Stamp 10.21.
commit : 6979736b4bcdc57e4699eadcca44b699fd1afd29
author : Tom Lane <[email protected]>
date : Mon, 9 May 2022 17:21:58 -0400
committer: Tom Lane <[email protected]>
date : Mon, 9 May 2022 17:21:58 -0400
M configure
M configure.in
M doc/bug.template
M src/include/pg_config.h.win32
M src/interfaces/libpq/libpq.rc.in
M src/port/win32ver.rc
Last-minute updates for release notes.
commit : db034c2e2e45216439c14e2b7bb99260914d610f
author : Tom Lane <[email protected]>
date : Mon, 9 May 2022 14:29:53 -0400
committer: Tom Lane <[email protected]>
date : Mon, 9 May 2022 14:29:53 -0400
Security: CVE-2022-1552
M doc/src/sgml/release-10.sgml
Fix core dump in transformValuesClause when there are no columns.
commit : 4eabaffcada25206c3f8f89f7cb0b27ffb9372bc
author : Tom Lane <[email protected]>
date : Mon, 9 May 2022 14:15:37 -0400
committer: Tom Lane <[email protected]>
date : Mon, 9 May 2022 14:15:37 -0400
The parser code that transformed VALUES from row-oriented to
column-oriented lists failed if there were zero columns.
You can't write that straightforwardly (though probably you
should be able to), but the case can be reached by expanding
a "tab.*" reference to a zero-column table.
Per bug #17477 from Wang Ke. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/analyze.c
M src/test/regress/expected/select.out
M src/test/regress/sql/select.sql
Revert "Disallow infinite endpoints in generate_series() for timestamps."
commit : 86a21803c7d860bd0cbd8c81c34cc9250d4cc173
author : Tom Lane <[email protected]>
date : Mon, 9 May 2022 11:02:37 -0400
committer: Tom Lane <[email protected]>
date : Mon, 9 May 2022 11:02:37 -0400
This reverts commit eafdf9de06e9b60168f5e47cedcfceecdc6d4b5f
and its back-branch counterparts. Corey Huinker pointed out that
we'd discussed this exact change back in 2016 and rejected it,
on the grounds that there's at least one usage pattern with LIMIT
where an infinite endpoint can usefully be used. Perhaps that
argument needs to be re-litigated, but there's no time left before
our back-branch releases. To keep our options open, restore the
status quo ante; if we do end up deciding to change things, waiting
one more quarter won't hurt anything.
Rather than just doing a straight revert, I added a new test case
demonstrating the usage with LIMIT. That'll at least remind us of
the issue if we forget again.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CADkLM=dzw0Pvdqp5yWKxMd+VmNkAMhG=4ku7GnCZxebWnzmz3Q@mail.gmail.com
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
In REFRESH MATERIALIZED VIEW, set user ID before running user code.
commit : f26d5702857a9c027f84850af48b0eea0f3aa15c
author : Noah Misch <[email protected]>
date : Mon, 9 May 2022 08:35:08 -0700
committer: Noah Misch <[email protected]>
date : Mon, 9 May 2022 08:35:08 -0700
It intended to, but did not, achieve this. Adopt the new standard of
setting user ID just after locking the relation. Back-patch to v10 (all
supported versions).
Reviewed by Simon Riggs. Reported by Alvaro Herrera.
Security: CVE-2022-1552
M src/backend/commands/matview.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Make relation-enumerating operations be security-restricted operations.
commit : ef792f7856dea2576dcd9cab92b2b05fe955696b
author : Noah Misch <[email protected]>
date : Mon, 9 May 2022 08:35:08 -0700
committer: Noah Misch <[email protected]>
date : Mon, 9 May 2022 08:35:08 -0700
When a feature enumerates relations and runs functions associated with
all found relations, the feature's user shall not need to trust every
user having permission to create objects. BRIN-specific functionality
in autovacuum neglected to account for this, as did pg_amcheck and
CLUSTER. An attacker having permission to create non-temp objects in at
least one schema could execute arbitrary SQL functions under the
identity of the bootstrap superuser. CREATE INDEX (not a
relation-enumerating operation) and REINDEX protected themselves too
late. This change extends to the non-enumerating amcheck interface.
Back-patch to v10 (all supported versions).
Sergey Shinderuk, reviewed (in earlier versions) by Alexander Lakhin.
Reported by Alexander Lakhin.
Security: CVE-2022-1552
M contrib/amcheck/expected/check_btree.out
M contrib/amcheck/sql/check_btree.sql
M contrib/amcheck/verify_nbtree.c
M src/backend/access/brin/brin.c
M src/backend/catalog/index.c
M src/backend/commands/cluster.c
M src/backend/commands/indexcmds.c
M src/backend/utils/init/miscinit.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Translation updates
commit : b05f4ae1fc2460b76b37118f3b435c60d01280cf
author : Peter Eisentraut <[email protected]>
date : Mon, 9 May 2022 12:27:36 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 9 May 2022 12:27:36 +0200
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: c71132f2b3f9275a610a2a1a8c39d76fcef092ab
M src/backend/po/de.po
M src/backend/po/fr.po
M src/backend/po/ru.po
M src/backend/po/sv.po
M src/bin/initdb/po/ru.po
M src/bin/initdb/po/sv.po
M src/bin/pg_archivecleanup/po/sv.po
M src/bin/pg_basebackup/po/sv.po
M src/bin/pg_config/po/ru.po
M src/bin/pg_config/po/sv.po
M src/bin/pg_controldata/po/sv.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_ctl/po/sv.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/sv.po
M src/bin/pg_resetwal/po/sv.po
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_rewind/po/sv.po
M src/bin/pg_test_fsync/po/sv.po
M src/bin/pg_test_timing/po/sv.po
M src/bin/pg_upgrade/po/sv.po
M src/bin/pg_waldump/po/sv.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/sv.po
M src/bin/scripts/po/ru.po
M src/bin/scripts/po/sv.po
M src/interfaces/ecpg/ecpglib/po/sv.po
M src/interfaces/ecpg/preproc/po/sv.po
M src/interfaces/libpq/po/de.po
M src/interfaces/libpq/po/fr.po
M src/interfaces/libpq/po/ru.po
M src/interfaces/libpq/po/sv.po
M src/pl/plperl/po/ru.po
M src/pl/plperl/po/sv.po
M src/pl/plpgsql/src/po/sv.po
M src/pl/plpython/po/sv.po
M src/pl/tcl/po/sv.po
Disable 031_recovery_conflict.pl until after minor releases.
commit : 390b489eff76743206c2ddb6aeb80f05026040c5
author : Andres Freund <[email protected]>
date : Sun, 8 May 2022 17:59:30 -0700
committer: Andres Freund <[email protected]>
date : Sun, 8 May 2022 17:59:30 -0700
f40d362a667 disabled part of 031_recovery_conflict.pl due to instability
that's not trivial to fix in the back branches. That fixed most of the
issues. But there was one more failure (on lapwing / REL_10_STABLE).
That failure looks like it might be caused by a genuine problem. Disable the
test until after the set of releases, to avoid packagers etc potentially
having to fight with a test failure they can't do anything about.
Discussion: https://postgr.es/m/[email protected]
Backpatch: 10-14
M src/test/recovery/t/031_recovery_conflict.pl
Release notes for 14.3, 13.7, 12.11, 11.16, 10.21.
commit : b2aed8cbeadfc29267b36ced24d69f23b9049d08
author : Tom Lane <[email protected]>
date : Sun, 8 May 2022 12:36:38 -0400
committer: Tom Lane <[email protected]>
date : Sun, 8 May 2022 12:36:38 -0400
M doc/src/sgml/release-10.sgml
Fix back-patch of "Under has_wal_read_bug, skip .../001_wal.pl."
commit : d0cf1ed0ed00dd4cfc54ce5b6979110869488764
author : Noah Misch <[email protected]>
date : Sat, 7 May 2022 09:12:56 -0700
committer: Noah Misch <[email protected]>
date : Sat, 7 May 2022 09:12:56 -0700
Per buildfarm members tadarida, snapper, and kittiwake. Back-patch to
v10 (all supported versions).
M contrib/bloom/t/001_wal.pl
Under has_wal_read_bug, skip contrib/bloom/t/001_wal.pl.
commit : c027e51d0fa3cb87662f14ff5b5f1dbb604743ac
author : Noah Misch <[email protected]>
date : Sat, 7 May 2022 00:33:15 -0700
committer: Noah Misch <[email protected]>
date : Sat, 7 May 2022 00:33:15 -0700
Per buildfarm members snapper and kittiwake. Back-patch to v10 (all
supported versions).
Discussion: https://postgr.es/m/[email protected]
M contrib/bloom/t/001_wal.pl
Temporarily skip recovery deadlock test in back branches.
commit : 441fa7a63ad4ed9377e08c6295b9f94f16a4e61e
author : Andres Freund <[email protected]>
date : Fri, 6 May 2022 09:01:08 -0700
committer: Andres Freund <[email protected]>
date : Fri, 6 May 2022 09:01:08 -0700
The recovery deadlock test has a timing issue that was fixed in 5136967f1eb in
HEAD. Unfortunately the same fix doesn't quite work in the back branches: 1)
adjust_conf() doesn't exist, which is easy enough to work around 2) a restart
cleares the recovery conflict stats < 15.
These issues can be worked around, but given the upcoming set of minor
releases, skip the problematic test for now. The buildfarm doesn't show
failures in other parts of 031_recovery_conflict.pl.
Discussion: https://postgr.es/m/[email protected]
Backpatch: 10-14
M src/test/recovery/t/031_recovery_conflict.pl
Update time zone data files to tzdata release 2022a.
commit : bc334ec9f1c133a81d36a8915fa8d43ae3234c94
author : Tom Lane <[email protected]>
date : Thu, 5 May 2022 14:54:53 -0400
committer: Tom Lane <[email protected]>
date : Thu, 5 May 2022 14:54:53 -0400
DST law changes in Palestine. Historical corrections for
Chile and Ukraine.
M src/timezone/data/tzdata.zi
Revert "Fix timing issue in deadlock recovery conflict test."
commit : 14f9d8b7e3b3341489fabee381ab3094b59cb5f7
author : Andres Freund <[email protected]>
date : Wed, 4 May 2022 14:15:20 -0700
committer: Andres Freund <[email protected]>
date : Wed, 4 May 2022 14:15:20 -0700
This reverts commit 3197e0f5ae9d58a0dd45e1bf8646ce83791bb2de.
M src/test/recovery/t/031_recovery_conflict.pl
Fix timing issue in deadlock recovery conflict test.
commit : 3197e0f5ae9d58a0dd45e1bf8646ce83791bb2de
author : Andres Freund <[email protected]>
date : Wed, 4 May 2022 12:50:38 -0700
committer: Andres Freund <[email protected]>
date : Wed, 4 May 2022 12:50:38 -0700
Per buildfarm members longfin and skink.
Discussion: https://postgr.es/m/[email protected]
Backpatch: 10-
M src/test/recovery/t/031_recovery_conflict.pl
Backpatch 031_recovery_conflict.pl.
commit : b1f35a36e6d595fb7a7d60fb04b54fca0f7f65fd
author : Andres Freund <[email protected]>
date : Mon, 2 May 2022 18:29:35 -0700
committer: Andres Freund <[email protected]>
date : Mon, 2 May 2022 18:29:35 -0700
The prior commit showed that the introduction of recovery conflict tests was a
good idea. Without these tests it's hard to know that the fix didn't break
something...
031_recovery_conflict.pl was introduced in 9f8a050f68d and extended in
21e184403bf.
Discussion: https://postgr.es/m/[email protected]
Backpatch: 10-14
A src/test/recovery/t/031_recovery_conflict.pl
Fix possibility of self-deadlock in ResolveRecoveryConflictWithBufferPin().
commit : cbc47ad7ef6051a58cc954c28db44034e7e542be
author : Andres Freund <[email protected]>
date : Mon, 2 May 2022 18:25:00 -0700
committer: Andres Freund <[email protected]>
date : Mon, 2 May 2022 18:25:00 -0700
The tests added in 9f8a050f68d failed nearly reliably on FreeBSD in CI, and
occasionally on the buildfarm. That turns out to be caused not by a bug in the
test, but by a longstanding bug in recovery conflict handling.
The standby timeout handler, used by ResolveRecoveryConflictWithBufferPin(),
executed SendRecoveryConflictWithBufferPin() inside a signal handler. A bad
idea, because the deadlock timeout handler (or a spurious latch set) could
have interrupted ProcWaitForSignal(). If unlucky that could cause a
self-deadlock on ProcArrayLock, if the deadlock check is in
SendRecoveryConflictWithBufferPin()->CancelDBBackends().
To fix, set a flag in StandbyTimeoutHandler(), and check the flag in
ResolveRecoveryConflictWithBufferPin().
Subsequently the recovery conflict tests will be backpatched.
Discussion: https://postgr.es/m/[email protected]
Backpatch: 10-
M src/backend/storage/ipc/standby.c
Backpatch addition of wait_for_log(), pump_until().
commit : e8a0cf9b20a12d9cb7687a1363cdeb261ef16472
author : Andres Freund <[email protected]>
date : Mon, 2 May 2022 18:09:44 -0700
committer: Andres Freund <[email protected]>
date : Mon, 2 May 2022 18:09:44 -0700
These were originally introduced in a2ab9c06ea1 and a2ab9c06ea1, as they are
needed by a about-to-be-backpatched test.
Discussion: https://postgr.es/m/[email protected]
Backpatch: 10-14
M src/test/perl/PostgreSQL/Test/Utils.pm
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
Fix typo in comment.
commit : d6ae041c87488af37bc1f8b30697c3df1c24e65f
author : Etsuro Fujita <[email protected]>
date : Mon, 2 May 2022 16:45:09 +0900
committer: Etsuro Fujita <[email protected]>
date : Mon, 2 May 2022 16:45:09 +0900
M src/backend/storage/ipc/latch.c
Inhibit mingw CRT's auto-globbing of command line arguments
commit : 58887d58c6addd2ed727d57520c6c4629d59f89a
author : Andrew Dunstan <[email protected]>
date : Mon, 25 Apr 2022 15:02:13 -0400
committer: Andrew Dunstan <[email protected]>
date : Mon, 25 Apr 2022 15:02:13 -0400
For some reason by default the mingw C Runtime takes it upon itself to
expand program arguments that look like shell globbing characters. That
has caused much scratching of heads and mis-attribution of the causes of
some TAP test failures, so stop doing that.
This removes an inconsistency with Windows binaries built with MSVC,
which have no such behaviour.
Per suggestion from Noah Misch.
Backpatch to all live branches.
Discussion: https://postgr.es/m/[email protected]
M src/common/exec.c
Support new perl module namespace in stable branches
commit : 5d223e4e0691911d1addbacbb6d843ac5c8af53a
author : Andrew Dunstan <[email protected]>
date : Thu, 21 Apr 2022 09:38:20 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 21 Apr 2022 09:38:20 -0400
Commit b3b4d8e68a moved our perl test modules to a better namespace
structure, but this has made life hard for people wishing to backpatch
improvements in the TAP tests. Here we alleviate much of that difficulty
by implementing the new module names on top of the old modules, mostly
by using a little perl typeglob aliasing magic, so that we don't have a
dual maintenance burden. This should work both for the case where a new
test is backpatched and the case where a fix to an existing test that
uses the new namespace is backpatched.
Reviewed by Michael Paquier
Per complaint from Andres Freund
Discussion: https://postgr.es/m/[email protected]
Applied to branches 10 through 14
A src/test/perl/PostgreSQL/Test/Cluster.pm
A src/test/perl/PostgreSQL/Test/Utils.pm
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
Fix CLUSTER tuplesorts on abbreviated expressions.
commit : a903895b3fe49c4ceb6fbcea73e357493245771d
author : Peter Geoghegan <[email protected]>
date : Wed, 20 Apr 2022 17:17:33 -0700
committer: Peter Geoghegan <[email protected]>
date : Wed, 20 Apr 2022 17:17:33 -0700
CLUSTER sort won't use the datum1 SortTuple field when clustering
against an index whose leading key is an expression. This makes it
unsafe to use the abbreviated keys optimization, which was missed by the
logic that sets up SortSupport state. Affected tuplesorts output tuples
in a completely bogus order as a result (the wrong SortSupport based
comparator was used for the leading attribute).
This issue is similar to the bug fixed on the master branch by recent
commit cc58eecc5d. But it's a far older issue, that dates back to the
introduction of the abbreviated keys optimization by commit 4ea51cdfe8.
Backpatch to all supported versions.
Author: Peter Geoghegan <[email protected]>
Author: Thomas Munro <[email protected]>
Discussion: https://postgr.es/m/CA+hUKG+bA+bmwD36_oDxAoLrCwZjVtST2fqe=b4=qZcmU7u89A@mail.gmail.com
Backpatch: 10-
M src/backend/utils/sort/tuplesort.c
M src/test/regress/expected/cluster.out
M src/test/regress/sql/cluster.sql
Disallow infinite endpoints in generate_series() for timestamps.
commit : a1e4782a0bca3f19f05042c6f05675dbf4fcacc7
author : Tom Lane <[email protected]>
date : Wed, 20 Apr 2022 18:08:15 -0400
committer: Tom Lane <[email protected]>
date : Wed, 20 Apr 2022 18:08:15 -0400
Such cases will lead to infinite loops, so they're of no practical
value. The numeric variant of generate_series() already threw error
for this, so borrow its message wording.
Per report from Richard Wesley. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Fix breakage in AlterFunction().
commit : 409bcfda6bd62479f4f7e745c4a536757dbe9637
author : Tom Lane <[email protected]>
date : Tue, 19 Apr 2022 23:03:59 -0400
committer: Tom Lane <[email protected]>
date : Tue, 19 Apr 2022 23:03:59 -0400
An ALTER FUNCTION command that tried to update both the function's
proparallel property and its proconfig list failed to do the former,
because it stored the new proparallel value into a tuple that was
no longer the interesting one. Carelessness in 7aea8e4f2.
(I did not bother with a regression test, because the only likely
future breakage would be for someone to ignore the comment I added
and add some other field update after the heap_modify_tuple step.
A test using existing function properties could not catch that.)
Per report from Bryn Llewellyn. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/functioncmds.c
Fix the check to limit sync workers.
commit : be46985bed8a071a4f7d6016f955f3a71f3b04ca
author : Amit Kapila <[email protected]>
date : Tue, 19 Apr 2022 09:38:25 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 19 Apr 2022 09:38:25 +0530
We don't allow to invoke more sync workers once we have reached the sync
worker limit per subscription. But the check to enforce this also doesn't
allow to launch an apply worker if it gets restarted.
This code was introduced by commit de43897122 but we caught the problem
only with the test added by recent commit c91f71b9dc which started failing
occasionally in the buildfarm.
As per buildfarm.
Diagnosed-by: Amit Kapila, Masahiko Sawada, Tomas Vondra
Author: Amit Kapila
Backpatch-through: 10
Discussion: https://postgr.es/m/CAH2L28vddB_NFdRVpuyRBJEBWjz4BSyTB=_ektNRH8NJ1jf95g@mail.gmail.com
https://postgr.es/m/[email protected]
M src/backend/replication/logical/launcher.c
Avoid invalid array reference in transformAlterTableStmt().
commit : 2ceec5bb396bceeb40e7ac7089041c856e9c21e5
author : Tom Lane <[email protected]>
date : Mon, 18 Apr 2022 12:16:45 -0400
committer: Tom Lane <[email protected]>
date : Mon, 18 Apr 2022 12:16:45 -0400
Don't try to look at the attidentity field of system attributes,
because they're not there in the TupleDescAttr array. Sometimes
this is harmless because we accidentally pick up a zero, but
otherwise we'll report "no owned sequence found" from an attempt
to alter a system attribute. (It seems possible that a SIGSEGV
could occur, too, though I've not seen it in testing.)
It's not in this function's charter to complain that you can't
alter a system column, so instead just hard-wire an assumption
that system attributes aren't identities. I didn't bother with
a regression test because the appearance of the bug is very
erratic.
Per bug #17465 from Roman Zharkov. Back-patch to all supported
branches. (There's not actually a live bug before v12, because
before that get_attidentity() did the right thing anyway.
But for consistency I changed the test in the older branches too.)
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/parse_utilcmd.c
Add a temp-install prerequisite to src/interfaces/ecpg "checktcp".
commit : ad845d640ec5f5951c38f00c764a28e7bfeee40e
author : Noah Misch <[email protected]>
date : Sat, 16 Apr 2022 17:43:54 -0700
committer: Noah Misch <[email protected]>
date : Sat, 16 Apr 2022 17:43:54 -0700
The target failed, tested $PATH binaries, or tested a stale temporary
installation. Commit c66b438db62748000700c9b90b585e756dd54141 missed
this. Back-patch to v10 (all supported versions).
M src/interfaces/ecpg/Makefile
Rethink the delay-checkpoint-end mechanism in the back-branches.
commit : 48b398943b78a42c61771a1210c70ffa0b017164
author : Robert Haas <[email protected]>
date : Thu, 14 Apr 2022 11:10:18 -0400
committer: Robert Haas <[email protected]>
date : Thu, 14 Apr 2022 11:10:18 -0400
The back-patch of commit bbace5697df12398e87ffd9879171c39d27f5b33 had
the unfortunate effect of changing the layout of PGPROC in the
back-branches, which could break extensions. This happened because it
changed the delayChkpt from type bool to type int. So, change it back,
and add a new bool delayChkptEnd field instead. The new field should
fall within what used to be padding space within the struct, and so
hopefully won't cause any extensions to break.
Per report from Markus Wanner and discussion with Tom Lane and others.
Patch originally by me, somewhat revised by Markus Wanner per a
suggestion from Michael Paquier. A very similar patch was developed
by Kyotaro Horiguchi, but I failed to see the email in which that was
posted before writing one of my own.
Discussion: http://postgr.es/m/CA+Tgmoao-kUD9c5nG5sub3F7tbo39+cdr8jKaOVEs_1aBWcJ3Q@mail.gmail.com
Discussion: http://postgr.es/m/[email protected]
M src/backend/access/transam/multixact.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xloginsert.c
M src/backend/catalog/storage.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/ipc/procarray.c
M src/include/storage/proc.h
M src/include/storage/procarray.h
pageinspect: Fix handling of all-zero pages
commit : 12d32b7bc15183be063516732d7458c58e2d2156
author : Michael Paquier <[email protected]>
date : Thu, 14 Apr 2022 15:09:46 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 14 Apr 2022 15:09:46 +0900
Getting from get_raw_page() an all-zero page is considered as a valid
case by the buffer manager and it can happen for example when finding a
corrupted page with zero_damaged_pages enabled (using zero_damaged_pages
to look at corrupted pages happens), or after a crash when a relation
file is extended before any WAL for its new data is generated (before a
vacuum or autovacuum job comes in to do some cleanup).
However, all the functions of pageinspect, as of the index AMs (except
hash that has its own idea of new pages), heap, the FSM or the page
header have never worked with all-zero pages, causing various crashes
when going through the page internals.
This commit changes all the pageinspect functions to be compliant with
all-zero pages, where the choice is made to return NULL or no rows for
SRFs when finding a new page. get_raw_page() still works the same way,
returning a batch of zeros in the bytea of the page retrieved. A hard
error could be used but NULL, while more invasive, is useful when
scanning relation files in full to get a batch of results for a single
relation in one query. Tests are added for all the code paths
impacted.
Reported-by: Daria Lepikhova
Author: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 10
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/btreefuncs.c
M contrib/pageinspect/expected/brin.out
M contrib/pageinspect/expected/btree.out
M contrib/pageinspect/expected/gin.out
M contrib/pageinspect/expected/hash.out
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/fsmfuncs.c
M contrib/pageinspect/ginfuncs.c
M contrib/pageinspect/rawpage.c
M contrib/pageinspect/sql/brin.sql
M contrib/pageinspect/sql/btree.sql
M contrib/pageinspect/sql/gin.sql
M contrib/pageinspect/sql/hash.sql
M contrib/pageinspect/sql/page.sql
Doc: tweak textsearch.sgml for SEO purposes.
commit : 2e1f30270b04067e2dcdf4362a05677dbed21912
author : Tom Lane <[email protected]>
date : Tue, 12 Apr 2022 18:21:04 -0400
committer: Tom Lane <[email protected]>
date : Tue, 12 Apr 2022 18:21:04 -0400
Google seems to like to return textsearch.html for queries about
GIN and GiST indexes, even though it's not a primary reference
for either. It seems likely that that's because those keywords
appear in the page title. Since "GIN and GiST Index Types" is
not a very apposite title for this material anyway, rename the
section in hopes of stopping that.
Also provide explicit links to the GIN and GiST chapters, to help
anyone who finds their way to this page regardless.
Per gripe from Jan Piotrowski. Back-patch to supported branches.
(Unfortunately Google is likely to continue returning the 9.1
version of this page, but improving that situation is a matter
for the www team.)
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/textsearch.sgml
Fix postgres_fdw to check shippability of sort clauses properly.
commit : 728fc0f9a2dd5f8b99be8cb6fc44171e4deb0490
author : Tom Lane <[email protected]>
date : Thu, 31 Mar 2022 14:29:24 -0400
committer: Tom Lane <[email protected]>
date : Thu, 31 Mar 2022 14:29:24 -0400
postgres_fdw would push ORDER BY clauses to the remote side without
verifying that the sort operator is safe to ship. Moreover, it failed
to print a suitable USING clause if the sort operator isn't default
for the sort expression's type. The net result of this is that the
remote sort might not have anywhere near the semantics we expect,
which'd be disastrous for locally-performed merge joins in particular.
We addressed similar issues in the context of ORDER BY within an
aggregate function call in commit 7012b132d, but failed to notice
that query-level ORDER BY was broken. Thus, much of the necessary
logic already existed, but it requires refactoring to be usable
in both cases.
Back-patch to all supported branches. In HEAD only, remove the
core code's copy of find_em_expr_for_rel, which is no longer used
and really should never have been pushed into equivclass.c in the
first place.
Ronan Dunklau, per report from David Rowley;
reviews by David Rowley, Ranier Vilela, and myself
Discussion: https://postgr.es/m/CAApHDvr4OeC2DBVY--zVP83-K=bYrTD7F8SZDhN4g+pj2f2S-A@mail.gmail.com
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/sql/postgres_fdw.sql
Add missing newline in one libpq error message.
commit : a6bd1f0eef3dae3918315734cabd88ed07122dc3
author : Tom Lane <[email protected]>
date : Thu, 31 Mar 2022 11:24:26 -0400
committer: Tom Lane <[email protected]>
date : Thu, 31 Mar 2022 11:24:26 -0400
Oversight in commit a59c79564. Back-patch, as that was.
Noted by Peter Eisentraut.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-secure-openssl.c
doc: Fix typo in ANALYZE documentation
commit : 3b6d2b460b5af1e62f45c751997a23e5c674ad03
author : Daniel Gustafsson <[email protected]>
date : Thu, 31 Mar 2022 12:03:33 +0200
committer: Daniel Gustafsson <[email protected]>
date : Thu, 31 Mar 2022 12:03:33 +0200
Commit 61fa6ca79b3 accidentally wrote constrast instead of contrast.
Backpatch-through: 10
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/analyze.sgml
Fix typo in comment.
commit : 0598bed99efa6e2d5fa43bbccbcca0358c6e1e75
author : Etsuro Fujita <[email protected]>
date : Wed, 30 Mar 2022 19:00:09 +0900
committer: Etsuro Fujita <[email protected]>
date : Wed, 30 Mar 2022 19:00:09 +0900
M src/backend/commands/copy.c
Document autoanalyze limitations for partitioned tables
commit : c1e9cfaef9759442dde94f23275b4969b0a631b7
author : Tomas Vondra <[email protected]>
date : Mon, 28 Mar 2022 14:27:36 +0200
committer: Tomas Vondra <[email protected]>
date : Mon, 28 Mar 2022 14:27:36 +0200
When dealing with partitioned tables, counters for partitioned tables
are not updated when modifying child tables. This means autoanalyze may
not update optimizer statistics for the parent relations, which can
result in poor plans for some queries.
It's worth documenting this limitation, so that people are aware of it
and can take steps to mitigate it (e.g. by setting up a script executing
ANALYZE regularly).
Backpatch to v10. Older branches are affected too, of couse, but we no
longer maintain those.
Author: Justin Pryzby
Reviewed-by: Zhihong Yu, Tomas Vondra
Backpatch-through: 10
Discussion: https://postgr.es/m/20210913035409.GA10647%40telsasoft.com
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/ref/analyze.sgml
pageinspect: Add more sanity checks to prevent out-of-bound reads
commit : ab80b0fbbbc4ab3e6a2d14407d1e57757adff6fb
author : Michael Paquier <[email protected]>
date : Sun, 27 Mar 2022 17:54:08 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 27 Mar 2022 17:54:08 +0900
A couple of code paths use the special area on the page passed by the
function caller, expecting to find some data in it. However, feeding
an incorrect page can lead to out-of-bound reads when trying to access
the page special area (like a heap page that has no special area,
leading PageGetSpecialPointer() to grab a pointer outside the allocated
page).
The functions used for hash and btree indexes have some protection
already against that, while some other functions using a relation OID
as argument would make sure that the access method involved is correct,
but functions taking in input a raw page without knowing the relation
the page is attached to would run into problems.
This commit improves the set of checks used in the code paths of BRIN,
btree (including one check if a leaf page is found with a non-zero
level), GIN and GiST to verify that the page given in input has a
special area size that fits with each access method, which is done
though PageGetSpecialSize(), becore calling PageGetSpecialPointer().
The scope of the checks done is limited to work with pages that one
would pass after getting a block with get_raw_page(), as it is possible
to craft byteas that could bypass existing code paths. Having too many
checks would also impact the usability of pageinspect, as the existing
code is very useful to look at the content details in a corrupted page,
so the focus is really to avoid out-of-bound reads as this is never a
good thing even with functions whose execution is limited to
superusers.
The safest approach could be to rework the functions so as these fetch a
block using a relation OID and a block number, but there are also cases
where using a raw page is useful.
Tests are added to cover all the code paths that needed such checks, and
an error message for hash indexes is reworded to fit better with what
this commit adds.
Reported-By: Alexander Lakhin
Author: Julien Rouhaud, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 10
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/btreefuncs.c
M contrib/pageinspect/expected/brin.out
M contrib/pageinspect/expected/btree.out
M contrib/pageinspect/expected/gin.out
M contrib/pageinspect/expected/hash.out
M contrib/pageinspect/ginfuncs.c
M contrib/pageinspect/hashfuncs.c
M contrib/pageinspect/sql/brin.sql
M contrib/pageinspect/sql/btree.sql
M contrib/pageinspect/sql/gin.sql
M contrib/pageinspect/sql/hash.sql
Suppress compiler warning in relptr_store().
commit : 403a8c76293bdef17bf5ee8af6be92ee4eba6bb1
author : Tom Lane <[email protected]>
date : Sat, 26 Mar 2022 14:29:29 -0400
committer: Tom Lane <[email protected]>
date : Sat, 26 Mar 2022 14:29:29 -0400
clang 13 with -Wextra warns that "performing pointer subtraction with
a null pointer has undefined behavior" in the places where freepage.c
tries to set a relptr variable to constant NULL. This appears to be
a compiler bug, but it's unlikely to get fixed instantly. Fortunately,
we can work around it by introducing an inline support function, which
seems like a good change anyway because it removes the macro's existing
double-evaluation hazard.
Backpatch to v10 where this code was introduced.
Patch by me, based on an idea of Andres Freund's.
Discussion: https://postgr.es/m/[email protected]
M src/include/utils/relptr.h
Fix possible recovery trouble if TRUNCATE overlaps a checkpoint.
commit : 57f618310f837101ba13757baf95bae37e3766ac
author : Robert Haas <[email protected]>
date : Thu, 24 Mar 2022 14:50:06 -0400
committer: Robert Haas <[email protected]>
date : Thu, 24 Mar 2022 14:50:06 -0400
If TRUNCATE causes some buffers to be invalidated and thus the
checkpoint does not flush them, TRUNCATE must also ensure that the
corresponding files are truncated on disk. Otherwise, a replay
from the checkpoint might find that the buffers exist but have
the wrong contents, which may cause replay to fail.
Report by Teja Mupparti. Patch by Kyotaro Horiguchi, per a design
suggestion from Heikki Linnakangas, with some changes to the
comments by me. Review of this and a prior patch that approached
the issue differently by Heikki Linnakangas, Andres Freund, Álvaro
Herrera, Masahiko Sawada, and Tom Lane.
Discussion: http://postgr.es/m/BYAPR06MB6373BF50B469CA393C614257ABF00@BYAPR06MB6373.namprd06.prod.outlook.com
M src/backend/access/transam/multixact.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xloginsert.c
M src/backend/catalog/storage.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/lmgr/proc.c
M src/include/storage/proc.h
M src/include/storage/procarray.h
Don't call fwrite() with len == 0 when writing out relcache init file.
commit : 5f4615b9def5c7a4218c7e17f318e4b2eb2b1747
author : Andres Freund <[email protected]>
date : Wed, 23 Mar 2022 13:05:25 -0700
committer: Andres Freund <[email protected]>
date : Wed, 23 Mar 2022 13:05:25 -0700
Noticed via -fsanitize=undefined.
Backpatch to all branches, for the same reasons as 46ab07ffda9.
Discussion: https://postgr.es/m/[email protected]
Backpatch: 10-
M src/backend/utils/cache/relcache.c
configure: check for dlsym instead of dlopen.
commit : 2a919372655d64a5ed848b84a09b54d685aec446
author : Andres Freund <[email protected]>
date : Wed, 23 Mar 2022 12:43:14 -0700
committer: Andres Freund <[email protected]>
date : Wed, 23 Mar 2022 12:43:14 -0700
When building with sanitizers the sanitizer library provides dlopen, but not
dlsym(), making configure think that -ldl isn't needed. Just checking for
dlsym() ought to suffice, hard to see dlsym() being provided without dlopen()
also being provided.
Backpatch to all branches, for the same reasons as 46ab07ffda9.
Reviewed-By: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch: 10-
M configure
M configure.in
pg_upgrade: Upgrade an Assert to a real 'if' test
commit : d8718edf6017ba647abb2715d6196ca53bb3aec6
author : Alvaro Herrera <[email protected]>
date : Wed, 23 Mar 2022 19:23:51 +0100
committer: Alvaro Herrera <[email protected]>
date : Wed, 23 Mar 2022 19:23:51 +0100
It seems possible for the condition being tested to be true in
production, and nobody would never know (except when some data
eventually becomes corrupt?).
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m//[email protected]
M src/bin/pg_rewind/parsexlog.c
Fix "missing continuation record" after standby promotion
commit : 9ade3c09a5d11f4be91b5e180832abd2dbc2c55d
author : Alvaro Herrera <[email protected]>
date : Wed, 23 Mar 2022 18:22:10 +0100
committer: Alvaro Herrera <[email protected]>
date : Wed, 23 Mar 2022 18:22:10 +0100
Invalidate abortedRecPtr and missingContrecPtr after a missing
continuation record is successfully skipped on a standby. This fixes a
PANIC caused when a recently promoted standby attempts to write an
OVERWRITE_RECORD with an LSN of the previously read aborted record.
Backpatch to 10 (all stable versions).
Author: Sami Imseih <[email protected]>
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
M src/test/recovery/t/026_overwrite_contrecord.pl
Add missing dependency of pg_dumpall to WIN32RES.
commit : 083031aa99d739f5e8ff26d90e488ed4ac14892e
author : Andres Freund <[email protected]>
date : Tue, 22 Mar 2022 08:22:02 -0700
committer: Andres Freund <[email protected]>
date : Tue, 22 Mar 2022 08:22:02 -0700
When cross-building to windows, or building with mingw on windows, the build
could fail with
x86_64-w64-mingw32-gcc: error: win32ver.o: No such file or director
because pg_dumpall didn't depend on WIN32RES, but it's recipe references
it. The build nevertheless succeeded most of the time, due to
pg_dump/pg_restore having the required dependency, causing win32ver.o to be
built.
Reported-By: Thomas Munro <[email protected]>
Discussion: https://postgr.es/m/CA+hUKGJeekpUPWW6yCVdf9=oBAcCp86RrBivo4Y4cwazAzGPng@mail.gmail.com
Backpatch: 10-, omission present on all live branches
M src/bin/pg_dump/Makefile
Fix failures in SSL tests caused by out-of-tree keys and certificates
commit : 8138bd4a567e05b77b205519c24ef6caa4b716d6
author : Michael Paquier <[email protected]>
date : Tue, 22 Mar 2022 13:21:58 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 22 Mar 2022 13:21:58 +0900
This issue is environment-sensitive, where the SSL tests could fail in
various way by feeding on defaults provided by sslcert, sslkey,
sslrootkey, sslrootcert, sslcrl and sslcrldir coming from a local setup,
as of ~/.postgresql/ by default. Horiguchi-san has reported two
failures, but more advanced testing from me (aka inclusion of garbage
SSL configuration in ~/.postgresql/ for all the configuration
parameters) has showed dozens of failures that can be triggered in the
whole test suite.
History has showed that we are not good when it comes to address such
issues, fixing them locally like in dd87799, and such problems keep
appearing. This commit strengthens the entire test suite to put an end
to this set of problems by embedding invalid default values in all the
connection strings used in the tests. The invalid values are prefixed
in each connection string, relying on the follow-up values passed in the
connection string to enforce any invalid value previously set. Note
that two tests related to CRLs are required to fail with certain pre-set
configurations, but we can rely on enforcing an empty value instead
after the invalid set of values.
Reported-by: Kyotaro Horiguchi
Reviewed-by: Andrew Dunstan, Daniel Gustafsson, Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
backpatch-through: 10
M src/test/ssl/t/001_ssltests.pl
Fix assorted missing logic for GroupingFunc nodes.
commit : 2afa031ac552995e2b6ba5371388bff280e7cfe8
author : Tom Lane <[email protected]>
date : Mon, 21 Mar 2022 17:44:29 -0400
committer: Tom Lane <[email protected]>
date : Mon, 21 Mar 2022 17:44:29 -0400
The planner needs to treat GroupingFunc like Aggref for many purposes,
in particular with respect to processing of the argument expressions,
which are not to be evaluated at runtime. A few places hadn't gotten
that memo, notably including subselect.c's processing of outer-level
aggregates. This resulted in assertion failures or wrong plans for
cases in which a GROUPING() construct references an outer aggregation
level.
Also fix missing special cases for GroupingFunc in cost_qual_eval
(resulting in wrong cost estimates for GROUPING(), although it's
not clear that that would affect plan shapes in practice) and in
ruleutils.c (resulting in excess parentheses in pretty-print mode).
Per bug #17088 from Yaoguang Chen. Back-patch to all supported
branches.
Richard Guo, Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/nodes/nodeFuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/subselect.c
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Doc: fix our example systemd script.
commit : 0de9b999d14f7c529c0a47b24198442b404db084
author : Tom Lane <[email protected]>
date : Sun, 20 Mar 2022 12:39:40 -0400
committer: Tom Lane <[email protected]>
date : Sun, 20 Mar 2022 12:39:40 -0400
The example used "TimeoutSec=0", but systemd's documented way to get
the desired effect is "TimeoutSec=infinity".
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/runtime.sgml
doc: Mention SET TABLESPACE clause for ALTER MATERIALIZED VIEW
commit : 49c5cd5c4e7ac408dc0ae64fa77b16a4e0c63c8b
author : Michael Paquier <[email protected]>
date : Sat, 19 Mar 2022 16:38:02 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 19 Mar 2022 16:38:02 +0900
This command flavor is supported, but there was nothing in the
documentation about it.
Author: Yugo Nagata
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 10
M doc/src/sgml/ref/alter_materialized_view.sgml
Fix incorrect xmlschema output for types timetz and timestamptz.
commit : e6fd4a3daff6e9d2a5dcceeae90a4b01bf1b7645
author : Tom Lane <[email protected]>
date : Fri, 18 Mar 2022 16:01:42 -0400
committer: Tom Lane <[email protected]>
date : Fri, 18 Mar 2022 16:01:42 -0400
The output of table_to_xmlschema() and allied functions includes
a regex describing valid values for these types ... but the regex
was itself invalid, as it failed to escape a literal "+" sign.
Report and fix by Renan Soares Lopes. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/xml.c
M src/test/regress/expected/xmlmap.out
M src/test/regress/expected/xmlmap_1.out
M src/test/regress/sql/xmlmap.sql
Revert applying column aliases to the output of whole-row Vars.
commit : 205214c8b8ccc40b64f9dad4061f12c4dc28fefd
author : Tom Lane <[email protected]>
date : Thu, 17 Mar 2022 18:18:05 -0400
committer: Tom Lane <[email protected]>
date : Thu, 17 Mar 2022 18:18:05 -0400
In commit bf7ca1587, I had the bright idea that we could make the
result of a whole-row Var (that is, foo.*) track any column aliases
that had been applied to the FROM entry the Var refers to. However,
that's not terribly logically consistent, because now the output of
the Var is no longer of the named composite type that the Var claims
to emit. bf7ca1587 tried to handle that by changing the output
tuple values to be labeled with a blessed RECORD type, but that's
really pretty disastrous: we can wind up storing such tuples onto
disk, whereupon they're not readable by other sessions.
The only practical fix I can see is to give up on what bf7ca1587
tried to do, and say that the column names of tuples produced by
a whole-row Var are always those of the underlying named composite
type, query aliases or no. While this introduces some inconsistencies,
it removes others, so it's not that awful in the abstract. What *is*
kind of awful is to make such a behavioral change in a back-patched
bug fix. But corrupt data is worse, so back-patched it will be.
(A workaround available to anyone who's unhappy about this is to
introduce an extra level of sub-SELECT, so that the whole-row Var is
referring to the sub-SELECT's output and not to a named table type.
Then the Var is of type RECORD to begin with and there's no issue.)
Per report from Miles Delahunty. The faulty commit dates to 9.5,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/execTuples.c
M src/test/regress/expected/rowtypes.out
M src/test/regress/sql/rowtypes.sql
Fix race between DROP TABLESPACE and checkpointing.
commit : fe197872cfe8d29b3134568df06b3fba2cdd4c18
author : Thomas Munro <[email protected]>
date : Wed, 16 Mar 2022 17:20:24 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 16 Mar 2022 17:20:24 +1300
Commands like ALTER TABLE SET TABLESPACE may leave files for the next
checkpoint to clean up. If such files are not removed by the time DROP
TABLESPACE is called, we request a checkpoint so that they are deleted.
However, there is presently a window before checkpoint start where new
unlink requests won't be scheduled until the following checkpoint. This
means that the checkpoint forced by DROP TABLESPACE might not remove the
files we expect it to remove, and the following ERROR will be emitted:
ERROR: tablespace "mytblspc" is not empty
To fix, add a call to AbsorbSyncRequests() just before advancing the
unlink cycle counter. This ensures that any unlink requests forwarded
prior to checkpoint start (i.e., when ckpt_started is incremented) will
be processed by the current checkpoint. Since AbsorbSyncRequests()
performs memory allocations, it cannot be called within a critical
section, so we also need to move SyncPreCheckpoint() to before
CreateCheckPoint()'s critical section.
This is an old bug, so back-patch to all supported versions.
Author: Nathan Bossart <[email protected]>
Reported-by: Nathan Bossart <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/20220215235845.GA2665318%40nathanxps13
M src/backend/access/transam/xlog.c
M src/backend/storage/smgr/md.c
pageinspect: Fix memory context allocation of page in brin_revmap_data()
commit : 07e343513b85ea152f4cc2d3afeb2d4c87bbda77
author : Michael Paquier <[email protected]>
date : Wed, 16 Mar 2022 12:30:07 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 16 Mar 2022 12:30:07 +0900
This caused the function to fail, as the aligned copy of the raw page
given by the function caller was not saved in the correct memory
context, which needs to be multi_call_memory_ctx in this case.
Issue introduced by 076f4d9.
Per buildfarm members sifika, mylodon and longfin. I have reproduced
that locally with macos.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 10
M contrib/pageinspect/brinfuncs.c
pageinspect: Fix handling of page sizes and AM types
commit : c42b88932b9ddf6d4f6f509e62aec6d1f2e73111
author : Michael Paquier <[email protected]>
date : Wed, 16 Mar 2022 11:21:00 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 16 Mar 2022 11:21:00 +0900
This commit fixes a set of issues related to the use of the SQL
functions in this module when the caller is able to pass down raw page
data as input argument:
- The page size check was fuzzy in a couple of places, sometimes
looking after only a sub-range, but what we are looking for is an exact
match on BLCKSZ. After considering a few options here, I have settled
down to do a generalization of get_page_from_raw(). Most of the SQL
functions already used that, and this is not strictly required if not
accessing an 8-byte-wide value from a raw page, but this feels safer in
the long run for alignment-picky environment, particularly if a code
path begins to access such values. This also reduces the number of
strings that need to be translated.
- The BRIN function brin_page_items() uses a Relation but it did not
check the access method of the opened index, potentially leading to
crashes. All the other functions in need of a Relation already did
that.
- Some code paths could fail on elog(), but we should to use ereport()
for failures that can be triggered by the user.
Tests are added to stress all the cases that are fixed as of this
commit, with some junk raw pages (\set VERBOSITY ensures that this works
across all page sizes) and unexpected index types when functions open
relations.
Author: Michael Paquier, Justin Prysby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 10
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/btreefuncs.c
M contrib/pageinspect/expected/brin.out
M contrib/pageinspect/expected/btree.out
M contrib/pageinspect/expected/gin.out
M contrib/pageinspect/expected/hash.out
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/fsmfuncs.c
M contrib/pageinspect/hashfuncs.c
M contrib/pageinspect/rawpage.c
M contrib/pageinspect/sql/brin.sql
M contrib/pageinspect/sql/btree.sql
M contrib/pageinspect/sql/gin.sql
M contrib/pageinspect/sql/hash.sql
M contrib/pageinspect/sql/page.sql
Introduce PG_TEST_TIMEOUT_DEFAULT for TAP suite non-elapsing timeouts.
commit : 7f2e1befc039029ce5be36088f6872ad393d007c
author : Noah Misch <[email protected]>
date : Fri, 4 Mar 2022 18:53:13 -0800
committer: Noah Misch <[email protected]>
date : Fri, 4 Mar 2022 18:53:13 -0800
Slow hosts may avoid load-induced, spurious failures by setting
environment variable PG_TEST_TIMEOUT_DEFAULT to some number of seconds
greater than 180. Developers may see faster failures by setting that
environment variable to some lesser number of seconds. In tests, write
$PostgreSQL::Test::Utils::timeout_default wherever the convention has
been to write 180. This change raises the default for some briefer
timeouts. Back-patch to v10 (all supported versions).
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/pgbench/t/022_cic.pl
M src/bin/pgbench/t/023_cic_2pc.pl
M src/bin/scripts/t/080_pg_isready.pl
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
M src/test/recovery/t/006_logical_decoding.pl
M src/test/recovery/t/010_logical_decoding_timelines.pl
M src/test/recovery/t/017_shm.pl
Fix bogus casting in BlockIdGetBlockNumber().
commit : 18c04d1574568fd68e4f498483f3b10b11018c68
author : Tom Lane <[email protected]>
date : Thu, 3 Mar 2022 19:03:17 -0500
committer: Tom Lane <[email protected]>
date : Thu, 3 Mar 2022 19:03:17 -0500
This macro cast the result to BlockNumber after shifting, not before,
which is the wrong thing. Per the C spec, the uint16 fields would
promote to int not unsigned int, so that (for 32-bit int) the shift
potentially shifts a nonzero bit into the sign position. I doubt
there are any production systems where this would actually end with
the wrong answer, but it is undefined behavior per the C spec, and
clang's -fsanitize=undefined option reputedly warns about it on some
platforms. (I can't reproduce that right now, but the code is
undeniably wrong per spec.) It's easy to fix by casting to
BlockNumber (uint32) in the proper places.
It's been wrong for ages, so back-patch to all supported branches.
Report and patch by Zhihong Yu (cosmetic tweaking by me)
Discussion: https://postgr.es/m/CALNJ-vT9r0DSsAOw9OXVJFxLENoVS_68kJ5x0p44atoYH+H4dg@mail.gmail.com
M src/include/storage/block.h
Clean up assorted failures under clang's -fsanitize=undefined checks.
commit : cfbe87bd6f644e2b9b172516d94b971ed94dfc89
author : Tom Lane <[email protected]>
date : Thu, 3 Mar 2022 18:13:24 -0500
committer: Tom Lane <[email protected]>
date : Thu, 3 Mar 2022 18:13:24 -0500
Most of these are cases where we could call memcpy() or other libc
functions with a NULL pointer and a zero count, which is forbidden
by POSIX even though every production version of libc allows it.
We've fixed such things before in a piecemeal way, but apparently
never made an effort to try to get them all. I don't claim that
this patch does so either, but it gets every failure I observe in
check-world, using clang 12.0.1 on current RHEL8.
numeric.c has a different issue that the sanitizer doesn't like:
"ln(-1.0)" will compute log10(0) and then try to assign the
resulting -Inf to an integer variable. We don't actually use the
result in such a case, so there's no live bug.
Back-patch to all supported branches, with the idea that we might
start running a buildfarm member that tests this case. This includes
back-patching c1132aae3 (Check the size in COPY_POINTER_FIELD),
which previously silenced some of these issues in copyfuncs.c.
Discussion: https://postgr.es/m/CALNJ-vT9r0DSsAOw9OXVJFxLENoVS_68kJ5x0p44atoYH+H4dg@mail.gmail.com
M contrib/pgcrypto/px.c
M src/backend/access/heap/heapam.c
M src/backend/access/transam/xact.c
M src/backend/nodes/copyfuncs.c
M src/backend/storage/ipc/shm_mq.c
M src/backend/utils/adt/numeric.c
M src/backend/utils/time/snapmgr.c
M src/backend/utils/time/tqual.c
M src/fe_utils/print.c
Allow root-owned SSL private keys in libpq, not only the backend.
commit : 9050999efea72eb763fd9349d295761f4ff4da4d
author : Tom Lane <[email protected]>
date : Wed, 2 Mar 2022 11:57:02 -0500
committer: Tom Lane <[email protected]>
date : Wed, 2 Mar 2022 11:57:02 -0500
This change makes libpq apply the same private-key-file ownership
and permissions checks that we have used in the backend since commit
9a83564c5. Namely, that the private key can be owned by either the
current user or root (with different file permissions allowed in the
two cases). This allows system-wide management of key files, which
is just as sensible on the client side as the server, particularly
when the client is itself some application daemon.
Sync the comments about this between libpq and the backend, too.
Back-patch of a59c79564 and 50f03473e into all supported branches.
David Steele
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
M src/backend/libpq/be-secure-openssl.c
M src/interfaces/libpq/fe-secure-openssl.c
Disallow execution of SPI functions during plperl function compilation.
commit : 4936747c477270e133e622bf6d7a12fa246cf3a7
author : Tom Lane <[email protected]>
date : Fri, 25 Feb 2022 17:40:21 -0500
committer: Tom Lane <[email protected]>
date : Fri, 25 Feb 2022 17:40:21 -0500
Perl can be convinced to execute user-defined code during compilation
of a plperl function (or at least a plperlu function). That's not
such a big problem as long as the activity is confined within the
Perl interpreter, and it's not clear we could do anything about that
anyway. However, if such code tries to use plperl's SPI functions,
we have a bigger problem. In the first place, those functions are
likely to crash because current_call_data->prodesc isn't set up yet.
In the second place, because it isn't set up, we lack critical info
such as whether the function is supposed to be read-only. And in
the third place, this path allows code execution during function
validation, which is strongly discouraged because of the potential
for security exploits. Hence, reject execution of the SPI functions
until compilation is finished.
While here, add check_spi_usage_allowed() calls to various functions
that hadn't gotten the memo about checking that. I think that perhaps
plperl_sv_to_literal may have been intentionally omitted on the grounds
that it was safe at the time; but if so, the addition of transforms
functionality changed that. The others are more recently added and
seem to be flat-out oversights.
Per report from Mark Murawski. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/pl/plperl/plperl.c
Fix temporary object cleanup failing due to toast access without snapshot.
commit : cebc54b90bda3ce43dc60faf216d2d149eebd3f3
author : Andres Freund <[email protected]>
date : Sat, 19 Feb 2022 12:27:20 -0800
committer: Andres Freund <[email protected]>
date : Sat, 19 Feb 2022 12:27:20 -0800
When cleaning up temporary objects during process exit the cleanup could fail
with:
FATAL: cannot fetch toast data without an active snapshot
The bug is caused by RemoveTempRelationsCallback() not setting up a
snapshot. If an object with toasted catalog data needs to be cleaned up,
init_toast_snapshot() could fail with the above error.
Most of the time however the the problem is masked due to cached catalog
snapshots being returned by GetOldestSnapshot(). But dropping an object can
cause catalog invalidations to be emitted. If no further catalog accesses are
necessary between the invalidation processing and the next toast datum
deletion, the bug becomes visible.
It's easy to miss this bug because it typically happens after clients
disconnect and the FATAL error just ends up in the log.
Luckily temporary table cleanup at the next use of the same temporary schema
or during DISCARD ALL does not have the same problem.
Fix the bug by pushing a snapshot in RemoveTempRelationsCallback(). Also add
isolation tests for temporary object cleanup, including objects with toasted
catalog data.
A future HEAD only commit will add more assertions.
Reported-By: Miles Delahunty
Author: Andres Freund
Discussion: https://postgr.es/m/CAOFAq3BU5Mf2TTvu8D9n_ZOoFAeQswuzk7yziAb7xuw_qyw5gw@mail.gmail.com
Backpatch: 10-
M src/backend/catalog/namespace.c
A src/test/isolation/expected/temp-schema-cleanup.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/temp-schema-cleanup.spec
Remove most msys special processing in TAP tests
commit : 2b02cadb5cbc4645adf0f8f7297d633e01258ee2
author : Andrew Dunstan <[email protected]>
date : Sat, 19 Feb 2022 11:28:35 -0500
committer: Andrew Dunstan <[email protected]>
date : Sat, 19 Feb 2022 11:28:35 -0500
Following migration of Windows buildfarm members running TAP tests to
use of ucrt64 perl for those tests, special processing for msys perl is
no longer necessary and so is removed.
Backpatch to release 10
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_ctl/t/001_start_stop.pl
M src/bin/pg_rewind/t/RewindTest.pm
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
M src/test/recovery/t/cp_history_files
Remove PostgreSQL::Test::Utils::perl2host completely
commit : b30c62bd42ebdee366919b9cfcf62a489031b571
author : Andrew Dunstan <[email protected]>
date : Sun, 20 Feb 2022 08:55:06 -0500
committer: Andrew Dunstan <[email protected]>
date : Sun, 20 Feb 2022 08:55:06 -0500
Commit f1ac4a74de disabled this processing, and as nothing has broken (as
expected) here we proceed to remove the routine and adjust all the call
sites.
Backpatch to release 10
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
M src/test/recovery/t/025_stuck_on_old_timeline.pl
Improve subscriber's error message for wrong publication relkind.
commit : 17a8c4c3a274e2c04bf5f564f20398deba53cd55
author : Tom Lane <[email protected]>
date : Tue, 15 Feb 2022 12:21:28 -0500
committer: Tom Lane <[email protected]>
date : Tue, 15 Feb 2022 12:21:28 -0500
Pre-v13 versions only support logical replication from plain tables,
while v13 and later also allow partitioned tables to be published.
If you tried to subscribe an older server to such a publication,
you got "table XXX not found on publisher", which is pretty
unhelpful/confusing. Arrange to deliver a more on-point error
message. As commit c314c147c did in v13, remove the relkind check
from the query WHERE clause altogether, so that "not there"
is distinguishable from "wrong relkind".
Per report from Radoslav Nedyalkov. Patch v10-v12.
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/tablesync.c
WAL log unchanged toasted replica identity key attributes.
commit : b5e060473273b7ef95ec919aee0c934e659dd63e
author : Amit Kapila <[email protected]>
date : Mon, 14 Feb 2022 08:45:54 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 14 Feb 2022 08:45:54 +0530
Currently, during UPDATE, the unchanged replica identity key attributes
are not logged separately because they are getting logged as part of the
new tuple. But if they are stored externally then the untoasted values are
not getting logged as part of the new tuple and logical replication won't
be able to replicate such UPDATEs. So we need to log such attributes as
part of the old_key_tuple during UPDATE.
Reported-by: Haiying Tang
Author: Dilip Kumar and Amit Kapila
Reviewed-by: Alvaro Herrera, Haiying Tang, Andres Freund
Backpatch-through: 10
Discussion: https://postgr.es/m/OS0PR01MB611342D0A92D4F4BF26C0F47FB229@OS0PR01MB6113.jpnprd01.prod.outlook.com
M contrib/test_decoding/expected/toast.out
M doc/src/sgml/ref/alter_table.sgml
M src/backend/access/heap/heapam.c
Fix memory leak in IndexScan node with reordering
commit : 2e9033a548041151e32f070cecc9dee154e8fb98
author : Alexander Korotkov <[email protected]>
date : Mon, 14 Feb 2022 03:26:55 +0300
committer: Alexander Korotkov <[email protected]>
date : Mon, 14 Feb 2022 03:26:55 +0300
Fix ExecReScanIndexScan() to free the referenced tuples while emptying the
priority queue. Backpatch to all supported versions.
Discussion: https://postgr.es/m/CAHqSB9gECMENBQmpbv5rvmT3HTaORmMK3Ukg73DsX5H7EJV7jw%40mail.gmail.com
Author: Aliaksandr Kalenik
Reviewed-by: Tom Lane, Alexander Korotkov
Backpatch-through: 10
M src/backend/executor/nodeIndexscan.c
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Don't use_physical_tlist for an IOS with non-returnable columns.
commit : b6f869beeab06981c223ff35088fdb8a5422584d
author : Tom Lane <[email protected]>
date : Fri, 11 Feb 2022 15:23:52 -0500
committer: Tom Lane <[email protected]>
date : Fri, 11 Feb 2022 15:23:52 -0500
createplan.c tries to save a runtime projection step by specifying
a scan plan node's output as being exactly the table's columns, or
index's columns in the case of an index-only scan, if there is not a
reason to do otherwise. This logic did not previously pay attention
to whether an index's columns are returnable. That worked, sort of
accidentally, until commit 9a3ddeb51 taught setrefs.c to reject plans
that try to read a non-returnable column. I have no desire to loosen
setrefs.c's new check, so instead adjust use_physical_tlist() to not
try to optimize this way when there are non-returnable column(s).
Per report from Ryan Kelly. Like the previous patch, back-patch
to all supported branches.
Discussion: https://postgr.es/m/CAHUie24ddN+pDNw7fkhNrjrwAX=fXXfGZZEHhRuofV_N_ftaSg@mail.gmail.com
M src/backend/optimizer/plan/createplan.c
M src/test/regress/expected/gist.out
M src/test/regress/sql/gist.sql
Make pg_ctl stop/restart/promote recheck postmaster aliveness.
commit : 34523933b8dc118da0693d55a3b01e9d02be0949
author : Tom Lane <[email protected]>
date : Thu, 10 Feb 2022 16:49:39 -0500
committer: Tom Lane <[email protected]>
date : Thu, 10 Feb 2022 16:49:39 -0500
"pg_ctl stop/restart" checked that the postmaster PID is valid just
once, as a side-effect of sending the stop signal, and then would
wait-till-timeout for the postmaster.pid file to go away. This
neglects the case wherein the postmaster dies uncleanly after we
signal it. Similarly, once "pg_ctl promote" has sent the signal,
it'd wait for the corresponding on-disk state change to occur
even if the postmaster dies.
I'm not sure how we've managed not to notice this problem, but it
seems to explain slow execution of the 017_shm.pl test script on AIX
since commit 4fdbf9af5, which added a speculative "pg_ctl stop" with
the idea of making real sure that the postmaster isn't there. In the
test steps that kill-9 and then restart the postmaster, it's possible
to get past the initial signal attempt before kill() stops working
for the doomed postmaster. If that happens, pg_ctl waited till
PGCTLTIMEOUT before giving up ... and the buildfarm's AIX members
have that set very high.
To fix, include a "kill(pid, 0)" test (similar to what
postmaster_is_alive uses) in these wait loops, so that we'll
give up immediately if the postmaster PID disappears.
While here, I chose to refactor those loops out of where they were.
do_stop() and do_restart() can perfectly well share one copy of the
wait-for-stop loop, and it seems desirable to put a similar function
beside that for wait-for-promote.
Back-patch to all supported versions, since pg_ctl's wait logic
is substantially identical in all, and we're seeing the slow test
behavior in all branches.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_ctl/pg_ctl.c
Use gendef instead of pexports for building windows .def files
commit : a5eef80c2c7c37177aac149b612234b90f6131ca
author : Andrew Dunstan <[email protected]>
date : Thu, 10 Feb 2022 13:44:05 -0500
committer: Andrew Dunstan <[email protected]>
date : Thu, 10 Feb 2022 13:44:05 -0500
Modern msys systems lack pexports but have gendef instead, so use that.
Discussion: https://postgr.es/m/[email protected]
Backpatch to release 9.4 to enable building with perl on older branches.
Before that pexports is not used for plperl.
M src/pl/plperl/GNUmakefile
M src/pl/plpython/Makefile
M src/pl/tcl/Makefile
Use Test::Builder::todo_start(), replacing $::TODO.
commit : 6da62ff14c57ec9ab68bc7fa590eca7b3d1bd671
author : Noah Misch <[email protected]>
date : Wed, 9 Feb 2022 18:16:59 -0800
committer: Noah Misch <[email protected]>
date : Wed, 9 Feb 2022 18:16:59 -0800
Some pre-2017 Test::More versions need perfect $Test::Builder::Level
maintenance to find the variable. Buildfarm member snapper reported an
overall failure that the file intended to hide via the TODO construct.
That trouble was reachable in v11 and v10. For later branches, this
serves as defense in depth. Back-patch to v10 (all supported versions).
Discussion: https://postgr.es/m/[email protected]
M src/bin/pgbench/t/023_cic_2pc.pl
Fix back-patch of "Avoid race in RelationBuildDesc() ..."
commit : eac3c4007044f0bb6ee380d056931944611d22c2
author : Noah Misch <[email protected]>
date : Wed, 9 Feb 2022 18:16:56 -0800
committer: Noah Misch <[email protected]>
date : Wed, 9 Feb 2022 18:16:56 -0800
The back-patch of commit fdd965d074d46765c295223b119ca437dbcac973 broke
CLOBBER_CACHE_ALWAYS for v9.6 through v13. It updated the
InvalidateSystemCaches() call for CLOBBER_CACHE_RECURSIVELY, neglecting
the one for CLOBBER_CACHE_ALWAYS. Back-patch to v13, v12, v11, and v10.
Reviewed by Tomas Vondra. Reported by Tomas Vondra.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/cache/inval.c
Remove ppport.h's broken re-implementation of eval_pv().
commit : 1a8b6f055567b553376bb6b9e295379b613baeea
author : Tom Lane <[email protected]>
date : Tue, 8 Feb 2022 19:25:56 -0500
committer: Tom Lane <[email protected]>
date : Tue, 8 Feb 2022 19:25:56 -0500
Recent versions of Devel::PPPort try to redefine eval_pv() to
dodge a bug in pre-5.31 Perl versions. Unfortunately the redefinition
fails on compilers that don't support statements nested within
expressions. However, we aren't actually interested in this bug fix,
since we always call eval_pv() with croak_on_error = FALSE.
So, until there's an upstream fix for this breakage, just comment
out the macro to revert to the older behavior.
Per report from Wei Sun, as well as previous buildfarm failure
on pademelon (which I'd unfortunately not looked at carefully
enough to understand the cause). Back-patch to all supported
versions, since we're using the same ppport.h in all.
Discussion: https://postgr.es/m/[email protected]
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=pademelon&dt=2022-02-02%2001%3A22%3A58
M src/pl/plperl/ppport.h