Fix incorrect comments for Memoize struct
commit : 3f712ea6dc40a0c9d5cf96907655343039601473
author : David Rowley <drowley@postgresql.org>
date : Thu, 19 May 2022 17:14:56 +1200
committer: David Rowley <drowley@postgresql.org>
date : Thu, 19 May 2022 17:14:56 +1200
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/0635f5aa-4973-8dc2-4e4e-df9fd5778a65@enterprisedb.com
Backpatch-through: 14, where Memoize was added
M src/include/nodes/plannodes.h
Update xml_1.out and xml_2.out
commit : e8b93c6e2810306887dc8477aab88ae08cf335f9
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 18 May 2022 23:19:53 +0200
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 18 May 2022 23:19:53 +0200
Commit 0fbf01120023 should have updated them but didn't.
M src/test/regress/expected/xml_1.out
M src/test/regress/expected/xml_2.out
Check column list length in XMLTABLE/JSON_TABLE alias
commit : 94edb85d253e5ae066481e4e369ad9e35dea41b5
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 18 May 2022 20:28:31 +0200
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 18 May 2022 20:28:31 +0200
We weren't checking the length of the column list in the alias clause of
an XMLTABLE or JSON_TABLE function (a "tablefunc" RTE), and it was
possible to make the server crash by passing an overly long one. Fix it
by throwing an error in that case, like the other places that deal with
alias lists.
In passing, modify the equivalent test used for join RTEs to look like
the other ones, which was different for no apparent reason.
This bug came in when XMLTABLE was born in version 10; backpatch to all
stable versions.
Reported-by: Wang Ke <krking@zju.edu.cn>
Discussion: https://postgr.es/m/17480-1c9d73565bb28e90@postgresql.org
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_relation.c
M src/test/regress/expected/int2.out
M src/test/regress/expected/join.out
M src/test/regress/expected/with.out
M src/test/regress/expected/xml.out
M src/test/regress/sql/int2.sql
M src/test/regress/sql/join.sql
M src/test/regress/sql/with.sql
M src/test/regress/sql/xml.sql
Fix incorrect row estimates used for Memoize costing
commit : 23c2b76a8322233ce0b2c135c10ad52d48a4a22b
author : David Rowley <drowley@postgresql.org>
date : Mon, 16 May 2022 16:08:37 +1200
committer: David Rowley <drowley@postgresql.org>
date : Mon, 16 May 2022 16:08:37 +1200
In order to estimate the cache hit ratio of a Memoize node, one of the
inputs we require is the estimated number of times the Memoize node will
be rescanned. The higher this number, the large the cache hit ratio is
likely to become. Unfortunately, the value being passed as the number of
"calls" to the Memoize was incorrectly using the Nested Loop's
outer_path->parent->rows instead of outer_path->rows. This failed to
account for the fact that the outer_path might be parameterized by some
upper-level Nested Loop.
This problem could lead to Memoize plans appearing more favorable than
they might actually be. It could also lead to extended executor startup
times when work_mem values were large due to the planner setting overly
large MemoizePath->est_entries resulting in the Memoize hash table being
initially made much larger than might be required.
Fix this simply by passing outer_path->rows rather than
outer_path->parent->rows. Also, adjust the expected regression test
output for a plan change.
Reported-by: Pavel Stehule
Author: David Rowley
Discussion: https://postgr.es/m/CAFj8pRAMp%3DQsMi6sPQJ4W3hczoFJRvyXHJV3AZAZaMyTVM312Q%40mail.gmail.com
Backpatch-through: 14, where Memoize was introduced
M src/backend/optimizer/path/joinpath.c
M src/test/regress/expected/join.out
Fix control file update done in restartpoints still running after promotion
commit : 6dced63b411b920f7d0296337e4b3c8cba3e2e30
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 16 May 2022 11:26:22 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 16 May 2022 11:26:22 +0900
If a cluster is promoted (aka the control file shows a state different
than DB_IN_ARCHIVE_RECOVERY) while CreateRestartPoint() is still
processing, this function could miss an update of the control file for
"checkPoint" and "checkPointCopy" but still do the recycling and/or
removal of the past WAL segments, assuming that the to-be-updated LSN
values should be used as reference points for the cleanup. This causes
a follow-up restart attempting crash recovery to fail with a PANIC on a
missing checkpoint record if the end-of-recovery checkpoint triggered by
the promotion did not complete while the cluster abruptly stopped or
crashed before the completion of this checkpoint. The PANIC would be
caused by the redo LSN referred in the control file as located in a
segment already gone, recycled by the previous restartpoint with
"checkPoint" out-of-sync in the control file.
This commit fixes the update of the control file during restartpoints so
as "checkPoint" and "checkPointCopy" are updated even if the cluster has
been promoted while a restartpoint is running, to be on par with the set
of WAL segments actually recycled in the end of CreateRestartPoint().
7863ee4 has fixed this problem already on master, but the release timing
of the latest point versions did not let me enough time to study and fix
that on all the stable branches.
Reported-by: Fujii Masao, Rui Zhao
Author: Kyotaro Horiguchi
Reviewed-by: Nathan Bossart, Michael Paquier
Discussion: https://postgr.es/m/20220316.102444.2193181487576617583.horikyota.ntt@gmail.com
Backpatch-through: 10
M src/backend/access/transam/xlog.c
Make pull_var_clause() handle GroupingFuncs exactly like Aggrefs.
commit : ac51c9fba5a796ff434c0a7e05a5fd081690b21c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 12 May 2022 11:31:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 12 May 2022 11:31:46 -0400
This follows in the footsteps of commit 2591ee8ec by removing one more
ill-advised shortcut from planning of GroupingFuncs. It's true that
we don't intend to execute the argument expression(s) at runtime, but
we still have to process any Vars appearing within them, or we risk
failure at setrefs.c time (or more fundamentally, in EXPLAIN trying
to print such an expression). Vars in upper plan nodes have to have
referents in the next plan level, whether we ever execute 'em or not.
Per bug #17479 from Michael J. Sullivan. Back-patch to all supported
branches.
Richard Guo
Discussion: https://postgr.es/m/17479-6260deceaf0ad304@postgresql.org
M src/backend/optimizer/util/var.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Fix the logical replication timeout during large transactions.
commit : d6da71fa8f28faa68823e163f318ffb38a7a9a54
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 11 May 2022 10:51:04 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 11 May 2022 10:51:04 +0530
The problem is that we don't send keep-alive messages for a long time
while processing large transactions during logical replication where we
don't send any data of such transactions. This can happen when the table
modified in the transaction is not published or because all the changes
got filtered. We do try to send the keep_alive if necessary at the end of
the transaction (via WalSndWriteData()) but by that time the
subscriber-side can timeout and exit.
To fix this we try to send the keepalive message if required after
processing certain threshold of changes.
Reported-by: Fabrice Chapuis
Author: Wang wei and Amit Kapila
Reviewed By: Masahiko Sawada, Euler Taveira, Hou Zhijie, Hayato Kuroda
Backpatch-through: 10
Discussion: https://postgr.es/m/CAA5-nLARN7-3SLU_QUxfy510pmrYK6JJb=bk3hcgemAM_pAv+w@mail.gmail.com
M src/backend/replication/logical/logical.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/replication/walsender.c
M src/include/replication/logical.h
Improve setup of environment values for commands in MSVC's vcregress.pl
commit : ca9e9b08e453523314a3b8e87d1894edb23b6e8d
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 11 May 2022 10:22:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 11 May 2022 10:22:29 +0900
The current setup assumes that commands for lz4, zstd and gzip always
exist by default if not enforced by a user's environment. However,
vcpkg, as one example, installs libraries but no binaries, so this
default setup to assume that a command should always be present would
cause failures. This commit improves the detection of such external
commands as follows:
* If a ENV value is available, trust the environment/user and use it.
* If a ENV value is not available, check its execution by looking in the
current PATH, by launching a simple "$command --version" (that should be
portable enough).
** On execution failure, ignore ENV{command}.
** On execution success, set ENV{command} = "$command".
Note that this new rule applies to gzip, lz4 and zstd but not tar that
we assume will always exist. Those commands are set up in the
environment only when using bincheck and taptest. The CI includes all
those commands and I have checked that their setup is correct there. I
have also tested this change in a MSVC environment where we have none of
those commands.
While on it, remove the references to lz4 from the documentation and
vcregress.pl in ~v13. --with-lz4 has been added in v14~ so there is no
point to have this information in these older branches.
Reported-by: Andrew Dunstan
Reviewed-by: Andrew Dunstan
Discussion: https://postgr.es/m/14402151-376b-a57a-6d0c-10ad12608e12@dunslane.net
Backpatch-through: 10
M src/tools/msvc/vcregress.pl
configure: don't probe for libldap_r if libldap is 2.5 or newer.
commit : 12736e7dc326c404ae0f12ebdf5d1dbb954ec5f6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 May 2022 18:42:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 May 2022 18:42:02 -0400
In OpenLDAP 2.5 and later, libldap itself is always thread-safe and
there's never a libldap_r. Our existing coding dealt with that
by assuming it wouldn't find libldap_r if libldap is thread-safe.
But that rule fails to cope if there are multiple OpenLDAP versions
visible, as is likely to be the case on macOS in particular. We'd
end up using shiny new libldap in the backend and a hoary libldap_r
in libpq.
Instead, once we've found libldap, check if it's >= 2.5 (by
probing for a function introduced then) and don't bother looking
for libldap_r if so. While one can imagine library setups that
this'd still give the wrong answer for, they seem unlikely to
occur in practice.
Per report from Peter Eisentraut. Back-patch to all supported branches.
Discussion: https://postgr.es/m/fedacd7c-2a38-25c9-e7ff-dea549d0e979@enterprisedb.com
M configure
M configure.ac
Stamp 14.3.
commit : 0e9cc8c0f3bc411de888a77318d8854ccba90e80
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 May 2022 17:14:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 May 2022 17:14:45 -0400
M configure
M configure.ac
Last-minute updates for release notes.
commit : 0ae1d09575bbadd2e7a9ada5721a88c8ee0b6be3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 May 2022 14:29:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 May 2022 14:29:53 -0400
Security: CVE-2022-1552
M doc/src/sgml/release-14.sgml
Fix core dump in transformValuesClause when there are no columns.
commit : ab2f783921734a96aa1baf4f3ea165292b62aecf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 May 2022 14:15:37 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 May 2022 14:15:37 -0400
The parser code that transformed VALUES from row-oriented to
column-oriented lists failed if there were zero columns.
You can't write that straightforwardly (though probably you
should be able to), but the case can be reached by expanding
a "tab.*" reference to a zero-column table.
Per bug #17477 from Wang Ke. Back-patch to all supported branches.
Discussion: https://postgr.es/m/17477-0af3c6ac6b0a6ae0@postgresql.org
M src/backend/parser/analyze.c
M src/test/regress/expected/select.out
M src/test/regress/sql/select.sql
Revert "Disallow infinite endpoints in generate_series() for timestamps."
commit : 9b5797ca54f5ad74740220c1be070eda4c21c82c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 May 2022 11:02:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 May 2022 11:02:36 -0400
This reverts commit eafdf9de06e9b60168f5e47cedcfceecdc6d4b5f
and its back-branch counterparts. Corey Huinker pointed out that
we'd discussed this exact change back in 2016 and rejected it,
on the grounds that there's at least one usage pattern with LIMIT
where an infinite endpoint can usefully be used. Perhaps that
argument needs to be re-litigated, but there's no time left before
our back-branch releases. To keep our options open, restore the
status quo ante; if we do end up deciding to change things, waiting
one more quarter won't hurt anything.
Rather than just doing a straight revert, I added a new test case
demonstrating the usage with LIMIT. That'll at least remind us of
the issue if we forget again.
Discussion: https://postgr.es/m/3603504.1652068977@sss.pgh.pa.us
Discussion: https://postgr.es/m/CADkLM=dzw0Pvdqp5yWKxMd+VmNkAMhG=4ku7GnCZxebWnzmz3Q@mail.gmail.com
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
In REFRESH MATERIALIZED VIEW, set user ID before running user code.
commit : 677a494789062ca88e0142a17bedd5415f6ab0aa
author : Noah Misch <noah@leadboat.com>
date : Mon, 9 May 2022 08:35:08 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 9 May 2022 08:35:08 -0700
It intended to, but did not, achieve this. Adopt the new standard of
setting user ID just after locking the relation. Back-patch to v10 (all
supported versions).
Reviewed by Simon Riggs. Reported by Alvaro Herrera.
Security: CVE-2022-1552
M src/backend/commands/matview.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Make relation-enumerating operations be security-restricted operations.
commit : ab49ce7c3414ac19e4afb386d7843ce2d2fb8bda
author : Noah Misch <noah@leadboat.com>
date : Mon, 9 May 2022 08:35:08 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 9 May 2022 08:35:08 -0700
When a feature enumerates relations and runs functions associated with
all found relations, the feature's user shall not need to trust every
user having permission to create objects. BRIN-specific functionality
in autovacuum neglected to account for this, as did pg_amcheck and
CLUSTER. An attacker having permission to create non-temp objects in at
least one schema could execute arbitrary SQL functions under the
identity of the bootstrap superuser. CREATE INDEX (not a
relation-enumerating operation) and REINDEX protected themselves too
late. This change extends to the non-enumerating amcheck interface.
Back-patch to v10 (all supported versions).
Sergey Shinderuk, reviewed (in earlier versions) by Alexander Lakhin.
Reported by Alexander Lakhin.
Security: CVE-2022-1552
M contrib/amcheck/expected/check_btree.out
M contrib/amcheck/sql/check_btree.sql
M contrib/amcheck/verify_nbtree.c
M src/backend/access/brin/brin.c
M src/backend/catalog/index.c
M src/backend/commands/cluster.c
M src/backend/commands/indexcmds.c
M src/backend/utils/init/miscinit.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Translation updates
commit : e5b5a21356233739a552063fa70d4f5b245edb9a
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 9 May 2022 12:26:57 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 9 May 2022 12:26:57 +0200
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: b7586f1542a8ffdfd1416e425f55e4e89c9a9505
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/backend/po/ru.po
M src/backend/po/sv.po
M src/bin/initdb/po/ru.po
M src/bin/pg_amcheck/po/de.po
M src/bin/pg_amcheck/po/fr.po
M src/bin/pg_amcheck/po/ru.po
M src/bin/pg_amcheck/po/sv.po
M src/bin/pg_basebackup/po/sv.po
M src/bin/pg_config/po/ru.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/sv.po
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_rewind/po/sv.po
M src/bin/pg_waldump/po/ru.po
M src/bin/psql/po/de.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/sv.po
M src/bin/scripts/po/ru.po
M src/interfaces/libpq/po/de.po
M src/interfaces/libpq/po/fr.po
M src/interfaces/libpq/po/ru.po
M src/interfaces/libpq/po/sv.po
M src/pl/plperl/po/ru.po
Disable 031_recovery_conflict.pl until after minor releases.
commit : 55e5a5e0fa16dbe023ebdab35628fc73454c1dd4
author : Andres Freund <andres@anarazel.de>
date : Sun, 8 May 2022 17:59:30 -0700
committer: Andres Freund <andres@anarazel.de>
date : Sun, 8 May 2022 17:59:30 -0700
f40d362a667 disabled part of 031_recovery_conflict.pl due to instability
that's not trivial to fix in the back branches. That fixed most of the
issues. But there was one more failure (on lapwing / REL_10_STABLE).
That failure looks like it might be caused by a genuine problem. Disable the
test until after the set of releases, to avoid packagers etc potentially
having to fight with a test failure they can't do anything about.
Discussion: https://postgr.es/m/3447060.1652032749@sss.pgh.pa.us
Backpatch: 10-14
M src/test/recovery/t/031_recovery_conflict.pl
Release notes for 14.3, 13.7, 12.11, 11.16, 10.21.
commit : 2867d9b9697ffc34133ad988782ae0af88811c4d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 May 2022 12:36:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 May 2022 12:36:38 -0400
M doc/src/sgml/release-14.sgml
Fix back-patch of "Under has_wal_read_bug, skip .../001_wal.pl."
commit : e599ea09698ae6ae45bfbb04fbe0bda4e3c04ab0
author : Noah Misch <noah@leadboat.com>
date : Sat, 7 May 2022 09:12:56 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 7 May 2022 09:12:56 -0700
Per buildfarm members tadarida, snapper, and kittiwake. Back-patch to
v10 (all supported versions).
M contrib/bloom/t/001_wal.pl
Under has_wal_read_bug, skip contrib/bloom/t/001_wal.pl.
commit : aca4d561cb9f567537d12d9d8fbc45ae1528bc9d
author : Noah Misch <noah@leadboat.com>
date : Sat, 7 May 2022 00:33:15 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 7 May 2022 00:33:15 -0700
Per buildfarm members snapper and kittiwake. Back-patch to v10 (all
supported versions).
Discussion: https://postgr.es/m/20220116210241.GC756210@rfd.leadboat.com
M contrib/bloom/t/001_wal.pl
Temporarily skip recovery deadlock test in back branches.
commit : f40d362a667f6cf36f17d7565e4d7565fb7910f3
author : Andres Freund <andres@anarazel.de>
date : Fri, 6 May 2022 09:01:08 -0700
committer: Andres Freund <andres@anarazel.de>
date : Fri, 6 May 2022 09:01:08 -0700
The recovery deadlock test has a timing issue that was fixed in 5136967f1eb in
HEAD. Unfortunately the same fix doesn't quite work in the back branches: 1)
adjust_conf() doesn't exist, which is easy enough to work around 2) a restart
cleares the recovery conflict stats < 15.
These issues can be worked around, but given the upcoming set of minor
releases, skip the problematic test for now. The buildfarm doesn't show
failures in other parts of 031_recovery_conflict.pl.
Discussion: https://postgr.es/m/20220506155827.dfnaheq6ufylwrqf@alap3.anarazel.de
Backpatch: 10-14
M src/test/recovery/t/031_recovery_conflict.pl
Backpatch addition of pump_until() more completely.
commit : 7fa95bb0ac8e98799a98a08006a11feefcb614a6
author : Andres Freund <andres@anarazel.de>
date : Fri, 6 May 2022 08:38:19 -0700
committer: Andres Freund <andres@anarazel.de>
date : Fri, 6 May 2022 08:38:19 -0700
In a2ab9c06ea1 I just backpatched the introduction of pump_until(), without
changing the existing local definitions (as 6da65a3f9a9). The necessary
changes seemed more verbose than desirable. However, that leads to warnings,
as I failed to realize...
Backpatch to all versions containing pump_until() calls before
f74496dd611 (there's none in 10).
Discussion: https://postgr.es/m/2808491.1651802860@sss.pgh.pa.us
Discussion: https://postgr.es/m/18b37361-b482-b9d8-f30d-6115cd5ce25c@enterprisedb.com
Backpatch: 11-14
M src/test/recovery/t/013_crash_restart.pl
M src/test/recovery/t/022_crash_temp_files.pl
First-draft release notes for 14.3.
commit : 66ca1427a4963012fd565b922d0a67a8a8930d1f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 5 May 2022 18:27:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 5 May 2022 18:27:31 -0400
As usual, the release notes for other branches will be made by cutting
these down, but put them up for community review first.
M doc/src/sgml/release-14.sgml
Update time zone data files to tzdata release 2022a.
commit : 77ee14ed96e94f16f4ff355637d563f29e893bc2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 5 May 2022 14:54:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 5 May 2022 14:54:53 -0400
DST law changes in Palestine. Historical corrections for
Chile and Ukraine.
M src/timezone/data/tzdata.zi
Revert "Fix timing issue in deadlock recovery conflict test."
commit : 6e2924b57793d662aa48bc88ac9e97d5d8eb90cf
author : Andres Freund <andres@anarazel.de>
date : Wed, 4 May 2022 14:20:24 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 4 May 2022 14:20:24 -0700
This reverts commit 5136967f1eb194bf7598668f9893b997199935c1.
M src/test/recovery/t/031_recovery_conflict.pl
Fix timing issue in deadlock recovery conflict test.
commit : 5136967f1eb194bf7598668f9893b997199935c1
author : Andres Freund <andres@anarazel.de>
date : Wed, 4 May 2022 12:50:38 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 4 May 2022 12:50:38 -0700
Per buildfarm members longfin and skink.
Discussion: https://postgr.es/m/20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de
Backpatch: 10-
M src/test/recovery/t/031_recovery_conflict.pl
Backpatch 031_recovery_conflict.pl.
commit : f74496dd611ee4d345072b8eff7e7175fb5eb47d
author : Andres Freund <andres@anarazel.de>
date : Mon, 2 May 2022 18:26:09 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 2 May 2022 18:26:09 -0700
The prior commit showed that the introduction of recovery conflict tests was a
good idea. Without these tests it's hard to know that the fix didn't break
something...
031_recovery_conflict.pl was introduced in 9f8a050f68d and extended in
21e184403bf.
Discussion: https://postgr.es/m/20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de
Backpatch: 10-14
A src/test/recovery/t/031_recovery_conflict.pl
Fix possibility of self-deadlock in ResolveRecoveryConflictWithBufferPin().
commit : 9ab3b2bdbb5dc4ff857685eae5645d7c35839055
author : Andres Freund <andres@anarazel.de>
date : Mon, 2 May 2022 18:25:00 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 2 May 2022 18:25:00 -0700
The tests added in 9f8a050f68d failed nearly reliably on FreeBSD in CI, and
occasionally on the buildfarm. That turns out to be caused not by a bug in the
test, but by a longstanding bug in recovery conflict handling.
The standby timeout handler, used by ResolveRecoveryConflictWithBufferPin(),
executed SendRecoveryConflictWithBufferPin() inside a signal handler. A bad
idea, because the deadlock timeout handler (or a spurious latch set) could
have interrupted ProcWaitForSignal(). If unlucky that could cause a
self-deadlock on ProcArrayLock, if the deadlock check is in
SendRecoveryConflictWithBufferPin()->CancelDBBackends().
To fix, set a flag in StandbyTimeoutHandler(), and check the flag in
ResolveRecoveryConflictWithBufferPin().
Subsequently the recovery conflict tests will be backpatched.
Discussion: https://postgr.es/m/20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de
Backpatch: 10-
M src/backend/storage/ipc/standby.c
Backpatch addition of wait_for_log(), pump_until().
commit : 5ab8e8014801dd6bc05809e7ba994c013e9ee86b
author : Andres Freund <andres@anarazel.de>
date : Mon, 2 May 2022 18:09:42 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 2 May 2022 18:09:42 -0700
These were originally introduced in a2ab9c06ea1 and a2ab9c06ea1, as they are
needed by a about-to-be-backpatched test.
Discussion: https://postgr.es/m/20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de
Backpatch: 10-14
M src/test/perl/PostgreSQL/Test/Utils.pm
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
Fix typo in comment.
commit : 24c58f7a2a5c762f35ffe8aaa00fa08caeb881fe
author : Etsuro Fujita <efujita@postgresql.org>
date : Mon, 2 May 2022 16:45:02 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Mon, 2 May 2022 16:45:02 +0900
M src/backend/storage/ipc/latch.c
Disable asynchronous execution if using gating Result nodes.
commit : ebb79024152348227250404fd06c35c56cbc672e
author : Etsuro Fujita <efujita@postgresql.org>
date : Thu, 28 Apr 2022 15:15:02 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Thu, 28 Apr 2022 15:15:02 +0900
mark_async_capable_plan(), which is called from create_append_plan() to
determine whether subplans are async-capable, failed to take into
account that the given subplan created from a given subpath might
include a gating Result node if the subpath is a SubqueryScanPath or
ForeignPath, causing a segmentation fault there when the subplan created
from a SubqueryScanPath includes the Result node, or causing
ExecAsyncRequest() to throw an error about an unrecognized node type
when the subplan created from a ForeignPath includes the Result node,
because in the latter case the Result node was unintentionally
considered as async-capable, but we don't currently support executing
Result nodes asynchronously. Fix by modifying mark_async_capable_plan()
to disable asynchronous execution in such cases. Also, adjust code in
the ProjectionPath case in mark_async_capable_plan(), for consistency
with other cases, and adjust/improve comments there.
is_async_capable_path() added in commit 27e1f1456, which was rewritten
to mark_async_capable_plan() in a later commit, has the same issue,
causing the error at execution mentioned above, so back-patch to v14
where the aforesaid commit went in.
Per report from Justin Pryzby.
Etsuro Fujita, reviewed by Zhihong Yu and Justin Pryzby.
Discussion: https://postgr.es/m/20220408124338.GK24419%40telsasoft.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/optimizer/plan/createplan.c
Inhibit mingw CRT's auto-globbing of command line arguments
commit : 71f394667c824257ddf6e8359cba746e621a2d16
author : Andrew Dunstan <andrew@dunslane.net>
date : Mon, 25 Apr 2022 15:02:13 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Mon, 25 Apr 2022 15:02:13 -0400
For some reason by default the mingw C Runtime takes it upon itself to
expand program arguments that look like shell globbing characters. That
has caused much scratching of heads and mis-attribution of the causes of
some TAP test failures, so stop doing that.
This removes an inconsistency with Windows binaries built with MSVC,
which have no such behaviour.
Per suggestion from Noah Misch.
Backpatch to all live branches.
Discussion: https://postgr.es/m/20220423025927.GA1274057@rfd.leadboat.com
M src/common/exec.c
Remove some recently-added pg_dump test cases.
commit : 75a006beef6c502a16bc4e1c0dc4d67d2cd5de9c
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 25 Apr 2022 09:14:19 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 25 Apr 2022 09:14:19 -0400
Commit d2d35479796c3510e249d6fc72adbd5df918efbf included a pretty
extensive set of test cases, and some of them don't work on all
of our Windows machines. This happens because IPC::Run expands
its arguments as shell globs on a few machines, but doesn't on most
of the buildfarm. It might be good to fix that problem systematically
somehow, but in the meantime, there are enough test cases for this
commit that it seems OK to just remove the ones that are failing.
Discussion: http://postgr.es/m/3a190754-b2b0-d02b-dcfd-4ec1610ffbcb@dunslane.net
Discussion: http://postgr.es/m/CA+TgmoYRGUcFBy6VgN0+Pn4f6Wv=2H0HZLuPHqSy6VC8Ba7vdg@mail.gmail.com
M src/bin/pg_dump/t/002_pg_dump.pl
Fix incautious CTE matching in rewriteSearchAndCycle().
commit : dff6c77faf0c6bcf1d407dc3ffef9e169166c714
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 23 Apr 2022 12:16:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 23 Apr 2022 12:16:12 -0400
This function looks for a reference to the recursive WITH CTE,
but it checked only the CTE name not ctelevelsup, so that it could
seize on a lower CTE that happened to have the same name. This
would result in planner failures later, either weird errors such as
"could not find attribute 2 in subquery targetlist", or crashes
or assertion failures. The code also merely Assert'ed that it found
a matching entry, which is not guaranteed at all by the parser.
Per bugs #17320 and #17318 from Zhiyong Wu.
Thanks to Kyotaro Horiguchi for investigation.
Discussion: https://postgr.es/m/17320-70e37868182512ab@postgresql.org
Discussion: https://postgr.es/m/17318-2eb65a3a611d2368@postgresql.org
M src/backend/rewrite/rewriteSearchCycle.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Remove inadequate assertion check in CTE inlining.
commit : da22ef388a2469c3d7d11a8c97a3c41cc0016f4a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 21 Apr 2022 17:58:52 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 21 Apr 2022 17:58:52 -0400
inline_cte() expected to find exactly as many references to the
target CTE as its cterefcount indicates. While that should be
accurate for the tree as emitted by the parser, there are some
optimizations that occur upstream of here that could falsify it,
notably removal of unused subquery output expressions.
Trying to make the accounting 100% accurate seems expensive and
doomed to future breakage. It's not really worth it, because
all this code is protecting is downstream assumptions that every
referenced CTE has a plan. Let's convert those assertions to
regular test-and-elog just in case there's some actual problem,
and then drop the failing assertion.
Per report from Tomas Vondra (thanks also to Richard Guo for
analysis). Back-patch to v12 where the faulty code came in.
Discussion: https://postgr.es/m/29196a1e-ed47-c7ca-9be2-b1c636816183@enterprisedb.com
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/subselect.c
M src/include/nodes/pathnodes.h
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Support new perl module namespace in stable branches
commit : b235d41d9646c531864ecc680fd9ec5da9217051
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 21 Apr 2022 07:39:25 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 21 Apr 2022 07:39:25 -0400
Commit b3b4d8e68a moved our perl test modules to a better namespace
structure, but this has made life hard for people wishing to backpatch
improvements in the TAP tests. Here we alleviate much of that difficulty
by implementing the new module names on top of the old modules, mostly
by using a little perl typeglob aliasing magic, so that we don't have a
dual maintenance burden. This should work both for the case where a new
test is backpatched and the case where a fix to an existing test that
uses the new namespace is backpatched.
Reviewed by Michael Paquier
Per complaint from Andres Freund
Discussion: https://postgr.es/m/20220418141530.nfxtkohefvwnzncl@alap3.anarazel.de
Applied to branches 10 through 14
A src/test/perl/PostgreSQL/Test/Cluster.pm
A src/test/perl/PostgreSQL/Test/Utils.pm
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
postgres_fdw: Disable batch insert when BEFORE ROW INSERT triggers exist.
commit : 89d349b0a0239504325e5f63755f3bc1e1742478
author : Etsuro Fujita <efujita@postgresql.org>
date : Thu, 21 Apr 2022 15:30:02 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Thu, 21 Apr 2022 15:30:02 +0900
Previously, we allowed this, but such triggers might query the table to
insert into and act differently if the tuples that have already been
processed and prepared for insertion are not there, so disable it in
such cases.
Back-patch to v14 where batch insert was added.
Discussion: https://postgr.es/m/CAPmGK16_uPqsmgK0-LpLSUk54_BoK13bPrhxhfjSoSTVz414hA%40mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
Fix CLUSTER tuplesorts on abbreviated expressions.
commit : e4521841a1ee6917633bd28b34496c1627588731
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 20 Apr 2022 17:17:41 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 20 Apr 2022 17:17:41 -0700
CLUSTER sort won't use the datum1 SortTuple field when clustering
against an index whose leading key is an expression. This makes it
unsafe to use the abbreviated keys optimization, which was missed by the
logic that sets up SortSupport state. Affected tuplesorts output tuples
in a completely bogus order as a result (the wrong SortSupport based
comparator was used for the leading attribute).
This issue is similar to the bug fixed on the master branch by recent
commit cc58eecc5d. But it's a far older issue, that dates back to the
introduction of the abbreviated keys optimization by commit 4ea51cdfe8.
Backpatch to all supported versions.
Author: Peter Geoghegan <pg@bowt.ie>
Author: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CA+hUKG+bA+bmwD36_oDxAoLrCwZjVtST2fqe=b4=qZcmU7u89A@mail.gmail.com
Backpatch: 10-
M src/backend/utils/sort/tuplesort.c
M src/test/regress/expected/cluster.out
M src/test/regress/sql/cluster.sql
Disallow infinite endpoints in generate_series() for timestamps.
commit : e346329470081f5919e83b1acb2d0f2442c1fc3d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Apr 2022 18:08:15 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Apr 2022 18:08:15 -0400
Such cases will lead to infinite loops, so they're of no practical
value. The numeric variant of generate_series() already threw error
for this, so borrow its message wording.
Per report from Richard Wesley. Back-patch to all supported branches.
Discussion: https://postgr.es/m/91B44E7B-68D5-448F-95C8-B4B3B0F5DEAF@duckdblabs.com
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Allow db.schema.table patterns, but complain about random garbage.
commit : 4a66300acd8c788998615ffc077b7d7be57afceb
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 20 Apr 2022 11:02:35 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 20 Apr 2022 11:02:35 -0400
psql, pg_dump, and pg_amcheck share code to process object name
patterns like 'foo*.bar*' to match all tables with names starting in
'bar' that are in schemas starting with 'foo'. Before v14, any number
of extra name parts were silently ignored, so a command line '\d
foo.bar.baz.bletch.quux' was interpreted as '\d bletch.quux'. In v14,
as a result of commit 2c8726c4b0a496608919d1f78a5abc8c9b6e0868, we
instead treated this as a request for table quux in a schema named
'foo.bar.baz.bletch'. That caused problems for people like Justin
Pryzby who were accustomed to copying strings of the form
db.schema.table from messages generated by PostgreSQL itself and using
them as arguments to \d.
Accordingly, revise things so that if an object name pattern contains
more parts than we're expecting, we throw an error, unless there's
exactly one extra part and it matches the current database name.
That way, thisdb.myschema.mytable is accepted as meaning just
myschema.mytable, but otherdb.myschema.mytable is an error, and so
is some.random.garbage.myschema.mytable.
Mark Dilger, per report from Justin Pryzby and discussion among
various people.
Discussion: https://www.postgresql.org/message-id/20211013165426.GD27491%40telsasoft.com
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/pg_amcheck/pg_amcheck.c
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/psql/describe.c
M src/fe_utils/string_utils.c
M src/include/fe_utils/string_utils.h
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
Stabilize streaming tests in test_decoding.
commit : 7891a0d5d9fa53a0ba55ff70d77db54c7e50998f
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 20 Apr 2022 08:48:25 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 20 Apr 2022 08:48:25 +0530
We have some streaming tests that rely on the size of changes which can
fail if there are additional changes like invalidation messages by
background activity like auto analyze. Avoid such failures by increasing
autovacuum_naptime to a reasonably high value (1d).
Author: Dilip Kumar
Backpatch-through: 14
Discussion: https://postgr.es/m/1958043.1650129119@sss.pgh.pa.us
M contrib/test_decoding/logical.conf
Fix breakage in AlterFunction().
commit : 08a9e7a8c7917233926802aaea94a5529a747a50
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Apr 2022 23:03:59 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Apr 2022 23:03:59 -0400
An ALTER FUNCTION command that tried to update both the function's
proparallel property and its proconfig list failed to do the former,
because it stored the new proparallel value into a tuple that was
no longer the interesting one. Carelessness in 7aea8e4f2.
(I did not bother with a regression test, because the only likely
future breakage would be for someone to ignore the comment I added
and add some other field update after the heap_modify_tuple step.
A test using existing function properties could not catch that.)
Per report from Bryn Llewellyn. Back-patch to all supported branches.
Discussion: https://postgr.es/m/8AC9A37F-99BD-446F-A2F7-B89AD0022774@yugabyte.com
M src/backend/commands/functioncmds.c
Fix extract epoch from interval calculation
commit : 7a8d8219ccbf0963f2171ea23e1616ee6a7823ba
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 19 Apr 2022 20:38:53 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 19 Apr 2022 20:38:53 +0200
The new numeric code for extract epoch from interval accidentally
truncated the DAYS_PER_YEAR value to an integer, leading to results
that mismatched the floating-point interval_part calculations.
The commit a2da77cdb4661826482ebf2ddba1f953bc74afe4 that introduced
this actually contains the regression test change that this reverts.
I suppose this was missed at the time.
Reported-by: Joseph Koshakow <koshy44@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/CAAvxfHd5n%3D13NYA2q_tUq%3D3%3DSuWU-CufmTf-Ozj%3DfrEgt7pXwQ%40mail.gmail.com
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/interval.out
Fix the check to limit sync workers.
commit : c9dea58e270236a8d256a686f71f6bab960f0b80
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 19 Apr 2022 08:54:37 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 19 Apr 2022 08:54:37 +0530
We don't allow to invoke more sync workers once we have reached the sync
worker limit per subscription. But the check to enforce this also doesn't
allow to launch an apply worker if it gets restarted.
This code was introduced by commit de43897122 but we caught the problem
only with the test added by recent commit c91f71b9dc which started failing
occasionally in the buildfarm.
As per buildfarm.
Diagnosed-by: Amit Kapila, Masahiko Sawada, Tomas Vondra
Author: Amit Kapila
Backpatch-through: 10
Discussion: https://postgr.es/m/CAH2L28vddB_NFdRVpuyRBJEBWjz4BSyTB=_ektNRH8NJ1jf95g@mail.gmail.com
https://postgr.es/m/f90d2b03-4462-ce95-a524-d91464e797c8@enterprisedb.com
M src/backend/replication/logical/launcher.c
Avoid invalid array reference in transformAlterTableStmt().
commit : e805735a83f7bb5bfe773b9690f96a52ba2b84b2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 18 Apr 2022 12:16:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 18 Apr 2022 12:16:45 -0400
Don't try to look at the attidentity field of system attributes,
because they're not there in the TupleDescAttr array. Sometimes
this is harmless because we accidentally pick up a zero, but
otherwise we'll report "no owned sequence found" from an attempt
to alter a system attribute. (It seems possible that a SIGSEGV
could occur, too, though I've not seen it in testing.)
It's not in this function's charter to complain that you can't
alter a system column, so instead just hard-wire an assumption
that system attributes aren't identities. I didn't bother with
a regression test because the appearance of the bug is very
erratic.
Per bug #17465 from Roman Zharkov. Back-patch to all supported
branches. (There's not actually a live bug before v12, because
before that get_attidentity() did the right thing anyway.
But for consistency I changed the test in the older branches too.)
Discussion: https://postgr.es/m/17465-f2a554a6cb5740d3@postgresql.org
M src/backend/parser/parse_utilcmd.c
Fix race in TAP test 002_archiving.pl when restoring history file
commit : 8bcf90c7a67caf3009d7e529e15266cfd833c573
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 18 Apr 2022 11:40:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 18 Apr 2022 11:40:14 +0900
This test, introduced in df86e52, uses a second standby to check that
it is able to remove correctly RECOVERYHISTORY and RECOVERYXLOG at the
end of recovery. This standby uses the archives of the primary to
restore its contents, with some of the archive's contents coming from
the first standby previously promoted. In slow environments, it was
possible that the test did not check what it should, as the history file
generated by the promotion of the first standby may not be stored yet on
the archives the second standby feeds on. So, it could be possible that
the second standby selects an incorrect timeline, without restoring a
history file at all.
This commits adds a wait phase to make sure that the history file
required by the second standby is archived before this cluster is
created. This relies on poll_query_until() with pg_stat_file() and an
absolute path, something not supported in REL_10_STABLE.
While on it, this adds a new test to check that the history file has
been restored by looking at the logs of the second standby. This
ensures that a RECOVERYHISTORY, whose removal needs to be checked,
is created in the first place. This should make the test more robust.
This test has been introduced by df86e52, but it came in light as an
effect of the bug fixed by acf1dd42, where the extra restore_command
calls made the test much slower.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/YlT23IvsXkGuLzFi@paquier.xyz
Backpatch-through: 11
M src/test/recovery/t/002_archiving.pl
Add a temp-install prerequisite to src/interfaces/ecpg "checktcp".
commit : acd0eb635e859127c456c2166ed9455a1fdac60e
author : Noah Misch <noah@leadboat.com>
date : Sat, 16 Apr 2022 17:43:54 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 16 Apr 2022 17:43:54 -0700
The target failed, tested $PATH binaries, or tested a stale temporary
installation. Commit c66b438db62748000700c9b90b585e756dd54141 missed
this. Back-patch to v10 (all supported versions).
M src/interfaces/ecpg/Makefile
Rethink the delay-checkpoint-end mechanism in the back-branches.
commit : 10520f4346876aad4941797c2255a21bdac74739
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 14 Apr 2022 11:10:07 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 14 Apr 2022 11:10:07 -0400
The back-patch of commit bbace5697df12398e87ffd9879171c39d27f5b33 had
the unfortunate effect of changing the layout of PGPROC in the
back-branches, which could break extensions. This happened because it
changed the delayChkpt from type bool to type int. So, change it back,
and add a new bool delayChkptEnd field instead. The new field should
fall within what used to be padding space within the struct, and so
hopefully won't cause any extensions to break.
Per report from Markus Wanner and discussion with Tom Lane and others.
Patch originally by me, somewhat revised by Markus Wanner per a
suggestion from Michael Paquier. A very similar patch was developed
by Kyotaro Horiguchi, but I failed to see the email in which that was
posted before writing one of my own.
Discussion: http://postgr.es/m/CA+Tgmoao-kUD9c5nG5sub3F7tbo39+cdr8jKaOVEs_1aBWcJ3Q@mail.gmail.com
Discussion: http://postgr.es/m/20220406.164521.17171257901083417.horikyota.ntt@gmail.com
M src/backend/access/transam/multixact.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xloginsert.c
M src/backend/catalog/storage.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/ipc/procarray.c
M src/include/storage/proc.h
M src/include/storage/procarray.h
pageinspect: Fix handling of all-zero pages
commit : df6bbe73b86cda28472b9ecc61ac2deff5b2837b
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Apr 2022 15:09:26 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Apr 2022 15:09:26 +0900
Getting from get_raw_page() an all-zero page is considered as a valid
case by the buffer manager and it can happen for example when finding a
corrupted page with zero_damaged_pages enabled (using zero_damaged_pages
to look at corrupted pages happens), or after a crash when a relation
file is extended before any WAL for its new data is generated (before a
vacuum or autovacuum job comes in to do some cleanup).
However, all the functions of pageinspect, as of the index AMs (except
hash that has its own idea of new pages), heap, the FSM or the page
header have never worked with all-zero pages, causing various crashes
when going through the page internals.
This commit changes all the pageinspect functions to be compliant with
all-zero pages, where the choice is made to return NULL or no rows for
SRFs when finding a new page. get_raw_page() still works the same way,
returning a batch of zeros in the bytea of the page retrieved. A hard
error could be used but NULL, while more invasive, is useful when
scanning relation files in full to get a batch of results for a single
relation in one query. Tests are added for all the code paths
impacted.
Reported-by: Daria Lepikhova
Author: Michael Paquier
Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru
Backpatch-through: 10
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/btreefuncs.c
M contrib/pageinspect/expected/brin.out
M contrib/pageinspect/expected/btree.out
M contrib/pageinspect/expected/gin.out
M contrib/pageinspect/expected/gist.out
M contrib/pageinspect/expected/hash.out
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/fsmfuncs.c
M contrib/pageinspect/ginfuncs.c
M contrib/pageinspect/gistfuncs.c
M contrib/pageinspect/rawpage.c
M contrib/pageinspect/sql/brin.sql
M contrib/pageinspect/sql/btree.sql
M contrib/pageinspect/sql/gin.sql
M contrib/pageinspect/sql/gist.sql
M contrib/pageinspect/sql/hash.sql
M contrib/pageinspect/sql/page.sql
Prevent access to no-longer-pinned buffer in heapam_tuple_lock().
commit : c590e514a90ddc9237a3438efb05be074d43452b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Apr 2022 13:35:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Apr 2022 13:35:02 -0400
heap_fetch() used to have a "keep_buf" parameter that told it to return
ownership of the buffer pin to the caller after finding that the
requested tuple TID exists but is invisible to the specified snapshot.
This was thoughtlessly removed in commit 5db6df0c0, which broke
heapam_tuple_lock() (formerly EvalPlanQualFetch) because that function
needs to do more accesses to the tuple even if it's invisible. The net
effect is that we would continue to touch the page for a microsecond or
two after releasing pin on the buffer. Usually no harm would result;
but if a different session decided to defragment the page concurrently,
we could see garbage data and mistakenly conclude that there's no newer
tuple version to chain up to. (It's hard to say whether this has
happened in the field. The bug was actually found thanks to a later
change that allowed valgrind to detect accesses to non-pinned buffers.)
The most reasonable way to fix this is to reintroduce keep_buf,
although I made it behave slightly differently: buffer ownership
is passed back only if there is a valid tuple at the requested TID.
In HEAD, we can just add the parameter back to heap_fetch().
To avoid an API break in the back branches, introduce an additional
function heap_fetch_extended() in those branches.
In HEAD there is an additional, less obvious API change: tuple->t_data
will be set to NULL in all cases where buffer ownership is not returned,
in particular when the tuple exists but fails the time qual (and
!keep_buf). This is to defend against any other callers attempting to
access non-pinned buffers. We concluded that making that change in back
branches would be more likely to introduce problems than cure any.
In passing, remove a comment about heap_fetch that was obsoleted by
9a8ee1dc6.
Per bug #17462 from Daniil Anisimov. Back-patch to v12 where the bug
was introduced.
Discussion: https://postgr.es/m/17462-9c98a0f00df9bd36@postgresql.org
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/include/access/heapam.h
Docs: wording improvement for compute_query_id = regress
commit : ea669b8088380cc0bc7c48ab8581ea5fba1c5b4f
author : David Rowley <drowley@postgresql.org>
date : Wed, 13 Apr 2022 21:29:35 +1200
committer: David Rowley <drowley@postgresql.org>
date : Wed, 13 Apr 2022 21:29:35 +1200
It's more accurate to say that the query identifier is not shown when
compute_query_id = regress rather than to say it is hidden.
This change (ebf6c5249) appeared in v14, so it makes sense to backpatch
this small adjustment to keep the documents consistent between v14 and
master.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20220411020336.GB26620@telsasoft.com
Backpatch-through: 14, where compute_query_id = regress was added
M doc/src/sgml/config.sgml
Docs: adjust pg_upgrade syntax to mark -B as optional
commit : e286be5d05311ec060a4359fcbc53eb98b914d42
author : David Rowley <drowley@postgresql.org>
date : Wed, 13 Apr 2022 11:19:34 +1200
committer: David Rowley <drowley@postgresql.org>
date : Wed, 13 Apr 2022 11:19:34 +1200
This was made optional in 959f6d6a1.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20220411020336.GB26620@telsasoft.com
Backpatch-through: 13, where -B was made optional
M doc/src/sgml/ref/pgupgrade.sgml
Doc: tweak textsearch.sgml for SEO purposes.
commit : 8320a34d38e220d99eb24fbd09536b12711fb102
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 12 Apr 2022 18:21:04 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 12 Apr 2022 18:21:04 -0400
Google seems to like to return textsearch.html for queries about
GIN and GiST indexes, even though it's not a primary reference
for either. It seems likely that that's because those keywords
appear in the page title. Since "GIN and GiST Index Types" is
not a very apposite title for this material anyway, rename the
section in hopes of stopping that.
Also provide explicit links to the GIN and GiST chapters, to help
anyone who finds their way to this page regardless.
Per gripe from Jan Piotrowski. Back-patch to supported branches.
(Unfortunately Google is likely to continue returning the 9.1
version of this page, but improving that situation is a matter
for the www team.)
Discussion: https://postgr.es/m/164978902252.1276550.9330175733459697101@wrigleys.postgresql.org
M doc/src/sgml/textsearch.sgml
Docs: avoid confusing use of the word "synchronized"
commit : 3a95dfe4025f48fb57262ce590ba397a80a1a7b6
author : David Rowley <drowley@postgresql.org>
date : Wed, 13 Apr 2022 09:16:33 +1200
committer: David Rowley <drowley@postgresql.org>
date : Wed, 13 Apr 2022 09:16:33 +1200
It's misleading to call the data directory the "synchronized data
directory" when discussing a crash scenario when using pg_rewind's
--no-sync option. Here we just remove the word "synchronized" to avoid
any possible confusion.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20220411020336.GB26620@telsasoft.com
Backpatch-through: 12, where --no-sync was added
M doc/src/sgml/ref/pg_rewind.sgml
Suppress "variable 'pagesaving' set but not used" warning.
commit : a65747b1c787cc85dc59f088543f3d333f310eee
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Apr 2022 17:03:35 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Apr 2022 17:03:35 -0400
With asserts disabled, late-model clang notices that this variable
is incremented but never otherwise read.
Discussion: https://postgr.es/m/3171401.1649275153@sss.pgh.pa.us
M src/backend/access/nbtree/nbtdedup.c
Remove race condition in 022_crash_temp_files.pl test.
commit : 9a7229948c70945ca6ef0b36adfe61b74f4fdaf5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 5 Apr 2022 20:44:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 5 Apr 2022 20:44:01 -0400
It's possible for the query that "waits for restart" to complete a
successful iteration before the postmaster has noticed its SIGKILL'd
child and begun the restart cycle. (This is a bit hard to believe
perhaps, but it's been seen at least twice in the buildfarm, mainly
on ancient platforms that likely have quirky schedulers.)
To provide a more secure interlock, wait for the other session
we're using to report that it's been forcibly shut down.
Patch by me, based on a suggestion from Andres Freund.
Back-patch to v14 where this test case came in.
Discussion: https://postgr.es/m/1801850.1649047827@sss.pgh.pa.us
M src/test/recovery/t/022_crash_temp_files.pl
Update some tests in 013_crash_restart.pl.
commit : 8803df4ea960117f42f38ee82a74e954e72e0339
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Apr 2022 22:10:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Apr 2022 22:10:07 -0400
The expected backend message after SIGQUIT changed in commit
7e784d1dc, but we missed updating this test case. Also, experience
shows that we might sometimes get "could not send data to server"
instead of either of the libpq messages the test is looking for.
Per report from Mark Dilger. Back-patch to v14 where the
backend message changed.
Discussion: https://postgr.es/m/17BD82D7-49AC-40C9-8204-E7ADD30321A0@enterprisedb.com
M src/test/recovery/t/013_crash_restart.pl
Doc: Remove MultiXact wraparound section link.
commit : 32558a8bc3c0035e7b738e47b00ff6b4bfc96da8
author : Peter Geoghegan <pg@bowt.ie>
date : Sat, 2 Apr 2022 09:49:06 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Sat, 2 Apr 2022 09:49:06 -0700
Remove circular "25.1.5.1. Multixacts And Wraparound" link that
references the section that the link itself appears in. An explanation
of MultiXactId age appears only a few sentences before the link, so
there's no question that the link is superfluous at best.
Oversight in commit d5409295.
Author: Peter Geoghegan <pg@bowt.ie>
Backpatch: 14-
M doc/src/sgml/maintenance.sgml
Remove obsolete comment
commit : d480ae069e6c13dcb719b8c6aaf7a5540196a1b3
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 2 Apr 2022 07:27:26 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 2 Apr 2022 07:27:26 +0200
accidentally left behind by 4cb658af70027c3544fb843d77b2e84028762747
M src/include/utils/rel.h
libpq: Fix pkg-config without OpenSSL
commit : 7a27892750e966998381e6c34cdfec30ac5e3906
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 1 Apr 2022 17:12:56 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 1 Apr 2022 17:12:56 +0200
Do not add OpenSSL dependencies to libpq pkg-config file if OpenSSL is
not enabled. Oversight in beff361bc1edc24ee5f8b2073a1e5e4c92ea66eb.
Author: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/20220331163759.32665-1-fontaine.fabrice%40gmail.com
M src/interfaces/libpq/Makefile
Fix postgres_fdw to check shippability of sort clauses properly.
commit : 9f9489aa2eefeaf4a6517b0b14f8f22938c2e3f3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 31 Mar 2022 14:29:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 31 Mar 2022 14:29:24 -0400
postgres_fdw would push ORDER BY clauses to the remote side without
verifying that the sort operator is safe to ship. Moreover, it failed
to print a suitable USING clause if the sort operator isn't default
for the sort expression's type. The net result of this is that the
remote sort might not have anywhere near the semantics we expect,
which'd be disastrous for locally-performed merge joins in particular.
We addressed similar issues in the context of ORDER BY within an
aggregate function call in commit 7012b132d, but failed to notice
that query-level ORDER BY was broken. Thus, much of the necessary
logic already existed, but it requires refactoring to be usable
in both cases.
Back-patch to all supported branches. In HEAD only, remove the
core code's copy of find_em_expr_for_rel, which is no longer used
and really should never have been pushed into equivclass.c in the
first place.
Ronan Dunklau, per report from David Rowley;
reviews by David Rowley, Ranier Vilela, and myself
Discussion: https://postgr.es/m/CAApHDvr4OeC2DBVY--zVP83-K=bYrTD7F8SZDhN4g+pj2f2S-A@mail.gmail.com
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/sql/postgres_fdw.sql
Add missing newline in one libpq error message.
commit : 402279afe48d26c6657bec55256bd91d924e4086
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 31 Mar 2022 11:24:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 31 Mar 2022 11:24:26 -0400
Oversight in commit a59c79564. Back-patch, as that was.
Noted by Peter Eisentraut.
Discussion: https://postgr.es/m/7f85ef6d-250b-f5ec-9867-89f0b16d019f@enterprisedb.com
M src/interfaces/libpq/fe-secure-openssl.c
doc: Fix typo in ANALYZE documentation
commit : c5479178441ea68341afa2d2bf41738dabbefc71
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 31 Mar 2022 12:03:33 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 31 Mar 2022 12:03:33 +0200
Commit 61fa6ca79b3 accidentally wrote constrast instead of contrast.
Backpatch-through: 10
Discussion: https://postgr.es/m/88903179-5ce2-3d4d-af43-7830372bdcb6@enterprisedb.com
M doc/src/sgml/ref/analyze.sgml
Fix typo in comment.
commit : 637afee327f4690afc3f16fd76123cd4b7af1d95
author : Etsuro Fujita <efujita@postgresql.org>
date : Wed, 30 Mar 2022 19:00:02 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Wed, 30 Mar 2022 19:00:02 +0900
M src/backend/commands/copyfrom.c
Revert "Fix replay of create database records on standby"
commit : adc943b4e1fe26c3085e81c65f5b185c7ff0611a
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 29 Mar 2022 15:36:21 +0200
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 29 Mar 2022 15:36:21 +0200
This reverts commit 49d9cfc68bf4. The approach taken by this patch has
problems, so we'll come up with a radically different fix.
Discussion: https://postgr.es/m/CA+TgmoYcUPL+WOJL2ZzhH=zmrhj0iOQ=iCFM0SuYqBbqZEamEg@mail.gmail.com
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogutils.c
M src/backend/commands/dbcommands.c
M src/backend/commands/tablespace.c
M src/include/access/xlogutils.h
M src/tools/pgindent/typedefs.list
Document autoanalyze limitations for partitioned tables
commit : 6b262f353ef710704a9388d32924b8c10c7e6eaa
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 28 Mar 2022 14:27:36 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 28 Mar 2022 14:27:36 +0200
When dealing with partitioned tables, counters for partitioned tables
are not updated when modifying child tables. This means autoanalyze may
not update optimizer statistics for the parent relations, which can
result in poor plans for some queries.
It's worth documenting this limitation, so that people are aware of it
and can take steps to mitigate it (e.g. by setting up a script executing
ANALYZE regularly).
Backpatch to v10. Older branches are affected too, of couse, but we no
longer maintain those.
Author: Justin Pryzby
Reviewed-by: Zhihong Yu, Tomas Vondra
Backpatch-through: 10
Discussion: https://postgr.es/m/20210913035409.GA10647%40telsasoft.com
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/ref/analyze.sgml
Fix NULL input behaviour of pg_stat_get_replication_slot().
commit : c1a0d7d1c4b77d978fe93edc0889589148cbad23
author : Andres Freund <andres@anarazel.de>
date : Sun, 27 Mar 2022 21:39:43 -0700
committer: Andres Freund <andres@anarazel.de>
date : Sun, 27 Mar 2022 21:39:43 -0700
pg_stat_get_replication_slot() accidentally was marked as non-strict, crashing
when called with NULL input. As it's already released, introduce an explicit
NULL check in 14, fix the catalog in HEAD.
Bumps catversion in HEAD.
Discussion: https://postgr.es/m/20220326212432.s5n2maw6kugnpyxw@alap3.anarazel.de
Backpatch: 14-, where replication slot stats were introduced
M src/backend/utils/adt/pgstatfuncs.c
M src/test/regress/expected/stats.out
M src/test/regress/sql/stats.sql
waldump: fix use-after-free in search_directory().
commit : 6839aa7a6921d133a51e6162b9ee710ab712293d
author : Andres Freund <andres@anarazel.de>
date : Wed, 23 Mar 2022 16:38:43 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 23 Mar 2022 16:38:43 -0700
After closedir() dirent->d_name is not valid anymore. As there alerady are a
few places relying on the limited lifetime of pg_waldump, do so here as well,
and just pg_strdup() the string.
The bug was introduced in fc49e24fa69a.
Found by UBSan, run locally.
Backpatch: 11-, like fc49e24fa69 itself.
M src/bin/pg_waldump/pg_waldump.c
Fix breakage of get_ps_display() in the PS_USE_NONE case.
commit : 3f7a59c59b4873a3b39838401d590f9d030a3500
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 27 Mar 2022 12:57:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 27 Mar 2022 12:57:46 -0400
Commit 8c6d30f21 caused this function to fail to set *displen
in the PS_USE_NONE code path. If the variable's previous value
had been negative, that'd lead to a memory clobber at some call
sites. We'd managed not to notice due to very thin test coverage
of such configurations, but this appears to explain buildfarm member
lorikeet's recent struggles.
Credit to Andrew Dunstan for spotting the problem. Back-patch
to v13 where the bug was introduced.
Discussion: https://postgr.es/m/136102.1648320427@sss.pgh.pa.us
M src/backend/utils/misc/ps_status.c
pageinspect: Add more sanity checks to prevent out-of-bound reads
commit : 27d384441851fe28ced962665632f2d1b914f11b
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 27 Mar 2022 17:53:51 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 27 Mar 2022 17:53:51 +0900
A couple of code paths use the special area on the page passed by the
function caller, expecting to find some data in it. However, feeding
an incorrect page can lead to out-of-bound reads when trying to access
the page special area (like a heap page that has no special area,
leading PageGetSpecialPointer() to grab a pointer outside the allocated
page).
The functions used for hash and btree indexes have some protection
already against that, while some other functions using a relation OID
as argument would make sure that the access method involved is correct,
but functions taking in input a raw page without knowing the relation
the page is attached to would run into problems.
This commit improves the set of checks used in the code paths of BRIN,
btree (including one check if a leaf page is found with a non-zero
level), GIN and GiST to verify that the page given in input has a
special area size that fits with each access method, which is done
though PageGetSpecialSize(), becore calling PageGetSpecialPointer().
The scope of the checks done is limited to work with pages that one
would pass after getting a block with get_raw_page(), as it is possible
to craft byteas that could bypass existing code paths. Having too many
checks would also impact the usability of pageinspect, as the existing
code is very useful to look at the content details in a corrupted page,
so the focus is really to avoid out-of-bound reads as this is never a
good thing even with functions whose execution is limited to
superusers.
The safest approach could be to rework the functions so as these fetch a
block using a relation OID and a block number, but there are also cases
where using a raw page is useful.
Tests are added to cover all the code paths that needed such checks, and
an error message for hash indexes is reworded to fit better with what
this commit adds.
Reported-By: Alexander Lakhin
Author: Julien Rouhaud, Michael Paquier
Discussion: https://postgr.es/m/16527-ef7606186f0610a1@postgresql.org
Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru
Backpatch-through: 10
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/btreefuncs.c
M contrib/pageinspect/expected/brin.out
M contrib/pageinspect/expected/btree.out
M contrib/pageinspect/expected/gin.out
M contrib/pageinspect/expected/gist.out
M contrib/pageinspect/expected/hash.out
M contrib/pageinspect/ginfuncs.c
M contrib/pageinspect/gistfuncs.c
M contrib/pageinspect/hashfuncs.c
M contrib/pageinspect/sql/brin.sql
M contrib/pageinspect/sql/btree.sql
M contrib/pageinspect/sql/gin.sql
M contrib/pageinspect/sql/gist.sql
M contrib/pageinspect/sql/hash.sql
Suppress compiler warning in relptr_store().
commit : 0144c9c7e73dc75f9cd5297c7c380d55927beb75
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 26 Mar 2022 14:29:29 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 26 Mar 2022 14:29:29 -0400
clang 13 with -Wextra warns that "performing pointer subtraction with
a null pointer has undefined behavior" in the places where freepage.c
tries to set a relptr variable to constant NULL. This appears to be
a compiler bug, but it's unlikely to get fixed instantly. Fortunately,
we can work around it by introducing an inline support function, which
seems like a good change anyway because it removes the macro's existing
double-evaluation hazard.
Backpatch to v10 where this code was introduced.
Patch by me, based on an idea of Andres Freund's.
Discussion: https://postgr.es/m/48826.1648310694@sss.pgh.pa.us
M src/include/utils/relptr.h
Harden TAP tests that intentionally corrupt page checksums.
commit : 579cef5faf11b63a312691e086415b37d7888b5a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 25 Mar 2022 14:23:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 25 Mar 2022 14:23:26 -0400
The previous method for doing that was to write zeroes into a
predetermined set of page locations. However, there's a roughly
1-in-64K chance that the existing checksum will match by chance,
and yesterday several buildfarm animals started to reproducibly
see that, resulting in test failures because no checksum mismatch
was reported.
Since the checksum includes the page LSN, test success depends on
the length of the installation's WAL history, which is affected by
(at least) the initial catalog contents, the set of locales installed
on the system, and the length of the pathname of the test directory.
Sooner or later we were going to hit a chance match, and today is
that day.
Harden these tests by specifically inverting the checksum field and
leaving all else alone, thereby guaranteeing that the checksum is
incorrect.
In passing, fix places that were using seek() to set up for syswrite(),
a combination that the Perl docs very explicitly warn against. We've
probably escaped problems because no regular buffered I/O is done on
these filehandles; but if it ever breaks, we wouldn't deserve or get
much sympathy.
Although we've only seen problems in HEAD, now that we recognize the
environmental dependencies it seems like it might be just a matter
of time until someone manages to hit this in back-branch testing.
Hence, back-patch to v11 where we started doing this kind of test.
Discussion: https://postgr.es/m/3192026.1648185780@sss.pgh.pa.us
M contrib/amcheck/t/001_verify_heapam.pl
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_checksums/t/002_actions.pl
M src/test/perl/PostgresNode.pm
Fix replay of create database records on standby
commit : ffd28516e69983d44616451b808b1c9d4f3c2b60
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 25 Mar 2022 13:16:21 +0100
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 25 Mar 2022 13:16:21 +0100
Crash recovery on standby may encounter missing directories when
replaying create database WAL records. Prior to this patch, the standby
would fail to recover in such a case. However, the directories could be
legitimately missing. Consider a sequence of WAL records as follows:
CREATE DATABASE
DROP DATABASE
DROP TABLESPACE
If, after replaying the last WAL record and removing the tablespace
directory, the standby crashes and has to replay the create database
record again, the crash recovery must be able to move on.
This patch adds a mechanism similar to invalid-page tracking, to keep a
tally of missing directories during crash recovery. If all the missing
directory references are matched with corresponding drop records at the
end of crash recovery, the standby can safely continue following the
primary.
Backpatch to 13, at least for now. The bug is older, but fixing it in
older branches requires more careful study of the interactions with
commit e6d8069522c8, which appeared in 13.
A new TAP test file is added to verify the condition. However, because
it depends on commit d6d317dbf615, it can only be added to branch
master. I (Álvaro) manually verified that the code behaves as expected
in branch 14. It's a bit nervous-making to leave the code uncovered by
tests in older branches, but leaving the bug unfixed is even worse.
Also, the main reason this fix took so long is precisely that we
couldn't agree on a good strategy to approach testing for the bug, so
perhaps this is the best we can do.
Diagnosed-by: Paul Guo <paulguo@gmail.com>
Author: Paul Guo <paulguo@gmail.com>
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Author: Asim R Praveen <apraveen@pivotal.io>
Discussion: https://postgr.es/m/CAEET0ZGx9AvioViLf7nbR_8tH9-=27DN5xWJ2P9-ROH16e4JUA@mail.gmail.com
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogutils.c
M src/backend/commands/dbcommands.c
M src/backend/commands/tablespace.c
M src/include/access/xlogutils.h
M src/tools/pgindent/typedefs.list
Fix possible recovery trouble if TRUNCATE overlaps a checkpoint.
commit : bbace5697df12398e87ffd9879171c39d27f5b33
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 24 Mar 2022 14:32:24 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 24 Mar 2022 14:32:24 -0400
If TRUNCATE causes some buffers to be invalidated and thus the
checkpoint does not flush them, TRUNCATE must also ensure that the
corresponding files are truncated on disk. Otherwise, a replay
from the checkpoint might find that the buffers exist but have
the wrong contents, which may cause replay to fail.
Report by Teja Mupparti. Patch by Kyotaro Horiguchi, per a design
suggestion from Heikki Linnakangas, with some changes to the
comments by me. Review of this and a prior patch that approached
the issue differently by Heikki Linnakangas, Andres Freund, Álvaro
Herrera, Masahiko Sawada, and Tom Lane.
Discussion: http://postgr.es/m/BYAPR06MB6373BF50B469CA393C614257ABF00@BYAPR06MB6373.namprd06.prod.outlook.com
M src/backend/access/transam/multixact.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xloginsert.c
M src/backend/catalog/storage.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/lmgr/proc.c
M src/include/storage/proc.h
M src/include/storage/procarray.h
Don't try to translate NULL in GetConfigOptionByNum().
commit : 81045e1e1c3370fb7e57c8841b0a7b6aab31831b
author : Andres Freund <andres@anarazel.de>
date : Wed, 23 Mar 2022 13:05:59 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 23 Mar 2022 13:05:59 -0700
Noticed via -fsanitize=undefined. Introduced when a few columns in
GetConfigOptionByNum() / pg_settings started to be translated in 72be8c29a /
PG 12.
Backpatch to all affected branches, for the same reasons as 46ab07ffda9.
Discussion: https://postgr.es/m/20220323173537.ll7klrglnp4gn2um@alap3.anarazel.de
Backpatch: 12-
M src/backend/utils/misc/guc.c
Don't call fwrite() with len == 0 when writing out relcache init file.
commit : 89a94c24aadd8f40084339ea3d16e2e14fa9f629
author : Andres Freund <andres@anarazel.de>
date : Wed, 23 Mar 2022 13:05:25 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 23 Mar 2022 13:05:25 -0700
Noticed via -fsanitize=undefined.
Backpatch to all branches, for the same reasons as 46ab07ffda9.
Discussion: https://postgr.es/m/20220323173537.ll7klrglnp4gn2um@alap3.anarazel.de
Backpatch: 10-
M src/backend/utils/cache/relcache.c
configure: check for dlsym instead of dlopen.
commit : e52e9bd57f33da1c4a915e24d0a24759a23cb5fd
author : Andres Freund <andres@anarazel.de>
date : Wed, 23 Mar 2022 12:43:14 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 23 Mar 2022 12:43:14 -0700
When building with sanitizers the sanitizer library provides dlopen, but not
dlsym(), making configure think that -ldl isn't needed. Just checking for
dlsym() ought to suffice, hard to see dlsym() being provided without dlopen()
also being provided.
Backpatch to all branches, for the same reasons as 46ab07ffda9.
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20220323173537.ll7klrglnp4gn2um@alap3.anarazel.de
Backpatch: 10-
M configure
M configure.ac
pg_upgrade: Upgrade an Assert to a real 'if' test
commit : 9814c708c657a9f21fd1a36b5dc0f49ca2d01499
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 23 Mar 2022 19:23:51 +0100
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 23 Mar 2022 19:23:51 +0100
It seems possible for the condition being tested to be true in
production, and nobody would never know (except when some data
eventually becomes corrupt?).
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m//202109040001.zky3wgv2qeqg@alvherre.pgsql
M src/bin/pg_rewind/parsexlog.c
Fix "missing continuation record" after standby promotion
commit : caaeb88ff7ae84a3bbf0ee497a4bda18cf9261cd
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 23 Mar 2022 18:22:10 +0100
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 23 Mar 2022 18:22:10 +0100
Invalidate abortedRecPtr and missingContrecPtr after a missing
continuation record is successfully skipped on a standby. This fixes a
PANIC caused when a recently promoted standby attempts to write an
OVERWRITE_RECORD with an LSN of the previously read aborted record.
Backpatch to 10 (all stable versions).
Author: Sami Imseih <simseih@amazon.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/44D259DE-7542-49C4-8A52-2AB01534DCA9@amazon.com
M src/backend/access/transam/xlog.c
M src/test/recovery/t/026_overwrite_contrecord.pl
Try to stabilize vacuum test.
commit : cd3a5055f952ca2811e63d86558dd3e1f4147f62
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 23 Mar 2022 14:31:18 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 23 Mar 2022 14:31:18 +1300
As commits b700f96c and 3414099c did for the reloptions test, make
sure VACUUM can always truncate the table as expected.
Back-patch to 12, where vacuum_truncate arrived.
Discussion: https://postgr.es/m/CAD21AoCNoWjYkdEtr%2BVDoF9v__V905AedKZ9iF%3DArgCtrbxZqw%40mail.gmail.com
M src/test/regress/expected/vacuum.out
M src/test/regress/sql/vacuum.sql
Add missing dependency of pg_dumpall to WIN32RES.
commit : 2d608c9607584a30ff718d2c9f211ef3c692a315
author : Andres Freund <andres@anarazel.de>
date : Tue, 22 Mar 2022 08:22:02 -0700
committer: Andres Freund <andres@anarazel.de>
date : Tue, 22 Mar 2022 08:22:02 -0700
When cross-building to windows, or building with mingw on windows, the build
could fail with
x86_64-w64-mingw32-gcc: error: win32ver.o: No such file or director
because pg_dumpall didn't depend on WIN32RES, but it's recipe references
it. The build nevertheless succeeded most of the time, due to
pg_dump/pg_restore having the required dependency, causing win32ver.o to be
built.
Reported-By: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CA+hUKGJeekpUPWW6yCVdf9=oBAcCp86RrBivo4Y4cwazAzGPng@mail.gmail.com
Backpatch: 10-, omission present on all live branches
M src/bin/pg_dump/Makefile
Fix failures in SSL tests caused by out-of-tree keys and certificates
commit : fdb1be4962ca080c007308ddb7163bf76310bb7c
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 22 Mar 2022 13:21:33 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 22 Mar 2022 13:21:33 +0900
This issue is environment-sensitive, where the SSL tests could fail in
various way by feeding on defaults provided by sslcert, sslkey,
sslrootkey, sslrootcert, sslcrl and sslcrldir coming from a local setup,
as of ~/.postgresql/ by default. Horiguchi-san has reported two
failures, but more advanced testing from me (aka inclusion of garbage
SSL configuration in ~/.postgresql/ for all the configuration
parameters) has showed dozens of failures that can be triggered in the
whole test suite.
History has showed that we are not good when it comes to address such
issues, fixing them locally like in dd87799, and such problems keep
appearing. This commit strengthens the entire test suite to put an end
to this set of problems by embedding invalid default values in all the
connection strings used in the tests. The invalid values are prefixed
in each connection string, relying on the follow-up values passed in the
connection string to enforce any invalid value previously set. Note
that two tests related to CRLs are required to fail with certain pre-set
configurations, but we can rely on enforcing an empty value instead
after the invalid set of values.
Reported-by: Kyotaro Horiguchi
Reviewed-by: Andrew Dunstan, Daniel Gustafsson, Kyotaro Horiguchi
Discussion: https://postgr.es/m/20220316.163658.1122740600489097632.horikyota.ntt@gmail.com
backpatch-through: 10
M src/test/ssl/t/001_ssltests.pl
Fix assorted missing logic for GroupingFunc nodes.
commit : 48b6035f0f72c312aaa22733cb88f5290f473ac0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Mar 2022 17:44:29 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Mar 2022 17:44:29 -0400
The planner needs to treat GroupingFunc like Aggref for many purposes,
in particular with respect to processing of the argument expressions,
which are not to be evaluated at runtime. A few places hadn't gotten
that memo, notably including subselect.c's processing of outer-level
aggregates. This resulted in assertion failures or wrong plans for
cases in which a GROUPING() construct references an outer aggregation
level.
Also fix missing special cases for GroupingFunc in cost_qual_eval
(resulting in wrong cost estimates for GROUPING(), although it's
not clear that that would affect plan shapes in practice) and in
ruleutils.c (resulting in excess parentheses in pretty-print mode).
Per bug #17088 from Yaoguang Chen. Back-patch to all supported
branches.
Richard Guo, Tom Lane
Discussion: https://postgr.es/m/17088-e33882b387de7f5c@postgresql.org
M src/backend/nodes/nodeFuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/subselect.c
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Fix risk of deadlock failure while dropping a partitioned index.
commit : 05ccf974cd75d3a95869c4ce4b96fdbc10e663cf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Mar 2022 12:22:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Mar 2022 12:22:13 -0400
DROP INDEX needs to lock the index's table before the index itself,
else it will deadlock against ordinary queries that acquire the
relation locks in that order. This is correctly mechanized for
plain indexes by RangeVarCallbackForDropRelation; but in the case of
a partitioned index, we neglected to lock the child tables in advance
of locking the child indexes. We can fix that by traversing the
inheritance tree and acquiring the needed locks in RemoveRelations,
after we have acquired our locks on the parent partitioned table and
index.
While at it, do some refactoring to eliminate confusion between
the actual and expected relkind in RangeVarCallbackForDropRelation.
We can save a couple of syscache lookups too, by having that function
pass back info that RemoveRelations will need.
Back-patch to v11 where partitioned indexes were added.
Jimmy Yih, Gaurab Dey, Tom Lane
Discussion: https://postgr.es/m/BYAPR05MB645402330042E17D91A70C12BD5F9@BYAPR05MB6454.namprd05.prod.outlook.com
M src/backend/commands/tablecmds.c
A src/test/isolation/expected/partition-drop-index-locking.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/partition-drop-index-locking.spec
Doc: fix our example systemd script.
commit : a1efc8f8c820a1e94eb26f7c93e4c6f6e9b277d6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Mar 2022 12:39:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Mar 2022 12:39:40 -0400
The example used "TimeoutSec=0", but systemd's documented way to get
the desired effect is "TimeoutSec=infinity".
Discussion: https://postgr.es/m/164770078557.670.5467111518383664377@wrigleys.postgresql.org
M doc/src/sgml/runtime.sgml
doc: Mention SET TABLESPACE clause for ALTER MATERIALIZED VIEW
commit : c422e8da7ec26f7f43c05b6020fd80fd24d3c8a2
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 19 Mar 2022 16:37:39 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 19 Mar 2022 16:37:39 +0900
This command flavor is supported, but there was nothing in the
documentation about it.
Author: Yugo Nagata
Discussion: https://postgr.es/m/20220316133337.5dc9740abfa24c25ec9f67f5@sraoss.co.jp
Backpatch-through: 10
M doc/src/sgml/ref/alter_materialized_view.sgml
Fix incorrect xmlschema output for types timetz and timestamptz.
commit : ae8ec7febac1b414681c16022e6c655a62a2540a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Mar 2022 16:01:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Mar 2022 16:01:42 -0400
The output of table_to_xmlschema() and allied functions includes
a regex describing valid values for these types ... but the regex
was itself invalid, as it failed to escape a literal "+" sign.
Report and fix by Renan Soares Lopes. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/7f6fabaa-3f8f-49ab-89ca-59fbfe633105@me.com
M src/backend/utils/adt/xml.c
M src/test/regress/expected/xmlmap.out
M src/test/regress/expected/xmlmap_1.out
M src/test/regress/sql/xmlmap.sql
Revert applying column aliases to the output of whole-row Vars.
commit : 1d072bd2030af0f2eaa522449028ff160f71ebf8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Mar 2022 18:18:05 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Mar 2022 18:18:05 -0400
In commit bf7ca1587, I had the bright idea that we could make the
result of a whole-row Var (that is, foo.*) track any column aliases
that had been applied to the FROM entry the Var refers to. However,
that's not terribly logically consistent, because now the output of
the Var is no longer of the named composite type that the Var claims
to emit. bf7ca1587 tried to handle that by changing the output
tuple values to be labeled with a blessed RECORD type, but that's
really pretty disastrous: we can wind up storing such tuples onto
disk, whereupon they're not readable by other sessions.
The only practical fix I can see is to give up on what bf7ca1587
tried to do, and say that the column names of tuples produced by
a whole-row Var are always those of the underlying named composite
type, query aliases or no. While this introduces some inconsistencies,
it removes others, so it's not that awful in the abstract. What *is*
kind of awful is to make such a behavioral change in a back-patched
bug fix. But corrupt data is worse, so back-patched it will be.
(A workaround available to anyone who's unhappy about this is to
introduce an extra level of sub-SELECT, so that the whole-row Var is
referring to the sub-SELECT's output and not to a named table type.
Then the Var is of type RECORD to begin with and there's no issue.)
Per report from Miles Delahunty. The faulty commit dates to 9.5,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/2950001.1638729947@sss.pgh.pa.us
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/execTuples.c
M src/test/regress/expected/rowtypes.out
M src/test/regress/sql/rowtypes.sql
Fix publish_as_relid with multiple publications
commit : 677a1dc0ca0f33220ba1ea8067181a72b4aff536
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 16 Mar 2022 16:42:47 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 16 Mar 2022 16:42:47 +0100
Commit 83fd4532a7 allowed publishing of changes via ancestors, for
publications defined with publish_via_partition_root. But the way
the ancestor was determined in get_rel_sync_entry() was incorrect,
simply updating the same variable. So with multiple publications,
replicating different ancestors, the outcome depended on the order
of publications in the list - the value from the last loop was used,
even if it wasn't the top-most ancestor.
This is a probably rare situation, as in most cases publications do
not overlap, so each partition has exactly one candidate ancestor
to replicate as and there's no ambiguity.
Fixed by tracking the "ancestor level" for each publication, and
picking the top-most ancestor. Adds a test case, verifying the
correct ancestor is used for publishing the changes and that this
does not depend on order of publications in the list.
Older releases have another bug in this loop - once all actions are
replicated, the loop is terminated, on the assumption that inspecting
additional publications is unecessary. But that misses the fact that
those additional applications may replicate different ancestors.
Fixed by removal of this break condition. We might still terminate the
loop in some cases (e.g. when replicating all actions and the ancestor
is the partition root).
Backpatch to 13, where publish_via_partition_root was introduced.
Initial report and fix by me, test added by Hou zj. Reviews and
improvements by Amit Kapila.
Author: Tomas Vondra, Hou zj, Amit Kapila
Reviewed-by: Amit Kapila, Hou zj
Discussion: https://postgr.es/m/d26d24dd-2fab-3c48-0162-2b7f84a9c893%40enterprisedb.com
M src/backend/replication/pgoutput/pgoutput.c
M src/test/subscription/t/013_partition.pl
Fix default signature length for gist_ltree_ops
commit : 7d30f59da714ab25f82f4bc24f30cba1022d8cb9
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 16 Mar 2022 11:41:18 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 16 Mar 2022 11:41:18 +0300
911e702077 implemented operator class parameters including the signature length
in ltree. Previously, the signature length for gist_ltree_ops was 8. Because
of bug 911e702077 the default signature length for gist_ltree_ops became 28 for
ltree 1.1 (where options method is NOT provided) and 8 for ltree 1.2 (where
options method is provided). This commit changes the default signature length
for ltree 1.1 to 8.
Existing gist_ltree_ops indexes might be corrupted in various scenarios.
Thus, we have to recommend reindexing all the gist_ltree_ops indexes after
the upgrade.
Reported-by: Victor Yegorov
Reviewed-by: Tomas Vondra, Tom Lane, Andres Freund, Nikita Glukhov
Reviewed-by: Andrew Dunstan
Author: Tomas Vondra, Alexander Korotkov
Discussion: https://postgr.es/m/17406-71e02820ae79bb40%40postgresql.org
Discussion: https://postgr.es/m/d80e0a55-6c3e-5b26-53e3-3c4f973f737c%40enterprisedb.com
M contrib/ltree/ltree.h
M contrib/ltree/ltree_gist.c
Fix race between DROP TABLESPACE and checkpointing.
commit : 26e00793980996fada8d61181d45070ce9fee600
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 16 Mar 2022 17:20:24 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 16 Mar 2022 17:20:24 +1300
Commands like ALTER TABLE SET TABLESPACE may leave files for the next
checkpoint to clean up. If such files are not removed by the time DROP
TABLESPACE is called, we request a checkpoint so that they are deleted.
However, there is presently a window before checkpoint start where new
unlink requests won't be scheduled until the following checkpoint. This
means that the checkpoint forced by DROP TABLESPACE might not remove the
files we expect it to remove, and the following ERROR will be emitted:
ERROR: tablespace "mytblspc" is not empty
To fix, add a call to AbsorbSyncRequests() just before advancing the
unlink cycle counter. This ensures that any unlink requests forwarded
prior to checkpoint start (i.e., when ckpt_started is incremented) will
be processed by the current checkpoint. Since AbsorbSyncRequests()
performs memory allocations, it cannot be called within a critical
section, so we also need to move SyncPreCheckpoint() to before
CreateCheckPoint()'s critical section.
This is an old bug, so back-patch to all supported versions.
Author: Nathan Bossart <nathandbossart@gmail.com>
Reported-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20220215235845.GA2665318%40nathanxps13
M src/backend/access/transam/xlog.c
M src/backend/storage/sync/sync.c
pageinspect: Fix memory context allocation of page in brin_revmap_data()
commit : dc5b3bdae31690e11839079e15b74ba0df0aa167
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 16 Mar 2022 12:29:47 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 16 Mar 2022 12:29:47 +0900
This caused the function to fail, as the aligned copy of the raw page
given by the function caller was not saved in the correct memory
context, which needs to be multi_call_memory_ctx in this case.
Issue introduced by 076f4d9.
Per buildfarm members sifika, mylodon and longfin. I have reproduced
that locally with macos.
Discussion: https://postgr.es/m/YjFPOtfCW6yLXUeM@paquier.xyz
Backpatch-through: 10
M contrib/pageinspect/brinfuncs.c
Fix documentation typo in commit 5e6368b4.
commit : ea70f6945029b622ea28d53c7a32354d1d033246
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 16 Mar 2022 15:53:28 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 16 Mar 2022 15:53:28 +1300
Back-patch to 14.
M doc/src/sgml/monitoring.sgml
Fix waiting in RegisterSyncRequest().
commit : 1396b5c6ed2bf46433c77ad0ce7f1ddaa984c224
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 16 Mar 2022 15:35:42 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 16 Mar 2022 15:35:42 +1300
If we run out of space in the checkpointer sync request queue (which is
hopefully rare on real systems, but common with very small buffer pool),
we wait for it to drain. While waiting, we should report that as a wait
event so that users know what is going on, and also handle postmaster
death, since otherwise the loop might never terminate if the
checkpointer has exited.
Back-patch to 12. Although the problem exists in earlier releases too,
the code is structured differently before 12 so I haven't gone any
further for now, in the absence of field complaints.
Reported-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20220226213942.nb7uvb2pamyu26dj%40alap3.anarazel.de
M doc/src/sgml/monitoring.sgml
M src/backend/storage/sync/sync.c
M src/backend/utils/activity/wait_event.c
M src/include/utils/wait_event.h
pageinspect: Fix handling of page sizes and AM types
commit : b61e6214cface3def6241295edadf4eeaf4c884d
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 16 Mar 2022 11:20:47 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 16 Mar 2022 11:20:47 +0900
This commit fixes a set of issues related to the use of the SQL
functions in this module when the caller is able to pass down raw page
data as input argument:
- The page size check was fuzzy in a couple of places, sometimes
looking after only a sub-range, but what we are looking for is an exact
match on BLCKSZ. After considering a few options here, I have settled
down to do a generalization of get_page_from_raw(). Most of the SQL
functions already used that, and this is not strictly required if not
accessing an 8-byte-wide value from a raw page, but this feels safer in
the long run for alignment-picky environment, particularly if a code
path begins to access such values. This also reduces the number of
strings that need to be translated.
- The BRIN function brin_page_items() uses a Relation but it did not
check the access method of the opened index, potentially leading to
crashes. All the other functions in need of a Relation already did
that.
- Some code paths could fail on elog(), but we should to use ereport()
for failures that can be triggered by the user.
Tests are added to stress all the cases that are fixed as of this
commit, with some junk raw pages (\set VERBOSITY ensures that this works
across all page sizes) and unexpected index types when functions open
relations.
Author: Michael Paquier, Justin Prysby
Discussion: https://postgr.es/m/20220218030020.GA1137@telsasoft.com
Backpatch-through: 10
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/btreefuncs.c
M contrib/pageinspect/expected/brin.out
M contrib/pageinspect/expected/btree.out
M contrib/pageinspect/expected/gin.out
M contrib/pageinspect/expected/gist.out
M contrib/pageinspect/expected/hash.out
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/fsmfuncs.c
M contrib/pageinspect/gistfuncs.c
M contrib/pageinspect/hashfuncs.c
M contrib/pageinspect/rawpage.c
M contrib/pageinspect/sql/brin.sql
M contrib/pageinspect/sql/btree.sql
M contrib/pageinspect/sql/gin.sql
M contrib/pageinspect/sql/gist.sql
M contrib/pageinspect/sql/hash.sql
M contrib/pageinspect/sql/page.sql
Wake up for latches in CheckpointWriteDelay().
commit : 78c0f85e435bf327dcf2c1aded47b51263b32b82
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 16 Mar 2022 13:37:58 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 16 Mar 2022 13:37:58 +1300
The checkpointer shouldn't ignore its latch. Other backends may be
waiting for it to drain the request queue. Hopefully real systems don't
have a full queue often, but the condition is reached easily when
shared_buffers is small.
This involves defining a new wait event, which will appear in the
pg_stat_activity view often due to spread checkpoints.
Back-patch only to 14. Even though the problem exists in earlier
branches too, it's hard to hit there. In 14 we stopped using signal
handlers for latches on Linux, *BSD and macOS, which were previously
hiding this problem by interrupting the sleep (though not reliably, as
the signal could arrive before the sleep begins; precisely the problem
latches address).
Reported-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20220226213942.nb7uvb2pamyu26dj%40alap3.anarazel.de
M doc/src/sgml/monitoring.sgml
M src/backend/postmaster/checkpointer.c
M src/backend/utils/activity/wait_event.c
M src/include/utils/wait_event.h
Back-patch LLVM 14 API changes.
commit : d9f7ad54e552262ee0090e88d5abd3e04fcdeac8
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 16 Mar 2022 11:35:00 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 16 Mar 2022 11:35:00 +1300
Since LLVM 14 has stopped changing and is about to be released,
back-patch the following changes from the master branch:
e6a7600202105919bffd62b3dfd941f4a94e082b
807fee1a39de6bb8184082012e643951abb9ad1d
a56e7b66010f330782243de9e25ac2a6596be0e1
Back-patch to 11, where LLVM JIT support came in.
M src/backend/jit/llvm/Makefile
M src/backend/jit/llvm/llvmjit_error.cpp
M src/backend/jit/llvm/llvmjit_inline.cpp
Restore the previous semantics of get_constraint_index().
commit : 8dcd1c3564f04bc1f71020c150b31deea07b7a95
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 11 Mar 2022 13:47:26 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 11 Mar 2022 13:47:26 -0500
Commit 8b069ef5d changed this function to look at pg_constraint.conindid
rather than searching pg_depend. That was a good performance improvement,
but it failed to preserve the exact semantics. The old code would only
return an index that was "owned by" (internally dependent on) the
specified constraint, whereas the new code will also return indexes that
are just referenced by foreign key constraints. This confuses ALTER
TABLE, which was implicitly expecting the previous semantics, into
failing with errors like
ERROR: relation 146621 has multiple clustered indexes
or
ERROR: "pk_attbl" is not an index for table "atref"
We can fix this without reverting the performance improvement by adding
a contype check in get_constraint_index(). Another way could be to
make ALTER TABLE check it, but I'm worried that extension code could
also have subtle dependencies on the old semantics.
Tom Lane and Japin Li, per bug #17409 from Holly Roberts.
Back-patch to v14 where the error crept in.
Discussion: https://postgr.es/m/17409-52871dda8b5741cb@postgresql.org
M src/backend/utils/cache/lsyscache.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
doc: Add ALTER/DROP ROUTINE to the event trigger matrix
commit : 8f091572873c072ff844b0e2e18088ec51e4b03f
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 9 Mar 2022 14:59:18 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 9 Mar 2022 14:59:18 +0900
ALTER ROUTINE triggers the events ddl_command_start and ddl_command_end,
and DROP ROUTINE triggers sql_drop, ddl_command_start and
ddl_command_end, but this was not mention on the matrix table.
Reported-by: Leslie Lemaire
Discussion: https://postgr.es/m/164647533363.646.5802968483136493025@wrigleys.postgresql.org
Backpatch-through: 11
M doc/src/sgml/event-trigger.sgml
Introduce PG_TEST_TIMEOUT_DEFAULT for TAP suite non-elapsing timeouts.
commit : f60bb3e0a9044058a3df1d75bb900e77e51ad4ed
author : Noah Misch <noah@leadboat.com>
date : Fri, 4 Mar 2022 18:53:13 -0800
committer: Noah Misch <noah@leadboat.com>
date : Fri, 4 Mar 2022 18:53:13 -0800
Slow hosts may avoid load-induced, spurious failures by setting
environment variable PG_TEST_TIMEOUT_DEFAULT to some number of seconds
greater than 180. Developers may see faster failures by setting that
environment variable to some lesser number of seconds. In tests, write
$PostgreSQL::Test::Utils::timeout_default wherever the convention has
been to write 180. This change raises the default for some briefer
timeouts. Back-patch to v10 (all supported versions).
Discussion: https://postgr.es/m/20220218052842.GA3627003@rfd.leadboat.com
M contrib/amcheck/t/002_cic.pl
M contrib/amcheck/t/003_cic_2pc.pl
M src/bin/pg_ctl/t/004_logrotate.pl
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/psql/t/010_tab_completion.pl
M src/bin/scripts/t/080_pg_isready.pl
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
M src/test/recovery/t/003_recovery_targets.pl
M src/test/recovery/t/006_logical_decoding.pl
M src/test/recovery/t/010_logical_decoding_timelines.pl
M src/test/recovery/t/013_crash_restart.pl
M src/test/recovery/t/017_shm.pl
M src/test/recovery/t/019_replslot_limit.pl
M src/test/recovery/t/021_row_visibility.pl
M src/test/recovery/t/022_crash_temp_files.pl
M src/test/recovery/t/024_archive_recovery.pl
M src/test/subscription/t/015_stream.pl
Fix pg_regress to print the correct postmaster address on Windows.
commit : a008c03dd722f3a9ea44b13ca2cc6066fd90ddc1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Mar 2022 13:23:58 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Mar 2022 13:23:58 -0500
pg_regress reported "Unix socket" as the default location whenever
HAVE_UNIX_SOCKETS is defined. However, that's not been accurate
on Windows since 8f3ec75de. Update this logic to match what libpq
actually does now.
This is just cosmetic, but still it's potentially misleading.
Back-patch to v13 where 8f3ec75de came in.
Discussion: https://postgr.es/m/3894060.1646415641@sss.pgh.pa.us
M src/interfaces/libpq/fe-connect.c
M src/test/regress/pg_regress.c
Fix bogus casting in BlockIdGetBlockNumber().
commit : 5c9d17e94c5cc0d24c1f0dbfe030000238d65afb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Mar 2022 19:03:17 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Mar 2022 19:03:17 -0500
This macro cast the result to BlockNumber after shifting, not before,
which is the wrong thing. Per the C spec, the uint16 fields would
promote to int not unsigned int, so that (for 32-bit int) the shift
potentially shifts a nonzero bit into the sign position. I doubt
there are any production systems where this would actually end with
the wrong answer, but it is undefined behavior per the C spec, and
clang's -fsanitize=undefined option reputedly warns about it on some
platforms. (I can't reproduce that right now, but the code is
undeniably wrong per spec.) It's easy to fix by casting to
BlockNumber (uint32) in the proper places.
It's been wrong for ages, so back-patch to all supported branches.
Report and patch by Zhihong Yu (cosmetic tweaking by me)
Discussion: https://postgr.es/m/CALNJ-vT9r0DSsAOw9OXVJFxLENoVS_68kJ5x0p44atoYH+H4dg@mail.gmail.com
M src/include/storage/block.h
Clean up assorted failures under clang's -fsanitize=undefined checks.
commit : b0bc196e52e606fe0116fb63da20f57fb577745b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Mar 2022 18:13:24 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Mar 2022 18:13:24 -0500
Most of these are cases where we could call memcpy() or other libc
functions with a NULL pointer and a zero count, which is forbidden
by POSIX even though every production version of libc allows it.
We've fixed such things before in a piecemeal way, but apparently
never made an effort to try to get them all. I don't claim that
this patch does so either, but it gets every failure I observe in
check-world, using clang 12.0.1 on current RHEL8.
numeric.c has a different issue that the sanitizer doesn't like:
"ln(-1.0)" will compute log10(0) and then try to assign the
resulting -Inf to an integer variable. We don't actually use the
result in such a case, so there's no live bug.
Back-patch to all supported branches, with the idea that we might
start running a buildfarm member that tests this case. This includes
back-patching c1132aae3 (Check the size in COPY_POINTER_FIELD),
which previously silenced some of these issues in copyfuncs.c.
Discussion: https://postgr.es/m/CALNJ-vT9r0DSsAOw9OXVJFxLENoVS_68kJ5x0p44atoYH+H4dg@mail.gmail.com
M contrib/pgcrypto/px.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_visibility.c
M src/backend/access/transam/clog.c
M src/backend/access/transam/xact.c
M src/backend/nodes/copyfuncs.c
M src/backend/storage/ipc/shm_mq.c
M src/backend/utils/adt/numeric.c
M src/backend/utils/time/snapmgr.c
M src/fe_utils/print.c
Allow root-owned SSL private keys in libpq, not only the backend.
commit : 2a1f84636dc335a3edf53a8361ae44bb2ae00093
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Mar 2022 11:57:02 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Mar 2022 11:57:02 -0500
This change makes libpq apply the same private-key-file ownership
and permissions checks that we have used in the backend since commit
9a83564c5. Namely, that the private key can be owned by either the
current user or root (with different file permissions allowed in the
two cases). This allows system-wide management of key files, which
is just as sensible on the client side as the server, particularly
when the client is itself some application daemon.
Sync the comments about this between libpq and the backend, too.
Back-patch of a59c79564 and 50f03473e into all supported branches.
David Steele
Discussion: https://postgr.es/m/f4b7bc55-97ac-9e69-7398-335e212f7743@pgmasters.net
M doc/src/sgml/libpq.sgml
M src/backend/libpq/be-secure-common.c
M src/interfaces/libpq/fe-secure-openssl.c
Disallow execution of SPI functions during plperl function compilation.
commit : ac910bb2328ff107359d009aaf821c01559cef19
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 25 Feb 2022 17:40:21 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 25 Feb 2022 17:40:21 -0500
Perl can be convinced to execute user-defined code during compilation
of a plperl function (or at least a plperlu function). That's not
such a big problem as long as the activity is confined within the
Perl interpreter, and it's not clear we could do anything about that
anyway. However, if such code tries to use plperl's SPI functions,
we have a bigger problem. In the first place, those functions are
likely to crash because current_call_data->prodesc isn't set up yet.
In the second place, because it isn't set up, we lack critical info
such as whether the function is supposed to be read-only. And in
the third place, this path allows code execution during function
validation, which is strongly discouraged because of the potential
for security exploits. Hence, reject execution of the SPI functions
until compilation is finished.
While here, add check_spi_usage_allowed() calls to various functions
that hadn't gotten the memo about checking that. I think that perhaps
plperl_sv_to_literal may have been intentionally omitted on the grounds
that it was safe at the time; but if so, the addition of transforms
functionality changed that. The others are more recently added and
seem to be flat-out oversights.
Per report from Mark Murawski. Back-patch to all supported branches.
Discussion: https://postgr.es/m/9acdf918-7fff-4f40-f750-2ffa84f083d2@intellasoft.net
M src/pl/plperl/plperl.c
pg_waldump: Fix error message for WAL files smaller than XLOG_BLCKSZ.
commit : 9ff7fd9063ce918e458a643b53eda1cf95f7d115
author : Andres Freund <andres@anarazel.de>
date : Fri, 25 Feb 2022 10:30:05 -0800
committer: Andres Freund <andres@anarazel.de>
date : Fri, 25 Feb 2022 10:30:05 -0800
When opening a WAL file smaller than XLOG_BLCKSZ (e.g. 0 bytes long) while
determining the wal_segment_size, pg_waldump checked errno, despite errno not
being set by the short read. Resulting in a bogus error message.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/20220214.181847.775024684568733277.horikyota.ntt@gmail.com
Backpatch: 11-, the bug was introducedin fc49e24fa
M src/bin/pg_waldump/pg_waldump.c
Fix data loss on crash after sorted GiST index build.
commit : 7d80e93fb1f1059ba3691b2584bb16992d7d734e
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 24 Feb 2022 14:34:06 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 24 Feb 2022 14:34:06 +0200
If a checkpoint happens during the index build, and the system crashes
after the checkpoint and the index build have finished, the data written
to the index before the checkpoint started could be lost. The checkpoint
won't have fsync'd it, and it won't be replayed at crash recovery either.
Fix by calling smgrimmedsync() after the index build, just like in B-tree
index build.
Backpatch to v14 where the sorted GiST index build was introduced.
Reported-by: Melanie Plageman
Discussion: https://www.postgresql.org/message-id/CAAKRu_ZJJynimxKj5xYBSziL62-iEtPE+fx-B=JzR=jUtP92mw@mail.gmail.com
M src/backend/access/gist/gistbuild.c
Re-allow underscore as first character of custom GUC names.
commit : dd7c0597918a6922a43a1f2e7085e6276a6216ec
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Feb 2022 11:10:46 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Feb 2022 11:10:46 -0500
Commit 3db826bd5 intended that valid_custom_variable_name's
rules for valid identifiers match those of scan.l. However,
I (tgl) had some kind of brain fade and put "_" in the wrong
list.
Fix by Japin Li, per bug #17415 from Daniel Polski.
Discussion: https://postgr.es/m/17415-ebdb683d7e09a51c@postgresql.org
M src/backend/utils/misc/guc.c
Add compute_query_id = regress
commit : 627c79a1e87d9ec4a8a8a0c5be8564ba74e221ea
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 22 Feb 2022 10:23:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 22 Feb 2022 10:23:49 +0900
"regress" is a new mode added to compute_query_id aimed at facilitating
regression testing when a module computing query IDs is loaded into the
backend, like pg_stat_statements. It works the same way as "auto",
meaning that query IDs are computed if a module enables it, except that
query IDs are hidden in EXPLAIN outputs to ensure regression output
stability.
Like any GUCs of the kind (force_parallel_mode, etc.), this new
configuration can be added to an instance's postgresql.conf, or just
passed down with PGOPTIONS at command level. compute_query_id uses an
enum for its set of option values, meaning that this addition ensures
ABI compatibility.
Using this new configuration mode allows installcheck-world to pass when
running the tests on an instance with pg_stat_statements enabled,
stabilizing the test output while checking the paths doing query ID
computations.
Reported-by: Anton Melnikov
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/1634283396.372373993@f75.i.mail.ru
Discussion: https://postgr.es/m/YgHlxgc/OimuPYhH@paquier.xyz
Backpatch-through: 14
M doc/src/sgml/config.sgml
M src/backend/commands/explain.c
M src/backend/utils/misc/guc.c
M src/include/utils/queryjumble.h
Fix temporary object cleanup failing due to toast access without snapshot.
commit : 7bbfe5994165aef9e903b1919d625fe87d4f146e
author : Andres Freund <andres@anarazel.de>
date : Sat, 19 Feb 2022 12:27:20 -0800
committer: Andres Freund <andres@anarazel.de>
date : Sat, 19 Feb 2022 12:27:20 -0800
When cleaning up temporary objects during process exit the cleanup could fail
with:
FATAL: cannot fetch toast data without an active snapshot
The bug is caused by RemoveTempRelationsCallback() not setting up a
snapshot. If an object with toasted catalog data needs to be cleaned up,
init_toast_snapshot() could fail with the above error.
Most of the time however the the problem is masked due to cached catalog
snapshots being returned by GetOldestSnapshot(). But dropping an object can
cause catalog invalidations to be emitted. If no further catalog accesses are
necessary between the invalidation processing and the next toast datum
deletion, the bug becomes visible.
It's easy to miss this bug because it typically happens after clients
disconnect and the FATAL error just ends up in the log.
Luckily temporary table cleanup at the next use of the same temporary schema
or during DISCARD ALL does not have the same problem.
Fix the bug by pushing a snapshot in RemoveTempRelationsCallback(). Also add
isolation tests for temporary object cleanup, including objects with toasted
catalog data.
A future HEAD only commit will add more assertions.
Reported-By: Miles Delahunty
Author: Andres Freund
Discussion: https://postgr.es/m/CAOFAq3BU5Mf2TTvu8D9n_ZOoFAeQswuzk7yziAb7xuw_qyw5gw@mail.gmail.com
Backpatch: 10-
M src/backend/catalog/namespace.c
A src/test/isolation/expected/temp-schema-cleanup.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/temp-schema-cleanup.spec
Remove most msys special processing in TAP tests
commit : 8b5cd373ba033977ef0100f52871dd6b38b7239e
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 20 Feb 2022 11:48:45 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 20 Feb 2022 11:48:45 -0500
Following migration of Windows buildfarm members running TAP tests to
use of ucrt64 perl for those tests, special processing for msys perl is
no longer necessary and so is removed.
Backpatch to release 10
Discussion: https://postgr.es/m/c65a8781-77ac-ea95-d185-6db291e1baeb@dunslane.net
M src/bin/pg_ctl/t/001_start_stop.pl
M src/bin/pg_rewind/t/RewindTest.pm
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
M src/test/recovery/t/021_row_visibility.pl
M src/test/recovery/t/cp_history_files
Remove PostgreSQL::Test::Utils::perl2host completely
commit : 652ff988fbf635c04618803fad06b863bf28bdb9
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 20 Feb 2022 08:55:06 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 20 Feb 2022 08:55:06 -0500
Commit f1ac4a74de disabled this processing, and as nothing has broken (as
expected) here we proceed to remove the routine and adjust all the call
sites.
Backpatch to release 10
Discussion: https://postgr.es/m/0ba775a2-8aa0-0d56-d780-69427cf6f33d@dunslane.net
Discussion: https://postgr.es/m/20220125023609.5ohu3nslxgoygihl@alap3.anarazel.de
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_checksums/t/002_actions.pl
M src/bin/pg_verifybackup/t/003_corruption.pl
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/scripts/t/090_reindexdb.pl
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
M src/test/recovery/t/014_unlogged_reinit.pl
M src/test/recovery/t/017_shm.pl
M src/test/recovery/t/018_wal_optimize.pl
M src/test/recovery/t/025_stuck_on_old_timeline.pl
Suppress warning about stack_base_ptr with late-model GCC.
commit : 2e30d77a196810f7e9d57709a942b1826fd1bb43
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Feb 2022 22:45:34 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Feb 2022 22:45:34 -0500
GCC 12 complains that set_stack_base is storing the address of
a local variable in a long-lived pointer. This is an entirely
reasonable warning (indeed, it just helped us find a bug);
but that behavior is intentional here. We can work around it
by using __builtin_frame_address(0) instead of a specific local
variable; that produces an address a dozen or so bytes different,
in my testing, but we don't care about such a small difference.
Maybe someday a compiler lacking that function will start to issue
a similar warning, but we'll worry about that when it happens.
Patch by me, per a suggestion from Andres Freund. Back-patch to
v12, which is as far back as the patch will go without some pain.
(Recently-established project policy would permit a back-patch as
far as 9.2, but I'm disinclined to expend the work until GCC 12
is much more widespread.)
Discussion: https://postgr.es/m/3773792.1645141467@sss.pgh.pa.us
M config/c-compiler.m4
M configure
M configure.ac
M src/backend/postmaster/postmaster.c
M src/backend/tcop/postgres.c
M src/backend/utils/init/miscinit.c
M src/include/pg_config.h.in
M src/tools/msvc/Solution.pm
Doc: Update documentation for modifying postgres_fdw foreign tables.
commit : a9e186da81d9965789b1a7ab869329eabe3595c3
author : Etsuro Fujita <efujita@postgresql.org>
date : Wed, 16 Feb 2022 15:15:02 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Wed, 16 Feb 2022 15:15:02 +0900
Document that they can be modified using COPY as well.
Back-patch to v11 where commit 3d956d956 added support for COPY in
postgres_fdw.
M doc/src/sgml/postgres-fdw.sgml
WAL log unchanged toasted replica identity key attributes.
commit : 04645bbcae72c71e5bf28a7a0c96efa851e13ddb
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 14 Feb 2022 08:07:46 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 14 Feb 2022 08:07:46 +0530
Currently, during UPDATE, the unchanged replica identity key attributes
are not logged separately because they are getting logged as part of the
new tuple. But if they are stored externally then the untoasted values are
not getting logged as part of the new tuple and logical replication won't
be able to replicate such UPDATEs. So we need to log such attributes as
part of the old_key_tuple during UPDATE.
Reported-by: Haiying Tang
Author: Dilip Kumar and Amit Kapila
Reviewed-by: Alvaro Herrera, Haiying Tang, Andres Freund
Backpatch-through: 10
Discussion: https://postgr.es/m/OS0PR01MB611342D0A92D4F4BF26C0F47FB229@OS0PR01MB6113.jpnprd01.prod.outlook.com
M contrib/test_decoding/expected/toast.out
M doc/src/sgml/ref/alter_table.sgml
M src/backend/access/heap/heapam.c
Fix memory leak in IndexScan node with reordering
commit : c76665edce3d4db4e509723e91742569e9f46817
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 14 Feb 2022 03:26:55 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 14 Feb 2022 03:26:55 +0300
Fix ExecReScanIndexScan() to free the referenced tuples while emptying the
priority queue. Backpatch to all supported versions.
Discussion: https://postgr.es/m/CAHqSB9gECMENBQmpbv5rvmT3HTaORmMK3Ukg73DsX5H7EJV7jw%40mail.gmail.com
Author: Aliaksandr Kalenik
Reviewed-by: Tom Lane, Alexander Korotkov
Backpatch-through: 10
M src/backend/executor/nodeIndexscan.c
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Fix thinko in PQisBusy().
commit : ae27b1acc43d58badad356a4da8fc0c72d46d7fb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Feb 2022 13:23:20 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Feb 2022 13:23:20 -0500
In commit 1f39a1c06 I made PQisBusy consider conn->write_failed, but
that is now looking like complete brain fade. In the first place, the
logic is quite wrong: it ought to be like "and not" rather than "or".
This meant that once we'd gotten into a write_failed state, PQisBusy
would always return true, probably causing the calling application to
iterate its loop until PQconsumeInput returns a hard failure thanks
to connection loss. That's not what we want: the intended behavior
is to return an error PGresult, which the application probably has
much cleaner support for.
But in the second place, checking write_failed here seems like the
wrong thing anyway. The idea of the write_failed mechanism is to
postpone handling of a write failure until we've read all we can from
the server; so that flag should not interfere with input-processing
behavior. (Compare 7247e243a.) What we *should* check for is
status = CONNECTION_BAD, ie, socket already closed. (Most places that
close the socket don't touch asyncStatus, but they do reset status.)
This primarily ensures that if PQisBusy() returns true then there is
an open socket, which is assumed by several call sites in our own
code, and probably other applications too.
While at it, fix a nearby thinko in libpq's my_sock_write: we should
only consult errno for res < 0, not res == 0. This is harmless since
pqsecure_raw_write would force errno to zero in such a case, but it
still could confuse readers.
Noted by Andres Freund. Backpatch to v12 where 1f39a1c06 came in.
Discussion: https://postgr.es/m/20220211011025.ek7exh6owpzjyudn@alap3.anarazel.de
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-secure-openssl.c
Don't use_physical_tlist for an IOS with non-returnable columns.
commit : 277e744ae176ef8e411a9e6c5b2e649322ee6bdd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 11 Feb 2022 15:23:52 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 11 Feb 2022 15:23:52 -0500
createplan.c tries to save a runtime projection step by specifying
a scan plan node's output as being exactly the table's columns, or
index's columns in the case of an index-only scan, if there is not a
reason to do otherwise. This logic did not previously pay attention
to whether an index's columns are returnable. That worked, sort of
accidentally, until commit 9a3ddeb51 taught setrefs.c to reject plans
that try to read a non-returnable column. I have no desire to loosen
setrefs.c's new check, so instead adjust use_physical_tlist() to not
try to optimize this way when there are non-returnable column(s).
Per report from Ryan Kelly. Like the previous patch, back-patch
to all supported branches.
Discussion: https://postgr.es/m/CAHUie24ddN+pDNw7fkhNrjrwAX=fXXfGZZEHhRuofV_N_ftaSg@mail.gmail.com
M src/backend/optimizer/plan/createplan.c
M src/test/regress/expected/gist.out
M src/test/regress/sql/gist.sql
Make pg_ctl stop/restart/promote recheck postmaster aliveness.
commit : 1e8c5cf7c6c2668da69b77cea91f74ca75e67c56
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Feb 2022 16:49:39 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Feb 2022 16:49:39 -0500
"pg_ctl stop/restart" checked that the postmaster PID is valid just
once, as a side-effect of sending the stop signal, and then would
wait-till-timeout for the postmaster.pid file to go away. This
neglects the case wherein the postmaster dies uncleanly after we
signal it. Similarly, once "pg_ctl promote" has sent the signal,
it'd wait for the corresponding on-disk state change to occur
even if the postmaster dies.
I'm not sure how we've managed not to notice this problem, but it
seems to explain slow execution of the 017_shm.pl test script on AIX
since commit 4fdbf9af5, which added a speculative "pg_ctl stop" with
the idea of making real sure that the postmaster isn't there. In the
test steps that kill-9 and then restart the postmaster, it's possible
to get past the initial signal attempt before kill() stops working
for the doomed postmaster. If that happens, pg_ctl waited till
PGCTLTIMEOUT before giving up ... and the buildfarm's AIX members
have that set very high.
To fix, include a "kill(pid, 0)" test (similar to what
postmaster_is_alive uses) in these wait loops, so that we'll
give up immediately if the postmaster PID disappears.
While here, I chose to refactor those loops out of where they were.
do_stop() and do_restart() can perfectly well share one copy of the
wait-for-stop loop, and it seems desirable to put a similar function
beside that for wait-for-promote.
Back-patch to all supported versions, since pg_ctl's wait logic
is substantially identical in all, and we're seeing the slow test
behavior in all branches.
Discussion: https://postgr.es/m/20220210023537.GA3222837@rfd.leadboat.com
M src/bin/pg_ctl/pg_ctl.c
Use gendef instead of pexports for building windows .def files
commit : 92f60f536eceabcefb2528d4bf5366fef4a1536d
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 10 Feb 2022 13:44:05 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 10 Feb 2022 13:44:05 -0500
Modern msys systems lack pexports but have gendef instead, so use that.
Discussion: https://postgr.es/m/3ccde7a9-e4f9-e194-30e0-0936e6ad68ba@dunslane.net
Backpatch to release 9.4 to enable building with perl on older branches.
Before that pexports is not used for plperl.
M src/pl/plperl/GNUmakefile
M src/pl/plpython/Makefile
M src/pl/tcl/Makefile
Make timeout.c more robust against missed timer interrupts.
commit : 2e211c16612a127bea861e416bdc22ff39c54866
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Feb 2022 11:52:20 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Feb 2022 11:52:20 -0500
Commit 09cf1d522 taught schedule_alarm() to not do anything if
the next requested event is after when we expect the next interrupt
to fire. However, if somehow an interrupt gets lost, we'll continue
to not do anything indefinitely, even after the "next interrupt" time
is obviously in the past. Thus, one missed interrupt can break
timeout scheduling for the life of the session. Michael Harris
reported a scenario where a bug in a user-defined function caused this
to happen, so you don't even need to assume kernel bugs exist to think
this is worth fixing. We can make things more robust at little cost
by detecting the case where signal_due_at is before "now" and forcing
a new setitimer call to occur. This isn't a completely bulletproof
fix of course; but in our typical usage pattern where we frequently set
timeouts and clear them before they are reached, the interrupt will
get re-enabled after at most one timeout interval, which with a little
luck will be before we really need it.
While here, let's mark signal_due_at as volatile, since the signal
handler can both examine and set it. I'm not sure there's any
actual risk given that signal_pending is already volatile, but
it's surely questionable.
Backpatch to v14 where this logic came in.
Michael Harris and Tom Lane
Discussion: https://postgr.es/m/CADofcAWbMrvgwSMqO4iG_iD3E2v8ZUrC-_crB41my=VMM02-CA@mail.gmail.com
M src/backend/utils/misc/timeout.c
Set SNI ClientHello extension to localhost in tests
commit : 5f00ef065ec8f086efeec93083fe250f81a3f585
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 10 Feb 2022 14:23:36 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 10 Feb 2022 14:23:36 +0100
The connection strings in the SSL client tests were using the host
set up from Cluster.pm which is a temporary pathname. When SNI is
enabled we pass the host to OpenSSL in order to set the server name
indication ClientHello extension via SSL_set_tlsext_host_name.
OpenSSL doesn't validate the hostname apart from checking the max
length, but LibreSSL checks for RFC 5890 conformance which results
in errors during testing as the pathname from Cluster.pm is not a
valid hostname.
Fix by setting the host explicitly to localhost, as that's closer
to the intent of the test.
Backpatch through 14 where SNI support came in.
Reported-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/17391-304f81bcf724b58b@postgresql.org
Backpatch-through: 14
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
Use Test::Builder::todo_start(), replacing $::TODO.
commit : 1a83297d222487070386b55b032657345d7ec64a
author : Noah Misch <noah@leadboat.com>
date : Wed, 9 Feb 2022 18:16:59 -0800
committer: Noah Misch <noah@leadboat.com>
date : Wed, 9 Feb 2022 18:16:59 -0800
Some pre-2017 Test::More versions need perfect $Test::Builder::Level
maintenance to find the variable. Buildfarm member snapper reported an
overall failure that the file intended to hide via the TODO construct.
That trouble was reachable in v11 and v10. For later branches, this
serves as defense in depth. Back-patch to v10 (all supported versions).
Discussion: https://postgr.es/m/20220202055556.GB2745933@rfd.leadboat.com
M contrib/amcheck/t/003_cic_2pc.pl
Test honestly for <sys/signalfd.h>.
commit : c23461a22a1ead0a34dc209c2918ec4ccaf61475
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Feb 2022 14:24:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Feb 2022 14:24:55 -0500
Commit 6a2a70a02 supposed that any platform having <sys/epoll.h>
would also have <sys/signalfd.h>. It turns out there are still a
few people using platforms where that's not so, so we'd better make
a separate configure probe for it. But since it took this long to
notice, I'm content with the decision to not have a separate code
path for epoll-only machines; we'll just fall back to using poll()
for these stragglers.
Per gripe from Gabriela Serventi. Back-patch to v14 where this
code came in.
Discussion: https://postgr.es/m/CAHOHWE-JjJDfcYuLAAEO7Jk07atFAU47z8TzHzg71gbC0aMy=g@mail.gmail.com
M configure
M configure.ac
M src/backend/storage/ipc/latch.c
M src/include/pg_config.h.in
M src/tools/msvc/Solution.pm
Remove ppport.h's broken re-implementation of eval_pv().
commit : e327291e4a96571c85b6a9b140e1a8116fb47ec6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Feb 2022 19:25:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Feb 2022 19:25:56 -0500
Recent versions of Devel::PPPort try to redefine eval_pv() to
dodge a bug in pre-5.31 Perl versions. Unfortunately the redefinition
fails on compilers that don't support statements nested within
expressions. However, we aren't actually interested in this bug fix,
since we always call eval_pv() with croak_on_error = FALSE.
So, until there's an upstream fix for this breakage, just comment
out the macro to revert to the older behavior.
Per report from Wei Sun, as well as previous buildfarm failure
on pademelon (which I'd unfortunately not looked at carefully
enough to understand the cause). Back-patch to all supported
versions, since we're using the same ppport.h in all.
Discussion: https://postgr.es/m/tencent_2EFCC8BA0107B6EC0F97179E019A8A43C806@qq.com
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=pademelon&dt=2022-02-02%2001%3A22%3A58
M src/pl/plperl/ppport.h
Stamp 14.2.
commit : 78b1ddc753b3e5e3f3e9eb0e7519b48146018394
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Feb 2022 16:15:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Feb 2022 16:15:36 -0500
M configure
M configure.ac
Translation updates
commit : 3c879e61d9ad5968e88e12620e270808b6cb76f5
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 7 Feb 2022 13:31:57 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 7 Feb 2022 13:31:57 +0100
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 063c497a909612d444c7c7188482db9aef86200f
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/backend/po/ru.po
M src/backend/po/sv.po
M src/bin/initdb/po/el.po
M src/bin/initdb/po/sv.po
M src/bin/pg_amcheck/po/el.po
M src/bin/pg_amcheck/po/ru.po
M src/bin/pg_basebackup/nls.mk
M src/bin/pg_basebackup/po/de.po
A src/bin/pg_basebackup/po/el.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_basebackup/po/fr.po
M src/bin/pg_basebackup/po/ru.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_dump/po/el.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/sv.po
M src/bin/pg_resetwal/po/el.po
M src/bin/pg_resetwal/po/sv.po
M src/bin/pg_rewind/po/sv.po
M src/bin/pg_upgrade/po/ru.po
M src/bin/pg_upgrade/po/sv.po
M src/bin/psql/po/de.po
M src/bin/psql/po/el.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/sv.po
M src/bin/scripts/po/el.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/ru.po
M src/bin/scripts/po/sv.po
M src/interfaces/ecpg/preproc/po/el.po
M src/interfaces/libpq/po/de.po
M src/interfaces/libpq/po/es.po
M src/interfaces/libpq/po/fr.po
M src/interfaces/libpq/po/ru.po
M src/interfaces/libpq/po/sv.po
M src/pl/plperl/po/ru.po
M src/pl/plpgsql/src/po/el.po
Release notes for 14.2, 13.6, 12.10, 11.15, 10.20.
commit : 1d9ccc0a6d3c1b3d5c52e6e427a9e1cb824b0a90
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Feb 2022 14:24:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Feb 2022 14:24:55 -0500
M doc/src/sgml/release-14.sgml
Doc: be clearer that foreign-table partitions need user-added constraints.
commit : d0cd7b77265cf39e0e8984ba9d5bec0c155dccdb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Feb 2022 12:55:44 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Feb 2022 12:55:44 -0500
A very well-informed user might deduce this from what we said already,
but I'd bet against it. Lay it out explicitly.
While here, rewrite the comment about tuple routing to be more
intelligible to an average SQL user.
Per bug #17395 from Alexander Lakhin. Back-patch to v11. (The text
in this area is different in v10 and I'm not sufficiently excited
about this point to adapt the patch.)
Discussion: https://postgr.es/m/17395-8c326292078d1a57@postgresql.org
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/create_foreign_table.sgml
Test, don't just Assert, that mergejoin's inputs are in order.
commit : d13a838e1c19df5358d250b8d469544d7ab3dfda
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Feb 2022 11:59:30 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Feb 2022 11:59:30 -0500
There are two Asserts in nodeMergejoin.c that are reachable if
the input data is not in the expected order. This seems way too
fragile. Alexander Lakhin reported a case where the assertions
could be triggered with misconfigured foreign-table partitions,
and bitter experience with unstable operating system collation
definitions suggests another easy route to hitting them. Neither
Assert is in a place where we can't afford one more test-and-branch,
so replace 'em with plain test-and-elog logic.
Per bug #17395. While the reported symptom is relatively recent,
collation changes could happen anytime, so back-patch to all
supported branches.
Discussion: https://postgr.es/m/17395-8c326292078d1a57@postgresql.org
M src/backend/executor/nodeMergejoin.c
First-draft release notes for 14.2.
commit : ab22eea83169c8d0eb15050ce61cbe3d7dae4de6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Feb 2022 14:55:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Feb 2022 14:55:00 -0500
As usual, the release notes for older branches will be made by cutting
these down, but put them up for community review first.
M doc/src/sgml/release-14.sgml
Fix compiler warning in non-assert builds, introduced in f862d57057f.
commit : 2a3958e4d9b9b1e82adefa11f3e69ea5c16efb45
author : Andres Freund <andres@anarazel.de>
date : Thu, 3 Feb 2022 10:44:26 -0800
committer: Andres Freund <andres@anarazel.de>
date : Thu, 3 Feb 2022 10:44:26 -0800
Discussion: https://postgr.es/m/20220203183655.ralgkh54sdcgysmn@alap3.anarazel.de
Backpatch: 14-, like f862d57057f
M src/backend/executor/nodeForeignscan.c
Further fix for EvalPlanQual with mix of local and foreign partitions.
commit : 7b0cec2fa05a123ed96163d7ac8485384570b3e0
author : Etsuro Fujita <efujita@postgresql.org>
date : Thu, 3 Feb 2022 15:15:01 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Thu, 3 Feb 2022 15:15:01 +0900
We assume that direct-modify ForeignScan nodes cannot be re-evaluated
during EvalPlanQual processing, but the rework for inherited
UPDATE/DELETE in commit 86dc90056 changed things, without considering
that, so that such ForeignScan nodes get called as part of the
EvalPlanQual subtree during EvalPlanQual processing in the case of an
inherited UPDATE/DELETE where the inheritance set contains foreign
target relations. To avoid re-evaluating such ForeignScan nodes during
EvalPlanQual processing, commit c3928b467 modified nodeForeignscan.c,
but the assumption made there that ExecForeignScan() should never be
called for such ForeignScan nodes during EvalPlanQual processing turned
out to be wrong in some cases, leading to a segmentation fault or a
"cannot re-evaluate a Foreign Update or Delete during EvalPlanQual"
error.
Fix by modifying nodeForeignscan.c further to avoid re-evaluating such
ForeignScan nodes even in ExecForeignScan()/ExecReScanForeignScan()
during EvalPlanQual processing. Since this makes non-reachable the
test-and-elog added to ForeignNext() by commit c3928b467 that produced
the aforesaid error, convert the test-and-elog to an Assert.
Per bug #17355 from Alexander Lakhin. Back-patch to v14 where both
commits came in.
Patch by me, reviewed and tested by Alexander Lakhin and Amit Langote.
Discussion: https://postgr.es/m/17355-de8e362eb7001a96@postgresql.org
M src/backend/executor/nodeForeignscan.c
doc: clarify syntax notation, particularly parentheses
commit : 25560761289e5dbe21514c9ef2cf7272d9181488
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 2 Feb 2022 21:53:51 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 2 Feb 2022 21:53:51 -0500
Also move TCL syntax to the PL/tcl section.
Reported-by: davs2rt@gmail.com
Discussion: https://postgr.es/m/164308146320.12460.3590769444508751574@wrigleys.postgresql.org
Backpatch-through: 10
M doc/src/sgml/notation.sgml
M doc/src/sgml/pltcl.sgml
doc: Fix mistake in PL/Python documentation
commit : ee57467cb6b2ce1e25d367cbd64cc60ae9ba0eb3
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 2 Feb 2022 09:14:26 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 2 Feb 2022 09:14:26 +0100
Small thinko introduced by 94aceed317730953476bec490ce0148b2af3c383
Reported-by: nassehk@gmail.com
M doc/src/sgml/plpython.sgml
Replace use of deprecated Python module distutils.sysconfig, take 2.
commit : 803f0b1739065b2d1d46c4a72c99b46cdd354f60
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Feb 2022 19:03:41 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Feb 2022 19:03:41 -0500
With Python 3.10, configure spits out warnings about the module
distutils.sysconfig being deprecated and scheduled for removal in
Python 3.12. Change the uses in configure to use the module sysconfig
instead. The logic stays largely the same, although we have to
rely on INCLUDEPY instead of the deprecated get_python_inc function.
Note that sysconfig exists since Python 2.7, so this moves the
minimum required version up from Python 2.6 (or 2.4, before v13).
Also, sysconfig didn't exist in Python 3.1, so the minimum 3.x
version is now 3.2.
Back-patch of commit bd233bdd8 into all supported branches.
In v10, this also includes back-patching v11's beff4bb9c, primarily
because this opinion is clearly out-of-date:
While at it, get rid of the code's assumption that both the major and
minor numbers contain exactly one digit. That will foreseeably be
broken by Python 3.10 in perhaps four or five years. That's far enough
out that we probably don't need to back-patch this.
Peter Eisentraut, Tom Lane, Andres Freund
Discussion: https://postgr.es/m/c74add3c-09c4-a9dd-1a03-a846e5b2fc52@enterprisedb.com
M config/python.m4
M configure
M doc/src/sgml/installation.sgml
Revert "plperl: Fix breakage of c89f409749c in back branches."
commit : 00fdfde6dc726dbbee7e7e3e25a6406e2c76c4f6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 31 Jan 2022 15:07:39 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 31 Jan 2022 15:07:39 -0500
This reverts commits d81cac47a et al. We shouldn't need that
hack after the preceding commits.
Discussion: https://postgr.es/m/20220131015130.shn6wr2fzuymerf6@alap3.anarazel.de
M src/pl/plperl/plperl.c
plperl: update ppport.h to Perl 5.34.0.
commit : 2d44912cf79f1f174a83f587e1c4328315b8a0fa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 31 Jan 2022 15:01:05 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 31 Jan 2022 15:01:05 -0500
Also apply the changes suggested by running
perl ppport.h --compat-version=5.8.0
And remove some no-longer-required NEED_foo declarations.
Dagfinn Ilmari Mannsåker
Back-patch of commit 05798c9f7 into all supported branches.
At the time we thought this update was mostly cosmetic, but the
lack of it has caused trouble, while the patch itself hasn't.
Discussion: https://postgr.es/m/87y278s6iq.fsf@wibble.ilmari.org
Discussion: https://postgr.es/m/20220131015130.shn6wr2fzuymerf6@alap3.anarazel.de
M src/pl/plperl/plperl.c
M src/pl/plperl/plperl.h
M src/pl/plperl/ppport.h
plperl: Fix breakage of c89f409749c in back branches.
commit : d81cac47a829c3e2c729ef98acaf132adb7ff3ae
author : Andres Freund <andres@anarazel.de>
date : Sun, 30 Jan 2022 17:53:53 -0800
committer: Andres Freund <andres@anarazel.de>
date : Sun, 30 Jan 2022 17:53:53 -0800
ppport.h was only updated in 05798c9f7f0 (master). Unfortunately my commit
c89f409749c uses PERL_VERSION_LT which came in with that update. Breaking most
buildfarm animals.
I should have noticed that...
We might want to backpatch the ppport update instead, but for now lets get the
buildfarm green again.
Discussion: https://postgr.es/m/20220131015130.shn6wr2fzuymerf6@alap3.anarazel.de
Backpatch: 10-14, master doesn't need it
M src/pl/plperl/plperl.c
plperl: windows: Use Perl_setlocale on 5.28+, fixing compile failure.
commit : 8484e38126d7522069e44b81cd41ef53f700b2e1
author : Andres Freund <andres@anarazel.de>
date : Sun, 30 Jan 2022 14:29:04 -0800
committer: Andres Freund <andres@anarazel.de>
date : Sun, 30 Jan 2022 14:29:04 -0800
For older versions we need our own copy of perl's setlocale(), because it was
not exposed (why we need the setlocale in the first place is explained in
plperl_init_interp) . The copy stopped working in 5.28, as some of the used
macros are not public anymore. But Perl_setlocale is available in 5.28, so
use that.
Author: Victor Wagner <vitus@wagner.pp.ru>
Reviewed-By: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://postgr.es/m/20200501134711.08750c5f@antares.wagner.home
Backpatch: all versions
M src/pl/plperl/plperl.c
Fix failure to validate the result of select_common_type().
commit : c025067f6d3fbffd1acc8c70ec9d2ecc5be8c90b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 29 Jan 2022 11:41:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 29 Jan 2022 11:41:12 -0500
Although select_common_type() has a failure-return convention, an
apparent successful return just provides a type OID that *might* work
as a common supertype; we've not validated that the required casts
actually exist. In the mainstream use-cases that doesn't matter,
because we'll proceed to invoke coerce_to_common_type() on each input,
which will fail appropriately if the proposed common type doesn't
actually work. However, a few callers didn't read the (nonexistent)
fine print, and thought that if they got back a nonzero OID then the
coercions were sure to work.
This affects in particular the recently-added "anycompatible"
polymorphic types; we might think that a function/operator using
such types matches cases it really doesn't. A likely end result
of that is unexpected "ambiguous operator" errors, as for example
in bug #17387 from James Inform. Another, much older, case is that
the parser might try to transform an "x IN (list)" construct to
a ScalarArrayOpExpr even when the list elements don't actually have
a common supertype.
It doesn't seem desirable to add more checking to select_common_type
itself, as that'd just slow down the mainstream use-cases. Instead,
write a separate function verify_common_type that performs the
missing checks, and add a call to that where necessary. Likewise add
verify_common_type_from_oids to go with select_common_type_from_oids.
Back-patch to v13 where the "anycompatible" types came in. (The
symptom complained of in bug #17387 doesn't appear till v14, but
that's just because we didn't get around to converting || to use
anycompatible till then.) In principle the "x IN (list)" fix could
go back all the way, but I'm not currently convinced that it makes
much difference in real-world cases, so I won't bother for now.
Discussion: https://postgr.es/m/17387-5dfe54b988444963@postgresql.org
M src/backend/parser/parse_coerce.c
M src/backend/parser/parse_expr.c
M src/include/parser/parse_coerce.h
M src/test/regress/expected/arrays.out
M src/test/regress/expected/expressions.out
M src/test/regress/sql/arrays.sql
M src/test/regress/sql/expressions.sql
Fix incorrect memory context switch in COPY TO execution
commit : b30282fccfa34238492c0f64808e7b4aa4ef72b3
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 29 Jan 2022 10:23:17 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 29 Jan 2022 10:23:17 +0900
c532d15 has split the logic of COPY commands into multiple files, one
change being to move the internals of BeginCopy() to BeginCopyTo().
Originally the code was written so as we'd switch back-and-forth between
the current execution memory context and the dedicated memory context
for the COPY command, and this refactoring has introduced an extra
switch to the current memory context from the COPY context once
BeginCopyTo() is done with the past logic coming from BeginCopy().
The code was correctly doing the analyze, rewrite and planning phases in
the COPY context, but it was not assigning "copy_file" (FILE* used when
copying to a source file) and "filename" in the COPY context, making the
COPY status data inconsistent.
Author: Bharath Rupireddy
Reviewed-by: Japin Li
Discussion: https://postgr.es/m/CALj2ACWvVa69foi9jhHFY=2BuHxAoYboyE+vXQTARwxZcJnVrQ@mail.gmail.com
Backpatch-through: 14
M src/backend/commands/copyto.c
Fix typo in comment.
commit : d99166ed4f6e9f370289a90dd6678c06bfffd41f
author : Etsuro Fujita <efujita@postgresql.org>
date : Fri, 28 Jan 2022 15:45:01 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Fri, 28 Jan 2022 15:45:01 +0900
M src/backend/executor/nodeForeignscan.c
Prevent memory context logging from sending log message to connected client.
commit : 6e7ee55e727771d97d0fce531a5cda350bb3fa84
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 28 Jan 2022 11:24:42 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 28 Jan 2022 11:24:42 +0900
When pg_log_backend_memory_contexts() is executed, the target backend
should use LOG_SERVER_ONLY to log its memory contexts, to prevent them
from being sent to its connected client regardless of client_min_messages.
But previously the backend unexpectedly used LOG to log the message
"logging memory contexts of PID %d" and it could be sent to the client.
This is a bug in memory context logging.
To fix the bug, this commit changes that message so that it's logged with
LOG_SERVER_ONLY.
Back-patch to v14 where pg_log_backend_memory_contexts() was added.
Author: Fujii Masao
Reviewed-by: Bharath Rupireddy, Atsushi Torikoshi
Discussion: https://postgr.es/m/82c12f36-86f7-5e72-79af-7f5c37f6cad7@oss.nttdata.com
M src/backend/utils/mmgr/mcxt.c
Fix ordering of XIDs in ProcArrayApplyRecoveryInfo
commit : fb2f8e534aaefa649ada8df3e4863ebad32229f7
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 27 Jan 2022 17:53:53 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 27 Jan 2022 17:53:53 +0100
Commit 8431e296ea reworked ProcArrayApplyRecoveryInfo to sort XIDs
before adding them to KnownAssignedXids. But the XIDs are sorted using
xidComparator, which compares the XIDs simply as uint32 values, not
logically. KnownAssignedXidsAdd() however expects XIDs in logical order,
and calls TransactionIdFollowsOrEquals() to enforce that. If there are
XIDs for which the two orderings disagree, an error is raised and the
recovery fails/restarts.
Hitting this issue is fairly easy - you just need two transactions, one
started before the 4B limit (e.g. XID 4294967290), the other sometime
after it (e.g. XID 1000). Logically (4294967290 <= 1000) but when
compared using xidComparator we try to add them in the opposite order.
Which makes KnownAssignedXidsAdd() fail with an error like this:
ERROR: out-of-order XID insertion in KnownAssignedXids
This only happens during replica startup, while processing RUNNING_XACTS
records to build the snapshot. Once we reach STANDBY_SNAPSHOT_READY, we
skip these records. So this does not affect already running replicas,
but if you restart (or create) a replica while there are transactions
with XIDs for which the two orderings disagree, you may hit this.
Long-running transactions and frequent replica restarts increase the
likelihood of hitting this issue. Once the replica gets into this state,
it can't be started (even if the old transactions are terminated).
Fixed by sorting the XIDs logically - this is fine because we're dealing
with normal XIDs (because it's XIDs assigned to backends) and from the
same wraparound epoch (otherwise the backends could not be running at
the same time on the primary node). So there are no problems with the
triangle inequality, which is why xidComparator compares raw values.
Investigation and root cause analysis by Abhijit Menon-Sen. Patch by me.
This issue is present in all releases since 9.4, however releases up to
9.6 are EOL already so backpatch to 10 only.
Reviewed-by: Abhijit Menon-Sen
Reviewed-by: Alvaro Herrera
Backpatch-through: 10
Discussion: https://postgr.es/m/36b8a501-5d73-277c-4972-f58a4dce088a%40enterprisedb.com
M src/backend/storage/ipc/procarray.c
M src/backend/utils/adt/xid.c
M src/include/utils/builtins.h
Improve msys2 detection for TAP tests
commit : 999dc1d2650883b817ea6d4e393ca8ba84a8687d
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 27 Jan 2022 08:17:42 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 27 Jan 2022 08:17:42 -0500
Perl instances on some msys toolchains (e.g. UCRT64) have their
configured osname set to 'MSWin32' rather than 'msys'. The test for
the msys2 platform is adjusted accordingly.
Backpatch to release 14.
M src/test/perl/TestLib.pm
postgres_fdw: Fix handling of a pending asynchronous request in postgresReScanForeignScan().
commit : d1cca9441dc488b9e4d82cf34c9cacc998e6470f
author : Etsuro Fujita <efujita@postgresql.org>
date : Thu, 27 Jan 2022 16:15:02 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Thu, 27 Jan 2022 16:15:02 +0900
Commit 27e1f1456 failed to process a pending asynchronous request made
for a given ForeignScan node in postgresReScanForeignScan() (if any) in
cases where we would only reset the next_tuple counter in that function,
contradicting the assumption that there should be no pending
asynchronous requests that have been made for async-capable subplans for
the parent Append node after ReScan. This led to an assert failure in
an assert-enabled build. I think this would also lead to mis-rewinding
the cursor in that function in the case where we have already fetched
one batch for the ForeignScan node and the asynchronous request has been
made for the second batch, because even in that case we would just reset
the counter when called from that function, so we would fail to execute
MOVE BACKWARD ALL.
To fix, modify that function to process the asynchronous request before
restarting the scan.
While at it, add a comment to a function to match other places.
Per bug #17344 from Alexander Lakhin. Back-patch to v14 where the
aforesaid commit came in.
Patch by me. Test case by Alexander Lakhin, adjusted by me. Reviewed
and tested by Alexander Lakhin and Dmitry Dolgov.
Discussion: https://postgr.es/m/17344-226b78b00de73a7e@postgresql.org
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
On sparc64+ext4, suppress test failures from known WAL read failure.
commit : d94a95cce9eb8582f3bb48fd745f21749b317e83
author : Noah Misch <noah@leadboat.com>
date : Wed, 26 Jan 2022 18:06:19 -0800
committer: Noah Misch <noah@leadboat.com>
date : Wed, 26 Jan 2022 18:06:19 -0800
Buildfarm members kittiwake, tadarida and snapper began to fail
frequently when commits 3cd9c3b921977272e6650a5efbeade4203c4bca2 and
f47ed79cc8a0cfa154dc7f01faaf59822552363f added tests of concurrency, but
the problem was reachable before those commits. Back-patch to v10 (all
supported versions).
Discussion: https://postgr.es/m/20220116210241.GC756210@rfd.leadboat.com
M contrib/amcheck/t/003_cic_2pc.pl
M src/test/perl/TestLib.pm
Fix pg_hba_file_rules for authentication method cert
commit : 4afae689ead4ffbfdbaac1cd72794c729446596e
author : Magnus Hagander <magnus@hagander.net>
date : Wed, 26 Jan 2022 09:52:41 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Wed, 26 Jan 2022 09:52:41 +0100
For authentication method cert, clientcert=verify-full is implied. But
the pg_hba_file_rules entry would incorrectly show clientcert=verify-ca.
Per bug #17354
Reported-By: Feike Steenbergen
Reviewed-By: Jonathan Katz
Backpatch-through: 12
M src/backend/libpq/hba.c
Revert "graceful shutdown" changes for Windows, in back branches only.
commit : 75674c7ec1b1607e7013b5cebcb22d9c8b4b2cb6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 25 Jan 2022 12:17:40 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 25 Jan 2022 12:17:40 -0500
This reverts commits 6051857fc and ed52c3707, but only in the back
branches. Further testing has shown that while those changes do fix
some things, they also break others; in particular, it looks like
walreceivers fail to detect walsender-initiated connection close
reliably if the walsender shuts down this way. We'll keep trying to
improve matters in HEAD, but it now seems unwise to push these changes
into stable releases.
Discussion: https://postgr.es/m/CA+hUKG+OeoETZQ=Qw5Ub5h3tmwQhBmDA=nuNO3KG=zWfUypFAw@mail.gmail.com
M src/backend/libpq/pqcomm.c
Consider parallel awareness when removing single-child Appends
commit : 357ff66153b4791d32c45724d8a0d3c91d89c937
author : David Rowley <drowley@postgresql.org>
date : Tue, 25 Jan 2022 21:14:27 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 25 Jan 2022 21:14:27 +1300
8edd0e794 added some code to remove Append and MergeAppend nodes when they
contained a single child node. As it turned out, this was unsafe to do
when the Append/MergeAppend was parallel_aware and the child node was not.
Removing the Append/MergeAppend, in this case, could lead to the child plan
being called multiple times by parallel workers when it was unsafe to do
so.
Here we fix this by just not removing the Append/MergeAppend when the
parallel_aware flag of the parent and child node don't match.
Reported-by: Yura Sokolov
Bug: #17335
Discussion: https://postgr.es/m/b59605fecb20ba9ea94e70ab60098c237c870628.camel%40postgrespro.ru
Backpatch-through: 12, where 8edd0e794 was first introduced
M src/backend/optimizer/plan/setrefs.c
doc: Fix some grammar
commit : 29d9da36b7b132476ef7b31a5dace7bc7410cc82
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 25 Jan 2022 10:49:30 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 25 Jan 2022 10:49:30 +0900
This is an extraction of the user-visible changes done in 410aa24,
including all the relevant documentation parts.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20220124030001.GQ23027@telsasoft.com
Backpatch-through: 10
M doc/src/sgml/config.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/pg_receivewal.sgml
M doc/src/sgml/ref/select_into.sgml
Fix limitations on what SQL commands can be issued to a walsender.
commit : 1efcc5946d595a168a1abf737030f2e0ee1f4d43
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 24 Jan 2022 15:33:34 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 24 Jan 2022 15:33:34 -0500
In logical replication mode, a WalSender is supposed to be able
to execute any regular SQL command, as well as the special
replication commands. Poor design of the replication-command
parser caused it to fail in various cases, notably:
* semicolons embedded in a command, or multiple SQL commands
sent in a single message;
* dollar-quoted literals containing odd numbers of single
or double quote marks;
* commands starting with a comment.
The basic problem here is that we're trying to run repl_scanner.l
across the entire input string even when it's not a replication
command. Since repl_scanner.l does not understand all of the
token types known to the core lexer, this is doomed to have
failure modes.
We certainly don't want to make repl_scanner.l as big as scan.l,
so instead rejigger stuff so that we only lex the first token of
a non-replication command. That will usually look like an IDENT
to repl_scanner.l, though a comment would end up getting reported
as a '-' or '/' single-character token. If the token is a replication
command keyword, we push it back and proceed normally with repl_gram.y
parsing. Otherwise, we can drop out of exec_replication_command()
without examining the rest of the string.
(It's still theoretically possible for repl_scanner.l to fail on
the first token; but that could only happen if it's an unterminated
single- or double-quoted string, in which case you'd have gotten
largely the same error from the core lexer too.)
In this way, repl_gram.y isn't involved at all in handling general
SQL commands, so we can get rid of the SQLCmd node type. (In
the back branches, we can't remove it because renumbering enum
NodeTag would be an ABI break; so just leave it sit there unused.)
I failed to resist the temptation to clean up some other sloppy
coding in repl_scanner.l while at it. The only externally-visible
behavior change from that is it now accepts \r and \f as whitespace,
same as the core lexer.
Per bug #17379 from Greg Rychlewski. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/17379-6a5c6cfb3f1f5e77@postgresql.org
M src/backend/replication/repl_gram.y
M src/backend/replication/repl_scanner.l
M src/backend/replication/walsender.c
M src/include/replication/walsender_private.h
Remember to reset yy_start state when firing up repl_scanner.l.
commit : ef9706bbc8ce917a366e4640df8c603c9605817a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 24 Jan 2022 12:09:46 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 24 Jan 2022 12:09:46 -0500
Without this, we get odd behavior when the previous cycle of
lexing exited in a non-default exclusive state. Every other
copy of this code is aware that it has to do BEGIN(INITIAL),
but repl_scanner.l did not get that memo.
The real-world impact of this is probably limited, since most
replication clients would abandon their connection after getting
a syntax error. Still, it's a bug.
This mistake is old, so back-patch to all supported branches.
Discussion: https://postgr.es/m/1874781.1643035952@sss.pgh.pa.us
M src/backend/replication/repl_scanner.l
pg_dump: avoid useless query in binary_upgrade_set_type_oids_by_type_oid
commit : 1042de69dbb6f3c0aa92a741a68aeca23f4ed45e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 Jan 2022 13:54:24 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 Jan 2022 13:54:24 -0500
Commit 6df7a9698 wrote appendPQExpBuffer where it should have
written printfPQExpBuffer. This resulted in re-issuing the
previous query along with the desired one, which very accidentally
had no negative consequences except for some wasted cycles.
Back-patch to v14 where that came in.
Discussion: https://postgr.es/m/1714711.1642962663@sss.pgh.pa.us
M src/bin/pg_dump/pg_dump.c
Suppress variable-set-but-not-used warning from clang 13.
commit : ef4edf88df6592b6a29649f3ae69d44cef478291
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 Jan 2022 11:09:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 Jan 2022 11:09:00 -0500
In the normal configuration where GEQO_DEBUG isn't defined,
recent clang versions have started to complain that geqo_main.c
accumulates the edge_failures count but never does anything
with it. As a minimal back-patchable fix, insert a void cast
to silence this warning. (I'd speculated about ripping out the
GEQO_DEBUG logic altogether, but I don't think we'd wish to
back-patch that.)
Per recently-established project policy, this is a candidate
for back-patching into out-of-support branches: it suppresses
an annoying compiler warning but changes no behavior. Hence,
back-patch all the way to 9.2.
Discussion: https://postgr.es/m/CA+hUKGLTSZQwES8VNPmWO9AO0wSeLt36OCPDAZTccT1h7Q7kTQ@mail.gmail.com
M src/backend/optimizer/geqo/geqo_main.c
Correct type of front_pathkey to PathKey
commit : 72ac4d71b54fdac5efc9ecfc23663f26c20ddcf9
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 23 Jan 2022 03:36:55 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 23 Jan 2022 03:36:55 +0100
In sort_inner_and_outer we iterate a list of PathKey elements, but the
variable is declared as (List *). This mistake is benign, because we
only pass the pointer to lcons() and never dereference it.
This exists since ~2004, but it's confusing. So fix and backpatch to all
supported branches.
Backpatch-through: 10
Discussion: https://postgr.es/m/bf3a6ea1-a7d8-7211-0669-189d5c169374%40enterprisedb.com
M src/backend/optimizer/path/joinpath.c
Check syscache result in AlterStatistics
commit : a192243c75be0f142f829b3e73548bc7cbd3b90d
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 23 Jan 2022 02:49:41 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 23 Jan 2022 02:49:41 +0100
The syscache lookup may return NULL even for valid OID, for example due
to a concurrent DROP STATISTICS, so a HeapTupleIsValid is necessary.
Without it, it may fail with a segfault.
Reported by Alexander Lakhin, patch by me. Backpatch to 13, where ALTER
STATISTICS ... SET STATISTICS was introduced.
Backpatch-through: 13
Discussion: https://postgr.es/m/17372-bf3b6e947e35ae77%40postgresql.org
M src/backend/commands/statscmds.c
Flush table's relcache during ALTER TABLE ADD PRIMARY KEY USING INDEX.
commit : 3839e29c58b44639bb6800f3415af87f4d1fb655
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 Jan 2022 13:32:40 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 Jan 2022 13:32:40 -0500
Previously, unless we had to add a NOT NULL constraint to the column,
this command resulted in updating only the index's relcache entry.
That's problematic when replication behavior is being driven off the
existence of a primary key: other sessions (and ours too for that
matter) failed to recalculate their opinion of whether the table can
be replicated. Add a relcache invalidation to fix it.
This has been broken since pg_class.relhaspkey was removed in v11.
Before that, updating the table's relhaspkey value sufficed to cause
a cache flush. Hence, backpatch to v11.
Report and patch by Hou Zhijie
Discussion: https://postgr.es/m/OS0PR01MB5716EBE01F112C62F8F9B786947B9@OS0PR01MB5716.jpnprd01.prod.outlook.com
M src/backend/catalog/index.c
M src/test/regress/expected/publication.out
M src/test/regress/sql/publication.sql
Fix race condition in gettext() initialization in libpq and ecpglib.
commit : f4ebf0dbea835da85c21988321258a587a567dbd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Jan 2022 15:36:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Jan 2022 15:36:12 -0500
In libpq and ecpglib, multiple threads can concurrently enter the
initialization logic for message localization. Since we set the
its-done flag before actually doing the work, it'd be possible
for some threads to reach gettext() before anyone has called
bindtextdomain(). Barring bugs in libintl itself, this would not
result in anything worse than failure to localize some early
messages. Nonetheless, it's a bug, and an easy one to fix.
Noted while investigating bug #17299 from Clemens Zeidler
(much thanks to Liam Bowen for followup investigation on that).
It currently appears that that actually *is* a bug in libintl itself,
but that doesn't let us off the hook for this bit.
Back-patch to all supported versions.
Discussion: https://postgr.es/m/17299-7270741958c0b1ab@postgresql.org
Discussion: https://postgr.es/m/CAE7q7Eit4Eq2=bxce=Fm8HAStECjaXUE=WBQc-sDDcgJQ7s7eg@mail.gmail.com
M src/interfaces/ecpg/ecpglib/misc.c
M src/interfaces/libpq/fe-misc.c
fsync pg_logical/mappings in CheckPointLogicalRewriteHeap().
commit : 2b7dbe4bd5872ae5c6780e1a4121c2a5a8401e7c
author : Andres Freund <andres@anarazel.de>
date : Fri, 21 Jan 2022 11:22:55 -0800
committer: Andres Freund <andres@anarazel.de>
date : Fri, 21 Jan 2022 11:22:55 -0800
While individual logical rewrite files were synced to disk, the directory was
not. On some filesystems that could lead to loosing directory entries after a
crash.
Reported-By: Tom Lane <tgl@sss.pgh.pa.us>
Author: Nathan Bossart <bossartn@amazon.com>
Discussion: https://postgr.es/m/867F2E29-2782-4869-970E-B984C6D35A8F@amazon.com
Backpatch: 10-
M src/backend/access/heap/rewriteheap.c
postgres_fdw: Fix subabort cleanup of connections used in asynchronous execution.
commit : 700e8fe6d414ebaf9820fef364097d9b83cae7fd
author : Etsuro Fujita <efujita@postgresql.org>
date : Fri, 21 Jan 2022 17:45:02 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Fri, 21 Jan 2022 17:45:02 +0900
Commit 27e1f1456 resets the per-connection states of connections used to
scan foreign tables asynchronously during abort cleanup at main
transaction end, but it failed to do so during subabort cleanup at
subtransaction end, leading to a segmentation fault when re-executing an
asynchronous-foreign-table-scan query in a transaction that was
cancelled in a subtransaction of it.
Fix by modifying pgfdw_abort_cleanup() to reset the per-connection state
of a given connection also when called for subabort cleanup. Also,
modify that function to do the reset in both the abort-cleanup and
subabort-cleanup cases if necessary, to save cycles, and improve a
comment on it a little bit.
Back-patch to v14 where the aforesaid commit came in.
Reviewed by Alexander Pyhalov
Discussion: https://postgr.es/m/CAPmGK14cCV-JA7kNsyt2EUTKvZ4xkr2LNRthi1U1C3cqfGppAw@mail.gmail.com
M contrib/postgres_fdw/connection.c
Fix one-off bug causing missing commit timestamps for subtransactions
commit : 84db5169d4715904b0bffa25509f3c77fcd52188
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Jan 2022 14:54:47 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Jan 2022 14:54:47 +0900
The logic in charge of writing commit timestamps (enabled with
track_commit_timestamp) for subtransactions had a one-bug bug,
where it would be possible that commit timestamps go missing for the
last subtransaction committed.
While on it, simplify a bit the iteration logic in the loop writing the
commit timestamps, as per suggestions from Kyotaro Horiguchi and Tom
Lane, so as some variable initializations are not part of the loop
itself.
Issue introduced in 73c986a.
Analyzed-by: Alex Kingsborough
Author: Alex Kingsborough, Kyotaro Horiguchi
Discussion: https://postgr.es/m/73A66172-4050-4F2A-B7F1-13508EDA2144@amazon.com
Backpatch-through: 10
M src/backend/access/transam/commit_ts.c
Tighten TAP tests' tracking of postmaster state some more.
commit : cf680bd6538a5a05020ab4d193b43563e9eb0c5e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 20 Jan 2022 17:28:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 20 Jan 2022 17:28:07 -0500
Commits 6c4a8903b et al. had a couple of deficiencies:
* The logic I added to Cluster::start to see if a PID file is present
could be fooled by a stale PID file left over from a previous
postmaster. To fix, if we're not sure whether we expect to find a
running postmaster or not, validate the PID using "kill 0".
* 017_shm.pl has a loop in which it just issues repeated Cluster::start
calls; this will fail if some invocation fails but leaves self->_pid
set. Per buildfarm results, the above fix is not enough to make this
safe: we might have "validated" a PID for a postmaster that exits
immediately after we look. Hence, match each failed start call with
a stop call that will get us back to the self->_pid == undef state.
Add a fail_ok option to Cluster::stop to make this work.
Discussion: https://postgr.es/m/CA+hUKGKV6fOHvfiPt8=dOKzvswjAyLoFoJF1iQXMNpi7+hD1JQ@mail.gmail.com
M src/test/perl/PostgresNode.pm
M src/test/recovery/t/017_shm.pl
Allow clean.bat to be run from anywhere
commit : 156a846d93ee2dab365a89ddd1469ef7a9d1b92e
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 20 Jan 2022 10:13:18 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 20 Jan 2022 10:13:18 -0500
This was omitted from c3879a7b4c which modified the other msvc .bat
files.
Per request from Juan José Santamaría Flecha
Discussion: https://postgr.es/m/CAC+AXB0_fxYGbQoaYjCA8um7TTbOVP4L9aXnVmHwK8WzaT4gdA@mail.gmail.com
Backpatch to all live branches.
M src/tools/msvc/clean.bat
Try to stabilize reloptions test, again.
commit : b9dd162205bf2e1a20ce21a80e6ea497c7980ffe
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 20 Jan 2022 22:11:48 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 20 Jan 2022 22:11:48 +1300
Since the test requires reproducible behavior from VACUUM, and since
DISABLE_PAGE_SKIPPING doesn't actually disable all forms of page
skipping, let's use a temporary table to avoid contention.
Back-patch to 12, like commit 3414099c.
Discussion: https://postgr.es/m/20220120052404.sonrhq3f3qgplpzj%40alap3.anarazel.de
M src/test/regress/expected/reloptions.out
M src/test/regress/sql/reloptions.sql
doc: Mention the level of locks taken on objects in COMMENT
commit : a9009f59fff148b11588cfccdc3989d3363cfab1
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Jan 2022 16:54:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Jan 2022 16:54:54 +0900
This information was nowhere to be found. This adds one note on the
page of COMMENT, and one note in the section dedicated to explicit
locking, both telling that a SHARE UPDATE EXCLUSIVE lock is taken on the
object commented.
Author: Nikolai Berkoff
Reviewed-by: Laurenz Albe
Discussion: https://postgr.es/m/_0HDHIGcCdCsUyXn22QwI2FEuNR6Fs71rtgGX6hfyBlUh5rrnE2qMmvIFu9EY4Pijr2gUmJEAXCjuNU2Oxku9TryLp9CdHllpsCfN3gD0-Y=@pm.me
Backpatch-through: 10
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/ref/comment.sgml
TAP tests: check for postmaster.pid anyway when "pg_ctl start" fails.
commit : b0623625a0a30d66e991deab6488c59bb63ea20e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 19 Jan 2022 16:29:09 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 19 Jan 2022 16:29:09 -0500
"pg_ctl start" might start a new postmaster and then return failure
anyway, for example if PGCTLTIMEOUT is exceeded. If there is a
postmaster there, it's still incumbent on us to shut it down at
script end, so check for the PID file even though we are about
to fail.
This has been broken all along, so back-patch to all supported branches.
Discussion: https://postgr.es/m/647439.1642622744@sss.pgh.pa.us
M src/test/perl/PostgresNode.pm
doc: Fix description of pg_replication_origin_oid() in error case
commit : 0f1367c9f381aae523f6557c0ee28a800b3eace7
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Jan 2022 10:37:47 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Jan 2022 10:37:47 +0900
This function returns NULL if the replication origin given in input
argument does not exist, contrary to what the docs described
previously.
Author: Ian Barwick
Discussion: https://postgr.es/m/CAB8KJ=htJjBL=103URqjOxV2mqb4rjphDpMeKdyKq_QXt6h05w@mail.gmail.com
Backpatch-through: 10
M doc/src/sgml/func.sgml
Try to stabilize the reloptions test.
commit : ff4d0d8cc5e2e3823bf3428e707ee33c13d401f6
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Jan 2022 07:03:01 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Jan 2022 07:03:01 +1300
Where we test vacuum_truncate's effects, sometimes this is failing to
truncate as expected on the build farm. That could be explained by page
skipping, so disable it explicitly, with the theory that commit fe246d1c
didn't go far enough.
Back-patch to 12, where the vacuum_truncate tests were added.
Discussion: https://postgr.es/m/CA%2BhUKGLT2UL5_JhmBzUgkdyKfc%3D5J-gJSQJLysMs4rqLUKLAzw%40mail.gmail.com
M src/test/regress/expected/reloptions.out
M src/test/regress/sql/reloptions.sql
Fix psql \d's query for identifying parent triggers.
commit : 3886785b4ce59147b92319e7204ab64a9d8801cb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 Jan 2022 21:18:49 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 Jan 2022 21:18:49 -0500
The original coding (from c33869cc3) failed with "more than one row
returned by a subquery used as an expression" if there were unrelated
triggers of the same tgname on parent partitioned tables. (That's
possible because statement-level triggers don't get inherited.) Fix
by applying LIMIT 1 after sorting the candidates by inheritance level.
Also, wrap the subquery in a CASE so that we don't have to execute it at
all when the trigger is visibly non-inherited. Aside from saving some
cycles, this avoids the need for a confusing and undocumented NULLIF().
While here, tweak the format of the emitted query to look a bit
nicer for "psql -E", and add some explanation of this subquery,
because it badly needs it.
Report and patch by Justin Pryzby (with some editing by me).
Back-patch to v13 where the faulty code came in.
Discussion: https://postgr.es/m/20211217154356.GJ17618@telsasoft.com
M src/bin/psql/describe.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Avoid calling gettext() in signal handlers.
commit : 4e8726566e4d7c0e712d16c0dc7a27d212ece090
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 Jan 2022 13:30:04 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 Jan 2022 13:30:04 -0500
It seems highly unlikely that gettext() can be relied on to be
async-signal-safe. psql used to understand that, but someone got
it wrong long ago in the src/bin/scripts/ version of handle_sigint,
and then the bad idea was perpetuated when those two versions were
unified into src/fe_utils/cancel.c.
I'm unsure why there have not been field complaints about this
... maybe gettext() is signal-safe once it's translated at least
one message? But we have no business assuming any such thing.
In cancel.c (v13 and up), I preserved our ability to localize
"Cancel request sent" messages by invoking gettext() before
the signal handler is set up. In earlier branches I just made
src/bin/scripts/ not localize those messages, as psql did then.
(Just for extra unsafety, the src/bin/scripts/ version was
invoking fprintf() from a signal handler. Sigh.)
Noted while fixing signal-safety issues in PQcancel() itself.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/2937814.1641960929@sss.pgh.pa.us
M src/fe_utils/cancel.c
Avoid calling strerror[_r] in PQcancel().
commit : 050949877004f003b7d95fddf8f572a32f3a2f70
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 Jan 2022 12:52:44 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 Jan 2022 12:52:44 -0500
PQcancel() is supposed to be safe to call from a signal handler,
and indeed psql uses it that way. All of the library functions
it uses are specified to be async-signal-safe by POSIX ...
except for strerror. Neither plain strerror nor strerror_r
are considered safe. When this code was written, back in the
dark ages, we probably figured "oh, strerror will just index
into a constant array of strings" ... but in any locale except C,
that's unlikely to be true. Probably the reason we've not heard
complaints is that (a) this error-handling code is unlikely to be
reached in normal use, and (b) in many scenarios, localized error
strings would already have been loaded, after which maybe it's
safe to call strerror here. Still, this is clearly unacceptable.
The best we can do without relying on strerror is to print the
decimal value of errno, so make it do that instead. (This is
probably not much loss of user-friendliness, given that it is
hard to get a failure here.)
Back-patch to all supported branches.
Discussion: https://postgr.es/m/2937814.1641960929@sss.pgh.pa.us
M src/interfaces/libpq/fe-connect.c
Teach hash_ok_operator() that record_eq is only sometimes hashable.
commit : 17da9d4c28297fd699cbbda969a9f64c4c09c665
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 16 Jan 2022 16:39:26 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 16 Jan 2022 16:39:26 -0500
The need for this was foreseen long ago, but when record_eq
actually became hashable (in commit 01e658fa7), we missed updating
this spot.
Per bug #17363 from Elvis Pranskevichus. Back-patch to v14 where
the faulty commit came in.
Discussion: https://postgr.es/m/17363-f6d42fd0d726be02@postgresql.org
M src/backend/optimizer/plan/subselect.c
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
Fix psql's tab-completion of enum label values.
commit : d91d4338e05486f943361260b9d4ad54421d3b44
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 16 Jan 2022 14:59:20 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 16 Jan 2022 14:59:20 -0500
Since enum labels have to be single-quoted, this part of the
tab completion machinery got side-swiped by commit cd69ec66c.
A side-effect of that commit is that (at least with some versions
of Readline) the text string passed for completion will omit the
leading quote mark of the enum label literal. Libedit still acts
the same as before, though, so adapt COMPLETE_WITH_ENUM_VALUE so
that it can cope with either convention.
Also, when we fail to find any valid completion, set
rl_completion_suppress_quote = 1. Otherwise readline will
go ahead and append a closing quote, which is unwanted.
Per report from Peter Eisentraut. Back-patch to v13 where
cd69ec66c came in.
Discussion: https://postgr.es/m/8ca82d89-ec3d-8b28-8291-500efaf23b25@enterprisedb.com
M src/bin/psql/t/010_tab_completion.pl
M src/bin/psql/tab-complete.c
Build inherited extended stats on partitioned tables
commit : ea212bd95fd2456953ca8995cf2a04d3db82ac19
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 15 Jan 2022 18:17:20 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 15 Jan 2022 18:17:20 +0100
Commit 859b3003de disabled building of extended stats for inheritance
trees, to prevent updating the same catalog row twice. While that
resolved the issue, it also means there are no extended stats for
declaratively partitioned tables, because there are no data in the
non-leaf relations.
That also means declaratively partitioned tables were not affected by
the issue 859b3003de addressed, which means this is a regression
affecting queries that calculate estimates for the whole inheritance
tree as a whole (which includes e.g. GROUP BY queries).
But because partitioned tables are empty, we can invert the condition
and build statistics only for the case with inheritance, without losing
anything. And we can consider them when calculating estimates.
It may be necessary to run ANALYZE on partitioned tables, to collect
proper statistics. For declarative partitioning there should no prior
statistics, and it might take time before autoanalyze is triggered. For
tables partitioned by inheritance the statistics may include data from
child relations (if built 859b3003de), contradicting the current code.
Report and patch by Justin Pryzby, minor fixes and cleanup by me.
Backpatch all the way back to PostgreSQL 10, where extended statistics
were introduced (same as 859b3003de).
Author: Justin Pryzby
Reported-by: Justin Pryzby
Backpatch-through: 10
Discussion: https://postgr.es/m/20210923212624.GI831%40telsasoft.com
M src/backend/commands/analyze.c
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/utils/adt/selfuncs.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Ignore extended statistics for inheritance trees
commit : 2cc007fd0359870b4a1e959af0ac94e61df47240
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 15 Jan 2022 02:15:23 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 15 Jan 2022 02:15:23 +0100
Since commit 859b3003de we only build extended statistics for individual
relations, ignoring the child relations. This resolved the issue with
updating catalog tuple twice, but we still tried to use the statistics
when calculating estimates for the whole inheritance tree. When the
relations contain very distinct data, it may produce bogus estimates.
This is roughly the same issue 427c6b5b9 addressed ~15 years ago, and we
fix it the same way - by ignoring extended statistics when calculating
estimates for the inheritance tree as a whole. We still consider
extended statistics when calculating estimates for individual child
relations, of course.
This may result in plan changes due to different estimates, but if the
old statistics were not describing the inheritance tree particularly
well it's quite likely the new plans is actually better.
Report and patch by Justin Pryzby, minor fixes and cleanup by me.
Backpatch all the way back to PostgreSQL 10, where extended statistics
were introduced (same as 859b3003de).
Author: Justin Pryzby
Reported-by: Justin Pryzby
Backpatch-through: 10
Discussion: https://postgr.es/m/20210923212624.GI831%40telsasoft.com
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/utils/adt/selfuncs.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Fix possible HOT corruption when RECENTLY_DEAD changes to DEAD while pruning.
commit : dad1539aec28ae3d024b00ae322f02f7fe0c41ee
author : Andres Freund <andres@anarazel.de>
date : Fri, 10 Dec 2021 20:12:26 -0800
committer: Andres Freund <andres@anarazel.de>
date : Fri, 10 Dec 2021 20:12:26 -0800
Since dc7420c2c92 the horizon used for pruning is determined "lazily". A more
accurate horizon is built on-demand, rather than in GetSnapshotData(). If a
horizon computation is triggered between two HeapTupleSatisfiesVacuum() calls
for the same tuple, the result can change from RECENTLY_DEAD to DEAD.
heap_page_prune() can process the same tid multiple times (once following an
update chain, once "directly"). When the result of HeapTupleSatisfiesVacuum()
of a tuple changes from RECENTLY_DEAD during the first access, to DEAD in the
second, the "tuple is DEAD and doesn't chain to anything else" path in
heap_prune_chain() can end up marking the target of a LP_REDIRECT ItemId
unused.
Initially not easily visible,
Once the target of a LP_REDIRECT ItemId is marked unused, a new tuple version
can reuse it. At that point the corruption may become visible, as index
entries pointing to the "original" redirect item, now point to a unrelated
tuple.
To fix, compute HTSV for all tuples on a page only once. This fixes the entire
class of problems of HTSV changing inside heap_page_prune(). However,
visibility changes can obviously still occur between HTSV checks inside
heap_page_prune() and outside (e.g. in lazy_scan_prune()).
The computation of HTSV is now done in bulk, in heap_page_prune(), rather than
on-demand in heap_prune_chain(). Besides being a bit simpler, it also is
faster: Memory accesses can happen sequentially, rather than in the order of
HOT chains.
There are other causes of HeapTupleSatisfiesVacuum() results changing between
two visibility checks for the same tuple, even before dc7420c2c92. E.g.
HEAPTUPLE_INSERT_IN_PROGRESS can change to HEAPTUPLE_DEAD when a transaction
aborts between the two checks. None of the these other visibility status
changes are known to cause corruption, but heap_page_prune()'s approach makes
it hard to be confident.
A patch implementing a more fundamental redesign of heap_page_prune(), which
fixes this bug and simplifies pruning substantially, has been proposed by
Peter Geoghegan in
https://postgr.es/m/CAH2-WzmNk6V6tqzuuabxoxM8HJRaWU6h12toaS-bqYcLiht16A@mail.gmail.com
However, that redesign is larger change than desirable for backpatching. As
the new design still benefits from the batched visibility determination
introduced in this commit, it makes sense to commit this narrower fix to 14
and master, and then commit Peter's improvement in master.
The precise sequence required to trigger the bug is complicated and hard to do
exercise in an isolation test (until we have wait points). Due to that the
isolation test initially posted at
https://postgr.es/m/20211119003623.d3jusiytzjqwb62p%40alap3.anarazel.de
and updated in
https://postgr.es/m/20211122175914.ayk6gg6nvdwuhrzb%40alap3.anarazel.de
isn't committable.
A followup commit will introduce additional assertions, to detect problems
like this more easily.
Bug: #17255
Reported-By: Alexander Lakhin <exclusion@gmail.com>
Debugged-By: Andres Freund <andres@anarazel.de>
Debugged-By: Peter Geoghegan <pg@bowt.ie>
Author: Andres Freund <andres@andres@anarazel.de>
Reviewed-By: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/20211122175914.ayk6gg6nvdwuhrzb@alap3.anarazel.de
Backpatch: 14-, the oldest branch containing dc7420c2c92
M src/backend/access/heap/pruneheap.c
Revert error handling improvements for cryptohashes
commit : ad5b6f248ab288c3252d8122d12a1eb410d4a0b6
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 14 Jan 2022 11:25:39 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 14 Jan 2022 11:25:39 +0900
This reverts commits ab27df2, af8d530 and 3a0cced, that introduced
pg_cryptohash_error(). In order to make the core code able to pass down
the new error types that this introduced, some of the MD5-related
routines had to be reworked, causing an ABI breakage, but we found that
some external extensions rely on them. Maintaining compatibility
outweights the error report benefits, so just revert the change in v14.
Reported-by: Laurenz Albe
Discussion: https://postgr.es/m/9f0c0a96d28cf14fc87296bbe67061c14eb53ae8.camel@cybertec.at
M contrib/passwordcheck/passwordcheck.c
M contrib/pgcrypto/internal-sha2.c
M contrib/pgcrypto/internal.c
M contrib/uuid-ossp/uuid-ossp.c
M src/backend/commands/user.c
M src/backend/libpq/auth-scram.c
M src/backend/libpq/auth.c
M src/backend/libpq/crypt.c
M src/backend/replication/backup_manifest.c
M src/backend/utils/adt/cryptohashfuncs.c
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
M src/common/md5_common.c
M src/include/common/cryptohash.h
M src/include/common/md5.h
M src/include/libpq/crypt.h
M src/include/libpq/scram.h
M src/interfaces/libpq/fe-auth.c
M src/tools/pgindent/typedefs.list
Fix ruleutils.c's dumping of whole-row Vars in more contexts.
commit : 4aee39ddb8fa748a6beea2bc1b48882990c226a7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 13 Jan 2022 17:49:26 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 13 Jan 2022 17:49:26 -0500
Commit 7745bc352 intended to ensure that whole-row Vars would be
printed with "::type" decoration in all contexts where plain
"var.*" notation would result in star-expansion, notably in
ROW() and VALUES() constructs. However, it missed the case of
INSERT with a single-row VALUES, as reported by Timur Khanjanov.
Nosing around ruleutils.c, I found a second oversight: the
code for RowCompareExpr generates ROW() notation without benefit
of an actual RowExpr, and naturally it wasn't in sync :-(.
(The code for FieldStore also does this, but we don't expect that
to generate strictly parsable SQL anyway, so I left it alone.)
Back-patch to all supported branches.
Discussion: https://postgr.es/m/efaba6f9-4190-56be-8ff2-7a1674f9194f@intrans.baku.az
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/create_view.out
M src/test/regress/sql/create_view.sql
Fix incorrect comments in hmac.c and hmac_openssl.c
commit : 3c1ffd02dd058666ddf27d7e621ab5bbc4b769bb
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 13 Jan 2022 09:43:44 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 13 Jan 2022 09:43:44 +0900
Both files referred to pg_hmac_ctx->data, which, I guess, comes from the
early versions of the patch that has resulted in commit e6bdfd9.
Author: Sergey Shinderuk
Discussion: https://postgr.es/m/8cbb56dd-63d6-a581-7a65-25a97ac4be03@postgrespro.ru
Backpatch-through: 14
M src/common/hmac.c
M src/common/hmac_openssl.c
Fix memory leak in indexUnchanged hint mechanism.
commit : 41ee68a91f0fc8e1f6583e4f153d79578adb265c
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 12 Jan 2022 15:41:02 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 12 Jan 2022 15:41:02 -0800
Commit 9dc718bd added a "logically unchanged by UPDATE" hinting
mechanism, which is currently used within nbtree indexes only (see
commit d168b666). This mechanism determined whether or not the incoming
item is a logically unchanged duplicate (a duplicate needed only for
MVCC versioning purposes) once per row updated per non-HOT update. This
approach led to memory leaks which were noticeable with an UPDATE
statement that updated sufficiently many rows, at least on tables that
happen to have an expression index.
On HEAD, fix the issue by adding a cache to the executor's per-index
IndexInfo struct.
Take a different approach on Postgres 14 to avoid an ABI break: simply
pass down the hint to all indexes unconditionally with non-HOT UPDATEs.
This is deemed acceptable because the hint is currently interpreted
within btinsert() as "perform a bottom-up index deletion pass if and
when the only alternative is splitting the leaf page -- prefer to delete
any LP_DEAD-set items first". nbtree must always treat the hint as a
noisy signal about what might work, as a strategy of last resort, with
costs imposed on non-HOT updaters. (The same thing might not be true
within another index AM that applies the hint, which is why the original
behavior is preserved on HEAD.)
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Klaudie Willis <Klaudie.Willis@protonmail.com>
Diagnosed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/261065.1639497535@sss.pgh.pa.us
Backpatch: 14-, where the hinting mechanism was added.
M src/backend/executor/execIndexing.c
Fix comment related to pg_cryptohash_error()
commit : af8d530e47588d909a3c8d400b21a273e50498d6
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 12 Jan 2022 12:40:04 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 12 Jan 2022 12:40:04 +0900
One of the comments introduced in b69aba7 was worded a bit weirdly, so
improve it.
Reported-by: Sergey Shinderuk
Discussion: https://postgr.es/m/71b9a5d2-a3bf-83bc-a243-93dcf0bcfb3b@postgrespro.ru
Backpatch-through: 14
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
Clean up error message reported after \password encryption failure.
commit : ab27df2490e3a5dfb8ecb3de66818d16645ef3dd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Jan 2022 12:03:06 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Jan 2022 12:03:06 -0500
Experimenting with FIPS mode enabled, I saw
regression=# \password joe
Enter new password for user "joe":
Enter it again:
could not encrypt password: disabled for FIPS
out of memory
because PQencryptPasswordConn was still of the opinion that "out of
memory" is always appropriate to print.
Minor oversight in b69aba745. Like that one, back-patch to v14.
M src/interfaces/libpq/fe-auth.c
Improve error handling of cryptohash computations
commit : 3a0cced86d3b1bbbf0aaa0fb34e2b15edd643979
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 11 Jan 2022 09:55:24 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 11 Jan 2022 09:55:24 +0900
The existing cryptohash facility was causing problems in some code paths
related to MD5 (frontend and backend) that relied on the fact that the
only type of error that could happen would be an OOM, as the MD5
implementation used in PostgreSQL ~13 (the in-core implementation is
used when compiling with or without OpenSSL in those older versions),
could fail only under this circumstance.
The new cryptohash facilities can fail for reasons other than OOMs, like
attempting MD5 when FIPS is enabled (upstream OpenSSL allows that up to
1.0.2, Fedora and Photon patch OpenSSL 1.1.1 to allow that), so this
would cause incorrect reports to show up.
This commit extends the cryptohash APIs so as callers of those routines
can fetch more context when an error happens, by using a new routine
called pg_cryptohash_error(). The error states are stored within each
implementation's internal context data, so as it is possible to extend
the logic depending on what's suited for an implementation. The default
implementation requires few error states, but OpenSSL could report
various issues depending on its internal state so more is needed in
cryptohash_openssl.c, and the code is shaped so as we are always able to
grab the necessary information.
The core code is changed to adapt to the new error routine, painting
more "const" across the call stack where the static errors are stored,
particularly in authentication code paths on variables that provide
log details. This way, any future changes would warn if attempting to
free these strings. The MD5 authentication code was also a bit blurry
about the handling of "logdetail" (LOG sent to the postmaster), so
improve the comments related that, while on it.
The origin of the problem is 87ae969, that introduced the centralized
cryptohash facility. Extra changes are done for pgcrypto in v14 for the
non-OpenSSL code path to cope with the improvements done by this
commit.
Reported-by: Michael Mühlbeyer
Author: Michael Paquier
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/89B7F072-5BBE-4C92-903E-D83E865D9367@trivadis.com
Backpatch-through: 14
M contrib/passwordcheck/passwordcheck.c
M contrib/pgcrypto/internal-sha2.c
M contrib/pgcrypto/internal.c
M contrib/uuid-ossp/uuid-ossp.c
M src/backend/commands/user.c
M src/backend/libpq/auth-scram.c
M src/backend/libpq/auth.c
M src/backend/libpq/crypt.c
M src/backend/replication/backup_manifest.c
M src/backend/utils/adt/cryptohashfuncs.c
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
M src/common/md5_common.c
M src/include/common/cryptohash.h
M src/include/common/md5.h
M src/include/libpq/crypt.h
M src/include/libpq/scram.h
M src/interfaces/libpq/fe-auth.c
M src/tools/pgindent/typedefs.list
Doc: fix bogus example about ambiguous timestamps.
commit : 05cdda68f8f6e63cb87bc1776236d7619cd89fb7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Jan 2022 11:46:16 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Jan 2022 11:46:16 -0500
I had a brain fade in commit d32899157, and used 2:30AM as the
example timestamp for both spring-forward and fall-back cases.
But it's not actually ambiguous at all in the fall-back case,
because that transition is from 2AM to 1AM under USA rules.
Fix the example to use 1:30AM, which *is* ambiguous.
Noted while answering a question from Aleksander Alekseev.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/2191355.1641828552@sss.pgh.pa.us
M doc/src/sgml/datetime.sgml
Avoid warning about uninitialized value in MSVC python3 tests
commit : 1cd46f168fda4d819e315eacf3d4f3d5aa84859c
author : Andrew Dunstan <andrew@dunslane.net>
date : Mon, 10 Jan 2022 10:08:44 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Mon, 10 Jan 2022 10:08:44 -0500
Juan José Santamaría Flecha
Backpatch to all live branches
M src/tools/msvc/vcregress.pl
Fix results of index-only scans on btree_gist char(N) indexes.
commit : 043c1e1a73aace8b459b3290ff7a50c9b124fb33
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 8 Jan 2022 14:54:39 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 8 Jan 2022 14:54:39 -0500
If contrib/btree_gist is used to make a GIST index on a char(N)
(bpchar) column, and that column is retrieved via an index-only
scan, what came out had all trailing spaces removed. Since
that doesn't happen in any other kind of table scan, this is
clearly a bug. The cause is that gbt_bpchar_compress() strips
trailing spaces (using rtrim1) before a new index entry is made.
That was probably a good idea when this code was first written,
but since we invented index-only scans, it's not so good.
One answer could be to mark this opclass as incapable of index-only
scans. But to do so, we'd need an extension module version bump,
followed by manual action by DBAs to install the updated version
of btree_gist. And it's not really a desirable place to end up,
anyway.
Instead, let's fix the code by removing the unwanted space-stripping
action and adjusting the opclass's comparison logic to ignore
trailing spaces as bpchar normally does. This will not hinder
cases that work today, since index searches with this logic will
act the same whether trailing spaces are stored or not. It will
not by itself fix the problem of getting space-stripped results
from index-only scans, of course. Users who care about that can
REINDEX affected indexes after installing this update, to immediately
replace all improperly-truncated index entries. Otherwise, it can
be expected that the index's behavior will change incrementally as
old entries are replaced by new ones.
Per report from Alexander Lakhin. Back-patch to all supported branches.
Discussion: https://postgr.es/m/696c995b-b37f-5526-f45d-04abe713179f@gmail.com
M contrib/btree_gist/btree_text.c
M contrib/btree_gist/expected/char.out
M contrib/btree_gist/expected/char_1.out
Fix issues with describe queries of extended statistics in psql
commit : f5bea8360691eeaef7ad3b6163b3e2bbb9c7c466
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 8 Jan 2022 16:45:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 8 Jan 2022 16:45:14 +0900
This addresses some problems in the describe queries used for extended
statistics:
- Two schema qualifications for the text type were missing for \dX.
- The list of extended statistics listed for a table through \d was
ordered based on the object OIDs, but it is more consistent with the
other commands to order by namespace and then by object name.
- A couple of aliases were not used in \d. These are removed.
This is similar to commits 1f092a3 and 07f8a9e.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20220107022235.GA14051@telsasoft.com
Backpatch-through: 14
M src/bin/psql/describe.c
Update copyright for 2022
commit : 61c8da50cb39ab41c3a7a0122d6943f72bb0798e
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 7 Jan 2022 19:04:56 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 7 Jan 2022 19:04:56 -0500
Backpatch-through: 10
M COPYRIGHT
M doc/src/sgml/legal.sgml
Allow MSVC .bat wrappers to be called from anywhere
commit : a7772e8748a7783776f0095e0d7068bcf42e3d69
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 7 Jan 2022 16:07:45 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 7 Jan 2022 16:07:45 -0500
Instead of using a hardcoded or default path to the perl file the .bat
file is a wrapper for, we use a path that means the file is found in
the same directory as the .bat file.
Patch by Anton Voloshin, slightly tweaked by me.
Backpatch to all live branches
Discussion: https://postgr.es/m/2b7a674b-5fb0-d264-75ef-ecc7a31e54f8@postgrespro.ru
M src/tools/msvc/build.bat
M src/tools/msvc/install.bat
M src/tools/msvc/pgbison.bat
M src/tools/msvc/pgflex.bat
M src/tools/msvc/vcregress.bat
Prevent altering partitioned table's rowtype, if it's used elsewhere.
commit : f285d95839e05608ee99dba399abe860d3c026c5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 6 Jan 2022 16:46:46 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 6 Jan 2022 16:46:46 -0500
We disallow altering a column datatype within a regular table,
if the table's rowtype is used as a column type elsewhere,
because we lack code to go around and rewrite the other tables.
This restriction should apply to partitioned tables as well, but it
was not checked because ATRewriteTables and ATPrepAlterColumnType
were not on the same page about who should do it for which relkinds.
Per bug #17351 from Alexander Lakhin. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/17351-6db1870f3f4f612a@postgresql.org
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
doc: Remove link to JSON support in the SQL specification
commit : 9ae0a4c702324110dde366f9370f9b6c02b4dee8
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 6 Jan 2022 11:41:40 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 6 Jan 2022 11:41:40 +0900
The link used in the documentation is dead, and the only options to have
an access to this part of the SQL specification are not free. Like any
other books referred, just remove the link to keep some neutrality but
keep its reference.
Reported-by: Erik Rijkers
Discussion: https://postgr.es/m/989abd7d-af30-ab52-1201-bf0b4f33b872@xs4all.nl
Backpatch-through: 12
M doc/src/sgml/biblio.sgml
Reduce relcache access in WAL sender streaming logical changes
commit : 5ddfebded41f7754b3c4fc1b576ca33736efcab6
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 5 Jan 2022 10:27:47 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 5 Jan 2022 10:27:47 +0900
get_rel_sync_entry(), which is called each time a change needs to be
logically replicated, is a rather hot code path in the WAL sender
sending logical changes. This code path was doing a relcache access on
relkind and relpartition for each logical change, but we only need to
know this information when building or re-building the cached
information for a relation.
Some measurements prove that this is noticeable in perf profiles,
particularly when attempting to replicate changes from relations that
are not published as these cause less overhead in the WAL sender,
delaying further the replication of changes for relations that are
published.
Issue introduced in 83fd453.
Author: Hou Zhijie
Reviewed-by: Kyotaro Horiguchi, Euler Taveira
Discussion: https://postgr.es/m/OS0PR01MB5716E863AA9E591C1F010F7A947D9@OS0PR01MB5716.jpnprd01.prod.outlook.com
Backpatch-through: 13
M src/backend/replication/pgoutput/pgoutput.c
Fix silly mistake in Assert
commit : f9db153c282de599a7d5ae2694f50fe10fba8461
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 4 Jan 2022 13:21:23 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 4 Jan 2022 13:21:23 -0300
M src/backend/access/heap/heapam.c
Allow special SKIP LOCKED condition in Assert()
commit : f185f35a83d980033824c3522b7b97ab1c2de71e
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 4 Jan 2022 13:01:05 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 4 Jan 2022 13:01:05 -0300
Under concurrency, it is possible for two sessions to be merrily locking
and releasing a tuple and marking it again as HEAP_XMAX_INVALID all the
while a third session attempts to lock it, miserably fails at it, and
then contemplates life, the universe and everything only to eventually
fail an assertion that said bit is not set. Before SKIP LOCKED that was
indeed a reasonable expectation, but alas! commit df630b0dd5ea falsified
it.
This bug is as old as time itself, and even older, if you think time
begins with the oldest supported branch. Therefore, backpatch to all
supported branches.
Author: Simon Riggs <simon.riggs@enterprisedb.com>
Discussion: https://postgr.es/m/CANbhV-FeEwMnN8yuMyss7if1ZKjOKfjcgqB26n8pqu1e=q0ebg@mail.gmail.com
M src/backend/access/heap/heapam.c
Fix index-only scan plans, take 2.
commit : d228af79d0f206f9bd24856b2c1df0909c8db1ec
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Jan 2022 15:42:27 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Jan 2022 15:42:27 -0500
Commit 4ace45677 failed to fix the problem fully, because the
same issue of attempting to fetch a non-returnable index column
can occur when rechecking the indexqual after using a lossy index
operator. Moreover, it broke EXPLAIN for such indexquals (which
indicates a gap in our test cases :-().
Revert the code changes of 4ace45677 in favor of adding a new field
to struct IndexOnlyScan, containing a version of the indexqual that
can be executed against the index-returned tuple without using any
non-returnable columns. (The restrictions imposed by check_index_only
guarantee this is possible, although we may have to recompute indexed
expressions.) Support construction of that during setrefs.c
processing by marking IndexOnlyScan.indextlist entries as resjunk
if they can't be returned, rather than removing them entirely.
(We could alternatively require setrefs.c to look up the IndexOptInfo
again, but abusing resjunk this way seems like a reasonably safe way
to avoid needing to do that.)
This solution isn't great from an API-stability standpoint: if there
are any extensions out there that build IndexOnlyScan structs directly,
they'll be broken in the next minor releases. However, only a very
invasive extension would be likely to do such a thing. There's no
change in the Path representation, so typical planner extensions
shouldn't have a problem.
As before, back-patch to all supported branches.
Discussion: https://postgr.es/m/3179992.1641150853@sss.pgh.pa.us
Discussion: https://postgr.es/m/17350-b5bdcf476e5badbb@postgresql.org
M src/backend/commands/explain.c
M src/backend/executor/nodeIndexonlyscan.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/include/nodes/execnodes.h
M src/include/nodes/plannodes.h
M src/test/regress/expected/gist.out
M src/test/regress/sql/gist.sql
pg_stat_statements: Remove obsolete comment
commit : 52d50261dbd353a34027d057ce2e0b892922fa58
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 3 Jan 2022 17:34:53 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 3 Jan 2022 17:34:53 +0900
Since 4f0b096, pgss_store() does nothing if compute_query_id is disabled
or if no other module computed a query identifier, but the top comment
of this function did not reflect that. This behavior is already
documented in its own code path, and this just removes the inconsistent
comment.
Author: Kyotaro Horiguchi
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/20211122.153823.1325120762360533122.horikyota.ntt@gmail.com
Backpatch-through: 14
M contrib/pg_stat_statements/pg_stat_statements.c
Fix typo
commit : f0c5866113fb1d85ada6dc3fe67e4909d94bacc2
author : Magnus Hagander <magnus@hagander.net>
date : Sun, 2 Jan 2022 17:03:32 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Sun, 2 Jan 2022 17:03:32 +0100
Reported-By: Eric Mutta
Backpatch-through: 10
Discussion: https://postgr.es/m/164052477973.21665.7888120874624887609@wrigleys.postgresql.org
M doc/src/sgml/high-availability.sgml
Fix index-only scan plans when not all index columns can be returned.
commit : cabea571d11e020c8c72dfb8ddced441745d7238
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 1 Jan 2022 16:12:03 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 1 Jan 2022 16:12:03 -0500
If an index has both returnable and non-returnable columns, and one of
the non-returnable columns is an expression using a Var that is in a
returnable column, then a query returning that expression could result
in an index-only scan plan that attempts to read the non-returnable
column, instead of recomputing the expression from the returnable
column as intended.
To fix, redefine the "indextlist" list of an IndexOnlyScan plan node
as containing null Consts in place of any non-returnable columns.
This solves the problem by preventing setrefs.c from falsely matching
to such entries. The executor is happy since it only cares about the
exposed types of the entries, and ruleutils.c doesn't care because a
correct plan won't reference those entries. I considered some other
ways to prevent setrefs.c from doing the wrong thing, but this way
seems good since (a) it allows a very localized fix, (b) it makes
the indextlist structure more compact in many cases, and (c) the
indextlist is now a more faithful representation of what the index AM
will actually produce, viz. nulls for any non-returnable columns.
This is easier to hit since we introduced included columns, but it's
possible to construct failing examples without that, as per the
added regression test. Hence, back-patch to all supported branches.
Per bug #17350 from Louis Jachiet.
Discussion: https://postgr.es/m/17350-b5bdcf476e5badbb@postgresql.org
M src/backend/optimizer/plan/createplan.c
M src/include/nodes/plannodes.h
M src/test/regress/expected/gist.out
M src/test/regress/sql/gist.sql
Revert b2a459edf "Fix GRANTED BY support in REVOKE ROLE statements"
commit : cb4f1be43ad2c60ea80b07d3345d8a52ca9f763f
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 30 Dec 2021 13:23:47 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 30 Dec 2021 13:23:47 +0100
The reverted commit attempted to fix SQL specification compliance for
the cases which 6aaaa76bb left. This however broke existing behavior
which takes precedence over spec compliance so revert. The introduced
tests are left after the revert since the codepath isn't well covered.
Per bug report 17346. Backpatch down to 14 where it was introduced.
Reported-by: Andrew Bille <andrewbille@gmail.com>
Discussion: https://postgr.es/m/17346-f72b28bd1a341060@postgresql.org
M src/backend/commands/user.c
M src/backend/parser/gram.y
M src/test/regress/expected/privileges.out
Fix overly generic name in with.sql test.
commit : f7d7ac23d19240a90bbb8d91cb231ef7f883dd02
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 30 Dec 2021 16:09:53 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 30 Dec 2021 16:09:53 +1300
Avoid the name "test". In the 10 branch, this could clash with
alter_table.sql, as seen in the build farm. That other instance was
already renamed in later branches by commit 2cf8c7aa, but it's good to
future-proof the name here too.
Back-patch to 10.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA%2BhUKGJf4RAXUyAYVUcQawcptX%3DnhEco3SYpuPK5cCbA-F1eLA%40mail.gmail.com
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
doc: clarify when expression indexes evaluate their expressions
commit : 2ff209953e0a7b4ce9df1c1ba15783345ef5659f
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 22 Dec 2021 16:29:16 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 22 Dec 2021 16:29:16 -0500
Only non-HOT updates evaluate the index expression.
Reported-by: Chris Lowder
Discussion: https://postgr.es/m/163967385701.26064.15365003480975321072@wrigleys.postgresql.org
Backpatch-through: 10
M doc/src/sgml/indices.sgml
Correct comment and some documentation about REPLICA_IDENTITY_INDEX
commit : 420f9ac1b77cca142d4ab7919a4f2b703c1278b8
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Dec 2021 16:38:38 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Dec 2021 16:38:38 +0900
catalog/pg_class.h was stating that REPLICA_IDENTITY_INDEX with a
dropped index is equivalent to REPLICA_IDENTITY_DEFAULT. The code tells
a different story, as it is equivalent to REPLICA_IDENTITY_NOTHING.
The behavior exists since the introduction of replica identities, and
fe7fd4e even added tests for this case but I somewhat forgot to fix this
comment.
While on it, this commit reorganizes the documentation about replica
identities on the ALTER TABLE page, and a note is added about the case
of dropped indexes with REPLICA_IDENTITY_INDEX.
Author: Michael Paquier, Wei Wang
Reviewed-by: Euler Taveira
Discussion: https://postgr.es/m/OS3PR01MB6275464AD0A681A0793F56879E759@OS3PR01MB6275.jpnprd01.prod.outlook.com
Backpatch-through: 10
M doc/src/sgml/ref/alter_table.sgml
M src/include/catalog/pg_class.h
Remove assertion for ALTER TABLE .. DETACH PARTITION CONCURRENTLY
commit : 8a22a40b2cfd1ea3218663153e5c6d7ebc68d2bf
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Dec 2021 15:38:05 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Dec 2021 15:38:05 +0900
One code path related to this flavor of ALTER TABLE was checking that
the relation to detach has to be a normal table or a partitioned table,
which would fail if using the command with a different relation kind.
Views, sequences and materialized views cannot be part of a partition
tree, so these would cause the command to fail anyway, but the assertion
was triggered. Foreign tables can be part of a partition tree, and
again the assertion would have failed. The simplest solution is just to
remove this assertion, so as we get the same failure as the
non-concurrent code path.
While on it, add a regression test in postgres_fdw for the concurrent
partition detach of a foreign table, as per a suggestion from Alexander
Lakhin.
Issue introduced in 71f4c8c.
Reported-by: Alexander Lakhin
Author: Michael Paquier, Alexander Lakhin
Reviewed-by: Peter Eisentraut, Kyotaro Horiguchi
Discussion: https://postgr.es/m/17339-a9e09aaf38a3457a@postgresql.org
Backpatch-through: 14
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/commands/tablecmds.c
Ensure casting to typmod -1 generates a RelabelType.
commit : f9a8bc9f27e6c7569f30fc86b82129647068f6a0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 16 Dec 2021 15:36:02 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 16 Dec 2021 15:36:02 -0500
Fix the code changed by commit 5c056b0c2 so that we always generate
RelabelType, not something else, for a cast to unspecified typmod.
Otherwise planner optimizations might not happen.
It appears we missed this point because the previous experiments were
done on type numeric: the parser undesirably generates a call on the
numeric() length-coercion function, but then numeric_support()
optimizes that down to a RelabelType, so that everything seems fine.
It misbehaves for types that have a non-optimized length coercion
function, such as bpchar.
Per report from John Naylor. Back-patch to all supported branches,
as the previous patch eventually was. Unfortunately, that no longer
includes 9.6 ... we really shouldn't put this type of change into a
nearly-EOL branch.
Discussion: https://postgr.es/m/CAFBsxsEfbFHEkouc+FSj+3K1sHipLPbEC67L0SAe-9-da8QtYg@mail.gmail.com
M src/backend/parser/parse_coerce.c
M src/test/regress/expected/expressions.out
M src/test/regress/sql/expressions.sql
Adjust behavior of some env settings for the TAP tests of MSVC
commit : 4ddbd7456a2d1a2f06f0860151bcbc720bf75e55
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 15 Dec 2021 10:40:08 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 15 Dec 2021 10:40:08 +0900
edc2332 has introduced in vcregress.pl some control on the environment
variables LZ4, TAR and GZIP_PROGRAM to allow any TAP tests to be able
use those commands. This makes the settings more consistent with
src/Makefile.global.in, as the same default gets used for Make and MSVC
builds.
Each parameter can be changed in buildenv.pl, but as a default gets
assigned after loading buldenv.pl, it is not possible to unset any of
these, and using an empty value would not work with "||=" either. As
some environments may not have a compatible command in their PATH (tar
coming from MinGW is an issue, for one), this could break tests without
an exit path to bypass any failing test. This commit changes things so
as the default values for LZ4, TAR and GZIP_PROGRAM are assigned before
loading buildenv.pl, not after. This way, we keep the same amount of
compatibility as a GNU build with the same defaults, and it becomes
possible to unset any of those values.
While on it, this adds some documentation about those three variables in
the section dedicated to the TAP tests for MSVC.
Per discussion with Andrew Dunstan.
Discussion: https://postgr.es/m/YbGYe483803il3X7@paquier.xyz
Backpatch-through: 10
M doc/src/sgml/install-windows.sgml
M src/tools/msvc/vcregress.pl
Fix datatype confusion in logtape.c's right_offset().
commit : 25c7faa1feeeff040242100d61b1f8914db3d3ea
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 14 Dec 2021 11:46:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 14 Dec 2021 11:46:36 -0500
This could only matter if (a) long is wider than int, and (b) the heap
of free blocks exceeds UINT_MAX entries, which seems pretty unlikely.
Still, it's a theoretical bug, so backpatch to v13 where the typo came
in (in commit c02fdc922).
In passing, also make swap_nodes() use consistent datatypes.
Ma Liangzhu
Discussion: https://postgr.es/m/17336-fc4e522d26a750fd@postgresql.org
M src/backend/utils/sort/logtape.c
Remove assertion for replication origins in PREPARE TRANSACTION
commit : 4be3e005e5da931aa816e3c04e1217c33de52fd6
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Dec 2021 10:58:25 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Dec 2021 10:58:25 +0900
When using replication origins, pg_replication_origin_xact_setup() is an
optional choice to be able to set a LSN and a timestamp to mark the
origin, which would be additionally added to WAL for transaction commits
or aborts (including 2PC transactions). An assertion in the code path
of PREPARE TRANSACTION assumed that this data should always be set, so
it would trigger when using replication origins without setting up an
origin LSN. Some tests are added to cover more this kind of scenario.
Oversight in commit 1eb6d65.
Per discussion with Amit Kapila and Masahiko Sawada.
Discussion: https://postgr.es/m/YbbBfNSvMm5nIINV@paquier.xyz
Backpatch-through: 11
M contrib/test_decoding/expected/replorigin.out
M contrib/test_decoding/sql/replorigin.sql
M src/backend/access/transam/twophase.c
Doc: de-document unimplemented geometric operators.
commit : a80fd2ae4cd4058b745959de1092f73810077779
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 13 Dec 2021 17:49:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 13 Dec 2021 17:49:36 -0500
In commit 791090bd7, I made an effort to fill in documentation
for all geometric operators listed in pg_operator. However,
it now appears that at least some of the omissions may have been
intentional, because some of those operator entries point at
unimplemented stub functions. Remove those from the docs again.
(In HEAD, poly_distance stays, because c5c192d7b just added an
implementation for it.)
Per complaint from Anton Voloshin.
Discussion: https://postgr.es/m/3426566.1638832718@sss.pgh.pa.us
M doc/src/sgml/func.sgml
isolationtester: append session name to application_name.
commit : 5b5455b035c62a710ebd6facb164112e36e9fa49
author : Andres Freund <andres@anarazel.de>
date : Mon, 13 Dec 2021 12:02:42 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 13 Dec 2021 12:02:42 -0800
When writing / debugging an isolation test it sometimes is useful to see which
session holds what lock etc. To make it easier, both as part of spec files and
interactively, append the session name to application_name. Since b1907d688
application_name already contains the test name, this appends the session's
name to that.
insert-conflict-specconflict did something like this manually, which can now
be removed.
As we have done lately with other test infrastructure improvements, backpatch
this change, to make it easier to backpatch tests.
Author: Andres Freund <andres@anarazel.de>
Reviewed-By: Michael Paquier <michael@paquier.xyz>
Reviewed-By: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/20211211012052.2blmzcmxnxqawd2z@alap3.anarazel.de
Backpatch: 10-, to make backpatching of tests easier.
M src/test/isolation/expected/insert-conflict-specconflict.out
M src/test/isolation/isolationtester.c
M src/test/isolation/specs/insert-conflict-specconflict.spec
Fix alignment in multirange_get_range() function
commit : 7615edd1db92b0bf909037480e8d9fcdb1be5f4b
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 13 Dec 2021 17:17:33 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 13 Dec 2021 17:17:33 +0300
The multirange_get_range() function fails when two boundaries of the same
range have different alignments. Fix that by adding proper pointer alignment.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/17300-dced2d01ddeb1f2f%40postgresql.org
Backpatch-through: 14
M src/backend/utils/adt/multirangetypes.c
M src/test/regress/expected/multirangetypes.out
M src/test/regress/expected/rangetypes.out
M src/test/regress/sql/multirangetypes.sql
M src/test/regress/sql/rangetypes.sql
Fix some typos with {a,an}
commit : a04fc56d03b33780612cf7630bb3f1a9388a9a02
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 9 Dec 2021 15:20:45 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 9 Dec 2021 15:20:45 +0900
One of the changes impacts the documentation, so backpatch.
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+Pu6+c+r3mY24VT7u+H+E_s6vMr5OdRiZ8NT3EOa-E5Lmw@mail.gmail.com
Backpatch-through: 14
M contrib/amcheck/verify_nbtree.c
M doc/src/sgml/ecpg.sgml
M src/backend/access/table/toast_helper.c
M src/backend/catalog/pg_depend.c
M src/backend/jit/llvm/llvmjit_error.cpp
M src/backend/replication/logical/reorderbuffer.c
Fix double publish of child table's data.
commit : 614b77d65a3859ad8d2c1c28475e98ddee06dc50
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 9 Dec 2021 08:49:50 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 9 Dec 2021 08:49:50 +0530
We publish the child table's data twice for a publication that has both
child and parent tables and is published with publish_via_partition_root
as true. This happens because subscribers will initiate synchronization
using both parent and child tables, since it gets both as separate tables
in the initial table list.
Ensure that pg_publication_tables returns only parent tables in such
cases.
Author: Hou Zhijie
Reviewed-by: Greg Nancarrow, Amit Langote, Vignesh C, Amit Kapila
Backpatch-through: 13
Discussion: https://postgr.es/m/OS0PR01MB57167F45D481F78CDC5986F794B99@OS0PR01MB5716.jpnprd01.prod.outlook.com
M src/backend/catalog/pg_publication.c
M src/test/regress/expected/publication.out
M src/test/regress/sql/publication.sql
M src/test/subscription/t/013_partition.pl
Doc: improve xfunc-c-type-table.
commit : 3d603431975b4ca920c698abd7f0083a3fabe827
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 8 Dec 2021 16:54:31 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 8 Dec 2021 16:54:31 -0500
List types numeric and timestamptz, which don't seem to have ever been
included here. Restore bigint, which was no-doubt-accidentally deleted
in v12. Fix some errors, or at least obsolete usages (nobody declares
float arguments as "float8*" anymore, even though they might be that
under the hood). Re-alphabetize. Remove the seeming claim that this
is a complete list of built-in types.
Per question from Oskar Stenberg.
Discussion: https://postgr.es/m/HE1PR03MB2971DE2527ECE1E99D6C19A8F96E9@HE1PR03MB2971.eurprd03.prod.outlook.com
M doc/src/sgml/xfunc.sgml
Revert "Check that we have a working tar before trying to use it"
commit : e4d73d089f4a1200c5d24f9d9dd014d911567f53
author : Andrew Dunstan <andrew@dunslane.net>
date : Wed, 8 Dec 2021 16:45:39 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Wed, 8 Dec 2021 16:45:39 -0500
This reverts commit f920f7e799c587228227ec94356c760e3f3d5f2b.
The patch in effect fixed a problem we didn't have and caused another
instead.
Backpatch to release 14 like original
Discussion: https://postgr.es/m/3655283.1638977975@sss.pgh.pa.us
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
Check that we have a working tar before trying to use it
commit : 90c08ed11332b766769b2e5ba114fb7ecb34e401
author : Andrew Dunstan <andrew@dunslane.net>
date : Wed, 8 Dec 2021 10:21:35 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Wed, 8 Dec 2021 10:21:35 -0500
Issue exposed by commit edc2332550 and the buildfarm.
Backpatch to release 14 where this usage started.
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
Fix origin timestamp during decoding of ROLLBACK PREPARED operation.
commit : f2e1730ee9bc1a8a3875a69a174cfb365b210821
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 8 Dec 2021 15:21:12 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 8 Dec 2021 15:21:12 +0530
This happens because we were passing incorrect arguments to
ReorderBufferFinishPrepared().
Author: Masahiko Sawada
Reviewed-by: Vignesh C
Backpatch-through: 14
Discussion: https://postgr.es/m/CAD21AoBqhUqgDZUhUVnnwKRubPDNJ6m6fJDPgok3E5cWJLL+pA@mail.gmail.com
M src/backend/replication/logical/decode.c
Fix corruption of toast indexes with REINDEX CONCURRENTLY
commit : 64ab21f0e5deb80ff67f953c6e3c8640a007a929
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 8 Dec 2021 11:01:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 8 Dec 2021 11:01:14 +0900
REINDEX CONCURRENTLY run on a toast index or a toast relation could
corrupt the target indexes rebuilt, as a backend running in parallel
that manipulates toast values would directly release the lock on the
toast relation when its local operation is done, rather than releasing
the lock once the transaction that manipulated the toast values
committed.
The fix done here is simple: we now hold a ROW EXCLUSIVE lock on the
toast relation when saving or deleting a toast value until the
transaction working on them is committed, so as a concurrent reindex
happening in parallel would be able to wait for any activity and see any
new rows inserted (or deleted).
An isolation test is added to check after the case fixed here, which is
a bit fancy by design as it relies on allow_system_table_mods to rename
the toast table and its index to fixed names. This way, it is possible
to reindex them directly without any dependency on the OID of the
underlying relation. Note that this could not use a DO block either, as
REINDEX CONCURRENTLY cannot be run in a transaction block. The test is
backpatched down to 13, where it is possible, thanks to c4a7a39, to use
allow_system_table_mods in a test suite.
Reported-by: Alexey Ermakov
Analyzed-by: Andres Freund, Noah Misch
Author: Michael Paquier
Reviewed-by: Nathan Bossart
Discussion: https://postgr.es/m/17268-d2fb426e0895abd4@postgresql.org
Backpatch-through: 12
M src/backend/access/common/toast_internals.c
A src/test/isolation/expected/reindex-concurrently-toast.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/reindex-concurrently-toast.spec
Enable settings used in TAP tests for MSVC builds
commit : 93094232c8958a42fdd066ef52d4673108e768e3
author : Andrew Dunstan <andrew@dunslane.net>
date : Tue, 7 Dec 2021 14:53:39 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Tue, 7 Dec 2021 14:53:39 -0500
Certain settings from configuration or the Makefile infrastructure are
used by the TAP tests, but were not being set up by vcregress.pl. This
remedies those omissions. This should increase test coverage, especially
on the buildfarm.
Reviewed by Noah Misch
Discussion: https://postgr.es/m/17093da5-e40d-8335-d53a-2bd803fc38b0@dunslane.net
Backpatch to all live branches.
M src/tools/msvc/vcregress.pl
On Windows, also call shutdown() while closing the client socket.
commit : ea5ecdadf6eacdc1ef5c67648ebaed2efcb0d07a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 7 Dec 2021 13:34:06 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 7 Dec 2021 13:34:06 -0500
Further experimentation shows that commit 6051857fc is not sufficient
when using (some versions of?) OpenSSL. The reason is obscure, but
calling shutdown(socket, SD_SEND) improves matters.
Per testing by Andrew Dunstan and Alexander Lakhin.
Back-patch as before.
Discussion: https://postgr.es/m/af5e0bf3-6a61-bb97-6cba-061ddf22ff6b@dunslane.net
M src/backend/libpq/pqcomm.c
Doc: Fix misleading wording of CRL parameters
commit : b6360aa46ce966f77fba290637f51f5bb2de12a9
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 3 Dec 2021 14:15:50 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 3 Dec 2021 14:15:50 +0100
ssl_crl_file and ssl_crl_dir are both used to for client certificate
revocation, not server certificates. The description for the params
could be easily misread to mean the opposite however, as evidenced
by the bugreport leading to this fix. Similarly, expand sslcrl and
and sslcrldir to explicitly mention server certificates. While there
also mention sslcrldir where previously only sslcrl was discussed.
Backpatch down to v10, with the CRL dir fixes down to 14 where they
were introduced.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/20211202.135441.590555657708629486.horikyota.ntt@gmail.com
Discussion: https://postgr.es/m/CABWY_HCBUCjY1EJHrEGePGEaSZ5b29apgTohCyygtsqe_ySYng@mail.gmail.com
Backpatch-through: 10
M doc/src/sgml/config.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/runtime.sgml
postgres_fdw: Fix unexpected reporting of empty message.
commit : 5cb86a2622f616cbacc0c4fc05a2d34e458863b6
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 3 Dec 2021 17:35:29 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 3 Dec 2021 17:35:29 +0900
pgfdw_report_error() in postgres_fdw gets a message from PGresult or
PGconn to report an error received from a remote server. Previously
if it could get a message from neither of them, it reported empty
message unexpectedly. The cause of this issue was that pgfdw_report_error()
didn't handle properly the case where no message could be obtained
and its local variable message_primary was set to '\0'.
This commit improves pgfdw_report_error() so that it reports the message
"could not obtain ..." when it gets no message and message_primary
is set to '\0'. This is the same behavior as when message_primary is NULL.
dblink_res_error() in dblink has the same issue, so this commit also
improves it in the same way.
Back-patch to all supported branches.
Author: Fujii Masao
Reviewed-by: Bharath Rupireddy
Discussion: https://postgr.es/m/477c16c8-7ea4-20fc-38d5-ed3a77ed616c@oss.nttdata.com
M contrib/dblink/dblink.c
M contrib/postgres_fdw/connection.c
On Windows, close the client socket explicitly during backend shutdown.
commit : 4cd2928543c1148278f8c1f393ee58dd72816df0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 2 Dec 2021 17:14:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 2 Dec 2021 17:14:43 -0500
It turns out that this is necessary to keep Winsock from dropping any
not-yet-sent data, such as an error message explaining the reason for
process termination. It's pretty weird that the implicit close done
by the kernel acts differently from an explicit close, but it's hard
to argue with experimental results.
Independently submitted by Alexander Lakhin and Lars Kanis (comments
by me, though). Back-patch to all supported branches.
Discussion: https://postgr.es/m/90b34057-4176-7bb0-0dbb-9822a5f6425b@greiz-reinsdorf.de
Discussion: https://postgr.es/m/16678-253e48d34dc0c376@postgresql.org
M src/backend/libpq/pqcomm.c
Move into separate file all the SQL queries used in pg_upgrade tests
commit : b6dac98b05618c22bf357e5fcf23ad2419013439
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 2 Dec 2021 10:31:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 2 Dec 2021 10:31:29 +0900
The existing pg_upgrade/test.sh and the buildfarm code have been holding
the same set of SQL queries when doing cross-version upgrade tests to
adapt the objects created by the regression tests before the upgrade
(mostly, incompatible or non-existing objects need to be dropped from
the origin, perhaps re-created).
This moves all those SQL queries into a new, separate, file with a set
of \if clauses to handle the version checks depending on the old version
of the cluster to-be-upgraded.
The long-term plan is to make the buildfarm code re-use this new SQL
file, so as committers are able to fix any compatibility issues in the
tests of pg_upgrade with a refresh of the core code, without having to
poke at the buildfarm client. Note that this is only able to handle the
main regression test suite, and that nothing is done yet for contrib
modules yet (these have more issues like their database names).
A backpatch down to 10 is done, adapting the version checks as this
script needs to be only backward-compatible, so as it becomes possible
to clean up a maximum amount of code within the buildfarm client.
Author: Justin Pryzby, Michael Paquier
Discussion: https://postgr.es/m/20201206180248.GI24052@telsasoft.com
Backpatch-through: 10
M src/bin/pg_upgrade/test.sh
A src/bin/pg_upgrade/upgrade_adapt.sql
Avoid leaking memory during large-scale REASSIGN OWNED BY operations.
commit : 8f4b0200e15a5684f6331f94caea002370754257
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 1 Dec 2021 13:44:47 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 1 Dec 2021 13:44:47 -0500
The various ALTER OWNER routines tend to leak memory in
CurrentMemoryContext. That's not a problem when they're only called
once per command; but in this usage where we might be touching many
objects, it can amount to a serious memory leak. Fix that by running
each call in a short-lived context.
(DROP OWNED BY likely has a similar issue, except that you'll probably
run out of lock table space before noticing. REASSIGN is worth fixing
since for most non-table object types, it won't take any lock.)
Back-patch to all supported branches. Unfortunately, in the back
branches this helps to only a limited extent, since the sinval message
queue bloats quite a lot in this usage before commit 3aafc030a,
consuming memory more or less comparable to what's actually leaked.
Still, it's clearly a leak with a simple fix, so we might as well fix it.
Justin Pryzby, per report from Guillaume Lelarge
Discussion: https://postgr.es/m/CAECtzeW2DAoioEGBRjR=CzHP6TdL=yosGku8qZxfX9hhtrBB0Q@mail.gmail.com
M src/backend/catalog/pg_shdepend.c
Doc: Add "Attach Partition" limitation during logical replication.
commit : 4b8eec716a524b9c9f4a65660a7006628c1d8da3
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 1 Dec 2021 10:18:27 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 1 Dec 2021 10:18:27 +0530
ATTACHing a table into a partition tree whose root is published using a
publication with publish_via_partition_root set to true does not result in
the table's existing contents being replicated. This happens because
subscriber doesn't consider replicating the newly attached partition as
the root table is already in a 'ready' state.
This behavior was introduced in PG13 (83fd4532a7) where we allowed to
publish partition changes via ancestors.
We can consider fixing this limitation in the future.
Author: Amit Langote
Reviewed-by: Hou Zhijie, Amit Kapila
Backpatch-through: 13
Discussion: https://postgr.es/m/OS0PR01MB5716E97F00732B52DC2BBC2594989@OS0PR01MB5716.jpnprd01.prod.outlook.com
M doc/src/sgml/ref/create_publication.sgml
Cope with cross-compiling when checking for a random-number source.
commit : 175edafd1f30a78643359b56c5545b5e7aabfb50
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 30 Nov 2021 17:18:04 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 30 Nov 2021 17:18:04 -0500
Commit 16f96c74d neglected to consider the possibility of cross-compiling,
causing cross-compiles to fail at the configure stage unless you'd
selected --with-openssl. Since we're now more or less assuming that
/dev/urandom is available everywhere, it seems reasonable to assume
that the cross-compile target has it too, rather than failing.
Per complaint from Vincas Dargis. Back-patch to v14 where this came in.
Discussion: https://postgr.es/m/0dc14a31-acaf-8cae-0df4-a87339b22bd9@gmail.com
M configure
M configure.ac
Fix compatibility thinko for fstat() on standard streams in win32stat.c
commit : 5550a9c385198ac51d2f3b17be2f6ff9f886d76d
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 30 Nov 2021 09:55:56 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 30 Nov 2021 09:55:56 +0900
GetFinalPathNameByHandleA() cannot be used in compilation environments
where _WIN32_WINNT < 0x0600, meaning at least Windows XP used by some
buildfarm members under MinGW that Postgres still needs to support.
This was reported as a compilation warning by the buildfarm, but this is
actually worse than the report as the code would have not worked.
Instead, this switches to GetFileInformationByHandle() that is able to
fail for standard streams and succeed for redirected ones, which is what
we are looking for herein the code emulating fstat(). We also know that
it is able to work in all the environments still supported, thanks to
the existing logic of win32stat.c.
Issue introduced by 10260c7, so backpatch down to 14.
Reported-by: Justin Pryzby, via buildfarm member jacana
Author: Michael Paquier
Reviewed-by: Juan José Santamaría Flecha
Discussion: https://postgr.es/m/20211129050122.GK17618@telsasoft.com
Backpatch-through: 14
M src/port/win32stat.c
Doc: improve documentation about ORDER BY in matviews.
commit : ce43b3dc52c90082981f2f1de5351a95fc21c405
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 29 Nov 2021 12:13:13 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 29 Nov 2021 12:13:13 -0500
Remove the confusing use of ORDER BY in an example materialized
view. It adds nothing to the example, but might encourage
people to follow bad practice. Clarify REFRESH MATERIALIZED
VIEW's note about whether view ordering is retained (it isn't).
Maciek Sakrejda
Discussion: https://postgr.es/m/CAOtHd0D-OvrUU0C=4hX28p4BaSE1XL78BAQ0VcDaLLt8tdUzsg@mail.gmail.com
M doc/src/sgml/ref/refresh_materialized_view.sgml
M doc/src/sgml/rules.sgml
Harden be-gssapi-common.h for headerscheck
commit : 1b1e4bfe7da770ad159c0f4291925a3597a65dbc
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 26 Nov 2021 17:00:29 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 26 Nov 2021 17:00:29 -0300
Surround the contents with a test that the feature is enabled by
configure, to silence header checking tools on systems without GSSAPI
installed.
Backpatch to 12, where the file appeared.
Discussion: https://postgr.es/m/202111161709.u3pbx5lxdimt@alvherre.pgsql
M src/include/libpq/be-gssapi-common.h
Document units for max_slot_wal_keep_size
commit : 2187a229410f12168d48eb41fce22ba348b2b757
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 26 Nov 2021 14:31:57 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 26 Nov 2021 14:31:57 -0300
The doc blurb failed to mention units, as well as lacking the point
about changeability.
Backpatch to 13.
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reported by: b1000101@pm.me
Discussion: https://postgr.es/m/163760291192.26193.10801700492025355788@wrigleys.postgresql.org
M doc/src/sgml/config.sgml
Fix determination of broken LSN in OVERWRITTEN_CONTRECORD
commit : d24dac9549067716f5b4d8e44ccf87dd5274bfcd
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 26 Nov 2021 11:14:27 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 26 Nov 2021 11:14:27 -0300
In commit ff9f111bce24 I mixed up inconsistent definitions of the LSN of
the first record in a page, when the previous record ends exactly at the
page boundary. The correct LSN is adjusted to skip the WAL page header;
I failed to use that when setting XLogReaderState->overwrittenRecPtr,
so at WAL replay time VerifyOverwriteContrecord would refuse to let
replay continue past that record.
Backpatch to 10. 9.6 also contains this bug, but it's no longer being
maintained.
Discussion: https://postgr.es/m/45597.1637694259@sss.pgh.pa.us
M src/backend/access/transam/xlogreader.c
Fix GRANTED BY support in REVOKE ROLE statements
commit : 371087d006e04991080bf17cf2287db38d3ea92e
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 26 Nov 2021 14:02:01 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 26 Nov 2021 14:02:01 +0100
Commit 6aaaa76bb added support for the GRANTED BY clause in GRANT and
REVOKE statements, but missed adding support for checking the role in
the REVOKE ROLE case. Fix by checking that the parsed role matches the
CURRENT_ROLE/CURRENT_USER requirement, and also add some tests for it.
Backpatch to v14 where GRANTED BY support was introduced.
Discussion: https://postgr.es/m/B7F6699A-A984-4943-B9BF-CEB84C003527@yesql.se
Backpatch-through: 14
M src/backend/commands/user.c
M src/backend/parser/gram.y
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Remove unneeded Python includes
commit : 1cc13b83eb125fb63eeb1249aa5b28d06fbd2b6c
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 25 Nov 2021 14:19:22 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 25 Nov 2021 14:19:22 +0100
Inluding <compile.h> and <eval.h> has not been necessary since Python
2.4, since they are included via <Python.h>. Morever, <eval.h> is
being removed in Python 3.11. So remove these includes.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/84884.1637723223%40sss.pgh.pa.us
M src/pl/plpython/plpython.h
Block ALTER TABLE .. DROP NOT NULL on columns in replica identity index
commit : e415916e24ccf28a88d87182e716beace9820753
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 25 Nov 2021 15:05:24 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 25 Nov 2021 15:05:24 +0900
Replica identities that depend directly on an index rely on a set of
properties, one of them being that all the columns defined in this index
have to be marked as NOT NULL. There was a hole in the logic with ALTER
TABLE DROP NOT NULL, where it was possible to remove the NOT NULL
property of a column part of an index used as replica identity, so block
it to avoid problems with logical decoding down the road.
The same check was already done columns part of a primary key, so the
fix is straight-forward.
Author: Haiying Tang, Hou Zhijie
Reviewed-by: Dilip Kumar, Michael Paquier
Discussion: https://postgr.es/m/OS0PR01MB6113338C102BEE8B2FFC5BD9FB619@OS0PR01MB6113.jpnprd01.prod.outlook.com
Backpatch-through: 10
M src/backend/commands/tablecmds.c
M src/test/regress/expected/replica_identity.out
M src/test/regress/sql/replica_identity.sql
Fix fstat() emulation on Windows with standard streams
commit : d2198b4593cca21390907a1d6762e3a0357298db
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 25 Nov 2021 12:17:05 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 25 Nov 2021 12:17:05 +0900
The emulation of fstat() in win32stat.c caused two issues with the
existing in-core callers, failing on EINVAL when using a stream as
argument:
- psql's \copy would crash when using a stream.
- pg_recvlogical would fail with -f -.
The tests in copyselect.sql from the main test suite covers the first
case, and there is a TAP test for the second case. However, in both
cases, as the standard streams are always redirected, automated tests
did not notice those issues, requiring a terminal on Windows to be
reproducible.
This issue has been introduced in bed9075, and the origin of the problem
is that GetFileInformationByHandle() does not work directly on streams,
so this commit adds an extra code path to emulate and return a set of
stats that match best with the reality. Note that redirected streams
rely on handles that can be queried with GetFileInformationByHandle(),
but we can rely on GetFinalPathNameByHandleA() to detect this case.
Author: Dmitry Koval, Juan José Santamaría Flecha
Discussion: https://postgr.es/m/17288-6b58a91025a8a8a3@postgresql.org
Backpatch-through: 14
M src/port/win32stat.c
Doc: improve documentation about nextval()/setval().
commit : 6d1bdd5e0712c0c3460d68dc2175f842530b1f83
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 24 Nov 2021 13:37:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 24 Nov 2021 13:37:12 -0500
Clarify that the results of nextval and setval are not guaranteed
persistent until the calling transaction commits. Some people
seem to have drawn the opposite conclusion from the statement that
these functions are never rolled back, so re-word to avoid saying
it quite that way.
Discussion: https://postgr.es/m/CAKU4AWohO=NfM-4KiZWvdc+z3c1C9FrUBR6xnReFJ6sfy0i=Lw@mail.gmail.com
M doc/src/sgml/func.sgml
Fix missing space in docs.
commit : 88482d75cce5f3527ebad1cf42ec59a355f4b033
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 24 Nov 2021 18:32:56 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 24 Nov 2021 18:32:56 +0200
Author: Japin Li
Discussion: https://www.postgresql.org/message-id/MEYP282MB1669C36E5F733C2EFBDCB80BB6619@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
M doc/src/sgml/arch-dev.sgml
Flush Memoize cache when non-key parameters change, take 2
commit : c2dc7b9e15039ce90c0a631ff291feb81d2c8051
author : David Rowley <drowley@postgresql.org>
date : Wed, 24 Nov 2021 23:29:56 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 24 Nov 2021 23:29:56 +1300
It's possible that a subplan below a Memoize node contains a parameter
from above the Memoize node. If this parameter changes then cache entries
may become out-dated due to the new parameter value.
Previously Memoize was mistakenly not aware of this. We fix this here by
flushing the cache whenever a parameter that's not part of the cache
key changes.
Bug: #17213
Reported by: Elvis Pranskevichus
Author: David Rowley
Discussion: https://postgr.es/m/17213-988ed34b225a2862@postgresql.org
Backpatch-through: 14, where Memoize was added
M src/backend/executor/nodeMemoize.c
M src/backend/nodes/bitmapset.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/clauses.c
M src/include/nodes/execnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/clauses.h
M src/test/regress/expected/memoize.out
M src/test/regress/sql/memoize.sql
Add support for Visual Studio 2022 in build scripts
commit : 0e681fa458750b08a5c7885820320fb85cd9ddf1
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 24 Nov 2021 13:03:55 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 24 Nov 2021 13:03:55 +0900
Documentation and any code paths related to VS are updated to keep the
whole consistent. Similarly to 2017 and 2019, the version of VS and the
version of nmake that we use to determine which code paths to use for
the build are still inconsistent in their own way.
Backpatch down to 10, so as buildfarm members are able to use this new
version of Visual Studio on all the stable branches supported.
Author: Hans Buschmann
Discussion: https://postgr.es/m/1633101364685.39218@nidsa.net
Backpatch-through: 10
M doc/src/sgml/install-windows.sgml
M src/tools/msvc/MSBuildProject.pm
M src/tools/msvc/README
M src/tools/msvc/Solution.pm
M src/tools/msvc/VSObjectFactory.pm
Revert "Flush Memoize cache when non-key parameters change"
commit : 7933bc0d13616844ed66271942eac12c3a7f0488
author : David Rowley <drowley@postgresql.org>
date : Wed, 24 Nov 2021 15:28:34 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 24 Nov 2021 15:28:34 +1300
This reverts commit f94edb06ab60545a0e9e90aedc39e3b81b2b9e3f.
M src/backend/executor/nodeMemoize.c
M src/backend/nodes/bitmapset.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/clauses.c
M src/include/nodes/execnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/clauses.h
M src/test/regress/expected/memoize.out
M src/test/regress/sql/memoize.sql
Flush Memoize cache when non-key parameters change
commit : f94edb06ab60545a0e9e90aedc39e3b81b2b9e3f
author : David Rowley <drowley@postgresql.org>
date : Wed, 24 Nov 2021 14:57:07 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 24 Nov 2021 14:57:07 +1300
It's possible that a subplan below a Memoize node contains a parameter
from above the Memoize node. If this parameter changes then cache entries
may become out-dated due to the new parameter value.
Previously Memoize was mistakenly not aware of this. We fix this here by
flushing the cache whenever a parameter that's not part of the cache
key changes.
Bug: #17213
Reported by: Elvis Pranskevichus
Author: David Rowley
Discussion: https://postgr.es/m/17213-988ed34b225a2862@postgresql.org
Backpatch-through: 14, where Memoize was added
M src/backend/executor/nodeMemoize.c
M src/backend/nodes/bitmapset.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/clauses.c
M src/include/nodes/execnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/clauses.h
M src/test/regress/expected/memoize.out
M src/test/regress/sql/memoize.sql
Allow Memoize to operate in binary comparison mode
commit : 6c32c0977783fae217b5eaa1d22d26c96e5b0085
author : David Rowley <drowley@postgresql.org>
date : Wed, 24 Nov 2021 10:07:38 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 24 Nov 2021 10:07:38 +1300
Memoize would always use the hash equality operator for the cache key
types to determine if the current set of parameters were the same as some
previously cached set. Certain types such as floating points where -0.0
and +0.0 differ in their binary representation but are classed as equal by
the hash equality operator may cause problems as unless the join uses the
same operator it's possible that whichever join operator is being used
would be able to distinguish the two values. In which case we may
accidentally return in the incorrect rows out of the cache.
To fix this here we add a binary mode to Memoize to allow it to the
current set of parameters to previously cached values by comparing
bit-by-bit rather than logically using the hash equality operator. This
binary mode is always used for LATERAL joins and it's used for normal
joins when any of the join operators are not hashable.
Reported-by: Tom Lane
Author: David Rowley
Discussion: https://postgr.es/m/3004308.1632952496@sss.pgh.pa.us
Backpatch-through: 14, where Memoize was added
M contrib/postgres_fdw/expected/postgres_fdw.out
M src/backend/commands/explain.c
M src/backend/executor/nodeMemoize.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/pathnode.c
M src/backend/utils/adt/datum.c
M src/include/nodes/execnodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/pathnode.h
M src/include/utils/datum.h
M src/test/regress/expected/join.out
M src/test/regress/expected/memoize.out
M src/test/regress/expected/subselect.out
M src/test/regress/sql/memoize.sql
Adjust pg_dump's priority ordering for casts.
commit : 0fdf67476ccbcc60bba185fd08a90b95cd91ce2d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Nov 2021 17:16:29 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Nov 2021 17:16:29 -0500
When a stored expression depends on a user-defined cast, the backend
records the dependency as being on the cast's implementation function
--- or indeed, if there's no cast function involved but just
RelabelType or CoerceViaIO, no dependency is recorded at all. This
is problematic for pg_dump, which is at risk of dumping things in the
wrong order leading to restore failures. Given the lack of previous
reports, the risk isn't that high, but it can be demonstrated if the
cast is used in some view whose rowtype is then used as an input or
result type for some other function. (That results in the view
getting hoisted into the functions portion of the dump, ahead of
the cast.)
A logically bulletproof fix for this would require including the
cast's OID in the parsed form of the expression, whence it could be
extracted by dependency.c, and then the stored dependency would force
pg_dump to do the right thing. Such a change would be fairly invasive,
and certainly not back-patchable. Moreover, since we'd prefer that
an expression using cast syntax be equal() to one doing the same
thing by explicit function call, the cast OID field would have to
have special ignored-by-comparisons semantics, making things messy.
So, let's instead fix this by a very simple hack in pg_dump: change
the object-type priority order so that casts are initially sorted
before functions, immediately after types. This fixes the problem
in a fairly direct way for casts that have no implementation function.
For those that do, the implementation function will be hoisted to just
before the cast by the dependency sorting step, so that we still have
a valid dump order. (I'm not sure that this provides a full guarantee
of no problems; but since it's been like this for many years without
any previous reports, this is probably enough to fix it in practice.)
Per report from Дмитрий Иванов.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/CAPL5KHoGa3uvyKp6z6m48LwCnTsK+LRQ_mcA4uKGfqAVSEjV_A@mail.gmail.com
M src/bin/pg_dump/pg_dump_sort.c
Pacify perlcritic.
commit : 72842a5756857a1ce6e2b2bd7a162cdf1a279fa9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Nov 2021 15:57:31 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Nov 2021 15:57:31 -0500
Per buildfarm.
M config/check_modules.pl
Fix pg_dump --inserts mode for generated columns with dropped columns.
commit : aedc4600d84dd1ebd9386e72ac78f329b8be4804
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Nov 2021 15:25:48 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Nov 2021 15:25:48 -0500
If a table contains a generated column that's preceded by a dropped
column, dumpTableData_insert failed to account for the dropped
column, and would emit DEFAULT placeholder(s) in the wrong column(s).
This resulted in failures at restore time. The default COPY code path
did not have this bug, likely explaining why it wasn't noticed sooner.
While we're fixing this, we can be a little smarter about the
situation: (1) avoid unnecessarily fetching the values of generated
columns, (2) omit generated columns from the output, too, if we're
using --column-inserts. While these modes aren't expected to be
as high-performance as the COPY path, we might as well be as
efficient as we can; it doesn't add much complexity.
Per report from Дмитрий Иванов.
Back-patch to v12 where generated columns came in.
Discussion: https://postgr.es/m/CAPL5KHrkBniyQt5e1rafm5DdXvbgiiqfEQEJ9GjtVzN71Jj5pA@mail.gmail.com
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/t/002_pg_dump.pl
Probe $PROVE not $PERL while checking for modules needed by TAP tests.
commit : e9af18c6c5512566c409c2ff99083fc1d89ff314
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Nov 2021 12:54:52 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Nov 2021 12:54:52 -0500
Normally "prove" and "perl" come from the same Perl installation,
but we support the case where they don't (mainly because the MSys
buildfarm animals need this). In that case, AX_PROG_PERL_MODULES
is completely the wrong thing to use, because it's checking what
"perl" has. Instead, make a little TAP test script including the
required modules, and run that under "prove".
We don't need ax_prog_perl_modules.m4 at all after this change,
so remove it.
Back-patch to all supported branches, for the buildfarm's benefit.
(In v10, this also back-patches the effects of commit 264eb03aa.)
Andrew Dunstan and Tom Lane, per an observation by Noah Misch
Discussion: https://postgr.es/m/E1moZHS-0002Cu-Ei@gemulon.postgresql.org
M aclocal.m4
D config/ax_prog_perl_modules.m4
A config/check_modules.pl
M configure
M configure.ac
Add missing words in comment
commit : c985a43df379e044e0de43c41daa1ba0847de459
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 22 Nov 2021 12:38:41 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 22 Nov 2021 12:38:41 -0300
Reported by Zhihong Yu.
Discussion: https://postgr.es/m/CALNJ-vR6uZivg_XkB1zKjEXeyZDEgoYanFXB-++1kBT9yZQoUw@mail.gmail.com
M src/backend/commands/tablecmds.c
pg_receivewal, pg_recvlogical: allow canceling initial password prompt.
commit : 3bd7556bbe0ade42552e3a094c3b0fd99039854e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Nov 2021 14:13:35 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Nov 2021 14:13:35 -0500
Previously it was impossible to terminate these programs via control-C
while they were prompting for a password. We can fix that trivially
for their initial password prompts, by moving setup of the SIGINT
handler from just before to just after their initial GetConnection()
calls.
This fix doesn't permit escaping out of later re-prompts, but those
should be exceedingly rare, since the user's password or the server's
authentication setup would have to have changed meanwhile. We
considered applying a fix similar to commit 46d665bc2, but that
seemed more complicated than it'd be worth. Moreover, this way is
back-patchable, which that wasn't.
The misbehavior exists in all supported versions, so back-patch to all.
Tom Lane and Nathan Bossart
Discussion: https://postgr.es/m/747443.1635536754@sss.pgh.pa.us
M src/bin/pg_basebackup/pg_receivewal.c
M src/bin/pg_basebackup/pg_recvlogical.c
Fix SP-GiST scan initialization logic for binary-compatible cases.
commit : 6d07cbc509d83d299552c4155b3511cf71c67276
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Nov 2021 14:29:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Nov 2021 14:29:56 -0500
Commit ac9099fc1 rearranged the logic in spgGetCache() that determines
the index's attType (nominal input data type) and leafType (actual
type stored in leaf index tuples). Turns out this broke things for
the case where (a) the actual input data type is different from the
nominal type, (b) the opclass's config function leaves leafType
defaulted, and (c) the opclass has no "compress" function. (b) caused
us to assign the actual input data type as leafType, and then since
that's not attType, we complained that a "compress" function is
required. For non-polymorphic opclasses, condition (a) arises in
binary-compatible cases, such as using SP-GiST text_ops for a varchar
column, or using any opclass on a domain over its nominal input type.
To fix, use attType for leafType when the index's declared column type
is different from but binary-compatible with attType. Do this only in
the defaulted-leafType case, to avoid overriding any explicit
selection made by the opclass.
Per bug #17294 from Ilya Anfimov. Back-patch to v14.
Discussion: https://postgr.es/m/17294-8f6c7962ce877edc@postgresql.org
M src/backend/access/spgist/spgutils.c
M src/test/regress/expected/spgist.out
M src/test/regress/sql/spgist.sql
Fix parallel operations that prevent oldest xmin from advancing.
commit : ead49ebc07a7eaae4d57008dd7fd1f3d71fa1ec4
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 19 Nov 2021 09:14:09 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 19 Nov 2021 09:14:09 +0530
While determining xid horizons, we skip over backends that are running
Vacuum. We also ignore Create Index Concurrently, or Reindex Concurrently
for the purposes of computing Xmin for Vacuum. But we were not setting the
flags corresponding to these operations when they are performed in
parallel which was preventing Xid horizon from advancing.
The optimization related to skipping Create Index Concurrently, or Reindex
Concurrently operations was implemented in PG-14 but the fix is the same
for the Parallel Vacuum as well so back-patched till PG-13.
Author: Masahiko Sawada
Reviewed-by: Amit Kapila
Backpatch-through: 13
Discussion: https://postgr.es/m/CAD21AoCLQqgM1sXh9BrDFq0uzd3RBFKi=Vfo6cjjKODm0Onr5w@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/storage/ipc/procarray.c
M src/include/storage/proc.h
Use appropriate -Wno-warning switches when compiling bitcode.
commit : ed1c261a00e51f7f216911df2278827ba8cc211f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Nov 2021 14:50:13 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Nov 2021 14:50:13 -0500
We use "clang" to compile bitcode files for LLVM inlining. That might
be different from the build's main C compiler, so it needs its own set
of compiler flags. To simplify configure, we don't bother adding any
-W switches to that flag set; there's little need since the main build
will show us any warnings. However, if we don't want to see unwanted
warnings, we still have to add any -Wno-warning switches we'd normally
use with clang.
This escaped notice before commit 9ff47ea41, which tried to add
-Wno-compound-token-split-by-macro; buildfarm animals using mismatched
CC and CLANG still showed those warnings. I'm not sure why we never
saw any effects from the lack of -Wno-unused-command-line-argument
(maybe that's only activated by -Wall?). clang does not currently
support -Wno-format-truncation or -Wno-stringop-truncation, although
in the interests of future-proofing and consistency I included tests
for those.
Back-patch to v11 where we started building bitcode files.
Discussion: https://postgr.es/m/2921539.1637254619@sss.pgh.pa.us
M configure
M configure.ac
Fix quoting of ACL item in table for upgrade binary compatibility checks
commit : 048f3ee618552790125a5412b5c8a4b48a36a480
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 18 Nov 2021 12:52:56 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 18 Nov 2021 12:52:56 +0900
Per buildfarm member prion, that runs the regression tests under a role
name that uses a hyphen. Issue introduced by 835bcba.
Discussion: https://postgr.es/m/YZW4MvzCZ+hQ34vw@paquier.xyz
Backpatch-through: 12
M src/test/regress/expected/type_sanity.out
M src/test/regress/sql/type_sanity.sql
Add table to regression tests for binary-compatibility checks in pg_upgrade
commit : cf3d79aa31f2acb11a03475c33e8e962dcb935a1
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 18 Nov 2021 10:37:25 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 18 Nov 2021 10:37:25 +0900
This commit adds to the main regression test suite a table with all
the in-core data types (some exceptions apply). This table is not
dropped, so as pg_upgrade would be able to check the binary
compatibility of the types tracked in the table. If a new type is added
in core, this part of the tests would need a refresh but the tests are
designed to fail if that were to happen.
As this is useful for upgrades and that these rely on the objects
created in the regression test suite of the old version upgraded from,
a backpatch down to 12 is done, which is the last point where a binary
incompatible change has been done (7c15cef). This will hopefully be
enough to find out if something gets broken during the development of a
new version of Postgres, so as it is possible to take actions in
pg_upgrade itself in this case (like 0ccfc28 for sql_identifier).
An area that is not covered yet is related to external modules, which
may create their own types. The testing infrastructure of pg_upgrade is
not integrated yet with the external modules stored in core
(src/test/modules/ or contrib/, all use the same database name for their
tests so there would be an overlap). This could be improved in the
future.
Author: Justin Pryzby
Reviewed-by: Jacob Champion, Peter Eisentraut, Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/20201206180248.GI24052@telsasoft.com
Backpatch-through: 12
M src/test/regress/expected/sanity_check.out
M src/test/regress/expected/type_sanity.out
M src/test/regress/sql/type_sanity.sql
Clean up error handling in pg_basebackup's walmethods.c.
commit : 53c4a580e4ab90295a0a3f0814089ac978901b3f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Nov 2021 14:16:34 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Nov 2021 14:16:34 -0500
The error handling here was a mess, as a result of a fundamentally
bad design (relying on errno to keep its value much longer than is
safe to assume) as well as a lot of just plain sloppiness, both as
to noticing errors at all and as to reporting the correct errno.
Moreover, the recent addition of LZ4 compression broke things
completely, because liblz4 doesn't use errno to report errors.
To improve matters, keep the error state in the DirectoryMethodData or
TarMethodData struct, and add a string field so we can handle cases
that don't set errno. (The tar methods already had a version of this,
but it can be done more efficiently since all these cases use a
constant error string.) Make the dir and tar methods handle errors
in basically identical ways, which they didn't before.
This requires copying errno into the state struct in a lot of places,
which is a bit tedious, but it has the virtue that we can get rid of
ad-hoc code to save and restore errno in a number of places ... not
to mention that it fixes other places that should've saved/restored
errno but neglected to.
In passing, fix some pointlessly static buffers to be ordinary
local variables.
There remains an issue about exactly how to handle errors from
fsync(), but that seems like material for its own patch.
While the LZ4 problems are new, all the rest of this is fixes for
old bugs, so backpatch to v10 where walmethods.c was introduced.
Patch by me; thanks to Michael Paquier for review.
Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us
M src/bin/pg_basebackup/walmethods.c
Handle close() failures more robustly in pg_dump and pg_basebackup.
commit : 6b413b41b45fe0412ee55b29f23ce31ce894fc90
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Nov 2021 13:08:25 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Nov 2021 13:08:25 -0500
Coverity complained that applying get_gz_error after a failed gzclose,
as we did in one place in pg_basebackup, is unsafe. I think it's
right: it's entirely likely that the call is touching freed memory.
Change that to inspect errno, as we do for other gzclose calls.
Also, be careful to initialize errno to zero immediately before any
gzclose() call where we care about the error status. (There are
some calls where we don't, because we already failed at some previous
step.) This ensures that we don't get a misleadingly irrelevant
error code if gzclose() fails in a way that doesn't set errno.
We could work harder at that, but it looks to me like all such cases
are basically can't-happen if we're not misusing zlib, so it's
not worth the extra notational cruft that would be required.
Also, fix several places that simply failed to check for close-time
errors at all, mostly at some remove from the close or gzclose itself;
and one place that did check but didn't bother to report the errno.
Back-patch to v12. These mistakes are older than that, but between
the frontend logging API changes that happened in v12 and the fact
that frontend code can't rely on %m before that, the patch would need
substantial revision to work in older branches. It doesn't quite
seem worth the trouble given the lack of related field complaints.
Patch by me; thanks to Michael Paquier for review.
Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_directory.c
M src/bin/pg_dump/pg_backup_tar.c
Fix display of SQL-standard function's arguments in INSERT/SELECT.
commit : 5d5779aeaf0780210de0e51dabeee8c9ceedb0ee
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Nov 2021 11:31:31 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Nov 2021 11:31:31 -0500
If a SQL-standard function body contains an INSERT ... SELECT statement,
any function parameters referenced within the SELECT were always printed
in $N style, rather than using the parameter name if any. While not
strictly incorrect, this wasn't the intention, and it's inconsistent
with the way that such parameters would be printed in any other kind
of statement.
The cause is that the recursion to get_query_def from
get_insert_query_def neglected to pass down the context->namespaces
list, passing constant NIL instead. This is a very ancient oversight,
but AFAICT it had no visible consequences before commit e717a9a18
added an outermost namespace with function parameters. We don't allow
INSERT ... SELECT as a sub-query, except in a top-level WITH clause,
where it couldn't contain any outer references that might need to access
upper namespaces. So although that's arguably a bug, I don't see any
point in changing it before v14.
In passing, harden the code added to get_parameter by e717a9a18 so that
it won't crash if a PARAM_EXTERN Param appears in an unexpected place.
Per report from Erki Eessaar. Code fix by me, regression test case
by Masahiko Sawada.
Discussion: https://postgr.es/m/AM9PR01MB8268347BED344848555167FAFE949@AM9PR01MB8268.eurprd01.prod.exchangelabs.com
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/create_function_3.out
M src/test/regress/sql/create_function_3.sql
Doc: add see-also references to CREATE PUBLICATION.
commit : 51d4213689ff30af6db4b7ecb410358dd5c35491
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 17 Nov 2021 13:34:41 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 17 Nov 2021 13:34:41 +0100
The "See also" section on the reference page for CREATE PUBLICATION
didn't match the cross references on CREATE SUBSCRIPTION and their
ALTER counterparts. Fixed by adding an xref to the CREATE and ALTER
SUBSCRIPTION pages. Backpatch down to v10 where CREATE PUBLICATION
was introduced.
Author: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAHut+PvGWd3-Ktn96c-z6uq-8TGVVP=TPOkEovkEfntoo2mRhw@mail.gmail.com
Backpatch-through: 10
M doc/src/sgml/ref/create_publication.sgml
Invalidate relcache when changing REPLICA IDENTITY index.
commit : 232fd72a5eab3ab429497796bce6e7da9e67ca3f
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 16 Nov 2021 08:34:24 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 16 Nov 2021 08:34:24 +0530
When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache was not being invalidated. This leads to skipping update/delete
operations during apply on the subscriber side as the columns required to
search corresponding rows won't get logged.
Author: Tang Haiying, Hou Zhijie
Reviewed-by: Euler Taveira, Amit Kapila
Backpatch-through: 10
Discussion: https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/backend/commands/tablecmds.c
M src/test/subscription/t/100_bugs.pl
Make psql's \password default to CURRENT_USER, not PQuser(conn).
commit : 99389cb66bb743ef6b94edc77034bdd9bf5e9928
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Nov 2021 14:55:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Nov 2021 14:55:32 -0500
The documentation says plainly that \password acts on "the current user"
by default. What it actually acted on, or tried to, was the username
used to log into the current session. This is not the same thing if
one has since done SET ROLE or SET SESSION AUTHENTICATION. Aside from
the possible surprise factor, it's quite likely that the current role
doesn't have permissions to set the password of the original role.
To fix, use "SELECT CURRENT_USER" to get the role name to act on.
(This syntax works with servers at least back to 7.0.) Also, in
hopes of reducing confusion, include the role name that will be
acted on in the password prompt.
The discrepancy from the documentation makes this a bug, so
back-patch to all supported branches.
Patch by me; thanks to Nathan Bossart for review.
Discussion: https://postgr.es/m/747443.1635536754@sss.pgh.pa.us
M src/bin/psql/command.c
Fix memory overrun when querying pg_stat_slru
commit : 5f81a480d537956efc0b9bf2eee7d651a03c596a
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 12 Nov 2021 21:50:04 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 12 Nov 2021 21:50:04 +0900
pg_stat_get_slru() in pgstatfuncs.c would point to one element after the
end of the array PgStat_SLRUStats when finishing to scan its entries.
This had no direct consequences as no data from the extra memory area
was read, but static analyzers would rightfully complain here. So let's
be clean.
While on it, this adds one regression test in the area reserved for
system views.
Reported-by: Alexander Kozhemyakin, via AddressSanitizer
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/17280-37da556e86032070@postgresql.org
Backpatch-through: 13
M src/backend/utils/adt/pgstatfuncs.c
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/sysviews.sql
Report any XLogReadRecord() error in XlogReadTwoPhaseData().
commit : 675cd765c2a5d5f2f020a5bf647086c178612abf
author : Noah Misch <noah@leadboat.com>
date : Thu, 11 Nov 2021 17:10:18 -0800
committer: Noah Misch <noah@leadboat.com>
date : Thu, 11 Nov 2021 17:10:18 -0800
Buildfarm members kittiwake and tadarida have witnessed errors at this
site. The site discarded key facts. Back-patch to v10 (all supported
versions).
Reviewed by Michael Paquier and Tom Lane.
Discussion: https://postgr.es/m/20211107013157.GB790288@rfd.leadboat.com
M src/backend/access/transam/twophase.c
Restore lock level to set vacuum flags
commit : 9aa91cb33b70c5bc207e1a136d60f742acd67c32
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 11 Nov 2021 11:03:29 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 11 Nov 2021 11:03:29 -0300
Commit 27838981be9d mistakenly reduced the lock level from exclusive to
shared that is acquired to set PGPROC->statusFlags; this was reverted
by dcfff74fb166, but failed to do so in one spot. Fix it.
Backpatch to 14.
Noted by Andres Freund.
Discussion: https://postgr.es/m/20211111020724.ggsfhcq3krq5r4hb@alap3.anarazel.de
M src/backend/replication/slot.c
Fix buffer overrun in unicode string normalization with empty input
commit : b609db7155d9c94011a6e16d0e6a60d62e322694
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 11 Nov 2021 15:01:45 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 11 Nov 2021 15:01:45 +0900
PostgreSQL 13 and newer versions are directly impacted by that through
the SQL function normalize(), which would cause a call of this function
to write one byte past its allocation if using in input an empty
string after recomposing the string with NFC and NFKC. Older versions
(v10~v12) are not directly affected by this problem as the only code
path using normalization is SASLprep in SCRAM authentication that
forbids the case of an empty string, but let's make the code more robust
anyway there so as any out-of-core callers of this function are covered.
The solution chosen to fix this issue is simple, with the addition of a
fast-exit path if the decomposed string is found as empty. This would
only happen for an empty string as at its lowest level a codepoint would
be decomposed as itself if it has no entry in the decomposition table or
if it has a decomposition size of 0.
Some tests are added to cover this issue in v13~. Note that an empty
string has always been considered as normalized (grammar "IS NF[K]{C,D}
NORMALIZED", through the SQL function is_normalized()) for all the
operations allowed (NFC, NFD, NFKC and NFKD) since this feature has been
introduced as of 2991ac5. This behavior is unchanged but some tests are
added in v13~ to check after that.
I have also checked "make normalization-check" in src/common/unicode/,
while on it (works in 13~, and breaks in older stable branches
independently of this commit).
The release notes should just mention this commit for v13~.
Reported-by: Matthijs van der Vleuten
Discussion: https://postgr.es/m/17277-0c527a373794e802@postgresql.org
Backpatch-through: 10
M src/common/unicode_norm.c
M src/test/regress/expected/unicode.out
M src/test/regress/sql/unicode.sql
Clean up compilation warnings coming from PL/Perl with clang-12~
commit : 56eb02ce79a8c14ac4fdf3494465e3fac3d4d45b
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 11 Nov 2021 10:51:08 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 11 Nov 2021 10:51:08 +0900
clang-12 has introduced -Wcompound-token-split-by-macro, that is causing
a large amount of warnings when building PL/Perl because of its
interactions with upstream Perl. This commit adds one -Wno to CFLAGS at
./configure time if the flag is supported by the compiler to silence all
those warnings.
Upstream perl has fixed this issue, but it is going to take some time
before this is spread across the buildfarm, and we have noticed that
some animals would be useful with an extra -Werror to help with the
detection of incorrect placeholders (see b0cf544), dangomushi being
one.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/YYr3qYa/R3Gw+Sbg@paquier.xyz
Backpatch-through: 10
M configure
M configure.ac
Doc: improve protocol spec for logical replication Type messages.
commit : 74da4c71d6b1fc20c546ab042135f444dd21e734
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 10 Nov 2021 13:12:58 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 10 Nov 2021 13:12:58 -0500
protocol.sgml documented the layout for Type messages, but completely
dropped the ball otherwise, failing to explain what they are, when
they are sent, or what they're good for. While at it, do a little
copy-editing on the description of Relation messages.
In passing, adjust the comment for apply_handle_type() to make it
clearer that we choose not to do anything when receiving a Type
message, not that we think it has no use whatsoever.
Per question from Stefen Hillman.
Discussion: https://postgr.es/m/CAPgW8pMknK5pup6=T4a_UG=Cz80Rgp=KONqJmTdHfaZb0RvnFg@mail.gmail.com
M doc/src/sgml/protocol.sgml
M src/backend/replication/logical/worker.c
Fix instability in 026_overwrite_contrecord.pl test.
commit : 3aa858c8932e91f74a2e7a7c3c237b345c4f0847
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 9 Nov 2021 18:40:19 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 9 Nov 2021 18:40:19 -0500
We've seen intermittent failures in this test on slower buildfarm
machines, which I think can be explained by assuming that autovacuum
emitted some additional WAL. Disable autovacuum to stabilize it.
In passing, use stringwise not numeric comparison to compare
WAL file names. Doesn't matter at present, but they are
hex strings not decimal ...
Discussion: https://postgr.es/m/1372189.1636499287@sss.pgh.pa.us
M src/test/recovery/t/026_overwrite_contrecord.pl
Stamp 14.1.
commit : 0a455b8d61d8fc5a7d1fdc152667f9ba1fd27fda
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Nov 2021 16:58:24 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Nov 2021 16:58:24 -0500
M configure
M configure.ac
Last-minute updates for release notes.
commit : ddc6917c314ddcf16f56dd2cfd52fbc721760b07
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Nov 2021 14:02:16 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Nov 2021 14:02:16 -0500
Security: CVE-2021-23214, CVE-2021-23222
M doc/src/sgml/release-14.sgml
libpq: reject extraneous data after SSL or GSS encryption handshake.
commit : 30547d7913098502cdc93c06f77c3629af51b24c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Nov 2021 11:14:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Nov 2021 11:14:56 -0500
libpq collects up to a bufferload of data whenever it reads data from
the socket. When SSL or GSS encryption is requested during startup,
any additional data received with the server's yes-or-no reply
remained in the buffer, and would be treated as already-decrypted data
once the encryption handshake completed. Thus, a man-in-the-middle
with the ability to inject data into the TCP connection could stuff
some cleartext data into the start of a supposedly encryption-protected
database session.
This could probably be abused to inject faked responses to the
client's first few queries, although other details of libpq's behavior
make that harder than it sounds. A different line of attack is to
exfiltrate the client's password, or other sensitive data that might
be sent early in the session. That has been shown to be possible with
a server vulnerable to CVE-2021-23214.
To fix, throw a protocol-violation error if the internal buffer
is not empty after the encryption handshake.
Our thanks to Jacob Champion for reporting this problem.
Security: CVE-2021-23222
M doc/src/sgml/protocol.sgml
M src/interfaces/libpq/fe-connect.c
Reject extraneous data after SSL or GSS encryption handshake.
commit : 9d5a76b8d18d6584341f754090c3007a54215f45
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Nov 2021 11:01:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Nov 2021 11:01:43 -0500
The server collects up to a bufferload of data whenever it reads data
from the client socket. When SSL or GSS encryption is requested
during startup, any additional data received with the initial
request message remained in the buffer, and would be treated as
already-decrypted data once the encryption handshake completed.
Thus, a man-in-the-middle with the ability to inject data into the
TCP connection could stuff some cleartext data into the start of
a supposedly encryption-protected database session.
This could be abused to send faked SQL commands to the server,
although that would only work if the server did not demand any
authentication data. (However, a server relying on SSL certificate
authentication might well not do so.)
To fix, throw a protocol-violation error if the internal buffer
is not empty after the encryption handshake.
Our thanks to Jacob Champion for reporting this problem.
Security: CVE-2021-23214
M src/backend/libpq/pqcomm.c
M src/backend/postmaster/postmaster.c
M src/include/libpq/libpq.h
Translation updates
commit : 5a75612022703a198383f3226d35f78b6075ff30
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 8 Nov 2021 10:06:30 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 8 Nov 2021 10:06:30 +0100
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: f54c1d7c2c97bb2a238a149e407023a9bc007b06
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/backend/po/ru.po
M src/backend/po/sv.po
M src/bin/initdb/po/es.po
M src/bin/initdb/po/fr.po
M src/bin/initdb/po/sv.po
M src/bin/pg_amcheck/nls.mk
M src/bin/pg_amcheck/po/es.po
M src/bin/pg_amcheck/po/fr.po
M src/bin/pg_amcheck/po/ru.po
A src/bin/pg_amcheck/po/sv.po
M src/bin/pg_archivecleanup/po/fr.po
M src/bin/pg_archivecleanup/po/sv.po
M src/bin/pg_basebackup/po/fr.po
M src/bin/pg_basebackup/po/sv.po
M src/bin/pg_checksums/po/fr.po
M src/bin/pg_checksums/po/sv.po
M src/bin/pg_config/po/fr.po
M src/bin/pg_config/po/sv.po
M src/bin/pg_controldata/po/fr.po
M src/bin/pg_controldata/po/sv.po
M src/bin/pg_ctl/po/fr.po
M src/bin/pg_ctl/po/sv.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/sv.po
M src/bin/pg_resetwal/po/es.po
M src/bin/pg_resetwal/po/fr.po
M src/bin/pg_resetwal/po/sv.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_rewind/po/sv.po
M src/bin/pg_test_fsync/po/sv.po
M src/bin/pg_test_timing/nls.mk
A src/bin/pg_test_timing/po/sv.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/fr.po
M src/bin/pg_upgrade/po/sv.po
M src/bin/pg_verifybackup/po/fr.po
M src/bin/pg_verifybackup/po/sv.po
M src/bin/pg_waldump/po/fr.po
M src/bin/pg_waldump/po/sv.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/sv.po
M src/bin/scripts/po/de.po
M src/bin/scripts/po/fr.po
M src/bin/scripts/po/ru.po
M src/bin/scripts/po/sv.po
M src/interfaces/ecpg/ecpglib/po/sv.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/fr.po
M src/interfaces/ecpg/preproc/po/sv.po
M src/interfaces/libpq/po/sv.po
M src/pl/plperl/po/sv.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpgsql/src/po/ru.po
M src/pl/plpgsql/src/po/sv.po
M src/pl/plpython/po/sv.po
M src/pl/tcl/po/sv.po
Fix incorrect hash equality operator bug in Memoize
commit : 1f194ed6c26e67ce92437729fc46193cc6e913d2
author : David Rowley <drowley@postgresql.org>
date : Mon, 8 Nov 2021 14:41:13 +1300
committer: David Rowley <drowley@postgresql.org>
date : Mon, 8 Nov 2021 14:41:13 +1300
In v14, because we don't have a field in RestrictInfo to cache both the
left and right type's hash equality operator, we just restrict the scope
of Memoize to only when the left and right types of a RestrictInfo are the
same.
In master we add another field to RestrictInfo and cache both hash
equality operators.
Reported-by: Jaime Casanova
Author: David Rowley
Discussion: https://postgr.es/m/20210929185544.GB24346%40ahch-to
Backpatch-through: 14
M src/backend/optimizer/plan/initsplan.c
Release notes for 14.1, 13.5, 12.9, 11.14, 10.19, 9.6.24.
commit : bb003edbb7b1e8fb2ce488e2f047e5e1982b95fd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 7 Nov 2021 14:21:50 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 7 Nov 2021 14:21:50 -0500
M doc/src/sgml/release-14.sgml
Reset lastOverflowedXid on standby when needed
commit : b0f6bd48f35ddfe4195804eb8197f477c88da6d2
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 6 Nov 2021 18:31:21 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 6 Nov 2021 18:31:21 +0300
Currently, lastOverflowedXid is never reset. It's just adjusted on new
transactions known to be overflowed. But if there are no overflowed
transactions for a long time, snapshots could be mistakenly marked as
suboverflowed due to wraparound.
This commit fixes this issue by resetting lastOverflowedXid when needed
altogether with KnownAssignedXids.
Backpatch to all supported versions.
Reported-by: Stan Hu
Discussion: https://postgr.es/m/CAMBWrQ%3DFp5UAsU_nATY7EMY7NHczG4-DTDU%3DmCvBQZAQ6wa2xQ%40mail.gmail.com
Author: Kyotaro Horiguchi, Alexander Korotkov
Reviewed-by: Stan Hu, Simon Riggs, Nikolay Samokhvalov, Andrey Borodin, Dmitry Dolgov
M src/backend/storage/ipc/procarray.c
Second-draft release notes for 14.1.
commit : 01a11c77be05dd5254d0656418b544c8f1a8031c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 6 Nov 2021 11:56:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 6 Nov 2021 11:56:33 -0400
Add latest commits. Fix some typos and infelicitous wording
(thanks to Justin Pryzby for proof-reading).
M doc/src/sgml/release-14.sgml
Fix handling of NaN values in BRIN minmax multi
commit : f7829feb759ef54d9adcc696fa8345e280ed423b
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 6 Nov 2021 01:25:31 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 6 Nov 2021 01:25:31 +0100
When calculating distance between float4/float8 values, we need to be a
bit more careful about NaN values in order not to trigger assert. We
consider NaN values to be equal (distace 0.0) and in infinite distance
from all other values.
On builds without asserts, this issue is mostly harmless - the ranges
may be merged in less efficient order, but the index is still correct.
Per report from Andreas Seltenreich. Backpatch to 14, where this new
BRIN opclass was introduced.
Reported-by: Andreas Seltenreich
Discussion: https://postgr.es/m/87r1bw9ukm.fsf@credativ.de
M src/backend/access/brin/brin_minmax_multi.c
M src/test/regress/expected/brin_multi.out
M src/test/regress/sql/brin_multi.sql
First-draft release notes for 14.1.
commit : 39387228c8b7043d168bada0c64e9f59e83285f5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Nov 2021 20:26:18 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Nov 2021 20:26:18 -0400
As usual, the release notes for other branches will be made by cutting
these down, but put them up for community review first.
Also as usual for a .1 release, there are some entries here that
are not really relevant for v14 because they already appeared in 14.0.
Those'll be removed later.
M doc/src/sgml/release-14.sgml
Avoid crash in rare case of concurrent DROP
commit : 02e20bb2dcaa523782a8fe7de1c2ddfefaeff033
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 5 Nov 2021 12:29:35 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 5 Nov 2021 12:29:35 -0300
When a role being dropped contains is referenced by catalog objects that
are concurrently also being dropped, a crash can result while trying to
construct the string that describes the objects. Suppress that by
ignoring objects whose descriptions are returned as NULL.
The majority of relevant codesites were already cautious about this
already; we had just missed a couple.
This is an old bug, so backpatch all the way back.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/17126-21887f04508cb5c8@postgresql.org
M src/backend/catalog/dependency.c
M src/backend/catalog/pg_shdepend.c
Pipeline mode disallows multicommand strings
commit : c829fbc3dc93d43bc499499164ac42c4edb1cb70
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 5 Nov 2021 11:40:03 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 5 Nov 2021 11:40:03 -0300
... so mention that in appropriate places of the libpq docs.
Backpatch to 14.
Reported-by: RekGRpth <rekgrpth@gmail.com>
Discussion: https://postgr.es/m/17235-53bb38fc5be593dc@postgresql.org
M doc/src/sgml/libpq.sgml
Update alternative expected output file.
commit : f4e3b62710c0eb6e952f7744c6c7e647817f687d
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 3 Nov 2021 19:38:17 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 3 Nov 2021 19:38:17 +0200
Previous commit added a test to 'largeobject', but neglected the
alternative expected output file 'largeobject_1.source'. Per failure
on buildfarm animal 'hamerkop'.
Discussion: https://www.postgresql.org/message-id/DBA08346-9962-4706-92D1-230EE5201C10@yesql.se
M src/test/regress/output/largeobject_1.source
Fix snapshot reference leak if lo_export fails.
commit : 4ebd740cd3f3afe5523c5947be836edf8cccdc27
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 3 Nov 2021 10:28:52 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 3 Nov 2021 10:28:52 +0200
If lo_export() fails to open the target file or to write to it, it leaks
the created LargeObjectDesc and its snapshot in the top-transaction
context and resource owner. That's pretty harmless, it's a small leak
after all, but it gives the user a "Snapshot reference leak" warning.
Fix by using a short-lived memory context and no resource owner for
transient LargeObjectDescs that are opened and closed within one function
call. The leak is easiest to reproduce with lo_export() on a directory
that doesn't exist, but in principle the other lo_* functions could also
fail.
Backpatch to all supported versions.
Reported-by: Andrew B
Reviewed-by: Alvaro Herrera
Discussion: https://www.postgresql.org/message-id/32bf767a-2d65-71c4-f170-122f416bab7e@iki.fi
M src/backend/libpq/be-fsstubs.c
M src/backend/storage/large_object/inv_api.c
M src/test/regress/input/largeobject.source
M src/test/regress/output/largeobject.source
Fix parallel amvacuumcleanup safety bug.
commit : f6162c020c880189255e77e33bde74a2b951eaae
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 2 Nov 2021 19:52:10 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 2 Nov 2021 19:52:10 -0700
Commit b4af70cb inverted the return value of the function
parallel_processing_is_safe(), but missed the amvacuumcleanup test.
Index AMs that don't support parallel cleanup at all were affected.
The practical consequences of this bug were not very serious. Hash
indexes are affected, but since they just return the number of blocks
during hashvacuumcleanup anyway, it can't have had much impact.
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAD21AoA-Em+aeVPmBbL_s1V-ghsJQSxYL-i3JP8nTfPiD1wjKw@mail.gmail.com
Backpatch: 14-, where commit b4af70cb appears.
M src/backend/access/heap/vacuumlazy.c
Don't overlook indexes during parallel VACUUM.
commit : 61a86ed55ba169044b9a692542bad1b05341147b
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 2 Nov 2021 12:06:16 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 2 Nov 2021 12:06:16 -0700
Commit b4af70cb, which simplified state managed by VACUUM, performed
refactoring of parallel VACUUM in passing. Confusion about the exact
details of the tasks that the leader process is responsible for led to
code that made it possible for parallel VACUUM to miss a subset of the
table's indexes entirely. Specifically, indexes that fell under the
min_parallel_index_scan_size size cutoff were missed. These indexes are
supposed to be vacuumed by the leader (alongside any parallel unsafe
indexes), but weren't vacuumed at all. Affected indexes could easily
end up with duplicate heap TIDs, once heap TIDs were recycled for new
heap tuples. This had generic symptoms that might be seen with almost
any index corruption involving structural inconsistencies between an
index and its table.
To fix, make sure that the parallel VACUUM leader process performs any
required index vacuuming for indexes that happen to be below the size
cutoff. Also document the design of parallel VACUUM with these
below-size-cutoff indexes.
It's unclear how many users might be affected by this bug. There had to
be at least three indexes on the table to hit the bug: a smaller index,
plus at least two additional indexes that themselves exceed the size
cutoff. Cases with just one additional index would not run into
trouble, since the parallel VACUUM cost model requires two
larger-than-cutoff indexes on the table to apply any parallel
processing. Note also that autovacuum was not affected, since it never
uses parallel processing.
Test case based on tests from a larger patch to test parallel VACUUM by
Masahiko Sawada.
Many thanks to Kamigishi Rei for her invaluable help with tracking this
problem down.
Author: Peter Geoghegan <pg@bowt.ie>
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Reported-By: Kamigishi Rei <iijima.yun@koumakan.jp>
Reported-By: Andrew Gierth <andrew@tao11.riddles.org.uk>
Diagnosed-By: Andres Freund <andres@anarazel.de>
Bug: #17245
Discussion: https://postgr.es/m/17245-ddf06aaf85735f36@postgresql.org
Discussion: https://postgr.es/m/20211030023740.qbnsl2xaoh2grq3d@alap3.anarazel.de
Backpatch: 14-, where the refactoring commit appears.
M src/backend/access/heap/vacuumlazy.c
M src/include/commands/vacuum.h
A src/test/regress/expected/vacuum_parallel.out
M src/test/regress/parallel_schedule
A src/test/regress/sql/vacuum_parallel.sql
Fix variable lifespan in ExecInitCoerceToDomain().
commit : 16a56774fade2da2f44f9f6d6899a8113d5725fc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Nov 2021 13:36:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Nov 2021 13:36:47 -0400
This undoes a mistake in 1ec7679f1: domainval and domainnull were
meant to live across loop iterations, but they were incorrectly
moved inside the loop. The effect was only to emit useless extra
EEOP_MAKE_READONLY steps, so it's not a big deal; nonetheless,
back-patch to v13 where the mistake was introduced.
Ranier Vilela
Discussion: https://postgr.es/m/CAEudQAqXuhbkaAp-sGH6dR6Nsq7v28_0TPexHOm6FiDYqwQD-w@mail.gmail.com
M src/backend/executor/execExpr.c
Avoid O(N^2) behavior in SyncPostCheckpoint().
commit : 08cfa5981e17d9daaa861520a1d41259748732b8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Nov 2021 11:31:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Nov 2021 11:31:54 -0400
As in commits 6301c3ada and e9d9ba2a4, avoid doing repetitive
list_delete_first() operations, since that would be expensive when
there are many files waiting to be unlinked. This is a slightly
larger change than in those cases. We have to keep the list state
valid for calls to AbsorbSyncRequests(), so it's necessary to invent a
"canceled" field instead of immediately deleting PendingUnlinkEntry
entries. Also, because we might not be able to process all the
entries, we need a new list primitive list_delete_first_n().
list_delete_first_n() is almost list_copy_tail(), but it modifies the
input List instead of making a new copy. I found a couple of existing
uses of the latter that could profitably use the new function. (There
might be more, but the other callers look like they probably shouldn't
overwrite the input List.)
As before, back-patch to v13.
Discussion: https://postgr.es/m/CD2F0E7F-9822-45EC-A411-AE56F14DEA9F@amazon.com
M src/backend/nodes/list.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/parse_func.c
M src/backend/storage/sync/sync.c
M src/include/nodes/pg_list.h
Avoid some other O(N^2) hazards in list manipulation.
commit : ad87bf355214c47221263825b3a541062c71aea7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Nov 2021 16:24:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Nov 2021 16:24:40 -0400
In the same spirit as 6301c3ada, fix some more places where we were
using list_delete_first() in a loop and thereby risking O(N^2)
behavior. It's not clear that the lists manipulated in these spots
can get long enough to be really problematic ... but it's not clear
that they can't, either, and the fixes are simple enough.
As before, back-patch to v13.
Discussion: https://postgr.es/m/CD2F0E7F-9822-45EC-A411-AE56F14DEA9F@amazon.com
M contrib/pg_trgm/trgm_regexp.c
M src/backend/executor/nodeAgg.c
M src/backend/jit/llvm/llvmjit.c
Handle XLOG_OVERWRITE_CONTRECORD in DecodeXLogOp
commit : 494ec0037e03854cd2c8a3e01c3c207d2886ce01
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 1 Nov 2021 13:07:23 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 1 Nov 2021 13:07:23 -0300
Failing to do so results in inability of logical decoding to process the
WAL stream. Handle it by doing nothing.
Backpatch all the way back.
Reported-by: Petr Jelínek <petr.jelinek@enterprisedb.com>
M src/backend/replication/logical/decode.c
Preserve opclass parameters across REINDEX CONCURRENTLY
commit : f255de9a456531d917a7488ad9d084201dedcfd2
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 1 Nov 2021 11:40:22 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 1 Nov 2021 11:40:22 +0900
The opclass parameter Datums from the old index are fetched in the same
way as for predicates and expressions, by grabbing them directly from
the system catalogs. They are then copied into the new IndexInfo that
will be used for the creation of the new copy.
This caused the new index to be rebuilt with default parameters rather
than the ones pre-defined by a user. The only way to get back a new
index with correct opclass parameters would be to recreate a new index
from scratch.
The issue has been introduced by 911e702.
Author: Michael Paquier
Reviewed-by: Zhihong Yu
Discussion: https://postgr.es/m/YX0CG/QpLXcPr8HJ@paquier.xyz
Backpatch-through: 13
M src/backend/catalog/index.c
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Don't try to read a multi-GB pg_stat_statements file in one call.
commit : 7104e0b2457d1dc2d0e1c97e5053706e64a0fc50
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 31 Oct 2021 19:13:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 31 Oct 2021 19:13:48 -0400
Windows fails on a request to read() more than INT_MAX bytes,
and perhaps other platforms could have similar issues. Let's
adjust this code to read at most 1GB per call.
(One would not have thought the file could get that big, but now
we have a field report of trouble, so it can. We likely ought to
add some mechanism to limit the size of the query-texts file
separately from the size of the hash table. That is not this
patch, though.)
Per bug #17254 from Yusuke Egashira. It's been like this for
awhile, so back-patch to all supported branches.
Discussion: https://postgr.es/m/17254-a926c89dc03375c2@postgresql.org
M contrib/pg_stat_statements/pg_stat_statements.c
Avoid O(N^2) behavior when the standby process releases many locks.
commit : 8424dfced790a5c2886ac33f9ce33eb57bf99f09
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 31 Oct 2021 15:31:29 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 31 Oct 2021 15:31:29 -0400
When replaying a transaction that held many exclusive locks on the
primary, a standby server's startup process would expend O(N^2)
effort on manipulating the list of locks. This code was fine when
written, but commit 1cff1b95a made repetitive list_delete_first()
calls inefficient, as explained in its commit message. Fix by just
iterating the list normally, and releasing storage only when done.
(This'd be inadequate if we needed to recover from an error occurring
partway through; but we don't.)
Back-patch to v13 where 1cff1b95a came in.
Nathan Bossart
Discussion: https://postgr.es/m/CD2F0E7F-9822-45EC-A411-AE56F14DEA9F@amazon.com
M src/backend/storage/ipc/standby.c
Demote pg_unreachable() in heapam to an assertion.
commit : bd9f4cf0ee5336ce4f9f3103521740baccfb3825
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 29 Oct 2021 10:53:46 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 29 Oct 2021 10:53:46 -0700
Commit d168b66682, which overhauled index deletion, added a
pg_unreachable() to the end of a sort comparator used when sorting heap
TIDs from an index page. This allows the compiler to apply
optimizations that assume that the heap TIDs from the index AM must
always be unique.
That doesn't seem like a good idea now, given recent reports of
corruption involving duplicate TIDs in indexes on Postgres 14. Demote
to an assertion, just in case.
Backpatch: 14-, where index deletion was overhauled.
M src/backend/access/heap/heapam.c
Update time zone data files to tzdata release 2021e.
commit : 0c8a40b391191793afe8f1e202765a5b71bccea5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 29 Oct 2021 11:38:18 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 29 Oct 2021 11:38:18 -0400
DST law changes in Fiji, Jordan, Palestine, and Samoa. Historical
corrections for Barbados, Cook Islands, Guyana, Niue, Portugal, and
Tonga.
Also, the Pacific/Enderbury zone has been renamed to Pacific/Kanton.
The following zones have been merged into nearby, more-populous zones
whose clocks have agreed since 1970: Africa/Accra, America/Atikokan,
America/Blanc-Sablon, America/Creston, America/Curacao,
America/Nassau, America/Port_of_Spain, Antarctica/DumontDUrville,
and Antarctica/Syowa.
M src/timezone/data/tzdata.zi
M src/timezone/known_abbrevs.txt
Improve contrib/amcheck's tests for CREATE INDEX CONCURRENTLY.
commit : b1f943d2aa7eba9d7f0f11d3ba66c92f814cca78
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Oct 2021 11:45:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Oct 2021 11:45:14 -0400
Commits fdd965d07 and 3cd9c3b92 tested CREATE INDEX CONCURRENTLY by
launching two separate pgbench runs concurrently. This was needed so
that only a single client thread would run CREATE INDEX CONCURRENTLY,
avoiding deadlock between two CICs. However, there's a better way,
which is to use an advisory lock to prevent concurrent CICs. That's
better in part because the test code is shorter and more readable, but
mostly because it automatically scales things to launch an appropriate
number of CICs relative to the number of INSERT transactions.
As committed, typically half to three-quarters of the CIC transactions
were pointless because the INSERT transactions had already stopped.
In passing, remove background_pgbench, which was added to support
these tests and isn't needed anymore. We can always put it back
if we find a use for it later.
Back-patch to v12; older pgbench versions lack the
conditional-execution features needed for this method.
Tom Lane and Andrey Borodin
Discussion: https://postgr.es/m/139687.1635277318@sss.pgh.pa.us
M contrib/amcheck/t/002_cic.pl
M contrib/amcheck/t/003_cic_2pc.pl
M src/test/perl/PostgresNode.pm
doc: Fix link to SELinux user guide in sepgsql page
commit : da7d0fb159460ddb37798e91d5489c2b6da3bd0d
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 28 Oct 2021 09:26:13 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 28 Oct 2021 09:26:13 +0900
Reported-by: Anton Voloshin
Discussion: https://postgr.es/m/15a86d4e-a237-1acd-18a2-fd69730f1ab9@postgrespro.ru
Backpatch-through: 10
M doc/src/sgml/sepgsql.sgml
Fix ordering of items in nbtree error message.
commit : 6cac3433965ada5ea30fdaa4911eabd0bf59cb68
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 27 Oct 2021 13:09:01 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 27 Oct 2021 13:09:01 -0700
Oversight in commit a5213adf.
Backpatch: 13-, just like commit a5213adf.
M src/backend/access/nbtree/nbtinsert.c
Further harden nbtree posting split code.
commit : d078fe83d592618d9137e50306566c5a34ceb8aa
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 27 Oct 2021 12:10:45 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 27 Oct 2021 12:10:45 -0700
Add more defensive checks around posting list split code. These should
detect corruption involving duplicate table TIDs earlier and more
reliably than any existing check.
Follow up to commit 8f72bbac.
Discussion: https://postgr.es/m/CAH2-WzkrSY_kjyd1_M5xJK1uM0govJXMxPn8JUSvwcUOiHuWVw@mail.gmail.com
Backpatch: 13-, where nbtree deduplication was introduced.
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtsearch.c
Clarify that --system reindexes system catalogs *only*
commit : a0b6520ecfd6bd938851ec386241d2e3c689e117
author : Magnus Hagander <magnus@hagander.net>
date : Wed, 27 Oct 2021 16:20:02 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Wed, 27 Oct 2021 16:20:02 +0200
Make this more clear both in the help message and docs.
Reviewed-By: Michael Paquier
Backpatch-through: 9.6
Discussion: https://postgr.es/m/CABUevEw6Je0WUFTLhPKOk4+BoBuDrE-fKw3N4ckqgDBMFu4paA@mail.gmail.com
M doc/src/sgml/ref/reindexdb.sgml
M src/bin/scripts/reindexdb.c
Ensure that slots are zeroed before use
commit : 1ed1f801cdd114822a5f08409f23a7f22be0b9fb
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 26 Oct 2021 10:40:08 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 26 Oct 2021 10:40:08 +0200
The previous coding relied on the memory for the slots being zeroed
elsewhere, which while it was true in this case is not an contract
which is guaranteed to hold. Explicitly clear the tts_isnull array
to ensure that the slots are filled from a known state.
Backpatch to v14 where the catalog multi-inserts were introduced.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAJ7c6TP0AowkUgNL6zcAK-s5HYsVHVBRWfu69FRubPpfwZGM9A@mail.gmail.com
Backpatch-through: 14
M src/backend/catalog/heap.c
M src/backend/catalog/pg_shdepend.c
Revert "Remove unused wait events."
commit : a6a0ae127e76da58e0396cd0f21a97066da2115b
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 26 Oct 2021 08:19:33 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 26 Oct 2021 08:19:33 +0530
This reverts commit 671eb8f34404d24c8f16ae40e94becb38afd93bb. The removed
wait events are used by some extensions and removal of these would force a
recompile of those extensions. We don't want that for released branches.
Discussion: https://postgr.es/m/E1mdOBY-0005j2-QL@gemulon.postgresql.org
M doc/src/sgml/monitoring.sgml
M src/backend/utils/activity/wait_event.c
M src/include/utils/wait_event.h
Reject huge_pages=on if shared_memory_type=sysv.
commit : 181361a0c2fec635de10e101f683da9bddbb8178
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 26 Oct 2021 12:54:55 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 26 Oct 2021 12:54:55 +1300
It doesn't work (it could, but hasn't been implemented).
Back-patch to 12, where shared_memory_type arrived.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Reviewed-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/163271880203.22789.1125998876173795966@wrigleys.postgresql.org
M doc/src/sgml/config.sgml
M src/backend/port/sysv_shmem.c
Fix CREATE INDEX CONCURRENTLY for the newest prepared transactions.
commit : a5b9a0000e14663fd814dd51648a544fa40140a4
author : Noah Misch <noah@leadboat.com>
date : Sat, 23 Oct 2021 18:36:38 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 23 Oct 2021 18:36:38 -0700
The purpose of commit 8a54e12a38d1545d249f1402f66c8cde2837d97c was to
fix this, and it sufficed when the PREPARE TRANSACTION completed before
the CIC looked for lock conflicts. Otherwise, things still broke. As
before, in a cluster having used CIC while having enabled prepared
transactions, queries that use the resulting index can silently fail to
find rows. It may be necessary to reindex to recover from past
occurrences; REINDEX CONCURRENTLY suffices. Fix this for future index
builds by making CIC wait for arbitrarily-recent prepared transactions
and for ordinary transactions that may yet PREPARE TRANSACTION. As part
of that, have PREPARE TRANSACTION transfer locks to its dummy PGPROC
before it calls ProcArrayClearTransaction(). Back-patch to 9.6 (all
supported versions).
Andrey Borodin, reviewed (in earlier versions) by Andres Freund.
Discussion: https://postgr.es/m/01824242-AA92-4FE9-9BA7-AEBAFFEA3D0C@yandex-team.ru
A contrib/amcheck/t/003_cic_2pc.pl
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xact.c
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lock.c
M src/backend/utils/cache/inval.c
M src/include/access/twophase.h
M src/include/storage/lock.h
Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURRENTLY.
commit : dde966efb286ddc222f17013a8cf4561a3b67a5e
author : Noah Misch <noah@leadboat.com>
date : Sat, 23 Oct 2021 18:36:38 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 23 Oct 2021 18:36:38 -0700
CIC and REINDEX CONCURRENTLY assume backends see their catalog changes
no later than each backend's next transaction start. That failed to
hold when a backend absorbed a relevant invalidation in the middle of
running RelationBuildDesc() on the CIC index. Queries that use the
resulting index can silently fail to find rows. Fix this for future
index builds by making RelationBuildDesc() loop until it finishes
without accepting a relevant invalidation. It may be necessary to
reindex to recover from past occurrences; REINDEX CONCURRENTLY suffices.
Back-patch to 9.6 (all supported versions).
Noah Misch and Andrey Borodin, reviewed (in earlier versions) by Andres
Freund.
Discussion: https://postgr.es/m/20210730022548.GA1940096@gust.leadboat.com
A contrib/amcheck/t/002_cic.pl
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/relcache.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/include/utils/inval.h
M src/include/utils/relcache.h
M src/test/perl/PostgresNode.pm
M src/tools/pgindent/typedefs.list
doc: Describe calculation method of streaming start for pg_receivewal
commit : affec9f6a2dec174b333b8d4c43948af9d825b46
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 23 Oct 2021 14:43:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 23 Oct 2021 14:43:42 +0900
The documentation was imprecise about the starting LSN used for WAL
streaming if nothing can be found in the local archive directory
defined with the pg_receivewal command, so be more talkative on this
matter.
Extracted from a larger patch by the same author.
Author: Ronan Dunklau, Michael Paquier
Discussion: https://postgr.es/m/18708360.4lzOvYHigE@aivenronan
Backpatch-through: 10
M doc/src/sgml/ref/pg_receivewal.sgml
Fix frontend version of sh_error() in simplehash.h.
commit : 8cee4be6dc931c74870f57a7697b031838876da4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Oct 2021 16:43:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Oct 2021 16:43:38 -0400
The code does not expect sh_error() to return, but the patch
that made this header usable in frontend didn't get that memo.
While here, plaster unlikely() on the tests that decide whether
to invoke sh_error(), and add our standard copyright notice.
Noted by Andres Freund. Back-patch to v13 where this frontend
support came in.
Discussion: https://postgr.es/m/0D54435C-1199-4361-9D74-2FBDCF8EA164@anarazel.de
M src/include/lib/simplehash.h
pg_dump: fix mis-dumping of non-global default privileges.
commit : 3ad2c2455be0bd42f1ca22907850b93c3afc575b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Oct 2021 15:22:25 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Oct 2021 15:22:25 -0400
Non-global default privilege entries should be dumped as-is,
not made relative to the default ACL for their object type.
This would typically only matter if one had revoked some
on-by-default privileges in a global entry, and then wanted
to grant them again in a non-global entry.
Per report from Boris Korzun. This is an old bug, so back-patch
to all supported branches.
Neil Chen, test case by Masahiko Sawada
Discussion: https://postgr.es/m/111621616618184@mail.yandex.ru
Discussion: https://postgr.es/m/CAA3qoJnr2+1dVJObNtfec=qW4Z0nz=A9+r5bZKoTSy5RDjskMw@mail.gmail.com
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/t/002_pg_dump.pl
Add module build directory to the PATH for TAP tests
commit : 52c0c11367fdac78a59997de2b24031c0501fbe7
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 22 Oct 2021 09:11:52 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 22 Oct 2021 09:11:52 -0400
For non-MSVC builds this is make's $(CURDIR), while for MSVC builds it
is $topdir/$Config/$module. The directory is added as the second element
in the PATH, so that the install location takes precedence, but the
added PATH element takes precedence over the rest of the PATH.
The reason for this is to allow tests to find built products that are
not installed, such as the libpq_pipeline test driver.
The libpq_pipeline test is adjusted to take advantage of this.
Based on a suggestion from Andres Freund.
Backpatch to release 14.
Discussion: https://postgr.es/m/4941f5a5-2d50-1a0e-6701-14c5fefe92d6@dunslane.net
M src/Makefile.global.in
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
M src/tools/msvc/vcregress.pl
Back-patch "Add parent table name in an error in reorderbuffer.c."
commit : 9e84f6a7213314f6f6dcb66651525d1b7eda8591
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 21 Oct 2021 09:24:59 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 21 Oct 2021 09:24:59 +0530
This was originally done in commit 5e77625b26 for 15 only, as a
troubleshooting aid but multiple people showed interest in back-patching
this.
Author: Jeremy Schneider
Reviewed-by: Amit Kapila
Backpatch-through: 9.6
Discussion: https://postgr.es/m/808ed65b-994c-915a-361c-577f088b837f@amazon.com
M src/backend/replication/logical/reorderbuffer.c
Remove unused wait events.
commit : 671eb8f34404d24c8f16ae40e94becb38afd93bb
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 21 Oct 2021 08:07:08 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 21 Oct 2021 08:07:08 +0530
Commit 464824323e introduced the wait events which were neither used by
that commit nor by follow-up commits for that work.
Author: Masahiro Ikeda
Backpatch-through: 14, where it was introduced
Discussion: https://postgr.es/m/ff077840-3ab2-04dd-bbe4-4f5dfd2ad481@oss.nttdata.com
M doc/src/sgml/monitoring.sgml
M src/backend/utils/activity/wait_event.c
M src/include/utils/wait_event.h
Fix corruption of pg_shdepend when copying deps from template database
commit : 5040c96415a062a1016e0a6b9a4dc9f26a7f356e
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 21 Oct 2021 10:39:07 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 21 Oct 2021 10:39:07 +0900
Using for a new database a template database with shared dependencies
that need to be copied over was causing a corruption of pg_shdepend
because of an off-by-one computation error of the index number used for
the values inserted with a slot.
Issue introduced by e3931d0. Monitoring the rest of the code, there are
no similar mistakes.
Reported-by: Sven Klemm
Author: Aleksander Alekseev
Reviewed-by: Daniel Gustafsson, Michael Paquier
Discussion: https://postgr.es/m/CAJ7c6TP0AowkUgNL6zcAK-s5HYsVHVBRWfu69FRubPpfwZGM9A@mail.gmail.com
Backpatch-through: 14
M src/backend/catalog/pg_shdepend.c
Protect against collation variations in test
commit : 7182788552b7b7d7fca226af2ec281789b1abff3
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 20 Oct 2021 13:05:42 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 20 Oct 2021 13:05:42 -0300
Discussion: https://postgr.es/m/YW/MYdSRQZtPFBWR@paquier.xyz
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
Fix build of MSVC with OpenSSL 3.0.0
commit : 81aefaea82934e577e9d81fcb1f809a8d75bbf5c
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 20 Oct 2021 16:48:57 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 20 Oct 2021 16:48:57 +0900
The build scripts of Visual Studio would fail to detect properly a 3.0.0
build as the check on the second digit was failing. This is adjusted
where needed, allowing the builds to complete. Note that the MSIs of
OpenSSL mentioned in the documentation have not changed any library
names for Win32 and Win64, making this change straight-forward.
Reported-by: htalaco, via github
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/YW5XKYkq6k7OtrFq@paquier.xyz
Backpatch-through: 9.6
M src/tools/msvc/Solution.pm
Ensure correct lock level is used in ALTER ... RENAME
commit : 3ce3fb2f7dc66fef67c8184b96245d74372b729e
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 19 Oct 2021 19:08:45 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 19 Oct 2021 19:08:45 -0300
Commit 1b5d797cd4f7 intended to relax the lock level used to rename
indexes, but inadvertently allowed *any* relation to be renamed with a
lowered lock level, as long as the command is spelled ALTER INDEX.
That's undesirable for other relation types, so retry the operation with
the higher lock if the relation turns out not to be an index.
After this fix, ALTER INDEX <sometable> RENAME will require access
exclusive lock, which it didn't before.
Author: Nathan Bossart <bossartn@amazon.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reported-by: Onder Kalaci <onderk@microsoft.com>
Discussion: https://postgr.es/m/PH0PR21MB1328189E2821CDEC646F8178D8AE9@PH0PR21MB1328.namprd21.prod.outlook.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
Adapt src/test/ldap/t/001_auth.pl to work with openldap 2.5.
commit : 533315b68f8a00784686500f29ee780244341bc2
author : Andres Freund <andres@anarazel.de>
date : Tue, 19 Oct 2021 10:14:49 -0700
committer: Andres Freund <andres@anarazel.de>
date : Tue, 19 Oct 2021 10:14:49 -0700
ldapsearch's deprecated -h/-p arguments were removed, need to use -H now -
which has been around for over 20 years.
As perltidy insists on reflowing the parameters anyway, change order and
"phrasing" to yield a less confusing layout (per suggestion from Tom Lane).
Discussion: https://postgr.es/m/20211009233850.wvr6apcrw2ai6cnj@alap3.anarazel.de
Backpatch: 11-, where the tests were added.
M src/test/ldap/t/001_auth.pl
Fix assignment to array of domain over composite.
commit : 04dae19f4d5067b35b421dcd43175a5059747800
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Oct 2021 13:54:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Oct 2021 13:54:45 -0400
An update such as "UPDATE ... SET fld[n].subfld = whatever"
failed if the array elements were domains rather than plain
composites. That's because isAssignmentIndirectionExpr()
failed to cope with the CoerceToDomain node that would appear
in the expression tree in this case. The result would typically
be a crash, and even if we accidentally didn't crash, we'd not
correctly preserve other fields of the same array element.
Per report from Onder Kalaci. Back-patch to v11 where arrays of
domains came in.
Discussion: https://postgr.es/m/PH0PR21MB132823A46AA36F0685B7A29AD8BD9@PH0PR21MB1328.namprd21.prod.outlook.com
M src/backend/executor/execExpr.c
M src/test/regress/expected/domain.out
M src/test/regress/sql/domain.sql
Remove bogus assertion in transformExpressionList().
commit : f627fd547a34bb3ffba5763a1ab404c2777debe4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Oct 2021 11:35:15 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Oct 2021 11:35:15 -0400
I think when I added this assertion (in commit 8f889b108), I was only
thinking of the use of transformExpressionList at top level of INSERT
and VALUES. But it's also called by transformRowExpr(), which can
certainly occur in an UPDATE targetlist, so it's inappropriate to
suppose that p_multiassign_exprs must be empty. Besides, since the
input is not expected to contain ResTargets, there's no reason it
should contain MultiAssignRefs either. Hence this code need not
be concerned about the state of p_multiassign_exprs, and we should
just drop the assertion.
Per bug #17236 from ocean_li_996. It's been wrong for years,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/17236-3210de9bcba1d7ca@postgresql.org
M src/backend/parser/parse_target.c
Fix bug in TOC file error message printing
commit : 3e2f32b01d3b6868022c8381b28e8fc5399baeb9
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 19 Oct 2021 12:59:54 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 19 Oct 2021 12:59:54 +0200
If the blob TOC file cannot be parsed, the error message was failing
to print the filename as the variable holding it was shadowed by the
destination buffer for parsing. When the filename fails to parse,
the error will print an empty string:
./pg_restore -d foo -F d dump
pg_restore: error: invalid line in large object TOC file "": ..
..instead of the intended error message:
./pg_restore -d foo -F d dump
pg_restore: error: invalid line in large object TOC file "dump/blobs.toc": ..
Fix by renaming both variables as the shared name was too generic to
store either and still convey what the variable held.
Backpatch all the way down to 9.6.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/A2B151F5-B32B-4F2C-BA4A-6870856D9BDE@yesql.se
Backpatch-through: 9.6
M src/bin/pg_dump/pg_backup_directory.c
Fix sscanf limits in pg_basebackup and pg_dump
commit : 121be6a665aaf64e0fe45b424cd26b53b384dc31
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 19 Oct 2021 12:59:50 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 19 Oct 2021 12:59:50 +0200
Make sure that the string parsing is limited by the size of the
destination buffer.
In pg_basebackup the available values sent from the server
is limited to two characters so there was no risk of overflow.
In pg_dump the buffer is bounded by MAXPGPATH, and thus the limit
must be inserted via preprocessor expansion and the buffer increased
by one to account for the terminator. There is no risk of overflow
here, since in this case, the buffer scanned is smaller than the
destination buffer.
Backpatch the pg_basebackup fix to 11 where it was introduced, and
the pg_dump fix all the way down to 9.6.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/B14D3D7B-F98C-4E20-9459-C122C67647FB@yesql.se
Backpatch-through: 11 and 9.6
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_dump/pg_backup_directory.c
Block ALTER INDEX/TABLE index_name ALTER COLUMN colname SET (options)
commit : b1b797ec71a1a8cc6bb500313f4373bcb5aac1ff
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 19 Oct 2021 11:04:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 19 Oct 2021 11:04:00 +0900
The grammar of this command run on indexes with column names has always
been authorized by the parser, and it has never been documented.
Since 911e702, it is possible to define opclass parameters as of CREATE
INDEX, which actually broke the old case of ALTER INDEX/TABLE where
relation-level parameters n_distinct and n_distinct_inherited could be
defined for an index (see 76a47c0 and its thread where this point has
been touched, still remained unused). Attempting to do that in v13~
would cause the index to become unusable, as there is a new dedicated
code path to load opclass parameters instead of the relation-level ones
previously available. Note that it is possible to fix things with a
manual catalog update to bring the relation back online.
This commit disables this command for now as the use of column names for
indexes does not make sense anyway, particularly when it comes to index
expressions where names are automatically computed. One way to properly
support this case properly in the future would be to use column numbers
when it comes to indexes, in the same way as ALTER INDEX .. ALTER COLUMN
.. SET STATISTICS.
Partitioned indexes were already blocked, but not indexes. Some tests
are added for both cases.
There was some code in ANALYZE to enforce n_distinct to be used for an
index expression if the parameter was defined, but just remove it for
now until/if there is support for this (note that index-level parameters
never had support in pg_dump either, previously), so this was just dead
code.
Reported-by: Matthijs van der Vleuten
Author: Nathan Bossart, Michael Paquier
Reviewed-by: Vik Fearing, Dilip Kumar
Discussion: https://postgr.es/m/17220-15d684c6c2171a83@postgresql.org
Backpatch-through: 13
M src/backend/commands/analyze.c
M src/backend/commands/tablecmds.c
M src/test/regress/expected/btree_index.out
M src/test/regress/sql/btree_index.sql
Invalidate partitions of table being attached/detached
commit : 72d0642172571be7b44d54728deb609d7285d9d0
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 18 Oct 2021 19:08:25 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 18 Oct 2021 19:08:25 -0300
Failing to do that, any direct inserts/updates of those partitions
would fail to enforce the correct constraint, that is, one that
considers the new partition constraint of their parent table.
Backpatch to 10.
Reported by: Hou Zhijie <houzj.fnst@fujitsu.com>
Author: Amit Langote <amitlangote09@gmail.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Nitin Jadhav <nitinjadhavpostgres@gmail.com>
Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com>
Discussion: https://postgr.es/m/OS3PR01MB5718DA1C4609A25186D1FBF194089%40OS3PR01MB5718.jpnprd01.prod.outlook.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
Reset properly snapshot export state during transaction abort
commit : 5b353aaff69c6099aebd45f1a63cb38b9a7ef75c
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 18 Oct 2021 11:56:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 18 Oct 2021 11:56:48 +0900
During a replication slot creation, an ERROR generated in the same
transaction as the one creating a to-be-exported snapshot would have
left the backend in an inconsistent state, as the associated static
export snapshot state was not being reset on transaction abort, but only
on the follow-up command received by the WAL sender that created this
snapshot on replication slot creation. This would trigger inconsistency
failures if this session tried to export again a snapshot, like during
the creation of a replication slot.
Note that a snapshot export cannot happen in a transaction block, so
there is no need to worry resetting this state for subtransaction
aborts. Also, this inconsistent state would very unlikely show up to
users. For example, one case where this could happen is an
out-of-memory error when building the initial snapshot to-be-exported.
Dilip found this problem while poking at a different patch, that caused
an error in this code path for reasons unrelated to HEAD.
Author: Dilip Kumar
Reviewed-by: Michael Paquier, Zhihong Yu
Discussion: https://postgr.es/m/CAFiTN-s0zA1Kj0ozGHwkYkHwa5U0zUE94RSc_g81WrpcETB5=w@mail.gmail.com
Backpatch-through: 9.6
M src/backend/access/transam/xact.c
M src/backend/replication/logical/snapbuild.c
M src/include/replication/snapbuild.h
Avoid core dump in pg_dump when dumping from pre-8.3 server.
commit : 3e4c8db931e1a9b760b22382f7c9e0e28e732a2c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 16 Oct 2021 15:02:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 16 Oct 2021 15:02:55 -0400
Commit f0e21f2f6 missed adding a tgisinternal output column
to getTriggers' query for pre-8.3 servers. Back-patch to v11,
like that commit.
M src/bin/pg_dump/pg_dump.c
Make pg_dump acquire lock on partitioned tables that are to be dumped.
commit : b5152e3ba688e7023f0e5f1a1b82b77bedadaafa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 16 Oct 2021 12:23:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 16 Oct 2021 12:23:57 -0400
It was clearly the intent to do so all along, but the original coding
fat-fingered this by checking the wrong array element. We fixed it
in passing in 403a3d91c, but that later got reverted, and we forgot
to keep this bug fix.
Most of the time this'd be relatively harmless, since once we lock
any of the partitioned table's leaf partitions, that would suffice
to prevent major DDL on the partitioned table itself. However, a
childless partitioned table would get dumped with no relevant lock
whatsoever, possibly allowing dump failure or inconsistent output.
Unlike 403a3d91c, there are no versioning concerns, since every server
version that has partitioned tables will allow you to lock one.
Back-patch to v10 where partitioned tables were introduced.
Discussion: https://postgr.es/m/1018205.1634346327@sss.pgh.pa.us
M src/bin/pg_dump/pg_dump.c
Fix PostgresNode install_path sanity tests that fail on Windows
commit : c697d8a39b9001281aa62249afb01d4ab5e63703
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 15 Oct 2021 12:56:29 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 15 Oct 2021 12:56:29 -0400
Backpatch to 14 where install_path was introduced.
M src/test/perl/PostgresNode.pm
Remove unstable pg_amcheck tests.
commit : 5863115e4cb16bd905ed78a481457c5ff92ebef4
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 14 Oct 2021 14:50:25 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 14 Oct 2021 14:50:25 -0700
Recent pg_amcheck bugfix commit d2bf06db added a test case that the
buildfarm has shown to be non-portable. It doesn't particularly seem
worth keeping anyway. Remove it.
Discussion: https://postgr.es/m/CAH2-Wz=7HKJ9WzAh7+M0JfwJ1yfT9qoE+KPa3P7iGToPOtGhXg@mail.gmail.com
Backpatch: 14-, just like the original commit.
D src/bin/pg_amcheck/t/006_bad_targets.pl
Check criticalSharedRelcachesBuilt in GetSharedSecurityLabel().
commit : 0b90f1c4c329f2be8ede10722a99a856020c0169
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 14 Oct 2021 12:24:22 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 14 Oct 2021 12:24:22 -0700
An extension may want to call GetSecurityLabel() on a shared object
before the shared relcaches are fully initialized. For instance, a
ClientAuthentication_hook might want to retrieve the security label on
a role.
Discussion: https://postgr.es/m/ecb7af0b26e3be1d96d291c8453a86f1f82d9061.camel@j-davis.com
Backpatch-through: 9.6
M src/backend/commands/seclabel.c
Fix planner error with pulling up subquery expressions into function RTEs.
commit : fd059ac2e461af922c7a18a7ac1cab8ad56c6353
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 14 Oct 2021 12:43:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 14 Oct 2021 12:43:43 -0400
If a function-in-FROM laterally references the output of some sub-SELECT
earlier in the FROM clause, and we are able to flatten that sub-SELECT
into the outer query, the expression(s) copied into the function RTE
missed being processed by eval_const_expressions. This'd lead to trouble
and probable crashes at execution if such expressions contained
named-argument function call syntax or functions with defaulted arguments.
The bug is masked if the query contains any explicit JOIN syntax, which
may help explain why we'd not noticed.
Per bug #17227 from Bernd Dorn. This is an oversight in commit 7266d0997,
so back-patch to v13 where that came in.
Discussion: https://postgr.es/m/17227-5a28ed1512189fa4@postgresql.org
M src/backend/optimizer/plan/planner.c
M src/test/regress/expected/rangefuncs.out
M src/test/regress/sql/rangefuncs.sql
Change recently added test code for stability
commit : 79c7fe1af82d42211baf37e8dc865288b251cc6d
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 13 Oct 2021 18:49:27 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 13 Oct 2021 18:49:27 -0300
The test code added with ff9f111bce24 fails under valgrind, and probably
other slow cases too, because if (say) autovacuum runs in between and
produces WAL of its own, the large INSERT fails to account for that in
the LSN calculations. Rewrite to use a DO loop.
Per complaint from Andres Freund
Backpatch to all branches.
Discussion: https://postgr.es/m/20211013180338.5guyqzpkcisqugrl@alap3.anarazel.de
M src/test/recovery/t/026_overwrite_contrecord.pl
pg_amcheck: avoid unhelpful verification attempts.
commit : dd58194cf563c9d69821162861506d5d9acceddd
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 13 Oct 2021 14:08:11 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 13 Oct 2021 14:08:11 -0700
Avoid calling contrib/amcheck functions with relations that are
unsuitable for checking. Specifically, don't attempt verification of
temporary relations, or indexes whose pg_index entry indicates that the
index is invalid, or not ready.
These relations are not supported by any of the contrib/amcheck
functions, for reasons that are pretty fundamental. For example, the
implementation of REINDEX CONCURRENTLY can add its own "transient"
pg_index entries, which has rather unclear implications for the B-Tree
verification functions, at least in the general case -- so they just
treat it as an error. It falls to the amcheck caller (in this case
pg_amcheck) to deal with the situation at a higher level.
pg_amcheck now simply treats these conditions as additional "visibility
concerns" when it queries system catalogs. This is a little arbitrary.
It seems to have the least problems among any of the available
alternatives.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Reported-By: Alexander Lakhin <exclusion@gmail.com>
Reviewed-By: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Robert Haas <robertmhaas@gmail.com>
Bug: #17212
Discussion: https://postgr.es/m/17212-34dd4a1d6bba98bf@postgresql.org
Backpatch: 14-, where pg_amcheck was introduced.
M doc/src/sgml/ref/pg_amcheck.sgml
M src/bin/pg_amcheck/pg_amcheck.c
A src/bin/pg_amcheck/t/006_bad_targets.pl
postgres_fdw: Move comments about elog level in (sub)abort cleanup.
commit : 419d27b1a28ae0b8ebb465610bcc67f36e78adfd
author : Etsuro Fujita <efujita@postgresql.org>
date : Wed, 13 Oct 2021 19:00:01 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Wed, 13 Oct 2021 19:00:01 +0900
The comments were misplaced when adding postgres_fdw. Fix that by
moving the comments to more appropriate functions.
Author: Etsuro Fujita
Backpatch-through: 9.6
Discussion: https://postgr.es/m/CAPmGK164sAXQtC46mDFyu6d-T25Mzvh5qaRNkit06VMmecYnOA%40mail.gmail.com
M contrib/postgres_fdw/connection.c
Fix use-after-free with multirange types in CREATE TYPE
commit : 922e15c47647af1b15af0a92742c8af69e83c823
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 13 Oct 2021 16:38:15 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 13 Oct 2021 16:38:15 +0900
The code was freeing the name of the multirange type function stored in
the parse tree but it should not do that. Event triggers could for
example look at such a corrupted parsed tree with a ddl_command_end
event.
Author: Alex Kozhemyakin, Sergey Shinderuk
Reviewed-by: Peter Eisentraut, Michael Paquier
Discussion: https://postgr.es/m/d5042d46-b9cd-6efb-219a-71ed0cf45bc8@postgrespro.ru
Backpatch-through: 14
M src/backend/commands/typecmds.c
Fix tests of pg_upgrade across different major versions
commit : f4e1c8892b9e90a9d3ccae21db04a1215b9312a5
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 13 Oct 2021 09:22:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 13 Oct 2021 09:22:00 +0900
This fixes a set of issues that cause different breakages or annoyances
when using pg_upgrade's test.sh to do upgrades across different major
versions:
- test.sh is completely broken when using v14 as new version because of
the removal of testtablespace/ as Makefile rule. Older versions of
pg_regress don't support --make-tablespacedir, blocking the creation of
the tablespace. In order to fix that, it is simple enough to create
those directories in the script itself, but only do that when an old
version is involved. This fix is needed on HEAD and REL_14_STABLE.
- The script would fail when using PG <= v11 as old version because of
WITH OIDS relations not supported in v12. In order to fix this, this
steals a method from the buildfarm that uses a DO block to change all
the relations marked as WITH OIDS, allowing pg_upgrade to pass. This is
more portable than using ALTER TABLE queries on the relations causing
issues. This is fixed down to v12, and authored originally by Andrew
Dunstan.
- Not using --extra-float-digits=0 with v11 as old version causes
a lot of diffs in the dumps, making the whole unreadable. This gets
only done when using v11 as old version. This is fixed down to v12.
The buildfarm code uses that already.
Note that the addition of --wal-segsize and --allow-group-access breaks
the script when using v10 or older at initdb time as these got added in
11. 10 would be EOL'd next year and nobody has complained about those
problems yet, so nothing is done about that. This means that this
commit fixes upgrade tests using test.sh with v11 as minimum older
version, up to HEAD, and that it is enough to apply this change down to
12. The old and new dumps still generate diffs, still require manual
checks, and more could be done to reduce the noise, but this allows the
tests to run with a rather minimal amount of them.
I have tested this commit and test.sh with v11 as minimum across all the
branches where this is applied. Note that this commit has no impact on
the normal pg_upgrade test run with a simple "make check".
Author: Justin Pryzby, Andrew Dunstan, Michael Paquier
Discussion: https://postgr.es/m/20201206180248.GI24052@telsasoft.com
Backpatch-through: 12
M src/bin/pg_upgrade/test.sh
Doc: normalize vacuum_multixact_failsafe_age ID.
commit : 070c402b4a5d3cc40b3551d42763aea9afb6288d
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 12 Oct 2021 10:59:22 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 12 Oct 2021 10:59:22 -0700
Author: Pavel Luzanov <p.luzanov@postgrespro.ru>
Discussion: https://postgr.es/m/c71a3cfc-a267-3d9f-1b44-fbd668d0ab10@postgrespro.ru
Backpatch: 14-, where the failsafe was introduced.
M doc/src/sgml/config.sgml
M doc/src/sgml/release-14.sgml
Add more $Test::Builder::Level in the TAP tests
commit : d834ebcf23208b3ae2109c0cae9af077202a27a4
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 12 Oct 2021 11:16:20 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 12 Oct 2021 11:16:20 +0900
Incrementing the level of the call stack reported is useful for
debugging purposes as it allows to control which part of the test is
exactly failing, especially if a test is structured with subroutines
that call routines from Test::More.
This adds more incrementations of $Test::Builder::Level where debugging
gets improved (for example it does not make sense for some paths like
pg_rewind where long subroutines are used).
A note is added to src/test/perl/README about that, based on a
suggestion from Andrew Dunstan and a wording coming from both of us.
Usage of Test::Builder::Level has spread in 12, so a backpatch down to
this version is done.
Reviewed-by: Andrew Dunstan, Peter Eisentraut, Daniel Gustafsson
Discussion: https://postgr.es/m/YV1CCFwgM1RV1LeS@paquier.xyz
Backpatch-through: 12
M contrib/amcheck/t/001_verify_heapam.pl
M contrib/test_decoding/t/001_repl_stats.pl
M src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
M src/bin/pg_verifybackup/t/005_bad_manifest.pl
M src/bin/psql/t/010_tab_completion.pl
M src/test/kerberos/t/001_auth.pl
M src/test/perl/README
M src/test/recovery/t/001_stream_rep.pl
M src/test/recovery/t/003_recovery_targets.pl
M src/test/recovery/t/007_sync_rep.pl
M src/test/recovery/t/009_twophase.pl
M src/test/recovery/t/018_wal_optimize.pl
Make autovacuum launcher more responsive to pg_log_backend_memory_contexts().
commit : 62e821ad28e1f08ea9734d7338bdebd783228a1c
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 12 Oct 2021 09:50:17 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 12 Oct 2021 09:50:17 +0900
Previously when pg_log_backend_memory_contexts() sent the request to
the autovacuum launcher, it could take more than several seconds to
log its memory contexts. Because the function (HandleAutoVacLauncherInterrupts)
to process any new interrupts that autovacuum launcher received
didn't handle the request for logging of memory contexts. This commit changes
the function so that it handles the request, to make autovacuum launcher
more responsitve to pg_log_backend_memory_contexts().
Back-patch to v14 where pg_log_backend_memory_contexts() was added.
Author: Koyu Tanigawa
Reviewed-by: Bharath Rupireddy, Atsushi Torikoshi
Discussion: https://postgr.es/m/0aae3e074face409b35153451be5cc11@oss.nttdata.com
M src/backend/postmaster/autovacuum.c
amcheck: Skip unlogged relations in Hot Standby.
commit : e7712155ea08c2f86dc32dccb5df2b230733be31
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 11 Oct 2021 17:21:46 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 11 Oct 2021 17:21:46 -0700
Have verify_heapam.c treat unlogged relations as if they were simply
empty when in Hot Standby mode. This brings it in line with
verify_nbtree.c, which has handled unlogged relations in the same way
since bugfix commit 6754fe65a4. This was an oversight in commit
866e24d47d, which extended contrib/amcheck to check heap relations.
In passing, lower the verbosity used when reporting that a relation has
been skipped like this, from NOTICE to DEBUG1. This is appropriate
because the skipping behavior is only an implementation detail, needed
to work around the fact that unlogged tables don't have smgr-level
storage for their main fork when in Hot Standby mode.
Affected unlogged relations should be considered "trivially verified",
not skipped over. They are verified in the same sense that a totally
empty relation can be verified. This behavior seems least surprising
overall, since unlogged relations on a replica will initially be empty
if and when the replica is promoted and Hot Standby ends.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Reviewed-By: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-Wzk_pukOFY7JmdiFLsrz+Pd3V8OwgC1TH2Vd5BH5ZgK4bA@mail.gmail.com
Backpatch: 14-, where heapam verification was introduced.
M contrib/amcheck/verify_heapam.c
M contrib/amcheck/verify_nbtree.c
Fix EXPLAIN of SEARCH BREADTH FIRST queries some more.
commit : 2c25db32eedb9696c4b900c70ebef03683babf24
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Oct 2021 11:56:52 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Oct 2021 11:56:52 -0400
Commit 3f50b8263 had an oversight: formerly, to deparse expressions
attached to a plan node, it was only necessary to update the
deparse_namespace ancestors list alongside calling set_deparse_plan.
Now it's necessary to update the ancestors list *first*, because
set_deparse_plan consults it, and one call site got that wrong.
This error was masked in most cases because explain.c uses just one
List object for the ancestors list, updating it in-place as the plan
is scanned, so that we accidentally had the right List assigned to
dpns->ancestors before it was needed. It would fail only if a
WorkTableScan node were the first one that we tried to deparse a
subexpression of.
Per report from Markus Winand. Like the previous patch,
back-patch to v14.
Discussion: https://postgr.es/m/648B0505-AA57-42C2-A2DA-E551DE46FA15@winand.at
M src/backend/utils/adt/ruleutils.c
postgres_fdw: Fix comments in connection.c.
commit : 2051fd30a4fa3b62cdb244f5b9748c1cb3b39845
author : Etsuro Fujita <efujita@postgresql.org>
date : Thu, 7 Oct 2021 18:15:01 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Thu, 7 Oct 2021 18:15:01 +0900
Commit 27e1f1456 missed updating some comments.
Reviewed-by: Bharath Rupireddy
Backpatch-through: 14
Discussion: https://postgr.es/m/CAPmGK15Q2Nm6U%2Ba_GwskrWFEVBZ9_3VKOvRrprGufpx91M_3Sw%40mail.gmail.com
M contrib/postgres_fdw/connection.c
Add missing word to comment in joinrels.c.
commit : ef2e107f548cfbd22060e92c0f435616cac2f57c
author : Etsuro Fujita <efujita@postgresql.org>
date : Thu, 7 Oct 2021 17:45:01 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Thu, 7 Oct 2021 17:45:01 +0900
Author: Amit Langote
Backpatch-through: 13
Discussion: https://postgr.es/m/CA%2BHiwqGQNbtamQ_9DU3osR1XiWR4wxWFZurPmN6zgbdSZDeWmw%40mail.gmail.com
M src/backend/optimizer/path/joinrels.c
Fix null-pointer crash in postgres_fdw's conversion_error_callback.
commit : 12ff678e1d657fc94a1cfa90f1b85dd9bd79e1e6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Oct 2021 15:50:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Oct 2021 15:50:24 -0400
Commit c7b7311f6 adjusted conversion_error_callback to always use
information from the query's rangetable, to avoid doing catalog lookups
in an already-failed transaction. However, as a result of the utterly
inadequate documentation for make_tuple_from_result_row, I failed to
realize that fsstate could be NULL in some contexts. That led to a
crash if we got a conversion error in such a context. Fix by falling
back to the previous coding when fsstate is NULL. Improve the
commentary, too.
Per report from Andrey Borodin. Back-patch to 9.6, like the previous
patch.
Discussion: https://postgr.es/m/08916396-55E4-4D68-AB3A-BD6066F9E5C0@yandex-team.ru
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
Fix corner-case loss of precision in numeric_power().
commit : 8e26b868d5cfe9c746578aa3b172b6f24a5a310c
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Wed, 6 Oct 2021 13:19:25 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Wed, 6 Oct 2021 13:19:25 +0100
This fixes a loss of precision that occurs when the first input is
very close to 1, so that its logarithm is very small.
Formerly, during the initial low-precision calculation to estimate the
result weight, the logarithm was computed to a local rscale that was
capped to NUMERIC_MAX_DISPLAY_SCALE (1000). However, the base may be
as close as 1e-16383 to 1, hence its logarithm may be as small as
1e-16383, and so the local rscale needs to be allowed to exceed 16383,
otherwise all precision is lost, leading to a poor choice of rscale
for the full-precision calculation.
Fix this by removing the cap on the local rscale during the initial
low-precision calculation, as we already do in the full-precision
calculation. This doesn't change the fact that the initial calculation
is a low-precision approximation, computing the logarithm to around 8
significant digits, which is very fast, especially when the base is
very close to 1.
Patch by me, reviewed by Alvaro Herrera.
Discussion: https://postgr.es/m/CAEZATCV-Ceu%2BHpRMf416yUe4KKFv%3DtdgXQAe5-7S9tD%3D5E-T1g%40mail.gmail.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Fix warning in TAP test of pg_verifybackup
commit : ae254356f94af220841f3c59dcd4d820449ca0fc
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 6 Oct 2021 13:28:30 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 6 Oct 2021 13:28:30 +0900
Oversight in a3fcbcd.
Reported-by: Thomas Munro
Discussion: https://postgr.es/m/CA+hUKGKnajZEwe91OTjro9kQLCMGGFHh2vvFn8tgHgbyn4bF9w@mail.gmail.com
Backpatch-through: 13
M src/bin/pg_verifybackup/t/007_wal.pl
Doc: improve description of UNION/INTERSECT/EXCEPT syntax.
commit : cb8a5a588eb7325d17780a7a8871ae3b49adc497
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 5 Oct 2021 10:24:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 5 Oct 2021 10:24:14 -0400
queries.sgml failed to mention the rather important point that
INTERSECT binds more tightly than UNION or EXCEPT. I thought
it could also use more discussion of the role of parentheses
in these constructs.
Per gripe from Christopher Painter-Wakefield.
Discussion: https://postgr.es/m/163338891727.12510.3939775743980651160@wrigleys.postgresql.org
M doc/src/sgml/queries.sgml
doc: remove URL for ICU explorer/locexp
commit : 1f00a2902bf422cb15fc66145f1e349fdb97689b
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 4 Oct 2021 17:10:59 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 4 Oct 2021 17:10:59 -0400
The old URL was HTTP 404 and the git link didn't build. Also update two
other ICU links. If we ever get a good link we will add it back.
Reported-by: Anton Voloshin
Author: Laurenz Albe
Backpatch-through: 10
M doc/src/sgml/charset.sgml
Fix TestLib::slurp_file() with offset on windows.
commit : c4465cd09e3ab034ced25e623d5760e9ce437f6c
author : Andres Freund <andres@anarazel.de>
date : Mon, 4 Oct 2021 13:28:06 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 4 Oct 2021 13:28:06 -0700
3c5b0685b921 used setFilePointer() to set the position of the filehandle, but
passed the wrong filehandle, always leaving the position at 0. Instead of just
fixing that, remove use of setFilePointer(), we have a perl fd at this point,
so we can just use perl's seek().
Additionally, the perl filehandle wasn't closed, just the windows filehandle.
Reviewed-By: Andrew Dunstan <andrew@dunslane.net>
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20211003173038.64mmhgxctfqn7wl6@alap3.anarazel.de
Backpatch: 9.6-, like 3c5b0685b921
M src/test/perl/TestLib.pm
Update our mapping of Windows time zone names some more.
commit : 919c08d909f766bb51c5c617714364a91bb90d9d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Oct 2021 14:52:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Oct 2021 14:52:17 -0400
Per discussion, let's just follow CLDR's default zone mappings
faithfully. There are two changes here that are clear improvements:
* Mapping "Greenwich Standard Time" to Atlantic/Reykjavik is actually
a better fit than using London, because Iceland hasn't observed DST
since 1968, so this is more nearly what people might expect.
* Since the "Samoa" zone is specified to be UTC+13:00, we must map
it to Pacific/Apia not Pacific/Samoa; the latter refers to American
Samoa which is now on the other side of the date line.
The rest of these changes look like they're choosing the most populous
IANA zone as representative. Whatever the details, we're just going
to say "if you don't like this mapping, complain to CLDR".
Discussion: https://postgr.es/m/3266414.1633045628@sss.pgh.pa.us
M src/bin/initdb/findtimezone.c
Doc: fix minor issues in GiST support function documentation.
commit : 5f46180070d108f2f2aec6a9d3d488d3ef55b42e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Oct 2021 13:34:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Oct 2021 13:34:31 -0400
gist.sgml and xindex.sgml hadn't been fully updated for the
addition of a sortsupport support function (commit 16fa9b2b3).
xindex.sgml also missed that the compress and decompress support
functions are optional, an apparently far older oversight.
In passing, fix gratuitous inconsistencies in wording and
capitalization.
Noted by E. Rogov. Back-patch to v14; the residual issues
before that aren't significant enough to bother with.
Discussion: https://postgr.es/m/163335322905.12519.5711557029494638051@wrigleys.postgresql.org
M doc/src/sgml/gist.sgml
M doc/src/sgml/xindex.sgml
Fix snapshot builds during promotion of hot standby node with 2PC
commit : 828f7f0009edda3f78dd789102cd02a394f94938
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 4 Oct 2021 14:05:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 4 Oct 2021 14:05:48 +0900
Some specific logic is done at the end of recovery when involving 2PC
transactions:
1) Call RecoverPreparedTransactions(), to recover the state of 2PC
transactions into memory (re-acquire locks, etc.).
2) ShutdownRecoveryTransactionEnvironment(), to move back to normal
operations, mainly cleaning up recovery locks and KnownAssignedXids
(including any 2PC transaction tracked previously).
3) Switch XLogCtl->SharedRecoveryState to RECOVERY_STATE_DONE, which is
the tipping point for any process calling RecoveryInProgress() to check
if the cluster is still in recovery or not.
Any snapshot taken between steps 2) and 3) would be empty, causing any
transaction relying on a snapshot at this point to potentially corrupt
data as there could still be some 2PC transactions to track, with
RecentXmin moving backwards on successive calls to GetSnapshotData() in
the same transaction.
As SharedRecoveryState is the point to take into account to know if it
is safe to discard KnownAssignedXids, this commit moves step 2) after
step 3), so as we can never finish with empty snapshots.
This exists since the introduction of hot standby, so backpatch all the
way down. The window with incorrect snapshots is extremely small, but I
have seen it when running 023_pitr_prepared_xact.pl, as did buildfarm
member fairywren. Thomas Munro also found it independently. Special
thanks to Andres Freund for taking the time to analyze this issue.
Reported-by: Thomas Munro, Michael Paquier
Analyzed-by: Andres Freund
Discussion: https://postgr.es/m/20210422203603.fdnh3fu2mmfp2iov@alap3.anarazel.de
Backpatch-through: 9.6
M src/backend/access/transam/xlog.c
Fix checking of query type in plpgsql's RETURN QUERY command.
commit : e0eba586b1b8448639ab4170fe39270d73dc2e2a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 3 Oct 2021 13:21:20 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 3 Oct 2021 13:21:20 -0400
Prior to v14, we insisted that the query in RETURN QUERY be of a type
that returns tuples. (For instance, INSERT RETURNING was allowed,
but not plain INSERT.) That happened indirectly because we opened a
cursor for the query, so spi.c checked SPI_is_cursor_plan(). As a
consequence, the error message wasn't terribly on-point, but at least
it was there.
Commit 2f48ede08 lost this detail. Instead, plain RETURN QUERY
insisted that the query be a SELECT (by checking for SPI_OK_SELECT)
while RETURN QUERY EXECUTE failed to check the query type at all.
Neither of these changes was intended.
The only convenient place to check this in the EXECUTE case is inside
_SPI_execute_plan, because we haven't done parse analysis until then.
So we need to pass down a flag saying whether to enforce that the
query returns tuples. Fortunately, we can squeeze another boolean
into struct SPIExecuteOptions without an ABI break, since there's
padding space there. (It's unlikely that any extensions would
already be using this new struct, but preserving ABI in v14 seems
like a smart idea anyway.)
Within spi.c, it seemed like _SPI_execute_plan's parameter list
was already ridiculously long, and I didn't want to make it longer.
So I thought of passing SPIExecuteOptions down as-is, allowing that
parameter list to become much shorter. This makes the patch a bit
more invasive than it might otherwise be, but it's all internal to
spi.c, so that seems fine.
Per report from Marc Bachmann. Back-patch to v14 where the
faulty code came in.
Discussion: https://postgr.es/m/1F2F75F0-27DF-406F-848D-8B50C7EEF06A@gmail.com
M doc/src/sgml/spi.sgml
M src/backend/executor/spi.c
M src/include/executor/spi.h
M src/pl/plpgsql/src/pl_exec.c
M src/test/regress/expected/plpgsql.out
M src/test/regress/sql/plpgsql.sql
Update our mapping of Windows time zone names using CLDR info.
commit : fa8db48791c58444ab35d6f2419074a717fbb9b7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 2 Oct 2021 16:05:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 2 Oct 2021 16:05:42 -0400
This corrects a bunch of entries in win32_tzmap[], and adds a few
new ones, based on the CLDR project's windowsZones.xml file.
Non-cosmetic changes fall into four main categories:
* Flat-out errors:
US/Aleutan doesn't exist
America/Salvador doesn't exist
Asia/Baku is wrong for Yerevan
Asia/Dhaka (Bangladesh) is wrong for Astana (Kazakhstan)
Europe/Bucharest is wrong for Chisinau
America/Mexico_City is wrong for Chetumal
America/Buenos_Aires is wrong for Cayenne
America/Caracas has its own zone, so poor fit for La Paz
US/Eastern is wrong for Haiti
US/Eastern is wrong for Indiana (East)
Asia/Karachi is wrong for Tashkent
Etc/UTC+12 doesn't exist
Signs of Etc/GMT zones were backwards
* Judgment calls:
(These changes follow CLDR's choices, except for the first one)
Use Europe/London for "Greenwich Standard Time", since that seems much
more likely than Africa/Casablanca to be what people will think that
zone name means. CLDR has Atlantic/Reykjavik here, but that's no better.
Asia/Shanghai seems a better fit than Hong Kong for "China Standard
Time".
Europe/Sarajevo is now a link to Belgrade, ie "Central Europe Standard
Time"; so use Warsaw for "Central European Standard Time".
America/Sao_Paulo seems more representative than Araguaina for
"E. South America Standard Time".
Africa/Johannesburg seems more representative than Harare for
"South Africa Standard Time".
* New Windows zone names:
"Israel Standard Time"
"Kaliningrad Standard Time"
"Russia Time Zone N" for various N
"Singapore Standard Time"
"South Sudan Standard Time"
"W. Central Africa Standard Time"
"West Bank Standard Time"
"Yukon Standard Time"
Some of these replace older spellings, but I kept the older spellings
too in case our code runs on a machine with the older data.
* Replace aliases (tzdb Links) with underlying city-named zones:
(This tracks tzdb's longstanding practice, and reduces inconsistency
with the rest of the entries, as well as with CLDR.)
US/Alaska
Asia/Kuwait
Asia/Muscat
Canada/Atlantic
Australia/Canberra
Canada/Saskatchewan
US/Central
US/Eastern
US/Hawaii
US/Mountain
Canada/Newfoundland
US/Pacific
Back-patch to all supported branches, as is our usual practice for
time zone data updates.
Discussion: https://postgr.es/m/3266414.1633045628@sss.pgh.pa.us
M src/bin/initdb/findtimezone.c
Re-alphabetize the win32_tzmap[] array.
commit : 81464999bc92ebf9b813e1c7bd8b333e29511fb5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 2 Oct 2021 16:05:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 2 Oct 2021 16:05:10 -0400
The original intent seems to have been to sort case-insensitively
by the Windows zone name, but various changes over the years did
not get that memo. This commit just moves a few entries to
restore exact alphabetic order, to ease comparison to the outputs
of processing scripts.
Back-patch to all supported branches, as is our usual practice for
time zone data updates.
Discussion: https://postgr.es/m/3266414.1633045628@sss.pgh.pa.us
M src/bin/initdb/findtimezone.c
Reference test binary using TESTDIR in 001_libpq_pipeline.pl.
commit : 0baa33da0fd123aff215c505116dc60bc04dae90
author : Andres Freund <andres@anarazel.de>
date : Wed, 29 Sep 2021 18:02:32 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 29 Sep 2021 18:02:32 -0700
The previous approach didn't really work on windows, due to the PATH separator
being ';' not ':'. Instead of making the PATH change more complicated,
reference the binary using the TESTDIR environment.
Reported-By: Andres Freund <andres@anarazel.de>
Suggested-By: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/20210930214040.odkdd42vknvzifm6@alap3.anarazel.de
Backpatch: 14-, where the test was introduced.
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
Error out if SKIP LOCKED and WITH TIES are both specified
commit : 20047609d39cc4d30d6b266ed3a8b418b3ce5f78
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 1 Oct 2021 18:29:18 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 1 Oct 2021 18:29:18 -0300
Both bugs #16676[1] and #17141[2] illustrate that the combination of
SKIP LOCKED and FETCH FIRST WITH TIES break expectations when it comes
to rows returned to other sessions accessing the same row. Since this
situation is detectable from the syntax and hard to fix otherwise,
forbid for now, with the potential to fix in the future.
[1] https://postgr.es/m/16676-fd62c3c835880da6@postgresql.org
[2] https://postgr.es/m/17141-913d78b9675aac8e@postgresql.org
Backpatch-through: 13, where WITH TIES was introduced
Author: David Christensen <david.christensen@crunchydata.com>
Discussion: https://postgr.es/m/CAOxo6XLPccCKru3xPMaYDpa+AXyPeWFs+SskrrL+HKwDjJnLhg@mail.gmail.com
M doc/src/sgml/ref/select.sgml
M src/backend/commands/matview.c
M src/backend/parser/gram.y
M src/test/regress/expected/limit.out
M src/test/regress/sql/limit.sql
Remove unstable, unnecessary test; fix typo
commit : 0ce67bce00717d8934b599ab4409e28b33c612ea
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 1 Oct 2021 18:03:11 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 1 Oct 2021 18:03:11 -0300
Commit ff9f111bce24 added some test code that's unportable and doesn't
add meaningful coverage. Remove it rather than try and get it to work
everywhere.
While at it, fix a typo in a log message added by the aforementioned
commit.
Backpatch to 14.
Discussion: https://postgr.es/m/3000074.1632947632@sss.pgh.pa.us
M src/backend/access/transam/xlog.c
M src/test/recovery/t/026_overwrite_contrecord.pl
D src/test/recovery/t/idiosyncratic_copy
Fix memory leak in pg_hmac
commit : a5e83ad79c282421f32224c5152d6182de34da35
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 1 Oct 2021 22:47:05 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 1 Oct 2021 22:47:05 +0200
The intermittent h buffer was not freed, causing it to leak. Backpatch
through 14 where HMAC was refactored to the current API.
Author: Sergey Shinderuk <s.shinderuk@postgrespro.ru>
Discussion: https://postgr.es/m/af07e620-7e28-a742-4637-2bc44aa7c2be@postgrespro.ru
Backpatch-through: 14
M src/common/hmac.c
Avoid believing incomplete MCV-only stats in get_variable_range().
commit : a54509bfd952085a24224480ff74d5cbd558c407
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 1 Oct 2021 14:59:35 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 1 Oct 2021 14:59:35 -0400
get_variable_range() would incautiously believe that statistics
containing only an MCV list are sufficient to derive a range estimate.
That's okay for an enum-like column that contains only MCVs, but
otherwise the estimate could be pretty bad. Make it report that the
range is indeterminate unless the MCVs plus nullfrac account for
the whole table.
I don't think this needs a dedicated test case, since a quick code
coverage check verifies that the existing regression tests traverse
all the alternatives. There is room to doubt that a future-proof
test case could be built anyway, given that the submitted example
accidentally doesn't fail before v11.
Per bug #17207 from Simon Perepelitsa. Back-patch to v10.
In principle this has been broken all along, but I'm hesitant to
make such changes in 9.6, since if anyone is unhappy with 9.6.24's
behavior there will be no second chance to fix it.
Discussion: https://postgr.es/m/17207-5265aefa79e333b4@postgresql.org
M src/backend/utils/adt/selfuncs.c
Fix Portal snapshot tracking to handle subtransactions properly.
commit : e6adaa1795d593edd616b9541fff0920637f9721
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 1 Oct 2021 11:10:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 1 Oct 2021 11:10:12 -0400
Commit 84f5c2908 forgot to consider the possibility that
EnsurePortalSnapshotExists could run inside a subtransaction with
lifespan shorter than the Portal's. In that case, the new active
snapshot would be popped at the end of the subtransaction, leaving
a dangling pointer in the Portal, with mayhem ensuing.
To fix, make sure the ActiveSnapshot stack entry is marked with
the same subtransaction nesting level as the associated Portal.
It's certainly safe to do so since we won't be here at all unless
the stack is empty; hence we can't create an out-of-order stack.
Let's also apply this logic in the case where PortalRunUtility
sets portalSnapshot, just to be sure that path can't cause similar
problems. It's slightly less clear that that path can't create
an out-of-order stack, so add an assertion guarding it.
Report and patch by Bertrand Drouvot (with kibitzing by me).
Back-patch to v11, like the previous commit.
Discussion: https://postgr.es/m/ff82b8c5-77f4-3fe7-6028-fcf3303e82dd@amazon.com
M src/backend/access/transam/xact.c
M src/backend/tcop/pquery.c
M src/backend/utils/mmgr/portalmem.c
M src/backend/utils/time/snapmgr.c
M src/include/utils/portal.h
M src/include/utils/snapmgr.h
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/sql/plpgsql_transaction.sql
Doc: Move pg_stat_replication_slots view to "Collected Statistics Views" section.
commit : 8de4a31720c2c2ce55f73f4af4b3cdc6cdcca225
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 1 Oct 2021 08:31:41 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 1 Oct 2021 08:31:41 +0530
Commit 9868167500 added pg_stat_replication_slots view to monitor
ReorderBuffer stats but mistakenly added it under
"Dynamic Statistics Views" section in the docs whereas it belongs to
"Collected Statistics Views" section.
Author: Amit Kapila
Reviewed-by: Masahiko Sawada
Backpatch-through: 14, where it was introduced
Discussion: https://postgr.es/m/CAA4eK1Kb5ur=OC-G4cAsqPOjoVe+S8LNw1WmUY8Owasjk8o5WQ@mail.gmail.com
M doc/src/sgml/monitoring.sgml
Remove gratuitous environment dependency in 002_types.pl test.
commit : afc6081f6ea8fa053dafe14e609d1d8e885410f8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 30 Sep 2021 16:23:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 30 Sep 2021 16:23:10 -0400
Computing related timestamps by subtracting "N days" is sensitive
to the prevailing timezone, since we interpret that as "same local
time on the N'th prior day". Even though the intervals in question
are only two to four days, through remarkable bad luck they managed
to cross the end of Ramadan in 2014, causing the test's output to
change if timezone is set to Africa/Casablanca. (Maybe in other
Muslim areas as well; I didn't check.) There's absolutely no reason
for this test to exercise interval subtraction, so just get rid of
that and use plain timestamptz constants representing the intended
values.
Per report from Andres Freund. Back-patch to v10 where this test
script came in.
Discussion: https://postgr.es/m/20210930183641.7lh4jhvpipvromca@alap3.anarazel.de
M src/test/subscription/t/002_types.pl
Repair two portability oversights of new test
commit : e3731bac52cf049bed965aa4f96cb073ed044b68
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 30 Sep 2021 10:01:03 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 30 Sep 2021 10:01:03 -0300
First, as pointed out by Tom Lane and Michael Paquier, I failed to
realize that Windows' PostgresNode needs an extra pg_hba.conf line
(added by PostgresNode->set_replication_conf, called internally by
->init() when 'allows_streaming=>1' is given -- but I purposefully
omitted that). I think a good fix should be to have nodes with only
'has_archiving=>1' set up for replication too, but that's a bigger
discussion. Fix it by calling ->set_replication_conf, which is not
unprecedented, as pointed out by Andrew Dunstan.
I also forgot to uncomment a ->finish() call for a pumpable IPC::Run
file descriptor. Apparently this is innocuous in almost all platforms.
Backpatch to 14. The older branches were added this file too, but not
this particular part of the test.
Discussion: https://postgr.es/m/3000074.1632947632@sss.pgh.pa.us
Discussion: https://postgr.es/m/YVT7qwhR8JmC2kfz@paquier.xyz
M src/test/recovery/t/026_overwrite_contrecord.pl
Fix WAL replay in presence of an incomplete record
commit : 64a8687a68914aa3f5a0867885777a1294eceb1c
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 29 Sep 2021 11:21:51 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 29 Sep 2021 11:21:51 -0300
Physical replication always ships WAL segment files to replicas once
they are complete. This is a problem if one WAL record is split across
a segment boundary and the primary server crashes before writing down
the segment with the next portion of the WAL record: WAL writing after
crash recovery would happily resume at the point where the broken record
started, overwriting that record ... but any standby or backup may have
already received a copy of that segment, and they are not rewinding.
This causes standbys to stop following the primary after the latter
crashes:
LOG: invalid contrecord length 7262 at A8/D9FFFBC8
because the standby is still trying to read the continuation record
(contrecord) for the original long WAL record, but it is not there and
it will never be. A workaround is to stop the replica, delete the WAL
file, and restart it -- at which point a fresh copy is brought over from
the primary. But that's pretty labor intensive, and I bet many users
would just give up and re-clone the standby instead.
A fix for this problem was already attempted in commit 515e3d84a0b5, but
it only addressed the case for the scenario of WAL archiving, so
streaming replication would still be a problem (as well as other things
such as taking a filesystem-level backup while the server is down after
having crashed), and it had performance scalability problems too; so it
had to be reverted.
This commit fixes the problem using an approach suggested by Andres
Freund, whereby the initial portion(s) of the split-up WAL record are
kept, and a special type of WAL record is written where the contrecord
was lost, so that WAL replay in the replica knows to skip the broken
parts. With this approach, we can continue to stream/archive segment
files as soon as they are complete, and replay of the broken records
will proceed across the crash point without a hitch.
Because a new type of WAL record is added, users should be careful to
upgrade standbys first, primaries later. Otherwise they risk the standby
being unable to start if the primary happens to write such a record.
A new TAP test that exercises this is added, but the portability of it
is yet to be seen.
This has been wrong since the introduction of physical replication, so
backpatch all the way back. In stable branches, keep the new
XLogReaderState members at the end of the struct, to avoid an ABI
break.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Nathan Bossart <bossartn@amazon.com>
Discussion: https://postgr.es/m/202108232252.dh7uxf6oxwcy@alvherre.pgsql
M src/backend/access/rmgrdesc/xlogdesc.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/include/access/xlog_internal.h
M src/include/access/xlogreader.h
M src/include/catalog/pg_control.h
A src/test/recovery/t/026_overwrite_contrecord.pl
A src/test/recovery/t/idiosyncratic_copy
M src/tools/pgindent/typedefs.list
doc: PG 14 relnotes, improve cache invalidation wording
commit : 4f2c75316b2b767a838aa9fefb6e4944ace34f23
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 29 Sep 2021 10:27:53 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 29 Sep 2021 10:27:53 -0400
Reported-by: Simon Riggs (privately)
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
pgbench: Fix handling of socket errors during benchmark.
commit : 8231c500ed742431263052cd48fbb859d7f7d7ea
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 29 Sep 2021 21:01:10 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 29 Sep 2021 21:01:10 +0900
Previously socket errors such as invalid socket or socket wait method failures
during benchmark caused pgbench to exit with status 0. Instead, errors during
the run should result in exit status 2.
Back-patch to v12 where pgbench started reporting exit status.
Original complaint and patch by Hayato Kuroda.
Author: Yugo Nagata, Fabien COELHO
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/TYCPR01MB5870057375ACA8A73099C649F5349@TYCPR01MB5870.jpnprd01.prod.outlook.com
M src/bin/pgbench/pgbench.c
pgbench: Correct log level of message output when socket wait method fails.
commit : 8021334d3710cb030ed1b6de764a0bb2233e5265
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 29 Sep 2021 20:35:00 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 29 Sep 2021 20:35:00 +0900
The failure of socket wait method like "select()" doesn't terminate pgbench.
So the log level of error message when that failure happens should be ERROR.
But previously FATAL was used in that case.
Back-patch to v13 where pgbench started using common logging API.
Author: Yugo Nagata, Fabien COELHO
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/20210617005934.8bd37bf72efd5f1b38e6f482@sraoss.co.jp
M src/bin/pgbench/pgbench.c
Clarify use of "statistics objects" in the code
commit : 2cf9cf5d7b5e66202389618a4c08945da352b35c
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 29 Sep 2021 15:29:45 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 29 Sep 2021 15:29:45 +0900
The code inconsistently used "statistic object" or "statistics" where
the correct term, as discussed, is actually "statistics object". This
improves the state of the code to be more consistent.
While on it, fix an incorrect error message introduced in a4d75c8. This
error should never happen, as the code states, but it would be
misleading.
Author: Justin Pryzby
Reviewed-by: Álvaro Herrera, Michael Paquier
Discussion: https://postgr.es/m/20210924215827.GS831@telsasoft.com
Backpatch-through: 14
M src/backend/commands/statscmds.c
M src/backend/commands/tablecmds.c
M src/backend/parser/parse_utilcmd.c
M src/backend/statistics/extended_stats.c
M src/backend/utils/adt/selfuncs.c
doc: Fix some typos and markups
commit : 2a27dbaeb9088718baa25bc97f2e049e063412db
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 29 Sep 2021 11:56:36 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 29 Sep 2021 11:56:36 +0900
Author: Ekaterina Kiryanova
Discussion: https://postgr.es/m/8a14e78f-6991-7a6e-4711-fe376635f2ad@postgrespro.ru
Backpatch-through: 14
M doc/src/sgml/btree.sgml
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/alter_subscription.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/spgist.sgml
M doc/src/sgml/test-decoding.sgml
Fix instability in contrib/bloom TAP tests.
commit : cea5624f6a23a100fb93aee8b40bdc3a8cb52081
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 28 Sep 2021 17:34:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 28 Sep 2021 17:34:31 -0400
It turns out that the instability complained of in commit d3c09b9b1
has an embarrassingly simple explanation. The test script waits for
the standby to flush incoming WAL to disk, but it should wait for
the WAL to be replayed, since we are testing for the effects of that
to be visible.
While at it, use wait_for_catchup instead of reinventing that logic,
and adjust $Test::Builder::Level to improve future error reports.
Back-patch to v12 where the necessary infrastructure came in
(cf. aforesaid commit). Also back-patch 7d1aa6bf1 so that the
test will actually get run.
Discussion: https://postgr.es/m/2854602.1632852664@sss.pgh.pa.us
M contrib/bloom/Makefile
M contrib/bloom/t/001_wal.pl
doc: adjust attributions in PG 14 release notes
commit : 2f283d036da8e4c74fd07022cb685efee4e345bd
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 28 Sep 2021 14:15:39 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 28 Sep 2021 14:15:39 -0400
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
Properly schema-prefix reference to pg_catalog.pg_get_statisticsobjdef_columns
commit : febbb2f52c99b16261387daba4931d621dfbef0f
author : Magnus Hagander <magnus@hagander.net>
date : Tue, 28 Sep 2021 16:23:18 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Tue, 28 Sep 2021 16:23:18 +0200
Author: Tatsuro Yamada
Backpatch-through: 14
Discussion: https://www.postgresql.org/message-id/7ad8cd13-db5b-5cf6-8561-dccad1a934cb@nttcom.co.jp
M src/bin/psql/describe.c
Stamp 14.0.
commit : 86a4dc1e6f29d1992a2afa3fac1a0b0a6e84568c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 27 Sep 2021 16:57:41 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 27 Sep 2021 16:57:41 -0400
M configure
M configure.ac
Translation updates
commit : 9f535e4aeafc4e427355760178ecc89055f96942
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 27 Sep 2021 09:22:27 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 27 Sep 2021 09:22:27 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 941ca560d0b36a8bace8432b06302ca003829f42
M src/bin/psql/po/de.po
Update list of acknowledgments in release notes
commit : 3cc51338438acb38ea14cebe6fd979f00ad5bebf
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 27 Sep 2021 08:59:03 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 27 Sep 2021 08:59:03 +0200
current through e8b39cebdaf042dfeeb31d2f48f0fe7b33886210
M doc/src/sgml/release-14.sgml
Fix typos in docs
commit : e8b39cebdaf042dfeeb31d2f48f0fe7b33886210
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 26 Sep 2021 19:18:23 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 26 Sep 2021 19:18:23 +0900
Author: Justin Pryzby
Discussion: https://postgr.es/m/20210924215827.GS831@telsasoft.com
Backpatch-through: 9.6
M doc/src/sgml/datatype.sgml
Doc: final(?) updates for 14.0 release notes.
commit : 765f677f364100072160e7af37288eb1df2ff355
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Sep 2021 11:36:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Sep 2021 11:36:43 -0400
Add the customary short list of major features. Set release date.
Discussion: https://postgr.es/m/1489855.1631986639@sss.pgh.pa.us
M doc/src/sgml/release-14.sgml
Doc: extend warnings about collation-mismatch hazards in postgres_fdw.
commit : 02c4e3533926f4e7398611b6349d7c438b86c63b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Sep 2021 10:53:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Sep 2021 10:53:54 -0400
Be a little more vocal about the risks of remote collations not
matching local ones. Actually fixing these risks seems hard,
and I've given up on the idea that it might be back-patchable.
So the best we can do for the back branches is add documentation.
Per discussion of bug #16583 from Jiří Fejfar.
Discussion: https://postgr.es/m/2438715.1632510693@sss.pgh.pa.us
M doc/src/sgml/postgres-fdw.sgml
Add alternative output for OpenSSL 3 without legacy loaded
commit : 6d0001aabf2a49180a236d9c2a7ecdf24e0cdb37
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sat, 25 Sep 2021 11:27:28 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sat, 25 Sep 2021 11:27:28 +0200
OpenSSL 3 introduced the concept of providers to support modularization,
and moved the outdated ciphers to the new legacy provider. In case it's
not loaded in the users openssl.cnf file there will be a lot of regress
test failures, so add alternative outputs covering those.
Also document the need to load the legacy provider in order to use older
ciphers with OpenSSL-enabled pgcrypto.
This will be backpatched to all supported version once there is sufficient
testing in the buildfarm of OpenSSL 3.
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/FEF81714-D479-4512-839B-C769D2605F8A@yesql.se
A contrib/pgcrypto/expected/blowfish_1.out
A contrib/pgcrypto/expected/cast5_1.out
A contrib/pgcrypto/expected/des_1.out
A contrib/pgcrypto/expected/pgp-decrypt_1.out
A contrib/pgcrypto/expected/pgp-pubkey-decrypt_1.out
M doc/src/sgml/pgcrypto.sgml
Disable OpenSSL EVP digest padding in pgcrypto
commit : 4fa2b15e1c9cae79afe17c14591074111b0d4093
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sat, 25 Sep 2021 11:27:20 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sat, 25 Sep 2021 11:27:20 +0200
The PX layer in pgcrypto is handling digest padding on its own uniformly
for all backend implementations. Starting with OpenSSL 3.0.0, DecryptUpdate
doesn't flush the last block in case padding is enabled so explicitly
disable it as we don't use it.
This will be backpatched to all supported version once there is sufficient
testing in the buildfarm of OpenSSL 3.
Reviewed-by: Peter Eisentraut, Michael Paquier
Discussion: https://postgr.es/m/FEF81714-D479-4512-839B-C769D2605F8A@yesql.se
M contrib/pgcrypto/openssl.c
doc: Improve description of index vacuuming with GUCs
commit : bfe1ead94488986008771c0d295c2725bab952cb
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 25 Sep 2021 15:11:52 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 25 Sep 2021 15:11:52 +0900
Index vacuums may happen multiple times depending on the number of dead
tuples stored, as of maintenance_work_mem for a manual VACUUM. For
autovacuum, this is controlled by autovacuum_work_mem instead, if set.
The documentation mentioned the former, but not the latter in the
context of autovacuum.
Reported-by: Nikolai Berkoff
Author: Laurenz Albe, Euler Taveira
Discussion: https://postgr.es/m/161545365522.10134.12195402324485546870@wrigleys.postgresql.org
Backpatch-through: 9.6
M doc/src/sgml/monitoring.sgml
doc: Add missing markup in CREATE EVENT TRIGGER page
commit : 3eea9139ee46124746e942293d57e3fb1d3c0e09
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 25 Sep 2021 14:48:09 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 25 Sep 2021 14:48:09 +0900
Reported-by: rir
Discussion: https://postgr.es/m/20210924183658.3syyitp3yuxjv2fp@localhost
Backpatch-through: 9.6
M doc/src/sgml/ref/create_event_trigger.sgml
Add missing $Test::Builder::Level settings
commit : e536a2683439c3dd4ea6d339a878fa4430e3174d
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 23 Sep 2021 22:49:20 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 23 Sep 2021 22:49:20 +0200
One of these was accidentally removed by c50624c. The others are
added by analogy.
Discussion: https://www.postgresql.org/message-id/ae1143fb-455c-c80f-ed66-78d45bd93303@enterprisedb.com
M src/test/authentication/t/001_password.pl
M src/test/authentication/t/002_saslprep.pl
M src/test/kerberos/t/001_auth.pl
M src/test/ldap/t/001_auth.pl
Split macros from visibilitymap.h into a separate header
commit : 7186f07189baf069c54718315b81e65d87f4c0c1
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 23 Sep 2021 19:59:03 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 23 Sep 2021 19:59:03 +0300
That allows to include just visibilitymapdefs.h from file.c, and in turn,
remove include of postgres.h from relcache.h.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/20210913232614.czafiubr435l6egi%40alap3.anarazel.de
Author: Alexander Korotkov
Reviewed-by: Andres Freund, Tom Lane, Alvaro Herrera
Backpatch-through: 13
M src/bin/pg_upgrade/file.c
M src/include/access/visibilitymap.h
A src/include/access/visibilitymapdefs.h
M src/include/utils/relcache.h
Release memory allocated by dependency_degree
commit : abb2f9144ba1b7ac806f3779f53ae2f6174cd2d9
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 21 Sep 2021 01:13:11 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 21 Sep 2021 01:13:11 +0200
Calculating degree of a functional dependency may allocate a lot of
memory - we have released mot of the explicitly allocated memory, but
e.g. detoasted varlena values were left behind. That may be an issue,
because we consider a lot of dependencies (all combinations), and the
detoasting may happen for each one again.
Fixed by calling dependency_degree() in a dedicated context, and
resetting it after each call. We only need the calculated dependency
degree, so we don't need to copy anything.
Backpatch to PostgreSQL 10, where extended statistics were introduced.
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com
M src/backend/statistics/dependencies.c
Free memory after building each statistics object
commit : bb7628e55eda6f450f0f824d11c9a34b11f6bb12
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 21 Sep 2021 01:14:11 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 21 Sep 2021 01:14:11 +0200
Until now, all extended statistics on a given relation were built in the
same memory context, without resetting. Some of the memory was released
explicitly, but not all of it - for example memory allocated while
detoasting values is hard to free. This is how it worked since extended
statistics were introduced in PostgreSQL 10, but adding support for
extended stats on expressions made the issue somewhat worse as it
increases the number of statistics to build.
Fixed by adding a memory context which gets reset after building each
statistics object (all the statistics kinds included in it). Resetting
it after building each statistics kind would be even better, but it
would require more invasive changes and copying of results, making it
harder to backpatch.
Backpatch to PostgreSQL 10, where extended statistics were introduced.
Author: Justin Pryzby
Reported-by: Justin Pryzby
Reviewed-by: Tomas Vondra
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com
M src/backend/statistics/extended_stats.c
Invalidate all partitions for a partitioned table in publication.
commit : 9eff8593265929d3a1fcdee375bd0a801c12b367
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 22 Sep 2021 08:13:37 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 22 Sep 2021 08:13:37 +0530
Updates/Deletes on a partition were allowed even without replica identity
after the parent table was added to a publication. This would later lead
to an error on subscribers. The reason was that we were not invalidating
the partition's relcache and the publication information for partitions
was not getting rebuilt. Similarly, we were not invalidating the
partitions' relcache after dropping a partitioned table from a publication
which will prohibit Updates/Deletes on its partition without replica
identity even without any publication.
Reported-by: Haiying Tang
Author: Hou Zhijie and Vignesh C
Reviewed-by: Vignesh C and Amit Kapila
Backpatch-through: 13
Discussion: https://postgr.es/m/OS0PR01MB6113D77F583C922F1CEAA1C3FBD29@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/backend/catalog/pg_publication.c
M src/backend/commands/publicationcmds.c
M src/include/catalog/pg_publication.h
M src/include/commands/publicationcmds.h
M src/test/regress/expected/publication.out
M src/test/regress/sql/publication.sql
Fix "single value strategy" index deletion issue.
commit : e665129c4727004e7a7c12c86d077abc750b3307
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 21 Sep 2021 18:57:31 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 21 Sep 2021 18:57:31 -0700
It is not appropriate for deduplication to apply single value strategy
when triggered by a bottom-up index deletion pass. This wastes cycles
because later bottom-up deletion passes will overinterpret older
duplicate tuples that deduplication actually just skipped over "by
design". It also makes bottom-up deletion much less effective for low
cardinality indexes that happen to cross a meaningless "index has single
key value per leaf page" threshold.
To fix, slightly narrow the conditions under which deduplication's
single value strategy is considered. We already avoided the strategy
for a unique index, since our high level goal must just be to buy time
for VACUUM to run (not to buy space). We'll now also avoid it when we
just had a bottom-up pass that reported failure. The two cases share
the same high level goal, and already overlapped significantly, so this
approach is quite natural.
Oversight in commit d168b666, which added bottom-up index deletion.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WznaOvM+Gyj-JQ0X=JxoMDxctDTYjiEuETdAGbF5EUc3MA@mail.gmail.com
Backpatch: 14-, where bottom-up deletion was introduced.
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtinsert.c
M src/include/access/nbtree.h
Fix places in TestLib.pm in need of adaptation to the output of Msys perl
commit : 90251ff199858844fa450ba9614092c06c67fc4f
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Sep 2021 08:43:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Sep 2021 08:43:00 +0900
Contrary to the output of native perl, Msys perl generates outputs with
CRLFs characters. There are already places in the TAP code where CRLFs
(\r\n) are automatically converted to LF (\n) on Msys, but we missed a
couple of places when running commands and using their output for
comparison, that would lead to failures.
This problem has been found thanks to the test added in 5adb067 using
TestLib::command_checks_all(), but after a closer look more code paths
were missing a filter.
This is backpatched all the way down to prevent any surprises if a new
test is introduced in stable branches.
Reviewed-by: Andrew Dunstan, Álvaro Herrera
Discussion: https://postgr.es/m/1252480.1631829409@sss.pgh.pa.us
Backpatch-through: 9.6
M src/test/perl/TestLib.pm
Fix misevaluation of STABLE parameters in CALL within plpgsql.
commit : 2ad5f963e1306aa6f4cf96076a230e96529d2237
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 21 Sep 2021 19:06:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 21 Sep 2021 19:06:33 -0400
Before commit 84f5c2908, a STABLE function in a plpgsql CALL
statement's argument list would see an up-to-date snapshot,
because exec_stmt_call would push a new snapshot. I got rid of
that because the possibility of the snapshot disappearing within
COMMIT made it too hard to manage a snapshot across the CALL
statement. That's fine so far as the procedure itself goes,
but I forgot to think about the possibility of STABLE functions
within the CALL argument list. As things now stand, those'll
be executed with the Portal's snapshot as ActiveSnapshot,
keeping them from seeing updates more recent than Portal startup.
(VOLATILE functions don't have a problem because they take their
own snapshots; which indeed is also why the procedure itself
doesn't have a problem. There are no STABLE procedures.)
We can fix this by pushing a new snapshot transiently within
ExecuteCallStmt itself. Popping the snapshot before we get
into the procedure proper eliminates the management problem.
The possibly-useless extra snapshot-grab is slightly annoying,
but it's no worse than what happened before 84f5c2908.
Per bug #17199 from Alexander Nawratil. Back-patch to v11,
like the previous patch.
Discussion: https://postgr.es/m/17199-1ab2561f0d94af92@postgresql.org
M src/backend/commands/functioncmds.c
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/sql/plpgsql_transaction.sql
Document XLOG_INCLUDE_XID a little better
commit : c1d1ae1db23796e4d1b04f5c087944722cf1446a
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Sep 2021 19:47:53 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Sep 2021 19:47:53 -0300
I noticed that commit 0bead9af484c left this flag undocumented in
XLogSetRecordFlags, which led me to discover that the flag doesn't
actually do what the one comment on it said it does. Improve the
situation by adding some more comments.
Backpatch to 14, where the aforementioned commit appears.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/202109212119.c3nhfp64t2ql@alvherre.pgsql
M src/backend/access/transam/xloginsert.c
M src/include/access/xlog.h
M src/include/access/xlogrecord.h
Stamp 14rc1.
commit : 8aed7f7a2ec46ad60c84c3c97300210850c3fbc8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Sep 2021 17:33:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Sep 2021 17:33:01 -0400
M configure
M configure.ac
Remove overzealous index deletion assertion.
commit : 955a6a893498a5d3af544d9c0b1c292b19ec1690
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 20 Sep 2021 14:26:24 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 20 Sep 2021 14:26:24 -0700
A broken HOT chain is not an unexpected condition, even when the offset
number points past the end of the page's line pointer array.
heap_prune_chain() does not (and never has) treated this condition as
unexpected, so derivative code in heap_index_delete_tuples() shouldn't
do so either.
Oversight in commit 4228817449.
The assertion can probably only fail on Postgres 14 and master. Earlier
releases don't have commit 3c3b8a4b, which taught VACUUM to truncate the
line pointer array of heap pages. Backpatch all the same, just to be
consistent.
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/17197-9438f31f46705182@postgresql.org
Backpatch: 12-, just like commit 4228817449.
M src/backend/access/heap/heapam.c
Translation updates
commit : 3e8525aab86e78593844c9b395be40938582ebaf
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Sep 2021 16:23:13 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Sep 2021 16:23:13 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 10b675b81a3a04bac460cb049e0b7b6e17fb4795
M src/backend/nls.mk
M src/backend/po/de.po
M src/backend/po/fr.po
D src/backend/po/id.po
M src/backend/po/ja.po
D src/backend/po/pl.po
D src/backend/po/pt_BR.po
M src/backend/po/ru.po
D src/backend/po/tr.po
M src/backend/po/uk.po
M src/bin/initdb/nls.mk
M src/bin/initdb/po/el.po
M src/bin/initdb/po/fr.po
D src/bin/initdb/po/he.po
D src/bin/initdb/po/it.po
M src/bin/initdb/po/ja.po
D src/bin/initdb/po/pl.po
D src/bin/initdb/po/pt_BR.po
M src/bin/initdb/po/ru.po
M src/bin/initdb/po/tr.po
M src/bin/initdb/po/uk.po
D src/bin/initdb/po/vi.po
M src/bin/initdb/po/zh_CN.po
M src/bin/pg_amcheck/nls.mk
M src/bin/pg_amcheck/po/de.po
M src/bin/pg_amcheck/po/el.po
M src/bin/pg_amcheck/po/fr.po
M src/bin/pg_amcheck/po/ja.po
A src/bin/pg_amcheck/po/ru.po
M src/bin/pg_amcheck/po/uk.po
M src/bin/pg_amcheck/po/zh_CN.po
M src/bin/pg_archivecleanup/nls.mk
M src/bin/pg_archivecleanup/po/el.po
M src/bin/pg_archivecleanup/po/ja.po
D src/bin/pg_archivecleanup/po/pl.po
M src/bin/pg_archivecleanup/po/ru.po
M src/bin/pg_archivecleanup/po/tr.po
M src/bin/pg_archivecleanup/po/uk.po
D src/bin/pg_archivecleanup/po/vi.po
M src/bin/pg_archivecleanup/po/zh_CN.po
M src/bin/pg_basebackup/nls.mk
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_basebackup/po/fr.po
D src/bin/pg_basebackup/po/he.po
D src/bin/pg_basebackup/po/it.po
M src/bin/pg_basebackup/po/ja.po
D src/bin/pg_basebackup/po/pl.po
D src/bin/pg_basebackup/po/pt_BR.po
M src/bin/pg_basebackup/po/ru.po
M src/bin/pg_basebackup/po/uk.po
D src/bin/pg_basebackup/po/vi.po
M src/bin/pg_basebackup/po/zh_CN.po
M src/bin/pg_checksums/po/el.po
M src/bin/pg_checksums/po/ja.po
M src/bin/pg_checksums/po/ru.po
M src/bin/pg_checksums/po/uk.po
M src/bin/pg_checksums/po/zh_CN.po
M src/bin/pg_config/nls.mk
M src/bin/pg_config/po/el.po
M src/bin/pg_config/po/fr.po
M src/bin/pg_config/po/ja.po
D src/bin/pg_config/po/nb.po
D src/bin/pg_config/po/ro.po
M src/bin/pg_config/po/ru.po
D src/bin/pg_config/po/ta.po
M src/bin/pg_config/po/tr.po
M src/bin/pg_config/po/uk.po
M src/bin/pg_config/po/zh_CN.po
D src/bin/pg_config/po/zh_TW.po
M src/bin/pg_controldata/nls.mk
M src/