Stamp 15.5.
commit : 1e7f81e90741795d547c0290b4a82d84d518faac
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 6 Nov 2023 17:06:45 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 6 Nov 2023 17:06:45 -0500
M configure
M configure.ac
Last-minute updates for release notes.
commit : 95c96ba1d073afeadf193c7a9681a2c665dddead
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 6 Nov 2023 13:26:33 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 6 Nov 2023 13:26:33 -0500
Security: CVE-2023-5868, CVE-2023-5869, CVE-2023-5870
M doc/src/sgml/release-15.sgml
Detect integer overflow while computing new array dimensions.
commit : 3bc6bc3ee2dea4812203b3649096049af97fb13b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 6 Nov 2023 10:56:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 6 Nov 2023 10:56:43 -0500
array_set_element() and related functions allow an array to be
enlarged by assigning to subscripts outside the current array bounds.
While these places were careful to check that the new bounds are
allowable, they neglected to consider the risk of integer overflow
in computing the new bounds. In edge cases, we could compute new
bounds that are invalid but get past the subsequent checks,
allowing bad things to happen. Memory stomps that are potentially
exploitable for arbitrary code execution are possible, and so is
disclosure of server memory.
To fix, perform the hazardous computations using overflow-detecting
arithmetic routines, which fortunately exist in all still-supported
branches.
The test cases added for this generate (after patching) errors that
mention the value of MaxArraySize, which is platform-dependent.
Rather than introduce multiple expected-files, use psql's VERBOSITY
parameter to suppress the printing of the message text. v11 psql
lacks that parameter, so omit the tests in that branch.
Our thanks to Pedro Gallegos for reporting this problem.
Security: CVE-2023-5869
M src/backend/utils/adt/arrayfuncs.c
M src/backend/utils/adt/arrayutils.c
M src/include/utils/array.h
M src/test/regress/expected/arrays.out
M src/test/regress/sql/arrays.sql
Compute aggregate argument types correctly in transformAggregateCall().
commit : 4f4a422fbb15e49ff5e9dc410cd8713d63fdda24
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 6 Nov 2023 10:38:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 6 Nov 2023 10:38:00 -0500
transformAggregateCall() captures the datatypes of the aggregate's
arguments immediately to construct the Aggref.aggargtypes list.
This seems reasonable because the arguments have already been
transformed --- but there is an edge case where they haven't been.
Specifically, if we have an unknown-type literal in an ANY argument
position, nothing will have been done with it earlier. But if we
also have DISTINCT, then addTargetToGroupList() converts the literal
to "text" type, resulting in the aggargtypes list not matching the
actual runtime type of the argument. The end result is that the
aggregate tries to interpret a "text" value as being of type
"unknown", that is a zero-terminated C string. If the text value
contains no zero bytes, this could result in disclosure of server
memory following the text literal value.
To fix, move the collection of the aggargtypes list to the end
of transformAggregateCall(), after DISTINCT has been handled.
This requires slightly more code, but not a great deal.
Our thanks to Jingzhou Fu for reporting this problem.
Security: CVE-2023-5868
M src/backend/parser/parse_agg.c
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql
Set GUC "is_superuser" in all processes that set AuthenticatedUserId.
commit : fbc3719094150f5718cc55a80b3c98edf4188ef9
author : Noah Misch <noah@leadboat.com>
date : Mon, 6 Nov 2023 06:14:13 -0800
committer: Noah Misch <noah@leadboat.com>
date : Mon, 6 Nov 2023 06:14:13 -0800
It was always false in single-user mode, in autovacuum workers, and in
background workers. This had no specifically-identified security
consequences, but non-core code or future work might make it
security-relevant. Back-patch to v11 (all supported versions).
Jelte Fennema-Nio. Reported by Jelte Fennema-Nio.
M src/backend/utils/init/miscinit.c
Ban role pg_signal_backend from more superuser backend types.
commit : 595c988c90024acdc56479d0a631a232a9d19e36
author : Noah Misch <noah@leadboat.com>
date : Mon, 6 Nov 2023 06:14:13 -0800
committer: Noah Misch <noah@leadboat.com>
date : Mon, 6 Nov 2023 06:14:13 -0800
Documentation says it cannot signal "a backend owned by a superuser".
On the contrary, it could signal background workers, including the
logical replication launcher. It could signal autovacuum workers and
the autovacuum launcher. Block all that. Signaling autovacuum workers
and those two launchers doesn't stall progress beyond what one could
achieve other ways. If a cluster uses a non-core extension with a
background worker that does not auto-restart, this could create a denial
of service with respect to that background worker. A background worker
with bugs in its code for responding to terminations or cancellations
could experience those bugs at a time the pg_signal_backend member
chooses. Back-patch to v11 (all supported versions).
Reviewed by Jelte Fennema-Nio. Reported by Hemanth Sandrana and
Mahendrakar Srinivasarao.
Security: CVE-2023-5870
M src/backend/storage/ipc/signalfuncs.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Translation updates
commit : 8913ed121e28dc354791dc3137633b7d390ed86a
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 6 Nov 2023 13:16:22 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 6 Nov 2023 13:16:22 +0100
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 15fb3bd712561df7018c37a08ced1b71a05d4c31
M src/backend/nls.mk
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/backend/po/ja.po
A src/backend/po/ka.po
M src/backend/po/ru.po
M src/backend/po/sv.po
M src/backend/po/uk.po
M src/bin/initdb/po/es.po
M src/bin/initdb/po/ru.po
M src/bin/pg_basebackup/po/ru.po
M src/bin/pg_config/po/ru.po
M src/bin/pg_controldata/po/fr.po
M src/bin/pg_controldata/po/ru.po
M src/bin/pg_ctl/po/fr.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_dump/po/ja.po
M src/bin/pg_dump/po/ka.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/sv.po
M src/bin/pg_dump/po/uk.po
M src/bin/pg_resetwal/po/fr.po
M src/bin/pg_resetwal/po/ru.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_upgrade/po/de.po
M src/bin/pg_upgrade/po/fr.po
M src/bin/pg_upgrade/po/ja.po
M src/bin/pg_upgrade/po/ru.po
M src/bin/pg_verifybackup/po/ru.po
M src/bin/pg_waldump/po/ru.po
M src/bin/pg_waldump/po/sv.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/it.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/zh_CN.po
M src/bin/scripts/po/de.po
M src/bin/scripts/po/fr.po
M src/bin/scripts/po/ja.po
M src/bin/scripts/po/ru.po
M src/interfaces/libpq/po/ru.po
M src/interfaces/libpq/po/zh_CN.po
M src/pl/plpython/po/ka.po
M src/pl/plpython/po/ru.po
M src/pl/plpython/po/sv.po
M src/pl/plpython/po/uk.po
M src/pl/tcl/po/fr.po
Release notes for 16.1, 15.5, 14.10, 13.13, 12.17, 11.22.
commit : bea89120530ebd2d10b57f0f37436c2054997a3c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 5 Nov 2023 13:14:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 5 Nov 2023 13:14:07 -0500
M doc/src/sgml/release-15.sgml
doc: \copy can get data values \. and end-of-input confused
commit : a5bee67c36cc05507e919ed1e2711a9c51f38afb
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 3 Nov 2023 13:57:59 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 3 Nov 2023 13:57:59 -0400
Reported-by: Svante Richter
Discussion: https://postgr.es/m/fcd57e4-8f23-4c3e-a5db-2571d09208e2@beta.fastmail.com
Backpatch-through: 11
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/copy.c
doc: CREATE DATABASE doesn't copy db-level perms. from template
commit : 31a89ddeb8ebe1904a4a2fc8914f577c6ceb9dee
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 3 Nov 2023 13:39:50 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 3 Nov 2023 13:39:50 -0400
Reported-by: david@kapitaltrading.com
Discussion: https://postgr.es/m/166007719137.995877.13951579839074751714@wrigleys.postgresql.org
Backpatch-through: 11
M doc/src/sgml/manage-ag.sgml
pg_upgrade: Add missing newline to message
commit : 04d2d605f3a4d0d90f35e89c66a19c338ede0dd8
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 3 Nov 2023 11:50:25 -0400
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 3 Nov 2023 11:50:25 -0400
This was the backport of 2e3dc8c148, but in older releases the newline
must be in the message.
M src/bin/pg_upgrade/check.c
Doc: update CREATE RULE ref page's hoary discussion of views.
commit : 58d3283b675639a72faaef669371d71423be441b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 3 Nov 2023 11:48:23 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 3 Nov 2023 11:48:23 -0400
This text left one with the impression that an ON SELECT rule could
be attached to a plain table, which has not been true since commit
264c06820 (meaning the text was already misleading when written,
evidently by me in 96bd67f61). However, it didn't get really bad
until b23cd185f removed the convert-a-table-to-a-view logic, which
had made it possible for scripts that thought they were attaching
ON SELECTs to tables to still work.
Rewrite into a form that makes it clear that an ON SELECT rule
is better regarded as an implementation detail of a view.
Pre-v16, point out that adding ON SELECT to a table actually
converts it to a view.
Per bug #18178 from Joshua Uyehara. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/18178-05534d7064044d2d@postgresql.org
M doc/src/sgml/ref/create_rule.sgml
doc: ALTER DEFAULT PRIVILEGES does not affect inherited roles
commit : 1595724d00eb152296cfecae557a836963a67679
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 3 Nov 2023 09:51:53 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 3 Nov 2023 09:51:53 -0400
Reported-by: Jordi GutiƩrrez Hermoso
Discussion: https://postgr.es/m/72652d72e1816bfc3c05d40f9e0e0373d07823c8.camel@octave.org
Co-authored-by: Laurenz Albe
Backpatch-through: 11
M doc/src/sgml/ref/alter_default_privileges.sgml
Be more wary about NULL values for GUC string variables.
commit : ae33659d4239ac53ba502f44277e6958bcba7981
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 2 Nov 2023 11:47:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 2 Nov 2023 11:47:33 -0400
get_explain_guc_options() crashed if a string GUC marked GUC_EXPLAIN
has a NULL boot_val. Nosing around found a couple of other places
that seemed insufficiently cautious about NULL string values, although
those are likely unreachable in practice. Add some commentary
defining the expectations for NULL values of string variables,
in hopes of forestalling future additions of more such bugs.
Xing Guo, Aleksander Alekseev, Tom Lane
Discussion: https://postgr.es/m/CACpMh+AyDx5YUpPaAgzVwC1d8zfOL4JoD-uyFDnNSa1z0EsDQQ@mail.gmail.com
M src/backend/utils/misc/guc.c
M src/include/utils/guc_tables.h
Fix 003_check_guc.pl when loading modules with custom GUCs
commit : 2001aab86099e17dd4ea8dce362e225d50c26493
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 2 Nov 2023 12:38:28 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 2 Nov 2023 12:38:28 +0900
The test missed that custom GUCs need to be ignored from the list of
parameters that can exist in postgresql.conf.sample. This caused the
test to fail on a server where such a module is loaded, when using
EXTRA_INSTALL and TEMP_CONFIG, for instance.
Author: Anton A. Melnikov
Discussion: https://postgr.es/m/fc5509ce-5144-4dac-8d13-21793da44fc5@postgrespro.ru
Backpatch-through: 15
M src/test/modules/test_misc/t/003_check_guc.pl
doc: Replace reference to ERRCODE_RAISE_EXCEPTION by "raise_exception"
commit : bbbf1510bc414ce335aa778f0301dc46ad6a6bfd
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 2 Nov 2023 07:33:02 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 2 Nov 2023 07:33:02 +0900
This part of the documentation refers to exceptions as handled by
PL/pgSQL, and using the internal error code is confusing.
Per thinko in 66bde49d96a9.
Reported-by: Euler Taveira, Bruce Momjian
Discussion: https://postgr.es/m/ZUEUnLevXyW7DlCs@momjian.us
Backpatch-through: 11
M doc/src/sgml/plpgsql.sgml
doc: improve ALTER SYSTEM description of value list quoting
commit : d21b1706a900d35e7be513080fe9e0a32c43554c
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 31 Oct 2023 10:21:32 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 31 Oct 2023 10:21:32 -0400
Reported-by: splarv@ya.ru
Discussion: https://postgr.es/m/167105927893.1897.13227723035830709578@wrigleys.postgresql.org
Backpatch-through: 11
M doc/src/sgml/ref/alter_system.sgml
doc: add function argument and query parameter limits
commit : a1e2107862e3dcba87e014088b670db3977732dc
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 31 Oct 2023 09:23:09 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 31 Oct 2023 09:23:09 -0400
Also reorder entries and add commas.
Reported-by: David G. Johnston
Discussion: https://postgr.es/m/CAKFQuwYeNPxeocV3_0+Zx=_Xwvg+sNyEMdzyG5s2E2e0hZLQhg@mail.gmail.com
Author: David G. Johnston (partial)
Backpatch-through: 12
M doc/src/sgml/limits.sgml
doc: 1-byte varlena headers can be used for user PLAIN storage
commit : cbf6c07f4fcef9393689c2da7d442cc0cd1c0d4b
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 31 Oct 2023 09:10:35 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 31 Oct 2023 09:10:35 -0400
This also updates some C comments.
Reported-by: suchithjn22@gmail.com
Discussion: https://postgr.es/m/167336599095.2667301.15497893107226841625@wrigleys.postgresql.org
Author: Laurenz Albe (doc patch)
Backpatch-through: 11
M doc/src/sgml/storage.sgml
M src/backend/access/common/heaptuple.c
M src/backend/utils/adt/rangetypes.c
Adjust the order of the prechecks in pgrowlocks()
commit : 1360683530a67a6e494412c1cc16a6ad00e8e2f9
author : David Rowley <drowley@postgresql.org>
date : Tue, 31 Oct 2023 16:43:01 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 31 Oct 2023 16:43:01 +1300
4b8266415 added a precheck to pgrowlocks() to ensure the given object's
pg_class.relam is HEAP_TABLE_AM_OID, however, that check was put before
another check which was checking if the given object was a partitioned
table. Since the pg_class.relam is always InvalidOid for partitioned
tables, if pgrowlocks() was called passing a partitioned table, then the
"only heap AM is supported" error would be raised instead of the intended
error about the given object being a partitioned table.
Here we simply move the pg_class.relam check to after the check that
verifies that we are in fact working with a normal (non-partitioned)
table.
Reported-by: jian he
Discussion: https://postgr.es/m/CACJufxFaSp_WguFCf0X98951zFVX+dXFnF1mxAb-G3g1HiHOow@mail.gmail.com
Backpatch-through: 12, where 4b8266415 was introduced.
M contrib/pgrowlocks/pgrowlocks.c
Diagnose !indisvalid in more SQL functions.
commit : e633e9b132ffda7f4afd9248c7385bdd282d32d4
author : Noah Misch <noah@leadboat.com>
date : Mon, 30 Oct 2023 14:46:05 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 30 Oct 2023 14:46:05 -0700
pgstatindex failed with ERRCODE_DATA_CORRUPTED, of the "can't-happen"
class XX. The other functions succeeded on an empty index; they might
have malfunctioned if the failed index build left torn I/O or other
complex state. Report an ERROR in statistics functions pgstatindex,
pgstatginindex, pgstathashindex, and pgstattuple. Report DEBUG1 and
skip all index I/O in maintenance functions brin_desummarize_range,
brin_summarize_new_values, brin_summarize_range, and
gin_clean_pending_list. Back-patch to v11 (all supported versions).
Discussion: https://postgr.es/m/20231001195309.a3@google.com
M contrib/pgstattuple/pgstatindex.c
M contrib/pgstattuple/pgstattuple.c
M src/backend/access/brin/brin.c
M src/backend/access/gin/ginfast.c
amcheck: Distinguish interrupted page deletion from corruption.
commit : 6f81386a9cea39b83e24943224569ec18557163c
author : Noah Misch <noah@leadboat.com>
date : Mon, 30 Oct 2023 14:46:05 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 30 Oct 2023 14:46:05 -0700
This prevents false-positive reports about "the first child of leftmost
target page is not leftmost of its level", "block %u is not leftmost"
and "left link/right link pair". They appeared if amcheck ran before
VACUUM cleaned things, after a cluster exited recovery between the
first-stage and second-stage WAL records of a deletion. Back-patch to
v11 (all supported versions).
Reviewed by Peter Geoghegan.
Discussion: https://postgr.es/m/20231005025232.c7.nmisch@google.com
M contrib/amcheck/Makefile
A contrib/amcheck/t/005_pitr.pl
M contrib/amcheck/verify_nbtree.c
btree_gin: Fix calculation of leftmost interval value.
commit : 5f06918399f07d46c15756bea84d8d5665ebfa57
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sun, 29 Oct 2023 11:14:35 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sun, 29 Oct 2023 11:14:35 +0000
Formerly, the value computed by leftmostvalue_interval() was a long
way short of the minimum possible interval value. As a result, an
index scan on a GIN index on an interval column with < or <= operators
would miss large negative interval values.
Fix by setting all fields of the leftmost interval to their minimum
values, ensuring that the result is less than any other possible
interval. Since this only affects index searches, no index rebuild is
necessary.
Back-patch to all supported branches.
Dean Rasheed, reviewed by Heikki Linnakangas.
Discussion: https://postgr.es/m/CAEZATCV80%2BgOfF8ehNUUfaKBZgZMDfCfL-g1HhWGb6kC3rpDfw%40mail.gmail.com
M contrib/btree_gin/btree_gin.c
M contrib/btree_gin/expected/interval.out
M contrib/btree_gin/sql/interval.sql
Fix intra-query memory leak when a SRF returns zero rows.
commit : 592cb11fbee08b4cdd660bb45b6767645c9cfdae
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Oct 2023 14:04:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Oct 2023 14:04:43 -0400
When looping around after finding that the set-returning function
returned zero rows for the current input tuple, ExecProjectSet
neglected to reset either of the two memory contexts it's
responsible for cleaning out. Typically this wouldn't cause much
problem, because once the SRF does return at least one row, the
contexts would get reset on the next call. However, if the SRF
returns no rows for many input tuples in succession, quite a lot
of memory could be transiently consumed.
To fix, make sure we reset both contexts while looping around.
Per bug #18172 from Sergei Kornilov. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/18172-9b8c5fc1d676ded3@postgresql.org
M src/backend/executor/nodeProjectSet.c
Remove PHOT from our default timezone abbreviations list.
commit : 85b98a70bbea1896464f767013eac0a6d2e3b938
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Oct 2023 11:54:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Oct 2023 11:54:40 -0400
Debian recently decided to split out a bunch of "obsolete" timezone
names into a new tzdata-legacy package, which isn't installed by
default. One of these zone names is Pacific/Enderbury, and that
breaks our regression tests (on --with-system-tzdata builds)
because our default timezone abbreviations list defines PHOT as
Pacific/Enderbury.
Pacific/Enderbury got renamed to Pacific/Kanton in tzdata 2021b,
so that in distros that still have this entry it's just a symlink
to Pacific/Kanton anyway. So one answer would be to redefine PHOT
as Pacific/Kanton. However, then things would fail if the
installed tzdata predates 2021b, which is recent enough that that
seems like a real problem.
Instead, let's just remove PHOT from the default list. That seems
likely to affect nobody in the real world, because (a) it was an
abbreviation that the tzdb crew made up in the first place, with
no evidence of real-world usage, and (b) the total human population
of the Phoenix Islands is less than two dozen persons, per Wikipedia.
If anyone does use this zone abbreviation they can easily put it back
via a custom abbreviations file.
We'll keep PHOT in the Pacific.txt reference file, but change it
to Pacific/Kanton there, as that definition seems more likely to
be useful to future readers of that file.
Per report from Victor Wagner. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/20231027152049.4b5c8044@wagner.wagner.home
M src/timezone/tznames/Default
M src/timezone/tznames/Pacific.txt
Fix minmax-multi distance for extreme interval values
commit : 2fbb2fcb0c63d6254aa93a1181a263df07e5c26d
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 27 Oct 2023 17:57:44 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 27 Oct 2023 17:57:44 +0200
When calculating distance for interval values, the code mostly mimicked
interval_mi, i.e. it built a new interval value for the difference.
That however does not work for sufficiently distant interval values,
when the difference overflows the interval range.
Instead, we can calculate the distance directly, without constructing
the intermediate (and unnecessary) interval value.
Backpatch to 14, where minmax-multi indexes were introduced.
Reported-by: Dean Rasheed
Reviewed-by: Ashutosh Bapat, Dean Rasheed
Backpatch-through: 14
Discussion: https://postgr.es/m/eef0ea8c-4aaa-8d0d-027f-58b1f35dd170@enterprisedb.com
M src/backend/access/brin/brin_minmax_multi.c
M src/test/regress/expected/brin_multi.out
M src/test/regress/sql/brin_multi.sql
Fix minmax-multi on infinite date/timestamp values
commit : d04a9283b7070c36578a6e2b1726b251508f978d
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 27 Oct 2023 17:57:28 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 27 Oct 2023 17:57:28 +0200
Make sure that infinite values in date/timestamp columns are treated as
if in infinite distance. Infinite values should not be merged with other
values, leaving them as outliers. The code however returned distance 0
in this case, so that infinite values were merged first. While this does
not break the index (i.e. it still produces correct query results), it
may make it much less efficient.
We don't need explicit handling of infinite date/timestamp values when
calculating distances, because those values are represented as extreme
but regular values (e.g. INT64_MIN/MAX for the timestamp type).
We don't need an exact distance, just a value that is much larger than
distanced between regular values. With the added cast to double values,
we can simply subtract the values.
The regression test queries a value in the "gap" and checks the range
was properly eliminated by the BRIN index.
This only affects minmax-multi indexes on timestamp/date columns with
infinite values, which is not very common in practice. The affected
indexes may need to be rebuilt.
Backpatch to 14, where minmax-multi indexes were introduced.
Reported-by: Ashutosh Bapat
Reviewed-by: Ashutosh Bapat, Dean Rasheed
Backpatch-through: 14
Discussion: https://postgr.es/m/eef0ea8c-4aaa-8d0d-027f-58b1f35dd170@enterprisedb.com
M src/backend/access/brin/brin_minmax_multi.c
M src/test/regress/expected/brin_multi.out
M src/test/regress/sql/brin_multi.sql
Fix calculation in brin_minmax_multi_distance_date
commit : 088233f8db6c22cbff6c29c212ce138236a5bcc1
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 27 Oct 2023 17:57:11 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 27 Oct 2023 17:57:11 +0200
When calculating the distance between date values, make sure to subtract
them in the right order, i.e. (larger - smaller).
The distance is used to determine which values to merge, and is expected
to be a positive value. The code unfortunately did the subtraction in
the opposite order, i.e. (smaller - larger), thus producing negative
values and merging values the most distant values first.
The resulting index is correct (i.e. produces correct results), but may
be significantly less efficient. This affects all minmax-multi indexes
on date columns.
Backpatch to 14, where minmax-multi indexes were introduced.
Reported-by: Ashutosh Bapat
Reviewed-by: Ashutosh Bapat, Dean Rasheed
Backpatch-through: 14
Discussion: https://postgr.es/m/eef0ea8c-4aaa-8d0d-027f-58b1f35dd170@enterprisedb.com
M src/backend/access/brin/brin_minmax_multi.c
M src/test/regress/expected/brin_multi.out
M src/test/regress/sql/brin_multi.sql
Fix overflow when calculating timestamp distance in BRIN
commit : daa7b0d7ce149b0f7f9d68dfae0c2483a7ba5e98
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 27 Oct 2023 17:56:27 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 27 Oct 2023 17:56:27 +0200
When calculating distances for timestamp values for BRIN minmax-multi
indexes, we need to be careful about overflows for extreme values. If
the value overflows into a negative value, the index may be inefficient.
The new regression test checks this for the timestamp type by adding a
table with enough values to force range compaction/merging. The values
are close to min/max, which means a risk of overflow.
Fixed by converting the int64 values to double first, before calculating
the distance. This prevents the overflow. We may lose some precision, of
course, but that's good enough. In the worst case we build a slightly
less efficient index, but for large distances this won't matter.
This only affects minmax-multi indexes on timestamp columns, with ranges
containing values sufficiently distant to cause an overflow. That seems
like a fairly rare case in practice.
Backpatch to 14, where minmax-multi indexes were introduced.
Reported-by: Ashutosh Bapat
Reviewed-by: Ashutosh Bapat, Dean Rasheed
Backpatch-through: 14
Discussion: https://postgr.es/m/eef0ea8c-4aaa-8d0d-027f-58b1f35dd170@enterprisedb.com
M src/backend/access/brin/brin_minmax_multi.c
M src/test/regress/expected/brin_multi.out
M src/test/regress/sql/brin_multi.sql
Doc: remove misleading info about ecpg's CONNECT/DISCONNECT DEFAULT.
commit : 733e99de4ec7cd2fcf291a2f8259d66b75f0a0b3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 25 Oct 2023 17:34:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 25 Oct 2023 17:34:47 -0400
As far as I can see, ecpg has no notion of a "default" open
connection. You can do "CONNECT TO DEFAULT" but that just specifies
letting libpq use all its default connection parameters --- the
resulting connection is not special subsequently. In particular,
SET CONNECTION = DEFAULT and DISCONNECT DEFAULT simply act on a
connection named DEFAULT, if you've made one; they do not have
special lookup rules. But the documentation of these commands
makes it look like they do.
Simplest fix, I think, is just to remove the paras suggesting that
DEFAULT is special here.
Also, SET CONNECTION *does* have one special lookup rule, which
is that it recognizes CURRENT as an alias for the currently selected
connection. SET CONNECTION = CURRENT is a no-op, so it's pretty
useless, but nonetheless it does something different from selecting
a connection by name; so we'd better document it.
Per report from Sylvain Frandaz. Back-patch to all supported
versions.
Discussion: https://postgr.es/m/169824721149.1769274.1553568436817652238@wrigleys.postgresql.org
M doc/src/sgml/ecpg.sgml
doc: Fix some typos and grammar
commit : d8c3ed4f2981821a62fbd83509db28cfa49e8994
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 25 Oct 2023 09:41:11 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 25 Oct 2023 09:41:11 +0900
Author: Ekaterina Kiryanova, Elena Indrupskaya, Oleg Sibiryakov, Maxim
Yablokov
Discussion: https://postgr.es/m/7aad518b-3e6d-47f3-9184-b1d69cb412e7@postgrespro.ru
Backpatch-through: 11
M doc/src/sgml/ref/create_foreign_table.sgml
Fix problems when a plain-inheritance parent table is excluded.
commit : 1268e7378123a4fccc73f709fb0cb0021a2e8d5a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 24 Oct 2023 14:48:28 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 24 Oct 2023 14:48:28 -0400
When an UPDATE/DELETE/MERGE's target table is an old-style
inheritance tree, it's possible for the parent to get excluded
from the plan while some children are not. (I believe this is
only possible if we can prove that a CHECK ... NO INHERIT
constraint on the parent contradicts the query WHERE clause,
so it's a very unusual case.) In such a case, ExecInitModifyTable
mistakenly concluded that the first surviving child is the target
table, leading to at least two bugs:
1. The wrong table's statement-level triggers would get fired.
2. In v16 and up, it was possible to fail with "invalid perminfoindex
0 in RTE with relid nnnn" due to the child RTE not having permissions
data included in the query plan. This was hard to reproduce reliably
because it did not occur unless the update triggered some non-HOT
index updates.
In v14 and up, this is easy to fix by defining ModifyTable.rootRelation
to be the parent RTE in plain inheritance as well as partitioned cases.
While the wrong-triggers bug also appears in older branches, the
relevant code in both the planner and executor is quite a bit
different, so it would take a good deal of effort to develop and
test a suitable patch. Given the lack of field complaints about the
trigger issue, I'll desist for now. (Patching v11 for this seems
unwise anyway, given that it will have no more releases after next
month.)
Per bug #18147 from Hans Buschmann.
Amit Langote and Tom Lane
Discussion: https://postgr.es/m/18147-6fc796538913ee88@postgresql.org
M src/backend/executor/nodeModifyTable.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/util/pathnode.c
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/test/regress/expected/inherit.out
M src/test/regress/sql/inherit.sql
Doc: indexUnchanged is strictly a hint.
commit : be2502947f299a8f8f076e49eb74289558654368
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 24 Oct 2023 09:27:23 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 24 Oct 2023 09:27:23 -0700
Clearly spell out the limitations of aminsert()'s indexUnchanged hinting
mechanism in the index AM documentation.
Oversight in commit 9dc718bd, which added the "logically unchanged
index" hint (which is used to trigger bottom-up index deletion).
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAH2-WzmU_BQ=-H9L+bxTSMQBqHMjp1DSwGypvL0gKs+dTOfkKg@mail.gmail.com
Backpatch: 14-, where indexUnchanged hinting was introduced.
M doc/src/sgml/indexam.sgml
Log LLVM library version in configure output.
commit : 0e49e23782c55237f54aa60ed7a2b755b47cd4a2
author : Thomas Munro <tmunro@postgresql.org>
date : Sun, 22 Oct 2023 14:17:00 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sun, 22 Oct 2023 14:17:00 +1300
When scanning build farm results, it's useful to be able to see which
version is in use. For the Meson build system, this information was
already displayed.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/4022690.1697852728%40sss.pgh.pa.us
M config/llvm.m4
M configure
Fix min_dynamic_shared_memory on Windows.
commit : f72790b2952d75c885cf812b38d59d1586bf039e
author : Thomas Munro <tmunro@postgresql.org>
date : Sun, 22 Oct 2023 10:04:55 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sun, 22 Oct 2023 10:04:55 +1300
When min_dynamic_shared_memory is set above 0, we try to find space in a
pre-allocated region of the main shared memory area instead of calling
dsm_impl_XXX() routines to allocate more. The dsm_pin_segment() and
dsm_unpin_segment() routines had a bug: they called dsm_impl_XXX()
routines even for main region segments. Nobody noticed before now
because those routines do nothing on Unix, but on Windows they'd fail
while attempting to duplicate an invalid Windows HANDLE. Add the
missing gating.
Back-patch to 14, where commit 84b1c63a added this feature. Fixes
pgsql-bugs bug #18165.
Reported-by: Maxime Boyer <maxime.boyer@cra-arc.gc.ca>
Tested-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/18165-bf4f525cea6e51de%40postgresql.org
M src/backend/storage/ipc/dsm.c
Dodge a compiler bug affecting timetz_zone/timetz_izone.
commit : 5463319acd19771ac0c986d305f66ea8fb3cc801
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 20 Oct 2023 13:40:15 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 20 Oct 2023 13:40:15 -0400
This avoids a compiler bug occurring in AIX's xlc, even in pretty
late-model revisions. Buildfarm testing has now confirmed that
only 64-bit xlc is affected. Although we are contemplating
dropping support for xlc in v17, it's still supported in the
back branches, so we need this fix.
Back-patch of code changes from HEAD commit 19fa97731.
(The test cases were already back-patched, in 4a427b82c et al.)
Discussion: https://postgr.es/m/CA+hUKGK=DOC+hE-62FKfZy=Ybt5uLkrg3zCZD-jFykM-iPn8yw@mail.gmail.com
M src/backend/utils/adt/date.c
Doc: update CREATE OPERATOR's statement about => as an operator.
commit : d568d2fab4be8c74804df40b733925ac92dc0dd8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 20 Oct 2023 13:01:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 20 Oct 2023 13:01:02 -0400
This doco said that use of => as an operator "is deprecated".
It's been fully disallowed since 865f14a2d back in 9.5, but
evidently that commit missed updating this statement.
Do so now.
M doc/src/sgml/ref/create_operator.sgml
Improve pglz_decompress's defenses against corrupt compressed data.
commit : 985ac5ce292ab084fe56b90528df6a2496f2c507
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 18 Oct 2023 20:43:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 18 Oct 2023 20:43:17 -0400
When processing a match tag, check to see if the claimed "off"
is more than the distance back to the output buffer start.
If it is, then the data is corrupt, and what's more we would
fetch from outside the buffer boundaries and potentially incur
a SIGSEGV. (Although the odds of that seem relatively low, given
that "off" can't be more than 4K.)
Back-patch to v13; before that, this function wasn't really
trying to protect against bad data.
Report and fix by Flavien Guedez.
Discussion: https://postgr.es/m/01fc0593-e31e-463d-902c-dd43174acee2@oopacity.net
M src/common/pg_lzcompress.c
jit: Changes for LLVM 17.
commit : b60e3ac7603dfae3f810418b1fb5856d9fc7a328
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 18 Oct 2023 22:15:54 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 18 Oct 2023 22:15:54 +1300
Changes required by https://llvm.org/docs/NewPassManager.html.
Back-patch to 12, leaving the final release of 11 unchanged, consistent
with earlier decision not to back-patch LLVM 16 support either.
Author: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKG%2BWXznXCyTgCADd%3DHWkP9Qksa6chd7L%3DGCnZo-MBgg9Lg%40mail.gmail.com
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_wrap.cpp
jit: Supply LLVMGlobalGetValueType() for LLVM < 8.
commit : b2e0977886630b2bf749937fd694f0c3c1392210
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 19 Oct 2023 03:01:55 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 19 Oct 2023 03:01:55 +1300
Commit 37d5babb used this C API function while adding support for LLVM
16 and opaque pointers, but it's not available in LLVM 7 and older.
Provide it in our own llvmjit_wrap.cpp. It just calls a C++ function
that pre-dates LLVM 3.9, our minimum target.
Back-patch to 12, like 37d5babb.
Discussion: https://postgr.es/m/CA%2BhUKGKnLnJnWrkr%3D4mSGhE5FuTK55FY15uULR7%3Dzzc%3DwX4Nqw%40mail.gmail.com
M src/backend/jit/llvm/llvmjit_wrap.cpp
M src/include/jit/llvmjit.h
jit: Support opaque pointers in LLVM 16.
commit : eed1feb3fee1a558b67b04cbd709f31142f071d5
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 18 Oct 2023 22:09:05 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 18 Oct 2023 22:09:05 +1300
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_deform.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/jit/llvm/llvmjit_types.c
M src/backend/jit/llvm/llvmjit_wrap.cpp
M src/include/jit/llvmjit.h
M src/include/jit/llvmjit_emit.h
pg_upgrade: Fix test name in 002_pg_upgrade.pl
commit : c4e561c1e029461430c86d81a1956ddddb4fbe13
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 18 Oct 2023 18:23:50 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 18 Oct 2023 18:23:50 +0900
Author: Hou Zhijie
Discussion: https://postgr.es/m/TYAPR01MB5724A40D47E71F4717357EC694D5A@TYAPR01MB5724.jpnprd01.prod.outlook.com
Backpatch-through: 15
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
Back-patch test cases for timetz_zone/timetz_izone.
commit : 29231dbd40cbebe7651ca718d084b5b5ed3a9f00
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Oct 2023 13:55:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Oct 2023 13:55:45 -0400
Per code coverage reports, we had zero regression test coverage
of these functions. That came back to bite us, as apparently
that's allowed us to miss discovering misbehavior of this code
with AIX's xlc compiler. Install relevant portions of the
test cases added in 97957fdba, 2f0472030, 19fa97731.
(Assuming the expected outcome that the xlc problem does appear
in back branches, a code fix will follow.)
Discussion: https://postgr.es/m/CA+hUKGK=DOC+hE-62FKfZy=Ybt5uLkrg3zCZD-jFykM-iPn8yw@mail.gmail.com
M src/test/regress/expected/timetz.out
M src/test/regress/sql/timetz.sql
Avoid calling proc_exit() in processes forked by system().
commit : c9265ae80b6aa02daef620465800b03967433a39
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 17 Oct 2023 10:42:06 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 17 Oct 2023 10:42:06 -0500
The SIGTERM handler for the startup process immediately calls
proc_exit() for the duration of the restore_command, i.e., a call
to system(). This system() call forks a new process to execute the
shell command, and this child process inherits the parent's signal
handlers. If both the parent and child processes receive SIGTERM,
both will attempt to call proc_exit(). This can end badly. For
example, both processes will try to remove themselves from the
PGPROC shared array.
To fix this problem, this commit adds a check in
StartupProcShutdownHandler() to see whether MyProcPid == getpid().
If they match, this is the parent process, and we can proc_exit()
like before. If they do not match, this is a child process, and we
just emit a message to STDERR (in a signal safe manner) and
_exit(), thereby skipping any problematic exit callbacks.
This commit also adds checks in proc_exit(), ProcKill(), and
AuxiliaryProcKill() that verify they are not being called within
such child processes.
Suggested-by: Andres Freund
Reviewed-by: Thomas Munro, Andres Freund
Discussion: https://postgr.es/m/Y9nGDSgIm83FHcad%40paquier.xyz
Discussion: https://postgr.es/m/20230223231503.GA743455%40nathanxps13
Backpatch-through: 11
M src/backend/postmaster/startup.c
M src/backend/storage/ipc/ipc.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/error/elog.c
M src/include/utils/elog.h
Ensure we have a snapshot while dropping ON COMMIT DROP temp tables.
commit : 0d1a7cd14ee130ae5a1b31dd80a6aadfe9da67de
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Oct 2023 14:06:11 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Oct 2023 14:06:11 -0400
Dropping a temp table could entail TOAST table access to clean out
toasted catalog entries, such as large pg_constraint.conbin strings
for complex CHECK constraints. If we did that via ON COMMIT DROP,
we triggered the assertion in init_toast_snapshot(), because
there was no provision for setting up a snapshot for the drop
actions. Fix that.
(I assume here that the adjacent truncation actions for ON COMMIT
DELETE ROWS don't have a similar problem: it doesn't seem like
nontransactional truncations would need to touch any toasted fields.
If that proves wrong, we could refactor a bit to have the same
snapshot acquisition cover that too.)
The test case added here does not fail before v15, because that
assertion was added in 277692220 which was not back-patched.
However, the race condition the assertion warns of surely
exists further back, so back-patch to all supported branches.
Per report from Richard Guo.
Discussion: https://postgr.es/m/CAMbWs4-x26=_QxxgdJyNbiCDzvtr2WV5ZDso_v-CukKEe6cBZw@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/temp.out
M src/test/regress/sql/temp.sql
Move extra code out of the Pre/PostRestoreCommand() section.
commit : 882e522d64686ade162091a572118da2562f3569
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 16 Oct 2023 12:43:24 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 16 Oct 2023 12:43:24 -0500
If SIGTERM is received within this section, the startup process
will immediately proc_exit() in the signal handler, so it is
inadvisable to include any more code than is required there (as
such code is unlikely to be compatible with doing proc_exit() in a
signal handler). This commit moves the code recently added to this
section (see 1b06d7bac9 and 7fed801135) to outside of the section.
This ensures that the startup process only calls proc_exit() in its
SIGTERM handler for the duration of the system() call, which is how
this code worked from v8.4 to v14.
Reported-by: Michael Paquier, Thomas Munro
Analyzed-by: Andres Freund
Suggested-by: Tom Lane
Reviewed-by: Michael Paquier, Robert Haas, Thomas Munro, Andres Freund
Discussion: https://postgr.es/m/Y9nGDSgIm83FHcad%40paquier.xyz
Discussion: https://postgr.es/m/20230223231503.GA743455%40nathanxps13
Backpatch-through: 15
M src/backend/access/transam/xlogarchive.c
Update the documentation on recovering from (M)XID exhaustion.
commit : fdfb141a7467ef61396e6f697a45d0470285c657
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 16 Oct 2023 12:57:39 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 16 Oct 2023 12:57:39 -0400
The old documentation encourages entering single-user mode for no
reason, which is a bad plan in most cases. Instead, discourage users
from doing that, and explain the limited cases in which it may be
desirable.
The old documentation claims that running VACUUM as anyone but the
superuser can't possibly work, which is not really true, because it
might be that some other user has enough permissions to VACUUM all
the tables that matter. Weaken the language just a bit.
The old documentation claims that you can't run any commands
when near XID exhaustion, which is false because you can still
run commands that don't require an XID, like a SELECT without a
locking clause.
The old documentation doesn't clearly explain that it's a good idea
to get rid of prepared transactons, long-running transactions, and
replication slots that are preventing (M)XID horizon advancement.
Spell out the steps to do that.
Also, discourage the use of VACUUM FULL and VACUUM FREEZE in
this type of scenario.
Back-patch to v14. Much of this is good advice on all supported
versions, but before 60f1f09ff44308667ef6c72fbafd68235e55ae27
the chances of VACUUM failing in multi-user mode were much higher.
Alexander Alekseev, John Naylor, Robert Haas, reviewed at various
times by Peter Geoghegan, Hannu Krosing, and Andres Freund.
Discussion: http://postgr.es/m/CA+TgmoYtsUDrzaHcmjFhLzTk1VEv29mO_u-MT+XWHrBJ_4nD8A@mail.gmail.com
M doc/src/sgml/maintenance.sgml
Try to handle torn reads of pg_control in frontend.
commit : 5e39884d322a7dd1dd595fbaaeb1f88c8907d3a6
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 16 Oct 2023 17:10:13 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 16 Oct 2023 17:10:13 +1300
Some of our src/bin tools read the control file without any kind of
interlocking against concurrent writes from the server. At least ext4
and ntfs can expose partially modified contents when you do that.
For now, we'll try to tolerate this by retrying up to 10 times if the
checksum doesn't match, until we get two reads in a row with the same
bad checksum. This is not guaranteed to reach the right conclusion, but
it seems very likely to. Thanks to Tom Lane for this suggestion.
Various ideas for interlocking or atomicity were considered too
complicated, unportable or expensive given the lack of field reports,
but remain open for future reconsideration.
Back-patch as far as 12. It doesn't seem like a good idea to put a
heuristic change for a very rare problem into the final release of 11.
Reviewed-by: Anton A. Melnikov <aamelnikov@inbox.ru>
Reviewed-by: David Steele <david@pgmasters.net>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20221123014224.xisi44byq3cf5psi%40awork3.anarazel.de
M src/common/controldata_utils.c
Acquire ControlFileLock in relevant SQL functions.
commit : 606be8a35d9726b6c0468428b6262e58d9e308af
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 16 Oct 2023 10:43:47 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 16 Oct 2023 10:43:47 +1300
Commit dc7d70ea added functions that read the control file, but didn't
acquire ControlFileLock. With unlucky timing, file systems that have
weak interlocking like ext4 and ntfs could expose partially overwritten
contents, and the checksum would fail.
Back-patch to all supported releases.
Reviewed-by: David Steele <david@pgmasters.net>
Reviewed-by: Anton A. Melnikov <aamelnikov@inbox.ru>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20221123014224.xisi44byq3cf5psi%40awork3.anarazel.de
M src/backend/utils/misc/pg_controldata.c
Dissociate btequalimage() from interval_ops, ending its deduplication.
commit : 782be0f7124a7e45acf122c1f4988937b94ddb4e
author : Noah Misch <noah@leadboat.com>
date : Sat, 14 Oct 2023 16:33:51 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 14 Oct 2023 16:33:51 -0700
Under interval_ops, some equal values are distinguishable. One such
pair is '24:00:00' and '1 day'. With that being so, btequalimage()
breaches the documented contract for the "equalimage" btree support
function. This can cause incorrect results from index-only scans.
Users should REINDEX any btree indexes having interval-type columns.
After updating, pg_amcheck will report an error for almost all such
indexes. This fix makes interval_ops simply omit the support function,
like numeric_ops does. Back-pack to v13, where btequalimage() first
appeared. In back branches, for the benefit of old catalog content,
btequalimage() code will return false for type "interval". Going
forward, back-branch initdb will include the catalog change.
Reviewed by Peter Geoghegan.
Discussion: https://postgr.es/m/20231011013317.22.nmisch@google.com
M contrib/amcheck/verify_nbtree.c
M src/backend/utils/adt/datum.c
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_opfamily.dat
M src/test/regress/expected/opr_sanity.out
Don't spuriously report FD_SETSIZE exhaustion on Windows.
commit : 1102f4ece30ed734ca2193fedac74395490aee99
author : Noah Misch <noah@leadboat.com>
date : Sat, 14 Oct 2023 15:54:46 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 14 Oct 2023 15:54:46 -0700
Starting on 2023-08-03, this intermittently terminated a "pgbench -C"
test in CI. It could affect a high-client-count "pgbench" without "-C".
While parallel reindexdb and vacuumdb reach the same problematic check,
sufficient client count and/or connection turnover is less plausible for
them. Given the lack of examples from the buildfarm or from manual
builds, reproducing this must entail rare operating system
configurations. Also correct the associated error message, which was
wrong for non-Windows. Back-patch to v12, where the pgbench check first
appeared. While v11 vacuumdb has the problematic check, reaching it
with typical vacuumdb usage is implausible.
Reviewed by Thomas Munro.
Discussion: https://postgr.es/m/CA+hUKG+JwvTNdcyJTriy9BbtzF1veSRQ=9M_ZKFn9_LqE7Kp7Q@mail.gmail.com
M src/bin/pgbench/pgbench.c
M src/fe_utils/parallel_slot.c
Fix runtime partition pruning for HASH partitioned tables
commit : 1e81d3e6e0a0e35e52e109742c5ce3738c3e87c6
author : David Rowley <drowley@postgresql.org>
date : Fri, 13 Oct 2023 01:13:36 +1300
committer: David Rowley <drowley@postgresql.org>
date : Fri, 13 Oct 2023 01:13:36 +1300
This could only affect HASH partitioned tables with at least 2 partition
key columns.
If partition pruning was delayed until execution and the query contained
an IS NULL qual on one of the partitioned keys, and some subsequent
partitioned key was being compared to a non-Const, then this could result
in a crash due to the incorrect keyno being used to calculate the
stateidx for the expression evaluation code.
Here we fix this by properly skipping partitioned keys which have a
nullkey set. Effectively, this must be the same as what's going on
inside perform_pruning_base_step().
Sergei Glukhov also provided a patch, but that's not what's being used
here.
Reported-by: Sergei Glukhov
Reviewed-by: tender wang, Sergei Glukhov
Discussion: https://postgr.es/m/d05b26fa-af54-27e1-f693-6c31590802fa@postgrespro.ru
Backpatch-through: 11, where runtime partition pruning was added.
M src/backend/executor/execPartition.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Doc: fix grammatical errors for enable_partitionwise_aggregate
commit : 245e0f4e8de565ec87c1ec64bfd89d846f3d496e
author : David Rowley <drowley@postgresql.org>
date : Thu, 12 Oct 2023 21:16:22 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 12 Oct 2023 21:16:22 +1300
Author: Andrew Atkinson
Reviewed-by: Ashutosh Bapat
Discussion: https://postgr.es/m/CAG6XLEnC%3DEgq0YHRic2kWWDs4xwQnQ_kBA6qhhzAq1-pO_9Tfw%40mail.gmail.com
Backpatch-through: 11, where enable_partitionwise_aggregate was added
M doc/src/sgml/config.sgml
Fix incorrect step generation in HASH partition pruning
commit : 916adc7c50bbfe6e53d80dd66b61c580f0f3c89e
author : David Rowley <drowley@postgresql.org>
date : Thu, 12 Oct 2023 19:52:05 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 12 Oct 2023 19:52:05 +1300
get_steps_using_prefix_recurse() incorrectly assumed that it could stop
recursive processing of the 'prefix' list when cur_keyno was one before
the step_lastkeyno. Since hash partition pruning can prune using IS
NULL quals, and these IS NULL quals are not present in the 'prefix'
list, then that logic could cause more levels of recursion than what is
needed and lead to there being no more items in the 'prefix' list to
process. This would manifest itself as a crash in some code that
expected the 'start' ListCell not to be NULL.
Here we adjust the logic so that instead of stopping recursion at 1 key
before the step_lastkeyno, we just look at the llast(prefix) item and
ensure we only recursively process up until just before whichever the last
key is. This effectively allows keys to be missing in the 'prefix' list.
This change does mean that step_lastkeyno is no longer needed, so we
remove that from the static functions. I also spent quite some time
reading this code and testing it to try to convince myself that there
are no other issues. That resulted in the irresistible temptation of
rewriting some comments, many of which were just not true or inconcise.
Reported-by: Sergei Glukhov
Reviewed-by: Sergei Glukhov, tender wang
Discussion: https://postgr.es/m/2f09ce72-315e-2a33-589a-8519ada8df61@postgrespro.ru
Backpatch-through: 11, where partition pruning was introduced.
M src/backend/partitioning/partprune.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
doc: clarify that SSPI and GSSAPI are interchangeable
commit : 5143f764a74adb830b801ad8425ef78ee088573a
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 10 Oct 2023 16:51:08 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 10 Oct 2023 16:51:08 -0400
Reported-by: tpo_deb@sourcepole.ch
Discussion: https://postgr.es/m/167846222574.1803490.15815104179136215862@wrigleys.postgresql.org
Backpatch-through: 11
M doc/src/sgml/client-auth.sgml
doc: foreign servers with pushdown need matching collation
commit : 78691d1f98168206daf7e464a3e9f5cff4bbb7f6
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 10 Oct 2023 16:04:56 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 10 Oct 2023 16:04:56 -0400
Reported-by: Pete Storer
Discussion: https://postgr.es/m/BL0PR05MB66283C57D72E321591AE4EB1F3CE9@BL0PR05MB6628.namprd05.prod.outlook.com
Backpatch-through: 11
M doc/src/sgml/ref/create_server.sgml
doc: add SSL configuration section reference
commit : f20e0f82b99d3e7f7ff31e159c98d031acf24682
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 10 Oct 2023 15:54:28 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 10 Oct 2023 15:54:28 -0400
Reported-by: Steve Atkins
Discussion: https://postgr.es/m/B82E80DD-1452-4175-B19C-564FE46705BA@blighty.com
Backpatch-through: 11
M doc/src/sgml/client-auth.sgml
doc: document the need to analyze partitioned tables
commit : 119fd40dce3d3ce238631d4c86bc92e57babed90
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 10 Oct 2023 15:14:18 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 10 Oct 2023 15:14:18 -0400
Autovacuum does not do it.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210913035409.GA10647@telsasoft.com
Backpatch-through: 11
M doc/src/sgml/maintenance.sgml
Fix bug in GenericXLogFinish().
commit : b9bb02620d4b1d19bde03cbea1bc2a52eff3b380
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 10 Oct 2023 11:01:13 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 10 Oct 2023 11:01:13 -0700
Mark the buffers dirty before writing WAL.
Discussion: https://postgr.es/m/25104133-7df8-cae3-b9a2-1c0aaa1c094a@iki.fi
Reviewed-by: Heikki Linnakangas
Backpatch-through: 11
M src/backend/access/transam/generic_xlog.c
Doc: use CURRENT_USER not USER in plpgsql trigger examples.
commit : 6db384f2f94882743152234bece5fa99730def91
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Oct 2023 11:29:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Oct 2023 11:29:21 -0400
While these two built-in functions do exactly the same thing,
CURRENT_USER seems preferable to use in documentation examples.
It's easier to look up if the reader is unsure what it is.
Also, this puts these examples in sync with an adjacent example
that already used CURRENT_USER.
Per question from Kirk Parker.
Discussion: https://postgr.es/m/CANwZ8rmN_Eb0h0hoMRS8Feftaik0z89PxVsKg+cP+PctuOq=Qg@mail.gmail.com
M doc/src/sgml/plpgsql.sgml
Remove extra parenthesis from comment.
commit : bb912563059c789932efc423ad2092b7638249be
author : Etsuro Fujita <efujita@postgresql.org>
date : Fri, 6 Oct 2023 18:30:03 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Fri, 6 Oct 2023 18:30:03 +0900
M src/backend/storage/ipc/procarray.c
Fix memory leak in Memoize code
commit : 689af6db6c76a7966efdb702326f85fa0d8a04ef
author : David Rowley <drowley@postgresql.org>
date : Thu, 5 Oct 2023 20:31:51 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 5 Oct 2023 20:31:51 +1300
Ensure we switch to the per-tuple memory context to prevent any memory
leaks of detoasted Datums in MemoizeHash_hash() and MemoizeHash_equal().
Reported-by: Orlov Aleksej
Author: Orlov Aleksej, David Rowley
Discussion: https://postgr.es/m/83281eed63c74e4f940317186372abfd%40cft.ru
Backpatch-through: 14, where Memoize was added
M src/backend/executor/nodeMemoize.c
Avoid memory size overflow when allocating backend activity buffer
commit : 95e91da66cf5bbaf46c34c9386c93004f28d6719
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 3 Oct 2023 15:37:19 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 3 Oct 2023 15:37:19 +0900
The code in charge of copying the contents of PgBackendStatus to local
memory could fail on memory allocation because of an overflow on the
amount of memory to use. The overflow can happen when combining a high
value track_activity_query_size (max at 1MB) with a large
max_connections, when both multiplied get higher than INT32_MAX as both
parameters treated as signed integers. This could for example trigger
with the following functions, all calling pgstat_read_current_status():
- pg_stat_get_backend_subxact()
- pg_stat_get_backend_idset()
- pg_stat_get_progress_info()
- pg_stat_get_activity()
- pg_stat_get_db_numbackends()
The change to use MemoryContextAllocHuge() has been introduced in
8d0ddccec636, so backpatch down to 12.
Author: Jakub Wartak
Discussion: https://postgr.es/m/CAKZiRmw8QSNVw2qNK-dznsatQqz+9DkCquxP0GHbbv1jMkGHMA@mail.gmail.com
Backpatch-through: 12
M src/backend/utils/activity/backend_status.c
Fail hard on out-of-memory failures in xlogreader.c
commit : afc79591de8e2facc3e22f8746ee2b7e59a2a741
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 3 Oct 2023 10:25:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 3 Oct 2023 10:25:14 +0900
This commit changes the WAL reader routines so as a FATAL for the
backend or exit(FAILURE) for the frontend is triggered if an allocation
for a WAL record decode fails in walreader.c, rather than treating this
case as bogus data, which would be equivalent to the end of WAL. The
key is to avoid palloc_extended(MCXT_ALLOC_NO_OOM) in walreader.c,
relying on plain palloc() calls.
The previous behavior could make WAL replay finish too early than it
should. For example, crash recovery finishing earlier may corrupt
clusters because not all the WAL available locally was replayed to
ensure a consistent state. Out-of-memory failures would show up
randomly depending on the memory pressure on the host, but one simple
case would be to generate a large record, then replay this record after
downsizing a host, as Ethan Mertz originally reported.
This relies on bae868caf222, as the WAL reader routines now do the
memory allocation required for a record only once its header has been
fully read and validated, making xl_tot_len trustable. Making the WAL
reader react differently on out-of-memory or bogus record data would
require ABI changes, so this is the safest choice for stable branches.
Also, it is worth noting that 3f1ce973467a has been using a plain
palloc() in this code for some time now.
Thanks to Noah Misch and Thomas Munro for the discussion.
Like the other commit, backpatch down to 12, leaving out v11 that will
be EOL'd soon. The behavior of considering a failed allocation as bogus
data comes originally from 0ffe11abd3a0, where the record length
retrieved from its header was not entirely trustable.
Reported-by: Ethan Mertz
Discussion: https://postgr.es/m/ZRKKdI5-RRlta3aF@paquier.xyz
Backpatch-through: 12
M src/backend/access/transam/xlogreader.c
Fix omission of column-level privileges in selective pg_restore.
commit : 10e705bd2445ae5a7276b1fa3e1c52c6be522055
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Oct 2023 13:27:51 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Oct 2023 13:27:51 -0400
In a selective restore, ACLs for a table should be dumped if the
table is selected to be dumped. However, if the table has both
table-level and column-level ACLs, only the table-level ACL was
restored. This happened because _tocEntryRequired assumed that
an ACL could have only one dependency (the one on its table),
and punted if there was more than one. But since commit ea9125304,
column-level ACLs also depend on the table-level ACL if any, to
ensure correct ordering in parallel restores. To fix, adjust the
logic in _tocEntryRequired to ignore dependencies on ACLs.
I extended a test case in 002_pg_dump.pl so that it purports to
test for this; but in fact the test passes even without the fix.
That's because this bug only manifests during a selective restore,
while the scenarios 002_pg_dump.pl tests include only selective dumps.
Perhaps somebody would like to extend the script so that it can test
scenarios including selective restore, but I'm not touching that.
Euler Taveira and Tom Lane, per report from Kong Man.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/DM4PR11MB73976902DBBA10B1D652F9498B06A@DM4PR11MB7397.namprd11.prod.outlook.com
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/t/002_pg_dump.pl
Flush WAL stats in bgwriter
commit : 0684d1949fb43ec21480db627cd13a282ca41f6e
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 2 Oct 2023 12:39:35 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 2 Oct 2023 12:39:35 +0300
bgwriter can write out WAL, but did not flush the WAL pgstat counters,
so the writes were not seen in pg_stat_wal.
Back-patch to v14, where pg_stat_wal was introduced.
Author: Nazir Bilal Yavuz
Reviewed-by: Matthias van de Meent, Kyotaro Horiguchi
Discussion: https://www.postgresql.org/message-id/CAN55FZ2FPYngovZstr%3D3w1KSEHe6toiZwrurbhspfkXe5UDocg%40mail.gmail.com
M src/backend/postmaster/bgwriter.c
Fix datalen calculation in tsvectorrecv().
commit : 55e188a15e8ab0fdafca8cc612f9e0d1a1802bde
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Oct 2023 13:16:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Oct 2023 13:16:47 -0400
After receiving position data for a lexeme, tsvectorrecv()
advanced its "datalen" value by (npos+1)*sizeof(WordEntry)
where the correct calculation is (npos+1)*sizeof(WordEntryPos).
This accidentally failed to render the constructed tsvector
invalid, but it did result in leaving some wasted space
approximately equal to the space consumed by the position data.
That could have several bad effects:
* Disk space is wasted if the received tsvector is stored into a
table as-is.
* A legal tsvector could get rejected with "maximum total lexeme
length exceeded" if the extra space pushes it over the MAXSTRPOS
limit.
* In edge cases, the finished tsvector could be assigned a length
larger than the allocated size of its palloc chunk, conceivably
leading to SIGSEGV when the tsvector gets copied somewhere else.
The odds of a field failure of this sort seem low, though valgrind
testing could probably have found this.
While we're here, let's express the calculation as
"sizeof(uint16) + npos * sizeof(WordEntryPos)" to avoid the type
pun implicit in the "npos + 1" formulation. It's not wrong
given that WordEntryPos had better be 2 bytes to avoid padding
problems, but it seems clearer this way.
Report and patch by Denis Erokhin. Back-patch to all supported
versions.
Discussion: https://postgr.es/m/009801d9f2d9$f29730c0$d7c59240$@datagile.ru
M src/backend/utils/adt/tsvector.c
In COPY FROM, fail cleanly when unsupported encoding conversion is needed.
commit : 95fd5c89ffa18f6958f0135013046f798cced72b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Oct 2023 12:09:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Oct 2023 12:09:26 -0400
In recent releases, such cases fail with "cache lookup failed for
function 0" rather than complaining that the conversion function
doesn't exist as prior versions did. Seems to be a consequence of
sloppy refactoring in commit f82de5c46. Add the missing error check.
Per report from Pierre Fortin. Back-patch to v14 where the
oversight crept in.
Discussion: https://postgr.es/m/20230929163739.3bea46e5.pfortin@pfortin.com
M src/backend/commands/copyfrom.c
Fix briefly showing old progress stats for ANALYZE on inherited tables.
commit : 5ae245664830219bab2e2ada11d2379875f4501a
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sat, 30 Sep 2023 17:03:50 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sat, 30 Sep 2023 17:03:50 +0300
ANALYZE on a table with inheritance children analyzes all the child
tables in a loop. When stepping to next child table, it updated the
child rel ID value in the command progress stats, but did not reset
the 'sample_blks_total' and 'sample_blks_scanned' counters.
acquire_sample_rows() updates 'sample_blks_total' as soon as the scan
starts and 'sample_blks_scanned' after processing the first block, but
until then, pg_stat_progress_analyze would display a bogus combination
of the new child table relid with old counter values from the
previously processed child table. Fix by resetting 'sample_blks_total'
and 'sample_blks_scanned' to zero at the same time that
'current_child_table_relid' is updated.
Backpatch to v13, where pg_stat_progress_analyze view was introduced.
Reported-by: Justin Pryzby
Discussion: https://www.postgresql.org/message-id/20230122162345.GP13860%40telsasoft.com
M src/backend/commands/analyze.c
Fix EvalPlanQual rechecking during MERGE.
commit : 3c1a1af91d9504e4ce0319f48e83f7c2d5765969
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sat, 30 Sep 2023 10:55:24 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sat, 30 Sep 2023 10:55:24 +0100
Under some circumstances, concurrent MERGE operations could lead to
inconsistent results, that varied according the plan chosen. This was
caused by a lack of rowmarks on the source relation, which meant that
EvalPlanQual rechecking was not guaranteed to return the same source
tuples when re-running the join query.
Fix by ensuring that preprocess_rowmarks() sets up PlanRowMarks for
all non-target relations used in MERGE, in the same way that it does
for UPDATE and DELETE.
Per bug #18103. Back-patch to v15, where MERGE was introduced.
Dean Rasheed, reviewed by Richard Guo.
Discussion: https://postgr.es/m/18103-c4386baab8e355e3%40postgresql.org
M src/backend/executor/README
M src/backend/executor/nodeModifyTable.c
M src/backend/optimizer/plan/planner.c
M src/include/nodes/execnodes.h
M src/include/nodes/plannodes.h
A src/test/isolation/expected/merge-join.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/merge-join.spec
M src/test/regress/expected/merge.out
M src/test/regress/expected/with.out
Remove environment sensitivity in pl/tcl regression test.
commit : ef595bf74459aac3d944714719bd33b87fbc0df1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 29 Sep 2023 20:20:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 29 Sep 2023 20:20:57 -0400
Add "-gmt 1" to our test invocations of the Tcl "clock" command,
so that they do not consult the timezone environment. While it
doesn't really matter which timezone is used here, it does
matter that the command not fall over entirely. We've now
discovered that at least on FreeBSD, "clock scan" will fail if
/etc/localtime is missing. It seems worth making the test
insensitive to that.
Per Tomas Vondras' buildfarm animal dikkop. Thanks to
Thomas Munro for the diagnosis.
Discussion: https://postgr.es/m/316d304a-1dcd-cea1-3d6c-27f794727a06@enterprisedb.com
M src/pl/tcl/expected/pltcl_setup.out
M src/pl/tcl/sql/pltcl_setup.sql
Suppress macOS warnings about duplicate libraries in link commands.
commit : be3398ea155c263d23486365e9f56074c79211c7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 29 Sep 2023 14:07:30 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 29 Sep 2023 14:07:30 -0400
As of Xcode 15 (macOS Sonoma), the linker complains about duplicate
references to the same library. We see warnings about libpgport and
libpgcommon being duplicated in many client executables. This is a
consequence of the hack introduced in commit 6b7ef076b to list
libpgport before libpq while not removing it from $(LIBS).
(Commit 8396447cd later applied the same rule to libpgcommon.)
The concern in 6b7ef076b was to ensure that the client executable
wouldn't unintentionally depend on pgport functions from libpq.
That concern is obsolete on any platform for which we can do symbol
export control, because if we can then the pgport functions in libpq
won't be exposed anyway. Hence, we can fix this problem by just
removing libpgport and libpgcommon from $(libpq_pgport), and letting
clients depend on the occurrences in $(LIBS).
In the back branches, do that only on macOS (which we know has
symbol export control). In HEAD, let's be more aggressive and
remove the extra libraries everywhere. The only still-supported
platforms that lack export control are MinGW/Cygwin, and it
doesn't seem worth sweating over ABI stability details for those
(or if somebody does care, it'd probably be possible to perform
symbol export control for those too). As well as being simpler,
this might give some microscopic improvement in build time.
The meson build system is not changed here, as it doesn't have
this particular disease, though it does have some related issues
that we'll fix separately.
Discussion: https://postgr.es/m/467042.1695766998@sss.pgh.pa.us
M src/Makefile.global.in
Doc: improve description of dump/restore's --clean and --if-exists.
commit : f8b55e845069d85203d7b727cf8a81a99df03d90
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 29 Sep 2023 13:13:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 29 Sep 2023 13:13:54 -0400
Try to make these option descriptions a little clearer for novices.
Per gripe from Attila GulyƔs.
Discussion: https://postgr.es/m/169590536647.3727336.11070254203649648453@wrigleys.postgresql.org
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_restore.sgml
doc: Change statistics function xref to the right target
commit : 231e6b98fc3e23f6216ddde086505d7bc6683187
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 29 Sep 2023 15:55:37 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 29 Sep 2023 15:55:37 +0200
Commit 7d3b7011b added a link to the statistics functions, which at the
time were anchored under the section for statistics views. aebe989477a
added a separate section for statistics functions, but the link was not
updated to point to the new anchor. Fix by changing the xref.
Backpatch to all supported branches.
Author: Peter Smith <peter.b.smith@fujitsu.com>
Discussion: https://postgr.es/m/CAHut+Ptr0jKzNNtWnssLq+3jNhbyaBseqf6NPrWHk08mQFRoTg@mail.gmail.com
Backpatch-through: 11
M doc/src/sgml/func.sgml
Fix btmarkpos/btrestrpos array key wraparound bug.
commit : cac37c1a1b7c64e60832b764e3828a9aca95bc48
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 28 Sep 2023 16:29:32 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 28 Sep 2023 16:29:32 -0700
nbtree's mark/restore processing failed to correctly handle an edge case
involving array key advancement and related search-type scan key state.
Scans with ScalarArrayScalarArrayOpExpr quals requiring mark/restore
processing (for a merge join) could incorrectly conclude that an
affected array/scan key must not have advanced during the time between
marking and restoring the scan's position.
As a result of all this, array key handling within btrestrpos could skip
a required call to _bt_preprocess_keys(). This confusion allowed later
primitive index scans to overlook tuples matching the true current array
keys. The scan's search-type scan keys would still have spurious values
corresponding to the final array element(s) -- not values matching the
first/now-current array element(s).
To fix, remember that "array key wraparound" has taken place during the
ongoing btrescan in a flag variable stored in the scan's state, and use
that information at the point where btrestrpos decides if another call
to _bt_preprocess_keys is required.
Oversight in commit 70bc5833, which taught nbtree to handle array keys
during mark/restore processing, but missed this subtlety. That commit
was itself a bug fix for an issue in commit 9e8da0f7, which taught
nbtree to handle ScalarArrayOpExpr quals natively.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WzkgP3DDRJxw6DgjCxo-cu-DKrvjEv_ArkP2ctBJatDCYg@mail.gmail.com
Backpatch: 11- (all supported branches).
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtutils.c
M src/include/access/nbtree.h
Fix checking of index expressions in CompareIndexInfo().
commit : 59371f1aeb9a6a3d296c0278dba4e4a8b9a53f65
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Sep 2023 14:05:25 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Sep 2023 14:05:25 -0400
This code was sloppy about comparison of index columns that
are expressions. It didn't reliably reject cases where one
index has an expression where the other has a plain column,
and it could index off the start of the attmap array, leading
to a Valgrind complaint (though an actual crash seems unlikely).
I'm not sure that the expression-vs-column sloppiness leads
to any visible problem in practice, because the subsequent
comparison of the two expression lists would reject cases
where the indexes have different numbers of expressions
overall. Maybe we could falsely match indexes having the
same expressions in different column positions, but it'd
require unlucky contents of the word before the attmap array.
It's not too surprising that no problem has been reported
from the field. Nonetheless, this code is clearly wrong.
Per bug #18135 from Alexander Lakhin. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/18135-532f4a755e71e4d2@postgresql.org
M src/backend/catalog/index.c
Add missing TidRangePath handling in print_path()
commit : a0bfd65993948f5d299e37f33c658cdee79bb237
author : David Rowley <drowley@postgresql.org>
date : Fri, 29 Sep 2023 00:03:26 +1300
committer: David Rowley <drowley@postgresql.org>
date : Fri, 29 Sep 2023 00:03:26 +1300
Tid Range scans were added back in bb437f995. That commit forgot to add
handling for TidRangePaths in print_path().
Only people building with OPTIMIZER_DEBUG might have noticed this, which
likely is the reason it's taken 4 years for anyone to notice.
Author: Andrey Lepikhov
Reported-by: Andrey Lepikhov
Discussion: https://postgr.es/m/379082d6-1b6a-4cd6-9ecf-7157d8c08635@postgrespro.ru
Backpatch-through: 14, where bb437f995 was introduced
M src/backend/optimizer/path/allpaths.c
Fix typo in src/backend/access/transam/README.
commit : 0db39e633812e8bda24067904ee8bc262e9eae25
author : Etsuro Fujita <efujita@postgresql.org>
date : Thu, 28 Sep 2023 19:45:04 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Thu, 28 Sep 2023 19:45:04 +0900
M src/backend/access/transam/README
unaccent: Tweak value of PYTHON when building without Python support
commit : 8a9e4e84e3cf75a01df5bf26fd35ffc9576b40f1
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 27 Sep 2023 14:41:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 27 Sep 2023 14:41:21 +0900
As coded, the module's Makefile would fail to set a value for PYTHON as
it checked if the variable is defined. When compiling without
--with-python, PYTHON is defined and set to an empty value, so the
existing check is not able to do its work.
This commit switches the rule to check if the value is empty rather than
defined, allowing the generation of unaccent.rules even if --with-python
is not used as long as "python" exists. BISON and FLEX do the same in
pgxs.mk, for instance.
Thinko in f85a485f89e2.
Author: Japin Li
Discussion: https://postgr.es/m/MEYP282MB1669F86C0DC7B4DC48489CB0B6C3A@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
Backpatch-through: 13
M contrib/unaccent/Makefile
Stop using "-multiply_defined suppress" on macOS.
commit : 78f17fb97a7e404d577778f048ec0d6da0ef2404
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 26 Sep 2023 21:06:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 26 Sep 2023 21:06:21 -0400
We started to use this linker switch in commit 9df308697 of
2004-07-13, which was in the OS X 10.3 era. Apparently it's been a
no-op since around OS X 10.9. Apple's most recent toolchain version
actively complains about it, so it's time to get rid of it.
Discussion: https://postgr.es/m/467042.1695766998@sss.pgh.pa.us
M src/Makefile.shlib
doc: clarify the effect of concurrent work_mem allocations
commit : c3f7e91a191bfce657ed27ea59781fb427c46c06
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 26 Sep 2023 19:44:22 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 26 Sep 2023 19:44:22 -0400
Reported-by: Sami Imseih
Discussion: https://postgr.es/m/66590882-F48C-4A25-83E3-73792CF8C51F@amazon.com
Backpatch-through: 11
M doc/src/sgml/config.sgml
doc: clarify handling of time zones with "time with time zone"
commit : 7bb0f4a25d1b2bff7f2938176cd59a704c625b86
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 26 Sep 2023 19:23:59 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 26 Sep 2023 19:23:59 -0400
Reported-by: davecramer@postgres.rocks
Discussion: https://postgr.es/m/168451942371.714.9173574930845904336@wrigleys.postgresql.org
Backpatch-through: 11
M doc/src/sgml/datatype.sgml
doc: clarify the behavior of unopenable listen_addresses
commit : cf93f173c1eee21298f21633b83e06c4cacab9ea
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 26 Sep 2023 19:02:18 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 26 Sep 2023 19:02:18 -0400
Reported-by: Gurjeet Singh
Discussion: https://postgr.es/m/CABwTF4WYPD9ov-kcSq1+J+ZJ5wYDQLXquY6Lu2cvb-Y7pTpSGA@mail.gmail.com
Backpatch-through: 11
M doc/src/sgml/config.sgml
doc: pg_upgrade, clarify standby servers must remain running
commit : 29dec9f04313fff014a43941edd1ebfeaeecb20e
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 26 Sep 2023 18:54:10 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 26 Sep 2023 18:54:10 -0400
Also mention that mismatching primary/standby LSNs should never
happen.
Reported-by: Nikolay Samokhvalov
Discussion: https://postgr.es/m/CAM527d8heqkjG5VrvjU3Xjsqxg41ufUyabD9QZccdAxnpbRH-Q@mail.gmail.com
Backpatch-through: 11
M doc/src/sgml/ref/pgupgrade.sgml
doc: mention GROUP BY columns can reference target col numbers
commit : 3a9a1b821e07f1d223d2161e46286ee8d134163c
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 26 Sep 2023 17:31:06 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 26 Sep 2023 17:31:06 -0400
Reported-by: hape <postgres-hape@gmx.de>
Discussion: https://postgr.es/m/168871536004.379168.9352636188330923805@wrigleys.postgresql.org
Backpatch-through: 11
M doc/src/sgml/ref/select.sgml
Fix another bug in parent page splitting during GiST index build.
commit : c3c284b379555df411a17f21a7c34b416c56e462
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 26 Sep 2023 14:14:49 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 26 Sep 2023 14:14:49 +0300
Yet another bug in the ilk of commits a7ee7c851 and 741b88435. In
741b88435, we took care to clear the memorized location of the
downlink when we split the parent page, because splitting the parent
page can move the downlink. But we missed that even *updating* a tuple
on the parent can move it, because updating a tuple on a gist page is
implemented as a delete+insert, so the updated tuple gets moved to the
end of the page.
This commit fixes the bug in two different ways (belt and suspenders):
1. Clear the downlink when we update a tuple on the parent page, even
if it's not split. This the same approach as in commits a7ee7c851
and 741b88435.
I also noticed that gistFindCorrectParent did not clear the
'downlinkoffnum' when it stepped to the right sibling. Fix that
too, as it seems like a clear bug even though I haven't been able
to find a test case to hit that.
2. Change gistFindCorrectParent so that it treats 'downlinkoffnum'
merely as a hint. It now always first checks if the downlink is
still at that location, and if not, it scans the page like before.
That's more robust if there are still more cases where we fail to
clear 'downlinkoffnum' that we haven't yet uncovered. With this,
it's no longer necessary to meticulously clear 'downlinkoffnum',
so this makes the previous fixes unnecessary, but I didn't revert
them because it still seems nice to clear it when we know that the
downlink has moved.
Also add the test case using the same test data that Alexander
posted. I tried to reduce it to a smaller test, and I also tried to
reproduce this with different test data, but I was not able to, so
let's just include what we have.
Backpatch to v12, like the previous fixes.
Reported-by: Alexander Lakhin
Discussion: https://www.postgresql.org/message-id/18129-caca016eaf0c3702@postgresql.org
M contrib/intarray/expected/_int.out
M contrib/intarray/sql/_int.sql
M src/backend/access/gist/gist.c
Fix behavior of "force" in pgstat_report_wal()
commit : 802fcb9ed16672dcc179977c9dc561fa12afae7b
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 26 Sep 2023 09:30:39 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 26 Sep 2023 09:30:39 +0900
As implemented in 5891c7a8ed8f, setting "force" to true in
pgstat_report_wal() causes the routine to not wait for the pgstat
shmem lock if it cannot be acquired, in which case the WAL and I/O
statistics finish by not being flushed. The origin of the confusion
comes from pgstat_flush_wal() and pgstat_flush_io(), that use "nowait"
as sole argument. The I/O stats are new in v16.
This is the opposite behavior of what has been used in
pgstat_report_stat(), where "force" is the opposite of "nowait". In
this case, when "force" is true, the routine sets "nowait" to false,
which would cause the routine to wait for the pgstat shmem lock,
ensuring that the stats are always flushed. When "force" is false,
"nowait" is set to true, and the stats would only not be flushed if the
pgstat shmem lock can be acquired, returning immediately without
flushing the stats if the lock cannot be acquired.
This commit changes pgstat_report_wal() so as "force" has the same
behavior as in pgstat_report_stat(). There are currently three callers
of pgstat_report_wal():
- Two in the checkpointer where force=true during a shutdown and the
main checkpointer loop. Now the code behaves so as the stats are always
flushed.
- One in the main loop of the bgwriter, where force=false. Now the code
behaves so as the stats would not be flushed if the pgstat shmem lock
could not be acquired.
Before this commit, some stats on WAL and I/O could have been lost after
a shutdown, for example.
Reported-by: Ryoga Yoshida
Author: Ryoga Yoshida, Michael Paquier
Discussion: https://postgr.es/m/f87a4d7be70530606b864fd1df91718c@oss.nttdata.com
Backpatch-through: 15
M src/backend/utils/activity/pgstat_wal.c
doc: Tell about "vcregress taptest" for regression tests on Windows
commit : 3c065d5a7ecd61ba3b896be300aa0d98ad6993a2
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 26 Sep 2023 08:16:43 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 26 Sep 2023 08:16:43 +0900
There was no mention of this command in the documentation, and it is
useful to run the TAP tests of a target source directory.
Author: Yugo Nagata
Discussion: https://postgr.es/m/20230925153204.926d685d347ee1c8f527090c@sraoss.co.jp
Backpatch-through: 11
M doc/src/sgml/install-windows.sgml
Fix edge-case for xl_tot_len broken by bae868ca.
commit : 99d334a187ae5c283aca1c31b9fad0cea140a4b2
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 26 Sep 2023 09:07:26 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 26 Sep 2023 09:07:26 +1300
bae868ca removed a check that was still needed. If you had an
xl_tot_len at the end of a page that was too small for a record header,
but not big enough to span onto the next page, we'd immediately perform
the CRC check using a bogus large length. Because of arbitrary coding
differences between the CRC implementations on different platforms,
nothing very bad happened on common modern systems. On systems using
the _sb8.c fallback we could segfault.
Restore that check, add a new assertion and supply a test for that case.
Back-patch to 12, like bae868ca.
Tested-by: Tom Lane <tgl@sss.pgh.pa.us>
Tested-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGLCkTT7zYjzOxuLGahBdQ%3DMcF%3Dz5ZvrjSOnW4EDhVjT-g%40mail.gmail.com
M src/backend/access/transam/xlogreader.c
M src/test/recovery/t/039_end_of_wal.pl
pg_dump: tests: Correct test condition for invalid databases
commit : 9dc3c5472ed99a1046a4d7884270a35cfad29d1d
author : Andres Freund <andres@anarazel.de>
date : Mon, 25 Sep 2023 11:50:02 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 25 Sep 2023 11:50:02 -0700
For some reason I used not_like = { pg_dumpall_dbprivs => 1, } in the test
condition of one of the tests added in in c66a7d75e65. That doesn't make sense
for two reasons: 1) not_like isn't a valid test condition 2) the database
should not be dumped in any of the tests. Due to 1), the test achieved its
goal, but clearly the formulation is confusing. Instead use like => {}, with
a comment explaining why.
Reported-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/3ddf79f2-8b7b-a093-11d2-5c739bc64f86@eisentraut.org
Backpatch: 11-, like c66a7d75e65
M src/bin/pg_dump/t/002_pg_dump.pl
Collect dependency information for parsed CallStmts.
commit : 0e59266a5daf31ff5c9976dae1b328b6a0e52b3e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Sep 2023 14:41:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Sep 2023 14:41:57 -0400
Parse analysis of a CallStmt will inject mutable information,
for instance the OID of the called procedure, so that subsequent
DDL may create a need to re-parse the CALL. We failed to detect
this for CALLs in plpgsql routines, because no dependency information
was collected when putting a CallStmt into the plan cache. That
could lead to misbehavior or strange errors such as "cache lookup
failed".
Before commit ee895a655, the issue would only manifest for CALLs
appearing in atomic contexts, because we re-planned non-atomic
CALLs every time through anyway.
It is now apparent that extract_query_dependencies() probably
needs a special case for every utility statement type for which
stmt_requires_parse_analysis() returns true. I wanted to add
something like Assert(!stmt_requires_parse_analysis(...)) when
falling out of extract_query_dependencies_walker without doing
anything, but there are API issues as well as a more fundamental
point: stmt_requires_parse_analysis is supposed to be applied to
raw parser output, so it'd be cheating to assume it will give the
correct answer for post-parse-analysis trees. I contented myself
with adding a comment.
Per bug #18131 from Christian Stork. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/18131-576854e79c5cd264@postgresql.org
M src/backend/optimizer/plan/setrefs.c
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/sql/plpgsql_call.sql
Limit to_tsvector_byid's initial array allocation to something sane.
commit : 71bb73f60be71085a8f3094a64d0883cef3c0afc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Sep 2023 11:50:28 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Sep 2023 11:50:28 -0400
The initial estimate of the number of distinct ParsedWords is just
that: an estimate. Don't let it exceed what palloc is willing to
allocate. If in fact we need more entries, we'll eventually fail
trying to enlarge the array. But if we don't, this allows success on
inputs that currently draw "invalid memory alloc request size".
Per bug #18080 from Uwe Binder. Back-patch to all supported branches.
Discussion: https://postgr.es/m/18080-d5c5e58fef8c99b7@postgresql.org
M src/backend/tsearch/to_tsany.c
pg_upgrade: check for types removed in pg12
commit : 8845d8597d7725690cecbd6282b3cb1a70000f72
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 25 Sep 2023 14:34:06 +0200
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 25 Sep 2023 14:34:06 +0200
Commit cda6a8d01d39 removed a few datatypes, but didn't update
pg_upgrade --check to throw error if these types are used. So the users
find that pg_upgrade --check tells them that everything is fine, only to
fail when the real upgrade is attempted.
Reviewed-by: Tristan Partin <tristan@neon.tech>
Reviewed-by: Suraj Kharage <suraj.kharage@enterprisedb.com>
Discussion: https://postgr.es/m/202309201654.ng4ksea25mti@alvherre.pgsql
M src/bin/pg_upgrade/check.c
Don't use Perl pack('Q') in 039_end_of_wal.pl.
commit : 21b4c3ca0b22761eccdc55c265a12c6296c10068
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 23 Sep 2023 14:13:06 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 23 Sep 2023 14:13:06 +1200
'Q' for 64 bit integers turns out not to work on 32 bit Perl, as
revealed by the build farm. Use 'II' instead, and deal with endianness.
Back-patch to 12, like bae868ca.
Discussion: https://postgr.es/m/ZQ4r1vHcryBsSi_V%40paquier.xyz
M src/test/recovery/t/039_end_of_wal.pl
Don't trust unvalidated xl_tot_len.
commit : f4d152edd8f355e96b30d4bbf9c6a9cb7a20c38f
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 23 Sep 2023 10:26:24 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 23 Sep 2023 10:26:24 +1200
xl_tot_len comes first in a WAL record. Usually we don't trust it to be
the true length until we've validated the record header. If the record
header was split across two pages, previously we wouldn't do the
validation until after we'd already tried to allocate enough memory to
hold the record, which was bad because it might actually be garbage
bytes from a recycled WAL file, so we could try to allocate a lot of
memory. Release 15 made it worse.
Since 70b4f82a4b5, we'd at least generate an end-of-WAL condition if the
garbage 4 byte value happened to be > 1GB, but we'd still try to
allocate up to 1GB of memory bogusly otherwise. That was an
improvement, but unfortunately release 15 tries to allocate another
object before that, so you could get a FATAL error and recovery could
fail.
We can fix both variants of the problem more fundamentally using
pre-existing page-level validation, if we just re-order some logic.
The new order of operations in the split-header case defers all memory
allocation based on xl_tot_len until we've read the following page. At
that point we know that its first few bytes are not recycled data, by
checking its xlp_pageaddr, and that its xlp_rem_len agrees with
xl_tot_len on the preceding page. That is strong evidence that
xl_tot_len was truly the start of a record that was logged.
This problem was most likely to occur on a standby, because
walreceiver.c recycles WAL files without zeroing out trailing regions of
each page. We could fix that too, but that wouldn't protect us from
rare crash scenarios where the trailing zeroes don't make it to disk.
With reliable xl_tot_len validation in place, the ancient policy of
considering malloc failure to indicate corruption at end-of-WAL seems
quite surprising, but changing that is left for later work.
Also included is a new TAP test to exercise various cases of end-of-WAL
detection by writing contrived data into the WAL from Perl.
Back-patch to 12. We decided not to put this change into the final
release of 11.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Michael Paquier <michael@paquier.xyz>
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com> (the idea, not the code)
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Sergei Kornilov <sk@zsrv.org>
Reviewed-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/17928-aa92416a70ff44a2%40postgresql.org
M src/backend/access/transam/xlogreader.c
M src/test/perl/PostgreSQL/Test/Utils.pm
A src/test/recovery/t/039_end_of_wal.pl
Doc: copy-edit the introductory para for the pg_class catalog.
commit : bd82b8d9b21364be5e41dae98520f76e23a6c8e2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Sep 2023 14:52:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Sep 2023 14:52:36 -0400
The previous wording had a faint archaic whiff to it, and more
importantly used "catalogs" as a verb, which while cutely
self-referential seems likely to provoke confusion in this
particular context. Also consistently use "kind" not "type" to
refer to the different kinds of relations distinguished by relkind.
Per gripe from Martin Nash. Back-patch to supported versions.
Discussion: https://postgr.es/m/169518739902.3727338.4793815593763320945@wrigleys.postgresql.org
M doc/src/sgml/catalogs.sgml
Avoid potential pfree on NULL on OpenSSL errors
commit : 9dc85806d8beb1d4b0cc5461cc3a7d2e44c6e782
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 22 Sep 2023 11:18:25 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 22 Sep 2023 11:18:25 +0200
Guard against the pointer being NULL before pfreeing upon an error
returned from OpenSSL. Also handle errors from X509_NAME_print_ex
which also can return -1 on memory allocation errors.
Backpatch down to v15 where the code was added.
Author: Sergey Shinderuk <s.shinderuk@postgrespro.ru>
Discussion: https://postgr.es/m/8db5374d-32e0-6abb-d402-40762511eff2@postgrespro.ru
Backpatch-through: v15
M src/backend/libpq/be-secure-openssl.c
Fix COMMIT/ROLLBACK AND CHAIN in the presence of subtransactions.
commit : 77dc8160278925f453a1b7560326021f971b9b74
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 21 Sep 2023 23:11:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 21 Sep 2023 23:11:31 -0400
In older branches, COMMIT/ROLLBACK AND CHAIN failed to propagate
the current transaction's properties to the new transaction if
there was any open subtransaction (unreleased savepoint).
Instead, some previous transaction's properties would be restored.
This is because the "if (s->chain)" check in CommitTransactionCommand
examined the wrong instance of the "chain" flag and falsely
concluded that it didn't need to save transaction properties.
Our regression tests would have noticed this, except they used
identical transaction properties for multiple tests in a row,
so that the faulty behavior was not distinguishable from correct
behavior.
Commit 12d768e70 fixed the problem in v15 and later, but only rather
accidentally, because I removed the "if (s->chain)" test to avoid a
compiler warning, while not realizing that the warning was flagging a
real bug.
In v14 and before, remove the if-test and save transaction properties
unconditionally; just as in the newer branches, that's not expensive
enough to justify thinking harder.
Add the comment and extra regression test to v15 and later to
forestall any future recurrence, but there's no live bug in those
branches.
Patch by me, per bug #18118 from Liu Xiang. Back-patch to v12 where
the AND CHAIN feature was added.
Discussion: https://postgr.es/m/18118-4b72fcbb903aace6@postgresql.org
M src/backend/access/transam/xact.c
M src/test/regress/expected/transactions.out
M src/test/regress/sql/transactions.sql
Update comment about set_join_pathlist_hook().
commit : da421219e42900af13f2d432fbf6612464ac724c
author : Etsuro Fujita <efujita@postgresql.org>
date : Thu, 21 Sep 2023 19:45:04 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Thu, 21 Sep 2023 19:45:04 +0900
The comment introduced by commit e7cb7ee14 was a bit too terse, which
could lead to extensions doing different things within the hook function
than we intend to allow. Extend the comment to explain what they can do
within the hook function.
Back-patch to all supported branches.
In passing, I rephrased a nearby comment that I recently added to the
back branches.
Reviewed by David Rowley and Andrei Lepikhov.
Discussion: https://postgr.es/m/CAPmGK15SBPA1nr3Aqsdm%2BYyS-ay0Ayo2BRYQ8_A2To9eLqwopQ%40mail.gmail.com
M src/backend/optimizer/path/joinpath.c
Fix GiST README's explanation of the NSN cross-check.
commit : bd7a18e70714bd57c1375a93eb2e64378dbc7d99
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 19 Sep 2023 11:53:51 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 19 Sep 2023 11:53:51 +0300
The text got the condition backwards, it's "NSN > LSN", not "NSN < LSN".
While we're at it, expand it a little for clarity.
Reviewed-by: Daniel Gustafsson
Discussion: https://www.postgresql.org/message-id/4cb46e18-e688-524a-0f73-b1f03ed5d6ee@iki.fi
M src/backend/access/gist/README
Fix assertion failure with PL/Python exceptions
commit : 9e0ce80f3d0c217c33034984fce022a329d8961f
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 19 Sep 2023 08:31:24 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 19 Sep 2023 08:31:24 +0900
PLy_elog() was not able to handle correctly cases where a SPI called
failed, which would fill in a DETAIL string able to trigger an
assertion. We may want to improve this infrastructure so as it is able
to provide any extra detail information provided by an error stack, but
this is left as a future improvement as it could impact existing error
stacks and any applications that depend on them. For now, the assertion
is removed and a regression test is added to cover the case of a failure
with a detail string.
This problem exists since 2bd78eb8d51c, so backpatch all the way down
with tweaks to the regression tests output added where required.
Author: Alexander Lakhin
Discussion: https://postgr.es/m/18070-ab9c171cbf4ebb0f@postgresql.org
Backpatch-through: 11
M src/pl/plpython/expected/plpython_error.out
M src/pl/plpython/expected/plpython_error_5.out
M src/pl/plpython/plpy_elog.c
M src/pl/plpython/sql/plpython_error.sql
Don't crash if cursor_to_xmlschema is used on a non-data-returning Portal.
commit : 95f54f0d034eb69349f9134a01441a3cdd311419
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 18 Sep 2023 14:27:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 18 Sep 2023 14:27:47 -0400
cursor_to_xmlschema() assumed that any Portal must have a tupDesc,
which is not so. Add a defensive check.
It's plausible that this mistake occurred because of the rather
poorly chosen name of the lookup function SPI_cursor_find(),
which in such cases is returning something that isn't very much
like a cursor. Add some documentation to try to forestall future
errors of the same ilk.
Report and patch by Boyu Yang (docs changes by me). Back-patch
to all supported branches.
Discussion: https://postgr.es/m/dd343010-c637-434c-a8cb-418f53bda3b8.yangboyu.yby@alibaba-inc.com
M doc/src/sgml/spi.sgml
M src/backend/utils/adt/xml.c
Track nesting depth correctly when drilling down into RECORD Vars.
commit : 2679a107a152de3f88602a41f8651ec82e4d6205
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 15 Sep 2023 17:01:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 15 Sep 2023 17:01:26 -0400
expandRecordVariable() failed to adjust the parse nesting structure
correctly when recursing to inspect an outer-level Var. This could
result in assertion failures or core dumps in corner cases.
Likewise, get_name_for_var_field() failed to adjust the deparse
namespace stack correctly when recursing to inspect an outer-level
Var. In this case the likely result was a "bogus varno" error
while deparsing a view.
Per bug #18077 from Jingzhou Fu. Back-patch to all supported
branches.
Richard Guo, with some adjustments by me
Discussion: https://postgr.es/m/18077-b9db97c6e0ab45d8@postgresql.org
M src/backend/parser/parse_target.c
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/rowtypes.out
M src/test/regress/sql/rowtypes.sql
Revert "Improve error message on snapshot import in snapmgr.c"
commit : fe60434af76ffa5fe57711378b7af06ee2e918d3
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Sep 2023 16:00:39 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Sep 2023 16:00:39 +0900
This reverts commit a0d87bcd9b57, following a remark from Andres Frend
that the new error can be triggered with an incorrect SET TRANSACTION
SNAPSHOT command without being really helpful for the user as it uses
the internal file name.
Discussion: https://postgr.es/m/20230914020724.hlks7vunitvtbbz4@awork3.anarazel.de
Backpatch-through: 11
M src/backend/utils/time/snapmgr.c
Improve error message on snapshot import in snapmgr.c
commit : 19885ab246d81a787797e92d20c5e54414f1ddee
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Sep 2023 10:30:27 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Sep 2023 10:30:27 +0900
When a snapshot file fails to be read in ImportSnapshot(), it would
issue an ERROR as "invalid snapshot identifier" when opening a stream
for it in read-only mode. This error message is reworded to be the same
as all the other messages used in this case on failure, which is useful
when debugging this area.
Thinko introduced by bb446b689b66 where snapshot imports have been
added. A backpatch down to 11 is done as this can improve any work
related to snapshot imports in older branches.
Author: Bharath Rupireddy
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/CALj2ACWmr=3KdxDkm8h7Zn1XxBoF6hdzq8WQyMn2y1OL5RYFrg@mail.gmail.com
Backpatch-through: 11
M src/backend/utils/time/snapmgr.c
Fix incorrect logic in plan dependency recording
commit : 17a3f1c3475a4a31ea0af9c95efdccd07dd4247c
author : David Rowley <drowley@postgresql.org>
date : Thu, 14 Sep 2023 11:27:00 +1200
committer: David Rowley <drowley@postgresql.org>
date : Thu, 14 Sep 2023 11:27:00 +1200
Both 50e17ad28 and 29f45e299 mistakenly tried to record a plan dependency
on a function but mistakenly inverted the OidIsValid test. This meant
that we'd record a dependency only when the function's Oid was
InvalidOid. Clearly this was meant to *not* record the dependency in
that case.
50e17ad28 made this mistake first, then in v15 29f45e299 copied the same
mistake.
Reported-by: Tom Lane
Backpatch-through: 14, where 50e17ad28 first made this mistake
Discussion: https://postgr.es/m/2277537.1694301772@sss.pgh.pa.us
M src/backend/optimizer/plan/setrefs.c
Fix exception safety bug in typcache.c.
commit : a26cc0334f8b02c949b68d6ba9061eeb1fa75498
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 13 Sep 2023 14:32:24 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 13 Sep 2023 14:32:24 +1200
If an out-of-memory error was thrown at an unfortunate time,
ensure_record_cache_typmod_slot_exists() could leak memory and leave
behind a global state that produced an infinite loop on the next call.
Fix by merging RecordCacheArray and RecordIdentifierArray into a single
array. With only one allocation or re-allocation, there is no
intermediate state.
Back-patch to all supported releases.
Reported-by: "James Pang (chaolpan)" <chaolpan@cisco.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/PH0PR11MB519113E738814BDDA702EDADD6EFA%40PH0PR11MB5191.namprd11.prod.outlook.com
M src/backend/utils/cache/typcache.c
M src/tools/pgindent/typedefs.list
Skip psql's TAP test for query cancellation entirely on Windows
commit : 75b4f930d5db097e2db7ce5235476c1bf6f70da4
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 13 Sep 2023 09:53:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 13 Sep 2023 09:53:54 +0900
This changes 020_cancel.pl so as the test is entirely skipped on
Windows. This test was already doing nothing under WIN32, except
initializing and starting a node without using it so this shaves a few
test cycles.
Author: Yugo NAGATA
Reviewed-by: Fabien Coelho
Discussion: https://postgr.es/m/20230810125935.22c2922ea5250ba79358965b@sraoss.co.jp
Backpatch-through: 15
M src/bin/psql/t/020_cancel.pl
Fix uninitialized access to InitialRunningXacts during decoding after ERROR.
commit : c7256e6564fa580ecaad97b6a68e57652d4b09d4
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 12 Sep 2023 10:23:17 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 12 Sep 2023 10:23:17 +0530
The transactions and subtransactions array that was allocated under
snapshot builder memory context and recorded during decoding was not
cleared in case of errors. This can result in an assertion failure if we
attempt to retry logical decoding within the same session. To address this
issue, we register a callback function under the snapshot builder memory
context to clear the recorded transactions and subtransactions array along
with the context.
This problem doesn't exist in PG16 and HEAD as instead of using
InitialRunningXacts, we added the list of transaction IDs and
sub-transaction IDs, that have modified catalogs and are running during
snapshot serialization, to the serialized snapshot (see commit 7f13ac8123).
Author: Hou Zhijie
Reviewed-by: Amit Kapila
Backpatch-through: 11
Discussion: http://postgr.es/m/18055-ab3beed9f4b7b7d6@postgresql.org
M src/backend/replication/logical/snapbuild.c
doc: remove mention of backslash doubling in strings
commit : ad6508c104f746e56e0e86c03febecc8d66b0b98
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 8 Sep 2023 17:25:15 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 8 Sep 2023 17:25:15 -0400
Reported-by: Laurenz Albe
Discussion: https://postgr.es/m/0b03f91a875fb44182f5bed9e1d404ed6d138066.camel@cybertec.at
Author: Laurenz Albe
Backpatch-through: 11
M doc/src/sgml/syntax.sgml
Stabilize subscription stats test.
commit : 461a7fad7c670037d5308d043609462e2ff83951
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 8 Sep 2023 22:50:53 +0900
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 8 Sep 2023 22:50:53 +0900
The new test added by commit 68a59f9e9 disables the subscription and
manually drops the associated replication slot. However, since
disabling the subsubscription doesn't wait for a walsender to release
the replication slot and exit, pg_drop_replication_slot() could
fail. Avoid failure by adding a wait for the replication slot to
become inactive.
Reported-by: Hou Zhijie, as per buildfarm
Reviewed-by: Hou Zhijie
Discussion: https://postgr.es/m/OS0PR01MB571682316378379AA34854F694E9A%40OS0PR01MB5716.jpnprd01.prod.outlook.com
Backpatch-through: 15
M src/test/subscription/t/026_stats.pl
doc: Extend documentation of PG_TEST_EXTRA
commit : 87d6e0127acaf5563d22fcc310483f1d960c85c5
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 8 Sep 2023 11:34:43 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 8 Sep 2023 11:34:43 +0200
Extend the PG_TEST_EXTRA documentation to mention resource intensive
tests as well. The previous wording only mentioned special software
and security in the main paragraph, with resource usage listed on one
of the tests in the list.
Backpatch to v15 where f47ed79cc8 added wal_consistenct_checking as
a PG_TEST_EXTRA target.
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CAN55FZ0OthTuBdiNkaX2BvxuHdK4Y1MVEb8_uEuD1yHMPmT9Og@mail.gmail.com
Backpatch-through: 15
M doc/src/sgml/regress.sgml
pg_basebackup: Generate valid temporary slot names under PQbackendPID()
commit : 574bff7bd23cc95c60bff733fe1adc16787e2476
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 7 Sep 2023 14:12:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 7 Sep 2023 14:12:29 +0900
pgbouncer can cause PQbackendPID() to return negative values due to it
filling be_pid with random bytes (even these days pid_max can only be
set up to 2^22 on 64b machines on Linux, for example, so this cannot
happen with normal PID numbers). When this happens, pg_basebackup may
generate a temporary slot name that may not be accepted by the parser,
leading to spurious failures, like:
pg_basebackup: error: could not send replication command
ERROR: replication slot name "pg_basebackup_-1201966863" contains
invalid character
This commit fixes that problem by formatting the result from
PQbackendPID() as an unsigned integer when creating the temporary
replication slot name, so as the invalid character is gone and the
command can be parsed.
Author: Jelte Fennema
Reviewed-by: Daniel Gustafsson, Nishant Sharma
Discussion: https://postgr.es/m/CAGECzQQOGvYfp8ziF4fWQ_o8s2K7ppaoWBQnTmdakn3s-4Z=5g@mail.gmail.com
Backpatch-through: 11
M src/bin/pg_basebackup/pg_basebackup.c
Disable 031_recovery_conflict.pl in 15 and 16.
commit : e13de491391322e3d66251cc04dc7970e22c3484
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 7 Sep 2023 11:47:42 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 7 Sep 2023 11:47:42 +1200
This test fails due to known bugs in the test and the server. Those
will be fixed in master shortly and possibly back-patched a bit later,
but in the meantime it is unhelpful for package maintainers if the tests
randomly fail, and it's not a good time to make complex changes in 16.
This had already been done for older branches prior to 15's release.
Now we're about to release 16, and Debian's test builds are regularly
failing on one architecture, so let's do the same for 15 and 16.
Reported-by: Christoph Berg <myon@debian.org>
Reported-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Discussion: https://postgr.es/m/CALj2ACVr8au2J_9D88UfRCi0JdWhyQDDxAcSVav0B0irx9nXEg%40mail.gmail.com
M src/test/recovery/t/031_recovery_conflict.pl
doc: mention that to_char() values are rounded
commit : 18aecf5e15a6d6fb655a6db9041147704cd0b9cf
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 6 Sep 2023 16:52:24 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 6 Sep 2023 16:52:24 -0400
Reported-by: barsikdacat@gmail.com
Diagnosed-by: Laurenz Albe
Discussion: https://postgr.es/m/168991536429.626.9957835774751337210@wrigleys.postgresql.org
Author: Laurenz Albe
Backpatch-through: 11
M doc/src/sgml/func.sgml
doc: mention libpq regression tests
commit : de35fb89cd672d4ccc0bbffcadc061434a905565
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 5 Sep 2023 13:05:28 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 5 Sep 2023 13:05:28 -0400
Reported-by: Ryo Matsumura
Discussion: https://postgr.es/m/TYCPR01MB11316B3FB56EE54D70BF0CEF6E8E4A@TYCPR01MB11316.jpnprd01.prod.outlook.com
Backpatch-through: 11
M doc/src/sgml/regress.sgml
Unify gratuitously different error messages
commit : 3a788447d85e2643f70a8d50f16a526328bfc092
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 5 Sep 2023 11:36:55 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 5 Sep 2023 11:36:55 +0200
Fixup for commit 37188cea0c.
M src/bin/pg_dump/pg_dump.c
Fix out-of-bound read in gtsvector_picksplit()
commit : 88aa4a049acaaea4db2d7946154ae72110910f32
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 4 Sep 2023 14:55:51 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 4 Sep 2023 14:55:51 +0900
This could lead to an imprecise choice when splitting an index page of a
GiST index on a tsvector, deciding which entries should remain on the
old page and which entries should move to a new page.
This is wrong since tsearch2 has been moved into core with commit
140d4ebcb46e, so backpatch all the way down. This error has been
spotted by valgrind.
Author: Alexander Lakhin
Discussion: https://postgr.es/m/17950-6c80a8d2b94ec695@postgresql.org
Backpatch-through: 11
M src/backend/utils/adt/tsgistidx.c
Fix handling of shared statistics with dropped databases
commit : 5a9325fdde1d8ab8031fa09ebd0f63dc3f967a0a
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 4 Sep 2023 08:04:43 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 4 Sep 2023 08:04:43 +0900
Dropping a database while a connection is attempted on it was able to
lead to the presence of valid database entries in shared statistics.
The issue is that MyDatabaseId was getting set too early than it should,
as, if the connection attempted on the dropped database fails when
renamed or dropped, the shutdown callback of the shared statistics would
finish by re-inserting a correct entry related to the database already
dropped.
As analyzed by the bug reporters, this issue could lead to phantom
entries in the database list maintained by the autovacuum launcher
(in rebuild_database_list()) if the database dropped was part of the
database list when it was still valid. After the database was dropped,
it would remain the highest on the list of databases to considered by
the autovacuum worker as things to process. This would prevent
autovacuum jobs to happen on all the other databases still present.
The commit fixes this issue by delaying setting MyDatabaseId until the
database existence has been re-checked with the second scan on
pg_database after getting a shared lock on it, and by switching
pgstat_update_dbstats() so as nothing happens if MyDatabaseId is not
valid.
Issue introduced by 5891c7a8ed8f, so backpatch down to 15.
Reported-by: Will Mortensen, Jacob Speidel
Analyzed-by: Will Mortensen, Jacob Speidel
Author: Andres Freund
Discussion: https://postgr.es/m/17973-bca1f7d5c14f601e@postgresql.org
Backpatch-through: 15
M src/backend/utils/activity/pgstat_database.c
M src/backend/utils/init/postinit.c
postgres_fdw: Fix test for parameterized foreign scan.
commit : 3daee7fb0871e52cb272bda8e0b5deee9153b6c1
author : Etsuro Fujita <efujita@postgresql.org>
date : Wed, 30 Aug 2023 17:15:03 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Wed, 30 Aug 2023 17:15:03 +0900
Commit e4106b252 should have updated this test, but did not; back-patch
to all supported branches.
Reviewed by Richard Guo.
Discussion: http://postgr.es/m/CAPmGK15nR0NXLSCKQAcqbZbTzrzd5MozowWnTnGfPkayndF43Q%40mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Avoid possible overflow with ltsGetFreeBlock() in logtape.c
commit : 5180160c177466a319981f68895a14ba4d4eb234
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 30 Aug 2023 08:03:51 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 30 Aug 2023 08:03:51 +0900
nFreeBlocks, defined as a long, stores the number of free blocks in a
logical tape. ltsGetFreeBlock() has been using an int to store the
value of nFreeBlocks, which could lead to overflows on platforms where
long and int are not the same size (in short everything except Windows
where long is 4 bytes).
The problematic intermediate variable is switched to be a long instead
of an int.
Issue introduced by c02fdc9223015, so backpatch down to 13.
Author: Ranier vilela
Reviewed-by: Peter Geoghegan, David Rowley
Discussion: https://postgr.es/m/CAEudQApLDWCBR_xmwNjGBrDo+f+S4E87x3s7-+hoaKqYdtC4JQ@mail.gmail.com
Backpatch-through: 13
M src/backend/utils/sort/logtape.c
Initialize ListenSocket array earlier.
commit : 0c10240600ad3d7a139737467184777ea822f593
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 29 Aug 2023 09:09:40 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 29 Aug 2023 09:09:40 +0300
After commit b0bea38705, syslogger prints 63 warnings about failing to
close a listen socket at postmaster startup. That's because the
syslogger process forks before the ListenSockets array is initialized,
so ClosePostmasterPorts() calls "close(0)" 64 times. The first call
succeeds, because fd 0 is stdin.
This has been like this since commit 9a86f03b4e in version 13, which
moved the SysLogger_Start() call to before initializing ListenSockets.
We just didn't notice until commit b0bea38705 added the LOG message.
Reported by Michael Paquier and Jeff Janes.
Author: Michael Paquier
Discussion: https://www.postgresql.org/message-id/ZOvvuQe0rdj2slA9%40paquier.xyz
Discussion: https://www.postgresql.org/message-id/ZO0fgDwVw2SUJiZx@paquier.xyz#482670177eb4eaf4c9f03c1eed963e5f
Backpatch-through: 13
M src/backend/postmaster/postmaster.c
Avoid unnecessary plancache revalidation of utility statements.
commit : 8700851352a89e9e78da9df31fd6d3a75df98470
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Aug 2023 12:02:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Aug 2023 12:02:40 -0400
Revalidation of a plancache entry (after a cache invalidation event)
requires acquiring a snapshot. Normally that is harmless, but not
if the cached statement is one that needs to run without acquiring a
snapshot. We were already aware of that for TransactionStmts,
but for some reason hadn't extrapolated to the other statements that
PlannedStmtRequiresSnapshot() knows mustn't set a snapshot. This can
lead to unexpected failures of commands such as SET TRANSACTION
ISOLATION LEVEL. We can fix it in the same way, by excluding those
command types from revalidation.
However, we can do even better than that: there is no need to
revalidate for any statement type for which parse analysis, rewrite,
and plan steps do nothing interesting, which is nearly all utility
commands. To mechanize this, invent a parser function
stmt_requires_parse_analysis() that tells whether parse analysis does
anything beyond wrapping a CMD_UTILITY Query around the raw parse
tree. If that's what it does, then rewrite and plan will just
skip the Query, so that it is not possible for the same raw parse
tree to produce a different plan tree after cache invalidation.
stmt_requires_parse_analysis() is basically equivalent to the
existing function analyze_requires_snapshot(), except that for
obscure reasons that function omits ReturnStmt and CallStmt.
It is unclear whether those were oversights or intentional.
I have not been able to demonstrate a bug from not acquiring a
snapshot while analyzing these commands, but at best it seems mighty
fragile. It seems safer to acquire a snapshot for parse analysis of
these commands too, which allows making stmt_requires_parse_analysis
and analyze_requires_snapshot equivalent.
In passing this fixes a second bug, which is that ResetPlanCache
would exclude ReturnStmts and CallStmts from revalidation.
That's surely *not* safe, since they contain parsable expressions.
Per bug #18059 from Pavel Kulakov. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/18059-79c692f036b25346@postgresql.org
M src/backend/parser/analyze.c
M src/backend/utils/cache/plancache.c
M src/include/parser/analyze.h
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/sql/plpgsql_call.sql
Update DECLARE_INDEX documentation
commit : 908f711d2398f87af1c7982c4376d04a09737faf
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 24 Aug 2023 13:59:40 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 24 Aug 2023 13:59:40 +0200
Update source code comment changes belonging to the changes in
6a6389a08b.
Discussion: https://www.postgresql.org/message-id/flat/75ae5875-3abc-dafc-8aec-73247ed41cde@eisentraut.org
M src/include/catalog/genbki.h
ci: Make compute resources for CI configurable
commit : e4693c68a497129ea3913148027a6cee3b522d5d
author : Andres Freund <andres@anarazel.de>
date : Wed, 23 Aug 2023 15:15:29 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 23 Aug 2023 15:15:29 -0700
See prior commit for an explanation for the goal of the change and why it had
to be split into two commits.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/20230808021541.7lbzdefvma7qmn3w@awork3.anarazel.de
Backpatch: 15-, where CI support was added
M .cirrus.tasks.yml
A .cirrus.yml
M src/tools/ci/README
ci: Prepare to make compute resources for CI configurable
commit : 284465e1b95ef5f561b0fadead2468c0c63c0f82
author : Andres Freund <andres@anarazel.de>
date : Wed, 23 Aug 2023 15:15:29 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 23 Aug 2023 15:15:29 -0700
cirrus-ci will soon restrict the amount of free resources every user gets (as
have many other CI providers). For most users of CI that should not be an
issue. But e.g. for cfbot it will be an issue.
To allow configuring different resources on a per-repository basis, introduce
infrastructure for overriding the task execution environment. Unfortunately
this is not entirely trivial, as yaml anchors have to be defined before their
use, and cirrus-ci only allows injecting additional contents at the end of
.cirrus.yml.
To deal with that, move the definition of the CI tasks to
.cirrus.tasks.yml. The main .cirrus.yml is loaded first, then, if defined, the
file referenced by the REPO_CI_CONFIG_GIT_URL variable, will be added,
followed by the contents of .cirrus.tasks.yml. That allows
REPO_CI_CONFIG_GIT_URL to override the yaml anchors defined in .cirrus.yml.
Unfortunately git's default merge / rebase strategy does not handle copied
files, just renamed ones. To avoid painful rebasing over this change, this
commit just renames .cirrus.yml to .cirrus.tasks.yml, without adding a new
.cirrus.yml. That's done in the followup commit, which moves the relevant
portion of .cirrus.tasks.yml to .cirrus.yml. Until that is done,
REPO_CI_CONFIG_GIT_URL does not fully work.
The subsequent commit adds documentation for how to configure custom compute
resources to src/tools/ci/README
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/20230808021541.7lbzdefvma7qmn3w@awork3.anarazel.de
Backpatch: 15-, where CI support was added
A .cirrus.star
R100 .cirrus.yml .cirrus.tasks.yml
ci: Use VMs for SanityCheck and CompilerWarnings
commit : 4fdfd0629d77af19582ca4d59c0e7fe99619f048
author : Andres Freund <andres@anarazel.de>
date : Wed, 23 Aug 2023 12:30:38 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 23 Aug 2023 12:30:38 -0700
The main reason for this change is to reduce different ways of executing
tasks, making it easier to use custom compute resources for cfbot. A secondary
benefit is that the tasks seem slightly faster this way, apparently the
increased startup overhead is outweighed by reduced runtime overhead.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/20230808021541.7lbzdefvma7qmn3w@awork3.anarazel.de
Backpatch: 15-, where CI support was added
M .cirrus.yml
ci: Move execution method of tasks into yaml templates
commit : 462f4df0a86d2c497a1739f386835d246ea8645e
author : Andres Freund <andres@anarazel.de>
date : Wed, 23 Aug 2023 12:30:23 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 23 Aug 2023 12:30:23 -0700
This is done in preparation for making the compute resources for CI
configurable. It also looks cleaner.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/20230808021541.7lbzdefvma7qmn3w@awork3.anarazel.de
Backpatch: 15-, where CI support was added
M .cirrus.yml
ci: Don't specify amount of memory
commit : 89daa5ae307a56017ce47e7dc652284ce547012a
author : Andres Freund <andres@anarazel.de>
date : Wed, 23 Aug 2023 12:30:11 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 23 Aug 2023 12:30:11 -0700
The number of CPUs is the cost-determining factor. Most instance types that
run tests have more memory/core than what we specified, there's no real
benefit in wasting that.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/20230808021541.7lbzdefvma7qmn3w@awork3.anarazel.de
Backpatch: 15-, where CI support was added
M .cirrus.yml
doc: Replace list of drivers and PLs with wiki link
commit : f4f63577b665f85916c8ec85ec70b7fd8a8cad1d
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 23 Aug 2023 14:13:07 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 23 Aug 2023 14:13:07 +0200
The list of external language drivers and procedural languages was
never complete or exhaustive, and rather than attempting to manage
it the content has migrated to the wiki. This replaces the tables
altogether with links to the wiki as we regularly get requests for
adding various projects, which we reject without any clear policy
for why or how the content should be managed.
The threads linked to below are the most recent discussions about
this, the archives contain many more.
Backpatch to all supported branches since the list on the wiki
applies to all branches.
Author: Jonathan Katz <jkatz@postgresql.org>
Discussion: https://postgr.es/m/169165415312.635.10247434927885764880@wrigleys.postgresql.org
Discussion: https://postgr.es/m/169177958824.635.11087800083040275266@wrigleys.postgresql.org
Backpatch-through: v11
M doc/src/sgml/external-projects.sgml
Fix pg_dump assertion failure when dumping pg_catalog.
commit : 1d9976d1bd0067099753820c427139696d38b4a1
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 22 Aug 2023 11:21:36 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 22 Aug 2023 11:21:36 -0700
Commit 396d348b04 did not account for the default collation.
Also, use pg_log_warning() instead of Assert().
Discussion: https://postgr.es/m/ce071503fee88334aa70f360e6e4ea14d48305ee.camel%40j-davis.com
Reviewed-by: Michael Paquier
Backpatch-through: 15
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/t/002_pg_dump.pl
Cache by-reference missing values in a long lived context
commit : 75f323aa1c1ec88f9781cb6d9307e42b820b26f0
author : Andrew Dunstan <andrew@dunslane.net>
date : Tue, 22 Aug 2023 11:57:08 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Tue, 22 Aug 2023 11:57:08 -0400
Attribute missing values might be needed past the lifetime of the tuple
descriptors from which they are extracted. To avoid possibly using
pointers for by-reference values which might thus be left dangling, we
cache a datumCopy'd version of the datum in the TopMemoryContext. Since
we first search for the value this only needs to be done once per
session for any such value.
Original complaint from Tom Lane, idea for mitigation by Andrew Dunstan,
tweaked by Tom Lane.
Backpatch to version 11 where missing values were introduced.
Discussion: https://postgr.es/m/1306569.1687978174@sss.pgh.pa.us
M src/backend/access/common/heaptuple.c
M src/tools/pgindent/typedefs.list
Fix pg_stat_reset_single_table_counters() for shared relations
commit : ad8753a3a717709f5072817636e6845a2d690162
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 21 Aug 2023 13:33:08 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 21 Aug 2023 13:33:08 +0900
This commit fixes the function of $subject for shared relations. This
feature has been added by e042678. Unfortunately, this new behavior got
removed by 5891c7a when moving statistics to shared memory.
Reported-by: Mitsuru Hinata
Author: Masahiro Ikeda
Reviewed-by: Kyotaro Horiguchi, Masahiko Sawada
Discussion: https://postgr.es/m/7cc69f863d9b1bc677544e3accd0e4b4@oss.nttdata.com
Backpatch-through: 15
M src/backend/utils/adt/pgstatfuncs.c
M src/test/regress/expected/stats.out
M src/test/regress/sql/stats.sql
ci: macos: use cached macports install
commit : 62017cb51ff9f1ff6aa94bdf26d9cc290d4ffe48
author : Andres Freund <andres@anarazel.de>
date : Sat, 19 Aug 2023 12:40:45 -0700
committer: Andres Freund <andres@anarazel.de>
date : Sat, 19 Aug 2023 12:40:45 -0700
A significant chunk of the time on the macos CI task is spent installing
packages using homebrew. The downloads of the packages are cached, but the
installation needs to happen every time. We can't cache the whole homebrew
installation, because it is too large due to pre-installed packages.
Speed this up by installing packages using macports and caching the
installation as .dmg. That's a lot faster than unpacking a tarball.
In addition, don't install llvm - it wasn't enabled when building, so it's
just a waste of time/space.
This substantially speeds up the mac CI time, both in the cold cache and in
the warm cache case (the latter from ~1m20s to ~5s).
It doesn't seem great to have diverging sources of packages for CI between
branches, so backpatch to 15 (where CI was added).
Discussion: https://postgr.es/m/20230805202539.r3umyamsnctysdc7@awork3.anarazel.de
Backpatch: 15-, where CI was added
M .cirrus.yml
A src/tools/ci/ci_macports_packages.sh
Invalidate smgr_targblock in smgrrelease().
commit : d23e60e6673ab946b769d727c62fc65e146f17f3
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 17 Aug 2023 15:45:13 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 17 Aug 2023 15:45:13 +1200
In rare circumstances involving relfilenode reuse, it might have been
possible for smgr_targblock to finish up pointing past the end.
Oversight in b74e94dc. Back-patch to 15.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/CA%2BhUKGJ8NTvqLHz6dqbQnt2c8XCki4r2QvXjBQcXpVwxTY_pvA%40mail.gmail.com
M src/backend/storage/smgr/smgr.c
Remove test from commit fa2e874946.
commit : 1bc19dfcfea71e7ed03eff9d72072579bd678b4c
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 10 Aug 2023 10:16:59 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 10 Aug 2023 10:16:59 -0700
The fix itself is fine, but the test revealed other problems related
to parallel query that are not easily fixable. Remove the test for
now to fix the buildfarm.
Discussion: https://postgr.es/m/88825.1691665432@sss.pgh.pa.us
Backpatch-through: 11
D src/test/isolation/expected/search-path-inval.out
M src/test/isolation/isolation_schedule
D src/test/isolation/specs/search-path-inval.spec
Recalculate search_path after ALTER ROLE.
commit : 9f3343e40df6fc92a3913eeb8e240e0f5c6a5cfc
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 7 Aug 2023 15:12:49 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 7 Aug 2023 15:12:49 -0700
Renaming a role can affect the meaning of the special string $user, so
must cause search_path to be recalculated.
Discussion: https://postgr.es/m/186761d32c0255debbdf50b6310b581b9c973e6c.camel@j-davis.com
Reviewed-by: Nathan Bossart, Michael Paquier
Backpatch-through: 11
M src/backend/catalog/namespace.c
A src/test/isolation/expected/search-path-inval.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/search-path-inval.spec