Stamp 14.9.
commit : 86648dcdaec67b83cec20a9d25b45ec089a7c624
author : Tom Lane <[email protected]>
date : Mon, 7 Aug 2023 16:10:05 -0400
committer: Tom Lane <[email protected]>
date : Mon, 7 Aug 2023 16:10:05 -0400
M configure
M configure.ac
Last-minute updates for release notes.
commit : 93eb661713338b2b51f36ec397e42397b5e441e2
author : Tom Lane <[email protected]>
date : Mon, 7 Aug 2023 12:50:15 -0400
committer: Tom Lane <[email protected]>
date : Mon, 7 Aug 2023 12:50:15 -0400
Security: CVE-2023-39417, CVE-2023-39418
M doc/src/sgml/release-14.sgml
Reject substituting extension schemas or owners matching ["$'\].
commit : d4648a74be07bfb23b449c722303c320297c0327
author : Noah Misch <[email protected]>
date : Mon, 7 Aug 2023 06:05:56 -0700
committer: Noah Misch <[email protected]>
date : Mon, 7 Aug 2023 06:05:56 -0700
Substituting such values in extension scripts facilitated SQL injection
when @extowner@, @extschema@, or @extschema:...@ appeared inside a
quoting construct (dollar quoting, '', or ""). No bundled extension was
vulnerable. Vulnerable uses do appear in a documentation example and in
non-bundled extensions. Hence, the attack prerequisite was an
administrator having installed files of a vulnerable, trusted,
non-bundled extension. Subject to that prerequisite, this enabled an
attacker having database-level CREATE privilege to execute arbitrary
code as the bootstrap superuser. By blocking this attack in the core
server, there's no need to modify individual extensions. Back-patch to
v11 (all supported versions).
Reported by Micah Gate, Valerie Woolard, Tim Carey-Smith, and Christoph
Berg.
Security: CVE-2023-39417
M src/backend/commands/extension.c
M src/test/modules/test_extensions/Makefile
M src/test/modules/test_extensions/expected/test_extensions.out
M src/test/modules/test_extensions/sql/test_extensions.sql
A src/test/modules/test_extensions/test_ext_extschema–1.0.sql
A src/test/modules/test_extensions/test_ext_extschema.control
Translation updates
commit : 6186e2775e0c159e9e4fc4d8df23f8d5e66dccce
author : Peter Eisentraut <[email protected]>
date : Mon, 7 Aug 2023 12:23:24 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 7 Aug 2023 12:23:24 +0200
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 2ac4b26db0a9032bce0eb018f6f742cea5847118
M src/backend/po/de.po
M src/backend/po/fr.po
M src/backend/po/ja.po
M src/backend/po/zh_CN.po
Don't Memoize lateral joins with volatile join conditions
commit : bf315354e0c330665f4b48a8e1aa6892c4681ad2
author : David Rowley <[email protected]>
date : Mon, 7 Aug 2023 22:15:50 +1200
committer: David Rowley <[email protected]>
date : Mon, 7 Aug 2023 22:15:50 +1200
The use of Memoize was already disabled in normal joins when the join
conditions had volatile functions per the code in
match_opclause_to_indexcol(). Ordinarily, the parameterization for the
inner side of a nested loop will be an Index Scan or at least eventually
lead to an index scan (perhaps nested several joins deep). However, for
lateral joins, that's not the case and seq scans can be parameterized
too, so we can't rely on match_opclause_to_indexcol().
Here we explicitly check the parameterization for volatile functions and
don't consider the generation of a Memoize path when such functions
are present.
Author: Richard Guo
Discussion: https://postgr.es/m/CAMbWs49nHFnHbpepLsv_yF3qkpCS4BdB-v8HoJVv8_=Oat0u_w@mail.gmail.com
Backpatch-through: 14, where Memoize was introduced
M src/backend/optimizer/path/joinpath.c
Release notes for 15.4, 14.9, 13.12, 12.16, 11.21.
commit : 7c9f0c0c029479e109aed546ce0106a1cb9674a5
author : Tom Lane <[email protected]>
date : Sat, 5 Aug 2023 16:47:04 -0400
committer: Tom Lane <[email protected]>
date : Sat, 5 Aug 2023 16:47:04 -0400
M doc/src/sgml/release-14.sgml
Doc: update documentation for creating custom scan paths.
commit : 1ecd74396902983329955a248bed499dca4c4706
author : Etsuro Fujita <[email protected]>
date : Thu, 3 Aug 2023 17:45:04 +0900
committer: Etsuro Fujita <[email protected]>
date : Thu, 3 Aug 2023 17:45:04 +0900
Commit f49842d1e added a new callback for custom scan paths, but missed
updating the documentation.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/CAPmGK15ODkN%2B%3DhkBCufj1HBW0x5OTb65Xuy7ryXchMdiCMpx_g%40mail.gmail.com
M doc/src/sgml/custom-scan.sgml
Update comments on CustomPath struct.
commit : 1ea0424a5b8ea5ce6defd072fc8a48d1ddc3304e
author : Etsuro Fujita <[email protected]>
date : Thu, 3 Aug 2023 17:15:04 +0900
committer: Etsuro Fujita <[email protected]>
date : Thu, 3 Aug 2023 17:15:04 +0900
Commit e7cb7ee14 allowed custom scan providers to create CustomPath
paths for join relations as well, but missed updating the comments.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/CAPmGK15ODkN%2B%3DhkBCufj1HBW0x5OTb65Xuy7ryXchMdiCMpx_g%40mail.gmail.com
M src/include/nodes/pathnodes.h
Fix ReorderBufferCheckMemoryLimit() comment.
commit : 2e3741fb5733201f80443ba9b4583f0e3f3df86e
author : Masahiko Sawada <[email protected]>
date : Wed, 2 Aug 2023 15:01:05 +0900
committer: Masahiko Sawada <[email protected]>
date : Wed, 2 Aug 2023 15:01:05 +0900
Commit 7259736a6 updated the comment but it was not correct since
ReorderBufferLargestStreamableTopTXN() returns only top-level
transactions.
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAD21AoA9XB7OR86BqvrCe2dMYX%2BZv3-BvVmjF%3DGY2z6jN-kqjg%40mail.gmail.com
Backpatch-through: 14
M src/backend/replication/logical/reorderbuffer.c
Fix overly strict Assert in jsonpath code
commit : f457f2ef140a9a218ded6befed6d7dc625aac3ec
author : David Rowley <[email protected]>
date : Wed, 2 Aug 2023 01:41:21 +1200
committer: David Rowley <[email protected]>
date : Wed, 2 Aug 2023 01:41:21 +1200
This was failing for queries which try to get the .type() of a
jpiLikeRegex. For example:
select jsonb_path_query('["string", "string"]',
'($[0] like_regex ".{7}").type()');
Reported-by: Alexander Kozhemyakin
Bug: #18035
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12, where SQL/JSON path was added.
M src/backend/utils/adt/jsonpath.c
Disallow replacing joins with scans in problematic cases.
commit : b0e390e6d1d68b92e9983840941f8f6d9e083fe0
author : Etsuro Fujita <[email protected]>
date : Fri, 28 Jul 2023 15:45:04 +0900
committer: Etsuro Fujita <[email protected]>
date : Fri, 28 Jul 2023 15:45:04 +0900
Commit e7cb7ee14, which introduced the infrastructure for FDWs and
custom scan providers to replace joins with scans, failed to add support
handling of pseudoconstant quals assigned to replaced joins in
createplan.c, leading to an incorrect plan without a gating Result node
when postgres_fdw replaced a join with such a qual.
To fix, we could add the support by 1) modifying the ForeignPath and
CustomPath structs to store the list of RestrictInfo nodes to apply to
the join, as in JoinPaths, if they represent foreign and custom scans
replacing a join with a scan, and by 2) modifying create_scan_plan() in
createplan.c to use that list in that case, instead of the
baserestrictinfo list, to get pseudoconstant quals assigned to the join;
but #1 would cause an ABI break. So fix by modifying the infrastructure
to just disallow replacing joins with such quals.
Back-patch to all supported branches.
Reported by Nishant Sharma. Patch by me, reviewed by Nishant Sharma and
Richard Guo.
Discussion: https://postgr.es/m/CADrsxdbcN1vejBaf8a%2BQhrZY5PXL-04mCd4GDu6qm6FigDZd6Q%40mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/util/restrictinfo.c
M src/include/optimizer/restrictinfo.h
Raise fixed token-length limit in hba.c.
commit : 341996248e4d720556689e5fb3da7a408cf94228
author : Tom Lane <[email protected]>
date : Thu, 27 Jul 2023 12:07:48 -0400
committer: Tom Lane <[email protected]>
date : Thu, 27 Jul 2023 12:07:48 -0400
Historically, hba.c limited tokens in the authentication configuration
files (pg_hba.conf and pg_ident.conf) to less than 256 bytes. We have
seen a few reports of this limit causing problems; notably, for
moderately-complex LDAP configurations. Increase the limit to 10240
bytes as a low-risk stop-gap solution.
In v13 and earlier, this also requires raising MAX_LINE, the limit
on overall line length. I'm hesitant to make this code consume
too much stack space, so I only raised that to 20480 bytes.
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/hba.c
Make test_decoding ddl.out shorter
commit : c5c5832600e9dfa4f690d1f4af536c3fd6d5d7e9
author : Alvaro Herrera <[email protected]>
date : Mon, 24 Jul 2023 17:48:06 +0200
committer: Alvaro Herrera <[email protected]>
date : Mon, 24 Jul 2023 17:48:06 +0200
Some of the test_decoding test output was extremely wide, because it
deals with massive toasted values, and the aligned mode causes psql to
produce 200kB of whitespace and dashes. Change to unaligned mode
temporarily to avoid that behavior.
Backpatch to 14, where it applies cleanly.
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/expected/ddl.out
M contrib/test_decoding/sql/ddl.sql
Guard against null plan pointer in CachedPlanIsSimplyValid().
commit : 10fd061bbc66351bb9cdfc20b65d0c928620e21c
author : Tom Lane <[email protected]>
date : Thu, 20 Jul 2023 14:23:46 -0400
committer: Tom Lane <[email protected]>
date : Thu, 20 Jul 2023 14:23:46 -0400
If both the passed-in plan pointer and plansource->gplan are
NULL, CachedPlanIsSimplyValid would think that the plan pointer
is possibly-valid and try to dereference it. For the one extant
call site in plpgsql, this situation doesn't normally happen
which is why we've not noticed. However, it appears to be possible
if the previous use of the cached plan failed, as per report from
Justin Pryzby. Add an extra check to prevent crashing.
Back-patch to v13 where this code was added.
Discussion: https://postgr.es/m/ZLlV+STFz1l/[email protected]
M src/backend/utils/cache/plancache.c
Doc: improve description of IN and row-constructor comparisons.
commit : 8286b432c32dbec8b3109c5b09a902adb2385c52
author : Tom Lane <[email protected]>
date : Wed, 19 Jul 2023 11:00:34 -0400
committer: Tom Lane <[email protected]>
date : Wed, 19 Jul 2023 11:00:34 -0400
IN and NOT IN work fine on records and arrays, so just say that
they accept "expressions" not "scalar expressions". I think that
that phrasing was meant to say that they don't work on set-returning
expressions, but that's not the common meaning of "scalar".
Revise the description of row-constructor comparisons to make it
perhaps a bit less confusing. (This partially reverts some
dubious wording changes made by commit f56651519.)
Per gripe from Ilya Nenashev. Back-patch to supported branches.
In HEAD and v16, also drop a NOTE about pre-8.2 behavior, which
is hopefully no longer of interest to anybody.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
Doc: fix out-of-date example of SPI usage.
commit : 2e9135b940936c2d6a08b397da495e0ebc016c40
author : Tom Lane <[email protected]>
date : Tue, 18 Jul 2023 11:59:39 -0400
committer: Tom Lane <[email protected]>
date : Tue, 18 Jul 2023 11:59:39 -0400
The "count" argument of SPI_exec() only limits execution when
the query is actually returning rows. This was not the case
before PG 9.0, so this example was correct when written; but
we missed updating it in commit 2ddc600f8. Extend the example
to show the behavior both with and without RETURNING.
While here, improve the commentary and markup for the rest
of the example.
David G. Johnston and Tom Lane, per report from Curt Kolovson.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/CANhYJV6HWtgz_qjx_APfK0PAgLUzY-2vjLuj7i_o=TZF1LAQew@mail.gmail.com
M doc/src/sgml/spi.sgml
Fix indentation in twophase.c
commit : 763d26205ad0245721a5efe1107fa3c398ff2695
author : Michael Paquier <[email protected]>
date : Tue, 18 Jul 2023 14:04:50 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 18 Jul 2023 14:04:50 +0900
This has been missed in cb0cca1, noticed before buildfarm member koel
has been able to complain while poking at a different patch. Like the
other commit, backpatch all the way down to limit the odds of merge
conflicts.
Backpatch-through: 11
M src/backend/access/transam/twophase.c
Fix recovery of 2PC transaction during crash recovery
commit : 442749100d34a213f71c73d98bc7b43478832e3a
author : Michael Paquier <[email protected]>
date : Tue, 18 Jul 2023 13:44:31 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 18 Jul 2023 13:44:31 +0900
A crash in the middle of a checkpoint with some two-phase state data
already flushed to disk by this checkpoint could cause a follow-up crash
recovery to recover twice the same transaction, once from what has been
found in pg_twophase/ at the beginning of recovery and a second time
when replaying its corresponding record.
This would lead to FATAL failures in the startup process during
recovery, where the same transaction would have a state recovered twice
instead of once:
LOG: recovering prepared transaction 731 from shared memory
LOG: recovering prepared transaction 731 from shared memory
FATAL: lock ExclusiveLock on object 731/0/0 is already held
This issue is fixed by skipping the addition of any 2PC state coming
from a record whose equivalent 2PC state file has already been loaded in
TwoPhaseState at the beginning of recovery by restoreTwoPhaseData(),
which is OK as long as the system has not reached a consistent state.
The timing to get a messed up recovery processing is very racy, and
would very unlikely happen. The thread that has reported the issue has
demonstrated the bug using injection points to force a PANIC in the
middle of a checkpoint.
Issue introduced in 728bd99, so backpatch all the way down.
Reported-by: "suyu.cmj" <[email protected]>
Author: "suyu.cmj" <[email protected]>
Author: Michael Paquier
Discussion: https://postgr.es/m/109e6994-b971-48cb-84f6-829646f18b4c.mengjuan.cmj@alibaba-inc.com
Backpatch-through: 11
M src/backend/access/transam/twophase.c
Add indisreplident to fields refreshed by RelationReloadIndexInfo()
commit : 7af65523ab8b63d2e27b898a995388179a802c71
author : Michael Paquier <[email protected]>
date : Fri, 14 Jul 2023 11:16:08 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 14 Jul 2023 11:16:08 +0900
RelationReloadIndexInfo() is a fast-path used for index reloads in the
relation cache, and it has always forgotten about updating
indisreplident, which is something that would happen after an index is
selected for a replica identity. This can lead to incorrect cache
information provided when executing a command in a transaction context
that updates indisreplident.
None of the code paths currently on HEAD that need to check upon
pg_index.indisreplident fetch its value from the relation cache, always
relying on a fresh copy on the syscache. Unfortunately, this may not be
the case of out-of-core code, that could see out-of-date value.
Author: Shruthi Gowda
Reviewed-by: Robert Haas, Dilip Kumar, Michael Paquier
Discussion: https://postgr.es/m/CAASxf_PBcxax0wW-3gErUyftZ0XrCs3Lrpuhq4-Z3Fak1DoW7Q@mail.gmail.com
Backpatch-through: 11
M src/backend/utils/cache/relcache.c
Fix updates of indisvalid for partitioned indexes
commit : 954cc2139c035e5806a7acd3e662e1144ef3b74b
author : Michael Paquier <[email protected]>
date : Fri, 14 Jul 2023 10:13:17 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 14 Jul 2023 10:13:17 +0900
indisvalid is switched to true for partitioned indexes when all its
partitions have valid indexes when attaching a new partition, up to the
top-most parent if all its leaves are themselves valid when dealing with
multiple layers of partitions.
The copy of the tuple from pg_index used to switch indisvalid to true
came from the relation cache, which is incorrect. Particularly, in the
case reported by Shruthi Gowda, executing a series of commands in a
single transaction would cause the validation of partitioned indexes to
use an incorrect version of a pg_index tuple, as indexes are reloaded
after an invalidation request with RelationReloadIndexInfo(), a much
faster version than a full index cache rebuild. In this case, the
limited information updated in the cache leads to an incorrect version
of the tuple used. One of the symptoms reported was the following
error, with a replica identity update, for instance:
"ERROR: attempted to update invisible tuple"
This is incorrect since 8b08f7d, so backpatch all the way down.
Reported-by: Shruthi Gowda
Author: Michael Paquier
Reviewed-by: Shruthi Gowda, Dilip Kumar
Discussion: https://postgr.es/m/CAASxf_PBcxax0wW-3gErUyftZ0XrCs3Lrpuhq4-Z3Fak1DoW7Q@mail.gmail.com
Backpatch-through: 11
M src/backend/commands/tablecmds.c
M src/test/regress/expected/indexing.out
M src/test/regress/sql/indexing.sql
Handle DROP DATABASE getting interrupted
commit : d11efe83038538a7b5169c679ceee457f5753877
author : Andres Freund <[email protected]>
date : Thu, 13 Jul 2023 13:03:33 -0700
committer: Andres Freund <[email protected]>
date : Thu, 13 Jul 2023 13:03:33 -0700
Until now, when DROP DATABASE got interrupted in the wrong moment, the removal
of the pg_database row would also roll back, even though some irreversible
steps have already been taken. E.g. DropDatabaseBuffers() might have thrown
out dirty buffers, or files could have been unlinked. But we continued to
allow connections to such a corrupted database.
To fix this, mark databases invalid with an in-place update, just before
starting to perform irreversible steps. As we can't add a new column in the
back branches, we use pg_database.datconnlimit = -2 for this purpose.
An invalid database cannot be connected to anymore, but can still be
dropped.
Unfortunately we can't easily add output to psql's \l to indicate that some
database is invalid, it doesn't fit in any of the existing columns.
Add tests verifying that a interrupted DROP DATABASE is handled correctly in
the backend and in various tools.
Reported-by: Evgeny Morozov <[email protected]>
Author: Andres Freund <[email protected]>
Reviewed-by: Daniel Gustafsson <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch: 11-, bug present in all supported versions
M doc/src/sgml/catalogs.sgml
M src/backend/commands/dbcommands.c
M src/backend/commands/vacuum.c
M src/backend/postmaster/autovacuum.c
M src/backend/utils/init/postinit.c
M src/bin/pg_amcheck/pg_amcheck.c
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/scripts/clusterdb.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/t/011_clusterdb_all.pl
M src/bin/scripts/t/050_dropdb.pl
M src/bin/scripts/t/091_reindexdb_all.pl
M src/bin/scripts/t/101_vacuumdb_all.pl
M src/bin/scripts/vacuumdb.c
M src/include/catalog/pg_database.h
A src/test/recovery/t/037_invalid_database.pl
Release lock after encountering bogs row in vac_truncate_clog()
commit : e246fd42363fcfd61cb98fb338c307e52747973f
author : Andres Freund <[email protected]>
date : Thu, 13 Jul 2023 13:03:33 -0700
committer: Andres Freund <[email protected]>
date : Thu, 13 Jul 2023 13:03:33 -0700
When vac_truncate_clog() encounters bogus datfrozenxid / datminmxid values, it
returns early. Unfortunately, until now, it did not release
WrapLimitsVacuumLock. If the backend later tries to acquire
WrapLimitsVacuumLock, the session / autovacuum worker hangs in an
uncancellable way. Similarly, other sessions will hang waiting for the
lock. However, if the backend holding the lock exited or errored out for some
reason, the lock was released.
The bug was introduced as a side effect of 566372b3d643.
It is interesting that there are no production reports of this problem. That
is likely due to a mix of bugs leading to bogus values having gotten less
common, process exit releasing locks and instances of hangs being hard to
debug for "normal" users.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/vacuum.c
Remove unnecessary pfree() in g_intbig_compress().
commit : a6991f763df8d2125ba2d53ef241bcdc6afc26cf
author : Tom Lane <[email protected]>
date : Thu, 13 Jul 2023 13:07:51 -0400
committer: Tom Lane <[email protected]>
date : Thu, 13 Jul 2023 13:07:51 -0400
GiST compress functions (like all GiST opclass functions) are
supposed to be called in short-lived memory contexts, so that
minor memory leaks in them are not of concern, and indeed
explicit pfree's are likely slightly counterproductive.
But this one in g_intbig_compress() is more than
slightly counterproductive, because it's guarded by
"if (in != DatumGetArrayTypeP(entry->key))" which means
that if this test succeeds, we've detoasted the datum twice.
(And to add insult to injury, the extra detoast result is
leaked.) Let's just drop the whole stanza, relying on the
GiST temporary context mechanism to clean up in good time.
The analogous bit in g_int_compress() is
if (r != (ArrayType *) DatumGetPointer(entry->key))
pfree(r);
which doesn't have the gratuitous-detoast problem so
I left it alone. Perhaps there is a case for removing
unnecessary pfree's more widely, but I'm not sure if it's
worth the code churn.
The potential extra decompress seems expensive enough to
justify calling this a (minor) performance bug and
back-patching.
Konstantin Knizhnik, Matthias van de Meent, Tom Lane
Discussion: https://postgr.es/m/CAEze2Wi86=DxErfvf+SCB2UKmU2amKOF60BKuJOX=w-RojRn0A@mail.gmail.com
M contrib/intarray/_intbig_gist.c
Be more rigorous about local variables in PostgresMain().
commit : 48582cf9ed965d4748335199da1b43101310f2a0
author : Tom Lane <[email protected]>
date : Mon, 10 Jul 2023 12:14:34 -0400
committer: Tom Lane <[email protected]>
date : Mon, 10 Jul 2023 12:14:34 -0400
Since PostgresMain calls sigsetjmp, any local variables that are not
marked "volatile" have a risk of unspecified behavior. In practice
this means that when control returns via longjmp, such variables might
get reset to their values as of the time of sigsetjmp, depending on
whether the compiler chose to put them in registers or on the stack.
We were careful about this for "send_ready_for_query", but not the
other local variables.
In the case of the timeout_enabled flags, resetting them to
their initial "false" states is actually good, since we do
"disable_all_timeouts()" in the longjmp cleanup code path. If that
does not happen, we risk uselessly calling "disable_timeout()" later,
which is harmless but a little bit expensive. Let's explicitly reset
these flags so that the behavior is correct and platform-independent.
(This change means that we really don't need the new "volatile"
markings after all, but let's install them anyway since any change
in this logic could re-introduce a problem.)
There is no issue for "firstchar" and "input_message" because those
are explicitly reinitialized each time through the query processing
loop. To make that clearer, move them to be declared inside the loop.
That leaves us with all the function-lifespan locals except the
sigjmp_buf itself marked as volatile, which seems like a good policy
to have going forward.
Because of the possibility of extra disable_timeout() calls, this
seems worth back-patching.
Sergey Shinderuk and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/tcop/postgres.c
Fix ALTER EXTENSION SET SCHEMA with objects outside an extension's schema
commit : 235e716bc24df2fbc89f0e99a29f63ec4c58772c
author : Michael Paquier <[email protected]>
date : Mon, 10 Jul 2023 09:40:15 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 10 Jul 2023 09:40:15 +0900
As coded, the code would use as a base comparison the namespace OID from
the first object scanned in pg_depend when switching its namespace
dependency entry to the new one, and use it as a base of comparison for
any follow-up checks. It would also be used as the old namespace OID to
switch *from* for the extension's pg_depend entry. Hence, if the first
object scanned has a namespace different than the one stored in the
extension, we would finish by:
- Not checking that the extension objects map with the extension's
schema.
- Not switching the extension -> namespace dependency entry to the new
namespace provided by the user, making ALTER EXTENSION ineffective.
This issue exists since this command has been introduced in d9572c4 for
relocatable extension, so backpatch all the way down to 11. The test
case has been provided by Heikki, that I have tweaked a bit to show the
effects on pg_depend for the extension.
Reported-by: Heikki Linnakangas
Author: Michael Paquier, Heikki Linnakangas
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M src/backend/commands/extension.c
Doc: update old reference to "result cache"
commit : 571377dfb83d128a104ec2220aae7ca992e036b0
author : David Rowley <[email protected]>
date : Sun, 9 Jul 2023 16:15:52 +1200
committer: David Rowley <[email protected]>
date : Sun, 9 Jul 2023 16:15:52 +1200
During the PostgreSQL 14 cycle, the Memoize executor node was briefly
called "Result Cache" until it was renamed in 83f4fcc65. That commit
missed one reference.
Reported-by: Paul A Jungwirth
Packpatch-through: 14, where Memoize was added
Discussion: https://postgr.es/m/CA+renyX=40YXhsfPTzn13oNOPO3TJ12CK9GX-2P2pvnQiScefA@mail.gmail.com
M doc/src/sgml/config.sgml
Fix tmpdir issues with commit e213de8e78
commit : 75414c69899148156d87c337cec681c292a329e9
author : Andrew Dunstan <[email protected]>
date : Sat, 8 Jul 2023 12:34:25 -0400
committer: Andrew Dunstan <[email protected]>
date : Sat, 8 Jul 2023 12:34:25 -0400
Commit e213de8e78 fixed a problem with path lengths to a tempdir on
Windows, but caused problems on at least some Unix systems where the
system tempdir is on a different file system. To work around this, only
used the system temdir for the destination of pg_replslot on Windows,
and otherwise restore the old behaviour.
Backpatch to relase 14 like the previous patch.
Problem exposed by a myriad of buildfarm animals.
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
Use shorter location for pg_replslot in pg_basebackup test
commit : 38342df830f0e6303ed0c17b58f8b1c9e310605e
author : Andrew Dunstan <[email protected]>
date : Sat, 8 Jul 2023 11:21:58 -0400
committer: Andrew Dunstan <[email protected]>
date : Sat, 8 Jul 2023 11:21:58 -0400
The symlink to a longer location tripped up some Windows limit on
buildfarm animal fairywren when running with meson, which uses slightly
longer paths.
Backpatch to release 14 to keep the script in sync. Before that the
script skipped all symlink related tests on Windows.
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
Fix type of iterator variable in SH_START_ITERATE
commit : 1508b57d4b7a67c1febb5dd71a17c9a08359c8e4
author : Andres Freund <[email protected]>
date : Thu, 6 Jul 2023 08:34:17 -0700
committer: Andres Freund <[email protected]>
date : Thu, 6 Jul 2023 08:34:17 -0700
Also add comment to make the reasoning behind the Assert() more explicit (per
Tom).
Reported-by: Ranier Vilela
Discussion: https://postgr.es/m/CAEudQAocXNJ6s1VLz+hMamLAQAiewRoW17OJ6-+9GACKfj6iPQ@mail.gmail.com
Backpatch: 11-
M src/include/lib/simplehash.h
Skip pg_baseback long filename test if path too long on Windows
commit : fb0f05576a0a6a452fe206a9e1d181f8f82dd882
author : Andrew Dunstan <[email protected]>
date : Thu, 6 Jul 2023 12:27:40 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 6 Jul 2023 12:27:40 -0400
On Windows, it's sometimes difficult to create a file with a path longer
than 255 chars, and if it can be created it might not be seen by the
archiver. This can be triggered by the test for tar backups with
filenames greater than 100 bytes. So we skip that test if the path would
exceed 255.
Backpatch to all live branches.
Reviewed by Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
WAL-log the creation of the init fork of unlogged indexes.
commit : 32f327f681d5ea86582c50e3eaf8f52a9ee65754
author : Heikki Linnakangas <[email protected]>
date : Thu, 6 Jul 2023 17:25:29 +0300
committer: Heikki Linnakangas <[email protected]>
date : Thu, 6 Jul 2023 17:25:29 +0300
We create a file, so we better WAL-log it. In practice, all the
built-in index AMs and all extensions that I'm aware of write a
metapage to the init fork, which is WAL-logged, and replay of the
metapage implicitly creates the fork too. But if ambuildempty() didn't
write any page, we would miss it.
This can be seen with dummy_index_am. Set up replication, create a
'dummy_index_am' index on an unlogged table, and look at the files
created in the replica: the init fork is not created on the
replica. Dummy_index_am doesn't do anything with the relation files,
however, so it doesn't lead to any user-visible errors.
Backpatch to all supported versions.
Reviewed-by: Robert Haas
Discussion: https://www.postgresql.org/message-id/6e5bbc08-cdfc-b2b3-9e23-1a914b9850a9%40iki.fi
M src/backend/catalog/index.c
Revert the commits related to allowing page lock to conflict among parallel group members.
commit : cf3e29851278e18b7b4997cd0bd90edb372c69da
author : Amit Kapila <[email protected]>
date : Thu, 6 Jul 2023 08:15:37 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 6 Jul 2023 08:15:37 +0530
This commit reverts the work done by commits 3ba59ccc89 and 72e78d831a.
Those commits were incorrect in asserting that we never acquire any other
heavy-weight lock after acquring page lock other than relation extension
lock. We can acquire a lock on catalogs while doing catalog look up after
acquring page lock.
This won't impact any existing feature but we need to think some other way
to achieve this before parallelizing other write operations or even
improving the parallelism in vacuum (like allowing multiple workers
for an index).
Reported-by: Jaime Casanova
Author: Amit Kapila
Backpatch-through: 13
Discussion: https://postgr.es/m/CAJKUy5jffnRKNvRHKQ0LynRb0RJC-o4P8Ku3x9vGAVLwDBWumQ@mail.gmail.com
M src/backend/optimizer/plan/planner.c
M src/backend/storage/lmgr/README
M src/backend/storage/lmgr/deadlock.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
Fix leak of LLVM "fatal-on-oom" section counter.
commit : bfb493dbae7e460a7786daa7ba1522cc3eb6dac3
author : Heikki Linnakangas <[email protected]>
date : Wed, 5 Jul 2023 13:13:13 +0300
committer: Heikki Linnakangas <[email protected]>
date : Wed, 5 Jul 2023 13:13:13 +0300
llvm_release_context() called llvm_enter_fatal_on_oom(), but was missing
the corresponding llvm_leave_fatal_on_oom() call. As a result, if JIT was
used at all, we were almost always in the "fatal-on-oom" state.
It only makes a difference if you use an extension written in C++, and
run out of memory in a C++ 'new' call. In that case, you would get a
PostgreSQL FATAL error, instead of the default behavior of throwing a
C++ exception.
Back-patch to all supported versions.
Reviewed-by: Daniel Gustafsson
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/backend/jit/llvm/llvmjit.c
Ensure that creation of an empty relfile is fsync'd at checkpoint.
commit : d85bf0719e71f483cdeb537cd0045913f587f743
author : Heikki Linnakangas <[email protected]>
date : Tue, 4 Jul 2023 17:57:03 +0300
committer: Heikki Linnakangas <[email protected]>
date : Tue, 4 Jul 2023 17:57:03 +0300
If you create a table and don't insert any data into it, the relation file
is never fsync'd. You don't lose data, because an empty table doesn't have
any data to begin with, but if you crash and lose the file, subsequent
operations on the table will fail with "could not open file" error.
To fix, register an fsync request in mdcreate(), like we do for mdwrite().
Per discussion, we probably should also fsync the containing directory
after creating a new file. But that's a separate and much wider issue.
Backpatch to all supported versions.
Reviewed-by: Andres Freund, Thomas Munro
Discussion: https://www.postgresql.org/message-id/d47d8122-415e-425c-d0a2-e0160829702d%40iki.fi
M src/backend/storage/smgr/md.c
Adjust kerberos and ldap tests for Homebrew on ARM
commit : 070bf5cda56a4750e09818ae203959ed4985137f
author : Peter Eisentraut <[email protected]>
date : Tue, 4 Jul 2023 11:14:53 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 4 Jul 2023 11:14:53 +0200
The Homebrew package manager changed its default installation prefix
for the new architecture, so a couple of tests need tweaks to find
binaries.
This is a partial backpatch of dc513bc654.
M src/test/kerberos/t/001_auth.pl
M src/test/ldap/t/001_auth.pl
Re-bin segment when memory pages are freed.
commit : b7ec66731dd8d47fdadc71e38c07dd561fd2953b
author : Thomas Munro <[email protected]>
date : Tue, 4 Jul 2023 15:16:34 +1200
committer: Thomas Munro <[email protected]>
date : Tue, 4 Jul 2023 15:16:34 +1200
It's OK to be lazy about re-binning memory segments when allocating,
because that can only leave segments in a bin that's too high. We'll
search higher bins if necessary while allocating next time, and
also eventually re-bin, so no memory can become unreachable that way.
However, when freeing memory, the largest contiguous range of free pages
might go up, so we should re-bin eagerly to make sure we don't leave the
segment in a bin that is too low for get_best_segment() to find.
The re-binning code is moved into a function of its own, so it can be
called whenever free pages are returned to the segment's free page map.
Back-patch to all supported releases.
Author: Dongming Liu <[email protected]>
Reviewed-by: Robert Haas <[email protected]> (earlier version)
Reviewed-by: Thomas Munro <[email protected]>
Discussion: https://postgr.es/m/CAL1p7e8LzB2LSeAXo2pXCW4%2BRya9s0sJ3G_ReKOU%3DAjSUWjHWQ%40mail.gmail.com
M src/backend/utils/mmgr/dsa.c
Fix race in SSI interaction with gin fast path.
commit : fb663f38790eb5f7aa994f617c6edda81f4e774c
author : Thomas Munro <[email protected]>
date : Mon, 3 Jul 2023 16:20:01 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 3 Jul 2023 16:20:01 +1200
The ginfast.c code previously checked for conflicts in before locking
the relevant buffer, leaving a window where a RW conflict could be
missed. Re-order.
There was also a place where buffer ID and block number were confused
while trying to predicate-lock a page, noted by visual inspection.
Back-patch to all supported releases. Fixes one more problem discovered
with the reproducer from bug #17949, in this case when Dmitry tried
other index types.
Reported-by: Artem Anisimov <[email protected]>
Reported-by: Dmitry Dolgov <[email protected]>
Reviewed-by: Heikki Linnakangas <[email protected]>
Discussion: https://postgr.es/m/17949-a0f17035294a55e2%40postgresql.org
M src/backend/access/gin/ginfast.c
M src/backend/access/gin/ginget.c
Fix race in SSI interaction with bitmap heap scan.
commit : 3f7d3a77e18dab4f6d1b2652e525069645c24980
author : Thomas Munro <[email protected]>
date : Mon, 3 Jul 2023 16:18:20 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 3 Jul 2023 16:18:20 +1200
When performing a bitmap heap scan, we don't want to miss concurrent
writes that occurred after we observed the heap's rs_nblocks, but before
we took predicate locks on index pages. Therefore, we can't skip
fetching any heap tuples that are referenced by the index, because we
need to test them all with CheckForSerializableConflictOut(). The
old optimization that would ignore any references to blocks >=
rs_nblocks gets in the way of that requirement, because it means that
concurrent writes in that window are ignored.
Removing that optimization shouldn't affect correctness at any isolation
level, because any new tuples shouldn't be visible to an MVCC snapshot.
There also shouldn't be any error-causing references to heap blocks past
the end, because we should have held at least an AccessShareLock on the
table before the index scan. It can't get smaller while our transaction
is running. For now, though, we'll keep the optimization at lower
levels to avoid making unnecessary changes in a bug fix.
Back-patch to all supported releases. In release 11, the code is in a
different place but not fundamentally different. Fixes one aspect of
bug #17949.
Reported-by: Artem Anisimov <[email protected]>
Reviewed-by: Dmitry Dolgov <[email protected]>
Reviewed-by: Heikki Linnakangas <[email protected]>
Discussion: https://postgr.es/m/17949-a0f17035294a55e2%40postgresql.org
M src/backend/access/heap/heapam_handler.c
Fix race in SSI interaction with empty btrees.
commit : ae6d536ed0dcb5e29126975d4a07eb308fdc5cfa
author : Thomas Munro <[email protected]>
date : Mon, 3 Jul 2023 16:16:27 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 3 Jul 2023 16:16:27 +1200
When predicate-locking btrees, we have a special case for completely
empty btrees, since there is no page to lock. This was racy, because,
without buffer lock held, a matching key could be inserted between the
_bt_search() and the PredicateLockRelation() calls.
Fix, by rechecking _bt_search() after taking the relation-level SIREAD
lock, if using SERIALIZABLE isolation and an empty btree is discovered.
Back-patch to all supported releases. Fixes one aspect of bug #17949.
Reported-by: Artem Anisimov <[email protected]>
Reviewed-by: Dmitry Dolgov <[email protected]>
Reviewed-by: Heikki Linnakangas <[email protected]>
Discussion: https://postgr.es/m/17949-a0f17035294a55e2%40postgresql.org
M src/backend/access/nbtree/nbtsearch.c
Remove expensive test of postgres_fdw batch inserts
commit : 5396b188c92213216248fa8f311c0f031a137e2a
author : Tomas Vondra <[email protected]>
date : Mon, 3 Jul 2023 18:16:58 +0200
committer: Tomas Vondra <[email protected]>
date : Mon, 3 Jul 2023 18:16:58 +0200
The test inserted 70k rows into a foreign table, in order to verify
correct behavior with more than 65535 parameters, and was added in
response to a bug report.
However, this is rather expensive, especially when running the tests
under valgrind, CLOBBER_CACHE_ALWAYS etc. It doesn't seem worth it to
keep running the test, so remove it from all branches (14+).
Backpatch-through: 14
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Use older package name in pg_basebackup test
commit : 8d3e1718d5bb89ddcbfc3bbc4b912f343bf3c13f
author : Andrew Dunstan <[email protected]>
date : Mon, 3 Jul 2023 10:46:49 -0400
committer: Andrew Dunstan <[email protected]>
date : Mon, 3 Jul 2023 10:46:49 -0400
Commit 83ed4de20f inadvertently used the new package names. In version
14 or older, use TestLib intead of using PostgreSQL::Test::Utils
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
Improve pg_basebackup long file name test Windows robustness
commit : 83ed4de20fdb60457d315065fc7895dfb51108be
author : Andrew Dunstan <[email protected]>
date : Mon, 3 Jul 2023 10:06:26 -0400
committer: Andrew Dunstan <[email protected]>
date : Mon, 3 Jul 2023 10:06:26 -0400
Creation of a file with a very long name can create problems on Windows
due to its file path limits. Work around that by creating the file via a
symlink with a shorter name.
Error displayed by buildfarm animal fairywren.o
Backpatch to all live branches
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
Make PG_TEST_NOCLEAN work for temporary directories in TAP tests
commit : c8987ea90cf2b02269f4cc7584b8d1a28e9376b4
author : Michael Paquier <[email protected]>
date : Mon, 3 Jul 2023 10:06:16 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 3 Jul 2023 10:06:16 +0900
When set, this environment variable was only effective for data
directories but not for all the other temporary files created by
PostgreSQL::Test::Utils. Keeping the temporary files after a successful
run can be useful for debugging purposes.
The documentation is updated to reflect the new behavior, with contents
available in doc/ since v16 and in src/test/perl/README since v15.
Author: Jacob Champion
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/CAAWbhmgHtDH1SGZ+Fw05CsXtE0mzTmjbuUxLB9mY9iPKgM6cUw@mail.gmail.com
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M src/test/perl/TestLib.pm
Fix oversight in handling of modifiedCols since f24523672d
commit : 260dbf19a5b4269feb6d29887aed94454602d8ac
author : Tomas Vondra <[email protected]>
date : Sun, 2 Jul 2023 20:29:01 +0200
committer: Tomas Vondra <[email protected]>
date : Sun, 2 Jul 2023 20:29:01 +0200
Commit f24523672d fixed a memory leak by moving the modifiedCols bitmap
into the per-row memory context. In the case of AFTER UPDATE triggers,
the bitmap is however referenced from an event kept until the end of the
query, resulting in a use-after-free bug.
Fixed by copying the bitmap into the AfterTriggerEvents memory context,
which is the one where we keep the trigger events. There's only one
place that needs to do the copy, but the memory context may not exist
yet. Doing that in a separate function seems more readable.
Report by Alexander Pyhalov, fix by me. Backpatch to 13, where the
bitmap was added to the event by commit 71d60e2aa0.
Reported-by: Alexander Pyhalov
Backpatch-through: 13
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/trigger.c
Fix memory leak in Incremental Sort rescans
commit : c1affa38c73b432d358c5b19b697e03a59b355ab
author : Tomas Vondra <[email protected]>
date : Sun, 2 Jul 2023 18:54:09 +0200
committer: Tomas Vondra <[email protected]>
date : Sun, 2 Jul 2023 18:54:09 +0200
The Incremental Sort had a couple issues, resulting in leaking memory
during rescans, possibly triggering OOM. The code had a couple of
related flaws:
1. During rescans, the sort states were reset but then also set to NULL
(despite the comment saying otherwise). ExecIncrementalSort then
sees NULL and initializes a new sort state, leaking the memory used
by the old one.
2. Initializing the sort state also automatically rebuilt the info about
presorted keys, leaking the already initialized info. presorted_keys
was also unnecessarily reset to NULL.
Patch by James Coleman, based on patches by Laurenz Albe and Tom Lane.
Backpatch to 13, where Incremental Sort was introduced.
Author: James Coleman, Laurenz Albe, Tom Lane
Reported-by: Laurenz Albe, Zu-Ming Jiang
Backpatch-through: 13
Discussion: https://postgr.es/m/b2bd02dff61af15e3526293e2771f874cf2a3be7.camel%40cybertec.at
Discussion: https://postgr.es/m/db03c582-086d-e7cd-d4a1-3bc722f81765%40inf.ethz.ch
M src/backend/executor/nodeIncrementalSort.c
doc: PG _14_ relnotes, remove duplicate commit comment
commit : 49d1d3c2c896f610ddff835c63648828aa9a6d3e
author : Bruce Momjian <[email protected]>
date : Fri, 30 Jun 2023 08:37:15 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 30 Jun 2023 08:37:15 -0400
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
Fix marking of indisvalid for partitioned indexes at creation
commit : 663b35f2df6261e5531d97967bca04d95a61ee15
author : Michael Paquier <[email protected]>
date : Fri, 30 Jun 2023 13:54:56 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 30 Jun 2023 13:54:56 +0900
The logic that introduced partitioned indexes missed a few things when
invalidating a partitioned index when these are created, still the code
is written to handle recursions:
1) If created from scratch because a mapping index could not be found,
the new index created could be itself invalid, if for example it was a
partitioned index with one of its leaves invalid.
2) A CCI was missing when indisvalid is set for a parent index, leading
to inconsistent trees when recursing across more than one level for a
partitioned index creation if an invalidation of the parent was
required.
This could lead to the creation of a partition index tree where some of
the partitioned indexes are marked as invalid, but some of the parents
are marked valid, which is not something that should happen (as
validatePartitionedIndex() defines, indisvalid is switched to true for a
partitioned index iff all its partitions are themselves valid).
This patch makes sure that indisvalid is set to false on a partitioned
index if at least one of its partition is invalid. The flag is set to
true if *all* its partitions are valid.
The regression test added in this commit abuses of a failed concurrent
index creation, marked as invalid, that maps with an index created on
its partitioned table afterwards.
Reported-by: Alexander Lakhin
Reviewed-by: Alexander Lakhin
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M src/backend/commands/indexcmds.c
M src/test/regress/expected/indexing.out
M src/test/regress/sql/indexing.sql
Fix order of operations in ExecEvalFieldStoreDeForm().
commit : 0789b82a97924cec14f651f678eebc56c09da17f
author : Tom Lane <[email protected]>
date : Thu, 29 Jun 2023 10:19:10 -0400
committer: Tom Lane <[email protected]>
date : Thu, 29 Jun 2023 10:19:10 -0400
If the given composite datum is toasted out-of-line,
DatumGetHeapTupleHeader will perform database accesses to detoast it.
That can invalidate the result of get_cached_rowtype, as documented
(perhaps not plainly enough) in that function's API spec; which leads
to strange errors or crashes when we try to use the TupleDesc to read
the tuple. In short then, trying to update a field of a composite
column could fail intermittently if the overall column value is wide
enough to require toasting.
We can fix the bug at no cost by just changing the order of
operations, since we don't need the TupleDesc until after detoasting.
(Other callers of get_cached_rowtype appear to get this right already,
so there's only one bug.)
Note that the added regression test case reveals this bug reliably
only with debug_discard_caches/CLOBBER_CACHE_ALWAYS.
Per bug #17994 from Alexander Lakhin. Sadly, this patch does not fix
the missing-values issue revealed in the bug discussion; we'll need
some more work to cover that.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execExprInterp.c
M src/test/regress/expected/rowtypes.out
M src/test/regress/sql/rowtypes.sql
Remove inappropriate raw_expression_tree_walker() code
commit : 6bc7873da1c3e9f1d870fa2d671eb450ea6c9753
author : Peter Eisentraut <[email protected]>
date : Thu, 29 Jun 2023 10:30:55 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 29 Jun 2023 10:30:55 +0200
It was walking into the ColumnDef->compression field, which is not a
node but a string. This code is currently not reachable (because the
compression field is only set in situations that don't go through
raw_expression_tree_walker()), but if it had been, this could have
behaved erratically.
M src/backend/nodes/nodeFuncs.c
pg_stat_statements: Fix second comment related to entry resets
commit : 7e8349cbd7f74f906d7b72841e66a96f93b516be
author : Michael Paquier <[email protected]>
date : Thu, 29 Jun 2023 09:17:34 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 29 Jun 2023 09:17:34 +0900
This should have been part of dc73db6, but it got lost in the mix.
Oversight in 6b4d23f.
Author: Japin Li
Discussion: https://postgr.es/m/MEYP282MB1669FC91C764E277821936D3B624A@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
Backpatch-through: 14
M contrib/pg_stat_statements/pg_stat_statements.c
pg_stat_statements: Fix incorrect comment with entry resets
commit : aa4b11e8be7d0b417d2abe15a175e82dbba82616
author : Michael Paquier <[email protected]>
date : Thu, 29 Jun 2023 08:05:10 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 29 Jun 2023 08:05:10 +0900
Oversight in 6b4d23f.
Author: Japin Li, Richard Guo
Discussion: https://postgr.es/m/MEYP282MB1669FC91C764E277821936D3B624A@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
Backpatch-through: 14
M contrib/pg_stat_statements/pg_stat_statements.c
Ignore invalid indexes when enforcing index rules in ALTER TABLE ATTACH PARTITION
commit : 6160e221d53db19b3cccb844de55276597a7c608
author : Michael Paquier <[email protected]>
date : Wed, 28 Jun 2023 15:57:48 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 28 Jun 2023 15:57:48 +0900
A portion of ALTER TABLE .. ATTACH PARTITION is to ensure that the
partition being attached to the partitioned table has a correct set of
indexes, so as there is a consistent index mapping between the
partitioned table and its new-to-be partition. However, as introduced
in 8b08f7d, the current logic could choose an invalid index as a match,
which is something that can exist when dealing with more than two levels
of partitioning, like attaching a partitioned table (that has
partitions, with an index created by CREATE INDEX ON ONLY) to another
partitioned table.
A partitioned index with indisvalid set to false is equivalent to an
incomplete partition tree, meaning that an invalid partitioned index
does not have indexes defined in all its partitions. Hence, choosing an
invalid partitioned index can create inconsistent partition index trees,
where the parent attaching to is valid, but its partition may be
invalid.
In the report from Alexander Lakhin, this showed up as an assertion
failure when validating an index. Without assertions enabled, the
partition index tree would be actually broken, as indisvalid should
be switched to true for a partitioned index once all its partitions are
themselves valid. With two levels of partitioning, the top partitioned
table used a valid index and was able to link to an invalid index stored
on its partition, itself a partitioned table.
I have studied a few options here (like the possibility to switch
indisvalid to false for the parent), but came down to the conclusion
that we'd better rely on a simple rule: invalid indexes had better never
be chosen, so as the partition attached uses and creates indexes that
the parent expects. Some regression tests are added to provide some
coverage. Note that the existing coverage is not impacted.
This is a problem since partitioned indexes exist, so backpatch all the
way down to v11.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/[email protected]
Backpatch-through: 11
M src/backend/commands/tablecmds.c
M src/test/regress/expected/indexing.out
M src/test/regress/sql/indexing.sql
Fix comment on clearing padding.
commit : 0c3fb8ac5f17d19a1d9df24594050dcf0bda8bc1
author : Heikki Linnakangas <[email protected]>
date : Tue, 27 Jun 2023 10:11:31 +0300
committer: Heikki Linnakangas <[email protected]>
date : Tue, 27 Jun 2023 10:11:31 +0300
Author: Japin Li
Discussion: https://www.postgresql.org/message-id/MEYP282MB16696317B5DA7D0D92306149B627A@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
M contrib/pg_stat_statements/pg_stat_statements.c
Check for interrupts and stack overflow in TParserGet().
commit : 4c61afa47c618a11ad78b91dcd083e73b6332e82
author : Tom Lane <[email protected]>
date : Sat, 24 Jun 2023 17:18:08 -0400
committer: Tom Lane <[email protected]>
date : Sat, 24 Jun 2023 17:18:08 -0400
TParserGet() recurses for some token types, meaning it's possible
to drive it to stack overflow. Since this is a minority behavior,
I chose to add the check_stack_depth() call to the two places that
recurse rather than doing it during every single call.
While at it, add CHECK_FOR_INTERRUPTS(), because this can run
unpleasantly long for long inputs.
Per bug #17995 from Zuming Jiang. This is old, so back-patch
to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/tsearch/wparser_def.c
doc: rename "decades" to be more generic
commit : c1589923c681877ee06e352922a942ed1d6d9bb9
author : Bruce Momjian <[email protected]>
date : Fri, 23 Jun 2023 22:50:55 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 23 Jun 2023 22:50:55 -0400
Reported-by: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M doc/src/sgml/history.sgml
Fix incorrect error message in libpq_pipeline
commit : 451ca5c1e699c4448875fc12d9ed6d84a2e7816d
author : Michael Paquier <[email protected]>
date : Fri, 23 Jun 2023 17:50:28 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 23 Jun 2023 17:50:28 +0900
One of the tests for the pipeline mode with portal description expects a
non-NULL PQgetResult, but used an incorrect error message on failure,
telling that PQgetResult being NULL was the expected result.
Author: Jelte Fennema
Discussion: https://postgr.es/m/CAGECzQTkShHecFF+EZrm94Lbsu2ej569T=bz+PjMbw9Aiioxuw@mail.gmail.com
Backpatch-through: 14
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Doc: Clarify the behavior of triggers/rules in a logical subscriber.
commit : 991983fcf650a5cecaefff5dc10a55ff865f57cb
author : Amit Kapila <[email protected]>
date : Thu, 22 Jun 2023 12:16:51 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 22 Jun 2023 12:16:51 +0530
By default, triggers and rules do not fire on a logical replication
subscriber based on the "session_replication_role" GUC being set to
"replica". However, the docs in the logical replication section assumed
that the reader understood how this GUC worked. This modifies the docs to
be more explicit and links back to the GUC itself.
Author: Jonathan Katz, Peter Smith
Reviewed-by: Vignesh C, Euler Taveira
Backpatch-through: 11
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/logical-replication.sgml
Doc: mention that extended stats aren't used for joins
commit : 8145b5e138c7f26594fb93e11e6ebd5bf2aba586
author : David Rowley <[email protected]>
date : Thu, 22 Jun 2023 12:47:53 +1200
committer: David Rowley <[email protected]>
date : Thu, 22 Jun 2023 12:47:53 +1200
Statistics defined by the CREATE STATISTICS command are only used to
assist with the selectivity estimations of base relations, never for
joins. Here we mention this fact in the notes section of the CREATE
STATISTICS command.
Discussion: https://postgr.es/m/CAApHDvrMuVgDOrmg_EtFDZ=AOovq6EsJNnHH1ddyZ8EqL4yzMw@mail.gmail.com
Backpatch-through: 11
M doc/src/sgml/ref/create_statistics.sgml
nbtree VACUUM: cope with topparent inconsistencies.
commit : 63fa0deb31c4b18a6dbf15fc8ff64ac010ea77b2
author : Peter Geoghegan <[email protected]>
date : Wed, 21 Jun 2023 17:41:54 -0700
committer: Peter Geoghegan <[email protected]>
date : Wed, 21 Jun 2023 17:41:54 -0700
Avoid "right sibling %u of block %u is not next child" errors when
vacuuming a corrupt nbtree index. Just LOG the issue and press on.
That way VACUUM will have a decent chance of finishing off all required
processing for the index (and for the table as a whole).
This is similar to recent work from commit 5abff197, as well as work
from commit 5b861baa (later backpatched as commit 43e409ce), which
taught nbtree VACUUM to keep going when its "re-find" check fails. The
hardening added by this commit takes place directly after the "re-find"
check, right before the critical section for the first stage of page
deletion.
Author: Peter Geoghegan <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wz=dayg0vjs4+er84TS9ami=csdzjpuiCGbEw=idhwqhzQ@mail.gmail.com
Backpatch: 11- (all supported versions).
M src/backend/access/nbtree/nbtpage.c
doc: update PG history as over "three decades"
commit : 43b28fc39f2224da7f68b3dfc1850adc68e7f9c7
author : Bruce Momjian <[email protected]>
date : Wed, 21 Jun 2023 19:20:07 -0400
committer: Bruce Momjian <[email protected]>
date : Wed, 21 Jun 2023 19:20:07 -0400
Reported-by: Pierre <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M doc/src/sgml/history.sgml
Avoid Assert failure when processing empty statement in aborted xact.
commit : 120ea65b8a0c7fd5c8f45030ec6a7fd979eaafc3
author : Tom Lane <[email protected]>
date : Wed, 21 Jun 2023 11:07:11 -0400
committer: Tom Lane <[email protected]>
date : Wed, 21 Jun 2023 11:07:11 -0400
exec_parse_message() wants to create a cached plan in all cases,
including for empty input. The empty-input path does not have
a test for being in an aborted transaction, making it possible
that plancache.c will fail due to trying to do database lookups
even though there's no real work to do.
One solution would be to throw an aborted-transaction error in
this path too, but it's not entirely clear whether the lack of
such an error was intentional or whether some clients might be
relying on non-error behavior. Instead, let's hack plancache.c
so that it treats empty statements with the same logic it
already had for transaction control commands, ensuring that it
can soldier through even in an already-aborted transaction.
Per bug #17983 from Alexander Lakhin. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/cache/plancache.c
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
Disable use of archiving in 009_twophase.pl
commit : 2634926906ded28212abb110e3152337467535b3
author : Michael Paquier <[email protected]>
date : Wed, 21 Jun 2023 16:16:24 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 21 Jun 2023 16:16:24 +0900
This partially reverts 68cb5af, as using archiving to enforce the
rename of the last partial segment of the old timeline at promotion to
use .partial as suffix is impacting the tests when it does switchovers.
As showed by the logs gathered by the CI in the tests that failed, a new
standby may fail to find the WAL segment it needs to follow a promoted
instance with its timeline jump, as it got renamed to .partial.
This problem would manifest as a run timeout with 009_twophase.pl, as
the new standby repeatedly requests a segment from the promoted primary
that it would not find.
Reported-by: Nathan Bossart
Discussion: https://postgr.es/m/20230621043345.GA787473@nathanxps13
Backpatch-through: 13
M src/test/recovery/t/009_twophase.pl
Fix the errhint message and docs for drop subscription failure.
commit : 0b79042701ff845b9a580443c822d52566c1ec24
author : Amit Kapila <[email protected]>
date : Wed, 21 Jun 2023 10:09:28 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 21 Jun 2023 10:09:28 +0530
The existing errhint message and docs were missing the fact that we can't
disassociate from the slot unless the subscription is disabled.
Author: Robert Sjöblom, Peter Smith
Reviewed-by: Peter Eisentraut, Amit Kapila
Backpatch-through: 11
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/drop_subscription.sgml
M src/backend/commands/subscriptioncmds.c
Fix hash join when inner hashkey expressions contain Params.
commit : d911dce14d766a16c4e5b98b54960ef4df857c3b
author : Tom Lane <[email protected]>
date : Tue, 20 Jun 2023 17:47:36 -0400
committer: Tom Lane <[email protected]>
date : Tue, 20 Jun 2023 17:47:36 -0400
If the inner-side expressions contain PARAM_EXEC Params, we must
re-hash whenever the values of those Params change. The executor
mechanism for that exists already, but we failed to invoke it because
finalize_plan() neglected to search the Hash.hashkeys field for
Params. This allowed a previous scan's hash table to be re-used
when it should not be, leading to rows missing from the join's output.
(I believe incorrectly-included join rows are impossible however,
since checking the real hashclauses would reject false matches.)
This bug is very ancient, dating probably to d24d75ff1 of 7.4.
Sadly, this simple fix depends on the plan representational changes
made by 2abd7ae9b, so it will only work back to v12. I thought
about trying to make some kind of hack for v11, but I'm leery
of putting code significantly different from what is used in the
newer branches into a nearly-EOL branch. Seeing that the bug
escaped detection for a full twenty years, problematic cases
must be rare; so I don't feel too awful about leaving v11 as-is.
Per bug #17985 from Zuming Jiang. Back-patch to v12.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/plan/subselect.c
M src/test/regress/expected/join_hash.out
M src/test/regress/sql/join_hash.sql
Enable archiving in recovery TAP test 009_twophase.pl
commit : e6317d9b507cc7021f07a6712fa86bd893a2d469
author : Michael Paquier <[email protected]>
date : Tue, 20 Jun 2023 10:25:45 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 20 Jun 2023 10:25:45 +0900
This is a follow-up of f663b00, that has been committed to v13 and v14,
tweaking the TAP test for two-phase transactions so as it provides
coverage for the bug that has been fixed. This change is done in its
own commit for clarity, as v15 and HEAD did not show the problematic
behavior, still missed coverage for it.
While on it, this adds a comment about the dependency of the last
partial segment rename and RecoverPreparedTransactions() at the end of
recovery, as that can be easy to miss.
Author: Michael Paquier
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/backend/access/transam/xlog.c
M src/test/recovery/t/009_twophase.pl
Fix failure at promotion with 2PC transactions and archiving enabled
commit : f663b009189ff175a8d134a6cc6e26b5940a1449
author : Michael Paquier <[email protected]>
date : Tue, 20 Jun 2023 09:36:35 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 20 Jun 2023 09:36:35 +0900
When archiving is enabled, a promotion request would fail with the
following error when some 2PC transaction needs to be recovered from
WAL, preventing the promotion to complete:
FATAL: requested WAL segment pg_wal/000000010000000000000001 has already been removed
The origin of the problem is that the last partial segment of the old
timeline is renamed before recovering the 2PC data via
RecoverPreparedTransactions() at the end of recovery, causing the FATAL
because the segment wanted is now renamed with a .partial suffix. This
commit reorders a bit the end-of-recovery actions so as the execution of
recovery_end_command, the cleanup of the old segments of the old
timeline (RemoveNonParentXlogFiles) and the last partial segment rename
are done after the 2PC transaction data is recovered with
RecoverPreparedTransactions(). This makes the order of these
end-of-recovery actions more consistent with ~15, at the exception of
the end-of-recovery checkpoint that still needs to happen before all the
actions reordered here in v13 and v14, contrary to what 15~ does.
v15 and newer versions have "fixed" this problem somewhat accidentally
with 811051c, where the end-of-recovery actions got reordered. In this
case, the recovery of 2PC transactions happens before the renaming of
the last partial segment of the old timeline.
v13 and v14 are the versions that can easily see this problem as per the
refactoring of 38a95731 where XLogReaderState is reset in
XLogBeginRead() before reading the 2PC transaction data. v11 and v12
could also see this problem, but may finish by reading the 2PC data from
some of the WAL buffers instead. Perhaps something could be done for
these two branches, but I am not really excited about doing something on
these per the lack of complaints and per the fact that v11 is soon going
to be EOL'd soon (there is always a risk of breaking something).
Note that the TAP test 009_twophase.pl is able to exhibit the issue if
it enables archiving on the primary node, which does not impact the test
coverage as restore_command would remain unused. This is something that
should be changed on v15 and HEAD as well, so this will be changed in a
separate commit for clarity.
Author: Julian Markwort
Reviewed-by: Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/backend/access/transam/xlog.c
Don't use partial unique indexes for unique proofs in the planner
commit : 73f1c17fc31b3582ad8ab8fed33e9c9c1067045b
author : David Rowley <[email protected]>
date : Mon, 19 Jun 2023 13:01:58 +1200
committer: David Rowley <[email protected]>
date : Mon, 19 Jun 2023 13:01:58 +1200
Here we adjust relation_has_unique_index_for() so that it no longer makes
use of partial unique indexes as uniqueness proofs. It is incorrect to
use these as the predicates used by check_index_predicates() to set
predOK makes use of not only baserestrictinfo quals as proofs, but also
qual from join conditions. For relation_has_unique_index_for()'s case, we
need to know the relation is unique for a given set of columns before any
joins are evaluated, so if predOK was only set to true due to some join
qual, then it's unsafe to use such indexes in
relation_has_unique_index_for(). The final plan may not even make use
of that index, which could result in reading tuples that are not as
unique as the planner previously expected them to be.
Bug: #17975
Reported-by: Tor Erik Linnerud
Backpatch-through: 11, all supported versions
Discussion: https://postgr.es/m/17975-98a90c156f25c952%40postgresql.org
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/plan/analyzejoins.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Fix typo in comment.
commit : 3f157d085bd72a69fabd6e2621800204c19cc76f
author : Amit Langote <[email protected]>
date : Fri, 16 Jun 2023 10:04:22 +0900
committer: Amit Langote <[email protected]>
date : Fri, 16 Jun 2023 10:04:22 +0900
Back-patch down to 11.
Author: Sho Kato (<[email protected]>)
Discussion: https://postgr.es/m/TYCPR01MB68499042A33BC32241193AAF9F5BA%40TYCPR01MB6849.jpnprd01.prod.outlook.com
M src/backend/rewrite/rewriteHandler.c
intarray: Prevent out-of-bound memory reads with gist__int_ops
commit : 019a40d61959cf9f35795bcd1bc1f3d8a15476c1
author : Michael Paquier <[email protected]>
date : Thu, 15 Jun 2023 13:45:40 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 15 Jun 2023 13:45:40 +0900
As gist__int_ops stands in intarray, it is possible to store GiST
entries for leaf pages that can cause corruptions when decompressed.
Leaf nodes are stored as decompressed all the time by the compression
method, and the decompression method should map with that, retrieving
the contents of the page without doing any decompression. However, the
code authorized the insertion of leaf page data with a higher number of
array items than what can be supported, generating a NOTICE message to
inform about this matter (199 for a 8k page, for reference). When
calling the decompression method, a decompression would be attempted on
this leaf node item but the contents should be retrieved as they are.
The NOTICE message generated when dealing with the compression of a leaf
page and too many elements in the input array for gist__int_ops has been
introduced by 08ee64e, removing the marker stored in the array to track
if this is actually a leaf node. However, it also missed the fact that
the decompression path should do nothing for a leaf page. Hence, as the
code stand, a too-large array would be stored as uncompressed but the
decompression path would attempt a decompression rather that retrieving
the contents as they are.
This leads to various problems. First, even if 08ee64e tried to address
that, it is possible to do out-of-bound chunk writes with a large input
array, with the backend informing about that with WARNINGs. On
decompression, retrieving the stored leaf data would lead to incorrect
memory reads, leading to crashes or even worse.
Perhaps somebody would be interested in expanding the number of array
items that can be handled in a leaf page for this operator in the
future, which would require revisiting the choice done in 08ee64e, but
based on the lack of reports about this problem since 2005 it does not
look so. For now, this commit prevents the insertion of data for leaf
pages when using more array items that the code can handle on
decompression, switching the NOTICE message to an ERROR. If one wishes
to use more array items, gist__intbig_ops is an optional choice.
While on it, use ERRCODE_PROGRAM_LIMIT_EXCEEDED as error code when a
limit is reached, because that's what the module is facing in such
cases.
Author: Ankit Kumar Pandey, Alexander Lakhin
Reviewed-by: Richard Guo, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M contrib/intarray/_int_gist.c
M contrib/intarray/expected/_int.out
M contrib/intarray/sql/_int.sql
Correctly update hasSubLinks while mutating a rule action.
commit : d1423c52e3f00a7d00c4e3517fab0ed9b9330e3a
author : Tom Lane <[email protected]>
date : Tue, 13 Jun 2023 15:58:37 -0400
committer: Tom Lane <[email protected]>
date : Tue, 13 Jun 2023 15:58:37 -0400
rewriteRuleAction neglected to check for SubLink nodes in the
securityQuals of range table entries. This could lead to failing
to convert such a SubLink to a SubPlan, resulting in assertion
crashes or weird errors later in planning.
In passing, fix some poor coding in rewriteTargetView:
we should not pass the source parsetree's hasSubLinks
field to ReplaceVarsFromTargetList's outer_hasSubLinks.
ReplaceVarsFromTargetList knows enough to ignore that
when a Query node is passed, but it's still confusing
and bad precedent: if we did try to update that flag
we'd be updating a stale copy of the parsetree.
Per bug #17972 from Alexander Lakhin. This has been broken since
we added RangeTblEntry.securityQuals (although the presented test
case only fails back to 215b43cdc), so back-patch all the way.
Discussion: https://postgr.es/m/[email protected]
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/updatable_views.out
M src/test/regress/sql/updatable_views.sql
Accept fractional seconds in jsonpath's datetime() method.
commit : 5eaa05f637179b6847f9efc98ca07a9aa1479e47
author : Tom Lane <[email protected]>
date : Mon, 12 Jun 2023 10:54:28 -0400
committer: Tom Lane <[email protected]>
date : Mon, 12 Jun 2023 10:54:28 -0400
Commit 927d9abb6 purported to make datetime() accept any string
that could be output for a datetime value by to_jsonb(). But it
overlooked the possibility of fractional seconds being present,
so that cases as simple as to_jsonb(now()) would defeat it.
Fix by adding formats that include ".US" to the list in
executeDateTimeMethod(). (Note that while this is nominally
microseconds, it'll do the right thing for fractions with
fewer than six digits.)
In passing, re-order the list to restore the datatype ordering
specified in its comment. The violation accidentally did not
break anything; but the next edit might be less lucky, so add
more comments.
Per report from Tim Field. Back-patch to v13 where datetime()
was added, like the previous patch.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/jsonb_jsonpath.out
M src/test/regress/sql/jsonb_jsonpath.sql
hstore: Tighten key/value parsing check for whitespaces
commit : e0e6829459b0e0566dc4493675522afb69d39f36
author : Michael Paquier <[email protected]>
date : Mon, 12 Jun 2023 09:14:14 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 12 Jun 2023 09:14:14 +0900
isspace() can be locale-sensitive depending on the platform, causing
hstore to consider as whitespaces characters it should not see as such.
For example, U+0105, being decoded as 0xC4 0x85 in UTF-8, would be
discarded from the input given.
This problem is similar to 9ae2661, though it was missed that hstore
can also manipulate non-ASCII inputs, so replace the existing isspace()
calls with scanner_isspace().
This problem exists for a long time, so backpatch all the way down.
Author: Evan Jones
Discussion: https://postgr.es/m/CA+HWA9awUW0+RV_gO9r1ABZwGoZxPztcJxPy8vMFSTbTfi4jig@mail.gmail.com
Backpatch-through: 11
M contrib/hstore/Makefile
A contrib/hstore/expected/hstore_utf8.out
A contrib/hstore/expected/hstore_utf8_1.out
M contrib/hstore/hstore_io.c
A contrib/hstore/sql/hstore_utf8.sql
Fix missing initializations of MyProc.delayChkptEnd
commit : c6043fcbb23875926f3b674b2e22dfaa9c6cf300
author : Michael Paquier <[email protected]>
date : Sun, 11 Jun 2023 10:33:46 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 11 Jun 2023 10:33:46 +0900
This commit fixes an oversight introduced in 10520f4, that has added
delayChkptEnd to PGPROC to avoid ABI breakages on stable branches, where
two spots have missed to initialize this variable (delayChkpt was
switched back from int to bool, and it was initialized as 0 so there was
no consequences for it):
- InitProcess(), where the per-process data structures of a backend are
initialized.
- InitAuxiliaryProcess(), same but for auxiliary processes.
An interruption during relation truncation while this flag is set could
cause an assertion failure when a follow-up process does a relation
truncation while reusing the same PGPROC entry. A second effect could
be incorrect checkpoint end delays.
While on it, add an assertion in ProcArrayClearTransaction() for
delayChkptEnd to be in line with 5788e25. This is needed only for v14.
This issue affects v11~v14, but not v15~, as we use a single field
called delayChkptFlags to delay checkpoints there.
Author: suyu.cmj ([email protected])
Reviewed-by: Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/9c3d2a49-db5f-43cb-840b-d58f9a684295.mengjuan.cmj@alibaba-inc.com
Backpatch-through: 11
M src/backend/storage/ipc/procarray.c
M src/backend/storage/lmgr/proc.c
Refactor routine to find single log content pattern in TAP tests
commit : 28af91b4e79763c810c01efaafbfd10740fd5990
author : Michael Paquier <[email protected]>
date : Fri, 9 Jun 2023 11:56:41 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 9 Jun 2023 11:56:41 +0900
The same routine to check if a specific pattern can be found in the
server logs was copied over four different test scripts. This refactors
the whole to use a single routine located in PostgreSQL::Test::Cluster,
named log_contains, to grab the contents of the server logs and check
for a specific pattern.
On HEAD, the code previously used assumed that slurp_file() could not
handle an undefined offset, setting it to zero, but slurp_file() does
do an extra fseek() before retrieving the log contents only if an offset
is defined. In two places, the test was retrieving the full log
contents with slurp_file() after calling substr() to apply an offset,
ignoring that slurp_file() would be able to handle that.
Backpatch all the way down to ease the introduction of new tests that
could rely on the new routine.
Author: Vignesh C
Reviewed-by: Andrew Dunstan, Dagfinn Ilmari Mannsåker, Michael Paquier
Discussion: https://postgr.es/m/CALDaNm0YSiLpjCmajwLfidQrFOrLNKPQir7s__PeVvh9U3uoTQ@mail.gmail.com
Backpatch-through: 11
M src/test/perl/PostgresNode.pm
M src/test/recovery/t/019_replslot_limit.pl
M src/test/recovery/t/033_replay_tsp_drops.pl
Refactor log check logic for connect_ok/fails in PostgreSQL::Test::Cluster
commit : 30469a6ed4248b4543e165355a7793329ac041df
author : Michael Paquier <[email protected]>
date : Fri, 9 Jun 2023 09:37:34 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 9 Jun 2023 09:37:34 +0900
This commit refactors a bit the code in charge of checking for log
patterns when connections fail or succeed, by moving the log pattern
checks into their own routine, for clarity. This has come up as
something to improve while discussing the refactoring of find_in_log().
Backpatch down to 14 where these routines are used, to ease the
introduction of new tests that could rely on them.
Author: Vignesh C, Michael Paquier
Discussion: https://postgr.es/m/CALDaNm0YSiLpjCmajwLfidQrFOrLNKPQir7s__PeVvh9U3uoTQ@mail.gmail.com
Backpatch-through: 14
M src/test/perl/PostgresNode.pm
doc: Fix example command for ALTER FOREIGN TABLE ... OPTIONS.
commit : fd3def395081850eddb83cac3c2c0e6862db7f61
author : Fujii Masao <[email protected]>
date : Thu, 8 Jun 2023 20:11:52 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 8 Jun 2023 20:11:52 +0900
In the documentation, previously the example command for
ALTER FOREIGN TABLE ... OPTIONS incorrectly included both
the option name and value with the DROP operation.
The correct syntax for the DROP operation requires only
the name of the option to be specified. This commit fixes
the example by removing the option value from the DROP operation.
Back-patch to all supported versions.
Author: Mehmet Emin KARAKAS <[email protected]>
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CANQrdXAHzbcEYhjGoe5A42OmfvdQhHFJzyKj9gJvHuDKyOF5Ng@mail.gmail.com
M doc/src/sgml/ref/alter_foreign_table.sgml
Use per-tuple context in ExecGetAllUpdatedCols
commit : 7f528e96c501704202b86edaf0ae115fe809ac5b
author : Tomas Vondra <[email protected]>
date : Wed, 7 Jun 2023 16:48:50 +0200
committer: Tomas Vondra <[email protected]>
date : Wed, 7 Jun 2023 16:48:50 +0200
Commit fc22b6623b (generated columns) replaced ExecGetUpdatedCols() with
ExecGetAllUpdatedCols() in a couple places handling UPDATE (triggers and
lock mode). However, ExecGetUpdatedCols() did exec_rt_fetch() while
ExecGetAllUpdatedCols() also allocates memory through bms_union()
without paying attention to the memory context and happened to use the
long-lived ExecutorState, leaking the memory until the end of the query.
The amount of leaked memory is proportional to the number of (updated)
attributes, types of UPDATE triggers, and the number of processed rows
(which for UPDATE ... FROM ... may be much higher than updated rows).
Fixed by switching to the per-tuple context in GetAllUpdatedColumns().
This is fine for all in-core callers, but external callers may need to
copy the result. But we're not aware of any such callers.
Note the issue was introduced by fc22b6623b, but the macros were later
renamed by f50e888990.
Backpatch to 12, where the issue was introduced.
Reported-by: Tomas Vondra
Reviewed-by: Andres Freund, Tom Lane, Jakub Wartak
Backpatch-through: 12
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execUtils.c
Initialize 'recordXtime' to silence compiler warning.
commit : 525ec837e108db1a41159b8d2caee26b19bae849
author : Heikki Linnakangas <[email protected]>
date : Tue, 6 Jun 2023 20:30:53 +0300
committer: Heikki Linnakangas <[email protected]>
date : Tue, 6 Jun 2023 20:30:53 +0300
In reality, recordXtime will always be set by the getRecordTimestamp
call, but the compiler doesn't necessarily see that.
Back-patch to all supported versions.
Author: Tristan Partin
Discussion: https://www.postgresql.org/message-id/CT5MN8E11U0M.1NYNCHXYUHY41@gonk
M src/backend/access/transam/xlog.c
Doc: explain about dependency tracking for new-style SQL functions.
commit : 1b322c1faee965855ce393b5b23c3d3dc8cb6a2d
author : Tom Lane <[email protected]>
date : Sun, 4 Jun 2023 13:27:34 -0400
committer: Tom Lane <[email protected]>
date : Sun, 4 Jun 2023 13:27:34 -0400
5.14 Dependency Tracking was not updated when we added new-style
SQL functions. Improve that.
Noted by Sami Imseih. Back-patch to v14 where
new-style SQL functions came in.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ddl.sgml
Fix pg_dump's failure to honor dependencies of SQL functions.
commit : d6f549d7a6275e16668faf8cddc95d6759506108
author : Tom Lane <[email protected]>
date : Sun, 4 Jun 2023 13:05:54 -0400
committer: Tom Lane <[email protected]>
date : Sun, 4 Jun 2023 13:05:54 -0400
A new-style SQL function can contain a parse-time dependency
on a unique index, much as views and matviews can (such cases
arise from GROUP BY and ON CONFLICT clauses, for example).
To dump and restore such a function successfully, pg_dump must
postpone the function until after the unique index is created,
which will happen in the post-data part of the dump. Therefore
we have to remove the normal constraint that functions are
dumped in pre-data. Add code similar to the existing logic
that handles this for matviews. I added test cases for both
as well, since code coverage tests showed that we weren't
testing the matview logic.
Per report from Sami Imseih. Back-patch to v14 where
new-style SQL functions came in.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/pg_dump_sort.c
M src/bin/pg_dump/t/002_pg_dump.pl
doc: add missing "the" in LATERAL sentence.
commit : 3d8aefece4b98e8e85e8ed3f27efab732d2a4d71
author : Bruce Momjian <[email protected]>
date : Thu, 1 Jun 2023 10:22:16 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 1 Jun 2023 10:22:16 -0400
Backpatch-through: 11
M doc/src/sgml/queries.sgml
nbtree VACUUM: cope with right sibling link corruption.
commit : 322c9b340adae0b48eed6fe822cbb700a2e6cf34
author : Peter Geoghegan <[email protected]>
date : Thu, 25 May 2023 15:32:53 -0700
committer: Peter Geoghegan <[email protected]>
date : Thu, 25 May 2023 15:32:53 -0700
Avoid "right sibling's left-link doesn't match" errors when vacuuming a
corrupt nbtree index. Just LOG the issue and press on. That way VACUUM
will have a decent chance of finishing off all required processing for
the index (and for the table as a whole).
This error was seen in the field from time to time (it's more than a
theoretical risk), so giving VACUUM the ability to press on like this
has real value. Nothing short of a REINDEX is expected to fix the
underlying index corruption, so giving up (by throwing an error) risks
making a bad situation far worse. Anything that blocks forward progress
by VACUUM like this might go unnoticed for a long time. This could
eventually lead to a wraparound/xidStopLimit outage.
Note that _bt_unlink_halfdead_page() has always been able to bail on
page deletion when the target page's left sibling page was in an
inconsistent state. It now does the same thing (returns false to back
out of the second phase of deletion) when it notices sibling link
corruption in the target page's right sibling page.
This is similar to the work from commit 5b861baa (later backpatched as
commit 43e409ce), which taught nbtree to press on with vacuuming an
index when page deletion fails to "re-find" a downlink in the target
page's parent page. The "re-find" check seems to make VACUUM bail on
page deletion more often in practice, but there is no reason to take any
chances here.
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Heikki Linnakangas <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wzko2q2kP1+UvgJyP9g0mF4hopK0NtQZcxwvMv9_ytGhkQ@mail.gmail.com
Backpatch: 11- (all supported versions).
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
Fix misbehavior of EvalPlanQual checks with multiple result relations.
commit : f8320cc72ddb91182c43ea23e2a721e10daa9b13
author : Tom Lane <[email protected]>
date : Fri, 19 May 2023 14:26:34 -0400
committer: Tom Lane <[email protected]>
date : Fri, 19 May 2023 14:26:34 -0400
The idea of EvalPlanQual is that we replace the query's scan of the
result relation with a single injected tuple, and see if we get a
tuple out, thereby implying that the injected tuple still passes the
query quals. (In join cases, other relations in the query are still
scanned normally.) This logic was not updated when commit 86dc90056
made it possible for a single DML query plan to have multiple result
relations, when the query target relation has inheritance or partition
children. We replaced the output for the current result relation
successfully, but other result relations were still scanned normally;
thus, if any other result relation contained a tuple satisfying the
quals, we'd think the EPQ check passed, even if it did not pass for
the injected tuple itself. This would lead to update or delete
actions getting performed when they should have been skipped due to
a conflicting concurrent update in READ COMMITTED isolation mode.
Fix by blocking all sibling result relations from emitting tuples
during an EvalPlanQual recheck. In the back branches, the fix is
complicated a bit by the need to not change the size of struct
EPQState (else we'd have ABI-breaking changes in offsets in
struct ModifyTableState). Like the back-patches of 3f7836ff6
and 4b3e37993, add a separately palloc'd struct to avoid that.
The logic is the same as in HEAD otherwise.
This is only a live bug back to v14 where 86dc90056 came in.
However, I chose to back-patch the test cases further, on the
grounds that this whole area is none too well tested. I skipped
doing so in v11 though because none of the test applied cleanly,
and it didn't quite seem worth extra work for a branch with only
six months to live.
Per report from Ante Krešić (via Aleksander Alekseev)
Discussion: https://postgr.es/m/CAJ7c6TMBTN3rcz4=AjYhLPD_w3FFT0Wq_C15jxCDn8U4tZnH1g@mail.gmail.com
M src/backend/executor/execMain.c
M src/backend/executor/execScan.c
M src/backend/executor/nodeModifyTable.c
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
M src/test/isolation/expected/eval-plan-qual.out
M src/test/isolation/specs/eval-plan-qual.spec
Avoid naming conflict between transactions.sql and namespace.sql.
commit : 4cdda71d4dc33dd35e8cdbca5dd2e2a7a1234092
author : Tom Lane <[email protected]>
date : Fri, 19 May 2023 10:57:46 -0400
committer: Tom Lane <[email protected]>
date : Fri, 19 May 2023 10:57:46 -0400
Commits 681d9e462 et al added a test case in namespace.sql that
implicitly relied on there not being a table "public.abc".
However, the concurrently-run transactions.sql test creates precisely
such a table, so with the right timing you'd get a failure.
Creating a table named as generically as "abc" in a common schema
seems like bad practice, so fix this by changing the name of
transactions.sql's table. (Compare 2cf8c7aa4.)
Marina Polyakova
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/transactions.out
M src/test/regress/sql/transactions.sql
pageinspect: Fix gist_page_items() with included columns
commit : e72580232c8e61e65af124fb0780c50918eee54c
author : Michael Paquier <[email protected]>
date : Fri, 19 May 2023 12:38:18 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 19 May 2023 12:38:18 +0900
Non-leaf pages of GiST indexes contain key attributes, leaf pages
contain both key and non-key attributes, and gist_page_items() ignored
the handling of non-key attributes. This caused a few problems when
using gist_page_items() on a GiST index with INCLUDE:
- On a non-leaf page, the function would crash.
- On a leaf page, the function would work, but miss to display all the
values for included attributes.
This commit fixes gist_page_items() to handle such cases in a more
appropriate way, and now displays the values of key and non-key
attributes for each item separately in a style consistent with what
ruleutils.c would generate for the attribute list, depending on the page
type dealt with. In a way similar to how a record is displayed, values
would be double-quoted for key or non-key attributes if required.
ruleutils.c did not provide a routine able to control if non-key
attributes should be displayed, so an extended() routine for index
definitions is added to work around the leaf and non-leaf page
differences.
While on it, this commit fixes a third problem related to the amount of
data reported for key attributes. The code originally relied on
BuildIndexValueDescription() (used for error reports on constraints)
that would not print all the data stored in the index but the index
opclass's input type, so this limited the amount of information
available. This switch makes gist_page_items() much cheaper as there is
no need to run ACL checks for each item printed, which is not an issue
anyway as superuser rights are required to execute the functions of
pageinspect. Opclasses whose data cannot be displayed can rely on
gist_page_items_bytea().
The documentation of this function was slightly incorrect for the
output results generated on HEAD and v15, so adjust it on these
branches.
Author: Alexander Lakhin, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 14
M contrib/pageinspect/expected/gist.out
M contrib/pageinspect/gistfuncs.c
M contrib/pageinspect/sql/gist.sql
M doc/src/sgml/pageinspect.sgml
M src/backend/utils/adt/ruleutils.c
M src/include/utils/ruleutils.h
Fix handling of empty ranges and NULLs in BRIN
commit : 40d465cafce0825539a02edeb72966555766fc18
author : Tomas Vondra <[email protected]>
date : Fri, 19 May 2023 00:00:22 +0200
committer: Tomas Vondra <[email protected]>
date : Fri, 19 May 2023 00:00:22 +0200
BRIN indexes did not properly distinguish between summaries for empty
(no rows) and all-NULL ranges, treating them as essentially the same
thing. Summaries were initialized with allnulls=true, and opclasses
simply reset allnulls to false when processing the first non-NULL value.
This however produces incorrect results if the range starts with a NULL
value (or a sequence of NULL values), in which case we forget the range
contains NULL values when adding the first non-NULL value.
This happens because the allnulls flag is used for two separate
purposes - to mark empty ranges (not representing any rows yet) and
ranges containing only NULL values.
Opclasses don't know which of these cases it is, and so don't know
whether to set hasnulls=true. Setting the flag in both cases would make
it correct, but it would also make BRIN indexes useless for queries with
IS NULL clauses. All ranges start empty (and thus allnulls=true), so all
ranges would end up with either allnulls=true or hasnulls=true.
The severity of the issue is somewhat reduced by the fact that it only
happens when adding values to an existing summary with allnulls=true.
This can happen e.g. for small tables (because a summary for the first
range exists for all BRIN indexes), or for tables with large fraction of
NULL values in the indexed columns.
Bulk summarization (e.g. during CREATE INDEX or automatic summarization)
that processes all values at once is not affected by this issue. In this
case the flags were updated in a slightly different way, not forgetting
the NULL values.
To identify empty ranges we use a new flag, stored in an unused bit in
the BRIN tuple header so the on-disk format remains the same. A matching
flag is added to BrinMemTuple, into a 3B gap after bt_placeholder.
That means there's no risk of ABI breakage, although we don't actually
pass the BrinMemTuple to any public API.
We could also skip storing index tuples for empty summaries, but then
we'd have to always process such ranges - even if there are no rows in
large parts of the table (e.g. after a bulk DELETE), it would still
require reading the pages etc. So we store them, but ignore them when
building the bitmap.
Backpatch to 11. The issue exists since BRIN indexes were introduced in
9.5, but older releases are already EOL.
Backpatch-through: 11
Reviewed-by: Justin Pryzby, Matthias van de Meent, Alvaro Herrera
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_tuple.c
M src/include/access/brin_tuple.h
M src/test/modules/brin/expected/summarization-and-inprogress-insertion.out
M src/test/modules/brin/specs/summarization-and-inprogress-insertion.spec
Fix handling of NULLs when merging BRIN summaries
commit : 3f1356e5d60a310b9ceb73497ec49af1b69bafa8
author : Tomas Vondra <[email protected]>
date : Thu, 18 May 2023 13:00:31 +0200
committer: Tomas Vondra <[email protected]>
date : Thu, 18 May 2023 13:00:31 +0200
When merging BRIN summaries, union_tuples() did not correctly update the
target hasnulls/allnulls flags. When merging all-NULL summary into a
summary without any NULL values, the result had both flags set to false
(instead of having hasnulls=true).
This happened because the code only considered the hasnulls flags,
ignoring the possibility the source summary has allnulls=true.
Discovered while investigating issues with handling empty BRIN ranges
and handling of NULL values, but it's a separate problem (has nothing to
do with empty ranges).
Fixed by considering both flags on the source summary, and updating the
hasnulls flag on the target summary.
Backpatch to 11. The bug exists since 9.5 (where BRIN indexes were
introduced), but those releases are EOL already.
Discussion: https://postgr.es/m/9d993d0d-e431-2196-9ccc-0554d0e60154%40enterprisedb.com
M src/backend/access/brin/brin.c
Ensure Soundex difference() function handles empty input sanely.
commit : ccd3623256220b944d9da00df75d91ef4d550362
author : Tom Lane <[email protected]>
date : Tue, 16 May 2023 10:53:42 -0400
committer: Tom Lane <[email protected]>
date : Tue, 16 May 2023 10:53:42 -0400
fuzzystrmatch's difference() function assumes that _soundex()
always initializes its output buffer fully. This was not so for
the case of a string containing no alphabetic characters, resulting
in unstable output and Valgrind complaints.
Fix by using memset() to fill the whole buffer in the early-exit
case. Also make some cosmetic improvements (I didn't care for the
random switches between "instr[0]" and "*instr" notation).
Report and diagnosis by Alexander Lakhin (bug #17935).
Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M contrib/fuzzystrmatch/expected/fuzzystrmatch.out
M contrib/fuzzystrmatch/fuzzystrmatch.c
M contrib/fuzzystrmatch/sql/fuzzystrmatch.sql
Doc: Fix link to fillfactor reloption.
commit : 7deeb02f6d717b5b6aa0a6eb885a477767a8beaa
author : Peter Geoghegan <[email protected]>
date : Wed, 10 May 2023 10:49:46 -0700
committer: Peter Geoghegan <[email protected]>
date : Wed, 10 May 2023 10:49:46 -0700
Fix a link from the "Heap-Only Tuples" documentation section.
Previously, its "fillfactor" link pointed to the "CREATE TABLE"
command's documentation. Now the link directly points to the fillfactor
storage parameter documentation (which is about half way into the
"CREATE TABLE" sect1).
Oversight in commit 115464bb.
Backpatch: 12-, the first version with a usable reloption link.
M doc/src/sgml/storage.sgml