Align organization wording in copyright statement
commit : 5987553fde2cb22b69cf9c7d71a92573b7a3b7c9
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 16 May 2025 11:20:07 -0400
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 16 May 2025 11:20:07 -0400
This aligns the copyright and legal notice wordig with commit
a233a603bab8 and pgweb commit 2d764dbc083ab8. Backpatch down
to all supported versions.
Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Dave Page <dpage@pgadmin.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/744E414E-3F52-404C-97FB-ED9B3AA37DC8@yesql.se
Backpatch-through: 13
M COPYRIGHT
M doc/src/sgml/legal.sgml
Fix Assert failure in XMLTABLE parser
commit : fe29b2a1dab2b3d81e6c350679b7d9b0e1c8c850
author : Richard Guo <rguo@postgresql.org>
date : Thu, 15 May 2025 17:09:04 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Thu, 15 May 2025 17:09:04 +0900
In an XMLTABLE expression, columns can be marked NOT NULL, and the
parser internally fabricates an option named "is_not_null" to
represent this. However, the parser also allows users to specify
arbitrary option names. This creates a conflict: a user can
explicitly use "is_not_null" as an option name and assign it a
non-Boolean value, which violates internal assumptions and triggers an
assertion failure.
To fix, this patch checks whether a user-supplied name collides with
the internally reserved option name and raises an error if so.
Additionally, the internal name is renamed to "__pg__is_not_null" to
further reduce the risk of collision with user-defined names.
Reported-by: Евгений Горбанев <gorbanyoves@basealt.ru>
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Alvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/6bac9886-65bf-4cec-96bd-e304159f28db@basealt.ru
Backpatch-through: 15
M src/backend/parser/gram.y
M src/test/regress/expected/xml.out
M src/test/regress/expected/xml_1.out
M src/test/regress/expected/xml_2.out
M src/test/regress/sql/xml.sql
Add explicit initialization for all PlannerGlobal fields
commit : 2c0ed86d393670c7054d051490063de771f1791e
author : Richard Guo <rguo@postgresql.org>
date : Wed, 14 May 2025 09:59:31 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Wed, 14 May 2025 09:59:31 +0900
When creating a new PlannerGlobal node in standard_planner(), most
fields are explicitly initialized, but a few are not. This doesn't
cause any functional issues, as makeNode() zeroes all fields by
default. However, the inconsistency is undesirable from a clarity and
maintenance perspective.
This patch explicitly initializes the remaining fields to improve
consistency and readability.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAMbWs4-TgQHNOiouqGcuHoBqbJjWyx4UxGKxUY3FrF4trGbcPA@mail.gmail.com
M src/backend/optimizer/plan/planner.c
Fix order of parameters in POD documentation
commit : 6e289f2d5da02b450f00fb57f0facc60bd73c0d1
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 13 May 2025 07:29:14 -0400
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 13 May 2025 07:29:14 -0400
The documentation for log_check() had the parameters in the wrong
order. Also while there, rename %parameters to %params to better
documentation for similar functions which use %params. Backpatch
down to v14 where this was introduced.
Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/9F503B5-32F2-45D7-A0AE-952879AD65F1@yesql.se
Backpatch-through: 14
M src/test/perl/PostgreSQL/Test/Cluster.pm
Fix the race condition in the test added by 7c99dc587.
commit : 8ede692de512750ecb6930213e6d78ed90ef06d0
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 13 May 2025 09:54:29 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 13 May 2025 09:54:29 +0530
After executing ALTER SUBSCRIPTION tap_sub SET PUBLICATION, we did not
wait for the new walsender process to restart. As a result, an INSERT
executed immediately after the ALTER could be decoded and skipped,
considering it is not part of any subscribed publication. And, the old
apply worker could also confirm the LSN of such an INSERT. This could
cause the replication to resume from a point after the INSERT. In such
cases, we miss the expected warning about the missing publication.
To fix this, ensure the walsender has restarted before continuing after
ALTER SUBSCRIPTION.
Reported-by: Tom Lane as per CI
Author: vignesh C <vignesh21@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/1230066.1745992333@sss.pgh.pa.us
M src/test/subscription/t/024_add_drop_pub.pl
Add tab-complete for ALTER DOMAIN ADD [CONSTRAINT]
commit : dbf42b84ac7b4e1dfd1ac6e618e247100b6aa08b
author : Álvaro Herrera <alvherre@kurilemu.de>
date : Sun, 11 May 2025 10:16:45 -0400
committer: Álvaro Herrera <alvherre@kurilemu.de>
date : Sun, 11 May 2025 10:16:45 -0400
We can add tab-completion with "CHECK (" and "NOT NULL" after ALTER
DOMAIN ADD [CONSTRAINT].
ALTER DOMAIN dom ADD -> CHECK (
ALTER DOMAIN dom ADD -> NOT NULL
ALTER DOMAIN dom ADD -> CONSTRAINT
ALTER DOMAIN dom ADD CONSTRAINT nm -> CHECK (
ALTER DOMAIN dom ADD CONSTRAINT nm -> NOT NULL
Author: jian he <jian.universality@gmail.com>
Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://postgr.es/m/CACJufxG_f6LzAT_McC-kKmQWpuWnOYKyNBw8Kv3xzTjPqmeHcA@mail.gmail.com
M src/bin/psql/tab-complete.in.c
Fix comment of tsquerysend()
commit : 0588656366687887d06b0b1f79c9adc142ea1595
author : Álvaro Herrera <alvherre@kurilemu.de>
date : Sun, 11 May 2025 09:47:10 -0400
committer: Álvaro Herrera <alvherre@kurilemu.de>
date : Sun, 11 May 2025 09:47:10 -0400
The comment describes the order in which fields are sent, and it had one
of the fields in the wrong place.
This has been wrong since e6dbcb72fafa (2008), so backpatch all the way
back.
Author: Emre Hasegeli <emre@hasegeli.com>
Discussion: https://postgr.es/m/CAE2gYzzf38bR_R=izhpMxAmqHXKeM5ajkmukh4mNs_oXfxcMCA@mail.gmail.com
M src/backend/utils/adt/tsquery.c
relcache: Avoid memory leak on tables with no CHECK constraints
commit : dc9a2d54fd2571c21c074103cc8ddd93e840b985
author : Álvaro Herrera <alvherre@kurilemu.de>
date : Sun, 11 May 2025 09:22:12 -0400
committer: Álvaro Herrera <alvherre@kurilemu.de>
date : Sun, 11 May 2025 09:22:12 -0400
As complained about by Valgrind, in commit a379061a22a8 I failed to
realize that I was causing rd_att->constr->check to become allocated
when no CHECK constraints exist; previously it'd remain NULL. (This was
my bug, not the mentioned commit author's). Fix by making the
allocation conditional, and set ->check to NULL if unallocated.
Reported-by: Yasir <yasir.hussain.shah@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/202505082025.57ijx3qrbx7u@alvherre.pgsql
M src/backend/utils/cache/relcache.c
Sort includes in alphabetical order
commit : 7b2ad4342603c9796bf28dbe84aca247b2bfa9f8
author : Álvaro Herrera <alvherre@kurilemu.de>
date : Sun, 11 May 2025 09:15:05 -0400
committer: Álvaro Herrera <alvherre@kurilemu.de>
date : Sun, 11 May 2025 09:15:05 -0400
Added by commit 042a66291b04, no backpatch needed.
M src/backend/utils/adt/mcxtfuncs.c
Fix incorrect "return NULL" in BumpAllocLarge().
commit : d4a7e4e179f946e12708f758d30a07f931fd5a84
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 10 May 2025 20:22:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 10 May 2025 20:22:39 -0400
This must be "return MemoryContextAllocationFailure(context, size, flags)"
instead. The effect of this oversight is that if we got a malloc
failure right here, the code would act as though MCXT_ALLOC_NO_OOM
had been specified, whether it was or not. That would likely lead
to a null-pointer-dereference crash at the unsuspecting call site.
Noted while messing with a patch to improve our Valgrind leak
detection support. Back-patch to v17 where this code came in.
M src/backend/utils/mmgr/bump.c
Remove GLOBALTABLESPACE_OID assert for locked buffers.
commit : 4a4ee0c2c1e53401924101945ac3d517c0a8a559
author : Noah Misch <noah@leadboat.com>
date : Sat, 10 May 2025 07:36:27 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 10 May 2025 07:36:27 -0700
Commit f4ece891fc2f3f96f0571720a1ae30db8030681b added the assertion in
an attempt to catch some defects even after VACUUM FULL or REINDEX.
However, IsCatalogTextUniqueIndexOid(tag.relNumber) always returns false
after a relfilenode change, provoking unintended assertion failures.
Reported-by: Adam Guo <adamguo@amazon.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Bug: #18912
Discussion: https://postgr.es/m/18912-a41c9bd0e0ad19b1@postgresql.org
M src/backend/storage/buffer/bufmgr.c
doc PG 18 relnotes: mv. hash joins and GROUP BY item to General
commit : 99ddf8615c215bb81655574b9a9f7e8ebb2cb89e
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 9 May 2025 23:39:34 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 9 May 2025 23:39:34 -0400
Reported-by: David Rowley
Discussion: https://postgr.es/m/CAApHDvqJz+Zf7a6abisqoTGottDSRD+YPx=aQSgCsCKD476vGA@mail.gmail.com
M doc/src/sgml/release-18.sgml
aio: Use runtime arguments with injections points in tests
commit : c259ba881c1098e75b53d5b232600b49e3301058
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 10 May 2025 12:36:57 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 10 May 2025 12:36:57 +0900
This cleans up the code related to the testing infrastructure of AIO
that used injection points, switching the test code to use the new
facility for injection points added by 371f2db8b05e rather than tweaks
to pass and reset arguments to the callbacks run.
This removes all the dependencies to USE_INJECTION_POINTS in the AIO
code. pgaio_io_call_inj(), pgaio_inj_io_get() and pgaio_inj_cur_handle
are now gone.
Reviewed-by: Greg Burd <greg@burd.me>
Discussion: https://postgr.es/m/Z_y9TtnXubvYAApS@paquier.xyz
M src/backend/storage/aio/aio.c
M src/backend/storage/aio/method_worker.c
M src/include/storage/aio_internal.h
M src/test/modules/test_aio/test_aio.c
injection_points: Add support and tests for runtime arguments
commit : 36e5fda6326045f4523568d4c9d682ee5d2f0867
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 10 May 2025 07:40:25 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 10 May 2025 07:40:25 +0900
This commit provides some test coverage for the runtime arguments of
injection points, for both INJECTION_POINT_CACHED() and
INJECTION_POINT(), as extended in 371f2db8b05e.
The SQL functions injection_points_cached() and injection_points_run()
are extended so as it is possible to pass an optional string value to
them.
Reviewed-by: Greg Burd <greg@burd.me>
Discussion: https://postgr.es/m/Z_y9TtnXubvYAApS@paquier.xyz
M src/test/modules/injection_points/expected/injection_points.out
M src/test/modules/injection_points/injection_points–1.0.sql
M src/test/modules/injection_points/injection_points.c
M src/test/modules/injection_points/sql/injection_points.sql
Add support for runtime arguments in injection points
commit : 371f2db8b05e4d46cbf489f05cbfc4d6ed6976d4
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 10 May 2025 06:56:26 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 10 May 2025 06:56:26 +0900
The macros INJECTION_POINT() and INJECTION_POINT_CACHED() are extended
with an optional argument that can be passed down to the callback
attached when an injection point is run, giving to callbacks the
possibility to manipulate a stack state given by the caller. The
existing callbacks in modules injection_points and test_aio have their
declarations adjusted based on that.
da7226993fd4 (core AIO infrastructure) and 93bc3d75d8e1 (test_aio) and
been relying on a set of workarounds where a static variable called
pgaio_inj_cur_handle is used as runtime argument in the injection point
callbacks used by the AIO tests, in combination with a TRY/CATCH block
to reset the argument value. The infrastructure introduced in this
commit will be reused for the AIO tests, simplifying them.
Reviewed-by: Greg Burd <greg@burd.me>
Discussion: https://postgr.es/m/Z_y9TtnXubvYAApS@paquier.xyz
M doc/src/sgml/xfunc.sgml
M src/backend/access/gin/ginbtree.c
M src/backend/access/heap/heapam.c
M src/backend/access/index/genam.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/xlog.c
M src/backend/commands/indexcmds.c
M src/backend/executor/nodeAgg.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/libpq/be-secure.c
M src/backend/postmaster/autovacuum.c
M src/backend/storage/aio/aio.c
M src/backend/tcop/backend_startup.c
M src/backend/tcop/postgres.c
M src/backend/utils/cache/catcache.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/init/postinit.c
M src/backend/utils/misc/injection_point.c
M src/include/utils/injection_point.h
M src/test/modules/injection_points/injection_points.c
M src/test/modules/test_aio/test_aio.c
M src/test/modules/test_slru/test_multixact.c
doc PG 18 relnotes: fix missing parens for crc32c()
commit : 89372d0aaa4a6f0e560acdf9014c5ad66fdde1b1
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 9 May 2025 14:15:54 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 9 May 2025 14:15:54 -0400
Reported-by: Steven Niu
Discussion: https://postgr.es/m/CABBtG=ejqK58cFWpw3etVZfQfhjC-qOqV+9GQWRnLO+p9wYMbw@mail.gmail.com
M doc/src/sgml/release-18.sgml
Skip RSA-PSS ssl test when using LibreSSL.
commit : 95129709fd9b153226c1874549d37fa9246b78aa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 May 2025 12:29:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 May 2025 12:29:01 -0400
Presently, LibreSSL does not have working support for RSA-PSS,
so disable that test. Per discussion at
https://marc.info/?l=libressl&m=174664225002441&w=2
they do intend to fix this, but it's a ways off yet.
Reported-by: Thomas Munro <thomas.munro@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CA+hUKG+fLqyweHqFSBcErueUVT0vDuSNWui-ySz3+d_APmq7dw@mail.gmail.com
Backpatch-through: 15
M src/test/ssl/t/002_scram.pl
Hack one ssl test case to pass with current LibreSSL.
commit : 75d73331d0146871c846e849be626cb220e7e013
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 May 2025 11:53:51 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 May 2025 11:53:51 -0400
With LibreSSL, our test of error logging for cert chain depths > 0
reports the wrong certificate. This is almost certainly their bug
not ours, so just tweak the test to accept their answer.
No back-patch needed, since this test case wasn't enabled before
e0f373ee4.
Reported-by: Thomas Munro <thomas.munro@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CA+hUKG+fLqyweHqFSBcErueUVT0vDuSNWui-ySz3+d_APmq7dw@mail.gmail.com
M src/test/ssl/t/001_ssltests.pl
Centralize ssl tests' check for whether we're using LibreSSL.
commit : 0aaf69965dbd92072911cc46e4b28b9a37c38fad
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 May 2025 11:50:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 May 2025 11:50:33 -0400
Right now there's only one caller, so that this is merely
an exercise in shoving code from one module to another,
but there will shortly be another one. It seems better to
avoid having two copies of this highly-subject-to-change test.
Back-patch to v15, where we first introduced some tests that
don't work with LibreSSL.
Reported-by: Thomas Munro <thomas.munro@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CA+hUKG+fLqyweHqFSBcErueUVT0vDuSNWui-ySz3+d_APmq7dw@mail.gmail.com
Backpatch-through: 15
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/SSL/Backend/OpenSSL.pm
M src/test/ssl/t/SSL/Server.pm
doc: Put new options in consistent order on man pages
commit : bc35adee8d7ad38e7bef40052f196be55decddec
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 9 May 2025 08:55:05 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 9 May 2025 08:55:05 +0200
M doc/src/sgml/ref/pg_combinebackup.sgml
M doc/src/sgml/ref/pg_resetwal.sgml
M doc/src/sgml/ref/pg_restore.sgml
M doc/src/sgml/ref/pg_verifybackup.sgml
M doc/src/sgml/ref/pgupgrade.sgml
Use 'void *' for arbitrary buffers, 'uint8 *' for byte arrays
commit : b28c59a6cd089902e66a91e0d0974da34d1c922b
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 8 May 2025 22:01:25 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 8 May 2025 22:01:25 +0300
A 'void *' argument suggests that the caller might pass an arbitrary
struct, which is appropriate for functions like libc's read/write, or
pq_sendbytes(). 'uint8 *' is more appropriate for byte arrays that
have no structure, like the cancellation keys or SCRAM tokens. Some
places used 'char *', but 'uint8 *' is better because 'char *' is
commonly used for null-terminated strings. Change code around SCRAM,
MD5 authentication, and cancellation key handling to follow these
conventions.
Discussion: https://www.postgresql.org/message-id/61be9e31-7b7d-49d5-bc11-721800d89d64@eisentraut.org
M contrib/dblink/dblink.c
M contrib/postgres_fdw/connection.c
M src/backend/libpq/auth-scram.c
M src/backend/libpq/auth.c
M src/backend/libpq/crypt.c
M src/backend/storage/ipc/procsignal.c
M src/backend/utils/init/globals.c
M src/common/base64.c
M src/common/md5_common.c
M src/common/scram-common.c
M src/include/common/base64.h
M src/include/common/md5.h
M src/include/common/scram-common.h
M src/include/libpq/auth.h
M src/include/libpq/crypt.h
M src/include/libpq/pqcomm.h
M src/include/miscadmin.h
M src/include/storage/procsignal.h
M src/interfaces/libpq/fe-auth-scram.c
M src/interfaces/libpq/fe-auth.c
M src/interfaces/libpq/fe-cancel.c
M src/interfaces/libpq/fe-misc.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-int.h
Use more mundane 'int' type for cancel key lengths in libpq
commit : 965213d9c56a671086525a65f5427653b4a66350
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 8 May 2025 22:01:20 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 8 May 2025 22:01:20 +0300
The documented max length of a cancel key is 256 bytes, so it fits in
uint8. It nevertheless seems weird to not just use 'int', like in
commit 0f1433f053 for the backend.
Discussion: https://www.postgresql.org/message-id/61be9e31-7b7d-49d5-bc11-721800d89d64%40eisentraut.org
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-int.h
PG 18 relnotes: adjust RETURNING new/old item
commit : 9d710a1ac0912244934277a99fe4666df3d2b825
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 8 May 2025 11:11:08 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 8 May 2025 11:11:08 -0400
Reported-by: jian he
Discussion: https://postgr.es/m/CACJufxFM1avdwu=OrTx_uMAjTDbFOj1Gp7mnNHOofTVj9QtmRw@mail.gmail.com
M doc/src/sgml/release-18.sgml
doc: Fix title markup for AT TIME ZONE and AT LOCAL
commit : 8fcc6487809efa5508a4b70049402236a53be84d
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 8 May 2025 13:53:16 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 8 May 2025 13:53:16 +0200
The title for AT TIME ZONE and AT LOCAL was accidentally wrapping the
"and" in the <literal> tag. Backpatch to v17 where it was introduced
in 97957fdbaa42.
Author: Noboru Saito <noborusai@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Tatsuo Ishii <ishii@postgresql.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAAM3qn+7QUWW9R6_YwPKXmky0xGE4n63U3EsxZeWE_QtogeU8g@mail.gmail.com
Backpatch-through: 17
M doc/src/sgml/func.sgml
Track the number of presorted outer pathkeys in MergePath
commit : c06e909c26f070dee78f73c35565d6f4a4ffdcda
author : Richard Guo <rguo@postgresql.org>
date : Thu, 8 May 2025 18:21:32 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Thu, 8 May 2025 18:21:32 +0900
When creating an explicit Sort node for the outer path of a mergejoin,
we need to determine the number of presorted keys of the outer path to
decide whether explicit incremental sort can be applied. Currently,
this is done by repeatedly calling pathkeys_count_contained_in.
This patch caches the number of presorted outer pathkeys in MergePath,
allowing us to save several calls to pathkeys_count_contained_in. It
can be considered a complement to the changes in commit 828e94c9d.
Reported-by: David Rowley <dgrowleyml@gmail.com>
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://postgr.es/m/CAApHDvqvBireB_w6x8BN5txdvBEHxVgZBt=rUnpf5ww5P_E_ww@mail.gmail.com
M src/backend/foreign/foreign.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/pathnode.c
M src/include/nodes/pathnodes.h
M src/include/optimizer/cost.h
M src/include/optimizer/pathnode.h
Suppress unnecessary explicit sorting for EPQ mergejoin path
commit : 773db22269d474fab46d25e9e15b1e55252cf92c
author : Richard Guo <rguo@postgresql.org>
date : Thu, 8 May 2025 18:20:18 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Thu, 8 May 2025 18:20:18 +0900
When building a ForeignPath for a joinrel, if there's a possibility
that EvalPlanQual will be executed, we must identify a suitable path
for EPQ checks. If the outer or inner path of the chosen path is a
ForeignPath representing a pushed-down join, we replace it with its
fdw_outerpath to ensure that the EPQ check path consists entirely of
local joins.
If the chosen path is a MergePath, and its outer or inner path is a
ForeignPath that is not already well enough ordered, the MergePath
will have non-NIL outersortkeys or innersortkeys indicating the
desired ordering to be created by an explicit Sort node. If we then
replace the outer or inner path with its corresponding fdw_outerpath,
and that path is already sufficiently ordered, we end up in an
inconsistent state: the MergePath has non-NIL outersortkeys or
innersortkeys, and its input path is already properly ordered. This
inconsistency can result in an Assert failure or the addition of a
redundant Sort node.
To fix, check if the new outer or inner path of a MergePath is already
properly sorted, and set its outersortkeys or innersortkeys to NIL if
so.
Bug: #18902
Reported-by: Nikita Kalinin <n.kalinin@postgrespro.ru>
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://postgr.es/m/18902-71c1bed2b9f7c46f@postgresql.org
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/foreign/foreign.c
doc PG 18 relnotes: adjust pg_log_backend_memory_contexts()
commit : 9fef27a83b3112655509241107834086628f7fa8
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 7 May 2025 21:11:16 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 7 May 2025 21:11:16 -0400
Reported-by: David Rowley
Discussion: https://postgr.es/m/CAApHDvrGLBqs_Vm9COMY7uBDvUDMKds7RwC20YjEPf+XRTY9XQ@mail.gmail.com
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: add pg_log_backend_memory_contexts() mention
commit : f8d49aa130b213d66beae0f574917c1884f7b2a9
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 7 May 2025 20:36:21 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 7 May 2025 20:36:21 -0400
Now zero-based.
Reported-by: David Rowley
Discussion: https://postgr.es/m/CAApHDvqMfTBdfwc0Z-tHXLnBMKJLYEZDApgUzA7x_PUDZsY3GA@mail.gmail.com
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: adjust pgbench per-script reporting item
commit : 69aca072ebf64dcf36958b6958ce4388480fcba2
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 7 May 2025 16:56:26 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 7 May 2025 16:56:26 -0400
Also run src/tools/add_commit_links.pl for a previous commit.
Reported-by: Yugo Nagata
Discussion: https://postgr.es/m/20250507195941.c6e1b48c73f062b727f686a8@sraoss.co.jp
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: mention GROUP SET fixes
commit : 3bd5271729c5496e3712c9ffbb002a6bef99a4c3
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 7 May 2025 16:39:26 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 7 May 2025 16:39:26 -0400
Reported-by: Richard Guo
Discussion: https://postgr.es/m/CAMbWs4_asKPqTCt0h9pp=zHc9vmPcnczbHeF6Xkxn1LhLapcTQ@mail.gmail.com
M doc/src/sgml/release-18.sgml
Remove pg_replication_origin's TOAST table.
commit : 16bf24e0e471c975f0877d6612eacdae7ce4a30e
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 7 May 2025 14:47:36 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 7 May 2025 14:47:36 -0500
A few places that access this catalog don't set up an active
snapshot before potentially accessing its TOAST table. However,
roname (the replication origin name) is the only varlena column, so
this is only a problem if the name requires out-of-line storage.
This commit removes its TOAST table to avoid needing to set up a
snapshot. It also places a limit on replication origin names so
that attempts to set long names will fail with a more user-friendly
error. Those chosen limit of 512 bytes should be sufficient to
avoid "row is too big" errors independent of BLCKSZ, but it should
also be lenient enough for all reasonable use-cases.
Bumps catversion.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Euler Taveira <euler@eulerto.com>
Reviewed-by: Nisha Moond <nisha.moond412@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/ZvMSUPOqUU-VNADN%40nathan
M doc/src/sgml/func.sgml
M src/backend/catalog/catalog.c
M src/backend/replication/logical/origin.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_replication_origin.h
M src/include/replication/origin.h
M src/test/regress/expected/misc_functions.out
M src/test/regress/expected/misc_sanity.out
M src/test/regress/sql/misc_functions.sql
M src/test/regress/sql/misc_sanity.sql
Prevent premature nbtree array advancement.
commit : 5f4d98d4f3718cf7a0597c37f3ee4a4485de6ef8
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 7 May 2025 15:20:42 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 7 May 2025 15:20:42 -0400
nbtree array index scans could fail to return matching tuples in rare
cases where the missed tuples cover key space that the scan's arrays
incorrectly indicate has already been read. These cases involved nearby
tuples with NULL values that were evaluated using a skip array key while
in pstate.forcenonrequired mode.
To fix, prevent forcenonrequired mode from prematurely advancing the
scan's array keys beyond key space that the scan has yet to read tuples
from: reset the scan's array keys (to the first elements in the current
scan direction) before the _bt_checkkeys call for pstate.finaltup. That
way _bt_checkkeys starts from a clean slate, which ensures that it will
call _bt_advance_array_keys (while passing it sktrig_required=true).
This reliably restores the invariant that the scan's arrays always
accurately track its progress through the index's key space (at least
when the scan is "between pages").
Oversight in commit 8a510275, which optimized nbtree search scan key
comparisons.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://postgr.es/m/CAH2-WzmodSE+gpTd1CRGU9ez8ytyyDS+Kns2r9NzgUp1s56kpw@mail.gmail.com
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtutils.c
nbtree: tighten up array recheck rules.
commit : 7e25c9363a82b6974c1ca2303ae8ded98af3bb24
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 7 May 2025 15:17:40 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 7 May 2025 15:17:40 -0400
Be more conservative when performing a scheduled recheck of an nbtree
scan's array keys once on the next page, having set so->scanBehind: back
out of reading the page (perform another primitive scan instead) when
the next page's high key/finaltup has an untruncated prefix of matching
values and truncated suffix attributes associated with lower-order keys.
In other words, stop assuming that the lower-order keys have been
satisfied by the truncated suffix attributes in this context (only do so
when considering scheduling a recheck within _bt_advance_array_keys).
The new behavior is more logical: if the next page read after setting
so->scanBehind can only contain tuples that are themselves "behind the
scan", that's reason enough to cut our losses. In general, when we set
so->scanBehind, we only expect to perform one recheck on the next page
to make a final decision about whether or not to continue the current
primitive index scan. It seems unprincipled for the recheck to allow a
_bt_readpage to continue unless the scan's arrays will advance/unless
the page might actually contain relevant tuples.
In practice it is highly unlikely that things will line up like this
(the untruncated prefix of attribute values from the next page's high
key is seldom an exact match for their corresponding array's current
element following array advancement on the original/previous page).
That gives us all the more reason to keep things simple and consistent.
This was arguably an oversight in commit 9a2e2a285a, which improved
nbtree array primitive scan scheduling.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WzkXzJajgyW-pCQ7vaDPhaT3huU+Zw_j448rpCBEsu2YOQ@mail.gmail.com
M src/backend/access/nbtree/nbtutils.c
pg_dumpall: Add --sequence-data.
commit : acea3fc49f006a08e6c540ac0bacc2c2577a5ff9
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 7 May 2025 13:36:51 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 7 May 2025 13:36:51 -0500
I recently added this option to pg_dump, but I forgot to add it to
pg_dumpall, too. There's probably little use for it at the moment,
but we will need it if/when we teach pg_upgrade to use pg_dumpall
to dump the database schemas.
Oversight in commit 9c49f0e8cd.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/aBE8rHFo922xQUwh%40nathan
M doc/src/sgml/ref/pg_dumpall.sgml
M src/bin/pg_dump/pg_dumpall.c
Refactor ChangeVarNodesExtended() using the custom callback
commit : ab42d643c14509cf1345588f55d798284b11a91e
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 7 May 2025 11:10:16 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 7 May 2025 11:10:16 +0300
fc069a3a6319 implemented Self-Join Elimination (SJE) and put related logic
to ChangeVarNodes_walker(). This commit provides refactoring to remove the
SJE-related logic from ChangeVarNodes_walker() but adds a custom callback to
ChangeVarNodesExtended(), which has a chance to process a node before
ChangeVarNodes_walker(). Passing this callback to ChangeVarNodesExtended()
allows SJE-related node handling to be kept within the analyzejoins.c.
Reported-by: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/CAMbWs49PE3CvnV8vrQ0Dr%3DHqgZZmX0tdNbzVNJxqc8yg-8kDQQ%40mail.gmail.com
Author: Andrei Lepikhov <lepihov@gmail.com>
Author: Alexander Korotkov <aekorotkov@gmail.com>
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/rewrite/rewriteManip.c
M src/include/rewrite/rewriteManip.h
M src/tools/pgindent/typedefs.list
doc: Put some psql documentation pieces back into alphabetical order
commit : 2448c7a9e00536bac8704bff402bc02c2b2f23de
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 4 May 2025 20:38:42 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 4 May 2025 20:38:42 +0200
M doc/src/sgml/ref/psql-ref.sgml
Remove some tabs in C string literals
commit : c0cf28255178f1d6e20a26c8df0a4cfdd0ab3b24
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 4 May 2025 13:04:46 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 4 May 2025 13:04:46 +0200
M src/test/modules/oauth_validator/oauth_hook_client.c
doc: Add link to table
commit : c11bd5f500f0b774a766ed67b769d63abdbdae91
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 4 May 2025 12:43:10 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 4 May 2025 12:43:10 +0200
Formal tables should generally have an xref in the text that points to
them. Add them here.
M doc/src/sgml/protocol.sgml
doc: Fix up spacing around verbatim DocBook elements
commit : a2c6d84acd72d6c43ab7361fb34fa4f758da7a41
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 4 May 2025 12:39:21 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 4 May 2025 12:39:21 +0200
M doc/src/sgml/func.sgml
Fix some comments related to IO workers
commit : c4c236ab5c416131baa7109cdc25ccaadd4d7158
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 7 May 2025 14:55:57 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 7 May 2025 14:55:57 +0900
IO workers are treated as auxiliary processes. The comments fixed in
this commit stated that there could be only one auxiliary process of
each BackendType at the same time. This is not true for IO workers, as
up to MAX_IO_WORKERS of them can co-exist at the same time.
Author: Cédric Villemain <Cedric.Villemain@data-bene.io>
Co-authored-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/e4a3ac45-abce-4b58-a043-b4a31cd11113@Data-Bene.io
M src/backend/storage/ipc/procsignal.c
M src/backend/storage/ipc/sinvaladt.c
M src/include/miscadmin.h
Fix whitespace
commit : 09a47c68e2fce446849ce0262eff1bfbf921fa18
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 7 May 2025 07:01:03 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 7 May 2025 07:01:03 +0200
M contrib/dblink/t/001_auth_scram.pl
M contrib/pg_stat_statements/sql/squashing.sql
doc PG 18 relnotes: adjust partition planning item
commit : b560ce7884a32ba6409adab340b9732dcd210284
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 6 May 2025 21:15:44 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 6 May 2025 21:15:44 -0400
Reported-by: David Rowley
Discussion: https://postgr.es/m/CAApHDvqgK7uqPZAwxsfBiFhvBHHB0txaUxhUrdwG4d5Mik_RnA@mail.gmail.com
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: small adjustments regarding options
commit : ada78f9bef2eb47b0711a9eec07529c0da7c5672
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 6 May 2025 17:17:46 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 6 May 2025 17:17:46 -0400
Reported-by: jian he
Discussion: https://postgr.es/m/CACJufxH1jo=hv77AK0HUJYBBMuPmr6+JT+8g-yovuJmHUPGOZQ@mail.gmail.com
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: move partition locking item to General Perf
commit : 575f6003eddbaf0d08195e1e48aee9a741518d6d
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 6 May 2025 16:03:56 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 6 May 2025 16:03:56 -0400
Reported-by: Amit Langote
Discussion: https://postgr.es/m/CA+HiwqE+8Pui_NCCC7zgacnet0Cf3tc_vU+P=nhLDES-8xuCUw@mail.gmail.com
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: adjust partition items
commit : 45750c6cfe8d1ba6018366642a35e661b9d6f3af
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 6 May 2025 15:45:03 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 6 May 2025 15:45:03 -0400
Reported-by: David Rowley
Discussion: https://postgr.es/m/CAApHDvo+BrVTXMBPjNXBTnAovJWN9+-dYc0kN7rSDqdNvpggZQ@mail.gmail.com
M doc/src/sgml/release-18.sgml
Stamp 18beta1.
commit : caa76b91a60681dff0bf193b64d4dcdc1014e036
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 May 2025 16:25:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 May 2025 16:25:46 -0400
M configure
M configure.ac
M meson.build
doc PG 18 relnotes: reword OAuth item
commit : c0e6aace02b6c47e7b99d49cc0e88918349ac30a
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 5 May 2025 15:42:03 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 5 May 2025 15:42:03 -0400
Reported-by: Jacob Champion
Discussion: https://postgr.es/m/CAOYmi+mEQOqBSJas5V5t__b+6h_MLxyy3JFrVJEq638fnNxi0A@mail.gmail.com
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: add mention of pg_stat_reset_backend_stats()
commit : 0de2e1c8b5423c07700112f44499a5fdc0ab0991
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 5 May 2025 14:56:58 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 5 May 2025 14:56:58 -0400
This is for WAL statistics.
Reported-by: Bertrand Drouvot
Discussion: https://postgr.es/m/aBjGlj+Yi++fVRQt@ip-10-97-1-34.eu-west-3.compute.internal
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: adjust hash item
commit : 092e72a93023fc9cea9cf673ab577ccd8537fb69
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 5 May 2025 12:30:35 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 5 May 2025 12:30:35 -0400
Reported-by: David Rowley
Discussion: https://postgr.es/m/CAApHDvrNmGncNgZMh2oBG5K-+4d1LGJgzrz7180OcHRT1VFojw@mail.gmail.com
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: split partition optimizer item into two
commit : cf847d6340a6620251315fccae3bd87d2116d308
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 5 May 2025 11:59:56 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 5 May 2025 11:59:56 -0400
Reported-by: David Rowley
Discussion: https://postgr.es/m/CAApHDvohfoJ0D9eiUuVyHU_kq2Y7A_jAjWVsUt0Fm7Gw1Q=1cQ@mail.gmail.com
M doc/src/sgml/release-18.sgml
With GB18030, prevent SIGSEGV from reading past end of allocation.
commit : 627acc3caa74caa736b2c5587e944d2ea510ea67
author : Noah Misch <noah@leadboat.com>
date : Mon, 5 May 2025 04:52:04 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 5 May 2025 04:52:04 -0700
With GB18030 as source encoding, applications could crash the server via
SQL functions convert() or convert_from(). Applications themselves
could crash after passing unterminated GB18030 input to libpq functions
PQescapeLiteral(), PQescapeIdentifier(), PQescapeStringConn(), or
PQescapeString(). Extension code could crash by passing unterminated
GB18030 input to jsonapi.h functions. All those functions have been
intended to handle untrusted, unterminated input safely.
A crash required allocating the input such that the last byte of the
allocation was the last byte of a virtual memory page. Some malloc()
implementations take measures against that, making the SIGSEGV hard to
reach. Back-patch to v13 (all supported versions).
Author: Noah Misch <noah@leadboat.com>
Author: Andres Freund <andres@anarazel.de>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Backpatch-through: 13
Security: CVE-2025-4207
M src/backend/utils/mb/mbutils.c
M src/common/jsonapi.c
M src/common/wchar.c
M src/include/mb/pg_wchar.h
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-misc.c
M src/test/modules/test_escape/test_escape.c
M src/test/regress/expected/conversion.out
M src/test/regress/sql/conversion.sql
Refactor test_escape.c for additional ways of testing.
commit : 5be213caaa1a9a65dfdbbf400b6a53b5e743b8d1
author : Noah Misch <noah@leadboat.com>
date : Mon, 5 May 2025 04:52:04 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 5 May 2025 04:52:04 -0700
Start the file with static functions not specific to pe_test_vectors
tests. This way, new tests can use them without disrupting the file's
layout. Change report_result() PQExpBuffer arguments to plain strings.
Back-patch to v13 (all supported versions), for the next commit.
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Backpatch-through: 13
Security: CVE-2025-4207
M src/test/modules/test_escape/test_escape.c
Translation updates
commit : 18c4fff640b5a5a12efc61fb202355675b9ce904
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 5 May 2025 12:04:49 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 5 May 2025 12:04:49 +0200
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: f90ee4803c30491e5c49996b973b8a30de47bfb2
M src/backend/po/es.po
M src/backend/po/ja.po
M src/backend/po/ka.po
M src/backend/po/ko.po
M src/backend/po/ru.po
M src/backend/po/sv.po
M src/backend/po/uk.po
M src/bin/initdb/po/de.po
M src/bin/initdb/po/es.po
M src/bin/initdb/po/fr.po
M src/bin/initdb/po/ja.po
M src/bin/initdb/po/ka.po
M src/bin/initdb/po/ko.po
M src/bin/initdb/po/pt_BR.po
M src/bin/initdb/po/ru.po
M src/bin/initdb/po/sv.po
M src/bin/initdb/po/uk.po
M src/bin/initdb/po/zh_CN.po
M src/bin/pg_amcheck/po/de.po
M src/bin/pg_amcheck/po/es.po
M src/bin/pg_amcheck/po/fr.po
M src/bin/pg_amcheck/po/ja.po
M src/bin/pg_amcheck/po/ka.po
M src/bin/pg_amcheck/po/ko.po
M src/bin/pg_amcheck/po/ru.po
M src/bin/pg_amcheck/po/sv.po
M src/bin/pg_amcheck/po/uk.po
M src/bin/pg_archivecleanup/po/de.po
M src/bin/pg_archivecleanup/po/es.po
M src/bin/pg_archivecleanup/po/fr.po
M src/bin/pg_archivecleanup/po/ja.po
M src/bin/pg_archivecleanup/po/ka.po
M src/bin/pg_archivecleanup/po/ko.po
M src/bin/pg_archivecleanup/po/ru.po
M src/bin/pg_archivecleanup/po/sv.po
M src/bin/pg_archivecleanup/po/uk.po
M src/bin/pg_archivecleanup/po/zh_CN.po
M src/bin/pg_basebackup/po/de.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_basebackup/po/fr.po
M src/bin/pg_basebackup/po/ja.po
M src/bin/pg_basebackup/po/ka.po
M src/bin/pg_basebackup/po/ko.po
M src/bin/pg_basebackup/po/ru.po
M src/bin/pg_basebackup/po/sv.po
M src/bin/pg_basebackup/po/uk.po
M src/bin/pg_checksums/po/de.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_checksums/po/fr.po
M src/bin/pg_checksums/po/ja.po
M src/bin/pg_checksums/po/ka.po
M src/bin/pg_checksums/po/ko.po
M src/bin/pg_checksums/po/ru.po
M src/bin/pg_checksums/po/sv.po
M src/bin/pg_checksums/po/uk.po
M src/bin/pg_combinebackup/po/LINGUAS
M src/bin/pg_combinebackup/po/de.po
A src/bin/pg_combinebackup/po/es.po
A src/bin/pg_combinebackup/po/fr.po
M src/bin/pg_combinebackup/po/ja.po
M src/bin/pg_combinebackup/po/ka.po
A src/bin/pg_combinebackup/po/ko.po
A src/bin/pg_combinebackup/po/ru.po
A src/bin/pg_combinebackup/po/sv.po
A src/bin/pg_combinebackup/po/uk.po
M src/bin/pg_config/po/es.po
M src/bin/pg_config/po/fr.po
M src/bin/pg_config/po/ka.po
M src/bin/pg_config/po/ko.po
M src/bin/pg_config/po/ru.po
M src/bin/pg_config/po/sv.po
M src/bin/pg_config/po/uk.po
M src/bin/pg_config/po/zh_CN.po
M src/bin/pg_controldata/po/es.po
M src/bin/pg_controldata/po/fr.po
M src/bin/pg_controldata/po/ja.po
M src/bin/pg_controldata/po/ka.po
M src/bin/pg_controldata/po/ko.po
M src/bin/pg_controldata/po/ru.po
M src/bin/pg_controldata/po/sv.po
M src/bin/pg_controldata/po/uk.po
M src/bin/pg_controldata/po/zh_TW.po
M src/bin/pg_ctl/po/de.po
M src/bin/pg_ctl/po/es.po
M src/bin/pg_ctl/po/fr.po
M src/bin/pg_ctl/po/ja.po
M src/bin/pg_ctl/po/ka.po
M src/bin/pg_ctl/po/ko.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_ctl/po/sv.po
M src/bin/pg_ctl/po/uk.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_dump/po/ja.po
M src/bin/pg_dump/po/ka.po
M src/bin/pg_dump/po/ko.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/sv.po
M src/bin/pg_dump/po/uk.po
M src/bin/pg_resetwal/po/es.po
M src/bin/pg_resetwal/po/fr.po
M src/bin/pg_resetwal/po/ja.po
M src/bin/pg_resetwal/po/ka.po
M src/bin/pg_resetwal/po/ko.po
M src/bin/pg_resetwal/po/ru.po
M src/bin/pg_resetwal/po/sv.po
M src/bin/pg_resetwal/po/uk.po
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/ja.po
M src/bin/pg_rewind/po/ka.po
M src/bin/pg_rewind/po/ko.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_rewind/po/sv.po
M src/bin/pg_rewind/po/uk.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_fsync/po/fr.po
M src/bin/pg_test_fsync/po/ka.po
M src/bin/pg_test_fsync/po/ko.po
M src/bin/pg_test_fsync/po/ru.po
M src/bin/pg_test_fsync/po/sv.po
M src/bin/pg_test_fsync/po/uk.po
M src/bin/pg_test_timing/po/es.po
M src/bin/pg_test_timing/po/fr.po
M src/bin/pg_test_timing/po/ka.po
M src/bin/pg_test_timing/po/ko.po
M src/bin/pg_test_timing/po/ru.po
M src/bin/pg_test_timing/po/sv.po
M src/bin/pg_test_timing/po/uk.po
M src/bin/pg_upgrade/po/de.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/fr.po
M src/bin/pg_upgrade/po/ja.po
M src/bin/pg_upgrade/po/ka.po
M src/bin/pg_upgrade/po/ko.po
M src/bin/pg_upgrade/po/ru.po
M src/bin/pg_upgrade/po/sv.po
M src/bin/pg_upgrade/po/uk.po
M src/bin/pg_verifybackup/po/de.po
M src/bin/pg_verifybackup/po/ja.po
M src/bin/pg_verifybackup/po/ka.po
M src/bin/pg_waldump/po/de.po
M src/bin/pg_waldump/po/es.po
M src/bin/pg_waldump/po/fr.po
M src/bin/pg_waldump/po/ka.po
M src/bin/pg_waldump/po/ko.po
M src/bin/pg_waldump/po/ru.po
M src/bin/pg_waldump/po/sv.po
M src/bin/pg_waldump/po/uk.po
M src/bin/pg_walsummary/po/LINGUAS
A src/bin/pg_walsummary/po/es.po
A src/bin/pg_walsummary/po/fr.po
M src/bin/pg_walsummary/po/ka.po
A src/bin/pg_walsummary/po/ko.po
A src/bin/pg_walsummary/po/ru.po
A src/bin/pg_walsummary/po/sv.po
A src/bin/pg_walsummary/po/uk.po
M src/bin/psql/po/cs.po
M src/bin/psql/po/de.po
M src/bin/psql/po/el.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/it.po
M src/bin/psql/po/ja.po
M src/bin/psql/po/ko.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/sv.po
M src/bin/psql/po/uk.po
M src/bin/scripts/po/de.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/fr.po
M src/bin/scripts/po/ja.po
M src/bin/scripts/po/ka.po
M src/bin/scripts/po/ko.po
M src/bin/scripts/po/pt_BR.po
M src/bin/scripts/po/ru.po
M src/bin/scripts/po/sv.po
M src/bin/scripts/po/uk.po
M src/bin/scripts/po/zh_CN.po
M src/interfaces/ecpg/ecpglib/po/es.po
M src/interfaces/ecpg/ecpglib/po/fr.po
M src/interfaces/ecpg/ecpglib/po/ru.po
M src/interfaces/ecpg/ecpglib/po/sv.po
M src/interfaces/ecpg/ecpglib/po/uk.po
M src/interfaces/ecpg/preproc/po/de.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/fr.po
M src/interfaces/ecpg/preproc/po/ja.po
M src/interfaces/ecpg/preproc/po/ka.po
M src/interfaces/ecpg/preproc/po/ko.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/ecpg/preproc/po/sv.po
M src/interfaces/ecpg/preproc/po/uk.po
M src/interfaces/libpq/po/de.po
M src/interfaces/libpq/po/ja.po
M src/interfaces/libpq/po/ka.po
M src/pl/plperl/po/es.po
M src/pl/plperl/po/fr.po
M src/pl/plperl/po/ko.po
M src/pl/plperl/po/ru.po
M src/pl/plperl/po/sv.po
M src/pl/plperl/po/uk.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpgsql/src/po/fr.po
M src/pl/plpgsql/src/po/ko.po
M src/pl/plpgsql/src/po/ru.po
M src/pl/plpgsql/src/po/sv.po
M src/pl/plpgsql/src/po/uk.po
M src/pl/plpython/po/es.po
M src/pl/plpython/po/fr.po
M src/pl/plpython/po/ru.po
M src/pl/plpython/po/sv.po
M src/pl/plpython/po/uk.po
M src/pl/plpython/po/zh_CN.po
M src/pl/tcl/po/de.po
M src/pl/tcl/po/es.po
M src/pl/tcl/po/fr.po
M src/pl/tcl/po/ja.po
M src/pl/tcl/po/ka.po
M src/pl/tcl/po/ko.po
M src/pl/tcl/po/ru.po
M src/pl/tcl/po/sv.po
M src/pl/tcl/po/uk.po
M src/pl/tcl/po/zh_CN.po
doc PG 18 relnotes: adjust COPY and REJECT_LIMIT items
commit : b3754dcc9ff7aba2268e98ecf4b5546353305cd2
author : Bruce Momjian <bruce@momjian.us>
date : Sun, 4 May 2025 22:37:20 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sun, 4 May 2025 22:37:20 -0400
Reported-by: Atsushi Torikoshi
Discussion: https://postgr.es/m/CAM6-o=CEF6tKAjtGMEOd45YySwNRXPu8d_zyYq=fhnia9hOU6Q@mail.gmail.com
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: move and clarify constraint items
commit : d83981c24be76b64205158b59b6c05cf118187c3
author : Bruce Momjian <bruce@momjian.us>
date : Sun, 4 May 2025 22:08:20 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sun, 4 May 2025 22:08:20 -0400
Reported-by: Álvaro Herrera
Discussion: https://postgr.es/m/202505041135.cpo7zgdcya2u@alvherre.pgsql
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: add commit for cancel key and protocol neg.
commit : 8c9eec540dc4ba684b74e3ccab8413f358a2f93d
author : Bruce Momjian <bruce@momjian.us>
date : Sun, 4 May 2025 21:44:39 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sun, 4 May 2025 21:44:39 -0400
Reported-by: Jelte Fennema-Nio
Discussion: https://postgr.es/m/CAGECzQQehQrhkNNXvLiBgE3odBbTPG=9PzV8F4Oqq3kOorK0Sw@mail.gmail.com
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: fix libpq wording
commit : a675149e87706d01e4007150a0124b89bdef08be
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 3 May 2025 18:49:40 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 3 May 2025 18:49:40 -0400
Reported-by: Jelte Fennema-Nio
Discussion: https://postgr.es/m/CAGECzQT4804OLOP+nDBxDpMw3Soq=g+fKOE7NryBHggy4GgEcg@mail.gmail.com
M doc/src/sgml/release-18.sgml
Revert "Refactor ChangeVarNodesExtended() using the custom callback"
commit : 2782f3b8454676c9df408aa0bc960efca0f3eb22
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 3 May 2025 22:42:05 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 3 May 2025 22:42:05 +0300
This reverts commit 250a718aadad68793e82103282247556a46a3cfc.
It shouldn't be pushed during the release freeze.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/E1uBIbY-000owH-0O%40gemulon.postgresql.org
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/rewrite/rewriteManip.c
M src/include/rewrite/rewriteManip.h
M src/tools/pgindent/typedefs.list
Refactor ChangeVarNodesExtended() using the custom callback
commit : 250a718aadad68793e82103282247556a46a3cfc
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 3 May 2025 22:30:52 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 3 May 2025 22:30:52 +0300
fc069a3a6319 implemented Self-Join Elimination (SJE) and put related logic
to ChangeVarNodes_walker(). This commit provides refactoring to remove the
SJE-related logic from ChangeVarNodes_walker() but adds a custom callback to
ChangeVarNodesExtended(), which has a chance to process a node before
ChangeVarNodes_walker(). Passing this callback to ChangeVarNodesExtended()
allows SJE-related node handling to be kept within the analyzejoins.c.
Reported-by: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/CAMbWs49PE3CvnV8vrQ0Dr%3DHqgZZmX0tdNbzVNJxqc8yg-8kDQQ%40mail.gmail.com
Author: Andrei Lepikhov <lepihov@gmail.com>
Author: Alexander Korotkov <aekorotkov@gmail.com>
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/rewrite/rewriteManip.c
M src/include/rewrite/rewriteManip.h
M src/tools/pgindent/typedefs.list
doc: update guidelines on non-ASCII characters in docs
commit : fb21ed6c38da719149443e78048790c0347e2c9b
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 3 May 2025 14:45:26 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 3 May 2025 14:45:26 -0400
M doc/src/sgml/README.non-ASCII
doc PG 18 relnotes: add GROUP BY column elimination item
commit : 24987c6f0687576da97b593c6dd5740ccdd23277
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 3 May 2025 12:57:18 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 3 May 2025 12:57:18 -0400
With a nod to PG 9.6.
Reported-by: jian he
Discussion: https://postgr.es/m/CACJufxEqs=EXZETwtaOooTFhZrtxvSWg8M2uPfzjNtS3wQ6Dzw@mail.gmail.com
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: move protocol version item to "server"
commit : 04b269da56db966bd439bf4634a0f33c166eae2e
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 3 May 2025 12:19:31 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 3 May 2025 12:19:31 -0400
Reported-by: Jelte Fennema-Nio
Discussion: https://postgr.es/m/CAGECzQSTBgTsDJPxOHWKo7106-YnnYQGzpzNJdis+xTKGUhu2g@mail.gmail.com
M doc/src/sgml/release-18.sgml
Fix memory allocation/copy mistakes.
commit : 5201bba266d7127397e08a2d954faa5a996c0139
author : Etsuro Fujita <efujita@postgresql.org>
date : Sat, 3 May 2025 20:00:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Sat, 3 May 2025 20:00:00 +0900
The previous code was allocating more memory and copying more data than
necessary because it specified the wrong PgStat_KindInfo member as the
size argument for MemoryContextAlloc and memcpy, respectively.
Although these issues exist since 5891c7a8e, there have been no reports
from the field. So for now, it seems sufficient to fix them in master.
Author: Etsuro Fujita <etsuro.fujita@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Gurjeet Singh <gurjeet@singh.im>
Discussion: https://postgr.es/m/CAPmGK15eTRCZTnfgQ4EuBNo%3DQLYGFEbXS_7m2dXqtkcT7L8qrQ%40mail.gmail.com
M src/backend/utils/activity/pgstat.c
Fix typos in comments.
commit : 6e91b9c16ff53fadf75a545aabe5907f1c046d2f
author : Etsuro Fujita <efujita@postgresql.org>
date : Sat, 3 May 2025 19:10:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Sat, 3 May 2025 19:10:00 +0900
Also adjust the phrasing in the comments.
Author: Etsuro Fujita <etsuro.fujita@gmail.com>
Author: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Gurjeet Singh <gurjeet@singh.im>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAPmGK17%3DPHSDZ%2B0G6jcj12buyyE1bQQc3sbp1Wxri7tODT-SDw%40mail.gmail.com
Backpatch-through: 15
M src/backend/utils/activity/pgstat_database.c
M src/backend/utils/activity/pgstat_function.c
M src/backend/utils/activity/pgstat_relation.c
M src/backend/utils/activity/pgstat_subscription.c
doc PG 18 relnotes: update chapter tags for recent commit
commit : 9fd989ff9937966a8c3f28bfb6cce8f8071b3289
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 May 2025 20:10:10 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 May 2025 20:10:10 -0400
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: adjust libpq trace & potocol version items
commit : 9f8fcadb2087d3178dc01de37e59ba9a0c70dd31
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 May 2025 20:09:12 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 May 2025 20:09:12 -0400
Reported-by: Jelte Fennema-Nio
Discussion: https://postgr.es/m/CAGECzQQj0r_JX38fa-_kepp9UaMzCcujRAYaJG2+fPks1b8MVg@mail.gmail.com
M doc/src/sgml/release-18.sgml
doc PG 18 relnotes: reword and reorder items
commit : aa82ebdc297073ab1bb51cfc0322a6f9e81a13f5
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 May 2025 19:58:54 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 May 2025 19:58:54 -0400
Also move ssl_groups to a more appropriate section.
Reported-by: Jacob Champion (ssl_groups item)
Discussion: https://postgr.es/m/CAOYmi+k_zpGaDOrwV46_j-O-a_hSWxcXM6h8vccq45Y28deP-g@mail.gmail.com
M doc/src/sgml/release-18.sgml
Avoid treating nonrequired nbtree keys as required.
commit : 0f08df406822bfa697dbcabef55728a8cb3e7fdb
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 2 May 2025 17:50:58 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 2 May 2025 17:50:58 -0400
Consistently prevent nbtree array advancement from treating a scankey as
required when operating in pstate.forcenonrequired mode. Otherwise, we
risk a NULL pointer dereference. This was possible in the path where
_bt_check_compare is called to recheck a tuple that advanced all of the
scan's arrays to matching values: its continuescan=false handling
expects _bt_advance_array_keys to have been called with a valid pstate,
but it'll always be NULL during sktrig_required=false calls (which is
how _bt_advance_array_keys must be called when pstate.forcenonrequired).
Oversight in commit 8a510275, which optimized nbtree search scan key
comparisons.
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://postgr.es/m/CAHgHdKsn2W=gPBmj7p6MjQFvxB+zZDBkwTSg0o3f5Hh8rkRrsA@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WzmodSE+gpTd1CRGU9ez8ytyyDS+Kns2r9NzgUp1s56kpw@mail.gmail.com
M src/backend/access/nbtree/nbtutils.c
Fix memory leak in _gin_parallel_merge
commit : 1681a70df3d68b6f9dc82645f97f8d4668edc42f
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 2 May 2025 23:04:39 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 2 May 2025 23:04:39 +0200
To insert the merged GIN entries in _gin_parallel_merge, the leader
calls ginEntryInsert(). This may allocate memory, e.g. for a new leaf
tuple. This was allocated in the PortalContext, and kept until the end
of the index build. For most GIN indexes the amount of leaked memory is
negligible, but for custom opclasses with large keys it may cause OOMs.
Fixed by calling ginEntryInsert() in a temporary memory context, reset
after each insert. Other ginEntryInsert() callers do this too, except
that the context is reset after batches of inserts. More frequent resets
don't seem to hurt performance, it may even help it a bit.
Report and fix by Vinod Sridharan.
Author: Vinod Sridharan <vsridh90@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CAFMdLD4p0VBd8JG=Nbi=BKv6rzFAiGJ_sXSFrw-2tNmNZFO5Kg@mail.gmail.com
M src/backend/access/gin/gininsert.c
Don't use a tuplestore if we don't have to for SQL-language functions.
commit : e83a8ae44729bfdd15d31101bd2c99be98cfc691
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 2 May 2025 16:16:20 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 2 May 2025 16:16:20 -0400
We only need a tuplestore if we're actually going to accumulate
multiple result tuples. Obviously then we don't need one for non-set-
returning functions; but even a SRF doesn't need one if we decide to
use "lazyEval" (one row at a time) mode. In these cases, it's
sufficient to use the junkfilter's result slot to hold the single row
that's due to be returned. We just need to "materialize" that slot
to ensure it holds onto the data past shutdown of the sub-executor.
The original intent of this patch was partially to save a few cycles
(by not putting tuples into a tuplestore only to pull them back out
immediately), but mostly to ensure that we don't use a tuplestore
in non-set-returning functions. That's because I had concerns
about whether a tuplestore is safe to keep across queries,
which was possible for functions invoked via long-lived FmgrInfos
such as those kept in the typcache. There are no cases where SRFs
are called that way, so getting rid of the tuplestore in non-SRFs
should make things safer.
However, it emerges that running fmgr_sql in a short-lived context
(as 595d1efed made it do) makes the existing coding unsafe anyway:
we can end up with a long-lived TupleTableSlot holding a freeable
reference to a short-lived tuple, resulting in a double-free crash.
Not trying to pull tuples out of the tuplestore using that slot
dodges the problem, so I'm going to commit this now rather than
invent a band-aid solution for v18.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/2443532.1744919968@sss.pgh.pa.us
Discussion: https://postgr.es/m/9f975803-1a1c-4f21-b987-f572e110e860@gmail.com
M src/backend/executor/functions.c
Handle self-referencing FKs correctly in partitioned tables
commit : c83a38758d2a9b4a72ec8f7e218f7546457f265d
author : Álvaro Herrera <alvherre@kurilemu.de>
date : Fri, 2 May 2025 21:25:50 +0200
committer: Álvaro Herrera <alvherre@kurilemu.de>
date : Fri, 2 May 2025 21:25:50 +0200
For self-referencing foreign keys in partitioned tables, we weren't
handling creation of pg_constraint rows during CREATE TABLE PARTITION AS
as well as ALTER TABLE ATTACH PARTITION. This is an old bug -- mostly,
we broke this in 614a406b4ff1 while trying to fix it (so 12.13, 13.9,
14.6 and 15.0 and up all behave incorrectly). This commit reverts part
of that with additional fixes for full correctness, and installs more
tests to verify the parts we broke, not just the catalog contents but
also the user-visible behavior.
Backpatch to all live branches. In branches 13 and 14, commit
46a8c27a7226 changed the behavior during DETACH to drop a FK
constraint rather than trying to repair it, because the complete fix of
repairing catalog constraints was problematic due to lack of previous
fixes. For this reason, the test behavior in those branches is a bit
different. However, as best as I can tell, the fix works correctly
there.
In release notes we have to recommend that all self-referencing foreign
keys on partitioned tables be recreated if partitions have been created
or attached after the FK was created, keeping in mind that violating
rows might already be present on the referencing side.
Reported-by: Guillaume Lelarge <guillaume@lelarge.info>
Reported-by: Matthew Gabeler-Lee <fastcat@gmail.com>
Reported-by: Luca Vallisa <luca.vallisa@gmail.com>
Discussion: https://postgr.es/m/CAECtzeWHCA+6tTcm2Oh2+g7fURUJpLZb-=pRXgeWJ-Pi+VU=_w@mail.gmail.com
Discussion: https://postgr.es/m/18156-a44bc7096f0683e6@postgresql.org
Discussion: https://postgr.es/m/CAAT=myvsiF-Attja5DcWoUWh21R12R-sfXECY2-3ynt8kaOqjw@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/expected/triggers.out
M src/test/regress/sql/foreign_key.sql
Doc: correct spelling of meson switch.
commit : ac557793d478c34967a862e7104e524820e24cdd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 2 May 2025 15:12:49 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 2 May 2025 15:12:49 -0400
It's --auto-features not --auto_features.
Reported-by: Egor Chindyaskin <kyzevan23@mail.ru>
Discussion: https://postgr.es/m/172465652540.862882.17808523044292761256@wrigleys.postgresql.org
Discussion: https://postgr.es/m/1979661.1746212726@sss.pgh.pa.us
Backpatch-through: 16
M doc/src/sgml/installation.sgml
oauth: Correct SSL dependency for libpq-oauth.a
commit : 3db68212a39351804b2b4fb70a05cab76be32ce7
author : Jacob Champion <jchampion@postgresql.org>
date : Fri, 2 May 2025 10:45:12 -0700
committer: Jacob Champion <jchampion@postgresql.org>
date : Fri, 2 May 2025 10:45:12 -0700
libpq-oauth.a includes libpq-int.h, which includes OpenSSL headers. The
Autoconf side picks up the necessary include directories via CPPFLAGS,
but Meson needs the dependency to be made explicit.
Reported-by: Nathan Bossart <nathandbossart@gmail.com>
Tested-by: Nathan Bossart <nathandbossart@gmail.com>
Tested-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/aBTgjDfrdOZmaPgv%40nathan
M src/interfaces/libpq-oauth/meson.build
Make "directory" setting work with extension_control_path
commit : 81eaaa2c41dd29b06edf8b234753debbd1a581d5
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 2 May 2025 16:25:40 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 2 May 2025 16:25:40 +0200
The extension_control_path setting (commit 4f7f7b03758) did not
support extensions that set a custom "directory" setting in their
control file. Very few extensions use that and during the discussion
on the previous commit it was suggested to maybe remove that
functionality. But a fix was easier than initially thought, so this
just adds that support. The fix is to use the control->control_dir as
a share dir to return the path of the extension script files.
To make this work more sensibly overall, the directory suffix
"extension" is no longer to be included in the extension_control_path
value. To quote the patch, it would be
-extension_control_path = '/usr/local/share/postgresql/extension:/home/my_project/share/extension:$system'
+extension_control_path = '/usr/local/share/postgresql:/home/my_project/share:$system'
During the initial patch, there was some discussion on which of these
two approaches would be better, and the committed patch was a 50/50
decision. But the support for the "directory" setting pushed it the
other way, and also it seems like many people didn't like the previous
behavior much.
Author: Matheus Alcantara <mths.dev@pm.me>
Reviewed-by: Christoph Berg <myon@debian.org>
Reviewed-by: David E. Wheeler <david@justatheory.com>
Discussion: https://www.postgresql.org/message-id/flat/aAi1VACxhjMhjFnb%40msg.df7cb.de#0cdf7b7d727cc593b029650daa3c4fbc
M doc/src/sgml/config.sgml
M doc/src/sgml/extend.sgml
M src/backend/commands/extension.c
M src/test/modules/test_extensions/t/001_extension_control_path.pl
doc: first draft of the PG 18 release notes
commit : a724c7889f74cc4e76e1979f90808decbc744c79
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 1 May 2025 22:36:58 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 1 May 2025 22:36:58 -0400
M doc/src/sgml/release-18.sgml
Doc: stop implying recommendation of insecure search_path value.
commit : c6a26e4ccd1bf090ab755df2f597103798284d3a
author : Noah Misch <noah@leadboat.com>
date : Thu, 1 May 2025 16:51:59 -0700
committer: Noah Misch <noah@leadboat.com>
date : Thu, 1 May 2025 16:51:59 -0700
SQL "SET search_path = 'pg_catalog, pg_temp'" is silently equivalent to
"SET search_path = pg_temp, pg_catalog, "pg_catalog, pg_temp"" instead
of the intended "SET search_path = pg_catalog, pg_temp". (The intent
was a two-element search path. With the single quotes, it instead
specifies one element with a comma and a space in the middle of the
element.) In addition to the SET statement, this affects SET clauses of
CREATE FUNCTION, ALTER ROLE, and ALTER DATABASE. It does not affect the
set_config() SQL function.
Though the documentation did not show an insecure command, remove single
quotes that could entice a reader to write an insecure command.
Back-patch to v13 (all supported versions).
Reported-by: Sven Klemm <sven@timescale.com>
Author: Sven Klemm <sven@timescale.com>
Backpatch-through: 13
M doc/src/sgml/extend.sgml
doc: Flesh out extension docs for the "prefix" make variable
commit : 0064020680c1293ec6ce9fe2588e33d6aebdb015
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 1 May 2025 22:10:36 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 1 May 2025 22:10:36 +0200
The variable is a bit magical in how it requires "postgresql" or
"pgsql" to be part of the path, and files end up in its "share" and
"lib" subdirectories. So mention all that and show an example of
setting "extension_control_path" and "dynamic_library_path" to use
those locations.
Author: David E. Wheeler <david@justatheory.com>
Reviewed-by: Matheus Alcantara <matheusssilv97@gmail.com>
Reviewed-by: Christoph Berg <myon@debian.org>
Discussion: https://www.postgresql.org/message-id/6B5BF07B-8A21-48E3-858C-1DC22F3A28B4@justatheory.com
M doc/src/sgml/extend.sgml
oauth: Fix Autoconf build on macOS
commit : 4ea1254f35b27ed35a389e720ab2e614fc1e1d4d
author : Jacob Champion <jchampion@postgresql.org>
date : Thu, 1 May 2025 12:35:52 -0700
committer: Jacob Champion <jchampion@postgresql.org>
date : Thu, 1 May 2025 12:35:52 -0700
Oversight in b0635bfda. -lintl is necessary for gettext on Mac, which
libpq-oauth depends on via pgport/pgcommon. (I'd incorrectly removed
this change from an earlier version of the patch, where it was suggested
by Peter Eisentraut.)
Per buildfarm member indri.
M src/interfaces/libpq-oauth/Makefile
oauth: Move the builtin flow into a separate module
commit : b0635bfda0535a7fc36cd11d10eecec4e2a96330
author : Jacob Champion <jchampion@postgresql.org>
date : Thu, 1 May 2025 09:14:30 -0700
committer: Jacob Champion <jchampion@postgresql.org>
date : Thu, 1 May 2025 09:14:30 -0700
The additional packaging footprint of the OAuth Curl dependency, as well
as the existence of libcurl in the address space even if OAuth isn't
ever used by a client, has raised some concerns. Split off this
dependency into a separate loadable module called libpq-oauth.
When configured using --with-libcurl, libpq.so searches for this new
module via dlopen(). End users may choose not to install the libpq-oauth
module, in which case the default flow is disabled.
For static applications using libpq.a, the libpq-oauth staticlib is a
mandatory link-time dependency for --with-libcurl builds. libpq.pc has
been updated accordingly.
The default flow relies on some libpq internals. Some of these can be
safely duplicated (such as the SIGPIPE handlers), but others need to be
shared between libpq and libpq-oauth for thread-safety. To avoid
exporting these internals to all libpq clients forever, these
dependencies are instead injected from the libpq side via an
initialization function. This also lets libpq communicate the offsets of
PGconn struct members to libpq-oauth, so that we can function without
crashing if the module on the search path came from a different build of
Postgres. (A minor-version upgrade could swap the libpq-oauth module out
from under a long-running libpq client before it does its first load of
the OAuth flow.)
This ABI is considered "private". The module has no SONAME or version
symlinks, and it's named libpq-oauth-<major>.so to avoid mixing and
matching across Postgres versions. (Future improvements may promote this
"OAuth flow plugin" to a first-class concept, at which point we would
need a public API to replace this anyway.)
Additionally, NLS support for error messages in b3f0be788a was
incomplete, because the new error macros weren't being scanned by
xgettext. Fix that now.
Per request from Tom Lane and Bruce Momjian. Based on an initial patch
by Daniel Gustafsson, who also contributed docs changes. The "bare"
dlopen() concept came from Thomas Munro. Many people reviewed the design
and implementation; thank you!
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Christoph Berg <myon@debian.org>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Wolfgang Walther <walther@technowledgy.de>
Discussion: https://postgr.es/m/641687.1742360249%40sss.pgh.pa.us
M config/programs.m4
M configure
M configure.ac
M doc/src/sgml/installation.sgml
M doc/src/sgml/libpq.sgml
M meson.build
M src/Makefile.global.in
M src/interfaces/Makefile
A src/interfaces/libpq-oauth/Makefile
A src/interfaces/libpq-oauth/README
A src/interfaces/libpq-oauth/exports.txt
A src/interfaces/libpq-oauth/meson.build
R094 src/interfaces/libpq/fe-auth-oauth-curl.c src/interfaces/libpq-oauth/oauth-curl.c
A src/interfaces/libpq-oauth/oauth-curl.h
A src/interfaces/libpq-oauth/oauth-utils.c
A src/interfaces/libpq-oauth/oauth-utils.h
M src/interfaces/libpq/Makefile
M src/interfaces/libpq/exports.txt
M src/interfaces/libpq/fe-auth-oauth.c
M src/interfaces/libpq/fe-auth-oauth.h
M src/interfaces/libpq/meson.build
M src/interfaces/libpq/nls.mk
M src/makefiles/meson.build
M src/test/modules/oauth_validator/meson.build
M src/test/modules/oauth_validator/t/002_client.pl
Remove extra "not" in pg_upgrade documentation.
commit : a3ef0b570c56f7bb15e4aa5caf0125fff92a557a
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 1 May 2025 09:31:36 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 1 May 2025 09:31:36 -0500
Oversight in commit cb45dc3afb.
Reported-by: Erik Rijkers <er@xs4all.nl>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Discussion: https://postgr.es/m/7b856277-62ad-80f0-36e1-a134ec3c9cab%40xs4all.nl
M doc/src/sgml/ref/pgupgrade.sgml
doc: Warn that ts_headline() output is not HTML-safe.
commit : d73d4cfdfce897bd42ef6466e6cece5d35a91779
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 1 May 2025 11:03:43 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 1 May 2025 11:03:43 +0100
Add a documentation warning to ts_headline() pointing out that, when
working with untrusted input documents, the output is not guaranteed
to be safe for direct inclusion in web pages. This is because, while
it does remove some XML tags from the input, it doesn't remove all
HTML markup, and so the result may be unsafe (e.g., it might permit
XSS attacks).
To guard against that, all HTML markup should be removed from the
input, making it plain text, or the output should be passed through an
HTML sanitizer.
In addition, document precisely what the default text search parser
recognises as valid XML tags, since that's what determines which XML
tags ts_headline() will remove.
Reported-by: Richard Neill <richard.neill@telos.digital>
Author: Dean Rasheed <dean.a.rasheed@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Backpatch-through: 13
M doc/src/sgml/textsearch.sgml
doc: Improve explanations when a table rewrite is needed
commit : 06c4f3ae804e6680ff03a09c1f89dfb1ca3d90e9
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 1 May 2025 08:56:43 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 1 May 2025 08:56:43 +0200
Further improvement for commit 11bd8318602. That commit confused
identity and generated columns; fix that. Also, virtual generated
columns have since been added; add more details about that. Also some
small rewordings and reformattings to further improve clarity.
Reviewed-by: Robert Treat <rob@xzilla.net>
Discussion: https://postgr.es/m/00e6eb5f5c793b8ef722252c7a519c9a@oss.nttdata.com
M doc/src/sgml/ref/alter_table.sgml
Adjust overstrong nbtree skip array assertion.
commit : 9d924dbb37103b647c72a5252ad20770b8bae3a1
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Apr 2025 23:15:51 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Apr 2025 23:15:51 -0400
Make an nbtree array preprocessing assertion account for scans that add
fewer skip arrays than initially expected due to preprocessing finding
an unsatisfiable array qual.
Oversight in commit 92fe23d9.
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://postgr.es/m/CAHgHdKtQMhHy5qcB3KqCcGiW-Rp8P7KzUFRa9ZMKUiv6zen7LQ@mail.gmail.com
M src/backend/access/nbtree/nbtpreprocesskeys.c
doc: Mention cost-based delays for total_[auto]{vacuum,analyze}_time
commit : 92ee8a4df5b529efbc2455d7a171054c88b43a33
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 1 May 2025 08:52:19 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 1 May 2025 08:52:19 +0900
30a6ed0ce4b has added four attributes to pg_stat_all_tables to track the
cumulative time spent in [auto]vacuum and [auto]analyze. It was not
mentioned that the vacuum cost-based delays are included in these
numbers, which could be confusing now that the delays are included in
the vacuum progress view (bb8dff9995f2).
This commit adds an extra note about this matter.
Reported-by: Magnus Hagander <magnus@hagander.net>
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/CABUevEz9v1ZNToPyD98JnWDGZgG=SmPZKkSNzU9hXQ-nGTQF0g@mail.gmail.com
M doc/src/sgml/monitoring.sgml
Convert strncpy to strlcpy
commit : 45e7e8ca9e9382e8f5e51fe0c085ea7f5d408ac8
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 30 Apr 2025 23:00:47 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 30 Apr 2025 23:00:47 +0200
We try to avoid using strncpy() due to the ease of which it can
be misused. Convert this callsite to use strlcpy() instead to
match similar codepaths in this file.
Suggested-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/2a796830-de2d-4030-b480-d673f6cc5d94@eisentraut.org
M src/backend/utils/mmgr/mcxt.c
doc: Add missing reference to track_cost_delay_timing.
commit : 2d6745a66b12e980fb12bd06d583a66231b5ed5c
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 30 Apr 2025 14:45:54 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 30 Apr 2025 14:45:54 -0500
Oversight in commit bb8dff9995.
M doc/src/sgml/monitoring.sgml
vacuumdb: Don't skip empty relations in --missing-stats-only mode.
commit : 987910502420b77eaf7c3e807f6b0fc1cffced29
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 30 Apr 2025 14:12:59 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 30 Apr 2025 14:12:59 -0500
Presently, --missing-stats-only skips relations with reltuples set
to 0 because empty relations don't get optimizer statistics.
However, before v14, a reltuples value of 0 was ambiguous: it could
either mean the relation is empty, or it could mean that it hadn't
yet been vacuumed or analyzed. (Commit 3d351d916b taught v14 and
newer to use -1 for the latter case.) This ambiguity can cause
--missing-stats-only to inadvertently skip relations that need
optimizer statistics after upgrades to v18 and newer (since
reltuples is now transferred from the old cluster).
To fix, simply remove the check for reltuples != 0. This will
cause --missing-stats-only to analyze some empty tables, but that
doesn't seem too terrible a trade-off.
Reported-by: Christoph Berg <myon@debian.org>
Reviewed-by: Christoph Berg <myon@debian.org>
Discussion: https://postgr.es/m/aAjyvW5_fRGNr7yF%40msg.df7cb.de
M src/bin/scripts/vacuumdb.c
Further adjust guidance for running vacuumdb after pg_upgrade.
commit : d5f1b6a75ba3f55a78959c957c3fd25d9569057c
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 30 Apr 2025 14:12:59 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 30 Apr 2025 14:12:59 -0500
Since pg_upgrade does not transfer the cumulative statistics used
to trigger autovacuum and autoanalyze, the server may take much
longer than expected to process them post-upgrade. Currently, we
recommend analyzing only relations for which optimizer statistics
were not transferred by using the --analyze-in-stages and
--missing-stats-only options. This commit appends another
recommendation to analyze all relations to update the relevant
cumulative statistics by using the --analyze-only option. This is
similar to the recommendation for pg_stat_reset().
Reported-by: Christoph Berg <myon@debian.org>
Reviewed-by: Christoph Berg <myon@debian.org>
Discussion: https://postgr.es/m/aAfxfKC82B9NvJDj%40msg.df7cb.de
M doc/src/sgml/ref/pgupgrade.sgml
M src/bin/pg_upgrade/check.c
doc: Alphabetize long options for pg_dump[all].
commit : f60420cff66a9089a9b431f9c07f4a29aae4990a
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 30 Apr 2025 13:07:51 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 30 Apr 2025 13:07:51 -0500
The current ordering strategy for these pages is to list the short
options in alphabetical order followed by the long options in
alphabetical order. If an option has both a short variant and a
long variant, the short variant takes precedence. This commit
moves a few recently added options to match this style. We should
probably adjust all pages and --help output to list the long and
short options in one combined alphabetical list (with the long
variants taking precedence), but that is a much larger change, so
it is left as a future exercise.
Oversights in commits a5cf808be5, 1fd1bd8710, and bde2fb797a.
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/aBFBtsltgu3-IU1d%40nathan
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
Update time zone data files to tzdata release 2025b.
commit : 368c3fbf9da96787d4e7cae61e11518d72f75071
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Apr 2025 11:13:49 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Apr 2025 11:13:49 -0400
DST law changes in Chile: there is a new time zone America/Coyhaique
for Chile's Aysén Region, to account for it changing to UTC-03
year-round and thus diverging from America/Santiago.
Historical corrections for Iran.
Backpatch-through: 13
M src/timezone/data/tzdata.zi
Typo and doc fixups for memory context reporting
commit : f8c115a6cb6531ca86e7b0febd02698135827337
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 30 Apr 2025 11:10:27 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 30 Apr 2025 11:10:27 +0200
This fixes comment and docs typos as well as a small documentation
change to make it clearer. Found via post-commit review.
Author: Rahila Syed <rahilasyed90@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAH2L28vt16C9xTuK+K7QZvtA3kCNWXOEiT=gEekUw3Xxp9LVQw@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/mmgr/mcxt.c
Add missing string terminator
commit : d2a1ed1727a8ef45eab1a8ddb3d375c1ce839aac
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 30 Apr 2025 10:34:08 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 30 Apr 2025 10:34:08 +0200
When copying the string strncpy won't add nul termination since
the string length is equal to the length specified. Explicitly
set a nul terminator after copying to properly terminate. Found
via post-commit review.
Author: Rahila Syed <rahilasyed90@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAH2L28vt16C9xTuK+K7QZvtA3kCNWXOEiT=gEekUw3Xxp9LVQw@mail.gmail.com
M src/backend/utils/mmgr/mcxt.c
Add 918e7287e to .git-blame-ignore-revs
commit : 991407ae86785ef38bb1ebfea679a70bcd76574d
author : David Rowley <drowley@postgresql.org>
date : Wed, 30 Apr 2025 19:27:56 +1200
committer: David Rowley <drowley@postgresql.org>
date : Wed, 30 Apr 2025 19:27:56 +1200
M .git-blame-ignore-revs
Fix broken indentation
commit : 918e7287ed20eb1fe280ab6c4056ccf94dcd53a8
author : David Rowley <drowley@postgresql.org>
date : Wed, 30 Apr 2025 19:18:30 +1200
committer: David Rowley <drowley@postgresql.org>
date : Wed, 30 Apr 2025 19:18:30 +1200
I forgot to run pgindent in d8555e522.
Reported-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Discussion: https://postgr.es/m/156083c9-eac0-418d-9667-92dec4d6d6cd@oss.nttdata.com
M src/backend/utils/adt/jsonfuncs.c
Fix a couple of comment typos
commit : d8555e522eca00f1b4b6cd0b75269b7176fb7931
author : David Rowley <drowley@postgresql.org>
date : Wed, 30 Apr 2025 13:40:46 +1200
committer: David Rowley <drowley@postgresql.org>
date : Wed, 30 Apr 2025 13:40:46 +1200
Author: Junwang Zhao <zhjwpku@gmail.com>
Discussion: https://postgr.es/m/CAEG8a3+MRwDKc4YSFKKPKq7Y+vMufVC5u94wM5KZPB2CbgCxnQ@mail.gmail.com
M src/backend/access/transam/multixact.c
M src/backend/utils/adt/jsonfuncs.c
Give up on running with NetBSD/OpenBSD's default semaphore settings.
commit : 810a8b1c8051d4e8822967a96f133692698386de
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Apr 2025 17:27:52 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Apr 2025 17:27:52 -0400
This reverts commit 38da053463bef32adf563ddee5277d16d2b6c5af, which
attempted to preserve our ability to start with only 60 semaphores.
Subsequent changes (particularly 55b454d0e) have put that idea pretty
much permanently out of reach: people wishing to use Postgres v18 on
OpenBSD or NetBSD will have no choice but to increase those platforms'
default values of SEMMNI and SEMMNS.
Hence, revert 38da05346's changes in SEMAS_PER_SET and the minimum
tested value of max_connections. Adjust a comment from the subsequent
patch 6d0154196, and tweak the wording in runtime.sgml to make it
clear that changing SEMMNI/SEMMNS is no longer even a little bit
optional on these platforms.
Although 38da05346 was later back-patched into v17, leave that branch
alone: it's still capable of starting with 60 semaphores, and there's
no reason to break that.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/E1tuZNv-0037Gs-34@gemulon.postgresql.org
Discussion: https://postgr.es/m/1052019.1745947915@sss.pgh.pa.us
M doc/src/sgml/runtime.sgml
M src/backend/port/sysv_sema.c
M src/bin/initdb/initdb.c
oauth: Classify oauth_client_secret as a password
commit : e974f1c2164bc677d55f98edaf99f80c0b6b89d9
author : Jacob Champion <jchampion@postgresql.org>
date : Tue, 29 Apr 2025 13:08:55 -0700
committer: Jacob Champion <jchampion@postgresql.org>
date : Tue, 29 Apr 2025 13:08:55 -0700
Tell UIs to hide the value of oauth_client_secret, like the other
passwords. Due to the previous commit, this does not affect postgres_fdw
and dblink, but add a comment to try to warn others of the hazard in the
future.
Reported-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250415191435.55.nmisch%40google.com
M src/interfaces/libpq/fe-connect.c
oauth: Disallow OAuth connections via postgres_fdw/dblink
commit : d2e7d2a09d7d17656418f8c859074db690af19ec
author : Jacob Champion <jchampion@postgresql.org>
date : Tue, 29 Apr 2025 13:08:24 -0700
committer: Jacob Champion <jchampion@postgresql.org>
date : Tue, 29 Apr 2025 13:08:24 -0700
A subsequent commit will reclassify oauth_client_secret from dispchar=""
to dispchar="*", so that UIs will treat it like a secret. For our FDWs,
this change will move that option from SERVER to USER MAPPING, which we
need to avoid.
But upon further discussion, we don't really want our FDWs to use our
builtin Device Authorization flow at all, for several reasons:
- the URL and code would be printed to the server logs, not sent over
the client connection
- tokens are not cached/refreshed, so every single connection has to be
manually authorized by a user with a browser
- oauth_client_secret needs to belong to the foreign server, but options
on SERVER are publicly accessible
- all non-superusers would need password_required=false, which is
dangerous
Future OAuth work can use FDWs as a motivating use case. But for now,
disallow all oauth_* connection options for these two extensions.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250415191435.55.nmisch%40google.com
M contrib/dblink/dblink.c
M contrib/dblink/expected/dblink.out
M contrib/dblink/sql/dblink.sql
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
Bump the minimum supported Python version to 3.6.8
commit : 45363fca6372a04e90d1c2628c3cbfe8894b811b
author : Jacob Champion <jchampion@postgresql.org>
date : Tue, 29 Apr 2025 13:04:19 -0700
committer: Jacob Champion <jchampion@postgresql.org>
date : Tue, 29 Apr 2025 13:04:19 -0700
Python 3.2 is no longer tested by the buildfarm, and there are only a
handful of buildfarm animals running versions older than 3.6, which
itself went end-of-life in 2021. Python 3.6.8 is the default version
shipped in RHEL8, so that seems like a reasonable baseline for PG18.
Now that we use the Python Limited API as of 0793ab810, older versions
of Python should continue functioning for users of PL/Python in
particular, so soften the language from "required" to "supported".
Wording by Tom Lane. Separate from the review of the patch itself,
several people provided input on the choice of cutoff: Christoph Berg,
Devrim Gündüz, Florents Tselai, Jelte Fennema-Nio, and Renan Alves
Fonseca. Thank you!
Suggested-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/16098.1745079444%40sss.pgh.pa.us
M doc/src/sgml/installation.sgml
Fix whitespace typo in string
commit : eec34099c3bfe0ed4b33e77f38cab3b221cd3b37
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Apr 2025 19:08:40 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Apr 2025 19:08:40 +0200
M src/bin/pg_resetwal/pg_resetwal.c
initdb: Do not report default autovacuum_worker_slots.
commit : 2b49492eda3f2a30ee41038b3132338d2587f433
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 29 Apr 2025 11:41:42 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 29 Apr 2025 11:41:42 -0500
Commit 6d01541960 taught initdb to lower the default value of
autovacuum_worker_slots for systems with very few semaphores. It
also added a "fake" report for the chosen value, i.e., initdb
prints a message about selecting the default, but the value was
already selected in a previous test. Per discussion, this is not a
precedent we want to set, and it seems unnecessary to report
everything derived from max_connections, so let's remove the "fake"
report.
Reported-by: Peter Eisentraut <peter@eisentraut.org>
Suggested-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/de722583-4ba4-4063-bc41-e20684978116%40eisentraut.org
M src/bin/initdb/initdb.c
doc: adjust max_files_per_process again
commit : faced8e6a40b469bf9603db868beb6604c95df32
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 29 Apr 2025 10:30:08 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 29 Apr 2025 10:30:08 -0400
Reported-by: Andres Freund
Discussion: https://postgr.es/m/5yqochswkulckuzzrwgv2nqdrfh4k4coc4uwq4lvgzkfwnbjbd@46igbiwjabn2
M doc/src/sgml/config.sgml
doc: clarify new behavior of max_files_per_process
commit : 9a9e60fed39268bb85b59dde3cdb0b0166c85cc5
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 29 Apr 2025 09:45:19 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 29 Apr 2025 09:45:19 -0400
M doc/src/sgml/config.sgml
doc: Small example improvement
commit : 913c60b067aa1a618420fd0372c4df3cf5501863
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Apr 2025 14:40:49 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Apr 2025 14:40:49 +0200
Add a comment character before a line annotation, so that the query
can be used as presented.
Reported-by: Yaroslav Saburov <y.saburov@gmail.com>
Author: Euler Taveira <euler@eulerto.com>
Reviewed-by: Robert Treat <rob@xzilla.net>
Discussion: https://www.postgresql.org/message-id/flat/174393459040.678.17810152410419444783%40wrigleys.postgresql.org
M doc/src/sgml/query.sgml
Fixes for ChangeVarNodes_walker()
commit : 2260c7f6d90ecf76d3806d32890a0890688b41e8
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Apr 2025 14:34:44 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Apr 2025 14:34:44 +0300
This commit fixes two bug in ChangeVarNodes_walker() function.
* When considering RestrictInfo, walk down to its clauses based on the
presense of relid to be deleted not just in clause_relids but also in
required_relids.
* Incrementally adjust num_base_rels based on the change of clause_relids
instead of recalculating it using clause_relids, which could contain
outer-join relids.
Reported-by: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/CAMbWs49PE3CvnV8vrQ0Dr%3DHqgZZmX0tdNbzVNJxqc8yg-8kDQQ%40mail.gmail.com
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
M src/backend/rewrite/rewriteManip.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
pg_restore: Improve --help synopsis
commit : 15b1b4dd3fccfd2576f44841863153c312de011b
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Apr 2025 11:32:49 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Apr 2025 11:32:49 +0200
The --help synopsis should only be one line. This rephrases the first
line a bit to reflect the new functionality of restoring multiple
databases from pg_dumpall output. Additional explanations are better
kept in the man page.
M src/bin/pg_dump/pg_restore.c
pg_restore: Put new option in consistent order in --help output
commit : dadc58f50ac9697907e04422615d79762533496c
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Apr 2025 10:59:05 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Apr 2025 10:59:05 +0200
Also make the description a bit more consistent with similar options.
M src/bin/pg_dump/pg_restore.c
Fix assertion failure during decoding from synced slots.
commit : 3ff2a1f0c9e8d016f3288f4da3962eb1f1592cd8
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 29 Apr 2025 12:52:05 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 29 Apr 2025 12:52:05 +0530
The slot synchronization skips updating the confirmed_flush LSN of the
local slot if the local slot has a newer catalog_xmin or restart_lsn, but
still allows updating the two_phase and two_phase_at fields of the slot.
This opens up a window for the prepared transactions between old
confirmed_flush LSN and two_phase_at to unexpectedly get decoded and sent
to the downstream after promotion. Then, while decoding the commit
prepared the assert will fail, which expects that the prepare hasn't been
sent to the downstream.
The fix is to skip updating the other slot fields when we are skipping to
update the confirmed_flush LSN of the slot.
We didn't backpatch this commit as two_phase_at was not synced in back
branches, which means prepared transactions won't be unexpectedly sent to
downstream.
We discovered this problem while analyzing BF failure reported in the
discussion link.
Reliably reproducing this issue without a debugger is difficult. Given
its rarity, adding specific injection point to test it doesn't seem
worthwhile, so we won't be adding a dedicated test case.
Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/OS0PR01MB5716B44052000EB91EFAE60E94BC2@OS0PR01MB5716.jpnprd01.prod.outlook.com
M src/backend/replication/logical/slotsync.c
pg_verifybackup: Message style improvements
commit : ef1811ac9a7b9c5accde1652c22fb041cb0f34b3
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Apr 2025 09:19:15 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Apr 2025 09:19:15 +0200
M src/bin/pg_verifybackup/astreamer_verify.c
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/bin/pg_verifybackup/t/003_corruption.pl
M src/bin/pg_verifybackup/t/004_options.pl
test_slru: Fix incorrect format placeholders
commit : c893245ec388499426ac609812ab76a416343c7a
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Apr 2025 09:07:53 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Apr 2025 09:07:53 +0200
Before commit a0ed19e0a9e there was a cast around these, but the cast
inadvertently changed the signedness, but that made the format
placeholder correct. Commit a0ed19e0a9e removed the casts, so now the
format placeholders had the wrong signedness.
M src/test/modules/test_slru/test_slru.c
Doc: Specify the interaction of publish_generated_columns with column list.
commit : 9807617a9293f3fce867fbde99e0840cb5695a12
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 29 Apr 2025 09:01:43 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 29 Apr 2025 09:01:43 +0530
Author: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CAHut+PtnjLiNFFh-3f9cXH0wnwqjdkTjQNbVmZdZ1y+zKt_PPg@mail.gmail.com
M doc/src/sgml/ref/create_publication.sgml
Add maintenance_io_concurrency flag to some read stream users
commit : f132815fd717f26613dcc3888608044eac91a7ac
author : Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 28 Apr 2025 14:19:45 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 28 Apr 2025 14:19:45 -0400
Index vacuuming and [auto]prewarm AIO concurrency should be governed by
maintenance_io_concurrency. As such, pass those read stream users the
READ_STREAM_MAINTENANCE flag which will calculate their read stream
distance with maintenance_io_concurrency instead of
effective_io_concurrency. This was an oversight in the original commits
making those operations use the read stream API.
Discussion: https://postgr.es/m/flat/CAAKRu_aopDxTo4b41Mt_7Zc-z0_ngocrY8SFCCY6Aph1HgwuNw%40mail.gmail.com
M contrib/pg_prewarm/autoprewarm.c
M contrib/pg_prewarm/pg_prewarm.c
M src/backend/access/gist/gistvacuum.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/spgist/spgvacuum.c
Fix obsolete nbtree array advancement comment.
commit : ce72e7e02e1c9075f8dd492f55281398d0721eb8
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 28 Apr 2025 12:49:17 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 28 Apr 2025 12:49:17 -0400
Checking if another primitive scan is required after all once the next
leaf page was moved from _bt_checkkeys to its _bt_readpage caller by
commit 9a2e2a28. Update a comment that incorrectly described the
recheck mechanism as something that takes place in _bt_checkkeys.
Also fix an older typo in related code comments.
M src/backend/access/nbtree/nbtutils.c
Make NULL tuple values always advance skip arrays.
commit : b75fedcab7916e1c955dbab565b7ad0cee6b37c6
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 28 Apr 2025 12:11:08 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 28 Apr 2025 12:11:08 -0400
_bt_check_compare neglected to handle a case that can arise when the
scan's keys are temporarily treated as nonrequired, as an optimization:
whenever a NULL tuple value was encountered that had a skip array whose
current element wasn't already NULL, _bt_check_compare failed to advance
the array to the NULL element. This allowed _bt_check_compare to fail
to return matching tuples containing a NULL value (though only with an
array column that came before a skip array column with NULLs, and only
during _bt_readpage calls that set pstate.forcenonrequired=true on a
page where the higher-order column also had to advance).
To fix, teach _bt_check_compare to handle this case just like any other
case where a skip array key is unsatisfied and must be advanced directly
(due to the key being considered a nonrequired key).
Oversight in commit 8a510275, which optimized nbtree search scan key
comparisons with skip arrays.
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://postgr.es/m/CAHgHdKtLFWZcjr87hMH0hYDHgcifu4Tj7iHz-xh8qsJREt5cqA@mail.gmail.com
M src/backend/access/nbtree/nbtutils.c
Fix pg_dump for inherited validated not-null constraints
commit : 0e13b13d26e870cb18fe6ecf9f8697ddfcf2c740
author : Álvaro Herrera <alvherre@kurilemu.de>
date : Mon, 28 Apr 2025 16:25:06 +0200
committer: Álvaro Herrera <alvherre@kurilemu.de>
date : Mon, 28 Apr 2025 16:25:06 +0200
When a child constraint is validated and the parent constraint it
derives from isn't, pg_dump must be coerced into printing the child
constraint; failing to do would result in a dump that restores the
constraint as not valid, which would be incorrect.
Co-authored-by: jian he <jian.universality@gmail.com>
Co-authored-by: Álvaro Herrera <alvherre@kurilemu.de>
Reported-by: jian he <jian.universality@gmail.com>
Message-id: https://postgr.es/m/CACJufxGHNNMc0E2JphUqJMzD3=bwRSuAEVBF5ekgkG8uY0Q3hg@mail.gmail.com
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/test/regress/expected/constraints.out
M src/test/regress/sql/constraints.sql
pg_combinebackup: Message style improvements
commit : c061000311029d2ef0129ad5eee32e698221d43d
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 28 Apr 2025 14:26:49 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 28 Apr 2025 14:26:49 +0200
M src/bin/pg_combinebackup/copy_file.c
M src/bin/pg_combinebackup/pg_combinebackup.c
Restore comments in ChangeVarNodesExtended()
commit : 73e736137616b86d8bc21ec3cba098e187d883d5
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 28 Apr 2025 11:20:22 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 28 Apr 2025 11:20:22 +0300
This commit restores comments in ChangeVarNodesExtended(), which were
accidentally removed by fc069a3a6319.
Reported-by: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/CAMbWs49PE3CvnV8vrQ0Dr%3DHqgZZmX0tdNbzVNJxqc8yg-8kDQQ%40mail.gmail.com
M src/backend/rewrite/rewriteManip.c
Fix xmin advancement during fast_forward decoding.
commit : aaf9e95e8764093ab55baf27c35705f406f592f9
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 28 Apr 2025 11:35:54 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 28 Apr 2025 11:35:54 +0530
During logical decoding, we advance catalog_xmin of logical too early in
fast_forward mode, resulting in required catalog data being removed by
vacuum. This mode is normally used to advance the slot without processing
the changes, but we still can't let the slot's xmin to advance to an
incorrect value.
Commit f49a80c481 fixed a similar issue where the logical slot's
catalog_xmin was getting advanced prematurely during non-fast-forward
mode. During xl_running_xacts processing, instead of directly advancing
the slot's xmin to the oldest running xid in the record, it allowed the
xmin to be held back for snapshots that can be used for
not-yet-replayed transactions, as those might consider older txns as
running too. However, it missed the fact that the same problem can happen
during fast_forward mode decoding, as we won't build a base snapshot in
that mode, and the future call to get_changes from the same slot can miss
seeing the required catalog changes leading to incorrect reslts.
This commit allows building the base snapshot even in fast_forward mode to
prevent the early advancement of xmin.
Reported-by: Amit Kapila <amit.kapila16@gmail.com>
Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Backpatch-through: 13
Discussion: https://postgr.es/m/CAA4eK1LqWncUOqKijiafe+Ypt1gQAQRjctKLMY953J79xDBgAg@mail.gmail.com
Discussion: https://postgr.es/m/OS0PR01MB57163087F86621D44D9A72BF94BB2@OS0PR01MB5716.jpnprd01.prod.outlook.com
M contrib/test_decoding/expected/oldest_xmin.out
M contrib/test_decoding/specs/oldest_xmin.spec
M src/backend/replication/logical/decode.c
Remove circular #include's between wait_event.h and wait_event_types.h
commit : b225c5e76ed1053e505e392423b0dab065a3b813
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 28 Apr 2025 09:08:15 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 28 Apr 2025 09:08:15 +0900
wait_event_types.h is generated by the code, and included wait_event.h.
wait_event.h did the opposite move, including wait_event_types.h,
causing a circular dependency between both.
wait_event_types.h only needs to now about the wait event classes, so
this information is moved into its own file, and wait_event_types.h uses
this new header so as it does not depend anymore on wait_event.h.
Note that such errors can be found with clang-tidy, with commands like
this one:
clang-tidy source_file.c --checks=misc-header-include-cycle -- \
-I/install/path/include/ -I/install/path/include/server/
Issue introduced by fa88928470b5.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/350192.1745768770@sss.pgh.pa.us
M src/backend/utils/activity/generate-wait_event_types.pl
A src/include/utils/wait_classes.h
M src/include/utils/wait_event.h
Disallow removing placeholders during Self-Join Elimination.
commit : 1aa7cf9eb85972aaf2969306e84f5fc794fbef7f
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 28 Apr 2025 01:40:42 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 28 Apr 2025 01:40:42 +0300
fc069a3a6319 implements Self-Join Elimination (SJE), which can remove base
relations when appropriate. However, regressions tests for SJE only cover
the case when placeholder variables (PHVs) are evaluated and needed only
in a single base rel. If this baserel is removed due to SJE, its clauses,
including PHVs, will be transferred to the keeping relation. Removing these
PHVs may trigger an error on plan creation -- thanks to the b3ff6c742f6c for
detecting that.
This commit skips removal of PHVs during SJE. This might also happen that
we skip the removal of some PHVs that could be removed. However, the overhead
of extra PHVs is small compared to the complexity of analysis needed to remove
them.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Alena Rybakina <a.rybakina@postgrespro.ru>
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Reviewed-by: Richard Guo <guofenglinux@gmail.com>
M src/backend/optimizer/plan/analyzejoins.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Remove inappropriate inclusions of c.h and postgres_fe.h.
commit : 2f5b05620330baed1648a1a6e2f231c7ba1368d1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 27 Apr 2025 16:58:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 27 Apr 2025 16:58:57 -0400
Per our usual policy, Postgres header files should not include these;
the decision as to which one to use is to be made in the calling .c
file instead.
These errors aren't particularly new, but I'm not feeling a need
to back-patch these changes; it's mostly just neatnik-ism.
M src/bin/pg_combinebackup/copy_file.h
M src/include/fe_utils/option_utils.h
M src/include/fe_utils/query_utils.h
Don't use double-quotes in #include's of system headers, redux.
commit : 94b84a60729e15ee071739c3aec7fdd781123aab
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 27 Apr 2025 13:23:19 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 27 Apr 2025 13:23:19 -0400
This cleans up some loose ends left by commit e8ca9ed1d. I hadn't
looked closely enough at these places before, but now I have.
The use of double-quoted #includes for Perl headers in plperl_system.h
seems to be simply a mistake introduced in 6c944bf3c and faithfully
copied forward since then. (I had thought possibly it was required
by some weird Windows build setup, but there's no evidence of that in
our history.)
The occurrences in SectionMemoryManager.h and SectionMemoryManager.cpp
evidently stem from those files' origin as LLVM code. It's
understandable that LLVM would treat their own files as needing
double-quoted #includes; but they're still system headers to us.
I also applied the same check to *.c files, and found a few other
random incorrect usages in both directions.
Our ECPG headers and test files routinely use angle brackets to refer
to ECPG headers. I left those usages alone, since it seems reasonable
for an ECPG user to regard those headers as system headers.
M src/backend/jit/llvm/SectionMemoryManager.cpp
M src/backend/storage/buffer/bufmgr.c
M src/bin/initdb/initdb.c
M src/bin/pg_dump/compress_gzip.c
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/include/jit/SectionMemoryManager.h
M src/pl/plperl/plperl_system.h
Remove circular #include's between plpython.h and plpy_util.h.
commit : 2311f193ea7db23f68f5471ae2c35940fc0d4794
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 27 Apr 2025 11:43:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 27 Apr 2025 11:43:02 -0400
plpython.h included plpy_util.h, simply on the grounds that "it's
easier to just include it everywhere". However, plpy_util.h must
include plpython.h, or it won't pass headerscheck. While the
resulting circularity doesn't have any immediate bad effect,
it's poor design. We have seen serious messes arise in the past
from overly-broad inclusion footprints created by such circularities,
so let's establish a project policy against it.
To fix, just replace *.c files' inclusions of plpython.h with
plpy_util.h. They'll pull in plpython.h indirectly; indeed, almost
all have already done so via inclusions of other plpy_xxx.h headers.
(Any extensions using plpython.h can do likewise without breaking
the compatibility of their code with prior Postgres versions.)
Reported-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/aAxQ6fcY5QQV1lo3@ip-10-97-1-34.eu-west-3.compute.internal
M contrib/hstore_plpython/hstore_plpython.c
M contrib/jsonb_plpython/jsonb_plpython.c
M contrib/ltree_plpython/ltree_plpython.c
M src/pl/plpython/plpy_cursorobject.c
M src/pl/plpython/plpy_elog.c
M src/pl/plpython/plpy_exec.c
M src/pl/plpython/plpy_main.c
M src/pl/plpython/plpy_planobject.c
M src/pl/plpython/plpy_plpymodule.c
M src/pl/plpython/plpy_procedure.c
M src/pl/plpython/plpy_resultobject.c
M src/pl/plpython/plpy_spi.c
M src/pl/plpython/plpy_subxactobject.c
M src/pl/plpython/plpy_typeio.c
M src/pl/plpython/plpy_util.c
M src/pl/plpython/plpython.h
Don't use double-quotes in #include's of system headers.
commit : e8ca9ed1d260e214f02f7f5464f125fe86e55597
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 26 Apr 2025 20:30:27 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 26 Apr 2025 20:30:27 -0400
While few if any C compilers will complain about this, it's
inconsistent with our other #include's of the same headers.
There are some other questionable usages in
src/include/jit/SectionMemoryManager.h and
src/pl/plperl/plperl_system.h, but perhaps those have a
reason to be like that. I can't see that these do.
Noticed while fooling around with a script to do analysis
of our header cross-inclusions.
M src/include/storage/pmsignal.h
Eliminate divide in new fast-path locking code
commit : 936457419d797dfaf37191db53d5efb1279eae6d
author : David Rowley <drowley@postgresql.org>
date : Sun, 27 Apr 2025 11:53:40 +1200
committer: David Rowley <drowley@postgresql.org>
date : Sun, 27 Apr 2025 11:53:40 +1200
c4d5cb71d2 adjusted the fast-path locking code to allow some
configuration of the number of fast-path locking slots via the
max_locks_per_transaction GUC. In that commit the FAST_PATH_REL_GROUP()
macro used integer division to determine the fast-path locking group slot
to use for the lock.
The divisor in this case is always a power-of-two value. Here we swap
out the divide by a bitwise-AND, which is a significantly faster
operation to perform.
In passing, adjust the code that's setting FastPathLockGroupsPerBackend
so that it's more clear that the value being set is a power-of-two.
Also, adjust some comments in the area which contained some magic
numbers. It seems better to justify the 1024 upper limit in the
location where the #define is made instead of where it is used.
Author: David Rowley <drowleyml@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CAApHDvodr3bcnpxcs7+k-3cFwYR0tP-BYhyd2PpDhe-bCx9i=g@mail.gmail.com
M src/backend/storage/lmgr/lock.c
M src/backend/utils/init/postinit.c
M src/include/storage/proc.h
Match parameter in new function to earlier equivalents
commit : 27757677ca4da566ff21f65c58a008818afb5b06
author : John Naylor <john.naylor@postgresql.org>
date : Sun, 27 Apr 2025 03:03:52 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Sun, 27 Apr 2025 03:03:52 +0700
Oversight in commit 3c6e8c123.
M src/include/port/pg_crc32c.h
M src/port/pg_crc32c_sse42.c
doc: improve wording of vacuum_max_eager_freeze_failure_rate
commit : 10e8176950b14ab7710a1ea65cad3179b52edbe3
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 26 Apr 2025 11:41:23 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 26 Apr 2025 11:41:23 -0400
M doc/src/sgml/config.sgml
aio: Improve debug logging around waiting for IOs
commit : 039bfc457e436dc5c6b1d6ef996048ae19ff480d
author : Andres Freund <andres@anarazel.de>
date : Fri, 25 Apr 2025 12:31:42 -0400
committer: Andres Freund <andres@anarazel.de>
date : Fri, 25 Apr 2025 12:31:42 -0400
Trying to investigate a bug report by Alexander Lakhin made it apparent that
the debug logging around waiting for IO completion is insufficient. Fix that.
Discussion: https://postgr.es/m/h4in2db37vepagmi2oz5vvqymjasc5gyb4lpqkunj4eusu274i@37jpd3c2spd3
M src/backend/storage/aio/aio.c
Fix bug allowing io_combine_limit > io_max_combine_combine limit
commit : 500b61769fa27e9f00a03c3f544f70da3bf3d679
author : Andres Freund <andres@anarazel.de>
date : Fri, 25 Apr 2025 12:18:27 -0400
committer: Andres Freund <andres@anarazel.de>
date : Fri, 25 Apr 2025 12:18:27 -0400
10f66468475 intended to limit the value of io_combine_limit to the minimum of
io_combine_limit and io_max_combine_limit. To avoid issues with interdependent
GUCs, it introduced io_combine_limit_guc and set io_combine_limit in assign
hooks. That plan was thwarted by guc_tables.c accidentally still referencing
io_combine_limit, instead of io_combine_limit_guc. That lead to the GUC
machinery overriding the work done in the assign hooks, potentially leaving
io_combine_limit with a too high value.
The consequence of this bug was that when running with io_combine_limit >
io_combine_limit_guc the AIO machinery would not have reserved large enough
iovec and IO data arrays, with one IO's arrays overlapping with another IO's,
leading to total confusion.
To make such a problem easier to detect in the future, add assertions to
pgaio_io_set_handle_data_* checking the length is smaller than
io_max_combine_limit (not just PG_IOV_MAX).
It'd be nice to have a few tests for this, but it's not entirely obvious how
to do so portably.
As remarked upon by Tom, the GUC assignment hooks really shouldn't set the
underlying variable, that's the job of the GUC machinery. Change that as well.
Discussion: https://postgr.es/m/c5jyqnuwrpigd35qe7xdypxsisdjrdba5iw63mhcse4mzjogxo@qdjpv22z763f
M src/backend/commands/variable.c
M src/backend/storage/aio/aio_callback.c
M src/backend/utils/misc/guc_tables.c
aio: Fix crash potential for pg_aios views due to late state update
commit : 0d9114b7040d7503096e2897f4c856b17d461f6a
author : Andres Freund <andres@anarazel.de>
date : Fri, 25 Apr 2025 12:03:41 -0400
committer: Andres Freund <andres@anarazel.de>
date : Fri, 25 Apr 2025 12:03:41 -0400
pgaio_io_reclaim() reset the fields in PgAioHandle before updating the state
to IDLE or incrementing the generation. For most things that's OK, but for
pg_get_aios() it is not - if it copied the PgAioHandle while fields were being
reset, we wouldn't detect that and could call
pgaio_io_get_target_description() with ioh->target == PGAIO_TID_INVALID,
leading to a crash.
Fix this issue by incrementing the generation and state earlier, before
resetting.
Also add an assertion to pgaio_io_get_target_description() for the target to
be valid - that'd have made this case a bit easier to debug. While at it,
add/update a few related assertions.
Author: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/062daca9-dfad-4750-9da8-b13388301ad9@gmail.com
M src/backend/storage/aio/aio.c
M src/backend/storage/aio/aio_target.c
Fix incorrect format placeholders
commit : 76d52e71659149d20d616c8a94c02793cedce066
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Apr 2025 16:49:30 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Apr 2025 16:49:30 +0200
Before commit a0ed19e0a9e there was a cast around these, but the cast
inadvertently changed the signedness, but that made the format
placeholder correct. Commit a0ed19e0a9e removed the casts, so now the
format placeholders had the wrong signedness.
M src/fe_utils/print.c
Fix terminology in comment and message
commit : 385959bdeafb2e4c4b02adb55b78bde6eac616f7
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Apr 2025 16:26:28 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Apr 2025 16:26:28 +0200
Should be "bracket" not "brace" for [].
M src/interfaces/ecpg/preproc/variable.c
Small code consistency improvement
commit : 0787646e1dce966395f211fb9475dcab32daae70
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Apr 2025 12:56:40 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Apr 2025 12:56:40 +0200
Adjust the way the increment operators are placed to be consistent
throughout the function. Fixup for commit commit c1da7281060.
M src/bin/pg_dump/connectdb.c
Fix typo in test file name added in commit 4909b38af0.
commit : 50b8ad30f7541ff48f50a42be9d6bd59b8ce4074
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 25 Apr 2025 12:46:02 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 25 Apr 2025 12:46:02 +0530
Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Backpatch-through: 13
Discussion: https://postgr.es/m/CANhcyEXsObdjkjxEnq10aJumDpa5J6aiPzgTh_w4KCWRYHLw6Q@mail.gmail.com
M contrib/test_decoding/Makefile
R100 contrib/test_decoding/expected/invalidation_distrubution.out contrib/test_decoding/expected/invalidation_distribution.out
M contrib/test_decoding/meson.build
R100 contrib/test_decoding/specs/invalidation_distrubution.spec contrib/test_decoding/specs/invalidation_distribution.spec
doc: remove unnecessary secondary index terms for replication settings.
commit : 632f62dcecc78fe66f06a1ce1b0c57d16f32e191
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 25 Apr 2025 14:58:14 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 25 Apr 2025 14:58:14 +0900
Previously, config.sgml included secondary index terms for
max_replication_slots and max_active_replication_origins. These are
no longer necessary, as each parameter now has a single distinct index entry.
The secondary index terms were originally useful because
max_active_replication_origins was part of max_replication_slots,
and separate index entries helped users locate each setting. However,
commit 04ff636cbce split them into independent parameters,
making the secondary terms redundant.
This commit removes the unnecessary secondary index entries to
simplify the documentation.
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Euler Taveira <euler@eulerto.com>
Reviewed-by: Robert Treat <rob@xzilla.net>
Discussion: https://postgr.es/m/e825e7a7-4877-441d-93c1-25377db36c31@oss.nttdata.com
M doc/src/sgml/config.sgml
doc: simplify new EXPLAIN ANALYZE BUFFERS description
commit : 6389db23209a3a9b99759b26fbd26a9da5693116
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 24 Apr 2025 22:02:35 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 24 Apr 2025 22:02:35 -0400
M doc/src/sgml/ref/explain.sgml
psql: Fix assertion failures with pipeline mode
commit : 3631612eae9c2def99151c4f36b1b3771f53cba7
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 24 Apr 2025 12:22:53 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 24 Apr 2025 12:22:53 +0900
A correct cocktail of COPY FROM, SELECT and/or DML queries and
\syncpipeline was able to break the logic in charge of discarding
results of a pipeline, done in discardAbortedPipelineResults(). Such
sequence make the backend generate a FATAL, due to a protocol
synchronization loss.
This problem comes down to the fact that we did not consider the case of
libpq returning a PGRES_FATAL_ERROR when discarding the results of an
aborted pipeline. The discarding code is changed so as this result
status is handled as a special case, with the caller of
discardAbortedPipelineResults() being responsible for consuming the
result.
A couple of tests are added to cover the problems reported, bringing an
interesting gain in coverage as there were no tests in the tree covering
the case of protocol synchronization loss.
Issue introduced by 41625ab8ea3d.
Reported-by: Alexander Kozhemyakin <a.kozhemyakin@postgrespro.ru>
Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Co-authored-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/ebf6ce77-b180-4d6b-8eab-71f641499ddf@postgrespro.ru
M src/bin/psql/common.c
M src/bin/psql/t/001_basic.pl
Add sanity check for dshash entries when reading pgstats file
commit : 923ae50cf55986efb5f3feb0f15253a69a5e224c
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 24 Apr 2025 09:20:01 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 24 Apr 2025 09:20:01 +0900
Not having this check would produce a core dump at startup when running
pgstat_read_statsfile(), in the case where the information of a stats
kind for an entry in the dshash could not be found. The same check
already happens for fixed-numbered stats and entries that are stored
with their names. This issue can be seen with custom stats kinds.
Note that this problem can be reproduced what what is in the core code:
- Tweak the test module injection_points to not load the fixed-numbered
stats part, leaving only the variable-numbered stats.
- Create an instance with injection_points defined in
shared_preload_libraries.
- Create a pgstats entry by attaching and running a point.
- Restart the server without shared_preload_libraries. The startup
process detects that something is wrong and reports a WARNING.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/aAieZAvM+K1d89R2@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/utils/activity/pgstat.c
Avoid possibly-theoretical OOM crash hazard in hash_create().
commit : bc19f63f80109737e2610c81f65d33fc93246b2e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Apr 2025 16:04:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Apr 2025 16:04:42 -0400
One place in hash_create() used DynaHashAlloc() as a convenient
shorthand for MemoryContextAlloc(). That was fine when it was
written, but it stopped being fine when 9c911ec06 changed
DynaHashAlloc() to use MCXT_ALLOC_NO_OOM (mea culpa). Change
the code to call plain MemoryContextAlloc() as intended.
I think that this bug may be unreachable in practice, since we now
always create AllocSets with some space already allocated, so that
an OOM failure here for a non-shared hash table should be impossible
(with a hash table name of reasonable length anyway). And there
aren't enough shared hash tables to make a crash for one of those
probable. Nonetheless it's clearly not operating as designed, so
back-patch to v16 where 9c911ec06 came in.
Reported-by: Maksim Korotkov <m.korotkov@postgrespro.ru>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/219bdccd460510efaccf90b57e5e5ef2@postgrespro.ru
Backpatch-through: 16
M src/backend/utils/hash/dynahash.c
oauth: Support Python 3.6 in tests
commit : 005ccae0f2d68ad3d44141a748635f3dd4a46c1f
author : Jacob Champion <jchampion@postgresql.org>
date : Wed, 23 Apr 2025 11:16:45 -0700
committer: Jacob Champion <jchampion@postgresql.org>
date : Wed, 23 Apr 2025 11:16:45 -0700
RHEL8 ships a patched 3.6.8 as its base Python version, and I
accidentally let some newer Python-isms creep into oauth_server.py
during development.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Tested-by: Renan Alves Fonseca <renanfonseca@gmail.com>
Tested-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/16098.1745079444%40sss.pgh.pa.us
M src/test/modules/oauth_validator/t/oauth_server.py
Maintain RelIdToTypeIdCacheHash in TypeCacheOpcCallback()
commit : bb78e4267817b5c1f5a8ff5b0b93fe756e514bb0
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 23 Apr 2025 20:14:32 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 23 Apr 2025 20:14:32 +0300
b85a9d046efd introduced a new RelIdToTypeIdCacheHash, whose entries should
exist for typecache entries with TCFLAGS_HAVE_PG_TYPE_DATA flag set or any
of TCFLAGS_OPERATOR_FLAGS set or tupDesc set. However, TypeCacheOpcCallback(),
which resets TCFLAGS_OPERATOR_FLAGS, was forgotten to update
RelIdToTypeIdCacheHash.
This commit adds a delete_rel_type_cache_if_needed() call to the
TypeCacheOpcCallback() function to maintain RelIdToTypeIdCacheHash after
resetting TCFLAGS_OPERATOR_FLAGS.
Also, this commit fixes the name of the delete_rel_type_cache_if_needed()
function in its mentions in the comments.
Reported-by: Noah Misch
Discussion: https://postgr.es/m/20250411203241.e9.nmisch%40google.com
M src/backend/utils/cache/typcache.c
Properly prepare varinfos in estimate_multivariate_bucketsize()
commit : 9f404d7922e8831dc49bfa225530ba5309900e4e
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 23 Apr 2025 20:13:51 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 23 Apr 2025 20:13:51 +0300
To estimate with extended statistics, we need to clear the varnullingrels
field in the expression, and duplicates are not allowed in the GroupVarInfo
list. We might re-use add_unique_group_var(), but we don't do so for two
reasons.
1) We must keep the origin_rinfos list ordered exactly the same way as
varinfos.
2) add_unique_group_var() is designed for estimate_num_groups(), where a
larger number of groups is worse. While estimating the number of hash
buckets, we have the opposite: a lesser number of groups is worse.
Therefore, we don't have to remove "known equal" vars: the removed var
may valuably contribute to the multivariate statistics to grow the number
of groups.
This commit adds custom code to estimate_multivariate_bucketsize() to
initialize varinfos properly.
Reported-by: Robins Tharakan <tharakan@gmail.com>
Discussion: https://postgr.es/m/18885-da51324078588253%40postgresql.org
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
M src/backend/utils/adt/selfuncs.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Change the names generated for child foreign key constraints.
commit : 3db61db48ef5b8898f7e85f98548fdec79d76524
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Apr 2025 12:03:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Apr 2025 12:03:02 -0400
When a foreign key constraint is placed on a partitioned table, we
actually make two pg_constraint entries associated with that table.
(I have my doubts about the wisdom of that, but it's been like that
since v12 and post-feature-freeze is no time to be messing with such
entrenched decisions.) The second "child" entry always had a name
generated according to the default rule, "table_column(s)_fkey[nnn]",
even if the primary entry had an unrelated user-specified name. The
trouble with doing that is that the default name could collide with
the user-specified name of some other constraint on the same table.
While we were willing to adjust the generated name to avoid
collisions, that only helps if it's made second; if it's made first
then creation of the other constraint would fail, potentially causing
dump/reload or pg_upgrade failures.
The core of the problem here is that we're infringing on user
namespace, so I doubt that there's any 100% solution other than to
find a way to not need the "child" entry. In the meantime, it seems
like it'd be an improvement to make the child's name be the name of
the parent constraint with an underscore and digit(s) appended as
necessary to make it unique. This rule can in theory fail in the same
way, but it seems much less probable; for one thing, this rule is
guaranteed not to match primary entries having auto-generated names.
(While an auto-generated primary name isn't user-specified to begin
with, it acts like that during dump/reload, so collisions against such
names are definitely possible.)
An additional bonus, visible in some of the regression test cases
that change here, arises from the fact that some error messages
cite the child constraint's name not the parent's. In the
previous approach the two names could be completely unrelated,
leading to user confusion --- the more so since psql's \d command
hides child constraints. With this approach it's hopefully much
clearer which constraint-the-user-knows-about is failing.
However, that does mean that there's user-visible behavior change
occurring here, making it seem like not something to back-patch.
I feel it's not too late for v18, though.
Reported-by: Kirill Reshke <reshkekirill@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Alvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/CALdSSPhGitjpTfzEMJN-Y2x+Q-5QChSxAsmSJ1-E8mQJLkHOqQ@mail.gmail.com
M src/backend/catalog/pg_constraint.c
M src/backend/commands/tablecmds.c
M src/test/isolation/expected/detach-partition-concurrently-2.out
M src/test/isolation/expected/detach-partition-concurrently-4.out
M src/test/isolation/expected/fk-partitioned-1.out
M src/test/isolation/expected/fk-partitioned-2.out
M src/test/regress/expected/constraints.out
M src/test/regress/expected/foreign_key.out
M src/test/regress/expected/without_overlaps.out
M src/test/regress/sql/foreign_key.sql
Allocate JsonLexContexts on the heap to avoid warnings
commit : 994a100b37ad8c2fb8282a9fce91a16b4c832277
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 23 Apr 2025 11:02:05 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 23 Apr 2025 11:02:05 +0200
The stack allocated JsonLexContexts, in combination with codepaths
using goto, were causing warnings when compiling with LTO enabled
as the optimizer is unable to figure out that is safe. Rather than
contort the code with workarounds for this simply heap allocate the
structs instead as these are not in any performance critical paths.
Author: Daniel Gustafsson <daniel@yesql.se>
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/2074634.1744839761@sss.pgh.pa.us
M src/interfaces/libpq/fe-auth-oauth.c
M src/test/modules/test_json_parser/test_json_parser_incremental.c
psql: Rework TAP routine psql_fails_like() to define WAL sender context
commit : 0ff95e0a5be1372bfba9db284ea17c8e0e5da3a0
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 23 Apr 2025 15:33:07 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 23 Apr 2025 15:33:07 +0900
The routine was coded so as a WAL sender was always used, state required
only for one failure test related to START_REPLICATION. This test is
changed so as a WAL sender is used by passing a replication option to
psql_fails_like(), instead of forcing the use of a WAL sender for all
the tests.
This has come up as useful in the context of a separate bug fix where
we are looking at extending tests for some failure scenarios. These
tests need to happen in the context of a normal backend, and not a WAL
sender where the extended query protocol cannot be used.
Discussion: https://postgr.es/m/aAXkJIOildLUA7vQ@paquier.xyz
M src/bin/psql/t/001_basic.pl
Fix an oversight in 3f28b2fcac.
commit : 0e091ce409e06cd89baf81306cc92c496d8ba58d
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 23 Apr 2025 11:08:24 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 23 Apr 2025 11:08:24 +0530
Commit 3f28b2fcac tried to ensure that the replication origin shouldn't be
advanced in case of an ERROR in the apply worker, so that it can request
the same data again after restart. However, it is possible that an ERROR
was caught and handled by a (say PL/pgSQL) function, and the apply worker
continues to apply further changes, in which case, we shouldn't reset the
replication origin.
Ensure to reset the origin only when the apply worker exits after an
ERROR.
Commit 3f28b2fcac added new function geterrlevel, which we removed in HEAD
as part of this commit, but kept it in backbranches to avoid breaking any
applications. A separate case can be made to have such a function even for
HEAD.
Reported-by: Shawn McCoy <shawn.the.mccoy@gmail.com>
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Backpatch-through: 16, where it was introduced
Discussion: https://postgr.es/m/CALsgZNCGARa2mcYNVTSj9uoPcJo-tPuWUGECReKpNgTpo31_Pw@mail.gmail.com
M src/backend/replication/logical/worker.c
M src/backend/utils/error/elog.c
M src/include/utils/elog.h
M src/test/subscription/t/100_bugs.pl
Remove assertion based on pending_since in pgstat_report_stat()
commit : 1f7878c33cab8b741ecb0d145809cdd3000677be
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 23 Apr 2025 13:53:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 23 Apr 2025 13:53:29 +0900
This assertion, based on pending_since (timestamp used to prevent stats
reports to be too frequent or should a partial flush happen), is reached
when it is found that no data can be flushed but a previous call of
pgstat_report_stat() determined that some stats data has been found as
in need of a flush. So pending_since is set when some stats data is
pending (in non-force mode) or if report attempts are too frequent, and
reset to 0 once all stats have been flushed.
Since 5cbbe70a9cc6, WAL senders have begun to report their stats on a
periodic basis for IO stats in v16~ and backend stats on HEAD, creating
some friction with the concurrent pgstat_report_stat() calls that can
happen in the context of a WAL sender (shutdown callback doing a final
report or backend-related code paths). This problem is the cause of
spurious failures in the TAP tests.
In theory, this assertion can be also reached in v15, even if that's
very unlikely. For example, a process, say a background worker, could
do periodic and direct stats flushes with concurrent calls of
pgstat_report_stat() that could cause conflicting values of
pending_since. This can be done with WAL or SLRU stats flushes using
pgstat_flush_wal() or pgstat_slru_flush(). HEAD makes this situation
easier to happen with custom cumulative stats.
This commit removes the assertion altogether, per discussion, as it is
more useful to keep the state of things as they are for the WAL sender.
The assertion could use a special state based on for example
am_walsender, but I doubt that this would be meaningful in the long run
based on the other arguments raised while discussing this issue.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/1489124.1744685908@sss.pgh.pa.us
Discussion: https://postgr.es/m/dwrkeszz6czvtkxzr5mqlciy652zau5qqnm3cp5f3p2po74ppk@omg4g3cc6dgq
Backpatch-through: 15
M src/backend/utils/activity/pgstat.c
Re-enable SSL connect_fails tests, and fix related race conditions.
commit : e0f373ee42a40a41bdfc025a1641d351580991c4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Apr 2025 15:10:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Apr 2025 15:10:50 -0400
Cluster.pm's connect_fails routine has long had the ability to
sniff the postmaster log file for expected messages after a
connection failure. However, that's always had a race condition:
on some platforms it's possible for psql to exit and the test
script to slurp up the postmaster log before the backend process
has been able to write out its final log messages. Back in
commit 55828a6b6 we disabled a bunch of tests after discovering
that, and the aim of this patch is to re-enable them.
(The sibling function connect_ok doesn't seem to have a similar
problem, mainly because the messages we look for come out during
the authentication handshake, so that if psql reports successful
connection they should certainly have been emitted already.)
The solution used here is borrowed from 002_connection_limits.pl's
connect_fails_wait routine: set the server's log_min_messages setting
to DEBUG2 so that the postmaster will log child-process exit, and then
wait till we see that log entry before checking for the messages we
are actually interested in.
If a TAP test uses connect_fails' log_like or log_unlike options, and
forgets to set log_min_messages, those connect_fails calls will now
hang until timeout. Fixing up the existing callers shows that we had
several other TAP tests that were in theory vulnerable to the same
problem. It's unclear whether the lack of failures is just luck, or
lack of buildfarm coverage, or perhaps there is some obscure timing
effect that only manifests in SSL connections. In any case, this
change should in principle make those other call sites more robust.
I'm not inclined to back-patch though, unless sometime we observe
an actual failure in one of them.
Reported-by: Andrew Dunstan <andrew@dunslane.net>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/984fca80-85a8-4c6f-a5cc-bb860950b435@dunslane.net
M src/test/authentication/t/001_password.pl
M src/test/authentication/t/003_peer.pl
M src/test/kerberos/t/001_auth.pl
M src/test/ldap/t/001_auth.pl
M src/test/modules/oauth_validator/t/001_server.pl
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/test/ssl/t/001_ssltests.pl
Avoid depending on post-UPDATE row order in float4/float8 tests.
commit : da83b1ea10c2b7937d4c9e922465321749c6785b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Apr 2025 14:24:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Apr 2025 14:24:21 -0400
While heapam reproduces the insertion order of rows well, updates
can move rows to varying places depending on autovacuum activity.
In most regression tests we've guarded against getting variable
results due to that, but float4.sql and float8.sql had escaped
notice so far because they update tables that are too small for
autovacuum to pay attention to.
With increasing interest in non-heap table AMs, it seems worth
allowing for update behaviors that are not like heapam's. Hence,
add ORDER BY to stabilize the results in case the updates put
the rows in a different order. (We'll continue to assume that a
seqscan will reproduce original insertion order, though. Removing
that assumption would require vastly-more-invasive test changes.)
Author: Pavel Borisov <pashkin.elfe@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CALT9ZEExHAnBoBVQzQuWPMKUbapF5-FBO3fdeYG3s2tuWQz1NQ@mail.gmail.com
M src/test/regress/expected/float4-misrounded-input.out
M src/test/regress/expected/float4.out
M src/test/regress/expected/float8.out
M src/test/regress/sql/float4.sql
M src/test/regress/sql/float8.sql
gen_node_support.pl: improve error message for unclosed struct.
commit : eaf582806c0d6b19c081c5afecd9ddb7003e6f80
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Apr 2025 13:56:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Apr 2025 13:56:31 -0400
This error message was 'runaway "struct_name"', which isn't all
that clear; I think 'could not find closing brace for "struct_name"'
is better. Also, provide the location of the struct start using the
script's usual '$file:$lineno' style.
Bug: #18901
Reported-by: Clemens Ruck <clemens.ruck@t-online.de>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18901-424272abe01357e6@postgresql.org
M src/backend/nodes/gen_node_support.pl
doc: Mention naming convention used by injection points
commit : e29df428a1dca4112aad640c889a9a54642759c9
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 22 Apr 2025 12:41:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 22 Apr 2025 12:41:29 +0900
All the injection points used in the tree have relied on an implied
rule: their names should be made of lower-case characters, with dashes
between the words used.
This commit adds a light mention about that in the docs, encouraging the
practice.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://postgr.es/m/OSCPR01MB14966E14C1378DEE51FB7B7C5F5B32@OSCPR01MB14966.jpnprd01.prod.outlook.com
Backpatch-through: 17
M doc/src/sgml/xfunc.sgml
Doc: reword text explaining the --maintenance-db option
commit : 0b06459f3c0a515c922f2a22974acea9db38c024
author : David Rowley <drowley@postgresql.org>
date : Tue, 22 Apr 2025 14:54:22 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 22 Apr 2025 14:54:22 +1200
The previous text was a little clumsy. Here we improve that.
Author: David Rowley <dgrowleyml@gmail.com>
Reported-by: Noboru Saito <noborusai@gmail.com>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/CAAM3qnJtv5YbjpwDfVOYN2gZ9zGSLFM1UGJgptSXmwfifOZJFQ@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/ref/clusterdb.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
Rename injection point for invalidation messages at end of transaction
commit : 02c63f9438e082ac58284eed52a9a46015a83432
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 22 Apr 2025 10:01:38 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 22 Apr 2025 10:01:38 +0900
This injection point was named "AtEOXact_Inval-with-transInvalInfo", not
respecting the implied naming convention that injection points should
use lower-case characters, with terms separated by dashes. All the
other points defined in the tree follow this style, so let's be more
consistent.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://postgr.es/m/OSCPR01MB14966E14C1378DEE51FB7B7C5F5B32@OSCPR01MB14966.jpnprd01.prod.outlook.com
Backpatch-through: 17
M src/backend/utils/cache/inval.c
M src/test/modules/injection_points/expected/syscache-update-pruned.out
M src/test/modules/injection_points/expected/syscache-update-pruned_1.out
M src/test/modules/injection_points/specs/syscache-update-pruned.spec
Doc: various fixups
commit : 5e6f9a9c4e7edbe110eab25f9fc05a6d136b950a
author : David Rowley <drowley@postgresql.org>
date : Tue, 22 Apr 2025 11:10:08 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 22 Apr 2025 11:10:08 +1200
* Use <symbol> tags for CONNECTION_* #defines
We were using an inconsistent mix of <literal> and sometimes <function>
tags.
* Use <application> tag for libpq
There was a mix of <literal> and <productname>
Also fix a whitespace issue.
None of these seem critical enough mistakes to backpatch.
Author: Noboru Saito <noborusai@gmail.com>
Discussion: https://postgr.es/m/CAAM3qnJtv5YbjpwDfVOYN2gZ9zGSLFM1UGJgptSXmwfifOZJFQ@mail.gmail.com
M doc/src/sgml/libpq.sgml
M doc/src/sgml/logicaldecoding.sgml
Doc: fix incorrect punctuation
commit : d010cc6ccacf88926255d3ae2d67dad3e332ae8a
author : David Rowley <drowley@postgresql.org>
date : Tue, 22 Apr 2025 11:04:04 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 22 Apr 2025 11:04:04 +1200
Author: Noboru Saito <noborusai@gmail.com>
Discussion: https://postgr.es/m/CAAM3qnJtv5YbjpwDfVOYN2gZ9zGSLFM1UGJgptSXmwfifOZJFQ@mail.gmail.com
Backpatch-through: 17
M doc/src/sgml/monitoring.sgml
Fix INITCAP() word boundaries for PG_UNICODE_FAST.
commit : 90260e2ec6bbfc3dfa9d9501ab75c535de52f677
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 21 Apr 2025 12:34:58 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 21 Apr 2025 12:34:58 -0700
Word boundaries are based on whether a character is alphanumeric or
not. For the PG_UNICODE_FAST collation, alphanumeric includes
non-ASCII digits; whereas for the PG_C_UTF8 collation, it only
includes digits 0-9. Pass down the right information from the
pg_locale_t into initcap_wbnext to differentiate the behavior.
Reported-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250417135841.33.nmisch@google.com
M src/backend/utils/adt/pg_locale_builtin.c
M src/common/unicode/case_test.c
M src/test/regress/expected/collate.utf8.out
M src/test/regress/sql/collate.utf8.sql
Use the same cmd_context throughout a walsender's lifetime.
commit : 80b727eb9deab589a8648750bc20f1623d5acd3e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Apr 2025 12:09:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Apr 2025 12:09:36 -0400
exec_replication_command created a cmd_context to work in and
then deleted it on exit. This is pretty dangerous because
some replication commands start/finish transactions. In the
wake of commit 1afe31f03, that could lead to re-selecting a
CurrentMemoryContext that's already been deleted, leading to
hilarity such as a memory context that is its own parent.
To fix, let's make the cmd_context persist across
exec_replication_command calls; instead of deleting it, we'll just
reset it each time. In this way it retains the same identity and
there's no problem if transaction abort restores it as the working
context. It probably even saves a few microseconds to do this.
This fix also ensures that exec_replication_command returns to the
caller (PostgresMain) with the same context active that had been
when it was called (probably MessageContext). The previous
coding could get that wrong too.
Reported-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAO6_XqoJA7-_G6t7Uqe5nWF3nj+QBGn4F6Ptp=rUGDr0zo+KvA@mail.gmail.com
M src/backend/replication/walsender.c
M src/test/subscription/t/100_bugs.pl
MemoryContextCreate: assert parent is valid and different from node.
commit : 5ec8b01c30e7ea34bb42592ad9d34d4b02ea593d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Apr 2025 11:34:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Apr 2025 11:34:36 -0400
The case of "node == parent" might seem impossible, since we just
allocated the new node. But it's possible if parent is a dangling
reference to a recently-deleted context. In fact, given aset.c's
habit of recycling contexts, it's actually rather likely if that's so.
If we'd had this assertion before, it would have simplified debugging
a recently-identified walsender issue.
Reported-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAO6_XqoJA7-_G6t7Uqe5nWF3nj+QBGn4F6Ptp=rUGDr0zo+KvA@mail.gmail.com
M src/backend/utils/mmgr/mcxt.c
doc: Fix memory context level in pg_log_backend_memory_contexts() example.
commit : 706cbed351037fb5e886815506515d1281e62d40
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 21 Apr 2025 14:53:25 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 21 Apr 2025 14:53:25 +0900
Commit d9e03864b6b changed the memory context level numbers shown by
pg_log_backend_memory_contexts() to be 1-based. However, the example in
the documentation was not updated and still used 0-based numbering.
This commit updates the example to match the current 1-based output.
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: David Rowley <drowleyml@gmail.com>
Discussion: https://postgr.es/m/1ad6d388-1b43-400d-bec9-36d52f755f74@oss.nttdata.com
M doc/src/sgml/func.sgml
Fix a few more duplicate words in comments
commit : 78eda9e2648ab3f8c0df0d4aca53eb0ad6399c59
author : David Rowley <drowley@postgresql.org>
date : Mon, 21 Apr 2025 13:50:50 +1200
committer: David Rowley <drowley@postgresql.org>
date : Mon, 21 Apr 2025 13:50:50 +1200
Similar to 84fd3bc14 but these ones were found using a regex that can span
multiple lines.
Author: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvrMcr8XD107H3NV=WHgyBcu=sx5+7=WArr-n_cWUqdFXQ@mail.gmail.com
M src/backend/storage/buffer/freelist.c
M src/backend/utils/adt/ruleutils.c
M src/interfaces/libpq/fe-connect.c
Fix a few duplicate words in comments
commit : 84fd3bc141039b32f25c3c9f6157163a56ab5f66
author : David Rowley <drowley@postgresql.org>
date : Mon, 21 Apr 2025 10:41:18 +1200
committer: David Rowley <drowley@postgresql.org>
date : Mon, 21 Apr 2025 10:41:18 +1200
These are all new to v18
Author: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvrMcr8XD107H3NV=WHgyBcu=sx5+7=WArr-n_cWUqdFXQ@mail.gmail.com
M contrib/amcheck/verify_heapam.c
M contrib/pg_buffercache/pg_buffercache_pages.c
M src/backend/access/nbtree/nbtpreprocesskeys.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/executor/execPartition.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/mmgr/mcxt.c
M src/bin/pg_upgrade/relfilenumber.c
M src/include/storage/aio.h
Comment on need to MarkBufferDirty() if omitting DELAY_CHKPT_START.
commit : 818013665259d4242ba641aad705ebe5a3e2db8e
author : Noah Misch <noah@leadboat.com>
date : Sun, 20 Apr 2025 12:00:17 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 20 Apr 2025 12:00:17 -0700
Blocking checkpoint phase 2 requires MarkBufferDirty() and
BUFFER_LOCK_EXCLUSIVE; neither suffices by itself. transam/README documents
this, citing SyncOneBuffer(). Update the DELAY_CHKPT_START documentation to
say this. Expand the heap_inplace_update_and_unlock() comment that cites
XLogSaveBufferForHint() as precedent, since heap_inplace_update_and_unlock()
could have opted not to use DELAY_CHKPT_START.
Commit 8e7e672cdaa6bfec85d4d5dd9be84159df23bb41 added DELAY_CHKPT_START to
heap_inplace_update_and_unlock(). Since commit
bc6bad88572501aecaa2ac5d4bc900ac0fd457d5 reverted it in non-master branches,
no back-patch.
Discussion: https://postgr.es/m/20250406180054.26.nmisch@google.com
M src/backend/access/heap/heapam.c
M src/include/storage/proc.h
Test restartpoints in archive recovery.
commit : 714bd9e3a733a98da47531a750182ee5981cc3ed
author : Noah Misch <noah@leadboat.com>
date : Sun, 20 Apr 2025 08:28:48 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 20 Apr 2025 08:28:48 -0700
v14 commit 1f95181b44c843729caaa688f74babe9403b5850 and its v13
equivalent caused timing-dependent failures in archive recovery, at
restartpoints. The symptom was "invalid magic number 0000 in log
segment X, offset 0", "unexpected pageaddr X in log segment Y, offset 0"
[X < Y], or an assertion failure. Commit
3635a0a35aafd3bfa80b7a809bc6e91ccd36606a and predecessors back-patched
v15 changes to fix that. This test reproduces the problem
probabilistically, typically in less than 1000 iterations of the test.
Hence, buildfarm and CI runs would have surfaced enough failures to get
attention within a day.
Reported-by: Arun Thirupathi <arunth@google.com>
Discussion: https://postgr.es/m/20250306193013.36.nmisch@google.com
Backpatch-through: 13
M src/test/recovery/meson.build
A src/test/recovery/t/045_archive_restartpoint.pl
Avoid ERROR at ON COMMIT DELETE ROWS after relhassubclass=f.
commit : 2d5350cfbdc6e61124214d68713295bb060bc541
author : Noah Misch <noah@leadboat.com>
date : Sun, 20 Apr 2025 08:28:48 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 20 Apr 2025 08:28:48 -0700
Commit 7102070329d8147246d2791321f9915c3b5abf31 fixed a similar bug, but
it missed the case of database-wide ANALYZE ("use_own_xacts" mode).
Commit a07e03fd8fa7daf4d1356f7cb501ffe784ea6257 changed consequences
from silent discard of a pg_class stats (relpages et al.) update to
ERROR "tuple to be updated was already modified". Losing a relpages
update of an ON COMMIT DELETE ROWS table was negligible, but a
COMMIT-time error isn't negligible. Back-patch to v13 (all supported
versions).
Reported-by: Richard Guo <guofenglinux@gmail.com
Reported-by: Robins Tharakan <tharakan@gmail.com>
Discussion: https://postgr.es/m/CAMbWs4-XwMKMKJ_GT=p3_-_=j9rQSEs1FbDFUnW9zHuKPsPNEQ@mail.gmail.com
Backpatch-through: 13
M src/backend/commands/vacuum.c
A src/test/regress/expected/maintain_every.out
M src/test/regress/parallel_schedule
A src/test/regress/sql/maintain_every.sql
Fix issue with ORDER BY / DISTINCT aggregates and FILTER
commit : d47f922246b54f0290811951e0b73a3d6110437c
author : David Rowley <drowley@postgresql.org>
date : Sun, 20 Apr 2025 22:12:07 +1200
committer: David Rowley <drowley@postgresql.org>
date : Sun, 20 Apr 2025 22:12:07 +1200
1349d2790 added support so that aggregate functions with an ORDER BY or
DISTINCT clause could make use of presorted inputs to avoid an implicit
sort within nodeAgg.c. That commit failed to consider that a FILTER
clause may exist that filters rows before the aggregate function
arguments are evaluated. That can be problematic if an aggregate
argument contains an expression which could error out during evaluation.
It's perfectly valid to want to have a FILTER clause which eliminates
such values, and with the pre-sorted path added in 1349d2790, it was
possible that the planner would produce a plan with a Sort node above
the Aggregate to perform the sort on the aggregate's arguments long before
the Aggregate node would filter out the non-matching values.
Here we fix this by inspecting ORDER BY / DISTINCT aggregate functions
which have a FILTER clause to see if the aggregate's arguments are
anything more complex than a Var or a Const. Evaluating these isn't
going to cause an error. If we find any non-Var, non-Const parameters
then the planner will now opt to perform the sort in the Aggregate node
for these aggregates, i.e. disable the presorted aggregate optimization.
An alternative fix would have been to completely disallow the presorted
optimization for Aggrefs with any FILTER clause, but that wasn't done as
that could cause large performance regressions for queries that see
significant gains from 1349d2790 due to presorted results coming in from
an Index Scan.
Backpatch to 16, where 1349d2790 was introduced
Author: David Rowley <dgrowleyml@gmail.com>
Reported-by: Kaimeh <kkaimeh@gmail.com>
Diagnosed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAK-%2BJz9J%3DQ06-M7cDJoPNeYbz5EZDqkjQbJnmRyQyzkbRGsYkA%40mail.gmail.com
Backpatch-through: 16
M src/backend/optimizer/plan/planner.c
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/sqljson.out
M src/test/regress/sql/aggregates.sql
psql: Split extended query protocol meta-commands in --help=commands
commit : 78231baaf967166776de8262072acb8d589d3b6a
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 20 Apr 2025 08:34:38 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 20 Apr 2025 08:34:38 +0900
Compared to v17 with only \bind able to do extended query protocol work,
v18 has now a total of 11 meta-commands related to the extended query
protocol. These were all listed under the "General" section of the
--help=commands output and are specialized, bloating the output
generated.
All these meta-commands are moved into a new section called "Extended
Query Protocol", listed at the end of --help=commands.
This split has been suggested by Noah Misch.
Discussion: https://postgr.es/m/20250415213450.1f.nmisch@google.com
M src/bin/psql/help.c
psql: Improve descriptions of \\flush[request] in --help
commit : 5743d122fcf35edba801609fcde7410610ed0f19
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 20 Apr 2025 08:16:57 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 20 Apr 2025 08:16:57 +0900
Noah has reported that the current wording was confusing compared to the
description of the underlying libpq routine. The new wording is from
me.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250415213450.1f.nmisch@google.com
M src/bin/psql/help.c
psql: Fix incorrect status code returned by \getresults
commit : 5ee7bd944ee812e988a7dee52613ee1c80c13579
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 20 Apr 2025 08:15:39 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 20 Apr 2025 08:15:39 +0900
When an invalid number of results is requested for \getresults, the
status code returned by exec_command_getresults() was PSQL_CMD_SKIP_LINE
and not PSQL_CMD_ERROR.
This led to incorrect behaviors, with ON_ERROR_STOP for example.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250415213450.1f.nmisch@google.com
M src/bin/psql/command.c
Be more wary of corrupt data in pageinspect's heap_page_items().
commit : d05996340d7a4bc32c464be085727ccddf28376b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 19 Apr 2025 16:37:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 19 Apr 2025 16:37:42 -0400
The original intent in heap_page_items() was to return nulls, not
throw an error or crash, if an item was sufficiently corrupt that
we couldn't safely extract data from it. However, commit d6061f83a
utterly missed that memo, and not only put in an un-length-checked
copy of the tuple's data section, but also managed to break the check
on sane nulls-bitmap length. Either mistake could possibly lead to
a SIGSEGV crash if the tuple is corrupt.
Bug: #18896
Reported-by: Dmitry Kovalenko <d.kovalenko@postgrespro.ru>
Author: Dmitry Kovalenko <d.kovalenko@postgrespro.ru>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18896-add267b8e06663e3@postgresql.org
Backpatch-through: 13
M contrib/pageinspect/heapfuncs.c
Fix typos and grammar in the code
commit : 88e947136b47664b6936b35542f2d1eda0c90588
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 19 Apr 2025 19:17:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 19 Apr 2025 19:17:42 +0900
The large majority of these have been introduced by recent commits done
in the v18 development cycle.
Author: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/9a7763ab-5252-429d-a943-b28941e0e28b@gmail.com
M contrib/amcheck/verify_common.h
M contrib/amcheck/verify_gin.c
M contrib/pg_buffercache/pg_buffercache_pages.c
M contrib/pg_overexplain/expected/pg_overexplain.out
M contrib/pg_overexplain/pg_overexplain.c
M contrib/postgres_fdw/t/001_auth_scram.pl
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/regress.sgml
M src/backend/access/gin/gininsert.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/transam/xlog.c
M src/backend/catalog/catalog.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/backend/executor/nodeSeqscan.c
M src/backend/nodes/queryjumblefuncs.c
M src/backend/postmaster/postmaster.c
M src/backend/storage/aio/README.md
M src/backend/storage/aio/method_worker.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/page/bufpage.c
M src/backend/utils/adt/mcxtfuncs.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/mmgr/mcxt.c
M src/bin/pg_dump/t/006_pg_dumpall.pl
M src/bin/pg_upgrade/relfilenumber.c
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
M src/bin/psql/common.c
M src/bin/scripts/t/100_vacuumdb.pl
M src/include/access/gin_tuple.h
M src/include/nodes/pathnodes.h
M src/include/storage/aio_subsys.h
M src/interfaces/libpq/fe-auth-oauth-curl.c
M src/interfaces/libpq/fe-cancel.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-protocol3.c
M src/port/pg_localeconv_r.c
M src/port/pg_popcount_aarch64.c
M src/test/modules/oauth_validator/t/002_client.pl
M src/test/modules/test_aio/t/001_aio.pl
M src/test/recovery/t/035_standby_logical_decoding.pl
M src/test/subscription/t/007_ddl.pl
Rename injection points used in AIO tests
commit : 114f7fa81c72637d75b574269f2076dcc1104e24
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 19 Apr 2025 18:53:35 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 19 Apr 2025 18:53:35 +0900
The format of the injection point names used by the AIO code does not
match the existing naming convention used everywhere else in the code,
so let's be consistent. These points are used in test_aio.
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Discussion: https://postgr.es/m/Z_yTB80bdu1sYDqJ@paquier.xyz
M src/backend/storage/aio/aio.c
M src/backend/storage/aio/method_worker.c
M src/test/modules/test_aio/test_aio.c
Make pg_upgrade log message with control file path translatable.
commit : 3aad76a0a970faf22ea38dd61b5668d480b5a92f
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 18 Apr 2025 18:35:40 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 18 Apr 2025 18:35:40 +0900
Commit 173c97812ff replaced the hardcoded "global/pg_control" in pg_upgrade
log message with a string literal concatenation of XLOG_CONTROL_FILE macro.
However, this change made the message untranslatable.
This commit fixes the issue by using %s with XLOG_CONTROL_FILE instead of
that literal concatenation, allowing the message to be translated properly.
It also wraps the file path in double quotes for consistency with similar
log messages.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Masao Fujii <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/20250407.155546.2129693791769531891.horikyota.ntt@gmail.com
M src/bin/pg_upgrade/controldata.c
Doc: fix missing comma at the end of a line.
commit : 05883bd6e556bb4d86c7ece4c37080d00cdb0d34
author : Tatsuo Ishii <ishii@postgresql.org>
date : Fri, 18 Apr 2025 09:38:46 +0900
committer: Tatsuo Ishii <ishii@postgresql.org>
date : Fri, 18 Apr 2025 09:38:46 +0900
Backpatch to 17, where the line was added.
Reported by Noboru Saito while he was working on translating the file
into Japanese.
Discussion: https://postgr.es/m/20250417.203047.1321297410457834775.ishii%40postgresql.org
Reported-by: Noboru Saito <noborusai@gmail.com>
Reviewed-by: Daniel Gustafs <daniel@yesql.se>
Backpatch-through: 17
M doc/src/sgml/libpq.sgml
Fixup various older misuses of appendPQExpBuffer
commit : 1bd08f6ba56ac9b23dc46ab634d82327395e882e
author : David Rowley <drowley@postgresql.org>
date : Fri, 18 Apr 2025 12:15:08 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 18 Apr 2025 12:15:08 +1200
Use appendPQExpBufferStr when there are no parameters and
appendPQExpBufferChar when the string length is 1.
Unlike 3fae25cbb, which fixed this issue for code that was new to v18,
this one fixes up instances which exist in the backbranches. We've
historically tried to maintain this standard and if we're going to
continue doing that, then we won't be doing that selectively based on
when the code was introduced. Now seems like a good time to flush out the
existing misuses. Waiting until v19 just prolongs their existence in
terms of released versions that the misuses exist in.
Author: David Rowley <drowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvoARMvPeXTTC0HnpARBHn-WgVstc8XFCyMGOzvgu_1HvQ@mail.gmail.com
M src/bin/initdb/initdb.c
M src/bin/pg_basebackup/pg_createsubscriber.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/psql/common.c
M src/bin/scripts/vacuumdb.c
Make levels 1-based in pg_log_backend_memory_contexts()
commit : d9e03864b6b4c7fe2988393c22e2355786c4c1f7
author : David Rowley <drowley@postgresql.org>
date : Fri, 18 Apr 2025 09:04:28 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 18 Apr 2025 09:04:28 +1200
Both pg_get_process_memory_contexts() and pg_backend_memory_contexts
have 1-based levels, whereas pg_log_backend_memory_contexts() was using
0-based levels. Align these.
This results in slightly saner behavior from MemoryContextStatsDetail()
in regards to the max_level. Previously it would stop at 1 level before
the maximum requested level rather than at that level.
Reported-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Author: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Author: David Rowley <drowleyml@gmail.com
Reviewed-by: Melih Mutlu <m.melihmutlu@gmail.com>
Reviewed-by: Rahila Syed <rahilasyed90@gmail.com>
Discussion: https://postgr.es/m/395ea5d4fe190480efa95bf533485c70@oss.nttdata.com
M src/backend/utils/mmgr/mcxt.c
Suppress "may be used uninitialized" warnings from older compilers.
commit : fc5e966f73f06ab2b3e7fa488fc2712d77639947
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 16:47:04 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 16:47:04 -0400
The "children" list won't be used until "got_children" has been set
true, but older compilers don't get that; about half a dozen
buildfarm animals are warning about this. Issue added by 11ff192b5.
While here, improve slightly-shaky grammar in comment.
Discussion: https://postgr.es/m/2057835.1744833309@sss.pgh.pa.us
M src/backend/commands/tablecmds.c
Portability fix: isdigit() must be passed an unsigned char.
commit : 4aad2cb7707dfb239eaaad29a8a7155027d8e8b8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 16:33:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 16:33:21 -0400
Oversight in commit 40b9c2701, per buildfarm member mamba.
M src/bin/pg_dump/pg_restore.c
Cache typlens of a SQL function's input arguments.
commit : 0400ae4a68831250347407463453447daa9548df
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 12:56:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 12:56:40 -0400
This gets rid of repetitive get_typlen calls in postquel_sub_params,
which show up as costing a few percent of the runtime in simple test
cases (more with more parameters).
In combination with the preceding patches, this gets us most of the
way back down to the amount of per-call overhead that functions.c
had before commit 0dca5d68d. There are some more things that could
be done, but this seems like an okay place to stop for v18.
M src/backend/executor/functions.c
Make SQLFunctionCache long-lived again.
commit : 0313c5dc627a1407344617fa8dd84ce1374ec915
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 12:56:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 12:56:31 -0400
At this point, the only data structures we allocate directly in
fcontext are the SQLFunctionCache struct itself, the ParamListInfo
struct, and the execution_state array, all of which are small and
perfectly capable of being re-used across executions of the same
FmgrInfo. Hence, let's give them the same lifespan as the FmgrInfo.
This step gets rid of the separate SQLFunctionLink struct and makes
fn_extra point to SQLFunctionCache again. We also get rid of the
separate fcontext memory context and allocate these items directly
in fn_mcxt.
For notational simplicity, SQLFunctionCache still has an fcontext
field, but it's just a copy of fn_mcxt.
The motivation for this is to allow these structures to live as
long as the FmgrInfo and be re-used across calls, restoring the
original design without its propensity for memory leaks. This
gets rid of some per-call overhead that we added in 0dca5d68d.
We also make an effort to re-use the JunkFilter and result slot.
Those might need to change if the function definition changes,
so we compromise by rebuilding them if the cached plan changes.
This also moves the tuplestore into fn_mcxt so that it can be
re-used across calls, again undoing a change made in 0dca5d68d.
M src/backend/executor/functions.c
Split some storage out to separate subcontexts of fcontext.
commit : f45a5444ee7612bcca31172035c9638fed77afcc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 12:56:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 12:56:21 -0400
Put the JunkFilter and its result slot (and thence also
some subsidiary data such as the result tupledesc) into a
separate subcontext "jfcontext". This doesn't accomplish
a lot at this point, because we make a new JunkFilter each
time through the SQL function. However, the plan is to make
the fcontext long-lived, and that raises the possibility
that we'll need a new JunkFilter because the plan for the
result-generating query changes. A separate context makes
it easy to free the obsoleted data when that happens.
Also, instead of always running the sub-executor in fcontext,
make a separate context for it if we're doing lazy eval of
a SRF, and otherwise just run it inside CurrentMemoryContext.
M src/backend/executor/functions.c
Make functions.c mostly run in a short-lived memory context.
commit : 595d1efeda11186ac6850f5e0bfec877da363e1e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 12:56:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 12:56:08 -0400
Previously, much of this code ran with CurrentMemoryContext set
to be the function's fcontext, so that we tended to leak a lot of
stuff there. Commit 0dca5d68d dealt with that by releasing the
fcontext at the completion of each SQL function call, but we'd
like to go back to the previous approach of allowing the fcontext
to be query-lifespan. To control the leakage problem, rearrange
the code so that we mostly run in the memory context that fmgr_sql
is called in (which we expect to be short-lived). Notably, this
means that parsing/planning is all done in the short-lived context
and doesn't leak cruft into fcontext.
This patch also fixes the allocation of execution_state records
so that we don't leak them across executions. I set that up
with a re-usable array that contains at least as many
execution_state structs as we need for the current querytree.
The chain structure is still there, but it's not really doing
much for us, and maybe somebody will be motivated to get rid
of it. I'm not though.
This incidentally also moves the call of BlessTupleDesc to be
with the code that creates the JunkFilter. That doesn't make
much difference now, but a later patch will reduce the number
of times the JunkFilter gets made, and we needn't bless the
results any more often than that.
We still leak a fair amount in fcontext, particularly when
executing utility statements, but that's material for a
separate patch step; the point here is only to get rid of
unintentional allocations in fcontext.
M src/backend/executor/functions.c
Minor performance improvement for SQL-language functions.
commit : 09b07c29532fe7db87cbfe1c54355cfc80291b6c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 12:55:58 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Apr 2025 12:55:58 -0400
Late in the development of commit 0dca5d68d, I added a step to copy
the result tlist we extract from the cached final query, because
I was afraid that that might not last as long as the JunkFilter that
we're passing it off to. However, that turns out to cost a noticeable
number of cycles, and it's really quite unnecessary because the
JunkFilter will not examine that tlist after it's been created.
(ExecFindJunkAttribute would use it, but we don't use that function
on this JunkFilter.) Hence, remove the copy step. For safety,
reset the might-become-dangling jf_targetList pointer to NIL.
In passing, remove DR_sqlfunction.cxt, which we don't use anymore;
it's confusing because it's not entirely clear which context it
ought to point at.
M src/backend/executor/functions.c
Assert lack of hazardous buffer locks before possible catalog read.
commit : f4ece891fc2f3f96f0571720a1ae30db8030681b
author : Noah Misch <noah@leadboat.com>
date : Thu, 17 Apr 2025 05:00:30 -0700
committer: Noah Misch <noah@leadboat.com>
date : Thu, 17 Apr 2025 05:00:30 -0700
Commit 0bada39c83a150079567a6e97b1a25a198f30ea3 fixed a bug of this kind,
which existed in all branches for six days before detection. While the
probability of reaching the trouble was low, the disruption was extreme. No
new backends could start, and service restoration needed an immediate
shutdown. Hence, add this to catch the next bug like it.
The new check in RelationIdGetRelation() suffices to make autovacuum detect
the bug in commit 243e9b40f1b2dd09d6e5bf91ebf6e822a2cd3704 that led to commit
0bada39. This also checks in a number of similar places. It replaces each
Assert(IsTransactionState()) that pertained to a conditional catalog read.
No back-patch for now, but a back-patch of commit 243e9b4 should back-patch
this, too. A back-patch could omit the src/test/regress changes, since back
branches won't gain new index columns.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/20250410191830.0e.nmisch@google.com
Discussion: https://postgr.es/m/10ec0bc3-5933-1189-6bb8-5dec4114558e@gmail.com
M src/backend/catalog/catalog.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/lmgr/lwlock.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/cache/catcache.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/mb/mbutils.c
M src/include/catalog/catalog.h
M src/include/storage/bufmgr.h
M src/include/storage/lwlock.h
M src/include/utils/relcache.h
M src/test/regress/expected/type_sanity.out
M src/test/regress/regress.c
M src/test/regress/sql/type_sanity.sql
pg_dump: Set private_date pointer to NULL in callback
commit : b669293e3432ee8fdcd44854a945837bb18eea5c
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 17 Apr 2025 12:58:00 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 17 Apr 2025 12:58:00 +0200
The end callback for ZStandard compression frees the private_data
but didn't set the pointer to NULL after freeing. This is not a
bug as the code is right now, since nothing is dereferencing the
pointer upon returning from the callback but it is good practice
to do.
Author: Alexander Kuznetsov <kuznetsovam@altlinux.org>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/efaee52b-9550-44ca-8633-ea86076b3283@altlinux.org
M src/bin/pg_dump/compress_zstd.c
pg_dump: Fix incorrect archive format shown in error message.
commit : e4b0f86e1fe38147b0bcf6e50862d0899a31201c
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 17 Apr 2025 09:52:47 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 17 Apr 2025 09:52:47 +0900
In pg_dump and pg_restore, _allocAH() calls _discoverArchiveFormat() to
determine the archive format when the input format is unknown one.
If the input or discovered format is unrecognized, it reports an error
including the archive format number.
If discovered format is unrecognized, its number should be shown in
the error message. But previously the error message mistakenly showed
the originally requested format number (i.e., unknown one) instead of
the discovered one, due to referencing the wrong variable in the error
message.
This commit corrects the issue by using the appropriate variable in
the error message.
This fix has no practical impact since _discoverArchiveFormat() never
returns an unrecognized format and that error mesasge is actually
never output. Therefore, while the issue exists in back branches,
it's not worth the trouble and buildfarm cycles to back-patch.
So this fix is applied only to the master branch.
Author: Mahendra Singh Thalor <mahi6run@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/CAKYtNAqu+N-Ab2Fq6wzNSOm_-0N-BMneanYNV1+6kFDXjva1Eg@mail.gmail.com
M src/bin/pg_dump/pg_backup_archiver.c
Another unintentional behavior change in commit e9931bfb75.
commit : 2e5353be2534aed99a714f99c8a193f85777e64a
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 16 Apr 2025 16:46:22 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 16 Apr 2025 16:46:22 -0700
Reported-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250412123430.8c.nmisch@google.com
M src/backend/regex/regc_pg_locale.c
Improve comment in regc_pg_locale.c.
commit : b107744ce73782a996aaa65dfd9d1bb3c543503d
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 16 Apr 2025 16:46:16 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 16 Apr 2025 16:46:16 -0700
Reported-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250412123430.8c.nmisch@google.com
M src/backend/regex/regc_pg_locale.c
Fixup various new-to-v18 usages of appendPQExpBuffer
commit : 3fae25cbb35aa885c411fb51f55a64bb80dc5844
author : David Rowley <drowley@postgresql.org>
date : Thu, 17 Apr 2025 11:37:55 +1200
committer: David Rowley <drowley@postgresql.org>
date : Thu, 17 Apr 2025 11:37:55 +1200
Use appendPQExpBufferStr when there are no parameters and
appendPQExpBufferChar when the string length is 1.
Author: David Rowley <drowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvoARMvPeXTTC0HnpARBHn-WgVstc8XFCyMGOzvgu_1HvQ@mail.gmail.com
M src/bin/pg_basebackup/pg_createsubscriber.c
M src/bin/pg_dump/pg_dump.c
M src/interfaces/libpq/fe-auth-oauth-curl.c
M src/test/modules/test_escape/test_escape.c
Improve comments for estimate_multivariate_ndistinct()
commit : f3281f9f9395099724b55ddc52991a92795c6068
author : David Rowley <drowley@postgresql.org>
date : Thu, 17 Apr 2025 11:03:24 +1200
committer: David Rowley <drowley@postgresql.org>
date : Thu, 17 Apr 2025 11:03:24 +1200
estimate_multivariate_ndistinct() is coded to assume the caller handles
passing it a list of GroupVarInfos with unique 'var' fields over the
entire list. 6bb6a62f3 added code which didn't ensure this and that
could result in estimate_multivariate_ndistinct() erroring out with:
ERROR: corrupt MVNDistinct entry
This occurred because estimate_multivariate_ndistinct() first searches
for a set of stats that match to at least two of the given GroupVarInfos
and then later assumes that the MVNDistinctItem.items array of the
best matching stats will have an entry for those two columns. If the
GroupVarInfos List contained a duplicate entry then the same column could
be matched to twice and that could trick the code into thinking we have
>= 2 columns matched in cases where only a single distinct column has been
matched. This could result in a failure to find the correct
MVNDistinctItem in the stats as the array containing those never
contains an item for single columns.
Here we make it more clear that the function needs a distinct set of
GroupVarInfos and also tidy up a few other comments to make things a bit
easier to follow.
Author: David Rowley <drowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvocZCUhM9W9mJ39d6oQz7ePKoqFnao_347mvC-A7QatcQ@mail.gmail.com
M src/backend/utils/adt/selfuncs.c
Sync declarations and definitions of two new tablecmds.c functions.
commit : ab3d8afc7f426cb562d96a673f620fc6fa404ebd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Apr 2025 17:59:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Apr 2025 17:59:08 -0400
Buildfarm member drongo complained because the definitions of these
functions used "const Oid foo" where the forward declarations just
had "Oid foo". (I'm a bit surprised that drongo seems to be the only
complainant.) I chose to fix this by removing the "consts" because
(a) I'm generally not a fan of using const that way, and (b) it was
a minority usage even within these two functions, let alone compared
to the rest of our code base.
Oversight in commit eec0040c4, so no need for back-patch.
M src/backend/commands/tablecmds.c
Elide not-null constraint checks on child tables during PK creation
commit : 11ff192b5bb707ba9ec13a0b6c7468874403abb3
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 16 Apr 2025 21:50:22 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 16 Apr 2025 21:50:22 +0200
We were unnecessarily acquiring AccessExclusiveLock on all child tables
when "ALTER TABLE ONLY sometab ADD PRIMARY KEY" was run on their parent
table, an oversight in commit 14e87ffa5c54. This caused deadlocks
during pg_restore of partitioned tables.
The reason to acquire the AEL was that we need to verify that child
tables have the involved columns already marked as not-null; but if the
parent table has an inheritable not-null constraint, then all children
must necessarily be in the correct state already, so we can skip the
check, which avoids acquiring the lock. Reorder the code so that it
works that way. This doesn't change things in the case where the
constraint doesn't exist, but that case is of lesser importance because
it doesn't occur during parallel pg_restore.
While at it, reword some errmsg() and add errhint() to similar cases in
related but not adjacent code.
Diagnosed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://postgr.es/m/67469c1c-38bc-7d94-918a-67033f5dd731@gmx.net
Discussion: https://postgr.es/m/2045026.1743801143@sss.pgh.pa.us
Discussion: https://postgr.es/m/1280408.1744650810@sss.pgh.pa.us
M src/backend/catalog/pg_constraint.c
M src/backend/commands/tablecmds.c
M src/test/regress/expected/constraints.out
M src/test/regress/expected/inherit.out
M src/test/regress/sql/constraints.sql
Update pg_config.h.in with libnuma changes
commit : 1fd3566ebc1fa626e0fa9bc0265542fd0638d39a
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 16 Apr 2025 20:16:57 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 16 Apr 2025 20:16:57 +0200
Add macros from autoheader which were accidentally omitted in
commit 65c298f61fc. There is no function change by this as no
code is currently using the missing macro.
Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/CF6D7D7F-E1C4-45BE-9019-0F4B4BC7C135@yesql.se
M src/include/pg_config.h.in
Fix pg_dump --clean with partitioned indexes.
commit : 1fc3403626d4aa99a81996435549e47c56c16897
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Apr 2025 13:31:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Apr 2025 13:31:44 -0400
We'd try to drop the partitions of a partitioned index separately,
which is disallowed by the backend, leading to an error during
restore. While the error is harmless, it causes problems if you
try to use --single-transaction mode.
Fortunately, there seems no need to do a DROP at all, since the
partition will go away silently when we drop either the parent index
or the partition's table. So just make the DROP conditional on not
being a partition.
Reported-by: jian he <jian.universality@gmail.com>
Author: jian he <jian.universality@gmail.com>
Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CACJufxF0QSdkjFKF4di-JGWN6CSdQYEAhGPmQJJCdkSZtd=oLg@mail.gmail.com
Backpatch-through: 13
M src/bin/pg_dump/pg_dump.c
pg_restore cleanups
commit : 40b9c27014d90d77c61e8f5c77ddeb84fa9a6b69
author : Andrew Dunstan <andrew@dunslane.net>
date : Wed, 16 Apr 2025 11:58:44 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Wed, 16 Apr 2025 11:58:44 -0400
. remove unnecessary oid_string list stuff
. use pg_get_line_buf() instead of open-coding it
. cleaner parsing of map.dat lines
Reverts 2b69afbe50d add new list type simple_oid_string_list to fe-utils/simple_list
Author: Álvaro Herrera <alvherre@kurilemu.de>
Author: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/202504141220.343fmoxfsbj4@alvherre.pgsql
M src/bin/pg_dump/pg_restore.c
M src/fe_utils/simple_list.c
M src/include/fe_utils/simple_list.h
M src/tools/pgindent/typedefs.list
Fix an incorrect check in get_memoize_path
commit : 3b35f9a4c5e0ed85c367b1397dac02fe7bd5b8a8
author : Richard Guo <rguo@postgresql.org>
date : Wed, 16 Apr 2025 10:55:44 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Wed, 16 Apr 2025 10:55:44 +0900
Memoize typically marks cache entries as complete after fully scanning
the inner side of a join. However, in the case of unique joins, we
skip to the next outer tuple as soon as the first matching inner tuple
is found, leaving no opportunity to scan the inner side to completion.
To work around that, we mark cache entries as complete after fetching
the first matching inner tuple in unique joins.
This approach is only safe when all of the join's restriction clauses
are parameterized; otherwise, there is no guarantee that reading just
one tuple from the inner side is sufficient.
Currently, we check for this by verifying that the number of clauses
in ppi_clauses is no less than the number of the join's restriction
clauses. However, this check isn't entirely reliable, as ppi_clauses
includes join clauses available from all outer rels, not just the
current outer rel. This means the check could pass even if a
restriction clause isn't parameterized, as long as another join
clause, which doesn't belong to the current join, is included in
ppi_clauses.
To fix this, we explicitly check whether each restriction clause of
the current join is present in ppi_clauses.
While we're here, remove the XXX comment from the modified code, as
it's not justified; in certain cases, it's not possible to move a join
clause to the inner side.
This is arguably a bugfix, but no backpatch given the lack of field
reports.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: wenhui qiu <qiuwenhuifx@gmail.com>
Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>
Discussion: https://postgr.es/m/CAMbWs4-8JPouj=wBDj4DhK-WO4+Xdx=A2jbjvvyyTBQneJ1=BQ@mail.gmail.com
M src/backend/optimizer/path/joinpath.c
doc: Fix typos in documentation
commit : 5ee476294c517ce0024e865e6f51c4ce34bc044e
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 15 Apr 2025 21:32:18 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 15 Apr 2025 21:32:18 +0200
This fixes a set of typos introduced during the v18 development
cycle.
Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/7038B4C5-2742-42B1-A8F0-0FFEAECF02A7@yesql.se
M doc/src/sgml/amcheck.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/system-views.sgml
Fix failure for generated column with a not-null domain constraint.
commit : 7c872849407730fa01e2c13b2d47483bc3ff6e7e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Apr 2025 12:08:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Apr 2025 12:08:34 -0400
If a GENERATED column is declared to have a domain data type where
the domain's constraints disallow null values, INSERT commands failed
because we built a targetlist that included coercing a null constant
to the domain's type. The failure occurred even when the generated
value would have been perfectly OK. This is adjacent to the issues
fixed in 0da39aa76, but we didn't notice for lack of testing a domain
with such a constraint.
We aren't going to use the result of the targetlist entry for the
generated column --- ExecComputeStoredGenerated will overwrite it.
So it's not really necessary that it have the exact datatype of
the generated column. This patch fixes the problem by changing
the targetlist entry to be a null Const of the domain's base type,
which should be sufficiently legal. (We do have to tweak
ExecCheckPlanOutput to accept the situation, though.)
This has been broken since we implemented generated columns.
However, this patch only applies easily as far back as v14, partly
because I (tgl) only carried 0da39aa76 back that far, but mostly
because v14 significantly refactored the handling of INSERT/UPDATE
targetlists. Given the lack of field complaints and the short
remaining support lifetime of v13, I judge the cost-benefit ratio
not good for devising a version that would work in v13.
Reported-by: jian he <jian.universality@gmail.com>
Author: jian he <jian.universality@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CACJufxG59tip2+9h=rEv-ykOFjt0cbsPVchhi0RTij8bABBA0Q@mail.gmail.com
Backpatch-through: 14
M src/backend/executor/nodeModifyTable.c
M src/backend/optimizer/prep/preptlist.c
M src/test/regress/expected/generated_stored.out
M src/test/regress/expected/generated_virtual.out
M src/test/regress/sql/generated_stored.sql
M src/test/regress/sql/generated_virtual.sql
doc: Fix missing whitespace in pg_restore documentation.
commit : f840f8ee304cb756638fd82254e3d00150a9b409
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 15 Apr 2025 23:15:06 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 15 Apr 2025 23:15:06 +0900
Previously, a space was missing between "<option>--exclude-schema</option>"
and "for" in the pg_restore documentation. This commit fixes the typo by
adding the missing whitespace.
Back-patch to v17 where the typo was added.
Author: Lele Gaifax <lele@metapensiero.it>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/87lds3ysm0.fsf@metapensiero.it
Backpatch-through: 17
M doc/src/sgml/ref/pg_restore.sgml
pg_combinebackup: Fix incorrect code documentation
commit : 7ae13170ba556e51a8e89f1b8a16535a857263ab
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 15 Apr 2025 15:27:08 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 15 Apr 2025 15:27:08 +0200
The code comment for parse_oid accidentally used the wrong parameter
when referring to the location of the last backup. Also, while there,
improve sentence wording by removing a superfluous word.
Backpatch to v17 where pg_combinebackup was addedd
Author: Amul Sul <sulamul@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/CAAJ_b95ecWgzcS4K3Dx0E_Yp-SLwK5JBasFgioKMSjhQLw9xvg@mail.gmail.com
Backpatch-through: 17
M src/bin/pg_combinebackup/pg_combinebackup.c
Fix incorrect format placeholders
commit : c55df7c6eae5a5c6f91cd029fb91913db7f2089c
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Apr 2025 08:56:33 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Apr 2025 08:56:33 +0200
BlockNumber is unsigned int. Fix for commit 14ffaece0fb.
M contrib/amcheck/verify_gin.c
Add more source files to pg_verifybackup/nls.mk
commit : 7cd171a5d2c4b99eceb2659ce8c88d9b1bc3bf1e
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Apr 2025 08:32:46 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Apr 2025 08:32:46 +0200
also related to commit 8dfd3129027
M src/bin/pg_verifybackup/nls.mk
Doc: use "an SQL" consistently rather than "a SQL"
commit : b51f86e49a7f119004c0ce5d0be89cdf98309141
author : David Rowley <drowley@postgresql.org>
date : Mon, 14 Apr 2025 11:55:18 +1200
committer: David Rowley <drowley@postgresql.org>
date : Mon, 14 Apr 2025 11:55:18 +1200
Per the precedent set by 04539e73f, adjust article prefixes for "SQL" to
use "an" consistently rather than "a", i.e., "an es-que-ell" rather than
"a sequel".
Both of these are new to v18. Also see b1b13d2b5, d866f0374 and
7bdd489d3.
M doc/src/sgml/ref/copy.sgml
Mark sslkeylogfile as Debug option
commit : 2970c75dd9824a0eeb1d1aeb5a42f59d5c9904d8
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sun, 13 Apr 2025 21:52:29 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sun, 13 Apr 2025 21:52:29 +0200
Mark the sslkeylogile option as "D" debug as this truly is a debug
option, and it will allow postgres_fdw et.al to filter it out as
well. Also update the display length to match that for an ssl key
as they are both filename based inputs.
Author: Daniel Gustafsson <daniel@yesql.se>
Reported-by: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/CAOYmi+=5GyBKpu7bU4D_xkAnYJTj=rMzGaUvHO99-DpNG_YKcw@mail.gmail.com
M src/interfaces/libpq/fe-connect.c
Make AIO error test more portable
commit : 64e193f5dda2075ecc6356625992e190a4347df6
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 13 Apr 2025 14:39:45 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 13 Apr 2025 14:39:45 -0400
Alpine Linux's C library (musl) spells one error message differently.
Reported-by: Wolfgang Walther
M src/test/modules/test_aio/t/001_aio.pl
Free memory properly in pg_restore.c
commit : f09088a01d3372fdfe5da12dd0b2e24989f0caa6
author : Andrew Dunstan <andrew@dunslane.net>
date : Sat, 12 Apr 2025 14:54:48 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sat, 12 Apr 2025 14:54:48 -0400
Thinko in commit 39729ec01d2. Mea maxima culpa.
Per Mahendra Singh Thalor <mahi6run@gmail.com>
M src/bin/pg_dump/pg_restore.c
Doc: do a little copy-editing on Index Storage Parameters list.
commit : 78637a8be20e49d8cbfdecf2d6db1f45b9695211
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Apr 2025 13:42:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Apr 2025 13:42:31 -0400
Add a paragraph break per suggestion from David G. Johnston.
Use a consistent voice for all the different parameter
descriptions, and fix a couple of grammatical issues.
Reported-by: Igor Korot <ikorot01@gmail.com>
Co-authored-by: "David G. Johnston" <david.g.johnston@gmail.com>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA+FnnTz=EW1VQRpWB9J+G-NSchrPFcw4nR7d0JqzEK9jWKB35A@mail.gmail.com
M doc/src/sgml/ref/create_index.sgml
Fix GIN's shimTriConsistentFn to not corrupt its input.
commit : e708ffe79df07fe26801869f391f4c227615dd42
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Apr 2025 12:27:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Apr 2025 12:27:46 -0400
Commit 0f21db36d made an assumption that GIN triConsistentFns
would not modify their input entryRes[] arrays. But in fact,
the "shim" triConsistentFn that we use for opclasses that don't
supply their own did exactly that, potentially leading to wrong
answers from a GIN index search. Through bad luck, none of the
test cases that we have for such opclasses exposed the bug.
One response to this could be that the assumption of consistency check
functions not modifying entryRes[] arrays is a bad one, but it still
seems reasonable to me. Notably, shimTriConsistentFn is itself
assuming that with respect to the underlying boolean consistentFn,
so it's sure being self-centered in supposing that it gets to do so.
Fortunately, it's quite simple to fix shimTriConsistentFn to restore
the entry-time state of entryRes[], so let's do that instead.
This issue doesn't affect any core GIN opclasses, since they all
supply their own triConsistentFns. It does affect contrib modules
btree_gin, hstore, and intarray.
Along the way, I (tgl) noticed that shimTriConsistentFn failed to
pick up on a "recheck" flag returned by its first call to the boolean
consistentFn. This may be only a latent problem, since it would be
unlikely for a consistentFn to set recheck for the all-false case
and not any other cases. (Indeed, none of our contrib modules do
that.) Nonetheless, it's formally wrong.
Reported-by: Vinod Sridharan <vsridh90@gmail.com>
Author: Vinod Sridharan <vsridh90@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAFMdLD7XzsXfi1+DpTqTgrD8XU0i2C99KuF=5VHLWjx4C1pkcg@mail.gmail.com
Backpatch-through: 13
M contrib/intarray/expected/_int.out
M contrib/intarray/sql/_int.sql
M src/backend/access/gin/ginlogic.c
Harmonize function parameter names for Postgres 18.
commit : a6cab6a78ee58356967a447fa2c6414732856986
author : Peter Geoghegan <pg@bowt.ie>
date : Sat, 12 Apr 2025 12:07:36 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Sat, 12 Apr 2025 12:07:36 -0400
Make sure that function declarations use names that exactly match the
corresponding names from function definitions in a few places. These
inconsistencies were all introduced during Postgres 18 development.
This commit was written with help from clang-tidy, by mechanically
applying the same rules as similar clean-up commits (the earliest such
commit was commit 035ce1fe).
M contrib/pg_overexplain/pg_overexplain.c
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/access/gin/gininsert.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execPartition.c
M src/backend/nodes/queryjumblefuncs.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/smgr/md.c
M src/backend/utils/adt/acl.c
M src/backend/utils/adt/pg_locale_builtin.c
M src/backend/utils/adt/pg_locale_icu.c
M src/backend/utils/mmgr/mcxt.c
M src/bin/scripts/reindexdb.c
M src/common/unicode_case.c
M src/include/commands/explain.h
M src/include/libpq/oauth.h
M src/include/optimizer/cost.h
M src/include/optimizer/planmain.h
M src/include/port/pg_crc32c.h
M src/include/replication/slot.h
M src/include/storage/lwlock.h
M src/include/storage/md.h
M src/include/utils/pg_locale.h
M src/include/utils/tuplestore.h
M src/test/modules/oauth_validator/fail_validator.c
M src/test/modules/oauth_validator/magic_validator.c
M src/test/modules/oauth_validator/validator.c
Fix instability with WAL fsync test in stats.sql
commit : fdb69dd582427e4976897d544867e65a0e767002
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 12 Apr 2025 13:09:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 12 Apr 2025 13:09:48 +0900
A backend using wal_sync_method set to "open_sync" or "open_datasync"
would fail the test checking the WAL sync data in pg_stat_io. These
modes guarantee that a sync is done when WAL is written to disk, and the
data checked by the test is not incremented in this case,
issue_xlog_fsync() doing nothing.
Oversight in commit a051e71e28a1.
Author: Sami Imseih <samimseih@gmail.com>
Discussion: https://postgr.es/m/CAA5RZ0uxwg3xAi4nvdBMJ-zJQEeyg+RotuU+ebM2F6CKmnvaYA@mail.gmail.com
M src/test/regress/expected/stats.out
M src/test/regress/sql/stats.sql
Fix recently introduced typos
commit : 847bbb21f8c4eb0e2b47417684ad2ba9255c9e80
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 11 Apr 2025 22:17:12 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 11 Apr 2025 22:17:12 +0200
This fixes typos in docs and comments introduced during the v18
development cycle, to keep them from ending up in backbranches.
Author: Jacob Brazeal <jacob.brazeal@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CA+COZaCgGua25f2hSrjrDLJcJJAHkwoKgTTqUy-wyL1=64JNjw@mail.gmail.com
M doc/src/sgml/indexam.sgml
M src/backend/statistics/attribute_stats.c
M src/backend/storage/lmgr/lwlock.c
M src/backend/utils/activity/pgstat_backend.c
M src/backend/utils/cache/inval.c
M src/tools/testwrap
Add missing space in pg_restore documentation.
commit : 5822bf21d50eb3a4938066951727294e3d33ca6c
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 11 Apr 2025 10:05:32 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 11 Apr 2025 10:05:32 -0500
Oversight in commit 1495eff7bd.
M doc/src/sgml/ref/pg_restore.sgml
Add missing source file to pg_verifybackup/nls.mk
commit : 914ea1c93c0e446a0cd174497fd6a22fd6071c5e
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 11 Apr 2025 10:53:36 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 11 Apr 2025 10:53:36 +0200
added by commit 8dfd3129027
M src/bin/pg_verifybackup/nls.mk
Add missing source file to pg_dump/nls.mk
commit : b63cbacb86cf7fa75ca872633ce55fd327b5c7e7
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 11 Apr 2025 10:28:59 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 11 Apr 2025 10:28:59 +0200
added by commit c1da7281060
M src/bin/pg_dump/nls.mk
Add missing source file to pg_upgrade/nls.mk
commit : 9e0e1cfc3ef2f683be514b9ef7a7567404fc327f
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 11 Apr 2025 10:26:51 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 11 Apr 2025 10:26:51 +0200
added by commit 40e2e5e92b7
M src/bin/pg_upgrade/nls.mk
Add missing PGDLLIMPORT markings
commit : 7d430a57281f0050b66e5eb9eb587193e4cc54ed
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 11 Apr 2025 08:56:39 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 11 Apr 2025 08:56:39 +0200
Discussion: https://www.postgresql.org/message-id/flat/25095db5-b595-4b85-9100-d358907c25b5%40eisentraut.org
M src/include/libpq/oauth.h
M src/include/postmaster/postmaster.h
M src/include/storage/bufmgr.h
M src/include/storage/md.h
M src/include/storage/smgr.h
Fix race with synchronous_standby_names at startup
commit : 2e57790836c636d579871b13de13850e013f511c
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 11 Apr 2025 10:00:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 11 Apr 2025 10:00:21 +0900
synchronous_standby_names cannot be reloaded safely by backends, and the
checkpointer is in charge of updating a state in shared memory if the
GUC is enabled in WalSndCtl, to let the backends know if they should
wait or not for a given LSN. This provides a strict control on the
timing of the waiting queues if the GUC is enabled or disabled, then
reloaded. The checkpointer is also in charge of waking up the backends
that could be waiting for a LSN when the GUC is disabled.
This logic had a race condition at startup, where it would be possible
for backends to not wait for a LSN even if synchronous_standby_names is
enabled. This would cause visibility issues with transactions that we
should be waiting for but they were not. The problem lasts until the
checkpointer does its initial update of the shared memory state when it
loads synchronous_standby_names.
In order to take care of this problem, the shared memory state in
WalSndCtl is extended to detect if it has been initialized by the
checkpointer, and not only check if synchronous_standby_names is
defined. In WalSndCtlData, sync_standbys_defined is renamed to
sync_standbys_status, a bits8 able to know about two states:
- If the shared memory state has been initialized. This flag is set by
the checkpointer at startup once, and never removed.
- If synchronous_standby_names is known as defined in the shared memory
state. This is the same as the previous sync_standbys_defined in
WalSndCtl.
This method gives a way for backends to decide what they should do until
the shared memory area is initialized, and they now ultimately fall back
to a check on the GUC value in this case, which is the best thing that
can be done.
Fortunately, SyncRepUpdateSyncStandbysDefined() is called immediately by
the checkpointer when this process starts, so the window is very narrow.
It is possible to enlarge the problematic window by making the
checkpointer wait at the beginning of SyncRepUpdateSyncStandbysDefined()
with a hardcoded sleep for example, and doing so has showed that a 2PC
visibility test is indeed failing. On machines slow enough, this bug
would cause spurious failures.
In 17~, we have looked at the possibility of adding an injection point
to have a reproducible test, but as the problematic window happens at
early startup, we would need to invent a way to make an injection point
optionally persistent across restarts when attached, something that
would be fine for this case as it would involve the checkpointer. This
issue is quite old, and can be reproduced on all the stable branches.
Author: Melnikov Maksim <m.melnikov@postgrespro.ru>
Co-authored-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/163fcbec-900b-4b07-beaa-d2ead8634bec@postgrespro.ru
Backpatch-through: 13
M src/backend/replication/syncrep.c
M src/backend/replication/walsender.c
M src/include/replication/walsender_private.h
Add code comment explaining ins_since_vacuum and aborted inserts
commit : 530050d8d2850d0453bb56e2bfa7cae216ee8a18
author : David Rowley <drowley@postgresql.org>
date : Fri, 11 Apr 2025 11:36:21 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 11 Apr 2025 11:36:21 +1200
Sami complained that there's a discrepancy between n_mod_since_analyze
and n_ins_since_vacuum, as the former only accounts for committed changes
and the latter tracks committed and aborted inserts. Nobody seemed
overly concerned that this would cause any concerning issues. The
repercussions, from what I can tell, are limited to causing an
autovacuum to trigger for inserts sooner than it otherwise might. For
typical ratios of commits to aborts, it's unlikely to ever be noticed.
Fixing things to make it so n_ins_since_vacuum only displays committed
inserts would require an additional field in PgStat_TableCounts, which
does not quite seem worthwhile at this stage. This commit just adds a
comment with some details to mention that we know about it, which will
hopefully prevent repeat discussions.
Reported-by: Sami Imseih <samimseih@gmail.com>
Author: David Rowley <drowleyml@gmail.com>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Discussion: https://postgr.es/m/CAApHDvpgV3a-R2EGmPOh0L-x3pHbZpM3y4dySWfy+UqUazwDQA@mail.gmail.com
M src/backend/utils/activity/pgstat_relation.c
Fix fat fingering in 22cb6d28950
commit : 39729ec01d25dbe12e0dd8322c68f242650235c9
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 10 Apr 2025 19:05:54 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 10 Apr 2025 19:05:54 -0400
Per Rainier Vilela
M src/bin/pg_dump/pg_restore.c
Improve various new-to-v18 appendStringInfo calls
commit : 928394b664bc4afef2fe6dc69a70e4074886e065
author : David Rowley <drowley@postgresql.org>
date : Fri, 11 Apr 2025 10:07:22 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 11 Apr 2025 10:07:22 +1200
Similar to 8461424fd, here we adjust a few new locations which were not
using the most suitable appendStringInfo* function for the intended
purpose.
Author: David Rowley <drowleyml@gmail.com
Discussion: https://postgr.es/m/CAApHDvqJnNjueb=Eoj8K+8n0g7nj_AcPWSiCj5RNV4fDejAfqA@mail.gmail.com
M contrib/dblink/dblink.c
M contrib/pg_overexplain/pg_overexplain.c
M src/backend/commands/explain.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/conflict.c
M src/backend/replication/logical/relation.c
M src/backend/replication/logical/tablesync.c
M src/backend/utils/adt/ri_triggers.c
M src/bin/pg_dump/pg_restore.c
Rename global variable backing DSA area
commit : 55ef7abf88c07e9f716846e645e1628c667c54a7
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 10 Apr 2025 22:40:27 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 10 Apr 2025 22:40:27 +0200
The global variable backing the DSA area for Memory Context stats
reporting had a too generic name, rename to be more descriptive.
Independently reported by Peter and Laurenz.
Author: Daniel Gustafsson <daniel@yesql.se>
Reported-by: Peter Eisentraut <peter@eisentraut.org>
Reported-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/d51172bd4e7f4b07a18a0288ca1b1c28a71a5f6a.camel@cybertec.at
Discussion: https://postgr.es/m/25095db5-b595-4b85-9100-d358907c25b5@eisentraut.org
M src/backend/utils/adt/mcxtfuncs.c
M src/backend/utils/mmgr/mcxt.c
M src/include/utils/memutils.h
Fix memory leak in pg_restore.c
commit : 22cb6d289500ef22163a9d7cf2afa496f41b4d4c
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 10 Apr 2025 14:54:39 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 10 Apr 2025 14:54:39 -0400
Oversight in 1495eff7bdb
Author: Ranier Vilela <ranier.vf@gmail.com>
M src/bin/pg_dump/pg_restore.c
Doc: remove long-obsolete advice about generated constraint names.
commit : d89335eea67c1cb55b03d5235320cc107287b59a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Apr 2025 14:49:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Apr 2025 14:49:10 -0400
It's been twenty years since we generated constraint names that
look like "$N". So this advice about double-quoting such names
is well past its sell-by date, and now it merely seems confusing.
Reported-by: Yaroslav Saburov <y.saburov@gmail.com>
Author: "David G. Johnston" <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/174393459040.678.17810152410419444783@wrigleys.postgresql.org
Backpatch-through: 13
M doc/src/sgml/ddl.sgml
Remove useless check for negative result of ip_addrsize().
commit : f27eb0325b7b2cff3b880fa669913693849521dc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Apr 2025 14:18:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Apr 2025 14:18:07 -0400
By inspection, ip_addrsize() can't return a negative result.
(If it could, we'd have way bigger problems elsewhere.)
So delete useless check in network_send(). Most C compilers
are probably perfectly capable of removing this code by
themselves, but it's confusing/misleading.
Bug: #18889
Reported-by: Daniel Elishakov <dan-eli@mail.ru>
Discussion: https://postgr.es/m/18889-73d4f19e953a629e@postgresql.org
M src/backend/utils/adt/network.c
Further cleanup for directory creation on pg_dump/pg_dumpall
commit : 4170298b6ecff7ce697b81e13d9a81e3b825798c
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 10 Apr 2025 12:11:36 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 10 Apr 2025 12:11:36 -0400
Instead of two separate (and different) implementations, refactor to use
a single common routine.
Along the way, remove use of a hardcoded file permissions constant in
favor of the common project setting for directory creation.
Author: Mahendra Singh Thalor <mahi6run@gmail.com>
Discussion: https://postgr.es/m/CAKYtNApihL8X1h7XO-zOjznc8Ca66Aevgvhc9zOTh6DBh2iaeA@mail.gmail.com
M src/bin/pg_dump/dumputils.c
M src/bin/pg_dump/dumputils.h
M src/bin/pg_dump/pg_backup_directory.c
M src/bin/pg_dump/pg_dumpall.c
Fix data loss in logical replication.
commit : 4909b38af034fa4d2c67c5c71fd8509f870c1695
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 10 Apr 2025 13:14:40 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 10 Apr 2025 13:14:40 +0530
Data loss can happen when the DDLs like ALTER PUBLICATION ... ADD TABLE ...
or ALTER TYPE ... that don't take a strong lock on table happens
concurrently to DMLs on the tables involved in the DDL. This happens
because logical decoding doesn't distribute invalidations to concurrent
transactions and those transactions use stale cache data to decode the
changes. The problem becomes bigger because we keep using the stale cache
even after those in-progress transactions are finished and skip the
changes required to be sent to the client.
This commit fixes the issue by distributing invalidation messages from
catalog-modifying transactions to all concurrent in-progress transactions.
This allows the necessary rebuild of the catalog cache when decoding new
changes after concurrent DDL.
We observed performance regression primarily during frequent execution of
*publication DDL* statements that modify the published tables. The
regression is minor or nearly nonexistent for DDLs that do not affect the
published tables or occur infrequently, making this a worthwhile cost to
resolve a longstanding data loss issue.
An alternative approach considered was to take a strong lock on each
affected table during publication modification. However, this would only
address issues related to publication DDLs (but not the ALTER TYPE ...)
and require locking every relation in the database for publications
created as FOR ALL TABLES, which is impractical.
The bug exists in all supported branches, but we are backpatching till 14.
The fix for 13 requires somewhat bigger changes than this fix, so the fix
for that branch is still under discussion.
Reported-by: hubert depesz lubaczewski <depesz@depesz.com>
Reported-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Tested-by: Benoit Lobréau <benoit.lobreau@dalibo.com>
Backpatch-through: 14
Discussion: https://postgr.es/m/de52b282-1166-1180-45a2-8d8917ca74c6@enterprisedb.com
Discussion: https://postgr.es/m/CAD21AoAenVqiMjpN-PvGHL1N9DWnHSq673bfgr6phmBUzx=kLQ@mail.gmail.com
M contrib/test_decoding/Makefile
A contrib/test_decoding/expected/invalidation_distrubution.out
M contrib/test_decoding/meson.build
A contrib/test_decoding/specs/invalidation_distrubution.spec
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/include/replication/reorderbuffer.h
Fix incorrect format placeholders
commit : 9ad19295e934e261bb22463d71af3f69699053ea
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 10 Apr 2025 08:04:35 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 10 Apr 2025 08:04:35 +0200
for commits 8f427187db7, 6ee3b91bad2
M src/bin/pg_dump/pg_dump.c
Update wording in optimizer/README for EquivalenceClasses
commit : d7c04db27aeba1718d8059eae9a6e6a5eedc01b8
author : David Rowley <drowley@postgresql.org>
date : Thu, 10 Apr 2025 17:33:58 +1200
committer: David Rowley <drowley@postgresql.org>
date : Thu, 10 Apr 2025 17:33:58 +1200
d69d45a5a changed how em_is_child members are stored in
EquivalenceClasses. Children are no longer stored in the ec_members
list. optimizer/README mentioned that most operations "should ignore
child members", but that felt a little untrue now since child members
are now stored in a separate place, they simply won't be found by the
normal means of looking (a foreach loop over ec_members), and if you don't
find them, there's technically no need to "ignore" them.
Here we tweak the wording slightly to reflect the new storage location
for child members.
Reported-by: Amit Langote <amitlangote09@gmail.com>
Author: Amit Langote <amitlangote09@gmail.com>
Author: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CA+HiwqE8v=EuAP_3F_A2xn8zWx+nG_etW_Fe_DvKO-Fkx=+DdQ@mail.gmail.com
M src/backend/optimizer/README
Cosmetic fixes for pg_createsubscriber's -all option.
commit : d438515c29c3cc539c79292c040335804183ca3e
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 10 Apr 2025 10:23:01 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 10 Apr 2025 10:23:01 +0530
Author: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CAHut+PsmSCQ-ENSDQ0YOUcsgzT=GG-E9jyXBvxd51A_dMXH5XA@mail.gmail.com
M doc/src/sgml/ref/pg_createsubscriber.sgml
M src/bin/pg_basebackup/pg_createsubscriber.c
ci: Check for missing dependencies in meson builds
commit : d15acc915de196df799ba000dd6842bbd4a60339
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 9 Apr 2025 19:34:27 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 9 Apr 2025 19:34:27 +0200
Extends the Linux and Windows meson builds with a check for missing
dependencies by running
ninja -t missingdeps
after the build. This highlights unindended dependencies.
Reviewed-by: Andres Freund <andres@anarazel.de>
https://postgr.es/m/CALdSSPi5fj0a7UG7Fmw2cUD1uWuckU_e8dJ+6x-bJEokcSXzqA@mail.gmail.com
M .cirrus.tasks.yml
Cleanup of pg_numa.c
commit : 3887d0cfeb3315bc4c6219bd2eb90e89cf487fa8
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 9 Apr 2025 19:32:17 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 9 Apr 2025 19:32:17 +0200
This moves/renames some of the functions defined in pg_numa.c:
* pg_numa_get_pagesize() is renamed to pg_get_shmem_pagesize(), and
moved to src/backend/storage/ipc/shmem.c. The new name better reflects
that the page size is not related to NUMA, and it's specifically about
the page size used for the main shared memory segment.
* move pg_numa_available() to src/backend/storage/ipc/shmem.c, i.e. into
the backend (which more appropriate for functions callable from SQL).
While at it, improve the comment to explain what page size it returns.
* remove unnecessary includes from src/port/pg_numa.c, adding
unnecessary dependencies (src/port should be suitable for frontent).
These were either leftovers or unnecessary thanks to the other changes
in this commit.
This eliminates unnecessary dependencies on backend symbols, which we
don't want in src/port.
Reported-by: Kirill Reshke <reshkekirill@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
https://postgr.es/m/CALdSSPi5fj0a7UG7Fmw2cUD1uWuckU_e8dJ+6x-bJEokcSXzqA@mail.gmail.com
M contrib/pg_buffercache/pg_buffercache_pages.c
M src/backend/storage/ipc/shmem.c
M src/include/port/pg_numa.h
M src/include/storage/shmem.h
M src/port/pg_numa.c
pg_upgrade: Mention that we preserve database OIDs in a comment.
commit : e2665efd0f4d2ae90d3245f9425d379d75c66645
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 9 Apr 2025 14:27:08 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 9 Apr 2025 14:27:08 -0500
Oversight in commit aa01051418.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/4055696.1744134682%40sss.pgh.pa.us
M src/bin/pg_upgrade/pg_upgrade.c
Fix performance issue in deadlock-parallel isolation test.
commit : 837cc73af29cd9c63515a6f2a36f54dd703a3a3f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Apr 2025 12:28:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Apr 2025 12:28:34 -0400
With debug_discard_caches = 1, the runtime of this test script
increased by about a factor of 10 after commit 0dca5d68d. That's
causing some of our buildfarm animals to fail with a timeout.
The reason for the increased time is that now we are re-planning
some intentionally-non-inlineable SQL functions on every execution,
where the previous coding held onto the original plans throughout
the outer query. The previous behavior was arguably quite buggy,
so I don't think 0dca5d68d deserves blame here. But we would
like this test script to not take so long.
To fix, instead of forcing a "parallel safe" label via a
non-inlineable SQL function, apply it directly to the advisory-lock
functions by making internal-language aliases for them. A small
problem is that the advisory-lock functions return void but this
test would really like them to return integer 1. I cheated here by
declaring the aliases as returning "int". That's perhaps undue
familiarity with the implementation of PG_RETURN_VOID(), but that
hasn't changed in twenty years and is unlikely to do so in the next
twenty. That gets us an integer 0 result, and then an inline-able
wrapper to convert that to an integer 1 allows the rest of the script
to remain unchanged.
For me, this reduces the runtime with debug_discard_caches = 1
by about 100x, making the test comfortably faster than before
instead of slower.
Discussion: https://postgr.es/m/136163.1744179562@sss.pgh.pa.us
M src/test/isolation/specs/deadlock-parallel.spec
Fix test races between syscache-update-pruned.spec and autovacuum.
commit : 5bbc596391aeaf9042536cc25ce71f1e4fce6db8
author : Noah Misch <noah@leadboat.com>
date : Wed, 9 Apr 2025 07:23:39 -0700
committer: Noah Misch <noah@leadboat.com>
date : Wed, 9 Apr 2025 07:23:39 -0700
This spec fails ~3% of my Valgrind runs, and the spec has failed on Valgrind
buildfarm member skink at a similar rate. Two problems contributed to that:
- A competing buffer pin triggered VACUUM's lazy_scan_noprune() path, causing
"tuples missed: 1 dead from 1 pages not removed due to cleanup lock
contention". FREEZE fixes that.
- The spec ran lazy VACUUM immediately after VACUUM FULL. The spec implicitly
assumed lazy VACUUM prunes the one tuple that VACUUM FULL made dead. First
wait for old snapshots, making that assumption reliable.
This also adds two forms of defense in depth:
- Wait for snapshots using shared catalog pruning rules (VISHORIZON_SHARED).
This avoids the removable cutoff moving backward when an XID-bearing
autoanalyze process runs in another database. That may never happen in this
test, but it's cheap insurance.
- Use lazy VACUUM option DISABLE_PAGE_SKIPPING. Commit
c2dc1a79767a0f947e1145f82eb65dfe4360d25f did this for a related requirement
in other tests, but I suspect FREEZE is necessary and sufficient in all
these tests.
Back-patch to v17, where the test first appeared.
Reported-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/sv3taq4e6ea4qckimien3nxp3sz4b6cw6sfcy4nhwl52zpur4g@h6i6tohxmizu
Backpatch-through: 17
M src/test/modules/injection_points/expected/syscache-update-pruned.out
M src/test/modules/injection_points/expected/syscache-update-pruned_1.out
M src/test/modules/injection_points/regress_injection.c
M src/test/modules/injection_points/specs/syscache-update-pruned.spec
Update config.guess and config.sub
commit : 306dd6e727bf7a4e76f02cb31c2e22ac3b0deca3
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 9 Apr 2025 12:12:57 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 9 Apr 2025 12:12:57 +0200
M config/config.guess
M config/config.sub
Fix a few oversights in the longer cancel keys patch
commit : 0f1433f05356e8de7c4125e5192a54355e12359d
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 9 Apr 2025 13:11:42 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 9 Apr 2025 13:11:42 +0300
Change MyCancelKeyLength's type from uint8 to int. While it always
fits in a uint8, plain int is less surprising, as there's no
particular reason for it to be uint8.
Fix one ProcSignalInit caller that passed 'false' instead of NULL for
the pointer argument.
Author: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/61be9e31-7b7d-49d5-bc11-721800d89d64@eisentraut.org
M src/backend/postmaster/auxprocess.c
M src/backend/utils/init/globals.c
M src/include/miscadmin.h
Perform missed catversion bump
commit : ef366b7d7e3e66ca55ce80c9dfa3254ada727754
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 9 Apr 2025 09:29:12 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 9 Apr 2025 09:29:12 +0200
Commit c57971034e69ca renamed an argument for a function but missed
to bump the catversion to reflect this.
Reported-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvqOega=dPtu3h2C5fJWJEuaGCMDib_sVfhKQqgUNJVmFA@mail.gmail.com
M src/include/catalog/catversion.h
Doc: note that two examples in optimizer/README are oversimplified.
commit : dd496eedeaac7cc9023e8fbca709475868ed5d9a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Apr 2025 23:03:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Apr 2025 23:03:33 -0400
These examples fail to account for join clauses generated by
EquivalenceClasses, but since we haven't mentioned EquivalenceClasses
yet it seems like it'd just add confusion to make them fully accurate.
Instead, parenthetically note that they're oversimplified.
Reported-by: Zeyuan Hu <ferrishu3886@gmail.com>
Co-authored-by: David Rowley <dgrowleyml@gmail.com>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CACvHWmYFo+60yMqKJajDDvKN5EM41YHrCT3oxukwXmGAqpWvyw@mail.gmail.com
M src/backend/optimizer/README
Adjust AdjustUpgrade.pm for commit b1720fe63.
commit : b65b9da568fd93f8fe3b41f4074f225eead104d2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Apr 2025 20:21:03 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Apr 2025 20:21:03 -0400
Need to delete the functions we no longer have available from
the dumps to be reloaded from old versions.
Per buildfarm.
M src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
Move contrib/spi testing from core regression tests to contrib/spi.
commit : b1720fe63f344adeb8a75b22e8f31b127c814f35
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Apr 2025 19:12:03 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Apr 2025 19:12:03 -0400
It's weird to have the core regression tests depending on contrib
code, and coverage testing shows that those test queries add nothing
to the core-code coverage of the core tests. So pull those test bits
out and put them into ordinary test scripts inside contrib/spi/,
making that more like other contrib modules.
Aside from being structurally nicer, anything we can take out of the
core tests (which are executed multiple times per check-world run)
and put into tests executed only once should be a win. It doesn't
look like this change will buy a whole lot of milliseconds, but a
cycle saved is a cycle earned.
Also, there is some discussion around possibly removing refint and/or
autoinc altogether. I don't know if that will happen, but we'd
certainly need to decouple them from the core tests to do so.
The tests for autoinc were quite intertwined with the undocumented
"ttdummy" trigger in regress.c. That made the tests very hard to
understand and contributed nothing to autoinc's testing either.
So I just deleted ttdummy and rewrote the autoinc tests without it.
I realized while doing this that the description of autoinc in
the SGML docs is not a great description of what the function
actually does, so the patch includes some updates to those docs.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/3872677.1744077559@sss.pgh.pa.us
A contrib/spi/.gitignore
M contrib/spi/Makefile
A contrib/spi/expected/autoinc.out
A contrib/spi/expected/refint.out
M contrib/spi/meson.build
A contrib/spi/sql/autoinc.sql
A contrib/spi/sql/refint.sql
M doc/src/sgml/contrib-spi.sgml
M src/test/regress/GNUmakefile
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/test_setup.out
M src/test/regress/expected/triggers.out
M src/test/regress/meson.build
M src/test/regress/regress.c
M src/test/regress/sql/alter_table.sql
M src/test/regress/sql/test_setup.sql
M src/test/regress/sql/triggers.sql
Rename argument in pg_get_process_memory_contexts().
commit : c57971034e69ca5705ac2be893a80ea82aca978b
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 8 Apr 2025 23:09:13 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 8 Apr 2025 23:09:13 +0200
During development the third argument to pg_get_process_memory_contexts
was a retry count, but it was changed to a timeout instead. The param
name was accidentally left in pg_proc.dat though. Fix by renaming to
the correct parameter name.
Author: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/3eb40b3e-45c7-426a-b7f8-81f7d05a9b53@oss.nttdata.com
M src/include/catalog/pg_proc.dat
Fix incorrect format placeholder
commit : 8969194b73dbc9642a0302225f686f9febd95c90
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 8 Apr 2025 19:12:03 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 8 Apr 2025 19:12:03 +0200
for commit 749a9e20c97
M contrib/pgcrypto/crypt-gensalt.c
Prevent 006_transfer_modes.pl from leaving files behind.
commit : b0a4c3e88b8b527679216dca5e2e9cbd49514e99
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 8 Apr 2025 10:57:31 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 8 Apr 2025 10:57:31 -0500
This test was leaving files like delete_old_cluster.{sh,bat} in the
source directory for VPATH and meson builds. To fix, change the
directory to tmp_check before running the test, as was done in
commits 15b6d21553, 8af917be6b, and c462b054ba.
Oversight in commit af0d4901c1.
Reported-by: Andrew Dunstan <andrew@dunslane.net> (on Discord)
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/Z_RHkG770w3SE0yU%40nathan
M src/bin/pg_upgrade/t/006_transfer_modes.pl
ci: Add MBUILD_TARGET for NetBSD and OpenBSD
commit : 88edd661c881562869e8a02f135c897cdd55103f
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 8 Apr 2025 15:28:29 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 8 Apr 2025 15:28:29 +0200
Commit b2bdb972c0 added MBUILD_TARGET to ensure that meson builds
the tests before running them, this adds MBUILD_TARGET to OpenBSD
and NetBSD builds as well where it was missing.
No backpatching since OpenBSD and NetBSD support does not exist
in the backbranch CI.
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAN55FZ2LNnRrtL+cpSdEg44fQcLPq_GjJjfNa0vz+xqEdq=ZHw@mail.gmail.com
M .cirrus.tasks.yml
pg_buffercache: Change page_num type to bigint
commit : 91f1fe90c7d42641201bd2c474bd86d703b5e830
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 8 Apr 2025 11:52:02 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 8 Apr 2025 11:52:02 +0200
The page_num was defined as integer, which should be sufficient for the
near future (with 4K pages it's 8TB). But it's virtually free to return
bigint, and get a wider range. This was agreed on the thread, but I
forgot to tweak this in ba2a3c2302f1.
While at it, make the data types in CREATE VIEW a bit more consistent.
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.co
M contrib/pg_buffercache/pg_buffercache–1.5–1.6.sql
M contrib/pg_buffercache/pg_buffercache_pages.c
M doc/src/sgml/pgbuffercache.sgml
doc: Correct pg_shmem_allocations_numa.size data type
commit : b8a6078ca8f446a6271b26101de75ee48e288a3a
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 8 Apr 2025 11:45:01 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 8 Apr 2025 11:45:01 +0200
The code in pg_get_shmem_allocations_numa() returned 'size' as int64,
but the docs said int32.
Report and fix by Noriyoshi Shinoda.
Reported-by: Noriyoshi Shinoda <noriyoshi.shinoda@hpe.com>
Discussion: https://postgr.es/m/DM4PR84MB1734308EB741A6ECFF040C27EEAA2@DM4PR84MB1734.NAMPRD84.PROD.OUTLOOK.COM
M doc/src/sgml/system-views.sgml
Fix uninitialized index information access during apply.
commit : 12eece5fd54c2aa3dbdefb6de7f18566f5c00357
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 8 Apr 2025 15:35:42 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 8 Apr 2025 15:35:42 +0530
The issue happens when building conflict information during apply of
INSERT or UPDATE operations that violate unique constraints on leaf
partitions.
The problem was introduced in commit 9ff68679b5, which removed the
redundant calls to ExecOpenIndices/ExecCloseIndices. The previous code was
relying on the redundant ExecOpenIndices call in
apply_handle_tuple_routing() to build the index information required for
unique key conflict detection.
The fix is to delay building the index information until a conflict is
detected instead of relying on ExecOpenIndices to do the same. The
additional benefit of this approach is that it avoids building index
information when there is no conflict.
Author: Hou Zhijie <houzj.fnst@fujitsu.com>
Reviewed-by:Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/TYAPR01MB57244ADA33DDA57119B9D26494A62@TYAPR01MB5724.jpnprd01.prod.outlook.com
M src/backend/executor/execIndexing.c
M src/backend/executor/execReplication.c
M src/backend/replication/logical/worker.c
M src/test/subscription/t/035_conflicts.pl
Fix typo in docs.
commit : 7ea21f4ee2721be4239657e74c2ae8f88fb5a3ef
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 8 Apr 2025 22:02:24 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 8 Apr 2025 22:02:24 +1200
Typo in previous commit.
M doc/src/sgml/ref/alter_database.sgml
Introduce file_copy_method setting.
commit : f78ca6f3ebbbff8c675c34b8ee61047223073866
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 8 Apr 2025 20:52:47 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 8 Apr 2025 20:52:47 +1200
It can be set to either COPY (the default) or CLONE if the system
supports it. CLONE causes callers of copydir(), currently CREATE
DATABASE ... STRATEGY=FILE_COPY and ALTER DATABASE ... SET TABLESPACE =
..., to use copy_file_range (Linux, FreeBSD) or copyfile (macOS) to copy
files instead of a read-write loop over the contents.
CLONE gives the kernel the opportunity to share block ranges on
copy-on-write file systems and push copying down to storage on others,
depending on configuration. On some systems CLONE can be used to clone
large databases quickly with CREATE DATABASE ... TEMPLATE=source
STRATEGY=FILE_COPY.
Other operating systems could be supported; patches welcome.
Co-authored-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGLM%2Bt%2BSwBU-cHeMUXJCOgBxSHLGZutV5zCwY4qrCcE02w%40mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/alter_database.sgml
M doc/src/sgml/ref/create_database.sgml
M src/backend/storage/file/copydir.c
M src/backend/utils/activity/wait_event_names.txt
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/storage/copydir.h
M src/tools/pgindent/typedefs.list
Add function to get memory context stats for processes
commit : 042a66291b04f473cbc72f95f07438abd75ae3a9
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 8 Apr 2025 11:06:56 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 8 Apr 2025 11:06:56 +0200
This adds a function for retrieving memory context statistics
and information from backends as well as auxiliary processes.
The intended usecase is cluster debugging when under memory
pressure or unanticipated memory usage characteristics.
When calling the function it sends a signal to the specified
process to submit statistics regarding its memory contexts
into dynamic shared memory. Each memory context is returned
in detail, followed by a cumulative total in case the number
of contexts exceed the max allocated amount of shared memory.
Each process is limited to use at most 1Mb memory for this.
A summary can also be explicitly requested by the user, this
will return the TopMemoryContext and a cumulative total of
all lower contexts.
In order to not block on busy processes the caller specifies
the number of seconds during which to retry before timing out.
In the case where no statistics are published within the set
timeout, the last known statistics are returned, or NULL if
no previously published statistics exist. This allows dash-
board type queries to continually publish even if the target
process is temporarily congested. Context records contain a
timestamp to indicate when they were submitted.
Author: Rahila Syed <rahilasyed90@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Discussion: https://postgr.es/m/CAH2L28v8mc9HDt8QoSJ8TRmKau_8FM_HKS41NeO9-6ZAkuZKXw@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/catalog/system_views.sql
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/interrupt.c
M src/backend/postmaster/pgarch.c
M src/backend/postmaster/startup.c
M src/backend/postmaster/walsummarizer.c
M src/backend/storage/ipc/ipci.c
M src/backend/storage/ipc/procsignal.c
M src/backend/storage/lmgr/lwlock.c
M src/backend/storage/lmgr/proc.c
M src/backend/tcop/postgres.c
M src/backend/utils/activity/wait_event_names.txt
M src/backend/utils/adt/mcxtfuncs.c
M src/backend/utils/init/globals.c
M src/backend/utils/init/postinit.c
M src/backend/utils/mmgr/mcxt.c
M src/include/catalog/pg_proc.dat
M src/include/miscadmin.h
M src/include/storage/lwlock.h
M src/include/storage/procsignal.h
M src/include/utils/memutils.h
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/sysviews.sql
M src/tools/pgindent/typedefs.list
Increase BAS_BULKREAD based on effective_io_concurrency
commit : 15f0cb26b530b6725a37391738cfc62d4745c49b
author : Andres Freund <andres@anarazel.de>
date : Tue, 8 Apr 2025 02:41:03 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 8 Apr 2025 02:41:03 -0400
Before, BAS_BULKREAD was always of size 256kB. With the default
io_combine_limit of 16, that only allowed 1-2 IOs to be in flight -
insufficient even on very low latency storage.
We don't just want to increase the size to a much larger hardcoded value, as
very large rings (10s of MBs of of buffers), appear to have negative
performance effects when reading in data that the OS has cached (but not when
actually needing to do IO).
To address this, increase the size of BAS_BULKREAD to allow for
io_combine_limit * effective_io_concurrency buffers getting read in. To
prevent the ring being much larger than useful, limit the increased size with
GetPinLimit().
The formula outlined above keeps the ring size to sizes for which we have not
observed performance regressions, unless very large effective_io_concurrency
values are used together with large shared_buffers setting.
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/lqwghabtu2ak4wknzycufqjm5ijnxhb4k73vzphlt2a3wsemcd@gtftg44kdim6
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah@brqs62irg4dt
M src/backend/storage/buffer/freelist.c
Add pg_buffercache_evict_{relation,all} functions
commit : dcf7e1697ba75ce7883ad7c6bc26ba24422eb892
author : Andres Freund <andres@anarazel.de>
date : Tue, 8 Apr 2025 02:16:51 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 8 Apr 2025 02:16:51 -0400
In addition to the added functions, the pg_buffercache_evict() function now
shows whether the buffer was flushed.
pg_buffercache_evict_relation(): Evicts all shared buffers in a
relation at once.
pg_buffercache_evict_all(): Evicts all shared buffers at once.
Both functions provide mechanism to evict multiple shared buffers at
once. They are designed to address the inefficiency of repeatedly calling
pg_buffercache_evict() for each individual buffer, which can be time-consuming
when dealing with large shared buffer pools. (e.g., ~477ms vs. ~2576ms for
16GB of fully populated shared buffers).
These functions are intended for developer testing and debugging
purposes and are available to superusers only.
Minimal tests for the new functions are included. Also, there was no test for
pg_buffercache_evict(), test for this added too.
No new extension version is needed, as it was already increased this release
by ba2a3c2302f.
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Aidar Imamov <a.imamov@postgrespro.ru>
Reviewed-by: Joseph Koshakow <koshy44@gmail.com>
Discussion: https://postgr.es/m/CAN55FZ0h_YoSqqutxV6DES1RW8ig6wcA8CR9rJk358YRMxZFmw%40mail.gmail.com
M contrib/pg_buffercache/expected/pg_buffercache.out
M contrib/pg_buffercache/pg_buffercache–1.5–1.6.sql
M contrib/pg_buffercache/pg_buffercache_pages.c
M contrib/pg_buffercache/sql/pg_buffercache.sql
M doc/src/sgml/pgbuffercache.sgml
M src/backend/storage/buffer/bufmgr.c
M src/include/storage/bufmgr.h
M src/test/modules/test_aio/test_aio.c
Speedup child EquivalenceMember lookup in planner
commit : d69d45a5a956e930dc91b3ca09a0188bf9fe2176
author : David Rowley <drowley@postgresql.org>
date : Tue, 8 Apr 2025 18:09:57 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 8 Apr 2025 18:09:57 +1200
When planning queries to partitioned tables, we clone all
EquivalenceMembers belonging to the partitioned table into em_is_child
EquivalenceMembers for each non-pruned partition. For partitioned tables
with large numbers of partitions, this meant the ec_members list could
become large and code searching that list would become slow. Effectively,
the more partitions which were present, the more searches needed to be
performed for operations such as find_ec_member_matching_expr() during
create_plan() and the more partitions present, the longer these searches
would take, i.e., a quadratic slowdown.
To fix this, here we adjust how we store EquivalenceMembers for
em_is_child members. Instead of storing these directly in ec_members,
these are now stored in a new array of Lists in the EquivalenceClass,
which is indexed by the relid. When we want to find EquivalenceMembers
belonging to a certain child relation, we can narrow the search to the
array element for that relation.
To make EquivalenceMember lookup easier and to reduce the amount of code
change, this commit provides a pair of functions to allow iteration over
the EquivalenceMembers of an EC which also handles finding the child
members, if required. Callers that never need to look at child members
can remain using the foreach loop over ec_members, which will now often
be faster due to only parent-level members being stored there.
The actual performance increases here are highly dependent on the number
of partitions and the query being planned. Performance increases can be
visible with as few as 8 partitions, but the speedup is marginal for
such low numbers of partitions. The speedups become much more visible
with a few dozen to hundreds of partitions. With some tested queries
using 56 partitions, the planner was around 3x faster than before. For
use cases with thousands of partitions, these are likely to become
significantly faster. Some testing has shown planner speedups of 60x or
more with 8192 partitions.
Author: Yuya Watari <watari.yuya@gmail.com>
Co-authored-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
Reviewed-by: Alena Rybakina <lena.ribackina@yandex.ru>
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Tested-by: Thom Brown <thom@linux.com>
Tested-by: newtglobal postgresql_contributors <postgresql_contributors@newtglobalcorp.com>
Discussion: https://postgr.es/m/CAJ2pMkZNCgoUKSE%2B_5LthD%2BKbXKvq6h2hQN8Esxpxd%2Bcxmgomg%40mail.gmail.com
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/path/pathkeys.c
M src/backend/optimizer/plan/analyzejoins.c
M src/include/nodes/pathnodes.h
M src/include/optimizer/paths.h
M src/tools/pgindent/typedefs.list
Stabilize 035_standby_logical_decoding.pl.
commit : 105b2cb336173d7c62e26ad104682175ddad4cff
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 8 Apr 2025 09:38:02 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 8 Apr 2025 09:38:02 +0530
Some tests try to invalidate logical slots on the standby server by
running VACUUM on the primary. The problem is that xl_running_xacts was
getting generated and replayed before the VACUUM command, leading to the
advancement of the active slot's catalog_xmin. Due to this, active slots
were not getting invalidated, leading to test failures.
We fix it by skipping the generation of xl_running_xacts for the required
tests with the help of injection points. As the required interface for
injection points was not present in back branches, we fixed the failing
tests in them by disallowing the slot to become active for the required
cases (where rows_removed conflict could be generated).
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Backpatch-through: 16, where it was introduced
Discussion: https://postgr.es/m/Z6oQXc8LmiTLfwLA@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/storage/ipc/standby.c
M src/test/recovery/t/035_standby_logical_decoding.pl
Fix PG 17 [NOT] NULL optimization bug for domains
commit : 46b4ba533cee9f64a60714d91607e74362abf67f
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 7 Apr 2025 21:33:42 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 7 Apr 2025 21:33:42 -0400
A PG 17 optimization allowed columns with NOT NULL constraints to skip
table scans for IS NULL queries, and to skip IS NOT NULL checks for IS
NOT NULL queries. This didn't work for domain types, since domain types
don't follow the IS NULL/IS NOT NULL constraint logic. To fix, disable
this optimization for domains for PG 17+.
Reported-by: Jan Behrens
Diagnosed-by: Tom Lane
Discussion: https://postgr.es/m/Z37p0paENWWUarj-@momjian.us
Backpatch-through: 17
M doc/src/sgml/ref/create_domain.sgml
M src/backend/optimizer/plan/initsplan.c
Flush the IO statistics of active WAL senders more frequently
commit : 039549d70f6aa2daa3714a13752a08fa8ca2fb05
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Apr 2025 07:57:19 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Apr 2025 07:57:19 +0900
WAL senders do not flush their statistics until they exit, limiting the
monitoring possible for live processes. This is penalizing when WAL
senders are running for a long time, like in streaming or logical
replication setups, because it is not possible to know the amount of IO
they generate while running.
This commit makes WAL senders more aggressive with their statistics
flush, using an internal of 1 second, with the flush timing calculated
based on the existing GetCurrentTimestamp() done before the sleeps done
to wait for some activity. Note that the sleep done for logical and
physical WAL senders happens in two different code paths, so the stats
flushes need to happen in these two places.
One test is added for the physical WAL sender case, and one for the
logical WAL sender case. This can be done in a stable fashion by
relying on the WAL generated by the TAP tests in combination with a
stats reset while a server is running, but only on HEAD as WAL data has
been added to pg_stat_io in a051e71e28a1.
This issue exists since a9c70b46dbe and the introduction of pg_stat_io,
so backpatch down to v16.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/Z73IsKBceoVd4t55@ip-10-97-1-34.eu-west-3.compute.internal
Backpatch-through: 16
M src/backend/replication/walsender.c
M src/test/recovery/t/001_stream_rep.pl
M src/test/subscription/t/001_rep_changes.pl
Add pg_buffercache_numa view with NUMA node info
commit : ba2a3c2302f1248496322eba917b17a421499388
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 7 Apr 2025 22:56:57 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 7 Apr 2025 22:56:57 +0200
Introduces a new view pg_buffercache_numa, showing NUMA memory nodes
for individual buffers. For each buffer the view returns an entry for
each memory page, with the associated NUMA node.
The database blocks and OS memory pages may have different size - the
default block size is 8KB, while the memory page is 4K (on x86). But
other combinations are possible, depending on configure parameters,
platform, etc. This means buffers may overlap with multiple memory
pages, each associated with a different NUMA node.
To determine the NUMA node for a buffer, we first need to touch the
memory pages using pg_numa_touch_mem_if_required, otherwise we might get
status -2 (ENOENT = The page is not present), indicating the page is
either unmapped or unallocated.
The view may be relatively expensive, especially when accessed for the
first time in a backend, as it touches all memory pages to get reliable
information about the NUMA node. This may also force allocation of the
shared memory.
Author: Jakub Wartak <jakub.wartak@enterprisedb.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
M contrib/pg_buffercache/Makefile
A contrib/pg_buffercache/expected/pg_buffercache_numa.out
A contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
M contrib/pg_buffercache/meson.build
A contrib/pg_buffercache/pg_buffercache–1.5–1.6.sql
M contrib/pg_buffercache/pg_buffercache.control
M contrib/pg_buffercache/pg_buffercache_pages.c
A contrib/pg_buffercache/sql/pg_buffercache_numa.sql
M doc/src/sgml/pgbuffercache.sgml
M src/tools/pgindent/typedefs.list
Introduce pg_shmem_allocations_numa view
commit : 8cc139bec34a2971b0682a04eb52ce7b3f5bb425
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 7 Apr 2025 22:54:49 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 7 Apr 2025 22:54:49 +0200
Introduce new pg_shmem_alloctions_numa view with information about how
shared memory is distributed across NUMA nodes. For each shared memory
segment, the view returns one row for each NUMA node backing it, with
the total amount of memory allocated from that node.
The view may be relatively expensive, especially when executed for the
first time in a backend, as it has to touch all memory pages to get
reliable information about the NUMA node. This may also force allocation
of the shared memory.
Unlike pg_shmem_allocations, the view does not show anonymous shared
memory allocations. It also does not show memory allocated using the
dynamic shared memory infrastructure.
Author: Jakub Wartak <jakub.wartak@enterprisedb.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
M doc/src/sgml/system-views.sgml
M src/backend/catalog/system_views.sql
M src/backend/storage/ipc/shmem.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
A src/test/regress/expected/numa.out
A src/test/regress/expected/numa_1.out
M src/test/regress/expected/privileges.out
M src/test/regress/expected/rules.out
M src/test/regress/parallel_schedule
A src/test/regress/sql/numa.sql
M src/test/regress/sql/privileges.sql
Add support for basic NUMA awareness
commit : 65c298f61fc70f2f960437c05649f71b862e2c48
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 7 Apr 2025 22:51:49 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 7 Apr 2025 22:51:49 +0200
Add basic NUMA awareness routines, using a minimal src/port/pg_numa.c
portability wrapper and an optional build dependency, enabled by
--with-libnuma configure option. For now this is Linux-only, other
platforms may be supported later.
A built-in SQL function pg_numa_available() allows checking NUMA
support, i.e. that the server was built/linked with the NUMA library.
The main function introduced is pg_numa_query_pages(), which allows
determining the NUMA node for individual memory pages. Internally the
function uses move_pages(2) syscall, as it allows batching, and is more
efficient than get_mempolicy(2).
Author: Jakub Wartak <jakub.wartak@enterprisedb.com>
Co-authored-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
M .cirrus.tasks.yml
M configure
M configure.ac
M doc/src/sgml/func.sgml
M doc/src/sgml/installation.sgml
M meson.build
M meson_options.txt
M src/Makefile.global.in
M src/backend/utils/misc/guc_tables.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pg_config.h.in
A src/include/port/pg_numa.h
M src/include/storage/pg_shmem.h
M src/makefiles/meson.build
M src/port/Makefile
M src/port/meson.build
A src/port/pg_numa.c
Use specific collation where needed in new test
commit : 17bcf4f5450430f67b744c225566c9e0e6413e95
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 7 Apr 2025 21:58:06 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 7 Apr 2025 21:58:06 +0200
Oversight in commit a379061a22a8.
Per Czech buildfarm members jay and hippopotamus.
M src/test/regress/expected/constraints.out
M src/test/regress/sql/constraints.sql
Fix some issues in contrib/spi/refint.c.
commit : 8cfbdf8f4dfbb2204df32fbc7d5dcd4d1b465723
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Apr 2025 15:54:09 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Apr 2025 15:54:09 -0400
check_foreign_key incorrectly used a single cache entry for its saved
plans for a 'c' (cascade) trigger, although there are two different
queries to execute depending on whether it fires for an update or a
delete. This caused the wrong things to be done if both types of
event occur in one session. (This was indeed visible in the triggers
regression test, but apparently nobody ever questioned it.) To fix,
add the operation type to the cache key.
Its debug log output failed to distinguish update from delete
events, too.
Also, change the intended trigger usage from BEFORE ROW to AFTER ROW,
and add checks insisting on that usage. BEFORE is really rather
unsafe, since if there are other BEFORE triggers they might change or
cancel the operation we are trying to check. AFTER triggers are the
standard way to propagate changes to other rows, so we should follow
that way here.
In passing, remove a useless duplicate lookup of the cache entry.
This code is mostly intended as a documentation example, so we
won't consider a back-patch.
Author: Dmitrii Bondar <d.bondar@postgrespro.ru>
Reviewed-by: Paul Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: Lilian Ontowhee <ontowhee@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/79755a2b18ed4fe5e29da6a87a1e00d1@postgrespro.ru
M contrib/spi/refint.c
M doc/src/sgml/contrib-spi.sgml
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
aio: Make AIO more compatible with valgrind
commit : 8e293e689bab0267d26e3705fe1d537cd43e633a
author : Andres Freund <andres@anarazel.de>
date : Mon, 7 Apr 2025 15:20:30 -0400
committer: Andres Freund <andres@anarazel.de>
date : Mon, 7 Apr 2025 15:20:30 -0400
In some edge cases valgrind flags issues with the memory referenced by
IOs. All of the cases addressed in this change are false positives.
Most of the false positives are caused by UnpinBuffer[NoOwner] marking buffer
data as inaccessible. This happens even though the AIO subsystem still holds a
pin. That's good, there shouldn't be accesses to the buffer outside of AIO
related code until it is pinned by "user" code again. But it requires some
explicit work - if the buffer is not pinned by the current backend, we need to
explicitly mark the buffer data accessible/inaccessible while executing
completion callbacks.
That however causes a cascading issue in IO workers: After the completion
callbacks for a buffer is executed, the page is marked as inaccessible. If
subsequently the same worker is executing IO targeting the same buffer, we
would get an error, as the memory is still marked inaccessible. To avoid that,
we need to explicitly mark the memory as accessible in IO workers.
Another issue is that IO executed in workers or via io_uring will not mark
memory as DEFINED. In the case of workers that is because valgrind does not
track memory definedness across processes. For io_uring that is because
valgrind does not understand io_uring, and therefore its IOs never mark memory
as defined, whether the completions are processed in the defining process or
in another context. It's not entirely clear how to best solve that. The
current user of AIO is not affected, as it explicitly marks buffers as DEFINED
& NOACCESS anyway. Defer solving this issue until we have a user with
different needs.
Per buildfarm animal skink.
Reviewed-by: Noah Misch <noah@leadboat.com>
Co-authored-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/3pd4322mogfmdd5nln3zphdwhtmq3rzdldqjwb2sfqzcgs22lf@ok2gletdaoe6
M src/backend/storage/aio/aio_io.c
M src/backend/storage/aio/method_worker.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/smgr/smgr.c
M src/include/storage/aio_internal.h
localbuf: Add Valgrind buffer access instrumentation
commit : 8ab4241b9f4f73f2168bcaebc990f8b0a6b7bc81
author : Andres Freund <andres@anarazel.de>
date : Mon, 7 Apr 2025 15:20:30 -0400
committer: Andres Freund <andres@anarazel.de>
date : Mon, 7 Apr 2025 15:20:30 -0400
This mirrors 1e0dfd166b3 (+ 46ef520b9566), for temporary table buffers. This
is mainly interesting right now because the AIO work currently triggers
spurious valgrind errors, and the fix for that is cleaner if temp buffers
behave the same as shared buffers.
This requires one change beyond the annotations themselves, namely to pin
local buffers while writing them out in FlushRelationBuffers().
Reviewed-by: Noah Misch <noah@leadboat.com>
Co-authored-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/3pd4322mogfmdd5nln3zphdwhtmq3rzdldqjwb2sfqzcgs22lf@ok2gletdaoe6
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
doc: Fix a typo in pg_recvlogical documentation.
commit : a13d49014dea39bbe436fd6c28fc65f927038ed7
author : Masahiko Sawada <msawada@postgresql.org>
date : Mon, 7 Apr 2025 12:13:08 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Mon, 7 Apr 2025 12:13:08 -0700
Oversight in cf2655a9029a.
Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Discussion: https://postgr.es/m/OS3PR01MB5718DD1466E2B9043448AE5094AA2@OS3PR01MB5718.jpnprd01.prod.outlook.com
M doc/src/sgml/ref/pg_recvlogical.sgml
Follow-up fixes for SHA-2 patch (commit 749a9e20c).
commit : 969ab9d4f5d141ac25db092a0e5cd004d3d60a60
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Apr 2025 14:14:28 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Apr 2025 14:14:28 -0400
This changes the check for valid characters in the salt string to
only allow plain ASCII letters and digits. The previous coding was
locale-dependent which doesn't really seem like a great idea here;
moreover it could not work correctly in multibyte encodings.
This fixes a careless pointer-use-after-pfree, too.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Andres Freund <andres@anarazel.de>
Author: Bernd Helmle <mailings@oopsware.de>
Discussion: https://postgr.es/m/6fab35422df6b6b9727fdcc243c5fa1c667dd3b5.camel@oopsware.de
M contrib/pgcrypto/crypt-sha.c
Fix erroneous construction of functions' dependencies on transforms.
commit : b73e6d71a89f92436c71467acda1d47552f568a0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Apr 2025 13:31:27 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Apr 2025 13:31:27 -0400
The list of transform objects that a function should use is specified
in CREATE FUNCTION's TRANSFORM clause, and then represented indirectly
in pg_proc.protrftypes. However, ProcedureCreate completely ignored
that for purposes of constructing pg_depend entries, and instead made
the function depend on any transforms that exist for its parameter or
return data types. This is bad in both directions: the function could
be made dependent on a transform it does not actually use, or it
could try to use a transform that's since been dropped. (The latter
scenario would require use of a transform that's not for any of the
parameter or return types, but that seems legit for cases where the
function performs SQL operations internally.)
To fix, pass in the list of transform objects that CreateFunction
identified, and build pg_depend entries from that not from the
parameter/return types. This results in changes in the expected
test outputs in contrib/bool_plperl, which I guess are due to
different ordering of pg_depend entries -- that test case is
surely not exercising either of the problem scenarios.
This fix is not back-patchable as-is: changing the signature of
ProcedureCreate seems too risky in stable branches. We could
do something like making ProcedureCreate a wrapper around
ProcedureCreateExt or so. However, I'm more inclined to do
nothing in the back branches. We had no field complaints up to
now, so the hazards don't seem to be a big issue in practice.
And we couldn't do anything about existing pg_depend entries,
so a back-patched fix would result in a mishmash of dependencies
created according to different rules. That cure could be worse
than the disease, perhaps.
I bumped catversion just to lay down a marker that the expected
contents of pg_depend are a bit different than before.
Reported-by: Chapman Flack <jcflack@acm.org>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/3112950.1743984111@sss.pgh.pa.us
M contrib/bool_plperl/expected/bool_plperl.out
M contrib/bool_plperl/expected/bool_plperlu.out
M src/backend/catalog/pg_aggregate.c
M src/backend/catalog/pg_proc.c
M src/backend/commands/functioncmds.c
M src/backend/commands/typecmds.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.h
Allow NOT NULL constraints to be added as NOT VALID
commit : a379061a22a8fdf421e1a457cc6af8503def6252
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 7 Apr 2025 19:19:50 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 7 Apr 2025 19:19:50 +0200
This allows them to be added without scanning the table, and validating
them afterwards without holding access exclusive lock on the table after
any violating rows have been deleted or fixed.
Doing ALTER TABLE ... SET NOT NULL for a column that has an invalid
not-null constraint validates that constraint. ALTER TABLE .. VALIDATE
CONSTRAINT is also supported. There are various checks on whether an
invalid constraint is allowed in a child table when the parent table has
a valid constraint; this should match what we do for enforced/not
enforced constraints.
pg_attribute.attnotnull is now only an indicator for whether a not-null
constraint exists for the column; whether it's valid or invalid must be
queried in pg_constraint. Applications can continue to query
pg_attribute.attnotnull as before, but now it's possible that NULL rows
are present in the column even when that's set to true.
For backend internal purposes, we cache the nullability status in
CompactAttribute->attnullability that each tuple descriptor carries
(replacing CompactAttribute.attnotnull, which was a mirror of
Form_pg_attribute.attnotnull). During the initial tuple descriptor
creation, based on the pg_attribute scan, we set this to UNRESTRICTED if
pg_attribute.attnotnull is false, or to UNKNOWN if it's true; then we
update the latter to VALID or INVALID depending on the pg_constraint
scan. This flag is also copied when tupledescs are copied.
Comparing tuple descs for equality must also compare the
CompactAttribute.attnullability flag and return false in case of a
mismatch.
pg_dump deals with these constraints by storing the OIDs of invalid
not-null constraints in a separate array, and running a query to obtain
their properties. The regular table creation SQL omits them entirely.
They are then dealt with in the same way as "separate" CHECK
constraints, and dumped after the data has been loaded. Because no
additional pg_dump infrastructure was required, we don't bump its
version number.
I decided not to bump catversion either, because the old catalog state
works perfectly in the new world. (Trying to run with new catalog state
and the old server version would likely run into issues, however.)
System catalogs do not support invalid not-null constraints (because
commit 14e87ffa5c54 didn't allow them to have pg_constraint rows
anyway.)
Author: Rushabh Lathia <rushabh.lathia@gmail.com>
Author: Jian He <jian.universality@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Tested-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CAGPqQf0KitkNack4F5CFkFi-9Dqvp29Ro=EpcWt=4_hs-Rt+bQ@mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/alter_table.sgml
M src/backend/access/common/tupdesc.c
M src/backend/catalog/heap.c
M src/backend/catalog/pg_constraint.c
M src/backend/commands/tablecmds.c
M src/backend/jit/llvm/llvmjit_deform.c
M src/backend/optimizer/util/plancat.c
M src/backend/parser/gram.y
M src/backend/utils/cache/relcache.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/psql/describe.c
M src/include/access/tupdesc.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_constraint.h
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/constraints.out
M src/test/regress/sql/alter_table.sql
M src/test/regress/sql/constraints.sql
Clean up error messages from 1495eff7bdb
commit : b52a4a5f285df49399fe6deefa1bf63dc88cd3d1
author : Andrew Dunstan <andrew@dunslane.net>
date : Mon, 7 Apr 2025 12:22:41 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Mon, 7 Apr 2025 12:22:41 -0400
Quote file names, and mostly avoid hard coded file names. Along the way
make a few other minor improvements.
Discussion: https://postgr.es/m/20250407.152721.1397761902317499205.horikyota.ntt@gmail.com
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_dump/pg_restore.c
Add local-address escape "%L" to log_line_prefix.
commit : 3516ea768c92ad0abc16dc4dd97e5eef6eab4237
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Apr 2025 10:57:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Apr 2025 10:57:26 -0400
This escape shows the numeric server IP address that the client
has connected to. Unix-socket connections will show "[local]".
Non-client processes (e.g. background processes) will show "[none]".
We expect that this option will be of interest to only a fairly
small number of users. Therefore the implementation is optimized
for the case where it's not used (that is, we don't do the string
conversion until we have to), and we've not added the field to
csvlog or jsonlog formats.
Author: Greg Sabino Mullane <htamfids@gmail.com>
Reviewed-by: Cary Huang <cary.huang@highgo.ca>
Reviewed-by: David Steele <david@pgmasters.net>
Reviewed-by: Jim Jones <jim.jones@uni-muenster.de>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAKAnmmK-U+UicE-qbNU23K--Q5XTLdM6bj+gbkZBZkjyjrd3Ow@mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/utils/error/elog.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/libpq/libpq-be.h
Revert "Use workaround of __builtin_setjmp only on MINGW on MSVCRT"
commit : 8f5e419484c3efb613d971ec25b9bf344db3d0b0
author : Andrew Dunstan <andrew@dunslane.net>
date : Mon, 7 Apr 2025 11:01:15 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Mon, 7 Apr 2025 11:01:15 -0400
This reverts commit c313fa4602defe1be947370ab5b217ca163a1e3c.
This is found to cause issues on x86_64 Windows even when using UCRT.
Discussion: https://postgr.es/m/3312149.1744001936@sss.pgh.pa.us
M src/include/c.h
read_stream: Fix overflow hazard with large shared buffers
commit : 8ce79483dc47df11159f506cf51bacec9f874055
author : Andres Freund <andres@anarazel.de>
date : Mon, 7 Apr 2025 08:47:30 -0400
committer: Andres Freund <andres@anarazel.de>
date : Mon, 7 Apr 2025 08:47:30 -0400
If the limit returned by GetAdditionalPinLimit() is large, the buffer_limit
variable in read_stream_start_pending_read() can overflow. While the code is
careful to limit buffer_limit PG_INT16_MAX, we subsequently add the number of
forwarded buffers.
The overflow can lead to assertion failures, crashes or wrong query results
when using large shared buffers.
It seems easier to avoid this if we make the buffer_limit variable an int,
instead of an int16. Do so, and clamp buffer_limit after adding the number of
forwarded buffers.
It's possible we might want to address this and related issues more widely by
changing to int instead of int16 more widely, but since the consequences of
this bug can be confusing, it seems better to fix it now.
This bug was introduced in ed0b87caaca.
Discussion: https://postgr.es/m/ewvz3cbtlhrwqk7h6ca6cctiqh7r64ol3pzb3iyjycn2r5nxk5@tnhw3a5zatlr
M src/backend/storage/aio/read_stream.c
Remove GUC_NOT_IN_SAMPLE from enable_self_join_elimination
commit : 717d0e8dd945edea7c7e875bdb2553238a94f885
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 7 Apr 2025 16:28:54 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 7 Apr 2025 16:28:54 +0300
fc069a3a6319 implements Self-Join Elimination (SJE) and provides a new GUC
variable: enable_self_join_elimination. This new GUC variable was marked
as GUC_NOT_IN_SAMPLE. However, enable_self_join_elimination is documented
and is not different from any other enable_* GUCs. Thus, remove
GUC_NOT_IN_SAMPLE from it and add it to the postgresql.conf.sample.
Discussion: https://postgr.es/m/CAPpHfdsqMTEsmxk3aQwt6xPz%2BKpUELO%3D6fzmER9ZRGrbs4uMfA%40mail.gmail.com
Author: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
psql: Clarify help message for WATCH_INTERVAL
commit : ae60947643722da257a59f8f16f855add307583f
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 7 Apr 2025 13:44:58 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 7 Apr 2025 13:44:58 +0200
The help message for WATCH_INTERVAL was hard to interpret and didn't
follow the style of other messages, this updates it to nake it fit in
better and be easier to interpret.
Author: Daniel Gustafsson <daniel@yesql.se>
Reported-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/20250326.120732.1167093737847500721.horikyota.ntt@gmail.com
M src/bin/psql/help.c
Fix grammar in log message of pg_restore.c
commit : d6f118444db320d13b70e7fbb8eac3ab2113b93d
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Apr 2025 15:37:34 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Apr 2025 15:37:34 +0900
Introduced by 1495eff7bdb0.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/20250407.151359.72428746612514925.horikyota.ntt@gmail.com
M src/bin/pg_dump/pg_restore.c
libpq: Fix some issues in TAP tests for service files
commit : 2c7bd2ba507e273f2d7fe1b2f6d30775ed4f3c09
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Apr 2025 12:55:09 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Apr 2025 12:55:09 +0900
The valid service file was not correctly shaped, as append_to_file() was
called with an array as input. This is changed so as the parameter and
value pairs from the valid connection string are appended to the valid
service file one by one.
Even with the first issue fixed, the tests should fail. However, they
have been passing because all the connection attempts relied on the
default values given to PGPORT and PGHOST from the node when using
Cluster.pm's connect_ok() and connect_fails(), rather than the data in
the service file. The test is updated to use an interesting trick: a
dummy node is initialized but not started, and all the connection
attempts are done through it. This ensures that the data inside the
service file is used for all the connection tests. Note that breaking
the contents of the valid service file on purpose makes all the tests
that rely on it fail.
Issues introduced by 72c2f36d5727.
Author: Andrew Jackson <andrewjackson947@gmail.com>
Discussion: https://postgr.es/m/CAKK5BkG_6_YSaebM6gG=8EuKaY7_VX1RFgYeySuwFPh8FZY73g@mail.gmail.com
M src/interfaces/libpq/t/006_service.pl
Clarify comment for worst-case allocation in quote_literal_cstr()
commit : c36eda259130faf3559b751b9e81a5c7e2bee0d7
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Apr 2025 10:02:12 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Apr 2025 10:02:12 +0900
palloc() is invoked with a specific formula for its allocation size in
quote_literal_cstr(). This wastes some memory, but the size is large
enough to cover even the worst-case scenarios.
No explanations were given about the reasons behind these numbers. This
commit adds more documentation about all that.
Author: Steve Chavez <steve@supabase.io>
Discussion: https://postgr.es/m/CAGRrpzZ9bToRWS+fAnjxDJrxwZN1QcJ-y1Pn2yg=Hst6rydLtw@mail.gmail.com
M src/backend/utils/adt/quote.c
Fix use-after-free in pgstat_fetch_stat_backend_by_pid()
commit : 3191a593d6dea56def460d06adc779f2aca44976
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Apr 2025 09:51:40 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Apr 2025 09:51:40 +0900
stats_fetch_consistency set to "snapshot" causes the backend entry
"beentry" retrieved by pgstat_get_beentry_by_proc_number() to be reset
at the beginning of pgstat_fetch_stat_backend() when fetching the
backend pgstats entry. As coded, "beentry" was being accessed after
being freed. This commit moves all the accesses to "beentry" to happen
before calling pgstat_fetch_stat_backend(), fixing the problem.
This problem could be reached by calling the SQL functions
pg_stat_get_backend_io() or pg_stat_get_backend_wal().
Issue caught by valgrind.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/f1788cc0-253a-4a3a-aee0-1b8ab9538736@gmail.com
M src/backend/utils/activity/pgstat_backend.c
Use XLOG_CONTROL_FILE macro consistently for control file name.
commit : 173c97812ffcc48082355df71470974c33379d3f
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 7 Apr 2025 09:27:33 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 7 Apr 2025 09:27:33 +0900
The XLOG_CONTROL_FILE macro (defined in access/xlog_internal.h)
represents the control file name. While some parts of the codebase already
use this macro, others previously hardcoded the file name as a string.
This commit replaces those hardcoded strings with the macro,
ensuring consistent usage throughout the code. This makes future
maintenance easier and improves searchability, for example when
grepping for control file usage.
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Masao Fujii <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/0841ec77-47e5-452a-adb4-c6fa55d605fc@postgrespro.ru
M src/backend/backup/basebackup.c
M src/backend/postmaster/postmaster.c
M src/bin/pg_combinebackup/pg_combinebackup.c
M src/bin/pg_controldata/pg_controldata.c
M src/bin/pg_rewind/filemap.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_verifybackup/astreamer_verify.c
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/common/controldata_utils.c
doc: Clarify project naming
commit : a233a603bab80fdf6da2b4908c98f166273aa0cb
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 7 Apr 2025 00:03:18 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 7 Apr 2025 00:03:18 +0200
Clarify the project naming in the history section of the docs
to match the recent license preamble changes.
Backpatch to all supported versions.
Author: Dave Page <dpage@pgadmin.org>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CA+OCxozLzK2+Jc14XZyWXSp6L9Ot+3efwXUE35FJG=fsbib2EA@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/history.sgml
Clean up checking for pg_dumpall output directory
commit : 643a1a61985bef25904965053020057200d4ae48
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 6 Apr 2025 17:00:58 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 6 Apr 2025 17:00:58 -0400
Coverity objected to the original code, and in any case this is much
cleaner, using the existing routine pg_check_dir() instead of rolling
its own test.
Per suggestion from Tom Lane.
M src/bin/pg_dump/pg_dumpall.c
Doc: fix PDF "contents ... exceed the available area" warnings.
commit : 218ab682751dfe75667456019364187758d4871a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Apr 2025 16:27:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Apr 2025 16:27:31 -0400
Tweak column widths in a new table, similarly to some previous
fixes such as b62381d9a.
Per buildfarm.
M doc/src/sgml/pgcrypto.sgml
pg_upgrade: Fix memory leak in check_for_unicode_update().
commit : de48056ec7d237e6a48dce907804ac26c87311ec
author : Nathan Bossart <nathan@postgresql.org>
date : Sun, 6 Apr 2025 15:11:41 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Sun, 6 Apr 2025 15:11:41 -0500
This function was initializing the "task" variable before a couple
of early returns. To fix, postpone the initialization until just
before it's needed.
Per Coverity.
Discussion: https://postgr.es/m/Z_KMsUH2-FEbiNjC%40nathan
M src/bin/pg_upgrade/check.c
aio: Avoid spurious coverity warning
commit : 57dec20fd4691900fbe118699d32640fc538dc20
author : Andres Freund <andres@anarazel.de>
date : Fri, 4 Apr 2025 15:15:39 -0400
committer: Andres Freund <andres@anarazel.de>
date : Fri, 4 Apr 2025 15:15:39 -0400
PgAioResult.result is never accessed in the relevant path, but coverity
complains about an uninitialized access anyway. So just zero-initialize the
whole thing. While at it, reduce the scope of the variable.
Reported-by: Ranier Vilela <ranier.vf@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CAEudQApsKqd-s+fsUQ0OmxJAMHmBSXxrAz3dCs+uvqb3iRtjSw@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
Fix memory leaks in px_crypt_shacrypt().
commit : 8ab6ef2bb8d9a00d205742932eb0b6daf62b34e0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Apr 2025 11:57:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Apr 2025 11:57:22 -0400
Per Coverity. I don't think these are of any actual significance
since the function ought to be invoked in a short-lived context.
Still, if it's trying to be neat it should get it right.
Also const-ify a constant and fix up typedef formatting.
M contrib/pgcrypto/crypt-sha.c
M src/tools/pgindent/typedefs.list
Use "(void)" to mark pgstat_lock_entry(..., false) calls.
commit : 2e4ccf1b4508cc337bb4d0afff1e32a049d549fc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Apr 2025 11:37:09 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Apr 2025 11:37:09 -0400
This should silence Coverity's complaints about the result being
sometimes ignored.
I'm inclined to think that these routines are simply misdesigned,
because sometimes it's okay to ignore the result and sometimes it
isn't, and we have no way to enforce the latter. But for now
I just added a comment.
M src/backend/utils/activity/pgstat.c
M src/backend/utils/activity/pgstat_database.c
M src/backend/utils/activity/pgstat_shmem.c
Avoid unnecessary copying of a string in pg_restore.c
commit : 5e1915439085014140314979c4dd5e23bd677cac
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 6 Apr 2025 09:21:09 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 6 Apr 2025 09:21:09 -0400
Coverity complained about a possible overrun in the copy, but there is
no actual need to copy the string at all.
M src/bin/pg_dump/pg_restore.c
Fix a couple of memory leaks in pg_restore.c
commit : 6d5417e634b3841dcc44bdb43f5586bcde33ddb4
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 6 Apr 2025 09:08:02 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 6 Apr 2025 09:08:02 -0400
per complaint from Coverity.
M src/bin/pg_dump/pg_restore.c
Relax ordering-related hardcoded btree requirements in planning
commit : a8025f544854ad8b865c6b4509030ee84aa8f4a0
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 6 Apr 2025 14:43:51 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 6 Apr 2025 14:43:51 +0200
There were several places in ordering-related planning where a
requirement for btree was hardcoded but an amcanorder index could
suffice. This fixes that. We just need to do the necessary mapping
between strategy numbers and compare types and adjust some related
APIs so that this works independent of btree strategy numbers. For
instance, non-btree amcanorder indexes can now be used to support
sorting and merge joins. Also, predtest.c works independent of btree
strategy numbers now.
To avoid performance regressions, some details on btree and other
built-in index types are still hardcoded as shortcuts, but other index
types now have access to the same features by providing the required
flags and callbacks.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Co-authored-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/access/index/amapi.c
M src/backend/executor/nodeMergejoin.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/path/pathkeys.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/plancat.c
M src/backend/optimizer/util/predtest.c
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_expr.c
M src/backend/utils/adt/selfuncs.c
M src/backend/utils/cache/lsyscache.c
M src/backend/utils/sort/sortsupport.c
M src/include/utils/lsyscache.h
M src/include/utils/selfuncs.h
M src/tools/pgindent/typedefs.list
Revert "Put enable_self_join_elimination into postgresql.conf.sample"
commit : 3a1a7c5a7071c75676c15b26e242c7df17560bd1
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 6 Apr 2025 14:30:20 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 6 Apr 2025 14:30:20 +0300
This reverts commit c2d329260cd8.
Reported-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/D292EB44-806E-439A-82A4-491A1BA59E7A%40yesql.se
M src/backend/utils/misc/postgresql.conf.sample
Put enable_self_join_elimination into postgresql.conf.sample
commit : c2d329260cd885a937014cecd38b51bdeae65216
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 6 Apr 2025 13:24:16 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 6 Apr 2025 13:24:16 +0300
fc069a3a6319 implements Self-Join Elimination (SJE) and provides a new
GUC variable: enable_self_join_elimination. This commit adds
enable_self_join_elimination to the postgresql.conf.sample, as it was
forgotten in the original commit.
Discussion: https://postgr.es/m/CAHewXN%3D%2Bghd6O6im46q7j2u6c3H6vkXtXmF%3D_v4CfGSnjje8PA%40mail.gmail.com
Author: Tender Wang <tndrwang@gmail.com>
M src/backend/utils/misc/postgresql.conf.sample
Compute CRC32C using AVX-512 instructions where available
commit : 3c6e8c123896584f1be1fe69aaf68dcb5eb094d5
author : John Naylor <john.naylor@postgresql.org>
date : Sun, 6 Apr 2025 14:04:30 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Sun, 6 Apr 2025 14:04:30 +0700
The previous implementation of CRC32C on x86 relied on the native
CRC32 instruction from the SSE 4.2 extension, which operates on
up to 8 bytes at a time. We can get a substantial speedup by using
carryless multiplication on SIMD registers, processing 64 bytes per
loop iteration. Shorter inputs fall back to ordinary CRC instructions.
On Intel Tiger Lake hardware (2020), CRC is now 50% faster for inputs
between 64 and 112 bytes, and 3x faster for 256 bytes.
The VPCLMULQDQ instruction on 512-bit registers has been available
on Intel hardware since 2019 and AMD since 2022. There is an older
variant for 128-bit registers, but at least on Zen 2 it performs worse
than normal CRC instructions for short inputs.
We must now do a runtime check, even for builds that target SSE
4.2. This doesn't matter in practice for WAL (arguably the most
critical case), because since commit e2809e3a1 the final computation
with the 20-byte WAL header is inlined and unrolled when targeting
that extension. Compared with two direct function calls, testing
showed equal or slightly faster performance in performing an indirect
function call on several dozen bytes followed by inlined instructions
on constant input of 20 bytes.
The MIT-licensed implementation was generated with the "generate"
program from
https://github.com/corsix/fast-crc32/
Based on: "Fast CRC Computation for Generic Polynomials Using PCLMULQDQ
Instruction" V. Gopal, E. Ozturk, et al., 2009
Co-authored-by: Raghuveer Devulapalli <raghuveer.devulapalli@intel.com>
Co-authored-by: Paul Amonson <paul.d.amonson@intel.com>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de> (earlier version)
Reviewed-by: Matthew Sterrett <matthewsterrett2@gmail.com> (earlier version)
Tested-by: Raghuveer Devulapalli <raghuveer.devulapalli@intel.com>
Tested-by: David Rowley <<dgrowleyml@gmail.com>> (earlier version)
Discussion: https://postgr.es/m/BL1PR11MB530401FA7E9B1CA432CF9DC3DC192@BL1PR11MB5304.namprd11.prod.outlook.com
Discussion: https://postgr.es/m/PH8PR11MB82869FF741DFA4E9A029FF13FBF72@PH8PR11MB8286.namprd11.prod.outlook.com
M config/c-compiler.m4
M configure
M configure.ac
M meson.build
M src/include/pg_config.h.in
M src/include/port/pg_crc32c.h
M src/port/meson.build
M src/port/pg_crc32c_sse42.c
M src/port/pg_crc32c_sse42_choose.c
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
Quote filename in error message
commit : 683df3f4de00bf50b20eae92369e006badf7cd57
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sat, 5 Apr 2025 22:10:28 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sat, 5 Apr 2025 22:10:28 +0200
Project standard is to quote filenames in error and log messages, which
commit 2da74d8d640 missed in two error messages.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reported-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/20250404.120328.103562371975971823.horikyota.ntt@gmail.com
M src/interfaces/libpq/fe-secure-openssl.c
Fix parse_cte.c's failure to examine sub-WITHs in DML statements.
commit : 691836405f1ecae2368a0f2fdbfee3b2cb53375b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Apr 2025 15:01:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Apr 2025 15:01:33 -0400
makeDependencyGraphWalker thought that only SelectStmt nodes could
contain a WithClause. Which was true in our original implementation
of WITH, but astonishingly we missed updating this code when we added
the ability to attach WITH to INSERT/UPDATE/DELETE (and later MERGE).
Moreover, since it was coded to deliberately block recursion to a
WithClause, even updating raw_expression_tree_walker didn't save it.
The upshot of this was that we didn't see references to outer CTE
names appearing within an inner WITH, and would neither complain about
disallowed recursion nor account for such references when sorting CTEs
into a usable order. The lack of complaints about this is perhaps not
so surprising, because typical usage of WITH wouldn't hit either case.
Still, it's pretty broken; failing to detect recursion here leads to
assert failures or worse later on.
Fix by factoring out the processing of sub-WITHs into a new function
WalkInnerWith, and invoking that for all the statement types that
can have WITH.
Bug: #18878
Reported-by: Yu Liang <luy70@psu.edu>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18878-a26fa5ab6be2f2cf@postgresql.org
Backpatch-through: 13
M src/backend/parser/parse_cte.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Add modern SHA-2 based password hashes to pgcrypto.
commit : 749a9e20c9790006f3af47f7a8faf4ad8dc358d9
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 5 Apr 2025 19:16:58 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 5 Apr 2025 19:16:58 +0200
This adapts the publicly available reference implementation on
https://www.akkadia.org/drepper/SHA-crypt.txt and adds the new hash
algorithms sha256crypt and sha512crypt to crypt() and gen_salt()
respectively.
Author: Bernd Helmle <mailings@oopsware.de>
Reviewed-by: Japin Li <japinli@hotmail.com>
Discussion: https://postgr.es/m/c763235a2757e2f5f9e3e27268b9028349cef659.camel@oopsware.de
M contrib/pgcrypto/Makefile
M contrib/pgcrypto/crypt-gensalt.c
A contrib/pgcrypto/crypt-sha.c
A contrib/pgcrypto/expected/crypt-shacrypt.out
M contrib/pgcrypto/meson.build
M contrib/pgcrypto/px-crypt.c
M contrib/pgcrypto/px-crypt.h
A contrib/pgcrypto/sql/crypt-shacrypt.sql
M doc/src/sgml/pgcrypto.sgml
Avoid double transformation of json_array()'s subquery.
commit : e33f2335a9d9754ccf3bf7181085cfa581ee03c3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Apr 2025 12:13:35 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Apr 2025 12:13:35 -0400
transformJsonArrayQueryConstructor() applied transformStmt() to
the same subquery tree twice. While this causes no issue in many
cases, there are some where it causes a coredump, thanks to the
parser's habit of scribbling on its input.
Fix by making a copy before the first transformation (compare
0f43083d1). This is quite brute-force, but then so is the
whole business of transforming the input twice. Per discussion
in the bug thread, this implementation of json_array() parsing
should be replaced completely. But that will take some work
and will surely not be back-patchable, so for the moment let's
take the easy way out.
Oversight in 7081ac46a. Back-patch to v16 where that came in.
Bug: #18877
Reported-by: Yu Liang <luy70@psu.edu>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18877-c3c3ad75845833bb@postgresql.org
Backpatch-through: 16
M src/backend/parser/parse_expr.c
M src/test/regress/expected/sqljson.out
M src/test/regress/sql/sqljson.sql
Clean up from commit 1495eff7bdb
commit : 5db3bf7391d77ae86bc9b5f580141e022803b744
author : Andrew Dunstan <andrew@dunslane.net>
date : Sat, 5 Apr 2025 08:00:24 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sat, 5 Apr 2025 08:00:24 -0400
Fix some comments, and remove the hacky way of quoting database names in
favor of appendStringLiteralConn.
M src/bin/pg_dump/pg_restore.c
Set log_statement=none in t/002_pg_upgrade.pl
commit : 64fba9c61787c98d019c3b5099e21286b61a56d5
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 5 Apr 2025 11:41:01 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 5 Apr 2025 11:41:01 +0200
This should make the test a wee bit faster on high-load machines (e.g.,
when running under valgrind).
Per complaint from Andres Freund.
Discussion: https://postgr.es/m/cwbcyjp2ts7o7xgy5y5gwtcd4zltvncsj67el7xgci7xbwrhlu@k363vk5tce4g
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
pg_dump: Tiny header cleanup
commit : 4be6a74cfb1ef4c50740e4c2f97a3b478c16de73
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 5 Apr 2025 11:22:40 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 5 Apr 2025 11:22:40 +0200
In commits 9c02e3a986da and 8ec0aaeae094, Nathan added a duplicate
TocEntry typedef forward declaration (plus assorted #ifdef hackery to
avoid C99 preprocessor issues) to deal with some very old untidyness
regarding DefnDumperPtr function prototype being located in pg_backup.h.
But there's no reason to have the DefnDumperPtr typedef (and the
accompanying DataDumperPtr typedef) in that file at all; they are better
placed in pg_backup_archiver.h, the internal header, because they are
only used internally. That also requires zero #ifdef hackery, so move
them there.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/202504042140.qo66ggw6wzsz@alvherre.pgsql
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.h
pg_dump: Fix query for gathering attribute stats on older versions.
commit : f0d0083f52f994989b1bb6723e7bd0fad9f586cd
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 21:05:30 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 21:05:30 -0500
Commit 9c02e3a986 taught pg_dump to retrieve attribute statistics
for 64 relations at a time. pg_dump supports dumping from v9.2 and
newer versions, but our query for retrieving statistics for
multiple relations uses WITH ORDINALITY and multi-argument
UNNEST(), both of which were introduced in v9.4. To fix, we resort
to gathering statistics for a single relation at a time on versions
older than v9.4.
Per buildfarm member crake.
Author: Corey Huinker <corey.huinker@gmail.com>
Discussion: https://postgr.es/m/Z_BcWVMvlUIJ_iuZ%40nathan
M src/bin/pg_dump/pg_dump.c
Repair misbehavior with duplicate entries in FK SET column lists.
commit : 43b8e6c4abfb632bbff5b3a26dc39bf3f188afff
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Apr 2025 20:11:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Apr 2025 20:11:48 -0400
Since v15 we've had an option to apply a foreign key constraint's
ON DELETE SET DEFAULT or SET NULL action to just some of the
referencing columns. There was not a check for duplicate entries in
the list of columns-to-set, though. That caused a potential memory
stomp in CreateConstraintEntry(), which incautiously assumed that
the list of columns-to-set couldn't be longer than the number of key
columns. Even after fixing that, the case doesn't work because you
get an error like "multiple assignments to same column" from the SQL
command that is generated to do the update.
We could either raise an error for duplicate columns or silently
suppress the dups, and after a bit of thought I chose to do the
latter. This is motivated by the fact that duplicates in the FK
column list are legal, so it's not real clear why duplicates
in the columns-to-set list shouldn't be. Of course there's no
need to actually set the column more than once.
I left in the fix in CreateConstraintEntry() too, just because
it didn't seem like such low-level code ought to be making
assumptions about what it's handed.
Bug: #18879
Reported-by: Yu Liang <luy70@psu.edu>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18879-259fc59d072bd4d7@postgresql.org
Backpatch-through: 15
M src/backend/catalog/pg_constraint.c
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
functions.c: copy trees from source_list before parse analysis etc.
commit : 0f43083d16f4be7c01efa80d05d0eef5e5ff69d3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Apr 2025 18:26:51 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Apr 2025 18:26:51 -0400
This is yet another bit of fallout from the fact that backend/parser
(like other code) feels free to scribble on the parse tree it's
handed. In this case that resulted in modifying the
relatively-short-lived copy in the cached function's source_list.
That would be fine since we only need each source_list tree once
... except that if the parser fails after making some changes,
the function cache entry remains as-is and will still be there
if the user tries to execute the function again. Then we have
problems because we're feeding a non-pristine tree to the parser.
The most expedient fix is a quick copyObject(). I considered
other answers like somehow marking the cache entry invalid
temporarily, but that would add complexity and I'm not sure
it's worth it. In typical scenarios we'd only do this once
per function query per session.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/6d442183-102c-498a-81d1-eeeb086cdc5a@gmail.com
M src/backend/executor/functions.c
Fix a couple of error messages and tests for them
commit : 2ef57908067ab29c22ae13f7775fe2afc330e8f6
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 4 Apr 2025 17:02:02 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 4 Apr 2025 17:02:02 -0400
oversights in 1495eff7bdb and 289f74d0cb2. Mea culpa.
M src/bin/pg_dump/pg_restore.c
M src/bin/pg_dump/t/001_basic.pl
M src/bin/pg_dump/t/006_pg_dumpall.pl
Prevent redeclaration of typedef TocEntry.
commit : 8ec0aaeae09482925d2d15ce4a91f6953bdb1566
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 15:56:23 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 15:56:23 -0500
Commit 9c02e3a986 added a forward declaration for this typedef that
caused redeclarations, which is not valid in C99. To fix, add some
preprocessor guards to avoid a redefinition, as is done elsewhere
(e.g., commit 382092a0cd).
Per buildfarm.
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.h
Add more TAP tests for pg_dumpall
commit : 289f74d0cb247ebdb443fec65fb2500531c961b4
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 3 Apr 2025 14:45:52 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 3 Apr 2025 14:45:52 -0400
Author: Matheus Alcantara <matheusssilv97@gmail.com>
Author: Mahendra Singh Thalor <mahi6run@gmail.com>
M src/bin/pg_dump/meson.build
M src/bin/pg_dump/t/001_basic.pl
A src/bin/pg_dump/t/006_pg_dumpall.pl
Non text modes for pg_dumpall, correspondingly change pg_restore
commit : 1495eff7bdb0779cc54ca04f3bd768f647240df2
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 4 Apr 2025 10:05:38 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 4 Apr 2025 10:05:38 -0400
pg_dumpall acquires a new -F/--format option, with the same meanings as
pg_dump. The default is p, meaning plain text. For any other value, a
directory is created containing two files, globals.data and map.dat. The
first contains SQL for restoring the global data, and the second
contains a map from oids to database names. It will also contain a
subdirectory called databases, inside which it will create archives in
the specified format, named using the database oids.
In these casess the -f argument is required.
If pg_restore encounters a directory containing globals.dat, and no
toc.dat, it restores the global settings and then restores each
database.
pg_restore acquires two new options: -g/--globals-only which suppresses
restoration of any databases, and --exclude-database which inhibits
restoration of particualr database(s) in the same way the same option
works in pg_dumpall.
Author: Mahendra Singh Thalor <mahi6run@gmail.com>
Co-authored-by: Andrew Dunstan <andrew@dunslane.net>
Reviewed-by: jian he <jian.universality@gmail.com>
Reviewed-by: Srinath Reddy <srinath2133@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/cb103623-8ee6-4ba5-a2c9-f32e3a4933fa@dunslane.net
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_restore.sgml
M src/bin/pg_dump/parallel.c
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_tar.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_dump/pg_restore.c
M src/bin/pg_dump/t/001_basic.pl
add new list type simple_oid_string_list to fe-utils/simple_list
commit : 2b69afbe50d5e39cc7d9703b3ab7acc4495a54ea
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 28 Mar 2025 18:10:24 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 28 Mar 2025 18:10:24 -0400
This type contains both an oid and a string.
This will be used in forthcoming changes to pg_restore.
Author: Andrew Dunstan <andrew@dunslane.net>
M src/fe_utils/simple_list.c
M src/include/fe_utils/simple_list.h
M src/tools/pgindent/typedefs.list
Move common pg_dump code related to connections to a new file
commit : c1da7281060d646f863e920a1aac3b9dbc997672
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 4 Apr 2025 10:04:35 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 4 Apr 2025 10:04:35 -0400
ConnectDatabase is used by pg_dumpall, pg_restore and pg_dump so move
common code to new file.
new file name: connectdb.c
Author: Mahendra Singh Thalor <mahi6run@gmail.com>
M src/bin/pg_dump/Makefile
A src/bin/pg_dump/connectdb.c
A src/bin/pg_dump/connectdb.h
M src/bin/pg_dump/meson.build
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
Remove unused function parameters in pg_backup_archiver.c.
commit : ff3a7f0b686068d093933a7bdedca1fa57fc5158
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 14:55:04 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 14:55:04 -0500
Thanks to commit 9c02e3a986, which modified some of the changes
from commit a0a4601765, we can remove the now-unused ArchiveHandle
parameter from _tocEntryRestorePass() and move_to_ready_heap().
Reviewed-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://postgr.es/m/Z-3x2AnPCP331JA3%40nathan
M src/bin/pg_dump/pg_backup_archiver.c
pg_dump: Retrieve attribute statistics in batches.
commit : 9c02e3a986daa865ecdc2e3d8183e2d83b8f4824
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 14:51:08 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 14:51:08 -0500
Currently, pg_dump gathers attribute statistics with a query per
relation, which can cause pg_dump to take significantly longer,
especially when there are many relations. This commit addresses
this by teaching pg_dump to gather attribute statistics for 64
relations at a time. Some simple tests showed this was the optimal
batch size, but performance may vary depending on the workload.
Our lookahead code determines the next batch of relations by
searching the TOC sequentially for relevant entries. This approach
assumes that we will dump all such entries in TOC order, which
unfortunately isn't true for dump formats that use
RestoreArchive(). RestoreArchive() does multiple passes through
the TOC and selectively dumps certain groups of entries each time.
This is particularly problematic for index stats and a subset of
matview stats; both are in SECTION_POST_DATA, but matview stats
that depend on matview data are dumped in RESTORE_PASS_POST_ACL,
while all other stats are dumped in RESTORE_PASS_MAIN. To handle
this, this commit moves all statistics data entries in
SECTION_POST_DATA to RESTORE_PASS_POST_ACL, which ensures that we
always dump them in TOC order. A convenient side effect of this
change is that we can revert a decent chunk of commit a0a4601765,
but that is left for a follow-up commit.
Author: Corey Huinker <corey.huinker@gmail.com>
Co-authored-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://postgr.es/m/CADkLM%3Dc%2Br05srPy9w%2B-%2BnbmLEo15dKXYQ03Q_xyK%2BriJerigLQ%40mail.gmail.com
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_dump.c
pg_dump: Reduce memory usage of dumps with statistics.
commit : 7d5c83b4e90c7156655f98b7312a30ae5eeb4d27
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 14:51:08 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 14:51:08 -0500
Right now, pg_dump stores all generated commands for statistics in
memory. These commands can be quite large and therefore can
significantly increase pg_dump's memory footprint. To fix, wait
until we are about to write out the commands before generating
them, and be sure to free the commands after writing. This is
implemented via a new defnDumper callback that works much like the
dataDumper one but is specifically designed for TOC entries.
Custom dumps that include data might write the TOC twice (to update
data offset information), which would ordinarily cause pg_dump to
run the attribute statistics queries twice. However, as a hack, we
save the length of the written-out entry in the first pass and skip
over it in the second. While there is no known technical issue
with executing the queries multiple times and rewriting the
results, it's expensive and feels risky, so let's avoid it.
As an exception, we _do_ execute the queries twice for the tar
format. This format does a second pass through the TOC to generate
the restore.sql file. pg_restore doesn't use this file, so even if
the second round of queries returns different results than the
first, it won't corrupt the output; the archive and restore.sql
file will just have different content. A follow-up commit will
teach pg_dump to gather attribute statistics in batches, which our
testing indicates more than makes up for the added expense of
running the queries twice.
Author: Corey Huinker <corey.huinker@gmail.com>
Co-authored-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://postgr.es/m/CADkLM%3Dc%2Br05srPy9w%2B-%2BnbmLEo15dKXYQ03Q_xyK%2BriJerigLQ%40mail.gmail.com
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_dump.c
Skip second WriteToc() call for custom-format dumps without data.
commit : e3cc039a7d930ffe86e706944a2b3368bd3ef506
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 14:51:08 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 14:51:08 -0500
Presently, "pg_dump --format=custom" calls WriteToc() twice. The
second call updates the data offset information, which allegedly
makes parallel pg_restore significantly faster. However, if we're
not dumping any data, there are no data offsets to update, so we
can skip this step.
Reviewed-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://postgr.es/m/Z9c1rbzZegYQTOQE%40nathan
M src/bin/pg_dump/pg_backup_custom.c
Use streaming read I/O in autoprewarm
commit : d9c7911e1a5f354b7c34b7f53a52e0a690e7ebc9
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 4 Apr 2025 15:25:45 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 4 Apr 2025 15:25:45 -0400
Make a read stream for each valid fork of each valid relation
represented in the autoprewarm dump file and prewarm those blocks
through the read stream API instead of by directly invoking
ReadBuffer().
Co-authored-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Co-authored-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Andrey M. Borodin <x4mmm@yandex-team.ru> (earlier versions)
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com> (earlier versions)
Reviewed-by: Matheus Alcantara <mths.dev@pm.me> (earlier versions)
Discussion: https://postgr.es/m/flat/CAN55FZ3n8Gd%2BhajbL%3D5UkGzu_aHGRqnn%2BxktXq2fuds%3D1AOR6Q%40mail.gmail.com
M contrib/pg_prewarm/autoprewarm.c
M src/tools/pgindent/typedefs.list
Refactor autoprewarm_database_main() in preparation for read stream
commit : 6acab8bdbcda735ef47b1bb0ba2284d6c465cd88
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 4 Apr 2025 15:25:27 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 4 Apr 2025 15:25:27 -0400
Autoprewarm prewarms blocks from a dump file representing the contents
of shared buffers at the time it was dumped. It uses a sorted array of
BlockInfoRecords, each representing a block from one of the cluster's
databases and tables.
autoprewarm_database_main() prewarms all the blocks from a single
database. It is optimized to ensure we don't try to open the same
relation or fork over and over again if it has been dropped or is
invalid. The main loop handled this by carefully setting various local
variables to sentinel values when a run of blocks should be skipped.
This method won't work with the read stream API. The read stream
callback must be able to advance the current position in the
BlockInfoRecord array to allow for reading ahead additional blocks,
however a read stream maps 1-1 with a relation and fork combination. So,
the main loop in autoprewarm_database_main() must also advance the
position in the array of BlockInfoRecords to skip invalid relations and
forks. This split control doesn't fit well with the current flow control
in autoprewarm_database_main()
To make it compatible with the read stream API, change
autoprewarm_database_main() to explicitly fast-forward in the
BlockInfoRecords array past the blocks belonging to an invalid relation
or fork.
This commit only implements the new control flow -- it does not use the
read stream API.
Co-authored-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Co-authored-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/flat/CAN55FZ3n8Gd%2BhajbL%3D5UkGzu_aHGRqnn%2BxktXq2fuds%3D1AOR6Q%40mail.gmail.com
M contrib/pg_prewarm/autoprewarm.c
Remove superfluous autoprewarm check
commit : 7f848cb788ee05b9a4896ad2e19b9ee724577439
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 4 Apr 2025 15:25:17 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 4 Apr 2025 15:25:17 -0400
autoprewarm_database_main() prewarms blocks from the same database. It
is passed an array of sorted BlockInfoRecords and a start and stop index
into the array. The range represented should include only blocks
belonging to global objects or blocks from a single database. Remove an
unnecessary check that the current block is from the same database and
add an assert to ensure this invariant remains. Doing so removes a
special case that makes future refactoring to accommodate read
streamifying autoprewarm easier.
Noticed off-list by Andres Freund
M contrib/pg_prewarm/autoprewarm.c
Avoid extra index searches through preprocessing.
commit : b3f1a13f22f9e28842ee5fbd08b7ec805e27aaac
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 4 Apr 2025 14:14:08 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 4 Apr 2025 14:14:08 -0400
Transform low_compare and high_compare nbtree skip array inequalities
(with opclasses that offer skip support) in such a way as to allow
_bt_first to consistently apply later keys when it descends the tree.
This can lower the number of index searches for multi-column scans that
use a ">" key on one of the index's prefix columns (or use a "<" key,
when scanning backwards) when it precedes some later lower-order key.
For example, an index qual "WHERE a > 5 AND b = 2" will now be converted
to "WHERE a >= 6 AND b = 2" by a new preprocessing step that takes place
after low_compare and high_compare have been finalized. That way, the
initial call to _bt_first can use "WHERE a >= 6 AND b = 2" to find an
initial position, rather than just using "WHERE a > 5" -- "b = 2" can be
applied during every _bt_first call. There's a decent chance that this
will allow such a scan to avoid the extra search that might otherwise be
needed to determine the lowest "a" value still satisfying "WHERE a > 5".
The transformation process can only lower the total number of index
pages read when the use of a more restrictive set of initial positioning
keys in _bt_first actually allows the scan to land on some later leaf
page directly, relative to the unoptimized case (or on an earlier leaf
page directly, when scanning backwards). But the savings can really add
up in cases where an affected skip array comes after some other array.
For example, a scan indexqual "WHERE x IN (1, 2, 3) AND y > 5 AND z = 2"
can save as many as 3 _bt_first calls by applying the new transformation
to its "y" array (up to 1 extra search can be avoided per "x" element).
Follow-up to commit 92fe23d9, which added nbtree skip scan.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wz=FJ78K3WsF3iWNxWnUCY9f=Jdg3QPxaXE=uYUbmuRz5Q@mail.gmail.com
M src/backend/access/nbtree/nbtpreprocesskeys.c
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Improve nbtree skip scan primitive scan scheduling.
commit : 21a152b37f36c9563d1b0b058fe1436baf578b4c
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 4 Apr 2025 13:58:05 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 4 Apr 2025 13:58:05 -0400
Don't allow nbtree scans with skip arrays to end any primitive scan on
its first leaf page without giving some consideration to how many times
the scan's arrays advanced while changing at least one skip array
(though continue not caring about the number of array advancements that
only affected SAOP arrays, even during skip scans with SAOP arrays).
Now when a scan performs more than 3 such array advancements in the
course of reading a single leaf page, it is taken as a signal that the
next page is unlikely to be skippable. We'll therefore continue the
ongoing primitive index scan, at least until we can perform a recheck
against the next page's finaltup.
Testing has shown that this new heuristic occasionally makes all the
difference with skip scans that were expected to rely on the "passed
first page" heuristic added by commit 9a2e2a28. Without it, there is a
remaining risk that certain kinds of skip scans will never quite manage
to clear the initial hurdle of performing a primitive scan that lasts
beyond its first leaf page (or that such a skip scan will only clear
that initial hurdle when it has already wasted noticeably-many cycles
due to inefficient primitive scan scheduling).
Follow-up to commits 92fe23d9 and 9a2e2a28.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wz=RVdG3zWytFWBsyW7fWH7zveFvTHed5JKEsuTT0RCO_A@mail.gmail.com
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtutils.c
M src/include/access/nbtree.h
pg_recvlogical: Add --failover option.
commit : cf2655a9029aff63dd567dbbdcdee15ec969905d
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 4 Apr 2025 10:39:57 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 4 Apr 2025 10:39:57 -0700
This new option instructs pg_recvlogical to create the logical
replication slot with the failover option enabled. It can be used in
conjunction with the --create-slot option.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Michael Banck <mbanck@gmx.net>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/OSCPR01MB14966C54097FC83AF19F3516BF5AC2@OSCPR01MB14966.jpnprd01.prod.outlook.com
M doc/src/sgml/ref/pg_recvlogical.sgml
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_basebackup/pg_receivewal.c
M src/bin/pg_basebackup/pg_recvlogical.c
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_basebackup/streamutil.h
M src/bin/pg_basebackup/t/030_pg_recvlogical.pl
Oversight in commit b81ffa13e3.
commit : 3556c89321e8baa2242288bd4f015efd1e9d6be0
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 4 Apr 2025 10:28:52 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 4 Apr 2025 10:28:52 -0700
Should warn if a materialized view may be affected, as well.
M src/bin/pg_upgrade/check.c
Further optimize nbtree search scan key comparisons.
commit : 8a510275dd6b343b8e1646e4754078eab8ef3ab5
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 4 Apr 2025 12:27:52 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 4 Apr 2025 12:27:52 -0400
Postgres 17 commit e0b1ee17 added two complementary optimizations to
nbtree: the "prechecked" and "firstmatch" optimizations. _bt_readpage
was made to avoid needlessly evaluating keys that are guaranteed to be
satisfied by applying page-level context. "prechecked" did this for
keys required in the current scan direction, while "firstmatch" did it
for keys required in the opposite-to-scan direction only.
The "prechecked" design had a number of notable issues. It didn't
account for the fact that an = array scan key's sk_argument field might
need to advance at the point of the page precheck (it didn't check the
precheck tuple against the key's array, only the key's sk_argument,
which needlessly made it ineffective in cases involving stepping to a
page having advanced the scan's arrays using a truncated high key).
"prechecked" was also completely ineffective when only one scan key
wasn't guaranteed to be satisfied by every tuple (it didn't recognize
that it was still safe to avoid evaluating other, earlier keys).
The "firstmatch" optimization had similar limitations. It could only be
applied after _bt_readpage found its first matching tuple, regardless of
why any earlier tuples failed to satisfy the scan's index quals. This
allowed unsatisfied non-required scan keys to impede the optimization.
Replace both optimizations with a new optimization, without any of these
limitations: the "startikey" optimization. Affected _bt_readpage calls
generate a page-level key offset ("startikey"), that their _bt_checkkeys
calls can then start at. This is an offset to the first key that isn't
known to be satisfied by every tuple on the page.
Although this is independently useful work, its main goal is to avoid
performance regressions with index scans that use skip arrays, but still
never manage to skip over irrelevant leaf pages. We must avoid wasting
CPU cycles on overly granular skip array maintenance in these cases.
The new "startikey" optimization helps with this by selectively
disabling array maintenance for the duration of a _bt_readpage call.
This has no lasting consequences for the scan's array keys (they'll
still reliably track the scan's progress through the index's key space
whenever the scan is "between pages").
Skip scan adds skip arrays during preprocessing using simple, static
rules, and decides how best to navigate/apply the scan's skip arrays
dynamically, at runtime. The "startikey" optimization enables this
approach. As a result of all this, the planner doesn't need to generate
distinct, competing index paths (one path for skip scan, another for an
equivalent traditional full index scan). The overall effect is to make
scan runtime close to optimal, even when the planner works off an
incorrect cardinality estimate. Scans will also perform well given a
skipped column with data skew: individual groups of pages with many
distinct values (in respect of a skipped column) can be read about as
efficiently as before -- without the scan being forced to give up on
skipping over other groups of pages that are provably irrelevant.
Many scans that cannot possibly skip will still benefit from the use of
skip arrays, since they'll allow the "startikey" optimization to be as
effective as possible (by allowing preprocessing to mark all the scan's
keys as required). A scan that uses a skip array on "a" for a qual
"WHERE a BETWEEN 0 AND 1_000_000 AND b = 42" is often much faster now,
even when every tuple read by the scan has its own distinct "a" value.
However, there are still some remaining regressions, affecting certain
trickier cases.
Scans whose index quals have several range skip arrays, each on some
high cardinality column, can still be slower than they were before the
introduction of skip scan -- even with the new "startikey" optimization.
There are also known regressions affecting very selective index scans
that use a skip array. The underlying issue with such selective scans
is that they never get as far as reading a second leaf page, and so will
never get a chance to consider applying the "startikey" optimization.
In principle, all regressions could be avoided by teaching preprocessing
to not add skip arrays whenever they aren't expected to help, but it
seems best to err on the side of robust performance.
Follow-up to commit 92fe23d9, which added nbtree skip scan.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Reviewed-By: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wz=Y93jf5WjoOsN=xvqpMjRy-bxCE037bVFi-EasrpeUJA@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WznWDK45JfNPNvDxh6RQy-TaCwULaM5u5ALMXbjLBMcugQ@mail.gmail.com
M src/backend/access/nbtree/nbtpreprocesskeys.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtutils.c
M src/include/access/nbtree.h
Add nbtree skip scan optimization.
commit : 92fe23d93aa3bbbc40fca669cabc4a4d7975e327
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 4 Apr 2025 12:27:04 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 4 Apr 2025 12:27:04 -0400
Teach nbtree multi-column index scans to opportunistically skip over
irrelevant sections of the index given a query with no "=" conditions on
one or more prefix index columns. When nbtree is passed input scan keys
derived from a predicate "WHERE b = 5", new nbtree preprocessing steps
output "WHERE a = ANY(<every possible 'a' value>) AND b = 5" scan keys.
That is, preprocessing generates a "skip array" (and an output scan key)
for the omitted prefix column "a", which makes it safe to mark the scan
key on "b" as required to continue the scan. The scan is therefore able
to repeatedly reposition itself by applying both the "a" and "b" keys.
A skip array has "elements" that are generated procedurally and on
demand, but otherwise works just like a regular ScalarArrayOp array.
Preprocessing can freely add a skip array before or after any input
ScalarArrayOp arrays. Index scans with a skip array decide when and
where to reposition the scan using the same approach as any other scan
with array keys. This design builds on the design for array advancement
and primitive scan scheduling added to Postgres 17 by commit 5bf748b8.
Testing has shown that skip scans of an index with a low cardinality
skipped prefix column can be multiple orders of magnitude faster than an
equivalent full index scan (or sequential scan). In general, the
cardinality of the scan's skipped column(s) limits the number of leaf
pages that can be skipped over.
The core B-Tree operator classes on most discrete types generate their
array elements with the help of their own custom skip support routine.
This infrastructure gives nbtree a way to generate the next required
array element by incrementing (or decrementing) the current array value.
It can reduce the number of index descents in cases where the next
possible indexable value frequently turns out to be the next value
stored in the index. Opclasses that lack a skip support routine fall
back on having nbtree "increment" (or "decrement") a skip array's
current element by setting the NEXT (or PRIOR) scan key flag, without
directly changing the scan key's sk_argument. These sentinel values
behave just like any other value from an array -- though they can never
locate equal index tuples (they can only locate the next group of index
tuples containing the next set of non-sentinel values that the scan's
arrays need to advance to).
A skip array's range is constrained by "contradictory" inequality keys.
For example, a skip array on "x" will only generate the values 1 and 2
given a qual such as "WHERE x BETWEEN 1 AND 2 AND y = 66". Such a skip
array qual usually has near-identical performance characteristics to a
comparable SAOP qual "WHERE x = ANY('{1, 2}') AND y = 66". However,
improved performance isn't guaranteed. Much depends on physical index
characteristics.
B-Tree preprocessing is optimistic about skipping working out: it
applies static, generic rules when determining where to generate skip
arrays, which assumes that the runtime overhead of maintaining skip
arrays will pay for itself -- or lead to only a modest performance loss.
As things stand, these assumptions are much too optimistic: skip array
maintenance will lead to unacceptable regressions with unsympathetic
queries (queries whose scan can't skip over many irrelevant leaf pages).
An upcoming commit will address the problems in this area by enhancing
_bt_readpage's approach to saving cycles on scan key evaluation, making
it work in a way that directly considers the needs of = array keys
(particularly = skip array keys).
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Masahiro Ikeda <masahiro.ikeda@nttdata.com>
Reviewed-By: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Tomas Vondra <tomas@vondra.me>
Reviewed-By: Aleksander Alekseev <aleksander@timescale.com>
Reviewed-By: Alena Rybakina <a.rybakina@postgrespro.ru>
Discussion: https://postgr.es/m/CAH2-Wzmn1YsLzOGgjAQZdn1STSG_y8qP__vggTaPAYXJP+G4bw@mail.gmail.com
M doc/src/sgml/btree.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/indices.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/xindex.sgml
M src/backend/access/index/indexam.c
M src/backend/access/nbtree/nbtcompare.c
M src/backend/access/nbtree/nbtpreprocesskeys.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/access/nbtree/nbtvalidate.c
M src/backend/commands/opclasscmds.c
M src/backend/utils/adt/Makefile
M src/backend/utils/adt/date.c
M src/backend/utils/adt/meson.build
M src/backend/utils/adt/selfuncs.c
A src/backend/utils/adt/skipsupport.c
M src/backend/utils/adt/timestamp.c
M src/backend/utils/adt/uuid.c
M src/include/access/amapi.h
M src/include/access/nbtree.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_proc.dat
A src/include/utils/skipsupport.h
M src/test/regress/expected/alter_generic.out
M src/test/regress/expected/btree_index.out
M src/test/regress/expected/create_index.out
M src/test/regress/expected/psql.out
M src/test/regress/sql/alter_generic.sql
M src/test/regress/sql/btree_index.sql
M src/test/regress/sql/create_index.sql
M src/tools/pgindent/typedefs.list
Stabilize regression test from c0962a113.
commit : 3ba2cdaa45416196fadc7163998cda7b4e07e7d7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Apr 2025 11:57:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Apr 2025 11:57:26 -0400
Per buildfarm.
Co-authored-by: Alena Rybakina <a.rybakina@postgrespro.ru>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/srnuqlttuimzmvoulhsrbgvj4vnul6b65osswvua7sfkqsvmuy@yg7apybpxp34
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
Fix autoprewarm neglect of tablespaces
commit : 64e7fa43a948a82f97e305f020f924c02671a434
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 4 Apr 2025 11:34:06 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 4 Apr 2025 11:34:06 -0400
While prewarming blocks from a dump file, autoprewarm_database_main()
mistakenly ignored tablespace when detecting the beginning of the next
relation to prewarm. Because RelFileNumbers are only unique within a
tablespace, autoprewarm could miss prewarming blocks from a
relation with the same RelFileNumber in a different tablespace.
Though this situation is likely rare in practice, it's best to make the
code correct. Do so by explicitly checking for the RelFileNumber when
detecting a new relation.
Reported-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/97c36982-603b-494a-95f4-aaf2a12ac27e%40iki.fi
M contrib/pg_prewarm/autoprewarm.c
Add commit e1a8b1ad58 to .git-blame-ignore-revs.
commit : 742317a80f89a5d6476c20b3d07330f3ec0d4357
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 09:41:59 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 09:41:59 -0500
M .git-blame-ignore-revs
Re-pgindent pg_largeobject.c after commit 0d6c477664.
commit : e1a8b1ad587112e67fdc5aa7b388631dde4dbdda
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 09:38:22 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 4 Apr 2025 09:38:22 -0500
M src/backend/catalog/pg_largeobject.c
Convert 'x IN (VALUES ...)' to 'x = ANY ...' then appropriate
commit : c0962a113d1f2f94cb7222a7ca025a67e9ce3860
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 4 Apr 2025 16:01:50 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 4 Apr 2025 16:01:50 +0300
This commit implements the automatic conversion of 'x IN (VALUES ...)' into
ScalarArrayOpExpr. That simplifies the query tree, eliminating the appearance
of an unnecessary join.
Since VALUES describes a relational table, and the value of such a list is
a table row, the optimizer will likely face an underestimation problem due to
the inability to estimate cardinality through MCV statistics. The cardinality
evaluation mechanism can work with the array inclusion check operation.
If the array is small enough (< 100 elements), it will perform a statistical
evaluation element by element.
We perform the transformation in the convert_ANY_sublink_to_join() if VALUES
RTE is proper and the transformation is convertible. The conversion is only
possible for operations on scalar values, not rows. Also, we currently
support the transformation only when it ends up with a constant array.
Otherwise, the evaluation of non-hashed SAOP might be slower than the
corresponding Hash Join with VALUES.
Discussion: https://postgr.es/m/0184212d-1248-4f1f-a42d-f5cb1c1976d2%40tantorlabs.com
Author: Alena Rybakina <a.rybakina@postgrespro.ru>
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Ivan Kush <ivan.kush@tantorlabs.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/optimizer/util/clauses.c
M src/include/optimizer/subselect.h
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
Extract make_SAOP_expr() function from match_orclause_to_indexcol()
commit : d48d2e2dc8be50d3ca13305b5699384329b15433
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 4 Apr 2025 16:01:28 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 4 Apr 2025 16:01:28 +0300
This commit extracts the code to generate ScalarArrayOpExpr on top of the list
of expressions from match_orclause_to_indexcol() into a separate function
make_SAOP_expr(). This function was extracted to be used in optimization for
conversion of 'x IN (VALUES ...)' to 'x = ANY ...'. make_SAOP_expr() is
placed in clauses.c file as only two additional headers were needed there
compared with other places.
Discussion: https://postgr.es/m/0184212d-1248-4f1f-a42d-f5cb1c1976d2%40tantorlabs.com
Author: Alena Rybakina <a.rybakina@postgrespro.ru>
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Ivan Kush <ivan.kush@tantorlabs.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/util/clauses.c
M src/include/optimizer/optimizer.h
Fix crash/valgrind error
commit : ee1ae8b99f964ca1edc038dd558e541832164d52
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 4 Apr 2025 14:45:53 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 4 Apr 2025 14:45:53 +0200
Fix for commit 9ef1851685b: We have to skip indexes where sortopfamily
is NULL. This takes the place of the previous btree check. Detected
by valgrind on the buildfarm.
M src/backend/utils/adt/selfuncs.c
docs: Clarify that NULL arg to set_config() means reset to default
commit : b4f453f6ab71eb72cb3a46a5be18e07e5cc285df
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 4 Apr 2025 15:17:17 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 4 Apr 2025 15:17:17 +0300
Author: David G. Johnston <david.g.johnston@gmail.com>
Reviewed-by: Zhang Mingli <zmlpostgres@gmail.com>
Discussion: https://www.postgresql.org/message-id/CAKFQuwY0SK6JdCci1VJX6xsztRXgGeVEY-grkENZx%2B3CZpyPcQ@mail.gmail.com
M doc/src/sgml/func.sgml
Relax assertion in finding correct GiST parent
commit : 7afca7edef751b8d7c0f5b6402ffcefc11c67fdd
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 4 Apr 2025 13:49:00 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 4 Apr 2025 13:49:00 +0300
Commit 28d3c2ddcf introduced an assertion that if the memorized
downlink location in the insertion stack isn't valid, the parent's
LSN should've changed too. Turns out that was too strict. In
gistFindCorrectParent(), if we walk right, we update the parent's
block number and clear its memorized 'downlinkoffnum'. That triggered
the assertion on next call to gistFindCorrectParent(), if the parent
needed to be split too. Relax the assertion, so that it's OK if
downlinkOffnum is InvalidOffsetNumber.
Backpatch to v13-, all supported versions. The assertion was added in
commit 28d3c2ddcf in v12.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://www.postgresql.org/message-id/18396-03cac9beb2f7aac3@postgresql.org
M src/backend/access/gist/gist.c
Allow "COPY table TO" command to copy rows from materialized views.
commit : 534874fac0b34535c9a5ab9257d6574f78423578
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 4 Apr 2025 19:32:00 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 4 Apr 2025 19:32:00 +0900
Previously, "COPY table TO" command worked only with plain tables and
did not support materialized views, even when they were populated and
had physical storage. To copy rows from materialized views,
"COPY (query) TO" command had to be used, instead.
This commit extends "COPY table TO" to support populated materialized
views directly, improving usability and performance, as "COPY table TO"
is generally faster than "COPY (query) TO". Note that copying from
unpopulated materialized views will still result in an error.
Author: jian he <jian.universality@gmail.com>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/CACJufxHVxnyRYy67hiPePNCPwVBMzhTQ6FaL9_Te5On9udG=yg@mail.gmail.com
M doc/src/sgml/ref/copy.sgml
M src/backend/commands/copyto.c
M src/test/regress/expected/copy.out
M src/test/regress/sql/copy.sql
Support non-btree indexes in get_actual_variable_range()
commit : 9ef1851685b718264de47bf543505cf3ec25aaea
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 4 Apr 2025 12:10:28 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 4 Apr 2025 12:10:28 +0200
This was previously not supported because the btree strategy numbers
were hardcoded. Now we can support this for any index that has the
required strategy mapping support and the required operators.
If an index scan used for get_actual_variable_range() requires
recheck, we now just ignore it instead of erroring out. With btree we
knew this couldn't happen, but now it might.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Co-authored-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/utils/adt/selfuncs.c
Extend ALTER DEFAULT PRIVILEGES to define default privileges for large objects.
commit : 0d6c4776647feeee26f3e29fff6a5edb222fa260
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 4 Apr 2025 19:02:17 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 4 Apr 2025 19:02:17 +0900
Previously, ALTER DEFAULT PRIVILEGES did not support large objects.
This meant that to grant privileges to users other than the owner,
permissions had to be manually assigned each time a large object
was created, which was inconvenient.
This commit extends ALTER DEFAULT PRIVILEGES to allow defining default
access privileges for large objects. With this change, specified privileges
will automatically apply to newly created large objects, making privilege
management more efficient.
As a side effect, this commit introduces the new keyword OBJECTS
since it's used in the syntax of ALTER DEFAULT PRIVILEGES.
Original patch by Haruka Takatsuka, with some fixes and tests by Yugo Nagata,
and rebased by Laurenz Albe.
Author: Takatsuka Haruka <harukat@sraoss.co.jp>
Co-authored-by: Yugo Nagata <nagata@sraoss.co.jp>
Co-authored-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Masao Fujii <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/20240424115242.236b499b2bed5b7a27f7a418@sraoss.co.jp
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/alter_default_privileges.sgml
M src/backend/catalog/aclchk.c
M src/backend/catalog/objectaddress.c
M src/backend/catalog/pg_largeobject.c
M src/backend/parser/gram.y
M src/bin/pg_dump/dumputils.c
M src/bin/pg_dump/pg_dump.c
M src/bin/psql/describe.c
M src/bin/psql/tab-complete.in.c
M src/include/catalog/pg_default_acl.h
M src/include/parser/kwlist.h
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Use standard die() signal handler in walreceiver
commit : 6e9c81836e101bc7f37ddc5e2f6ab58d62efcb24
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 4 Apr 2025 12:31:00 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 4 Apr 2025 12:31:00 +0300
This gets rid of the bespoken ProcessWalRcvInterrupts() function,
which lets walreceiver terminate at any CHECK_FOR_INTERRUPTS() call.
And it's less code anyway.
We can now use the standard libpqsrv_connect_params() libpq wrapper
from libpq-be-fe-helpers.h, removing more code. We attempted to do
that earlier already in commit 728f86fec6, but that was reverted
because it didn't call ProcessWalRcvInterrupts() and therefore didn't
react to shutdown requests. Now that ProcessWalRcvInterrupts() is
gone, it works. As stated in that commit, this also leads to
libpqwalreceiver reserving file descriptors for libpq conncetions,
which is nice.
Author: Andres Freund <andres@anarazel.de> (the earlier commit)
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Yura Sokolov <y.sokolov@postgrespro.ru>
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/walreceiver.c
M src/backend/tcop/postgres.c
M src/include/replication/walreceiver.h
Convert PathKey to use CompareType
commit : 8123e91f5aeb26c6e4cf583bb61c99281485af83
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 4 Apr 2025 11:13:54 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 4 Apr 2025 11:13:54 +0200
Change the PathKey struct to use CompareType to record the sort
direction instead of hardcoding btree strategy numbers. The
CompareType is then converted to the index-type-specific strategy when
the plan is created.
This reduces the number of places btree strategy numbers are
hardcoded, and it's a self-contained subset of a larger effort to
allow non-btree indexes to behave like btrees.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Co-authored-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/path/pathkeys.c
M src/backend/optimizer/plan/createplan.c
M src/include/nodes/pathnodes.h
M src/include/optimizer/paths.h
doc: Clarify the system value for sslrootcert
commit : daa16893faa96246d758eb2cc27e2f75c5308296
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 4 Apr 2025 09:47:36 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 4 Apr 2025 09:47:36 +0200
The documentation for the special value "system" for sslrootcert could
be misinterpreted to mean the default operating system CA store, which
it may be, but it's defined to be the default CA store of the SSL lib
used.
Backpatch down to v16 where support for the system value was added.
Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: George MacKerron <george@mackerron.co.uk>
Discussion: https://postgr.es/m/B3CBBAA3-6EA3-4AB7-8619-4BBFAB93DDB4@yesql.se
Backpatch-through: 16
M doc/src/sgml/libpq.sgml
M doc/src/sgml/runtime.sgml
pg_createsubscriber: Improve error messages.
commit : 898c131b58a0b62833e10f3556f993d0d385f941
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 4 Apr 2025 10:58:59 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 4 Apr 2025 10:58:59 +0530
Consistently, an option name is used in the error messages where
applicable. Also, change the code to use pg_fatal() instead of a
combination of pg_log_error() and exit().
Author: vignesh C <vignesh21@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CALDaNm0HxF1RH27LP7VisLzNsSJbssy8a64M5p6UduDaBq6-ag@mail.gmail.com
M src/bin/pg_basebackup/pg_createsubscriber.c
M src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
Fix logical decoding test to correctly check slot removal on standby.
commit : d5d85f1881615f8fb9ee9ecfed4d77f550ec1c75
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 4 Apr 2025 13:32:46 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 4 Apr 2025 13:32:46 +0900
The regression test for logical decoding verifies whether a logical slot
is correctly dropped on a standby when its associated database is dropped.
However, the test mistakenly retrieved slot information from the primary
instead of the standby, causing incorrect behavior.
This commit fixes the issue by ensuring the test correctly checks the slot
on the standby.
Back-patch to all supported versions.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/1fdfd020-a509-403c-bd8f-a04664aba148@oss.nttdata.com
Backpatch-through: 13
M src/test/recovery/t/010_logical_decoding_timelines.pl
Fix logical decoding regression tests to correctly check slot existence.
commit : c754bdd8a2e5ae388d48dc2a219f9f6b65e3ca3a
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 4 Apr 2025 13:09:06 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 4 Apr 2025 13:09:06 +0900
The regression tests for logical decoding verify whether a logical slot
exists or has been dropped. Previously, these tests attempted to
retrieve "slot_name" from the result of slot(), but since "slot_name" was
not included in the result, slot()->{'slot_name'} always returned undef,
leading to incorrect behavior.
This commit fixes the issue by checking the "plugin" field in the result
of slot() instead, ensuring the tests properly verify slot existence.
Back-patch to all supported versions.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/OSCPR01MB149667EC4E738769CA80B7EA5F5AE2@OSCPR01MB14966.jpnprd01.prod.outlook.com
Backpatch-through: 13
M src/test/recovery/t/006_logical_decoding.pl
M src/test/recovery/t/010_logical_decoding_timelines.pl
Revert "Improve accounting for memory used by shared hash tables"
commit : 1aff1dc8dfa7584ddbc695e1aae0f767db66326f
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 4 Apr 2025 04:28:59 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 4 Apr 2025 04:28:59 +0200
This reverts commit f5930f9a98ea65d659d41600a138e608988ad122.
This broke the expansion of private hash tables, which reallocates the
directory. But that's impossible when it's allocated together with the
other fields, and dir_realloc() failed with BogusFree. Clearly, this
needs rethinking.
Discussion: https://postgr.es/m/CAApHDvriCiNkm=v521AP6PKPfyWkJ++jqZ9eqX4cXnhxLv8w-A@mail.gmail.com
M src/backend/storage/ipc/shmem.c
M src/backend/utils/hash/dynahash.c
M src/include/utils/hsearch.h
Make derived clause lookup in EquivalenceClass more efficient
commit : 88f55bc97622bce000a8c90f8ef58dacc926de19
author : Amit Langote <amitlan@postgresql.org>
date : Fri, 4 Apr 2025 10:41:17 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Fri, 4 Apr 2025 10:41:17 +0900
Derived clauses are stored in ec_derives, a List of RestrictInfos.
These clauses are later looked up by matching the left and right
EquivalenceMembers along with the clause's parent EC.
This linear search becomes expensive in queries with many joins or
partitions, where ec_derives may contain thousands of entries. In
particular, create_join_clause() can spend significant time scanning
this list.
To improve performance, introduce a hash table (ec_derives_hash) that
is built when the list reaches 32 entries -- the same threshold used
for join_rel_hash. The original list is retained alongside the hash
table to support EC merging and serialization
(_outEquivalenceClass()).
Each clause is stored in the hash table using a canonicalized key: the
EquivalenceMember with the lower memory address is placed in the key
before the one with the higher memory address. This avoids storing or
searching for both permutations of the same clause. For clauses
involving a constant EM, the key places NULL in the first slot and the
non-constant EM in the second.
The hash table is initialized using list_length(ec_derives_list) as
the size hint. simplehash internally adjusts this to the next power of
two after dividing by the fillfactor, so this typically results in at
least 64 buckets near the threshold -- avoiding immediate resizing
while adapting to the actual number of entries.
The lookup logic for derived clauses is now centralized in
ec_search_derived_clause_for_ems(), which consults the hash table when
available and falls back to the list otherwise.
The new ec_clear_derived_clauses() always frees ec_derives_list, even
though some of the original code paths that cleared the old
ec_derives field did not. This ensures consistent cleanup and avoids
leaking memory when large lists are discarded.
An assertion originally placed in find_derived_clause_for_ec_member()
is moved into ec_search_derived_clause_for_ems() so that it is
enforced consistently, regardless of whether the hash table or list is
used for lookup.
This design incorporates suggestions by David Rowley, who proposed
both the key canonicalization and the initial sizing approach to
balance memory usage and CPU efficiency.
Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Tested-by: Dmitry Dolgov <9erthalion6@gmail.com>
Tested-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Tested-by: Amit Langote <amitlangote09@gmail.com>
Tested-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAExHW5vZiQtWU6moszLP5iZ8gLX_ZAUbgEX0DxGLx9PGWCtqUg@mail.gmail.com
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/plan/analyzejoins.c
M src/include/nodes/pathnodes.h
M src/include/optimizer/paths.h
M src/tools/pgindent/typedefs.list
Add assertion to verify derived clause has constant RHS
commit : 887160d1beaec607187261ab1a9411040da3d7b5
author : Amit Langote <amitlan@postgresql.org>
date : Fri, 4 Apr 2025 10:40:04 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Fri, 4 Apr 2025 10:40:04 +0900
find_derived_clause_for_ec_member() searches for a previously-derived
clause that equates a non-constant EquivalenceMember to a constant.
It is only called for EquivalenceClasses with ec_has_const set, and
with a non-constant member the EquivalenceMember to search for.
The matched clause is expected to have the non-constant member on the
left-hand side and the constant EquivalenceMember on the right.
Assert that the RHS is indeed a constant, to catch violations of this
structure and enforce assumptions made by
generate_base_implied_equalities_const().
Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/CAExHW5scMxyFRqOFE6ODmBiW2rnVBEmeEcA-p4W_CyuEikURdA@mail.gmail.com
M src/backend/optimizer/path/equivclass.c
Use AIO batchmode for bitmap heap scans
commit : 67be093562b6b345c170417312dff22f467055ba
author : Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 3 Apr 2025 18:23:02 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 3 Apr 2025 18:23:02 -0400
Previously bitmap heap scan was not AIO batchmode safe because of the
visibility map reads potentially done for the "skip fetch" optimization
(which skipped fetching tuples from the heap if the pages were all
visible and none of the columns were used in the query).
The skip fetch optimization implementation was found to have bugs and
was removed in 459e7bf8e2f8, so we can safely enable batchmode for
bitmap heap scans.
M src/backend/access/heap/heapam.c
Remove misleading read stream asserts in a few users
commit : 54a3615f15b9b2e1433bc47cfb150317885d7c1f
author : Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 3 Apr 2025 18:22:37 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 3 Apr 2025 18:22:37 -0400
Several read stream users asserted that the read stream was exhausted
after looping on that very condition. It was pointed out in an a
review of an as-of-yet uncommitted read stream user [1] that this was
confusing and could lead the reader to think there was a possibility of
some kind of race condition. Remove these asserts.
[1] https://postgr.es/m/F9ACE8D0-B807-4A17-B6BD-87EF0717983D%40yesql.se
M contrib/amcheck/verify_heapam.c
M src/backend/access/gist/gistvacuum.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/spgist/spgvacuum.c
Fix oversight in commit 0dca5d68d.
commit : dbd437e670b88ca67f6b2477e853412a0e82a8cc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Apr 2025 16:03:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Apr 2025 16:03:06 -0400
As coded, fmgr_sql() would get an assertion failure for a SQL function
that has an empty body and is declared to return some type other than
VOID. Typically you'd never get that far because fmgr_sql_validator()
would reject such a definition (I suspect that's how come I managed to
miss the bug). But if check_function_bodies is off or the function is
polymorphic, the validation check wouldn't get made.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/0fde377a-3870-4d18-946a-ce008ee5bb88@gmail.com
M src/backend/executor/functions.c
M src/test/regress/expected/create_function_sql.out
M src/test/regress/sql/create_function_sql.sql
oauth: Remove timeout from t/002_client when not needed
commit : 46c4c7cbc6d562d5f1b784fea9527c998c190b99
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 3 Apr 2025 20:41:09 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 3 Apr 2025 20:41:09 +0200
The connect_timeout=1 setting for the --hang-forever test was left in
place and used by later tests, causing unexpected timeouts on slower
buildfarm animals. Remove it when no longer needed.
Per buildfarm member skink, reported by Andres on Discord.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Reported-by: Andres Freund <andres@anarazel.de>
M src/test/modules/oauth_validator/t/002_client.pl
oauth: Fix build on platforms without epoll/kqueue
commit : 8ae0a37932d81082ec7e4b10eb304a83c561408e
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 3 Apr 2025 20:37:52 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 3 Apr 2025 20:37:52 +0200
register_socket() missed a variable declaration if neither
HAVE_SYS_EPOLL_H nor HAVE_SYS_EVENT_H was defined.
While we're fixing that, adjust the tests to check pg_config.h for one
of the multiplexer implementations, rather than assuming that Windows is
the only platform without support. (Christoph reported this on
hurd-amd64, an experimental Debian.)
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Reported-by: Christoph Berg <myon@debian.org>
Discussion: https://postgr.es/m/Z-sPFl27Y0ZC-VBl%40msg.df7cb.de
M src/interfaces/libpq/fe-auth-oauth-curl.c
M src/test/modules/oauth_validator/t/001_server.pl
Fix unintentional 'NULL' string literal in pg_upgrade.
commit : 945126234bbc03a5a8f40e46383f550c08fce08e
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 3 Apr 2025 11:04:37 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 3 Apr 2025 11:04:37 -0700
Introduced in 2a083ab807.
Discussion: https://postgr.es/m/e852442da35b4f31acc600ed98bbee0f12e65e0c.camel@j-davis.com
Reviewed-by: Michael Paquier <michael@paquier.xyz>
M src/bin/pg_upgrade/pg_upgrade.c
pg_upgrade check for Unicode-dependent relations.
commit : b81ffa13e35693af8accf00f1694bbeca6809fb6
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 3 Apr 2025 10:45:38 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 3 Apr 2025 10:45:38 -0700
This check will not cause an upgrade failure, only a warning.
Discussion: https://postgr.es/m/ef03d678b39a64392f4b12e0f59d1495c740969e.camel%40j-davis.com
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
M src/bin/pg_upgrade/check.c
Restrict copying of invalidated replication slots.
commit : fd09c1316bc37773d115d918dded660a7f5f08fb
author : Masahiko Sawada <msawada@postgresql.org>
date : Thu, 3 Apr 2025 10:30:00 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Thu, 3 Apr 2025 10:30:00 -0700
Previously, invalidated logical and physical replication slots could
be copied using the pg_copy_logical_replication_slot and
pg_copy_physical_replication_slot functions. Replication slots that
were invalidated for reasons other than WAL removal retained their
restart_lsn. This meant that a new slot copied from an invalidated
slot could have a restart_lsn pointing to a WAL segment that might
have already been removed.
This commit restricts the copying of invalidated replication slots.
Backpatch to v16, where slots could retain their restart_lsn when
invalidated for reasons other than WAL removal.
For v15 and earlier, this check is not required since slots can only
be invalidated due to WAL removal, and existing checks already handle
this issue.
Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CANhcyEU65aH0VYnLiu%3DOhNNxhnhNhwcXBeT-jvRe1OiJTo_Ayg%40mail.gmail.com
Backpatch-through: 16
M doc/src/sgml/func.sgml
M src/backend/replication/slotfuncs.c
M src/test/recovery/t/035_standby_logical_decoding.pl
Remove duplicate set of print_notnull
commit : f104192e52fe01592689c6a896fb29facf03bd12
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 3 Apr 2025 17:34:25 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 3 Apr 2025 17:34:25 +0200
I inserted the second one by mistake in commit 14e87ffa5c54.
Reported-by: jian he <jian.universality@gmail.com>
Confirmed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CACJufxFqckBFxPfCixHHbOr0zMLksviTj2m3o12-tErfx_PvTg@mail.gmail.com
M src/bin/pg_dump/pg_dump.c
Add missing declarations to pg_config.h.in
commit : b82e7eddb023ade0739002b3ef05939ea6937e57
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 3 Apr 2025 13:57:27 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 3 Apr 2025 13:57:27 +0200
Add missing pg_config.h.in declarations from 09be39112654
where the corresponding autoconf/meson declarations were
added.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/70145721-6949-4ABF-BB54-63F866488DF8@yesql.se
M src/include/pg_config.h.in
libpq: Add support for dumping SSL key material to file
commit : 2da74d8d6400975bf73fb0df97c3943ad3ed9a36
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 3 Apr 2025 13:16:43 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 3 Apr 2025 13:16:43 +0200
This adds a new connection parameter which instructs libpq to
write out keymaterial clientside into a file in order to make
connection debugging with Wireshark and similar tools possible.
The file format used is the standardized NSS format.
Author: Abhishek Chanda <abhishek.becs@gmail.com>
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/CAKiP-K85C8uQbzXKWf5wHQPkuygGUGcufke713iHmYWOe9q2dA@mail.gmail.com
M configure
M configure.ac
M doc/src/sgml/libpq.sgml
M meson.build
M src/include/pg_config.h.in
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/libpq-int.h
M src/test/ssl/t/001_ssltests.pl
Add support for sorted gist index builds to btree_gist
commit : e4309f73f698851a2f7d49ca5e98e3e188400891
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 3 Apr 2025 13:46:35 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 3 Apr 2025 13:46:35 +0300
This enables sortsupport in the btree_gist extension for faster builds
of gist indexes.
Sorted gist index build strategy is the new default now. Regression
tests are unchanged (except for one small change in the 'enum' test to
add coverage for enum values added later) and are using the sorted
build strategy instead.
One version of this was committed a long time ago already, in commit
9f984ba6d2, but it was quickly reverted because of buildfarm
failures. The failures were presumably caused by some small bugs, but
we never got around to debug and commit it again. This patch was
written from scratch, implementing the same idea, with some fragments
and ideas from the original patch.
Author: Bernd Helmle <mailings@oopsware.de>
Author: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://www.postgresql.org/message-id/64d324ce2a6d535d3f0f3baeeea7b25beff82ce4.camel@oopsware.de
M contrib/btree_gist/Makefile
M contrib/btree_gist/btree_bit.c
M contrib/btree_gist/btree_bool.c
M contrib/btree_gist/btree_bytea.c
M contrib/btree_gist/btree_cash.c
M contrib/btree_gist/btree_date.c
M contrib/btree_gist/btree_enum.c
M contrib/btree_gist/btree_float4.c
M contrib/btree_gist/btree_float8.c
A contrib/btree_gist/btree_gist–1.8–1.9.sql
M contrib/btree_gist/btree_gist.control
M contrib/btree_gist/btree_inet.c
M contrib/btree_gist/btree_int2.c
M contrib/btree_gist/btree_int4.c
M contrib/btree_gist/btree_int8.c
M contrib/btree_gist/btree_interval.c
M contrib/btree_gist/btree_macaddr.c
M contrib/btree_gist/btree_macaddr8.c
M contrib/btree_gist/btree_numeric.c
M contrib/btree_gist/btree_oid.c
M contrib/btree_gist/btree_text.c
M contrib/btree_gist/btree_time.c
M contrib/btree_gist/btree_ts.c
M contrib/btree_gist/btree_utils_var.h
M contrib/btree_gist/btree_uuid.c
M contrib/btree_gist/expected/enum.out
M contrib/btree_gist/meson.build
M contrib/btree_gist/sql/enum.sql
M doc/src/sgml/btree-gist.sgml
Fix boilerplate comments in btree_gist
commit : 9370978da8ac8f0e31867859282b0f2be724fe48
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 3 Apr 2025 13:39:33 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 3 Apr 2025 13:39:33 +0300
A few of these were copy-pasted wrong, like the comment "Bytea ops" in
btree_numeric.c. Instead of fixing the incorrect ones, replace them
all with generic comment "GiST support functions".
Also tidy up the inconsistent newlines between various functions while
we're at it.
M contrib/btree_gist/btree_bit.c
M contrib/btree_gist/btree_bool.c
M contrib/btree_gist/btree_bytea.c
M contrib/btree_gist/btree_cash.c
M contrib/btree_gist/btree_date.c
M contrib/btree_gist/btree_enum.c
M contrib/btree_gist/btree_float4.c
M contrib/btree_gist/btree_float8.c
M contrib/btree_gist/btree_inet.c
M contrib/btree_gist/btree_int2.c
M contrib/btree_gist/btree_int4.c
M contrib/btree_gist/btree_int8.c
M contrib/btree_gist/btree_interval.c
M contrib/btree_gist/btree_macaddr.c
M contrib/btree_gist/btree_macaddr8.c
M contrib/btree_gist/btree_numeric.c
M contrib/btree_gist/btree_oid.c
M contrib/btree_gist/btree_text.c
M contrib/btree_gist/btree_time.c
M contrib/btree_gist/btree_ts.c
M contrib/btree_gist/btree_uuid.c
Update Unicode data to Unicode 16.0.0
commit : 82a46cca99fa967cc3c6c2cf55254dd44eebbcfd
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Apr 2025 11:54:14 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Apr 2025 11:54:14 +0200
Reviewed-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://www.postgresql.org/message-id/flat/146349e4-4687-4321-91af-f235572490a8@eisentraut.org
M src/Makefile.global.in
M src/common/unicode/meson.build
M src/include/common/unicode_case_table.h
M src/include/common/unicode_category_table.h
M src/include/common/unicode_east_asian_fw_table.h
M src/include/common/unicode_nonspacing_table.h
M src/include/common/unicode_norm_hashfunc.h
M src/include/common/unicode_norm_table.h
M src/include/common/unicode_normprops_table.h
M src/include/common/unicode_version.h
plpython: Add test for returning Python set from SETOF function
commit : 231064aa0fbcdf12b7248e8d8d43a75482afbf1f
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Apr 2025 11:04:28 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Apr 2025 11:04:28 +0200
This is claimed in the documentation but there was a no test case for
it.
Reported-by: Bogdan Grigorenko <gri.bogdan.2020@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/173543330569.680.6706329879058172623%40wrigleys.postgresql.org
M src/pl/plpython/expected/plpython_setof.out
M src/pl/plpython/sql/plpython_setof.sql
Doc: Improve -R option added in e5aeed4b80.
commit : d1d83827ba4d8c5c87cd59a164a2c7a99f897512
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 3 Apr 2025 14:27:13 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 3 Apr 2025 14:27:13 +0530
Author: Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/CAHut+PvJPnaL=70SbBe3fYg2nq74Z=Yv4X=zRpUWYfOi-q6=2w@mail.gmail.com
M doc/src/sgml/ref/pg_createsubscriber.sgml
002_pg_upgrade.pl: Move pg_dump test code for better stability
commit : 8806e4e8deb1e21715e031e17181d904825a410e
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 3 Apr 2025 10:16:24 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 3 Apr 2025 10:16:24 +0200
The alleged "statistics pg_dump bug" that prevented us from enabling
stats dumping in commit 172259afb563 wasn't a pg_dump bug after all: it
was just a side effect of not running pg_dump at the right time (namely,
before giving autovacuum some time to do its thing and then disabling it
to stabilize things). Move the code around to fix this problem and
enable statistics dumping.
Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Diagnosed-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://postgr.es/m/5f3703fd7f27da62a8f3615218f937507f522347.camel@j-davis.com
Discussion: https://postgr.es/m/CAExHW5sDm+aGb7A4EXK=X9rkrmSPDgc03EdADt=wWkdMO=XPSA@mail.gmail.com
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
002_pg_upgrade.pl: rename some variables for clarity
commit : abe56227b2e213755dd3e194c530f5f06467bd7c
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 3 Apr 2025 09:56:58 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 3 Apr 2025 09:56:58 +0200
This renames %node_params to %old_node_params, @initdb_params to
@old_initdb_params, and adds separate @new_initdb_params and
%new_node_params rather than reusing the former in confusing ways.
Extracted from a larger patch from the same author.
Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CAExHW5sDm+aGb7A4EXK=X9rkrmSPDgc03EdADt=wWkdMO=XPSA@mail.gmail.com
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
Remove duplicated comment in get_relation_constraints
commit : ea5d3f52334a1693a8b9e110095718c79a678a5f
author : Richard Guo <rguo@postgresql.org>
date : Thu, 3 Apr 2025 16:43:53 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Thu, 3 Apr 2025 16:43:53 +0900
The check for non-inheritable constraints is performed later, and the
same comment is included at that point.
While we're here, remove one extraneous blank line.
Author: jian he <jian.universality@gmail.com>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Reviewed-by: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/CACJufxETi6x86S8EkH8mRfOcm2AenoE9t1pyCFVMpU34gVhF3w@mail.gmail.com
M src/backend/optimizer/util/plancat.c
Update Unicode data to CLDR 47
commit : 84fea854c990a59d00d2fec15f81a96565d4bce4
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Apr 2025 09:19:28 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Apr 2025 09:19:28 +0200
No actual changes result.
M src/Makefile.global.in
Update code comment
commit : bbf24fe2f19a426097c208c355e61e257cd76bac
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Apr 2025 09:18:04 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Apr 2025 09:18:04 +0200
Commit 4e7f62bc386 added a new input file to a script but didn't
update the comment listing the input files.
M src/common/unicode/generate-unicode_case_table.pl
Fix update-unicode make target
commit : 34f04aa6533b6f6f9894907cd29f5d52d9e0df92
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Apr 2025 09:15:59 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Apr 2025 09:15:59 +0200
The addition of SpecialCasing.txt by commit 286a365b9c2 was not added
to the make target dependencies, so the invoked script would fail
because the required file wasn't downloaded first. (The meson version
appears to work correctly.)
M src/common/unicode/Makefile
Fix slot synchronization for two_phase enabled slots.
commit : 4868c96bc8c60145958f1a83bbe9409718a696cb
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 3 Apr 2025 12:07:46 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 3 Apr 2025 12:07:46 +0530
The issue is that the transactions prepared before two-phase decoding is
enabled can fail to replicate to the subscriber after being committed on a
promoted standby following a failover. This is because the two_phase_at
field of a slot, which tracks the LSN from which two-phase decoding
starts, is not synchronized to standby servers. Without two_phase_at, the
logical decoding might incorrectly identify prepared transaction as
already replicated to the subscriber after promotion of standby server,
causing them to be skipped.
To address the issue on HEAD, the two_phase_at field of the slot is
exposed by the pg_replication_slots view and allows the slot
synchronization to copy this value to the corresponding synced slot on the
standby server.
This bug is likely to occur if the user toggles the two_phase option to
true after initial slot creation. Given that altering the two_phase option
of a replication slot is not allowed in PostgreSQL 17, this bug is less
likely to occur. We can't change the view/function definition in
backbranch so we can't push the same fix but we are brainstorming an
appropriate solution for PG17.
Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/TYAPR01MB5724CC7C288535BBCEEE65DA94A72@TYAPR01MB5724.jpnprd01.prod.outlook.com
M doc/src/sgml/system-views.sgml
M src/backend/catalog/system_views.sql
M src/backend/replication/logical/slotsync.c
M src/backend/replication/slotfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/recovery/t/040_standby_failover_slots_sync.pl
M src/test/regress/expected/rules.out
Remove unnecessary type violation in tsvectorrecv().
commit : a7187c3723b41057522038c5e5db329d84f41ac4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Apr 2025 16:17:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Apr 2025 16:17:43 -0400
compareentry() is declared to work on WordEntryIN structs, but
tsvectorrecv() is using it in two places to work on WordEntry
structs. This is almost okay, since WordEntry is the first
field of WordEntryIN. But on machines with 8-byte pointers,
WordEntryIN will have a larger alignment spec than WordEntry,
and it's at least theoretically possible that the compiler
could generate code that depends on the larger alignment.
Given the lack of field reports, this may be just a hypothetical bug
that upsets nothing except sanitizer tools. Or it may be real on
certain hardware but nobody's tried to use tsvectorrecv() on such
hardware. In any case we should fix it, and the fix is trivial:
just change compareentry() so that it works on WordEntry without any
mention of WordEntryIN. We can also get rid of the quite-useless
intermediate function WordEntryCMP.
Bug: #18875
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18875-07a29c49c825a608@postgresql.org
Backpatch-through: 13
M src/backend/utils/adt/tsvector.c
Add test for HeapBitmapScan's broken skip_fetch optimization
commit : 24da5b239a4b50798fb2cbe46597cc932a5210f2
author : Andres Freund <andres@anarazel.de>
date : Wed, 2 Apr 2025 14:58:39 -0400
committer: Andres Freund <andres@anarazel.de>
date : Wed, 2 Apr 2025 14:58:39 -0400
In the previous commit HeapBitmapScan's skip_fetch optimization was removed,
due to being broken in not easily fixable ways. Add a test that verifies we
don't re-introduce this bug if somebody tries to re-add the feature.
Only add the test to master for now, it's possible it's not entirely
stable. That seems sufficient, as we're not going to re-introduce the feature
on the backbranches. I did verify that the test passes on all branches. If the
test turns out to be unproblematic, we can backpatch it later, should we feel
a need to do so.
Discussion: https://postgr.es/m/CAEze2Wg3gXXZTr6_rwC+s4-o2ZVFB5F985uUSgJTsECx6AmGcQ@mail.gmail.com
A src/test/isolation/expected/index-only-bitmapscan.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/index-only-bitmapscan.spec
Remove HeapBitmapScan's skip_fetch optimization
commit : 459e7bf8e2f8ab894dc613fa8555b74c4eef6969
author : Andres Freund <andres@anarazel.de>
date : Wed, 2 Apr 2025 14:25:17 -0400
committer: Andres Freund <andres@anarazel.de>
date : Wed, 2 Apr 2025 14:25:17 -0400
The optimization does not take the removal of TIDs by a concurrent vacuum into
account. The concurrent vacuum can remove dead TIDs and make pages ALL_VISIBLE
while those dead TIDs are referenced in the bitmap. This can lead to a
skip_fetch scan returning too many tuples.
It likely would be possible to implement this optimization safely, but we
don't have the necessary infrastructure in place. Nor is it clear that it's
worth building that infrastructure, given how limited the skip_fetch
optimization is.
In the backbranches we just disable the optimization by always passing
need_tuples=true to table_beginscan_bm(). We can't perform API/ABI changes in
the backbranches and we want to make the change as minimal as possible.
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reported-By: Konstantin Knizhnik <knizhnik@garret.ru>
Discussion: https://postgr.es/m/CAEze2Wg3gXXZTr6_rwC+s4-o2ZVFB5F985uUSgJTsECx6AmGcQ@mail.gmail.com
Backpatch-through: 13
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/include/access/heapam.h
M src/include/access/tableam.h
Change SQL-language functions to use the plan cache.
commit : 0dca5d68d7bebf2c1036fd84875533afef6df992
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Apr 2025 14:05:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Apr 2025 14:05:50 -0400
In the historical implementation of SQL functions (if they don't get
inlined), we built plans for all the contained queries at first call
within an outer query, and then re-used those plans for the duration
of the outer query, and then forgot everything. This was not ideal,
not least because the plans could not be customized to specific values
of the function's parameters. Our plancache infrastructure seems
mature enough to be used here. That will solve both the problem with
not being able to build custom plans and the problem with not being
able to share work across successive outer queries.
Aside from those performance concerns, this change fixes a
longstanding bugaboo with SQL functions: you could not write DDL that
would affect later statements in the same function. That's mostly
still true with new-style SQL functions, since the results of parse
analysis are baked into the stored query trees (and protected by
dependency records). But for old-style SQL functions, it will now
work much as it does with PL/pgSQL functions, because we delay parse
analysis and planning of each query until we're ready to run it.
Some edge cases that require replanning are now handled better too;
see for example the new rowsecurity test, where we now detect an RLS
context change that was previously missed.
One other edge-case change that might be worthy of a release note
is that we now insist that a SQL function's result be generated
by the physically-last query within it. Previously, if the last
original query was deleted by a DO INSTEAD NOTHING rule, we'd be
willing to take the result from the preceding query instead.
This behavior was undocumented except in source-code comments,
and it seems hard to believe that anyone's relying on it.
Along the way to this feature, we needed a few infrastructure changes:
* The plancache can now take either a raw parse tree or an
analyzed-but-not-rewritten Query as the starting point for a
CachedPlanSource. If given a Query, it is caller's responsibility
that nothing will happen to invalidate that form of the query.
We use this for new-style SQL functions, where what's in pg_proc is
serialized Query(s) and we trust the dependency mechanism to disallow
DDL that would break those.
* The plancache now offers a way to invoke a post-rewrite callback
to examine/modify the rewritten parse tree when it is rebuilding
the parse trees after a cache invalidation. We need this because
SQL functions sometimes adjust the parse tree to make its output
exactly match the declared result type; if the plan gets rebuilt,
that has to be re-done.
* There is a new backend module utils/cache/funccache.c that
abstracts the idea of caching data about a specific function
usage (a particular function and set of input data types).
The code in it is moved almost verbatim from PL/pgSQL, which
has done that for a long time. We use that logic now for
SQL-language functions too, and maybe other PLs will have use
for it in the future.
Author: Alexander Pyhalov <a.pyhalov@postgrespro.ru>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://postgr.es/m/8216639.NyiUUSuA9g@aivenlaptop
M doc/src/sgml/xfunc.sgml
M src/backend/catalog/pg_proc.c
M src/backend/executor/functions.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/analyze.c
M src/backend/utils/cache/Makefile
A src/backend/utils/cache/funccache.c
M src/backend/utils/cache/meson.build
M src/backend/utils/cache/plancache.c
M src/include/executor/functions.h
M src/include/parser/analyze.h
A src/include/utils/funccache.h
M src/include/utils/plancache.h
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_funcs.c
M src/pl/plpgsql/src/pl_handler.c
M src/pl/plpgsql/src/plpgsql.h
M src/test/modules/test_extensions/expected/test_extensions.out
M src/test/regress/expected/create_function_sql.out
M src/test/regress/expected/rangefuncs.out
M src/test/regress/expected/rowsecurity.out
M src/test/regress/sql/create_function_sql.sql
M src/test/regress/sql/rowsecurity.sql
M src/tools/pgindent/typedefs.list
Add GiST and btree sortsupport routines for range types
commit : e9e7b66044c9e3dfa76fd1599d5703acd3e4a3f5
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 19:51:28 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 19:51:28 +0300
For GiST, having a sortsupport function allows building the index
using the "sorted build" method, which is much faster.
For b-tree, the sortsupport routine doesn't give any new
functionality, but speeds up sorting a tiny bit. The difference is not
very significant, about 2% in cursory testing on my laptop, because
the range type comparison function has quite a lot of overhead from
detoasting. In any case, since we have the function for GiST anyway,
we might as well register it for the btree opfamily too.
Author: Bernd Helmle <mailings@oopsware.de>
Discussion: https://www.postgresql.org/message-id/64d324ce2a6d535d3f0f3baeeea7b25beff82ce4.camel@oopsware.de
M src/backend/utils/adt/rangetypes.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_proc.dat
docs: Fix column count attribute in table
commit : ea3f9b6da34a1a4dc2c0c118789587c2a85c78d7
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 18:17:47 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 18:17:47 +0300
Nothing seems to actually depend on the attribute, as the docs built
successfully, but let's be tidy.
Reported offlist by Matthias van de Meent
M doc/src/sgml/protocol.sgml
Improve accounting for PredXactList, RWConflictPool and PGPROC
commit : 46df9487d973d00c83a7962935063f3a6518939c
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 2 Apr 2025 17:08:34 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 2 Apr 2025 17:08:34 +0200
Various places allocated shared memory by first allocating a small chunk
using ShmemInitStruct(), followed by ShmemAlloc() calls to allocate more
memory. Unfortunately, ShmemAlloc() does not update ShmemIndex, so this
affected pg_shmem_allocations - it only shown the initial chunk.
This commit modifies the following allocations, to allocate everything
as a single chunk, and then split it internally.
- PredXactList
- RWConflictPool
- PGPROC structures
- Fast-Path Lock Array
The fast-path lock array is allocated separately, not as a part of the
PGPROC structures allocation.
Author: Rahila Syed <rahilasyed90@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CAH2L28vHzRankszhqz7deXURxKncxfirnuW68zD7+hVAqaS5GQ@mail.gmail.com
M src/backend/storage/lmgr/predicate.c
M src/backend/storage/lmgr/proc.c
Improve accounting for memory used by shared hash tables
commit : f5930f9a98ea65d659d41600a138e608988ad122
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 2 Apr 2025 17:02:21 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 2 Apr 2025 17:02:21 +0200
pg_shmem_allocations tracks the memory allocated by ShmemInitStruct(),
but for shared hash tables that covered only the header and hash
directory. The remaining parts (segments and buckets) were allocated
later using ShmemAlloc(), which does not update the shmem accounting.
Thus, these allocations were not shown in pg_shmem_allocations.
This commit improves the situation by allocating all the hash table
parts at once, using a single ShmemInitStruct() call. This way the
ShmemIndex entries (and thus pg_shmem_allocations) better reflect the
proper size of the hash table.
This affects allocations for private (non-shared) hash tables too, as
the hash_create() code is shared. For non-shared tables this however
makes no practical difference.
This changes the alignment a bit. ShmemAlloc() aligns the chunks using
CACHELINEALIGN(), which means some parts (header, directory, segments)
were aligned this way. Allocating all parts as a single chunk removes
this (implicit) alignment. We've considered adding explicit alignment,
but we've decided not to - it seems to be merely a coincidence due to
using the ShmemAlloc() API, not due to necessity.
Author: Rahila Syed <rahilasyed90@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CAH2L28vHzRankszhqz7deXURxKncxfirnuW68zD7+hVAqaS5GQ@mail.gmail.com
M src/backend/storage/ipc/shmem.c
M src/backend/utils/hash/dynahash.c
M src/include/utils/hsearch.h
Need to do CommandCounterIncrement after StoreAttrMissingVal.
commit : bd178960c69bae972c274af8102da9018df8196a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Apr 2025 11:13:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Apr 2025 11:13:01 -0400
Without this, an additional change to the same pg_attribute row
within the same command will fail. This is possible at least with
ALTER TABLE ADD COLUMN on a multiple-inheritance-pathway structure.
(Another potential hazard is that immediately-following operations
might not see the missingval.)
Introduced by 95f650674, which split the former coding that
used a single pg_attribute update to change both atthasdef and
atthasmissing/attmissingval into two updates, but missed that
this should entail two CommandCounterIncrements as well. Like
that fix, back-patch through v13.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/025a3ffa-5eff-4a88-97fb-8f583b015965@gmail.com
Backpatch-through: 13
M src/backend/commands/tablecmds.c
M src/test/regress/expected/inherit.out
M src/test/regress/sql/inherit.sql
docs: Add a new section and a table listing protocol versions
commit : b05751220b0cab7bdf9b729f4fcf6f2adb2d1b46
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 16:41:51 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 16:41:51 +0300
Move the discussion on protocol versions and version negotiation to a
new "Protocol versions" section. Add a table listing all the different
protocol versions, starting from the obsolete protocol version 2, and
the PostgreSQL versions that support each.
Discussion: https://www.postgresql.org/message-id/69f53970-1d55-4165-9151-6fb524e36af9@iki.fi
M doc/src/sgml/protocol.sgml
Make cancel request keys longer
commit : a460251f0a1ac987f0225203ff9593704da0b1a9
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 16:41:48 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 16:41:48 +0300
Currently, the cancel request key is a 32-bit token, which isn't very
much entropy. If you want to cancel another session's query, you can
brute-force it. In most environments, an unauthorized cancellation of
a query isn't very serious, but it nevertheless would be nice to have
more protection from it. Hence make the key longer, to make it harder
to guess.
The longer cancellation keys are generated when using the new protocol
version 3.2. For connections using version 3.0, short 4-bytes keys are
still used.
The new longer key length is not hardcoded in the protocol anymore,
the client is expected to deal with variable length keys, up to 256
bytes. This flexibility allows e.g. a connection pooler to add more
information to the cancel key, which might be useful for finding the
connection.
Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions)
Discussion: https://www.postgresql.org/message-id/508d0505-8b7a-4864-a681-e7e5edfe32aa@iki.fi
M doc/src/sgml/protocol.sgml
M src/backend/storage/ipc/procsignal.c
M src/backend/tcop/backend_startup.c
M src/backend/tcop/postgres.c
M src/backend/utils/init/globals.c
M src/backend/utils/init/postinit.c
M src/include/libpq/pqcomm.h
M src/include/miscadmin.h
M src/include/storage/procsignal.h
M src/interfaces/libpq/fe-cancel.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-int.h
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
libpq: Add min/max_protocol_version connection options
commit : 285613c60a7aff5daaf281c67002483b0d26e715
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 16:41:45 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 16:41:45 +0300
All supported version of the PostgreSQL server send the
NegotiateProtocolVersion message when an unsupported minor protocol
version is requested by a client. But many other applications that
implement the PostgreSQL protocol (connection poolers, or other
databases) do not, and the same is true for PostgreSQL server versions
older than 9.3. Connecting to such other applications thus fails if a
client requests a protocol version different than 3.0.
This patch adds a max_protocol_version connection option to libpq that
specifies the protocol version that libpq should request from the
server. Currently only 3.0 is supported, but that will change in a
future commit that bumps the protocol version. Even after that version
bump the default will likely stay 3.0 for the time being. Once more of
the ecosystem supports the NegotiateProtocolVersion message we might
want to change the default to the latest minor version.
This also adds the similar min_protocol_version connection option, to
allow the client to specify that connecting should fail if a lower
protocol version is attempted by the server. This can be used to
ensure that certain protocol features are used, which can be
particularly useful if those features impact security.
Author: Jelte Fennema-Nio <postgres@jeltef.nl>
Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions)
Discussion: https://www.postgresql.org/message-id/CAGECzQTfc_O%2BHXqAo5_-xG4r3EFVsTefUeQzSvhEyyLDba-O9w@mail.gmail.com
Discussion: https://www.postgresql.org/message-id/CAGECzQRbAGqJnnJJxTdKewTsNOovUt4bsx3NFfofz3m2j-t7tA@mail.gmail.com
M doc/src/sgml/libpq.sgml
M doc/src/sgml/protocol.sgml
M src/include/libpq/pqcomm.h
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-int.h
M src/test/modules/libpq_pipeline/libpq_pipeline.c
libpq: Handle NegotiateProtocolVersion message differently
commit : 5070349102af12832c8528651c8ed18b16346323
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 16:41:42 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 16:41:42 +0300
Previously libpq would always error out if the server sends a
NegotiateProtocolVersion message. This was fine because libpq only
supported a single protocol version and did not support any protocol
parameters. But in the upcoming commits, we will introduce a new
protocol version and the NegotiateProtocolVersion message starts to
actually be used.
This patch modifies the client side checks to allow a range of
supported protocol versions, instead of only allowing the exact
version that was requested. Currently this "range" only contains the
3.0 version, but in a future commit we'll change this.
Also clarify the error messages, making them suitable for the world
where libpq will support multiple protocol versions and protocol
extensions.
Note that until the later commits that introduce new protocol version,
this change does not have any behavioural effect, because libpq will
only request version 3.0 and will never send protocol parameters, and
therefore will never receive a NegotiateProtocolVersion message from
the server.
Author: Jelte Fennema-Nio <postgres@jeltef.nl>
Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions)
Discussion: https://www.postgresql.org/message-id/CAGECzQTfc_O%2BHXqAo5_-xG4r3EFVsTefUeQzSvhEyyLDba-O9w@mail.gmail.com
Discussion: https://www.postgresql.org/message-id/CAGECzQRbAGqJnnJJxTdKewTsNOovUt4bsx3NFfofz3m2j-t7tA@mail.gmail.com
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-int.h
Fix code comment
commit : 748e98d05b752e05c0f860f35f17bb0ba83631ed
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 2 Apr 2025 14:34:24 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 2 Apr 2025 14:34:24 +0200
The changes made in commit d2b4b4c2259 contained incorrect comments:
They said that certain forward declarations were necessary to "avoid
including pathnodes.h here", but the file is itself pathnodes.h! So
change the comment to just say it's a forward declaration in one case,
and in the other case we don't need the declaration at all because it
already appeared earlier in the file.
M src/include/nodes/pathnodes.h
Add timingsafe_bcmp(), for constant-time memory comparison
commit : 09be39112654c3f158098fdb5f820143c0330763
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 15:32:40 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 15:32:40 +0300
timingsafe_bcmp() should be used instead of memcmp() or a naive
for-loop, when comparing passwords or secret tokens, to avoid leaking
information about the secret token by timing. This commit just
introduces the function but does not change any existing code to use
it yet.
Co-authored-by: Jelte Fennema-Nio <github-tech@jeltef.nl>
Discussion: https://www.postgresql.org/message-id/7b86da3b-9356-4e50-aa1b-56570825e234@iki.fi
M configure
M configure.ac
M meson.build
M src/include/port.h
M src/port/meson.build
A src/port/timingsafe_bcmp.c
docs: Update phrase on message lengths in the protocol
commit : 85d799ba8a7fe3f6a462fcccbb449e08d21f4ea4
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 15:32:33 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 2 Apr 2025 15:32:33 +0300
The reasoning for why all the message formats are parseable without
the explicit message length field is anachronistic; the real reason is
that protocol version 2 did not have a message length field. There's
nothing wrong with relying on the message length, like we do in the
CopyData messags, even though it often still makes sense to have
length fields for individual parts in messages.
Discussion: https://www.postgresql.org/message-id/02a4eed2-98f0-4796-9d4f-12128ff44fe0@iki.fi
M doc/src/sgml/protocol.sgml
tests: Fix incompatibility of test_aio with *_FORCE_RELEASE
commit : a6285b150ad308fbba3ca832fccd199ba38fac60
author : Andres Freund <andres@anarazel.de>
date : Wed, 2 Apr 2025 07:57:11 -0400
committer: Andres Freund <andres@anarazel.de>
date : Wed, 2 Apr 2025 07:57:11 -0400
The test added in 93bc3d75d8e failed in a build with RELCACHE_FORCE_RELEASE
and CATCACHE_FORCE_RELEASE defined. The test intentionally forgets to exit
batchmode - normally that would trigger an error at the end of the
transaction, which the test verifies. However, with RELCACHE_FORCE_RELEASE
and CATCACHE_FORCE_RELEASE defined, we get other code (output function lookup)
entering batchmode and erroring out because batchmode isn't allowed to be
entered recursively.
Fix that by changing the queries in question to not output any rows. That's
not exactly pretty, but seems to avoid the problem reliably.
Eventually we might want to make RELCACHE_FORCE_RELEASE and
CATCACHE_FORCE_RELEASE GUCs, so we can disable them where necessary - this
isn't the first test having difficulty with those debug options. But that's
for later.
Per buildfarm member prion.
Discussion: https://postgr.es/m/uc62i6vi5gd4bi6wtjj5poadqxolgy55e7ihkmf3mthjegb6zl@zqo7xez7sc2r
M src/test/modules/test_aio/t/001_aio.pl
tests: Cope with WARNINGs during failed CREATE DB on windows
commit : 43dca8a11624d02dde2b4bd348d77b7045c0dfbc
author : Andres Freund <andres@anarazel.de>
date : Wed, 2 Apr 2025 07:08:10 -0400
committer: Andres Freund <andres@anarazel.de>
date : Wed, 2 Apr 2025 07:08:10 -0400
The test added in 93bc3d75d8e sometimes fails on windows, due to warnings like
WARNING: some useless files may be left behind in old database directory "base/16514"
The reason for that is createdb_failure_callback() does not ensure that there
are no open file descriptors for files in the partially created,
to-be-dropped, database. We do take care in dropdb(), but that involves
waiting for checkpoints and a ProcSignalBarrier, which we probably don't want
to do in an error callback. This should probably be fixed one day, but for
now 001_aio.pl needs to cope.
Per buildfarm animals fairywren and drongo.
Discussion: https://postgr.es/m/uc62i6vi5gd4bi6wtjj5poadqxolgy55e7ihkmf3mthjegb6zl@zqo7xez7sc2r
M src/test/modules/test_aio/t/001_aio.pl
Add support for NOT ENFORCED in foreign key constraints
commit : eec0040c4bcd650993bb058ebdf61ab94171fda4
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 2 Apr 2025 13:30:13 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 2 Apr 2025 13:30:13 +0200
This expands the NOT ENFORCED constraint flag, previously only
supported for CHECK constraints (commit ca87c415e2f), to foreign key
constraints.
Normally, when a foreign key constraint is created on a table, action
and check triggers are added to maintain data integrity. With this
patch, if a constraint is marked as NOT ENFORCED, integrity checks are
no longer required, making these triggers unnecessary. Consequently,
when creating a NOT ENFORCED foreign key constraint, triggers will not
be created, and the constraint will be marked as NOT VALID.
Similarly, if an existing foreign key constraint is changed to NOT
ENFORCED, the associated triggers will be dropped, and the constraint
will also be marked as NOT VALID. Conversely, if a NOT ENFORCED
foreign key constraint is changed to ENFORCED, the necessary triggers
will be created, and the will be changed to VALID by performing
necessary validation.
Since not-enforced foreign key constraints have no triggers, the
shortcut used for example in psql and pg_dump to skip looking for
foreign keys if the relation is known not to have triggers no longer
applies. (It already didn't work for partitioned tables.)
Author: Amul Sul <sulamul@gmail.com>
Reviewed-by: Joel Jacobson <joel@compiler.org>
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: jian he <jian.universality@gmail.com>
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Isaac Morland <isaac.morland@gmail.com>
Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com>
Tested-by: Triveni N <triveni.n@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/CAAJ_b962c5AcYW9KUt_R_ER5qs3fUGbe4az-SP-vuwPS-w-AGA@mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_table.sgml
M src/backend/catalog/pg_constraint.c
M src/backend/catalog/sql_features.txt
M src/backend/commands/tablecmds.c
M src/backend/optimizer/util/plancat.c
M src/backend/parser/gram.y
M src/backend/parser/parse_utilcmd.c
M src/backend/utils/cache/relcache.c
M src/bin/pg_dump/pg_dump.c
M src/bin/psql/describe.c
M src/include/nodes/parsenodes.h
M src/include/utils/rel.h
M src/test/regress/expected/constraints.out
M src/test/regress/expected/foreign_key.out
M src/test/regress/expected/inherit.out
M src/test/regress/sql/foreign_key.sql
M src/test/regress/sql/inherit.sql
tests: Cope with io_method in TEMP_CONFIG in test_aio
commit : 327d987df1e72a9b146f312df0a5ed34ef148720
author : Andres Freund <andres@anarazel.de>
date : Wed, 2 Apr 2025 06:56:29 -0400
committer: Andres Freund <andres@anarazel.de>
date : Wed, 2 Apr 2025 06:56:29 -0400
If io_method is set in TEMP_CONFIG the test added in 93bc3d75d8e fails,
because it assumes the io_method specified at initdb is actually used.
Fix that by appending the io_method again, after initdb (and thus after
TEMP_CONFIG has been added by Cluster.pm).
Per buildfarm animal bumblebee
Discussion: https://postgr.es/m/zh5u22wbpcyfw2ddl3lsvmsxf4yvsrvgxqwwmfjddc4c2khsgp@gfysyjsaelr5
M src/test/modules/test_aio/t/001_aio.pl
Get rid of WALBufMappingLock
commit : bc22dc0e0ddc2dcb6043a732415019cc6b6bf683
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 2 Apr 2025 12:44:24 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 2 Apr 2025 12:44:24 +0300
Allow multiple backends to initialize WAL buffers concurrently. This way
`MemSet((char *) NewPage, 0, XLOG_BLCKSZ);` can run in parallel without
taking a single LWLock in exclusive mode.
The new algorithm works as follows:
* reserve a page for initialization using XLogCtl->InitializeReserved,
* ensure the page is written out,
* once the page is initialized, try to advance XLogCtl->InitializedUpTo and
signal to waiters using XLogCtl->InitializedUpToCondVar condition
variable,
* repeat previous steps until we reserve initialization up to the target
WAL position,
* wait until concurrent initialization finishes using a
XLogCtl->InitializedUpToCondVar.
Now, multiple backends can, in parallel, concurrently reserve pages,
initialize them, and advance XLogCtl->InitializedUpTo to point to the latest
initialized page.
Author: Yura Sokolov <y.sokolov@postgrespro.ru>
Co-authored-by: Alexander Korotkov <aekorotkov@gmail.com>
Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Tested-by: Michael Paquier <michael@paquier.xyz>
M src/backend/access/transam/xlog.c
M src/backend/utils/activity/wait_event_names.txt
M src/include/storage/lwlocklist.h
Improve error message when standby does accept connections.
commit : b53b88109f94bd81ed0ac580035a936000bc2865
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 2 Apr 2025 15:13:01 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 2 Apr 2025 15:13:01 +0900
Even after reaching the minimum recovery point, if there are long-lived
write transactions with 64 subtransactions on the primary, the recovery
snapshot may not yet be ready for hot standby, delaying read-only
connections on the standby. Previously, when read-only connections were
not accepted due to this condition, the following error message was logged:
FATAL: the database system is not yet accepting connections
DETAIL: Consistent recovery state has not been yet reached.
This DETAIL message was misleading because the following message was
already logged in this case:
LOG: consistent recovery state reached
This contradiction, i.e., indicating that the recovery state was consistent
while also stating it wasn’t, caused confusion.
This commit improves the error message to better reflect the actual state:
FATAL: the database system is not yet accepting connections
DETAIL: Recovery snapshot is not yet ready for hot standby.
HINT: To enable hot standby, close write transactions with more than 64 subtransactions on the primary server.
To implement this, the commit introduces a new postmaster signal,
PMSIGNAL_RECOVERY_CONSISTENT. When the startup process reaches
a consistent recovery state, it sends this signal to the postmaster,
allowing it to correctly recognize that state.
Since this is not a clear bug, the change is applied only to the master
branch and is not back-patched.
Author: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Co-authored-by: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Yugo Nagata <nagata@sraoss.co.jp>
Discussion: https://postgr.es/m/02db8cd8e1f527a8b999b94a4bee3165@oss.nttdata.com
M doc/src/sgml/high-availability.sgml
M src/backend/access/transam/xlogrecovery.c
M src/backend/postmaster/postmaster.c
M src/backend/tcop/backend_startup.c
M src/include/storage/pmsignal.h
M src/include/tcop/backend_startup.h
Doc: add information about partition locking
commit : 121d774caea4c93c8b36fb20a17ef774e60894d6
author : David Rowley <drowley@postgresql.org>
date : Wed, 2 Apr 2025 14:02:44 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 2 Apr 2025 14:02:44 +1300
The documentation around locking of partitions for the executor startup
phase of run-time partition pruning wasn't clear about which partitions
were being locked. Fix that.
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://postgr.es/m/CAApHDvp738G75HfkKcfXaf3a8s%3D6mmtOLh46tMD0D2hAo1UCzA%40mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/ddl.sgml
aio: Add errcontext for processing I/Os for another backend
commit : b3219c69fc1e161df8d380c464b3f2cce3b6cab9
author : Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 1 Apr 2025 19:53:07 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 1 Apr 2025 19:53:07 -0400
Push an ErrorContextCallback adding additional detail about the process
performing the I/O and the owner of the I/O when those are not the same.
For io_method worker, this adds context specifying which process owns
the I/O that the I/O worker is processing.
For io_method io_uring, this adds context only when a backend is
*completing* I/O for another backend. It specifies the pid of the owning
process.
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/rdml3fpukrqnas7qc5uimtl2fyytrnu6ymc2vjf2zuflbsjuul%40hyizyjsexwmm
M src/backend/storage/aio/method_io_uring.c
M src/backend/storage/aio/method_worker.c
Fix planner's failure to identify multiple hashable ScalarArrayOpExprs
commit : b136db07c651e5048d44add0a01d5fc01b6fe0dc
author : David Rowley <drowley@postgresql.org>
date : Wed, 2 Apr 2025 11:56:29 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 2 Apr 2025 11:56:29 +1300
50e17ad28 (v14) and 29f45e299 (v15) made it so the planner could identify
IN and NOT IN clauses which have Const lists as right-hand arguments and
when an appropriate hash function is available for the data types, mark
the ScalarArrayOpExpr as hashable so the executor could execute it more
optimally by building and probing a hash table during expression
evaluation.
These commits both worked correctly when there was only a single
ScalarArrayOpExpr in the given expression being processed by the
planner, but when there were multiple, only the first was checked and any
subsequent ones were not identified, which resulted in less optimal
expression evaluation during query execution for all but the first found
ScalarArrayOpExpr.
Backpatch to 14, where 50e17ad28 was introduced.
Author: David Geier <geidav.pg@gmail.com>
Discussion: https://postgr.es/m/29a76f51-97b0-4c07-87b7-ec8e3b5345c9@gmail.com
Backpatch-through: 14
M src/backend/optimizer/util/clauses.c
Introduce a SQL-callable function array_sort(anyarray).
commit : 6c12ae09f5a5d6c153eaea7901542591dc28fb9e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Apr 2025 18:03:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Apr 2025 18:03:55 -0400
Create a function that will sort the elements of an array
according to the element type's sort order. If the array
has more than one dimension, the sub-arrays of the first
dimension are sorted per normal array-comparison rules,
leaving their contents alone.
In support of this, add pg_type.typarray to the set of fields
cached by the typcache.
Author: Junwang Zhao <zhjwpku@gmail.com>
Co-authored-by: Jian He <jian.universality@gmail.com>
Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://postgr.es/m/CAEG8a3J41a4dpw_-F94fF-JPRXYxw-GfsgoGotKcjs9LVfEEvw@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/array_userfuncs.c
M src/backend/utils/cache/typcache.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/utils/typcache.h
M src/test/regress/expected/arrays.out
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/arrays.sql
M src/test/regress/sql/collate.icu.utf8.sql
M src/tools/pgindent/typedefs.list
Fix detection and handling of strchrnul() for macOS 15.4.
commit : 6da2ba1d8a031984eb016fed6741bb2ac945f19d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Apr 2025 16:49:51 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Apr 2025 16:49:51 -0400
As of 15.4, macOS has strchrnul(), but access to it is blocked behind
a check for MACOSX_DEPLOYMENT_TARGET >= 15.4. But our does-it-link
configure check finds it, so we try to use it, and fail with the
present default deployment target (namely 15.0). This accounts for
today's buildfarm failures on indri and sifaka.
This is the identical problem that we faced some years ago when Apple
introduced preadv and pwritev in the same way. We solved that in
commit f014b1b9b by using AC_CHECK_DECLS instead of AC_CHECK_FUNCS
to check the functions' availability. So do the same now for
strchrnul(). Interestingly, we already had a workaround for
"the link check doesn't agree with <string.h>" cases with glibc,
which we no longer need since only the header declaration is being
checked.
Testing this revealed that the meson version of this check has never
worked, because it failed to use "-Werror=unguarded-availability-new".
(Apparently nobody's tried to build with meson on macOS versions that
lack preadv/pwritev as standard.) Adjust that while at it. Also,
we had never put support for "-Werror=unguarded-availability-new"
into v13, but we need that now.
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Co-authored-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/385134.1743523038@sss.pgh.pa.us
Backpatch-through: 13
M configure
M configure.ac
M meson.build
M src/include/pg_config.h.in
M src/port/snprintf.c
Use workaround of __builtin_setjmp only on MINGW on MSVCRT
commit : c313fa4602defe1be947370ab5b217ca163a1e3c
author : Andrew Dunstan <andrew@dunslane.net>
date : Tue, 1 Apr 2025 16:24:59 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Tue, 1 Apr 2025 16:24:59 -0400
MSVCRT is not present Windows/ARM64 and the workaround is not
necessary on any UCRT based toolchain.
Author: Lars Kanis <lars@greiz-reinsdorf.de>
Discussion: https://postgr.es/m/CAHXCYb2OjNHtoGVKyXtXmw4B3bUXwJX6M-Lcp1KcMCRUMLOocA@mail.gmail.com
M src/include/c.h
aio: Minor comment improvements
commit : e19dc74491e68e1df05c99c4937929cef3092a3b
author : Andres Freund <andres@anarazel.de>
date : Tue, 1 Apr 2025 16:06:48 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 1 Apr 2025 16:06:48 -0400
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/usbwzckj7q3jhfx3ann3nrfnukmupbs35axvq5zfyeo6nvrzrm@onjhxs2du4st
M src/backend/storage/smgr/md.c
M src/backend/storage/smgr/smgr.c
aio: Add README.md explaining higher level design
commit : fdd146a8ef2bc0c24e8d27e3773feeb8ef0a9c25
author : Andres Freund <andres@anarazel.de>
date : Tue, 1 Apr 2025 16:06:48 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 1 Apr 2025 16:06:48 -0400
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
Discussion: https://postgr.es/m/20210223100344.llw5an2aklengrmn@alap3.anarazel.de
Discussion: https://postgr.es/m/stj36ea6yyhoxtqkhpieia2z4krnam7qyetc57rfezgk4zgapf@gcnactj4z56m
A src/backend/storage/aio/README.md
M src/backend/storage/aio/aio.c
doc: Adjust some notes about pg_upgrade's file transfer modes.
commit : 5aec7e07fb9fd5d8d046a2b19c793d7e4cecf5e4
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 1 Apr 2025 14:37:47 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 1 Apr 2025 14:37:47 -0500
--copy-file-range and --swap were not mentioned in a few places
that discuss the available file transfer modes. This entire page
would likely benefit from an overhaul, but that's v19 material at
this point.
Oversights in commits d93627bcbe and 626d7236b6.
M doc/src/sgml/ref/pgupgrade.sgml
md: Add comment & assert to buffer-zeroing path in md[start]readv()
commit : 00066aa1733d84109f7569a7202c3915d8289d3a
author : Andres Freund <andres@anarazel.de>
date : Tue, 1 Apr 2025 13:50:39 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 1 Apr 2025 13:50:39 -0400
mdreadv() has a codepath to zero out buffers when a read returns zero bytes,
guarded by a check for zero_damaged_pages || InRecovery.
The InRecovery codepath to zero out buffers in mdreadv() appears to be
unreachable. The only known paths to reach mdreadv()/mdstartreadv() in
recovery are XLogReadBufferExtended(), vm_readbuf(), and fsm_readbuf(), each
of which takes care to extend the relation if necessary. This looks to either
have been the case for a long time, or the code was never reachable.
The zero_damaged_pages path is incomplete, as missing segments are not
created.
Putting blocks into the buffer-pool that do not exist on disk is rather
problematic, as such blocks will, at least initially, not be found by scans
that rely on smgrnblocks(), as they are beyond EOF. It also can cause weird
problems with relation extension, as relation extension does not expect blocks
beyond EOF to exist.
Therefore we would like to remove that path.
mdstartreadv(), which I added in e5fe570b51c, does not implement this zeroing
logic. I had started a discussion about that a while ago (linked below), but
forgot to act on the conclusion of the discussion, namely to disable the
in-memory-zeroing behavior.
We could certainly implement equivalent zeroing logic in mdstartreadv(), but
it would have to be more complicated due to potential differences in the
zero_damaged_pages setting between the definer and completor of IO. Given that
we want to remove the logic, that does not seem worth implementing the
necessary logic.
For now, put an Assert(false) and comments documenting this choice into
mdreadv() and comments documenting the deprecation of the path in mdreadv()
and the non-implementation of it in mdstartreadv(). If we, during testing,
discover that we do need the path, we can implement it at that time.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/postgr.es/m/20250330024513.ac.nmisch@google.com
Discussion: https://postgr.es/m/postgr.es/m/3qxxsnciyffyf3wyguiz4besdp5t5uxvv3utg75cbcszojlz7p@uibfzmnukkbd
M src/backend/storage/smgr/md.c
aio: Add test_aio module
commit : 93bc3d75d8e1aabdc256ff6da2282266dca82537
author : Andres Freund <andres@anarazel.de>
date : Tue, 1 Apr 2025 13:47:46 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 1 Apr 2025 13:47:46 -0400
To make the tests possible, a few functions from bufmgr.c/localbuf.c had to be
exported, via buf_internals.h.
Reviewed-by: Noah Misch <noah@leadboat.com>
Co-authored-by: Andres Freund <andres@anarazel.de>
Co-authored-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/include/storage/buf_internals.h
M src/test/modules/Makefile
M src/test/modules/meson.build
A src/test/modules/test_aio/.gitignore
A src/test/modules/test_aio/Makefile
A src/test/modules/test_aio/meson.build
A src/test/modules/test_aio/t/001_aio.pl
A src/test/modules/test_aio/t/002_io_workers.pl
A src/test/modules/test_aio/test_aio–1.0.sql
A src/test/modules/test_aio/test_aio.c
A src/test/modules/test_aio/test_aio.control
aio: Add pg_aios view
commit : 60f566b4f24362f54003569d5f6e24137bcd83fc
author : Andres Freund <andres@anarazel.de>
date : Tue, 1 Apr 2025 13:30:33 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 1 Apr 2025 13:30:33 -0400
The new view lists all IO handles that are currently in use and is mainly
useful for PG developers, but may also be useful when tuning PG.
Bumps catversion.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
M doc/src/sgml/system-views.sgml
M src/backend/catalog/system_views.sql
M src/backend/storage/aio/Makefile
A src/backend/storage/aio/aio_funcs.c
M src/backend/storage/aio/meson.build
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/privileges.out
M src/test/regress/expected/rules.out
M src/test/regress/sql/privileges.sql
docs: Add acronym and glossary entries for I/O and AIO
commit : 46250cdcb037e302ec5f0734ea0d06e2d21193e0
author : Andres Freund <andres@anarazel.de>
date : Tue, 1 Apr 2025 13:30:33 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 1 Apr 2025 13:30:33 -0400
These are fairly basic, but better than nothing. While there are several
opportunities to link to these entries, this patch does not add any. They will
however be referenced by future patches.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250326183102.92.nmisch@google.com
M doc/src/sgml/acronyms.sgml
M doc/src/sgml/glossary.sgml
Verify roundtrip dump/restore of regression database
commit : 172259afb563d35001410dc6daad78b250924038
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 1 Apr 2025 18:50:40 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 1 Apr 2025 18:50:40 +0200
Add a test to pg_upgrade's test suite that verifies that
dump-restore-dump of regression database produces equivalent output to
dumping it directly. This was already being tested by running
pg_upgrade itself, but non-binary-upgrade mode was not being covered.
The regression database has accrued, over time, a sufficient collection
of interesting objects to ensure good coverage, but there hasn't been a
concerted effort to be completely exhaustive, so it is likely still
possible to have more.
This'd belong more naturally in the pg_dump test suite, but we chose to
put it in src/bin/pg_upgrade/t/002_pg_upgrade.pl because we need a run
of the regression tests which is already done here, so this has less
total test runtime impact. Also, experiments have shown that using
parallel dump/restore is slightly faster, so we use --format=directory -j2.
This test has already reported pg_dump bugs, as fixed in fd41ba93e463,
74563f6b9021, d611f8b1587b, 4694aedf63bf.
Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/CAExHW5uF5V=Cjecx3_Z=7xfh4rg2Wf61PT+hfquzjBqouRzQJQ@mail.gmail.com
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
M src/test/perl/Makefile
A src/test/perl/PostgreSQL/Test/AdjustDump.pm
M src/test/perl/meson.build
Remove a stray "pgrminclude" annotation
commit : 764d501d24baab8ead6dc3bf7bb0dbd13ea86084
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Apr 2025 15:27:28 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Apr 2025 15:27:28 +0200
We don't use those anymore. Fix for commit 8492feb98f6.
M src/backend/access/gin/gininsert.c
Fix minor C type confusion
commit : 113ecf1f8c8c783d4c5ffb4ff934fcb441b3bc2a
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Apr 2025 15:25:42 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Apr 2025 15:25:42 +0200
Returning false instead of NULL gets a compiler error under gcc-14
-std=gnu23, and it appears to have been unintentional. Fix for commit
8492feb98f6.
M src/backend/utils/sort/tuplesortvariants.c
heapam: Only set tuple's block once per page in pagemode
commit : 2904324a88f672b2ecc22735279c16d6e1ee178c
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 1 Apr 2025 13:24:27 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 1 Apr 2025 13:24:27 +0300
Due to splitting the block id into two 16 bit integers, BlockIdSet()
is more expensive than one might think. Doing it once per returned
tuple shows up as a small but reliably reproducible cost. It's simple
enough to set the block number just once per block in pagemode, so do
so.
Author: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/lxzj26ga6ippdeunz6kuncectr5gfuugmm2ry22qu6hcx6oid6@lzx3sjsqhmt6
M src/backend/access/heap/heapam.c
Use function attributes for SSE 4.2 even when targeting that extension
commit : af0c248557aecb335462d980cb7319bdf85a5c66
author : John Naylor <john.naylor@postgresql.org>
date : Tue, 1 Apr 2025 12:01:58 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Tue, 1 Apr 2025 12:01:58 +0700
On Red Hat 9 systems (or similar), the packaged gcc targets x86-64-v2,
but clang does not. This has caused build failures in the wake of
commit e2809e3a1 when building --with-llvm.
The most expedient fix is to use the same function attributes for
the inlined function as we do for the global function.
Reported-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> (plus members skimmer and bumblebee)
Diagnosed-by: Tom Lane <tgl@sss.pgh.pa.us>
Tested-by: Todd Cook <cookt@blackduck.com>
Discussion: https://postgr.es/m/CANWCAZZSxs3a1YRKehkgk2OHKbrVn+xZ+AWW8Co2R_f70NqqmA@mail.gmail.com
M src/include/port/pg_crc32c.h
Fix failing regression test on x86-32 machines
commit : 3dbdf86c6364d6b8470fa4067096062cea62405a
author : David Rowley <drowley@postgresql.org>
date : Tue, 1 Apr 2025 10:52:25 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 1 Apr 2025 10:52:25 +1300
95d6e9af0 added code to display the tuplestore storage type for
WindowAgg nodes and added a test to ensure the "Disk" storage method was
working correctly by setting work_mem to 64 and running a test which
caused the WindowAgg to go to disk. Seemingly, the number of rows
chosen there wasn't quite enough for that to happen in x86 32-bit.
Fix this by increasing the number of rows slightly.
I suspect the buildfarm didn't catch this as MEMORY_CONTEXT_CHECKING
builds will use a bit more memory for MemoryChunks to store the
requested_size and also because of the additional space to store the
chunk's sentinel byte.
Reported-by: Christoph Berg <myon@debian.org>
Discussion: https://postgr.es/m/Z-q3ZAM4OhE-4UiI@msg.df7cb.de
M src/test/regress/expected/explain.out
M src/test/regress/sql/explain.sql
Fix accidentally-harmless thinko in psqlscan_test_variable().
commit : 2fd3e2fa5c908d05ac4d8b6c2c80fd50c506f32e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 31 Mar 2025 12:16:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 31 Mar 2025 12:16:10 -0400
This code was passing literal strings to psqlscan_emit,
which is quite contrary to that function's specification:
"If you pass it something that is not part of the yytext
string, you are making a mistake". It accidentally worked
anyway, even in non-safe_encoding mode. psqlscan_emit
would compute a garbage "reference" pointer, but would
never dereference that since the passed string is all-ASCII.
So there's no live bug today, but that is a happenstance
outcome of psqlscan_emit's current implementation.
Let's make psqlscan_test_variable do what it's supposed to,
namely append directly to the output buffer. This is just
future-proofing against possible changes in psqlscan_emit,
so I don't feel a need to back-patch.
M src/fe_utils/psqlscan.l
doc: Mention clock synchronization recommendation for hot_standby_feedback
commit : 0fcf02ad456862d8523a1918e2882e7697a52896
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 31 Mar 2025 16:54:50 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 31 Mar 2025 16:54:50 +0200
hot_standby_feedback mechanics assume that clocks are synchronized,
but it was not clear from documentation.
Author: Jakub Wartak <jakub.wartak@enterprisedb.com>
Reviewed-by: Euler Taveira <euler@eulerto.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/CAKZiRmwBcALLrDgCyEhHP1enUxtPMjyNM_d1A2Lng3_6Rf4Qfw%40mail.gmail.com
M doc/src/sgml/config.sgml
Inline CRC computation for small fixed-length input on x86
commit : e2809e3a1015697832ee4d37b75ba1cd0caac0f0
author : John Naylor <john.naylor@postgresql.org>
date : Fri, 28 Feb 2025 16:27:30 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Fri, 28 Feb 2025 16:27:30 +0700
pg_crc32c.h now has a simplified copy of the loop in pg_crc32c_sse42.c
suitable for inlining where possible.
This may slightly reduce contention for the WAL insertion lock,
but that hasn't been tested. The motivation for this change is avoid
regressing for a future commit that will use a function pointer for
non-constant input in all x86 builds.
While it's technically possible to make a similar change for Arm and
LoongArch, there are some questions about how inlining should work
since those platforms prefer stricter alignment. There are also no
immediate plans to add additional implementations for them.
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Raghuveer Devulapalli <raghuveer.devulapalli@intel.com>
Discussion: https://postgr.es/m/CANWCAZZEiTzhZcuwTiJ2=opiNpAUn1vuDRu1N02z61AthwRZLA@mail.gmail.com
Discussion: https://postgr.es/m/CANWCAZYRhLHArpyfV4uRK-Rw9N5oV5HMkkKtBehcuTjNOMwCZg@mail.gmail.com
M src/include/port/pg_crc32c.h
Add relallfrozen to pg_dump statistics.
commit : 4694aedf63bf5b5d91f766cb6d6d6d14a9e4434b
author : Jeff Davis <jdavis@postgresql.org>
date : Sun, 30 Mar 2025 22:14:06 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Sun, 30 Mar 2025 22:14:06 -0700
Author: Corey Huinker <corey.huinker@gmail.com>
Discussion: https://postgr.es/m/CADkLM=desCuf3dVHasADvdUVRmb-5gO0mhMO5u9nzgv6i7U86Q@mail.gmail.com
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
Enable IO concurrency on all systems
commit : 2a5e709e721cf5f890cde51755b84cfe25d1c4d9
author : Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 19:14:55 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 19:14:55 -0400
Previously effective_io_concurrency and maintenance_io_concurrency could not
be set above 0 on machines without fadvise support. AIO enables IO concurrency
without such support, via io_method=worker.
Currently only subsystems using the read stream API will take advantage of
this. Other users of maintenance_io_concurrency (like recovery prefetching)
which leverage OS advice directly will not benefit from this change. In those
cases, maintenance_io_concurrency will have no effect on I/O behavior.
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CAAKRu_atGgZePo=_g6T3cNtfMf0QxpvoUh5OUqa_cnPdhLd=gw@mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/alter_tablespace.sgml
M doc/src/sgml/ref/create_tablespace.sgml
M src/backend/access/common/reloptions.c
M src/backend/commands/variable.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/initdb/initdb.c
M src/include/storage/bufmgr.h
M src/include/utils/guc_hooks.h
read_stream: Introduce and use optional batchmode support
commit : ae3df4b34155d4a3811762e78d5076e64df66fba
author : Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 18:30:36 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 18:30:36 -0400
Submitting IO in larger batches can be more efficient than doing so
one-by-one, particularly for many small reads. It does, however, require
the ReadStreamBlockNumberCB callback to abide by the restrictions of AIO
batching (c.f. pgaio_enter_batchmode()). Basically, the callback may not:
a) block without first calling pgaio_submit_staged(), unless a
to-be-waited-on lock cannot be part of a deadlock, e.g. because it is
never held while waiting for IO.
b) directly or indirectly start another batch pgaio_enter_batchmode()
As this requires care and is nontrivial in some cases, batching is only
used with explicit opt-in.
This patch adds an explicit flag (READ_STREAM_USE_BATCHING) to read_stream and
uses it where appropriate.
There are two cases where batching would likely be beneficial, but where we
aren't using it yet:
1) bitmap heap scans, because the callback reads the VM
This should soon be solved, because we are planning to remove the use of
the VM, due to that not being sound.
2) The first phase of heap vacuum
This could be made to support batchmode, but would require some care.
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
M contrib/amcheck/verify_heapam.c
M contrib/pg_prewarm/pg_prewarm.c
M contrib/pg_visibility/pg_visibility.c
M src/backend/access/gist/gistvacuum.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/spgist/spgvacuum.c
M src/backend/commands/analyze.c
M src/backend/storage/aio/read_stream.c
M src/backend/storage/buffer/bufmgr.c
M src/include/storage/read_stream.h
aio: Basic read_stream adjustments for real AIO
commit : f4d0730bbca1b338b8f06389d04816c6c9f06e58
author : Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 18:26:44 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 18:26:44 -0400
Adapt the read stream logic for real AIO:
- If AIO is enabled, we shouldn't issue advice, but if it isn't, we should
continue issuing advice
- AIO benefits from reading ahead with direct IO
- If effective_io_concurrency=0, pass READ_BUFFERS_SYNCHRONOUSLY to
StartReadBuffers() to ensure synchronous IO execution
There are further improvements we should consider:
- While in read_stream_look_ahead(), we can use AIO batch submission mode for
increased efficiency. That however requires care to avoid deadlocks and thus
done separately.
- It can be beneficial to defer starting new IOs until we can issue multiple
IOs at once. That however requires non-trivial heuristics to decide when to
do so.
Reviewed-by: Noah Misch <noah@leadboat.com>
Co-authored-by: Andres Freund <andres@anarazel.de>
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
M src/backend/storage/aio/read_stream.c
docs: Reframe track_io_timing related docs as wait time
commit : b27f8637ea708c4b6113f0d29a76cb411b430a27
author : Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 18:04:40 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 18:04:40 -0400
With AIO it does not make sense anymore to track the time for each individual
IO, as multiple IOs can be in-flight at the same time. Instead we now track
the time spent *waiting* for IOs.
This should be reflected in the docs. While, so far, we only do a subset of
reads, and no other operations, via AIO, describing the GUC and view columns
as measuring IO waits is accurate for synchronous and asynchronous IO.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/5dzyoduxlvfg55oqtjyjehez5uoq6hnwgzor4kkybkfdgkj7ag@rbi4gsmzaczk
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
bufmgr: Use AIO in StartReadBuffers()
commit : 12ce89fd0708207f21a8888e546b9670a847ad4f
author : Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 18:02:23 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 18:02:23 -0400
This finally introduces the first actual use of AIO. StartReadBuffers() now
uses the AIO routines to issue IO.
As the implementation of StartReadBuffers() is also used by the functions for
reading individual blocks (StartReadBuffer() and through that
ReadBufferExtended()) this means all buffered read IO passes through the AIO
paths. However, as those are synchronous reads, actually performing the IO
asynchronously would be rarely beneficial. Instead such IOs are flagged to
always be executed synchronously. This way we don't have to duplicate a fair
bit of code.
When io_method=sync is used, the IO patterns generated after this change are
the same as before, i.e. actual reads are only issued in WaitReadBuffers() and
StartReadBuffers() may issue prefetch requests. This allows to bypass most of
the actual asynchronicity, which is important to make a change as big as this
less risky.
One thing worth calling out is that, if IO is actually executed
asynchronously, the precise meaning of what track_io_timing is measuring has
changed. Previously it tracked the time for each IO, but that does not make
sense when multiple IOs are executed concurrently. Now it only measures the
time actually spent waiting for IO. A subsequent commit will adjust the docs
for this.
While AIO is now actually used, the logic in read_stream.c will often prevent
using sufficiently many concurrent IOs. That will be addressed in the next
commit.
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Co-authored-by: Andres Freund <andres@anarazel.de>
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
Discussion: https://postgr.es/m/20210223100344.llw5an2aklengrmn@alap3.anarazel.de
Discussion: https://postgr.es/m/stj36ea6yyhoxtqkhpieia2z4krnam7qyetc57rfezgk4zgapf@gcnactj4z56m
M src/backend/storage/buffer/bufmgr.c
M src/include/storage/bufmgr.h
bufmgr: Implement AIO read support
commit : 047cba7fa0f8c6930b0dd1d93d98c736ef1e4a5a
author : Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 17:28:03 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 17:28:03 -0400
This commit implements the infrastructure to perform asynchronous reads into
the buffer pool.
To do so, it:
- Adds readv AIO callbacks for shared and local buffers
It may be worth calling out that shared buffer completions may be run in a
different backend than where the IO started.
- Adds an AIO wait reference to BufferDesc, to allow backends to wait for
in-progress asynchronous IOs
- Adapts StartBufferIO(), WaitIO(), TerminateBufferIO(), and their localbuf.c
equivalents, to be able to deal with AIO
- Moves the code to handle BM_PIN_COUNT_WAITER into a helper function, as it
now also needs to be called on IO completion
As of this commit, nothing issues AIO on shared/local buffers. A future commit
will update StartReadBuffers() to do so.
Buffer reads executed through this infrastructure will report invalid page /
checksum errors / warnings differently than before:
In the error case the error message will cover all the blocks that were
included in the read, rather than just the reporting the first invalid
block. If more than one block is invalid, the error will include information
about the range of the read, the first invalid block and the number of invalid
pages, with a HINT towards the server log for per-block details.
For the warning case (i.e. zero_damaged_buffers) we would previously emit one
warning message for each buffer in a multi-block read. Now there is only a
single warning message for the entire read, again referring to the server log
for more details in case of multiple checksum failures within a single larger
read.
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
Discussion: https://postgr.es/m/20210223100344.llw5an2aklengrmn@alap3.anarazel.de
Discussion: https://postgr.es/m/stj36ea6yyhoxtqkhpieia2z4krnam7qyetc57rfezgk4zgapf@gcnactj4z56m
M src/backend/storage/aio/aio_callback.c
M src/backend/storage/buffer/README
M src/backend/storage/buffer/buf_init.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/backend/storage/page/bufpage.c
M src/include/storage/aio.h
M src/include/storage/buf_internals.h
M src/include/storage/bufmgr.h
M src/include/storage/bufpage.h
aio: Add WARNING result status
commit : ef64fe26bad92a7b8425767cdbbe8b946d4637f0
author : Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 16:10:51 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 16:10:51 -0400
If an IO succeeds, but issues a warning, e.g. due to a page verification
failure with zero_damaged_pages, we want to issue that warning in the context
of the issuer of the IO, not the process that executes the completion (always
the case for worker).
It's already possible for a completion callback to report a custom error
message, we just didn't have a result status that allowed a user of AIO to
know that a warning should be emitted even though the IO request succeeded.
All that's needed for that is a dedicated PGAIO_RS_ value.
Previously there were not enough bits in PgAioResult.id for the new
value. Increase. While at that, add defines for the amount of bits and static
asserts to check that the widths are appropriate.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250329212929.a6.nmisch@google.com
M src/backend/storage/aio/aio.c
M src/backend/storage/aio/aio_callback.c
M src/include/storage/aio.h
M src/include/storage/aio_types.h
Let caller of PageIsVerified() control ignore_checksum_failure
commit : d445990adc419f435360f0dcd91c05c082f5fa3f
author : Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 16:10:51 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 16:10:51 -0400
For AIO the completion of a read into shared buffers (i.e. verifying the page
including the checksum, updating the BufferDesc to reflect the IO) can happen
in a different backend than the backend that started the IO. As
ignore_checksum_failure can differ between backends, we need to allow the
caller of PageIsVerified() control whether to ignore checksum failures.
The commit leaves a gap in the PIV_* values, as an upcoming commit, which
depends on this commit, will add PIV_LOG_LOG, which better fits just after
PIV_LOG_WARNING.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250329212929.a6.nmisch@google.com
M src/backend/catalog/storage.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/page/bufpage.c
M src/include/storage/bufpage.h
pgstat: Allow checksum errors to be reported in critical sections
commit : b96d3c389755fc5d20f4a5b9ded58b68541b8ba3
author : Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 16:10:51 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 16:10:51 -0400
For AIO we execute completion callbacks in critical sections (to ensure that
AIO can in the future be used for WAL, which in turn requires that we can call
completion callbacks in critical sections, to get the resources for WAL
io). To report checksum errors a backend now has to call
pgstat_prepare_report_checksum_failure(), before entering a critical section,
which guarantees the relevant pgstats entry is in shared memory, the relevant
DSM segment is mapped into the backend's memory and the address is known via a
PgStat_EntryRef.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/wkjj4p2rmkevutkwc6tewoovdqznj6c6nvjmvii4oo5wmbh5sr@retq7d6uqs4j
M src/backend/backup/basebackup.c
M src/backend/catalog/storage.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/utils/activity/pgstat_database.c
M src/include/pgstat.h
Add errhint_internal()
commit : 4244cf68769773ba30b868354f1f2fe93238e98b
author : Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 16:10:51 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sun, 30 Mar 2025 16:10:51 -0400
We have errmsg_internal(), errdetail_internal(), but not errhint_internal().
Sometimes it is useful to output a hint with already translated format
string (e.g. because there different messages depending on the condition). For
message/detail we do that with the _internal() variants, but we can't do that
with hint today. It's possible to work around that that by using something
like
str = psprintf(translated_format, args);
ereport(...
errhint("%s", str);
but that's not exactly pretty and makes it harder to avoid memory leaks.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/ym3dqpa4xcvoeknewcw63x77vnqdosbqcetjinb2zfoh65k55m@m4ozmwhr6lk6
M src/backend/utils/error/elog.c
M src/include/utils/elog.h
Remove incidental md5() function use from test
commit : 49b82522f13fa5f756687f5609f687877fc970ff
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 30 Mar 2025 13:22:39 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 30 Mar 2025 13:22:39 +0200
Replace md5() with sha256() in tests introduced in 14ffaece0fb5, to
allow test to pass in OpenSSL FIPS mode.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/3518736.1743307492@sss.pgh.pa.us
M contrib/amcheck/expected/check_gin.out
M contrib/amcheck/sql/check_gin.sql
localbuf: Track pincount in BufferDesc as well
commit : d6d8054dc72d4844f52f1552f8d3074c16987e32
author : Andres Freund <andres@anarazel.de>
date : Sat, 29 Mar 2025 16:36:51 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sat, 29 Mar 2025 16:36:51 -0400
For AIO on temporary table buffers the AIO subsystem needs to be able to
ensure a pin on a buffer while AIO is going on, even if the IO issuing query
errors out. Tracking the buffer in LocalRefCount does not work, as it would
cause CheckForLocalBufferLeaks() to assert out.
Instead, also track the refcount in BufferDesc.state, not just
LocalRefCount. This also makes local buffers behave a bit more akin to shared
buffers.
Note that we still don't need locking, AIO completion callbacks for local
buffers are executed in the issuing session (i.e. nobody else has access to
the BufferDesc).
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
M src/backend/storage/buffer/localbuf.c
aio, bufmgr: Comment fixes/improvements
commit : 08ccd56ac765496a152ba50cf0ae743c39396f52
author : Andres Freund <andres@anarazel.de>
date : Sat, 29 Mar 2025 14:45:42 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sat, 29 Mar 2025 14:45:42 -0400
Some of these comments have been wrong for a while (12f3867f5534), some I
recently introduced (da7226993fd, 55b454d0e14). This includes an update to a
comment in FlushBuffer(), which will be copied in a future commit.
These changes seem big enough to be worth doing in separate commits.
Suggested-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250319212530.80.nmisch@google.com
M src/backend/postmaster/postmaster.c
M src/backend/storage/buffer/bufmgr.c
M src/include/storage/aio.h
M src/include/storage/aio_internal.h
aio: Implement support for reads in smgr/md/fd
commit : 50cb7505b3010736b9a7922e903931534785f3aa
author : Andres Freund <andres@anarazel.de>
date : Sat, 29 Mar 2025 13:38:35 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sat, 29 Mar 2025 13:38:35 -0400
This implements the following:
1) An smgr AIO target, for AIO on smgr files. This should be usable not just
for md.c but also other SMGR implementation if we ever get them.
2) readv support in fd.c, which requires a small bit of infrastructure work in
fd.c
3) smgr.c and md.c support for readv
There still is nothing performing AIO, but as of this commit it would be
possible.
As part of this change FileGetRawDesc() actually ensures that the file is
opened - previously it was basically not usable. It's used to reopen a file in
IO workers.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
Discussion: https://postgr.es/m/20210223100344.llw5an2aklengrmn@alap3.anarazel.de
Discussion: https://postgr.es/m/stj36ea6yyhoxtqkhpieia2z4krnam7qyetc57rfezgk4zgapf@gcnactj4z56m
M src/backend/storage/aio/aio_callback.c
M src/backend/storage/aio/aio_target.c
M src/backend/storage/file/fd.c
M src/backend/storage/smgr/md.c
M src/backend/storage/smgr/smgr.c
M src/include/storage/aio.h
M src/include/storage/aio_types.h
M src/include/storage/fd.h
M src/include/storage/md.h
M src/include/storage/smgr.h
Fix mis-attribution of checksum failure stats to the wrong database
commit : dee80024688c32fc259a3cff040b609b837a35ec
author : Andres Freund <andres@anarazel.de>
date : Sat, 29 Mar 2025 13:38:35 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sat, 29 Mar 2025 13:38:35 -0400
Checksum failure stats could be attributed to the wrong database in two cases:
- when a read of a shared relation encountered a checksum error , it would be
attributed to the current database, instead of the "database" representing
shared relations
- when using CREATE DATABASE ... STRATEGY WAL_LOG checksum errors in the
source database would be attributed to the current database
The checksum stats reporting via PageIsVerifiedExtended(PIV_REPORT_STAT) does
not have access to the information about what database a page belongs to.
This fixes the issue by removing PIV_REPORT_STAT and delegating the
responsibility to report stats to the caller, which now can learn about the
number of stats via a new optional argument.
As this changes the signature of PageIsVerifiedExtended() and all callers
should adapt to the new signature, use the occasion to rename the function to
PageIsVerified() and remove the compatibility macro.
We could instead have fixed this by adding information about the database to
the args of PageIsVerified(), but there are soon-to-be-applied patches that
need to separate the stats reporting from the PageIsVerified() call
anyway. Those patches also include testing for the failure paths, something we
inexplicably have not had.
As there is no caller of pgstat_report_checksum_failure() left, remove it.
It'd be possible, but awkward to fix this in the back branches. We considered
doing the work not quite worth it, as mis-attributed stats should still elicit
concern. The emitted error messages do allow to attribute the errors
correctly.
Discussion: https://postgr.es/m/5tyic6epvdlmd6eddgelv47syg2b5cpwffjam54axp25xyq2ga@ptwkinxqo3az
Discussion: https://postgr.es/m/mglpvvbhighzuwudjxzu4br65qqcxsnyvio3nl4fbog3qknwhg@e4gt7npsohuz
M src/backend/catalog/storage.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/page/bufpage.c
M src/backend/utils/activity/pgstat_database.c
M src/include/pgstat.h
M src/include/storage/bufpage.h
amcheck: Add a GIN index to the CREATE INDEX CONCURRENTLY tests
commit : 68f97aeadb8e50794addaf5c1f8e9a67f75691b8
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 29 Mar 2025 16:46:49 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 29 Mar 2025 16:46:49 +0100
The existing CREATE INDEX CONCURRENTLY tests checking only B-Tree, but
can be cheaply extended to also check GIN. This helps increasing test
coverage for GIN amcheck, especially related to handling concurrent page
splits and posting list trees.
This already helped to identify several issues during development of the
GIN amcheck support.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Reviewed-By: Tomas Vondra <tomas.vondra@enterprisedb.com>
Reviewed-By: Kirill Reshke <reshkekirill@gmail.com>
Discussion: https://postgr.es/m/BC221A56-977C-418E-A1B8-9EFC881D80C5%40enterprisedb.com
M contrib/amcheck/t/002_cic.pl
M contrib/amcheck/t/003_cic_2pc.pl
amcheck: Add a test with GIN index on JSONB data
commit : ca738bdc4c9d32c05a6abd125ce495dfb82be492
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 29 Mar 2025 16:46:34 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 29 Mar 2025 16:46:34 +0100
Extend the existing test of GIN checks to also include an index on JSONB
data, using the jsonb_path_ops opclass. This is a common enough usage of
GIN that it makes sense to have better test coverage for it.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Reviewed-By: Tomas Vondra <tomas.vondra@enterprisedb.com>
Reviewed-By: Kirill Reshke <reshkekirill@gmail.com>
Discussion: https://postgr.es/m/BC221A56-977C-418E-A1B8-9EFC881D80C5%40enterprisedb.com
M contrib/amcheck/expected/check_gin.out
M contrib/amcheck/sql/check_gin.sql
amcheck: Fix indentation in verify_gin.c
commit : ec4327d106be745534592e8aff14effb716f4dc9
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 29 Mar 2025 16:42:13 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 29 Mar 2025 16:42:13 +0100
I forgot to reindent the code after a couple last-minute adjustments
just before committing 14ffaece0fb53fed8ddbc46d2b353e1c4834863a.
Discussion: https://postgr.es/m/45AC9B0A-2B45-40EE-B08F-BDCF5739D1E1%40yandex-team.ru
M contrib/amcheck/verify_gin.c
Fix "‘static’ is not at beginning of declaration" warning
commit : 116e851db5b3e51c962cc3468ba3055291cd8697
author : Andres Freund <andres@anarazel.de>
date : Sat, 29 Mar 2025 10:48:59 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sat, 29 Mar 2025 10:48:59 -0400
b98be8a2a2a used "const static" instead of "static const". We normally use the
latter form.
Discussion: https://postgr.es/m/z4mc2hzecahyq3paupfsouhuupmzmgum45md3k5my6bmo7gvn7@z5j26doqamqy
M src/port/pg_localeconv_r.c
amcheck: Add gin_index_check() to verify GIN index
commit : 14ffaece0fb53fed8ddbc46d2b353e1c4834863a
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 29 Mar 2025 15:43:55 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 29 Mar 2025 15:43:55 +0100
Adds a new function, validating two kinds of invariants on a GIN index:
- parent-child consistency: Paths in a GIN graph have to contain
consistent keys. Tuples on parent pages consistently include tuples
from child pages; parent tuples do not require any adjustments.
- balanced-tree / graph: Each internal page has at least one downlink,
and can reference either only leaf pages or only internal pages.
The GIN verification is based on work by Grigory Kryachko, reworked by
Heikki Linnakangas and with various improvements by Andrey Borodin.
Investigation and fixes for multiple bugs by Kirill Reshke.
Author: Grigory Kryachko <GSKryachko@gmail.com>
Author: Heikki Linnakangas <hlinnaka@iki.fi>
Author: Andrey Borodin <amborodin@acm.org>
Reviewed-By: José Villanova <jose.arthur@gmail.com>
Reviewed-By: Aleksander Alekseev <aleksander@timescale.com>
Reviewed-By: Nikolay Samokhvalov <samokhvalov@gmail.com>
Reviewed-By: Andres Freund <andres@anarazel.de>
Reviewed-By: Tomas Vondra <tomas.vondra@enterprisedb.com>
Reviewed-By: Kirill Reshke <reshkekirill@gmail.com>
Reviewed-By: Mark Dilger <mark.dilger@enterprisedb.com>
Reviewed-By: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/45AC9B0A-2B45-40EE-B08F-BDCF5739D1E1%40yandex-team.ru
M contrib/amcheck/Makefile
A contrib/amcheck/amcheck–1.4–1.5.sql
M contrib/amcheck/amcheck.control
A contrib/amcheck/expected/check_gin.out
M contrib/amcheck/meson.build
A contrib/amcheck/sql/check_gin.sql
A contrib/amcheck/verify_gin.c
M doc/src/sgml/amcheck.sgml
M src/tools/pgindent/typedefs.list
pgbench: Make set_random_seed() 64-bit everywhere.
commit : 53a2a1564ae450a46a7c565756ab536b84150e36
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 29 Mar 2025 15:24:42 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 29 Mar 2025 15:24:42 +0100
Delete an intermediate variable, a redundant cast, a use of long and a
use of long long. scanf() the seed directly into a uint64, now that we
can do that with SCNu64 from <inttypes.h>.
The previous coding was from pre-C99 times when %lld might not have been
there, so it read into an unsigned long. Therefore behavior varied
by OS, and --random-seed would accept either 32 or 64 bit seeds. Now
it's the same everywhere.
Author: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/b936d2fb-590d-49c3-a615-92c3a88c6c19%40eisentraut.org
M src/bin/pgbench/pgbench.c
amcheck: Move common routines into a separate module
commit : d70b17636ddf1ea2c71d1c7bc477372b36ccb66b
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 29 Mar 2025 15:14:47 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 29 Mar 2025 15:14:47 +0100
Before performing checks on an index, we need to take some safety
measures that apply to all index AMs. This includes:
* verifying that the index can be checked - Only selected AMs are
supported by amcheck (right now only B-Tree). The index has to be
valid and not a temporary index from another session.
* changing (and then restoring) user's security context
* obtaining proper locks on the index (and table, if needed)
* discarding GUC changes from the index functions
Until now this was implemented in the B-Tree amcheck module, but it's
something every AM will have to do. So relocate the code into a new
module verify_common for reuse.
The shared steps are implemented by amcheck_lock_relation_and_check(),
receiving the AM-specific verification as a callback. Custom parameters
may be supplied using a pointer.
Author: Andrey Borodin <amborodin@acm.org>
Reviewed-By: José Villanova <jose.arthur@gmail.com>
Reviewed-By: Aleksander Alekseev <aleksander@timescale.com>
Reviewed-By: Nikolay Samokhvalov <samokhvalov@gmail.com>
Reviewed-By: Andres Freund <andres@anarazel.de>
Reviewed-By: Tomas Vondra <tomas@vondra.me>
Reviewed-By: Mark Dilger <mark.dilger@enterprisedb.com>
Reviewed-By: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Kirill Reshke <reshkekirill@gmail.com>
Discussion: https://postgr.es/m/45AC9B0A-2B45-40EE-B08F-BDCF5739D1E1%40yandex-team.ru
M contrib/amcheck/Makefile
M contrib/amcheck/expected/check_btree.out
M contrib/amcheck/meson.build
A contrib/amcheck/verify_common.c
A contrib/amcheck/verify_common.h
M contrib/amcheck/verify_nbtree.c
M src/tools/pgindent/typedefs.list
Fix grammar in GIN README
commit : fb9dff76635d4c32198f30a3cb503588d557d156
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 29 Mar 2025 15:14:09 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 29 Mar 2025 15:14:09 +0100
Author: Kirill Reshke <reshkekirill@gmail.com>
Discussion: https://postgr.es/m/CALdSSPgu9uAhVYojQ0yjG%3Dq5MaqmiSLUJPhz%2B-u7cA6K6Mc9UA%40mail.gmail.com
M src/backend/access/gin/README
Fix MERGE with DO NOTHING actions into a partitioned table.
commit : 8b6a0e2392b9fc3f9b821da705797bb54c982dc1
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sat, 29 Mar 2025 09:58:40 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sat, 29 Mar 2025 09:58:40 +0000
ExecInitPartitionInfo() duplicates much of the logic in
ExecInitMerge(), except that it failed to handle DO NOTHING
actions. This would cause an "unknown action in MERGE WHEN clause"
error if a MERGE with any DO NOTHING actions attempted to insert into
a partition not already initialised by ExecInitModifyTable().
Bug: #18871
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Gurjeet Singh <gurjeet@singh.im>
Discussion: https://postgr.es/m/18871-b44e3c96de3bd2e8%40postgresql.org
Backpatch-through: 15
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/test/regress/expected/merge.out
M src/test/regress/sql/merge.sql
Use PRI?64 instead of "ll?" in format strings (continued).
commit : a0ed19e0a9efe93b3b83d6e3fe8f77656be253a2
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 29 Mar 2025 10:30:08 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 29 Mar 2025 10:30:08 +0100
Continuation of work started in commit 15a79c73, after initial trial.
Author: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/b936d2fb-590d-49c3-a615-92c3a88c6c19%40eisentraut.org
M contrib/file_fdw/file_fdw.c
M contrib/pageinspect/btreefuncs.c
M contrib/pageinspect/hashfuncs.c
M contrib/pg_prewarm/pg_prewarm.c
M src/backend/access/brin/brin.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/rmgrdesc/clogdesc.c
M src/backend/access/rmgrdesc/committsdesc.c
M src/backend/access/rmgrdesc/mxactdesc.c
M src/backend/access/rmgrdesc/xactdesc.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/xloginsert.c
M src/backend/access/transam/xlogreader.c
M src/backend/backup/basebackup_incremental.c
M src/backend/backup/walsummaryfuncs.c
M src/backend/catalog/catalog.c
M src/backend/commands/analyze.c
M src/backend/commands/copy.c
M src/backend/commands/copyfrom.c
M src/backend/commands/copyfromparse.c
M src/backend/commands/explain.c
M src/backend/commands/sequence.c
M src/backend/commands/vacuum.c
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/storage/ipc/ipc.c
M src/backend/storage/ipc/signalfuncs.c
M src/backend/utils/activity/pgstat.c
M src/backend/utils/activity/pgstat_replslot.c
M src/backend/utils/activity/pgstat_shmem.c
M src/backend/utils/activity/pgstat_xact.c
M src/backend/utils/adt/datetime.c
M src/backend/utils/adt/varlena.c
M src/backend/utils/adt/xid8funcs.c
M src/backend/utils/error/csvlog.c
M src/backend/utils/error/elog.c
M src/backend/utils/error/jsonlog.c
M src/backend/utils/mmgr/mcxt.c
M src/backend/utils/sort/logtape.c
M src/backend/utils/sort/tuplesort.c
M src/bin/pg_basebackup/pg_createsubscriber.c
M src/bin/pg_combinebackup/pg_combinebackup.c
M src/bin/pg_combinebackup/write_manifest.c
M src/bin/pg_controldata/pg_controldata.c
M src/bin/pg_resetwal/pg_resetwal.c
M src/bin/pg_rewind/libpq_source.c
M src/bin/pg_verifybackup/astreamer_verify.c
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/fe_utils/print.c
M src/include/lib/radixtree.h
M src/test/modules/test_copy_callbacks/test_copy_callbacks.c
M src/test/modules/test_slru/test_slru.c
M src/test/modules/xid_wraparound/xid_wraparound.c
Matview statistics depend on matview data.
commit : a0a4601765b896079eb82a9d5cfa1f41154fcfdb
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 28 Mar 2025 16:12:55 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 28 Mar 2025 16:12:55 -0700
REFRESH MATERIALIZED VIEW replaces the storage, which resets
statistics, so statistics must be restored afterward.
If both statistics and data are being dumped for a materialized view,
add a dependency from the former to the latter. Defer the statistics
to SECTION_POST_DATA, and use RESTORE_PASS_POST_ACL.
Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CAExHW5s47kmubpbbRJzSM-Zfe0Tj2O3GBagB7YAyE8rQ-V24Uw@mail.gmail.com
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/pg_dump_sort.c
Make group_similar_or_args() reorder clause list as little as possible
commit : 775a06d44c04e323158826ec256386e7211e154d
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 28 Mar 2025 23:36:49 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 28 Mar 2025 23:36:49 +0200
Currently, group_similar_or_args() permutes original positions of clauses
independently on whether it manages to find any groups of similar clauses.
While we are not providing any strict warranties on saving the original order
of OR-clauses, it is preferred that the original order be modified as little
as possible.
This commit changes the reordering algorithm of group_similar_or_args() in
the following way. We reorder each group of similar clauses so that the
first item of the group stays in place, but all the other items are moved
after it. So, if there are no similar clauses, the order of clauses stays
the same. When there are some groups, only required reordering happens while
the rest of the clauses remain in their places.
Reported-by: Andrei Lepikhov <lepihov@gmail.com>
Discussion: https://postgr.es/m/3ac7c436-81e1-4191-9caf-b0dd70b51511%40gmail.com
Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com>
Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Alena Rybakina <a.rybakina@postgrespro.ru>
M src/backend/optimizer/path/indxpath.c
M src/test/regress/expected/create_index.out
M src/test/regress/expected/join.out
M src/test/regress/expected/partition_join.out
M src/test/regress/sql/create_index.sql
Optimize popcount functions with ARM SVE intrinsics.
commit : 519338ace410d9b1ffb13176b8802b0307ff0531
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 28 Mar 2025 16:20:20 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 28 Mar 2025 16:20:20 -0500
This commit introduces SVE implementations of pg_popcount{32,64}.
Unlike the Neon versions, we need an additional configure-time
check to determine if the compiler supports SVE intrinsics, and we
need a runtime check to determine if the current CPU supports SVE
instructions. Our testing showed that the SVE implementations are
much faster for larger inputs and are comparable to the status
quo for smaller inputs.
Author: "Devanga.Susmitha@fujitsu.com" <Devanga.Susmitha@fujitsu.com>
Co-authored-by: "Chiranmoy.Bhattacharya@fujitsu.com" <Chiranmoy.Bhattacharya@fujitsu.com>
Co-authored-by: "Malladi, Rama" <ramamalladi@hotmail.com>
Reviewed-by: John Naylor <johncnaylorls@gmail.com>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Discussion: https://postgr.es/m/010101936e4aaa70-b474ab9e-b9ce-474d-a3ba-a3dc223d295c-000000%40us-west-2.amazonses.com
Discussion: https://postgr.es/m/OSZPR01MB84990A9A02A3515C6E85A65B8B2A2%40OSZPR01MB8499.jpnprd01.prod.outlook.com
M config/c-compiler.m4
M configure
M configure.ac
M meson.build
M src/include/pg_config.h.in
M src/include/port/pg_bitutils.h
M src/port/pg_popcount_aarch64.c
Revert "Tidy up locale thread safety in ECPG library."
commit : 3c8e463b0d885e0d976f6a13a1fb78187b25c86f
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 21:27:37 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 21:27:37 +0100
This reverts commit 8e993bff5326b00ced137c837fce7cd1e0ecae14.
It causes various build failures on the buildfarm, to be investigated.
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
M configure
M configure.ac
M meson.build
M src/include/pg_config.h.in
M src/include/port.h
M src/include/port/win32_port.h
M src/interfaces/ecpg/ecpglib/connect.c
M src/interfaces/ecpg/ecpglib/data.c
M src/interfaces/ecpg/ecpglib/descriptor.c
M src/interfaces/ecpg/ecpglib/ecpglib_extern.h
M src/interfaces/ecpg/ecpglib/execute.c
M src/interfaces/ecpg/pgtypeslib/dt_common.c
M src/interfaces/ecpg/pgtypeslib/interval.c
M src/interfaces/ecpg/pgtypeslib/numeric.c
M src/port/Makefile
D src/port/locale.c
M src/port/meson.build
M src/port/snprintf.c
Optimize popcount functions with ARM Neon intrinsics.
commit : 6be53c27673a5fca64a00a684c36c29db6ca33a5
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 28 Mar 2025 14:49:35 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 28 Mar 2025 14:49:35 -0500
This commit introduces Neon implementations of pg_popcount{32,64},
pg_popcount(), and pg_popcount_masked(). As in simd.h, we assume
that all available AArch64 hardware supports Neon, so we don't need
any new configure-time or runtime checks. Some compilers already
emit Neon instructions for these functions, but our hand-rolled
implementations for pg_popcount() and pg_popcount_masked()
performed better in testing, likely due to better instruction-level
parallelism.
Author: "Chiranmoy.Bhattacharya@fujitsu.com" <Chiranmoy.Bhattacharya@fujitsu.com>
Reviewed-by: John Naylor <johncnaylorls@gmail.com>
Discussion: https://postgr.es/m/010101936e4aaa70-b474ab9e-b9ce-474d-a3ba-a3dc223d295c-000000%40us-west-2.amazonses.com
M src/include/port/pg_bitutils.h
M src/port/Makefile
M src/port/meson.build
M src/port/pg_bitutils.c
A src/port/pg_popcount_aarch64.c
Fix crash if LockErrorCleanup() is called twice
commit : 51a0382e8d8793b5cc89b69285e5ecdffe03c2bf
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 28 Mar 2025 20:19:17 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 28 Mar 2025 20:19:17 +0200
The refactoring in commit 3c0fd64fec removed the clearing of
awaitedLock from LockErrorCleanup(). It's still needed, otherwise
LockErrorCleanup() during abort processing will try to update the
LOCALLOCK struct even after the lock has already been released. Put it
back.
Reported-by: Richard Guo <guofenglinux@gmail.com>
Reported-by: Robins Tharakan <tharakan@gmail.com>
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAMbWs4_dNX1SzBmvFdoY-LxJh_4W_BjtVd5i008ihfU-wFF=eg@mail.gmail.com
Discussion: https://www.postgresql.org/message-id/18832-38e5575b1bbd7277@postgresql.org
Discussion: https://www.postgresql.org/message-id/e11a30e5-c0d8-491d-8546-3a1b50c10ad4@gmail.com
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
M src/include/storage/lock.h
Rename TRY_POPCNT_FAST to TRY_POPCNT_X86_64.
commit : 9ac6f7e7ceb6e7c9b168bbf02bbe4662782dfa6e
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 28 Mar 2025 12:27:47 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 28 Mar 2025 12:27:47 -0500
This macro protects x86_64-specific code, and a subsequent commit
will introduce AArch64-specific versions of that code. To prevent
confusion, let's rename it to clearly indicate that it's for
x86_64. We should likely move this code to its own file (perhaps
merging it with the AVX-512 popcount code), but that is left as a
future exercise.
Reviewed-by: "Chiranmoy.Bhattacharya@fujitsu.com" <Chiranmoy.Bhattacharya@fujitsu.com>
Reviewed-by: John Naylor <johncnaylorls@gmail.com>
Discussion: https://postgr.es/m/010101936e4aaa70-b474ab9e-b9ce-474d-a3ba-a3dc223d295c-000000%40us-west-2.amazonses.com
M src/include/port/pg_bitutils.h
M src/port/pg_bitutils.c
M src/port/pg_popcount_avx512.c
Fix timestamp overflow in UUIDv7 implementation.
commit : a5419bc72e22337a16655aa28e2e20ecb65f65c3
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 28 Mar 2025 09:39:11 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 28 Mar 2025 09:39:11 -0700
The uuidv7_interval() function previously converted a shifted
microsecond-precision timestamp (64-bit integer) to another 64-bit
integer representing a timestamp with nanosecond precision. This
conversion caused overflow for dates beyond the year 2262. The
millisecond and sub-millisecond parts were then extracted from this
nanosecond-precision timestamp and stored in UUIDv7 values.
With this commit, the millisecond and sub-millisecond parts are stored
directly into the UUIDv7 value without being converted back to a
nanosecond precision timestamp. Following RFC 9562, the timestamp is
stored as an unsigned integer, enabling support for dates up to the
year 10889.
Reported and fixed by Andrey Borodin, with cosmetic changes and
regression tests by me.
Reported-by: Andrey Borodin <x4mmm@yandex-team.ru>
Author: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/96DEC2D9-659A-40E8-B7BA-AF5D162A9E21@yandex-team.ru
M src/backend/utils/adt/uuid.c
M src/test/regress/expected/uuid.out
M src/test/regress/sql/uuid.sql
Tidy up locale thread safety in ECPG library.
commit : 8e993bff5326b00ced137c837fce7cd1e0ecae14
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 16:15:57 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 16:15:57 +0100
Remove setlocale() and _configthreadlocal() as fallback strategy on
systems that don't have uselocale(), where ECPG tries to control
LC_NUMERIC formatting on input and output of floating point numbers. It
was probably broken on some systems (NetBSD), and the code was also
quite messy and complicated, with obsolete configure tests (Windows).
It was also arguably broken, or at least had unstated environmental
requirements, if pgtypeslib code was called directly.
Instead, introduce PG_C_LOCALE to refer to the "C" locale as a locale_t
value. It maps to the special constant LC_C_LOCALE when defined by libc
(macOS, NetBSD), or otherwise uses a process-lifetime locale_t that is
allocated on first use, just as ECPG previously did itself. The new
replacement might be more widely useful. Then change the float parsing
and printing code to pass that to _l() functions where appropriate.
Unfortunately the portability of those functions is a bit complicated.
First, many obvious and useful _l() functions are missing from POSIX,
though most standard libraries define some of them anyway. Second,
although the thread-safe save/restore technique can be used to replace
the missing ones, Windows and NetBSD refused to implement standard
uselocale(). They might have a point: "wide scope" uselocale() is hard
to combine with other code and error-prone, especially in library code.
Luckily they have the _l() functions we want so far anyway. So we have
to be prepared for both ways of doing things:
1. In ECPG, use strtod_l() for parsing, and supply a port.h replacement
using uselocale() over a limited scope if missing.
2. Inside our own snprintf.c, use three different approaches to format
floats. For frontend code, call libc's snprintf_l(), or wrap libc's
snprintf() in uselocale() if it's missing. For backend code, snprintf.c
can keep assuming that the global locale's LC_NUMERIC is "C" and call
libc's snprintf() without change, for now.
(It might eventually be possible to call our in-tree Ryū routines to
display floats in snprintf.c, given the C-locale-always remit of our
in-tree snprintf(), but this patch doesn't risk changing anything that
complicated.)
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Tristan Partin <tristan@partin.io>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
M configure
M configure.ac
M meson.build
M src/include/pg_config.h.in
M src/include/port.h
M src/include/port/win32_port.h
M src/interfaces/ecpg/ecpglib/connect.c
M src/interfaces/ecpg/ecpglib/data.c
M src/interfaces/ecpg/ecpglib/descriptor.c
M src/interfaces/ecpg/ecpglib/ecpglib_extern.h
M src/interfaces/ecpg/ecpglib/execute.c
M src/interfaces/ecpg/pgtypeslib/dt_common.c
M src/interfaces/ecpg/pgtypeslib/interval.c
M src/interfaces/ecpg/pgtypeslib/numeric.c
M src/port/Makefile
A src/port/locale.c
M src/port/meson.build
M src/port/snprintf.c
Cast result of i64abs() back to int64
commit : 2247281c470502c799fc2153a3583c025e894a9c
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 14:05:45 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 14:05:45 +0100
Without the cast, the return type could be long or long long,
depending on what int64 is underneath. This doesn't affect code
correctness, but it could result in format-mismatch warnings when
attempting to printf such values using PRId64.
Reported-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+hUKGJc4s+Wyb3EFOQNN9VVK+Qv40r2LK41o9PkS9ThxviTvQ@mail.gmail.com
M src/include/c.h
pg_overexplain: Use PG_MODULE_MAGIC_EXT.
commit : 83ccc85859f30af44785b0dee830f7071d99ec4f
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 28 Mar 2025 09:16:29 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 28 Mar 2025 09:16:29 -0400
I committed this contrib module just after Tom committed
55527368bd07248e91e3d37a782bf66b76f06865; adjust it to match.
Author: Man Zeng <zengman@halodbtech.com>
Discussion: http://postgr.es/m/174313513707.60295.16516085012903412705.pgcf@coridan.postgresql.org
M contrib/pg_overexplain/pg_overexplain.c
pg_overexplain: Call previous hooks as appropriate.
commit : 9f0c36aea0fba7d2b737964d34ed582edf644283
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 28 Mar 2025 08:59:33 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 28 Mar 2025 08:59:33 -0400
It makes no sense to remember the previous values of the hook variables
and then never bother calling those functions. Thanks to Andrei for
spotting my goof.
Author: Andrei Lepikhov <lepihov@gmail.com>
Discussion: http://postgr.es/m/41a344e3-ffb1-4296-8ba7-801f1e9642e5@gmail.com
M contrib/pg_overexplain/pg_overexplain.c
Add support for not-null constraints on virtual generated columns
commit : cdc168ad4b22ea4183f966688b245cabb5935d1f
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 13:53:37 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 13:53:37 +0100
This was left out of the original patch for virtual generated columns
(commit 83ea6c54025).
This just involves a bit of extra work in the executor to expand the
generation expressions and run a "IS NOT NULL" test against them.
There is also a bit of work to make sure that not-null constraints are
checked during a table rewrite.
Author: jian he <jian.universality@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Navneet Kumar <thanit3111@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/CACJufxHArQysbDkWFmvK+D1TPHQWWTxWN15cMuUaTYX3xhQXgg@mail.gmail.com
M src/backend/catalog/heap.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execMain.c
M src/backend/parser/parse_utilcmd.c
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
M src/test/regress/expected/generated_virtual.out
M src/test/regress/sql/generated_virtual.sql
Modernize some code a bit
commit : 747ddd38cbf6d32bca496e69c1efb2ae4fe333cc
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 10:49:15 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 10:49:15 +0100
Modernize code in ExecRelCheck() and ExecConstraints() a bit,
preparing the way for some new code.
Co-authored-by: jian he <jian.universality@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Navneet Kumar <thanit3111@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/CACJufxHArQysbDkWFmvK+D1TPHQWWTxWN15cMuUaTYX3xhQXgg@mail.gmail.com
M src/backend/executor/execMain.c
Rename a node field for clarity
commit : 9a9ead1105482fc292eccf707697da2ebcc578e5
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 09:50:01 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 09:50:01 +0100
Rename ResultRelInfo.ri_ConstraintExprs to ri_CheckConstraintExprs.
This reflects its specific purpose better and avoids confusion with
adjacent fields with similar but distinct purposes.
Discussion: https://postgr.es/m/CACJufxHArQysbDkWFmvK+D1TPHQWWTxWN15cMuUaTYX3xhQXgg@mail.gmail.com
M src/backend/executor/execMain.c
M src/include/nodes/execnodes.h
pg_createsubscriber: Add '--all' option.
commit : fb2ea12f42b9453853be043b8ed107e136e1ccb7
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 28 Mar 2025 12:26:39 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 28 Mar 2025 12:26:39 +0530
The '--all' option indicates that the tool queries the source server
(publisher) for all databases and creates subscriptions on the target
server (subscriber) for databases with matching names. Without this user
needs to explicitly specify all databases by using -d option for each
database.
This simplifies converting a physical standby to a logical subscriber,
particularly during upgrades.
The options '--database', '--publication', '--subscription', and
'--replication-slot' cannot be used when '--all' is specified.
Author: Shubham Khanna <khannashubham1197@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Euler Taveira <euler@eulerto.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Shlok Kyal <shlok.kyal.oss@gmail.com>
Discussion: https://postgr.es/m/CAHv8RjKhA=_h5vAbozzJ1Opnv=KXYQHQ-fJyaMfqfRqPpnC2bA@mail.gmail.com
M doc/src/sgml/ref/pg_createsubscriber.sgml
M src/bin/pg_basebackup/pg_createsubscriber.c
M src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
Use thread-safe strftime_l() instead of strftime().
commit : 890fc826c91f153ae8c084d3a008f91798495b13
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 07:13:43 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 28 Mar 2025 07:13:43 +0100
This removes some setlocale() calls and a lot of commentary about how
dangerous that is. strftime_l() is from POSIX 2008, and on Windows we
use _wcsftime_l().
While here, adjust error message for strftime_l() failure: it does not
in practice set errno (even though POSIX says it could), so no %m.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/CA%2BhUKGJqVe0%2BPv9dvC9dSums_PXxGo9SWcxYAMBguWJUGbWz-A%40mail.gmail.com
M src/backend/main/main.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_locale_libc.c
M src/include/utils/pg_locale.h
Stablize tests added in 3abe9dc188.
commit : 474d7a1fd858d5ee138cf4504dcd93a9a3123a7d
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 28 Mar 2025 11:03:05 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 28 Mar 2025 11:03:05 +0530
The problem is that after the ALTER SUBSCRIPTION tap_sub SET PUBLICATION
command, we didn't wait for the new walsender to start on the publisher.
Immediately after ALTER, we performed Insert and expected it to replicate.
However, the replication could start from a point after the INSERT location,
and as the subscription isn't copying initial data, we could miss such an
Insert.
The fix is to wait for connection to be established between publisher and
subscriber before starting DML operations that are expected to replicate.
As per CI.
Reported-by: Andres Freund <andres@anarazel.de>
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Discussion: https://postgr.es/m/CALDaNm2ms1deM5EYNLFEfESv_Kw=Y4AiTB0LP=qGS-UpFwGbPg@mail.gmail.com
M src/test/subscription/t/007_ddl.pl
Fix guc_malloc calls for consistency and OOM checks
commit : 058b5152f02ef86c98a795c14dbd6a8e195f4fd1
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 27 Mar 2025 22:57:34 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 27 Mar 2025 22:57:34 +0100
check_createrole_self_grant and check_synchronized_standby_slots
were allocating memory on a LOG elevel without checking if the
allocation succeeded or not, which would have led to a segfault
on allocation failure.
On top of that, a number of callsites were using the ERROR level,
relying on erroring out rather than returning false to allow the
GUC machinery handle it gracefully. Other callsites used WARNING
instead of LOG. While neither being not wrong, this changes all
check_ functions do it consistently with LOG.
init_custom_variable gets a promoted elevel to FATAL to keep
the guc_malloc error handling in line with the rest of the
error handling in that function which already call FATAL. If
we encounter an OOM in this callsite there is no graceful
handling to be had, better to error out hard.
Backpatch the fix to check_createrole_self_grant down to v16
and the fix to check_synchronized_standby_slots down to v17
where they were introduced.
Author: Daniel Gustafsson <daniel@yesql.se>
Reported-by: Nikita <pm91.arapov@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Bug: #18845
Discussion: https://postgr.es/m/18845-582c6e10247377ec@postgresql.org
Backpatch-through: 16
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogrecovery.c
M src/backend/commands/user.c
M src/backend/commands/variable.c
M src/backend/replication/slot.c
M src/backend/storage/file/fd.c
M src/backend/tcop/backend_startup.c
M src/backend/tcop/postgres.c
M src/backend/utils/error/elog.c
M src/backend/utils/misc/guc.c
Use streaming read I/O in heap amcheck
commit : 043799fa08c2c71f35816ca067951266d2e9ebe0
author : Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 27 Mar 2025 14:02:40 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 27 Mar 2025 14:02:40 -0400
Instead of directly invoking ReadBuffer() for each unskippable block in
the heap relation, verify_heapam() now uses the read stream API to
acquire the next buffer to check for corruption.
Author: Matheus Alcantara <matheusssilv97@gmail.com>
Co-authored-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Reviewed-by: jian he <jian.universality@gmail.com>
Discussion: https://postgr.es/m/flat/CAFY6G8eLyz7%2BsccegZYFj%3D5tAUR-GZ9uEq4Ch5gvwKqUwb_hCA%40mail.gmail.com
M contrib/amcheck/verify_heapam.c
M src/tools/pgindent/typedefs.list
Prevent assertion failure in contrib/pg_freespacemap.
commit : 4623d71443de40781135a7040079ab4538974a6f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Mar 2025 13:20:23 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Mar 2025 13:20:23 -0400
Applying pg_freespacemap() to a relation lacking storage (such as a
view) caused an assertion failure, although there was no ill effect
in non-assert builds. Add an error check for that case.
Bug: #18866
Reported-by: Robins Tharakan <tharakan@gmail.com>
Author: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Euler Taveira <euler@eulerto.com>
Discussion: https://postgr.es/m/18866-d68926d0f1c72d44@postgresql.org
Backpatch-through: 13
M contrib/pg_freespacemap/pg_freespacemap.c
Avoid mixing designated and non-designated field initializers.
commit : d66997dfe8fe5a2fc6e25a64e78309ca2094f396
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Mar 2025 11:06:30 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Mar 2025 11:06:30 -0400
As revised by commit 9324c8c58, PG_MODULE_MAGIC constructed a
struct initializer containing both designated fields and a
non-designated "0". That's okay in C, but not in C++, with
the result that extensions written in C++ failed to compile.
Change it to use only designated field initializers.
Author: Yurii Rashkovskii <yrashk@omnigres.com>
Discussion: https://postgr.es/m/CAG=VW14mctsR543gpzLCuJ9JgJqwa=ptmBfGvxEjs+k8Jf7-Bg@mail.gmail.com
M src/include/fmgr.h
psql: Fix incorrect equality comparison
commit : 0f3604a518f8b3fd35ffc344d85c71693ded0dde
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 27 Mar 2025 14:09:25 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 27 Mar 2025 14:09:25 +0100
Commit 1a759c83278 contained an incorrect equality comparison
which was discovered by Coverity.
Reported-by: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/CAEudQApfAWzLo+oSuy2byXktdr7R8KJC_ACT5VV8fontrL35Pw@mail.gmail.com
M src/bin/psql/variables.c
pg_overexplain: Filter out actual row count from test result.
commit : 081ec08e6a148b06a0179bf8ac50524b1f142b68
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 27 Mar 2025 09:00:46 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 27 Mar 2025 09:00:46 -0400
Per buildfarm, these are not stable. In particular, 1/8 is sometimes
0.12 and sometimes 0.13.
M contrib/pg_overexplain/expected/pg_overexplain.out
M contrib/pg_overexplain/sql/pg_overexplain.sql
Remove the query_id_squash_values GUC
commit : 9fbd53dea5d513a78ca04834101ca1aa73b63e59
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 27 Mar 2025 13:33:37 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 27 Mar 2025 13:33:37 +0100
Commit 62d712ecfd94 introduced the capability to calculate the same
queryId for queries with different lengths of constants in a list for an
IN clause. This behavior was originally enabled with a GUC
query_id_squash_values. After a discussion about the value of such a
GUC, it was decided to back out of the use of a GUC and make the
squashing behavior the only available option.
Author: Sami Imseih <samimseih@gmail.com>
Discussion: https://postgr.es/m/Z-LZyygkkNyA8-kR@msg.df7cb.de
Discussion: https://postgr.es/m/CA+q6zcVTK-3C-8NWV1oY2NZrvtnMCDqnyYYyk1T7WMUG65MeOQ@mail.gmail.com
M contrib/pg_stat_statements/expected/dml.out
M contrib/pg_stat_statements/expected/squashing.out
M contrib/pg_stat_statements/sql/squashing.sql
M doc/src/sgml/config.sgml
M doc/src/sgml/pgstatstatements.sgml
M src/backend/nodes/queryjumblefuncs.c
M src/backend/postmaster/launch_backend.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/nodes/queryjumble.h
Expand test a bit
commit : 5d5f415816a60a3c5c5e4420eff55e73edfbd9f5
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 27 Mar 2025 12:11:15 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 27 Mar 2025 12:11:15 +0100
Make pg_constraint output in inherit test show the convalidated column
as well. This shows the interaction between convalidated and
conenforced.
This is extracted from a larger patch so that this reformatting isn't
distracting there.
Author: Amul Sul <amul.sul@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/CAAJ_b962c5AcYW9KUt_R_ER5qs3fUGbe4az-SP-vuwPS-w-AGA@mail.gmail.com
M src/test/regress/expected/inherit.out
M src/test/regress/sql/inherit.sql
Provide thread-safe pg_localeconv_r().
commit : b98be8a2a2a6196dc24631dfe4d8785b86800d23
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 27 Mar 2025 07:52:22 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 27 Mar 2025 07:52:22 +0100
This involves four different implementation strategies:
1. For Windows, we now require _configthreadlocale() to be available
and work (commit f1da075d9a0), and the documentation says that the
object returned by localeconv() is in thread-local memory.
2. For glibc, we translate to nl_langinfo_l() calls, because it
offers the same information that way as an extension, and that API is
thread-safe.
3. For macOS/*BSD, use localeconv_l(), which is thread-safe.
4. For everything else, use uselocale() to set the locale for the
thread, and use a big ugly lock to defend against the returned object
being concurrently clobbered. In practice this currently means only
Solaris.
The new call is used in pg_locale.c, replacing calls to setlocale() and
localeconv().
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/CA%2BhUKGJqVe0%2BPv9dvC9dSums_PXxGo9SWcxYAMBguWJUGbWz-A%40mail.gmail.com
M configure
M configure.ac
M meson.build
M src/backend/utils/adt/pg_locale.c
M src/include/pg_config.h.in
M src/include/port.h
M src/port/Makefile
M src/port/meson.build
A src/port/pg_localeconv_r.c
Simplify syntax for ALTER TABLE ALTER CONSTRAINT NO INHERIT
commit : 4a02af8b1a5f3f8221c7381bc4de08a3d830f682
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 27 Mar 2025 09:24:52 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 27 Mar 2025 09:24:52 +0100
Commit d45597f72fe5 introduced the ability to change a not-null
constraint from NO INHERIT to INHERIT and vice versa, but we included
the SET noise word in the syntax for it. The SET turns out not to be
necessary and goes against what the SQL standard says for other ALTER
TABLE subcommands, so remove it.
This changes the way this command is processed for constraint types
other than not-null, so there are some error message changes.
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Suraj Kharage <suraj.kharage@enterprisedb.com>
Discussion: https://postgr.es/m/202503251602.vsxaehsyaoac@alvherre.pgsql
M doc/src/sgml/ref/alter_table.sgml
M src/backend/commands/tablecmds.c
M src/backend/parser/gram.y
M src/test/regress/expected/foreign_key.out
M src/test/regress/expected/inherit.out
M src/test/regress/sql/inherit.sql
libpq: Add TAP tests for service files and names
commit : 72c2f36d572763201430a35414138c1b9ec3a106
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Mar 2025 16:01:38 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Mar 2025 16:01:38 +0900
This commit adds a set of regression tests that checks various patterns
with service names and service files, with:
- Service file with no contents, used as default for PGSERVICEFILE to
prevent any lookups at the HOME directory of an environment where the
test is run.
- Service file with valid service name and its section.
- Service file at the root of PGSYSCONFDIR, named pg_service.conf.
- Missing service file.
- Service name defined as a connection parameter or as PGSERVICE.
Note that PGSYSCONFDIR is set to always point at a temporary directory
created by the test, so as we never try to look at SYSCONFDIR.
This set of tests has come up as a useful independent addition while
discussing a patch that adds an equivalent of PGSERVICEFILE as a
connection parameter as there have never been any tests for service
files and service names. Torsten Foertsch and Ryo Kanbayashi have
provided a basic implementation, that I have expanded to what is
introduced in this commit.
Author: Torsten Foertsch <tfoertsch123@gmail.com>
Author: Ryo Kanbayashi <kanbayashi.dev@gmail.com>
Author: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAKkG4_nCjx3a_F3gyXHSPWxD8Sd8URaM89wey7fG_9g7KBkOCQ@mail.gmail.com
M src/interfaces/libpq/meson.build
A src/interfaces/libpq/t/006_service.pl
Optimize Query jumble
commit : ad9a23bc4f51eea829ee99c7d9940b7ac9f523e3
author : David Rowley <drowley@postgresql.org>
date : Thu, 27 Mar 2025 18:34:34 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 27 Mar 2025 18:34:34 +1300
f31aad9b0 adjusted query jumbling so it no longer ignores NULL nodes
during the jumble. This added some overhead. Here we tune a few
things to make jumbling faster again. This makes jumbling perform
similar or even slightly faster than prior to that change.
Author: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAApHDvreP04nhTKuYsPw0F-YN+4nr4f=L72SPeFb81jfv+2c7w@mail.gmail.com
M src/backend/nodes/queryjumblefuncs.c
Fix query jumbling to account for NULL nodes
commit : f31aad9b07b7a6ef693d7059830bd2f1504976ae
author : David Rowley <drowley@postgresql.org>
date : Thu, 27 Mar 2025 18:23:00 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 27 Mar 2025 18:23:00 +1300
Previously NULL nodes were ignored. This could cause issues where the
computed query ID could match for queries where fields that are next to
each other in their Node struct where one field was NULL and the other
non-NULL. For example, the Query struct had distinctClause and sortClause
next to each other. If someone wrote;
SELECT DISTINCT c1 FROM t;
and then;
SELECT c1 FROM t ORDER BY c1;
these would produce the same query ID since, in the first query, we
ignored the NULL sortClause and appended the jumble bytes for the
distictClause. In the latter query, since we did nothing for the NULL
distinctClause then jumble the non-NULL sortClause, and since the node
representation stored is the same in both cases, the query IDs were
identical.
Here we fix this by always accounting for NULL nodes by recording that
we saw a NULL in the jumble buffer. This fixes the issue as the order that
the NULL is recorded isn't the same in the above two queries.
Author: Bykov Ivan <i.bykov@modernsys.ru>
Author: Michael Paquier <michael@paquier.xyz>
Author: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/aafce7966e234372b2ba876c0193f1e9%40localhost.localdomain
M contrib/pg_stat_statements/expected/select.out
M contrib/pg_stat_statements/sql/select.sql
M src/backend/nodes/queryjumblefuncs.c
M src/include/nodes/queryjumble.h
doc: Correct description of values used in FSM for indexes
commit : 44fe6ceb51f001689048c19ea3ea53fbf2572581
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Mar 2025 10:20:41 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Mar 2025 10:20:41 +0900
The implementation of FSM for indexes is simpler than heap, where 0 is
used to track if a page is in-use and (BLCKSZ - 1) if a page is free.
One comment in indexfsm.c and one description in the documentation of
pg_freespacemap were incorrect about that.
Author: Alex Friedman <alexf01@gmail.com>
Discussion: https://postgr.es/m/71eef655-c192-453f-ac45-2772fec2cb04@gmail.com
Backpatch-through: 13
M doc/src/sgml/pgfreespacemap.sgml
M src/backend/storage/freespace/indexfsm.c
aio: Add io_method=io_uring
commit : c325a7633fcb33dbd73f46ddbbe91e95ddf3b227
author : Andres Freund <andres@anarazel.de>
date : Tue, 18 Mar 2025 14:40:05 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 18 Mar 2025 14:40:05 -0400
Performing AIO using io_uring can be considerably faster than
io_method=worker, particularly when lots of small IOs are issued, as
a) the context-switch overhead for worker based AIO becomes more significant
b) the number of IO workers can become limiting
io_uring, however, is linux specific and requires an additional compile-time
dependency (liburing).
This implementation is fairly simple and there are substantial optimization
opportunities.
The description of the existing AIO_IO_COMPLETION wait event is updated to
make the difference between it and the new AIO_IO_URING_EXECUTION clearer.
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Jakub Wartak <jakub.wartak@enterprisedb.com>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
Discussion: https://postgr.es/m/20210223100344.llw5an2aklengrmn@alap3.anarazel.de
Discussion: https://postgr.es/m/stj36ea6yyhoxtqkhpieia2z4krnam7qyetc57rfezgk4zgapf@gcnactj4z56m
M .cirrus.tasks.yml
M doc/src/sgml/config.sgml
M src/backend/storage/aio/Makefile
M src/backend/storage/aio/aio.c
M src/backend/storage/aio/aio_io.c
M src/backend/storage/aio/meson.build
A src/backend/storage/aio/method_io_uring.c
M src/backend/storage/lmgr/lwlock.c
M src/backend/utils/activity/wait_event_names.txt
M src/backend/utils/misc/postgresql.conf.sample
M src/include/storage/aio.h
M src/include/storage/aio_internal.h
M src/include/storage/lwlock.h
M src/tools/pgindent/typedefs.list
aio: Add liburing dependency
commit : 8eadd5c73c44708ecd45b9fd3ac54a550511d16f
author : Andres Freund <andres@anarazel.de>
date : Tue, 18 Mar 2025 14:40:05 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 18 Mar 2025 14:40:05 -0400
Will be used in a subsequent commit, to implement io_method=io_uring. Kept
separate for easier review.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
M .cirrus.tasks.yml
M configure
M configure.ac
M doc/src/sgml/installation.sgml
M meson.build
M meson_options.txt
M src/Makefile.global.in
M src/backend/Makefile
M src/include/pg_config.h.in
M src/makefiles/meson.build
doc: Mention possible ephemeral discrepancies in pg_stat_activity
commit : f056f75dafd0025d26efaca026a87f14c079a130
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Mar 2025 08:07:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Mar 2025 08:07:54 +0900
Ephemeral inconsistencies across multiple attributes of pg_stat_activity
can exist as the system is designed to be efficient with a low overhead.
This question is raised by users from time to time based on the data
read in the view, so let's add a note in the docs about this
possibility.
Author: Alex Friedman <alexf01@gmail.com>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Discussion: https://postgr.es/m/8a275154-a654-44b0-ab37-197802f04c7b@gmail.com
M doc/src/sgml/monitoring.sgml
aio: Rename pgaio_io_prep_* to pgaio_io_start_*
commit : 9469d7fdd2bcf2327ad91a1022885953a461942a
author : Andres Freund <andres@anarazel.de>
date : Wed, 26 Mar 2025 16:10:29 -0400
committer: Andres Freund <andres@anarazel.de>
date : Wed, 26 Mar 2025 16:10:29 -0400
The old naming pattern (mirroring liburing's naming) was inconsistent with
the (not yet introduced) callers. It seems better to get rid of the
inconsistency now than to grow more users of the odd naming.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250326001915.bc.nmisch@google.com
M src/backend/storage/aio/aio.c
M src/backend/storage/aio/aio_callback.c
M src/backend/storage/aio/aio_io.c
M src/backend/storage/aio/aio_target.c
M src/include/storage/aio.h
M src/include/storage/aio_internal.h
aio: Pass result of local callbacks to ->report_return
commit : f321ec237a54912b28d81acc4bfc4e434d9339c7
author : Andres Freund <andres@anarazel.de>
date : Wed, 26 Mar 2025 16:06:54 -0400
committer: Andres Freund <andres@anarazel.de>
date : Wed, 26 Mar 2025 16:06:54 -0400
Otherwise the results of e.g. temp table buffer verification errors will not
reach bufmgr.c. Obviously that's not right. Found while expanding the tests
for invalid buffer contents.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250326001915.bc.nmisch@google.com
M src/backend/storage/aio/aio.c
M src/backend/storage/aio/aio_callback.c
M src/include/storage/aio_internal.h
aio: Be more paranoid about interrupts
commit : 96da9050a57aece4a48ab34a84bc3b3412708a20
author : Andres Freund <andres@anarazel.de>
date : Wed, 26 Mar 2025 16:06:54 -0400
committer: Andres Freund <andres@anarazel.de>
date : Wed, 26 Mar 2025 16:06:54 -0400
As reported by Noah, it's possible, although practically very unlikely, that
interrupts could be processed in between pgaio_io_reopen() and
pgaio_io_perform_synchronously(). Prevent that by explicitly holding
interrupts.
It also seems good to add an assertion to pgaio_io_before_prep() to ensure
that interrupts are held, as otherwise FDs referenced by the IO could be
closed during interrupt processing. All code in the aio series currently runs
the code with interrupts held, but it seems better to be paranoid.
Reviewed-by: Noah Misch <noah@leadboat.com>
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20250324002939.5c.nmisch@google.com
M src/backend/storage/aio/aio_io.c
M src/backend/storage/aio/method_worker.c
pg_overexplain: SET jit=off when running tests.
commit : 47a1f076a7c9789134a29dc738db0152e5f71b4c
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 26 Mar 2025 15:43:25 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 26 Mar 2025 15:43:25 -0400
Per buildfarm.
M contrib/pg_overexplain/expected/pg_overexplain.out
M contrib/pg_overexplain/sql/pg_overexplain.sql
Fix oversights in commit 8d5ceb113e3f7ddb627bd40b26438a9d2fa05512
commit : de65c4dade0a36026f3d0745dc8134186dab49cf
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 26 Mar 2025 14:22:45 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 26 Mar 2025 14:22:45 -0400
It added bogus whitespace at the end of a line in the documentation.
It should not have done that.
The pg_overexplain tests must SET debug_parallel_query = false,
not just RESET debug_parallel_query, or we get failures on test
machines that make debug_parallel_query = true the defualt.
M contrib/pg_overexplain/expected/pg_overexplain.out
M contrib/pg_overexplain/sql/pg_overexplain.sql
M doc/src/sgml/pgoverexplain.sgml
pg_overexplain: Additional EXPLAIN options for debugging.
commit : 8d5ceb113e3f7ddb627bd40b26438a9d2fa05512
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 26 Mar 2025 13:52:21 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 26 Mar 2025 13:52:21 -0400
There's a fair amount of information in the Plan and PlanState trees
that isn't printed by any existing EXPLAIN option. This means that,
when working on the planner, it's often necessary to rely on facilities
such as debug_print_plan, which produce excessively voluminous
output. Hence, use the new EXPLAIN extension facilities to implement
EXPLAIN (DEBUG) and EXPLAIN (RANGE_TABLE) as extensions to the core
EXPLAIN facility.
A great deal more could be done here, and the specific choices about
what to print and how are definitely arguable, but this is at least
a starting point for discussion and a jumping-off point for possible
future improvements.
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviweed-by: Andrei Lepikhov <lepihov@gmail.com> (who didn't like it)
Discussion: http://postgr.es/m/CA+TgmoZfvQUBWQ2P8iO30jywhfEAKyNzMZSR+uc2xr9PZBw6eQ@mail.gmail.com
M contrib/Makefile
M contrib/meson.build
A contrib/pg_overexplain/.gitignore
A contrib/pg_overexplain/Makefile
A contrib/pg_overexplain/expected/pg_overexplain.out
A contrib/pg_overexplain/meson.build
A contrib/pg_overexplain/pg_overexplain.c
A contrib/pg_overexplain/sql/pg_overexplain.sql
M doc/src/sgml/contrib.sgml
M doc/src/sgml/filelist.sgml
A doc/src/sgml/pgoverexplain.sgml
M src/tools/pgindent/typedefs.list
Keep the decompressed filter in brin_bloom_union
commit : 818245506c285e0325141dabb1ced45057937502
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 26 Mar 2025 16:50:13 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 26 Mar 2025 16:50:13 +0100
The brin_bloom_union() function combines two BRIN summaries, by merging
one filter into the other. With bloom, we have to decompress the filters
first, but the function failed to update the summary to store the merged
filter. As a consequence, the index may be missing some of the data, and
return false negatives.
This issue exists since BRIN bloom indexes were introduced in Postgres
14, but at that point the union function was called only when two
sessions happened to summarize a range concurrently, which is rare. It
got much easier to hit in 17, as parallel builds use the union function
to merge summaries built by workers.
Fixed by storing a pointer to the decompressed filter, and freeing the
original one. Free the second filter too, if it was decompressed. The
freeing is not strictly necessary, because the union is called in
short-lived contexts, but it's tidy.
Backpatch to 14, where BRIN bloom indexes were introduced.
Reported by Arseniy Mukhin, investigation and fix by me.
Reported-by: Arseniy Mukhin
Discussion: https://postgr.es/m/18855-1cf1c8bcc22150e6%40postgresql.org
Backpatch-through: 14
M src/backend/access/brin/brin_bloom.c
Use PG_MODULE_MAGIC_EXT in our installable shared libraries.
commit : 55527368bd07248e91e3d37a782bf66b76f06865
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 26 Mar 2025 11:11:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 26 Mar 2025 11:11:02 -0400
It seems potentially useful to label our shared libraries with version
information, now that a facility exists for retrieving that. This
patch labels them with the PG_VERSION string. There was some
discussion about using semantic versioning conventions, but that
doesn't seem terribly helpful for modules with no SQL-level presence;
and for those that do have SQL objects, we typically expect them
to support multiple revisions of the SQL definitions, so it'd still
not be very helpful.
I did not label any of src/test/modules/. It seems unnecessary since
we don't install those, and besides there ought to be someplace that
still provides test coverage for the original PG_MODULE_MAGIC macro.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/dd4d1b59-d0fe-49d5-b28f-1e463b68fa32@gmail.com
M contrib/amcheck/verify_nbtree.c
M contrib/auth_delay/auth_delay.c
M contrib/basebackup_to_shell/basebackup_to_shell.c
M contrib/basic_archive/basic_archive.c
M contrib/bloom/blinsert.c
M contrib/bool_plperl/bool_plperl.c
M contrib/btree_gin/btree_gin.c
M contrib/btree_gist/btree_gist.c
M contrib/citext/citext.c
M contrib/cube/cube.c
M contrib/dblink/dblink.c
M contrib/dict_int/dict_int.c
M contrib/dict_xsyn/dict_xsyn.c
M contrib/earthdistance/earthdistance.c
M contrib/file_fdw/file_fdw.c
M contrib/fuzzystrmatch/fuzzystrmatch.c
M contrib/hstore/hstore_io.c
M contrib/hstore_plperl/hstore_plperl.c
M contrib/hstore_plpython/hstore_plpython.c
M contrib/intarray/_int_op.c
M contrib/isn/isn.c
M contrib/jsonb_plperl/jsonb_plperl.c
M contrib/jsonb_plpython/jsonb_plpython.c
M contrib/lo/lo.c
M contrib/ltree/ltree_op.c
M contrib/ltree_plpython/ltree_plpython.c
M contrib/pageinspect/rawpage.c
M contrib/passwordcheck/passwordcheck.c
M contrib/pg_buffercache/pg_buffercache_pages.c
M contrib/pg_freespacemap/pg_freespacemap.c
M contrib/pg_logicalinspect/pg_logicalinspect.c
M contrib/pg_prewarm/pg_prewarm.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_surgery/heap_surgery.c
M contrib/pg_trgm/trgm_op.c
M contrib/pg_visibility/pg_visibility.c
M contrib/pg_walinspect/pg_walinspect.c
M contrib/pgcrypto/pgcrypto.c
M contrib/pgrowlocks/pgrowlocks.c
M contrib/pgstattuple/pgstattuple.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/seg/seg.c
M contrib/sepgsql/hooks.c
M contrib/spi/autoinc.c
M contrib/spi/insert_username.c
M contrib/spi/moddatetime.c
M contrib/spi/refint.c
M contrib/sslinfo/sslinfo.c
M contrib/tablefunc/tablefunc.c
M contrib/tcn/tcn.c
M contrib/test_decoding/test_decoding.c
M contrib/tsm_system_rows/tsm_system_rows.c
M contrib/tsm_system_time/tsm_system_time.c
M contrib/unaccent/unaccent.c
M contrib/uuid-ossp/uuid-ossp.c
M contrib/xml2/xpath.c
M src/backend/jit/llvm/llvmjit.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/snowball/dict_snowball.c
M src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
M src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
M src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
M src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
M src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
M src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
M src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
M src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
M src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
M src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
M src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
M src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
M src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
M src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
M src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
M src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
M src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
M src/pl/plperl/plperl.c
M src/pl/plpgsql/src/pl_handler.c
M src/pl/plpython/plpy_main.c
M src/pl/tcl/pltcl.c
M src/test/regress/regress.c
Introduce PG_MODULE_MAGIC_EXT macro.
commit : 9324c8c580655800331b0582b770e88c01b7a5c4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 26 Mar 2025 10:59:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 26 Mar 2025 10:59:42 -0400
This macro allows dynamically loaded shared libraries (modules) to
provide a wired-in module name and version, and possibly other
compile-time-constant fields in future. This information can be
retrieved with the new pg_get_loaded_modules() function.
This feature is expected to be particularly useful for modules
that do not have any exposed SQL functionality and thus are
not associated with a SQL-level extension object. But even for
modules that do belong to extensions, being able to verify the
actual code version can be useful.
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Yurii Rashkovskii <yrashk@omnigres.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/dd4d1b59-d0fe-49d5-b28f-1e463b68fa32@gmail.com
M contrib/auto_explain/auto_explain.c
M contrib/auto_explain/t/001_auto_explain.pl
M doc/src/sgml/func.sgml
M doc/src/sgml/xfunc.sgml
M src/backend/commands/extension.c
M src/backend/utils/fmgr/dfmgr.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/fmgr.h
M src/tools/pgindent/typedefs.list
Move GSSAPI includes into its own header
commit : e92c0632c1473fe57383c58f0dfdde3bae7044f4
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 26 Mar 2025 15:31:46 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 26 Mar 2025 15:31:46 +0100
Due to a conflict in macro names on Windows between <wincrypt.h>
and <openssl/ssl.h> these headers need to be included using a
predictable pattern with an undef to handle that. The GSSAPI
header <gssapi.h> does include <wincrypt.h> which cause problems
with compiling PostgreSQL using MSVC when OpenSSL and GSSAPI are
both enabled in the tree. Rather than fixing piecemeal for each
file including gssapi headers, move the the includes and undef
to a new file which should be used to centralize the logic.
This patch is a reworked version of a patch by Imran Zaheer
proposed earlier in the thread. Once this has proven effective
in master we should look at backporting this as the problem
exist at least since v16.
Author: Daniel Gustafsson <daniel@yesql.se>
Co-authored-by: Imran Zaheer <imran.zhir@gmail.com>
Reported-by: Dave Page <dpage@pgadmin.org>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/20240708173204.3f3xjilglx5wuzx6@awork3.anarazel.de
M contrib/sslinfo/sslinfo.c
M src/include/libpq/be-gssapi-common.h
M src/include/libpq/libpq-be.h
A src/include/libpq/pg-gssapi.h
M src/interfaces/libpq/libpq-int.h
psql: Make test robust against locale variations
commit : 1eb399366ed658d41b8795a0a25599a8599bbcfb
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 26 Mar 2025 13:20:56 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 26 Mar 2025 13:20:56 +0100
The test committed in 1a759c83278 was prone to failing when using
locales with a different decimal separator. Since the test value
isn't the important part, change to using an integer instead.
Author: Daniel Gustafsson <daniel@yesql.se>
Reported-by: Pavel Stehule <pavel.stehule@gmail.com>
Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://postgr.es/m/CAFj8pRDE=7uW7QP4rg-OQLE2i-puYsUUt+eHE-L6_b_J9w=eWg@mail.gmail.com
M src/bin/psql/t/001_basic.pl
dblink: SCRAM authentication pass-through
commit : 3642df265d09779443a9f44f5cb873df40974e89
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 26 Mar 2025 10:05:49 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 26 Mar 2025 10:05:49 +0100
This enables SCRAM authentication for dblink (using dblink_fdw) when
connecting to a foreign server without having to store a plain-text
password on user mapping options
This uses the same approach as it was implemented for postgres_fdw in
commit 761c79508e7. (It also contains the equivalent of the
subsequent fixes 76563f88cfb and d2028e9bbc1.)
Author: Matheus Alcantara <mths.dev@pm.me>
Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/CAFY6G8ercA1KES%3DE_0__R9QCTR805TTyYr1No8qF8ZxmMg8z2Q%40mail.gmail.com
M contrib/dblink/Makefile
M contrib/dblink/dblink.c
M contrib/dblink/meson.build
A contrib/dblink/t/001_auth_scram.pl
M doc/src/sgml/dblink.sgml
M doc/src/sgml/postgres-fdw.sgml
Add support for gamma() and lgamma() functions.
commit : a3b6dfd410699a19209be3cb3c7a74e5ff1da830
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Wed, 26 Mar 2025 09:35:53 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Wed, 26 Mar 2025 09:35:53 +0000
These are useful general-purpose math functions which are included in
POSIX and C99, and are commonly included in other math libraries, so
expose them as SQL-callable functions.
Author: Dean Rasheed <dean.a.rasheed@gmail.com>
Reviewed-by: Stepan Neretin <sncfmgg@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Dmitry Koval <d.koval@postgrespro.ru>
Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com>
Discussion: https://postgr.es/m/CAEZATCXpGyfjXCirFk9au+FvM0y2Ah+2-0WSJx7MO368ysNUPA@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/float.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/float8.out
M src/test/regress/sql/float8.sql
Fix integer-overflow problem in scram_SaltedPassword()
commit : 7c82b4f711877b175142bb2b2a6e2c2ee2429441
author : Richard Guo <rguo@postgresql.org>
date : Wed, 26 Mar 2025 17:46:51 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Wed, 26 Mar 2025 17:46:51 +0900
Setting the iteration count for SCRAM secret generation to INT_MAX
will cause an infinite loop in scram_SaltedPassword() due to integer
overflow, as the loop uses the "i <= iterations" comparison. To fix,
use "i < iterations" instead.
Back-patch to v16 where the user-settable GUC scram_iterations has
been added.
Author: Kevin K Biju <kevinkbiju@gmail.com>
Reviewed-by: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAM45KeEMm8hnxdTOxA98qhfZ9CzGDdgy3mxgJmy0c+2WwjA6Zg@mail.gmail.com
M src/common/scram-common.c
Use relation name instead of OID in query jumbling for RangeTblEntry
commit : 787514b30bb7dd0b3484d6cb717e3b9aafc06c4a
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Mar 2025 15:21:05 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Mar 2025 15:21:05 +0900
custom_query_jumble (introduced in 5ac462e2b7ac as a node field
attribute) is now assigned to the expanded reference name "eref" of
RangeTblEntry, adding in the query jumble computation the non-qualified
aliased relation name, without the list of column names. The relation
OID is removed from the query jumbling.
The effects of this change can be seen in the tests added by
3430215fe35f, where pg_stat_statements (PGSS) entries are now grouped
using the relation name, ignoring the relation search_path may point at.
For example, these two relations are different, but are now grouped in a
single PGSS entry as they are assigned the same query ID:
CREATE TABLE foo1.tab (a int);
CREATE TABLE foo2.tab (b int);
SET search_path = 'foo1';
SELECT count(*) FROM tab;
SET search_path = 'foo2';
SELECT count(*) FROM tab;
SELECT count(*) FROM foo1.tab;
SELECT count(*) FROM foo2.tab;
SELECT query, calls FROM pg_stat_statements WHERE query ~ 'FROM tab';
query | calls
--------------------------+-------
SELECT count(*) FROM tab | 4
(1 row)
It is still possible to use an alias in the FROM clause to split these.
This behavior is useful for relations re-created with the same name,
where queries based on such relations would be grouped in the same
PGSS entry. For permanent schemas, it should not really matter in
practice. The main benefit is for workloads that use a lot of temporary
relations, which are usually re-created with the same name continuously.
These can be a heavy source of bloat in PGSS depending on the workload.
Such entries can now be grouped together, improving the user experience.
The original idea from Christoph Berg used catalog lookups to find
temporary relations, something that the query jumble has never done, and
it could cause some performance regressions. The idea to use
RangeTblEntry.eref and the relation name, applying the same rules for
all relations, temporary and not temporary, has been proposed by Tom
Lane. The documentation additions have been suggested by Sami Imseih.
Author: Michael Paquier <michael@paquier.xyz>
Co-authored-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: Christoph Berg <myon@debian.org>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Discussion: https://postgr.es/m/Z9iWXKGwkm8RAC93@msg.df7cb.de
M contrib/pg_stat_statements/expected/select.out
M doc/src/sgml/pgstatstatements.sgml
M src/backend/nodes/queryjumblefuncs.c
M src/include/nodes/parsenodes.h
postgres_fdw: Fix tests on some Windows variants
commit : d2028e9bbc1af94d2ac390146293f02b548c95c9
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 26 Mar 2025 06:56:52 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 26 Mar 2025 06:56:52 +0100
The tests introduced by commit 76563f88cfb only work when Unix-domain
sockets are available. This is optional on Windows, and buildfarm
member drongo runs without them. To fix, skip the test if Unix-domain
sockets are not enabled.
M contrib/postgres_fdw/t/001_auth_scram.pl
Add pg_dump --with-{schema|data|statistics} options.
commit : bde2fb797aaebcbe06bf60f330ba5a068f17dda7
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 25 Mar 2025 17:36:38 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 25 Mar 2025 17:36:38 -0700
By adding the positive variants of options, in addition to the
negative variants that already exist, users can be explicit about what
pg_dump should produce.
Discussion: https://postgr.es/m/bd0513e4b1ea2b2f2d06f02720c6579711cb62a6.camel@j-davis.com
Reviewed-by: Corey Huinker <corey.huinker@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_restore.sgml
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_dump/pg_restore.c
M src/bin/pg_dump/t/002_pg_dump.pl
Fix two issues with custom_query_jumble in gen_node_support.pl
commit : 27ee6ede6bc9ee287f321d396c5c777083d40ecd
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Mar 2025 09:06:36 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Mar 2025 09:06:36 +0900
A node field marked with custom_query_jumble and query_jumble_ignore
would generate some code of a custom routine. The script is changed so
as custom_query_jumble behaves like the other options in this case,
query_jumble_ignore taking priority, with no code generated.
A comment related to the code generated for node types was misplaced.
Thinkos introduced in 5ac462e2b7ac.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/1324036.1742945060@sss.pgh.pa.us
M src/backend/nodes/gen_node_support.pl
Fix order of -I switches for building pg_regress.o.
commit : cb36f8ec2124b0c495da057a2f55735fe858be0e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 25 Mar 2025 20:03:56 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 25 Mar 2025 20:03:56 -0400
We need the -I switch for libpq_srcdir to come before any -I switches
injected by configure. Otherwise there is a risk of pulling in a
mismatched version of libpq_fe.h from someplace like
/usr/local/include, if the platform has another Postgres version
installed there. This evidently accounts for today's buildfarm
failures on "anaconda".
In principle the -I switch for src/port/ is at similar hazard, and has
been for a very long time. But the only .h files we keep there are
pg_config_paths.h and pthread-win32.h, neither of which get installed
on Unix-ish systems, so the odds of picking up a conflicting header
seem pretty small. That doubtless accounts for the lack of prior
reports.
Back-patch to v17 where pg_regress acquired a build dependency on
libpq_fe.h. We could go back further to fix the hazard for src/port/
in older branches, but it seems unlikely to be worth troubling over.
Reported-by: Nathan Bossart <nathandbossart@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/Z-MhRzoc7t-nPUQG@nathan
Backpatch-through: 17
M src/test/regress/GNUmakefile
pg_stat_statements: Add more tests with temp tables and namespaces
commit : 3430215fe35f96e4be355c93e275f00656800602
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Mar 2025 07:25:23 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Mar 2025 07:25:23 +0900
These tests provide coverage for RangeTblEntry and how query jumbling
works with search_path, as well as the case where relations are
re-created, generating a different query ID as the relation OID is used
in the computation.
A patch is under discussion to switch to a different approach based on
the relation name, and there was no test coverage for this area,
including how queries are currently grouped with search_path. This is
useful to track how the situation changes between HEAD and any patches
proposed.
Christoph has proposed the test with ON COMMIT DROP temporary tables,
and I have written the second part.
Author: Christoph Berg <myon@debian.org>
Author: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/Z9iWXKGwkm8RAC93@msg.df7cb.de
M contrib/pg_stat_statements/expected/select.out
M contrib/pg_stat_statements/sql/select.sql
pg_upgrade: Add --swap for faster file transfer.
commit : 626d7236b65da50423df7de035e86f273cd36b49
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 25 Mar 2025 16:02:35 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 25 Mar 2025 16:02:35 -0500
This new option instructs pg_upgrade to move the data directories
from the old cluster to the new cluster and then to replace the
catalog files with those generated for the new cluster. This mode
can outperform --link, --clone, --copy, and --copy-file-range,
especially on clusters with many relations.
However, this mode creates many garbage files in the old cluster,
which can prolong the file synchronization step if
--sync-method=syncfs is used. To handle that, we recommend using
--sync-method=fsync with this mode, and pg_upgrade internally uses
"initdb --sync-only --no-sync-data-files" for file synchronization.
pg_upgrade will synchronize the catalog files as they are
transferred. We assume that the database files transferred from
the old cluster were synchronized prior to upgrade.
This mode also complicates reverting to the old cluster, so we
recommend restoring from backup upon failure during or after file
transfer. We did consider teaching pg_upgrade how to generate a
revert script for such failures, but we decided against it due to
the rarity of failing during file transfer, the complexity of
generating the script, and the potential for misusing the script.
The new mode is limited to clusters located in the same file
system. With some effort, we could probably support upgrades
between different file systems, but this mode is unlikely to offer
much benefit if we have to copy the files across file system
boundaries.
It is also limited to upgrades from version 10 or newer. There are
a few known obstacles for using swap mode to upgrade from older
versions. For example, the visibility map format changed in v9.6,
and the sequence tuple format changed in v10. In fact, swap mode
omits the --sequence-data option in its uses of pg_dump and instead
reuses the old cluster's sequence data files. While teaching swap
mode to deal with these kinds of changes is surely possible (and we
may have to deal with similar problems in the future, anyway), it
doesn't seem worth the effort to support upgrades from
long-unsupported versions.
Reviewed-by: Greg Sabino Mullane <htamfids@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/Zyvop-LxLXBLrZil%40nathan
M doc/src/sgml/ref/pgupgrade.sgml
M src/bin/pg_upgrade/TESTING
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/dump.c
M src/bin/pg_upgrade/file.c
M src/bin/pg_upgrade/info.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/relfilenumber.c
M src/bin/pg_upgrade/t/006_transfer_modes.pl
M src/common/file_utils.c
M src/include/common/file_utils.h
pg_dump: Add --sequence-data.
commit : 9c49f0e8cd7d59e240f5da88decf2d62d8a4ad0d
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 25 Mar 2025 16:02:35 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 25 Mar 2025 16:02:35 -0500
This new option instructs pg_dump to dump sequence data when the
--no-data, --schema-only, or --statistics-only option is specified.
This was originally considered for commit a7e5457db8, but it was
left out at that time because there was no known use-case. A
follow-up commit will use this to optimize pg_upgrade's file
transfer step.
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/Zyvop-LxLXBLrZil%40nathan
M doc/src/sgml/ref/pg_dump.sgml
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/pg_upgrade/dump.c
M src/test/modules/test_pg_dump/t/001_base.pl
initdb: Add --no-sync-data-files.
commit : cf131fa942855acd42e8283185837566c7e8c15e
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 25 Mar 2025 16:02:35 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 25 Mar 2025 16:02:35 -0500
This new option instructs initdb to skip synchronizing any files
in database directories, the database directories themselves, and
the tablespace directories, i.e., everything in the base/
subdirectory and any other tablespace directories. Other files,
such as those in pg_wal/ and pg_xact/, will still be synchronized
unless --no-sync is also specified. --no-sync-data-files is
primarily intended for internal use by tools that separately ensure
the skipped files are synchronized to disk. A follow-up commit
will use this to help optimize pg_upgrade's file transfer step.
The --sync-method=fsync implementation of this option makes use of
a new exclude_dir parameter for walkdir(). When not NULL,
exclude_dir specifies a directory to skip processing. The
--sync-method=syncfs implementation of this option just skips
synchronizing the non-default tablespace directories. This means
that initdb will still synchronize some or all of the database
files, but there's not much we can do about that.
Discussion: https://postgr.es/m/Zyvop-LxLXBLrZil%40nathan
M doc/src/sgml/ref/initdb.sgml
M src/bin/initdb/initdb.c
M src/bin/initdb/t/001_initdb.pl
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_checksums/pg_checksums.c
M src/bin/pg_combinebackup/pg_combinebackup.c
M src/bin/pg_rewind/file_ops.c
M src/common/file_utils.c
M src/include/common/file_utils.h
Stats: use schemaname/relname instead of regclass.
commit : 650ab8aaf1957863ae14c80265e79f5d903b49fd
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 25 Mar 2025 11:16:06 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 25 Mar 2025 11:16:06 -0700
For import and export, use schemaname/relname rather than
regclass.
This is more natural during export, fits with the other arguments
better, and it gives better control over error handling in case we
need to downgrade more errors to warnings.
Also, use text for the argument types for schemaname, relname, and
attname so that casts to "name" are not required.
Author: Corey Huinker <corey.huinker@gmail.com>
Discussion: https://postgr.es/m/CADkLM=ceOSsx_=oe73QQ-BxUFR2Cwqum7-UP_fPe22DBY0NerA@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/statistics/attribute_stats.c
M src/backend/statistics/relation_stats.c
M src/backend/statistics/stat_utils.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/statistics/stat_utils.h
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Minor doc update for commit 99f8f3fbbc.
commit : 2a420f7995e415f4813fccf1c42ab29a3a32182f
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 25 Mar 2025 11:15:52 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 25 Mar 2025 11:15:52 -0700
Author: Corey Huinker <corey.huinker@gmail.com>
M doc/src/sgml/func.sgml
psql: Make default \watch interval configurable
commit : 1a759c83278fcdae11ee5da8318b646b9d47129c
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 25 Mar 2025 17:53:33 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 25 Mar 2025 17:53:33 +0100
The default interval for \watch to wait between executing queries,
when executed without a specified interval, was hardcoded to two
seconds. This adds the new variable WATCH_INTERVAL which is used
to set the default interval, making it configurable for the user.
This makes \watch the first command which has a user configurable
default setting.
Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Reviewed-by: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Greg Sabino Mullane <htamfids@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/B2FD26B4-8F64-4552-A603-5CC3DF1C7103@yesql.se
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/help.c
M src/bin/psql/settings.h
M src/bin/psql/startup.c
M src/bin/psql/t/001_basic.pl
M src/bin/psql/variables.c
M src/bin/psql/variables.h
pg_basebackup: Add missing PQclear in error path
commit : a19db082749662a933d0bf72722982b1ee2c4725
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 25 Mar 2025 17:24:23 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 25 Mar 2025 17:24:23 +0100
This adds a missing PQclear in the error path of StreamLogicalLog, a
fix in the same vein as e889422d98e with an equivalent low impact.
Author: Steven Niu <niushiji@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/c4b1c627-a3e4-4347-a670-1e28a43ce0eb@gmail.com
M src/bin/pg_basebackup/pg_recvlogical.c
refactor: Pass relation OID instead of Relation to createForeignKeyCheckTriggers()
commit : ef7a5af77d449ee2b426f627c1f7472740350040
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Mar 2025 16:45:28 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Mar 2025 16:45:28 +0100
Currently, createForeignKeyCheckTriggers() takes a Relation type as
its first argument, but it doesn't use that argument directly.
Instead, it fetches the relation OID by calling RelationGetRelid().
Therefore, it would be more consistent with other functions (e.g.,
createForeignKeyCheckTriggers()) to pass the relation OID directly
instead of the whole Relation.
Author: Amul Sul <amul.sul@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/CAAJ_b962c5AcYW9KUt_R_ER5qs3fUGbe4az-SP-vuwPS-w-AGA@mail.gmail.com
M src/backend/commands/tablecmds.c
refactor: Split ATExecAlterConstraintInternal()
commit : 639238b978fefe53b8412f2cc0265f6a2512887a
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Mar 2025 16:18:00 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Mar 2025 16:18:00 +0100
Split ATExecAlterConstraintInternal() into two functions:
ATExecAlterConstrDeferrability() and
ATExecAlterConstrInheritability(). This simplifies the code and
avoids unnecessary confusion caused by recursive code, which isn't
needed for ATExecAlterConstrInheritability().
(This also takes over the changes in commit 64224a834ce, as the new
AlterConstrDeferrabilityRecurse() is essentially the old
ATExecAlterChildConstr().)
Author: Amul Sul <amul.sul@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/CAAJ_b962c5AcYW9KUt_R_ER5qs3fUGbe4az-SP-vuwPS-w-AGA@mail.gmail.com
M src/backend/commands/tablecmds.c
refactor: Move some code that updates pg_constraint to a separate function
commit : a3280e2a494fe369a041ebdefdf2afb515952c18
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Mar 2025 14:37:22 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Mar 2025 14:37:22 +0100
This extracts common/duplicate code for different ALTER CONSTRAINT
variants into a common function. We plan to add more variants that
would use the same code.
Author: Amul Sul <amul.sul@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/CAAJ_b962c5AcYW9KUt_R_ER5qs3fUGbe4az-SP-vuwPS-w-AGA@mail.gmail.com
M src/backend/commands/tablecmds.c
Small fixes for Add ALTER TABLE ... ALTER CONSTRAINT ... SET [NO] INHERIT
commit : f4b2a62ae3147ab262eaa21f8be76e569ffd8dfa
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Mar 2025 13:40:24 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Mar 2025 13:40:24 +0100
Small fixes for commit f4e53e10b6c: Add missing calls to
InvokeObjectPostAlterHook() and also CacheInvalidateRelcache(). The
former change could have a user-visible effect. The latter omission
might have caused other bugs, but it is not clear whether one actually
existed. With these changes, the code is now more consistent with
similar ALTER CONSTRAINT variants, especially the ones that set the
deferrability.
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/CAF1DzPVfOW6Kk=7SSh7LbneQDJWh=PbJrEC_Wkzc24tHOyQWGg@mail.gmail.com
M src/backend/commands/tablecmds.c
postgres_fdw: Remove redundant check in semijoin_target_ok()
commit : 62f36d6924c740b5a566d64c563789d20cd4a5e1
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 25 Mar 2025 12:48:48 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 25 Mar 2025 12:48:48 +0200
If a var belongs to the innerrel of the joinrel, it's not possible that
it belongs to the outerrel. This commit removes the redundant check from
the if-clause but keeps it as an assertion.
Discussion: https://postgr.es/m/flat/CAHewXN=8aW4hd_W71F7Ua4+_w0=bppuvvTEBFBF6G0NuSXLwUw@mail.gmail.com
Author: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Alexander Pyhalov <a.yhalov@postgrespro.ru>
Backpatch-through: 17
M contrib/postgres_fdw/postgres_fdw.c
libpq: Deprecate pg_int64.
commit : 3c86223c998280ea313480d319ec39f802453218
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 25 Mar 2025 20:17:53 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 25 Mar 2025 20:17:53 +1300
Previously we used pg_int64 in three function prototypes in libpq. It
was added by commit 461ef73f to expose the platform-dependent type used
for int64 in the C89 era. As of commit 962da900 it is defined as
standard int64_t, and the dust seems to have settled.
Let's just use int64_t directly in these three client-facing functions
instead of (yet) another name. We've required C99 and thus <stdint.h>
since PostgreSQL 12, C89 and C++98 compilers are long gone, and client
applications very likely use standard types for their own 64-bit needs.
This also cleans up the obscure placement of a new #include <stdint.h>
directive in postgres_ext.h, required for the new definition. The
typedef was hiding in there for historical reasons, but it doesn't fit
postgres_ext.h's own description of its purpose and there is no evidence
of client applications including postgres_ext.h directly to see it.
Keep a typedef marked deprecated for backward compatibility, but move it
into libpq-fe.h where it was used.
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/CA%2BhUKGKn_EkNNGMY5RzMcKP%2Ba6urT4JF%3DCPhw_zHtQwjvX6P2g%40mail.gmail.com
M doc/src/sgml/libpq.sgml
M doc/src/sgml/lobj.sgml
M src/include/postgres_ext.h
M src/interfaces/libpq/fe-lobj.c
M src/interfaces/libpq/libpq-fe.h
M src/test/examples/testlo64.c
Generalize index support in network support function
commit : be1cc9aaf5b79f07bc0483a0c95366c77a844d0a
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Mar 2025 07:11:56 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Mar 2025 07:11:56 +0100
The network (inet) support functions currently only supported a
hardcoded btree operator family. With the generalized compare type
facility, we can generalize this to support any operator family from
any index type that supports the required operators.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Co-authored-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/utils/adt/network.c
Add support for custom_query_jumble as a node field attribute
commit : 5ac462e2b7ac97685de9f34b17296640220d0bfc
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 25 Mar 2025 14:18:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 25 Mar 2025 14:18:00 +0900
This option gives the possibility for query jumble to define a custom
routine for the field of a Node, extending support for
custom_query_jumble as a node field attribute. When dealing with
complex node structures, this can be simpler than having to enforce a
custom function across a full node.
Custom functions need to be defined in queryjumblefuncs.c, named as
_jumble${node}_${field}(), and use in input the JumbleState, the node
and its field. The field is not really required if we have the Node,
but it makes custom implementations somewhat easier to think about. The
code generated by gen_node_support.pl uses a macro called
JUMBLE_CUSTOM(), hiding the internals of the logic inside
queryjumblefuncs.c.
This will be used by an upcoming patch manipulating adding a custom
routine into a field of RangeTblEntry, but this facility can become
useful in more cases.
Reviewed-by: Christoph Berg <myon@debian.org>
Discussion: https://postgr.es/m/Z9y43-dRvb4EtxQ0@paquier.xyz
M src/backend/nodes/gen_node_support.pl
M src/backend/nodes/queryjumblefuncs.c
M src/include/nodes/nodes.h
Remove 'additional' pointer from TupleHashEntryData.
commit : 626df47ad9db809dc8f93330175ab95b75914721
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 24 Mar 2025 22:06:02 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 24 Mar 2025 22:06:02 -0700
Reduces memory required for hash aggregation by avoiding an allocation
and a pointer in the TupleHashEntryData structure. That structure is
used for all buckets, whether occupied or not, so the savings is
substantial.
Discussion: https://postgr.es/m/AApHDvpN4v3t_sdz4dvrv1Fx_ZPw=twSnxuTEytRYP7LFz5K9A@mail.gmail.com
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
M src/backend/executor/execGrouping.c
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
Add ExecCopySlotMinimalTupleExtra().
commit : a0942f441ed651f6345d969b7a8f4774eda1fceb
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 24 Mar 2025 22:05:53 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 24 Mar 2025 22:05:53 -0700
Allows an "extra" argument that allocates extra memory at the end of
the MinimalTuple. This is important for callers that need to store
additional data, but do not want to perform an additional allocation.
Suggested-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvppeqw2pNM-+ahBOJwq2QmC0hOAGsmCpC89QVmEoOvsdg@mail.gmail.com
M src/backend/access/common/heaptuple.c
M src/backend/executor/execTuples.c
M src/backend/executor/nodeGatherMerge.c
M src/backend/utils/sort/tuplesortvariants.c
M src/backend/utils/sort/tuplestore.c
M src/include/access/htup_details.h
M src/include/executor/tuptable.h
Create accessor functions for TupleHashEntry.
commit : 4d143509cbfae0207c35abffae7b0e3b4d078349
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 24 Mar 2025 22:05:41 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 24 Mar 2025 22:05:41 -0700
Refactor for upcoming optimizations.
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/1cc3b400a0e8eead18ff967436fa9e42c0c14cfb.camel@j-davis.com
M src/backend/executor/execGrouping.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeSetOp.c
M src/backend/executor/nodeSubplan.c
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
HashAgg: use Bump allocator for hash TupleHashTable entries.
commit : cc721c459d3783bbdb4beb1bbfa802a5328d15a2
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 24 Mar 2025 22:05:33 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 24 Mar 2025 22:05:33 -0700
The entries aren't freed until the entire hash table is destroyed, so
use the Bump allocator to improve allocation speed, avoid wasting
space on the chunk header, and avoid wasting space due to the
power-of-two allocations.
Discussion: https://postgr.es/m/CAApHDvqv1aNB4cM36FzRwivXrEvBO_LsG_eQ3nqDXTjECaatOQ@mail.gmail.com
Reviewed-by: David Rowley
M src/backend/executor/execUtils.c
M src/backend/executor/nodeAgg.c
M src/include/nodes/execnodes.h
Fix the typo in the test case added in 73eba5004a.
commit : cc4331605ad81ad3040704470ff56904624eef75
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 25 Mar 2025 09:39:53 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 25 Mar 2025 09:39:53 +0530
Author: vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/CALDaNm2ms1deM5EYNLFEfESv_Kw=Y4AiTB0LP=qGS-UpFwGbPg@mail.gmail.com
Discussion: https://postgr.es/m/CABdArM7FW-_dnthGkg2s0fy1HhUB8C3ELA0gZX1kkbs1ZZoV3Q@mail.gmail.com
M src/test/subscription/t/035_conflicts.pl
Fix an oversight in 3abe9dc188.
commit : b87ced747df74fd094aba3902bca3d3c6b5b247d
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 25 Mar 2025 09:26:23 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 25 Mar 2025 09:26:23 +0530
Forgot to update the comment atop one of the functions.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Discussion: https://postgr.es/m/OSCPR01MB1496623BE1125B44614494E7AF5A72@OSCPR01MB14966.jpnprd01.prod.outlook.com
M src/backend/replication/pgoutput/pgoutput.c
postgres_fdw: Avoid pulling up restrict infos from subqueries
commit : 023fb512755ffb64949eb6de8464c259ca70f1bd
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 25 Mar 2025 05:49:47 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 25 Mar 2025 05:49:47 +0200
Semi-join joins below left/right join are deparsed as
subqueries. Thus, we can't refer to subqueries vars from upper relations.
This commit avoids pulling conditions from them.
Reported-by: Robins Tharakan <tharakan@gmail.com>
Bug: #18852
Discussion: https://postgr.es/m/CAEP4nAzryLd3gwcUpFBAG9MWyDfMRX8ZjuyY2XXjyC_C6k%2B_Zw%40mail.gmail.com
Author: Alexander Pyhalov <a.pyhalov@postgrespro.ru>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Backpatch-through: 17
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
Redefine max_files_per_process to control additionally opened files
commit : adb5f85fa5a00d6d5759df55feb16dc22b0fc8d7
author : Andres Freund <andres@anarazel.de>
date : Mon, 24 Mar 2025 18:20:18 -0400
committer: Andres Freund <andres@anarazel.de>
date : Mon, 24 Mar 2025 18:20:18 -0400
Until now max_files_per_process=N limited each backend to open N files in
total (minus a safety factor), even if there were already more files opened in
postmaster and inherited by backends. Change max_files_per_process to control
how many additional files each process is allowed to open.
The main motivation for this is the patch to add io_method=io_uring, which
needs to open one file for each backend. Without this patch, even if
RLIMIT_NOFILE is high enough, postmaster will fail in set_max_safe_fds() if
started with a high max_connections. The cause of the failure is that, until
now, set_max_safe_fds() subtracted the already open files from
max_files_per_process.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/w6uiicyou7hzq47mbyejubtcyb2rngkkf45fk4q7inue5kfbeo@bbfad3qyubvs
Discussion: https://postgr.es/m/CAGECzQQh6VSy3KG4pN1d=h9J=D1rStFCMR+t7yh_Kwj-g87aLQ@mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/storage/file/fd.c
M src/backend/utils/misc/guc_tables.c
Expand comment for isset_offset.
commit : 7d559c8580f722dc40f2c34573e53e33626679e0
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 24 Mar 2025 15:47:02 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 24 Mar 2025 15:47:02 -0500
This field was added in commit 0164a0f9ee to provide a way to
determine whether a storage parameter was explicitly set for the
relation or if it just picked up the default value. In most cases,
this can be accomplished by giving the storage parameter a special
out-of-range default value (e.g., the
autovacuum_vacuum_insert_threshold storage parameter defaults to
-2), but this approach doesn't work in all cases. For example, a
Boolean storage parameter cannot be given an out-of-range default,
so we need another way to discover the source of its value.
Reported-by: "David G. Johnston" <david.g.johnston@gmail.com>
Reviewed-by: "David G. Johnston" <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/CAKFQuwYKtEUYKS%2B18gRs-xPhn0qOJgM2KGyyWVCODHuVn9F-XQ%40mail.gmail.com
M src/include/access/reloptions.h
Fix bitmapheapscan incorrect recheck of NULL tuples
commit : aea916fe555a351fe20bf31d56f5f0d027d9db61
author : Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 24 Mar 2025 16:40:59 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 24 Mar 2025 16:40:59 -0400
The bitmap heap scan skip fetch optimization skips fetching the heap
block when a page is set all-visible in the visibility map and no
columns from the table are needed to satisfy the query.
2b73a8cd33b and c3953226a07 changed the control flow of bitmap heap scan
to use the read stream API. The read stream API returns buffers
containing blocks to the user. To make this work with the skip fetch
optimization, we keep a count of the empty tuples we need to emit for
all the blocks skipped and only emit the empty tuples after processing
the next block fetched from the heap or at the end of the scan.
It's incorrect to recheck NULL tuples, so we must set `recheck` to false
before yielding control back to BitmapHeapNext(). This was done before
emitting any remaining empty tuples at the end of the scan but not for
empty tuples emitted during the scan. This meant that if a page fetched
from the heap did require recheck and set `recheck` to true and then we
emitted empty tuples for subsequent blocks, we would get wrong results.
Fix this by always setting `recheck` to false before emitting empty
tuples.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Tested-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/496f7acd-881c-4df3-9bd3-8f8534dfec26%40gmail.com
M src/backend/access/heap/heapam_handler.c
M src/test/regress/expected/bitmapops.out
M src/test/regress/sql/bitmapops.sql
Fix typo
commit : 0e3e0ec06b995f6809f315752cbf5ff67902e095
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 24 Mar 2025 17:36:44 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 24 Mar 2025 17:36:44 +0100
M src/test/authentication/t/001_password.pl
Allow pg_recvlogical --drop-slot to work without --dbname.
commit : c68100aa4313cf1cef826cf2d50f67164d13ebe4
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 25 Mar 2025 00:18:27 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 25 Mar 2025 00:18:27 +0900
When pg_recvlogical was introduced in 9.4, the --dbname option was not
required for --drop-slot. Without it, pg_recvlogical --drop-slot connected
using a replication connection (not tied to a specific database) and
was able to drop both physical and logical replication slots, similar to
pg_receivewal --drop-slot.
However, commit 0c013e08cfb unintentionally changed this behavior in 9.5,
making pg_recvlogical always check whether it's connected to a specific
database and fail if it's not. This change was expected for --create-slot
and --start, which handle logical replication slots and require a database
connection, but it was unnecessary for --drop-slot, which should work with
any replication connection. As a result, --dbname became a required option
for --drop-slot.
This commit removes that restriction, restoring the original behavior and
allowing pg_recvlogical --drop-slot to work without specifying --dbname.
Although this issue originated from an unintended change, it has existed
for a long time without complaints or bug reports, and the documentation
never explicitly stated that --drop-slot should work without --dbname.
Therefore, the change is not treated as a bug fix and is applied only to
master.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/b15ecf4f-e5af-4fbb-82c2-a425f453e0b2@oss.nttdata.com
M src/bin/pg_basebackup/pg_recvlogical.c
M src/bin/pg_basebackup/t/030_pg_recvlogical.pl
doc: Clarify required options for each action in pg_recvlogical.
commit : dfc13428a9088dd37ec8ed64de37fb1655355f9a
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 25 Mar 2025 00:14:38 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 25 Mar 2025 00:14:38 +0900
Each pg_recvlogical action requires specific options. For example,
--slot, --dbname, and --file must be specified with the --start action.
Previously, the documentation did not clearly outline these requirements.
This commit updates the documentation to explicitly state
the necessary options for each action.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Co-authored-by: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/OSCPR01MB14966930B4357BAE8C9D68A8AF5C72@OSCPR01MB14966.jpnprd01.prod.outlook.com
M doc/src/sgml/ref/pg_recvlogical.sgml
postgres_fdw: improve security checks
commit : 76563f88cfbd91696e7ebe568dead648f2d229ff
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 24 Mar 2025 14:09:51 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 24 Mar 2025 14:09:51 +0100
SCRAM pass-through should not bypass the FDW security check as it was
implemented for postgres_fdw in commit 761c79508e7.
This commit improves the security check by adding new SCRAM
pass-through checks to ensure that the required SCRAM connection
options are not overwritten by the user mapping or foreign server
options. This is meant to match the security requirements for a
password-using connection.
Since libpq has no SCRAM-specific equivalent of
PQconnectionUsedPassword(), we enforce this instead by making the
use_scram_passthrough option of postgres_fdw imply
require_auth=scram-sha-256. This means that if use_scram_passthrough
is set, some situations that might otherwise have worked are
preempted, for example GSSAPI with delegated credentials. This could
be enhanced in the future if there is desire for more flexibility.
Reported-by: Jacob Champion <jacob.champion@enterprisedb.com>
Author: Matheus Alcantara <mths.dev@pm.me>
Co-authored-by: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/CAFY6G8ercA1KES%3DE_0__R9QCTR805TTyYr1No8qF8ZxmMg8z2Q%40mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/t/001_auth_scram.pl
M doc/src/sgml/postgres-fdw.sgml
psql: use consistent alias for pg_description
commit : a8eeb22f171437f11ca2098f9b3094545263d8cd
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 24 Mar 2025 14:31:28 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 24 Mar 2025 14:31:28 +0100
Author:Jelte Fennema-Nio <github-tech@jeltef.nl>
Suggested-By: Michael Banck <mbanck@gmx.net>
Discussion: https://www.postgresql.org/message-id/67813520.170a0220.183245.7bf0%40mx.google.com
M src/bin/psql/describe.c
psql: show default extension version in \dx output
commit : d696406a9b255546bc1716d07199103abd8bb785
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 24 Mar 2025 14:25:05 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 24 Mar 2025 14:25:05 +0100
Reviewed-By: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-By: Michael Banck <mbanck@gmx.net>
Reviewed-By: Yugo Nagata <nagata@sraoss.co.jp>
Reviewed-By: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-By: Jelte Fennema-Nio <postgres@jeltef.nl>
Discussion: https://postgr.es/m/CABUevEyTMyXC6OvCWkj+rPnHrfi8_Rw_+DD_jzgFFNPqgf+Oig@mail.gmail.com
M src/bin/psql/describe.c
M src/test/regress/expected/psql.out
Add test case for when subscriber table is missing a column
commit : 19c6eb06c51f4da70e2ea0f1bdb64a0142e8e2aa
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 24 Mar 2025 12:13:32 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 24 Mar 2025 12:13:32 +0200
We haven't had bugs in this area, but there's some not-entirely
trivial code to detect that case, so it seems good to have test
coverage for it.
Author: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://www.postgresql.org/message-id/CAHut%2BPtX8P0EGhsk9p%3DhQGUHrzxeCSzANXSMKOvYiLX-EjdyNw@mail.gmail.com
M src/test/subscription/t/008_diff_schema.pl
Detect and Log multiple_unique_conflicts type conflict.
commit : 73eba5004a06a744b6b8570e42432b9e9f75997b
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 24 Mar 2025 12:30:44 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 24 Mar 2025 12:30:44 +0530
Introduce a new conflict type, multiple_unique_conflicts, to handle cases
where an incoming row during logical replication violates multiple UNIQUE
constraints.
Previously, the apply worker detected and reported only the first
encountered key conflict (insert_exists/update_exists), causing repeated
failures as each constraint violation needs to be handled one by one
making the process slow and error-prone.
With this patch, the apply worker checks all unique constraints upfront
once the first key conflict is detected and reports
multiple_unique_conflicts if multiple violations exist. This allows users
to resolve all conflicts at once by deleting all conflicting tuples rather
than dealing with them individually or skipping the transaction.
In the future, this will also allow us to specify different resolution
handlers for such a conflict type.
Add the stats for this conflict type in pg_stat_subscription_stats.
Author: Nisha Moond <nisha.moond412@gmail.com>
Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Discussion: https://postgr.es/m/CABdArM7FW-_dnthGkg2s0fy1HhUB8C3ELA0gZX1kkbs1ZZoV3Q@mail.gmail.com
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/executor/execReplication.c
M src/backend/replication/logical/conflict.c
M src/backend/replication/logical/worker.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/replication/conflict.h
M src/test/regress/expected/rules.out
M src/test/subscription/meson.build
A src/test/subscription/t/035_conflicts.pl
M src/tools/pgindent/typedefs.list
Add tests for POSITION(bytea, bytea)
commit : 35a92b7c2520cca3df9ecddab1dcad14ff71ec0b
author : David Rowley <drowley@postgresql.org>
date : Mon, 24 Mar 2025 19:32:02 +1300
committer: David Rowley <drowley@postgresql.org>
date : Mon, 24 Mar 2025 19:32:02 +1300
Previously there was no coverage for this function.
Author: Aleksander Alekseev <aleksander@timescale.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Rustam ALLAKOV <rustamallakov@gmail.com>
Discussion: https://postgr.es/m/CAJ7c6TMT6XCooMVKnCd_tR2oBdGcnjefSeCDCv8jzKy9VkWA5w@mail.gmail.com
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
Allow plugins to set a 64-bit plan identifier in PlannedStmt
commit : 2a0cd38da5ccf70461c51a489ee7d25fcd3f26be
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 24 Mar 2025 13:23:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 24 Mar 2025 13:23:42 +0900
This field can be optionally set in a PlannedStmt through the planner
hook, giving extensions the possibility to assign an identifier related
to a computed plan. The backend is changed to report it in the backend
entry of a process running (including the extended query protocol), with
semantics and APIs to set or get it similar to what is used for the
existing query ID (introduced in the backend via 4f0b0966c8). The plan
ID is reset at the same timing as the query ID. Currently, this
information is not added to the system view pg_stat_activity; extensions
can access it through PgBackendStatus.
Some patches have been proposed to provide some features in the planning
area, where a plan identifier is used as a key to know the plan involved
(for statistics, plan storage and manipulations, etc.), and the point of
this commit is to provide an anchor in the backend that extensions can
rely on for future work. The reset of the plan identifier is
controlled by core and follows the same pattern as the query identifier
added in 4f0b0966c8.
The contents of this commit are extracted from a larger set proposed
originally by Lukas Fittl, that Sami Imseih has proposed as an
independent change, with a few tweaks sprinkled by me.
Author: Lukas Fittl <lukas@fittl.com>
Author: Sami Imseih <samimseih@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAP53Pkyow59ajFMHGpmb1BK9WHDypaWtUsS_5DoYUEfsa_Hktg@mail.gmail.com
Discussion: https://postgr.es/m/CAA5RZ0vyWd4r35uUBUmhngv8XqeiJUkJDDKkLf5LCoWxv-t_pw@mail.gmail.com
M src/backend/executor/execParallel.c
M src/backend/optimizer/plan/planner.c
M src/backend/tcop/postgres.c
M src/backend/utils/activity/backend_status.c
M src/include/nodes/plannodes.h
M src/include/utils/backend_status.h
psql: Add tab completion for VACUUM and ANALYZE ... ONLY option.
commit : 8a3e4011f02dd2789717c633e74fefdd3b648386
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 Mar 2025 17:16:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 Mar 2025 17:16:08 -0400
Improve psql's tab completion for VACUUM and ANALYZE by supporting
the ONLY option introduced in 62ddf7ee9.
In passing, simplify some of the VACUUM patterns by making use
of MatchAnyN.
Author: Umar Hayat <postgresql.wizard@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Discussion: https://postgr.es/m/CAD68Dp3L6yW_nWs+MWBs6s8tKLRzXaQdQgVRm4byZe0L-hRD8g@mail.gmail.com
M src/bin/psql/tab-complete.in.c
Fix rare assertion failure in standby, if primary is restarted
commit : 2817525f0d56075e1f3a14c0dc6a180b337d8aed
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sun, 23 Mar 2025 20:41:16 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sun, 23 Mar 2025 20:41:16 +0200
During hot standby, ExpireAllKnownAssignedTransactionIds() and
ExpireOldKnownAssignedTransactionIds() functions mark old transactions
as no-longer running, but they failed to update xactCompletionCount
and latestCompletedXid. AFAICS it would not lead to incorrect query
results, because those functions effectively turn in-progress
transactions into aborted transactions and an MVCC snapshot considers
both as "not visible". But it could surprise GetSnapshotDataReuse()
and trigger the "TransactionIdPrecedesOrEquals(TransactionXmin,
RecentXmin))" assertion in it, if the apparent xmin in a backend would
move backwards. We saw this happen when GetCatalogSnapshot() would
reuse an older catalog snapshot, when GetTransactionSnapshot() had
already advanced TransactionXmin.
The bug goes back all the way to commit 623a9ba79b in v14 that
introduced the snapshot reuse mechanism, but it started to happen more
frequently with commit 952365cded6 which removed a
GetTransactionSnapshot() call from backend startup. That made it more
likely for ExpireOldKnownAssignedTransactionIds() to be called between
GetCatalogSnapshot() and the first GetTransactionSnapshot() in a
backend.
Andres Freund first spotted this assertion failure on buildfarm member
'skink'. Reproduction and analysis by Tomas Vondra.
Backpatch-through: 14
Discussion: https://www.postgresql.org/message-id/oey246mcw43cy4qw2hqjmurbd62lfdpcuxyqiu7botx3typpax%40h7o7mfg5zmdj
M src/backend/storage/ipc/procarray.c
Fix "make clean" for new TAP suite.
commit : f0446384ea7c4274894d7f5b215bfc2496ace85d
author : Noah Misch <noah@leadboat.com>
date : Sun, 23 Mar 2025 06:12:02 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 23 Mar 2025 06:12:02 -0700
Commit 28f04984f0c240b76e61f00cd247554fbc850056 missed this.
M src/interfaces/ecpg/preproc/Makefile
aio: Change prefix of PgAioResultStatus values to PGAIO_RS_
commit : ca3067cc573d7c1ed617cfb959ae8507be17f1aa
author : Andres Freund <andres@anarazel.de>
date : Sat, 22 Mar 2025 17:30:44 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sat, 22 Mar 2025 17:30:44 -0400
The previous prefix wasn't consistent with the naming of other AIO related
enum values. It seems best to rename it before the users are introduced.
Reported-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_Yb+JzQpNsgUxCB0gBi+sE-mi_HmcJF6ALnmO4W+UgwpA@mail.gmail.com
M src/backend/storage/aio/aio.c
M src/backend/storage/aio/aio_callback.c
M src/backend/storage/aio/aio_init.c
M src/include/storage/aio_types.h
plpgsql: make WHEN OTHERS distinct from WHEN SQLSTATE '00000'.
commit : 58fdca2204de5f683f025df37553e5e69cb6adb1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 Mar 2025 14:17:00 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 Mar 2025 14:17:00 -0400
The catchall exception condition OTHERS was represented as
sqlerrstate == 0, which was a poor choice because that comes
out the same as SQLSTATE '00000'. While we don't issue that
as an error code ourselves, there isn't anything particularly
stopping users from doing so. Use -1 instead, which can't
match any allowed SQLSTATE string.
While at it, invent a macro PLPGSQL_OTHERS to use instead of
a hard-coded magic number.
While this seems like a bug fix, I'm inclined not to back-patch.
It seems barely possible that someone has written code like this
and would be annoyed by changing the behavior in a minor release.
Reported-by: David Fiedler <david.fido.fiedler@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAHjN70-=H5EpTOuZVbC8mPvRS5EfZ4MY2=OUdVDWoyGvKhb+Rw@mail.gmail.com
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/plpgsql.h
Improve nbtree array primitive scan scheduling.
commit : 9a2e2a285a149490a69a7bd92dd618bb7ca975b3
author : Peter Geoghegan <pg@bowt.ie>
date : Sat, 22 Mar 2025 13:02:18 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Sat, 22 Mar 2025 13:02:18 -0400
Add a new scheduling heuristic: don't end the ongoing primitive index
scan immediately (at the point where _bt_advance_array_keys notices that
the next set of matching tuples must be on a later page) if the primscan
already managed to step right/left from its first leaf page. Schedule a
recheck against the next sibling leaf page's finaltup instead.
The new heuristic tends to avoid scenarios where the top-level scan
repeatedly starts and ends primitive index scans that each read only one
leaf page from a group of neighboring leaf pages. Affected top-level
scans will now tend to step forward (or backward) through the index
instead, without wasting cycles on descending the index anew.
The recheck mechanism isn't exactly new. But up until now it has only
been used to deal with edge cases involving high key finaltups with one
or more truncated -inf attributes that _bt_advance_array_keys deemed
"provisionally satisfied" (satisfied for the purposes of allowing the
scan to step onto the next page, subject to recheck once on that page).
The mechanism was added by commit 5bf748b8, which invented the general
concept of primitive scan scheduling. It was later enhanced by commit
79fa7b3b, which taught it about cases involving -inf attributes that
satisfy inequality scan keys required in the opposite-to-scan direction
only (arguably, they should have been covered by the earliest version).
Now the recheck mechanism can be applied based on scan-level heuristics,
which have nothing to do with truncated high keys. Now rechecks might
be performed by _bt_readpage when scanning in _either_ scan direction.
The theory behind the new heuristic is that any primitive scan that
makes it past its first leaf page is one that is already likely to have
arrays whose key values match index tuples that are closely clustered
together in the index. The rules that determine whether we ever get
past the first page are still conservative (that'll still only happen
when pstate.finaltup strongly suggests that it's the right thing to do).
Surviving past the first leaf page is a strong signal in itself.
Preparation for an upcoming patch that will add skip scan optimizations
to nbtree. That'll work by adding skip arrays, which behave similarly
to SAOP arrays, but generate their elements procedurally and on-demand.
Note that this commit isn't specifically concerned with skip arrays; the
scheduling logic doesn't (and won't) condition anything on whether the
scan uses skip arrays, SAOP arrays, or some combination of the two
(which seems like a good general principle for _bt_advance_array_keys).
While the problems that this commit ameliorates are more likely with
skip arrays (at least in practice), SAOP arrays (or those with very
dense, contiguous array elements) are also affected.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wzkz0wPe6+02kr+hC+JJNKfGtjGTzpG3CFVTQmKwWNrXNw@mail.gmail.com
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtutils.c
M src/include/access/nbtree.h
Use streaming read I/O in SP-GiST vacuuming
commit : e215166c9c810950cff101cc098e66c8758538fa
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 21 Mar 2025 17:49:20 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 21 Mar 2025 17:49:20 -0400
Like 69273b818b1df did for GiST vacuuming, make SP-GiST vacuum use the
read stream API for vacuuming physically contiguous index pages.
Concurrent insertions may cause SP-GiST index tuples to be redirected.
While vacuuming, these are added to a pending list which is later
processed to ensure no dead tuples are left behind. Pages containing
such tuples are still read by directly calling ReadBuffer() and do not
use the read stream API.
Author: Andrey M. Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/37432403-8657-403B-9CDF-5A642BECDD81%40yandex-team.ru
M src/backend/access/spgist/spgvacuum.c
Fix ps display for IO workers.
commit : e51ca405ed823b28358f216642b5a80f1f886f5d
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 22 Mar 2025 09:57:45 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 22 Mar 2025 09:57:45 +1300
This code must have missed a memo about the backend type description
being supplied automatically these days, and was duplicating that
information.
Before: "io worker io worker: N"
After: "io worker N"
M src/backend/storage/aio/method_worker.c
Revert inappropriate weakening of an Assert in plpgsql.
commit : 16a3ae504e2030583851c8f45cbed14b0c546d6e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Mar 2025 15:55:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Mar 2025 15:55:06 -0400
Commit 682ce911f modified exec_save_simple_expr to accept a Param
in the tlist of a Gather node, rather than the normal case of a Var
referencing the Gather's input. It turns out that this was a kluge
to work around the bug later fixed in 0f7ec8d9c, namely that setrefs.c
was failing to replace Params in upper plan nodes with Var references
to the same Params appearing in the child tlists. With that fixed,
there seems no reason to continue to allow a Param here. (Moreover,
even if we did expect a Param here, the semantically correct thing
to do would be to take the Param as the expression being sought.
Whatever it may represent, it is *not* a reference to the child.)
Hence, revert that part of 682ce911f.
That all happened a long time ago. However, since the net effect
here is just to tighten an Assert condition, I'm content to change
it only in master.
Discussion: https://postgr.es/m/1565347.1742572349@sss.pgh.pa.us
M src/pl/plpgsql/src/pl_exec.c
Add GUC option to control maximum active replication origins.
commit : 04ff636cbce4b91fba1f334e1bc0dc88686e7b2d
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Mar 2025 12:20:15 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Mar 2025 12:20:15 -0700
This commit introduces a new GUC option max_active_replication_origins
to control the maximum number of active replication
origins. Previously, this was controlled by
'max_replication_slots'. Having a separate GUC option provides better
flexibility for setting up subscribers, as they may not require
replication slots (for cascading replication) but always require
replication origins.
Author: Euler Taveira <euler@eulerto.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/b81db436-8262-4575-b7c4-bc0c1551000b@app.fastmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/ref/pg_createsubscriber.sgml
M src/backend/replication/logical/launcher.c
M src/backend/replication/logical/origin.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/pg_basebackup/pg_createsubscriber.c
M src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/t/004_subscription.pl
M src/include/replication/origin.h
Place "extern" declaration in the right part of pg_class.h.
commit : 0e032a2240a98669546023f7707030ea3ce4ded7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Mar 2025 15:14:15 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Mar 2025 15:14:15 -0400
errdetail_relkind_not_supported() was declared within
EXPOSE_TO_CLIENT_CODE, which is mistaken since that function
isn't available client-side. While relatively harmless,
this isn't good precedent.
Discussion: https://postgr.es/m/1134562.1742507765@sss.pgh.pa.us
M src/include/catalog/pg_class.h
Label the contents of pg_*_d.h files a little better.
commit : cd72c1b76e36525fb6b23418d232df661634afd5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Mar 2025 15:09:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Mar 2025 15:09:46 -0400
Make genbki.pl emit some boilerplate comments identifying the
sections of the pg_*_d.h files that it generates. This is in
hopes of making them slightly more readable, in case people
look at those files and not the pg_*.h/pg_*.dat originals.
Discussion: https://postgr.es/m/1134562.1742507765@sss.pgh.pa.us
M src/backend/catalog/genbki.pl
Use streaming read I/O in GiST vacuuming
commit : 69273b818b1df82c36b2b2acb592db3d0743cc7c
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 21 Mar 2025 14:05:36 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 21 Mar 2025 14:05:36 -0400
Like c5c239e26e387 did for btree vacuuming, make GiST vacuum use the
read stream API for sequentially processed pages.
Because it is possible for concurrent insertions to relocate unprocessed
index entries to already vacuumed pages, GiST vacuum must backtrack and
reprocess those pages. These pages are still read with explicit
ReadBuffer() calls.
Author: Andrey M. Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/EFEBED92-18D1-4C0F-A4EB-CD47072EF071%40yandex-team.ru
M src/backend/access/gist/gistvacuum.c
Assorted trivial cleanup of c5c239e26e
commit : 3f850c3fc5442084d13122be7f54335e4d017bef
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 21 Mar 2025 14:05:21 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 21 Mar 2025 14:05:21 -0400
c5c239e26e made btree vacuum use the read stream API. Though it used
functions declared in read_stream.h, it relied on transitively including
it. Explicitly include that file. Also remove an extraneous newline and
decrease the scope of one of the local variables in btvacuumscan().
M src/backend/access/nbtree/nbtree.c
Fix plpgsql's handling of simple expressions in scrollable cursors.
commit : 7fe312f6092f47724a29e30fb2992bd2bc287687
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Mar 2025 11:30:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Mar 2025 11:30:42 -0400
exec_save_simple_expr did not account for the possibility that
standard_planner would stick a Materialize node atop the plan
of even a simple Result, if CURSOR_OPT_SCROLL is set. This led
to an "unexpected plan node type" error.
This is a very old bug, but it'd only be reached by declaring a
cursor for a "SELECT simple-expression" query and explicitly
marking it scrollable, which is an odd thing to do. So the lack
of prior reports isn't too surprising.
Bug: #18859
Reported-by: Olleg Samoylov <splarv@ya.ru>
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18859-0d5f28ac99a37059@postgresql.org
Backpatch-through: 13
M src/pl/plpgsql/src/expected/plpgsql_simple.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_simple.sql
Use streaming read I/O in btree vacuuming
commit : c5c239e26e3876c05b3a2c5e8989817fd38ceed1
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 21 Mar 2025 09:07:33 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 21 Mar 2025 09:07:33 -0400
Btree vacuum processes all index pages in physical order. Now it uses
the read stream API to get the next buffer instead of explicitly
invoking ReadBuffer().
It is possible for concurrent insertions to cause page splits during
index vacuuming. This can lead to index entries that have yet to be
vacuumed being moved to pages that have already been vacuumed. Btree
vacuum code handles this by backtracking to reprocess those pages. So,
while sequentially encountered pages are now read through the
read stream API, backtracked pages are still read with explicit
ReadBuffer() calls.
Author: Andrey Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Junwang Zhao <zhjwpku@gmail.com>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Discussion: https://postgr.es/m/flat/CAAKRu_bW1UOyup%3DjdFw%2BkOF9bCaAm%3D9UpiyZtbPMn8n_vnP%2Big%40mail.gmail.com#3b3a84132fc683b3ee5b40bc4c2ea2a5
M src/backend/access/nbtree/nbtree.c
Change one loop in ATRewriteTable to use 1-based attnums
commit : 1d617a20284f887cb9cdfe5693eec155e8016517
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 21 Mar 2025 10:55:06 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 21 Mar 2025 10:55:06 +0100
All TupleDescAttr() calls in tablecmds.c that aren't in loops across all
attributes use AttrNumber-style indexes (1-based); there was only one
place in ATRewriteTable that was stashing 0-based indexes in a list for
later processing. Switch that to use attnums for consistency.
Author: jian he <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CACJufxEoYA5ScUr2=CmA1xcpaS_1ixneDbEkVU77X1ctGxY2mA@mail.gmail.com
M src/backend/commands/tablecmds.c
Support buffer forwarding in StartReadBuffers().
commit : ce1a75c4feada55e7435edc5aac4b85afec734e6
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 21 Mar 2025 19:04:58 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 21 Mar 2025 19:04:58 +1300
StartReadBuffers() reports a short read when it finds a cached block
that ends a range needing I/O by updating the caller's *nblocks. It
doesn't want to have to unpin the trailing hit that it knows the caller
wants, so the v17 version used sleight of hand in the name of
simplicity: it included it in *nblocks as if it were part of the I/O,
but internally tracked the shorter real I/O size in io_buffers_len (now
removed).
This API change "forwards" the delimiting buffer to the next call. It's
still pinned, and still stored in the caller's array, but *nblocks no
longer includes stray buffers that are not really part of the operation.
The expectation is that the caller still wants the rest of the blocks
and will call again starting from that point, and now it can pass the
already pinned buffer back in (or choose not to and release it).
The change is needed for the coming asynchronous I/O version's larger
version of the problem: by definition it must move BM_IO_IN_PROGRESS
negotiation from WaitReadBuffers() to StartReadBuffers(), but it might
already have many buffers pinned before it discovers a need to split an
I/O. (The current synchronous I/O version hides that detail from
callers by looping over smaller reads if required to make all covered
buffers valid in WaitReadBuffers(), so it looks like one operation but
it might occasionally be several under the covers.)
Aside from avoiding unnecessary pin traffic, this will also be important
for later work on out-of-order streams: you can't prioritize data that
is already available right now if that fact is hidden from you.
The new API is natural for read_stream.c (see ed0b87ca). After a short
read it leaves forwarded buffers where they fell in its circular queue
for the continuing call to pick up.
Single-block StartReadBuffer() and traditional ReadBuffer() share code
but are not affected by the change. They don't do multi-block I/O.
Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions)
Discussion: https://postgr.es/m/CA%2BhUKGK_%3D4CVmMHvsHjOVrK6t4F%3DLBpFzsrr3R%2BaJYN8kcTfWg%40mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
M src/include/storage/bufmgr.h
Support buffer forwarding in read_stream.c.
commit : ed0b87caaca1eb3daa0f76f32499da7628286919
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 21 Mar 2025 17:34:22 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 21 Mar 2025 17:34:22 +1300
In preparation for a follow-up change to the buffer manager, teach
read_stream.c to manage buffers "forwarded" from one StartReadBuffers()
call to the next after a short read. This involves a small amount of
extra book-keeping, and opens the way for lower levels to split I/O
operations without having to drop pins, as required for efficient
handling of various edge cases.
Concretely, the "buffers" argument will change from an out parameter to
an in/out parameter. Buffer queue elements must be initialized on first
use and cleared after they're consumed, but forwarded buffers are left
where they fall ahead of the current pending read in the queue, ready
for use by the operation that continues where a short read left off.
The stream also needs to count them for pin limit management and release
them on reset/early end.
Tested-by: Andres Freund <andres@anarazel.de> (earlier versions)
Discussion: https://postgr.es/m/CA%2BhUKGK_%3D4CVmMHvsHjOVrK6t4F%3DLBpFzsrr3R%2BaJYN8kcTfWg%40mail.gmail.com
M src/backend/storage/aio/read_stream.c
doc: Remove incorrect description about dropping replication slots.
commit : 14413d0ef5df4df4cad36cfe657e3f9d1f94bc2f
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 21 Mar 2025 12:56:39 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 21 Mar 2025 12:56:39 +0900
pg_drop_replication_slot() can drop replication slots created on
a different database than the one where it is executed. This behavior
has been in place since PostgreSQL 9.4, when pg_drop_replication_slot()
was introduced.
However, commit ff539d mistakenly added the following incorrect
description in the documentation:
For logical slots, this must be called when connected to
the same database the slot was created on.
This commit removes that incorrect statement. A similar mistake was
also present in the documentation for the DROP_REPLICATION_SLOT
command, which has now been corrected as well.
Back-patch to all supported versions.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/OSCPR01MB14966C6BE304B5BB2E58D4009F5DE2@OSCPR01MB14966.jpnprd01.prod.outlook.com
Backpatch-through: 13
M doc/src/sgml/func.sgml
M doc/src/sgml/protocol.sgml
Simplify EXPLAIN code for Memoize
commit : 00b52c3db66c5ec2593463c43371525deff95347
author : David Rowley <drowley@postgresql.org>
date : Fri, 21 Mar 2025 13:40:05 +1300
committer: David Rowley <drowley@postgresql.org>
date : Fri, 21 Mar 2025 13:40:05 +1300
This removes a needless special case for Memoize's FORMAT TEXT EXPLAIN
output.
ExplainPropertyText() outputs the same thing in text mode as the
special-case code was doing, so removing the special-case code results in
the same EXPLAIN output, just with less code.
It seems like a good idea to fix this to help prevent future changes in
this area from copying the same pattern.
Author: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Reported-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/88a71bcd-0b5c-4d0b-8107-757e96f402d5@tantorlabs.com
M src/backend/commands/explain.c
bufmgr: Improve stats when a buffer is read in concurrently
commit : 202b12774d092baf7e5d3309aa3cb0cd1e8a606b
author : Andres Freund <andres@anarazel.de>
date : Thu, 20 Mar 2025 19:58:22 -0400
committer: Andres Freund <andres@anarazel.de>
date : Thu, 20 Mar 2025 19:58:22 -0400
Previously we would have the following inaccuracies when a backend tried to
read in a buffer, but that buffer was read in concurrently by another backend:
- the read IO was double-counted in the global buffer access stats (pgBufferUsage)
- the buffer hit was not accounted for in:
- global buffer access statistics
- pg_stat_io
- relation level IO stats
- vacuum cost balancing
While trying to read in a buffer that is concurrently read in by another
backend is not a common occurrence, it's also not that rare, e.g. due to
concurrent sequential scans on the same relation. This scenario has become
more likely in PG 17, due to the introducing of read streams, which can pin
multiple buffers before calling StartBufferIO() for all the buffers.
This behaviour has historically grown, but there doesn't seem to be any reason
to continue with the wrong accounting.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_Zk-B08AzPsO-6680LUHLOCGaNJYofaxTFseLa=OepV1g@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
Show plperl version in the meson setup summary.
commit : 12604593e9f3b3460f7359a39d25731aff6beb88
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 20 Mar 2025 18:55:29 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 20 Mar 2025 18:55:29 -0400
Also, use perl 'version' instead of 'api_versionstring' to sync with
the configure script.
Author: Roman Zharkov <r.zharkov@postgrespro.ru>
Discussion: https://postgr.es/m/93e7f77bf4e1ef4640e4ee733f9e2a78@postgrespro.ru
M meson.build
smgr: Hold interrupts in most smgr functions
commit : fc51a60dd45bb39d21d09a890da2f21ac8e61532
author : Andres Freund <andres@anarazel.de>
date : Thu, 20 Mar 2025 17:32:59 -0400
committer: Andres Freund <andres@anarazel.de>
date : Thu, 20 Mar 2025 17:32:59 -0400
We need to hold interrupts across most of the smgr.c/md.c functions, as
otherwise interrupt processing, e.g. due to a < ERROR elog/ereport, can
trigger procsignal processing, which in turn can trigger smgrreleaseall(). As
the relevant code is not reentrant, we quickly end up in a bad situation.
The only reason we haven't noticed this before is that there is only one
non-error ereport called in affected routines, in register_dirty_segments(),
and that one is extremely rarely reached. If one enables fd.c's FDDEBUG it's
easy to reproduce crashes.
It seems better to put the HOLD_INTERRUPTS()/RESUME_INTERRUPTS() in smgr.c,
instead of trying to push them down to md.c where possible: For one, every
smgr implementation would be vulnerable, for another, a good bit of smgr.c
code itself is affected too.
Eventually we might want a more targeted solution, allowing e.g. a networked
smgr implementation to be interrupted, but many other, more complicated,
problems would need to be fixed for that to be viable (e.g. smgr.c is often
called with interrupts already held).
One could argue this should be backpatched, but the existing < ERROR
elog/ereports that can be reached with unmodified sources are unlikely to be
reached. On balance the risk of backpatching seems higher than the gain - at
least for now.
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/3vae7l5ozvqtxmd7rr7zaeq3qkuipz365u3rtim5t5wdkr6f4g@vkgf2fogjirl
M src/backend/storage/smgr/smgr.c
Be more paranoid in configure's checks for CRC and POPCNT intrinsics.
commit : fdb5dd6331e305f797bb589747f056062c305f0b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 20 Mar 2025 16:23:09 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 20 Mar 2025 16:23:09 -0400
In these tests, we need to verify not only that the compiler has heard
of these intrinsics, but that lower-level tools cope with them too.
(For example, the assembler must also know the instructions, and on
some platforms there might be library support involved.) The hazard
is that the compiler might optimize away the calls altogether,
allowing the configure check to succeed only to have the build fail
later if lower-level support is missing. The existing code tried to
prevent that by ensuring that the result of the intrinsic is used
for something, but that's really insufficient because we were feeding
constant input to it. So the compiler would be perfectly entitled to
optimize away the calls anyway. Fix by making the inputs into global
variables. (Hypothetically, LTO optimization could still remove the
code --- but that's well past where we'd be likely to hit trouble.)
It is not known that any current compiler would actually optimize
away these calls, and even if that happened it would be unlikely
that any problem would manifest. Our concern for this stems from
largely-bygone days when it was common to install gcc on platforms
with some other native compiler, so that a compiler-vs-library
support discrepancy was more probable. Still, there's little
point in defending against such cases in a way that is visibly
incomplete.
I'm content to fix this in master for now; we can back-patch if
any indication appears that it's a live problem for someone.
Discussion: https://postgr.es/m/3368102.1741993462@sss.pgh.pa.us
M config/c-compiler.m4
M configure
M meson.build
Add an additional hook for EXPLAIN option validation.
commit : 50ba65e73325cf55fedb3e1f14673d816726923b
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 20 Mar 2025 13:47:55 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 20 Mar 2025 13:47:55 -0400
Commit c65bc2e1d14a2d4daed7c1921ac518f2c5ac3d17 made it possible for
loadable modules to add EXPLAIN options. Normally, any necessary
validation can be performed by the hook function passed to
RegisterExtensionExplainOption, but if a loadable module wants to sanity
check options against each other, that needs to be done after the entire
options list has been processed. So, add an additional hook for that
purpose.
Author: Sami Imseih <samimseih@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: http://postgr.es/m/CAA5RZ0vOcJF91O2e5AQN+V6guMNLMhJx83dxALf-iUZ-hLGO_Q@mail.gmail.com
M src/backend/commands/explain_state.c
M src/include/commands/explain_state.h
Add test for pg_upgrade file transfer modes.
commit : af0d4901c1c39b3e3425522834708e0992592bb8
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 20 Mar 2025 11:08:42 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 20 Mar 2025 11:08:42 -0500
This new test checks all of pg_upgrade's file transfer modes. For
each mode, we verify that pg_upgrade either succeeds (and some test
objects successfully reach the new version) or fails with an error
that indicates the mode is not supported on the current platform.
For cross-version tests, we also check that pg_upgrade transfers
non-default tablespaces. (Tablespaces can't be tested on same
version upgrades because of the version-specific subdirectory
conflict, but we might be able to enable such tests once we teach
pg_upgrade how to handle in-place tablespaces.)
Suggested-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/Zyvop-LxLXBLrZil%40nathan
M src/bin/pg_upgrade/meson.build
A src/bin/pg_upgrade/t/006_transfer_modes.pl
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/test/perl/PostgreSQL/Test/Utils.pm
Add vacuum_truncate configuration parameter.
commit : 0164a0f9ee12e0eff9e4c661358a272ecd65c2d4
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 20 Mar 2025 10:16:50 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 20 Mar 2025 10:16:50 -0500
This new parameter works just like the storage parameter of the
same name: if set to true (which is the default), autovacuum and
VACUUM attempt to truncate any empty pages at the end of the table.
It is primarily intended to help users avoid locking issues on hot
standbys. The setting can be overridden with the storage parameter
or VACUUM's TRUNCATE option.
Since there's presently no way to determine whether a Boolean
storage parameter is explicitly set or has just picked up the
default value, this commit also introduces an isset_offset member
to relopt_parse_elt.
Suggested-by: Will Storey <will@summercat.com>
Author: Nathan Bossart <nathandbossart@gmail.com>
Co-authored-by: Gurjeet Singh <gurjeet@singh.im>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Robert Treat <rob@xzilla.net>
Discussion: https://postgr.es/m/Z2DE4lDX4tHqNGZt%40dev.null
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/vacuum.sgml
M src/backend/access/common/reloptions.c
M src/backend/commands/vacuum.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/access/reloptions.h
M src/include/commands/vacuum.h
M src/include/utils/guc_tables.h
M src/include/utils/rel.h
M src/test/regress/expected/vacuum.out
M src/test/regress/sql/vacuum.sql
Revert workarounds for -Wmissing-braces false positives on old GCC
commit : 618c64ffd3967cb5313b4b11e1e1043a074f2139
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 20 Mar 2025 11:25:58 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 20 Mar 2025 11:25:58 +0100
We have collected several instances of a workaround for GCC bug 53119,
which caused false-positive compiler warnings. This bug has long been
fixed, but was still seen on the buildfarm, most recently on lapwing
with gcc (Debian 4.7.2-5). (The GCC bug tracker mentions that a fix
was backported to 4.7.4 and 4.8.3.)
That compiler no longer runs warning-free since commit 6fdd5d95634, so
we don't need to keep these workarounds. And furthermore, the
consensus appears to be that we don't want to keep supporting that era
of platform anymore at all.
This reverts the following commits:
d937904cce6a3d82e4f9c2127de7b59105a134b3
506428d091760650971433f6bc083531c307b368
b449afb582bb9015bfbb85abc10ce122aef9ec70
6392f2a0968c20ecde4d27b6652703ad931fce92
bad0763a4d7be3005eae35d460c73ac4bc7ebaad
5e0c761d0a13c7b4f7c5de618ac38560d74d74d0
and makes a few similar fixes to newer code.
Discussion: https://www.postgresql.org/message-id/flat/e170d61f-01ab-4cf9-ab68-91cd1fac62c5%40eisentraut.org
Discussion: https://www.postgresql.org/message-id/flat/CA%2BTgmoYEAm-KKZibAP3hSqbTFTjUd47XtVcf3xSFDpyecXX9uQ%40mail.gmail.com
M contrib/pg_prewarm/autoprewarm.c
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/optimizer/path/costsize.c
M src/backend/storage/smgr/bulk_write.c
M src/bin/pg_waldump/rmgrdesc.c
M src/bin/pgbench/pgbench.c
M src/common/blkreftable.c
M src/common/file_utils.c
M src/interfaces/libpq/fe-auth-oauth-curl.c
Fix extension control path tests
commit : b7076c1e7f43f61515fa9a8c4f75e4329aef69ce
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 20 Mar 2025 10:53:59 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 20 Mar 2025 10:53:59 +0100
Change expected extension to be installed from amcheck to plpgsql since
not all build farm animals has the contrib module installed.
Author: Matheus Alcantara <mths.dev@pm.me>
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/E7C7BFFB-8857-48D4-A71F-88B359FADCFD@justatheory.com
M src/test/modules/test_extensions/t/001_extension_control_path.pl
Fix typo in comment
commit : 47929324c522cecfa8e0f64c6833f1dc3a67a735
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 20 Mar 2025 10:44:12 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 20 Mar 2025 10:44:12 +0100
M src/backend/storage/file/buffile.c
pg_createsubscriber: Add -R publications option.
commit : e5aeed4b8020424fa882c3d1969d00c2b2d8cf30
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 20 Mar 2025 12:21:54 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 20 Mar 2025 12:21:54 +0530
This patch introduces a new '-R'/'--remove' option in the
'pg_createsubscriber' utility to specify the object types to be removed
from the subscriber. Currently, we add support to specify 'publications'
as an object type. In the future, other object types like failover-slots
could be added.
This feature allows optionally to remove publications on the subscriber
that were replicated from the primary server (before running this tool)
during physical replication. Users may want to retain these publications
in case they want some pre-existing subscribers to point to the newly
created subscriber.
Author: Shubham Khanna <khannashubham1197@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Reviewed-by: Euler Taveira <euler@eulerto.com>
Reviewed-by: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Nisha Moond <nisha.moond412@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CAHv8RjL4OvoYafofTb_U_JD5HuyoNowBoGpMfnEbhDSENA74Kg@mail.gmail.com
M doc/src/sgml/ref/pg_createsubscriber.sgml
M src/bin/pg_basebackup/pg_createsubscriber.c
M src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
meson: Flush stdout in testwrap
commit : 5941946d0934b9eccb0d5bfebd40b155249a0130
author : Andres Freund <andres@anarazel.de>
date : Wed, 19 Mar 2025 09:04:09 -0400
committer: Andres Freund <andres@anarazel.de>
date : Wed, 19 Mar 2025 09:04:09 -0400
Otherwise the progress won't reliably be displayed during a test.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/kx6xu7suexal5vwsxpy7ybgkcznx6hgywbuhkr6qabcwxjqax2@i4pcpk75jvaa
Backpatch-through: 16
M src/tools/testwrap
Update a code comment
commit : 190dc27998d5b7b4c36e12bebe62f7176f4b4507
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 19 Mar 2025 10:39:06 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 19 Mar 2025 10:39:06 +0100
The comment explained that ALTER TABLE ADD CONSTRAINT USING INDEX is
only supported with a btree index. (This is not being changed.) The
reason is to keep upgrades robust, as explained there. The other part
of the comment, that btree is the only unique index kind anyway, is
somewhat less true as we're trying to enable unique indexes other than
btree, and it's irrelevant to this check. There is a check for
indisunique earlier already. So just remove this part of the comment.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/parser/parse_utilcmd.c
extension_control_path
commit : 4f7f7b0375854e2f89876473405a8f21c95012af
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 19 Mar 2025 06:57:20 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 19 Mar 2025 06:57:20 +0100
The new GUC extension_control_path specifies a path to look for
extension control files. The default value is $system, which looks in
the compiled-in location, as before.
The path search uses the same code and works in the same way as
dynamic_library_path.
Some use cases of this are: (1) testing extensions during package
builds, (2) installing extensions outside security-restricted
containers like Python.app (on macOS), (3) adding extensions to
PostgreSQL running in a Kubernetes environment using operators such as
CloudNativePG without having to rebuild the base image for each new
extension.
There is also a tweak in Makefile.global so that it is possible to
install extensions using PGXS into an different directory than the
default, using 'make install prefix=/else/where'. This previously
only worked when specifying the subdirectories, like 'make install
datadir=/else/where/share pkglibdir=/else/where/lib', for purely
implementation reasons. (Of course, without the path feature,
installing elsewhere was rarely useful.)
Author: Peter Eisentraut <peter@eisentraut.org>
Co-authored-by: Matheus Alcantara <matheusssilv97@gmail.com>
Reviewed-by: David E. Wheeler <david@justatheory.com>
Reviewed-by: Gabriele Bartolini <gabriele.bartolini@enterprisedb.com>
Reviewed-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
Reviewed-by: Niccolò Fei <niccolo.fei@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E7C7BFFB-8857-48D4-A71F-88B359FADCFD@justatheory.com
M doc/src/sgml/config.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/ref/create_extension.sgml
M src/Makefile.global.in
M src/backend/commands/extension.c
M src/backend/utils/fmgr/dfmgr.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/commands/extension.h
M src/include/fmgr.h
M src/test/modules/test_extensions/Makefile
M src/test/modules/test_extensions/meson.build
A src/test/modules/test_extensions/t/001_extension_control_path.pl
psql: Allow queries terminated by semicolons while in pipeline mode
commit : 2cce0fe440fb3f252a7be70a89298168009a2c15
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Mar 2025 13:34:59 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Mar 2025 13:34:59 +0900
Currently, the only way to pipe queries in an ongoing pipeline (in a
\startpipeline block) is to leverage the meta-commands able to create
extended queries such as \bind, \parse or \bind_named.
While this is good enough for testing the backend with pipelines, it has
been mentioned that it can also be very useful to allow queries
terminated by semicolons to be appended to a pipeline. For example, it
would be possible to migrate existing psql scripts to use pipelines by
just adding a set of \startpipeline and \endpipeline meta-commands,
making such scripts more efficient.
Doing such a change is proving to be simple in psql: queries terminated
by semicolons can be executed through PQsendQueryParams() without any
parameters set when the pipeline mode is active, instead of
PQsendQuery(), the default, like pgbench. \watch is still forbidden
while in a pipeline, as it expects its results to be processed
synchronously.
The large portion of this commit consists in providing more test
coverage, with mixes of extended queries appended in a pipeline by \bind
and friends, and queries terminated by semicolons.
This improvement has been suggested by Daniel Vérité.
Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/d67b9c19-d009-4a50-8020-1a0ea92366a1@manitou-mail.org
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/common.c
M src/test/regress/expected/psql_pipeline.out
M src/test/regress/sql/psql_pipeline.sql
Fix compiler warning for commit 434dbf69.
commit : 0b53c08677a6515786bde9d4471b42ef7289759e
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 17:26:07 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 17:26:07 +1300
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
M src/interfaces/libpq/fe-auth-oauth-curl.c
oauth: Simplify copy of PGoauthBearerRequest
commit : 1cf4c56480f883bec50753e092da51ceaf3cea67
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 16:58:48 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 16:58:48 +1300
Follow-up to 03366b61d. Since there are no more const members in the
PGoauthBearerRequest struct, the previous memcpy() can be replaced with
simple assignment.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/p4bd7mn6dxr2zdak74abocyltpfdxif4pxqzixqpxpetjwt34h%40qc6jgfmoddvq
M src/interfaces/libpq/fe-auth-oauth.c
oauth: Improve validator docs on interruptibility
commit : 873c0fd67872426e88ac07cbb7ae07457416aafa
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 16:58:06 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 16:58:06 +1300
Andres pointed out that EINTR handling is inadequate for real-world use
cases. Direct module writers to our wait APIs instead.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/p4bd7mn6dxr2zdak74abocyltpfdxif4pxqzixqpxpetjwt34h%40qc6jgfmoddvq
M doc/src/sgml/oauth-validators.sgml
oauth: Disallow synchronous DNS in libcurl
commit : d7e40845f923ed5091b334b1018a547e39846415
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 16:56:19 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 16:56:19 +1300
There is concern that a blocking DNS lookup in libpq could stall a
backend process (say, via FDW). Since there's currently no strong
evidence that synchronous DNS is a popular option, disallow it entirely
rather than warning at configure time. We can revisit if anyone
complains.
Per query from Andres Freund.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/p4bd7mn6dxr2zdak74abocyltpfdxif4pxqzixqpxpetjwt34h%40qc6jgfmoddvq
M config/programs.m4
M configure
M meson.build
oauth: Fix postcondition for set_timer on macOS
commit : 434dbf6907ec8fafa6862a0f00385f293e63ac0e
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 16:22:52 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 16:22:52 +1300
On macOS, readding an EVFILT_TIMER to a kqueue does not appear to clear
out previously queued timer events, so checks for timer expiration do
not work correctly during token retrieval. Switching to IPv4-only
communication exposes the problem, because libcurl is no longer clearing
out other timeouts related to Happy Eyeballs dual-stack handling.
Fully remove and re-register the kqueue timer events during each call to
set_timer(), to clear out any stale expirations.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/CAOYmi%2Bn4EDOOUL27_OqYT2-F2rS6S%2B3mK-ppWb2Ec92UEoUbYA%40mail.gmail.com
M src/interfaces/libpq/fe-auth-oauth-curl.c
oauth: Use IPv4-only issuer in oauth_validator tests
commit : 8d9d5843b55f47d24031165f99b07d41715b93e9
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 16:16:15 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 16:16:15 +1300
The test authorization server implemented in oauth_server.py does not
listen on IPv6. Most of the time, libcurl happily falls back to IPv4
after failing its initial connection, but on NetBSD, something is
consistently showing up on the unreserved IPv6 port and causing a test
failure.
Rather than deal with dual-stack details across all test platforms,
change the issuer to enforce the use of IPv4 only. (This elicits more
punishing timeout behavior from libcurl, so it's a useful change from
the testing perspective as well.)
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Reported-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CAOYmi%2Bn4EDOOUL27_OqYT2-F2rS6S%2B3mK-ppWb2Ec92UEoUbYA%40mail.gmail.com
M src/test/modules/oauth_validator/t/001_server.pl
M src/test/modules/oauth_validator/t/OAuth/Server.pm
M src/test/modules/oauth_validator/t/oauth_server.py
Ensure first ModifyTable rel initialized if all are pruned
commit : 28317de723b60b61c40e7de4341a3029f698ddaf
author : Amit Langote <amitlan@postgresql.org>
date : Wed, 19 Mar 2025 12:14:24 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Wed, 19 Mar 2025 12:14:24 +0900
Commit cbc127917e introduced tracking of unpruned relids to avoid
processing pruned relations, and changed ExecInitModifyTable() to
initialize only unpruned result relations. As a result, MERGE
statements that prune all target partitions can now lead to crashes
or incorrect behavior during execution.
The crash occurs because some executor code paths rely on
ModifyTableState.resultRelInfo[0] being present and initialized,
even when no result relations remain after pruning. For example,
ExecMerge() and ExecMergeNotMatched() use the first resultRelInfo
to determine the appropriate action. Similarly,
ExecInitPartitionInfo() assumes that at least one result relation
exists.
To preserve these assumptions, ExecInitModifyTable() now includes the
first result relation in the initialized result relation list if all
result relations for that ModifyTable were pruned. To enable that,
ExecDoInitialPruning() ensures the first relation is locked if it was
pruned and locking is necessary.
To support this exception to the pruning logic, PlannedStmt now
includes a list of RT indexes identifying the first result relation
of each ModifyTable node in the plan. This allows
ExecDoInitialPruning() to check whether each such relation was
pruned and, if so, lock it if necessary.
Bug: #18830
Reported-by: Robins Tharakan <tharakan@gmail.com>
Diagnozed-by: Tender Wang <tndrwang@gmail.com>
Diagnozed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Co-authored-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Discussion: https://postgr.es/m/18830-1f31ea1dc930d444%40postgresql.org
M src/backend/commands/explain.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeModifyTable.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/setrefs.c
M src/include/executor/execPartition.h
M src/include/executor/executor.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Increase io_combine_limit range to 1MB.
commit : 06fb5612c970b3af95aca3db5a955669b07537ca
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 15:23:12 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 15:23:12 +1300
The default of 128kB is unchanged, but the upper limit is changed from
32 blocks to 128 blocks, unless the operating system's IOV_MAX is too
low. Some other RDBMSes seem to cap their multi-block buffer pool I/O
around this number, and it seems useful to allow experimentation.
The concrete change is to our definition of PG_IOV_MAX, which provides
the maximum for io_combine_limit and io_max_combine_limit. It also
affects a couple of other places that work with arrays of struct iovec
or smaller objects on the stack, so we still don't want to use the
system IOV_MAX directly without a clamp: it is not under our control and
likely to be 1024. 128 seems acceptable for our current usage.
For Windows, we can't use real scatter/gather yet, so we continue to
define our own IOV_MAX value of 16 and emulate preadv()/pwritev() with
loops. Someone would need to research the trade-offs of raising that
number.
NB if trying to see this working: you might temporarily need to hack
BAS_BULKREAD to be bigger, since otherwise the obvious way of "a very
big SELECT" is limited by that for now.
Suggested-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CA%2BhUKG%2B2T9p-%2BzM6Eeou-RAJjTML6eit1qn26f9twznX59qtCA%40mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/storage/aio/read_stream.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/port/pg_iovec.h
Introduce io_max_combine_limit.
commit : 10f6646847515b1ab02735c24b04abaf1996f65f
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 11:40:56 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 19 Mar 2025 11:40:56 +1300
The existing io_combine_limit can be changed by users. The new
io_max_combine_limit is fixed at server startup time, and functions as a
silent clamp on the user setting. That in itself is probably quite
useful, but the primary motivation is:
aio_init.c allocates shared memory for all asynchronous IOs including
some per-block data, and we didn't want to waste memory you'd never used
by assuming they could be up to PG_IOV_MAX. This commit already halves
the size of 'AioHandleIov' and 'AioHandleData'. A follow-up commit can
now expand PG_IOV_MAX without affecting that.
Since our GUC system doesn't support dependencies or cross-checks
between GUCs, the user-settable one now assigns a "raw" value to
io_combine_limit_guc, and the lower of io_combine_limit_guc and
io_max_combine_limit is maintained in io_combine_limit.
Reviewed-by: Andres Freund <andres@anarazel.de> (earlier version)
Discussion: https://postgr.es/m/CA%2BhUKG%2B2T9p-%2BzM6Eeou-RAJjTML6eit1qn26f9twznX59qtCA%40mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/commands/variable.c
M src/backend/storage/aio/aio_init.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/storage/bufmgr.h
M src/include/utils/guc_hooks.h
Fix copy-paste error related to the autovacuum launcher in pgstat_io.c
commit : 17d8bba6dad12e14a7cafca9ef5eef21e577e9c3
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Mar 2025 08:52:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Mar 2025 08:52:10 +0900
Autovacuum launchers perform no WAL IO reads, but pgstat_tracks_io_op()
was tracking them as an allowed combination for the "init" and "normal"
contexts.
This caused the "read", "read_bytes" and "read_time" attributes of
pg_stat_io to show zeros for the autovacuum launcher rather than NULL.
NULL means that a combination of IO object, IO context and IO operation
has no meaning for a backend type. Zero is the same as telling that a
combination is relevant, and that WAL reads are possible in an
autovacuum launcher, but it is not relevant.
Copy-pasto introduced in a051e71e28a1.
Author: Ranier Vilela <ranier.vf@gmail.com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CAEudQAopEMAPiUqE7BvDV+x2fUPmKmb9RrsaoDR+hhQzLKg4PQ@mail.gmail.com
M src/backend/utils/activity/pgstat_io.c
Fix assertion failure in parallel vacuum with minimal maintenance_work_mem setting.
commit : f4290f20dd4d6f75e01fbb87304c3b8f31d1cfea
author : Masahiko Sawada <msawada@postgresql.org>
date : Tue, 18 Mar 2025 16:37:02 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Tue, 18 Mar 2025 16:37:02 -0700
bbf668d66fbf lowered the minimum value of maintenance_work_mem to
64kB. However, in parallel vacuum cases, since the initial underlying
DSA size is 256kB, it attempts to perform a cycle of index vacuuming
and table vacuuming with an empty TID store, resulting in an assertion
failure.
This commit ensures that at least one page is processed before index
vacuuming and table vacuuming begins.
Backpatch to 17, where the minimum maintenance_work_mem value was
lowered.
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAD21AoCEAmbkkXSKbj4dB+5pJDRL4ZHxrCiLBgES_g_g8mVi1Q@mail.gmail.com
Backpatch-through: 17
M src/backend/access/heap/vacuumlazy.c
M src/test/regress/expected/vacuum.out
M src/test/regress/sql/vacuum.sql
Optimize check for pending backend IO stats
commit : 6d3ea48ff1aea5fb1ccfed69424bf93a8643b4a4
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Mar 2025 08:03:06 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Mar 2025 08:03:06 +0900
This commit changes the backend stats code so as we rely on a single
boolean rather than a repeated check based on pg_memory_is_all_zeros()
in the code, making it cheaper should PgStat_PendingIO get bigger in
size.
The frequency of backend stats reports is not a bottleneck, but there is
no reason to not make that cheaper, and the logic is simple as the only
entry points updating backend IO stats are pgstat_count_backend_io_op()
and pgstat_count_backend_io_op_time().
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/Z8WYf1jyy4MwOveQ@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/utils/activity/pgstat_backend.c
Add commit 796bdda484 to .git-blame-ignore-revs.
commit : 7fb418f020815a146c5c9f33e3b344ea9d6df3c7
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 18 Mar 2025 17:00:23 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 18 Mar 2025 17:00:23 -0500
M .git-blame-ignore-revs
Update guidance for running vacuumdb after pg_upgrade.
commit : c9d502eb68094d817fe94c2e954f6fdbb62f4d48
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 18 Mar 2025 16:32:56 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 18 Mar 2025 16:32:56 -0500
Now that pg_upgrade can carry over most optimizer statistics, we
should recommend using vacuumdb's new --missing-stats-only option
to only analyze relations that are missing statistics.
Reviewed-by: John Naylor <johncnaylorls@gmail.com>
Discussion: https://postgr.es/m/Z5O1bpcwDrMgyrYy%40nathan
M doc/src/sgml/ref/pgupgrade.sgml
M src/bin/pg_upgrade/check.c
vacuumdb: Add option for analyzing only relations missing stats.
commit : edba754f052ea0c02287080004aa6051eaa4f597
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 18 Mar 2025 16:32:56 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 18 Mar 2025 16:32:56 -0500
This commit adds a new --missing-stats-only option that can be used
with --analyze-only or --analyze-in-stages. When this option is
specified, vacuumdb will analyze a relation if it lacks any
statistics for a column, expression index, or extended statistics
object. This new option is primarily intended for use after
pg_upgrade (since it can now retain most optimizer statistics), but
it might be useful in other situations, too.
Author: Corey Huinker <corey.huinker@gmail.com>
Co-authored-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: John Naylor <johncnaylorls@gmail.com>
Discussion: https://postgr.es/m/Z5O1bpcwDrMgyrYy%40nathan
M doc/src/sgml/ref/vacuumdb.sgml
M src/bin/scripts/t/100_vacuumdb.pl
M src/bin/scripts/vacuumdb.c
M src/test/perl/PostgreSQL/Test/Cluster.pm
vacuumdb: Teach vacuum_one_database() to reuse query results.
commit : 9c03c8d187210f112371aab881b379bd4800562c
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 18 Mar 2025 16:32:55 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 18 Mar 2025 16:32:55 -0500
Presently, each call to vacuum_one_database() queries the catalogs
to retrieve the list of tables to process. A follow-up commit will
add a "missing stats only" feature to --analyze-in-stages, which
requires saving the catalog query results (since tables without
statistics will have them after the first stage). This commit adds
a new parameter to vacuum_one_database() that specifies either a
previously-retrieved list or a place to return the catalog query
results. Note that nothing uses this new parameter yet.
Author: Corey Huinker <corey.huinker@gmail.com>
Co-authored-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: John Naylor <johncnaylorls@gmail.com>
Discussion: https://postgr.es/m/Z5O1bpcwDrMgyrYy%40nathan
M src/bin/scripts/vacuumdb.c
Doc: manually break lines in wide UUID examples.
commit : a6524105d20b190fb4f5b2116e044e29be88f215
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 18 Mar 2025 15:35:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 18 Mar 2025 15:35:13 -0400
Buildfarm member crake has been complaining "WARNING: The contents of
fo:inline line 1 exceed the available area in the inline-progression
direction by 20500 millipoints. (See position 23808:106)" since
ba57dcfdc went in. The other doc-building animals are not showing
this warning, and I don't see it on my RHEL8 workstation either, but
I was able to reproduce it on a Fedora 41 box. So apparently this
is due to a recent-ish change in DocBook's line-breaking heuristics,
which caused it to cope less well with the UUIDs in these examples.
Put in some zero-width spaces to encourage the PDF toolchain to
break these lines in a better place. (Only one of these examples
actually needs this today, but I marked up all three to ensure that
they get wrapped in a consistent way.)
M doc/src/sgml/func.sgml
smgr: Make SMgrRelation initialization safer against errors
commit : 499faf9063a4c5a9985d7ac657b86e8e34e02199
author : Andres Freund <andres@anarazel.de>
date : Tue, 18 Mar 2025 13:43:10 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 18 Mar 2025 13:43:10 -0400
In case the smgr_open callback failed, the ->pincount field would not be
initialized and the relation would not be put onto the unpinned_relns list.
This buglet was introduced in 21d9c3ee4ef7, in 17.
Discussion: https://postgr.es/m/3vae7l5ozvqtxmd7rr7zaeq3qkuipz365u3rtim5t5wdkr6f4g@vkgf2fogjirl
Backpatch-through: 17
M src/backend/storage/smgr/smgr.c
Introduce squashing of constant lists in query jumbling
commit : 62d712ecfd940f60e68bde5b6972b6859937c412
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 18 Mar 2025 18:56:11 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 18 Mar 2025 18:56:11 +0100
pg_stat_statements produces multiple entries for queries like
SELECT something FROM table WHERE col IN (1, 2, 3, ...)
depending on the number of parameters, because every element of
ArrayExpr is individually jumbled. Most of the time that's undesirable,
especially if the list becomes too large.
Fix this by introducing a new GUC query_id_squash_values which modifies
the node jumbling code to only consider the first and last element of a
list of constants, rather than each list element individually. This
affects both the query_id generated by query jumbling, as well as
pg_stat_statements query normalization so that it suppresses printing of
the individual elements of such a list.
The default value is off, meaning the previous behavior is maintained.
Author: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Sergey Dudoladov (mysterious, off-list)
Reviewed-by: David Geier <geidav.pg@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: Sutou Kouhei <kou@clear-code.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Marcos Pegoraro <marcos@f10.com.br>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Zhihong Yu <zyu@yugabyte.com>
Tested-by: Yasuo Honda <yasuo.honda@gmail.com>
Tested-by: Sergei Kornilov <sk@zsrv.org>
Tested-by: Maciek Sakrejda <m.sakrejda@gmail.com>
Tested-by: Chengxi Sun <sunchengxi@highgo.com>
Tested-by: Jakub Wartak <jakub.wartak@enterprisedb.com>
Discussion: https://postgr.es/m/CA+q6zcWtUbT_Sxj0V6HY6EZ89uv5wuG5aefpe_9n0Jr3VwntFg@mail.gmail.com
M contrib/pg_stat_statements/Makefile
A contrib/pg_stat_statements/expected/squashing.out
M contrib/pg_stat_statements/meson.build
M contrib/pg_stat_statements/pg_stat_statements.c
A contrib/pg_stat_statements/sql/squashing.sql
M doc/src/sgml/config.sgml
M doc/src/sgml/pgstatstatements.sgml
M src/backend/nodes/gen_node_support.pl
M src/backend/nodes/queryjumblefuncs.c
M src/backend/postmaster/launch_backend.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/nodes/nodes.h
M src/include/nodes/primnodes.h
M src/include/nodes/queryjumble.h
aio: Add io_method=worker
commit : 247ce06b883d7b3a40d08312dc03dfb37fbff212
author : Andres Freund <andres@anarazel.de>
date : Tue, 18 Mar 2025 10:52:33 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 18 Mar 2025 10:52:33 -0400
The previous commit introduced the infrastructure to start io_workers. This
commit actually makes the workers execute IOs.
IO workers consume IOs from a shared memory submission queue, run traditional
synchronous system calls, and perform the shared completion handling
immediately. Client code submits most requests by pushing IOs into the
submission queue, and waits (if necessary) using condition variables. Some
IOs cannot be performed in another process due to lack of infrastructure for
reopening the file, and must processed synchronously by the client code when
submitted.
For now the default io_method is changed to "worker". We should re-evaluate
that around beta1, we might want to be careful and set the default to "sync"
for 18.
Reviewed-by: Noah Misch <noah@leadboat.com>
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Co-authored-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
Discussion: https://postgr.es/m/20210223100344.llw5an2aklengrmn@alap3.anarazel.de
Discussion: https://postgr.es/m/stj36ea6yyhoxtqkhpieia2z4krnam7qyetc57rfezgk4zgapf@gcnactj4z56m
M doc/src/sgml/config.sgml
M src/backend/storage/aio/aio.c
M src/backend/storage/aio/aio_init.c
M src/backend/storage/aio/method_worker.c
M src/backend/utils/activity/wait_event_names.txt
M src/backend/utils/misc/postgresql.conf.sample
M src/include/storage/aio.h
M src/include/storage/aio_internal.h
M src/include/storage/lwlocklist.h
M src/tools/pgindent/typedefs.list
aio: Infrastructure for io_method=worker
commit : 55b454d0e14084c841a034073abbf1a0ea937a45
author : Andres Freund <andres@anarazel.de>
date : Tue, 18 Mar 2025 10:52:33 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 18 Mar 2025 10:52:33 -0400
This commit contains the basic, system-wide, infrastructure for
io_method=worker. It does not yet actually execute IO, this commit just
provides the infrastructure for running IO workers, kept separate for easier
review.
The number of IO workers can be adjusted with a PGC_SIGHUP GUC. Eventually
we'd like to make the number of workers dynamically scale up/down based on the
current "IO load".
To allow the number of IO workers to be increased without a restart, we need
to reserve PGPROC entries for the workers unconditionally. This has been
judged to be worth the cost. If it turns out to be problematic, we can
introduce a PGC_POSTMASTER GUC to control the maximum number.
As io workers might be needed during shutdown, e.g. for AIO during the
shutdown checkpoint, a new PMState phase is added. IO workers are shut down
after the shutdown checkpoint has been performed and walsender/archiver have
shut down, but before the checkpointer itself shuts down. See also
87a6690cc69.
Updates PGSTAT_FILE_FORMAT_ID due to the addition of a new BackendType.
Reviewed-by: Noah Misch <noah@leadboat.com>
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Co-authored-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
Discussion: https://postgr.es/m/20210223100344.llw5an2aklengrmn@alap3.anarazel.de
Discussion: https://postgr.es/m/stj36ea6yyhoxtqkhpieia2z4krnam7qyetc57rfezgk4zgapf@gcnactj4z56m
M doc/src/sgml/config.sgml
M src/backend/postmaster/launch_backend.c
M src/backend/postmaster/pmchild.c
M src/backend/postmaster/postmaster.c
M src/backend/storage/aio/Makefile
M src/backend/storage/aio/meson.build
A src/backend/storage/aio/method_worker.c
M src/backend/tcop/postgres.c
M src/backend/utils/activity/pgstat_backend.c
M src/backend/utils/activity/pgstat_io.c
M src/backend/utils/activity/wait_event_names.txt
M src/backend/utils/init/miscinit.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/miscadmin.h
M src/include/pgstat.h
M src/include/storage/aio_subsys.h
A src/include/storage/io_worker.h
M src/include/storage/proc.h
M src/test/regress/expected/stats.out
Fix headerscheck warning.
commit : 549ea06e4217aca10d3a73dc09cf5018c51bc23a
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 18 Mar 2025 08:37:07 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 18 Mar 2025 08:37:07 -0700
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/93731.1742310701@sss.pgh.pa.us
M src/common/unicode/generate-unicode_case_table.pl
M src/common/unicode_case.c
M src/include/common/unicode_case_table.h
Silence compiler warning.
commit : 4078da6c478039bbafdd58138957dc47db545935
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 18 Mar 2025 10:54:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 18 Mar 2025 10:54:10 -0400
Assorted buildfarm members are complaining about "'process_list' may
be used uninitialized in this function" since f76892c9f, presumably
because they don't trust that the switch case labels are exhaustive.
We can silence that by initializing the variable to NULL. Should
a switch fall-through actually happen, we'll get SIGSEGV at the
first use, which is as good as an Assert.
M src/bin/scripts/reindexdb.c
Add X25519 to the default set of curves
commit : daa02c6bd9262adeb44f4a9ed9d94fa6259afd94
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 18 Mar 2025 15:26:27 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 18 Mar 2025 15:26:27 +0100
Since many clients default to the X25519 curve in the TLS handshake,
the fact that the server by defualt doesn't support it cause an extra
roundtrip for each TLS connection. By adding multiple curves, which
is supported since 3d1ef3a15c3eb68da, we can reduce the risk of extra
roundtrips.
Author: Daniel Gustafsson <daniel@yesql.se>
Co-authored-by: Jacob Champion <jacob.champion@enterprisedb.com>
Reported-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/20240616234612.6cslu7nqexquvwj7@awork3.anarazel.de
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/test/ssl/t/SSL/Server.pm
Add some new hooks so extensions can add details to EXPLAIN.
commit : 4fd02bf7cf94c3b6807dcf0b13e076de94f1e4ff
author : Robert Haas <rhaas@postgresql.org>
date : Tue, 18 Mar 2025 09:09:34 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Tue, 18 Mar 2025 09:09:34 -0400
Specifically, add a per-node hook that is called after the per-node
information has been displayed but before we display children, and a
per-query hook that is called after existing query-level information
is printed. This assumes that extension-added information should
always go at the end rather than the beginning or the middle, but
that seems like an acceptable limitation for simplicity. It also
assumes that extensions will only want to add information, not remove
or reformat existing details; those also seem like acceptable
restrictions, at least for now.
If multiple EXPLAIN extensions are used, the order in which any
additional details are printed is likely to depend on the order in
which the modules are loaded. That seems OK, since the user may
have opinions about the order in which output should appear, and the
extension author can't really know whether their stuff is more or
less important to a particular user than some other extension.
Discussion: http://postgr.es/m/CA+TgmoYSzg58hPuBmei46o8D3SKX+SZoO4K_aGQGwiRzvRApLg@mail.gmail.com
Reviewed-by: Srinath Reddy <srinath2133@gmail.com>
Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
M src/backend/commands/explain.c
M src/include/commands/explain.h
Simplify reindexdb coding
commit : f76892c9ff7e5f2dcb4073310d1a5273f47d1d9d
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 18 Mar 2025 14:21:26 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 18 Mar 2025 14:21:26 +0100
get_parallel_object_list() was trying to serve two masters, and it was
doing a bad job at both. In particular, it treated the given user_list
as an output argument, but only sometimes. This was confusing, and the
two paths through it didn't really have all that much in common, so the
complexity wasn't buying us much. Split it in two:
get_parallel_tables_list() handles the straightforward cases for
schemas, databases and tables, takes one list as argument and returns
another list.
A new function get_parallel_tabidx_list() handles the case for indexes.
This takes a list as argument and outputs two lists, just like
get_parallel_object_list used to do, but now the API is clearer (IMO
anyway). Another difference is that accompanying the list of indexes
now we have a list of tables as an OID list rather than a
fully-qualified table name list. This makes some comparisons easier,
and we don't really need the names of the tables, just their OIDs.
(This requires atooid, which requires <stdlib.h>).
Author: Ranier Vilela <ranier.vf@gmail.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/CAEudQArfqr0-s0VVPSEh=0kgOgBJvFNdGW=xSL5rBcr0WDMQYQ@mail.gmail.com
M src/bin/scripts/reindexdb.c
M src/bin/scripts/t/090_reindexdb.pl
Increase default maintenance_io_concurrency to 16
commit : cc6be07ebde2aef7cc0507f997f563ce77c00a00
author : Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 18 Mar 2025 09:08:10 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 18 Mar 2025 09:08:10 -0400
Since its introduction in fc34b0d9de27a, the default
maintenance_io_concurrency has been larger than the default
effective_io_concurrency. maintenance_io_concurrency primarily
controlled prefetching done on behalf of the whole system, for
operations like recovery. Therefore it makes sense for it to have a
value equal to or greater than effective_io_concurrency, which controls
I/O concurrency for reading a relation in a bitmap heap scan.
ff79b5b2ab increased effective_io_concurrency to 16, so we'll increase
maintenance_io_concurrency as well. For now, though, we'll keep the
defaults of effective_io_concurrency and maintenance_io_concurrency
equal to one another (16).
On fast, high IOPs systems, significantly higher values of
maintenance_io_concurrency are observably beneficial [1]. However, such
values would flood low IOPs systems and increase overall system I/O
latency.
It is worth mentioning that since 9256822608f and c3e775e608f,
maintenance_io_concurrency also controls the I/O concurrency of each
vacuum worker. Since many autovacuum workers may be simultaneously
issuing I/Os, we want to keep maintenance_io_concurrency appropriately
conservative.
[1] https://postgr.es/m/c5d52837-6256-0556-ac8c-d6d3d558820a%40enterprisedb.com
Suggested-by: Jakub Wartak <jakub.wartak@enterprisedb.com>
Discussion: https://postgr.es/m/CAKZiRmxdHQaU%2B2Zpe6d%3Dx%3D0vigJ1sfWwwVYLJAf%3Dud_wQ_VcUw%40mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/utils/misc/postgresql.conf.sample
M src/include/storage/bufmgr.h
Fix indentation again.
commit : 796bdda484c838313959f65e2b700f14ac7c0e66
author : Robert Haas <rhaas@postgresql.org>
date : Tue, 18 Mar 2025 09:02:36 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Tue, 18 Mar 2025 09:02:36 -0400
Because somehow I manage to keep forgetting this.
M src/include/commands/explain.h
Make it possible for loadable modules to add EXPLAIN options.
commit : c65bc2e1d14a2d4daed7c1921ac518f2c5ac3d17
author : Robert Haas <rhaas@postgresql.org>
date : Tue, 18 Mar 2025 08:41:12 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Tue, 18 Mar 2025 08:41:12 -0400
Modules can use RegisterExtensionExplainOption to register new
EXPLAIN options, and GetExplainExtensionId, GetExplainExtensionState,
and SetExplainExtensionState to store related state inside the
ExplainState object.
Since this substantially increases the amount of code that needs
to handle ExplainState-related tasks, move a few bits of existing
code to a new file explain_state.c and add the rest of this
infrastructure there.
See the comments at the top of explain_state.c for further
explanation of how this mechanism works.
This does not yet provide a way for such such options to do anything
useful. The intention is that we'll add hooks for that purpose in a
separate commit.
Discussion: http://postgr.es/m/CA+TgmoYSzg58hPuBmei46o8D3SKX+SZoO4K_aGQGwiRzvRApLg@mail.gmail.com
Reviewed-by: Srinath Reddy <srinath2133@gmail.com>
Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
M contrib/auto_explain/auto_explain.c
M contrib/file_fdw/file_fdw.c
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/commands/Makefile
M src/backend/commands/createas.c
M src/backend/commands/explain.c
M src/backend/commands/explain_dr.c
M src/backend/commands/explain_format.c
A src/backend/commands/explain_state.c
M src/backend/commands/meson.build
M src/backend/commands/prepare.c
M src/backend/executor/execAmi.c
M src/backend/tcop/pquery.c
M src/include/commands/explain.h
A src/include/commands/explain_state.h
M src/include/commands/prepare.h
M src/include/nodes/extensible.h
M src/tools/pgindent/typedefs.list
Allow non-btree unique indexes for matviews
commit : 9d6db8bec19413cd0167f1e59d1af005a997bd3e
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 18 Mar 2025 11:29:15 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 18 Mar 2025 11:29:15 +0100
We were rejecting non-btree indexes in some cases owing to the
inability to determine the equality operators for other index AMs;
that problem no longer exists, because we can look up the equality
operator using COMPARE_EQ.
Stop rejecting these indexes, but instead rely on all unique indexes
having equality operators. Unique indexes must have equality
operators.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/commands/matview.c
Allow non-btree unique indexes for partition keys
commit : f278e1fe300ab1b7d43c3efb55a29aa17e5f5dda
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 18 Mar 2025 11:25:36 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 18 Mar 2025 11:25:36 +0100
We were rejecting non-btree indexes in some cases owing to the
inability to determine the equality operators for other index AMs;
that problem no longer exists, because we can look up the equality
operator using COMPARE_EQ. The problem of not knowing the strategy
number for equality in other index AMs is already resolved.
Stop rejecting the indexes upfront, and instead reject any for which
the equality operator lookup fails.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/commands/indexcmds.c
Add some opfamily support functions to lsyscache.c
commit : 7317e641268fb9b08d32519920adf1f16c8591ea
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 18 Mar 2025 11:17:43 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 18 Mar 2025 11:17:43 +0100
Add get_opfamily_method() and get_opfamily_member_for_cmptype() in
lsyscache.c. No callers yet, but we'll add some soon. This is part
of generalizing some parts of the code away from having btree
hardcoded and use CompareType instead.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/utils/cache/lsyscache.c
M src/include/utils/lsyscache.h
Fix typo.
commit : 122a9af5def2db78f2c2131958eab8873bfee93b
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 18 Mar 2025 14:18:09 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 18 Mar 2025 14:18:09 +0530
Author: vignesh C <vignesh21@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CALDaNm1KqJ0VFfDJRPbfYi9Shz6LHFEE-Ckn+eqsePfKhebv9w@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Use correct variable name in publicationcmds.c.
commit : 01e27aab05f5f83ab4b79688795848d787dab738
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 18 Mar 2025 14:06:51 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 18 Mar 2025 14:06:51 +0530
subid was used at few places for publicationid in publicationcmds.c/.h.
Author: vignesh C <vignesh21@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CALDaNm1KqJ0VFfDJRPbfYi9Shz6LHFEE-Ckn+eqsePfKhebv9w@mail.gmail.com
M src/backend/commands/publicationcmds.c
M src/include/commands/publicationcmds.h
Fix the test 005_char_signedness.
commit : c462b054ba605d23c1ec139fcca3d758ac139026
author : Masahiko Sawada <msawada@postgresql.org>
date : Mon, 17 Mar 2025 21:34:10 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Mon, 17 Mar 2025 21:34:10 -0700
pg_upgrade test 005_char_signedness was leaving files like
delete_old_cluster.sh in the source directory for VPATH and meson
builds. The fix is to change the directory to tmp_check before running
the test.
Reported-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: http://postgr.es/m/CA+TgmoYg5e4oznn0XGoJ3+mceG1qe_JJt34rF2JLwvGS5T1hgQ@mail.gmail.com
M src/bin/pg_upgrade/t/005_char_signedness.pl
psql: Add \sendpipeline to send query buffers while in a pipeline
commit : 17caf6644546eccb4a1f6e0137ea1d0f58da8c16
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 18 Mar 2025 09:41:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 18 Mar 2025 09:41:21 +0900
In the initial pipeline support for psql added in 41625ab8ea3d, \g was
used as the way to push extended query into an ongoing pipeline. \gx
was blocked.
These two meta-commands have format-related options that can be applied
when fetching a query result (expanded, etc.). As the results of a
pipeline are fetched asynchronously, not at the moment of the
meta-command execution but at the moment of a \getresults or a
\endpipeline, authorizing \g while blocking \gx leads to a confusing
implementation, making one think that psql should be smart enough to
remember the output format options defined from the time when \g or \gx
were executed. Doing so would lead to more code complications when
retrieving a batch of results. There is an extra argument other than
simplicity here: the output format options defined at the point of a
\getresults or a \endpipeline execution should be what affect the output
format for a batch of results.
To avoid any confusion, we have settled to the introduction of a new
meta-command called \sendpipeline, replacing \g when within a pipeline.
An advantage of this design is that it is possible to add new options
specific to pipelines when sending a query buffer, independent of \g
and \gx, should it prove to be necessary.
Most of the changes of this commit happen in the regression tests, where
\g is replaced by \sendpipeline. More tests are added to check that \g
is not allowed.
Per discussion between the author, Daniel Vérité and me.
Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/ad4b9f1a-f7fe-4ab8-8546-90754726d0be@manitou-mail.org
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/help.c
M src/bin/psql/tab-complete.in.c
M src/test/regress/expected/psql.out
M src/test/regress/expected/psql_pipeline.out
M src/test/regress/sql/psql.sql
M src/test/regress/sql/psql_pipeline.sql
aio: Add core asynchronous I/O infrastructure
commit : da7226993fd4b73d8b40abb7167d124eada97f2e
author : Andres Freund <andres@anarazel.de>
date : Mon, 17 Mar 2025 18:51:33 -0400
committer: Andres Freund <andres@anarazel.de>
date : Mon, 17 Mar 2025 18:51:33 -0400
The main motivations to use AIO in PostgreSQL are:
a) Reduce the time spent waiting for IO by issuing IO sufficiently early.
In a few places we have approximated this using posix_fadvise() based
prefetching, but that is fairly limited (no completion feedback, double the
syscalls, only works with buffered IO, only works on some OSs).
b) Allow to use Direct-I/O (DIO).
DIO can offload most of the work for IO to hardware and thus increase
throughput / decrease CPU utilization, as well as reduce latency. While we
have gained the ability to configure DIO in d4e71df6, it is not yet usable
for real world workloads, as every IO is executed synchronously.
For portability, the new AIO infrastructure allows to implement AIO using
different methods. The choice of the AIO method is controlled by the new
io_method GUC. As of this commit, the only implemented method is "sync",
i.e. AIO is not actually executed asynchronously. The "sync" method exists to
allow to bypass most of the new code initially.
Subsequent commits will introduce additional IO methods, including a
cross-platform method implemented using worker processes and a linux specific
method using io_uring.
To allow different parts of postgres to use AIO, the core AIO infrastructure
does not need to know what kind of files it is operating on. The necessary
behavioral differences for different files are abstracted as "AIO
Targets". One example target would be smgr. For boring portability reasons,
all targets currently need to be added to an array in aio_target.c. This
commit does not implement any AIO targets, just the infrastructure for
them. The smgr target will be added in a later commit.
Completion (and other events) of IOs for one type of file (i.e. one AIO
target) need to be reacted to differently, based on the IO operation and the
callsite. This is made possible by callbacks that can be registered on
IOs. E.g. an smgr read into a local buffer does not need to update the
corresponding BufferDesc (as there is none), but a read into shared buffers
does. This commit does not contain any callbacks, they will be added in
subsequent commits.
For now the AIO infrastructure only understands READV and WRITEV operations,
but it is expected that more operations will be added. E.g. fsync/fdatasync,
flush_range and network operations like send/recv.
As of this commit, nothing uses the AIO infrastructure. Later commits will add
an smgr target, md.c and bufmgr.c callbacks and then finally use AIO for
read_stream.c IO, which, in one fell swoop, will convert all read stream users
to AIO.
The goal is to use AIO in many more places. There are patches to use AIO for
checkpointer and bgwriter that are reasonably close to being ready. There also
are prototypes to use it for WAL, relation extension, backend writes and many
more. Those prototypes were important to ensure the design of the AIO
subsystem is not too limiting (e.g. WAL writes need to happen in critical
sections, which influenced a lot of the design).
A future commit will add an AIO README explaining the AIO architecture and how
to use the AIO subsystem. The README is added later, as it references details
only added in later commits.
Many many more people than the folks named below have contributed with
feedback, work on semi-independent patches etc. E.g. various folks have
contributed patches to use the read stream infrastructure (added by Thomas in
b5a9b18cd0b) in more places. Similarly, a *lot* of folks have contributed to
the CI infrastructure, which I had started to work on to make adding AIO
feasible.
Some of the work by contributors has gone into the "v1" prototype of AIO,
which heavily influenced the current design of the AIO subsystem. None of the
code from that directly survives, but without the prototype, the current
version of the AIO infrastructure would not exist.
Similarly, the reviewers below have not necessarily looked at the current
design or the whole infrastructure, but have provided very valuable input. I
am to blame for problems, not they.
Author: Andres Freund <andres@anarazel.de>
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Co-authored-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Co-authored-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Jakub Wartak <jakub.wartak@enterprisedb.com>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Antonin Houska <ah@cybertec.at>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
Discussion: https://postgr.es/m/20210223100344.llw5an2aklengrmn@alap3.anarazel.de
Discussion: https://postgr.es/m/stj36ea6yyhoxtqkhpieia2z4krnam7qyetc57rfezgk4zgapf@gcnactj4z56m
M src/backend/storage/aio/Makefile
M src/backend/storage/aio/aio.c
A src/backend/storage/aio/aio_callback.c
M src/backend/storage/aio/aio_init.c
A src/backend/storage/aio/aio_io.c
A src/backend/storage/aio/aio_target.c
M src/backend/storage/aio/meson.build
A src/backend/storage/aio/method_sync.c
M src/backend/utils/activity/wait_event_names.txt
M src/include/storage/aio.h
A src/include/storage/aio_internal.h
A src/include/storage/aio_types.h
M src/tools/pgindent/typedefs.list
aio: Basic subsystem initialization
commit : 02844012b304ba80d1c48d51f6fe10bb622490cc
author : Andres Freund <andres@anarazel.de>
date : Mon, 17 Mar 2025 18:51:33 -0400
committer: Andres Freund <andres@anarazel.de>
date : Mon, 17 Mar 2025 18:51:33 -0400
This commit just does the minimal wiring up of the AIO subsystem, added in the
next commit, to the rest of the system. The next commit contains more details
about motivation and architecture.
This commit is kept separate to make it easier to review, separating the
changes across the tree, from the implementation of the new subsystem.
We discussed squashing this commit with the main commit before merging AIO,
but there has been a mild preference for keeping it separate.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
M doc/src/sgml/config.sgml
M src/backend/access/transam/xact.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/bgwriter.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/pgarch.c
M src/backend/postmaster/walsummarizer.c
M src/backend/postmaster/walwriter.c
M src/backend/replication/walsender.c
M src/backend/storage/aio/Makefile
A src/backend/storage/aio/aio.c
A src/backend/storage/aio/aio_init.c
M src/backend/storage/aio/meson.build
M src/backend/storage/ipc/ipci.c
M src/backend/utils/init/postinit.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/backend/utils/resowner/resowner.c
A src/include/storage/aio.h
A src/include/storage/aio_subsys.h
M src/include/utils/guc.h
M src/include/utils/guc_hooks.h
M src/include/utils/resowner.h
M src/tools/pgindent/typedefs.list
Add commit 203c1b4cc4 to .git-blame-ignore-revs.
commit : 65db3963ae7154b8f01e4d73dc6b1ffd81c70e1e
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 17 Mar 2025 15:58:02 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 17 Mar 2025 15:58:02 -0500
M .git-blame-ignore-revs
Fix indentation.
commit : 203c1b4cc49455364b6bcab8034900d1c016b9cd
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 17 Mar 2025 16:05:35 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 17 Mar 2025 16:05:35 -0400
Commit 99aeb84703177308c1541e2d11c09fdc59acb724 wasn't fully
reindented prior to commit.
M src/bin/pg_combinebackup/copy_file.c
pg_upgrade: Remove some dead code.
commit : 7e05df430be36ad34d9c797b21f56a4b9c64bbc8
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 17 Mar 2025 13:18:14 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 17 Mar 2025 13:18:14 -0500
Since commit e469f0aaf3, tablespace_suffix can't be empty.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/Z9hc3mkYFKR56Xof%40nathan
M src/bin/pg_upgrade/check.c
tests: Expand temp table tests to some pin related matters
commit : 1a22a8a0f1319e3d80cab0962374f235359b59ff
author : Andres Freund <andres@anarazel.de>
date : Mon, 17 Mar 2025 14:12:44 -0400
committer: Andres Freund <andres@anarazel.de>
date : Mon, 17 Mar 2025 14:12:44 -0400
Added tests:
- recovery from running out of unpinned local buffers
- that we don't run out of unpinned buffers due to read stream (only recently
fixed, in 92fc6856cb4)
- temp tables can't be dropped while in use by cursors
Discussion: weskknhckugbdm2yt7sa2uq53xlsax67gcdkac34sanb7qpd3p@hcc2wadao5wy
Discussion: https://postgr.es/m/ge6nsuddurhpmll3xj22vucvqwp4agqz6ndtcf2mhyeydzarst@l75dman5x53p
M src/test/regress/expected/temp.out
M src/test/regress/parallel_schedule
M src/test/regress/sql/temp.sql
pg_combinebackup: Add -k, --link option.
commit : 99aeb84703177308c1541e2d11c09fdc59acb724
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 17 Mar 2025 14:03:14 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 17 Mar 2025 14:03:14 -0400
This is similar to pg_upgrade's --link option, except that here we won't
typically be able to use it for every input file: sometimes we will need
to reconstruct a complete backup from blocks stored in different files.
However, when a whole file does need to be copied, we can use an
optimized copying strategy: see the existing --clone and
--copy-file-range options and the code to use CopyFile() on Windows.
This commit adds a new strategy: add a hard link to an existing file.
Making a hard link doesn't actually copy anything, but it makes sense
for the code to treat it as doing so.
This is useful when the input directories are merely staging directories
that will be removed once the restore is complete. In such cases, there
is no need to actually copy the data, and making a bunch of new hard
links can be very quick. However, it would be quite dangerous to use it
if the input directories might later be reused for any other purpose,
since starting postgres on the output directory would destructively
modify the input directories. For that reason, using this new option
causes pg_combinebackup to emit a warning about the danger involved.
Author: Israel Barth Rubio <barthisrael@gmail.com>
Co-authored-by: Robert Haas <robertmhaas@gmail.com> (cosmetic changes)
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Discussion: http://postgr.es/m/CA+TgmoaEFsYHsMefNaNkU=2SnMRufKE3eVJxvAaX=OWgcnPmPg@mail.gmail.com
M doc/src/sgml/ref/pg_combinebackup.sgml
M src/bin/pg_combinebackup/copy_file.c
M src/bin/pg_combinebackup/copy_file.h
M src/bin/pg_combinebackup/meson.build
M src/bin/pg_combinebackup/pg_combinebackup.c
A src/bin/pg_combinebackup/t/010_hardlink.pl
Unify wording of user-facing "row security" messages.
commit : ed762e94253d5dd7810da13b8dd802baa876c55e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 Mar 2025 12:53:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 Mar 2025 12:53:50 -0400
Row-level security is mostly referred to as "row security" in
user-facing messages. Commit cd3c45125 introduced one inconsistent
use of "row level security"; make that one match the rest.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/20250317.135305.573764276033358827.horikyota.ntt@gmail.com
M src/bin/pg_dump/pg_restore.c
Fix inconsistent quoting for some options in TAP tests
commit : 3943f5cff6a66639d295a6f1922e54b79d2ab16f
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Mar 2025 14:07:12 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Mar 2025 14:07:12 +0900
This commit addresses some inconsistencies with how the options of some
routines from PostgreSQL/Test/ are written, mainly for init() and
init_from_backup() in Cluster.pm. These are written as unquoted, except
in the locations updated here.
Changes extracted from a larger patch by the same author.
Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://postgr.es/m/87jz8rzf3h.fsf@wibble.ilmari.org
M contrib/auto_explain/t/001_auto_explain.pl
M contrib/basebackup_to_shell/t/001_basic.pl
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_basebackup/t/011_in_place_tablespace.pl
M src/bin/pg_combinebackup/t/008_promote.pl
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/test/postmaster/t/002_connection_limits.pl
M src/test/ssl/t/002_scram.pl
M src/test/ssl/t/003_sslinfo.pl
Apply more consistent style for command options in TAP tests
commit : 19c6e92b13b2336d1db1b236457ab15d0894b506
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Mar 2025 12:42:23 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Mar 2025 12:42:23 +0900
This commit reshapes the grammar of some commands to apply a more
consistent style across the board, following rules similar to
ce1b0f9da03e:
- Elimination of some pointless used-once variables.
- Use of long options, to self-document better the options used.
- Use of fat commas to link option names and their assigned values,
including redirections, so as perltidy can be tricked to put them
together.
Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://postgr.es/m/87jz8rzf3h.fsf@wibble.ilmari.org
M contrib/auto_explain/t/001_auto_explain.pl
M contrib/basebackup_to_shell/t/001_basic.pl
M contrib/sepgsql/t/001_sepgsql.pl
M src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
M src/bin/pg_resetwal/t/002_corrupted.pl
M src/bin/pg_rewind/t/009_growing_files.pl
M src/bin/pg_rewind/t/RewindTest.pm
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
M src/bin/pg_upgrade/t/003_logical_slots.pl
M src/bin/pg_upgrade/t/005_char_signedness.pl
M src/bin/pg_verifybackup/t/002_algorithm.pl
M src/bin/pg_verifybackup/t/003_corruption.pl
M src/bin/pg_verifybackup/t/004_options.pl
M src/bin/pg_verifybackup/t/008_untar.pl
M src/bin/pg_verifybackup/t/009_extract.pl
M src/bin/pg_verifybackup/t/010_client_untar.pl
M src/bin/pg_waldump/t/001_basic.pl
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/pgbench/t/002_pgbench_no_server.pl
M src/bin/psql/t/001_basic.pl
M src/bin/psql/t/020_cancel.pl
M src/interfaces/libpq/t/001_uri.pl
M src/interfaces/libpq/t/005_negotiate_encryption.pl
M src/test/modules/commit_ts/t/003_standby_2.pl
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
M src/test/modules/ssl_passphrase_callback/t/001_testfunc.pl
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/perl/PostgreSQL/Test/BackgroundPsql.pm
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/test/perl/PostgreSQL/Test/Kerberos.pm
M src/test/perl/PostgreSQL/Test/Utils.pm
M src/test/postmaster/t/002_connection_limits.pl
M src/test/recovery/t/001_stream_rep.pl
M src/test/recovery/t/006_logical_decoding.pl
M src/test/recovery/t/013_crash_restart.pl
M src/test/recovery/t/017_shm.pl
M src/test/recovery/t/021_row_visibility.pl
M src/test/recovery/t/022_crash_temp_files.pl
M src/test/recovery/t/032_relfilenode_reuse.pl
M src/test/recovery/t/035_standby_logical_decoding.pl
M src/test/recovery/t/040_standby_failover_slots_sync.pl
M src/test/recovery/t/041_checkpoint_at_promote.pl
M src/tools/pg_bsd_indent/t/001_pg_bsd_indent.pl
Revert "Add redo LSN to pgstats files"
commit : 5721e5453ebc59360b6756fe72d7499c4a02177c
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Mar 2025 08:35:12 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Mar 2025 08:35:12 +0900
This reverts commit b860848232aa, that was added as a prerequisite for
the support of pgstats data flush across checkpoints, linking a pgstats
file to a specific checkpoint redo LSN.
As reported, this is proving to be currently problematic when going
through a pg_upgrade, that does direct manipulations of the control file
in the new cluster. The LSN stored in the pgstats file is not able to
cope with any changes done in the control file by pg_upgrade yet,
causing the pgstats file to be discarded when starting the new cluster
after overriding its redo LSN (one is a `pg_resetwal -l` where the new
cluster's start LSN is bumped by a hardcoded value of 8 segments, see
copy_xact_xlog_xid).
The least painful path going forward is likely going to be a refactor of
the pgstats code so as it is possible to read and write some of its data
with some routines in src/common/, so as pg_upgrade or pg_resetwal are
able to update its data. The main point is that we are going to need a
LSN in the stats file should we make it written at checkpoint time and
not only as part of a shutdown sequence. It is too late to dive into
these details for v18, so let's revert the change, and let's try to
figure out all the details in the next release cycle. The pgstats file
is currently only written as part of a shutdown sequence, and its
contents are still lost on crash, same as older releases.
Bump PGSTAT_FILE_FORMAT_ID.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/2563883.1741826489@sss.pgh.pa.us
M src/backend/access/transam/xlog.c
M src/backend/utils/activity/pgstat.c
M src/include/pgstat.h
pg_dump, pg_dumpall, pg_restore: Add --no-policies option.
commit : cd3c45125d2d92e86ad7530b162562a23d063c26
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 16 Mar 2025 18:08:15 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 16 Mar 2025 18:08:15 -0400
Add --no-policies option to control row level security policy handling
in dump and restore operations. When this option is used, both CREATE
POLICY commands and ALTER TABLE ... ENABLE ROW LEVEL SECURITY commands
are excluded from dumps and skipped during restores.
This is useful in scenarios where policies need to be redefined in the
target system or when moving data between environments with different
security requirements.
Author: Nikolay Samokhvalov <nik@postgres.ai>
Reviewed-by: Greg Sabino Mullane <htamfids@gmail.com>
Reviewed-by: Jim Jones <jim.jones@uni-muenster.de>
Reviewed-by: newtglobal postgresql_contributors <postgresql_contributors@newtglobalcorp.com>
Discussion: https://postgr.es/m/CAM527d8kG2qPKvbfJ=OYJkT7iRNd623Bk+m-a4ngm+nyHYsHog@mail.gmail.com
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_restore.sgml
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_dump/pg_restore.c
M src/bin/pg_dump/t/002_pg_dump.pl
contrib/isn: Make weak mode a GUC setting, and fix related functions.
commit : 44890442398cf3a65230d53167e61905d2b0d348
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 16 Mar 2025 13:45:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 16 Mar 2025 13:45:48 -0400
isn's weak mode used to be a simple static variable, settable only
via the isn_weak(boolean) function. This wasn't optimal, as this
means it doesn't respect transactions nor respond to RESET ALL.
This patch makes isn.weak a GUC parameter instead, so that
it acts like any other user-settable parameter.
The isn_weak() functions are retained for backwards compatibility.
But we must fix their volatility markings: they were marked IMMUTABLE
which is surely incorrect, and PARALLEL RESTRICTED which isn't right
for GUC-related functions either. Mark isn_weak(boolean) as
VOLATILE and PARALLEL UNSAFE, matching set_config(). Mark isn_weak()
as STABLE and PARALLEL SAFE, matching current_setting().
Reported-by: Viktor Holmberg <v@viktorh.net>
Diagnosed-by: Daniel Gustafsson <daniel@yesql.se>
Author: Viktor Holmberg <v@viktorh.net>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/790bc1f9-74dc-4b50-94d2-8147315b1556@Spark
M contrib/isn/Makefile
M contrib/isn/expected/isn.out
A contrib/isn/isn–1.2–1.3.sql
M contrib/isn/isn.c
M contrib/isn/isn.control
M contrib/isn/isn.h
M contrib/isn/meson.build
M contrib/isn/sql/isn.sql
M doc/src/sgml/isn.sgml
reindexdb: Fix the index-level REINDEX with multiple jobs
commit : 682c5be25c28192c56e9d5e2a9ca14673a2fcf4b
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 16 Mar 2025 13:28:22 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 16 Mar 2025 13:28:22 +0200
47f99a407d introduced a parallel index-level REINDEX. The code was written
assuming that running run_reindex_command() with 'async == true' can schedule
a number of queries for a connection. That's not true, and the second query
sent using run_reindex_command() will wait for the completion of the previous
one.
This commit fixes that by putting REINDEX commands for the same table into a
single query.
Also, this commit removes the 'async' argument from run_reindex_command(),
as only its call always passes 'async == true'.
Reported-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/202503071820.j25zn3lo4hvn%40alvherre.pgsql
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Backpatch-through: 17
M src/bin/scripts/reindexdb.c
pg_createsubscriber: Remove some code bloat in the atexit() callback
commit : 83e5763d4dddd9b24b738d8335b3cbfc2742e4ac
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 16 Mar 2025 19:20:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 16 Mar 2025 19:20:49 +0900
This commit adjusts some code added by e117cfb2f6c6 in the atexit()
callback of pg_createsubscriber.c, in charge of performing post-failure
cleanup actions. The code loops over all the databases specified, and
it is changed here to rely on a single LogicalRepInfo for each database
rather than always using LogicalRepInfos, simplifying its logic.
Author: Peter Smith <smithpb2250@gmail.com>
Discussion: https://postgr.es/m/CAHut+PtdBSVi4iH7BObDVwDNVwOpn+H3fezOBdSTtENx+rhNMw@mail.gmail.com
M src/bin/pg_basebackup/pg_createsubscriber.c
localbuf: Introduce StartLocalBufferIO()
commit : 771ba90298e2b79a94678c96f7960274a7896feb
author : Andres Freund <andres@anarazel.de>
date : Sat, 15 Mar 2025 12:30:07 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sat, 15 Mar 2025 12:30:07 -0400
To initiate IO on a shared buffer we have StartBufferIO(). For temporary table
buffers no similar function exists - likely because the code for that
currently is very simple due to the lack of concurrency.
However, the upcoming AIO support will make it possible to re-encounter a
local buffer, while the buffer already is the target of IO. In that case we
need to wait for already in-progress IO to complete. This commit makes it
easier to add the necessary code, by introducing StartLocalBufferIO().
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_b9anbWzEs5AAF9WCvcEVmgz-1AkHSQ-CLLy-p7WHzvFw@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/include/storage/buf_internals.h
localbuf: Introduce FlushLocalBuffer()
commit : 4b4d33b9ea9ff6bdc813b5b7b1aa4a6a3a2a2d5c
author : Andres Freund <andres@anarazel.de>
date : Sat, 15 Mar 2025 12:30:07 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sat, 15 Mar 2025 12:30:07 -0400
Previously we had two paths implementing writing out temporary table
buffers. For shared buffers, the logic for that is centralized in
FlushBuffer(). Introduce FlushLocalBuffer() to do the same for local buffers.
Besides being a nice cleanup on its own, it also makes an upcoming change
slightly easier.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_b9anbWzEs5AAF9WCvcEVmgz-1AkHSQ-CLLy-p7WHzvFw@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/include/storage/buf_internals.h
localbuf: Introduce TerminateLocalBufferIO()
commit : dd6f2618f681e699cb5f2122a3f036beaa89f992
author : Andres Freund <andres@anarazel.de>
date : Sat, 15 Mar 2025 12:30:07 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sat, 15 Mar 2025 12:30:07 -0400
Previously TerminateLocalBufferIO() was open-coded in multiple places, which
doesn't seem like a great idea. While TerminateLocalBufferIO() currently is
rather simple, an upcoming patch requires additional code to be added to
TerminateLocalBufferIO(), making this modification particularly worthwhile.
For some reason FlushRelationBuffers() previously cleared BM_JUST_DIRTIED,
even though that's never set for temporary buffers. This is not carried over
as part of this change.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_b9anbWzEs5AAF9WCvcEVmgz-1AkHSQ-CLLy-p7WHzvFw@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/include/storage/buf_internals.h
localbuf: Introduce InvalidateLocalBuffer()
commit : 0762a151b0e018944694ccac07e521adcdf7a06f
author : Andres Freund <andres@anarazel.de>
date : Sat, 15 Mar 2025 12:30:07 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sat, 15 Mar 2025 12:30:07 -0400
Previously, there were three copies of this code, two of them
identical. There's no good reason for that.
This change is nice on its own, but the main motivation is the AIO patchset,
which needs to add extra checks the deduplicated code, which of course is
easier if there is only one version.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_b9anbWzEs5AAF9WCvcEVmgz-1AkHSQ-CLLy-p7WHzvFw@mail.gmail.com
M src/backend/storage/buffer/localbuf.c
localbuf: Fix dangerous coding pattern in GetLocalVictimBuffer()
commit : fa6af9b25e4b449e6e006d9b3434315c0e8e4402
author : Andres Freund <andres@anarazel.de>
date : Sat, 15 Mar 2025 12:30:07 -0400
committer: Andres Freund <andres@anarazel.de>
date : Sat, 15 Mar 2025 12:30:07 -0400
If PinLocalBuffer() were to modify the buf_state, the buf_state in
GetLocalVictimBuffer() would be out of date. Currently that does not happen,
as PinLocalBuffer() only modifies the buf_state if adjust_usagecount=true and
GetLocalVictimBuffer() passes false.
However, it's easy to make this not the case anymore - it cost me a few hours
to debug the consequences.
The minimal fix would be to just refetch the buf_state after after calling
PinLocalBuffer(), but the same danger exists in later parts of the
function. Instead, declare buf_state in the narrower scopes and re-read the
state in conditional branches. Besides being safer, it also fits well with
an upcoming set of cleanup patches that move the contents of the conditional
branches in GetLocalVictimBuffer() into helper functions.
I "broke" this in 794f2594479.
Arguably this should be backpatched, but as the relevant functions are not
exported and there is no actual misbehaviour, I chose to not backpatch, at
least for now.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_b9anbWzEs5AAF9WCvcEVmgz-1AkHSQ-CLLy-p7WHzvFw@mail.gmail.com
M src/backend/storage/buffer/localbuf.c
Silence perl critic
commit : 5eabd91a83adae75f53b61857343660919fef4c7
author : Andrew Dunstan <andrew@dunslane.net>
date : Sat, 15 Mar 2025 17:41:54 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sat, 15 Mar 2025 17:41:54 -0400
Commit 27bdec06841 uses a loop variable that is not strictly local to
the loop. Perlcritic disapproves, and there's really no reason as the
variable is not used outside the loop.
Per buildfarm animals koel and crake.
M src/common/unicode/generate-unicode_case_table.pl
Optimization for lower(), upper(), casefold() functions.
commit : 27bdec06841d1bb004ca7627eac97808b08a7ac7
author : Jeff Davis <jdavis@postgresql.org>
date : Sat, 15 Mar 2025 13:00:50 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Sat, 15 Mar 2025 13:00:50 -0700
Improve performance and reduce table sizes for case mapping.
The main case mapping table stores only 16-bit offsets, which can be
used to look up the mapped code point in any of the case tables (fold,
lower, upper, or title case). Simple case pairs point to the same
offsets.
Generate a function in generate-unicode_case_table.pl that consists of
a nested branches to test for specific codepoint ranges that determine
the offset in the main table.
Other approaches were considered, such as representing these ranges as
another structure (rather than branches in a generated function), or a
different approach such as a radix tree, or perfect hashing. The
author implemented and tested these alternatives and settled on the
generated branches.
Author: Alexander Borisov <lex.borisov@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/7cac7e66-9a3b-4e3f-a997-42aa0c401f80%40gmail.com
M src/common/unicode/generate-unicode_case_table.pl
M src/common/unicode_case.c
M src/include/common/unicode_case_table.h
Remove table AM callback scan_bitmap_next_block
commit : c3953226a07527a1e2f7f410b83e1a7021f42888
author : Melanie Plageman <melanieplageman@gmail.com>
date : Sat, 15 Mar 2025 10:37:46 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Sat, 15 Mar 2025 10:37:46 -0400
After pushing the bitmap iterator into table-AM specific code (as part
of making bitmap heap scan use the read stream API in 2b73a8cd33b7),
scan_bitmap_next_block() no longer returns the current block number.
Since scan_bitmap_next_block() isn't returning any relevant information
to bitmap table scan code, it makes more sense to get rid of it.
Now, bitmap table scan code only calls table_scan_bitmap_next_tuple(),
and the heap AM implementation of scan_bitmap_next_block() is a local
helper in heapam_handler.c.
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/flat/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
M src/backend/access/heap/heapam_handler.c
M src/backend/access/table/tableamapi.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/backend/optimizer/util/plancat.c
M src/include/access/tableam.h
BitmapHeapScan uses the read stream API
commit : 2b73a8cd33b745c5b8a7f44322f86642519e3a40
author : Melanie Plageman <melanieplageman@gmail.com>
date : Sat, 15 Mar 2025 10:34:42 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Sat, 15 Mar 2025 10:34:42 -0400
Make Bitmap Heap Scan use the read stream API instead of invoking
ReadBuffer() for each block indicated by the bitmap.
The read stream API handles prefetching, so remove all of the explicit
prefetching from bitmap heap scan code.
Now, heap table AM implements a read stream callback which uses the
bitmap iterator to return the next required block to the read stream
code.
Tomas Vondra conducted extensive regression testing of this feature.
Andres Freund, Thomas Munro, and I analyzed regressions and Thomas Munro
patched the read stream API.
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Tested-by: Tomas Vondra <tomas@vondra.me>
Tested-by: Andres Freund <andres@anarazel.de>
Tested-by: Thomas Munro <thomas.munro@gmail.com>
Tested-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/flat/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/include/access/tableam.h
M src/include/nodes/execnodes.h
Separate TBM[Shared|Private]Iterator and TBMIterateResult
commit : 944e81bf99db2b5b70b8a389d4f273534da73f74
author : Melanie Plageman <melanieplageman@gmail.com>
date : Sat, 15 Mar 2025 10:10:51 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Sat, 15 Mar 2025 10:10:51 -0400
Remove the TBMIterateResult member from the TBMPrivateIterator and
TBMSharedIterator and make tbm_[shared|private_]iterate() take a
TBMIterateResult as a parameter.
This allows tidbitmap API users to manage multiple TBMIterateResults per
scan. This is required for bitmap heap scan to use the read stream API,
with which there may be multiple I/Os in flight at once, each one with a
TBMIterateResult.
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/d4bb26c9-fe07-439e-ac53-c0e244387e01%40vondra.me
M src/backend/access/gin/ginget.c
M src/backend/access/gin/ginscan.c
M src/backend/access/heap/heapam_handler.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/backend/nodes/tidbitmap.c
M src/include/access/gin_private.h
M src/include/nodes/tidbitmap.h
Simplify distance heuristics in read_stream.c.
commit : 799959dc7cf0e2462601bea8d07b6edec3fa0c4f
author : Thomas Munro <tmunro@postgresql.org>
date : Sun, 16 Mar 2025 03:04:08 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sun, 16 Mar 2025 03:04:08 +1300
Make the distance control heuristics simpler and more aggressive in
preparation for asynchronous I/O.
The v17 version of read_stream.c made a conservative choice to limit the
look-ahead distance when streaming sequential blocks, because it
couldn't benefit very much from looking ahead further yet. It had a
three-behavior model where only random I/O would rapidly increase the
look-ahead distance, to support read-ahead advice. Sequential I/O would
move it towards the io_combine_limit setting, just enough to build one
full-sized synchronous I/O at a time, and then expect kernel read-ahead
to avoid I/O stalls.
That already left I/O performance on the table with advice-based I/O
concurrency, since sequential blocks could be followed by random jumps,
eg with the proposed streaming Bitmap Heap Scan patch.
It is time to delete the cautious middle option and adjust the distance
based on recent I/O needs only, since asynchronous reads will need to be
started ahead of time whether random or sequential. It is still limited
by io_combine_limit, *_io_concurrency, buffer availability and
strategy ring size, as before.
Reviewed-by: Andres Freund <andres@anarazel.de> (earlier version)
Tested-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGK_%3D4CVmMHvsHjOVrK6t4F%3DLBpFzsrr3R%2BaJYN8kcTfWg%40mail.gmail.com
M src/backend/storage/aio/read_stream.c
Improve read_stream.c advice for dense streams.
commit : 7ea8cd15661e3b0da4b57be2f25fdd512951576f
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 15 Mar 2025 10:23:59 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 15 Mar 2025 10:23:59 +1300
read_stream.c tries not to issue read-ahead advice when it thinks the
kernel's own read-ahead should be active, ie when using buffered I/O and
reading sequential blocks. It previously gave up too easily, and issued
advice only for the first read of up to io_combine_limit blocks in a
larger range of sequential blocks after random jump. The following read
could suffer an avoidable I/O stall.
Fix, by continuing to issue advice until the corresponding preadv()
calls catch up with the start of the region we're currently issuing
advice for, if ever. That's when the kernel actually sees the
sequential pattern. Advice is now disabled only when the stream is
entirely sequential as far as we can see in the look-ahead window, or
in other words, when a sequential region is larger than we can cover
with the current io_concurrency and io_combine_limit settings.
While refactoring the advice control logic, also get rid of the
"suppress_advice" argument that was passed around between functions to
skip useless posix_fadvise() calls immediately followed by preadv().
read_stream_start_pending_read() can figure that out, so let's
concentrate knowledge of advice heuristics in fewer places (our goal
being to make advice-based I/O concurrency a legacy mode soon).
The problem cases were revealed by Tomas Vondra's extensive regression
testing with many different disk access patterns using Melanie
Plageman's streaming Bitmap Heap Scan patch, in a battle against the
venerable always-issue-advice-and-always-one-block-at-a-time code.
Reviewed-by: Andres Freund <andres@anarazel.de> (earlier version)
Reported-by: Melanie Plageman <melanieplageman@gmail.com>
Reported-by: Tomas Vondra <tomas@vondra.me>
Reported-by: Andres Freund <andres@anarazel.de>
Tested-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGK_%3D4CVmMHvsHjOVrK6t4F%3DLBpFzsrr3R%2BaJYN8kcTfWg%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2BhUKGJ3HSWciQCz8ekP1Zn7N213RfA4nbuotQawfpq23%2Bw-5Q%40mail.gmail.com
M src/backend/storage/aio/read_stream.c
doc: Explain more thoroughly when a table rewrite is needed
commit : 11bd8318602fc2282a6201f714c15461dc2009c6
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 14 Mar 2025 20:44:59 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 14 Mar 2025 20:44:59 +0100
Author: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Reviewed-by: Robert Treat <rob@xzilla.net>
Discussion: https://postgr.es/m/00e6eb5f5c793b8ef722252c7a519c9a@oss.nttdata.com
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/alter_table.sgml
Doc: remove obsolete comment.
commit : 1c9242b2cdc358b8be9c4e9967823a24a7807525
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 Mar 2025 14:08:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 Mar 2025 14:08:47 -0400
This para should have been removed by 2f9661311, which made it
both false and irrelevant. Noted while looking at SQL function
plancache patch.
M src/include/utils/plancache.h
Add GUC option to log lock acquisition failures.
commit : 6d376c3b0d1e79c318d2a1c04097025784e28377
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 14 Mar 2025 23:14:12 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 14 Mar 2025 23:14:12 +0900
This commit introduces a new GUC, log_lock_failure, which controls whether
a detailed log message is produced when a lock acquisition fails. Currently,
it only supports logging lock failures caused by SELECT ... NOWAIT.
The log message includes information about all processes holding or
waiting for the lock that couldn't be acquired, helping users analyze and
diagnose the causes of lock failures.
Currently, this option does not log failures from SELECT ... SKIP LOCKED,
as that could generate excessive log messages if many locks are skipped,
causing unnecessary noise.
This mechanism can be extended in the future to support for logging
lock failures from other commands, such as LOCK TABLE ... NOWAIT.
Author: Yuki Seino <seinoyu@oss.nttdata.com>
Co-authored-by: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Discussion: https://postgr.es/m/411280a186cc26ef7034e0f2dfe54131@oss.nttdata.com
M doc/src/sgml/config.sgml
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/storage/lmgr.h
M src/include/storage/lock.h
M src/include/storage/proc.h
Optimize iteration over PGPROC for fast-path lock searches.
commit : e80171d57c25caf4362a7de17e96195f905386ea
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 14 Mar 2025 22:49:29 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 14 Mar 2025 22:49:29 +0900
This commit improves efficiency in FastPathTransferRelationLocks()
and GetLockConflicts(), which iterate over PGPROCs to search for
fast-path locks.
Previously, these functions recalculated the fast-path group during
every loop iteration, even though it remained constant. This update
optimizes the process by calculating the group once and reusing it
throughout the loop.
The functions also now skip empty fast-path groups, avoiding
unnecessary scans of their slots. Additionally, groups belonging to
inactive backends (with pid=0) are always empty, so checking
the group is sufficient to bypass these backends, further enhancing
performance.
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/07d5fd6a-71f1-4ce8-8602-4cc6883f4bd1@oss.nttdata.com
M src/backend/storage/lmgr/lock.c
Simplify and generalize PrepareSortSupportFromIndexRel()
commit : a359d3701991d040bf7b66cfa7052546eb923c38
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 14 Mar 2025 10:34:08 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 14 Mar 2025 10:34:08 +0100
PrepareSortSupportFromIndexRel() was accepting btree strategy numbers
purely for the purpose of comparing it later against btree strategies
to determine if the sort direction was forward or reverse. Change
that. Instead, pass a bool directly, to indicate the same without an
unfortunate assumption that a strategy number refers specifically to a
btree strategy. (This is similar in spirit to commits 0d2aa4d4937 and
c594f1ad2ba.)
(This could arguably be simplfied further by having the callers fill
in ssup_reverse directly. But this way, it preserves consistency by
having all PrepareSortSupport*() variants be responsible for filling
in ssup_reverse.)
Moreover, remove the hardcoded check against BTREE_AM_OID, and check
against amcanorder instead, which is the actual requirement.
Co-authored-by: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/access/nbtree/nbtsort.c
M src/backend/utils/sort/sortsupport.c
M src/backend/utils/sort/tuplesortvariants.c
M src/include/utils/sortsupport.h
Remove direct handling of reloptions for toast tables
commit : 1548c3a30436dd825cfbf57923c6766b2fddd355
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 14 Mar 2025 09:28:51 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 14 Mar 2025 09:28:51 +0100
It doesn't actually work, even with allow_system_table_mods turned on:
the ALTER TABLE operation is rejected by ATSimplePermissions(), so even
the error message we're adding in this commit is unreachable.
Add a test case for it.
Author: Nikolay Shaplov <dhyan@nataraj.su>
Discussion: https://postgr.es/m/1913854.tdWV9SEqCh@thinkpad-pgpro
M src/backend/commands/tablecmds.c
M src/test/modules/unsafe_tests/expected/alter_system_table.out
M src/test/modules/unsafe_tests/sql/alter_system_table.sql
Respect changing pin limits in read_stream.c.
commit : 92fc6856cb4c598f99c58b862bc34aebc6f2ec25
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 14 Mar 2025 20:39:43 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 14 Mar 2025 20:39:43 +1300
To avoid pinning too much of the buffer pool at once, read_stream.c
previously used LimitAdditionalPins(). The coding was naive, and only
considered the available buffers at stream construction time.
This commit checks before each StartReadBuffers() call with
GetAdditionalPinLimit(). The result might change over time due to pins
acquired outside this stream by the same backend. No extra CPU cycles
are added to the all-buffered fast-path code, but the I/O-starting path
now considers the up-to-date remaining buffer limit.
In practice it was quite difficult to exceed limits and cause any real
problems in v17, so no back-patch for now, but proposed changes will
make it easier.
Per code review from Andres, in the course of testing his AIO patches.
Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions)
Discussion: https://postgr.es/m/CA%2BhUKGK_%3D4CVmMHvsHjOVrK6t4F%3DLBpFzsrr3R%2BaJYN8kcTfWg%40mail.gmail.com
M src/backend/storage/aio/read_stream.c
Activate Python "Limited API" in PL/Python
commit : 0793ab810038999c0659b3aad9525aa7ef4c8c26
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 14 Mar 2025 08:17:40 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 14 Mar 2025 08:17:40 +0100
This allows building PL/Python against any Python 3.x version and
using another Python 3.x version at run time. This is useful for
installers that want to run against a separately downloaded Python, so
that they don't have to bundle it themselves.
This builds on the earlier patch to only use APIs supported by the
Limited API.
At the moment, this is not activated on MSVC because that leads to
build failures that no one could explain or cared enough to address.
This could be done later.
Reviewed-by: Jakob Egger <jakob@eggerapps.at>
Discussion: https://www.postgresql.org/message-id/flat/ee410de1-1e0b-4770-b125-eeefd4726a24@eisentraut.org
M src/pl/plpython/plpython.h
Swap order of extern/static and pg_nodiscard
commit : 05cbd6cb22d235d2e52ac309266a5fc50053dc16
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 14 Mar 2025 07:18:07 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 14 Mar 2025 07:18:07 +0100
When pg_nodiscard was first added, the C standard draft had it as a
function specifier, and so the code comment about placement was
written with that in mind. The final C23 standard has it as an
attribute and the placement rules are a bit different for that.
Specifically, it needs to be before extern or static. (Or at least
both current clang and gcc require that.) So just swap these. (To be
clear: The current implementation with gcc attributes doesn't care.
This change is just for maximum forward compatibility for non-gcc
compilers.) This also keeps the order consistent with the previously
introduced pg_noreturn. Also update the code comment to reflect the
mentioned developments since its introduction.
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/flat/pxr5b3z7jmkpenssra5zroxi7qzzp6eswuggokw64axmdixpnk@zbwxuq7gbbcw
M src/include/c.h
M src/include/common/base64.h
M src/include/nodes/pg_list.h
M src/include/storage/buffile.h
M src/include/utils/guc.h
M src/include/utils/palloc.h
Improve buffer manager API for backend pin limits.
commit : 01261fb07888642efa98ba8d4117654bfd2a413d
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 14 Mar 2025 15:10:43 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 14 Mar 2025 15:10:43 +1300
Previously the support functions assumed that the caller needed one pin
to make progress, and could optionally use some more, allowing enough
for every connection to do the same. Add a couple more functions for
callers that want to know:
* what the maximum possible number could be, irrespective of currently
held pins, for space planning purposes
* how many additional pins they could acquire right now, without the
special case allowing one pin, for callers that already hold pins and
could already make progress even if no extra pins are available
The pin limit logic began in commit 31966b15. This refactoring is
better suited to read_stream.c, which will be adjusted to respect the
remaining limit as it changes over time in a follow-up commit. It also
computes MaxProportionalPins up front, to avoid performing divisions
whenever a caller needs to check the balance.
Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions)
Discussion: https://postgr.es/m/CA%2BhUKGK_%3D4CVmMHvsHjOVrK6t4F%3DLBpFzsrr3R%2BaJYN8kcTfWg%40mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/include/storage/bufmgr.h
Fix ALTER SUBSCRIPTION ... SET PUBLICATION ... command.
commit : 7c99dc587a010a0c40d72a0e435111ca7a371c02
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 14 Mar 2025 08:57:40 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 14 Mar 2025 08:57:40 +0530
The problem is that ALTER SUBSCRIPTION ... SET PUBLICATION ... will lead
to restarting of apply worker and after the restart, the apply worker will
use the existing slot and replication origin corresponding to the
subscription. Now, it is possible that before the restart, the origin has
not been updated, and the WAL start location points to a location before
where PUBLICATION pointed to by SET PUBLICATION doesn't exist, and that
can lead to an error like: "ERROR: publication "pub1" does not exist".
Once this error occurs, apply worker will never be able to proceed and
will always return the same error.
We decided to skip loading the publication if the publication does not
exist. The publication is loaded later and updates the relation entry when
the publication gets created.
We decided not to backpatch this as this is a behaviour change, and we don't
see field reports. This problem has been found by intermittent buildfarm
failures.
Author: vignesh C <vignesh21@gmail.com>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/flat/CALDaNm0-n8FGAorM%2BbTxkzn%2BAOUyx5%3DL_XmnvOP6T24%2B-NcBKg%40mail.gmail.com
Discussion: https://postgr.es/m/CAA4eK1+T-ETXeRM4DHWzGxBpKafLCp__5bPA_QZfFQp7-0wj4Q@mail.gmail.com
M src/backend/replication/pgoutput/pgoutput.c
M src/test/subscription/t/024_add_drop_pub.pl
Fix ARRAY_SUBLINK and ARRAY[] for int2vector and oidvector input.
commit : 4618045bee4a6d3efcb489c319649d8dd9aaa738
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 13 Mar 2025 16:07:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 13 Mar 2025 16:07:55 -0400
If the given input_type yields valid results from both
get_element_type and get_array_type, initArrayResultAny believed the
former and treated the input as an array type. However this is
inconsistent with what get_promoted_array_type does, leading to
situations where the output of an ARRAY() subquery is labeled with
the wrong type: it's labeled as oidvector[] but is really a 2-D
array of OID. That at least results in strange output, and can
result in crashes if further processing such as unnest() is applied.
AFAIK this is only possible with the int2vector and oidvector
types, which are special-cased to be treated mostly as true arrays
even though they aren't quite.
Fix by switching the logic to match get_promoted_array_type by
testing get_array_type not get_element_type, and remove an Assert
thereby made pointless. (We need not introduce a symmetrical
check for get_element_type in the other if-branch, because
initArrayResultArr will check it.) This restores the behavior
that existed before bac27394a introduced initArrayResultAny:
the output really is int2vector[] or oidvector[].
Comparable confusion exists when an input of an ARRAY[] construct
is int2vector or oidvector: transformArrayExpr decides it's dealing
with a multidimensional array constructor, and we end up with
something that's a multidimensional OID array but is alleged to be
of type oidvector. I have not found a crashing case here, but it's
easy to demonstrate totally-wrong results. Adjust that code so
that what you get is an oidvector[] instead, for consistency with
ARRAY() subqueries. (This change also makes these types work like
domains-over-arrays in this context, which seems correct.)
Bug: #18840
Reported-by: yang lei <ylshiyu@126.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18840-fbc9505f066e50d6@postgresql.org
Backpatch-through: 13
M src/backend/parser/parse_expr.c
M src/backend/utils/adt/arrayfuncs.c
M src/test/regress/expected/arrays.out
M src/test/regress/sql/arrays.sql
ATExecSetRelOptions: Reduce scope of 'isnull' variable
commit : c7fc8808a91ed1b5810abb5f6043be7b6d58dbcf
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 13 Mar 2025 18:15:59 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 13 Mar 2025 18:15:59 +0100
Author: Nikolay Shaplov <dhyan@nataraj.su>
Reviewed-by: Timur Magomedov <t.magomedov@postgrespro.ru>
Discussion: https://postgr.es/m/1913854.tdWV9SEqCh@thinkpad-pgpro
M src/backend/commands/tablecmds.c
Make lwlocknames.h generated file less ugly
commit : da0f0582e81e2fc42e19feaece56364acfb3f640
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 13 Mar 2025 17:38:21 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 13 Mar 2025 17:38:21 +0100
We can make the output look a bit better by aligning each lock's
definition, so add some padding space to achieve that. This change
makes no practical difference, but casual onlookers will be less
distracted by (lack of) whitespace.
Author: Gurjeet Singh <gurjeet@singh.im>
Discussion: https://postgr.es/m/CABwTF4VxfwDtRV-H22_XK4XeDogaV-Vaobu+af5U=8ZAZn9ZZQ@mail.gmail.com
M src/backend/storage/lmgr/generate-lwlocknames.pl
Add reverse(bytea).
commit : 0697b23906adabccba41fa7bf19909eb98bdf0dd
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 13 Mar 2025 11:20:53 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 13 Mar 2025 11:20:53 -0500
This commit introduces a function for reversing the order of the
bytes in binary strings.
Bumps catversion.
Author: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://postgr.es/m/CAJ7c6TMe0QVRuNssUArbMi0bJJK32%2BzNA3at5m3osrBQ25MHuw%40mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/varlena.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
Fix copy-and-paste mistake in error message
commit : bb25276205b45b60b5a1ddb7db3b66f501a02164
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 13 Mar 2025 15:17:08 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 13 Mar 2025 15:17:08 +0100
Introduced in commit a68159ff2b3.
M src/backend/storage/file/fd.c
pg_noreturn to replace pg_attribute_noreturn()
commit : 3691edfab97187789b8a1cbb9dce4acf0ecd8f5a
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 13 Mar 2025 12:25:14 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 13 Mar 2025 12:25:14 +0100
We want to support a "noreturn" decoration on more compilers besides
just GCC-compatible ones, but for that we need to move the decoration
in front of the function declaration instead of either behind it or
wherever, which is the current style afforded by GCC-style attributes.
Also rename the macro to "pg_noreturn" to be similar to the C11
standard "noreturn".
pg_noreturn is now supported on all compilers that support C11 (using
_Noreturn), as well as GCC-compatible ones (using __attribute__, as
before), as well as MSVC (using __declspec). (When PostgreSQL
requires C11, the latter two variants can be dropped.)
Now, all supported compilers effectively support pg_noreturn, so the
extra code for !HAVE_PG_ATTRIBUTE_NORETURN can be dropped.
This also fixes a possible problem if third-party code includes
stdnoreturn.h, because then the current definition of
#define pg_attribute_noreturn() __attribute__((noreturn))
would cause an error.
Note that the C standard does not support a noreturn attribute on
function pointer types. So we have to drop these here. There are
only two instances at this time, so it's not a big loss. In one case,
we can make up for it by adding the pg_noreturn to a wrapper function
and adding a pg_unreachable(), in the other case, the latter was
already done before.
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/flat/pxr5b3z7jmkpenssra5zroxi7qzzp6eswuggokw64axmdixpnk@zbwxuq7gbbcw
M contrib/dblink/dblink.c
M contrib/pgcrypto/px.h
M src/backend/access/transam/xlogrecovery.c
M src/backend/backup/basebackup_incremental.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/launch_backend.c
M src/backend/postmaster/postmaster.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/walsender.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/fmgr/dfmgr.c
M src/backend/utils/hash/dynahash.c
M src/backend/utils/mmgr/slab.c
M src/bin/pg_combinebackup/load_manifest.c
M src/bin/pg_dump/pg_backup_utils.h
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/bin/pg_verifybackup/pg_verifybackup.h
M src/bin/pgbench/pgbench.h
M src/common/parse_manifest.c
M src/include/bootstrap/bootstrap.h
M src/include/c.h
M src/include/commands/defrem.h
M src/include/common/parse_manifest.h
M src/include/mb/pg_wchar.h
M src/include/parser/parse_relation.h
M src/include/parser/scanner.h
M src/include/postmaster/autovacuum.h
M src/include/postmaster/bgworker_internals.h
M src/include/postmaster/bgwriter.h
M src/include/postmaster/pgarch.h
M src/include/postmaster/postmaster.h
M src/include/postmaster/startup.h
M src/include/postmaster/syslogger.h
M src/include/postmaster/walsummarizer.h
M src/include/postmaster/walwriter.h
M src/include/replication/slotsync.h
M src/include/replication/walreceiver.h
M src/include/replication/walsender_private.h
M src/include/storage/ipc.h
M src/include/storage/lock.h
M src/include/tcop/backend_startup.h
M src/include/tcop/tcopprot.h
M src/include/utils/elog.h
M src/include/utils/float.h
M src/include/utils/help_config.h
M src/include/utils/memutils_internal.h
M src/interfaces/ecpg/preproc/preproc_extern.h
M src/pl/plpgsql/src/plpgsql.h
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/test_shm_mq/test_shm_mq.h
M src/test/modules/worker_spi/worker_spi.c
M src/timezone/zic.c
M src/tools/pg_bsd_indent/err.h
Fix incorrect handling of subquery pullup
commit : cc5d98525d43c22b98f360ef0f2c8d7dc57f04dc
author : Richard Guo <rguo@postgresql.org>
date : Thu, 13 Mar 2025 16:36:03 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Thu, 13 Mar 2025 16:36:03 +0900
When pulling up a subquery, if the subquery's target list items are
used in grouping set columns, we need to wrap them in PlaceHolderVars.
This ensures that expressions retain their separate identity so that
they will match grouping set columns when appropriate.
In 90947674f, we decided to wrap subquery outputs that are non-var
expressions in PlaceHolderVars. This prevents const-simplification
from merging them into the surrounding expressions after subquery
pullup, which could otherwise lead to failing to match those
subexpressions to grouping set columns, with the effect that they'd
not go to null when expected.
However, that left some loose ends. If the subquery's target list
contains two or more identical Var expressions, we can still fail to
match the Var expression to the expected grouping set expression.
This is not related to const-simplification, but rather to how we
match expressions to lower target items in setrefs.c.
For sort/group expressions, we use ressortgroupref matching, which
works well. For other expressions, we primarily rely on comparing the
expressions to determine if they are the same. Therefore, we need a
way to prevent setrefs.c from matching the expression to some other
identical ones.
To fix, wrap all subquery outputs in PlaceHolderVars if the parent
query uses grouping sets, ensuring that they preserve their separate
identity throughout the whole planning process.
Reported-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Discussion: https://postgr.es/m/CAMbWs4-meSahaanKskpBn0KKxdHAXC1_EJCVWHxEodqirrGJnw@mail.gmail.com
M src/backend/optimizer/prep/prepjointree.c
M src/backend/rewrite/rewriteManip.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
M src/tools/pgindent/typedefs.list
Remove code setting wrap_non_vars to true for UNION ALL subqueries
commit : 4c4961171577e292220d050c338020f2c6bf4c76
author : Richard Guo <rguo@postgresql.org>
date : Thu, 13 Mar 2025 16:34:28 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Thu, 13 Mar 2025 16:34:28 +0900
In pull_up_simple_subquery and pull_up_constant_function, there is
code that sets wrap_non_vars to true when dealing with an appendrel
member. The goal is to wrap subquery outputs that are not simple Vars
in PlaceHolderVars, ensuring that what we pull up doesn't get merged
into a surrounding expression during later processing, which could
cause it to fail to match the expression actually available from the
appendrel.
However, this is unnecessary. When pulling up an appendrel child
subquery, the only part of the upper query that could reference the
appendrel child yet is the translated_vars list of the associated
AppendRelInfo that we just made for this child. Furthermore, we do
not want to force use of PHVs in the AppendRelInfo, as there is no
outer join between. In fact, perform_pullup_replace_vars always sets
wrap_non_vars to false before performing pullup_replace_vars on the
AppendRelInfo.
This patch simply removes the code that sets wrap_non_vars to true for
UNION ALL subqueries.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Discussion: https://postgr.es/m/CAMbWs4-VXDEi1v+hZYLxpOv0riJxHsCkCH1f46tLnhonEAyGCQ@mail.gmail.com
M src/backend/optimizer/prep/prepjointree.c
Refactor convert_case() to prepare for optimizations.
commit : d3b2e5e1ab5ca3e9738d20409debfdd299e65a2e
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 12 Mar 2025 21:51:52 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 12 Mar 2025 21:51:52 -0700
Upcoming optimizations will add complexity to convert_case(). This
patch reorganizes slightly so that the complexity can be contained
within the logic to convert the case of a single character, rather
than mixing it in with logic to iterate through the string.
Reviewed-by: Alexander Borisov <lex.borisov@gmail.com>
Discussion: https://postgr.es/m/44005c3d-88f4-4a26-981f-fd82dfa8e313@gmail.com
M src/common/unicode_case.c
Avoid invalidating all RelationSyncCache entries on publication rename.
commit : 3abe9dc18892b9f69bb48a2eb21fbe5cf348a489
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 13 Mar 2025 09:03:45 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 13 Mar 2025 09:03:45 +0530
On Publication rename, we need to only invalidate the RelationSyncCache
entries corresponding to relations that are part of the publication being
renamed.
As part of this patch, we introduce a new invalidation message to
invalidate the cache maintained by the logical decoding output plugin. We
can't use existing relcache invalidation for this purpose, as that would
unnecessarily cause relcache invalidations in other backends.
This will improve performance by building fewer relation cache entries
during logical replication.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Reviewed-by: Hou Zhijie <houzj.fnst@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/OSCPR01MB14966C09AA201EFFA706576A7F5C92@OSCPR01MB14966.jpnprd01.prod.outlook.com
M src/backend/access/rmgrdesc/standbydesc.c
M src/backend/commands/alter.c
M src/backend/commands/publicationcmds.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/utils/cache/inval.c
M src/include/commands/publicationcmds.h
M src/include/pg_config_manual.h
M src/include/storage/sinval.h
M src/include/utils/inval.h
M src/test/subscription/t/007_ddl.pl
Fix read_stream.c for changing io_combine_limit.
commit : 75da2bece670059f3c1a3628dfbc3d24cc9638b8
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 13 Mar 2025 15:43:34 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 13 Mar 2025 15:43:34 +1300
In a couple of places, read_stream.c assumed that io_combine_limit would
be stable during the lifetime of a stream. That is not true in at least
one unusual case: streams held by CURSORs where you could change the GUC
between FETCH commands, with unpredictable results.
Fix, by storing stream->io_combine_limit and referring only to that
after construction. This mirrors the treatment of the other important
setting {effective,maintenance}_io_concurrency, which is stored in
stream->max_ios.
One of the cases was the queue overflow space, which was sized for
io_combine_limit and could be overrun if the GUC was increased. Since
that coding was a little hard to follow, also introduce a variable for
better readability instead of open-coding the arithmetic. Doing so
revealed an off-by-one thinko while clamping max_pinned_buffers to
INT16_MAX, though that wasn't a live bug due to the current limits on
GUC values.
Back-patch to 17.
Discussion: https://postgr.es/m/CA%2BhUKG%2B2T9p-%2BzM6Eeou-RAJjTML6eit1qn26f9twznX59qtCA%40mail.gmail.com
M src/backend/storage/aio/read_stream.c
Fix copy-paste error in datum_to_jsonb_internal()
commit : d4f79865d4207bd93fa00f521638c289f036e276
author : Amit Langote <amitlan@postgresql.org>
date : Thu, 13 Mar 2025 09:56:36 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Thu, 13 Mar 2025 09:56:36 +0900
Commit 3c152a27b06 mistakenly repeated JSONTYPE_JSON in a condition,
omitting JSONTYPE_CAST. As a result, datum_to_jsonb_internal() failed
to reject inputs that were casts (e.g., from an enum to json as in the
example below) when used as keys in JSON constructors.
This led to a crash in cases like:
SELECT JSON_OBJECT('happy'::mood: '123'::jsonb);
where 'happy'::mood is implicitly cast to json. The missing check
meant such casted values weren’t properly rejected as invalid
(non-scalar) JSON keys.
Reported-by: Maciek Sakrejda <maciek@pganalyze.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Maciek Sakrejda <maciek@pganalyze.com>
Discussion: https://postgr.es/m/CADXhmgTJtJZK9A3Na_ry+Xrq-ghjcejBRhcRMzWZvbd__QdgJA@mail.gmail.com
Backpatch-through: 17
M src/backend/utils/adt/jsonb.c
M src/test/regress/expected/sqljson.out
M src/test/regress/sql/sqljson.sql
pg_rewind: Add dbname to primary_conninfo when using --write-recovery-conf.
commit : 4ecdd4110d5cbaf107c0c85d16df78dffe0a9574
author : Masahiko Sawada <msawada@postgresql.org>
date : Wed, 12 Mar 2025 16:56:04 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Wed, 12 Mar 2025 16:56:04 -0700
This commit enhances pg_rewind's --write-recovery-conf option to
include the dbname in the generated primary_conninfo value when
specified in the --source-server option. With this modification, the
rewound server can connect to the primary server without manual
configuration file modifications when sync_replication_slots is
enabled.
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Discussion: https://postgr.es/m/CAD21AoAkW=Ht0k9dVoBTCcqLiiZ2MXhVr+d=j2T_EZMerGrLWQ@mail.gmail.com
M doc/src/sgml/ref/pg_rewind.sgml
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_basebackup/streamutil.h
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/t/RewindTest.pm
M src/fe_utils/recovery_gen.c
M src/include/fe_utils/recovery_gen.h
Add b955df443 to .git-blame-ignore-revs
commit : cdc1471cc7626ffd44416abe92d37de038a0a0ec
author : David Rowley <drowley@postgresql.org>
date : Thu, 13 Mar 2025 12:44:26 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 13 Mar 2025 12:44:26 +1300
M .git-blame-ignore-revs
Fix indentation issue
commit : b955df443405e056fd9047ef819a1465654f9d79
author : David Rowley <drowley@postgresql.org>
date : Thu, 13 Mar 2025 12:41:44 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 13 Mar 2025 12:41:44 +1300
Introduced recently by 9e088f7dd
Per buildfarm member koel
M contrib/pg_logicalinspect/pg_logicalinspect.c
Fix compiler warning in pg_logicalinspect.
commit : 9e088f7dd8cd1f14813d0919a68564d55cbff18a
author : Masahiko Sawada <msawada@postgresql.org>
date : Wed, 12 Mar 2025 14:23:56 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Wed, 12 Mar 2025 14:23:56 -0700
Oversight in bd65cb3cd48.
Reported-by: David Rowley <dgrowleyml@gmail.com>
Reported-by: Nathan Bossart <nathandbossart@gmail.com>
Author: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvqrhFfnetbcwgGkJ=z63T8HfQ_OyP=vX8BYiXyxFKt67w@mail.gmail.com
M contrib/pg_logicalinspect/pg_logicalinspect.c
Rename alloc/free functions in reorderbuffer.c
commit : ac4494646daee27df20f7cded3ad49be24e9a98c
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 12 Mar 2025 22:03:39 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 12 Mar 2025 22:03:39 +0200
There used to be bespoken pools for these structs to reduce the
palloc/pfree overhead, but that was ripped out a long time ago and
replaced with the generic, cheaper generational memory allocator
(commit a4ccc1cef5). The Get/Return terminology made sense with the
pools, as you "got" an object from the pool and "returned" it later,
but now it just looks weird. Rename to Alloc/Free.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/c9e43d2d-8e83-444f-b111-430377368989@iki.fi
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h
Remove count_one_bits() in acl.c.
commit : 025e7e1eb4b884a187ade19f2ed479b256306a82
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 12 Mar 2025 15:01:52 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 12 Mar 2025 15:01:52 -0500
The only caller, select_best_grantor(), can instead use
pg_popcount64(). This isn't performance-critical code, but we
might as well use the centralized implementation. While at it, add
some test coverage for this part of select_best_grantor().
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/Z9GtL7Nm6hsYyJnF%40nathan
M src/backend/utils/adt/acl.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Increase default effective_io_concurrency to 16
commit : ff79b5b2aba02d720f9b7fff644dd50ce07b8c6e
author : Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 12 Mar 2025 15:56:59 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 12 Mar 2025 15:56:59 -0400
The default effective_io_concurrency has been 1 since it was introduced
in b7b8f0b6096d2ab6e. Referencing the associated discussion [1], it
seems 1 was chosen as a conservative value that seemed unlikely to cause
regressions.
Experimentation on high latency cloud storage as well as fast, local
nvme storage (see Discussion link) shows that even slightly higher
values improve query timings substantially. 1 actually performs worse
than 0 [2]. With effective_io_concurrency 1, we are not prefetching
enough to avoid I/O stalls, but we are issuing extra syscalls.
The new default is 16, which should be more appropriate for common
hardware while still avoiding flooding low IOPs devices with I/O
requests.
[1] https://www.postgresql.org/message-id/flat/FDDBA24E-FF4D-4654-BA75-692B3BA71B97%40enterprisedb.com
[2] https://www.postgresql.org/message-id/CAAKRu_Zv08Cic%3DqdCfzrQabpEXGrd9Z9UOW5svEVkCM6%3DFXA9g%40mail.gmail.com
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CAAKRu_Z%2BJa-mwXebOoOERMMUMvJeRhzTjad4dSThxG0JLXESxw%40mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/utils/misc/postgresql.conf.sample
M src/include/storage/bufmgr.h
Handle interrupts while waiting on Append's async subplans
commit : af717317a04f5217728ce296edf4a581eb7e6ea0
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 12 Mar 2025 20:53:09 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 12 Mar 2025 20:53:09 +0200
We did not wake up on interrupts while waiting on async events on an
async-capable append node. For example, if you tried to cancel the
query, nothing would happen until one of the async subplans becomes
readable. To fix, add WL_LATCH_SET to the WaitEventSet.
Backpatch down to v14 where async Append execution was introduced.
Discussion: https://www.postgresql.org/message-id/37a40570-f558-40d3-b5ea-5c2079b3b30b@iki.fi
M src/backend/executor/nodeAppend.c
Build whole-row Vars the same way during parsing and planning.
commit : f4e7756ef9a437f30a4dc5ded41b8824a9d291b9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 Mar 2025 11:47:19 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 Mar 2025 11:47:19 -0400
makeWholeRowVar() has different rules for constructing a
whole-row Var depending on the kind of RTE it's representing.
This turns out to be problematic because the rewriter and planner
can convert view RTEs and set-returning-function RTEs into
subquery RTEs; so a whole-row Var made during planning might
look different from one made by the parser. In isolation this
doesn't cause any problem, but if a query contains Vars made
both ways for the same varno, there are cross-checks in the
executor that will complain. This manifests for UPDATE, DELETE,
and MERGE queries that use whole-row table references.
To fix, we need makeWholeRowVar() to produce the same result
from an inlined RTE as it would have for the original. For
an inlined view, we can use RangeTblEntry.relid to detect
that this had been a view RTE. For inlined SRFs, make a
data structure definition change akin to commit 47bb9db75,
and say that we won't clear RangeTblEntry.functions until
the end of planning. That allows makeWholeRowVar() to
repeat what it would have done with the unmodified RTE.
Reported-by: Duncan Sands <duncan.sands@deepbluecap.com>
Reported-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Diagnosed-by: Tender Wang <tndrwang@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Discussion: https://postgr.es/m/3518c50a-ab18-482f-b916-a37263622501@deepbluecap.com
Backpatch-through: 13
M src/backend/nodes/makefuncs.c
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/returning.out
M src/test/regress/sql/returning.sql
Add connection establishment duration logging
commit : 18cd15e706ac1f2d6b1c49847a82774ca143352f
author : Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 12 Mar 2025 11:33:08 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 12 Mar 2025 11:33:08 -0400
Add log_connections option 'setup_durations' which logs durations of
several key parts of connection establishment and backend setup.
For an incoming connection, starting from when the postmaster gets a
socket from accept() and ending when the forked child backend is first
ready for query, there are multiple steps that could each take longer
than expected due to external factors. This logging provides visibility
into authentication and fork duration as well as the end-to-end
connection establishment and backend initialization time.
To make this portable, the timings captured in the postmaster (socket
creation time, fork initiation time) are passed through the
BackendStartupData.
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Reviewed-by: Guillaume Lelarge <guillaume.lelarge@dalibo.com>
Discussion: https://postgr.es/m/flat/CAAKRu_b_smAHK0ZjrnL5GRxnAVWujEXQWpLXYzGbmpcZd3nLYw%40mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/postmaster/launch_backend.c
M src/backend/postmaster/postmaster.c
M src/backend/tcop/backend_startup.c
M src/backend/tcop/postgres.c
M src/backend/utils/init/postinit.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/miscadmin.h
M src/include/tcop/backend_startup.h
M src/include/utils/timestamp.h
M src/test/authentication/t/001_password.pl
M src/tools/pgindent/typedefs.list
Modularize log_connections output
commit : 9219093cab2607f34ac70612a65430a9c519157f
author : Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 12 Mar 2025 11:33:01 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 12 Mar 2025 11:33:01 -0400
Convert the boolean log_connections GUC into a list GUC comprised of the
connection aspects to log.
This gives users more control over the volume and kind of connection
logging.
The current log_connections options are 'receipt', 'authentication', and
'authorization'. The empty string disables all connection logging. 'all'
enables all available connection logging.
For backwards compatibility, the most common values for the
log_connections boolean are still supported (on, off, 1, 0, true, false,
yes, no). Note that previously supported substrings of on, off, true,
false, yes, and no are no longer supported.
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/flat/CAAKRu_b_smAHK0ZjrnL5GRxnAVWujEXQWpLXYzGbmpcZd3nLYw%40mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/libpq/auth.c
M src/backend/postmaster/postmaster.c
M src/backend/tcop/backend_startup.c
M src/backend/utils/init/postinit.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/postmaster/postmaster.h
M src/include/tcop/backend_startup.h
M src/include/utils/guc_hooks.h
M src/test/authentication/t/001_password.pl
M src/tools/pgindent/typedefs.list
Remove initialization from PendingBackendStats
commit : f554a95379a9adef233d21b1e1e8981a8f5f8de3
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 12 Mar 2025 20:37:43 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 12 Mar 2025 20:37:43 +0900
9a8dd2c5a6d has added an initialization to PendingBackendStats, which
has been causing compilation warnings in the buildfarm. This code does
not strictly require it as PendingBackendStats is always initialized
with memset(0), so let's remove it.
Per report from multiple buildfarm members, like ayu and batfish, via
Tom Lane.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/1870853.1741749264@sss.pgh.pa.us
M src/backend/utils/activity/pgstat_backend.c
Prepare for Python "Limited API" in PL/Python
commit : 72a3d0462b9a7f6265267950668af0c0246e7c01
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 12 Mar 2025 08:49:37 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 12 Mar 2025 08:49:37 +0100
Using the Python Limited API would allow building PL/Python against
any Python 3.x version and using another Python 3.x version at run
time. This commit does not activate that, but it prepares the code to
only use APIs supported by the Limited API.
Implementation details:
- Convert static types to heap types
(https://docs.python.org/3/howto/isolating-extensions.html#heap-types).
- Replace PyRun_String() with component functions.
- Replace PyList_SET_ITEM() with PyList_SetItem().
This was previously committed as c47e8df815c and then reverted because
it wasn't working under Python older than 3.8. That has been fixed in
this version. There was a Python API change/bugfix between 3.7 and
3.8 that directly affects this patch. The relevant commit is
<https://github.com/python/cpython/commit/364f0b0f19c>. The
workarounds described there have been applied in this patch, and it
has been confirmed to work with Python 3.6 and 3.7.
Reviewed-by: Jakob Egger <jakob@eggerapps.at>
Discussion: https://www.postgresql.org/message-id/flat/ee410de1-1e0b-4770-b125-eeefd4726a24@eisentraut.org
M src/pl/plpython/plpy_cursorobject.c
M src/pl/plpython/plpy_planobject.c
M src/pl/plpython/plpy_procedure.c
M src/pl/plpython/plpy_resultobject.c
M src/pl/plpython/plpy_subxactobject.c
M src/pl/plpython/plpy_typeio.c
Doc: silence A4 PDF build warnings.
commit : c872516d8fe5ba3ce27e7020fad887d90d308e29
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Mar 2025 23:35:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Mar 2025 23:35:39 -0400
Commit 0fbceae84 put a "&zwsp;" in almost but not quite the correct
place to avoid "The contents of fo:block line 1 exceed the available
area" warnings. Per buildfarm.
M doc/src/sgml/perform.sgml
Improve snapmgr.c comment
commit : 043745c3a01fece996e4818945138320d10261bf
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 11 Mar 2025 23:28:38 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 11 Mar 2025 23:28:38 +0200
Add more details on the different kinds of snapshots, how to use them,
and how the active snapshot stack works.
Discussion: https://www.postgresql.org/message-id/7c56f180-b9e1-481e-8c1d-efa63de3ecbb@iki.fi
M src/backend/utils/time/snapmgr.c
Assert that a snapshot is active or registered before it's used
commit : 8076c00592e40e8dbd1fce7a98b20d4bf075e4ba
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 11 Mar 2025 23:20:34 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 11 Mar 2025 23:20:34 +0200
The comment in GetTransactionSnapshot() said that you "should call
RegisterSnapshot or PushActiveSnapshot on the returned snap if it is
to be used very long". That felt too unclear to me. Make the comment
more strongly worded.
To enforce that rule and to catch potential bugs where a snapshot
might get invalidated while it's still in use, add an assertion to
HeapTupleSatisfiesMVCC() to check that the snapshot is registered or
pushed to active stack. No new bugs were found by this, but it seems
like good future-proofing. It's not a great place for the check;
HeapTupleSatisfiesMVCC() is in fact safe to call with an unregistered
snapshot, and the assertion won't catch other unsafe uses. But it goes
a long way in practice.
Fix a few cases that were playing fast and loose with that and just
assumed that the snapshot cannot be invalidated during a scan. Those
assumptions were not wrong, but they're not performance critical, so
let's drop the excuses and just register the snapshot. These were
false positives found by the new assertion.
Discussion: https://www.postgresql.org/message-id/7c56f180-b9e1-481e-8c1d-efa63de3ecbb@iki.fi
M src/backend/access/heap/heapam_visibility.c
M src/backend/access/index/genam.c
M src/backend/commands/dbcommands.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/time/snapmgr.c
pg_logicalinspect: Fix possible crash when passing a directory path.
commit : bd65cb3cd48a7a5ce48b26f8031ad3968efed87e
author : Masahiko Sawada <msawada@postgresql.org>
date : Tue, 11 Mar 2025 09:56:40 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Tue, 11 Mar 2025 09:56:40 -0700
Previously, pg_logicalinspect functions were too trusting of their
input and blindly passed it to SnapBuildRestoreSnapshot(). If the
input pointed to a directory, the server could a PANIC error while
attempting to fsync_fname() with isdir=false on a directory.
This commit adds validation checks for input filenames and passes the
LSN extracted from the filename to SnapBuildRestoreSnapshot() instead
of the filename itself. It also adds regression tests for various
input patterns and permission checks.
Bug: #18828
Reported-by: Robins Tharakan <tharakan@gmail.com>
Co-authored-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Co-authored-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/18828-0f4701c635064211@postgresql.org
M contrib/pg_logicalinspect/Makefile
A contrib/pg_logicalinspect/expected/pg_logicalinspect.out
M contrib/pg_logicalinspect/pg_logicalinspect.c
A contrib/pg_logicalinspect/sql/pg_logicalinspect.sql
M src/backend/replication/logical/snapbuild.c
M src/include/replication/snapbuild_internal.h
pg_logicalinspect: Stabilize isolation tests.
commit : a49927f04cd0c66e1bc41f551160fce50f382d8c
author : Masahiko Sawada <msawada@postgresql.org>
date : Tue, 11 Mar 2025 09:30:00 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Tue, 11 Mar 2025 09:30:00 -0700
The previous isolation tests did not account for the possibility that
the background writer or the checkpointer could write a RUNNING_XACTS
record, which could cause logical decoding to produce more logical
snapshots than expected.
This commit modifies the isolation tests to verify that at least one
logical snapshot contains the expected number of committed or ongoing
catalog-change transactions.
Per buildfarm member skink.
Reported-by: Andres Freund <andres@anarazel.de>
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/5qbxud4pvnvmtuoi7weiizm5hmumxaeohx4vztfhrwlfhyz6rj@buh4435mllwo
M contrib/pg_logicalinspect/expected/logical_inspect.out
M contrib/pg_logicalinspect/specs/logical_inspect.spec
Improve EXPLAIN's display of window functions.
commit : 8b1b342544b69b281ffd3aafe594aec629ec4d3c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Mar 2025 11:19:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Mar 2025 11:19:54 -0400
Up to now we just punted on showing the window definitions used
in a plan, with window function calls represented as "OVER (?)".
To improve that, show the window definition implemented by each
WindowAgg plan node, and reference their window names in OVER.
For nameless window clauses generated by "OVER (...)", assign
unique names w1, w2, etc.
In passing, re-order the properties shown for a WindowAgg node
so that the Run Condition (if any) appears after the Window
property and before the Filter (if any). This seems more
sensible since the Run Condition is associated with the Window
and acts before the Filter.
Thanks to David G. Johnston and Álvaro Herrera for design
suggestions.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/144530.1741469955@sss.pgh.pa.us
M contrib/postgres_fdw/expected/postgres_fdw.out
M src/backend/commands/explain.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/planner.c
M src/backend/utils/adt/ruleutils.c
M src/include/nodes/plannodes.h
M src/include/utils/ruleutils.h
M src/test/regress/expected/box.out
M src/test/regress/expected/create_index_spgist.out
M src/test/regress/expected/explain.out
M src/test/regress/expected/generated_virtual.out
M src/test/regress/expected/groupingsets.out
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/polygon.out
M src/test/regress/expected/select_parallel.out
M src/test/regress/expected/sqljson.out
M src/test/regress/expected/window.out
M src/test/regress/sql/explain.sql
nbtree: Make BTMaxItemSize into object-like macro.
commit : 426ea611171da4e60ab4f3863fa3cc3683ae9547
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 11 Mar 2025 10:35:56 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 11 Mar 2025 10:35:56 -0400
Make nbtree's "1/3 of a page limit" BTMaxItemSize function-like macro
(which accepts a "page" argument) into an object-like macro that can be
used from code that doesn't have convenient access to an nbtree page.
Preparation for an upcoming patch that adds skip scan to nbtree.
Parallel index scans that use skip scan will serialize datums (not just
SAOP array subscripts) when scheduling primitive scans. BTMaxItemSize
will be used by btestimateparallelscan to determine how much DSM to
request.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-Wz=H_RG5weNGeUG_TkK87tRBnH9mGCQj6WpM4V4FNWKv2g@mail.gmail.com
M contrib/amcheck/verify_nbtree.c
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/access/nbtree/nbtxlog.c
M src/include/access/nbtree.h
Show index search count in EXPLAIN ANALYZE, take 2.
commit : 0fbceae841cb5a31b13d3f284ac8fdd19822eceb
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 11 Mar 2025 09:20:50 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 11 Mar 2025 09:20:50 -0400
Expose the count of index searches/index descents in EXPLAIN ANALYZE's
output for index scan/index-only scan/bitmap index scan nodes. This
information is particularly useful with scans that use ScalarArrayOp
quals, where the number of index searches can be unpredictable due to
implementation details that interact with physical index characteristics
(at least with nbtree SAOP scans, since Postgres 17 commit 5bf748b8).
The information shown also provides useful context when EXPLAIN ANALYZE
runs a plan with an index scan node that successfully applied the skip
scan optimization (set to be added to nbtree by an upcoming patch).
The instrumentation works by teaching all index AMs to increment a new
nsearches counter whenever a new index search begins. The counter is
incremented at exactly the same point that index AMs already increment
the pg_stat_*_indexes.idx_scan counter (we're counting the same event,
but at the scan level rather than the relation level). Parallel queries
have workers copy their local counter struct into shared memory when an
index scan node ends -- even when it isn't a parallel aware scan node.
An earlier version of this patch that only worked with parallel aware
scans became commit 5ead85fb (though that was quickly reverted by commit
d00107cd following "debug_parallel_query=regress" buildfarm failures).
Our approach doesn't match the approach used when tracking other index
scan related costs (e.g., "Rows Removed by Filter:"). It is comparable
to the approach used in similar cases involving costs that are only
readily accessible inside an access method, not from the executor proper
(e.g., "Heap Blocks:" output for a Bitmap Heap Scan, which was recently
enhanced to show per-worker costs by commit 5a1e6df3, using essentially
the same scheme as the one used here). It is necessary for index AMs to
have direct responsibility for maintaining the new counter, since the
counter might need to be incremented multiple times per amgettuple call
(or per amgetbitmap call). But it is also necessary for the executor
proper to manage the shared memory now used to transfer each worker's
counter struct to the leader.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Robert Haas <robertmhaas@gmail.com>
Reviewed-By: Tomas Vondra <tomas@vondra.me>
Reviewed-By: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAH2-WzkRqvaqR2CTNqTZP0z6FuL4-3ED6eQB0yx38XBNj1v-4Q@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-Wz=PKR6rB7qbx+Vnd7eqeB5VTcrW=iJvAsTsKbdG+kW_UA@mail.gmail.com
M contrib/bloom/blscan.c
M doc/src/sgml/bloom.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/ref/explain.sgml
M doc/src/sgml/rules.sgml
M src/backend/access/brin/brin.c
M src/backend/access/gin/ginscan.c
M src/backend/access/gist/gistget.c
M src/backend/access/hash/hashsearch.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/index/genam.c
M src/backend/access/index/indexam.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/spgist/spgscan.c
M src/backend/commands/explain.c
M src/backend/executor/execIndexing.c
M src/backend/executor/execParallel.c
M src/backend/executor/execReplication.c
M src/backend/executor/nodeBitmapIndexscan.c
M src/backend/executor/nodeIndexonlyscan.c
M src/backend/executor/nodeIndexscan.c
M src/backend/utils/adt/selfuncs.c
M src/include/access/genam.h
M src/include/access/relscan.h
M src/include/executor/nodeBitmapIndexscan.h
M src/include/executor/nodeIndexonlyscan.h
M src/include/executor/nodeIndexscan.h
M src/include/nodes/execnodes.h
M src/test/regress/expected/brin_multi.out
M src/test/regress/expected/memoize.out
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/select.out
M src/test/regress/sql/memoize.sql
M src/test/regress/sql/partition_prune.sql
M src/tools/pgindent/typedefs.list
Update nls.mk for newly added file
commit : 12c5f797ea6a8e96de661e3838410b9775061796
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 Mar 2025 13:42:03 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 Mar 2025 13:42:03 +0100
Commit f18231e8175 moved some code to a new file, but the new file
wasn't added to nls.mk.
M src/interfaces/ecpg/preproc/nls.mk
BRIN: be more strict about required support procs
commit : 17ce344f86f1a9a9002d8b668ea7d77234ec3e60
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 11 Mar 2025 12:50:35 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 11 Mar 2025 12:50:35 +0100
With improperly defined operator classes, it's possible to get a
Postgres crash because we'd try to invoke a procedure that doesn't
exist. This is because the code is being a bit too trusting that the
opclass is correctly defined. Add some ereport(ERROR)s for cases where
mandatory support procedures are not defined, transforming the crashes
into errors.
The particular case that was reported is an incomplete opclass in
PostGIS.
Backpatch all the way down to 13.
Reported-by: Tobias Wendorff <tobias.wendorff@tu-dortmund.de>
Diagnosed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/fb6d9a35-6c8e-4869-af80-0a4944a793a4@tu-dortmund.de
M src/backend/access/brin/brin_bloom.c
M src/backend/access/brin/brin_inclusion.c
M src/backend/access/brin/brin_minmax_multi.c
Add special case fast-paths for strict functions
commit : d35d32d7112bc632c6a305e9dffdec0082bbdf00
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 11 Mar 2025 12:02:42 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 11 Mar 2025 12:02:42 +0100
Many STRICT function calls will have one or two arguments, in which
case we can speed up checking for NULL input by avoiding setting up
a loop over the arguments. This adds EEOP_FUNCEXPR_STRICT_1 and the
corresponding EEOP_FUNCEXPR_STRICT_2 for functions with one and two
arguments respectively.
Author: Andres Freund <andres@anarazel.de>
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Andreas Karlsson <andreas@proxel.se>
Discussion: https://postgr.es/m/415721CE-7D2E-4B74-B5D9-1950083BA03E@yesql.se
Discussion: https://postgr.es/m/20191023163849.sosqbfs5yenocez3@alap3.anarazel.de
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/include/executor/execExpr.h
Replace EEOP_DONE with special steps for return/no return
commit : 8dd7c7cd0a2605d5301266a6b67a569d6a305106
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 11 Mar 2025 12:02:38 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 11 Mar 2025 12:02:38 +0100
Knowing when the side-effects of an expression is the intended result
of the execution, rather than the returnvalue, is important for being
able generate more efficient JITed code. This replaces EEOP_DONE with
two new steps: EEOP_DONE_RETURN and EEOP_DONE_NO_RETURN. Expressions
which return a value should use the former step; expressions used for
their side-effects which don't return value should use the latter.
Author: Andres Freund <andres@anarazel.de>
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Andreas Karlsson <andreas@proxel.se>
Discussion: https://postgr.es/m/415721CE-7D2E-4B74-B5D9-1950083BA03E@yesql.se
Discussion: https://postgr.es/m/20191023163849.sosqbfs5yenocez3@alap3.anarazel.de
M src/backend/executor/README
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/nodeAgg.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/include/executor/execExpr.h
M src/include/executor/executor.h
Move RemoveInheritedConstraint() call slightly earlier
commit : dabccf45139a8c7c3c2e7683a943c31077e55a78
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 Mar 2025 10:43:48 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 Mar 2025 10:43:48 +0100
This change is harmless and does not affect the existing intended
operation. It is necessary for a subsequent patch operation (NOT
ENFORCED foreign keys), where we may need to change the child
constraint to enforced. In this case, we would create the necessary
triggers and queue the constraint for validation, so it is important
to remove any unnecessary constraints before proceeding.
This is a small change that could have been included in the previous
"split tryAttachPartitionForeignKey" refactoring patch (commit
1d26c2d2c4b), but was kept separate to highlight the changes.
Author: Amul Sul <amul.sul@enterprisedb.com>
Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAAJ_b962c5AcYW9KUt_R_ER5qs3fUGbe4az-SP-vuwPS-w-AGA%40mail.gmail.com
M src/backend/commands/tablecmds.c
refactor: Split tryAttachPartitionForeignKey()
commit : 1d26c2d2c4b8d833a63ba403bc9313db2c091aa7
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 Mar 2025 09:33:36 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 Mar 2025 09:33:36 +0100
Split tryAttachPartitionForeignKey() into three functions:
AttachPartitionForeignKey(), RemoveInheritedConstraint(), and
DropForeignKeyConstraintTriggers(), so they can be reused in some
subsequent patches for the NOT ENFORCED feature.
Author: Amul Sul <amul.sul@enterprisedb.com>
Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAAJ_b962c5AcYW9KUt_R_ER5qs3fUGbe4az-SP-vuwPS-w-AGA%40mail.gmail.com
M src/backend/commands/tablecmds.c
refactor: re-add ATExecAlterChildConstr()
commit : 64224a834ce4c899e878db27b14ea3412a6ddbed
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 Mar 2025 08:40:42 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 Mar 2025 08:40:42 +0100
ATExecAlterChildConstr() was removed in commit 80d7f990496, but it is
needed in some subsequent patches for the NOT ENFORCED feature, to
recurse over child constraints. This adds it back in slightly altered
form.
Author: Amul Sul <amul.sul@enterprisedb.com>
Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAAJ_b962c5AcYW9KUt_R_ER5qs3fUGbe4az-SP-vuwPS-w-AGA%40mail.gmail.com
M src/backend/commands/tablecmds.c
Add WAL data to backend statistics
commit : 76def4cdd7c2b32d19e950a160f834392ea51744
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 11 Mar 2025 09:04:11 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 11 Mar 2025 09:04:11 +0900
This commit adds per-backend WAL statistics, providing the same
information as pg_stat_wal, except that it is now possible to know how
much WAL activity is happening in each backend rather than an overall
aggregate of all the activity. Like pg_stat_wal, the implementation
relies on pgWalUsage, tracking the difference of activity between two
reports to pgstats.
This data can be retrieved with a new system function called
pg_stat_get_backend_wal(), that returns one tuple based on the PID
provided in input. Like pg_stat_get_backend_io(), this is useful when
joined with pg_stat_activity to get a live picture of the WAL generated
for each running backend, showing how the activity is [un]balanced.
pgstat_flush_backend() gains a new flag value, able to control the flush
of the WAL stats.
This commit relies mostly on the infrastructure provided by
9aea73fc61d4, that has introduced backend statistics.
Bump catalog version. A bump of PGSTAT_FILE_FORMAT_ID is not required,
as backend stats do not persist on disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/Z3zqc4o09dM/Ezyz@ip-10-97-1-34.eu-west-3.compute.internal
M doc/src/sgml/monitoring.sgml
M src/backend/utils/activity/pgstat_backend.c
M src/backend/utils/activity/pgstat_wal.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/utils/pgstat_internal.h
M src/test/regress/expected/stats.out
M src/test/regress/sql/stats.sql
tests: Make postmaster/002_connection_limits deal verbose logs
commit : 59a1592e39fed164d4d6437d62a55aef36e866a5
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Mar 2025 19:11:32 -0400
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Mar 2025 19:11:32 -0400
When log_error_verbosity=verbose is configured the test would hand (and then
fail), because of the sqlstate being added between log level and message. Make
regex cope.
Reported-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/c7ba6bd0-3701-43d1-9087-017777fe9cd2%40dunslane.net
M src/test/postmaster/t/002_connection_limits.pl
CREATE INDEX: do update index stats if autovacuum=off.
commit : 29d6808edebb3eced76e2acbbcf453dd693ac6bf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Mar 2025 17:49:27 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Mar 2025 17:49:27 -0400
This fixes a thinko from commit d611f8b15. The intent was to prevent
updating the stats of the pre-existing heap if autovacuum is off,
but it also disabled updating the stats of the just-created index.
There is AFAICS no good reason to do the latter, since there could not
be any pre-existing stats to refrain from overwriting, and the zeroed
stats that are there to begin with are very unlikely to be useful.
Moreover, the change broke our cross-version upgrade tests again.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/1116282.1741374848@sss.pgh.pa.us
M src/backend/catalog/index.c
Fix a few more redundant calls of GetLatestSnapshot()
commit : f7c566a1a276c5f228891c7599df1c0b8878eb35
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 10 Mar 2025 18:54:58 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 10 Mar 2025 18:54:58 +0200
Commit 2367503177 fixed this in RelationFindReplTupleByIndex(), but I
missed two other similar cases.
Per report from Ranier Vilela.
Discussion: https://www.postgresql.org/message-id/CAEudQArUT1dE45WN87F-Gb7XMy_hW6x1DFd3sqdhhxP-RMDa0Q@mail.gmail.com
Backpatch-through: 13
M src/backend/executor/execReplication.c
Fix snapshot used in logical replication index lookup
commit : 23675031774c1644c32ff052a1c3e9fb87261023
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 10 Mar 2025 17:07:38 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 10 Mar 2025 17:07:38 +0200
The function calls GetLatestSnapshot() to acquire a fresh snapshot,
makes it active, and was meant to pass it to table_tuple_lock(), but
instead called GetLatestSnapshot() again to acquire yet another
snapshot. It was harmless because the heap AM and all other known
table AMs ignore the 'snapshot' argument anyway, but let's be tidy.
In the long run, this perhaps should be redesigned so that snapshot
was not needed in the first place. The table AM API uses TID +
snapshot as the unique identifier for the row version, which is
questionable when the row came from an index scan with a Dirty
snapshot. You might lock a different row version when you use a
different snapshot in the table_tuple_lock() call (a fresh MVCC
snapshot) than in the index scan (DirtySnapshot). However, in the heap
AM and other AMs where the TID alone identifies the row version, it
doesn't matter. So for now, just fix the obvious albeit harmless bug.
This has been wrong ever since the table AM API was introduced in
commit 5db6df0c01, so backpatch to all supported versions.
Discussion: https://www.postgresql.org/message-id/83d243d6-ad8d-4307-8b51-2ee5844f6230@iki.fi
Backpatch-through: 13
M src/backend/executor/execReplication.c
Doc: improve description of window function processing.
commit : 9f87e2593febf0ff024c11b9a97703bce0eee7be
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Mar 2025 10:22:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Mar 2025 10:22:08 -0400
The previous wording talked about a "single pass over the data",
which can be read as promising more than intended (to wit, that only
one WindowAgg plan node will be used). What we promise is only what
the SQL spec requires, namely that the data not get re-sorted between
window functions with compatible PARTITION BY/ORDER BY clauses.
Adjust the wording in hopes of making this clearer.
Reported-by: Christopher Inokuchi <cinokuchi@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/CABde6B5va2wMsnM79u_x=n9KUgfKQje_pbLROEBmA9Ru5XWidw@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/queries.sgml
Use extended stats for precise estimation of bucket size in hash join
commit : 6bb6a62f3cc45624c601d5270673a17447734629
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 10 Mar 2025 13:42:00 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 10 Mar 2025 13:42:00 +0200
Recognizing the real-life complexity where columns in the table often have
functional dependencies, PostgreSQL's estimation of the number of distinct
values over a set of columns can be underestimated (or much rarely,
overestimated) when dealing with multi-clause JOIN. In the case of hash
join, it can end up with a small number of predicted hash buckets and, as
a result, picking non-optimal merge join.
To improve the situation, we introduce one additional stage of bucket size
estimation - having two or more join clauses estimator lookup for extended
statistics and use it for multicolumn estimation. Clauses are grouped into
lists, each containing expressions referencing the same relation. The result
of the multicolumn estimation made over such a list is combined with others
according to the caller's logic. Clauses that are not estimated are returned
to the caller for further estimation.
Discussion: https://postgr.es/m/52257607-57f6-850d-399a-ec33a654457b%40postgrespro.ru
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Andy Fan <zhihui.fan1213@gmail.com>
Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Reviewed-by: Alena Rybakina <lena.ribackina@yandex.ru>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
M src/backend/optimizer/path/costsize.c
M src/backend/utils/adt/selfuncs.c
M src/include/utils/selfuncs.h
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Teach Append to consider tuple_fraction when accumulating subpaths.
commit : fae535da0ac2a8d0bb279cc66d62b0dcc4b5409b
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 10 Mar 2025 13:38:39 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 10 Mar 2025 13:38:39 +0200
This change is dedicated to more active usage of IndexScan and parameterized
NestLoop paths in partitioned cases under an Append node, as it already works
with plain tables. As newly added regression tests demonstrate, it should
provide more smartness to the partitionwise technique.
With an indication of how many tuples are needed, it may be more meaningful
to use the 'fractional branch' subpaths of the Append path list, which are
more optimal for this specific number of tuples. Planning on a higher level,
if the optimizer needs all the tuples, it will choose non-fractional paths.
In the case when, during execution, Append needs to return fewer tuples than
declared by tuple_fraction, it would not be harmful to use the 'intermediate'
variant of paths. However, it will earn a considerable profit if a sensible
set of tuples is selected.
The change of the existing regression test demonstrates the positive outcome
of this feature: instead of scanning the whole table, the optimizer prefers
to use a parameterized scan, being aware of the only single tuple the join
has to produce to perform the query.
Discussion: https://www.postgresql.org/message-id/flat/CAN-LCVPxnWB39CUBTgOQ9O7Dd8DrA_tpT1EY3LNVnUuvAX1NjA%40mail.gmail.com
Author: Nikita Malakhov <hukutoc@gmail.com>
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Andy Fan <zhihuifan1213@163.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/plan/planner.c
M src/test/regress/expected/partition_join.out
M src/test/regress/expected/union.out
M src/test/regress/sql/partition_join.sql
Remove support for temporal RESTRICT foreign keys
commit : b83e8a2ca2eb381ea0a48e5b2d4e4cdb74febc45
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 Mar 2025 11:29:54 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 Mar 2025 11:29:54 +0100
It isn't clear how these should behave, so let's wait to implement them
until we are sure how to do it.
This feature was initially added by commit 89f908a6d0a, so it hasn't
been released yet.
Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://postgr.es/m/e773bc11-4ac1-40de-bb91-814e02f05b6d%40eisentraut.org
M doc/src/sgml/ref/create_table.sgml
M src/backend/commands/tablecmds.c
M src/test/regress/expected/without_overlaps.out
M src/test/regress/sql/without_overlaps.sql
Fix incorrect #endif comment
commit : e033696596566d422a0eae47adca371a210ed730
author : David Rowley <drowley@postgresql.org>
date : Mon, 10 Mar 2025 13:36:04 +1300
committer: David Rowley <drowley@postgresql.org>
date : Mon, 10 Mar 2025 13:36:04 +1300
Noticed while reading code in this area.
M src/include/partitioning/partdesc.h
Fix incorrect assertion in libpqwalreceiver
commit : 03f8e9a7fee0ad4d87bfe48cb22dbca08fd8e6c4
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sun, 9 Mar 2025 20:40:45 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sun, 9 Mar 2025 20:40:45 +0200
Was supposed to check the length of the array, but was checking its
size in bytes.
Author: Jacob Brazeal <jacob.brazeal@gmail.com>
Discussion: https://www.postgresql.org/message-id/CA%2BCOZaA_9afJxj9ZuO73U5P7WXP%2BZM9NGnZvTDCmBFz0FGP%2BwA@mail.gmail.com
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
Fix test name and username used in failed connection attempts
commit : 2a943afcff444bade64ad5209f8b248d8d449383
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sun, 9 Mar 2025 19:47:55 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sun, 9 Mar 2025 19:47:55 +0200
The first failed connection tests the "regular" connections limit, not
the reserved limit.
In the second failed connection, the username doesn't really matter,
but since the previous successful connections used "regress_reserved",
it seems weird to switch back to "regress_regular" for the
expected-to-fail attempt.
Discussion: https://www.postgresql.org/message-id/fd5e9523-78d3-4270-86b2-fd1b1eeb4fc9@iki.fi
M src/test/postmaster/t/002_connection_limits.pl
Don't try to parallelize array_agg() on an anonymous record type.
commit : fedfcf66506fa9bf2259a88ae711e7ad7bde0011
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 Mar 2025 13:11:20 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 Mar 2025 13:11:20 -0400
This doesn't work because record_recv requires the typmod that
identifies the specific record type (in our session) and
array_agg_deserialize has no convenient way to get that information.
The result is an "input of anonymous composite types is not
implemented" error.
We could probably make this work if we had to, but it does not seem
worth the trouble, given that it took this long to get a field report.
Just shut off parallelization, as though record_recv didn't exist.
Oversight in commit 16fd03e95. Back-patch to v16 where that
came in.
Reported-by: Kirill Zdornyy <kirill@dineserve.com>
Diagnosed-by: Richard Guo <guofenglinux@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/atLI5Kce2ie1zcYjU0w_kjtVaxiYbYGTihrkLDmGZQnRDD4pnXukIATaABbnIj9pUnelC4ESvCXMm4HAyHg-v61XABaKpERj0A2IXzJZM7g=@dineserve.com
Backpatch-through: 16
M src/backend/parser/parse_agg.c
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql
doc: Adjust note about pg_upgrade's --jobs option.
commit : 3c472a18296e473270b6ff611b898592263a6ed1
author : Nathan Bossart <nathan@postgresql.org>
date : Sat, 8 Mar 2025 14:28:16 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Sat, 8 Mar 2025 14:28:16 -0600
Presently, this section lists a couple of parallelized parts of
pg_upgrade and suggests a starting point for setting the --jobs
option. The list of parallelized tasks is not particularly
actionable, and the phrasing for the --jobs recommendation is
confusing to some readers.
This commit attempts to improve this section by eliminating the
list of parallelized tasks and instead highlighting that --jobs is
most useful for clusters with multiple databases or tablespaces.
Additionally, the recommendation for setting --jobs is simplified
to suggest starting with the number of CPU cores.
Reported-by: Magnus Hagander <magnus@hagander.net>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Magnus Hagander <magnus@hagander.net>
Discussion: https://postgr.es/m/Z8dBn_5iGLNuYiPo%40nathan
M doc/src/sgml/ref/pgupgrade.sgml
Don't convert to and from floats in pg_dump.
commit : 1852aea3f526b124b95585f7b1f655f3af04afd5
author : Jeff Davis <jdavis@postgresql.org>
date : Sat, 8 Mar 2025 11:25:36 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Sat, 8 Mar 2025 11:25:36 -0800
Commit 8f427187db improved performance by remembering relation stats
as native types rather than issuing a new query for each relation.
Using native types is fine for integers like relpages; but reltuples
is floating point. The commit controllled for that complexity by using
setlocale(LC_NUMERIC, "C"). After that, Alexander Lakhin found a
problem in pg_strtof(), fixed in 00d61a08c5.
While we aren't aware of any more problems with that approach, it
seems wise to just use a string the whole way for floating point
values, as Corey's original patch did, and get rid of the
setlocale(). Integers are still converted to native types to avoid
wasting memory.
Co-authored-by: Corey Huinker <corey.huinker@gmail.com>
Discussion: https://postgr.es/m/3049348.1740855411@sss.pgh.pa.us
Discussion: https://postgr.es/m/560cca3781740bd69881bb07e26eb8f65b09792c.camel%40j-davis.com
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
Clear errno before calling strtol() in spell.c.
commit : 7fb880102138e45f58bf626cb2a4599bade8b172
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 8 Mar 2025 11:24:22 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 8 Mar 2025 11:24:22 -0500
Per POSIX, a caller of strtol() that wishes to check for errors must
set errno to 0 beforehand. Several places in spell.c neglected that,
so that they risked delivering a false overflow error in case errno
had been ERANGE already. Given the lack of field reports, this case
may be unreachable at present --- but it's surely trouble waiting to
happen, so fix it.
Author: Jacob Brazeal <jacob.brazeal@gmail.com>
Discussion: https://postgr.es/m/CA+COZaBhsq6EromFm+knMJfzK6nTpG23zJ+K2=nfUQQXcj_xcQ@mail.gmail.com
Backpatch-through: 13
M src/backend/tsearch/spell.c
Make parallel nbtree index scans use an LWLock.
commit : 67fc4c9fd7fab7004b656e0cc27826c75d7ea7ad
author : Peter Geoghegan <pg@bowt.ie>
date : Sat, 8 Mar 2025 11:10:14 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Sat, 8 Mar 2025 11:10:14 -0500
Teach parallel nbtree index scans to use an LWLock (not a spinlock) to
protect the scan's shared descriptor state.
Preparation for an upcoming patch that will add skip scan optimizations
to nbtree. That patch will create the need to occasionally allocate
memory while the scan descriptor is locked, while copying datums that
were serialized by another backend.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wz=PKR6rB7qbx+Vnd7eqeB5VTcrW=iJvAsTsKbdG+kW_UA@mail.gmail.com
M src/backend/access/nbtree/nbtpreprocesskeys.c
M src/backend/access/nbtree/nbtree.c
M src/backend/storage/lmgr/lwlock.c
M src/backend/utils/activity/wait_event_names.txt
M src/include/storage/lwlock.h
Make amcanorder independent of amconsistentordering
commit : 8021c77769e90cc804121d61a1bb7bcc4652d48b
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 8 Mar 2025 09:37:06 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 8 Mar 2025 09:37:06 +0100
Follow-up to commit af4002b381d: Make amconsistentordering not depend
on amcanorder. Although they are related, they are independent
properties.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/E1tngY6-0000UL-2n%40gemulon.postgresql.org
M src/backend/utils/cache/lsyscache.c
Fix typo
commit : 661781f3a3ed37249cd468db27050ab3d4d662b7
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 8 Mar 2025 08:06:30 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 8 Mar 2025 08:06:30 +0100
Duplicate assignment in commit af4002b381d should have been a
different field. (But it didn't affect the outcome.)
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/E1tngY6-0000UL-2n%40gemulon.postgresql.org
M src/backend/access/hash/hash.c
Use stricter ordering in regression test query for pg_stat_io
commit : 21f653cc0024100f8ecc279162631f2b1ba8c46c
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 8 Mar 2025 13:39:57 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 8 Mar 2025 13:39:57 +0900
The query introduced in 8b532771a099 is proving to have ordering issues
under at least the locale cs_CZ. This commit updates the query to use a
stricter ordering.
Per reports from buildfarm members hippopotamus and jay.
M src/test/regress/expected/stats.out
M src/test/regress/sql/stats.sql
Add regression test listing all the possible tuples in pg_stat_io
commit : 8b532771a09975f274b35823aa355d8ca90c77db
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 8 Mar 2025 12:22:41 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 8 Mar 2025 12:22:41 +0900
pg_stat_io returns a set of tuples based on a combination of three
properties (BackendType, IOObject and IOContext) and
pgstat_tracks_io_object() to decide if a BackendType should return a
tuple based on a pair made of an IOObject and an IOContext.
This commit adds a regression test to track all the combinations
supported. This is useful to know which tuples are relevant when adding
a new BackendType to the set or when touching pgstat_tracks_io_object(),
and I have noticed while playing with this area that it is not
complicated to break it without the regression tests noticing a
difference in some cases.
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/Z8exfAehbVbEKXW5@paquier.xyz
M src/test/regress/expected/stats.out
M src/test/regress/sql/stats.sql
Improve check for detection of pending data in backend statistics
commit : 9a8dd2c5a6d9d2538444c156e6b273b1b4f4e3a4
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 8 Mar 2025 10:56:30 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 8 Mar 2025 10:56:30 +0900
The callback pgstat_backend_have_pending_cb() is used as a way for
pg_stat_report() to detect if there is any pending data for backend
statistics.
It did not include a check based on pgstat_tracks_backend_bktype(), that
discards processes whose backend types do not support backend
statistics. The logic is not a problem on HEAD, as processes that do
not support backend statistics cannot touch PendingBackendStats, so the
callback would always report that there is no pending data in this case.
However, we would run into trouble once backend statistics include
portions of pending stats that are not always zeroed, like pgWalUsage.
There is no reason for pgstat_backend_have_pending_cb() to not check
for pgstat_tracks_backend_bktype(), anyway, and this pattern is safer in
the long run, so let's update the code to do so.
While on it, this commit adds a proper initialization to
PendingBackendStats.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Co-authored-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/Z8l6EMM4ImVoWRkg@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/utils/activity/pgstat_backend.c
nbtree: refine _bt_readnextpage contract comments.
commit : 8e167e618893b59ee45317c43055002ba71f955b
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 7 Mar 2025 18:35:13 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 7 Mar 2025 18:35:13 -0500
Another minor follow-up commit for commit 1bd4bc85, which changed the
_bt_readnextpage contract.
M src/backend/access/nbtree/nbtsearch.c
Assert that wrapper_handler()'s argument is within expected range.
commit : 088f8e2d568eaa963fe68a15287002081314fad8
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 7 Mar 2025 15:23:09 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 7 Mar 2025 15:23:09 -0600
pqsignal() already does a similar check, but strange Valgrind
reports have us wondering if wrapper_handler() is somehow getting
called with an invalid signal number.
Reported-by: Tomas Vondra <tomas@vondra.me>
Suggested-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/ace01111-f9ac-4f61-b1b1-8e9379415444%40vondra.me
Backpatch-through: 17
M src/port/pqsignal.c
Include column name in build_attrmap_by_position's error reports.
commit : 34c3c5ce1c0b2a27de212076c2e5aaa8e82fac31
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Mar 2025 13:24:09 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Mar 2025 13:24:09 -0500
Formerly we only provided the column number, but it's frequently
more useful to mention the column name. The input tupdesc often
doesn't have useful column names, but the output tupdesc usually
contains user-supplied names, so report that one.
Author: Marcos Pegoraro <marcos@f10.com.br>
Co-authored-by: jian he <jian.universality@gmail.com>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Co-authored-by: Erik Wienhold <ewie@ewie.name>
Reviewed-by: Vladlen Popolitov <v.popolitov@postgrespro.ru>
Discussion: https://postgr.es/m/CAB-JLwanky28gjAMdnMh1CjyO1b2zLdr6UOA1-oY9G7PVL9KKQ@mail.gmail.com
M src/backend/access/common/attmap.c
M src/pl/plpgsql/src/expected/plpgsql_record.out
M src/pl/plpgsql/src/sql/plpgsql_record.sql
M src/test/regress/expected/plpgsql.out
tests: Don't fail due to high default timeout in postmaster/003_start_stop
commit : b48832cddbf57a3428f1a553e395f1870e1b0cb5
author : Andres Freund <andres@anarazel.de>
date : Fri, 7 Mar 2025 13:09:16 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 7 Mar 2025 13:09:16 -0500
Some BF animals use very high timeouts due to their slowness. Unfortunately
postmaster/003_start_stop fails if a high timeout is configured, due to
authentication_timeout having a fairly low max.
As this test is reasonably fast, the easiest fix seems to be to cap the
timeout to 600.
Per buildfarm animal skink.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/ggflhkciwdyotpoie323chu2c2idpjk5qimrn462encwx2io7s@thmcxl7i6dpw
M src/test/postmaster/t/003_start_stop.pl
tests: Fix race condition in postmaster/002_connection_limits
commit : 71d1ed6fe129935ce2764c5d34924512ed046f69
author : Andres Freund <andres@anarazel.de>
date : Fri, 7 Mar 2025 13:09:16 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 7 Mar 2025 13:09:16 -0500
The test occasionally failed due to unexpected connection limit errors being
encountered after having waited for FATAL errors on another connection. These
spurious failures were caused by the the backend reporting FATAL errors to the
client before detaching from the PGPROC entry. Adding a sleep(1) before
proc_exit() makes it easy to reproduce that problem.
To fix the issue, add a helper function that waits for postmaster to notice
the process having exited. For now this is implemented by waiting for the
DEBUG2 message that postmaster logs in that case. That's not the prettiest
fix, but simple. If we notice this problem elsewhere, it might be worthwhile
to make this more general, e.g. by adding an injection point.
Reported-by: Tomas Vondra <tomas@vondra.me>
Diagnosed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Tested-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/ggflhkciwdyotpoie323chu2c2idpjk5qimrn462encwx2io7s@thmcxl7i6dpw
M src/test/postmaster/t/002_connection_limits.pl
doc: Add missing decimal places to example rowcount.
commit : d3fc7a51208b3f4f2be2476d44aa2542f52879de
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 7 Mar 2025 09:00:53 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 7 Mar 2025 09:00:53 -0500
Commit 95dbd827f2edc4d10bebd7e840a0bd6782cf69b7 updated a bunch
of similar cases in the documentation, but missed this one.
Author: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Reviewed-by: Fabrízio de Royes Mello <fabriziomello@gmail.com>
M doc/src/sgml/perform.sgml
Improve possible performance regression
commit : 7f24c0274385ea3d4c797cc2fd60d9a55f3a8d74
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 Mar 2025 11:20:26 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 Mar 2025 11:20:26 +0100
Commit ce62f2f2a0a introduced calls to GetIndexAmRoutineByAmId() in
lsyscache.c functions. This call is a bit more expensive than a
simple syscache lookup. So rearrange the nesting so that we call that
one last and do the cheaper checks first.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/E1tngY6-0000UL-2n%40gemulon.postgresql.org
M src/backend/utils/cache/lsyscache.c
Rename amcancrosscompare
commit : af4002b381d86df6479962953d82f03ecb4e2e06
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 Mar 2025 10:51:53 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 Mar 2025 10:51:53 +0100
After more discussion about commit ce62f2f2a0a, rename the index AM
property amcancrosscompare to two separate properties
amconsistentequality and amconsistentordering. Also improve the
documentation and update some comments that were previously missed.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/E1tngY6-0000UL-2n%40gemulon.postgresql.org
M contrib/bloom/blutils.c
M doc/src/sgml/indexam.sgml
M src/backend/access/brin/brin.c
M src/backend/access/gin/ginutil.c
M src/backend/access/gist/gist.c
M src/backend/access/hash/hash.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/spgist/spgutils.c
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/utils/cache/lsyscache.c
M src/include/access/amapi.h
M src/test/modules/dummy_index_am/dummy_index_am.c
Allow casting between bytea and integer types.
commit : 6da469badaffec32f8a804181cca279561467378
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Fri, 7 Mar 2025 09:31:18 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Fri, 7 Mar 2025 09:31:18 +0000
This allows smallint, integer, and bigint values to be cast to and
from bytea. The bytea value is the two's complement representation of
the integer, with the most significant byte first. For example:
1234::bytea -> \x000004d2
(-1234)::bytea -> \xfffffb2e
Author: Aleksander Alekseev <aleksander@timescale.com>
Reviewed-by: Joel Jacobson <joel@compiler.org>
Reviewed-by: Yugo Nagata <nagata@sraoss.co.jp>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Discussion: https://postgr.es/m/CAJ7c6TPtOp6%2BkFX5QX3fH1SVr7v65uHr-7yEJ%3DGMGQi5uhGtcA%40mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/varlena.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_cast.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/opr_sanity.out
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
CREATE INDEX: don't update table stats if autovacuum=off.
commit : d611f8b1587b8f30caa7c0da99ae5d28e914d54f
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 6 Mar 2025 19:36:34 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 6 Mar 2025 19:36:34 -0800
We previously fixed this for binary upgrade in 71b66171d0, but a
similar problem remained when dumping statistics without data.
Fix by not opportunistically updating table stats during CREATE INDEX
when autovacuum is disabled. For stats to be stable at all, the server
needs to be aware that it should not take every opportunity to update
stats. Per discussion, autovacuum=off is a signal that the user
expects stats to be stable; though if necessary, we could create
a more specific mode in the future.
Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CAExHW5vf9D+8-a5_BEX3y=2y_xY9hiCxV1=C+FnxDvfprWvkng@mail.gmail.com
Discussion: https://postgr.es/m/ca81cbf6e6ea2af838df972801ad4da52640a503.camel%40j-davis.com
M src/backend/catalog/index.c
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Revert "vacuumdb: Add option for analyzing only relations missing stats."
commit : 19e57f4f78e4354d9a21c284868373d28bb0d368
author : John Naylor <john.naylor@postgresql.org>
date : Fri, 7 Mar 2025 10:35:21 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Fri, 7 Mar 2025 10:35:21 +0700
This reverts commit 5f8eb25706b62923c53172e453c8a4dedd877a3d, which in
my branch by mistake.
M doc/src/sgml/ref/vacuumdb.sgml
M src/bin/scripts/t/102_vacuumdb_stages.pl
M src/bin/scripts/vacuumdb.c
M src/test/perl/PostgreSQL/Test/Cluster.pm
Doc: correct aggressive vacuum threshold for multixact members storage
commit : fcabc3adf889ebf2c2d8d1c084989f5fd95aac2f
author : John Naylor <john.naylor@postgresql.org>
date : Fri, 7 Mar 2025 10:22:56 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Fri, 7 Mar 2025 10:22:56 +0700
The threshold is two billion members, which was interpreted as 2GB
in the documentation. Fix to reflect that each member takes up five
bytes, which translates to about 10GB. This is not exact, because of
page boundaries. While at it, mention the maximum size 20GB.
This has been wrong since commit c552e171d16e, so backpatch to
version 14.
Author: Alex Friedman <alexf01@gmail.com>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/CACbFw60UOk6fCC02KsyT3OfU9Dnuq5roYxdw2aFisiN_p1L0bg@mail.gmail.com
Backpatch-through: 14
M doc/src/sgml/maintenance.sgml
vacuumdb: Add option for analyzing only relations missing stats.
commit : 5f8eb25706b62923c53172e453c8a4dedd877a3d
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 4 Feb 2025 15:07:54 -0600
committer: John Naylor <john.naylor@postgresql.org>
date : Tue, 4 Feb 2025 15:07:54 -0600
This commit adds a new --missing-only option that can be used in
conjunction with --analyze-only and --analyze-in-stages. When this
option is specified, vacuumdb will generate ANALYZE commands for a
relation if it is missing any statistics it should ordinarily have.
For example, if a table has statistics for one column but not
another, we will analyze the whole table. A similar principle
applies to extended statistics, expression indexes, and table
inheritance.
Co-authored-by: Corey Huinker <corey.huinker@gmail.com>
Reviewed-by: TODO
Discussion: https://postgr.es/m/Z5O1bpcwDrMgyrYy%40nathan
M doc/src/sgml/ref/vacuumdb.sgml
M src/bin/scripts/t/102_vacuumdb_stages.pl
M src/bin/scripts/vacuumdb.c
M src/test/perl/PostgreSQL/Test/Cluster.pm
Fix race condition in TAP test 007_pre_auth
commit : e2080261cc8c1a962708843dc4c806e19fb2c44e
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 7 Mar 2025 08:12:45 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 7 Mar 2025 08:12:45 +0900
The authentication test added in c76db55c9085 expects a backend to start
and wait at the injection point "init-pre-auth". A query is used to
retrieve the PID of the backend waiting at authentication, but its WHERE
clause was too soft, checking only for a backend in a "starting" state.
As proved by the CI, this WHERE clause is not enough. There is a small
window between the moment when the backend is reported as "starting" in
its backend entry and the moment when it waits in its injection point,
and it was possible for the test to return the PID of a backend process
not yet waiting in the injection point, causing spurious failures. This
issue is fixed by tweaking the query retrieving the PID of the backend
waiting before authentication so as we check for "init-pre-auth" in its
wait_event. An extra check based on the backend_type is added, based on
a suggestion by Jacob, to be more cautious.
Error spotted by the CI on Windows, but it could happen anywhere, as
long as the authentication path is slow enough compared to the TAP test.
Reported-by: Andres Freund <andres@anarazel.de>
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Co-authored-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/soexrl7oeyku24bj3czupxmv27ow35u6edymp5y3oyoysbe2kb@r3tgoos2xp2x
M src/test/authentication/t/007_pre_auth.pl
reindexdb: move PQfinish() calls to the right place
commit : 24503fa95cd8cd57a8d695a910056a1cfcbacefc
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 6 Mar 2025 18:14:41 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 6 Mar 2025 18:14:41 +0100
get_parallel_object_list() has no business closing a connection it did
not create. Make things more sensible by closing the connection at the
level where it is created, in reindex_one_database().
Extracted from a larger patch by the same author. However, the patch as
submitted not only was not described as containing this change, but in
addition it contained a fatal flaw whereby reindexdb would crash and
fail across all of its TAP test, which is why I list myself as
co-author.
Author: Ranier Vilela <ranier.vf@gmail.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/CAEudQArfqr0-s0VVPSEh=0kgOgBJvFNdGW=xSL5rBcr0WDMQYQ@mail.gmail.com
M src/bin/scripts/reindexdb.c
Fix some performance issues in GIN query startup.
commit : 0f21db36d663fcf0789290902c84cc460ef0df8b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 6 Mar 2025 11:54:27 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 6 Mar 2025 11:54:27 -0500
If a GIN index search had a lot of search keys (for example,
"jsonbcol ?| array[]" with tens of thousands of array elements),
both ginFillScanKey() and startScanKey() took O(N^2) time.
Worse, those loops were uncancelable for lack of CHECK_FOR_INTERRUPTS.
The problem in ginFillScanKey() is the brute-force search key
de-duplication done in ginFillScanEntry(). The most expedient
solution seems to be to just stop trying to de-duplicate once
there are "too many" search keys. We could imagine working harder,
say by using a sort-and-unique algorithm instead of brute force
compare-all-the-keys. But it seems unlikely to be worth the trouble.
There is no correctness issue here, since the code already allowed
duplicate keys if any extra_data is present.
The problem in startScanKey() is the loop that attempts to identify
the first non-required search key. In the submitted test case, that
vainly tests all the key positions, and each iteration takes O(N)
time. One part of that is that it's reinitializing the entryRes[]
array from scratch each time, which is entirely unnecessary given
that the triConsistentFn isn't supposed to scribble on its input.
We can easily adjust the array contents incrementally instead.
The other part of it is that the triConsistentFn may itself take
O(N) time (and does in this test case). This is all extremely
brute force: in simple cases with AND or OR semantics, we could
know without any looping whatever that all or none of the keys
are required. But GIN opclasses don't have any API for exposing
that knowledge, so at least in the short run there is little to
be done about that. Put in a CHECK_FOR_INTERRUPTS so that at
least the loop is cancelable.
These two changes together resolve the primary complaint that
the test query doesn't respond promptly to cancel interrupts.
Also, while they don't completely eliminate the O(N^2) behavior,
they do provide quite a nice speedup for mid-sized examples.
Bug: #18831
Reported-by: Niek <niek.brasa@hitachienergy.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18831-e845ac44ebc5dd36@postgresql.org
Backpatch-through: 13
M src/backend/access/gin/ginget.c
M src/backend/access/gin/ginscan.c
Further fix for json_strip_nulls documentation
commit : e33969abc1934cc7fd92d539e51a2b8ae46d6a33
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 6 Mar 2025 10:24:03 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 6 Mar 2025 10:24:03 -0500
Oversight in commit 4603903d294.
Author: Shinoda, Noriyoshi (SXD Japan FSI) <noriyoshi.shinoda@hpe.com>
M doc/src/sgml/func.sgml
Remove extraneous commas in json{b}_strip_nulls documentation
commit : 0e76f253f4f0bf3d8a85e88dbea62a09be1f3ff8
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 6 Mar 2025 08:46:15 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 6 Mar 2025 08:46:15 -0500
Oversight in commit 4603903d294.
Author: Ian Lawrence Barwick <barwick@gmail.com>
M doc/src/sgml/func.sgml
Avoid invalidating all RelationSyncCache entries on publication change.
commit : 588acf6d0ec15d9384c2f712585c0f56936d7bac
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 6 Mar 2025 14:19:38 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 6 Mar 2025 14:19:38 +0530
On change of publication via ALTER PUBLICATION ... SET/ADD/DROP commands,
we were invalidating all the relations present in relation sync cache
maintained by pgoutput. We need to invalidate only the relation entries
that are changed as part of publication DDL.
We have ensured that the publication DDL execution generated the
invalidations required to invalidate impacted relation sync entries in
RelationSyncCache.
This improves the performance by avoiding building the cache entries for
the cases where a publication has many tables but only one of them is
dropped.
Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Hou Zhijie <houzj.fnst@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/OSCPR01MB14966C09AA201EFFA706576A7F5C92@OSCPR01MB14966.jpnprd01.prod.outlook.com
M src/backend/replication/pgoutput/pgoutput.c
Organize and deduplicate statistics import tests.
commit : 1d33de9d683722bfa14e436e30a8f5b999a4e2e7
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 6 Mar 2025 00:19:22 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 6 Mar 2025 00:19:22 -0800
Author: Corey Huinker <corey.huinker@gmail.com>
Reported-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_bWEqUfxhODfJ-XbZC75vq=P6DYOKK6biyey=yM1Ah3Hg@mail.gmail.com
Discussion: https://postgr.es/m/CADkLM=f1n2_Vomq0gKab7xdxDHmJGgn=DE48P8fzQOp3Mrs1Qg@mail.gmail.com
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Address stats export review comments.
commit : f9f4b43b8dc0174797f939af986d8c9bb52269eb
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 6 Mar 2025 00:11:12 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 6 Mar 2025 00:11:12 -0800
Per discussion, did not use Jian He's patch exactly.
Reported-by: jian he <jian.universality@gmail.com>
Reviewed-by: Corey Huinker <corey.huinker@gmail.com>
Discussion: https://postgr.es/m/CACJufxFVq=tq9u1zrHWYSbMi1T07gS9Ff0LJScMco4HZmtZ1xw@mail.gmail.com
Discussion: https://postgr.es/m/CADkLM=f1n2_Vomq0gKab7xdxDHmJGgn=DE48P8fzQOp3Mrs1Qg@mail.gmail.com
M src/bin/pg_dump/pg_backup_archiver.c
Address stats import review comments.
commit : 298944e8d80252820531309e5b73d7de7d85e2df
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 5 Mar 2025 23:07:25 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 5 Mar 2025 23:07:25 -0800
Reported-by: jian he <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CACJufxHG9MBQozbJQ4JRBcRbUO+t+sx4qLZX092rS_9b4SR_EA@mail.gmail.com
M src/backend/statistics/relation_stats.c
M src/backend/statistics/stat_utils.c
Fix compiler warnings about typedef redefinitions
commit : 39de4f157d3ac0b889bb276c2487fe160578f967
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 6 Mar 2025 03:10:22 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 6 Mar 2025 03:10:22 +0200
Clang with -Wtypedef-redefinition produced warnings:
src/include/storage/latch.h:122:3: error: redefinition of typedef 'Latch' is a C11 feature [-Werror,-Wtypedef-redefinition]
Per buildfarm
M src/include/storage/waiteventset.h
Add more monitoring data for WAL writes in the WAL receiver
commit : 7f7f324eb5a4754ab9fa1c68dfef952555a94d1b
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 6 Mar 2025 09:39:45 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 6 Mar 2025 09:39:45 +0900
This commit adds two improvements related to the monitoring of WAL
writes for the WAL receiver.
First, write counts and timings are now counted in pg_stat_io for the
WAL receiver. These have been discarded from pg_stat_wal in
ff99918c625a due to performance concerns, related to the fact that we
still relied on an on-disk file for the stats back then, even with
track_wal_io_timing to avoid the overhead of the timestamp calculations.
This implementation is simpler than the original proposal as it is
possible to rely on the APIs of pgstat_io.c to do the job. Like the
fsync and read data, track_wal_io_timing needs to be enabled to track
the timings.
Second, a wait event is added around the pg_pwrite() call in charge of
the writes, using the exiting WAIT_EVENT_WAL_WRITE. This is useful as
the WAL receiver data is tracked in pg_stat_activity.
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/Z8gFnH4o3jBm5BRz@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/replication/walreceiver.c
Split WaitEventSet functions to separate source file
commit : 393e0d2314050576c9c039853fdabe7f685a4f47
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 6 Mar 2025 01:26:16 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 6 Mar 2025 01:26:16 +0200
latch.c now only contains the Latch related functions, which build on
the WaitEventSet abstraction. Most of the platform-dependent stuff is
now in waiteventset.c.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/8a507fb6-df28-49d3-81a5-ede180d7f0fb@iki.fi
M src/backend/libpq/pqsignal.c
M src/backend/postmaster/postmaster.c
M src/backend/storage/ipc/Makefile
M src/backend/storage/ipc/latch.c
M src/backend/storage/ipc/meson.build
A src/backend/storage/ipc/waiteventset.c
M src/backend/utils/init/miscinit.c
M src/include/storage/latch.h
A src/include/storage/waiteventset.h
Use ModifyWaitEvent to update exit_on_postmaster_death
commit : 84e5b2f07a5e8ba983ff0f6e71b063b27f45f346
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 6 Mar 2025 01:26:12 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 6 Mar 2025 01:26:12 +0200
This is in preparation for splitting WaitEventSet related functions to
a separate source file. That will hide the details of WaitEventSet
from WaitLatch, so it must use an exposed function instead of
modifying WaitEventSet->exit_on_postmaster_death directly.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/8a507fb6-df28-49d3-81a5-ede180d7f0fb@iki.fi
M src/backend/storage/ipc/latch.c
ecpg: Fix compiler warning in ecpg build with Meson.
commit : 9f25b9f7392267c5628af09f498086ad9f33ccbd
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 6 Mar 2025 08:22:30 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 6 Mar 2025 08:22:30 +0900
Previously, Meson could produce a warning about the use of 'deps' in ecpg:
WARNING: Project targets '>=0.54' but uses a feature introduced in '0.60.0': list.<plus>. The right-hand operand was not a list.
The right-hand operand of 'deps' should be a list. This commit fixes
the warning by wrapping it with square brackets.
This issue was introduced in commit 28f04984f0c.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/CAOYmi+ks8wO06Ymxduw2h_eQJ_D4_jHGeyMK0P=p5Q3psnEdMA@mail.gmail.com
M src/interfaces/ecpg/preproc/meson.build
Remove unused ShutdownLatchSupport() function
commit : a98e4dee63ce594ed32d1b004385d6ffa8b57dc5
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 5 Mar 2025 23:46:29 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 5 Mar 2025 23:46:29 +0200
The only caller was removed in commit 80a8f95b3b. I don't foresee
needing it any time soon, and I'm working on some big changes in this
area, so let's remove it out of the way.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/8a507fb6-df28-49d3-81a5-ede180d7f0fb@iki.fi
M src/backend/storage/ipc/latch.c
M src/include/storage/latch.h
ci: Remove installation of libcurl
commit : 153836b99acb5c432b9e2326dbd75ac239f3c320
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 5 Mar 2025 22:12:20 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 5 Mar 2025 22:12:20 +0100
The CI images come with libcurl pre-installed since commit a119426
in the pg-vm-images repository so remove the installation commands
from the Cirrus tasks. Installation of libcurl packages was added
in the OAuth patchset which introduced the dependency, a backpatch
is thus not applicable.
Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/8745B9D8-D897-4302-BD4C-FC18F291ECB7@yesql.se
M .cirrus.tasks.yml
ci: Document what makes certain tasks special
commit : d4a6c847ca0afc40a558c53995eaaf393cf66785
author : Andres Freund <andres@anarazel.de>
date : Wed, 5 Mar 2025 13:19:28 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 5 Mar 2025 13:19:28 -0500
To increase coverage without drastically increasing CI resource usage, we have
different CI tasks test different things (e.g. the linux tasks use
sanitizers). Unfortunately that can create confusing situations where CI
fails on some OS, but not others, without the problem appearing to be platform
dependent.
To, partially, address that, add a comment, prefixed with SPECIAL, to each
task that we use to test in some non-default way.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/321570.1741195755@sss.pgh.pa.us
M .cirrus.tasks.yml
ci: freebsd: Specify debug_parallel_query=regress
commit : 0a2f5df88168e3d8c50e3a26bf1ca45c61f9d6f2
author : Andres Freund <andres@anarazel.de>
date : Wed, 5 Mar 2025 13:19:28 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 5 Mar 2025 13:19:28 -0500
A lot of buildfarm animals run with debug_parallel_query=regress, while CI
didn't test that. That lead to the annoying situation of only noticing related
test instabilities after merging changes upstream.
FreeBSD was chosen because it's a relatively fast task. It also tests
debug_write_read_parse_plan_trees etc, which probably is exercised a bit more
heavily with debug_parallel_query=regress.
Discussion: https://postgr.es/m/zbuk4mlov22yfoktf5ub3lwjw2b7ezwphwolbplthepda42int@h6wpvq7orc44
M .cirrus.tasks.yml
ci: Upgrade FreeBSD image
commit : ad40644eb8582fc1f537330b76ae7893cacf7b21
author : Andres Freund <andres@anarazel.de>
date : Wed, 5 Mar 2025 10:33:47 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 5 Mar 2025 10:33:47 -0500
Upgrade to the current stable version. To avoid needing commits like this in
the future, the CI image name now doesn't contain the OS version number
anymore.
Backpatch to all versions with CI support, we don't want to generate CI images
for multiple FreeBSD versions.
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CAN55FZ3_P4JJ6tWZafjf-_XbHgG6DQGXhH-y6Yp78_bwBJjcww@mail.gmail.com
Backpatch-through: 15
M .cirrus.tasks.yml
Revert "Show index search count in EXPLAIN ANALYZE."
commit : d00107cd63e780753aa25563fa37603369997d0c
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 5 Mar 2025 10:27:31 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 5 Mar 2025 10:27:31 -0500
This reverts commit 5ead85fbc81162ab1594f656b036a22e814f96b3.
This commit shows test failures with debug_parallel_query=regress. The
underlying issue needs to be debugged, so revert for now.
M contrib/bloom/blscan.c
M doc/src/sgml/bloom.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/ref/explain.sgml
M doc/src/sgml/rules.sgml
M src/backend/access/brin/brin.c
M src/backend/access/gin/ginscan.c
M src/backend/access/gist/gistget.c
M src/backend/access/hash/hashsearch.c
M src/backend/access/index/genam.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/spgist/spgscan.c
M src/backend/commands/explain.c
M src/include/access/relscan.h
M src/test/regress/expected/brin_multi.out
M src/test/regress/expected/memoize.out
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/select.out
M src/test/regress/sql/memoize.sql
M src/test/regress/sql/partition_prune.sql
Allow json{b}_strip_nulls to remove null array elements
commit : 4603903d294bbdd644afecf9b5970827db6d1ff5
author : Andrew Dunstan <andrew@dunslane.net>
date : Wed, 5 Mar 2025 09:50:34 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Wed, 5 Mar 2025 09:50:34 -0500
An additional paramater ("strip_in_arrays") is added to these functions.
It defaults to false. If true, then null array elements are removed as
well as null valued object fields. JSON that just consists of a single
null is not affected.
Author: Florents Tselai <florents.tselai@gmail.com>
Discussion: https://postgr.es/m/4BCECCD5-4F40-4313-9E98-9E16BEB0B01D@gmail.com
M doc/src/sgml/func.sgml
M src/backend/catalog/system_functions.sql
M src/backend/utils/adt/jsonfuncs.c
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/json.out
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/json.sql
M src/test/regress/sql/jsonb.sql
Show index search count in EXPLAIN ANALYZE.
commit : 5ead85fbc81162ab1594f656b036a22e814f96b3
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 5 Mar 2025 09:36:48 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 5 Mar 2025 09:36:48 -0500
Expose the count of index searches/index descents in EXPLAIN ANALYZE's
output for index scan nodes. This information is particularly useful
with scans that use ScalarArrayOp quals, where the number of index scans
isn't predictable in advance (at least not with optimizations like the
one added to nbtree by Postgres 17 commit 5bf748b8). It will also be
useful when EXPLAIN ANALYZE shows details of an nbtree index scan that
uses skip scan optimizations set to be introduced by an upcoming patch.
The instrumentation works by teaching index AMs to increment a new
nsearches counter whenever a new index search begins. The counter is
incremented at exactly the same point that index AMs must already
increment the index's pg_stat_*_indexes.idx_scan counter (we're counting
the same event, but at the scan level rather than the relation level).
The new counter is stored in the scan descriptor (IndexScanDescData),
which explain.c reaches by going through the scan node's PlanState.
This approach doesn't match the approach used when tracking other index
scan specific costs (e.g., "Rows Removed by Filter:"). It is similar to
the approach used in other cases where we must track costs that are only
readily accessible inside an access method, and not from the executor
(e.g., "Heap Blocks:" output for a Bitmap Heap Scan). It is inherently
necessary to maintain a counter that can be incremented multiple times
during a single amgettuple call (or amgetbitmap call), and directly
exposing PlanState.instrument to index access methods seems unappealing.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Tomas Vondra <tomas@vondra.me>
Reviewed-By: Robert Haas <robertmhaas@gmail.com>
Reviewed-By: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wz=PKR6rB7qbx+Vnd7eqeB5VTcrW=iJvAsTsKbdG+kW_UA@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WzkRqvaqR2CTNqTZP0z6FuL4-3ED6eQB0yx38XBNj1v-4Q@mail.gmail.com
M contrib/bloom/blscan.c
M doc/src/sgml/bloom.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/ref/explain.sgml
M doc/src/sgml/rules.sgml
M src/backend/access/brin/brin.c
M src/backend/access/gin/ginscan.c
M src/backend/access/gist/gistget.c
M src/backend/access/hash/hashsearch.c
M src/backend/access/index/genam.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/spgist/spgscan.c
M src/backend/commands/explain.c
M src/include/access/relscan.h
M src/test/regress/expected/brin_multi.out
M src/test/regress/expected/memoize.out
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/select.out
M src/test/regress/sql/memoize.sql
M src/test/regress/sql/partition_prune.sql
Rename some signal and interrupt handling functions for consistency
commit : 635f580120b99f6df71d7c12749b22acde61c5ad
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 5 Mar 2025 16:22:26 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 5 Mar 2025 16:22:26 +0200
The usual pattern for handling a signal is that the signal handler
sets a flag and calls SetLatch(MyLatch), and CHECK_FOR_INTERRUPTS() or
other code that is part of a wait loop calls another function to deal
with it. The naming of the functions involved was a bit inconsistent,
however. CHECK_FOR_INTERRUPTS() calls ProcessInterrupts() to do the
heavy-lifting, but the analogous functions in aux processes were
called HandleMainLoopInterrupts(), HandleStartupProcInterrupts(),
etc. Similarly, most subroutines of ProcessInterrupts() were called
Process*(), but some were called Handle*().
To make things less confusing, rename all the functions that are part
of the overall signal/interrupt handling system but are not executed
in a signal handler to e.g. ProcessSomething(), rather than
HandleSomething(). The "Process" prefix is now consistently used in
the non-signal-handler functions, and the "Handle" prefix in functions
that are part of signal handlers, except for some completely unrelated
functions that clearly have nothing to do with signal or interrupt
handling.
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://www.postgresql.org/message-id/8a384b26-1499-41f6-be33-64b801fb98b8@iki.fi
M src/backend/access/transam/parallel.c
M src/backend/access/transam/xlogrecovery.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/bgwriter.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/interrupt.c
M src/backend/postmaster/pgarch.c
M src/backend/postmaster/startup.c
M src/backend/postmaster/walsummarizer.c
M src/backend/postmaster/walwriter.c
M src/backend/replication/logical/applyparallelworker.c
M src/backend/tcop/postgres.c
M src/include/access/parallel.h
M src/include/postmaster/interrupt.h
M src/include/postmaster/startup.h
M src/include/replication/logicalworker.h
Add ALTER TABLE ... ALTER CONSTRAINT ... SET [NO] INHERIT
commit : f4e53e10b6ce0eedeb98caa4356facb47c7bb9cb
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 5 Mar 2025 13:50:22 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 5 Mar 2025 13:50:22 +0100
This allows to redefine an existing non-inheritable constraint to be
inheritable, which allows to straighten up situations with NO INHERIT
constraints so that thay can become normal constraints without having to
re-verify existing data. For existing inheritance children this may
require creating additional constraints, if they don't exist already.
It also allows to do the opposite, if only for symmetry.
Author: Suraj Kharage <suraj.kharage@enterprisedb.com>
Reviewed-by: jian he <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CAF1DzPVfOW6Kk=7SSh7LbneQDJWh=PbJrEC_Wkzc24tHOyQWGg@mail.gmail.com
M doc/src/sgml/ref/alter_table.sgml
M src/backend/commands/tablecmds.c
M src/backend/parser/gram.y
M src/include/nodes/parsenodes.h
M src/test/regress/expected/inherit.out
M src/test/regress/sql/inherit.sql
Fix some gaps in pg_stat_io with WAL receiver and WAL summarizer
commit : f4694e0f35b218238cbc87bcf8f8f5c6639bb1d4
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 5 Mar 2025 10:17:39 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 5 Mar 2025 10:17:39 +0900
The WAL receiver and WAL summarizer processes gain each one a call to
pgstat_report_wal(), to make sure that they report their WAL statistics
to pgstats, gathering data for pg_stat_io.
In the WAL receiver, the stats reports are timed with status updates sent
to the primary, that depend on wal_receiver_status_interval and
wal_receiver_timeout. This is a conservative choice, but perhaps we
could be more aggressive with the frequency of the stats reports. An
interesting historical fact is that the WAL receiver does writes and
syncs of WAL, but it has never reported its statistics to pgstats in
pg_stat_wal.
In the WAL summarizer, the stats reports are done each time the process
waits for WAL.
While on it, pg_stat_io is adjusted so as these two processes do not
report any rows when IOObject is not WAL, making the view easier to use
with less rows.
Two tests are added in TAP, checking statistics for the WAL summarizer
and the WAL receiver. Status updates in the WAL receiver are currently
possible in the recovery test 001_stream_rep.pl.
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/Z8UKZyVSHUUQJHNb@paquier.xyz
M src/backend/postmaster/walsummarizer.c
M src/backend/replication/walreceiver.c
M src/backend/utils/activity/pgstat_io.c
M src/bin/pg_walsummary/t/002_blocks.pl
M src/test/recovery/t/001_stream_rep.pl
psql: Fix memory leak with \gx used within a pipeline
commit : 54d23601b978d2552696fb7fe35ae5d6102ea2cb
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 5 Mar 2025 07:56:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 5 Mar 2025 07:56:03 +0900
While inside a pipeline, \gx is currently forbidden and will make
exec_command_g() exit early. There was a memory leak in this code path,
so let's fix it.
Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/CAO6_XqqFVQjLjZQiL7xdwLpzZEy1ghO_JWvCFPM_OmwF9s7XdA@mail.gmail.com
M src/bin/psql/command.c
Enforce memory limit during parallel GIN builds
commit : b229c10164770769c3b5033785917ca7a43a2471
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 4 Mar 2025 20:37:55 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 4 Mar 2025 20:37:55 +0100
Index builds are expected to respect maintenance_work_mem, just like
other maintenance operations. For serial builds this is done simply by
flushing the buffer in ginBuildCallback() into the index. But with
parallel builds it's more complicated, because there are multiple places
that can allocate memory.
ginBuildCallbackParallel() does the same thing as ginBuildCallback(),
except that the accumulated items are written into tuplesort. Then the
entries with the same key get merged - first in the worker, then in the
leader - and the TID lists may get (arbitrarily) long. It's unlikely it
would exceed the memory limit, but it's possible. We address this by
evicting some of the data if the list gets too long.
We can't simply dump the whole in-memory TID list. The GIN index bulk
insert code expects to see TIDs in monotonic order; it may fail if the
TIDs go backwards. If the TID lists overlap, evicting the whole current
TID list would break this (a later entry might add "old" TID values into
the already-written part).
In the workers this is not an issue, because the lists never overlap.
But the leader may see overlapping lists produced by the workers.
We can however derive a safe "horizon" TID - the entries (for a given
key) are sorted by (key, first TID), which means no future list can add
values before the last "first TID" we've seen. This patch tracks the
"frozen" part of the TID list, which we know can't change by merging
additional TID lists. If needed, we can evict this part of the list.
We don't want to do this too often - the smaller lists we evict, the
more expensive it'll be to merge them in the next step (especially in
the leader). Therefore we only trim the list if we have at least 1024
frozen items, and if the whole list is at least 64kB large.
These thresholds are somewhat arbitrary and conservative. We might
calculate the values from maintenance_work_mem, but tests show that does
not really improve anything (time, compression ratio, ...). So we stick
to these conservative values to release memory faster.
Author: Tomas Vondra
Reviewed-by: Matthias van de Meent, Andy Fan, Kirill Reshke
Discussion: https://postgr.es/m/6ab4003f-a8b8-4d75-a67f-f25ad98582dc%40enterprisedb.com
M src/backend/access/gin/gininsert.c
pg_upgrade: Check for the expected error message in TAP tests.
commit : f52345995d3691de7b0a888903b6cfed1497c672
author : Masahiko Sawada <msawada@postgresql.org>
date : Tue, 4 Mar 2025 11:16:12 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Tue, 4 Mar 2025 11:16:12 -0800
Since pg_upgrade prints its error messages on stdout, we can't use
command_fails_like() to check if it fails for the right reason. This
commit uses command_checks_all() in pg_upgrade TAP tests to check the
exit status and stdout, enabling proper verification of error
reasons.
Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://postgr.es/m/87tt8h1vb7.fsf@wibble.ilmari.org
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
M src/bin/pg_upgrade/t/004_subscription.pl
M src/bin/pg_upgrade/t/005_char_signedness.pl
Fix ALTER TABLE error message
commit : 7bbc46213de9301795da90923a015070239c1fcc
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 4 Mar 2025 20:07:30 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 4 Mar 2025 20:07:30 +0100
This bogus error message was introduced in 2013 by commit f177cbfe676d,
because of misunderstanding the processCASbits() API; at the time, no
test cases were added that would be affected by this change. Only in
ca87c415e2fc was one added (along with a couple of typos), with an XXX
note that the error message was bogus. Fix the whole, add some test
cases.
Backpatch all the way back.
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/202503041822.aobpqke3igvb@alvherre.pgsql
M src/backend/parser/gram.y
M src/test/regress/expected/constraints.out
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/constraints.sql
M src/test/regress/sql/foreign_key.sql
Refactor Copy{From|To}GetRoutine() to use pass-by-reference argument.
commit : bacbc4863b3bfb79b9577f11f2c77e4df9f66d66
author : Masahiko Sawada <msawada@postgresql.org>
date : Tue, 4 Mar 2025 10:38:41 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Tue, 4 Mar 2025 10:38:41 -0800
The change improves efficiency by eliminating unnecessary copying of
CopyFormatOptions.
The coverity also complained about inefficiencies caused by
pass-by-value.
Oversight in 7717f6300 and 2e4127b6d.
Reported-by: Junwang Zhao <zhjwpku@gmail.com>
Reported-by: Tom Lane <tgl@sss.pgh.pa.us> (per reports from coverity)
Author: Sutou Kouhei <kou@clear-code.com>
Reviewed-by: Junwang Zhao <zhjwpku@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAEG8a3L6YCpPksTQMzjD_CvwDEhW3D_t=5md9BvvdOs5k+TA=Q@mail.gmail.com
M src/backend/commands/copyfrom.c
M src/backend/commands/copyto.c
Compress TID lists when writing GIN tuples to disk
commit : 0b2a45a5d1f2b088640a7eb7817ca6a0513a2717
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 4 Mar 2025 19:02:04 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 4 Mar 2025 19:02:04 +0100
When serializing GIN tuples to tuplesorts during parallel index builds,
we can significantly reduce the amount of data by compressing the TID
lists. The GIN opclasses may produce a lot of data (depending on how
many keys are extracted from each row), and the TID compression is very
efficient and effective.
If the number of distinct keys is high, the first worker pass (reading
data from the table and writing them into a private tuplesort) may not
benefit from the compression very much. It is likely to spill data to
disk before the TID lists get long enough for the compression to help.
The second pass (writing the merged data into the shared tuplesort) is
more likely to benefit from compression.
The compression can be seen as a way to reduce the amount of disk space
needed by the parallel builds, because the data is written twice. First
into the per-worker tuplesorts, then into the shared tuplesort.
Author: Tomas Vondra
Reviewed-by: Matthias van de Meent, Andy Fan, Kirill Reshke
Discussion: https://postgr.es/m/6ab4003f-a8b8-4d75-a67f-f25ad98582dc%40enterprisedb.com
M src/backend/access/gin/gininsert.c
M src/tools/pgindent/typedefs.list
Add .gitignore entry for ecpg test detritus.
commit : 9b4bdf876a2e4fc9db80cb0137911a8d53bdbe1e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 4 Mar 2025 12:58:04 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 4 Mar 2025 12:58:04 -0500
Oversight in commit 28f04984f.
M src/interfaces/ecpg/preproc/.gitignore
Make FP_LOCK_SLOTS_PER_BACKEND look like a function
commit : c878de1db438f9f4747374c286d4319cc527eec2
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 4 Mar 2025 18:33:09 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 4 Mar 2025 18:33:09 +0100
The FP_LOCK_SLOTS_PER_BACKEND macro looks like a constant, but it
depends on the max_locks_per_transaction GUC, and thus can change. This
is non-obvious and confusing, so make it look more like a function by
renaming it to FastPathLockSlotsPerBackend().
While at it, use the macro when initializing fast-path shared memory,
instead of using the formula.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/ffiwtzc6vedo6wb4gbwelon5nefqg675t5c7an2ta7pcz646cg%40qwmkdb3l4ett
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/init/postinit.c
M src/include/storage/proc.h
Add regression tests for pg_stat_progress_copy.tuples_skipped.
commit : 91ecb5e0bc7f8373ec9ff7d188bfd0d45f13a333
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 4 Mar 2025 23:56:49 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 4 Mar 2025 23:56:49 +0900
This commit adds tests to verify that tuples_skipped in pg_stat_progress_copy
works as expected. While existing tests checked other fields, tuples_skipped
was previously untested.
This improves test coverage and ensures accurate tracking of skipped tuples.
Author: Jian He <jian.universality@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Josef Šimánek <josef.simanek@gmail.com>
Discussion: https://postgr.es/m/CACJufxFazq-bfyhiO0KBojR=yOr84E25Rqf6mHB0Ow0KPidkKw@mail.gmail.com
M src/test/regress/expected/copy.out
M src/test/regress/sql/copy.sql
Fix outdated comment
commit : d2e7068392b22218b83b7e275b77968027ac3d29
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 4 Mar 2025 15:33:19 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 4 Mar 2025 15:33:19 +0200
Commit bc971f4025 replaced the latch-setting mechanism that the
comment talked about with a condition variable. And before that,
commit 2258e76f90 moved the code so that the comment got detached from
the loop that it talked about, so move the comment closer to the loop.
M src/backend/access/transam/xlogutils.c
doc: Expand version compatibility for pg_basebackup features
commit : ad13490be0005b0f467879d8f88db003b7494138
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 4 Mar 2025 12:08:27 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 4 Mar 2025 12:08:27 +0100
This updates the paragraph on backwards compatitibility for server
features to include --incremental which only works on servers with
v17 or newer. Backpatch down to v17 where incremental backup was
added.
Author: David G. Johnston <David.G.Johnston@Gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAKFQuwZYfZyeTkS3g2Ovw84TsxHa796xnf-u5kfgn_auyxZk0Q@mail.gmail.com
Backpatch-through: 17
M doc/src/sgml/ref/pg_basebackup.sgml
Fix accidental use of = instead of ==
commit : 3abbd8dbeb8f0e4764ad6edb1ab3a71d4c3023f0
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 4 Mar 2025 09:45:01 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 4 Mar 2025 09:45:01 +0100
Fix for commit 630f9a43cec. It used = instead of ==. The result
would be an incorrect error message.
Author: Jacob Brazeal <jacob.brazeal@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/CA%2BCOZaC-JMbhQ4O0Q8V1Bxa0R%2BNex_RN9D6UyuLPiEx_CK4Heg%40mail.gmail.com
M src/backend/commands/indexcmds.c
Fix ALTER TABLE ADD VIRTUAL GENERATED COLUMN when table rewrite
commit : f011acdd61fc296ec8822ca4edc18e32b6bd2b56
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 4 Mar 2025 09:18:32 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 4 Mar 2025 09:18:32 +0100
demo:
CREATE TABLE gtest20a (a int PRIMARY KEY, b int GENERATED ALWAYS AS (a * 2) VIRTUAL);
ALTER TABLE gtest20a ADD COLUMN c float8 DEFAULT RANDOM() CHECK (b < 60);
ERROR: no generation expression found for column number 2 of table "pg_temp_17306"
In ATRewriteTable, the variable OIDNewHeap (if valid) corresponding
pg_attrdef default expression entry was not populated. So OIDNewHeap
cannot be used to call expand_generated_columns_in_expr or
build_generation_expression. Therefore in ATRewriteTable, we can only
use the existing relation to expand the generated expression.
Author: jian he <jian.universality@gmail.com>
Reviewed-by: Srinath Reddy <srinath2133@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CACJufxEJ%3DFoajabWXjszo_yrQeKSxdZ87KJqBW373rSbajKGAA%40mail.gmail.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/generated_stored.out
M src/test/regress/expected/generated_virtual.out
M src/test/regress/sql/generated_stored.sql
M src/test/regress/sql/generated_virtual.sql
Avoid NullTest deduction for clone clauses
commit : 716a051aaccdef9296c41826034d85c196549146
author : Richard Guo <rguo@postgresql.org>
date : Tue, 4 Mar 2025 16:11:03 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Tue, 4 Mar 2025 16:11:03 +0900
In commit b262ad440, we introduced an optimization that reduces an IS
NOT NULL qual on a column defined as NOT NULL to constant true, and an
IS NULL qual on a NOT NULL column to constant false, provided we can
prove that the input expression of the NullTest is not nullable by any
outer join. This deduction happens after we have generated multiple
clones of the same qual condition to cope with commuted-left-join
cases.
However, performing the NullTest deduction for clone clauses can be
unsafe, because we don't have a reliable way to determine if the input
expression of a NullTest is non-nullable: nullingrel bits in clone
clauses may not reflect reality, so we dare not draw conclusions from
clones about whether Vars are guaranteed not-null.
To fix, we check whether the given RestrictInfo is a clone clause in
restriction_is_always_true and restriction_is_always_false, and avoid
performing any reduction if it is.
There are several ensuing plan changes in predicate.out, and we have
to modify the tests to ensure that they continue to test what they are
intended to. Additionally, this fix causes the test case added in
f00ab1fd1 to no longer trigger the bug that commit fixed, so we also
remove that test case.
Back-patch to v17 where this bug crept in.
Reported-by: Ronald Cruz <cruz@rentec.com>
Diagnosed-by: Tom Lane <tgl@sss.pgh.pa.us>
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/f5320d3d-77af-4ce8-b9c3-4715ff33f213@rentec.com
Backpatch-through: 17
M src/backend/optimizer/plan/initsplan.c
M src/test/regress/expected/predicate.out
M src/test/regress/sql/predicate.sql
ecpg: Add TAP test for the ecpg command.
commit : 28f04984f0c240b76e61f00cd247554fbc850056
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 4 Mar 2025 14:58:46 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 4 Mar 2025 14:58:46 +0900
This commit adds a TAP test to verify that the ecpg command correctly
detects unsupported or disallowed statements in input files and reports
the appropriate error or warning messages.
This test helps catch bugs like the one introduced in commit 3d009e45bd,
which broke ecpg's handling of unsupported COPY FROM STDIN statements,
later fixed by commit 94b914f601b.
Author: Ryo Kanbayashi <kanbayashi.dev@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/CANOn0EzoMyxA1m-quDS1UeQUq6FNki6+GGiGucgr9tm2R78rKw@mail.gmail.com
M src/interfaces/ecpg/preproc/Makefile
M src/interfaces/ecpg/preproc/meson.build
A src/interfaces/ecpg/preproc/t/001_ecpg_err_warn_msg.pl
A src/interfaces/ecpg/preproc/t/002_ecpg_err_warn_msg_informix.pl
A src/interfaces/ecpg/preproc/t/err_warn_msg.pgc
A src/interfaces/ecpg/preproc/t/err_warn_msg_informix.pgc
Split pgstat_bestart() into three different routines
commit : c76db55c9085d0b7984ea337576e45a7d1268b97
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 4 Mar 2025 14:09:44 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 4 Mar 2025 14:09:44 +0900
pgstat_bestart(), used post-authentication to set up a backend entry
in the PgBackendStatus array, so as its data becomes visible in
pg_stat_activity and related catalogs, has its logic divided into three
routines with this commit, called in order at different steps of the
backend initialization:
* pgstat_bestart_initial() sets up the backend entry with a minimal
amount of information, reporting it with a new BackendState called
STATE_STARTING while waiting for backend initialization and client
authentication to complete. The main benefit that this offers is
observability, so as it is possible to monitor the backend activity
during authentication. This step happens earlier than in the logic
prior to this commit. pgstat_beinit() happens earlier as well, before
authentication.
* pgstat_bestart_security() reports the SSL/GSS status of the
connection, once authentication completes. Auxiliary processes, for
example, do not need to call this step, hence it is optional. This
step is called after performing authentication, same as previously.
* pgstat_bestart_final() reports the user and database IDs, takes the
entry out of STATE_STARTING, and reports its application_name. This is
called as the last step of the three, once authentication completes.
An injection point is added, with a test checking that the "starting"
phase of a backend entry is visible in pg_stat_activity. Some follow-up
patches are planned to take advantage of this refactoring with more
information provided in backend entries during authentication (LDAP
hanging was a problem for the author, initially).
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CAOYmi+=60deN20WDyCoHCiecgivJxr=98s7s7-C8SkXwrCfHXg@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/postmaster/auxprocess.c
M src/backend/utils/activity/backend_status.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/init/postinit.c
M src/include/utils/backend_status.h
M src/test/authentication/Makefile
M src/test/authentication/meson.build
A src/test/authentication/t/007_pre_auth.pl
Add more assertions in palloc0() and palloc_extended()
commit : 40d3f8274499cb1dd349f60f2e5915f907acce6e
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 4 Mar 2025 10:53:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 4 Mar 2025 10:53:10 +0900
palloc() includes an assertion checking that an alloc() implementation
never returns NULL for all MemoryContextMethods.
This commit adds a similar assertion in palloc0(). In palloc_extend(),
a different assertion is added, checking that MCXT_ALLOC_NO_OOM is set
when an alloc() routine returns NULL. These additions can be useful to
catch errors when implementing a new set of MemoryContextMethods
routines.
Author: Andreas Karlsson <andreas@proxel.se>
Discussion: https://postgr.es/m/507e8eba-2035-4a12-a777-98199a66beb8@proxel.se
M src/backend/utils/mmgr/mcxt.c
doc: Convert UUID functions list to table format.
commit : ba57dcfdcda824e92eeb30b8863be8bf784908a8
author : Masahiko Sawada <msawada@postgresql.org>
date : Mon, 3 Mar 2025 15:44:01 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Mon, 3 Mar 2025 15:44:01 -0800
Convert the list of UUID functions into a table for better
readability. This commit also adds references to the UUID type section
and includes descriptions of different UUID generation algorithm
versions.
Author: Andy Alsup <bluesbreaker@gmail.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CADOZ7s7OHag+r6w+BzKw2xgb3fVtAD-pU=_N9-9pSe5W1TB+xQ@mail.gmail.com
M doc/src/sgml/datatype.sgml
M doc/src/sgml/func.sgml
Allow => syntax for named cursor arguments in plpgsql.
commit : 246dedc5d028800b393920184e5e6319c6805253
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Mar 2025 18:00:05 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Mar 2025 18:00:05 -0500
We've traditionally accepted "name := value" syntax for
cursor arguments in plpgsql. But it turns out that the
equivalent statements in Oracle use "name => value".
Since we accept both forms of punctuation for function
arguments, it makes sense to do the same here.
Author: Pavel Stehule <pavel.stehule@gmail.com>
Reviewed-by: Gilles Darold <gilles@darold.net>
Discussion: https://postgr.es/m/CAFj8pRA3d0ARQEMbABa1n6q25AUdNmyO8aGs56XNf9pD4sRMjQ@mail.gmail.com
M doc/src/sgml/plpgsql.sgml
M src/pl/plpgsql/src/pl_gram.y
M src/test/regress/expected/plpgsql.out
M src/test/regress/sql/plpgsql.sql
ci: Use a RAM disk for NetBSD and OpenBSD.
commit : b6904afae40d94c2f34bc3c94aee1cdbe0bcfbe9
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 4 Mar 2025 11:18:29 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 4 Mar 2025 11:18:29 +1300
Put the RAM disk setup for all three *BSD CI tasks into a common script,
replacing the old FreeBSD-specific one from commit 0265e5c1. This makes
them run 3 times and a bit over 2 times faster, respectively.
NetBSD and FreeBSD now share the same one-liner to mount tmpfs. OpenBSD
needs a GCP-image specific recipe that knows where to steal an unused
disk partition needed to reserve swap space for an mfs RAM disk, because
its tmpfs is deprecated and currently broken. The configured size is
enough for our current tests but could potentially need future
expansion. Thanks to Bilal for the disklabel incantation.
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGJJ-XrPhN%2BQA4ZUfYAAXcwOSDty9t0vE9Z8__AdacKnQg%40mail.gmail.com
M .cirrus.tasks.yml
D src/tools/ci/gcp_freebsd_repartition.sh
A src/tools/ci/gcp_ram_disk.sh
Trigger more frequent autovacuums with relallfrozen
commit : 06eae9e6218ab2acf64ea497bad0360e4c90e32d
author : Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 3 Mar 2025 14:42:00 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 3 Mar 2025 14:42:00 -0500
Calculate the insert threshold for triggering an autovacuum of a
relation based on the number of unfrozen pages.
By only considering the unfrozen portion of the table when calculating
how many tuples to add to the insert threshold, we can trigger more
frequent vacuums of insert-heavy tables. This increases the chances of
vacuuming those pages when they still reside in shared buffers
This also increases the number of autovacuums triggered by tuples
inserted and not by wraparound risk. We prefer to freeze these pages
during insert-triggered autovacuums, as anti-wraparound vacuums are not
automatically canceled by conflicting lock requests.
We calculate the unfrozen percentage of the table using the recently
added (99f8f3fbbc8f) relallfrozen column of pg_class.
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Greg Sabino Mullane <htamfids@gmail.com>
Reviewed-by: Robert Treat <rob@xzilla.net>
Reviewed-by: wenhui qiu <qiuwenhuifx@gmail.com>
Discussion: https://postgr.es/m/flat/CAAKRu_aj-P7YyBz_cPNwztz6ohP%2BvWis%3Diz3YcomkB3NpYA--w%40mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M src/backend/postmaster/autovacuum.c
M src/backend/utils/misc/postgresql.conf.sample
Simplify some logic around setting pg_attribute.atthasdef.
commit : 35c8dd9e1176ae0c3cb060b0da9cb2bba925363c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Mar 2025 13:35:48 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Mar 2025 13:35:48 -0500
DefineRelation was of the opinion that it could usefully pre-fill
atthasdef flags to eliminate work for StoreAttrDefault. This is not
the case, however: the tupledesc that it's filling is not the one that
InsertPgAttributeTuples will work from. The tupledesc used there is
made by RelationBuildLocalRelation, which deliberately doesn't copy
atthasdef. Moreover, if this did happen as the code thinks, it would
be wrong for the case of plain "DEFAULT NULL" clauses, since we detect
and ignore simple-null-Const defaults later on. Hence, remove the
useless code.
It also emerges that it's not really worth a special-case path in
StoreAttrDefault() for atthasdef already being set, because as far as
we can see that never happens: cases where an existing default gets
updated always do RemoveAttrDefault first, so as to clean up
possibly-no-longer-correct dependency entries. If it were the case
the code would still work, anyway.
Also remove a nearby comment made moot by 5eaa0e92e.
Author: jian he <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CACJufxHFssPvkP1we7WMhPD_1kwgbG52o=kQgL+TnVoX5LOyCQ@mail.gmail.com
M src/backend/catalog/pg_attrdef.c
M src/backend/commands/tablecmds.c
Remove now-dead code in StoreAttrDefault().
commit : 4528768d98f83070d625207dcb526982fb18f1ce
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Mar 2025 13:09:20 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Mar 2025 13:09:20 -0500
StoreAttrDefault() is no longer responsible for filling
attmissingval, so remove the code for that.
Get rid of RawColumnDefault.missingMode, too, as we no longer
need that to pass information around.
While here, clean up some sloppy coding in StoreAttrDefault(),
such as failure to use XXXGetDatum macros. These aren't bugs
but they're not good code either.
Reported-by: jian he <jian.universality@gmail.com>
Author: jian he <jian.universality@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CACJufxHFssPvkP1we7WMhPD_1kwgbG52o=kQgL+TnVoX5LOyCQ@mail.gmail.com
M src/backend/catalog/heap.c
M src/backend/catalog/pg_attrdef.c
M src/backend/commands/tablecmds.c
M src/include/catalog/heap.h
M src/include/catalog/pg_attrdef.h
Fix broken handling of domains in atthasmissing logic.
commit : 95f650674d2ceea1ba6440a9b0ae89ed3867fd7e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Mar 2025 12:43:29 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Mar 2025 12:43:29 -0500
If a domain type has a default, adding a column of that type (without
any explicit DEFAULT clause) failed to install the domain's default
value in existing rows, instead leaving the new column null. This
is unexpected, and it used to work correctly before v11. The cause
is confusion in the atthasmissing mechanism about which default value
to install: we'd only consider installing an explicitly-specified
default, and then we'd decide that no table rewrite is needed.
To fix, take the responsibility for filling attmissingval out of
StoreAttrDefault, and instead put it into ATExecAddColumn's existing
logic that derives the correct value to fill the new column with.
Also, centralize the logic that determines the need for
default-related table rewriting there, instead of spreading it over
four or five places.
In the back branches, we'll leave the attmissingval-filling code
in StoreAttrDefault even though it's now dead, for fear that some
extension may be depending on that functionality to exist there.
A separate HEAD-only patch will clean up the now-useless code.
Reported-by: jian he <jian.universality@gmail.com>
Author: jian he <jian.universality@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CACJufxHFssPvkP1we7WMhPD_1kwgbG52o=kQgL+TnVoX5LOyCQ@mail.gmail.com
Backpatch-through: 13
M src/backend/catalog/heap.c
M src/backend/catalog/pg_attrdef.c
M src/backend/commands/tablecmds.c
M src/include/catalog/heap.h
M src/test/regress/expected/fast_default.out
M src/test/regress/sql/fast_default.sql
Add relallfrozen to pg_class
commit : 99f8f3fbbc8f743290844e8c676d39dad11c5d5d
author : Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 3 Mar 2025 11:18:05 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 3 Mar 2025 11:18:05 -0500
Add relallfrozen, an estimate of the number of pages marked all-frozen
in the visibility map.
pg_class already has relallvisible, an estimate of the number of pages
in the relation marked all-visible in the visibility map. This is used
primarily for planning.
relallfrozen, together with relallvisible, is useful for estimating the
outstanding number of all-visible but not all-frozen pages in the
relation for the purposes of scheduling manual VACUUMs and tuning vacuum
freeze parameters.
A future commit will use relallfrozen to trigger more frequent vacuums
on insert-focused workloads with significant volume of frozen data.
Bump catalog version
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Robert Treat <rob@xzilla.net>
Reviewed-by: Corey Huinker <corey.huinker@gmail.com>
Reviewed-by: Greg Sabino Mullane <htamfids@gmail.com>
Discussion: https://postgr.es/m/flat/CAAKRu_aj-P7YyBz_cPNwztz6ohP%2BvWis%3Diz3YcomkB3NpYA--w%40mail.gmail.com
M doc/src/sgml/catalogs.sgml
M src/backend/access/heap/vacuumlazy.c
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/commands/analyze.c
M src/backend/commands/cluster.c
M src/backend/commands/vacuum.c
M src/backend/statistics/relation_stats.c
M src/backend/utils/cache/relcache.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_class.h
M src/include/commands/vacuum.h
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Allow parallel CREATE INDEX for GIN indexes
commit : 8492feb98f6df3f0f03e84ed56f0d1cbb2ac514c
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 3 Mar 2025 16:53:03 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 3 Mar 2025 16:53:03 +0100
Allow using parallel workers to build a GIN index, similarly to BTREE
and BRIN. For large tables this may result in significant speedup when
the build is CPU-bound.
The work is divided so that each worker builds index entries on a subset
of the table, determined by the regular parallel scan used to read the
data. Each worker uses a local tuplesort to sort and merge the entries
for the same key. The TID lists do not overlap (for a given key), which
means the merge sort simply concatenates the two lists. The merged
entries are written into a shared tuplesort for the leader.
The leader needs to merge the sorted entries again, before writing them
into the index. But this way a significant part of the work happens in
the workers, and the leader is left with merging fewer large entries,
which is more efficient.
Most of the parallelism infrastructure is a simplified copy of the code
used by BTREE indexes, omitting the parts irrelevant for GIN indexes
(e.g. uniqueness checks).
Original patch by me, with reviews and substantial improvements by
Matthias van de Meent, certainly enough to make him a co-author.
Author: Tomas Vondra, Matthias van de Meent
Reviewed-by: Matthias van de Meent, Andy Fan, Kirill Reshke
Discussion: https://postgr.es/m/6ab4003f-a8b8-4d75-a67f-f25ad98582dc%40enterprisedb.com
M src/backend/access/gin/gininsert.c
M src/backend/access/gin/ginutil.c
M src/backend/access/transam/parallel.c
M src/backend/utils/sort/tuplesortvariants.c
M src/include/access/gin.h
M src/include/access/gin_private.h
A src/include/access/gin_tuple.h
M src/include/utils/tuplesort.h
M src/tools/pgindent/typedefs.list
Handle auxiliary processes in SQL functions of backend statistics
commit : 3f1db99bfabbb9d4afc41f362d9801512f4c7c65
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 3 Mar 2025 09:57:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 3 Mar 2025 09:57:48 +0900
This commit impacts the following SQL functions, authorizing the access
to the PGPROC entries of auxiliary processes when attempting to fetch or
reset backend-level pgstats entries:
- pg_stat_reset_backend_stats()
- pg_stat_get_backend_io()
This is relevant since a051e71e28a1 for at least the WAL summarizer, WAL
receiver and WAL writer processes, that has changed the backend
statistics to authorize these three following the addition of WAL I/O
statistics in pg_stat_io and backend statistics. The code is more
flexible with future changes written this way, adapting automatically to
any updates done in pgstat_tracks_backend_bktype().
While on it, pgstat_report_wal() gains a call to pgstat_flush_backend(),
making sure that backend I/O statistics are updated when calling this
routine. This makes the statistics report correctly for the WAL writer.
WAL receiver and WAL summarizer do not call pgstat_report_wal() yet
(spoiler: both should). It should be possible to lift some of the
existing restrictions for other auxiliary processes, as well, but this
is left as future work.
Reported-by: Rahila Syed <rahilasyed90@gmail.com>
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/CAH2L28v9BwN8_y0k6FQ591=0g2Hj_esHLGj3bP38c9nmVykoiA@mail.gmail.com
M src/backend/utils/activity/pgstat_backend.c
M src/backend/utils/activity/pgstat_wal.c
M src/backend/utils/adt/pgstatfuncs.c
postgres_fdw: Extend postgres_fdw_get_connections to return remote backend PID.
commit : fe186bda78c4a0ccbb691489c744e6317f648434
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 3 Mar 2025 08:51:30 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 3 Mar 2025 08:51:30 +0900
This commit adds a new "remote_backend_pid" output column to
the postgres_fdw_get_connections function. It returns the process ID of
the remote backend, on the foreign server, handling the connection.
This enhancement is useful for troubleshooting, monitoring, and reporting.
For example, if a connection is unexpectedly closed by the foreign server,
the remote backend's PID can help diagnose the cause.
No extension version bump is needed, as commit c297a47c5f already
handled it for v18~.
Author: Sagar Dilip Shedge <sagar.shedge92@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/CAPhYifF25q5xUQWXETfKwhc0YVa_6+tfG9Kw4bCvCjpCWxYs2A@mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw–1.1–1.2.sql
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Use PRI*64 instead of "ll*" in format strings (minimal trial)
commit : 15a79c73111f0c9738ee81b796f7de5bfeb3aedc
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 2 Mar 2025 13:53:03 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 2 Mar 2025 13:53:03 +0100
Old: errmsg("hello %llu", (unsigned long long) x)
New: errmsg("hello %" PRIu64, x)
And likewise for everything printf-like.
In the past we had to use long long so localized format strings remained
architecture independent in message catalogs. Although long long is
expected to be 64 bit everywhere, if we hadn't also cast the int64
values, we'd have generated compiler warnings on systems where int64 was
long.
Now that int64 is int64_t, C99 understand how to format them using
<inttypes.h> macros, the casts are not necessary, and the gettext()
tools recognize the macros and defer expansion until load time. (And if
we ever manage to get -Wformat-signedness to work for us, that'd help
with these too, but not the type-system-clobbering casts.)
This particular patch converts only pg_checksums.c to the new system,
to allow testing of the translation toolchain for everyone. If this
works okay, a later patch will convert most of the rest.
Author: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/b936d2fb-590d-49c3-a615-92c3a88c6c19%40eisentraut.org
M src/bin/pg_checksums/pg_checksums.c
Fix pg_strtof() to not crash on NULL endptr.
commit : 00d61a08c5fadca869c66a988b722e543d8e7100
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 1 Mar 2025 14:22:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 1 Mar 2025 14:22:56 -0500
We had managed not to notice this simple oversight because none
of our calls exercised the case --- until commit 8f427187d.
That led to pg_dump crashing on any platform that uses this code
(currently Cygwin and Mingw).
Even though there's no immediate bug in the back branches, backpatch,
because a non-POSIX-compliant strtof() substitute is trouble waiting
to happen for extensions or future back-patches.
Diagnosed-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/339b3902-4e98-4e31-a744-94e43b7b9292@gmail.com
Backpatch-through: 13
M src/port/strtof.c
Set amcancrosscompare to true for hash
commit : 56ba0463d38c17400a090353db47cf455ce01c3d
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 1 Mar 2025 09:15:27 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 1 Mar 2025 09:15:27 +0100
This was missed in the refactoring in patch ce62f2f2a0a, which thus
created a regression.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E1tngY6-0000UL-2n%40gemulon.postgresql.org
M src/backend/access/hash/hash.c
Work around OAuth/EVFILT_TIMER quirk on NetBSD.
commit : c301a0a74a8aa9ec4fe1f364e05aa17a649f7260
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 1 Mar 2025 13:56:33 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 1 Mar 2025 13:56:33 +1300
NetBSD's EVFILT_TIMER doesn't like zero timeouts, as introduced by
commit b3f0be788. Steal the workaround from the same problem on Linux
from a few lines up: round zero up to one. Do this only for NetBSD, as
the other systems with the kevent() API accept zero and shouldn't have
to insert a small bogus wait.
Future improvement ideas:
* when NetBSD < 10 falls out of support, we could try NODE_ABSTIME for
the "fire now" meaning if timeout == 0
* when libcurl tells us to start a 0ms timer and call it back, we could
figure out how to handle that more directly without involving the
kernel (the current architecture doesn't make that straightforward)
Failures with EINVAL errors could be seen on the new optional NetBSD CI
task that we're trying to keep green as a candidate for inclusion as
default-enabled CI task. The NetBSD build farm animals aren't testing
OAuth yet, so no breakage there.
Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/CA%2BhUKGJ%2BWyJ26QGvO_nkgvbxgw%2B03U4EQ4Hxw%2BQBft6Np%2BXW7w%40mail.gmail.com
M src/interfaces/libpq/fe-auth-oauth-curl.c
Re-export NextCopyFromRawFields() to copy.h.
commit : 8a1012b35d6123d8ee1818fce68f435459e297b7
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 28 Feb 2025 15:11:41 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 28 Feb 2025 15:11:41 -0800
Commit 7717f630069 removed NextCopyFromRawFields() from copy.h. While
it was hoped that NextCopyFrom() could serve as an alternative,
certain use cases still require NextCopyFromRawFields(). For instance,
extensions like file_text_array_fdw, which process source data with an
unknown number of columns, rely on this function.
Per buildfarm member crake.
Reported-by: Andrew Dunstan <andrew@dunslane.net>
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Reviewed-by: Sutou Kouhei <kou@clear-code.com>
Discussion: https://postgr.es/m/5c7e1ac8-5083-4c08-af19-cb9ade2f16ce@dunslane.net
M src/backend/commands/copyfromparse.c
M src/include/commands/copy.h
Adjust auto_explain's GUC descriptions.
commit : e636da92002857c24f8b66b6ac4a47ff91803c06
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 28 Feb 2025 16:05:51 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 28 Feb 2025 16:05:51 -0600
This commit adjusts auto_explain's GUC descriptions to follow the
style guidelines established by commit 977d865c36. Specifically,
it ensures the accepted special values are listed in a consistent
manner.
Author: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Discussion: https://postgr.es/m/e82d4647-ce7f-45c7-9b01-fb900a050767%40tantorlabs.com
M contrib/auto_explain/auto_explain.c
Tweak regex to avoid a bug in Perl 5.16.3.
commit : 8b49392b270b4ac0b9f5c210e2a503546841e832
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 28 Feb 2025 15:20:22 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 28 Feb 2025 15:20:22 -0500
For some reason, 5.16.3 (and perhaps slightly earlier/later versions)
go into an infinite loop with the version-replacement regex installed
by commit fc0d0ce97. We can work around that by using an explicit
"\n" instead of the line-start metacharacter "^".
Reported-by: Sami Imseih <samimseih@gmail.com>
Discussion: https://postgr.es/m/CAA5RZ0u9dV3CdKqkqdusA_RdvBkwWe0c0rxcFWj++VYoutFYSw@mail.gmail.com
M src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
Refactor COPY FROM to use format callback functions.
commit : 7717f63006935de00fafd000bff450280508adf1
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 28 Feb 2025 10:29:36 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 28 Feb 2025 10:29:36 -0800
This commit introduces a new CopyFromRoutine struct, which is a set of
callback routines to read tuples in a specific format. It also makes
COPY FROM with the existing formats (text, CSV, and binary) utilize
these format callbacks.
This change is a preliminary step towards making the COPY FROM command
extensible in terms of input formats.
Similar to 2e4127b6d2d, this refactoring contributes to a performance
improvement by reducing the number of "if" branches that need to be
checked on a per-row basis when sending field representations in text
or CSV mode. The performance benchmark results showed ~5% performance
gain in text or CSV mode.
Author: Sutou Kouhei <kou@clear-code.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Reviewed-by: Junwang Zhao <zhjwpku@gmail.com>
Discussion: https://postgr.es/m/20231204.153548.2126325458835528809.kou@clear-code.com
M src/backend/commands/copyfrom.c
M src/backend/commands/copyfromparse.c
M src/include/commands/copy.h
M src/include/commands/copyapi.h
M src/include/commands/copyfrom_internal.h
M src/tools/pgindent/typedefs.list
Avoid including explain.h in explain_format.h and explain_dr.h
commit : 77cb08be510623421fc727f35980de5107eea735
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 28 Feb 2025 13:17:29 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 28 Feb 2025 13:17:29 -0500
As per a suggestion from Tom Lane, we do this by declaring "struct
ExplainState" here and refer to that rather than "ExplainState".
Also per Tom, CreateExplainSerializeDestReceiver was still defined
in explain.h in addition to explain_dr.h. Remove leftover prototype.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: http://postgr.es/m/CA+TgmoYtaad3i21V0jqua-fbr+CR0ix6uBvEX8_s6BG96abd=g@mail.gmail.com
M src/backend/commands/explain_dr.c
M src/backend/tcop/dest.c
M src/include/commands/explain.h
M src/include/commands/explain_dr.h
M src/include/commands/explain_format.h
Fix missing space in EXPLAIN ANALYZE output.
commit : 51d3e279c3e1f6548794d5e7b5206ad1e080568f
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 28 Feb 2025 13:02:03 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 28 Feb 2025 13:02:03 -0500
Commit ddb17e387aa28d61521227377b00f997756b8a27 introduced this
regression. Ideally, the regression tests would have caught this
mistake, but apparently they don't test with timing enabled,
presumably because that would make the output vary.
Author: Thom Brown <thom@linux.com>
Reviewed-by: Fabrízio de Royes Mello <fabriziomello@gmail.com>
Discussion: http://postgr.es/m/CAA-aLv6nq=UeiyvM7_Mxgo9TVBzs2oh46b9vfyLzuyVEz3j1-g@mail.gmail.com
M src/backend/commands/explain.c
Adjust pg_dump tag for relation stats.
commit : 424ededc580b03e1bcf8aff18a735e519c80061f
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 27 Feb 2025 20:40:21 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 27 Feb 2025 20:40:21 -0800
Do not use fmtId(), just use dobj->name directly, like for table data.
M src/bin/pg_dump/pg_dump.c
Invent pgstat_fetch_stat_backend_by_pid()
commit : c2a50ac678eb5ccee271aef3e7ed146ac395a32b
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 28 Feb 2025 11:20:31 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 28 Feb 2025 11:20:31 +0900
This code is extracted from pg_stat_get_backend_io() in pgstatfuncs.c,
so as it can be shared with other areas that need backend pgstats
entries while having the benefits of the various sanity checks
refactored here. As per its name, this retrieves backend statistics
based on a PID, with the option of retrieving a BackendType if given in
input.
Currently, this is used for the backend-level IO statistics. The next
move would be to reuse that for the backend-level WAL statistics.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/Z3zqc4o09dM/Ezyz@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/utils/activity/pgstat_backend.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/pgstat.h
pg_upgrade: Fix inconsistency in memory freeing
commit : 2a083ab807db6d9e2e0e3aa82ee8f6ff9fc44c8d
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 28 Feb 2025 10:15:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 28 Feb 2025 10:15:29 +0900
The function in charge of freeing the memory from a result created by
PQescapeIdentifier() has to be PQfreemem(), to ensure that both
allocation and free come from libpq.
One spot in pg_upgrade was not respecting that for pg_database's
datlocale (daticulocale in v16) when the collation provider is libc (aka
datlocale/daticulocale is NULL) with an allocation done using
pg_strdup() and a free with PQfreemem(). The code is changed to always
use PQescapeLiteral() when processing the input.
Oversight in 9637badd9f92. This commit is similar to 48e4ae9a0707 and
5b94e2753439.
Author: Michael Paquier <michael@paquier.xyz>
Co-authored-by: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/Z601RQxTmIUohdkV@paquier.xyz
Backpatch-through: 16
M src/bin/pg_upgrade/pg_upgrade.c
Refactor COPY TO to use format callback functions.
commit : 2e4127b6d2d8f3a9d67a21d1905703d5be46970c
author : Masahiko Sawada <msawada@postgresql.org>
date : Thu, 27 Feb 2025 15:03:52 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Thu, 27 Feb 2025 15:03:52 -0800
This commit introduces a new CopyToRoutine struct, which is a set of
callback routines to copy tuples in a specific format. It also makes
the existing formats (text, CSV, and binary) utilize these format
callbacks.
This change is a preliminary step towards making the COPY TO command
extensible in terms of output formats.
Additionally, this refactoring contributes to a performance
improvement by reducing the number of "if" branches that need to be
checked on a per-row basis when sending field representations in text
or CSV mode. The performance benchmark results showed ~5% performance
gain in text or CSV mode.
Author: Sutou Kouhei <kou@clear-code.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Reviewed-by: Junwang Zhao <zhjwpku@gmail.com>
Discussion: https://postgr.es/m/20231204.153548.2126325458835528809.kou@clear-code.com
M src/backend/commands/copyto.c
A src/include/commands/copyapi.h
M src/tools/pgindent/typedefs.list
Create explain_dr.c and move DestReceiver-related code there.
commit : 555960a0fbf0590a744f36e90e69e2501dc06146
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 27 Feb 2025 13:14:16 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 27 Feb 2025 13:14:16 -0500
explain.c has grown rather large, and the code that deals with the
DestReceiver that supports the SERIALIZE option is pretty easily severable
from the rest of explain.c; hence, move it to a separate file.
Reviewed-by: Peter Geoghegan <pg@bowt.ie>
Discussion: http://postgr.es/m/CA+TgmoYutMw1Jgo8BWUmB3TqnOhsEAJiYO=rOQufF4gPLWmkLQ@mail.gmail.com
M src/backend/commands/Makefile
M src/backend/commands/explain.c
A src/backend/commands/explain_dr.c
M src/backend/commands/meson.build
A src/include/commands/explain_dr.h
Create explain_format.c and move relevant code there.
commit : 9173e8b604636633a8e3aca54bb56a437bffa718
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 27 Feb 2025 12:37:10 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 27 Feb 2025 12:37:10 -0500
explain.c has grown rather large, so move various functions that
are principally concerned with output generation to a new source
file, explain_format.c, instead of lumping them in with everything
else that is part of explain.c
Reviewed-by: Peter Geoghegan <pg@bowt.ie>
Discussion: http://postgr.es/m/CA+TgmoYutMw1Jgo8BWUmB3TqnOhsEAJiYO=rOQufF4gPLWmkLQ@mail.gmail.com
M contrib/auto_explain/auto_explain.c
M contrib/file_fdw/file_fdw.c
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/commands/Makefile
M src/backend/commands/explain.c
A src/backend/commands/explain_format.c
M src/backend/commands/meson.build
M src/backend/commands/prepare.c
M src/include/commands/explain.h
A src/include/commands/explain_format.h
EXPLAIN: Always use two fractional digits for row counts.
commit : 95dbd827f2edc4d10bebd7e840a0bd6782cf69b7
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 27 Feb 2025 11:25:18 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 27 Feb 2025 11:25:18 -0500
Commit ddb17e387aa28d61521227377b00f997756b8a27 attempted to avoid
confusing users by displaying digits after the decimal point only when
nloops > 1, since it's impossible to have a fraction row count after a
single iteration. However, this made the regression tests unstable since
parallal queries will have nloops>1 for all nodes below the Gather or
Gather Merge in normal cases, but if the workers don't start in time and
the leader finishes all the work, they will suddenly have nloops==1,
making it unpredictable whether the digits after the decimal point would
be displayed or not. Although 44cbba9a7f51a3888d5087fc94b23614ba2b81f2
seemed to fix the immediate failures, it may still be the case that there
are lower-probability failures elsewhere in the regression tests.
Various fixes are possible here. For example, it has previously been
proposed that we should try to display the digits after the decimal
point only if rows/nloops is an integer, but currently rows is storead
as a float so it's not theoretically an exact quantity -- precision
could be lost in extreme cases. It has also been proposed that we
should try to display the digits after the decimal point only if we're
under some sort of construct that could potentially cause looping
regardless of whether it actually does. While such ideas are not
without merit, this patch adopts the much simpler solution of always
display two decimal digits. If that approach stands up to scrutiny
from the buildfarm and human users, it spares us the trouble of doing
anything more complex; if not, we can reassess.
This commit incidentally reverts 44cbba9a7f51a3888d5087fc94b23614ba2b81f2,
which should no longer be needed.
Author: Robert Haas <robertmhaas@gmail.com>
Author: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Discussion: http://postgr.es/m/CA+TgmoazzVHn8sFOMFAEwoqBTDxKT45D7mvkyeHgqtoD2cn58Q@mail.gmail.com
M contrib/pg_stat_statements/expected/level_tracking.out
M contrib/postgres_fdw/expected/postgres_fdw.out
M doc/src/sgml/auto-explain.sgml
M doc/src/sgml/bloom.sgml
M doc/src/sgml/jit.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/planstats.sgml
M doc/src/sgml/ref/explain.sgml
M doc/src/sgml/rules.sgml
M src/backend/commands/explain.c
M src/test/regress/expected/brin_multi.out
M src/test/regress/expected/explain.out
M src/test/regress/expected/gin.out
M src/test/regress/expected/incremental_sort.out
M src/test/regress/expected/matview.out
M src/test/regress/expected/memoize.out
M src/test/regress/expected/merge.out
M src/test/regress/expected/misc_functions.out
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/select.out
M src/test/regress/expected/select_into.out
M src/test/regress/expected/select_parallel.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/tidscan.out
M src/test/regress/sql/explain.sql
Generalize hash and ordering support in amapi
commit : ce62f2f2a0a48d021f250ba84dfcab5d45ddc914
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 27 Feb 2025 17:03:31 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 27 Feb 2025 17:03:31 +0100
Stop comparing access method OID values against HASH_AM_OID and
BTREE_AM_OID, and instead check the IndexAmRoutine for an index to see
if it advertises its ability to perform the necessary ordering,
hashing, or cross-type comparing functionality. A field amcanorder
already existed, this uses it more widely. Fields amcanhash and
amcancrosscompare are added for the other purposes.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M contrib/bloom/blutils.c
M doc/src/sgml/indexam.sgml
M src/backend/access/brin/brin.c
M src/backend/access/gin/ginutil.c
M src/backend/access/gist/gist.c
M src/backend/access/hash/hash.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/spgist/spgutils.c
M src/backend/commands/opclasscmds.c
M src/backend/executor/nodeIndexscan.c
M src/backend/utils/cache/lsyscache.c
M src/include/access/amapi.h
M src/test/modules/dummy_index_am/dummy_index_am.c
M src/test/regress/expected/alter_generic.out
Avoid unnecessary computation of pgbench's script line number.
commit : 6eb8a1a4f90c542d7ce9dcc381528fcb81390ab9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Feb 2025 10:57:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Feb 2025 10:57:55 -0500
ParseScript only needs the lineno for meta-commands, so let's not
bother computing it otherwise. While this doesn't save much given
the previous patch, there's no point in doing unnecessary work.
While we're at it, avoid calling psql_scan_get_location() twice for
a meta-command.
One reason for making this change is that the line number computed
in ParseScript's main loop was actually wrong in most cases: it
would point just past the semicolon of the previous SQL command,
not at what the user thinks the current command's line number is.
We could add some code to skip whitespace before capturing the line
number, but it would be pretty pointless at present. Just move the
call to avoid the temptation to rely on that value. (Once we've
lexed the backslash, the computed line number will be right.)
This change also means that pgbench never inquires about the
location before it's lexed something, so that the care taken in
the previous patch to behave sanely in that case is unnecessary.
It seems best to keep that logic, though, as future callers
might depend on it.
Author: Daniel Vérité <daniel@manitou-mail.org>
Discussion: https://postgr.es/m/84a8a89e-adb8-47a9-9d34-c13f7150ee45@manitou-mail.org
M src/bin/pgbench/pgbench.c
Get rid of O(N^2) script-parsing overhead in pgbench.
commit : c8c74ad7e1cbc71b616f8ae786776c521729976b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Feb 2025 10:53:38 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Feb 2025 10:53:38 -0500
pgbench wants to record the starting line number of each command
in its scripts. It was computing that by scanning from the script
start and counting newlines, so that O(N^2) work had to be done
for an N-command script. In a script with 50K lines, this adds
up to about 10 seconds on my machine.
To add insult to injury, the results were subtly wrong, because
expr_scanner_offset() scanned to find the NUL that flex inserts
at the end of the current token --- and before the first yylex
call, no such NUL has been inserted. So we ended by computing the
script's last line number not its first one. This was visible only
in case of \gset at the start of a script, which perhaps accounts
for the lack of complaints.
To fix, steal an idea from plpgsql and track the current lexer
ending position and line count as we advance through the script.
(It's a bit simpler than plpgsql since we can't need to back up.)
Also adjust a couple of other places that were invoking scans
from script start when they didn't really need to. I made a new
psqlscan function psql_scan_get_location() that replaces both
expr_scanner_offset() and expr_scanner_get_lineno(), since in
practice expr_scanner_get_lineno() was only being invoked to find
the line number of the current lexer end position.
Reported-by: Daniel Vérité <daniel@manitou-mail.org>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/84a8a89e-adb8-47a9-9d34-c13f7150ee45@manitou-mail.org
M src/bin/pgbench/exprscan.l
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/pgbench.h
M src/fe_utils/psqlscan.l
M src/include/fe_utils/psqlscan.h
M src/include/fe_utils/psqlscan_int.h
Get rid of ojrelid local variable in remove_rel_from_query()
commit : e167191dc146b65146fbd32e147be30dd8f1f166
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 23 Feb 2025 23:06:33 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 23 Feb 2025 23:06:33 +0200
As spotted by Coverity, the calculation of ojrelid mixes signed and unsigned
types causes possible overflow and undefined behavior. Instead of trying to
fix the expression, this commit eliminates the relied local variable. The
explicit branching is used to replace the -1 value. That, in turn, requires
changing the signature of the remove_rel_from_eclass() function.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/914330.1740330169%40sss.pgh.pa.us
Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>
M src/backend/optimizer/plan/analyzejoins.c
Remove arbitrary cap on read_stream.c buffer queue.
commit : 55918f798bc2d1846eea0d805fbec91d4e1816e0
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 27 Feb 2025 14:15:15 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 27 Feb 2025 14:15:15 +1300
Previously the internal queue of buffers was capped at max_ios * 4,
though not less than io_combine_limit, at allocation time. That was
done in the first version based on conservative theories about resource
usage and heuristics pending later work. The configured I/O depth could
not always be reached with dense random streams generated by ANALYZE,
VACUUM, the proposed Bitmap Heap Scan patch, and also sequential streams
with the proposed AIO subsystem to name some examples.
The new formula is (max_ios + 1) * io_combine_limit, enough buffers for
the full configured I/O concurrency level using the full configured I/O
combine size, plus the buffers from one finished but not yet consumed
full-sized I/O. Significantly more memory would be needed for high GUC
values if the client code requests a large per-buffer data size, but
that is discouraged (existing and proposed stream users try to keep it
under a few words, if not zero).
With this new formula, an intermediate variable could have overflowed
under maximum GUC values, so its data type is adjusted to cope.
Discussion: https://postgr.es/m/CA%2BhUKGK_%3D4CVmMHvsHjOVrK6t4F%3DLBpFzsrr3R%2BaJYN8kcTfWg%40mail.gmail.com
M src/backend/storage/aio/read_stream.c
pg_amcheck: Fix inconsistency in memory freeing
commit : 48e4ae9a0707b22cf874a4e8e531a07077318424
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Feb 2025 14:05:51 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Feb 2025 14:05:51 +0900
The function in charge of freeing the memory from a result created by
PQescapeIdentifier() has to be PQfreemem(), to ensure that both
allocation and free come from libpq, but one spot in pg_amcheck was
missing that.
Oversight in b859d94c6389.
Author: Ranier Vilela <ranier.vf@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/CAEudQArD_nKSnYCNUZiPPsJ2tNXgRmLbXGSOrH1vpOF_XtP0Vg@mail.gmail.com
Discussion: https://postgr.es/m/CAEudQArbTWVSbxq608GRmXJjnNSQ0B6R7CSffNnj2hPWMUsRNg@mail.gmail.com
Backpatch-through: 14
M src/bin/pg_amcheck/pg_amcheck.c
Fix the race condition in ReplicationSlotAcquire().
commit : 8709dccc793da0c0c6619cafa182c8e67a871154
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 27 Feb 2025 09:47:04 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 27 Feb 2025 09:47:04 +0530
After commit f41d8468dd, a process could acquire and use a replication
slot that had just been invalidated, leading to failures while accessing
WAL.
To ensure that we don't accidentally start using invalid slots, we must
perform the invalidation check after acquiring the slot or under the
spinlock where we associate the slot with a particular process. We choose
the earlier method to keep the code simple.
Reported-by: Hou Zhijie <houzj.fnst@fujitsu.com>
Author: Nisha Moond <nisha.moond412@gmail.com>
Reviewed-by: Hou Zhijie <houzj.fnst@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CABdArM7J-LbGoMPGUPiFiLOyB_TZ5+YaZb=HMES0mQqzVTn8Gg@mail.gmail.com
M src/backend/replication/slot.c
Doc: Additional clarification for -d option of pg_createsubscriber.
commit : 845511a72ad01838c9e1766e031d2862cd021801
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 27 Feb 2025 08:50:03 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 27 Feb 2025 08:50:03 +0530
Author: vignesh C <vignesh21@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CALDaNm0zsFUYpe-tLha+-sp3K8KmBXu0o=LUN=8FFtxMLYikPA@mail.gmail.com
M doc/src/sgml/ref/pg_createsubscriber.sgml
Refactor code of pg_stat_get_wal() building result tuple
commit : 495864a4cf16530c5174f264668c87b780c4cf22
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Feb 2025 11:54:36 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Feb 2025 11:54:36 +0900
This commit adds to pgstatfuncs.c a new routine called
pg_stat_wal_build_tuple(), helper routine for pg_stat_get_wal(). This
is in charge of filling one tuple based on the contents of
PgStat_WalStats retrieved from pgstats.
This refactoring will be used by an upcoming patch introducing
backend-level WAL statistics, simplifying the main patch. Note that
it is not possible for stats_reset to be NULL in pg_stat_wal; backend
statistics need to be able to handle this case.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/Z3zqc4o09dM/Ezyz@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/utils/adt/pgstatfuncs.c
Fix possible double-release of spinlock in procsignal.c
commit : 62ec3e1f6786181431210643a2d427b9a98b8af8
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Feb 2025 09:43:06 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Feb 2025 09:43:06 +0900
9d9b9d46f3c5 has added spinlocks to protect the fields in ProcSignal
flags, introducing a code path in ProcSignalInit() where a spinlock
could be released twice if the pss_pid field of a ProcSignalSlot is
found as already set. Multiple spinlock releases have no effect with
most spinlock implementations, but this could cause the code to run into
issues when the spinlock is acquired concurrently by a different
process.
This sanity check on pss_pid generates a LOG that can be delayed until
after the spinlock is released as, like older versions up to v17, the
code expects the initialization of the ProcSignalSlot to happen even if
pss_pid is found incorrect. The code is changed so as the old pss_pid
is read while holding the slot's spinlock, with the LOG from the sanity
check generated after releasing the spinlock, preventing the double
release.
Author: Maksim Melnikov <m.melnikov@postgrespro.ru>
Co-authored-by: Maxim Orlov <orlovmg@gmail.com>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/dca47527-2d8b-4e3b-b5a0-e2deb73371a4@postgrespro.ru
M src/backend/storage/ipc/procsignal.c
Remove stray diff introduced by a5cbdeb98a.
commit : 15df9d7b5123b2b478886175c17cd0c0359d9996
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 26 Feb 2025 13:37:14 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 26 Feb 2025 13:37:14 -0800
Reported-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/Z77IkjmmfbFfNh3f@paquier.xyz
M src/include/catalog/pg_proc.dat
Use attnum to identify index columns in pg_restore_attribute_stats().
commit : 40e27d04b4f643cfb78af8db42a1f2e700ec9876
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 26 Feb 2025 16:36:11 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 26 Feb 2025 16:36:11 -0500
Previously we used attname for both table and index columns, but
that is problematic for indexes because their attnames are assigned
by internal rules that don't guarantee to preserve the names across
dump and reload. (This is what's causing the remaining buildfarm
failures in cross-version-upgrade tests.) Fortunately we can use
attnum instead, since there's no such thing as adding or dropping
columns in an existing index. We met this same problem previously
with ALTER INDEX ... SET STATISTICS, and solved it the same way,
cf commit 5b6d13eec.
In pg_restore_attribute_stats() itself, we accept either attnum or
attname, but the policy used by pg_dump is to always use attname
for tables and attnum for indexes.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Author: Corey Huinker <corey.huinker@gmail.com>
Discussion: https://postgr.es/m/1457469.1740419458@sss.pgh.pa.us
M doc/src/sgml/func.sgml
M src/backend/statistics/attribute_stats.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Revert "Prepare for Python "Limited API" in PL/Python"
commit : f734c9fc3a91959c2473a1e33fd9b60116902175
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 26 Feb 2025 21:58:38 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 26 Feb 2025 21:58:38 +0100
This reverts commit c47e8df815c1c45f4e4fc90d5817d67ab088279f.
That commit makes the plpython tests crash with Python 3.6.* and
3.7.*. It will need further investigation and testing, so revert for
now.
M src/pl/plpython/plpy_cursorobject.c
M src/pl/plpython/plpy_planobject.c
M src/pl/plpython/plpy_procedure.c
M src/pl/plpython/plpy_resultobject.c
M src/pl/plpython/plpy_subxactobject.c
M src/pl/plpython/plpy_typeio.c
Fix a typo in 005_char_signedness.pl test.
commit : 945a9e3832c3ede20e2c575b796a4f16687a1949
author : Masahiko Sawada <msawada@postgresql.org>
date : Wed, 26 Feb 2025 11:10:03 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Wed, 26 Feb 2025 11:10:03 -0800
The test in 005_char_signedness.pl was missing a dash in the
--set-char-signedness option. Although the test didn't fail since it
doesn't check the error message, it resulted in an unexpected error
message instead of the intended one.
Oversight in 1aab680591.
Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://postgr.es/m/87tt8h1vb7.fsf@wibble.ilmari.org
M src/bin/pg_upgrade/t/005_char_signedness.pl
Prepare for Python "Limited API" in PL/Python
commit : c47e8df815c1c45f4e4fc90d5817d67ab088279f
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 26 Feb 2025 16:14:16 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 26 Feb 2025 16:14:16 +0100
Using the Python Limited API would allow building PL/Python against
any Python 3.x version and using another Python 3.x version at run
time. This commit does not activate that, but it prepares the code to
only use APIs supported by the Limited API.
Implementation details:
- Convert static types to heap types
(https://docs.python.org/3/howto/isolating-extensions.html#heap-types).
- Replace PyRun_String() with component functions.
- Replace PyList_SET_ITEM() with PyList_SetItem().
Reviewed-by: Jakob Egger <jakob@eggerapps.at>
Discussion: https://www.postgresql.org/message-id/flat/ee410de1-1e0b-4770-b125-eeefd4726a24@eisentraut.org
M src/pl/plpython/plpy_cursorobject.c
M src/pl/plpython/plpy_planobject.c
M src/pl/plpython/plpy_procedure.c
M src/pl/plpython/plpy_resultobject.c
M src/pl/plpython/plpy_subxactobject.c
M src/pl/plpython/plpy_typeio.c
Adding new PgStat_WalCounters structure in pgstat.h
commit : 0e42d31b0b2273c376ce9de946b59d155fac589c
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Feb 2025 16:48:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Feb 2025 16:48:54 +0900
This new structure contains the counters and the data related to the WAL
activity statistics gathered from WalUsage, separated into its own
structure so as it can be shared across more than one Stats structure in
pg_stat.h.
This refactoring will be used by an upcoming patch introducing
backend-level WAL statistics.
Bump PGSTAT_FILE_FORMAT_ID.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/Z3zqc4o09dM/Ezyz@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/utils/activity/pgstat_wal.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/pgstat.h
M src/tools/pgindent/typedefs.list
Remove pgstat_flush_wal()
commit : d7cbeaf261da346d8c745870da1d31075ae0313c
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Feb 2025 15:29:51 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Feb 2025 15:29:51 +0900
All the processes that generate WAL should call pgstat_report_wal() to
report all their statistics related to WAL, and this is already what
happens in the tree. Keeping pgstat_report_wal() is confusing while the
other routine is encouraged.
This routine is not required since fc415edf8ca8, where it was lastly
used in pgstat_report_stat() before an equivalent callback existed.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/Z71oPkJJICrRB5Ws@paquier.xyz
M src/backend/utils/activity/pgstat_wal.c
M src/include/utils/pgstat_internal.h
Add two-phase option in pg_createsubscriber.
commit : e117cfb2f6c67fe4ba46720bc6917da3dbd48c10
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 26 Feb 2025 11:12:50 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 26 Feb 2025 11:12:50 +0530
This patch introduces the '--enable-two-phase' option to the
'pg_createsubscriber' utility, allowing users to enable two-phase commit
for all subscriptions during their creation.
Note that even without this option users can enable the two_phase option
for the subscriptions created by pg_createsubscriber. However, it requires
the subscription to be disabled first which could be inconvenient for
users.
When two-phase commit is enabled, prepared transactions are sent to the
subscriber at the time of 'PREPARE TRANSACTION', and they are processed as
two-phase transactions on the subscriber as well. If disabled, prepared
transactions are sent only when committed and are processed immediately by
the subscriber.
Author: Shubham Khanna <khannashubham1197@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Ajin Cherian <itsajin@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CAHv8RjLPdFP=kA5LNSmWZ=+GMXmO+LczvV6p9HJjsXxZz10KGA@mail.gmail.com
M doc/src/sgml/ref/pg_createsubscriber.sgml
M src/bin/pg_basebackup/pg_createsubscriber.c
M src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
Improve FATAL message for invalid TLI history at recovery
commit : adc6032fa8824e7653252b02abc6a59d8b9d01a6
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Feb 2025 14:26:16 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Feb 2025 14:26:16 +0900
The original message did not mention where the checkpoint record LSN was
found, a control file or a backup_label file. A couple of LOG messages
are generated before this FATAL check is reached, providing more details
about the way recovery is set up. However, knowing this information in
this specific message is useful for debugging. This is also useful for
instances where log_min_messages is set to FATAL or more, where LOG
messages do not show up.
Author: Benoit Lobréau <benoit.lobreau@dalibo.com>
Reviewed-by: David Steele <david@pgbackrest.org>
Discussion: https://postgr.es/m/4ed10bc8-5513-4d8e-8643-8abcaa08336d@dalibo.com
M src/backend/access/transam/xlogrecovery.c
pg_dump: prepare attribute stats query.
commit : 6ee3b91bad269b8f1acee38607bd17923efd6e06
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 25 Feb 2025 19:50:50 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 25 Feb 2025 19:50:50 -0800
Follow precedent in pg_dump for preparing queries to improve
performance. Also, simplify the query by removing unnecessary joins.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Andres Freund <andres@anarazel.de>
Co-authored-by: Corey Huinker <corey.huinker@gmail.com>
Co-authored-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://postgr.es/m/CADkLM=dRMC6t8gp9GVf6y6E_r5EChQjMAAh_vPyih_zMiq0zvA@mail.gmail.com
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_dump.c
Avoid unnecessary relation stats query in pg_dump.
commit : 8f427187db78a549af316a69d54f5969db5f4951
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 25 Feb 2025 19:49:49 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 25 Feb 2025 19:49:49 -0800
The few fields we need can be easily collected in getTables() and
getIndexes() and stored in RelStatsInfo.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Andres Freund <andres@anarazel.de>
Co-authored-by: Corey Huinker <corey.huinker@gmail.com>
Co-authored-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://postgr.es/m/CADkLM=f0a43aTd88xW4xCFayEF25g-7hTrHX_WhV40HyocsUGg@mail.gmail.com
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
Re-add GUC track_wal_io_timing
commit : 6c349d83b607cfe20a685a5f66a4fa0e24dbe458
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Feb 2025 09:49:59 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Feb 2025 09:49:59 +0900
This commit is a rework of 2421e9a51d20, about which Andres Freund has
raised some concerns as it is valuable to have both track_io_timing and
track_wal_io_timing in some cases, as the WAL write and fsync paths can
be a major bottleneck for some workloads. Hence, it can be relevant to
not calculate the WAL timings in environments where pg_test_timing
performs poorly while capturing some IO data under track_io_timing for
the non-WAL IO paths. The opposite can be also true: it should be
possible to disable the non-WAL timings and enable the WAL timings (the
previous GUC setups allowed this possibility).
track_wal_io_timing is added back in this commit, controlling if WAL
timings should be calculated in pg_stat_io for the read, fsync and write
paths, as done previously with pg_stat_wal. pg_stat_wal previously
tracked only the sync and write parts (now removed), read stats is new
data tracked in pg_stat_io, all three are aggregated if
track_wal_io_timing is enabled. The read part matters during recovery
or if a XLogReader is used.
Extra note: more control over if the types of timings calculated in
pg_stat_io could be done with a GUC that lists pairs of (IOObject,IOOp).
Reported-by: Andres Freund <andres@anarazel.de>
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Co-authored-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/3opf2wh2oljco6ldyqf7ukabw3jijnnhno6fjb4mlu6civ5h24@fcwmhsgmlmzu
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogrecovery.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/backend/storage/smgr/md.c
M src/backend/utils/activity/pgstat_backend.c
M src/backend/utils/activity/pgstat_io.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/access/xlog.h
M src/include/pgstat.h
Remove redundant pg_set_*_stats() variants.
commit : a5cbdeb98af9a8d4f683fbffa69cd74be1f4a084
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 25 Feb 2025 16:15:47 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 25 Feb 2025 16:15:47 -0800
After commit f3dae2ae58, the primary purpose of separating the
pg_set_*_stats() from the pg_restore_*_stats() variants was
eliminated.
Leave pg_restore_relation_stats() and pg_restore_attribute_stats(),
which satisfy both purposes, and remove pg_set_relation_stats() and
pg_set_attribute_stats().
Reviewed-by: Corey Huinker <corey.huinker@gmail.com>
Discussion: https://postgr.es/m/1457469.1740419458@sss.pgh.pa.us
M doc/src/sgml/func.sgml
M src/backend/catalog/system_functions.sql
M src/backend/statistics/attribute_stats.c
M src/backend/statistics/relation_stats.c
M src/backend/statistics/stat_utils.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/statistics/stat_utils.h
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Change _mdfd_segpath() to return paths by value
commit : ecbff4378beecb0b1d12fc758538005a69821db1
author : Andres Freund <andres@anarazel.de>
date : Tue, 25 Feb 2025 09:02:07 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 25 Feb 2025 09:02:07 -0500
This basically mirrors the changes done in the predecessor commit. While there
isn't currently a need to get these paths in critical sections, it seems a
shame to unnecessarily allocate memory in these paths now that relpath()
doesn't allocate anymore.
Discussion: https://postgr.es/m/xeri5mla4b5syjd5a25nok5iez2kr3bm26j2qn4u7okzof2bmf@kwdh2vf7npra
M src/backend/storage/smgr/md.c
M src/tools/pgindent/typedefs.list
Change relpath() et al to return path by value
commit : 37c87e63f9e1a2d76db54fedcdf91d3895d200a6
author : Andres Freund <andres@anarazel.de>
date : Tue, 25 Feb 2025 09:02:07 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 25 Feb 2025 09:02:07 -0500
For AIO, and also some other recent patches, we need the ability to call
relpath() in a critical section. Until now that was not feasible, as it
allocated memory.
The fact that relpath() allocated memory also made it awkward to use in log
messages because we had to take care to free the memory afterwards. Which we
e.g. didn't do for when zeroing out an invalid buffer.
We discussed other solutions, e.g. filling a pre-allocated buffer that's
passed to relpath(), but they all came with plenty downsides or were larger
projects. The easiest fix seems to be to make relpath() return the path by
value.
To be able to return the path by value we need to determine the maximum length
of a relation path. This patch adds a long #define that computes the exact
maximum, which is verified to be correct in a regression test.
As this change the signature of relpath(), extensions using it will need to
adapt their code. We discussed leaving a backward-compat shim in place, but
decided it's not worth it given the use of relpath() doesn't seem widespread.
Discussion: https://postgr.es/m/xeri5mla4b5syjd5a25nok5iez2kr3bm26j2qn4u7okzof2bmf@kwdh2vf7npra
M src/backend/access/rmgrdesc/smgrdesc.c
M src/backend/access/rmgrdesc/xactdesc.c
M src/backend/access/transam/xlogutils.c
M src/backend/backup/basebackup_incremental.c
M src/backend/catalog/catalog.c
M src/backend/catalog/storage.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/backend/storage/smgr/md.c
M src/backend/utils/adt/dbsize.c
M src/bin/pg_rewind/filemap.c
M src/common/relpath.c
M src/include/common/relpath.h
M src/test/regress/expected/misc_functions.out
M src/test/regress/regress.c
M src/test/regress/sql/misc_functions.sql
M src/tools/pgindent/typedefs.list
Remove obsolete Python version check
commit : 32c393f9f1f148febcd741e7067e9537825587cc
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Feb 2025 14:11:38 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Feb 2025 14:11:38 +0100
The checked version is already the current minimum supported version
(3.2).
Discussion: https://www.postgresql.org/message-id/flat/ee410de1-1e0b-4770-b125-eeefd4726a24@eisentraut.org
M src/pl/plpython/plpy_exec.c
Eliminate code duplication in replace_rte_variables callbacks
commit : 363a6e8c6fcf9f3e19fe673ae02554645974a388
author : Richard Guo <rguo@postgresql.org>
date : Tue, 25 Feb 2025 16:11:34 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Tue, 25 Feb 2025 16:11:34 +0900
The callback functions ReplaceVarsFromTargetList_callback and
pullup_replace_vars_callback are both used to replace Vars in an
expression tree that reference a particular RTE with items from a
targetlist, and they both need to expand whole-tuple references and
deal with OLD/NEW RETURNING list Vars. As a result, currently there
is significant code duplication between these two functions.
This patch introduces a new function, ReplaceVarFromTargetList, to
perform the replacement and calls it from both callback functions,
thereby eliminating code duplication.
Author: Dean Rasheed <dean.a.rasheed@gmail.com>
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Jian He <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CAEZATCWhr=FM4X5kCPvVs-g2XEk+ceLsNtBK_zZMkqFn9vUjsw@mail.gmail.com
M src/backend/optimizer/prep/prepjointree.c
M src/backend/rewrite/rewriteManip.c
M src/include/rewrite/rewriteManip.h
Expand virtual generated columns in the planner
commit : 1e4351af329f2949c679a215f63c51d663ecd715
author : Richard Guo <rguo@postgresql.org>
date : Tue, 25 Feb 2025 16:10:25 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Tue, 25 Feb 2025 16:10:25 +0900
Commit 83ea6c540 added support for virtual generated columns that are
computed on read. All Var nodes in the query that reference virtual
generated columns must be replaced with the corresponding generation
expressions. Currently, this replacement occurs in the rewriter.
However, this approach has several issues. If a Var referencing a
virtual generated column has any varnullingrels, those varnullingrels
need to be propagated into the generation expression. Failing to do
so can lead to "wrong varnullingrels" errors and improper outer-join
removal.
Additionally, if such a Var comes from the nullable side of an outer
join, we may need to wrap the generation expression in a
PlaceHolderVar to ensure that it is evaluated at the right place and
hence is forced to null when the outer join should do so. In certain
cases, such as when the query uses grouping sets, we also need a
PlaceHolderVar for anything that is not a simple Var to isolate
subexpressions. Failure to do so can result in incorrect results.
To fix these issues, this patch expands the virtual generated columns
in the planner rather than in the rewriter, and leverages the
pullup_replace_vars architecture to avoid code duplication. The
generation expressions will be correctly marked with nullingrel bits
and wrapped in PlaceHolderVars when needed by the pullup_replace_vars
callback function. This requires handling the OLD/NEW RETURNING list
Vars in pullup_replace_vars_callback, as it may now deal with Vars
referencing the result relation instead of a subquery.
The "wrong varnullingrels" error was reported by Alexander Lakhin.
The incorrect result issue and the improper outer-join removal issue
were reported by Richard Guo.
Author: Richard Guo <guofenglinux@gmail.com>
Author: Dean Rasheed <dean.a.rasheed@gmail.com>
Reviewed-by: Jian He <jian.universality@gmail.com>
Discussion: https://postgr.es/m/75eb1a6f-d59f-42e6-8a78-124ee808cda7@gmail.com
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/rewrite/rewriteManip.c
M src/include/nodes/primnodes.h
M src/include/optimizer/prep.h
M src/include/rewrite/rewriteHandler.h
M src/include/rewrite/rewriteManip.h
M src/test/regress/expected/generated_virtual.out
M src/test/regress/sql/generated_virtual.sql
Fix untranslatable string concatenation in pg_upgrade
commit : 560a842d639f28497ab6df08ac0305240be79803
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 25 Feb 2025 15:53:32 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 25 Feb 2025 15:53:32 +0900
Oversight in 1aab6805919b.
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20250225.140953.1271748916018759840.horikyota.ntt@gmail.com
M src/bin/pg_upgrade/option.c
Doc: Fix pg_copy_logical_replication_slot description.
commit : 5b8f2ccc0a93375acb64a457817e61f400404a1f
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 25 Feb 2025 09:42:07 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 25 Feb 2025 09:42:07 +0530
This commit documents that the failover option is not copied when using
the pg_copy_logical_replication_slot function.
In passing, we modify the comments in the function clarifying the reason
for this behavior.
Reported-by: <duffieldzane@gmail.com>
Author: Hou Zhijie <houzj.fnst@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Backpatch-through: 17, where it was introduced
Discussion: https://postgr.es/m/173976850802.682632.11315364077431550250@wrigleys.postgresql.org
M doc/src/sgml/func.sgml
M src/backend/replication/slotfuncs.c
Missing doc update for f3dae2ae58.
commit : 15601fa21a5509b83a871c39ab98671f560da905
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 24 Feb 2025 17:27:32 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 24 Feb 2025 17:27:32 -0800
M doc/src/sgml/func.sgml
Do not use in-place updates for statistics import.
commit : f3dae2ae5856dec9935a51e53216400566ef8d4f
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 24 Feb 2025 17:10:59 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 24 Feb 2025 17:10:59 -0800
The use of in-place updates was originally there to follow the
precedent of ANALYZE and to reduce the potential for bloat on
pg_class. Per discussion, it's not worth the risks.
Reported-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/cpdanvzykcb5o64rmapkx6n5gjypoce3y52hff7ocxupgpbxu4@53jmlyvukijo
M src/backend/statistics/relation_stats.c
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
psql: Add pipeline status to prompt and some state variables
commit : 3ce357584e795762aa6dc45fafc21dccea7e5ec6
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 25 Feb 2025 10:07:24 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 25 Feb 2025 10:07:24 +0900
This commit adds %P to psql prompts, able to report the status of a
pipeline depending on PQpipelineStatus(): on, off or abort.
The following variables are added to report the state of an ongoing
pipeline:
- PIPELINE_SYNC_COUNT: reports the number of piped syncs.
- PIPELINE_COMMAND_COUNT: reports the number of piped commands, a
command being either \bind, \bind_named, \close or \parse.
- PIPELINE_RESULT_COUNT: reports the results available to read with
\getresults.
These variables can be used with \echo or in a prompt, using "%:name:"
in PROMPT1, PROMPT2 or PROMPT3. Some basic regression tests are added
for these. The suggestion to use variables to show the details about
the status counters comes from me. The original patch proposed was less
extensible, hardcoding the output in the prompt.
Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/CAO6_XqroE7JuMEm1sWz55rp9fAYX2JwmcP_3m_v51vnOFdsLiQ@mail.gmail.com
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/common.c
M src/bin/psql/prompt.c
M src/bin/psql/startup.c
M src/test/regress/expected/psql_pipeline.out
M src/test/regress/sql/psql_pipeline.sql
Fix bug in cbc127917 to handle nested Append correctly
commit : cbb9086c9ef64b020cb9036f50afc14644bbf734
author : Amit Langote <amitlan@postgresql.org>
date : Tue, 25 Feb 2025 09:21:17 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Tue, 25 Feb 2025 09:21:17 +0900
A non-leaf partition with a subplan that is an Append node was
omitted from PlannedStmt.unprunableRelids because it was mistakenly
included in PlannerGlobal.prunableRelids due to the way
PartitionedRelPruneInfo.leafpart_rti_map[] is constructed. This
happened when a non-leaf partition used an unflattened Append or
MergeAppend. As a result, ExecGetRangeTableRelation() reported an
error when called from CreatePartitionPruneState() to process the
partition's own PartitionPruneInfo, since it was treated as prunable
when it should not have been.
Reported-by: Alexander Lakhin <exclusion@gmail.com> (via sqlsmith)
Diagnosed-by: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://postgr.es/m/74839af6-aadc-4f60-ae77-ae65f94bf607@gmail.com
M src/backend/executor/execPartition.c
M src/backend/partitioning/partprune.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Fix assertion when decoding XLOG_PARAMETER_CHANGE on promoted primary.
commit : 48796a98d5aed68e349118d4e05ac26b76c765b0
author : Masahiko Sawada <msawada@postgresql.org>
date : Mon, 24 Feb 2025 14:03:04 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Mon, 24 Feb 2025 14:03:04 -0800
When a standby replays an XLOG_PARAMETER_CHANGE record that lowers
wal_level below logical, we invalidate all logical slots in hot
standby mode. However, if this record was replayed while not in hot
standby mode, logical slots could remain valid even after promotion,
potentially causing an assertion failure during WAL record decoding.
To fix this issue, this commit adds a check for hot_standby status
when restoring a logical replication slot on standbys. This check
ensures that logical slots are invalidated when they become
incompatible due to insufficient wal_level during recovery.
Backpatch to v16 where logical decoding on standby was introduced.
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/CAD21AoABoFwGY_Rh2aeE6tEq3HkJxf0c6UeOXn4VV9v6BAQPSw%40mail.gmail.com
Backpatch-through: 16
M src/backend/replication/slot.c
M src/test/recovery/t/035_standby_logical_decoding.pl
oauth: Rename macro to avoid collisions on Windows
commit : d1146dc2a72e96294045f7feb6d0f29082e1c71c
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 24 Feb 2025 22:20:37 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 24 Feb 2025 22:20:37 +0100
Our json parsing defined the macros OPTIONAL and REQUIRED to decorate the
structs with for increased readability. This however collides with macros
in the <windef.h> header on Windows.
../src/interfaces/libpq/fe-auth-oauth-curl.c:398:9: warning: "OPTIONAL" redefined
398 | #define OPTIONAL false
| ^~~~~~~~
In file included from D:/a/_temp/msys64/ucrt64/include/windef.h:9,
from D:/a/_temp/msys64/ucrt64/include/windows.h:69,
from D:/a/_temp/msys64/ucrt64/include/winsock2.h:23,
from ../src/include/port/win32_port.h:60,
from ../src/include/port.h:24,
from ../src/include/c.h:1331,
from ../src/include/postgres_fe.h:28,
from ../src/interfaces/libpq/fe-auth-oauth-curl.c:16:
include/minwindef.h:65:9: note: this is the location of the previous definition
65 | #define OPTIONAL
| ^~~~~~~~
Rename to avoid compilation errors in anticipation of implementing
support for Windows.
Reported-by: Dave Cramer (on PostgreSQL Hacking Discord)
M src/interfaces/libpq/fe-auth-oauth-curl.c
oauth: Fix incorrect const markers in struct
commit : 03366b61dfe596796a19368f851859a4764229b9
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 24 Feb 2025 22:20:29 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 24 Feb 2025 22:20:29 +0100
Two members in PGoauthBearerRequest were incorrectly marked as const.
While in there, align the name of the struct with the typedef as per
project style.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/912516.1740329361@sss.pgh.pa.us
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/libpq-fe.h
Delay extraction of TIDBitmap per page offsets
commit : bfe56cdf9a4e07edca46254a88efd9ef17421cd7
author : Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 24 Feb 2025 16:07:55 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 24 Feb 2025 16:07:55 -0500
Pages from the bitmap created by the TIDBitmap API can be exact or
lossy. The TIDBitmap API extracts the tuple offsets from exact pages
into an array for the convenience of the caller.
This was done in tbm_private|shared_iterate() right after advancing the
iterator. However, as long as tbm_private|shared_iterate() set a
reference to the PagetableEntry in the TBMIterateResult, the offset
extraction can be done later.
Waiting to extract the tuple offsets has a few benefits. For the shared
iterator case, it allows us to extract the offsets after dropping the
shared iterator state lock, reducing time spent holding a contended
lock.
Separating the iteration step and extracting the offsets later also
allows us to avoid extracting the offsets for prefetched blocks. Those
offsets were never used, so the overhead of extracting and storing them
was wasted.
The real motivation for this change, however, is that future commits
will make bitmap heap scan use the read stream API. This requires a
TBMIterateResult per issued block. By removing the array of tuple
offsets from the TBMIterateResult and only extracting the offsets when
they are used, we reduce the memory required for per buffer data
substantially.
Suggested-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGLHbKP3jwJ6_%2BhnGi37Pw3BD5j2amjV3oSk7j-KyCnY7Q%40mail.gmail.com
M src/backend/access/gin/ginget.c
M src/backend/access/gin/ginscan.c
M src/backend/access/heap/heapam_handler.c
M src/backend/nodes/tidbitmap.c
M src/include/access/gin_private.h
M src/include/nodes/tidbitmap.h
Add lossy indicator to TBMIterateResult
commit : b8778c4cd8bc924ce5347cb1ab10dfbf34130559
author : Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 24 Feb 2025 16:07:50 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 24 Feb 2025 16:07:50 -0500
TBMIterateResult->ntuples is -1 when the page in the bitmap is lossy.
Add an explicit lossy indicator so that we can move ntuples out of the
TBMIterateResult in a future commit.
Discussion: https://postgr.es/m/CA%2BhUKGLHbKP3jwJ6_%2BhnGi37Pw3BD5j2amjV3oSk7j-KyCnY7Q%40mail.gmail.com
M src/backend/access/gin/ginget.c
M src/backend/access/heap/heapam_handler.c
M src/backend/nodes/tidbitmap.c
M src/include/nodes/tidbitmap.h
Fix comment for MAX_BACKENDS.
commit : c56e8af75e081383e05cf544f372506c8da4efe7
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 24 Feb 2025 15:02:09 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 24 Feb 2025 15:02:09 -0600
This comment mentions that we check that the configured number of
backends does not exceed MAX_BACKENDS in RegisterBackgroundWorker()
and relevant GUC check hooks, neither of which has those checks
anymore. To fix, adjust this comment to say that we do the check
in InitializeMaxBackends().
Oversights in commits 6bc8ef0b7f and 0b1fe1413e.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/Z7zOEzz8lNjaU9yf%40nathan
M src/include/storage/procnumber.h
libpq: Trace all NegotiateProtocolVersion fields
commit : e87c14b19ed458f6e07d2df338e5000453ad785d
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 24 Feb 2025 12:03:25 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 24 Feb 2025 12:03:25 -0500
Previously, the names of the unsupported protocol options were not
traced. Since NegotiateProtocolVersion has not really been used yet,
that has not mattered much, but we hope to use it eventually, so let's
fix this.
Author: Jelte Fennema-Nio <postgres@jeltef.nl>
Discussion: https://postgr.es/m/CAGECzQTfc_O+HXqAo5_-xG4r3EFVsTefUeQzSvhEyyLDba-O9w@mail.gmail.com
M src/interfaces/libpq/fe-trace.c
libpq: Add PQfullProtocolVersion to exports.txt
commit : c9d94ea2158b7b16f8a7edbb18e504068771c91f
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 24 Feb 2025 11:44:29 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 24 Feb 2025 11:44:29 -0500
This is necessary to be able to actually use the function on Windows;
bug introduced in commit cdb6b0fdb0b2face270406905d31f8f513b015cc.
Author: Jelte Fennema-Nio <postgres@jeltef.nl>
Discussion: https://postgr.es/m/CAGECzQTfc_O+HXqAo5_-xG4r3EFVsTefUeQzSvhEyyLDba-O9w@mail.gmail.com
M src/interfaces/libpq/exports.txt
Fix confusion about data type of pg_class.relpages and relallvisible.
commit : 9de2cc455eb9c60dda979c32985df967522a8ab2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 24 Feb 2025 11:16:04 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 24 Feb 2025 11:16:04 -0500
Although they're exposed as int4 in pg_class, relpages and
relallvisible are really of type BlockNumber, that is uint32.
Correct type puns in relation_statistics_update() and remove
inappropriate range-checks. The type puns are only cosmetic
issues, but the range checks would cause failures with huge
relations.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Author: Corey Huinker <corey.huinker@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/614341.1740269035@sss.pgh.pa.us
M src/backend/statistics/relation_stats.c
pg_amcheck: PQclear query results
commit : e889422d98e066dcf095fa275fbcb14ffd45708c
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 24 Feb 2025 16:03:19 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 24 Feb 2025 16:03:19 +0100
While the potential memory leak is small, ensure to PQclear the query
results before disconnecting.
Author: Jiao Shuntian <312199339@qq.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/tencent_F34922C91C41E76C734773E767C9FBDB9906@qq.com
M src/bin/pg_amcheck/pg_amcheck.c
Add static asserts for MAX_BACKENDS limiting factors
commit : 5ee75e32fa734e8ae85ff096f1523be8e16e1b8d
author : Andres Freund <andres@anarazel.de>
date : Mon, 24 Feb 2025 05:39:27 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 24 Feb 2025 05:39:27 -0500
So far the various dependencies were documented in the comment above
MAX_BACKENDS, but not checked.
Discussion: https://postgr.es/m/CA+COZaBO_s3LfALq=b+HcBHFSOEGiApVjrRacCe4VP9m7CJsNQ@mail.gmail.com
M src/backend/storage/lmgr/deadlock.c
M src/backend/utils/cache/inval.c
M src/include/storage/buf_internals.h
bufmgr: Make it easier to change number of buffer state bits
commit : 418451bfe161e4b86c29f098e4d93b2347d35774
author : Andres Freund <andres@anarazel.de>
date : Mon, 24 Feb 2025 05:39:17 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 24 Feb 2025 05:39:17 -0500
In an upcoming commit I'd like to change the number of bits for the usage
count (the current max is 5, fitting in three bits, but we reserve four
bits). Until now that required adjusting a bunch of magic constants, now the
constants are defined based on the number of bits reserved.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/lxzj26ga6ippdeunz6kuncectr5gfuugmm2ry22qu6hcx6oid6@lzx3sjsqhmt6
Discussion: https://postgr.es/m/riivolmg6uzfvpzfn6wjo3ghwt42rcec43ok6mv4oenfg654y7@x7dbposbskwd
M src/include/storage/buf_internals.h
Base LWLock limits directly on MAX_BACKENDS
commit : cd3ccf88aacb43b7232d6834dc886ff8538c6ce8
author : Andres Freund <andres@anarazel.de>
date : Mon, 24 Feb 2025 05:39:17 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 24 Feb 2025 05:39:17 -0500
Jacob reported that comments for LW_SHARED_MASK referenced a MAX_BACKENDS
limit of 2^23-1, but that MAX_BACKENDS is actually limited to 2^18-1. The
limit was lowered in 48354581a49c, but the comment in lwlock.c wasn't updated.
Instead of just fixing the comment, it seems better to directly base the
lwlock defines on MAX_BACKENDS and add static assertions to ensure that there
is enough space. That way there's no comment that can go out of sync in the
future.
As part of that change I noticed that for some reason the high bit wasn't used
for flags, which seems somewhat odd. Redefine the flag values to start at the
highest bit.
Reported-by: Jacob Brazeal <jacob.brazeal@gmail.com>
Reviewed-by: Jacob Brazeal <jacob.brazeal@gmail.com>
Discussion: https://postgr.es/m/CA+COZaBO_s3LfALq=b+HcBHFSOEGiApVjrRacCe4VP9m7CJsNQ@mail.gmail.com
M src/backend/storage/lmgr/lwlock.c
Move MAX_BACKENDS to procnumber.h
commit : 6394a3a61c0deae34ad9e94505d319cebe9e09a1
author : Andres Freund <andres@anarazel.de>
date : Mon, 24 Feb 2025 05:39:17 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 24 Feb 2025 05:39:17 -0500
MAX_BACKENDS influences many things besides postmaster. I e.g. noticed that we
don't have static assertions ensuring BUF_REFCOUNT_MASK is big enough for
MAX_BACKENDS, adding them would require including postmaster.h in
buf_internals.h which doesn't seem right.
While at that, add MAX_BACKENDS_BITS, as that's useful in various places for
static assertions (to be added in subsequent commits).
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/wptizm4qt6yikgm2pt52xzyv6ycmqiutloyvypvmagn7xvqkce@d4xuv3mylpg4
M src/backend/storage/lmgr/lwlock.c
M src/backend/utils/adt/xid8funcs.c
M src/backend/utils/init/postinit.c
M src/backend/utils/misc/guc_tables.c
M src/include/postmaster/postmaster.h
M src/include/storage/procnumber.h
Silence warning in older versions of Valgrind
commit : 0600d276d485a8222eb6b571b08889c9f539e86f
author : John Naylor <john.naylor@postgresql.org>
date : Mon, 24 Feb 2025 18:03:29 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Mon, 24 Feb 2025 18:03:29 +0700
Due to misunderstanding on my part, commit 235328ee4 did not go far
enough to silence older versions of Valgrind. For those, it was the bit
scan that was problematic, not the subsequent bit-masking operation. To
fix, use the unaligned path for the trailing bytes. Since we don't have
a bit scan here anymore, also remove some comments and endian-specific
coding around that.
Reported-by: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Discussion: https://postgr.es/m/f3aa2d45-3b28-41c5-9499-a1bc30e0f8ec@postgrespro.ru
Backpatch-through: 17
M src/include/common/hashfn_unstable.h
Remove read/sync fields from pg_stat_wal and GUC track_wal_io_timing
commit : 2421e9a51d20bb83154e54a16ce628f9249fa907
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 24 Feb 2025 09:51:56 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 24 Feb 2025 09:51:56 +0900
The four following attributes are removed from pg_stat_wal:
* wal_write
* wal_sync
* wal_write_time
* wal_sync_time
a051e71e28a1 has added an equivalent of this information in pg_stat_io
with more granularity as this now spreads across the backend types, IO
context and IO objects. So, keeping the same information in pg_stat_wal
has little benefits.
Another benefit of this commit is the removal of PendingWalStats,
simplifying an upcoming patch to add per-backend WAL statistics, which
already support IO statistics and which have access to the write/sync
stats data of WAL.
The GUC track_wal_io_timing, that was used to enable or disable the
aggregation of the write and sync timings for WAL, is also removed.
pgstat_prepare_io_time() is simplified.
Bump catalog version.
Bump PGSTAT_FILE_FORMAT_ID, due to the update of PgStat_WalStats.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/Z7RkQ0EfYaqqjgz/@ip-10-97-1-34.eu-west-3.compute.internal
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogrecovery.c
M src/backend/catalog/system_views.sql
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/backend/storage/smgr/md.c
M src/backend/utils/activity/pgstat_io.c
M src/backend/utils/activity/pgstat_wal.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/access/xlog.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
M src/tools/pgindent/typedefs.list
Ignore hash's relallvisible when checking pg_upgrade from pre-v10.
commit : fc0d0ce978752493868496be6558fa17b7c4c3cf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 Feb 2025 14:16:26 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 Feb 2025 14:16:26 -0500
Our cross-version upgrade tests have been failing for some pre-v10
source versions since commit 1fd1bd871. This turns out to be
because relallvisible may change for tables that have hash indexes,
because the upgrade process forcibly reindexes such indexes to
deal with the changes made in v10.
Fortunately, the set of tables that have such indexes is small
and won't change anymore in those branches. So just hack up
AdjustUpgrade.pm to not compare the relallvisible values of
those specific tables.
While here, also tighten the regex that suppresses comparison
of version fields.
Discussion: https://postgr.es/m/812817.1740277228@sss.pgh.pa.us
M src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
backend libpq void * argument for binary data
commit : 454c182f8542890d0e2eac85f70d9a254a34fce3
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 23 Feb 2025 14:26:39 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 23 Feb 2025 14:26:39 +0100
Change some backend libpq functions to take void * for binary data
instead of char *. This removes the need for numerous casts.
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/fd1fcedb-3492-4fc8-9e3e-74b97f2db6c7%40eisentraut.org
M src/backend/libpq/pqcomm.c
M src/backend/libpq/pqformat.c
M src/backend/tcop/backend_startup.c
M src/backend/utils/adt/varbit.c
M src/include/libpq/libpq.h
M src/include/libpq/pqformat.h
SnapBuildRestoreContents() void * argument for binary data
commit : ebdccead1647aec1122810dad498438d9964f35f
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 24 Sep 2024 12:18:31 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 24 Sep 2024 12:18:31 +0200
Change internal snapbuild API function to take void * for binary data
instead of char *. This removes the need for numerous casts.
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/fd1fcedb-3492-4fc8-9e3e-74b97f2db6c7%40eisentraut.org
M src/backend/replication/logical/snapbuild.c
Add more tests for utility commands in pipelines
commit : a4e986ef5a4698b43a3a9b11791b282734938ed1
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 23 Feb 2025 16:43:07 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 23 Feb 2025 16:43:07 +0900
This commit checks interactions with pipelines and implicit transaction
blocks for the following commands that have their own behaviors when
used in pipelines depending on their order in a pipeline and sync
requests:
- SET LOCAL
- REINDEX CONCURRENTLY
- VACUUM
- Subtransactions (SAVEPOINT, ROLLBACK TO SAVEPOINT)
These scenarios could be tested only with pgbench previously. The
meta-commands of psql controlling pipelines make these easier to
implement, debug, and they can be run in a SQL script.
Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/CAO6_XqroE7JuMEm1sWz55rp9fAYX2JwmcP_3m_v51vnOFdsLiQ@mail.gmail.com
M src/test/regress/expected/psql_pipeline.out
M src/test/regress/sql/psql_pipeline.sql
jsonb internal API void * argument for binary data
commit : f98765f0ce90e465e35c809f4b3caaeeb3d24610
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 23 Feb 2025 08:34:55 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 23 Feb 2025 08:34:55 +0100
Change some internal jsonb API functions to take void * for binary
data instead of char *. This removes the need for numerous casts.
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/fd1fcedb-3492-4fc8-9e3e-74b97f2db6c7%40eisentraut.org
M src/backend/utils/adt/jsonb_util.c
Documentation fixups for dumping statistics.
commit : cb45dc3afb05ce5e5267b1a9be4725d0be7ee33c
author : Jeff Davis <jdavis@postgresql.org>
date : Sat, 22 Feb 2025 10:03:11 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Sat, 22 Feb 2025 10:03:11 -0800
Reported-by: Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
Reported-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/OSCPR01MB149665630030E7F54FDA8B27BF5C72@OSCPR01MB14966.jpnprd01.prod.outlook.com
Discussion: https://postgr.es/m/25d26774-25fa-46f2-9888-c6a707d1fef7@dunslane.net
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_restore.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M src/bin/pg_upgrade/check.c
Change \conninfo to use tabular format
commit : bba2fbc6238b2a0a7f348fbbb5c31ffa7623bc39
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 22 Feb 2025 10:05:26 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 22 Feb 2025 10:05:26 +0100
(Initially the proposal was to keep \conninfo alone and add this feature
as \conninfo+, but we decided against keeping the original.)
Also display more fields than before, though not as many as were
suggested during the discussion. In particular, we don't show 'role'
nor 'session authorization', for both which a case can probably be made.
These can be added as followup commits, if we agree to it.
Some (most?) reviewers actually reviewed rather different versions of
the patch and do not necessarily endorse the current one.
Co-authored-by: Maiquel Grassi <grassi@hotmail.com.br>
Co-authored-by: Hunaid Sohail <hunaidpgml@gmail.com>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Sami Imseih <simseih@amazon.com>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Reviewed-by: Jim Jones <jim.jones@uni-muenster.de>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Pavel Luzanov <p.luzanov@postgrespro.ru>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Reviewed-by: Erik Wienhold <ewie@ewie.name>
Discussion: https://postgr.es/m/CP8P284MB24965CB63DAC00FC0EA4A475EC462@CP8P284MB2496.BRAP284.PROD.OUTLOOK.COM
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
Remove unstable test suite added by 525392d57
commit : 4f1b6e5bb4fe9bc74395d30d689b28e9cda654a5
author : Amit Langote <amitlan@postgresql.org>
date : Sat, 22 Feb 2025 15:19:23 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Sat, 22 Feb 2025 15:19:23 +0900
The 'cached-plan-inval' test suite, introduced in 525392d57 under
src/test/modules/delay_execution, aimed to verify that cached plan
invalidation triggers replanning after deferred locks are taken.
However, its ExecutorStart_hook-based approach relies on lock timing
assumptions that, in retrospect, are fragile. This instability was
exposed by failures on BF animal trilobite, which builds with
CLOBBER_CACHE_ALWAYS.
One option was to dynamically disable the cache behavior that causes
the test suite to fail by setting "debug_discard_caches = 0", but it
seems better to remove the suite. The risk of future failures due to
other cache flush hazards outweighs the benefit of catching real
breakage in the backend behavior it tests.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/2990641.1740117879@sss.pgh.pa.us
M src/test/modules/delay_execution/Makefile
M src/test/modules/delay_execution/delay_execution.c
D src/test/modules/delay_execution/expected/cached-plan-inval.out
M src/test/modules/delay_execution/meson.build
D src/test/modules/delay_execution/specs/cached-plan-inval.spec
Allow lwlocks to be disowned
commit : f8d7f29b3e81db59b95e4b5baaa6943178c89fd8
author : Andres Freund <andres@anarazel.de>
date : Fri, 21 Feb 2025 20:55:23 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 21 Feb 2025 20:55:23 -0500
To implement AIO writes, the backend initiating writes needs to transfer the
lock ownership to the AIO subsystem, so the lock held during the write can be
released in another backend.
Other backends need to be able to "complete" an asynchronously started IO to
avoid deadlocks (consider e.g. one backend starting IO for a buffer and then
waiting for a heavyweight lock held by another relation followed by the
current holder of the heavyweight lock waiting for the IO to complete).
To that end, this commit adds LWLockDisown() and LWLockReleaseDisowned(). If
code uses LWLockDisown() it's the code's responsibility to ensure that the
lock is released in case of errors.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/1f6b50a7-38ef-4d87-8246-786d39f46ab9@iki.fi
M src/backend/storage/lmgr/lwlock.c
M src/include/storage/lwlock.h
Adjust EXPLAIN test case to filter out "Actual Rows" values.
commit : 44cbba9a7f51a3888d5087fc94b23614ba2b81f2
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 21 Feb 2025 19:15:44 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 21 Feb 2025 19:15:44 -0500
Per the buildfarm, these tests appear to be unstable in the wake of
commit ddb17e387aa28d61521227377b00f997756b8a27. I'm not sure that
just hiding this output is the right way forward, because I think
there may be other test cases that will fail with lower probability
even after this fix. However, it's hard to tell right now, because
this is failing on a number of buildfarm animals. So let's try this
for now to either get a clearer picture of what else is broken, or
as a stopgap until we decide what the permanent fix should be, or
perhaps this will be the permanent fix after all.
M src/test/regress/expected/explain.out
M src/test/regress/sql/explain.sql
Avoid race condition between "GRANT role" and "DROP ROLE".
commit : 98fc31d6499163a0a781aa6f13582a07f09cd7c6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Feb 2025 17:07:01 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Feb 2025 17:07:01 -0500
Concurrently dropping either the granted role or the grantee
does not stop GRANT from completing, instead resulting in a
dangling role reference in pg_auth_members. That's relatively
harmless in the short run, but inconsistent catalog entries
are not a good thing.
This patch solves the problem by adding the granted and grantee
roles as explicit shared dependencies of the pg_auth_members entry.
That's a bit indirect, but it works because the pg_shdepend code
applies the necessary locking and rechecking.
Commit 6566133c5 previously established similar handling for
the grantor column of pg_auth_members; it's not clear why it
didn't cover the other two role OID columns.
A side-effect of this approach is that DROP OWNED BY will now drop
pg_auth_members entries that mention the target role as either the
granted or grantee role. That's clearly appropriate for the
grantee, since we'll drop its other privileges too. It doesn't
seem too far out of line for the granted role, since we're
presumably about to drop it and besides we're removing all reasons
why it'd matter to be a member of it. (One could argue that this
makes DropRole's code to auto-drop pg_auth_members entries
unnecessary, but I chose to leave it in place since perhaps some
people's workflows expect that to work without a DROP OWNED BY.)
Note to patch readers: CreateRole's first CommandCounterIncrement
call is now unconditional, because this change creates another
case in which it's needed, and it seemed to be more trouble than
it's worth to preserve that micro-optimization.
Arguably this is a bug fix, but the fact that it changes the
expected contents of pg_shdepend seems like not a great thing
to do in the stable branches, and perhaps we don't want the
change in DROP OWNED BY semantics there either. On the other
hand, I opted not to force a catversion bump in HEAD, because
the presence or absence of these entries doesn't matter for
most purposes.
Reported-by: Virender Singla <virender.cse@gmail.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CAM6Zo8woa62ZFHtMKox6a4jb8qQ=w87R2L0K8347iE-juQL2EA@mail.gmail.com
M doc/src/sgml/ref/drop_owned.sgml
M src/backend/commands/user.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Allow EXPLAIN to indicate fractional rows.
commit : ddb17e387aa28d61521227377b00f997756b8a27
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 21 Feb 2025 16:10:44 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 21 Feb 2025 16:10:44 -0500
When nloops > 1, we now display two digits after the decimal point,
rather than none. This is important because what we print is actually
planstate->instrument->ntuples / nloops, and sometimes what you want
to know is planstate->instrument->ntuples. You can estimate that by
multiplying the displayed row count by the displayed nloops value, but
the fact that the displayed value is rounded makes that inexact. It's
still inexact even if we show these two extra decimal places, but less
so. Perhaps we will agree on a way to further improve this output later,
but for now this seems better than doing nothing.
Author: Ibrar Ahmed <ibrar.ahmad@gmail.com>
Author: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Greg Stark <stark@mit.edu>
Reviewed-by: Naeem Akhter <akhternaeem@gmail.com>
Reviewed-by: Hamid Akhtar <hamid.akhtar@percona.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andrei Lepikhov <a.lepikhov@postgrespro.ru>
Reviewed-by: Guillaume Lelarge <guillaume@lelarge.info>
Reviewed-by: Matheus Alcantara <matheusssilv97@gmail.com>
Reviewed-by: Alena Rybakina <a.rybakina@postgrespro.ru>
Discussion: http://postgr.es/m/603c8f070905281830g2e5419c4xad2946d149e21f9d%40mail.gmail.com
M src/backend/commands/explain.c
M src/test/regress/expected/explain.out
M src/test/regress/expected/memoize.out
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/partition_prune.sql
Add test 005_char_signedness.pl to meson.build.
commit : 78d3f48895029e2c7c579fc38c07f2a197f85ed9
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Feb 2025 12:31:16 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Feb 2025 12:31:16 -0800
Oversight in a8238f87f98 where the test has been added.
Discussion: https://postgr.es/m/CB11ADBC-0C3F-4FE0-A678-666EE80CBB07%40amazon.com
M src/bin/pg_upgrade/meson.build
Fix pg_dumpall to cope with dangling OIDs in pg_auth_members.
commit : 29d75b25b567e5c91b895a6148211399c9c97554
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Feb 2025 13:37:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Feb 2025 13:37:12 -0500
There is a race condition between "GRANT role" and "DROP ROLE",
which allows GRANT to install pg_auth_members entries that refer to
dropped roles. (Commit 6566133c5 prevented that for the grantor
field, but not for the granted or grantee roles.) We'll soon fix
that, at least in HEAD, but pg_dumpall needs to cope with the
situation in case of pre-existing inconsistency. As pg_dumpall
stands, it will emit invalid commands like 'GRANT foo TO ""',
which causes pg_upgrade to fail. Fix it to emit warnings and skip
those GRANTs, instead.
There was some discussion of removing the problem by changing
dumpRoleMembership's query to use JOIN not LEFT JOIN, but that
would result in silently ignoring such entries. It seems better
to produce a warning.
Pre-v16 branches already coped with dangling grantor OIDs by simply
omitting the GRANTED BY clause. I left that behavior as-is, although
it's somewhat inconsistent with the behavior of later branches.
Reported-by: Virender Singla <virender.cse@gmail.com>
Discussion: https://postgr.es/m/CAM6Zo8woa62ZFHtMKox6a4jb8qQ=w87R2L0K8347iE-juQL2EA@mail.gmail.com
Backpatch-through: 13
M src/bin/pg_dump/pg_dumpall.c
Fix an issue with index scan using pg_trgm due to char signedness on different architectures.
commit : dfd8e6c73eeaa8a4d53851ba01e8d7b1c0bef0e3
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Feb 2025 10:27:39 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Feb 2025 10:27:39 -0800
GIN and GiST indexes utilizing pg_trgm's opclasses store sorted
trigrams within index tuples. When comparing and sorting each trigram,
pg_trgm treats each character as a 'char[3]' type in C. However, the
char type in C can be interpreted as either signed char or unsigned
char, depending on the platform, if the signedness is not explicitly
specified. Consequently, during replication between different CPU
architectures, there was an issue where index scans on standby servers
could not locate matching index tuples due to the differing treatment
of character signedness.
This change introduces comparison functions for trgm that explicitly
handle signed char and unsigned char. The appropriate comparison
function will be dynamically selected based on the character
signedness stored in the control file. Therefore, upgraded clusters
can utilize the indexes without rebuilding, provided the cluster
upgrade occurs on platforms with the same character signedness as the
original cluster initialization.
The default char signedness information was introduced in 44fe30fdab6,
so no backpatch.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CB11ADBC-0C3F-4FE0-A678-666EE80CBB07%40amazon.com
M contrib/pg_trgm/trgm.h
M contrib/pg_trgm/trgm_op.c
pg_upgrade: Add --set-char-signedness to set the default char signedness of new cluster.
commit : 1aab6805919b84a2f2477132d96f2147ed36150e
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Feb 2025 10:23:39 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Feb 2025 10:23:39 -0800
This change adds a new option --set-char-signedness to pg_upgrade. It
enables user to set arbitrary signedness during pg_upgrade. This helps
cases where user who knew they copied the v17 source cluster from
x86 (signedness=true) to ARM (signedness=false) can pg_upgrade
properly without the prerequisite of acquiring an x86 VM.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CB11ADBC-0C3F-4FE0-A678-666EE80CBB07%40amazon.com
M doc/src/sgml/ref/pgupgrade.sgml
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/t/005_char_signedness.pl
pg_upgrade: Preserve default char signedness value from old cluster.
commit : a8238f87f980848c2d69c105555c4383e20e7670
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Feb 2025 10:19:40 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Feb 2025 10:19:40 -0800
Commit 44fe30fdab6 introduced the 'default_char_signedness' field in
controlfile. Newly created database clusters always set this field to
'signed'.
This change ensures that pg_upgrade updates the
'default_char_signedness' to 'unsigned' if the source database cluster
has signedness=false. For source clusters from v17 or earlier, which
lack the 'default_char_signedness' information, pg_upgrade assumes the
source cluster was initialized on the same platform where pg_upgrade
is running. It then sets the 'default_char_signedness' value according
to the current platform's default character signedness.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CB11ADBC-0C3F-4FE0-A678-666EE80CBB07%40amazon.com
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/pg_upgrade.h
A src/bin/pg_upgrade/t/005_char_signedness.pl
pg_resetwal: Add --char-signedness option to change the default char signedness.
commit : 30666d1857d7a785331f5805c20ec19430500466
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Feb 2025 10:14:36 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Feb 2025 10:14:36 -0800
With the newly added option --char-signedness, pg_resetwal updates the
default char signedness flag in the controlfile. This option is
primarily intended for an upcoming patch that pg_upgrade supports
preserving the default char signedness during upgrades, and is not
meant for manual operation.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CB11ADBC-0C3F-4FE0-A678-666EE80CBB07%40amazon.com
M doc/src/sgml/ref/pg_resetwal.sgml
M src/bin/pg_resetwal/pg_resetwal.c
M src/bin/pg_resetwal/t/001_basic.pl
Add default_char_signedness field to ControlFileData.
commit : 44fe30fdab6746a287163e7cc093fd36cda8eb92
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Feb 2025 10:12:08 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 21 Feb 2025 10:12:08 -0800
The signedness of the 'char' type in C is
implementation-dependent. For instance, 'signed char' is used by
default on x86 CPUs, while 'unsigned char' is used on aarch
CPUs. Previously, we accidentally let C implementation signedness
affect persistent data. This led to inconsistent results when
comparing char data across different platforms.
This commit introduces a new 'default_char_signedness' field in
ControlFileData to store the signedness of the 'char' type. While this
change does not encourage the use of 'char' without explicitly
specifying its signedness, this field can be used as a hint to ensure
consistent behavior for pre-v18 data files that store data sorted by
the 'char' type on disk (e.g., GIN and GiST indexes), especially in
cross-platform replication scenarios.
Newly created database clusters unconditionally set the default char
signedness to true. pg_upgrade (with an upcoming commit) changes this
flag for clusters if the source database cluster has
signedness=false. As a result, signedness=false setting will become
rare over time. If we had known about the problem during the last
development cycle that forced initdb (v8.3), we would have made all
clusters signed or all clusters unsigned. Making pg_upgrade the only
source of signedness=false will cause the population of database
clusters to converge toward that retrospective ideal.
Bump catalog version (for the catalog changes) and PG_CONTROL_VERSION
(for the additions in ControlFileData).
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CB11ADBC-0C3F-4FE0-A678-666EE80CBB07%40amazon.com
M doc/src/sgml/func.sgml
M src/backend/access/transam/xlog.c
M src/backend/utils/misc/pg_controldata.c
M src/bin/pg_controldata/pg_controldata.c
M src/include/access/xlog.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_control.h
M src/include/catalog/pg_proc.dat
doc: clarify default checksum behavior in non-master branches
commit : 901a1cf8b4a291a3a22021bf7acb516c1d278e53
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Feb 2025 13:03:29 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Feb 2025 13:03:29 -0500
Also simplify and correct data checksum wording in master now that it is
the default. PG 13 did not have the awkward wording.
Reported-by: Felix <afripowered@gmail.com>
Reviewed-by: Laurenz Albe
Discussion: https://postgr.es/m/173928241056.707.3989867022954178032@wrigleys.postgresql.org
Backpatch-through: 14
M doc/src/sgml/amcheck.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/pageinspect.sgml
M doc/src/sgml/wal.sgml
doc: remove non-breaking space in SGML files, causes make error
commit : 6ea0734e41de4f7f39c0df05736c1714aa004b61
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Feb 2025 12:15:53 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Feb 2025 12:15:53 -0500
M doc/src/sgml/installation.sgml
Make test portlock logic work with meson
commit : 32ce58e9e97723958d43e09fc3203a302c1eb4b5
author : Andres Freund <andres@anarazel.de>
date : Fri, 21 Feb 2025 11:16:57 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 21 Feb 2025 11:16:57 -0500
Previously the portlock logic, added in 9b4eafcaf41, didn't actually work
properly when the tests were run via meson. 9b4eafcaf41 used the
MESON_BUILD_ROOT environment variable to determine the directory for the port
lock directory, but that's never set for running the tests. That meant that
each test used its own portlock dir, unless the PG_TEST_PORT_DIR environment
variable was set.
Fix the problem by setting top_builddir for the environment. That's also used
for the autoconf/make build.
Backpatch back to 16, where meson support was added.
Reported-by: Zharkov Roman <r.zharkov@postgrespro.ru>
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Backpatch-through: 16
M meson.build
M src/test/perl/PostgreSQL/Test/Cluster.pm
Fix cross-version upgrades with XMLSERIALIZE(NO INDENT)
commit : 665cafe8a4bd23b4f14194b0671d3ef09dc55864
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Feb 2025 20:37:31 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Feb 2025 20:37:31 +0900
Dumps from versions older than v16 do not know about NO INDENT in a
XMLSERIALIZE() clause. This commit adjusts AdjustUpgrade.pm so as NO
INDENT is discarded in the contents of the new dump adjusted for
comparison when the old version is v15 or older. This should be enough
to make the cross-version upgrade tests pass.
Per report from buildfarm member crake. Oversight in 984410b92326.
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/88b183f1-ebf9-4f51-9144-3704380ccae7@dunslane.net
Backpatch-through: 16
M src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
Support text position search functions with nondeterministic collations
commit : 329304c9012b2ac6d906afeb18062f9080dceef9
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 21 Feb 2025 12:21:17 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 21 Feb 2025 12:21:17 +0100
This allows using text position search functions with nondeterministic
collations. These functions are
- position, strpos
- replace
- split_part
- string_to_array
- string_to_table
which all use common internal infrastructure.
There was previously no internal implementation of this, so it was met
with a not-supported error. This adds the internal implementation and
removes the error.
Unlike with deterministic collations, the search cannot use any
byte-by-byte optimized techniques but has to go substring by
substring. We also need to consider that the found match could have a
different length than the needle and that there could be substrings of
different length matching at a position. In most cases, we need to
find the longest such substring (greedy semantics), but this can be
configured by each caller.
Reviewed-by: Euler Taveira <euler@eulerto.com>
Discussion: https://www.postgresql.org/message-id/flat/582b2613-0900-48ca-8b0d-340c06f4d400@eisentraut.org
M src/backend/utils/adt/varlena.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
doc: Add links to olsen93 and ong90 in bibliography
commit : 41336bf085599892b37ecfeace1576d9ae9a599a
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 21 Feb 2025 11:28:42 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 21 Feb 2025 11:28:42 +0100
The bibliography entries for olsen93 and ong90 lacked links to
online copies. While ong90 is available in digital form, the
olsen93 thesis is only available as a physical copy in the UCB
library. To save people from searching for it, we still link
to it via the UCB library page.
Reported-by: jian he <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CACJufxFcJYdRvzgt59N26XjFp2tFFUXu+VN+x8Uo0NbDUCMCbw@mail.gmail.com
M doc/src/sgml/biblio.sgml
Fix a WARNING for data origin discrepancies.
commit : b4e0d0c53fabe7d19b6a081d6c2b98aa4df55295
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 21 Feb 2025 14:34:40 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 21 Feb 2025 14:34:40 +0530
Previously, a WARNING was issued at the time of defining a subscription
with origin=NONE only when the publisher subscribed to the same table from
other publishers, indicating potential data origination from different
origins. However, the publisher can subscribe to the partition ancestors
or partition children of the table from other publishers, which could also
result in mixed-origin data inclusion. So, give a WARNING in those cases
as well.
Reported-by: Sergey Tatarintsev <s.tatarintsev@postgrespro.ru>
Author: Hou Zhijie <houzj.fnst@fujitsu.com>
Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Backpatch-through: 16, where it was introduced
Discussion: https://postgr.es/m/5eda6a9c-63cf-404d-8a49-8dcb116a29f3@postgrespro.ru
M doc/src/sgml/ref/create_subscription.sgml
M src/backend/commands/subscriptioncmds.c
M src/test/subscription/t/030_origin.pl
Add missing deparsing of [NO] IDENT to XMLSERIALIZE()
commit : 984410b923263cac901fa81e0efbe523e9c36df3
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Feb 2025 17:30:56 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Feb 2025 17:30:56 +0900
NO INDENT is the default, and is added if no explicit indentation
flag was provided with XMLSERIALIZE().
Oversight in 483bdb2afec9.
Author: Jim Jones <jim.jones@uni-muenster.de>
Discussion: https://postgr.es/m/bebd457e-5b43-46b3-8fc6-f6a6509483ba@uni-muenster.de
Backpatch-through: 16
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/xml.out
M src/test/regress/expected/xml_1.out
M src/test/regress/expected/xml_2.out
M src/test/regress/sql/xml.sql
Drop opcintype from index AM strategy translation API
commit : 7d6d2c4bbd730bd9af191d46d4fb01d5f5c30cf1
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 21 Feb 2025 08:34:35 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 21 Feb 2025 08:34:35 +0100
The type argument wasn't actually really necessary. It was a remnant
of converting the API of the gist strategy translation from using
opclass to using opfamily+opcintype (commits c09e5a6a016,
622f678c102). For looking up the gist translation function, we used
the convention "amproclefttype = amprocrighttype = opclass's
opcintype" (see pg_amproc.h). But each operator family should only
have one translation function, and getting the right type for the
lookup is sometimes cumbersome and fragile, so this is all
unnecessarily complicated.
To simplify this, change the gist stategy support procedure to take
"any", "any" as argument. (This is arbitrary but seems intuitive.
The alternative of using InvalidOid as argument(s) upsets various DDL
commands, so it's not practical.) Then we don't need opcintype for
the lookup, and we can remove it from all the API layers introduced by
commit c09e5a6a016.
This also adds some more documentation about the correct signature of
the gist support function and adds more checks in gistvalidate().
This was previously underspecified. (It relied implicitly on
convention mentioned above.)
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M contrib/btree_gist/btree_gist–1.7–1.8.sql
M doc/src/sgml/gist.sgml
M src/backend/access/gist/gistutil.c
M src/backend/access/gist/gistvalidate.c
M src/backend/access/hash/hash.c
M src/backend/access/index/amapi.c
M src/backend/access/nbtree/nbtree.c
M src/backend/catalog/index.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execReplication.c
M src/backend/replication/logical/relation.c
M src/include/access/amapi.h
M src/include/access/gist.h
M src/include/access/hash.h
M src/include/access/nbtree.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_amproc.dat
backend launchers void * arguments for binary data
commit : 7202d72787d3b93b692feae62ee963238580c877
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 21 Feb 2025 08:03:33 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 21 Feb 2025 08:03:33 +0100
Change backend launcher functions to take void * for binary data
instead of char *. This removes the need for numerous casts.
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/fd1fcedb-3492-4fc8-9e3e-74b97f2db6c7%40eisentraut.org
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/bgwriter.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/launch_backend.c
M src/backend/postmaster/pgarch.c
M src/backend/postmaster/postmaster.c
M src/backend/postmaster/startup.c
M src/backend/postmaster/syslogger.c
M src/backend/postmaster/walsummarizer.c
M src/backend/postmaster/walwriter.c
M src/backend/replication/logical/slotsync.c
M src/backend/replication/walreceiver.c
M src/backend/tcop/backend_startup.c
M src/include/postmaster/autovacuum.h
M src/include/postmaster/bgworker_internals.h
M src/include/postmaster/bgwriter.h
M src/include/postmaster/pgarch.h
M src/include/postmaster/postmaster.h
M src/include/postmaster/startup.h
M src/include/postmaster/syslogger.h
M src/include/postmaster/walsummarizer.h
M src/include/postmaster/walwriter.h
M src/include/replication/slotsync.h
M src/include/replication/walreceiver.h
M src/include/tcop/backend_startup.h
Fix for pg_restore_attribute_stats().
commit : b50a554cc84066577f0f0a3baafe2f1fac302006
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 20 Feb 2025 22:31:22 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 20 Feb 2025 22:31:22 -0800
Use RelationGetIndexExpressions() rather than rd_indexprs directly.
Author: Corey Huinker <corey.huinker@gmail.com>
M src/backend/statistics/attribute_stats.c
psql: Add support for pipelines
commit : 41625ab8ea3d0a2656dd0f067f1f0b61df63af97
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Feb 2025 11:19:59 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Feb 2025 11:19:59 +0900
With \bind, \parse, \bind_named and \close, it is possible to issue
queries from psql using the extended protocol. However, it was not
possible to send these queries using libpq's pipeline mode. This
feature has two advantages:
- Testing. Pipeline tests were only possible with pgbench, using TAP
tests. It now becomes possible to have more SQL tests that are able to
stress the backend with pipelines and extended queries. More tests will
be added in a follow-up commit that were discussed on some other
threads. Some external projects in the community had to implement their
own facility to work around this limitation.
- Emulation of custom workloads, with more control over the actions
taken by a client with libpq APIs. It is possible to emulate more
workload patterns to bottleneck the backend with the extended query
protocol.
This patch adds six new meta-commands to be able to control pipelines:
* \startpipeline starts a new pipeline. All extended queries are queued
until the end of the pipeline are reached or a sync request is sent and
processed.
* \endpipeline ends an existing pipeline. All queued commands are sent
to the server and all responses are processed by psql.
* \syncpipeline queues a synchronisation request, without flushing the
commands to the server, equivalent of PQsendPipelineSync().
* \flush, equivalent of PQflush().
* \flushrequest, equivalent of PQsendFlushRequest()
* \getresults reads the server's results for the queries in a pipeline.
Unsent data is automatically pushed when \getresults is called. It is
possible to control the number of results read in a single meta-command
execution with an optional parameter, 0 means that all the results
should be read.
Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Discussion: https://postgr.es/m/CAO6_XqroE7JuMEm1sWz55rp9fAYX2JwmcP_3m_v51vnOFdsLiQ@mail.gmail.com
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/common.c
M src/bin/psql/help.c
M src/bin/psql/settings.h
M src/bin/psql/tab-complete.in.c
M src/test/regress/expected/psql.out
A src/test/regress/expected/psql_pipeline.out
M src/test/regress/parallel_schedule
M src/test/regress/sql/psql.sql
A src/test/regress/sql/psql_pipeline.sql
Add braces for if block with large comment in psql's common.c
commit : 40af897eb777bc8a6afca14195587e79e57a5c06
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Feb 2025 09:18:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Feb 2025 09:18:49 +0900
A patch touching this area of the code is under review, and this format
makes the readability of the code slightly harder to parse.
Extracted from a larger patch by the same author.
Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/CAO6_XqroE7JuMEm1sWz55rp9fAYX2JwmcP_3m_v51vnOFdsLiQ@mail.gmail.com
M src/bin/psql/common.c
Add missing entry to oauth_validator test .gitignore
commit : 2c53dec7f4407c022f8b83e1a63fe0ae1bbb4dc2
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 20 Feb 2025 21:29:21 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 20 Feb 2025 21:29:21 +0100
Commit b3f0be788 accidentally missed adding the oauth client test
binary to the relevant .gitignore.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/2839306.1740082041@sss.pgh.pa.us
M src/test/modules/oauth_validator/.gitignore
Remove various unnecessary (char *) casts
commit : 3e4d868615408370fe6c2977c32f45a47c372b00
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 20 Feb 2025 19:49:27 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 20 Feb 2025 19:49:27 +0100
Remove a number of (char *) casts that are unnecessary. Or in some
cases, rewrite the code to make the purpose of the cast clearer.
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/fd1fcedb-3492-4fc8-9e3e-74b97f2db6c7%40eisentraut.org
M contrib/ltree/ltxtquery_io.c
M contrib/pg_trgm/trgm_gist.c
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/access/transam/xlog.c
M src/backend/libpq/auth.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/makefuncs.c
M src/backend/storage/page/bufpage.c
M src/backend/utils/cache/relmapper.c
M src/backend/utils/error/jsonlog.c
M src/bin/pg_basebackup/pg_receivewal.c
M src/bin/pg_dump/compress_gzip.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/psql/describe.c
M src/common/controldata_utils.c
M src/interfaces/ecpg/ecpglib/descriptor.c
M src/interfaces/ecpg/ecpglib/execute.c
M src/interfaces/ecpg/pgtypeslib/numeric.c
Trial fix for old cross-version upgrades.
commit : ab84d0ff806dd791ea9da5f1ca302daf3cf42980
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 20 Feb 2025 10:21:24 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 20 Feb 2025 10:21:24 -0800
Per buildfarm and reports, it seems that 9.X to 18 upgrades were
failing after commit 1fd1bd8710 due to an incorrect regex. Loosen the
regex to accommodate older versions.
Reported-by: vignesh C <vignesh21@gmail.com>
Reported-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/CALDaNm3GUs+U8Nt4S=V5zmb+K8-RfAc03vRENS0teeoq0Lc6Tw@mail.gmail.com
Discussion: https://postgr.es/m/ea4cbbc1-c5a5-43d1-9618-8ff3f2155bfe@dunslane.net
M src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
Ignore blank lines in pgindent exclude files
commit : 8e4d72573cc8b8bdc081661c0a3a76d6573eaa38
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 20 Feb 2025 11:36:07 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 20 Feb 2025 11:36:07 -0500
Currently a blank line matches everything, which is almost never what
someone would want. If they really want that they can use a wildcard
regex to do it.
Author: Zsolt Parragi <zsolt.parragi@percona.com>
Discussion: https://postgr.es/m/CAN4CZFNka+2q3=-Dithr4w65RJfwPaV92T62spEzLn+T4MgcMg@mail.gmail.com
M src/tools/pgindent/pgindent
cirrus: Temporarily fix libcurl link error
commit : 9d9a71002a1c1a04005c40d9cd5a6d3db50d32b0
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 20 Feb 2025 16:25:47 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 20 Feb 2025 16:25:47 +0100
On FreeBSD the ftp/curl port appears to be missing a minimum
version dependency on libssh2, so the following starts showing
up after upgrading to curl 8.11.1_1:
libcurl.so.4: Undefined symbol "libssh2_session_callback_set2"
Awaiting an upgrade of the FreeBSD CI images to version 14, work
around the issue.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAOYmi+kZAka0sdxCOBxsQc2ozEZGZKHWU_9nrPXg3sG1NJ-zJw@mail.gmail.com
M .cirrus.tasks.yml
Add support for OAUTHBEARER SASL mechanism
commit : b3f0be788afc17d2206e1ae1c731d8aeda1f2f59
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 20 Feb 2025 16:25:17 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 20 Feb 2025 16:25:17 +0100
This commit implements OAUTHBEARER, RFC 7628, and OAuth 2.0 Device
Authorization Grants, RFC 8628. In order to use this there is a
new pg_hba auth method called oauth. When speaking to a OAuth-
enabled server, it looks a bit like this:
$ psql 'host=example.org oauth_issuer=... oauth_client_id=...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
Device authorization is currently the only supported flow so the
OAuth issuer must support that in order for users to authenticate.
Third-party clients may however extend this and provide their own
flows. The built-in device authorization flow is currently not
supported on Windows.
In order for validation to happen server side a new framework for
plugging in OAuth validation modules is added. As validation is
implementation specific, with no default specified in the standard,
PostgreSQL does not ship with one built-in. Each pg_hba entry can
specify a specific validator or be left blank for the validator
installed as default.
This adds a requirement on libcurl for the client side support,
which is optional to build, but the server side has no additional
build requirements. In order to run the tests, Python is required
as this adds a https server written in Python. Tests are gated
behind PG_TEST_EXTRA as they open ports.
This patch has been a multi-year project with many contributors
involved with reviews and in-depth discussions: Michael Paquier,
Heikki Linnakangas, Zhihong Yu, Mahendrakar Srinivasarao, Andrey
Chudnovsky and Stephen Frost to name a few. While Jacob Champion
is the main author there have been some levels of hacking by others.
Daniel Gustafsson contributed the validation module and various bits
and pieces; Thomas Munro wrote the client side support for kqueue.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Antonin Houska <ah@cybertec.at>
Reviewed-by: Kashif Zeeshan <kashi.zeeshan@gmail.com>
Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com
M .cirrus.tasks.yml
M config/programs.m4
M configure
M configure.ac
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/filelist.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/libpq.sgml
A doc/src/sgml/oauth-validators.sgml
M doc/src/sgml/postgres.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/regress.sgml
M meson.build
M meson_options.txt
M src/Makefile.global.in
M src/backend/libpq/Makefile
A src/backend/libpq/auth-oauth.c
M src/backend/libpq/auth.c
M src/backend/libpq/hba.c
M src/backend/libpq/meson.build
M src/backend/libpq/pg_hba.conf.sample
M src/backend/utils/adt/hbafuncs.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
A src/include/common/oauth-common.h
M src/include/libpq/auth.h
M src/include/libpq/hba.h
A src/include/libpq/oauth.h
M src/include/pg_config.h.in
M src/interfaces/libpq/Makefile
M src/interfaces/libpq/exports.txt
A src/interfaces/libpq/fe-auth-oauth-curl.c
A src/interfaces/libpq/fe-auth-oauth.c
A src/interfaces/libpq/fe-auth-oauth.h
M src/interfaces/libpq/fe-auth.c
M src/interfaces/libpq/fe-auth.h
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
M src/interfaces/libpq/meson.build
M src/makefiles/meson.build
M src/test/authentication/t/001_password.pl
M src/test/modules/Makefile
M src/test/modules/meson.build
A src/test/modules/oauth_validator/.gitignore
A src/test/modules/oauth_validator/Makefile
A src/test/modules/oauth_validator/README
A src/test/modules/oauth_validator/fail_validator.c
A src/test/modules/oauth_validator/magic_validator.c
A src/test/modules/oauth_validator/meson.build
A src/test/modules/oauth_validator/oauth_hook_client.c
A src/test/modules/oauth_validator/t/001_server.pl
A src/test/modules/oauth_validator/t/002_client.pl
A src/test/modules/oauth_validator/t/OAuth/Server.pm
A src/test/modules/oauth_validator/t/oauth_server.py
A src/test/modules/oauth_validator/validator.c
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/tools/pgindent/pgindent
M src/tools/pgindent/typedefs.list
Transfer statistics during pg_upgrade.
commit : 1fd1bd871012732e3c6c482667d2f2c56f1a9395
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 20 Feb 2025 01:29:06 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 20 Feb 2025 01:29:06 -0800
Add support to pg_dump for dumping stats, and use that during
pg_upgrade so that statistics are transferred during upgrade. In most
cases this removes the need for a costly re-analyze after upgrade.
Some statistics are not transferred, such as extended statistics or
statistics with a custom stakind.
Now pg_dump accepts the options --schema-only, --no-schema,
--data-only, --no-data, --statistics-only, and --no-statistics; which
allow all combinations of schema, data, and/or stats. The options are
named this way to preserve compatibility with the previous
--schema-only and --data-only options.
Statistics are in SECTION_DATA, unless the object itself is in
SECTION_POST_DATA.
The stats are represented as calls to pg_restore_relation_stats() and
pg_restore_attribute_stats().
Author: Corey Huinker, Jeff Davis
Reviewed-by: Jian He
Discussion: https://postgr.es/m/CADkLM=fzX7QX6r78fShWDjNN3Vcr4PVAnvXxQ4DiGy6V=0bCUA@mail.gmail.com
Discussion: https://postgr.es/m/CADkLM%3DcB0rF3p_FuWRTMSV0983ihTRpsH%2BOCpNyiqE7Wk0vUWA%40mail.gmail.com
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_restore.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_directory.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/pg_dump_sort.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_dump/pg_restore.c
M src/bin/pg_dump/t/001_basic.pl
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/pg_upgrade/dump.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
M src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
M src/test/recovery/t/027_stream_regress.pl
M src/tools/pgindent/typedefs.list
Improve errdetail message added by ac0e33136a.
commit : 7da344b9f84f0c63590a34136f3fa5d0ab128657
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 20 Feb 2025 14:02:29 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 20 Feb 2025 14:02:29 +0530
Make it consistent with other similar messages.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Discussion: https://postgr.es/m/20250220.140839.1444694904721968348.horikyota.ntt@gmail.com
M src/backend/replication/slot.c
Don't lock partitions pruned by initial pruning
commit : 525392d5727f469e9a5882e1d728917a4be56147
author : Amit Langote <amitlan@postgresql.org>
date : Thu, 20 Feb 2025 17:09:48 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Thu, 20 Feb 2025 17:09:48 +0900
Before executing a cached generic plan, AcquireExecutorLocks() in
plancache.c locks all relations in a plan's range table to ensure the
plan is safe for execution. However, this locks runtime-prunable
relations that will later be pruned during "initial" runtime pruning,
introducing unnecessary overhead.
This commit defers locking for such relations to executor startup and
ensures that if the CachedPlan is invalidated due to concurrent DDL
during this window, replanning is triggered. Deferring these locks
avoids unnecessary locking overhead for pruned partitions, resulting
in significant speedup, particularly when many partitions are pruned
during initial runtime pruning.
* Changes to locking when executing generic plans:
AcquireExecutorLocks() now locks only unprunable relations, that is,
those found in PlannedStmt.unprunableRelids (introduced in commit
cbc127917e), to avoid locking runtime-prunable partitions
unnecessarily. The remaining locks are taken by
ExecDoInitialPruning(), which acquires them only for partitions that
survive pruning.
This deferral does not affect the locks required for permission
checking in InitPlan(), which takes place before initial pruning.
ExecCheckPermissions() now includes an Assert to verify that all
relations undergoing permission checks, none of which can be in the
set of runtime-prunable relations, are properly locked.
* Plan invalidation handling:
Deferring locks introduces a window where prunable relations may be
altered by concurrent DDL, invalidating the plan. A new function,
ExecutorStartCachedPlan(), wraps ExecutorStart() to detect and handle
invalidation caused by deferred locking. If invalidation occurs,
ExecutorStartCachedPlan() updates CachedPlan using the new
UpdateCachedPlan() function and retries execution with the updated
plan. To ensure all code paths that may be affected by this handle
invalidation properly, all callers of ExecutorStart that may execute a
PlannedStmt from a CachedPlan have been updated to use
ExecutorStartCachedPlan() instead.
UpdateCachedPlan() replaces stale plans in CachedPlan.stmt_list. A new
CachedPlan.stmt_context, created as a child of CachedPlan.context,
allows freeing old PlannedStmts while preserving the CachedPlan
structure and its statement list. This ensures that loops over
statements in upstream callers of ExecutorStartCachedPlan() remain
intact.
ExecutorStart() and ExecutorStart_hook implementations now return a
boolean value indicating whether plan initialization succeeded with a
valid PlanState tree in QueryDesc.planstate, or false otherwise, in
which case QueryDesc.planstate is NULL. Hook implementations are
required to call standard_ExecutorStart() at the beginning, and if it
returns false, they should do the same without proceeding.
* Testing:
To verify these changes, the delay_execution module tests scenarios
where cached plans become invalid due to changes in prunable relations
after deferred locks.
* Note to extension authors:
ExecutorStart_hook implementations must verify plan validity after
calling standard_ExecutorStart(), as explained earlier. For example:
if (prev_ExecutorStart)
plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
plan_valid = standard_ExecutorStart(queryDesc, eflags);
if (!plan_valid)
return false;
<extension-code>
return true;
Extensions accessing child relations, especially prunable partitions,
via ExecGetRangeTableRelation() must now ensure their RT indexes are
present in es_unpruned_relids (introduced in commit cbc127917e), or
they will encounter an error. This is a strict requirement after this
change, as only relations in that set are locked.
The idea of deferring some locks to executor startup, allowing locks
for prunable partitions to be skipped, was first proposed by Tom Lane.
Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions)
Reviewed-by: David Rowley <dgrowleyml@gmail.com> (earlier versions)
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (earlier versions)
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Junwang Zhao <zhjwpku@gmail.com>
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
M contrib/auto_explain/auto_explain.c
M contrib/pg_stat_statements/pg_stat_statements.c
M src/backend/commands/copyto.c
M src/backend/commands/createas.c
M src/backend/commands/explain.c
M src/backend/commands/extension.c
M src/backend/commands/matview.c
M src/backend/commands/portalcmds.c
M src/backend/commands/prepare.c
M src/backend/commands/trigger.c
M src/backend/executor/README
M src/backend/executor/execMain.c
M src/backend/executor/execParallel.c
M src/backend/executor/execPartition.c
M src/backend/executor/execUtils.c
M src/backend/executor/functions.c
M src/backend/executor/spi.c
M src/backend/tcop/postgres.c
M src/backend/tcop/pquery.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/mmgr/portalmem.c
M src/include/commands/explain.h
M src/include/commands/trigger.h
M src/include/executor/execdesc.h
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
M src/include/utils/plancache.h
M src/include/utils/portal.h
M src/test/modules/delay_execution/Makefile
M src/test/modules/delay_execution/delay_execution.c
A src/test/modules/delay_execution/expected/cached-plan-inval.out
M src/test/modules/delay_execution/meson.build
A src/test/modules/delay_execution/specs/cached-plan-inval.spec
Include schema/table publications even with exclude options in dump.
commit : 4aa6fa3cd0a2422f1bb47db837b82f2b53d4c065
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 20 Feb 2025 11:25:29 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 20 Feb 2025 11:25:29 +0530
The current implementation inconsistently includes public schema but not
information_schema when those are specified in FOR TABLES IN SCHMEA ...
Apart from that, the current behavior for publications w.r.t exclude table
and schema (--exclude-table, --exclude-schema) option differs from what we
do at other places. We try to avoid including publications for
corresponding tables or schemas when an exclude-table or exclude-schema
option is given, unlike what we do for views using functions defined in a
particular schema or a subscription pointing to publications with their
corresponding exclude options.
I decided not to backpatch this as it leads to a behavior change and we don't
see any field report for current behavior.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Author: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/1270733.1734134272@sss.pgh.pa.us
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/t/002_pg_dump.pl
doc: Fix typo in section "WAL configuration"
commit : f11674f8dfe811390dfeaf6f3c3a554dbeb790e1
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 14:22:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 14:22:00 +0900
pg_stat_io has an attribute named fsync_time, not sync_time.
Oversight in 2f70871c2bc1.
Discussion: https://postgr.es/m/Z7RkQ0EfYaqqjgz/@ip-10-97-1-34.eu-west-3.compute.internal
M doc/src/sgml/wal.sgml
doc: Add details about object "wal" in pg_stat_io
commit : 4538bd3f1dde065820438ea461ad30c993982d92
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 14:16:23 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 14:16:23 +0900
This commit adds a short description of what kind of activity is tracked
in pg_stat_io for the object "wal", with a link pointing to the section
"WAL configuration" that has a lot of details on the matter.
This should perhaps have been added in a051e71e28a1, but things are what
they are.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/Z7RkQ0EfYaqqjgz/@ip-10-97-1-34.eu-west-3.compute.internal
M doc/src/sgml/monitoring.sgml
doc: Recommend pg_stat_io rather than pg_stat_wal in WAL configuration
commit : 2f70871c2bc18d103d5dd25ff9d91e0da29412ee
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 13:55:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 13:55:00 +0900
Since a051e71e28a1, pg_stat_io is able to track statistics for the WAL
activity, providing an equivalent of pg_stat_wal with more granularity
for the fsyncs/writes counts and timings, as the data is split across
backend types.
This commit now recommends pg_stat_io rather than pg_stat_wal in the
section "WAL configuration", some of the latter's attributes being
candidate for removal in a follow-up commit.
Extracted from a larger patch by the same author.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/Z7RkQ0EfYaqqjgz/@ip-10-97-1-34.eu-west-3.compute.internal
M doc/src/sgml/wal.sgml
Fix FATAL message for invalid recovery timeline at beginning of recovery
commit : 71f17823ba010296da9946bd906bb8bcad6325bc
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 10:42:20 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 10:42:20 +0900
If the requested recovery timeline is not reachable, the logged
checkpoint and timeline should to be the values read from the
backup_label when it is defined. The message generated used the values
from the control file in this case, which is fine when recovering from
the control file without a backup_label, but not if there is a
backup_label.
Issue introduced in ee994272ca50. v15 has introduced xlogrecovery.c and
more simplifications in this area (4a92a1c3d1c3, a27048cbcb58), making
this change a bit simpler to think about, so backpatch only down to this
version.
Author: David Steele <david@pgbackrest.org>
Reviewed-by: Andrey M. Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Benoit Lobréau <benoit.lobreau@dalibo.com>
Discussion: https://postgr.es/m/c3d617d4-1696-4aa7-8a4d-5a7d19cc5618@pgbackrest.org
Backpatch-through: 15
M src/backend/access/transam/xlogrecovery.c
pgbench: Increase RLIMIT_NOFILE if necessary
commit : d38bab5edd60dbe6309512b6c8daea37ce579b70
author : Andres Freund <andres@anarazel.de>
date : Wed, 19 Feb 2025 19:35:09 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 19 Feb 2025 19:35:09 -0500
pgbench already had code to check if the soft rlimit is too low for the
specified number of connections. If too low, it errored out, telling the user
to increase the limit.
However, we can do better: If the hard limit allows, increase the soft limit
to be sufficiently for the number of connections.
It is common for the soft limit to be considerably lower than the hard limit,
due to the danger of soft limits > 1024 breaking programs that use the
select(2), as explained in [1].
[1]: https://0pointer.net/blog/file-descriptor-limits.html
Author: Jelte Fennema-Nio <postgres@jeltef.nl>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAGECzQQh6VSy3KG4pN1d%3Dh9J%3DD1rStFCMR%2Bt7yh_Kwj-g87aLQ%40mail.gmail.com
M src/bin/pgbench/pgbench.c
test_escape: Fix output of --help
commit : 9b1cb58c5f2127ddc682778b486739059ea2479d
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 09:30:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 09:30:54 +0900
The short option name -f was not listed, only its long option name
--force-unsupported.
Author: Japin Li
Discussion: https://postgr.es/m/ME0P300MB04452BD1FB1B277D4C1C20B9B6C52@ME0P300MB0445.AUSP300.PROD.OUTLOOK.COM
Backpatch-through: 13
M src/test/modules/test_escape/test_escape.c
Correct relation size estimate with low fillfactor
commit : 9ba7bcc894f1cafb0793a3a62658bd91db1dbecf
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 19 Feb 2025 23:51:48 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 19 Feb 2025 23:51:48 +0100
Since commit 29cf61ade3, table_block_relation_estimate_size() considers
fillfactor when estimating number of rows in a relation before the first
ANALYZE. The formula however did not consider tuples may be larger than
available space determined by fillfactor, ending with density 0. This
ultimately means the relation was estimated to contain a single row.
The executor however places at least one tuple per page, even with very
low fillfactor values, so the density should be at least 1. Fixed by
clamping the density estimate using clamp_row_est().
Reported by Heikki Linnakangas. Fix by me, with regression test inspired
by example provided by Heikki.
Backpatch to 17, where the issue was introduced.
Reported-by: Heikki Linnakangas
Backpatch-through: 17
Discussion: https://postgr.es/m/2bf9d973-7789-4937-a7ca-0af9fb49c71e@iki.fi
M src/backend/access/table/tableam.c
M src/test/regress/expected/stats.out
M src/test/regress/sql/stats.sql
Assert that ExecOpenIndices and ExecCloseIndices are not repeated.
commit : e596e077bbb3b512bbc80610d64dc007a5761ce6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 19 Feb 2025 16:45:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 19 Feb 2025 16:45:12 -0500
These functions should be called at most once per ResultRelInfo;
it's wasteful to do otherwise, and certainly the pattern of
opening twice and then closing twice is a bad idea. Moreover,
aminsertcleanup functions might not be prepared to be called twice,
as the just-hardened code in BRIN demonstrates.
This amounts to an API change, since such coding patterns were
safe even if wasteful before v17. Hence, apply to HEAD only.
(Extension code violating this new rule faces some risk in v17,
but we just fixed brininsertcleanup and there are probably few
other aminsertcleanup functions as yet. So the odds of breaking
usable code seem higher than the odds of doing something useful
with a back-patch.)
Bug: #18815
Reported-by: Sergey Belyashov <sergey.belyashov@gmail.com>
Discussion: https://postgr.es/m/18815-2a0407cc7f40b327@postgresql.org
M src/backend/executor/execIndexing.c
Fix crash in brininsertcleanup during logical replication.
commit : 9ff68679b50f291d1c38e09748721a9732439508
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 19 Feb 2025 16:35:15 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 19 Feb 2025 16:35:15 -0500
Logical replication crashes if the subscriber's partitioned table
has a BRIN index. There are two independently blamable causes,
and this patch fixes both:
1. brininsertcleanup fails if called twice for the same IndexInfo,
because it half-destroys its BrinInsertState but leaves it still
linked from ii_AmCache. brininsert would also fail in that state,
so it's pretty hard to see any advantage to this coding. Fully
remove the BrinInsertState, instead, so that a new brininsert
call would create a new cache.
2. A logical replication subscriber sometimes does ExecOpenIndices
twice on the same ResultRelInfo, followed by doing ExecCloseIndices
twice; the second call reaches the brininsertcleanup bug. Quite
aside from tickling unexpected cases in aminsertcleanup methods,
this seems very wasteful, because the IndexInfos built in the
first ExecOpenIndices call are just lost during the second call,
and have to be rebuilt at possibly-nontrivial cost. We should
establish a coding rule that you don't do that.
The problematic coding is that when the target table is partitioned,
apply_handle_tuple_routing calls ExecFindPartition which does
ExecOpenIndices (and expects that ExecCleanupTupleRouting will
close the indexes again). Using the ResultRelInfo made by
ExecFindPartition, it calls apply_handle_delete_internal or
apply_handle_insert_internal, both of which think they need to do
ExecOpenIndices/ExecCloseIndices for themselves. They do in the main
non-partitioned code paths, but not here. The simplest fix is to pull
their ExecOpenIndices/ExecCloseIndices calls out and put them in the
call sites for the non-partitioned cases. (We could have refactored
apply_handle_update_internal similarly, but I did not do so today
because there's no bug there: the partitioned code path doesn't
call it.)
Also, remove the always-duplicative open/close calls within
apply_handle_tuple_routing itself.
Since brininsertcleanup and indeed the whole aminsertcleanup mechanism
are new in v17, there's no observable bug in older branches. A case
could be made for trying to avoid these duplicative open/close calls
in the older branches, but for now it seems not worth the trouble and
risk of new bugs.
Bug: #18815
Reported-by: Sergey Belyashov <sergey.belyashov@gmail.com>
Discussion: https://postgr.es/m/18815-2a0407cc7f40b327@postgresql.org
Backpatch-through: 17
M src/backend/access/brin/brin.c
M src/backend/replication/logical/worker.c
M src/test/subscription/t/013_partition.pl
Consider BufFiles when adjusting hashjoin parameters
commit : a1b4f289beec19534e5e34e1a962c5828b306ce2
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 19 Feb 2025 20:29:26 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 19 Feb 2025 20:29:26 +0100
Until now ExecChooseHashTableSize() considered only the size of the
in-memory hash table, and ignored the memory needed for the batch files.
Which can be a significant amount, because each batch needs two BufFiles
(each with a BLCKSZ buffer). The same issue applies to increasing the
number of batches during execution.
It's also possible to trigger a "batch explosion", e.g. due to duplicate
values or skew. We've seen reports of joins with hundreds of thousands
(or even millions) of batches, consuming gigabytes of memory, triggering
OOM errors. These cases may be fairly rare, but it's clearly possible to
hit them.
These issues can't be prevented during planning. Even if we improve
that, it does not help with execution-time batch explosion. We can
however reduce the impact and use as little memory as possible.
This patch improves the behavior by adjusting how the memory is divided
between the hash table and batch files. It may be better to use fewer
batch files, even if it means the hash table will exceed the limit.
The capacity of the hash node may be increased either by doubling he
number of batches, or doubling the size of the in-memory hash table. The
outcome is the same, but the memory usage may be very different. For low
nbatch values it's better to add batches, for high nbatch values it's
better to allow a larger hash table.
The patch considers both options, both during the initial sizing and
then during execution, to minimize how much the limit gets exceeded.
It might seem this patch is relaxing the memory limit - allowing it to
be exceeded. But that's not really the case. It has always been like
that, except the memory used by batches was ignored.
Allowing the hash table to grow may also prevent the batch explosion.
If there's a large batch that can't be split (due to hash collisions or
duplicate values), at some point the memory limit will increase enough
for the batch to fit into the hash table.
This patch was in the works for a long time. The early versions were
posted in 2019, and revived every year or two when we happened to get
the next report of OOM due to a hashjoin batch explosion. Each of those
patch versions were reviewed by a couple people. I'm mentioning only
Melanie Plageman and Robert Haas, because they reviewed the last
version, and the older patches are very different.
Reviewed-by: Melanie Plageman, Robert Haas
Discussion: https://postgr.es/m/7bed6c08-72a0-4ab9-a79c-e01fcdd0940f@vondra.me
Discussion: https://postgr.es/m/20190504003414.bulcbnge3rhwhcsh%40development
Discussion: https://postgr.es/m/20190428141901.5dsbge2ka3rxmpk6%40development
M src/backend/executor/nodeHash.c
tests: BackgroundPsql: Fix potential for lost errors on windows
commit : 8b886a4e34886bf2d1dd4f5e7d95f4ddadc3afa8
author : Andres Freund <andres@anarazel.de>
date : Wed, 19 Feb 2025 10:45:48 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 19 Feb 2025 10:45:48 -0500
This addresses various corner cases in BackgroundPsql:
- On windows stdout and stderr may arrive out of order, leading to errors not
being reported, or attributed to the wrong statement.
To fix, emit the "query-separation banner" on both stdout and stderr and
wait for both.
- Very occasionally the "query-separation banner" would not get removed, because
we waited until the banner arrived, but then replaced the banner plus
newline.
To fix, wait for banner and newline.
- For interactive psql replacing $banner\n is not sufficient, interactive psql
outputs \r\n.
- For interactive psql, where commands are echoed to stdout, the \echo
command, rather than its output, would be matched.
This would sometimes lead to output from the prior query, or wait_connect(),
being returned in the next command.
This also affected wait_connect(), leading to sometimes sending queries to
psql before the connection actually was established.
While debugging these issues I also found that it's hard to know whether a
query separation banner was attributed to the right query. Make that easier by
counting the queries each BackgroundPsql instance has emitted and include the
number in the banner.
Also emit psql stdout/stderr in query() and wait_connect() as Test::More
notes, without that it's rather hard to debug some issues in CI and buildfarm.
As this can cause issues not just to-be-added tests, but also existing ones,
backpatch the fix to all supported versions.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/wmovm6xcbwh7twdtymxuboaoarbvwj2haasd3sikzlb3dkgz76@n45rzycluzft
Backpatch-through: 13
M src/test/perl/PostgreSQL/Test/BackgroundPsql.pm
Add ATAlterConstraint struct for ALTER .. CONSTRAINT
commit : 80d7f990496b1c7be61d9a00a2635b7d96b96197
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 19 Feb 2025 13:06:13 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 19 Feb 2025 13:06:13 +0100
Replace the use of Constraint with a new ATAlterConstraint struct, which
allows us to pass additional information. No functionality is added by
this commit. This is necessary for future work that allows altering
constraints in other ways.
I (Álvaro) took the liberty of restructuring the code for ALTER
CONSTRAINT beyond what Amul did. The original coding before Amul's
patch was unnecessarily baroque, and this change makes things simpler
by removing one level of subroutine. Also, partly remove the assumption
that only partitioned tables are relevant (by passing sensible 'recurse'
arguments) and no longer ignore whether ONLY was specified. I say
'partly' because the current coding only walks down via the 'conparentid'
relationship, which is only used for partitioned tables; but future
patches could handle ONLY or not for other types of constraint changes
for legacy inheritance trees too.
Author: Amul Sul <sulamul@gmail.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/CAAJ_b94bfgPV-8Mw_HwSBeheVwaK9=5s+7+KbBj_NpwXQFgDGg@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/backend/parser/gram.y
M src/include/nodes/parsenodes.h
M src/tools/pgindent/typedefs.list
Improve statistics estimation for single-column GROUP BY in sub-queries
commit : e983ee9380fcac746f9b6b7c360d255d9fcd0907
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 19 Feb 2025 11:56:54 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 19 Feb 2025 11:56:54 +0200
This commit follows the idea of the 4767bc8ff2. If sub-query has only one
GROUP BY column, we can consider its output variable as being unique. We can
employ this fact in the statistics to make more precise estimations in the
upper query block.
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
M src/backend/utils/adt/selfuncs.c
M src/include/utils/selfuncs.h
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Add a test for commit ac0e33136a using the injection point.
commit : 8a695d7998be67445b9cd8e67faa684d4e87a40d
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 19 Feb 2025 14:52:32 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 19 Feb 2025 14:52:32 +0530
This test uses an injection point to bypass the time overhead caused by
the idle_replication_slot_timeout GUC, which has a minimum value of one
minute.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Author: Nisha Moond <nisha.moond412@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CALj2ACW4aUe-_uFQOjdWCEN-xXoLGhmvRFnL8SNw_TZ5nJe+aw@mail.gmail.com
M src/backend/replication/slot.c
M src/test/recovery/meson.build
A src/test/recovery/t/044_invalidate_inactive_slots.pl
Add support for LIKE in CREATE FOREIGN TABLE
commit : 302cf15759233e654512979286ce1a5c3b36625f
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Feb 2025 15:50:37 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Feb 2025 15:50:37 +0900
LIKE enables the creation of foreign tables based on the column
definitions, constraints and objects of the defined source relation(s).
This feature mirrors the behavior of CREATE TABLE LIKE, but ignores
the INCLUDING sub-options that do not make sense for foreign tables:
INDEXES, COMPRESSION, IDENTITY and STORAGE. The supported sub-options
are COMMENTS, CONSTRAINTS, DEFAULTS, GENERATED and STATISTICS, mapping
with the clauses already supported by the command.
Note that the restriction with LIKE in CREATE FOREIGN TABLE was added in
a0c6dfeecfcc.
Author: Zhang Mingli
Reviewed-by: Álvaro Herrera, Sami Imseih, Michael Paquier
Discussion: https://postgr.es/m/42d3f855-2275-4361-a42a-826172ca2dc4@Spark
M doc/src/sgml/ref/create_foreign_table.sgml
M src/backend/parser/parse_utilcmd.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
doc: Fix some issues with JSON_TABLE() examples
commit : e7563e3c75a83898eff495533b4736093c73778a
author : Amit Langote <amitlan@postgresql.org>
date : Wed, 19 Feb 2025 15:08:17 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Wed, 19 Feb 2025 15:08:17 +0900
1. Remove an unused PASSING variable.
2. Adjust formatting of JSON data used in an example to be valid
under strict mode
Reported-by: Miłosz Chmura <mieszko4@gmail.com>
Author: Robert Treat <rob@xzilla.net>
Discussion: https://postgr.es/m/173859550337.1071.4748984213168572913@wrigleys.postgresql.org
M doc/src/sgml/func.sgml
Invalidate inactive replication slots.
commit : ac0e33136abc4668c9b08e1ba7db69afe1e0e2c3
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 19 Feb 2025 09:29:50 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 19 Feb 2025 09:29:50 +0530
This commit introduces idle_replication_slot_timeout GUC that allows
inactive slots to be invalidated at the time of checkpoint. Because
checkpoints happen checkpoint_timeout intervals, there can be some lag
between when the idle_replication_slot_timeout was exceeded and when the
slot invalidation is triggered at the next checkpoint. To avoid such lags,
users can force a checkpoint to promptly invalidate inactive slots.
Note that the idle timeout invalidation mechanism is not applicable for
slots that do not reserve WAL or for slots on the standby server that are
synced from the primary server (i.e., standby slots having 'synced' field
'true'). Synced slots are always considered to be inactive because they
don't perform logical decoding to produce changes.
The slots can become inactive for a long period if a subscriber is down
due to a system error or inaccessible because of network issues. If such a
situation persists, it might be more practical to recreate the subscriber
rather than attempt to recover the node and wait for it to catch up which
could be time-consuming.
Then, external tools could create replication slots (e.g., for migrations
or upgrades) that may fail to remove them if an error occurs, leaving
behind unused slots that take up space and resources. Manually cleaning
them up can be tedious and error-prone, and without intervention, these
lingering slots can cause unnecessary WAL retention and system bloat.
As the duration of idle_replication_slot_timeout is in minutes, any test
using that would be time-consuming. We are planning to commit a follow up
patch for tests by using the injection point framework.
Author: Nisha Moond <nisha.moond412@gmail.com>
Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Hou Zhijie <houzj.fnst@fujitsu.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/CALj2ACW4aUe-_uFQOjdWCEN-xXoLGhmvRFnL8SNw_TZ5nJe+aw@mail.gmail.com
Discussion: https://postgr.es/m/OS0PR01MB5716C131A7D80DAE8CB9E88794FC2@OS0PR01MB5716.jpnprd01.prod.outlook.com
M doc/src/sgml/config.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/system-views.sgml
M src/backend/access/transam/xlog.c
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/utils/adt/timestamp.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/pg_basebackup/pg_createsubscriber.c
M src/bin/pg_upgrade/server.c
M src/include/replication/slot.h
M src/include/utils/guc_hooks.h
M src/include/utils/timestamp.h
M src/tools/pgindent/typedefs.list
Update to latest Snowball sources.
commit : b464e51ab32fbf09cf5d9c911a8e26f491ad1f44
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 18 Feb 2025 21:13:46 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 18 Feb 2025 21:13:46 -0500
It's been some time since we did this, partly because the upstream
snowball project hasn't formally tagged a new release since 2021.
The main motivation for doing it now is to absorb a bug fix
(their commit e322673a841d9abd69994ae8cd20e191090b6ef4), which
prevents a null pointer dereference crash if SN_create_env() gets
a malloc failure at just the wrong point. We'll patch the back
branches with only that change, but we might as well do the full
sync dance on HEAD.
Aside from a bunch of mostly-minor tweaks to existing stemmers, this
update adds a new stemmer for Estonian. It also removes the existing
stemmer for Romanian using ISO-8859-2 encoding. Upstream apparently
concluded that ISO-8859-2 doesn't provide an adequate representation
of some Romanian characters, and the UTF-8 implementation should be
used instead.
While at it, update the README's instructions for doing a sync,
which have not been adjusted during the addition of meson tooling.
Thanks to Maksim Korotkov for discovering the null-pointer
bug and submitting the fix to upstream snowball.
Reported-by: Maksim Korotkov <m.korotkov@postgrespro.ru>
Discussion: https://postgr.es/m/1d1a46-67ab1000-21-80c451@83151435
M doc/src/sgml/textsearch.sgml
M src/backend/snowball/Makefile
M src/backend/snowball/README
M src/backend/snowball/dict_snowball.c
M src/backend/snowball/libstemmer/api.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_basque.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_catalan.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_danish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_dutch.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_english.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_finnish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_french.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_german.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_indonesian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_irish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_italian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_norwegian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_porter.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_portuguese.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_spanish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_swedish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_2_hungarian.c
D src/backend/snowball/libstemmer/stem_ISO_8859_2_romanian.c
M src/backend/snowball/libstemmer/stem_KOI8_R_russian.c
M src/backend/snowball/libstemmer/stem_UTF_8_arabic.c
M src/backend/snowball/libstemmer/stem_UTF_8_armenian.c
M src/backend/snowball/libstemmer/stem_UTF_8_basque.c
M src/backend/snowball/libstemmer/stem_UTF_8_catalan.c
M src/backend/snowball/libstemmer/stem_UTF_8_danish.c
M src/backend/snowball/libstemmer/stem_UTF_8_dutch.c
M src/backend/snowball/libstemmer/stem_UTF_8_english.c
A src/backend/snowball/libstemmer/stem_UTF_8_estonian.c
M src/backend/snowball/libstemmer/stem_UTF_8_finnish.c
M src/backend/snowball/libstemmer/stem_UTF_8_french.c
M src/backend/snowball/libstemmer/stem_UTF_8_german.c
M src/backend/snowball/libstemmer/stem_UTF_8_greek.c
M src/backend/snowball/libstemmer/stem_UTF_8_hindi.c
M src/backend/snowball/libstemmer/stem_UTF_8_hungarian.c
M src/backend/snowball/libstemmer/stem_UTF_8_indonesian.c
M src/backend/snowball/libstemmer/stem_UTF_8_irish.c
M src/backend/snowball/libstemmer/stem_UTF_8_italian.c
M src/backend/snowball/libstemmer/stem_UTF_8_lithuanian.c
M src/backend/snowball/libstemmer/stem_UTF_8_nepali.c
M src/backend/snowball/libstemmer/stem_UTF_8_norwegian.c
M src/backend/snowball/libstemmer/stem_UTF_8_porter.c
M src/backend/snowball/libstemmer/stem_UTF_8_portuguese.c
M src/backend/snowball/libstemmer/stem_UTF_8_romanian.c
M src/backend/snowball/libstemmer/stem_UTF_8_russian.c
M src/backend/snowball/libstemmer/stem_UTF_8_serbian.c
M src/backend/snowball/libstemmer/stem_UTF_8_spanish.c
M src/backend/snowball/libstemmer/stem_UTF_8_swedish.c
M src/backend/snowball/libstemmer/stem_UTF_8_tamil.c
M src/backend/snowball/libstemmer/stem_UTF_8_turkish.c
M src/backend/snowball/libstemmer/stem_UTF_8_yiddish.c
M src/backend/snowball/libstemmer/utilities.c
M src/backend/snowball/meson.build
M src/backend/snowball/snowball_create.pl
M src/bin/initdb/initdb.c
M src/include/snowball/libstemmer/header.h
D src/include/snowball/libstemmer/stem_ISO_8859_2_romanian.h
A src/include/snowball/libstemmer/stem_UTF_8_estonian.h
Fix unsafe access to BufferDescriptors
commit : 71d02dc478d574c75bd0af82cec774c7b9059a61
author : Richard Guo <rguo@postgresql.org>
date : Wed, 19 Feb 2025 11:05:35 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Wed, 19 Feb 2025 11:05:35 +0900
When considering a local buffer, the GetBufferDescriptor() call in
BufferGetLSNAtomic() would be retrieving a shared buffer with a bad
buffer ID. Since the code checks whether the buffer is shared before
using the retrieved BufferDesc, this issue did not lead to any
malfunction. Nonetheless this seems like trouble waiting to happen,
so fix it by ensuring that GetBufferDescriptor() is only called when
we know the buffer is shared.
Author: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/CAHewXNku-o46-9cmUgyv6LkSZ25doDrWq32p=oz9kfD8ovVJMg@mail.gmail.com
Backpatch-through: 13
M src/backend/storage/buffer/bufmgr.c
Fix freeing a child join's SpecialJoinInfo
commit : c39392ebaeeff7dafbc255844bcc605aaef01d7d
author : Richard Guo <rguo@postgresql.org>
date : Wed, 19 Feb 2025 10:02:32 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Wed, 19 Feb 2025 10:02:32 +0900
In try_partitionwise_join, we try to break down the join between two
partitioned relations into joins between matching partitions. To
achieve this, we iterate through each pair of partitions from the two
joining relations and create child join relations for them. To reduce
memory accumulation during each iteration, one step we take is freeing
the SpecialJoinInfos created for the child joins.
A child join's SpecialJoinInfo is a copy of the parent join's
SpecialJoinInfo, with some members being translated copies of their
counterparts in the parent. However, when freeing the bitmapset
members in a child join's SpecialJoinInfo, we failed to check whether
they were translated copies. As a result, we inadvertently freed the
members that were still in use by the parent SpecialJoinInfo, leading
to crashes when those freed members were accessed.
To fix, check if each member of the child join's SpecialJoinInfo is a
translated copy and free it only if that's the case. This requires
passing the parent join's SpecialJoinInfo as a parameter to
free_child_join_sjinfo.
Back-patch to v17 where this bug crept in.
Bug: #18806
Reported-by: 孟令彬 <m_lingbin@126.com>
Diagnosed-by: Tender Wang <tndrwang@gmail.com>
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/18806-d70b0c9fdf63dcbf@postgresql.org
Backpatch-through: 17
M src/backend/optimizer/path/joinrels.c
M src/test/regress/expected/partition_join.out
M src/test/regress/sql/partition_join.sql
test_escape: Fix handling of short options in getopt_long()
commit : aef6f907f63106437d059e1a0a2b696b067fb0af
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Feb 2025 09:45:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Feb 2025 09:45:42 +0900
This addresses two errors in the module, based on the set of options
supported:
- '-c', for --conninfo, was not listed.
- '-f', for --force-unsupported, was not listed.
While on it, these are now listed in an alphabetical order.
Author: Japin Li
Discussion: https://postgr.es/m/ME0P300MB04451FB20CE0346A59C25CADB6FA2@ME0P300MB0445.AUSP300.PROD.OUTLOOK.COM
Backpatch-through: 13
M src/test/modules/test_escape/test_escape.c
Make the description of some GUCs more consistent
commit : f2e4c2b2039e8e1944c6b50531910cc389f47ace
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Feb 2025 08:42:35 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Feb 2025 08:42:35 +0900
This commit improves the description of a couple of GUCs, to be more
consistent with the style of their surroundings:
* array_nulls
* enable_self_join_elimination
* optimize_bounded_sort
* row_security
* synchronize_seqscans
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20250218.103240.1422205966404509831.horikyota.ntt@gmail.com
M src/backend/utils/misc/guc_tables.c
doc: add example of sign mismatch with POSIX/ISO-8601 time zones
commit : 06dc1ffd24096f7c71d1abeaa9e96fec4db9313d
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 18 Feb 2025 15:51:31 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 18 Feb 2025 15:51:31 -0500
Author: Laurenz Albe
Discussion: https://postgr.es/m/eb4d1e15c6822c1937be1491118500dd9201492f.camel@cybertec.at
M doc/src/sgml/datatype.sgml
M doc/src/sgml/func.sgml
Update outdated comments in nodeAgg.c.
commit : a1f7f80bfef105c8c02153e8832d4106df3cf459
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 18 Feb 2025 10:28:05 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 18 Feb 2025 10:28:05 -0800
Author: Zhang Mingli
Reviewed-by: Richard Guo
Discussion: https://postgr.es/m/198a8d1e-0792-4e7f-828e-902aa342f36e@Spark
M src/backend/executor/nodeAgg.c
Reduce scope of heap vacuum per_buffer_data
commit : c623e8593ec4ee6987f3cd9350ced7caf8526ed2
author : Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 18 Feb 2025 09:28:10 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 18 Feb 2025 09:28:10 -0500
Move lazy_scan_heap()'s per_buffer_data variable into a tighter scope.
In lazy_scan_heap()'s phase I heap vacuuming, the read stream API
returns a pointer to the next block number to vacuum. As long as
read_stream_next_buffer() returns a valid buffer, per_buffer_data should
always be valid.
Move per_buffer_data into a tighter scope and make sure it is reset to
NULL on each iteration so that we get a core dump instead of bogus data
from a previous block if something goes wrong in the read stream API.
Suggested-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/626104.1739729538%40sss.pgh.pa.us
M src/backend/access/heap/vacuumlazy.c
Add PGErrorVerbosity to typedefs.list
commit : 95ef3d902950317650ccea4b9a8bef47936eae35
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 18 Feb 2025 13:23:13 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 18 Feb 2025 13:23:13 +0100
PGErrorVerbosity was missing which resulted in incorrect whitespace
alignment going back all the way to e3860ffa4dd0. No backpatch for
this though since we don't pgindent backbranches.
Author: Jelte Fennema-Nio <postgres@jeltef.nl>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAGECzQTVi8n-HW4Q27je-b9ckQk7zf6bS_it42gNvQu+DX0NCQ@mail.gmail.com
M src/include/utils/elog.h
M src/tools/pgindent/typedefs.list
Fix poorly written regression test
commit : 593509202f669dbc4a9db33bb3aca2bd68f7ab5c
author : David Rowley <drowley@postgresql.org>
date : Wed, 19 Feb 2025 00:42:22 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 19 Feb 2025 00:42:22 +1300
bd10ec529 added code to allow redundant functionally dependent GROUP BY
columns to be removed using unique indexes and NOT NULL constraints as
proofs of functional dependency. In that commit, I (David) added a test
to ensure that when there are multiple indexes available to remove columns
that we pick the index that allows us to remove the most columns. This
test was faulty as it assumed the t3 table's primary key index was valid
to use as functional dependency proof, but that's not the case since
that's defined as deferrable.
Here we adjust the tests added by that commit to use the t2 table instead.
That's defined with a non-deferrable primary key.
Author: songjinzhou <tsinghualucky912@foxmail.com>
Author: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Japin Li <japinli@hotmail.com>
Discussion: https://postgr.es/m/tencent_CD414C79D39668455DF80D35143B87634C08@qq.com
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql
Raise a WARNING for max_slot_wal_keep_size in pg_createsubscriber.
commit : 217919dd0954f54402e8d0a38cd203a740754077
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 18 Feb 2025 12:15:43 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 18 Feb 2025 12:15:43 +0530
During the pg_createsubscriber execution, it is possible that the required
WAL is removed from the primary/publisher node due to
'max_slot_wal_keep_size'.
This patch raises a WARNING during the '--dry-run' mode if the
'max_slot_wal_keep_size' is set to a non-default value on the
primary/publisher node.
Author: Shubham Khanna <khannashubham1197@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/CAHv8Rj+deqsQXOMa7Tck8CBQUbsua=+4AuMVQ2=MPM0f-ZHbjA@mail.gmail.com
M doc/src/sgml/ref/pg_createsubscriber.sgml
M src/bin/pg_basebackup/pg_createsubscriber.c
Specialize intarray sorting
commit : 53d3daa491be458e543dd5bf24d40595e588e4e7
author : John Naylor <john.naylor@postgresql.org>
date : Tue, 18 Feb 2025 11:04:55 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Tue, 18 Feb 2025 11:04:55 +0700
There is at least one report in the field of storing millions of
integers in arrays, so it seems like a good time to specialize
intarray's qsort function. In doing so, streamline the comparators:
Previously there were three, two for each direction for sorting
and one passed to qunique_arg. To preserve the early exit in the
case of descending input, pass the direction as an argument to
the comparator. This requires giving up duplicate detection, which
previously allowed skipping the qunique_arg() call. Testing showed
no regressions this way.
In passing, get rid of nearby checks that the input has at least
two elements, since preserving them would make some macros less
readable. These are not necessary for correctness, and seem like
premature optimizations.
Author: Andrey M. Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/098A3E67-E4A6-4086-9C66-B1EAEB1DFE1C@yandex-team.ru
M contrib/intarray/_int.h
M contrib/intarray/_int_tool.c
Doc: Improve pg_replication_slots.inactive_since description.
commit : 164bac92f08ccddd6701d44a5338d72c22f7b5c2
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 18 Feb 2025 09:23:43 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 18 Feb 2025 09:23:43 +0530
Author: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CAHut+PssvVMTWVtUPto6HbPO8pgVsvtzndt_FdBomA_Oq4zf3w@mail.gmail.com
M doc/src/sgml/system-views.sgml
Fix typo in 2a8a0067.
commit : 2509b857cc1067be628291864289f27325760477
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 18 Feb 2025 14:44:59 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 18 Feb 2025 14:44:59 +1300
Builds configured with Valgrind but without assertions would fail due to
a typo in the recent change. This should be included when back-patching
2a8a0067 into v17.
M src/backend/storage/aio/read_stream.c
Fix translator notes in comments
commit : 9cdc21b533d43829eb1337c8b16e147f6de7c53e
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 17 Feb 2025 20:23:34 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 17 Feb 2025 20:23:34 +0100
The translator comments detailing what a %s inclusion refers to were
accidentally including too many address types. In practice this is
not a problem since it's not a translated string, but to minimize any
risk of confusion let's fix them anwyays. Even though this exists in
backbranches there is little use for backpatch as the translation work
has already happened there, so let's avoid the churn.
Author: Japin Li <japinli@hotmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/ME0P300MB04458DE627480614ABE639D2B6FB2@ME0P300MB0445.AUSP300.PROD.OUTLOOK.COM
M src/backend/libpq/pqcomm.c
Add tab completion for ALTER USER/ROLE RESET
commit : c407d5426b877b41be87f9e679e321bb2c42e47d
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 17 Feb 2025 17:55:23 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 17 Feb 2025 17:55:23 +0100
Currently tab completion for ALTER USER RESET shows a list of all
configuration parameters that may be set on a role, irrespectively of
which parameters are actually set. This patch improves tab completion to
offer only parameters that are set.
Author: Robins Tharakan
Reviewed-By: Tomas Vondra
Discussion: https://postgr.es/m/CAEP4nAzqiT6VbVC5r3nq5byLTnPzjniVGzEMpYcnAHQyNzEuaw%40mail.gmail.com
M src/bin/psql/tab-complete.in.c
Add tab completion for ALTER DATABASE RESET
commit : 9df8727c5067916580f82da8b249136391de8544
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 17 Feb 2025 17:49:53 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 17 Feb 2025 17:49:53 +0100
Currently tab completion for ALTER DATABASE RESET shows a list of all
configuration parameters that may be set on a database, irrespectively
of which parameters are actually set. This patch improves tab completion
to offer only parameters that are set.
Author: Robins Tharakan
Reviewed-By: Tomas Vondra
Discussion: https://postgr.es/m/CAEP4nAzqiT6VbVC5r3nq5byLTnPzjniVGzEMpYcnAHQyNzEuaw%40mail.gmail.com
M src/bin/psql/tab-complete.in.c
Implement Self-Join Elimination
commit : fc069a3a6319b5bf40d2f0f1efceae1c9b7a68a8
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 13 Feb 2025 00:56:03 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 13 Feb 2025 00:56:03 +0200
The Self-Join Elimination (SJE) feature removes an inner join of a plain
table to itself in the query tree if it is proven that the join can be
replaced with a scan without impacting the query result. Self-join and
inner relation get replaced with the outer in query, equivalence classes,
and planner info structures. Also, the inner restrictlist moves to the
outer one with the removal of duplicated clauses. Thus, this optimization
reduces the length of the range table list (this especially makes sense for
partitioned relations), reduces the number of restriction clauses and,
in turn, selectivity estimations, and potentially improves total planner
prediction for the query.
This feature is dedicated to avoiding redundancy, which can appear after
pull-up transformations or the creation of an EquivalenceClass-derived clause
like the below.
SELECT * FROM t1 WHERE x IN (SELECT t3.x FROM t1 t3);
SELECT * FROM t1 WHERE EXISTS (SELECT t3.x FROM t1 t3 WHERE t3.x = t1.x);
SELECT * FROM t1,t2, t1 t3 WHERE t1.x = t2.x AND t2.x = t3.x;
In the future, we could also reduce redundancy caused by subquery pull-up
after unnecessary outer join removal in cases like the one below.
SELECT * FROM t1 WHERE x IN
(SELECT t3.x FROM t1 t3 LEFT JOIN t2 ON t2.x = t1.x);
Also, it can drastically help to join partitioned tables, removing entries
even before their expansion.
The SJE proof is based on innerrel_is_unique() machinery.
We can remove a self-join when for each outer row:
1. At most, one inner row matches the join clause;
2. Each matched inner row must be (physically) the same as the outer one;
3. Inner and outer rows have the same row mark.
In this patch, we use the next approach to identify a self-join:
1. Collect all merge-joinable join quals which look like a.x = b.x;
2. Add to the list above the baseretrictinfo of the inner table;
3. Check innerrel_is_unique() for the qual list. If it returns false, skip
this pair of joining tables;
4. Check uniqueness, proved by the baserestrictinfo clauses. To prove the
possibility of self-join elimination, the inner and outer clauses must
match exactly.
The relation replacement procedure is not trivial and is partly combined
with the one used to remove useless left joins. Tests covering this feature
were added to join.sql. Some of the existing regression tests changed due
to self-join removal logic.
Discussion: https://postgr.es/m/flat/64486b0b-0404-e39e-322d-0801154901f3%40postgrespro.ru
Author: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
Author: Alexander Kuzmenkov <a.kuzmenkov@postgrespro.ru>
Co-authored-by: Alexander Korotkov <aekorotkov@gmail.com>
Co-authored-by: Alena Rybakina <lena.ribackina@yandex.ru>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Simon Riggs <simon@2ndquadrant.com>
Reviewed-by: Jonathan S. Katz <jkatz@postgresql.org>
Reviewed-by: David Rowley <david.rowley@2ndquadrant.com>
Reviewed-by: Thomas Munro <thomas.munro@enterprisedb.com>
Reviewed-by: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Hywel Carver <hywel@skillerwhale.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Zhihong Yu <zyu@yugabyte.com>
Reviewed-by: Greg Stark <stark@mit.edu>
Reviewed-by: Jaime Casanova <jcasanov@systemguards.com.ec>
Reviewed-by: Michał Kłeczek <michal@kleczek.org>
Reviewed-by: Alena Rybakina <lena.ribackina@yandex.ru>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
M doc/src/sgml/config.sgml
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/optimizer/plan/planmain.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/rewrite/rewriteManip.c
M src/backend/utils/misc/guc_tables.c
M src/include/nodes/pathnodes.h
M src/include/optimizer/optimizer.h
M src/include/optimizer/paths.h
M src/include/optimizer/planmain.h
M src/include/rewrite/rewriteManip.h
M src/test/regress/expected/equivclass.out
M src/test/regress/expected/join.out
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/equivclass.sql
M src/test/regress/sql/join.sql
M src/tools/pgindent/typedefs.list
Revert: Get rid of WALBufMappingLock
commit : 3fb58625d18fd226cb929c9700d0db72ac92c075
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 17 Feb 2025 12:35:28 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 17 Feb 2025 12:35:28 +0200
This commit reverts 6a2275b895. Buildfarm failure on batta spots some
concurrency issue, which requires further investigation.
M src/backend/access/transam/xlog.c
M src/backend/utils/activity/wait_event_names.txt
M src/include/storage/lwlocklist.h
Fix an oversight in cbc127917 to handle MERGE correctly
commit : 75dfde13639a57513e23e58c961a69c5bbf017b6
author : Amit Langote <amitlan@postgresql.org>
date : Mon, 17 Feb 2025 16:12:03 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Mon, 17 Feb 2025 16:12:03 +0900
ExecInitModifyTable() forgot to trim MERGE-related lists to exclude
entries for result relations pruned during initial pruning, so fix
that.
While at it, make the function's use of the pruned resultRelations
list, rather than ModifyTable.resultRelations, more consistent.
Reported-by: Alexander Lakhin <exclusion@gmail.com> (via sqlsmith)
Reviewed-by: Junwang Zhao <zhjwpku@gmail.com>
Discussion: https://postgr.es/m/e72c94d9-e5f9-4753-9bc1-69d72bd54b8a@gmail.com
M src/backend/executor/nodeModifyTable.c
M src/include/nodes/execnodes.h
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Add information about WAL buffers full to VACUUM/ANALYZE (VERBOSE)
commit : 6a8a7ce476bd5f042999a04a0d767b38ea6faec0
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Feb 2025 15:09:51 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Feb 2025 15:09:51 +0900
This commit adds the information about the number of times WAL buffers
have been full to the logs generated by VACUUM/ANALYZE (VERBOSE) and in
the logs generated by autovacuum, complementing the existing information
stored by WalUsage.
This is the last part of the backend code where the value of
wal_buffers_full can be reported, similarly to all the other fields of
WalUsage. 320545bfcfee and ce5bcc4a9f26 have done the same for EXPLAIN
and pgss.
Author: Bertrand Drouvot
Reviewed-by: Ilia Evdokimov
Discussion: https://postgr.es/m/Z6SOha5YFFgvpwQY@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/analyze.c
Add information about WAL buffers being full to EXPLAIN (WAL)
commit : 320545bfcfee9c6826d9ae96a880ec82fc03d860
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Feb 2025 14:50:33 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Feb 2025 14:50:33 +0900
This is similar to ce5bcc4a9f26, relying on the addition of
wal_buffers_full to WalUsage. This time, the information is added to
the output generated by EXPLAIN (WAL).
Author: Bertrand Drouvot
Reviewed-by: Ilia Evdokimov
Discussion: https://postgr.es/m/Z6SOha5YFFgvpwQY@ip-10-97-1-34.eu-west-3.compute.internal
M doc/src/sgml/ref/explain.sgml
M src/backend/commands/explain.c
pg_stat_statements: Add wal_buffers_full
commit : ce5bcc4a9f26484746f82d23584c8e342cba9b10
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Feb 2025 13:55:17 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Feb 2025 13:55:17 +0900
wal_buffers_full tracks the number of times WAL buffers become full,
giving hints to be able to tune the GUC wal_buffers.
Up to now, this information was only available in pg_stat_wal. With
this field available in WalUsage since eaf502747bac, exposing it in
pg_stat_statements is straight-forward, and it offers more granularity
at query level.
pg_stat_statements does not need a version bump as one has been done in
commit cf54a2c00254 for this development cycle.
Author: Bertrand Drouvot
Reviewed-by: Ilia Evdokimov
Discussion: https://postgr.es/m/Z6SOha5YFFgvpwQY@ip-10-97-1-34.eu-west-3.compute.internal
M contrib/pg_stat_statements/expected/oldextversions.out
M contrib/pg_stat_statements/pg_stat_statements–1.11–1.12.sql
M contrib/pg_stat_statements/pg_stat_statements.c
M doc/src/sgml/pgstatstatements.sgml
Move wal_buffers_full from PgStat_PendingWalStats to WalUsage
commit : eaf502747bacee0122668eb1ba3979f86b8d8342
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Feb 2025 13:14:28 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Feb 2025 13:14:28 +0900
wal_buffers_full has been introduced in pg_stat_wal in 8d9a935965f, as
some information providing metrics for the tuning of the GUC
wal_buffers. WalUsage has been introduced before that in df3b181499.
Moving this field is proving to be beneficial for several reasons:
- This information can now be made available in more layers, providing
more granularity than just pg_stat_wal, on a per-query basis: EXPLAIN,
pgss and VACUUM/ANALYZE logs.
- A patch is under discussion to provide statistics for WAL at backend
level, and this move simplifies a bit the handling of pending
statistics. The remaining data in PgStat_PendingWalStats now relates to
write/sync counters and times, with equivalents present in pg_stat_io,
that backend statistics are able to already track. So this should cut
all the dependencies between PgStat_PendingWalStats and WAL stats at
backend level.
As of this change, wal_buffers_full only shows in pg_stat_wal.
Author: Bertrand Drouvot
Reviewed-by: Ilia Evdokimov
Discussion: https://postgr.es/m/Z6SOha5YFFgvpwQY@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/access/transam/xlog.c
M src/backend/executor/instrument.c
M src/backend/utils/activity/pgstat_wal.c
M src/include/executor/instrument.h
M src/include/pgstat.h
Get rid of WALBufMappingLock
commit : 6a2275b8953a4462d44daf001bdd60b3d48f0946
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 17 Feb 2025 04:19:01 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 17 Feb 2025 04:19:01 +0200
Allow multiple backends to initialize WAL buffers concurrently. This way
`MemSet((char *) NewPage, 0, XLOG_BLCKSZ);` can run in parallel without
taking a single LWLock in exclusive mode.
The new algorithm works as follows:
* reserve a page for initialization using XLogCtl->InitializeReserved,
* ensure the page is written out,
* once the page is initialized, try to advance XLogCtl->InitializedUpTo and
signal to waiters using XLogCtl->InitializedUpToCondVar condition
variable,
* repeat previous steps until we reserve initialization up to the target
WAL position,
* wait until concurrent initialization finishes using a
XLogCtl->InitializedUpToCondVar.
Now, multiple backends can, in parallel, concurrently reserve pages,
initialize them, and advance XLogCtl->InitializedUpTo to point to the latest
initialized page.
Author: Yura Sokolov <y.sokolov@postgrespro.ru>
Co-authored-by: Alexander Korotkov <aekorotkov@gmail.com>
Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com>
M src/backend/access/transam/xlog.c
M src/backend/utils/activity/wait_event_names.txt
M src/include/storage/lwlocklist.h
Adjust tuples estimate for appendrels
commit : fbc0fe9a2efeb6172f93ad74a9656fdac5de2507
author : Richard Guo <rguo@postgresql.org>
date : Mon, 17 Feb 2025 11:13:15 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Mon, 17 Feb 2025 11:13:15 +0900
In set_append_rel_size(), we currently set rel->tuples to rel->rows
for an appendrel. Generally, rel->tuples is the raw number of tuples
in the relation and rel->rows is the estimated number of tuples after
the relation's restriction clauses have been applied. Although an
appendrel itself doesn't directly enforce any quals today, its child
relations may. Therefore, setting rel->tuples equal to rel->rows for
an appendrel isn't always appropriate.
Doing so can lead to issues in cost estimates in some cases. For
instance, when estimating the number of distinct values from an
appendrel, we would not be able to adjust the estimate based on the
restriction selectivity.
This patch addresses this by setting an appendrel's tuples to the
total number of tuples accumulated from each live child, which better
aligns with reality.
This is arguably a bug, but nobody has complained about that until
now, so no back-patch.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Alena Rybakina <a.rybakina@postgrespro.ru>
Discussion: https://postgr.es/m/CAMbWs4_TG_+kVn6fjG-5GYzzukrNK57=g9eUo4gsrUG26OFawg@mail.gmail.com
M src/backend/optimizer/path/allpaths.c
M src/test/regress/expected/inherit.out
M src/test/regress/sql/inherit.sql
In fmtIdEnc(), handle failure of enlargePQExpBuffer().
commit : a7f95859ef7763ffb7f87894553ab25f5268f33c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 16 Feb 2025 12:46:35 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 16 Feb 2025 12:46:35 -0500
Coverity complained that we weren't doing that, and it's right.
This fix just makes fmtIdEnc() honor the general convention that OOM
causes a PQExpBuffer to become marked "broken", without any immediate
error. In the pretty-unlikely case that we actually did hit OOM here,
the end result would be to return an empty string to the caller,
probably resulting in invalid SQL syntax in an issued command (if
nothing else went wrong, which is even more unlikely). It's tempting
to throw an "out of memory" error if the buffer becomes broken, but
there's not a lot of point in doing that only here and not in hundreds
of other PQExpBuffer-using places in pg_dump and similar callers.
The whole issue could do with some non-time-crunched redesign, perhaps.
This is a followup to the fixes for CVE-2025-1094, and should be
included if cherry-picking those fixes.
M src/fe_utils/string_utils.c
Make escaping functions retain trailing bytes of an invalid character.
commit : 9f45e6a91d8460ac0b1f30e6ae3eefb185b8d0ab
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 15 Feb 2025 16:20:21 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 15 Feb 2025 16:20:21 -0500
Instead of dropping the trailing byte(s) of an invalid or incomplete
multibyte character, replace only the first byte with a known-invalid
sequence, and process the rest normally. This seems less likely to
confuse incautious callers than the behavior adopted in 5dc1e42b4.
While we're at it, adjust PQescapeStringInternal to produce at most
one bleat about invalid multibyte characters per string. This
matches the behavior of PQescapeInternal, and avoids the risk of
producing tons of repetitive junk if a long string is simply given
in the wrong encoding.
This is a followup to the fixes for CVE-2025-1094, and should be
included if cherry-picking those fixes.
Author: Andres Freund <andres@anarazel.de>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://postgr.es/m/20250215012712.45@rfd.leadboat.com
Backpatch-through: 13
M src/fe_utils/string_utils.c
M src/interfaces/libpq/fe-exec.c
Fix explicit valgrind interaction in read_stream.c.
commit : 2a8a00674e973dec4d7d74d5e16c7d7cdedf2be5
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 15 Feb 2025 13:13:19 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 15 Feb 2025 13:13:19 +1300
By calling wipe_mem() on per-buffer data memory that has been released,
we are also telling Valgrind that the memory is "noaccess". We need to
set it to "undefined" before giving it to the registered callback to
fill in, when a slot is reused.
As discovered by build farm animal skink when the VACUUM streamification
patches landed (the first users of per-buffer data).
Pushing to master only for now, to clear the error on skink. It's also
possible that external code might discover the per-buffer data feature
in v17, and reasonable to expect Valgrind not to produce spurious
memcheck reports, but the back-patch is deferred until after the
imminent minor release is out of the way.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Tested-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKG%2Bg6aXpi2FEHqeLOzE%2BxYw%3DOV%2B-N5jhOEnnV%2BF0USM9xA%40mail.gmail.com
M src/backend/storage/aio/read_stream.c
Fix PQescapeLiteral()/PQescapeIdentifier() length handling
commit : efdadeb2238f5b5f0f32df2f3e7d277d1faf1c4f
author : Andres Freund <andres@anarazel.de>
date : Fri, 14 Feb 2025 17:44:28 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 14 Feb 2025 17:44:28 -0500
In 5dc1e42b4fa I fixed bugs in various escape functions, unfortunately as part
of that I introduced a new bug in PQescapeLiteral()/PQescapeIdentifier(). The
bug is that I made PQescapeInternal() just use strlen(), rather than taking
the specified input length into account.
That's bad, because it can lead to including input that wasn't intended to be
included (in case len is shorter than null termination of the string) and
because it can lead to reading invalid memory if the input string is not null
terminated.
Expand test_escape to this kind of bug:
a) for escape functions with length support, append data that should not be
escaped and check that it is not
b) add valgrind requests to detect access of bytes that should not be touched
Author: Tom Lane <tgl@sss.pgh.pa.us>
Author: Andres Freund <andres@anarazel.de
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/Z64jD3u46gObCo1p@pryzbyj2023
Backpatch: 13
M src/interfaces/libpq/fe-exec.c
M src/test/modules/test_escape/test_escape.c
Add delay time to VACUUM/ANALYZE (VERBOSE) and autovacuum logs.
commit : 7720082ae53280857f1eacb86134a9a759290f40
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 14 Feb 2025 14:53:28 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 14 Feb 2025 14:53:28 -0600
Commit bb8dff9995 added this information to the
pg_stat_progress_vacuum and pg_stat_progress_analyze system views.
This commit adds the same information to the output of VACUUM and
ANALYZE with the VERBOSE option and to the autovacuum logs.
Suggested-by: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/ZmaXmWDL829fzAVX%40ip-10-97-1-34.eu-west-3.compute.internal
M doc/src/sgml/config.sgml
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/analyze.c
pgcrypto: Add support for CFB mode in AES encryption
commit : 9ad1b3d01f369f1b204324857e58d9283ff0a527
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 14 Feb 2025 21:18:37 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 14 Feb 2025 21:18:37 +0100
Cipher Feedback Mode, CFB, is a self-synchronizing stream cipher which
is very similar to CBC performed in reverse. Since OpenSSL supports it,
we can easily plug it into the existing cipher selection code without
any need for infrastructure changes.
This patch was simultaneously submitted by Umar Hayat and Vladyslav
Nebozhyn, the latter whom suggested the feauture. The committed patch
is Umar's version.
Author: Umar Hayat <postgresql.wizard@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/CAPBGcbxo9ASzq14VTpQp3mnUJ5omdgTWUJOvWV0L6nNigWE5jw@mail.gmail.com
M contrib/pgcrypto/expected/rijndael.out
M contrib/pgcrypto/openssl.c
M contrib/pgcrypto/sql/rijndael.sql
M doc/src/sgml/pgcrypto.sgml
Use PqMsg_Progress macro in HandleParallelMessage().
commit : 760bf588de91daa5426e3d7aaf77de5abde063b4
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 14 Feb 2025 12:57:13 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 14 Feb 2025 12:57:13 -0600
Commit a99cc6c6b4 introduced the PqMsg_Progress macro but missed
updating HandleParallelMessage() accordingly.
Backpatch-through: 17
M src/backend/access/transam/parallel.c
Use streaming read I/O in VACUUM's third phase
commit : c3e775e608f2a6d0bcfba147bf08a506827cc567
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 14 Feb 2025 12:57:03 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 14 Feb 2025 12:57:03 -0500
Make vacuum's third phase (its second pass over the heap), which reaps
dead items collected in the first phase and marks them as reusable, use
the read stream API. This commit adds a new read stream callback,
vacuum_reap_lp_read_stream_next(), that looks ahead in the TidStore and
returns the next block number to read for vacuum.
Author: Melanie Plageman <melanieplageman@gmail.com>
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGKN3oy0bN_3yv8hd78a4%2BM1tJC9z7mD8%2Bf%2ByA%2BGeoFUwQ%40mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Use streaming read I/O in VACUUM's first phase
commit : 9256822608f3b0636cfd88c3436cdceda7003cf3
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 14 Feb 2025 12:56:57 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 14 Feb 2025 12:56:57 -0500
Make vacuum's first phase, which prunes and freezes tuples and records
dead TIDs, use the read stream API by by converting
heap_vac_scan_next_block() to a read stream callback.
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_aLwANZpxHc0tC-6OT0OQT4TftDGkKAO5yigMUOv_Tcsw%40mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Convert heap_vac_scan_next_block() boolean parameters to flags
commit : 32acad7d1d0a83a1815e15fe28b421d3471ef774
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 14 Feb 2025 12:56:50 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 14 Feb 2025 12:56:50 -0500
The read stream API only allows one piece of extra per block state to be
passed back to the API user (per_buffer_data). lazy_scan_heap() needs
two pieces of per-buffer data: whether or not the block was all-visible
in the visibility map and whether or not it was eagerly scanned.
Convert these two pieces of information to flags so that they can be
populated by heap_vac_scan_next_block() and returned to
lazy_scan_heap(). A future commit will turn heap_vac_scan_next_block()
into the read stream callback for heap phase I vacuuming.
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_bmx33jTqATP5GKNFYwAg02a9dDtk4U_ciEjgBHZSVkOQ%40mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Describe special values in GUC descriptions more consistently.
commit : 977d865c36cc6ed4e813b8545b09d06f51534011
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 14 Feb 2025 10:44:30 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 14 Feb 2025 10:44:30 -0600
Many GUCs accept special values like -1 or an empty string to
disable the feature, use a system default, etc. While the
documentation consistently lists these special values, the GUC
descriptions do not. Many such descriptions fail to mention the
special values, and those that do vary in phrasing and placement.
This commit aims to bring some consistency to this area by applying
the following rules:
* Special values should be listed at the end of the long
description.
* Descriptions should use numerals (e.g., "0") instead of words
(e.g., "zero").
* Special value mentions should be concise and direct (e.g., "0
disables the timeout.", "An empty string means use the operating
system setting.").
* Multiple special values should be listed in ascending order.
Of course, there are exceptions, such as
max_pred_locks_per_relation and search_path, whose special values
are too complex to include. And there are cases like
listen_addresses, where the meaning of an empty string is arguably
too obvious to include. In those cases, I've refrained from adding
special value information to the GUC description.
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: "David G. Johnston" <david.g.johnston@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/Z6aIy4aywxUZHAo6%40nathan
M src/backend/utils/misc/guc_tables.c
M src/include/utils/guc_tables.h
Fix assertion on dereferenced object
commit : 67a02341572e0dcf27a290953317a39c7f15cc82
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 14 Feb 2025 11:50:56 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 14 Feb 2025 11:50:56 +0100
Commit 27cc7cd2bc8a accidentally placed the assertion ensuring
that the pointer isn't NULL after it had already been accessed.
Fix by moving the pointer dereferencing to after the assertion.
Backpatch to all supported branches.
Author: Dmitry Koval <d.koval@postgrespro.ru>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/1618848d-cdc7-414b-9c03-08cf4bef4408@postgrespro.ru
Backpatch-through: 13
M src/backend/executor/execMain.c
Remove obsolete comment.
commit : 9e17ac997f921c9de99125a97a0f34ae50f5c498
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 14 Feb 2025 13:16:05 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 14 Feb 2025 13:16:05 +1300
Commit 755a4c10d19d prevented StartReadBuffers() from crossing md.c
segment boundaries in one operation, but a comment about that
possibility remained.
M src/backend/storage/buffer/bufmgr.c
Remove unused parameter from execute_extension_script().
commit : 432c30dc4ee05bda56f8de9f3da8bb958400ade9
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 13 Feb 2025 16:47:42 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 13 Feb 2025 16:47:42 -0600
This function's schemaOid parameter appears to have never been used
for anything.
Author: Yugo Nagata <nagata@sraoss.co.jp>
Reviewed-by: Fabrízio de Royes Mello <fabriziomello@gmail.com>
Discussion: https://postgr.es/m/20250214010218.550ebe4ec1a7c7811a7fa2bb%40sraoss.co.jp
M src/backend/commands/extension.c
Remove unnecessary (char *) casts [xlog]
commit : ed5e5f071033c8bdaabc8d9cd015f89aa3ccfeef
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 Feb 2025 22:03:29 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 Feb 2025 22:03:29 +0100
Remove (char *) casts no longer needed after XLogRegisterData() and
XLogRegisterBufData() argument type change.
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/fd1fcedb-3492-4fc8-9e3e-74b97f2db6c7%40eisentraut.org
M contrib/pg_visibility/pg_visibility.c
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_pageops.c
M src/backend/access/brin/brin_revmap.c
M src/backend/access/gin/ginbtree.c
M src/backend/access/gin/gindatapage.c
M src/backend/access/gin/ginentrypage.c
M src/backend/access/gin/ginfast.c
M src/backend/access/gin/ginutil.c
M src/backend/access/gin/ginvacuum.c
M src/backend/access/gist/gistxlog.c
M src/backend/access/hash/hash.c
M src/backend/access/hash/hashinsert.c
M src/backend/access/hash/hashovfl.c
M src/backend/access/hash/hashpage.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/rewriteheap.c
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/spgist/spgdoinsert.c
M src/backend/access/spgist/spgvacuum.c
M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlog.c
M src/backend/catalog/storage.c
M src/backend/commands/dbcommands.c
M src/backend/commands/sequence.c
M src/backend/commands/tablecmds.c
M src/backend/commands/tablespace.c
M src/backend/replication/logical/message.c
M src/backend/replication/logical/origin.c
M src/backend/storage/ipc/standby.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/relmapper.c
M src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c
XLogRegisterData, XLogRegisterBufData void * argument for binary data
commit : cdaeff9b39d9ea0cdd8a0676dc6aac7d719b4541
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 13 Feb 2025 10:33:14 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 13 Feb 2025 10:33:14 +0100
Change XLogRegisterData() and XLogRegisterBufData() functions to take
void * for binary data instead of char *. This will remove the need
for numerous casts (done in a separate commit for clarity).
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/fd1fcedb-3492-4fc8-9e3e-74b97f2db6c7%40eisentraut.org
M src/backend/access/transam/README
M src/backend/access/transam/xloginsert.c
M src/include/access/xlog_internal.h
M src/include/access/xloginsert.h
Fix MakeTransitionCaptureState() to return a consistent result
commit : 773c51dd39ada5f107a3656377a9611ff89132f1
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 13 Feb 2025 16:30:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 13 Feb 2025 16:30:58 +0900
When an UPDATE trigger referencing a new table and a DELETE trigger
referencing an old table are both present, MakeTransitionCaptureState()
returns an inconsistent result for UPDATE commands in its set of flags
and tuplestores holding the TransitionCaptureState for transition
tables.
As proved by the test added here, this issue causes a crash in v14 and
earlier versions (down to 11, actually, older versions do not support
triggers on partitioned tables) during cross-partition updates on a
partitioned table. v15 and newer versions are safe thanks to
7103ebb7aae8.
This commit fixes the function so that it returns a consistent state
by using portions of the changes made in commit 7103ebb7aae8 for v13 and
v14. v15 and newer versions are slightly tweaked to match with the
older versions, mainly for consistency across branches.
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20250207.150238.968446820828052276.horikyota.ntt@gmail.com
Backpatch-through: 13
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Rename RBTXN_PREPARE to RBTXN_IS_PREPARE for better clarification.
commit : abfb29648f9adcde84656afde1d50bc8b8e9b6e0
author : Masahiko Sawada <msawada@postgresql.org>
date : Wed, 12 Feb 2025 16:55:00 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Wed, 12 Feb 2025 16:55:00 -0800
RBTXN_PREPARE flag and rbtxn_prepared macro could be misinterpreted as
either indicating the transaction type (e.g. a prepared transaction or
a normal transaction) or its currentstate (e.g. skipped or its prepare
message is sent), especially after commit 072ee847ad4 introduced the
RBTXN_SENT_PREPARE flag and the rbtxn_sent_prepare macro.
The RBTXN_PREPARE flag (and its corresponding macro) have been renamed
to RBTXN_IS_PREPARE to explicitly indicate the transaction
type. Therefore, this commit also adds the RBTXN_IS_PREPARE flag to
the transaction that is a prepared transaction and has been skipped,
which previously had only the RBTXN_SKIPPED_PREPARE flag.
Reviewed-by: Amit Kapila, Peter Smith
Discussion: https://postgr.es/m/CAA4eK1KgNmBsG%3D155E7QQ6TX9RoWnM4z5Z20SvsbwxSe_QXYsg%40mail.gmail.com
M src/backend/replication/logical/proto.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/include/replication/reorderbuffer.h
Skip logical decoding of already-aborted transactions.
commit : 072ee847ad4c3fb52b0c24f7dddbe0798bd70c24
author : Masahiko Sawada <msawada@postgresql.org>
date : Wed, 12 Feb 2025 16:31:34 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Wed, 12 Feb 2025 16:31:34 -0800
Previously, transaction aborts were detected concurrently only during
system catalog scans while replaying a transaction in streaming mode.
This commit adds an additional CLOG lookup to check the transaction
status, allowing the logical decoding to skip changes also when it
doesn't touch system catalogs, if the transaction is already
aborted. This optimization enhances logical decoding performance,
especially for large transactions that have already been rolled back,
as it avoids unnecessary disk or network I/O.
To avoid potential slowdowns caused by frequent CLOG lookups for small
transactions (most of which commit), the CLOG lookup is performed only
for large transactions before eviction. The performance benchmark
results showed there is not noticeable performance regression due to
CLOG lookups.
Reviewed-by: Amit Kapila, Peter Smith, Vignesh C, Ajin Cherian
Reviewed-by: Dilip Kumar, Andres Freund
Discussion: https://postgr.es/m/CAD21AoDht9Pz_DFv_R2LqBTBbO4eGrpa9Vojmt5z5sEx3XwD7A@mail.gmail.com
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/expected/stream.out
M contrib/test_decoding/sql/stats.sql
M contrib/test_decoding/sql/stream.sql
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h
Remove unneeded volatile qualifier in fmgr.c.
commit : 9e66a2b7841a5c1f929103c82c73244d3f5f3416
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 12 Feb 2025 15:45:40 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 12 Feb 2025 15:45:40 -0600
Currently, the save_nestlevel variable in fmgr_security_definer()
is marked volatile. While this may have been necessary when it was
used in a PG_CATCH section (as explained in the comment for PG_TRY
in elog.h), it appears to have been unnecessary since commit
82a47982f3, which removed its use in a PG_CATCH section.
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/Z6xbAgXKY2L-3d5Q%40jrouhaud
M src/backend/utils/fmgr/fmgr.c
Clean up impenetrable logic in pg_basebackup/receivelog.c.
commit : fd602f29c19d4f483f54d93abe240c12219d9f51
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 Feb 2025 16:07:23 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 Feb 2025 16:07:23 -0500
Coverity complained about possible double free of HandleCopyStream's
"copybuf". AFAICS it's mistaken, but it is easy to see why it's
confused, because management of that buffer is impossibly confusing.
It's unreasonable that HandleEndOfCopyStream frees the buffer in some
cases but not others, updates the caller's state for that in no case,
and has not a single comment about how complicated that makes things.
Let's put all the responsibility for freeing copybuf in the actual
owner of that variable, HandleCopyStream. This results in one more
PQfreemem call than before, but the logic is far easier to follow,
both for humans and machines.
Since this isn't (quite) actually broken, no back-patch.
M src/bin/pg_basebackup/receivelog.c
Fix minor memory leaks in pg_dump.
commit : fcd77a6873018c06a34761b7be0c0d1fd847fdd5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 Feb 2025 15:46:31 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 Feb 2025 15:46:31 -0500
Coverity reported the two oversights in getPublicationTables.
Valgrind found the one in determineNotNullFlags.
The mistakes in getPublicationTables seem too minor to be worth
back-patching. determineNotNullFlags could be run enough times
to matter, but that code is new in v18. So, no back-patch.
M src/bin/pg_dump/pg_dump.c
ci: Collect core files on NetBSD and OpenBSD
commit : c45963c5d5cc0a3b8739ba158f781452a3fd89c5
author : Andres Freund <andres@anarazel.de>
date : Wed, 12 Feb 2025 09:40:20 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 12 Feb 2025 09:40:20 -0500
Support for NetBSD and OpenBSD operating systems have been added to CI in the
prior commit. Now add support for collect core files and generating backtraces
using for all core files.
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CAN55FZ32ySyYa06k9MFd+VY5vHhUyBpvgmJUZae5PihjzaurVg@mail.gmail.com
M .cirrus.tasks.yml
M src/tools/ci/cores_backtrace.sh
ci: Test NetBSD and OpenBSD
commit : e291573534237c154609f320d774ef293c0cd2ee
author : Andres Freund <andres@anarazel.de>
date : Wed, 12 Feb 2025 09:40:07 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 12 Feb 2025 09:40:07 -0500
NetBSD and OpenBSD Postgres CI images are now generated [1], but aren't yet
utilized for Postgres' CI. This commit adds CI support for them.
For now the tasks will be manually triggered, to save on CI credits.
[1] https://github.com/anarazel/pg-vm-images
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CAN55FZ32ySyYa06k9MFd+VY5vHhUyBpvgmJUZae5PihjzaurVg@mail.gmail.com
M .cirrus.tasks.yml
M .cirrus.yml
M src/tools/ci/README
meson: Fix failure to detect bsd_auth.h presence
commit : b64d83115cba32b3baf7bd985fc628efbc2dbc3b
author : Andres Freund <andres@anarazel.de>
date : Wed, 12 Feb 2025 08:15:53 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 12 Feb 2025 08:15:53 -0500
bsd_auth.h file needs to be included after 'sys/types.h', as documented in
https://man.openbsd.org/authenticate.3
The reason a similar looking stanza works for autoconf is that autoconf
automatically adds AC_INCLUDES_DEFAULT, which in turn includes sys/types.h.
Backpatch to all versions with meson support.
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/637haqqyhg2wlz7q6wq25m2qupe67g7f2uupngzui64zypy4x2@ysr2xnmynmu4
Backpatch-through: 16
M meson.build
Fix issue in recovery test 041_checkpoint_at_promote
commit : 0fc68c8421b656f2ad03e788d97ff0c59da60e6f
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 12 Feb 2025 17:58:25 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 12 Feb 2025 17:58:25 +0900
The phase of the test waiting for a restartpoint to complete was not
working as intended, due to a log_contains() call incorrectly
written.
The problem reported by the author could be simply reproduced by
removing the injection_points_wakeup() call: the test succeeds rather
than waiting for the restartpoint completion. In most cases, the
restartpoint completion is fast enough that the test offered the wanted
coverage. On slow machines, it could have become unreliable.
Oversight in 6782709df81f.
Author: Nitin Jadhav
Discussion: https://postgr.es/m/CAMm1aWa_6u+o52r7h7G6pX-oWD0Qraf0ee17Ma50qxGS0B_Rzg@mail.gmail.com
Backpatch-through: 17
M src/test/recovery/t/041_checkpoint_at_promote.pl
Fix some inconsistencies with memory freeing in pg_createsubscriber
commit : 5b94e2753439b699b78aacbb53e89bf0e814becc
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 12 Feb 2025 17:11:43 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 12 Feb 2025 17:11:43 +0900
The correct function documented to free the memory allocated for the
result returned by PQescapeIdentifier() and PQescapeLiteral() is
PQfreemem(). pg_createsubscriber.c relied on pg_free() instead, which
is not incorrect as both do a free() internally, but inconsistent with
the documentation.
While on it, this commit fixes a small memory leak introduced by
4867f8a555ce, as the code of pg_createsubscriber makes this effort.
Author: Ranier Vilela
Reviewed-by: Euler Taveira
Discussion: https://postgr.es/m/CAEudQAp=AW5dJXrGLbC_aZg_9nOo=42W7uLDRONFQE-gcgnkgQ@mail.gmail.com
Backpatch-through: 17
M src/bin/pg_basebackup/pg_createsubscriber.c
Remove unnecessary (char *) casts [checksum]
commit : 1b5841d4615045e48c3a0019e36f91e4e71d0912
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 12 Feb 2025 08:59:48 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 12 Feb 2025 08:59:48 +0100
Remove some (char *) casts related to uses of the pg_checksum_page()
function. These casts are useless, because everything involved
already has the right type. Moreover, these casts actually silently
discarded a const qualifier. The declaration of a higher-level
function needs to be adjusted to fix that.
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/fd1fcedb-3492-4fc8-9e3e-74b97f2db6c7%40eisentraut.org
M contrib/pageinspect/rawpage.c
M src/backend/storage/page/bufpage.c
M src/include/storage/bufpage.h
Remove unnecessary (char *) casts [mem]
commit : 827b4060a8e35047c1adc9ca2ab3d8e7ad905df0
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 12 Feb 2025 08:50:13 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 12 Feb 2025 08:50:13 +0100
Remove (char *) casts around memory functions such as memcmp(),
memcpy(), or memset() where the cast is useless. Since these
functions don't take char * arguments anyway, these casts are at best
complicated casts to (void *), about which see commit 7f798aca1d5.
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/fd1fcedb-3492-4fc8-9e3e-74b97f2db6c7%40eisentraut.org
M contrib/pg_trgm/trgm_gist.c
M contrib/xml2/xpath.c
M src/backend/access/common/heaptuple.c
M src/backend/access/heap/heapam_xlog.c
M src/backend/access/table/toast_helper.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/backend/bootstrap/bootstrap.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/replication/logical/decode.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/backend/storage/file/fd.c
M src/backend/storage/page/bufpage.c
M src/backend/tcop/postgres.c
M src/backend/utils/activity/backend_status.c
M src/interfaces/ecpg/compatlib/informix.c
M src/interfaces/ecpg/ecpglib/misc.c
M src/interfaces/libpq/fe-lobj.c
M src/interfaces/libpq/fe-secure-gssapi.c
Remove unnecessary (char *) casts [string]
commit : 506183bce73a2b22308a54876f0a56a249bc26e9
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 12 Feb 2025 08:49:18 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 12 Feb 2025 08:49:18 +0100
Remove (char *) casts around string functions where the arguments or
result already have the right type and the cast is useless (or worse,
potentially casts away a qualifier, but this doesn't appear to be the
case here).
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/fd1fcedb-3492-4fc8-9e3e-74b97f2db6c7%40eisentraut.org
M contrib/fuzzystrmatch/dmetaphone.c
M doc/src/sgml/gist.sgml
M src/backend/replication/walreceiver.c
M src/backend/replication/walreceiverfuncs.c
M src/backend/utils/activity/backend_status.c
M src/interfaces/ecpg/pgtypeslib/common.c
Doc: Fix punctuation errors
commit : 0bc34ad69204dded9cd06d03c3ced656eb137dc3
author : John Naylor <john.naylor@postgresql.org>
date : Wed, 12 Feb 2025 13:12:51 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Wed, 12 Feb 2025 13:12:51 +0700
Author: 斉藤登 <noborusai@gmail.com>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/CAAM3qnL6i-BSu5rB2+KiHLjMCOXiQEiPMBvEj7F1CgUzZMooLA@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/config.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/ref/pgbench.sgml
Add cost-based vacuum delay time to progress views.
commit : bb8dff9995f2cf501376772898bcbcf58aa05cde
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 11 Feb 2025 16:38:14 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 11 Feb 2025 16:38:14 -0600
This commit adds the amount of time spent sleeping due to
cost-based delay to the pg_stat_progress_vacuum and
pg_stat_progress_analyze system views. A new configuration
parameter named track_cost_delay_timing, which is off by default,
controls whether this information is gathered. For vacuum, the
reported value includes the sleep time of any associated parallel
workers. However, parallel workers only report their sleep time
once per second to avoid overloading the leader process.
Bumps catversion.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Co-authored-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: Sergei Kornilov <sk@zsrv.org>
Discussion: https://postgr.es/m/ZmaXmWDL829fzAVX%40ip-10-97-1-34.eu-west-3.compute.internal
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/commands/vacuum.c
M src/backend/commands/vacuumparallel.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/catalog/catversion.h
M src/include/commands/progress.h
M src/include/commands/vacuum.h
M src/test/regress/expected/rules.out
Add is_analyze parameter to vacuum_delay_point().
commit : e5b0b0ce150972bf162a059430d84e5f8e07cf30
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 11 Feb 2025 16:38:14 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 11 Feb 2025 16:38:14 -0600
This function is used in both vacuum and analyze code paths, and a
follow-up commit will require distinguishing between the two. This
commit forces callers to specify whether they are in a vacuum or
analyze path, but it does not use that information for anything
yet.
Author: Nathan Bossart <nathandbossart@gmail.com>
Co-authored-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/ZmaXmWDL829fzAVX%40ip-10-97-1-34.eu-west-3.compute.internal
M contrib/bloom/blvacuum.c
M contrib/file_fdw/file_fdw.c
M src/backend/access/gin/ginfast.c
M src/backend/access/gin/ginvacuum.c
M src/backend/access/gist/gistvacuum.c
M src/backend/access/hash/hash.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/spgist/spgvacuum.c
M src/backend/commands/analyze.c
M src/backend/commands/vacuum.c
M src/backend/tsearch/ts_typanalyze.c
M src/backend/utils/adt/array_typanalyze.c
M src/backend/utils/adt/rangetypes_typanalyze.c
M src/include/commands/vacuum.h
Limit pgbench COPY FREEZE to ordinary relations
commit : d0d649e91685081e6c5334d6d01ab4cd735b397d
author : Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 11 Feb 2025 16:51:28 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 11 Feb 2025 16:51:28 -0500
pgbench client-side data generation uses COPY FREEZE to load data for most
tables. COPY FREEZE isn't supported for partitioned tables and since pgbench
only supports partitioning pgbench_accounts, pgbench used a hard-coded check to
skip COPY FREEZE and use plain COPY for a partitioned pgbench_accounts.
If the user has manually partitioned one of the other pgbench tables, this
causes client-side data generation to error out with:
ERROR: cannot perform COPY FREEZE on a partitioned table
Fix this by limiting COPY FREEZE to ordinary tables (RELKIND_RELATION).
Author: Sergey Tatarintsev <s.tatarintsev@postgrespro.ru>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/flat/97f55fca-8a7b-4da8-b413-7d1c57010676%40postgrespro.ru
M doc/src/sgml/ref/pgbench.sgml
M src/bin/pgbench/pgbench.c
Injection points for hash aggregation.
commit : 38172d1856b34792a5ee60eaa0d883166e90d33d
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 11 Feb 2025 11:26:25 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 11 Feb 2025 11:26:25 -0800
Requires adding a guard against shift-by-32. Previously, that was
impossible because the number of partitions was always greater than 1,
but a new injection point can force the number of partitions to 1.
Discussion: https://postgr.es/m/ff4e59305e5d689e03cd256a736348d3e7958f8f.camel@j-davis.com
M src/backend/executor/nodeAgg.c
M src/test/modules/injection_points/Makefile
A src/test/modules/injection_points/expected/hashagg.out
M src/test/modules/injection_points/meson.build
A src/test/modules/injection_points/sql/hashagg.sql
Eagerly scan all-visible pages to amortize aggressive vacuum
commit : 052026c9b903380b428a4c9ba2ec90726db81288
author : Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 11 Feb 2025 13:52:19 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 11 Feb 2025 13:52:19 -0500
Aggressive vacuums must scan every unfrozen tuple in order to advance
the relfrozenxid/relminmxid. Because data is often vacuumed before it is
old enough to require freezing, relations may build up a large backlog
of pages that are set all-visible but not all-frozen in the visibility
map. When an aggressive vacuum is triggered, all of these pages must be
scanned. These pages have often been evicted from shared buffers and
even from the kernel buffer cache. Thus, aggressive vacuums often incur
large amounts of extra I/O at the expense of foreground workloads.
To amortize the cost of aggressive vacuums, eagerly scan some
all-visible but not all-frozen pages during normal vacuums.
All-visible pages that are eagerly scanned and set all-frozen in the
visibility map are counted as successful eager freezes and those not
frozen are counted as failed eager freezes.
If too many eager scans fail in a row, eager scanning is temporarily
suspended until a later portion of the relation. The number of failures
tolerated is configurable globally and per table.
To effectively amortize aggressive vacuums, we cap the number of
successes as well. Capping eager freeze successes also limits the amount
of potentially wasted work if these pages are modified again before the
next aggressive vacuum. Once we reach the maximum number of blocks
successfully eager frozen, eager scanning is disabled for the remainder
of the vacuum of the relation.
Original design idea from Robert Haas, with enhancements from
Andres Freund, Tomas Vondra, and me
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Robert Treat <rob@xzilla.net>
Reviewed-by: Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/flat/CAAKRu_ZF_KCzZuOrPrOqjGVe8iRVWEAJSpzMgRQs%3D5-v84cXUg%40mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/ref/create_table.sgml
M src/backend/access/common/reloptions.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/vacuum.c
M src/backend/postmaster/autovacuum.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/psql/tab-complete.in.c
M src/include/commands/vacuum.h
M src/include/utils/rel.h
config: Rename "Asynchronous Behavior" to "I/O"
commit : 4dd09a1d415d143386d5e6dc9519615a8b18f850
author : Andres Freund <andres@anarazel.de>
date : Tue, 11 Feb 2025 11:39:20 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 11 Feb 2025 11:39:20 -0500
"I/O" seems more descriptive than "Asynchronous Behavior", given that some of
the GUCs in the section don't relate to anything asynchronous.
Most other abbreviations in the config sections are un-abbreviated, but
"Input/Output" seems less likely to be helpful than just IO or I/O.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/x3tlw2jk5gm3r3mv47hwrshffyw7halpczkfbk3peksxds7bvc@lguk43z3bsyq
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/utils/guc_tables.h
config: Split "Worker Processes" out of "Asynchronous Behavior"
commit : 740766d37cdf466438c0ccb6c4c8acaa62a7fc90
author : Andres Freund <andres@anarazel.de>
date : Tue, 11 Feb 2025 11:39:19 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 11 Feb 2025 11:39:19 -0500
Having all the worker related GUCs in the same section as IO controlling GUCs
doesn't really make sense. Create a separate section for them.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/x3tlw2jk5gm3r3mv47hwrshffyw7halpczkfbk3peksxds7bvc@lguk43z3bsyq
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/utils/guc_tables.h
Allow extension functions to participate in in-place updates.
commit : c366d2bdba7c3b9b2cca1429d4535866e231ca55
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Feb 2025 12:49:34 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Feb 2025 12:49:34 -0500
Commit 1dc5ebc90 allowed PL/pgSQL to perform in-place updates
of expanded-object variables that are being updated with
assignments like "x := f(x, ...)". However this was allowed
only for a hard-wired list of functions f(), since we need to
be sure that f() will not modify the variable if it fails.
It was always envisioned that we should make that extensible,
but at the time we didn't have a good way to do so. Since
then we've invented the idea of "support functions" to allow
attaching specialized optimization knowledge to functions,
and that is a perfect mechanism for doing this.
Hence, adjust PL/pgSQL to use a support function request instead
of hard-wired logic to decide if in-place update is safe.
Preserve the previous optimizations by creating support functions
for the three functions that were previously hard-wired.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com>
Discussion: https://postgr.es/m/CACxu=vJaKFNsYxooSnW1wEgsAO5u_v1XYBacfVJ14wgJV_PYeg@mail.gmail.com
M src/backend/utils/adt/array_userfuncs.c
M src/backend/utils/adt/arraysubs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/nodes/supportnodes.h
M src/pl/plpgsql/src/expected/plpgsql_array.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_array.sql
M src/tools/pgindent/typedefs.list
Implement new optimization rule for updates of expanded variables.
commit : 6c7251db0ce12a83dd03c840e7d0ff7516b27229
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Feb 2025 12:34:59 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Feb 2025 12:34:59 -0500
If a read/write expanded variable is declared locally to the
assignment statement that is updating it, and it is referenced
exactly once in the assignment RHS, then we can optimize the
operation as a direct update of the expanded value, whether
or not the function(s) operating on it can be trusted not to
modify the value before throwing an error. This works because
if an error does get thrown, we no longer care what value the
variable has.
In cases where that doesn't work, fall back to the previous
rule that checks for safety of the top-level function.
In any case, postpone determination of whether these optimizations
are feasible until we are executing a Param referencing the target
variable and that variable holds a R/W expanded object. While the
previous incarnation of exec_check_rw_parameter was pretty cheap,
this is a bit less so, and our plan to invoke support functions
will make it even less so. So avoiding the check for variables
where it couldn't be useful should be a win.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com>
Discussion: https://postgr.es/m/CACxu=vJaKFNsYxooSnW1wEgsAO5u_v1XYBacfVJ14wgJV_PYeg@mail.gmail.com
M src/include/executor/execExpr.h
M src/pl/plpgsql/src/expected/plpgsql_array.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/plpgsql.h
M src/pl/plpgsql/src/sql/plpgsql_array.sql
M src/tools/pgindent/typedefs.list
Detect whether plpgsql assignment targets are "local" variables.
commit : 36fb9ef269a092a1203c9482bdf9b3e963728fd9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Feb 2025 12:27:15 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Feb 2025 12:27:15 -0500
Mark whether the target of a potentially optimizable assignment
is "local", in the sense of being declared inside any exception
block that could trap an error thrown from the assignment.
(This implies that we needn't preserve the variable's value
in case of an error. This patch doesn't do anything with the
knowledge, but the next one will.)
Normally, this requires a post-parsing scan of the function's
parse tree, since we don't know while parsing a BEGIN ...
construct whether we will find EXCEPTION at its end. However,
if there are no BEGIN ... EXCEPTION blocks in the function at
all, then all assignments are local, even those to variables
representing function arguments. We optimize that common case
by initializing the target_is_local flags to "true", and fixing
them up with a post-scan only if we found EXCEPTION.
Note that variables' default-value expressions are never interesting
for expanded-variable optimization, since they couldn't contain a
reference to the target variable anyway. But the code is set up
to compute their target_param and target_is_local correctly anyway,
for consistency and in case someone thinks of a use for that data.
I added a bit of plpgsql_dumptree support to help verify that this
code sets the flags as expected. I also added a plpgsql_dumptree
call in plpgsql_compile_inline. It was at best an oversight that
"#option dump" didn't work in a DO block; now it does.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com>
Discussion: https://postgr.es/m/CACxu=vJaKFNsYxooSnW1wEgsAO5u_v1XYBacfVJ14wgJV_PYeg@mail.gmail.com
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_funcs.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
Preliminary refactoring of plpgsql expression construction.
commit : a654af21ae522cc8e867e052defd16f76ffaef2e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Feb 2025 12:20:05 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Feb 2025 12:20:05 -0500
This short and boring patch simply moves the responsibility for
initializing PLpgSQL_expr.target_param into plpgsql parsing,
rather than doing it at first execution of the expr as before.
This doesn't save anything in terms of runtime, since the work was
trivial and done only once per expr anyway. But it makes the info
available during parsing, which will be useful for the next step.
Likewise set PLpgSQL_expr.func during parsing. According to the
comments, this was once impossible; but it's certainly possible
since we invented the plpgsql_curr_compile variable. Again, this
saves little runtime, but it seems far cleaner conceptually.
While at it, I reordered stuff in struct PLpgSQL_expr to make it
clearer which fields are filled when, and merged some duplicative
code in pl_gram.y.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com>
Discussion: https://postgr.es/m/CACxu=vJaKFNsYxooSnW1wEgsAO5u_v1XYBacfVJ14wgJV_PYeg@mail.gmail.com
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
Refactor pl_funcs.c to provide a usage-independent tree walker.
commit : 6a7283dd2f1c573f2cf3daaf14e8a246577da57d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Feb 2025 12:14:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 Feb 2025 12:14:12 -0500
We haven't done this up to now because there was only one use-case,
namely plpgsql_free_function_memory's search for expressions to clean
up. However an upcoming patch has another need for walking plpgsql
functions' statement trees, so let's create sharable tree-walker
infrastructure in the same style as expression_tree_walker().
This patch actually makes the code shorter, although that's
mainly down to having used a more compact coding style. (I didn't
write a separate subroutine for each statement type, and I made
use of some newer notations like foreach_ptr.)
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com>
Discussion: https://postgr.es/m/CACxu=vJaKFNsYxooSnW1wEgsAO5u_v1XYBacfVJ14wgJV_PYeg@mail.gmail.com
M src/pl/plpgsql/src/pl_funcs.c
Replace AssertMacro() with Assert() when not in macro
commit : 6998db59c2959c4f280a9088054e6dbf7178efe0
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 Feb 2025 11:12:05 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 Feb 2025 11:12:05 +0100
This was forgotten to be changed in commit 9c727360bcc.
M src/include/storage/bufmgr.h
Fix indentation of comment in plannodes.h
commit : c9238ad853258c3f72d8671df5690f526c973f68
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 11 Feb 2025 07:40:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 11 Feb 2025 07:40:03 +0900
Oversight in commit 3d17d7d7fb7a. Worth noting that pgindent was fine
as-is.
Author: Sami Imseih
Discussion: https://postgr.es/m/CAA5RZ0t80hP2aTv97QtEJy39GkxKmDBVDiTBApfiuTa4O=TEWQ@mail.gmail.com
M src/include/nodes/plannodes.h
Adapt appendPsqlMetaConnect() to the new fmtId() encoding expectations.
commit : 5bf12323b6b8b05790aab6876555568898f4fc81
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Feb 2025 16:30:03 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Feb 2025 16:30:03 -0500
We need to tell fmtId() what encoding to assume, but this function
doesn't know that. Fortunately we can fix that without changing the
function's API, because we can just use SQL_ASCII. That's because
database names in connection requests are effectively binary not text:
no encoding-aware processing will happen on them.
This fixes XversionUpgrade failures seen in the buildfarm. The
alternative of having pg_upgrade use setFmtEncoding() is unappetizing,
given that it's connecting to multiple databases that may have
different encodings.
Andres Freund, Noah Misch, Tom Lane
Security: CVE-2025-1094
M src/fe_utils/string_utils.c
Lock table in ShareUpdateExclusive when importing index stats.
commit : 9f12da78d9538016efccbf17e6f816f1c286b7f5
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 10 Feb 2025 12:25:24 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 10 Feb 2025 12:25:24 -0800
Follow locking behavior of ANALYZE when importing statistics. In
particular, when importing index statistics, the table must be locked
in ShareUpdateExclusive mode. Fixes bug reportd by Jian He.
ANALYZE doesn't update statistics on partitioned indexes, and the
locking requirements are slightly different for in-place updates on
partitioned indexes versus normal indexes. To be conservative, lock
both the partitioned table and the partitioned index in
ShareUpdateExclusive mode when importing stats for a partitioned
index.
Author: Corey Huinker
Reported-by: Jian He
Reviewed-by: Michael Paquier
Discussion: https://www.postgresql.org/message-id/CACJufxGreTY7qsCV8%2BBkuv0p5SXGTScgh%3DD%2BDq6%3D%2B_%3DXTp7FWg%40mail.gmail.com
M src/backend/statistics/stat_utils.c
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Fix type in test_escape test
commit : 979205e47bdc094787b10142e84a7634f7c720dd
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 12:09:23 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 12:09:23 -0500
On machines where char is unsigned this could lead to option parsing looping
endlessly. It's also too narrow a type on other hardware.
Found via Tom Lane's monitoring of the buildfarm.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Security: CVE-2025-1094
Backpatch-through: 13
M src/test/modules/test_escape/test_escape.c
docs: EUC_TW can be up to four bytes wide, not three
commit : 32c34006b24bf7432c2d0f47da10492de7253d52
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:37 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:37 -0500
Backpatch-through: 13
Security: CVE-2025-1094
M doc/src/sgml/charset.sgml
Add test of various escape functions
commit : ac00ff1c96081ea4981f1001209b4719f5cd82ac
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:37 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:37 -0500
As highlighted by the prior commit, writing correct escape functions is less
trivial than one might hope.
This test module tries to verify that different escaping functions behave
reasonably. It e.g. tests:
- Invalidly encoded input to an escape function leads to invalidly encoded
output
- Trailing incomplete multi-byte characters are handled sensibly
- Escaped strings are parsed as single statement by psql's parser (which
derives from the backend parser)
There are further tests that would be good to add. But even in the current
state it was rather useful for writing the fix in the prior commit.
Reviewed-by: Noah Misch <noah@leadboat.com>
Backpatch-through: 13
Security: CVE-2025-1094
M src/test/modules/Makefile
M src/test/modules/meson.build
A src/test/modules/test_escape/.gitignore
A src/test/modules/test_escape/Makefile
A src/test/modules/test_escape/meson.build
A src/test/modules/test_escape/t/001_test_escape.pl
A src/test/modules/test_escape/test_escape.c
M src/tools/pgindent/typedefs.list
Fix handling of invalidly encoded data in escaping functions
commit : 5dc1e42b4fa6a4434afa7d7cdcf0291351a7b873
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:37 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:37 -0500
Previously invalidly encoded input to various escaping functions could lead to
the escaped string getting incorrectly parsed by psql. To be safe, escaping
functions need to ensure that neither invalid nor incomplete multi-byte
characters can be used to "escape" from being quoted.
Functions which can report errors now return an error in more cases than
before. Functions that cannot report errors now replace invalid input bytes
with a byte sequence that cannot be used to escape the quotes and that is
guaranteed to error out when a query is sent to the server.
The following functions are fixed by this commit:
- PQescapeLiteral()
- PQescapeIdentifier()
- PQescapeString()
- PQescapeStringConn()
- fmtId()
- appendStringLiteral()
Reported-by: Stephen Fewer <stephen_fewer@rapid7.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Backpatch-through: 13
Security: CVE-2025-1094
M src/fe_utils/string_utils.c
M src/interfaces/libpq/fe-exec.c
Specify the encoding of input to fmtId()
commit : 3e98c8ce50e46d58b91bf3ea806e995296dc5b91
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:37 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:37 -0500
This commit adds fmtIdEnc() and fmtQualifiedIdEnc(), which allow to specify
the encoding as an explicit argument. Additionally setFmtEncoding() is
provided, which defines the encoding when no explicit encoding is provided, to
avoid breaking all code using fmtId().
All users of fmtId()/fmtQualifiedId() are either converted to the explicit
version or a call to setFmtEncoding() has been added.
This commit does not yet utilize the now well-defined encoding, that will
happen in a subsequent commit.
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Backpatch-through: 13
Security: CVE-2025-1094
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/psql/command.c
M src/bin/scripts/common.c
M src/bin/scripts/createdb.c
M src/bin/scripts/createuser.c
M src/bin/scripts/dropdb.c
M src/bin/scripts/dropuser.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/vacuumdb.c
M src/fe_utils/string_utils.c
M src/include/fe_utils/string_utils.h
Add pg_encoding_set_invalid()
commit : 4dc28963533704fc7dd922b9447467466a233d89
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:37 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:37 -0500
There are cases where we cannot / do not want to error out for invalidly
encoded input. In such cases it can be useful to replace e.g. an incomplete
multi-byte characters with bytes that will trigger an error when getting
validated as part of a larger string.
Unfortunately, until now, for some encoding no such sequence existed. For
those encodings this commit removes one previously accepted input combination
- we consider that to be ok, as the chosen bytes are outside of the valid
ranges for the encodings, we just previously failed to detect that.
As we cannot add a new field to pg_wchar_table without breaking ABI, this is
implemented "in-line" in the newly added function.
Author: Noah Misch <noah@leadboat.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Backpatch-through: 13
Security: CVE-2025-1094
M src/common/wchar.c
M src/include/mb/pg_wchar.h
M src/test/regress/expected/conversion.out
M src/test/regress/regress.c
M src/test/regress/sql/conversion.sql
Reformat node comments in plannodes.h
commit : 3d17d7d7fb7a4603b48acb275b5a416f110db464
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 10 Feb 2025 09:58:25 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 10 Feb 2025 09:58:25 +0900
This is similar to d575051b9af9 but this time for the comments in
plannodes.h to avoid long lines, which is useful if adding per-field
annotations with pg_node_attr() to these planner structures.
Some patches are under discussion to add such properties to planner
fields, which is something that may or may not happen, and this change
makes future proposals easier to work on and review, which being more
consistent in style with the parse nodes.
Author: Sami Imseih
Discussion: https://postgr.es/m/Z5xTb5iBHVGns35R@paquier.xyz
M src/include/nodes/plannodes.h
Cache NO ACTION foreign keys separately from RESTRICT foreign keys
commit : 9926f854d07784d72aada16c8038b558fbc45d13
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 9 Feb 2025 13:43:56 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 9 Feb 2025 13:43:56 +0100
Now that we generate different SQL for temporal NO ACTION vs RESTRICT
foreign keys, we should cache their query plans with different keys.
Since the key also includes the constraint oid, this shouldn't be
necessary, but we have been seeing build farm failures that suggest we
might be sometimes using a cached NO ACTION plan to implement a RESTRICT
constraint.
Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+renyUApHgSZF9-nd-a0+OPGharLQLO=mDHcY4_qQ0+noCUVg@mail.gmail.com
M src/backend/utils/adt/ri_triggers.c
Make TLS write functions' buffer arguments pointers const
commit : a9258629edabd461f998ec7d06cdf554e8a6a6ec
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 9 Feb 2025 12:10:25 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 9 Feb 2025 12:10:25 +0100
This also makes it match the equivalent APIs in libpq.
Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/fd1fcedb-3492-4fc8-9e3e-74b97f2db6c7%40eisentraut.org
M src/backend/libpq/be-secure-gssapi.c
M src/backend/libpq/be-secure-openssl.c
M src/backend/libpq/be-secure.c
M src/backend/libpq/pqcomm.c
M src/include/libpq/libpq-be.h
M src/include/libpq/libpq.h
Refactor TAP test code for file comparisons into new routine in Utils.pm
commit : 169208092f5c98a6021b23b38f03a5d65f84ad96
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 9 Feb 2025 16:52:33 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 9 Feb 2025 16:52:33 +0900
This unifies the output used should any differences be found in the
files provided, information that 027_stream_regress did not show on
failures. TAP tests of pg_combinebackup and pg_upgrade now rely on the
refactored routine, reducing the dependency to the diff command. The
callers of this routine can optionally specify a custom line-comparison
function.
There are a couple of tests that still use directly a diff command:
001_pg_bsd_indent, 017_shm and test_json_parser's 003. These rely on
different properties and are left out for now.
Extracted from a larger patch by the same author.
Author: Ashutosh Bapat
Discussion: https://postgr.es/m/Z6RQS-tMzGYjlA-H@paquier.xyz
M src/bin/pg_combinebackup/t/002_compare_backups.pl
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
M src/test/perl/PostgreSQL/Test/Utils.pm
M src/test/recovery/t/027_stream_regress.pl
PDF docs build: avoid spurious "warn" in build logs.
commit : ecb8226af63dc8f1c0859977102764704368693b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Feb 2025 22:12:38 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Feb 2025 22:12:38 -0500
Improve on e4c886519 so that the string "warn" appears in
the output when there's a problem, and not when there isn't.
This should silence noise I've been seeing in my buildfarm
warning scraper.
M doc/src/sgml/Makefile
Fix pgbench performance issue induced by commit af35fe501.
commit : fb056564ec5bc1c18dd670c963c893cdb6de927e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Feb 2025 13:41:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Feb 2025 13:41:42 -0500
Commit af35fe501 caused "pgbench -i" to emit a '\r' character
for each data row loaded (when stderr is a terminal).
That's effectively invisible on-screen, but it causes the
connected terminal program to consume a lot of cycles.
It's even worse if you're connected over ssh, as the data
then has to pass through the ssh tunnel.
Simplest fix is to move the added logic inside the if-tests
that check whether to print a progress line. We could do
it another way that avoids duplicating these few lines,
but on the whole this seems the most transparent way to
write it.
Like the previous commit, back-patch to all supported versions.
Reported-by: Andres Freund <andres@anarazel.de>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/4k4drkh7bcmdezq6zbkhp25mnrzpswqi2o75d5uv2eeg3aq6q7@b7kqdmzzwzgb
Backpatch-through: 13
M src/bin/pgbench/pgbench.c
Doc: clarify behavior of timestamptz input some more.
commit : 11bba6e49407cbaa3a9cd317bb3a1f08b8c5609b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Feb 2025 12:40:41 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Feb 2025 12:40:41 -0500
Try to make it absolutely plain that we don't retain the
originally specified time zone, only the UTC timestamp.
While at it, make glossary entries for "UTC" and "GMT".
Author: Robert Treat <rob@xzilla.net>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/173796426022.1064.9135167366862649513@wrigleys.postgresql.org
Backpatch-through: 13
M doc/src/sgml/datatype.sgml
M doc/src/sgml/glossary.sgml
Allow non-btree speculative insertion indexes
commit : b92c03342dcd69d259262b06b5c290e27249cb11
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 Feb 2025 11:17:25 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 Feb 2025 11:17:25 +0100
Previously, only btrees were supported as the arbiter index for
speculative insertion because there was no way to get the equality
strategy number for other index methods. We have this now (commit
c09e5a6a016), so we can support this.
At the moment, only btree supports unique indexes, so this does not
change anything in practice, but it would allow another index method
that has amcanunique to be supported.
Co-authored-by: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/catalog/index.c
Support non-btree indexes for foreign keys
commit : bfe21b760e039163b91279f58fd09a0298572775
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 Feb 2025 11:17:25 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 Feb 2025 11:17:25 +0100
Previously, only btrees were supported as the referenced unique index
for foreign keys because there was no way to get the equality strategy
number for other index methods. We have this now (commit
c09e5a6a016), so we can support this. In fact, this is now just a
special case of the existing generalized "period" foreign key
support, since that already knows how to lookup equality strategy
numbers.
Note that this does not change the requirement that the referenced
index needs to be unique, and at the moment, only btree supports that,
so this does not change anything in practice, but it would allow
another index method that has amcanunique to be supported.
Co-authored-by: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/commands/tablecmds.c
Virtual generated columns
commit : 83ea6c54025bea67bcd4949a6d58d3fc11c3e21b
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 Feb 2025 09:09:34 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 Feb 2025 09:09:34 +0100
This adds a new variant of generated columns that are computed on read
(like a view, unlike the existing stored generated columns, which are
computed on write, like a materialized view).
The syntax for the column definition is
... GENERATED ALWAYS AS (...) VIRTUAL
and VIRTUAL is also optional. VIRTUAL is the default rather than
STORED to match various other SQL products. (The SQL standard makes
no specification about this, but it also doesn't know about VIRTUAL or
STORED.) (Also, virtual views are the default, rather than
materialized views.)
Virtual generated columns are stored in tuples as null values. (A
very early version of this patch had the ambition to not store them at
all. But so much stuff breaks or gets confused if you have tuples
where a column in the middle is completely missing. This is a
compromise, and it still saves space over being forced to use stored
generated columns. If we ever find a way to improve this, a bit of
pg_upgrade cleverness could allow for upgrades to a newer scheme.)
The capabilities and restrictions of virtual generated columns are
mostly the same as for stored generated columns. In some cases, this
patch keeps virtual generated columns more restricted than they might
technically need to be, to keep the two kinds consistent. Some of
that could maybe be relaxed later after separate careful
considerations.
Some functionality that is currently not supported, but could possibly
be added as incremental features, some easier than others:
- index on or using a virtual column
- hence also no unique constraints on virtual columns
- extended statistics on virtual columns
- foreign-key constraints on virtual columns
- not-null constraints on virtual columns (check constraints are supported)
- ALTER TABLE / DROP EXPRESSION
- virtual column cannot have domain type
- virtual columns are not supported in logical replication
The tests in generated_virtual.sql have been copied over from
generated_stored.sql with the keyword replaced. This way we can make
sure the behavior is mostly aligned, and the differences can be
visible. Some tests for currently not supported features are
currently commented out.
Reviewed-by: Jian He <jian.universality@gmail.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Tested-by: Shlok Kyal <shlok.kyal.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/a368248e-69e4-40be-9c07-6c3b5880b0a6@eisentraut.org
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/sql/page.sql
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_foreign_table.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/trigger.sgml
M src/backend/access/common/tupdesc.c
M src/backend/access/heap/heapam_handler.c
M src/backend/catalog/heap.c
M src/backend/catalog/pg_publication.c
M src/backend/commands/analyze.c
M src/backend/commands/indexcmds.c
M src/backend/commands/publicationcmds.c
M src/backend/commands/statscmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/execMain.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeModifyTable.c
M src/backend/parser/gram.y
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_utilcmd.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/utils/cache/relcache.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/psql/describe.c
M src/include/access/tupdesc.h
M src/include/catalog/catversion.h
M src/include/catalog/heap.h
M src/include/catalog/pg_attribute.h
M src/include/executor/nodeModifyTable.h
M src/include/nodes/execnodes.h
M src/include/nodes/parsenodes.h
M src/include/parser/kwlist.h
M src/include/rewrite/rewriteHandler.h
M src/pl/plperl/expected/plperl_trigger.out
M src/pl/plperl/plperl.c
M src/pl/plperl/sql/plperl_trigger.sql
M src/pl/plpython/expected/plpython_trigger.out
M src/pl/plpython/plpy_typeio.c
M src/pl/plpython/sql/plpython_trigger.sql
M src/pl/tcl/expected/pltcl_trigger.out
M src/pl/tcl/pltcl.c
M src/pl/tcl/sql/pltcl_trigger.sql
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/expected/create_table_like.out
M src/test/regress/expected/fast_default.out
M src/test/regress/expected/generated_stored.out
A src/test/regress/expected/generated_virtual.out
M src/test/regress/expected/publication.out
M src/test/regress/expected/rowsecurity.out
M src/test/regress/expected/stats_ext.out
M src/test/regress/parallel_schedule
M src/test/regress/sql/collate.icu.utf8.sql
M src/test/regress/sql/create_table_like.sql
M src/test/regress/sql/fast_default.sql
M src/test/regress/sql/generated_stored.sql
A src/test/regress/sql/generated_virtual.sql
M src/test/regress/sql/publication.sql
M src/test/regress/sql/rowsecurity.sql
M src/test/regress/sql/stats_ext.sql
M src/test/subscription/t/011_generated.pl
M src/test/subscription/t/028_row_filter.pl
M src/test/subscription/t/031_column_list.pl
Track unpruned relids to avoid processing pruned relations
commit : cbc127917e04a978a788b8bc9d35a70244396d5b
author : Amit Langote <amitlan@postgresql.org>
date : Fri, 7 Feb 2025 17:15:09 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Fri, 7 Feb 2025 17:15:09 +0900
This commit introduces changes to track unpruned relations explicitly,
making it possible for top-level plan nodes, such as ModifyTable and
LockRows, to avoid processing partitions pruned during initial
pruning. Scan-level nodes, such as Append and MergeAppend, already
avoid the unnecessary processing by accessing partition pruning
results directly via part_prune_index. In contrast, top-level nodes
cannot access pruning results directly and need to determine which
partitions remain unpruned.
To address this, this commit introduces a new bitmapset field,
es_unpruned_relids, which the executor uses to track the set of
unpruned relations. This field is referenced during plan
initialization to skip initializing certain nodes for pruned
partitions. It is initialized with PlannedStmt.unprunableRelids,
a new field that the planner populates with RT indexes of relations
that cannot be pruned during runtime pruning. These include relations
not subject to partition pruning and those required for execution
regardless of pruning.
PlannedStmt.unprunableRelids is computed during set_plan_refs() by
removing the RT indexes of runtime-prunable relations, identified
from PartitionPruneInfos, from the full set of relation RT indexes.
ExecDoInitialPruning() then updates es_unpruned_relids by adding
partitions that survive initial pruning.
To support this, PartitionedRelPruneInfo and PartitionedRelPruningData
now include a leafpart_rti_map[] array that maps partition indexes to
their corresponding RT indexes. The former is used in set_plan_refs()
when constructing unprunableRelids, while the latter is used in
ExecDoInitialPruning() to convert partition indexes returned by
get_matching_partitions() into RT indexes, which are then added to
es_unpruned_relids.
These changes make it possible for ModifyTable and LockRows nodes to
process only relations that remain unpruned after initial pruning.
ExecInitModifyTable() trims lists, such as resultRelations,
withCheckOptionLists, returningLists, and updateColnosLists, to
consider only unpruned partitions. It also creates ResultRelInfo
structs only for these partitions. Similarly, child RowMarks for
pruned relations are skipped.
By avoiding unnecessary initialization of structures for pruned
partitions, these changes improve the performance of updates and
deletes on partitioned tables during initial runtime pruning.
Due to ExecInitModifyTable() changes as described above, EXPLAIN on a
plan for UPDATE and DELETE that uses runtime initial pruning no longer
lists partitions pruned during initial pruning.
Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions)
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
M src/backend/commands/copyfrom.c
M src/backend/executor/execMain.c
M src/backend/executor/execParallel.c
M src/backend/executor/execPartition.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeLockRows.c
M src/backend/executor/nodeMergeAppend.c
M src/backend/executor/nodeModifyTable.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/partitioning/partprune.c
M src/backend/replication/logical/worker.c
M src/backend/replication/pgoutput/pgoutput.c
M src/include/executor/execPartition.h
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
injection_points: Tweak more permutation in isolation test "basic"
commit : 926c7fce03603b49dd2529e37d47b805730584fc
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 7 Feb 2025 13:58:22 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 7 Feb 2025 13:58:22 +0900
The CI has reported that using a marker to force the output of the
detach step to happen after the wait step was not enough, as
isolationtester has managed to report the detach step as waiting before
the wait step finishes in some runs.
src/test/isolation/README tells that there is a more drastic method to
enforce the ordering of the output: an empty step positioned just after
the wait step can force the wait step to complete before the detach step
begins. This method has been able to pass 10 runs in the CI here, while
HEAD seems to fail 15~20% of the time in the CF bot.
Discussion: https://postgr.es/m/Z6WO8FbqK_FHmrzC@paquier.xyz
M src/test/modules/injection_points/expected/basic.out
M src/test/modules/injection_points/specs/basic.spec
Move SQL tests of pg_stat_io for WAL data to recovery test 029_stats_restart
commit : 428fadb7e97e3c95180d75dea73de99c491a9f65
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 7 Feb 2025 09:42:31 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 7 Feb 2025 09:42:31 +0900
Three tests in the main regression test suite are proving to not be
portable across multiple runs on a deployed cluster as stats of
pg_stat_io are reset. Problems happen for tests on:
- Writes of WAL in the init context, when creating a WAL segment.
- Syncs of WAL in the init context, when creating a WAL segment.
- Reads of WAL in the normal context, requiring a WAL record to be read.
For a `make check`, this could rely on the checkpoint record read by the
startup process when starting the cluster, something that is not going
to work for a deployed node.
Two of the three tests are moved to the recovery TAP test
029_stats_restart, where we already check the consistency of stats
data. The test for syncs is dropped as TAP can run with fsync=off. The
other two are checked with some data from a freshly-initialized cluster.
Per discussion with Tom Lane, Bertrand Drouvot and Nazir Bilal Yavuz.
Discussion: https://postgr.es/m/915687.1738780322@sss.pgh.pa.us
M src/test/recovery/t/029_stats_restart.pl
M src/test/regress/expected/stats.out
M src/test/regress/sql/stats.sql
Disallow COPY FREEZE on foreign tables.
commit : 401a6956fa69c9202cbc14c09ba8a9c430b90cac
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 6 Feb 2025 15:23:40 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 6 Feb 2025 15:23:40 -0600
This didn't actually work: the COPY succeeds, but the FREEZE
optimization isn't applied. There doesn't seem to be an easy way
to support FREEZE on foreign tables, so let's follow the precedent
established by commit 5c9a5513a3 by raising an error early. This
is arguably a bug fix, but due to the lack of reports, the minimal
discussion on the mailing list, and the potential to break existing
scripts, I am not back-patching it for now.
Author: Sami Imseih <samimseih@gmail.com>
Reviewed-by: Zhang Mingli <zmlpostgres@gmail.com>
Discussion: https://postgr.es/m/CAA5RZ0ujeNgKpE3OrLtR%3DeJGa5LkGMekFzQTwjgw%3DrzaLufQLQ%40mail.gmail.com
M doc/src/sgml/ref/copy.sgml
M src/backend/commands/copyfrom.c
M src/test/regress/expected/copy.out
M src/test/regress/sql/copy.sql
libpq: Handle asynchronous actions during SASL
commit : a99a32e43ed72bd4fdb0950d2359fa4aa50fab76
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 6 Feb 2025 22:19:21 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 6 Feb 2025 22:19:21 +0100
This adds the ability for a SASL mechanism to signal PQconnectPoll()
that some arbitrary work, external to the Postgres connection, is
required for authentication to continue. There is no consumer for
this capability as part of this commit, it is infrastructure which
is required for future work on supporting the OAUTHBEARER mechanism.
To ensure that threads are not blocked waiting for the SASL mechanism
to make long-running calls, the mechanism communicates with the top-
level client via the "altsock": a file or socket descriptor, opaque to
this layer of libpq, which is signaled when work is ready to be done
again. The altsock temporarily replaces the regular connection
descriptor, so existing PQsocket() clients should continue to operate
correctly using their existing polling implementations.
For a mechanism to use this it should set an authentication callback,
conn->async_auth(), and a cleanup callback, conn->cleanup_async_auth(),
and return SASL_ASYNC during the exchange. It should then assign
conn->altsock during the first call to async_auth(). When the cleanup
callback is called, either because authentication has succeeded or
because the connection is being dropped, the altsock must be released
and disconnected from the PGconn object.
This was extracted from the larger OAUTHBEARER patchset which has
been developed, and reviewed by many, over several years and it is
thus likely that some reviewer credit of much earlier versions has
been accidentally omitted.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Antonin Houska <ah@cybertec.at>
Discussion: https://postgr.es/m/CAOYmi+kJqzo6XsR9TEhvVfeVNQ-TyFM5LATypm9yoQVYk=4Wrw@mail.gmail.com
M src/interfaces/libpq/fe-auth-sasl.h
M src/interfaces/libpq/fe-auth-scram.c
M src/interfaces/libpq/fe-auth.c
M src/interfaces/libpq/fe-auth.h
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-misc.c
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
Remove support for linking with libeay32 and ssleay32
commit : 44ec09575145e908be8130de701e5c83995d1abe
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 6 Feb 2025 20:26:46 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 6 Feb 2025 20:26:46 +0100
The OpenSSL project stopped using the eay names back in 2016
on platforms other than Microsoft Windows, and version 1.1.0
removed the names from Windows as well. Since we now require
OpenSSL 1.1.1 we can remove support for using the eay names
from our tree as well.
Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/3C445F8E-D43E-4970-9CD9-A54882197714@yesql.se
Discussion: https://postgr.es/m/CAHrt6656W9OnFomQTHBGYDcM5CKZ7hcgzFt8L+N0ezBZfcN3zA@mail.gmail.com
M configure
M configure.ac
M contrib/pgcrypto/Makefile
M contrib/sslinfo/Makefile
M src/interfaces/libpq/Makefile
M src/test/modules/ssl_passphrase_callback/Makefile
Fix autovacuum_vacuum_max_threshold's GUC description.
commit : 527f8fec220b4c3b84d353543e9741b76d396a11
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 6 Feb 2025 11:59:12 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 6 Feb 2025 11:59:12 -0600
Most GUCs that accept a special value to disable the feature
mention it in their GUC description. This commit adds that
information to autovacuum_vacuum_max_threshold's description.
Oversight in commit 306dc520b9.
M src/backend/utils/misc/guc_tables.c
pgcrypto: Remove static storage class from variables
commit : affd38e55a3cabbc4c64198dde71c22a94877690
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 6 Feb 2025 15:13:40 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 6 Feb 2025 15:13:40 +0100
Variables p, sp and ep were labeled with static storage class
but are all assigned before use so they cannot carry any data
across calls. Fix by removing the static label.
Also while in there, make the magic variable const as it will
never change.
Author: Japin Li <japinli@hotmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/ME0P300MB0445096B67ACE8CE25772F00B6F72@ME0P300MB0445.AUSP300.PROD.OUTLOOK.COM
M contrib/pgcrypto/crypt-md5.c
injection_points: Re-enable permutation in isolation test "basic"
commit : 9e020050b8fa8e184bc1d58e6a4bc1edfa76cb8c
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 6 Feb 2025 10:39:41 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 6 Feb 2025 10:39:41 +0900
This test has been disabled in 9f00edc22888 due to an instable expected
output, where it would be possible for the wait step to report its
result after the detach step is done. The expected output was ordered
so as the detach would always report last.
Isolation test permutations have the option to use markers to control
the ordering for cases like this one, as documented in
src/test/isolation/README. The permutation is enabled once again, this
time with a marker added so as the detach step reports only once the
wait step has finished, ensuring a correct output ordering.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Z6MBZTX5EqQ6A8Zc@paquier.xyz
M src/test/modules/injection_points/expected/basic.out
M src/test/modules/injection_points/specs/basic.spec
Introduce autovacuum_vacuum_max_threshold.
commit : 306dc520b9dfd6014613961962a89940a431a069
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 5 Feb 2025 15:48:18 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 5 Feb 2025 15:48:18 -0600
One way autovacuum chooses tables to vacuum is by comparing the
number of updated or deleted tuples with a value calculated using
autovacuum_vacuum_threshold and autovacuum_vacuum_scale_factor.
The threshold specifies the base value for comparison, and the
scale factor specifies the fraction of the table size to add to it.
This strategy ensures that smaller tables are vacuumed after fewer
updates/deletes than larger tables, which is reasonable in many
cases but can result in infrequent vacuums on very large tables.
This is undesirable for a couple of reasons, such as very large
tables incurring a huge amount of bloat between vacuums.
This new parameter provides a way to set a limit on the value
calculated with autovacuum_vacuum_threshold and
autovacuum_vacuum_scale_factor so that very large tables are
vacuumed more frequently. By default, it is set to 100,000,000
tuples, but it can be disabled by setting it to -1. It can also be
adjusted for individual tables by changing storage parameters.
Author: Nathan Bossart <nathandbossart@gmail.com>
Co-authored-by: Frédéric Yhuel <frederic.yhuel@dalibo.com>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Michael Banck <mbanck@gmx.net>
Reviewed-by: Joe Conway <mail@joeconway.com>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: wenhui qiu <qiuwenhuifx@gmail.com>
Reviewed-by: Vinícius Abrahão <vinnix.bsd@gmail.com>
Reviewed-by: Robert Treat <rob@xzilla.net>
Reviewed-by: Alena Rybakina <a.rybakina@postgrespro.ru>
Discussion: https://postgr.es/m/956435f8-3b2f-47a6-8756-8c54ded61802%40dalibo.com
M doc/src/sgml/config.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/ref/create_table.sgml
M src/backend/access/common/reloptions.c
M src/backend/postmaster/autovacuum.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/psql/tab-complete.in.c
M src/include/postmaster/autovacuum.h
M src/include/utils/rel.h
Show more-intuitive titles for psql commands \dt, \di, etc.
commit : a14707da564e8c94bd123f0e3a75e194fd7ef56a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 5 Feb 2025 12:45:58 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 5 Feb 2025 12:45:58 -0500
If exactly one relation type is requested in a command of the \dtisv
family, say "tables", "indexes", etc instead of "relations". This
should cover the majority of actual uses, without creating a huge
number of new translatable strings. The error messages for no
matching relations are adjusted as well.
In passing, invent "pg_log_error_internal()" to be used for frontend
error messages that don't seem to need translation, analogously to
errmsg_internal() in the backend. The implementation is a bit cheesy,
being just a macro to prevent xgettext from recognizing a trigger
keyword. This won't avoid a useless gettext lookup cycle at runtime
--- but surely we don't care about an extra microsecond or two in
what's supposed to be a can't-happen case. I (tgl) also made
"pg_fatal_internal()", though it's not used in this patch.
Author: Greg Sabino Mullane <htamfids@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAKAnmm+7o93fQV-RFkGaN1QnP-0D4d3JTykD+cLueqjDMKdfag@mail.gmail.com
M src/bin/psql/describe.c
M src/include/common/logging.h
M src/test/regress/expected/dependency.out
M src/test/regress/expected/psql.out
doc: Update links which returned 404
commit : ee4667f0184d87a2e58822e6a9edad563fa164fd
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 5 Feb 2025 13:58:40 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 5 Feb 2025 13:58:40 +0100
Two links in the isn module documentation were pointing to tools
which had been moved, resulting in 404 error responses. Update
to the new URLs for the tools. The link to the Sequoia 2000 page
in the history section was no longer working, and since the page
is no longer available online update our link to point at the
paper instead which is on a stable URL.
These links exist in all versions of the documentation so backpatch
to all supported branches.
Author: Daniel Gustafsson <daniel@yesql.se>
Reported-by: charukiewicz@protonmail.com
Discussion: https://postgr.es/m/173679670185.705.8565555804465055355@wrigleys.postgresql.org
Backpatch-through: 13
M doc/src/sgml/biblio.sgml
M doc/src/sgml/history.sgml
M doc/src/sgml/isn.sgml
Avoid updating inactive_since for invalid replication slots.
commit : 0ec3c295e7594ed3af86bca1a4b4be269c2f069d
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 5 Feb 2025 08:56:14 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 5 Feb 2025 08:56:14 +0530
It is possible for the inactive_since value of an invalid replication slot
to be updated multiple times, which is unexpected behavior like during the
release of the slot or at the time of restart. This is harmless because
invalid slots are not allowed to be accessed but it is not prudent to
update invalid slots. We are planning to invalidate slots due to other
reasons like idle time and it will look odd that the slot's inactive_since
displays the recent time in this field after invalidated due to idle time.
So, this patch ensures that the inactive_since field of slots is not
updated for invalid slots.
In the passing, ensure to use the same inactive_since time for all the
slots at restart while restoring them from the disk.
Author: Nisha Moond <nisha.moond412@gmail.com>
Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Hou Zhijie <houzj.fnst@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CABdArM7QdifQ_MHmMA=Cc4v8+MeckkwKncm2Nn6tX9wSCQ-+iw@mail.gmail.com
M doc/src/sgml/system-views.sgml
M src/backend/replication/logical/slotsync.c
M src/backend/replication/slot.c
M src/include/replication/slot.h
meson: ci: ensure tests are built before running them
commit : b2bdb972c0df3b7091e39d1a31484e5e065519f4
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
Meson 1.7 stopped building all the dependencies of tests as part of the
default build target. But it does breaks CI because we only built the default
target before running the test, and ran the tests with --no-rebuild.
The simplest fix would be to remove --no-rebuild from MTEST_ARGS, but it seems
better to explicitly build the test dependencies, so compiler warnings /
errors are visible as part of the build step.
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Backpatch: 16-, where meson was added
M .cirrus.tasks.yml
meson: Add missing dependencies for libpq tests
commit : 26aca4de43d5b17916cd9204e9e2d1bf0964f6cb
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
The missing dependency was, e.g., visible when doing
ninja clean && ninja meson-test-prereq && meson test --no-rebuild --suite setup --suite libpq
This is a bit more complicated than other related fixes, because until now
libpq's tests depended on 'frontend_code', which includes a dependency on
fe_utils, which in turns on libpq. That in turn required
src/interfaces/libpq/test to be entered from the top-level, not from
libpq/meson.build. Because of that the test definitions in libpq/meson.build
could not declare a dependency on the binaries defined in
libpq/test/meson.build.
To fix this, this commit creates frontend_no_fe_utils_code, which allows us to
recurse into libpq/test from withing libpq/meson.build.
Apply this to all branches with meson support, as part of an effort to fix
incorrect test dependencies that can lead to test failures.
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Discussion: https://postgr.es/m/bdba588f-69a9-4f3e-9b95-62d07210a32e@eisentraut.org
Backpatch: 16-, where meson support was added
M meson.build
M src/interfaces/libpq/meson.build
M src/interfaces/libpq/test/meson.build
meson: Add missing dependencies to libpq_pipeline test
commit : c89525d57bce57f759042c746b15676e0ea30ce2
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
The missing dependency was, e.g., visible when doing
ninja clean && ninja meson-test-prereq && meson test --no-rebuild --suite setup --suite libpq_pipeline
Apply this to all branches with meson support, as part of an effort to fix
incorrect test dependencies that can lead to test failures.
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Discussion: https://postgr.es/m/bdba588f-69a9-4f3e-9b95-62d07210a32e@eisentraut.org
Backpatch: 16-, where meson support was added
M src/test/modules/libpq_pipeline/meson.build
meson: Add test dependencies for test_json_parser
commit : 1be5c37372432d88925d4bc6ab08b1249b558a8f
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
This is required to ensure correct test dependencies, previously
the test binaries would not necessarily be built.
The missing dependency was, e.g., visible when doing
ninja clean && ninja meson-test-prereq && m test --no-rebuild --suite setup --suite test_json_parser
Apply this to all branches with meson support, as part of an effort to fix
incorrect test dependencies that can lead to test failures.
Author: Peter Eisentraut <peter@eisentraut.org>
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Discussion: https://postgr.es/m/bdba588f-69a9-4f3e-9b95-62d07210a32e@eisentraut.org
Backpatch: 17-, where test_json_parser was added
M src/test/modules/test_json_parser/meson.build
meson: Add pg_regress_ecpg to ecpg test dependencies
commit : 69b05d581eca97fd4de5860cd91ea457fcbfd161
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
This is required to ensure correct test dependencies, previously
pg_regress_ecpg would not necessarily be built.
The missing dependency was, e.g., visible when doing
ninja clean && ninja meson-test-prereq && meson test --no-rebuild --suite setup --suite ecpg
Apply this to all branches with meson support, as part of an effort to fix
incorrect test dependencies that can lead to test failures.
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Discussion: https://postgr.es/m/bdba588f-69a9-4f3e-9b95-62d07210a32e@eisentraut.org
Backpatch: 16-, where meson support was added
M src/interfaces/ecpg/test/meson.build
meson: Improve dependencies for tmp_install test target
commit : 74ef4855b08125390f9142517d5e8eff2257900c
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:55 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:55 -0500
The missing dependency was, e.g., visible when doing
ninja clean && ninja meson-test-prereq && meson test --no-rebuild --suite setup --suite cube
because meson (and thus its internal meson-test-prereq target) did not know
about a lot of the required targets.
Previously tmp_install did not actually depend on the relevant files being
built. That was mostly not visible, because "meson test" currently uses the
'default' targets as a test's dependency if no dependency is specified.
However, there are plans to narrow that on the meson side, to make it quicker
to run tests.
Apply this to all branches with meson support, as part of an effort to fix
incorrect test dependencies that can lead to test failures.
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Discussion: https://postgr.es/m/bdba588f-69a9-4f3e-9b95-62d07210a32e@eisentraut.org
Backpatch: 16-, where meson support was added
M meson.build
meson: Narrow dependencies for 'install-quiet' target
commit : c2ede6640ca2d0f2966ef6aaa5d9b245642cdb53
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:55 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:55 -0500
Previously test dependencies, which are not actually installed, were
unnecessarily built.
Apply this to all branches with meson support, as part of an effort to fix
incorrect test dependencies that can lead to test failures.
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Discussion: https://postgr.es/m/bdba588f-69a9-4f3e-9b95-62d07210a32e@eisentraut.org
Backpatch: 16-, where meson support was added
M meson.build
pg_controldata: Fix possible errors on corrupted pg_control
commit : ff1975ddd0270703306dee56601b17742d0934b6
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 5 Feb 2025 00:15:17 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 5 Feb 2025 00:15:17 +0200
Protect against malformed timestamps. Also protect against negative WalSegSz
as it triggers division by zero:
((0x100000000UL) / (WalSegSz)) can turn into zero in
XLogFileName(xlogfilename, ControlFile->checkPointCopy.ThisTimeLineID,
segno, WalSegSz);
because if WalSegSz is -1 then by arithmetic rules in C we get
0x100000000UL / 0xFFFFFFFFFFFFFFFFUL == 0.
Author: Ilyasov Ian <ianilyasov@outlook.com>
Author: Anton Voloshin <a.voloshin@postgrespro.ru>
Backpatch-through: 13
M src/bin/pg_controldata/pg_controldata.c
Allow usage of match_orclause_to_indexcol() for joins
commit : 627d63419e22054551327216d2b2de3e6977fade
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 4 Feb 2025 23:21:49 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 4 Feb 2025 23:21:49 +0200
This commit allows transformation of OR-clauses into SAOP's for index scans
within nested loop joins. That required the following changes.
1. Make match_orclause_to_indexcol() and group_similar_or_args() understand
const-ness in the same way as match_opclause_to_indexcol(). This
generally makes our approach more uniform.
2. Make match_join_clauses_to_index() pass OR-clauses to
match_clause_to_index().
3. Also switch match_join_clauses_to_index() to use list_append_unique_ptr()
for adding clauses to *joinorclauses. That avoids possible duplicates
when processing the same clauses with different indexes. Previously such
duplicates were elimited in match_clause_to_index(), but now
group_similar_or_args() each time generates distinct copies of grouped
OR clauses.
Discussion: https://postgr.es/m/CAPpHfdv%2BjtNwofg-p5z86jLYZUTt6tR17Wy00ta0dL%3DwHQN3ZA%40mail.gmail.com
Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Alena Rybakina <a.rybakina@postgrespro.ru>
Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com>
M src/backend/optimizer/path/indxpath.c
M src/test/regress/expected/create_index.out
M src/test/regress/expected/join.out
M src/test/regress/expected/partition_join.out
M src/test/regress/sql/create_index.sql
M src/test/regress/sql/join.sql
Revise the header comment for match_clause_to_indexcol()
commit : 23ef119f58e3eaa29bedc69320fd61b108ab8c10
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 4 Feb 2025 23:18:47 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 4 Feb 2025 23:18:47 +0200
Since d4378c0005e6, match_clause_to_indexcol() doesn't always return NULL
for an OR clause. This commit reflects that in the function header comment.
Reported-by: Pavel Borisov <pashkin.elfe@gmail.com>
M src/backend/optimizer/path/indxpath.c
vacuumdb: Add missing PQfinish() calls to vacuum_one_database().
commit : f3e4aeb744da6acf909e9c5a7a83338fba1984a7
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 4 Feb 2025 13:26:57 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 4 Feb 2025 13:26:57 -0600
A few of the version checks in vacuum_one_database() do not call
PQfinish() before exiting. This precedent was unintentionally
established in commit 00d1e88d36, and while it's probably not too
problematic, it seems better to properly close the connection.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/Z6JAwqN1I8ljTuXp%40nathan
Backpatch-through: 13
M src/bin/scripts/vacuumdb.c
sepgsql: update TAP test to use fat comma style
commit : cc2c9fa6960c4abd5e4f4bcafb35fd3ba7b5ee98
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 4 Feb 2025 15:51:42 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 4 Feb 2025 15:51:42 +0100
Adopt the style introduced by commit ce1b0f9da03 to this new test
file.
Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/87y0yv2har.fsf@wibble.ilmari.org
M contrib/sepgsql/t/001_sepgsql.pl
Add data for WAL in pg_stat_io and backend statistics
commit : a051e71e28a12342a4fb39a3c149a197159f9c46
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 4 Feb 2025 16:50:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 4 Feb 2025 16:50:00 +0900
This commit adds WAL IO stats to both pg_stat_io view and per-backend IO
statistics (pg_stat_get_backend_io()). This change is possible since
f92c854cf406, as WAL IO is not counted in blocks in some code paths
where its stats data is measured (like WAL read in xlogreader.c).
IOContext gains IOCONTEXT_INIT and IOObject IOOBJECT_WAL, with the
following combinations allowed:
- IOOBJECT_WAL/IOCONTEXT_NORMAL is used to track I/O operations done on
already-created WAL segments.
- IOOBJECT_WAL/IOCONTEXT_INIT is used for tracking I/O operations done
when initializing WAL segments.
The core changes are done in pg_stat_io.c, backend statistics inherit
them. Backend statistics and pg_stat_io are now available for the WAL
writer, the WAL receiver and the WAL summarizer processes.
I/O timing data is controlled by the GUC track_io_timing, like the
existing data of pg_stat_io for consistency. The timings related to
IOOBJECT_WAL show up if the GUC is enabled (disabled by default).
Bump pgstats file version, due to the additions in IOObject and
IOContext, impacting the amount of data written for the fixed-numbered
IO stats kind in the pgstats file.
Author: Nazir Bilal Yavuz
Reviewed-by: Bertrand Drouvot, Nitin Jadhav, Amit Kapila, Michael
Paquier, Melanie Plageman, Bharath Rupireddy
Discussion: https://postgr.es/m/CAN55FZ3AiQ+ZMxUuXnBpd0Rrh1YhwJ5FudkHg=JU0P+-W8T4Vg@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogrecovery.c
M src/backend/utils/activity/pgstat_backend.c
M src/backend/utils/activity/pgstat_io.c
M src/include/pgstat.h
M src/test/regress/expected/stats.out
M src/test/regress/sql/stats.sql
Integrate GistTranslateCompareType() into IndexAmTranslateCompareType()
commit : 622f678c10202c8a0b350794d504eeef7b773e90
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 3 Feb 2025 08:14:27 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 3 Feb 2025 08:14:27 +0100
This turns GistTranslateCompareType() into a callback function of the
gist index AM instead of a standalone function. The existing callers
are changed to use IndexAmTranslateCompareType(). This then makes
that code not hardcoded toward gist.
This means in particular that the temporal keys code is now
independent of gist. Also, this generalizes commit 74edabce7a3, so
other index access methods other than the previously hardcoded ones
could now work as REPLICA IDENTITY in a logical replication
subscriber.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Co-authored-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/access/gist/gist.c
M src/backend/access/gist/gistutil.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execReplication.c
M src/backend/replication/logical/relation.c
M src/include/access/gist.h
M src/include/executor/executor.h
Fix incorrect range in pg_regress comment.
commit : 43a15eb9400dba2b0b97be72d1a3745a6a6f7136
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 2 Feb 2025 22:37:13 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 2 Feb 2025 22:37:13 -0500
A comment in pg_regress incorrectly stated that alternative
output files could be named test_{i}.out with 0 < i <= 9.
However, the valid range is actually 0 <= i <= 9.
(The user-facing docs have this right already.)
Author: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Discussion: https://postgr.es/m/6e6c4dea-07a1-4a83-9bb7-77b9b3324c37@tantorlabs.com
M src/test/regress/pg_regress.c
Improve comment on top of pgstat_count_io_op_time()
commit : b998fedab74c6cf4e564c17f4171881759a093ee
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 3 Feb 2025 11:19:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 3 Feb 2025 11:19:58 +0900
This commit adds more documentation to pgstat_count_io_op_time() in
pgstat_io.c, explaining its internals for pgstat_count_buffer_*(),
pgBufferUsage and the contexts where these are used.
Extracted from a larger patch by the same author.
Author: Nazir Bilal Yavuz
Discussion: https://postgr.es/m/CAN55FZ3AiQ+ZMxUuXnBpd0Rrh1YhwJ5FudkHg=JU0P+-W8T4Vg@mail.gmail.com
M src/backend/utils/activity/pgstat_io.c
Fix typo in xlog.c
commit : fcce828529ee36ac452aead60fe95648bde3256c
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 3 Feb 2025 09:22:45 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 3 Feb 2025 09:22:45 +0900
"recovery" is not a verb. Introduced in 68cb5af46cd8.
M src/backend/access/transam/xlog.c
Convert strategies to and from compare types
commit : c09e5a6a01659a66dd84f3e745694999d3414ddd
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 2 Feb 2025 10:26:04 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 2 Feb 2025 10:26:04 +0100
For each Index AM, provide a mapping between operator strategies and
the system-wide generic concept of a comparison type. For example,
for btree, BTLessStrategyNumber maps to and from COMPARE_LT. Numerous
places in the planner and executor think directly in terms of btree
strategy numbers (and a few in terms of hash strategy numbers.) These
should be converted over subsequent commits to think in terms of
CompareType instead. (This commit doesn't make any use of this API
yet.)
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M contrib/bloom/blutils.c
M doc/src/sgml/indexam.sgml
M src/backend/access/brin/brin.c
M src/backend/access/gist/gist.c
M src/backend/access/hash/hash.c
M src/backend/access/index/amapi.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/spgist/spgutils.c
M src/include/access/amapi.h
M src/include/access/cmptype.h
M src/include/access/hash.h
M src/include/access/nbtree.h
M src/tools/pgindent/typedefs.list
Move CompareType to separate header file
commit : 119fc30dd5bd918819b864107ddc8baac51f4d22
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 2 Feb 2025 08:11:57 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 2 Feb 2025 08:11:57 +0100
We'll want to make use of it in more places, and we'd prefer to not
have to include all of primnodes.h everywhere.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M contrib/btree_gist/btree_gist.c
M src/backend/nodes/Makefile
M src/backend/nodes/gen_node_support.pl
A src/include/access/cmptype.h
M src/include/nodes/meson.build
M src/include/nodes/primnodes.h
Mention jsonlog in description of logging_collector in GUC table
commit : d61b9662b09ea55d175680cc31b24768d95972f6
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 2 Feb 2025 11:31:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 2 Feb 2025 11:31:21 +0900
logging_collector was only mentioning stderr and csvlog, and forgot
about jsonlog. Oversight in dc686681e079, that has added support for
jsonlog in log_destination.
While on it, the description in the GUC table is tweaked to be more
consistent with the documentation and postgresql.conf.sample.
Author: Umar Hayat
Reviewed-by: Ashutosh Bapat, Tom Lane
Discussion: https://postgr.es/m/CAD68Dp1K_vBYqBEukHw=1jF7e76t8aszGZTFL2ugi=H7r=a7MA@mail.gmail.com
Backpatch-through: 13
M src/backend/utils/misc/guc_tables.c
Add get_opfamily_name() function
commit : 43493cceda2fb75e07a63cec46000407661d505d
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 24 Jan 2025 22:58:13 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 24 Jan 2025 22:58:13 +0100
This refactors and simplifies various existing code to make use of the
new function.
Reviewed-by: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M contrib/bloom/blvalidate.c
M src/backend/access/brin/brin_validate.c
M src/backend/access/gin/ginvalidate.c
M src/backend/access/gist/gistvalidate.c
M src/backend/access/hash/hashvalidate.c
M src/backend/access/nbtree/nbtvalidate.c
M src/backend/access/spgist/spgvalidate.c
M src/backend/commands/indexcmds.c
M src/backend/utils/cache/lsyscache.c
M src/include/utils/lsyscache.h
Rename GistTranslateStratnum() to GistTranslateCompareType()
commit : a5709b5bb293a554913c4b1f6a9c58d1915ba3f7
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 1 Feb 2025 10:18:46 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 1 Feb 2025 10:18:46 +0100
Follow up to commit 630f9a43cec. The previous name had become
confusing, because it doesn't actually translate a strategy number but
a CompareType into a strategy number. We might add the inverse at
some point, which would then probably be called something like
GistTranslateStratnum.
Reviewed-by: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/access/gist/gistutil.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execReplication.c
M src/include/access/gist.h
Add script to keep .editorconfig in sync with .gitattributes
commit : 2452e71ff2fe642ff35535d48a728aeca2d61a8d
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 1 Feb 2025 10:01:16 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 1 Feb 2025 10:01:16 +0100
Our repo already contained an .editorconfig file, but it was not kept
up to date with .gitattributes. This adds a script that keeps these
files in sync. A big advantage of the editorconfig file is that it
many editors/IDEs get automatically configured to trim trailing
newlines and add a final newline on save, while .gitattributes only
complains about these problems instead of automatically fixing them.
This also adds rules to .gitattributes for Python files as well as for
C files in pg_bsd_indent directory (which have a different tab_width
than most C files due to being vendored in).
Author: Jelte Fennema-Nio <github-tech@jeltef.nl>
Discussion: https://www.postgresql.org/message-id/flat/CAGECzQQGzbroAXi+Yicp3HvcCo4=g84kaOgjuvQ5MW9F0ubOGg@mail.gmail.com
M .editorconfig
M .gitattributes
A src/tools/generate_editorconfig.py
Add commit 76aa615943 to .git-blame-ignore-revs
commit : 79e872fedb4d2f73baa0a80ba572480774edc61b
author : Amit Langote <amitlan@postgresql.org>
date : Sat, 1 Feb 2025 16:36:18 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Sat, 1 Feb 2025 16:36:18 +0900
M .git-blame-ignore-revs
Doc: add commentary about cowboy assignment of maintenance_work_mem.
commit : 53a49365052026907afff7613929710d1e7f0da0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 31 Jan 2025 15:17:15 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 31 Jan 2025 15:17:15 -0500
Whilst working on commit 041e8b95b I happened to notice that
parallel_vacuum_main() assigns directly to the maintenance_work_mem
GUC. This is definitely not per project conventions, so I tried to
fix it to use SetConfigOption(). But that fails with "parameter
cannot be set during a parallel operation". It doesn't seem worth
working on a cleaner answer, at least not till we have a few more
instances of similar problems. But add some commentary, just so
nobody gets the idea that this is an approved way to set a GUC.
M src/backend/commands/vacuumparallel.c
Remove obsolete restriction on the range of log_rotation_size.
commit : d4c3a6b8ad830882066122081a7141ecd573f45d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 31 Jan 2025 14:36:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 31 Jan 2025 14:36:56 -0500
When syslogger.c was first written, we didn't want to assume that
all platforms have 64-bit ftello. But we've been assuming that
since v13 (cf commit 799d22461), so let's use that in syslogger.c
and allow log_rotation_size to range up to INT_MAX kilobytes.
The old code effectively limited log_rotation_size to 2GB regardless
of platform. While nobody's complained, that doesn't seem too far
away from what might be thought reasonable these days.
I noticed this while searching for instances of "1024L" in connection
with commit 041e8b95b. These were the last such instances.
(We still have instances of L-suffixed literals, but most of them
are associated with wait intervals for pg_usleep or similar functions.
I don't see any urgent reason to change that.)
M src/backend/postmaster/syslogger.c
M src/backend/utils/misc/guc_tables.c
Get rid of our dependency on type "long" for memory size calculations.
commit : 041e8b95b8cd251bfec6a3c9c3dd6614de6a4c9b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 31 Jan 2025 13:52:40 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 31 Jan 2025 13:52:40 -0500
Consistently use "Size" (or size_t, or in some places int64 or double)
as the type for variables holding memory allocation sizes. In most
places variables' data types were fine already, but we had an ancient
habit of computing bytes from kilobytes-units GUCs with code like
"work_mem * 1024L". That risks overflow on Win64 where they did not
make "long" as wide as "size_t". We worked around that by restricting
such GUCs' ranges, so you couldn't set work_mem et al higher than 2GB
on Win64. This patch removes that restriction, after replacing such
calculations with "work_mem * (Size) 1024" or variants of that.
It should be noted that this patch was constructed by searching
outwards from the GUCs that have MAX_KILOBYTES as upper limit.
So I can't positively guarantee there are no other places doing
memory-size arithmetic in int or long variables. I do however feel
pretty confident that increasing MAX_KILOBYTES on Win64 is safe now.
Also, nothing in our code should be dealing in multiple-gigabyte
allocations without authorization from a relevant GUC, so it seems
pretty likely that this search caught everything that could be at
risk of overflow.
Author: Vladlen Popolitov <v.popolitov@postgrespro.ru>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/1a01f0-66ec2d80-3b-68487680@27595217
M src/backend/access/gin/ginfast.c
M src/backend/access/gin/ginget.c
M src/backend/access/gin/gininsert.c
M src/backend/access/hash/hash.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/commands/vacuumparallel.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeBitmapIndexscan.c
M src/backend/executor/nodeBitmapOr.c
M src/backend/nodes/tidbitmap.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/utils/sort/tuplestore.c
M src/include/executor/hashjoin.h
M src/include/nodes/tidbitmap.h
M src/include/utils/dsa.h
M src/include/utils/guc.h
M src/test/modules/test_bloomfilter/test_bloomfilter.c
require_auth: prepare for multiple SASL mechanisms
commit : f8d8581ed882b79b512daaa7f71ca19c8eafcaef
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 31 Jan 2025 15:47:28 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 31 Jan 2025 15:47:28 +0100
Prior to this patch, the require_auth implementation assumed that
the AuthenticationSASL protocol message was using SCRAM-SHA-256.
In preparation for future SASL mechanisms, like OAUTHBEARER, split
the implementation into two tiers: the first checks the acceptable
AUTH_REQ_* codes, and the second checks acceptable mechanisms if
AUTH_REQ_SASL et.al are permitted.
conn->allowed_sasl_mechs contains a list of pointers to acceptable
mechanisms, and pg_SASL_init() will bail if the selected mechanism
isn't contained in this array.
Since there's only one mechansism supported right now, one branch
of the second tier cannot be exercised yet and is protected by an
Assert(false) call. This assertion will need to be removed when
the next mechanism is added.
This patch is extracted from a larger body of work aimed at adding
support for OAUTHBEARER in libpq.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/CAOYmi+kJqzo6XsR9TEhvVfeVNQ-TyFM5LATypm9yoQVYk=4Wrw@mail.gmail.com
M src/interfaces/libpq/fe-auth.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/libpq-int.h
M src/test/authentication/t/001_password.pl
Move PG_MAX_AUTH_TOKEN_LENGTH to libpq/auth.h
commit : e21d6f297158db1383a7c9a668ebe1048f2eac39
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 31 Jan 2025 15:39:35 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 31 Jan 2025 15:39:35 +0100
Future SASL mechanism, like OAUTHBEARER, will use this as a limit on
token messages coming from the client, so promote it to the header
file to make it available.
This patch is extracted from a larger body of work aimed at adding
support for OAUTHBEARER in libpq.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/CAOYmi+kJqzo6XsR9TEhvVfeVNQ-TyFM5LATypm9yoQVYk=4Wrw@mail.gmail.com
M src/backend/libpq/auth.c
M src/include/libpq/auth.h
doc: Fix pg_buffercache_evict() title
commit : 59d6c03956193f622c069a4ab985bade27384ac4
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 31 Jan 2025 10:44:21 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 31 Jan 2025 10:44:21 +0100
Use <function> rather than <structname> in the <title> to be consistent
with how other functions in this module are documented. Also suffix the
function name with () for consistency.
Backpatch to v17 where pg_buffercache_evict was introduced.
Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAExHW5uKWH8CuZc9NCb8XxSQc6uzvACV0cScebm54kF763ERAw@mail.gmail.com
Backpatch-through: 17
M doc/src/sgml/pgbuffercache.sgml
Fix bad indentation introduced in commit d47cbf474
commit : 76aa615943049c04efd36ab4765c06eda89cdfea
author : Amit Langote <amitlan@postgresql.org>
date : Fri, 31 Jan 2025 16:44:24 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Fri, 31 Jan 2025 16:44:24 +0900
Per buildfarm member koel
M src/backend/executor/execPartition.c
Perform runtime initial pruning outside ExecInitNode()
commit : d47cbf474ecbd449a47c8c1b4aaa1874f7698271
author : Amit Langote <amitlan@postgresql.org>
date : Fri, 31 Jan 2025 15:47:15 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Fri, 31 Jan 2025 15:47:15 +0900
This commit builds on the prior change that moved PartitionPruneInfos
out of individual plan nodes into a list in PlannedStmt, making it
possible to initialize PartitionPruneStates without traversing the
plan tree and perform runtime initial pruning before ExecInitNode()
initializes the plan trees. These tasks are now handled in a new
routine, ExecDoInitialPruning(), which is called by InitPlan()
before calling ExecInitNode() on various plan trees.
ExecDoInitialPruning() performs the initial pruning and saves the
result -- a Bitmapset of indexes for surviving child subnodes -- in
es_part_prune_results, a list in EState.
PartitionPruneStates created for initial pruning are stored in
es_part_prune_states, another list in EState, for later use during
exec pruning. Both lists are parallel to es_part_prune_infos, which
holds the PartitionPruneInfos from PlannedStmt, enabling shared
indexing.
PartitionPruneStates initialized in ExecDoInitialPruning() now
include only the PartitionPruneContexts for initial pruning steps.
Exec pruning contexts are initialized later in
ExecInitPartitionExecPruning() when the parent plan node is
initialized, as the exec pruning step expressions depend on the parent
node's PlanState.
The existing function PartitionPruneFixSubPlanMap() has been
repurposed for this initialization to avoid duplicating a similar
loop structure for finding PartitionedRelPruningData to initialize
exec pruning contexts for. It has been renamed to
InitExecPruningContexts() to reflect its new primary responsibility.
The original logic to "fix subplan maps" remains intact but is now
encapsulated within the renamed function.
This commit removes two obsolete Asserts in partkey_datum_from_expr().
The ExprContext used for pruning expression evaluation is now
independent of the parent PlanState, making these Asserts unnecessary.
By centralizing pruning logic and decoupling it from the plan
initialization step (ExecInitNode()), this change sets the stage for
future patches that will use the result of initial pruning to
save the overhead of redundant processing for pruned partitions.
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeMergeAppend.c
M src/backend/partitioning/partprune.c
M src/include/executor/execPartition.h
M src/include/nodes/execnodes.h
M src/include/nodes/plannodes.h
Raise an error while trying to acquire an invalid slot.
commit : f41d8468ddea34170fe19fdc17b5a247e7d3ac78
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 31 Jan 2025 10:27:35 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 31 Jan 2025 10:27:35 +0530
Once a replication slot is invalidated, it cannot be altered or used to
fetch changes. However, a process could still acquire an invalid slot and
fail later.
For example, if a process acquires a logical slot that was invalidated due
to wal_removed, it will eventually fail in CreateDecodingContext() when
attempting to access the removed WAL. Similarly, for physical replication
slots, even if the slot is invalidated and invalidation_reason is set to
wal_removed, the walsender does not currently check for invalidation when
starting physical replication. Instead, replication starts, and an error
is only reported later while trying to access WAL. Similarly, we prohibit
modifying slot properties for invalid slots but give the error for the
same after acquiring the slot.
This patch improves error handling by detecting invalid slots earlier at
the time of slot acquisition which is the first step. This also helped in
unifying different ERROR messages at different places and gave a
consistent message for invalid slots. This means that the message for
invalid slots will change to a generic message.
This will also be helpful for future patches where we are planning to
invalidate slots due to more reasons like idle_timeout because we don't
have to modify multiple places in such cases and avoid the chances of
missing out on a particular place.
Author: Nisha Moond <nisha.moond412@gmail.com>
Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CABdArM6pBL5hPnSQ+5nEVMANcF4FCH7LQmgskXyiLY75TMnKpw@mail.gmail.com
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/logicalfuncs.c
M src/backend/replication/logical/slotsync.c
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/walsender.c
M src/backend/utils/adt/pg_upgrade_support.c
M src/include/replication/slot.h
M src/test/recovery/t/019_replslot_limit.pl
M src/test/recovery/t/035_standby_logical_decoding.pl
injection_points: Add routine able to drop all stats
commit : a632cd354d35e1c3352061c375b4458876c9f10a
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 31 Jan 2025 12:41:39 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 31 Jan 2025 12:41:39 +0900
This serves as an example of how to use the new function introduced in
ce5c620fb625, pgstat_drop_matching_entries(), with a callback able to
filter the entries dropped.
A SQL function named injection_points_stats_drop() is added with some
tests.
Author: Lukas Fitti
Discussion: https://postgr.es/m/CAP53PkwuFbo3NkwZgxwNRMjMfqPEqidD-SggaoQ4ijotBVLJAA@mail.gmail.com
M src/test/modules/injection_points/injection_points–1.0.sql
M src/test/modules/injection_points/injection_stats.c
M src/test/modules/injection_points/t/001_stats.pl
Add pgstat_drop_matching_entries() to pgstats
commit : ce5c620fb6252dca00d3856d5f09d56c7f1215d0
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 31 Jan 2025 12:27:19 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 31 Jan 2025 12:27:19 +0900
This allows users of the cumulative statistics to drop entries in the
shared hash stats table, deleting as well local references. Callers of
this function can optionally define a callback able to filter which
entries to drop, similarly to pgstat_reset_matching_entries() with its
callback do_reset().
pgstat_drop_all_entries() is refactored so as it uses this new function.
Author: Lukas Fitti
Discussion: https://postgr.es/m/CAP53PkwuFbo3NkwZgxwNRMjMfqPEqidD-SggaoQ4ijotBVLJAA@mail.gmail.com
M src/backend/utils/activity/pgstat_shmem.c
M src/include/utils/pgstat_internal.h
Fix comment of StrategySyncStart()
commit : 1e380fa7d82df8a1730b80a911f50952c711f2a8
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 31 Jan 2025 11:05:57 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 31 Jan 2025 11:05:57 +0900
The top comment of StrategySyncStart() mentions BufferSync(), but this
function calls BgBufferSync(), not BufferSync().
Oversight in 9cd00c457e6a.
Author: Ashutosh Bapat
Discussion: https://postgr.es/m/CAExHW5tgkjag8i-s=RFrCn5KAWDrC4zEPPkfUKczfccPOxBRQQ@mail.gmail.com
Backpatch-through: 13
M src/backend/storage/buffer/freelist.c
Use "ssize_t" not "long" in max_stack_depth-related code.
commit : b9d232b9de89aa9282f9a2e7c85f783bcd334af2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 30 Jan 2025 16:44:47 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 30 Jan 2025 16:44:47 -0500
This change adapts these functions to the machine's address width
without depending on "long" to be the right size. (It isn't on
Win64, for example.) While it seems unlikely anyone would care
to run with a stack depth limit exceeding 2GB, this is part of a
general push to avoid using type "long" to represent memory sizes.
It's convenient to use ssize_t rather than the perhaps-more-obvious
choice of size_t/Size, because the code involved depends on working
with a signed data type. Our MAX_KILOBYTES limit already ensures
that ssize_t will be sufficient to represent the maximum value of
max_stack_depth.
Extracted from a larger patch by Vladlen, plus additional hackery
by me.
Author: Vladlen Popolitov <v.popolitov@postgrespro.ru>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/1a01f0-66ec2d80-3b-68487680@27595217
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/stack_depth.c
M src/include/miscadmin.h
Avoid integer overflow while testing wal_skip_threshold condition.
commit : b9aa4166fa3823d4f1f76286ca21fcfa991ce036
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 30 Jan 2025 15:36:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 30 Jan 2025 15:36:07 -0500
smgrDoPendingSyncs had two distinct risks of integer overflow while
deciding which way to ensure durability of a newly-created relation.
First, it accumulated the total size of all forks in a variable of
type BlockNumber (uint32). While we restrict an individual fork's
size to fit in that, I don't believe there's such a restriction on
all of them added together. Second, it proceeded to multiply the
sum by BLCKSZ, which most certainly could overflow a uint32.
(The exact expression is total_blocks * BLCKSZ / 1024. The
compiler might choose to optimize that to total_blocks * 8,
which is not at quite as much risk of overflow as a literal
reading would be, but it's still wrong.)
If an overflow did occur it could lead to a poor choice to
shove a very large relation into WAL instead of fsync'ing it.
This wouldn't be fatal, but it could be inefficient.
Change total_blocks to uint64 which should be plenty, and
rearrange the comparison calculation to be overflow-safe.
I noticed this while looking for ramifications of the proposed
change in MAX_KILOBYTES. It's not entirely clear to me why
wal_skip_threshold is limited to MAX_KILOBYTES in the
first place, but in any case this code is unsafe regardless
of the range of wal_skip_threshold.
Oversight in c6b92041d which introduced wal_skip_threshold,
so back-patch to v13.
Discussion: https://postgr.es/m/1a01f0-66ec2d80-3b-68487680@27595217
Backpatch-through: 13
M src/backend/catalog/storage.c
Move BitmapTableScan per-scan setup into a helper
commit : a5358c14b2fe2210a1ac0b836f8d54723043fba2
author : Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 30 Jan 2025 15:26:55 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 30 Jan 2025 15:26:55 -0500
Add BitmapTableScanSetup(), a helper which contains all of the code that
must be done on every scan of the table in a bitmap table scan. This
includes scanning the index, building the bitmap, and setting up the
scan descriptors.
Pushing this setup into a helper function makes BitmapHeapNext() more
readable.
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CAN55FZ1vXu%2BZdT0_MM-i1vbTdfHHf0KR3cK6R5gs6dNNNpyrJw%40mail.gmail.com
M src/backend/executor/nodeBitmapHeapscan.c
Simplify executor's handling of CaseTestExpr & CoerceToDomainValue.
commit : 115a365519bfd53a65bf17d253b26902eff0c337
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 30 Jan 2025 13:21:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 30 Jan 2025 13:21:42 -0500
Instead of deciding at runtime whether to read from casetest.value
or caseValue_datum, split EEOP_CASE_TESTVAL into two opcodes and
make the decision during expression compilation. Similarly for
EEOP_DOMAIN_TESTVAL. This actually results in net less code,
mainly because llvmjit_expr.c's code for handling these opcodes
gets shorter. The performance gain is doubtless negligible, but
this seems worth changing anyway on grounds of simplicity and
understandability.
Author: Andreas Karlsson <andreas@proxel.se>
Co-authored-by: Xing Guo <higuoxing@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CACpMh+AiBYAWn+D1aU7Rsy-V1tox06Cbc0H3qA7rwL5zdJ=anQ@mail.gmail.com
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/include/executor/execExpr.h
Doc: Generated column replication.
commit : 6252b1eaf82bb8db361341d1c8651e43b29be816
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 30 Jan 2025 11:09:18 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 30 Jan 2025 11:09:18 +0530
Commit 7054186c4e added the support to publish generated stored columns.
This patch adds detailed documentation for that feature.
Author: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E%40gmail.com
Discussion: https://postgr.es/m/CAHut+PsYmAvKhUjA1AaR1rxLdeSBKiBko8wKyf4_H8nEEqDuOg@mail.gmail.com
M doc/src/sgml/ddl.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/ref/create_publication.sgml
Move PartitionPruneInfo out of plan nodes into PlannedStmt
commit : bb3ec16e14ded1d23a46d3c7e623a965164fa124
author : Amit Langote <amitlan@postgresql.org>
date : Thu, 30 Jan 2025 11:57:32 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Thu, 30 Jan 2025 11:57:32 +0900
This moves PartitionPruneInfo from plan nodes to PlannedStmt,
simplifying traversal by centralizing all PartitionPruneInfo
structures in a single list in it, which holds all instances for the
main query and its subqueries. Instead of plan nodes (Append or
MergeAppend) storing PartitionPruneInfo pointers, they now reference
an index in this list.
A bitmapset field is added to PartitionPruneInfo to store the RT
indexes corresponding to the apprelids field in Append or MergeAppend.
This allows execution pruning logic to verify that it operates on the
correct plan node, mainly to facilitate debugging.
Duplicated code in set_append_references() and
set_mergeappend_references() is refactored into a new function,
register_pruneinfo(). This updates RT indexes by applying rtoffet
and adds PartitionPruneInfo to the global list in PlannerGlobal.
By allowing pruning to be performed without traversing the plan tree,
this change lays the groundwork for runtime initial pruning to occur
independently of plan tree initialization.
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> (earlier version)
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
M src/backend/executor/execMain.c
M src/backend/executor/execParallel.c
M src/backend/executor/execPartition.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeMergeAppend.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/partitioning/partprune.c
M src/include/executor/execPartition.h
M src/include/nodes/execnodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/partitioning/partprune.h
Require callers of coerce_to_domain() to supply base type/typmod.
commit : ba0da16bd054de854c200a2dfa0b21c70e0300cf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 29 Jan 2025 15:42:25 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 29 Jan 2025 15:42:25 -0500
In view of the issue fixed in commit 0da39aa76, it no longer seems
like a great idea for coerce_to_domain() to offer to perform a lookup
that its caller probably should have done already. The caller should
be providing a value of the domain's base type, so it's hard to
envision a valid case where it hasn't looked up that type. After
0da39aa76 there is only one caller using the option for internal
lookup, and that one can trivially be rearranged to not do that.
So this seems more like a bug-encouraging misfeature than a useful
shortcut; let's get rid of it (in HEAD only, there's no need to
break any external callers in back branches).
Discussion: https://postgr.es/m/1865579.1738113656@sss.pgh.pa.us
M src/backend/parser/parse_coerce.c
Handle default NULL insertion a little better.
commit : 0da39aa7667b06e16189d318f7850d559d446d52
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 29 Jan 2025 15:31:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 29 Jan 2025 15:31:55 -0500
If a column is omitted in an INSERT, and there's no column default,
the code in preptlist.c generates a NULL Const to be inserted.
Furthermore, if the column is of a domain type, we wrap the Const
in CoerceToDomain, so as to throw a run-time error if the domain
has a NOT NULL constraint. That's fine as far as it goes, but
there are two problems:
1. We're being sloppy about the type/typmod that the Const is
labeled with. It really should have the domain's base type/typmod,
since it's the input to CoerceToDomain not the output. This can
result in coerce_to_domain inserting a useless length-coercion
function (useless because it's being applied to a null). The
coercion would typically get const-folded away later, but it'd
be better not to create it in the first place.
2. We're not applying expression preprocessing (specifically,
eval_const_expressions) to the resulting expression tree.
The planner's primary expression-preprocessing pass already happened,
so that means the length coercion step and CoerceToDomain node miss
preprocessing altogether.
This is at the least inefficient, since it means the length coercion
and CoerceToDomain will actually be executed for each inserted row,
though they could be const-folded away in most cases. Worse, it
seems possible that missing preprocessing for the length coercion
could result in an invalid plan (for example, due to failing to
perform default-function-argument insertion). I'm not aware of
any live bug of that sort with core datatypes, and it might be
unreachable for extension types as well because of restrictions of
CREATE CAST, but I'm not entirely convinced that it's unreachable.
Hence, it seems worth back-patching the fix (although I only went
back to v14, as the patch doesn't apply cleanly at all in v13).
There are several places in the rewriter that are building null
domain constants the same way as preptlist.c. While those are
before the planner and hence don't have any reachable bug, they're
still applying a length coercion that will be const-folded away
later, uselessly wasting cycles. Hence, make a utility routine
that all of these places can call to do it right.
Making this code more careful about the typmod assigned to the
generated NULL constant has visible but cosmetic effects on some
of the plans shown in contrib/postgres_fdw's regression tests.
Discussion: https://postgr.es/m/1865579.1738113656@sss.pgh.pa.us
Backpatch-through: 14
M contrib/postgres_fdw/expected/postgres_fdw.out
M src/backend/optimizer/prep/preptlist.c
M src/backend/parser/parse_coerce.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/rewrite/rewriteManip.c
M src/include/parser/parse_coerce.h
Avoid breaking SJIS encoding while de-backslashing Windows paths.
commit : 6cddecdfb00b35c1d0256600d76f68c702dedec7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 29 Jan 2025 14:24:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 29 Jan 2025 14:24:36 -0500
When running on Windows, canonicalize_path() converts '\' to '/'
to prevent confusing the Windows command processor. It was
doing that in a non-encoding-aware fashion; but in SJIS there
are valid two-byte characters whose second byte matches '\'.
So encoding corruption ensues if such a character is used in
the path.
We can fairly easily fix this if we know which encoding is
in use, but a lot of our utilities don't have much of a clue
about that. After some discussion we decided we'd settle for
fixing this only in psql, and assuming that its value of
client_encoding matches what the user is typing.
It seems hopeless to get the server to deal with the problematic
characters in database path names, so we'll just declare that
case to be unsupported. That means nothing need be done in
the server, nor in utility programs whose only contact with
file path names is for database paths. But psql frequently
deals with client-side file paths, so it'd be good if it
didn't mess those up.
Bug: #18735
Reported-by: Koichi Suzuki <koichi.suzuki@enterprisedb.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Koichi Suzuki <koichi.suzuki@enterprisedb.com>
Discussion: https://postgr.es/m/18735-4acdb3998bb9f2b1@postgresql.org
Backpatch-through: 13
M src/bin/psql/command.c
M src/bin/psql/copy.c
M src/include/port.h
M src/port/path.c
Make BufferIsExclusiveLocked and BufferIsDirty work for local buffers.
commit : f6ff75f79689d2d0c5bb241435d7fc6a63273223
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 29 Jan 2025 13:23:31 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 29 Jan 2025 13:23:31 -0500
These functions tried to check the state of the buffer's content lock
even for local buffers. Since we don't use the content lock for a
local buffer, that would lead to a "false" result from
LWLockHeldByMeInMode, which would mean a misleading "false" answer
from BufferIsExclusiveLocked (we'd rather that case always return
"true") or an assertion failure in BufferIsDirty.
The core code never applies these two functions to local buffers,
and apparently no extensions do either, since we've not heard
complaints. Still, in the name of future-proofing, let's fix
them to act as though a pinned local buffer is content-locked.
Author: Srinath Reddy <srinath2133@gmail.com>
Discussion: https://postgr.es/m/19396ef77f8.1098c4a1810508.2255483659262451647@zohocorp.com
M src/backend/storage/buffer/bufmgr.c
Fix grammatical typos around possessive "its"
commit : 128897b101e0a7bc8621abac746ea99d444d83ae
author : John Naylor <john.naylor@postgresql.org>
date : Wed, 29 Jan 2025 14:28:20 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Wed, 29 Jan 2025 14:28:20 +0700
Some places spelled it "it's", which is short for "it is".
In passing, fix a couple other nearby grammatical errors.
Author: Jacob Brazeal <jacob.brazeal@gmail.com>
Discussion: https://postgr.es/m/CA+COZaAO8g1KJCV0T48=CkJMjAnnfTGLWOATz+2aCh40c2Nm+g@mail.gmail.com
M contrib/amcheck/verify_nbtree.c
M src/backend/access/spgist/spgscan.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/worker.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/lwlock.c
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/adt/tsrank.c
M src/backend/utils/sort/tuplesort.c
M src/bin/pg_combinebackup/reconstruct.c
M src/fe_utils/parallel_slot.c
M src/include/jit/SectionMemoryManager.h
M src/include/nodes/plannodes.h
M src/pl/plpgsql/src/pl_exec.c
M src/test/modules/test_slru/t/001_multixact.pl
M src/test/recovery/t/035_standby_logical_decoding.pl
Revert "Speed up tail processing when hashing aligned C strings, take two"
commit : 235328ee4ae49cd5abf3c1c9b81f3b809507e5f2
author : John Naylor <john.naylor@postgresql.org>
date : Wed, 29 Jan 2025 13:35:43 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Wed, 29 Jan 2025 13:35:43 +0700
This reverts commit a365d9e2e8c1ead27203a4431211098292777d3b.
Older versions of Valgrind raise an error, so go back to the bytewise
loop for the final word in the input.
Reported-by: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Discussion: https://postgr.es/m/a3a959f6-14b8-4819-ac04-eaf2aa2e868d@postgrespro.ru
Backpatch-through: 17
M src/include/common/hashfn_unstable.h
Improve test coverage of network address functions
commit : 4f071349c0c28543fa8b84a5ce0b2f619c089ace
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 29 Jan 2025 08:49:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 29 Jan 2025 08:49:48 +0900
The following functions were not covered by any tests:
- abbrev(inet)
- set_masklen(cidr)
- set_masklen(inet)
- netmask(inet)
- hostmask(inet)
While on it, this improves the output of some of the existing queries in
the test inet to use better aliases.
Author: Aleksander Alekseev
Reviewed-by: Jacob Champion, Keisuke Kuroda, Tom Lane
Discussion: https://postgr.es/m/CAJ7c6TOyZ9bGNrDK6Z3Q0gr9ow8ZpOm+=+01mpE0dsdH4C+u9A@mail.gmail.com
M src/test/regress/expected/inet.out
M src/test/regress/sql/inet.sql
Rename pubgencols_type to pubgencols in pg_publication.
commit : 75eb9766ec201b62f264554019757716093e2a2f
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 28 Jan 2025 10:42:46 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 28 Jan 2025 10:42:46 +0530
The column added in commit e65dbc9927, pubgencols_type, was inconsistent
with the naming conventions of other columns in the pg_publication
catalog.
Author: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Discussion: https://postgr.es/m/CALDaNm1u-ufVOW-RUsXSooqzkpohxfZYy=z78fbcr_9Pq5hbCg@mail.gmail.com
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/pg_publication.c
M src/backend/commands/publicationcmds.c
M src/backend/utils/cache/relcache.c
M src/bin/pg_dump/pg_dump.c
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_publication.h
Track per-relation cumulative time spent in [auto]vacuum and [auto]analyze
commit : 30a6ed0ce4bb18212ec38cdb537ea4b43bc99b83
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 28 Jan 2025 09:57:32 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 28 Jan 2025 09:57:32 +0900
This commit adds four fields to the statistics of relations, aggregating
the amount of time spent for each operation on a relation:
- total_vacuum_time, for manual vacuum.
- total_autovacuum_time, for vacuum done by the autovacuum daemon.
- total_analyze_time, for manual analyze.
- total_autoanalyze_time, for analyze done by the autovacuum daemon.
This gives users the option to derive the average time spent for these
operations with the help of the related "count" fields.
Bump catalog version (for the catalog changes) and PGSTAT_FILE_FORMAT_ID
(for the additions in PgStat_StatTabEntry).
Author: Sami Imseih
Reviewed-by: Bertrand Drouvot, Michael Paquier
Discussion: https://postgr.es/m/CAA5RZ0uVOGBYmPEeGF2d1B_67tgNjKx_bKDuL+oUftuoz+=Y1g@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/access/heap/vacuumlazy.c
M src/backend/catalog/system_views.sql
M src/backend/commands/analyze.c
M src/backend/utils/activity/pgstat_relation.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
doc: Meson is not experimental on Windows
commit : 5afaba6297a8dd6999c8bc9f517a3ad38bd39652
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 27 Jan 2025 12:02:00 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 27 Jan 2025 12:02:00 +0100
The installation documentation stated that using Meson is
experimental. But since this is the only way to build using Visual
Studio on Windows, this would imply that that whole build procedure is
experimental, which isn't true. So qualify this statement a bit more.
We keep the statement that Meson is experimental on other platforms,
since it doesn't have full, confirmed feature parity with the make
build system.
Author: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/flat/a3e76618-4cb5-4d54-a71c-da4fb8ba571b@eisentraut.org
M doc/src/sgml/installation.sgml
Print out error position for some ALTER TABLE ALTER COLUMN type
commit : 65281391a937293db7fa747be218def0e9794550
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 27 Jan 2025 13:51:23 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 27 Jan 2025 13:51:23 +0900
A ParseState exists in ATPrepAlterColumnType() since its introduction
in 077db40fa1f3, and it has never relied on a query string that could be
used to point at a location in the origin string on error.
The output of some regression tests are updated, showing the error
location where applicable. Six error strings are upgraded with the
error location.
Author: Jian He
Discussion: https://postgr.es/m/CACJufxGfbPfWLjcEz33G9eW_epDW0UDi2H05i9eSTPKGJ4rxSA@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/generated_stored.out
M src/test/regress/expected/typed_table.out
pg_amcheck: Fix test failure on Windows with non-existing role
commit : 14793f47192b5eb02197cd03afb88559d7514b76
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 27 Jan 2025 08:00:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 27 Jan 2025 08:00:03 +0900
For SSPI auth extra users need to be explicitly allowed, or we get
"SSPI authentication failed" instead of the expected "role does not
exist" error.
This report also means that the test has never worked on Windows since
its introduction in 9706092839db, because it has always bumped on an
authentication failure rather than an error about the role not existing.
Oversight in eef4a33f62f7, that has added a pattern check on the error
generated by the command.
Per report from Tom Lane, via buildfarm member drongo.
Author: Dagfinn Ilmari Mannsåker
Reviewed-by: Andrew Dunstan
Discussion: https://postgr.es/m/379085.1737734611@sss.pgh.pa.us
M src/bin/pg_amcheck/t/002_nonesuch.pl
Test postmaster with program_options_handling_ok() et al.
commit : 2f12df7eb46d5eb428a77a8aa5dc951011418935
author : Noah Misch <noah@leadboat.com>
date : Sun, 26 Jan 2025 09:39:05 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sun, 26 Jan 2025 09:39:05 -0800
Most executables already get that testing. To occupy the customary
001_basic.pl name, this renumbers the new-in-October tests of
src/test/postmaster/t.
Reviewed by Thomas Munro.
Discussion: https://postgr.es/m/20241215022701.a1.nmisch@google.com
M src/test/postmaster/meson.build
A src/test/postmaster/t/001_basic.pl
R100 src/test/postmaster/t/001_connection_limits.pl src/test/postmaster/t/002_connection_limits.pl
R100 src/test/postmaster/t/002_start_stop.pl src/test/postmaster/t/003_start_stop.pl
Add missing CommandCounterIncrement
commit : 0a16c8326c5a14abd180eeefe5e5ee4263513c2a
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sun, 26 Jan 2025 17:34:28 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sun, 26 Jan 2025 17:34:28 +0100
For commit b663b9436e75 I thought this was useless, but turns out not to
be for the case where a partitioned table has two identical foreign key
constraints which can both be matched by the same constraint in a
partition during attach. This CCI makes the match search for the second
constraint in the parent ignore the constraint in the child that has
already been matched by the first constraint in the parent.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/c599253c-1ccd-4161-80fc-c9065e037a09@gmail.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
At update of non-LP_NORMAL TID, fail instead of corrupting page header.
commit : d28cd3e7b21c41a710fd9c188a57dad99f28805f
author : Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
The right mix of DDL and VACUUM could corrupt a catalog page header such
that PageIsVerified() durably fails, requiring a restore from backup.
This affects only catalogs that both have a syscache and have DDL code
that uses syscache tuples to construct updates. One of the test
permutations shows a variant not yet fixed.
This makes !TransactionIdIsValid(TM_FailureData.xmax) possible with
TM_Deleted. I think core and PGXN are indifferent to that.
Per bug #17821 from Alexander Lakhin. Back-patch to v13 (all supported
versions). The test case is v17+, since it uses INJECTION_POINT.
Discussion: https://postgr.es/m/17821-dd8c334263399284@postgresql.org
M src/backend/access/heap/heapam.c
M src/backend/utils/cache/inval.c
M src/include/access/tableam.h
M src/test/modules/injection_points/Makefile
A src/test/modules/injection_points/expected/syscache-update-pruned.out
A src/test/modules/injection_points/expected/syscache-update-pruned_1.out
M src/test/modules/injection_points/injection_points–1.0.sql
M src/test/modules/injection_points/meson.build
A src/test/modules/injection_points/regress_injection.c
A src/test/modules/injection_points/specs/syscache-update-pruned.spec
Merge copies of converting an XID to a FullTransactionId.
commit : 81772a495ec98d36eabf5cc294e7031a9545c5c1
author : Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
Assume twophase.c is the performance-sensitive caller, and preserve its
choice of unlikely() branch hint. Add some retrospective rationale for
that choice. Back-patch to v17, for the next commit to use it.
Reviewed (in earlier versions) by Michael Paquier.
Discussion: https://postgr.es/m/17821-dd8c334263399284@postgresql.org
Discussion: https://postgr.es/m/20250116010051.f3.nmisch@google.com
M contrib/amcheck/verify_heapam.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xlogreader.c
M src/backend/utils/adt/xid8funcs.c
M src/include/access/transam.h
Disable runningcheck for src/test/modules/injection_points/specs.
commit : 4f6ec3831d3e6a237eb6197451472e70282654db
author : Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
Directory "injection_points" has specified NO_INSTALLCHECK since before
commit c35f419d6efbdf1a050250d84b687e6705917711 added the specs, but
that commit neglected to disable the corresponding meson runningcheck.
The alternative would be to enable "make installcheck" for ISOLATION,
but the GNU make build system lacks a concept of setting NO_INSTALLCHECK
for REGRESS without also setting it for ISOLATION. Back-patch to v17,
where that commit first appeared, to avoid surprises when back-patching
additional specs.
Discussion: https://postgr.es/m/17821-dd8c334263399284@postgresql.org
M src/test/modules/injection_points/meson.build
Test ECPG decadd(), decdiv(), decmul(), and decsub() for risnull() input.
commit : 7819a25cd101b574f5422edb00fe3376fbb646da
author : Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
Since commit 757fb0e5a9a61ac8d3a67e334faeea6dc0084b3f, these
Informix-compat functions return 0 without changing the output
parameter. Initialize the output parameter before the test call, making
that obvious. Before this, the expected test output has been depending
on freed stack memory. "gcc -ftrivial-auto-var-init=pattern" revealed
that. Back-patch to v13 (all supported versions).
Discussion: https://postgr.es/m/20250106192748.cf.nmisch@google.com
M src/interfaces/ecpg/test/compat_informix/dec_test.pgc
M src/interfaces/ecpg/test/expected/compat_informix-dec_test.c
M src/interfaces/ecpg/test/expected/compat_informix-dec_test.stdout
Doc: recommend "psql -X" for restoring pg_dump scripts.
commit : d83a108c10a3ec886a24c620a915aa2c5bc023aa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Jan 2025 12:42:05 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Jan 2025 12:42:05 -0500
This practice avoids possible problems caused by non-default psql
options, such as disabling AUTOCOMMIT.
Author: Shinya Kato <Shinya11.Kato@oss.nttdata.com>
Reviewed-by: Robert Treat <rob@xzilla.net>
Discussion: https://postgr.es/m/96ff23a5d858ff72ca8e823a014d16fe@oss.nttdata.com
Backpatch-through: 13
M doc/src/sgml/backup.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
Change shutdown sequence to terminate checkpointer last
commit : 87a6690cc69530703b7da7e72769bae2ac5b2e77
author : Andres Freund <andres@anarazel.de>
date : Sat, 25 Jan 2025 11:37:13 -0500
committer: Andres Freund <andres@anarazel.de>
date : Sat, 25 Jan 2025 11:37:13 -0500
The main motivation for this change is to have a process that can serialize
stats after all other processes have terminated. Serializing stats already
happens in checkpointer, even though walsenders can be active longer.
The only reason the current shutdown sequence does not actively cause problems
is that walsender currently does not generate any stats. However, there is an
upcoming patch changing that.
Another need for this change originates in the AIO patchset, where IO
workers (which, in some edge cases, can emit stats of their own) need to run
while the shutdown checkpoint is being written.
This commit changes the shutdown sequence so checkpointer is signalled (via
SIGINT) to trigger writing the shutdown checkpoint without also causing
checkpointer to exit. Once checkpointer wrote the shutdown checkpoint it
notifies postmaster via PMSIGNAL_XLOG_IS_SHUTDOWN and waits for the
termination signal (SIGUSR2, as before). Checkpointer now is terminated after
all children, other than dead-end children and logger, have been terminated,
tracked using the new PM_WAIT_CHECKPOINTER PMState.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/kgng5nrvnlv335evmsuvpnh354rw7qyazl73kdysev2cr2v5zu@m3cfzxicm5kp
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/postmaster.c
M src/backend/utils/activity/wait_event_names.txt
M src/include/storage/pmsignal.h
Tighten pg_restore's recognition of its -F (format) option values.
commit : 04ace176e08f2c694bb66b5b91cbd9d4d0bd77ea
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Jan 2025 11:24:16 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Jan 2025 11:24:16 -0500
Instead of checking just the first letter, match the whole string
using pg_strcasecmp. Per the documentation, we allow either just
the first letter (e.g. "c") or the whole name ("custom"); but we
will no longer accept random variations such as "chump". This
matches pg_dump's longstanding parsing code for the same option.
Also for consistency with pg_dump, recognize "p"/"plain". We don't
support it, but we can give a more helpful error message than
"unrecognized archive format".
Author: Srinath Reddy <srinath2133@gmail.com>
Discussion: https://postgr.es/m/CAFC+b6pfK-BGcWW1kQmtxVrCh-JGjB2X02rLPQs_ZFaDGjZDsQ@mail.gmail.com
M src/bin/pg_dump/pg_restore.c
Fix PDF doc build.
commit : d2ca16bb509c453eac181c1ef805ff55289df779
author : Jeff Davis <jdavis@postgresql.org>
date : Sat, 25 Jan 2025 00:12:30 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Sat, 25 Jan 2025 00:12:30 -0800
Reported-by: Tom Lane
Discussion: https://postgr.es/m/608525.1737781222@sss.pgh.pa.us
M doc/src/sgml/func.sgml
Use the correct sizeof() in BufFileLoadBuffer
commit : 38273b5f831bc7651fc479eacf3644e96a551af8
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 25 Jan 2025 00:36:48 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 25 Jan 2025 00:36:48 +0100
The sizeof() call should reference buffer.data, because that's the
buffer we're reading data into, not the whole PGAlignedBuffer union.
This was introduced by 44cac93464, which replaced the simple buffer
with a PGAlignedBuffer field.
It's benign, because the buffer is the largest field of the union, so
the sizes are the same. But it's easy to trip over this in a patch, so
fix and backpatch. Commit 44cac93464 went into 12, but that's EOL.
Backpatch-through: 13
Discussion: https://postgr.es/m/928bdab1-6567-449f-98c4-339cd2203b87@vondra.me
M src/backend/storage/file/buffile.c
Add SQL function CASEFOLD().
commit : bfc5992069cf00b189af83d96a83ae5ebb65e938
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 24 Jan 2025 14:56:22 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 24 Jan 2025 14:56:22 -0800
Useful for caseless matching. Similar to LOWER(), but avoids edge-case
problems with using LOWER() for caseless matching.
For collations that support it, CASEFOLD() handles characters with
more than two case variations or multi-character case variations. Some
characters may fold to uppercase. The results of case folding are also
more stable across Unicode versions than LOWER() or UPPER().
Discussion: https://postgr.es/m/a1886ddfcd8f60cb3e905c93009b646b4cfb74c5.camel%40j-davis.com
Reviewed-by: Ian Lawrence Barwick
M doc/src/sgml/func.sgml
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/oracle_compat.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_locale_builtin.c
M src/backend/utils/adt/pg_locale_icu.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/utils/formatting.h
M src/include/utils/pg_locale.h
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/expected/collate.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
M src/test/regress/sql/collate.utf8.sql
postmaster: Adjust which processes we expect to have exited
commit : f15538cd27d4eeb7d665263a3d7b5700362d7eb0
author : Andres Freund <andres@anarazel.de>
date : Fri, 24 Jan 2025 17:00:10 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 24 Jan 2025 17:00:10 -0500
Comments and code stated that we expect checkpointer to have been signalled in
case of immediate shutdown / fatal errors, but didn't treat archiver and
walsenders the same. That doesn't seem right.
I had started digging through the history to see where this oddity was
introduced, but it's not the fault of a single commit.
Instead treat archiver, checkpointer, and walsenders the same.
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/kgng5nrvnlv335evmsuvpnh354rw7qyazl73kdysev2cr2v5zu@m3cfzxicm5kp
M src/backend/postmaster/postmaster.c
postmaster: Commonalize FatalError paths
commit : 463a2ebd9fda4fa94833838d0a372f7fd53b5b8a
author : Andres Freund <andres@anarazel.de>
date : Fri, 24 Jan 2025 17:08:31 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 24 Jan 2025 17:08:31 -0500
This includes some behavioral changes:
- Previously PM_WAIT_XLOG_ARCHIVAL wasn't handled in HandleFatalError(), that
doesn't seem quite right.
- Previously a fatal error in PM_WAIT_XLOG_SHUTDOWN lead to jumping back to
PM_WAIT_BACKENDS, no we go to PM_WAIT_DEAD_END. Jumping backwards doesn't
seem quite right and we didn't do so when checkpointer failed to fork during
a shutdown.
- Previously a checkpointer fork failure didn't call SetQuitSignalReason(),
which would lead to quickdie() reporting
"terminating connection because of unexpected SIGQUIT signal"
which seems even worse than the PMQUIT_FOR_CRASH message. If I saw that in
the log I'd suspect somebody outside of postgres sent SIGQUITs
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/kgng5nrvnlv335evmsuvpnh354rw7qyazl73kdysev2cr2v5zu@m3cfzxicm5kp
M src/backend/postmaster/postmaster.c
postmaster: Move code to switch into FatalError state into function
commit : 8edd8c77c88e75822334ccb8376d2c151d6e5615
author : Andres Freund <andres@anarazel.de>
date : Fri, 24 Jan 2025 17:00:10 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 24 Jan 2025 17:00:10 -0500
There are two places switching to FatalError mode, behaving somewhat
differently. An upcoming commit will introduce a third. That doesn't seem seem
like a good idea.
This commit just moves the FatalError related code from HandleChildCrash()
into its own function, a subsequent commit will evolve the state machine
change to be suitable for other callers.
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/kgng5nrvnlv335evmsuvpnh354rw7qyazl73kdysev2cr2v5zu@m3cfzxicm5kp
M src/backend/postmaster/postmaster.c
postmaster: Don't repeatedly transition to crashing state
commit : f0b7ab7251390544c5e79d1d9c96da254181fc4f
author : Andres Freund <andres@anarazel.de>
date : Fri, 24 Jan 2025 17:00:10 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 24 Jan 2025 17:00:10 -0500
Previously HandleChildCrash() skipped logging and signalling child exits if
already in an immediate shutdown or in FatalError state, but still
transitioned server state in response to a crash. That's redundant.
In the other place we transition to FatalError, we do take care to not do so
when already in FatalError state.
To make it easier to combine different paths for entering FatalError state,
only do so once in HandleChildCrash().
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/kgng5nrvnlv335evmsuvpnh354rw7qyazl73kdysev2cr2v5zu@m3cfzxicm5kp
M src/backend/postmaster/postmaster.c
postmaster: Don't open-code TerminateChildren() in HandleChildCrash()
commit : d239c1a8e5b6ac467b3479bf3840e3d297a40bef
author : Andres Freund <andres@anarazel.de>
date : Fri, 24 Jan 2025 17:00:10 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 24 Jan 2025 17:00:10 -0500
After removing the duplication no user of sigquit_child() remains, therefore
remove it.
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/kgng5nrvnlv335evmsuvpnh354rw7qyazl73kdysev2cr2v5zu@m3cfzxicm5kp
M src/backend/postmaster/postmaster.c
checkpointer: Request checkpoint via latch instead of signal
commit : 4d271e3ec25138be360b5a6ee40ec8bfa12459e0
author : Andres Freund <andres@anarazel.de>
date : Fri, 24 Jan 2025 17:00:10 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 24 Jan 2025 17:00:10 -0500
The motivation for this change is that a future commit will use SIGINT for
another purpose (postmaster requesting WAL access to be shut down) and that
there no other signals that we could readily use (see code comment for the
reason why SIGTERM shouldn't be used). But it's also a tad nicer / more
efficient to use SetLatch(), as it avoids sending signals when checkpointer
already is busy.
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/kgng5nrvnlv335evmsuvpnh354rw7qyazl73kdysev2cr2v5zu@m3cfzxicm5kp
M src/backend/postmaster/checkpointer.c
Make jsonb casts to scalar types translate JSON null to SQL NULL.
commit : a5579a90af05814eb5dc2fd5f68ce803899d2504
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 24 Jan 2025 13:20:44 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 24 Jan 2025 13:20:44 -0500
Formerly, these cases threw an error "cannot cast jsonb null to type
<whatever>". That seems less than helpful though. It's also
inconsistent with the behavior of the ->> operator, which translates
JSON null to SQL NULL, as do some other jsonb functions.
Discussion: https://postgr.es/m/3851203.1722552717@sss.pgh.pa.us
M src/backend/utils/adt/jsonb.c
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql
Fix copy-and-paste typo
commit : 13a255c195c9911d6b66179f5c2344597dc47155
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 24 Jan 2025 17:45:29 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 24 Jan 2025 17:45:29 +0100
M contrib/bloom/blvalidate.c
pgcrypto: Make it possible to disable built-in crypto
commit : 035f99cbebe5ffcaf52f8370394446cd59621ab7
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 24 Jan 2025 14:25:08 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 24 Jan 2025 14:25:08 +0100
When using OpenSSL and/or the underlying operating system in FIPS
mode no non-FIPS certified crypto implementations should be used.
While that is already possible by just not invoking the built-in
crypto in pgcrypto, this adds a GUC which prohibit the code from
being called. This doesn't change the FIPS status of PostgreSQL
but can make it easier for sites which target FIPS compliance to
ensure that violations cannot occur.
Author: Daniel Gustafsson <daniel@yesql.se>
Author: Joe Conway <mail@joeconway.com>
Reviewed-by: Joe Conway <mail@joeconway.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Discussion: https://postgr.es/m/16b4a157-9ea1-44d0-b7b3-4c85df5de97b@joeconway.com
M contrib/pgcrypto/expected/crypt-des.out
M contrib/pgcrypto/openssl.c
M contrib/pgcrypto/pgcrypto.c
M contrib/pgcrypto/px-crypt.c
M contrib/pgcrypto/px.h
M contrib/pgcrypto/sql/crypt-des.sql
M doc/src/sgml/pgcrypto.sgml
pgcrypto: Add function to check FIPS mode
commit : 924d89a354750976cdd271d1dfc6c1e97cbb8851
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 24 Jan 2025 14:18:40 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 24 Jan 2025 14:18:40 +0100
This adds a SQL callable function for reading and returning the status
of FIPS configuration of OpenSSL. If OpenSSL is operating with FIPS
enabled it will return true, otherwise false. As this adds a function
to the SQL file, bump the extension version to 1.4.
Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Joe Conway <mail@joeconway.com>
Discussion: https://postgr.es/m/8f979145-e206-475a-a31b-73c977a4134c@joeconway.com
M contrib/pgcrypto/Makefile
M contrib/pgcrypto/meson.build
M contrib/pgcrypto/openssl.c
A contrib/pgcrypto/pgcrypto–1.3–1.4.sql
M contrib/pgcrypto/pgcrypto.c
M contrib/pgcrypto/pgcrypto.control
M contrib/pgcrypto/px.h
M doc/src/sgml/pgcrypto.sgml
Fix instability in recently added regression tests
commit : c44c2d2759876463dcbab2eb608e33ed2e772d12
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 24 Jan 2025 12:54:46 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 24 Jan 2025 12:54:46 +0100
We missed the usual ORDER BY clause.
Author: Amul Sul <amul.sul@enterprisedb.com>
Discussion: https://postgr.es/m/CAAJ_b974U3Vvf-qGwFyZ73DFHqyFJP9TOmuiXR2Kp8KVcJtP6w@mail.gmail.com
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
Convert sepgsql tests to TAP
commit : aeb8ea361a0a321a0e1cbc79a4cd3ec0b1191bf2
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 24 Jan 2025 11:37:20 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 24 Jan 2025 11:37:20 +0100
Add a TAP test for sepgsql. This automates the previously required
manual setup before the test. The actual tests are still run by
pg_regress, as before, but now called from within the TAP Perl script.
The previous manual test script (test_sepgsql) is left in place, since
its purpose is (also) to test whether a running instance was properly
initialized for sepgsql. But it has been changed to call pg_regress
directly and no longer require make.
Reviewed-by: Andreas Karlsson <andreas@proxel.se>
Discussion: https://www.postgresql.org/message-id/flat/651a5baf-5c45-4a5a-a202-0c8453a4ebf8@eisentraut.org
M contrib/sepgsql/.gitignore
M contrib/sepgsql/Makefile
M contrib/sepgsql/meson.build
A contrib/sepgsql/t/001_sepgsql.pl
M contrib/sepgsql/test_sepgsql
M doc/src/sgml/regress.sgml
M doc/src/sgml/sepgsql.sgml
M meson.build
M src/Makefile.global.in
meson: Fix sepgsql installation
commit : 02ed3c2bdcefab453b548bc9c7e0e8874a502790
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 24 Jan 2025 10:26:12 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 24 Jan 2025 10:26:12 +0100
The sepgsql.sql file should be installed under share/contrib/, not
share/extension/, since it is not an extension. This makes it match
what make install does.
Discussion: https://www.postgresql.org/message-id/flat/651a5baf-5c45-4a5a-a202-0c8453a4ebf8@eisentraut.org
M contrib/sepgsql/meson.build
initdb: Convert tests to use long options with fat comma style
commit : fd4c4ede708fc5affb04362bb5a8bdc3136fa963
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 24 Jan 2025 15:19:38 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 24 Jan 2025 15:19:38 +0900
This is similar to ce1b0f9da03e, but this time this rule is applied to
some of the TAP tests of initdb.
Author: Dagfinn Ilmari Mannsåker
Discussion: https://postgr.es/m/878qr146ra.fsf@wibble.ilmari.org
M src/bin/initdb/t/001_initdb.pl
Return yyparse() result not via global variable
commit : 473a575e05979b4dbb28b3f2544f4ec8f184ce65
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 24 Jan 2025 06:55:39 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 24 Jan 2025 06:55:39 +0100
Instead of passing the parse result from yyparse() via a global
variable, pass it via a function output argument.
This complements earlier work to make the parsers reentrant.
Discussion: Discussion: https://www.postgresql.org/message-id/flat/eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org
M src/backend/nls.mk
M src/backend/replication/repl_gram.y
M src/backend/replication/repl_scanner.l
M src/backend/replication/syncrep.c
M src/backend/replication/syncrep_gram.y
M src/backend/replication/syncrep_scanner.l
M src/backend/replication/walsender.c
M src/bin/pgbench/exprparse.y
M src/bin/pgbench/exprscan.l
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/pgbench.h
M src/include/replication/syncrep.h
M src/include/replication/walsender_private.h
M src/pl/plpgsql/src/nls.mk
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/pl_scanner.c
M src/pl/plpgsql/src/plpgsql.h
Doc: Fix a typo introduced in 4a0e7314f1.
commit : 6fc4fc42da07c47a9b8d014ab18d005cf8b3e176
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 24 Jan 2025 08:25:21 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 24 Jan 2025 08:25:21 +0530
Author: Erik Rijkers <er@xs4all.nl>
Discussion: https://postgr.es/m/6e625c81-968e-42d0-802d-edfaf9cfac11@xs4all.nl
M doc/src/sgml/logical-replication.sgml
Doc: Fix column name in pg_publication catalog.
commit : 117f9f328e0f146a4b6de517501f4d5d994040a9
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 24 Jan 2025 08:11:29 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 24 Jan 2025 08:11:29 +0530
Commit e65dbc9927 incorrectly spelled the column name in the
pg_publication catalog. In passing make the order of columns in the doc
match the actual catalog.
Author: Shinoda, Noriyoshi <noriyoshi.shinoda@hpe.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Discussion: https://postgr.es/m/DM4PR84MB1734F8F140E4477580761F93EEE02@DM4PR84MB1734.NAMPRD84.PROD.OUTLOOK.COM
M doc/src/sgml/catalogs.sgml
Don't ask for bug reports about pthread_is_threaded_np() != 0.
commit : 4f15759bdcddd23e874526a6b2c0ff86e0beb042
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 23 Jan 2025 14:23:04 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 23 Jan 2025 14:23:04 -0500
We thought that this condition was unreachable in ExitPostmaster,
but actually it's possible if you have both a misconfigured locale
setting and some other mistake that causes PostmasterMain to bail
out before reaching its own check of pthread_is_threaded_np().
Given the lack of other reports, let's not ask for bug reports if
this occurs; instead just give the same hint as in PostmasterMain.
Bug: #18783
Reported-by: anani191181515@gmail.com
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/18783-d1873b95a59b9103@postgresql.org
Discussion: https://postgr.es/m/206317.1737656533@sss.pgh.pa.us
Backpatch-through: 13
M src/backend/postmaster/postmaster.c
Ensure that AFTER triggers run as the instigating user.
commit : 01463e1cccd33fb11b33a4dd6dbebcad3c534102
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 23 Jan 2025 12:25:45 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 23 Jan 2025 12:25:45 -0500
With deferred triggers, it is possible that the current role changes
between the time when the trigger is queued and the time it is
executed (for example, the triggering data modification could have
been executed in a SECURITY DEFINER function).
Up to now, deferred trigger functions would run with the current role
set to whatever was active at commit time. That does not matter for
foreign-key constraints, whose correctness doesn't depend on the
current role. But for user-written triggers, the current role
certainly can matter.
Hence, fix things so that AFTER triggers are fired under the role
that was active when they were queued, matching the behavior of
BEFORE triggers which would have actually fired at that time.
(If the trigger function is marked SECURITY DEFINER, that of course
overrides this, as it always has.)
This does not create any new security exposure: if you do DML on a
table owned by a hostile user, that user has always had various ways
to exploit your permissions, such as the aforementioned BEFORE
triggers, default expressions, etc. It might remove some security
exposure, because the old behavior could potentially expose some
other role besides the one directly modifying the table.
There was discussion of making a larger change, such as running as
the trigger's owner. However, that would break the common idiom of
capturing the value of CURRENT_USER in a trigger for auditing/logging
purposes. This change will make no difference in the typical scenario
where the current role doesn't change before commit.
Arguably this is a bug fix, but it seems too big a semantic change
to consider for back-patching.
Author: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Joseph Koshakow <koshy44@gmail.com>
Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://postgr.es/m/77ee784cf248e842f74588418f55c2931e47bd78.camel@cybertec.at
M doc/src/sgml/trigger.sgml
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Add support for Unicode case folding.
commit : 4e7f62bc386a479593e4e8ecfb94370f5a88e522
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 23 Jan 2025 09:06:50 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 23 Jan 2025 09:06:50 -0800
Expand case mapping tables to include entries for case folding, which
are parsed from CaseFolding.txt.
Discussion: https://postgr.es/m/a1886ddfcd8f60cb3e905c93009b646b4cfb74c5.camel%40j-davis.com
M src/common/unicode/Makefile
M src/common/unicode/case_test.c
M src/common/unicode/generate-unicode_case_table.pl
M src/common/unicode/meson.build
M src/common/unicode_case.c
M src/include/common/unicode_case.h
M src/include/common/unicode_case_table.h
Reverse the search order in afterTriggerAddEvent().
commit : 7921927bbb9d4a80ced9283b27c26eedb638f555
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 23 Jan 2025 11:08:05 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 23 Jan 2025 11:08:05 -0500
When scanning existing AfterTriggerSharedData records in search
of a match to the event being queued, we were examining the
records from oldest to newest. But it makes more sense to do
the opposite. The newest record is likely to be from the current
query, while the oldest is likely to be from some previous command
in the same transaction, which will likely have different details.
There aren't expected to be very many active AfterTriggerSharedData
records at once, so that this change is unlikely to make any
spectacular difference. Still, having added a nontrivially-expensive
bms_equal call to this loop yesterday, I feel a need to shave cycles
where possible.
Discussion: https://postgr.es/m/4166712.1737583961@sss.pgh.pa.us
M src/backend/commands/trigger.c
Allow NOT VALID foreign key constraints on partitioned tables
commit : b663b9436e7509b5e73c8c372539f067cd6e66c1
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 23 Jan 2025 15:54:38 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 23 Jan 2025 15:54:38 +0100
This feature was intentionally omitted when FKs were first implemented
for partitioned tables, and had been requested a few times; the
usefulness is clear.
Validation can happen for each partition individually, which is useful
to contain the number of locks held and the duration; or it can be
executed for the partitioning hierarchy as a single command, which
validates all child constraints that haven't been validated already.
This is also useful to implement NOT ENFORCED constraints on top.
Author: Amul Sul <sulamul@gmail.com>
Discussion: https://postgr.es/m/CAAJ_b96Bp=-ZwihPPtuaNX=SrZ0U6ZsXD3+fgARO0JuKa8v2jQ@mail.gmail.com
M doc/src/sgml/ref/alter_table.sgml
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
Fix buildfarm failure introduced by commit e65dbc9927.
commit : b35434b134b1e1a306649a8c63c5c46afdcf7b1e
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 23 Jan 2025 17:47:15 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 23 Jan 2025 17:47:15 +0530
The patch had incorrectly specified the default value for
publish_generated_columns during the query formation in pg_dump.
Author: Vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/CAA4eK1KfZYTD8Hpi9TD1KaB8rNUBR9baUvTxa5wYyZDGbEaa6g@mail.gmail.com
M src/bin/pg_dump/pg_dump.c
Convert macros to static inline functions (htup_details.h, itup.h)
commit : 34694ec888d6783c3222134305a86e533b43071d
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 23 Jan 2025 12:07:38 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 23 Jan 2025 12:07:38 +0100
Discussion: https://www.postgresql.org/message-id/flat/5b558da8-99fb-0a99-83dd-f72f05388517@enterprisedb.com
M contrib/pageinspect/heapfuncs.c
M src/include/access/htup_details.h
M src/include/access/itup.h
Add some const decorations (htup.h)
commit : b15b8c5cf882e725c4f4f4628ac6f4d054065b4c
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 23 Jan 2025 12:07:38 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 23 Jan 2025 12:07:38 +0100
Discussion: https://www.postgresql.org/message-id/flat/5b558da8-99fb-0a99-83dd-f72f05388517@enterprisedb.com
M src/backend/access/heap/heapam.c
M src/backend/utils/time/combocid.c
M src/include/access/htup.h
Change publication's publish_generated_columns option type to enum.
commit : e65dbc9927fb86aa3c8a914ede6a6ae934384f5a
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 23 Jan 2025 15:28:37 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 23 Jan 2025 15:28:37 +0530
The current boolean publish_generated_columns option only supports a
binary choice, which is insufficient for future enhancements where
generated columns can be of different types (e.g., stored or virtual). The
supported values for the publish_generated_columns option are 'none' and
'stored'.
Author: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/d718d219-dd47-4a33-bb97-56e8fc4da994@eisentraut.org
Discussion: https://postgr.es/m/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E@gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/create_publication.sgml
M src/backend/catalog/pg_publication.c
M src/backend/commands/publicationcmds.c
M src/backend/replication/logical/proto.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/utils/cache/relcache.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_publication.h
M src/include/commands/publicationcmds.h
M src/include/replication/logicalproto.h
M src/test/regress/expected/publication.out
M src/test/regress/sql/publication.sql
M src/test/subscription/t/011_generated.pl
M src/tools/pgindent/typedefs.list
Add error pattern checks for some TAP tests for non-existing objects
commit : eef4a33f62f7300765b5ffa8c966fa2fba50d176
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 23 Jan 2025 16:03:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 23 Jan 2025 16:03:48 +0900
Some tests are updated to use command_fails_like(), gaining a check for
the error output generated. The test changed in pg_amcheck has come up
after noticing that an incorrect option name still made the test to
pass, while the command failed. The three other tests changed in
src/bin/scripts/ have been noticed by me, in passing.
Author: Dagfinn Ilmari Mannsåker, Michael Paquier
Discussion: https://postgr.es/m/87bjvy50cs.fsf@wibble.ilmari.org
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/scripts/t/010_clusterdb.pl
M src/bin/scripts/t/050_dropdb.pl
M src/bin/scripts/t/070_dropuser.pl
Improve TAP tests of pg_basebackup
commit : 858b4db378f2ba1dacd078b3b052027c3518fd13
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 23 Jan 2025 15:15:36 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 23 Jan 2025 15:15:36 +0900
This addresses some minor issues with the TAP tests of pg_basebackup:
- Remove three duplicated tests used for incorrect option combinations.
- Add more pattern checks for commands doomed to fail, to make sure that
the error generated is the expected one. These are for tests related to
the tablespace mapping and incorrect option combinations.
- Fix the description of one test for the case of backup target versus
format.
Issues noticed while reviewing this area of the tests.
Discussion: https://postgr.es/m/87bjvy50cs.fsf@wibble.ilmari.org
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
Support RN (roman-numeral format) in to_number().
commit : 172e6b3adb2e2749883ad0746172e4cf97139961
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 22 Jan 2025 15:18:40 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 22 Jan 2025 15:18:40 -0500
We've long had roman-numeral output support in to_char(),
but lacked the reverse conversion. Here it is.
Author: Hunaid Sohail <hunaidpgml@gmail.com>
Reviewed-by: Maciek Sakrejda <m.sakrejda@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CAMWA6ybh4M1VQqpmnu2tfSwO+3gAPeA8YKnMHVADeB=XDEvT_A@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/formatting.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Fix comment about AVX-512 popcount support.
commit : f0ee648527e54c465e398d993db892af61a228da
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 22 Jan 2025 14:11:37 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 22 Jan 2025 14:11:37 -0600
Since commit f78667bd91, we've used __attribute__((target(...)))
instead of extra compiler flags for AVX-512 support, but this
comment still says that we put the code in a separate file because
it might require extra compiler flags. Let's just remove that part
of the comment.
M src/include/port/pg_bitutils.h
Repair incorrect handling of AfterTriggerSharedData.ats_modifiedcols.
commit : ea68ea6320ff84f55cf30dff1af662fc0bf5dafa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 22 Jan 2025 11:58:20 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 22 Jan 2025 11:58:20 -0500
This patch fixes two distinct errors that both ultimately trace
to commit 71d60e2aa, which added the ats_modifiedcols field.
The more severe error is that ats_modifiedcols wasn't accounted for
in afterTriggerAddEvent's scanning loop that looks for a pre-existing
duplicate AfterTriggerSharedData. Thus, a new event could be
incorrectly matched to an AfterTriggerSharedData that has a different
value of ats_modifiedcols, resulting in the wrong tg_updatedcols
bitmap getting passed to the trigger whenever it finally gets fired.
We'd not noticed because (a) few triggers consult tg_updatedcols,
and (b) we had no tests exercising a case where such a trigger was
called as an AFTER trigger. In the test case added by this commit,
contrib/lo's trigger fails to remove a large object when expected
because (without this fix) it thinks the LO OID column hasn't changed.
The other problem was introduced by commit ce5aaea8c, which copied the
modified-columns bitmap into trigger-related storage. It made a copy
for every trigger event, whereas what we really want is to make a new
copy only when we make a new AfterTriggerSharedData entry. (We could
imagine adding extra logic to reduce the number of bitmap copies still
more, but it doesn't look worthwhile at the moment.) In a simple test
of an UPDATE of 10000000 rows with a single AFTER trigger, this thinko
roughly tripled the amount of memory consumed by the pending-triggers
data structures, from 160446744 to 480443440 bytes.
Fixing the first problem requires introducing a bms_equal() call into
afterTriggerAddEvent's scanning loop, which is slightly annoying from
a speed perspective. However, getting rid of the excessive bms_copy()
calls from the second problem balances that out; overall speed of
trigger operations is the same or slightly better, in my tests.
Discussion: https://postgr.es/m/3496294.1737501591@sss.pgh.pa.us
Backpatch-through: 13
M contrib/lo/expected/lo.out
M contrib/lo/sql/lo.sql
M src/backend/commands/trigger.c
Fix \dRp+ output when describing publications with a lower server version.
commit : 991974bb48886201948cd8d3f4ea7bce2c6bda4b
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 22 Jan 2025 15:27:37 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 22 Jan 2025 15:27:37 +0530
The psql was not careful that the new column "Generated columns" won't be
present in the lower version. This was introduced in recent commit
7054186c4e.
Author: Vignesh C
Reviewed-by: Peter Smith
Discussion: https://postgr.es/m/CALDaNm3OcXdY0EzDEKAfaK9gq2B67Mfsgxu93+_249ohyts=0g@mail.gmail.com
M src/bin/psql/describe.c
Additional tests for stored generated columns
commit : 41084409f635453efce03f1114880189b4f6ce4c
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 22 Jan 2025 07:32:21 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 22 Jan 2025 07:32:21 +0100
Some additional tests have been created during the development of
virtual generated columns (not included here). This commit adds
equivalent tests to the existing test set for stored generated
columns. This includes expanded tests related to MERGE, subqueries,
whole-row references, permissions, domains, partitioning, and
triggers.
Author: Peter Eisentraut <peter@eisentraut.org>
Co-authored-by: jian he <jian.universality@gmail.com>
Co-authored-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/a368248e-69e4-40be-9c07-6c3b5880b0a6@eisentraut.org
M src/test/regress/expected/generated_stored.out
M src/test/regress/sql/generated_stored.sql
Improve grammar of options for command arrays in TAP tests
commit : ce1b0f9da03e1cebc293f60b378079b22bd7cc0f
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Jan 2025 14:47:13 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Jan 2025 14:47:13 +0900
This commit rewrites a good chunk of the command arrays in TAP tests
with a grammar based on the following rules:
- Fat commas are used between option names and their values, making it
clear to both humans and perltidy that values and names are bound
together. This is particularly useful for the readability of multi-line
command arrays, and there are plenty of them in the TAP tests. Most of
the test code is updated to use this style. Some commands used
parenthesis to show the link, or attached values and options in a single
string. These are updated to use fat commas instead.
- Option names are switched to use their long names, making them more
self-documented. Based on a suggestion by Andrew Dunstan.
- Add some trailing commas after the last item in multi-line arrays,
which is a common perl style.
Not all the places are taken care of, but this covers a very good chunk
of them.
Author: Dagfinn Ilmari Mannsåker
Reviewed-by: Michael Paquier, Peter Smith, Euler Taveira
Discussion: https://postgr.es/m/87jzc46d8u.fsf@wibble.ilmari.org
M contrib/basebackup_to_shell/t/001_basic.pl
M src/bin/initdb/t/001_initdb.pl
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/bin/pg_amcheck/t/005_opclass_damage.pl
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_basebackup/t/011_in_place_tablespace.pl
M src/bin/pg_basebackup/t/020_pg_receivewal.pl
M src/bin/pg_basebackup/t/030_pg_recvlogical.pl
M src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
M src/bin/pg_checksums/t/002_actions.pl
M src/bin/pg_combinebackup/t/002_compare_backups.pl
M src/bin/pg_combinebackup/t/003_timeline.pl
M src/bin/pg_combinebackup/t/004_manifest.pl
M src/bin/pg_combinebackup/t/005_integrity.pl
M src/bin/pg_combinebackup/t/006_db_file_copy.pl
M src/bin/pg_combinebackup/t/007_wal_level_minimal.pl
M src/bin/pg_combinebackup/t/008_promote.pl
M src/bin/pg_combinebackup/t/009_no_full_file.pl
M src/bin/pg_ctl/t/001_start_stop.pl
M src/bin/pg_ctl/t/002_status.pl
M src/bin/pg_ctl/t/003_promote.pl
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/pg_dump/t/003_pg_dump_with_server.pl
M src/bin/pg_dump/t/004_pg_dump_parallel.pl
M src/bin/pg_dump/t/005_pg_dump_filterfile.pl
M src/bin/pg_dump/t/010_dump_connstr.pl
M src/bin/pg_resetwal/t/001_basic.pl
M src/bin/pg_resetwal/t/002_corrupted.pl
M src/bin/pg_rewind/t/001_basic.pl
M src/bin/pg_rewind/t/006_options.pl
M src/bin/pg_rewind/t/007_standby_source.pl
M src/bin/pg_rewind/t/008_min_recovery_point.pl
M src/bin/pg_rewind/t/009_growing_files.pl
M src/bin/pg_rewind/t/010_keep_recycled_wals.pl
M src/bin/pg_rewind/t/RewindTest.pm
M src/bin/pg_test_fsync/t/001_basic.pl
M src/bin/pg_test_timing/t/001_basic.pl
M src/bin/pg_upgrade/t/004_subscription.pl
M src/bin/pg_verifybackup/t/001_basic.pl
M src/bin/pg_verifybackup/t/003_corruption.pl
M src/bin/pg_verifybackup/t/004_options.pl
M src/bin/pg_verifybackup/t/006_encoding.pl
M src/bin/pg_verifybackup/t/007_wal.pl
M src/bin/pg_verifybackup/t/010_client_untar.pl
M src/bin/pg_waldump/t/001_basic.pl
M src/bin/pg_waldump/t/002_save_fullpage.pl
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/psql/t/001_basic.pl
M src/bin/scripts/t/010_clusterdb.pl
M src/bin/scripts/t/011_clusterdb_all.pl
M src/bin/scripts/t/020_createdb.pl
M src/bin/scripts/t/040_createuser.pl
M src/bin/scripts/t/080_pg_isready.pl
M src/bin/scripts/t/090_reindexdb.pl
M src/bin/scripts/t/091_reindexdb_all.pl
M src/bin/scripts/t/100_vacuumdb.pl
M src/bin/scripts/t/101_vacuumdb_all.pl
M src/test/recovery/t/001_stream_rep.pl
M src/test/recovery/t/003_recovery_targets.pl
M src/test/recovery/t/017_shm.pl
M src/test/recovery/t/024_archive_recovery.pl
M src/test/recovery/t/027_stream_regress.pl
M src/test/ssl/t/001_ssltests.pl
Doc: Update the interaction of tablesync with wal_retrieve_retry_interval.
commit : 4a0e7314f11ee03adfe9df945598c068b4179314
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 22 Jan 2025 10:54:53 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 22 Jan 2025 10:54:53 +0530
In passing, update the documentation that explains the process of initial
data replication to explicitly state that it uses a table synchronization
worker.
Author: Vignesh C
Reviewed-by: Peter Smith, Shlok Kyal, Amit Kapila
Discussion: https://postgr.es/m/CALDaNm3RxGcD4cDAV5Q0_A4n06F3+AAMpxiyND9Zn0dB86hFmg@mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/logical-replication.sgml
Run perltidy
commit : be31ac25191b26a8a1db345a727545959654f4cb
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Jan 2025 10:13:59 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Jan 2025 10:13:59 +0900
A follow-up patch will adjust the TAP tests to follow a more-structured
format for option lists in commands, that perltidy is able to cope
better with. Putting the tree first in a clean state makes the next
change a bit easier. v20230309 has been used.
Author: Dagfinn Ilmari Mannsåker
Discussion: https://postgr.es/m/87jzc46d8u.fsf@wibble.ilmari.org
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_combinebackup/t/008_promote.pl
M src/bin/pg_combinebackup/t/009_no_full_file.pl
M src/bin/pg_verifybackup/t/002_algorithm.pl
M src/bin/pg_verifybackup/t/003_corruption.pl
M src/bin/pg_verifybackup/t/004_options.pl
M src/bin/pg_verifybackup/t/008_untar.pl
M src/bin/pg_verifybackup/t/010_client_untar.pl
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/test/recovery/t/035_standby_logical_decoding.pl
M src/test/recovery/t/040_standby_failover_slots_sync.pl
M src/test/ssl/t/SSL/Server.pm
M src/tools/add_commit_links.pl
M src/tools/pg_bsd_indent/t/001_pg_bsd_indent.pl
Doc: simplify the tutorial's window-function examples.
commit : 4907ba304c346051a6535e67c043779755a78e84
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 21 Jan 2025 14:43:21 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 21 Jan 2025 14:43:21 -0500
For the purposes of this discussion, row_number() is just as good
as rank(), and its behavior is easier to understand and describe.
So let's switch the examples to using row_number().
Along the way to checking the results given in the tutorial,
I found it helpful to extract the empsalary table we use in the
regression tests, which is evidently the same data that was used
to make these results. So I shoved that into advanced.source
to improve the coverage of that file a little. (There's still
several pages of the tutorial that are not included in it,
but at least now 3.5 Window Functions is covered.)
Suggested-by: "David G. Johnston" <david.g.johnston@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/173737973383.1070.1832752929070067441@wrigleys.postgresql.org
M doc/src/sgml/advanced.sgml
M src/tutorial/advanced.source
Reword recent error messages: "should" -> "must"
commit : db19a5061ce954320f47a65c169081cbb2d920f8
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Jan 2025 15:24:49 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Jan 2025 15:24:49 +0100
Most were introduced in the 17 timeframe. The ones in wparser_def.c are
very old.
I also changed "JSON path expression for column \"%s\" should return
single item without wrapper" to "JSON path expression for column \"%s\"
must return single item when no wrapper is requested" to avoid
ambiguity.
Backpatch to 17.
Crickets: https://postgr.es/m/202501131819.26ors7oouafu@alvherre.pgsql
M src/backend/tsearch/wparser_def.c
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/sqljson_jsontable.out
M src/test/regress/expected/sqljson_queryfuncs.out
Fix detach of a partition that has a toplevel FK to a partitioned table
commit : 9b21f203dd879a6a7eb3b8629f3d8cae0c1381a5
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Jan 2025 14:53:46 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Jan 2025 14:53:46 +0100
In common cases, foreign keys are defined on the toplevel partitioned
table; but if instead one is defined on a partition and references a
partitioned table, and the referencing partition is detached, we would
examine the pg_constraint row on the partition being detached, and fail
to realize that the sub-constraints must be left alone. This causes the
ALTER TABLE DETACH process to fail with
ERROR: could not find ON INSERT check triggers of foreign key constraint NNN
This is similar but not quite the same as what was fixed by
53af9491a043. This bug doesn't affect branches earlier than 15, because
the detach procedure was different there, so we only backpatch down to
15.
Fix by skipping such modifying constraints that are children of other
constraints being detached.
Author: Amul Sul <sulamul@gmail.com>
Diagnosys-by: Sami Imseih <samimseih@gmail.com>
Discussion: https://postgr.es/m/CAAJ_b97GuPh6wQPbxQS-Zpy16Oh+0aMv-w64QcGrLhCOZZ6p+g@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
Fix NO ACTION temporal foreign keys when the referenced endpoints change
commit : 1772d554b089c4779eaa13ae1487721e3080d119
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 21 Jan 2025 14:34:44 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 21 Jan 2025 14:34:44 +0100
If a referenced UPDATE changes the temporal start/end times, shrinking
the span the row is valid, we get a false return from
ri_Check_Pk_Match(), but overlapping references may still be valid, if
their reference didn't overlap with the removed span.
We need to consider what span(s) are still provided in the referenced
table. Instead of returning that from ri_Check_Pk_Match(), we can
just look it up in the main SQL query.
Reported-by: Sam Gabrielsson <sam@movsom.se>
Author: Paul Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+renyUApHgSZF9-nd-a0+OPGharLQLO=mDHcY4_qQ0+noCUVg@mail.gmail.com
M src/backend/catalog/pg_constraint.c
M src/backend/commands/tablecmds.c
M src/backend/utils/adt/ri_triggers.c
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_operator.dat
M src/test/regress/expected/without_overlaps.out
M src/test/regress/sql/without_overlaps.sql
Improve whitespace in without_overlaps test
commit : 888d4523f0c2f7e410739feff3ff97c5f3b9b3ea
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 21 Jan 2025 12:14:49 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 21 Jan 2025 12:14:49 +0100
Make some indentation better and more consistent. Extracted from
another patch with some actual test changes.
Discussion: https://www.postgresql.org/message-id/flat/CA+renyUApHgSZF9-nd-a0+OPGharLQLO=mDHcY4_qQ0+noCUVg@mail.gmail.com
M src/test/regress/expected/without_overlaps.out
M src/test/regress/sql/without_overlaps.sql
Improve generated_stored test
commit : 44b61efb7928a10e93408b374fa8df1effc8ef4c
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 21 Jan 2025 08:13:40 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 21 Jan 2025 08:13:40 +0100
The test table names gtest11s and gtest12s were way originally chosen
to signify "stored", when the idea was to have virtual columns in the
same test file. This is no longer the idea, so this naming is
irrelevant. (The upcoming feature of virtual generated columns will
have a test file that is initially a copy of generated_stored.sql, and
this random difference will be even more annoying then.) Clean this
up by dropping the suffix.
Discussion: https://www.postgresql.org/message-id/flat/a368248e-69e4-40be-9c07-6c3b5880b0a6@eisentraut.org
M src/test/regress/expected/generated_stored.out
M src/test/regress/sql/generated_stored.sql
Refactor ExecScan() to allow inlining of its core logic
commit : fb9f955025f7609fd3da0d7e33b77438ddc765de
author : Amit Langote <amitlan@postgresql.org>
date : Tue, 21 Jan 2025 12:53:03 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Tue, 21 Jan 2025 12:53:03 +0900
This commit refactors ExecScan() by moving its tuple-fetching,
filtering, and projection logic into an inline-able function,
ExecScanExtended(), defined in src/include/executor/execScan.h.
ExecScanExtended() accepts parameters for EvalPlanQual state,
qualifiers (ExprState), and projection (ProjectionInfo).
Specialized variants of the execution function of a given Scan node
(for example, ExecSeqScan() for SeqScan) can then pass const-NULL for
unused parameters. This allows the compiler to inline the logic and
eliminate unnecessary branches or checks. Each variant function thus
contains only the necessary code, optimizing execution for scans
where these features are not needed.
The variant function to be used is determined in the ExecInit*()
function of the node and assigned to the ExecProcNode function pointer
in the node's PlanState, effectively turning runtime checks and
conditional branches on the NULLness of epqstate, qual, and projInfo
into static ones, provided the compiler successfully eliminates
unnecessary checks from the inlined code of ExecScanExtended().
Currently, only ExecSeqScan() is modified to take advantage of this
inline-ability. Other Scan nodes might benefit from such specialized
variant functions but that is left as future work.
Benchmarks performed by Junwang Zhao, David Rowley and myself show up
to a 5% reduction in execution time for queries that rely heavily on
Seq Scans. The most significant improvements were observed in
scenarios where EvalPlanQual, qualifiers, and projection were not
required, but other cases also benefit from reduced runtime overhead
due to the inlining and removal of unnecessary code paths.
The idea for this patch first came from Andres Freund in an off-list
discussion. The refactoring approach implemented here is based on a
proposal by David Rowley, significantly improving upon the patch I
(amitlan) initially proposed.
Suggested-by: Andres Freund <andres@anarazel.de>
Co-authored-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Junwang Zhao <zhjwpku@gmail.com>
Tested-by: Junwang Zhao <zhjwpku@gmail.com>
Tested-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CA+HiwqGaH-otvqW_ce-paL=96JvU4j+Xbuk+14esJNDwefdkOg@mail.gmail.com
M src/backend/executor/execScan.c
M src/backend/executor/nodeSeqscan.c
A src/include/executor/execScan.h
Rework handling of pending data for backend statistics
commit : 4feba03d8b925c4cdda460927611d6b17fb01803
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 21 Jan 2025 11:30:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 21 Jan 2025 11:30:42 +0900
9aea73fc61d4 has added support for backend statistics, relying on
PgStat_EntryRef->pending for its data pending for flush. This design
lacks in flexibility, because the pending list does some memory
allocation, making it unsuitable if incrementing counters in critical
sections.
Pending data of backend statistics is reworked so the implementation
does not depend on PgStat_EntryRef->pending anymore, relying on a static
area of memory to store the counters that are flushed when stats are
reported to the pgstats dshash. An advantage of this approach is to
allow the pending data to be manipulated in critical sections; some
patches are under discussion and require that.
The pending data is tracked by PendingBackendStats, local to
pgstat_backend.c. Two routines are introduced to allow IO statistics to
update the backend-side counters. have_static_pending_cb and
flush_static_cb are used for the flush, instead of flush_pending_cb.
Author: Bertrand Drouvot, Michael Paquier
Discussion: https://postgr.es/m/66efowskppsns35v5u2m7k4sdnl7yoz5bo64tdjwq7r5lhplrz@y7dme5xwh2r5
M src/backend/utils/activity/pgstat.c
M src/backend/utils/activity/pgstat_backend.c
M src/backend/utils/activity/pgstat_io.c
M src/backend/utils/activity/pgstat_relation.c
M src/include/pgstat.h
M src/include/utils/pgstat_internal.h
Rename some pgstats callbacks related to flush of entries
commit : 28de66cee5f45e2f173fa60dd6867c810ecabe38
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 21 Jan 2025 10:12:39 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 21 Jan 2025 10:12:39 +0900
The two callbacks have_fixed_pending_cb and flush_fixed_cb have been
introduced in fc415edf8ca8 to provide a way for fixed-numbered
statistics to control the flush of their data. These are renamed to
respectively have_static_pending_cb and flush_static_cb. The
restriction that these only apply to fixed-numbered stats is removed.
A follow-up patch will make use of them for backend statistics. This
stats kind is variable-numbered, and patches are under discussion to
track WAL data for IO and backend stats which cannot use
PgStat_EntryRef->pending as pending data would be touched in critical
sections, where no memory allocation can happen.
Per discussion with Andres Freund.
Author: Bertrand Drouvot
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/66efowskppsns35v5u2m7k4sdnl7yoz5bo64tdjwq7r5lhplrz@y7dme5xwh2r5
M src/backend/utils/activity/pgstat.c
M src/include/utils/pgstat_internal.h
Update time zone data files to tzdata release 2025a.
commit : 60c513f8fa17b83878bc2267b4d1e77dccd38cea
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jan 2025 16:49:15 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jan 2025 16:49:15 -0500
DST law changes in Paraguay.
Historical corrections for the Philippines.
Backpatch-through: 13
M src/timezone/data/tzdata.zi
Avoid using timezone Asia/Manila in regression tests.
commit : 8108674f0e5639baebcf03b54b7ccf9e9a8662a2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jan 2025 15:47:53 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jan 2025 15:47:53 -0500
The freshly-released 2025a version of tzdata has a refined estimate
for the longitude of Manila, changing their value for LMT in
pre-standardized-timezone days. This changes the output of one of
our test cases. Since we need to be able to run with system tzdata
files that may or may not contain this update, we'd better stop
making that specific test.
I switched it to use Asia/Singapore, which has a roughly similar UTC
offset. That LMT value hasn't changed in tzdb since 2003, so we can
hope that it's well established.
I also noticed that this set of make_timestamptz tests only exercises
zones east of Greenwich, which seems rather sad, and was not the
original intent AFAICS. (We've already changed these tests once
to stabilize their results across tzdata updates, cf 66b737cd9;
it looks like I failed to consider the UTC-offset-sign aspect then.)
To improve that, add a test with Pacific/Honolulu. That LMT offset
is also quite old in tzdb, so we'll cross our fingers that it doesn't
get improved.
Reported-by: Christoph Berg <cb@df7cb.de>
Discussion: https://postgr.es/m/Z46inkznCxesvDEb@msg.df7cb.de
Backpatch-through: 13
M src/include/datatype/timestamp.h
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamptz.sql
Improve generated_stored test
commit : 86749ea3b7660b09cfb951a253c24d2975f189f6
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Jan 2025 15:27:33 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Jan 2025 15:27:33 +0100
It makes more sense to put the catalog sanity check at the end of the
test rather than at the beginning, so that it can also check whatever
the tests did rather than just whatever happened before the tests.
Suggested-by: jian he <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/a368248e-69e4-40be-9c07-6c3b5880b0a6@eisentraut.org
M src/test/regress/expected/generated_stored.out
M src/test/regress/sql/generated_stored.sql
Add some more use of Page/PageData rather than char *
commit : 443a8e4ae364bacc4016f569439546e53b80fd66
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Jan 2025 13:05:50 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Jan 2025 13:05:50 +0100
Discussion: https://www.postgresql.org/message-id/flat/692ee0da-49da-4d32-8dca-da224cc2800e@eisentraut.org
M src/backend/access/transam/xloginsert.c
M src/include/access/xloginsert.h
Add const qualifiers to bufpage.h
commit : 4f4a1d853a5c2d31f11c56939c44aa917e77c1da
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Jan 2025 10:53:47 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Jan 2025 10:53:47 +0100
This makes use of the new PageData type.
PageGetSpecialPointer() had to be turned back into a macro, because it
is used in a way that sometimes it takes const and returns const and
sometimes takes non-const and returns non-const.
Discussion: https://www.postgresql.org/message-id/flat/692ee0da-49da-4d32-8dca-da224cc2800e@eisentraut.org
M src/backend/storage/page/bufpage.c
M src/include/storage/bufpage.h
Add PageData C type
commit : 6e4df237fbb532e7807cdd97f8b0b0d85093b9ee
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Jan 2025 10:53:47 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Jan 2025 10:53:47 +0100
This adds the C type PageData and makes the existing type Page a
pointer to it. This follows the usual PostgreSQL C type naming scheme
of Foo/FooData pairs. (Prior to commit ddbba3aac86, PageData existed
as an unrelated type.) The type definitions are compatible, so this
doesn't change anything except some of the naming.
Discussion: https://www.postgresql.org/message-id/flat/692ee0da-49da-4d32-8dca-da224cc2800e@eisentraut.org
M src/include/storage/bufpage.h
M src/tools/pgindent/typedefs.list
Fix latch event policy that hid socket events.
commit : 73f6b9a3b0fe161cd3d35b1e3109cd47864a4b33
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 20 Jan 2025 15:17:47 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 20 Jan 2025 15:17:47 +1300
If a WaitEventSetWait() caller asks for multiple events, an already set
latch would previously prevent other events from being reported at the
same time. Now, we'll also poll the kernel for other events that would
fit in the caller's output buffer with a zero wait time. This policy
change doesn't affect callers that ask for only one event.
The main caller affected is the postmaster. If its latch is set
extremely frequently by backends launching workers and workers exiting,
we don't want it to handle only those jobs and ignore incoming client
connections.
Back-patch to 16 where the postmaster began using the API. The
fast-return policy changed here is older than that, but doesn't cause
any known problems in earlier releases.
Reported-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/Z1n5UpAiGDmFcMmd%40nathan
M src/backend/storage/ipc/latch.c
Fix header check for continuation records where standbys could be stuck
commit : 6cf1647d87e7cd423d71525a8759b75c4e4a47ec
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 20 Jan 2025 09:29:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 20 Jan 2025 09:29:42 +0900
XLogPageRead() checks immediately for an invalid WAL record header on a
standby, to be able to handle the case of continuation records that need
to be read across two different sources. As written, the check was too
generic, applying to any target LSN. Based on an analysis by Kyotaro
Horiguchi, what really matters is to make sure that the page header is
checked when attempting to read a LSN at the boundary of a segment, to
handle the case of a continuation record that spawns across multiple
pages when dealing with multiple segments, as WAL receivers are spawned
they request WAL from the beginning of a segment. This fix has been
proposed by Kyotaro Horiguchi.
This could cause standbys to loop infinitely when dealing with a
continuation record during a timeline jump, in the case where the
contents of the record in the follow-up page are invalid.
Some regression tests are added to check such scenarios, able to
reproduce the original problem. In the test, the contents of a
continuation record are overwritten with junk zeros on its follow-up
page, and replayed on standbys. This is inspired by 039_end_of_wal.pl,
and is enough to show how standbys should react on promotion by not
being stuck. Without the fix, the test would fail with a timeout. The
test to reproduce the problem has been written by Alexander Kukushkin.
The original check has been introduced in 066871980183, for a similar
problem.
Author: Kyotaro Horiguchi, Alexander Kukushkin
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/CAFh8B=mozC+e1wGJq0H=0O65goZju+6ab5AU7DEWCSUA2OtwDg@mail.gmail.com
Backpatch-through: 13
M src/backend/access/transam/xlogrecovery.c
M src/test/recovery/meson.build
A src/test/recovery/t/043_no_contrecord_switch.pl
Remove PrintBufferDescs() and PrintPinnedBufs().
commit : 23d7562018b2c772aec26f4641de211d8a930b26
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 19 Jan 2025 14:00:22 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 19 Jan 2025 14:00:22 -0500
These have been #ifdef'd out for a long time, and in fact have
been uncompilable since commit 48354581a of 2016-04-10. The
fact that nobody noticed for so long demonstrates their lack of
usefulness, so let's remove them rather than fix them.
Author: Jacob Brazeal <jacob.brazeal@gmail.com>
Discussion: https://postgr.es/m/CA+COZaB+9CN_f63PPRoVhHjYmCwwmb_9CWLxqCJdMWDqs1a-JA@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
M src/include/storage/bufmgr.h
Be clearer about when jsonapi's need_escapes is needed
commit : ea5ff5833c7d4ae727a5acfc590c848b520775d0
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 19 Jan 2025 09:09:58 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 19 Jan 2025 09:09:58 -0500
Most operations beyond pure json parsing need to set need_escapes to
true to get access to field names and string scalars. Document this
fact more explicitly.
Slightly tweaked patch from:
Author: Corey Huinker <corey.huinker@gmail.com>
Discussion: https://postgr.es/m/CADkLM=c49Vkfg2+A8ubSuEtaGEjuaKZXCA6SrXA8kdwHjx3uxQ@mail.gmail.com
M src/common/jsonapi.c
M src/include/common/jsonapi.h
Support PG_UNICODE_FAST locale in the builtin collation provider.
commit : d3d0983169130a9b81e3fe48d5c2ca4931480956
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 17 Jan 2025 15:56:30 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 17 Jan 2025 15:56:30 -0800
The PG_UNICODE_FAST locale uses code point sort order (fast,
memcmp-based) combined with Unicode character semantics. The character
semantics are based on Unicode full case mapping.
Full case mapping can map a single codepoint to multiple codepoints,
such as "ß" uppercasing to "SS". Additionally, it handles
context-sensitive mappings like the "final sigma", and it uses
titlecase mappings such as "Dž" when titlecasing (rather than plain
uppercase mappings).
Importantly, the uppercasing of "ß" as "SS" is specifically mentioned
by the SQL standard. In Postgres, UCS_BASIC uses plain ASCII semantics
for case mapping and pattern matching, so if we changed it to use the
PG_UNICODE_FAST locale, it would offer better compliance with the
standard. For now, though, do not change the behavior of UCS_BASIC.
Discussion: https://postgr.es/m/ddfd67928818f138f51635712529bc5e1d25e4e7.camel@j-davis.com
Discussion: https://postgr.es/m/27bb0e52-801d-4f73-a0a4-02cfdd4a9ada@eisentraut.org
Reviewed-by: Peter Eisentraut, Daniel Verite
M doc/src/sgml/charset.sgml
M doc/src/sgml/ref/create_collation.sgml
M doc/src/sgml/ref/create_database.sgml
M doc/src/sgml/ref/initdb.sgml
M src/backend/regex/regc_pg_locale.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_locale_builtin.c
M src/bin/initdb/initdb.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_collation.dat
M src/include/utils/pg_locale.h
M src/test/regress/expected/collate.utf8.out
M src/test/regress/sql/collate.utf8.sql
Support Unicode full case mapping and conversion.
commit : 286a365b9c25479f8ad82043ed136748733adfa6
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 17 Jan 2025 15:56:20 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 17 Jan 2025 15:56:20 -0800
Generate tables from Unicode SpecialCasing.txt to support more
sophisticated case mapping behavior:
* support case mappings to multiple codepoints, such as "ß"
uppercasing to "SS"
* support conditional case mappings, such as the "final sigma"
* support titlecase variants, such as "dž" uppercasing to "DŽ" but
titlecasing to "Dž"
Discussion: https://postgr.es/m/ddfd67928818f138f51635712529bc5e1d25e4e7.camel@j-davis.com
Discussion: https://postgr.es/m/27bb0e52-801d-4f73-a0a4-02cfdd4a9ada@eisentraut.org
Reviewed-by: Peter Eisentraut, Daniel Verite
M src/backend/utils/adt/pg_locale_builtin.c
M src/common/unicode/Makefile
M src/common/unicode/case_test.c
M src/common/unicode/generate-unicode_case_table.pl
M src/common/unicode/meson.build
M src/common/unicode_case.c
M src/include/common/unicode_case.h
M src/include/common/unicode_case_table.h
M src/tools/pgindent/typedefs.list
vacuumdb: Fix comment for vacuum_one_database().
commit : 6a9b2a631aa3dbac5b351cd0c45631cbc6e8d19e
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 17 Jan 2025 15:23:14 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 17 Jan 2025 15:23:14 -0600
Since commit e0c2933a76, vacuum_one_database() always uses a
catalog query to discover the tables to process, but this comment
still notes the special case for which we used a catalog query
before that commit. Let's just remove that note.
Also, commit 7781f4e3e7 renamed the "tables" parameter to "objects"
but missed updating this comment. This commit fixes that as well.
M src/bin/scripts/vacuumdb.c
Add documentation about calling version-1 C functions from C.
commit : 86e4efc52b664b60f9494e062bbf0454cabcb61f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Jan 2025 14:37:38 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Jan 2025 14:37:38 -0500
This topic wasn't really covered before, so fill in some details.
Author: Florents Tselai <florents.tselai@gmail.com>
Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/90853055-5BBD-493D-91E5-721677C7C59B@gmail.com
M doc/src/sgml/xfunc.sgml
M src/tutorial/funcs.c
M src/tutorial/funcs.source
Fix parsing of qualified relation names in RETURNING.
commit : 43830ecb8a9b6a1bc322298a77a5e0d87c2e83d4
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Fri, 17 Jan 2025 10:35:07 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Fri, 17 Jan 2025 10:35:07 +0000
Given a qualified refname, refnameNamespaceItem() will search for a
matching namespace item by relation OID, rather than by name. Commit
80feb727c8 broke this by adding additional namespace items for OLD and
NEW in the RETURNING list, which have the same relation OID, causing
ambiguity. Fix this by ignoring these in the search, which is correct
since they don't match the qualified relation name, and so there is no
real ambiguity.
Reported by Richard Guo.
Discussion: https://postgr.es/m/CAMbWs49MBjWYWDROJ8MZ%3DY%2B4UgRQa10wzik1tWrD5yto9eoGXg%40mail.gmail.com
M src/backend/parser/parse_relation.c
M src/test/regress/expected/returning.out
M src/test/regress/sql/returning.sql
Speed up hex_encode with bytewise lookup
commit : e24d77080b365265f3c854f64e71518622883678
author : John Naylor <john.naylor@postgresql.org>
date : Wed, 15 Jan 2025 13:28:26 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Wed, 15 Jan 2025 13:28:26 +0700
Previously, hex_encode looked up each nibble of the input
separately. We now use a larger lookup table containing the two-byte
encoding of every possible input byte, resulting in a 1/3 reduction
in encoding time.
Reviewed by Tom Lane, Michael Paquier, Nathan Bossart, David Rowley
Discussion: https://postgr.es/m/CANWCAZZvXuJMgqMN4u068Yqa19CEjS31tQKZp_qFFFbgYfaXqQ%40mail.gmail.com
M src/backend/utils/adt/encode.c
Remove flex version checks
commit : 0869ea43e9c2257819e5c8719e28fe3b5c527911
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 17 Jan 2025 08:35:52 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 17 Jan 2025 08:35:52 +0100
Remove the flex version checks from configure and meson. The cutoff
versions are all so ancient that this is no longer relevant, and what
the actual cutoff should be is a bit fuzzy.
This also removes the ancient behavior that configure would also
accept a "lex" program if it is actuall flex. This aligns the check
with meson in this respect.
For future reference, as of this commit, these are relevant flex
versions:
- The hard required minimum is flex 2.5.34 as of commit b1ef48980dd,
but this has not actually been tested.
- Prior to this, the minimum enforced by configure/meson was flex
2.5.35, which is the oldest present in the buildfarm right now.
- As of commit 6fdd5d95634, the oldest version that will compile
without warnings due to flex-generated code is flex 2.5.36.
- The oldest version that probably still has some practical relevance
is flex 2.5.37, which ships with CentOS/RHEL 7.
Discussion: https://www.postgresql.org/message-id/1a204ccd-7ae6-478c-a431-407b5c48ccc6@eisentraut.org
M config/programs.m4
M configure
M doc/src/sgml/installation.sgml
M meson.build
M src/backend/utils/misc/guc-file.l
Add pg_nodiscard decorations to base64 functions
commit : b0eff1098860f88dd385e6882ff7b1063d267213
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 17 Jan 2025 08:06:24 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 17 Jan 2025 08:06:24 +0100
The result of pg_b64_encode() and pg_b64_decode() should be checked
for errors. This attribute could detect mistakes such as those fixed
in commit ff030ebe250 and d278541be42.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAEudQAq-3yHsSdWoOOaw%2BgAQYgPMpMGuB5pt2yCXgv-YuxG2Hg%40mail.gmail.com
M src/include/common/base64.h
Revert recent changes related to handling of 2PC files at recovery
commit : a6c70f68cdeb3bed591a2dd7d421bf3536ba5216
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 17 Jan 2025 13:27:39 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 17 Jan 2025 13:27:39 +0900
This commit reverts 8f67f994e8ea (down to v13) and c3de0f9eed38 (down to
v17), as these are proving to not be completely correct regarding two
aspects:
- In v17 and newer branches, c3de0f9eed38's check for epoch handling is
incorrect, and does not correctly handle frozen epochs. A logic closer
to widen_snapshot_xid() should be used. The 2PC code should try to
integrate deeper with FullTransactionIds, 5a1dfde8334b being not enough.
- In v13 and newer branches, 8f67f994e8ea is a workaround for the real
issue, which is that we should not attempt CLOG lookups without reaching
consistency. This exists since 728bd991c3c4, and this is reachable with
ProcessTwoPhaseBuffer() called by restoreTwoPhaseData() at the beginning
of recovery.
Per discussion with Noah Misch.
Discussion: https://postgr.es/m/20250116010051.f3.nmisch@google.com
Backpatch-through: 13
M src/backend/access/transam/twophase.c
M src/test/recovery/t/009_twophase.pl
Remove redefinitions of SIG_* macros in win32_port.h.
commit : 0dc9c7d200e59b19b2e683f78afa1ee4cfcf3c21
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 16 Jan 2025 20:55:24 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 16 Jan 2025 20:55:24 -0600
It is not clear why these were originally added. One hypothesis is
that an ancient version of MinGW didn't define them. In any case,
they appear to now be superfluous, so let's remove them. If
nothing else, the buildfarm might offer us clues to their origins.
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/Z4chOKfnthRH71mw%40nathan
M src/include/port/win32_port.h
Fix setrefs.c's failure to do expression processing on prune steps.
commit : bf826ea062979da02395fb2e8961feeef47604da
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 16 Jan 2025 20:40:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 16 Jan 2025 20:40:07 -0500
We should run the expression subtrees of PartitionedRelPruneInfo
structs through fix_scan_expr. Failure to do so means that
AlternativeSubPlans within those expressions won't be cleaned up
properly, resulting in "unrecognized node type" errors since v14.
It seems fairly likely that at least some of the other steps done
by fix_scan_expr are important here as well, resulting in as-yet-
undetected bugs. Therefore, I've chosen to back-patch this to
all supported branches including v13, even though the known
symptom doesn't manifest in v13.
Per bug #18778 from Alexander Lakhin.
Discussion: https://postgr.es/m/18778-24cd399df6c806af@postgresql.org
M src/backend/optimizer/plan/setrefs.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Add and use BitmapHeapScanDescData struct
commit : f7a8fc10ccb882249f8624b937f2c3b467d07bd6
author : Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 16 Jan 2025 18:42:39 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 16 Jan 2025 18:42:39 -0500
Move the several members of HeapScanDescData which are specific to
Bitmap Heap Scans into a new struct, BitmapHeapScanDescData, which
inherits from HeapScanDescData.
This reduces the size of the HeapScanDescData for other types of scans
and will allow us to add additional bitmap heap scan-specific members in
the future without fear of bloating the HeapScanDescData.
Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/c736f6aa-8b35-4e20-9621-62c7c82e2168%40vondra.me
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/include/access/heapam.h
M src/tools/pgindent/typedefs.list
Rework macro pgstat_is_ioop_tracked_in_bytes()
commit : 7b6468cc9523d7d923487d212281af51f7f3dae2
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 17 Jan 2025 08:26:17 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 17 Jan 2025 08:26:17 +0900
As written, it was triggering a compilation warning for old versions of
clang, as reported by buildfarm members ayu, batfish and demoiselle.
Forcing a cast with "unsigned int" should fix the warning.
While on it, the macro is moved to pgstat.h, closer to the declaration
of IOOp, per suggestion from Tom Lane.
Reported-by: Tom Lane
Reviewed-by: Bertrand Drouvot, Tom Lane, Nazir Bilal Yavuz
Discussion: https://postgr.es/m/1272824.1736961543@sss.pgh.pa.us
M src/backend/utils/activity/pgstat_io.c
M src/include/pgstat.h
Convert libpgport's pqsignal() to a void function.
commit : d4a43b283751b23d32bbfa1ecc2cad2d16e3dde9
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 16 Jan 2025 16:41:05 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 16 Jan 2025 16:41:05 -0600
The protections added by commit 3b00fdba9f introduced race
conditions to this function that can lead to bogus return values.
Since nobody seems to inspect the return value, this is of little
consequence, but it would have been nice to convert it to a void
function to avoid any possibility of a bogus return value. I
originally thought that doing so would have required also modifying
legacy-pqsignal.c's version of the function (which would've
required an SONAME bump), but commit 9a45a89c38 gave
legacy-pqsignal.c its own dedicated extern for pqsignal(), thereby
decoupling it enough that libpgport's pqsignal() can be modified.
This commit also adds an assertion for the return value of
sigaction()/signal(). Since a failure most likely indicates a
coding error, and nobody has ever bothered to check pqsignal()'s
return value, it's probably not worth the effort to do anything
fancier.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/Z4chOKfnthRH71mw%40nathan
M src/include/port.h
M src/port/pqsignal.c
Avoid calling pqsignal() with invalid signals on Windows frontends.
commit : 5cda4fdb0beb64453b634d0ab966912965c7b8f6
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 16 Jan 2025 15:56:39 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 16 Jan 2025 15:56:39 -0600
As noted by the comment at the top of port/pqsignal.c, Windows
frontend programs can only use pqsignal() with the 6 signals
required by C. Most places avoid using invalid signals via #ifndef
WIN32, but initdb and pg_test_fsync check whether the signal itself
is defined, which doesn't work because win32_port.h defines many
extra signals for the signal emulation code. pg_regress seems to
have missed the memo completely. These issues aren't causing any
real problems today because nobody checks the return value of
pqsignal(), but a follow-up commit will add some error checking.
To fix, surround all frontend calls to pqsignal() that use signals
that are invalid on Windows with #ifndef WIN32. We cannot simply
skip defining the extra signals in win32_port.h for frontends
because they are needed in places such as pgkill().
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/Z4chOKfnthRH71mw%40nathan
M src/bin/initdb/initdb.c
M src/bin/pg_test_fsync/pg_test_fsync.c
M src/test/regress/pg_regress.c
Seek zone abbreviations in the IANA data before timezone_abbreviations.
commit : d7674c9fab09d5bab427ba3b9b7a20b169aba715
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 16 Jan 2025 14:11:19 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 16 Jan 2025 14:11:19 -0500
If a time zone abbreviation used in datetime input is defined in
the currently active timezone, use that definition in preference
to looking in the timezone_abbreviations list. That allows us to
correctly handle abbreviations that have different meanings in
different timezones. Also, it eliminates an inconsistency between
datetime input and datetime output: the non-ISO datestyles for
timestamptz have always printed abbreviations taken from the IANA
data, not from timezone_abbreviations. Before this fix, it was
possible to demonstrate cases where casting a timestamp to text
and back fails or changes the value significantly because of that
inconsistency.
While this change removes the ability to override the IANA data about
an abbreviation known in the current zone, it's not clear that there's
any real use-case for doing so. But it is clear that this makes life
a lot easier for dealing with abbreviations that have conflicts across
different time zones.
Also update the pg_timezone_abbrevs view to report abbreviations
that are recognized via the IANA data, and *not* report any
timezone_abbreviations entries that are thereby overridden.
Under the hood, there are now two SRFs, one that pulls the IANA
data and one that pulls timezone_abbreviations entries. They're
combined by logic in the view. This approach was useful for
debugging (since the functions can be called on their own).
While I don't intend to document the functions explicitly,
they might be useful to call directly.
Also improve DecodeTimezoneAbbrev's caching logic so that it can
cache zone abbreviations found in the IANA data. Without that,
this patch would have caused a noticeable degradation of the
runtime of timestamptz_in.
Per report from Aleksander Alekseev and additional investigation.
Discussion: https://postgr.es/m/CAJ7c6TOATjJqvhnYsui0=CO5XFMF4dvTGH+skzB--jNhqSQu5g@mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/datetime.sgml
M doc/src/sgml/system-views.sgml
M src/backend/catalog/system_views.sql
M src/backend/commands/variable.c
M src/backend/utils/adt/datetime.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgtime.h
M src/include/utils/datetime.h
M src/test/regress/expected/horology.out
M src/test/regress/expected/rules.out
M src/test/regress/expected/sysviews.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/horology.sql
M src/test/regress/sql/sysviews.sql
M src/test/regress/sql/timestamptz.sql
M src/timezone/localtime.c
M src/tools/pgindent/typedefs.list
Make pg_interpret_timezone_abbrev() check sp->defaulttype too.
commit : bc10219b9c8931ff4f872b3e799da2208101c574
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 16 Jan 2025 12:43:03 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 16 Jan 2025 12:43:03 -0500
This omission caused it to not recognize the furthest-back zone
abbreviation when working with timezone data compiled with relatively
recent zic (2018f or newer). Older versions of zic produced a dummy
DST transition at the Big Bang, so that the oldest abbreviation could
always be found in the sp->types[] array; but newer versions don't do
that, so that we must examine defaulttype as well as the types[] array
to be sure of seeing all the abbreviations.
While this has been broken for six or so years, we'd managed not
to notice for two reasons: (1) many platforms are still using
ancient zic for compatibility reasons, so that the issue did not
manifest in builds using --with-system-tzdata; (2) the oldest
zone abbreviation is almost always "LMT", which we weren't
supporting anyway (but an upcoming patch will accept that).
While at it, update pg_next_dst_boundary() to use sp->defaulttype
as the time type for non-DST zones and times before the oldest
DST transition. The existing code there predates upstream's
invention of the sp->defaulttype field, and its heuristic for
finding the oldest time type has now been subsumed into the
code that sets sp->defaulttype.
Possibly this should be back-patched, but I'm not currently aware
of any visible consequences of this bug in released branches.
Per report from Aleksander Alekseev and additional investigation.
Discussion: https://postgr.es/m/CAJ7c6TOATjJqvhnYsui0=CO5XFMF4dvTGH+skzB--jNhqSQu5g@mail.gmail.com
M src/timezone/localtime.c
Fix nbtree contradictory array element comment.
commit : 901bd4a65ab9bc0025be0fc8fe1fce3c69e7b78e
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 16 Jan 2025 11:26:41 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 16 Jan 2025 11:26:41 -0500
Oversight in commit 5bf748b8, which enhanced nbtree ScalarArrayOp
execution.
M src/backend/access/nbtree/nbtpreprocesskeys.c
Split ATExecValidateConstraint into reusable pieces
commit : 86374c9a0e304520044ab8b2cb84849c7db4a9fa
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 16 Jan 2025 16:44:24 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 16 Jan 2025 16:44:24 +0100
With this, we have separate functions to add validation requests to
ALTER TABLE's phase 3 queue for check and foreign key constraints, which
allows reusing them in future commits -- particularly this will allow us
to perform validation of invalid foreign key constraints in partitioned
tables.
We could have let the check constraint code alone since we don't need to
reuse that for anything at this point, but it seems cleaner and more
consistent to do both at the same time.
Author: Amul Sul <sulamul@gmail.com>
Discussion: https://postgr.es/m/CAAJ_b96Bp=-ZwihPPtuaNX=SrZ0U6ZsXD3+fgARO0JuKa8v2jQ@mail.gmail.com
M src/backend/commands/tablecmds.c
Add OLD/NEW support to RETURNING in DML queries.
commit : 80feb727c869cc0b2e12bd1543bafa449be9c8e2
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 16 Jan 2025 14:57:35 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 16 Jan 2025 14:57:35 +0000
This allows the RETURNING list of INSERT/UPDATE/DELETE/MERGE queries
to explicitly return old and new values by using the special aliases
"old" and "new", which are automatically added to the query (if not
already defined) while parsing its RETURNING list, allowing things
like:
RETURNING old.colname, new.colname, ...
RETURNING old.*, new.*
Additionally, a new syntax is supported, allowing the names "old" and
"new" to be changed to user-supplied alias names, e.g.:
RETURNING WITH (OLD AS o, NEW AS n) o.colname, n.colname, ...
This is useful when the names "old" and "new" are already defined,
such as inside trigger functions, allowing backwards compatibility to
be maintained -- the interpretation of any existing queries that
happen to already refer to relations called "old" or "new", or use
those as aliases for other relations, is not changed.
For an INSERT, old values will generally be NULL, and for a DELETE,
new values will generally be NULL, but that may change for an INSERT
with an ON CONFLICT ... DO UPDATE clause, or if a query rewrite rule
changes the command type. Therefore, we put no restrictions on the use
of old and new in any DML queries.
Dean Rasheed, reviewed by Jian He and Jeff Davis.
Discussion: https://postgr.es/m/CAEZATCWx0J0-v=Qjc6gXzR=KtsdvAE7Ow=D=mu50AgOe+pvisQ@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/dml.sgml
M doc/src/sgml/ref/delete.sgml
M doc/src/sgml/ref/insert.sgml
M doc/src/sgml/ref/merge.sgml
M doc/src/sgml/ref/update.sgml
M doc/src/sgml/rules.sgml
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/execMain.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeModifyTable.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/nodes/makefuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/optimizer/util/appendinfo.c
M src/backend/optimizer/util/clauses.c
M src/backend/optimizer/util/paramassign.c
M src/backend/optimizer/util/plancat.c
M src/backend/optimizer/util/var.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_merge.c
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_target.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/rewrite/rewriteManip.c
M src/backend/utils/adt/ruleutils.c
M src/include/catalog/catversion.h
M src/include/executor/execExpr.h
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
M src/include/nodes/parsenodes.h
M src/include/nodes/plannodes.h
M src/include/nodes/primnodes.h
M src/include/optimizer/optimizer.h
M src/include/optimizer/paramassign.h
M src/include/parser/analyze.h
M src/include/parser/parse_node.h
M src/include/parser/parse_relation.h
M src/include/rewrite/rewriteManip.h
M src/interfaces/ecpg/preproc/parse.pl
M src/test/isolation/expected/merge-update.out
M src/test/isolation/specs/merge-update.spec
M src/test/regress/expected/merge.out
M src/test/regress/expected/returning.out
M src/test/regress/expected/rules.out
M src/test/regress/expected/updatable_views.out
M src/test/regress/sql/merge.sql
M src/test/regress/sql/returning.sql
M src/test/regress/sql/rules.sql
M src/test/regress/sql/updatable_views.sql
M src/tools/pgindent/typedefs.list
Remove dead code
commit : 7407b2d48cf37bc8847ae6c47dde2164ef2faa34
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 16 Jan 2025 14:37:28 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 16 Jan 2025 14:37:28 +0100
As of commit 9895b35cb88, AlterDomainAddConstraint() can only be
called with constraints of type CONSTR_CHECK and CONSTR_NOTNULL. So
all the code to check for and reject other constraint type values is
dead and can be removed.
Author: jian he <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CACJufxHitd5LGLBSSAPShhtDWxT0ViVKTHinkYW-skBX93TcpA@mail.gmail.com
M src/backend/commands/typecmds.c
refactor: split ATExecAlterConstrRecurse()
commit : 7a947ed25b547256c5e05793467ff0f23991073f
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 16 Jan 2025 13:22:01 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 16 Jan 2025 13:22:01 +0100
This splits out a couple of subroutines from
ATExecAlterConstrRecurse(). This makes the main function a bit
smaller, and a future patch (NOT ENFORCED foreign-key constraints)
will also want to call some of the pieces separately.
Author: Amul Sul <amul.sul@enterprisedb.com>
Reviewed-by: jian he <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAAJ_b962c5AcYW9KUt_R_ER5qs3fUGbe4az-SP-vuwPS-w-AGA%40mail.gmail.com
M src/backend/commands/tablecmds.c
Fix error handling of pg_b64_decode()
commit : d278541be422f348b10c89c638942c25229c5e07
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 16 Jan 2025 09:02:21 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 16 Jan 2025 09:02:21 +0100
Fix for commit 761c79508e7. The previous error handling logic was not
quite correct.
Discussion: https://www.postgresql.org/message-id/flat/CAEudQAq-3yHsSdWoOOaw%2BgAQYgPMpMGuB5pt2yCXgv-YuxG2Hg%40mail.gmail.com
M src/interfaces/libpq/fe-connect.c
Check return of pg_b64_encode() for error
commit : ff030ebe25022bdb90cecb7bfe45c6f863e3be30
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 16 Jan 2025 08:35:57 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 16 Jan 2025 08:35:57 +0100
Forgotten in commit 761c79508e7.
Author: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAEudQAq-3yHsSdWoOOaw%2BgAQYgPMpMGuB5pt2yCXgv-YuxG2Hg%40mail.gmail.com
M contrib/postgres_fdw/connection.c
Doc: Improve the Replica Identity information.
commit : 965b2cc0a400bc0bc56525f8ae12ddd93c3d5bbd
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 16 Jan 2025 08:33:02 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 16 Jan 2025 08:33:02 +0530
This commit improves the Replica Identity information and clarifies its
related restrictions.
Reported-by: James Coleman
Author: Peter Smith
Co-authored-by: Robert Treat
Reviewed-by: Laurenz Albe, Amit Kapila
Discussion: https://postgr.es/m/CAAaqYe_=7qFSqW7qavvhVy58mmzk1uSQ0RReRiUHyKO5znvr7g@mail.gmail.com
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/ref/alter_table.sgml
Move routines to manipulate WAL into PostgreSQL::Test::Cluster
commit : 32a18cc0a73d8deb1e81d6ef8a9d5a6f300ed397
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 16 Jan 2025 09:25:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 16 Jan 2025 09:25:29 +0900
These facilities were originally in the recovery TAP test
039_end_of_wal.pl. A follow-up bug fix with a TAP test doing similar
WAL manipulations requires them, and all these had better not be
duplicated due to their complexity. The routine names are tweaked to
use "wal" more consistently, similarly to the existing "advance_wal".
In v14 and v13, the new routines are moved to PostgresNode.pm.
039_end_of_wal.pl is updated to use the refactored routines, without
changing its coverage.
Reviewed-by: Alexander Kukushkin
Discussion: https://postgr.es/m/CAFh8B=mozC+e1wGJq0H=0O65goZju+6ab5AU7DEWCSUA2OtwDg@mail.gmail.com
Backpatch-through: 13
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/test/recovery/t/039_end_of_wal.pl
Fix cpluspluscheck for "Change gist stratnum function to use CompareType"
commit : d5221c49a302ee7f3964449cbc730b01ce4e7d40
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 23:11:08 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 23:11:08 +0100
Commit 630f9a43cec introduced an enum forward declaration, which
doesn't work in C++. To fix, just include the header file to get the
type.
M src/include/access/gist.h
Add more general summary to vacuumlazy.c
commit : 3edc67d337c2e498dad1cd200e460f7c63e512e6
author : Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 15 Jan 2025 14:16:38 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 15 Jan 2025 14:16:38 -0500
Add more comments at the top of vacuumlazy.c on heap relation vacuuming
implementation.
Previously vacuumlazy.c only had details related to dead TID storage.
This commit adds a more general summary to help future developers
understand the heap relation vacuum design and implementation at a high
level.
Reviewed-by: Alena Rybakina, Robert Haas, Andres Freund, Bilal Yavuz
Discussion: https://postgr.es/m/flat/CAAKRu_ZF_KCzZuOrPrOqjGVe8iRVWEAJSpzMgRQs%3D5-v84cXUg%40mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Add a bit of documentation related to IWYU
commit : 44512e7c952f798c9912476ac5fc2132493f9af7
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 18:57:53 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 18:57:53 +0100
Add some basic information about IWYU to src/tools/pginclude/README.
Discussion: https://www.postgresql.org/message-id/flat/9395d484-eff4-47c2-b276-8e228526c8ae@eisentraut.org
M src/tools/pginclude/README
IWYU pragmas for catalog headers
commit : fecc8021e1c851d2f180cc96521314a0daf2b30b
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 18:57:53 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 18:57:53 +0100
Add "IWYU pragma: export" annotations in each catalog header file so
that, for instance, including "catalog/pg_aggregate.h" is considered
acceptable in place of "catalog/pg_aggregate_d.h". This is very
common and it seems better to silence IWYU about it than trying to fix
this up.
Discussion: https://www.postgresql.org/message-id/flat/9395d484-eff4-47c2-b276-8e228526c8ae@eisentraut.org
M src/include/catalog/pg_aggregate.h
M src/include/catalog/pg_am.h
M src/include/catalog/pg_amop.h
M src/include/catalog/pg_amproc.h
M src/include/catalog/pg_attrdef.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_auth_members.h
M src/include/catalog/pg_authid.h
M src/include/catalog/pg_cast.h
M src/include/catalog/pg_class.h
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_conversion.h
M src/include/catalog/pg_database.h
M src/include/catalog/pg_db_role_setting.h
M src/include/catalog/pg_default_acl.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_description.h
M src/include/catalog/pg_enum.h
M src/include/catalog/pg_event_trigger.h
M src/include/catalog/pg_extension.h
M src/include/catalog/pg_foreign_data_wrapper.h
M src/include/catalog/pg_foreign_server.h
M src/include/catalog/pg_foreign_table.h
M src/include/catalog/pg_index.h
M src/include/catalog/pg_inherits.h
M src/include/catalog/pg_init_privs.h
M src/include/catalog/pg_language.h
M src/include/catalog/pg_largeobject.h
M src/include/catalog/pg_largeobject_metadata.h
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_opclass.h
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_opfamily.h
M src/include/catalog/pg_parameter_acl.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_policy.h
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_publication.h
M src/include/catalog/pg_publication_namespace.h
M src/include/catalog/pg_publication_rel.h
M src/include/catalog/pg_range.h
M src/include/catalog/pg_replication_origin.h
M src/include/catalog/pg_rewrite.h
M src/include/catalog/pg_seclabel.h
M src/include/catalog/pg_sequence.h
M src/include/catalog/pg_shdepend.h
M src/include/catalog/pg_shdescription.h
M src/include/catalog/pg_shseclabel.h
M src/include/catalog/pg_statistic.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/catalog/pg_subscription.h
M src/include/catalog/pg_subscription_rel.h
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_transform.h
M src/include/catalog/pg_trigger.h
M src/include/catalog/pg_ts_config.h
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_ts_parser.h
M src/include/catalog/pg_ts_template.h
M src/include/catalog/pg_type.h
M src/include/catalog/pg_user_mapping.h
IWYU widely useful pragmas
commit : 74938d132093bc0ad99a5420da2e99edaa3734f9
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 18:57:53 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 18:57:53 +0100
Add various widely useful "IWYU pragma" annotations, such as
- Common header files such as c.h, postgres.h should be "always_keep".
- System headers included in c.h, postgres.h etc. should be considered
"export".
- Some portability headers such as getopt_long.h should be
"always_keep", so they are not considered superfluous on some
platforms.
- Certain system headers included from portability headers should be
considered "export" because the purpose of the portability header is
to wrap them.
- Superfluous includes marked as "for backward compatibility" get a
formal IWYU annotation.
- Generated header included in utils/syscache.h is marked exported.
This is a very commonly used include and this avoids lots of
complaints.
Discussion: https://www.postgresql.org/message-id/flat/9395d484-eff4-47c2-b276-8e228526c8ae@eisentraut.org
M src/include/c.h
M src/include/getopt_long.h
M src/include/pg_getopt.h
M src/include/pg_trace.h
M src/include/pgstat.h
M src/include/port/pg_iovec.h
M src/include/port/pg_pthread.h
M src/include/postgres.h
M src/include/postgres_ext.h
M src/include/postgres_fe.h
M src/include/utils/syscache.h
M src/interfaces/libpq/libpq-int.h
postgres_fdw: SCRAM authentication pass-through
commit : 761c79508e7fbc33c1b11754bdde4bd03ce9cbb3
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 17:55:18 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 17:55:18 +0100
This enables SCRAM authentication for postgres_fdw when connecting to
a foreign server without having to store a plain-text password on user
mapping options.
This is done by saving the SCRAM ClientKey and ServeryKey from the
client authentication and using those instead of the plain-text
password for the server-side SCRAM exchange. The new foreign-server
or user-mapping option "use_scram_passthrough" enables this.
Co-authored-by: Matheus Alcantara <mths.dev@pm.me>
Co-authored-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/27b29a35-9b96-46a9-bc1a-914140869dac@gmail.com
M contrib/postgres_fdw/Makefile
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/meson.build
M contrib/postgres_fdw/option.c
A contrib/postgres_fdw/t/001_auth_scram.pl
M doc/src/sgml/libpq.sgml
M doc/src/sgml/postgres-fdw.sgml
M src/backend/libpq/auth-scram.c
M src/include/libpq/libpq-be.h
M src/interfaces/libpq/fe-auth-scram.c
M src/interfaces/libpq/fe-auth.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/libpq-int.h
Downgrade error in object_aclmask_ext() to internal
commit : b6463ea6ef3e46b32be96a23f3a9f47357847ce4
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 16:53:53 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 16:53:53 +0100
The "does not exist" error in object_aclmask_ext() was written as
ereport(), suggesting that it is user-facing. This is problematic:
get_object_class_descr() is meant to be for internal errors only and
does not support translation.
For the has_xxx_privilege functions, the error has not been
user-facing since commit 403ac226ddd. The remaining users are
pg_database_size() and pg_tablespace_size(). The call stack here is
pretty deep and this dependency is not obvious. Here we can put in an
explicit existence check with a bespoke error message early in the
function.
Then we can downgrade the error in object_aclmask_ext() to a normal
"cache lookup failed" internal error.
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/flat/da2f8942-be6d-48d0-ac1c-a053370a6b1f@eisentraut.org
M src/backend/catalog/aclchk.c
M src/backend/utils/adt/dbsize.c
Downgrade errors in object_ownercheck() to internal
commit : de9037d0d085f25d3c220a45bfbb7abc5794a0c8
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 16:53:53 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 16:53:53 +0100
The "does not exist" errors in object_ownership() were written as
ereport(), suggesting that they are user-facing. But no code path
except one can reach this function without first checking that the
object exists. If this were actually a user-facing error message,
then there would be some problems: get_object_class_descr() is meant
to be for internal errors only and does not support translation.
The one case that can reach this without first checking the object
existence is from be_lo_unlink(). (This makes some sense since large
objects are referred to by their OID directly.) In this one case, we
can add a line of code to check the object existence explicitly,
consistent with other LO code.
For the rest, downgrade the error messages to elog()s. The new
message wordings are the same as in DropObjectById().
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/flat/da2f8942-be6d-48d0-ac1c-a053370a6b1f@eisentraut.org
M src/backend/catalog/aclchk.c
M src/backend/libpq/be-fsstubs.c
Drop warning-free support for Flex 2.5.35
commit : 6fdd5d9563439285df200f7282f0ca68152ef72d
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 15:35:08 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 15:35:08 +0100
This removes all the various workarounds for avoiding compiler
warnings with Flex 2.5.35. Several recent patches have added
additional warnings that would either need to be fixed along the lines
of the existing workarounds, or we decide to no longer care about
this, which we do here.
Flex 2.5.35 is extremely outdated, and you can't even download it
anymore from any of the Flex project sites, so it's nearly impossible
to support.
After this, using Flex 2.5.35 will still work, but the generated code
will produce numerous compiler warnings.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/1a204ccd-7ae6-478c-a431-407b5c48ccc6@eisentraut.org
M src/Makefile.global.in
M src/backend/parser/Makefile
M src/backend/parser/meson.build
M src/backend/parser/scan.l
M src/bin/pgbench/exprscan.l
M src/bin/psql/Makefile
M src/bin/psql/meson.build
M src/bin/psql/psqlscanslash.l
M src/fe_utils/Makefile
M src/fe_utils/meson.build
M src/fe_utils/psqlscan.l
D src/tools/fix-old-flex-code.pl
M src/tools/pgflex
Change gist stratnum function to use CompareType
commit : 630f9a43cece93cb4a5c243b30e34abce6a89514
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 11:28:39 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 11:28:39 +0100
This changes commit 7406ab623fe in that the gist strategy number
mapping support function is changed to use the CompareType enum as
input, instead of the "well-known" RT*StrategyNumber strategy numbers.
This is a bit cleaner, since you are not dealing with two sets of
strategy numbers. Also, this will enable us to subsume this system
into a more general system of using CompareType to define operator
semantics across index methods.
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M contrib/btree_gist/btree_gist–1.7–1.8.sql
M contrib/btree_gist/btree_gist.c
M contrib/btree_gist/expected/stratnum.out
M contrib/btree_gist/sql/stratnum.sql
M doc/src/sgml/gist.sgml
M doc/src/sgml/xindex.sgml
M src/backend/access/gist/gistutil.c
M src/backend/access/gist/gistvalidate.c
M src/backend/catalog/pg_constraint.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execReplication.c
M src/include/access/gist.h
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_proc.dat
M src/include/commands/defrem.h
M src/include/nodes/primnodes.h
M src/test/regress/expected/misc_functions.out
M src/test/regress/sql/misc_functions.sql
Rename RowCompareType to CompareType
commit : 6339f6468e8217f556e38482626250dc72d7cd00
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 08:31:46 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jan 2025 08:31:46 +0100
RowCompareType served as a way to describe the fundamental meaning of
an operator, notionally independent of an operator class (although so
far this was only really supported for btrees). Its original purpose
was for use inside RowCompareExpr, and it has also found some small
use outside, such as for get_op_btree_interpretation().
We want to expand this now, as a more general way to describe operator
semantics for other index access methods, including gist (to improve
GistTranslateStratnum()) and others not written yet. To avoid future
confusion, we rename the type to CompareType and the symbols from
ROWCOMPARE_XXX to COMPARE_XXX to reflect their more general purpose.
Reviewed-by: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/nodeIndexscan.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/util/predtest.c
M src/backend/parser/parse_expr.c
M src/backend/utils/cache/lsyscache.c
M src/include/executor/execExpr.h
M src/include/nodes/primnodes.h
M src/tools/pgindent/typedefs.list
Avoid symbol collisions between pqsignal.c and legacy-pqsignal.c.
commit : 9a45a89c38f3257b13e09edf382e32fa28b918c2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 14 Jan 2025 18:50:24 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 14 Jan 2025 18:50:24 -0500
In the name of ABI stability (that is, to avoid a library major
version bump for libpq), libpq still exports a version of pqsignal()
that we no longer want to use ourselves. However, since that has
the same link name as the function exported by src/port/pqsignal.c,
there is a link ordering dependency determining which version will
actually get used by code that uses libpq as well as libpgport.a.
It now emerges that the wrong version has been used by pgbench and
psql since commit 06843df4a rearranged their link commands. This
can result in odd failures in pgbench with the -T switch, since its
SIGALRM handler will now not be marked SA_RESTART. psql may have
some edge-case problems in \watch, too.
Since we don't want to depend on link ordering effects anymore,
let's fix this in the same spirit as b6c7cfac8: use macros to change
the actual link names of the competing functions. We cannot change
legacy-pqsignal.c's exported name of course, so the victim has to be
src/port/pqsignal.c.
In master, rename its exported name to be pqsignal_fe in frontend or
pqsignal_be in backend. (We could perhaps have gotten away with using
the same symbol in both cases, but since the FE and BE versions now
work a little differently, it seems advisable to use different names.)
In back branches, rename to pqsignal_fe in frontend but keep it as
pqsignal in backend. The frontend change could affect third-party
code that is calling pqsignal from libpgport.a or libpgport_shlib.a,
but only if the code is compiled against port.h from a different minor
release than libpgport. Since we don't support using libpgport as a
shared library, it seems unlikely that there will be such a problem.
I left the backend symbol unchanged to avoid an ABI break for
extensions. This means that the link ordering hazard still exists
for any extension that links against libpq. However, none of our own
extensions use both pqsignal() and libpq, and we're not making things
any worse for third-party extensions that do.
Report from Andy Fan, diagnosis by Fujii Masao, patch by me.
Back-patch to all supported branches, as 06843df4a was.
Discussion: https://postgr.es/m/87msfz5qv2.fsf@163.com
M src/include/port.h
M src/interfaces/libpq/legacy-pqsignal.c
M src/port/pqsignal.c
Synchronize guc_tables.c categories with vacuum docs categories
commit : 2ae98ea5ab8f22ece06bb206e6a298ecc5770ed4
author : Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 14 Jan 2025 15:30:13 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 14 Jan 2025 15:30:13 -0500
ca9c6a5680d consolidated most of the vacuum-related GUCs' documentation
into a new subsection. af2317652d5daf8b then enforced this order in
postgresql.conf.sample. This commit reorganizes the GUC groups in
guc_tables.c/h to match the updated ordering in the docs.
Reported-by: Álvaro Herrera
Reviewed-by: Álvaro Herrera, Alena Rybakina
Discussion: https://postgr.es/m/202501132046.m4mcvxxswznu%40alvherre.pgsql
M src/backend/utils/misc/guc_tables.c
M src/include/utils/guc_tables.h
psql: Add option to use expanded mode to all list commands.
commit : 00f4c2959d631c7851da21a512885d1deab28649
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 14 Jan 2025 16:29:15 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 14 Jan 2025 16:29:15 +0000
This allows "x" to be appended to any psql list-like meta-command,
forcing its output to be displayed in expanded mode. This improves
readability in cases where the output is very wide. For example,
"\dfx+" (or equivalently "\df+x") will produce a list of functions,
with additional details, in expanded mode.
This works with all \d* meta-commands, plus \l, \z, and \lo_list, with
the one exception that the expanded mode option "x" cannot be appended
to "\d" by itself, since "\dx" already means something else.
Dean Rasheed, reviewed by Greg Sabino Mullane.
Discussion: https://postgr.es/m/CAEZATCVXJk3KsmCncf7PAVbxdDAUDm3QzDgGT7mBYySWikuOYw@mail.gmail.com
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/help.c
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
ecpg: Restore detection of unsupported COPY FROM STDIN.
commit : 94b914f601bd15829df6f83c0246342ca92630b1
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 15 Jan 2025 01:23:02 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 15 Jan 2025 01:23:02 +0900
The ecpg command includes code to warn about unsupported COPY FROM STDIN
statements in input files. However, since commit 3d009e45bd,
this functionality has been broken due to a bug introduced in that commit,
causing ecpg to fail to detect the statement.
This commit resolves the issue, restoring ecpg's ability to detect
COPY FROM STDIN and issue a warning as intended.
Back-patch to all supported versions.
Author: Ryo Kanbayashi
Reviewed-by: Hayato Kuroda, Tom Lane
Discussion: https://postgr.es/m/CANOn0Ez_t5uDCUEV8c1YORMisJiU5wu681eEVZzgKwOeiKhkqQ@mail.gmail.com
M src/interfaces/ecpg/preproc/ecpg.addons
Consistently spell "leakproof" without a hyphen.
commit : 4cb560b53f8f9e9a94d593f9416c107ed8202110
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 14 Jan 2025 13:50:54 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 14 Jan 2025 13:50:54 +0000
The overwhelming majority of places already did this, but a small
handful of places had a hyphen.
Yugo Nagata.
Discussion: https://postgr.es/m/CAEZATCXnnuORE2BoGwHw2zbtVvsPOLhbfVmEk9GxRzK%2Bx3OW-Q%40mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/planstats.sgml
M doc/src/sgml/rules.sgml
M src/backend/statistics/extended_stats.c
M src/backend/utils/adt/selfuncs.c
M src/include/catalog/pg_proc.h
psql: Add leakproof indicator to \df+, \do+, \dAo+, and \dC+ output.
commit : 2355e51110e7c687c125a5958f12a462931de996
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 14 Jan 2025 13:23:24 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 14 Jan 2025 13:23:24 +0000
This allows users to determine whether particular functions are
leakproof, and whether the underlying functions used by operators and
casts are leakproof. This is useful to determine whether indexes can
be used in queries on security barrier views or tables with row-level
security policies.
Yugo Nagata, reviewed by Erik Wienhold and Dean Rasheed.
Discussion: https://postgr.es/m/20240701220817.483f9b645b95611f8b1f65da%40sranhm.sraoss.co.jp
M doc/src/sgml/planstats.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/rules.sgml
M src/bin/psql/describe.c
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
Fix catcache invalidation of a list entry that's being built
commit : af8cd1639ab298ba8fae62cd9583f1c10a5068e1
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 14 Jan 2025 14:28:49 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 14 Jan 2025 14:28:49 +0200
If a new catalog tuple is inserted that belongs to a catcache list
entry, and cache invalidation happens while the list entry is being
built, the list entry might miss the newly inserted tuple.
To fix, change the way we detect concurrent invalidations while a
catcache entry is being built. Keep a stack of entries that are being
built, and apply cache invalidation to those entries in addition to
the real catcache entries. This is similar to the in-progress list in
relcache.c.
Back-patch to all supported versions.
Reviewed-by: Noah Misch
Discussion: https://www.postgresql.org/message-id/2234dc98-06fe-42ed-b5db-ac17384dc880@iki.fi
M src/backend/utils/cache/catcache.c
M src/backend/utils/cache/inval.c
M src/include/utils/catcache.h
M src/test/modules/test_misc/meson.build
A src/test/modules/test_misc/t/007_catcache_inval.pl
M src/tools/pgindent/typedefs.list
Bump PGSTAT_FILE_FORMAT_ID
commit : ce9a74707d4cf7768cff06298d09c7f7e823341d
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jan 2025 15:17:22 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jan 2025 15:17:22 +0900
Oversight in f92c854cf406, that has changed the definition of
PgStat_BktypeIO, impacting PgStat_IO which is the on-disk data for IO
pgstats data.
M src/include/pgstat.h
Fix potential integer overflow in bringetbitmap()
commit : 720e529840d5a1087a34db445f624c5ece46eb38
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jan 2025 15:12:56 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jan 2025 15:12:56 +0900
This function expects an "int64" as result and stores the number of
pages to add to the index scan bitmap as an "int", multiplying its final
result by 10. For a relation large enough, this can theoretically
overflow if counting more than (INT32_MAX / 10) pages, knowing that the
number of pages is upper-bounded by MaxBlockNumber.
To avoid the overflow, this commit redefines "totalpages", used to
calculate the result, to be an "int64" rather than an "int".
Reported-by: Evgeniy Gorbanyov
Author: James Hunter
Discussion: https://www.postgresql.org/message-id/07704817-6fa0-460c-b1cf-cd18f7647041@basealt.ru
Backpatch-through: 13
M src/backend/access/brin/brin.c
Move information about pgstats kinds into its own header pgstat_kind.h
commit : d35ea27e51c05cbe3575d50a6b99d64f20a3a742
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jan 2025 12:43:07 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jan 2025 12:43:07 +0900
This includes all the definitions for the various PGSTAT_KIND_* values,
the range allowed for custom stats kinds and some macros related all
that.
One use-case behind this split is the possibility to use this
information for frontend tools, without having to rely on pgstat.h and a
backend footprint.
Author: Michael Paquier
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Z24fyb3ipXKR38oS@paquier.xyz
M src/include/pgstat.h
A src/include/utils/pgstat_kind.h
Remove assertion in pgstat_count_io_op()
commit : d2181b321852d4dbea8a909d2f039ebdcbf0e009
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jan 2025 12:19:51 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jan 2025 12:19:51 +0900
An equivalent check is done with pgstat_is_ioop_tracked_in_bytes(), so
there is no need for this extra one. Small cleanup that should have
been included in f92c854cf406.
Author: Nazir Bilal Yavuz
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/CAN55FZ0oqxBaaHAEsj=xFqkzE3n5P=3RA1V_igXwL-RV7QRzyw@mail.gmail.com
M src/backend/utils/activity/pgstat_io.c
Make pg_stat_io count IOs as bytes instead of blocks for some operations
commit : f92c854cf406a5ad34c9aa92416d578819704aa2
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jan 2025 12:14:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jan 2025 12:14:29 +0900
Currently in pg_stat_io view, IOs are counted as blocks of size
BLCKSZ. There are two limitations with this design:
* The actual number of I/O requests sent to the kernel is lower because
I/O requests may be merged before being sent. Additionally, it gives
the impression that all I/Os are done in block size, which shadows the
benefits of merging I/O requests.
* Some patches are under work to extend pg_stat_io for the tracking of
operations that may not be linked to the block size. For example, WAL
read IOs are done in variable bytes and it is not possible to correctly
show these IOs in pg_stat_io view, and we want to keep all this data in
a single system view rather than spread it across multiple relations to
ease monitoring.
WaitReadBuffers() can now be tracked as a single read operation
worth N blocks. Same for ExtendBufferedRelShared() and
ExtendBufferedRelLocal() for extensions.
Three columns are added to pg_stat_io for reads, writes and extensions
for the byte calculations. op_bytes, which was always hardcoded to
BLCKSZ, is removed. IO backend statistics are updated to reflect these
changes.
Bump catalog version.
Author: Nazir Bilal Yavuz
Reviewed-by: Bertrand Drouvot, Melanie Plageman
Discussion: https://postgr.es/m/CAN55FZ0oqxBaaHAEsj=xFqkzE3n5P=3RA1V_igXwL-RV7QRzyw@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/backend/storage/smgr/md.c
M src/backend/utils/activity/pgstat_backend.c
M src/backend/utils/activity/pgstat_io.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
Revert "TupleHashTable: store additional data along with tuple."
commit : b4a07f532b40a64fb4714a3f7ab6063435411edb
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 13 Jan 2025 14:14:07 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 13 Jan 2025 14:14:07 -0800
This reverts commit e0ece2a981ee9068f50c4423e303836c2585eb02 due to
performance regressions.
Reported-by: David Rowley
M src/backend/executor/execGrouping.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeSetOp.c
M src/backend/executor/nodeSubplan.c
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
Reorder vacuum GUCs in postgresql.conf.sample to match docs
commit : af2317652d5daf8b382cc65936731c4a3c0aaa4c
author : Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 13 Jan 2025 15:21:04 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Mon, 13 Jan 2025 15:21:04 -0500
ca9c6a5680d consolidated most of vacuum-related GUCs' documentation into
a new subsection. It neglected, however, to reorganize
postgresql.conf.sample to match the new order. Do this now.
Reported-by: Álvaro Herrera
Discussion: https://postgr.es/m/202501110902.5banlseavz7c%40alvherre.pgsql
M src/backend/utils/misc/postgresql.conf.sample
Add BTOPTIONS_PROC comments to nbtree.h.
commit : 1c854eb8931a8ce2ba8e2cc4d0a209a45d836de5
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 13 Jan 2025 15:02:14 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 13 Jan 2025 15:02:14 -0500
Add comments explaining the purpose of B-Tree support function 5 to
nbtree.h for consistency (all other support functions were already
described by nearby comments).
This fixes what was arguably an oversight in commit 911e702077, or in
follow-up doc commit 15cb2bd2 (which documented support function 5 in
btree.sgml, but neglected to add anything to nbtree.h).
M src/include/access/nbtree.h
Move nbtree preprocessing into new .c file.
commit : 597b1ffbf12352a3863a894f16741864aaf2242f
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 13 Jan 2025 12:15:00 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 13 Jan 2025 12:15:00 -0500
Quite a bit of code within nbtutils.c is only called during nbtree
preprocessing. Move that code into a new .c file, nbtpreprocesskeys.c.
Also reorder some of the functions within the new file for clarity.
This commit has no functional impact. It is strictly mechanical.
Author: Peter Geoghegan <pg@bowt.ie>
Suggested-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/CAH2-WznwNn1BDOpWxHBUK1f3Rdw8pO9UCenWXnvT=n9GO8GnLA@mail.gmail.com
Discussion: https://postgr.es/m/86930045-5df5-494a-b4f1-815bc3fbcce0%40iki.fi
M src/backend/access/nbtree/Makefile
M src/backend/access/nbtree/meson.build
A src/backend/access/nbtree/nbtpreprocesskeys.c
M src/backend/access/nbtree/nbtutils.c
M src/include/access/nbtree.h
Add commit 6e826278f1 to .git-blame-ignore-revs.
commit : a8a762bc46e43bfaf4c03fc50f805fdf53f11888
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 13 Jan 2025 09:37:32 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 13 Jan 2025 09:37:32 -0600
M .git-blame-ignore-revs
Fix pgindent damage
commit : 6e826278f1ebd9967c0f8adda29c8960a812e344
author : Richard Guo <rguo@postgresql.org>
date : Mon, 13 Jan 2025 11:27:32 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Mon, 13 Jan 2025 11:27:32 +0900
Oversight in commit e0ece2a98.
M src/backend/executor/execGrouping.c
M src/include/nodes/execnodes.h
Fix HBA option count
commit : 97698cc517c75d3cbb9c680aab3afd36e471b83a
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sun, 12 Jan 2025 23:44:39 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sun, 12 Jan 2025 23:44:39 +0100
Commit 27a1f8d108 missed updating the max HBA option count to
account for the new option added. Fix by bumping the counter
and adjust the relevant comment to match. Backpatch down to
all supported branches like the erroneous commit.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/286764.1736697356@sss.pgh.pa.us
Backpatch-through: v13
M src/backend/utils/adt/hbafuncs.c
Fix JsonExpr deparsing to quote variable names in the PASSING clause.
commit : a93e2a1e25a6d5410abb1446637c4d9a4f24e35d
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sun, 12 Jan 2025 13:35:12 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sun, 12 Jan 2025 13:35:12 +0000
When deparsing a JsonExpr, variable names in the PASSING clause were
not quoted. However, since they are parsed as ColLabel tokens, some
variable names require double quotes to ensure that they are properly
interpreted. Fix by using quote_identifier() in the deparsing code.
This oversight was limited to the SQL/JSON query functions
JSON_EXISTS(), JSON_QUERY(), and JSON_VALUE().
Back-patch to v17, where these functions were added.
Dean Rasheed, reviewed by Tom Lane.
Discussion: https://postgr.es/m/CAEZATCXTpAS%3DncfLNTZ7YS6O5puHeLg_SUYAit%2Bcs7wsrd9Msg%40mail.gmail.com
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/sqljson_queryfuncs.out
M src/test/regress/sql/sqljson_queryfuncs.sql
Fix XMLTABLE() deparsing to quote namespace names if necessary.
commit : d673eefd410cd7f5f95e418990146189f44811af
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sun, 12 Jan 2025 12:54:32 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sun, 12 Jan 2025 12:54:32 +0000
When deparsing an XMLTABLE() expression, XML namespace names were not
quoted. However, since they are parsed as ColLabel tokens, some names
require double quotes to ensure that they are properly interpreted.
Fix by using quote_identifier() in the deparsing code.
Back-patch to all supported versions.
Dean Rasheed, reviewed by Tom Lane.
Discussion: https://postgr.es/m/CAEZATCXTpAS%3DncfLNTZ7YS6O5puHeLg_SUYAit%2Bcs7wsrd9Msg%40mail.gmail.com
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/xml.out
M src/test/regress/expected/xml_1.out
M src/test/regress/expected/xml_2.out
M src/test/regress/sql/xml.sql
Repair memory leaks in plpython.
commit : 29dfffae0a6db6cb880ae873169f5512ddab703d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 11 Jan 2025 11:45:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 11 Jan 2025 11:45:56 -0500
PLy_spi_execute_plan (PLyPlan.execute) and PLy_cursor_plan
(plpy.cursor) use PLy_output_convert to convert Python values
into Datums that can be passed to the query-to-execute. But they
failed to pay much attention to its warning that it can leave "cruft
generated along the way" behind. Repeated use of these methods can
result in a substantial memory leak for the duration of the calling
plpython function.
To fix, make a temporary memory context to invoke PLy_output_convert
in. This also lets us get rid of the rather fragile code that was
here for retail pfree's of the converted Datums. Indeed, we don't
need the PLyPlanObject.values field anymore at all, though I left it
in place in the back branches in the name of ABI stability.
Mat Arye and Tom Lane, per report from Mat Arye. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/CADsUR0DvVgnZYWwnmKRK65MZg7YLUSTDLV61qdnrwtrAJgU6xw@mail.gmail.com
M src/pl/plpython/plpy_cursorobject.c
M src/pl/plpython/plpy_planobject.c
M src/pl/plpython/plpy_planobject.h
M src/pl/plpython/plpy_spi.c
Add support for NOT ENFORCED in CHECK constraints
commit : ca87c415e2fccf81cec6fd45698dde9fae0ab570
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 11 Jan 2025 10:45:17 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 11 Jan 2025 10:45:17 +0100
This adds support for the NOT ENFORCED/ENFORCED flag for constraints,
with support for check constraints.
The plan is to eventually support this for foreign key constraints,
where it is typically more useful.
Note that CHECK constraints do not currently support ALTER operations,
so changing the enforceability of an existing constraint isn't
possible without dropping and recreating it. This could be added
later.
Author: Amul Sul <amul.sul@enterprisedb.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: jian he <jian.universality@gmail.com>
Tested-by: Triveni N <triveni.n@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/CAAJ_b962c5AcYW9KUt_R_ER5qs3fUGbe4az-SP-vuwPS-w-AGA@mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/information_schema.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_foreign_table.sgml
M doc/src/sgml/ref/create_table.sgml
M src/backend/access/common/tupdesc.c
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/information_schema.sql
M src/backend/catalog/pg_constraint.c
M src/backend/catalog/sql_features.txt
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/commands/typecmds.c
M src/backend/executor/execMain.c
M src/backend/nodes/makefuncs.c
M src/backend/optimizer/util/plancat.c
M src/backend/parser/gram.y
M src/backend/parser/parse_utilcmd.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/cache/relcache.c
M src/include/access/tupdesc.h
M src/include/catalog/catversion.h
M src/include/catalog/heap.h
M src/include/catalog/pg_constraint.h
M src/include/nodes/parsenodes.h
M src/include/parser/kwlist.h
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/constraints.out
M src/test/regress/expected/create_table_like.out
M src/test/regress/expected/domain.out
M src/test/regress/expected/inherit.out
M src/test/regress/sql/alter_table.sql
M src/test/regress/sql/constraints.sql
M src/test/regress/sql/create_table_like.sql
M src/test/regress/sql/domain.sql
M src/test/regress/sql/inherit.sql
Fix a compiler warning in initStringInfo().
commit : 72ceb21b029433dd82f29182894dce63e639b4d4
author : Tatsuo Ishii <ishii@postgresql.org>
date : Sat, 11 Jan 2025 15:52:37 +0900
committer: Tatsuo Ishii <ishii@postgresql.org>
date : Sat, 11 Jan 2025 15:52:37 +0900
Fix a compiler warning found by Cfbot. This was caused by commit
bb86e85e442.
M src/common/stringinfo.c
Fix redefinition of type in commit e0ece2a981.
commit : ceb2855522940d5aaae4c6e5eed493d60e3196ce
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 10 Jan 2025 17:45:27 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 10 Jan 2025 17:45:27 -0800
M src/backend/executor/execGrouping.c
TupleHashTable: store additional data along with tuple.
commit : e0ece2a981ee9068f50c4423e303836c2585eb02
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 10 Jan 2025 17:14:37 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 10 Jan 2025 17:14:37 -0800
Previously, the caller needed to allocate the memory and the
TupleHashTable would store a pointer to it. That wastes space for the
palloc overhead as well as the size of the pointer itself.
Now, the TupleHashTable relies on the caller to correctly specify the
additionalsize, and allocates that amount of space. The caller can
then request a pointer into that space.
Discussion: https://postgr.es/m/b9cbf0219a9859dc8d240311643ff4362fd9602c.camel@j-davis.com
Reviewed-by: Heikki Linnakangas
M src/backend/executor/execGrouping.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeSetOp.c
M src/backend/executor/nodeSubplan.c
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
Make verify_compact_attribute available in non-assert builds
commit : 34c6e652425fde42c2746f749e31d196fc0d5538
author : David Rowley <drowley@postgresql.org>
date : Sat, 11 Jan 2025 13:45:54 +1300
committer: David Rowley <drowley@postgresql.org>
date : Sat, 11 Jan 2025 13:45:54 +1300
6f3820f37 adjusted the assert-enabled validation of the CompactAttribute
to call a new external function to perform the validation. That commit
made it so the function was only available when building with
USE_ASSERT_CHECKING, and because TupleDescCompactAttr() is a static
inline function, the call to verify_compact_attribute() was compiled
into any extension which uses TupleDescCompactAttr(). This caused issues
for such extensions when loading the assert-enabled extension into
PostgreSQL versions without asserts enabled due to that function being
unavailable in core.
To fix this, make verify_compact_attribute() available unconditionally,
but make it do nothing unless building with USE_ASSERT_CHECKING.
Author: Andrew Kane <andrew@ankane.org>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAOdR5yHfMEMW00XGo=v1zCVUS6Huq2UehXdvKnwtXPTcZwXhmg@mail.gmail.com
M src/backend/access/common/tupdesc.c
M src/include/access/tupdesc.h
Add new StringInfo APIs to allow callers to specify the buffer size.
commit : a9dcbb4d5c00b703e727e14055ed70c525a86418
author : Tatsuo Ishii <ishii@postgresql.org>
date : Sat, 11 Jan 2025 08:02:56 +0900
committer: Tatsuo Ishii <ishii@postgresql.org>
date : Sat, 11 Jan 2025 08:02:56 +0900
Previously StringInfo APIs allocated buffers with fixed initial
allocation size of 1024 bytes. This may be too large and inappropriate
for some callers that can do with smaller memory buffers. To fix this,
introduce new APIs that allow callers to specify initial buffer size.
extern StringInfo makeStringInfoExt(int initsize);
extern void initStringInfoExt(StringInfo str, int initsize);
Existing APIs (makeStringInfo() and initStringInfo()) are changed to
call makeStringInfoExt and initStringInfoExt respectively (via inline
helper functions makeStringInfoInternal and initStringInfoInternal),
with the default buffer size of 1024.
Reviewed-by: Nathan Bossart, David Rowley, Michael Paquier, Gurjeet Singh
Discussion: https://postgr.es/m/20241225.123704.1194662271286702010.ishii%40postgresql.org
M src/common/stringinfo.c
M src/include/lib/stringinfo.h
Consolidate docs for vacuum-related GUCs in new subsection
commit : ca9c6a5680d7c7dece0f7209ee7ce20c9dfe0840
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 10 Jan 2025 18:20:16 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 10 Jan 2025 18:20:16 -0500
GUCs related to vacuum's freezing behavior were documented in a
subsection of the Client Connection Defaults documentation. These GUCs
don't belong there, as they affect the freezing behavior of all vacuums
-- including autovacuums.
There wasn't a clear alternative location, so this commit makes a new
"Server Configuration" docs subsection, "Vacuuming", with a subsection
for "Freezing". It also moves the "Automatic Vacuuming" subsection and
the docs on GUCs controlling cost-based vacuum delay under the new
"Vacuuming" subsection.
The other vacuum-related GUCs under the "Resource Consumption"
subsection have been left in their current location, as they seem to fit
there.
The GUCs' documentation was largely lifted and shifted. The only
modification made was the addition of a few missing <literal> tags.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/flat/CAAKRu_aQUOaMYrcjNuXeSkJtaX9oRUzKP57bsYbC0gVVWS%2BcbA%40mail.gmail.com
M doc/src/sgml/config.sgml
Fix missing ldapscheme option in pg_hba_file_rules()
commit : 27a1f8d108114ec463101a812f83542c257f8169
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 10 Jan 2025 22:02:58 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 10 Jan 2025 22:02:58 +0100
The ldapscheme option was missed when inspecing the HbaLine for
assembling rows for the pg_hba_file_rules function. Backpatch
to all supported versions.
Author: Laurenz Albe <laurenz.albe@cybertec.at>
Reported-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Bug: 18769
Discussion: https://postgr.es/m/18769-dd8610cbc0405172@postgresql.org
Backpatch-through: v13
M src/backend/utils/adt/hbafuncs.c
Fix obsolete nbtree README left link remarks.
commit : 5b14ec0a48cca3addfc7262c86e60475b7d84c02
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 10 Jan 2025 15:42:17 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 10 Jan 2025 15:42:17 -0500
Oversight in commit 1bd4bc85, which made nbtree backwards scans operate
off of a copy of each page's left link as of the time of its call to
_bt_readpage.
M src/backend/access/nbtree/README
Use a non-locking initial test in TAS_SPIN on AArch64.
commit : 3d0b4b1068018f624d5ef7c9f90b536ed58345b5
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 10 Jan 2025 13:18:04 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 10 Jan 2025 13:18:04 -0600
Our testing showed that this is helpful at sufficiently high
contention levels and doesn't hurt performance on smaller machines.
The new TAS_SPIN macro for AArch64 is identical to the ones added
for PPC and x86_64 (see commits bc2a050d40 and b03d196be0).
Reported-by: Salvatore Dipietro
Reviewed-by: Jingtang Zhang, Andres Freund
Tested-by: Tom Lane
Discussion: https://postgr.es/m/ZxgDEb_VpWyNZKB_%40nathan
M src/include/storage/s_lock.h
postmaster: Rename some shutdown related PMState phase names
commit : 28e7a9968e183a6b8dddaff5b513b0656ba349aa
author : Andres Freund <andres@anarazel.de>
date : Fri, 10 Jan 2025 11:08:17 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 10 Jan 2025 11:08:17 -0500
The previous names weren't particularly clear. Future patches will add more
shutdown phases, making it even more important to have understandable shutdown
phases.
Suggested-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/d2cd8fd3-396a-4390-8f0b-74be65e72899@iki.fi
M src/backend/postmaster/postmaster.c
M src/backend/replication/README
postmaster: Make btmask_add() variadic
commit : e84712c73820839abfaa29c20cee0d3cabbf8f1c
author : Andres Freund <andres@anarazel.de>
date : Fri, 10 Jan 2025 11:08:17 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 10 Jan 2025 11:08:17 -0500
Suggested-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/d2cd8fd3-396a-4390-8f0b-74be65e72899@iki.fi
M src/backend/postmaster/postmaster.c
postmaster: Introduce variadic btmask_all_except()
commit : 7e957cbb50aa425ef10a2fd0903595ae9e256d5f
author : Andres Freund <andres@anarazel.de>
date : Fri, 10 Jan 2025 11:08:17 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 10 Jan 2025 11:08:17 -0500
Upcoming patches would otherwise need btmask_all_except3().
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/w3z6w3g4aovivs735nk4pzjhmegntecesm3kktpebchegm5o53@aonnq2kn27xi
M src/backend/postmaster/postmaster.c
postmaster: Improve logging of signals sent by postmaster
commit : 40d4031abd0da3d84543b050e1ced2da775a3274
author : Andres Freund <andres@anarazel.de>
date : Fri, 10 Jan 2025 11:08:17 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 10 Jan 2025 11:08:17 -0500
Previously many, in some cases important, signals we never logged. In other
cases the signal name was only included numerically.
As part of this, change the debug log level the signal is logged at to DEBUG3,
previously some where DEBUG2, some DEBUG4.
Also move from direct use of kill() to signal the av launcher to
signal_child(). There doesn't seem to be a reason for directly using kill().
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/kgng5nrvnlv335evmsuvpnh354rw7qyazl73kdysev2cr2v5zu@m3cfzxicm5kp
M src/backend/postmaster/postmaster.c
postmaster: Update pmState via a wrapper function
commit : 7148cbbdc65d68167967e67ed155c7f1635bc8e7
author : Andres Freund <andres@anarazel.de>
date : Fri, 10 Jan 2025 11:08:17 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 10 Jan 2025 11:08:17 -0500
This makes logging of state changes easier - state transitions are now logged
at DEBUG1. Without that logging it was surprisingly hard to understand the
current state of the system while debugging.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/kgng5nrvnlv335evmsuvpnh354rw7qyazl73kdysev2cr2v5zu@m3cfzxicm5kp
M src/backend/postmaster/postmaster.c
Adjust signature of cluster_rel() and its subroutines
commit : cc811f92bac5c80253c8a22e43409722cab4c05b
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 10 Jan 2025 13:09:38 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 10 Jan 2025 13:09:38 +0100
cluster_rel() receives the OID of the relation to process, which it
opens and locks; but then its subroutine copy_table_data() also receives
the relation OID and opens it by itself. This is a bit wasteful. It's
better to have cluster_rel() receive the relation already open, and pass
it down to its subroutines as necessary; then cluster_rel closes the rel
before returning. This simplifies things.
But a better motivation to make this change is that a future command to
do logical-decoding-based "concurrent VACUUM FULL" will need to release
all locks on the relation (and possibly on the clustering index) at some
point. Since it makes little sense to keep the relation reference
without the lock, the cluster_rel() function will also close it (and
the index). With this arrangement, neither the function nor its
subroutines need open extra references, which, again, makes things simpler.
Author: Antonin Houska <ah@cybertec.at>
Discussion: https://postgr.es/m/82651.1720540558@antos
M src/backend/commands/cluster.c
M src/backend/commands/matview.c
M src/backend/commands/vacuum.c
M src/include/commands/cluster.h
Fix UNION planner datatype issue
commit : 23100645104f63bf7c35304f36f8a7d91f5ddd73
author : David Rowley <drowley@postgresql.org>
date : Fri, 10 Jan 2025 14:30:25 +1300
committer: David Rowley <drowley@postgresql.org>
date : Fri, 10 Jan 2025 14:30:25 +1300
66c0185a3 gave the planner the ability to have union child queries
provide the union planner with pre-sorted input so that UNION queries
could be more efficiently implemented using Merge Append.
That commit overlooked checking that the UNION target list and the union
child target list's types all match. In some corner cases, this could
result in the planner producing sorts using the sort operator of the
top-level UNION's target list type rather than of the union child's
target list's type. The implications of this range from silently
working correctly, despite using the wrong sort operator all the way up
to a segmentation fault.
Here we fix by adjusting the planner so it makes no attempt to have the
subquery produce pre-sorted results when the data type of the UNION
target list and the types from the subquery target list don't match
exactly.
Backpatch to 17, where 66c0185a3 was introduced.
Reported-by: Jason Smith <dqetool@126.com>
Diagnosed-by: Tom Lane <tgl@sss.pgh.pa.us>
Bug: 18764
Discussion: https://postgr.es/m/18764-63ad667ea26e877a%40postgresql.org
Backpatch-through: 17
M src/backend/optimizer/plan/planner.c
Merge pgstat_count_io_op_n() and pgstat_count_io_op()
commit : f0bf7857befefa3ff00c7d2ac5c611c525f5cd3e
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 10 Jan 2025 09:57:27 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 10 Jan 2025 09:57:27 +0900
The pgstat_count_io_op() function, which counts a single I/O operation,
wraps pgstat_count_io_op_n() with a counter value of 1. The latter is
declared in pgstat.h and used nowhere in the code, so let's remove it in
favor of the former.
This change makes also the code more symmetric with
pgstat_count_io_op_time(), that already uses a similar set of arguments,
except that it counts also the I/O time. This will ease a bit the
integration of a follow-up patch that adds byte-level tracking in
pg_stat_io for some of its attributes, lifting the current restriction
based on BLCKSZ as all I/O operations are assumed to be block-based.
Author: Nazir Bilal Yavuz
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/CAN55FZ32ze812=yjyZg1QeXhKvACUM_Nu0_gyPQcUKKuVHL5xA@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/backend/utils/activity/pgstat_io.c
M src/include/pgstat.h
Refactor some code related to backend statistics
commit : 2c14037bb57c091b9f0bcbd36fa62138601beb55
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 10 Jan 2025 09:00:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 10 Jan 2025 09:00:48 +0900
This commit changes the way pending backend statistics are tracked by
moving them into a new structure called PgStat_BackendPending, removing
PgStat_BackendPendingIO. PgStat_BackendPending currently only includes
PgStat_PendingIO for the pending I/O stats.
pgstat_flush_backend() is extended with a "flags" argument to control
which parts of the stats of a backend should be flushed.
With this refactoring, it becomes easier to plug into backend statistics
more data. A patch to add information related to WAL in this stats kind
is under discussion.
Author: Bertrand Drouvot
Discussion: https://postgr.es/m/Z3zqc4o09dM/Ezyz@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/utils/activity/pgstat.c
M src/backend/utils/activity/pgstat_backend.c
M src/backend/utils/activity/pgstat_io.c
M src/backend/utils/activity/pgstat_relation.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/pgstat.h
M src/include/utils/pgstat_internal.h
M src/tools/pgindent/typedefs.list
Fix an ALTER GROUP ... DROP USER error message.
commit : 39e3bcae44868734504c3decbb6708c36a5a84b2
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 9 Jan 2025 17:10:13 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 9 Jan 2025 17:10:13 -0600
This error message stated the privileges required to add a member
to a group even if the user was trying to drop a member:
postgres=> alter group a drop user b;
ERROR: permission denied to alter role
DETAIL: Only roles with the ADMIN option on role "a" may add members.
Since the required privileges for both operations are the same, we
can fix this by modifying the message to mention both adding and
dropping members:
postgres=> alter group a drop user b;
ERROR: permission denied to alter role
DETAIL: Only roles with the ADMIN option on role "a" may add or drop members.
Author: ChangAo Chen
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/tencent_FAA0D00E3514AAF0BBB6322542A6094FEF05%40qq.com
Backpatch-through: 16
M src/backend/commands/user.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Use @extschema:name@ notation in contrib transform modules.
commit : bebe9040388bb2292585eab712fe4d29a71843fb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 9 Jan 2025 15:16:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 9 Jan 2025 15:16:56 -0500
Harden hstore_plperl, hstore_plpython, and ltree_plpython
against search-path-based attacks by using @extschema:name@
notation to refer to the underlying hstore or ltree data type.
This allows removal of the previous documentation warning
suggesting that they must be installed in the same schema as
the underlying data type. In passing, also improve a para in
extend.sgml to suggest using @extschema:name@ for such purposes.
Discussion: https://postgr.es/m/692480.1736021695@sss.pgh.pa.us
M contrib/hstore_plperl/hstore_plperl–1.0.sql
M contrib/hstore_plperl/hstore_plperlu–1.0.sql
M contrib/hstore_plpython/hstore_plpython3u–1.0.sql
M contrib/ltree_plpython/ltree_plpython3u–1.0.sql
M doc/src/sgml/extend.sgml
M doc/src/sgml/hstore.sgml
M doc/src/sgml/ltree.sgml
Simplify signature of RewriteTable
commit : ebd8fc7e47fdad6adb68aad341d95c541d7325c3
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 9 Jan 2025 14:17:12 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 9 Jan 2025 14:17:12 +0100
This function doesn't need the lockmode to be passed: it was being used
to lock the new heap, but that's bogus, because the only caller has
already obtained the appropriate lock on the new heap (which is
unimportant anyway, because the relation's creation is not yet committed
and so no other session can see it).
Noticed while reviewed Antonin Houska's patch to add VACUUM FULL
CONCURRENTLY.
M src/backend/commands/tablecmds.c
doc: Clarify synchronous_standby_names parameter.
commit : 6313a76b355e3640aba4cc80456a65c2bbc55a80
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 9 Jan 2025 21:04:49 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 9 Jan 2025 21:04:49 +0900
The synchronous_standby_names GUC allows specifying num_sync,
the number of synchronous standbys transactions must wait for
replies from. This value must be an integer greater than zero.
This commit updates the documentation to clarify this requirement.
Reported-by: Asphator <asphator@gmail.com>
Discussion: https://postgr.es/m/18663-b02f75cb919f1b60@postgresql.org
M doc/src/sgml/config.sgml
Fix SLRU bank selection code
commit : 69ab44651422c49a6256d1b6cca6c20b5060ad92
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 9 Jan 2025 07:33:52 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 9 Jan 2025 07:33:52 +0100
The originally submitted code (using bit masking) was correct when the
number of slots was restricted to be a power of two -- but that
limitation was removed during development that led to commit
53c2a97a9266, which made the bank selection code incorrect. This led to
always using a smaller number of banks than available. Change said code
to use integer modulo instead, which works correctly with an arbitrary
number of banks.
It's likely that we could improve on this to avoid runtime use of
integer division. But with this change we're, at least, not wasting
memory on unused banks, and more banks mean less contention, which is
likely to have a much higher performance impact than a single
instruction's latency.
Author: Yura Sokolov <y.sokolov@postgrespro.ru>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/9444dc46-ca47-43ed-9058-89c456316306@postgrespro.ru
M src/backend/access/transam/slru.c
M src/include/access/slru.h
Fix off_t overflow in pg_basebackup on Windows.
commit : 970b97eeb8f036e534abe183fd41604671f7674d
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 9 Jan 2025 13:17:36 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 9 Jan 2025 13:17:36 +1300
walmethods.c used off_t to navigate around a pg_wal.tar file that could
exceed 2GB, which doesn't work on Windows and would fail with misleading
errors. Use pgoff_t instead.
Back-patch to all supported branches.
Author: Davinder Singh <davinder.singh@enterprisedb.com>
Reported-by: Jakub Wartak <jakub.wartak@enterprisedb.com>
Discussion: https://postgr.es/m/CAKZiRmyM4YnokK6Oenw5JKwAQ3rhP0YTz2T-tiw5dAQjGRXE3Q%40mail.gmail.com
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_basebackup/walmethods.h
Provide 64-bit ftruncate() and lseek() on Windows.
commit : 026762dae39d6efcbfa99a18a15fdfeecbd5b9cc
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 9 Jan 2025 12:10:26 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 9 Jan 2025 12:10:26 +1300
Change our ftruncate() macro to use the 64-bit variant of chsize(), and
add a new macro to redirect lseek() to _lseeki64().
Back-patch to all supported releases, in preparation for a bug fix.
Tested-by: Davinder Singh <davinder.singh@enterprisedb.com>
Discussion: https://postgr.es/m/CAKZiRmyM4YnokK6Oenw5JKwAQ3rhP0YTz2T-tiw5dAQjGRXE3Q%40mail.gmail.com
M src/include/port.h
M src/include/port/win32_port.h
M src/pl/plperl/plperl_system.h
Fix duplicate typedef from commit a2f17f004d.
commit : 229e7793d96954739d3fb9b37e2ccf77c2803f07
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 8 Jan 2025 15:25:05 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 8 Jan 2025 15:25:05 -0800
Reported-by: Thomas Munro
M src/include/utils/pg_locale.h
Control collation behavior with a method table.
commit : a2f17f004d229f69a32cfa80904b95edcbc68f95
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 8 Jan 2025 14:26:33 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 8 Jan 2025 14:26:33 -0800
Previously, behavior branched based on the provider. A method table is
less error-prone and more flexible.
The ctype behavior will be addressed in an upcoming commit.
Reviewed-by: Andreas Karlsson
Discussion: https://postgr.es/m/2830211e1b6e6a2e26d845780b03e125281ea17b.camel%40j-davis.com
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_locale_icu.c
M src/backend/utils/adt/pg_locale_libc.c
M src/include/utils/pg_locale.h
Move code for collation version into provider-specific files.
commit : 4f5cef2607c1f8804d4b54250642aaf586745b0e
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 8 Jan 2025 13:54:07 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 8 Jan 2025 13:54:07 -0800
Author: Andreas Karlsson
Discussion: https://postgr.es/m/4548a168-62cd-457b-8d06-9ba7b985c477%40proxel.se
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_locale_builtin.c
M src/backend/utils/adt/pg_locale_icu.c
M src/backend/utils/adt/pg_locale_libc.c
Disallow NAMEDTUPLESTORE RTEs in stored views, rules, etc.
commit : 3c49d462dbcfaff7bb77b1fec2c73c1079d25433
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 8 Jan 2025 16:35:54 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 8 Jan 2025 16:35:54 -0500
A named tuplestore is necessarily a transient object, so it makes
no sense to reference one in a persistent object such as a view.
We didn't previously prevent that, with the result that if you
tried you would get some weird failure about how the executor
couldn't find the tuplestore.
We can mechanize a check for this case cheaply by making dependency
extraction complain if it comes across such an RTE. This is a
plausible way of dealing with it since part of the problem is that we
have no way to make a pg_depend representation of a named tuplestore.
Report and fix by Yugo Nagata. Although this is an old problem,
it's a very weird corner case and there have been no reports from
end users. So it seems sufficient to fix it in master.
Discussion: https://postgr.es/m/20240726160714.e74d0db579f2c017e1ca0b7e@sraoss.co.jp
M src/backend/catalog/dependency.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Set exit status for pgindent if pg_bsd_indent fails
commit : b20fe54c9c2194fec65db73b2778a014e7823ae0
author : Andrew Dunstan <andrew@dunslane.net>
date : Wed, 8 Jan 2025 10:56:12 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Wed, 8 Jan 2025 10:56:12 -0500
Also document the exit codes in the script.
The new exit code is 3, and is not overridden by the exit code set in
--check mode.
Author: Ashutosh Bapat
Discussion: https://postgr.es/m/CAExHW5sPRSiFeLdP-u1Fa5ba7YS2f0gvLjmKOobopKadJwQ_GQ@mail.gmail.com
M src/tools/pgindent/pgindent
plpgsql: pure parser and reentrant scanner
commit : 7b27f5fd36cb3270e8ac25aefd73b552663d1392
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 8 Jan 2025 09:20:01 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 8 Jan 2025 09:20:01 +0100
The plpgsql scanner is a wrapper around the core scanner, which
already uses the flex %option reentrant. This patch only pushes up a
few levels the place where the scanner handle is allocated. Before,
it was allocated in pl_scanner.c in a global variable, so to the
outside the scanner was not reentrant. Now, it is allocated in
pl_comp.c and is passed as an argument to yyparse(), similar to how it
is handled in other reentrant scanners.
Also use flex yyextra to handle context information, instead of global
variables. Again, this uses the existing yyextra support in the core
scanner. This complements the other changes to make the scanner
reentrant.
The bison option %pure-parser is used to make the generated parser
pure. This happens in the usual way, since plpgsql has its own bison
parser definition.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org
M src/pl/plpgsql/src/nls.mk
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/pl_scanner.c
M src/pl/plpgsql/src/plpgsql.h
Remove useless function declaration
commit : b18464fad4af6ab3e115fd0e377ac7dce706a81e
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 8 Jan 2025 08:21:05 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 8 Jan 2025 08:21:05 +0100
This function apparently never existed.
M src/pl/plpgsql/src/plpgsql.h
pg_freespacemap: Fix declaration of pg_freespace(regclass)
commit : e0c3d5122e6a4114af3092b7938c9b43d619e62c
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 8 Jan 2025 13:16:43 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 8 Jan 2025 13:16:43 +0900
This function called generate_series() without enforcing its input
argument types, making possible for an attacker to catch this call, by
defining for example a generate_series(int,bigint).
The internals of pg_freespace(regclass) are changed to force the use of
bigint for the inputs of generate_series(). A more consistent style is
applied for all its hardcoded values, while on it.
Issue introduced in 3f323eba89fb.
Reported-by: Noah Misch
Reviewed-by: Noah Misch
Discussion: https://postgr.es/m/20250106190428.ec.nmisch@google.com
M contrib/pg_freespacemap/pg_freespacemap–1.2–1.3.sql
ExecInitAgg: update aggstate->numaggs and ->numtrans earlier.
commit : 3f482940dbcbd15834a67894f4d9efdf5ceb7e16
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 7 Jan 2025 15:13:50 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 7 Jan 2025 15:13:50 -0800
Functions hash_agg_entry_size() and build_hash_tables() make use of
those values for memory size estimates.
Because this change only affects memory estimates, don't backpatch.
Discussion: https://postgr.es/m/7530bd8783b1a78d53a3c70383e38d8da0a5ffe5.camel%40j-davis.com
M src/backend/executor/nodeAgg.c
nodeSetOp.c: missing additionalsize for BuildTupleHashTable().
commit : 32ddfaffd1493b27f200afb3919d3fd2e6a200ef
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 7 Jan 2025 14:55:53 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 7 Jan 2025 14:55:53 -0800
Provide additionalsize argument, which can affect the calculations for
'nbuckets'. Also, future work for Hash Aggregation will rely on the
correct additionalsize.
Discussion: https://postgr.es/m/7530bd8783b1a78d53a3c70383e38d8da0a5ffe5.camel%40j-davis.com
M src/backend/executor/nodeSetOp.c
Remove unused TupleHashTableData->entrysize.
commit : 8a96faedc408b447acd1570d2f51300bcec34959
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 7 Jan 2025 14:49:18 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 7 Jan 2025 14:49:18 -0800
Discussion: https://postgr.es/m/7530bd8783b1a78d53a3c70383e38d8da0a5ffe5.camel%40j-davis.com
M src/backend/executor/execGrouping.c
M src/include/nodes/execnodes.h
Add missing typedefs.list entry for AggStatePerGroupData.
commit : 834c9e807cf12e355f06479a8b1f7a82aba77315
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 7 Jan 2025 14:32:37 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 7 Jan 2025 14:32:37 -0800
Discussion: https://postgr.es/m/7530bd8783b1a78d53a3c70383e38d8da0a5ffe5.camel%40j-davis.com
M src/include/executor/nodeAgg.h
M src/tools/pgindent/typedefs.list
Use PqMsg_* macros in postgres.c.
commit : 4a68d5008869afd819b03075d69bf102dd2f1bda
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 7 Jan 2025 15:34:19 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 7 Jan 2025 15:34:19 -0600
Commit f4b54e1ed9, which introduced macros for protocol characters,
missed updating a couple of places in postgres.c.
Author: Dave Cramer
Reviewed-by: Fabrízio de Royes Mello
Discussion: https://postgr.es/m/CADK3HHJUVBPoVOmFesPB-fN8_dYt%2BQELV2UB6jxOW2Z40qF-qw%40mail.gmail.com
Backpatch-through: 17
M src/backend/tcop/postgres.c
Add passwordcheck.min_password_length.
commit : f7e1b3828a930639aefb5746a214cb5ddf734361
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 7 Jan 2025 15:06:40 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 7 Jan 2025 15:06:40 -0600
This new parameter can be used to change the minimum allowed
password length (in bytes). Note that it has no effect if a user
supplies a pre-encrypted password.
Author: Emanuele Musella, Maurizio Boriani
Reviewed-by: Tomas Vondra, Bertrand Drouvot, Japin Li
Discussion: https://postgr.es/m/CA%2BugDNyYtHOtWCqVD3YkSVYDWD_1fO8Jm_ahsDGA5dXhbDPwrQ%40mail.gmail.com
M contrib/passwordcheck/expected/passwordcheck.out
M contrib/passwordcheck/expected/passwordcheck_1.out
M contrib/passwordcheck/passwordcheck.c
M contrib/passwordcheck/sql/passwordcheck.sql
M doc/src/sgml/passwordcheck.sgml
Lower default value of autovacuum_worker_slots in initdb as needed.
commit : 6d01541960e2a42c92ef67be9a1c4842cd4458d6
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 7 Jan 2025 14:38:55 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 7 Jan 2025 14:38:55 -0600
Commit c758119e5b increased the default number of semaphores
required for autovacuum workers from 3 to 16. Unfortunately, some
systems have very low default settings for SEMMNS, and this change
moved the minimum required for Postgres well beyond that limit (see
commit 38da053463 for more details).
With this commit, initdb will lower the default value for
autovacuum_worker_slots as needed, just like it already does for
parameters such as max_connections and shared_buffers. We test
for (max_connections / 6) slots, which conveniently has the
following properties:
* For the initial max_connections default of 100, the default of
autovacuum_worker_slots will be 16, which is its initial default
value specified in the documentation and in guc_tables.c.
* For the lowest possible max_connections default of 25, the
default of autovacuum_worker_slots will be 4, which means we only
need one additional semaphore for autovacuum workers (as compared
to before commit c758119e5b). This leaves some wiggle room for
new auxiliary workers, etc. on systems with low SEMMNS, and it
ensures that the default number of slots will be greater than or
equal to the default value of autovacuum_max_workers (3).
Reported-by: Tom Lane
Suggested-by: Andres Freund
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/1346002.1736198977%40sss.pgh.pa.us
M doc/src/sgml/config.sgml
M src/bin/initdb/initdb.c
Fix error message wording
commit : 0e5b14410e2bb54d191bbb7bb21a674ccabe768e
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 7 Jan 2025 20:07:32 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 7 Jan 2025 20:07:32 +0100
The originals are ambiguous and a bit out of style.
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/202412141243.efesjyyvzxsz@alvherre.pgsql
M src/backend/parser/parse_expr.c
M src/interfaces/ecpg/test/expected/sql-sqljson.stderr
M src/test/regress/expected/sqljson.out
Fix meson detection of a couple of 64 bit builtins.
commit : c4782c44101ef9e50daa9e1d69821d98f58057fd
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 8 Jan 2025 07:15:28 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 8 Jan 2025 07:15:28 +1300
A couple of checks were missed by commit 962da900, so we would fail to
detect the features.
Reported-by: Юрий Соколов <y.sokolov@postgrespro.ru>
Discussion: https://postgr.es/m/42C25E2A-6519-4549-9F47-6B0686E83836%40postgrespro.ru
M meson.build
Remove unnecessary code to handle CONSTR_NOTNULL
commit : 5b291d1c9c09d75982c3270bfa61d4e822087b6a
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 7 Jan 2025 16:49:41 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 7 Jan 2025 16:49:41 +0100
Commit 14e87ffa5c54 needlessly added support for CONSTR_NOTNULL entries
to StoreConstraints. It's dead code, so remove it.
To make the situation regarding constraint creation clearer, change
comments in heap_create_with_catalog, StoreConstraints, MergeAttributes
to explain which types of constraint are used on each.
Author: 何建 (Jian He) <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CACJufxFxzqrCiUNfjJ0tQU+=nKQkQCGtGzUBude=SMOwj5VNjQ@mail.gmail.com
M src/backend/catalog/heap.c
M src/backend/commands/tablecmds.c
Improve nbtree unsatisfiable RowCompare detection.
commit : ec986020decff322723cf7b3a2696803d082ad17
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 7 Jan 2025 10:38:30 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 7 Jan 2025 10:38:30 -0500
Move nbtree's detection of RowCompare quals that are unsatisfiable due
to having a NULL in their first row element: rather than detecting these
cases at the point where _bt_first builds its insertion scan key, do so
earlier, during preprocessing proper. This brings the RowCompare case
in line every other case involving an unsatisfiable-due-to-NULL qual.
nbtree now consistently detects such unsatisfiable quals -- even when
they happen to involve a key that isn't examined by _bt_first at all.
Affected cases thereby avoid useless full index scans that cannot
possibly return any matching rows.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAH2-WzmySVXst2hFrOATC-zw1Byg1XC-jYUS314=mzuqsNwk+Q@mail.gmail.com
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtutils.c
M src/test/regress/expected/btree_index.out
M src/test/regress/expected/create_index.out
M src/test/regress/sql/btree_index.sql
M src/test/regress/sql/create_index.sql
nbtree: Simplify _bt_first parallel scan handling.
commit : 428a99b58960a2772c3a29235eb694054f666a28
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 7 Jan 2025 10:29:46 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 7 Jan 2025 10:29:46 -0500
This new structure relieves _bt_first from having separate calls to
_bt_start_array_keys for the serial case and parallel case. This saves
code, and seems clearer.
Follow-up to work from commits 4e6e375b and b5ee4e52.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wz=XjUZjBjHJdhTvuH5MwoJObWGoM2RG2LyFg5WUdWyk=A@mail.gmail.com
M src/backend/access/nbtree/nbtsearch.c
Remove unused parameter in lookup_var_attr_stats
commit : 2f8b4007dbbac5aa9605414974c802e415176df4
author : Richard Guo <rguo@postgresql.org>
date : Tue, 7 Jan 2025 11:24:14 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Tue, 7 Jan 2025 11:24:14 +0900
The parameter 'rel' in lookup_var_attr_stats was once used to draw an
ERROR when ANALYZE failed to acquire sufficient data to build extended
statistics. bf2a691e0 changed the logic to raise a WARNING in the
caller instead. As a result, this parameter is no longer needed and
can be removed. Since this is a static function, we can always easily
reintroduce the parameter if it's ever needed in the future.
Author: Ilia Evdokimov
Reviewed-by: Fabrízio de Royes Mello
Discussion: https://postgr.es/m/b3880f22-5808-4206-88d4-1553a81c3440@tantorlabs.com
M src/backend/statistics/extended_stats.c
Allow changing autovacuum_max_workers without restarting.
commit : c758119e5bfb47b38cf957f9a5a37ceae96fa9b3
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 6 Jan 2025 15:01:22 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 6 Jan 2025 15:01:22 -0600
This commit introduces a new parameter named
autovacuum_worker_slots that controls how many autovacuum worker
slots to reserve during server startup. Modifying this new
parameter's value does require a server restart, but it should
typically be set to the upper bound of what you might realistically
need to set autovacuum_max_workers. With that new parameter in
place, autovacuum_max_workers can now be changed with a SIGHUP
(e.g., pg_ctl reload).
If autovacuum_max_workers is set higher than
autovacuum_worker_slots, a WARNING is emitted, and the server will
only start up to autovacuum_worker_slots workers at a given time.
If autovacuum_max_workers is set to a value less than the number of
currently-running autovacuum workers, the existing workers will
continue running, but no new workers will be started until the
number of running autovacuum workers drops below
autovacuum_max_workers.
Reviewed-by: Sami Imseih, Justin Pryzby, Robert Haas, Andres Freund, Yogesh Sharma
Discussion: https://postgr.es/m/20240410212344.GA1824549%40nathanxps13
M doc/src/sgml/config.sgml
M doc/src/sgml/runtime.sgml
M src/backend/access/transam/xlog.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/pmchild.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/init/postinit.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/postmaster/autovacuum.h
M src/test/perl/PostgreSQL/Test/Cluster.pm
Remove duplicate definitions in proc.h
commit : 5e68f6119268d8c76b860bb56014335697c687db
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 6 Jan 2025 11:56:03 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 6 Jan 2025 11:56:03 +0200
These are also present in procnumber.h
Reported-by: Peter Eisentraut
Discussion: https://www.postgresql.org/message-id/bd04d675-4672-4f87-800a-eb5d470c15fc@eisentraut.org
M src/include/storage/proc.h
flex code modernization: Replace YY_EXTRA_TYPE define with flex option
commit : b1ef48980ddd082a90ed39aa5914af45e53059e6
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 6 Jan 2025 09:47:58 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 6 Jan 2025 09:47:58 +0100
Replace #define YY_EXTRA_TYPE with %option extra-type. The latter is
the way recommended by the flex manual (available since flex 2.5.34).
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org
M src/backend/parser/scan.l
M src/backend/replication/repl_scanner.l
M src/backend/replication/syncrep_scanner.l
M src/backend/utils/adt/jsonpath_scan.l
M src/bin/psql/psqlscanslash.l
M src/fe_utils/psqlscan.l
doc: Clarify log level for VERBOSE messages in maintenance commands.
commit : 632384d0eb142f243fdd6059fde715319cfd05c9
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 6 Jan 2025 17:24:10 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 6 Jan 2025 17:24:10 +0900
VERBOSE messages from ANALYZE, CLUSTER, REINDEX, and VACUUM are logged
at the INFO level, but this detail was missing from the documentation.
This commit updates the docs to mention the log level for these messages.
Author: Masahiro Ikeda
Reviewed-by: Yugo Nagata
Discussion: https://postgr.es/m/b4a4b7916982dccd9607c8efb3ce5116@oss.nttdata.com
M doc/src/sgml/ref/analyze.sgml
M doc/src/sgml/ref/cluster.sgml
M doc/src/sgml/ref/reindex.sgml
M doc/src/sgml/ref/vacuum.sgml
Always use the caller-provided context for radix tree leaves
commit : 3e70da2781e86d07cd88f568ebeb38ddf7da8e8e
author : John Naylor <john.naylor@postgresql.org>
date : Fri, 20 Dec 2024 14:48:24 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Fri, 20 Dec 2024 14:48:24 +0700
Previously, it would not have worked for a caller to pass a slab
context, since it would have been used for other things which likely
had incompatible size. In an attempt to be helpful and avoid possible
space wastage due to aset's power-of-two rounding, RT_CREATE would
create an additional slab context if the value type was fixed-length
and larger than pointer size. The problem was, we have since added
the bump context type, and the generation context was a possibility as
well, so silently overriding the caller's choice may actually be worse.
Commit e8a6f1f908d arranged so that the caller-provided context is
used only for leaves, so it's safe for the caller to use slab here
if they wish. As demonstration, use slab in one of the radix tree
regression tests.
Reviewed by Masahiko Sawada
Discussion: https://postgr.es/m/CANWCAZZDCo4k5oURg_pPxM6+WZ1oiG=sqgjmQiELuyP0Vtrwig@mail.gmail.com
M src/include/lib/radixtree.h
M src/test/modules/test_radixtree/test_radixtree.c
Get rid of radix tree's general purpose memory context
commit : e8a6f1f908d03b836c8b5379b449346ad32c1ba1
author : John Naylor <john.naylor@postgresql.org>
date : Fri, 20 Dec 2024 13:04:18 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Fri, 20 Dec 2024 13:04:18 +0700
Previously, this was notionally used only for the entry point of the
tree and as a convenient parent for other contexts.
For shared memory, the creator previously allocated the entry point
in this context, but attaching backends didn't have access to that,
so they just used the caller's context. For the sake of consistency,
allocate every instance of an entry point in the caller's context.
For local memory, allocate the control object in the caller's context
as well. This commit also makes the "leaf context" the notional parent
of the child contexts used for nodes, so it's a bit of a misnomer,
but a future commit will make the node contexts independent rather
than children, so leave it this way for now to avoid code churn.
The memory context parameter for RT_CREATE is now unused in the case
of shared memory, so remove it and adjust callers to match.
In passing, remove unused "context" member from struct TidStore,
which seems to have been an oversight.
Reviewed by Masahiko Sawada
Discussion: https://postgr.es/m/CANWCAZZDCo4k5oURg_pPxM6+WZ1oiG=sqgjmQiELuyP0Vtrwig@mail.gmail.com
M src/backend/access/common/tidstore.c
M src/include/lib/radixtree.h
M src/test/modules/test_radixtree/test_radixtree.c
Use caller's memory context for radix tree iteration state
commit : 960013f2a1f9e51aad85c91aceeb442508be6032
author : John Naylor <john.naylor@postgresql.org>
date : Sat, 21 Dec 2024 10:55:31 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Sat, 21 Dec 2024 10:55:31 +0700
Typically only one iterator is present at any time, so it's overkill
to devote an entire context for this. Get rid of it and use the
caller's context.
This is tidy-up work, so no backpatch in this form. However, a
hypothetical extension to v17 that tried to start iteration from
an attaching backend would result in a crash, so that'll be fixed
separately in a way that doesn't change behavior in core.
Patch by me, reported and reviewed by Masahiko Sawada
Discussion: https://postgr.es/m/CAD21AoBB2U47V=F+wQRB1bERov_of5=BOZGaybjaV8FLQyqG3Q@mail.gmail.com
M src/include/lib/radixtree.h
Remove useless configure check
commit : 9a8313dabe5ebb6a99d9a165d3c9bc3fa0213088
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 5 Jan 2025 11:30:48 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 5 Jan 2025 11:30:48 +0100
The test for "decltype" as a variant of "typeof" apparently never
worked (see also commit 3582b223d49), so remove it.
Discussion: https://www.postgresql.org/message-id/flat/795b1c54-c64a-47f9-8fa3-880dcab59975%40eisentraut.org
M config/c-compiler.m4
M configure
M meson.build
meson: Fix missing name arguments of cc.compiles() calls
commit : 6549a02a51e88960cccb938999a43203e335d0cc
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 5 Jan 2025 11:30:48 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 5 Jan 2025 11:30:48 +0100
Without it, the check won't show up in the meson setup/configure
output.
Discussion: https://www.postgresql.org/message-id/flat/795b1c54-c64a-47f9-8fa3-880dcab59975%40eisentraut.org
M meson.build
Document strange jsonb sort order for empty top level arrays
commit : 30f017626308a06cf0c0c82a706a1ba1b07df34a
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 3 Jan 2025 09:23:46 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 3 Jan 2025 09:23:46 -0500
Slightly faulty logic in the original jsonb code (commit d9134d0a355)
results in an empty top level array sorting less than a json null. We
can't change the sort order now since it would affect btree indexes over
jsonb, so document the anomaly.
Backpatch to all live branches (13 .. 17)
In master, also add a code comment noting the anomaly.
Reported-by: Yan Chengpen
Reviewed-by: Jian He
Discussion: https://postgr.es/m/OSBPR01MB45199DD8DA2D1CECD50518188E272@OSBPR01MB4519.jpnprd01.prod.outlook.com
M doc/src/sgml/json.sgml
M src/backend/utils/adt/jsonb_util.c
Ignore nullingrels when looking up statistics
commit : e28033fe1af8037e0fec8bb3a32fabbe18ac06b1
author : Richard Guo <rguo@postgresql.org>
date : Thu, 2 Jan 2025 17:56:59 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Thu, 2 Jan 2025 17:56:59 +0900
When looking up statistical data about an expression, we do not need
to concern ourselves with the outer joins that could null the
Vars/PHVs contained in the expression. Accounting for nullingrels in
the expression could cause estimate_num_groups to count the same Var
multiple times if it's marked with different nullingrels. This is
incorrect, and could lead to "ERROR: corrupt MVNDistinct entry" when
searching for multivariate n-distinct.
Furthermore, the nullingrels could prevent us from matching an
expression to expressional index columns or to the expressions in
extended statistics, leading to inaccurate estimates.
To fix, strip out all the nullingrels from the expression before we
look up statistical data about it. There is one ensuing plan change
in the regression tests, but it looks reasonable and does not
compromise its original purpose.
This patch could result in plan changes, but it fixes an actual bug,
so back-patch to v16 where the outer-join-aware-Var infrastructure was
introduced.
Author: Richard Guo
Discussion: https://postgr.es/m/CAMbWs4-2Z4k+nFTiZe0Qbu5n8juUWenDAtMzi98bAZQtwHx0-w@mail.gmail.com
M src/backend/utils/adt/selfuncs.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Fix outdated CHUNKHDRSZ value in nodeAgg.c
commit : d93bb8163c9cdbaa922ce6b1f3d044fc589d0fa6
author : David Rowley <drowley@postgresql.org>
date : Thu, 2 Jan 2025 22:04:09 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 2 Jan 2025 22:04:09 +1300
CHUNKHDRSZ was defined as 16 bytes, which was true when that code went in,
but since c6e0fe1f2, 8 is a more accurate value. Here we adjust it to use
sizeof(MemoryChunk), which is normally 8, or 16 for cassert builds.
c6e0fe1f2 first appeared in v16, so this is technically wrong in v16 up
to master, but let's apply this only to master as adjusting this does
influence the estimated number of batches in the aggregate costing code
and we don't want to cause plan instability in released versions.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/CAApHDvpMpRQvsTqZo3FinXkgytwxwF8sCyZm83xDj-1s_hLe+w@mail.gmail.com
M src/backend/executor/nodeAgg.c
Fix an assortment of spelling mistakes and typos
commit : 11012c503759f8018d8831bc6eb1f998eba7ad25
author : David Rowley <drowley@postgresql.org>
date : Thu, 2 Jan 2025 12:42:01 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 2 Jan 2025 12:42:01 +1300
Author: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/5812a0b9-b0cf-4151-9a14-d9f00e4f2858@gmail.com
M src/backend/access/heap/heapam.c
M src/backend/access/heap/pruneheap.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/access/transam/xlog.c
M src/backend/catalog/pg_constraint.c
M src/backend/catalog/pg_depend.c
M src/backend/catalog/storage.c
M src/backend/commands/explain.c
M src/backend/optimizer/path/indxpath.c
M src/backend/postmaster/postmaster.c
M src/backend/replication/logical/decode.c
M src/backend/statistics/attribute_stats.c
M src/backend/statistics/stat_utils.c
M src/backend/storage/lmgr/lock.c
M src/backend/utils/adt/cash.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_rewind/parsexlog.c
M src/include/catalog/reformat_dat_file.pl
M src/include/commands/copyfrom_internal.h
M src/include/jit/SectionMemoryManager.h
M src/test/postmaster/t/002_start_stop.pl
Update copyright for 2025
commit : 50e6eb731d98ab6d0e625a0b87fb327b172bbebd
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 1 Jan 2025 11:21:55 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 1 Jan 2025 11:21:55 -0500
Backpatch-through: 13
M COPYRIGHT
M config/check_modules.pl
M configure
M configure.ac
M contrib/amcheck/meson.build
M contrib/amcheck/t/001_verify_heapam.pl
M contrib/amcheck/t/002_cic.pl
M contrib/amcheck/t/003_cic_2pc.pl
M contrib/amcheck/t/004_verify_nbtree_unique.pl
M contrib/amcheck/t/005_pitr.pl
M contrib/amcheck/verify_heapam.c
M contrib/amcheck/verify_nbtree.c
M contrib/auth_delay/auth_delay.c
M contrib/auth_delay/meson.build
M contrib/auto_explain/auto_explain.c
M contrib/auto_explain/meson.build
M contrib/auto_explain/t/001_auto_explain.pl
M contrib/basebackup_to_shell/basebackup_to_shell.c
M contrib/basebackup_to_shell/meson.build
M contrib/basebackup_to_shell/t/001_basic.pl
M contrib/basic_archive/basic_archive.c
M contrib/basic_archive/meson.build
M contrib/bloom/blcost.c
M contrib/bloom/blinsert.c
M contrib/bloom/bloom.h
M contrib/bloom/blscan.c
M contrib/bloom/blutils.c
M contrib/bloom/blvacuum.c
M contrib/bloom/blvalidate.c
M contrib/bloom/meson.build
M contrib/bloom/t/001_wal.pl
M contrib/bool_plperl/meson.build
M contrib/btree_gin/meson.build
M contrib/btree_gist/meson.build
M contrib/citext/meson.build
M contrib/cube/meson.build
M contrib/dblink/dblink.c
M contrib/dblink/meson.build
M contrib/dict_int/dict_int.c
M contrib/dict_int/meson.build
M contrib/dict_xsyn/dict_xsyn.c
M contrib/dict_xsyn/meson.build
M contrib/earthdistance/meson.build
M contrib/file_fdw/file_fdw.c
M contrib/file_fdw/meson.build
M contrib/fuzzystrmatch/daitch_mokotoff.c
M contrib/fuzzystrmatch/daitch_mokotoff_header.pl
M contrib/fuzzystrmatch/fuzzystrmatch.c
M contrib/fuzzystrmatch/meson.build
M contrib/hstore/hstore_subs.c
M contrib/hstore/meson.build
M contrib/hstore_plperl/meson.build
M contrib/hstore_plpython/meson.build
M contrib/intagg/meson.build
M contrib/intarray/_int_selfuncs.c
M contrib/intarray/bench/bench.pl
M contrib/intarray/bench/create_test.pl
M contrib/intarray/meson.build
M contrib/isn/isn.c
M contrib/isn/isn.h
M contrib/isn/meson.build
M contrib/jsonb_plperl/meson.build
M contrib/jsonb_plpython/meson.build
M contrib/lo/meson.build
M contrib/ltree/meson.build
M contrib/ltree_plpython/meson.build
M contrib/meson.build
M contrib/oid2name/meson.build
M contrib/oid2name/t/001_basic.pl
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/fsmfuncs.c
M contrib/pageinspect/ginfuncs.c
M contrib/pageinspect/gistfuncs.c
M contrib/pageinspect/hashfuncs.c
M contrib/pageinspect/heapfuncs.c
M contrib/pageinspect/meson.build
M contrib/pageinspect/pageinspect.h
M contrib/pageinspect/rawpage.c
M contrib/passwordcheck/meson.build
M contrib/passwordcheck/passwordcheck.c
M contrib/pg_buffercache/meson.build
M contrib/pg_freespacemap/meson.build
M contrib/pg_logicalinspect/meson.build
M contrib/pg_logicalinspect/pg_logicalinspect.c
M contrib/pg_prewarm/autoprewarm.c
M contrib/pg_prewarm/meson.build
M contrib/pg_prewarm/pg_prewarm.c
M contrib/pg_prewarm/t/001_basic.pl
M contrib/pg_stat_statements/meson.build
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/t/010_restart.pl
M contrib/pg_surgery/heap_surgery.c
M contrib/pg_surgery/meson.build
M contrib/pg_trgm/meson.build
M contrib/pg_trgm/trgm_regexp.c
M contrib/pg_visibility/meson.build
M contrib/pg_visibility/pg_visibility.c
M contrib/pg_visibility/t/001_concurrent_transaction.pl
M contrib/pg_visibility/t/002_corrupt_vm.pl
M contrib/pg_walinspect/meson.build
M contrib/pg_walinspect/pg_walinspect.c
M contrib/pgcrypto/meson.build
M contrib/pgrowlocks/meson.build
M contrib/pgstattuple/meson.build
M contrib/pgstattuple/pgstatapprox.c
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/meson.build
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/shippable.c
M contrib/seg/meson.build
M contrib/seg/seg-validate.pl
M contrib/seg/sort-segments.pl
M contrib/sepgsql/database.c
M contrib/sepgsql/dml.c
M contrib/sepgsql/hooks.c
M contrib/sepgsql/label.c
M contrib/sepgsql/launcher
M contrib/sepgsql/meson.build
M contrib/sepgsql/proc.c
M contrib/sepgsql/relation.c
M contrib/sepgsql/schema.c
M contrib/sepgsql/selinux.c
M contrib/sepgsql/sepgsql.h
M contrib/sepgsql/uavc.c
M contrib/spi/meson.build
M contrib/sslinfo/meson.build
M contrib/tablefunc/meson.build
M contrib/tablefunc/tablefunc.c
M contrib/tcn/meson.build
M contrib/tcn/tcn.c
M contrib/test_decoding/meson.build
M contrib/test_decoding/t/001_repl_stats.pl
M contrib/test_decoding/test_decoding.c
M contrib/tsm_system_rows/meson.build
M contrib/tsm_system_rows/tsm_system_rows.c
M contrib/tsm_system_time/meson.build
M contrib/tsm_system_time/tsm_system_time.c
M contrib/unaccent/meson.build
M contrib/unaccent/unaccent.c
M contrib/uuid-ossp/meson.build
M contrib/uuid-ossp/uuid-ossp.c
M contrib/vacuumlo/meson.build
M contrib/vacuumlo/t/001_basic.pl
M contrib/vacuumlo/vacuumlo.c
M contrib/xml2/meson.build
M doc/src/sgml/generate-errcodes-table.pl
M doc/src/sgml/generate-keywords-table.pl
M doc/src/sgml/generate-targets-meson.pl
M doc/src/sgml/legal.sgml
M doc/src/sgml/lobj.sgml
M doc/src/sgml/meson.build
M doc/src/sgml/targets-meson.txt
M meson.build
M meson_options.txt
M src/backend/Makefile
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_bloom.c
M src/backend/access/brin/brin_inclusion.c
M src/backend/access/brin/brin_minmax.c
M src/backend/access/brin/brin_minmax_multi.c
M src/backend/access/brin/brin_pageops.c
M src/backend/access/brin/brin_revmap.c
M src/backend/access/brin/brin_tuple.c
M src/backend/access/brin/brin_validate.c
M src/backend/access/brin/brin_xlog.c
M src/backend/access/brin/meson.build
M src/backend/access/common/attmap.c
M src/backend/access/common/bufmask.c
M src/backend/access/common/detoast.c
M src/backend/access/common/heaptuple.c
M src/backend/access/common/indextuple.c
M src/backend/access/common/meson.build
M src/backend/access/common/printsimple.c
M src/backend/access/common/printtup.c
M src/backend/access/common/relation.c
M src/backend/access/common/reloptions.c
M src/backend/access/common/scankey.c
M src/backend/access/common/session.c
M src/backend/access/common/syncscan.c
M src/backend/access/common/tidstore.c
M src/backend/access/common/toast_compression.c
M src/backend/access/common/toast_internals.c
M src/backend/access/common/tupconvert.c
M src/backend/access/common/tupdesc.c
M src/backend/access/gin/ginarrayproc.c
M src/backend/access/gin/ginbtree.c
M src/backend/access/gin/ginbulk.c
M src/backend/access/gin/gindatapage.c
M src/backend/access/gin/ginentrypage.c
M src/backend/access/gin/ginfast.c
M src/backend/access/gin/ginget.c
M src/backend/access/gin/gininsert.c
M src/backend/access/gin/ginlogic.c
M src/backend/access/gin/ginpostinglist.c
M src/backend/access/gin/ginscan.c
M src/backend/access/gin/ginutil.c
M src/backend/access/gin/ginvacuum.c
M src/backend/access/gin/ginvalidate.c
M src/backend/access/gin/ginxlog.c
M src/backend/access/gin/meson.build
M src/backend/access/gist/gist.c
M src/backend/access/gist/gistbuild.c
M src/backend/access/gist/gistbuildbuffers.c
M src/backend/access/gist/gistget.c
M src/backend/access/gist/gistproc.c
M src/backend/access/gist/gistscan.c
M src/backend/access/gist/gistsplit.c
M src/backend/access/gist/gistutil.c
M src/backend/access/gist/gistvacuum.c
M src/backend/access/gist/gistvalidate.c
M src/backend/access/gist/gistxlog.c
M src/backend/access/gist/meson.build
M src/backend/access/hash/hash.c
M src/backend/access/hash/hash_xlog.c
M src/backend/access/hash/hashfunc.c
M src/backend/access/hash/hashinsert.c
M src/backend/access/hash/hashovfl.c
M src/backend/access/hash/hashpage.c
M src/backend/access/hash/hashsearch.c
M src/backend/access/hash/hashsort.c
M src/backend/access/hash/hashutil.c
M src/backend/access/hash/hashvalidate.c
M src/backend/access/hash/meson.build
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/heap/heapam_visibility.c
M src/backend/access/heap/heapam_xlog.c
M src/backend/access/heap/heaptoast.c
M src/backend/access/heap/hio.c
M src/backend/access/heap/meson.build
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/rewriteheap.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/heap/visibilitymap.c
M src/backend/access/index/amapi.c
M src/backend/access/index/amvalidate.c
M src/backend/access/index/genam.c
M src/backend/access/index/indexam.c
M src/backend/access/index/meson.build
M src/backend/access/meson.build
M src/backend/access/nbtree/meson.build
M src/backend/access/nbtree/nbtcompare.c
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/access/nbtree/nbtsplitloc.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/access/nbtree/nbtvalidate.c
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/rmgrdesc/brindesc.c
M src/backend/access/rmgrdesc/clogdesc.c
M src/backend/access/rmgrdesc/committsdesc.c
M src/backend/access/rmgrdesc/dbasedesc.c
M src/backend/access/rmgrdesc/genericdesc.c
M src/backend/access/rmgrdesc/gindesc.c
M src/backend/access/rmgrdesc/gistdesc.c
M src/backend/access/rmgrdesc/hashdesc.c
M src/backend/access/rmgrdesc/heapdesc.c
M src/backend/access/rmgrdesc/logicalmsgdesc.c
M src/backend/access/rmgrdesc/meson.build
M src/backend/access/rmgrdesc/mxactdesc.c
M src/backend/access/rmgrdesc/nbtdesc.c
M src/backend/access/rmgrdesc/relmapdesc.c
M src/backend/access/rmgrdesc/replorigindesc.c
M src/backend/access/rmgrdesc/rmgrdesc_utils.c
M src/backend/access/rmgrdesc/seqdesc.c
M src/backend/access/rmgrdesc/smgrdesc.c
M src/backend/access/rmgrdesc/spgdesc.c
M src/backend/access/rmgrdesc/standbydesc.c
M src/backend/access/rmgrdesc/tblspcdesc.c
M src/backend/access/rmgrdesc/xactdesc.c
M src/backend/access/rmgrdesc/xlogdesc.c
M src/backend/access/sequence/meson.build
M src/backend/access/sequence/sequence.c
M src/backend/access/spgist/meson.build
M src/backend/access/spgist/spgdoinsert.c
M src/backend/access/spgist/spginsert.c
M src/backend/access/spgist/spgkdtreeproc.c
M src/backend/access/spgist/spgproc.c
M src/backend/access/spgist/spgquadtreeproc.c
M src/backend/access/spgist/spgscan.c
M src/backend/access/spgist/spgtextproc.c
M src/backend/access/spgist/spgutils.c
M src/backend/access/spgist/spgvacuum.c
M src/backend/access/spgist/spgvalidate.c
M src/backend/access/spgist/spgxlog.c
M src/backend/access/table/meson.build
M src/backend/access/table/table.c
M src/backend/access/table/tableam.c
M src/backend/access/table/tableamapi.c
M src/backend/access/table/toast_helper.c
M src/backend/access/tablesample/bernoulli.c
M src/backend/access/tablesample/meson.build
M src/backend/access/tablesample/system.c
M src/backend/access/tablesample/tablesample.c
M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/generic_xlog.c
M src/backend/access/transam/meson.build
M src/backend/access/transam/multixact.c
M src/backend/access/transam/parallel.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/access/transam/timeline.c
M src/backend/access/transam/transam.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/twophase_rmgr.c
M src/backend/access/transam/varsup.c
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogarchive.c
M src/backend/access/transam/xlogbackup.c
M src/backend/access/transam/xlogfuncs.c
M src/backend/access/transam/xloginsert.c
M src/backend/access/transam/xlogprefetcher.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogrecovery.c
M src/backend/access/transam/xlogstats.c
M src/backend/access/transam/xlogutils.c
M src/backend/archive/meson.build
M src/backend/archive/shell_archive.c
M src/backend/backup/backup_manifest.c
M src/backend/backup/basebackup.c
M src/backend/backup/basebackup_copy.c
M src/backend/backup/basebackup_gzip.c
M src/backend/backup/basebackup_incremental.c
M src/backend/backup/basebackup_lz4.c
M src/backend/backup/basebackup_progress.c
M src/backend/backup/basebackup_sink.c
M src/backend/backup/basebackup_target.c
M src/backend/backup/basebackup_throttle.c
M src/backend/backup/basebackup_zstd.c
M src/backend/backup/meson.build
M src/backend/backup/walsummary.c
M src/backend/backup/walsummaryfuncs.c
M src/backend/bootstrap/bootparse.y
M src/backend/bootstrap/bootscanner.l
M src/backend/bootstrap/bootstrap.c
M src/backend/bootstrap/meson.build
M src/backend/catalog/Catalog.pm
M src/backend/catalog/Makefile
M src/backend/catalog/aclchk.c
M src/backend/catalog/catalog.c
M src/backend/catalog/dependency.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/indexing.c
M src/backend/catalog/information_schema.sql
M src/backend/catalog/meson.build
M src/backend/catalog/namespace.c
M src/backend/catalog/objectaccess.c
M src/backend/catalog/objectaddress.c
M src/backend/catalog/partition.c
M src/backend/catalog/pg_aggregate.c
M src/backend/catalog/pg_attrdef.c
M src/backend/catalog/pg_cast.c
M src/backend/catalog/pg_class.c
M src/backend/catalog/pg_collation.c
M src/backend/catalog/pg_constraint.c
M src/backend/catalog/pg_conversion.c
M src/backend/catalog/pg_db_role_setting.c
M src/backend/catalog/pg_depend.c
M src/backend/catalog/pg_enum.c
M src/backend/catalog/pg_inherits.c
M src/backend/catalog/pg_largeobject.c
M src/backend/catalog/pg_namespace.c
M src/backend/catalog/pg_operator.c
M src/backend/catalog/pg_parameter_acl.c
M src/backend/catalog/pg_proc.c
M src/backend/catalog/pg_publication.c
M src/backend/catalog/pg_range.c
M src/backend/catalog/pg_shdepend.c
M src/backend/catalog/pg_subscription.c
M src/backend/catalog/pg_type.c
M src/backend/catalog/storage.c
M src/backend/catalog/system_functions.sql
M src/backend/catalog/system_views.sql
M src/backend/catalog/toasting.c
M src/backend/commands/aggregatecmds.c
M src/backend/commands/alter.c
M src/backend/commands/amcmds.c
M src/backend/commands/analyze.c
M src/backend/commands/async.c
M src/backend/commands/cluster.c
M src/backend/commands/collationcmds.c
M src/backend/commands/comment.c
M src/backend/commands/constraint.c
M src/backend/commands/conversioncmds.c
M src/backend/commands/copy.c
M src/backend/commands/copyfrom.c
M src/backend/commands/copyfromparse.c
M src/backend/commands/copyto.c
M src/backend/commands/createas.c
M src/backend/commands/dbcommands.c
M src/backend/commands/define.c
M src/backend/commands/discard.c
M src/backend/commands/dropcmds.c
M src/backend/commands/event_trigger.c
M src/backend/commands/explain.c
M src/backend/commands/extension.c
M src/backend/commands/foreigncmds.c
M src/backend/commands/functioncmds.c
M src/backend/commands/indexcmds.c
M src/backend/commands/lockcmds.c
M src/backend/commands/matview.c
M src/backend/commands/meson.build
M src/backend/commands/opclasscmds.c
M src/backend/commands/operatorcmds.c
M src/backend/commands/policy.c
M src/backend/commands/portalcmds.c
M src/backend/commands/prepare.c
M src/backend/commands/proclang.c
M src/backend/commands/publicationcmds.c
M src/backend/commands/schemacmds.c
M src/backend/commands/seclabel.c
M src/backend/commands/sequence.c
M src/backend/commands/statscmds.c
M src/backend/commands/subscriptioncmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/tablespace.c
M src/backend/commands/trigger.c
M src/backend/commands/tsearchcmds.c
M src/backend/commands/typecmds.c
M src/backend/commands/user.c
M src/backend/commands/vacuum.c
M src/backend/commands/vacuumparallel.c
M src/backend/commands/variable.c
M src/backend/commands/view.c
M src/backend/executor/execAmi.c
M src/backend/executor/execAsync.c
M src/backend/executor/execCurrent.c
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/execGrouping.c
M src/backend/executor/execIndexing.c
M src/backend/executor/execJunk.c
M src/backend/executor/execMain.c
M src/backend/executor/execParallel.c
M src/backend/executor/execPartition.c
M src/backend/executor/execProcnode.c
M src/backend/executor/execReplication.c
M src/backend/executor/execSRF.c
M src/backend/executor/execScan.c
M src/backend/executor/execTuples.c
M src/backend/executor/execUtils.c
M src/backend/executor/functions.c
M src/backend/executor/instrument.c
M src/backend/executor/meson.build
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeBitmapAnd.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/backend/executor/nodeBitmapIndexscan.c
M src/backend/executor/nodeBitmapOr.c
M src/backend/executor/nodeCtescan.c
M src/backend/executor/nodeCustom.c
M src/backend/executor/nodeForeignscan.c
M src/backend/executor/nodeFunctionscan.c
M src/backend/executor/nodeGather.c
M src/backend/executor/nodeGatherMerge.c
M src/backend/executor/nodeGroup.c
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeHashjoin.c
M src/backend/executor/nodeIncrementalSort.c
M src/backend/executor/nodeIndexonlyscan.c
M src/backend/executor/nodeIndexscan.c
M src/backend/executor/nodeLimit.c
M src/backend/executor/nodeLockRows.c
M src/backend/executor/nodeMaterial.c
M src/backend/executor/nodeMemoize.c
M src/backend/executor/nodeMergeAppend.c
M src/backend/executor/nodeMergejoin.c
M src/backend/executor/nodeModifyTable.c
M src/backend/executor/nodeNamedtuplestorescan.c
M src/backend/executor/nodeNestloop.c
M src/backend/executor/nodeProjectSet.c
M src/backend/executor/nodeRecursiveunion.c
M src/backend/executor/nodeResult.c
M src/backend/executor/nodeSamplescan.c
M src/backend/executor/nodeSeqscan.c
M src/backend/executor/nodeSetOp.c
M src/backend/executor/nodeSort.c
M src/backend/executor/nodeSubplan.c
M src/backend/executor/nodeSubqueryscan.c
M src/backend/executor/nodeTableFuncscan.c
M src/backend/executor/nodeTidrangescan.c
M src/backend/executor/nodeTidscan.c
M src/backend/executor/nodeUnique.c
M src/backend/executor/nodeValuesscan.c
M src/backend/executor/nodeWindowAgg.c
M src/backend/executor/nodeWorktablescan.c
M src/backend/executor/spi.c
M src/backend/executor/tqueue.c
M src/backend/executor/tstoreReceiver.c
M src/backend/foreign/foreign.c
M src/backend/foreign/meson.build
M src/backend/jit/jit.c
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_deform.c
M src/backend/jit/llvm/llvmjit_error.cpp
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/jit/llvm/llvmjit_inline.cpp
M src/backend/jit/llvm/llvmjit_types.c
M src/backend/jit/llvm/llvmjit_wrap.cpp
M src/backend/jit/llvm/meson.build
M src/backend/jit/meson.build
M src/backend/lib/bipartite_match.c
M src/backend/lib/bloomfilter.c
M src/backend/lib/dshash.c
M src/backend/lib/hyperloglog.c
M src/backend/lib/ilist.c
M src/backend/lib/integerset.c
M src/backend/lib/knapsack.c
M src/backend/lib/meson.build
M src/backend/lib/pairingheap.c
M src/backend/lib/rbtree.c
M src/backend/libpq/auth-sasl.c
M src/backend/libpq/auth-scram.c
M src/backend/libpq/auth.c
M src/backend/libpq/be-fsstubs.c
M src/backend/libpq/be-gssapi-common.c
M src/backend/libpq/be-secure-common.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/libpq/be-secure-openssl.c
M src/backend/libpq/be-secure.c
M src/backend/libpq/crypt.c
M src/backend/libpq/hba.c
M src/backend/libpq/ifaddr.c
M src/backend/libpq/meson.build
M src/backend/libpq/pqcomm.c
M src/backend/libpq/pqformat.c
M src/backend/libpq/pqmq.c
M src/backend/libpq/pqsignal.c
M src/backend/main/main.c
M src/backend/main/meson.build
M src/backend/meson.build
M src/backend/nodes/bitmapset.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/extensible.c
M src/backend/nodes/gen_node_support.pl
M src/backend/nodes/list.c
M src/backend/nodes/makefuncs.c
M src/backend/nodes/meson.build
M src/backend/nodes/multibitmapset.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/params.c
M src/backend/nodes/print.c
M src/backend/nodes/queryjumblefuncs.c
M src/backend/nodes/read.c
M src/backend/nodes/readfuncs.c
M src/backend/nodes/tidbitmap.c
M src/backend/nodes/value.c
M src/backend/optimizer/geqo/geqo_copy.c
M src/backend/optimizer/geqo/geqo_eval.c
M src/backend/optimizer/geqo/geqo_main.c
M src/backend/optimizer/geqo/geqo_misc.c
M src/backend/optimizer/geqo/geqo_pool.c
M src/backend/optimizer/geqo/geqo_random.c
M src/backend/optimizer/geqo/geqo_selection.c
M src/backend/optimizer/geqo/meson.build
M src/backend/optimizer/meson.build
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/clausesel.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/path/joinrels.c
M src/backend/optimizer/path/meson.build
M src/backend/optimizer/path/pathkeys.c
M src/backend/optimizer/path/tidpath.c
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/meson.build
M src/backend/optimizer/plan/planagg.c
M src/backend/optimizer/plan/planmain.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/meson.build
M src/backend/optimizer/prep/prepagg.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/optimizer/prep/prepqual.c
M src/backend/optimizer/prep/preptlist.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/appendinfo.c
M src/backend/optimizer/util/clauses.c
M src/backend/optimizer/util/inherit.c
M src/backend/optimizer/util/joininfo.c
M src/backend/optimizer/util/meson.build
M src/backend/optimizer/util/orclauses.c
M src/backend/optimizer/util/paramassign.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/placeholder.c
M src/backend/optimizer/util/plancat.c
M src/backend/optimizer/util/predtest.c
M src/backend/optimizer/util/relnode.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/optimizer/util/tlist.c
M src/backend/optimizer/util/var.c
M src/backend/parser/analyze.c
M src/backend/parser/check_keywords.pl
M src/backend/parser/gram.y
M src/backend/parser/gramparse.h
M src/backend/parser/meson.build
M src/backend/parser/parse_agg.c
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_coerce.c
M src/backend/parser/parse_collate.c
M src/backend/parser/parse_cte.c
M src/backend/parser/parse_enr.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_func.c
M src/backend/parser/parse_jsontable.c
M src/backend/parser/parse_merge.c
M src/backend/parser/parse_node.c
M src/backend/parser/parse_oper.c
M src/backend/parser/parse_param.c
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_target.c
M src/backend/parser/parse_type.c
M src/backend/parser/parse_utilcmd.c
M src/backend/parser/parser.c
M src/backend/parser/scan.l
M src/backend/parser/scansup.c
M src/backend/partitioning/meson.build
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partdesc.c
M src/backend/partitioning/partprune.c
M src/backend/po/meson.build
M src/backend/port/atomics.c
M src/backend/port/meson.build
M src/backend/port/posix_sema.c
M src/backend/port/sysv_sema.c
M src/backend/port/sysv_shmem.c
M src/backend/port/tas/sunstudio_sparc.s
M src/backend/port/tas/sunstudio_x86.s
M src/backend/port/win32/crashdump.c
M src/backend/port/win32/meson.build
M src/backend/port/win32/signal.c
M src/backend/port/win32/socket.c
M src/backend/port/win32/timer.c
M src/backend/port/win32_sema.c
M src/backend/port/win32_shmem.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/auxprocess.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/bgwriter.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/fork_process.c
M src/backend/postmaster/interrupt.c
M src/backend/postmaster/launch_backend.c
M src/backend/postmaster/meson.build
M src/backend/postmaster/pgarch.c
M src/backend/postmaster/pmchild.c
M src/backend/postmaster/postmaster.c
M src/backend/postmaster/startup.c
M src/backend/postmaster/syslogger.c
M src/backend/postmaster/walsummarizer.c
M src/backend/postmaster/walwriter.c
M src/backend/regex/meson.build
M src/backend/regex/regc_pg_locale.c
M src/backend/regex/regexport.c
M src/backend/regex/regprefix.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/libpqwalreceiver/meson.build
M src/backend/replication/logical/applyparallelworker.c
M src/backend/replication/logical/conflict.c
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/launcher.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/logicalfuncs.c
M src/backend/replication/logical/meson.build
M src/backend/replication/logical/message.c
M src/backend/replication/logical/origin.c
M src/backend/replication/logical/proto.c
M src/backend/replication/logical/relation.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/slotsync.c
M src/backend/replication/logical/snapbuild.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/backend/replication/meson.build
M src/backend/replication/pgoutput/meson.build
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/replication/repl_gram.y
M src/backend/replication/repl_scanner.l
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/syncrep.c
M src/backend/replication/syncrep_gram.y
M src/backend/replication/syncrep_scanner.l
M src/backend/replication/walreceiver.c
M src/backend/replication/walreceiverfuncs.c
M src/backend/replication/walsender.c
M src/backend/rewrite/meson.build
M src/backend/rewrite/rewriteDefine.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/rewrite/rewriteManip.c
M src/backend/rewrite/rewriteRemove.c
M src/backend/rewrite/rewriteSearchCycle.c
M src/backend/rewrite/rewriteSupport.c
M src/backend/rewrite/rowsecurity.c
M src/backend/snowball/dict_snowball.c
M src/backend/snowball/meson.build
M src/backend/snowball/snowball.sql.in
M src/backend/snowball/snowball_create.pl
M src/backend/snowball/snowball_func.sql.in
M src/backend/statistics/attribute_stats.c
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/statistics/mcv.c
M src/backend/statistics/meson.build
M src/backend/statistics/mvdistinct.c
M src/backend/statistics/relation_stats.c
M src/backend/statistics/stat_utils.c
M src/backend/storage/aio/meson.build
M src/backend/storage/aio/read_stream.c
M src/backend/storage/buffer/buf_init.c
M src/backend/storage/buffer/buf_table.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/freelist.c
M src/backend/storage/buffer/localbuf.c
M src/backend/storage/buffer/meson.build
M src/backend/storage/file/buffile.c
M src/backend/storage/file/copydir.c
M src/backend/storage/file/fd.c
M src/backend/storage/file/fileset.c
M src/backend/storage/file/meson.build
M src/backend/storage/file/reinit.c
M src/backend/storage/file/sharedfileset.c
M src/backend/storage/freespace/freespace.c
M src/backend/storage/freespace/fsmpage.c
M src/backend/storage/freespace/indexfsm.c
M src/backend/storage/freespace/meson.build
M src/backend/storage/ipc/barrier.c
M src/backend/storage/ipc/dsm.c
M src/backend/storage/ipc/dsm_impl.c
M src/backend/storage/ipc/dsm_registry.c
M src/backend/storage/ipc/ipc.c
M src/backend/storage/ipc/ipci.c
M src/backend/storage/ipc/latch.c
M src/backend/storage/ipc/meson.build
M src/backend/storage/ipc/pmsignal.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/procsignal.c
M src/backend/storage/ipc/shm_mq.c
M src/backend/storage/ipc/shm_toc.c
M src/backend/storage/ipc/shmem.c
M src/backend/storage/ipc/signalfuncs.c
M src/backend/storage/ipc/sinval.c
M src/backend/storage/ipc/sinvaladt.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/large_object/inv_api.c
M src/backend/storage/large_object/meson.build
M src/backend/storage/lmgr/condition_variable.c
M src/backend/storage/lmgr/deadlock.c
M src/backend/storage/lmgr/generate-lwlocknames.pl
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/lwlock.c
M src/backend/storage/lmgr/meson.build
M src/backend/storage/lmgr/predicate.c
M src/backend/storage/lmgr/proc.c
M src/backend/storage/lmgr/s_lock.c
M src/backend/storage/meson.build
M src/backend/storage/page/bufpage.c
M src/backend/storage/page/checksum.c
M src/backend/storage/page/itemptr.c
M src/backend/storage/page/meson.build
M src/backend/storage/smgr/bulk_write.c
M src/backend/storage/smgr/md.c
M src/backend/storage/smgr/meson.build
M src/backend/storage/smgr/smgr.c
M src/backend/storage/sync/meson.build
M src/backend/storage/sync/sync.c
M src/backend/tcop/backend_startup.c
M src/backend/tcop/cmdtag.c
M src/backend/tcop/dest.c
M src/backend/tcop/fastpath.c
M src/backend/tcop/meson.build
M src/backend/tcop/postgres.c
M src/backend/tcop/pquery.c
M src/backend/tcop/utility.c
M src/backend/tsearch/Makefile
M src/backend/tsearch/dict.c
M src/backend/tsearch/dict_ispell.c
M src/backend/tsearch/dict_simple.c
M src/backend/tsearch/dict_synonym.c
M src/backend/tsearch/dict_thesaurus.c
M src/backend/tsearch/meson.build
M src/backend/tsearch/regis.c
M src/backend/tsearch/spell.c
M src/backend/tsearch/to_tsany.c
M src/backend/tsearch/ts_locale.c
M src/backend/tsearch/ts_parse.c
M src/backend/tsearch/ts_selfuncs.c
M src/backend/tsearch/ts_typanalyze.c
M src/backend/tsearch/ts_utils.c
M src/backend/tsearch/wparser.c
M src/backend/tsearch/wparser_def.c
M src/backend/utils/Gen_dummy_probes.pl
M src/backend/utils/Gen_fmgrtab.pl
M src/backend/utils/Makefile
M src/backend/utils/activity/Makefile
M src/backend/utils/activity/backend_progress.c
M src/backend/utils/activity/backend_status.c
M src/backend/utils/activity/generate-wait_event_types.pl
M src/backend/utils/activity/meson.build
M src/backend/utils/activity/pgstat.c
M src/backend/utils/activity/pgstat_archiver.c
M src/backend/utils/activity/pgstat_backend.c
M src/backend/utils/activity/pgstat_bgwriter.c
M src/backend/utils/activity/pgstat_checkpointer.c
M src/backend/utils/activity/pgstat_database.c
M src/backend/utils/activity/pgstat_function.c
M src/backend/utils/activity/pgstat_io.c
M src/backend/utils/activity/pgstat_relation.c
M src/backend/utils/activity/pgstat_replslot.c
M src/backend/utils/activity/pgstat_shmem.c
M src/backend/utils/activity/pgstat_slru.c
M src/backend/utils/activity/pgstat_subscription.c
M src/backend/utils/activity/pgstat_wal.c
M src/backend/utils/activity/pgstat_xact.c
M src/backend/utils/activity/wait_event.c
M src/backend/utils/activity/wait_event_funcs.c
M src/backend/utils/activity/wait_event_names.txt
M src/backend/utils/adt/acl.c
M src/backend/utils/adt/amutils.c
M src/backend/utils/adt/array_expanded.c
M src/backend/utils/adt/array_selfuncs.c
M src/backend/utils/adt/array_typanalyze.c
M src/backend/utils/adt/array_userfuncs.c
M src/backend/utils/adt/arrayfuncs.c
M src/backend/utils/adt/arraysubs.c
M src/backend/utils/adt/arrayutils.c
M src/backend/utils/adt/ascii.c
M src/backend/utils/adt/bool.c
M src/backend/utils/adt/char.c
M src/backend/utils/adt/cryptohashfuncs.c
M src/backend/utils/adt/date.c
M src/backend/utils/adt/datetime.c
M src/backend/utils/adt/datum.c
M src/backend/utils/adt/dbsize.c
M src/backend/utils/adt/domains.c
M src/backend/utils/adt/encode.c
M src/backend/utils/adt/enum.c
M src/backend/utils/adt/expandeddatum.c
M src/backend/utils/adt/expandedrecord.c
M src/backend/utils/adt/float.c
M src/backend/utils/adt/format_type.c
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/genfile.c
M src/backend/utils/adt/geo_ops.c
M src/backend/utils/adt/geo_selfuncs.c
M src/backend/utils/adt/geo_spgist.c
M src/backend/utils/adt/hbafuncs.c
M src/backend/utils/adt/int.c
M src/backend/utils/adt/int8.c
M src/backend/utils/adt/json.c
M src/backend/utils/adt/jsonb.c
M src/backend/utils/adt/jsonb_gin.c
M src/backend/utils/adt/jsonb_op.c
M src/backend/utils/adt/jsonb_util.c
M src/backend/utils/adt/jsonbsubs.c
M src/backend/utils/adt/jsonfuncs.c
M src/backend/utils/adt/jsonpath.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/adt/jsonpath_gram.y
M src/backend/utils/adt/jsonpath_internal.h
M src/backend/utils/adt/jsonpath_scan.l
M src/backend/utils/adt/levenshtein.c
M src/backend/utils/adt/like.c
M src/backend/utils/adt/like_match.c
M src/backend/utils/adt/like_support.c
M src/backend/utils/adt/lockfuncs.c
M src/backend/utils/adt/mac.c
M src/backend/utils/adt/mac8.c
M src/backend/utils/adt/mcxtfuncs.c
M src/backend/utils/adt/meson.build
M src/backend/utils/adt/misc.c
M src/backend/utils/adt/multirangetypes.c
M src/backend/utils/adt/multirangetypes_selfuncs.c
M src/backend/utils/adt/name.c
M src/backend/utils/adt/network_gist.c
M src/backend/utils/adt/network_selfuncs.c
M src/backend/utils/adt/network_spgist.c
M src/backend/utils/adt/numeric.c
M src/backend/utils/adt/numutils.c
M src/backend/utils/adt/oid.c
M src/backend/utils/adt/oracle_compat.c
M src/backend/utils/adt/orderedsetaggs.c
M src/backend/utils/adt/partitionfuncs.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_locale_builtin.c
M src/backend/utils/adt/pg_locale_icu.c
M src/backend/utils/adt/pg_locale_libc.c
M src/backend/utils/adt/pg_lsn.c
M src/backend/utils/adt/pg_upgrade_support.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/adt/pseudorandomfuncs.c
M src/backend/utils/adt/pseudotypes.c
M src/backend/utils/adt/quote.c
M src/backend/utils/adt/rangetypes.c
M src/backend/utils/adt/rangetypes_gist.c
M src/backend/utils/adt/rangetypes_selfuncs.c
M src/backend/utils/adt/rangetypes_spgist.c
M src/backend/utils/adt/rangetypes_typanalyze.c
M src/backend/utils/adt/regexp.c
M src/backend/utils/adt/regproc.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/adt/rowtypes.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/adt/selfuncs.c
M src/backend/utils/adt/tid.c
M src/backend/utils/adt/timestamp.c
M src/backend/utils/adt/trigfuncs.c
M src/backend/utils/adt/tsginidx.c
M src/backend/utils/adt/tsgistidx.c
M src/backend/utils/adt/tsquery.c
M src/backend/utils/adt/tsquery_cleanup.c
M src/backend/utils/adt/tsquery_gist.c
M src/backend/utils/adt/tsquery_op.c
M src/backend/utils/adt/tsquery_rewrite.c
M src/backend/utils/adt/tsquery_util.c
M src/backend/utils/adt/tsrank.c
M src/backend/utils/adt/tsvector.c
M src/backend/utils/adt/tsvector_op.c
M src/backend/utils/adt/tsvector_parser.c
M src/backend/utils/adt/uuid.c
M src/backend/utils/adt/varbit.c
M src/backend/utils/adt/varchar.c
M src/backend/utils/adt/varlena.c
M src/backend/utils/adt/version.c
M src/backend/utils/adt/waitfuncs.c
M src/backend/utils/adt/windowfuncs.c
M src/backend/utils/adt/xid.c
M src/backend/utils/adt/xid8funcs.c
M src/backend/utils/adt/xml.c
M src/backend/utils/cache/attoptcache.c
M src/backend/utils/cache/catcache.c
M src/backend/utils/cache/evtcache.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/lsyscache.c
M src/backend/utils/cache/meson.build
M src/backend/utils/cache/partcache.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/cache/relfilenumbermap.c
M src/backend/utils/cache/relmapper.c
M src/backend/utils/cache/spccache.c
M src/backend/utils/cache/syscache.c
M src/backend/utils/cache/ts_cache.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/errcodes.txt
M src/backend/utils/error/assert.c
M src/backend/utils/error/csvlog.c
M src/backend/utils/error/elog.c
M src/backend/utils/error/jsonlog.c
M src/backend/utils/error/meson.build
M src/backend/utils/fmgr/dfmgr.c
M src/backend/utils/fmgr/fmgr.c
M src/backend/utils/fmgr/funcapi.c
M src/backend/utils/fmgr/meson.build
M src/backend/utils/generate-errcodes.pl
M src/backend/utils/hash/dynahash.c
M src/backend/utils/hash/meson.build
M src/backend/utils/hash/pg_crc.c
M src/backend/utils/init/globals.c
M src/backend/utils/init/meson.build
M src/backend/utils/init/miscinit.c
M src/backend/utils/init/postinit.c
M src/backend/utils/init/usercontext.c
M src/backend/utils/mb/Unicode/Makefile
M src/backend/utils/mb/Unicode/UCS_to_BIG5.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_CN.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_JIS_2004.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_JP.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_KR.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_TW.pl
M src/backend/utils/mb/Unicode/UCS_to_GB18030.pl
M src/backend/utils/mb/Unicode/UCS_to_JOHAB.pl
M src/backend/utils/mb/Unicode/UCS_to_SHIFT_JIS_2004.pl
M src/backend/utils/mb/Unicode/UCS_to_SJIS.pl
M src/backend/utils/mb/Unicode/UCS_to_UHC.pl
M src/backend/utils/mb/Unicode/UCS_to_most.pl
M src/backend/utils/mb/Unicode/convutils.pm
M src/backend/utils/mb/conv.c
M src/backend/utils/mb/conversion_procs/Makefile
M src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
M src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
M src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
M src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
M src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
M src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
M src/backend/utils/mb/conversion_procs/meson.build
M src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
M src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
M src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
M src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
M src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
M src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
M src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
M src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
M src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
M src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
M src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
M src/backend/utils/mb/mbutils.c
M src/backend/utils/mb/meson.build
M src/backend/utils/mb/stringinfo_mb.c
M src/backend/utils/meson.build
M src/backend/utils/misc/conffiles.c
M src/backend/utils/misc/guc-file.l
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/guc_funcs.c
M src/backend/utils/misc/guc_internal.h
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/help_config.c
M src/backend/utils/misc/injection_point.c
M src/backend/utils/misc/meson.build
M src/backend/utils/misc/pg_config.c
M src/backend/utils/misc/pg_controldata.c
M src/backend/utils/misc/pg_rusage.c
M src/backend/utils/misc/ps_status.c
M src/backend/utils/misc/queryenvironment.c
M src/backend/utils/misc/rls.c
M src/backend/utils/misc/sampling.c
M src/backend/utils/misc/stack_depth.c
M src/backend/utils/misc/superuser.c
M src/backend/utils/misc/timeout.c
M src/backend/utils/misc/tzparser.c
M src/backend/utils/mmgr/alignedalloc.c
M src/backend/utils/mmgr/aset.c
M src/backend/utils/mmgr/bump.c
M src/backend/utils/mmgr/dsa.c
M src/backend/utils/mmgr/freepage.c
M src/backend/utils/mmgr/generation.c
M src/backend/utils/mmgr/mcxt.c
M src/backend/utils/mmgr/memdebug.c
M src/backend/utils/mmgr/meson.build
M src/backend/utils/mmgr/portalmem.c
M src/backend/utils/mmgr/slab.c
M src/backend/utils/postprocess_dtrace.sed
M src/backend/utils/probes.d
M src/backend/utils/resowner/meson.build
M src/backend/utils/resowner/resowner.c
M src/backend/utils/sort/logtape.c
M src/backend/utils/sort/meson.build
M src/backend/utils/sort/sharedtuplestore.c
M src/backend/utils/sort/sortsupport.c
M src/backend/utils/sort/tuplesort.c
M src/backend/utils/sort/tuplesortvariants.c
M src/backend/utils/sort/tuplestore.c
M src/backend/utils/time/combocid.c
M src/backend/utils/time/meson.build
M src/backend/utils/time/snapmgr.c
M src/bin/Makefile
M src/bin/initdb/Makefile
M src/bin/initdb/findtimezone.c
M src/bin/initdb/initdb.c
M src/bin/initdb/meson.build
M src/bin/initdb/po/meson.build
M src/bin/initdb/t/001_initdb.pl
M src/bin/meson.build
M src/bin/pg_amcheck/Makefile
M src/bin/pg_amcheck/meson.build
M src/bin/pg_amcheck/pg_amcheck.c
M src/bin/pg_amcheck/po/meson.build
M src/bin/pg_amcheck/t/001_basic.pl
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/bin/pg_amcheck/t/005_opclass_damage.pl
M src/bin/pg_archivecleanup/meson.build
M src/bin/pg_archivecleanup/po/meson.build
M src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
M src/bin/pg_basebackup/Makefile
M src/bin/pg_basebackup/astreamer_inject.c
M src/bin/pg_basebackup/astreamer_inject.h
M src/bin/pg_basebackup/meson.build
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_basebackup/pg_createsubscriber.c
M src/bin/pg_basebackup/pg_receivewal.c
M src/bin/pg_basebackup/pg_recvlogical.c
M src/bin/pg_basebackup/po/meson.build
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/receivelog.h
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_basebackup/streamutil.h
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_basebackup/t/011_in_place_tablespace.pl
M src/bin/pg_basebackup/t/020_pg_receivewal.pl
M src/bin/pg_basebackup/t/030_pg_recvlogical.pl
M src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_basebackup/walmethods.h
M src/bin/pg_checksums/Makefile
M src/bin/pg_checksums/meson.build
M src/bin/pg_checksums/pg_checksums.c
M src/bin/pg_checksums/po/meson.build
M src/bin/pg_checksums/t/001_basic.pl
M src/bin/pg_checksums/t/002_actions.pl
M src/bin/pg_combinebackup/Makefile
M src/bin/pg_combinebackup/backup_label.c
M src/bin/pg_combinebackup/backup_label.h
M src/bin/pg_combinebackup/copy_file.c
M src/bin/pg_combinebackup/copy_file.h
M src/bin/pg_combinebackup/load_manifest.c
M src/bin/pg_combinebackup/load_manifest.h
M src/bin/pg_combinebackup/meson.build
M src/bin/pg_combinebackup/pg_combinebackup.c
M src/bin/pg_combinebackup/po/meson.build
M src/bin/pg_combinebackup/reconstruct.c
M src/bin/pg_combinebackup/reconstruct.h
M src/bin/pg_combinebackup/t/001_basic.pl
M src/bin/pg_combinebackup/t/002_compare_backups.pl
M src/bin/pg_combinebackup/t/003_timeline.pl
M src/bin/pg_combinebackup/t/004_manifest.pl
M src/bin/pg_combinebackup/t/005_integrity.pl
M src/bin/pg_combinebackup/t/006_db_file_copy.pl
M src/bin/pg_combinebackup/t/007_wal_level_minimal.pl
M src/bin/pg_combinebackup/t/008_promote.pl
M src/bin/pg_combinebackup/t/009_no_full_file.pl
M src/bin/pg_combinebackup/write_manifest.c
M src/bin/pg_combinebackup/write_manifest.h
M src/bin/pg_config/Makefile
M src/bin/pg_config/meson.build
M src/bin/pg_config/pg_config.c
M src/bin/pg_config/po/meson.build
M src/bin/pg_config/t/001_pg_config.pl
M src/bin/pg_controldata/Makefile
M src/bin/pg_controldata/meson.build
M src/bin/pg_controldata/po/meson.build
M src/bin/pg_controldata/t/001_pg_controldata.pl
M src/bin/pg_ctl/Makefile
M src/bin/pg_ctl/meson.build
M src/bin/pg_ctl/pg_ctl.c
M src/bin/pg_ctl/po/meson.build
M src/bin/pg_ctl/t/001_start_stop.pl
M src/bin/pg_ctl/t/002_status.pl
M src/bin/pg_ctl/t/003_promote.pl
M src/bin/pg_ctl/t/004_logrotate.pl
M src/bin/pg_dump/Makefile
M src/bin/pg_dump/common.c
M src/bin/pg_dump/compress_gzip.c
M src/bin/pg_dump/compress_gzip.h
M src/bin/pg_dump/compress_io.c
M src/bin/pg_dump/compress_io.h
M src/bin/pg_dump/compress_lz4.c
M src/bin/pg_dump/compress_lz4.h
M src/bin/pg_dump/compress_none.c
M src/bin/pg_dump/compress_none.h
M src/bin/pg_dump/compress_zstd.c
M src/bin/pg_dump/compress_zstd.h
M src/bin/pg_dump/dumputils.c
M src/bin/pg_dump/dumputils.h
M src/bin/pg_dump/filter.c
M src/bin/pg_dump/filter.h
M src/bin/pg_dump/meson.build
M src/bin/pg_dump/parallel.c
M src/bin/pg_dump/parallel.h
M src/bin/pg_dump/pg_backup_directory.c
M src/bin/pg_dump/pg_backup_utils.c
M src/bin/pg_dump/pg_backup_utils.h
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/pg_dump_sort.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_dump/po/meson.build
M src/bin/pg_dump/t/001_basic.pl
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/pg_dump/t/003_pg_dump_with_server.pl
M src/bin/pg_dump/t/004_pg_dump_parallel.pl
M src/bin/pg_dump/t/005_pg_dump_filterfile.pl
M src/bin/pg_dump/t/010_dump_connstr.pl
M src/bin/pg_resetwal/Makefile
M src/bin/pg_resetwal/meson.build
M src/bin/pg_resetwal/pg_resetwal.c
M src/bin/pg_resetwal/po/meson.build
M src/bin/pg_resetwal/t/001_basic.pl
M src/bin/pg_resetwal/t/002_corrupted.pl
M src/bin/pg_rewind/Makefile
M src/bin/pg_rewind/datapagemap.c
M src/bin/pg_rewind/datapagemap.h
M src/bin/pg_rewind/file_ops.c
M src/bin/pg_rewind/file_ops.h
M src/bin/pg_rewind/filemap.c
M src/bin/pg_rewind/filemap.h
M src/bin/pg_rewind/libpq_source.c
M src/bin/pg_rewind/local_source.c
M src/bin/pg_rewind/meson.build
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/pg_rewind.h
M src/bin/pg_rewind/po/meson.build
M src/bin/pg_rewind/rewind_source.h
M src/bin/pg_rewind/t/001_basic.pl
M src/bin/pg_rewind/t/002_databases.pl
M src/bin/pg_rewind/t/003_extrafiles.pl
M src/bin/pg_rewind/t/004_pg_xlog_symlink.pl
M src/bin/pg_rewind/t/005_same_timeline.pl
M src/bin/pg_rewind/t/006_options.pl
M src/bin/pg_rewind/t/007_standby_source.pl
M src/bin/pg_rewind/t/008_min_recovery_point.pl
M src/bin/pg_rewind/t/009_growing_files.pl
M src/bin/pg_rewind/t/010_keep_recycled_wals.pl
M src/bin/pg_rewind/t/RewindTest.pm
M src/bin/pg_rewind/timeline.c
M src/bin/pg_test_fsync/meson.build
M src/bin/pg_test_fsync/pg_test_fsync.c
M src/bin/pg_test_fsync/po/meson.build
M src/bin/pg_test_fsync/t/001_basic.pl
M src/bin/pg_test_timing/meson.build
M src/bin/pg_test_timing/po/meson.build
M src/bin/pg_test_timing/t/001_basic.pl
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/dump.c
M src/bin/pg_upgrade/exec.c
M src/bin/pg_upgrade/file.c
M src/bin/pg_upgrade/function.c
M src/bin/pg_upgrade/info.c
M src/bin/pg_upgrade/meson.build
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/parallel.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/po/meson.build
M src/bin/pg_upgrade/relfilenumber.c
M src/bin/pg_upgrade/server.c
M src/bin/pg_upgrade/t/001_basic.pl
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
M src/bin/pg_upgrade/t/003_logical_slots.pl
M src/bin/pg_upgrade/t/004_subscription.pl
M src/bin/pg_upgrade/tablespace.c
M src/bin/pg_upgrade/task.c
M src/bin/pg_upgrade/util.c
M src/bin/pg_upgrade/version.c
M src/bin/pg_verifybackup/astreamer_verify.c
M src/bin/pg_verifybackup/meson.build
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/bin/pg_verifybackup/pg_verifybackup.h
M src/bin/pg_verifybackup/po/meson.build
M src/bin/pg_verifybackup/t/001_basic.pl
M src/bin/pg_verifybackup/t/002_algorithm.pl
M src/bin/pg_verifybackup/t/003_corruption.pl
M src/bin/pg_verifybackup/t/004_options.pl
M src/bin/pg_verifybackup/t/005_bad_manifest.pl
M src/bin/pg_verifybackup/t/006_encoding.pl
M src/bin/pg_verifybackup/t/007_wal.pl
M src/bin/pg_verifybackup/t/008_untar.pl
M src/bin/pg_verifybackup/t/009_extract.pl
M src/bin/pg_verifybackup/t/010_client_untar.pl
M src/bin/pg_waldump/compat.c
M src/bin/pg_waldump/meson.build
M src/bin/pg_waldump/pg_waldump.c
M src/bin/pg_waldump/po/meson.build
M src/bin/pg_waldump/t/001_basic.pl
M src/bin/pg_waldump/t/002_save_fullpage.pl
M src/bin/pg_walsummary/Makefile
M src/bin/pg_walsummary/meson.build
M src/bin/pg_walsummary/pg_walsummary.c
M src/bin/pg_walsummary/po/meson.build
M src/bin/pg_walsummary/t/001_basic.pl
M src/bin/pg_walsummary/t/002_blocks.pl
M src/bin/pgbench/exprparse.y
M src/bin/pgbench/exprscan.l
M src/bin/pgbench/meson.build
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/pgbench.h
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/pgbench/t/002_pgbench_no_server.pl
M src/bin/pgevent/Makefile
M src/bin/pgevent/meson.build
M src/bin/psql/Makefile
M src/bin/psql/command.c
M src/bin/psql/command.h
M src/bin/psql/common.c
M src/bin/psql/common.h
M src/bin/psql/copy.c
M src/bin/psql/copy.h
M src/bin/psql/create_help.pl
M src/bin/psql/crosstabview.c
M src/bin/psql/crosstabview.h
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/bin/psql/gen_tabcomplete.pl
M src/bin/psql/help.c
M src/bin/psql/help.h
M src/bin/psql/input.c
M src/bin/psql/input.h
M src/bin/psql/large_obj.c
M src/bin/psql/large_obj.h
M src/bin/psql/mainloop.c
M src/bin/psql/mainloop.h
M src/bin/psql/meson.build
M src/bin/psql/po/meson.build
M src/bin/psql/prompt.c
M src/bin/psql/prompt.h
M src/bin/psql/psqlscanslash.h
M src/bin/psql/psqlscanslash.l
M src/bin/psql/settings.h
M src/bin/psql/startup.c
M src/bin/psql/stringutils.c
M src/bin/psql/stringutils.h
M src/bin/psql/t/001_basic.pl
M src/bin/psql/t/010_tab_completion.pl
M src/bin/psql/t/020_cancel.pl
M src/bin/psql/tab-complete.h
M src/bin/psql/tab-complete.in.c
M src/bin/psql/variables.c
M src/bin/psql/variables.h
M src/bin/scripts/Makefile
M src/bin/scripts/clusterdb.c
M src/bin/scripts/common.c
M src/bin/scripts/common.h
M src/bin/scripts/createdb.c
M src/bin/scripts/createuser.c
M src/bin/scripts/dropdb.c
M src/bin/scripts/dropuser.c
M src/bin/scripts/meson.build
M src/bin/scripts/pg_isready.c
M src/bin/scripts/po/meson.build
M src/bin/scripts/reindexdb.c
M src/bin/scripts/t/010_clusterdb.pl
M src/bin/scripts/t/011_clusterdb_all.pl
M src/bin/scripts/t/020_createdb.pl
M src/bin/scripts/t/040_createuser.pl
M src/bin/scripts/t/050_dropdb.pl
M src/bin/scripts/t/070_dropuser.pl
M src/bin/scripts/t/080_pg_isready.pl
M src/bin/scripts/t/090_reindexdb.pl
M src/bin/scripts/t/091_reindexdb_all.pl
M src/bin/scripts/t/100_vacuumdb.pl
M src/bin/scripts/t/101_vacuumdb_all.pl
M src/bin/scripts/t/102_vacuumdb_stages.pl
M src/bin/scripts/t/200_connstr.pl
M src/bin/scripts/vacuumdb.c
M src/common/archive.c
M src/common/base64.c
M src/common/binaryheap.c
M src/common/blkreftable.c
M src/common/checksum_helper.c
M src/common/compression.c
M src/common/config_info.c
M src/common/controldata_utils.c
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
M src/common/d2s.c
M src/common/d2s_full_table.h
M src/common/d2s_intrinsics.h
M src/common/encnames.c
M src/common/exec.c
M src/common/f2s.c
M src/common/fe_memutils.c
M src/common/file_perm.c
M src/common/file_utils.c
M src/common/hashfn.c
M src/common/hmac.c
M src/common/hmac_openssl.c
M src/common/ip.c
M src/common/jsonapi.c
M src/common/keywords.c
M src/common/kwlookup.c
M src/common/link-canary.c
M src/common/logging.c
M src/common/md5.c
M src/common/md5_common.c
M src/common/md5_int.h
M src/common/meson.build
M src/common/parse_manifest.c
M src/common/percentrepl.c
M src/common/pg_get_line.c
M src/common/pg_lzcompress.c
M src/common/pg_prng.c
M src/common/pgfnames.c
M src/common/psprintf.c
M src/common/relpath.c
M src/common/restricted_token.c
M src/common/rmtree.c
M src/common/ryu_common.h
M src/common/saslprep.c
M src/common/scram-common.c
M src/common/sha1.c
M src/common/sha1_int.h
M src/common/sha2.c
M src/common/sha2_int.h
M src/common/sprompt.c
M src/common/string.c
M src/common/stringinfo.c
M src/common/unicode/case_test.c
M src/common/unicode/category_test.c
M src/common/unicode/generate-norm_test_table.pl
M src/common/unicode/generate-unicode_case_table.pl
M src/common/unicode/generate-unicode_category_table.pl
M src/common/unicode/generate-unicode_east_asian_fw_table.pl
M src/common/unicode/generate-unicode_nonspacing_table.pl
M src/common/unicode/generate-unicode_norm_table.pl
M src/common/unicode/generate-unicode_normprops_table.pl
M src/common/unicode/generate-unicode_version.pl
M src/common/unicode/meson.build
M src/common/unicode/norm_test.c
M src/common/unicode_case.c
M src/common/unicode_category.c
M src/common/unicode_norm.c
M src/common/username.c
M src/common/wait_error.c
M src/common/wchar.c
M src/fe_utils/Makefile
M src/fe_utils/archive.c
M src/fe_utils/astreamer_file.c
M src/fe_utils/astreamer_gzip.c
M src/fe_utils/astreamer_lz4.c
M src/fe_utils/astreamer_tar.c
M src/fe_utils/astreamer_zstd.c
M src/fe_utils/cancel.c
M src/fe_utils/conditional.c
M src/fe_utils/connect_utils.c
M src/fe_utils/mbprint.c
M src/fe_utils/meson.build
M src/fe_utils/option_utils.c
M src/fe_utils/parallel_slot.c
M src/fe_utils/print.c
M src/fe_utils/psqlscan.l
M src/fe_utils/query_utils.c
M src/fe_utils/recovery_gen.c
M src/fe_utils/simple_list.c
M src/fe_utils/string_utils.c
M src/include/access/amapi.h
M src/include/access/amvalidate.h
M src/include/access/attmap.h
M src/include/access/attnum.h
M src/include/access/brin.h
M src/include/access/brin_internal.h
M src/include/access/brin_page.h
M src/include/access/brin_pageops.h
M src/include/access/brin_revmap.h
M src/include/access/brin_tuple.h
M src/include/access/brin_xlog.h
M src/include/access/bufmask.h
M src/include/access/clog.h
M src/include/access/commit_ts.h
M src/include/access/detoast.h
M src/include/access/genam.h
M src/include/access/generic_xlog.h
M src/include/access/gin.h
M src/include/access/gin_private.h
M src/include/access/ginblock.h
M src/include/access/ginxlog.h
M src/include/access/gist.h
M src/include/access/gist_private.h
M src/include/access/gistscan.h
M src/include/access/gistxlog.h
M src/include/access/hash.h
M src/include/access/hash_xlog.h
M src/include/access/heapam.h
M src/include/access/heapam_xlog.h
M src/include/access/heaptoast.h
M src/include/access/hio.h
M src/include/access/htup.h
M src/include/access/htup_details.h
M src/include/access/itup.h
M src/include/access/multixact.h
M src/include/access/nbtree.h
M src/include/access/nbtxlog.h
M src/include/access/parallel.h
M src/include/access/printsimple.h
M src/include/access/printtup.h
M src/include/access/relation.h
M src/include/access/reloptions.h
M src/include/access/relscan.h
M src/include/access/rewriteheap.h
M src/include/access/rmgrdesc_utils.h
M src/include/access/rmgrlist.h
M src/include/access/sdir.h
M src/include/access/sequence.h
M src/include/access/session.h
M src/include/access/skey.h
M src/include/access/slru.h
M src/include/access/spgist.h
M src/include/access/spgist_private.h
M src/include/access/spgxlog.h
M src/include/access/stratnum.h
M src/include/access/subtrans.h
M src/include/access/syncscan.h
M src/include/access/sysattr.h
M src/include/access/table.h
M src/include/access/tableam.h
M src/include/access/tidstore.h
M src/include/access/timeline.h
M src/include/access/toast_compression.h
M src/include/access/toast_helper.h
M src/include/access/toast_internals.h
M src/include/access/transam.h
M src/include/access/tsmapi.h
M src/include/access/tupconvert.h
M src/include/access/tupdesc.h
M src/include/access/tupdesc_details.h
M src/include/access/tupmacs.h
M src/include/access/twophase.h
M src/include/access/twophase_rmgr.h
M src/include/access/valid.h
M src/include/access/visibilitymap.h
M src/include/access/visibilitymapdefs.h
M src/include/access/xact.h
M src/include/access/xlog.h
M src/include/access/xlog_internal.h
M src/include/access/xlogarchive.h
M src/include/access/xlogbackup.h
M src/include/access/xlogdefs.h
M src/include/access/xloginsert.h
M src/include/access/xlogprefetcher.h
M src/include/access/xlogreader.h
M src/include/access/xlogrecord.h
M src/include/access/xlogrecovery.h
M src/include/access/xlogstats.h
M src/include/access/xlogutils.h
M src/include/archive/archive_module.h
M src/include/archive/shell_archive.h
M src/include/backup/backup_manifest.h
M src/include/backup/basebackup.h
M src/include/backup/basebackup_incremental.h
M src/include/backup/basebackup_sink.h
M src/include/backup/basebackup_target.h
M src/include/backup/walsummary.h
M src/include/bootstrap/bootstrap.h
M src/include/c.h
M src/include/catalog/Makefile
M src/include/catalog/binary_upgrade.h
M src/include/catalog/catalog.h
M src/include/catalog/catversion.h
M src/include/catalog/dependency.h
M src/include/catalog/duplicate_oids
M src/include/catalog/genbki.h
M src/include/catalog/heap.h
M src/include/catalog/index.h
M src/include/catalog/indexing.h
M src/include/catalog/meson.build
M src/include/catalog/namespace.h
M src/include/catalog/objectaccess.h
M src/include/catalog/objectaddress.h
M src/include/catalog/partition.h
M src/include/catalog/pg_aggregate.dat
M src/include/catalog/pg_aggregate.h
M src/include/catalog/pg_am.dat
M src/include/catalog/pg_am.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amop.h
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_amproc.h
M src/include/catalog/pg_attrdef.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_auth_members.h
M src/include/catalog/pg_authid.dat
M src/include/catalog/pg_authid.h
M src/include/catalog/pg_cast.dat
M src/include/catalog/pg_cast.h
M src/include/catalog/pg_class.dat
M src/include/catalog/pg_class.h
M src/include/catalog/pg_collation.dat
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_control.h
M src/include/catalog/pg_conversion.dat
M src/include/catalog/pg_conversion.h
M src/include/catalog/pg_database.dat
M src/include/catalog/pg_database.h
M src/include/catalog/pg_db_role_setting.h
M src/include/catalog/pg_default_acl.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_description.h
M src/include/catalog/pg_enum.h
M src/include/catalog/pg_event_trigger.h
M src/include/catalog/pg_extension.h
M src/include/catalog/pg_foreign_data_wrapper.h
M src/include/catalog/pg_foreign_server.h
M src/include/catalog/pg_foreign_table.h
M src/include/catalog/pg_index.h
M src/include/catalog/pg_inherits.h
M src/include/catalog/pg_init_privs.h
M src/include/catalog/pg_language.dat
M src/include/catalog/pg_language.h
M src/include/catalog/pg_largeobject.h
M src/include/catalog/pg_largeobject_metadata.h
M src/include/catalog/pg_namespace.dat
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_opclass.h
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_opfamily.h
M src/include/catalog/pg_parameter_acl.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_policy.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_publication.h
M src/include/catalog/pg_publication_namespace.h
M src/include/catalog/pg_publication_rel.h
M src/include/catalog/pg_range.dat
M src/include/catalog/pg_range.h
M src/include/catalog/pg_replication_origin.h
M src/include/catalog/pg_rewrite.h
M src/include/catalog/pg_seclabel.h
M src/include/catalog/pg_sequence.h
M src/include/catalog/pg_shdepend.h
M src/include/catalog/pg_shdescription.h
M src/include/catalog/pg_shseclabel.h
M src/include/catalog/pg_statistic.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/catalog/pg_subscription.h
M src/include/catalog/pg_subscription_rel.h
M src/include/catalog/pg_tablespace.dat
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_transform.h
M src/include/catalog/pg_trigger.h
M src/include/catalog/pg_ts_config.dat
M src/include/catalog/pg_ts_config.h
M src/include/catalog/pg_ts_config_map.dat
M src/include/catalog/pg_ts_config_map.h
M src/include/catalog/pg_ts_dict.dat
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_ts_parser.dat
M src/include/catalog/pg_ts_parser.h
M src/include/catalog/pg_ts_template.dat
M src/include/catalog/pg_ts_template.h
M src/include/catalog/pg_type.dat
M src/include/catalog/pg_type.h
M src/include/catalog/pg_user_mapping.h
M src/include/catalog/reformat_dat_file.pl
M src/include/catalog/renumber_oids.pl
M src/include/catalog/storage.h
M src/include/catalog/storage_xlog.h
M src/include/catalog/toasting.h
M src/include/catalog/unused_oids
M src/include/commands/alter.h
M src/include/commands/async.h
M src/include/commands/cluster.h
M src/include/commands/collationcmds.h
M src/include/commands/comment.h
M src/include/commands/conversioncmds.h
M src/include/commands/copy.h
M src/include/commands/copyfrom_internal.h
M src/include/commands/createas.h
M src/include/commands/dbcommands.h
M src/include/commands/dbcommands_xlog.h
M src/include/commands/defrem.h
M src/include/commands/discard.h
M src/include/commands/event_trigger.h
M src/include/commands/explain.h
M src/include/commands/extension.h
M src/include/commands/lockcmds.h
M src/include/commands/matview.h
M src/include/commands/policy.h
M src/include/commands/portalcmds.h
M src/include/commands/prepare.h
M src/include/commands/proclang.h
M src/include/commands/progress.h
M src/include/commands/publicationcmds.h
M src/include/commands/schemacmds.h
M src/include/commands/seclabel.h
M src/include/commands/sequence.h
M src/include/commands/subscriptioncmds.h
M src/include/commands/tablecmds.h
M src/include/commands/tablespace.h
M src/include/commands/trigger.h
M src/include/commands/typecmds.h
M src/include/commands/vacuum.h
M src/include/commands/view.h
M src/include/common/archive.h
M src/include/common/base64.h
M src/include/common/blkreftable.h
M src/include/common/checksum_helper.h
M src/include/common/compression.h
M src/include/common/config_info.h
M src/include/common/connect.h
M src/include/common/controldata_utils.h
M src/include/common/cryptohash.h
M src/include/common/fe_memutils.h
M src/include/common/file_perm.h
M src/include/common/file_utils.h
M src/include/common/hashfn.h
M src/include/common/hashfn_unstable.h
M src/include/common/hmac.h
M src/include/common/int.h
M src/include/common/int128.h
M src/include/common/ip.h
M src/include/common/jsonapi.h
M src/include/common/keywords.h
M src/include/common/kwlookup.h
M src/include/common/link-canary.h
M src/include/common/logging.h
M src/include/common/md5.h
M src/include/common/openssl.h
M src/include/common/parse_manifest.h
M src/include/common/percentrepl.h
M src/include/common/pg_prng.h
M src/include/common/relpath.h
M src/include/common/restricted_token.h
M src/include/common/saslprep.h
M src/include/common/scram-common.h
M src/include/common/sha1.h
M src/include/common/sha2.h
M src/include/common/shortest_dec.h
M src/include/common/string.h
M src/include/common/unicode_case.h
M src/include/common/unicode_case_table.h
M src/include/common/unicode_category.h
M src/include/common/unicode_category_table.h
M src/include/common/unicode_norm.h
M src/include/common/unicode_norm_hashfunc.h
M src/include/common/unicode_norm_table.h
M src/include/common/unicode_version.h
M src/include/common/username.h
M src/include/datatype/timestamp.h
M src/include/executor/execAsync.h
M src/include/executor/execExpr.h
M src/include/executor/execParallel.h
M src/include/executor/execPartition.h
M src/include/executor/execdebug.h
M src/include/executor/execdesc.h
M src/include/executor/executor.h
M src/include/executor/functions.h
M src/include/executor/hashjoin.h
M src/include/executor/instrument.h
M src/include/executor/nodeAgg.h
M src/include/executor/nodeAppend.h
M src/include/executor/nodeBitmapAnd.h
M src/include/executor/nodeBitmapHeapscan.h
M src/include/executor/nodeBitmapIndexscan.h
M src/include/executor/nodeBitmapOr.h
M src/include/executor/nodeCtescan.h
M src/include/executor/nodeCustom.h
M src/include/executor/nodeForeignscan.h
M src/include/executor/nodeFunctionscan.h
M src/include/executor/nodeGather.h
M src/include/executor/nodeGatherMerge.h
M src/include/executor/nodeGroup.h
M src/include/executor/nodeHash.h
M src/include/executor/nodeHashjoin.h
M src/include/executor/nodeIncrementalSort.h
M src/include/executor/nodeIndexonlyscan.h
M src/include/executor/nodeIndexscan.h
M src/include/executor/nodeLimit.h
M src/include/executor/nodeLockRows.h
M src/include/executor/nodeMaterial.h
M src/include/executor/nodeMemoize.h
M src/include/executor/nodeMergeAppend.h
M src/include/executor/nodeMergejoin.h
M src/include/executor/nodeModifyTable.h
M src/include/executor/nodeNamedtuplestorescan.h
M src/include/executor/nodeNestloop.h
M src/include/executor/nodeProjectSet.h
M src/include/executor/nodeRecursiveunion.h
M src/include/executor/nodeResult.h
M src/include/executor/nodeSamplescan.h
M src/include/executor/nodeSeqscan.h
M src/include/executor/nodeSetOp.h
M src/include/executor/nodeSort.h
M src/include/executor/nodeSubplan.h
M src/include/executor/nodeSubqueryscan.h
M src/include/executor/nodeTableFuncscan.h
M src/include/executor/nodeTidrangescan.h
M src/include/executor/nodeTidscan.h
M src/include/executor/nodeUnique.h
M src/include/executor/nodeValuesscan.h
M src/include/executor/nodeWindowAgg.h
M src/include/executor/nodeWorktablescan.h
M src/include/executor/spi.h
M src/include/executor/spi_priv.h
M src/include/executor/tablefunc.h
M src/include/executor/tqueue.h
M src/include/executor/tstoreReceiver.h
M src/include/executor/tuptable.h
M src/include/fe_utils/archive.h
M src/include/fe_utils/astreamer.h
M src/include/fe_utils/cancel.h
M src/include/fe_utils/conditional.h
M src/include/fe_utils/connect_utils.h
M src/include/fe_utils/mbprint.h
M src/include/fe_utils/option_utils.h
M src/include/fe_utils/parallel_slot.h
M src/include/fe_utils/print.h
M src/include/fe_utils/psqlscan.h
M src/include/fe_utils/psqlscan_int.h
M src/include/fe_utils/query_utils.h
M src/include/fe_utils/recovery_gen.h
M src/include/fe_utils/simple_list.h
M src/include/fe_utils/string_utils.h
M src/include/fmgr.h
M src/include/foreign/fdwapi.h
M src/include/foreign/foreign.h
M src/include/funcapi.h
M src/include/getopt_long.h
M src/include/jit/jit.h
M src/include/jit/llvmjit.h
M src/include/jit/llvmjit_emit.h
M src/include/lib/binaryheap.h
M src/include/lib/bipartite_match.h
M src/include/lib/bloomfilter.h
M src/include/lib/dshash.h
M src/include/lib/hyperloglog.h
M src/include/lib/ilist.h
M src/include/lib/integerset.h
M src/include/lib/knapsack.h
M src/include/lib/pairingheap.h
M src/include/lib/qunique.h
M src/include/lib/radixtree.h
M src/include/lib/rbtree.h
M src/include/lib/simplehash.h
M src/include/lib/sort_template.h
M src/include/lib/stringinfo.h
M src/include/libpq/auth.h
M src/include/libpq/be-fsstubs.h
M src/include/libpq/be-gssapi-common.h
M src/include/libpq/crypt.h
M src/include/libpq/ifaddr.h
M src/include/libpq/libpq-be-fe-helpers.h
M src/include/libpq/libpq-be.h
M src/include/libpq/libpq-fs.h
M src/include/libpq/libpq.h
M src/include/libpq/pqcomm.h
M src/include/libpq/pqformat.h
M src/include/libpq/pqmq.h
M src/include/libpq/pqsignal.h
M src/include/libpq/protocol.h
M src/include/libpq/sasl.h
M src/include/libpq/scram.h
M src/include/mb/pg_wchar.h
M src/include/mb/stringinfo_mb.h
M src/include/meson.build
M src/include/miscadmin.h
M src/include/nodes/bitmapset.h
M src/include/nodes/execnodes.h
M src/include/nodes/extensible.h
M src/include/nodes/lockoptions.h
M src/include/nodes/makefuncs.h
M src/include/nodes/memnodes.h
M src/include/nodes/meson.build
M src/include/nodes/miscnodes.h
M src/include/nodes/multibitmapset.h
M src/include/nodes/nodeFuncs.h
M src/include/nodes/nodes.h
M src/include/nodes/params.h
M src/include/nodes/parsenodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/pg_list.h
M src/include/nodes/plannodes.h
M src/include/nodes/primnodes.h
M src/include/nodes/print.h
M src/include/nodes/queryjumble.h
M src/include/nodes/readfuncs.h
M src/include/nodes/replnodes.h
M src/include/nodes/subscripting.h
M src/include/nodes/supportnodes.h
M src/include/nodes/tidbitmap.h
M src/include/nodes/value.h
M src/include/optimizer/appendinfo.h
M src/include/optimizer/clauses.h
M src/include/optimizer/cost.h
M src/include/optimizer/geqo.h
M src/include/optimizer/geqo_copy.h
M src/include/optimizer/geqo_gene.h
M src/include/optimizer/geqo_misc.h
M src/include/optimizer/geqo_mutation.h
M src/include/optimizer/geqo_pool.h
M src/include/optimizer/geqo_random.h
M src/include/optimizer/geqo_recombination.h
M src/include/optimizer/geqo_selection.h
M src/include/optimizer/inherit.h
M src/include/optimizer/joininfo.h
M src/include/optimizer/optimizer.h
M src/include/optimizer/orclauses.h
M src/include/optimizer/paramassign.h
M src/include/optimizer/pathnode.h
M src/include/optimizer/paths.h
M src/include/optimizer/placeholder.h
M src/include/optimizer/plancat.h
M src/include/optimizer/planmain.h
M src/include/optimizer/planner.h
M src/include/optimizer/prep.h
M src/include/optimizer/restrictinfo.h
M src/include/optimizer/subselect.h
M src/include/optimizer/tlist.h
M src/include/parser/analyze.h
M src/include/parser/kwlist.h
M src/include/parser/parse_agg.h
M src/include/parser/parse_clause.h
M src/include/parser/parse_coerce.h
M src/include/parser/parse_collate.h
M src/include/parser/parse_cte.h
M src/include/parser/parse_enr.h
M src/include/parser/parse_expr.h
M src/include/parser/parse_func.h
M src/include/parser/parse_merge.h
M src/include/parser/parse_node.h
M src/include/parser/parse_oper.h
M src/include/parser/parse_param.h
M src/include/parser/parse_relation.h
M src/include/parser/parse_target.h
M src/include/parser/parse_type.h
M src/include/parser/parse_utilcmd.h
M src/include/parser/parser.h
M src/include/parser/parsetree.h
M src/include/parser/scanner.h
M src/include/parser/scansup.h
M src/include/partitioning/partbounds.h
M src/include/partitioning/partdefs.h
M src/include/partitioning/partdesc.h
M src/include/partitioning/partprune.h
M src/include/pch/meson.build
M src/include/pg_config_manual.h
M src/include/pg_getopt.h
M src/include/pg_trace.h
M src/include/pgstat.h
M src/include/pgtar.h
M src/include/pgtime.h
M src/include/port.h
M src/include/port/atomics.h
M src/include/port/atomics/arch-arm.h
M src/include/port/atomics/arch-ppc.h
M src/include/port/atomics/arch-x86.h
M src/include/port/atomics/fallback.h
M src/include/port/atomics/generic-gcc.h
M src/include/port/atomics/generic-msvc.h
M src/include/port/atomics/generic-sunpro.h
M src/include/port/atomics/generic.h
M src/include/port/pg_bitutils.h
M src/include/port/pg_bswap.h
M src/include/port/pg_crc32c.h
M src/include/port/pg_iovec.h
M src/include/port/pg_lfind.h
M src/include/port/simd.h
M src/include/port/win32_port.h
M src/include/port/win32ntdll.h
M src/include/portability/instr_time.h
M src/include/portability/mem.h
M src/include/postgres.h
M src/include/postgres_fe.h
M src/include/postmaster/autovacuum.h
M src/include/postmaster/auxprocess.h
M src/include/postmaster/bgworker.h
M src/include/postmaster/bgworker_internals.h
M src/include/postmaster/bgwriter.h
M src/include/postmaster/fork_process.h
M src/include/postmaster/interrupt.h
M src/include/postmaster/pgarch.h
M src/include/postmaster/postmaster.h
M src/include/postmaster/startup.h
M src/include/postmaster/syslogger.h
M src/include/postmaster/walsummarizer.h
M src/include/postmaster/walwriter.h
M src/include/regex/regexport.h
M src/include/replication/conflict.h
M src/include/replication/decode.h
M src/include/replication/logical.h
M src/include/replication/logicallauncher.h
M src/include/replication/logicalproto.h
M src/include/replication/logicalrelation.h
M src/include/replication/logicalworker.h
M src/include/replication/message.h
M src/include/replication/origin.h
M src/include/replication/output_plugin.h
M src/include/replication/pgoutput.h
M src/include/replication/reorderbuffer.h
M src/include/replication/slot.h
M src/include/replication/slotsync.h
M src/include/replication/snapbuild.h
M src/include/replication/snapbuild_internal.h
M src/include/replication/syncrep.h
M src/include/replication/walreceiver.h
M src/include/replication/walsender.h
M src/include/replication/walsender_private.h
M src/include/replication/worker_internal.h
M src/include/rewrite/prs2lock.h
M src/include/rewrite/rewriteDefine.h
M src/include/rewrite/rewriteHandler.h
M src/include/rewrite/rewriteManip.h
M src/include/rewrite/rewriteRemove.h
M src/include/rewrite/rewriteSearchCycle.h
M src/include/rewrite/rewriteSupport.h
M src/include/rewrite/rowsecurity.h
M src/include/snowball/header.h
M src/include/statistics/extended_stats_internal.h
M src/include/statistics/stat_utils.h
M src/include/statistics/statistics.h
M src/include/storage/barrier.h
M src/include/storage/block.h
M src/include/storage/buf.h
M src/include/storage/buf_internals.h
M src/include/storage/buffile.h
M src/include/storage/bufmgr.h
M src/include/storage/bufpage.h
M src/include/storage/bulk_write.h
M src/include/storage/checksum.h
M src/include/storage/checksum_impl.h
M src/include/storage/condition_variable.h
M src/include/storage/copydir.h
M src/include/storage/dsm.h
M src/include/storage/dsm_impl.h
M src/include/storage/dsm_registry.h
M src/include/storage/fd.h
M src/include/storage/fileset.h
M src/include/storage/freespace.h
M src/include/storage/fsm_internals.h
M src/include/storage/indexfsm.h
M src/include/storage/ipc.h
M src/include/storage/item.h
M src/include/storage/itemid.h
M src/include/storage/itemptr.h
M src/include/storage/large_object.h
M src/include/storage/latch.h
M src/include/storage/lmgr.h
M src/include/storage/lock.h
M src/include/storage/lockdefs.h
M src/include/storage/lwlock.h
M src/include/storage/lwlocklist.h
M src/include/storage/md.h
M src/include/storage/meson.build
M src/include/storage/off.h
M src/include/storage/pg_sema.h
M src/include/storage/pg_shmem.h
M src/include/storage/pmsignal.h
M src/include/storage/predicate.h
M src/include/storage/predicate_internals.h
M src/include/storage/proc.h
M src/include/storage/procarray.h
M src/include/storage/proclist.h
M src/include/storage/proclist_types.h
M src/include/storage/procnumber.h
M src/include/storage/procsignal.h
M src/include/storage/read_stream.h
M src/include/storage/reinit.h
M src/include/storage/relfilelocator.h
M src/include/storage/s_lock.h
M src/include/storage/sharedfileset.h
M src/include/storage/shm_mq.h
M src/include/storage/shm_toc.h
M src/include/storage/shmem.h
M src/include/storage/sinval.h
M src/include/storage/sinvaladt.h
M src/include/storage/smgr.h
M src/include/storage/spin.h
M src/include/storage/standby.h
M src/include/storage/standbydefs.h
M src/include/storage/sync.h
M src/include/tcop/backend_startup.h
M src/include/tcop/cmdtag.h
M src/include/tcop/cmdtaglist.h
M src/include/tcop/deparse_utility.h
M src/include/tcop/dest.h
M src/include/tcop/fastpath.h
M src/include/tcop/pquery.h
M src/include/tcop/tcopprot.h
M src/include/tcop/utility.h
M src/include/tsearch/dicts/regis.h
M src/include/tsearch/dicts/spell.h
M src/include/tsearch/ts_cache.h
M src/include/tsearch/ts_locale.h
M src/include/tsearch/ts_public.h
M src/include/tsearch/ts_type.h
M src/include/tsearch/ts_utils.h
M src/include/utils/acl.h
M src/include/utils/aclchk_internal.h
M src/include/utils/array.h
M src/include/utils/arrayaccess.h
M src/include/utils/ascii.h
M src/include/utils/attoptcache.h
M src/include/utils/backend_progress.h
M src/include/utils/backend_status.h
M src/include/utils/builtins.h
M src/include/utils/bytea.h
M src/include/utils/catcache.h
M src/include/utils/combocid.h
M src/include/utils/conffiles.h
M src/include/utils/date.h
M src/include/utils/datetime.h
M src/include/utils/datum.h
M src/include/utils/dsa.h
M src/include/utils/dynahash.h
M src/include/utils/elog.h
M src/include/utils/evtcache.h
M src/include/utils/expandeddatum.h
M src/include/utils/expandedrecord.h
M src/include/utils/float.h
M src/include/utils/fmgrtab.h
M src/include/utils/formatting.h
M src/include/utils/freepage.h
M src/include/utils/geo_decls.h
M src/include/utils/guc.h
M src/include/utils/guc_hooks.h
M src/include/utils/guc_tables.h
M src/include/utils/help_config.h
M src/include/utils/hsearch.h
M src/include/utils/index_selfuncs.h
M src/include/utils/inet.h
M src/include/utils/injection_point.h
M src/include/utils/inval.h
M src/include/utils/json.h
M src/include/utils/jsonb.h
M src/include/utils/jsonfuncs.h
M src/include/utils/jsonpath.h
M src/include/utils/logtape.h
M src/include/utils/lsyscache.h
M src/include/utils/memdebug.h
M src/include/utils/memutils.h
M src/include/utils/memutils_internal.h
M src/include/utils/memutils_memorychunk.h
M src/include/utils/meson.build
M src/include/utils/multirangetypes.h
M src/include/utils/numeric.h
M src/include/utils/palloc.h
M src/include/utils/partcache.h
M src/include/utils/pg_crc.h
M src/include/utils/pg_locale.h
M src/include/utils/pg_lsn.h
M src/include/utils/pg_rusage.h
M src/include/utils/pgstat_internal.h
M src/include/utils/pidfile.h
M src/include/utils/plancache.h
M src/include/utils/portal.h
M src/include/utils/queryenvironment.h
M src/include/utils/rangetypes.h
M src/include/utils/regproc.h
M src/include/utils/rel.h
M src/include/utils/relcache.h
M src/include/utils/relfilenumbermap.h
M src/include/utils/relmapper.h
M src/include/utils/relptr.h
M src/include/utils/reltrigger.h
M src/include/utils/resowner.h
M src/include/utils/rls.h
M src/include/utils/ruleutils.h
M src/include/utils/sampling.h
M src/include/utils/selfuncs.h
M src/include/utils/sharedtuplestore.h
M src/include/utils/snapmgr.h
M src/include/utils/snapshot.h
M src/include/utils/sortsupport.h
M src/include/utils/spccache.h
M src/include/utils/syscache.h
M src/include/utils/timeout.h
M src/include/utils/timestamp.h
M src/include/utils/tuplesort.h
M src/include/utils/tuplestore.h
M src/include/utils/typcache.h
M src/include/utils/tzparser.h
M src/include/utils/uuid.h
M src/include/utils/varbit.h
M src/include/utils/varlena.h
M src/include/utils/wait_event.h
M src/include/utils/xid8.h
M src/include/utils/xml.h
M src/include/varatt.h
M src/include/windowapi.h
M src/interfaces/ecpg/compatlib/Makefile
M src/interfaces/ecpg/compatlib/meson.build
M src/interfaces/ecpg/ecpglib/Makefile
M src/interfaces/ecpg/ecpglib/meson.build
M src/interfaces/ecpg/ecpglib/po/meson.build
M src/interfaces/ecpg/include/meson.build
M src/interfaces/ecpg/meson.build
M src/interfaces/ecpg/pgtypeslib/Makefile
M src/interfaces/ecpg/pgtypeslib/meson.build
M src/interfaces/ecpg/preproc/Makefile
M src/interfaces/ecpg/preproc/c_kwlist.h
M src/interfaces/ecpg/preproc/ecpg.c
M src/interfaces/ecpg/preproc/ecpg_kwlist.h
M src/interfaces/ecpg/preproc/keywords.c
M src/interfaces/ecpg/preproc/meson.build
M src/interfaces/ecpg/preproc/parse.pl
M src/interfaces/ecpg/preproc/parser.c
M src/interfaces/ecpg/preproc/pgc.l
M src/interfaces/ecpg/preproc/po/meson.build
M src/interfaces/ecpg/test/compat_informix/meson.build
M src/interfaces/ecpg/test/compat_oracle/meson.build
M src/interfaces/ecpg/test/connect/meson.build
M src/interfaces/ecpg/test/meson.build
M src/interfaces/ecpg/test/pg_regress_ecpg.c
M src/interfaces/ecpg/test/pgtypeslib/meson.build
M src/interfaces/ecpg/test/preproc/meson.build
M src/interfaces/ecpg/test/sql/meson.build
M src/interfaces/ecpg/test/thread/meson.build
M src/interfaces/libpq/Makefile
M src/interfaces/libpq/fe-auth-sasl.h
M src/interfaces/libpq/fe-auth-scram.c
M src/interfaces/libpq/fe-auth.c
M src/interfaces/libpq/fe-auth.h
M src/interfaces/libpq/fe-cancel.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-gssapi-common.c
M src/interfaces/libpq/fe-gssapi-common.h
M src/interfaces/libpq/fe-lobj.c
M src/interfaces/libpq/fe-misc.c
M src/interfaces/libpq/fe-print.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/fe-secure-common.c
M src/interfaces/libpq/fe-secure-common.h
M src/interfaces/libpq/fe-secure-gssapi.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/fe-secure.c
M src/interfaces/libpq/fe-trace.c
M src/interfaces/libpq/legacy-pqsignal.c
M src/interfaces/libpq/libpq-events.c
M src/interfaces/libpq/libpq-events.h
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
M src/interfaces/libpq/meson.build
M src/interfaces/libpq/po/meson.build
M src/interfaces/libpq/pqexpbuffer.c
M src/interfaces/libpq/pqexpbuffer.h
M src/interfaces/libpq/pthread-win32.c
M src/interfaces/libpq/t/001_uri.pl
M src/interfaces/libpq/t/002_api.pl
M src/interfaces/libpq/t/003_load_balance_host_list.pl
M src/interfaces/libpq/t/004_load_balance_dns.pl
M src/interfaces/libpq/t/005_negotiate_encryption.pl
M src/interfaces/libpq/test/libpq_testclient.c
M src/interfaces/libpq/test/libpq_uri_regress.c
M src/interfaces/libpq/test/meson.build
M src/interfaces/libpq/win32.c
M src/interfaces/meson.build
M src/makefiles/meson.build
M src/meson.build
M src/pl/meson.build
M src/pl/plperl/meson.build
M src/pl/plperl/plc_perlboot.pl
M src/pl/plperl/plc_trusted.pl
M src/pl/plperl/plperl.h
M src/pl/plperl/plperl_opmask.pl
M src/pl/plperl/plperl_system.h
M src/pl/plperl/po/meson.build
M src/pl/plperl/text2macro.pl
M src/pl/plpgsql/meson.build
M src/pl/plpgsql/src/generate-plerrcodes.pl
M src/pl/plpgsql/src/meson.build
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_funcs.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/pl_handler.c
M src/pl/plpgsql/src/pl_reserved_kwlist.h
M src/pl/plpgsql/src/pl_scanner.c
M src/pl/plpgsql/src/pl_unreserved_kwlist.h
M src/pl/plpgsql/src/plpgsql.h
M src/pl/plpgsql/src/po/meson.build
M src/pl/plpython/generate-spiexceptions.pl
M src/pl/plpython/meson.build
M src/pl/plpython/plpython.h
M src/pl/plpython/plpython_system.h
M src/pl/plpython/po/meson.build
M src/pl/tcl/generate-pltclerrcodes.pl
M src/pl/tcl/meson.build
M src/pl/tcl/po/meson.build
M src/port/bsearch_arg.c
M src/port/chklocale.c
M src/port/dirent.c
M src/port/dirmod.c
M src/port/explicit_bzero.c
M src/port/getpeereid.c
M src/port/kill.c
M src/port/meson.build
M src/port/mkdtemp.c
M src/port/noblock.c
M src/port/open.c
M src/port/path.c
M src/port/pg_bitutils.c
M src/port/pg_crc32c_armv8.c
M src/port/pg_crc32c_armv8_choose.c
M src/port/pg_crc32c_loongarch.c
M src/port/pg_crc32c_sb8.c
M src/port/pg_crc32c_sse42.c
M src/port/pg_crc32c_sse42_choose.c
M src/port/pg_popcount_avx512.c
M src/port/pg_strong_random.c
M src/port/pgcheckdir.c
M src/port/pgsleep.c
M src/port/pgstrcasecmp.c
M src/port/pgstrsignal.c
M src/port/pqsignal.c
M src/port/pthread_barrier_wait.c
M src/port/quotes.c
M src/port/snprintf.c
M src/port/strerror.c
M src/port/strlcpy.c
M src/port/strnlen.c
M src/port/strtof.c
M src/port/system.c
M src/port/win32common.c
M src/port/win32dlopen.c
M src/port/win32env.c
M src/port/win32error.c
M src/port/win32fdatasync.c
M src/port/win32fseek.c
M src/port/win32gai_strerror.c
M src/port/win32getrusage.c
M src/port/win32link.c
M src/port/win32ntdll.c
M src/port/win32pread.c
M src/port/win32pwrite.c
M src/port/win32security.c
M src/port/win32setlocale.c
M src/port/win32stat.c
M src/port/win32ver.rc
M src/test/authentication/Makefile
M src/test/authentication/meson.build
M src/test/authentication/t/001_password.pl
M src/test/authentication/t/002_saslprep.pl
M src/test/authentication/t/003_peer.pl
M src/test/authentication/t/004_file_inclusion.pl
M src/test/authentication/t/005_sspi.pl
M src/test/authentication/t/006_login_trigger.pl
M src/test/examples/testlo.c
M src/test/examples/testlo64.c
M src/test/icu/Makefile
M src/test/icu/meson.build
M src/test/icu/t/010_database.pl
M src/test/isolation/isolation_main.c
M src/test/isolation/isolationtester.h
M src/test/isolation/meson.build
M src/test/isolation/specparse.y
M src/test/isolation/specscanner.l
M src/test/kerberos/Makefile
M src/test/kerberos/meson.build
M src/test/kerberos/t/001_auth.pl
M src/test/ldap/LdapServer.pm
M src/test/ldap/Makefile
M src/test/ldap/meson.build
M src/test/ldap/t/001_auth.pl
M src/test/ldap/t/002_bindpasswd.pl
M src/test/locale/sort-test.pl
M src/test/meson.build
M src/test/modules/brin/meson.build
M src/test/modules/brin/t/01_workitems.pl
M src/test/modules/brin/t/02_wal_consistency.pl
M src/test/modules/commit_ts/meson.build
M src/test/modules/commit_ts/t/001_base.pl
M src/test/modules/commit_ts/t/002_standby.pl
M src/test/modules/commit_ts/t/003_standby_2.pl
M src/test/modules/commit_ts/t/004_restart.pl
M src/test/modules/delay_execution/delay_execution.c
M src/test/modules/delay_execution/meson.build
M src/test/modules/dummy_index_am/dummy_index_am.c
M src/test/modules/dummy_index_am/meson.build
M src/test/modules/dummy_seclabel/dummy_seclabel.c
M src/test/modules/dummy_seclabel/meson.build
M src/test/modules/gin/meson.build
M src/test/modules/injection_points/injection_points.c
M src/test/modules/injection_points/injection_stats.c
M src/test/modules/injection_points/injection_stats.h
M src/test/modules/injection_points/injection_stats_fixed.c
M src/test/modules/injection_points/meson.build
M src/test/modules/injection_points/t/001_stats.pl
M src/test/modules/ldap_password_func/Makefile
M src/test/modules/ldap_password_func/ldap_password_func.c
M src/test/modules/ldap_password_func/t/001_mutated_bindpasswd.pl
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/libpq_pipeline/meson.build
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
M src/test/modules/meson.build
M src/test/modules/plsample/meson.build
M src/test/modules/plsample/plsample.c
M src/test/modules/spgist_name_ops/meson.build
M src/test/modules/spgist_name_ops/spgist_name_ops.c
M src/test/modules/ssl_passphrase_callback/meson.build
M src/test/modules/ssl_passphrase_callback/t/001_testfunc.pl
M src/test/modules/test_bloomfilter/meson.build
M src/test/modules/test_bloomfilter/test_bloomfilter.c
M src/test/modules/test_copy_callbacks/meson.build
M src/test/modules/test_copy_callbacks/test_copy_callbacks.c
M src/test/modules/test_custom_rmgrs/meson.build
M src/test/modules/test_custom_rmgrs/t/001_basic.pl
M src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c
M src/test/modules/test_ddl_deparse/meson.build
M src/test/modules/test_ddl_deparse/test_ddl_deparse.c
M src/test/modules/test_dsa/meson.build
M src/test/modules/test_dsa/test_dsa.c
M src/test/modules/test_dsm_registry/meson.build
M src/test/modules/test_dsm_registry/test_dsm_registry.c
M src/test/modules/test_extensions/meson.build
M src/test/modules/test_ginpostinglist/meson.build
M src/test/modules/test_ginpostinglist/test_ginpostinglist.c
M src/test/modules/test_integerset/meson.build
M src/test/modules/test_integerset/test_integerset.c
M src/test/modules/test_json_parser/meson.build
M src/test/modules/test_json_parser/t/001_test_json_parser_incremental.pl
M src/test/modules/test_json_parser/t/002_inline.pl
M src/test/modules/test_json_parser/t/003_test_semantic.pl
M src/test/modules/test_json_parser/t/004_test_parser_perf.pl
M src/test/modules/test_json_parser/test_json_parser_incremental.c
M src/test/modules/test_json_parser/test_json_parser_perf.c
M src/test/modules/test_lfind/meson.build
M src/test/modules/test_lfind/test_lfind.c
M src/test/modules/test_misc/meson.build
M src/test/modules/test_misc/t/001_constraint_validation.pl
M src/test/modules/test_misc/t/002_tablespace.pl
M src/test/modules/test_misc/t/003_check_guc.pl
M src/test/modules/test_misc/t/004_io_direct.pl
M src/test/modules/test_misc/t/005_timeouts.pl
M src/test/modules/test_misc/t/006_signal_autovacuum.pl
M src/test/modules/test_oat_hooks/meson.build
M src/test/modules/test_oat_hooks/test_oat_hooks.c
M src/test/modules/test_parser/meson.build
M src/test/modules/test_parser/test_parser.c
M src/test/modules/test_pg_dump/meson.build
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/modules/test_predtest/meson.build
M src/test/modules/test_predtest/test_predtest.c
M src/test/modules/test_radixtree/meson.build
M src/test/modules/test_radixtree/test_radixtree.c
M src/test/modules/test_rbtree/meson.build
M src/test/modules/test_rbtree/test_rbtree.c
M src/test/modules/test_regex/meson.build
M src/test/modules/test_regex/test_regex.c
M src/test/modules/test_resowner/meson.build
M src/test/modules/test_resowner/test_resowner_basic.c
M src/test/modules/test_resowner/test_resowner_many.c
M src/test/modules/test_rls_hooks/meson.build
M src/test/modules/test_rls_hooks/test_rls_hooks.c
M src/test/modules/test_rls_hooks/test_rls_hooks.h
M src/test/modules/test_shm_mq/meson.build
M src/test/modules/test_shm_mq/setup.c
M src/test/modules/test_shm_mq/test.c
M src/test/modules/test_shm_mq/test_shm_mq.h
M src/test/modules/test_shm_mq/worker.c
M src/test/modules/test_slru/meson.build
M src/test/modules/test_slru/t/001_multixact.pl
M src/test/modules/test_slru/test_multixact.c
M src/test/modules/test_slru/test_slru.c
M src/test/modules/test_tidstore/meson.build
M src/test/modules/test_tidstore/test_tidstore.c
M src/test/modules/typcache/meson.build
M src/test/modules/unsafe_tests/meson.build
M src/test/modules/worker_spi/meson.build
M src/test/modules/worker_spi/t/001_worker_spi.pl
M src/test/modules/worker_spi/worker_spi.c
M src/test/modules/xid_wraparound/meson.build
M src/test/modules/xid_wraparound/t/001_emergency_vacuum.pl
M src/test/modules/xid_wraparound/t/002_limits.pl
M src/test/modules/xid_wraparound/t/003_wraparounds.pl
M src/test/modules/xid_wraparound/xid_wraparound.c
M src/test/perl/Makefile
M src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
M src/test/perl/PostgreSQL/Test/BackgroundPsql.pm
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/test/perl/PostgreSQL/Test/Kerberos.pm
M src/test/perl/PostgreSQL/Test/RecursiveCopy.pm
M src/test/perl/PostgreSQL/Test/SimpleTee.pm
M src/test/perl/PostgreSQL/Test/Utils.pm
M src/test/perl/PostgreSQL/Version.pm
M src/test/perl/meson.build
M src/test/postmaster/Makefile
M src/test/postmaster/meson.build
M src/test/postmaster/t/001_connection_limits.pl
M src/test/postmaster/t/002_start_stop.pl
M src/test/recovery/Makefile
M src/test/recovery/meson.build
M src/test/recovery/t/001_stream_rep.pl
M src/test/recovery/t/002_archiving.pl
M src/test/recovery/t/003_recovery_targets.pl
M src/test/recovery/t/004_timeline_switch.pl
M src/test/recovery/t/005_replay_delay.pl
M src/test/recovery/t/006_logical_decoding.pl
M src/test/recovery/t/007_sync_rep.pl
M src/test/recovery/t/008_fsm_truncation.pl
M src/test/recovery/t/009_twophase.pl
M src/test/recovery/t/010_logical_decoding_timelines.pl
M src/test/recovery/t/012_subtransactions.pl
M src/test/recovery/t/013_crash_restart.pl
M src/test/recovery/t/014_unlogged_reinit.pl
M src/test/recovery/t/015_promotion_pages.pl
M src/test/recovery/t/016_min_consistency.pl
M src/test/recovery/t/017_shm.pl
M src/test/recovery/t/018_wal_optimize.pl
M src/test/recovery/t/019_replslot_limit.pl
M src/test/recovery/t/020_archive_status.pl
M src/test/recovery/t/021_row_visibility.pl
M src/test/recovery/t/022_crash_temp_files.pl
M src/test/recovery/t/023_pitr_prepared_xact.pl
M src/test/recovery/t/024_archive_recovery.pl
M src/test/recovery/t/025_stuck_on_old_timeline.pl
M src/test/recovery/t/026_overwrite_contrecord.pl
M src/test/recovery/t/027_stream_regress.pl
M src/test/recovery/t/028_pitr_timelines.pl
M src/test/recovery/t/029_stats_restart.pl
M src/test/recovery/t/030_stats_cleanup_replica.pl
M src/test/recovery/t/031_recovery_conflict.pl
M src/test/recovery/t/032_relfilenode_reuse.pl
M src/test/recovery/t/033_replay_tsp_drops.pl
M src/test/recovery/t/034_create_database.pl
M src/test/recovery/t/035_standby_logical_decoding.pl
M src/test/recovery/t/036_truncated_dropped.pl
M src/test/recovery/t/037_invalid_database.pl
M src/test/recovery/t/038_save_logical_slots_shutdown.pl
M src/test/recovery/t/039_end_of_wal.pl
M src/test/recovery/t/040_standby_failover_slots_sync.pl
M src/test/recovery/t/041_checkpoint_at_promote.pl
M src/test/recovery/t/042_low_level_backup.pl
M src/test/regress/GNUmakefile
M src/test/regress/meson.build
M src/test/regress/pg_regress.c
M src/test/regress/pg_regress.h
M src/test/regress/pg_regress_main.c
M src/test/regress/regress.c
M src/test/ssl/Makefile
M src/test/ssl/meson.build
M src/test/ssl/sslfiles.mk
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
M src/test/ssl/t/003_sslinfo.pl
M src/test/ssl/t/SSL/Backend/OpenSSL.pm
M src/test/ssl/t/SSL/Server.pm
M src/test/subscription/Makefile
M src/test/subscription/meson.build
M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/002_types.pl
M src/test/subscription/t/003_constraints.pl
M src/test/subscription/t/004_sync.pl
M src/test/subscription/t/005_encoding.pl
M src/test/subscription/t/006_rewrite.pl
M src/test/subscription/t/007_ddl.pl
M src/test/subscription/t/008_diff_schema.pl
M src/test/subscription/t/009_matviews.pl
M src/test/subscription/t/010_truncate.pl
M src/test/subscription/t/011_generated.pl
M src/test/subscription/t/012_collation.pl
M src/test/subscription/t/013_partition.pl
M src/test/subscription/t/014_binary.pl
M src/test/subscription/t/015_stream.pl
M src/test/subscription/t/016_stream_subxact.pl
M src/test/subscription/t/017_stream_ddl.pl
M src/test/subscription/t/018_stream_subxact_abort.pl
M src/test/subscription/t/019_stream_subxact_ddl_abort.pl
M src/test/subscription/t/020_messages.pl
M src/test/subscription/t/021_twophase.pl
M src/test/subscription/t/022_twophase_cascade.pl
M src/test/subscription/t/023_twophase_stream.pl
M src/test/subscription/t/024_add_drop_pub.pl
M src/test/subscription/t/025_rep_changes_for_schema.pl
M src/test/subscription/t/026_stats.pl
M src/test/subscription/t/027_nosuperuser.pl
M src/test/subscription/t/028_row_filter.pl
M src/test/subscription/t/029_on_error.pl
M src/test/subscription/t/030_origin.pl
M src/test/subscription/t/031_column_list.pl
M src/test/subscription/t/032_subscribe_use_index.pl
M src/test/subscription/t/033_run_as_table_owner.pl
M src/test/subscription/t/034_temporal.pl
M src/test/subscription/t/100_bugs.pl
M src/timezone/meson.build
M src/timezone/pgtz.c
M src/timezone/pgtz.h
M src/timezone/tznames/meson.build
M src/tools/PerfectHash.pm
M src/tools/add_commit_links.pl
M src/tools/check_bison_recursion.pl
M src/tools/copyright.pl
M src/tools/fix-old-flex-code.pl
M src/tools/gen_export.pl
M src/tools/gen_keywordlist.pl
M src/tools/git_changelog
M src/tools/ifaddrs/Makefile
M src/tools/mark_pgdllimport.pl
M src/tools/msvc_gendef.pl
M src/tools/pg_bsd_indent/Makefile
M src/tools/pg_bsd_indent/meson.build
M src/tools/pg_bsd_indent/t/001_pg_bsd_indent.pl
M src/tools/pginclude/headerscheck
M src/tools/pgindent/pgindent
M src/tools/testint128.c
M src/tools/version_stamp.pl
M src/tools/win32tzlist.pl
M src/tutorial/complex.source
M src/tutorial/syscat.source
Update obsolete reference to plpgsql's gram.y file.
commit : 98b1efd6ef6a1612019ef9ed6e44b79c132352ce
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Dec 2024 14:33:45 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Dec 2024 14:33:45 -0500
This was evidently missed in 05346c131, which renamed that
file to pl_gram.y.
Japin Li
Discussion: https://postgr.es/m/ME0P300MB0445F7CA7456C2AC67D37A01B6092@ME0P300MB0445.AUSP300.PROD.OUTLOOK.COM
M src/pl/plpgsql/src/plpgsql.h
injection_points: Tweak variable-numbered stats to work with pending data
commit : b757abefc0419c9b742222e1dfe70c73aff67364
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 18:48:18 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 18:48:18 +0900
As coded, the module was not using pending entries to store its data
locally before doing a flush to the central dshash with a timed
pgstat_report_stat() call. Hence, the flush callback was defined, but
finished by being not used. As a template, this is more efficient than
the original logic of updating directly the shared memory entries as
this reduces the interactions that need to be done with the pgstats
hash table in shared memory.
injection_stats_flush_cb() was also missing a pgstat_unlock_entry(), so
add one, while on it.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Z3JbLhKFFm6kKfT8@ip-10-97-1-34.eu-west-3.compute.internal
M src/test/modules/injection_points/injection_stats.c
Fix memory leak in pgoutput with relation attribute map
commit : c9b3d4909bbfcaa8cc2ed434d12a562fb30fe422
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 13:33:09 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 13:33:09 +0900
pgoutput caches the attribute map of a relation, that is free()'d only
when validating a RelationSyncEntry. However, this code path is not
taken when calling any of the SQL functions able to do some logical
decoding, like pg_logical_slot_{get,peek}_changes(), leaking some memory
into CacheMemoryContext on repeated calls.
To address this, a relation's attribute map is allocated in
PGOutputData's cachectx, free()'d at the end of the execution of these
SQL functions when logical decoding ends. This is available down to 15.
v13 and v14 have a similar leak, which will be dealt with later.
Reported-by: Masahiko Sawada
Author: Vignesh C
Reviewed-by: Hou Zhijie
Discussion: https://postgr.es/m/CAD21AoDkAhQVSukOfH3_reuF-j4EU0-HxMqU3dU+bSTxsqT14Q@mail.gmail.com
Discussion: https://postgr.es/m/CALDaNm1hewNAsZ_e6FF52a=9drmkRJxtEPrzCB6-9mkJyeBBqA@mail.gmail.com
Backpatch-through: 15
M src/backend/replication/pgoutput/pgoutput.c
Remove redundant wording in pg_statistic.h
commit : ebf2ab40e5e1b0545257e8b27a1d48e9488c23f4
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 12:18:45 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 12:18:45 +0900
Author: Junwang Zhao
Discussion: https://postgr.es/m/CAEG8a3JbMCHna=N5ZSx6huLnTDfW34kw7Pf2n8+3M-9UrrwesA@mail.gmail.com
M src/include/catalog/pg_statistic.h
Fix failures with incorrect epoch handling for 2PC files at recovery
commit : 7e125b20eed65e61b07e2e9ef6901467e4f14272
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 09:58:02 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 09:58:02 +0900
At the beginning of recovery, an orphaned two-phase file in an epoch
different than the one defined in the checkpoint record could not be
removed based on the assumptions that AdjustToFullTransactionId() relies
on, assuming that all files would be either from the current epoch or
from the previous epoch.
If the checkpoint epoch was 0 while the 2PC file was orphaned and in the
future, AdjustToFullTransactionId() would underflow the epoch used to
build the 2PC file path. In non-assert builds, this would create a
WARNING message referring to a 2PC file with an epoch of "FFFFFFFF" (or
UINT32_MAX), as an effect of the underflow calculation, leaving the
orphaned file around.
Some tests are added with dummy 2PC files in the past and the future,
checking that these are properly removed.
Issue introduced by 5a1dfde8334b, that has switched two-phase state
files to use FullTransactionIds.
Reported-by: Vitaly Davydov
Author: Michael Paquier
Reviewed-by: Vitaly Davydov
Discussion: https://postgr.es/m/13b5b6-676c3080-4d-531db900@47931709
Backpatch-through: 17
M src/backend/access/transam/twophase.c
M src/test/recovery/t/009_twophase.pl
Fix handling of orphaned 2PC files in the future at recovery
commit : e3584258154fac40e16f757fadd3361688a166d2
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 08:06:07 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 08:06:07 +0900
Before 728bd991c3c4, that has improved the support for 2PC files during
recovery, the initial logic scanning files in pg_twophase was done so as
files in the future of the transaction ID horizon were checked first,
followed by a check if a transaction ID is aborted or committed which
could involve a pg_xact lookup. After this commit, these checks have
been done in reverse order.
Files detected as in the future do not have a state that can be checked
in pg_xact, hence this caused recovery to fail abruptly should an
orphaned 2PC file in the future of the transaction ID horizon exist in
pg_twophase at the beginning of recovery.
A test is added to check for this scenario, using an empty 2PC with a
transaction ID large enough to be in the future when running the test.
This test is added in 16 and older versions for now. 17 and newer
versions are impacted by a second bug caused by the addition of the
epoch in the 2PC file names. An equivalent test will be added in these
branches in a follow-up commit, once the second set of issues reported
are fixed.
Author: Vitaly Davydov, Michael Paquier
Discussion: https://postgr.es/m/11e597-676ab680-8d-374f23c0@145466129
Backpatch-through: 13
M src/backend/access/transam/twophase.c
contrib/pageinspect: Use SQL-standard function bodies.
commit : 68ff25eef125a5eaf49a52ec794a0bf55a917b27
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Dec 2024 14:58:05 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Dec 2024 14:58:05 -0500
In the same spirit as 969bbd0fa, 13e3796c9, 3f323eba8.
Tom Lane and Ronan Dunklau
Discussion: https://postgr.es/m/3316564.aeNJFYEL58@aivenlaptop
M contrib/pageinspect/Makefile
M contrib/pageinspect/meson.build
A contrib/pageinspect/pageinspect–1.12–1.13.sql
M contrib/pageinspect/pageinspect.control
contrib/xml2: Use SQL-standard function bodies.
commit : 667368fd26de5846d6f6b8ab889ea9489b9b22bc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Dec 2024 13:53:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Dec 2024 13:53:00 -0500
In the same spirit as 969bbd0fa, 13e3796c9, 3f323eba8.
Tom Lane and Ronan Dunklau
Discussion: https://postgr.es/m/3316564.aeNJFYEL58@aivenlaptop
M contrib/xml2/Makefile
M contrib/xml2/meson.build
A contrib/xml2/xml2–1.1–1.2.sql
M contrib/xml2/xml2.control
contrib/citext: Use SQL-standard function bodies.
commit : 97a5a16849eb4642e38d9447ab40708d3f302e17
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Dec 2024 13:37:35 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Dec 2024 13:37:35 -0500
In the same spirit as 969bbd0fa, 13e3796c9, 3f323eba8.
Tom Lane and Ronan Dunklau
Discussion: https://postgr.es/m/3316564.aeNJFYEL58@aivenlaptop
M contrib/citext/Makefile
A contrib/citext/citext–1.7–1.8.sql
M contrib/citext/citext.control
M contrib/citext/meson.build
Fix overly large values/nulls arrays
commit : eb53ff5517e50018539747d75a02d493abc2d232
author : David Rowley <drowley@postgresql.org>
date : Sun, 29 Dec 2024 23:57:43 +1300
committer: David Rowley <drowley@postgresql.org>
date : Sun, 29 Dec 2024 23:57:43 +1300
These arrays were sized with Natts_pg_trigger (19) when they should have
been sized with Natts_pg_event_trigger (7). We'd better fix this as
it's clearly a mistake and it could become problematic if
pg_event_trigger were to gain a dozen or so more columns in the future.
No backpatch as there's no actual bug and the column count on those
tables isn't going to change in released versions.
Author: Xin Zhang <zhanghien@qq.com>
Discussion: https://postgr.es/m/tencent_05AD0FB321A414EC3661204D2102AA6EF605@qq.com
M src/backend/commands/event_trigger.c
Replace PGPROC.isBackgroundWorker with isRegularBackend.
commit : 508a97ee4984597ea60c383f52adefd47178189f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Dec 2024 16:21:54 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Dec 2024 16:21:54 -0500
Commit 34486b609 effectively redefined isBackgroundWorker as meaning
"not a regular backend", whereas before it had the narrower
meaning of AmBackgroundWorkerProcess(). For clarity, rename the
field to isRegularBackend and invert its sense.
Discussion: https://postgr.es/m/1808397.1735156190@sss.pgh.pa.us
M src/backend/access/transam/twophase.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/lmgr/proc.c
M src/include/storage/proc.h
Exclude parallel workers from connection privilege/limit checks.
commit : 34486b6092e700089d22586df211c4e8b0412136
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Dec 2024 16:08:50 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Dec 2024 16:08:50 -0500
Cause parallel workers to not check datallowconn, rolcanlogin, and
ACL_CONNECT privileges. The leader already checked these things
(except for rolcanlogin which might have been checked for a different
role). Re-checking can accomplish little except to induce unexpected
failures in applications that might not even be aware that their query
has been parallelized. We already had the principle that parallel
workers rely on their leader to pass a valid set of authorization
information, so this change just extends that a bit further.
Also, modify the ReservedConnections, datconnlimit and rolconnlimit
logic so that these limits are only enforced against regular backends,
and only regular backends are counted while checking if the limits
were already reached. Previously, background processes that had an
assigned database or role were subject to these limits (with rather
random exclusions for autovac workers and walsenders), and the set of
existing processes that counted against each limit was quite haphazard
as well. The point of these limits, AFAICS, is to ensure the
availability of PGPROC slots for regular backends. Since all other
types of processes have their own separate pools of PGPROC slots, it
makes no sense either to enforce these limits against them or to count
them while enforcing the limit.
While edge-case failures of these sorts have been possible for a
long time, the problem got a good deal worse with commit 5a2fed911
(CVE-2024-10978), which caused parallel workers to make some of these
checks using the leader's current role where before we had used its
AuthenticatedUserId, thus allowing parallel queries to fail after
SET ROLE. The previous behavior was fairly accidental and I have
no desire to return to it.
This patch includes reverting 73c9f91a1, which was an emergency hack
to suppress these same checks in some cases. It wasn't complete,
as shown by a recent bug report from Laurenz Albe. We can also revert
fd4d93d26 and 492217301, which hacked around the same problems in one
regression test.
In passing, remove the special case for autovac workers in
CheckMyDatabase; it seems cleaner to have AutoVacWorkerMain pass
the INIT_PG_OVERRIDE_ALLOW_CONNS flag, now that that does what's
needed.
Like 5a2fed911, back-patch to supported branches (which sadly no
longer includes v12).
Discussion: https://postgr.es/m/1808397.1735156190@sss.pgh.pa.us
M src/backend/access/transam/parallel.c
M src/backend/access/transam/twophase.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/bgworker.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/init/miscinit.c
M src/backend/utils/init/postinit.c
M src/include/miscadmin.h
M src/include/storage/proc.h
M src/test/modules/worker_spi/worker_spi.c
Reserve a PGPROC slot and semaphore for the slotsync worker process.
commit : 2bdf1b2a2efc7bbbe2db3213153c14615485eadd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Dec 2024 12:30:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Dec 2024 12:30:42 -0500
The need for this was missed in commit 93db6cbda, with the result
being that if we launch a slotsync worker it would consume one of
the PGPROCs in the max_connections pool. That could lead to inability
to launch the worker, or to subsequent failures of connection requests
that should have succeeded according to the configured settings.
Rather than create some one-off infrastructure to support this,
let's group the slotsync worker with the existing autovac launcher
in a new category of "special worker" processes. These are kind of
like auxiliary processes, but they cannot use that infrastructure
because they need to be able to run transactions.
For the moment, make these processes share the PGPROC freelist
used for autovac workers (which previously supplied the autovac
launcher too). This is partly to avoid an ABI change in v17,
and partly because it seems silly to have a freelist with
at most two members. This might be worth revisiting if we grow
enough workers in this category.
Tom Lane and Hou Zhijie. Back-patch to v17.
Discussion: https://postgr.es/m/1808397.1735156190@sss.pgh.pa.us
M src/backend/storage/lmgr/proc.c
M src/backend/utils/init/postinit.c
M src/include/miscadmin.h
M src/include/storage/proc.h
In REASSIGN OWNED of a database, lock the tuple as mandated.
commit : ff90ee61450921e03fe3af4e0f75337a4bbd0584
author : Noah Misch <noah@leadboat.com>
date : Sat, 28 Dec 2024 07:16:22 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 28 Dec 2024 07:16:22 -0800
Commit aac2c9b4fde889d13f859c233c2523345e72d32b mandated such locking
and attempted to fulfill that mandate, but it missed REASSIGN OWNED.
Hence, it remained possible to lose VACUUM's inplace update of
datfrozenxid if a REASSIGN OWNED processed that database at the same
time. This didn't affect the other inplace-updated catalog, pg_class.
For pg_class, REASSIGN OWNED calls ATExecChangeOwner() instead of the
generic AlterObjectOwner_internal(), and ATExecChangeOwner() fulfills
the locking mandate.
Like in GRANT, implement this by following the locking protocol for any
catalog subject to the generic AlterObjectOwner_internal(). It would
suffice to do this for IsInplaceUpdateOid() catalogs only. Back-patch
to v13 (all supported versions).
Kirill Reshke. Reported by Alexander Kukushkin.
Discussion: https://postgr.es/m/CAFh8B=mpKjAy4Cuun-HP-f_vRzh2HSvYFG3rhVfYbfEBUhBAGg@mail.gmail.com
M src/backend/catalog/objectaddress.c
M src/backend/commands/alter.c
M src/include/catalog/objectaddress.h
M src/test/regress/expected/database.out
M src/test/regress/sql/database.sql
Speedup tuple deformation with additional function inlining
commit : 58a359e585d0281ecab4d34cab9869e7eb4e4ca3
author : David Rowley <drowley@postgresql.org>
date : Sat, 28 Dec 2024 12:20:42 +1300
committer: David Rowley <drowley@postgresql.org>
date : Sat, 28 Dec 2024 12:20:42 +1300
This adjusts slot_deform_heap_tuple() to add special-case loops to
eliminate much of the branching that was done within the body of the
main deform loop.
Previously, while looping over each attribute to deform,
slot_deform_heap_tuple() would always recheck if the given attribute was
NULL by looking at HeapTupleHasNulls() and if so, went on to check the
tuple's NULL bitmap. Since many tuples won't contain any NULLs, we can
just check HeapTupleHasNulls() once and when there are no NULLs, use a
more compact version of the deforming loop which contains no NULL checking
code at all.
The same is possible for the "slow" mode checking part of the loop. That
variable was checked several times for each attribute, once to determine
if the offset to the attribute value could be taken from the attcacheoff,
and again to check if the offset could be cached for next time.
These "slow" checks can mostly be eliminated by instead having multiple
loops. Initially, we can start in the non-slow loop and break out of
that loop if and only if we must stop caching the offset. This
eliminates branching for both slow and non-slow deforming methods. The
amount of code required for the no nulls / non-slow version is very
small. It's possible to have separate loops like this due to the fact
that once we move into slow mode, we never need to switch back into
non-slow mode for a given tuple.
We have the compiler take care of writing out the multiple required
loops by having a pg_attribute_always_inline function which gets called
various times passing in constant values for the "slow" and "hasnulls"
parameters. This allows the compiler to eliminate const-false branches
and remove comparisons for const-true ones.
This commit has shown overall query performance increases of around 5-20%
in deform-heavy OLAP-type workloads.
Author: David Rowley
Reviewed-by: Victor Yegorov
Discussion: https://postgr.es/m/CAGnEbog92Og2CpC2S8=g_HozGsWtt_3kRS1sXjLz0jKSoCNfLw@mail.gmail.com
Discussion: https://postgr.es/m/CAApHDvo9e0XG71WrefYaRv5n4xNPLK4k8LjD0mSR3c9KR2vi2Q@mail.gmail.com
M src/backend/executor/execTuples.c
Improve handling of date_trunc() units for infinite input values
commit : d85ce012f99f63249bb45a78fcecb7a2383920b1
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 27 Dec 2024 13:32:40 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 27 Dec 2024 13:32:40 +0900
Previously, if an infinite value was passed to date_trunc(), then the
same infinite value would always be returned regardless of the field
unit given by the caller. This commit updates the function so that an
error is returned when an invalid unit is passed to date_trunc() with an
infinite value.
This matches the behavior of date_trunc() with a finite value and
date_part() with an infinite value, making the handling of interval,
timestamp and timestamptz more consistent across the board for these two
functions.
Some tests are added to cover all these new failure cases, with an
unsupported unit and infinite values for the three data types. There
were no test cases in core that checked all these patterns up to now.
Author: Joseph Koshakow
Discussion: https://postgr.es/m/CAAvxfHc4084dGzEJR0_pBZkDuqbPGc5wn7gK_M0XR_kRiCdUJQ@mail.gmail.com
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/interval.out
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/interval.sql
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Remove unused totalrows parameter in compute_expr_stats
commit : 61cac71c23686f47c785dc69d3c15cb4304d106f
author : David Rowley <drowley@postgresql.org>
date : Fri, 27 Dec 2024 10:51:22 +1300
committer: David Rowley <drowley@postgresql.org>
date : Fri, 27 Dec 2024 10:51:22 +1300
The totalrows parameter in compute_expr_stats is unused, so remove it.
This is a static function, so the parameter can easily be added again if
it's ever needed.
Author: Ilia Evdokimov <ilya.evdokimov@tantorlabs.ru>
Discussion: https://postgr.es/m/667b92d2-f953-4fcb-9377-3765f5b94187@tantorlabs.com
M src/backend/statistics/extended_stats.c
plpgsql: Rename a variable for clarity
commit : 3f2d72b4934945da76f6bc60dfe3fc5ca42e7526
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 26 Dec 2024 11:11:14 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 26 Dec 2024 11:11:14 +0100
Rename "core_yy_extra_type core_yy" to "core_yy_extra". The previous
name was a bit unclear and confusing. The new name matches the name
used elsewhere for the same purpose, for example in
src/backend/parser/gramparse.h.
M src/pl/plpgsql/src/pl_scanner.c
Fix typo in comment of compute_return_type() in functioncmds.c
commit : a86cfcae7c58049c7a3a60118ef3f53afd92b3fa
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 26 Dec 2024 12:53:55 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 26 Dec 2024 12:53:55 +0900
Author: Japin Li
Discussion: https://postgr.es/m/ME0P300MB0445D51BCFA8680F0B35FD6EB60C2@ME0P300MB0445.AUSP300.PROD.OUTLOOK.COM
M src/backend/commands/functioncmds.c
meson: Export all libcommon functions in Windows builds
commit : 07f902bd760791527511be9fc47440039bc86b5c
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 25 Dec 2024 19:22:25 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 25 Dec 2024 19:22:25 +0200
This fixes "unresolved external symbol" errors with extensions that
use functions from libpgport that need special CFLAGS to
compile. Currently, that includes the CRC-32 functions.
Commit 2571c1d5cc did this for libcommon, but I missed that libpqport
has the same issue.
Reported-by: Tom Lane
Backpatch-through: 16, where Meson was introduced
Discussion: https://www.postgresql.org/message-id/CAOdR5yF0krWrxycA04rgUKCgKugRvGWzzGLAhDZ9bzNv8g0Lag@mail.gmail.com
M src/port/meson.build
Add commit 301de6a6f60 to .git-blame-ignore-revs.
commit : 4c9b453c91b201264c6d60a3de197115c8feaee0
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Dec 2024 18:17:29 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Dec 2024 18:17:29 +0100
M .git-blame-ignore-revs
Partial pgindent of .l and .y files
commit : 301de6a6f609cb3ad2d9d31fd8db9ae6c71e6dea
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Dec 2024 17:52:42 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Dec 2024 17:52:42 +0100
Trying to clean up the code a bit while we're working on these files
for the reentrant scanner/pure parser patches. This cleanup only
touches the code sections after the second '%%' in each file, via a
manually-supervised and locally hacked up pgindent.
M contrib/cube/cubeparse.y
M contrib/cube/cubescan.l
M contrib/seg/segscan.l
M src/backend/bootstrap/bootscanner.l
M src/backend/parser/gram.y
M src/backend/parser/scan.l
M src/backend/replication/repl_scanner.l
M src/backend/replication/syncrep_scanner.l
M src/backend/utils/adt/jsonpath_gram.y
M src/backend/utils/adt/jsonpath_scan.l
M src/backend/utils/misc/guc-file.l
M src/bin/pgbench/exprparse.y
M src/bin/psql/psqlscanslash.l
M src/fe_utils/psqlscan.l
M src/interfaces/ecpg/preproc/pgc.l
M src/pl/plpgsql/src/pl_gram.y
meson: Export all libcommon functions in Windows builds
commit : 2571c1d5cc5ed2620d601a7e12179bd951f2e202
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 25 Dec 2024 18:14:18 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 25 Dec 2024 18:14:18 +0200
This fixes "unresolved external symbol" errors with extensions that
use functions from libcommon. This was reported with pgvector.
Reported-by: Andrew Kane
Author: Vladlen Popolitov
Backpatch-through: 16, where Meson was introduced
Discussion: https://www.postgresql.org/message-id/CAOdR5yF0krWrxycA04rgUKCgKugRvGWzzGLAhDZ9bzNv8g0Lag@mail.gmail.com
M src/common/meson.build
guc: reentrant scanner
commit : d663f150b5edd5ec4a5a3bbc24f9ad7e40f1abbb
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Dec 2024 14:18:07 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Dec 2024 14:18:07 +0100
Use the flex %option reentrant to make the generated scanner
reentrant, and perhaps eventually thread-safe, but that will require
additional work.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Andreas Karlsson <andreas@proxel.se>
Discussion: https://www.postgresql.org/message-id/flat/eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org
M src/backend/utils/misc/guc-file.l
jsonpath scanner: reentrant scanner
commit : 2a7425d7eef9b4b848763fb920fefa23c3c43f31
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 24 Dec 2024 23:42:41 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 24 Dec 2024 23:42:41 +0100
Use the flex %option reentrant to make the generated scanner
reentrant and thread-safe. Note: The parser was already pure.
Simplify flex scan buffer management: Instead of constructing the
buffer from pieces and then using yy_scan_buffer(), we can just use
yy_scan_string(), which does the same thing internally. (Actually, we
use yy_scan_bytes() here because we already have the length.)
Use flex yyextra to handle context information, instead of global
variables. This complements the other changes to make the scanner
reentrant.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Andreas Karlsson <andreas@proxel.se>
Discussion: https://www.postgresql.org/message-id/flat/eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org
M src/backend/nls.mk
M src/backend/utils/adt/jsonpath_gram.y
M src/backend/utils/adt/jsonpath_internal.h
M src/backend/utils/adt/jsonpath_scan.l
Fix nbtree symbol name comment reference.
commit : 9b254895c367a3b962bcdd3e35f8590005106e8a
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 24 Dec 2024 14:06:16 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 24 Dec 2024 14:06:16 -0500
Oversight in commit 5bf748b86b.
M src/backend/access/nbtree/nbtsearch.c
syncrep parser: pure parser and reentrant scanner
commit : db6856c9913f1af08b5d7bde442fae362913190a
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 2 Dec 2024 10:35:37 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 2 Dec 2024 10:35:37 +0100
Use the flex %option reentrant and the bison option %pure-parser to
make the generated scanner and parser pure, reentrant, and
thread-safe.
Make the generated scanner use palloc() etc. instead of malloc() etc.
Previously, we only used palloc() for the buffer, but flex would still
use malloc() for its internal structures. Now, all the memory is
under palloc() control.
Simplify flex scan buffer management: Instead of constructing the
buffer from pieces and then using yy_scan_buffer(), we can just use
yy_scan_string(), which does the same thing internally.
The previous code was necessary because we allocated the buffer with
palloc() and the rest of the state was handled by malloc(). But this
is no longer the case; everything is under palloc() now.
Use flex yyextra to handle context information, instead of global
variables. This complements the other changes to make the scanner
reentrant.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Andreas Karlsson <andreas@proxel.se>
Discussion: https://www.postgresql.org/message-id/flat/eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org
M src/backend/nls.mk
M src/backend/replication/syncrep.c
M src/backend/replication/syncrep_gram.y
M src/backend/replication/syncrep_scanner.l
M src/include/replication/syncrep.h
replication parser: pure parser and reentrant scanner
commit : e4a8fb8fefb903fe601f7415098a4fe39a14069a
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 2 Dec 2024 10:35:37 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 2 Dec 2024 10:35:37 +0100
Use the flex %option reentrant and the bison option %pure-parser to
make the generated scanner and parser pure, reentrant, and
thread-safe.
Make the generated scanner use palloc() etc. instead of malloc() etc.
Previously, we only used palloc() for the buffer, but flex would still
use malloc() for its internal structures. As a result, there could be
some small memory leaks in case of uncaught errors. Now, all the
memory is under palloc() control, so there are no more such issues.
Simplify flex scan buffer management: Instead of constructing the
buffer from pieces and then using yy_scan_buffer(), we can just use
yy_scan_string(), which does the same thing internally.
The previous code was necessary because we allocated the buffer with
palloc() and the rest of the state was handled by malloc(). But this
is no longer the case; everything is under palloc() now.
Use flex yyextra to handle context information, instead of global
variables. This complements the other changes to make the scanner
reentrant.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Co-authored-by: Andreas Karlsson <andreas@proxel.se>
Reviewed-by: Andreas Karlsson <andreas@proxel.se>
Discussion: https://www.postgresql.org/message-id/flat/eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org
M src/backend/nls.mk
M src/backend/replication/repl_gram.y
M src/backend/replication/repl_scanner.l
M src/backend/replication/walsender.c
M src/include/replication/walsender_private.h
Remove pgrminclude and associated scripts
commit : 5af699066f81547aa75c093544fbd57289d30284
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 24 Dec 2024 14:02:42 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 24 Dec 2024 14:02:42 +0100
Per git log, the last time someone tried to do something with
pgrminclude was around 2011. And it's always had a tendency of
causing trouble when it was active. Also, pgcominclude is redundant
with headerscheck.
Discussion: https://www.postgresql.org/message-id/flat/2d4dc7b2-cb2e-49b1-b8ca-ba5f7024f05b%40eisentraut.org
M src/tools/pginclude/README
D src/tools/pginclude/pgcheckdefines
D src/tools/pginclude/pgcompinclude
D src/tools/pginclude/pgdefine
D src/tools/pginclude/pgfixinclude
D src/tools/pginclude/pgrminclude
Remove pgrminclude annotations
commit : 1eb7cb21c2a2c9091bc4cdf09d37d68559af8a7d
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 24 Dec 2024 11:48:08 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 24 Dec 2024 11:48:08 +0100
Per git log, the last time someone tried to do something with
pgrminclude was around 2011. Many (not all) of the "pgrminclude
ignore" annotations are of a newer date but seem to have just been
copied around during refactorings and file moves and don't seem to
reflect an actual need anymore.
There have been some parallel experiments with include-what-you-use
(IWYU) annotations, but these don't seem to correspond very strongly
to pgrminclude annotations, so there is no value in keeping the
existing ones even for that kind of thing.
So, wipe them all away. We can always add new ones in the future
based on actual needs.
Discussion: https://www.postgresql.org/message-id/flat/2d4dc7b2-cb2e-49b1-b8ca-ba5f7024f05b%40eisentraut.org
M src/backend/access/brin/brin.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/regex/regerror.c
M src/backend/utils/adt/inet_net_pton.c
M src/bin/initdb/initdb.c
M src/bin/pg_amcheck/pg_amcheck.c
M src/bin/scripts/common.h
M src/include/common/ip.h
M src/include/common/relpath.h
M src/include/libpq/hba.h
M src/include/libpq/ifaddr.h
M src/include/pg_trace.h
M src/include/snowball/header.h
M src/pl/plpgsql/src/pl_comp.c
M src/pl/tcl/pltcl.c
Fix race condition in TupleDescCompactAttr assert code
commit : 6f3820f37aba94232468365bae7ba5de697fe993
author : David Rowley <drowley@postgresql.org>
date : Tue, 24 Dec 2024 14:54:24 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 24 Dec 2024 14:54:24 +1300
5983a4cff added CompactAttribute as an abbreviated alternative to
FormData_pg_attribute to allow more cache-friendly processing in tasks
related to TupleDescs. That commit contained some assert-only code to
check that the CompactAttribute had been populated correctly, however,
the method used to do that checking caused the TupleDesc's
CompactAttribute to be zeroed before it was repopulated and compared to
the snapshot taken before the memset call. This caused issues as the type
cache caches TupleDescs in shared memory which can be used by multiple
backend processes at the same time. There was a window of time between
the zero and repopulation of the CompactAttribute where another process
would mistakenly think that the CompactAttribute is invalid due to the
memset.
To fix this, instead of taking a snapshot of the CompactAttribute and
calling populate_compact_attribute() and comparing the snapshot to the
freshly populated TupleDesc's CompactAttribute, refactor things so we
can just populate a temporary CompactAttribute on the stack. This way
we don't touch the TupleDesc's memory.
Reported-by: Alexander Lakhin, SQLsmith
Discussion: https://postgr.es/m/ca3a256a-5d12-42db-aabe-a75a030d9fb9@gmail.com
M src/backend/access/common/tupdesc.c
M src/include/access/tupdesc.h
Try to avoid semaphore-related test failures on NetBSD/OpenBSD.
commit : 38da053463bef32adf563ddee5277d16d2b6c5af
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Dec 2024 16:46:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Dec 2024 16:46:07 -0500
These two platforms have a remarkably tight default limit on the
number of SysV semaphores in the system: SEMMNS is only 60
out-of-the-box. Unless manual action is taken to raise that,
we'll only be able to allocate 3 sets of 16 usable semaphores
each, leading to initdb setting max_connections to just 20.
That's problematic because the core regression tests expect
to be able to launch 20 concurrent sessions, leaving us with
no headroom. This seems to be the cause of intermittent
buildfarm failures on some machines.
While there's no getting around the fact that you'd better raise
SEMMNS for production use on these platforms, it does seem desirable
for "make check" to pass reliably without that. We can make that
happen, at least for awhile longer, with two small changes:
* Change sysv_sema.c's SEMAS_PER_SET to 19, so that we can eat up
all of the available semas not just most of them.
* Change initdb to make the smallest max_connections value it will
consider be 25 not 20.
As of HEAD this will leave us with four free semaphores (using the
default values for other relevant parameters such as max_wal_senders).
So we won't need to consider this again until we've invented five
more background processes. Maybe by then we can switch both these
platforms to some other semaphore API.
For the moment, do this only in master; there've not been field
complaints that might justify a back-patch.
Discussion: https://postgr.es/m/db2773a2-aca0-43d0-99c1-060efcd9954e@gmail.com
M doc/src/sgml/runtime.sgml
M src/backend/port/sysv_sema.c
M src/bin/initdb/initdb.c
Reset btpo_cycleid in nbtree VACUUM's REDO routine.
commit : da9517fb3a0979f8d75823199f7a231e0269963b
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 23 Dec 2024 15:46:00 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 23 Dec 2024 15:46:00 -0500
Reset btpo_cycleid to 0 in btree_xlog_vacuum for consistency with
_bt_delitems_vacuum (the corresponding original execution code). This
makes things neater.
There might be some performance benefit to being consistent like this.
When btvacuumpage doesn't call _bt_delitems_vacuum, it can still
proactively reset btpo_cycleid to 0 via a separate hint-like update
mechanism (it does so whenever it sees that it isn't already set to 0).
And so it's possible that being consistent about resetting btpo_cycleid
like this will save work later on, after standby promotion: subsequent
VACUUMs won't need to clear btpo_cycleid using the hint-like update
mechanism as often as they otherwise would.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/CAH2-Wz=+LDFxn9NZyEsCo8ifcyKt6+n-VLyygySEHgMz+oynqw@mail.gmail.com
M src/backend/access/nbtree/nbtxlog.c
postgres_fdw: re-issue cancel requests a few times if necessary.
commit : c431986de16cc7e7af8ba3af45d846f279606fd3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Dec 2024 15:14:30 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Dec 2024 15:14:30 -0500
Despite the best efforts of commit 0e5c82380, we're still seeing
occasional failures of postgres_fdw's query_cancel test in the
buildfarm. Investigation suggests that its 100ms timeout is
still not enough to reliably ensure that the remote side starts
the query before receiving the cancel request --- and if it
hasn't, it will just discard the request because it's idle.
We discussed allowing a cancel request to kill the next-received
query, but that would have wide and perhaps unpleasant side-effects.
What seems safer is to make postgres_fdw do what a human user would
likely do, which is issue another cancel request if the first one
didn't seem to do anything. We'll keep the same overall 30 second
grace period before concluding things are broken, but issue additional
cancel requests after 1 second, then 2 more seconds, then 4, then 8.
(The next one in series is 16 seconds, but we'll hit the 30 second
timeout before that.)
Having done that, revert the timeout in query_cancel.sql to 10 ms.
That will still be enough on most machines, most of the time, for
the remote query to start; but now we're intentionally risking the
race condition occurring sometimes in the buildfarm, so that the
repeat-cancel code path will get some testing.
As before, back-patch to v17. We might eventually contemplate
back-patching this further, and/or adding similar logic to dblink.
But given the lack of field complaints to date, this feels like
mostly an exercise in test case stabilization, so v17 is enough.
Discussion: https://postgr.es/m/colnv3lzzmc53iu5qoawynr6qq7etn47lmggqr65ddtpjliq5d@glkveb4m6nop
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/query_cancel.out
M contrib/postgres_fdw/sql/query_cancel.sql
Don't allow GetTransactionSnapshot() in logical decoding
commit : 1585ff7387dbcc5657522b2ed87ffd58fd555ee9
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 23 Dec 2024 12:42:55 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 23 Dec 2024 12:42:55 +0200
A historic snapshot should only be used for catalog access, not
general queries. We never call GetTransactionSnapshot() during logical
decoding, which is good because it wouldn't be very sensible, so the
code to deal with that was unreachable and untested. Turn it into an
error, to avoid doing that in the future either.
Discussion: https://www.postgresql.org/message-id/a868fe78-ddb4-4b0a-9b96-873d91d93cfd@iki.fi
M src/backend/utils/time/snapmgr.c
Remove unnecessary GetTransactionSnapshot() calls
commit : 952365cded635e54c4177399c0280cb7a5e34c11
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 23 Dec 2024 12:42:39 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 23 Dec 2024 12:42:39 +0200
In get_database_list() and get_subscription_list(), the
GetTransactionSnapshot() call is not required because the catalog
table scans use the catalog snapshot, which is held until the end of
the scan. See table_beginscan_catalog(), which calls
RegisterSnapshot(GetCatalogSnapshot(relid)).
In InitPostgres, it's a little less obvious that it's not required,
but still true I believe. All the catalog lookups in InitPostgres()
also use the catalog snapshot, and the looked up values are copied
while still holding the snapshot.
Furthermore, as the removed FIXME comments said, calling
GetTransactionSnapshot() didn't really prevent MyProc->xmin from being
reset anyway.
Discussion: https://www.postgresql.org/message-id/7c56f180-b9e1-481e-8c1d-efa63de3ecbb@iki.fi
M src/backend/postmaster/autovacuum.c
M src/backend/replication/logical/launcher.c
M src/backend/utils/init/postinit.c
Fix incorrect source filename references
commit : 7ec4b9ff80d92fee3c41eb1a069cc32d0ec11da1
author : David Rowley <drowley@postgresql.org>
date : Mon, 23 Dec 2024 19:41:49 +1300
committer: David Rowley <drowley@postgresql.org>
date : Mon, 23 Dec 2024 19:41:49 +1300
Jian He reported the src/include/utility/tcop.h one and the remainder
were found by using a script to look for src/* and check that we have a
filename or directory of that name.
In passing, fix some out-date comments.
Reported-by: Jian He <jian.universality@gmail.com>
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/CACJufxGoE3H-7VgO02=PrR4SNuVWDVbfTyUnwO0HvS-Lxurnog@mail.gmail.com
M src/backend/commands/aggregatecmds.c
M src/backend/commands/define.c
M src/backend/commands/functioncmds.c
M src/backend/commands/operatorcmds.c
M src/backend/commands/typecmds.c
M src/backend/tcop/utility.c
M src/backend/utils/misc/guc_internal.h
M src/fe_utils/astreamer_file.c
M src/fe_utils/astreamer_gzip.c
M src/fe_utils/astreamer_lz4.c
M src/fe_utils/astreamer_tar.c
M src/fe_utils/astreamer_zstd.c
M src/include/fe_utils/astreamer.h
Fix some comments related to library unloading
commit : 7f97b4734f937db6f8dab1bbf8bbaab349e6c9b1
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 23 Dec 2024 14:46:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 23 Dec 2024 14:46:49 +0900
Library unloading has never been supported with its code removed in
ab02d702ef08, and there were some comments still mentioning that it was
a possible operation.
ChangAo has noticed the incorrect references in dfmgr.c, while I have
noticed the other ones while scanning the rest of the tree for similar
mistakes.
Author: ChangAo Chen, Michael Paquier
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/tencent_1D09840A1632D406A610C8C4E2491D74DB0A@qq.com
M contrib/auto_explain/auto_explain.c
M contrib/passwordcheck/passwordcheck.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/sepgsql/hooks.c
M src/backend/utils/fmgr/dfmgr.c
Update TransactionXmin when MyProc->xmin is updated
commit : 578a7fe7b6f8484f6d7caa2fda288abb3fe87aa0
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sat, 21 Dec 2024 23:42:39 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sat, 21 Dec 2024 23:42:39 +0200
GetSnapshotData() set TransactionXmin = MyProc->xmin, but when
SnapshotResetXmin() advanced MyProc->xmin, it did not advance
TransactionXmin correspondingly. That meant that TransactionXmin could
be older than MyProc->xmin, and XIDs between than TransactionXmin and
the real MyProc->xmin could be vacuumed away. One known consequence is
in pg_subtrans lookups: we might try to look up the status of an XID
that was already truncated away.
Back-patch to all supported versions.
Reviewed-by: Andres Freund
Discussion: https://www.postgresql.org/message-id/d27a046d-a1e4-47d1-a95c-fbabe41debb4@iki.fi
M src/backend/utils/time/snapmgr.c
Optimize alignment calculations in tuple form/deform
commit : db448ce5ad36a2754e4e75900b180260143aacf8
author : David Rowley <drowley@postgresql.org>
date : Sat, 21 Dec 2024 09:43:26 +1300
committer: David Rowley <drowley@postgresql.org>
date : Sat, 21 Dec 2024 09:43:26 +1300
Here we convert CompactAttribute.attalign from a char, which is directly
derived from pg_attribute.attalign into a uint8, which stores the number
of bytes to align the column's value by in the tuple.
This allows tuple deformation and tuple size calculations to move away
from using the inefficient att_align_nominal() macro, which manually
checks each TYPALIGN_* char to translate that into the alignment bytes
for the given type. Effectively, this commit changes those to TYPEALIGN
calls, which are branchless and only perform some simple arithmetic with
some bit-twiddling.
The removed branches were often mispredicted by CPUs, especially so in
real-world tables which often contain a mishmash of different types
with different alignment requirements.
Author: David Rowley
Reviewed-by: Andres Freund, Victor Yegorov
Discussion: https://postgr.es/m/CAApHDvrBztXP3yx=NKNmo3xwFAFhEdyPnvrDg3=M0RhDs+4vYw@mail.gmail.com
M contrib/amcheck/verify_heapam.c
M contrib/pageinspect/heapfuncs.c
M src/backend/access/brin/brin_tuple.c
M src/backend/access/common/attmap.c
M src/backend/access/common/heaptuple.c
M src/backend/access/common/indextuple.c
M src/backend/access/common/tupdesc.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/execTuples.c
M src/backend/jit/llvm/llvmjit_deform.c
M src/include/access/tupdesc.h
M src/include/access/tupmacs.h
Mark CatalogSnapshotData static
commit : 1f81b48a9d567ae9074ab1f3233eae9997b3d7bd
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 19 Dec 2024 18:02:11 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 19 Dec 2024 18:02:11 +0200
Like CurrentSnapshotData, it should not be accessed directly outside
snapmgr.c.
M src/backend/utils/time/snapmgr.c
M src/include/utils/snapmgr.h
Fix variable reference in comment
commit : d5a7bd5670c4a59bea506f5999101a8e40ef256c
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 20 Dec 2024 19:36:33 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 20 Dec 2024 19:36:33 +0200
This used to say "nsubxcnt isn't decreased when subtransactions
abort", but there's no variable called nsubxcnt. Commit 8548ddc61b
changed it to "subxcnt", among other typo fixes, but that was wrong
too: the comment actually talks about txn->nsubtxns. That's the field
that's incremented but never decremented and is used for the
allocation earlier in the function.
M src/backend/replication/logical/reorderbuffer.c
Fix overflow danger in SampleHeapTupleVisible(), take 2
commit : 94bb6c4410d81ccc019bb60d8aedb73dbc85df76
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 20 Dec 2024 09:41:41 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 20 Dec 2024 09:41:41 -0500
28328ec87b45725 addressed one overflow danger in
SampleHeapTupleVisible() but introduced another, albeit a less likely
one. Modify the binary search code to remove this danger.
Reported-by: Richard Guo
Reviewed-by: Richard Guo, Ranier Vilela
Discussion: https://postgr.es/m/CAMbWs4_bE%2BNscChbKWzw6HZOipCUyXfA5133qvoXQ654D3B2gQ%40mail.gmail.com
M src/backend/access/heap/heapam_handler.c
Fix corruption when relation truncation fails.
commit : 38c579b08988e6f1a5bd74241d0a1001421d8015
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 20 Dec 2024 21:53:25 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 20 Dec 2024 21:53:25 +1300
RelationTruncate() does three things, while holding an
AccessExclusiveLock and preventing checkpoints:
1. Logs the truncation.
2. Drops buffers, even if they're dirty.
3. Truncates some number of files.
Step 2 could previously be canceled if it had to wait for I/O, and step
3 could and still can fail in file APIs. All orderings of these
operations have data corruption hazards if interrupted, so we can't give
up until the whole operation is done. When dirty pages were discarded
but the corresponding blocks were left on disk due to ERROR, old page
versions could come back from disk, reviving deleted data (see
pgsql-bugs #18146 and several like it). When primary and standby were
allowed to disagree on relation size, standbys could panic (see
pgsql-bugs #18426) or revive data unknown to visibility management on
the primary (theorized).
Changes:
* WAL is now unconditionally flushed first
* smgrtruncate() is now called in a critical section, preventing
interrupts and causing PANIC on file API failure
* smgrtruncate() has a new parameter for existing fork sizes,
because it can't call smgrnblocks() itself inside a critical section
The changes apply to RelationTruncate(), smgr_redo() and
pg_truncate_visibility_map(). That last is also brought up to date with
other evolutions of the truncation protocol.
The VACUUM FileTruncate() failure mode had been discussed in older
reports than the ones referenced below, with independent analysis from
many people, but earlier theories on how to fix it were too complicated
to back-patch. The more recently invented cancellation bug was
diagnosed by Alexander Lakhin. Other corruption scenarios were spotted
by me while iterating on this patch and earlier commit 75818b3a.
Back-patch to all supported releases.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reported-by: rootcause000@gmail.com
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/18146-04e908c662113ad5%40postgresql.org
Discussion: https://postgr.es/m/18426-2d18da6586f152d6%40postgresql.org
M contrib/pg_visibility/pg_visibility.c
M src/backend/catalog/storage.c
M src/backend/storage/smgr/md.c
M src/backend/storage/smgr/smgr.c
M src/include/storage/md.h
M src/include/storage/smgr.h
Remove pg_attribute.attcacheoff column
commit : 02a8d0c45253eb54e57b1974c8627e5be3e1d852
author : David Rowley <drowley@postgresql.org>
date : Fri, 20 Dec 2024 23:22:37 +1300
committer: David Rowley <drowley@postgresql.org>
date : Fri, 20 Dec 2024 23:22:37 +1300
The column is no longer needed as the offset is now cached in the
CompactAttribute struct per commit 5983a4cff.
Author: David Rowley
Reviewed-by: Andres Freund, Victor Yegorov
Discussion: https://postgr.es/m/CAApHDvrBztXP3yx=NKNmo3xwFAFhEdyPnvrDg3=M0RhDs+4vYw@mail.gmail.com
M doc/src/sgml/catalogs.sgml
M src/backend/access/common/tupdesc.c
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/utils/cache/relcache.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_attribute.h
M src/test/regress/expected/type_sanity.out
M src/test/regress/sql/type_sanity.sql
Relax regression test for fsync check of backend-level stats
commit : 546371599e767340599cdff3e276728016e560cc
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 20 Dec 2024 19:00:18 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 20 Dec 2024 19:00:18 +0900
One test added in 9aea73fc61d4 did not take into account that the
backend may have some fsync even after a checkpoint. Let's relax it to
be more flexible.
Per report from buildfarm member grassquit, via Alexander Lakhin.
Author: Bertrand Drouvot
Discussion: https://postgr.es/m/6143ab0a-9e88-4790-8d9d-50ba45657761@gmail.com
M src/test/regress/expected/stats.out
M src/test/regress/sql/stats.sql
Introduce CompactAttribute array in TupleDesc, take 2
commit : 5983a4cffc31640fda6643f10146a5b72b203eaa
author : David Rowley <drowley@postgresql.org>
date : Fri, 20 Dec 2024 22:31:26 +1300
committer: David Rowley <drowley@postgresql.org>
date : Fri, 20 Dec 2024 22:31:26 +1300
The new compact_attrs array stores a few select fields from
FormData_pg_attribute in a more compact way, using only 16 bytes per
column instead of the 104 bytes that FormData_pg_attribute uses. Using
CompactAttribute allows performance-critical operations such as tuple
deformation to be performed without looking at the FormData_pg_attribute
element in TupleDesc which means fewer cacheline accesses.
For some workloads, tuple deformation can be the most CPU intensive part
of processing the query. Some testing with 16 columns on a table
where the first column is variable length showed around a 10% increase in
transactions per second for an OLAP type query performing aggregation on
the 16th column. However, in certain cases, the increases were much
higher, up to ~25% on one AMD Zen4 machine.
This also makes pg_attribute.attcacheoff redundant. A follow-on commit
will remove it, thus shrinking the FormData_pg_attribute struct by 4
bytes.
Author: David Rowley
Reviewed-by: Andres Freund, Victor Yegorov
Discussion: https://postgr.es/m/CAApHDvrBztXP3yx=NKNmo3xwFAFhEdyPnvrDg3=M0RhDs+4vYw@mail.gmail.com
M contrib/amcheck/verify_heapam.c
M contrib/pageinspect/gistfuncs.c
M contrib/pageinspect/heapfuncs.c
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/access/brin/brin_inclusion.c
M src/backend/access/brin/brin_tuple.c
M src/backend/access/common/attmap.c
M src/backend/access/common/heaptuple.c
M src/backend/access/common/indextuple.c
M src/backend/access/common/tupdesc.c
M src/backend/access/gin/ginbulk.c
M src/backend/access/gin/ginget.c
M src/backend/access/gist/gist.c
M src/backend/access/gist/gistbuild.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/heap/heaptoast.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/access/spgist/spgdoinsert.c
M src/backend/access/spgist/spgutils.c
M src/backend/access/table/toast_helper.c
M src/backend/catalog/index.c
M src/backend/commands/copy.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/execJunk.c
M src/backend/executor/execTuples.c
M src/backend/executor/functions.c
M src/backend/executor/nodeMemoize.c
M src/backend/executor/nodeModifyTable.c
M src/backend/executor/nodeValuesscan.c
M src/backend/executor/tstoreReceiver.c
M src/backend/jit/llvm/llvmjit_deform.c
M src/backend/optimizer/util/plancat.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/utils/adt/expandedrecord.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/cache/relcache.c
M src/include/access/htup_details.h
M src/include/access/itup.h
M src/include/access/tupdesc.h
M src/include/access/tupmacs.h
M src/tools/pgindent/typedefs.list
Remove final mention of FREEZE_PAGE from comments
commit : 8ac0021b6f10928a46b7f3d1b25bc21c0ac7f8c5
author : Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 19 Dec 2024 18:52:19 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 19 Dec 2024 18:52:19 -0500
b7493e1ab35 removed leftover mentions of XLOG_HEAP2_FREEZE_PAGE records
from comments but neglected to remove one mention of FREEZE_PAGE.
Reported off-list by Alexander Lakhin
M src/backend/access/transam/clog.c
Get rid of old version of BuildTupleHashTable().
commit : e0a2721f7c169b50617fed797d7336cd8f10bf77
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Dec 2024 18:07:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Dec 2024 18:07:00 -0500
It was reasonable to preserve the old API of BuildTupleHashTable()
in the back branches, but in HEAD we should actively discourage use
of that version. There are no remaining callers in core, so just
get rid of it. Then rename BuildTupleHashTableExt() back to
BuildTupleHashTable().
While at it, fix up the miserably-poorly-maintained header comment
for BuildTupleHashTable[Ext]. It looks like more than one patch in
this area has had the opinion that updating comments is beneath them.
Discussion: https://postgr.es/m/538343.1734646986@sss.pgh.pa.us
M src/backend/executor/execGrouping.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeRecursiveunion.c
M src/backend/executor/nodeSetOp.c
M src/backend/executor/nodeSubplan.c
M src/include/executor/executor.h
Use ExecGetCommonSlotOps infrastructure in more places.
commit : f0b900086a370e45f730138d55da4f260d24809c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Dec 2024 17:07:14 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Dec 2024 17:07:14 -0500
Append, MergeAppend, and RecursiveUnion can all use the support
functions added in commit 276279295. The first two can report a
fixed result slot type if all their children return the same fixed
slot type. That does nothing for the append step itself, but might
allow optimizations in the parent plan node. RecursiveUnion can
optimize tuple hash table operations in the same way as SetOp now
does.
Patch by me; thanks to Richard Guo and David Rowley for review.
Discussion: https://postgr.es/m/1850138.1731549611@sss.pgh.pa.us
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeMergeAppend.c
M src/backend/executor/nodeRecursiveunion.c
Improve planner's handling of SetOp plans.
commit : 8d96f57d5cc79c0c51050bb707c19bf07d2895eb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Dec 2024 17:02:25 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Dec 2024 17:02:25 -0500
Remove the code for inserting flag columns in the inputs of a SetOp.
That was the only reason why there would be resjunk columns in a
set-operations plan tree, so we can get rid of some code that
supported that, too.
Get rid of choose_hashed_setop() in favor of building Paths for
the hashed and sorted alternatives, and letting them fight it out
within add_path().
Remove set_operation_ordered_results_useful(), which was giving wrong
answers due to examining the wrong ancestor node: we need to examine
the immediate SetOperationStmt parent not the topmost node. Instead
make each caller of recurse_set_operations() pass down the relevant
parent node. (This thinko seems to have led only to wasted planning
cycles and possibly-inferior plans, not wrong query answers. Perhaps
we should back-patch it, but I'm not doing so right now.)
Teach generate_nonunion_paths() to consider pre-sorted inputs for
sorted SetOps, rather than always generating a Sort node.
Patch by me; thanks to Richard Guo and David Rowley for review.
Discussion: https://postgr.es/m/1850138.1731549611@sss.pgh.pa.us
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/pathnode.c
M src/include/optimizer/prep.h
M src/test/regress/expected/subselect.out
M src/test/regress/expected/union.out
M src/test/regress/sql/subselect.sql
M src/test/regress/sql/union.sql
Convert SetOp to read its inputs as outerPlan and innerPlan.
commit : 27627929528e24a547d1058a5444b35491057a56
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Dec 2024 16:23:45 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Dec 2024 16:23:45 -0500
The original design for set operations involved appending the two
input relations into one and adding a flag column that allows
distinguishing which side each row came from. Then the SetOp node
pries them apart again based on the flag. This is bizarre. The
only apparent reason to do it is that when sorting, we'd only need
one Sort node not two. But since sorting is at least O(N log N),
sorting all the data is actually worse than sorting each side
separately --- plus, we have no chance of taking advantage of
presorted input. On top of that, adding the flag column frequently
requires an additional projection step that adds cycles, and then
the Append node isn't free either. Let's get rid of all of that
and make the SetOp node have two separate children, using the
existing outerPlan/innerPlan infrastructure.
This initial patch re-implements nodeSetop.c and does a bare minimum
of work on the planner side to generate correctly-shaped plans.
In particular, I've tried not to change the cost estimates here,
so that the visible changes in the regression test results will only
involve removal of useless projection steps and not any changes in
whether to use sorted vs hashed mode.
For SORTED mode, we combine successive identical tuples from each
input into groups, and then merge-join the groups. The tuple
comparisons now use SortSupport instead of simple equality, but
the group-formation part should involve roughly the same number of
tuple comparisons as before. The cross-comparisons between left and
right groups probably add to that, but I'm not sure to quantify how
many more comparisons we might need.
For HASHED mode, nodeSetop's logic is almost the same as before,
just refactored into two separate loops instead of one loop that
has an assumption that it will see all the left-hand inputs first.
In both modes, I added early-exit logic to not bother reading the
right-hand relation if the left-hand input is empty, since neither
INTERSECT nor EXCEPT modes can produce any output if the left input
is empty. This could have been done before in the hashed mode, but
not in sorted mode. Sorted mode can also stop as soon as it exhausts
the left input; any remaining right-hand tuples cannot have matches.
Also, this patch adds some infrastructure for detecting whether
child plan nodes all output the same type of tuple table slot.
If they do, the hash table logic can use slightly more efficient
code based on assuming that that's the input slot type it will see.
We'll make use of that infrastructure in other plan node types later.
Patch by me; thanks to Richard Guo and David Rowley for review.
Discussion: https://postgr.es/m/1850138.1731549611@sss.pgh.pa.us
M src/backend/executor/execUtils.c
M src/backend/executor/nodeSetOp.c
M src/backend/optimizer/README
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/pathnode.c
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/pathnode.h
M src/test/regress/expected/subselect.out
M src/test/regress/expected/union.out
M src/test/regress/sql/union.sql
M src/tools/pgindent/typedefs.list
Remove extra prefetch iterator setup for Bitmap Table Scan
commit : 2128cebcdb2f32303baf200fa2ccb2947366c636
author : Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 19 Dec 2024 11:55:18 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 19 Dec 2024 11:55:18 -0500
1a0da347a7ac98db replaced Bitmap Table Scan's separate private and
shared bitmap iterators with a unified iterator. It accidentally set up
the prefetch iterator twice for non-parallel bitmap table scans. Remove
the extra set up call to tbm_begin_iterate().
M src/backend/executor/nodeBitmapHeapscan.c
Fix bitmap table scan crash on iterator release
commit : 754c610e13b820370db4c02010a4c8c5dbd1edbd
author : Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 19 Dec 2024 11:55:03 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Thu, 19 Dec 2024 11:55:03 -0500
1a0da347a7ac98db replaced Bitmap Table Scan's individual private and
shared iterators with a unified iterator. It neglected, however, to
check if the iterator had already been cleaned up before doing so on
rescan. Add this check both on rescan and end scan to be safe.
Reported-by: Richard Guo
Author: Richard Guo
Discussion: https://postgr.es/m/CAMbWs48nrhcLY1kcd-u9oD%2B6yiS631F_8Fx8ZGsO-BYDwH%2Bbyw%40mail.gmail.com
M src/backend/executor/nodeBitmapHeapscan.c
M src/backend/nodes/tidbitmap.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Avoid nbtree index scan SAOP scanBehind confusion.
commit : 31b0a8f040042c1dfb9ac359fffbb6b8f9375999
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 19 Dec 2024 11:08:55 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 19 Dec 2024 11:08:55 -0500
Consistently reset so->scanBehind at the beginning of nbtree array
advancement, even during sktrig_required=false calls (calls where array
advancement is triggered by an unsatisfied non-required array scan key).
Otherwise, it's possible for queries to fail to return all relevant
tuples to the scan given a low-order required scan key that was
previously deemed "satisfied" by a truncated high key attribute value.
This only happened at the point where a later non-required array scan
key needed to be "advanced" once on the next leaf page (that is, once
the right sibling of the truncated high key page was reached).
The underlying issue was that later code within _bt_advance_array_keys
assumed that the so->scanBehind flag must have been set using the
current page's high key (not the previous page's high key). Any later
successful recheck call to _bt_check_compare would therefore spuriously
be prevented from making _bt_advance_array_keys return true, based on
the faulty belief that the truncated attribute must be from the scan's
current tuple (i.e. the non-pivot tuple at the start of the next page).
_bt_advance_array_keys would return false for the tuple, ultimately
resulting in _bt_checkkeys failing to return a matching tuple.
Oversight in commit 5bf748b8, which enhanced nbtree ScalarArrayOp
execution.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WzkJKncfqyAUTeuB5GgRhT1vhsWO2q11dbZNqKmvjopP_g@mail.gmail.com
Backpatch: 17-, where commit 5bf748b8 first appears.
M src/backend/access/nbtree/nbtutils.c
bootstrap: pure parser and reentrant scanner
commit : 3e4bacb171001644583ac14e29ae1b09ce818c92
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 19 Dec 2024 15:37:44 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 19 Dec 2024 15:37:44 +0100
Use the flex %option reentrant and the bison option %pure-parser to
make the generated scanner and parser pure, reentrant, and
thread-safe.
Make the generated scanner use palloc() etc. instead of malloc() etc.
For the bootstrap scanner and parser, reentrancy and memory management
aren't that important, but we make this change here anyway so that all
the scanners and parsers in the backend use a similar set of options
and APIs.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Andreas Karlsson <andreas@proxel.se>
Discussion: https://www.postgresql.org/message-id/flat/eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org
M src/backend/bootstrap/bootparse.y
M src/backend/bootstrap/bootscanner.l
M src/backend/bootstrap/bootstrap.c
M src/include/bootstrap/bootstrap.h
Small whitespace improvement
commit : 399d0f1e11b5438c6dc82e55a22a0f402855b2ac
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 19 Dec 2024 13:00:31 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 19 Dec 2024 13:00:31 +0100
Author: Andreas Karlsson <andreas@proxel.se>
Discussion: https://www.postgresql.org/message-id/flat/eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org
M src/backend/replication/repl_scanner.l
Prevent redeclaration of typedef yyscan_t
commit : 382092a0cd2c75613f3df889f7b821577e6773d7
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 19 Dec 2024 11:21:06 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 19 Dec 2024 11:21:06 +0100
Fix for 1f0de66ea2a: We need to prevent redeclaration of typedef
yyscan_t. (This will work with C11 but not currently with C99.) The
generated scanner files provide their own typedef, but we also need to
provide one for the interfaces that we expose. So we need to add some
preprocessor guards to avoid a redefinition. (This is how the
generated scanner files do it internally as well.) This way
everything now works independent of the order in which things are
included.
Discussion: https://www.postgresql.org/message-id/flat/eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org
M contrib/cube/cubedata.h
M contrib/seg/segdata.h
Add backend-level statistics to pgstats
commit : 9aea73fc61d4e77e000724ce0b2f896590a10e03
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 19 Dec 2024 13:19:22 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 19 Dec 2024 13:19:22 +0900
This adds a new variable-numbered statistics kind in pgstats, where the
object ID key of the stats entries is based on the proc number of the
backends. This acts as an upper-bound for the number of stats entries
that can exist at once. The entries are created when a backend starts
after authentication succeeds, and are removed when the backend exits,
making the stats entry exist for as long as their backend is up and
running. These are not written to the pgstats file at shutdown (note
that write_to_file is disabled, as a safety measure).
Currently, these stats include only information about the I/O generated
by a backend, using the same layer as pg_stat_io, except that it is now
possible to know how much activity is happening in each backend rather
than an overall aggregate of all the activity. A function called
pg_stat_get_backend_io() is added to access this data depending on the
PID of a backend. The existing structure could be expanded in the
future to add more information about other statistics related to
backends, depending on requirements or ideas.
Auxiliary processes are not included in this set of statistics. These
are less interesting to have than normal backends as they have dedicated
entries in pg_stat_io, and stats kinds of their own.
This commit includes also pg_stat_reset_backend_stats(), function able
to reset all the stats associated to a single backend.
Bump catalog version and PGSTAT_FILE_FORMAT_ID.
Author: Bertrand Drouvot
Reviewed-by: Álvaro Herrera, Kyotaro Horiguchi, Michael Paquier, Nazir
Bilal Yavuz
Discussion: https://postgr.es/m/ZtXR+CtkEVVE/LHF@ip-10-97-1-34.eu-west-3.compute.internal
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_functions.sql
M src/backend/utils/activity/Makefile
M src/backend/utils/activity/backend_status.c
M src/backend/utils/activity/meson.build
M src/backend/utils/activity/pgstat.c
A src/backend/utils/activity/pgstat_backend.c
M src/backend/utils/activity/pgstat_io.c
M src/backend/utils/activity/pgstat_relation.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/utils/pgstat_internal.h
M src/test/regress/expected/stats.out
M src/test/regress/sql/stats.sql
M src/tools/pgindent/typedefs.list
Extract logic filling pg_stat_get_io()'s tuplestore into its own routine
commit : ff7c40d7fd6a218f31fcf6f2c23c544c85934b24
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 19 Dec 2024 10:16:02 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 19 Dec 2024 10:16:02 +0900
This commit adds pg_stat_io_build_tuples(), a helper routine for
pg_stat_get_io(), that fills its result tuplestore based on the contents
of PgStat_BktypeIO. This will be used in a follow-up commit that uses
the same structures as pg_stat_io for reporting, including the same
object types and contexts, but for a different statistics kind.
Author: Bertrand Drouvot, Michael Paquier
Discussion: https://postgr.es/m/ZtXR+CtkEVVE/LHF@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/utils/adt/pgstatfuncs.c
Optimize grouping equality checks with virtual slots
commit : 08cdb079d4a8a82c687321e9ffe0a3d3fbcc0551
author : David Rowley <drowley@postgresql.org>
date : Thu, 19 Dec 2024 13:57:21 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 19 Dec 2024 13:57:21 +1300
8f4ee9626 fixed an old Assert failure that could happen when the slot
type used to look up the hash table for BuildTupleHashTableExt() users
wasn't a TTSOpsMinimalTuple slot. The fix for that in the back branches
had to be to pass the TupleTableSlotOps as NULL, however in master,
since we have the inputOps parameter as was added by d96d1d515, we can
pass that down instead.
At least one caller uses a fixed slot that's always TTSOpsVirtual, so
passing down inputOps for these cases allows ExecBuildGroupingEqual() to
skip adding the EEOP_INNER_FETCHSOME ExprEvalStep.
This should increase the performance of hashed subplans very slightly.
Author: Tom Lane, David Rowley
Discussion: https://postgr.es/m/2543667.1734483723@sss.pgh.pa.us
M src/backend/executor/execGrouping.c
Fix Assert failure in WITH RECURSIVE UNION queries
commit : 8f4ee962691ed36302be3333c65422ed9e082743
author : David Rowley <drowley@postgresql.org>
date : Thu, 19 Dec 2024 13:11:39 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 19 Dec 2024 13:11:39 +1300
If the non-recursive part of a recursive CTE ended up using
TTSOpsBufferHeapTuple as the table slot type, then a duplicate value
could cause an Assert failure in CheckOpSlotCompatibility() when
checking the hash table for the duplicate value. The expected slot type
for the deform step was TTSOpsMinimalTuple so the Assert failed when the
TTSOpsBufferHeapTuple slot was used.
This is a long-standing bug which we likely didn't notice because it
seems much more likely that the non-recursive term would have required
projection and used a TTSOpsVirtual slot, which CheckOpSlotCompatibility
is ok with.
There doesn't seem to be any harm done here other than the Assert
failure. Both TTSOpsMinimalTuple and TTSOpsBufferHeapTuple slot types
require tuple deformation, so the EEOP_*_FETCHSOME ExprState step would
have properly existed in the ExprState.
The solution is to pass NULL for the ExecBuildGroupingEqual's 'lops'
parameter. This means the ExprState's EEOP_*_FETCHSOME step won't
expect a fixed slot type. This makes CheckOpSlotCompatibility() happy as
no checking is performed when the ExprEvalStep is not expecting a fixed
slot type.
Reported-by: Richard Guo
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/CAMbWs4-8U9q2LAtf8+ghV11zeUReA3AmrYkxzBEv0vKnDxwkKA@mail.gmail.com
Backpatch-through: 13, all supported versions
M src/backend/executor/execGrouping.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Remove leftover mentions of XLOG_HEAP2_FREEZE_PAGE records
commit : b7493e1ab353349855f553a4b4bee094cda4d9cc
author : Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 18 Dec 2024 18:47:21 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 18 Dec 2024 18:47:21 -0500
f83d709760d merged the separate XLOG_HEAP2_FREEZE_PAGE records into a
new combined prune, freeze, and vacuum record with opcode
XLOG_HEAP2_PRUNE_VACUUM_SCAN. Remove the last few references to
XLOG_HEAP2_FREEZE_PAGE records which were accidentally left behind.
Reported-by: Tomas Vondra
Reviewed-by: Robert Haas
Discussion: https://postgr.es/m/CA%2BTgmoY1tYff-1CEn8kYt5FsOrynTbtr%3DUZw%3D7mTC1Hv1HpeBQ%40mail.gmail.com
M src/backend/access/heap/pruneheap.c
Bitmap Table Scans use unified TBMIterator
commit : 1a0da347a7ac98db6964feb5e3063fc6e8fc92a0
author : Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 18 Dec 2024 18:43:39 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 18 Dec 2024 18:43:39 -0500
With the repurposing of TBMIterator as an interface for both parallel
and serial iteration through TIDBitmaps in commit 7f9d4187e7bab10329cc,
bitmap table scans may now use it.
Modify bitmap table scan code to use the TBMIterator. This requires
moving around a bit of code, so a few variables are initialized
elsewhere.
Author: Melanie Plageman
Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/c736f6aa-8b35-4e20-9621-62c7c82e2168%40vondra.me
M src/backend/access/heap/heapam_handler.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/include/access/relscan.h
M src/include/access/tableam.h
M src/include/nodes/execnodes.h
M src/include/nodes/tidbitmap.h
Add common interface for TBMIterators
commit : 7f9d4187e7bab10329cc00aff34cfba08248d4be
author : Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 18 Dec 2024 18:19:28 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 18 Dec 2024 18:19:28 -0500
Add and use TBMPrivateIterator, which replaces the current TBMIterator
for serial use cases, and repurpose TBMIterator to be a unified
interface for both the serial ("private") and parallel ("shared") TID
Bitmap iterator interfaces. This encapsulation simplifies call sites for
callers supporting both parallel and serial TID Bitmap access.
TBMIterator is not yet used in this commit.
Author: Melanie Plageman
Reviewed-by: Tomas Vondra, Heikki Linnakangas
Discussion: https://postgr.es/m/063e4eb4-32d9-439e-a0b1-75565a9835a8%40iki.fi
M src/backend/access/gin/ginget.c
M src/backend/access/gin/ginscan.c
M src/backend/access/heap/heapam_handler.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/backend/nodes/tidbitmap.c
M src/include/access/gin_private.h
M src/include/access/relscan.h
M src/include/nodes/execnodes.h
M src/include/nodes/tidbitmap.h
M src/tools/pgindent/typedefs.list
Fix overflow danger in SampleHeapTupleVisible()
commit : 28328ec87b45725f62bed1104d99c8b3220d1675
author : Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 18 Dec 2024 18:16:43 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 18 Dec 2024 18:16:43 -0500
68d9662be1c4b70 made HeapScanDesc->rs_ntuples unsigned but neglected to
change how it was being used in SampleHeapTupleVisible().
Return early if rs_ntuples is 0 to avoid overflowing and incorrectly
executing the loop code in SampleHeapTupleVisible().
Reported-by: Ranier Vilela
Discussion: https://postgr.es/m/CAEudQAot_xQoZyPZjpj1aBUPrPykY5mOPHGyvfe%3Djz%2BWowdA3A%40mail.gmail.com
M src/backend/access/heap/heapam_handler.c
Make rs_cindex and rs_ntuples unsigned
commit : 68d9662be1c4b705123a0e292974fb4be661294c
author : Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 18 Dec 2024 11:47:38 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Wed, 18 Dec 2024 11:47:38 -0500
HeapScanDescData.rs_cindex and rs_ntuples can't be less than 0. All scan
types using the heap scan descriptor expect these values to be >= 0.
Make that expectation clear by making rs_cindex and rs_ntuples unsigned.
Also remove the test in heapam_scan_bitmap_next_tuple() that checks if
rs_cindex < 0. This was never true, but now that rs_cindex is unsigned,
it makes even less sense.
While we are at it, initialize both rs_cindex and rs_ntuples to 0 in
initscan().
Author: Melanie Plageman
Reviewed-by: Dilip Kumar
Discussion: https://postgr.es/m/CAAKRu_ZxF8cDCM_BFi_L-t%3DRjdCZYP1usd1Gd45mjHfZxm0nZw%40mail.gmail.com
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/include/access/heapam.h
seg: pure parser and reentrant scanner
commit : 1f0de66ea2a5549a3768c67434e28a136c280571
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 18 Dec 2024 08:47:53 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 18 Dec 2024 08:47:53 +0100
Use the flex %option reentrant and the bison option %pure-parser to
make the generated scanner and parser pure, reentrant, and
thread-safe.
Make the generated scanner use palloc() etc. instead of malloc() etc.
Previously, we only used palloc() for the buffer, but flex would still
use malloc() for its internal structures. As a result, there could be
some small memory leaks in case of uncaught errors. (We do catch
normal syntax errors as soft errors.) Now, all the memory is under
palloc() control, so there are no more such issues.
Simplify flex scan buffer management: Instead of constructing the
buffer from pieces and then using yy_scan_buffer(), we can just use
yy_scan_string(), which does the same thing internally.
The previous code was necessary because we allocated the buffer with
palloc() and the rest of the state was handled by malloc(). But this
is no longer the case; everything is under palloc() now.
(We could even get rid of the yylex_destroy() call and just let the
memory context cleanup handle everything. But for now, we preserve
the existing behavior.)
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Andreas Karlsson <andreas@proxel.se>
Discussion: https://www.postgresql.org/message-id/flat/eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org
M contrib/seg/seg.c
M contrib/seg/segdata.h
M contrib/seg/segparse.y
M contrib/seg/segscan.l
cube: pure parser and reentrant scanner
commit : 802fe923e3cd4b2f51080ed1a9a0958024b00479
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 18 Dec 2024 08:47:34 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 18 Dec 2024 08:47:34 +0100
Use the flex %option reentrant and the bison option %pure-parser to
make the generated scanner and parser pure, reentrant, and
thread-safe.
Make the generated scanner use palloc() etc. instead of malloc() etc.
Previously, we only used palloc() for the buffer, but flex would still
use malloc() for its internal structures. As a result, there could be
some small memory leaks in case of uncaught errors. (We do catch
normal syntax errors as soft errors.) Now, all the memory is under
palloc() control, so there are no more such issues.
Simplify flex scan buffer management: Instead of constructing the
buffer from pieces and then using yy_scan_buffer(), we can just use
yy_scan_string(), which does the same thing internally. (Actually, we
use yy_scan_bytes() here because we already have the length.)
The previous code was necessary because we allocated the buffer with
palloc() and the rest of the state was handled by malloc(). But this
is no longer the case; everything is under palloc() now.
(We could even get rid of the yylex_destroy() call and just let the
memory context cleanup handle everything. But for now, we preserve
the existing behavior.)
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Andreas Karlsson <andreas@proxel.se>
Discussion: https://www.postgresql.org/message-id/flat/eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org
M contrib/cube/cube.c
M contrib/cube/cubedata.h
M contrib/cube/cubeparse.y
M contrib/cube/cubescan.l
psql: Add more information about service name
commit : 477728b5d6fa16461b81cd22b0568fec1eab97ac
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 18 Dec 2024 15:16:12 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 18 Dec 2024 15:16:12 +0900
This commit adds support for the following items in psql, able to show a
service name, when available:
- Variable SERVICE.
- Substitution %s in PROMPT{1,2,3}.
This relies on 4b99fed7541e, that has made the service name available in
PGconn for libpq.
Author: Michael Banck
Reviewed-by: Greg Sabino Mullane
Discussion: https://postgr.es/m/6723c612.050a0220.1567f4.b94a@mx.google.com
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/prompt.c
libpq: Add service name to PGconn and PQservice()
commit : 4b99fed7541e330b669fe488a274c0c69490391c
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 18 Dec 2024 14:53:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 18 Dec 2024 14:53:42 +0900
This commit adds one field to PGconn for the database service name (if
any), with PQservice() as routine to retrieve it. Like the other
routines of this area, NULL is returned as result if the connection is
NULL.
A follow-up patch will make use of this feature to be able to display
the service name in the psql prompt.
Author: Michael Banck
Reviewed-by: Greg Sabino Mullane
Discusion: https://postgr.es/m/6723c612.050a0220.1567f4.b94a@mx.google.com
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/exports.txt
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
Fix memory leak in pg_restore with zstd-compressed data.
commit : 3f06324705aeb5a4c67b6c08f2016c4c3c756723
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Dec 2024 22:31:26 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Dec 2024 22:31:26 -0500
EndCompressorZstd() neglected to free everything. This was
most visible with a lot of large objects in the dump.
Per report from Tomasz Szypowski. Back-patch to v16
where this code came in.
Discussion: https://postgr.es/m/DU0PR04MB94193D038A128EF989F922D199042@DU0PR04MB9419.eurprd04.prod.outlook.com
M src/bin/pg_dump/compress_zstd.c
Fix incorrect slot type in BuildTupleHashTableExt
commit : d96d1d5152f30d15678e08e75b42756101b7cab6
author : David Rowley <drowley@postgresql.org>
date : Wed, 18 Dec 2024 12:05:55 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 18 Dec 2024 12:05:55 +1300
0f5738202 adjusted the execGrouping.c code so it made use of ExprStates to
generate hash values. That commit made a wrong assumption that the slot
type to pass to ExecBuildHash32FromAttrs() is always &TTSOpsMinimalTuple.
That's not the case as the slot type depends on the slot type passed to
LookupTupleHashEntry(), which for nodeRecursiveunion.c, could be any of
the current slot types.
Here we fix this by adding a new parameter to BuildTupleHashTableExt()
to allow the slot type to be passed in. In the case of nodeSubplan.c
and nodeAgg.c the slot type is always &TTSOpsVirtual, so for both of
those cases, it's beneficial to pass the known slot type as that allows
ExecBuildHash32FromAttrs() to skip adding the tuple deform step to the
resulting ExprState. Another possible fix would have been to have
ExecBuildHash32FromAttrs() set "fetch.kind" to NULL so that
ExecComputeSlotInfo() always determines the EEOP_INNER_FETCHSOME is
required, however, that option isn't favorable as slows down aggregation
and hashed subplan evaluation due to the extra (needless) deform step.
Thanks to Nathan Bossart for bisecting to find the offending commit
based on Paul's report.
Reported-by: Paul Ramsey <pramsey@cleverelephant.ca>
Discussion: https://postgr.es/m/99F064C1-B3EB-4BE7-97D2-D2A0AA487A71@cleverelephant.ca
M src/backend/executor/execGrouping.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeRecursiveunion.c
M src/backend/executor/nodeSetOp.c
M src/backend/executor/nodeSubplan.c
M src/include/executor/executor.h
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Accommodate very large dshash tables.
commit : 84f1b0b031e6914c41623102b93fed8ab0e51253
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 17 Dec 2024 15:24:45 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 17 Dec 2024 15:24:45 -0600
If a dshash table grows very large (e.g., the dshash table for
cumulative statistics when there are millions of tables), resizing
it may fail with an error like:
ERROR: invalid DSA memory alloc request size 1073741824
To fix, permit dshash resizing to allocate more than 1 GB by
providing the DSA_ALLOC_HUGE flag.
Reported-by: Andreas Scherbaum
Author: Matthias van de Meent
Reviewed-by: Cédric Villemain, Michael Paquier, Andres Freund
Discussion: https://postgr.es/m/80a12d59-0d5e-4c54-866c-e69cd6536471%40pgug.de
Backpatch-through: 13
M src/backend/lib/dshash.c
Skip useless calculation of join RTE column names during EXPLAIN.
commit : 7a80e381d16c642d00ec6146ccdf1262a159c69e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Dec 2024 15:52:05 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Dec 2024 15:52:05 -0500
There's no need for set_simple_column_names() to compute unique
column names for join RTEs, because a finished plan tree will
not contain any join alias Vars that we could need names for.
Its other, internal callers will not pass it any join RTEs
anyway, so the upshot is we can just skip join RTEs here.
Aside from getting rid of a klugy against-its-documentation use of
set_relation_column_names, this can speed up EXPLAIN substantially
when considering many-join queries, because the upper join RTEs
tend to have a lot of columns.
Sami Imseih, with cosmetic changes by me
Discussion: https://postgr.es/m/CAA5RZ0th3q-0p1pri58z9grG8r8azmEBa8o1rtkwhLmJg_cH+g@mail.gmail.com
M src/backend/utils/adt/ruleutils.c
Count pages set all-visible and all-frozen in VM during vacuum
commit : dc6acfd910b868351f115382fd92f95e5345992c
author : Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 17 Dec 2024 14:13:27 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 17 Dec 2024 14:13:27 -0500
Heap vacuum already counts and logs pages with newly frozen tuples. Now
count and log the number of pages newly set all-visible and all-frozen
in the visibility map.
Pages that are all-visible but not all-frozen are debt for future
aggressive vacuums. The counts of newly all-visible and all-frozen pages
give us insight into the rate at which this debt is being accrued and
paid down.
Author: Melanie Plageman
Reviewed-by: Masahiko Sawada, Alastair Turner, Nitin Jadhav, Andres Freund, Bilal Yavuz, Tomas Vondra
Discussion: https://postgr.es/m/flat/CAAKRu_ZQe26xdvAqo4weHLR%3DivQ8J4xrSfDDD8uXnh-O-6P6Lg%40mail.gmail.com#6d8d2b4219394f774889509bf3bdc13d,
https://postgr.es/m/ctdjzroezaxmiyah3gwbwm67defsrwj2b5fpfs4ku6msfpxeia%40mwjyqlhwr2wu
M src/backend/access/heap/vacuumlazy.c
Make visibilitymap_set() return previous state of vmbits
commit : 4b565a198b524469df9f3286a6f12778acd9d564
author : Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 17 Dec 2024 14:13:18 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 17 Dec 2024 14:13:18 -0500
It can be useful to know the state of a relation page's VM bits before
visibilitymap_set(). visibilitymap_set() has the old value on hand, so
returning it is simple. This commit does not use visibilitymap_set()'s
new return value.
Author: Melanie Plageman
Reviewed-by: Masahiko Sawada, Andres Freund, Nitin Jadhav, Bilal Yavuz
Discussion: https://postgr.es/m/flat/CAAKRu_ZQe26xdvAqo4weHLR%3DivQ8J4xrSfDDD8uXnh-O-6P6Lg%40mail.gmail.com#6d8d2b4219394f774889509bf3bdc13d,
https://postgr.es/m/ctdjzroezaxmiyah3gwbwm67defsrwj2b5fpfs4ku6msfpxeia%40mwjyqlhwr2wu
M src/backend/access/heap/visibilitymap.c
M src/include/access/visibilitymap.h
Rename LVRelState->frozen_pages
commit : f020baa0662c73aa7e470b1e3856281199ce1ee4
author : Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 17 Dec 2024 14:13:00 -0500
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Tue, 17 Dec 2024 14:13:00 -0500
Rename frozen_pages to new_frozen_tuple_pages in LVRelState, the struct
used for tracking state during vacuuming of a heap relation.
frozen_pages sounds like it tracks pages set all-frozen. That is a
misnomer. It only includes pages with at least one newly frozen tuple.
It also includes pages that are not all-frozen.
Author: Melanie Plageman
Reviewed-by: Andres Freund, Masahiko Sawada, Nitin Jadhav, Bilal Yavuz
Discussion: https://postgr.es/m/ctdjzroezaxmiyah3gwbwm67defsrwj2b5fpfs4ku6msfpxeia%40mwjyqlhwr2wu
M src/backend/access/heap/vacuumlazy.c
Set max_safe_fds whenever we create shared memory and semaphores.
commit : 21fb39cb07938d29f2c7cfcf627d1b3d6b48e61c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Dec 2024 12:23:26 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Dec 2024 12:23:26 -0500
Formerly we skipped this in bootstrap/check mode and in single-user
mode. That's bad in check mode because it may allow accepting a
value of max_connections that doesn't actually work: on platforms
where semaphores consume file descriptors, there may not be enough
free FDs left over to satisfy fd.c, causing postmaster start to
fail. It's also not great in single-user mode, because fd.c will
operate with just the minimum allowable value of max_safe_fds,
resulting in excess file open/close overhead if anything moderately
complicated is done in single-user mode. (There may be some penalty
for bootstrap mode too, though probably not much.)
Discussion: https://postgr.es/m/2081982.1734393311@sss.pgh.pa.us
M src/backend/bootstrap/bootstrap.c
M src/backend/tcop/postgres.c
Set the stack_base_ptr in main(), not in random other places.
commit : c91963da1302e8dd490bde115f3956f7d2f1258d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Dec 2024 12:08:39 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Dec 2024 12:08:39 -0500
Previously we did this in PostmasterMain() and InitPostmasterChild(),
which meant that stack depth checking was disabled in non-postmaster
server processes, for instance in single-user mode. That seems like
a fairly bad idea, since there's no a-priori restriction on the
complexity of queries we will run in single-user mode. Moreover, this
led to not having quite the same stack depth limit in all processes,
which likely has no real-world effect but it offends my inner neatnik.
Setting the depth in main() guarantees that check_stack_depth() is
armed and has a consistent interpretation of stack depth in all forms
of server processes.
While at it, move the code associated with checking the stack depth
out of tcop/postgres.c (which was never a great home for it) into
a new file src/backend/utils/misc/stack_depth.c.
Discussion: https://postgr.es/m/2081982.1734393311@sss.pgh.pa.us
M src/backend/main/main.c
M src/backend/postmaster/postmaster.c
M src/backend/tcop/postgres.c
M src/backend/utils/init/miscinit.c
M src/backend/utils/misc/Makefile
M src/backend/utils/misc/meson.build
A src/backend/utils/misc/stack_depth.c
M src/include/miscadmin.h
M src/include/tcop/tcopprot.h
Detect version mismatch in brin_page_items
commit : 957ba9ff14066782a42ebb974913b2fc616c99e1
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 17 Dec 2024 16:47:23 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 17 Dec 2024 16:47:23 +0100
Commit dae761a87ed modified brin_page_items() to return the new "empty"
flag for each BRIN range. But the new output parameter was added in the
middle, which may cause crashes when using the new binary with old
function definition.
The ideal solution would be to introduce API versioning similar to what
pg_stat_statements does, but it's too late for that as PG17 was already
released (so we can't introduce a new extension version). We could do
something similar in brin_page_items() by checking the number of output
columns (and ignoring the new flag), but it doesn't seem very nice.
Instead, simply error out and suggest updating the extension to the
latest version. pageinspect is a superuser-only extension, and there's
not much reason to run an older version. Moreover, there's a precedent
for this approach in 691e8b2e18.
Reported by Ľuboslav Špilák, investigation and patch by me. Backpatch to
17, same as dae761a87ed.
Reported-by: Ľuboslav Špilák
Reviewed-by: Michael Paquier, Hayato Kuroda, Peter Geoghegan
Backpatch-through: 17
Discussion: https://postgr.es/m/VI1PR02MB63331C3D90E2104FD12399D38A5D2@VI1PR02MB6333.eurprd02.prod.outlook.com
Discussion: https://postgr.es/m/flat/3385a58f-5484-49d0-b790-9a198a0bf236@vondra.me
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/expected/oldextversions.out
M contrib/pageinspect/sql/oldextversions.sql
Update comments about index parallel builds
commit : 8cd44db42a40056a701a4ae03d73bf2cd05781b1
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 17 Dec 2024 15:40:07 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 17 Dec 2024 15:40:07 +0100
Commit b43757171470 allowed parallel builds for BRIN, but left behind
two comments claiming only btree indexes support parallel builds.
Reported by Egor Rogov, along with similar issues in SGML docs.
Backpatch to 17, where parallel builds for BRIN were introduced.
Reported-by: Egor Rogov
Backpatch-through: 17
Discussion: https://postgr.es/m/114e2d5d-125e-07d8-94aa-5ad175fb7443@postgrespro.ru
M src/backend/catalog/index.c
M src/backend/optimizer/plan/planner.c
Remove ts_locale.c's lowerstr()
commit : fb1a18810f07fc3b722392103d67ce8ed188b63d
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 17 Dec 2024 14:04:55 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 17 Dec 2024 14:04:55 +0100
lowerstr() and lowerstr_with_len() in ts_locale.c do the same thing as
str_tolower() that the rest of the system uses, except that the former
don't use the common locale provider framework but instead use the
global libc locale settings.
This patch replaces uses of lowerstr*() with str_tolower(...,
DEFAULT_COLLATION_OID). For instances that use a libc locale
globally, this will result in exactly the same behavior. For
instances that use other locale providers, you now get consistent
behavior and are no longer dependent on the libc locale settings (for
this case; there are others).
Most uses of these functions are for processing dictionary and
configuration files. In those cases, using the default collation
seems appropriate. At least we don't have a more specific collation
available. But the code in contrib/pg_trgm should really depend on
the collation of the columns being processed. This is not done here,
this can be done in a separate patch.
(You can probably construct some edge cases where this change would
create some locale-related upgrade incompatibility, for example if
before you used a combination of ICU and a differently-behaving libc
locale. We can document this in the release notes, but I don't think
there is anything more we can do about this.)
Reviewed-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://www.postgresql.org/message-id/flat/653f3b84-fc87-45a7-9a0c-bfb4fcab3e7d%40eisentraut.org
M contrib/dict_xsyn/dict_xsyn.c
M contrib/pg_trgm/trgm_op.c
M contrib/pg_trgm/trgm_regexp.c
M src/backend/snowball/dict_snowball.c
M src/backend/tsearch/dict_ispell.c
M src/backend/tsearch/dict_simple.c
M src/backend/tsearch/dict_synonym.c
M src/backend/tsearch/spell.c
M src/backend/tsearch/ts_locale.c
M src/backend/tsearch/ts_utils.c
M src/include/tsearch/ts_locale.h
M src/include/tsearch/ts_public.h
Remove ts_locale.c's t_isdigit(), t_isspace(), t_isprint()
commit : d3aad4ac57c5592ade77916404e6d8a989a1d6a1
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 17 Dec 2024 12:48:58 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 17 Dec 2024 12:48:58 +0100
These do the same thing as the standard isdigit(), isspace(), and
isprint() but with multibyte and encoding support. But all the
callers are only interested in analyzing single-byte ASCII characters.
So this extra layer is overkill and we can replace the uses with the
standard functions.
All the t_is*() functions in ts_locale.c are under scrutiny because
they don't use the common locale provider framework but instead use
the global libc locale settings. For the functions being touched by
this patch, we don't need all that anyway, as mentioned above, so the
simplest solution is to just remove them. The few remaining t_is*()
functions will need a different treatment in a separate patch.
pg_trgm has some compile-time options with macros such as
KEEPONLYALNUM. These are not documented, and the non-default variant
is not supported by any test cases. As part of this undertaking, I'm
removing the non-default variant, as it is in the way of cleanup. So
in this case, the not-KEEPONLYALNUM code path is gone.
Reviewed-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://www.postgresql.org/message-id/flat/653f3b84-fc87-45a7-9a0c-bfb4fcab3e7d%40eisentraut.org
M contrib/dict_xsyn/dict_xsyn.c
M contrib/ltree/ltree_io.c
M contrib/ltree/ltxtquery_io.c
M contrib/pg_trgm/trgm.h
M contrib/unaccent/unaccent.c
M src/backend/tsearch/dict_synonym.c
M src/backend/tsearch/dict_thesaurus.c
M src/backend/tsearch/spell.c
M src/backend/tsearch/ts_locale.c
M src/backend/tsearch/ts_utils.c
M src/backend/utils/adt/tsquery.c
M src/backend/utils/adt/tsvector_parser.c
M src/include/tsearch/ts_locale.h
Avoid unnecessary wrapping for more complex expressions
commit : 60be3f9f0a64a071822c0711f69c7b5f6467d84f
author : Richard Guo <rguo@postgresql.org>
date : Tue, 17 Dec 2024 19:53:01 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Tue, 17 Dec 2024 19:53:01 +0900
When pulling up a subquery that is under an outer join, if the
subquery's target list contains a strict expression that uses a
subquery variable, it's okay to pull up the expression without
wrapping it in a PlaceHolderVar: if the subquery variable is forced to
NULL by the outer join, the expression result will come out as NULL
too.
If the strict expression does not contain any subquery variables, the
current code always wraps it in a PlaceHolderVar. While this is not
incorrect, the analysis could be tighter: if the strict expression
contains any variables of rels that are under the same lowest nulling
outer join as the subquery, we can also avoid wrapping it. This is
safe because if the subquery variable is forced to NULL by the outer
join, the variables of rels that are under the same lowest nulling
outer join will also be forced to NULL, resulting in the expression
evaluating to NULL as well. Therefore, it's not necessary to force
the expression to be evaluated below the outer join. It could be
beneficial to get rid of such PHVs because they could imply lateral
dependencies, which force us to resort to nestloop joins.
This patch checks if the lateral references in the strict expression
contain any variables of rels under the same lowest nulling outer join
as the subquery, and avoids wrapping the expression in that case.
This is fundamentally a generalization of the optimizations for bare
Vars and PHVs introduced in commit f64ec81a8.
No backpatch as this could result in plan changes.
Author: Richard Guo
Discussion: https://postgr.es/m/CAMbWs4_ENtfRdLaM_bXAxiKRYO7DmwDBDG4_2=VTDi0mJP-jAw@mail.gmail.com
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
Doc: Fix the wrong link on pg_createsubscriber page.
commit : 2364f61488ec1b4bc40d3a73dba4385e8ede65a0
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 17 Dec 2024 15:08:29 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 17 Dec 2024 15:08:29 +0530
Commit 84db9a0eb1 has added the incorrect link to
'initial data synchronization'. It was a subsection of Row Filter and
didn't provide the required information.
Author: Peter Smith
Reviewed-by: Vignesh C, Pavel Luzanov
Backpatch-through: 17, where it was introduced
Discussion: https://postgr.es/m/CAHut+PtnA4DB_pcv4TDr4NjUSM1=P2N_cuZx5DX09k7LVmaqUA@mail.gmail.com
M doc/src/sgml/ref/pg_createsubscriber.sgml
Tweak some comments related to variable-numbered stats in pgstat.c
commit : fee2b3ea2ecd0da0c88832b37ac0d9f6b3bfb9a9
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 17 Dec 2024 14:32:35 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 17 Dec 2024 14:32:35 +0900
These comments referred to database objects, but depending on the stats
kind dealt with this may not be true.
Issues found while reviewing a different patch in this area.
Discussion: https://postgr.es/m/ZtXR+CtkEVVE/LHF@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/utils/activity/pgstat.c
Print out error position for some more DDLs
commit : 0f23dedc91760271aefe9e3c52d677c079bb7bce
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 17 Dec 2024 09:44:06 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 17 Dec 2024 09:44:06 +0900
The following commands gain some information about the error position in
the query, should they fail when looking at the type used:
- CREATE TYPE (LIKE)
- CREATE TABLE OF
Both are related to typenameType() where the type name lookup is done.
These calls gain the ParseState that already exists in these paths.
Author: Kirill Reshke, Jian He
Reviewed-by: Álvaro Herrera, Michael Paquier
Discussion: https://postgr.es/m/CALdSSPhqfvKbDwqJaY=yEePi_aq61GmMpW88i6ZH7CMG_2Z4Cg@mail.gmail.com
M src/backend/commands/typecmds.c
M src/backend/parser/parse_utilcmd.c
M src/test/regress/expected/float8.out
M src/test/regress/expected/typed_table.out
pg_combinebackup: Fix PITR comparison test in 002_compare_backups
commit : e116b703f0b9b551cb413c8f2cfe231a9ce73f42
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 17 Dec 2024 09:23:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 17 Dec 2024 09:23:49 +0900
The test was creating both the dumps to compare from the same database
on the same node, so it would never detect any mismatches when comparing
the logical dumps of the two servers.
Fixing this issue has revealed that there is a difference in the dumps:
the tablespaces paths are different. This commit uses compare_text()
with a custom comparison function to erase the difference (slightly
tweaked to be able to work with WIN32 and non-WIN32 paths). This way,
the non-relevant parts of the tablespace path are ignored from the check
with the basic structure of the query string still compared.
Author: Dagfinn Ilmari Mannsåker
Discussion: https://postgr.es/m/87h67653ns.fsf@wibble.ilmari.org
Backpatch-through: 17
M src/bin/pg_combinebackup/t/002_compare_backups.pl
doc: Mention BRIN indexes support parallel builds
commit : 0cc7da4e234c3027cc1f96ffcb1365bf96301587
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 19:08:52 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 19:08:52 +0100
Two places in the documentation suggest B-tree is the only index access
method allowing parallel builds. Commit b4375717 added parallel builds
for BRIN too, but failed to update the docs. So fix that, and backpatch
to 17, where parallel BRIN builds were introduced.
Author: Egor Rogov
Backpatch-through: 17
Discussion: https://postgr.es/m/114e2d5d-125e-07d8-94aa-5ad175fb7443@postgrespro.ru
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/create_index.sgml
psql: Tab completion for JOIN ... USING column list
commit : 3429145d42e0aee0565328c8ca8378ed17b300dc
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 18:12:29 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 18:12:29 +0100
For JOIN ... USING, offer attribute names for the first member of the
column list.
Author: Andreas Karlsson
Reviewed-By: Tomas Vondra
Discussion: https://postgr.es/m/3a7e27bc-d6ed-4cb0-9b21-f21143fc1b37@proxel.se
M src/bin/psql/tab-complete.in.c
psql: Tab completion for JOIN ... ON/USING
commit : a01f6fa6ad5e232e1bd38c05d443875ae3ba7ee8
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 18:08:30 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 18:08:30 +0100
Offer ON/USING clauses for join types that require join conditions (i.e.
anything except for NATURAL/CROSS joins).
Author: Andreas Karlsson
Reviewed-By: Tomas Vondra
Discussion: https://postgr.es/m/3a7e27bc-d6ed-4cb0-9b21-f21143fc1b37@proxel.se
M src/bin/psql/tab-complete.in.c
psql: Tab completion for LATERAL joins
commit : 5dd5786b94cea4652035a5dc55c103ed09b0365b
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 17:55:00 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 17:55:00 +0100
When listing selectable objects after a JOIN, offer also LATERAL.
Author: Andreas Karlsson
Reviewed-By: Tomas Vondra
Discussion: https://postgr.es/m/3a7e27bc-d6ed-4cb0-9b21-f21143fc1b37@proxel.se
M src/bin/psql/tab-complete.in.c
Refactor string case conversion into provider-specific files.
commit : 86a5d6006aff956a5e00982b7628177fa7dc5027
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 16 Dec 2024 09:35:18 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 16 Dec 2024 09:35:18 -0800
Create API entry points pg_strlower(), etc., that work with any
provider and give the caller control over the destination
buffer. Then, move provider-specific logic into pg_locale_builtin.c,
pg_locale_icu.c, and pg_locale_libc.c as appropriate.
Discussion: https://postgr.es/m/7aa46d77b377428058403723440862d12a8a129a.camel@j-davis.com
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_locale_builtin.c
M src/backend/utils/adt/pg_locale_icu.c
M src/backend/utils/adt/pg_locale_libc.c
M src/include/utils/pg_locale.h
psql: Tab completion for CREATE MATERIALIZED VIEW ... USING
commit : de1e29885730851787b467449f525ff6fc7d69fa
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 16:46:56 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 16:46:56 +0100
The tab completion didn't offer USING for CREATE MATERIALIZED VIEW, so
add it, and offer a list of access methods, followed by SELECT.
Author: Kirill Reshke
Reviewed-By: Karina Litskevich
Discussion: https://postgr.es/m/CALdSSPhVELkvutquqrDB=Ujfq_Pjz=6jn-kzh+291KPNViLTfw@mail.gmail.com
M src/bin/psql/tab-complete.in.c
psql: Tab completion for CREATE TEMP TABLE ... USING
commit : 1e1f70c34a8e8cef8d7f55ecb73ef22771c2e21b
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 16:38:35 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 16:38:35 +0100
The USING keyword was offered only for persistent tables, not for
temporary ones. So improve that.
Author: Kirill Reshke
Reviewed-By: Karina Litskevich
Discussion: https://postgr.es/m/CALdSSPhVELkvutquqrDB=Ujfq_Pjz=6jn-kzh+291KPNViLTfw@mail.gmail.com
M src/bin/psql/tab-complete.in.c
psql: Tab completion for ALTER TYPE ... CASCADE/RESTRICT
commit : 8f11ef80c5458f0d12eefa1c6fbba81af8bb3042
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 16:20:04 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 16:20:04 +0100
Updates table completion for ALTER TYPE to offer CASCADE/RESTRICT for a
number of actions on attributes:
ALTER TYPE ... ADD/DROP/RENAME ATTRIBUTE ... [CASCADE|RESTRICT]
ALTER TYPE ... TYPE ... [CASCADE|RESTRICT]
Author: Kirill Reshke
Reviewed-By: Karina Litskevich
Discussion: https://postgr.es/m/CALdSSPhVELkvutquqrDB=Ujfq_Pjz=6jn-kzh+291KPNViLTfw@mail.gmail.com
M src/bin/psql/tab-complete.in.c
psql: Tab completion for ALTER TYPE ... ADD ATTRIBUTE
commit : e0275c380c3f4c11c0d6b9005b930a4d7e4b6862
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 15:53:36 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 15:53:36 +0100
Improve psql tab completion for ALTER TYPE ... ADD ATTRIBUTE to offer a
list of existing data types (until now no options were offered).
Author: Kirill Reshke
Reviewed-By: Karina Litskevich
Discussion: https://postgr.es/m/CALdSSPhVELkvutquqrDB=Ujfq_Pjz=6jn-kzh+291KPNViLTfw@mail.gmail.com
M src/bin/psql/tab-complete.in.c
Make 009_twophase.pl test pass with recovery_min_apply_delay set
commit : 1dfeb6af7f87d5117389870765c210bf6b6b8ed1
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 16 Dec 2024 15:56:38 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 16 Dec 2024 15:56:38 +0200
The test failed if you ran the regression tests with TEMP_CONFIG with
recovery_min_apply_delay = '500ms'. Fix the race condition by waiting
for transaction to be applied in the replica, like in a few other
tests.
The failing test was introduced in commit cbfbda7841. Backpatch to all
supported versions like that commit (except v12, which is no longer
supported).
Reported-by: Alexander Lakhin
Discussion: https://www.postgresql.org/message-id/09e2a70a-a6c2-4b5c-aeae-040a7449c9f2@gmail.com
M src/test/recovery/t/009_twophase.pl
Print out error position for CREATE DOMAIN
commit : 39240bcad56dc51a7896d04a1e066efcf988b58f
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 16 Dec 2024 14:52:11 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 16 Dec 2024 14:52:11 +0900
This is simply done by pushing down the ParseState available in
ProcessUtility() to DefineDomain(), giving more information about the
position of an error when running a CREATE DOMAIN query.
Most of the queries impacted by this change have been added previously
in 0172b4c9449e.
Author: Kirill Reshke, Jian He
Reviewed-by: Álvaro Herrera, Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/CALdSSPhqfvKbDwqJaY=yEePi_aq61GmMpW88i6ZH7CMG_2Z4Cg@mail.gmail.com
M src/backend/commands/typecmds.c
M src/backend/tcop/utility.c
M src/include/commands/typecmds.h
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/expected/collate.linux.utf8.out
M src/test/regress/expected/collate.out
M src/test/regress/expected/collate.windows.win1252.out
M src/test/regress/expected/domain.out
Add some tests for encoding conversion in COPY TO/FROM
commit : 3ad8b840ce8b1d7279f2d0d5fb7d346c0a6a3e8d
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 16 Dec 2024 11:23:38 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 16 Dec 2024 11:23:38 +0900
This adds a couple of tests to trigger encoding conversion when input
and server encodings do not match in COPY FROM/TO, or need_transcoding
set to true in the COPY state data. These tests rely on UTF8 <-> LATIN1
for the valid cases as LATIN1 accepts any bytes, and UTF8 <-> EUC_JP for
some of the invalid cases where a character cannot be understood,
causing a conversion failure.
Both ENCODING and client_encoding are covered. Test suggested by Andres
Freund.
Author: Sutou Kouhei
Discussion: https://postgr.es/m/20240206222445.hzq22pb2nye7rm67@awork3.anarazel.de
A src/test/regress/expected/copyencoding.out
A src/test/regress/expected/copyencoding_1.out
M src/test/regress/parallel_schedule
A src/test/regress/sql/copyencoding.sql
Declare a couple of variables inside not outside a PG_TRY block.
commit : bf9165bb0c5cea71e1a9cfa0c584c4d176f6c36f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 15 Dec 2024 15:50:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 15 Dec 2024 15:50:07 -0500
I went through the buildfarm's reports of "warning: variable 'foo'
might be clobbered by 'longjmp' or 'vfork' [-Wclobbered]". As usual,
none of them are live problems according to my understanding of the
effects of setjmp/longjmp, to wit that local variables might revert
to their values as of PG_TRY entry, due to being kept in registers.
But I did happen to notice that XmlTableGetValue's "cstr" variable
doesn't need to be declared outside the PG_TRY block at all (thus
giving further proof that the -Wclobbered warning has little
connection to real problems). We might as well move it inside,
and "cur" too, in hopes of eliminating one of the bogus warnings.
M src/backend/utils/adt/xml.c
pgbench: fix misprocessing of some nested \if constructs.
commit : 530f89e648da3a5505920322dcd4e00e15559f66
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 15 Dec 2024 14:14:14 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 15 Dec 2024 14:14:14 -0500
An \if command appearing within a false (not-to-be-executed) \if
branch was incorrectly treated the same as \elif. This could allow
statements within the inner \if to be executed when they should
not be. Also the missing inner \if stack entry would result in an
assertion failure (in assert-enabled builds) when the final \endif
is reached.
Report and patch by Michail Nikolaev. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/CANtu0oiA1ke=SP6tauhNqkUdv5QFsJtS1p=aOOf_iU+EhyKkjQ@mail.gmail.com
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
doc: Clarify old WAL files are kept until they are summarized.
commit : 56499315a74f97d220373e396903c389d85c8933
author : Fujii Masao <fujii@postgresql.org>
date : Sun, 15 Dec 2024 11:18:18 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Sun, 15 Dec 2024 11:18:18 +0900
The documentation in wal.sgml explains that old WAL files cannot be
removed or recycled until they are archived (when WAL archiving is used)
or replicated (when using replication slots). However, it did not mention
that, similarly, old WAL files are also kept until they are summarized
if WAL summarization is enabled. This commit adds that clarification
to the documentation.
Back-patch to v17 where WAL summarization was added.
Author: Fujii Masao
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/fd0eb0a5-f43b-4e06-b450-cbca011b6cff@oss.nttdata.com
M doc/src/sgml/wal.sgml
contrib/earthdistance: Use SQL-standard function bodies.
commit : 969bbd0fafc0f4d9ef504ca98a127c8d945f71a0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 14 Dec 2024 16:07:18 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 14 Dec 2024 16:07:18 -0500
The @extschema:name@ feature added by 72a5b1fc8 allows us to
make earthdistance's references to the cube extension fully
search-path-secure, so long as all those references are
resolved at extension installation time not runtime.
To do that, we must convert earthdistance's SQL functions to
the new SQL-standard style; but we wanted to do that anyway.
The functions can be updated in our customary style by running
CREATE OR REPLACE FUNCTION in an extension update script.
However, there's still problems in the "CREATE DOMAIN earth"
command: its references to cube functions could be captured
by hostile objects in earthdistance's installation schema,
if that's not where the cube extension is. Worse, the reference
to the cube type itself as the domain's base could be captured,
and that's not something we could fix after-the-fact in the
update script.
What I've done about that is to change the "CREATE DOMAIN earth"
command in the base script earthdistance--1.1.sql. Ordinarily,
changing a released extension script is forbidden; but I think
it's okay here since the results of successful (non-trojaned)
script execution will be identical to before.
A good deal of care is still needed to make the extension's scripts
proof against search-path-based attacks. We have to make sure that
all the function and operator invocations have exact argument-type
matches, to forestall attacks based on supplying a better match.
Fortunately earthdistance isn't very big, so I've just gone through
it and inspected each call to be sure of that. The only actual code
changes needed were to spell all floating-point constants in the style
'-1'::float8, rather than depending on runtime type conversions and/or
negations. (I'm not sure that the shortcuts previously used were
attackable, but removing run-time effort is a good thing anyway.)
I believe that this fixes earthdistance enough that we could
mark it trusted and remove the warnings about it that were
added by 7eeb1d986; but I've not done that here.
The primary reason for dealing with this now is that we've
received reports of pg_upgrade failing for databases that use
earthdistance functions in contexts like generated columns.
That's a consequence of 2af07e2f7 having restricted the search_path
used while evaluating such expressions. The only way to fix that
is to make the earthdistance functions independent of run-time
search_path. This patch is very much nicer than the alternative of
attaching "SET search_path" clauses to earthdistance's functions:
it is more secure and doesn't create a run-time penalty. Therefore,
I've chosen to back-patch this to v16 where @extschema:name@
was added. It won't help unless users update to 16.7 and issue
"ALTER EXTENSION earthdistance UPDATE" before upgrading to 17,
but at least there's now a way to deal with the problem without
manual intervention in the dump/restore process.
Tom Lane and Ronan Dunklau
Discussion: https://postgr.es/m/3316564.aeNJFYEL58@aivenlaptop
Discussion: https://postgr.es/m/6a6439f1-8039-44e2-8fb9-59028f7f2014@mailbox.org
M contrib/earthdistance/Makefile
A contrib/earthdistance/earthdistance–1.1–1.2.sql
M contrib/earthdistance/earthdistance–1.1.sql
M contrib/earthdistance/earthdistance.control
M contrib/earthdistance/meson.build
Refactor some SQL/JSON error messages
commit : 62b7a9a7784eb8b3871368726fc3b8df17d87f32
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 14 Dec 2024 12:55:00 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 14 Dec 2024 12:55:00 +0100
Turn type names into "%s" specifiers to 1) avoid getting them translated
and 2) reduce the total number of messages.
M src/backend/utils/adt/jsonpath_exec.c
Fix warnings about declaration of environ on MinGW.
commit : 7bc9a8bdd2d6f6da664572456f226c54e9c9e3dd
author : Thomas Munro <tmunro@postgresql.org>
date : Sun, 15 Dec 2024 00:36:30 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sun, 15 Dec 2024 00:36:30 +1300
POSIX says that the global variable environ shouldn't be declared in a
header, and that you have to declare it yourself. MinGW declares it in
<stdlib.h> with some macrology that messes up our declarations. Visual
Studio doesn't warn (there are clues that it may also declare it, but if
so, apparently compatibly). Suppress our declarations, on MinGW only.
This clears the last warnings on CI's optional MinGW task, and hopefully
on build farm animal fairywren too.
Like 1319997d, no back-patch for now as it's not known to be breaking
anything, and my humble goal is just to keep the MinGW build clean going
forward.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (earlier version)
Discussion: https://postgr.es/m/CA%2BhUKGJLMh%2B6W5E4M_jSFb43gnrA_-Q6-%2BBf3HkBXyGfRFcBsQ%40mail.gmail.com
M src/backend/postmaster/postmaster.c
M src/backend/utils/misc/ps_status.c
M src/test/regress/regress.c
Remove EXTENSION_DONT_CHECK_SIZE from md.c.
commit : 48c142f78d90fcdcbc0557a4bcdc4f47ec32b333
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 14 Dec 2024 20:59:58 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 14 Dec 2024 20:59:58 +1300
Commits 7bb3102c and 3eb77eba removed the only user of the
EXTENSION_DONT_CHECK_SIZE flag, which had previously been required to
checkpoint truncated relations. Since 7bb3102c, segments have been
opened directly for synchronization without calling _mdfd_getseg(), so
it doesn't need a mode that tolerates non-final short segments. Remove
the redundant flag and associated comments.
Reported-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/nyj4k7yur5t27rtygvx2i2lrlp6rqfvvhoiiwx4fznynksf2et%404hj2sp42alpe
M src/backend/storage/smgr/md.c
Fix typo
commit : c72ca3ddec15b73d9f9dc3e21b8db5d958b0dad7
author : John Naylor <john.naylor@postgresql.org>
date : Sat, 14 Dec 2024 09:52:08 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Sat, 14 Dec 2024 09:52:08 +0700
Ryo Kanbayashi
Discussion: https://postgr.es/m/CANOn0ExEQiPVrzkjULkENVac_n4Lknm6dxsU69MSncQap0kJVA%40mail.gmail.com
M src/backend/optimizer/util/restrictinfo.c
Fix possible crash in pg_dump with identity sequences.
commit : 7b8cb9cd6a76b1b6a77032a1c352b258b6abfd39
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 13 Dec 2024 14:21:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 13 Dec 2024 14:21:36 -0500
If an owned sequence is considered interesting, force its owning
table to be marked interesting too. This ensures, in particular,
that we'll fetch the owning table's column names so we have the
data needed for ALTER TABLE ... ADD GENERATED. Previously there were
edge cases where pg_dump could get SIGSEGV due to not having filled in
the column names. (The known case is where the owning table has been
made part of an extension while its identity sequence is not a member;
but there may be others.)
Also, if it's an identity sequence, force its dumped-components mask
to exactly match the owning table: dump definition only if we're
dumping the table's definition, dump data only if we're dumping the
table's data, etc. This generalizes the code introduced in commit
b965f2617 that set the sequence's dump mask to NONE if the owning
table's mask is NONE. That's insufficient to prevent failures,
because for example the table's mask might only request dumping ACLs,
which would lead us to still emit ALTER TABLE ADD GENERATED even
though we didn't create the table. It seems better to treat an
identity sequence as though it were an inseparable aspect of the
table, matching the treatment used in the backend's dependency logic.
Perhaps this policy needs additional refinement, but let's wait to
see some field use-cases before changing it further.
While here, add a comment in pg_dump.h warning against writing tests
like "if (dobj->dump == DUMP_COMPONENT_NONE)", which was a bug in this
case. There is one other example in getPublicationNamespaces, which
if it's not a bug is at least remarkably unclear and under-documented.
Changing that requires a separate discussion, however.
Per report from Artur Zakirov. Back-patch to all supported branches.
Discussion: https://postgr.es/m/CAKNkYnwXFBf136=u9UqUxFUVagevLQJ=zGd5BsLhCsatDvQsKQ@mail.gmail.com
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
Rewrite maybe_reread_subscription() comment
commit : 3191eccd8a9bff1715f2e4fab86d2932a556185e
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 13 Dec 2024 07:41:36 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 13 Dec 2024 07:41:36 +0100
One sentence was gramatically wrong, but also too terse. Expand on it.
M src/backend/replication/logical/worker.c
Dump not-null constraints on inherited columns correctly
commit : fd41ba93e4630921a72ed5127cd0d552a8f3f8fc
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 13 Dec 2024 07:38:49 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 13 Dec 2024 07:38:49 +0100
With not-null constraints defined in child tables for columns that are
coming from their parent tables, we were printing ALTER TABLE SET NOT
NULL commands that were missing the constraint name, so the original
constraint name was being lost, which is bogus. Fix by instead adding
a table-constraint constraint declaration with the correct constraint
name in the CREATE TABLE instead.
Oversight in commit 14e87ffa5c54.
We could have fixed it by changing the ALTER TABLE SET NOT NULL to ALTER
TABLE ADD CONSTRAINT, but I'm not sure that's any better. A potential
problem here might be that if sent to a non-Postgres server, the new
pg_dump output would fail because the "CONSTRAINT foo NOT NULL colname"
syntax isn't SQL-conforming. However, Postgres' implementation of
inheritance is already non-SQL-conforming, so that'd likely fail anyway.
This problem was only noticed by Ashutosh's proposed test framework for
pg_dump, https://postgr.es/m/CAExHW5uF5V=Cjecx3_Z=7xfh4rg2Wf61PT+hfquzjBqouRzQJQ@mail.gmail.com
Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/CAExHW5tbdgAKDfqjDJ-7Fk6PJtHg8D4zUF6FQ4H2Pq8zK38Nyw@mail.gmail.com
M src/bin/pg_dump/pg_dump.c
Revert "Don't truncate database and user names in startup packets."
commit : a0ff56e2d3ff1db3de727b33b2dac985ccc43ef8
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 12 Dec 2024 15:52:04 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 12 Dec 2024 15:52:04 -0600
This reverts commit 562bee0fc13dc95710b8db6a48edad2f3d052f2e.
We received a report from the field about this change in behavior,
so it seems best to revert this commit and to add proper
multibyte-aware truncation as a follow-up exercise.
Fixes bug #18711.
Reported-by: Adam Rauch
Reviewed-by: Tom Lane, Bertrand Drouvot, Bruce Momjian, Thomas Munro
Discussion: https://postgr.es/m/18711-7503ee3e449d2c47%40postgresql.org
Backpatch-through: 17
M src/backend/tcop/backend_startup.c
Adjust some comments about structure properties in pg_stat.h
commit : 4766438aa317e85f3d762847b2b009f91f530b6f
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 12 Dec 2024 16:59:22 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 12 Dec 2024 16:59:22 +0900
One comment of PgStat_TableCounts mentioned that its pending stats use
memcmp() to check for the all-zero case if there is any activity. This
is not true since 07e9e28b56, as pg_memory_is_all_zeros() is used.
PgStat_FunctionCounts incorrectly documented that it relied on memcpy().
This has never been correct, and not relevant because function
statistics do not have an all-zero check for pending stats.
Checkpoint and bgwriter statistics have been always relying on memcmp()
or pg_memory_is_all_zeros() (since 07e9e28b56 for the latter), and never
mentioned the dependency on event counters for their all-zero checks.
Let's document these properties, like the table statistics.
Author: Bertrand Drouvot
Discussion: https://postgr.es/m/Z1hNLvcPgVLPxCoc@ip-10-97-1-34.eu-west-3.compute.internal
M src/include/pgstat.h
Detect redundant GROUP BY columns using UNIQUE indexes
commit : bd10ec529796a13670645e6acd640c6f290df020
author : David Rowley <drowley@postgresql.org>
date : Thu, 12 Dec 2024 15:28:38 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 12 Dec 2024 15:28:38 +1300
d4c3a156c added support that when the GROUP BY contained all of the
columns belonging to a relation's PRIMARY KEY, all other columns
belonging to that relation would be removed from the GROUP BY clause.
That's possible because all other columns are functionally dependent on
the PRIMARY KEY and those columns alone ensure the groups are distinct.
Here we expand on that optimization and allow it to work for any unique
indexes on the table rather than just the PRIMARY KEY index. This
normally requires that all columns in the index are defined with NOT NULL,
however, we can relax that requirement when the index is defined with
NULLS NOT DISTINCT.
When there are multiple suitable indexes to allow columns to be removed,
we prefer the index with the least number of columns as this allows us
to remove the highest number of GROUP BY columns. One day, we may want to
revisit that decision as it may make more sense to use the narrower set of
columns in terms of the width of the data types and stored/queried data.
This also adjusts the code to make use of RelOptInfo.indexlist rather
than looking up the catalog tables.
In passing, add another short-circuit path to allow bailing out earlier
in cases where it's certainly not possible to remove redundant GROUP BY
columns. This early exit is now cheaper to do than when this code was
originally written as 00b41463c made it cheaper to check for empty
Bitmapsets.
Patch originally by Zhang Mingli and later worked on by jian he, but after
I (David) worked on it, there was very little of the original left.
Author: Zhang Mingli, jian he, David Rowley
Reviewed-by: jian he, Andrei Lepikhov
Discussion: https://postgr.es/m/327990c8-b9b2-4b0c-bffb-462249f82de0%40Spark
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/util/plancat.c
M src/include/nodes/pathnodes.h
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql
Improve the test case from 5668a857d
commit : d8f335156c57f0df6ae3b1ec31e55979838eb882
author : Richard Guo <rguo@postgresql.org>
date : Thu, 12 Dec 2024 11:21:51 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Thu, 12 Dec 2024 11:21:51 +0900
In commit 5668a857d, we fixed an issue with incorrect results in right
semi joins and introduced a test case to verify the fix. The test
case involves SubPlans and InitPlans, which may not be immediately
apparent in relation to the issue we addressed.
This patch simplifies the test case with a more straightforward query.
Per discussion with Melanie Plageman.
Author: Richard Guo
Discussion: https://postgr.es/m/CAAKRu_a-Cip2XCXp13fmxq+T9BhLAVApHTyjr94awL2mbXHC-Q@mail.gmail.com
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Add some regression tests for missing DDL patterns
commit : 0172b4c9449e92a3988f669d9e7e9000454d16ce
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 12 Dec 2024 11:16:45 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 12 Dec 2024 11:16:45 +0900
The following commands gain increased coverage for some of the errors
they can trigger:
- ALTER TABLE .. ALTER COLUMN
- CREATE DOMAIN
- CREATE TYPE (LIKE)
This has come up while discussing the possibility to add more
information about the location of the error in such queries, and it
is useful on its own as there was no coverage until now for the
patterns added in this commit.
Author: Jian He, Kirill Reshke
Reviewed-By: Álvaro Herrera, Michael Paquier
Discussion: https://postgr.es/m/CALdSSPhqfvKbDwqJaY=yEePi_aq61GmMpW88i6ZH7CMG_2Z4Cg@mail.gmail.com
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/domain.out
M src/test/regress/expected/float8.out
M src/test/regress/expected/identity.out
M src/test/regress/sql/alter_table.sql
M src/test/regress/sql/domain.sql
M src/test/regress/sql/float8.sql
M src/test/regress/sql/identity.sql
Defer remove_useless_groupby_columns() work until query_planner()
commit : 430a5952deb3bfbfe1e2537315d44427b7c41fb1
author : David Rowley <drowley@postgresql.org>
date : Thu, 12 Dec 2024 14:22:15 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 12 Dec 2024 14:22:15 +1300
Traditionally, remove_useless_groupby_columns() was called during
grouping_planner() directly after the call to preprocess_groupclause().
While in many ways, it made sense to populate the field and remove the
functionally dependent columns from processed_groupClause at the same
time, it's just that doing so had the disadvantage that
remove_useless_groupby_columns() was being called before the RelOptInfos
were populated for the relations mentioned in the query. Not having
RelOptInfos available meant we needed to manually query the catalog tables
to get the required details about the primary key constraint for the
table.
Here we move the remove_useless_groupby_columns() call to
query_planner() and put it directly after the RelOptInfos are populated.
This is fine to do as processed_groupClause still isn't final at this
point as it can still be modified inside standard_qp_callback() by
make_pathkeys_for_sortclauses_extended().
This commit is just a refactor and simply moves
remove_useless_groupby_columns() into initsplan.c. A planned follow-up
commit will adjust that function so it uses RelOptInfo instead of doing
catalog lookups and also teach it how to use unique indexes as proofs to
expand the cases where we can remove functionally dependent columns from
the GROUP BY.
Reviewed-by: Andrei Lepikhov, jian he
Discussion: https://postgr.es/m/CAApHDvqLezKwoEBBQd0dp4Y9MDkFBDbny0f3SzEeqOFoU7Z5+A@mail.gmail.com
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/planmain.c
M src/backend/optimizer/plan/planner.c
M src/include/optimizer/planmain.h
Add UUID version 7 generation function.
commit : 78c5e141e9c139fc2ff36a220334e4aa25e1b0eb
author : Masahiko Sawada <msawada@postgresql.org>
date : Wed, 11 Dec 2024 15:54:41 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Wed, 11 Dec 2024 15:54:41 -0800
This commit introduces the uuidv7() SQL function, which generates UUID
version 7 as specified in RFC 9652. UUIDv7 combines a Unix timestamp
in milliseconds and random bits, offering both uniqueness and
sortability.
In our implementation, the 12-bit sub-millisecond timestamp fraction
is stored immediately after the timestamp, in the space referred to as
"rand_a" in the RFC. This ensures additional monotonicity within a
millisecond. The rand_a bits also function as a counter. We select a
sub-millisecond timestamp so that it monotonically increases for
generated UUIDs within the same backend, even when the system clock
goes backward or when generating UUIDs at very high
frequency. Therefore, the monotonicity of generated UUIDs is ensured
within the same backend.
This commit also expands the uuid_extract_timestamp() function to
support UUID version 7.
Additionally, an alias uuidv4() is added for the existing
gen_random_uuid() SQL function to maintain consistency.
Bump catalog version.
Author: Andrey Borodin
Reviewed-by: Sergey Prokhorenko, Przemysław Sztoch, Nikolay Samokhvalov
Reviewed-by: Peter Eisentraut, Jelte Fennema-Nio, Aleksander Alekseev
Reviewed-by: Masahiko Sawada, Lukas Fittl, Michael Paquier, Japin Li
Reviewed-by: Marcos Pegoraro, Junwang Zhao, Stepan Neretin
Reviewed-by: Daniel Vérité
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
M doc/src/sgml/datatype.sgml
M doc/src/sgml/func.sgml
M src/backend/utils/adt/uuid.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/uuid.out
M src/test/regress/sql/uuid.sql
Fix further fallout from EXPLAIN ANALYZE BUFFERS change
commit : 89988ac5891b3d41725472a65e50ae4e192313aa
author : David Rowley <drowley@postgresql.org>
date : Thu, 12 Dec 2024 09:50:00 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 12 Dec 2024 09:50:00 +1300
c2a4078eb adjusted EXPLAIN ANALYZE to default the BUFFERS to ON. This
(hopefully) fixes the last remaining issue with regression test failures
with -D RELCACHE_FORCE_RELEASE -D CATCACHE_FORCE_RELEASE builds, where
the planner accesses more buffers due to the cold caches.
Discussion: https://postgr.es/m/CAApHDvqLdzgz77JsE-yTki3w9UiKQ-uTMLRctazcu+99-ips3g@mail.gmail.com
M contrib/pg_stat_statements/expected/level_tracking.out
M contrib/pg_stat_statements/sql/level_tracking.sql
Use pg_memory_is_all_zeros() in pgstatfuncs.c.
commit : e8d59294282bd01bc06f2af13c79b9155024a917
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 11 Dec 2024 14:19:14 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 11 Dec 2024 14:19:14 -0600
There are a few places in this file that use memset() and memcmp()
to determine whether a section of memory is all zeros. This commit
modifies them to use pg_memory_is_all_zeros() instead. These
aren't expected to be hot code paths, but this may optimize them a
bit. Plus, this allows us to remove some variables that were only
needed for the memset() and memcmp().
Author: Bertrand Drouvot
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/Z1hNubHfvMxlW6eu%40ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/utils/adt/pgstatfuncs.c
Unmark gen_random_uuid() function leakproof.
commit : 398d3e3b5b8f4c81795e07655f28036839cc0550
author : Masahiko Sawada <msawada@postgresql.org>
date : Wed, 11 Dec 2024 10:35:57 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Wed, 11 Dec 2024 10:35:57 -0800
The functions without arguments don't need to be marked
leakproof. This commit unmarks gen_random_uuid() leakproof for
consistency with upcoming UUID generation functions. Also, this commit
adds a regression test to prevent reintroducing such cases.
Bump catalog version.
Reported-by: Peter Eisentraut
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/CAD21AoBE1ePPWY1NQEgk3DkqjYzLPZwYTzCySHm0e%2B9a69PfZw%40mail.gmail.com
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/opr_sanity.out
M src/test/regress/sql/opr_sanity.sql
Fix a memory leak in dumping functions with TRANSFORMs
commit : 0e033f5b6de569e712d5f94b77878b8ffacf6397
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 11 Dec 2024 12:48:22 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 11 Dec 2024 12:48:22 +0100
The gneration of the dump clause for functions with TRANSFORM
calls would leak the memory for holding the result of the Oid
array parsing. Fix by freeing.
While in there, switch to using pg_malloc instead of palloc in
order to be consistent with the rest of the file.
Author: Oleg Tselebrovskiy <o.tselebrovskiy@postgrespro.ru>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/baf1ae4511288e5b421f41e79a3df1a0@postgrespro.ru
M src/bin/pg_dump/pg_dump.c
Add missing BUFFERS OFF in regression tests, take 2
commit : 9df2a4b9316fae76477187bb2b64197169f5c346
author : David Rowley <drowley@postgresql.org>
date : Wed, 11 Dec 2024 23:16:44 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 11 Dec 2024 23:16:44 +1300
Similar to 9fa1aaa65, but running with -D RELCACHE_FORCE_RELEASE and
-D CATCACHE_FORCE_RELEASE yielded some additional missing places that
needed BUFFERS OFF.
Discussion: https://postgr.es/m/CANNMO++W7MM8T0KyXN3ZheXXt-uLVM3aEtZd+WNfZ=obxffUiA@mail.gmail.com
M src/test/regress/expected/matview.out
M src/test/regress/expected/misc_functions.out
M src/test/regress/sql/matview.sql
M src/test/regress/sql/misc_functions.sql
Add missing BUFFERS OFF in select_into regression tests
commit : 9fa1aaa6525af044385dbcefe9d9abb5d57578cd
author : David Rowley <drowley@postgresql.org>
date : Wed, 11 Dec 2024 22:56:36 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 11 Dec 2024 22:56:36 +1300
c2a4078eb adjusted EXPLAIN ANALYZE to include BUFFERS by default, but
a few tests in select_into.sql neglected to add BUFFERS OFF. The
failing tests seem unlikely to ever access buffers during execution, but
they certainly could during planning.
Per buildfarm member kestrel, tayra and calliphoridae.
Discussion: https://postgr.es/m/CANNMO++W7MM8T0KyXN3ZheXXt-uLVM3aEtZd+WNfZ=obxffUiA@mail.gmail.com
M src/test/regress/expected/select_into.out
M src/test/regress/sql/select_into.sql
Enable BUFFERS with EXPLAIN ANALYZE by default
commit : c2a4078ebad71999dd451ae7d4358be3c9290b07
author : David Rowley <drowley@postgresql.org>
date : Wed, 11 Dec 2024 22:35:11 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 11 Dec 2024 22:35:11 +1300
The topic of turning EXPLAIN's BUFFERS option on with the ANALYZE option
has come up a few times over the past few years. In many ways, doing this
seems like a good idea as it may be more obvious to users why a given
query is running more slowly than they might expect. Also, from my own
(David's) personal experience, I've seen users posting to the mailing
lists with two identical plans, one slow and one fast asking why their
query is sometimes slow. In many cases, this is due to additional reads.
Having BUFFERS on by default may help reduce some of these questions, and
if not, make it more obvious to the user before they post, or save a
round-trip to the mailing list when additional I/O effort is the cause of
the slowness.
The general consensus is that we want BUFFERS on by default with
ANALYZE. However, there were more than zero concerns raised with doing
so. The primary reason against is the additional verbosity, making it
harder to read large plans. Another concern was that buffer information
isn't always useful so may not make sense to have it on by default.
It's currently December, so let's commit this to see if anyone comes
forward with a strong objection against making this change. We have over
half a year remaining in the v18 cycle where we could still easily consider
reverting this if someone were to come forward with a convincing enough
reason as to why doing this is a bad idea.
There were two patches independently submitted to achieve this goal, one
by me and the other by Guillaume. This commit is a mix of both of these
patches with some additional work done by me to adjust various
additional places in the documentation which include EXPLAIN ANALYZE
output.
Author: Guillaume Lelarge, David Rowley
Reviewed-by: Robert Haas, Greg Sabino Mullane, Michael Christofides
Discussion: https://postgr.es/m/CANNMO++W7MM8T0KyXN3ZheXXt-uLVM3aEtZd+WNfZ=obxffUiA@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/bloom.sgml
M doc/src/sgml/jit.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/planstats.sgml
M doc/src/sgml/ref/explain.sgml
M src/backend/commands/explain.c
M src/test/regress/expected/brin_multi.out
M src/test/regress/expected/explain.out
M src/test/regress/expected/incremental_sort.out
M src/test/regress/expected/memoize.out
M src/test/regress/expected/merge.out
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/select.out
M src/test/regress/expected/select_into.out
M src/test/regress/expected/select_parallel.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/tidscan.out
M src/test/regress/sql/brin_multi.sql
M src/test/regress/sql/explain.sql
M src/test/regress/sql/incremental_sort.sql
M src/test/regress/sql/memoize.sql
M src/test/regress/sql/merge.sql
M src/test/regress/sql/partition_prune.sql
M src/test/regress/sql/select.sql
M src/test/regress/sql/select_into.sql
M src/test/regress/sql/select_parallel.sql
M src/test/regress/sql/subselect.sql
M src/test/regress/sql/tidscan.sql
Use ExprStates for hashing in GROUP BY and SubPlans
commit : 0f5738202b812a976e8612c85399b52d16a0abb6
author : David Rowley <drowley@postgresql.org>
date : Wed, 11 Dec 2024 13:47:16 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 11 Dec 2024 13:47:16 +1300
This speeds up obtaining hash values for GROUP BY and hashed SubPlans by
using the ExprState support for hashing, thus allowing JIT compilation for
obtaining hash values for these operations.
This, even without JIT compilation, has been shown to improve Hash
Aggregate performance in some cases by around 15% and hashed NOT IN
queries in one case by over 30%, however, real-world cases are likely to
see smaller gains as the test cases used were purposefully designed to
have high hashing overheads by keeping the hash table small to prevent
additional memory overheads that would be a factor when working with large
hash tables.
In passing, fix a hypothetical bug in ExecBuildHash32Expr() so that the
initial value is stored directly in the ExprState's result field if
there are no expressions to hash. None of the current users of this
function use an initial value, so the bug is only hypothetical.
Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>
Discussion: https://postgr.es/m/CAApHDvpYSO3kc9UryMevWqthTBrxgfd9djiAjKHMPUSQeX9vdQ@mail.gmail.com
M src/backend/executor/execExpr.c
M src/backend/executor/execGrouping.c
M src/backend/executor/nodeSubplan.c
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
Use in-place updates for pg_restore_relation_stats().
commit : a43567483c617fb046c805b61964d5168c9a0553
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 10 Dec 2024 16:30:37 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 10 Dec 2024 16:30:37 -0800
This matches the behavior of vac_update_relstats(), which is important
to avoid bloating pg_class.
Author: Corey Huinker
Discussion: https://postgr.es/m/CADkLM=fc3je+ufv3gsHqjjSSf+t8674RXpuXW62EL55MUEQd-g@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/statistics/relation_stats.c
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Improve reporting of pg_upgrade log files on test failure
commit : 8ede5016859e151003fdb662e13d11e43f372174
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 11 Dec 2024 08:48:47 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 11 Dec 2024 08:48:47 +0900
On failure, the pg_upgrade log files are automatically appended to the
test log file, but the information reported was inconsistent.
A header, with the log file name, was reported with note(), while the
log contents and a footer used print(), making it harder to diagnose
failures when these are split into console output and test log file
because the pg_upgrade log file path in the header may not be included
in the test log file.
The output is now consolidated so as the header uses print() rather than
note(). An extra note() is added to inform that the contents of a
pg_upgrade log file are appended to the test log file.
The diffs from the regression test suite and dump files all use print()
to show their contents on failure.
Author: Joel Jacobson
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/49f7e64a-b9be-4a90-a9fe-210a7740405e@app.fastmail.com
Backpatch-through: 15
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
Speedup Hash Joins with dedicated functions for ExprState hashing
commit : 50416cc4843a85fcb53507e21577cce16c75c65f
author : David Rowley <drowley@postgresql.org>
date : Wed, 11 Dec 2024 11:32:15 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 11 Dec 2024 11:32:15 +1300
Hashing of a single Var is a very common operation for ExprState to
perform. Here we add dedicated ExecJust* functions which helps speed up
Hash Joins by removing the interpretation overhead in ExecInterpExpr().
This change currently only affects Hash Joins on a single column. Hash
Joins with multiple join keys or an expression still run through
ExecInterpExpr().
Some testing has shown up to 10% query performance increases on recent AMD
hardware and nearly 7% increase on an Apple M2 for a query performing a
hash join with a large number of lookups on a small hash table.
This change was extracted from a larger patch which adjusts GROUP BY /
hashed subplans / hashed set operations to use ExprState hashing.
Discussion: https://postgr.es/m/CAApHDvr8Zc0ZgzVoCZLdHGOFNhiJeQ6vrUcS9V7N23zMWQb-eA@mail.gmail.com
M src/backend/executor/execExprInterp.c
Doc: add some commentary about ExecutorRun's NoMovement special case.
commit : 982890530379235974a7e99fd72e862ae40f1400
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 Dec 2024 17:17:28 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 Dec 2024 17:17:28 -0500
Robert Haas expressed concern about whether commit 3eea7a0c9 exposed
the parallel-execution machinery to a case it isn't tested for, namely
a second non-parallel execution of a plan after a parallel execution.
Investigation shows that that can't happen because of pquery.c's
manipulation of the scan direction, but it sure wasn't obvious to
start with. Add some commentary about that.
Discussion: https://postgr.es/m/CA+TgmoagyKQy=HFw+wLo0AKTYWHui+iKswZ8Jnqqd-cFby-WVg@mail.gmail.com
M src/backend/executor/execMain.c
Fix elog(FATAL) before PostmasterMain() or just after fork().
commit : 8b9cbf4922756d912309567f83f81e41f1737ac7
author : Noah Misch <noah@leadboat.com>
date : Tue, 10 Dec 2024 13:51:59 -0800
committer: Noah Misch <noah@leadboat.com>
date : Tue, 10 Dec 2024 13:51:59 -0800
Since commit 97550c0711972a9856b5db751539bbaf2f88884c, these failed with
"PANIC: proc_exit() called in child process" due to uninitialized or
stale MyProcPid. That was reachable if close() failed in
ClosePostmasterPorts() or setlocale(category, "C") failed, both
unlikely. Back-patch to v13 (all supported versions).
Discussion: https://postgr.es/m/20241208034614.45.nmisch@google.com
M src/backend/main/main.c
M src/backend/postmaster/fork_process.c
M src/backend/postmaster/postmaster.c
M src/port/pqsignal.c
Tests for logical replication with temporal keys
commit : 939b0908c87a7d82171097189b855e5d0d2dd716
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 10 Dec 2024 15:05:58 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 10 Dec 2024 15:05:58 +0100
This covers some cases that were previously failing before the
"Support for GiST in get_equal_strategy_number()" patch.
Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+renyUApHgSZF9-nd-a0+OPGharLQLO=mDHcY4_qQ0+noCUVg@mail.gmail.com
M src/test/subscription/meson.build
A src/test/subscription/t/034_temporal.pl
Support for GiST in get_equal_strategy_number()
commit : 74edabce7a33f428371f0da6c0338837fd9ac55e
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 10 Dec 2024 13:26:09 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 10 Dec 2024 13:26:09 +0100
A WITHOUT OVERLAPS primary key or unique constraint is accepted as a
REPLICA IDENTITY, since it guarantees uniqueness. But subscribers
applying logical decoding messages would fail because there was not
support for looking up the equals operator for a gist index. This
fixes that: For GiST indexes we can use the stratnum GiST support
function.
Reviewed-by: Paul Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+renyUApHgSZF9-nd-a0+OPGharLQLO=mDHcY4_qQ0+noCUVg@mail.gmail.com
M src/backend/executor/execReplication.c
Make the conditions in IsIndexUsableForReplicaIdentityFull() more explicit
commit : 13544e790ef8c4fe9043ba59276e5182ce9a623a
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 10 Dec 2024 13:11:34 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 10 Dec 2024 13:11:34 +0100
IsIndexUsableForReplicaIdentityFull() described a number of conditions
that a suitable index has to fulfill. But not all of these were
actually checked in the code. Instead, it appeared to rely on
get_equal_strategy_number() to filter out any indexes that are not
btree or hash. As we look to generalize index AM capabilities, this
would possibly break if we added additional support in
get_equal_strategy_number(). Instead, write out code to check for the
required capabilities explicitly. This shouldn't change any behaviors
at the moment.
Reviewed-by: Paul Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+renyUApHgSZF9-nd-a0+OPGharLQLO=mDHcY4_qQ0+noCUVg@mail.gmail.com
M src/backend/replication/logical/relation.c
Replace get_equal_strategy_number_for_am() by get_equal_strategy_number()
commit : a2a475b011cf7c25f5a09574def35b335af844ac
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 10 Dec 2024 12:53:27 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 10 Dec 2024 12:53:27 +0100
get_equal_strategy_number_for_am() gets the equal strategy number for
an AM. This currently only supports btree and hash. In the more
general case, this also depends on the operator class (see for example
GistTranslateStratnum()). To support that, replace this function with
get_equal_strategy_number() that takes an opclass and derives it from
there. (This function already existed before as a static function, so
the signature is kept for simplicity.)
This patch is only a refactoring, it doesn't add support for other
index AMs such as gist. This will be done separately.
Reviewed-by: Paul Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+renyUApHgSZF9-nd-a0+OPGharLQLO=mDHcY4_qQ0+noCUVg@mail.gmail.com
M src/backend/executor/execReplication.c
M src/backend/replication/logical/relation.c
M src/include/executor/executor.h
Improve internal logical replication error for missing equality strategy
commit : 321c287351f707c38c1fa94c1ac89bcc3134ed59
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 10 Dec 2024 12:30:42 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 10 Dec 2024 12:30:42 +0100
This "shouldn't happen", except right now it can with a temporal gist
index (to be fixed soon), because of missing gist support in
get_equal_strategy_number(). But right now, the error is not caught
right away, but instead you get the subsequent error about a "missing
operator 0". This makes the error more accurate.
Author: Paul Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+renyUApHgSZF9-nd-a0+OPGharLQLO=mDHcY4_qQ0+noCUVg@mail.gmail.com
M src/backend/executor/execReplication.c
Fix comments of GUC hooks for timezone_abbreviations
commit : d37e856410d0856cb851e11b2e0191edf6cde527
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 10 Dec 2024 13:02:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 10 Dec 2024 13:02:21 +0900
The GUC assign and check hooks used "assign_timezone_abbreviations",
which was incorrect.
Issue noticed while browsing this area of the code, introduced in
0a20ff54f5e6.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/Z1eV6Y8yk77GZhZI@paquier.xyz
Backpatch-through: 16
M src/backend/commands/variable.c
Fix outdated comment of scram_build_secret()
commit : 7b2690a5713e66c64313cea1cf881da3dcaaae2a
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 10 Dec 2024 12:54:09 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 10 Dec 2024 12:54:09 +0900
This routine documented that "iterations" would use a default value if
set to 0 by the caller. However, the iteration should always be set by
the caller to a value strictly more than 0, as documented by an
assertion.
Oversight in b577743000cd, that has made the iteration count of SCRAM
configurable.
Author: Matheus Alcantara
Discussion: https://postgr.es/m/ac858943-4743-44cd-b4ad-08a0c10cbbc8@gmail.com
Backpatch-through: 16
M src/common/scram-common.c
Include necessary header files in radixtree.h.
commit : 724890ffb75c703afc1e0287f5a66b94c2998799
author : Masahiko Sawada <msawada@postgresql.org>
date : Mon, 9 Dec 2024 13:07:06 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Mon, 9 Dec 2024 13:07:06 -0800
When #include'ing radixtree.h with RT_SHMEM, it could happen to raise
compiler errors due to missing some declarations of types and
functions.
This commit also removes the inclusion of postgres.h since it's
against our usual convention.
Backpatch to v17, where radixtree.h was introduced.
Reviewed-by: Heikki Linnakangas, Álvaro Herrera
Discussion: https://postgr.es/m/CAD21AoCU9YH%2Bb9Rr8YRw7UjmB%3D1zh8GKQkWNiuN9mVhMvkyrRg%40mail.gmail.com
Backpatch-through: 17
M src/include/lib/radixtree.h
Doc: fix incorrect EXPLAIN ANALYZE output for bloom indexes
commit : 36d0229b8ff5907ccff782e9425d1647401b7e99
author : David Rowley <drowley@postgresql.org>
date : Tue, 10 Dec 2024 09:24:43 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 10 Dec 2024 09:24:43 +1300
It looks like the example case was once modified to increase the number
of rows but the EXPLAIN ANALYZE output wasn't updated to reflect that.
Also adjust the text which discusses the index sizes. With the example
table size, the bloom index isn't quite 8 times more space efficient
than the btree indexes.
Discussion: https://postgr.es/m/CAApHDvovx8kQ0=HTt85gFDAwmTJHpCgiSvRmQZ_6u_g-vQYM_w@mail.gmail.com
Backpatch-through: 13, all supported versions
M doc/src/sgml/bloom.sgml
Fix small memory leaks in GUC checks
commit : 73a392d236965f14b84c0d09f011bda2cba6f8ca
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 9 Dec 2024 20:58:23 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 9 Dec 2024 20:58:23 +0100
Follow-up commit to a9d58bfe8a3a. Backpatch down to v16 where
this was added in order to keep the code consistent for future
backpatches.
Author: Tofig Aliev <t.aliev@postgrespro.ru>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/bba4313fdde9db46db279f96f3b748b1@postgrespro.ru
Backpatch-through: 16
M src/backend/commands/variable.c
Fix various overflow hazards in date and timestamp functions.
commit : 0a27c3d0f733c132a4c5122e749a619d02937a8a
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 9 Dec 2024 13:47:23 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 9 Dec 2024 13:47:23 -0600
This commit makes use of the overflow-aware routines in int.h to
fix a variety of reported overflow bugs in the date and timestamp
code. It seems unlikely that this fixes all such bugs in this
area, but since the problems seem limited to cases that are far
beyond any realistic usage, I'm not going to worry too much. Note
that for one bug, I've chosen to simply add a comment about the
overflow hazard because fixing it would require quite a bit of code
restructuring that doesn't seem worth the risk.
Since this is a bug fix, it could be back-patched, but given the
risk of conflicts with the new routines in int.h and the overall
risk/reward ratio of this patch, I've opted not to do so for now.
Fixes bug #18585 (except for the one case that's just commented).
Reported-by: Alexander Lakhin
Author: Matthew Kim, Nathan Bossart
Reviewed-by: Joseph Koshakow, Jian He
Discussion: https://postgr.es/m/31ad2cd1-db94-bdb3-f91a-65ffdb4bef95%40gmail.com
Discussion: https://postgr.es/m/18585-db646741dd649abd%40postgresql.org
M src/backend/utils/adt/date.c
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/timestamp.c
M src/include/common/int.h
M src/test/regress/expected/date.out
M src/test/regress/expected/horology.out
M src/test/regress/sql/date.sql
M src/test/regress/sql/horology.sql
Simplify executor's determination of whether to use parallelism.
commit : 3eea7a0c97e94f9570af87317ce3f6a41eb62768
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Dec 2024 14:38:19 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Dec 2024 14:38:19 -0500
Our parallel-mode code only works when we are executing a query
in full, so ExecutePlan must disable parallel mode when it is
asked to do partial execution. The previous logic for this
involved passing down a flag (variously named execute_once or
run_once) from callers of ExecutorRun or PortalRun. This is
overcomplicated, and unsurprisingly some of the callers didn't
get it right, since it requires keeping state that not all of
them have handy; not to mention that the requirements for it were
undocumented. That led to assertion failures in some corner
cases. The only state we really need for this is the existing
QueryDesc.already_executed flag, so let's just put all the
responsibility in ExecutePlan. (It could have been done in
ExecutorRun too, leading to a slightly shorter patch -- but if
there's ever more than one caller of ExecutePlan, it seems better
to have this logic in the subroutine than the callers.)
This makes those ExecutorRun/PortalRun parameters unnecessary.
In master it seems okay to just remove them, returning the
API for those functions to what it was before parallelism.
Such an API break is clearly not okay in stable branches,
but for them we can just leave the parameters in place after
documenting that they do nothing.
Per report from Yugo Nagata, who also reviewed and tested
this patch. Back-patch to all supported branches.
Discussion: https://postgr.es/m/20241206062549.710dc01cf91224809dd6c0e1@sraoss.co.jp
M contrib/auto_explain/auto_explain.c
M contrib/pg_stat_statements/pg_stat_statements.c
M src/backend/commands/copyto.c
M src/backend/commands/createas.c
M src/backend/commands/explain.c
M src/backend/commands/extension.c
M src/backend/commands/matview.c
M src/backend/commands/portalcmds.c
M src/backend/commands/prepare.c
M src/backend/executor/execMain.c
M src/backend/executor/execParallel.c
M src/backend/executor/functions.c
M src/backend/executor/spi.c
M src/backend/tcop/postgres.c
M src/backend/tcop/pquery.c
M src/include/executor/execdesc.h
M src/include/executor/executor.h
M src/include/tcop/pquery.h
M src/include/utils/portal.h
Remove remants of "snapshot too old"
commit : 4d8275046c36792afb3604677c0a53c8530388ae
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 9 Dec 2024 18:13:03 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 9 Dec 2024 18:13:03 +0200
Remove the 'whenTaken' and 'lsn' fields from SnapshotData. After the
removal of the "snapshot too old" feature, they were never set to a
non-zero value.
This largely reverts commit 3e2f3c2e423, which added the
OldestActiveSnapshot tracking, and the init_toast_snapshot()
function. That was only required for setting the 'whenTaken' and 'lsn'
fields. SnapshotToast is now a constant again, like SnapshotSelf and
SnapshotAny. I kept a thin get_toast_snapshot() wrapper around
SnapshotToast though, to check that you have a registered or active
snapshot. That's still a useful sanity check.
Reviewed-by: Nathan Bossart, Andres Freund, Tom Lane
Discussion: https://www.postgresql.org/message-id/cd4b4f8c-e63a-41c0-95f6-6e6cd9b83f6d@iki.fi
M contrib/amcheck/verify_heapam.c
M src/backend/access/common/toast_internals.c
M src/backend/access/heap/heaptoast.c
M src/backend/storage/ipc/procarray.c
M src/backend/utils/time/snapmgr.c
M src/include/access/genam.h
M src/include/access/toast_internals.h
M src/include/storage/predicate.h
M src/include/utils/snapmgr.h
M src/include/utils/snapshot.h
Avoid unnecessary wrapping for Vars and PHVs
commit : f64ec81a810ebd4649beb6c153844fa9ae1ecffe
author : Richard Guo <rguo@postgresql.org>
date : Mon, 9 Dec 2024 20:38:22 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Mon, 9 Dec 2024 20:38:22 +0900
When pulling up a lateral subquery that is under an outer join, the
current code always wraps a Var or PHV in the subquery's targetlist
into a new PlaceHolderVar if it is a lateral reference to something
outside the subquery. This is necessary when the Var/PHV references
the non-nullable side of the outer join from the nullable side: we
need to ensure that it is evaluated at the right place and hence is
forced to null when the outer join should do so. However, if the
referenced rel is under the same lowest nulling outer join, we can
actually omit the wrapping. That's safe because if the subquery
variable is forced to NULL by the outer join, the lateral reference
variable will come out as NULL too. It could be beneficial to get rid
of such PHVs because they imply lateral dependencies, which force us
to resort to nestloop joins.
This patch leverages the newly introduced nullingrel_info to check if
the nullingrels of the subquery RTE are a subset of those of the
laterally referenced rel, in order to determine if the referenced rel
is under the same lowest nulling outer join.
No backpatch as this could result in plan changes.
Author: Richard Guo
Reviewed-by: James Coleman, Dmitry Dolgov, Andrei Lepikhov
Discussion: https://postgr.es/m/CAMbWs48uk6C7Z9m_FNT8_21CMCk68hrgAsz=z6zpP1PNZMkeoQ@mail.gmail.com
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
Fix right-semi-joins in HashJoin rescans
commit : 5668a857de4f3f12066b2bbc626b77be4fc95ee5
author : Richard Guo <rguo@postgresql.org>
date : Mon, 9 Dec 2024 20:36:23 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Mon, 9 Dec 2024 20:36:23 +0900
When resetting a HashJoin node for rescans, if it is a single-batch
join and there are no parameter changes for the inner subnode, we can
just reuse the existing hash table without rebuilding it. However,
for join types that depend on the inner-tuple match flags in the hash
table, we need to reset these match flags to avoid incorrect results.
This applies to right, right-anti, right-semi, and full joins.
When I introduced "Right Semi Join" plan shapes in aa86129e1, I failed
to reset the match flags in the hash table for right-semi joins in
rescans. This oversight has been shown to produce incorrect results.
This patch fixes it.
Author: Richard Guo
Discussion: https://postgr.es/m/CAMbWs4-nQF9io2WL2SkD0eXvfPdyBc9Q=hRwfQHCGV2usa0jyA@mail.gmail.com
M src/backend/executor/nodeHashjoin.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Fix memory leak in pgoutput with publication list cache
commit : f0c569d7151532fbc9c016b01af49f04a9fb5278
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 9 Dec 2024 16:41:46 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 9 Dec 2024 16:41:46 +0900
The pgoutput module caches publication names in a list and frees it upon
invalidation. However, the code forgot to free the actual publication
names within the list elements, as publication names are pstrdup()'d in
GetPublication(). This would cause memory to leak in
CacheMemoryContext, bloating it over time as this context is not
cleaned.
This is a problem for WAL senders running for a long time, as an
accumulation of invalidation requests would bloat its cache memory
usage. A second case, where this leak is easier to see, involves a
backend calling SQL functions like pg_logical_slot_{get,peek}_changes()
which create a new decoding context with each execution. More
publications create more bloat.
To address this, this commit adds a new memory context within the
logical decoding context and resets it each time the publication names
cache is invalidated, based on a suggestion from Amit Kapila. This
ensures that the lifespan of the publication names aligns with that of
the logical decoding context.
This solution changes PGOutputData, which is fine for HEAD but it could
cause an ABI breakage in stable branches as the structure size would
change, so these are left out for now.
Analyzed-by: Michael Paquier, Jeff Davis
Author: Zhijie Hou
Reviewed-by: Michael Paquier, Masahiko Sawada, Euler Taveira
Discussion: https://postgr.es/m/Z0khf9EVMVLOc_YY@paquier.xyz
M src/backend/replication/pgoutput/pgoutput.c
M src/include/replication/pgoutput.h
Improve comment about dropped entries in pgstat.c
commit : 001a537b83ec6e2ab8fa8af44458b0502c94dd5e
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 9 Dec 2024 14:35:39 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 9 Dec 2024 14:35:39 +0900
pgstat_write_statsfile() discards any entries marked as dropped from
being written to the stats file at shutdown, and also included an
assertion based on the same condition.
The intention of the assertion is to track that no pgstats entries
should be left around as terminating backends should drop any entries
they still hold references on before the stats file is written by the
checkpointer, and it not worth taking down the server in this case if
there is a bug making that possible.
Let's improve the comment of this area to document clearly what's
intended.
Based on a discussion with Bertrand Drouvot and Anton A. Melnikov.
Author: Bertrand Drouvot
Discussion: https://postgr.es/m/a13e8cdf-b97a-4ecb-8f42-aaa367974e29@postgrespro.ru
Backpatch-through: 15
M src/backend/utils/activity/pgstat.c
Improve the error message introduced in commit 87ce27de696.
commit : 2d0152d6147bb7d12eb120b7e749a323bf91aa6e
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 9 Dec 2024 09:11:45 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 9 Dec 2024 09:11:45 +0530
The error detail message "Replica identity consists of an unpublished
generated column." implies that the entire replica identity is made up of
an unpublished generated column which may not be the case.
Reported-by: Peter Smith
Author: Shlok Kyal
Reviewed-by: Peter Smith, Amit Kapila
Discussion: https://postgr.es/m/CAHut+PuwMhKx0PhOA4APhJTLoBGNykbeCQpr_CuwGT-SkswG5w@mail.gmail.com
M src/backend/executor/execReplication.c
M src/test/regress/expected/publication.out
Fix invalidation of local pgstats references for entry reinitialization
commit : da99fedf8c1726326868d37a0e886caf9107d040
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 9 Dec 2024 10:45:28 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 9 Dec 2024 10:45:28 +0900
818119afccd3 has introduced the "generation" concept in pgstats entries,
incremented a counter when a pgstats entry is reinitialized, but it did
not count on the fact that backends still holding local references to
such entries need to be refreshed if the cache age is outdated. The
previous logic only updated local references when an entry was dropped,
but it needs also to consider entries that are reinitialized.
This matters for replication slot stats (as well as custom pgstats kinds
in 18~), where concurrent drops and creates of a slot could cause
incorrect stats to be locally referenced. This would lead to an
assertion failure at shutdown when writing out the stats file, as the
backend holding an outdated local reference would not be able to drop
during its shutdown sequence the stats entry that should be dropped, as
the last process holding a reference to the stats entry. The
checkpointer was then complaining about such an entry late in the
shutdown sequence, after the shutdown checkpoint is finished with the
control file updated, causing the stats file to not be generated. In
non-assert builds, the entry would just be skipped with the stats file
written.
Note that only logical replication slots use statistics.
A test case based on TAP is added to test_decoding, where a persistent
connection peeking at a slot's data is kept with concurrent drops and
creates of the same slot. This is based on the isolation test case that
Anton has sent. As it requires a node shutdown with a check to make
sure that the stats file is written with this specific sequence of
events, TAP is used instead.
Reported-by: Anton A. Melnikov
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/56bf8ff9-dd8c-47b2-872a-748ede82af99@postgrespro.ru
Backpatch-through: 15
M contrib/test_decoding/t/001_repl_stats.pl
M src/backend/utils/activity/pgstat_shmem.c
Fix possible crash during WindowAgg evaluation
commit : 1fe5a347e36f9f5f288b3574597d4e279fc72a65
author : David Rowley <drowley@postgresql.org>
date : Mon, 9 Dec 2024 14:23:21 +1300
committer: David Rowley <drowley@postgresql.org>
date : Mon, 9 Dec 2024 14:23:21 +1300
When short-circuiting WindowAgg node evaluation on the top-level
WindowAgg node using quals on monotonic window functions, because the
WindowAgg run condition can mean there's no need to evaluate subsequent
window function results in the same partition once the run condition
becomes false, it was possible that the executor would use stale results
from the previous invocation of the window function in some cases.
A fix for this was partially done by a5832722, but that commit only
fixed the issue for non-top-level WindowAgg nodes. I mistakenly thought
that the top-level WindowAgg didn't have this issue, but Jayesh's example
case clearly shows that's incorrect. At the time, I also thought that
this only affected 32-bit systems as all window functions which then
supported run conditions returned BIGINT, however, that's wrong as
ExecProject is still called and that could cause evaluation of any other
window function belonging to the same WindowAgg node, one of which may
return a byref type.
The only queries affected by this are WindowAggs with a "Run Condition"
which contains at least one window function with a byref result type,
such as lead() or lag() on a byref column. The window clause must also
contain a PARTITION BY clause (without a PARTITION BY, execution of the
WindowAgg stops immediately when the run condition becomes false and
there's no risk of using the stale results).
Reported-by: Jayesh Dehankar
Discussion: https://postgr.es/m/193261e2c4d.3dd3cd7c1842.871636075166132237@zohocorp.com
Backpatch-through: 15, where WindowAgg run conditions were added
M src/backend/executor/nodeWindowAgg.c
M src/test/regress/expected/window.out
M src/test/regress/sql/window.sql
Ensure that pg_amop/amproc entries depend on their lefttype/righttype.
commit : 3f9b9621766796983c37e192f73c5f8751872c18
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Dec 2024 15:56:28 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Dec 2024 15:56:28 -0500
Usually an entry in pg_amop or pg_amproc does not need a dependency on
its amoplefttype/amoprighttype/amproclefttype/amprocrighttype types,
because there is an indirect dependency via the argument types of its
referenced operator or procedure, or via the opclass it belongs to.
However, for some support procedures in some index AMs, the argument
types of the support procedure might not mention the column data type
at all. Also, the amop/amproc entry might be treated as "loose" in
the opfamily, in which case it lacks a dependency on any particular
opclass; or it might be a cross-type entry having a reference to a
datatype that is not its opclass' opcintype.
The upshot of all this is that there are cases where a datatype can
be dropped while leaving behind amop/amproc entries that mention it,
because there is no path in pg_depend showing that those entries
depend on that type. Such entries are harmless in normal activity,
because they won't get used, but they cause problems for maintenance
actions such as dropping the operator family. They also cause pg_dump
to produce bogus output. The previous commit put a band-aid on the
DROP OPERATOR FAMILY failure, but a real fix is needed.
To fix, add pg_depend entries showing that a pg_amop/pg_amproc entry
depends on its lefttype/righttype. To avoid bloating pg_depend too
much, skip this if the referenced operator or function has that type
as an input type. (I did not bother with considering the possible
indirect dependency via the opclass' opcintype; at least in the
reported case, that wouldn't help anyway.)
Probably, the reason this has escaped notice for so long is that
add-on datatypes and relevant opclasses/opfamilies are usually
packaged as extensions nowadays, so that there's no way to drop
a type without dropping the referencing opclasses/opfamilies too.
Still, in the absence of pg_depend entries there's nothing that
constrains DROP EXTENSION to drop the opfamily entries before the
datatype, so it seems possible for a DROP failure to occur anyway.
The specific case that was reported doesn't fail in v13, because
v13 prefers to attach the support procedure to the opclass not the
opfamily. But it's surely possible to construct other edge cases
that do fail in v13, so patch that too.
Per report from Yoran Heling. Back-patch to all supported branches.
Discussion: https://postgr.es/m/Z1MVCOh1hprjK5Sf@gmai021
M src/backend/commands/opclasscmds.c
M src/include/access/amapi.h
Make getObjectDescription robust against dangling amproc type links.
commit : c82003760d747a64f2decd9664a766485f66121a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Dec 2024 14:28:16 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Dec 2024 14:28:16 -0500
Yoran Heling reported a case where a data type could be dropped
while references to its OID remain behind in pg_amproc. This
causes getObjectDescription to fail, which blocks dropping the
operator family (since our DROP code likes to construct descriptions
of everything it's dropping). The proper fix for this requires
adding more pg_depend entries. But to allow DROP to go through with
already-corrupt catalogs, tweak getObjectDescription to print "???"
for the type instead of failing when it processes such an entry.
I changed the logic for pg_amop similarly, for consistency,
although it is not known that the problem can manifest in pg_amop.
Per report from Yoran Heling. Back-patch to all supported
branches (although the problem may be unreachable in v13).
Discussion: https://postgr.es/m/Z1MVCOh1hprjK5Sf@gmai021
M src/backend/catalog/objectaddress.c
Fix is_digit labeling of to_timestamp's FFn format codes.
commit : 3220ceaf774a2a11f86b33425d836e7072aed677
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Dec 2024 13:12:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Dec 2024 13:12:32 -0500
These format codes produce or consume strings of digits, so they
should be labeled with is_digit = true, but they were not.
This has effect in only one place, where is_next_separator()
is checked to see if the preceding format code should slurp up
all the available digits. Thus, with a format such as '...SSFF3'
with remaining input '12345', the 'SS' code would consume all
five digits (and then complain about seconds being out of range)
when it should eat only two digits.
Per report from Nick Davies. This bug goes back to d589f9446
where the FFn codes were introduced, so back-patch to v13.
Discussion: https://postgr.es/m/AM8PR08MB6356AC979252CFEA78B56678B6312@AM8PR08MB6356.eurprd08.prod.outlook.com
M src/backend/utils/adt/formatting.c
M src/test/regress/expected/horology.out
M src/test/regress/sql/horology.sql
doc: remove LC_COLLATE and LC_CTYPE from SHOW command
commit : 263a3f5f7f508167dbeafc2aefd5835b41d77481
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Dec 2024 12:55:55 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Dec 2024 12:55:55 +0100
The corresponding read-only server settings have been removed since
in PG16. See commit b0f6c437160db6.
Author: Pierre Giraud <pierre.giraud@dalibo.com>
Discussion: https://www.postgresql.org/message-id/flat/a75a2fb0-f4b3-4c0c-be3d-7a62d266d760%40dalibo.com
M doc/src/sgml/ref/show.sgml
Comment fix: "buffer context lock" to "buffer content lock".
commit : ffe003cae1ff28b698f1769e46e3dc14b5516070
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 6 Dec 2024 09:59:12 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 6 Dec 2024 09:59:12 -0800
The term "buffer context lock" is outdated as of commit 5d5087363d.
M src/backend/access/transam/xact.c
Remove useless casts to (const void *)
commit : 8743ea1b2eb3dfbb024f40e8d9952cd85596f552
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 6 Dec 2024 17:22:19 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 6 Dec 2024 17:22:19 +0100
Similar to commit 7f798aca1d5, but I didn't think to look for "const"
as well.
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/lmgr/lock.c
M src/bin/pg_dump/pg_backup_archiver.c
M src/test/modules/test_tidstore/test_tidstore.c
Fix printf format string warning on MinGW.
commit : 1319997df92447d9b473a538b109b2d1d560c637
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 6 Dec 2024 12:34:33 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 6 Dec 2024 12:34:33 +1300
Commit 517bf2d91 changed a printf format string to placate MinGW, which
at the time warned about "%lld". Current MinGW is now warning about the
replacement "%I64d". Reverting the change clears the warning on the
MinGW CI task, and hopefully it will clear it on build farm animal
fairywren too.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
Discussion: https://postgr.es/m/TYAPR01MB5866A71B744BE01B3BF71791F5AEA%40TYAPR01MB5866.jpnprd01.prod.outlook.com
M src/interfaces/ecpg/test/expected/sql-sqlda.c
M src/interfaces/ecpg/test/expected/sql-sqlda.stderr
M src/interfaces/ecpg/test/sql/sqlda.pgc
Remove pg_regex_collation
commit : 792b2c7e6d926e61e8ff3b33d3e22d7d74e7a437
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 5 Dec 2024 07:19:37 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 5 Dec 2024 07:19:37 +0100
We can also use the existing pg_regex_locale as the cache key, which
is the only use of this variable.
Reviewed-by: Jeff Davis <pgsql@j-davis.com>
Discussion: https://www.postgresql.org/message-id/flat/b1b92ae1-2e06-4619-a87a-4b4858e547ec%40eisentraut.org
M src/backend/regex/regc_pg_locale.c
Fix header inclusion order in c.h.
commit : 71cb352904c1833fe067d6f191269710fe2ca06f
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 5 Dec 2024 14:27:35 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 5 Dec 2024 14:27:35 +1300
Commit 962da900a added #include <stdint.h> to postgres_ext.h, which
broke c.h's header ordering rule.
The system headers on some systems would then lock down off_t's size in
private macros, before they'd had a chance to see our definition of
_FILE_OFFSET_BITS (and presumably other things). This was picked up by
perl's ABI compatibility checks on some 32 bit systems in the build
farm.
Move #include "postgres_ext.h" down below the system header section, and
make the comments clearer (thanks to Tom for the new wording).
Diagnosed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/2397643.1733347237%40sss.pgh.pa.us
M src/include/c.h
Provide a better error message for misplaced dispatch options.
commit : 76fd342496612c8432ef8f6c71794c935497d3c9
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 4 Dec 2024 15:04:15 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 4 Dec 2024 15:04:15 -0600
Before this patch, misplacing a special must-be-first option for
dispatching to a subprogram (e.g., postgres -D . --single) would
fail with an error like
FATAL: --single requires a value
This patch adjusts this error to more accurately complain that the
special option wasn't listed first. The aforementioned error
message now looks like
FATAL: --single must be first argument
The dispatch option parsing code has been refactored for use
wherever ParseLongOption() is called. Beyond the obvious advantage
of avoiding code duplication, this should prevent similar problems
when new dispatch options are added. Note that we assume that none
of the dispatch option names match another valid command-line
argument, such as the name of a configuration parameter.
Ideally, we'd remove this must-be-first requirement for these
options, but after some investigation, we decided that wasn't worth
the added complexity and behavior changes.
Author: Nathan Bossart, Greg Sabino Mullane
Reviewed-by: Greg Sabino Mullane, Peter Eisentraut, Álvaro Herrera, Tom Lane
Discussion: https://postgr.es/m/CAKAnmmJkZtZAiSryho%3DgYpbvC7H-HNjEDAh16F3SoC9LPu8rqQ%40mail.gmail.com
M src/backend/bootstrap/bootstrap.c
M src/backend/main/main.c
M src/backend/postmaster/postmaster.c
M src/backend/tcop/postgres.c
M src/include/postmaster/postmaster.h
M src/tools/pgindent/typedefs.list
Return actual error code from FOP failure in PDF build
commit : 24c1c6338719e4de1dd5f045418e64957781c595
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 4 Dec 2024 14:37:24 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 4 Dec 2024 14:37:24 -0500
Previously we returned "1" on error. Improvement on 77c189cdafe.
Backpatch-through: master
M doc/src/sgml/Makefile
Fix dead code
commit : dfbb092cff70ee4d4084b7e4e57228a89ff4f9f8
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 4 Dec 2024 16:43:07 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 4 Dec 2024 16:43:07 +0100
from commit 85b7efa1cdd
per Coverity report
M src/backend/utils/adt/like_match.c
Fix use-after-free in parallel_vacuum_reset_dead_items
commit : ccc8194e4275d0be557a9b94126a12b389520593
author : John Naylor <john.naylor@postgresql.org>
date : Wed, 4 Dec 2024 16:51:55 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Wed, 4 Dec 2024 16:51:55 +0700
parallel_vacuum_reset_dead_items used a local variable to hold a
pointer from the passed vacrel, purely as a shorthand. This pointer
was later freed and a new allocation was made and stored to the
struct. Then the local pointer was mistakenly referenced again.
This apparently happened not to break anything since the freed chunk
would have been put on the context's freelist, so it was accidentally
the same pointer anyway, in which case the DSA handle was correctly
updated. The minimal fix is to change two places so they access
dead_items through the vacrel. This coding style is a maintenance
hazard, so while at it get rid of most other similar usages, which
were inconsistently used anyway.
Analysis and patch by Vallimaharajan G, with further defensive coding
by me
Backpath to v17, when TidStore came in
Discussion: https://postgr.es/m/1936493cc38.68cb2ef27266.7456585136086197135@zohocorp.com
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/vacuumparallel.c
Simplify IsIndexUsableForReplicaIdentityFull()
commit : 7727049e8f663344d4d0457e1d9ec048d626f3d9
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 4 Dec 2024 08:33:28 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 4 Dec 2024 08:33:28 +0100
Take Relation as argument instead of IndexInfo. Building the
IndexInfo is an unnecessary intermediate step here.
A future patch wants to get some information that is in the relcache
but not in IndexInfo, so this will also help there.
Discussion: https://www.postgresql.org/message-id/333d3886-b737-45c3-93f4-594c96bb405d@eisentraut.org
M src/backend/replication/logical/relation.c
M src/backend/replication/logical/worker.c
M src/include/replication/logicalrelation.h
Ensure stored generated columns must be published when required.
commit : 87ce27de6963091f4a365f80bcdb06b9da098f00
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 4 Dec 2024 09:45:18 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 4 Dec 2024 09:45:18 +0530
Ensure stored generated columns that are part of REPLICA IDENTITY must be
published explicitly for UPDATE and DELETE operations to be published. We
can publish generated columns by listing them in the column list or by
enabling the publish_generated_columns option.
This commit changes the behavior of the test added in commit adedf54e65 by
giving an ERROR for the UPDATE operation in such cases. There is no way to
trigger the bug reported in commit adedf54e65 but we didn't remove the
corresponding code change because it is still relevant when replicating
changes from a publisher with version less than 18.
We decided not to backpatch this behavior change to avoid the risk of
breaking existing output plugins that may be sending generated columns by
default although we are not aware of any such plugin. Also, we didn't see
any reports related to this on STABLE branches which is another reason not
to backpatch this change.
Author: Shlok Kyal, Hou Zhijie
Reviewed-by: Vignesh C, Amit Kapila
Discussion: https://postgr.es/m/CANhcyEVw4V2Awe2AB6i0E5AJLNdASShGfdBLbUd1XtWDboymCA@mail.gmail.com
M doc/src/sgml/ref/create_publication.sgml
M src/backend/commands/publicationcmds.c
M src/backend/executor/execReplication.c
M src/backend/utils/cache/relcache.c
M src/include/catalog/pg_publication.h
M src/include/commands/publicationcmds.h
M src/test/regress/expected/publication.out
M src/test/regress/sql/publication.sql
M src/test/subscription/t/100_bugs.pl
Properly use $(AWK) in Makefile, not 'awk'
commit : 77c189cdafe3873b7273149fbc490cc11c431cc3
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Dec 2024 22:31:12 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Dec 2024 22:31:12 -0500
Fix for commit 498f1307569.
Backpatch-through: master
M doc/src/sgml/Makefile
Use <stdint.h> and <inttypes.h> for c.h integers.
commit : 962da900ac8f0927f1af2fd811ca67fa163c873a
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 4 Dec 2024 14:46:59 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 4 Dec 2024 14:46:59 +1300
Redefine our exact width types with standard C99 types and macros,
including int64_t, INT64_MAX, INT64_C(), PRId64 etc. We were already
using <stdint.h> types in a few places.
One complication is that Windows' <inttypes.h> uses format strings like
"%I64d", "%I32", "%I" for PRI*64, PRI*32, PTR*PTR, instead of mapping to
other standardized format strings like "%lld" etc as seen on other known
systems. Teach our snprintf.c to understand them.
This removes a lot of configure clutter, and should also allow 64-bit
numbers and other standard types to be used in localized messages
without casting.
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/ME3P282MB3166F9D1F71F787929C0C7E7B6312%40ME3P282MB3166.AUSP282.PROD.OUTLOOK.COM
M config/c-compiler.m4
M configure
M configure.ac
M meson.build
M src/Makefile.global.in
M src/bin/pg_waldump/pg_waldump.c
M src/bin/pgbench/pgbench.c
M src/include/.gitignore
M src/include/Makefile
M src/include/c.h
M src/include/meson.build
M src/include/pg_config.h.in
D src/include/pg_config_ext.h.in
D src/include/pg_config_ext.h.meson
M src/include/port/pg_bitutils.h
M src/include/postgres_ext.h
M src/include/utils/dsa.h
M src/interfaces/ecpg/ecpglib/typename.c
M src/interfaces/ecpg/include/ecpg_config.h.in
M src/interfaces/ecpg/include/meson.build
M src/interfaces/ecpg/include/pgtypes_interval.h
M src/interfaces/ecpg/include/sqltypes.h
M src/interfaces/ecpg/test/expected/compat_informix-sqlda.c
M src/port/pg_bitutils.c
M src/port/snprintf.c
Define __EXTENSIONS__ on Solaris, too.
commit : 3b08d5224d7df71cc111d8522cf6190fc02f6fb9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Dec 2024 20:21:23 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Dec 2024 20:21:23 -0500
Apparently, if you define _POSIX_C_SOURCE on Solaris,
that's interpreted as "you get ONLY what's defined by POSIX".
Results from BF member hake show that that breaks perl.h,
and doubtless it'd cause more problems if we got past that.
Adopt the suggestion from standards(7) that we also need to
define __EXTENSIONS__, in hopes of un-breaking things.
Discussion: https://postgr.es/m/1654508.1733162761@sss.pgh.pa.us
M configure
M configure.ac
M meson.build
Fix Makefile so invalid characters warning preserves error code
commit : 498f130756914e1bc3125ddfc7e5c86f38ded8e8
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Dec 2024 18:27:41 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Dec 2024 18:27:41 -0500
Fix for commit e4c8865196f.
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/88cb6ecf-22bb-431e-974b-1cd236a80364@eisentraut.org
Backpatch-through: master
M doc/src/sgml/Makefile
Now that we have non-Latin1 SGML detection, restore Latin1 chars
commit : 8b318a168a5830770722c501d7db0607649b3c33
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Dec 2024 17:09:49 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Dec 2024 17:09:49 -0500
This reverts the change in commit 641a5b7a144 that converted them to
HTML entities.
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/Z05ssoVheWI-rqax@momjian.us
Backpatch-through: master
M doc/src/sgml/charset.sgml
M doc/src/sgml/stylesheet-man.xsl
Move check for ucol_strcollUTF8 to pg_locale_icu.c
commit : 7167e05fc7d191bfd59f16d0852705d8f4a3fa08
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 3 Dec 2024 11:32:14 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 3 Dec 2024 11:32:14 -0800
The result of the check is only used by pg_locale_icu.c.
Author: Andreas Karlsson
Discussion: https://postgr.es/m/4548a168-62cd-457b-8d06-9ba7b985c477@proxel.se
M src/backend/utils/adt/pg_locale_icu.c
M src/include/utils/pg_locale.h
Define _POSIX_C_SOURCE as 200112L on Solaris.
commit : 32a7deb2a02bc3c81d61eeb943b03ed2010eaac9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Dec 2024 12:44:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Dec 2024 12:44:43 -0500
This is an attempt to suppress some compiler warnings that appeared in
the wake of commit 7f798aca1: it seems that by default Solaris/illumos
declares shmdt() to take "char *" not "void *". We'd like the system
headers to provide modern POSIX APIs, and POSIX 2001 seems to be as
modern as is available there.
illumos' standards(7) man page suggests that we might also need to
define __EXTENSIONS__, but let's see what happens with just this.
Discussion: https://postgr.es/m/1654508.1733162761@sss.pgh.pa.us
M configure
M configure.ac
M meson.build
Fix synchronized_standby_slots GUC check hook
commit : 3c5f9f12c807760f6d512957a863113b07a79dcb
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 3 Dec 2024 17:50:57 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 3 Dec 2024 17:50:57 +0100
The validate_sync_standby_slots subroutine requires an LWLock, so it
cannot run in processes without PGPROC; skip it there to avoid a crash.
This replaces the current test for ReplicationSlotCtl being not null,
which appears to be a solution for the same problem but less general.
I also rewrote a related comment that mentioned ReplicationSlotCtl in
StandbySlotsHaveCaughtup.
This code came in with commit bf279ddd1c28; backpatch to 17.
Reported-by: Gabriele Bartolini <gabriele.bartolini@enterprisedb.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Zhijie Hou <houzj.fnst@fujitsu.com>
Discussion: https://postgr.es/m/202411281216.sutbxtr6idnn@alvherre.pgsql
M src/backend/replication/slot.c
Drop "Lock" suffix from LWLock wait event names
commit : 1e5ef3a2a17974f574b90b86f491d74e32615987
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 3 Dec 2024 15:50:03 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 3 Dec 2024 15:50:03 +0100
Commit da952b415f44 unintentially reverted the SQL-visible part of
commit 14a910109126, which breaks queries joining pg_wait_events with
pg_stat_acivity. Remove the suffix again.
Backpatch to 17.
Reported-by: Christophe Courtois <christophe.courtois@dalibo.com>
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/18728-450924477056a339%40postgresql.org
Discussion: https://postgr.es/m/Z01w1+LihtRiS0Te@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/storage/lmgr/generate-lwlocknames.pl
M src/backend/storage/lmgr/lwlock.c
Update obsolete comment
commit : 4cc2a449802ec654b1b3ac4db698c3af4e7fd9f7
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 3 Dec 2024 14:46:31 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 3 Dec 2024 14:46:31 +0100
Commit 3aa0395d4ed3 made worrying about BKI_ROWTYPE_OID matching no
longer necessary, but this comment didn't get the memo.
M src/include/catalog/pg_class.dat
Fix handling of CREATE DOMAIN with GENERATED constraint syntax
commit : 84a67725cd11ffbd5c0e142b067ae90dc3e57270
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 3 Dec 2024 14:32:45 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 3 Dec 2024 14:32:45 +0100
Stuff like
CREATE DOMAIN foo AS int CONSTRAINT cc GENERATED ALWAYS AS (2) STORED
is not supported for domains, but the parser allows it, because it's
the same syntax as for table constraints. But CreateDomain() did not
explicitly handle all ConstrType values, so the above would get an
internal error like
ERROR: unrecognized constraint subtype: 4
Fix that by providing a user-facing error message for all ConstrType
values. Also, remove the switch default case, so future additions to
ConstrType are caught.
Reported-by: Jian He <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/CACJufxF8fmM=Dbm4pDFuV_nKGz2-No0k4YifhrF3-rjXTWJM3w@mail.gmail.com
M src/backend/commands/typecmds.c
Fix temporary memory leak in system table index scans
commit : 1acf10549e64c6a52ced570d712fcba1a2f5d1ec
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 3 Dec 2024 09:04:20 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 3 Dec 2024 09:04:20 +0100
Commit 811af9786b introduced palloc() calls into systable_beginscan()
and systable_beginscan_ordered(). But there was no pfree(), as is the
usual style.
It turns out that an ANALYZE of a partitioned table can invoke many
thousand system table index scans, and this memory is not cleaned up
until the end of the command, so this can temporarily leak quite a bit
of memory. Maybe there are improvements to be made at a higher level
about this, but for now, insert a couple of corresponding pfree()
calls to fix this particular issue.
Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/Z0XTfIq5xUtbkiIh@pryzbyj2023
M src/backend/access/index/genam.c
Perform provider-specific initialization in new functions.
commit : 1ba0782ce90cb4261098de59b49ae5cb2326566b
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 2 Dec 2024 23:20:32 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 2 Dec 2024 23:20:32 -0800
Reviewed-by: Andreas Karlsson
Discussion: https://postgr.es/m/4548a168-62cd-457b-8d06-9ba7b985c477@proxel.se
M src/backend/utils/adt/Makefile
M src/backend/utils/adt/meson.build
M src/backend/utils/adt/pg_locale.c
A src/backend/utils/adt/pg_locale_builtin.c
M src/backend/utils/adt/pg_locale_icu.c
M src/backend/utils/adt/pg_locale_libc.c
doc: Clarify some terms for pg_createsubscriber
commit : 8817e8d3a4e5f00a5583e56a17f5d49a4e21dafd
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 3 Dec 2024 16:21:07 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 3 Dec 2024 16:21:07 +0900
The last section of pg_createsubscriber used the terms
"publication-name", "replication-slot-name", and "subscription-name".
These terms are not defined on the page, which was confusing, and the
intention is clearly to refer to the values one would give to the
options --publication, --subscription and --replication-slot. Let's
simplify the documentation by mentioning the option switches, instead of
these terms.
Reported-by: Christophe Courtois
Author: Shubham Khanna
Reviewed-by: Vignesh C, Peter Smith
Discussion: https://postgr.es/m/173288198026.714.15127074046508836738@wrigleys.postgresql.org
Backpatch-through: 17
M doc/src/sgml/ref/pg_createsubscriber.sgml
Fix unintentional behavior change in commit e9931bfb75.
commit : e3fa2b037c6f0f435838e99200050dc54c306085
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 2 Dec 2024 21:59:02 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 2 Dec 2024 21:59:02 -0800
Prior to that commit, there was special case to use ASCII case mapping
behavior for the libc provider with a single-byte encoding when that's
the default collation. Commit e9931bfb75 mistakenly eliminated that
special case; this commit restores it.
Discussion: https://postgr.es/m/01a104f0d2179d756261e90d96fd65c36ad6fcf0.camel@j-davis.com
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/like.c
M src/backend/utils/adt/pg_locale.c
M src/include/utils/pg_locale.h
Revert "Introduce CompactAttribute array in TupleDesc"
commit : 4171c44c9b791da3c00386dc6d8e6b1842e3036b
author : David Rowley <drowley@postgresql.org>
date : Tue, 3 Dec 2024 17:12:38 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 3 Dec 2024 17:12:38 +1300
This reverts commit d28dff3f6cd6a7562fb2c211ac0fb74a33ffd032.
Quite a large number of buildfarm members didn't like this commit and
it's not yet clear why. Reverting this before too many animals turn
red.
Discussion: https://postgr.es/m/CAApHDvr9i6T5=iAwQCxFDgMsthr_obVxgwBaEJkC8KUH6yM3Hw@mail.gmail.com
M src/backend/access/common/heaptuple.c
M src/backend/access/common/indextuple.c
M src/backend/access/common/tupdesc.c
M src/backend/access/spgist/spgutils.c
M src/backend/catalog/index.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execTuples.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/cache/typcache.c
M src/include/access/htup_details.h
M src/include/access/itup.h
M src/include/access/tupdesc.h
M src/include/access/tupmacs.h
M src/tools/pgindent/typedefs.list
Introduce CompactAttribute array in TupleDesc
commit : d28dff3f6cd6a7562fb2c211ac0fb74a33ffd032
author : David Rowley <drowley@postgresql.org>
date : Tue, 3 Dec 2024 16:50:59 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 3 Dec 2024 16:50:59 +1300
The new compact_attrs array stores a few select fields from
FormData_pg_attribute in a more compact way, using only 16 bytes per
column instead of the 104 bytes that FormData_pg_attribute uses. Using
CompactAttribute allows performance-critical operations such as tuple
deformation to be performed without looking at the FormData_pg_attribute
element in TupleDesc which means fewer cacheline accesses. With this
change, NAMEDATALEN could be increased with a much smaller negative impact
on performance.
For some workloads, tuple deformation can be the most CPU intensive part
of processing the query. Some testing with 16 columns on a table
where the first column is variable length showed around a 10% increase in
transactions per second for an OLAP type query performing aggregation on
the 16th column. However, in certain cases, the increases were much
higher, up to ~25% on one AMD Zen4 machine.
This also makes pg_attribute.attcacheoff redundant. A follow-on commit
will remove it, thus shrinking the FormData_pg_attribute struct by 4
bytes.
Author: David Rowley
Discussion: https://postgr.es/m/CAApHDvrBztXP3yx=NKNmo3xwFAFhEdyPnvrDg3=M0RhDs+4vYw@mail.gmail.com
Reviewed-by: Andres Freund, Victor Yegorov
M src/backend/access/common/heaptuple.c
M src/backend/access/common/indextuple.c
M src/backend/access/common/tupdesc.c
M src/backend/access/spgist/spgutils.c
M src/backend/catalog/index.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execTuples.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/cache/typcache.c
M src/include/access/htup_details.h
M src/include/access/itup.h
M src/include/access/tupdesc.h
M src/include/access/tupmacs.h
M src/tools/pgindent/typedefs.list
doc Makefile: issue warning about chars that cannot be output
commit : e4c8865196f6ad6bb3473bcad1d2ad51147e4513
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 2 Dec 2024 21:25:12 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 2 Dec 2024 21:25:12 -0500
A follow-up improvement to commit 641a5b7a144.
Reported-by: Tatsuo Ishii, Tom Lane
Discussion: https://postgr.es/m/20241126.182513.1752581942460106099.ishii@postgresql.org
Backpatch-through: master
M doc/src/sgml/Makefile
Rework some code handling pg_subscription data in psql and pg_dump
commit : 08691ea958c2646b6aadefff878539eb0b860bb0
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 3 Dec 2024 09:48:12 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 3 Dec 2024 09:48:12 +0900
This commit fixes some inconsistencies found in the frontend code when
dealing with subscription catalog data.
The following changes are done:
- pg_subscription.h gains a EXPOSE_TO_CLIENT_CODE, so as more content
defined in pg_subscription.h becomes available in pg_subscription_d.h
for the frontend.
- In psql's describe.c, substream can be switched to use CppAsString2()
with its three LOGICALREP_STREAM_* values, with pg_subscription_d.h
included.
- pg_dump.c included pg_subscription.h, which is a header that should
only be used in the backend code. The code is updated to use
pg_subscription_d.h instead.
- pg_dump stored all the data from pg_subscription in SubscriptionInfo
with only strings, and a good chunk of them are boolean and char values.
Using strings is not necessary, complicates the code (see for example
two_phase_disabled[] removed here), and is inconsistent with the way
other catalogs' data is handled. The fields of SubscriptionInfo are
reordered to match with the order in its catalog, while on it.
Reviewed-by: Hayato Kuroda
Discussion: https://postgr.es/m/Z0lB2kp0ksHgmVuk@paquier.xyz
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/psql/describe.c
M src/include/catalog/pg_subscription.h
RelationTruncate() must set DELAY_CHKPT_START.
commit : 75818b3afbf850d600e0fcd1a3b03199077063f8
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 3 Dec 2024 09:27:05 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 3 Dec 2024 09:27:05 +1300
Previously, it set only DELAY_CHKPT_COMPLETE. That was important,
because it meant that if the XLOG_SMGR_TRUNCATE record preceded a
XLOG_CHECKPOINT_ONLINE record in the WAL, then the truncation would also
happen on disk before the XLOG_CHECKPOINT_ONLINE record was
written.
However, it didn't guarantee that the sync request for the truncation
was processed before the XLOG_CHECKPOINT_ONLINE record was written. By
setting DELAY_CHKPT_START, we guarantee that if an XLOG_SMGR_TRUNCATE
record is written to WAL before the redo pointer of a concurrent
checkpoint, the sync request queued by that operation must be processed
by that checkpoint, rather than being left for the following one.
This is a refinement of commit 412ad7a5563. Back-patch to all supported
releases, like that commit.
Author: Robert Haas <robertmhaas@gmail.com>
Reported-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKG%2B-2rjGZC2kwqr2NMLBcEBp4uf59QT1advbWYF_uc%2B0Aw%40mail.gmail.com
M src/backend/catalog/storage.c
Deprecate MD5 passwords.
commit : db6a4a985bc09d260d5c29848e3c97f080646a53
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 2 Dec 2024 13:30:07 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 2 Dec 2024 13:30:07 -0600
MD5 has been considered to be unsuitable for use as a cryptographic
hash algorithm for some time. Furthermore, MD5 password hashes in
PostgreSQL are vulnerable to pass-the-hash attacks, i.e., knowing
the username and hashed password is sufficient to authenticate.
The SCRAM-SHA-256 method added in v10 is not subject to these
problems and is considered to be superior to MD5.
This commit marks MD5 password support in PostgreSQL as deprecated
and to be removed in a future release. The documentation now
contains several deprecation notices, and CREATE ROLE and ALTER
ROLE now emit deprecation warnings when setting MD5 passwords. The
warnings can be disabled by setting the md5_password_warnings
parameter to "off".
Reviewed-by: Greg Sabino Mullane, Jim Nasby
Discussion: https://postgr.es/m/ZwbfpJJol7lDWajL%40nathan
M contrib/passwordcheck/expected/passwordcheck.out
M contrib/passwordcheck/expected/passwordcheck_1.out
M contrib/passwordcheck/sql/passwordcheck.sql
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/create_role.sgml
M doc/src/sgml/runtime.sgml
M src/backend/libpq/crypt.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/libpq/crypt.h
M src/test/regress/expected/password.out
M src/test/regress/expected/password_1.out
Add a planner support function for numeric generate_series().
commit : 97173536ed4b1c29dce0dc4119db136e142f60a2
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Mon, 2 Dec 2024 11:37:57 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Mon, 2 Dec 2024 11:37:57 +0000
This allows the planner to estimate the number of rows returned by
generate_series(numeric, numeric[, numeric]), when the input values
can be estimated at plan time.
Song Jinzhou, reviewed by Dean Rasheed and David Rowley.
Discussion: https://postgr.es/m/tencent_F43E7F4DD50EF5986D1051DE8DE547910206%40qq.com
Discussion: https://postgr.es/m/tencent_1F6D5B9A1545E02FD7D0EE508DFD056DE50A%40qq.com
M src/backend/utils/adt/numeric.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/misc_functions.out
M src/test/regress/sql/misc_functions.sql
Fix #include order in timestamp.c.
commit : 33152358453cc46f986483e9f82418fb8ba8eaef
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Mon, 2 Dec 2024 11:34:26 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Mon, 2 Dec 2024 11:34:26 +0000
Oversight in 036bdcec9f.
M src/backend/utils/adt/timestamp.c
Fix error code for referential action RESTRICT
commit : 086c84b23d99c2ad268f97508cd840efc1fdfd79
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 2 Dec 2024 08:18:36 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 2 Dec 2024 08:18:36 +0100
According to the SQL standard, if the referential action RESTRICT is
triggered, it has its own error code. We previously didn't use that,
we just used the error code for foreign key violation. But RESTRICT
is not necessarily an actual foreign key violation. The foreign key
might still be satisfied in theory afterwards, but the RESTRICT
setting prevents the action even then. So it's a separate kind of
error condition.
Discussion: https://www.postgresql.org/message-id/ea5b2777-266a-46fa-852f-6fca6ec480ad@eisentraut.org
M src/backend/utils/adt/ri_triggers.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/expected/foreign_key.out
M src/test/regress/expected/without_overlaps.out
Fix broken list-munging in ecpg's remove_variables().
commit : 2f696453d2b39fea800d5f7d8e5d3e1a2266de24
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Dec 2024 14:15:37 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Dec 2024 14:15:37 -0500
The loops over cursor argument variables neglected to ever advance
"prevvar". The code would accidentally do the right thing anyway
when removing the first or second list entry, but if it had to
remove the third or later entry then it would also remove all
entries between there and the first entry. AFAICS this would
only matter for cursors that reference out-of-scope variables,
which is a weird Informix compatibility hack; between that and
the lack of impact for short lists, it's not so surprising that
nobody has complained. Nonetheless it's a pretty obvious bug.
It would have been more obvious if these loops used a more standard
coding style for chasing the linked lists --- this business with the
"prev" pointer sometimes pointing at the current list entry is
confusing and overcomplicated. So rather than just add a minimal
band-aid, I chose to rewrite the loops in the same style we use
elsewhere, where the "prev" pointer is NULL until we are dealing with
a non-first entry and we save the "next" pointer at the top of the
loop. (Two of the four loops touched here are not actually buggy,
but it seems better to make them all look alike.)
Coverity discovered this problem, but not until 2b41de4a5 added code
to free no-longer-needed arguments structs. With that, the incorrect
link updates are possibly touching freed memory, and it complained
about that. Nonetheless the list corruption hazard is ancient, so
back-patch to all supported branches.
M src/interfaces/ecpg/preproc/variable.c
Avoid mislabeling of lateral references, redux.
commit : e032e4c7ddd0e1f7865b246ec18944365d4f8614
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 30 Nov 2024 12:42:19 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 30 Nov 2024 12:42:19 -0500
As I'd feared, commit 5c9d8636d was still a few bricks shy of a load.
We can't just leave pulled-up lateral-reference Vars with no new
nullingrels: we have to carefully compute what subset of the
to-be-replaced Var's nullingrels apply to them, else we still get
"wrong varnullingrels" errors. This is a bit tedious, but it looks
like we can use the nullingrel data this patch computes for other
purposes, enabling better optimization. We don't want to inject
unnecessary plan changes into stable branches though, so leave that
idea for a later HEAD-only patch.
Patch by me, but thanks to Richard Guo for devising a test case that
broke 5c9d8636d, and for preliminary investigation about how to fix
it. As before, back-patch to v16.
Discussion: https://postgr.es/m/E1tGn4j-0003zi-MP@gemulon.postgresql.org
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
M src/tools/pgindent/typedefs.list
doc: Fix typo
commit : 49ae9fd8b7081c075b74889449b29d47fd80eaf3
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 30 Nov 2024 08:43:46 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 30 Nov 2024 08:43:46 +0100
for commit 1e08905842f
Reported-by: Marcos Pegoraro <marcos@f10.com.br>
M doc/src/sgml/ddl.sgml
Small indenting fixes in jsonpath_scan.l
commit : 5d39becf8ba0080c98fee4b63575552f6800b012
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 29 Nov 2024 11:33:21 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 29 Nov 2024 11:33:21 +0100
Some lines were indented by an inconsistent number of spaces. While
we're here, also fix some code that used the newline after left
parenthesis style, which is obsolete.
M src/backend/utils/adt/jsonpath_scan.l
doc: Improve description of referential actions
commit : 1e08905842fbfa9301374f10571116bc5f0f7b8a
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 29 Nov 2024 08:52:28 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 29 Nov 2024 08:52:28 +0100
Some of the differences between NO ACTION and RESTRICT were not
explained fully.
Discussion: https://www.postgresql.org/message-id/ea5b2777-266a-46fa-852f-6fca6ec480ad@eisentraut.org
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/create_table.sgml
Add tests for foreign keys with case-insensitive collations
commit : 4a2dbfc6be45c4ce2d9ac53502f931e6b1d70318
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 29 Nov 2024 08:52:27 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 29 Nov 2024 08:52:27 +0100
Some of the behaviors of the different referential actions, such as
the difference between NO ACTION and RESTRICT are best illustrated
using a case-insensitive collation. So add some tests for that.
(What is actually being tested here is the behavior with values that
are "distinct" (binary different) but compare as equal. Another way
to do that would be with positive and negative zeroes with float
types. But this way seems nicer and more flexible.)
Discussion: https://www.postgresql.org/message-id/ea5b2777-266a-46fa-852f-6fca6ec480ad@eisentraut.org
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Skip not SOAP-supported indexes while transforming an OR clause into SAOP
commit : 5bba0546eecb32f4ff9388815727304823940ef6
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 29 Nov 2024 09:48:29 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 29 Nov 2024 09:48:29 +0200
There is no point in transforming OR-clauses into SAOP's if the target index
doesn't support SAOP scans anyway. This commit adds corresponding checks
to match_orclause_to_indexcol() and group_similar_or_args(). The first check
fixes the actual bug, while the second just saves some cycles.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/8174de69-9e1a-0827-0e81-ef97f56a5939%40gmail.com
Author: Alena Rybakina
Reviewed-by: Ranier Vilela, Alexander Korotkov, Andrei Lepikhov
M src/backend/optimizer/path/indxpath.c
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Fix typo in header comment for set_operation_ordered_results_useful
commit : b6612aedc53a6bf069eba5e356a8421ad6426486
author : David Rowley <drowley@postgresql.org>
date : Fri, 29 Nov 2024 15:56:24 +1300
committer: David Rowley <drowley@postgresql.org>
date : Fri, 29 Nov 2024 15:56:24 +1300
Reported-by: Richard Guo
Discussion: https://postgr.es/m/CAMbWs492vMy3XNjDZRtqtHfFTK6HVeDwhrEQH7eXGgF_h5Jnzw@mail.gmail.com
M src/backend/optimizer/prep/prepunion.c
psql: Sprinkle more CppAsString2() in describe.c
commit : 18954ce7f69b0a4725b1cf1845ed5fd3874e2a92
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 29 Nov 2024 08:53:09 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 29 Nov 2024 08:53:09 +0900
Like 91f5a4a000ea for pg_amcheck, this makes the code more
self-documented as there is less need to look in the headers what a
hardcoded value means. This touches queries related to procedures, AMs,
functions, databases, relations, constraints, collations, types and
extended stats, pulling into psql their *_d.h headers. The queries are
written the same way as originally.
There are still a couple of hardcoded values. These cannot be included
yet as they are not exposed in headers that are safe to use in frontend
code.
Note that describe.c was including pg_am.h that should be used only in
backend code. This is updated to use pg_am_d.h.
Reviewed-by: Daniel Gustafsson, Corey Huinker
Discussion: https://postgr.es/m/Zxb2hpca-pZc6zKe@paquier.xyz
M src/bin/psql/describe.c
Avoid mislabeling of lateral references when pulling up a subquery.
commit : 5c9d8636d30d0838c7432c2c75096b4cd7801971
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Nov 2024 17:33:16 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Nov 2024 17:33:16 -0500
If we are pulling up a subquery that's under an outer join, and
the subquery's target list contains a strict expression that uses
both a subquery variable and a lateral-reference variable, it's okay
to pull up the expression without wrapping it in a PlaceHolderVar.
That's safe because if the subquery variable is forced to NULL
by the outer join, the expression result will come out as NULL too,
so we don't have to force that outcome by evaluating the expression
below the outer join. It'd be correct to wrap in a PHV, but that can
lead to very significantly worse plans, since we'd then have to use
a nestloop plan to pass down the lateral reference to where the
expression will be evaluated.
However, when we do that, we should not mark the lateral reference
variable as being nulled by the outer join, because it isn't after
we pull up the expression in this way. So the marking logic added
by cb8e50a4a was incorrect in this detail, leading to "wrong
varnullingrels" errors from the consistency-checking logic in
setrefs.c. It seems to be sufficient to just not mark lateral
references at all in this case. (I have a nagging feeling that more
complexity may be needed in cases where there are several levels of
outer join, but some attempts to break it with that didn't succeed.)
Per report from Bertrand Mamasam. Back-patch to v16, as the previous
patch was.
Discussion: https://postgr.es/m/CACZ67_UA_EVrqiFXJu9XK50baEpH=ofEPJswa2kFxg6xuSw-ww@mail.gmail.com
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
Fix wording in comment
commit : 0c01f509a38f5b1f5fab26d98ae55d796541c2e4
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 28 Nov 2024 15:17:49 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 28 Nov 2024 15:17:49 +0100
Author: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/CAHut+PvE+2T2etdTaHi3n+xbCG_UYrshQuCbaAdJCFPpQGLwgQ@mail.gmail.com
M src/include/replication/reorderbuffer.h
psql: Add tab completion for COPY (MERGE ...
commit : 25ec329afa175a3a27d0d49ef5214d9249421de5
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 28 Nov 2024 09:14:41 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 28 Nov 2024 09:14:41 +0100
The underlying feature for this was added in PostgreSQL 17.
Author: Jian He <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/CACJufxEmNjxvf1deR1zBrJbjAMeCooooLRzZ+yaaBuqDKh_6-Q@mail.gmail.com
M src/bin/psql/tab-complete.in.c
Remove useless casts to (void *)
commit : 7f798aca1d5df290aafad41180baea0ae311b4ee
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 28 Nov 2024 08:19:22 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 28 Nov 2024 08:19:22 +0100
Many of them just seem to have been copied around for no real reason.
Their presence causes (small) risks of hiding actual type mismatches
or silently discarding qualifiers
Discussion: https://www.postgresql.org/message-id/flat/461ea37c-8b58-43b4-9736-52884e862820@eisentraut.org
M contrib/amcheck/verify_nbtree.c
M contrib/bloom/blinsert.c
M contrib/bloom/blutils.c
M contrib/btree_gist/btree_bit.c
M contrib/btree_gist/btree_bool.c
M contrib/btree_gist/btree_bytea.c
M contrib/btree_gist/btree_cash.c
M contrib/btree_gist/btree_date.c
M contrib/btree_gist/btree_enum.c
M contrib/btree_gist/btree_float4.c
M contrib/btree_gist/btree_float8.c
M contrib/btree_gist/btree_inet.c
M contrib/btree_gist/btree_int2.c
M contrib/btree_gist/btree_int4.c
M contrib/btree_gist/btree_int8.c
M contrib/btree_gist/btree_interval.c
M contrib/btree_gist/btree_macaddr.c
M contrib/btree_gist/btree_macaddr8.c
M contrib/btree_gist/btree_numeric.c
M contrib/btree_gist/btree_oid.c
M contrib/btree_gist/btree_text.c
M contrib/btree_gist/btree_time.c
M contrib/btree_gist/btree_ts.c
M contrib/btree_gist/btree_uuid.c
M contrib/file_fdw/file_fdw.c
M contrib/hstore/hstore_op.c
M contrib/intarray/_int_bool.c
M contrib/ltree/_ltree_gist.c
M contrib/ltree/_ltree_op.c
M contrib/pg_trgm/trgm_gist.c
M contrib/pgcrypto/openssl.c
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_minmax_multi.c
M src/backend/access/common/detoast.c
M src/backend/access/gin/ginbulk.c
M src/backend/access/gin/gininsert.c
M src/backend/access/gist/gist.c
M src/backend/access/gist/gistbuild.c
M src/backend/access/hash/hash.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/access/spgist/spginsert.c
M src/backend/access/spgist/spgutils.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xlogrecovery.c
M src/backend/catalog/dependency.c
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/objectaccess.c
M src/backend/catalog/pg_proc.c
M src/backend/catalog/pg_publication.c
M src/backend/commands/copyfrom.c
M src/backend/commands/foreigncmds.c
M src/backend/commands/indexcmds.c
M src/backend/commands/lockcmds.c
M src/backend/commands/policy.c
M src/backend/commands/publicationcmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/tablespace.c
M src/backend/commands/typecmds.c
M src/backend/commands/vacuum.c
M src/backend/commands/variable.c
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/execIndexing.c
M src/backend/executor/functions.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeSort.c
M src/backend/executor/nodeWindowAgg.c
M src/backend/libpq/hba.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/params.c
M src/backend/nodes/read.c
M src/backend/optimizer/geqo/geqo_main.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepagg.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/optimizer/util/appendinfo.c
M src/backend/optimizer/util/clauses.c
M src/backend/optimizer/util/predtest.c
M src/backend/optimizer/util/tlist.c
M src/backend/optimizer/util/var.c
M src/backend/parser/analyze.c
M src/backend/parser/parse_agg.c
M src/backend/parser/parse_collate.c
M src/backend/parser/parse_cte.c
M src/backend/parser/parse_node.c
M src/backend/parser/parse_param.c
M src/backend/parser/scan.l
M src/backend/partitioning/partprune.c
M src/backend/port/sysv_shmem.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/slot.c
M src/backend/replication/syncrep.c
M src/backend/rewrite/rewriteDefine.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/rewrite/rewriteManip.c
M src/backend/storage/buffer/buf_table.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/ipc/shmem.c
M src/backend/tcop/postgres.c
M src/backend/tsearch/dict_thesaurus.c
M src/backend/tsearch/spell.c
M src/backend/tsearch/ts_locale.c
M src/backend/tsearch/ts_parse.c
M src/backend/tsearch/wparser.c
M src/backend/utils/adt/acl.c
M src/backend/utils/adt/array_userfuncs.c
M src/backend/utils/adt/arrayfuncs.c
M src/backend/utils/adt/datetime.c
M src/backend/utils/adt/datum.c
M src/backend/utils/adt/domains.c
M src/backend/utils/adt/enum.c
M src/backend/utils/adt/expandedrecord.c
M src/backend/utils/adt/jsonb.c
M src/backend/utils/adt/jsonfuncs.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/adt/jsonpath_scan.l
M src/backend/utils/adt/lockfuncs.c
M src/backend/utils/adt/multirangetypes.c
M src/backend/utils/adt/orderedsetaggs.c
M src/backend/utils/adt/partitionfuncs.c
M src/backend/utils/adt/rangetypes.c
M src/backend/utils/adt/regexp.c
M src/backend/utils/adt/tsgistidx.c
M src/backend/utils/adt/tsrank.c
M src/backend/utils/adt/tsvector_op.c
M src/backend/utils/adt/xml.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/error/elog.c
M src/backend/utils/hash/dynahash.c
M src/backend/utils/mmgr/mcxt.c
M src/backend/utils/sort/tuplesortvariants.c
M src/backend/utils/sort/tuplestore.c
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_custom.c
M src/bin/pg_dump/pg_backup_directory.c
M src/bin/pg_dump/pg_backup_tar.c
M src/bin/psql/mainloop.c
M src/bin/psql/startup.c
M src/include/common/pg_prng.h
M src/interfaces/ecpg/ecpglib/connect.c
M src/interfaces/ecpg/ecpglib/descriptor.c
M src/interfaces/ecpg/ecpglib/memory.c
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_handler.c
M src/pl/plpython/plpy_exec.c
M src/test/isolation/isolationtester.c
M src/test/modules/test_regex/test_regex.c
Require sizeof(bool) == 1.
commit : 97525bc5c8ffb31475d23955d08e9ec9c1408f33
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 28 Nov 2024 11:48:07 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 28 Nov 2024 11:48:07 +1300
The C standard says that sizeof(bool) is implementation-defined, but we
know of no current systems where it is not 1. The last known systems
seem to have been Apple macOS/PowerPC 10.5 and Microsoft Visual C++ 4,
both long defunct.
PostgreSQL has always required sizeof(bool) == 1 for the definition of
bool that it used, but previously it would define its own type if the
system-provided bool had a different size. That was liable to cause
memory layout problems when interacting with system and third-party
libraries on (by now hypothetical) computers with wider _Bool, and now
C23 has introduced a new problem by making bool a built-in datatype
(like C++), so the fallback code doesn't even compile. We could
probably work around that, but then we'd be writing new untested code
for a computer that doesn't exist.
Instead, delete the unreachable and C23-uncompilable fallback code, and
let existing static assertions fail if the system-provided bool is too
wide. If we ever get a problem report from a real system, then it will
be time to figure out what to do about it in a way that also works on
modern compilers.
Note on C++: Previously we avoided including <stdbool.h> or trying to
define a new bool type in headers that might be included by C++ code.
These days we might as well just include <stdbool.h> unconditionally:
it should be visible to C++11 but do nothing, just as in C23. We
already include <stdint.h> without C++ guards in c.h, and that falls
under the same C99-compatibility section of the C++11 standard as
<stdbool.h>, so let's remove the guards here too.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/3198438.1731895163%40sss.pgh.pa.us
M configure
M configure.ac
M meson.build
M src/backend/jit/llvm/llvmjit_types.c
M src/backend/utils/fmgr/dfmgr.c
M src/include/c.h
M src/include/pg_config.h.in
M src/interfaces/ecpg/include/ecpg_config.h.in
M src/interfaces/ecpg/include/ecpglib.h
M src/interfaces/ecpg/include/meson.build
M src/pl/plperl/plperl_system.h
Use __attribute__((target(...))) for SSE4.2 CRC-32C support.
commit : 4b03a27fafc98e2a34e4e0b5ca44895211e021cc
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 27 Nov 2024 16:19:05 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 27 Nov 2024 16:19:05 -0600
Presently, we check for compiler support for the required
intrinsics both with and without the -msse4.2 compiler flag, and
then depending on the results of those checks, we pick which files
to compile with which flags. This is tedious and complicated, and
it results in unsustainable coding patterns such as separate files
for each portion of code that may need to be built with different
compiler flags.
This commit makes use of the newly-added support for
__attribute__((target(...))) in the SSE4.2 CRC-32C code. This
simplifies both the configure-time checks and the build scripts,
and it allows us to place the functions that use the intrinsics in
files that we otherwise do not want to build with special CPU
instructions (although this commit refrains from doing so). This
is also preparatory work for a proposed follow-up commit that will
further optimize the CRC-32C code with AVX-512 instructions.
While at it, this commit modifies meson's checks for SSE4.2 CRC
support to be the same as autoconf's. meson was choosing whether
to use a runtime check based purely on whether -msse4.2 is
required, while autoconf has long checked for the __SSE4_2__
preprocessor symbol to decide. meson's previous approach seems to
work just fine, but this change avoids needing to build multiple
test programs and to keep track of whether to actually use
pg_attribute_target().
Ideally we'd use __attribute__((target(...))) for ARMv8 CRC
support, too, but there's little point in doing so because until
clang 16, using the ARM intrinsics still requires special compiler
flags. Perhaps we can re-evaluate this decision after some time
has passed.
Author: Raghuveer Devulapalli
Discussion: https://postgr.es/m/PH8PR11MB8286BE735A463468415D46B5FB5C2%40PH8PR11MB8286.namprd11.prod.outlook.com
M config/c-compiler.m4
M configure
M configure.ac
M meson.build
M src/port/Makefile
M src/port/meson.build
M src/port/pg_crc32c_sse42.c
Make GUC_check_errdetail messages full sentences
commit : 6ba9892f5cb8c2f1c2592198d938cc8f5cf52edc
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 27 Nov 2024 19:46:06 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 27 Nov 2024 19:46:06 +0100
They were all missing punctuation, one was missing initial capital.
Per our message style guidelines.
No backpatch, to avoid breaking existing translations.
M src/backend/access/transam/slru.c
M src/backend/access/transam/xlogrecovery.c
M src/backend/commands/vacuum.c
M src/backend/replication/slot.c
M src/backend/replication/syncrep.c
M src/backend/storage/file/fd.c
M src/backend/utils/error/elog.c
Remove redundant relam initialization
commit : fd9924542bccc488731361861903e3f9f3cf11fa
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 27 Nov 2024 19:13:37 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 27 Nov 2024 19:13:37 +0100
This struct member is initialized again a few lines below in the same
function. This is cosmetic, so no backpatch.
Reported-by: Jingtang Zhang <mrdrivingduck@gmail.com>
Discussion: https://postgr.es/m/AFF74506-B925-46BB-B875-CF5A946170EB@gmail.com
M src/backend/utils/cache/relcache.c
ecpg: clean up some other assorted memory leaks.
commit : 2b41de4a5b429c53ae5fb5ff92b2e9822fd2cd9a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 27 Nov 2024 12:50:15 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 27 Nov 2024 12:50:15 -0500
Avoid leaking the prior value when updating the "connection"
state variable.
Ditto for ECPGstruct_sizeof. (It seems like this one ought to
be statement-local, but testing says it isn't, and I didn't
feel like diving deeper.)
The actual_type[] entries are statement-local, though, so
no need to mm_strdup() strings stored in them.
Likewise, sqlda variables are statement-local, so we can
loc_alloc them.
Also clean up sloppiness around management of the argsinsert and
argsresult lists.
progname changes are strictly to prevent valgrind from complaining
about leaked allocations.
With this, valgrind reports zero leakage in the ecpg preprocessor
for all of our ecpg regression test cases.
Discussion: https://postgr.es/m/2011420.1713493114@sss.pgh.pa.us
M src/interfaces/ecpg/preproc/descriptor.c
M src/interfaces/ecpg/preproc/ecpg.addons
M src/interfaces/ecpg/preproc/ecpg.c
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/output.c
M src/interfaces/ecpg/preproc/variable.c
ecpg: put all string-valued tokens returned by pgc.l in local storage.
commit : 85312d95e959bae16c5d0bbf79ae74bcd7fec1a9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 27 Nov 2024 12:44:03 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 27 Nov 2024 12:44:03 -0500
This didn't work earlier in the patch series (I think some of
the strings were ending up in data-type-related structures),
but apparently we're now clean enough for it. This considerably
reduces process-lifespan memory leakage.
Discussion: https://postgr.es/m/2011420.1713493114@sss.pgh.pa.us
M src/interfaces/ecpg/preproc/parser.c
M src/interfaces/ecpg/preproc/pgc.l
ecpg: fix some memory leakage of data-type-related structures.
commit : 0e6060790d6533847084770845c84e81862bff47
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 27 Nov 2024 12:41:20 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 27 Nov 2024 12:41:20 -0500
ECPGfree_type() and related functions were quite incomplete
about removing subsidiary data structures. Possibly this is
because ecpg wasn't careful to make sure said data structures
always had their own storage. Previous patches in this series
cleaned up a lot of that, and I had to add a couple more
mm_strdup's here.
Also, ecpg.trailer tended to overwrite struct_member_list[struct_level]
without bothering to free up its previous contents, thus potentially
leaking a lot of struct-member-related storage. Add
ECPGfree_struct_member() calls at appropriate points.
Discussion: https://postgr.es/m/2011420.1713493114@sss.pgh.pa.us
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/type.c
M src/interfaces/ecpg/preproc/type.h
M src/interfaces/ecpg/preproc/variable.c
jsonapi: add lexer option to keep token ownership
commit : 5c32c21afe6449a19b6dfafa17f29b71c9595e03
author : Andrew Dunstan <andrew@dunslane.net>
date : Wed, 27 Nov 2024 12:05:44 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Wed, 27 Nov 2024 12:05:44 -0500
Commit 0785d1b8b adds support for libpq as a JSON client, but
allocations for string tokens can still be leaked during parsing
failures. This is tricky to fix for the object_field semantic callbacks:
the field name must remain valid until the end of the object, but if a
parsing error is encountered partway through, object_field_end() won't
be invoked and the client won't get a chance to free the field name.
This patch adds a flag to switch the ownership of parsed tokens to the
lexer. When this is enabled, the client must make a copy of any tokens
it wants to persist past the callback lifetime, but the lexer will
handle necessary cleanup on failure.
Backend uses of the JSON parser don't need to use this flag, since the
parser's allocations will occur in a short lived memory context.
A -o option has been added to test_json_parser_incremental to exercise
the new setJsonLexContextOwnsTokens() API, and the test_json_parser TAP
tests make use of it. (The test program now cleans up allocated memory,
so that tests can be usefully run under leak sanitizers.)
Author: Jacob Champion
Discussion: https://postgr.es/m/CAOYmi+kb38EciwyBQOf9peApKGwraHqA7pgzBkvoUnw5BRfS1g@mail.gmail.com
M src/common/jsonapi.c
M src/include/common/jsonapi.h
M src/test/modules/test_json_parser/t/001_test_json_parser_incremental.pl
M src/test/modules/test_json_parser/t/002_inline.pl
M src/test/modules/test_json_parser/t/003_test_semantic.pl
M src/test/modules/test_json_parser/test_json_parser_incremental.c
ci: Fix cached MacPorts installation management
commit : 262283d5eec3d582a8645692fede2e8f5e6029bd
author : Andres Freund <andres@anarazel.de>
date : Wed, 27 Nov 2024 11:28:59 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 27 Nov 2024 11:28:59 -0500
1. The error reporting of "port setrequested list-of-packages..."
changed, hiding errors we were relying on to know if all packages in our
list were already installed. Use a loop instead.
2. The cached MacPorts installation was shared between PostgreSQL
major branches, though each branch wanted different packages. Add the
list of packages to cache key, so that different branches, when tested
in one github account/repo such as postgres/postgres, stop fighting with
each other, adding and removing packages.
Back-patch to 15 where CI began.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Suggested-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/au2uqfuy2nf43nwy2txmc5t2emhwij7kzupygto3d2ffgtrdgr%40ckvrlwyflnh2
M .cirrus.tasks.yml
M src/tools/ci/ci_macports_packages.sh
Look up backend type in pg_signal_backend() more cheaply.
commit : 61171a632d10ad7abe1d7ad891176266cc5ef0e1
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 27 Nov 2024 10:32:25 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 27 Nov 2024 10:32:25 -0600
Commit ccd38024bc, which introduced the pg_signal_autovacuum_worker
role, added a call to pgstat_get_beentry_by_proc_number() for the
purpose of determining whether the process is an autovacuum worker.
This function calls pgstat_read_current_status(), which can be
fairly expensive and may return cached, out-of-date information.
Since we just need to look up the target backend's BackendType, and
we already know its ProcNumber, we can instead inspect the
BackendStatusArray directly, which is much less expensive and
possibly more up-to-date. There are some caveats with this
approach (which are documented in the code), but it's still
substantially better than before.
Reported-by: Andres Freund
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/ujenaa2uabzfkwxwmfifawzdozh3ljr7geozlhftsuosgm7n7q%40g3utqqyyosb6
M src/backend/storage/ipc/signalfuncs.c
M src/backend/utils/activity/backend_status.c
M src/include/utils/backend_status.h
postmaster: Reduce verbosity of environment dump debug message
commit : 6a5bcf7f7dd1e3c1c669912d82591397dbe24a10
author : Andres Freund <andres@anarazel.de>
date : Wed, 27 Nov 2024 11:17:23 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 27 Nov 2024 11:17:23 -0500
Emitting each variable separately is unnecessarily verbose / hard to skim
over. Emit the whole thing in one ereport() to address that.
Also remove program name and function reference from the message. The former
doesn't seem particularly helpful and the latter is provided by the elog.c
infrastructure these days.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/leouteo5ozcrux3fepuhtbp6c56tbfd4naxeokidbx7m75cabz@hhw6g4urlowt
M src/backend/postmaster/postmaster.c
file_fdw: Add regression tests for ON_ERROR and other options.
commit : 631db1d4b251a500e22743fc3eec9fb8724baf67
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 27 Nov 2024 23:40:11 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 27 Nov 2024 23:40:11 +0900
This commit introduces regression tests to validate incorrect settings
for the ON_ERROR, LOG_VERBOSITY, and REJECT_LIMIT options in file_fdw.
Author: Atsushi Torikoshi
Reviewed-by: Fujii Masao
Suggested-by: Yugo Nagata
Discussion: https://postgr.es/m/20241113231706.09e5b5ea9640289312835be8@sraoss.co.jp
M contrib/file_fdw/expected/file_fdw.out
M contrib/file_fdw/sql/file_fdw.sql
pgbench: Ensure previous progress message is fully cleared when updating.
commit : af35fe501af52f90355ac184a17643932a26464f
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 27 Nov 2024 23:01:53 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 27 Nov 2024 23:01:53 +0900
During pgbench's table initialization, progress updates could display
leftover characters from the previous message if the new message
was shorter. This commit resolves the issue by appending spaces to
the current message to fully overwrite any remaining characters from
the previous line.
Back-patch to all the supported versions.
Author: Yushi Ogiwara, Tatsuo Ishii, Fujii Masao
Reviewed-by: Tatsuo Ishii, Fujii Masao
Discussion: https://postgr.es/m/9a9b8b95b6a709877ae48ad5b0c59bb9@oss.nttdata.com
M src/bin/pgbench/pgbench.c
Fix pg_get_constraintdef for NOT NULL constraints on domains
commit : 09d09d4297b9acbc2848ec35e8bf030d6c1fae18
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 27 Nov 2024 13:50:27 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 27 Nov 2024 13:50:27 +0100
We added pg_constraint rows for all not-null constraints, first for
tables and later for domains; but while the ones for tables were
reverted, the ones for domains were not. However, we did accidentally
revert ruleutils.c support for the ones on domains in 6f8bb7c1e961,
which breaks running pg_get_constraintdef() on them. Put that back.
This is only needed in branch 17, because we've reinstated this code in
branch master with commit 14e87ffa5c54. Add some new tests in both
branches.
I couldn't find anything else that needs de-reverting.
Reported-by: Erki Eessaar <erki.eessaar@taltech.ee>
Reviewed-by: Magnus Hagander <magnus@hagander.net>
Discussion: https://postgr.es/m/AS8PR01MB75110350415AAB8BBABBA1ECFE222@AS8PR01MB7511.eurprd01.prod.exchangelabs.com
M src/test/regress/expected/domain.out
M src/test/regress/sql/domain.sql
gitattributes: Add .cpp files to whitespace checks
commit : 0d884f570b72c5b030f7908032946078537ea121
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 11:15:53 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 11:15:53 +0100
Use the same rules as .c files.
M .gitattributes
Fix typo
commit : 41272784b9383910de737937f1987dd00246f728
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 11:12:09 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 11:12:09 +0100
from commit 9044fc1d45a
M src/tools/pgindent/exclude_file_patterns
Exclude LLVM files from whitespace checks
commit : 96447e9c817143071826c9a5b129f61d17c4ae2e
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 11:08:12 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 11:08:12 +0100
Commit 9044fc1d45a added some files from upstream LLVM. These files
have different whitespace rules, which make the git whitespace checks
powered by gitattributes fail. To fix, add those files to the exclude
list.
M .gitattributes
Revert "Blind attempt to fix _configthreadlocale() failures on MinGW."
commit : a62d90f2e5cb55d0c8998b63120d6db3c55df866
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 27 Nov 2024 22:56:41 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 27 Nov 2024 22:56:41 +1300
This reverts commit 2cf91ccb73ce888c44e3751548fb7c77e87335f2.
When using the old msvcrt.dll, MinGW would supply its own dummy version
of _configthreadlocale() that just returns -1 if you try to use it. For
a time we tolerated that to shut the build farm up. We would fall back
to code that was enough for the tests to pass, but it would surely have
risked crashing a real multithreaded program.
We don't need that kludge anymore, because we can count on ucrt. We
expect the real _configthreadlocale() to be present, and the ECPG tests
will now fail if it isn't. The workaround was dead code and it's time
to revert it.
(A later patch still under review proposes to remove this use of
_configthreadlocale() completely but we're unwinding this code in
steps.)
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/d9e7731c-ca1b-477c-9298-fa51e135574a%40eisentraut.org
M src/interfaces/ecpg/ecpglib/descriptor.c
M src/interfaces/ecpg/ecpglib/execute.c
Require ucrt if using MinGW.
commit : 1758d42446161f5dfae9b14791c5640239b86faa
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 27 Nov 2024 22:34:11 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 27 Nov 2024 22:34:11 +1300
Historically we tolerated the absence of various C runtime library
features for the benefit of the MinGW tool chain, because it used
ancient msvcrt.dll for a long period of time. It now uses ucrt by
default (like Windows 10+, Visual Studio 2015+), and that's the only
configuration we're testing.
In practice, we effectively required ucrt already in PostgreSQL 17, when
commit 8d9a9f03 required _create_locale etc, first available in
msvcr120.dll (Visual Studio 2013, the last of the pre-ucrt series of
runtimes), and for MinGW users that practically meant ucrt because it
was difficult or impossible to use msvcr120.dll. That may even not have
been the first such case, but old MinGW configurations had already
dropped off our testing radar so we weren't paying much attention.
This commit formalizes the requirement. It also removes a couple of
obsolete comments that discussed msvcrt.dll limitations, and some tests
of !defined(_MSC_VER) to imply msvcrt.dll. There are many more
anachronisms, but it'll take some time to figure out how to remove them
all. APIs affected relate to locales, UTF-8, threads, large files and
more.
Thanks to Peter Eisentraut for the documentation change. It's not
really necessary to talk about ucrt explicitly in such a short section,
since it's the default for MinGW-w64 and MSYS2. It's enough to prune
references and broken links to much older tools.
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/d9e7731c-ca1b-477c-9298-fa51e135574a%40eisentraut.org
M doc/src/sgml/installation.sgml
M src/backend/utils/adt/pg_locale.c
Remove configure check for _configthreadlocale().
commit : f1da075d9a0373c08af32e31dcbf0809ae4aec2f
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 27 Nov 2024 22:34:03 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 27 Nov 2024 22:34:03 +1300
All modern Windows systems have _configthreadlocale(). It was first
introduced in msvcr80.dll from Visual Studio 2005. Historically, MinGW
was stuck on even older msvcrt.dll, but added its own dummy
implementation of the function when using msvcrt.dll years ago anyway,
effectively rendering the configure test useless. In practice we don't
encounter the dummy anymore because modern MinGW uses ucrt.
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
M configure
M configure.ac
M meson.build
M src/include/pg_config.h.in
M src/interfaces/ecpg/ecpglib/descriptor.c
M src/interfaces/ecpg/ecpglib/ecpglib_extern.h
M src/interfaces/ecpg/ecpglib/execute.c
Improve slightly misleading internal error message
commit : 63e10988f8705cc56ac242fa21ec42dd87e99cbf
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 10:55:35 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 10:55:35 +0100
The error message was talking about RowCompareType but was actually
checking strategy numbers. While those are closely related, it is
better to be accurate.
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/access/nbtree/nbtutils.c
Fix buildfarm failure from commit 8fcd80258b.
commit : 9d7aa406d04ce51da38eb5b6402079f60a601aec
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 27 Nov 2024 14:54:26 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 27 Nov 2024 14:54:26 +0530
The test case was incorrectly matching the error code.
Author: Vignesh C
Discussion: https://postgr.es/m/CALDaNm0C5LPiTxkdqsxiyeaL=nuUP8t6ne81sp9jE0=MFz=-ew@mail.gmail.com
M src/test/subscription/t/011_generated.pl
Support LIKE with nondeterministic collations
commit : 85b7efa1cdd63c2fe2b70b725b8285743ee5787f
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 08:18:35 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 08:18:35 +0100
This allows for example using LIKE with case-insensitive collations.
There was previously no internal implementation of this, so it was met
with a not-supported error. This adds the internal implementation and
removes the error. The implementation follows the specification of
the SQL standard for this.
Unlike with deterministic collations, the LIKE matching cannot go
character by character but has to go substring by substring. For
example, if we are matching against LIKE 'foo%bar', we can't start by
looking for an 'f', then an 'o', but instead with have to find
something that matches 'foo'. This is because the collation could
consider substrings of different lengths to be equal. This is all
internal to MatchText() in like_match.c.
The changes in GenericMatchText() in like.c just pass through the
locale information to MatchText(), which was previously not needed.
This matches exactly Generic_Text_IC_like() below.
ILIKE is not affected. (It's unclear whether ILIKE makes sense under
nondeterministic collations.)
This also updates match_pattern_prefix() in like_support.c to support
optimizing the case of an exact pattern with nondeterministic
collations. This was already alluded to in the previous code.
(includes documentation examples from Daniel Vérité and test cases
from Paul A Jungwirth)
Reviewed-by: Jian He <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/700d2e86-bf75-4607-9cf2-f5b7802f6e88@eisentraut.org
M doc/src/sgml/charset.sgml
M doc/src/sgml/func.sgml
M src/backend/utils/adt/like.c
M src/backend/utils/adt/like_match.c
M src/backend/utils/adt/like_support.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Improve error message for replication of generated columns.
commit : 8fcd80258bcf43dab93d877a5de0ce3f4d2bd471
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 27 Nov 2024 09:09:20 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 27 Nov 2024 09:09:20 +0530
Currently, logical replication produces a generic error message when
targeting a subscriber-side table column that is either missing or
generated. The error message can be misleading for generated columns.
This patch introduces a specific error message to clarify the issue when
generated columns are involved.
Author: Shubham Khanna
Reviewed-by: Peter Smith, Vignesh C, Amit Kapila
Discussion: https://postgr.es/m/CAHv8RjJBvYtqU7OAofBizOmQOK2Q8h+w9v2_cQWxT_gO7er3Aw@mail.gmail.com
M src/backend/replication/logical/relation.c
M src/test/subscription/t/011_generated.pl
Handle better implicit transaction state of pipeline mode
commit : d0eb4297cc5bbfb02b64fdaaf5510415e92e5312
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 27 Nov 2024 09:31:22 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 27 Nov 2024 09:31:22 +0900
When using a pipeline, a transaction starts from the first command and
is committed with a Sync message or when the pipeline ends.
Functions like IsInTransactionBlock() or PreventInTransactionBlock()
were already able to understand a pipeline as being in a transaction
block, but it was not the case of CheckTransactionBlock(). This
function is called for example to generate a WARNING for SET LOCAL,
complaining that it is used outside of a transaction block.
The current state of the code caused multiple problems, like:
- SET LOCAL executed at any stage of a pipeline issued a WARNING, even
if the command was at least second in line where the pipeline is in a
transaction state.
- LOCK TABLE failed when invoked at any step of a pipeline, even if it
should be able to work within a transaction block.
The pipeline protocol assumes that the first command of a pipeline is
not part of a transaction block, and that any follow-up commands is
considered as within a transaction block.
This commit changes the backend so as an implicit transaction block is
started each time the first Execute message of a pipeline has finished
processing, with this implicit transaction block ended once a sync is
processed. The checks based on XACT_FLAGS_PIPELINING in the routines
checking if we are in a transaction block are not necessary: it is
enough to rely on the existing ones.
Some tests are added to pgbench, that can be backpatched down to v17
when \syncpipeline is involved and down to v14 where \startpipeline and
\endpipeline are available. This is unfortunately limited regarding the
error patterns that can be checked, but it provides coverage for various
pipeline combinations to check if these succeed or fail. These tests
are able to capture the case of SET LOCAL's WARNING. The author has
proposed a different feature to improve the coverage by adding similar
meta-commands to psql where error messages could be checked, something
more useful for the cases where commands cannot be used in transaction
blocks, like REINDEX CONCURRENTLY or VACUUM. This is considered as
future work for v18~.
Author: Anthonin Bonnefoy
Reviewed-by: Jelte Fennema-Nio, Michael Paquier
Discussion: https://postgr.es/m/CAO6_XqrWO8uNBQrSu5r6jh+vTGi5Oiyk4y8yXDORdE2jbzw8xw@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/protocol.sgml
M src/backend/access/transam/xact.c
M src/backend/tcop/postgres.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
Fix commit 641a5b7a144 for "nbsp" output in SVG files
commit : 6e80951f49f3bc18b5bdfb7e87bc2e0bcfb4af00
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 26 Nov 2024 13:07:53 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 26 Nov 2024 13:07:53 -0500
In commit 641a5b7a144, I removed "nbsp" characters from SVG files, not
realizing the SVG files were generated from GV files and that the "nbsp"
characters were caused by trailing ASCII spaces in GV files. This
commit restores the "nbsp" SVG characters and adds a GV comment about
how the trailing spaces cause the "nbsp" output.
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/2c5dd601-b245-4092-9c27-6d1ad51609df%40eisentraut.org
Backpatch-through: master
M doc/src/sgml/images/genetic-algorithm.gv
M doc/src/sgml/images/genetic-algorithm.svg
Distinguish between AcquireExternalFD and epoll_create1 / kqueue failing
commit : b8f9afc81f26ead612fcfe76301c01d110e36c7e
author : Andres Freund <andres@anarazel.de>
date : Tue, 26 Nov 2024 12:20:59 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 26 Nov 2024 12:20:59 -0500
The error messages in CreateWaitEventSet() made it hard to know whether the
syscall or AcquireExternalFD() failed. This is particularly relevant because
AcquireExternalFD() imposes a lower limit than what would cause syscalls fail
with EMFILE.
I did not change the message in libpqsrv_connect_prepare(), which is the one
other use of AcquireExternalFD() in our codebase, as the error message already
is less ambiguous.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/xjjx7r4xa7beixuu4qtkdhnwdbchrrpo3gaeb3jsbinvvdiat5@cwjw55mna5of
M src/backend/storage/ipc/latch.c
meson: Build pgevent as shared_module rather than shared_library
commit : 4ee130c6e03becefae94e66bf537cd1279823c68
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 26 Nov 2024 18:06:08 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 26 Nov 2024 18:06:08 +0100
This matches the behavior of the makefiles and the old MSVC build
system. The main effect is that the build result gets installed into
pkglibdir rather than bindir. The documentation says to locate the
library in pkglibdir, so this makes the code match the documentation
again.
Reviewed-by: Ryohei Takahashi (Fujitsu) <r.takahashi_2@fujitsu.com>
Discussion: https://www.postgresql.org/message-id/flat/TY3PR01MB118912125614599641CA881B782522%40TY3PR01MB11891.jpnprd01.prod.outlook.com
M src/bin/pgevent/meson.build
Clean up newlines following left parentheses
commit : e6c32d9fad18f5a1dcb13c13f02445061b60523b
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 26 Nov 2024 17:10:07 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 26 Nov 2024 17:10:07 +0100
Most came in during the 17 cycle, so backpatch there. Some
(particularly reorderbuffer.h) are very old, but backpatching doesn't
seem useful.
Like commits c9d297751959, c4f113e8fef9.
M contrib/bloom/bloom.h
M src/backend/backup/basebackup_incremental.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/cache/relcache.c
M src/bin/pg_combinebackup/load_manifest.c
M src/common/parse_manifest.c
M src/common/unicode/category_test.c
M src/include/common/parse_manifest.h
M src/include/replication/reorderbuffer.h
Improve InitShmemAccess() prototype
commit : 2a7b2d97171dd39dca7cefb91008a3c84ec003ba
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 26 Nov 2024 08:25:23 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 26 Nov 2024 08:25:23 +0100
The code comment said, 'the argument should be declared "PGShmemHeader
*seghdr", but we use void to avoid having to include ipc.h in
shmem.h.' We can achieve the original goal with a struct forward
declaration. (ipc.h was also not the correct header file.)
Discussion: https://www.postgresql.org/message-id/flat/cnthxg2eekacrejyeonuhiaezc7vd7o2uowlsbenxqfkjwgvwj@qgzu6eoqrglb
M src/backend/storage/ipc/shmem.c
M src/include/storage/shmem.h
Fix test case from a8ccf4e93
commit : e15e56713702c45402d2728f3d7d68df96b2fbbe
author : Richard Guo <rguo@postgresql.org>
date : Tue, 26 Nov 2024 11:12:57 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Tue, 26 Nov 2024 11:12:57 +0900
Commit a8ccf4e93 uses the same table name "distinct_tbl" in both
select_distinct.sql and select_distinct_on.sql, which could cause
conflicts when these two test scripts are run in parallel.
Fix by renaming the table in select_distinct_on.sql to
"distinct_on_tbl".
Per buildfarm (via Tom Lane)
Discussion: https://postgr.es/m/1572004.1732583549@sss.pgh.pa.us
M src/test/regress/expected/select_distinct_on.out
M src/test/regress/sql/select_distinct_on.sql
pg_amcheck: Use CppAsString2() for relkind and relpersistence in queries
commit : 91f5a4a000ea4979e5490e0a111c24f4486d7361
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 26 Nov 2024 09:45:34 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 26 Nov 2024 09:45:34 +0900
This utility has been using hardcoded values for relkind and
relpersistence in its queries generated. These queries are switched to
use CppAsString2() instead, with the values fetched directly from the
header of pg_class. This has the advantage of making the code more
self-documented, as it becomes unnecessary to look at a header for the
meaning of a value.
There should be no functional changes; the queries are generated the
same way as before this commit.
Reviewed-by: Nathan Bossart, Daniel Gustafsson, Álvaro Herrera, Karina
Litskevich
Discussion: https://postgr.es/m/ZxIvemDk0Ob1RGwh@paquier.xyz
M src/bin/pg_amcheck/pg_amcheck.c
Remove dead code in get_param_path_clause_serials()
commit : cc4c90cef918fc0cbd13af8b51838c5de5da73bb
author : Richard Guo <rguo@postgresql.org>
date : Tue, 26 Nov 2024 09:27:53 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Tue, 26 Nov 2024 09:27:53 +0900
The function get_param_path_clause_serials() is used to get the set of
pushed-down clauses enforced within a parameterized Path. Since we
don't currently support parameterized MergeAppend paths, and it
doesn't look like that is going to change anytime soon (as explained
in the comments for generate_orderedappend_paths), we don't need to
consider MergeAppendPath in this function.
This change won't make any measurable difference in performance; it's
just for clarity's sake.
Author: Richard Guo
Reviewed-by: Andrei Lepikhov
Discussion: https://postgr.es/m/CAMbWs4_Puie4DQ2ODvjQB_3CxYkUODnrJm8jn_ObMAcrjYNW7Q@mail.gmail.com
M src/backend/optimizer/util/relnode.c
Reordering DISTINCT keys to match input path's pathkeys
commit : a8ccf4e93a7eeaae66007bbf78cf9183ceb1b371
author : Richard Guo <rguo@postgresql.org>
date : Tue, 26 Nov 2024 09:25:18 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Tue, 26 Nov 2024 09:25:18 +0900
The ordering of DISTINCT items is semantically insignificant, so we
can reorder them as needed. In fact, in the parser, we absorb the
sorting semantics of the sortClause as much as possible into the
distinctClause, ensuring that one clause is a prefix of the other.
This can help avoid a possible need to re-sort.
In this commit, we attempt to adjust the DISTINCT keys to match the
input path's pathkeys. This can likewise help avoid re-sorting, or
allow us to use incremental-sort to save efforts.
For DISTINCT ON expressions, the parser already ensures that they
match the initial ORDER BY expressions. When reordering the DISTINCT
keys, we must ensure that the resulting pathkey list matches the
initial distinctClause pathkeys.
This introduces a new GUC, enable_distinct_reordering, which allows
the optimization to be disabled if needed.
Author: Richard Guo
Reviewed-by: Andrei Lepikhov
Discussion: https://postgr.es/m/CAMbWs48dR26cCcX0f=8bja2JKQPcU64136kHk=xekHT9xschiQ@mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/optimizer/path/pathkeys.c
M src/backend/optimizer/plan/planner.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/optimizer/optimizer.h
M src/test/regress/expected/select_distinct.out
M src/test/regress/expected/select_distinct_on.out
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/select_distinct.sql
M src/test/regress/sql/select_distinct_on.sql
Fix NULLIF()'s handling of read-write expanded objects.
commit : 5b8728cd7f9d3d93b6ff9b48887084fdf0a46e4f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Nov 2024 18:08:58 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Nov 2024 18:08:58 -0500
If passed a read-write expanded object pointer, the EEOP_NULLIF
code would hand that same pointer to the equality function
and then (unless equality was reported) also return the same
pointer as its value. This is no good, because a function that
receives a read-write expanded object pointer is fully entitled
to scribble on or even delete the object, thus corrupting the
NULLIF output. (This problem is likely unobservable with the
equality functions provided in core Postgres, but it's easy to
demonstrate with one coded in plpgsql.)
To fix, make sure the pointer passed to the equality function
is read-only. We can still return the original read-write
pointer as the NULLIF result, allowing optimization of later
operations.
Per bug #18722 from Alexander Lakhin. This has been wrong
since we invented expanded objects, so back-patch to all
supported branches.
Discussion: https://postgr.es/m/18722-fd9e645448cc78b4@postgresql.org
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/include/executor/execExpr.h
M src/test/regress/expected/case.out
M src/test/regress/sql/case.sql
Avoid "you don't own a lock of type ExclusiveLock" in GRANT TABLESPACE.
commit : 4ba84de459532a7a79616e756f8bfd45ba4b8048
author : Noah Misch <noah@leadboat.com>
date : Mon, 25 Nov 2024 14:42:35 -0800
committer: Noah Misch <noah@leadboat.com>
date : Mon, 25 Nov 2024 14:42:35 -0800
This WARNING appeared because SearchSysCacheLocked1() read
cc_relisshared before catcache initialization, when the field is false
unconditionally. On the basis of reading false there, it constructed a
locktag as though pg_tablespace weren't relisshared. Only shared
catalogs could be affected, and only GRANT TABLESPACE was affected in
practice. SearchSysCacheLocked1() callers use one other shared-relation
syscache, DATABASEOID. DATABASEOID is initialized by the end of
CheckMyDatabase(), making the problem unreachable for pg_database.
Back-patch to v13 (all supported versions). This has no known impact
before v16, where ExecGrant_common() first appeared. Earlier branches
avoid trouble by having a separate ExecGrant_Tablespace() that doesn't
use LOCKTAG_TUPLE. However, leaving this unfixed in v15 could ensnare a
future back-patch of a SearchSysCacheLocked1() call.
Reported by Aya Iwata.
Discussion: https://postgr.es/m/OS7PR01MB11964507B5548245A7EE54E70EA212@OS7PR01MB11964.jpnprd01.prod.outlook.com
M src/backend/utils/cache/syscache.c
M src/test/regress/expected/tablespace.out
M src/test/regress/sql/tablespace.sql
pg_dump: Add dumpSchema and dumpData derivative flags.
commit : 96a81c1be929d122719bd289f6e24824f37e1ff6
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 25 Nov 2024 16:36:37 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 25 Nov 2024 16:36:37 -0600
Various parts of pg_dump consult the --schema-only and --data-only
options to determine whether to run a section of code. While this
is simple enough for two mutually-exclusive options, it will become
progressively more complicated as more options are added. In
anticipation of that, this commit introduces new internal flags
called dumpSchema and dumpData, which are derivatives of
--schema-only and --data-only. This commit also removes the
schemaOnly and dataOnly members from the dump/restore options
structs to prevent their use elsewhere.
Note that this change neither adds new user-facing command-line
options nor changes the existing --schema-only and --data-only
options.
Author: Corey Huinker
Reviewed-by: Jeff Davis
Discussion: https://postgr.es/m/CADkLM%3DcQgghMJOS8EcAVBwRO4s1dUVtxGZv5gLPfZkQ1nL1gzA%40mail.gmail.com
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_restore.c
Clean up <stdbool.h> reference in meson.build.
commit : 648333a99f56058abd9abdeb87ab0b9e97709d91
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 26 Nov 2024 11:29:31 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 26 Nov 2024 11:29:31 +1300
Commit bc5a4dfc accidentally left a check for <stdbool.h> in
meson.build's header_checks. Synchronize with configure, which no
longer defines HAVE_STDBOOL_H.
There is still a reference to <stdbool.h> in an earlier test to see if
we need -std=c99 to get C99 features, like autoconf 2.69's
AC_PROG_CC_C99. (Therefore the test remove by this commit was
tautological since day one: you'd have copped "C compiler does not
support C99" before making it this far.)
Back-patch to 16, where meson begins.
M meson.build
Update configure probes for CFLAGS needed for ARM CRC instructions.
commit : 5980f1884fc911af120c98ad440b9546ed9012c5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Nov 2024 12:50:17 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Nov 2024 12:50:17 -0500
On ARM platforms where the baseline CPU target lacks CRC instructions,
we need to supply a -march flag to persuade the compiler to compile
such instructions. It turns out that our existing choice of
"-march=armv8-a+crc" has not worked for some time, because recent gcc
will interpret that as selecting software floating point, and then
will spit up if the platform requires hard-float ABI, as most do
nowadays. The end result was to silently fall back to software CRC,
which isn't very desirable since in practice almost all currently
produced ARM chips do have hardware CRC.
We can fix this by using "-march=armv8-a+crc+simd" to enable the
correct ABI choice. (This has no impact on the code actually
generated, since neither of the files we compile with this flag
does any floating-point stuff, let alone SIMD.) Keep the test for
"-march=armv8-a+crc" since that's required for soft-float ABI,
but try that second since most platforms we're likely to build on
use hard-float.
Since this isn't working as-intended on the last several years'
worth of gcc releases, back-patch to all supported branches.
Discussion: https://postgr.es/m/4496616.iHFcN1HehY@portable-bastien
M configure
M configure.ac
M meson.build
Support runtime CRC feature probing on NetBSD/ARM using sysctl().
commit : 4570b22666dd4c45a4c355f7d156d949c0247e2d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Nov 2024 11:53:26 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Nov 2024 11:53:26 -0500
Commit aac831caf left this as a to-do; here's code to do it.
Like the previous patch, this is HEAD-only for now.
Discussion: https://postgr.es/m/4496616.iHFcN1HehY@portable-bastien
M src/port/pg_crc32c_armv8_choose.c
Add support for Tcl 9
commit : 32a2aa77ef7eec5ca46cfdbe6b175435cac1af61
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 25 Nov 2024 08:03:16 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 25 Nov 2024 08:03:16 +0100
Tcl 9 changed several API functions to take Tcl_Size, which is
ptrdiff_t, instead of int, for 64-bit enablement. We have to change a
few local variables to be compatible with that. We also provide a
fallback typedef of Tcl_Size for older Tcl versions.
The affected variables are used for quantities that will not approach
values beyond the range of int, so this doesn't change any
functionality.
Reviewed-by: Tristan Partin <tristan@partin.io>
Discussion: https://www.postgresql.org/message-id/flat/bce0fe54-75b4-438e-b42b-8e84bc7c0e9c%40eisentraut.org
M src/pl/tcl/pltcl.c
Assume that <stdbool.h> conforms to the C standard.
commit : bc5a4dfcf7390145dc3ba3c1c18c5ce561b778cd
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 25 Nov 2024 13:11:28 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 25 Nov 2024 13:11:28 +1300
Previously we checked "for <stdbool.h> that conforms to C99" using
autoconf's AC_HEADER_STDBOOL macro. We've required C99 since PostgreSQL
12, so the test was redundant, and under C23 it was broken: autoconf
2.69's implementation doesn't understand C23's new empty header (the
macros it's looking for went away, replaced by language keywords).
Later autoconf versions fixed that, but let's just remove the
anachronistic test.
HAVE_STDBOOL_H and HAVE__BOOL will no longer be defined, but they
weren't directly tested in core or likely extensions (except in 11, see
below). PG_USE_STDBOOL (or USE_STDBOOL in 11 and 12) is still defined
when sizeof(bool) is 1, which should be true on all modern systems.
Otherwise we define our own bool type and values of size 1, which would
fail to compile under C23 as revealed by the broken test. (We'll
probably clean that dead code up in master, but here we want a minimal
back-patchable change.)
This came to our attention when GCC 15 recently started using using C23
by default and failed to compile the replacement code, as reported by
Sam James and build farm animal alligator.
Back-patch to all supported releases, and then two older versions that
also know about <stdbool.h>, per the recently-out-of-support policy[1].
12 requires C99 so it's much like the supported releases, but 11 only
assumes C89 so it now uses AC_CHECK_HEADERS instead of the overly picky
AC_HEADER_STDBOOL. (I could find no discussion of which historical
systems had <stdbool.h> but failed the conformance test; if they ever
existed, they surely aren't relevant to that policy's goals.)
[1] https://wiki.postgresql.org/wiki/Committing_checklist#Policies
Reported-by: Sam James <sam@gentoo.org>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org> (master version)
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (approach)
Discussion: https://www.postgresql.org/message-id/flat/87o72eo9iu.fsf%40gentoo.org
M configure
M configure.ac
M meson.build
M src/include/c.h
M src/include/pg_config.h.in
Remove the wrong assertion from match_orclause_to_indexcol()
commit : d4d11940df94ee13166dda70271ff3b97e43bc9b
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 25 Nov 2024 09:05:26 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 25 Nov 2024 09:05:26 +0200
Obviously, the constant could be zero. Also, add the relevant check to
regression tests.
Reported-by: Richard Guo
Discussion: https://postgr.es/m/CAMbWs4-siKJdtWhcbqk4Y-xG12do2Ckm1qw672GNsSnDqL9FQg%40mail.gmail.com
M src/backend/optimizer/path/indxpath.c
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Doc: Clarify the `inactive_since` field description.
commit : d05a387d9d78a65db2c960d4f309a2649d183066
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 25 Nov 2024 11:12:32 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 25 Nov 2024 11:12:32 +0530
Updated to specify that it represents the exact time a slot became
inactive, rather than the period of inactivity.
Reported-by: Peter Smith
Author: Bruce Momjian, Nisha Moond
Reviewed-by: Amit Kapila, Peter Smith
Backpatch-through: 17
Discussion: https://postgr.es/m/CAHut+PuvsyA5v8y7rYoY9mkDQzUhwaESM05yCByTMaDoRh30tA@mail.gmail.com
M doc/src/sgml/system-views.sgml
M src/backend/replication/logical/slotsync.c
M src/include/replication/slot.h
Simplify some SPI tests of PL/Python
commit : db80507d98462d95f2f7f8c1a58e55bb6337c3eb
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 25 Nov 2024 09:43:16 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 25 Nov 2024 09:43:16 +0900
These tests relied on both next() and __next__(), but only the former is
needed since Python 2 support has been removed, so let's simplify a bit
the tests.
Author: Erik Wienhold
Discussion: https://postgr.es/m/173209043143.2092749.13692266486972491694@wrigleys.postgresql.org
M src/pl/plpython/expected/plpython_spi.out
M src/pl/plpython/sql/plpython_spi.sql
doc: Fix example with __next__() in PL/Python function
commit : 2ff7c913d92d8540e8b00c54746ce2f565521256
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 25 Nov 2024 09:15:25 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 25 Nov 2024 09:15:25 +0900
Per PEP 3114, iterator.next() has been renamed to iterator.__next__(),
and one example in the documentation still used next(). This caused the
example provided to fail the function creation since Python 2 is not
supported anymore since 19252e8ec93.
Author: Erik Wienhold
Discussion: https://postgr.es/m/173209043143.2092749.13692266486972491694@wrigleys.postgresql.org
Backpatch-through: 15
M doc/src/sgml/plpython.sgml
Test "options=-crole=" and "ALTER DATABASE SET role".
commit : 5de08f136ad690cde9a99c09c398ddb0c9272f11
author : Noah Misch <noah@leadboat.com>
date : Sun, 24 Nov 2024 12:49:53 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sun, 24 Nov 2024 12:49:53 -0800
Commit 7b88529f4363994450bd4cd3c172006a8a77e222 fixed a regression
spanning these features, but it didn't test them. It did test code
paths sufficient for their present implementations, so no back-patch.
Reported by Matthew Woodcraft.
Discussion: https://postgr.es/m/87iksnsbhx.fsf@golux.woodcraft.me.uk
M src/test/modules/unsafe_tests/Makefile
M src/test/modules/unsafe_tests/expected/setconfig.out
M src/test/modules/unsafe_tests/meson.build
M src/test/modules/unsafe_tests/sql/setconfig.sql
Teach bitmap path generation about transforming OR-clauses to SAOP's
commit : ae4569161a27823793ca24825bbabce2a91a0bc9
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 24 Nov 2024 01:41:45 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 24 Nov 2024 01:41:45 +0200
When optimizer generates bitmap paths, it considers breaking OR-clause
arguments one-by-one. But now, a group of similar OR-clauses can be
transformed into SAOP during index matching. So, bitmap paths should
keep up.
This commit teaches bitmap paths generation machinery to group similar
OR-clauses into dedicated RestrictInfos. Those RestrictInfos are considered
both to match index as a whole (as SAOP), or to match as a set of individual
OR-clause argument one-by-one (the old way).
Therefore, bitmap path generation will takes advantage of OR-clauses to SAOP's
transformation. The old way of handling them is also considered. So, there
shouldn't be planning regression.
Discussion: https://postgr.es/m/CAPpHfdu5iQOjF93vGbjidsQkhHvY2NSm29duENYH_cbhC6x%2BMg%40mail.gmail.com
Author: Alexander Korotkov, Andrey Lepikhov
Reviewed-by: Alena Rybakina, Andrei Lepikhov, Jian he, Robert Haas
Reviewed-by: Peter Geoghegan
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/util/restrictinfo.c
M src/include/optimizer/restrictinfo.h
M src/test/regress/expected/create_index.out
M src/test/regress/expected/join.out
M src/test/regress/sql/create_index.sql
M src/tools/pgindent/typedefs.list
Transform OR-clauses to SAOP's during index matching
commit : d4378c0005e61b1bb78e88097ea6efcdddbe2d6e
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 24 Nov 2024 01:40:20 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 24 Nov 2024 01:40:20 +0200
This commit makes match_clause_to_indexcol() match
"(indexkey op C1) OR (indexkey op C2) ... (indexkey op CN)" expression
to the index while transforming it into "indexkey op ANY(ARRAY[C1, C2, ...])"
(ScalarArrayOpExpr node).
This transformation allows handling long OR-clauses with single IndexScan
avoiding diving them into a slower BitmapOr.
We currently restrict Ci to be either Const or Param to apply this
transformation only when it's clearly beneficial. However, in the future,
we might switch to a liberal understanding of constants, as it is in other
cases.
Discussion: https://postgr.es/m/567ED6CA.2040504%40sigaev.ru
Author: Alena Rybakina, Andrey Lepikhov, Alexander Korotkov
Reviewed-by: Peter Geoghegan, Ranier Vilela, Alexander Korotkov, Robert Haas
Reviewed-by: Jian He, Tom Lane, Nikolay Shaplov
M src/backend/optimizer/path/indxpath.c
M src/test/regress/expected/create_index.out
M src/test/regress/expected/join.out
M src/test/regress/expected/rowsecurity.out
M src/test/regress/expected/stats_ext.out
M src/test/regress/expected/uuid.out
M src/test/regress/sql/create_index.sql
M src/test/regress/sql/join.sql
M src/test/regress/sql/rowsecurity.sql
M src/test/regress/sql/stats_ext.sql
M src/test/regress/sql/uuid.sql
Disallow modifying statistics on system columns.
commit : 869ee4f10eca2acda3d2210198a46d5029a569fc
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 22 Nov 2024 12:07:46 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 22 Nov 2024 12:07:46 -0800
Reported-by: Heikki Linnakangas
Discussion: https://postgr.es/m/df3e1c41-4e6c-40ad-9636-98deefe488cd@iki.fi
M src/backend/statistics/attribute_stats.c
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Add INT64_HEX_FORMAT and UINT64_HEX_FORMAT to c.h.
commit : efdc7d747539fccc7c9a946bfcb4ded716f679f1
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 22 Nov 2024 12:41:57 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 22 Nov 2024 12:41:57 -0600
Like INT64_FORMAT and UINT64_FORMAT, these macros produce format
strings for 64-bit integers. However, INT64_HEX_FORMAT and
UINT64_HEX_FORMAT generate the output in hexadecimal instead of
decimal. Besides introducing these macros, this commit makes use
of them in several places. This was originally intended to be part
of commit 5d6187d2a2, but I left it out because I felt there was a
nonzero chance that back-patching these new macros into c.h could
cause problems with third-party code. We tend to be less cautious
with such changes in new major versions.
Note that UINT64_HEX_FORMAT was originally added in commit
ee1b30f128, but it was placed in test_radixtree.c, so it wasn't
widely available. This commit moves UINT64_HEX_FORMAT to c.h.
Discussion: https://postgr.es/m/ZwQvtUbPKaaRQezd%40nathan
M contrib/postgres_fdw/option.c
M src/backend/utils/error/csvlog.c
M src/backend/utils/error/elog.c
M src/backend/utils/error/jsonlog.c
M src/include/c.h
M src/test/modules/test_radixtree/test_radixtree.c
Add a couple of recent commits to .git-blame-ignore-revs.
commit : 8589876b796e1b78ccb38c3557f648f2b5f336ba
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 22 Nov 2024 12:17:35 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 22 Nov 2024 12:17:35 -0600
M .git-blame-ignore-revs
Make the memory layout of Port struct independent of USE_OPENSSL
commit : 8fb5936703e84945766b8425bfe54e0060dea231
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 22 Nov 2024 17:43:04 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 22 Nov 2024 17:43:04 +0200
Commit d39a49c1e4 added new fields to the struct, but missed the "keep
these last" comment on the previous fields. Add placeholder variables
so that the offsets of the fields are the same whether you build with
USE_OPENSSL or not. This is a courtesy to extensions that might peek
at the fields, to make the ABI the same regardless of the options used
to build PostgreSQL.
In reality, I don't expect any extensions to look at the 'raw_buf'
fields. Firstly, they are new in v17, so no one's written such
extensions yet. Secondly, extensions should have no business poking at
those fields anyway. Nevertheless, fix this properly on 'master'. On
v17, we mustn't change the memory layout, so just fix the comments.
Author: Jacob Champion
Discussion: https://www.postgresql.org/message-id/raw/CAOYmi%2BmKVJNzn5_TD_MK%3DhqO64r_w8Gb0FHCLk0oAkW-PJv8jQ@mail.gmail.com
M src/include/libpq/libpq-be.h
Fix data loss when restarting the bulk_write facility
commit : ee937f0409d5c75855861bf31f88eeb77623b411
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 22 Nov 2024 16:28:24 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 22 Nov 2024 16:28:24 +0200
If a user started a bulk write operation on a fork with existing data
to append data in bulk, the bulk_write machinery would zero out all
previously written pages up to the last page written by the new
bulk_write operation.
This is not an issue for PostgreSQL itself, because we never use the
bulk_write facility on a non-empty fork. But there are use cases where
it makes sense. TimescaleDB extension is known to do that to merge
partitions, for example.
Backpatch to v17, where the bulk_write machinery was introduced.
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reported-By: Erik Nordström <erik@timescale.com>
Reviewed-by: Erik Nordström <erik@timescale.com>
Discussion: https://www.postgresql.org/message-id/CACAa4VJ%2BQY4pY7M0ECq29uGkrOygikYtao1UG9yCDFosxaps9g@mail.gmail.com
M src/backend/storage/smgr/bulk_write.c
Use auxv to check for CRC32 instructions on ARM.
commit : aac831cafa6f3106dfcbd3298757801c299351fc
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 22 Nov 2024 16:45:19 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 22 Nov 2024 16:45:19 +1300
Previously we probed for CRC32 instructions by testing if they caused
SIGILL. Some have expressed doubts about that technique, the Linux
documentation advises not to use it, and it's not exactly beautiful.
Now that more operating systems expose CPU features to userspace via the
ELF loader in approximately the same way, let's use that instead.
This is expected to work on Linux, FreeBSD and recent OpenBSD.
OpenBSD/ARM has not been tested and is not present in our build farm,
but the API matches FreeBSD.
On macOS, compilers use a more recent baseline ISA so the runtime test
mechanism isn't reached. (A similar situation is expected for
Windows/ARM when that port lands.)
On NetBSD, runtime feature probing is lost for armv8-a builds. It looks
potentially doable with sysctl following the example of the cpuctl
program; patches are welcome.
No back-patch for now, since we don't have any evidence of actual
breakage from the previous technique.
Suggested-by: Bastien Roucariès <rouca@debian.org>
Discussion: https://postgr.es/m/4496616.iHFcN1HehY%40portable-bastien
M configure
M configure.ac
M meson.build
M src/include/pg_config.h.in
M src/port/pg_crc32c_armv8_choose.c
psql: Fix category of \parse in output of --help=commands and \?
commit : ea15816928c1bbcab749205a263d82daea28a3e0
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 22 Nov 2024 14:04:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 22 Nov 2024 14:04:21 +0900
\parse was listed under the category "Connection", which was incorrect.
Let's move it to "General" like the other meta-commands of the same type
(\bind, \bind_named and \close).
Oversight in commit d55322b0da60.
Discussion: https://postgr.es/m/Zz_x-NEKNeeRlAVc@paquier.xyz
M src/bin/psql/help.c
psql: Include \pset xheader_width in --help=commands|variables
commit : 768dfd8e6575f752f722d0a0ad69adebf0045a39
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 22 Nov 2024 12:17:37 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 22 Nov 2024 12:17:37 +0900
psql's --help was missed the description of the \pset variable
xheader_width, that should be listed when using \? or --help=commands,
and described for --help=variables.
Oversight in a45388d6e098.
Author: Pavel Luzanov
Discussion: https://postgr.es/m/1e3e06d6-0807-4e62-a9f6-c11481e6eb10@postgrespro.ru
Backpatch-through: 16
M src/bin/psql/help.c
jit: Use -mno-outline-atomics for bitcode on ARM.
commit : 78c09bd9f94e8d9b5dc65ebcad4145e71381ce9d
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 22 Nov 2024 14:53:21 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 22 Nov 2024 14:53:21 +1300
If the executable's .o files were produced by a compiler (probably gcc)
not using -moutline-atomics, and the corresponding .bc files were
produced by clang using -moutline-atomics (probably by default), then
the generated bitcode functions would have the target attribute
"+outline-atomics", and could fail at runtime when inlined. If the
target ISA at bitcode generation time was armv8-a (the most conservative
aarch64 target, no LSE), then LLVM IR atomic instructions would generate
calls to functions in libgcc.a or libclang_rt.*.a that switch between
LL/SC and faster LSE instructions depending on a runtime AT_HWCAP check.
Since the corresponding .o files didn't need those functions, they
wouldn't have been included in the executable, and resolution would
fail.
At least Debian and Ubuntu are known to ship gcc and clang compilers
that target armv8-a but differ on the use of outline atomics by default.
Fix, by suppressing the outline atomics attribute in bitcode explicitly.
Inline LL/SC instructions will be generated for atomic operations in
bitcode built for armv8-a. Only configure scripts are adjusted for now,
because the meson build system doesn't generate bitcode yet.
This doesn't seem to be a new phenomenon, so real cases of functions
using atomics that are inlined by JIT must be rare in the wild given how
long it took for a bug report to arrive. The reported case could be
reduced to:
postgres=# set jit_inline_above_cost = 0;
SET
postgres=# set jit_above_cost = 0;
SET
postgres=# select pg_last_wal_receive_lsn();
WARNING: failed to resolve name __aarch64_swp4_acq_rel
FATAL: fatal llvm error: Program used external function
'__aarch64_swp4_acq_rel' which could not be resolved!
The change doesn't affect non-ARM systems or later target ISAs.
Back-patch to all supported releases.
Reported-by: Alexander Kozhemyakin <a.kozhemyakin@postgrespro.ru>
Discussion: https://postgr.es/m/18610-37bf303f904fede3%40postgresql.org
M configure
M configure.ac
Add write_to_file to PgStat_KindInfo for pgstats kinds
commit : c06e71d1acaf490660ed4ae5c28b3079628746b0
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 22 Nov 2024 10:12:26 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 22 Nov 2024 10:12:26 +0900
This new field controls if entries of a stats kind should be written or
not to the on-disk pgstats file when shutting down an instance. This
affects both fixed and variable-numbered kinds.
This is useful for custom statistics by itself, and a patch is under
discussion to add a new builtin stats kind where the write of the stats
is not necessary. All the built-in stats kinds, as well as the two
custom stats kinds in the test module injection_points, set this flag to
"true" for now, so as stats entries are written to the on-disk pgstats
file.
Author: Bertrand Drouvot
Reviewed-by: Nazir Bilal Yavuz
Discussion: https://postgr.es/m/Zz7T47nHwYgeYwOe@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/utils/activity/pgstat.c
M src/include/utils/pgstat_internal.h
M src/test/modules/injection_points/injection_stats.c
M src/test/modules/injection_points/injection_stats_fixed.c
doc: clarify how logical replication takes its initial snapshot
commit : 4c4aaa19a6fed39e0eb0247625331c3df34d8211
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 21 Nov 2024 17:14:33 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 21 Nov 2024 17:14:33 -0500
Reported-by: Koen De Groote
Discussion: https://postgr.es/m/171606613152.686.7693963105919927503@wrigleys.postgresql.org
Backpatch-through: master
M doc/src/sgml/logical-replication.sgml
pgindent run
commit : 53dcba9be5746cc126bdb949bf81c29ea2cfc24d
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 21 Nov 2024 21:40:17 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 21 Nov 2024 21:40:17 +0100
for commit 79b575d3bc0
M src/backend/commands/tablecmds.c
Fix newly introduced 010_keep_recycled_wals.pl
commit : b5be29ecafe1653ec53f49cbc4789b9683648bc0
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 21 Nov 2024 17:04:26 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 21 Nov 2024 17:04:26 +0100
It failed to set the archive_command as it desired because of a syntax
problem. Oversight in commit 90bcc7c2db1d.
This bug doesn't cause the test to fail, because the test only checks
pg_rewind's output messages, not the actual outcome (and the outcome in
both cases is that the file is kept, not deleted). But in either case
the message about the file being kept is there, so it's hard to get
excited about doing much more.
Reported-by: Antonin Houska <ah@cybertec.at>
Author: Alexander Kukushkin <cyberdemn@gmail.com>
Discussion: https://postgr.es/m/7822.1732167825@antos
M src/bin/pg_rewind/t/010_keep_recycled_wals.pl
Fix outdated bit in README.tuplock
commit : 7300ff1bd8fa64c0feab1d490b709d553e9a438e
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 21 Nov 2024 16:54:36 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 21 Nov 2024 16:54:36 +0100
Apparently this information has been outdated since first committed,
because we adopted a different implementation during development per
reviews and this detail was not updated in the README.
This has been wrong since commit 0ac5ad5134f2 introduced the file in
2013. Backpatch to all live branches.
Reported-by: Will Mortensen <will@extrahop.com>
Discussion: https://postgr.es/m/CAMpnoC6yEQ=c0Rdq-J7uRedrP7Zo9UMp6VZyP23QMT68n06cvA@mail.gmail.com
M src/backend/access/heap/README.tuplock
Fix ALTER TABLE / REPLICA IDENTITY for temporal tables
commit : 79b575d3bc09cc67a7b5c2da454aeb35f4beccc5
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 21 Nov 2024 13:50:18 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 21 Nov 2024 13:50:18 +0100
REPLICA IDENTITY USING INDEX did not accept a GiST index. This should
be allowed when used as a temporal primary key.
Author: Paul Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://www.postgresql.org/message-id/04579cbf-b134-45e1-8f2d-8c54c849c1ee@illuminatedcomputing.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/without_overlaps.out
M src/test/regress/sql/without_overlaps.sql
Unify repetitive error messages
commit : da94e871e8877d1f1054277b24b604bf1c0036a6
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 21 Nov 2024 10:54:30 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 21 Nov 2024 10:54:30 +0100
M src/backend/commands/explain.c
Fix memory leak in pgoutput for the WAL sender
commit : ea792bfd93ab8ad4ef4e3d1a741b8595db143677
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 21 Nov 2024 15:14:02 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 21 Nov 2024 15:14:02 +0900
RelationSyncCache, the hash table in charge of tracking the relation
schemas sent through pgoutput, was forgetting to free the TupleDesc
associated to the two slots used to store the new and old tuples,
causing some memory to be leaked each time a relation is invalidated
when the slots of an existing relation entry are cleaned up.
This is rather hard to notice as the bloat is pretty minimal, but a
long-running WAL sender would be in trouble over time depending on the
workload. sysbench has proved to be pretty good at showing the problem,
coupled with some memory monitoring of the WAL sender.
Issue introduced in 52e4f0cd472d, that has added row filters for tables
logically replicated.
Author: Boyu Yang
Reviewed-by: Michael Paquier, Hou Zhijie
Discussion: https://postgr.es/m/DM3PR84MB3442E14B340E553313B5C816E3252@DM3PR84MB3442.NAMPRD84.PROD.OUTLOOK.COM
Backpatch-through: 15
M src/backend/replication/pgoutput/pgoutput.c
More logically order libpq func. includes, e.g., group GUC vals
commit : f95da9f0e091ddbc5d50ec6c11be772da009b24e
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 20 Nov 2024 17:09:17 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 20 Nov 2024 17:09:17 -0500
Reported-by: David Zhang
Discussion: https://postgr.es/m/65909efe-97c6-4863-af4e-21eb5a26dd1e@highgo.ca
Co-authored-by: David Zhang
Backpatch-through: master
M src/include/libpq/libpq.h
doc: clarify that jsonb_path_match() returns an SQL boolean
commit : 70236cf22f4c42338d25c07f7fd00bd5c5830eb5
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 20 Nov 2024 17:03:45 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 20 Nov 2024 17:03:45 -0500
Not a JSON boolean. Also clarify that other predicate check expressions
functions return a JSON boolean, not an SQL boolean.
Reported-by: jian he
Discussion: https://postgr.es/m/CACJufxH7tP1NXCHN1bUBXcEB=dv7-qE+ZjB3UxwK6Em+9Qzb9Q@mail.gmail.com
Backpatch-through: 17
M doc/src/sgml/func.sgml
clarify --no-comments option in --help and SGML files
commit : f722dd32de49af883b7b58d0ca028835972b7095
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 20 Nov 2024 14:48:31 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 20 Nov 2024 14:48:31 -0500
The previous commit, b38bac26e20, missed these cases for dump/restore.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/3495698.1731968093@sss.pgh.pa.us
Backpatch-through: master
M doc/src/sgml/ref/pg_dumpall.sgml
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_dump/pg_restore.c
Refine nbtree = redundancy preprocessing comment.
commit : 70743376987af2393e42f3cfc2def4a472070659
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 20 Nov 2024 13:37:08 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 20 Nov 2024 13:37:08 -0500
Spell out how a = key associated with a SAOP array renders a > key
against the same index column redundant at the relevant point inside
_bt_preprocess_keys.
Follow-up to commit 5bf748b8.
M src/backend/access/nbtree/nbtutils.c
Avoid assertion failure if a setop leaf query contains setops.
commit : 94131cd53c1552efe85c3e8120356c22cd658163
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Nov 2024 12:03:47 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Nov 2024 12:03:47 -0500
Ordinarily transformSetOperationTree will collect all UNION/
INTERSECT/EXCEPT steps into the setOperations tree of the topmost
Query, so that leaf queries do not contain any setOperations.
However, it cannot thus flatten a subquery that also contains
WITH, ORDER BY, FOR UPDATE, or LIMIT. I (tgl) forgot that in
commit 07b4c48b6 and wrote an assertion in rule deparsing that
a leaf's setOperations would always be empty.
If it were nonempty then we would want to parenthesize the subquery
to ensure that the output represents the setop nesting correctly
(e.g. UNION below INTERSECT had better get parenthesized). So
rather than just removing the faulty Assert, let's change it into
an additional case to check to decide whether to add parens. We
don't expect that the additional case will ever fire, but it's
cheap insurance.
Man Zeng and Tom Lane
Discussion: https://postgr.es/m/tencent_7ABF9B1F23B0C77606FC5FE3@qq.com
M src/backend/utils/adt/ruleutils.c
file_fdw: Add REJECT_LIMIT option to file_fdw.
commit : 6c8f670323d22acb62104f7f8f1b1a078dadd255
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 20 Nov 2024 23:53:19 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 20 Nov 2024 23:53:19 +0900
Commit 4ac2a9bece introduced the REJECT_LIMIT option for the COPY
command. This commit extends the support for this option to file_fdw.
As well as REJECT_LIMIT option for COPY, this option limits
the maximum number of erroneous rows that can be skipped.
If the number of data type conversion errors exceeds this limit,
accessing the file_fdw foreign table will fail with an error,
even when on_error = 'ignore' is specified.
Since the CREATE/ALTER FOREIGN TABLE commands require foreign
table options to be single-quoted, this commit updates
defGetCopyRejectLimitOption() to handle also string value for them,
in addition to int64 value for COPY command option.
Author: Atsushi Torikoshi
Reviewed-by: Fujii Masao, Yugo Nagata, Kirill Reshke
Discussion: https://postgr.es/m/bab68a9fc502b12693f0755b6f35f327@oss.nttdata.com
M contrib/file_fdw/data/agg.bad
M contrib/file_fdw/expected/file_fdw.out
M contrib/file_fdw/file_fdw.c
M contrib/file_fdw/sql/file_fdw.sql
M doc/src/sgml/file-fdw.sgml
M src/backend/commands/copy.c
doc: Fix section of functions age(xid) and mxid_age(xid)
commit : 15afb7d61c142a9254a6612c6774aff4f358fb69
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 20 Nov 2024 14:20:52 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 20 Nov 2024 14:20:52 +0900
In 17~, age(xid) and mxid_age(xid) were listed as deprecated. Based on
the discussion that led to 48b5aa3143, this is not intentional as this
could break many existing monitoring queries. Note that vacuumdb also
uses both of them.
In 16, both functions were listed under "Control Data Functions", which
is incorrect, so let's move them to the list of functions related to
transaction IDs and snapshots.
Author: Bertrand Drouvot
Discussion: https://postgr.es/m/Zzr2zZFyeFKXWe8a@ip-10-97-1-34.eu-west-3.compute.internal
Discussion: https://postgr.es/m/20231114013224.4z6oxa6p6va33rxr@awork3.anarazel.de
Backpatch-through: 16
M doc/src/sgml/func.sgml
Compare collations before merging UNION operations.
commit : a43d7a8c7cce00e3fcf330e201a3a94f97097e64
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Nov 2024 18:26:19 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Nov 2024 18:26:19 -0500
In the dim past we figured it was okay to ignore collations
when combining UNION set-operation nodes into a single N-way
UNION operation. I believe that was fine at the time, but
it stopped being fine when we added nondeterministic collations:
the semantics of distinct-ness are affected by those. v17 made
it even less fine by allowing per-child sorting operations to
be merged via MergeAppend, although I think we accidentally
avoided any live bug from that.
Add a check that collations match before deciding that two
UNION nodes are equivalent. I also failed to resist the
temptation to comment plan_union_children() a little better.
Back-patch to all supported branches (v13 now), since they
all have nondeterministic collations.
Discussion: https://postgr.es/m/3605568.1731970579@sss.pgh.pa.us
M src/backend/optimizer/prep/prepunion.c
Improve error message for database object stats manipulation functions.
commit : c166454496876818216e222890ab1ae788551c42
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 20 Nov 2024 02:00:50 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 20 Nov 2024 02:00:50 +0900
Previously, database object statistics manipulation functions like
pg_set_relation_stats() reported unclear error and hint messages
when executed during recovery. These messages were "internal",
making it difficult for users to understand the issue:
ERROR: cannot acquire lock mode ShareUpdateExclusiveLock on database objects while recovery is in progress
HINT: Only RowExclusiveLock or less can be acquired on database objects during recovery.
This commit updates the error handling so that, if these functions
are called during recovery, they produce clearer messages:
ERROR: recovery is in progress
HINT: Statistics cannot be modified during recovery.
The related documentation has also been updated to explicitly
clarify that these functions are not available during recovery.
Author: Fujii Masao
Reviewed-by: Heikki Linnakangas, Maxim Orlov
Discussion: https://postgr.es/m/6d313829-5f56-4a28-ae4b-bd01bf1ae791@oss.nttdata.com
M doc/src/sgml/func.sgml
M src/backend/statistics/attribute_stats.c
M src/backend/statistics/relation_stats.c
libpq: Improve error message when parsing URI parameters and keywords
commit : a3699daea2026de324ed7cc7115c36d3499010d3
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 19 Nov 2024 13:27:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 19 Nov 2024 13:27:42 +0900
The error message showing up when parameters or keywords include too
many whitespaces was "trailing data found", which was confusing because
there was no hint about what was actually wrong.
Issue introduced in 430ce189fc45, hence there is no need for a
backpatch.
Author: Yushi Ogiwara
Reviewed-by: Fujii Masao, Tom Lane, Bruce Momjian
Discussion: https://postgr.es/m/645bd22a53c4da8a1bc7e1e52d9d3b52@oss.nttdata.com
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/t/001_uri.pl
doc: clarify pg_dump --no-comments meaning as SQL comments
commit : b38bac26e203ffb4130a6e0f27146cb8d4bbc9c0
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 18 Nov 2024 16:30:33 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 18 Nov 2024 16:30:33 -0500
Discussion: https://postgr.es/m/ZyjdAjEsXbFPkD3t@momjian.us
Backpatch-through: master
M doc/src/sgml/ref/pg_dump.sgml
doc: clarify text about combining row-level policies
commit : cffca3665d8238dddf7afb4a3404866e375195c5
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 18 Nov 2024 15:34:59 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 18 Nov 2024 15:34:59 -0500
Reported-by: splarv@ya.ru
Discussion: https://postgr.es/m/173045909386.700.9231055113418242392@wrigleys.postgresql.org
Backpatch-through: master
M doc/src/sgml/ddl.sgml
nbtree: consistently use minoff variable.
commit : 18ea6b3d0da357b5874ab9cf41350d4473b01ee9
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 18 Nov 2024 13:35:28 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 18 Nov 2024 13:35:28 -0500
This was arguably an oversight in commit 29b64d1de7, which moved this
code from nbtutils.c to its nbtsearch.c caller.
M src/backend/access/nbtree/nbtsearch.c
Improve some code format in gist.c
commit : c1c09007e219ae68d1f8428a54baf68ccc1f8683
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 18 Nov 2024 13:41:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 18 Nov 2024 13:41:10 +0900
Author: Tender Wang
Discussion: https://postgr.es/m/CAHewXNmD=K7XmsHq=L1SyyzZYvwU4oaMG9EKSSMe4OrXfykLzg@mail.gmail.com
M src/backend/access/gist/gist.c
Use pg_memory_is_all_zeros() in PageIsVerifiedExtended()
commit : 03a42c9652f8cc2c447840e39418b862c48fd41d
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 18 Nov 2024 11:44:11 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 18 Nov 2024 11:44:11 +0900
Relying on pg_memory_is_all_zeros(), which would apply SIMD instructions
when dealing with an aligned page, is proving to be at least three times
faster than the original size_t-based comparisons when checking if a
BLCKSZ page is full of zeros. Note that PageIsVerifiedExtended() is
called each time a page is read from disk, and making it faster is a
good thing.
Author: Bertrand Drouvot
Discussion: https://postgr.es/m/CAApHDvq7P-JgFhgtxUPqhavG-qSDVUhyWaEX9M8_MNorFEijZA@mail.gmail.com
M src/backend/storage/page/bufpage.c
Optimize pg_memory_is_all_zeros() in memutils.h
commit : 5be1dabd2ae0cf48d927aad363c4b65507e38b25
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 18 Nov 2024 10:08:20 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 18 Nov 2024 10:08:20 +0900
pg_memory_is_all_zeros() is currently implemented to do only a
byte-per-byte comparison. While being sufficient for its existing
callers for pgstats entries, it could lead to performance regressions
should it be used for larger memory areas, like 8kB blocks, or even
future commits.
This commit optimizes the implementation of this function to be more
efficient for larger sizes, written in a way so that compilers can
optimize the code. This is portable across 32b and 64b architectures.
The implementation handles three cases, depending on the size of the
input provided:
* If less than sizeof(size_t), do a simple byte-by-byte comparison.
* If between sizeof(size_t) and (sizeof(size_t) * 8 - 1):
** Phase 1: byte-by-byte comparison, until the pointer is aligned.
** Phase 2: size_t comparisons, with aligned pointers, up to the last
aligned location possible.
** Phase 3: byte-by-byte comparison, until the end location.
* If more than (sizeof(size_t) * 8) bytes, this is the same as case 2
except that an additional phase is placed between Phase 1 and Phase 2,
with 8 * sizeof(size_t) comparisons using bitwise OR, to encourage
compilers to use SIMD instructions if available.
The last improvement proves to be at least 3 times faster than the
size_t comparisons, which is something currently used for the all-zero
page check in PageIsVerifiedExtended().
The optimization tricks that would encourage the use of SIMD
instructions have been suggested by David Rowley.
Author: Bertrand Drouvot
Reviewed-by: Michael Paquier, Ranier Vilela
Discussion: https://postgr.es/m/CAApHDvq7P-JgFhgtxUPqhavG-qSDVUhyWaEX9M8_MNorFEijZA@mail.gmail.com
M src/include/utils/memutils.h
Fix per-session activation of ALTER {ROLE|DATABASE} SET role.
commit : 7b88529f4363994450bd4cd3c172006a8a77e222
author : Noah Misch <noah@leadboat.com>
date : Fri, 15 Nov 2024 20:39:56 -0800
committer: Noah Misch <noah@leadboat.com>
date : Fri, 15 Nov 2024 20:39:56 -0800
After commit 5a2fed911a85ed6d8a015a6bafe3a0d9a69334ae, the catalog state
resulting from these commands ceased to affect sessions. Restore the
longstanding behavior, which is like beginning the session with a SET
ROLE command. If cherry-picking the CVE-2024-10978 fixes, default to
including this, too. (This fixes an unintended side effect of fixing
CVE-2024-10978.) Back-patch to v12, like that commit. The release team
decided to include v12, despite the original intent to halt v12 commits
earlier this week.
Tom Lane and Noah Misch. Reported by Etienne LAFARGE.
Discussion: https://postgr.es/m/CADOZwSb0UsEr4_UTFXC5k7=fyyK8uKXekucd+-uuGjJsGBfxgw@mail.gmail.com
M src/backend/utils/init/miscinit.c
M src/backend/utils/misc/guc.c
M src/test/modules/unsafe_tests/Makefile
A src/test/modules/unsafe_tests/expected/setconfig.out
M src/test/modules/unsafe_tests/meson.build
A src/test/modules/unsafe_tests/sql/setconfig.sql
Fix a possibility of logical replication slot's restart_lsn going backwards.
commit : e5ed873b1b4a395b06a5599100383a95c9846e5b
author : Masahiko Sawada <msawada@postgresql.org>
date : Fri, 15 Nov 2024 17:06:11 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Fri, 15 Nov 2024 17:06:11 -0800
Previously LogicalIncreaseRestartDecodingForSlot() accidentally
accepted any LSN as the candidate_lsn and candidate_valid after the
restart_lsn of the replication slot was updated, so it potentially
caused the restart_lsn to move backwards.
A scenario where this could happen in logical replication is: after a
logical replication restart, based on previous candidate_lsn and
candidate_valid values in memory, the restart_lsn advances upon
receiving a subscriber acknowledgment. Then, logical decoding restarts
from an older point, setting candidate_lsn and candidate_valid based
on an old RUNNING_XACTS record. Subsequent subscriber acknowledgments
then update the restart_lsn to an LSN older than the current value.
In the reported case, after WAL files were removed by a checkpoint,
the retreated restart_lsn prevented logical replication from
restarting due to missing WAL segments.
This change essentially modifies the 'if' condition to 'else if'
condition within the function. The previous code had an asymmetry in
this regard compared to LogicalIncreaseXminForSlot(), which does
almost the same thing for different fields.
The WAL removal issue was reported by Hubert Depesz Lubaczewski.
Backpatch to all supported versions, since the bug exists since 9.4
where logical decoding was introduced.
Reviewed-by: Tomas Vondra, Ashutosh Bapat, Amit Kapila
Discussion: https://postgr.es/m/Yz2hivgyjS1RfMKs%40depesz.com
Discussion: https://postgr.es/m/85fff40e-148b-4e86-b921-b4b846289132%40vondra.me
Backpatch-through: 13
M src/backend/replication/logical/logical.c
Avoid assertion due to disconnected NFA sub-graphs in regex parsing.
commit : b69bdcee9c9cfd8550c4e847d035f441fcee7d01
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 15 Nov 2024 18:23:38 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 15 Nov 2024 18:23:38 -0500
In commit 08c0d6ad6 which introduced "rainbow" arcs in regex NFAs,
I didn't think terribly hard about what to do when creating the color
complement of a rainbow arc. Clearly, the complement cannot match any
characters, and I took the easy way out by just not building any arcs
at all in the complement arc set. That mostly works, but Nikolay
Shaplov found a case where it doesn't: if we decide to delete that
sub-NFA later because it's inside a "{0}" quantifier, delsub()
suffered an assertion failure. That's because delsub() relies on
the target sub-NFA being fully connected. That was always true
before, and the best fix seems to be to restore that property.
Hence, invent a new arc type CANTMATCH that can be generated in
place of an empty color complement, and drop it again later when we
start NFA optimization. (At that point we don't need to do delsub()
any more, and besides there are other cases where NFA optimization can
lead to disconnected subgraphs.)
It appears that this bug has no consequences in a non-assert-enabled
build: there will be some transiently leaked NFA states/arcs, but
they'll get cleaned up eventually. Still, we don't like assertion
failures, so back-patch to v14 where rainbow arcs were introduced.
Per bug #18708 from Nikolay Shaplov.
Discussion: https://postgr.es/m/18708-f94f2599c9d2c005@postgresql.org
M src/backend/regex/regc_color.c
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/include/regex/regguts.h
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Remove unnecessary backslash from CopyFrom() code.
commit : 9a70f67667da32f8009364e1096e80ce6996c13c
author : Fujii Masao <fujii@postgresql.org>
date : Sat, 16 Nov 2024 01:59:33 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Sat, 16 Nov 2024 01:59:33 +0900
Commit 4ac2a9bece accidentally added an unnecessary backslash
to CopyFrom() code. This commit removes it.
Author: Yugo Nagata
Reviewed-by: Tender Wang
Discussion: https://postgr.es/m/20241112114609.4175a2e175282edd1463dbc6@sraoss.co.jp
M src/backend/commands/copyfrom.c
Fix collation handling for foreign keys
commit : 9321d2fdf808e93095c4f86565dc8c8178675841
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 15 Nov 2024 14:52:28 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 15 Nov 2024 14:52:28 +0100
Allowing foreign keys where the referenced and the referencing columns
have collations with different notions of equality is problematic.
This can only happen when using nondeterministic collations, for
example, if the referencing column is case-insensitive and the
referenced column is not, or vice versa. It does not happen if both
collations are deterministic.
To show one example:
CREATE COLLATION case_insensitive (provider = icu, deterministic = false, locale = 'und-u-ks-level2');
CREATE TABLE pktable (x text COLLATE "C" PRIMARY KEY);
CREATE TABLE fktable (x text COLLATE case_insensitive REFERENCES pktable ON UPDATE CASCADE ON DELETE CASCADE);
INSERT INTO pktable VALUES ('A'), ('a');
INSERT INTO fktable VALUES ('A');
BEGIN; DELETE FROM pktable WHERE x = 'a'; TABLE fktable; ROLLBACK;
BEGIN; DELETE FROM pktable WHERE x = 'A'; TABLE fktable; ROLLBACK;
Both of these DELETE statements delete the one row from fktable. So
this means that one row from fktable references two rows in pktable,
which should not happen. (That's why a primary key or unique
constraint is required on pktable.)
When nondeterministic collations were implemented, the SQL standard
available to yours truly said that referential integrity checks should
be performed with the collation of the referenced column, and so
that's how we implemented it. But this turned out to be a mistake in
the SQL standard, for the same reasons as above, that was later
(SQL:2016) fixed to require both collations to be the same. So that's
what we are aiming for here.
We don't have to be quite so strict. We can allow different
collations if they are both deterministic. This is also good for
backward compatibility.
So the new rule is that the collations either have to be the same or
both deterministic. Or in other words, if one of them is
nondeterministic, then both have to be the same.
Users upgrading from before that have affected setups will need to
make changes to their schemas (i.e., change one or both collations in
affected foreign-key relationships) before the upgrade will succeed.
Some of the nice test cases for the previous situation in
collate.icu.utf8.sql are now obsolete. They are changed to just check
the error checking of the new rule. Note that collate.sql already
contained a test for foreign keys with different deterministic
collations.
A bunch of code in ri_triggers.c that added a COLLATE clause to
enforce the referenced column's collation can be removed, because both
columns now have to have the same notion of equality, so it doesn't
matter which one to use.
Reported-by: Paul Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: Jian He <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/78d824e0-b21e-480d-a252-e4b84bc2c24b@illuminatedcomputing.com
M doc/src/sgml/ref/create_table.sgml
M src/backend/commands/tablecmds.c
M src/backend/utils/adt/ri_triggers.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Avoid deleting critical WAL segments during pg_rewind
commit : 90bcc7c2db1d346e1153748e627c2f922b67eae7
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 15 Nov 2024 12:53:12 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 15 Nov 2024 12:53:12 +0100
Previously, in unlucky cases, it was possible for pg_rewind to remove
certain WAL segments from the rewound demoted primary. In particular
this happens if those files have been marked for archival (i.e., their
.ready files were created) but not yet archived; the newly promoted node
no longer has such files because of them having been recycled, but they
are likely critical for recovery in the demoted node. If pg_rewind
removes them, recovery is not possible anymore.
Fix this by maintaining a hash table of files in this situation in the
scan that looks for a checkpoint, which the decide_file_actions phase
can consult so that it knows to preserve them.
Backpatch to 14. The problem also exists in 13, but that branch was not
blessed with commit eb00f1d4bf96, so this patch is difficult to apply
there. Users of older releases will just have to continue to be extra
careful when rewinding.
Co-authored-by: Полина Бунгина (Polina Bungina) <bungina@gmail.com>
Co-authored-by: Alexander Kukushkin <cyberdemn@gmail.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Discussion: https://postgr.es/m/CAAtGL4AhzmBRsEsaDdz7065T+k+BscNadfTqP1NcPmsqwA5HBw@mail.gmail.com
M src/bin/pg_rewind/filemap.c
M src/bin/pg_rewind/filemap.h
M src/bin/pg_rewind/meson.build
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
A src/bin/pg_rewind/t/010_keep_recycled_wals.pl
M src/tools/pgindent/typedefs.list
Proper object locking for GRANT/REVOKE
commit : d31bbfb6590e586f731345960311861d5eb4c23f
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 15 Nov 2024 10:53:54 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 15 Nov 2024 10:53:54 +0100
Refactor objectNamesToOids() to use get_object_address() internally if
possible. Not only does this save a lot of code, it also allows us to
use the object locking provided by get_object_address() for
GRANT/REVOKE. There was previously a code comment that complained
about the lack of locking in objectNamesToOids(), which is now fixed.
The check in ExecGrant_Type_check() is obsolete because
get_object_address_type() already does the same check.
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/bf72b82c-124d-4efa-a484-bb928e9494e4@eisentraut.org
M src/backend/catalog/aclchk.c
M src/test/isolation/expected/intra-grant-inplace.out
jit: Stop emitting some unnecessary instructions
commit : cfd7f36c83cdcf6322c7ea0d1d9ef62bc2b00375
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 15 Nov 2024 10:06:36 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 15 Nov 2024 10:06:36 +0200
In EEOP_BOOL_AND_STEP* and EEOP_BOOL_OR_STEP*, we emitted pointlesss
store instructions to store to resnull/resvalue values that were just
loaded from the same fields in the previous instructions. They will
surely get optimized away by LLVM if any optimizations are enabled,
but it's better to not emit them in the first place. In
EEOP_BOOL_NOT_STEP, similar story with resnull.
In EEOP_NULLIF, when it returns NULL, there was also a redundant store
to resvalue just after storing a 0 to it. The value of resvalue
doesn't matter when resnull is set, so in fact even storing the 0 is
unnecessary, but I kept that because we tend to do that for general
tidiness.
Author: Xing Guo <higuoxing@gmail.com>
Reviewed-by: Andreas Karlsson <andreas@proxel.se>
Discussion: https://www.postgresql.org/message-id/CACpMh%2BC%3Dg13WdvzLRSponsVWGgxwDSMzQWM4Gz0heOyaA0-N6g@mail.gmail.com
M src/backend/jit/llvm/llvmjit_expr.c
Add an assertion in get_object_address()
commit : e468ec0fddcd93e282da6685b75d5687c2a87f2a
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 15 Nov 2024 08:42:59 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 15 Nov 2024 08:42:59 +0100
Some places declared a Relation before calling get_object_address()
only to assert that the relation is NULL after the call.
The new assertion allows passing NULL as the relation argument at
those places making the code cleaner and easier to understand.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://www.postgresql.org/message-id/ZzG34eNrT83W/Orz@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/catalog/objectaddress.c
M src/backend/commands/alter.c
Fix race conditions with drop of reused pgstats entries
commit : 818119afccd342823e5b638be51f64ba36221318
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 15 Nov 2024 11:31:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 15 Nov 2024 11:31:58 +0900
This fixes a set of race conditions with cumulative statistics where a
shared stats entry could be dropped while it should still be valid in
the event when it is reused: an entry may refer to a different object
but requires the same hash key. This can happen with various stats
kinds, like:
- Replication slots that compute internally an index number, for
different slot names.
- Stats kinds that use an OID in the object key, where a wraparound
causes the same key to be used if an OID is used for the same object.
- As of PostgreSQL 18, custom pgstats kinds could also be an issue,
depending on their implementation.
This issue is fixed by introducing a counter called "generation" in the
shared entries via PgStatShared_HashEntry, initialized at 0 when an
entry is created and incremented when the same entry is reused, to avoid
concurrent issues on drop because of other backends still holding a
reference to it. This "generation" is copied to the local copy that a
backend holds when looking at an object, then cross-checked with the
shared entry to make sure that the entry is not dropped even if its
"refcount" justifies that if it has been reused.
This problem could show up when a backend shuts down and needs to
discard any entries it still holds, causing statistics to be removed
when they should not, or even an assertion failure. Another report
involved a failure in a standby after an OID wraparound, where the
startup process would FATAL on a "can only drop stats once", stopping
recovery abruptly. The buildfarm has been sporadically complaining
about the problem, as well, but the window is hard to reach with the
in-core tests.
Note that the issue can be reproduced easily by adding a sleep before
dshash_find() in pgstat_release_entry_ref() to enlarge the problematic
window while repeating test_decoding's isolation test oldest_xmin a
couple of times, for example, as pointed out by Alexander Lakhin.
Reported-by: Alexander Lakhin, Peter Smith
Author: Kyotaro Horiguchi, Michael Paquier
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/CAA4eK1KxuMVyAryz_Vk5yq3ejgKYcL6F45Hj9ZnMNBS-g+PuZg@mail.gmail.com
Discussion: https://postgr.es/m/17947-b9554521ad963c9c@postgresql.org
Backpatch-through: 15
M src/backend/utils/activity/pgstat_shmem.c
M src/include/utils/pgstat_internal.h
Pass MyPMChildSlot as an explicit argument to child process
commit : 5b007868577ae18d419c01ee739e5674fa8b2acf
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 14 Nov 2024 16:12:32 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 14 Nov 2024 16:12:32 +0200
All the other global variables passed from postmaster to child have
the same value in all the processes, while MyPMChildSlot is more like
a parameter to each child process.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/a102f15f-eac4-4ff2-af02-f9ff209ec66f@iki.fi
M src/backend/postmaster/launch_backend.c
M src/backend/postmaster/postmaster.c
M src/backend/postmaster/syslogger.c
M src/include/postmaster/postmaster.h
Assign a child slot to every postmaster child process
commit : a78af0427015449269fb7d9d8c6057cfcb740149
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 14 Nov 2024 16:12:28 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 14 Nov 2024 16:12:28 +0200
Previously, only backends, autovacuum workers, and background workers
had an entry in the PMChildFlags array. With this commit, all
postmaster child processes, including all the aux processes, have an
entry. Dead-end backends still don't get an entry, though, and other
processes that don't touch shared memory will never mark their
PMChildFlags entry as active.
We now maintain separate freelists for different kinds of child
processes. That ensures that there are always slots available for
autovacuum and background workers. Previously, pre-authentication
backends could prevent autovacuum or background workers from starting
up, by using up all the slots.
The code to manage the slots in the postmaster process is in a new
pmchild.c source file. Because postmaster.c is just so large.
Assigning pmsignal slot numbers is now pmchild.c's responsibility.
This replaces the PMChildInUse array in pmsignal.c.
Some of the comments in postmaster.c still talked about the "stats
process", but that was removed in commit 5891c7a8ed. Fix those while
we're at it.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/a102f15f-eac4-4ff2-af02-f9ff209ec66f@iki.fi
M src/backend/bootstrap/bootstrap.c
M src/backend/postmaster/Makefile
M src/backend/postmaster/launch_backend.c
M src/backend/postmaster/meson.build
A src/backend/postmaster/pmchild.c
M src/backend/postmaster/postmaster.c
M src/backend/postmaster/syslogger.c
M src/backend/storage/ipc/pmsignal.c
M src/backend/storage/lmgr/proc.c
M src/backend/tcop/postgres.c
M src/include/postmaster/bgworker_internals.h
M src/include/postmaster/postmaster.h
M src/include/postmaster/syslogger.h
M src/include/storage/pmsignal.h
M src/tools/pgindent/typedefs.list
Kill dead-end children when there's nothing else left
commit : bb861414fea31073f27aaab75a0ceaf3638d7985
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 14 Nov 2024 16:12:04 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 14 Nov 2024 16:12:04 +0200
Previously, the postmaster would never try to kill dead-end child
processes, even if there were no other processes left. A dead-end
backend will eventually exit, when authentication_timeout expires, but
if a dead-end backend is the only thing that's preventing the server
from shutting down, it seems better to kill it immediately. It's
particularly important, if there was a bug in the early startup code
that prevented a dead-end child from timing out and exiting normally.
Includes a test for that case where a dead-end backend previously
prevented the server from shutting down.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/a102f15f-eac4-4ff2-af02-f9ff209ec66f@iki.fi
M src/backend/postmaster/postmaster.c
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/test/postmaster/meson.build
A src/test/postmaster/t/002_start_stop.pl
Replace postmaster.c's own backend type codes with BackendType
commit : 18d67a8d7d30884655d65910b82781d9360819a6
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 14 Nov 2024 16:06:16 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 14 Nov 2024 16:06:16 +0200
Introduce a separate BackendType for dead-end children, so that we
don't need a separate dead_end flag.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/a102f15f-eac4-4ff2-af02-f9ff209ec66f@iki.fi
M src/backend/postmaster/launch_backend.c
M src/backend/postmaster/postmaster.c
M src/backend/utils/activity/pgstat_io.c
M src/backend/utils/init/miscinit.c
M src/include/miscadmin.h
M src/tools/pgindent/typedefs.list
Remove a useless cast to (void *) in hash_search() call
commit : a274bbb1b3655ae122bba792c8fb9ae8f0ac5b21
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 14 Nov 2024 09:30:14 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 14 Nov 2024 09:30:14 +0100
This pattern was previously cleaned up in 54a177a948b, but a new
instance snuck in around the same time in 31966b151e6.
M src/backend/storage/buffer/localbuf.c
contrib/lo: Use SQL-standard function bodies
commit : 13e3796c906befb18c45d4361b2ca0c4f55845c1
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Nov 2024 13:23:11 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Nov 2024 13:23:11 +0900
Author: Ronan Dunklau
Discussion: https://postgr.es/m/3316564.aeNJFYEL58@aivenlaptop
M contrib/lo/Makefile
A contrib/lo/lo–1.1–1.2.sql
M contrib/lo/lo.control
M contrib/lo/meson.build
xml2: Add tests for functions xpath_nodeset() and xpath_list()
commit : 93f9b4a93fef5dad585531692f53e42d6e9f7caf
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Nov 2024 13:10:36 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Nov 2024 13:10:36 +0900
These two functions with their different argument lists have never been
tested in this module, so let's add something.
Author: Ronan Dunklau
Discussion: https://postgr.es/m/ZzMSJkiNZhimjXWx@paquier.xyz
M contrib/xml2/expected/xml2.out
M contrib/xml2/expected/xml2_1.out
M contrib/xml2/sql/xml2.sql
contrib/lo: Add test for function lo_oid()
commit : 3ef038fc4f763f8f6a77c5b3406ab7d837c51c4d
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Nov 2024 12:24:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Nov 2024 12:24:00 +0900
Author: Ronan Dunklau
Discussion: https://postgr.es/m/ZzMSJkiNZhimjXWx@paquier.xyz
M contrib/lo/expected/lo.out
M contrib/lo/sql/lo.sql
Add nbtree amgettuple return item function.
commit : 4e6e375b0039a2afc5830ec5a161536428767e96
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 13 Nov 2024 09:50:57 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 13 Nov 2024 09:50:57 -0500
This makes it easier to add precondition assertions. We now assert that
the last call to _bt_readpage succeeded, and that the current item index
is within the bounds of the currPos items array.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Discussion: https://postgr.es/m/CAH2-WznFkEs9K1PtNruti5JjawY-dwj+gkaEh_k1ZE+1xLLGkA@mail.gmail.com
M src/backend/access/nbtree/nbtsearch.c
Fix pg_upgrade's cross-version tests when old < 18
commit : 38c18710b37a2d0e343a8d6b6333cff31498459b
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 13 Nov 2024 11:06:44 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 13 Nov 2024 11:06:44 +0100
Because in the 18 cycle we turned checksums on by default with commit
04bec894a04c, and pg_upgrade fails if the setting doesn't match in old
and new clusters, the built-in cross-version pg_upgrade test is failing
if the old version is older than 18. Fix the script so that it creates
the old cluster with checksums enabled (-k) in cross-version scenarios.
This went unnoticed because the buildfarm doesn't use the same test code
for cross-version testing.
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/202411071838.7fgkb7uvavvz@alvherre.pgsql
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
configure.ac: Remove useless AC_SUBST
commit : f05b5e6346802d58f23d2ba8370091eab7aa5f63
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 13 Nov 2024 10:29:31 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 13 Nov 2024 10:29:31 +0100
No longer used since commit 805e431a386.
M configure
M configure.ac
doc: Update pg_constraint.conexclop docs for WITHOUT OVERLAPS
commit : f683ba0867dacee295eba2e31fa0689a64f7fbe4
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 13 Nov 2024 09:05:02 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 13 Nov 2024 09:05:02 +0100
Fixup for commit fc0438b4e80.
Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://www.postgresql.org/message-id/57ea0668-5205-426e-b934-efc89f2186c2@illuminatedcomputing.com
M doc/src/sgml/catalogs.sgml
doc: Add PERIOD to ALTER TABLE reference docs
commit : d56af4c882e28a4dd91f96b66db6e5861b7b3f86
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 13 Nov 2024 08:53:08 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 13 Nov 2024 08:53:08 +0100
Commit 89f908a6d0a documented foreign keys with PERIOD in the CREATE
TABLE docs, but not in ALTER TABLE. This commit adds the new syntax
to the ALTER TABLE docs.
Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://www.postgresql.org/message-id/57ea0668-5205-426e-b934-efc89f2186c2@illuminatedcomputing.com
M doc/src/sgml/ref/alter_table.sgml
doc: Small improvement in CREATE TABLE / PERIOD documentation
commit : 94daf80bd1a2fb078e3a12b80e00f6c578008a08
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 13 Nov 2024 08:51:23 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 13 Nov 2024 08:51:23 +0100
Use placeholders that are more consistent and match the description
better. Fixup for commit 89f908a6d0a.
M doc/src/sgml/ref/create_table.sgml
doc: Add WITHOUT OVERLAPS to ALTER TABLE reference docs
commit : bf621059500b9b46dcd728d27e7db94fa26a5d2f
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 13 Nov 2024 08:42:34 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 13 Nov 2024 08:42:34 +0100
Commit fc0438b4e80 documented WITHOUT OVERLAPS in the CREATE TABLE
docs, but not in ALTER TABLE. This commit adds the new syntax to the
ALTER TABLE docs.
Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://www.postgresql.org/message-id/57ea0668-5205-426e-b934-efc89f2186c2@illuminatedcomputing.com
M doc/src/sgml/ref/alter_table.sgml
Fix comment in injection_point.c
commit : d74b590983d18b7c5bdc1679ce85b1600871d874
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 13 Nov 2024 13:58:09 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 13 Nov 2024 13:58:09 +0900
InjectionPointEntry->name was described as a hash key, which was fine
when introduced in d86d20f0ba79, but it is not now.
Oversight in 86db52a5062a, that has changed the way injection points are
stored in shared memory from a hash table to an array.
Backpatch-through: 17
M src/backend/utils/misc/injection_point.c
Fix obsolete nbtree page reuse FSM comment.
commit : 3be30d007514d504b1d0554a20527ee5ba7a6cfb
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 12 Nov 2024 22:09:00 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 12 Nov 2024 22:09:00 -0500
Oversight in commit d088ba5a.
M src/backend/access/nbtree/nbtxlog.c
Count contrib/bloom index scans in pgstat view.
commit : 93063e2e429b03e8b06295ede0f0ebb5199b656a
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 12 Nov 2024 20:57:45 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 12 Nov 2024 20:57:45 -0500
Maintain the pg_stat_user_indexes.idx_scan pgstat counter during
contrib/Bloom index scans.
Oversight in commit 9ee014fc, which added the Bloom index contrib
module.
Author: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Reviewed-By: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/c48839d881388ee401a01807c686004d@oss.nttdata.com
Backpatch: 13- (all supported branches).
M contrib/bloom/blscan.c
Add missing word in comment
commit : bfeeb065ea2c870cf4d9dfcd552d23d72432e692
author : Amit Langote <amitlan@postgresql.org>
date : Tue, 12 Nov 2024 15:56:51 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Tue, 12 Nov 2024 15:56:51 +0900
Discussion: https://postgr.es/m/CA+HiwqFgdp8=0_gi+DU0fPWZbg7qY3KZ_c1Wj1DEvzXC4BCnMQ@mail.gmail.com
M src/include/storage/smgr.h
Silence compilers about extractNotNullColumn()
commit : ff239c3bf4e8cc8b758450e82ff698100aa8efc0
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 12 Nov 2024 11:35:43 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 12 Nov 2024 11:35:43 +0100
Multiple buildfarm animals warn that a newly added Assert() is
impossible to fail; remove it to avoid the noise. While at it, use
direct assignment to obtain the value we need, avoiding an unnecessary
memcpy().
(I decided to remove the "pfree" call for the detoasted short-datum;
because this is only used for DDL, it's not problematic to leak such a
small allocation.)
Noted by Tom Lane about 14e87ffa5c54.
Discussion: https://postgr.es/m/3649828.1731083171@sss.pgh.pa.us
M src/backend/catalog/pg_constraint.c
pg_freespacemap: Use SQL-standard function bodies
commit : 3f323eba89fb1ea2220cfe48ea7d9529fffe5cb6
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 12 Nov 2024 17:28:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 12 Nov 2024 17:28:03 +0900
72a5b1fc8804 was the piece missing for the conversion of this module.
pg_freespace is bumped to 1.3, with its function pg_freespace(regclass)
converted to this new style.
There are other modules in the tree that need a similar treatment; these
will be handled later.
Author: Tom Lane
Reviewed-by: Ronan Dunklau
Discussion: https://postgr.es/m/3395418.1618352794@sss.pgh.pa.us
M contrib/pg_freespacemap/Makefile
M contrib/pg_freespacemap/meson.build
A contrib/pg_freespacemap/pg_freespacemap–1.2–1.3.sql
M contrib/pg_freespacemap/pg_freespacemap.control
Fix arrays comparison in CompareOpclassOptions()
commit : db22b900244d3c51918acce44cbe5bb6f6507d32
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 12 Nov 2024 01:44:20 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 12 Nov 2024 01:44:20 +0200
The current code calls array_eq() and does not provide FmgrInfo. This commit
provides initialization of FmgrInfo and uses C collation as the safe option
for text comparison because we don't know anything about the semantics of
opclass options.
Backpatch to 13, where opclass options were introduced.
Reported-by: Nicolas Maus
Discussion: https://postgr.es/m/18692-72ea398df3ec6712%40postgresql.org
Backpatch-through: 13
M contrib/pg_trgm/expected/pg_trgm.out
M contrib/pg_trgm/sql/pg_trgm.sql
M src/backend/commands/indexcmds.c
Parallel workers use AuthenticatedUserId for connection privilege checks.
commit : 73c9f91a1b6d3753451993c5f06ba4b5ccc1f127
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Nov 2024 17:05:53 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Nov 2024 17:05:53 -0500
Commit 5a2fed911 had an unexpected side-effect: the parallel worker
launched for the new test case would fail if it couldn't use a
superuser-reserved connection slot. The reason that test failed
while all our pre-existing ones worked is that the connection
privilege tests in InitPostgres had been based on the superuserness
of the leader's AuthenticatedUserId, but after the rearrangements
of 5a2fed911 we were testing the superuserness of CurrentUserId,
which the new test case deliberately made to be a non-superuser.
This all seems very accidental and probably not the behavior we really
want, but a security patch is no time to be redesigning things.
Pending some discussion about desirable semantics, hack it so that
InitPostgres continues to pay attention to the superuserness of
AuthenticatedUserId when starting a parallel worker.
Nathan Bossart and Tom Lane, per buildfarm member sawshark.
Security: CVE-2024-10978
M src/backend/utils/init/postinit.c
Fix cross-version upgrade tests.
commit : c4252c9ef0afea48a59402901c16fc970d838721
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Nov 2024 13:57:21 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Nov 2024 13:57:21 -0500
TestUpgradeXversion knows how to make the main regression database's
references to pg_regress.so be version-independent. But it doesn't
do that for plperl's database, so that the C function added by
commit b7e3a52a8 is causing cross-version upgrade test failures.
Path of least resistance is to just drop the function at the end
of the new test.
In <= v14, also take the opportunity to clean up the generated
test files.
Security: CVE-2024-10979
M src/pl/plperl/expected/plperl_env.out
M src/pl/plperl/sql/plperl_env.sql
Avoid bizarre meson behavior with backslashes in command arguments.
commit : a34c33fd2278157de140608523598567bfb7a32f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Nov 2024 12:20:08 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Nov 2024 12:20:08 -0500
meson makes the backslashes in text2macro.pl's --strip argument
into forward slashes, effectively disabling comment stripping.
That hasn't caused us issues before, but it breaks the test case
for b7e3a52a8. We don't really need the pattern to be adjustable,
so just hard-wire it into the script instead.
Context: https://github.com/mesonbuild/meson/issues/1564
Security: CVE-2024-10979
M src/pl/plperl/GNUmakefile
M src/pl/plperl/meson.build
M src/pl/plperl/text2macro.pl
Fix improper interactions between session_authorization and role.
commit : 5a2fed911a85ed6d8a015a6bafe3a0d9a69334ae
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Nov 2024 10:29:54 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Nov 2024 10:29:54 -0500
The SQL spec mandates that SET SESSION AUTHORIZATION implies
SET ROLE NONE. We tried to implement that within the lowest-level
functions that manipulate these settings, but that was a bad idea.
In particular, guc.c assumes that it doesn't matter in what order
it applies GUC variable updates, but that was not the case for these
two variables. This problem, compounded by some hackish attempts to
work around it, led to some security-grade issues:
* Rolling back a transaction that had done SET SESSION AUTHORIZATION
would revert to SET ROLE NONE, even if that had not been the previous
state, so that the effective user ID might now be different from what
it had been.
* The same for SET SESSION AUTHORIZATION in a function SET clause.
* If a parallel worker inspected current_setting('role'), it saw
"none" even when it should see something else.
Also, although the parallel worker startup code intended to cope
with the current role's pg_authid row having disappeared, its
implementation of that was incomplete so it would still fail.
Fix by fully separating the miscinit.c functions that assign
session_authorization from those that assign role. To implement the
spec's requirement, teach set_config_option itself to perform "SET
ROLE NONE" when it sets session_authorization. (This is undoubtedly
ugly, but the alternatives seem worse. In particular, there's no way
to do it within assign_session_authorization without incompatible
changes in the API for GUC assign hooks.) Also, improve
ParallelWorkerMain to directly set all the relevant user-ID variables
instead of relying on some of them to get set indirectly. That
allows us to survive not finding the pg_authid row during worker
startup.
In v16 and earlier, this includes back-patching 9987a7bf3 which
fixed a violation of GUC coding rules: SetSessionAuthorization
is not an appropriate place to be throwing errors from.
Security: CVE-2024-10978
M src/backend/access/transam/parallel.c
M src/backend/commands/variable.c
M src/backend/utils/init/miscinit.c
M src/backend/utils/misc/guc.c
M src/include/miscadmin.h
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Ensure cached plans are correctly marked as dependent on role.
commit : cd7ab57532bb4fbf2e636b1f7d132e6e2d9ac5fc
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 11 Nov 2024 09:00:00 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 11 Nov 2024 09:00:00 -0600
If a CTE, subquery, sublink, security invoker view, or coercion
projection references a table with row-level security policies, we
neglected to mark the plan as potentially dependent on which role
is executing it. This could lead to later executions in the same
session returning or hiding rows that should have been hidden or
returned instead.
Reported-by: Wolfgang Walther
Reviewed-by: Noah Misch
Security: CVE-2024-10976
Backpatch-through: 12
M src/backend/executor/functions.c
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/rowsecurity.out
M src/test/regress/sql/rowsecurity.sql
M src/tools/pgindent/typedefs.list
Block environment variable mutations from trusted PL/Perl.
commit : b7e3a52a877cffb42ec7208232e30a8e44231e01
author : Noah Misch <noah@leadboat.com>
date : Mon, 11 Nov 2024 06:23:43 -0800
committer: Noah Misch <noah@leadboat.com>
date : Mon, 11 Nov 2024 06:23:43 -0800
Many process environment variables (e.g. PATH), bypass the containment
expected of a trusted PL. Hence, trusted PLs must not offer features
that achieve setenv(). Otherwise, an attacker having USAGE privilege on
the language often can achieve arbitrary code execution, even if the
attacker lacks a database server operating system user.
To fix PL/Perl, replace trusted PL/Perl %ENV with a tied hash that just
replaces each modification attempt with a warning. Sites that reach
these warnings should evaluate the application-specific implications of
proceeding without the environment modification:
Can the application reasonably proceed without the modification?
If no, switch to plperlu or another approach.
If yes, the application should change the code to stop attempting
environment modifications. If that's too difficult, add "untie
%main::ENV" in any code executed before the warning. For example,
one might add it to the start of the affected function or even to
the plperl.on_plperl_init setting.
In passing, link to Perl's guidance about the Perl features behind the
security posture of PL/Perl.
Back-patch to v12 (all supported versions).
Andrew Dunstan and Noah Misch
Security: CVE-2024-10979
M doc/src/sgml/plperl.sgml
M src/pl/plperl/GNUmakefile
A src/pl/plperl/expected/plperl_env.out
M src/pl/plperl/meson.build
M src/pl/plperl/plc_trusted.pl
A src/pl/plperl/sql/plperl_env.sql
M src/test/regress/regress.c
Doc: Add links to clarify the max_replication_slots.
commit : 220cea9411e6164b1a70f8749a806f155f3dc124
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 11 Nov 2024 15:24:40 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 11 Nov 2024 15:24:40 +0530
The GUC max_replication_slots has a different meaning for sending servers
and subscribers. Add cross-links in each section for easy reference.
Author: Tristan Partin
Discussion: https://postgr.es/m/D5FNEPMMFHFX.1OQBCML0TU5AH@partin.io
M doc/src/sgml/config.sgml
Add two attributes to pg_stat_database for parallel workers activity
commit : e7a9496de90657e2161f68b3a5a9b2d9b0b7bb07
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 11 Nov 2024 10:40:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 11 Nov 2024 10:40:48 +0900
Two attributes are added to pg_stat_database:
* parallel_workers_to_launch, counting the total number of parallel
workers that were planned to be launched.
* parallel_workers_launched, counting the total number of parallel
workers actually launched.
The ratio of both fields can provide hints that there are not enough
slots available when launching parallel workers, also useful when
pg_stat_statements is not deployed on an instance (i.e. cf54a2c00254).
This commit relies on de3a2ea3b264, that has added two fields to EState,
that get incremented when executing Gather or GatherMerge nodes.
A test is added in select_parallel, where parallel workers are spawned.
Bump catalog version.
Author: Benoit Lobréau
Discussion: https://postgr.es/m/783bc7f7-659a-42fa-99dd-ee0565644e25@dalibo.com
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/executor/execMain.c
M src/backend/utils/activity/pgstat_database.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql
libpq: Bail out during SSL/GSS negotiation errors
commit : bf8835ea9717df50230c12133cff1b486dcc57be
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 11 Nov 2024 10:19:52 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 11 Nov 2024 10:19:52 +0900
This commit changes libpq so that errors reported by the backend during
the protocol negotiation for SSL and GSS are discarded by the client, as
these may include bytes that could be consumed by the client and write
arbitrary bytes to a client's terminal.
A failure with the SSL negotiation now leads to an error immediately
reported, without a retry on any other methods allowed, like a fallback
to a plaintext connection.
A failure with GSS discards the error message received, and we allow a
fallback as it may be possible that the error is caused by a connection
attempt with a pre-11 server, GSS encryption having been introduced in
v12. This was a problem only with v17 and newer versions; older
versions discard the error message already in this case, assuming a
failure caused by a lack of support for GSS encryption.
Author: Jacob Champion
Reviewed-by: Peter Eisentraut, Heikki Linnakangas, Michael Paquier
Security: CVE-2024-10977
Backpatch-through: 12
M doc/src/sgml/protocol.sgml
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/t/005_negotiate_encryption.pl
pg_stat_statements: Avoid some locking during PGSS entry scans
commit : 5d4298e75f252b162008ad0475f29349023573cb
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 11 Nov 2024 09:02:30 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 11 Nov 2024 09:02:30 +0900
A single PGSS entry's spinlock is used to be able to modify "counters"
without holding pgss->lock exclusively, as mentioned at the top of
pg_stat_statements.c and within pgssEntry.
Within a single pgssEntry, stats_since and minmax_stats_since are never
modified without holding pgss->lock exclusively, so there is no need to
hold an entry's spinlock when reading stats_since and
minmax_stats_since, as done when scanning all the PGSS entries for
function calls of pg_stat_statements().
This also restores the consistency between the code and the comments
about the entry's spinlock usage. This change is a performance
improvement (it can be argued that this is a logic bug), so there is no
need for a backpatch. This saves two instructions from being read while
holding an entry's spinlock.
Author: Karina Litskevich
Reviewed-by: Michael Paquier, wenhui qiu
Discussion: https://postgr.es/m/CACiT8ibhCmzbcOxM0v4pRLH3abk-95LPkt7_uC2JMP+miPjxsg@mail.gmail.com
M contrib/pg_stat_statements/pg_stat_statements.c
jit: Remove obsolete LLVM version guard.
commit : 29d66b2d2f80dba0ac7fa5453d4da2dfbdd5ed2b
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 11 Nov 2024 11:46:37 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 11 Nov 2024 11:46:37 +1300
Commit 9044fc1d needed a version guard when back-patched, but it is
redundant in master as of commit 972c2cd2, and I accidentally left it
in there.
M src/include/jit/llvmjit_backport.h
Fix sign-compare warnings in pg_iovec.h.
commit : 0fa688406595091c4fb6d65f2d409366cc11394e
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 8 Nov 2024 16:11:08 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 8 Nov 2024 16:11:08 -0600
The code in question (pg_preadv() and pg_pwritev()) has been around
for a while, but commit 15c9ac3629 moved it to a header file. If
third-party code that includes this header file is built with
-Wsign-compare on a system without preadv() or pwritev(), warnings
ensue. This commit fixes said warnings by casting the result of
pg_pread()/pg_pwrite() to size_t, which should be safe because we
will have already checked for a negative value.
Author: Wolfgang Walther
Discussion: https://postgr.es/m/16989737-1aa8-48fd-8dfe-b7ada06509ab%40technowledgy.de
Backpatch-through: 17
M src/include/port/pg_iovec.h
Assert consistency of currPage that ended scan.
commit : caca6d8d276ce57473e97050d19dfe84e59482c8
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 8 Nov 2024 16:34:41 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 8 Nov 2024 16:34:41 -0500
When _bt_readnextpage is called with our nbtree parallel scan already
seized (i.e. when it is directly called by _bt_first), we never expect a
prior call to _bt_readpage for lastcurrblkno to already indicate that
the scan should end -- the _bt_first caller's blkno must always be read.
After all, the "prior" _bt_readpage call (the call for lastcurrblkno)
probably took place in some other backend (and it might not even have
finished by the time our backend reaches _bt_first/_bt_readnextpage).
Add a documenting assertion to the path where _bt_readnextpage ends the
parallel scan based on information about lastcurrblkno from so->currPos.
Assert that the most recent _bt_readpage call that set so->currPos is in
fact lastcurrblkno's _bt_readpage call.
Follow-up to bugfix commit b5ee4e52.
M src/backend/access/nbtree/nbtsearch.c
Move check for USE_AVX512_POPCNT_WITH_RUNTIME_CHECK.
commit : 4225276e25403e70de623ca9afd2c061cba24ccc
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 8 Nov 2024 14:25:28 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 8 Nov 2024 14:25:28 -0600
Unlike TRY_POPCNT_FAST, which is defined in pg_bitutils.h, this
macro is defined in c.h (via pg_config.h), so we can check for it
earlier and avoid some unnecessary #includes on systems that lack
AVX-512 support.
Oversight in commit f78667bd91.
Discussion: https://postgr.es/m/Zy5K5Qmlb3Z4dsd4%40nathan
M src/port/pg_popcount_avx512.c
Improve fix for not entering parallel mode when holding interrupts.
commit : b8df690492568d7852b799b4eff3274fbbd91e78
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 8 Nov 2024 13:42:01 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 8 Nov 2024 13:42:01 -0500
Commit ac04aa84a put the shutoff for this into the planner, which is
not ideal because it doesn't prevent us from re-using a previously
made parallel plan. Revert the planner change and instead put the
shutoff into InitializeParallelDSM, modeling it on the existing code
there for recovering from failure to allocate a DSM segment.
However, that code path is mostly untested, and testing a bit harder
showed there's at least one bug: ExecHashJoinReInitializeDSM is not
prepared for us to have skipped doing parallel DSM setup. I also
thought the Assert in ReinitializeParallelWorkers is pretty
ill-advised, and replaced it with a silent Min() operation.
The existing test case added by ac04aa84a serves fine to test this
version of the fix, so no change needed there.
Patch by me, but thanks to Noah Misch for the core idea that we
could shut off worker creation when !INTERRUPTS_CAN_BE_PROCESSED.
Back-patch to v12, as ac04aa84a was.
Discussion: https://postgr.es/m/CAC-SaSzHUKT=vZJ8MPxYdC_URPfax+yoA1hKTcF4ROz_Q6z0_Q@mail.gmail.com
M src/backend/access/transam/parallel.c
M src/backend/executor/nodeHashjoin.c
M src/backend/optimizer/plan/planner.c
Avoid nbtree parallel scan currPos confusion.
commit : b5ee4e52026b03b19ba47cde27bed9cf740576cc
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 8 Nov 2024 13:10:10 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 8 Nov 2024 13:10:10 -0500
Commit 1bd4bc85, which refactored nbtree sibling link traversal, made
_bt_parallel_seize reset the scan's currPos so that things were
consistent with the state of a serial backend moving between pages.
This overlooked the fact that _bt_readnextpage relied on the existing
currPos state to decide when to end the scan -- even though it came from
before the scan was seized. As a result of all this, parallel nbtree
scans could needlessly behave like full index scans.
To fix, teach _bt_readnextpage to explicitly allow the use of an already
read page's so->currPos when deciding whether to end the scan -- even
during parallel index scans (allow it consistently now). This requires
moving _bt_readnextpage's seizure of the scan to earlier in its loop.
That way _bt_readnextpage either deals with the true so->currPos state,
or an initialized-by-_bt_parallel_seize currPos state set from when the
scan was seized. Now _bt_steppage (the most important _bt_readnextpage
caller) takes the same uniform approach to setting up its call using
details taken from so->currPos -- regardless of whether the scan happens
to be parallel or serial.
The new loop structure in _bt_readnextpage is prone to getting confused
by P_NONE blknos set when the rightmost or leftmost page was reached.
We could avoid that by adding an explicit check, but that would be ugly.
Avoid this problem by teaching _bt_parallel_seize to end the parallel
scan instead of returning a P_NONE next block/blkno. Doing things this
way was arguably a missed opportunity for commit 1bd4bc85. It allows us
to remove a similar "blkno == P_NONE" check from _bt_first.
Oversight in commit 1bd4bc85, which refactored sibling link traversal
(as part of optimizing nbtree backward scan locking).
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Diagnosed-By: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Reviewed-By: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Discussion: https://postgr.es/m/f8efb9c0f8d1a71b44fd7f8e42e49c25@oss.nttdata.com
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtutils.c
Add pg_constraint rows for not-null constraints
commit : 14e87ffa5c543b5f30ead7413084c25f7735039f
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 8 Nov 2024 13:28:48 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 8 Nov 2024 13:28:48 +0100
We now create contype='n' pg_constraint rows for not-null constraints on
user tables. Only one such constraint is allowed for a column.
We propagate these constraints to other tables during operations such as
adding inheritance relationships, creating and attaching partitions and
creating tables LIKE other tables. These related constraints mostly
follow the well-known rules of conislocal and coninhcount that we have
for CHECK constraints, with some adaptations: for example, as opposed to
CHECK constraints, we don't match not-null ones by name when descending
a hierarchy to alter or remove it, instead matching by the name of the
column that they apply to. This means we don't require the constraint
names to be identical across a hierarchy.
The inheritance status of these constraints can be controlled: now we
can be sure that if a parent table has one, then all children will have
it as well. They can optionally be marked NO INHERIT, and then children
are free not to have one. (There's currently no support for altering a
NO INHERIT constraint into inheriting down the hierarchy, but that's a
desirable future feature.)
This also opens the door for having these constraints be marked NOT
VALID, as well as allowing UNIQUE+NOT NULL to be used for functional
dependency determination, as envisioned by commit e49ae8d3bc58. It's
likely possible to allow DEFERRABLE constraints as followup work, as
well.
psql shows these constraints in \d+, though we may want to reconsider if
this turns out to be too noisy. Earlier versions of this patch hid
constraints that were on the same columns of the primary key, but I'm
not sure that that's very useful. If clutter is a problem, we might be
better off inventing a new \d++ command and not showing the constraints
in \d+.
For now, we omit these constraints on system catalog columns, because
they're unlikely to achieve anything.
The main difference to the previous attempt at this (b0e96f311985) is
that we now require that such a constraint always exists when a primary
key is in the column; we didn't require this previously which had a
number of unpalatable consequences. With this requirement, the code is
easier to reason about. For example:
- We no longer have "throwaway constraints" during pg_dump. We needed
those for the case where a table had a PK without a not-null
underneath, to prevent a slow scan of the data during restore of the
PK creation, which was particularly problematic for pg_upgrade.
- We no longer have to cope with attnotnull being set spuriously in
case a primary key is dropped indirectly (e.g., via DROP COLUMN).
Some bits of code in this patch were authored by Jian He.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Author: Bernd Helmle <mailings@oopsware.de>
Reviewed-by: 何建 (jian he) <jian.universality@gmail.com>
Reviewed-by: 王刚 (Tender Wang) <tndrwang@gmail.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Discussion: https://postgr.es/m/202408310358.sdhumtyuy2ht@alvherre.pgsql
M contrib/sepgsql/expected/alter.out
M contrib/test_decoding/expected/ddl.out
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/alter_foreign_table.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_foreign_table.sgml
M doc/src/sgml/ref/create_table.sgml
M src/backend/catalog/heap.c
M src/backend/catalog/information_schema.sql
M src/backend/catalog/pg_constraint.c
M src/backend/commands/tablecmds.c
M src/backend/commands/typecmds.c
M src/backend/nodes/makefuncs.c
M src/backend/optimizer/util/plancat.c
M src/backend/parser/gram.y
M src/backend/parser/parse_utilcmd.c
M src/backend/replication/logical/relation.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/cache/relcache.c
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/psql/describe.c
M src/bin/psql/t/010_tab_completion.pl
M src/include/catalog/catversion.h
M src/include/catalog/heap.h
M src/include/catalog/pg_constraint.h
M src/include/nodes/makefuncs.h
M src/include/nodes/parsenodes.h
M src/include/utils/relcache.h
M src/test/modules/test_ddl_deparse/expected/alter_table.out
M src/test/modules/test_ddl_deparse/expected/create_table.out
M src/test/modules/test_ddl_deparse/test_ddl_deparse.c
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/cluster.out
M src/test/regress/expected/constraints.out
M src/test/regress/expected/create_table.out
M src/test/regress/expected/create_table_like.out
M src/test/regress/expected/foreign_data.out
M src/test/regress/expected/foreign_key.out
M src/test/regress/expected/generated_stored.out
M src/test/regress/expected/identity.out
M src/test/regress/expected/index_including.out
M src/test/regress/expected/indexing.out
M src/test/regress/expected/inherit.out
M src/test/regress/expected/publication.out
M src/test/regress/expected/replica_identity.out
M src/test/regress/expected/rowsecurity.out
M src/test/regress/sql/alter_table.sql
M src/test/regress/sql/constraints.sql
M src/test/regress/sql/create_table_like.sql
M src/test/regress/sql/index_including.sql
M src/test/regress/sql/indexing.sql
M src/test/regress/sql/inherit.sql
M src/test/regress/sql/replica_identity.sql
Disallow partitionwise join when collations don't match
commit : 075acdd93388c080c0fb0aca5723144ad7a56dac
author : Amit Langote <amitlan@postgresql.org>
date : Fri, 8 Nov 2024 16:27:24 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Fri, 8 Nov 2024 16:27:24 +0900
If the collation of any join key column doesn’t match the collation of
the corresponding partition key, partitionwise joins can yield incorrect
results. For example, rows that would match under the join key collation
might be located in different partitions due to the partitioning
collation. In such cases, a partitionwise join would yield different
results from a non-partitionwise join, so disallow it in such cases.
Reported-by: Tender Wang <tndrwang@gmail.com>
Author: Jian He <jian.universality@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Junwang Zhao <zhjwpku@gmail.com>
Discussion: https://postgr.es/m/CAHewXNno_HKiQ6PqyLYfuqDtwp7KKHZiH1J7Pqyz0nr+PS2Dwg@mail.gmail.com
Backpatch-through: 12
M src/backend/optimizer/util/relnode.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Disallow partitionwise grouping when collations don't match
commit : 90fe6251c816547b9cb4e1895dd8e6620bae94e1
author : Amit Langote <amitlan@postgresql.org>
date : Fri, 8 Nov 2024 16:07:22 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Fri, 8 Nov 2024 16:07:22 +0900
If the collation of any grouping column doesn’t match the collation of
the corresponding partition key, partitionwise grouping can yield
incorrect results. For example, rows that would be grouped under the
grouping collation may end up in different partitions under the
partitioning collation. In such cases, full partitionwise grouping
would produce results that differ from those without partitionwise
grouping, so disallowed that.
Partial partitionwise aggregation is still allowed, as the Finalize
step reconciles partition-level aggregates with grouping requirements
across all partitions, ensuring that the final output remains
consistent.
This commit also fixes group_by_has_partkey() by ensuring the
RelabelType node is stripped from grouping expressions when matching
them to partition key expressions to avoid false mismatches.
Bug: #18568
Reported-by: Webbo Han <1105066510@qq.com>
Author: Webbo Han <1105066510@qq.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Reviewed-by: Jian He <jian.universality@gmail.com>
Discussion: https://postgr.es/m/18568-2a9afb6b9f7e6ed3@postgresql.org
Discussion: https://postgr.es/m/tencent_9D9103CDA420C07768349CC1DFF88465F90A@qq.com
Discussion: https://postgr.es/m/CAHewXNno_HKiQ6PqyLYfuqDtwp7KKHZiH1J7Pqyz0nr+PS2Dwg@mail.gmail.com
Backpatch-through: 12
M src/backend/optimizer/plan/planner.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Fix inconsistent RestrictInfo serial numbers
commit : f00ab1fd156ea9c6283ffa0be5cddaafecfb0a35
author : Richard Guo <rguo@postgresql.org>
date : Fri, 8 Nov 2024 11:21:11 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Fri, 8 Nov 2024 11:21:11 +0900
When we generate multiple clones of the same qual condition to cope
with outer join identity 3, we need to ensure that all the clones get
the same serial number. To achieve this, we reset the
root->last_rinfo_serial counter each time we produce RestrictInfo(s)
from the qual list (see deconstruct_distribute_oj_quals). This
approach works only if we ensure that we are not changing the qual
list in any way that'd affect the number of RestrictInfos built from
it.
However, with b262ad440, an IS NULL qual on a NOT NULL column might
result in an additional constant-FALSE RestrictInfo. And different
versions of the same qual clause can lead to different conclusions
about whether it can be reduced to constant-FALSE. This would affect
the number of RestrictInfos built from the qual list for different
versions, causing inconsistent RestrictInfo serial numbers across
multiple clones of the same qual. This inconsistency can confuse
users of these serial numbers, such as rebuild_joinclause_attr_needed,
and lead to planner errors such as "ERROR: variable not found in
subplan target lists".
To fix, reset the root->last_rinfo_serial counter after generating the
additional constant-FALSE RestrictInfo.
Back-patch to v17 where the issue crept in. In v17, I failed to make
a test case that would expose this bug, so no test case for v17.
Author: Richard Guo
Discussion: https://postgr.es/m/CAMbWs4-B6kafn+LmPuh-TYFwFyEm-vVj3Qqv7Yo-69CEv14rRg@mail.gmail.com
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/util/joininfo.c
M src/test/regress/expected/predicate.out
M src/test/regress/sql/predicate.sql
Fix __attribute__((target(...))) usage.
commit : 41b98ddb77bf49433f174f03383533dd09bdefec
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 7 Nov 2024 15:27:32 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 7 Nov 2024 15:27:32 -0600
The commonly supported way to specify multiple target options is to
surround the entire list with quotes and to use a comma (with no
extra spaces) as the delimiter.
Oversight in commit f78667bd91.
Discussion: https://postgr.es/m/Zy0jya8nF8CPpv3B%40nathan
M config/c-compiler.m4
M configure
M meson.build
M src/port/pg_popcount_avx512.c
Use __attribute__((target(...))) for AVX-512 support.
commit : f78667bd910ed5af90d927bbef30b181bc798ef7
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 7 Nov 2024 13:58:43 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 7 Nov 2024 13:58:43 -0600
Presently, we check for compiler support for the required
intrinsics both with and without extra compiler flags (e.g.,
-mxsave), and then depending on the results of those checks, we
pick which files to compile with which flags. This is tedious and
complicated, and it results in unsustainable coding patterns such
as separate files for each portion of code may need to be built
with different compiler flags.
This commit introduces support for __attribute__((target(...))) and
uses it for the AVX-512 code. This simplifies both the
configure-time checks and the build scripts, and it allows us to
place the functions that use the intrinsics in files that we
otherwise do not want to build with special CPU instructions. We
are careful to avoid using __attribute__((target(...))) on
compilers that do not understand it, but we still perform the
configure-time checks in case the compiler allows using the
intrinsics without it (e.g., MSVC).
A similar change could likely be made for some of the CRC-32C code,
but that is left as a future exercise.
Suggested-by: Andres Freund
Reviewed-by: Raghuveer Devulapalli, Andres Freund
Discussion: https://postgr.es/m/20240731205254.vfpap7uxwmebqeaf%40awork3.anarazel.de
M config/c-compiler.m4
M configure
M configure.ac
M meson.build
M src/Makefile.global.in
M src/include/c.h
M src/makefiles/meson.build
M src/port/Makefile
M src/port/meson.build
M src/port/pg_popcount_avx512.c
D src/port/pg_popcount_avx512_choose.c
doc: Reword ALTER TABLE ATTACH restriction on NO INHERIT constraints
commit : f56a01ebdbd591ddd1428672e53a7c9bc4a5d99d
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 7 Nov 2024 14:06:24 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 7 Nov 2024 14:06:24 +0100
The previous wording is easy to read incorrectly; this change makes it
simpler, less ambiguous, and less prominent.
Backpatch to all live branches.
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/202411051201.zody6mld7vkw@alvherre.pgsql
M doc/src/sgml/ref/alter_table.sgml
Clarify a foreign key error message
commit : d7a2b5bd8718a6207fb364318d7f7cccdf6219c3
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 7 Nov 2024 11:13:06 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 7 Nov 2024 11:13:06 +0100
Clarify the message about type mismatch in foreign key definition to
indicate which column the referencing and which is the referenced one.
Reported-by: Jian He <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/CACJufxEL82ao-aXOa=d_-Xip0bix-qdSyNc9fcWxOdkEZFko8w@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/enum.out
M src/test/regress/expected/foreign_key.out
M src/test/regress/expected/without_overlaps.out
Remove an obsolete comment in gistinsert()
commit : 987027bcc04397f963bb961092a83067c0e406bb
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 7 Nov 2024 15:13:50 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 7 Nov 2024 15:13:50 +0900
This is inconsistent since 1f7ef548ec2e where the definition of
gistFormTuple() has changed.
Author: Tender Wang
Reviewed-by: Aleksander Alekseev
Discussion: https://postgr.es/m/CAHewXNkjU95_HdioDVU=5yBq_Xt=GfBv=Od-0oKtiA006pWW7Q@mail.gmail.com
M src/backend/access/gist/gist.c
Replicate generated columns when 'publish_generated_columns' is set.
commit : 7054186c4ebe24e63254651e2ae9b36efae90d4e
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 7 Nov 2024 08:58:49 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 7 Nov 2024 08:58:49 +0530
This patch builds on the work done in commit 745217a051 by enabling the
replication of generated columns alongside regular column changes through
a new publication parameter: publish_generated_columns.
Example usage:
CREATE PUBLICATION pub1 FOR TABLE tab_gencol WITH (publish_generated_columns = true);
The column list takes precedence. If the generated columns are specified
in the column list, they will be replicated even if
'publish_generated_columns' is set to false. Conversely, if generated
columns are not included in the column list (assuming the user specifies a
column list), they will not be replicated even if
'publish_generated_columns' is true.
Author: Vignesh C, Shubham Khanna
Reviewed-by: Peter Smith, Amit Kapila, Hayato Kuroda, Shlok Kyal, Ajin Cherian, Hou Zhijie, Masahiko Sawada
Discussion: https://postgr.es/m/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E@gmail.com
M doc/src/sgml/ddl.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/create_publication.sgml
M src/backend/catalog/pg_publication.c
M src/backend/commands/publicationcmds.c
M src/backend/replication/logical/proto.c
M src/backend/replication/pgoutput/pgoutput.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/psql/describe.c
M src/bin/psql/tab-complete.in.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_publication.h
M src/include/replication/logicalproto.h
M src/test/regress/expected/psql.out
M src/test/regress/expected/publication.out
M src/test/regress/sql/publication.sql
M src/test/subscription/t/011_generated.pl
M src/test/subscription/t/031_column_list.pl
Improve handling of empty query results in BackgroundPsql::query()
commit : 70291a3c66eca599fd9f59f7f6051432b2020f4b
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 7 Nov 2024 12:11:27 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 7 Nov 2024 12:11:27 +0900
A newline is not added at the end of an empty query result, causing the
banner of the hardcoded \echo to not be discarded. This would reflect
on scripts that expect an empty result by showing the "QUERY_SEPARATOR"
in the output returned back to the caller, which was confusing.
This commit changes BackgroundPsql::query() so as empty results are able
to work correctly, making the first newline before the banner optional,
bringing more flexibility.
Note that this change affects 037_invalid_database.pl, where three
queries generated an empty result, with the script relying on the data
from the hardcoded banner to exist in the expected output. These
queries are changed to use query_safe(), leading to a simpler script.
The author has also proposed a test in a different patch where empty
results would exist when using BackgroundPsql.
Author: Jacob Champion
Reviewed-by: Andrew Dunstan, Michael Paquier
Discussion: https://postgr.es/m/CAOYmi+=60deN20WDyCoHCiecgivJxr=98s7s7-C8SkXwrCfHXg@mail.gmail.com
M src/test/perl/PostgreSQL/Test/BackgroundPsql.pm
M src/test/recovery/t/037_invalid_database.pl
Find invalid databases during upgrade check stage
commit : f638aafd1ea8505750ca9546e0e0b2e4cf31027d
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 6 Nov 2024 15:40:52 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 6 Nov 2024 15:40:52 +0100
Before continuing with the check start by checking that all databases
allow connections to avoid a hard fail without proper error reporting.
Inspired by a larger patch by Thomas Krennwallner.
Discussion: https://postgr.es/m/f9315bf0-e03e-4490-9f0d-5b6f7a6d9908@postsubmeta.net
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
Remove unused variable
commit : 1e37cc6e2c1f39565275b5dabf0f301c2ece95e1
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 6 Nov 2024 15:11:14 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 6 Nov 2024 15:11:14 +0100
The low variable has not been used since it was added in d168b666823
and can be safely removed. The variable is present in the Sedgewick
paper "Analysis of Shellsort and Related Algorithms" as a parameter
to the shellsort function, but our implementation does not use it.
Remove to improve readability of the code.
Author: Koki Nakamura <btnakamurakoukil@oss.nttdata.com>
Discussion: https://postgr.es/m/8aeb7b3eda53ca4c65fbacf8f43628fb@oss.nttdata.com
M src/backend/access/heap/heapam.c
doc: Remove event trigger firing matrix
commit : a0be94067e76c81a2bbba6a98b584b870024e363
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 6 Nov 2024 13:42:27 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 6 Nov 2024 13:42:27 +0100
This is difficult to maintain accurately, and it was probably already
somewhat incorrect, especially in the sql_drop and table_rewrite
categories.
The prior section already documented which DDL commands are *not*
supported (which was also slightly outdated), so let's expand that a
bit and just rely on that instead of listing out each command in full
detail.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Jian He <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CACJufxE_UAuxcM08BW5oVsg34v0cFWoEt8yBa5xSAoKLmL6LTQ%40mail.gmail.com
M doc/src/sgml/event-trigger.sgml
M src/backend/commands/event_trigger.c
M src/backend/tcop/utility.c
Monkey-patch LLVM code to fix ARM relocation bug.
commit : 9044fc1d45a0212fd123bd8f364eac058f60fed7
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 6 Nov 2024 22:04:44 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 6 Nov 2024 22:04:44 +1300
Supply a new memory manager for RuntimeDyld, to avoid crashes in
generated code caused by memory placement that can overflow a 32 bit
data type. This is a drop-in replacement for the
llvm::SectionMemoryManager class in the LLVM library, with Michael
Smith's proposed fix from
https://www.github.com/llvm/llvm-project/pull/71968.
We hereby slurp it into our own source tree, after moving into a new
namespace llvm::backport and making some minor adjustments so that it
can be compiled with older LLVM versions as far back as 12. It's harder
to make it work on even older LLVM versions, but it doesn't seem likely
that people are really using them so that is not investigated for now.
The problem could also be addressed by switching to JITLink instead of
RuntimeDyld, and that is the LLVM project's recommended solution as
the latter is about to be deprecated. We'll have to do that soon enough
anyway, and then when the LLVM version support window advances far
enough in a few years we'll be able to delete this code. Unfortunately
that wouldn't be enough for PostgreSQL today: in most relevant versions
of LLVM, JITLink is missing or incomplete.
Several other projects have already back-ported this fix into their fork
of LLVM, which is a vote of confidence despite the lack of commit into
LLVM as of today. We don't have our own copy of LLVM so we can't do
exactly what they've done; instead we have a copy of the whole patched
class so we can pass an instance of it to RuntimeDyld.
The LLVM project hasn't chosen to commit the fix yet, and even if it
did, it wouldn't be back-ported into the releases of LLVM that most of
our users care about, so there is not much point in waiting any longer
for that. If they make further changes and commit it to LLVM 19 or 20,
we'll still need this for older versions, but we may want to
resynchronize our copy and update some comments.
The changes that we've had to make to our copy can be seen by diffing
our SectionMemoryManager.{h,cpp} files against the ones in the tree of
the pull request. Per the LLVM project's license requirements, a copy
is in SectionMemoryManager.LICENSE.
This should fix the spate of crash reports we've been receiving lately
from users on large memory ARM systems.
Back-patch to all supported releases.
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Co-authored-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se> (license aspects)
Reported-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/CAO6_Xqr63qj%3DSx7HY6ZiiQ6R_JbX%2B-p6sTPwDYwTWZjUmjsYBg%40mail.gmail.com
M src/backend/jit/llvm/Makefile
A src/backend/jit/llvm/SectionMemoryManager.LICENSE
A src/backend/jit/llvm/SectionMemoryManager.cpp
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_wrap.cpp
M src/backend/jit/llvm/meson.build
A src/include/jit/SectionMemoryManager.h
M src/include/jit/llvmjit.h
A src/include/jit/llvmjit_backport.h
M src/tools/pginclude/headerscheck
M src/tools/pgindent/exclude_file_patterns
Remove unused #include's from bin .c files
commit : ecb5af779877ad3f84e60112ecf0b138e97b847c
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 6 Nov 2024 11:06:58 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 6 Nov 2024 11:06:58 +0100
as determined by IWYU
Similar to commit dbbca2cf299, but for bin and some related files.
Discussion: https://www.postgresql.org/message-id/flat/0df1d5b1-8ca8-4f84-93be-121081bde049%40eisentraut.org
M src/bin/pg_basebackup/pg_createsubscriber.c
M src/bin/pg_basebackup/pg_recvlogical.c
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_checksums/pg_checksums.c
M src/bin/pg_combinebackup/copy_file.h
M src/bin/pg_combinebackup/pg_combinebackup.c
M src/bin/pg_config/pg_config.c
M src/bin/pg_dump/common.c
M src/bin/pg_dump/compress_io.c
M src/bin/pg_dump/filter.c
M src/bin/pg_dump/parallel.c
M src/bin/pg_dump/pg_backup_custom.c
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_backup_utils.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump_sort.c
M src/bin/pg_dump/pg_restore.c
M src/bin/pg_rewind/datapagemap.h
M src/bin/pg_rewind/libpq_source.c
M src/bin/pg_rewind/local_source.c
M src/bin/pg_rewind/pg_rewind.h
M src/bin/pg_test_fsync/pg_test_fsync.c
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/relfilenumber.c
M src/bin/pg_walsummary/pg_walsummary.c
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/help.c
M src/bin/scripts/clusterdb.c
M src/bin/scripts/common.c
M src/bin/scripts/reindexdb.c
M src/common/controldata_utils.c
M src/common/string.c
M src/common/unicode_case.c
M src/fe_utils/astreamer_file.c
M src/fe_utils/astreamer_gzip.c
M src/fe_utils/astreamer_lz4.c
Extend Cluster.pm's background_psql() to be able to start asynchronously
commit : ba08edb065459fca62191c020362022efd42b522
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 6 Nov 2024 15:31:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 6 Nov 2024 15:31:14 +0900
This commit extends the constructor routine of BackgroundPsql.pm with a
new "wait" parameter. If set to 0, the routine returns without waiting
for psql to start, ready to consume input.
background_psql() in Cluster.pm gains the same "wait" parameter. The
default behavior is still to wait for psql to start. It becomes now
possible to not wait, giving to TAP scripts the possibility to perform
actions between a BackgroundPsql startup and its wait_connect() call.
Author: Jacob Champion
Discussion: https://postgr.es/m/CAOYmi+=60deN20WDyCoHCiecgivJxr=98s7s7-C8SkXwrCfHXg@mail.gmail.com
M src/test/perl/PostgreSQL/Test/BackgroundPsql.pm
M src/test/perl/PostgreSQL/Test/Cluster.pm
Fix hypothetical bug in ExprState building for hashing
commit : 87f81a55632107006e5006a2d0bc5272ebaeca64
author : David Rowley <drowley@postgresql.org>
date : Wed, 6 Nov 2024 09:16:00 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 6 Nov 2024 09:16:00 +1300
adf97c156 gave ExprStates the ability to hash expressions and return a
single hash value. That commit supports seeding the hash value with an
initial value to have that blended into the final hash value.
Here we fix a hypothetical bug where if there are zero expressions to
hash, the initial value is stored in the wrong location. The existing
code stored the initial value in an intermediate location expecting that
when the expressions were hashed that those steps would store the final
hash value in the ExprState.resvalue field. However, that wouldn't happen
when there are zero expressions to hash. The correct thing to do instead
is to have a special case for zero expressions and when we hit that case,
store the initial value directly in the ExprState.resvalue. The reason
that this is a hypothetical bug is that no code currently calls
ExecBuildHash32Expr passing a non-zero initial value.
Discussion: https://postgr.es/m/CAApHDvpMAL_zxbMRr1LOex3O7Y7R7ZN2i8iUFLQhqQiJMAg3qw@mail.gmail.com
M src/backend/executor/execExpr.c
Add a Git .mailmap file
commit : 7bdaa4b54208afe0d0fc691877da920016db6987
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 5 Nov 2024 13:56:02 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 5 Nov 2024 13:56:02 +0100
This adds a Git .mailmap to unify spellings of committer names.
Discussion: https://postgr.es/m/76773029-A7AD-4BAF-AFC2-E511D26E866D@yesql.se
A .mailmap
Silence meson warning about PG_TEST_EXTRA in src/Makefile.global.in
commit : e54688030cfe6fd2b3f9b7f437f2f3a4b75a6b35
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 5 Nov 2024 12:25:25 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 5 Nov 2024 12:25:25 +0200
Commit 99b937a44f introduced this warning when you run "meson setup":
Configuring Makefile.global using configuration
../src/meson.build:31: WARNING: The variable(s) 'PG_TEST_EXTRA' in the input file 'src/Makefile.global.in' are not present in the given configuration data.
To fix, add PG_TEST_EXTRA to the list of variables that are not needed
in the makefiles generated by meson. In meson builds, the makefiles
are only used for PGXS, not for building or testing the server itself.
Reported-by: Peter Eisentraut
Discussion: https://www.postgresql.org/message-id/5c380997-e270-425a-9542-e4ef36a285de@eisentraut.org
M src/makefiles/meson.build
Clear padding of PgStat_HashKey when handling pgstats entries
commit : 7d85d87f4d5c35fd5b2d38adaef63dfbfa542ccc
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 5 Nov 2024 09:39:43 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 5 Nov 2024 09:39:43 +0900
PgStat_HashKey is currently initialized in a way that could result in
random data if the structure has any padding bytes. The structure
has no padding bytes currently, fortunately, but it could become a
problem should the structure change at some point in the future.
The code is changed to use some memset(0) so as any padding would be
handled properly, as it would be surprising to see random failures in
the pgstats entry lookups. PgStat_HashKey is a structure internal to
pgstats, and an ABI change could be possible in the scope of a bug fix,
so backpatch down to 15 where this has been introduced.
Author: Bertrand Drouvot
Reviewed-by: Jelte Fennema-Nio, Michael Paquier
Discussion: https://postgr.es/m/Zyb7RW1y9dVfO0UH@ip-10-97-1-34.eu-west-3.compute.internal
Backpatch-through: 15
M src/backend/utils/activity/pgstat.c
M src/backend/utils/activity/pgstat_shmem.c
Use portable diff options in pg_bsd_indent's regression test.
commit : 0704aed0d851abe9ec297dbd009951f0feeb12aa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Nov 2024 18:08:48 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Nov 2024 18:08:48 -0500
We had been using "diff -upd", which evidently works for most people,
but Solaris's diff doesn't like it. (We'd not noticed because the
Solaris buildfarm animals weren't running this test until they were
upgraded to the latest buildfarm client script.) Change to "diff -U3"
which is what pg_regress has used for ages.
Per buildfarm (and off-list discussion with Noah Misch).
Back-patch to v16 where this test was added. In v16,
also back-patch the relevant part of 628c1d1f2 so that
the test script looks about the same in all branches.
M src/tools/pg_bsd_indent/t/001_pg_bsd_indent.pl
Revert pg_wal_replay_wait() stored procedure
commit : 3a7ae6b3d91e0d011dba1eb8a29e1836c6a33c75
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 4 Nov 2024 22:43:08 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 4 Nov 2024 22:43:08 +0200
This commit reverts 3c5db1d6b0, and subsequent improvements and fixes
including 8036d73ae3, 867d396ccd, 3ac3ec580c, 0868d7ae70, 85b98b8d5a,
2520226c95, 014f9f34d2, e658038772, e1555645d7, 5035172e4a, 6cfebfe88b,
73da6b8d1b, and e546989a26.
The reason for reverting is a set of remaining issues. Most notably, the
stored procedure appears to need more effort than the utility statement
to turn the backend into a "snapshot-less" state. This makes an approach
to use stored procedures questionable.
Catversion is bumped.
Discussion: https://postgr.es/m/Zyhj2anOPRKtb0xW%40paquier.xyz
M doc/src/sgml/func.sgml
M src/backend/access/transam/Makefile
M src/backend/access/transam/meson.build
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogfuncs.c
M src/backend/access/transam/xlogrecovery.c
D src/backend/access/transam/xlogwait.c
M src/backend/catalog/system_functions.sql
M src/backend/lib/pairingheap.c
M src/backend/storage/ipc/ipci.c
M src/backend/storage/lmgr/proc.c
M src/backend/tcop/pquery.c
M src/backend/utils/activity/wait_event_names.txt
D src/include/access/xlogwait.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/lib/pairingheap.h
M src/include/storage/lwlocklist.h
M src/test/recovery/meson.build
D src/test/recovery/t/043_wal_replay_wait.pl
M src/tools/pgindent/typedefs.list
doc: use more accurate URL for bug reporting
commit : 3293b718a01310c8ce765ace3cb15efcb956a84e
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 4 Nov 2024 15:07:38 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 4 Nov 2024 15:07:38 -0500
Reported-by: nat@makarevitch.org
Discussion: https://postgr.es/m/172947609746.699.14488791149769110078@wrigleys.postgresql.org
Backpatch-through: master
M doc/src/sgml/problems.sgml
pg_basebackup, pg_receivewal: fix failure to find password in ~/.pgpass.
commit : b1008c1f01ff5cd35d2394bde2532c40781e696f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Nov 2024 14:36:04 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Nov 2024 14:36:04 -0500
Sloppy refactoring in commit cca97ce6a caused these programs
to pass dbname = NULL to libpq if there was no "--dbname" switch
on the command line, where before "replication" would be passed.
This didn't break things completely, because the source server doesn't
care about the dbname specified for a physical replication connection.
However, it did cause libpq to fail to match a ~/.pgpass entry that
has "replication" in the dbname field. Restore the previous behavior
of passing "replication".
Also, closer inspection shows that if you do specify a dbname
in the connection string, that is what will be matched to ~/.pgpass,
not "replication". This was the pre-existing behavior so we should
not change it, but the SGML docs were pretty misleading about it.
Improve that.
Per bug #18685 from Toshi Harada. Back-patch to v17 where the
error crept in.
Discussion: https://postgr.es/m/18685-fee2dd142b9688f1@postgresql.org
Discussion: https://postgr.es/m/2702546.1730740456@sss.pgh.pa.us
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_receivewal.sgml
M src/bin/pg_basebackup/streamutil.c
doc: remove check of SVG files, since they are derived
commit : 32d07a000ffa9f5f6fb418a23d1bced22f20481d
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 4 Nov 2024 14:10:11 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 4 Nov 2024 14:10:11 -0500
revert of change from commit 641a5b7a144
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/2c5dd601-b245-4092-9c27-6d1ad51609df@eisentraut.org
Backpatch-through: master
M doc/src/sgml/Makefile
pg_dump: provide a stable sort order for rules.
commit : 350e6b8ea86c22c0b95c2e32a4e8d109255b5596
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Nov 2024 13:30:30 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Nov 2024 13:30:30 -0500
Previously, we sorted rules by schema name and then rule name;
if that wasn't unique, we sorted by rule OID. This can be
problematic for comparing dumps from databases with different
histories, especially since certain rule names like "_RETURN"
are very common. Let's make the sort key schema name, rule name,
table name, which should be unique. (This is the same behavior
we've long used for triggers and RLS policies.)
Andreas Karlsson
Discussion: https://postgr.es/m/b4e468d8-0cd6-42e6-ac8a-1d6afa6e0cf1@proxel.se
M src/bin/pg_dump/pg_dump_sort.c
Fix typo in comment of gistdoinsert().
commit : 215f7af27d622162a1e33883f71697af374b98cf
author : Masahiko Sawada <msawada@postgresql.org>
date : Mon, 4 Nov 2024 10:21:59 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Mon, 4 Nov 2024 10:21:59 -0800
Author: Tender Wang
Reviewed-by: Masahiko Sawada
Discussion: https://postgr.es/m/CAHewXN%3D3sH2sNw4nC3QGCEVw1Lftmw9m5y1Xje0bXK6ApDrsPQ%40mail.gmail.com
M src/backend/access/gist/gist.c
Fix obsolete _bt_first comments.
commit : 846cfe0dcc0281fd9064798ac3539e20379b9b91
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 4 Nov 2024 12:43:54 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 4 Nov 2024 12:43:54 -0500
_bt_first doesn't necessarily hold onto a buffer pin on success exit.
Fix header comments that claimed that we'll always hold onto a pin.
Oversight in commit 2ed5b87f96.
M src/backend/access/nbtree/nbtsearch.c
docs: Consistently use <optional> to indicate optional parameters
commit : 0d829703363ba70844ff545ccc73befb2a7fed2e
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 18:28:40 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 18:28:40 +0200
Some functions were using square brackets instead, replace them all
with <optional>.
Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Reviewed-by: jian he <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CACJufxFfUbSph5UUSsZbL4SitbuPuW%3DEccpKgEaZrjtrPPuadQ@mail.gmail.com
M doc/src/sgml/func.sgml
nbtree: Remove useless 'strat' local variable.
commit : b6558e4f837e5c8f923e405ef6ab957ac2bd095a
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 4 Nov 2024 11:04:30 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 4 Nov 2024 11:04:30 -0500
Remove a local variable that was used to avoid overwriting strat_total
with the = operator strategy when a >= operator strategy key was already
included in the initial positioning/insertion scan keys by _bt_first
(for backwards scans it would have to be a <= key that was included).
_bt_first's strat_total local variable now simply tracks the operator
strategy of the final scan key that was included in the scan's insertion
scan key (barring the case where the !used_all_subkeys row compare path
adjusts strat_total in its own way).
_bt_first already treated >= keys (or <= keys) as = keys for initial
positioning purposes. There is no good reason to remember that that was
what happened; no later _bt_first step cares about the distinction.
Note, in particular, that the insertion scan key's 'nextkey' and
'backward' fields will be initialized the same way regardless.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CAH2-Wz=PKR6rB7qbx+Vnd7eqeB5VTcrW=iJvAsTsKbdG+kW_UA@mail.gmail.com
M src/backend/access/nbtree/nbtsearch.c
Split ProcSleep function into JoinWaitQueue and ProcSleep
commit : 3c0fd64fec8ed6fa3987c33f076fcffbc3f268c3
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 17:59:24 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 17:59:24 +0200
Split ProcSleep into two functions: JoinWaitQueue and ProcSleep.
JoinWaitQueue is called while holding the partition lock, and inserts
the current process to the wait queue, while ProcSleep() does the
actual sleeping. ProcSleep() is now called without holding the
partition lock, and it no longer re-acquires the partition lock before
returning. That makes the wakeup a little cheaper. Once upon a time,
re-acquiring the partition lock was needed to prevent a signal handler
from longjmping out at a bad time, but these days our signal handlers
just set flags, and longjmping can only happen at points where we
explicitly run CHECK_FOR_INTERRUPTS().
If JoinWaitQueue detects an "early deadlock" before even joining the
wait queue, it returns without changing the shared lock entry, leaving
the cleanup of the shared lock entry to the caller. This makes the
handling of an early deadlock the same as the dontWait=true case.
One small user-visible side-effect of this refactoring is that we now
only set the 'ps' title to say "waiting" when we actually enter the
sleep, not when the lock is skipped because dontWait=true, or when a
deadlock is detected early before entering the sleep.
This eliminates the 'lockAwaited' global variable in proc.c, which was
largely redundant with 'awaitedLock' in lock.c
Note: Updating the local lock table is now the caller's responsibility.
JoinWaitQueue and ProcSleep are now only responsible for modifying the
shared state. Seems a little nicer that way.
Based on Thomas Munro's earlier patch and observation that ProcSleep
doesn't really need to re-acquire the partition lock.
Reviewed-by: Maxim Orlov
Discussion: https://www.postgresql.org/message-id/7c2090cd-a72a-4e34-afaa-6dd2ef31440e@iki.fi
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
M src/backend/tcop/postgres.c
M src/include/storage/lock.h
M src/include/storage/proc.h
pg_combinebackup: Error if incremental file exists in full backup.
commit : 0b1765d959508050dba023099e36325cfeec3ff9
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 4 Nov 2024 10:11:05 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 4 Nov 2024 10:11:05 -0500
Suppose that you run a command like "pg_combinebackup b1 b2 -o output",
but both b1 and b2 contain an INCREMENTAL.$something file in a directory
that is expected to contain relation files. This is an error, but the
previous code would not detect the problem and instead write a garbage
full file named $something to the output directory. This commit adds
code to detect the error and a test case to verify the behavior.
It's difficult to imagine that this will ever happen unless someone
is intentionally trying to break incremental backup, but per discussion,
let's consider that the lack of adequate sanity checking in this area is
a bug and back-patch to v17, where incremental backup was introduced.
Patch by me, reviewed by Bertrand Drouvot and Amul Sul.
Discussion: http://postgr.es/m/CA+TgmoaD7dBYPqe7kMtO0dyto7rd0rUh7joh=JPUSaFszKY6Pg@mail.gmail.com
M src/bin/pg_combinebackup/meson.build
M src/bin/pg_combinebackup/reconstruct.c
A src/bin/pg_combinebackup/t/009_no_full_file.pl
pg_combinebackup: When reconstructing, avoid double slash in filename.
commit : 6c24801b1717ff68695647068a5ad8ff88f7639d
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 4 Nov 2024 09:55:02 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 4 Nov 2024 09:55:02 -0500
This function is always called with a relative_path that ends in a
slash, so there's no need to insert a second one. So, don't. Instead,
add an assertion to verify that nothing gets broken in the future, and
adjust the comments.
While this is not a critical bug, the duplicate slash is visible in
error messages, which could create confusion, so back-patch to v17.
This is also better in that it keeps the code consistent across
branches.
Patch by me, reviewed by Bertrand Drouvot and Amul Sul.
Discussion: http://postgr.es/m/CA+TgmoaD7dBYPqe7kMtO0dyto7rd0rUh7joh=JPUSaFszKY6Pg@mail.gmail.com
M src/bin/pg_combinebackup/reconstruct.c
doc: fix typo in mvcc clarification in commit 2fa255ce9b9
commit : 7ac744e72c842f528bfbb303e615560f0689619b
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 4 Nov 2024 09:24:36 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 4 Nov 2024 09:24:36 -0500
Reported-by: Erik Rijkers (private email)
Backpatch-through: master
M doc/src/sgml/mvcc.sgml
Move TRACE calls into WaitOnLock()
commit : 6ae0897e42416d8954f0340ba7eca6275ed8c8b7
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 16:21:01 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 16:21:01 +0200
LockAcquire is a long and complex function. Pushing more stuff to its
subroutines makes it a little more manageable.
Reviewed-by: Maxim Orlov
Discussion: https://www.postgresql.org/message-id/7c2090cd-a72a-4e34-afaa-6dd2ef31440e@iki.fi
M src/backend/storage/lmgr/lock.c
Set MyProc->heldLocks in ProcSleep
commit : 0464f25b6aa156228982efd46303e8c8dd25cccc
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 16:20:57 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 16:20:57 +0200
Previously, ProcSleep()'s caller was responsible for setting
MyProc->heldLocks, and we had comments to remind about that. But it
seems simpler to make ProcSleep() itself responsible for it.
ProcSleep() already set the other info about the lock its waiting for
(waitLock, waitProcLock and waitLockMode), so it is natural for it to
set heldLocks too.
Reviewed-by: Maxim Orlov
Discussion: https://www.postgresql.org/message-id/7c2090cd-a72a-4e34-afaa-6dd2ef31440e@iki.fi
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
Clarify nbtree parallel scan _bt_endpoint contract.
commit : 62620b6aadcedf453989e6b426f6ef3284c64bed
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 4 Nov 2024 09:05:59 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 4 Nov 2024 09:05:59 -0500
_bt_endpoint is a helper function for _bt_first that's called whenever
no useful insertion scan key can be used, and we need to lock and read
either the leftmost or rightmost leaf page in the index. Simplify and
document its preconditions, relieving its _bt_first caller from having
to end the parallel scan when it returns false.
Also stop unnecessarily invalidating the current scan position in nearby
code in both _bt_first and _bt_endpoint. This seems to have been
copy-pasted from _bt_readnextpage, where invalidating the scan's current
position really is necessary.
Follow-up to the refactoring work in commit 1bd4bc85.
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
Fix comment in LockReleaseAll() on when locallock->nLock can be zero
commit : 1fe0466cf2cbb909b6553061620b6a6779319406
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 15:31:46 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 15:31:46 +0200
We reach this case also e.g. when a deadlock is detected, not only
when we run out of memory.
Reviewed-by: Maxim Orlov
Discussion: https://www.postgresql.org/message-id/7c2090cd-a72a-4e34-afaa-6dd2ef31440e@iki.fi
M src/backend/storage/lmgr/lock.c
Add PG_TEST_EXTRA configure option to the Make builds
commit : 99b937a44f087c133c4db42e1bf4a8990ea417b9
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 14:09:38 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 14:09:38 +0200
The Meson builds have PG_TEST_EXTRA as a configure-time variable,
which was not available in the Make builds. To ensure both build
systems are in sync, PG_TEST_EXTRA is now added as a configure-time
variable. It can be set like this:
./configure PG_TEST_EXTRA="kerberos, ssl, ..."
Note that to preserve the old behavior, this configure-time variable
is overridden by the PG_TEST_EXTRA environment variable when you run
the tests.
Author: Jacob Champion
Reviewed by: Ashutosh Bapat, Nazir Bilal Yavuz
M configure
M configure.ac
M src/Makefile.global.in
M src/test/Makefile
Make PG_TEST_EXTRA env var override the "meson setup" option
commit : 3d1aec225ac4a1199079150dabfd0d34c5054127
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 14:09:25 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 4 Nov 2024 14:09:25 +0200
"meson test" used to ignore the PG_TEST_EXTRA environment variable,
which meant that in order to run additional tests, you had to run
"meson setup -DPG_TEST_EXTRA=...". That's somewhat expensive, and not
consistent with autoconf builds. Allow PG_TEST_EXTRA environment
variable to override the setup-time option at run time, so that you
can do "PG_TEST_EXTRA=... meson test".
To implement this, the configuration time value is passed as an extra
"--pg-test-extra" argument to testwrap instead of adding it to the
test environment. If the environment variable is set at the time of
running test, testwrap uses the value from the environment variable
and ignores the --pg-test-extra option.
Now that "meson test" obeys the environment variable, we can remove it
from the "meson setup" steps in the CI script. It will now be picked
up from the environment variable like with "make check".
Author: Nazir Bilal Yavuzk, Ashutosh Bapat
Reviewed-by: Ashutosh Bapat with inputs from Tom Lane and Andrew Dunstan
M .cirrus.tasks.yml
M doc/src/sgml/installation.sgml
M meson.build
M meson_options.txt
M src/tools/testwrap
Doc: Update the behavior of generated columns in Logical Replication.
commit : 5b0c46ea0932e3be64081a277b5cc01fa9571689
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 4 Nov 2024 09:39:30 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 4 Nov 2024 09:39:30 +0530
Commit 745217a051 misses updating the new behavior of generated columns in
logical replication at a few places.
Reported-by: Peter Smith, Ajin Cherian
Author: Vignesh C
Discussion: https://postgr.es/m/CALDaNm2JOO7szz9+uaQbjmgZOfzbM_9tAQdFF8H5BjkQeaJs0A@mail.gmail.com
Discussion: https://postgr.es/m/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E@gmail.com
M doc/src/sgml/ddl.sgml
M doc/src/sgml/logical-replication.sgml
Add missing newlines at the end of two SQL files
commit : 027124a872d7b5dfddc69590af42f626b1727dba
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 3 Nov 2024 19:42:51 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 3 Nov 2024 19:42:51 +0900
arrays.sql was already missing it before 49d6c7d8daba, and I have just
noticed it thanks to this commit. The second one in test_slru has been
introduced by 768a9fd5535f.
M src/test/modules/test_slru/test_slru–1.0.sql
M src/test/regress/sql/arrays.sql
Suppress new "may be used uninitialized" warning.
commit : 825c72c071969bede8e15846a89a3bfa5b1be66b
author : Noah Misch <noah@leadboat.com>
date : Sat, 2 Nov 2024 19:42:52 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 2 Nov 2024 19:42:52 -0700
Buildfarm member mamba fails to deduce that the function never uses this
variable without initializing it. Back-patch to v12, like commit
b412f402d1e020c5dac94f3bf4a005db69519b99.
M src/backend/catalog/index.c
Fix inplace update buffer self-deadlock.
commit : 0bada39c83a150079567a6e97b1a25a198f30ea3
author : Noah Misch <noah@leadboat.com>
date : Sat, 2 Nov 2024 09:04:56 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 2 Nov 2024 09:04:56 -0700
A CacheInvalidateHeapTuple* callee might call
CatalogCacheInitializeCache(), which needs a relcache entry. Acquiring
a valid relcache entry might scan pg_class. Hence, to prevent
undetected LWLock self-deadlock, CacheInvalidateHeapTuple* callers must
not hold BUFFER_LOCK_EXCLUSIVE on buffers of pg_class. Move the
CacheInvalidateHeapTupleInplace() before the BUFFER_LOCK_EXCLUSIVE. No
back-patch, since I've reverted commit
243e9b40f1b2dd09d6e5bf91ebf6e822a2cd3704 from non-master branches.
Reported by Alexander Lakhin. Reviewed by Alexander Lakhin.
Discussion: https://postgr.es/m/10ec0bc3-5933-1189-6bb8-5dec4114558e@gmail.com
M src/backend/access/heap/heapam.c
M src/backend/utils/cache/inval.c
M src/include/utils/inval.h
Move I/O before the index_update_stats() buffer lock region.
commit : b412f402d1e020c5dac94f3bf4a005db69519b99
author : Noah Misch <noah@leadboat.com>
date : Sat, 2 Nov 2024 09:04:55 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 2 Nov 2024 09:04:55 -0700
Commit a07e03fd8fa7daf4d1356f7cb501ffe784ea6257 enlarged the work done
here under the pg_class heap buffer lock. Two preexisting actions are
best done before holding that lock. Both RelationGetNumberOfBlocks()
and visibilitymap_count() do I/O, and the latter might exclusive-lock a
visibility map buffer. Moving these reduces contention and risk of
undetected LWLock deadlock. Back-patch to v12, like that commit.
Discussion: https://postgr.es/m/20241031200139.b4@rfd.leadboat.com
M src/backend/catalog/index.c
doc: clarify text around MVCC example query
commit : 2fa255ce9b9536450c9f86bd7c15778f58cf3777
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 16:38:16 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 16:38:16 -0400
Reported-by: marlene.brandstaetter@cargonet.software
Discussion: https://postgr.es/m/167765529052.987840.12345375075704447735@wrigleys.postgresql.org
Backpatch-through: master
M doc/src/sgml/mvcc.sgml
doc: remove useless MERGE example
commit : 4a9effe45ee96f7105ba19285eff18a634bfee01
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 16:20:27 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 16:20:27 -0400
Reported-by: dwayne.towell@gmail.com
Discussion: https://postgr.es/m/167699245721.1902146.6479762301617101634@wrigleys.postgresql.org
Backpatch-through: master
M doc/src/sgml/ref/merge.sgml
doc: improve tablespace example query and link to helper funcs.
commit : e1a76db1a880c2e80ae96c40a31b0ee589e06562
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 15:54:16 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 15:54:16 -0400
Reported-by: Agustín
Discussion: https://postgr.es/m/172609721070.1128084.6724666076293146476@wrigleys.postgresql.org
Backpatch-through: master
M doc/src/sgml/manage-ag.sgml
doc: fix ALTER DOMAIN domain_constraint to spell out options
commit : 4200fea80e79851994b8eb5a64a3d4420456c977
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 13:54:28 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 13:54:28 -0400
It used to refer to CREATE DOMAIN, but CREATE DOMAIN allows NULL, while
ALTER DOMAIN does not.
Reported-by: elionescu@yahoo.com
Discussion: https://postgr.es/m/172225092461.915373.6103973717483380183@wrigleys.postgresql.org
Backpatch-through: 12
M doc/src/sgml/ref/alter_domain.sgml
doc: explain how the home directory is found on Unix-like syst.
commit : 94a8c19eed6cdaffad8939a41ed89e3c6afa7c94
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 13:32:21 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 13:32:21 -0400
Done for libpq, postgres-fdw, and psql.
Reported-by: marc@msys.ch
Discussion: https://postgr.es/m/CAKFQuwZ-T-zsVM7gApS9-XU9vGxC7Oa-UyRQPVcJFagNU=AjOw@mail.gmail.com
Backpatch-through: master
M doc/src/sgml/libpq.sgml
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/ref/psql-ref.sgml
doc: Add link to listen_addresses as cause of connection failure
commit : 1eb5564230a3b82f7f8e8b91b93114dbde66c811
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 13:14:46 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 13:14:46 -0400
Reported-by: k.man.113@gmail.com
Discussion: https://postgr.es/m/171494070007.703.17021965362263796980@wrigleys.postgresql.org
Backpatch-through: master
M doc/src/sgml/runtime.sgml
Update contrib/sepgsql regression tests for commit 89e51abcb.
commit : cf590c568d0535785d594f79dcdaafdfeb46ac2f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 1 Nov 2024 12:49:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 1 Nov 2024 12:49:43 -0400
Oversight revealed by buildfarm.
M contrib/sepgsql/expected/ddl.out
doc: improve build for non-Latin1 characters
commit : 641a5b7a1447954076728f259342c2f9201bb0b5
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 12:46:51 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 12:46:51 -0400
Add README.non-ASCII to explain non-ASCII doc behavior; some text moved
from release.sgml.
Change UTF8 SGML characters to use HTML entities.
Remove unnecessary UTF8 spaces.
Add SVG file check for check-nbsp target.
Add dummy 'pdf' Makefile target.
Reported-by: Yugo Nagata
Discussion: https://postgr.es/m/20241011114122.c90f8a871462da36f2e2afeb@sraoss.co.jp
Backpatch-through: master
M doc/src/sgml/Makefile
A doc/src/sgml/README.non-ASCII
M doc/src/sgml/charset.sgml
M doc/src/sgml/images/genetic-algorithm.svg
M doc/src/sgml/release.sgml
M doc/src/sgml/stylesheet-man.xsl
Clarify nbtree array preprocessing comment.
commit : fc7ddededb9ee466d20581b966e01a3566b9559f
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 1 Nov 2024 11:43:24 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 1 Nov 2024 11:43:24 -0400
Oversight in commit 5bf748b8.
M src/backend/access/nbtree/nbtutils.c
doc: remove mention of ActiveState for Perl and Tcl on Windows
commit : 6d5444c9ed4864be2429f6343d22c85684c4853d
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 11:30:54 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 1 Nov 2024 11:30:54 -0400
Replace with Strawberry Perl and Magicsplat Tcl.
Reported-by: Yasir Hussain
Discussion: https://postgr.es/m/CAA9OW9fAAM_WDYYpAquqF6j1hmfRMzHPsFkRfP5E6oSfkF=dMA@mail.gmail.com
Backpatch-through: 12
M doc/src/sgml/installation.sgml
Rename two functions that wake up other processes
commit : 368d8270c838a6c328fc24418603cb172b5e0390
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 1 Nov 2024 13:47:24 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 1 Nov 2024 13:47:24 +0200
Instead of talking about setting latches, which is a pretty low-level
mechanism, emphasize that they wake up other processes.
This is in preparation for replacing Latches with a new abstraction.
That's still work in progress, but this seems a little tidier anyway,
so let's get this refactoring out of the way already.
Discussion: https://www.postgresql.org/message-id/391abe21-413e-4d91-a650-b663af49500c%40iki.fi
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogrecovery.c
M src/backend/access/transam/xlogwait.c
M src/backend/postmaster/walsummarizer.c
M src/include/access/xlogwait.h
M src/include/postmaster/walsummarizer.h
Use ProcNumbers instead of direct Latch pointers to address other procs
commit : a9c546a5a3783810a1b665f246fc6d0a596d0606
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 1 Nov 2024 13:47:20 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 1 Nov 2024 13:47:20 +0200
This is in preparation for replacing Latches with a new abstraction.
That's still work in progress, but this seems a little tidier anyway,
so let's get this refactoring out of the way already.
Discussion: https://www.postgresql.org/message-id/391abe21-413e-4d91-a650-b663af49500c%40iki.fi
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogwait.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/walwriter.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walreceiverfuncs.c
M src/backend/storage/lmgr/proc.c
M src/include/access/xlogwait.h
M src/include/replication/walreceiver.h
M src/include/storage/proc.h
Remove use of pg_memory_is_all_zeros() in bufpage.c
commit : e819bbb7c82ac048ffd865ba3f2d2c4933923c77
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 1 Nov 2024 17:05:36 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 1 Nov 2024 17:05:36 +0900
After a closer lookup, this makes the all-zero check of the page more
expensive, so let's remove the new function call in bufpage.c. The
maths of the check were also incorrect, checking that the page was full
of zeros only for the first 1kB.
This brings back the code to the state it was at 49d6c7d8daba.
Per discussion with David Rowley and Bertrand Drouvot.
Discussion: https://postgr.es/m/CAApHDvrXzPAr3FxoBuB7b3D-okNoNA2jxLun1rW8Yw5wkbqusw@mail.gmail.com
M src/backend/storage/page/bufpage.c
Add pg_memory_is_all_zeros() in memutils.h
commit : 07e9e28b56db4d8533b62b3a921f1e74aae26b1f
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 1 Nov 2024 11:35:46 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 1 Nov 2024 11:35:46 +0900
This new function tests if a memory region starting at a given location
for a defined length is made only of zeroes. This unifies in a single
path the all-zero checks that were happening in a couple of places of
the backend code:
- For pgstats entries of relation, checkpointer and bgwriter, where
some "all_zeroes" variables were previously used with memcpy().
- For all-zero buffer pages in PageIsVerifiedExtended().
This new function uses the same forward scan as the check for all-zero
buffer pages, applying it to the three pgstats paths mentioned above.
Author: Bertrand Drouvot
Reviewed-by: Peter Eisentraut, Heikki Linnakangas, Peter Smith
Discussion: https://postgr.es/m/ZupUDDyf1hHI4ibn@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/storage/page/bufpage.c
M src/backend/utils/activity/pgstat_bgwriter.c
M src/backend/utils/activity/pgstat_checkpointer.c
M src/backend/utils/activity/pgstat_relation.c
M src/include/utils/memutils.h
Add SQL function array_reverse()
commit : 49d6c7d8dabad8c59ba21afe72ff32e79f822291
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 1 Nov 2024 10:32:19 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 1 Nov 2024 10:32:19 +0900
This function takes in input an array, and reverses the position of all
its elements. This operation only affects the first dimension of the
array, like array_shuffle().
The implementation structure is inspired by array_shuffle(), with a
subroutine called array_reverse_n() that may come in handy in the
future, should more functions able to reverse portions of arrays be
introduced.
Bump catalog version.
Author: Aleksander Alekseev
Reviewed-by: Ashutosh Bapat, Tom Lane, Vladlen Popolitov
Discussion: https://postgr.es/m/CAJ7c6TMpeO_ke+QGOaAx9xdJuxa7r=49-anMh3G5476e3CX1CA@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/array_userfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/arrays.out
M src/test/regress/sql/arrays.sql
Make all ereport() calls within gram.y provide error locations.
commit : 2d8bff603c9ee7b284b85016fd1a8849ee36368d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 31 Oct 2024 16:09:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 31 Oct 2024 16:09:22 -0400
This patch responds to a comment that I (tgl) made in the
discussion leading up to 774171c4f, that really all errors
occurring during raw parsing should provide error cursors.
Syntax errors reported by Bison will have one, and most of
the handwritten ereport's in gram.y already provide one,
but there were a few stragglers.
(It is not claimed that this handles every failure reachable
during raw parsing --- out-of-memory is an obvious exception.
But this makes a good start on cases that are likely to occur.)
While we're at it, clean up the reported positions for errors
associated with LIMIT/OFFSET clauses. Previously we were
relying on applying exprLocation() to the contained expressions,
but that leads to slightly odd cursor placement, e.g.
regression=# (select * from foo limit 10) limit 10;
ERROR: multiple LIMIT clauses not allowed
LINE 1: (select * from foo limit 10) limit 10;
^
We can afford to keep a little more state in the transient
SelectLimit structs in order to make that better.
Jian He and Tom Lane (extracted from a larger patch by Jian,
with some additional work by me)
Discussion: https://postgr.es/m/CACJufxEmONE3P2En=jopZy1m=cCCUs65M4+1o52MW5og9oaUPA@mail.gmail.com
M src/backend/parser/gram.y
M src/test/regress/expected/create_table.out
M src/test/regress/expected/limit.out
M src/test/regress/expected/sqljson.out
Add a parse location field to struct FunctionParameter.
commit : 89e51abcb2905d6beaa09a7d3603d6882ac27033
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 31 Oct 2024 15:53:58 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 31 Oct 2024 15:53:58 -0400
This allows an error cursor to be supplied for a bunch of
bad-function-definition errors that previously lacked one,
or that cheated a bit by pointing at the contained type name
when the error isn't really about that.
Bump catversion from an abundance of caution --- I don't think
this node type can actually appear in stored views/rules, but
better safe than sorry.
Jian He and Tom Lane (extracted from a larger patch by Jian,
with some additional work by me)
Discussion: https://postgr.es/m/CACJufxEmONE3P2En=jopZy1m=cCCUs65M4+1o52MW5og9oaUPA@mail.gmail.com
M src/backend/commands/functioncmds.c
M src/backend/nodes/nodeFuncs.c
M src/backend/parser/gram.y
M src/include/catalog/catversion.h
M src/include/nodes/parsenodes.h
M src/test/modules/test_ddl_deparse/expected/create_type.out
M src/test/modules/test_ddl_deparse/expected/opfamily.out
M src/test/modules/test_pg_dump/expected/test_pg_dump.out
M src/test/regress/expected/create_cast.out
M src/test/regress/expected/create_procedure.out
M src/test/regress/expected/create_type.out
M src/test/regress/expected/equivclass.out
M src/test/regress/expected/expressions.out
M src/test/regress/expected/float4-misrounded-input.out
M src/test/regress/expected/float4.out
M src/test/regress/expected/float8.out
M src/test/regress/expected/polymorphism.out
Fix refreshing physical relfilenumber on shared index
commit : b82c877e76e2398409e823773413079668cf1881
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 31 Oct 2024 18:24:48 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 31 Oct 2024 18:24:48 +0200
Buildfarm member 'prion', which is configured with
-DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE, failed with errors
like this:
ERROR: could not read blocks 0..0 in file "global/2672": read only 0 of 8192 bytes
while running a parallel test group that includes VACUUM FULL on some
catalog tables among other things. I was not able to reproduce that
just by running the tests with -DRELCACHE_FORCE_RELEASE
-DCATCACHE_FORCE_RELEASE, even though 'prion' hit it on first run
after commit 2b9b8ebbf8, so there might be something else that makes
it more susceptible to the race. However, I was able to reproduce it
by adding another test to the same test group that runs "vacuum full
pg_database" repeatedly.
The problem is that RelationReloadIndexInfo() no longer calls
RelationInitPhysicalAddr() on a nailed, shared index, when an
invalidation happens early during backend startup, before the critical
relcaches have been built. Before commit 2b9b8ebbf8, that was done by
RelationReloadNailed(), but it went missing from that path. Add it
back as an explicit step.
Broken by commit 2b9b8ebbf8, which refactored these functions.
Discussion: https://www.postgresql.org/message-id/db876575-8f5b-4193-a538-df7e1f92d47a%40iki.fi
M src/backend/utils/cache/relcache.c
Remove duplicate words in comments
commit : fb7e27abfbd4e1bae44ce29f5f2c6f6d1aabb5e2
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 31 Oct 2024 11:38:03 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 31 Oct 2024 11:38:03 +0100
A few comments contained duplicate "the" in sentences, fix by removing
one occurrence.
Author: Vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/CALDaNm2aEEiPwGJmPdzBxROVvs8n75yCjKz4K1f1B2TdWpzxTA@mail.gmail.com
M src/backend/executor/execExprInterp.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/postmaster/postmaster.c
Split RelationClearRelation into three different functions
commit : 2b9b8ebbf8a2f621d1cecc4db759700f9ce6ce66
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 31 Oct 2024 10:09:40 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 31 Oct 2024 10:09:40 +0200
The old RelationClearRelation function did different things depending
on the arguments and circumstances. It could:
a) remove the relation completely from relcache (rebuild == false),
b) mark the entry as invalid (rebuild == true, but not in xact), or
c) rebuild the entry (rebuild == true).
Different callers used it for different purposes, and often assumed a
particular behavior, which was confusing. Split it into three
different functions, one for each of the above actions (one of them,
RelationInvalidateRelation, was already added in commit e6cd857726).
Move the responsibility of choosing the action and calling the right
function to the callers.
Reviewed-by: jian he <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/9c9e8908-7b3e-4ce7-85a8-00c0e165a3d6%40iki.fi
M src/backend/utils/cache/relcache.c
Simplify call to rebuild relcache entry for indexes
commit : 8e2e2662214a0a6d0fb23a070ea78d691d8ced43
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 31 Oct 2024 10:02:58 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 31 Oct 2024 10:02:58 +0200
RelationClearRelation(rebuild == true) calls RelationReloadIndexInfo()
for indexes. We can rely on that in RelationIdGetRelation(), instead
of calling RelationReloadIndexInfo() directly. That simplifies the
code a little.
In the passing, add a comment in RelationBuildLocalRelation()
explaining why it doesn't call RelationInitIndexAccessInfo(). It's
because at index creation, it's called before the pg_index row has
been created. That's also the reason that RelationClearRelation()
still needs a special case to go through the full-blown rebuild if the
index support information in the relcache entry hasn't been populated
yet.
Reviewed-by: jian he <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/9c9e8908-7b3e-4ce7-85a8-00c0e165a3d6%40iki.fi
M src/backend/utils/cache/relcache.c
Remove unused field from SubPlanState struct
commit : 3974bc319639ad7da5934ba8d04d474a6f7fdee6
author : David Rowley <drowley@postgresql.org>
date : Thu, 31 Oct 2024 13:44:15 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 31 Oct 2024 13:44:15 +1300
bf6c614a2 did some conversion work to use ExprState instead of manually
calling equality functions to check if one set of values is not distinct
from another set. That patch removed many of the fields that became
redundant as a result of that change, but it forgot to remove
SubPlanState.tab_eq_funcs. Fix that.
In passing, fix the header comment for TupleHashEntryData to correctly
spell the field name it's talking about.
Author: Rafia Sabih <rafia.pghackers@gmail.com>
Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>
Discussion: https://postgr.es/m/CA+FpmFeycdombFzrjZw7Rmc29CVm4OOzCWwu=dVBQ6q=PX8SvQ@mail.gmail.com
Discussion: https://postgr.es/m/CAApHDvrWR2jYVhec=COyF2g2BE_ns91NDsCHAMFiXbyhEujKdQ@mail.gmail.com
M src/backend/executor/nodeSubplan.c
M src/include/nodes/execnodes.h
injection_points: Improve comment about disabled isolation permutation
commit : baa1ae04299f9e80ee8610a6bd84b9111f4ed615
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 31 Oct 2024 08:28:20 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 31 Oct 2024 08:28:20 +0900
9f00edc22888 has disabled a permutation due to failures in the CI for
FreeBSD environments, but this is a matter of timing. Let's document
properly why this type of permutation is a bad idea if relying on a wait
done in a SQL function, so as this can be avoided when implementing new
tests (this spec is also a template).
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/ZyCa2qsopKaw3W3K@paquier.xyz
M src/test/modules/injection_points/specs/basic.spec
nbtree: assert no scheduled primscan between pages.
commit : 492e6b54c604c39ecf5ebc812849185b20c4b1b6
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Oct 2024 15:53:26 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Oct 2024 15:53:26 -0400
Follow-up to bugfix commit 763d65ae. Technically this new assertion is
redundant with the assertion recently added to _bt_readpage by that same
commit, but it seems like a good idea to have both.
The new assertion makes it clear that we expect to call _bt_readnextpage
when there's another primitive index scan scheduled, though only when
needed as the final step of ending the current primitive scan.
M src/backend/access/nbtree/nbtsearch.c
Clarify nbtree array exhaustion comments.
commit : 81a25790f1f66319b4f088148b97dbb8badd4907
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Oct 2024 13:43:49 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Oct 2024 13:43:49 -0400
Strictly speaking, we only need to make sure to leave the scan's array
keys in their final positions (final for the current scan direction) to
handle SAOP array exhaustion because btgettuple might only return a
subset of the items for the final page (final for the current scan
direction), before the scan changes direction. While it's typical for
so->currPos to be invalidated shortly after the scan's arrays are first
exhausted, and while so->currPos invalidation does obviate the need to
leave the scan's arrays in any particular state, we can't rely on any of
that actually happening when handling array exhaustion. Adjust comments
to make all of that a lot clearer.
Oversight in commit 5bf748b8, which enhanced nbtree ScalarArrayOp
execution.
M src/backend/access/nbtree/nbtutils.c
Optimize sifting down in binaryheap.
commit : 849110dd3eec3e21c358e24f11c6d501d05eee72
author : Nathan Bossart <nathan@postgresql.org>
date : Wed, 30 Oct 2024 11:28:34 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Wed, 30 Oct 2024 11:28:34 -0500
Presently, each iteration of the loop in sift_down() will perform
3 comparisons if both children are larger than the parent node (2
for comparing each child to the parent node, and a third to compare
the children to each other). By first comparing the children to
each other and then comparing the larger child to the parent node,
we can accomplish the same thing with just 2 comparisons (while
also not affecting the number of comparisons in any other case).
Author: ChangAo Chen
Reviewed-by: Robert Haas
Discussion: https://postgr.es/m/tencent_0142D8DA90940B9930BCC08348BBD6D0BB07%40qq.com
M src/common/binaryheap.c
Stabilize jsonb_path_query test case.
commit : af21152268317323480caa790c4a6347110f8085
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Oct 2024 11:42:28 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Oct 2024 11:42:28 -0400
An operation like '12:34:56'::time_tz takes the UTC offset from
the prevailing time zone, which means that the results change
across DST transitions. One of the test cases added in ed055d249
failed to consider this.
Per report from Bernhard Wiedemann. Back-patch to v17, as the
test case was.
Discussion: https://postgr.es/m/ba8e1bc0-8a99-45b7-8397-3f2e94415e03@suse.de
M src/test/regress/expected/jsonb_jsonpath.out
M src/test/regress/sql/jsonb_jsonpath.sql
Fix bug in nbtree array primitive scan scheduling.
commit : 763d65ae254569e0bdbe2cbb27b184112765f24f
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Oct 2024 10:57:19 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Oct 2024 10:57:19 -0400
A bug in nbtree's handling of primitive index scan scheduling could lead
to wrong answers when a scrollable cursor was used with an index scan
that had a SAOP index qual. Wrong answers were only possible when the
scan direction changed after a primitive scan was scheduled, but before
_bt_next was asked to fetch the next tuple in line (i.e. for things to
break, _bt_next had to be denied the opportunity to step off the page in
the same direction as the one used when the primscan was scheduled).
Furthermore, the issue only occurred when the page in question happened
to be the first page to be visited by the entire top-level scan; the
issue hinged upon the cursor backing up to the absolute beginning of the
key space that it returns tuples from (fetching in the opposite scan
direction across a "primitive scan boundary" always worked correctly).
To fix, make _bt_next unset the "needs primitive index scan" flag when
it detects that the current scan direction is not the one that was used
by _bt_readpage back when the primitive scan in question was scheduled.
This fixes the cases that are known to be faulty, and also seems like a
good idea on general robustness grounds.
Affected scrollable cursor cases now avoid a spurious primitive index
scan when they fetch backwards to the absolute start of the key space to
be visited by their cursor. Fetching backwards now only returns those
tuples at the start of the scan, as expected. It'll also be okay to
once again fetch forwards from the start at that point, since the scan
will be left in a state that's exactly consistent with the state it was
in before any tuples were ever fetched, as expected.
Oversight in commit 5bf748b8, which enhanced nbtree ScalarArrayOp
execution.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-Wznv49bFsE2jkt4GuZ0tU2C91dEST=50egzjY2FeOcHL4Q@mail.gmail.com
Backpatch: 17-, where commit 5bf748b8 first appears.
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtutils.c
M src/include/access/nbtree.h
Fix some more bugs in foreign keys connecting partitioned tables
commit : 2d5fe514052ad82e46612e4c572f1c999be56a71
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 30 Oct 2024 10:54:03 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 30 Oct 2024 10:54:03 +0100
* In DetachPartitionFinalize() we were applying a tuple conversion map
to tuples that didn't need one, which can lead to erratic behavior if
a partitioned table has a partition with a different column order, as
reported by Alexander Lakhin. This was introduced by 53af9491a043.
Don't do that. Also, modify a recently added test case to exercise
this.
* The same function as well as CloneFkReferenced() were acquiring
AccessShareLock on a partition, only to have CreateTrigger() later
acquire ShareRowExclusiveLock on it. This can lead to deadlock by
lock escalation, unnecessarily. Avoid that by acquiring the stronger
lock to begin with. This probably dates back to branch 12, but I have
never seen a report of this being a problem in the field.
* Innocuous but wasteful: also introduced by 53af9491a043, we were
reading a pg_constraint tuple from syscache that we don't need, as
reported by Tender Wang. Don't.
Backpatch to 15.
Discussion: https://postgr.es/m/461e9c26-2076-8224-e119-84998b6a784e@gmail.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
meson: Add missing dependency to unicode test programs
commit : 2845cd1ca0cd2c11230f75d75405882e8bc35525
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 30 Oct 2024 08:30:00 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 30 Oct 2024 08:30:00 +0100
The test programs in src/common/unicode/ (case_test, category_test,
norm_test), don't build with meson if the nls option is enabled,
because a libintl dependency is missing. Fix that. (The makefiles
are ok.)
Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/flat/52db1d2b-4b96-473e-b323-a4b16a950fba%40eisentraut.org
M src/common/unicode/meson.build
Replicate generated columns when specified in the column list.
commit : 745217a051a9341e8c577ea59a87665d331d4af0
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 30 Oct 2024 12:36:26 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 30 Oct 2024 12:36:26 +0530
This commit allows logical replication to publish and replicate generated
columns when explicitly listed in the column list. We also ensured that
the generated columns were copied during the initial tablesync when they
were published.
We will allow to replicate generated columns even when they are not
specified in the column list (via a new publication option) in a separate
commit.
The motivation of this work is to allow replication for cases where the
client doesn't have generated columns. For example, the case where one is
trying to replicate data from Postgres to the non-Postgres database.
Author: Shubham Khanna, Vignesh C, Hou Zhijie
Reviewed-by: Peter Smith, Hayato Kuroda, Shlok Kyal, Amit Kapila
Discussion: https://postgr.es/m/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E@gmail.com
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/create_publication.sgml
M src/backend/catalog/pg_publication.c
M src/backend/replication/logical/proto.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/pgoutput/pgoutput.c
M src/include/replication/logicalproto.h
M src/test/regress/expected/publication.out
M src/test/regress/sql/publication.sql
M src/test/subscription/t/031_column_list.pl
Add missing CommandCounterIncrement() in stats import functions.
commit : f22e436bff779fee4e1ce49733ba5791d4634fb1
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 29 Oct 2024 10:14:23 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 29 Oct 2024 10:14:23 -0700
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/98b2fcf0-f701-369e-d63d-6be9739ce17c@gmail.com
M src/backend/statistics/attribute_stats.c
M src/backend/statistics/relation_stats.c
Unpin buffer before inplace update waits for an XID to end.
commit : 30d47ec8c6c78f46bcd7116b7effb6988b126e43
author : Noah Misch <noah@leadboat.com>
date : Tue, 29 Oct 2024 09:39:55 -0700
committer: Noah Misch <noah@leadboat.com>
date : Tue, 29 Oct 2024 09:39:55 -0700
Commit a07e03fd8fa7daf4d1356f7cb501ffe784ea6257 changed inplace updates
to wait for heap_update() commands like GRANT TABLE and GRANT DATABASE.
By keeping the pin during that wait, a sequence of autovacuum workers
and an uncommitted GRANT starved one foreground LockBufferForCleanup()
for six minutes, on buildfarm member sarus. Prevent, at the cost of a
bit of complexity. Back-patch to v12, like the earlier commit. That
commit and heap_inplace_lock() have not yet appeared in any release.
Discussion: https://postgr.es/m/20241026184936.ae.nmisch@google.com
M src/backend/access/heap/heapam.c
M src/backend/access/index/genam.c
M src/include/access/heapam.h
Update time zone data files to tzdata release 2024b.
commit : 502e7bf7f09c458ce66ce043b94864b535c5d5d0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Oct 2024 11:49:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Oct 2024 11:49:38 -0400
Historical corrections for Mexico, Mongolia, and Portugal.
Notably, Asia/Choibalsan is now an alias for Asia/Ulaanbaatar
rather than being a separate zone, mainly because the differences
between those zones were found to be based on untrustworthy data.
M src/timezone/data/tzdata.zi
M src/timezone/known_abbrevs.txt
M src/timezone/tznames/Default
M src/timezone/tznames/Europe.txt
Reduce variable scope and possibly useless palloc
commit : fcbd1bb6613fde5784b54c88e4d0398ab8e13694
author : David Rowley <drowley@postgresql.org>
date : Wed, 30 Oct 2024 01:38:42 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 30 Oct 2024 01:38:42 +1300
Move the CreateStmt down to the branch that it's used in, thus
preventing the makeNode() call in cases where the CreateStmt isn't used.
Author: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/CAEudQAq=06YPWPhS+yyTbCwv5JLKRz8rm3dWx6JR5Uj_d_fQDA@mail.gmail.com
M src/backend/commands/view.c
Doc: add detail about EXPLAIN's "Disabled" property
commit : 84b8fccbe5c21cc2a05f8cf91903cadc6ebfe680
author : David Rowley <drowley@postgresql.org>
date : Tue, 29 Oct 2024 23:28:12 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 29 Oct 2024 23:28:12 +1300
c01743aa4 and later 161320b4b adjusted the EXPLAIN output to show which
plan nodes were chosen despite being disabled by the various enable*
GUCs. Prior to e22253467, the disabledness of a node was only evident by
a large startup cost penalty. Since we now explicitly tag disabled nodes
with a boolean property in EXPLAIN, let's add some documentation to
provide some details about why and when disabled nodes can appear in the
plan.
Author: Laurenz Albe, David Rowley
Discussion: https://postgr.es/m/883729e429267214753d5e438c82c73a58c3db5d.camel@cybertec.at
M doc/src/sgml/perform.sgml
Add missing FATAL => 'all' to a use warnings in Perl
commit : 014720c6d9fb1285846f0b7cc1c80bdbe887a743
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Oct 2024 10:26:17 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 29 Oct 2024 10:26:17 +0100
Author: Anton Voloshin <a.voloshin@postgrespro.ru>
Discussion: https://www.postgresql.org/message-id/aa8a55d5-554a-4027-a491-1b0ca7c85f7a@postgrespro.ru
M src/test/modules/test_misc/t/006_signal_autovacuum.pl
doc: Add better description for rewrite functions in event triggers
commit : 4b7bba49e71266ec378e672a1aaa9f16823449c2
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 29 Oct 2024 15:35:01 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 29 Oct 2024 15:35:01 +0900
There are two functions that can be used in event triggers to get more
details about a rewrite happening on a relation. Both had a limited
documentation:
- pg_event_trigger_table_rewrite_reason() and
pg_event_trigger_table_rewrite_oid() were not mentioned in the main
event trigger section in the paragraph dedicated to the event
table_rewrite.
- pg_event_trigger_table_rewrite_reason() returns an integer which is a
bitmap of the reasons why a rewrite happens. There was no explanation
about the meaning of these values, forcing the reader to look at the
code to find out that these are defined in event_trigger.h.
While on it, let's add a comment in event_trigger.h where the
AT_REWRITE_* are defined, telling to update the documentation when
these values are changed.
Backpatch down to 13 as a consequence of 1ad23335f36b, where this area
of the documentation has been heavily reworked.
Author: Greg Sabino Mullane
Discussion: https://postgr.es/m/CAKAnmmL+Z6j-C8dAx1tVrnBmZJu+BSoc68WSg3sR+CVNjBCqbw@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/event-trigger.sgml
M doc/src/sgml/func.sgml
M src/include/commands/event_trigger.h
Doc: clarify enable_indexscan=off also disabled Index Only Scans
commit : dda781609f977315c9ecdefc1e29f9499393aaa4
author : David Rowley <drowley@postgresql.org>
date : Tue, 29 Oct 2024 16:24:10 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 29 Oct 2024 16:24:10 +1300
Disabling enable_indexscan has always also disabled Index Only Scans.
Here we make that more clear in the documentation in an attempt to
prevent future complaints complaining about this expected behavior.
Reported-by: Melanie Plageman
Author: David G. Johnston, David Rowley
Backpatch-through: 12, oldest supported version
Discussion: https://postgr.es/m/CAAKRu_atV=kovgpaLREyG68PB5+ncKvJ2UNoeRetEgyC3Yb5Sw@mail.gmail.com
M doc/src/sgml/config.sgml
Fix dependency of partitioned table and table AM with CREATE TABLE .. USING
commit : 49a23441ca7f314c25794a9a547011c0a858cfaa
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 29 Oct 2024 08:41:33 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 29 Oct 2024 08:41:33 +0900
A pg_depend entry between a partitioned table and its table access
method was missing when using CREATE TABLE .. USING with an unpinned
access method. DROP ACCESS METHOD could be used, while it should be
blocked if CASCADE is not specified, even if there was a partitioned
table that depends on the table access method. pg_class.relam would
then hold an orphaned OID value still pointing to the AM dropped.
The problem is fixed by adding a dependency between the partitioned
table and its table access method if set when the relation is created.
A test checking the contents of pg_depend in this case is added.
Issue introduced in 374c7a229042, that has added support for CREATE
TABLE .. USING for partitioned tables.
Reviewed-by: Alexander Lakhin
Discussion: https://postgr.es/m/18674-1ef01eceec278fab@postgresql.org
Backpatch-through: 17
M src/backend/catalog/heap.c
M src/test/regress/expected/create_am.out
M src/test/regress/sql/create_am.sql
Ensure we have a snapshot when updating pg_index in index_drop().
commit : 70b9adb98eab42bc12ddeea863be335b21b55675
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 28 Oct 2024 16:44:31 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 28 Oct 2024 16:44:31 -0500
I assumed that all index_drop() callers set an active snapshot
beforehand, but that is evidently not true. One counterexample is
autovacuum, which doesn't set an active snapshot when cleaning up
orphan temp indexes. To fix, unconditionally push an active
snapshot before updating pg_index in index_drop().
Oversight in commit b52adbad46.
Reported-by: Masahiko Sawada
Reviewed-by: Stepan Neretin, Masahiko Sawada
Discussion: https://postgr.es/m/CAD21AoBgF9etQrXbN9or_YHsmBRJHHNUEkhHp9rGK9CyQv5aTQ%40mail.gmail.com
M src/backend/catalog/index.c
Unify src/common/'s definitions of MaxAllocSize.
commit : 11b7de4a784135d7a9bce4a29ede092797749efc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Oct 2024 14:39:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Oct 2024 14:39:01 -0400
As threatened in the previous patch, define MaxAllocSize in
src/include/common/fe_memutils.h rather than having several
copies of it in different src/common/*.c files. This also
provides an opportunity to document it better.
While this would probably be safe to back-patch, I'll refrain
(for now anyway).
M src/common/psprintf.c
M src/common/saslprep.c
M src/common/stringinfo.c
M src/include/common/fe_memutils.h
Guard against enormously long input in pg_saslprep().
commit : bd2843167200f2607972c0defb916daa79ca265f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Oct 2024 14:33:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Oct 2024 14:33:55 -0400
Coverity complained that pg_saslprep() could suffer integer overflow,
leading to under-allocation of the output buffer, if the input string
exceeds SIZE_MAX/4. This hazard seems largely hypothetical, but it's
easy enough to defend against, so let's do so.
This patch creates a third place in src/common/ where we are locally
defining MaxAllocSize so that we can test against that in the same way
in backend and frontend compiles. That seems like about two places
too many, so the next patch will move that into common/fe_memutils.h.
I'm hesitant to do that in back branches however.
Back-patch to v14. The code looks similar in older branches, but
before commit 67a472d71 there was a separate test on the input string
length that prevented this hazard.
Per Coverity report.
M src/common/saslprep.c
Strip Windows newlines from extension script files manually.
commit : 6cfb3a33746990602c790199adc2debb2c4bbb87
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Oct 2024 13:07:29 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Oct 2024 13:07:29 -0400
Revert commit 924e03917 in favor of adding code to convert \r\n to \n
explicitly, on Windows only. The idea of letting text mode do the
work fails for a couple of reasons:
* Per Microsoft documentation, text mode also causes control-Z to be
interpreted as end-of-file. While it may be unlikely that extension
scripts contain control-Z, we've historically allowed it, and breaking
the case doesn't seem wise.
* Apparently, on some Windows configurations, "r" mode is interpreted
as binary not text mode. We could force it with "rt" but that would
be inconsistent with our code elsewhere, and it would still require
Windows-specific coding.
Thanks to Alexander Lakhin for investigation.
Discussion: https://postgr.es/m/79284195-4993-7b00-f6df-8db28ca60fa3@gmail.com
M src/backend/commands/extension.c
Fix WAL_DEBUG build
commit : 8a98822bcc37c03baf44ec39d8e88d9c24647b9d
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 28 Oct 2024 17:44:18 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 28 Oct 2024 17:44:18 +0100
broken by commit e18512c000e
Reported-by: Peter Geoghegan <pg@bowt.ie>
M src/backend/access/transam/xlog.c
nbtree: Minor sibling link traversal tweaks.
commit : 123474cbcebc17d3fd32503092df6e4beb8df0f6
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 28 Oct 2024 12:22:52 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 28 Oct 2024 12:22:52 -0400
Tweak some code comments for clarity, and relocate some local variable
declarations to the scope where they're actually used.
Follow-up to recent commit 1bd4bc85.
M src/backend/access/nbtree/nbtsearch.c
Fix overflow in bsearch_arg() with more than INT_MAX elements
commit : de5afddc3ba4aa41f341f62347ce40bb9ea9d78b
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 28 Oct 2024 14:07:38 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 28 Oct 2024 14:07:38 +0200
This was introduced in commit bfa2cee784, which replaced the old
bsearch_cmp() function we had in extended_stats.c with the current
implementation. The original discussion or commit message of
bfa2cee784 didn't mention where the new implementation came from, but
based on some googling, I'm guessing *BSD or libiberty, all of which
share this same code, with or without this fix.
Author: Ranier Vilela
Reviewed-by: Nathan Bossart
Backpatch-through: 14
Discussion: https://www.postgresql.org/message-id/CAEudQAp34o_8u6sGSVraLwuMv9F7T9hyHpePXHmRaxR2Aboi%2Bw%40mail.gmail.com
M src/port/bsearch_arg.c
Restore missing line to copyright notice
commit : 22bb889f7080d5a2596df2276ccbdb1ace1e4673
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 28 Oct 2024 13:08:43 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 28 Oct 2024 13:08:43 +0200
Commit 12c9423832 in May 2003 accidentally removed the last line of
the copyright notice in getopt.c. Put it back.
M src/port/getopt.c
Remove unused #include's from contrib, pl, test .c files
commit : 9be4e5d293b554d8a0800790c57fc707a3b5cf0f
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 28 Oct 2024 08:02:17 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 28 Oct 2024 08:02:17 +0100
as determined by IWYU
Similar to commit dbbca2cf299, but for contrib, pl, and src/test/.
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/flat/0df1d5b1-8ca8-4f84-93be-121081bde049%40eisentraut.org
M contrib/amcheck/verify_heapam.c
M contrib/amcheck/verify_nbtree.c
M contrib/auth_delay/auth_delay.c
M contrib/auto_explain/auto_explain.c
M contrib/basic_archive/basic_archive.c
M contrib/bloom/blcost.c
M contrib/bloom/blinsert.c
M contrib/bloom/blscan.c
M contrib/bloom/blutils.c
M contrib/bloom/blvacuum.c
M contrib/bloom/blvalidate.c
M contrib/btree_gin/btree_gin.c
M contrib/btree_gist/btree_bit.c
M contrib/btree_gist/btree_bool.c
M contrib/btree_gist/btree_bytea.c
M contrib/btree_gist/btree_date.c
M contrib/btree_gist/btree_enum.c
M contrib/btree_gist/btree_inet.c
M contrib/btree_gist/btree_interval.c
M contrib/btree_gist/btree_macaddr.c
M contrib/btree_gist/btree_macaddr8.c
M contrib/btree_gist/btree_text.c
M contrib/btree_gist/btree_time.c
M contrib/btree_gist/btree_ts.c
M contrib/btree_gist/btree_utils_num.h
M contrib/btree_gist/btree_utils_var.c
M contrib/btree_gist/btree_utils_var.h
M contrib/citext/citext.c
M contrib/dict_xsyn/dict_xsyn.c
M contrib/fuzzystrmatch/fuzzystrmatch.c
M contrib/intarray/_int_bool.c
M contrib/intarray/_int_selfuncs.c
M contrib/isn/isn.c
M contrib/lo/lo.c
M contrib/ltree/ltree_io.c
M contrib/ltree/ltree_op.c
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/fsmfuncs.c
M contrib/pageinspect/ginfuncs.c
M contrib/pageinspect/gistfuncs.c
M contrib/pageinspect/heapfuncs.c
M contrib/pg_freespacemap/pg_freespacemap.c
M contrib/pg_prewarm/autoprewarm.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_surgery/heap_surgery.c
M contrib/pg_visibility/pg_visibility.c
M contrib/pgcrypto/pgcrypto.c
M contrib/pgrowlocks/pgrowlocks.c
M contrib/pgstattuple/pgstatapprox.c
M contrib/pgstattuple/pgstatindex.c
M contrib/pgstattuple/pgstattuple.c
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/spi/moddatetime.c
M contrib/tablefunc/tablefunc.c
D contrib/tablefunc/tablefunc.h
M contrib/tsm_system_rows/tsm_system_rows.c
M contrib/tsm_system_time/tsm_system_time.c
M contrib/unaccent/unaccent.c
M contrib/xml2/xpath.c
M contrib/xml2/xslt_proc.c
M src/pl/plperl/plperl.c
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpython/plpy_cursorobject.c
M src/pl/plpython/plpy_exec.c
M src/pl/plpython/plpy_planobject.c
M src/pl/plpython/plpy_plpymodule.c
M src/pl/plpython/plpy_procedure.c
M src/pl/plpython/plpy_spi.c
M src/pl/plpython/plpy_util.c
M src/pl/tcl/pltcl.c
M src/test/modules/delay_execution/delay_execution.c
M src/test/modules/dummy_index_am/dummy_index_am.c
M src/test/modules/ldap_password_func/ldap_password_func.c
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/plsample/plsample.c
M src/test/modules/test_ddl_deparse/test_ddl_deparse.c
M src/test/modules/test_integerset/test_integerset.c
M src/test/modules/test_oat_hooks/test_oat_hooks.c
M src/test/modules/test_radixtree/test_radixtree.c
M src/test/modules/test_regex/test_regex.c
M src/test/modules/test_resowner/test_resowner_basic.c
M src/test/modules/test_resowner/test_resowner_many.c
M src/test/modules/test_rls_hooks/test_rls_hooks.c
M src/test/modules/test_shm_mq/setup.c
M src/test/modules/test_slru/test_multixact.c
M src/test/modules/test_tidstore/test_tidstore.c
M src/test/modules/worker_spi/worker_spi.c
M src/test/regress/pg_regress.c
M src/test/regress/regress.c
Change the default value of the streaming option to 'parallel'.
commit : 1bf1140be87230c71d0e7b29939f7e2b3d073aa1
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 28 Oct 2024 08:42:05 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 28 Oct 2024 08:42:05 +0530
Previously the default value of streaming option for a subscription was
'off'. The parallel option indicates that the changes in large
transactions (greater than logical_decoding_work_mem) are to be applied
directly via one of the parallel apply workers, if available.
The parallel mode was introduced in 16, but we refrain from enabling it by
default to avoid seeing any unpleasant behavior in the existing
applications. However we haven't found any such report yet, so this is a
good time to enable it by default.
Reported-by: Vignesh C
Author: Hayato Kuroda, Masahiko Sawada, Peter Smith, Amit Kapila
Discussion: https://postgr.es/m/CALDaNm1=MedhW23NuoePJTmonwsMSp80ddsw+sEJs0GUMC_kqQ@mail.gmail.com
M doc/src/sgml/ref/create_subscription.sgml
M src/backend/commands/subscriptioncmds.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/test/regress/expected/subscription.out
M src/test/subscription/t/022_twophase_cascade.pl
Set query ID for inner queries of CREATE TABLE AS and DECLARE
commit : 6b652e6ce85a977e4ca7b8cc045cf4f3457b2d7b
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 28 Oct 2024 09:03:20 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 28 Oct 2024 09:03:20 +0900
Some utility statements contain queries that can be planned and
executed: CREATE TABLE AS and DECLARE CURSOR. This commit adds query ID
computation for the inner queries executed by these two utility
commands, with and without EXPLAIN. This change leads to four new
callers of JumbleQuery() and post_parse_analyze_hook() so as extensions
can decide what to do with this new data.
Previously, extensions relying on the query ID, like pg_stat_statements,
were not able to track these nested queries as the query_id was 0.
For pg_stat_statements, this commit leads to additions under !toplevel
when pg_stat_statements.track is set to "all", as shown in its
regression tests. The output of EXPLAIN for these two utilities gains a
"Query Identifier" if compute_query_id is enabled.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
M contrib/pg_stat_statements/expected/level_tracking.out
M src/backend/commands/createas.c
M src/backend/commands/explain.c
M src/backend/commands/portalcmds.c
M src/backend/commands/prepare.c
M src/include/commands/explain.h
M src/include/commands/prepare.h
M src/test/regress/expected/explain.out
M src/test/regress/sql/explain.sql
Fix obsolete nbtree split buffer comment.
commit : 33b2fbe0504bd349c3bb86e8711f5531c8ca84b7
author : Peter Geoghegan <pg@bowt.ie>
date : Sun, 27 Oct 2024 10:38:24 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Sun, 27 Oct 2024 10:38:24 -0400
Oversight in commit d088ba5a.
M src/backend/access/nbtree/nbtinsert.c
Remove unused #include's from backend .c files
commit : e18512c000eeb0da3509c37ef4a3ce0530b396bd
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 27 Oct 2024 08:18:25 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 27 Oct 2024 08:18:25 +0100
as determined by IWYU
These are mostly issues that are new since commit dbbca2cf299.
Discussion: https://www.postgresql.org/message-id/flat/0df1d5b1-8ca8-4f84-93be-121081bde049%40eisentraut.org
M src/backend/access/brin/brin_bloom.c
M src/backend/access/gin/ginvacuum.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/access/sequence/sequence.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogwait.c
M src/backend/backup/basebackup.c
M src/backend/backup/basebackup_incremental.c
M src/backend/catalog/pg_depend.c
M src/backend/commands/analyze.c
M src/backend/commands/cluster.c
M src/backend/commands/copyfromparse.c
M src/backend/commands/createas.c
M src/backend/commands/indexcmds.c
M src/backend/commands/vacuum.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/backend/executor/nodeHash.c
M src/backend/lib/integerset.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/libpq/be-secure-openssl.c
M src/backend/main/main.c
M src/backend/parser/gram.y
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_jsontable.c
M src/backend/postmaster/auxprocess.c
M src/backend/postmaster/launch_backend.c
M src/backend/postmaster/postmaster.c
M src/backend/postmaster/walwriter.c
M src/backend/replication/logical/slotsync.c
M src/backend/replication/slotfuncs.c
M src/backend/storage/aio/read_stream.c
M src/backend/storage/buffer/buf_init.c
M src/backend/storage/file/fd.c
M src/backend/storage/ipc/ipci.c
M src/backend/storage/ipc/sinvaladt.c
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/s_lock.c
M src/backend/storage/page/checksum.c
M src/backend/storage/smgr/bulk_write.c
M src/backend/tcop/fastpath.c
M src/backend/tcop/postgres.c
M src/backend/utils/Gen_fmgrtab.pl
M src/backend/utils/activity/backend_progress.c
M src/backend/utils/activity/backend_status.c
M src/backend/utils/activity/pgstat.c
M src/backend/utils/activity/pgstat_relation.c
M src/backend/utils/activity/wait_event.c
M src/backend/utils/adt/float.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/adt/pg_upgrade_support.c
M src/backend/utils/adt/waitfuncs.c
M src/backend/utils/hash/pg_crc.c
M src/backend/utils/misc/injection_point.c
M src/backend/utils/misc/pg_config.c
Refactor the code to create a pg_locale_t into new function.
commit : 3aa2373c114124f62e80016d8939331fcb4d5586
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 25 Oct 2024 16:31:08 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 25 Oct 2024 16:31:08 -0700
Reviewed-by: Andreas Karlsson
Discussion: https://postgr.es/m/59da7ee4-5e1a-4727-b464-a603c6ed84cd@proxel.se
M src/backend/utils/adt/pg_locale.c
Read extension script files in text not binary mode.
commit : 924e03917d6f7c7536b2a1228b5422f175f58f9e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 25 Oct 2024 12:19:58 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 25 Oct 2024 12:19:58 -0400
This change affects only Windows, where it should cause DOS-style
newlines (\r\n) to be converted to plain \n during script loading.
This eliminates one potential discrepancy in the behavior of
extension script files between Windows and non-Windows. While
there's a small chance that this might cause undesirable behavior
changes for some extensions, it can also be argued that this may
remove behavioral surprises for others. An example is that in
the buildfarm, we are getting different results for the tests
added by commit 774171c4f depending on whether our git tree has
been checked out with Unix or DOS newlines.
The choice to use binary mode goes all the way back to our invention
of extensions in commit d9572c4e3. However, I suspect it was not
thought through carefully but was just a side-effect of the ready
availability of an almost-suitable function read_binary_file().
On balance, changing to text mode seems like a better answer than
other ways in which we might fix the inconsistent test results.
Discussion: https://postgr.es/m/2480333.1729784872@sss.pgh.pa.us
M src/backend/commands/extension.c
Make table_scan_bitmap_next_block() async-friendly
commit : de380a62b5dae610b3504b5036e5d5b1150cc4a4
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 25 Oct 2024 10:11:58 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 25 Oct 2024 10:11:58 -0400
Move all responsibility for indicating a block is exhuasted into
table_scan_bitmap_next_tuple() and advance the main iterator in
heap-specific code. This flow control makes more sense and is a step
toward using the read stream API for bitmap heap scans.
Previously, table_scan_bitmap_next_block() returned false to indicate
table_scan_bitmap_next_tuple() should not be called for the tuples on
the page. This happened both when 1) there were no visible tuples on the
page and 2) when the block returned by the iterator was past the end of
the table. BitmapHeapNext() (generic bitmap table scan code) handled the
case when the bitmap was exhausted.
It makes more sense for table_scan_bitmap_next_tuple() to return false
when there are no visible tuples on the page and
table_scan_bitmap_next_block() to return false when the bitmap is
exhausted or there are no more blocks in the table.
As part of this new design, TBMIterateResults are no longer used as a
flow control mechanism in BitmapHeapNext(), so we removed
table_scan_bitmap_next_tuple's TBMIterateResult parameter.
Note that the prefetch iterator is still saved in the
BitmapHeapScanState node and advanced in generic bitmap table scan code.
This is because 1) it was not necessary to change the prefetch iterator
location to change the flow control in BitmapHeapNext() 2) modifying
prefetch iterator management requires several more steps better split
over multiple commits and 3) the prefetch iterator will be removed once
the read stream API is used.
Author: Melanie Plageman
Reviewed-by: Tomas Vondra, Andres Freund, Heikki Linnakangas, Mark Dilger
Discussion: https://postgr.es/m/063e4eb4-32d9-439e-a0b1-75565a9835a8%40iki.fi
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/include/access/relscan.h
M src/include/access/tableam.h
M src/include/nodes/execnodes.h
Move EXPLAIN counter increment to heapam_scan_bitmap_next_block
commit : 7bd7aa4d30676de006636bb2c9c079c363d9d56c
author : Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 25 Oct 2024 10:11:46 -0400
committer: Melanie Plageman <melanieplageman@gmail.com>
date : Fri, 25 Oct 2024 10:11:46 -0400
Increment the lossy and exact page counters for EXPLAIN of bitmap heap
scans in heapam_scan_bitmap_next_block(). Note that other table AMs will
need to do this as well
Pushing the counters into heapam_scan_bitmap_next_block() is required to
be able to use the read stream API for bitmap heap scans. The bitmap
iterator must be advanced from inside the read stream callback, so
TBMIterateResults cannot be used as a flow control mechanism in
BitmapHeapNext().
Author: Melanie Plageman
Reviewed-by: Tomas Vondra, Heikki Linnakangas
Discussion: https://postgr.es/m/063e4eb4-32d9-439e-a0b1-75565a9835a8%40iki.fi
M src/backend/access/heap/heapam_handler.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/include/access/tableam.h
WAL-log inplace update before revealing it to other sessions.
commit : 8e7e672cdaa6bfec85d4d5dd9be84159df23bb41
author : Noah Misch <noah@leadboat.com>
date : Fri, 25 Oct 2024 06:51:03 -0700
committer: Noah Misch <noah@leadboat.com>
date : Fri, 25 Oct 2024 06:51:03 -0700
A buffer lock won't stop a reader having already checked tuple
visibility. If a vac_update_datfrozenid() and then a crash happened
during inplace update of a relfrozenxid value, datfrozenxid could
overtake relfrozenxid. That could lead to "could not access status of
transaction" errors. Back-patch to v12 (all supported versions). In
v14 and earlier, this also back-patches the assertion removal from
commit 7fcf2faf9c7dd473208fd6d5565f88d7f733782b.
Discussion: https://postgr.es/m/20240620012908.92.nmisch@google.com
M src/backend/access/heap/README.tuplock
M src/backend/access/heap/heapam.c
For inplace update, send nontransactional invalidations.
commit : 243e9b40f1b2dd09d6e5bf91ebf6e822a2cd3704
author : Noah Misch <noah@leadboat.com>
date : Fri, 25 Oct 2024 06:51:02 -0700
committer: Noah Misch <noah@leadboat.com>
date : Fri, 25 Oct 2024 06:51:02 -0700
The inplace update survives ROLLBACK. The inval didn't, so another
backend's DDL could then update the row without incorporating the
inplace update. In the test this fixes, a mix of CREATE INDEX and ALTER
TABLE resulted in a table with an index, yet relhasindex=f. That is a
source of index corruption. Back-patch to v12 (all supported versions).
The back branch versions don't change WAL, because those branches just
added end-of-recovery SIResetAll(). All branches change the ABI of
extern function PrepareToInvalidateCacheTuple(). No PGXN extension
calls that, and there's no apparent use case in extensions.
Reviewed by Nitin Motiani and (in earlier versions) Andres Freund.
Discussion: https://postgr.es/m/20240523000548.58.nmisch@google.com
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_xlog.c
M src/backend/access/rmgrdesc/heapdesc.c
M src/backend/access/rmgrdesc/standbydesc.c
M src/backend/access/transam/xact.c
M src/backend/catalog/index.c
M src/backend/commands/event_trigger.c
M src/backend/replication/logical/decode.c
M src/backend/utils/cache/catcache.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/syscache.c
M src/include/access/heapam_xlog.h
M src/include/access/xlog_internal.h
M src/include/storage/sinval.h
M src/include/utils/catcache.h
M src/include/utils/inval.h
M src/test/isolation/expected/inplace-inval.out
M src/test/isolation/specs/inplace-inval.spec
M src/tools/pgindent/typedefs.list
doc: Fix typo in pg_restore_*_stats function documentation
commit : 0fe173680e148984a150326b80c322a91ffa899d
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 25 Oct 2024 14:00:13 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 25 Oct 2024 14:00:13 +0200
Fix accidental typo from d32d146399, s/intepretation/interpretation/
M doc/src/sgml/func.sgml
Fix concurrrently in typcache_rel_type_cache.sql
commit : aa1e898dea666629bb5314fd25445545b9eb7b27
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 25 Oct 2024 13:07:07 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 25 Oct 2024 13:07:07 +0300
All injection points there must be local. Otherwise it affects parallel
tests.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/b3ybc66l6lhmtzj2n7ypumz5yjz7njc46sddsqshdtstgj74ah%40qgtn6nzokj6a
M src/test/modules/typcache/expected/typcache_rel_type_cache.out
M src/test/modules/typcache/sql/typcache_rel_type_cache.sql
Doc: Add a caution in alter publication.
commit : b8a046081c43ce4a0a462812165f34b6026e60ef
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 25 Oct 2024 14:19:05 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 25 Oct 2024 14:19:05 +0530
Clarify that altering the 'publish_via_partition_root' option can lead to
data loss or duplication when a partition root table is specified as the
replication target.
Reported-by: Maxim Boguk
Author: Hayato Kuroda
Reviewed-by: Amit Kapila, Peter Smith, Vignesh C
Discussion: https://postgr.es/m/18644-6866bbd22178ee16@postgresql.org
M doc/src/sgml/ref/alter_publication.sgml
pgbench: Fix typo.
commit : 7175ef870e24763a561821685160c5e35b2e8989
author : Tatsuo Ishii <ishii@postgresql.org>
date : Fri, 25 Oct 2024 15:06:16 +0900
committer: Tatsuo Ishii <ishii@postgresql.org>
date : Fri, 25 Oct 2024 15:06:16 +0900
Fix typo in commit cae0f3c405.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/70eaa41b-805b-ce19-6004-5a0dccd3f731%40gmail.com
M src/bin/pgbench/pgbench.c
Refactor code converting a publication name List to a StringInfo
commit : 248c2d19238c7616531caa8b3101cca4f656185c
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Oct 2024 12:02:04 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Oct 2024 12:02:04 +0900
The existing get_publications_str() is renamed to GetPublicationsStr()
and is moved to pg_subscription.c, so as it is possible to reuse it at
two locations of the tablesync code where the same logic was duplicated.
fetch_remote_table_info() was doing two List->StringInfo conversions
when dealing with a server of version 15 or newer. The conversion
happens only once now.
This refactoring leads to less code overall.
Author: Peter Smith
Reviewed-by: Michael Paquier, Masahiko Sawada
Discussion: https://postgr.es/m/CAHut+PtJMk4bKXqtpvqVy9ckknCgK9P6=FeG8zHF=6+Em_Snpw@mail.gmail.com
M src/backend/catalog/pg_subscription.c
M src/backend/commands/subscriptioncmds.c
M src/backend/replication/logical/tablesync.c
M src/include/catalog/pg_subscription.h
Add install rules for Kerberos.pm and AdjustUpgrade.pm
commit : 1564339bfe23b2d1aa6403d6a08a7eaae0373cf9
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Oct 2024 10:56:34 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Oct 2024 10:56:34 +0900
For the same reasons as c3a0818460a8, these can be useful for
out-of-core extension testing. Kerberos.pm has been moved to its
current path recently in 9f899562d420, and AdjustUpgrade.pm has been
introduced in 52585f8f072a, still both lacked [un]installation rules for
both meson and configure.
Reported-by: Ashutosh Bapat
Discussion: https://postgr.es/m/ZozqzznkDhfCG7Ng@paquier.xyz
M src/test/perl/Makefile
M src/test/perl/meson.build
injection_points: Disable one permutation in isolation test "basic"
commit : 9f00edc2288892a55e2bef464f555e9f5396ff1c
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Oct 2024 10:34:27 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Oct 2024 10:34:27 +0900
The first permutation done in the test does a wait, a wakeup then a
detach. It is proving to be unstable in the CI for FreeBSD (Windows and
Linux are stable). The failure shows that the wait is so slow to finish
after being woken up that the detach has the time to finish before the
wait, messing up with the expected output.
There may be a platform-specific issue going on here, but for now
disable this permutation to make the CI runs more stable.
Discussion: https://postgr.es/m/ZxrnSGdNtQWAxE3_@paquier.xyz
M src/test/modules/injection_points/expected/basic.out
M src/test/modules/injection_points/specs/basic.spec
Remove the RTE_GROUP RTE if we drop the groupClause
commit : ffe12d1d22e73d7bcda1f0ee9af33d04fab199b2
author : Richard Guo <rguo@postgresql.org>
date : Fri, 25 Oct 2024 09:52:34 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Fri, 25 Oct 2024 09:52:34 +0900
For an EXISTS subquery, the only thing that matters is whether it
returns zero or more than zero rows. Therefore, we remove certain SQL
features that won't affect that, among them the GROUP BY clauses.
After we drop the groupClause, we'd better remove the RTE_GROUP RTE
and clear the hasGroupRTE flag, as they depend on the groupClause.
Failing to do so could result in a bogus RTE_GROUP entry in the parent
query, leading to an assertion failure on the hasGroupRTE flag.
Reported-by: David Rowley
Author: Richard Guo
Discussion: https://postgr.es/m/CAApHDvp2_yht8uPLyWO-kVGWZhYvx5zjGfSrg4fBQ9fsC13V0g@mail.gmail.com
M src/backend/optimizer/plan/subselect.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Add functions pg_restore_relation_stats(), pg_restore_attribute_stats().
commit : d32d1463995c036853eeb9ec99cc367ffc7794ae
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 24 Oct 2024 12:08:00 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 24 Oct 2024 12:08:00 -0700
Similar to the pg_set_*_stats() functions, except with a variadic
signature that's designed to be more future-proof. Additionally, most
problems are reported as WARNINGs rather than ERRORs, allowing most
stats to be restored even if some cannot.
These functions are intended to be called from pg_dump to avoid the
need to run ANALYZE after an upgrade.
Author: Corey Huinker
Discussion: https://postgr.es/m/CADkLM=eErgzn7ECDpwFcptJKOk9SxZEk5Pot4d94eVTZsvj3gw@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/statistics/attribute_stats.c
M src/backend/statistics/relation_stats.c
M src/backend/statistics/stat_utils.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/statistics/stat_utils.h
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Generalize plpgsql's heuristic for importing expanded objects.
commit : 534d0ea6c2b915ac9745d2f070afacd7ba003d28
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Oct 2024 13:28:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Oct 2024 13:28:22 -0400
If a R/W expanded-object pointer is passed as a function parameter,
take ownership of the object, regardless of its type. Previously
this happened only for expanded arrays, but that was a result of
sloppy thinking. (If the plpgsql function did not end by returning
the object, the result would be to leak the object until the
surrounding memory context is cleaned up. That's not awful,
since non-expanded values have always been managed that way,
but we can do better.)
Per discussion with Michel Pelletier. There's a lot more to do
here to make plpgsql work efficiently with expanded objects that
aren't arrays, but this is an easy first step.
Discussion: https://postgr.es/m/CACxu=vJaKFNsYxooSnW1wEgsAO5u_v1XYBacfVJ14wgJV_PYeg@mail.gmail.com
M src/pl/plpgsql/src/pl_exec.c
Fix parallel worker tracking of new catalog relfilenumbers.
commit : 67bab53d64428e4d43386be0e077cd662bc9cad3
author : Noah Misch <noah@leadboat.com>
date : Thu, 24 Oct 2024 09:16:38 -0700
committer: Noah Misch <noah@leadboat.com>
date : Thu, 24 Oct 2024 09:16:38 -0700
Reunite RestorePendingSyncs() with RestoreRelationMap(). If
RelationInitPhysicalAddr() ran after RestoreRelationMap() but before
RestorePendingSyncs(), the relcache entry could cause RelationNeedsWAL()
to return true erroneously. Trouble required commands of the current
transaction to include REINDEX or CLUSTER of a system catalog. The
parallel leader correctly derived RelationNeedsWAL()==false from the new
relfilenumber, but the worker saw RelationNeedsWAL()==true. Worker
MarkBufferDirtyHint() then wrote unwanted WAL. Recovery of that
unwanted WAL could lose tuples like the system could before commit
c6b92041d38512a4176ed76ad06f713d2e6c01a8 introduced this tracking.
RestorePendingSyncs() and RestoreRelationMap() were adjacent till commit
126ec0bc76d044d3a9eb86538b61242bf7da6db4, so no back-patch for now.
Reviewed by Tom Lane.
Discussion: https://postgr.es/m/20241019232815.c6.nmisch@google.com
M src/backend/access/transam/parallel.c
Stop reading uninitialized memory in heap_inplace_lock().
commit : e947224cbbceca535fd00001ba459417e6917d1b
author : Noah Misch <noah@leadboat.com>
date : Thu, 24 Oct 2024 09:16:14 -0700
committer: Noah Misch <noah@leadboat.com>
date : Thu, 24 Oct 2024 09:16:14 -0700
Stop computing a never-used value. This removes the read; the read had
no functional implications. Back-patch to v12, like commit
a07e03fd8fa7daf4d1356f7cb501ffe784ea6257.
Reported by Alexander Lakhin.
Discussion: https://postgr.es/m/6c92f59b-f5bc-e58c-9bdd-d1f21c17c786@gmail.com
M src/backend/access/heap/heapam.c
M src/test/isolation/expected/intra-grant-inplace.out
M src/test/isolation/specs/intra-grant-inplace.spec
Refactor GetLockStatusData() to skip backends/groups without fast-path locks.
commit : 86c30cef4a4cae951b2c30f1790fd5a7b81a2946
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 25 Oct 2024 00:18:32 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 25 Oct 2024 00:18:32 +0900
Previously, GetLockStatusData() checked all slots for every backend
to gather fast-path lock data, which could be inefficient. This commit
refactors it by skipping backends with PID=0 (since they don't hold
fast-path locks) and skipping groups with no registered fast-path locks,
improving efficiency.
This refactoring is particularly beneficial, for example when
max_connections and max_locks_per_transaction are set high,
as it reduces unnecessary checks across numerous slots.
Author: Fujii Masao
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/a0a00c44-31e9-4c67-9846-fb9636213ac9@oss.nttdata.com
M src/backend/storage/lmgr/lock.c
Support configuring TLSv1.3 cipher suites
commit : 45188c2ea2391b7b24039e1632c726e2fc6b8008
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 24 Oct 2024 15:20:32 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 24 Oct 2024 15:20:32 +0200
The ssl_ciphers GUC can only set cipher suites for TLSv1.2, and lower,
connections. For TLSv1.3 connections a different OpenSSL API must be
used. This adds a new GUC, ssl_tls13_ciphers, which can be used to
configure a colon separated list of cipher suites to support when
performing a TLSv1.3 handshake.
Original patch by Erica Zhang with additional hacking by me.
Author: Erica Zhang <ericazhangy2021@qq.com>
Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Discussion: https://postgr.es/m/tencent_063F89FA72CCF2E48A0DF5338841988E9809@qq.com
M doc/src/sgml/config.sgml
M src/backend/libpq/be-secure-openssl.c
M src/backend/libpq/be-secure.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/libpq/libpq.h
M src/test/ssl/t/SSL/Server.pm
Support configuring multiple ECDH curves
commit : 3d1ef3a15c3eb68dae44b94e89d04c422b26fc16
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 24 Oct 2024 15:20:28 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 24 Oct 2024 15:20:28 +0200
The ssl_ecdh_curve GUC only accepts a single value, but the TLS
handshake can list multiple curves in the groups extension (the
extension has been renamed to contain more than elliptic curves).
This changes the GUC to accept a colon-separated list of curves.
This commit also renames the GUC to ssl_groups to match the new
nomenclature for the TLS extension.
Original patch by Erica Zhang with additional hacking by me.
Author: Erica Zhang <ericazhangy2021@qq.com>
Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Discussion: https://postgr.es/m/tencent_063F89FA72CCF2E48A0DF5338841988E9809@qq.com
M doc/src/sgml/config.sgml
M src/backend/libpq/be-secure-openssl.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/guc_tables.c
M src/backend/utils/misc/postgresql.conf.sample
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/SSL/Server.pm
Raise the minimum supported OpenSSL version to 1.1.1
commit : 6c66b7443cebf3ff09ea76416a20fb6bb1d32a52
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 24 Oct 2024 15:20:19 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 24 Oct 2024 15:20:19 +0200
Commit a70e01d4306fdbcd retired support for OpenSSL 1.0.2 in order to get
rid of the need for manual initialization of the library. This left our
API usage compatible with 1.1.0 which was defined as the minimum required
version. Also mention that 3.4 is the minimum version required when using
LibreSSL.
An upcoming commit will introduce support for configuring TLSv1.3 cipher
suites which require an API call in OpenSSL 1.1.1 and onwards. In order
to support this setting this commit will set v1.1.1 as the new minimum
required version. The version-specific call for randomness init added
in commit c3333dbc0c0 is removed as it's no longer needed.
Author: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/909A668B-06AD-47D1-B8EB-A164211AAD16@yesql.se
Discussion: https://postgr.es/m/tencent_063F89FA72CCF2E48A0DF5338841988E9809@qq.com
M configure
M configure.ac
M doc/src/sgml/installation.sgml
M meson.build
M src/include/pg_config.h.in
M src/port/pg_strong_random.c
Handle alphanumeric characters in matching GUC names
commit : f81855171f95c4440d1d9f41bc9e5ac47eafb7a0
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 24 Oct 2024 15:20:16 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 24 Oct 2024 15:20:16 +0200
The check for whether all GUCs are present in the sample config
file used the POSIX character class :alpha: which corresponds to
alphabet and not alphanumeric. Since GUC names can contain digits
as well we need to use the :alnum: character class instead.
Author: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/2CB04559-B1D8-4558-B6F0-8F09093D629F@yesql.se
M src/test/modules/test_misc/t/003_check_guc.pl
Add 'no_error' argument to pg_wal_replay_wait()
commit : e546989a269d5d73d283901aadcfda8c6d98e87b
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 24 Oct 2024 14:40:23 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 24 Oct 2024 14:40:23 +0300
This argument allow skipping throwing an error. Instead, the result status
can be obtained using pg_wal_replay_wait_status() function.
Catversion is bumped.
Reported-by: Michael Paquier
Discussion: https://postgr.es/m/ZtUF17gF0pNpwZDI%40paquier.xyz
Reviewed-by: Pavel Borisov
M doc/src/sgml/func.sgml
M src/backend/access/transam/xlogfuncs.c
M src/backend/access/transam/xlogwait.c
M src/backend/catalog/system_functions.sql
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/recovery/t/043_wal_replay_wait.pl
Refactor WaitForLSNReplay() to return the result of waiting
commit : 73da6b8d1b3e8b7541961c3534e584243cb0470e
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 24 Oct 2024 14:38:27 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 24 Oct 2024 14:38:27 +0300
Currently, WaitForLSNReplay() immediately throws an error if waiting for LSN
replay is not successful. This commit teaches WaitForLSNReplay() to return
the result of waiting, while making pg_wal_replay_wait() responsible for
throwing an appropriate error.
This is preparation to adding 'no_error' argument to pg_wal_replay_wait() and
new function pg_wal_replay_wait_status(), which returns the last wait result
status.
Additionally, we stop distinguishing situations when we find our instance to
be not in a recovery state before entering the waiting loop and inside
the waiting loop. Standby promotion may happen at any moment, even between
issuing a procedure call statement and pg_wal_replay_wait() doing a first
check of recovery status. Thus, there is no pointing distinguishing these
situations.
Also, since we may exit the waiting loop and see our instance not in recovery
without throwing an error, we need to deleteLSNWaiter() in that case. We do
this unconditionally for the sake of simplicity, even if standby was already
promoted after reaching the target LSN, the startup process surely already
deleted us.
Reported-by: Michael Paquier
Discussion: https://postgr.es/m/ZtUF17gF0pNpwZDI%40paquier.xyz
Reviewed-by: Michael Paquier, Pavel Borisov
M src/backend/access/transam/xlogfuncs.c
M src/backend/access/transam/xlogwait.c
M src/include/access/xlogwait.h
M src/tools/pgindent/typedefs.list
Make WaitForLSNReplay() issue FATAL on postmaster death
commit : 6cfebfe88b9a753152862b83d42d1103125b01bd
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 24 Oct 2024 14:38:06 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 24 Oct 2024 14:38:06 +0300
Reported-by: Michael Paquier
Discussion: https://postgr.es/m/ZvY2C8N4ZqgCFaLu%40paquier.xyz
Reviewed-by: Pavel Borisov
M src/backend/access/transam/xlogwait.c
Move LSN waiting declarations and definitions to better place
commit : 5035172e4ab58e4e8eef1bc60b0712fc59e0be31
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 24 Oct 2024 14:37:53 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 24 Oct 2024 14:37:53 +0300
3c5db1d6b implemented the pg_wal_replay_wait() stored procedure. Due to
the patch development history, the implementation resided in
src/backend/commands/waitlsn.c (src/include/commands/waitlsn.h for headers).
014f9f34d moved pg_wal_replay_wait() itself to
src/backend/access/transam/xlogfuncs.c near to the WAL-manipulation functions.
But most of the implementation stayed in place.
The code in src/backend/commands/waitlsn.c has nothing to do with commands,
but is related to WAL. So, this commit moves this code into
src/backend/access/transam/xlogwait.c (src/include/access/xlogwait.h for
headers).
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/18c0fa64-0475-415e-a1bd-665d922c5201%40eisentraut.org
Reviewed-by: Pavel Borisov
M src/backend/access/transam/Makefile
M src/backend/access/transam/meson.build
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogfuncs.c
M src/backend/access/transam/xlogrecovery.c
R099 src/backend/commands/waitlsn.c src/backend/access/transam/xlogwait.c
M src/backend/commands/Makefile
M src/backend/commands/meson.build
M src/backend/storage/ipc/ipci.c
M src/backend/storage/lmgr/proc.c
R094 src/include/commands/waitlsn.h src/include/access/xlogwait.h
Avoid looping over all type cache entries in TypeCacheRelCallback()
commit : b85a9d046efdd27775cbe7db9e92aad96aab4ada
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 24 Oct 2024 14:35:52 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 24 Oct 2024 14:35:52 +0300
Currently, when a single relcache entry gets invalidated,
TypeCacheRelCallback() has to loop over all type cache entries to find
appropriate typentry to invalidate. Unfortunately, using the syscache here
is impossible, because this callback could be called outside a transaction
and this makes impossible catalog lookups. This is why present commit
introduces RelIdToTypeIdCacheHash to map relation OID to its composite type
OID.
We are keeping RelIdToTypeIdCacheHash entry while corresponding type cache
entry have something to clean. Therefore, RelIdToTypeIdCacheHash shouldn't
get bloat in the case of temporary tables flood.
There are many places in lookup_type_cache() where syscache invalidation,
user interruption, or even error could occur. In order to handle this, we
keep an array of in-progress type cache entries. In the case of
lookup_type_cache() interruption this array is processed to keep
RelIdToTypeIdCacheHash in a consistent state.
Discussion: https://postgr.es/m/5812a6e5-68ae-4d84-9d85-b443176966a1%40sigaev.ru
Author: Teodor Sigaev
Reviewed-by: Aleksander Alekseev, Tom Lane, Michael Paquier, Roman Zharkov
Reviewed-by: Andrei Lepikhov, Pavel Borisov, Jian He, Alexander Lakhin
Reviewed-by: Artur Zakirov
M src/backend/access/transam/xact.c
M src/backend/utils/cache/typcache.c
M src/include/utils/typcache.h
M src/test/modules/Makefile
M src/test/modules/meson.build
A src/test/modules/typcache/.gitignore
A src/test/modules/typcache/Makefile
A src/test/modules/typcache/expected/typcache_rel_type_cache.out
A src/test/modules/typcache/meson.build
A src/test/modules/typcache/sql/typcache_rel_type_cache.sql
M src/tools/pgindent/typedefs.list
Update header comment for lookup_type_cache()
commit : c1500a1ba7e16ac9e98c6baf792f0be64a48e839
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 24 Oct 2024 14:34:16 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 24 Oct 2024 14:34:16 +0300
Describe the way we handle concurrent invalidation messages.
Discussion: https://postgr.es/m/CAPpHfdsQhwUrnB3of862j9RgHoJM--eRbifvBMvtQxpC57dxCA%40mail.gmail.com
Reviewed-by: Andrei Lepikhov, Artur Zakirov, Pavel Borisov
M src/backend/utils/cache/typcache.c
Track more precisely query locations for nested statements
commit : 499edb09741b8fad2de038361fb342aae6e6007f
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 24 Oct 2024 09:28:51 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 24 Oct 2024 09:28:51 +0900
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
M contrib/pg_stat_statements/expected/level_tracking.out
M contrib/pg_stat_statements/expected/planning.out
M contrib/pg_stat_statements/expected/select.out
M contrib/pg_stat_statements/expected/utility.out
M contrib/pg_stat_statements/sql/planning.sql
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/include/nodes/parsenodes.h
M src/include/parser/parse_node.h
Improve pg_set_attribute_stats() error message.
commit : 4b096c67e0eed81e287094b9692fff72b9ea3eef
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 23 Oct 2024 16:11:45 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 23 Oct 2024 16:11:45 -0700
Previously, an invalid attribute name was caught, but the error
message was unhelpful.
M src/backend/statistics/attribute_stats.c
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Fix typo in tidstore.h.
commit : 7b8b8dddd68aca184ae12e01d2f12d1d338e8f90
author : Masahiko Sawada <msawada@postgresql.org>
date : Wed, 23 Oct 2024 15:37:00 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Wed, 23 Oct 2024 15:37:00 -0700
An oversight in commit f6bef362c.
Reviewed-by: David Rowley
Discussion: https://postgr.es/m/CAD21AoB8MJ5OHtpUw1UEGf7spioFmP3PNH44KNx6Yb3FiZSwKA%40mail.gmail.com
M src/include/access/tidstore.h
Another documentation fixup.
commit : 0a3f9838218f6f7ff825bb0af00177681fef83ce
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 23 Oct 2024 10:28:31 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 23 Oct 2024 10:28:31 -0700
Reported-by: Erik Rijkers
M doc/src/sgml/func.sgml
Fix compiler warning.
commit : 56b1e88c804b0c7e32f77b4e2bd3cc42ebdfcc3f
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 23 Oct 2024 10:24:17 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 23 Oct 2024 10:24:17 -0700
Some buildfarm members complained about an always-true test in the
SOFT_ERROR_OCCURRED macro. Fix by reading the field directly rather
than using the macro.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/2144895.1729653514@sss.pgh.pa.us
M src/backend/statistics/attribute_stats.c
Documentation fixup.
commit : 07d00692c8da83fe2835900b2294e8966c2be247
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 23 Oct 2024 09:43:06 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 23 Oct 2024 09:43:06 -0700
Wrong return type for pg_clear_attribute_stats().
Author: Noriyoshi Shinoda
Discussion: https://postgr.es/m/DM4PR84MB17347944F27A552F0CCDF84CEE4C2@DM4PR84MB1734.NAMPRD84.PROD.OUTLOOK.COM
M doc/src/sgml/func.sgml
Fix incorrect struct reference in comment
commit : 940f7a562731084f7cb23c0a56e46056340ab664
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 23 Oct 2024 16:13:28 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 23 Oct 2024 16:13:28 +0200
SASL frontend mechanisms are implemented with pg_fe_sasl_mech and
not the _be_ variant which is the backend implementation. Spotted
while reading adjacent code.
M src/interfaces/libpq/fe-auth-sasl.h
Make SASL max message length configurable
commit : 6d16f9debae0a09a7db91d7f204bda043d04792c
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 23 Oct 2024 16:10:27 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 23 Oct 2024 16:10:27 +0200
The proposed OAUTHBEARER SASL mechanism will need to allow larger
messages in the exchange, since tokens are sent directly by the
client. Move this limit into the pg_be_sasl_mech struct so that
it can be changed per-mechanism.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAOYmi+nqX_5=Se0W0Ynrr55Fha3CMzwv_R9P3rkpHb=1kG7ZTQ@mail.gmail.com
M src/backend/libpq/auth-sasl.c
M src/backend/libpq/auth-scram.c
M src/include/libpq/sasl.h
doc: Fix INSERT statement syntax for identity columns
commit : 17b4aa77c3a161a9d0e7b08bd9931fe9fd051540
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 23 Oct 2024 14:58:17 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 23 Oct 2024 14:58:17 +0200
The INSERT statements in the examples were erroneously using
VALUE instead of VALUES. Backpatch to v17 where the examples
were added through a37bb7c1399.
Reported-by: shixiong327926@gmail.com
Discussion: https://postgr.es/m/172958472112.696.6075270400394560263@wrigleys.postgresql.org
Backpatch-through: 17
M doc/src/sgml/ddl.sgml
Remove unnecessary word in a comment
commit : 55e6d712aff83587c2e06899a794c52fbfb415f6
author : Amit Langote <amitlan@postgresql.org>
date : Wed, 23 Oct 2024 17:54:48 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Wed, 23 Oct 2024 17:54:48 +0900
Relations opened by the executor are only closed once in
ExecCloseRangeTableRelations(), so the word "again" in the comment
for ExecGetRangeTableRelation() is misleading and unnecessary.
Discussion: https://postgr.es/m/CA+HiwqHnw-zR+u060i3jp4ky5UR0CjByRFQz50oZ05de7wUg=Q@mail.gmail.com
Backpatch-through: 12
M src/backend/executor/execUtils.c
ecpg: Fix out-of-bound read in DecodeDateTime()
commit : a0bff38d133ac95fb56bbd868a128bba95ec46c9
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 23 Oct 2024 08:33:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 23 Oct 2024 08:33:54 +0900
It was possible for the code to read out-of-bound data from the
"day_tab" table with some crafted input data. Let's treat these as
invalid input as the month number is incorrect.
A test is added to test this case with a check on the errno returned by
the decoding routine. A test close to the new one added in this commit
was testing for a failure, but did not look at the errno generated, so
let's use this commit to also change it, adding a check on the errno
returned by DecodeDateTime().
Like the other test scripts, dt_test should likely be expanded to
include more checks based on the errnos generated in these code paths.
This is left as future work.
This issue exists since 2e6f97560a83, so backpatch all the way down.
Reported-by: Pavel Nekrasov
Author: Bruce Momjian, Pavel Nekrasov
Discussion: https://postgr.es/m/18614-6bbe00117352309e@postgresql.org
Backpatch-through: 12
M src/interfaces/ecpg/pgtypeslib/dt_common.c
M src/interfaces/ecpg/test/expected/pgtypeslib-dt_test.c
M src/interfaces/ecpg/test/expected/pgtypeslib-dt_test.stderr
M src/interfaces/ecpg/test/expected/pgtypeslib-dt_test.stdout
M src/interfaces/ecpg/test/pgtypeslib/dt_test.pgc
Add functions pg_set_attribute_stats() and pg_clear_attribute_stats().
commit : ce207d2a79017713c0293bf70132dc6d41166b03
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 22 Oct 2024 15:06:55 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 22 Oct 2024 15:06:55 -0700
Enable manipulation of attribute statistics. Only superficial
validation is performed, so it's possible to add nonsense, and it's up
to the planner (or other users of statistics) to behave reasonably in
that case.
Bump catalog version.
Author: Corey Huinker
Discussion: https://postgr.es/m/CADkLM=eErgzn7ECDpwFcptJKOk9SxZEk5Pot4d94eVTZsvj3gw@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/catalog/system_functions.sql
M src/backend/statistics/Makefile
A src/backend/statistics/attribute_stats.c
M src/backend/statistics/meson.build
M src/backend/statistics/stat_utils.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/statistics/stat_utils.h
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Change pg_*_relation_stats() functions to return type to void.
commit : dbe6bd4343d8cdb1b3cf1f66d6f66dd876a6c09d
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 22 Oct 2024 12:48:01 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 22 Oct 2024 12:48:01 -0700
These functions will either raise an ERROR or run to normal
completion, so no return value is necessary.
Bump catalog version.
Author: Corey Huinker
Discussion: https://postgr.es/m/CADkLM=cBF8rnphuTyHFi3KYzB9ByDgx57HwK9Rz2yp7S+Om87w@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/catalog/system_functions.sql
M src/backend/statistics/relation_stats.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/stats_import.out
Improve reporting of errors in extension script files.
commit : 774171c4f640853b1cf8747a4762631d2f5d25be
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Oct 2024 11:31:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Oct 2024 11:31:45 -0400
Previously, CREATE/ALTER EXTENSION gave basically no useful
context about errors reported while executing script files.
I think the idea was that you could run the same commands
manually to see the error, but that's often quite inconvenient.
Let's improve that.
If we get an error during raw parsing, we won't have a current
statement identified by a RawStmt node, but we should always get
a syntax error position. Show the portion of the script from
the last semicolon-newline before the error position to the first
one after it. There are cases where this might show only a
fragment of a statement, but that should be uncommon, and it
seems better than showing the whole script file.
Without an error cursor, if we have gotten past raw parsing (which
we probably have), we can report just the current SQL statement as
an item of error context.
In any case also report the script file name as error context,
since it might not be entirely obvious which of a series of
update scripts failed. We can also show an approximate script
line number in case whatever we printed of the query isn't
sufficiently identifiable.
The error-context code path is already exercised by some
test_extensions test cases, but add tests for the syntax-error
path.
Discussion: https://postgr.es/m/ZvV1ClhnbJLCz7Sm@msg.df7cb.de
M src/backend/commands/extension.c
M src/test/modules/test_extensions/Makefile
M src/test/modules/test_extensions/expected/test_extensions.out
M src/test/modules/test_extensions/meson.build
M src/test/modules/test_extensions/sql/test_extensions.sql
A src/test/modules/test_extensions/test_ext7–2.0–2.1bad.sql
A src/test/modules/test_extensions/test_ext7–2.0–2.2bad.sql
M src/tools/pgindent/typedefs.list
Improve parser's reporting of statement start locations.
commit : 14e5680eee19df8b41ef77330d0b3857f498e4f7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Oct 2024 11:26:05 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Oct 2024 11:26:05 -0400
Up to now, the parser's reporting of a statement's stmt_location
included any preceding whitespace or comments. This isn't really
desirable but was done to avoid accounting honestly for nonterminals
that reduce to empty. It causes problems for pg_stat_statements,
which partially compensates by manually stripping whitespace, but
is not bright enough to strip /*-style comments. There will be
more problems with an upcoming patch to improve reporting of errors
in extension scripts, so it's time to do something about this.
The thing we have to do to make it work right is to adjust
YYLLOC_DEFAULT to scan the inputs of each production to find the
first one that has a valid location (i.e., did not reduce to
empty). In theory this adds a little bit of per-reduction overhead,
but in practice it's negligible. I checked by measuring the time
to run raw_parser() on the contents of information_schema.sql, and
there was basically no change.
Having done that, we can rely on any nonterminal that didn't reduce
to completely empty to have a correct starting location, and we don't
need the kluges the stmtmulti production formerly used.
This should have a side benefit of allowing parse error reports to
include an error position in some cases where they formerly failed to
do so, due to trying to report the position of an empty nonterminal.
I did not go looking for an example though. The one previously known
case where that could happen (OptSchemaEltList) no longer needs the
kluge it had; but I rather doubt that that was the only case.
Discussion: https://postgr.es/m/ZvV1ClhnbJLCz7Sm@msg.df7cb.de
M contrib/pg_stat_statements/expected/select.out
M contrib/pg_stat_statements/sql/select.sql
M src/backend/nodes/queryjumblefuncs.c
M src/backend/parser/gram.y
ecpg: Refactor ecpg_log() to skip unnecessary calls to ECPGget_sqlca().
commit : 7c4d3fe272f3ab62b36f21e3fa6ae00751f00e31
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 22 Oct 2024 23:56:37 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 22 Oct 2024 23:56:37 +0900
Previously, ecpg_log() always called ECPGget_sqlca() to retrieve sqlca,
even though it was only needed for debug logging. This commit updates
ecpg_log() to call ECPGget_sqlca() only when debug logging is enabled.
Author: Yuto Sasaki
Reviewed-by: Alvaro Herrera, Tom Lane, Fujii Masao
Discussion: https://postgr.es/m/TY2PR01MB3628A85689649BABC9A1C6C3C1782@TY2PR01MB3628.jpnprd01.prod.outlook.com
M src/interfaces/ecpg/ecpglib/misc.c
Restructure foreign key handling code for ATTACH/DETACH
commit : 53af9491a0439720094a11b72602952d79f59ac7
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 22 Oct 2024 16:01:18 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 22 Oct 2024 16:01:18 +0200
... to fix bugs when the referenced table is partitioned.
The catalog representation we chose for foreign keys connecting
partitioned tables (in commit f56f8f8da6af) is inconvenient, in the
sense that a standalone table has a different way to represent the
constraint when referencing a partitioned table, than when the same
table becomes a partition (and vice versa). Because of this, we need to
create additional catalog rows on detach (pg_constraint and pg_trigger),
and remove them on attach. We were doing some of those things, but not
all of them, leading to missing catalog rows in certain cases.
The worst problem seems to be that we are missing action triggers after
detaching a partition, which means that you could update/delete rows
from the referenced partitioned table that still had referencing rows on
that table, the server failing to throw the required errors.
!!!
Note that this means existing databases with FKs that reference
partitioned tables might have rows that break relational integrity, on
tables that were once partitions on the referencing side of the FK.
Another possible problem is that trying to reattach a table
that had been detached would fail indicating that internal triggers
cannot be found, which from the user's point of view is nonsensical.
In branches 15 and above, we fix this by creating a new helper function
addFkConstraint() which is in charge of creating a standalone
pg_constraint row, and repurposing addFkRecurseReferencing() and
addFkRecurseReferenced() so that they're only the recursive routine for
each side of the FK, and they call addFkConstraint() to create
pg_constraint at each partitioning level and add the necessary triggers.
These new routines can be used during partition creation, partition
attach and detach, and foreign key creation. This reduces redundant
code and simplifies the flow.
In branches 14 and 13, we have a much simpler fix that consists on
simply removing the constraint on detach. The reason is that those
branches are missing commit f4566345cf40, which reworked the way this
works in a way that we didn't consider back-patchable at the time.
We opted to leave branch 12 alone, because it's different from branch 13
enough that the fix doesn't apply; and because it is going in EOL mode
very soon, patching it now might be worse since there's no way to undo
the damage if it goes wrong.
Existing databases might need to be repaired.
In the future we might want to rethink the catalog representation to
avoid this problem, but for now the code seems to do what's required to
make the constraints operate correctly.
Co-authored-by: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>
Co-authored-by: Tender Wang <tndrwang@gmail.com>
Co-authored-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reported-by: Guillaume Lelarge <guillaume@lelarge.info>
Reported-by: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>
Reported-by: Thomas Baehler (SBB CFF FFS) <thomas.baehler2@sbb.ch>
Discussion: https://postgr.es/m/20230420144344.40744130@karst
Discussion: https://postgr.es/m/20230705233028.2f554f73@karst
Discussion: https://postgr.es/m/GVAP278MB02787E7134FD691861635A8BC9032@GVAP278MB0278.CHEP278.PROD.OUTLOOK.COM
Discussion: https://postgr.es/m/18541-628a61bc267cd2d3@postgresql.org
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
M src/tools/pgindent/typedefs.list
Make all Perl warnings fatal in 043_wal_replay_wait.pl
commit : e1555645d729e0b91f644954e83e1ed51baa58df
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 22 Oct 2024 13:25:10 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 22 Oct 2024 13:25:10 +0300
This file was committed after c5385929593, but accidentally missed changing
all warnings into fatal errors.
Reported-by: Anton Voloshin
Discussion: https://postgr.es/m/aa8a55d5-554a-4027-a491-1b0ca7c85f7a%40postgrespro.ru
M src/test/recovery/t/043_wal_replay_wait.pl
Fix C23 compiler warning
commit : d2b4b4c2259e21ceaf05e393769b69728bfbee99
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 22 Oct 2024 08:12:43 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 22 Oct 2024 08:12:43 +0200
The approach of declaring a function pointer with an empty argument
list and hoping that the compiler will not complain about casting it
to another type no longer works with C23, because foo() is now
equivalent to foo(void).
We don't need to do this here. With a few struct forward declarations
we can supply a correct argument list without having to pull in
another header file.
(This is the only new warning with C23. Together with the previous
fix a67a49648d9, this makes the whole code compile cleanly under C23.)
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/95c6a9bf-d306-43d8-b880-664ef08f2944%40eisentraut.org
M src/include/nodes/pathnodes.h
pg_stat_statements: Add tests for nested queries with level tracking
commit : 45e0ba30fc40581f320fac17ad8b4e0676e1b3b5
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 22 Oct 2024 13:05:51 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 22 Oct 2024 13:05:51 +0900
There have never been any regression tests in PGSS for various query
patterns for nested queries combined with level tracking, like:
- Multi-statements.
- CREATE TABLE AS
- CREATE/REFRESH MATERIALIZED VIEW
- DECLARE CURSOR
- EXPLAIN, with a subset of the above supported.
- COPY.
All the tests added here track historical, sometimes confusing, existing
behaviors. For example, EXPLAIN stores two PGSS entries with the same
top-level query string but two different query IDs as one is calculated
for the top-level EXPLAIN (this part is right) and a second one for the
inner query in the EXPLAIN (this part is not right).
A couple of patches are under discussion to improve the situation, and
all the tests added here will prove useful to evaluate the changes
discussed.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
M contrib/pg_stat_statements/expected/level_tracking.out
M contrib/pg_stat_statements/sql/level_tracking.sql
Fix wrong assertion and poor error messages in "COPY (query) TO".
commit : 68ad9816c189958a0731d8d5ead7545878fdf15d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Oct 2024 15:08:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Oct 2024 15:08:22 -0400
If the query is rewritten into a NOTIFY command by a DO INSTEAD
rule, we'd get an assertion failure, or in non-assert builds
issue a rather confusing error message. Improve that.
Also fix a longstanding grammar mistake in a nearby error message.
Per bug #18664 from Alexander Lakhin. Back-patch to all supported
branches.
Tender Wang and Tom Lane
Discussion: https://postgr.es/m/18664-ffd0ebc2386598df@postgresql.org
M src/backend/commands/copyto.c
M src/test/regress/expected/copydml.out
M src/test/regress/sql/copydml.sql
Update outdated comment on WAL-logged locks with invalid XID
commit : 3c7d78427ea9d92d346ee9aada4fa483bd1059e8
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 21 Oct 2024 14:28:43 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 21 Oct 2024 14:28:43 +0300
We haven't generated those for a long time.
Discussion: https://www.postgresql.org/message-id/b439edfc-c5e5-43a9-802d-4cb51ec20646@iki.fi
M src/backend/storage/ipc/standby.c
Fix race condition in committing a serializable transaction
commit : 1a43de5e0a591b66ef3d92f3fac5aefcc2dc02fe
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 21 Oct 2024 09:49:21 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 21 Oct 2024 09:49:21 +0300
The finished transaction list can contain XIDs that are older than the
serializable global xmin. It's a short-lived state;
ClearOldPredicateLocks() removes any such transactions from the list,
and it's called whenever the global xmin advances. But if another
backend calls SummarizeOldestCommittedSxact() in that window, it will
call SerialAdd() on an XID that's older than the global xmin, or if
there are no more transactions running, when global xmin is
invalid. That trips the assertion in SerialAdd().
Fixes bug #18658 reported by Andrew Bille. Thanks to Alexander Lakhin
for analysis. Backpatch to all versions.
Discussion: https://www.postgresql.org/message-id/18658-7dab125ec688c70b%40postgresql.org
M src/backend/storage/lmgr/predicate.c
Fix grammar of a comment in bufmgr.c
commit : 57a36e890d3d60e6408bf6805e91c82f7b370873
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 21 Oct 2024 11:25:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 21 Oct 2024 11:25:29 +0900
Author: Junwang Zhao
Discussion: https://postgr.es/m/CAEG8a3L5YjxXCjx0LhkwHdDGsNgpFGEqH7SqtXRPNP+dwFMVZQ@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
injection_points: Add basic isolation test
commit : a7800cf498a36456bf4bd7a52e3b81cc225c5a65
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 21 Oct 2024 11:10:51 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 21 Oct 2024 11:10:51 +0900
This test can act as a template when implementing an isolation test with
injection points, and tracks in a much simpler way some of the behaviors
implied in the existing isolation test "inplace" that has been added in
c35f419d6efb. Particularly, a detach does not affect a backend wait; a
wait needs to be interrupted by a wakeup.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/ZxGTONm_ctQz--io@paquier.xyz
M src/test/modules/injection_points/Makefile
A src/test/modules/injection_points/expected/basic.out
M src/test/modules/injection_points/meson.build
A src/test/modules/injection_points/specs/basic.spec
Note that index_name in ALTER INDEX ATTACH PARTITION can be schema-qualified
commit : f1c141fe143cc362d6db918db5008c25e2df7a62
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sun, 20 Oct 2024 15:36:20 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Sun, 20 Oct 2024 15:36:20 +0200
Missed in 8b08f7d4820f; backpatch to all supported branches.
Reported-by: alvaro@datadoghq.com
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/172924785099.698.15236991344616673753@wrigleys.postgresql.org
M doc/src/sgml/ref/alter_index.sgml
SQL/JSON: Fix some oversights in commit b6e1157e7
commit : 11c87216d134a606938531e52edab6189bce6c2d
author : Amit Langote <amitlan@postgresql.org>
date : Sun, 20 Oct 2024 12:20:55 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Sun, 20 Oct 2024 12:20:55 +0900
The decision in b6e1157e7 to ignore raw_expr when evaluating a
JsonValueExpr was incorrect. While its value is not ultimately
used (since formatted_expr's value is), failing to initialize it
can lead to problems, for instance, when the expression tree in
raw_expr contains Aggref nodes, which must be initialized to
ensure the parent Agg node works correctly.
Also, optimize eval_const_expressions_mutator()'s handling of
JsonValueExpr a bit. Currently, when formatted_expr cannot be folded
into a constant, we end up processing it twice -- once directly in
eval_const_expressions_mutator() and again recursively via
ece_generic_processing(). This recursive processing is required to
handle raw_expr. To avoid the redundant processing of formatted_expr,
we now process raw_expr directly in eval_const_expressions_mutator().
Finally, update the comment of JsonValueExpr to describe the roles of
raw_expr and formatted_expr more clearly.
Bug: #18657
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Diagnosed-by: Fabio R. Sluzala <fabio3rs@gmail.com>
Diagnosed-by: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18657-1b90ccce2b16bdb8@postgresql.org
Backpatch-through: 16
M src/backend/executor/execExpr.c
M src/backend/optimizer/util/clauses.c
M src/include/nodes/primnodes.h
M src/test/regress/expected/sqljson.out
M src/test/regress/sql/sqljson.sql
Fix comment about pg_authid.
commit : 52475b4d30984e377542f964951bea57b0b26e92
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 19 Oct 2024 11:44:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 19 Oct 2024 11:44:14 -0400
pg_shadow is not "publicly readable". (pg_group is, but there seems
no need to make that distinction here.) Seems to be a thinko dating
clear back to 7762619e9.
Antonin Houska
Discussion: https://postgr.es/m/31926.1729252247@antos
M src/include/catalog/pg_authid.h
Disable autovacuum for tables in stats import tests.
commit : 779972e534c05c51be3e593b0681029afb1bd155
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 18 Oct 2024 10:57:46 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 18 Oct 2024 10:57:46 -0700
While we haven't observed any test instability, it seems like a good
idea to disable autovacuum during the stats import tests.
Author: Corey Huinker
Discussion: https://postgr.es/m/CADkLM=fajh1Lpcyr_XsMmq-9Z=SGk-u+_Zeac7Pt0RAN3uiVCg@mail.gmail.com
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Allow pg_set_relation_stats() to set relpages to -1.
commit : b391d882ff38da423dc99fc6224bde4be4100212
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 18 Oct 2024 10:44:15 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 18 Oct 2024 10:44:15 -0700
While the default value for relpages is 0, if a partitioned table with
at least one child has been analyzed, then the partititoned table will
have a relpages value of -1.
Author: Corey Huinker
Discussion: https://postgr.es/m/CADkLM=fajh1Lpcyr_XsMmq-9Z=SGk-u+_Zeac7Pt0RAN3uiVCg@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/statistics/relation_stats.c
M src/test/regress/expected/stats_import.out
M src/test/regress/sql/stats_import.sql
Optimize nbtree backwards scans.
commit : 1bd4bc85cac2b23484a6b568a752de0351c2cc5b
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 18 Oct 2024 11:25:32 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 18 Oct 2024 11:25:32 -0400
Make nbtree backwards scans optimistically access the next page to be
read to the left by following a prevPage block number that's now stashed
in currPos when the leaf page is first read. This approach matches the
one taken during forward scans, which follow a symmetric nextPage block
number from currPos. We stash both a prevPage and a nextPage, since the
scan direction might change (when fetching from a scrollable cursor).
Backwards scans will no longer need to lock the same page twice, except
in rare cases where the scan detects a concurrent page split (or page
deletion). Testing has shown this optimization to be particularly
effective during parallel index-only backwards scans: ~12% reductions in
query execution time are quite possible.
We're much better off being optimistic; concurrent left sibling page
splits are rare in general. It's possible that we'll need to lock more
pages than the pessimistic approach would have, but only when there are
_multiple_ concurrent splits of the left sibling page we now start at.
If there's just a single concurrent left sibling page split, the new
approach to scanning backwards will at least break even relative to the
old one (we'll acquire the same number of leaf page locks as before).
The optimization from this commit has long been contemplated by comments
added by commit 2ed5b87f96, which changed the rules for locking/pinning
during nbtree index scans. The approach that that commit introduced to
leaf level link traversal when scanning forwards is now more or less
applied all the time, regardless of the direction we're scanning in.
Following uniform conventions around sibling link traversal is simpler.
The only real remaining difference between our forward and backwards
handling is that our backwards handling must still detect and recover
from any concurrent left sibling splits (and concurrent page deletions),
as documented in the nbtree README. That is structured as a single,
isolated extra step that takes place in _bt_readnextpage.
Also use this opportunity to further simplify the functions that deal
with reading pages and traversing sibling links on the leaf level, and
to document their preconditions and postconditions (with respect to
things like buffer locks, buffer pins, and seizing the parallel scan).
This enhancement completely supersedes the one recently added by commit
3f44959f.
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAEze2WgpBGRgTTxTWVPXc9+PB6fc1a7t+VyGXHzfnrFXcQVxnA@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WzkBTuFv7W2+84jJT8mWZLXVL0GHq2hMUTn6c9Vw=eYrCw@mail.gmail.com
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtutils.c
M src/include/access/nbtree.h
Adjust documentation for configuring Linux huge pages.
commit : 9e2d813d59a90e832e04ede62b8d4b2197513902
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 18 Oct 2024 10:20:15 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 18 Oct 2024 10:20:15 -0500
The present wording about viewing shared_memory_size_in_huge_pages
seems to suggest that the parameter cannot be viewed after startup
at all, whereas the intent is to make it clear that you can't use
"postgres -C" to view this parameter while the server is running.
This commit rephrases this section to remove the ambiguity.
Author: Seino Yuki
Reviewed-by: Michael Paquier, David G. Johnston, Fujii Masao
Discussion: https://postgr.es/m/420584fd274f9ec4f337da55ffb3b790%40oss.nttdata.com
Backpatch-through: 15
M doc/src/sgml/runtime.sgml
Fix memory leaks from incorrect strsep() uses
commit : 4b652692e97923298d0c7b9c1c303fc648984d92
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 18 Oct 2024 11:28:54 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 18 Oct 2024 11:28:54 +0200
Commit 5d2e1cc117b introduced some strsep() uses, but it did the
memory management wrong in some cases. We need to keep a separate
pointer to the allocate memory so that we can free it later, because
strsep() advances the pointer we pass to it, and it at the end it
will be NULL, so any free() calls won't do anything.
(This fixes two of the four places changed in commit 5d2e1cc117b. The
other two don't have this problem.)
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/79692bf9-17d3-41e6-b9c9-fc8c3944222a@eisentraut.org
M src/common/logging.c
M src/test/regress/pg_regress.c
Fix strsep() use for SCRAM secrets parsing
commit : 24a36f91e32d6cbb8182c4cc6529d6e47587c988
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 18 Oct 2024 11:06:41 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 18 Oct 2024 11:06:41 +0200
The previous code (from commit 5d2e1cc117b) did not detect end of
string correctly, so it would fail to error out if fewer than the
expected number of fields were present, which could then later lead to
a crash when NULL string pointers are accessed.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Reported-by: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/79692bf9-17d3-41e6-b9c9-fc8c3944222a@eisentraut.org
M src/backend/libpq/auth-scram.c
Remove unused code for unlogged materialized views.
commit : 9272bdeac879ae44907422dd545f8d767b39c5ad
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 18 Oct 2024 17:18:57 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 18 Oct 2024 17:18:57 +0900
Commit 3bf3ab8c56 initially introduced support for unlogged
materialized views, but this was later disallowed by commit 3223b25ff7.
Additionally, commit d25f519107 added more code for handling
unlogged materialized views. This commit cleans up all unused
code related to them.
If unlogged materialized views had been supported in any official
release, psql would need to retain code to handle them for compatibility
with older servers. However, since they were never included in
an official release, this code is no longer necessary.
Author: Pixian Shi
Reviewed-by: Yugo Nagata, Fujii Masao
Discussion: https://postgr.es/m/CAAccyYKRZ=OvAvgowiSH+OELbStLP=p2Ht=R3CgT=OaNSH5DAA@mail.gmail.com
M src/backend/access/heap/heapam_handler.c
M src/bin/psql/describe.c
Fix description of PostgreSQL::Test::Cluster::wait_for_event()
commit : 19567b3eb4074c39b48f5557f268c3fb21dfa0f6
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 18 Oct 2024 13:49:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 18 Oct 2024 13:49:58 +0900
The arguments of the function were listed in an incorrect order in the
description of the routine. This information can be seen with perldoc.
Issue spotted while working on this area of the code.
Backpatch-through: 17
M src/test/perl/PostgreSQL/Test/Cluster.pm
Improve ThrowErrorData() comments for use with soft errors.
commit : eecd9138a0ef565366427a88866d0651530f7da4
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 17 Oct 2024 14:56:44 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 17 Oct 2024 14:56:44 -0700
Reviewed-by: Corey Huinker
Discussion: https://postgr.es/m/901ab7cf01957f92ea8b30b6feeb0eacfb7505fc.camel@j-davis.com
M src/backend/utils/error/elog.c
M src/include/nodes/miscnodes.h
ecpg: fix more minor mishandling of bad input in preprocessor.
commit : 1fed234f9faf1071d925434c5e9c14688fb4c77e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Oct 2024 15:28:32 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 17 Oct 2024 15:28:32 -0400
Don't get confused by an unmatched right brace in the input.
(Previously, this led to discarding information about file-level
variables and then possibly crashing.)
Detect, rather than crash on, an attempt to index into a non-array
variable.
As before, in the absence of field complaints I'm not too
excited about back-patching these.
Per valgrind testing by Alexander Lakhin.
Discussion: https://postgr.es/m/a239aec2-6c79-5fc9-9272-cea41158a360@gmail.com
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/variable.c
Fix extreme skew detection in Parallel Hash Join.
commit : 98c7c7152d2d4670a349c0fe48f05cf95c0b1185
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 17 Oct 2024 15:52:24 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 17 Oct 2024 15:52:24 +1300
After repartitioning the inner side of a hash join that would have
exceeded the allowed size, we check if all the tuples from a parent
partition moved to one child partition. That is evidence that it
contains duplicate keys and later attempts to repartition will also
fail, so we should give up trying to limit memory (for lack of a better
fallback strategy).
A thinko prevented the check from working correctly in partition 0 (the
one that is partially loaded into memory already). After
repartitioning, we should check for extreme skew if the *parent*
partition's space_exhausted flag was set, not the child partition's.
The consequence was repeated futile repartitioning until per-partition
data exceeded various limits including "ERROR: invalid DSA memory alloc
request size 1811939328", OS allocation failure, or temporary disk space
errors. (We could also do something about some of those symptoms, but
that's material for separate patches.)
This problem only became likely when PostgreSQL 16 introduced support
for Parallel Hash Right/Full Join, allowing NULL keys into the hash
table. Repartitioning always leaves NULL in partition 0, no matter how
many times you do it, because the hash value is all zero bits. That's
unlikely for other hashed values, but they might still have caused
wasted extra effort before giving up.
Back-patch to all supported releases.
Reported-by: Craig Milhiser <craig@milhiser.com>
Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>
Discussion: https://postgr.es/m/CA%2BwnhO1OfgXbmXgC4fv_uu%3DOxcDQuHvfoQ4k0DFeB0Qqd-X-rQ%40mail.gmail.com
M src/backend/executor/nodeHash.c
Remove superfluous forward declaration
commit : d893a299ce68f56522073a1b43d65764a552ae0d
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 17 Oct 2024 08:57:56 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 17 Oct 2024 08:57:56 +0200
The need for this was removed by commit dc9c3b0ff21.
M src/test/regress/pg_regress.h
Fix whitespace
commit : 6234a9ce0ef188d73caca1fe0f2ce79d2029009c
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 17 Oct 2024 08:42:58 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 17 Oct 2024 08:42:58 +0200
M src/test/regress/expected/merge.out
M src/test/regress/sql/merge.sql
Fix unnecessary casts of copyObject() result
commit : 665785d85f8b9a7250802d7fadd84841bb0aafa1
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 17 Oct 2024 08:36:14 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 17 Oct 2024 08:36:14 +0200
The result is already of the correct type, so these casts don't do
anything.
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/637eeea8-5663-460b-a114-39572c0f6c6e%40eisentraut.org
M src/backend/commands/trigger.c
M src/backend/nodes/nodeFuncs.c
M src/backend/rewrite/rewriteManip.c
Improve node type forward reference
commit : eafda78fc404c706da4cfa254dafb3e97f6cd111
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 17 Oct 2024 08:36:14 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 17 Oct 2024 08:36:14 +0200
Instead of using Node *, we can use an incomplete struct. That way,
everything has the correct type and fewer casts are required. This
technique is already used elsewhere in node type definitions.
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/637eeea8-5663-460b-a114-39572c0f6c6e%40eisentraut.org
M src/backend/commands/createas.c
M src/backend/parser/analyze.c
M src/include/nodes/primnodes.h
jsonapi: fully initialize dummy lexer
commit : 41b023946dfd20acbc4f3b14650a032eb46f5216
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 17 Oct 2024 08:23:46 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 17 Oct 2024 08:23:46 +0200
Valgrind reports that checks on lex->inc_state are undefined for the
"dummy lexer" used for incremental parsing, since it's only partially
initialized on the stack. This was introduced in 0785d1b8b2.
Zero-initialize the whole struct.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://www.postgresql.org/message-id/CAOYmi+n9QWr4gsAADZc6qFQjFViXQYVk=gBy_EvxuqsgPJcb_g@mail.gmail.com
M src/common/jsonapi.c
Fix unusual include style
commit : 342fb8a332007fb64fb1450e28194c36e9da3d6b
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 17 Oct 2024 08:14:45 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 17 Oct 2024 08:14:45 +0200
Project-internal header files should be included using " ", not < >.
M src/test/modules/test_rls_hooks/test_rls_hooks.h
Don't store intermediate hash values in ExprState->resvalue
commit : 9ca67658d19e6c258eb4021a326ed7d38b3ab75f
author : David Rowley <drowley@postgresql.org>
date : Thu, 17 Oct 2024 14:25:08 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 17 Oct 2024 14:25:08 +1300
adf97c156 made it so ExprStates could support hashing and changed Hash
Join to use that instead of manually extracting Datums from tuples and
hashing them one column at a time.
When hashing multiple columns or expressions, the code added in that
commit stored the intermediate hash value in the ExprState's resvalue
field. That was a mistake as steps may be injected into the ExprState
between each hashing step that look at or overwrite the stored
intermediate hash value. EEOP_PARAM_SET is an example of such a step.
Here we fix this by adding a new dedicated field for storing
intermediate hash values and adjust the code so that all apart from the
final hashing step store their result in the intermediate field.
In passing, rename a variable so that it's more aligned to the
surrounding code and also so a few lines stay within the 80 char margin.
Reported-by: Andres Freund
Reviewed-by: Alena Rybakina <a.rybakina@postgrespro.ru>
Discussion: https://postgr.es/m/CAApHDvqo9eenEFXND5zZ9JxO_k4eTA4jKMGxSyjdTrsmYvnmZw@mail.gmail.com
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/include/executor/execExpr.h
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Fix validation of COPY FORCE_NOT_NULL/FORCE_NULL for the all-column cases
commit : 089aac631b5ba53be0ecf8ea2e8d81388d69629c
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 17 Oct 2024 08:44:50 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 17 Oct 2024 08:44:50 +0900
This commit adds missing checks for COPY FORCE_NOT_NULL and FORCE_NULL
when applied to all columns via "*". These options now correctly
require CSV mode and are disallowed in COPY TO, making their behavior
consistent with FORCE_QUOTE.
Some regression tests are added to verify the correct behavior for the
all-columns case, including FORCE_QUOTE, which was not tested.
Backpatch down to 17, where support for the all-column grammar with
FORCE_NOT_NULL and FORCE_NULL has been added.
Author: Joel Jacobson
Reviewed-by: Zhang Mingli
Discussion: https://postgr.es/m/65030d1d-5f90-4fa4-92eb-f5f50389858e@app.fastmail.com
Backpatch-through: 17
M src/backend/commands/copy.c
M src/test/regress/expected/copy2.out
M src/test/regress/sql/copy2.sql
Rewrite some regression queries for option checks with COPY
commit : 03bf0d9a4ba0fb7a908b5a79eb80e0c983b51e87
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 17 Oct 2024 07:21:35 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 17 Oct 2024 07:21:35 +0900
Some queries in copy2 are there to check various option combinations,
and used "stdin" or "stdout" incompatible with the COPY TO or FROM
clauses combined with them, which was confusing. This commit rewrites
these queries to use a compatible grammar.
The coverage of the tests is unchanged. Like the original commit
451d1164b9d0, backpatch down to 16 where these have been introduced. A
follow-up commit will rely on this area of the tests for a bug fix.
Author: Joel Jacobson
Reviewed-by: Zhang Mingli
Discussion: https://postgr.es/m/65030d1d-5f90-4fa4-92eb-f5f50389858e@app.fastmail.com
Backpatch-through: 16
M src/test/regress/expected/copy2.out
M src/test/regress/sql/copy2.sql
nbtree: fix read page recheck typo.
commit : c0490b0ef773d8694d1a9bfbf4116a102c07458d
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 16 Oct 2024 17:38:38 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 16 Oct 2024 17:38:38 -0400
Oversight in commit 79fa7b3b.
M src/backend/access/nbtree/nbtsearch.c
Further refine _SPI_execute_plan's rule for atomic execution.
commit : c96de42c4b5f5d7bf785265cf26836326e28c6b6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Oct 2024 17:36:29 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Oct 2024 17:36:29 -0400
Commit 2dc1deaea turns out to have been still a brick shy of a load,
because CALL statements executing within a plpgsql exception block
could still pass the wrong snapshot to stable functions within the
CALL's argument list. That happened because standard_ProcessUtility
forces isAtomicContext to true if IsTransactionBlock is true, which
it always will be inside a subtransaction. Then ExecuteCallStmt
would think it does not need to push a new snapshot --- but
_SPI_execute_plan didn't do so either, since it thought it was in
nonatomic mode.
The best fix for this seems to be for _SPI_execute_plan to operate
in atomic execution mode if IsSubTransaction() is true, even when the
SPI context as a whole is non-atomic. This makes _SPI_execute_plan
have the same rules about when non-atomic execution is allowed as
_SPI_commit/_SPI_rollback have about when COMMIT/ROLLBACK are allowed,
which seems appropriately symmetric. (If anyone ever tries to allow
COMMIT/ROLLBACK inside a subtransaction, this would all need to be
rethought ... but I'm unconvinced that such a thing could be logically
consistent at all.)
For further consistency, also check IsSubTransaction() in
SPI_inside_nonatomic_context. That does not matter for its
one present-day caller StartTransaction, which can't be reached
inside a subtransaction. But if any other callers ever arise,
they'd presumably want this definition.
Per bug #18656 from Alexander Alehin. Back-patch to all
supported branches, like previous fixes in this area.
Discussion: https://postgr.es/m/18656-cade1780866ef66c@postgresql.org
M src/backend/executor/spi.c
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/sql/plpgsql_call.sql
Whitespace fixup from generated unicode tables.
commit : a7f2f6adc240a2823c2344b89e90bb630dea8803
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 16 Oct 2024 12:21:13 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 16 Oct 2024 12:21:13 -0700
When running the 'update-unicode' build target, generate files that
conform to pgindent whitespace rules.
M src/common/unicode/generate-unicode_case_table.pl
M src/common/unicode/generate-unicode_category_table.pl
Fix #include order from e839c8ecc9.
commit : b360d1762b55c3ff36e8b14c6a2286e5d4f61974
author : Jeff Davis <jdavis@postgresql.org>
date : Wed, 16 Oct 2024 12:09:16 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Wed, 16 Oct 2024 12:09:16 -0700
Reported-by: Alexander Korotkov
Discussion: https://postgr.es/m/CAPpHfduAiGSsvUc614Z-JOnyQffcMeJncWMF2HnUL8wFy4fuWA@mail.gmail.com
M src/backend/statistics/stat_utils.c
Reduce memory block size for decoded tuple storage to 8kB.
commit : 1b9b6cc3456be0f6ab929107293b31c333270bc1
author : Masahiko Sawada <msawada@postgresql.org>
date : Wed, 16 Oct 2024 12:08:05 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Wed, 16 Oct 2024 12:08:05 -0700
Commit a4ccc1cef introduced the Generation Context and modified the
logical decoding process to use a Generation Context with a fixed
block size of 8MB for storing tuple data decoded during logical
decoding (i.e., rb->tup_context). Several reports have indicated that
the logical decoding process can be terminated due to
out-of-memory (OOM) situations caused by excessive memory usage in
rb->tup_context.
This issue can occur when decoding a workload involving several
concurrent transactions, including a long-running transaction that
modifies tuples. By design, the Generation Context does not free a
memory block until all chunks within that block are
released. Consequently, if tuples modified by the long-running
transaction are stored across multiple memory blocks, these blocks
remain allocated until the long-running transaction completes, leading
to substantial memory fragmentation. The memory usage during logical
decoding, tracked by rb->size, does not account for memory
fragmentation, resulting in potentially much higher memory consumption
than the value of the logical_decoding_work_mem parameter.
Various improvement strategies were discussed in the relevant
thread. This change reduces the block size of the Generation Context
used in rb->tup_context from 8MB to 8kB. This modification
significantly decreases the likelihood of substantial memory
fragmentation occurring and is relatively straightforward to
backport. Performance testing across multiple platforms has confirmed
that this change will not introduce any performance degradation that
would impact actual operation.
Backport to all supported branches.
Reported-by: Alex Richman, Michael Guissine, Avi Weinberg
Reviewed-by: Amit Kapila, Fujii Masao, David Rowley
Tested-by: Hayato Kuroda, Shlok Kyal
Discussion: https://postgr.es/m/CAD21AoBTY1LATZUmvSXEssvq07qDZufV4AF-OHh9VD2pC0VY2A%40mail.gmail.com
Backpatch-through: 12
M src/backend/replication/logical/reorderbuffer.c
ecpg: fix some minor mishandling of bad input in preprocessor.
commit : 9b4bf5169064044ff082c61bf0783c4a65c08734
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Oct 2024 12:24:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Oct 2024 12:24:57 -0400
Avoid null-pointer crash when considering a cursor declaration
that's outside any C function (a case which is useless anyway).
Ensure a cursor for a prepared statement is marked as initially
not open. At worst, if we chanced to get not-already-zeroed memory
from malloc(), this oversight would result in failing to issue a
"cursor "foo" has been declared but not opened" warning that would
have been appropriate.
Avoid running off the end of the buffer when there are mismatched
square brackets following a variable name. This could lead to
SIGSEGV after reaching the end of memory.
Given the lack of field complaints, none of these seem to be worth
back-patching, but let's clean them up in HEAD.
Per valgrind testing by Alexander Lakhin.
Discussion: https://postgr.es/m/5f5bcecd-d7ec-b8c0-6c92-d1a7c6e0f639@gmail.com
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/variable.c
Normalize nbtree truncated high key array behavior.
commit : 79fa7b3b1a449680d9e51624834fbb4b32208659
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 16 Oct 2024 12:17:49 -0400
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 16 Oct 2024 12:17:49 -0400
Commit 5bf748b8 taught nbtree ScalarArrayOp index scans to decide when
and how to start the next primitive index scan based on physical index
characteristics. This included rules for deciding whether to start a
new primitive index scan (or whether to move onto the right sibling leaf
page instead) that specifically consider truncated lower-order columns
(-inf columns) from leaf page high keys.
These omitted columns were treated as satisfying the scan's required
scan keys, though only for scan keys marked required in the current scan
direction (forward). Scan keys that didn't get this behavior (those
marked required in the backwards direction only) usually didn't give the
scan reasonable cause to reposition itself to a later leaf page (via
another descent of the index in _bt_first), but _bt_advance_array_keys
would nevertheless always give up by forcing another call to _bt_first.
_bt_advance_array_keys was unwilling to allow the scan to continue onto
the next leaf page, to reconsider whether we really should start another
primitive scan based on the details of the sibling page's tuples. This
didn't match its behavior with similar cases involving keys required in
the current scan direction (forward), which seems unprincipled. It led
to an excessive number of primitive scans/index descents for queries
with a higher-order = array scan key (with dense, contiguous values)
mixed with a lower-order required > or >= scan key.
Bring > and >= strategy scan keys in line with other required scan key
types: treat truncated -inf scan keys as having satisfied scan keys
required in either scan direction (forwards and backwards alike) during
array advancement. That way affected scans can continue to the right
sibling leaf page. Advancement must now schedule an explicit recheck of
the right sibling page's high key in cases involving > or >= scan keys.
The recheck gives the scan a way to back out and start another primitive
index scan (we can't just rely on _bt_checkkeys with > or >= scan keys).
This work can be considered a stand alone optimization on top of the
work from commit 5bf748b8. But it was written in preparation for an
upcoming patch that will add skip scan to nbtree. In practice scans
that use "skip arrays" will tend to be much more sensitive to any
implementation deficiencies in this area.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CAH2-Wz=9A_UtM7HzUThSkQ+BcrQsQZuNhWOvQWK06PRkEp=SKQ@mail.gmail.com
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtutils.c
M src/include/access/nbtree.h
Fix typo in comment of transformJsonAggConstructor()
commit : c259b1578e006b2f808595fac4d8e45c5cce9d73
author : Amit Langote <amitlan@postgresql.org>
date : Wed, 16 Oct 2024 18:11:53 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Wed, 16 Oct 2024 18:11:53 +0900
An oversight of 3a8a1f3254b.
Reported-by: Tender Wang <tndrwang@gmail.com>
Author: Tender Wang <tndrwang@gmail.com>
Backpatch-through: 16
M src/backend/parser/parse_expr.c
initdb: Change default to using data checksums.
commit : 04bec894a04cb0d32533f1522ab81b7016141ff1
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 16 Oct 2024 08:45:09 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 16 Oct 2024 08:45:09 +0200
Checksums are now on by default. They can be disabled by the
previously added option --no-data-checksums.
Author: Greg Sabino Mullane <greg@turnstep.com>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://www.postgresql.org/message-id/flat/CAKAnmmKwiMHik5AHmBEdf5vqzbOBbcwEPHo4-PioWeAbzwcTOQ@mail.gmail.com
M doc/src/sgml/ref/initdb.sgml
M src/bin/initdb/initdb.c
M src/bin/initdb/t/001_initdb.pl
doc: Fix initdb option xreflabels
commit : 67846550dc6d7d90db4d98cdd8a8440725ef5c65
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 16 Oct 2024 08:28:12 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 16 Oct 2024 08:28:12 +0200
Generally, we don't want any overriding xreflabels in the options
list, so that we can link to options and the link renders as the
option name. The -g option did this differently and config.sgml made
use of that for a link. The new --no-data-checksums option (commit
983a588e0b8) apparently copied this pattern, but that seems like the
wrong direction, as a future patch revealed.
To fix, remove the two xreflabels and rewrite the link in config.sgml
with an explicit link text.
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/initdb.sgml
Add type cast to foreach_internal's loop variable.
commit : d5ca15ee54bf7faf038a38408e8adec971740966
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 15 Oct 2024 16:20:49 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 15 Oct 2024 16:20:49 -0500
C++ requires explicitly casting void pointers to the appropriate
pointer type, which means the foreach_ptr macro cannot be used in
C++ code without this change.
Author: Jelte Fennema-Nio
Reviewed-by: Bruce Momjian
Discussion: https://postgr.es/m/CAGECzQSYG3QfHrc-rOk2KbnB9iJOd7Qu-Xii1s-GTA%3D3JFt49Q%40mail.gmail.com
Backpatch-through: 17
M src/include/nodes/pg_list.h
Move clause_sides_match_join() into restrictinfo.h
commit : 2453196107de66cff0257feef2ff8585dcf9d924
author : David Rowley <drowley@postgresql.org>
date : Tue, 15 Oct 2024 21:14:21 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 15 Oct 2024 21:14:21 +1300
Two near-identical copies of clause_sides_match_join() existed in
joinpath.c and analyzejoins.c. Deduplicate this by moving the function
into restrictinfo.h.
It isn't quite clear that keeping the inline property of this function
is worthwhile, but this commit is just an exercise in code
deduplication. More effort would be required to determine if the inline
property is worth keeping.
Author: James Hunter <james.hunter.pg@gmail.com>
Discussion: https://postgr.es/m/CAJVSvF7Nm_9kgMLOch4c-5fbh3MYg%3D9BdnDx3Dv7Fcb64zr64Q%40mail.gmail.com
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/plan/analyzejoins.c
M src/include/optimizer/restrictinfo.h
Add contrib/pg_logicalinspect.
commit : 7cdfeee320e72162b62dddddee638e713c2b8680
author : Masahiko Sawada <msawada@postgresql.org>
date : Mon, 14 Oct 2024 17:22:02 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Mon, 14 Oct 2024 17:22:02 -0700
This module provides SQL functions that allow to inspect logical
decoding components.
It currently allows to inspect the contents of serialized logical
snapshots of a running database cluster, which is useful for debugging
or educational purposes.
Author: Bertrand Drouvot
Reviewed-by: Amit Kapila, Shveta Malik, Peter Smith, Peter Eisentraut
Reviewed-by: David G. Johnston
Discussion: https://postgr.es/m/ZscuZ92uGh3wm4tW%40ip-10-97-1-34.eu-west-3.compute.internal
M contrib/Makefile
M contrib/meson.build
A contrib/pg_logicalinspect/.gitignore
A contrib/pg_logicalinspect/Makefile
A contrib/pg_logicalinspect/expected/logical_inspect.out
A contrib/pg_logicalinspect/logicalinspect.conf
A contrib/pg_logicalinspect/meson.build
A contrib/pg_logicalinspect/pg_logicalinspect–1.0.sql
A contrib/pg_logicalinspect/pg_logicalinspect.c
A contrib/pg_logicalinspect/pg_logicalinspect.control
A contrib/pg_logicalinspect/specs/logical_inspect.spec
M doc/src/sgml/contrib.sgml
M doc/src/sgml/filelist.sgml
A doc/src/sgml/pglogicalinspect.sgml
M src/backend/replication/logical/snapbuild.c
M src/backend/utils/adt/arrayfuncs.c
M src/include/replication/snapbuild.h
M src/include/replication/snapbuild_internal.h
Move SnapBuild and SnapBuildOnDisk structs to snapshot_internal.h.
commit : e2fd615ecc177493b9a961a640ec0dcc4a25755c
author : Masahiko Sawada <msawada@postgresql.org>
date : Mon, 14 Oct 2024 17:19:33 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Mon, 14 Oct 2024 17:19:33 -0700
This commit moves the definitions of the SnapBuild and SnapBuildOnDisk
structs, related to logical snapshots, to the snapshot_internal.h
file. This change allows external tools, such as
pg_logicalinspect (with an upcoming patch), to access and utilize the
contents of logical snapshots.
Author: Bertrand Drouvot
Reviewed-by: Amit Kapila, Shveta Malik, Peter Smith
Discussion: https://postgr.es/m/ZscuZ92uGh3wm4tW%40ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/replication/logical/snapbuild.c
M src/include/replication/snapbuild.h
A src/include/replication/snapbuild_internal.h
ecpg: invent a saner syntax for ecpg.addons entries.
commit : dbedc461b4e7a9cb4d6f5777174bdf180edb95fd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 16:13:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 16:13:48 -0400
Put the rule type at the start not the end, and put spaces
between the constitutent token names instead of smashing them
into an illegible mess. This has no functional impact but
I think it makes the rules a great deal more readable.
Discussion: https://postgr.es/m/1185216.1724001216@sss.pgh.pa.us
M src/interfaces/ecpg/preproc/README.parser
M src/interfaces/ecpg/preproc/ecpg.addons
M src/interfaces/ecpg/preproc/parse.pl
Add commit 7f7474a8e4 to .git-blame-ignore-revs.
commit : 143e3a1ab8dfd4de507900e2e3a584db89155467
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 14 Oct 2024 15:09:39 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 14 Oct 2024 15:09:39 -0500
M .git-blame-ignore-revs
ecpg: add cross-checks to parse.pl for usage of internal tables.
commit : d2f41b4621de6d0e9eeaa4ea1d60cad190406f5e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 15:59:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 15:59:13 -0400
parse.pl contains several constant tables that describe tweaks
to be made to the backend grammar. In the same spirit as
00b0e7204, add cross-checks that each table entry is used at
least once (or exactly once if that's appropriate). This should
help catch cases where adjustments to the backend grammar cause
a table entry not to match as expected.
Per suggestion from Michael Paquier.
Discussion: https://postgr.es/m/ZsLVbjsc5x5Saesg@paquier.xyz
M src/interfaces/ecpg/preproc/parse.pl
Move libc-specific code from pg_locale.c into pg_locale_libc.c.
commit : 66ac94cdc79216e85f4c9e2d445c6f025653668e
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 14 Oct 2024 12:22:13 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 14 Oct 2024 12:22:13 -0700
Move implementation of pg_locale_t code for libc collations into
pg_locale_libc.c. Other locale-related code, such as
pg_perm_setlocale(), remains in pg_locale.c for now.
Discussion: https://postgr.es/m/flat/2830211e1b6e6a2e26d845780b03e125281ea17b.camel@j-davis.com
M src/backend/utils/adt/Makefile
M src/backend/utils/adt/meson.build
M src/backend/utils/adt/pg_locale.c
A src/backend/utils/adt/pg_locale_libc.c
ecpg: avoid breaking the IDENT precedence level in two.
commit : 9812138593f3e56735d93715a6dc2ed2d392611e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 15:41:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 15:41:42 -0400
Careless string hacking caused parse.pl to transform gram.y's
declaration
%nonassoc IDENT PARTITION RANGE ROWS ...
into
%nonassoc IDENT
%nonassoc CSTRING PARTITION RANGE ROWS ...
It turns out that this has no semantic impact, because the
generated preproc.c is exactly the same either way (if you
inject a blank line to keep line numbers the same).
Nonetheless, given the great emphasis that the commentary in
gram.y places on keeping those other keywords at the same
precedence level as IDENT, this seems like foolishly risking ecpg
behaving differently from the core parser. Adjust the code so
that CSTRING is added to the precedence line without breaking it
into two lines.
Discussion: https://postgr.es/m/2157151.1713540065@sss.pgh.pa.us
M src/interfaces/ecpg/preproc/parse.pl
Move ICU-specific code from pg_locale.c into pg_locale_icu.c.
commit : f244a2bb4c5520fc0271e638598abcbe7c9bc8cf
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 14 Oct 2024 12:13:20 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 14 Oct 2024 12:13:20 -0700
Discussion: https://postgr.es/m/flat/2830211e1b6e6a2e26d845780b03e125281ea17b.camel@j-davis.com
M src/backend/utils/adt/Makefile
M src/backend/utils/adt/meson.build
M src/backend/utils/adt/pg_locale.c
A src/backend/utils/adt/pg_locale_icu.c
ecpg: improve preprocessor's memory management.
commit : 1acd0f55274fab8c936779a0f2b738f6005cc48f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 13:55:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 13:55:08 -0400
Invent a notion of "local" storage that will automatically be
reclaimed at the end of each statement. Use this for location
strings as well as other visibly short-lived data within the parser.
Also, make cat_str and make_str return local storage and not free
their inputs, which allows dispensing with a whole lot of retail
mm_strdup calls. We do have to add some new ones in places where
a local-lifetime string needs to be added to a longer-lived data
structure, but on balance there are a lot less mm_strdup calls than
before.
In hopes of flushing out places where changes were necessary,
I changed YYLTYPE from "char *" to "const char *", which forced
const-ification of various function arguments that probably
should've been like that all along.
This still leaks somewhat more memory than v17, but that will be
cleaned up in future commits.
Discussion: https://postgr.es/m/2011420.1713493114@sss.pgh.pa.us
M src/interfaces/ecpg/preproc/descriptor.c
M src/interfaces/ecpg/preproc/ecpg.addons
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/output.c
M src/interfaces/ecpg/preproc/parser.c
M src/interfaces/ecpg/preproc/preproc_extern.h
M src/interfaces/ecpg/preproc/type.c
M src/interfaces/ecpg/preproc/type.h
M src/interfaces/ecpg/preproc/util.c
M src/interfaces/ecpg/preproc/variable.c
M src/tools/pgindent/typedefs.list
ecpg: move some functions into a new file ecpg/preproc/util.c.
commit : f18231e817599246fc99a798c9bf57ab785db91f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 13:47:59 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 13:47:59 -0400
mm_alloc and mm_strdup were in type.c, which seems a completely
random choice. No doubt the original author thought two small
functions didn't deserve their own file. But I'm about to add
some more memory-management stuff beside them, so let's put them
in a less surprising place. This seems like a better home for
mmerror, mmfatal, and the cat_str/make_str family, too.
Discussion: https://postgr.es/m/2011420.1713493114@sss.pgh.pa.us
M src/interfaces/ecpg/preproc/Makefile
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/preproc/meson.build
M src/interfaces/ecpg/preproc/preproc_extern.h
M src/interfaces/ecpg/preproc/type.c
A src/interfaces/ecpg/preproc/util.c
ecpg: re-implement preprocessor's string management.
commit : a542d5614bdb6430094556162b9ca2f01d35f9dc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 13:44:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 13:44:42 -0400
Most productions in the preprocessor grammar construct strings
representing SQL or C statements or fragments thereof. Instead
of returning these as <str> results of the productions, return
them as "location" values, taking advantage of Bison's flexibility
about what a location is. We aren't really giving up anything
thereby, since ecpg's error reports have always just given line
numbers, and that's tracked separately. The advantage of this
is that a single instance of the YYLLOC_DEFAULT macro can
perform all the work needed by the vast majority of productions,
including all the ones made automatically by parse.pl. This
avoids having large numbers of effectively-identical productions,
which tickles an optimization inefficiency in recent versions of
clang. (This patch reduces the compilation time for preproc.o
by more than 100-fold with clang 16, and is visibly helpful with
gcc too.) The compiled parser is noticeably smaller as well.
A disadvantage of this approach is that YYLLOC_DEFAULT is applied
before running the production's semantic action (if any). This
means it cannot use the method favored by cat_str() of free'ing
all the input strings; if the action needs to look at the input
strings, it'd be looking at dangling storage. As this stands,
therefore, it leaks memory like a sieve. This is already a big
patch though, and fixing the memory management seems like a
separable problem, so let's leave that for the next step.
(This does remove some free() calls that I'd have had to touch
anyway, in the expectation that the next step will manage
memory reclamation quite differently.)
Most of the changes here are mindless substitution of "@N" for
"$N" in grammar rules; see the changes to README.parser for
an explanation.
Discussion: https://postgr.es/m/2011420.1713493114@sss.pgh.pa.us
M src/interfaces/ecpg/preproc/README.parser
M src/interfaces/ecpg/preproc/ecpg.addons
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/ecpg.type
M src/interfaces/ecpg/preproc/output.c
M src/interfaces/ecpg/preproc/parse.pl
M src/interfaces/ecpg/preproc/parser.c
M src/interfaces/ecpg/preproc/preproc_extern.h
ecpg: major cleanup, simplification, and documentation of parse.pl.
commit : 6b005499447512abfa1e4add87dcf3e2859c4f9e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 13:37:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 13:37:33 -0400
Remove a lot of cruft, clean up and document what's left.
This produces the same preproc.y output as before, except for
fewer blank lines. (It's not like we're making any attempt to
match the layout of gram.y, so I removed the one bit of logic
that seemed to have that in mind.)
Discussion: https://postgr.es/m/2011420.1713493114@sss.pgh.pa.us
M src/interfaces/ecpg/preproc/parse.pl
ecpg: remove check_rules.pl.
commit : 293fd24425b59b9ac3386448c682c1f6c81fbbc8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 13:33:41 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 13:33:41 -0400
As noted in the previous commit, check_rules.pl is now entirely
redundant with checks made by parse.pl, or would be if it weren't
for the places where it's wrong. It's a waste of build cycles
and maintenance effort, so remove it.
Discussion: https://postgr.es/m/2011420.1713493114@sss.pgh.pa.us
M src/interfaces/ecpg/preproc/Makefile
D src/interfaces/ecpg/preproc/check_rules.pl
M src/interfaces/ecpg/preproc/meson.build
ecpg: clean up documentation of parse.pl, and add more input checking.
commit : 00b0e7204d530d878b188254b62008651030921a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 13:29:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Oct 2024 13:29:36 -0400
README.parser is the user's manual, such as it is, for parse.pl.
It's rather poorly written if you ask me; so try to improve it.
(More could be written here, but this at least covers the same
info in a more organized fashion.)
Also, the single solitary line of usage info in parse.pl itself
was a lie. Replace.
Add some error checks that the ecpg.addons entries meet the syntax
rules set forth in README.parser. One of them didn't, but
accidentally worked anyway because the logic in include_addon is
such that 'block' is the default behavior.
Also add a cross-check that each ecpg.addons entry is matched exactly
once in the backend grammar. This exposed that there are two dead
entries there --- they are dead because the %replace_types table in
parse.pl causes their nonterminals to be ignored altogether.
Removing them doesn't change the generated preproc.y file.
(This implies that check_rules.pl is completely worthless and should
be nuked: it adds build cycles and maintenance effort while failing
to reliably accomplish its one job of detecting dead rules. I'll
do that separately.)
Discussion: https://postgr.es/m/2011420.1713493114@sss.pgh.pa.us
M src/interfaces/ecpg/preproc/README.parser
M src/interfaces/ecpg/preproc/ecpg.addons
M src/interfaces/ecpg/preproc/parse.pl
Remove obsolete comment in reorderbuffer.h.
commit : 7be4ba4a9d560bc6dfb19749ad58cfc1a04d3a7c
author : Masahiko Sawada <msawada@postgresql.org>
date : Mon, 14 Oct 2024 09:53:05 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Mon, 14 Oct 2024 09:53:05 -0700
Commit 9fab40ad32e changed ReorderBuffer to use Slab Context for
allocating ReorderBufferTXN entries instead of using a caching
mechanism. The txn->node is no longer used as an element of the list
of preallocated ReorderBufferTXNs.
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAD21AoB1CTnX66Ji3zTCnjoPVC9OzYe0B6LygUHcxEB2RV-hFw%40mail.gmail.com
M src/include/replication/reorderbuffer.h
Use construct_array_builtin for FLOAT8OID instead of construct_array.
commit : 4681ad4b2fb06e3aca9e300358e62d9b066e9dce
author : Masahiko Sawada <msawada@postgresql.org>
date : Mon, 14 Oct 2024 09:49:29 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Mon, 14 Oct 2024 09:49:29 -0700
Commit d746021de1 introduced construct_array_builtin() for built-in
data types, but forgot some replacements linked to FLOAT8OID.
Author: Bertrand Drouvot
Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/CAD21AoCERkwmttY44dqUw%3Dm_9QCctu7W%2Bp6B7w_VqxRJA1Qq_Q%40mail.gmail.com
M src/backend/utils/adt/arrayfuncs.c
M src/backend/utils/adt/float.c
Track scan reversals in MergeJoin
commit : c594f1ad2ba7cbe66e8ae98da4ffed06491d1082
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Oct 2024 15:36:18 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Oct 2024 15:36:18 +0200
The MergeJoin struct was tracking "mergeStrategies", which were an
array of btree strategy numbers, purely for the purpose of comparing
it later against btree strategies to determine if the scan direction
was forward or reverse. Change that. Instead, track
"mergeReversals", an array of bool, to indicate the same without an
unfortunate assumption that a strategy number refers specifically to a
btree strategy.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/executor/nodeMergejoin.c
M src/backend/optimizer/plan/createplan.c
M src/include/nodes/plannodes.h
Track sort direction in SortGroupClause
commit : 0d2aa4d4937bb0500823edfc7d620f4e5fa45b9c
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Oct 2024 15:36:02 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Oct 2024 15:36:02 +0200
Functions make_pathkey_from_sortop() and transformWindowDefinitions(),
which receive a SortGroupClause, were determining the sort order
(ascending vs. descending) by comparing that structure's operator
strategy to BTLessStrategyNumber, but could just as easily have gotten
it from the SortGroupClause object, if it had such a field, so add
one. This reduces the number of places that hardcode the assumption
that the strategy refers specifically to a btree strategy, rather than
some other index AM's operators.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/E72EAA49-354D-4C2E-8EB9-255197F55330@enterprisedb.com
M src/backend/optimizer/path/pathkeys.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/planagg.c
M src/backend/parser/analyze.c
M src/backend/parser/parse_clause.c
M src/backend/utils/cache/lsyscache.c
M src/include/catalog/catversion.h
M src/include/nodes/parsenodes.h
Allow TAP tests to force checksums off when calling init()
commit : e7d0cf42b1acb185edc947a8732843966ea3c160
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Oct 2024 10:57:54 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Oct 2024 10:57:54 +0200
TAP tests can write
$node->init(no_data_checksums => 1);
to initialize a cluster explicitly without checksums. Currently, this
is the default, but this change allows running all tests with
checksums enabled, like
PG_TEST_INITDB_EXTRA_OPTS=--data-checksums meson test ...
And this also prepares the tests for when we switch the default to
checksums enabled.
The pg_checksums tests need to disable checksums so it can test its
own functionality of enabling checksums. The amcheck/pg_amcheck tests
need to disable checksums because they manually introduce corruption
that they want to detect, but with checksums enabled, the checksum
verification will fail before they even get to their work.
Author: Greg Sabino Mullane <greg@turnstep.com>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/CAKAnmmKwiMHik5AHmBEdf5vqzbOBbcwEPHo4-PioWeAbzwcTOQ@mail.gmail.com
M contrib/amcheck/t/001_verify_heapam.pl
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/bin/pg_checksums/t/002_actions.pl
M src/test/perl/PostgreSQL/Test/Cluster.pm
Run pgperltidy on newly-added test code
commit : 199ad00e4b59a04263fd79fb115aecce2fb0851b
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Oct 2024 11:07:04 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Oct 2024 11:07:04 +0200
From commit 85ec945b78 (but apparently not caught by 05d1b9b5c2).
M src/test/perl/PostgreSQL/Test/Cluster.pm
doc: Add anchors for COPY format descriptions
commit : 40f4f2fa659408fd4d52f2de46c9d17d4f3b7e1e
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 14 Oct 2024 10:15:33 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 14 Oct 2024 10:15:33 +0200
When answering support questions online it's helpful to be able to
refer to the specific format by using an anchored link.
Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://postgr.es/m/87edatit3t.fsf@wibble.ilmari.org
M doc/src/sgml/ref/copy.sgml
Remove traces of BeOS.
commit : a2d9a9b95ad933853dd18894393615d37257f43b
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Oct 2024 08:02:38 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Oct 2024 08:02:38 +0200
Commit 15abc7788e6 tolerated namespace pollution from BeOS system
headers. Commit 44f902122 de-supported BeOS. Since that stuff didn't
make it into the Meson build system, synchronize by removing from
configure.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Japin Li <japinli@hotmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (the idea, not the patch)
Discussion: https://postgr.es/m/ME3P282MB3166F9D1F71F787929C0C7E7B6312%40ME3P282MB3166.AUSP282.PROD.OUTLOOK.COM
M configure
M configure.ac
M src/include/c.h
M src/include/pg_config.h.in
M src/interfaces/ecpg/include/ecpg_config.h.in
M src/interfaces/ecpg/include/meson.build
M src/interfaces/ecpg/include/pgtypes_interval.h
psql: Fix \watch when using interval values less than 1ms
commit : 9f34cae1426fe5ab915c95b7e26b1efffbb2ca0f
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 14 Oct 2024 12:27:51 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 14 Oct 2024 12:27:51 +0900
Attempting to use an interval of time less than 1ms would cause \watch
to hang. This was confusing, so let's change the logic so as an
interval lower than 1ms behaves the same as 0.
Comments are added to mention that the internals of do_watch() had
better rely on "sleep_ms", the interval value in milliseconds. While on
it, this commit adds a test to check the behavior of interval values
less than 1ms.
\watch hanging for interval values less than 1ms existed before
6f9ee74d45aa, that has changed the code to support an interval value of
0.
Reported-by: Heikki Linnakangas
Author: Andrey M. Borodin, Michael Paquier
Discussion: https://postgr.es/m/88445e0e-3156-4b9d-afae-9a1a7b1631f6@iki.fi
Backpatch-through: 16
M src/bin/psql/command.c
M src/bin/psql/t/001_basic.pl
Fixup for pg_set_relation_stats().
commit : 35a015a60045c2daa532c7a9545818856f09d57a
author : Jeff Davis <jdavis@postgresql.org>
date : Sun, 13 Oct 2024 13:44:23 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Sun, 13 Oct 2024 13:44:23 -0700
Reported-by: Noriyoshi Shinoda
Discussion: https://postgr.es/m/DM4PR84MB17345E2DFF28A5557B7CBC3CEE7A2@DM4PR84MB1734.NAMPRD84.PROD.OUTLOOK.COM
M src/backend/statistics/relation_stats.c
Use MAX_PARALLEL_WORKER_LIMIT for max_parallel_maintenance_workers
commit : c0b74323dc7b47644f074cf98597723ca55b2309
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 13 Oct 2024 11:20:30 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 13 Oct 2024 11:20:30 +0900
max_parallel_maintenance_workers has been introduced in 9da0cc35284b,
and used a hardcoded limit of 1024 rather than this variable.
max_parallel_workers and max_parallel_workers_per_gather already used
MAX_PARALLEL_WORKER_LIMIT (1024) as their upper-bound since
6599c9ac3340.
Author: Matthias van de Meent
Reviewed-by: Zhang Mingli
Discussion: https://postgr.es/m/CAEze2WiCiJD+8Wig_wGPyn4vgdPjbnYXy2Rw+9KYi6izTMuP=w@mail.gmail.com
M src/backend/utils/misc/guc_tables.c
Correctly identify which EC members are computable at a plan node.
commit : 9f954177b1eb0f890a8e1f4ebaf90bea54c4871b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Oct 2024 14:56:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Oct 2024 14:56:08 -0400
find_computable_ec_member() had the wrong mental model of what
its primary caller prepare_sort_from_pathkeys() would do with
the selected EquivalenceClass member expression. We will not
compute the EC expression in a plan node atop the one returning
the passed-in targetlist; rather, the EC expression will be
computed as an additional column of that targetlist. So any
Var or quasi-Var used in the given tlist is also available to the
EC expression. In simple cases this makes no difference because
the given tlist is just a list of Vars or quasi-Vars --- but if
we are considering an appendrel member produced by flattening
a UNION ALL, the tlist may contain expressions, resulting in
failure to match and a "could not find pathkey item to sort"
error.
To fix, we can flatten both the tlist and the EC members with
pull_var_clause(), and then just check for subset-ness, so
that the code is actually shorter than before.
While this bug is quite old, the present patch only works back to
v13. We could possibly make it work in v12 by back-patching parts
of 375398244. On the whole though I don't like the risk/reward
ratio of that idea. v12's final release is next month, meaning
there would be no chance to correct matters if the patch causes a
regression. Since this failure has escaped notice for 14 years,
it's likely nobody will hit it in the field with v12.
Per bug #18652 from Alexander Lakhin.
Andrei Lepikhov and Tom Lane
Discussion: https://postgr.es/m/18652-deaa782ebcca85d1@postgresql.org
M src/backend/optimizer/path/equivclass.c
M src/test/regress/expected/inherit.out
M src/test/regress/sql/inherit.sql
Fix missed case for builtin collation provider.
commit : 98c5b191e74b65b74149bb436b057d7007d401a7
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 11 Oct 2024 16:57:48 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 11 Oct 2024 16:57:48 -0700
A missed check for the builtin collation provider could result in
falling through to call isalpha().
This does not appear to have practical consequences because it only
happens for characters in the ASCII range. Regardless, the builtin
provider should not be calling libc functions, so backpatch.
Discussion: https://postgr.es/m/1bd5a0a5192f82c22ee7527e825b18ab0028b2c7.camel@j-davis.com
Backpatch-through: 17
M src/backend/utils/adt/like_support.c
Create functions pg_set_relation_stats, pg_clear_relation_stats.
commit : e839c8ecc9352b7754e74f19ace013c0c0d18613
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 11 Oct 2024 16:55:11 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 11 Oct 2024 16:55:11 -0700
These functions are used to tweak statistics on any relation, provided
that the user has MAINTAIN privilege on the relation, or is the database
owner.
Bump catalog version.
Author: Corey Huinker
Discussion: https://postgr.es/m/CADkLM=eErgzn7ECDpwFcptJKOk9SxZEk5Pot4d94eVTZsvj3gw@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/catalog/system_functions.sql
M src/backend/statistics/Makefile
M src/backend/statistics/meson.build
A src/backend/statistics/relation_stats.c
A src/backend/statistics/stat_utils.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
A src/include/statistics/stat_utils.h
A src/test/regress/expected/stats_import.out
M src/test/regress/parallel_schedule
A src/test/regress/sql/stats_import.sql
Avoid mixing custom and OpenSSL BIO functions
commit : 6f782a2a1738ab96ee948a4ab33ca3defd39327b
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 11 Oct 2024 21:58:58 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 11 Oct 2024 21:58:58 +0200
PostgreSQL has for a long time mixed two BIO implementations, which can
lead to subtle bugs and inconsistencies. This cleans up our BIO by just
just setting up the methods we need. This patch does not introduce any
functionality changes.
The following methods are no longer defined due to not being needed:
- gets: Not used by libssl
- puts: Not used by libssl
- create: Sets up state not used by libpq
- destroy: Not used since libpq use BIO_NOCLOSE, if it was used it close
the socket from underneath libpq
- callback_ctrl: Not implemented by sockets
The following methods are defined for our BIO:
- read: Used for reading arbitrary length data from the BIO. No change
in functionality from the previous implementation.
- write: Used for writing arbitrary length data to the BIO. No change
in functionality from the previous implementation.
- ctrl: Used for processing ctrl messages in the BIO (similar to ioctl).
The only ctrl message which matters is BIO_CTRL_FLUSH used for
writing out buffered data (or signal EOF and that no more data
will be written). BIO_CTRL_FLUSH is mandatory to implement and
is implemented as a no-op since there is no intermediate buffer
to flush.
BIO_CTRL_EOF is the out-of-band method for signalling EOF to
read_ex based BIO's. Our BIO is not read_ex based but someone
could accidentally call BIO_CTRL_EOF on us so implement mainly
for completeness sake.
As the implementation is no longer related to BIO_s_socket or calling
SSL_set_fd, methods have been renamed to reference the PGconn and Port
types instead.
This also reverts back to using BIO_set_data, with our fallback, as a small
optimization as BIO_set_app_data require the ex_data mechanism in OpenSSL.
Author: David Benjamin <davidben@google.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAF8qwaCZ97AZWXtg_y359SpOHe+HdJ+p0poLCpJYSUxL-8Eo8A@mail.gmail.com
M src/backend/libpq/be-secure-openssl.c
M src/include/libpq/libpq-be.h
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/libpq-int.h
Add pg_ls_summariesdir().
commit : 4e1fad37872e49a711adad5d9870516e5c71a375
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 11 Oct 2024 11:02:09 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 11 Oct 2024 11:02:09 -0500
This function returns the name, size, and last modification time of
each regular file in pg_wal/summaries. This allows administrators
to grant privileges to view the contents of this directory without
granting privileges on pg_ls_dir(), which allows listing the
contents of many other directories. This commit also gives the
pg_monitor predefined role EXECUTE privileges on the new
pg_ls_summariesdir() function.
Bumps catversion.
Author: Yushi Ogiwara
Reviewed-by: Michael Paquier, Fujii Masao
Discussion: https://postgr.es/m/a0a3af15a9b9daa107739eb45aa9a9bc%40oss.nttdata.com
M doc/src/sgml/func.sgml
M src/backend/catalog/system_functions.sql
M src/backend/utils/adt/genfile.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/misc_functions.out
M src/test/regress/sql/misc_functions.sql
Mark consume_xids test functions VOLATILE and PARALLEL UNSAFE
commit : add77755ce851fae2f72be4b9c56a606f9362895
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 11 Oct 2024 11:09:09 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 11 Oct 2024 11:09:09 +0300
Both functions advance the transaction ID, which modifies the system
state. Thus, they should be marked as VOLATILE.
Additionally, they call the AssignTransactionId function, which cannot
be invoked in parallel mode, so they should be marked as PARALLEL
UNSAFE.
Author: Yushi Ogiwara <btogiwarayuushi@oss.nttdata.com>
Discussion: https://www.postgresql.org/message-id/18f01e4fd46448f88c7a1363050a9955@oss.nttdata.com
M src/test/modules/xid_wraparound/xid_wraparound–1.0.sql
Fix typo in connection limits test
commit : 682512dca870f46cf2c0b9e227702dffc5de5ef0
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 11 Oct 2024 10:04:23 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 11 Oct 2024 10:04:23 +0200
Spotted while doing post-commit review.
M src/test/postmaster/t/001_connection_limits.pl
Use deconstruct_array_builtin instead of deconstruct_array
commit : 099c572d3384f1cbac6f6a108f56de222c10c11e
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Oct 2024 09:54:18 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Oct 2024 09:54:18 +0200
Commit 062a84442424 introduced use of deconstruct_array when
deconstruct_array_builtin can be used instead. Do that to save some
code.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/Zwi5g2GzlUX1NqxR@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/catalog/pg_publication.c
pgbench: Improve result outputs related to failed transactions.
commit : cae0f3c40554e85cd004732f3f57b9cb936136d2
author : Tatsuo Ishii <ishii@postgresql.org>
date : Fri, 11 Oct 2024 13:40:23 +0900
committer: Tatsuo Ishii <ishii@postgresql.org>
date : Fri, 11 Oct 2024 13:40:23 +0900
Previously, per-script statistics were never output when all
transactions failed due to serialization or deadlock errors. However,
it is reasonable to report such information if there are ones even
when there are no successful transaction since these failed
transactions are now objects to be reported.
Meanwhile, if the total number of successful, skipped, and failed
transactions is zero, we don't have to report the number of failed
transactions as similar to the number of skipped transactions, which
avoids to print "NaN%" in lines on failed transaction reports.
Also, the number of transactions in per-script results now includes
skipped and failed transactions. It prevents to print "total of NaN%"
when any transactions are not successfully processed. The number of
transactions actually processed per-script and TPS based on it are now
output explicitly in a separate line.
Author: Yugo Nagata
Reviewed-by: Tatsuo Ishii
Discussion: https://postgr.es/m/20240921003544.2436ef8da9c5c8cb963c651b%40sraoss.co.jp
M src/bin/pgbench/pgbench.c
Adjust EXPLAIN's output for disabled nodes
commit : 161320b4b960ee4fe918959be6529ae9b106ea5a
author : David Rowley <drowley@postgresql.org>
date : Fri, 11 Oct 2024 17:19:59 +1300
committer: David Rowley <drowley@postgresql.org>
date : Fri, 11 Oct 2024 17:19:59 +1300
c01743aa4 added EXPLAIN output to display the plan node's disabled_node
count whenever that count is above 0. Seemingly, there weren't many
people who liked that output as each parent of a disabled node would
also have a "Disabled Nodes" output due to the way disabled_nodes is
accumulated towards the root plan node. It was often hard and sometimes
impossible to figure out which nodes were disabled from looking at
EXPLAIN. You might think it would be possible to manually add up the
numbers from the "Disabled Nodes" output of a given node's children to
figure out if that node has a higher disabled_nodes count than its
children, but that wouldn't have worked for Append and Merge Append nodes
if some disabled child nodes were run-time pruned during init plan. Those
children are not displayed in EXPLAIN.
Here we attempt to improve this output by only showing "Disabled: true"
against only the nodes which are explicitly disabled themselves. That
seems to be the output that's desired by the most people who voiced
their opinion. This is done by summing up the disabled_nodes of the
given node's children and checking if that number is less than the
disabled_nodes of the current node.
This commit also fixes a bug in make_sort() which was neglecting to set
the Sort's disabled_nodes field. This should have copied what was done
in cost_sort(), but it hadn't been updated. With the new output, the
choice to not maintain that field properly was clearly wrong as the
disabled-ness of the node was attributed to the Sort's parent instead.
Reviewed-by: Laurenz Albe, Alena Rybakina
Discussion: https://postgr.es/m/9e4ad616bebb103ec2084bf6f724cfc739e7fabb.camel@cybertec.at
M src/backend/commands/explain.c
M src/backend/optimizer/plan/createplan.c
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/btree_index.out
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/expected/explain.out
M src/test/regress/expected/incremental_sort.out
M src/test/regress/expected/inherit.out
M src/test/regress/expected/insert_conflict.out
M src/test/regress/expected/join.out
M src/test/regress/expected/memoize.out
M src/test/regress/expected/select_parallel.out
M src/test/regress/expected/sqljson_jsontable.out
M src/test/regress/expected/union.out
M src/test/regress/expected/xml.out
M src/test/regress/expected/xml_1.out
M src/test/regress/expected/xml_2.out
Don't hard-code the input file name in gen_tabcomplete.pl's output.
commit : c75c6f8d280a3364643fc51f9c9fdf2dee0213c2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Oct 2024 17:02:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Oct 2024 17:02:08 -0400
Use $ARGV[0], that is the specified input file name, in #line
directives generated by gen_tabcomplete.pl. This makes code
coverage reports work properly in the meson build system (where
the input file name will be a relative path).
Also fix up brain fade in the meson build rule for tab-complete.c:
we only need to write the input file name once not twice.
Jacob Champion (some cosmetic adjustments by me)
Discussion: https://postgr.es/m/CAOYmi+=+oWAoi8pqnH0MJQqsSn4ddzqDhqRQJvyiN2aJSWvw2w@mail.gmail.com
M src/bin/psql/gen_tabcomplete.pl
M src/bin/psql/meson.build
Avoid possible segfault in psql's tab completion.
commit : 95eb4cd4ff0648afc32abbfaacc9718f6b91bf7b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Oct 2024 16:17:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Oct 2024 16:17:38 -0400
Fix oversight in bd1276a3c: the "words_after_create" stanza in
psql_completion() requires previous_words_count > 0, since it uses
prev_wd. This condition was formerly assured by the if-else chain
above it, but no more. If there were no previous words then we'd
dereference an uninitialized pointer, possibly causing a segfault.
Report and patch by Anthonin Bonnefoy.
Discussion: https://postgr.es/m/CAO6_XqrSRE7c_i+D7Hm07K3+6S0jTAmMr60RY41XzaA29Ae5uA@mail.gmail.com
M src/bin/psql/tab-complete.in.c
Unbreak overflow test for attinhcount/coninhcount
commit : fd64ed60b62697984bb69a09a3ae19fbe2905eb6
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 10 Oct 2024 17:41:01 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 10 Oct 2024 17:41:01 +0200
Commit 90189eefc1e1 narrowed pg_attribute.attinhcount and
pg_constraint.coninhcount from 32 to 16 bits, but kept other related
structs with 32-bit wide fields: ColumnDef and CookedConstraint contain
an int 'inhcount' field which is itself checked for overflow on
increments, but there's no check that the values aren't above INT16_MAX
before assigning to the catalog columns. This means that a creative
user can get a inconsistent table definition and override some
protections.
Fix it by changing those other structs to also use int16.
Also, modernize style by using pg_add_s16_overflow for overflow testing
instead of checking for negative values.
We also have Constraint.inhcount, which is here removed completely.
This was added by commit b0e96f311985 and not removed by its revert at
6f8bb7c1e961. It is not needed by the upcoming not-null constraints
patch.
This is mostly academic, so we agreed not to backpatch to avoid ABI
problems.
Bump catversion because of the changes to parse nodes.
Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Co-authored-by: 何建 (jian he) <jian.universality@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/202410081611.up4iyofb5ie7@alvherre.pgsql
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/pg_constraint.c
M src/backend/commands/tablecmds.c
M src/include/catalog/catversion.h
M src/include/catalog/heap.h
M src/include/catalog/pg_constraint.h
M src/include/nodes/parsenodes.h
Improve descriptions of some pg_stat_checkpoints functions in pg_proc.dat.
commit : 1909835c28a144bf957254eea1f4a4dabafbbaee
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 11 Oct 2024 00:12:29 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 11 Oct 2024 00:12:29 +0900
Previously, the descriptions of pg_stat_get_checkpointer_num_requested(),
pg_stat_get_checkpointer_restartpoints_requested(),
and pg_stat_get_checkpointer_restartpoints_performed() in pg_proc.dat
referred to "backend". This was misleading because these functions report
the number of checkpoints or restartpoints requested or performed
by other than backends as well.
This commit removes "backend" from these descriptions to avoid confusion.
Bump catalog version.
Idea from Anton A. Melnikov
Author: Fujii Masao
Reviewed-by: Anton A. Melnikov
Discussion: https://postgr.es/m/8e5f353f-8b31-4a8e-9cfa-c037f22b4aee@postgrespro.ru
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Avoid crash in estimate_array_length with null root pointer.
commit : 5a4416192d220410779a4ef6d433e067ba7a7043
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Oct 2024 17:07:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Oct 2024 17:07:53 -0400
Commit 9391f7152 added a "PlannerInfo *root" parameter to
estimate_array_length, but failed to consider the possibility that
NULL would be passed for that, leading to a null pointer dereference.
We could rectify the particular case shown in the bug report by fixing
simplify_function/inline_function to pass through the root pointer.
However, as long as eval_const_expressions is documented to accept
NULL for root, similar hazards would remain. For now, let's just do
the narrow fix of hardening estimate_array_length to not crash.
Its behavior with NULL root will be the same as it was before
9391f7152, so this is not too awful.
Per report from Fredrik Widlert (via Paul Ramsey). Back-patch to v17
where 9391f7152 came in.
Discussion: https://postgr.es/m/518339E7-173E-45EC-A0FF-9A4A62AA4F40@cleverelephant.ca
M src/backend/optimizer/path/costsize.c
M src/backend/utils/adt/selfuncs.c
Apply GUC name from central table in more places of guc.c
commit : f3f06b13308e3c250bb8b0afee861744cda06cf6
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 9 Oct 2024 18:47:34 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 9 Oct 2024 18:47:34 +0900
The name extracted from the record of the GUC tables is applied to more
internal places of guc.c. This change has the advantage to simplify
parse_and_validate_value(), where the "name" was only used in elog
messages, while it was required to match with the name from the GUC
record.
pg_parameter_aclcheck() now passes the name of the GUC from its record
in two places rather than the caller's argument. The value given to
this function goes through convert_GUC_name_for_parameter_acl() that
does a simple ASCII downcasing.
Few GUCs mix character casing in core; one test is added for one of
these code paths with "IntervalStyle".
Author: Peter Smith, Michael Paquier
Discussion: https://postgr.es/m/ZwNh4vkc2NHJHnND@paquier.xyz
M src/backend/utils/misc/guc.c
M src/test/regress/expected/guc.out
M src/test/regress/sql/guc.sql
Allow pushdown of HAVING clauses with grouping sets
commit : 67a54b9e83d331eadd3a595e6c3bfec06288d2c4
author : Richard Guo <rguo@postgresql.org>
date : Wed, 9 Oct 2024 17:19:04 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Wed, 9 Oct 2024 17:19:04 +0900
In some cases, we may want to transfer a HAVING clause into WHERE in
hopes of eliminating tuples before aggregation instead of after.
Previously, we couldn't do this if there were any nonempty grouping
sets, because we didn't have a way to tell if the HAVING clause
referenced any columns that were nullable by the grouping sets, and
moving such a clause into WHERE could potentially change the results.
Now, with expressions marked nullable by grouping sets with the RT
index of the RTE_GROUP RTE, it is much easier to identify those
clauses that reference any nullable-by-grouping-sets columns: we just
need to check if the RT index of the RTE_GROUP RTE is present in the
clause. For other HAVING clauses, they can be safely pushed down.
Author: Richard Guo
Discussion: https://postgr.es/m/CAMbWs4-NpzPgtKU=hgnvyn+J-GanxQCjrUi7piNzZ=upiCV=2Q@mail.gmail.com
M src/backend/optimizer/plan/planner.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Consider explicit incremental sort for mergejoins
commit : 828e94c9d2fd87c06a75354361543119d9937068
author : Richard Guo <rguo@postgresql.org>
date : Wed, 9 Oct 2024 17:14:42 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Wed, 9 Oct 2024 17:14:42 +0900
For a mergejoin, if the given outer path or inner path is not already
well enough ordered, we need to do an explicit sort. Currently, we
only consider explicit full sort and do not account for incremental
sort.
In this patch, for the outer path of a mergejoin, we choose to use
explicit incremental sort if it is enabled and there are presorted
keys. For the inner path, though, we cannot use incremental sort
because it does not support mark/restore at present.
The rationale is based on the assumption that incremental sort is
always faster than full sort when there are presorted keys, a premise
that has been applied in various parts of the code. In addition, the
current cost model tends to favor incremental sort as being cheaper
than full sort in the presence of presorted keys, making it reasonable
not to consider full sort in such cases.
It could be argued that what if a mergejoin with an incremental sort
as the outer path is selected as the inner path of another mergejoin.
However, this should not be a problem, because mergejoin itself does
not support mark/restore either, and we will add a Material node on
top of it anyway in this case (see final_cost_mergejoin).
There is one ensuing plan change in the regression tests, and we have
to modify that test case to ensure that it continues to test what it
is intended to.
No backpatch as this could result in plan changes.
Author: Richard Guo
Reviewed-by: David Rowley, Tomas Vondra
Discussion: https://postgr.es/m/CAMbWs49x425QrX7h=Ux05WEnt8GS757H-jOP3_xsX5t1FoUsZw@mail.gmail.com
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/createplan.c
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/incremental_sort.sql
Remove incorrect function import from pgindent
commit : c4528fdfa903c74cf99837a554bd3c7e115bb366
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 9 Oct 2024 09:34:34 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 9 Oct 2024 09:34:34 +0200
Commit 149ac7d4559 which re-implemented pgindent in Perl explicitly
imported the devnull function from File::Spec, but the module does
not export anything. In recent versions of Perl calling a missing
import function cause a warning, which combined with warnings being
fatal cause pgindent to error out.
Backpatch to all supported versions.
Author: Erik Wienhold <ewie@ewie.name>
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discusson: https://postgr.es/m/2372cd74-11b0-46f9-b28e-8f9627215d19@ewie.name
Backpatch-through: v12
M src/tools/pgindent/pgindent
Allow roles created by new test to log in under SSPI.
commit : 97354286619904bbfd2607b14f3f0d01926e95d0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Oct 2024 19:46:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Oct 2024 19:46:39 -0400
Semi-blind attempt to fix 6a1d0d470 to work on Windows,
along the same lines as a70f2a57f. Per buildfarm.
M src/test/postmaster/t/001_connection_limits.pl
pg_stat_statements: Add columns to track parallel worker activity
commit : cf54a2c002544a4b7934deb44c895750aadb0a3c
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 9 Oct 2024 08:30:45 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 9 Oct 2024 08:30:45 +0900
The view pg_stat_statements gains two columns:
- parallel_workers_to_launch, the number of parallel workers planned to
be launched.
- parallel_workers_launched, the number of parallel workers actually
launched.
The ratio of both columns offers hints that parallel workers are lacking
on a per-statement basis, requiring some tuning, in coordination with
"calls", the number of times a query is executed.
As of now, these numbers are tracked within Gather and GatherMerge
nodes. They could be extended to utilities that make use of parallel
workers (parallel btree and brin, VACUUM).
The module is bumped to 1.12.
Author: Guillaume Lelarge
Discussion: https://postgr.es/m/CAECtzeWtTGOK0UgKXdDGpfTVSa5bd_VbUt6K6xn8P7X+_dZqKw@mail.gmail.com
M contrib/pg_stat_statements/Makefile
M contrib/pg_stat_statements/expected/oldextversions.out
A contrib/pg_stat_statements/expected/parallel.out
M contrib/pg_stat_statements/meson.build
A contrib/pg_stat_statements/pg_stat_statements–1.11–1.12.sql
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/pg_stat_statements.control
M contrib/pg_stat_statements/sql/oldextversions.sql
A contrib/pg_stat_statements/sql/parallel.sql
M doc/src/sgml/pgstatstatements.sgml
Introduce two fields in EState to track parallel worker activity
commit : de3a2ea3b264a5625cda6d8968f75e9bfb4dbdcb
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 9 Oct 2024 08:07:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 9 Oct 2024 08:07:48 +0900
These fields can be set by executor nodes to record how many parallel
workers were planned to be launched and how many of them have been
actually launched within the number initially planned. This data is
able to give an approximation of the parallel worker draught a system
is facing, making easier the tuning of related configuration parameters.
These fields will be used by some follow-up patches to populate other
parts of the system with their data.
Author: Guillaume Lelarge, Benoit Lobréau
Discussion: https://postgr.es/m/783bc7f7-659a-42fa-99dd-ee0565644e25@dalibo.com
Discussion: https://postgr.es/m/CAECtzeWtTGOK0UgKXdDGpfTVSa5bd_VbUt6K6xn8P7X+_dZqKw@mail.gmail.com
M src/backend/executor/execUtils.c
M src/backend/executor/nodeGather.c
M src/backend/executor/nodeGatherMerge.c
M src/include/nodes/execnodes.h
Silence assorted annoying test output.
commit : 01fce8dab16385573ba0f2ddd3d52ba3514a6562
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Oct 2024 14:13:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Oct 2024 14:13:01 -0400
Remove unnecessary chatter about "checking if IO::Socket::UNIX works";
our tests should never print anything on stderr unless there's a
problem.
Add .gitignore entry for temporary directory now being left behind
in src/test/postmaster.
M src/test/perl/PostgreSQL/Test/Cluster.pm
A src/test/postmaster/.gitignore
Add min and max aggregates for bytea type.
commit : 2d24fd942c7010247c5b7b54b416acf7f4592a4e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Oct 2024 13:52:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Oct 2024 13:52:14 -0400
Similar to a0f1fce80, although we chose to duplicate logic
rather than invoke byteacmp, primarily to avoid repeat detoasting.
Marat Buharov, Aleksander Alekseev
Discussion: https://postgr.es/m/CAPCEVGXiASjodos4P8pgyV7ixfVn-ZgG9YyiRZRbVqbGmfuDyg@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/varlena.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_aggregate.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/sql/aggregates.sql
Use aux process resource owner in walsender
commit : 57f37024712702350b714cf64215077f1d75c568
author : Andres Freund <andres@anarazel.de>
date : Tue, 8 Oct 2024 11:37:45 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 8 Oct 2024 11:37:45 -0400
AIO will need a resource owner to do IO. Right now we create a resowner
on-demand during basebackup, and we could do the same for AIO. But it seems
easier to just always create an aux process resowner.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/1f6b50a7-38ef-4d87-8246-786d39f46ab9@iki.fi
M src/backend/backup/basebackup.c
M src/backend/replication/walsender.c
M src/include/replication/walsender.h
bufmgr/smgr: Don't cross segment boundaries in StartReadBuffers()
commit : 755a4c10d19dbe432a1860cced914c570ff3becc
author : Andres Freund <andres@anarazel.de>
date : Tue, 8 Oct 2024 11:37:45 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 8 Oct 2024 11:37:45 -0400
With real AIO it doesn't make sense to cross segment boundaries with one
IO. Add smgrmaxcombine() to allow upper layers to query which buffers can be
merged.
We could continue to cross segment boundaries when not using AIO, but it
doesn't really make sense, because md.c will never be able to perform the read
across the segment boundary in one system call. Which means we'll mark more
buffers as undergoing IO than really makes sense - if another backend desires
to read the same blocks, it'll be blocked longer than necessary. So it seems
better to just never cross the boundary.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/1f6b50a7-38ef-4d87-8246-786d39f46ab9@iki.fi
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/smgr/md.c
M src/backend/storage/smgr/smgr.c
M src/include/storage/md.h
M src/include/storage/smgr.h
bufmgr: Return early in ScheduleBufferTagForWriteback() if fsync=off
commit : 488f826c729bd570c36df369fa8ac90c9a5a1b46
author : Andres Freund <andres@anarazel.de>
date : Tue, 8 Oct 2024 11:37:45 -0400
committer: Andres Freund <andres@anarazel.de>
date : Tue, 8 Oct 2024 11:37:45 -0400
As pg_flush_data() doesn't do anything with fsync disabled, there's no point
in tracking the buffer for writeback. Arguably the better fix would be to
change pg_flush_data() to flush data even with fsync off, but that's a
behavioral change, whereas this is just a small optimization.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/1f6b50a7-38ef-4d87-8246-786d39f46ab9@iki.fi
M src/backend/storage/buffer/bufmgr.c
Silence buildfarm warning chatter from bd1276a3c.
commit : c01fd93088da36f2473a6b1bad3fce3e04c685a1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Oct 2024 11:15:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Oct 2024 11:15:16 -0400
Buildfarm members using -Wextra complained about "warning: suggest
braces around empty body in an 'if' statement". Do it gcc's way,
though I see no actual readability benefit in this.
M src/bin/psql/tab-complete.in.c
Fix typo and run pgperltidy on newly-added test
commit : 05d1b9b5c286424dea2a91cf906c49c9d22ff4bf
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 8 Oct 2024 15:47:51 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 8 Oct 2024 15:47:51 +0300
From commit 85ec945b78.
M src/test/postmaster/t/001_connection_limits.pl
Use an shmem_exit callback to remove backend from PMChildFlags on exit
commit : 2bbc261ddbdfee2def5d14ee9fcc09c70bdf84e6
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 8 Oct 2024 15:06:34 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 8 Oct 2024 15:06:34 +0300
This seems nicer than having to duplicate the logic between
InitProcess() and ProcKill() for which child processes have a
PMChildFlags slot.
Move the MarkPostmasterChildActive() call earlier in InitProcess(),
out of the section protected by the spinlock.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/a102f15f-eac4-4ff2-af02-f9ff209ec66f@iki.fi
M src/backend/storage/ipc/pmsignal.c
M src/backend/storage/lmgr/proc.c
M src/include/storage/pmsignal.h
Add test for dead-end backends
commit : 85ec945b7880931cb506392101cb0b00209b78ba
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 8 Oct 2024 15:06:31 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 8 Oct 2024 15:06:31 +0300
The code path for launching a dead-end backend because we're out of
slots was not covered by any tests, so add one. (Some tests did hit
the case of launching a dead-end backend because the server is still
starting up, though, so the gap in our test coverage wasn't as big as
it sounds.)
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/a102f15f-eac4-4ff2-af02-f9ff209ec66f@iki.fi
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/test/postmaster/t/001_connection_limits.pl
Add test for connection limits
commit : 6a1d0d470e84f960a50cbe0cd1e2eee4b8c9baa5
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 8 Oct 2024 15:06:26 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 8 Oct 2024 15:06:26 +0300
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/a102f15f-eac4-4ff2-af02-f9ff209ec66f@iki.fi
M src/test/Makefile
M src/test/meson.build
A src/test/postmaster/Makefile
A src/test/postmaster/README
A src/test/postmaster/meson.build
A src/test/postmaster/t/001_connection_limits.pl
Doc: add check to detect non-breaking spaces in the docs.
commit : 5b7da5c261d1af1a5d6a275e1090b07de3654033
author : Tatsuo Ishii <ishii@postgresql.org>
date : Tue, 8 Oct 2024 20:25:18 +0900
committer: Tatsuo Ishii <ishii@postgresql.org>
date : Tue, 8 Oct 2024 20:25:18 +0900
There were multiple instances where accidentally adding non-breaking
space (nbsp, U+00A0, 0xc2a0 in UTF-8) to sgml files. This commit adds
additional checking to detect nbsp. You can check the nbsp by:
make -C doc/src/sgml check
or
make -C doc/src/sgml check-nbsp
Authors: Yugo Nagata, Daniel Gustafsson
Reviewed-by: Tatsuo Ishii, Daniel Gustafsson
Discussion: https://postgr.es/m/20240930.153404.202479334310259810.ishii%40postgresql.org
M doc/src/sgml/Makefile
Move check for binary mode and on_error option to the appropriate location.
commit : a39297ec026c556b1bf2963dde766db31a2c3d2d
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 8 Oct 2024 18:23:43 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 8 Oct 2024 18:23:43 +0900
Commit 9e2d870119 placed the check for binary mode and on_error
before default values were inserted, which was not ideal.
This commit moves the check to a more appropriate position
after default values are set.
Additionally, the comment incorrectly mentioned two checks before
inserting defaults, when there are actually three. This commit corrects
that comment.
Author: Atsushi Torikoshi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/8830518a-28ac-43a2-8a11-1676d9a3cdf8@oss.nttdata.com
M src/backend/commands/copy.c
Add REJECT_LIMIT option to the COPY command.
commit : 4ac2a9beceb10d44806d2cf157d5a931bdade39e
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 8 Oct 2024 18:19:58 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 8 Oct 2024 18:19:58 +0900
Previously, when ON_ERROR was set to 'ignore', the COPY command
would skip all rows with data type conversion errors, with no way to
limit the number of skipped rows before failing.
This commit introduces the REJECT_LIMIT option, allowing users to
specify the maximum number of erroneous rows that can be skipped.
If more rows encounter data type conversion errors than allowed by
REJECT_LIMIT, the COPY command will fail with an error, even when
ON_ERROR = 'ignore'.
Author: Atsushi Torikoshi
Reviewed-by: Junwang Zhao, Kirill Reshke, jian he, Fujii Masao
Discussion: https://postgr.es/m/63f99327aa6b404cc951217fa3e61fe4@oss.nttdata.com
M doc/src/sgml/ref/copy.sgml
M src/backend/commands/copy.c
M src/backend/commands/copyfrom.c
M src/include/commands/copy.h
M src/test/regress/expected/copy2.out
M src/test/regress/sql/copy2.sql
Stabilize the test added by commit 022564f60c.
commit : d759c1a0b864366d3592203c782027825ec4c8c8
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 8 Oct 2024 12:25:52 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 8 Oct 2024 12:25:52 +0530
The test was unstable in branches 14 and 15 as we were relying on the
number of changes in the table having a toast column to start streaming.
On branches >= 16, we have a GUC debug_logical_replication_streaming which
can stream each change, so the test was stable in those branches.
Change the test to use PREPARE TRANSACTION as that should make the result
consistent and test the code changed in 022564f60c.
Reported-by: Daniel Gustafsson as per buildfarm
Author: Hou Zhijie, Amit Kapila
Backpatch-through: 14
Discussion: https://postgr.es/m/8C2F86AA-981E-4803-B14D-E264C0255330@yesql.se
M contrib/test_decoding/expected/stream.out
M contrib/test_decoding/expected/twophase.out
M contrib/test_decoding/sql/stream.sql
M contrib/test_decoding/sql/twophase.sql
Improve style of two code paths
commit : 4572d59e3c9b0ef6c056544b6336561efd9bb5ec
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Oct 2024 10:51:20 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Oct 2024 10:51:20 +0900
In execGrouping.c, execTuplesMatchPrepare() was doing a memory
allocation that was not necessary when the number of columns was 0.
In foreign.c, pg_options_to_table() was assigning twice a variable to
the same value.
Author: Ranier Vilela
Discussion: https://postgr.es/m/CAEudQAqup0agbSzMjSLSTn=OANyCzxENF1+HrSYnr3WyZib7=Q@mail.gmail.com
M src/backend/executor/execGrouping.c
M src/backend/foreign/foreign.c
Fix search_path cache initialization.
commit : a9ed7d944965865e5c6b44e58602f621cd185856
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 7 Oct 2024 17:51:14 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 7 Oct 2024 17:51:14 -0700
The cache needs to be available very early, so don't rely on
InitializeSearchPath() to initialize the it.
Reported-by: Murat Efendioğlu
Discussion: https://postgr.es/m/CACbCzujQ4zS8MM1bx-==+tr+D3Hk5G1cjN4XkUQ+Q=cEpwhzqg@mail.gmail.com
Backpatch-through: 17
M src/backend/catalog/namespace.c
Fix test for password hash length limit.
commit : c3b80a7e98456904a2bf8f06a0cbf10f8dae1e83
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 7 Oct 2024 17:17:39 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 7 Oct 2024 17:17:39 -0500
In commit 8275325a06, I forgot to update password_1.out (an
alternative expected test output file added by commit 3c44e7d8d4),
so this test began failing on machines with FIPS mode enabled.
M src/test/regress/expected/password_1.out
vacuumdb: Schema-qualify operator in catalog query's WHERE clause.
commit : 8318f2b170d81e6792b90f2aa2b491a2b7add441
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 7 Oct 2024 16:49:20 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 7 Oct 2024 16:49:20 -0500
Commit 1ab67c9dfa, which modified this catalog query so that it
doesn't return temporary relations, forgot to schema-qualify the
operator. A comment earlier in the function implores us to fully
qualify everything in the query:
* Since we execute the constructed query with the default search_path
* (which could be unsafe), everything in this query MUST be fully
* qualified.
This commit fixes that. While at it, add a newline for consistency
with surrounding code.
Reviewed-by: Noah Misch
Discussion: https://postgr.es/m/ZwQJYcuPPUsF0reU%40nathan
Backpatch-through: 12
M src/bin/scripts/vacuumdb.c
Fix Y2038 issues with MyStartTime.
commit : 5d6187d2a28326b3414c394ec4c28c51bb513c44
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 7 Oct 2024 13:51:03 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 7 Oct 2024 13:51:03 -0500
Several places treat MyStartTime as a "long", which is only 32 bits
wide on some platforms. In reality, MyStartTime is a pg_time_t,
i.e., a signed 64-bit integer. This will lead to interesting bugs
on the aforementioned systems in 2038 when signed 32-bit integers
are no longer sufficient to store Unix time (e.g., "pg_ctl start"
hanging). To fix, ensure that MyStartTime is handled as a 64-bit
value everywhere. (Of course, users will need to ensure that
time_t is 64 bits wide on their system, too.)
Co-authored-by: Max Johnson
Discussion: https://postgr.es/m/CO1PR07MB905262E8AC270FAAACED66008D682%40CO1PR07MB9052.namprd07.prod.outlook.com
Backpatch-through: 12
M contrib/postgres_fdw/option.c
M src/backend/utils/error/csvlog.c
M src/backend/utils/error/elog.c
M src/backend/utils/error/jsonlog.c
M src/backend/utils/init/miscinit.c
M src/bin/pg_ctl/pg_ctl.c
Convert tab-complete's long else-if chain to a switch statement.
commit : f391d9dc93a24923c57bb0e044161d3f0b840770
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Oct 2024 12:22:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Oct 2024 12:22:10 -0400
Rename tab-complete.c to tab-complete.in.c, create the preprocessor
script gen_tabcomplete.pl, and install Makefile/meson.build rules
to create tab-complete.c from tab-complete.in.c. The preprocessor
converts match_previous_words' else-if chain into a switch and
populates tcpatterns[] with the data needed by the driver loop.
The initial HeadMatches/TailMatches/Matches test in each else-if arm
is now performed in a table-driven loop. Where we get a match, the
corresponding switch case is invoked to see if the match succeeds.
(It might not, if there were additional conditions in the original
else-if test.)
The total number of string comparisons done is just about the
same as it was in the previous coding; however, now that we
have table-driven logic underlying the handmade rules, there
is room to improve that. For now I haven't bothered because
tab completion is still plenty fast enough for human use.
If the number of rules keeps increasing, we might someday
need to do more in that area.
The immediate benefit of all this thrashing is that C compilers
frequently don't deal well with long else-if chains. On gcc 8.5.0,
this reduces the compile time of tab-complete.c by about a factor of
four, while MSVC is reported to crash outright with the previous
coding.
Discussion: https://postgr.es/m/2208466.1720729502@sss.pgh.pa.us
M src/bin/psql/.gitignore
M src/bin/psql/Makefile
A src/bin/psql/gen_tabcomplete.pl
M src/bin/psql/meson.build
R099 src/bin/psql/tab-complete.c src/bin/psql/tab-complete.in.c
Prepare tab-complete.c for preprocessing.
commit : bd1276a3c9433a9e2760db6ae6e73560d7d32a22
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Oct 2024 12:19:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Oct 2024 12:19:12 -0400
Separate out psql_completion's giant else-if chain of *Matches
tests into a new function. Add the infrastructure needed for
table-driven checking of the initial match of each completion
rule. As-is, however, the code continues to operate as it did.
The new behavior applies only if SWITCH_CONVERSION_APPLIED
is #defined, which it is not here. (The preprocessor added
in the next patch will add a #define for that.)
The first and last couple of bits of psql_completion are not
based on HeadMatches/TailMatches/Matches tests, so they stay
where they are; they won't become part of the switch.
This patch also fixes up a couple of if-conditions that didn't meet
the conditions enumerated in the comment for match_previous_words().
Those restrictions exist to simplify the preprocessor.
Discussion: https://postgr.es/m/2208466.1720729502@sss.pgh.pa.us
M src/bin/psql/tab-complete.c
M src/tools/pgindent/typedefs.list
Invent "MatchAnyN" option for tab-complete.c's Matches/MatchesCS.
commit : ef0938f7bdd0838857da38b9a660191ec8411696
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Oct 2024 12:13:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Oct 2024 12:13:02 -0400
This argument matches any number (including zero) of previous words.
Use it to replace the common coding pattern
if (HeadMatches("A", "B") && TailMatches("X", "Y"))
with
if (Matches("A", "B", MatchAnyN, "X", "Y"))
In itself this feature doesn't do much except (arguably) make the
code slightly shorter and more readable. However, it reduces the
number of complex if-condition patterns that have to be dealt with
in the next commits in this series.
While here, restructure the *Matches implementation functions so
that the actual work is done in functions that take a char **
array of pattern strings, and the versions taking variadic arguments
are thin wrappers around the array ones. This simplifies the
new Matches logic considerably. At the end of this patch series,
the array functions will be the only ones that are material to
performance, so having the variadic ones be wrappers makes sense.
Discussion: https://postgr.es/m/2208466.1720729502@sss.pgh.pa.us
M src/bin/psql/tab-complete.c
Restrict password hash length.
commit : 8275325a06ed91c053e046422a193dc6d56a70c5
author : Nathan Bossart <nathan@postgresql.org>
date : Mon, 7 Oct 2024 10:56:16 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Mon, 7 Oct 2024 10:56:16 -0500
Commit 6aa44060a3 removed pg_authid's TOAST table because the only
varlena column is rolpassword, which cannot be de-TOASTed during
authentication because we haven't selected a database yet and
cannot read pg_class. Since that change, attempts to set password
hashes that require out-of-line storage will fail with a "row is
too big" error. This error message might be confusing to users.
This commit places a limit on the length of password hashes so that
attempts to set long password hashes will fail with a more
user-friendly error. The chosen limit of 512 bytes should be
sufficient to avoid "row is too big" errors independent of BLCKSZ,
but it should also be lenient enough for all reasonable use-cases
(or at least all the use-cases we could imagine).
Reviewed-by: Tom Lane, Jonathan Katz, Michael Paquier, Jacob Champion
Discussion: https://postgr.es/m/89e8649c-eb74-db25-7945-6d6b23992394%40gmail.com
M src/backend/libpq/crypt.c
M src/include/libpq/crypt.h
M src/test/regress/expected/password.out
M src/test/regress/sql/password.sql
Fix fetching default toast value during decoding of in-progress transactions.
commit : 022564f60ca5cade8fd663906f3ee514573b4b5e
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 7 Oct 2024 15:38:45 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 7 Oct 2024 15:38:45 +0530
During logical decoding of in-progress transactions, we perform the toast
table scan while fetching the default toast value for an attribute. We
forgot to initialize the flag during this scan to indicate that the system
table scan is in progress. We need this flag to ensure that during logical
decoding we never directly access the tableam or heap APIs because we check
for concurrent aborts only in systable_* APIs.
Reported-by: Alexander Lakhin
Author: Takeshi Ideriha, Hou Zhijie
Reviewed-by: Amit Kapila, Hou Zhijie
Backpatch-through: 14
Discussion: https://postgr.es/m/18641-6687273b7f15269d@postgresql.org
M contrib/test_decoding/expected/stream.out
M contrib/test_decoding/sql/stream.sql
M src/backend/access/index/genam.c
doc: Quote value in SET NAMES documentation
commit : 6ae387eb6334e71c94347b50b711b6750ef71b50
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 7 Oct 2024 11:50:39 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 7 Oct 2024 11:50:39 +0200
The value passed to SET NAMES should be wrapped in single quotes.
Reported-by: jian he <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CACJufxG3EoUsbX4ZoMFkWrvBJcSCbPjdpRvPhuQN65fADc3mFg@mail.gmail.com
M doc/src/sgml/ref/set.sgml
doc: Add minimal C and SQL example to add a custom table AM handler
commit : e09fff7c980a8b545a18e119be41bf67cd2e9182
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Oct 2024 15:47:40 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Oct 2024 15:47:40 +0900
The documentation was rather sparse on this matter and there is no
extension in-core that shows how to do it. Adding a small example will
hopefully help newcomers. An advantage of writing things this way is
that the contents are not going to rot because of backend changes.
Author: Phil Eaton
Reviewed-by: Robert Haas, Fabrízio de Royes Mello
Discussion: https://postgr.es/m/CAByiw+r+CS-ojBDP7Dm=9YeOLkZTXVnBmOe_ajK=en8C_zB3_g@mail.gmail.com
M doc/src/sgml/tableam.sgml
Use camel case for "DateStyle" in some error messages
commit : 2e7c4abe5a88a1528236a13c9ef2e351a97fa022
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Oct 2024 12:36:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Oct 2024 12:36:00 +0900
This GUC is written as camel-case in most of the documentation and the
GUC table (but not postgresql.conf.sample), and two error messages
hardcoded it with lower case characters. Let's use a style more
consistent.
Most of the noise comes from the regression tests, updated to reflect
the GUC name in these error messages.
Author: Peter Smith
Reviewed-by: Peter Eisentraut, Álvaro Herrera
Discussion: https://postgr.es/m/CAHut+Pv-kSN8SkxSdoHano_wPubqcg5789ejhCDZAcLFceBR-w@mail.gmail.com
M src/backend/commands/variable.c
M src/backend/utils/adt/datetime.c
M src/test/regress/expected/date.out
M src/test/regress/expected/horology.out
Ignore not-yet-defined Portals in pg_cursors view.
commit : f8d9a9f21e2fec4f67af7b430b981b6e6e7a0906
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Oct 2024 16:03:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Oct 2024 16:03:48 -0400
pg_cursor() supposed that any Portal it finds in the hash table must
have sourceText set up, but there's an edge case where that is not so.
A newly-created Portal has sourceText = NULL, and that doesn't change
until PortalDefineQuery is called. In SPI_cursor_open_internal,
we perform GetCachedPlan between CreatePortal and PortalDefineQuery,
and it's possible for user-defined code to execute during that
planning and cause a fetch from the pg_cursors view, resulting in a
null-pointer-dereference crash. (It looks like the same could happen
in exec_bind_message, but I've not tried to provoke a failure there.)
I considered trying to fix this by setting sourceText sooner, but
there may be instances of this same calling pattern in extensions,
and we couldn't be sure they'd get the memo promptly. It seems
better to redefine pg_cursor as not showing Portals that have
not yet had PortalDefineQuery called on them, which we can do by
just skipping them if sourceText is still NULL.
(Before a1c692358, pg_cursor would instead return a row with NULL
in the statement column. We could revert to that behavior but it
doesn't really seem like a better definition, especially since our
documentation doesn't suggest that the column could be NULL.)
Per report from PetSerAl. Back-patch to all supported branches.
Discussion: https://postgr.es/m/CAKygsHTBXLXjwV43kpZa+Cs+XTiaeeJiZdL4cPBm9f4MTdw7wg@mail.gmail.com
M src/backend/utils/mmgr/portalmem.c
Move Cluster.pm initialization code to a more obvious place
commit : 70fea390cfa71c438998993bebfb584effd3e7df
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 6 Oct 2024 10:34:45 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 6 Oct 2024 10:34:45 -0400
Commit 460c0076e8 added some module intialization code to set signal
handlers. However, that code has now become somewhat buried, as later
commits added new subroutines. Therefore, move the initialization code
to the module's INIT block where it won't become obscured.
M src/test/perl/PostgreSQL/Test/Cluster.pm
libpq: Discard leading and trailing spaces for parameters and values in URIs
commit : 430ce189fc4521a1c197eee36b8ca6b37b29e86e
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 6 Oct 2024 18:23:02 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 6 Oct 2024 18:23:02 +0900
Integer values applied a parsing rule through pqParseIntParam() that
made URIs like this one working, even if these include spaces around
values:
"postgresql://localhost:5432/postgres?keepalives=1 &keepalives_idle=1 "
This commit changes the parsing so as spaces before and after parameters
and values are discarded, offering more consistency with the parsing
that already applied to libpq for integer values in URIs.
Note that %20 can be used in a URI for a space character. ECPGconnect()
has been discarded leading and trailing spaces around parameters and
values that for a long time, as well. Like f22e84df1dea, this is done
as a HEAD-only change.
Reviewed-by: Yuto Sasaki
Discussion: https://postgr.es/m/Zv3oWOfcrHTph7JK@paquier.xyz
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/t/001_uri.pl
Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.
commit : 68dfecbef210dc000271553cfcb2342989d4ca0f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Oct 2024 14:46:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Oct 2024 14:46:44 -0400
When instantiating an existing partitioned index for a new child
partition, we use generateClonedIndexStmt to build a suitable
IndexStmt to pass to DefineIndex. However, when DefineIndex needs
to recurse to instantiate a newly created partitioned index on an
existing child partition, it was doing copyObject on the given
IndexStmt and then applying a bunch of ad-hoc fixups. This has
a number of problems, primarily that it implies fresh lookups of
referenced objects such as opclasses and collations. Since commit
2af07e2f7 caused DefineIndex to restrict search_path internally, those
lookups could fail or deliver different results than the original one.
We can avoid those problems and save a few dozen lines of code by
using generateClonedIndexStmt in this code path too.
Another thing this fixes is incorrect propagation of parent-index
comments to child indexes (because the copyObject approach copies
the idxcomment field while generateClonedIndexStmt doesn't). I had
noticed this in connection with commit c01eb619a, but not run the
problem to ground.
I'm tempted to back-patch this further than v17, but the only thing
it's known to fix in older branches is the comment issue, which is
pretty minor and doesn't seem worth the risk of introducing new
issues in stable branches. (If anyone does care about that,
clearing idxcomment in the copied IndexStmt would be a safer fix.)
Per bug #18637 from usamoi. Back-patch to v17 where the search_path
change came in.
Discussion: https://postgr.es/m/18637-f51e314546e3ba2a@postgresql.org
M contrib/seg/Makefile
A contrib/seg/expected/partition.out
M contrib/seg/meson.build
A contrib/seg/sql/partition.sql
M src/backend/commands/indexcmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
Clean up WaitLatch calls that passed latch without WL_LATCH_SET
commit : f9ecb57a506ada358baa19287d2c438ad01cdbce
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sat, 5 Oct 2024 15:31:06 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sat, 5 Oct 2024 15:31:06 +0300
The 'latch' argument is ignored if WL_LATCH_SET is not given. Clarify
these calls by not pointlessly passing MyLatch.
Discussion: https://www.postgresql.org/message-id/391abe21-413e-4d91-a650-b663af49500c@iki.fi
M src/backend/libpq/be-secure-gssapi.c
M src/backend/libpq/be-secure-openssl.c
M src/backend/postmaster/walsummarizer.c
Remove unneeded #include
commit : 094ae071605d938c46b440b4c27b11f73a241422
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sat, 5 Oct 2024 15:09:32 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sat, 5 Oct 2024 15:09:32 +0300
Unneeded since commit d72731a704.
Discussion: https://www.postgresql.org/message-id/391abe21-413e-4d91-a650-b663af49500c@iki.fi
M src/include/storage/buf_internals.h
Remove unused latch
commit : 6c0c49f7d37dc29775947ac6edbc494c32f43fa6
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sat, 5 Oct 2024 15:09:27 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sat, 5 Oct 2024 15:09:27 +0300
It was left unused by commit bc971f4025, which replaced the latch
usage with a condition variable
Discussion: https://www.postgresql.org/message-id/391abe21-413e-4d91-a650-b663af49500c@iki.fi
M src/backend/replication/walsender.c
M src/include/replication/walsender_private.h
Reject non-ASCII locale names.
commit : adbb27ac89e07cfbd94ea07a53930addfc3895ee
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 5 Oct 2024 13:48:33 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 5 Oct 2024 13:48:33 +1300
Commit bf03cfd1 started scanning all available BCP 47 locale names on
Windows. This caused an abort/crash in the Windows runtime library if
the default locale name contained non-ASCII characters, because of our
use of the setlocale() save/restore pattern with "char" strings. After
switching to another locale with a different encoding, the saved name
could no longer be understood, and setlocale() would abort.
"Turkish_Türkiye.1254" is the example from recent reports, but there are
other examples of countries and languages with non-ASCII characters in
their names, and they appear in Windows' (old style) locale names.
To defend against this:
1. In initdb, reject non-ASCII locale names given explicity on the
command line, or returned by the operating system environment with
setlocale(..., ""), or "canonicalized" by the operating system when we
set it.
2. In initdb only, perform the save-and-restore with Windows'
non-standard wchar_t variant of setlocale(), so that it is not subject
to round trip failures stemming from char string encoding confusion.
3. In the backend, we don't have to worry about the save-and-restore
problem because we have already vetted the defaults, so we just have to
make sure that CREATE DATABASE also rejects non-ASCII names in any new
databases. SET lc_XXX doesn't suffer from the problem, but the ban
applies to it too because it uses check_locale(). CREATE COLLATION
doesn't suffer from the problem either, but it doesn't use
check_locale() so it is not included in the new ban for now, to minimize
the change.
Anyone who encounters the new error message should either create a new
duplicated locale with an ASCII-only name using Windows Locale Builder,
or consider using BCP 47 names like "tr-TR". Users already couldn't
initialize a cluster with "Turkish_Türkiye.1254" on PostgreSQL 16+, but
the new failure mode is an error message that explains why, instead of a
crash.
Back-patch to 16, where bf03cfd1 landed. Older versions are affected
in theory too, but only 16 and later are causing crash reports.
Reviewed-by: Andrew Dunstan <andrew@dunslane.net> (the idea, not the patch)
Reported-by: Haifang Wang (Centific Technologies Inc) <v-haiwang@microsoft.com>
Discussion: https://postgr.es/m/PH8PR21MB3902F334A3174C54058F792CE5182%40PH8PR21MB3902.namprd21.prod.outlook.com
M src/backend/utils/adt/pg_locale.c
M src/bin/initdb/initdb.c
ecpg: avoid adding whitespace around '&' in connection URLs.
commit : f22e84df1dea96c8f4b0f7369ea60607fbb9ce10
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Oct 2024 12:01:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Oct 2024 12:01:50 -0400
The preprocessor really should not have done this to begin with.
The space after '&' was masked by ECPGconnect's skipping spaces
before option keywords, and the space before by dint of libpq
being (mostly) insensitive to trailing space in option values.
We fixed the one known problem with that in 920d51979. Hence
this patch is mostly cosmetic, and we'll just change it in HEAD.
Discussion: https://postgr.es/m/TY2PR01MB36286A7B97B9A15793335D18C1772@TY2PR01MB3628.jpnprd01.prod.outlook.com
M src/interfaces/ecpg/ecpglib/connect.c
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/test/expected/connect-test5.c
M src/interfaces/ecpg/test/expected/connect-test5.stderr
Rename PageData to GenericXLogPageData
commit : ddbba3aac868132137782b02a4e81218cfd3b4b9
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 4 Oct 2024 12:47:35 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 4 Oct 2024 12:47:35 +0200
In the PostgreSQL C type naming schema, the type PageData should be
what the pointer of type Page points to. But in this case it's
actually an unrelated type local to generic_xlog.c. Rename that to a
more specific name. This makes room to possible add a PageData type
with the mentioned meaning, but this is not done here.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/001d457e-c118-4219-8132-e1846c2ae3c9%40eisentraut.org
M src/backend/access/transam/generic_xlog.c
M src/tools/pgindent/typedefs.list
Speed up numeric division by always using the "fast" algorithm.
commit : 9428c001f67540e6206b7a76adfc5377d9207777
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Fri, 4 Oct 2024 09:49:24 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Fri, 4 Oct 2024 09:49:24 +0100
Formerly there were two internal functions in numeric.c to perform
numeric division, div_var() and div_var_fast(). div_var() performed
division exactly to a specified rscale using Knuth's long division
algorithm, while div_var_fast() used the algorithm from the "FM"
library, which approximates each quotient digit using floating-point
arithmetic, and computes a truncated quotient with DIV_GUARD_DIGITS
extra digits. div_var_fast() could be many times faster than
div_var(), but did not guarantee correct results in all cases, and was
therefore only suitable for use in transcendental functions, where
small errors are acceptable.
This commit merges div_var() and div_var_fast() together into a single
function with an extra "exact" boolean parameter, which can be set to
false if the caller is OK with an approximate result. The new function
uses the faster algorithm from the "FM" library, except that when
"exact" is true, it does not truncate the computation with
DIV_GUARD_DIGITS extra digits, but instead performs the full-precision
computation, subtracting off complete multiples of the divisor for
each quotient digit. However, it is able to retain most of the
performance benefits of div_var_fast(), by delaying the propagation of
carries, allowing the inner loop to be auto-vectorized.
Since this may still lead to an inaccurate result, when "exact" is
true, it then inspects the remainder and uses that to adjust the
quotient, if necessary, to make it correct. In practice, the quotient
rarely needs to be adjusted, and never by more than one in the final
digit, though it's difficult to prove that, so the code allows for
larger adjustments, just in case.
In addition, use base-NBASE^2 arithmetic and a 64-bit dividend array,
similar to mul_var(), so that the number of iterations of the outer
loop is roughly halved. Together with the faster algorithm, this makes
div_var() up to around 20 times as fast as the old Knuth algorithm
when "exact" is true, and up to 2 or 3 times as fast as the old
div_var_fast() function when "exact" is false.
Dean Rasheed, reviewed by Joel Jacobson.
Discussion: https://postgr.es/m/CAEZATCVHR10BPDJSANh0u2+Sg6atO3mD0G+CjKDNRMD-C8hKzQ@mail.gmail.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Remove assertion checking query ID in execMain.c
commit : 4dd308730029f5d90bd188f181c2dd9d7630ea69
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 4 Oct 2024 12:51:17 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 4 Oct 2024 12:51:17 +0900
This assertion has been added by 24f520594809, but Alexander Lakhin has
proved that the ExecutorRun() one can be broken by using a PL function
that manipulates compute_query_id and track_activities, while the ones
in ExecutorFinish() and ExecutorEnd() could be triggered when cleaning
up portals at the beginning of a new query execution.
Discussion: https://postgr.es/m/b37d8e6c-e83d-e157-8865-1b2460a6aef2@gmail.com
M src/backend/executor/execMain.c
Fix wrong varnullingrels error for MERGE WHEN NOT MATCHED BY SOURCE.
commit : 259a0a99fe3d45dcf624788c1724d9989f3382dc
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 3 Oct 2024 13:48:32 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 3 Oct 2024 13:48:32 +0100
If a MERGE command contains WHEN NOT MATCHED BY SOURCE actions, the
source relation appears on the outer side of the join. Thus, any Vars
referring to the source in the merge join condition, actions, and
RETURNING list should be marked as nullable by the join, since they
are used in the ModifyTable node above the join. Note that this only
applies to the copy of join condition used in the executor to
distinguish MATCHED from NOT MATCHED BY SOURCE cases. Vars in the
original join condition, inside the join node itself, should not be
marked.
Failure to correctly mark these Vars led to a "wrong varnullingrels"
error in the final stage of query planning, in some circumstances. We
happened to get away without this in all previous tests, since they
all involved a ModifyTable node directly on top of the join node, so
that the top plan targetlist coincided with the output of the join,
and the varnullingrels check was more lax. However, if another plan
node, such as a one-time filter Result node, gets inserted between the
ModifyTable node and the join node, then a stricter check is applied,
which fails.
Per bug #18634 from Alexander Lakhin. Thanks to Tom Lane and Richard
Guo for review and analysis.
Back-patch to v17, where WHEN NOT MATCHED BY SOURCE support was added
to MERGE.
Discussion: https://postgr.es/m/18634-db5299c937877f2b%40postgresql.org
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/merge.out
M src/test/regress/sql/merge.sql
Fix incorrect non-strict join recheck in MERGE WHEN NOT MATCHED BY SOURCE.
commit : dddb5640c6d32f3d48994dac0f4e80d0f4567262
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 3 Oct 2024 12:53:03 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 3 Oct 2024 12:53:03 +0100
If a MERGE command contains WHEN NOT MATCHED BY SOURCE actions, the
merge join condition is used by the executor to distinguish MATCHED
from NOT MATCHED BY SOURCE cases. However, this qual is executed using
the output from the join subplan node, which nulls the output from the
source relation in the not matched case, and so the result may be
incorrect if the join condition is "non-strict" -- for example,
something like "src.col IS NOT DISTINCT FROM tgt.col".
Fix this by enhancing the join recheck condition with an additional
"src IS NOT NULL" check, so that it does the right thing when
evaluated using the output from the join subplan.
Noted by Tom Lane while investigating bug #18634 from Alexander
Lakhin.
Back-patch to v17, where WHEN NOT MATCHED BY SOURCE support was added
to MERGE.
Discussion: https://postgr.es/m/18634-db5299c937877f2b%40postgresql.org
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/merge.out
M src/test/regress/sql/merge.sql
Replace Unicode apostrophe with ASCII apostrophe
commit : 19531968e84557693576928b3184ccc549aa44c8
author : Amit Langote <amitlan@postgresql.org>
date : Thu, 3 Oct 2024 19:51:38 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Thu, 3 Oct 2024 19:51:38 +0900
In commit babb3993dbe9, I accidentally introduced a Unicode
apostrophe (U+2019). This commit replaces it with the ASCII
apostrophe (U+0027) for consistency.
Reported-by: Alexander Korotkov <aekorotkov@gmail.com>
Discussion: https://postgr.es/m/CAPpHfduNWMBjkJFtqXJremk6b6YQYO2s3_VEpnj-T_CaUNUYYQ@mail.gmail.com
M src/backend/commands/tablecmds.c
Refactor CopyFrom() in copyfrom.c.
commit : e55f025b059fb02b659f12f3003c160ed4674dc8
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 3 Oct 2024 15:59:16 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 3 Oct 2024 15:59:16 +0900
This commit simplifies CopyFrom() by removing the unnecessary local variable
'skipped', which tracked the number of rows skipped due to on_error = 'ignore'.
That count is already handled by cstate->num_errors, so the 'skipped' variable
was redundant.
Additionally, the condition on_error != COPY_ON_ERROR_STOP is removed.
Since on_error == COPY_ON_ERROR_IGNORE is already checked, and on_error
only has two values (ignore and stop), the additional check was redundant
and made the logic harder to read. Seemingly this was introduced
in preparation for a future patch, but the current checks don’t offer
clear value and have been removed to improve readability.
Author: Atsushi Torikoshi
Reviewed-by: Masahiko Sawada, Fujii Masao
Discussion: https://postgr.es/m/ab59dad10490ea3734cf022b16c24cfd@oss.nttdata.com
M src/backend/commands/copyfrom.c
file_fdw: Add on_error and log_verbosity options to file_fdw.
commit : a1c4c8a9e1e3a53996dafa1f4ee6d4f7de2c58b2
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 3 Oct 2024 15:57:32 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 3 Oct 2024 15:57:32 +0900
In v17, the on_error and log_verbosity options were introduced for
the COPY command. This commit extends support for these options
to file_fdw.
Setting on_error = 'ignore' for a file_fdw foreign table allows users
to query it without errors, even when the input file contains
malformed rows, by skipping the problematic rows.
Both on_error and log_verbosity options apply to SELECT and ANALYZE
operations on file_fdw foreign tables.
Author: Atsushi Torikoshi
Reviewed-by: Masahiko Sawada, Fujii Masao
Discussion: https://postgr.es/m/ab59dad10490ea3734cf022b16c24cfd@oss.nttdata.com
M contrib/file_fdw/expected/file_fdw.out
M contrib/file_fdw/file_fdw.c
M contrib/file_fdw/sql/file_fdw.sql
M doc/src/sgml/file-fdw.sgml
Add log_verbosity = 'silent' support to COPY command.
commit : e7834a1a251d4a28245377f383ff20a657ba8262
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 3 Oct 2024 15:55:37 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 3 Oct 2024 15:55:37 +0900
Previously, when the on_error option was set to ignore, the COPY command
would always log NOTICE messages for input rows discarded due to
data type incompatibility. Users had no way to suppress these messages.
This commit introduces a new log_verbosity setting, 'silent',
which prevents the COPY command from emitting NOTICE messages
when on_error = 'ignore' is used, even if rows are discarded.
This feature is particularly useful when processing malformed files
frequently, where a flood of NOTICE messages can be undesirable.
For example, when frequently loading malformed files via the COPY command
or querying foreign tables using file_fdw (with an upcoming patch to
add on_error support for file_fdw), users may prefer to suppress
these messages to reduce log noise and improve clarity.
Author: Atsushi Torikoshi
Reviewed-by: Masahiko Sawada, Fujii Masao
Discussion: https://postgr.es/m/ab59dad10490ea3734cf022b16c24cfd@oss.nttdata.com
M doc/src/sgml/ref/copy.sgml
M src/backend/commands/copy.c
M src/backend/commands/copyfrom.c
M src/bin/psql/tab-complete.c
M src/include/commands/copy.h
M src/test/regress/expected/copy2.out
M src/test/regress/sql/copy2.sql
Fix expression list handling in ATExecAttachPartition()
commit : babb3993dbe9c805c1d29fa275a5e8f4c2b40922
author : Amit Langote <amitlan@postgresql.org>
date : Thu, 3 Oct 2024 11:59:09 +0900
committer: Amit Langote <amitlan@postgresql.org>
date : Thu, 3 Oct 2024 11:59:09 +0900
This commit addresses two issues related to the manipulation of the
partition constraint expression list in ATExecAttachPartition().
First, the current use of list_concat() to combine the partition's
constraint (retrieved via get_qual_from_partbound()) with the parent
table’s partition constraint can lead to memory safety issues. After
calling list_concat(), the original constraint (partBoundConstraint)
might no longer be safe to access, as list_concat() may free or modify
it.
Second, there's a logical error in constructing the constraint for
validating against the default partition. The current approach
incorrectly includes a negated version of the parent table's partition
constraint, which is redundant, as it always evaluates to false for
rows in the default partition.
To resolve these issues, list_concat() is replaced with
list_concat_copy(), ensuring that partBoundConstraint remains unchanged
and can be safely reused when constructing the validation constraint
for the default partition.
This fix is not applied to back-branches, as there is no live bug and
the issue has not caused any reported problems in practice.
Nitin Jadhav posted a patch to address the memory safety issue, but I
decided to follow Alvaro Herrera's suggestion from the initial
discussion, as it allows us to fix both the memory safety and logical
issues.
Reported-by: Andres Freund <andres@anarazel.de>
Reported-by: Nitin Jadhav <nitinjadhavpostgres@gmail.com>
Reviewed-by: Junwang Zhao <zhjwpku@gmail.com>
Discussion: https://postgr.es/m/20231115165737.zeulb575cgrbqo74@awork3.anarazel.de
Discussion: https://postgr.es/m/CAMm1aWbmYHM3bqtjyMQ-a+4Ub=dgsb_2E3_up2cn=UGdHNrGTg@mail.gmail.com
M src/backend/commands/tablecmds.c
Remove support for unlogged on partitioned tables
commit : e2bab2d792044b55dd092bf1c2be0d570ccb9401
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 3 Oct 2024 10:55:02 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 3 Oct 2024 10:55:02 +0900
The following commands were allowed on partitioned tables, with
different effects:
1) ALTER TABLE SET [UN]LOGGED did not issue an error, and did not update
pg_class.relpersistence.
2) CREATE UNLOGGED TABLE was working with pg_class.relpersistence marked
as initially defined, but partitions did not inherit the UNLOGGED
property, which was confusing.
This commit causes the commands mentioned above to fail for partitioned
tables, instead.
pg_dump is tweaked so as partitioned tables marked as UNLOGGED ignore
the option when dumped from older server versions. pgbench needs a
tweak for --unlogged and --partitions=N to ignore the UNLOGGED option on
the partitioned tables created, its partitions still being unlogged.
Author: Michael Paquier
Reviewed-by: Nathan Bossart
Discussion: https://postgr.es/m/ZiiyGFTBNkqcMQi_@paquier.xyz
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_table.sgml
M src/backend/commands/tablecmds.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pgbench/pgbench.c
M src/test/regress/expected/create_table.out
M src/test/regress/sql/create_table.sql
Adjust json_manifest_per_file_callback API in one more place.
commit : 554d3a18f36264eeb1333655b8ddcd929befa6ec
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Oct 2024 20:27:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Oct 2024 20:27:45 -0400
Oversight in commit d94cf5ca7 (and in my testing of same).
Discussion: https://postgr.es/m/9468.1727895630@sss.pgh.pa.us
M src/backend/backup/basebackup_incremental.c
Parse libpq's "keepalives" option more like other integer options.
commit : 920d51979a99df29112396b2c75da7921ba7a7b0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Oct 2024 17:30:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Oct 2024 17:30:36 -0400
Use pqParseIntParam (nee parse_int_param) instead of using strtol
directly. This allows trailing whitespace, which the previous coding
didn't, and makes the spelling of the error message consistent with
other similar cases.
This seems to be an oversight in commit e7a221797, which introduced
parse_int_param. That fixed places that were using atoi(), but missed
this place which was randomly using strtol() instead.
Ordinarily I'd consider this minor cleanup not worth back-patching.
However, it seems that ecpg assumes it can add trailing whitespace
to URL parameters, so that use of the keepalives option fails in
that context. Perhaps that's worth improving as a separate matter.
In the meantime, back-patch this to all supported branches.
Yuto Sasaki (some further cleanup by me)
Discussion: https://postgr.es/m/TY2PR01MB36286A7B97B9A15793335D18C1772@TY2PR01MB3628.jpnprd01.prod.outlook.com
M src/interfaces/libpq/fe-connect.c
File size in a backup manifest should use uint64, not size_t.
commit : d94cf5ca7fad9cd81af5eac491bfbaf0facb9f6f
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 2 Oct 2024 09:59:04 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 2 Oct 2024 09:59:04 -0400
size_t is the size of an object in memory, not the size of a file on disk.
Thanks to Tom Lane for noting the error.
Discussion: http://postgr.es/m/1865585.1727803933@sss.pgh.pa.us
M src/bin/pg_combinebackup/load_manifest.c
M src/bin/pg_combinebackup/load_manifest.h
M src/bin/pg_combinebackup/write_manifest.c
M src/bin/pg_combinebackup/write_manifest.h
M src/bin/pg_verifybackup/astreamer_verify.c
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/bin/pg_verifybackup/pg_verifybackup.h
M src/common/parse_manifest.c
M src/include/common/parse_manifest.h
doc: Missing markup, punctuation and wordsmithing
commit : 7b2822ecf944a6aa429c05cc7f070001c3817934
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 2 Oct 2024 14:50:56 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 2 Oct 2024 14:50:56 +0200
Various improvements to the documentation like adding missing
markup, improving punctuation, ensuring consistent spelling of
words and minor wordsmithing.
Author: Oleg Sibiryakov <o.sibiryakov@postgrespro.ru>
Discussion: https://postgr.es/m/b7d0a03c-107e-48c7-a5c9-2c6f73cdf78f@postgrespro.ru
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/charset.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/copy.sgml
M doc/src/sgml/ref/create_subscription.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/sepgsql.sgml
M doc/src/sgml/syntax.sgml
M doc/src/sgml/system-views.sgml
M doc/src/sgml/trigger.sgml
M doc/src/sgml/user-manag.sgml
M doc/src/sgml/wal.sgml
Add fastpaths for when no objects are found
commit : 9c733951049bf3993c886d7f2c7459e7439a9793
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 2 Oct 2024 13:08:55 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 2 Oct 2024 13:08:55 +0200
If there are no objects found, there is no reason to inspect the
result columns and mallocing a zero-sized (which will be 1 byte
in reality) heap buffer for it. Add a fast-path for immediately
returning like how other object inspection functions are already
doing it.
Reviewed-by: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/C2F05B3C-1414-45DD-AE09-6FEE4D0F89BD@yesql.se
M src/bin/pg_dump/pg_dump.c
Remove superfluous PQExpBuffer resetting
commit : 1a123e3b136bf38ebf5f7a97b358306998f8a61f
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 2 Oct 2024 13:07:31 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 2 Oct 2024 13:07:31 +0200
Since the buffer was just created, there is no reason to immediately
reset it.
Reviewed-by: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/C2F05B3C-1414-45DD-AE09-6FEE4D0F89BD@yesql.se
M src/bin/pg_dump/pg_dump.c
doc: Add link to login event trigger example
commit : 94902b146fae024c03f80d496a0da5ba9751c9b5
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 2 Oct 2024 12:24:39 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 2 Oct 2024 12:24:39 +0200
The login event trigger is not listed on the trigger firing matrix
since it's not fired by a command. Add a link to the example code
page similar to how the other event triggers link to the matrix.
Reported-by: Marcos Pegoraro <marcos@f10.com.br>
Discussion: https://postgr.es/m/CAB-JLwYS+78rX02BZ3wJ9ykVrd2i3O1K+7jzvZKQ0evquyQiLQ@mail.gmail.com
M doc/src/sgml/event-trigger.sgml
Fix inconsistent reporting of checkpointer stats.
commit : 17cc5f666f6aada21eb3237974c50681ba4814ea
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 2 Oct 2024 11:17:47 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 2 Oct 2024 11:17:47 +0900
Previously, the pg_stat_checkpointer view and the checkpoint completion
log message could show different numbers for buffers written
during checkpoints. The view only counted shared buffers,
while the log message included both shared and SLRU buffers,
causing inconsistencies.
This commit resolves the issue by updating both the view and the log message
to separately report shared and SLRU buffers written during checkpoints.
A new slru_written column is added to the pg_stat_checkpointer view
to track SLRU buffers, while the existing buffers_written column now
tracks only shared buffers. This change would help users distinguish
between the two types of buffers, in the pg_stat_checkpointer view and
the checkpoint complete log message, respectively.
Bump catalog version.
Author: Nitin Jadhav
Reviewed-by: Bharath Rupireddy, Michael Paquier, Kyotaro Horiguchi, Robert Haas
Reviewed-by: Andres Freund, vignesh C, Fujii Masao
Discussion: https://postgr.es/m/CAMm1aWb18EpT0whJrjG+-nyhNouXET6ZUw0pNYYAe+NezpvsAA@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/access/transam/slru.c
M src/backend/access/transam/xlog.c
M src/backend/catalog/system_views.sql
M src/backend/utils/activity/pgstat_checkpointer.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/access/xlog.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
doc: Clarify name of files generated by pg_waldump --save-fullpage
commit : 506eede7111ae7c88bd3c21f653bbb65846bd4a5
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 2 Oct 2024 11:12:40 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 2 Oct 2024 11:12:40 +0900
The fork name is always separated with the block number by an underscore
in the names of the files generated, but the docs stuck them together
without a separator, which was confusing.
Author: Christoph Berg
Discussion: https://postgr.es/m/ZvxtSLiix9eceMRM@msg.df7cb.de
Backpatch-through: 16
M doc/src/sgml/ref/pg_waldump.sgml
Reject a copy EOF marker that has data ahead of it on the same line.
commit : da8a4c1666476648659dc5fb6fc01ba5cd5d16a1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Oct 2024 16:53:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Oct 2024 16:53:54 -0400
We have always documented that a copy EOF marker (\.) must appear
by itself on a line, and that is how psql interprets the rule.
However, the backend's actual COPY FROM logic only insists that
there not be data between the \. and the following newline.
Any data ahead of the \. is parsed as a final line of input.
It's hard to interpret this as anything but an ancient mistake
that we've faithfully carried forward. Continuing to allow it
is not cost-free, since it could mask client-side bugs that
unnecessarily backslash-escape periods (and thereby risk
accidentally creating an EOF marker). So, let's remove that
provision and throw error if the EOF marker isn't alone on its
line, matching what the documentation has said right along.
Adjust the relevant error messages to be clearer, too.
Discussion: https://postgr.es/m/ed659f37-a9dd-42a7-82b9-0da562cc4006@manitou-mail.org
M src/backend/commands/copyfromparse.c
M src/test/regress/expected/copy.out
M src/test/regress/sql/copy.sql
initdb: Add new option "--no-data-checksums"
commit : 983a588e0b864d5c016d5902217ba4b11fc82b4f
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Oct 2024 10:27:39 -0400
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Oct 2024 10:27:39 -0400
Right now this does nothing except override any earlier
--data-checksums option. But the idea is that --data-checksums could
become the default, and then this option would allow forcing it off
instead.
Author: Greg Sabino Mullane <greg@turnstep.com>
Discussion: https://www.postgresql.org/message-id/flat/CAKAnmmKwiMHik5AHmBEdf5vqzbOBbcwEPHo4-PioWeAbzwcTOQ@mail.gmail.com
M doc/src/sgml/ref/initdb.sgml
M src/bin/initdb/initdb.c
M src/bin/initdb/t/001_initdb.pl
Tweak docs to reduce possible impact of data checksums
commit : efd72a3d422b591bfec941d625df6fad820e021b
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Oct 2024 09:58:20 -0400
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Oct 2024 09:58:20 -0400
Author: Greg Sabino Mullane <greg@turnstep.com>
Discussion: https://www.postgresql.org/message-id/flat/CAKAnmmKwiMHik5AHmBEdf5vqzbOBbcwEPHo4-PioWeAbzwcTOQ@mail.gmail.com
M doc/src/sgml/ref/initdb.sgml
Use macro to define the number of enum values
commit : 10b721821d6d6e27e594549cf105476dc28514c8
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Oct 2024 09:30:24 -0400
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Oct 2024 09:30:24 -0400
Refactoring in the interest of code consistency, a follow-up to 2e068db56e31.
The argument against inserting a special enum value at the end of the enum
definition is that a switch statement might generate a compiler warning unless
it has a default clause.
Aleksander Alekseev, reviewed by Michael Paquier, Dean Rasheed, Peter Eisentraut
Discussion: https://postgr.es/m/CAJ7c6TMsiaV5urU_Pq6zJ2tXPDwk69-NKVh4AMN5XrRiM7N%2BGA%40mail.gmail.com
M contrib/pg_stat_statements/pg_stat_statements.c
M src/backend/postmaster/autovacuum.c
M src/bin/pg_dump/pg_backup.h
M src/include/storage/pmsignal.h
M src/include/storage/procsignal.h
Fix some pg_verifybackup issues reported by Coverity.
commit : fc1b2ce0ee9c9745c5c562b692e021344a3f719a
author : Robert Haas <rhaas@postgresql.org>
date : Tue, 1 Oct 2024 08:31:33 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Tue, 1 Oct 2024 08:31:33 -0400
Commit 8dfd3129027969fdd2d9d294220c867d2efd84aa introduced a few
problems. verify_tar_file() forgot to free a buffer; the leak can't
add up to anything material, but might as well fix it.
precheck_tar_backup_file() intended to return after reporting an
error but didn't actually do so. member_copy_control_data() could
try to copy zero bytes (and maybe Coverity thinks it can even be
trying to copy a negative number of bytes).
Per discussion with Tom Lane.
Discussion: http://postgr.es/m/1240823.1727629418@sss.pgh.pa.us
M src/bin/pg_verifybackup/astreamer_verify.c
M src/bin/pg_verifybackup/pg_verifybackup.c
Simplify checking for xlocale.h
commit : 9c2a6c5a5f4b94a93120009e623ae8bd153e6dbb
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Oct 2024 07:16:04 -0400
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Oct 2024 07:16:04 -0400
Instead of XXX_IN_XLOCALE_H for several features XXX, let's just
include <xlocale.h> if HAVE_XLOCALE_H. The reason for the extra
complication was apparently that some old glibc systems also had an
<xlocale.h>, and you weren't supposed to include it directly, but it's
gone now (as far as I can tell it was harmless to do so anyway).
Author: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
M config/c-library.m4
M configure
M configure.ac
M meson.build
M src/include/c.h
M src/include/pg_config.h.in
M src/include/utils/pg_locale.h
M src/interfaces/ecpg/ecpglib/ecpglib_extern.h
M src/port/chklocale.c
jit: Use opaque pointers in all supported LLVM versions.
commit : ee4859123e3d47aef8cfe078f7faee2ebcecb613
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Oct 2024 05:05:51 -0400
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Oct 2024 05:05:51 -0400
LLVM's opaque pointer change began in LLVM 14, but remained optional
until LLVM 16. When commit 37d5babb added opaque pointer support, we
didn't turn it on for LLVM 14 and 15 yet because we didn't want to risk
weird bitcode incompatibility problems in released branches of
PostgreSQL. (That might have been overly cautious, I don't know.)
Now that PostgreSQL 18 has dropped support for LLVM versions < 14, and
since it hasn't been released yet and no extensions or bitcode have been
built against it in the wild yet, we can be more aggressive. We can rip
out the support code and build system clutter that made opaque pointer
use optional.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussions: https://postgr.es/m/CA%2BhUKGLhNs5geZaVNj2EJ79Dx9W8fyWUU3HxcpZy55sMGcY%3DiA%40mail.gmail.com
M configure
M configure.ac
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/meson.build
M src/include/jit/llvmjit_emit.h
jit: Require at least LLVM 14, if enabled.
commit : 972c2cd2882b6dd7a2059d030d03e89dae47ede7
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Oct 2024 04:49:11 -0400
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 1 Oct 2024 04:49:11 -0400
Remove support for LLVM versions 10-13. The default on all non-EOL'd
OSes represented in our build farm will be at least LLVM 14 when
PostgreSQL 18 ships.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/CA%2BhUKGLhNs5geZaVNj2EJ79Dx9W8fyWUU3HxcpZy55sMGcY%3DiA%40mail.gmail.com
M config/llvm.m4
M configure
M doc/src/sgml/installation.sgml
M meson.build
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_error.cpp
M src/backend/jit/llvm/llvmjit_inline.cpp
M src/backend/jit/llvm/llvmjit_wrap.cpp
doc: Mention the connstring key word for PGSERVICE
commit : 1b4d52c3556b9abad10165ce7298062cf7eb7005
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 1 Oct 2024 10:20:14 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 1 Oct 2024 10:20:14 +0200
The documentation for the connection service file was mentioning
the environment variable early but not the connection string key
word until the last sentence and only then in an example. This
adds the keyword in the first paragraph to make it clearer
Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://postgr.es/m/87r09ibpke.fsf@wibble.ilmari.org
M doc/src/sgml/libpq.sgml
Fix race condition in COMMIT PREPARED causing orphaned 2PC files
commit : cf4401fe6cf56811343edcad29c96086c2c66481
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 1 Oct 2024 15:44:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 1 Oct 2024 15:44:03 +0900
COMMIT PREPARED removes on-disk 2PC files near its end, but the state
checked if a file is on-disk or not gets read from shared memory while
not holding the two-phase state lock.
Because of that, there was a small window where a second backend doing a
PREPARE TRANSACTION could reuse the GlobalTransaction put back into the
2PC free list by the COMMIT PREPARED, overwriting the "ondisk" flag read
afterwards by the COMMIT PREPARED to decide if its on-disk two-phase
state file should be removed, preventing the file deletion.
This commit fixes this issue so as the "ondisk" flag in the
GlobalTransaction is read while holding the two-phase state lock, not
from shared memory after its entry has been added to the free list.
Orphaned two-phase state files flushed to disk after a checkpoint are
discarded at the beginning of recovery. However, a truncation of
pg_xact/ would make the startup process issue a FATAL when it cannot
read the SLRU page holding the state of the transaction whose 2PC file
was orphaned, which is a necessary step to decide if the 2PC file should
be removed or not. Removing manually the file would be necessary in
this case.
Issue introduced by effe7d9552dd, so backpatch all the way down.
Mea culpa.
Author: wuchengwen
Discussion: https://postgr.es/m/tencent_A7F059B5136A359625C7B2E4A386B3C3F007@qq.com
Backpatch-through: 12
M src/backend/access/transam/twophase.c
Doc: replace unnecessary non-breaking space with ordinal space.
commit : 3b1a377defcfc01d8a3f964e39aa652766ffc188
author : Tatsuo Ishii <ishii@postgresql.org>
date : Tue, 1 Oct 2024 11:34:34 +0900
committer: Tatsuo Ishii <ishii@postgresql.org>
date : Tue, 1 Oct 2024 11:34:34 +0900
There were unnecessary non-breaking spaces (nbsp, U+00A0, 0xc2a0 in
UTF-8) in the docs. This commit replaces them with ASCII spaces
(0x20).
config.sgml is backpatched through 17.
ref/drop_extension.sgml is backpatched through 13.
Discussion: https://postgr.es/m/20240930.153404.202479334310259810.ishii%40postgresql.org
Reviewed-by: Yugo Nagata, Daniel Gustafsson
Backpatch-through: 17, 13
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/drop_extension.sgml
Expand assertion check for query ID reporting in executor
commit : 5deb56387e3a1d08e1e62bed031258db267a0ab5
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 1 Oct 2024 08:56:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 1 Oct 2024 08:56:21 +0900
As formulated, the assertion added in the executor by 24f520594809 to
check that a query ID is set had two problems:
- track_activities may be disabled while compute_query_id is enabled,
causing the query ID to not be reported to pg_stat_activity.
- debug_query_string may not be set in some context. The only path
where this would matter is visibly autovacuum, should parallel workers
be enabled there at some point. This is not the case currently.
There was no test showing the interactions between the query ID and
track_activities, so let's add one based on a scan of pg_stat_activity.
This assertion is still an experimentation at this stage, but let's see
if this shows more paths where query IDs are not properly set while they
should.
Discussion: https://postgr.es/m/Zvn5616oYXmpXyHI@paquier.xyz
M src/backend/executor/execMain.c
M src/test/regress/expected/guc.out
M src/test/regress/sql/guc.sql
Add missing command for pg_maintain in comment
commit : 102de3be73345a8de443355e055c10762b08cc4c
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 1 Oct 2024 00:01:32 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 1 Oct 2024 00:01:32 +0200
The comment in pg_class_aclmask_ext() which lists the allowed commands
for the pg_maintain role lacked LOCK TABLE.
Reported-by: Yusuke Sugie <btsugieyuusuke@oss.nttdata.com>
Reviewed-by: Yugo Nagata <nagata@sraoss.co.jp>
Discussion: https://postgr.es/m/034d3c60f5daba1919cd90f236b2e22d@oss.nttdata.com
M src/backend/catalog/aclchk.c
Do not treat \. as an EOF marker in CSV mode for COPY IN.
commit : 7702337489810f645b3501d99215c2b525c5abca
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Sep 2024 17:57:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Sep 2024 17:57:12 -0400
Since backslash is (typically) not special in CSV data, we should
not be treating \. as special either. The server historically did
this to keep CSV and TEXT modes more alike and to support V2 protocol;
but V2 protocol is long dead, and the inconsistency with CSV standards
is annoying. Remove that behavior in CopyReadLineText, and make some
minor consequent code simplifications.
On the client side, we need to fix psql so that it does not check
for \. except when reading data from STDIN (that is, the script
source). We must do that regardless of TEXT/CSV mode or there is
no way to end the COPY short of script EOF. Also, be careful
not to send the \. to the server in that case.
This is a small compatibility break in that other applications
beside psql may need similar adjustment. Also, using an older
version of psql with a v18 server may result in misbehavior
during CSV-mode COPY IN.
Daniel Vérité, reviewed by vignesh C, Robert Haas, and myself
Discussion: https://postgr.es/m/ed659f37-a9dd-42a7-82b9-0da562cc4006@manitou-mail.org
M doc/src/sgml/libpq.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/copy.sgml
M doc/src/sgml/ref/psql-ref.sgml
M src/backend/commands/copyfromparse.c
M src/backend/commands/copyto.c
M src/bin/psql/copy.c
M src/test/regress/expected/copy.out
M src/test/regress/sql/copy.sql
docs: Enhance the pg_stat_checkpointer view documentation.
commit : a19f83f87966f763991cc76404f8e42a36e7e842
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 1 Oct 2024 01:55:45 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 1 Oct 2024 01:55:45 +0900
This commit updates the documentation for the pg_stat_checkpointer view
to clarify what kind of checkpoints or restartpoints each counter tracks.
This makes it easier to understand the meaning of each counter.
Previously, the num_requested description included "backend,"
which could be misleading since requests come from other sources as well.
This commit also removes "backend" from the description of num_requested,
to avoid confusion.
Author: Fujii Masao
Reviewed-by: Anton A. Melnikov
Discussion: https://postgr.es/m/4640258e-d959-4cf0-903c-cd02389c3e05@oss.nttdata.com
M doc/src/sgml/monitoring.sgml
Remove incorrect entries in pg_walsummary's getopt_long call.
commit : 04c64e3fb35af090c249303b9df0a311f2a07593
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Sep 2024 12:06:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Sep 2024 12:06:54 -0400
For some reason this listed "-f" and "-w" as valid switches, though
the code doesn't implement any such thing nor do the docs mention
them. The effect of this was that if you tried to use one of these
switches, you'd get an unhelpful error message.
Yusuke Sugie
Discussion: https://postgr.es/m/68e72a2a70f4d84c1c7847b13bcdaef8@oss.nttdata.com
M src/bin/pg_walsummary/pg_walsummary.c
Don't disallow DROP of constraints ONLY on partitioned tables
commit : 4dea33ce765d65d8807d343ca6535a3d067a63da
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 30 Sep 2024 11:58:13 +0200
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 30 Sep 2024 11:58:13 +0200
This restriction seems to have come about due to some fuzzy thinking: in
commit 9139aa19423b we were adding a restriction against ADD constraint
ONLY on partitioned tables (which is sensible) and apparently we thought
the DROP case had to be symmetrical. However, it isn't, and the
comments about it are mistaken about the effect it would have. Remove
this limitation.
There have been no reports of users bothered by this limitation, so I'm
not backpatching it just yet. We can revisit this decision later, as needed.
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/202409261752.nbvlawkxsttf@alvherre.pgsql
Discussion: https://postgr.es/m/7682253a-6f79-6a92-00aa-267c4c412870@lab.ntt.co.jp
(about commit 9139aa19423b, previously not registered)
M doc/src/sgml/ref/alter_table.sgml
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
Bump catalog version for change in VariableSetStmt
commit : 4c7cd07aa62abc29e6885e95b5307e5e08bb3bf7
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Sep 2024 14:52:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Sep 2024 14:52:03 +0900
Oversight in dc68515968e8, as this breaks SQL functions with a SET
command.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/1364409.1727673407@sss.pgh.pa.us
M src/include/catalog/catversion.h
Show values of SET statements as constants in pg_stat_statements
commit : dc68515968e80d75f8106d0df05da346be537628
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Sep 2024 14:02:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Sep 2024 14:02:00 +0900
This is a continuation of work like 11c34b342bd7, done to reduce the
bloat of pg_stat_statements by applying more normalization to query
entries. This commit is able to detect and normalize values in
VariableSetStmt, resulting in:
SET conf_param = $1
Compared to other parse nodes, VariableSetStmt is embedded in much more
places in the parser, impacting many query patterns in
pg_stat_statements. A custom jumble function is used, with an extra
field in the node to decide if arguments should be included in the
jumbling or not, a location field being not enough for this purpose.
This approach allows for a finer tuning.
Clauses relying on one or more keywords are not normalized, for example:
* DEFAULT
* FROM CURRENT
* List of keywords. SET SESSION CHARACTERISTICS AS TRANSACTION,
where it is critical to differentiate different sets of options, is a
good example of why normalization should not happen.
Some queries use VariableSetStmt for some subclauses with SET, that also
have their values normalized:
- ALTER DATABASE
- ALTER ROLE
- ALTER SYSTEM
- CREATE/ALTER FUNCTION
ba90eac7a995 has added test coverage for most of the existing SET
patterns. The expected output of these tests shows the difference this
commit creates. Normalization could be perhaps applied to more portions
of the grammar but what is done here is conservative, and good enough as
a starting point.
Author: Greg Sabino Mullane, Michael Paquier
Discussion: https://postgr.es/m/36e5bffe-e989-194f-85c8-06e7bc88e6f7@amazon.com
Discussion: https://postgr.es/m/B44FA29D-EBD0-4DD9-ABC2-16F1CB087074@amazon.com
Discussion: https://postgr.es/m/CAKAnmmJtJY2jzQN91=2QAD2eAJAA-Per61eyO48-TyxEg-q0Rg@mail.gmail.com
M contrib/pg_stat_statements/expected/dml.out
M contrib/pg_stat_statements/expected/level_tracking.out
M contrib/pg_stat_statements/expected/utility.out
M contrib/pg_stat_statements/expected/wal.out
M src/backend/nodes/queryjumblefuncs.c
M src/backend/parser/gram.y
M src/include/nodes/parsenodes.h
Add num_done counter to the pg_stat_checkpointer view.
commit : 559efce1d684069acf234a5cb032acba84e70938
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 30 Sep 2024 11:56:05 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 30 Sep 2024 11:56:05 +0900
Checkpoints can be skipped when the server is idle. The existing num_timed and
num_requested counters in pg_stat_checkpointer track both completed and
skipped checkpoints, but there was no way to count only the completed ones.
This commit introduces the num_done counter, which tracks only completed
checkpoints, making it easier to see how many were actually performed.
Bump catalog version.
Author: Anton A. Melnikov
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/9ea77f40-818d-4841-9dee-158ac8f6e690@oss.nttdata.com
M doc/src/sgml/monitoring.sgml
M src/backend/access/transam/xlog.c
M src/backend/catalog/system_views.sql
M src/backend/postmaster/checkpointer.c
M src/backend/utils/activity/pgstat_checkpointer.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/access/xlog.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
reindexdb: Skip reindexing temporary tables and indexes.
commit : 20cfec896c6a20ca436f634b0ffa3582d7b9425c
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 30 Sep 2024 11:13:55 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 30 Sep 2024 11:13:55 +0900
Reindexing temp tables or indexes of other sessions is not allowed.
However, reindexdb in parallel mode previously listed them as
the objects to process, leading to failures.
This commit ensures reindexdb in parallel mode skips temporary tables
and indexes by adding a condition based on the relpersistence column
in pg_class to the object listing queries, preventing these issues.
Note that this commit does not affect reindexdb when temporary tables
or indexes are explicitly specified using the -t or -j options;
reindexdb in that case still does not skip them and can cause an error.
Back-patch to v13 where parallel mode was introduced in reindexdb.
Author: Fujii Masao
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/5f37ee56-14fb-44fe-9150-9eb97e10538b@oss.nttdata.com
M src/bin/scripts/reindexdb.c
Set query ID in parallel workers for vacuum, BRIN and btree
commit : 6fd5071909a2886c499871e61127f815fd9bb6a2
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Sep 2024 08:43:28 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Sep 2024 08:43:28 +0900
All these code paths use their own entry point when starting parallel
workers, but failed to set a query ID, even if they set a text query.
Hence, this data would be missed in pg_stat_activity for the worker
processes. The main entry point for parallel query processing,
ParallelQueryMain(), is already doing that by saving its query ID in a
dummy PlannedStmt, but not the others. The code is changed so as the
query ID of these queries is set in their shared state, and reported
back once the parallel workers start.
Some tests are added to show how the failures can happen for btree and
BRIN with a parallel build enforced, which are able to trigger a failure
in an assertion added by 24f520594809 in the recovery TAP test
027_stream_regress.pl where pg_stat_statements is always loaded. In
this case, the executor path was taken because the index expression
needs to be flattened when building its IndexInfo.
Alexander Lakhin has noticed the problem in btree, and I have noticed
that the issue was more spread. This is arguably a bug, but nobody has
complained about that until now, so no backpatch is done out of caution.
If folks would like to see a backpatch, well, let me know.
Reported-by: Alexander Lakhin
Reviewed-by: Sami Imseih
Discussion: https://postgr.es/m/cf3547c1-498a-6a61-7b01-819f902a251f@gmail.com
M src/backend/access/brin/brin.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/commands/vacuumparallel.c
M src/test/regress/expected/brin.out
M src/test/regress/expected/btree_index.out
M src/test/regress/sql/brin.sql
M src/test/regress/sql/btree_index.sql
Remove NULL dereference from RenameRelationInternal().
commit : 0d5a3d7574f8dabcbc229c2a06a9cb2d9a43c7c5
author : Noah Misch <noah@leadboat.com>
date : Sun, 29 Sep 2024 15:54:25 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 29 Sep 2024 15:54:25 -0700
Defect in last week's commit aac2c9b4fde889d13f859c233c2523345e72d32b,
per Coverity. Reaching this would need catalog corruption. Back-patch
to v12, like that commit.
M src/backend/commands/tablecmds.c
In passwordFromFile, don't leak the open file after stat failures.
commit : e9339782a631eeef01281bc7e1633dd6b970106e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Sep 2024 13:40:03 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Sep 2024 13:40:03 -0400
Oversight in e882bcae0. Per Coverity.
M src/interfaces/libpq/fe-connect.c
Avoid 037_invalid_database.pl hang under debug_discard_caches.
commit : c1ff2d8bc5be55e302731a16aaff563b7f03ed7c
author : Noah Misch <noah@leadboat.com>
date : Fri, 27 Sep 2024 15:28:56 -0700
committer: Noah Misch <noah@leadboat.com>
date : Fri, 27 Sep 2024 15:28:56 -0700
Back-patch to v12 (all supported versions).
M src/test/recovery/t/037_invalid_database.pl
doc: Note that CREATE MATERIALIZED VIEW restricts search_path.
commit : d8ebcac547d7c25c1378e0fc5810528f343ab8d5
author : Nathan Bossart <nathan@postgresql.org>
date : Fri, 27 Sep 2024 16:21:21 -0500
committer: Nathan Bossart <nathan@postgresql.org>
date : Fri, 27 Sep 2024 16:21:21 -0500