Stamp 14.0.
commit : 86a4dc1e6f29d1992a2afa3fac1a0b0a6e84568c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 27 Sep 2021 16:57:41 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 27 Sep 2021 16:57:41 -0400
M configure
M configure.ac
Translation updates
commit : 9f535e4aeafc4e427355760178ecc89055f96942
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 27 Sep 2021 09:22:27 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 27 Sep 2021 09:22:27 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 941ca560d0b36a8bace8432b06302ca003829f42
M src/bin/psql/po/de.po
Update list of acknowledgments in release notes
commit : 3cc51338438acb38ea14cebe6fd979f00ad5bebf
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 27 Sep 2021 08:59:03 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 27 Sep 2021 08:59:03 +0200
current through e8b39cebdaf042dfeeb31d2f48f0fe7b33886210
M doc/src/sgml/release-14.sgml
Fix typos in docs
commit : e8b39cebdaf042dfeeb31d2f48f0fe7b33886210
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 26 Sep 2021 19:18:23 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 26 Sep 2021 19:18:23 +0900
Author: Justin Pryzby
Discussion: https://postgr.es/m/20210924215827.GS831@telsasoft.com
Backpatch-through: 9.6
M doc/src/sgml/datatype.sgml
Doc: final(?) updates for 14.0 release notes.
commit : 765f677f364100072160e7af37288eb1df2ff355
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Sep 2021 11:36:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Sep 2021 11:36:43 -0400
Add the customary short list of major features. Set release date.
Discussion: https://postgr.es/m/1489855.1631986639@sss.pgh.pa.us
M doc/src/sgml/release-14.sgml
Doc: extend warnings about collation-mismatch hazards in postgres_fdw.
commit : 02c4e3533926f4e7398611b6349d7c438b86c63b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Sep 2021 10:53:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Sep 2021 10:53:54 -0400
Be a little more vocal about the risks of remote collations not
matching local ones. Actually fixing these risks seems hard,
and I've given up on the idea that it might be back-patchable.
So the best we can do for the back branches is add documentation.
Per discussion of bug #16583 from Jiří Fejfar.
Discussion: https://postgr.es/m/2438715.1632510693@sss.pgh.pa.us
M doc/src/sgml/postgres-fdw.sgml
Add alternative output for OpenSSL 3 without legacy loaded
commit : 6d0001aabf2a49180a236d9c2a7ecdf24e0cdb37
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sat, 25 Sep 2021 11:27:28 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sat, 25 Sep 2021 11:27:28 +0200
OpenSSL 3 introduced the concept of providers to support modularization,
and moved the outdated ciphers to the new legacy provider. In case it's
not loaded in the users openssl.cnf file there will be a lot of regress
test failures, so add alternative outputs covering those.
Also document the need to load the legacy provider in order to use older
ciphers with OpenSSL-enabled pgcrypto.
This will be backpatched to all supported version once there is sufficient
testing in the buildfarm of OpenSSL 3.
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/FEF81714-D479-4512-839B-C769D2605F8A@yesql.se
A contrib/pgcrypto/expected/blowfish_1.out
A contrib/pgcrypto/expected/cast5_1.out
A contrib/pgcrypto/expected/des_1.out
A contrib/pgcrypto/expected/pgp-decrypt_1.out
A contrib/pgcrypto/expected/pgp-pubkey-decrypt_1.out
M doc/src/sgml/pgcrypto.sgml
Disable OpenSSL EVP digest padding in pgcrypto
commit : 4fa2b15e1c9cae79afe17c14591074111b0d4093
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sat, 25 Sep 2021 11:27:20 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sat, 25 Sep 2021 11:27:20 +0200
The PX layer in pgcrypto is handling digest padding on its own uniformly
for all backend implementations. Starting with OpenSSL 3.0.0, DecryptUpdate
doesn't flush the last block in case padding is enabled so explicitly
disable it as we don't use it.
This will be backpatched to all supported version once there is sufficient
testing in the buildfarm of OpenSSL 3.
Reviewed-by: Peter Eisentraut, Michael Paquier
Discussion: https://postgr.es/m/FEF81714-D479-4512-839B-C769D2605F8A@yesql.se
M contrib/pgcrypto/openssl.c
doc: Improve description of index vacuuming with GUCs
commit : bfe1ead94488986008771c0d295c2725bab952cb
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 25 Sep 2021 15:11:52 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 25 Sep 2021 15:11:52 +0900
Index vacuums may happen multiple times depending on the number of dead
tuples stored, as of maintenance_work_mem for a manual VACUUM. For
autovacuum, this is controlled by autovacuum_work_mem instead, if set.
The documentation mentioned the former, but not the latter in the
context of autovacuum.
Reported-by: Nikolai Berkoff
Author: Laurenz Albe, Euler Taveira
Discussion: https://postgr.es/m/161545365522.10134.12195402324485546870@wrigleys.postgresql.org
Backpatch-through: 9.6
M doc/src/sgml/monitoring.sgml
doc: Add missing markup in CREATE EVENT TRIGGER page
commit : 3eea9139ee46124746e942293d57e3fb1d3c0e09
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 25 Sep 2021 14:48:09 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 25 Sep 2021 14:48:09 +0900
Reported-by: rir
Discussion: https://postgr.es/m/20210924183658.3syyitp3yuxjv2fp@localhost
Backpatch-through: 9.6
M doc/src/sgml/ref/create_event_trigger.sgml
Add missing $Test::Builder::Level settings
commit : e536a2683439c3dd4ea6d339a878fa4430e3174d
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 23 Sep 2021 22:49:20 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 23 Sep 2021 22:49:20 +0200
One of these was accidentally removed by c50624c. The others are
added by analogy.
Discussion: https://www.postgresql.org/message-id/ae1143fb-455c-c80f-ed66-78d45bd93303@enterprisedb.com
M src/test/authentication/t/001_password.pl
M src/test/authentication/t/002_saslprep.pl
M src/test/kerberos/t/001_auth.pl
M src/test/ldap/t/001_auth.pl
Split macros from visibilitymap.h into a separate header
commit : 7186f07189baf069c54718315b81e65d87f4c0c1
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 23 Sep 2021 19:59:03 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 23 Sep 2021 19:59:03 +0300
That allows to include just visibilitymapdefs.h from file.c, and in turn,
remove include of postgres.h from relcache.h.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/20210913232614.czafiubr435l6egi%40alap3.anarazel.de
Author: Alexander Korotkov
Reviewed-by: Andres Freund, Tom Lane, Alvaro Herrera
Backpatch-through: 13
M src/bin/pg_upgrade/file.c
M src/include/access/visibilitymap.h
A src/include/access/visibilitymapdefs.h
M src/include/utils/relcache.h
Release memory allocated by dependency_degree
commit : abb2f9144ba1b7ac806f3779f53ae2f6174cd2d9
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 21 Sep 2021 01:13:11 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 21 Sep 2021 01:13:11 +0200
Calculating degree of a functional dependency may allocate a lot of
memory - we have released mot of the explicitly allocated memory, but
e.g. detoasted varlena values were left behind. That may be an issue,
because we consider a lot of dependencies (all combinations), and the
detoasting may happen for each one again.
Fixed by calling dependency_degree() in a dedicated context, and
resetting it after each call. We only need the calculated dependency
degree, so we don't need to copy anything.
Backpatch to PostgreSQL 10, where extended statistics were introduced.
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com
M src/backend/statistics/dependencies.c
Free memory after building each statistics object
commit : bb7628e55eda6f450f0f824d11c9a34b11f6bb12
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 21 Sep 2021 01:14:11 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 21 Sep 2021 01:14:11 +0200
Until now, all extended statistics on a given relation were built in the
same memory context, without resetting. Some of the memory was released
explicitly, but not all of it - for example memory allocated while
detoasting values is hard to free. This is how it worked since extended
statistics were introduced in PostgreSQL 10, but adding support for
extended stats on expressions made the issue somewhat worse as it
increases the number of statistics to build.
Fixed by adding a memory context which gets reset after building each
statistics object (all the statistics kinds included in it). Resetting
it after building each statistics kind would be even better, but it
would require more invasive changes and copying of results, making it
harder to backpatch.
Backpatch to PostgreSQL 10, where extended statistics were introduced.
Author: Justin Pryzby
Reported-by: Justin Pryzby
Reviewed-by: Tomas Vondra
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com
M src/backend/statistics/extended_stats.c
Invalidate all partitions for a partitioned table in publication.
commit : 9eff8593265929d3a1fcdee375bd0a801c12b367
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 22 Sep 2021 08:13:37 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 22 Sep 2021 08:13:37 +0530
Updates/Deletes on a partition were allowed even without replica identity
after the parent table was added to a publication. This would later lead
to an error on subscribers. The reason was that we were not invalidating
the partition's relcache and the publication information for partitions
was not getting rebuilt. Similarly, we were not invalidating the
partitions' relcache after dropping a partitioned table from a publication
which will prohibit Updates/Deletes on its partition without replica
identity even without any publication.
Reported-by: Haiying Tang
Author: Hou Zhijie and Vignesh C
Reviewed-by: Vignesh C and Amit Kapila
Backpatch-through: 13
Discussion: https://postgr.es/m/OS0PR01MB6113D77F583C922F1CEAA1C3FBD29@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/backend/catalog/pg_publication.c
M src/backend/commands/publicationcmds.c
M src/include/catalog/pg_publication.h
M src/include/commands/publicationcmds.h
M src/test/regress/expected/publication.out
M src/test/regress/sql/publication.sql
Fix "single value strategy" index deletion issue.
commit : e665129c4727004e7a7c12c86d077abc750b3307
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 21 Sep 2021 18:57:31 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 21 Sep 2021 18:57:31 -0700
It is not appropriate for deduplication to apply single value strategy
when triggered by a bottom-up index deletion pass. This wastes cycles
because later bottom-up deletion passes will overinterpret older
duplicate tuples that deduplication actually just skipped over "by
design". It also makes bottom-up deletion much less effective for low
cardinality indexes that happen to cross a meaningless "index has single
key value per leaf page" threshold.
To fix, slightly narrow the conditions under which deduplication's
single value strategy is considered. We already avoided the strategy
for a unique index, since our high level goal must just be to buy time
for VACUUM to run (not to buy space). We'll now also avoid it when we
just had a bottom-up pass that reported failure. The two cases share
the same high level goal, and already overlapped significantly, so this
approach is quite natural.
Oversight in commit d168b666, which added bottom-up index deletion.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WznaOvM+Gyj-JQ0X=JxoMDxctDTYjiEuETdAGbF5EUc3MA@mail.gmail.com
Backpatch: 14-, where bottom-up deletion was introduced.
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtinsert.c
M src/include/access/nbtree.h
Fix places in TestLib.pm in need of adaptation to the output of Msys perl
commit : 90251ff199858844fa450ba9614092c06c67fc4f
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Sep 2021 08:43:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Sep 2021 08:43:00 +0900
Contrary to the output of native perl, Msys perl generates outputs with
CRLFs characters. There are already places in the TAP code where CRLFs
(\r\n) are automatically converted to LF (\n) on Msys, but we missed a
couple of places when running commands and using their output for
comparison, that would lead to failures.
This problem has been found thanks to the test added in 5adb067 using
TestLib::command_checks_all(), but after a closer look more code paths
were missing a filter.
This is backpatched all the way down to prevent any surprises if a new
test is introduced in stable branches.
Reviewed-by: Andrew Dunstan, Álvaro Herrera
Discussion: https://postgr.es/m/1252480.1631829409@sss.pgh.pa.us
Backpatch-through: 9.6
M src/test/perl/TestLib.pm
Fix misevaluation of STABLE parameters in CALL within plpgsql.
commit : 2ad5f963e1306aa6f4cf96076a230e96529d2237
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 21 Sep 2021 19:06:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 21 Sep 2021 19:06:33 -0400
Before commit 84f5c2908, a STABLE function in a plpgsql CALL
statement's argument list would see an up-to-date snapshot,
because exec_stmt_call would push a new snapshot. I got rid of
that because the possibility of the snapshot disappearing within
COMMIT made it too hard to manage a snapshot across the CALL
statement. That's fine so far as the procedure itself goes,
but I forgot to think about the possibility of STABLE functions
within the CALL argument list. As things now stand, those'll
be executed with the Portal's snapshot as ActiveSnapshot,
keeping them from seeing updates more recent than Portal startup.
(VOLATILE functions don't have a problem because they take their
own snapshots; which indeed is also why the procedure itself
doesn't have a problem. There are no STABLE procedures.)
We can fix this by pushing a new snapshot transiently within
ExecuteCallStmt itself. Popping the snapshot before we get
into the procedure proper eliminates the management problem.
The possibly-useless extra snapshot-grab is slightly annoying,
but it's no worse than what happened before 84f5c2908.
Per bug #17199 from Alexander Nawratil. Back-patch to v11,
like the previous patch.
Discussion: https://postgr.es/m/17199-1ab2561f0d94af92@postgresql.org
M src/backend/commands/functioncmds.c
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/sql/plpgsql_transaction.sql
Document XLOG_INCLUDE_XID a little better
commit : c1d1ae1db23796e4d1b04f5c087944722cf1446a
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Sep 2021 19:47:53 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Sep 2021 19:47:53 -0300
I noticed that commit 0bead9af484c left this flag undocumented in
XLogSetRecordFlags, which led me to discover that the flag doesn't
actually do what the one comment on it said it does. Improve the
situation by adding some more comments.
Backpatch to 14, where the aforementioned commit appears.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/202109212119.c3nhfp64t2ql@alvherre.pgsql
M src/backend/access/transam/xloginsert.c
M src/include/access/xlog.h
M src/include/access/xlogrecord.h
Stamp 14rc1.
commit : 8aed7f7a2ec46ad60c84c3c97300210850c3fbc8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Sep 2021 17:33:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Sep 2021 17:33:01 -0400
M configure
M configure.ac
Remove overzealous index deletion assertion.
commit : 955a6a893498a5d3af544d9c0b1c292b19ec1690
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 20 Sep 2021 14:26:24 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 20 Sep 2021 14:26:24 -0700
A broken HOT chain is not an unexpected condition, even when the offset
number points past the end of the page's line pointer array.
heap_prune_chain() does not (and never has) treated this condition as
unexpected, so derivative code in heap_index_delete_tuples() shouldn't
do so either.
Oversight in commit 4228817449.
The assertion can probably only fail on Postgres 14 and master. Earlier
releases don't have commit 3c3b8a4b, which taught VACUUM to truncate the
line pointer array of heap pages. Backpatch all the same, just to be
consistent.
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/17197-9438f31f46705182@postgresql.org
Backpatch: 12-, just like commit 4228817449.
M src/backend/access/heap/heapam.c
Translation updates
commit : 3e8525aab86e78593844c9b395be40938582ebaf
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Sep 2021 16:23:13 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Sep 2021 16:23:13 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 10b675b81a3a04bac460cb049e0b7b6e17fb4795
M src/backend/nls.mk
M src/backend/po/de.po
M src/backend/po/fr.po
D src/backend/po/id.po
M src/backend/po/ja.po
D src/backend/po/pl.po
D src/backend/po/pt_BR.po
M src/backend/po/ru.po
D src/backend/po/tr.po
M src/backend/po/uk.po
M src/bin/initdb/nls.mk
M src/bin/initdb/po/el.po
M src/bin/initdb/po/fr.po
D src/bin/initdb/po/he.po
D src/bin/initdb/po/it.po
M src/bin/initdb/po/ja.po
D src/bin/initdb/po/pl.po
D src/bin/initdb/po/pt_BR.po
M src/bin/initdb/po/ru.po
M src/bin/initdb/po/tr.po
M src/bin/initdb/po/uk.po
D src/bin/initdb/po/vi.po
M src/bin/initdb/po/zh_CN.po
M src/bin/pg_amcheck/nls.mk
M src/bin/pg_amcheck/po/de.po
M src/bin/pg_amcheck/po/el.po
M src/bin/pg_amcheck/po/fr.po
M src/bin/pg_amcheck/po/ja.po
A src/bin/pg_amcheck/po/ru.po
M src/bin/pg_amcheck/po/uk.po
M src/bin/pg_amcheck/po/zh_CN.po
M src/bin/pg_archivecleanup/nls.mk
M src/bin/pg_archivecleanup/po/el.po
M src/bin/pg_archivecleanup/po/ja.po
D src/bin/pg_archivecleanup/po/pl.po
M src/bin/pg_archivecleanup/po/ru.po
M src/bin/pg_archivecleanup/po/tr.po
M src/bin/pg_archivecleanup/po/uk.po
D src/bin/pg_archivecleanup/po/vi.po
M src/bin/pg_archivecleanup/po/zh_CN.po
M src/bin/pg_basebackup/nls.mk
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_basebackup/po/fr.po
D src/bin/pg_basebackup/po/he.po
D src/bin/pg_basebackup/po/it.po
M src/bin/pg_basebackup/po/ja.po
D src/bin/pg_basebackup/po/pl.po
D src/bin/pg_basebackup/po/pt_BR.po
M src/bin/pg_basebackup/po/ru.po
M src/bin/pg_basebackup/po/uk.po
D src/bin/pg_basebackup/po/vi.po
M src/bin/pg_basebackup/po/zh_CN.po
M src/bin/pg_checksums/po/el.po
M src/bin/pg_checksums/po/ja.po
M src/bin/pg_checksums/po/ru.po
M src/bin/pg_checksums/po/uk.po
M src/bin/pg_checksums/po/zh_CN.po
M src/bin/pg_config/nls.mk
M src/bin/pg_config/po/el.po
M src/bin/pg_config/po/fr.po
M src/bin/pg_config/po/ja.po
D src/bin/pg_config/po/nb.po
D src/bin/pg_config/po/ro.po
M src/bin/pg_config/po/ru.po
D src/bin/pg_config/po/ta.po
M src/bin/pg_config/po/tr.po
M src/bin/pg_config/po/uk.po
M src/bin/pg_config/po/zh_CN.po
D src/bin/pg_config/po/zh_TW.po
M src/bin/pg_controldata/nls.mk
M src/bin/pg_controldata/po/el.po
M src/bin/pg_controldata/po/fr.po
M src/bin/pg_controldata/po/ja.po
D src/bin/pg_controldata/po/pl.po
D src/bin/pg_controldata/po/pt_BR.po
M src/bin/pg_controldata/po/ru.po
M src/bin/pg_controldata/po/uk.po
D src/bin/pg_controldata/po/vi.po
M src/bin/pg_controldata/po/zh_CN.po
M src/bin/pg_ctl/nls.mk
M src/bin/pg_ctl/po/cs.po
M src/bin/pg_ctl/po/el.po
M src/bin/pg_ctl/po/fr.po
D src/bin/pg_ctl/po/he.po
M src/bin/pg_ctl/po/ja.po
D src/bin/pg_ctl/po/pl.po
D src/bin/pg_ctl/po/pt_BR.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_ctl/po/uk.po
M src/bin/pg_ctl/po/zh_CN.po
M src/bin/pg_dump/nls.mk
M src/bin/pg_dump/po/cs.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_dump/po/el.po
M src/bin/pg_dump/po/fr.po
D src/bin/pg_dump/po/he.po
D src/bin/pg_dump/po/it.po
M src/bin/pg_dump/po/ja.po
D src/bin/pg_dump/po/pl.po
D src/bin/pg_dump/po/pt_BR.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/uk.po
M src/bin/pg_dump/po/zh_CN.po
M src/bin/pg_resetwal/nls.mk
M src/bin/pg_resetwal/po/el.po
M src/bin/pg_resetwal/po/fr.po
D src/bin/pg_resetwal/po/it.po
M src/bin/pg_resetwal/po/ja.po
D src/bin/pg_resetwal/po/pl.po
D src/bin/pg_resetwal/po/pt_BR.po
M src/bin/pg_resetwal/po/ru.po
M src/bin/pg_resetwal/po/uk.po
M src/bin/pg_resetwal/po/zh_CN.po
M src/bin/pg_rewind/nls.mk
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/el.po
M src/bin/pg_rewind/po/fr.po
D src/bin/pg_rewind/po/it.po
M src/bin/pg_rewind/po/ja.po
D src/bin/pg_rewind/po/ko.po
D src/bin/pg_rewind/po/pl.po
D src/bin/pg_rewind/po/pt_BR.po
M src/bin/pg_rewind/po/ru.po
D src/bin/pg_rewind/po/tr.po
M src/bin/pg_rewind/po/uk.po
M src/bin/pg_rewind/po/zh_CN.po
M src/bin/pg_test_fsync/po/el.po
M src/bin/pg_test_fsync/po/ja.po
M src/bin/pg_test_fsync/po/ru.po
M src/bin/pg_test_fsync/po/uk.po
M src/bin/pg_test_fsync/po/zh_CN.po
M src/bin/pg_test_timing/nls.mk
D src/bin/pg_test_timing/po/cs.po
M src/bin/pg_test_timing/po/el.po
M src/bin/pg_test_timing/po/ja.po
D src/bin/pg_test_timing/po/ko.po
D src/bin/pg_test_timing/po/pl.po
M src/bin/pg_test_timing/po/ru.po
D src/bin/pg_test_timing/po/sv.po
D src/bin/pg_test_timing/po/tr.po
M src/bin/pg_test_timing/po/uk.po
D src/bin/pg_test_timing/po/vi.po
M src/bin/pg_test_timing/po/zh_CN.po
M src/bin/pg_upgrade/po/cs.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/fr.po
M src/bin/pg_upgrade/po/ja.po
M src/bin/pg_upgrade/po/ru.po
M src/bin/pg_upgrade/po/tr.po
M src/bin/pg_upgrade/po/uk.po
M src/bin/pg_upgrade/po/zh_CN.po
M src/bin/pg_verifybackup/po/el.po
M src/bin/pg_verifybackup/po/fr.po
M src/bin/pg_verifybackup/po/ja.po
M src/bin/pg_verifybackup/po/ru.po
M src/bin/pg_verifybackup/po/uk.po
M src/bin/pg_verifybackup/po/zh_CN.po
M src/bin/pg_waldump/nls.mk
M src/bin/pg_waldump/po/cs.po
M src/bin/pg_waldump/po/el.po
M src/bin/pg_waldump/po/es.po
M src/bin/pg_waldump/po/ja.po
M src/bin/pg_waldump/po/ru.po
M src/bin/pg_waldump/po/tr.po
M src/bin/pg_waldump/po/uk.po
D src/bin/pg_waldump/po/vi.po
M src/bin/pg_waldump/po/zh_CN.po
M src/bin/psql/nls.mk
M src/bin/psql/po/de.po
M src/bin/psql/po/el.po
M src/bin/psql/po/fr.po
D src/bin/psql/po/he.po
M src/bin/psql/po/ja.po
D src/bin/psql/po/pl.po
D src/bin/psql/po/pt_BR.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/uk.po
M src/bin/psql/po/zh_CN.po
D src/bin/psql/po/zh_TW.po
M src/bin/scripts/nls.mk
M src/bin/scripts/po/cs.po
M src/bin/scripts/po/de.po
A src/bin/scripts/po/el.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/fr.po
D src/bin/scripts/po/he.po
D src/bin/scripts/po/it.po
M src/bin/scripts/po/ja.po
D src/bin/scripts/po/pl.po
D src/bin/scripts/po/pt_BR.po
M src/bin/scripts/po/ru.po
M src/bin/scripts/po/tr.po
M src/bin/scripts/po/uk.po
M src/bin/scripts/po/zh_CN.po
M src/interfaces/ecpg/ecpglib/nls.mk
A src/interfaces/ecpg/ecpglib/po/el.po
M src/interfaces/ecpg/ecpglib/po/ja.po
M src/interfaces/ecpg/ecpglib/po/ru.po
M src/interfaces/ecpg/ecpglib/po/uk.po
M src/interfaces/ecpg/ecpglib/po/zh_CN.po
M src/interfaces/ecpg/preproc/nls.mk
M src/interfaces/ecpg/preproc/po/cs.po
M src/interfaces/ecpg/preproc/po/de.po
A src/interfaces/ecpg/preproc/po/el.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/fr.po
M src/interfaces/ecpg/preproc/po/ja.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/ecpg/preproc/po/tr.po
M src/interfaces/ecpg/preproc/po/uk.po
M src/interfaces/ecpg/preproc/po/zh_CN.po
M src/interfaces/libpq/nls.mk
M src/interfaces/libpq/po/el.po
M src/interfaces/libpq/po/fr.po
D src/interfaces/libpq/po/he.po
D src/interfaces/libpq/po/it.po
M src/interfaces/libpq/po/ja.po
D src/interfaces/libpq/po/pl.po
D src/interfaces/libpq/po/pt_BR.po
M src/interfaces/libpq/po/ru.po
D src/interfaces/libpq/po/tr.po
M src/interfaces/libpq/po/uk.po
M src/interfaces/libpq/po/zh_CN.po
D src/interfaces/libpq/po/zh_TW.po
M src/pl/plperl/nls.mk
M src/pl/plperl/po/el.po
M src/pl/plperl/po/fr.po
M src/pl/plperl/po/ja.po
M src/pl/plperl/po/ru.po
M src/pl/plperl/po/uk.po
M src/pl/plperl/po/zh_CN.po
D src/pl/plperl/po/zh_TW.po
M src/pl/plpgsql/src/nls.mk
M src/pl/plpgsql/src/po/de.po
A src/pl/plpgsql/src/po/el.po
M src/pl/plpgsql/src/po/fr.po
M src/pl/plpgsql/src/po/ja.po
D src/pl/plpgsql/src/po/ro.po
M src/pl/plpgsql/src/po/ru.po
M src/pl/plpgsql/src/po/uk.po
M src/pl/plpgsql/src/po/zh_CN.po
D src/pl/plpgsql/src/po/zh_TW.po
M src/pl/plpython/po/el.po
M src/pl/plpython/po/fr.po
M src/pl/plpython/po/ja.po
M src/pl/plpython/po/ru.po
M src/pl/plpython/po/uk.po
M src/pl/plpython/po/zh_CN.po
M src/pl/tcl/nls.mk
M src/pl/tcl/po/el.po
M src/pl/tcl/po/fr.po
M src/pl/tcl/po/ja.po
D src/pl/tcl/po/pt_BR.po
D src/pl/tcl/po/ro.po
M src/pl/tcl/po/ru.po
M src/pl/tcl/po/uk.po
M src/pl/tcl/po/zh_CN.po
D src/pl/tcl/po/zh_TW.po
doc: Replace characters that the PDF build cannot handle
commit : 89a967e9b5262db1cea62a5725dd7bd7489d240d
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Sep 2021 10:05:46 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Sep 2021 10:05:46 +0200
A few characters in the acknowledgments list cannot be handled by the
PDF build, so replace with a similar ASCII character.
M doc/src/sgml/release-14.sgml
Update list of acknowledgments in release notes
commit : 3609b453cd655060562f8750ce0ce4bdddf43a57
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Sep 2021 09:18:17 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 20 Sep 2021 09:18:17 +0200
current through 66061077155d68463ec00604ba7d6f0ae69716e8
M doc/src/sgml/release-14.sgml
Disallow extended statistics on system columns
commit : 66061077155d68463ec00604ba7d6f0ae69716e8
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 20 Sep 2021 00:34:57 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 20 Sep 2021 00:34:57 +0200
Since introduction of extended statistics, we've disallowed references
to system columns. So for example
CREATE STATISTICS s ON ctid FROM t;
would fail. But with extended statistics on expressions, it was possible
to work around this limitation quite easily
CREATE STATISTICS s ON (ctid::text) FROM t;
This is an oversight in a4d75c86bf, fixed by adding a simple check.
Backpatch to PostgreSQL 14, where support for extended statistics on
expressions was introduced.
Backpatch-through: 14
Discussion: https://postgr.es/m/20210816013255.GS10479%40telsasoft.com
M src/backend/commands/statscmds.c
Doc: further tweaking of v14 release notes.
commit : 2a9a34c3100d8c0311c63baaa435be38a7d7aae5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 19 Sep 2021 12:10:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 19 Sep 2021 12:10:34 -0400
A recent question reminded me that the notes' description of
commit 86dc90056 rather undersold its benefits.
Discussion: https://postgr.es/m/4a3115d4-0fb2-e214-93e3-9a9d0974b883@deepbluecap.com
M doc/src/sgml/release-14.sgml
Doc: fix typos.
commit : 55abb37506e9af4bbc4d8410ac8469b9d38e84e7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 19 Sep 2021 11:36:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 19 Sep 2021 11:36:53 -0400
"PGcon" should be "PGconn". Noted by D. Frey.
Discussion: https://postgr.es/m/163191739352.4680.16994248583642672629@wrigleys.postgresql.org
M doc/src/sgml/lobj.sgml
Doc: copy-editing for v14 release notes.
commit : f8dacf125db42547579629be1936fa613bad8070
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 18 Sep 2021 17:09:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 18 Sep 2021 17:09:46 -0400
Improve various item descriptions. Rearrange some things into
(IMO) more logical order. Fix missing markup and dubious
choices of link destinations. Drop a couple of items that
were later back-patched or otherwise don't seem to need
to be documented here.
M doc/src/sgml/release-14.sgml
Doc: update v14 release notes through today.
commit : 68b6ed42e69f920c67902e33d4a648bd39e0a19b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 18 Sep 2021 13:46:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 18 Sep 2021 13:46:07 -0400
Account for recent commits, notably reversion of 0827e8af7.
Strip trailing spaces.
M doc/src/sgml/release-14.sgml
pageinspect: Make page deletion elog less chatty.
commit : 55934416d2c50cdc9fba044a97fb14da1c779589
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 17 Sep 2021 14:19:50 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 17 Sep 2021 14:19:50 -0700
An elog that reports the value of a transaction ID stored on a deleted
nbtree page was added by commit e5d8a999, which taught page deletion to
store full 64-bit XIDs. It seems very chatty on further reflection, so
lower its elevel from NOTICE to DEBUG2.
Author: Peter Geoghegan <pg@bowt.ie>
Backpatch: 14-, just like the nbtree XID enhancement.
M contrib/pageinspect/btreefuncs.c
Fix pull_varnos to cope with translated PlaceHolderVars.
commit : 4d5b4483db1c6370861ca968edd753ab4dc03f9d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Sep 2021 15:41:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Sep 2021 15:41:16 -0400
Commit 55dc86eca changed pull_varnos to use (if possible) the associated
ph_eval_at for a PlaceHolderVar. I missed a fine point though: we might
be looking at a PHV in the quals or tlist of a child appendrel, in which
case we need to compute a ph_eval_at value that's been translated in the
same way that the PHV itself has been (cf. adjust_appendrel_attrs).
Fortunately, enough info is available in the PlaceHolderInfo to make
such translation possible without additional outside data, so we don't
need another round of uglification of planner APIs. This is a little
bit complicated, but since it's a hard-to-hit corner case, I'm not much
worried about adding cycles here.
Per report from Jaime Casanova. Back-patch to v12, like the previous
commit.
Discussion: https://postgr.es/m/20210915230959.GB17635@ahch-to
M src/backend/optimizer/util/var.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Fix EXPLAIN to handle SEARCH BREADTH FIRST queries.
commit : 388726753b638fb9938883bdd057b2ffe6f950f5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 16 Sep 2021 10:45:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 16 Sep 2021 10:45:42 -0400
The rewriter transformation for SEARCH BREADTH FIRST produces a
FieldSelect on a Var of type RECORD, where the Var references the
recursive union's worktable output. EXPLAIN VERBOSE failed to handle
this case, because it only expected such Vars to appear in CteScans
not WorkTableScans. Fix that, and add some test cases exercising
EXPLAIN on SEARCH and CYCLE queries.
In principle this oversight is an old bug, but it seems that the
case is unreachable without SEARCH BREADTH FIRST, because the
parser fails when attempting to create such a reference manually.
So for today I'll just patch HEAD/v14. Someday we might find that
the code portion of this patch needs to be back-patched further.
Per report from Atsushi Torikoshi.
Discussion: https://postgr.es/m/5bafa66ad529e11860339565c9e7c166@oss.nttdata.com
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Message style improvements
commit : f46dc96fcc652d5c29343e0eb4fa8e86a9468c36
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 16 Sep 2021 14:48:52 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 16 Sep 2021 14:48:52 +0200
M src/backend/commands/typecmds.c
M src/backend/libpq/auth.c
M src/backend/libpq/be-secure-openssl.c
M src/backend/utils/adt/jsonbsubs.c
M src/backend/utils/adt/multirangetypes.c
M src/test/regress/expected/jsonb.out
Fix performance regression from session statistics.
commit : 7890a423470b3a763aa22faf69a44cb6a2df8f0e
author : Andres Freund <andres@anarazel.de>
date : Thu, 16 Sep 2021 02:02:40 -0700
committer: Andres Freund <andres@anarazel.de>
date : Thu, 16 Sep 2021 02:02:40 -0700
Session statistics, as introduced by 960869da08, had several shortcomings:
- an additional GetCurrentTimestamp() call that also impaired the accuracy of
the data collected
This can be avoided by passing the current timestamp we already have in
pgstat_report_stat().
- an additional statistics UDP packet sent every 500ms
This is solved by adding the new statistics to PgStat_MsgTabstat.
This is conceptually ugly, because session statistics are not
table statistics. But the struct already contains data unrelated
to tables, so there is not much damage done.
Connection and disconnection are reported in separate messages, which
reduces the number of additional messages to two messages per session and a
slight increase in PgStat_MsgTabstat size (but the same number of table
stats fit).
- Session time computation could overflow on systems where long is 32 bit.
Reported-By: Andres Freund <andres@anarazel.de>
Author: Andres Freund <andres@anarazel.de>
Author: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/20210801205501.nyxzxoelqoo4x2qc%40alap3.anarazel.de
Backpatch: 14-, where the feature was introduced.
M src/backend/postmaster/pgstat.c
M src/backend/tcop/postgres.c
M src/backend/utils/activity/backend_status.c
M src/include/pgstat.h
M src/tools/pgindent/typedefs.list
Fix variable shadowing in procarray.c.
commit : 92a8d7610ea0f440a80328ced342b782180a3f43
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 16 Sep 2021 13:06:21 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 16 Sep 2021 13:06:21 +0900
ProcArrayGroupClearXid function has a parameter named "proc",
but the same name was used for its local variables. This commit fixes
this variable shadowing, to improve code readability.
Back-patch to all supported versions, to make future back-patching
easy though this patch is classified as refactoring only.
Reported-by: Ranier Vilela
Author: Ranier Vilela, Aleksander Alekseev
https://postgr.es/m/CAEudQAqyoTZC670xWi6w-Oe2_Bk1bfu2JzXz6xRfiOUzm7xbyQ@mail.gmail.com
M src/backend/storage/ipc/procarray.c
Use int instead of size_t in procarray.c.
commit : fe8821ca7da4bc1f72688cece0b7f2c3076d813d
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 16 Sep 2021 12:52:30 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 16 Sep 2021 12:52:30 +0900
All size_t variables declared in procarray.c are actually int ones.
Let's use int instead of size_t for those variables. Which would
reduce Wsign-compare compiler warnings.
Back-patch to v14 where commit 941697c3c1 added size_t variables
in procarray.c, to make future back-patching easy though
this patch is classified as refactoring only.
Reported-by: Ranier Vilela
Author: Ranier Vilela, Aleksander Alekseev
https://postgr.es/m/CAEudQAqyoTZC670xWi6w-Oe2_Bk1bfu2JzXz6xRfiOUzm7xbyQ@mail.gmail.com
M src/backend/storage/ipc/procarray.c
Disallow LISTEN in background workers.
commit : d84d62b6225b4af86cd9837b71f6fd2cf33fe80c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 15 Sep 2021 12:31:56 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 15 Sep 2021 12:31:56 -0400
It's possible to execute user-defined SQL in some background processes;
for example, logical replication workers can fire triggers. This opens
the possibility that someone would try to execute LISTEN in such a
context. But since only regular backends ever call
ProcessNotifyInterrupt, no messages would actually be received, and
thus the registered listener would simply prevent the message queue
from being cleaned. Eventually NOTIFY would stop working, which is bad.
Perhaps someday somebody will invent infrastructure to make listening
in a background worker actually useful. In the meantime, forbid it.
Back-patch to v13, which is where we introduced the MyBackendType
variable. It'd be a lot harder to implement the check without that,
and it doesn't seem worth the trouble.
Discussion: https://postgr.es/m/153243441449.1404.2274116228506175596@wrigleys.postgresql.org
M src/backend/tcop/utility.c
Fix hash_array
commit : 9b2fd490577bc957429f337cfd72869eb8ef08c9
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Sep 2021 11:59:34 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Sep 2021 11:59:34 +0200
Commit 054adca641ac1279dc8d9b74fda41948ac35e9a9 neglected to
initialize the type_id field of the synthesized type cache entry, so
it would make a new one on every call.
Also, better use the per-function memory context for this; otherwise
it leaks memory.
Discussion: https://www.postgresql.org/message-id/flat/17158-8a2ba823982537a4%40postgresql.org
M src/backend/utils/adt/arrayfuncs.c
doc: Clarify refresh options for DROP PUBLICATION
commit : 6e6adbddbce1391ffb676ce9779f24c25a6a9f09
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 15 Sep 2021 09:54:45 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 15 Sep 2021 09:54:45 +0200
The available refresh options are specified as refresh_options under
REFRESH PUBLICATION, and DROP PUBLICATION itself has an option named
refresh. Clarify what we mean by refresh options to avoid confusion.
Backpatch through v14 where ALTER SUBSCRIPTION ... DROP PUBLICATION
was introduced.
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Discussion: https://postgr.es/m/CAD21AoCm1wJ3A8Q9EmBjRbShYkJ+o+Oa_z9O0hvwhvhUa2BSyg@mail.gmail.com
Backpatch-through: 14
M doc/src/sgml/ref/alter_subscription.sgml
Send NOTIFY signals during CommitTransaction.
commit : 0eff10a008447bc6f6594547e2fc04756849eaed
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 14 Sep 2021 17:18:25 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 14 Sep 2021 17:18:25 -0400
Formerly, we sent signals for outgoing NOTIFY messages within
ProcessCompletedNotifies, which was also responsible for sending
relevant ones of those messages to our connected client. It therefore
had to run during the main-loop processing that occurs just before
going idle. This arrangement had two big disadvantages:
* Now that procedures allow intra-command COMMITs, it would be
useful to send NOTIFYs to other sessions immediately at COMMIT
(though, for reasons of wire-protocol stability, we still shouldn't
forward them to our client until end of command).
* Background processes such as replication workers would not send
NOTIFYs at all, since they never execute the client communication
loop. We've had requests to allow triggers running in replication
workers to send NOTIFYs, so that's a problem.
To fix these things, move transmission of outgoing NOTIFY signals
into AtCommit_Notify, where it will happen during CommitTransaction.
Also move the possible call of asyncQueueAdvanceTail there, to
ensure we don't bloat the async SLRU if a background worker sends
many NOTIFYs with no one listening.
We can also drop the call of asyncQueueReadAllNotifications,
allowing ProcessCompletedNotifies to go away entirely. That's
because commit 790026972 added a call of ProcessNotifyInterrupt
adjacent to PostgresMain's call of ProcessCompletedNotifies,
and that does its own call of asyncQueueReadAllNotifications,
meaning that we were uselessly doing two such calls (inside two
separate transactions) whenever inbound notify signals coincided
with an outbound notify. We need only set notifyInterruptPending
to ensure that ProcessNotifyInterrupt runs, and we're done.
The existing documentation suggests that custom background workers
should call ProcessCompletedNotifies if they want to send NOTIFY
messages. To avoid an ABI break in the back branches, reduce it
to an empty routine rather than removing it entirely. Removal
will occur in v15.
Although the problems mentioned above have existed for awhile,
I don't feel comfortable back-patching this any further than v13.
There was quite a bit of churn in adjacent code between 12 and 13.
At minimum we'd have to also backpatch 51004c717, and a good deal
of other adjustment would also be needed, so the benefit-to-risk
ratio doesn't look attractive.
Per bug #15293 from Michael Powers (and similar gripes from others).
Artur Zakirov and Tom Lane
Discussion: https://postgr.es/m/153243441449.1404.2274116228506175596@wrigleys.postgresql.org
M doc/src/sgml/bgworker.sgml
M src/backend/access/transam/xact.c
M src/backend/commands/async.c
M src/backend/tcop/postgres.c
M src/include/commands/async.h
Fix planner error with multiple copies of an AlternativeSubPlan.
commit : 29aa0ce361d16a79e4ebf7561cbb16ed1d0e2211
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 14 Sep 2021 15:11:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 14 Sep 2021 15:11:21 -0400
It's possible for us to copy an AlternativeSubPlan expression node
into multiple places, for example the scan quals of several
partition children. Then it's possible that we choose a different
one of the alternatives as optimal in each place. Commit 41efb8340
failed to consider this scenario, so its attempt to remove "unused"
subplans could remove subplans that were still used elsewhere.
Fix by delaying the removal logic until we've examined all the
AlternativeSubPlans in a given query level. (This does assume that
AlternativeSubPlans couldn't get copied to other query levels, but
for the foreseeable future that's fine; cf qual_is_pushdown_safe.)
Per report from Rajkumar Raghuwanshi. Back-patch to v14
where the faulty logic came in.
Discussion: https://postgr.es/m/CAKcux6==O3NNZC3bZ2prRYv3cjm3_Zw1GfzmOjEVqYN4jub2+Q@mail.gmail.com
M src/backend/optimizer/plan/setrefs.c
M src/include/nodes/pathnodes.h
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
jit: Do not try to shut down LLVM state in case of LLVM triggered errors.
commit : 4e86887e0922f20add67e2473c7baae8c7f05d5e
author : Andres Freund <andres@anarazel.de>
date : Mon, 13 Sep 2021 18:07:19 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 13 Sep 2021 18:07:19 -0700
If an allocation failed within LLVM it is not safe to call back into LLVM as
LLVM is not generally safe against exceptions / stack-unwinding. Thus errors
while in LLVM code are promoted to FATAL. However llvm_shutdown() did call
back into LLVM even in such cases, while llvm_release_context() was careful
not to do so.
We cannot generally skip shutting down LLVM, as that can break profiling. But
it's OK to do so if there was an error from within LLVM.
Reported-By: Jelte Fennema <Jelte.Fennema@microsoft.com>
Author: Andres Freund <andres@anarazel.de>
Author: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/AM5PR83MB0178C52CCA0A8DEA0207DC14F7FF9@AM5PR83MB0178.EURPRD83.prod.outlook.com
Backpatch: 11-, where jit was introduced
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_error.cpp
M src/include/jit/llvmjit.h
Fix potential for compiler warning in GlobalVisTestFor().
commit : 0d0bbee5e3ee2eabe3a1f6081cbc4226520764c0
author : Andres Freund <andres@anarazel.de>
date : Mon, 13 Sep 2021 16:50:10 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 13 Sep 2021 16:50:10 -0700
In d9d8aa9bb9a I added a defensive NULL assignment to protect against a
not-too-smart compiler warning about unitialized variable use after the
switch. Unfortunately I only did so on master and forgot to adjust that for
14.
Stephen noticed that there actually is a compiler warning :(.
Reported-By: Stephen Frost <sfrost@snowman.net>
Discussion: https://postgr.es/m/20210827224639.GX17906@tamriel.snowman.net
M src/backend/storage/ipc/procarray.c
Clear conn->errorMessage at successful completion of PQconnectdb().
commit : 896a0c44f93b0b449fdf7e6c3973deab5fdfed4f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 13 Sep 2021 16:53:11 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 13 Sep 2021 16:53:11 -0400
Commits ffa2e4670 and 52a10224e caused libpq's connection-establishment
functions to usually leave a nonempty string in the connection's
errorMessage buffer, even after a successful connection. While that
was intentional on my part, more sober reflection says that it wasn't
a great idea: the string would be a bit confusing. Also this broke at
least one application that checked for connection success by examining
the errorMessage, instead of using PQstatus() as documented. Let's
clear the buffer at success exit, restoring the pre-v14 behavior.
Discussion: https://postgr.es/m/4170264.1620321747@sss.pgh.pa.us
M src/interfaces/libpq/fe-connect.c
Fix EXIT out of outermost block in plpgsql.
commit : 4ffd3fe4d6ccc9ff36271e5a21ea3b065621b982
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 13 Sep 2021 12:42:03 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 13 Sep 2021 12:42:03 -0400
Ordinarily, using EXIT this way would draw "control reached end of
function without RETURN". However, if the function is one where we
don't require an explicit RETURN (such as a DO block), that should
not happen. It did anyway, because add_dummy_return() neglected to
account for the case.
Per report from Herwig Goemans. Back-patch to all supported branches.
Discussion: https://postgr.es/m/868ae948-e3ca-c7ec-95a6-83cfc08ef750@gmail.com
M src/pl/plpgsql/src/expected/plpgsql_control.out
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/sql/plpgsql_control.sql
doc: fix PG 14 release note typo
commit : 10faeb7ed5049afb70d26561e9991d52c363bd8e
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 13 Sep 2021 10:42:16 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 13 Sep 2021 10:42:16 -0400
Reported-by: Robert Haas
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
Doc: Remove type information for import_generated in postgres-fdw.sgml.
commit : 749945c2425b7c83d95b0359733f4839049fdcb8
author : Etsuro Fujita <efujita@postgresql.org>
date : Mon, 13 Sep 2021 17:30:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Mon, 13 Sep 2021 17:30:00 +0900
The type information for FDW options is only added to HEAD; remove this
from back branches. Oversight in commit aa769f80e.
Apply the patch to v12, v13, and v14.
Discussion: https://postgr.es/m/CAPmGK14z92twaKwRoccHbbh5Va5vbRDZcTYYTx50+0JTQ8xx_g@mail.gmail.com
M doc/src/sgml/postgres-fdw.sgml
Fix reorder buffer memory accounting for toast changes.
commit : f5e0ff4631b5402950db69c989c88c4c6295504b
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 13 Sep 2021 10:35:00 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 13 Sep 2021 10:35:00 +0530
While processing toast changes in logical decoding, we rejigger the
tuple change to point to in-memory toast tuples instead to on-disk toast
tuples. And, to make sure the memory accounting is correct, we were
subtracting the old change size and then after re-computing the new tuple,
re-adding its size at the end. Now, if there is any error before we add
the new size, we will release the changes and that will update the
accounting info (subtracting the size from the counters). And we were
underflowing there which leads to an assertion failure in assert enabled
builds and wrong memory accounting in reorder buffer otherwise.
Author: Bertrand Drouvot
Reviewed-by: Amit Kapila
Backpatch-through: 13, where memory accounting was introduced
Discussion: https://postgr.es/m/92b0ee65-b8bd-e42d-c082-4f3f4bf12d34@amazon.com
M src/backend/replication/logical/reorderbuffer.c
Fix error handling with threads on OOM in ECPG connection logic
commit : cc057fb315e24b78552f5b7ac05792418042e71b
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 13 Sep 2021 13:24:04 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 13 Sep 2021 13:24:04 +0900
An out-of-memory failure happening when allocating the structures to
store the connection parameter keywords and values would mess up with
the set of connections saved, as on failure the pthread mutex would
still be hold with the new connection object listed but free()'d.
Rather than just unlocking the mutex, which would leave the static list
of connections into an inconsistent state, move the allocation for the
structures of the connection parameters before beginning the test
manipulation. This ensures that the list of connections and the
connection mutex remain consistent all the time in this code path.
This error is unlikely going to happen, but this could mess up badly
with ECPG clients in surprising ways, so backpatch all the way down.
Reported-by: ryancaicse
Discussion: https://postgr.es/m/17186-b4cfd8f0eb4d1dee@postgresql.org
Backpatch-through: 9.6
M src/interfaces/ecpg/ecpglib/connect.c
Make pg_regexec() robust against out-of-range search_start.
commit : b33283cbd336adbf982c21aac1399130c8ffaaa9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 11 Sep 2021 15:19:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 11 Sep 2021 15:19:31 -0400
If search_start is greater than the length of the string, we should just
return REG_NOMATCH immediately. (Note that the equality case should
*not* be rejected, since the pattern might be able to match zero
characters.) This guards various internal assumptions that the min of a
range of string positions is not more than the max. Violation of those
assumptions could allow an attempt to fetch string[search_start-1],
possibly causing a crash.
Jaime Casanova pointed out that this situation is reachable with the
new regexp_xxx functions that accept a user-specified start position.
I don't believe it's reachable via any in-core call site in v14 and
below. However, extensions could possibly call pg_regexec with an
out-of-range search_start, so let's back-patch the fix anyway.
Discussion: https://postgr.es/m/20210911180357.GA6870@ahch-to
M src/backend/regex/regexec.c
Fix some anomalies with NO SCROLL cursors.
commit : d844cd75a6764a1dd2d8b3410144df9ebf8a0f04
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 10 Sep 2021 13:18:32 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 10 Sep 2021 13:18:32 -0400
We have long forbidden fetching backwards from a NO SCROLL cursor,
but the prohibition didn't extend to cases in which we rewind the
query altogether and then re-fetch forwards. I think the reason is
that this logic was mainly meant to protect plan nodes that can't
be run in the reverse direction. However, re-reading the query output
is problematic if the query is volatile (which includes SELECT FOR
UPDATE, not just queries with volatile functions): the re-read can
produce different results, which confuses the cursor navigation logic
completely. Another reason for disliking this approach is that some
code paths will either fetch backwards or rewind-and-fetch-forwards
depending on the distance to the target row; so that seemingly
identical use-cases may or may not draw the "cursor can only scan
forward" error. Hence, let's clean things up by disallowing rewind
as well as fetch-backwards in a NO SCROLL cursor.
Ordinarily we'd only make such a definitional change in HEAD, but
there is a third reason to consider this change now. Commit ba2c6d6ce
created some new user-visible anomalies for non-scrollable cursors
WITH HOLD, in that navigation in the cursor result got confused if the
cursor had been partially read before committing. The only good way
to resolve those anomalies is to forbid rewinding such a cursor, which
allows removal of the incorrect cursor state manipulations that
ba2c6d6ce added to PersistHoldablePortal.
To minimize the behavioral change in the back branches (including
v14), refuse to rewind a NO SCROLL cursor only when it has a holdStore,
ie has been held over from a previous transaction due to WITH HOLD.
This should avoid breaking most applications that have been sloppy
about whether to declare cursors as scrollable. We'll enforce the
prohibition across-the-board beginning in v15.
Back-patch to v11, as ba2c6d6ce was.
Discussion: https://postgr.es/m/3712911.1631207435@sss.pgh.pa.us
M src/backend/commands/portalcmds.c
M src/backend/tcop/pquery.c
M src/test/regress/expected/portals.out
M src/test/regress/sql/portals.sql
Avoid fetching from an already-terminated plan.
commit : b7056c0a25e545f4c1efcc1e2a6118e4626e377b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 9 Sep 2021 13:36:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 9 Sep 2021 13:36:31 -0400
Some plan node types don't react well to being called again after
they've already returned NULL. PortalRunSelect() has long dealt
with this by calling the executor with NoMovementScanDirection
if it sees that we've already run the portal to the end. However,
commit ba2c6d6ce overlooked this point, so that persisting an
already-fully-fetched cursor would fail if it had such a plan.
Per report from Tomas Barton. Back-patch to v11, as the faulty
commit was. (I've omitted a test case because the type of plan
that causes a problem isn't all that stable.)
Discussion: https://postgr.es/m/CAPV2KRjd=ErgVGbvO2Ty20tKTEZZr6cYsYLxgN_W3eAo9pf5sw@mail.gmail.com
M src/backend/commands/portalcmds.c
pgbench: Stop counting skipped transactions as soon as timer is exceeded.
commit : b27d0cd3147e2a0215253f61944c392a30265db8
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 10 Sep 2021 01:28:17 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 10 Sep 2021 01:28:17 +0900
When throttling is used, transactions that lag behind schedule by
more than the latency limit are counted and reported as skipped.
Previously, there was the case where pgbench counted all skipped
transactions even if the timer specified in -T option was exceeded.
This could take a very long time to do that especially when unrealistically
high rate setting in -R option caused quite a lot of transactions that
lagged behind schedule. This could prevent pgbench from ending
immediately, and so pgbench could look like it got stuck to users.
To fix the issue, this commit changes pgbench so that it stops counting
skipped transactions as soon as the timer is exceeded. The timer can
make pgbench end soon even when there are lots of skipped transactions
that have not been counted yet.
Note that there is no guarantee that all skipped transactions are
counted under -T though there is under -t. This is OK in practice
because it's very unlikely to happen with realistic setting. Also this is
not the issue that this commit newly introdues. There used to be
the case where pgbench ended without counting all skipped
transactions since before.
Back-patch to v14. Per discussion, we decided not to bother
back-patch to the stable branches because it's hard to imagine
the issue happens in practice (with realistic setting).
Author: Yugo Nagata, Fabien COELHO
Reviewed-by: Greg Sabino Mullane, Fujii Masao
Discussion: https://postgr.es/m/20210613040151.265ff59d832f835bbcf8b3ba@sraoss.co.jp
M src/bin/pgbench/pgbench.c
Check for relation length overrun soon enough.
commit : 7430c774209cd98bbc33076cc3c07497c1086d9a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 9 Sep 2021 11:45:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 9 Sep 2021 11:45:48 -0400
We don't allow relations to exceed 2^32-1 blocks, because block
numbers are 32 bits and the last possible block number is reserved
to mean InvalidBlockNumber. There is a check for this in mdextend,
but that's really way too late, because the smgr API requires us to
create a buffer for the block-to-be-added, and we do not want to
have any buffer with blocknum InvalidBlockNumber. (Such a case
can trigger assertions in bufmgr.c, plus I think it might confuse
ReadBuffer's logic for data-past-EOF later on.) So put the check
into ReadBuffer.
Per report from Christoph Berg. It's been like this forever,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/YTn1iTkUYBZfcODk@msg.credativ.de
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/smgr/md.c
Fix issue with WAL archiving in standby.
commit : b5ec22bf5e8f19964f7c7d7ef357ff947a38c7fc
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 9 Sep 2021 23:58:05 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 9 Sep 2021 23:58:05 +0900
Previously, walreceiver always closed the currently-opened WAL segment
and created its archive notification file, after it finished writing
the current segment up and received any WAL data that should be
written into the next segment. If walreceiver exited just before
any WAL data in the next segment arrived at standby, it did not
create the archive notification file of the current segment
even though that's known completed. This behavior could cause
WAL archiving of the segment to be delayed until subsequent
restartpoints or checkpoints created its notification file.
To fix the issue, this commit changes walreceiver so that it creates
an archive notification file of a current WAL segment immediately
if that's known completed before receiving next WAL data.
Back-patch to all supported branches.
Reported-by: Kyotaro Horiguchi
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20200630.165503.1465894182551545886.horikyota.ntt@gmail.com
M src/backend/replication/walreceiver.c
Avoid useless malloc/free traffic around getFormattedTypeName().
commit : 52c300df323e8ebb3d0613baa4711179d9f342f3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 8 Sep 2021 15:09:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 8 Sep 2021 15:09:42 -0400
Coverity complained that one caller of getFormattedTypeName() failed
to free the returned string. Which is true, but rather than fixing
that one, let's get rid of this tedious and error-prone requirement.
Now that getFormattedTypeName() caches its result, strdup'ing that
result and expecting the caller to free it accomplishes little except
to waste cycles. We do create a leak in the case where getTypes didn't
make a TypeInfo for the type, but that basically shouldn't ever happen.
Back-patch, as commit 6c450a861 was. This isn't a particularly
interesting bug fix, but the API change seems like a hazard for
future back-patching activity if we don't back-patch it.
M src/bin/pg_dump/pg_dump.c
Fix misleading comments about TOAST access macros.
commit : 67948a433e18a8561daf89dcc53e591a3fac659a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 8 Sep 2021 14:11:35 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 8 Sep 2021 14:11:35 -0400
Seems to have been my error in commit aeb1631ed.
Noted by Christoph Berg.
Discussion: https://postgr.es/m/YTeLipdnSOg4NNcI@msg.df7cb.de
M src/include/postgres.h
Fix rewriter to set hasModifyingCTE correctly on rewritten queries.
commit : 03d01d746b9338dd1568d37d26b344928f82c5ff
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 8 Sep 2021 12:05:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 8 Sep 2021 12:05:43 -0400
If we copy data-modifying CTEs from the original query to a replacement
query (from a DO INSTEAD rule), we must set hasModifyingCTE properly
in the replacement query. Failure to do this can cause various
unpleasantness, such as unsafe usage of parallel plans. The code also
neglected to propagate hasRecursive, though that's only cosmetic at
the moment.
A difficulty arises if the rule action is an INSERT...SELECT. We
attach the original query's RTEs and CTEs to the sub-SELECT Query, but
data-modifying CTEs are only allowed to appear in the topmost Query.
For the moment, throw an error in such cases. It would probably be
possible to avoid this error by attaching the CTEs to the top INSERT
Query instead; but that would require a bunch of new code to adjust
ctelevelsup references. Given the narrowness of the use-case, and
the need to back-patch this fix, it does not seem worth the trouble
for now. We can revisit this if we get field complaints.
Per report from Greg Nancarrow. Back-patch to all supported branches.
(The test case added here does not fail before v10, but there are
plenty of places checking top-level hasModifyingCTE in 9.6, so I have
no doubt that this code change is necessary there too.)
Greg Nancarrow and Tom Lane
Discussion: https://postgr.es/m/CAJcOf-f68DT=26YAMz_i0+Au3TcLO5oiHY5=fL6Sfuits6r+_w@mail.gmail.com
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Disable anonymous record hash support except in special cases
commit : 054adca641ac1279dc8d9b74fda41948ac35e9a9
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 8 Sep 2021 09:25:46 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 8 Sep 2021 09:25:46 +0200
Commit 01e658fa74 added hash support for row types. This also added
support for hashing anonymous record types, using the same approach
that the type cache uses for comparison support for record types: It
just reports that it works, but it might fail at run time if a
component type doesn't actually support the operation. We get away
with that for comparison because most types support that. But some
types don't support hashing, so the current state can result in
failures at run time where the planner chooses hashing over sorting,
whereas that previously worked if only sorting was an option.
We do, however, want the record hashing support for path tracking in
recursive unions, and the SEARCH and CYCLE clauses built on that. In
that case, hashing is the only plan option. So enable that, this
commit implements the following approach: The type cache does not
report that hashing is available for the record type. This undoes
that part of 01e658fa74. Instead, callers that require hashing no
matter what can override that result themselves. This patch only
touches the callers to make the aforementioned recursive query cases
work, namely the parse analysis of unions, as well as the hash_array()
function.
Reported-by: Sait Talha Nisanci <sait.nisanci@microsoft.com>
Bug: #17158
Discussion: https://www.postgresql.org/message-id/flat/17158-8a2ba823982537a4%40postgresql.org
M src/backend/parser/analyze.c
M src/backend/rewrite/rewriteSearchCycle.c
M src/backend/utils/adt/arrayfuncs.c
M src/backend/utils/cache/typcache.c
M src/include/parser/analyze.h
M src/test/regress/expected/union.out
M src/test/regress/sql/union.sql
Invalidate relcache for publications defined for all tables.
commit : 8db27fbc11974ef491fec97be3365277ed4fab20
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 8 Sep 2021 09:58:28 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 8 Sep 2021 09:58:28 +0530
Updates/Deletes on a relation were allowed even without replica identity
after we define the publication for all tables. This would later lead to
an error on subscribers. The reason was that for such publications we were
not invalidating the relcache and the publication information for
relations was not getting rebuilt. Similarly, we were not invalidating the
relcache after dropping of such publications which will prohibit
Updates/Deletes without replica identity even without any publication.
Author: Vignesh C and Hou Zhijie
Reviewed-by: Hou Zhijie, Kyotaro Horiguchi, Amit Kapila
Backpatch-through: 10, where it was introduced
Discussion: https://postgr.es/m/CALDaNm0pF6zeWqCA8TCe2sDuwFAy8fCqba=nHampCKag-qLixg@mail.gmail.com
M src/backend/catalog/dependency.c
M src/backend/commands/publicationcmds.c
M src/include/commands/publicationcmds.h
M src/test/regress/expected/publication.out
M src/test/regress/sql/publication.sql
Consistently use read-only instead of "read only"
commit : b7fd291042a846b04439f122cb81a41d3cd2e8de
author : Magnus Hagander <magnus@hagander.net>
date : Tue, 7 Sep 2021 21:59:25 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Tue, 7 Sep 2021 21:59:25 +0200
This affects one message and some documentation that used the format
"read only", unlike everything else that used read-only.
Backpatch-through: 14
Discussion: https://postgr.es/m/CABUevExuxKwn0YM3+wdSeQSvK6CRrJ-hewocGVX3R4-xVX4eMw@mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/mvcc.sgml
M src/backend/postmaster/postmaster.c
Finish reverting 3eda9fc09fd6b9a1aec2d0113c633c69c3214b4d.
commit : 8b895374cd9cf6989bcee5b6f70f65f2d3520224
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 7 Sep 2021 10:52:25 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 7 Sep 2021 10:52:25 -0400
Commit 67c33a114 should have set v14's catversion back to what it was
before 3eda9fc09, to avoid forcing a useless pg_upgrade cycle on users
of 14beta3. Do that now.
Discussion: https://postgr.es/m/2598498.1630702074@sss.pgh.pa.us
M src/include/catalog/catversion.h
Fix missing words in comment.
commit : e66add755df2eae5e874dc96743f3faaf4bced83
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 7 Sep 2021 10:28:55 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 7 Sep 2021 10:28:55 +0300
Introduced by commit c3928b467a, backpatch to v14 like that one.
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA+HiwqFQgNLS6VGntMcuJV6erBFV425xA6wBVnY=41GK4zC0Bw@mail.gmail.com
M src/backend/executor/nodeForeignscan.c
AIX: Fix missing libpq symbols by respecting SHLIB_EXPORTS.
commit : 47d54b6ba2749f5da72b7ab612e53e7f7b45b819
author : Noah Misch <noah@leadboat.com>
date : Mon, 6 Sep 2021 11:27:59 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 6 Sep 2021 11:27:59 -0700
We make each AIX shared library export all globals found in .o files
that originate in the library. That doesn't include symbols acquired by
-lpgcommon_shlib. That is good on average, but it became a problem for
libpq when commit e6afa8918c461c1dd80c5063a950518fa4e950cd moved five
official libpq API symbols into src/common. Fix this by implementing
the SHLIB_EXPORTS mechanism for AIX, so affected libraries export the
same symbols that they export on Linux. This reintroduces symbols
pg_encoding_to_char, pg_utf_mblen, pg_char_to_encoding,
pg_valid_server_encoding, and pg_valid_server_encoding_id. Back-patch
to v13, where the aforementioned commit first appeared. While a minor
release is usually the wrong time to add or remove symbol exports in
libpq or libecpg, we should expect users to want each documented symbol.
Tony Reix
Discussion: https://postgr.es/m/PR3PR02MB6396742E2FC3E77D37A920BC86C79@PR3PR02MB6396.eurprd02.prod.outlook.com
M src/Makefile.shlib
M src/port/README
Fix bogus timetz_zone() results for DYNTZ abbreviations.
commit : 599c73a91a0471465a84f12fe6a2e7236a825721
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 6 Sep 2021 11:29:52 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 6 Sep 2021 11:29:52 -0400
timetz_zone() delivered completely wrong answers if the zone was
specified by a dynamic TZ abbreviation, because it failed to account
for the difference between the POSIX conventions for field values in
struct pg_tm and the conventions used in PG-specific datetime code.
As a stopgap fix, just adjust the tm_year and tm_mon fields to match
PG conventions. This is fixed in a different way in HEAD (388e71af8)
but I don't want to back-patch the change of reference point.
Discussion: https://postgr.es/m/CAJ7c6TOMG8zSNEZtCn5SPe+cCk3Lfxb71ZaQwT2F4T7PJ_t=KA@mail.gmail.com
M src/backend/utils/adt/date.c
Fix pkg-config files for static linking
commit : 1e9afc868eb5a3b2f50629c2724d9fcdbe0afee2
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 6 Sep 2021 09:41:03 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 6 Sep 2021 09:41:03 +0200
Since ea53100d5 (PostgreSQL 12), the shipped pkg-config files have
been broken for statically linking libpq because libpgcommon and
libpgport are missing. This patch adds those two missing private
dependencies (in a non-hardcoded way).
Reported-by: Filip Gospodinov <f@gospodinov.ch>
Discussion: https://www.postgresql.org/message-id/flat/c7108bde-e051-11d5-a234-99beec01ce2a@gospodinov.ch
M src/Makefile.shlib
Further portability tweaks for float4/float8 hash functions.
commit : 718978d9daa7128e0f5e4e06c0fd402cdd63704f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 4 Sep 2021 16:29:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 4 Sep 2021 16:29:08 -0400
Attempting to make hashfloat4() look as much as possible like
hashfloat8(), I'd figured I could replace NaNs with get_float4_nan()
before widening to float8. However, results from protosciurus
and topminnow show that on some platforms that produces a different
bit-pattern from get_float8_nan(), breaking the intent of ce773f230.
Rearrange so that we use the result of get_float8_nan() for all NaN
cases. As before, back-patch.
M src/backend/access/hash/hashfunc.c
Minor improvements for psql help output.
commit : 5edbcbd5b94c696526e18bbb490ab2e9f4282826
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 4 Sep 2021 13:27:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 4 Sep 2021 13:27:55 -0400
Fix alphabetization of the output of "\?", and improve one description.
Update PageOutput counts where needed, fixing breakage from previous
patches.
Haiying Tang (PageOutput fixes by me)
Discussion: https://postgr.es/m/OS0PR01MB61136018064660F095CB57A8FB129@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/bin/psql/help.c
Revert "Avoid creating archive status ".ready" files too early"
commit : aa8bd0890bb29356a465d235dee59c1b08fa5fc5
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 4 Sep 2021 12:14:30 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 4 Sep 2021 12:14:30 -0400
This reverts commit 515e3d84a0b5 and equivalent commits in back
branches. This solution to the problem has a number of problems, so
we'll try again with a different approach.
Per note from Andres Freund
Discussion: https://postgr.es/m/20210831042949.52eqp5xwbxgrfank@alap3.anarazel.de
M src/backend/access/transam/xlog.c
M src/backend/postmaster/walwriter.c
M src/include/access/xlog.h
M src/include/access/xlogdefs.h
Remove arbitrary MAXPGPATH limit on command lengths in pg_ctl.
commit : 69d670e68ec9bd00b71ddc528274746790d7b6bd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 3 Sep 2021 21:04:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 3 Sep 2021 21:04:44 -0400
Replace fixed-length command buffers with psprintf() calls. We didn't
have anything as convenient as psprintf() when this code was written,
but now that we do, there's little reason for the limitation to
stand. Removing it eliminates some corner cases where (for example)
starting the postmaster with a whole lot of options fails.
Most individual file names that pg_ctl deals with are still restricted
to MAXPGPATH, but we've seldom had complaints about that limitation
so long as it only applies to one filename.
Back-patch to all supported branches.
Phil Krylov
Discussion: https://postgr.es/m/567e199c6b97ee19deee600311515b86@krylov.eu
M src/bin/pg_ctl/pg_ctl.c
Disallow creating an ICU collation if the DB encoding won't support it.
commit : 2cc018ba8f80a5236faf2c58dfcd18fee581dbbc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 3 Sep 2021 16:38:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 3 Sep 2021 16:38:55 -0400
Previously this was allowed, but the collation effectively vanished
into the ether because of the way lookup_collation() works: you could
not use the collation, nor even drop it. Seems better to give an
error up front than to leave the user wondering why it doesn't work.
(Because this test is in DefineCollation not CreateCollation, it does
not prevent pg_import_system_collations from creating ICU collations,
regardless of the initially-chosen encoding.)
Per bug #17170 from Andrew Bille. Back-patch to v10 where ICU support
was added.
Discussion: https://postgr.es/m/17170-95845cf3f0a9c36d@postgresql.org
M src/backend/commands/collationcmds.c
Set the volatility of the timestamptz version of date_bin() back to immutable
commit : 67c33a114f38edbd66f68d7b2c0cb7b03611ec48
author : John Naylor <john.naylor@postgresql.org>
date : Fri, 3 Sep 2021 13:38:15 -0400
committer: John Naylor <john.naylor@postgresql.org>
date : Fri, 3 Sep 2021 13:38:15 -0400
543f36b43d was too hasty in thinking that the volatility of date_bin()
had to match date_trunc(), since only the latter references
session_timezone.
Bump catversion
Per feedback from Aleksander Alekseev
Backpatch to v14, as the former commit was
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Fix portability issue in tests from commit ce773f230.
commit : 08b96a2b5243957acd3f7c3105af8b5b2d0da3ef
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 3 Sep 2021 10:01:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 3 Sep 2021 10:01:02 -0400
Modern POSIX seems to require strtod() to accept "-NaN", but there's
nothing about NaN in SUSv2, and some of our oldest buildfarm members
don't like it. Let's try writing it as -'NaN' instead; that seems
to produce the same result, at least on Intel hardware.
Per buildfarm.
M src/test/regress/expected/hash_func.out
M src/test/regress/sql/hash_func.sql
In count_usable_fds(), duplicate stderr not stdin.
commit : 6b54f12332a184bfd03524c012795678ddc992f4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 2 Sep 2021 18:53:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 2 Sep 2021 18:53:10 -0400
We had a complaint that the postmaster fails to start if the invoking
program closes stdin. That happens because count_usable_fds expects
to be able to dup(0), and if it can't, we conclude there are no free
FDs and go belly-up. So far as I can find, though, there is no other
place in the server that touches stdin, and it's not unreasonable to
expect that a daemon wouldn't use that file.
As a simple improvement, let's dup FD 2 (stderr) instead. Unlike stdin,
it *is* reasonable for us to expect that stderr be open; even if we are
configured not to touch it, common libraries such as libc might try to
write error messages there.
Per gripe from Mario Emmenlauer. Given the lack of previous complaints,
I'm not excited about pushing this into stable branches, but it seems
OK to squeeze it into v14.
Discussion: https://postgr.es/m/48bafc63-c30f-3962-2ded-f2e985d93e86@emmenlauer.de
M src/backend/storage/file/fd.c
Fix float4/float8 hash functions to produce uniform results for NaNs.
commit : 23c6bc581dc90d08421a13494f135681504ee4e6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 2 Sep 2021 17:24:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 2 Sep 2021 17:24:42 -0400
The IEEE 754 standard allows a wide variety of bit patterns for NaNs,
of which at least two ("NaN" and "-NaN") are pretty easy to produce
from SQL on most machines. This is problematic because our btree
comparison functions deem all NaNs to be equal, but our float hash
functions know nothing about NaNs and will happily produce varying
hash codes for them. That causes unexpected results from queries
that hash a column containing different NaN values. It could also
produce unexpected lookup failures when using a hash index on a
float column, i.e. "WHERE x = 'NaN'" will not find all the rows
it should.
To fix, special-case NaN in the float hash functions, not too much
unlike the existing special case that forces zero and minus zero
to hash the same. I arranged for the most vanilla sort of NaN
(that coming from the C99 NAN constant) to still have the same
hash code as before, to reduce the risk to existing hash indexes.
I dithered about whether to back-patch this into stable branches,
but ultimately decided to do so. It's a clear improvement for
queries that hash internally. If there is anybody who has -NaN
in a hash index, they'd be well advised to re-index after applying
this patch ... but the misbehavior if they don't will not be much
worse than the misbehavior they had before.
Per bug #17172 from Ma Liangzhu.
Discussion: https://postgr.es/m/17172-7505bea9e04e230f@postgresql.org
M src/backend/access/hash/hashfunc.c
M src/test/regress/expected/hash_func.out
M src/test/regress/sql/hash_func.sql
doc: Replace some uses of "which" by "that" in parallel.sgml
commit : 2c1981ec3c8e1c5a52a8e9513a832d362f4fb4c1
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 2 Sep 2021 11:35:56 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 2 Sep 2021 11:35:56 +0900
This makes the documentation more accurate grammatically.
Author: Elena Indrupskaya
Discussion: https://postgr.es/m/1c994b3d-951e-59bb-1ac2-7b9221c0e4cf@postgrespro.ru
Backpatch-through: 9.6
M doc/src/sgml/parallel.sgml
Doc: clarify how triggers relate to transactions.
commit : 95bc40f880a68dc092ca34c4813f2b27962f233d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 1 Sep 2021 17:24:59 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 1 Sep 2021 17:24:59 -0400
Laurenz Albe, per gripe from Nathan Long.
Discussion: https://postgr.es/m/161953360822.695.15805897835151971142@wrigleys.postgresql.org
M doc/src/sgml/ref/create_trigger.sgml
M doc/src/sgml/trigger.sgml
Identify simple column references in extended statistics
commit : 50ba70a957f9e018495a111fc4b5e5eb2ea62044
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 1 Sep 2021 17:41:54 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 1 Sep 2021 17:41:54 +0200
Until now, when defining extended statistics, everything except a plain
column reference was treated as complex expression. So for example "a"
was a column reference, but "(a)" would be an expression. In most cases
this does not matter much, but there were a couple strange consequences.
For example
CREATE STATISTICS s ON a FROM t;
would fail, because extended stats require at least two columns. But
CREATE STATISTICS s ON (a) FROM t;
would succeed, because that requirement does not apply to expressions.
Moreover, that statistics object is useless - the optimizer will always
use the regular statistics collected for attribute "a".
So do a bit more work to identify those expressions referencing a single
column, and translate them to a simple column reference. Backpatch to
14, where support for extended statistics on expressions was introduced.
Reported-by: Justin Pryzby
Backpatch-through: 14
Discussion: https://postgr.es/m/20210816013255.GS10479%40telsasoft.com
M src/backend/commands/statscmds.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
pgbench: Fix bug in measurement of disconnection delays.
commit : d760d942c73c9a161feefb7dc4a0004b9b4e7787
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 1 Sep 2021 17:05:13 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 1 Sep 2021 17:05:13 +0900
When -C/--connect option is specified, pgbench establishes and closes
the connection for each transaction. In this case pgbench needs to
measure the times taken for all those connections and disconnections,
to include the average connection time in the benchmark result.
But previously pgbench could not measure those disconnection delays.
To fix the bug, this commit makes pgbench measure the disconnection
delays whenever the connection is closed at the end of transaction,
if -C/--connect option is specified.
Back-patch to v14. Per discussion, we concluded not to back-patch to v13
or before because changing that behavior in stable branches would
surprise users rather than providing benefits.
Author: Yugo Nagata
Reviewed-by: Fabien COELHO, Tatsuo Ishii, Asif Rehman, Fujii Masao
Discussion: https://postgr.es/m/20210614151155.a393bc7d8fed183e38c9f52a@sraoss.co.jp
M src/bin/pgbench/pgbench.c
Fix the random test failure in 001_rep_changes.
commit : b7ad093d50e13aadfffb1662f53cd16a1c59e09d
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 1 Sep 2021 10:00:12 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 1 Sep 2021 10:00:12 +0530
The check to test whether the subscription workers were restarting after a
change in the subscription was failing. The reason was that the test was
assuming the walsender started before it reaches the 'streaming' state and
the walsender was exiting due to an error before that. Now, the walsender
was erroring out before reaching the 'streaming' state because it tries to
acquire the slot before the previous walsender has exited.
In passing, improve the die messages so that it is easier to investigate
the failures in the future if any.
Reported-by: Michael Paquier, as per buildfarm
Author: Ajin Cherian
Reviewed-by: Masahiko Sawada, Amit Kapila
Backpatch-through: 10, where this test was introduced
Discussion: https://postgr.es/m/YRnhFxa9bo73wfpV@paquier.xyz
M src/test/subscription/t/001_rep_changes.pl
VACUUM VERBOSE: Don't report "pages removed".
commit : 0d892cf73a13b3a32af438a059a168e711aa0a7f
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 31 Aug 2021 20:37:17 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 31 Aug 2021 20:37:17 -0700
It doesn't make any sense to report this information, since VACUUM
VERBOSE reports on heap relation truncation directly. This was an
oversight in commit 7ab96cf6, which made VACUUM VERBOSE output a little
more consistent with nearby autovacuum-specific log output. Adjust
comments that describe how this is supposed to work in passing.
Also bring truncation-related VACUUM VERBOSE output in line with the
convention established for VACUUM VERBOSE output by commit f4f4a649.
Author: Peter Geoghegan <pg@bowt.ie>
Backpatch: 14-, where VACUUM VERBOSE's output changed.
M src/backend/access/heap/vacuumlazy.c
Don't print extra parens around expressions in extended stats
commit : 4d1816ec26e877297608a850736afed962527d70
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 1 Sep 2021 00:42:32 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 1 Sep 2021 00:42:32 +0200
The code printing expressions for extended statistics doubled the
parens, producing results like ((a+1)), which is unnecessary and not
consistent with how we print expressions elsewhere.
Fixed by tweaking the code to produce just a single set of parens.
Reported by Mark Dilger, fix by me. Backpatch to 14, where support for
extended statistics on expressions was added.
Reported-by: Mark Dilger
Discussion: https://postgr.es/m/20210122040101.GF27167%40telsasoft.com
M src/backend/utils/adt/ruleutils.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/test/regress/expected/create_table_like.out
M src/test/regress/expected/stats_ext.out
Mark the timestamptz variant of date_bin() as stable
commit : 3eda9fc09fd6b9a1aec2d0113c633c69c3214b4d
author : John Naylor <john.naylor@postgresql.org>
date : Tue, 31 Aug 2021 14:15:22 -0400
committer: John Naylor <john.naylor@postgresql.org>
date : Tue, 31 Aug 2021 14:15:22 -0400
Previously, it was immutable by lack of marking. This is not
correct, since the time zone could change.
Bump catversion
Discussion: https://www.postgresql.org/message-id/CAFBsxsG2UHk8mOWL0tca%3D_cg%2B_oA5mVRNLhDF0TBw980iOg5NQ%40mail.gmail.com
Backpatch to v14, when this function came in
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
In pg_dump, avoid doing per-table queries for RLS policies.
commit : a20a9f26cefcf4e35ba7bb3d9e8672cb4ce1cf32
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 31 Aug 2021 15:04:05 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 31 Aug 2021 15:04:05 -0400
For no particularly good reason, getPolicies() queried pg_policy
separately for each table. We can collect all the policies in
a single query instead, and attach them to the correct TableInfo
objects using findTableByOid() lookups. On the regression
database, this reduces the number of queries substantially, and
provides a visible savings even when running against a local
server.
Per complaint from Hubert Depesz Lubaczewski. Since this is such
a simple fix and can have a visible performance benefit, back-patch
to all supported branches.
Discussion: https://postgr.es/m/20210826084430.GA26282@depesz.com
M src/bin/pg_dump/pg_dump.c
Cache the results of format_type() queries in pg_dump.
commit : 9407dbbcb5b587cbefc4af14d1612b49abcb143b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 31 Aug 2021 13:53:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 31 Aug 2021 13:53:33 -0400
There's long been a "TODO: there might be some value in caching
the results" annotation on pg_dump's getFormattedTypeName function;
but we hadn't gotten around to checking what it was costing us to
repetitively look up type names. It turns out that when dumping the
current regression database, about 10% of the total number of queries
issued are duplicative format_type() queries. However, Hubert Depesz
Lubaczewski reported a not-unusual case where these account for over
half of the queries issued by pg_dump. Individually these queries
aren't expensive, but when network lag is a factor, they add up to a
problem. We can very easily add some caching to getFormattedTypeName
to solve it.
Since this is such a simple fix and can have a visible performance
benefit, back-patch to all supported branches.
Discussion: https://postgr.es/m/20210826084430.GA26282@depesz.com
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
Rename the role in stats_ext to have regress_ prefix
commit : 4090ff2a99b76b7bd51534fb0f7013aa646d1e24
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 31 Aug 2021 19:21:29 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 31 Aug 2021 19:21:29 +0200
Commit 5be8ce82e8 added a new role to the stats_ext regression suite,
but the role name did not start with regress_ causing failures when
running with ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS. Fixed by
renaming the role to start with the expected regress_ prefix.
Backpatch-through: 10, same as the new regression test
Discussion: https://postgr.es/m/1F238937-7CC2-4703-A1B1-6DC225B8978A%40enterprisedb.com
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Fix lookup error in extended stats ownership check
commit : a371a5ba349d9c39e7754f7d0f195ac46ee87b3f
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 31 Aug 2021 18:03:05 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 31 Aug 2021 18:03:05 +0200
When an ownership check on extended statistics object failed, the code
was calling aclcheck_error_type to report the failure, which is clearly
wrong, resulting in cache lookup errors. Fix by calling aclcheck_error.
This issue exists since the introduction of extended statistics, so
backpatch all the way back to PostgreSQL 10. It went unnoticed because
there were no tests triggering the error, so add one.
Reported-by: Mark Dilger
Backpatch-through: 10, where extended stats were introduced
Discussion: https://postgr.es/m/1F238937-7CC2-4703-A1B1-6DC225B8978A%40enterprisedb.com
M src/backend/catalog/objectaddress.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Fix missed lock acquisition while inlining new-style SQL functions.
commit : 983d7033df6e03134572eb75dd76603fb9245211
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 31 Aug 2021 12:02:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 31 Aug 2021 12:02:36 -0400
When starting to use a query parsetree loaded from the catalogs,
we must begin by applying AcquireRewriteLocks(), to obtain the same
relation locks that the parser would have gotten if the query were
entered interactively, and to do some other cleanup such as dealing
with later-dropped columns. New-style SQL functions are just as
subject to this rule as other stored parsetrees; however, of the
places dealing with such functions, only init_sql_fcache had gotten
the memo. In particular, if we successfully inlined a new-style
set-returning SQL function that contained any relation references,
we'd either get an assertion failure or attempt to use those
relation(s) sans locks.
I also added AcquireRewriteLocks calls to fmgr_sql_validator and
print_function_sqlbody. Desultory experiments didn't demonstrate any
failures in those, but I suspect that I just didn't try hard enough.
Certainly we don't expect nearby code paths to operate without locks.
On the same logic of it-ought-to-have-the-same-effects-as-the-old-code,
call pg_rewrite_query() in fmgr_sql_validator, too. It's possible
that neither code path there needs to bother with rewriting, but
doing the analysis to prove that is beyond my goals for today.
Per bug #17161 from Alexander Lakhin.
Discussion: https://postgr.es/m/17161-048a1cdff8422800@postgresql.org
M src/backend/catalog/pg_proc.c
M src/backend/optimizer/util/clauses.c
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/create_function_3.out
M src/test/regress/sql/create_function_3.sql
Report tuple address in data-corruption error message
commit : eae08e21653c356f0e9223d26379527d653b71ed
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 30 Aug 2021 16:29:12 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 30 Aug 2021 16:29:12 -0400
Most data-corruption reports mention the location of the problem, but
this one failed to. Add it.
Backpatch all the way back. In 12 and older, also assign the
ERRCODE_DATA_CORRUPTED error code as was done in commit fd6ec93bf890 for
13 and later.
Discussion: https://postgr.es/m/202108191637.oqyzrdtnheir@alvherre.pgsql
M src/backend/access/heap/heapam_handler.c
psql: Fix name quoting on extended statistics
commit : ce6b662aae7abc8b533b0cfa8fff50a9001508b1
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 30 Aug 2021 14:01:29 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 30 Aug 2021 14:01:29 -0400
Per our message style guidelines, for human consumption we quote
qualified names as a whole rather than each part separately; but commits
bc085205c8a4 introduced a deviation for extended statistics and
a4d75c86bf15 copied it. I don't agree with this policy applying to
names shown by psql, but that's a poor reason to deviate from the
practice only in two obscure corners, so make said corners use the same
style as everywhere else.
Backpatch to 14. The first of these is older, but I'm not sure we want
to destabilize the psql output in the older branches for such a small
thing.
Discussion: https://postgr.es/m/20210828181618.GS26465@telsasoft.com
M src/bin/psql/describe.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/expected/stats_ext.out
pgbench: Avoid unnecessary measurement of connection delays.
commit : efe2382d5ade2c8d63f2aa1b48cc85691a2bfabd
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 30 Aug 2021 21:35:24 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 30 Aug 2021 21:35:24 +0900
Commit 547f04e734 changed pgbench so that it used the measurement result
of connection delays in its benchmark report only when -C/--connect option
is specified. But previously those delays were unnecessarily measured
even when that option is not specified. Which was a waste of cycles.
This commit improves pgbench so that it avoids such unnecessary measurement.
Back-patch to v14 where commit 547f04e734 first appeared.
Author: Yugo Nagata
Reviewed-by: Fabien COELHO, Asif Rehman, Fujii Masao
Discussion: https://postgr.es/m/20210614151155.a393bc7d8fed183e38c9f52a@sraoss.co.jp
M src/bin/pgbench/pgbench.c
Add list of acknowledgments to release notes
commit : 7af5c38eb9da5b0ae72c1dd3d847f43cd39d1f5a
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 30 Aug 2021 08:56:16 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 30 Aug 2021 08:56:16 +0200
This contains all individuals mentioned in the commit messages during
PostgreSQL 14 development.
current through ed740b06b18e1a23becd54c97ff229aba4c94349
M doc/src/sgml/release-14.sgml
Fix incorrect error code in StartupReplicationOrigin().
commit : 0a143c33f01257a0670af71205343b505f5bd89b
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 30 Aug 2021 09:22:28 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 30 Aug 2021 09:22:28 +0530
ERRCODE_CONFIGURATION_LIMIT_EXCEEDED was used for checksum failure, use
ERRCODE_DATA_CORRUPTED instead.
Reported-by: Tatsuhito Kasahara
Author: Tatsuhito Kasahara
Backpatch-through: 9.6, where it was introduced
Discussion: https://postgr.es/m/CAP0=ZVLHtYffs8SOWcFJWrBGoRzT9QQbk+_aP+E5AHLNXiOorA@mail.gmail.com
M src/backend/replication/logical/origin.c
Keep stats up to date for partitioned tables
commit : e1efc5b465c844969a0ed0d07e1364f3ce424d8c
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 28 Aug 2021 15:58:23 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 28 Aug 2021 15:58:23 -0400
In the long-going saga for analyze on partitioned tables, one thing I
missed while reverting 0827e8af70f4 is the maintenance of analyze count
and last analyze time for partitioned tables. This is a mostly trivial
change that enables users assess the need for invoking manual ANALYZE on
partitioned tables.
This patch, posted by Justin and modified a bit by me (Álvaro), can be
mostly traced back to Hosoya-san, though any problems introduced with
the scissors are mine.
Backpatch to 14, in line with 6f8127b73901.
Co-authored-by: Yuzuko Hosoya <yuzukohosoya@gmail.com>
Co-authored-by: Justin Pryzby <pryzby@telsasoft.com>
Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/20210816222810.GE10479@telsasoft.com
M src/backend/commands/analyze.c
M src/backend/postmaster/pgstat.c
psql \dX: reference regclass with "pg_catalog." prefix
commit : 359bcf775550aa577c86ea30a6d071487fcca1ed
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 28 Aug 2021 12:04:15 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 28 Aug 2021 12:04:15 -0400
Déjà vu of commit fc40ba1296a7, for another backslash command.
Strictly speaking this isn't a bug, but since all references to catalog
objects are schema-qualified, we might as well be consistent. The
omission first appeared in commit ad600bba0422 and replicated in
a4d75c86bf15; backpatch to 14.
Author: Justin Pryzby <pryzbyj@telsasoft.com>
Discussion: https://postgr.es/m/20210827193151.GN26465@telsasoft.com
M src/bin/psql/describe.c
psql \dP: reference regclass with "pg_catalog." prefix
commit : a32860b88f1ffd6ba5b8bc803aecf6bc0f87cf02
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 28 Aug 2021 11:45:47 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 28 Aug 2021 11:45:47 -0400
Strictly speaking this isn't a bug, but since all references to catalog
objects are schema-qualified, we might as well be consistent. The
omission first appeared in commit 1c5d9270e339, so backpatch to 12.
Author: Justin Pryzby <pryzbyj@telsasoft.com>
Discussion: https://postgr.es/m/20210827193151.GN26465@telsasoft.com
M src/bin/psql/describe.c
Fix data loss in wal_level=minimal crash recovery of CREATE TABLESPACE.
commit : 5513c09c89993cb0b2c1a4222b15722bcf71844b
author : Noah Misch <noah@leadboat.com>
date : Fri, 27 Aug 2021 23:33:23 -0700
committer: Noah Misch <noah@leadboat.com>
date : Fri, 27 Aug 2021 23:33:23 -0700
If the system crashed between CREATE TABLESPACE and the next checkpoint,
the result could be some files in the tablespace unexpectedly containing
no rows. Affected files would be those for which the system did not
write WAL; see the wal_skip_threshold documentation. Before v13, a
different set of conditions governed the writing of WAL; see v12's
<sect2 id="populate-pitr">. (The v12 conditions were broader in some
ways and narrower in others.) Users may want to audit non-default
tablespaces for unexpected short files. The bug could have truncated an
index without affecting the associated table, and reindexing the index
would fix that particular problem.
This fixes the bug by making create_tablespace_directories() more like
TablespaceCreateDbspace(). create_tablespace_directories() was
recursively removing tablespace contents, reasoning that WAL redo would
recreate everything removed that way. That assumption holds for other
wal_level values. Under wal_level=minimal, the old approach could
delete files for which no other copy existed. Back-patch to 9.6 (all
supported versions).
Reviewed by Robert Haas and Prabhat Sahu. Reported by Robert Haas.
Discussion: https://postgr.es/m/CA+TgmoaLO9ncuwvr2nN-J4VEP5XyAcy=zKiHxQzBbFRxxGxm0w@mail.gmail.com
M src/backend/commands/tablespace.c
M src/test/recovery/t/018_wal_optimize.pl
Count SP-GiST index scans in pg_stat statistics.
commit : e84d4810cd47d83a6a864fa33db74bcc5570dd76
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 27 Aug 2021 19:42:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 27 Aug 2021 19:42:42 -0400
Somehow, spgist overlooked the need to call pgstat_count_index_scan().
Hence, pg_stat_all_indexes.idx_scan and equivalent columns never
became nonzero for an SP-GiST index, although the related per-tuple
counters worked fine.
This fix works a bit differently from other index AMs, in that the
counter increment occurs in spgrescan not spggettuple/spggetbitmap.
It looks like this won't make the user-visible semantics noticeably
different, so I won't go to the trouble of introducing an is-this-
the-first-call flag just to make the counter bumps happen in the
same places.
Per bug #17163 from Christian Quest. Back-patch to all supported
versions.
Discussion: https://postgr.es/m/17163-b8c5cc88322a5e92@postgresql.org
M src/backend/access/spgist/spgscan.c
Use maintenance_io_concurrency for ANALYZE prefetch
commit : 9efa998a6403c5fe973ce5801d09ffa63e706eb6
author : Stephen Frost <sfrost@snowman.net>
date : Fri, 27 Aug 2021 19:23:11 -0400
committer: Stephen Frost <sfrost@snowman.net>
date : Fri, 27 Aug 2021 19:23:11 -0400
When prefetching pages for ANALYZE, we should be using
maintenance_io_concurrenty (by calling
get_tablespace_maintenance_io_concurrency(), not
get_tablespace_io_concurrency()).
ANALYZE prefetching was introduced in c6fc50c, so back-patch to 14.
Backpatch-through: 14
Reported-By: Egor Rogov
Discussion: https://postgr.es/m/9beada99-34ce-8c95-fadb-451768d08c64%40postgrespro.ru
M src/backend/commands/analyze.c
docs: Add command tags for SQL commands
commit : 8f6c110349769e2b6375cd01e632199a104dc4a1
author : Stephen Frost <sfrost@snowman.net>
date : Fri, 27 Aug 2021 18:25:34 -0400
committer: Stephen Frost <sfrost@snowman.net>
date : Fri, 27 Aug 2021 18:25:34 -0400
Commit 6c3ffd6 added a couple new predefined roles but didn't properly
wrap the SQL commands mentioned in the description of those roles with
command tags, so add them now.
Backpatch-through: 14
Reported-by: Michael Banck
Discussion: https://postgr.es/m/606d8b1c.1c69fb81.3df04.1a99@mx.google.com
M doc/src/sgml/user-manag.sgml
docs: clarify bgw_restart_time documentation
commit : 652804ebde82b6bb04b427becbcb4f6bea02ff90
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 27 Aug 2021 22:50:19 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 27 Aug 2021 22:50:19 +0200
Author: Dave Cramer <davecramer@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CADK3HHLZmqAQZ2ByPDQQ9yhGqax36kksq6sDkV0yYzsxw6ipvQ@mail.gmail.com
M doc/src/sgml/bgworker.sgml
track_io_timing logging: Don't special case 0 ms.
commit : 6a1095234e0fcfa5c2f618c7c841e8ffe3c6c712
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 27 Aug 2021 13:33:58 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 27 Aug 2021 13:33:58 -0700
Adjust track_io_timing related logging code added by commit 94d13d474d.
Make it consistent with other nearby autovacuum and autoanalyze logging
code by removing logic that suppressed zero millisecond outputs.
log_autovacuum_min_duration log output now reliably shows "read:" and
"write:" millisecond-based values in its report (when track_io_timing is
enabled).
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Stephen Frost <sfrost@snowman.net>
Discussion: https://postgr.es/m/CAH2-WznW0FNxSVQMSRazAMYNfZ6DR_gr5WE78hc6E1CBkkJpzw@mail.gmail.com
Backpatch: 14-, where the track_io_timing logging was introduced.
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/analyze.c
Reorder log_autovacuum_min_duration log output.
commit : fd134f374ef7a8cc0b2d878a92dfe2c7088e084b
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 27 Aug 2021 13:08:39 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 27 Aug 2021 13:08:39 -0700
This order seems more natural. It starts with details that are
particular to heap and index data structures, and ends with system-level
costs incurred during the autovacuum worker's VACUUM/ANALYZE operation.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WzkzxK6ahA9xxsOftRtBX_R0swuHZsvo4QUbak1Bz7hb7Q@mail.gmail.com
Backpatch: 14-, which enhanced the log output in various ways.
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/analyze.c
Handle interaction of regexp's makesearch and MATCHALL more honestly.
commit : 3068e45799327298a3f4c22b03db2aa48e2ab0da
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 27 Aug 2021 12:18:58 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 27 Aug 2021 12:18:58 -0400
Second thoughts about commit 824bf7190: we apply makesearch() to
an NFA after having determined whether it is a MATCHALL pattern.
Prepending ".*" doesn't make it non-MATCHALL, but it does change the
maximum possible match length, and makesearch() failed to update that.
This has no ill effects given the stylized usage of search NFAs, but
it seems like it's better to keep the data structure consistent. In
particular, fixing this allows more honest handling of the MATCHALL
check in matchuntil(): we can now assert that maxmatchall is infinity,
instead of lamely assuming that it should act that way.
In passing, improve the code in dump[c]nfa so that infinite maxmatchall
is printed as "inf" not a magic number.
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/backend/regex/rege_dfa.c
contrib/amcheck: Add heapam CHECK_FOR_INTERRUPTS().
commit : ba05dfe943046820312eaa20995ffc626936b8be
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 26 Aug 2021 18:42:18 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 26 Aug 2021 18:42:18 -0700
Add a CHECK_FOR_INTERRUPTS() call to make heap relation verification
responsive to query cancellations.
Author: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://postgr.es/m/CAH2-Wzk-9RtQgb2QiuLv8j2O0j9tSFKPmmch5nWSZhguUxvbrw%40mail.gmail.com
Backpatch: 14-, where amcheck heap verification was introduced.
M contrib/amcheck/verify_heapam.c
Remove redundant test.
commit : ed740b06b18e1a23becd54c97ff229aba4c94349
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 25 Aug 2021 11:06:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 25 Aug 2021 11:06:34 -0400
The condition "context_start < context_end" is strictly weaker
than "context_end - context_start >= 50", so we don't need both.
Oversight in commit ffd3944ab, noted by tanghy.fnst.
In passing, line-wrap a nearby test to make it more readable.
Discussion: https://postgr.es/m/OS0PR01MB61137C4054774F44E3A9DC89FBC69@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/backend/utils/adt/jsonfuncs.c
Fix broken snapshot handling in parallel workers.
commit : 11c1239881b399d1fe87186f1f8a10a505d4a692
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 25 Aug 2021 08:32:04 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 25 Aug 2021 08:32:04 -0400
Pengchengliu reported an assertion failure in a parallel woker while
performing a parallel scan using an overflowed snapshot. The proximate
cause is that TransactionXmin was set to an incorrect value. The
underlying cause is incorrect snapshot handling in parallel.c.
In particular, InitializeParallelDSM() was unconditionally calling
GetTransactionSnapshot(), because I (rhaas) mistakenly thought that
was always retrieving an existing snapshot whereas, at isolation
levels less than REPEATABLE READ, it's actually taking a new one. So
instead do this only at higher isolation levels where there actually
is a single snapshot for the whole transaction.
By itself, this is not a sufficient fix, because we still need to
guarantee that TransactionXmin gets set properly in the workers. The
easiest way to do that seems to be to install the leader's active
snapshot as the transaction snapshot if the leader did not serialize a
transaction snapshot. This doesn't affect the results of future
GetTrasnactionSnapshot() calls since those have to take a new snapshot
anyway; what we care about is the side effect of setting TransactionXmin.
Report by Pengchengliu. Patch by Greg Nancarrow, except for some comment
text which I supplied.
Discussion: https://postgr.es/m/002f01d748ac$eaa781a0$bff684e0$@tju.edu.cn
M src/backend/access/transam/parallel.c
Fix typo
commit : a599109e591426689d5f79041af45525b00ef7b2
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Aug 2021 10:14:51 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Aug 2021 10:14:51 +0200
M src/include/utils/typcache.h
Fix incorrect merge in ECPG code with DECLARE
commit : 8ab3452df8d5ff26cf52c089b986256b0c02d555
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 25 Aug 2021 15:16:55 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 25 Aug 2021 15:16:55 +0900
The same condition was repeated twice when comparing the connection used
by existing declared statement with the one coming from a fresh DECLARE
statement. This had no consequences, but let's keep the code clean.
Oversight in f576de1.
Author: Shenhao Wang
Discussion: https://postgr.es/m/OSBPR01MB42149653BC0AB0A49D23C1B8F2C69@OSBPR01MB4214.jpnprd01.prod.outlook.com
Backpatch-through: 14
M src/interfaces/ecpg/preproc/ecpg.header
Fix toast rewrites in logical decoding.
commit : 9d7a80ce0185bd0c38bd973638bb7e9a854cf9f8
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 25 Aug 2021 10:10:50 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 25 Aug 2021 10:10:50 +0530
Commit 325f2ec555 introduced pg_class.relwrite to skip operations on
tables created as part of a heap rewrite during DDL. It links such
transient heaps to the original relation OID via this new field in
pg_class but forgot to do anything about toast tables. So, logical
decoding was not able to skip operations on internally created toast
tables. This leads to an error when we tried to decode the WAL for the
next operation for which it appeared that there is a toast data where
actually it didn't have any toast data.
To fix this, we set pg_class.relwrite for internally created toast tables
as well which allowed skipping operations on them during logical decoding.
Author: Bertrand Drouvot
Reviewed-by: David Zhang, Amit Kapila
Backpatch-through: 11, where it was introduced
Discussion: https://postgr.es/m/b5146fb1-ad9e-7d6e-f980-98ed68744a7c@amazon.com
M contrib/test_decoding/expected/toast.out
M contrib/test_decoding/sql/toast.sql
M src/backend/catalog/toasting.c
M src/backend/commands/cluster.c
M src/backend/commands/tablecmds.c
M src/include/catalog/toasting.h
M src/include/commands/tablecmds.h
Doc: Tweak function prototype indentation for consistency.
commit : 22583edee7f4c6be6894b03c5cd93fd02e2a826a
author : Etsuro Fujita <efujita@postgresql.org>
date : Wed, 25 Aug 2021 13:00:01 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Wed, 25 Aug 2021 13:00:01 +0900
M doc/src/sgml/fdwhandler.sgml
ecpg: Remove trailing period from error message.
commit : d207df71f4af7de8e3e865e2635ae3a6b3a98d51
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 25 Aug 2021 09:56:04 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 25 Aug 2021 09:56:04 +0900
This commit improves the ecpg's error message that commit f576de1db1 updated,
so that it gets rid of trailing period and uppercases the command name
in the error message.
Back-patch to v14 where the error message exists.
Author: Kyotaro Horiguchi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/20210819.170315.1413060634876301811.horikyota.ntt@gmail.com
M src/interfaces/ecpg/preproc/ecpg.header
Avoid using ambiguous word "positive" in error message.
commit : ec619102aa56d3356fb6550a343af01ad477e4a2
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 25 Aug 2021 11:46:25 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 25 Aug 2021 11:46:25 +0900
There are two identical error messages about valid value of modulus for
hash partition, in PostgreSQL source code. Commit 0e1275fb07 improved
only one of them so that ambiguous word "positive" was avoided there,
and forgot to improve the other. This commit improves the other.
Which would reduce translator burden.
Back-pach to v11 where the error message exists.
Author: Kyotaro Horiguchi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/20210819.170315.1413060634876301811.horikyota.ntt@gmail.com
M src/backend/parser/parse_utilcmd.c
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/create_table.out
Improve error message about valid value for distance in phrase operator.
commit : 1d0567ec61fbc429a7717ab8d5027952c806a8fb
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 25 Aug 2021 11:43:56 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 25 Aug 2021 11:43:56 +0900
The distance in phrase operator must be an integer value between zero
and MAXENTRYPOS inclusive. But previously the error message about
its valid value included the information about its upper limit
but not lower limit (i.e., zero). This commit improves the error message
so that it also includes the information about its lower limit.
Back-patch to v9.6 where full-text phrase search was supported.
Author: Kyotaro Horiguchi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/20210819.170315.1413060634876301811.horikyota.ntt@gmail.com
M src/backend/utils/adt/tsquery.c
Fix regexp misbehavior with capturing parens inside "{0}".
commit : 244dd79923a16afcf5f9a8dea53fc3af2ac0f7db
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 24 Aug 2021 16:37:27 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 24 Aug 2021 16:37:27 -0400
Regexps like "(.){0}...\1" drew an "invalid backreference number".
That's not unreasonable on its face, since the capture group will
never be matched if it's iterated zero times. However, other engines
such as Perl's don't complain about this, nor do we throw an error for
related cases such as "(.)|\1", even though that backref can never
succeed either. Also, if the zero-iterations case happens at runtime
rather than compile time --- say, "(x)*...\1" when there's no "x" to
be found --- that's not an error, we just deem the backref to not
match. Making this even less defensible, no error was thrown for
nested cases such as "((.)){0}...\2"; and to add insult to injury,
those cases could result in assertion failures instead. (It seems
that nothing especially bad happened in non-assert builds, though.)
Let's just fix it so that no error is thrown and instead the backref
is deemed to never match, so that compile-time detection of no
iterations behaves the same as run-time detection.
Per report from Mark Dilger. This appears to be an aboriginal error
in Spencer's library, so back-patch to all supported versions.
Pre-v14, it turns out to also be necessary to back-patch one aspect of
commits cb76fbd7e/00116dee5, namely to create capture-node subREs with
the begin/end states of their subexpressions, not the current lp/rp
of the outer parseqatom invocation. Otherwise delsub complains that
we're trying to disconnect a state from itself. This is a bit scary
but code examination shows that it's safe: in the pre-v14 code, if we
want to wrap iteration around the subexpression, the first thing we do
is overwrite the atom's begin/end fields with new states. So the
bogus values didn't survive long enough to be used for anything, except
if no iteration is required, in which case it doesn't matter.
Discussion: https://postgr.es/m/A099E4A8-4377-4C64-A98C-3DEDDC075502@enterprisedb.com
M src/backend/regex/regcomp.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
M src/test/regress/expected/regex.out
M src/test/regress/sql/regex.sql
Fix Alter Subscription's Add/Drop Publication behavior.
commit : 5cfcd46e9d297eac627bf3183272713112ac5f60
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 24 Aug 2021 08:38:11 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 24 Aug 2021 08:38:11 +0530
The current refresh behavior tries to just refresh added/dropped
publications but that leads to removing wrong tables from subscription. We
can't refresh just the dropped publication because it is quite possible
that some of the tables are removed from publication by that time and now
those will remain as part of the subscription. Also, there is a chance
that the tables that were part of the publication being dropped are also
part of another publication, so we can't remove those.
So, we decided that by default, add/drop commands will also act like
REFRESH PUBLICATION which means they will refresh all the publications. We
can keep the old behavior for "add publication" but it is better to be
consistent with "drop publication".
Author: Hou Zhijie
Reviewed-by: Masahiko Sawada, Amit Kapila
Backpatch-through: 14, where it was introduced
Discussion: https://postgr.es/m/OS0PR01MB5716935D4C2CC85A6143073F94EF9@OS0PR01MB5716.jpnprd01.prod.outlook.com
M doc/src/sgml/ref/alter_subscription.sgml
M src/backend/commands/subscriptioncmds.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
A src/test/subscription/t/021_alter_sub_pub.pl
Prevent regexp back-refs from sometimes matching when they shouldn't.
commit : 779557bd22895420b48eba409d2286f1dea08c06
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Aug 2021 17:41:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Aug 2021 17:41:07 -0400
The recursion in cdissect() was careless about clearing match data
for capturing parentheses after rejecting a partial match. This
could allow a later back-reference to succeed when by rights it
should fail for lack of a defined referent.
To fix, think a little more rigorously about what the contract
between different levels of cdissect's recursion needs to be.
With the right spec, we can fix this using fewer rather than more
resets of the match data; the key decision being that a failed
sub-match is now explicitly responsible for clearing any matches
it may have set.
There are enough other cross-checks and optimizations in the code
that it's not especially easy to exhibit this problem; usually, the
match will fail as-expected. Plus, regexps that are even potentially
vulnerable are most likely user errors, since there's just not much
point in writing a back-ref that doesn't always have a referent.
These facts perhaps explain why the issue hasn't been detected,
even though it's almost certainly a couple of decades old.
Discussion: https://postgr.es/m/151435.1629733387@sss.pgh.pa.us
M src/backend/regex/regexec.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
M src/test/regress/expected/regex.out
M src/test/regress/sql/regex.sql
Avoid creating archive status ".ready" files too early
commit : e3fb6170e58c4325cd1e1eb22f96ef43c3b4152a
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 23 Aug 2021 15:50:35 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 23 Aug 2021 15:50:35 -0400
WAL records may span multiple segments, but XLogWrite() does not
wait for the entire record to be written out to disk before
creating archive status files. Instead, as soon as the last WAL page of
the segment is written, the archive status file is created, and the
archiver may process it. If PostgreSQL crashes before it is able to
write and flush the rest of the record (in the next WAL segment), the
wrong version of the first segment file lingers in the archive, which
causes operations such as point-in-time restores to fail.
To fix this, keep track of records that span across segments and ensure
that segments are only marked ready-for-archival once such records have
been completely written to disk.
This has always been wrong, so backpatch all the way back.
Author: Nathan Bossart <bossartn@amazon.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Ryo Matsumura <matsumura.ryo@fujitsu.com>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/CBDDFA01-6E40-46BB-9F98-9340F4379505@amazon.com
M src/backend/access/transam/xlog.c
M src/backend/postmaster/walwriter.c
M src/include/access/xlog.h
M src/include/access/xlogdefs.h
Fix backup manifests to generate correct WAL-Ranges across timelines
commit : 65b649fecb0c0bd2c2c1f075f94f45a74cdc41be
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 23 Aug 2021 11:09:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 23 Aug 2021 11:09:54 +0900
In a backup manifest, WAL-Ranges stores the range of WAL that is
required for the backup to be valid. pg_verifybackup would then
internally use pg_waldump for the checks based on this data.
When the timeline where the backup started was more than 1 with a
history file looked at for the manifest data generation, the calculation
of the WAL range for the first timeline to check was incorrect. The
previous logic used as start LSN the start position of the first
timeline, but it needs to use the start LSN of the backup. This would
cause failures with pg_verifybackup, or any tools making use of the
backup manifests.
This commit adds a test based on a logic using a self-promoted node,
making it rather cheap.
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20210818.143031.1867083699202617521.horikyota.ntt@gmail.com
Backpatch-through: 13
M src/backend/replication/backup_manifest.c
M src/bin/pg_verifybackup/t/007_wal.pl
Improve error messages about misuse of SELECT INTO.
commit : 8f6a52196aceb11a78eb1d673a03ab3ab72da231
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 21 Aug 2021 10:22:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 21 Aug 2021 10:22:14 -0400
Improve two places in plpgsql, and one in spi.c, where an error
message would confusingly tell you that you couldn't use a SELECT
query, when what you had written *was* a SELECT query. The actual
problem is that you can't use SELECT ... INTO in these contexts,
but the messages failed to make that apparent. Special-case
SELECT INTO to make these errors more helpful.
Also, fix the same spots in plpgsql, as well as several messages
in exec_eval_expr(), to not quote the entire complained-of query or
expression in the primary error message. That behavior very easily
led to violating our message style guideline about keeping the primary
error message short and single-line. Also, since the important part
of the message was after the inserted text, it could make the real
problem very hard to see. We can report the query or expression as
the first line of errcontext instead.
Per complaint from Roger Mason. Back-patch to v14, since (a) some
of these messages are new in v14 and (b) v14's translatable strings
are still somewhat in flux. The problem's older than that of
course, but I'm hesitant to change the behavior further back.
Discussion: https://postgr.es/m/1914708.1629474624@sss.pgh.pa.us
M src/backend/executor/spi.c
M src/pl/plpgsql/src/expected/plpgsql_array.out
M src/pl/plpgsql/src/pl_exec.c
Fix performance bug in regexp's citerdissect/creviterdissect.
commit : 57a2d4a1b3bb48ec422a2701f8df026123a98a27
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 20 Aug 2021 14:19:04 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 20 Aug 2021 14:19:04 -0400
After detecting a sub-match "dissect" failure (i.e., a backref match
failure) in the i'th sub-match of an iteration node, we should proceed
by adjusting the attempted length of the i'th submatch. As coded,
though, these functions changed the attempted length of the *last*
sub-match, and only after exhausting all possibilities for that would
they back up to adjust the next-to-last sub-match, and then the
second-from-last, etc; all of which is wasted effort, since only
changing the start or length of the i'th sub-match can possibly make
it succeed. This oversight creates the possibility for exponentially
bad performance. Fortunately the problem is masked in most cases by
optimizations or constraints applied elsewhere; which explains why
we'd not noticed it before. But it is possible to reach the problem
with fairly simple, if contrived, regexps.
Oversight in my commit 173e29aa5. That's pretty ancient now,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/1808998.1629412269@sss.pgh.pa.us
M src/backend/regex/regexec.c
Remove --quiet option from pg_amcheck
commit : 92ce7f527960ca672d2ad70e61442f4e5b3bb641
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 20 Aug 2021 12:44:54 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 20 Aug 2021 12:44:54 +0200
Using --quiet in combination with --no-strict-names didn't work as
documented, a warning message was still emitted. Since the --quiet
flag was working in an unconventional way to other utilities, fix
by removing the functionality instead.
Backpatch through 14 where pg_amcheck was introduced.
Bug: 17148
Reported-by: Chen Jiaoqian <chenjq.jy@fujitsu.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://postgr.es/m/17148-b5087318e2b04fc6@postgresql.org
Backpatch-through: 14
M doc/src/sgml/ref/pg_amcheck.sgml
M src/bin/pg_amcheck/pg_amcheck.c
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/005_opclass_damage.pl
pg_amcheck: Fix block number parsing on command line
commit : 88cfcbb79f8064705362d8cfc0dff23d3c16195f
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 20 Aug 2021 07:48:22 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 20 Aug 2021 07:48:22 +0200
The previous code wouldn't handle higher block numbers on systems
where sizeof(long) == 4.
Reviewed-by: Mark Dilger <mark.dilger@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/6a10a211-872b-3c4c-106b-909ae5fefa61%40enterprisedb.com
M src/bin/pg_amcheck/pg_amcheck.c
Avoid trying to lock OLD/NEW in a rule with FOR UPDATE.
commit : 4649003933522282f03643c5a50fd2eb2a31a783
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Aug 2021 12:12:35 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Aug 2021 12:12:35 -0400
transformLockingClause neglected to exclude the pseudo-RTEs for
OLD/NEW when processing a rule's query. This led to odd errors
or even crashes later on. This bug is very ancient, but it's
not terribly surprising that nobody noticed, since the use-case
for SELECT FOR UPDATE in a non-view rule is somewhere between
thin and non-existent. Still, crashing is not OK.
Per bug #17151 from Zhiyong Wu. Thanks to Masahiko Sawada
for analysis of the problem.
Discussion: https://postgr.es/m/17151-c03a3e6e4ec9aadb@postgresql.org
M src/backend/parser/analyze.c
M src/include/nodes/parsenodes.h
M src/test/regress/expected/rules.out
M src/test/regress/sql/rules.sql
Unset MyBEEntry, making elog.c's call to pgstat_get_my_query_id() safe.
commit : 18914f24ec6e704e81a5528cf09f3d54b23ef12b
author : Andres Freund <andres@anarazel.de>
date : Thu, 19 Aug 2021 04:59:06 -0700
committer: Andres Freund <andres@anarazel.de>
date : Thu, 19 Aug 2021 04:59:06 -0700
Previously log messages late during shutdown could end up using either another
backend's PgBackendStatus (multi user) or segfault (single user) because
pgstat_get_my_query_id()'s check for !MyBEEntry didn't filter out use after
pgstat_beshutdown_hook().
This became a bug in 4f0b0966c86, but was a bit fishy before. But given
there's no known problematic cases before 14, it doesn't seem worth
backpatching further.
Also fixes a wrong filename in a comment, introduced in e1025044.
Reported-By: Andres Freund <andres@anarazel.de>
Reviewed-By: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://postgr.es/m/Julien Rouhaud <rjuju123@gmail.com>
Backpatch: 14-
M src/backend/utils/activity/backend_status.c
Fix typo in protocol.sgml.
commit : e1915646658def5dd87331ac77fb9d8d0abd763b
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 19 Aug 2021 09:05:54 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 19 Aug 2021 09:05:54 +0530
The 'Stream Stop' message is misspelled as 'Stream End' in the docs.
Author: Masahiko Sawada
Backpatch-through: 14, where it was introduced
Discussion: https://postgr.es/m/CAD21AoDeScrsHhLyEPYqN3sydg6PxAPVBboK=30xJfUVihNZDA@mail.gmail.com
M doc/src/sgml/protocol.sgml
Revert refactoring of hex code to src/common/
commit : 1900c140554efdcaa94134705e9be7ce1437be9c
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 19 Aug 2021 09:20:19 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 19 Aug 2021 09:20:19 +0900
This is a combined revert of the following commits:
- c3826f8, a refactoring piece that moved the hex decoding code to
src/common/. This code was cleaned up by aef8948, as it originally
included no overflow checks in the same way as the base64 routines in
src/common/ used by SCRAM, making it unsafe for its purpose.
- aef8948, a more advanced refactoring of the hex encoding/decoding code
to src/common/ that added sanity checks on the result buffer for hex
decoding and encoding. As reported by Hans Buschmann, those overflow
checks are expensive, and it is possible to see a performance drop in
the decoding/encoding of bytea or LOs the longer they are. Simple SQLs
working on large bytea values show a clear difference in perf profile.
- ccf4e27, a cleanup made possible by aef8948.
The reverts of all those commits bring back the performance of hex
decoding and encoding back to what it was in ~13. Fow now and
post-beta3, this is the simplest option.
Reported-by: Hans Buschmann
Discussion: https://postgr.es/m/1629039545467.80333@nidsa.net
Backpatch-through: 14
M src/backend/replication/backup_manifest.c
M src/backend/utils/adt/encode.c
M src/backend/utils/adt/varlena.c
M src/common/Makefile
D src/common/hex.c
D src/include/common/hex.h
M src/include/common/sha2.h
M src/include/utils/builtins.h
M src/tools/msvc/Mkvcbuild.pm
Fix check_agg_arguments' examination of aggregate FILTER clauses.
commit : 61f9dae2ce7680d51035e5bb8668cf5227346eac
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 18 Aug 2021 18:12:51 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 18 Aug 2021 18:12:51 -0400
Recursion into the FILTER clause was mis-implemented, such that a
relevant Var or Aggref at the very top of the FILTER clause would
be ignored. (Of course, that'd have to be a plain boolean Var or
boolean-returning aggregate.) The consequence would be
mis-identification of the correct semantic level of the aggregate,
which could lead to not-per-spec query behavior. If the FILTER
expression is an aggregate, this could also lead to failure to issue
an expected "aggregate function calls cannot be nested" error, which
would likely result in a core dump later on, since the planner and
executor aren't expecting such cases to appear.
The root cause is that commit b560ec1b0 blindly copied some code
that assumed it's recursing into a List, and thus didn't examine the
top-level node. To forestall questions about why this call doesn't
look like the others, as well as possible future copy-and-paste
mistakes, let's change all three check_agg_arguments_walker calls in
check_agg_arguments, even though only the one for the filter clause
is really broken.
Per bug #17152 from Zhiyong Wu. This has been wrong since we
implemented FILTER, so back-patch to all supported versions.
(Testing suggests that pre-v11 branches manage to avoid crashing
in the bad-Aggref case, thanks to "redundant" checks in ExecInitAgg.
But I'm not sure how thorough that protection is, and anyway the
wrong-behavior issue remains, so fix 9.6 and 10 too.)
Discussion: https://postgr.es/m/17152-c7f906cc1a88e61b@postgresql.org
M src/backend/parser/parse_agg.c
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql
Fix pg_amcheck --skip option parameter handling
commit : 5310c61ecc14f23d28429f055c968a97d5e8b39c
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 18 Aug 2021 11:23:43 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 18 Aug 2021 11:23:43 +0200
The skip options set for all-visible and all-frozen were incorrect
as they used space rather than hyphen, causing a syntax error when
invoked. Also, the option for not skipping any pages at all, none,
was documented but not implemented.
Backpatch through 14 where pg_amcheck was introduced.
Bug: #17149
Reported-by: Chen Jiaoqian <chenjq.jy@fujitsu.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/17149-5918ea748da36b15@postgresql.org
Backpatch-through: 14
M src/bin/pg_amcheck/pg_amcheck.c
Prevent ALTER TYPE/DOMAIN/OPERATOR from changing extension membership.
commit : 8f51ee63df3a9022cfd07d7482b8f3f21ff8f46d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Aug 2021 14:29:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Aug 2021 14:29:22 -0400
If recordDependencyOnCurrentExtension is invoked on a pre-existing,
free-standing object during an extension update script, that object
will become owned by the extension. In our current code this is
possible in three cases:
* Replacing a "shell" type or operator.
* CREATE OR REPLACE overwriting an existing object.
* ALTER TYPE SET, ALTER DOMAIN SET, and ALTER OPERATOR SET.
The first of these cases is intentional behavior, as noted by the
existing comments for GenerateTypeDependencies. It seems like
appropriate behavior for CREATE OR REPLACE too; at least, the obvious
alternatives are not better. However, the fact that it happens during
ALTER is an artifact of trying to share code (GenerateTypeDependencies
and makeOperatorDependencies) between the CREATE and ALTER cases.
Since an extension script would be unlikely to ALTER an object that
didn't already belong to the extension, this behavior is not very
troubling for the direct target object ... but ALTER TYPE SET will
recurse to dependent domains, and it is very uncool for those to
become owned by the extension if they were not already.
Let's fix this by redefining the ALTER cases to never change extension
membership, full stop. We could minimize the behavioral change by
only changing the behavior when ALTER TYPE SET is recursing to a
domain, but that would complicate the code and it does not seem like
a better definition.
Per bug #17144 from Alex Kozhemyakin. Back-patch to v13 where ALTER
TYPE SET was added. (The other cases are older, but since they only
affect the directly-named object, there's not enough of a problem to
justify changing the behavior further back.)
Discussion: https://postgr.es/m/17144-e67d7a8f049de9af@postgresql.org
M src/backend/catalog/pg_depend.c
M src/backend/catalog/pg_operator.c
M src/backend/catalog/pg_type.c
M src/backend/commands/operatorcmds.c
M src/backend/commands/typecmds.c
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_type.h
Improved ECPG warning as suggested by Michael Paquier and removed test case that triggers the warning during regression tests.
commit : e2d6da0708df2c2e244d8748c26ebc49c0963ed5
author : Michael Meskes <meskes@postgresql.org>
date : Tue, 17 Aug 2021 14:58:33 +0200
committer: Michael Meskes <meskes@postgresql.org>
date : Tue, 17 Aug 2021 14:58:33 +0200
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/test/expected/sql-declare.c
M src/interfaces/ecpg/test/expected/sql-declare.stderr
M src/interfaces/ecpg/test/expected/sql-declare.stdout
M src/interfaces/ecpg/test/sql/declare.pgc
Set type identifier on BIO
commit : b88377ad65dfa714e1bd1c041421446065c07c9c
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 17 Aug 2021 14:27:37 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Tue, 17 Aug 2021 14:27:37 +0200
In OpenSSL there are two types of BIO's (I/O abstractions):
source/sink and filters. A source/sink BIO is a source and/or
sink of data, ie one acting on a socket or a file. A filter
BIO takes a stream of input from another BIO and transforms it.
In order for BIO_find_type() to be able to traverse the chain
of BIO's and correctly find all BIO's of a certain type they
shall have the type bit set accordingly, source/sink BIO's
(what PostgreSQL implements) use BIO_TYPE_SOURCE_SINK and
filter BIO's use BIO_TYPE_FILTER. In addition to these, file
descriptor based BIO's should have the descriptor bit set,
BIO_TYPE_DESCRIPTOR.
The PostgreSQL implementation didn't set the type bits, which
went unnoticed for a long time as it's only really relevant
for code auditing the OpenSSL installation, or doing similar
tasks. It is required by the API though, so this fixes it.
Backpatch through 9.6 as this has been wrong for a long time.
Author: Itamar Gafni
Discussion: https://postgr.es/m/SN6PR06MB39665EC10C34BB20956AE4578AF39@SN6PR06MB3966.namprd06.prod.outlook.com
Backpatch-through: 9.6
M src/backend/libpq/be-secure-openssl.c
M src/interfaces/libpq/fe-secure-openssl.c
doc: \123 and \x12 escapes in COPY are in database encoding.
commit : f90c05959ec3fb712cba889817e0a0689fa6fa89
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 17 Aug 2021 10:00:06 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 17 Aug 2021 10:00:06 +0300
The backslash sequences, including \123 and \x12 escapes, are interpreted
after encoding conversion. The docs failed to mention that.
Backpatch to all supported versions.
Reported-by: Andreas Grob
Discussion: https://www.postgresql.org/message-id/17142-9181542ca1df75ab%40postgresql.org
M doc/src/sgml/ref/copy.sgml
Revert analyze support for partitioned tables
commit : b3d24cc0f0aa882ceec0a74a99f94166c6fc3247
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 16 Aug 2021 17:27:52 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 16 Aug 2021 17:27:52 -0400
This reverts the following commits:
1b5617eb844cd2470a334c1d2eec66cf9b39c41a Describe (auto-)analyze behavior for partitioned tables
0e69f705cc1a3df273b38c9883fb5765991e04fe Set pg_class.reltuples for partitioned tables
41badeaba8beee7648ebe7923a41c04f1f3cb302 Document ANALYZE storage parameters for partitioned tables
0827e8af70f4653ba17ed773f123a60eadd9f9c9 autovacuum: handle analyze for partitioned tables
There are efficiency issues in this code when handling databases with
large numbers of partitions, and it doesn't look like there isn't any
trivial way to handle those. There are some other issues as well. It's
now too late in the cycle for nontrivial fixes, so we'll have to let
Postgres 14 users continue to manually deal with ANALYZE their
partitioned tables, and hopefully we can fix the issues for Postgres 15.
I kept [most of] be280cdad298 ("Don't reset relhasindex for partitioned
tables on ANALYZE") because while we added it due to 0827e8af70f4, it is
a good bugfix in its own right, since it affects manual analyze as well
as autovacuum-induced analyze, and there's no reason to revert it.
I retained the addition of relkind 'p' to tables included by
pg_stat_user_tables, because reverting that would require a catversion
bump.
Also, in pg14 only, I keep a struct member that was added to
PgStat_TabStatEntry to avoid breaking compatibility with existing stat
files.
Backpatch to 14.
Discussion: https://postgr.es/m/20210722205458.f2bug3z6qzxzpx2s@alap3.anarazel.de
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/ref/analyze.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/pg_restore.sgml
M src/backend/access/common/reloptions.c
M src/backend/commands/analyze.c
M src/backend/commands/tablecmds.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/pgstat.c
M src/include/pgstat.h
Refresh apply delay on reload of recovery_min_apply_delay at recovery
commit : f83d80ea1e57927471848d901843ddc6fd5ecacc
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 16 Aug 2021 12:11:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 16 Aug 2021 12:11:49 +0900
This commit ensures that the wait interval in the replay delay loop
waiting for an amount of time defined by recovery_min_apply_delay is
correctly handled on reload, recalculating the delay if this GUC value
is updated, based on the timestamp of the commit record being replayed.
The previous behavior would be problematic for example with replay
still waiting even if the delay got reduced or just cancelled. If the
apply delay was increased to a larger value, the wait would have just
respected the old value set, finishing earlier.
Author: Soumyadeep Chakraborty, Ashwin Agrawal
Reviewed-by: Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/CAE-ML+93zfr-HLN8OuxF0BjpWJ17O5dv1eMvSE5jsj9jpnAXZA@mail.gmail.com
Backpatch-through: 9.6
M src/backend/access/transam/xlog.c
Add RISC-V spinlock support in s_lock.h.
commit : 4ffbd55d9346b6a8ba391c6df9a0f692c4b61ace
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 13 Aug 2021 13:58:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 13 Aug 2021 13:58:47 -0400
Like the ARM case, just use gcc's __sync_lock_test_and_set();
that will compile into AMOSWAP.W.AQ which does what we need.
At some point it might be worth doing some work on atomic ops
for RISC-V, but this should be enough for a creditable port.
Back-patch to all supported branches, just in case somebody
wants to try them on RISC-V.
Marek Szuba
Discussion: https://postgr.es/m/dea97b6d-f55f-1f6d-9109-504aa7dfa421@gentoo.org
M src/include/storage/s_lock.h
pg_amcheck: Message style and structuring improvements
commit : e546d1d052f220b077cf4d9fee8e6d5bf85911c5
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 13 Aug 2021 17:15:03 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 13 Aug 2021 17:15:03 +0200
M src/bin/pg_amcheck/pg_amcheck.c
M src/bin/pg_amcheck/t/004_verify_heapam.pl
Fix connection handling for DEALLOCATE and DESCRIBE statements
commit : a945f5527322d18d02990a9772a893f741e7d8df
author : Michael Meskes <meskes@postgresql.org>
date : Fri, 13 Aug 2021 10:34:04 +0200
committer: Michael Meskes <meskes@postgresql.org>
date : Fri, 13 Aug 2021 10:34:04 +0200
After binding a statement to a connection with DECLARE STATEMENT the connection
was still not used for DEALLOCATE and DESCRIBE statements. This patch fixes
that, adds a missing warning and cleans up the code.
Author: Hayato Kuroda
Reviewed-by: Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/TYAPR01MB5866BA57688DF2770E2F95C6F5069%40TYAPR01MB5866.jpnprd01.prod.outlook.com
M src/interfaces/ecpg/preproc/descriptor.c
M src/interfaces/ecpg/preproc/ecpg.addons
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/ecpg.type
M src/interfaces/ecpg/preproc/output.c
M src/interfaces/ecpg/preproc/type.h
M src/interfaces/ecpg/test/expected/sql-declare.c
M src/interfaces/ecpg/test/expected/sql-declare.stderr
M src/interfaces/ecpg/test/expected/sql-declare.stdout
M src/interfaces/ecpg/test/sql/declare.pgc
Fix sslsni connparam boolean check
commit : ffff00a3556734f859f375b8c76c89f1d2920bcd
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 13 Aug 2021 10:32:16 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 13 Aug 2021 10:32:16 +0200
The check for sslsni only checked for existence of the parameter
but not for the actual value of the param. This meant that the
SNI extension was always turned on. Fix by inspecting the value
of sslsni and only activate the SNI extension iff sslsni has been
enabled. Also update the docs to be more in line with how other
boolean params are documented.
Backpatch to 14 where sslsni was first implemented.
Reviewed-by: Tom Lane
Backpatch-through: 14, where sslni was added
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-secure-openssl.c
Fix incorrect hash table resizing code in simplehash.h
commit : dc23c77d07af086574124ea5ca65acf9360b8691
author : David Rowley <drowley@postgresql.org>
date : Fri, 13 Aug 2021 16:41:56 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 13 Aug 2021 16:41:56 +1200
This fixes a bug in simplehash.h which caused an incorrect size mask to be
used when the hash table grew to SH_MAX_SIZE (2^32). The code was
incorrectly setting the size mask to 0 when the hash tables reached the
maximum possible number of buckets. This would result always trying to
use the 0th bucket causing an infinite loop of trying to grow the hash
table due to there being too many collisions.
Seemingly it's not that common for simplehash tables to ever grow this big
as this bug dates back to v10 and nobody seems to have noticed it before.
However, probably the most likely place that people would notice it would
be doing a large in-memory Hash Aggregate with something close to at least
2^31 groups.
After this fix, the code now works correctly with up to within 98% of 2^32
groups and will fail with the following error when trying to insert any
more items into the hash table:
ERROR: hash table size exceeded
However, the work_mem (or hash_mem_multiplier in newer versions) settings
will generally cause Hash Aggregates to spill to disk long before reaching
that many groups. The minimal test case I did took a work_mem setting of
over 192GB to hit the bug.
simplehash hash tables are used in a few other places such as Bitmap Index
Scans, however, again the size that the hash table can become there is
also limited to work_mem and it would take a relation of around 16TB
(2^31) pages and a very large work_mem setting to hit this. With smaller
work_mem values the table would become lossy and never grow large enough
to hit the problem.
Author: Yura Sokolov
Reviewed-by: David Rowley, Ranier Vilela
Discussion: https://postgr.es/m/b1f7f32737c3438136f64b26f4852b96@postgrespro.ru
Backpatch-through: 10, where simplehash.h was added
M src/include/lib/simplehash.h
Make EXEC_BACKEND more convenient on macOS.
commit : 8201b60565b8da465e75da4e4c6d2f01bf976a43
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 13 Aug 2021 10:38:22 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 13 Aug 2021 10:38:22 +1200
It's hard to disable ASLR on current macOS releases, for testing with
-DEXEC_BACKEND. You could already set the environment variable
PG_SHMEM_ADDR to something not likely to collide with mappings created
earlier in process startup. Let's also provide a default value that
works on current releases and architectures, for developer convenience.
As noted in the pre-existing comment, this is a horrible hack, but
-DEXEC_BACKEND is only used by Unix-based PostgreSQL developers for
testing some otherwise Windows-only code paths, so it seems excusable.
Back-patch to all supported branches.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20210806032944.m4tz7j2w47mant26%40alap3.anarazel.de
M src/backend/port/sysv_shmem.c
Use appropriate tuple descriptor in FDW batching
commit : 886531f3f403cae67a4537f0a2a1edc730f6b793
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 12 Aug 2021 21:32:53 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 12 Aug 2021 21:32:53 +0200
The FDW batching code was using the same tuple descriptor both for all
slots (regular and plan slots), but that's incorrect - the subplan may
use a different descriptor. Currently this is benign, because batching
is used only for INSERTs, and in that case the descriptors always match.
But that would change if we allow batching UPDATEs.
Fix by copying the appropriate tuple descriptor. Backpatch to 14, where
the FDW batching was implemented.
Author: Amit Langote
Backpatch-through: 14, where FDW batching was added
Discussion: https://postgr.es/m/CA%2BHiwqEWd5B0-e-RvixGGUrNvGkjH2s4m95%3DJcwUnyV%3Df0rAKQ%40mail.gmail.com
M src/backend/executor/nodeModifyTable.c
Fix segfault during EvalPlanQual with mix of local and foreign partitions.
commit : 6458ed18fe40ac91f29496128dd1a49c0ef2db5b
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 12 Aug 2021 11:02:29 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 12 Aug 2021 11:02:29 +0300
It's not sensible to re-evaluate a direct-modify Foreign Update or Delete
during EvalPlanQual. However, ExecInitForeignScan() can still get called
if a table mixes local and foreign partitions. EvalPlanQualStart() left
the es_result_relations array uninitialized in the child EPQ EState, but
ExecInitForeignScan() still expected to find it. That caused a segfault.
Fix by skipping the es_result_relations lookup during EvalPlanQual
processing. To make things a bit more robust, also skip the
BeginDirectModify calls, and add a runtime check that ExecForeignScan()
is not called on direct-modify foreign scans during EvalPlanQual
processing.
This is new in v14, commit 1375422c782. Before that, EvalPlanQualStart()
copied the whole ResultRelInfo array to the EPQ EState. Backpatch to v14.
Report and diagnosis by Andrey Lepikhov.
Discussion: https://www.postgresql.org/message-id/cb2b808d-cbaa-4772-76ee-c8809bafcf3d%40postgrespro.ru
M src/backend/executor/nodeForeignscan.c
Fix failure of btree_gin indexscans with "char" type and </<= operators.
commit : a4957b5a72dc9e169b78604f67357bb3dc49d826
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 Aug 2021 18:10:30 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 Aug 2021 18:10:30 -0400
As a result of confusion about whether the "char" type is signed or
unsigned, scans for index searches like "col < 'x'" or "col <= 'x'"
would start at the middle of the index not the left end, thus missing
many or all of the entries they should find. Fortunately, this
is not a symptom of index corruption. It's only the search logic
that is broken, and we can fix it without unpleasant side-effects.
Per report from Jason Kim. This has been wrong since btree_gin's
beginning, so back-patch to all supported branches.
Discussion: https://postgr.es/m/20210810001649.htnltbh7c63re42p@jasonk.me
M contrib/btree_gin/btree_gin.c
M contrib/btree_gin/expected/char.out
Stamp 14beta3.
commit : 26f5391df63445692c88e4540dcbb558e567044c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Aug 2021 16:47:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Aug 2021 16:47:06 -0400
M configure
M configure.ac
Translation updates
commit : 7f7a9c20621f52867100fa3acfb4d453004ed924
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 9 Aug 2021 11:51:59 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 9 Aug 2021 11:51:59 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 1234b3cdae465246e534cc4114129f18d3c04c38
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/ja.po
M src/backend/po/uk.po
M src/bin/initdb/po/de.po
M src/bin/initdb/po/el.po
M src/bin/initdb/po/es.po
M src/bin/initdb/po/fr.po
M src/bin/initdb/po/ja.po
M src/bin/initdb/po/uk.po
M src/bin/pg_amcheck/nls.mk
M src/bin/pg_amcheck/po/de.po
M src/bin/pg_amcheck/po/el.po
A src/bin/pg_amcheck/po/ja.po
A src/bin/pg_amcheck/po/uk.po
M src/bin/pg_archivecleanup/po/el.po
M src/bin/pg_archivecleanup/po/es.po
M src/bin/pg_archivecleanup/po/ja.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_basebackup/po/ja.po
M src/bin/pg_basebackup/po/uk.po
M src/bin/pg_checksums/po/el.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_checksums/po/ja.po
M src/bin/pg_config/po/el.po
M src/bin/pg_config/po/es.po
M src/bin/pg_config/po/ja.po
M src/bin/pg_controldata/po/el.po
M src/bin/pg_controldata/po/es.po
M src/bin/pg_controldata/po/ja.po
M src/bin/pg_ctl/po/el.po
M src/bin/pg_ctl/po/es.po
M src/bin/pg_ctl/po/ja.po
M src/bin/pg_ctl/po/uk.po
M src/bin/pg_dump/po/el.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/ja.po
M src/bin/pg_dump/po/uk.po
M src/bin/pg_resetwal/nls.mk
M src/bin/pg_resetwal/po/de.po
A src/bin/pg_resetwal/po/el.po
M src/bin/pg_resetwal/po/es.po
M src/bin/pg_resetwal/po/ja.po
M src/bin/pg_rewind/nls.mk
A src/bin/pg_rewind/po/el.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/ja.po
M src/bin/pg_rewind/po/uk.po
M src/bin/pg_test_fsync/po/el.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_fsync/po/ja.po
M src/bin/pg_test_fsync/po/uk.po
M src/bin/pg_test_timing/po/el.po
M src/bin/pg_test_timing/po/ja.po
M src/bin/pg_test_timing/po/uk.po
M src/bin/pg_upgrade/po/de.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/ja.po
M src/bin/pg_upgrade/po/uk.po
M src/bin/pg_verifybackup/po/el.po
M src/bin/pg_verifybackup/po/es.po
M src/bin/pg_verifybackup/po/ja.po
M src/bin/pg_verifybackup/po/uk.po
M src/bin/pg_waldump/po/el.po
M src/bin/pg_waldump/po/es.po
M src/bin/pg_waldump/po/ja.po
M src/bin/psql/po/de.po
M src/bin/psql/po/el.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ja.po
M src/bin/psql/po/uk.po
M src/bin/scripts/po/de.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/ja.po
M src/bin/scripts/po/uk.po
M src/interfaces/ecpg/ecpglib/po/es.po
M src/interfaces/ecpg/ecpglib/po/ja.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/ja.po
M src/interfaces/ecpg/preproc/po/uk.po
M src/interfaces/libpq/po/de.po
M src/interfaces/libpq/po/el.po
M src/interfaces/libpq/po/es.po
M src/interfaces/libpq/po/uk.po
M src/pl/plperl/po/el.po
M src/pl/plperl/po/es.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpython/po/el.po
M src/pl/plpython/po/es.po
M src/pl/tcl/po/el.po
M src/pl/tcl/po/es.po
Doc: Fix misleading statement about VACUUM memory limits
commit : b5815dd00ad32b413070b088d083754d7ec42d37
author : David Rowley <drowley@postgresql.org>
date : Mon, 9 Aug 2021 16:46:49 +1200
committer: David Rowley <drowley@postgresql.org>
date : Mon, 9 Aug 2021 16:46:49 +1200
In ec34040af I added a mention that there was no point in setting
maintenance_work_limit to anything higher than 1GB for vacuum, but that
was incorrect as ginInsertCleanup() also looks at what
maintenance_work_mem is set to during VACUUM and that's not limited to
1GB.
Here I attempt to make it more clear that the limitation is only around
the number of dead tuple identifiers that we can collect during VACUUM.
I've also added a note to autovacuum_work_mem to mention this limitation.
I didn't do that in ec34040af as I'd had some wrong-headed ideas about
just limiting the maximum value for that GUC to 1GB.
Author: David Rowley
Discussion: https://postgr.es/m/CAApHDvpGwOAvunp-E-bN_rbAs3hmxMoasm5pzkYDbf36h73s7w@mail.gmail.com
Backpatch-through: 9.6, same as ec34040af
M doc/src/sgml/config.sgml
Use ExplainPropertyInteger for queryid in EXPLAIN
commit : c26552f4fc4fb72c64103ea877ea3c2a251856ad
author : David Rowley <drowley@postgresql.org>
date : Mon, 9 Aug 2021 15:46:28 +1200
committer: David Rowley <drowley@postgresql.org>
date : Mon, 9 Aug 2021 15:46:28 +1200
This saves a few lines of code. Also add a comment to mention why we use
ExplainPropertyInteger instead of ExplainPropertyUInteger given that
queryid is a uint64 type.
Author: David Rowley
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAApHDvqhSLYpSU_EqUdN39w9Uvb8ogmHV7_3YhJ0S3aScGBjsg@mail.gmail.com
Backpatch-through: 14, where this code was originally added
M src/backend/commands/explain.c
doc: mention pg_upgrade extension script
commit : 9268fc34526427a48bc27b5ac4727c7fb4cddf7a
author : Bruce Momjian <bruce@momjian.us>
date : Sun, 8 Aug 2021 21:05:46 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sun, 8 Aug 2021 21:05:46 -0400
Since commit e462856a7a, pg_upgrade automatically creates a script to
update extensions, so mention that instead of ALTER EXTENSION.
Backpatch-through: 9.6
M doc/src/sgml/ref/pgupgrade.sgml
Doc: remove bogus <indexterm> items.
commit : c905e64d42fe0725ab7fe84b70a41b4c381aee07
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Aug 2021 15:35:30 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Aug 2021 15:35:30 -0400
Copy-and-pasteo in 665c5855e, evidently. The 9.6 docs toolchain
whined about duplicate index entries, though our modern toolchain
doesn't. In any case, these GUCs surely are not about the
default settings of these values.
M doc/src/sgml/config.sgml
Rethink regexp engine's backref-related compilation state.
commit : 5227d99896726b3c889e388f14a3593e104e36ee
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Aug 2021 11:56:29 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Aug 2021 11:56:29 -0400
I had committer's remorse almost immediately after pushing cb76fbd7e,
upon finding that removing capturing subexpressions' subREs from the
data structure broke my proposed patch for REG_NOSUB optimization.
Revert that data structure change. Instead, address the concern
about not changing capturing subREs' endpoints by not changing the
endpoints. We don't need to, because the point of that bit was just
to ensure that the atom has endpoints distinct from the outer state
pair that we're stringing the branch between. We already made
suitable states in the parenthesized-subexpression case, so the
additional ones were just useless overhead. This seems more
understandable than Spencer's original coding, and it ought to be
a shade faster too by saving a few state creations and arc changes.
(I actually see a couple percent improvement on Jacobson's web
corpus, though that's barely above the noise floor so I wouldn't
put much stock in that result.)
Also, fix the logic added by ea1268f63 to ensure that the subRE
recorded in v->subs[subno] is exactly the one with capno == subno.
Spencer's original coding recorded the child subRE of the capture
node, which is okay so far as having the right endpoint states is
concerned, but as of cb76fbd7e the capturing subRE itself always
has those endpoints too. I think the inconsistency is confusing
for the REG_NOSUB optimization.
As before, backpatch to v14.
Discussion: https://postgr.es/m/0203588E-E609-43AF-9F4F-902854231EE7@enterprisedb.com
M src/backend/regex/regcomp.c
Make regexp engine's backref-related compilation state more bulletproof.
commit : 5e6ad63c6db79a5515e7a4117b5de053530fd7d9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Aug 2021 22:27:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Aug 2021 22:27:02 -0400
Up to now, we remembered the definition of a capturing parenthesis
subexpression by storing a pointer to the associated subRE node.
That was okay before, because that subRE didn't get modified anymore
while parsing the rest of the regexp. However, in the wake of
commit ea1268f63, that's no longer true: the outer invocation of
parseqatom() feels free to scribble on that subRE. This seems to
work anyway, because the states we jam into the child atom in the
"prepare a general-purpose state skeleton" stanza aren't really
semantically different from the original endpoints of the child atom.
But that would be mighty easy to break, and it's definitely not how
things worked before.
Between this and the issue fixed in the prior commit, it seems best
to get rid of this dependence on subRE nodes entirely. We don't need
the whole child subRE for future backrefs, only its starting and ending
NFA states; so let's just store pointers to those.
Also, in the corner case where we make an extra subRE to handle
immediately-nested capturing parentheses, it seems like it'd be smart
to have the extra subRE have the same begin/end states as the original
child subRE does (s/s2 not lp/rp). I think that linking it from lp to
rp might actually be semantically wrong, though since Spencer's original
code did it that way, I'm not totally certain. Using s/s2 is certainly
not wrong, in any case.
Per report from Mark Dilger. Back-patch to v14 where the problematic
patches came in.
Discussion: https://postgr.es/m/0203588E-E609-43AF-9F4F-902854231EE7@enterprisedb.com
M src/backend/regex/regcomp.c
Fix use-after-free issue in regexp engine.
commit : f42ea8350db22725a251e98a5dafb4d2539c800f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Aug 2021 22:05:27 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Aug 2021 22:05:27 -0400
Commit cebc1d34e taught parseqatom() to optimize cases where a branch
contains only one, "messy", atom by getting rid of excess subRE nodes.
The way we really should do that is to keep the subRE built for the
"messy" child atom; but to avoid changing parseqatom's nominal API,
I made it delete that node after copying its fields to the outer subRE
made by parsebranch(). It seems that that actually worked at the time;
but it became dangerous after ea1268f63, because that later commit
allowed the lower invocation of parse() to return a subRE that was also
pointed to by some v->subs[] entry. This meant we could wind up with a
dangling pointer in v->subs[], allowing a later backref to misbehave,
but only if that subRE struct had been reused in between. So the damage
seems confined to cases like '((...))...(...\2'.
To fix, do what I should have done before and modify parseqatom's API
to make it possible for it to remove the caller's subRE instead of the
callee's. That's safer because we know that subRE isn't complete yet,
so noplace else will have a pointer to it.
Per report from Mark Dilger. Back-patch to v14 where the problematic
patches came in.
Discussion: https://postgr.es/m/0203588E-E609-43AF-9F4F-902854231EE7@enterprisedb.com
M src/backend/regex/regcomp.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
pg_amcheck: Message style improvements
commit : 51b95fb257a24aa4186960be8abc277774466218
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Aug 2021 20:34:49 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Aug 2021 20:34:49 +0200
M src/bin/pg_amcheck/pg_amcheck.c
Really fix the ambiguity in REFRESH MATERIALIZED VIEW CONCURRENTLY.
commit : 2c915905e3e82b1db28dce8630f8d20b3db316f2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Aug 2021 13:29:32 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Aug 2021 13:29:32 -0400
Rather than trying to pick table aliases that won't conflict with
any possible user-defined matview column name, adjust the queries'
syntax so that the aliases are only used in places where they can't be
mistaken for column names. Mostly this consists of writing "alias.*"
not just "alias", which adds clarity for humans as well as machines.
We do have the issue that "SELECT alias.*" acts differently from
"SELECT alias", but we can use the same hack ruleutils.c uses for
whole-row variables in SELECT lists: write "alias.*::compositetype".
We might as well revert to the original aliases after doing this;
they're a bit easier to read.
Like 75d66d10e, back-patch to all supported branches.
Discussion: https://postgr.es/m/2488325.1628261320@sss.pgh.pa.us
M src/backend/commands/matview.c
M src/test/regress/expected/matview.out
M src/test/regress/sql/matview.sql
pg_amcheck: Add missing translation markers
commit : 9b0d71725eb2ca6ea0aaffdc38be599b90e7dc56
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Aug 2021 13:36:59 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Aug 2021 13:36:59 +0200
M src/bin/pg_amcheck/nls.mk
M src/bin/pg_amcheck/pg_amcheck.c
Message style improvements
commit : d954019f0a60bb989ef6fe8e478b764d0d5f301c
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Aug 2021 12:09:22 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Aug 2021 12:09:22 +0200
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/analyze.c
Don't elide casting to typmod -1.
commit : e5f6493e3584ea7eec1f992f87639e7f186ae03e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Aug 2021 17:32:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Aug 2021 17:32:54 -0400
Casting a value that's already of a type with a specific typmod
to an unspecified typmod doesn't do anything so far as run-time
behavior is concerned. However, it really ought to change the
exposed type of the expression to match. Up to now,
coerce_type_typmod hasn't bothered with that, which creates gotchas
in contexts such as recursive unions. If for example one side of
the union is numeric(18,3), but it needs to be plain numeric to
match the other side, there's no direct way to express that.
This is easy enough to fix, by inserting a RelabelType to update the
exposed type of the expression. However, it's a bit nervous-making
to change this behavior, because it's stood for a really long time.
(I strongly suspect that it's like this in part because the logic
pre-dates the introduction of RelabelType in 7.0. The commit log
message for 57b30e8e2 is interesting reading here.) As a compromise,
we'll sneak the change into 14beta3, and consider back-patching to
stable branches if no complaints emerge in the next three months.
Discussion: https://postgr.es/m/CABNQVagu3bZGqiTjb31a8D5Od3fUMs7Oh3gmZMQZVHZ=uWWWfQ@mail.gmail.com
M src/backend/parser/parse_coerce.c
M src/test/regress/expected/expressions.out
M src/test/regress/sql/expressions.sql
Adjust the integer overflow tests in the numeric code.
commit : 0325565702d8fd18ba66cdeaad4d7f43744525b2
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Fri, 6 Aug 2021 21:30:25 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Fri, 6 Aug 2021 21:30:25 +0100
Formerly, the numeric code tested whether an integer value of a larger
type would fit in a smaller type by casting it to the smaller type and
then testing if the reverse conversion produced the original value.
That's perfectly fine, except that it caused a test failure on
buildfarm animal castoroides, most likely due to a compiler bug.
Instead, do these tests by comparing against PG_INT16/32_MIN/MAX. That
matches existing code in other places, such as int84(), which is more
widely tested, and so is less likely to go wrong.
While at it, add regression tests covering the numeric-to-int8/4/2
conversions, and adjust the recently added tests to the style of
434ddfb79a (on the v11 branch) to make failures easier to diagnose.
Per buildfarm via Tom Lane, reviewed by Tom Lane.
Discussion: https://postgr.es/m/2394813.1628179479%40sss.pgh.pa.us
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Add missing message punctuation
commit : c3a135b41eed3c719a0c34b8cc072835ff21b129
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 6 Aug 2021 22:11:02 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 6 Aug 2021 22:11:02 +0200
M src/backend/catalog/pg_type.c
M src/test/regress/expected/multirangetypes.out
Fix wording
commit : acd6b6e637fb670003d4ac2477212b4d51016f9a
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 6 Aug 2021 20:55:59 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 6 Aug 2021 20:55:59 +0200
M doc/src/sgml/rules.sgml
M src/backend/utils/adt/multirangetypes_selfuncs.c
M src/backend/utils/adt/rangetypes_selfuncs.c
M src/backend/utils/adt/rangetypes_spgist.c
M src/bin/pg_resetwal/pg_resetwal.c
Doc: remove commit 2945a488a from v14 release notes.
commit : 64b7a8353214a3fa84942366ffb66b36c7f3d2e3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 5 Aug 2021 19:09:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 5 Aug 2021 19:09:24 -0400
Now that this has been back-patched, it's no longer a new feature
for v14.
M doc/src/sgml/release-14.sgml
postgres_fdw: Fix issues with generated columns in foreign tables.
commit : 588d3f597c67fac6191658c3befc3b9120772905
author : Etsuro Fujita <efujita@postgresql.org>
date : Thu, 5 Aug 2021 20:00:01 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Thu, 5 Aug 2021 20:00:01 +0900
postgres_fdw imported generated columns from the remote tables as plain
columns, and caused failures like "ERROR: cannot insert a non-DEFAULT
value into column "foo"" when inserting into the foreign tables, as it
tried to insert values into the generated columns. To fix, we do the
following under the assumption that generated columns in a postgres_fdw
foreign table are defined so that they represent generated columns in
the underlying remote table:
* Send DEFAULT for the generated columns to the foreign server on insert
or update, not generated column values computed on the local server.
* Add to postgresImportForeignSchema() an option "import_generated" to
include column generated expressions in the definitions of foreign
tables imported from a foreign server. The option is true by default.
The assumption seems reasonable, because that would make a query of the
postgres_fdw foreign table return values for the generated columns that
are consistent with the generated expression.
While here, fix another issue in postgresImportForeignSchema(): it tried
to include column generated expressions as column default expressions in
the foreign table definitions when the import_default option was enabled.
Per bug #16631 from Daniel Cherniy. Back-patch to v12 where generated
columns were added.
Discussion: https://postgr.es/m/16631-e929fe9db0ffc7cf%40postgresql.org
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Fix division-by-zero error in to_char() with 'EEEE' format.
commit : ecbdbdfd9056a55e38f7922565c7ce46b1321907
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 5 Aug 2021 09:27:35 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 5 Aug 2021 09:27:35 +0100
This fixes a long-standing bug when using to_char() to format a
numeric value in scientific notation -- if the value's exponent is
less than -NUMERIC_MAX_DISPLAY_SCALE-1 (-1001), it produced a
division-by-zero error.
The reason for this error was that get_str_from_var_sci() divides its
input by 10^exp, which it produced using power_var_int(). However, the
underflow test in power_var_int() causes it to return zero if the
result scale is too small. That's not a problem for power_var_int()'s
only other caller, power_var(), since that limits the rscale to 1000,
but in get_str_from_var_sci() the exponent can be much smaller,
requiring a much larger rscale. Fix by introducing a new function to
compute 10^exp directly, with no rscale limit. This also allows 10^exp
to be computed more efficiently, without any numeric multiplication,
division or rounding.
Discussion: https://postgr.es/m/CAEZATCWhojfH4whaqgUKBe8D5jNHB8ytzemL-PnRx+KCTyMXmg@mail.gmail.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
pgbench: When using pipelining only do PQconsumeInput() when necessary.
commit : fa604e0dd07a39ba34f93d06ded8243280dffdeb
author : Andres Freund <andres@anarazel.de>
date : Wed, 4 Aug 2021 19:19:44 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 4 Aug 2021 19:19:44 -0700
Up to now we did a PQconsumeInput() for each pipelined query, asking the OS
for more input - which it often won't have, as all results might already have
been sent. That turns out to have a noticeable performance impact.
Alvaro Herrera reviewed the idea to add the PQisBusy() check, but not this
concrete patch.
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20210720180039.23rivhdft3l4mayn@alap3.anarazel.de
Backpatch: 14, where libpq/pgbench pipelining was introduced.
M src/bin/pgbench/pgbench.c
Make vacuum_index_cleanup reloption RELOPT_TYPE_ENUM.
commit : e8086bd3ba0ab73a18ac2293dd14f488734126ec
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 3 Aug 2021 21:53:40 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 3 Aug 2021 21:53:40 -0700
Oversight in commit 3499df0d, which generalized the reloption as a way
of giving users a way to consistently avoid VACUUM's index bypass
optimization.
Per off-list report from Nikolay Shaplov.
Backpatch: 14-, where index cleanup reloption was extended.
M src/backend/access/common/reloptions.c
C comment: correct heading of extension query
commit : 3a0ba31a32f2def545399549c2ace6951f98e47c
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Aug 2021 12:26:08 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Aug 2021 12:26:08 -0400
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210803161345.GZ12533@telsasoft.com
Backpatch-through: 9.6
M src/bin/pg_upgrade/version.c
doc: interval spill method for units greater than months
commit : 2c92bae3e944d7159611e3010c34fd91b696a27f
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Aug 2021 12:17:58 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Aug 2021 12:17:58 -0400
Units are _truncated_ to months, but only in back branches since the
recent commit.
Reported-by: Bryn Llewellyn
Discussion: https://postgr.es/m/BDAE4B56-3337-45A2-AC8A-30593849D6C0@yugabyte.com
Backpatch-through: 9.6 to 14
M doc/src/sgml/datatype.sgml
pg_upgrade: warn about extensions that need updating
commit : 4051a7775752bb6643b2fef45f30142b65726b1a
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Aug 2021 11:58:15 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Aug 2021 11:58:15 -0400
Also create a script that can be run to update them.
Reported-by: Dave Cramer
Discussion: https://postgr.es/m/CADK3HHKawwbOcGwMGnDuAf3-U8YfvTcS8jqDv3UM=niijs3MMA@mail.gmail.com
Backpatch-through: 9.6
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/version.c
pg_upgrade: improve docs about extension upgrades
commit : 9f6215b4c9e1f208a520c255eff7c6c1dece93c9
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Aug 2021 11:27:33 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Aug 2021 11:27:33 -0400
The previous wording was unclear about the steps needed to upgrade
extensions, and how to update them after pg_upgrade.
Reported-by: Dave Cramer
Discussion: https://postgr.es/m/CADK3HHKawwbOcGwMGnDuAf3-U8YfvTcS8jqDv3UM=niijs3MMA@mail.gmail.com
Backpatch-through: 9.6
M doc/src/sgml/ref/pgupgrade.sgml
doc: mention inheritance's tableoid can be used in partitioning
commit : c362570e83ab08e41e18437e158a8542dd8c3556
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Aug 2021 11:11:51 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Aug 2021 11:11:51 -0400
Previously tableoid was not mentioned in the partition doc section. We
only had a link to the "all the normal rules" of inheritance section.
Reported-by: michal.palenik@freemap.sk
Discussion: https://postgr.es/m/162627031219.693.11508199541771263335@wrigleys.postgresql.org
Backpatch-through: 10
M doc/src/sgml/ddl.sgml
doc: add example of using pg_dump with GNU split and gzip
commit : 924e1d0da958c82b79b225b43672532f9f36df9b
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Aug 2021 10:57:32 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 3 Aug 2021 10:57:32 -0400
This is only possible with GNU split, not other versions like BSD split.
Reported-by: jim@jdoherty.net
Discussion: https://postgr.es/m/162653459215.701.6323855956817776386@wrigleys.postgresql.org
Backpatch-through: 9.6
M doc/src/sgml/backup.sgml
Fix oversight in commit 1ec7fca8592178281cd5cdada0f27a340fb813fc.
commit : fb234086fe81fb1848934b6e1f6382611fc9ad4f
author : Etsuro Fujita <efujita@postgresql.org>
date : Mon, 2 Aug 2021 12:45:01 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Mon, 2 Aug 2021 12:45:01 +0900
I failed to account for the possibility that when
ExecAppendAsyncEventWait() notifies multiple async-capable nodes using
postgres_fdw, a preceding node might invoke process_pending_request() to
process a pending asynchronous request made by a succeeding node. In
that case the succeeding node should produce a tuple to return to the
parent Append node from tuples fetched by process_pending_request() when
notified. Repair.
Per buildfarm via Michael Paquier. Back-patch to v14, like the previous
commit.
Thanks to Tom Lane for testing.
Discussion: https://postgr.es/m/YQP0UPT8KmPiHTMs%40paquier.xyz
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/executor/nodeAppend.c
Use elog, not Assert, to report failure to provide an outer snapshot.
commit : ec410c985e6d360f666e39be5609f3c4da5edc8f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 31 Jul 2021 11:50:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 31 Jul 2021 11:50:14 -0400
As of commit 84f5c2908, executing SQL commands (via SPI or otherwise)
requires having either an active Portal, or a caller-established
active snapshot. We were simply Assert'ing that that's the case.
But we've now had a couple different reports of people testing
extensions that didn't meet this requirement, and were confused by
the resulting crash. Let's convert the Assert to a test-and-elog,
in hopes of making the issue clearer for extension authors.
Per gripes from Liu Huailing and RekGRpth. Back-patch to v11,
like the prior commit.
Discussion: https://postgr.es/m/OSZPR01MB6215671E3C5956A034A080DFBEEC9@OSZPR01MB6215.jpnprd01.prod.outlook.com
Discussion: https://postgr.es/m/17035-14607d308ac8643c@postgresql.org
M src/backend/tcop/pquery.c
Fix corner-case errors and loss of precision in numeric_power().
commit : 0d6b87497e32a5b15be8fa3247b8fae48358bb1b
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sat, 31 Jul 2021 11:23:48 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sat, 31 Jul 2021 11:23:48 +0100
This fixes a couple of related problems that arise when raising
numbers to very large powers.
Firstly, when raising a negative number to a very large integer power,
the result should be well-defined, but the previous code would only
cope if the exponent was small enough to go through power_var_int().
Otherwise it would throw an internal error, attempting to take the
logarithm of a negative number. Fix this by adding suitable handling
to the general case in power_var() to cope with negative bases,
checking for integer powers there.
Next, when raising a (positive or negative) number whose absolute
value is slightly less than 1 to a very large power, the result should
approach zero as the power is increased. However, in some cases, for
sufficiently large powers, this would lose all precision and return 1
instead of 0. This was due to the way that the local_rscale was being
calculated for the final full-precision calculation:
local_rscale = rscale + (int) val - ln_dweight + 8
The first two terms on the right hand side are meant to give the
number of significant digits required in the result ("val" being the
estimated result weight). However, this failed to account for the fact
that rscale is clipped to a maximum of NUMERIC_MAX_DISPLAY_SCALE
(1000), and the result weight might be less then -1000, causing their
sum to be negative, leading to a loss of precision. Fix this by
forcing the number of significant digits calculated to be nonnegative.
It's OK for it to be zero (when the result weight is less than -1000),
since the local_rscale value then includes a few extra digits to
ensure an accurate result.
Finally, add additional underflow checks to exp_var() and power_var(),
so that they consistently return zero for cases like this where the
result is indistinguishable from zero. Some paths through this code
already returned zero in such cases, but others were throwing overflow
errors.
Dean Rasheed, reviewed by Yugo Nagata.
Discussion: http://postgr.es/m/CAEZATCW6Dvq7+3wN3tt5jLj-FyOcUgT5xNoOqce5=6Su0bCR0w@mail.gmail.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Fix range check in ECPG numeric to int conversion
commit : f051b87ace6dda42c191b77858c8d099af0de46a
author : John Naylor <john.naylor@postgresql.org>
date : Fri, 30 Jul 2021 13:50:23 -0400
committer: John Naylor <john.naylor@postgresql.org>
date : Fri, 30 Jul 2021 13:50:23 -0400
The previous coding guarded against -INT_MAX instead of INT_MIN,
leading to -2147483648 being rejected as out of range.
Per bug #17128 from Kevin Sweet
Discussion: https://www.postgresql.org/message-id/flat/17128-55a8a879727a3e3a%40postgresql.org
Reviewed-by: Tom Lane
Backpatch to all supported branches
M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/pgtypeslib/numeric.c
M src/interfaces/ecpg/test/expected/pgtypeslib-num_test.c
M src/interfaces/ecpg/test/expected/pgtypeslib-num_test.stderr
M src/interfaces/ecpg/test/expected/pgtypeslib-num_test.stdout
M src/interfaces/ecpg/test/pgtypeslib/num_test.pgc
Update obsolete comment that still referred to CheckpointLock
commit : 99da905d5c4335d2df80f5cf05dad046a10d30fb
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 30 Jul 2021 12:52:49 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 30 Jul 2021 12:52:49 +0300
CheckpointLock was removed in commit d18e75664a, and commit ce197e91d0
updated a leftover comment in CreateCheckPoint, but there was another
copy of it in CreateRestartPoint still.
M src/backend/access/transam/xlog.c
postgres_fdw: Fix handling of pending asynchronous requests.
commit : 1cf7fb376acd48710c1c2c5cd97d11cd82e71ae4
author : Etsuro Fujita <efujita@postgresql.org>
date : Fri, 30 Jul 2021 17:00:01 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Fri, 30 Jul 2021 17:00:01 +0900
A pending asynchronous request is handled by process_pending_request(),
which previously not only processed an in-progress remote query but
performed ExecForeignScan() to produce a tuple to return to the local
server asynchronously from the result of the remote query. But that led
to a server crash when executing a query or led to an "InstrStartNode
called twice in a row" or "InstrEndLoop called on running node" failure
when doing EXPLAIN ANALYZE of it, in cases where the plan tree for it
contained multiple async-capable nodes accessing the same
initplan/subplan that contained multiple async-capable nodes scanning
the same foreign tables as for the parent async-capable nodes, as
reported by Andrey Lepikhov. The reason is that the second step in
process_pending_request() invoked when executing the initplan/subplan
for one of the parent async-capable nodes caused recursive execution of
the initplan/subplan for another of the parent async-capable nodes.
To fix, split process_pending_request() into the two steps and postpone
the second step until ForeignAsyncConfigureWait() is called for each of
the pending asynchronous requests. Also, in ExecAppendAsyncEventWait()
we assumed that FDWs would register at least one wait event in a
WaitEventSet created there when they were called from
ForeignAsyncConfigureWait() in that function, but allow FDWs to register
zero wait events in the WaitEventSet; modify ExecAppendAsyncEventWait()
to just return in that case.
Oversight in commit 27e1f1456. Back-patch to v14 where that commit went
in.
Andrey Lepikhov and Etsuro Fujita
Discussion: https://postgr.es/m/fe5eaa19-1704-e4a4-76ee-3b9d37ade399@postgrespro.ru
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/executor/nodeAppend.c
Remove unused argument in apply_handle_commit_internal().
commit : f4b939f1a3720209e1941b88e11e02ea5e671c1b
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 30 Jul 2021 08:21:59 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 30 Jul 2021 08:21:59 +0530
Oversight in commit 0926e96c49.
Author: Masahiko Sawada
Reviewed-By: Amit Kapila
Backpatch-through: 14, where it was introduced
Discussion: https://postgr.es/m/CAD21AoDeScrsHhLyEPYqN3sydg6PxAPVBboK=30xJfUVihNZDA@mail.gmail.com
M src/backend/replication/logical/worker.c
Close yet another race condition in replication slot test code
commit : f951f6f69c7e77acdd04776168126f00e269dcfb
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 29 Jul 2021 17:09:06 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 29 Jul 2021 17:09:06 -0400
Buildfarm shows that this test has a further failure mode when a
checkpoint starts earlier than expected, so we detect a "checkpoint
completed" line that's not the one we want. Change the config to try
and prevent this.
Per buildfarm
While at it, update one comment that was forgotten in commit
d18e75664a2f.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/20210729.162038.534808353849568395.horikyota.ntt@gmail.com
M src/backend/access/transam/xlog.c
M src/test/recovery/t/019_replslot_limit.pl
docs: Fix bit_count example output
commit : c73bba23ef07f88ce3860a06bbf56180e00b185d
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 29 Jul 2021 21:39:40 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 29 Jul 2021 21:39:40 +0200
The returnvalue for the bit_count(::bytea) example was assuming a
non-default value of standard_conforming_strings. This was fixed
in the tests in commit ebedd0c78.
Author: wangzk.fnstxz@fujitsu.com
Discussion: https://postgr.es/m/OSZPR01MB6551FFAC1088C82C3D799BE0FAEB9@OSZPR01MB6551.jpnprd01.prod.outlook.com
Backpatch-through: 14
M doc/src/sgml/func.sgml
Improve libpq's handling of OOM during error message construction.
commit : 43f1d2ab361ccbd5ba8b53578fd4bcea9a6344a6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Jul 2021 13:33:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Jul 2021 13:33:31 -0400
Commit ffa2e4670 changed libpq so that multiple error reports
occurring during one operation (a connection attempt or query)
are accumulated in conn->errorMessage, where before new ones
usually replaced any prior error. At least in theory, that makes
us more vulnerable to running out of memory for the errorMessage
buffer. If it did happen, the user would be left with just an
empty-string error report, which is pretty unhelpful.
We can improve this by relying on pqexpbuffer.c's existing "broken
buffer" convention to track whether we've hit OOM for the current
operation's error string, and then substituting a constant "out of
memory" string in the small number of places where the errorMessage
is read out.
While at it, apply the same method to similar OOM cases in
pqInternalNotice and pqGetErrorNotice3.
Back-patch to v14 where ffa2e4670 came in. In principle this could
go back further; but in view of the lack of field reports, the
hazard seems negligible in older branches.
Discussion: https://postgr.es/m/530153.1627425648@sss.pgh.pa.us
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-int.h
Avoid calling TestLib::perl2host on a symlinked directory
commit : c42d3d04d7d03782ae179bf92ea0eb9f2fa8f409
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 29 Jul 2021 12:15:03 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 29 Jul 2021 12:15:03 -0400
Certain versions of msys2/Windows have been observed to resolve symlinks
in perl2host rather than just follow them. This defeats using a
symlinked shorter path to a longer path, and makes certain tests fail.
We therefore call perl2host on the parent directory of the symlink and
thereafter just use that result.
Apply to release 14 where the problem has been observed.
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
Make TestLib::perl2host more consistent and robust
commit : 68011e17d098da66070a2d648a609625241f73f6
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 29 Jul 2021 12:15:03 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 29 Jul 2021 12:15:03 -0400
Sometimes cygpath has been observed to return a path with a trailing
slash. That can cause problems, Also, make "cygpath" usage
consistent with "pwd -W" with respect to the use of forward slashes.
Backpatch to release 14 where the current code was introduced.
M src/test/perl/TestLib.pm
Add missing exit() in pg_verifybackup when failing to find pg_waldump
commit : 67445deb7eca32d25721dffb228b009bfbe415d5
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 29 Jul 2021 10:59:56 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 29 Jul 2021 10:59:56 +0900
pg_verifybackup needs by default pg_waldump to check after a range of
WAL segments required for a backup, except if --no-parse-wal is
specified. The code checked for the presence of the binary pg_waldump
in an installation and reported an error, but it forgot to properly
exit(). This could lead to confusing errors reported.
Reviewed-by: Robert Haas, Fabien Coelho
Discussion: https://postgr.es/m/YQDMdB+B68yePFeT@paquier.xyz
Backpatch-through: 13
M src/bin/pg_verifybackup/pg_verifybackup.c
Update minimum recovery point on truncation during WAL replay of abort record.
commit : f2a3d7404e5d4aa17dbdf7299b1f0d548fe59b9d
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 29 Jul 2021 01:30:02 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 29 Jul 2021 01:30:02 +0900
If a file is truncated, we must update minRecoveryPoint. Once a file is
truncated, there's no going back; it would not be safe to stop recovery
at a point earlier than that anymore.
Commit 7bffc9b7bf changed xact_redo_commit() so that it updates
minRecoveryPoint on truncation, but forgot to change xact_redo_abort().
Back-patch to all supported versions.
Reported-by: mengjuan.cmj@alibaba-inc.com
Author: Fujii Masao
Reviewed-by: Heikki Linnakangas
Discussion: https://postgr.es/m/b029fce3-4fac-4265-968e-16f36ff4d075.mengjuan.cmj@alibaba-inc.com
M src/backend/access/transam/xact.c
Disallow negative strides in date_bin()
commit : fc0d9b8c224ff6b84113cefdca1ba9dde879d863
author : John Naylor <john.naylor@postgresql.org>
date : Wed, 28 Jul 2021 11:22:58 -0400
committer: John Naylor <john.naylor@postgresql.org>
date : Wed, 28 Jul 2021 11:22:58 -0400
It's not clear what the semantics of negative strides would be, so throw
an error instead.
Per report from Bauyrzhan Sakhariyev
Reviewed-by: Tom Lane, Michael Paquier
Discussion: https://www.postgresql.org/message-id/CAKpL73vZmLuFVuwF26FJ%2BNk11PVHhAnQRoREFcA03x7znRoFvA%40mail.gmail.com
Backpatch to v14
M doc/src/sgml/func.sgml
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Doc: Clarify lock levels taken during ATTACH PARTITION
commit : 5a8a9be08307a1c06fbed4510667de6b4f40fe56
author : David Rowley <drowley@postgresql.org>
date : Wed, 28 Jul 2021 15:02:12 +1200
committer: David Rowley <drowley@postgresql.org>
date : Wed, 28 Jul 2021 15:02:12 +1200
It wasn't all that clear which lock levels, if any, would be held on the
DEFAULT partition during an ATTACH PARTITION operation.
Also, clarify which locks will be taken if the DEFAULT partition or the
table being attached are themselves partitioned tables.
Here I'm only backpatching to v12 as before then we obtained an ACCESS
EXCLUSIVE lock on the partitioned table. It seems much less relevant to
mention which locks are taken on other tables when the partitioned table
itself is locked with an ACCESS EXCLUSIVE lock.
Author: Matthias van de Meent, David Rowley
Discussion: https://postgr.es/m/CAEze2WiTB6iwrV8W_J=fnrnZ7fowW3qu-8iQ8zCHP3FiQ6+o-A@mail.gmail.com
Backpatch-through: 12
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/alter_table.sgml
Set pg_setting.pending_restart when pertinent config lines are removed
commit : ad3b40eb26a30d376e8448eb1170421501f0fc6b
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Jul 2021 15:44:12 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Jul 2021 15:44:12 -0400
This changes the behavior of examining the pg_file_settings view after
changing a config option that requires restart. The user needs to know
that any change of such options does not take effect until a restart,
and this worked correctly if the line is edited without removing it.
However, for the case where the line is removed altogether, the flag
doesn't get set, because a flag was only set in set_config_option, but
that's not called for lines removed. Repair.
(Ref.: commits 62d16c7fc561 and a486e35706ea)
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/202107262302.xsfdfc5sb7sh@alvherre.pgsql
M src/backend/utils/misc/guc-file.l
Fix bugs in polymorphic-argument resolution for multiranges.
commit : b7ea0e8c41f1e512923267a57cd08df63115b783
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 27 Jul 2021 15:01:49 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 27 Jul 2021 15:01:49 -0400
We failed to deal with an UNKNOWN-type input for
anycompatiblemultirange; that should throw an error indicating
that we don't know how to resolve the multirange type.
We also failed to infer the type of an anycompatiblerange output
from an anycompatiblemultirange input or vice versa.
Per bug #17066 from Alexander Lakhin. Back-patch to v14
where multiranges were added.
Discussion: https://postgr.es/m/17066-16a37f6223a8470b@postgresql.org
M src/backend/parser/parse_coerce.c
M src/test/regress/expected/polymorphism.out
M src/test/regress/expected/rangefuncs.out
M src/test/regress/sql/polymorphism.sql
Avoid using ambiguous word "non-negative" in error messages.
commit : fd90f6ba7a75f51d14dac27219bb3755b893e251
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 28 Jul 2021 01:20:16 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 28 Jul 2021 01:20:16 +0900
The error messages using the word "non-negative" are confusing
because it's ambiguous about whether it accepts zero or not.
This commit improves those error messages by replacing it with
less ambiguous word like "greater than zero" or
"greater than or equal to zero".
Also this commit added the note about the word "non-negative" to
the error message style guide, to help writing the new error messages.
When postgres_fdw option fetch_size was set to zero, previously
the error message "fetch_size requires a non-negative integer value"
was reported. This error message was outright buggy. Therefore
back-patch to all supported versions where such buggy error message
could be thrown.
Reported-by: Hou Zhijie
Author: Bharath Rupireddy
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/OS0PR01MB5716415335A06B489F1B3A8194569@OS0PR01MB5716.jpnprd01.prod.outlook.com
M contrib/postgres_fdw/option.c
M doc/src/sgml/sources.sgml
M src/backend/partitioning/partbounds.c
M src/backend/utils/adt/tsquery_op.c
M src/test/modules/test_shm_mq/test.c
M src/test/regress/expected/hash_part.out
doc: for various substring funcs, document if only first match
commit : 5a353a2d43eeab7554a6986b04b0a9d98ad55c92
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Jul 2021 22:54:35 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Jul 2021 22:54:35 -0400
Reported-by: troy@frericks.us
Discussion: https://postgr.es/m/162614304115.701.2392941350859387646@wrigleys.postgresql.org
Backpatch-through: 13
M doc/src/sgml/func.sgml
pg_resetxlog: add option to set oldest xid & use by pg_upgrade
commit : 695b4a113abca380fd7aed26cefa039e4e8cd569
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Jul 2021 22:38:14 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Jul 2021 22:38:14 -0400
Add pg_resetxlog -u option to set the oldest xid in pg_control.
Previously -x set this value be -2 billion less than the -x value.
However, this causes the server to immediately scan all relation's
relfrozenxid so it can advance pg_control's oldest xid to be inside the
autovacuum_freeze_max_age range, which is inefficient and might disrupt
diagnostic recovery. pg_upgrade will use this option to better create
the new cluster to match the old cluster.
Reported-by: Jason Harvey, Floris Van Nee
Discussion: https://postgr.es/m/20190615183759.GB239428@rfd.leadboat.com, 87da83168c644fd9aae38f546cc70295@opammb0562.comp.optiver.com
Author: Bertrand Drouvot
Backpatch-through: 9.6
M doc/src/sgml/ref/pg_resetwal.sgml
M src/bin/pg_resetwal/pg_resetwal.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/pg_upgrade.h
psql \dX: check schema when listing statistics objects
commit : 611e42444b121370f561d860552beb7d28dacbf8
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 26 Jul 2021 17:12:28 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 26 Jul 2021 17:12:28 +0200
Commit ad600bba04 added psql command \dX listing extended statistics
objects, but it failed to consider search_path when selecting the
elements so some of the returned elements might be invisible.
The visibility was already considered for tab completion (added by
commit d99d58cdc8), so adding it to the query is fairly simple.
Reported and fix by Justin Pryzby, regression tests by me. Backpatch
to PostgreSQL 14, where \dX was introduced.
Batchpatch-through: 14
Author: Justin Pryzby
Reviewed-by: Tatsuro Yamada
Discussion: https://postgr.es/m/c027a541-5856-75a5-0868-341301e1624b%40nttcom.co.jp_1
M src/bin/psql/describe.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
doc: Fix command example to run regression tests with PGOPTIONS
commit : f2a37ddbb10177fd731109df4a63d10150a91c0d
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 26 Jul 2021 16:27:01 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 26 Jul 2021 16:27:01 +0900
The documentation mentioned the use of log_checkpoints, that cannot be
used in this context. This commit replaces log_checkpoints with
force_parallel_mode, a developer option useful to perform checks related
to parallelism.
Oversight in 854434c.
Author: Haiying Tang
Discussion: https://postgr.es/m/OS0PR01MB6113954B883ACEB2DDC973F2FBE59@OS0PR01MB6113.jpnprd01.prod.outlook.com
Backpatch-through: 14
M doc/src/sgml/regress.sgml
Harden pg_stat_statements tests against CLOBBER_CACHE_ALWAYS.
commit : e6d9544681b3f5c66a16c72bfd8dca450ec7536a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 25 Jul 2021 23:25:15 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 25 Jul 2021 23:25:15 -0400
Turns out the buildfarm hasn't been testing this, which will soon change.
Julien Rouhaud, per report from me
Discussion: https://postgr.es/m/42557.1627229005@sss.pgh.pa.us
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
Fix incorrect comment for get_agg_clause_costs
commit : dece64a941457686184654968ac1ded58b7d4ee8
author : David Rowley <drowley@postgresql.org>
date : Mon, 26 Jul 2021 14:56:09 +1200
committer: David Rowley <drowley@postgresql.org>
date : Mon, 26 Jul 2021 14:56:09 +1200
Adjust the header comment in get_agg_clause_costs so that it matches what
the function currently does. No recursive searching has been done ever
since 0a2bc5d61. It also does not determine the aggtranstype like the
comment claimed. That's all done in preprocess_aggref().
preprocess_aggref also now determines the numOrderedAggs, so remove the
mention that get_agg_clause_costs also calculates "counts".
Normally, since this is just an adjustment of a comment it might not be
worth back-patching, but since this code is new to PG14 and that version
is still in beta, then it seems worth having the comments match.
Discussion: https://postgr.es/m/CAApHDvrrGrTJFPELrjx0CnDtz9B7Jy2XYW3Z2BKifAWLSaJYwQ@mail.gmail.com
Backpatch-though: 14
M src/backend/optimizer/prep/prepagg.c
Fix a couple of memory leaks in src/bin/pg_basebackup/
commit : b0d28671964d61416248789e3cee4508ac6281b4
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 26 Jul 2021 11:14:08 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 26 Jul 2021 11:14:08 +0900
These have been introduced by 7fbe0c8, and could happen for
pg_basebackup and pg_receivewal.
Per report from Coverity for the ones in walmethods.c, I have spotted
the ones in receivelog.c after more review.
Backpatch-through: 10
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_basebackup/walmethods.h
Get rid of artificial restriction on hash table sizes on Windows.
commit : b154ee63bb659ce280d486db6bbbe77ddec105c5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 25 Jul 2021 14:02:27 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 25 Jul 2021 14:02:27 -0400
The point of introducing the hash_mem_multiplier GUC was to let users
reproduce the old behavior of hash aggregation, i.e. that it could use
more than work_mem at need. However, the implementation failed to get
the job done on Win64, where work_mem is clamped to 2GB to protect
various places that calculate memory sizes using "long int". As
written, the same clamp was applied to hash_mem. This resulted in
severe performance regressions for queries requiring a bit more than
2GB for hash aggregation, as they now spill to disk and there's no
way to stop that.
Getting rid of the work_mem restriction seems like a good idea, but
it's a big job and could not conceivably be back-patched. However,
there's only a fairly small number of places that are concerned with
the hash_mem value, and it turns out to be possible to remove the
restriction there without too much code churn or any ABI breaks.
So, let's do that for now to fix the regression, and leave the
larger task for another day.
This patch does introduce a bit more infrastructure that should help
with the larger task, namely pg_bitutils.h support for working with
size_t values.
Per gripe from Laurent Hasson. Back-patch to v13 where the
behavior change came in.
Discussion: https://postgr.es/m/997817.1627074924@sss.pgh.pa.us
Discussion: https://postgr.es/m/MN2PR15MB25601E80A9B6D1BA6F592B1985E39@MN2PR15MB2560.namprd15.prod.outlook.com
M src/backend/executor/execGrouping.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeMemoize.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/pathnode.c
M src/include/miscadmin.h
M src/include/port/pg_bitutils.h
Deduplicate choice of horizon for a relation procarray.c.
commit : 3d0a4636aa4c976e971c05c77e162fc70c61f40b
author : Andres Freund <andres@anarazel.de>
date : Sat, 24 Jul 2021 20:14:03 -0700
committer: Andres Freund <andres@anarazel.de>
date : Sat, 24 Jul 2021 20:14:03 -0700
5a1e1d83022 was a minimal bug fix for dc7420c2c92. To avoid future bugs of
that kind, deduplicate the choice of a relation's horizon into a new helper,
GlobalVisHorizonKindForRel().
As the code in question was only introduced in dc7420c2c92 it seems worth
backpatching this change as well, otherwise 14 will look different from all
other branches.
A different approach to this was suggested by Matthias van de Meent.
Author: Andres Freund
Discussion: https://postgr.es/m/20210621122919.2qhu3pfugxxp3cji@alap3.anarazel.de
Backpatch: 14, like 5a1e1d83022
M src/backend/storage/ipc/procarray.c
Fix check for conflicting session- vs transaction-level locks.
commit : 712ba6b8b73870fa9e3336df8d88f4bc5f824112
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 24 Jul 2021 18:35:52 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 24 Jul 2021 18:35:52 -0400
We have an implementation restriction that PREPARE TRANSACTION can't
handle cases where both session-lifespan and transaction-lifespan locks
are held on the same lockable object. (That's because we'd otherwise
need to acquire a new PROCLOCK entry during post-prepare cleanup, which
is an operation that might fail. The situation can only arise with odd
usages of advisory locks, so removing the restriction is probably not
worth the amount of effort it would take.) AtPrepare_Locks attempted
to enforce this, but its logic was many bricks shy of a load, because
it only detected cases where the session and transaction locks had the
same lockmode. Locks of different modes on the same object would lead
to the rather unhelpful message "PANIC: we seem to have dropped a bit
somewhere".
To fix, build a transient hashtable with one entry per locktag,
not one per locktag + mode, and use that to detect conflicts.
Per bug #17122 from Alexander Pyhalov. This bug is ancient,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/17122-04f3c32098a62233@postgresql.org
M src/backend/storage/lmgr/lock.c
M src/test/regress/expected/prepared_xacts.out
M src/test/regress/expected/prepared_xacts_1.out
M src/test/regress/sql/prepared_xacts.sql
Make printf("%s", NULL) print "(null)" instead of crashing.
commit : 89ad14cd7870bce199448c527e3130c36f80d2bf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 24 Jul 2021 13:41:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 24 Jul 2021 13:41:17 -0400
We previously took a hard-line attitude that callers should never print
a null string pointer, and doing so is worthy of an assertion failure
or crash. However, we've long since flushed out any easy-to-find bugs
of that nature. What remains is a lot of code that perhaps could fail
that way in hard-to-reach corner cases. For example, in something as
simple as
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_OBJECT),
errmsg("constraint \"%s\" for table \"%s\" does not exist",
conname, get_rel_name(relid))));
one must wonder whether it's completely guaranteed that get_rel_name
cannot return NULL in this context. If such a situation did occur,
the existing policy converts what might be a pretty minor bug into
a server crash condition. This is not good for robustness.
Hence, let's follow the lead of glibc and print "(null)" instead
of failing. We should, of course, still consider it a bug if that
behavior is reachable in ordinary use; but crashing seems less
desirable than not crashing.
This fix works across-the-board in v12 and up, where we always use
src/port/snprintf.c. Before that, on most platforms we're at the mercy
of the local libc, but it appears that Solaris 10 is the only supported
platform where we'd still get a crash. Most other platforms such as
*BSD, macOS, and Solaris 11 have adopted glibc's behavior at some
point. (AIX and HPUX just print "" not "(null)", but that's close
enough.) I've not checked what Windows' native printf would do, but
it doesn't matter because we've long used snprintf.c on that platform.
In v12 and up, also const-ify related code so that we're not casting
away const on the constant string. This is just neatnik-ism, since
next to no compilers will warn about that.
Discussion: https://postgr.es/m/17098-b960f3616c861f83@postgresql.org
M src/port/snprintf.c
Remove configure-time thread safety checking (thread_test.c).
commit : d5e913a81cbab063c53b0a06dc8894da89390676
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 24 Jul 2021 12:16:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 24 Jul 2021 12:16:40 -0400
This testing was useful when it was written, nigh twenty years ago,
but it seems fairly pointless for any platform built in the last
dozen or more years. (Compare also the comments at 8a2121185.)
Also we now have reports that the test program itself fails under
ThreadSanitizer. Rather than invest effort in fixing it, let's
just drop it, and assume that the few people who still care
already know they need to use --disable-thread-safety.
Back-patch into v14, for consistency with 8a2121185.
Discussion: https://postgr.es/m/CADhDkKzPSiNvA3Hyq+wSR_icuPmazG0cFe=YnC3U-CFcYLc8Xw@mail.gmail.com
D config/thread_test.c
M configure
M configure.ac
Fix division by zero error in date_bin
commit : 81322fc409743f9ef169cb7bd89b0d0113a0aaa1
author : John Naylor <john.naylor@postgresql.org>
date : Thu, 22 Jul 2021 17:34:19 -0400
committer: John Naylor <john.naylor@postgresql.org>
date : Thu, 22 Jul 2021 17:34:19 -0400
Bauyrzhan Sakhariyev, via Github
Backpatch to v14
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
jit: Don't inline functions that access thread-locals.
commit : b1c1b7af573bbc9fe9039c82ffb7d3a3c378fe4a
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 22 Jul 2021 14:11:17 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 22 Jul 2021 14:11:17 +1200
Code inlined by LLVM can crash or fail with "Relocation type not
implemented yet!" if it tries to access thread local variables. Don't
inline such code.
Back-patch to 11, where LLVM arrived. Bug #16696.
Author: Dmitry Marakasov <amdmi3@amdmi3.ru>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/16696-29d944a33801fbfe@postgresql.org
M src/backend/jit/llvm/llvmjit_inline.cpp
Doc: improve documentation about exponentiation operator.
commit : 227c4e57d6a70961e015ed4185facfc638afd048
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 21 Jul 2021 18:03:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 21 Jul 2021 18:03:33 -0400
Now that we're not having to wedge this into the straitjacket of
the old operator table format, we can add another example to
clarify the point about left-to-right associativity.
Per suggestion from mdione at grulic.org.ar.
https://postgr.es/m/162661954599.693.13700316547731859171@wrigleys.postgresql.org
M doc/src/sgml/func.sgml
Document "B" and "us" as accepted units in postgres.conf.sample
commit : 65cc77c9847022f895a12012085606d8553c5fff
author : John Naylor <john.naylor@postgresql.org>
date : Wed, 21 Jul 2021 10:17:07 -0400
committer: John Naylor <john.naylor@postgresql.org>
date : Wed, 21 Jul 2021 10:17:07 -0400
In postgresql.conf, memory and file size GUCs can be specified with "B"
(bytes) as of b06d8e58b. Likewise, time GUCs can be specified with "us"
(microseconds) as of caf626b2c. Update postgres.conf.sample to reflect
that fact.
Pavel Luzanov
Backpatch to v12, which is the earliest version that allows both of
these units. A separate commit will document the "B" case for v11.
Discussion: https://www.postgresql.org/message-id/flat/f10d16fc-8fa0-1b3c-7371-cb3a35a13b7a%40postgrespro.ru
M src/backend/utils/misc/postgresql.conf.sample
Add missing check of noError parameter in euc_tw_and_big5.c
commit : e5cebe1ae8895fd9793344e70f58714e0a1742a4
author : John Naylor <john.naylor@postgresql.org>
date : Wed, 21 Jul 2021 09:09:32 -0400
committer: John Naylor <john.naylor@postgresql.org>
date : Wed, 21 Jul 2021 09:09:32 -0400
Oversight in ea1b99a66
Yukun Wang
Backpatch to v14 where this parameter was introduced
Discussion: https://www.postgresql.org/message-id/flat/OS0PR01MB6003FCEFF0201EF21685FD33B4E39%40OS0PR01MB6003.jpnprd01.prod.outlook.com
M src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
doc: Document that only superusers can use pg_import_system_collations().
commit : 9c1d56a9b08f87303749e60f9df8cf5c26c279eb
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 21 Jul 2021 13:52:37 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 21 Jul 2021 13:52:37 +0900
Back-patch to v10 where pg_import_system_collations() was added.
Author: Atsushi Torikoshi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/b7f484692a3e283710032e68b7f40617@oss.nttdata.com
M doc/src/sgml/func.sgml
doc: PG 14 relnote adjustments
commit : f8d1333dec312f96673efc6314a0f1df99e38abc
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 20 Jul 2021 19:37:04 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 20 Jul 2021 19:37:04 -0400
Reported-by: Elena Indrupskaya
Discussion: https://postgr.es/m/38555778-a56b-4aca-2581-e05582fc9bcf@postgrespro.ru
Author: Elena Indrupskaya
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
Fix corner-case uninitialized-variable issues in plpgsql.
commit : 899564e010abf216a90c48d9a1ce3728b8b00266
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 20 Jul 2021 13:01:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 20 Jul 2021 13:01:48 -0400
If an error was raised during our initial attempt to check whether
a successfully-compiled expression is "simple", subsequent calls of
exec_stmt_execsql would suppose that stmt->mod_stmt was already computed
when it had not been. This could lead to assertion failures in debug
builds; in production builds the effect would typically be to act as
if INTO STRICT had been specified even when it had not been. Of course
that only matters if the subsequent attempt to execute the expression
succeeds, so that the problem can only be reached by fixing a failure
in some referenced, inline-able SQL function and then retrying the
calling plpgsql function in the same session.
(There might be even-more-obscure ways to change the expression's
behavior without changing the plpgsql function, but that one seems
like the only one people would be likely to hit in practice.)
The most foolproof way to fix this would be to arrange for
exec_prepare_plan to not set expr->plan until we've finished the
subsidiary simple-expression check. But it seems hard to do that
without creating reference-count leak issues. So settle for documenting
the hazard in a comment and fixing exec_stmt_execsql to test separately
for whether it's computed stmt->mod_stmt. (That adds a test-and-branch
per execution, but hopefully that's negligible in context.) In v11 and
up, also fix exec_stmt_call which had a variant of the same issue.
Per bug #17113 from Alexander Lakhin. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/17113-077605ce00e0e7ec@postgresql.org
M src/pl/plpgsql/src/expected/plpgsql_simple.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
M src/pl/plpgsql/src/sql/plpgsql_simple.sql
Fix some issues with WAL segment opening for pg_receivewal --compress
commit : 3a0d2d0cbaf349a1aba928bb7b14edeaffab5212
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 20 Jul 2021 12:12:47 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 20 Jul 2021 12:12:47 +0900
The logic handling the opening of new WAL segments was fuzzy when using
--compress if a partial, non-compressed, segment with the same base name
existed in the repository storing those files. In this case, using
--compress would cause the code to first check for the existence and the
size of a non-compressed segment, followed by the opening of a new
compressed, partial, segment. The code was accidentally working
correctly on most platforms as the buildfarm has proved, except
bowerbird where gzflush() could fail in this code path. It is wrong
anyway to take the code path used pre-padding when creating a new
partial, non-compressed, segment, so let's fix it.
Note that this issue exists when users mix successive runs of
pg_receivewal with or without compression, as discovered with the tests
introduced by ffc9dda.
While on it, this refactors the code so as code paths that need to know
about the ".gz" suffix are down from four to one in walmethods.c, easing
a bit the introduction of new compression methods. This addresses a
second issue where log messages generated for an unexpected failure
would not show the compressed segment name involved, which was
confusing, printing instead the name of the non-compressed equivalent.
Reported-by: Georgios Kokolatos
Discussion: https://postgr.es/m/YPDLz2x3o1aX2wRh@paquier.xyz
Backpatch-through: 10
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_basebackup/walmethods.h
Doc: vacuum_multixact_failsafe_age is multixact-based.
commit : e1cdf61726e8d70971cf29d98c7ab5806875ac0e
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 19 Jul 2021 17:20:24 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 19 Jul 2021 17:20:24 -0700
Oversight in commit 1e55e7d1, which added a wraparound failsafe
mechanism to VACUUM.
Backpatch: 14-, where VACUUM failsafe was introduced.
M doc/src/sgml/config.sgml
vacuumdb: Correct comment about --force-index-cleanup.
commit : 9a3d41a26fbb00c67b1ac00306b21ab83f171e08
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 19 Jul 2021 17:06:47 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 19 Jul 2021 17:06:47 -0700
Commit 3499df0d added a comment that incorrectly suggested that
--force-index-cleanup did not appear in the same major version as the
similar --no-index-cleanup option. In fact, both options are new to
PostgreSQL 14.
Backpatch: 14-, where both options were introduced.
M src/bin/scripts/vacuumdb.c
Make new replication slot test code even less racy
commit : 1e8751380836056fdf546433dcf0328bc18c9313
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 19 Jul 2021 17:21:07 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 19 Jul 2021 17:21:07 -0400
Further fix the test code in ead9e51e8236, this time by waiting until
the checkpoint has completed before moving on; this ensures that the
WAL segment removal has already happened when we create the next slot.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/20210719.111318.2042379313472032754.horikyota.ntt@gmail.com
M src/test/recovery/t/019_replslot_limit.pl
Don't allow to set replication slot_name as ''.
commit : 40295d158fd4d462c55e6debae19dcd43ab530a6
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 19 Jul 2021 10:54:21 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 19 Jul 2021 10:54:21 +0530
We don't allow to create replication slot_name as an empty string ('') via
SQL API pg_create_logical_replication_slot() but it is allowed to be set
via Alter Subscription command. This will lead to apply worker repeatedly
keep trying to stream data via slot_name '' and the user is not allowed to
create the slot with that name.
Author: Japin Li
Reviewed-By: Ranier Vilela, Amit Kapila
Backpatch-through: 10, where it was introduced
Discussion: https://postgr.es/m/MEYP282MB1669CBD98E721C77CA696499B61A9@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
M src/backend/commands/subscriptioncmds.c
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
doc: Mention CASCADE/RESTRICT for DROP STATISTICS
commit : 6d0dc1a7da7942915c8160caa379c58dfd1488b8
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 19 Jul 2021 12:39:50 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 19 Jul 2021 12:39:50 +0900
This grammar has no effect as there are no dependencies on statistics,
but it is supported by the parser. This is more consistent with the
other DROP commands.
Author: Vignesh C
Discussion: https://postgr.es/m/CALDaNm1LA=yNmzcSfy+0oe6CEAgsxXRf_-UutE3ZncFi8QkFNQ@mail.gmail.com
Backpatch-through: 10
M doc/src/sgml/ref/drop_statistics.sgml
Support for unnest(multirange)
commit : 244ad5415557812a6ac4dc5d6e2ae908361d82c3
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 18 Jul 2021 21:07:24 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 18 Jul 2021 21:07:24 +0300
It has been spotted that multiranges lack of ability to decompose them into
individual ranges. Subscription and proper expanded object representation
require substantial work, and it's too late for v14. This commit
provides the implementation of unnest(multirange), which is quite trivial.
unnest(multirange) is defined as a polymorphic procedure.
Catversion is bumped.
Reported-by: Jonathan S. Katz
Discussion: https://postgr.es/m/flat/60258efe-bd7e-4886-82e1-196e0cac5433%40postgresql.org
Author: Alexander Korotkov
Reviewed-by: Justin Pryzby, Jonathan S. Katz, Zhihong Yu, Tom Lane
Reviewed-by: Alvaro Herrera
M doc/src/sgml/func.sgml
M src/backend/utils/adt/multirangetypes.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/multirangetypes.out
M src/test/regress/sql/multirangetypes.sql
Make new replication slot test code less racy
commit : d8f3b021c618bf58c19f88fb86476793fd650336
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 17 Jul 2021 13:19:17 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 17 Jul 2021 13:19:17 -0400
The new test code added in ead9e51e8236 is racy -- it hinges on
shared-memory state, which changes before the WARNING message is logged.
Put it the other way around.
Backpatch to 13.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/202107161809.zclasccpfcg3@alvherre.pgsql
M src/test/recovery/t/019_replslot_limit.pl
Doc: document the current-transaction-modes GUCs.
commit : 4466b38ec6ffd4ddea135c5baff978b7b43f8257
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 17 Jul 2021 11:52:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 17 Jul 2021 11:52:54 -0400
We had documentation of default_transaction_isolation et al,
but for some reason not of transaction_isolation et al.
AFAICS this is just an ancient oversight, so repair.
Per bug #17077 from Yanliang Lei.
Discussion: https://postgr.es/m/17077-ade8e166a01e1374@postgresql.org
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/set_transaction.sgml
Fix pg_dump for disabled triggers on partitioned tables
commit : 3c5b7c6286215de41ba0a327d065876ff8bc26b2
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 16 Jul 2021 17:29:22 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 16 Jul 2021 17:29:22 -0400
pg_dump failed to preserve the 'enabled' flag (which can be not only
disabled, but also REPLICA or ALWAYS) for partitions which had it
changed from their respective parents. Attempt to handle that by
including a definition for such triggers in the dump, but replace the
standard CREATE TRIGGER line with an ALTER TRIGGER line.
Backpatch to 11, where these triggers can exist. In branches 11 and 12,
pick up a few test lines from commit b9b408c48724 to verify that
pg_upgrade is okay with these arrangements.
Co-authored-by: Justin Pryzby <pryzby@telsasoft.com>
Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20200930223450.GA14848@telsasoft.com
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/test/regress/expected/sanity_check.out
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Preserve firing-on state when cloning row triggers to partitions
commit : eef92de11e50837e4a0d02fc7269fdba7c97e583
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 16 Jul 2021 13:01:43 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 16 Jul 2021 13:01:43 -0400
When triggers are cloned from partitioned tables to their partitions,
the 'tgenabled' flag (origin/replica/always/disable) was not propagated.
Make it so that the flag on the trigger on partition is initially set to
the same value as on the partitioned table.
Add a test case to verify the behavior.
Backpatch to 11, where this appeared in commit 86f575948c77.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/20200930223450.GA14848@telsasoft.com
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/include/commands/trigger.h
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Advance old-segment horizon properly after slot invalidation
commit : e5bcbb10707b844471c67d5e5fd8226d1891ee97
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 16 Jul 2021 12:07:30 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 16 Jul 2021 12:07:30 -0400
When some slots are invalidated due to the max_slot_wal_keep_size limit,
the old segment horizon should move forward to stay within the limit.
However, in commit c6550776394e we forgot to call KeepLogSeg again to
recompute the horizon after invalidating replication slots. In cases
where other slots remained, the limits would be recomputed eventually
for other reasons, but if all slots were invalidated, the limits would
not move at all afterwards. Repair.
Backpatch to 13 where the feature was introduced.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reported-by: Marcin Krupowicz <mk@071.ovh>
Discussion: https://postgr.es/m/17103-004130e8f27782c9@postgresql.org
M src/backend/access/transam/xlog.c
M src/backend/replication/slot.c
M src/include/replication/slot.h
M src/test/recovery/t/019_replslot_limit.pl
doc: Spell checking
commit : 525115a332af0fd479503550fd4f82551b797aea
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 16 Jul 2021 10:35:38 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 16 Jul 2021 10:35:38 +0200
M doc/src/sgml/bki.sgml
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/json.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/pgstatstatements.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/release-14.sgml
M doc/src/sgml/wal.sgml
docs: fix inconsistencies in markup and case
commit : 9ffe128a6d616675b559a14a89f8b42a30084323
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 15 Jul 2021 23:22:59 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 15 Jul 2021 23:22:59 +0200
Ensure to properly mark up function parameters in text with <parameter>,
avoid using <acronym> for terms which aren't acronyms and properly place
the ", and" in a value list. The acronym removal is a follow-up to commit
fb72a7b8c3 which removed it for minmax-multi. In passing, also fix an
incorrectly cased word.
Author: Ekaterina Kiryanova <e.kiryanova@postgrespro.ru>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/c050ecbc-80b2-b360-3c1d-9fe6a6a11bb5@postgrespro.ru
Backpatch-through: v14
M doc/src/sgml/amcheck.sgml
M doc/src/sgml/brin.sgml
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
Ensure HAVE_DECL_XXX macros in MSVC builds match those in Unix.
commit : 081e86bd9e7e00af3cbf9bcdec8277f02d22e6b4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 15 Jul 2021 11:00:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 15 Jul 2021 11:00:43 -0400
Autoconf's AC_CHECK_DECLS() always defines HAVE_DECL_whatever
as 1 or 0, but some of the entries in msvc/Solution.pm showed
such symbols as "undef" instead of 0. Fix that for consistency.
There's no live bug in current usages AFAICS, but it's not hard
to imagine one creeping in if more-complex #if tests get added.
Back-patch to v13, which is as far back as Solution.pm contains
this data. The inconsistency still exists in the manually-filled
pg_config_ext.h.win32 files of older branches; but as long as the
problem is only latent, it doesn't seem worth the trouble to
clean things up there.
Discussion: https://postgr.es/m/3185430.1626133592@sss.pgh.pa.us
M src/tools/msvc/Solution.pm
Fix small inconsistencies in catalog definition of multirange operators
commit : 4d39d4e639b44503c9950f61ec882113385db7ec
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 15 Jul 2021 14:18:19 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 15 Jul 2021 14:18:19 +0300
This commit fixes the description of a couple of multirange operators and
oprjoin for another multirange operator. The change of oprjoin is more
cosmetic since both old and new functions return the same constant.
These cosmetic changes don't worth catalog incompatibility between 14beta2
and 14beta3. So, catversion isn't bumped.
Discussion: https://postgr.es/m/CAPpHfdv9OZEuZDqOQoUKpXhq%3Dmc-qa4gKCPmcgG5Vvesu7%3Ds1w%40mail.gmail.com
Backpatch-throgh: 14
M src/include/catalog/pg_operator.dat
Remove unnecessary assertion in postmaster.c
commit : b90063511a3ee41d037b2da0d4e0b3bc399140c5
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 15 Jul 2021 15:00:52 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 15 Jul 2021 15:00:52 +0900
A code path asserted that the archiver was dead, but a check made that
impossible to happen.
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/CALj2ACW=CYE1ars+2XyPTEPq0wQvru4c0dPZ=Nrn3EqNBkksvQ@mail.gmail.com
Backpatch-throgh: 14
M src/backend/postmaster/postmaster.c
Clarify description of pg_stat_statements columns
commit : 3b57d5af7435d0d21bdec392dc1ce7c13871df8b
author : Magnus Hagander <magnus@hagander.net>
date : Wed, 14 Jul 2021 11:04:15 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Wed, 14 Jul 2021 11:04:15 +0200
Reported-By: Peter Eisentraut
Backpatch-through: 14
Discussion: https://postgr.es/m/8f5e63b8-e8ed-0f80-d8c4-68222624c200@enterprisedb.com
M doc/src/sgml/pgstatstatements.sgml
Fix unexpected error messages for various flavors of ALTER TABLE
commit : 0c83eb2e0edb42f54a37fdeac85fb80eb5de2cca
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 14 Jul 2021 17:15:01 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 14 Jul 2021 17:15:01 +0900
Some commands of ALTER TABLE could fail with the following error:
ERROR: "tab" is of the wrong type
This error is unexpected, as all the code paths leading to
ATWrongRelkindError() should use a supported set of relkinds to generate
correct error messages. This commit closes the gap with such mistakes,
by adding all the missing relkind combinations. Tests are added to
check all the problems found. Note that some combinations are not used,
but these are left around as it could have an impact on applications
relying on this code.
2ed532e has done a much larger refactoring on HEAD to make such error
messages easier to manage in the long-term, so nothing is needed there.
Author: Kyotaro Horiguchi
Reviewed-by: Peter Eisentraut, Ahsan Hadi, Michael Paquier
Discussion: https://postgr.es/m/20210216.181415.368926598204753659.horikyota.ntt@gmail.com
Backpatch-through: 11
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/foreign_data.out
M src/test/regress/sql/alter_table.sql
M src/test/regress/sql/foreign_data.sql
Fix lack of message pluralization
commit : b4842a8d085e970ff24667954c0d861e911e16c1
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 14 Jul 2021 09:15:14 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 14 Jul 2021 09:15:14 +0200
M src/backend/storage/ipc/signalfuncs.c
Change the name of the Result Cache node to Memoize
commit : 47ca4836441d1c24f75a94d43af8bd72d4c8d057
author : David Rowley <drowley@postgresql.org>
date : Wed, 14 Jul 2021 12:45:00 +1200
committer: David Rowley <drowley@postgresql.org>
date : Wed, 14 Jul 2021 12:45:00 +1200
"Result Cache" was never a great name for this node, but nobody managed
to come up with another name that anyone liked enough. That was until
David Johnston mentioned "Node Memoization", which Tom Lane revised to
just "Memoize". People seem to like "Memoize", so let's do the rename.
Reviewed-by: Justin Pryzby
Discussion: https://postgr.es/m/20210708165145.GG1176@momjian.us
Backpatch-through: 14, where Result Cache was introduced
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M doc/src/sgml/release-14.sgml
M src/backend/commands/explain.c
M src/backend/executor/Makefile
M src/backend/executor/execAmi.c
M src/backend/executor/execParallel.c
M src/backend/executor/execProcnode.c
R065 src/backend/executor/nodeResultCache.c src/backend/executor/nodeMemoize.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/README
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/pathnode.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
A src/include/executor/nodeMemoize.h
D src/include/executor/nodeResultCache.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/cost.h
M src/include/optimizer/pathnode.h
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/join.out
R088 src/test/regress/expected/resultcache.out src/test/regress/expected/memoize.out
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/sysviews.out
M src/test/regress/parallel_schedule
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/join.sql
R088 src/test/regress/sql/resultcache.sql src/test/regress/sql/memoize.sql
M src/test/regress/sql/partition_prune.sql
M src/tools/pgindent/typedefs.list
Rename debug_invalidate_system_caches_always to debug_discard_caches.
commit : 6201fa3c166fe2383dd44a9dd5082bc748c2937a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Jul 2021 15:01:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Jul 2021 15:01:01 -0400
The name introduced by commit 4656e3d66 was agreed to be unreasonably
long. To match this change, rename initdb's recently-added
--clobber-cache option to --discard-caches.
Discussion: https://postgr.es/m/1374320.1625430433@sss.pgh.pa.us
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/regress.sgml
M doc/src/sgml/release-14.sgml
M src/backend/utils/adt/lockfuncs.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/misc/guc.c
M src/bin/initdb/initdb.c
M src/include/pg_config_manual.h
M src/include/utils/inval.h
M src/pl/plpgsql/src/expected/plpgsql_cache.out
M src/pl/plpgsql/src/expected/plpgsql_cache_1.out
M src/pl/plpgsql/src/expected/plpgsql_record.out
M src/pl/plpgsql/src/sql/plpgsql_cache.sql
M src/pl/plpgsql/src/sql/plpgsql_record.sql
M src/test/isolation/specs/deadlock-soft-2.spec
Robustify tuplesort's free_sort_tuple function
commit : a92709fed63428a711f36270a2b4bee039399d60
author : David Rowley <drowley@postgresql.org>
date : Tue, 13 Jul 2021 13:27:44 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 13 Jul 2021 13:27:44 +1200
41469253e went to the trouble of removing a theoretical bug from
free_sort_tuple by checking if the tuple was NULL before freeing it. Let's
make this a little more robust by also setting the tuple to NULL so that
should we be called again we won't end up doing a pfree on the already
pfree'd tuple. Per advice from Tom Lane.
Discussion: https://postgr.es/m/3188192.1626136953@sss.pgh.pa.us
Backpatch-through: 9.6, same as 41469253e
M src/backend/utils/sort/tuplesort.c
Fix theoretical bug in tuplesort
commit : a3b8d91ccd7964e5bec1066c9e598277294efd33
author : David Rowley <drowley@postgresql.org>
date : Tue, 13 Jul 2021 12:42:04 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 13 Jul 2021 12:42:04 +1200
This fixes a theoretical bug in tuplesort.c which, if a bounded sort was
used in combination with a byval Datum sort (tuplesort_begin_datum), when
switching the sort to a bounded heap in make_bounded_heap(), we'd call
free_sort_tuple(). The problem was that when sorting Datums of a byval
type, the tuple is NULL and free_sort_tuple() would free the memory for it
regardless of that. This would result in a crash.
Here we fix that simply by adding a check to see if the tuple is NULL
before trying to disassociate and free any memory belonging to it.
The reason this bug is only theoretical is that nowhere in the current
code base do we do tuplesort_set_bound() when performing a Datum sort.
However, let's backpatch a fix for this as if any extension uses the code
in this way then it's likely to cause problems.
Author: Ronan Dunklau
Discussion: https://postgr.es/m/CAApHDvpdoqNC5FjDb3KUTSMs5dg6f+XxH4Bg_dVcLi8UYAG3EQ@mail.gmail.com
Backpatch-through: 9.6, oldest supported version
M src/backend/utils/sort/tuplesort.c
Probe for preadv/pwritev in a more macOS-friendly way.
commit : e75da4f1b6c63523021e550f1a92c67d2b8a92fb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Jul 2021 19:17:35 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Jul 2021 19:17:35 -0400
Apple's mechanism for dealing with functions that are available
in only some OS versions confuses AC_CHECK_FUNCS, and therefore
AC_REPLACE_FUNCS. We can use AC_CHECK_DECLS instead, so long as
we enable -Werror=unguarded-availability-new. This allows people
compiling for macOS to control whether or not preadv/pwritev are
used by setting MACOSX_DEPLOYMENT_TARGET, rather than supplying
a back-rev SDK. (Of course, the latter still works, too.)
James Hilliard
Discussion: https://postgr.es/m/20210122193230.25295-1-james.hilliard1@gmail.com
M configure
M configure.ac
M src/include/pg_config.h.in
M src/include/port/pg_iovec.h
M src/tools/msvc/Solution.pm
doc: Fix typo in function prototype
commit : 9b450b1f1e5bbeac414bbb8e6a2f08fa40f470ff
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 12 Jul 2021 22:07:35 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 12 Jul 2021 22:07:35 +0200
M doc/src/sgml/indexam.sgml
Remove dead assignment to local variable.
commit : 233280803cb6b3fe6b905abc1a28913855f40dd6
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 12 Jul 2021 11:13:33 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 12 Jul 2021 11:13:33 +0300
This should have been removed in commit 7e30c186da, which split the loop
into two. Only the first loop uses the 'from' variable; updating it in
the second loop is bogus. It was never read after the first loop, so this
was harmless and surely optimized away by the compiler, but let's be tidy.
Backpatch to all supported versions.
Author: Ranier Vilela
Discussion: https://www.postgresql.org/message-id/CAEudQAoWq%2BAL3BnELHu7gms2GN07k-np6yLbukGaxJ1vY-zeiQ%40mail.gmail.com
M src/backend/access/nbtree/nbtxlog.c
doc: Make release note entries more accurate
commit : 166fcf4a6364769501dd8562db927abefdd310b2
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 12 Jul 2021 08:45:34 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 12 Jul 2021 08:45:34 +0200
Two tsquery-related release note entries had inaccuracies in the
before and after examples. Clean that up.
M doc/src/sgml/release-14.sgml
Revert "Fix issues with Windows' stat() for files pending on deletion"
commit : 5e60237ad1b4e2d159aec48ffd9914a633d2d6e0
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 12 Jul 2021 14:46:40 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 12 Jul 2021 14:46:40 +0900
This reverts commit 54fb8c7, as per the issues reported by fairywren
when it comes to MinGW because of the lack of microsoft_native_stat()
there. Using just stat() for MSVC is not sufficient to take care of the
concurrency problems with files pending on deletion. It may be possible
to paint some __MINGW64__ in the code to switch to a different
implementation of stat() in this build context, but I am not sure either
if relying on the implementation of stat() in MinGW to take care of the
problems we are trying to fix is enough or not. So this needs more
study.
Discussion: https://postgr.es/m/YOvOlfRrIO0yGtgw@paquier.xyz
Backpatch-through: 14
M src/port/open.c
M src/port/win32stat.c
Fix issues with Windows' stat() for files pending on deletion
commit : de1510e2f5a1826890b206253016ebfc592c2f0a
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 12 Jul 2021 13:02:45 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 12 Jul 2021 13:02:45 +0900
The code introduced by bed9075 to enhance the stat() implementation on
Windows for file sizes larger than 4GB fails to properly detect files
pending for deletion with its method based on NtQueryInformationFile()
or GetFileInformationByHandleEx(), as proved by Alexander Lakhin in a
custom TAP test of his own.
The method used in the implementation of open() to sleep and loop when
when failing on ERROR_ACCESS_DENIED (EACCES) is showing much more
stability, so switch to this method. This could still lead to issues if
the permission problem stays around for much longer than the timeout of
1 second used, but that should (hopefully) never happen in
performance-critical paths. Still, there could be a point in increasing
the timeouts for the sake of machines that handle heavy loads.
Note that WIN32's open() now uses microsoft_native_stat() as it should
be similar to stat() when working around issues with concurrent file
deletions.
I have spent some time testing this patch with pgbench in combination
of the SQL functions from genfile.c, as well as running the TAP test
provided on the thread with MSVC builds, and this looks much more
stable than the previous method.
Author: Alexander Lakhin
Reviewed-by: Tom Lane, Michael Paquier, Justin Pryzby
Discussion: https://postgr.es/m/c3427edf-d7c0-ff57-90f6-b5de3bb62709@gmail.com
Backpatch-through: 14
M src/port/open.c
M src/port/win32stat.c
Lock the extension during ALTER EXTENSION ADD/DROP.
commit : 69dfc36fd54d1620209d5d40f3c68006d96b0046
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 11 Jul 2021 12:54:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 11 Jul 2021 12:54:24 -0400
Although we were careful to lock the object being added or dropped,
we failed to get any sort of lock on the extension itself. This
allowed the ALTER to proceed in parallel with a DROP EXTENSION,
which is problematic for a couple of reasons. If both commands
succeeded we'd be left with a dangling link in pg_depend, which
would cause problems later. Also, if the ALTER failed for some
reason, it might try to print the extension's name, and that could
result in a crash or (in older branches) a silly error message
complaining about extension "(null)".
Per bug #17098 from Alexander Lakhin. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/17098-b960f3616c861f83@postgresql.org
M src/backend/commands/extension.c
Fix pgbench timestamp bugs.
commit : 5614a0f78eaaa51c07bd753ca854de238c730b84
author : Thomas Munro <tmunro@postgresql.org>
date : Sun, 11 Jul 2021 19:50:55 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Sun, 11 Jul 2021 19:50:55 +1200
Commit 547f04e changed pgbench to use microsecond accounting, but
introduced a couple of logging and aggregation bugs:
1. We print Unix epoch timestamps so that you can correlate them with
other logs, but these were inadvertently changed to use a
system-dependent reference epoch. Compute Unix timestamps, and begin
aggregation intervals on the boundaries of whole Unix epoch seconds, as
before.
2. The user-supplied aggregation interval needed to be scaled.
Back-patch to 14.
Author: Fabien COELHO <coelho@cri.ensmp.fr>
Author: Yugo NAGATA <nagata@sraoss.co.jp>
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reported-by: YoungHwan Joo <rulyox@gmail.com>
Reported-by: Gregory Smith <gregsmithpgsql@gmail.com>
Discussion: https://postgr.es/m/CAF7igB1r6wRfSCEAB-iZBKxkowWY6%2BdFF2jObSdd9%2BiVK%2BvHJg%40mail.gmail.com
Discussion: https://postgr.es/m/CAHLJuCW_8Vpcr0%3Dt6O_gozrg3wXXWXZXDioYJd3NhvKriqgpfQ%40mail.gmail.com
M src/bin/pgbench/pgbench.c
Fix assign_record_type_typmod().
commit : 10a07973cf72b32f32f0e05ada16c84580496d93
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 9 Jul 2021 14:15:48 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 9 Jul 2021 14:15:48 -0700
If an error occurred in the wrong place, it was possible to leave an
unintialized entry in the hash table, leading to a crash. Fixed.
Also, be more careful about the order of operations so that an
allocation error doesn't leak memory in CacheMemoryContext or
unnecessarily advance NextRecordTypmod.
Backpatch through version 11. Earlier versions (prior to 35ea75632a5)
do not exhibit the problem, because an uninitialized hash entry
contains a valid empty list.
Author: Sait Talha Nisanci <Sait.Nisanci@microsoft.com>
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/HE1PR8303MB009069D476225B9A9E194B8891779@HE1PR8303MB0090.EURPRD83.prod.outlook.com
Backpatch-through: 11
M src/backend/utils/cache/typcache.c
Fix busted test for ldap_initialize.
commit : ebc346e5bb72b2a68258f1be8791f2b90191cd38
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 10 Jul 2021 13:19:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 10 Jul 2021 13:19:31 -0400
Sigh ... I was expecting AC_CHECK_LIB to do something it didn't,
namely update LIBS. This led to not finding ldap_initialize.
Fix by moving the probe for ldap_initialize. In some sense this
is more correct anyway, since (at least for now) we care about
whether ldap_initialize exists in libldap not libldap_r.
Per buildfarm member elver and local testing.
Discussion: https://postgr.es/m/17083-a19190d9591946a7@postgresql.org
M configure
M configure.ac
Fix numeric_mul() overflow due to too many digits after decimal point.
commit : 06883d58ff29cf4fb8c32fd13ce9947796b9fb0f
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sat, 10 Jul 2021 12:45:00 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sat, 10 Jul 2021 12:45:00 +0100
This fixes an overflow error when using the numeric * operator if the
result has more than 16383 digits after the decimal point by rounding
the result. Overflow errors should only occur if the result has too
many digits *before* the decimal point.
Discussion: https://postgr.es/m/CAEZATCUmeFWCrq2dNzZpRj5+6LfN85jYiDoqm+ucSXhb9U2TbA@mail.gmail.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Un-break AIX build, take 2.
commit : 9ffad7ae7ffc632ff8d3822916c431b7e4a4df38
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 Jul 2021 16:59:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 Jul 2021 16:59:07 -0400
I incorrectly diagnosed the reason why hoverfly is unhappy.
Looking closer, it appears that it fails to link libldap
unless libssl is also present; so the problem was my
idea of clearing LIBS before making the check. Revert
to essentially the original coding, except that instead
of failing when libldap_r isn't there, use libldap.
Per buildfarm member hoverfly.
Discussion: https://postgr.es/m/17083-a19190d9591946a7@postgresql.org
M configure
M configure.ac
libpq: Fix sending queries in pipeline aborted state
commit : 1beaa654da61ee66857a3c5125682b629f62fdf9
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 9 Jul 2021 15:57:59 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 9 Jul 2021 15:57:59 -0400
When sending queries in pipeline mode, we were careless about leaving
the connection in the right state so that PQgetResult would behave
correctly; trying to read further results after sending a query after
having read a result with an error would sometimes hang. Fix by
ensuring internal libpq state is changed properly. All the state
changes were being done by the callers of pqAppendCmdQueueEntry(); it
would have become too repetitious to have this logic in each of them, so
instead put it all in that function and relieve callers of the
responsibility.
Add a test to verify this case. Without the code fix, this new test
hangs sometimes.
Also, document that PQisBusy() would return false when no queries are
pending result. This is not intuitively obvious, and NULL would be
obtained by calling PQgetResult() at that point, which is confusing.
Wording by Boris Kolpackov.
In passing, fix bogus use of "false" to mean "0", per Ranier Vilela.
Backpatch to 14.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reported-by: Boris Kolpackov <boris@codesynthesis.com>
Discussion: https://postgr.es/m/boris.20210624103805@codesynthesis.com
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-exec.c
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Un-break AIX build.
commit : 7f2eca6f9db6e2227b87d03204e201ee6d759b5b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 Jul 2021 14:15:41 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 Jul 2021 14:15:41 -0400
In commit d0a02bdb8, I'd supposed that uniformly probing for
ldap_bind would make the intent clearer. However, that seems
not to work on AIX, for obscure reasons (maybe it's a macro
there?). Revert to the former behavior of probing
ldap_simple_bind for thread-safe cases and ldap_bind otherwise.
Per buildfarm member hoverfly.
Discussion: https://postgr.es/m/17083-a19190d9591946a7@postgresql.org
M configure
M configure.ac
Avoid creating a RESULT RTE that's marked LATERAL.
commit : 1d98fdaed89c00465ef68fa2804967ea27b03abc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 Jul 2021 13:38:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 Jul 2021 13:38:24 -0400
Commit 7266d0997 added code to pull up simple constant function
results, converting the RTE_FUNCTION RTE to a dummy RTE_RESULT
RTE since it no longer need be scanned. But I forgot to clear
the LATERAL flag if the RTE has it set. If the function reduced
to a constant, it surely contains no lateral references so this
simplification is logically OK. It's needed because various other
places will Assert that RESULT RTEs aren't LATERAL.
Per bug #17097 from Yaoguang Chen. Back-patch to v13 where the
faulty code came in.
Discussion: https://postgr.es/m/17097-3372ef9f798fc94f@postgresql.org
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Update configure's probe for libldap to work with OpenLDAP 2.5.
commit : 5620ec83362d08b9f86c90c97c0a70031c4d0b2c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 Jul 2021 12:38:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 Jul 2021 12:38:55 -0400
The separate libldap_r is gone and libldap itself is now always
thread-safe. Unfortunately there seems no easy way to tell by
inspection whether libldap is thread-safe, so we have to take
it on faith that libldap is thread-safe if there's no libldap_r.
That should be okay, as it appears that libldap_r was a standard
part of the installation going back at least 20 years.
Report and patch by Adrian Ho. Back-patch to all supported
branches, since people might try to build any of them with
a newer OpenLDAP.
Discussion: https://postgr.es/m/17083-a19190d9591946a7@postgresql.org
M configure
M configure.ac
M src/include/pg_config.h.in
M src/tools/msvc/Solution.pm
Reject cases where a query in WITH rewrites to just NOTIFY.
commit : 39b6e85f135f7a1dcf43c0551d7d10e8c57b7fce
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 Jul 2021 11:02:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 9 Jul 2021 11:02:26 -0400
Since the executor can't cope with a utility statement appearing
as a node of a plan tree, we can't support cases where a rewrite
rule inserts a NOTIFY into an INSERT/UPDATE/DELETE command appearing
in a WITH clause of a larger query. (One can imagine ways around
that, but it'd be a new feature not a bug fix, and so far there's
been no demand for it.) RewriteQuery checked for this, but it
missed the case where the DML command rewrites to *only* a NOTIFY.
That'd lead to crashes later on in planning. Add the missed check,
and improve the level of testing of this area.
Per bug #17094 from Yaoguang Chen. It's been busted since WITH
was introduced, so back-patch to all supported branches.
Discussion: https://postgr.es/m/17094-bf15dff55eaf2e28@postgresql.org
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Remove more obsolete comments about semaphores.
commit : 8d48a3436dd83f7d6e3cde2d76f91f49f27ba16d
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 9 Jul 2021 17:51:48 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 9 Jul 2021 17:51:48 +1200
Commit 6753333f stopped using semaphores as the sleep/wake mechanism for
heavyweight locks, but some obsolete references to that scheme remained
in comments. As with similar commit 25b93a29, back-patch all the way.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CA%2BhUKGLafjB1uzXcy%3D%3D2L3cy7rjHkqOVn7qRYGBjk%3D%3DtMJE7Yg%40mail.gmail.com
M src/backend/storage/lmgr/lwlock.c
Fix incorrect return value in pg_size_pretty(bigint)
commit : 6de3a21bbc0601b22cd465311d10978acca7cc0b
author : David Rowley <drowley@postgresql.org>
date : Fri, 9 Jul 2021 14:04:40 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 9 Jul 2021 14:04:40 +1200
Due to how pg_size_pretty(bigint) was implemented, it's possible that when
given a negative number of bytes that the returning value would not match
the equivalent positive return value when given the equivalent positive
number of bytes. This was due to two separate issues.
1. The function used bit shifting to convert the number of bytes into
larger units. The rounding performed by bit shifting is not the same as
dividing. For example -3 >> 1 = -2, but -3 / 2 = -1. These two
operations are only equivalent with positive numbers.
2. The half_rounded() macro rounded towards positive infinity. This meant
that negative numbers rounded towards zero and positive numbers rounded
away from zero.
Here we fix #1 by dividing the values instead of bit shifting. We fix #2
by adjusting the half_rounded macro always to round away from zero.
Additionally, adjust the pg_size_pretty(numeric) function to be more
explicit that it's using division rather than bit shifting. A casual
observer might have believed bit shifting was used due to a static
function being named numeric_shift_right. However, that function was
calculating the divisor from the number of bits and performed division.
Here we make that more clear. This change is just cosmetic and does not
affect the return value of the numeric version of the function.
Here we also add a set of regression tests both versions of
pg_size_pretty() which test the values directly before and after the
function switches to the next unit.
This bug was introduced in 8a1fab36a. Prior to that negative values were
always displayed in bytes.
Author: Dean Rasheed, David Rowley
Discussion: https://postgr.es/m/CAEZATCXnNW4HsmZnxhfezR5FuiGgp+mkY4AzcL5eRGO4fuadWg@mail.gmail.com
Backpatch-through: 9.6, where the bug was introduced.
M src/backend/utils/adt/dbsize.c
M src/test/regress/expected/dbsize.out
M src/test/regress/sql/dbsize.sql
doc: minor PG 14 relnote changes
commit : 9893d5124b1a0a2d84fb8038bde3bf11491abf7a
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 8 Jul 2021 18:59:19 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 8 Jul 2021 18:59:19 -0400
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
doc: PG 14 relnote, move system view items to the proper sect.
commit : 6f566f0e9b1aab41eea1ca1991e04e3d8a79c198
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 8 Jul 2021 13:08:09 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 8 Jul 2021 13:08:09 -0400
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
doc: improve PG 14 relnote item about adding btree pages to FSM
commit : 049d3617a3bedb2a1d97d1621f36c861f52269a7
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 8 Jul 2021 12:27:30 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 8 Jul 2021 12:27:30 -0400
Wording confirmed by Peter Geoghegan.
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
Fix crash in postgres_fdw for provably-empty remote UPDATE/DELETE.
commit : 30a35bca3f9306f87ca8db095ddac2ba62156934
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Jul 2021 15:21:25 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Jul 2021 15:21:25 -0400
In 86dc90056, I'd written find_modifytable_subplan with the assumption
that if the immediate child of a ModifyTable is a Result, it must be
a projecting Result with a subplan. However, if the UPDATE or DELETE
has a provably-constant-false WHERE clause, that's not so: we'll
generate a dummy subplan with a childless Result. Add the missing
null-check so we don't crash on such cases.
Per report from Alexander Pyhalov.
Discussion: https://postgr.es/m/b9a6f53549456b2f3e2fd150dcd79d72@postgrespro.ru
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
doc: Fix description about pg_stat_statements.track_planning.
commit : e48f2afee631be42739e50fbefd758503e8dcf82
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 7 Jul 2021 21:54:47 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 7 Jul 2021 21:54:47 +0900
This commit fixes wrong wording like "a fewer kinds"
in the description about track_planning option.
Back-patch to v13 where pg_stat_statements.track_planning was added.
Author: Justin Pryzby
Reviewed-by: Julien Rouhaud, Fujii Masao
Discussion: https://postgr.es/m/20210418233615.GB7256@telsasoft.com
M doc/src/sgml/pgstatstatements.sgml
postgres_fdw: Tighten up allowed values for batch_size, fetch_size options.
commit : 4173477b3841749ce491c77b54a5940f6f3e9eb6
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 7 Jul 2021 11:13:40 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 7 Jul 2021 11:13:40 +0900
Previously the values such as '100$%$#$#', '9,223,372,' were accepted and
treated as valid integers for postgres_fdw options batch_size and fetch_size.
Whereas this is not the case with fdw_startup_cost and fdw_tuple_cost options
for which an error is thrown. This was because endptr was not used
while converting strings to integers using strtol.
This commit changes the logic so that it uses parse_int function
instead of strtol as it serves the purpose by returning false in case
if it is unable to convert the string to integer. Note that
this function also rounds off the values such as '100.456' to 100 and
'100.567' or '100.678' to 101.
While on this, use parse_real for fdw_startup_cost and fdw_tuple_cost options.
Since parse_int and parse_real are being used for reloptions and GUCs,
it is more appropriate to use in postgres_fdw rather than using strtol
and strtod directly.
Back-patch to v14.
Author: Bharath Rupireddy
Reviewed-by: Ashutosh Bapat, Tom Lane, Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVMO6wY5Pc4oe1OCgUOAtdjHuFsBDw8R5uoYR86eWFQDA@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Avoid doing catalog lookups in postgres_fdw's conversion_error_callback.
commit : 86d4914210e9e7f8f8c32defacc71744d0222feb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Jul 2021 12:36:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Jul 2021 12:36:12 -0400
As in 50371df26, this is a bad idea since the callback can't really
know what error is being thrown and thus whether or not it is safe
to attempt catalog accesses. Rather than pushing said accesses into
the mainline code where they'd usually be a waste of cycles, we can
look at the query's rangetable instead.
This change does mean that we'll be printing query aliases (if any
were used) rather than the table or column's true name. But that
doesn't seem like a bad thing: it's certainly a more useful definition
in self-join cases, for instance. In any case, it seems unlikely that
any applications would be depending on this detail, so it seems safe
to change.
Patch by me. Original complaint by Andres Freund; Bharath Rupireddy
noted the connection to conversion_error_callback.
Discussion: https://postgr.es/m/20210106020229.ne5xnuu6wlondjpe@alap3.anarazel.de
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
Doc: add info about timestamps with fractional-minute UTC offsets.
commit : 94911ec28f01d1babe61053977ca9f057983cb21
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Jul 2021 10:34:51 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Jul 2021 10:34:51 -0400
Our code has supported fractional-minute UTC offsets for ages, but
there was no mention of the possibility in the main docs, and only
a very indirect reference in Appendix B. Improve that.
Discussion: https://postgr.es/m/162543102827.697.5755498651217979813@wrigleys.postgresql.org
M doc/src/sgml/datatype.sgml
Reduce overhead of cache-clobber testing in LookupOpclassInfo().
commit : 07f1e06964adeec6d79bb24f3b1d50666694213b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Jul 2021 16:51:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Jul 2021 16:51:57 -0400
Commit 03ffc4d6d added logic to bypass all caching behavior in
LookupOpclassInfo when CLOBBER_CACHE_ALWAYS is enabled. It doesn't
look like I stopped to think much about what that would cost, but
recent investigation shows that the cost is enormous: it roughly
doubles the time needed for cache-clobber test runs.
There does seem to be value in this behavior when trying to test
the opclass-cache loading logic itself, but for other purposes the
cost is excessive. Hence, let's back off to doing this only when
debug_invalidate_system_caches_always is at least 3; or in older
branches, when CLOBBER_CACHE_RECURSIVELY is defined.
While here, clean up some other minor issues in LookupOpclassInfo.
Re-order the code so we aren't left with broken cache entries (leading
to later core dumps) in the unlikely case that we suffer OOM while
trying to allocate space for a new entry. (That seems to be my
oversight in 03ffc4d6d.) Also, in >= v13, stop allocating one array
entry too many. That's evidently left over from sloppy reversion in
851b14b0c.
Back-patch to all supported branches, mainly to reduce the runtime
of cache-clobbering buildfarm animals.
Discussion: https://postgr.es/m/1370856.1625428625@sss.pgh.pa.us
M src/backend/utils/cache/relcache.c
Rethink blocking annotations in detach-partition-concurrently-[34].
commit : 9fa6fe466c9d2eaba4fdd8091203ee61e74d71bf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Jul 2021 14:34:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Jul 2021 14:34:47 -0400
In 741d7f104, I tried to make the reports from canceled steps come out
after the pg_cancel_backend() steps, since that was the most common
ordering before. However, that doesn't ensure that a canceled step
doesn't report even later, as shown in a recent failure on buildfarm
member idiacanthus. Rather than complicating things even more with
additional annotations, let's just force the cancel's effect to be
reported first. It's not *that* unnatural-looking.
Back-patch to v14 where these test cases appeared.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=idiacanthus&dt=2021-07-02%2001%3A40%3A04
M src/test/isolation/expected/detach-partition-concurrently-3.out
M src/test/isolation/expected/detach-partition-concurrently-4.out
M src/test/isolation/specs/detach-partition-concurrently-3.spec
M src/test/isolation/specs/detach-partition-concurrently-4.spec
doc: Fix quoting markup
commit : 1479c5afdce2c57928dc1322bd71106858a0770f
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 5 Jul 2021 08:26:00 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 5 Jul 2021 08:26:00 +0200
M doc/src/sgml/appendix-obsolete-default-roles.sgml
Doc: Hash Indexes.
commit : f3690fbdeac43ab8547c212062ec515a55dd118c
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 5 Jul 2021 09:38:17 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 5 Jul 2021 09:38:17 +0530
A new chapter for Hash Indexes, designed to help users understand how
they work and when to use them.
Backpatch-through 10 where we have made hash indexes durable.
Author: Simon Riggs
Reviewed-By: Justin Pryzby, Amit Kapila
Discussion: https://postgr.es/m/CANbhV-HRjNPYgHo--P1ewBrFJ-GpZPb9_25P7=Wgu7s7hy_sLQ@mail.gmail.com
M doc/src/sgml/filelist.sgml
A doc/src/sgml/hash.sgml
M doc/src/sgml/postgres.sgml
doc: Mention requirement to --enable-tap-tests on section for TAP tests
commit : 95e2f925f1443f49b31e861a8d66dabca5528aaa
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 4 Jul 2021 20:59:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 4 Jul 2021 20:59:10 +0900
Author: Greg Sabino Mullane
Discussion: https://postgr.es/m/CAKAnmmJYH2FBn_+Vwd2FD5SaKn8hjhAXOCHpZc6n4wXaUaW_SA@mail.gmail.com
Backpatch-through: 9.6
M doc/src/sgml/regress.sgml
Doc: mention that VACUUM can't utilize over 1GB of RAM
commit : b88e5fe60eef5decd3ccfd73566559dc6136cb61
author : David Rowley <drowley@postgresql.org>
date : Sun, 4 Jul 2021 22:29:16 +1200
committer: David Rowley <drowley@postgresql.org>
date : Sun, 4 Jul 2021 22:29:16 +1200
Document that setting maintenance_work_mem to values over 1GB has no
effect on VACUUM.
Reported-by: Martín Marqués
Author: Laurenz Albe
Discussion: https://postgr.es/m/CABeG9LsZ2ozUMcqtqWu_-GiFKB17ih3p8wBHXcpfnHqhCnsc7A%40mail.gmail.com
Backpatch-through: 9.6, oldest supported release
M doc/src/sgml/config.sgml
doc: adjust "cities" example to be consistent with other SQL
commit : e033e260b9d0fcba969e5314be967429ade1b9ec
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Jul 2021 20:42:46 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Jul 2021 20:42:46 -0400
Reported-by: tom@crystae.net
Discussion: https://postgr.es/m/162345756191.14472.9754568432103008703@wrigleys.postgresql.org
Backpatch-through: 9.6
M doc/src/sgml/advanced.sgml
docs: clarify new aggressive vacuum mode for multi-xacts
commit : d54092955690e29656186ff5d4d315a3c56ccb37
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Jul 2021 18:00:30 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Jul 2021 18:00:30 -0400
Reported-by: eric.mutta@gmail.com
Discussion: https://postgr.es/m/162395467510.686.11947486273299446208@wrigleys.postgresql.org
Backpatch-through: 14
M doc/src/sgml/maintenance.sgml
Don't try to print data type names in slot_store_error_callback().
commit : 63a952167092bd2d457691202d71ace943e1b06a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 2 Jul 2021 16:04:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 2 Jul 2021 16:04:54 -0400
The existing code tried to do syscache lookups in an already-failed
transaction, which is problematic to say the least. After some
consideration of alternatives, the best fix seems to be to just drop
type names from the error message altogether. The table and column
names seem like sufficient localization. If the user is unsure what
types are involved, she can check the local and remote table
definitions.
Having done that, we can also discard the LogicalRepTypMap hash
table, which had no other use. Arguably, LOGICAL_REP_MSG_TYPE
replication messages are now obsolete as well; but we should
probably keep them in case some other use emerges. (The complexity
of removing something from the replication protocol would likely
outweigh any savings anyhow.)
Masahiko Sawada and Bharath Rupireddy, per complaint from Andres
Freund. Back-patch to v10 where this code originated.
Discussion: https://postgr.es/m/20210106020229.ne5xnuu6wlondjpe@alap3.anarazel.de
M src/backend/replication/logical/relation.c
M src/backend/replication/logical/worker.c
M src/include/replication/logicalrelation.h
doc: PG 14 relnotes, mention CONCURRENTLY improvements
commit : 56d467a07de1c203cfe78fea69cd8326fa297e23
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Jul 2021 14:46:31 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Jul 2021 14:46:31 -0400
Add items for vacuum not having to wait for CONCURRENTLY, and
CONCURRENTLY not having to wait for other CONCURRENTLY operations.
Reported-by: Simon Riggs
Discussion: https://postgr.es/m/CANbhV-EMM4nf7Ys-Yae_kY25dXT_3eiOXke2+yw44jgy+4jNsA@mail.gmail.com
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
doc: update PG 14 release notes
commit : a3a681f8590d9476c8ce94172316d3d339375fe6
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 1 Jul 2021 20:33:32 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 1 Jul 2021 20:33:32 -0400
Mostly addition of <literal> tags
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210629022547.GF21248@telsasoft.com
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
doc: Remove inappropriate <acronym> tags
commit : fb72a7b8c384e25d752b22a69e77d8227af65b4b
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 1 Jul 2021 22:23:37 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 1 Jul 2021 22:23:37 +0200
M doc/src/sgml/brin.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/release-14.sgml
add missing tag from commit b8c4261e5e
commit : 1da2ea0cccc7f9ce5eed3d6e62cba785e82a6a0d
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 1 Jul 2021 15:38:06 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 1 Jul 2021 15:38:06 -0400
M GNUmakefile.in
M doc/src/sgml/installation.sgml
doc: Clean up title case use
commit : 46bbe5d777722938496f177706cc87ff64512def
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 1 Jul 2021 21:27:39 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 1 Jul 2021 21:27:39 +0200
M doc/src/sgml/appendix-obsolete-default-roles.sgml
M doc/src/sgml/btree.sgml
M doc/src/sgml/logicaldecoding.sgml
Add new make targets world-bin and install-world-bin
commit : 100e9ae53f210c9e82671f1cc554aca945c4c180
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 1 Jul 2021 14:21:09 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 1 Jul 2021 14:21:09 -0400
These are the same as world and install-world respectively, but without
building or installing the documentation. There are many reasons for
wanting to be able to do this, including speed, lack of documentation
building tools, and wanting to build other formats of the documentation.
Plans for simplifying the buildfarm client code include using these
targets.
Backpatch to all live branches.
Discussion: https://postgr.es/m/6a421136-d462-b043-a8eb-e75b2861f3df@dunslane.net
M GNUmakefile.in
M doc/src/sgml/installation.sgml
Add --clobber-cache option to initdb, for CCA testing.
commit : d0477080174b227e6f5cbe0dd10c4d79abb7f3e6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 1 Jul 2021 13:33:05 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 1 Jul 2021 13:33:05 -0400
Commit 4656e3d66 replaced the "#define CLOBBER_CACHE_ALWAYS"
testing mechanism with a GUC, which has been a great help for
doing cache-clobber testing in more efficient ways; but there
is a gap in the implementation. The only way to do cache-clobber
testing during an initdb run is to use the old method with #define,
because one can't set the GUC from outside. Improve this by
adding a switch to initdb for the purpose.
(Perhaps someday we should let initdb pass through arbitrary
"-c NAME=VALUE" switches. Quoting difficulties dissuaded me
from attempting that right now, though.)
Back-patch to v14 where 4656e3d66 came in.
Discussion: https://postgr.es/m/1582507.1624227029@sss.pgh.pa.us
M doc/src/sgml/ref/initdb.sgml
M src/bin/initdb/initdb.c
Don't reset relhasindex for partitioned tables on ANALYZE
commit : be280cdad2985749e558212b0a5c8bdf9abb4e6a
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 1 Jul 2021 12:56:30 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 1 Jul 2021 12:56:30 -0400
Commit 0e69f705cc1a introduced code to analyze partitioned table;
however, that code fails to preserve pg_class.relhasindex correctly.
Fix by observing whether any indexes exist rather than accidentally
falling through to assuming none do.
Backpatch to 14.
Author: Alexander Pyhalov <a.pyhalov@postgrespro.ru>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Zhihong Yu <zyu@yugabyte.com>
Discussion: https://postgr.es/m/CALNJ-vS1R3Qoe5t4tbzxrkpBtzRbPq1dDcW4RmA_a+oqweF30w@mail.gmail.com
M src/backend/commands/analyze.c
M src/test/regress/expected/vacuum.out
M src/test/regress/sql/vacuum.sql
Fix prove_installcheck to use correct paths when used with PGXS
commit : c4774ce339beff4a8968ceecb86bbbfa3335ed2b
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 1 Jul 2021 08:29:10 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 1 Jul 2021 08:29:10 -0400
The prove_installcheck recipe in src/Makefile.global.in was emitting
bogus paths for a couple of elements when used with PGXS. Here we create
a separate recipe for the PGXS case that does it correctly. We also take
the opportunity to make the make the file more readable by breaking up
the prove_installcheck and prove_check recipes across several lines, and
to remove the setting for REGRESS_SHLIB to src/test/recovery/Makefile,
which is the only set of tests that actually need it.
Backpatch to all live branches
Discussion: https://postgr.es/m/f2401388-936b-f4ef-a07c-a0bcc49b3300@dunslane.net
M src/Makefile.global.in
M src/test/recovery/Makefile
Replace magic constants used in pg_stat_get_replication_slot().
commit : a9cb00a965c6ac0cd434733fae0948218702e501
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 30 Jun 2021 11:40:06 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 30 Jun 2021 11:40:06 +0530
A few variables have been using 10 as a magic constant while
PG_STAT_GET_REPLICATION_SLOT_COLS can be used instead.
Author: Masahiko Sawada
Reviewed-By: Amit Kapila
Backpatch-through: 14, where it was introduced
Discussion: https://postgr.es/m/CAD21AoBvqODDfmD17DkEuPCvV2KbruukXQ2Vwrv5Xi-TsAsTJA@mail.gmail.com
M src/backend/utils/adt/pgstatfuncs.c
Allow streaming the changes after speculative aborts.
commit : dfceed30abc191c097557357dd6db56e875bb7e1
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 30 Jun 2021 09:48:07 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 30 Jun 2021 09:48:07 +0530
Until now, we didn't allow to stream the changes in logical replication
till we receive speculative confirm or the next DML change record after
speculative inserts. The reason was that we never use to process
speculative aborts but after commit 4daa140a2f it is possible to process
them so we can allow streaming once we receive speculative abort after
speculative insertion.
We decided to backpatch to 14 where the feature for streaming in progress
transactions have been introduced as this is a minor change and makes that
functionality better.
Author: Amit Kapila
Reviewed-By: Dilip Kumar
Backpatch-through: 14
Discussion: https://postgr.es/m/CAA4eK1KdqmTCtrBR6oFfGELrLLbDLDedL6zACcsUOQuTJBj1vw@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
Fix incorrect PITR message for transaction ROLLBACK PREPARED
commit : 607a3a43bca573a02570f374fd98c82a0c2e12a1
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 30 Jun 2021 11:49:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 30 Jun 2021 11:49:10 +0900
Reaching PITR on such a transaction would cause the generation of a LOG
message mentioning a transaction committed, not aborted.
Oversight in 4f1b890.
Author: Simon Riggs
Discussion: https://postgr.es/m/CANbhV-GJ6KijeCgdOrxqMCQ+C8QiK657EMhCy4csjrPcEUFv_Q@mail.gmail.com
Backpatch-through: 9.6
M src/backend/access/transam/xlog.c
Fixes for multirange selectivity estimation
commit : 322e82b77ef4acb9697c6e4259292f5671cb85bb
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Jun 2021 23:18:09 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Jun 2021 23:18:09 +0300
* Fix enumeration of the multirange operators in calc_multirangesel() and
calc_multirangesel() switches.
* Add more regression tests for matching to empty ranges/multiranges.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/c5269c65-f967-77c5-ff7c-15e621c47f6a%40gmail.com
Author: Alexander Korotkov
Backpatch-through: 14, where multiranges were introduced
M src/backend/utils/adt/multirangetypes_selfuncs.c
M src/test/regress/expected/multirangetypes.out
M src/test/regress/sql/multirangetypes.sql
Fix libpq state machine in pipeline mode
commit : 690339fcd58759d213523f87071f992c4402e980
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 29 Jun 2021 15:01:29 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 29 Jun 2021 15:01:29 -0400
The original coding required that PQpipelineSync had been called before
the first call to PQgetResult, and failure to do that would result in an
unexpected NULL result being returned. Fix by setting the right state
when a query is sent, rather than leaving it unchanged and having
PQpipelineSync apply the necessary state change.
A new test case to verify the behavior is added, which relies on the new
PQsendFlushRequest() function added by commit a7192326c74d.
Backpatch to 14, where pipeline mode was added.
Reported-by: Boris Kolpackov <boris@codesynthesis.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/boris.20210616110321@codesynthesis.com
M src/interfaces/libpq/fe-exec.c
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
A src/test/modules/libpq_pipeline/traces/nosync.trace
Add PQsendFlushRequest to libpq
commit : 69cf1d5429d48d030d71d5dd4931084d422413cb
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 29 Jun 2021 14:37:39 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 29 Jun 2021 14:37:39 -0400
This new libpq function allows the application to send an 'H' message,
which instructs the server to flush its outgoing buffer.
This hasn't been needed so far because the Sync message already requests
a buffer; and I failed to realize that this was needed in pipeline mode
because PQpipelineSync also causes the buffer to be flushed. However,
sometimes it is useful to request a flush without establishing a
synchronization point.
Backpatch to 14, where pipeline mode was introduced in libpq.
Reported-by: Boris Kolpackov <boris@codesynthesis.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/202106252350.t76x73nt643j@alvherre.pgsql
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/exports.txt
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/libpq-fe.h
Fix bogus logic for reporting which hash partition conflicts.
commit : f8b51464c265696c1eab1c896bddc797beb9a13c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Jun 2021 14:34:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Jun 2021 14:34:31 -0400
Commit efbfb6424 added logic for reporting exactly which existing
partition conflicts when complaining that a new hash partition's
modulus isn't compatible with the existing ones. However, it
misunderstood the partitioning data structure, and would select
the wrong partition in some cases, or crash outright due to fetching
a bogus table OID in other cases.
Per bug #17076 from Alexander Lakhin. Fix by Amit Langote;
some further work on the code comments by me.
Discussion: https://postgr.es/m/17076-89a16ae835d329b9@postgresql.org
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/create_table.out
M src/test/regress/sql/create_table.sql
doc: Improve PG14 release notes
commit : caa0f07d2d4b69cd53b5bf7c847d9d65eadbbfee
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 28 Jun 2021 20:58:47 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 28 Jun 2021 20:58:47 -0400
Mostly markup improvements.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210625230456.GP29179@telsasoft.com
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
Don't use abort(3) in libpq's fe-print.c.
commit : cf1f545bf281d3dcb7b44e6b7f21a9369024fbf0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Jun 2021 14:17:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Jun 2021 14:17:42 -0400
Causing a core dump on out-of-memory seems pretty unfriendly,
and surely is far outside the expected behavior of a general-purpose
library. Just print an error message (as we did already) and return.
These functions unfortunately don't have an error return convention,
but code using them is probably just looking for a quick-n-dirty
print method and wouldn't bother to check anyway.
Although these functions are semi-deprecated, it still seems
appropriate to back-patch this. In passing, also back-patch
b90e6cef1, just to reduce cosmetic differences between the
branches.
Discussion: https://postgr.es/m/3122443.1624735363@sss.pgh.pa.us
M src/interfaces/libpq/fe-print.c
Don't depend on -fwrapv semantics in pgbench's random() function.
commit : 203c5aaaba56715168c1e80a45d4929120c9281b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Jun 2021 12:40:37 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Jun 2021 12:40:37 -0400
Instead use the common/int.h functions to check for integer overflow
in a more C-standard-compliant fashion. This is motivated by recent
failures on buildfarm member moonjelly, where it appears that
development-tip gcc is optimizing without regard to the -fwrapv
switch. Presumably that's a gcc bug that will be fixed soon, but
we might as well install cleaner coding here rather than wait.
(This does not address the question of whether we'll ever be able
to get rid of using -fwrapv. Testing shows that this spot is the
only place where doing so creates visible regression test failures,
but unfortunately that proves very little.)
Back-patch to v12. The common/int.h functions exist in v11, but
that branch doesn't use them in any client-side code. I judge
that this case isn't interesting enough in the real world to take
even a small risk of issues from being the first such use.
Tom Lane and Fabien Coelho
Discussion: https://postgr.es/m/73927.1624815543@sss.pgh.pa.us
M src/bin/pgbench/pgbench.c
Pre branch pgindent / pgperltidy run
commit : e1c1c30f635390b6a3ae4993e8cac213a33e6e3f
author : Andrew Dunstan <andrew@dunslane.net>
date : Mon, 28 Jun 2021 11:05:54 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Mon, 28 Jun 2021 11:05:54 -0400
Along the way make a slight adjustment to
src/include/utils/queryjumble.h to avoid an unused typedef.
M src/backend/access/heap/hio.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/heap.c
M src/backend/executor/nodeModifyTable.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/storage/ipc/procarray.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/include/nodes/execnodes.h
M src/include/utils/queryjumble.h
M src/test/perl/PostgresNode.pm
M src/test/recovery/t/005_replay_delay.pl
M src/test/recovery/t/025_stuck_on_old_timeline.pl
M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/010_truncate.pl
M src/test/subscription/t/013_partition.pl
M src/test/subscription/t/020_messages.pl
M src/tools/pgindent/typedefs.list
Message style improvements
commit : c31833779d5a4775d7220be4b9143bec66c9a9fd
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 28 Jun 2021 08:36:44 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 28 Jun 2021 08:36:44 +0200
M src/backend/access/common/toast_compression.c
M src/backend/catalog/catalog.c
M src/backend/catalog/pg_inherits.c
M src/backend/commands/tablecmds.c
M src/backend/postmaster/pgstat.c
M src/backend/storage/file/fd.c
M src/backend/storage/ipc/standby.c
M src/backend/tcop/fastpath.c
M src/backend/utils/adt/multirangetypes.c
M src/test/regress/expected/compression_1.out
M src/test/regress/expected/multirangetypes.out
Improve RelationGetIdentityKeyBitmap().
commit : ee3fdb8f3465b3a5937a7fe647b7b6584a600647
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 28 Jun 2021 10:56:53 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 28 Jun 2021 10:56:53 +0530
We were using RelationGetIndexList() to update the relation's replica
identity index but instead, we can directly use RelationGetReplicaIndex()
which uses the same functionality. This is a minor code readability
improvement.
Author: Japin Li
Reviewed-By: Takamichi Osumi, Amit Kapila
Discussion: https://postgr.es/m/4C99A862-69C8-431F-960A-81B1151F1B89@enterprisedb.com
M src/backend/utils/cache/relcache.c
Fix race condition in TransactionGroupUpdateXidStatus().
commit : b786304c2904a4e444fe740bbc2e0b69efacc19d
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 28 Jun 2021 09:29:38 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 28 Jun 2021 09:29:38 +0530
When we cannot immediately acquire XactSLRULock in exclusive mode at
commit time, we add ourselves to a list of processes that need their XIDs
status update. We do this if the clog page where we need to update the
current transaction status is the same as the group leader's clog page,
otherwise, we allow the caller to clear it by itself. Now, when we can't
add ourselves to any group, we were not clearing the current proc if it
has already become a member of some group which was leading to an
assertion failure when the same proc was assigned to another backend after
the current backend exits.
Reported-by: Alexander Lakhin
Bug: 17072
Author: Amit Kapila
Tested-By: Alexander Lakhin
Backpatch-through: 11, where it was introduced
Discussion: https://postgr.es/m/17072-2f8764857ef2c92a@postgresql.org
M src/backend/access/transam/clog.c
Change recovery_init_sync_method to PGC_SIGHUP.
commit : 34a8b64b4e5f0cd818e5cc7f98846de57938ea57
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 28 Jun 2021 15:17:43 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 28 Jun 2021 15:17:43 +1200
The setting has no effect except during startup. It's still nice to be
able to change it dynamically, which is expected to be pretty useful to
an admin following crash recovery when restarting the cluster is not so
appealing.
Per discussions following commits 2941138e6 and 61752afb2.
Author: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/20210529192321.GM2082%40telsasoft.com
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
Fix variable initialization with ALTER SUBSCRIPTION DROP PUBLICATION
commit : 79718c1c6c007c27e9c1b8e92bd96d17067606fa
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 28 Jun 2021 12:11:18 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 28 Jun 2021 12:11:18 +0900
copy_data is not a supported option with this sub-command of ALTER
SUBSCRIPTION, which would not make a variable related to it initialized
after parsing the option set in DefElems. A refresh could then refer to
it.
Author: Ranier Vilela
Reviewed-by: Peter Smith
Discussion: https://postgr.es/m/CAEudQAp5P8nr=ze2Gv=BMj=DJFZnrvendZCZcC-fos3QiDe2sg@mail.gmail.com
M src/backend/commands/subscriptioncmds.c
Add test for CREATE INDEX CONCURRENTLY with not-so-immutable predicate
commit : 09a69f6e23369847cf11cd03c999a0342d47bbcc
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 28 Jun 2021 11:17:05 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 28 Jun 2021 11:17:05 +0900
83158f7 has improved index_set_state_flags() so as it is possible to use
transactional updates when updating pg_index state flags, but there was
not really a test case which stressed directly the possibility it fixed.
This commit adds such a test, using a predicate that looks valid in
appearance but calls a stable function.
Author: Andrey Lepikhov
Discussion: https://postgr.es/m/9b905019-5297-7372-0ad2-e1a4bb66a719@postgrespro.ru
Backpatch-through: 9.6
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Remove memory leaks in isolationtester.
commit : 642c0697c96b9c6ba5d194653a329f7820565f01
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 27 Jun 2021 12:45:04 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 27 Jun 2021 12:45:04 -0400
specscanner.l leaked a kilobyte of memory per token of the spec file.
Apparently somebody thought that the introductory code block would be
executed once; but it's once per yylex() call.
A couple of functions in isolationtester.c leaked small amounts of
memory due to not bothering to free one-time allocations. Might
as well improve these so that valgrind gives this program a clean
bill of health. Also get rid of an ugly static variable.
Coverity complained about one of the one-time leaks, which led me
to try valgrind'ing isolationtester, which led to discovery of the
larger leak.
M src/test/isolation/isolationtester.c
M src/test/isolation/specscanner.l
Error message refactoring
commit : c302a6139072fed9410204fa9e751d95930e05ff
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 27 Jun 2021 09:41:16 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 27 Jun 2021 09:41:16 +0200
Take some untranslatable things out of the message and replace by
format placeholders, to reduce translatable strings and reduce
translation mistakes.
M src/backend/replication/logical/logical.c
M src/backend/statistics/extended_stats.c
M src/backend/utils/adt/numeric.c
Doc: update v14 release notes for revert of commit c0cb87fbb.
commit : dcffc9ba8a1e0ab1b0a57e9b9d38e3dc9960f83f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 26 Jun 2021 15:45:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 26 Jun 2021 15:45:16 -0400
M doc/src/sgml/release-14.sgml
Remove undesirable libpq dependency on stringinfo.c.
commit : 8ec00dc5cd70e0e579e9fbf8661bc46f5ccd8078
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 26 Jun 2021 14:20:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 26 Jun 2021 14:20:17 -0400
Commit c0cb87fbb unwisely introduced a dependency on the StringInfo
machinery in fe-connect.c. We must not use that in libpq, because
it will do a summary exit(1) if it hits OOM, and that is not
appropriate behavior for a general-purpose library. The goal of
allowing arbitrary line lengths in service files doesn't seem like
it's worth a lot of effort, so revert back to the previous method
of using a stack-allocated buffer and failing on buffer overflow.
This isn't an exact revert though. I kept that patch's refactoring
to have a single exit path, as that seems cleaner than having each
error path know what to do to clean up. Also, I made the fixed-size
buffer 1024 bytes not 256, just to push off the need for an expandable
buffer some more.
There is more to do here; in particular the lack of any mechanical
check for this type of mistake now seems pretty hazardous. But this
fix gets us back to the level of robustness we had in v13, anyway.
Discussion: https://postgr.es/m/daeb22ec6ca8ef61e94d766a9b35fb03cabed38e.camel@vmware.com
M src/interfaces/libpq/fe-connect.c
Remove non-existing variable reference in MSVC's Solution.pm
commit : d5a2c413fcdd187dc16c4fab16610af7d4849cc1
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 26 Jun 2021 13:52:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 26 Jun 2021 13:52:48 +0900
The version string is grabbed from PACKAGE_VERSION in pg_config.h in the
MSVC build since 8f4fb4c6, but an error message referenced a variable
that existed before that. This had no consequences except if one messes
up enough with the version number of the build.
Author: Anton Voloshin
Discussion: https://postgr.es/m/af79ee1b-9962-b299-98e1-f90a289e19e6@postgrespro.ru
Backpatch-through: 13
M src/tools/msvc/Solution.pm
Remove some useless logs from the TAP tests of pgbench
commit : 704e1dbd9aa29a0b46c356f1803ad55cbdef2c20
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 26 Jun 2021 12:39:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 26 Jun 2021 12:39:54 +0900
002_pgbench_no_server was printing some array pointers instead of the
actual contents of those arrays for the expected outputs of stdout and
stderr for a tested command. This does not add any new information that
can help with debugging as the test names allow to track failure
locations, if any.
This commit simply removes those logs as the rest of the printed
information is redundant with command_checks_all().
Per discussion with Andrew Dunstan and Álvaro Herrera.
Discussion: https://postgr.es/m/YNXNFaG7IgkzZanD@paquier.xyz
Backpatch-through: 11
M src/bin/pgbench/t/002_pgbench_no_server.pl
Remove unnecessary failure cases in RemoveRoleFromObjectPolicy().
commit : 5a0f1c8c0193f0dd7fba50c22d96781fa2414007
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 25 Jun 2021 13:59:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 25 Jun 2021 13:59:38 -0400
It's not really necessary for this function to open or lock the
relation associated with the pg_policy entry it's modifying. The
error checks it's making on the rel are if anything counterproductive
(e.g., if we don't want to allow installation of policies on system
catalogs, here is not the place to prevent that). In particular, it
seems just wrong to insist on an ownership check. That has the net
effect of forcing people to use superuser for DROP OWNED BY, which
surely is not an effect we want. Also there is no point in rebuilding
the dependencies of the policy expressions, which aren't being
changed. Lastly, locking the table also seems counterproductive; it's
not helping to prevent race conditions, since we failed to re-read the
pg_policy row after acquiring the lock. That means that concurrent
DDL would likely result in "tuple concurrently updated/deleted"
errors; which is the same behavior this code will produce, with less
overhead.
Per discussion of bug #17062. Back-patch to all supported versions,
as the failure cases this eliminates seem just as undesirable in 9.6
as in HEAD.
Discussion: https://postgr.es/m/1573181.1624220108@sss.pgh.pa.us
M src/backend/commands/policy.c
Doc: remove commit f560209c6 from v14 release notes.
commit : 8a80562d732c0da1ddcc9fb88dfb976f4b846577
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 25 Jun 2021 11:18:15 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 25 Jun 2021 11:18:15 -0400
Now that this has been back-patched, it's no longer a new feature
for v14.
M doc/src/sgml/release-14.sgml
Cleanup some code related to pgbench log checks in TAP tests
commit : 38ff135d9466c35b506b1049fedef73047907be0
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Jun 2021 20:15:24 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Jun 2021 20:15:24 +0900
This fixes a couple of problems within the so-said code of this commit
subject:
- Replace the use of open() with slurp_file(), fixing an issue reported
by buildfarm member fairywren whose perl installation keep around CRLF
characters, causing the matching patterns for the logs to fail.
- Remove the eval block, which is not really necessary.
This set of issues has come into light after fixing a different issue
with c13585fe, and this is wrong since this code has been introduced.
Reported-by: Andrew Dunstan, and buildfarm member fairywren
Author: Michael Paquier
Reviewed-by: Andrew Dunstan
Discussion: https://postgr.es/m/0f49303e-7784-b3ee-200b-cbf67be2eb9e@dunslane.net
Backpatch-through: 11
M src/bin/pgbench/t/001_pgbench_with_server.pl
Put option listing back into alphabetical order
commit : 3af10943ce21450e299b3915b9cad47cd90369e9
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Jun 2021 11:40:06 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Jun 2021 11:40:06 +0200
M doc/src/sgml/ref/vacuumdb.sgml
M src/bin/scripts/vacuumdb.c
Fixes in ALTER SUBSCRIPTION DROP PUBLICATION code
commit : e59d428f34297cd0eb67e3b4e4b8c8bc58504921
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Jun 2021 09:51:24 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Jun 2021 09:51:24 +0200
ALTER SUBSCRIPTION DROP PUBLICATION does not actually support
copy_data option, so remove it from tab completion.
Also, reword the error message that is thrown when all the
publications from a subscription are specified to be dropped.
Also, made few doc and cosmetic adjustments.
Author: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Reviewed-by: Japin Li <japinli@hotmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CALDaNm21RwsDzs4xj14ApteAF7auyyomHNnp+NEL-sH8m-jMvQ@mail.gmail.com
M doc/src/sgml/ref/alter_subscription.sgml
M src/backend/commands/subscriptioncmds.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/subscription.out
doc: Change reloption data type spelling for consistency
commit : 63e6d05bf34da58eef7cd1ed461b9c8315177a38
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Jun 2021 08:11:10 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Jun 2021 08:11:10 +0200
Use "floating point" rather than "float4", like everywhere else in
this context.
Author: Shinya11.Kato@nttdata.com
Discussion: https://www.postgresql.org/message-id/flat/TYAPR01MB28965989AF84B57FC351B97BC40F9@TYAPR01MB2896.jpnprd01.prod.outlook.com
M doc/src/sgml/ref/create_table.sgml
Remove redundant variable pageSize in gistinitpage
commit : a60c4c5c1a99746485123ae93fbd3e58c78e5d62
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Jun 2021 07:55:34 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 25 Jun 2021 07:55:34 +0200
In gistinitpage, pageSize variable looks redundant, instead just
pass BLCKSZ. This will be consistent with its peers BloomInitPage,
brin_page_init and SpGistInitPage.
Author: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/CALj2ACWj=V1k5591eeZK2sOg2FYuBUp6azFO8tMkBtGfXf8PMQ@mail.gmail.com
M src/backend/access/gist/gistutil.c
Doc: Fix minor formatting issue in logicaldecoding.sgml.
commit : 847c62ee76cbc237b3a204ca94b1b12811d698e3
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 25 Jun 2021 08:22:44 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 25 Jun 2021 08:22:44 +0530
Author: Guillaume Lelarge
Discussion: https://www.postgresql.org/message-id/CAECtzeXf3_oZoU6mgFCOy5+pDZ5n4XtH0Da4a5n_KacraVWiHQ@mail.gmail.com
M doc/src/sgml/logicaldecoding.sgml
doc: Add acronyms for MITM and SNI
commit : 15ff5401d1719aaf6c9a47e5abea517cc2bcbaf1
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Jun 2021 11:29:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Jun 2021 11:29:03 +0900
This adds MITM and SNI as acronyms, as the documentation already had
them marked up with <acronym>.
While on it, make sure to spell man-in-the-middle with dashes
consistently, and add acronyms for those new terms where appropriate.
Author: Daniel Gustafsson
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/CE12DD5C-4BB3-4166-BC9A-39779568734C@yesql.se
M doc/src/sgml/acronyms.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/libpq.sgml
Add more debugging information with log checks in TAP tests of pgbench
commit : 87b2124dfa0782a697ea7b90aff15a6f6117a720
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Jun 2021 09:56:44 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Jun 2021 09:56:44 +0900
fairywren is not happy with the pattern checks introduced by c13585f.
I am not sure if this outlines a bug in pgbench or if the regex patterns
used in the tests are too restrictive for this buildfarm member's
environment. This adds more debugging information to show the log
entries that do not match with the expected pattern, to help in finding
out what's happening. That seems like a good addition in the long-term
anyway as that may not be the only issue in this area.
Discussion: https://postgr.es/m/YNUad2HvgW+6eXyo@paquier.xyz
M src/bin/pgbench/t/001_pgbench_with_server.pl
doc: Move remove_temp_files_after_crash to section for developer options
commit : 797b0fc0b078c7b4c46ef9f2d9e47aa2d98c6c63
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Jun 2021 08:40:16 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Jun 2021 08:40:16 +0900
The main goal of this option is to allow inspecting temporary files for
debugging purposes, so moving the parameter there is natural.
Oversight in cd91de0.
Reported-by: Justin Pryzby
Author: Euler Taveira
Discussion: https://postgr.es/m/20210612004347.GP16435@telsasoft.com
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
Prepare for forthcoming LLVM 13 API change.
commit : 9b4e4cfe66ff133717c1b8ba3c2725d525c3e67c
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 25 Jun 2021 09:55:26 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 25 Jun 2021 09:55:26 +1200
LLVM 13 (due out in September) has changed the semantics of
LLVMOrcAbsoluteSymbols(), so we need to bump some reference counts to
avoid a double-free that causes crashes and bad query results.
A proactive change seems necessary to avoid having a window of time
where our respective latest releases would interact badly. It's
possible that the situation could change before then, though.
Thanks to Fabien Coelho for monitoring bleeding edge LLVM and Andres
Freund for tracking down the change.
Back-patch to 11, where the JIT code arrived.
Discussion: https://postgr.es/m/CA%2BhUKGLEy8mgtN7BNp0ooFAjUedDTJj5dME7NxLU-m91b85siA%40mail.gmail.com
M src/backend/jit/llvm/llvmjit.c
Fix pattern matching logic for logs in TAP tests of pgbench
commit : c13585fe9e55813cf9feac67fe7b65d3a78fff92
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Jun 2021 06:52:36 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 25 Jun 2021 06:52:36 +0900
The logic checking for the format of per-thread logs used grep() with
directly "$re", which would cause the test to consider all the logs as
a match without caring about their format at all. Using "/$re/" makes
grep() perform a regex test, which is what we want here.
While on it, improve some of the tests to be more picky with the
patterns expected and add more comments to describe the tests.
Issue discovered while digging into a separate patch.
Author: Fabien Coelho, Michael Paquier
Discussion: https://postgr.es/m/YNPsPAUoVDCpPOGk@paquier.xyz
Backpatch-through: 11
M src/bin/pgbench/t/001_pgbench_with_server.pl
Further stabilize postgres_fdw test.
commit : 802177090992511c610804da54a4603d4f50c594
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Jun 2021 15:02:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Jun 2021 15:02:13 -0400
The queries involving ft1_nopw don't stably return the same row
anymore. I surmise that an autovacuum hitting "S 1"."T 1"
right after the updates introduced by f61db909d/5843659d0 freed
some space, changing where subsequent insertions get stored.
It's only by good luck that these results were stable before,
though, since a LIMIT without ORDER BY isn't well defined,
and it's not like we've ever treated that table as append-only
in this test script.
Since we only really care whether these commands succeed or not,
just replace "SELECT *" with "SELECT 1".
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake&dt=2021-06-23%2019%3A52%3A08
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Another fix to relmapper race condition.
commit : 9b8ed0f52bffceaccf6fa650ffe482e7da20fbf2
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 24 Jun 2021 11:19:03 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 24 Jun 2021 11:19:03 +0300
In previous commit, I missed that relmap_redo() was also not acquiring the
RelationMappingLock. Thanks to Thomas Munro for pointing that out.
Backpatch-through: 9.6, like previous commit.
Discussion: https://www.postgresql.org/message-id/CA%2BhUKGLev%3DPpOSaL3WRZgOvgk217et%2BbxeJcRr4eR-NttP1F6Q%40mail.gmail.com
M src/backend/utils/cache/relmapper.c
Prevent race condition while reading relmapper file.
commit : b6d8d2073f228d9f7198f1f9826d8f6ab643c219
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 24 Jun 2021 10:45:23 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 24 Jun 2021 10:45:23 +0300
Contrary to the comment here, POSIX does not guarantee atomicity of a
read(), if another process calls write() concurrently. Or at least Linux
does not. Add locking to load_relmap_file() to avoid the race condition.
Fixes bug #17064. Thanks to Alexander Lakhin for the report and test case.
Backpatch-through: 9.6, all supported versions.
Discussion: https://www.postgresql.org/message-id/17064-bb0d7904ef72add3@postgresql.org
M src/backend/utils/cache/relmapper.c
Doc: Update logical replication message formats.
commit : f08722cf83ab1fabee948a4e5754bf6236ad700b
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 24 Jun 2021 11:51:58 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 24 Jun 2021 11:51:58 +0530
Commits 9de77b5453 and ac4645c015 missed to update the logical replication
message formats section in the docs.
Author: Brar Piening
Reviewed-by: Amit Kapila
Discussion: https://www.postgresql.org/message-id/cc70956c-e578-e54f-49e6-b5d68c89576f@gmx.de
M doc/src/sgml/protocol.sgml
Doc: Update caveats in synchronous logical replication.
commit : c66fb78ebb4f473bb4fd8773de3cda9339e14f81
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 24 Jun 2021 09:13:46 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 24 Jun 2021 09:13:46 +0530
Reported-by: Simon Riggs
Author: Takamichi Osumi
Reviewed-by: Amit Kapila
Backpatch-through: 9.6
Discussion: https://www.postgresql.org/message-id/20210222222847.tpnb6eg3yiykzpky@alap3.anarazel.de
M doc/src/sgml/logicaldecoding.sgml
Allow non-quoted identifiers as isolation test session/step names.
commit : a443c1b2d6a646cf90a8afc193c07ed12a2bf045
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Jun 2021 18:41:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Jun 2021 18:41:39 -0400
For no obvious reason, isolationtester has always insisted that
session and step names be written with double quotes. This is
fairly tedious and does little for test readability, especially
since the names that people actually choose almost always look
like normal identifiers. Hence, let's tweak the lexer to allow
SQL-like identifiers not only double-quoted strings.
(They're SQL-like, not exactly SQL, because I didn't add any
case-folding logic. Also there's no provision for U&"..." names,
not that anyone's likely to care.)
There is one incompatibility introduced by this change: if you write
"foo""bar" with no space, that used to be taken as two identifiers,
but now it's just one identifier with an embedded quote mark.
I converted all the src/test/isolation/ specfiles to remove
unnecessary double quotes, but stopped there because my
eyes were glazing over already.
Like 741d7f104, back-patch to all supported branches, so that this
isn't a stumbling block for back-patching isolation test changes.
Discussion: https://postgr.es/m/759113.1623861959@sss.pgh.pa.us
M contrib/test_decoding/specs/oldest_xmin.spec
M src/test/isolation/README
M src/test/isolation/specparse.y
M src/test/isolation/specs/aborted-keyrevoke.spec
M src/test/isolation/specs/alter-table-1.spec
M src/test/isolation/specs/alter-table-2.spec
M src/test/isolation/specs/alter-table-3.spec
M src/test/isolation/specs/alter-table-4.spec
M src/test/isolation/specs/async-notify.spec
M src/test/isolation/specs/classroom-scheduling.spec
M src/test/isolation/specs/create-trigger.spec
M src/test/isolation/specs/deadlock-hard.spec
M src/test/isolation/specs/deadlock-parallel.spec
M src/test/isolation/specs/deadlock-simple.spec
M src/test/isolation/specs/deadlock-soft-2.spec
M src/test/isolation/specs/deadlock-soft.spec
M src/test/isolation/specs/delete-abort-savept-2.spec
M src/test/isolation/specs/delete-abort-savept.spec
M src/test/isolation/specs/detach-partition-concurrently-1.spec
M src/test/isolation/specs/detach-partition-concurrently-2.spec
M src/test/isolation/specs/detach-partition-concurrently-3.spec
M src/test/isolation/specs/detach-partition-concurrently-4.spec
M src/test/isolation/specs/drop-index-concurrently-1.spec
M src/test/isolation/specs/eval-plan-qual-trigger.spec
M src/test/isolation/specs/eval-plan-qual.spec
M src/test/isolation/specs/fk-contention.spec
M src/test/isolation/specs/fk-deadlock.spec
M src/test/isolation/specs/fk-deadlock2.spec
M src/test/isolation/specs/fk-partitioned-1.spec
M src/test/isolation/specs/fk-partitioned-2.spec
M src/test/isolation/specs/freeze-the-dead.spec
M src/test/isolation/specs/horizons.spec
M src/test/isolation/specs/index-only-scan.spec
M src/test/isolation/specs/inherit-temp.spec
M src/test/isolation/specs/insert-conflict-do-nothing-2.spec
M src/test/isolation/specs/insert-conflict-do-nothing.spec
M src/test/isolation/specs/insert-conflict-do-update-2.spec
M src/test/isolation/specs/insert-conflict-do-update-3.spec
M src/test/isolation/specs/insert-conflict-do-update.spec
M src/test/isolation/specs/insert-conflict-specconflict.spec
M src/test/isolation/specs/lock-committed-keyupdate.spec
M src/test/isolation/specs/lock-committed-update.spec
M src/test/isolation/specs/lock-update-delete.spec
M src/test/isolation/specs/lock-update-traversal.spec
M src/test/isolation/specs/multiple-cic.spec
M src/test/isolation/specs/multiple-row-versions.spec
M src/test/isolation/specs/multixact-no-deadlock.spec
M src/test/isolation/specs/multixact-no-forget.spec
M src/test/isolation/specs/nowait-2.spec
M src/test/isolation/specs/nowait-3.spec
M src/test/isolation/specs/nowait-4.spec
M src/test/isolation/specs/nowait-5.spec
M src/test/isolation/specs/nowait.spec
M src/test/isolation/specs/partial-index.spec
M src/test/isolation/specs/partition-concurrent-attach.spec
M src/test/isolation/specs/partition-key-update-1.spec
M src/test/isolation/specs/partition-key-update-2.spec
M src/test/isolation/specs/partition-key-update-3.spec
M src/test/isolation/specs/partition-key-update-4.spec
M src/test/isolation/specs/plpgsql-toast.spec
M src/test/isolation/specs/predicate-gin.spec
M src/test/isolation/specs/predicate-gist.spec
M src/test/isolation/specs/predicate-hash.spec
M src/test/isolation/specs/predicate-lock-hot-tuple.spec
M src/test/isolation/specs/prepared-transactions-cic.spec
M src/test/isolation/specs/prepared-transactions.spec
M src/test/isolation/specs/project-manager.spec
M src/test/isolation/specs/propagate-lock-delete.spec
M src/test/isolation/specs/read-only-anomaly-2.spec
M src/test/isolation/specs/read-only-anomaly-3.spec
M src/test/isolation/specs/read-only-anomaly.spec
M src/test/isolation/specs/read-write-unique-2.spec
M src/test/isolation/specs/read-write-unique-3.spec
M src/test/isolation/specs/read-write-unique-4.spec
M src/test/isolation/specs/read-write-unique.spec
M src/test/isolation/specs/receipt-report.spec
M src/test/isolation/specs/referential-integrity.spec
M src/test/isolation/specs/reindex-concurrently.spec
M src/test/isolation/specs/reindex-schema.spec
M src/test/isolation/specs/ri-trigger.spec
M src/test/isolation/specs/sequence-ddl.spec
M src/test/isolation/specs/serializable-parallel-2.spec
M src/test/isolation/specs/serializable-parallel.spec
M src/test/isolation/specs/simple-write-skew.spec
M src/test/isolation/specs/skip-locked-2.spec
M src/test/isolation/specs/skip-locked-3.spec
M src/test/isolation/specs/skip-locked-4.spec
M src/test/isolation/specs/skip-locked.spec
M src/test/isolation/specs/temporal-range-integrity.spec
M src/test/isolation/specs/timeouts.spec
M src/test/isolation/specs/total-cash.spec
M src/test/isolation/specs/truncate-conflict.spec
M src/test/isolation/specs/tuplelock-conflict.spec
M src/test/isolation/specs/tuplelock-partition.spec
M src/test/isolation/specs/tuplelock-update.spec
M src/test/isolation/specs/tuplelock-upgrade-no-deadlock.spec
M src/test/isolation/specs/two-ids.spec
M src/test/isolation/specs/update-conflict-out.spec
M src/test/isolation/specs/update-locked-tuple.spec
M src/test/isolation/specs/vacuum-concurrent-drop.spec
M src/test/isolation/specs/vacuum-conflict.spec
M src/test/isolation/specs/vacuum-reltuples.spec
M src/test/isolation/specs/vacuum-skip-locked.spec
M src/test/isolation/specscanner.l
Doc: fix confusion about LEAKPROOF in syntax summaries.
commit : 2031e1668e5577e64cfed29da69a34903d5a5227
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Jun 2021 14:27:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Jun 2021 14:27:13 -0400
The syntax summaries for CREATE FUNCTION and allied commands
made it look like LEAKPROOF is an alternative to
IMMUTABLE/STABLE/VOLATILE, when of course it is an orthogonal
option. Improve that.
Per gripe from aazamrafeeque0. Thanks to David Johnston for
suggestions.
Discussion: https://postgr.es/m/162444349581.694.5818572718530259025@wrigleys.postgresql.org
M doc/src/sgml/ref/alter_function.sgml
M doc/src/sgml/ref/alter_routine.sgml
M doc/src/sgml/ref/create_function.sgml
Don't assume GSSAPI result strings are null-terminated.
commit : 126cdaf47af275f76b2f2ddb023bfdc6f018ae30
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Jun 2021 14:01:32 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Jun 2021 14:01:32 -0400
Our uses of gss_display_status() and gss_display_name() assumed
that the gss_buffer_desc strings returned by those functions are
null-terminated. It appears that they generally are, given the
lack of field complaints up to now. However, the available
documentation does not promise this, and some man pages
for gss_display_status() show examples that rely on the
gss_buffer_desc.length field instead of expecting null
termination. Also, we now have a report that on some
implementations, clang's address sanitizer is of the opinion
that the byte after the specified length is undefined.
Hence, change the code to rely on the length field instead.
This might well be cosmetic rather than fixing any real bug, but
it's hard to be sure, so back-patch to all supported branches.
While here, also back-patch the v12 changes that made pg_GSS_error
deal honestly with multiple messages available from
gss_display_status.
Per report from Sudheer H R.
Discussion: https://postgr.es/m/5372B6D4-8276-42C0-B8FB-BD0918826FC3@tekenlight.com
M src/backend/libpq/auth.c
M src/backend/libpq/be-gssapi-common.c
M src/interfaces/libpq/fe-gssapi-common.c
Improve display of query results in isolation tests.
commit : 4a054069a36032a59afceb07f3b837f09ab1a2e9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Jun 2021 11:12:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Jun 2021 11:12:31 -0400
Previously, isolationtester displayed SQL query results using some
ad-hoc code that clearly hadn't had much effort expended on it.
Field values longer than 14 characters weren't separated from
the next field, and usually caused misalignment of the columns
too. Also there was no visual separation of a query's result
from subsequent isolationtester output. This made test result
files confusing and hard to read.
To improve matters, let's use libpq's PQprint() function. Although
that's long since unused by psql, it's still plenty good enough
for the purpose here.
Like 741d7f104, back-patch to all supported branches, so that this
isn't a stumbling block for back-patching isolation test changes.
Discussion: https://postgr.es/m/582362.1623798221@sss.pgh.pa.us
M contrib/test_decoding/expected/concurrent_ddl_dml.out
M contrib/test_decoding/expected/concurrent_stream.out
M contrib/test_decoding/expected/delayed_startup.out
M contrib/test_decoding/expected/mxact.out
M contrib/test_decoding/expected/oldest_xmin.out
M contrib/test_decoding/expected/ondisk_startup.out
M contrib/test_decoding/expected/snapshot_transfer.out
M contrib/test_decoding/expected/subxact_without_top.out
M contrib/test_decoding/expected/twophase_snapshot.out
M src/test/isolation/expected/aborted-keyrevoke.out
M src/test/isolation/expected/alter-table-1.out
M src/test/isolation/expected/alter-table-2.out
M src/test/isolation/expected/alter-table-3.out
M src/test/isolation/expected/alter-table-4.out
M src/test/isolation/expected/async-notify.out
M src/test/isolation/expected/classroom-scheduling.out
M src/test/isolation/expected/create-trigger.out
M src/test/isolation/expected/deadlock-parallel.out
M src/test/isolation/expected/delete-abort-savept-2.out
M src/test/isolation/expected/delete-abort-savept.out
M src/test/isolation/expected/detach-partition-concurrently-1.out
M src/test/isolation/expected/detach-partition-concurrently-2.out
M src/test/isolation/expected/detach-partition-concurrently-3.out
M src/test/isolation/expected/detach-partition-concurrently-4.out
M src/test/isolation/expected/drop-index-concurrently-1.out
M src/test/isolation/expected/drop-index-concurrently-1_2.out
M src/test/isolation/expected/eval-plan-qual-trigger.out
M src/test/isolation/expected/eval-plan-qual.out
M src/test/isolation/expected/fk-partitioned-2.out
M src/test/isolation/expected/freeze-the-dead.out
M src/test/isolation/expected/horizons.out
M src/test/isolation/expected/inherit-temp.out
M src/test/isolation/expected/insert-conflict-do-nothing-2.out
M src/test/isolation/expected/insert-conflict-do-nothing.out
M src/test/isolation/expected/insert-conflict-do-update-2.out
M src/test/isolation/expected/insert-conflict-do-update-3.out
M src/test/isolation/expected/insert-conflict-do-update.out
M src/test/isolation/expected/insert-conflict-specconflict.out
M src/test/isolation/expected/lock-committed-keyupdate.out
M src/test/isolation/expected/lock-committed-update.out
M src/test/isolation/expected/lock-update-delete.out
M src/test/isolation/expected/lock-update-delete_1.out
M src/test/isolation/expected/lock-update-traversal.out
M src/test/isolation/expected/multiple-cic.out
M src/test/isolation/expected/multiple-row-versions.out
M src/test/isolation/expected/multixact-no-deadlock.out
M src/test/isolation/expected/multixact-no-forget.out
M src/test/isolation/expected/multixact-no-forget_1.out
M src/test/isolation/expected/nowait-2.out
M src/test/isolation/expected/nowait-3.out
M src/test/isolation/expected/nowait-4.out
M src/test/isolation/expected/nowait-4_1.out
M src/test/isolation/expected/nowait-5.out
M src/test/isolation/expected/nowait.out
M src/test/isolation/expected/partial-index.out
M src/test/isolation/expected/partition-concurrent-attach.out
M src/test/isolation/expected/partition-key-update-2.out
M src/test/isolation/expected/partition-key-update-3.out
M src/test/isolation/expected/partition-key-update-4.out
M src/test/isolation/expected/plpgsql-toast.out
M src/test/isolation/expected/predicate-gin.out
M src/test/isolation/expected/predicate-gist.out
M src/test/isolation/expected/predicate-hash.out
M src/test/isolation/expected/predicate-lock-hot-tuple.out
M src/test/isolation/expected/prepared-transactions-cic.out
M src/test/isolation/expected/prepared-transactions.out
M src/test/isolation/expected/project-manager.out
M src/test/isolation/expected/read-only-anomaly-2.out
M src/test/isolation/expected/read-only-anomaly-3.out
M src/test/isolation/expected/read-only-anomaly.out
M src/test/isolation/expected/read-write-unique-2.out
M src/test/isolation/expected/read-write-unique-3.out
M src/test/isolation/expected/read-write-unique-4.out
M src/test/isolation/expected/read-write-unique.out
M src/test/isolation/expected/receipt-report.out
M src/test/isolation/expected/referential-integrity.out
M src/test/isolation/expected/reindex-concurrently.out
M src/test/isolation/expected/ri-trigger.out
M src/test/isolation/expected/sequence-ddl.out
M src/test/isolation/expected/serializable-parallel-2.out
M src/test/isolation/expected/serializable-parallel.out
M src/test/isolation/expected/skip-locked-2.out
M src/test/isolation/expected/skip-locked-3.out
M src/test/isolation/expected/skip-locked-4.out
M src/test/isolation/expected/skip-locked-4_1.out
M src/test/isolation/expected/skip-locked.out
M src/test/isolation/expected/temporal-range-integrity.out
M src/test/isolation/expected/timeouts.out
M src/test/isolation/expected/total-cash.out
M src/test/isolation/expected/truncate-conflict.out
M src/test/isolation/expected/tuplelock-conflict.out
M src/test/isolation/expected/tuplelock-partition.out
M src/test/isolation/expected/tuplelock-update.out
M src/test/isolation/expected/tuplelock-upgrade-no-deadlock.out
M src/test/isolation/expected/two-ids.out
M src/test/isolation/expected/update-conflict-out.out
M src/test/isolation/expected/vacuum-reltuples.out
M src/test/isolation/isolationtester.c
M src/test/modules/brin/expected/summarization-and-inprogress-insertion.out
M src/test/modules/delay_execution/expected/partition-addition.out
M src/test/modules/delay_execution/expected/partition-removal-1.out
M src/test/modules/snapshot_too_old/expected/sto_using_cursor.out
M src/test/modules/snapshot_too_old/expected/sto_using_hash_index.out
M src/test/modules/snapshot_too_old/expected/sto_using_select.out
Add test case for obsoleting slot with active walsender, take 2
commit : 24043c27b46f873211177e3460ab09dc011802a5
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 23 Jun 2021 09:53:18 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 23 Jun 2021 09:53:18 -0400
The code to signal a running walsender when its reserved WAL size grows
too large is completely uncovered before this commit; this adds coverage
for that case.
This test involves sending SIGSTOP to walsender and walreceiver, then
advancing enough WAL for a checkpoint to trigger, then sending SIGCONT.
There's no precedent for STOP signalling in Perl tests, and my reading
of relevant manpages says it's likely to fail on Windows. Because of
this, this test is always skipped on that platform.
This version fixes a couple of rarely hit race conditions in the
previous attempt 09126984a263; most notably, both LOG string searches
are loops, not just the second one; we acquire the start-of-log position
before STOP-signalling; and reference the correct process name in the
test description. All per Tom Lane.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/202106102202.mjw4huiix7lo@alvherre.pgsql
M src/test/recovery/t/019_replslot_limit.pl
Use annotations to reduce instability of isolation-test results.
commit : 741d7f1047fe52da7ced6fa9cea661ce9320c8d4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Jun 2021 21:43:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Jun 2021 21:43:12 -0400
We've long contended with isolation test results that aren't entirely
stable. Some test scripts insert long delays to try to force stable
results, which is not terribly desirable; but other erratic failure
modes remain, causing unrepeatable buildfarm failures. I've spent a
fair amount of time trying to solve this by improving the server-side
support code, without much success: that way is fundamentally unable
to cope with diffs that stem from chance ordering of arrival of
messages from different server processes.
We can improve matters on the client side, however, by annotating
the test scripts themselves to show the desired reporting order
of events that might occur in different orders. This patch adds
three types of annotations to deal with (a) test steps that might or
might not complete their waits before the isolationtester can see them
waiting; (b) test steps in different sessions that can legitimately
complete in either order; and (c) NOTIFY messages that might arrive
before or after the completion of a step in another session. We might
need more annotation types later, but this seems to be enough to deal
with the instabilities we've seen in the buildfarm. It also lets us
get rid of all the long delays that were previously used, cutting more
than a minute off the runtime of the isolation tests.
Back-patch to all supported branches, because the buildfarm
instabilities affect all the branches, and because it seems desirable
to keep isolationtester's capabilities the same across all branches
to simplify possible future back-patching of tests.
Discussion: https://postgr.es/m/327948.1623725828@sss.pgh.pa.us
M contrib/test_decoding/expected/concurrent_ddl_dml.out
M src/test/isolation/README
M src/test/isolation/expected/alter-table-3.out
M src/test/isolation/expected/alter-table-4.out
M src/test/isolation/expected/deadlock-hard.out
M src/test/isolation/expected/deadlock-simple.out
M src/test/isolation/expected/detach-partition-concurrently-3.out
M src/test/isolation/expected/detach-partition-concurrently-4.out
M src/test/isolation/expected/eval-plan-qual-trigger.out
M src/test/isolation/expected/eval-plan-qual.out
M src/test/isolation/expected/fk-deadlock2_1.out
M src/test/isolation/expected/fk-deadlock2_2.out
M src/test/isolation/expected/fk-deadlock_1.out
M src/test/isolation/expected/fk-partitioned-1.out
M src/test/isolation/expected/fk-partitioned-2.out
M src/test/isolation/expected/insert-conflict-do-nothing-2.out
M src/test/isolation/expected/insert-conflict-specconflict.out
M src/test/isolation/expected/lock-committed-keyupdate.out
M src/test/isolation/expected/lock-update-delete_1.out
M src/test/isolation/expected/multiple-cic.out
D src/test/isolation/expected/multiple-cic_1.out
M src/test/isolation/expected/multixact-no-forget_1.out
M src/test/isolation/expected/nowait-4.out
M src/test/isolation/expected/nowait-4_1.out
M src/test/isolation/expected/nowait-5.out
M src/test/isolation/expected/partition-concurrent-attach.out
M src/test/isolation/expected/partition-key-update-1.out
M src/test/isolation/expected/partition-key-update-3.out
M src/test/isolation/expected/propagate-lock-delete.out
M src/test/isolation/expected/read-write-unique-2.out
M src/test/isolation/expected/read-write-unique-3.out
M src/test/isolation/expected/read-write-unique-4.out
M src/test/isolation/expected/read-write-unique.out
M src/test/isolation/expected/sequence-ddl.out
M src/test/isolation/expected/skip-locked-4_1.out
M src/test/isolation/expected/timeouts.out
M src/test/isolation/expected/tuplelock-update.out
M src/test/isolation/isolationtester.c
M src/test/isolation/isolationtester.h
M src/test/isolation/specparse.y
M src/test/isolation/specs/deadlock-hard.spec
M src/test/isolation/specs/deadlock-soft-2.spec
M src/test/isolation/specs/detach-partition-concurrently-3.spec
M src/test/isolation/specs/detach-partition-concurrently-4.spec
M src/test/isolation/specs/insert-conflict-specconflict.spec
M src/test/isolation/specs/multiple-cic.spec
M src/test/isolation/specs/timeouts.spec
M src/test/isolation/specs/tuplelock-update.spec
M src/test/isolation/specscanner.l
Restore the portal-level snapshot for simple expressions, too.
commit : d102aafb6259a6a412803d4b1d8c4f00aa17f67e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Jun 2021 17:48:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Jun 2021 17:48:39 -0400
Commits 84f5c2908 et al missed the need to cover plpgsql's "simple
expression" code path. If the first thing we execute after a
COMMIT/ROLLBACK is one of those, rather than a full-fledged SPI command,
we must explicitly do EnsurePortalSnapshotExists() to make sure we have
an outer snapshot. Note that it wouldn't be good enough to just push a
snapshot for the duration of the expression execution: what comes back
might be toasted, so we'd better have a snapshot protecting it.
The test case demonstrating this fact cheats a bit by marking a SQL
function immutable even though it fetches from a table. That's
nothing that users haven't been seen to do, though.
Per report from Jim Nasby. Back-patch to v11, like the previous fix.
Discussion: https://postgr.es/m/378885e4-f85f-fc28-6c91-c4d1c080bf26@amazon.com
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_transaction.sql
Add list of ignorable pgindent commits for git-blame.
commit : 8e638845ff6bb255ad1dea15831089a234535391
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 22 Jun 2021 09:06:32 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 22 Jun 2021 09:06:32 -0700
Add a .git-blame-ignore-revs file with a list of pgindent, pgperlyidy,
and reformat-dat-files commit hashes. Postgres hackers that configure
git to use the ignore file will get git-blame output that avoids
attributing line changes to the ignored indent commits. This makes
git-blame output much easier to work with in practice.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAH2-Wz=cVh3GHTP6SdLU-Gnmt2zRdF8vZkcrFdSzXQ=WhbWm9Q@mail.gmail.com
A .git-blame-ignore-revs
M src/tools/RELEASE_CHANGES
M src/tools/pgindent/README
Stamp 14beta2.
commit : bafad2c5b261a1449bed60ebac9e7918ebcc42b4
author : Joe Conway <mail@joeconway.com>
date : Mon, 21 Jun 2021 17:07:55 -0400
committer: Joe Conway <mail@joeconway.com>
date : Mon, 21 Jun 2021 17:07:55 -0400
M configure
M configure.ac
Use correct horizon when vacuuming catalog relations.
commit : 5a1e1d83022b976ebdec5cfa8f255c4278b75b8e
author : Andres Freund <andres@anarazel.de>
date : Mon, 21 Jun 2021 05:13:46 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 21 Jun 2021 05:13:46 -0700
In dc7420c2c92 I (Andres) accidentally used
RelationIsAccessibleInLogicalDecoding() as the sole condition to use the
non-shared catalog horizon in GetOldestNonRemovableTransactionId(). That is
incorrect, as RelationIsAccessibleInLogicalDecoding() checks whether wal_level
is logical.
The correct check, as done e.g. in GlobalVisTestFor(), is to check
IsCatalogRelation() and RelationIsAccessibleInLogicalDecoding().
The observed misbehavior of this bug was that there could be an endless loop
in lazy_scan_prune(), because the horizons used in heap_page_prune() and the
individual tuple liveliness checks did not match. Likely there are other
potential consequences as well.
A later commit will unify the determination which horizon has to be used, and
add additional assertions to make it easier to catch a bug like this.
Reported-By: Justin Pryzby <pryzby@telsasoft.com>
Diagnosed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAEze2Wg32Y9+WJfw=aofkRx1ZRFt_Ev6bNPc4PSaz7PjSFtZgQ@mail.gmail.com
M src/backend/storage/ipc/procarray.c
Fix assert failure in expand_grouping_sets
commit : 8d29d45d9b3cab95a866efbcdd9138b3d76741b3
author : David Rowley <drowley@postgresql.org>
date : Mon, 21 Jun 2021 23:11:23 +1200
committer: David Rowley <drowley@postgresql.org>
date : Mon, 21 Jun 2021 23:11:23 +1200
linitial_node() fails in assert enabled builds if the given pointer is
not of the specified type. Here the type is IntList. The code thought
it should be expecting List, but it was wrong.
In the existing tests which run this code the initial list element is
always NIL. Since linitial_node() allows NULL, we didn't trigger any
assert failures in the existing regression tests.
There is still some discussion as to whether we need a few more tests in
this area, but for now, since beta2 is looming, fix the bug first.
Bug: #17067
Discussion: https://postgr.es/m/17067-665d50fa321f79e0@postgresql.org
Reported-by: Yaoguang Chen
M src/backend/parser/parse_agg.c
Translation updates
commit : a7bb0ce58f56ee8907c3f49c52d99f502536c796
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 21 Jun 2021 12:32:14 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 21 Jun 2021 12:32:14 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 70796ae860c444c764bb591c885f22cac1c168ec
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/bin/initdb/po/el.po
M src/bin/initdb/po/es.po
M src/bin/pg_amcheck/nls.mk
A src/bin/pg_amcheck/po/el.po
A src/bin/pg_amcheck/po/es.po
M src/bin/pg_amcheck/po/fr.po
A src/bin/pg_amcheck/po/zh_CN.po
M src/bin/pg_archivecleanup/po/el.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_checksums/po/el.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_config/po/el.po
M src/bin/pg_config/po/es.po
M src/bin/pg_controldata/nls.mk
A src/bin/pg_controldata/po/el.po
M src/bin/pg_ctl/po/el.po
M src/bin/pg_ctl/po/es.po
M src/bin/pg_dump/po/el.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/zh_CN.po
M src/bin/pg_test_fsync/po/el.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_timing/po/el.po
M src/bin/pg_test_timing/po/es.po
M src/bin/pg_test_timing/po/zh_CN.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_verifybackup/nls.mk
A src/bin/pg_verifybackup/po/el.po
M src/bin/pg_verifybackup/po/es.po
M src/bin/pg_waldump/nls.mk
A src/bin/pg_waldump/po/el.po
M src/bin/psql/po/el.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/fr.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/fr.po
M src/interfaces/libpq/po/el.po
M src/interfaces/libpq/po/es.po
M src/interfaces/libpq/po/fr.po
M src/pl/plperl/nls.mk
A src/pl/plperl/po/el.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpython/nls.mk
A src/pl/plpython/po/el.po
M src/pl/tcl/po/el.po
Finish rename of PQtraceSetFlags() to PQsetTraceFlags().
commit : 047a259e35b9dde2dad5fd0e5d5d784bb327b848
author : Noah Misch <noah@leadboat.com>
date : Mon, 21 Jun 2021 02:48:11 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 21 Jun 2021 02:48:11 -0700
Jie Zhang
Discussion: https://postgr.es/m/TYWPR01MB767844835390EDD8DB276D75F90A9@TYWPR01MB7678.jpnprd01.prod.outlook.com
M doc/src/sgml/libpq.sgml
amcheck: Fix code comments
commit : 97b7134186490b36e01efc9d2feaf7038c666457
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 21 Jun 2021 11:17:49 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 21 Jun 2021 11:17:49 +0200
Code comments were claiming that verify_heapam() was checking
privileges on the relation it was operating on, but it didn't actually
do that. Perhaps earlier versions of the patch did that, but now the
access is regulated by privileges on the function. Remove the wrong
comments.
M contrib/amcheck/verify_heapam.c
doc: adjust PG 14 relnotes to be current
commit : 69a58bfe4ab05567a8fab8bdce7f3095ed06b99c
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 21 Jun 2021 01:09:32 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 21 Jun 2021 01:09:32 -0400
M doc/src/sgml/release-14.sgml
doc: add mention of +4GB windows file handling in PG14 relnotes
commit : 90855908b751d40f67352fa0252e0fcdaa7e317b
author : Bruce Momjian <bruce@momjian.us>
date : Sun, 20 Jun 2021 23:53:00 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sun, 20 Jun 2021 23:53:00 -0400
Reported-by: Masahiko Sawada
Discussion: https://postgr.es/m/CAD21AoCTHyouoGv-xt1qNjjvPbGMErLi0AJncByTvr66Nq7j8g@mail.gmail.com
M doc/src/sgml/release-14.sgml
Remove overzealous VACUUM failsafe assertions.
commit : e8f201ab82be234b2f103234cf9f262f9afeaeba
author : Peter Geoghegan <pg@bowt.ie>
date : Sun, 20 Jun 2021 18:14:00 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Sun, 20 Jun 2021 18:14:00 -0700
The failsafe can trigger when index processing is already disabled.
This can happen when VACUUM's INDEX_CLEANUP parameter is "off" and the
failsafe happens to trigger. Remove assertions that assume that index
processing is directly tied to the failsafe.
Oversight in commit c242baa4, which made it possible for the failsafe to
trigger in a two-pass strategy VACUUM that has yet to make its first
call to lazy_vacuum_all_indexes().
M src/backend/access/heap/vacuumlazy.c
Revert "Add test case for obsoleting slot with active walsender"
commit : 96795176810b979a2bf1f4563bdcb161679d5b95
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sun, 20 Jun 2021 12:28:08 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sun, 20 Jun 2021 12:28:08 -0400
This reverts commit 09126984a263; the test case added there failed once
in circumstances that remain mysterious. It seems better to remove the
test for now so that 14beta2 doesn't have random failures built in.
M src/test/recovery/t/019_replslot_limit.pl
Stabilize test case added by commit f61db909d.
commit : 5843659d091bfb6f2c60e010ea1fd00e55ee6ada
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Jun 2021 11:48:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Jun 2021 11:48:44 -0400
Buildfarm members ayu and tern have sometimes shown a different
plan than expected for this query. I'd been unable to reproduce
that before today, but I finally realized what is happening.
If there is a concurrent open transaction (probably an autovacuum
run in the buildfarm, but this can also be arranged manually),
then the index entries for the rows removed by the DELETE a few
lines up are not killed promptly, causing a change in the planner's
estimate of the extremal value of ft2.c1, which moves the rowcount
estimate for "c1 > 1100" by enough to change the join plan from
nestloop to hash.
To fix, change the query condition to "c1 > 1000", causing the
hash plan to be preferred whether or not a concurrent open
transaction exists. Since this UPDATE is tailored to be a no-op,
nothing else changes.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=ayu&dt=2021-06-09%2022%3A45%3A48
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=ayu&dt=2021-06-13%2022%3A38%3A18
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tern&dt=2021-06-20%2004%3A55%3A36
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Provide feature-test macros for libpq features added in v14.
commit : 6991e774e0304f5ef488cf1ae4fa79578b6ae3d5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 19 Jun 2021 11:44:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 19 Jun 2021 11:44:39 -0400
We had a request to provide a way to test at compile time for the
availability of the new pipeline features. More generally, it
seems like a good idea to provide a way to test via #ifdef for
all new libpq API features. People have been using the version
from pg_config.h for that; but that's more likely to represent the
server version than the libpq version, in the increasingly-common
scenario where they're different. It's safer if libpq-fe.h itself
is the source of truth about what features it offers.
Hence, establish a policy that starting in v14 we'll add a suitable
feature-is-present macro to libpq-fe.h when we add new API there.
(There doesn't seem to be much point in applying this policy
retroactively, but it's not too late for v14.)
Tom Lane and Alvaro Herrera, per suggestion from Boris Kolpackov.
Discussion: https://postgr.es/m/boris.20210617102439@codesynthesis.com
M src/interfaces/libpq/libpq-fe.h
Handle no replica identity index case in RelationGetIdentityKeyBitmap.
commit : 2731ce1bd550d08f3fdd7bcb1497af4b95170976
author : Amit Kapila <akapila@postgresql.org>
date : Sat, 19 Jun 2021 11:30:33 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Sat, 19 Jun 2021 11:30:33 +0530
Commit e7eea52b2d has introduced a new function
RelationGetIdentityKeyBitmap which omits to handle the case where there is
no replica identity index on a relation.
Author: Mark Dilger
Reviewed-by: Takamichi Osumi, Amit Kapila
Discussion: https://www.postgresql.org/message-id/4C99A862-69C8-431F-960A-81B1151F1B89@enterprisedb.com
M src/backend/utils/cache/relcache.c
M src/test/subscription/t/001_rep_changes.pl
Support disabling index bypassing by VACUUM.
commit : 3499df0dee8c4ea51d264a674df5b5e31991319a
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 18 Jun 2021 20:04:07 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 18 Jun 2021 20:04:07 -0700
Generalize the INDEX_CLEANUP VACUUM parameter (and the corresponding
reloption): make it into a ternary style boolean parameter. It now
exposes a third option, "auto". The "auto" option (which is now the
default) enables the "bypass index vacuuming" optimization added by
commit 1e55e7d1.
"VACUUM (INDEX_CLEANUP TRUE)" is redefined to once again make VACUUM
simply do any required index vacuuming, regardless of how few dead
tuples are encountered during the first scan of the target heap relation
(unless there are exactly zero). This gives users a way of opting out
of the "bypass index vacuuming" optimization, if for whatever reason
that proves necessary. It is also expected to be used by PostgreSQL
developers as a testing option from time to time.
"VACUUM (INDEX_CLEANUP FALSE)" does the same thing as it always has: it
forcibly disables both index vacuuming and index cleanup. It's not
expected to be used much in PostgreSQL 14. The failsafe mechanism added
by commit 1e55e7d1 addresses the same problem in a simpler way.
INDEX_CLEANUP can now be thought of as a testing and compatibility
option.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-By: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/CAH2-WznrBoCST4_Gxh_G9hA8NzGUbeBGnOUC8FcXcrhqsv6OHQ@mail.gmail.com
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/vacuum.sgml
M doc/src/sgml/ref/vacuumdb.sgml
M src/backend/access/common/reloptions.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/vacuum.c
M src/backend/postmaster/autovacuum.c
M src/bin/psql/tab-complete.c
M src/bin/scripts/vacuumdb.c
M src/include/commands/vacuum.h
M src/include/utils/rel.h
M src/test/regress/expected/vacuum.out
M src/test/regress/sql/vacuum.sql
Add test case for obsoleting slot with active walsender
commit : 09126984a2631db8dd6299f23954e0dede69735f
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 18 Jun 2021 18:42:00 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 18 Jun 2021 18:42:00 -0400
The code to signal a running walsender when its reserved WAL size grows
too large is completely uncovered before this commit; this adds coverage
for that case.
This test involves sending SIGSTOP to walsender and walreceiver and
running a checkpoint while advancing WAL, then sending SIGCONT. There's
no precedent for this coding in Perl tests, and my reading of relevant
manpages says it's likely to fail on Windows. Because of this, this
test is always skipped on that platform.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/202106102202.mjw4huiix7lo@alvherre.pgsql
M src/test/recovery/t/019_replslot_limit.pl
Fix misbehavior of DROP OWNED BY with duplicate polroles entries.
commit : d21fca084356946664bfce19d66d2df2bb873cbd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Jun 2021 18:00:09 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Jun 2021 18:00:09 -0400
Ordinarily, a pg_policy.polroles array wouldn't list the same role
more than once; but CREATE POLICY does not prevent that. If we
perform DROP OWNED BY on a role that is listed more than once,
RemoveRoleFromObjectPolicy either suffered an assertion failure
or encountered a tuple-updated-by-self error. Rewrite it to cope
correctly with duplicate entries, and add a CommandCounterIncrement
call to prevent the other problem.
Per discussion, there's other cleanup that ought to happen here,
but this seems like the minimum essential fix.
Per bug #17062 from Alexander Lakhin. It's been broken all along,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/17062-11f471ae3199ca23@postgresql.org
M src/backend/commands/policy.c
M src/test/regress/expected/rowsecurity.out
M src/test/regress/sql/rowsecurity.sql
Improve version reporting in pgbench.
commit : 84bee9610965331d5110971d8de390a5bbe2effc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Jun 2021 17:05:23 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Jun 2021 17:05:23 -0400
Commit 547f04e73 caused pgbench to start printing its version number,
which seems like a fine idea, but it needs a bit more work:
* Print the server version number too, when different.
* Print the PG_VERSION string, not some reconstructed approximation.
This patch copies psql's well-tested code for the same purpose.
Discussion: https://postgr.es/m/1226654.1624036821@sss.pgh.pa.us
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
Centralize the logic for protective copying of utility statements.
commit : 7c337b6b527b7052e6a751f966d5734c56f668b5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Jun 2021 11:22:58 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Jun 2021 11:22:58 -0400
In the "simple Query" code path, it's fine for parse analysis or
execution of a utility statement to scribble on the statement's node
tree, since that'll just be thrown away afterwards. However it's
not fine if the node tree is in the plan cache, as then it'd be
corrupted for subsequent executions. Up to now we've dealt with
that by having individual utility-statement functions apply
copyObject() if they were going to modify the tree. But that's
prone to errors of omission. Bug #17053 from Charles Samborski
shows that CREATE/ALTER DOMAIN didn't get this memo, and can
crash if executed repeatedly from plan cache.
In the back branches, we'll just apply a narrow band-aid for that,
but in HEAD it seems prudent to have a more principled fix that
will close off the possibility of other similar bugs in future.
Hence, let's hoist the responsibility for doing copyObject up into
ProcessUtility from its children, thus ensuring that it happens for
all utility statement types.
Also, modify ProcessUtility's API so that its callers can tell it
whether a copy step is necessary. It turns out that in all cases,
the immediate caller knows whether the node tree is transient, so
this doesn't involve a huge amount of code thrashing. In this way,
while we lose a little bit in the execute-from-cache code path due
to sometimes copying node trees that wouldn't be mutated anyway,
we gain something in the simple-Query code path by not copying
throwaway node trees. Statements that are complex enough to be
expensive to copy are almost certainly ones that would have to be
copied anyway, so the loss in the cache code path shouldn't be much.
(Note that this whole problem applies only to utility statements.
Optimizable statements don't have the issue because we long ago made
the executor treat Plan trees as read-only. Perhaps someday we will
make utility statement execution act likewise, but I'm not holding
my breath.)
Discussion: https://postgr.es/m/931771.1623893989@sss.pgh.pa.us
Discussion: https://postgr.es/m/17053-3ca3f501bbc212b4@postgresql.org
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/sepgsql/hooks.c
M src/backend/commands/copyto.c
M src/backend/commands/createas.c
M src/backend/commands/explain.c
M src/backend/commands/extension.c
M src/backend/commands/foreigncmds.c
M src/backend/commands/policy.c
M src/backend/commands/portalcmds.c
M src/backend/commands/prepare.c
M src/backend/commands/schemacmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/view.c
M src/backend/executor/functions.c
M src/backend/executor/spi.c
M src/backend/parser/parse_utilcmd.c
M src/backend/tcop/pquery.c
M src/backend/tcop/utility.c
M src/include/tcop/utility.h
Don't set a fast default for anything but a plain table
commit : 0a4efdc7ebf2584257b166c87e82797eb92815b5
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 18 Jun 2021 06:51:12 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 18 Jun 2021 06:51:12 -0400
The fast default code added in Release 11 omitted to check that the
table a fast default was being added to was a plain table. Thus one
could be added to a foreign table, which predicably blows up. Here we
perform that check.
In addition, on the back branches, since some of these might have
escaped into the wild, if we encounter a missing value for
an attribute of something other than a plain table we ignore it.
Fixes bug #17056
Backpatch to release 11,
Reviewed by: Andres Freund, Álvaro Herrera and Tom Lane
M src/backend/catalog/heap.c
M src/test/regress/expected/fast_default.out
M src/test/regress/sql/fast_default.sql
Make archiver process handle barrier events.
commit : 981524d2e3aa3f28d364c472e34f5386be846811
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 18 Jun 2021 17:57:09 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 18 Jun 2021 17:57:09 +0900
Commit d75288fb27 made WAL archiver process an auxiliary process.
An auxiliary process needs to handle barrier events but the commit
forgot to make archiver process do that.
Reported-by: Thomas Munro
Author: Fujii Masao
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/CA+hUKGLah2w1pWKHonZP_+EQw69=q56AHYwCgEN8GDzsRG_Hgw@mail.gmail.com
M src/backend/postmaster/pgarch.c
doc: Apply markup <productname> to OpenSSL more consistently
commit : f80979f659d39e238e95444e6752142799428078
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 18 Jun 2021 14:22:31 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 18 Jun 2021 14:22:31 +0900
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/CE12DD5C-4BB3-4166-BC9A-39779568734C@yesql.se
M doc/src/sgml/config.sgml
M doc/src/sgml/libpq.sgml
Tidy up GetMultiXactIdMembers()'s behavior on error
commit : d24c5658a80c8f5037e9e1948de311d3f3350f12
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 17 Jun 2021 14:50:42 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 17 Jun 2021 14:50:42 +0300
One of the error paths left *members uninitialized. That's not a live
bug, because most callers don't look at *members when the function
returns -1, but let's be tidy. One caller, in heap_lock_tuple(), does
"if (members != NULL) pfree(members)", but AFAICS it never passes an
invalid 'multi' value so it should not reach that error case.
The callers are also a bit inconsistent in their expectations.
heap_lock_tuple() pfrees the 'members' array if it's not-NULL, others
pfree() it if "nmembers >= 0", and others if "nmembers > 0". That's
not a live bug either, because the function should never return 0, but
add an Assert for that to make it more clear. I left the callers alone
for now.
I also moved the line where we set *nmembers. It wasn't wrong before,
but I like to do that right next to the 'return' statement, to make it
clear that it's always set on return.
Also remove one unreachable return statement after ereport(ERROR), for
brevity and for consistency with the similar if-block right after it.
Author: Greg Nancarrow with the additional changes by me
Backpatch-through: 9.6, all supported versions
M src/backend/access/transam/multixact.c
Document a few caveats in synchronous logical replication.
commit : 3cb828dbe26087e7754f49f3cfe3ed036d5af439
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 17 Jun 2021 09:56:05 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 17 Jun 2021 09:56:05 +0530
In a synchronous logical setup, locking [user] catalog tables can cause
deadlock. This is because logical decoding of transactions can lock
catalog tables to access them so exclusively locking those in transactions
can lead to deadlock. To avoid this users must refrain from having
exclusive locks on catalog tables.
Author: Takamichi Osumi
Reviewed-by: Vignesh C, Amit Kapila
Backpatch-through: 9.6
Discussion: https://www.postgresql.org/message-id/20210222222847.tpnb6eg3yiykzpky%40alap3.anarazel.de
M doc/src/sgml/logicaldecoding.sgml
Fix plancache refcount leak after error in ExecuteQuery.
commit : 131ea3e908d3c97a2fe1ab25cce5046dd5cb905f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Jun 2021 19:30:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Jun 2021 19:30:17 -0400
When stuffing a plan from the plancache into a Portal, one is
not supposed to risk throwing an error between GetCachedPlan and
PortalDefineQuery; if that happens, the plan refcount incremented
by GetCachedPlan will be leaked. I managed to break this rule
while refactoring code in 9dbf2b7d7. There is no visible
consequence other than some memory leakage, and since nobody is
very likely to trigger the relevant error conditions many times
in a row, it's not surprising we haven't noticed. Nonetheless,
it's a bug, so rearrange the order of operations to remove the
hazard.
Noted on the way to looking for a better fix for bug #17053.
This mistake is pretty old, so back-patch to all supported
branches.
M src/backend/commands/prepare.c
Fix copying data into slots with FDW batching
commit : 99cea49d6525e30bc3768e4ffb95377e7584dea1
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 16 Jun 2021 22:53:31 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 16 Jun 2021 22:53:31 +0200
Commit b676ac443b optimized handling of tuple slots with bulk inserts
into foreign tables, so that the slots are initialized only once and
reused for all batches. The data was however copied into the slots only
after the initialization, inserting duplicate values when the slot gets
reused. Fixed by moving the ExecCopySlot outside the init branch.
The existing postgres_fdw tests failed to catch this due to inserting
data into foreign tables without unique indexes, and then checking only
the number of inserted rows. This adds a new test with both a unique
index and a check of inserted values.
Reported-by: Alexander Pyhalov
Discussion: https://postgr.es/m/7a8cf8d56b3d18e5c0bccd6cd42d04ac%40postgrespro.ru
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/executor/nodeModifyTable.c
Improve SQLSTATE reporting in some replication-related code.
commit : 6b787d9e32005867ee3660d1ea20f447810a403d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Jun 2021 11:52:05 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Jun 2021 11:52:05 -0400
I started out with the goal of reporting ERRCODE_CONNECTION_FAILURE
when walrcv_connect() fails, but as I looked around I realized that
whoever wrote this code was of the opinion that errcodes are purely
optional. That's not my understanding of our project policy. Hence,
make sure that an errcode is provided in each ereport that (a) is
ERROR or higher level and (b) isn't arguably an internal logic error.
Also fix some very dubious existing errcode assignments.
While this is not per policy, it's also largely cosmetic, since few
of these cases could get reported to applications. So I don't
feel a need to back-patch.
Discussion: https://postgr.es/m/2189704.1623512522@sss.pgh.pa.us
M src/backend/commands/subscriptioncmds.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/backend/replication/walreceiver.c
Fix outdated comment that talked about seek position of WAL file.
commit : d0303bc8d2670d11c9df9220aa08a2c33529e100
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 16 Jun 2021 12:34:32 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 16 Jun 2021 12:34:32 +0300
Since commit c24dcd0cfd, we have been using pg_pread() to read the WAL
file, which doesn't change the seek position (unless we fall back to
the implementation in src/port/pread.c). Update comment accordingly.
Backpatch-through: 12, where we started to use pg_pread()
M src/backend/access/transam/xlog.c
Update another variant expected-result file.
commit : d3c878499c9d639ff06e0664d06b8c731e30c2fc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Jun 2021 16:11:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Jun 2021 16:11:45 -0400
This should have been updated in 533e9c6b0, but it was overlooked.
Given the lack of complaints, I won't bother back-patching.
M src/test/isolation/expected/lock-update-delete_1.out
Remove another orphan expected-result file.
commit : f6352a0d4e437ac8bc266f77df22d064592056c9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Jun 2021 16:09:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Jun 2021 16:09:14 -0400
aborted-keyrevoke_2.out was apparently needed when it was added (in
commit 0ac5ad513) to handle the case of serializable transaction mode.
However, the output in serializable mode actually matches the regular
aborted-keyrevoke.out file, and AFAICT has done so for a long time.
There's no need to keep dragging this variant along.
D src/test/isolation/expected/aborted-keyrevoke_2.out
Further refinement of stuck_on_old_timeline recovery test
commit : 54a5ed22016940d7ad5060ed62d23473924756a1
author : Andrew Dunstan <andrew@dunslane.net>
date : Tue, 15 Jun 2021 15:30:11 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Tue, 15 Jun 2021 15:30:11 -0400
TestLib::perl2host can take a file argument as well as a directory
argument, so that code becomes substantially simpler. Also add comments
on why we're using forward slashes, and why we're setting
PERL_BADLANG=0.
Discussion: https://postgr.es/m/e9947bcd-20ee-027c-f0fe-01f736b7e345@dunslane.net
M src/test/recovery/t/025_stuck_on_old_timeline.pl
Revert 29854ee8d1 due to buildfarm failures
commit : 817bb0a7d1e02df4643d37304aed8574cf43f629
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 15 Jun 2021 21:43:17 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 15 Jun 2021 21:43:17 +0300
Reported-by: Tom Lane
Discussion: https://postgr.es/m/CAPpHfdvcnw3x7jdV3r52p4%3D5S4WUxBCzcQKB3JukQHoicv1LSQ%40mail.gmail.com
M doc/src/sgml/func.sgml
M doc/src/sgml/rangetypes.sgml
M src/backend/commands/typecmds.c
M src/backend/utils/adt/multirangetypes.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_cast.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/multirangetypes.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/sql/multirangetypes.sql
M src/test/regress/sql/opr_sanity.sql
Remove unneeded field from VACUUM state.
commit : 958cfbcf2dd338e3179c2d8a35f48bde020eba60
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 15 Jun 2021 08:59:36 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 15 Jun 2021 08:59:36 -0700
Bugfix commit 5fc89376 effectively made the lock_waiter_detected field
from vacuumlazy.c's global state struct into private state owned by
lazy_truncate_heap(). Finish this off by replacing the struct field
with a local variable.
M src/backend/access/heap/vacuumlazy.c
Add missing type name "multirange" in docs chapter title
commit : ad2da246c69dd5ec55764d4b6066f3b0c0d24ca2
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 15 Jun 2021 16:06:32 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 15 Jun 2021 16:06:32 +0300
Discussion: https://postgr.es/m/CAFj8pRDioOxiJgmgw9TqQqZ3CxnJC4P5B2Oospf2eMgAjJuewA%40mail.gmail.com
Author: Pavel Stehule, Alexander Korotkov
Reviewed-by: Justin Pryzby, Tom Lane
M doc/src/sgml/func.sgml
Support for unnest(multirange) and cast multirange as an array of ranges
commit : 29854ee8d1ca4a46adb7e84deb17e6fb18e531cc
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 15 Jun 2021 15:59:20 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 15 Jun 2021 15:59:20 +0300
It has been spotted that multiranges lack of ability to decompose them into
individual ranges. Subscription and proper expanded object representation
require substantial work, and it's too late for v14. This commit
provides the implementation of unnest(multirange) and cast multirange as
an array of ranges, which is quite trivial.
unnest(multirange) is defined as a polymorphic procedure. The catalog
description of the cast underlying procedure is duplicated for each multirange
type because we don't have anyrangearray polymorphic type to use here.
Catversion is bumped.
Reported-by: Jonathan S. Katz
Discussion: https://postgr.es/m/flat/60258efe-bd7e-4886-82e1-196e0cac5433%40postgresql.org
Author: Alexander Korotkov
Reviewed-by: Justin Pryzby, Jonathan S. Katz, Zhihong Yu
M doc/src/sgml/func.sgml
M doc/src/sgml/rangetypes.sgml
M src/backend/commands/typecmds.c
M src/backend/utils/adt/multirangetypes.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_cast.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/multirangetypes.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/sql/multirangetypes.sql
M src/test/regress/sql/opr_sanity.sql
Fix decoding of speculative aborts.
commit : 4daa140a2f50e9a160bc180c3997ee13c7aabf9e
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 15 Jun 2021 08:28:36 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 15 Jun 2021 08:28:36 +0530
During decoding for speculative inserts, we were relying for cleaning
toast hash on confirmation records or next change records. But that
could lead to multiple problems (a) memory leak if there is neither a
confirmation record nor any other record after toast insertion for a
speculative insert in the transaction, (b) error and assertion failures
if the next operation is not an insert/update on the same table.
The fix is to start queuing spec abort change and clean up toast hash
and change record during its processing. Currently, we are queuing the
spec aborts for both toast and main table even though we perform cleanup
while processing the main table's spec abort record. Later, if we have a
way to distinguish between the spec abort record of toast and the main
table, we can avoid queuing the change for spec aborts of toast tables.
Reported-by: Ashutosh Bapat
Author: Dilip Kumar
Reviewed-by: Amit Kapila
Backpatch-through: 9.6, where it was introduced
Discussion: https://postgr.es/m/CAExHW5sPKF-Oovx_qZe4p5oM6Dvof7_P+XgsNAViug15Fm99jA@mail.gmail.com
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h
Update variant expected-result file.
commit : 0a1e80c5c4f094087257fc4284a87e0bc7bca591
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Jun 2021 21:58:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Jun 2021 21:58:26 -0400
This should have been updated in d2d8a229b, but it was overlooked.
According to 31a877f18 which added it, this file is meant to show the
results you get under default_transaction_isolation = serializable.
We've largely lost track of that goal in other isolation tests, but
as long as we've got this one, it should be right.
Noted while fooling about with the isolationtester.
M src/test/isolation/expected/drop-index-concurrently-1_2.out
Remove orphaned expected-result file.
commit : ffbe9dec13599fa786ea6567df1c6a3f3ee3c673
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Jun 2021 21:28:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Jun 2021 21:28:21 -0400
This should have been removed in 43e084197, which removed the
corresponding spec file. Noted while fooling about with the
isolationtester.
D src/test/isolation/expected/insert-conflict-toast_1.out
Remove pg_wait_for_backend_termination().
commit : 5f1df62a459b51ab3bb625a0ee5e655429254257
author : Noah Misch <noah@leadboat.com>
date : Mon, 14 Jun 2021 17:29:37 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 14 Jun 2021 17:29:37 -0700
It was unable to wait on a backend that had already left the procarray.
Users tolerant of that limitation can poll pg_stat_activity. Other
users can employ the "timeout" argument of pg_terminate_backend().
Reviewed by Bharath Rupireddy.
Discussion: https://postgr.es/m/20210605013236.GA208701@rfd.leadboat.com
M doc/src/sgml/func.sgml
M doc/src/sgml/release-14.sgml
M src/backend/catalog/system_functions.sql
M src/backend/storage/ipc/signalfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Copy-edit text for the pg_terminate_backend() "timeout" parameter.
commit : 0aac73e6a2602696d23aa7a9686204965f9093dc
author : Noah Misch <noah@leadboat.com>
date : Mon, 14 Jun 2021 17:29:37 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 14 Jun 2021 17:29:37 -0700
Revert the pg_description entry to its v13 form, since those messages
usually remain shorter and don't discuss individual parameters. No
catversion bump, since pg_description content does not impair backend
compatibility or application compatibility.
Justin Pryzby
Discussion: https://postgr.es/m/20210612182743.GY16435@telsasoft.com
M doc/src/sgml/func.sgml
M src/backend/storage/ipc/signalfuncs.c
M src/include/catalog/pg_proc.dat
Fix logic bug in 1632ea43682f
commit : 33c509956704e1d918077b51ccd954f2e201a8f5
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 14 Jun 2021 16:31:12 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 14 Jun 2021 16:31:12 -0400
I overlooked that one condition was logically inverted. The fix is a
little bit more involved than simply negating the condition, to make
the code easier to read.
Fix some outdated comments left by the same commit, while at it.
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/YMRlmB3/lZw8YBH+@paquier.xyz
M src/backend/replication/slot.c
doc: PG 14 relnotes fixes
commit : 86b222b09071d3918c5c55b164d22b2236d3f872
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 14 Jun 2021 16:14:04 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 14 Jun 2021 16:14:04 -0400
Items related to logical replication attribution and BRIN indexes
Reported-by: Tomas Vondra, John Naylor
Discussion: https://postgr.es/m/0db66294-a668-2caa-2b5e-a8db60b30662@enterprisedb.com, CAFBsxsH21KnteYdk33F1oZu2O726NSD6_XBq51Tn0jytsA1AnA@mail.gmail.com
M doc/src/sgml/release-14.sgml
doc: PG 14 relnote updates
commit : a2559d4093725592a3fd5da8a4c7ac7c883115a0
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 14 Jun 2021 16:03:18 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 14 Jun 2021 16:03:18 -0400
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210612034551.GU16435@telsasoft.com
M doc/src/sgml/release-14.sgml
doc: add PG 14 relnote item about array function references
commit : 25dfb5a831a1b8a83a8a68453b4bbe38a5ef737e
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 14 Jun 2021 12:49:05 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 14 Jun 2021 12:49:05 -0400
User-defined objects that reference some built-in array functions will
need to be recreated in PG 14.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210608225618.GR16435@telsasoft.com
M doc/src/sgml/release-14.sgml
Improve handling of dropped objects in pg_event_trigger_ddl_commands()
commit : 2d689babe3cb50dcb29f6ed595a61d56e518c0d8
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 14 Jun 2021 14:57:22 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 14 Jun 2021 14:57:22 +0900
An object found as dropped when digging into the list of objects
returned by pg_event_trigger_ddl_commands() could cause a cache lookup
error, as the calls grabbing for the object address and the type name
would fail if the object was missing.
Those lookup errors could be seen with combinations of ALTER TABLE
sub-commands involving identity columns. The lookup logic is changed in
this code path to get a behavior similar to any other SQL-callable
function by ignoring objects that are not found, taking advantage of
2a10fdc. The back-branches are not changed, as they require this commit
that is too invasive for stable branches.
While on it, add test cases to exercise event triggers with identity
columns, and stress more cases with the event ddl_command_end for
relations.
Author: Sven Klemm, Aleksander Alekseev, Michael Paquier
Discussion: https://postgr.es/m/CAMCrgp2R1cEXU53iYKtW6yVEp2_yKUz+z=3-CTrYpPP+xryRtg@mail.gmail.com
M src/backend/commands/event_trigger.c
M src/test/regress/expected/event_trigger.out
M src/test/regress/sql/event_trigger.sql
Remove forced toast recompression in VACUUM FULL/CLUSTER
commit : dbab0c07e5ba1f19a991da2d72972a8fe9a41bda
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 14 Jun 2021 09:25:50 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 14 Jun 2021 09:25:50 +0900
The extra checks added by the recompression of toast data introduced in
bbe0a81 is proving to have a performance impact on VACUUM or CLUSTER
even if no recompression is done. This is more noticeable with more
toastable columns that contain non-NULL values.
Improvements could be done to make those extra checks less expensive,
but that's not material for 14 at this stage, and we are not sure either
if the code path of VACUUM FULL/CLUSTER is adapted for this job.
Per discussion with several people, including Andres Freund, Robert
Haas, Álvaro Herrera, Tom Lane and myself.
Discussion: https://postgr.es/m/20210527003144.xxqppojoiwurc2iz@alap3.anarazel.de
M doc/src/sgml/ref/alter_table.sgml
M src/backend/access/heap/heapam_handler.c
M src/test/regress/expected/compression.out
M src/test/regress/expected/compression_1.out
M src/test/regress/sql/compression.sql
Work around portability issue with newer versions of mktime().
commit : f807e3410fdfc29ced6590c7c2afa76637e001ad
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 13 Jun 2021 14:32:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 13 Jun 2021 14:32:42 -0400
Recent glibc versions have made mktime() fail if tm_isdst is
inconsistent with the prevailing timezone; in particular it fails for
tm_isdst = 1 when the zone is UTC. (This seems wildly inconsistent
with the POSIX-mandated treatment of "incorrect" values for the other
fields of struct tm, so if you ask me it's a bug, but I bet they'll
say it's intentional.) This has been observed to cause cosmetic
problems when pg_restore'ing an archive created in a different
timezone.
To fix, do mktime() using the field values from the archive, and if
that fails try again with tm_isdst = -1. This will give a result
that's off by the UTC-offset difference from the original zone, but
that was true before, too. It's not terribly critical since we don't
do anything with the result except possibly print it. (Someday we
should flush this entire bit of logic and record a standard-format
timestamp in the archive instead. That's not okay for a back-patched
bug fix, though.)
Also, guard our only other use of mktime() by having initdb's
build_time_t() set tm_isdst = -1 not 0. This case could only have
an issue in zones that are DST year-round; but I think some do exist,
or could in future.
Per report from Wells Oliver. Back-patch to all supported
versions, since any of them might need to run with a newer glibc.
Discussion: https://postgr.es/m/CAOC+FBWDhDHO7G-i1_n_hjRzCnUeFO+H-Czi1y10mFhRWpBrew@mail.gmail.com
M src/bin/initdb/findtimezone.c
M src/bin/pg_dump/pg_backup_archiver.c
Further tweaks to stuck_on_old_timeline recovery test
commit : 9d97c3408319b43718e4b85bc694697db1af32c6
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 13 Jun 2021 07:10:41 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 13 Jun 2021 07:10:41 -0400
Translate path slashes on target directory path. This was confusing old
branches, but is applied to all branches for the sake of uniformity.
Perl is perfectly able to understand paths with forward slashes.
Along the way, restore the previous archive_wait query, for the sake of
uniformity with other tests, per gripe from Tom Lane.
M src/test/recovery/t/025_stuck_on_old_timeline.pl
Ignore more environment variables in pg_regress.c
commit : a9e0b3b08fe38d5e31f03ea96859ff5e413d4a38
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 13 Jun 2021 20:07:39 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 13 Jun 2021 20:07:39 +0900
This is similar to the work done in 8279f68 for TestLib.pm, where
environment variables set may cause unwanted failures if using a
temporary installation with pg_regress. The list of variables reset is
adjusted in each stable branch depending on what is supported.
Comments are added to remember that the lists in TestLib.pm and
pg_regress.c had better be kept in sync.
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/YMNR9GYDn+fHlMta@paquier.xyz
Backpatch-through: 9.6
M src/test/perl/TestLib.pm
M src/test/regress/pg_regress.c
Restore robustness of TAP tests that wait for postmaster restart.
commit : f452aaf7d4a96cfcecd6c60ccd294ffe7b8ea088
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Jun 2021 15:12:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Jun 2021 15:12:10 -0400
Several TAP tests use poll_query_until() to wait for the postmaster
to restart. They were checking to see if a trivial query
(e.g. "SELECT 1") succeeds. However, that's problematic in the wake
of commit 11e9caff8, because now that we feed said query to psql
via stdin, we risk IPC::Run whining about a SIGPIPE failure if psql
quits before reading the query. Hence, we can't use a nonempty
query in cases where we need to wait for connection failures to
stop happening.
Per the precedent of commits c757a3da0 and 6d41dd045, we can pass
"undef" as the query in such cases to ensure that IPC::Run has
nothing to write. However, then we have to say that the expected
output is empty, and this exposes a deficiency in poll_query_until:
if psql fails altogether and returns empty stdout, poll_query_until
will treat that as a success! That's because, contrary to its
documentation, it makes no actual check for psql failure, looking
neither at the exit status nor at stderr.
To fix that, adjust poll_query_until to insist on empty stderr as
well as a stdout match. (I experimented with checking exit status
instead, but it seems that psql often does exit(1) in cases that we
need to consider successes. That might be something to fix someday,
but it would be a non-back-patchable behavior change.)
Back-patch to v10. The test cases needing this exist only as far
back as v11, but it seems wise to keep poll_query_until's behavior
the same in v10, in case we back-patch another such test case in
future. (9.6 does not currently need this change, because in that
branch poll_query_until can't be told to accept empty stdout as
a success case.)
Per assorted buildfarm failures, mostly on hoverfly.
Discussion: https://postgr.es/m/CAA4eK1+zM6L4QSA1XMvXY_qqWwdUmqkOS1+hWvL8QcYEBGA1Uw@mail.gmail.com
M src/test/perl/PostgresNode.pm
M src/test/recovery/t/013_crash_restart.pl
M src/test/recovery/t/022_crash_temp_files.pl
Ensure pg_filenode_relation(0, 0) returns NULL.
commit : 1250aad42520fd5a3db68d6381997b7e1f9bb4aa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Jun 2021 13:29:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Jun 2021 13:29:24 -0400
Previously, a zero value for the relfilenode resulted in
a confusing error message about "unexpected duplicate".
This function returns NULL for other invalid relfilenode
values, so zero should be treated likewise.
It's been like this all along, so back-patch to all supported
branches.
Justin Pryzby
Discussion: https://postgr.es/m/20210612023324.GT16435@telsasoft.com
M src/backend/utils/adt/dbsize.c
Don't use Asserts to check for violations of replication protocol.
commit : fe6a20ce54cbbb6fcfe9f6675d563af836ae799a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Jun 2021 12:59:15 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Jun 2021 12:59:15 -0400
Using an Assert to check the validity of incoming messages is an
extremely poor decision. In a debug build, it should not be that easy
for a broken or malicious remote client to crash the logrep worker.
The consequences could be even worse in non-debug builds, which will
fail to make such checks at all, leading to who-knows-what misbehavior.
Hence, promote every Assert that could possibly be triggered by wrong
or out-of-order replication messages to a full test-and-ereport.
To avoid bloating the set of messages the translation team has to cope
with, establish a policy that replication protocol violation error
reports don't need to be translated. Hence, all the new messages here
use errmsg_internal(). A couple of old messages are changed likewise
for consistency.
Along the way, fix some non-idiomatic or outright wrong uses of
hash_search().
Most of these mistakes are new with the "streaming replication"
patch (commit 464824323), but a couple go back a long way.
Back-patch as appropriate.
Discussion: https://postgr.es/m/1719083.1623351052@sss.pgh.pa.us
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/worker.c
Fix new recovery test for use under msys
commit : c3652f976b7696a96a9c5606cc2d613af77e2e63
author : Andrew Dunstan <andrew@dunslane.net>
date : Sat, 12 Jun 2021 08:37:16 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sat, 12 Jun 2021 08:37:16 -0400
Commit caba8f0d43 wasn't quite right for msys, as demonstrated by
several buildfarm animals, including jacana and fairywren. We need to
use the msys perl in the archive command, but call it in such a way that
Windows will understand the path. Furthermore, inside the copy script we
need to convert a Windows path to an msys path.
M src/test/recovery/t/025_stuck_on_old_timeline.pl
M src/test/recovery/t/cp_history_files
Simplify some code in getObjectTypeDescription()
commit : b56b83aa0d6e044cf38d553f7a87f4b84708cac6
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 12 Jun 2021 16:29:11 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 12 Jun 2021 16:29:11 +0900
This routine is designed to never return an empty description or NULL,
providing description fallbacks even if missing objects are accepted,
but it included a code path where this was considered possible. All the
callers of this routine already consider NULL as not possible, so
change a bit the code to map with the assumptions of the callers, and
add more comments close to the callers of this routine to outline the
behavior expected.
This code is new as of 2a10fdc, so no backpatch is needed.
Discussion: https://postgr.es/m/YMNY6RGPBRCeLmFb@paquier.xyz
M src/backend/catalog/objectaddress.c
Improve log pattern detection in recently-added TAP tests
commit : bfd96b7a3dc26a8384d4185d274573fb6a46b033
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 12 Jun 2021 15:29:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 12 Jun 2021 15:29:48 +0900
ab55d74 has introduced some tests with rows found as missing in logical
replication subscriptions for partitioned tables, relying on a logic
with a lookup of the logs generated, scanning the whole file. This
commit makes the logic more precise, by scanning the logs only from the
position before the key queries are run to the position where we check
for the logs. This will reduce the risk of issues with log patterns
overlapping with each other if those tests get more complicated in the
future.
Per discussion with Tom Lane.
Discussion: https://postgr.es/m/YMP+Gx2S8meYYHW4@paquier.xyz
Backpatch-through: 13
M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/013_partition.pl
Improve and cleanup ProcArrayAdd(), ProcArrayRemove().
commit : d8e950d3ae7b33a2064a4fb39b7829334b0b47bc
author : Andres Freund <andres@anarazel.de>
date : Fri, 11 Jun 2021 21:22:21 -0700
committer: Andres Freund <andres@anarazel.de>
date : Fri, 11 Jun 2021 21:22:21 -0700
941697c3c1a changed ProcArrayAdd()/Remove() substantially. As reported by
zhanyi, I missed that due to the introduction of the PGPROC->pgxactoff
ProcArrayRemove() does not need to search for the position in
procArray->pgprocnos anymore - that's pgxactoff. Remove the search loop.
The removal of the search loop reduces assertion coverage a bit - but we can
easily do better than before by adding assertions to other loops over
pgprocnos elements.
Also do a bit of cleanup, mostly by reducing line lengths by introducing local
helper variables and adding newlines.
Author: zhanyi <w@hidva.com>
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/tencent_5624AA3B116B3D1C31CA9744@qq.com
M src/backend/storage/ipc/procarray.c
doc: remove extra right angle bracket in PG 14 relnotes
commit : d120e66fac87f768ea2289d2d923d0ee4262ec0f
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 11 Jun 2021 21:01:52 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 11 Jun 2021 21:01:52 -0400
Reported-by: Justin Pryzby
M doc/src/sgml/release-14.sgml
docs: fix incorrect indenting in PG 14 relnotes
commit : 0725913982e5e06895a32a9eeea2c59a13978927
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 11 Jun 2021 19:51:35 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 11 Jun 2021 19:51:35 -0400
M doc/src/sgml/release-14.sgml
Report sort phase progress in parallel btree build
commit : 5cc1cd502879d642da799e1fd12619d83d987369
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Jun 2021 19:07:32 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Jun 2021 19:07:32 -0400
We were already reporting it, but only after the parallel workers were
finished, which is visibly much later than what happens in a serial
build.
With this change we report it when the leader starts its own sort phase
when participating in the build (the normal case). Now this might
happen a little later than when the workers start their sorting phases,
but a) communicating the actual phase start from workers is likely to be
a hassle, and b) the sort phase start is pretty fuzzy anyway, since
sorting per se is actually initiated by tuplesort.c internally earlier
than tuplesort_performsort() is called.
Backpatch to pg12, where the progress reporting code for CREATE INDEX
went in.
Reported-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-by: Greg Nancarrow <gregn4422@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/1128176d-1eee-55d4-37ca-e63644422adb
M src/backend/access/nbtree/nbtsort.c
Fix multiple crasher bugs in partitioned-table replication logic.
commit : ab55d742eb7162c22ee60f1e15e07d2a60063c4e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 11 Jun 2021 16:12:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 11 Jun 2021 16:12:36 -0400
apply_handle_tuple_routing(), having detected and reported that
the tuple it needed to update didn't exist, tried to update that
tuple anyway, leading to a null-pointer dereference.
logicalrep_partition_open() failed to ensure that the
LogicalRepPartMapEntry it built for a partition was fully
independent of that for the partition root, leading to
trouble if the root entry was later freed or rebuilt.
Meanwhile, on the publisher's side, pgoutput_change() sometimes
attempted to apply execute_attr_map_tuple() to a NULL tuple.
The first of these was reported by Sergey Bernikov in bug #17055;
I found the other two while developing some test cases for this
sadly under-tested code.
Diagnosis and patch for the first issue by Amit Langote; patches
for the others by me; new test cases by me. Back-patch to v13
where this logic came in.
Discussion: https://postgr.es/m/17055-9ba800ec8522668b@postgresql.org
M src/backend/replication/logical/relation.c
M src/backend/replication/logical/worker.c
M src/backend/replication/pgoutput/pgoutput.c
M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/013_partition.pl
Add 'Portal Close' message to pipelined PQsendQuery()
commit : 4efcf47053eaf8dd88de2b1a89478df43d37d5c0
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Jun 2021 16:05:50 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Jun 2021 16:05:50 -0400
Commit acb7e4eb6b1c added a new implementation for PQsendQuery so that
it works in pipeline mode (by using extended query protocol), but it
behaves differently from the 'Q' message (in simple query protocol) used
by regular implementation: the new one doesn't close the unnamed portal.
Change the new code to have identical behavior to the old.
Reported-by: Yura Sokolov <y.sokolov@postgrespro.ru>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/202106072107.d4i55hdscxqj@alvherre.pgsql
M src/interfaces/libpq/fe-exec.c
M src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
Return ReplicationSlotAcquire API to its original form
commit : 1632ea43682fcea8836ea245771ae85b9e1bcd38
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Jun 2021 15:48:26 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Jun 2021 15:48:26 -0400
Per 96540f80f833; the awkward API introduced by c6550776394e is no
longer needed.
Author: Andres Freund <andres@anarazel.de>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20210408020913.zzprrlvqyvlt5cyy@alap3.anarazel.de
M src/backend/replication/logical/logicalfuncs.c
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/walsender.c
M src/include/replication/slot.h
Optimize creation of slots for FDW bulk inserts
commit : b676ac443b6a83558d4701b2dd9491c0b37e17c4
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 11 Jun 2021 20:19:48 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 11 Jun 2021 20:19:48 +0200
Commit b663a41363 introduced bulk inserts for FDW, but the handling of
tuple slots turned out to be problematic for two reasons. Firstly, the
slots were re-created for each individual batch. Secondly, all slots
referenced the same tuple descriptor - with reasonably small batches
this is not an issue, but with large batches this triggers O(N^2)
behavior in the resource owner code.
These two issues work against each other - to reduce the number of times
a slot has to be created/dropped, larger batches are needed. However,
the larger the batch, the more expensive the resource owner gets. For
practical batch sizes (100 - 1000) this would not be a big problem, as
the benefits (latency savings) greatly exceed the resource owner costs.
But for extremely large batches it might be much worse, possibly even
losing with non-batching mode.
Fixed by initializing tuple slots only once (and reusing them across
batches) and by using a new tuple descriptor copy for each slot.
Discussion: https://postgr.es/m/ebbbcc7d-4286-8c28-0272-61b4753af761%40enterprisedb.com
M src/backend/executor/nodeModifyTable.c
M src/include/nodes/execnodes.h
Fix race condition in invalidating obsolete replication slots
commit : 96540f80f8334a3f0f4a13f0d42e4565d8fa9eb7
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Jun 2021 12:16:14 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Jun 2021 12:16:14 -0400
The code added to mark replication slots invalid in commit c6550776394e
had the race condition that a slot can be dropped or advanced
concurrently with checkpointer trying to invalidate it. Rewrite the
code to close those races.
The changes to ReplicationSlotAcquire's API added with c6550776394e are
not necessary anymore. To avoid an ABI break in released branches, this
commit leaves that unchanged; it'll be changed in a master-only commit
separately.
Backpatch to 13, where this code first appeared.
Reported-by: Andres Freund <andres@anarazel.de>
Author: Andres Freund <andres@anarazel.de>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20210408001037.wfmk6jud36auhfqm@alap3.anarazel.de
M src/backend/replication/slot.c
Improve psql tab completion for options of subcriptions and publications
commit : d08237b5b494f96e72220bcef36a14a642969f16
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 11 Jun 2021 15:46:18 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 11 Jun 2021 15:46:18 +0900
The list of options provided by the tab completion was outdated for the
following commands:
- ALTER SUBSCRIPTION
- CREATE SUBSCRIPTION
- ALTER PUBLICATION
- CREATE PUBLICATION
Author: Vignesh C
Reviewed-by: Bharath Rupireddy
Discussion: https://postgr.es/m/CALDaNm18oHDFu6SFCHE=ZbiO153Fx7E-L1MG0YyScbaDV--U+A@mail.gmail.com
M src/bin/psql/tab-complete.c
Change position of field "transformed" in struct CreateStatsStmt.
commit : 13a1ca160dcfc316c9f4005891a312f5a84c5ca2
author : Noah Misch <noah@leadboat.com>
date : Thu, 10 Jun 2021 21:56:14 -0700
committer: Noah Misch <noah@leadboat.com>
date : Thu, 10 Jun 2021 21:56:14 -0700
Resolve the disagreement with nodes/*funcs.c field order in favor of the
latter, which is better-aligned with the IndexStmt field order. This
field is new in v14.
Discussion: https://postgr.es/m/20210611045546.GA573364@rfd.leadboat.com
M src/backend/parser/parse_utilcmd.c
M src/include/nodes/parsenodes.h
Rename PQtraceSetFlags() to PQsetTraceFlags().
commit : d0e750c0acaf31f60667b1635311bcef5ab38bbe
author : Noah Misch <noah@leadboat.com>
date : Thu, 10 Jun 2021 21:56:13 -0700
committer: Noah Misch <noah@leadboat.com>
date : Thu, 10 Jun 2021 21:56:13 -0700
We have a dozen PQset*() functions. PQresultSetInstanceData() and this
were the libpq setter functions having a different word order. Adopt
the majority word order.
Reviewed by Alvaro Herrera and Robert Haas, though this choice of name
was not unanimous.
Discussion: https://postgr.es/m/20210605060555.GA216695@rfd.leadboat.com
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/exports.txt
M src/interfaces/libpq/fe-trace.c
M src/interfaces/libpq/libpq-fe.h
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Use the correct article for abbreviations
commit : 04539e73faaaaa1c06c1407671910dceaffdfcd4
author : David Rowley <drowley@postgresql.org>
date : Fri, 11 Jun 2021 13:38:04 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 11 Jun 2021 13:38:04 +1200
We've accumulated quite a mix of instances of "an SQL" and "a SQL" in the
documents. It would be good to be a bit more consistent with these.
The most recent version of the SQL standard I looked at seems to prefer
"an SQL". That seems like a good lead to follow, so here we change all
instances of "a SQL" to become "an SQL". Most instances correctly use
"an SQL" already, so it also makes sense to use the dominant variation in
order to minimise churn.
Additionally, there were some other abbreviations that needed to be
adjusted. FSM, SSPI, SRF and a few others. Also fix some pronounceable,
abbreviations to use "a" instead of "an". For example, "a SASL" instead
of "an SASL".
Here I've only adjusted the documents and error messages. Many others
still exist in source code comments. Translator hint comments seem to be
the biggest culprit. It currently does not seem worth the churn to change
these.
Discussion: https://postgr.es/m/CAApHDvpML27UqFXnrYO1MJddsKVMQoiZisPvsAGhKE_tsKXquw%40mail.gmail.com
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/dblink.sgml
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/ltree.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/pageinspect.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/plpython.sgml
M doc/src/sgml/pltcl.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/alter_opfamily.sgml
M doc/src/sgml/ref/create_function.sgml
M doc/src/sgml/ref/create_opclass.sgml
M doc/src/sgml/ref/create_statistics.sgml
M doc/src/sgml/ref/create_trigger.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pgbench.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/tablefunc.sgml
M doc/src/sgml/tablesample-method.sgml
M doc/src/sgml/textsearch.sgml
M doc/src/sgml/trigger.sgml
M doc/src/sgml/typeconv.sgml
M doc/src/sgml/xfunc.sgml
M src/backend/executor/functions.c
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
Reconsider the handling of procedure OUT parameters.
commit : e56bce5d43789cce95d099554ae9593ada92b3b7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Jun 2021 17:11:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Jun 2021 17:11:36 -0400
Commit 2453ea142 redefined pg_proc.proargtypes to include the types of
OUT parameters, for procedures only. While that had some advantages
for implementing the SQL-spec behavior of DROP PROCEDURE, it was pretty
disastrous from a number of other perspectives. Notably, since the
primary key of pg_proc is name + proargtypes, this made it possible to
have multiple procedures with identical names + input arguments and
differing output argument types. That would make it impossible to call
any one of the procedures by writing just NULL (or "?", or any other
data-type-free notation) for the output argument(s). The change also
seems likely to cause grave confusion for client applications that
examine pg_proc and expect the traditional definition of proargtypes.
Hence, revert the definition of proargtypes to what it was, and
undo a number of complications that had been added to support that.
To support the SQL-spec behavior of DROP PROCEDURE, when there are
no argmode markers in the command's parameter list, we perform the
lookup both ways (that is, matching against both proargtypes and
proallargtypes), succeeding if we get just one unique match.
In principle this could result in ambiguous-function failures
that would not happen when using only one of the two rules.
However, overloading of procedure names is thought to be a pretty
rare usage, so this shouldn't cause many problems in practice.
Postgres-specific code such as pg_dump can defend against any
possibility of such failures by being careful to specify argmodes
for all procedure arguments.
This also fixes a few other bugs in the area of CALL statements
with named parameters, and improves the documentation a little.
catversion bump forced because the representation of procedures
with OUT arguments changes.
Discussion: https://postgr.es/m/3742981.1621533210@sss.pgh.pa.us
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/ref/alter_extension.sgml
M doc/src/sgml/ref/alter_procedure.sgml
M doc/src/sgml/ref/call.sgml
M doc/src/sgml/ref/comment.sgml
M doc/src/sgml/ref/drop_procedure.sgml
M doc/src/sgml/ref/drop_routine.sgml
M doc/src/sgml/ref/security_label.sgml
M doc/src/sgml/xfunc.sgml
M src/backend/catalog/namespace.c
M src/backend/catalog/pg_aggregate.c
M src/backend/catalog/pg_proc.c
M src/backend/commands/functioncmds.c
M src/backend/executor/functions.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/parser/parse_func.c
M src/backend/parser/parse_oper.c
M src/backend/utils/adt/regproc.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/fmgr/funcapi.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/include/catalog/catversion.h
M src/include/catalog/namespace.h
M src/include/catalog/pg_proc.h
M src/include/funcapi.h
M src/include/nodes/parsenodes.h
M src/include/optimizer/optimizer.h
M src/include/parser/parse_func.h
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_call.sql
M src/pl/plpython/expected/plpython_call.out
M src/pl/plpython/plpy_procedure.c
M src/pl/plpython/sql/plpython_call.sql
M src/pl/tcl/expected/pltcl_call.out
M src/pl/tcl/sql/pltcl_call.sql
M src/test/regress/expected/create_procedure.out
M src/test/regress/sql/create_procedure.sql
Rearrange logrep worker's snapshot handling some more.
commit : 3a09d75b4f6cabc8331e228b6988dbfcd9afdfbe
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Jun 2021 12:27:27 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Jun 2021 12:27:27 -0400
It turns out that worker.c's code path for TRUNCATE was also
careless about establishing a snapshot while executing user-defined
code, allowing the checks added by commit 84f5c2908 to fail when
a trigger is fired in that context.
We could just wrap Push/PopActiveSnapshot around the truncate call,
but it seems better to establish a policy of holding a snapshot
throughout execution of a replication step. To help with that and
possible future requirements, replace the previous ensure_transaction
calls with pairs of begin/end_replication_step calls.
Per report from Mark Dilger. Back-patch to v11, like the previous
changes.
Discussion: https://postgr.es/m/B4A3AF82-79ED-4F4C-A4E5-CD2622098972@enterprisedb.com
M src/backend/replication/logical/worker.c
Shut down EvalPlanQual machinery when LockRows node reaches the end.
commit : bb4aed46a5aeb00d2f1d8b8160feed339f4eaf12
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Jun 2021 11:15:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Jun 2021 11:15:13 -0400
Previously, we left the EPQ sub-executor alone until ExecEndLockRows.
This caused any buffer pins or other resources that it might hold to
remain held until ExecutorEnd, which in some code paths means that
they are held till the Portal is closed. That can cause user-visible
problems, such as blocking VACUUM; and it's unlike the behavior of
ordinary table-scanning nodes, which will have released all buffer
pins by the time they return an EOF indication.
We can make LockRows work more like other plan nodes by calling
EvalPlanQualEnd just before returning NULL. We still need to call it
in ExecEndLockRows in case the node was not run to completion, but in
the normal case the second call does nothing and costs little.
Per report from Yura Sokolov. In principle this is a longstanding
bug, but in view of the lack of other complaints and the low severity
of the consequences, I chose not to back-patch.
Discussion: https://postgr.es/m/4aa370cb91ecf2f9885d98b80ad1109c@postgrespro.ru
M src/backend/executor/nodeLockRows.c
Avoid ECPG test failures in some GSS-capable environments.
commit : 9bb5eecce645dd72853e3ed262bef7bf11cae566
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Jun 2021 10:45:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Jun 2021 10:45:31 -0400
Buildfarm member hamerkop has been reporting that two cases in
connect/test5.pgc show different error messages than the test expects,
because since commit ffa2e4670 libpq's connection failure messages
are exposing the fact that a GSS-encrypted connection was attempted
and failed. That's pretty interesting information in itself, and
I certainly don't wish to shoot the messenger, but we need to do
something to stabilize the ECPG results.
For the second of these two failure cases, we can add the
gssencmode=disable option to prevent the discrepancy. However,
that solution is problematic for the first failure, because the only
unique thing about that case is that it's testing a completely-omitted
connection target; there's noplace to add the option without defeating
the point of the test case. After some thrashing around with
alternative fixes that turned out to have undesirable side-effects,
the most workable answer is just to give up and remove that test case.
Perhaps we can revert this later, if we figure out why the GSS code
is misbehaving in hamerkop's environment.
Thanks to Michael Paquier for exploration of alternatives.
Discussion: https://postgr.es/m/YLRZH6CWs9N6Pusy@paquier.xyz
M src/interfaces/ecpg/test/connect/test5.pgc
M src/interfaces/ecpg/test/expected/connect-test5.c
M src/interfaces/ecpg/test/expected/connect-test5.stderr
Add some const decorations
commit : b29fa951ec519bdde153465e2caa6c0b7b3bcfc3
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 10 Jun 2021 16:21:48 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 10 Jun 2021 16:21:48 +0200
One of these functions is new in PostgreSQL 14; might as well start it
out right.
M src/backend/replication/logical/origin.c
M src/include/replication/origin.h
Adjust new test case to set wal_keep_size.
commit : 4dcb1d087aebc6fc2477ce4458ea82f548e2c1ee
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 10 Jun 2021 09:08:30 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 10 Jun 2021 09:08:30 -0400
Per buildfarm member conchuela and Kyotaro Horiguchi, it's possible
for the WAL segment that the cascading standby needs to be removed
too quickly. Hopefully this will prevent that.
Kyotaro Horiguchi
Discussion: http://postgr.es/m/20210610.101240.1270925505780628275.horikyota.ntt@gmail.com
M src/test/recovery/t/025_stuck_on_old_timeline.pl
Fix an asssortment of typos in brin_minmax_multi.c and mcv.c
commit : 55ba5973d9144a552661cf1fa4cbd228a3799212
author : David Rowley <drowley@postgresql.org>
date : Thu, 10 Jun 2021 20:13:44 +1200
committer: David Rowley <drowley@postgresql.org>
date : Thu, 10 Jun 2021 20:13:44 +1200
Discussion: https://postgr.es/m/CAApHDvrbyJNOPBws4RUhXghZ7+TBjtdO-rznTsqZECuowNorXg@mail.gmail.com
M src/backend/access/brin/brin_minmax_multi.c
M src/backend/statistics/mcv.c
Fix corner case failure of new standby to follow new primary.
commit : caba8f0d43fb679c6f9643456080408a6bc370e8
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 9 Jun 2021 16:17:00 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 9 Jun 2021 16:17:00 -0400
This only happens if (1) the new standby has no WAL available locally,
(2) the new standby is starting from the old timeline, (3) the promotion
happened in the WAL segment from which the new standby is starting,
(4) the timeline history file for the new timeline is available from
the archive but the WAL files for are not (i.e. this is a race),
(5) the WAL files for the new timeline are available via streaming,
and (6) recovery_target_timeline='latest'.
Commit ee994272ca50f70b53074f0febaec97e28f83c4e introduced this
logic and was an improvement over the previous code, but it mishandled
this case. If recovery_target_timeline='latest' and restore_command is
set, validateRecoveryParameters() can change recoveryTargetTLI to be
different from receiveTLI. If streaming is then tried afterward,
expectedTLEs gets initialized with the history of the wrong timeline.
It's supposed to be a list of entries explaining how to get to the
target timeline, but in this case it ends up with a list of entries
explaining how to get to the new standby's original timeline, which
isn't right.
Dilip Kumar and Robert Haas, reviewed by Kyotaro Horiguchi.
Discussion: http://postgr.es/m/CAFiTN-sE-jr=LB8jQuxeqikd-Ux+jHiXyh4YDiZMPedgQKup0g@mail.gmail.com
M src/backend/access/transam/xlog.c
A src/test/recovery/t/025_stuck_on_old_timeline.pl
A src/test/recovery/t/cp_history_files
Fix inconsistencies in psql --help=commands
commit : 845cad4d51cb12a34ea012dfe58af5ef490384fc
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 9 Jun 2021 16:24:52 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 9 Jun 2021 16:24:52 +0900
The set of subcommands supported by \dAp, \do and \dy was described
incorrectly in psql's --help. The documentation was already consistent
with the code.
Reported-by: inoas, from IRC
Author: Matthijs van der Vleuten
Reviewed-by: Neil Chen
Discussion: https://postgr.es/m/6a984e24-2171-4039-9050-92d55e7b23fe@www.fastmail.com
Backpatch-through: 9.6
M src/bin/psql/help.c
Force NO SCROLL for plpgsql's implicit cursors.
commit : be90098907475f3cfff7dc6d590641b9c2dae081
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Jun 2021 18:40:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Jun 2021 18:40:06 -0400
Further thought about bug #17050 suggests that it's a good idea
to use CURSOR_OPT_NO_SCROLL for the implicit cursor opened by
a plpgsql FOR-over-query loop. This ensures that, if somebody
commits inside the loop, PersistHoldablePortal won't try to
rewind and re-read the cursor. While we'd have selected NO_SCROLL
anyway if FOR UPDATE/SHARE appears in the query, there are other
hazards with volatile functions; and in any case, it's silly to
expend effort storing rows that we know for certain won't be needed.
(While here, improve the comment in exec_run_select, which was a bit
confused about the rationale for when we can use parallel mode.
Cursor operations aren't a hazard for nameless portals.)
This wasn't an issue until v11, which introduced the possibility
of persisting such cursors. Hence, back-patch to v11.
Per bug #17050 from Алексей Булгаков.
Discussion: https://postgr.es/m/17050-f77aa827dc85247c@postgresql.org
M src/pl/plpgsql/src/pl_exec.c
Avoid misbehavior when persisting a non-stable cursor.
commit : ba2c6d6cec000f0aeaeda4d56a23a335f6164860
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Jun 2021 17:50:15 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Jun 2021 17:50:15 -0400
PersistHoldablePortal has long assumed that it should store the
entire output of the query-to-be-persisted, which requires rewinding
and re-reading the output. This is problematic if the query is not
stable: we might get different row contents, or even a different
number of rows, which'd confuse the cursor state mightily.
In the case where the cursor is NO SCROLL, this is very easy to
solve: just store the remaining query output, without any rewinding,
and tweak the portal's cursor state to match. Aside from removing
the semantic problem, this could be significantly more efficient
than storing the whole output.
If the cursor is scrollable, there's not much we can do, but it
was already the case that scrolling a volatile query's result was
pretty unsafe. We can just document more clearly that getting
correct results from that is not guaranteed.
There are already prohibitions in place on using SCROLL with
FOR UPDATE/SHARE, which is one way for a SELECT query to have
non-stable results. We could imagine prohibiting SCROLL when
the query contains volatile functions, but that would be
expensive to enforce. Moreover, it could break applications
that work just fine, if they have functions that are in fact
stable but the user neglected to mark them so. So settle for
documenting the hazard.
While this problem has existed in some guise for a long time,
it got a lot worse in v11, which introduced the possibility
of persisting plpgsql cursors (perhaps implicit ones) even
when they violate the rules for what can be marked WITH HOLD.
Hence, I've chosen to back-patch to v11 but not further.
Per bug #17050 from Алексей Булгаков.
Discussion: https://postgr.es/m/17050-f77aa827dc85247c@postgresql.org
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/ref/declare.sgml
M src/backend/commands/portalcmds.c
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/sql/plpgsql_transaction.sql
doc: update release note item about the v2 wire protocol
commit : 444302ed56273e4c4006a9be319e60fa12a90346
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 8 Jun 2021 16:47:14 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 8 Jun 2021 16:47:14 -0400
Protocol v2 was last used in PG 7.3, not 7.2.
Reported-by: Tatsuo Ishii
Discussion: https://postgr.es/m/20210608.091329.906837606658882674.t-ishii@sraoss.co.jp
M doc/src/sgml/release-14.sgml
Adjust batch size in postgres_fdw to not use too many parameters
commit : cb92703384e2bb3fa0a690e5dbb95ad333c2b44c
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 8 Jun 2021 20:22:18 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 8 Jun 2021 20:22:18 +0200
The FE/BE protocol identifies parameters with an Int16 index, which
limits the maximum number of parameters per query to 65535. With
batching added to postges_fdw this limit is much easier to hit, as
the whole batch is essentially a single query, making this error much
easier to hit.
The failures are a bit unpredictable, because it also depends on the
number of columns in the query. So instead of just failing, this patch
tweaks the batch_size to not exceed the maximum number of parameters.
Reported-by: Hou Zhijie <houzj.fnst@cn.fujitsu.com>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Discussion: https://postgr.es/m/OS0PR01MB571603973C0AC2874AD6BF2594299%40OS0PR01MB5716.jpnprd01.prod.outlook.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/libpq-fe.h
Fix pg_visibility regression failure with CLOBBER_CACHE_ALWAYS
commit : d1f0aa7696917213485c03b076b573497a535076
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 8 Jun 2021 19:24:27 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 8 Jun 2021 19:24:27 +0200
Commit 8e03eb92e9 reverted a bit too much code, reintroducing one of the
issues fixed by 39b66a91bd - a page might have been left partially empty
after relcache invalidation.
Reported-By: Tom Lane
Author: Masahiko Sawada
Discussion: https://postgr.es/m/822752.1623032114@sss.pgh.pa.us
Discussion: https://postgr.es/m/CAD21AoA%3D%3Df2VSw3c-Cp_y%3DWLKHMKc1D6s7g3YWsCOvgaYPpJcg%40mail.gmail.com
M src/backend/access/heap/hio.c
Don't crash on empty statements in SQL-standard function bodies.
commit : bfeede9fa464ab707eb5ac5704742bf78bd7ac9e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Jun 2021 11:59:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Jun 2021 11:59:34 -0400
gram.y should discard NULL pointers (empty statements) when
assembling a routine_body_stmt_list, as it does for other
sorts of statement lists.
Julien Rouhaud and Tom Lane, per report from Noah Misch.
Discussion: https://postgr.es/m/20210606044418.GA297923@rfd.leadboat.com
M src/backend/parser/gram.y
M src/test/regress/expected/create_function_3.out
M src/test/regress/sql/create_function_3.sql
libpq: Fix SNI host handling
commit : 37e1cce4ddf0be362e3093cee55493aee41bc423
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 8 Jun 2021 15:37:54 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 8 Jun 2021 15:37:54 +0200
Fix handling of NULL host name (possibly by using hostaddr). It
previously crashed. Also, we should look at connhost, not pghost, to
handle multi-host specifications.
Also remove an unnecessary SSL_CTX_free().
Reported-by: Jacob Champion <pchampion@vmware.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/504c276ab6eee000bb23d571ea9b0ced4250774e.camel@vmware.com
M src/interfaces/libpq/fe-secure-openssl.c
Doc: Further update documentation for asynchronous execution.
commit : eab81953682d5087295afb911c93f36cb1533bd9
author : Etsuro Fujita <efujita@postgresql.org>
date : Tue, 8 Jun 2021 13:45:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Tue, 8 Jun 2021 13:45:00 +0900
Add a note about asynchronous execution by postgres_fdw when applied to
Append nodes that contain synchronous subplan(s) as well. Follow-up for
commit 27e1f1456.
Andrey Lepikhov and Etsuro Fujita
Discussion: https://postgr.es/m/58fa2aa5-07f5-80b5-59a1-fec8a349fee7%40postgrespro.ru
M doc/src/sgml/postgres-fdw.sgml
Reorder superuser check in pg_log_backend_memory_contexts()
commit : 4e47b02834827fa700627290fae02f89a450368c
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Jun 2021 08:53:12 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Jun 2021 08:53:12 +0900
The use of this function is limited to superusers and the code includes
a hardcoded check for that. However, the code would look for the PGPROC
entry to signal for the memory dump before checking if the user is a
superuser or not, which does not make sense if we know that an error
will be returned. Note that the code would let one know if a process
was a PostgreSQL process or not even for non-authorized users, which is
not the case now, but this avoids taking ProcArrayLock that will most
likely finish by being unnecessary.
Thanks to Julien Rouhaud and Tom Lane for the discussion.
Discussion: https://postgr.es/m/YLxw1uVGIAP5uMPl@paquier.xyz
M src/backend/utils/adt/mcxtfuncs.c
Add _outTidRangePath()
commit : 3bb309be7533e153d86390642e8a2d054bbe82c8
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 7 Jun 2021 21:32:53 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 7 Jun 2021 21:32:53 +0200
We have outNode() coverage for all path nodes, but this one was
missed when it was added.
M src/backend/nodes/outfuncs.c
Stabilize contrib/seg regression test.
commit : d16ebfbff74b30c83a4669a1df318cacfa7e52ca
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Jun 2021 14:52:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Jun 2021 14:52:42 -0400
If autovacuum comes along just after we fill table test_seg with
some data, it will update the stats to the point where we prefer
a plain indexscan over a bitmap scan, breaking the expected
output (as well as the point of the test case). To fix, just
force a bitmap scan to be chosen here.
This has evidently been wrong since commit de1d042f5. It's not
clear why we just recently saw any buildfarm failures due to it;
but prairiedog has failed twice on this test in the past week.
Hence, backpatch to v11 where this test case came in.
M contrib/seg/expected/seg.out
M contrib/seg/sql/seg.sql
Fix incautious handling of possibly-miscoded strings in client code.
commit : 42f94f56bf9559f0a3cf5f3ffde50655834694ee
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Jun 2021 14:15:25 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Jun 2021 14:15:25 -0400
An incorrectly-encoded multibyte character near the end of a string
could cause various processing loops to run past the string's
terminating NUL, with results ranging from no detectable issue to
a program crash, depending on what happens to be in the following
memory.
This isn't an issue in the server, because we take care to verify
the encoding of strings before doing any interesting processing
on them. However, that lack of care leaked into client-side code
which shouldn't assume that anyone has validated the encoding of
its input.
Although this is certainly a bug worth fixing, the PG security team
elected not to regard it as a security issue, primarily because
any untrusted text should be sanitized by PQescapeLiteral or
the like before being incorporated into a SQL or psql command.
(If an app fails to do so, the same technique can be used to
cause SQL injection, with probably much more dire consequences
than a mere client-program crash.) Those functions were already
made proof against this class of problem, cf CVE-2006-2313.
To fix, invent PQmblenBounded() which is like PQmblen() except it
won't return more than the number of bytes remaining in the string.
In HEAD we can make this a new libpq function, as PQmblen() is.
It seems imprudent to change libpq's API in stable branches though,
so in the back branches define PQmblenBounded as a macro in the files
that need it. (Note that just changing PQmblen's behavior would not
be a good idea; notably, it would completely break the escaping
functions' defense against this exact problem. So we just want a
version for those callers that don't have any better way of handling
this issue.)
Per private report from houjingyi. Back-patch to all supported branches.
M src/bin/psql/common.c
M src/bin/psql/psqlscanslash.l
M src/bin/psql/stringutils.c
M src/bin/psql/tab-complete.c
M src/bin/scripts/common.c
M src/common/jsonapi.c
M src/common/wchar.c
M src/fe_utils/print.c
M src/fe_utils/string_utils.c
M src/include/mb/pg_wchar.h
M src/interfaces/libpq/exports.txt
M src/interfaces/libpq/fe-misc.c
M src/interfaces/libpq/fe-print.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-fe.h
Fix portability issue in test indirect_toast
commit : 68a6d8a87006ba727d9662ec84c7a3d9de402df0
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Jun 2021 18:12:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Jun 2021 18:12:29 +0900
When run on a server using default_toast_compression set to LZ4, this
test would fail because of a consistency issue with the order of the
tuples treated. LZ4 causes one tuple to be stored inline instead of
getting externalized. As the goal of this test is to check after data
stored externally, stick to pglz as the compression algorithm used, so
as all data of this test is stored the way it should.
Analyzed-by: Dilip Kumar
Discussion: https://postgr.es/m/YLrDWxJgM8WWMoCg@paquier.xyz
M src/test/regress/expected/indirect_toast.out
M src/test/regress/sql/indirect_toast.sql
Remove two_phase variable from CreateReplicationSlotCmd struct.
commit : be57f21650d36449ec34a67b2d9af71126a663b3
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 7 Jun 2021 09:32:06 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 7 Jun 2021 09:32:06 +0530
Commit 19890a064e added the option to enable two_phase commits via
pg_create_logical_replication_slot but didn't extend the support of same
in replication protocol. However, by mistake, it added the two_phase
variable in CreateReplicationSlotCmd which is required only when we extend
the replication protocol.
Reported-by: Jeff Davis
Author: Ajin Cherian
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/64b9f783c6e125f18f88fbc0c0234e34e71d8639.camel@j-davis.com
M src/backend/replication/walsender.c
M src/include/nodes/replnodes.h
Fix rescanning of async-aware Append nodes.
commit : f3baaf28a6da588987b94a05a725894805c3eae9
author : Etsuro Fujita <efujita@postgresql.org>
date : Mon, 7 Jun 2021 12:45:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Mon, 7 Jun 2021 12:45:00 +0900
In cases where run-time pruning isn't required, the synchronous and
asynchronous subplans for an async-aware Append node determined using
classify_matching_subplans() should be re-used when rescanning the node,
but the previous code re-determined them using that function repeatedly
each time when rescanning the node, leading to incorrect results in a
normal build and an Assert failure in an Assert-enabled build as that
function doesn't assume that it's called repeatedly in such cases. Fix
the code as mentioned above.
My oversight in commit 27e1f1456.
While at it, initialize async-related pointers/variables to NULL/zero
explicitly in ExecInitAppend() and ExecReScanAppend(), just to be sure.
(The variables would have been set to zero before we get to the latter
function, but let's do so.)
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAPmGK16Q4B2_KY%2BJH7rb7wQbw54AUprp7TMekGTd2T1B62yysQ%40mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/executor/nodeAppend.c
Fix inconsistent equalfuncs.c behavior for FuncCall.funcformat.
commit : a65e9f3f1405b786673feec131879843432bf9a6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Jun 2021 15:46:58 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Jun 2021 15:46:58 -0400
Other equalfuncs.c checks on CoercionForm fields use
COMPARE_COERCIONFORM_FIELD (which makes them no-ops),
but commit 40c24bfef neglected to make _equalFuncCall
do likewise. Fix that.
This is only strictly correct if FuncCall.funcformat has
no semantic effect, instead just determining ruleutils.c
display formatting. 40c24bfef added a couple of checks
in parse analysis that could break that rule; but on closer
inspection, they're redundant, so just take them out again.
Per report from Noah Misch.
Discussion: https://postgr.es/m/20210606063331.GC297923@rfd.leadboat.com
M src/backend/nodes/equalfuncs.c
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_func.c
Add transformed flag to nodes/*funcs.c for CREATE STATISTICS
commit : d57ecebd128cdf2f4844a2ea4d35ff28d7d69be8
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 6 Jun 2021 20:52:58 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 6 Jun 2021 20:52:58 +0200
Commit a4d75c86bf added a new flag, tracking if the statement was
processed by transformStatsStmt(), but failed to add this flag to
nodes/*funcs.c.
Catversion bump, due to adding a flag to copy/equal/out functions.
Reported-by: Noah Misch
Discussion: https://postgr.es/m/ad7891d2-e90c-b446-9fe2-7419143847d7%40enterprisedb.com
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/include/catalog/catversion.h
Standardize nodes/*funcs.c cosmetics for ForeignScan.resultRelation.
commit : a2dee328bbe5b1979bbc6a784deb86a336c9cd74
author : Noah Misch <noah@leadboat.com>
date : Sun, 6 Jun 2021 00:08:21 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 6 Jun 2021 00:08:21 -0700
catversion bump due to readfuncs.c field order change.
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/include/catalog/catversion.h
doc: Simplify COMMENT and SECURITY LABEL documentation
commit : 5c25fd650a774cc4f16ac9c635830d9bc5797f61
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 5 Jun 2021 09:08:40 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 5 Jun 2021 09:08:40 +0200
Just say that objects that reside in schemas can be schema-qualified.
Don't list all possible such objects. The existing lists weren't
complete anyway.
Discussion: https://www.postgresql.org/message-id/flat/b2ec2234-67fe-d861-5cea-f526cd18c086%40enterprisedb.com
M doc/src/sgml/ref/comment.sgml
M doc/src/sgml/ref/security_label.sgml
doc: Make terminology in glossary consistent
commit : 01ddd2f7e411ba434473faebf00f5b5af84c0f64
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 5 Jun 2021 09:01:29 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 5 Jun 2021 09:01:29 +0200
Use "reside in" rather than "belong to" for objects in a schema.
Previous use was a mix of the two.
Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/202106021932.idmbjjaqk7ke@alvherre.pgsql
M doc/src/sgml/glossary.sgml
gitattributes: Add new entry to silence whitespace error
commit : e6159885b78e9b91b2adc3161c5f827d081f2b13
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 5 Jun 2021 07:57:31 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 5 Jun 2021 07:57:31 +0200
M .gitattributes
Fix subtransaction test for Python 3.10
commit : 4a682d85a1408e48ac529295c329ba2c17a44724
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 5 Jun 2021 07:16:34 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 5 Jun 2021 07:16:34 +0200
Starting with Python 3.10, the stacktrace looks differently:
- PL/Python function "subtransaction_exit_subtransaction_in_with", line 3, in <module>
- s.__exit__(None, None, None)
+ PL/Python function "subtransaction_exit_subtransaction_in_with", line 2, in <module>
+ with plpy.subtransaction() as s:
Using try/except specifically makes the error look always the same.
(See https://github.com/python/cpython/pull/25719 for the discussion
of this change in Python.)
Author: Honza Horak <hhorak@redhat.com>
Discussion: https://www.postgresql.org/message-id/flat/853083.1620749597%40sss.pgh.pa.us
RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1959080
M src/pl/plpython/expected/plpython_subtransaction.out
M src/pl/plpython/sql/plpython_subtransaction.sql
Fix postgres_fdw failure with whole-row Vars of type RECORD.
commit : f61db909dfb94f3411f8719916601a11a905b95e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Jun 2021 20:07:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Jun 2021 20:07:08 -0400
Commit 86dc90056 expects that FDWs can cope with whole-row Vars for
their tables, even if the Vars are marked with vartype RECORDOID.
Previously, whole-row Vars generated by the planner had vartype equal
to the relevant table's rowtype OID. (The point behind this change is
to enable sharing of resjunk columns across inheritance child tables.)
It turns out that postgres_fdw fails to cope with this, though through
bad fortune none of its test cases exposed that. Things mostly work,
but when we try to read back a value of such a Var, the expected
rowtype is not available to record_in(). Fortunately, it's not
difficult to hack up the tupdesc that controls this process to
substitute the foreign table's rowtype for RECORDOID. Thus we can
solve the runtime problem while still sharing the resjunk column
with other tables.
Per report from Alexander Pyhalov.
Discussion: https://postgr.es/m/7817fb9ebd6661cdf9b67dec6e129a78@postgrespro.ru
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
Doc: Fix some spelling mistakes
commit : 8f3c06c5d56fc0fa414bcf548860ac50a8fe5982
author : David Rowley <drowley@postgresql.org>
date : Fri, 4 Jun 2021 23:39:40 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 4 Jun 2021 23:39:40 +1200
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/7838B8EE-CFD6-48D7-AE2D-520D69FD84BD@yesql.se
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/ref/analyze.sgml
M doc/src/sgml/ref/pg_amcheck.sgml
Clean up some questionable usages of DatumGet* macros
commit : 8bdb36aab287f564eac534878bc0e1d873a4e3db
author : David Rowley <drowley@postgresql.org>
date : Fri, 4 Jun 2021 22:42:17 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 4 Jun 2021 22:42:17 +1200
This tidies up some questionable coding which made use of
DatumGetPointer() for Datums being passed into functions where the
parameter is expected to be a cstring. We saw no compiler warnings with
the old code as the Pointer type used in DatumGetPointer() happens to
be a char * rather than a void *. However, that's no excuse and we should
be using the correct macro for the job.
Here we also make use of OutputFunctionCall() rather than using
FunctionCall1() directly to call the type's output function.
OutputFunctionCall() is the standard way to do this. It casts the
returned value to a cstring for us.
In passing get rid of a duplicate call to strlen(). Most compilers will
likely optimize away the 2nd call, but there may be some that won't. In
any case, this just aligns the code to some other nearby code that already
does this.
Discussion: https://postgr.es/m/CAApHDvq1D=ehZ8hey8Hz67N+_Zth0GHO5wiVCfv1YcGPMXJq0A@mail.gmail.com
M src/backend/access/brin/brin_minmax_multi.c
Doc: fix bogus intarray index example.
commit : e4539386decae1c435767a69507cc7cbb11ac3ff
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Jun 2021 21:07:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Jun 2021 21:07:12 -0400
The siglen parameter is provided by gist__intbig_ops not
gist__int_ops.
Simon Norris
Discussion: https://postgr.es/m/11BF2AA9-17AE-432A-AFE1-584FB9FB079D@hillcrestgeo.ca
M doc/src/sgml/intarray.sgml
doc: Add description for PGSSLCRLDIR
commit : 1e809db86b160e697a56bf47358f7733475840d3
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 4 Jun 2021 09:46:15 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 4 Jun 2021 09:46:15 +0900
This was missing in the section dedicated to the supported environment
variables of libpq. Oversight in f5465fa.
Reviewed-by: Daniel Gustafsson, Kyotaro Horiguchi
Discussion: https://postgr.es/m/YLhI0mLoRkY3u4Wj@paquier.xyz
M doc/src/sgml/libpq.sgml
doc: Fix link reference for PGSSLMAXPROTOCOLVERSION
commit : 77e9d1b4884262fa09cd8d141c7eadad3affde8b
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 4 Jun 2021 09:33:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 4 Jun 2021 09:33:14 +0900
The link was pointing to the minimum protocol version. Incorrect as of
ff8ca5f.
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/F893F184-C645-4C21-A2BA-583441B7288F@yesql.se
Backpatch-through: 13
M doc/src/sgml/libpq.sgml
Adjust locations which have an incorrect copyright year
commit : 7fc26d11e370afe237631265714221364d7e7910
author : David Rowley <drowley@postgresql.org>
date : Fri, 4 Jun 2021 12:19:50 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 4 Jun 2021 12:19:50 +1200
A few patches committed after ca3b37487 mistakenly forgot to make the
copyright year 2021. Fix these.
Discussion: https://postgr.es/m/CAApHDvqyLmd9P2oBQYJ=DbrV8QwyPRdmXtCTFYPE08h+ip0UJw@mail.gmail.com
M contrib/pageinspect/gistfuncs.c
M src/backend/access/brin/brin_bloom.c
M src/backend/access/brin/brin_minmax_multi.c
M src/backend/rewrite/rewriteSearchCycle.c
M src/backend/utils/adt/jsonbsubs.c
M src/common/hex.c
M src/common/hmac.c
M src/common/hmac_openssl.c
M src/common/sha1.c
M src/common/sha1_int.h
M src/include/common/hex.h
M src/include/common/hmac.h
M src/include/common/sha1.h
M src/include/port/pg_iovec.h
M src/include/rewrite/rewriteSearchCycle.h
In PostgresNode.pm, don't pass SQL to psql on the command line
commit : 11e9caff82bc7326e2bc9782937cb03875050cc4
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 3 Jun 2021 16:08:33 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 3 Jun 2021 16:08:33 -0400
The Msys shell mangles certain patterns in its command line, so avoid
handing arbitrary SQL to psql on the command line and instead use
IPC::Run's redirection facility for stdin. This pattern is already
mostly whats used, but query_poll_until() was not doing the right thing.
Problem discovered on the buildfarm when a new TAP test failed on msys.
M src/test/perl/PostgresNode.pm
Fix incorrect permissions on pg_subscription.
commit : 3590680b85a8e51ef8df550e5a10dedd0d2dfd88
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Jun 2021 14:54:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Jun 2021 14:54:06 -0400
The documented intent is for all columns except subconninfo to be
publicly readable. However, this has been overlooked twice.
subsynccommit has never been readable since it was introduced,
nor has the oid column (which is important for joining).
Given the lack of previous complaints, it's not clear that it's
worth doing anything about this in the back branches. But there's
still time to fix it inexpensively for v14.
Per report from Israel Barth (via Euler Taveira).
Patch by Euler Taveira, possibly-vain comment updates by me.
Discussion: https://postgr.es/m/b8f7c17c-0041-46b6-acfe-2d1f5a985ab4@www.fastmail.com
M src/backend/catalog/system_views.sql
M src/include/catalog/catversion.h
M src/include/catalog/pg_subscription.h
Reduce risks of conflicts in internal queries of REFRESH MATVIEW CONCURRENTLY
commit : 187682c3217375c9b70417bf3235790f639e8e7e
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 3 Jun 2021 15:28:24 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 3 Jun 2021 15:28:24 +0900
The internal SQL queries used by REFRESH MATERIALIZED VIEW CONCURRENTLY
include some aliases for its diff and temporary relations with
rather-generic names: diff, newdata, newdata2 and mv. Depending on the
queries used for the materialized view, using CONCURRENTLY could lead to
some internal failures if the matview query and those internal aliases
conflict.
Those names have been chosen in 841c29c8. This commit switches instead
to a naming pattern which is less likely going to cause conflicts, based
on an idea from Thomas Munro, by appending _$ to those aliases. This is
not perfect as those new names could still conflict, but at least it has
the advantage to keep the code readable and simple while reducing the
likelihood of conflicts to be close to zero.
Reported-by: Mathis Rudolf
Author: Bharath Rupireddy
Reviewed-by: Bernd Helmle, Thomas Munro, Michael Paquier
Discussion: https://postgr.es/m/109c267a-10d2-3c53-b60e-720fcf44d9e8@credativ.de
Backpatch-through: 9.6
M src/backend/commands/matview.c
doc: Group options in pg_amcheck reference page
commit : cb3cffe694b6041c1de47b12b225e05f664c7285
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Jun 2021 06:55:04 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Jun 2021 06:55:04 +0200
The previous arrangement was just one big list, and the internal order
was not all consistent either. Now arrange the options by group and
sort them, the way it's already done in the --help output and one
other reference pages. Also fix some ordering in the --help output.
M doc/src/sgml/ref/pg_amcheck.sgml
M src/bin/pg_amcheck/pg_amcheck.c
Standardize usages of appendStringInfo and appendPQExpBuffer
commit : f736e188ce70bda34f04c9f381b7c5141dc20dce
author : David Rowley <drowley@postgresql.org>
date : Thu, 3 Jun 2021 16:38:03 +1200
committer: David Rowley <drowley@postgresql.org>
date : Thu, 3 Jun 2021 16:38:03 +1200
Fix a few places that were using appendStringInfo() when they should have
been using appendStringInfoString(). Also some cases of
appendPQExpBuffer() that would have been better suited to use
appendPQExpBufferChar(), and finally, some places that used
appendPQExpBuffer() when appendPQExpBufferStr() would have suited better.
There are no bugs are being fixed here. The aim is just to make the code
use the most optimal function for the job.
All the code being changed here is new to PG14. It makes sense to fix
these before we branch for PG15. There are a few other places that we
could fix, but those cases are older code so fixing those seems less
worthwhile as it may cause unnecessary back-patching pain in the future.
Author: Hou Zhijie
Discussion: https://postgr.es/m/OS0PR01MB5716732158B1C4142C6FE375943D9@OS0PR01MB5716.jpnprd01.prod.outlook.com
M src/backend/access/brin/brin_minmax_multi.c
M src/backend/access/heap/vacuumlazy.c
M src/bin/pg_amcheck/pg_amcheck.c
M src/bin/psql/describe.c
Ignore more environment variables in TAP tests
commit : 8279f68a1b13d58a0c9869a60081009d8995e4df
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 3 Jun 2021 11:50:56 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 3 Jun 2021 11:50:56 +0900
Various environment variables were not getting reset in the TAP tests,
which would cause failures depending on the tests or the environment
variables involved. For example, PGSSL{MAX,MIN}PROTOCOLVERSION could
cause failures in the SSL tests. Even worse, a junk value of
PGCLIENTENCODING makes a server startup fail. The list of variables
reset is adjusted in each stable branch depending on what is supported.
While on it, simplify a bit the code per a suggestion from Andrew
Dunstan, using a list of variables instead of doing single deletions.
Reviewed-by: Andrew Dunstan, Daniel Gustafsson
Discussion: https://postgr.es/m/YLbjjRpucIeZ78VQ@paquier.xyz
Backpatch-through: 9.6
M src/test/perl/TestLib.pm
Re-allow custom GUC names that have more than two components.
commit : 2955c2be79b35fa369c83fa3b5f44661cb88afa9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Jun 2021 18:50:15 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Jun 2021 18:50:15 -0400
Commit 3db826bd5 disallowed this case, but it turns out that some
people are depending on it. Since the core grammar has allowed
it since 3dc37cd8d, it seems like this code should fall in line.
Per bug #17045 from Robert Sosinski.
Discussion: https://postgr.es/m/17045-6a4a9f0d1513f72b@postgresql.org
M src/backend/utils/misc/guc.c
M src/test/regress/expected/guc.out
M src/test/regress/sql/guc.sql
Revert most of 39b66a91bd
commit : 8e03eb92e9ad54e2f1ed8b5a73617601f6262f81
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 3 Jun 2021 00:06:42 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 3 Jun 2021 00:06:42 +0200
Reverts most of commit 39b66a91bd, which was found to cause significant
regression for REFRESH MATERIALIZED VIEW. This means only rows inserted
by heap_multi_insert will benefit from the optimization, implemented in
commit 7db0cd2145.
Reported-by: Masahiko Sawada
Discussion: https://postgr.es/m/CAD21AoA%3D%3Df2VSw3c-Cp_y%3DWLKHMKc1D6s7g3YWsCOvgaYPpJcg%40mail.gmail.com
M src/backend/access/heap/heapam.c
M src/backend/access/heap/hio.c
Fix planner's row-mark code for inheritance from a foreign table.
commit : 889592344c48d3965567f331b4ea89dfe6447bce
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Jun 2021 14:38:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Jun 2021 14:38:14 -0400
Commit 428b260f8 broke planning of cases where row marks are needed
(SELECT FOR UPDATE, etc) and one of the query's tables is a foreign
table that has regular table(s) as inheritance children. We got the
reverse case right, but apparently were thinking that foreign tables
couldn't be inheritance parents. Not so; so we need to be able to
add a CTID junk column while adding a new child, not only a wholerow
junk column.
Back-patch to v12 where the faulty code came in.
Amit Langote
Discussion: https://postgr.es/m/CA+HiwqEmo3FV1LAQ4TVyS2h1WM=kMkZUmbNuZSCnfHvMcUcPeA@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/optimizer/util/inherit.c
Update plannodes.h's comments about PlanRowMark.
commit : 79c50ca57828e9f8375766b36cce1e2960eebf87
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Jun 2021 11:52:35 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Jun 2021 11:52:35 -0400
The reference here to different physical column numbers in inherited
UPDATE/DELETE plans is obsolete as of 86dc90056; remove it. Also
rework the text about inheritance cases to make it clearer.
M src/include/nodes/plannodes.h
Teach tab-complete.c about recently-added CREATE TYPE options.
commit : 9e3b3ff2664dd0b349d2a6d6f047128cb3489cf2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Jun 2021 10:44:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Jun 2021 10:44:16 -0400
Commit c7aba7c14 missed adding SUBSCRIPT here,
and commit 6df7a9698 missed adding MULTIRANGE_TYPE_NAME.
Haiying Tang and Tom Lane
Discussion: https://postgr.es/m/OS0PR01MB6113F9EDA46FA53BAA5445BDFB3D9@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/bin/psql/tab-complete.c
Remove unnecessary use of Time::HiRes from 013_crash_restart.pl.
commit : df466d30c6caa02e2215595fd83ca70be3199cec
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 2 Jun 2021 12:20:15 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 2 Jun 2021 12:20:15 +0900
The regression test 013_crash_restart.pl included "use Time::HiRes qw(usleep)",
but the usleep was not used there.
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/63ad1368-18e2-8900-8443-524bdfb1bef5@oss.nttdata.com
M src/test/recovery/t/013_crash_restart.pl
Add regression test for recovery pause.
commit : 6bbc5c5e96b08f6b8c7d28d10ed8dfe6c49dca30
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 2 Jun 2021 12:19:39 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 2 Jun 2021 12:19:39 +0900
Previously there was no regression test for recovery pause feature.
This commit adds the test that checks
- recovery can be paused or resumed expectedly
- pg_get_wal_replay_pause_state() reports the correct pause state
- the paused state ends and promotion continues if a promotion
is triggered while recovery is paused
Suggested-by: Michael Paquier
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Dilip Kumar
Discussion: https://postgr.es/m/YKNirzqM1HYyk5h4@paquier.xyz
M src/test/recovery/t/005_replay_delay.pl
Add Windows file version information to libpq_pipeline.exe.
commit : 42344ad0ed465ea988d8310d2f413d65329f55a8
author : Noah Misch <noah@leadboat.com>
date : Tue, 1 Jun 2021 18:04:15 -0700
committer: Noah Misch <noah@leadboat.com>
date : Tue, 1 Jun 2021 18:04:15 -0700
M src/test/modules/libpq_pipeline/Makefile
Fix missing gettimeofday() declaration.
commit : 49527a32ca97761d78efef732a4ac76a2fc086b2
author : Noah Misch <noah@leadboat.com>
date : Tue, 1 Jun 2021 18:04:14 -0700
committer: Noah Misch <noah@leadboat.com>
date : Tue, 1 Jun 2021 18:04:14 -0700
This avoids a warning under MinGW versions having gettimeofday(), per
buildfarm member walleye.
M src/interfaces/libpq/fe-trace.c
Reject SELECT ... GROUP BY GROUPING SETS (()) FOR UPDATE.
commit : 1103033aedc10295eb689a4b7158f21ef4c14a11
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Jun 2021 11:12:56 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Jun 2021 11:12:56 -0400
This case should be disallowed, just as FOR UPDATE with a plain
GROUP BY is disallowed; FOR UPDATE only makes sense when each row
of the query result can be identified with a single table row.
However, we missed teaching CheckSelectLocking() to check
groupingSets as well as groupClause, so that it would allow
degenerate grouping sets. That resulted in a bad plan and
a null-pointer dereference in the executor.
Looking around for other instances of the same bug, the only one
I found was in examine_simple_variable(). That'd just lead to
silly estimates, but it should be fixed too.
Per private report from Yaoguang Chen.
Back-patch to all supported branches.
M src/backend/parser/analyze.c
M src/backend/utils/adt/selfuncs.c
M src/test/regress/expected/errors.out
M src/test/regress/sql/errors.sql
pgoutput: Fix memory leak due to RelationSyncEntry.map.
commit : eb89cb43a0d0e401e71b8e2345b5f5bc8b2755a1
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 1 Jun 2021 14:14:02 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 1 Jun 2021 14:14:02 +0530
Release memory allocated when creating the tuple-conversion map and its
component TupleDescs when its owning sync entry is invalidated.
TupleDescs must also be freed when no map is deemed necessary, to begin
with.
Reported-by: Andres Freund
Author: Amit Langote
Reviewed-by: Takamichi Osumi, Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/MEYP282MB166933B1AB02B4FE56E82453B64D9@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
M src/backend/replication/pgoutput/pgoutput.c
M src/test/subscription/t/013_partition.pl
Fix error handling in replacement pthread_barrier_init().
commit : a40646e30d85e51a76fb620822d4d921b6802157
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 1 Jun 2021 11:22:22 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 1 Jun 2021 11:22:22 +1200
Commit 44bf3d50 incorrectly used an errno-style interface when supplying
missing pthread functionality (i.e. on macOS), but it should check for
and return error numbers directly.
M src/port/pthread_barrier_wait.c
Fix RADIUS error reporting in hba file parsing
commit : 7c544ecdad814ccda709cfb6aa7d62840c3a7486
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 31 May 2021 18:32:41 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 31 May 2021 18:32:41 +0200
The RADIUS-related checks in parse_hba_line() did not respect elevel
and did not fill in *err_msg. Also, verify_option_list_length()
pasted together error messages in an untranslatable way. To fix the
latter, remove the function and do the error checking inline. It's a
bit more verbose but only minimally longer, and it makes fixing the
first two issues straightforward.
Reviewed-by: Magnus Hagander <magnus@hagander.net>
Discussion: https://www.postgresql.org/message-id/flat/8381e425-8c23-99b3-15ec-3115001db1b2%40enterprisedb.com
M src/backend/libpq/hba.c
Fix mis-planning of repeated application of a projection.
commit : 6ee41a301e70fc8e4ad383bad22d695f66ccb0ac
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 31 May 2021 12:03:00 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 31 May 2021 12:03:00 -0400
create_projection_plan contains a hidden assumption (here made
explicit by an Assert) that a projection-capable Path will yield a
projection-capable Plan. Unfortunately, that assumption is violated
only a few lines away, by create_projection_plan itself. This means
that two stacked ProjectionPaths can yield an outcome where we try to
jam the upper path's tlist into a non-projection-capable child node,
resulting in an invalid plan.
There isn't any good reason to have stacked ProjectionPaths; indeed the
whole concept is faulty, since the set of Vars/Aggs/etc needed by the
upper one wouldn't necessarily be available in the output of the lower
one, nor could the lower one create such values if they weren't
available from its input. Hence, we can fix this by adjusting
create_projection_path to strip any top-level ProjectionPath from the
subpath it's given. (This amounts to saying "oh, we changed our
minds about what we need to project here".)
The test case added here only fails in v13 and HEAD; before that, we
don't attempt to shove the Sort into the parallel part of the plan,
for reasons that aren't entirely clear to me. However, all the
directly-related code looks generally the same as far back as v11,
where the hazard was introduced (by d7c19e62a). So I've got no faith
that the same type of bug doesn't exist in v11 and v12, given the
right test case. Hence, back-patch the code changes, but not the
irrelevant test case, into those branches.
Per report from Bas Poot.
Discussion: https://postgr.es/m/534fca83789c4a378c7de379e9067d4f@politie.nl
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/pathnode.c
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql
Raise a timeout to 180s, in test 010_logical_decoding_timelines.pl.
commit : d03eeab886baa1be73f8fc2938fb842d25a71fe8
author : Noah Misch <noah@leadboat.com>
date : Mon, 31 May 2021 00:29:58 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 31 May 2021 00:29:58 -0700
Per buildfarm member hornet. Also, update Pod documentation showing the
lower value. Back-patch to v10, where the test first appeared.
M src/test/perl/PostgresNode.pm
M src/test/recovery/t/010_logical_decoding_timelines.pl
Improve some error wording with multirange type parsing
commit : 12cc956664f159e97be71e33f15ec5f42e46b24e
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 31 May 2021 11:35:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 31 May 2021 11:35:00 +0900
Braces were referred in some error messages as only brackets (not curly
brackets or curly braces), which can be confusing as other types of
brackets could be used.
While on it, add one test to check after the case of junk characters
detected after a right brace.
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20210514.153153.1814935914483287479.horikyota.ntt@gmail.com
M src/backend/utils/adt/multirangetypes.c
M src/test/regress/expected/multirangetypes.out
M src/test/regress/sql/multirangetypes.sql
Doc: improve libpq service-file docs, avoid overspecifying pathnames.
commit : ba356a397de565c014384aa01a945aab7d50928c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 29 May 2021 14:27:37 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 29 May 2021 14:27:37 -0400
Clarify libpq.sgml's description of service file locations and
semantics. Avoid use of backtick'ed pg_config calls to describe
paths; that doesn't work on Windows, and even on Unix it's an
idiom that not all readers may be instantly familiar with.
Don't overspecify the locations of include files, instead writing
only as much as you'd use in #include directives. The previous text
in these places was incorrect for some installations, depending on
where "postgresql" is in the install path.
Our convention for referencing the user's home directory seems
to be "~", so change the one place that spelled it "$HOME".
install-windows.sgml follows the platform convention of spelling
file paths with "\", so change the one place that used "/".
Haiying Tang and Tom Lane
Discussion: https://postgr.es/m/162149020918.26174.7150424047314144297@wrigleys.postgresql.org
M doc/src/sgml/config.sgml
M doc/src/sgml/install-windows.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/pgbuffercache.sgml
Fix race condition when sharing tuple descriptors.
commit : b1d6538903504904db44679355ac109aeda01737
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 29 May 2021 14:48:15 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 29 May 2021 14:48:15 +1200
Parallel query processes that called BlessTupleDesc() for identical
tuple descriptors at the same moment could crash. There was code to
handle that rare case, but it dereferenced a bogus DSA pointer. Repair.
Back-patch to 11, where commit cc5f8136 added support for sharing tuple
descriptors in parallel queries.
Reported-by: Eric Thinnes <e.thinnes@gmx.de>
Discussion: https://postgr.es/m/99aaa2eb-e194-bf07-c29a-1a76b4f2bcf9%40gmx.de
M src/backend/utils/cache/typcache.c
fix syntax error
commit : d69fcb9caef1ac1f38241645d4fb9f7e0ce02a70
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 28 May 2021 09:35:11 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 28 May 2021 09:35:11 -0400
M src/tools/msvc/Solution.pm
Report configured port in MSVC built pg_config
commit : fb424ae85f6b1e32e545f13902d3ba3429be44df
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 28 May 2021 09:26:30 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 28 May 2021 09:26:30 -0400
This is a long standing omission, discovered when trying to write code
that relied on it.
Backpatch to all live branches.
M src/tools/msvc/Solution.pm
Fix VACUUM VERBOSE's LP_DEAD item pages output.
commit : 9afdea982420f9672b88e5c17d1ee8eec64105fc
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 27 May 2021 17:09:16 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 27 May 2021 17:09:16 -0700
Oversight in commit 5100010e.
M src/backend/access/heap/vacuumlazy.c
Reduce the range of OIDs reserved for genbki.pl.
commit : a4390abecf0f5152cff864e82b67e5f6c8489698
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 May 2021 15:55:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 May 2021 15:55:08 -0400
Commit ab596105b increased FirstBootstrapObjectId from 12000 to 13000,
but we've had some push-back about that. It's worrisome to reduce the
daylight between there and FirstNormalObjectId, because the number of
OIDs consumed during initdb for collation objects is hard to predict.
We can improve the situation by abandoning the assumption that these
OIDs must be globally unique. It should be sufficient for them to be
unique per-catalog. (Any code that's unhappy about that is broken
anyway, since no more than per-catalog uniqueness can be guaranteed
once the OID counter wraps around.) With that change, the largest OID
assigned during genbki.pl (starting from a base of 10000) is a bit
under 11000. This allows reverting FirstBootstrapObjectId to 12000
with reasonable confidence that that will be sufficient for many years
to come.
We are not, at this time, abandoning the expectation that
hand-assigned OIDs (below 10000) are globally unique. Someday that'll
likely be necessary, but the need seems years away still.
This is late for v14, but it seems worth doing it now so that
downstream software doesn't have to deal with the consequences of
a change in FirstBootstrapObjectId. In any case, we already
bought into forcing an initdb for beta2, so another catversion
bump won't hurt.
Discussion: https://postgr.es/m/1665197.1622065382@sss.pgh.pa.us
M doc/src/sgml/bki.sgml
M src/backend/catalog/genbki.pl
M src/include/access/transam.h
M src/include/catalog/catversion.h
Rethink definition of pg_attribute.attcompression.
commit : e6241d8e030fbd2746b3ea3f44e728224298f35b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 May 2021 13:24:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 May 2021 13:24:24 -0400
Redefine '\0' (InvalidCompressionMethod) as meaning "if we need to
compress, use the current setting of default_toast_compression".
This allows '\0' to be a suitable default choice regardless of
datatype, greatly simplifying code paths that initialize tupledescs
and the like. It seems like a more user-friendly approach as well,
because now the default compression choice doesn't migrate into table
definitions, meaning that changing default_toast_compression is
usually sufficient to flip an installation's behavior; one needn't
tediously issue per-column ALTER SET COMPRESSION commands.
Along the way, fix a few minor bugs and documentation issues
with the per-column-compression feature. Adopt more robust
APIs for SetIndexStorageProperties and GetAttributeCompression.
Bump catversion because typical contents of attcompression will now
be different. We could get away without doing that, but it seems
better to ensure v14 installations all agree on this. (We already
forced initdb for beta2, anyway.)
Discussion: https://postgr.es/m/626613.1621787110@sss.pgh.pa.us
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/storage.sgml
M src/backend/access/brin/brin_tuple.c
M src/backend/access/common/indextuple.c
M src/backend/access/common/toast_internals.c
M src/backend/access/common/tupdesc.c
M src/backend/access/heap/heapam_handler.c
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/heap.c
M src/backend/commands/tablecmds.c
M src/backend/parser/gram.y
M src/backend/utils/misc/guc.c
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_dump.c
M src/bin/psql/describe.c
M src/include/access/toast_compression.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_attribute.h
M src/test/regress/expected/compression.out
M src/test/regress/expected/compression_1.out
M src/test/regress/sql/compression.sql
Fix vpath build in libpq_pipeline test
commit : a717e5c771610cf8607f2423ab3ab6b5d30f44ea
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 27 May 2021 16:40:52 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 27 May 2021 16:40:52 +0200
The path needs to be set to refer to the build directory, not the
current directory, because that's actually the source directory at
that point.
fix for 6abc8c2596dbfcb24f9b4d954a1465b8015118c3
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
Add NO_INSTALL option to pgxs
commit : 6abc8c2596dbfcb24f9b4d954a1465b8015118c3
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 27 May 2021 13:58:13 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 27 May 2021 13:58:13 +0200
Apply in libpq_pipeline test makefile, so that the test file is not
installed into tmp_install.
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/cb9d16a6-760f-cd44-28d6-b091d5fb6ca7%40enterprisedb.com
M doc/src/sgml/extend.sgml
M src/makefiles/pgxs.mk
M src/test/modules/libpq_pipeline/Makefile
Fix MSVC scripts when building with GSSAPI/Kerberos
commit : 025110663448a8c877f4b591495f2e5d187d8936
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 27 May 2021 20:11:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 27 May 2021 20:11:00 +0900
The deliverables of upstream Kerberos on Windows are installed with
paths that do not match our MSVC scripts. First, the include folder was
named "inc/" in our scripts, but the upstream MSIs use "include/".
Second, the build would fail with 64-bit environments as the libraries
are named differently.
This commit adjusts the MSVC scripts to be compatible with the latest
installations of upstream, and I have checked that the compilation was
able to work with the 32-bit and 64-bit installations.
Special thanks to Kondo Yuta for the help in investigating the situation
in hamerkop, which had an incorrect configuration for the GSS
compilation.
Reported-by: Brian Ye
Discussion: https://postgr.es/m/162128202219.27274.12616756784952017465@wrigleys.postgresql.org
Backpatch-through: 9.6
M src/tools/msvc/Solution.pm
Replace run-time error check with assertion
commit : 388e75ad33489b77cfb9a8590a91e9287d8fb960
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 27 May 2021 09:52:12 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 27 May 2021 09:52:12 +0200
The error message was checking that the structures returned from the
parser matched expectations. That's something we usually use
assertions for, not a full user-facing error message. So replace that
with an assertion (hidden inside lfirst_node()).
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/452e9df8-ec89-e01b-b64a-8cc6ce830458%40enterprisedb.com
M src/backend/commands/statscmds.c
doc: Fix description of some GUCs in docs and postgresql.conf.sample
commit : 2941138e60fc711bd221b3264807f36cc079dfbb
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 27 May 2021 14:57:28 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 27 May 2021 14:57:28 +0900
The following parameters have been imprecise, or incorrect, about their
description (PGC_POSTMASTER or PGC_SIGHUP):
- autovacuum_work_mem (docs, as of 9.6~)
- huge_page_size (docs, as of 14~)
- max_logical_replication_workers (docs, as of 10~)
- max_sync_workers_per_subscription (docs, as of 10~)
- min_dynamic_shared_memory (docs, as of 14~)
- recovery_init_sync_method (postgresql.conf.sample, as of 14~)
- remove_temp_files_after_crash (docs, as of 14~)
- restart_after_crash (docs, as of 9.6~)
- ssl_min_protocol_version (docs, as of 12~)
- ssl_max_protocol_version (docs, as of 12~)
This commit adjusts the description of all these parameters to be more
consistent with the practice used for the others.
Revewed-by: Justin Pryzby
Discussion: https://postgr.es/m/YK2ltuLpe+FbRXzA@paquier.xyz
Backpatch-through: 9.6
M doc/src/sgml/config.sgml
M src/backend/utils/misc/postgresql.conf.sample
Fix assertion during streaming of multi-insert toast changes.
commit : 6f4bdf81529fdaf6744875b0be99ecb9bfb3b7e0
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 27 May 2021 07:59:43 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 27 May 2021 07:59:43 +0530
While decoding the multi-insert WAL we can't clean the toast untill we get
the last insert of that WAL record. Now if we stream the changes before we
get the last change, the memory for toast chunks won't be released and we
expect the txn to have streamed all changes after streaming. This
restriction is mainly to ensure the correctness of streamed transactions
and it doesn't seem worth uplifting such a restriction just to allow this
case because anyway we will stream the transaction once such an insert is
complete.
Previously we were using two different flags (one for toast tuples and
another for speculative inserts) to indicate partial changes. Now instead
we replaced both of them with a single flag to indicate partial changes.
Reported-by: Pavan Deolasee
Author: Dilip Kumar
Reviewed-by: Pavan Deolasee, Amit Kapila
Discussion: https://postgr.es/m/CABOikdN-_858zojYN-2tNcHiVTw-nhxPwoQS4quExeweQfG1Ug@mail.gmail.com
M contrib/test_decoding/expected/stream.out
M contrib/test_decoding/sql/stream.sql
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h
Fix typo in heapam.c
commit : 190fa5a00a8f9ecee8eef2c8e26136b772b94e19
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 26 May 2021 19:53:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 26 May 2021 19:53:03 +0900
Author: Hou Zhijie
Discussion: https://postgr.es/m/OS0PR01MB571612191738540B27A8DE5894249@OS0PR01MB5716.jpnprd01.prod.outlook.com
M src/backend/access/heap/heapam.c
Make detach-partition-concurrently-4 less timing sensitive
commit : eb43bdbf5104c183412aac0fccf8e515e60d9212
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 25 May 2021 19:32:22 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 25 May 2021 19:32:22 -0400
Same as 5e0b1aeb2dfe, for the companion test file.
This one seems lower probability (only two failures in a month of runs);
I was hardly able to reproduce a failure without a patch, so the fact
that I was also unable to reproduce one with it doesn't say anything.
We'll have to wait for further buildfarm results to see if we need any
further adjustments.
Discussion: https://postgr.es/m/20210524090712.GA3771394@rfd.leadboat.com
M src/test/isolation/expected/detach-partition-concurrently-4.out
M src/test/isolation/specs/detach-partition-concurrently-4.spec
Fix use of uninitialized variable in inline_function().
commit : e30e3fdea873e4e9517c490232ea1d3bcef6c643
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 25 May 2021 12:55:52 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 25 May 2021 12:55:52 -0400
Commit e717a9a18 introduced a code path that bypassed the call of
get_expr_result_type, which is not good because we need its rettupdesc
result to pass to check_sql_fn_retval. We'd failed to notice right
away because the code path in which check_sql_fn_retval uses that
argument is fairly hard to reach in this context. It's not impossible
though, and in any case inline_function would have no business
assuming that check_sql_fn_retval doesn't need that value.
To fix, move get_expr_result_type out of the if-block, which in
turn requires moving the construction of the dummy FuncExpr
out of it.
Per report from Ranier Vilela. (I'm bemused by the lack of any
compiler complaints...)
Discussion: https://postgr.es/m/CAEudQAqBqQpQ3HruWAGU_7WaMJ7tntpk0T8k_dVtNB46DqdBgw@mail.gmail.com
M src/backend/optimizer/util/clauses.c
Make detach-partition-concurrently-3 less timing-sensitive
commit : 5e0b1aeb2dfed4f1eb7ac5154c1573885a70db41
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 25 May 2021 12:53:29 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 25 May 2021 12:53:29 -0400
This recently added test has shown to be too sensitive to timing when
sending a cancel to a session waiting for a lock.
We fix this by running a no-op query in the blocked session immediately
after the cancel; this avoids the session that sent the cancel sending
another query immediately before the cancel has been reported.
Idea by Noah Misch.
With that fix, we sometimes see that the cancel error report is shown
only relative to the step that is cancelled, instead of together with
the step that sends the cancel. To increase the probability that both
steps are shown togeter, add a 0.1s sleep to the cancel. In normal
conditions this appears sufficient to silence most failures, but we'll
see that the slower buildfarm members say about it.
Reported-by: Takamichi Osumi <osumi.takamichi@fujitsu.com>
Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
M src/test/isolation/expected/detach-partition-concurrently-3.out
M src/test/isolation/specs/detach-partition-concurrently-3.spec
postgresql.conf.sample: Make vertical spacing consistent
commit : 8673a37c85fef00dd5b9c04197538142bec10542
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 May 2021 11:49:54 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 May 2021 11:49:54 +0200
M src/backend/utils/misc/postgresql.conf.sample
Fix memory leak when de-toasting compressed values in VACUUM FULL/CLUSTER
commit : fb0f5f0172edf9f63c8f70ea9c1ec043b61c770e
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 25 May 2021 14:27:18 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 25 May 2021 14:27:18 +0900
VACUUM FULL and CLUSTER can be used to enforce the use of the existing
compression method of a toastable column if a value currently stored is
compressed with a method that does not match the column's defined
method. The code in charge of decompressing and recompressing toast
values at rewrite left around the detoasted values, causing an
accumulation of memory allocated in TopTransactionContext.
When processing large relations, this could cause the system to run out
of memory. The detoasted values are not needed once their tuple is
rewritten, and this commit ensures that the necessary cleanup happens.
Issue introduced by bbe0a81d. The comments of the area are reordered a
bit while on it.
Reported-by: Andres Freund
Analyzed-by: Andres Freund
Author: Michael Paquier
Reviewed-by: Dilip Kumar
Discussion: https://postgr.es/m/20210521211929.pcehg6f23icwstdb@alap3.anarazel.de
M src/backend/access/heap/heapam_handler.c
Doc: Update logical decoding stats information.
commit : 0c6b92d9c6fb74255467573fde54f65139b26603
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 25 May 2021 10:51:45 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 25 May 2021 10:51:45 +0530
Add the information of pg_stat_replication_slots view along with other
system catalogs related to logical decoding.
Author: Vignesh C
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/20210319185247.ldebgpdaxsowiflw@alap3.anarazel.de
M doc/src/sgml/logicaldecoding.sgml
Improve docs and error messages for parallel vacuum.
commit : 0734b0e983443882ec509ab4501c30ba9b706f5f
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 25 May 2021 09:26:53 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 25 May 2021 09:26:53 +0530
The error messages, docs, and one of the options were using
'parallel degree' to indicate parallelism used by vacuum command. We
normally use 'parallel workers' at other places so change it for parallel
vacuum accordingly.
Author: Bharath Rupireddy
Reviewed-by: Dilip Kumar, Amit Kapila
Backpatch-through: 13
Discussion: https://postgr.es/m/CALj2ACWz=PYrrFXVsEKb9J1aiX4raA+UBe02hdRp_zqDkrWUiw@mail.gmail.com
M doc/src/sgml/ref/vacuumdb.sgml
M src/backend/commands/vacuum.c
M src/bin/scripts/vacuumdb.c
M src/test/regress/expected/vacuum.out
Disallow SSL renegotiation
commit : 01e6f1a842f406170e5f717305e4a6cf0e84b3ee
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 25 May 2021 10:10:09 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 25 May 2021 10:10:09 +0900
SSL renegotiation is already disabled as of 48d23c72, however this does
not prevent the server to comply with a client willing to use
renegotiation. In the last couple of years, renegotiation had its set
of security issues and flaws (like the recent CVE-2021-3449), and it
could be possible to crash the backend with a client attempting
renegotiation.
This commit takes one extra step by disabling renegotiation in the
backend in the same way as SSL compression (f9264d15) or tickets
(97d3a0b0). OpenSSL 1.1.0h has added an option named
SSL_OP_NO_RENEGOTIATION able to achieve that. In older versions
there is an option called SSL3_FLAGS_NO_RENEGOTIATE_CIPHERS that
was undocumented, and could be set within the SSL object created when
the TLS connection opens, but I have decided not to use it, as it feels
trickier to rely on, and it is not official. Note that this option is
not usable in OpenSSL < 1.1.0h as the internal contents of the *SSL
object are hidden to applications.
SSL renegotiation concerns protocols up to TLSv1.2.
Per original report from Robert Haas, with a patch based on a suggestion
by Andres Freund.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/YKZBXx7RhU74FlTE@paquier.xyz
Backpatch-through: 9.6
M src/backend/libpq/be-secure-openssl.c
Fix setrefs.c code for Result Cache nodes
commit : cba5c70b956810c61b3778f7041f92fbb8065acb
author : David Rowley <drowley@postgresql.org>
date : Tue, 25 May 2021 12:50:22 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 25 May 2021 12:50:22 +1200
Result Cache, added in 9eacee2e6 neglected to properly adjust the plan
references in setrefs.c. This could lead to the following error during
EXPLAIN:
ERROR: cannot decompile join alias var in plan tree
Fix that.
Bug: 17030
Reported-by: Hans Buschmann
Discussion: https://postgr.es/m/17030-5844aecae42fe223@postgresql.org
M src/backend/optimizer/plan/setrefs.c
M src/test/regress/expected/join.out
Consider triggering VACUUM failsafe during scan.
commit : c242baa4a831ac2e7dcaec85feb410aefa3a996e
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 24 May 2021 17:14:02 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 24 May 2021 17:14:02 -0700
The wraparound failsafe mechanism added by commit 1e55e7d1 handled the
one-pass strategy case (i.e. the "table has no indexes" case) by adding
a dedicated failsafe check. This made up for the fact that the usual
one-pass checks inside lazy_vacuum_all_indexes() cannot ever be reached
during a one-pass strategy VACUUM.
This approach failed to account for two-pass VACUUMs that opt out of
index vacuuming up-front. The INDEX_CLEANUP off case in the only case
that works like that.
Fix this by performing a failsafe check every 4GB during the first scan
of the heap, regardless of the details of the VACUUM. This eliminates
the special case, and will make the failsafe trigger more reliably.
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Andres Freund <andres@anarazel.de>
Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/20210424002921.pb3t7h6frupdqnkp@alap3.anarazel.de
M src/backend/access/heap/vacuumlazy.c
Doc: move some catalogs.sgml entries to the right place.
commit : 713a431c781fbfe1a22fae4991836077f0f4c513
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 24 May 2021 18:03:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 24 May 2021 18:03:47 -0400
pg_statistic_ext_data.stxdexpr was listed under the wrong catalog,
as was pg_stats_ext.exprs. Also there was a bogus entry for
pg_statistic_ext_data.stxexprs. Apparently a merge failure in
commit a4d75c86b.
Guillaume Lelarge and Tom Lane
Discussion: https://postgr.es/m/CAECtzeUHw+w64eUFVeV_2FJviAw6oZ0wNLkmU843ZH4hAQfiWg@mail.gmail.com
M doc/src/sgml/catalogs.sgml
Add missing NULL check when building Result Cache paths
commit : 99c5852e20a0987eca1c38ba0c09329d4076b6a0
author : David Rowley <drowley@postgresql.org>
date : Mon, 24 May 2021 12:37:11 +1200
committer: David Rowley <drowley@postgresql.org>
date : Mon, 24 May 2021 12:37:11 +1200
Code added in 9e215378d to disable building of Result Cache paths when
not all join conditions are part of the parameterization of a unique
join failed to first check if the inner path's param_info was set before
checking the param_info's ppi_clauses.
Add a check for NULL values here and just bail on trying to build the
path if param_info is NULL. lateral_vars are not considered when
deciding if the join is unique, so we're not missing out on doing the
optimization when there are lateral_vars and no param_info.
Reported-by: Coverity, via Tom Lane
Discussion: https://postgr.es/m/457998.1621779290@sss.pgh.pa.us
M src/backend/optimizer/path/joinpath.c
Re-order pg_attribute columns to eliminate some padding space.
commit : f5024d8d7b04de2f5f4742ab433cc38160354861
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 May 2021 12:12:09 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 May 2021 12:12:09 -0400
Now that attcompression is just a char, there's a lot of wasted
padding space after it. Move it into the group of char-wide
columns to save a net of 4 bytes per pg_attribute entry. While
we're at it, swap the order of attstorage and attalign to make for
a more logical grouping of these columns.
Also re-order actions in related code to match the new field ordering.
This patch also fixes one outright bug: equalTupleDescs() failed to
compare attcompression. That could, for example, cause relcache
reload to fail to adopt a new value following a change.
Michael Paquier and Tom Lane, per a gripe from Andres Freund.
Discussion: https://postgr.es/m/20210517204803.iyk5wwvwgtjcmc5w@alap3.anarazel.de
M doc/src/sgml/catalogs.sgml
M src/backend/access/common/tupdesc.c
M src/backend/access/spgist/spgutils.c
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/commands/tablecmds.c
M src/include/access/spgist_private.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_attribute.h
Be more verbose when the postmaster unexpectedly quits.
commit : bc2a389efb3b52d259cefd53c16cfa00742116f2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 May 2021 10:50:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 May 2021 10:50:21 -0400
Emit a LOG message when the postmaster stops because of a failure in
the startup process. There already is a similar message if we exit
for that reason during PM_STARTUP phase, so it seems inconsistent
that there was none if the startup process fails later on.
Also emit a LOG message when the postmaster stops after a crash
because restart_after_crash is disabled. This seems potentially
helpful in case DBAs (or developers) forget that that's set.
Also, it was the only remaining place where the postmaster would
do an abnormal exit without any comment as to why.
In passing, remove an unreachable call of ExitPostmaster(0).
Discussion: https://postgr.es/m/194914.1621641288@sss.pgh.pa.us
M src/backend/postmaster/postmaster.c
doc: word-wrap and indent PG 14 relnotes
commit : 8f73ed6b659464274eb9cc8358588b569960d0be
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 22 May 2021 22:25:05 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 22 May 2021 22:25:05 -0400
M doc/src/sgml/release-14.sgml
Fix access to no-longer-open relcache entry in logical-rep worker.
commit : b39630fd41f25b414d0ea9b30804f4105f2a0aff
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 May 2021 21:24:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 May 2021 21:24:48 -0400
If we redirected a replicated tuple operation into a partition child
table, and then tried to fire AFTER triggers for that event, the
relation cache entry for the child table was already closed. This has
no visible ill effects as long as the entry is still there and still
valid, but an unluckily-timed cache flush could result in a crash or
other misbehavior.
To fix, postpone the ExecCleanupTupleRouting call (which is what
closes the child table) until after we've fired triggers. This
requires a bit of refactoring so that the cleanup function can
have access to the necessary state.
In HEAD, I took the opportunity to simplify some of worker.c's
function APIs based on use of the new ApplyExecutionData struct.
However, it doesn't seem safe/practical to back-patch that aspect,
at least not without a lot of analysis of possible interactions
with a04daa97a.
In passing, add an Assert to afterTriggerInvokeEvents to catch
such cases. This seems worthwhile because we've grown a number
of fairly unstructured ways of calling AfterTriggerEndQuery.
Back-patch to v13, where worker.c grew the ability to deal with
partitioned target tables.
Discussion: https://postgr.es/m/3382681.1621381328@sss.pgh.pa.us
M src/backend/commands/trigger.c
M src/backend/replication/logical/worker.c
doc: PG 14 relnotes, adjust pg_{read|write}_all_data entry
commit : 7ce7d07e1c5fb33ee56bda235ae3d53f162f3bc0
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 22 May 2021 20:17:58 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 22 May 2021 20:17:58 -0400
Reported-by: Stephen Frost
Discussion: https://postgr.es/m/20210522232945.GO20766@tamriel.snowman.net
M doc/src/sgml/release-14.sgml
Update PG 14 relnotes for vacuum_cost_page_miss
commit : 8dcae7f0a3d6aba1afad1599ab18d259c417b4ee
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 22 May 2021 19:24:23 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 22 May 2021 19:24:23 -0400
Reported-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-WzmgSnDX9WVoxRZxuKeCy2MzLO9Dmo4+go0RzNW0VBdhmw@mail.gmail.com
M doc/src/sgml/release-14.sgml
Remove plpgsql's special-case code paths for SET/RESET.
commit : 30168be8f75b95183abccf48f0da7a64a0cfbd9f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 May 2021 10:25:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 May 2021 10:25:36 -0400
In the wake of 84f5c2908, it's no longer necessary for plpgsql to
handle SET/RESET specially. The point of that was just to avoid
taking a new transaction snapshot prematurely, which the regular code
path through _SPI_execute_plan() now does just fine (in fact better,
since it now does the right thing for LOCK too). Hence, rip out a
few lines of code, going back to the old way of treating SET/RESET
as a generic SQL command. This essentially reverts all but the
test cases from b981275b6.
Discussion: https://postgr.es/m/15990-eee2ac466b11293d@postgresql.org
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_funcs.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/pl_unreserved_kwlist.h
M src/pl/plpgsql/src/plpgsql.h
Fix planner's use of Result Cache with unique joins
commit : 9e215378d7fbb7d4615be917917c52f246cc6c61
author : David Rowley <drowley@postgresql.org>
date : Sat, 22 May 2021 16:22:27 +1200
committer: David Rowley <drowley@postgresql.org>
date : Sat, 22 May 2021 16:22:27 +1200
When the planner considered using a Result Cache node to cache results
from the inner side of a Nested Loop Join, it failed to consider that the
inner path's parameterization may not be the entire join condition. If
the join was marked as inner_unique then we may accidentally put the cache
in singlerow mode. This meant that entries would be marked as complete
after caching the first row. That was wrong as if only part of the join
condition was parameterized then the uniqueness of the unique join was not
guaranteed at the Result Cache's level. The uniqueness is only guaranteed
after Nested Loop applies the join filter. If subsequent rows were found,
this would lead to:
ERROR: cache entry already complete
This could have been fixed by only putting the cache in singlerow mode if
the entire join condition was parameterized. However, Nested Loop will
only read its inner side so far as the first matching row when the join is
unique, so that might mean we never get an opportunity to mark cache
entries as complete. Since non-complete cache entries are useless for
subsequent lookups, we just don't bother considering a Result Cache path
in this case.
In passing, remove the XXX comment that claimed the above ERROR might be
better suited to be an Assert. After there being an actual case which
triggered it, it seems better to keep it an ERROR.
Reported-by: David Christensen
Discussion: https://postgr.es/m/CAOxo6X+dy-V58iEPFgst8ahPKEU+38NZzUuc+a7wDBZd4TrHMQ@mail.gmail.com
M src/backend/executor/nodeResultCache.c
M src/backend/optimizer/path/joinpath.c
doc: complete adding XML markup to PG 14 relnotes
commit : 0cdaa05b40e9f28e5d6d58ccd06fe19f3cd920c9
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 21 May 2021 20:51:53 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 21 May 2021 20:51:53 -0400
M doc/src/sgml/release-14.sgml
doc: more XML markup for PG 14 release notes
commit : 55370f8db96c8416940ad0b05be7a00a9f059a9f
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 21 May 2021 16:16:56 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 21 May 2021 16:16:56 -0400
M doc/src/sgml/release-14.sgml
Disallow whole-row variables in GENERATED expressions.
commit : 4b10074453d182b5fc11a5667bab2ef8532ff3a6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 May 2021 15:12:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 May 2021 15:12:08 -0400
This was previously allowed, but I think that was just an oversight.
It's a clear violation of the rule that a generated column cannot
depend on itself or other generated columns. Moreover, because the
code was relying on the assumption that no such cross-references
exist, it was pretty easy to crash ALTER TABLE and perhaps other
places. Even if you managed not to crash, you got quite unstable,
implementation-dependent results.
Per report from Vitaly Ustinov.
Back-patch to v12 where GENERATED came in.
Discussion: https://postgr.es/m/CAM_DEiWR2DPT6U4xb-Ehigozzd3n3G37ZB1+867zbsEVtYoJww@mail.gmail.com
M src/backend/catalog/heap.c
M src/test/regress/expected/generated.out
M src/test/regress/sql/generated.sql
Fix usage of "tableoid" in GENERATED expressions.
commit : 2b0ee126bbf01cbfd657bd53c94f9284ba903ca2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 May 2021 15:02:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 May 2021 15:02:06 -0400
We consider this supported (though I've got my doubts that it's a
good idea, because tableoid is not immutable). However, several
code paths failed to fill the field in soon enough, causing such
a GENERATED expression to see zero or the wrong value. This
occurred when ALTER TABLE adds a new GENERATED column to a table
with existing rows, and during regular INSERT or UPDATE on a
foreign table with GENERATED columns.
Noted during investigation of a report from Vitaly Ustinov.
Back-patch to v12 where GENERATED came in.
Discussion: https://postgr.es/m/CAM_DEiWR2DPT6U4xb-Ehigozzd3n3G37ZB1+867zbsEVtYoJww@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/backend/executor/nodeModifyTable.c
M src/test/regress/expected/generated.out
M src/test/regress/sql/generated.sql
Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.
commit : 84f5c2908dad81e8622b0406beea580e40bb03ac
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 May 2021 14:03:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 May 2021 14:03:53 -0400
COMMIT/ROLLBACK necessarily destroys all snapshots within the session.
The original implementation of intra-procedure transactions just
cavalierly did that, ignoring the fact that this left us executing in
a rather different environment than normal. In particular, it turns
out that handling of toasted datums depends rather critically on there
being an outer ActiveSnapshot: otherwise, when SPI or the core
executor pop whatever snapshot they used and return, it's unsafe to
dereference any toasted datums that may appear in the query result.
It's possible to demonstrate "no known snapshots" and "missing chunk
number N for toast value" errors as a result of this oversight.
Historically this outer snapshot has been held by the Portal code,
and that seems like a good plan to preserve. So add infrastructure
to pquery.c to allow re-establishing the Portal-owned snapshot if it's
not there anymore, and add enough bookkeeping support that we can tell
whether it is or not.
We can't, however, just re-establish the Portal snapshot as part of
COMMIT/ROLLBACK. As in normal transaction start, acquiring the first
snapshot should wait until after SET and LOCK commands. Hence, teach
spi.c about doing this at the right time. (Note that this patch
doesn't fix the problem for any PLs that try to run intra-procedure
transactions without using SPI to execute SQL commands.)
This makes SPI's no_snapshots parameter rather a misnomer, so in HEAD,
rename that to allow_nonatomic.
replication/logical/worker.c also needs some fixes, because it wasn't
careful to hold a snapshot open around AFTER trigger execution.
That code doesn't use a Portal, which I suspect someday we're gonna
have to fix. But for now, just rearrange the order of operations.
This includes back-patching the recent addition of finish_estate()
to centralize the cleanup logic there.
This also back-patches commit 2ecfeda3e into v13, to improve the
test coverage for worker.c (it was that test that exposed that
worker.c's snapshot management is wrong).
Per bug #15990 from Andreas Wicht. Back-patch to v11 where
intra-procedure COMMIT was added.
Discussion: https://postgr.es/m/15990-eee2ac466b11293d@postgresql.org
M doc/src/sgml/spi.sgml
M src/backend/commands/functioncmds.c
M src/backend/executor/spi.c
M src/backend/replication/logical/worker.c
M src/backend/tcop/pquery.c
M src/backend/utils/mmgr/portalmem.c
M src/include/executor/spi.h
M src/include/tcop/pquery.h
M src/include/utils/portal.h
M src/pl/plpgsql/src/pl_exec.c
M src/test/isolation/expected/plpgsql-toast.out
M src/test/isolation/specs/plpgsql-toast.spec
Put some psql documentation pieces back into alphabetical order
commit : 124966c1a35b950210e12048e64533963960febd
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 21 May 2021 17:10:09 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 21 May 2021 17:10:09 +0200
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/help.c
Fix deadlock for multiple replicating truncates of the same table.
commit : 6d0eb38557155855539cd007f04736dc3b2ba16f
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 21 May 2021 07:54:27 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 21 May 2021 07:54:27 +0530
While applying the truncate change, the logical apply worker acquires
RowExclusiveLock on the relation being truncated. This allowed truncate on
the relation at a time by two apply workers which lead to a deadlock. The
reason was that one of the workers after updating the pg_class tuple tries
to acquire SHARE lock on the relation and started to wait for the second
worker which has acquired RowExclusiveLock on the relation. And when the
second worker tries to update the pg_class tuple, it starts to wait for
the first worker which leads to a deadlock. Fix it by acquiring
AccessExclusiveLock on the relation before applying the truncate change as
we do for normal truncate operation.
Author: Peter Smith, test case by Haiying Tang
Reviewed-by: Dilip Kumar, Amit Kapila
Backpatch-through: 11
Discussion: https://postgr.es/m/CAHut+PsNm43p0jM+idTvWwiGZPcP0hGrHMPK9TOAkc+a4UpUqw@mail.gmail.com
M src/backend/replication/logical/worker.c
M src/test/subscription/t/010_truncate.pl
Avoid detoasting failure after COMMIT inside a plpgsql FOR loop.
commit : f21fadafaf0fb5ea4c9622d915972651273d62ce
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 20 May 2021 18:32:37 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 20 May 2021 18:32:37 -0400
exec_for_query() normally tries to prefetch a few rows at a time
from the query being iterated over, so as to reduce executor
entry/exit overhead. Unfortunately this is unsafe if we have
COMMIT or ROLLBACK within the loop, because there might be
TOAST references in the data that we prefetched but haven't
yet examined. Immediately after the COMMIT/ROLLBACK, we have
no snapshots in the session, meaning that VACUUM is at liberty
to remove recently-deleted TOAST rows.
This was originally reported as a case triggering the "no known
snapshots" error in init_toast_snapshot(), but even if you miss
hitting that, you can get "missing toast chunk", as illustrated
by the added isolation test case.
To fix, just disable prefetching in non-atomic contexts. Maybe
there will be performance complaints prompting us to work harder
later, but it's not clear at the moment that this really costs
much, and I doubt we'd want to back-patch any complicated fix.
In passing, adjust that error message in init_toast_snapshot()
to be a little clearer about the likely cause of the problem.
Patch by me, based on earlier investigation by Konstantin Knizhnik.
Per bug #15990 from Andreas Wicht. Back-patch to v11 where
intra-procedure COMMIT was added.
Discussion: https://postgr.es/m/15990-eee2ac466b11293d@postgresql.org
M src/backend/access/common/toast_internals.c
M src/pl/plpgsql/src/pl_exec.c
M src/test/isolation/expected/plpgsql-toast.out
M src/test/isolation/specs/plpgsql-toast.spec
doc: change PG 14 relnotes as suggested by Justin Pryzby
commit : 4f586fe244a296d7c781de3f06c54755f2ae222b
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 20 May 2021 15:50:46 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 20 May 2021 15:50:46 -0400
M doc/src/sgml/release-14.sgml
Install PostgresVersion.pm
commit : bdbb2ce7d51e93ca2ec68e25e2fafb271b34e72d
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 20 May 2021 15:11:17 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 20 May 2021 15:11:17 -0400
A lamentable oversight on my part meant that when PostgresVersion.pm was
added in commit 4c4eaf3d19 provision to install it was not added to the
Makefile, so it was not installed along with the other perl modules.
M src/test/perl/Makefile
Clean up cpluspluscheck violation.
commit : 6d59a218c38adf5b993200a804713df4982a0c75
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 20 May 2021 13:03:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 20 May 2021 13:03:08 -0400
"typename" is a C++ keyword, so pg_upgrade.h fails to compile in C++.
Fortunately, there seems no likely reason for somebody to need to
do that. Nonetheless, it's project policy that all .h files should
pass cpluspluscheck, so rename the argument to fix that.
Oversight in 57c081de0; back-patch as that was. (The policy requiring
pg_upgrade.h to pass cpluspluscheck only goes back to v12, but it
seems best to keep this code looking the same in all branches.)
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/version.c
Use a more portable way to get the version string in PostgresNode
commit : 8bdd6f563aa2456de602e78991e6a9f61b8ec86d
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 20 May 2021 08:03:15 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 20 May 2021 08:03:15 -0400
Older versions of perl on Windows don't like the list form of pipe open,
and perlcritic doesn't like the string form of open, so we avoid both
with a simpler formulation using qx{}.
Per complaint from Amit Kapila.
M src/test/perl/PostgresNode.pm
Avoid creating testtablespace directories where not wanted.
commit : 413c1ef98e0c9c708c4a9a13a838a55b65b16a80
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 19 May 2021 14:04:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 19 May 2021 14:04:01 -0400
Recently we refactored things so that pg_regress makes the
"testtablespace" subdirectory used by the core regression tests,
instead of doing that in the makefiles. That had the undesirable
side effect of making such a subdirectory in every directory that
has "input" or "output" test files. Since these subdirectories
remain empty, git doesn't complain about them, but nonetheless
they're clutter.
To fix, invent an explicit --make-testtablespace-dir switch,
so that pg_regress only makes the subdirectory when explicitly
told to.
Discussion: https://postgr.es/m/2854388.1621284789@sss.pgh.pa.us
M src/test/regress/GNUmakefile
M src/test/regress/pg_regress.c
M src/tools/msvc/vcregress.pl
doc: revert 1e7d53bd01 so libpq chapter number is accessable
commit : 4f7d1c30966cc02fd5eba2f0d51d1f53be07d457
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 19 May 2021 11:22:21 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 19 May 2021 11:22:21 -0400
Fix PG 14 relnotes to use <link> instead of <xref>. This was discussed
in commit message 59fa7eb603.
M doc/src/sgml/libpq.sgml
M doc/src/sgml/release-14.sgml
doc: add xreflabel for libpq chapter, needed for PG 14 relnotes
commit : 1e7d53bd019e9d86ef1013308715622a2e400d3b
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 19 May 2021 11:01:28 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 19 May 2021 11:01:28 -0400
M doc/src/sgml/libpq.sgml
Fix pgbench permute tests.
commit : 0f516d039d8023163e82fa51104052306068dd69
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Wed, 19 May 2021 12:50:58 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Wed, 19 May 2021 12:50:58 +0100
One of the tests for the pgbench permute() function added by
6b258e3d68 fails on some 32-bit platforms, due to variations in the
floating point computations in getrand(). The remaining tests give
sufficient coverage, so just remove the failing test.
Reported by Christoph Berg. Analysis by Thomas Munro and Tom Lane.
Based on patch by Fabien Coelho.
Discussion: https://postgr.es/m/YKQnUoYV63GRJBDD@msg.df7cb.de
M src/bin/pgbench/t/001_pgbench_with_server.pl
Make standby promotion reset the recovery pause state to 'not paused'.
commit : 167bd4804995afd654bd97ca9486acbece24377e
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 19 May 2021 13:48:19 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 19 May 2021 13:48:19 +0900
If a promotion is triggered while recovery is paused, the paused state ends
and promotion continues. But previously in that case
pg_get_wal_replay_pause_state() returned 'paused' wrongly while a promotion
was ongoing.
This commit changes a standby promotion so that it marks the recovery
pause state as 'not paused' when it's triggered, to fix the issue.
Author: Fujii Masao
Reviewed-by: Dilip Kumar, Kyotaro Horiguchi
Discussion: https://postgr.es/m/f706876c-4894-0ba5-6f4d-79803eeea21b@oss.nttdata.com
M src/backend/access/transam/xlog.c
Fix 020_messages.pl test.
commit : 0a442a408b40d2c6710de7e5397cb2e769d8c630
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 19 May 2021 08:54:46 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 19 May 2021 08:54:46 +0530
We were not waiting for a publisher to catch up with the subscriber after
creating a subscription. Now, it can happen that apply worker starts
replication even after we have disabled the subscription in the test. This
will make the test expect that there is no active slot whereas there
exists one. Fix this symptom by allowing the publisher to wait for
catching up with the subscription.
It is not a good idea to ensure if the slot is still active by checking
for walsender existence as we release the slot after we clean up the
walsender related memory. Fix that by checking the slot status in
pg_replication_slots.
Also, it is better to avoid repeated enabling/disabling of the
subscription.
Finally, we make autovacuum off for this test to avoid any empty
transaction appearing in the test while consuming changes.
Reported-by: as per buildfarm
Author: Vignesh C
Reviewed-by: Amit Kapila, Michael Paquier
Discussion: https://postgr.es/m/CAA4eK1+uW1UGDHDz-HWMHMen76mKP7NJebOTZN4uwbyMjaYVww@mail.gmail.com
M src/test/subscription/t/020_messages.pl
doc: partial completion of XML markup for PG 14 release notes
commit : 6a5bde7d4f96ef153578eaeb624ae12e48b46e85
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 18 May 2021 23:21:47 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 18 May 2021 23:21:47 -0400
M doc/src/sgml/release-14.sgml
Fix issues in pg_stat_wal.
commit : d8735b8b4651f5ed50afc472e236a8e6120f07f2
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 19 May 2021 11:38:34 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 19 May 2021 11:38:34 +0900
1) Previously there were both pgstat_send_wal() and pgstat_report_wal()
in order to send WAL activity to the stats collector. With the former being
used by wal writer, the latter by most other processes. They were a bit
redundant and so this commit merges them into pgstat_send_wal() to
simplify the code.
2) Previously WAL global statistics counters were calculated and then
compared with zero-filled buffer in order to determine whether any WAL
activity has happened since the last submission. These calculation and
comparison were not cheap. This was regularly exercised even in read-only
workloads. This commit fixes the issue by making some WAL activity
counters directly be checked to determine if there's WAL activity stats
to send.
3) Previously pgstat_report_stat() did not check if there's WAL activity
stats to send as part of the "Don't expend a clock check if nothing to do"
check at the top. It's probably rare to have pending WAL stats without
also passing one of the other conditions, but for safely this commit
changes pgstat_report_stats() so that it checks also some WAL activity
counters at the top.
This commit also adds the comments about the design of WAL stats.
Reported-by: Andres Freund
Author: Masahiro Ikeda
Reviewed-by: Kyotaro Horiguchi, Atsushi Torikoshi, Andres Freund, Fujii Masao
Discussion: https://postgr.es/m/20210324232224.vrfiij2rxxwqqjjb@alap3.anarazel.de
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/pgstat.c
M src/include/executor/instrument.h
M src/include/pgstat.h
Add --no-toast-compression to pg_dumpall
commit : 694da1983e9569b2a2f96cd786ead6b8dba31f1d
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 19 May 2021 09:38:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 19 May 2021 09:38:48 +0900
This is an oversight from bbe0a81d, where the equivalent option exists
in pg_dump. This is useful to be able to reset the compression methods
cluster-wide when restoring the data based on default_toast_compression.
Reviewed-by: Daniel Gustafsson, Tom Lane
Discussion: https://postgr.es/m/YKHC+qCJvzCRVCpY@paquier.xyz
M doc/src/sgml/ref/pg_dumpall.sgml
M src/bin/pg_dump/pg_dumpall.c
doc: add PG 14 rel item about vacuum_cleanup_index_scale_factor
commit : 2e7c17837064297f25c427d58154dce8d4287302
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 18 May 2021 15:17:44 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 18 May 2021 15:17:44 -0400
M doc/src/sgml/release-14.sgml
Fix typo and outdated information in README.barrier
commit : 2ded19fa3a4dafbae80245710fa371d5163bdad4
author : David Rowley <drowley@postgresql.org>
date : Tue, 18 May 2021 09:54:56 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 18 May 2021 09:54:56 +1200
README.barrier didn't seem to get the memo when atomics were added. Fix
that.
Author: Tatsuo Ishii, David Rowley
Discussion: https://postgr.es/m/20210516.211133.2159010194908437625.t-ishii%40sraoss.co.jp
Backpatch-through: 9.6, oldest supported release
M src/backend/storage/lmgr/README.barrier
Stamp 14beta1.
commit : e4f9737fac77a5cb03a84d1f4038d300ffd28afd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 May 2021 16:11:18 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 May 2021 16:11:18 -0400
M configure
M configure.ac
Remove obsolete reference to winflex download
commit : cff8436f19e1c0c278f1ee96d450507fbd43f9ef
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 17 May 2021 21:54:36 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 17 May 2021 21:54:36 +0200
We used to distribute a binary version of flex for windows on our
download site, but it hasn't been working for many years. The "old
documentation" referenced was also for versions that have been EOL for
many years. So, remove it.
Discussion: https://postgr.es/m/CABUevEwXLJpVpab62f7AFXNWQ5=U0kvErCLq4VEsikidLyzSQg@mail.gmail.com
M doc/src/sgml/install-windows.sgml
doc: PG 14 relnotes adjustments from Fujii Masao
commit : fe2fb9ebcae8445fdb3915ecf8402a3a887effc2
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 17 May 2021 14:05:05 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 17 May 2021 14:05:05 -0400
M doc/src/sgml/release-14.sgml
Translation updates
commit : 6292b83074243db94df89271842bda0877cbc4ce
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 17 May 2021 14:30:27 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 17 May 2021 14:30:27 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 9bbd9c3714d0c76daaa806588b1fbf744aa60496
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/bin/initdb/po/es.po
M src/bin/pg_amcheck/nls.mk
A src/bin/pg_amcheck/po/de.po
M src/bin/pg_archivecleanup/nls.mk
A src/bin/pg_archivecleanup/po/el.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_checksums/nls.mk
A src/bin/pg_checksums/po/el.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_config/nls.mk
A src/bin/pg_config/po/el.po
M src/bin/pg_config/po/es.po
M src/bin/pg_ctl/po/es.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_test_fsync/nls.mk
A src/bin/pg_test_fsync/po/el.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_timing/nls.mk
A src/bin/pg_test_timing/po/el.po
M src/bin/pg_upgrade/po/de.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/fr.po
M src/bin/pg_verifybackup/po/es.po
M src/bin/psql/po/es.po
M src/bin/scripts/po/es.po
M src/interfaces/ecpg/preproc/po/de.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/libpq/po/es.po
M src/interfaces/libpq/po/fr.po
M src/pl/plpgsql/src/po/es.po
M src/pl/tcl/nls.mk
A src/pl/tcl/po/el.po
Fix wording in description of pg_stat_statements.toplevel
commit : f9e6d00df029144fd8f4ec70c52b5a1d2444f895
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 17 May 2021 10:59:54 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 17 May 2021 10:59:54 +0200
Incorrect wording got applied in 7531fcb1fcf.
Reported-By: Fujii Masao
Discussion: https://postgr.es/m/e5512912-eac9-b163-df2b-e2601ce06d27@oss.nttdata.com
M doc/src/sgml/pgstatstatements.sgml
Doc: Update documentation for asynchronous execution.
commit : 15fcd33e0694428d0567a6796891b759bc91e6f9
author : Etsuro Fujita <efujita@postgresql.org>
date : Mon, 17 May 2021 17:30:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Mon, 17 May 2021 17:30:00 +0900
Add a note of caution on the performance of asynchronous execution by
postgres_fdw. Follow-up for commit 27e1f1456.
Stephen Frost, a little bit expanded by me.
Discussion: https://postgr.es/m/20210506171224.GV20766%40tamriel.snowman.net
M doc/src/sgml/postgres-fdw.sgml
doc: update PG 14 relnotes from feedback by Tom, Alvaro, Julien
commit : 07af57dbad589bbef9d7178d9b1cb354412e823f
author : Bruce Momjian <bruce@momjian.us>
date : Sun, 16 May 2021 23:34:50 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sun, 16 May 2021 23:34:50 -0400
M doc/src/sgml/release-14.sgml
doc: remove XML comments around compute_query_id PG14 rel text
commit : f39b21e6a25c7269f50a709aa874e321e6f84b20
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 15 May 2021 17:30:45 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 15 May 2021 17:30:45 -0400
M doc/src/sgml/release-14.sgml
doc: update PG 14 release notes for compute_query_id change
commit : 6cb5346cb15d56e6ba8288b891c7098f0aecdadc
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 15 May 2021 17:26:26 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 15 May 2021 17:26:26 -0400
Also remove ALTER TYPE ...SUBSCRIPT, and update for all current commits.
M doc/src/sgml/release-14.sgml
Unbreak EXEC_BACKEND build
commit : 354f32d01dedc2c86a05be298a62cdae9710d203
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 15 May 2021 15:17:15 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 15 May 2021 15:17:15 -0400
Per buildfarm
M src/backend/postmaster/postmaster.c
Allow compute_query_id to be set to 'auto' and make it default
commit : cafde58b337e007cb6a719f5ab4dd6459d932a39
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 15 May 2021 14:13:09 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 15 May 2021 14:13:09 -0400
Allowing only on/off meant that all either all existing configuration
guides would become obsolete if we disabled it by default, or that we
would have to accept a performance loss in the default config if we
enabled it by default. By allowing 'auto' as a middle ground, the
performance cost is only paid by those who enable pg_stat_statements and
similar modules.
I only edited the release notes to comment-out a paragraph that is now
factually wrong; further edits are probably needed to describe the
related change in more detail.
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/20210513002623.eugftm4nk2lvvks3@nol
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/pg_stat_statements.conf
M doc/src/sgml/config.sgml
M doc/src/sgml/pgstatstatements.sgml
M doc/src/sgml/release-14.sgml
M src/backend/commands/explain.c
M src/backend/parser/analyze.c
M src/backend/postmaster/postmaster.c
M src/backend/tcop/postgres.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/backend/utils/misc/queryjumble.c
M src/include/utils/guc.h
M src/include/utils/queryjumble.h
Be more careful about barriers when releasing BackgroundWorkerSlots.
commit : 30d8bad494ad1f604295033e4f4de4b8f258fe74
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 15 May 2021 12:21:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 15 May 2021 12:21:06 -0400
ForgetBackgroundWorker lacked any memory barrier at all, while
BackgroundWorkerStateChange had one but unaccountably did
additional manipulation of the slot after the barrier. AFAICS,
the rule must be that the barrier is immediately before setting
or clearing slot->in_use.
It looks like back in 9.6 when ForgetBackgroundWorker was first
written, there might have been some case for not needing a
barrier there, but I'm not very convinced of that --- the fact
that the load of bgw_notify_pid is in the caller doesn't seem
to guarantee no memory ordering problem. So patch 9.6 too.
It's likely that this doesn't fix any observable bug on Intel
hardware, but machines with weaker memory ordering rules could
have problems here.
Discussion: https://postgr.es/m/4046084.1620244003@sss.pgh.pa.us
M src/backend/postmaster/bgworker.c
Harden nbtree deduplication posting split code.
commit : 8f72bbac3e4b1d1be9598e8edb9353fa5dc48138
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 14 May 2021 15:08:02 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 14 May 2021 15:08:02 -0700
Add a defensive "can't happen" error to code that handles nbtree posting
list splits (promote an existing assertion). This avoids a segfault in
the event of an insertion of a newitem that is somehow identical to an
existing non-pivot tuple in the index. An nbtree index should never
have two index tuples with identical TIDs.
This scenario is not particular unlikely in the event of any kind of
corruption that leaves the index in an inconsistent state relative to
the heap relation that is indexed. There are two known reports of
preventable hard crashes. Doing nothing seems unacceptable given the
general expectation that nbtree will cope reasonably well with corrupt
data.
Discussion: https://postgr.es/m/CAH2-Wz=Jr_d-dOYEEmwz0-ifojVNWho01eAqewfQXgKfoe114w@mail.gmail.com
Backpatch: 13-, where nbtree deduplication was introduced.
M src/backend/access/nbtree/nbtdedup.c
Prevent infinite insertion loops in spgdoinsert().
commit : c3c35a733c77b298d3cf7e7de2eeb4aea540a631
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 May 2021 15:07:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 May 2021 15:07:34 -0400
Formerly we just relied on operator classes that assert longValuesOK
to eventually shorten the leaf value enough to fit on an index page.
That fails since the introduction of INCLUDE-column support (commit
09c1c6ab4), because the INCLUDE columns might alone take up more
than a page, meaning no amount of leaf-datum compaction will get
the job done. At least with spgtextproc.c, that leads to an infinite
loop, since spgtextproc.c won't throw an error for not being able
to shorten the leaf datum anymore.
To fix without breaking cases that would otherwise work, add logic
to spgdoinsert() to verify that the leaf tuple size is decreasing
after each "choose" step. Some opclasses might not decrease the
size on every single cycle, and in any case, alignment roundoff
of the tuple size could obscure small gains. Therefore, allow
up to 10 cycles without additional savings before throwing an
error. (Perhaps this number will need adjustment, but it seems
quite generous right now.)
As long as we've developed this logic, let's back-patch it.
The back branches don't have INCLUDE columns to worry about, but
this seems like a good defense against possible bugs in operator
classes. We already know that an infinite loop here is pretty
unpleasant, so having a defense seems to outweigh the risk of
breaking things. (Note that spgtextproc.c is actually the only
known opclass with longValuesOK support, so that this is all moot
for known non-core opclasses anyway.)
Per report from Dilip Kumar.
Discussion: https://postgr.es/m/CAFiTN-uxP_soPhVG840tRMQTBmtA_f_Y8N51G7DKYYqDh7XN-A@mail.gmail.com
M doc/src/sgml/spgist.sgml
M src/backend/access/spgist/spgdoinsert.c
M src/test/modules/spgist_name_ops/expected/spgist_name_ops.out
M src/test/modules/spgist_name_ops/sql/spgist_name_ops.sql
Fix query-cancel handling in spgdoinsert().
commit : eb7a6b9229432dcb791f4bf0c44fe97bab661134
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 May 2021 13:26:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 May 2021 13:26:55 -0400
Knowing that a buggy opclass could cause an infinite insertion loop,
spgdoinsert() intended to allow its loop to be interrupted by query
cancel. However, that never actually worked, because in iterations
after the first, we'd be holding buffer lock(s) which would cause
InterruptHoldoffCount to be positive, preventing servicing of the
interrupt.
To fix, check if an interrupt is pending, and if so fall out of
the insertion loop and service the interrupt after we've released
the buffers. If it was indeed a query cancel, that's the end of
the matter. If it was a non-canceling interrupt reason, make use
of the existing provision to retry the whole insertion. (This isn't
as wasteful as it might seem, since any upper-level index tuples we
already created should be usable in the next attempt.)
While there's no known instance of such a bug in existing release
branches, it still seems like a good idea to back-patch this to
all supported branches, since the behavior is fairly nasty if a
loop does happen --- not only is it uncancelable, but it will
quickly consume memory to the point of an OOM failure. In any
case, this code is certainly not working as intended.
Per report from Dilip Kumar.
Discussion: https://postgr.es/m/CAFiTN-uxP_soPhVG840tRMQTBmtA_f_Y8N51G7DKYYqDh7XN-A@mail.gmail.com
M src/backend/access/spgist/spgdoinsert.c
Refactor CHECK_FOR_INTERRUPTS() to add flexibility.
commit : e47f93f981ccb70b4c4c5a0966e5fa0400e11a7e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 May 2021 12:54:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 May 2021 12:54:26 -0400
Split up CHECK_FOR_INTERRUPTS() to provide an additional macro
INTERRUPTS_PENDING_CONDITION(), which just tests whether an
interrupt is pending without attempting to service it. This is
useful in situations where the caller knows that interrupts are
blocked, and would like to find out if it's worth the trouble
to unblock them.
Also add INTERRUPTS_CAN_BE_PROCESSED(), which indicates whether
CHECK_FOR_INTERRUPTS() can be relied on to clear the pending interrupt.
This commit doesn't actually add any uses of the new macros,
but a follow-on bug fix will do so. Back-patch to all supported
branches to provide infrastructure for that fix.
Alvaro Herrera and Tom Lane
Discussion: https://postgr.es/m/20210513155351.GA7848@alvherre.pgsql
M src/backend/tcop/postgres.c
M src/include/miscadmin.h
Describe (auto-)analyze behavior for partitioned tables
commit : 1b5617eb844cd2470a334c1d2eec66cf9b39c41a
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 14 May 2021 13:10:52 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 14 May 2021 13:10:52 -0400
This explains the new behavior introduced by 0827e8af70f4 as well as
preexisting.
Author: Justin Pryzby <pryzby@telsasoft.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20210423180152.GA17270@telsasoft.com
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/ref/analyze.sgml
M doc/src/sgml/ref/pg_restore.sgml
doc: update PG 14 release notes with recent feedback
commit : 5eb1b27d20670b378508391fab01a6871a86a8e9
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 14 May 2021 13:01:03 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 14 May 2021 13:01:03 -0400
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210514020141.GQ27406@telsasoft.com
M doc/src/sgml/release-14.sgml
Message style improvements
commit : 09ae329957b739dfbaf722eb5624d0a71fdff3b4
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 14 May 2021 10:21:28 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 14 May 2021 10:21:28 +0200
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_rewind/pg_rewind.c
M src/interfaces/ecpg/preproc/ecpg.trailer
doc: PG 14 release notes, reorder items by significance
commit : 521d08a21a2b1ba7038ccc815b8bccc3c9be1351
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 13 May 2021 21:16:34 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 13 May 2021 21:16:34 -0400
M doc/src/sgml/release-14.sgml
Convert misleading while loop into an if condition
commit : 6cb93beddd33d00e0ce2ee55edfa32cd2a935394
author : David Rowley <drowley@postgresql.org>
date : Fri, 14 May 2021 12:26:11 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 14 May 2021 12:26:11 +1200
This seems to be leftover from ea15e1867 and from when we used to evaluate
SRFs at each node.
Since there is an unconditional "return" at the end of the loop body, only
1 loop is ever possible, so we can just change this into an if condition.
There is no actual bug being fixed here so no back-patch. It seems fine to
just fix this anomaly in master only.
Author: Greg Nancarrow
Discussion: https://postgr.es/m/CAJcOf-d7T1q0az-D8evWXnsuBZjigT04WkV5hCAOEJQZRWy28w@mail.gmail.com
M src/backend/executor/nodeResult.c
Fix autovacuum log output heap truncation issue.
commit : fbe9b80610fe17ed27ee318bdc5ba06ed86b1a71
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 13 May 2021 16:07:17 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 13 May 2021 16:07:17 -0700
The percentage of blocks from the table value reported by autovacuum log
output (following commit 5100010ee4d) should never exceed 100% because
it describes the state of the table back when lazy_vacuum() was called.
The value could nevertheless exceed 100% in the event of heap relation
truncation. We failed to compensate for how truncation affects
rel_pages.
Fix the faulty accounting by using the original rel_pages value instead
of the current/final rel_pages value.
Reported-By: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20210423204306.5osfpkt2ggaedyvy@alap3.anarazel.de
M src/backend/access/heap/vacuumlazy.c
doc: PG 14 release notes, adjust updates/deletes on partitions
commit : b2d0c7c96711843c6e47fce71335d43127f81647
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 13 May 2021 11:45:43 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 13 May 2021 11:45:43 -0400
M doc/src/sgml/release-14.sgml
Improve documentation example for jsonpath like_regex operator
commit : 9b7286c2b394381c937559a98f35df64a92ffbac
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 13 May 2021 16:10:21 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 13 May 2021 16:10:21 +0300
Make sample like_regex match string values of the root object instead of the
whole document. The corrected example seems to represent a more relevant
use case.
Backpatch to 12, when jsonpath was introduced.
Discussion: https://postgr.es/m/13440f8b-4c1f-5875-c8e3-f3f65606af2f%40xs4all.nl
Author: Erik Rijkers
Reviewed-by: Michael Paquier, Alexander Korotkov
Backpatch-through: 12
M doc/src/sgml/func.sgml
Prevent asynchronous execution of direct foreign-table modifications.
commit : a784859f4480ceaa05a00ca35311071ca33483d1
author : Etsuro Fujita <efujita@postgresql.org>
date : Thu, 13 May 2021 20:00:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Thu, 13 May 2021 20:00:00 +0900
Commits 27e1f1456 and 86dc90056, which were independently discussed,
cause a crash when executing an inherited foreign UPDATE/DELETE query
with asynchronous execution enabled, where children of an Append node
that is the direct/indirect child of the ModifyTable node are rewritten
so as to modify foreign tables directly by postgresPlanDirectModify();
as in that case the direct modifications are executed asynchronously,
which is not currently supported by asynchronous execution. Fix by
disabling asynchronous execution of the direct modifications in that
function.
Author: Etsuro Fujita
Reviewed-by: Amit Langote
Discussion: https://postgr.es/m/CAPmGK158e9sJOfuWxfn%2B0ynrspXQU3JhNjSCbaoeSzMvnga%2Bbw%40mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
pg_amcheck: Message style and formatting improvements
commit : 5a73a9e3b5b24cf2dd90ab4a7ae3724b2c12a0cc
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 13 May 2021 08:09:53 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 13 May 2021 08:09:53 +0200
M src/bin/pg_amcheck/pg_amcheck.c
Fix tests for replication slots stats.
commit : fc69509131c33c298e39dd25d542374e86aa3295
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 13 May 2021 10:14:07 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 13 May 2021 10:14:07 +0530
Some of the tests were not considering that the slot's spill stats could be
received by the stats collector after we have reset the stats. Remove
those tests and don't check total bytes decoded and sent to output plugin
in the spilled stats test as we can send the spilled stats to the stats
collector before actually sending the changes to output plugin.
Reported-by: Tom Lane as per buildfarm
Author: Vignesh C, Sawada Masahiko
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/20210319185247.ldebgpdaxsowiflw@alap3.anarazel.de
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
doc: update PG 14 release notes based on current feedback
commit : b35f827b68dc1e761e17f621fbf17c3ecd073cb0
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 12 May 2021 23:34:35 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 12 May 2021 23:34:35 -0400
M doc/src/sgml/release-14.sgml
Make saner the tab completion of INSERT and DELETE in psql
commit : 1906cc07d90a8e58fd381dba43c1085e9231f236
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 13 May 2021 09:48:28 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 13 May 2021 09:48:28 +0900
When specified directly as DML queries, INSERT was not getting always
completed to "INSERT INTO", same for DELETE with "DELETE FROM". This
makes the completion behavior more consistent for both commands, saving
a few keystrokes.
Commands on policies, triggers, grant/revoke, etc. require only DELETE
as completion keyword.
Author: Haiying Tang
Reviewed-by: Dilip Kumar, Julien Rouhaud
Discussion: https://postgr.es/m/OS0PR01MB61135AE2B07CCD1AB8C6A0F6FB549@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/bin/psql/tab-complete.c
Rename the logical replication global "wrconn"
commit : db16c656478b815627a03bb0a31833391a733eb0
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 12 May 2021 19:13:54 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 12 May 2021 19:13:54 -0400
The worker.c global wrconn is only meant to be used by logical apply/
tablesync workers, but there are other variables with the same name. To
reduce future confusion rename the global from "wrconn" to
"LogRepWorkerWalRcvConn".
While this is just cosmetic, it seems better to backpatch it all the way
back to 10 where this code appeared, to avoid future backpatching
issues.
Author: Peter Smith <smithpb2250@gmail.com>
Discussion: https://postgr.es/m/CAHut+Pu7Jv9L2BOEx_Z0UtJxfDevQSAUW2mJqWU+CtmDrEZVAg@mail.gmail.com
M src/backend/replication/logical/launcher.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/include/replication/worker_internal.h
Double-space commands in system_constraints.sql/system_functions.sql.
commit : 7dde98728a2ef6d48ef397ee783dd130fdb34e6b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 May 2021 18:41:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 May 2021 18:41:39 -0400
Previously, any error reported by the backend while reading
system_constraints.sql would report the entire file, not just the
particular command it was working on. (Ask me how I know.) Likewise,
there were chunks of system_functions.sql that would be read as one
command, which would be annoying if anything failed there.
The issue for system_constraints.sql is an oversight in commit
dfb75e478. I didn't try to trace down where the poor formatting
in system_functions.sql started, but it's certainly contrary to
the advice at the head of that file.
M src/backend/catalog/genbki.pl
M src/backend/catalog/system_functions.sql
Doc: update bki.sgml's statements about OID ranges.
commit : 1f9b0e6938054515b8c9df545437c3d347eed683
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 May 2021 17:41:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 May 2021 17:41:07 -0400
Commit ab596105b neglected to make the docs match the code.
M doc/src/sgml/bki.sgml
Do pre-release housekeeping on catalog data.
commit : 14472442861ca95cc9158518acdedf740c4bff55
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 May 2021 13:36:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 May 2021 13:36:06 -0400
Run renumber_oids.pl to move high-numbered OIDs down, as per pre-beta
tasks specified by RELEASE_CHANGES. For reference, the command was
./renumber_oids.pl --first-mapped-oid 8000 --target-oid 6150
M src/include/catalog/catversion.h
M src/include/catalog/pg_authid.dat
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_type.dat
Initial pgindent and pgperltidy run for v14.
commit : def5b065ff22a16a80084587613599fe15627213
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 May 2021 13:14:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 May 2021 13:14:10 -0400
Also "make reformat-dat-files".
The only change worthy of note is that pgindent messed up the formatting
of launcher.c's struct LogicalRepWorkerId, which led me to notice that
that struct wasn't used at all anymore, so I just took it out.
M contrib/amcheck/t/001_verify_heapam.pl
M contrib/amcheck/verify_heapam.c
M contrib/amcheck/verify_nbtree.c
M contrib/old_snapshot/time_mapping.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_bloom.c
M src/backend/access/brin/brin_minmax_multi.c
M src/backend/access/brin/brin_revmap.c
M src/backend/access/brin/brin_tuple.c
M src/backend/access/common/indextuple.c
M src/backend/access/common/toast_compression.c
M src/backend/access/common/toast_internals.c
M src/backend/access/common/tupconvert.c
M src/backend/access/gist/gistproc.c
M src/backend/access/gist/gistvalidate.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/heap/heapam_visibility.c
M src/backend/access/heap/hio.c
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/index/genam.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/varsup.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogfuncs.c
M src/backend/access/transam/xloginsert.c
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/Catalog.pm
M src/backend/catalog/aclchk.c
M src/backend/catalog/dependency.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/index.c
M src/backend/catalog/objectaddress.c
M src/backend/catalog/pg_inherits.c
M src/backend/catalog/pg_proc.c
M src/backend/catalog/pg_shdepend.c
M src/backend/catalog/pg_subscription.c
M src/backend/catalog/toasting.c
M src/backend/commands/analyze.c
M src/backend/commands/copyto.c
M src/backend/commands/explain.c
M src/backend/commands/extension.c
M src/backend/commands/indexcmds.c
M src/backend/commands/subscriptioncmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/commands/typecmds.c
M src/backend/commands/vacuum.c
M src/backend/executor/execAmi.c
M src/backend/executor/execAsync.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeGather.c
M src/backend/executor/nodeGatherMerge.c
M src/backend/executor/nodeIncrementalSort.c
M src/backend/executor/nodeModifyTable.c
M src/backend/jit/llvm/llvmjit.c
M src/backend/libpq/auth.c
M src/backend/libpq/be-secure-openssl.c
M src/backend/libpq/pqcomm.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/prep/preptlist.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/analyze.c
M src/backend/parser/parse_agg.c
M src/backend/parser/parse_cte.c
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_utilcmd.c
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partdesc.c
M src/backend/port/win32_shmem.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/postmaster/syslogger.c
M src/backend/replication/basebackup.c
M src/backend/replication/logical/launcher.c
M src/backend/replication/logical/origin.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/syncrep.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walsender.c
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/file/fd.c
M src/backend/storage/file/sharedfileset.c
M src/backend/storage/ipc/latch.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/procsignal.c
M src/backend/storage/ipc/signalfuncs.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/proc.c
M src/backend/storage/lmgr/spin.c
M src/backend/storage/page/bufpage.c
M src/backend/storage/sync/sync.c
M src/backend/tcop/postgres.c
M src/backend/utils/activity/backend_progress.c
M src/backend/utils/activity/backend_status.c
M src/backend/utils/activity/wait_event.c
M src/backend/utils/adt/acl.c
M src/backend/utils/adt/dbsize.c
M src/backend/utils/adt/genfile.c
M src/backend/utils/adt/lockfuncs.c
M src/backend/utils/adt/mcxtfuncs.c
M src/backend/utils/adt/name.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/rangetypes_typanalyze.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/adt/rowtypes.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/adt/selfuncs.c
M src/backend/utils/adt/timestamp.c
M src/backend/utils/adt/varlena.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/error/elog.c
M src/backend/utils/mb/Unicode/convutils.pm
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/queryjumble.c
M src/backend/utils/sort/logtape.c
M src/backend/utils/time/snapmgr.c
M src/bin/pg_amcheck/pg_amcheck.c
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/bin/pg_amcheck/t/005_opclass_damage.pl
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_tar.c
M src/bin/pg_dump/t/010_dump_connstr.pl
M src/bin/pg_rewind/t/001_basic.pl
M src/bin/pg_rewind/t/003_extrafiles.pl
M src/bin/pg_rewind/t/008_min_recovery_point.pl
M src/bin/pg_rewind/t/RewindTest.pm
M src/bin/pg_test_fsync/pg_test_fsync.c
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/exec.c
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/psql/tab-complete.c
M src/include/access/brin_tuple.h
M src/include/access/commit_ts.h
M src/include/access/nbtree.h
M src/include/access/relscan.h
M src/include/access/toast_compression.h
M src/include/access/transam.h
M src/include/access/xact.h
M src/include/catalog/pg_aggregate.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_type.dat
M src/include/catalog/renumber_oids.pl
M src/include/commands/copy.h
M src/include/commands/copyfrom_internal.h
M src/include/commands/defrem.h
M src/include/executor/execAsync.h
M src/include/executor/functions.h
M src/include/foreign/fdwapi.h
M src/include/lib/sort_template.h
M src/include/nodes/execnodes.h
M src/include/nodes/parsenodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/pg_config_manual.h
M src/include/pgstat.h
M src/include/storage/fd.h
M src/include/storage/proc.h
M src/include/utils/backend_progress.h
M src/include/utils/backend_status.h
M src/include/utils/builtins.h
M src/include/utils/selfuncs.h
M src/include/utils/wait_event.h
M src/interfaces/ecpg/preproc/ecpg.c
M src/interfaces/ecpg/preproc/parse.pl
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/fe-trace.c
M src/port/preadv.c
M src/test/authentication/t/001_password.pl
M src/test/kerberos/t/001_auth.pl
M src/test/modules/commit_ts/t/002_standby.pl
M src/test/modules/commit_ts/t/003_standby_2.pl
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
M src/test/perl/PostgresNode.pm
M src/test/perl/PostgresVersion.pm
M src/test/perl/TestLib.pm
M src/test/recovery/t/001_stream_rep.pl
M src/test/recovery/t/003_recovery_targets.pl
M src/test/recovery/t/007_sync_rep.pl
M src/test/recovery/t/011_crash_recovery.pl
M src/test/recovery/t/021_row_visibility.pl
M src/test/recovery/t/022_crash_temp_files.pl
M src/test/recovery/t/024_archive_recovery.pl
M src/test/regress/regress.c
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/SSLServer.pm
M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/004_sync.pl
M src/test/subscription/t/010_truncate.pl
M src/test/subscription/t/015_stream.pl
M src/test/subscription/t/016_stream_subxact.pl
M src/test/subscription/t/017_stream_ddl.pl
M src/test/subscription/t/018_stream_subxact_abort.pl
M src/test/subscription/t/019_stream_subxact_ddl_abort.pl
M src/test/subscription/t/020_messages.pl
M src/test/subscription/t/100_bugs.pl
M src/tools/msvc/Install.pm
M src/tools/msvc/Mkvcbuild.pm
M src/tools/msvc/Solution.pm
M src/tools/msvc/vcregress.pl
M src/tools/pgindent/typedefs.list
Simplify one use of ScanKey in pg_subscription.c
commit : e6ccd1ce1644d1b40b7981f8bc172394de524f99
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 12 May 2021 14:54:02 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 12 May 2021 14:54:02 +0900
The section of the code in charge of returning all the relations
associated to a subscription only need one ScanKey, but allocated two of
them. This code was introduced as a copy-paste from a different area on
the same file by 7c4f524, making the result confusing to follow.
Author: Peter Smith
Reviewed-by: Tom Lane, Julien Rouhaud, Bharath Rupireddy
Discussion: https://postgr.es/m/CAHut+PsLKe+rN3FjchoJsd76rx2aMsFTB7CTFxRgUP05p=kcpQ@mail.gmail.com
M src/backend/catalog/pg_subscription.c
Refactor some error messages for easier translation
commit : ec6e70c79fffe9292402ee602d3742a8c7d31bd2
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 12 May 2021 07:20:10 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 12 May 2021 07:20:10 +0200
M src/backend/access/common/toast_compression.c
M src/backend/parser/parse_coerce.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/xml.c
Fix EXPLAIN ANALYZE for async-capable nodes.
commit : a363bc6da96b14d27e1cae1bae97242eb6ade5e6
author : Etsuro Fujita <efujita@postgresql.org>
date : Wed, 12 May 2021 14:00:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Wed, 12 May 2021 14:00:00 +0900
EXPLAIN ANALYZE for an async-capable ForeignScan node associated with
postgres_fdw is done just by using instrumentation for ExecProcNode()
called from the node's callbacks, causing the following problems:
1) If the remote table to scan is empty, the node is incorrectly
considered as "never executed" by the command even if the node is
executed, as ExecProcNode() isn't called from the node's callbacks at
all in that case.
2) The command fails to collect timings for things other than
ExecProcNode() done in the node, such as creating a cursor for the
node's remote query.
To fix these problems, add instrumentation for async-capable nodes, and
modify postgres_fdw accordingly.
My oversight in commit 27e1f1456.
While at it, update a comment for the AsyncRequest struct in execnodes.h
and the documentation for the ForeignAsyncRequest API in fdwhandler.sgml
to match the code in ExecAsyncAppendResponse() in nodeAppend.c, and fix
typos in comments in nodeAppend.c.
Per report from Andrey Lepikhov, though I didn't use his patch.
Reviewed-by: Andrey Lepikhov
Discussion: https://postgr.es/m/2eb662bb-105d-fc20-7412-2f027cc3ca72%40postgrespro.ru
M contrib/auto_explain/auto_explain.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/fdwhandler.sgml
M src/backend/executor/execAsync.c
M src/backend/executor/execMain.c
M src/backend/executor/execProcnode.c
M src/backend/executor/instrument.c
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeForeignscan.c
M src/include/executor/instrument.h
M src/include/nodes/execnodes.h
Reduce runtime of privileges.sql test under CLOBBER_CACHE_ALWAYS.
commit : e135743ef07ea59088d09c459f41ee2eaabe95c3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 May 2021 20:59:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 May 2021 20:59:45 -0400
Several queries in the privileges regression test cause the planner
to apply the plpgsql function "leak()" to every element of the
histogram for atest12.b. Since commit 0c882e52a increased the size
of that histogram to 10000 entries, the test invokes that function
over 100000 times, which takes an absolutely unreasonable amount of
time in clobber-cache-always mode.
However, there's no real reason why that has to be a plpgsql
function: for the purposes of this test, all that matters is that
it not be marked leakproof. So we can replace the plpgsql
implementation with a direct call of int4lt, which has the same
behavior and is orders of magnitude faster. This is expected to
cut several hours off the buildfarm cycle time for CCA animals.
It has some positive impact in normal builds too, though that's
probably lost in the noise.
Back-patch to v13 where 0c882e52a came in.
Discussion: https://postgr.es/m/575884.1620626638@sss.pgh.pa.us
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Change data type of counters in BufferUsage and WalUsage from long to int64.
commit : d780d7c0882fe9a385102b292907baaceb505ed0
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 12 May 2021 09:56:34 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 12 May 2021 09:56:34 +0900
Previously long was used as the data type for some counters in BufferUsage
and WalUsage. But long is only four byte, e.g., on Windows, and it's entirely
possible to wrap a four byte counter. For example, emitting more than
four billion WAL records in one transaction isn't actually particularly rare.
To avoid the overflows of those counters, this commit changes the data type
of them from long to int64.
Suggested-by: Andres Freund
Author: Masahiro Ikeda
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/20201221211650.k7b53tcnadrciqjo@alap3.anarazel.de
Discussion: https://postgr.es/m/af0964ac-7080-1984-dc23-513754987716@oss.nttdata.com
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/explain.c
M src/include/executor/instrument.h
Tweak generation of Gen_dummy_probes.pl
commit : 0bf62931cae0db1294937eb9190b183494af4cf8
author : Andrew Dunstan <andrew@dunslane.net>
date : Tue, 11 May 2021 20:02:02 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Tue, 11 May 2021 20:02:02 -0400
Use a static prolog file instead of generating the prolog from the
existing perl script. Also, support generation of the file in a vpath
build.
Discussion: https://postgr.es/m/700620.1620662868@sss.pgh.pa.us
A src/backend/utils/Gen_dummy_probes.pl.prolog
M src/backend/utils/Makefile
M src/backend/utils/README.Gen_dummy_probes
Fix vcregress.pl's ancient misspelling of --max-connections.
commit : 0b85fa93e4575183aa5a71ebe3c6bae8d97704ed
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 May 2021 19:17:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 May 2021 19:17:07 -0400
I copied the existing spelling of "--max_connections", but
that's just wrong :-(. Evidently setting $ENV{MAX_CONNECTIONS}
has never actually worked in this script. Given the lack of
complaints, it's probably not worth back-patching a fix.
Per buildfarm.
Discussion: https://postgr.es/m/899209.1620759506@sss.pgh.pa.us
M src/tools/msvc/vcregress.pl
Get rid of the separate serial_schedule list of tests.
commit : 1df3555acc78dedc3ca25eb5e83649b3da1f298f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 May 2021 17:52:04 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 May 2021 17:52:04 -0400
Having to maintain two lists of regression test scripts has proven
annoyingly error-prone. We can achieve the effect of the
serial_schedule by running the parallel_schedule with
"--max_connections=1"; so do that and remove serial_schedule.
This causes cosmetic differences in the progress output, but it
doesn't seem worth restructuring pg_regress to avoid that.
Discussion: https://postgr.es/m/899209.1620759506@sss.pgh.pa.us
M src/test/regress/GNUmakefile
D src/test/regress/serial_schedule
M src/tools/msvc/vcregress.pl
doc: update PG 14 release notes based on feedback
commit : 5b2d09beaffa915edd6e74fcf030b13844d3326f
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 11 May 2021 17:40:44 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 11 May 2021 17:40:44 -0400
M doc/src/sgml/release-14.sgml
Replace opr_sanity test's binary_coercible() function with C code.
commit : 6303a5730914dfe6ef2709b28b225553315c573c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 May 2021 14:28:11 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 11 May 2021 14:28:11 -0400
opr_sanity's binary_coercible() function has always been meant
to match the parser's notion of binary coercibility, but it also
has always been a rather poor approximation of the parser's
real rules (as embodied in IsBinaryCoercible()). That hasn't
bit us so far, but it's predictable that it will eventually.
It also now emerges that implementing this check in plpgsql
performs absolutely horribly in clobber-cache-always testing.
(Perhaps we could do something about that, but I suspect it just
means that plpgsql is exploiting catalog caching to the hilt.)
Hence, let's replace binary_coercible() with a C shim that directly
invokes IsBinaryCoercible(), eliminating both the semantic hazard
and the performance issue.
Most of regress.c's C functions are declared in create_function_1,
but we can't simply move that to before opr_sanity/type_sanity
since those tests would complain about the resulting shell types.
I chose to split it into create_function_0 and create_function_1.
Since create_function_0 now runs as part of a parallel group while
create_function_1 doesn't, reduce the latter to create just those
functions that opr_sanity and type_sanity would whine about.
To make room for create_function_0 in the second parallel group
of tests, move tstypes to the third parallel group.
In passing, clean up some ordering deviations between
parallel_schedule and serial_schedule.
Discussion: https://postgr.es/m/292305.1620503097@sss.pgh.pa.us
M src/test/regress/expected/.gitignore
M src/test/regress/expected/conversion.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/expected/type_sanity.out
A src/test/regress/input/create_function_0.source
M src/test/regress/input/create_function_1.source
A src/test/regress/output/create_function_0.source
M src/test/regress/output/create_function_1.source
M src/test/regress/parallel_schedule
M src/test/regress/regress.c
M src/test/regress/serial_schedule
M src/test/regress/sql/.gitignore
M src/test/regress/sql/conversion.sql
M src/test/regress/sql/opr_sanity.sql
M src/test/regress/sql/type_sanity.sql
Fix typo
commit : 6d177e2813a2b4415539e2861b595583cc1a8f71
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 May 2021 09:06:49 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 11 May 2021 09:06:49 +0200
M src/backend/parser/parse_cte.c
M src/test/regress/expected/with.out
doc: update PG 14 release notes based on feedback so far
commit : ff51679220ce31091bfdbc96d2e90fc02ac6f329
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 10 May 2021 23:56:32 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 10 May 2021 23:56:32 -0400
M doc/src/sgml/release-14.sgml
Doc: Remove outdated note about run-time partition pruning
commit : 1692d0c3a3fc7716d7d00e0d657248cb98bf4df8
author : David Rowley <drowley@postgresql.org>
date : Tue, 11 May 2021 15:55:33 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 11 May 2021 15:55:33 +1200
The note is no longer true as of 86dc90056, so remove it.
Author: Amit Langote
Discussion: https://postgr.es/m/CA+HiwqFxQn7Hz1wT+wYgnf_9SK0c4BwOOwFFT8jcSZwJrd8HEA@mail.gmail.com
M doc/src/sgml/ddl.sgml
Add support for LZ4 build in MSVC scripts
commit : 9ca40dcd4d0cad43d95a9a253fafaa9a9ba7de24
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 11 May 2021 10:43:05 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 11 May 2021 10:43:05 +0900
Since its introduction in bbe0a81, compression of table data supports
LZ4, but nothing had been done within the MSVC scripts to allow users to
build the code with this library.
This commit closes the gap by extending the MSVC scripts to be able to
build optionally with LZ4. Getting libraries that can be used for
compilation and execution is possible as LZ4 can be compiled down to
MSVC 2010 using its source tarball. MinGW may require extra efforts to
be able to work, and I have been able to test this only with MSVC, still
this is better than nothing to give users a way to test the feature on
Windows.
Author: Dilip Kumar
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/YJPdNeF68XpwDDki@paquier.xyz
M doc/src/sgml/install-windows.sgml
M src/tools/msvc/Solution.pm
M src/tools/msvc/config_default.pl
Fix mishandling of resjunk columns in ON CONFLICT ... UPDATE tlists.
commit : 049e1e2edb06854d7cd9460c22516efaa165fbf8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 May 2021 11:02:29 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 May 2021 11:02:29 -0400
It's unusual to have any resjunk columns in an ON CONFLICT ... UPDATE
list, but it can happen when MULTIEXPR_SUBLINK SubPlans are present.
If it happens, the ON CONFLICT UPDATE code path would end up storing
tuples that include the values of the extra resjunk columns. That's
fairly harmless in the short run, but if new columns are added to
the table then the values would become accessible, possibly leading
to malfunctions if they don't match the datatypes of the new columns.
This had escaped notice through a confluence of missing sanity checks,
including
* There's no cross-check that a tuple presented to heap_insert or
heap_update matches the table rowtype. While it's difficult to
check that fully at reasonable cost, we can easily add assertions
that there aren't too many columns.
* The output-column-assignment cases in execExprInterp.c lacked
any sanity checks on the output column numbers, which seems like
an oversight considering there are plenty of assertion checks on
input column numbers. Add assertions there too.
* We failed to apply nodeModifyTable's ExecCheckPlanOutput() to
the ON CONFLICT UPDATE tlist. That wouldn't have caught this
specific error, since that function is chartered to ignore resjunk
columns; but it sure seems like a bad omission now that we've seen
this bug.
In HEAD, the right way to fix this is to make the processing of
ON CONFLICT UPDATE tlists work the same as regular UPDATE tlists
now do, that is don't add "SET x = x" entries, and use
ExecBuildUpdateProjection to evaluate the tlist and combine it with
old values of the not-set columns. This adds a little complication
to ExecBuildUpdateProjection, but allows removal of a comparable
amount of now-dead code from the planner.
In the back branches, the most expedient solution seems to be to
(a) use an output slot for the ON CONFLICT UPDATE projection that
actually matches the target table, and then (b) invent a variant of
ExecBuildProjectionInfo that can be told to not store values resulting
from resjunk columns, so it doesn't try to store into nonexistent
columns of the output slot. (We can't simply ignore the resjunk columns
altogether; they have to be evaluated for MULTIEXPR_SUBLINK to work.)
This works back to v10. In 9.6, projections work much differently and
we can't cheaply give them such an option. The 9.6 version of this
patch works by inserting a JunkFilter when it's necessary to get rid
of resjunk columns.
In addition, v11 and up have the reverse problem when trying to
perform ON CONFLICT UPDATE on a partitioned table. Through a
further oversight, adjust_partition_tlist() discarded resjunk columns
when re-ordering the ON CONFLICT UPDATE tlist to match a partition.
This accidentally prevented the storing-bogus-tuples problem, but
at the cost that MULTIEXPR_SUBLINK cases didn't work, typically
crashing if more than one row has to be updated. Fix by preserving
resjunk columns in that routine. (I failed to resist the temptation
to add more assertions there too, and to do some minor code
beautification.)
Per report from Andres Freund. Back-patch to all supported branches.
Security: CVE-2021-32028
M src/backend/access/heap/heapam.c
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/prep/preptlist.c
M src/include/executor/executor.h
M src/include/nodes/plannodes.h
M src/include/optimizer/prep.h
M src/test/regress/expected/update.out
M src/test/regress/sql/update.sql
Prevent integer overflows in array subscripting calculations.
commit : f02b9085ad2f6fefd9c5cdf85579cb9f0ff0f0ea
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 May 2021 10:44:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 May 2021 10:44:38 -0400
While we were (mostly) careful about ensuring that the dimensions of
arrays aren't large enough to cause integer overflow, the lower bound
values were generally not checked. This allows situations where
lower_bound + dimension overflows an integer. It seems that that's
harmless so far as array reading is concerned, except that array
elements with subscripts notionally exceeding INT_MAX are inaccessible.
However, it confuses various array-assignment logic, resulting in a
potential for memory stomps.
Fix by adding checks that array lower bounds aren't large enough to
cause lower_bound + dimension to overflow. (Note: this results in
disallowing cases where the last subscript position would be exactly
INT_MAX. In principle we could probably allow that, but there's a lot
of code that computes lower_bound + dimension and would need adjustment.
It seems doubtful that it's worth the trouble/risk to allow it.)
Somewhat independently of that, array_set_element() was careless
about possible overflow when checking the subscript of a fixed-length
array, creating a different route to memory stomps. Fix that too.
Security: CVE-2021-32027
M src/backend/executor/execExprInterp.c
M src/backend/utils/adt/array_userfuncs.c
M src/backend/utils/adt/arrayfuncs.c
M src/backend/utils/adt/arrayutils.c
M src/include/utils/array.h
Translation updates
commit : 6206454bdac1ccd6f6ed9d811e1a1139e663a8b9
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 May 2021 14:36:21 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 May 2021 14:36:21 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 1c361d3ac016b61715d99f2055dee050397e3f13
M src/backend/nls.mk
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/backend/po/ja.po
M src/backend/po/ko.po
M src/backend/po/ru.po
M src/backend/po/sv.po
A src/backend/po/uk.po
M src/bin/initdb/nls.mk
M src/bin/initdb/po/cs.po
M src/bin/initdb/po/de.po
A src/bin/initdb/po/el.po
M src/bin/initdb/po/es.po
M src/bin/initdb/po/fr.po
M src/bin/initdb/po/ja.po
M src/bin/initdb/po/ko.po
M src/bin/initdb/po/ru.po
M src/bin/initdb/po/uk.po
M src/bin/pg_amcheck/nls.mk
A src/bin/pg_amcheck/po/fr.po
M src/bin/pg_archivecleanup/po/cs.po
M src/bin/pg_archivecleanup/po/es.po
M src/bin/pg_archivecleanup/po/fr.po
M src/bin/pg_archivecleanup/po/ja.po
M src/bin/pg_archivecleanup/po/ko.po
M src/bin/pg_archivecleanup/po/ru.po
M src/bin/pg_archivecleanup/po/uk.po
M src/bin/pg_basebackup/nls.mk
M src/bin/pg_basebackup/po/cs.po
M src/bin/pg_basebackup/po/de.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_basebackup/po/fr.po
M src/bin/pg_basebackup/po/ja.po
M src/bin/pg_basebackup/po/ko.po
M src/bin/pg_basebackup/po/ru.po
A src/bin/pg_basebackup/po/uk.po
M src/bin/pg_checksums/nls.mk
M src/bin/pg_checksums/po/cs.po
M src/bin/pg_checksums/po/de.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_checksums/po/fr.po
M src/bin/pg_checksums/po/ja.po
M src/bin/pg_checksums/po/ko.po
M src/bin/pg_checksums/po/ru.po
A src/bin/pg_checksums/po/uk.po
A src/bin/pg_checksums/po/zh_CN.po
M src/bin/pg_config/po/cs.po
M src/bin/pg_config/po/de.po
M src/bin/pg_config/po/es.po
M src/bin/pg_config/po/fr.po
M src/bin/pg_config/po/ja.po
M src/bin/pg_config/po/ko.po
M src/bin/pg_config/po/ru.po
M src/bin/pg_config/po/uk.po
M src/bin/pg_controldata/po/cs.po
M src/bin/pg_controldata/po/es.po
M src/bin/pg_controldata/po/ja.po
M src/bin/pg_controldata/po/ko.po
M src/bin/pg_controldata/po/ru.po
M src/bin/pg_controldata/po/uk.po
M src/bin/pg_ctl/nls.mk
M src/bin/pg_ctl/po/cs.po
M src/bin/pg_ctl/po/de.po
A src/bin/pg_ctl/po/el.po
M src/bin/pg_ctl/po/es.po
M src/bin/pg_ctl/po/fr.po
M src/bin/pg_ctl/po/ja.po
M src/bin/pg_ctl/po/ko.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_ctl/po/uk.po
M src/bin/pg_dump/nls.mk
M src/bin/pg_dump/po/cs.po
M src/bin/pg_dump/po/de.po
A src/bin/pg_dump/po/el.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_dump/po/ja.po
M src/bin/pg_dump/po/ko.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/sv.po
A src/bin/pg_dump/po/uk.po
M src/bin/pg_resetwal/nls.mk
M src/bin/pg_resetwal/po/cs.po
M src/bin/pg_resetwal/po/es.po
M src/bin/pg_resetwal/po/ja.po
M src/bin/pg_resetwal/po/ko.po
M src/bin/pg_resetwal/po/ru.po
A src/bin/pg_resetwal/po/uk.po
M src/bin/pg_rewind/nls.mk
M src/bin/pg_rewind/po/cs.po
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/ja.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_rewind/po/sv.po
A src/bin/pg_rewind/po/uk.po
M src/bin/pg_test_fsync/po/de.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_fsync/po/fr.po
M src/bin/pg_test_fsync/po/ru.po
M src/bin/pg_test_fsync/po/uk.po
M src/bin/pg_test_timing/po/de.po
M src/bin/pg_test_timing/po/fr.po
M src/bin/pg_upgrade/nls.mk
M src/bin/pg_upgrade/po/cs.po
M src/bin/pg_upgrade/po/de.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/fr.po
M src/bin/pg_upgrade/po/ja.po
M src/bin/pg_upgrade/po/ko.po
M src/bin/pg_upgrade/po/ru.po
M src/bin/pg_upgrade/po/sv.po
A src/bin/pg_upgrade/po/uk.po
M src/bin/pg_verifybackup/nls.mk
A src/bin/pg_verifybackup/po/de.po
A src/bin/pg_verifybackup/po/es.po
M src/bin/pg_verifybackup/po/fr.po
A src/bin/pg_verifybackup/po/ja.po
A src/bin/pg_verifybackup/po/ko.po
A src/bin/pg_verifybackup/po/ru.po
M src/bin/pg_verifybackup/po/sv.po
A src/bin/pg_verifybackup/po/uk.po
A src/bin/pg_verifybackup/po/zh_CN.po
M src/bin/pg_waldump/nls.mk
M src/bin/pg_waldump/po/cs.po
M src/bin/pg_waldump/po/es.po
M src/bin/pg_waldump/po/ja.po
M src/bin/pg_waldump/po/ru.po
A src/bin/pg_waldump/po/uk.po
M src/bin/pg_waldump/po/zh_CN.po
M src/bin/psql/nls.mk
M src/bin/psql/po/cs.po
M src/bin/psql/po/de.po
A src/bin/psql/po/el.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ja.po
M src/bin/psql/po/ko.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/sv.po
M src/bin/psql/po/uk.po
M src/bin/scripts/nls.mk
M src/bin/scripts/po/cs.po
M src/bin/scripts/po/de.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/fr.po
M src/bin/scripts/po/ja.po
M src/bin/scripts/po/ko.po
M src/bin/scripts/po/ru.po
M src/bin/scripts/po/sv.po
A src/bin/scripts/po/uk.po
M src/interfaces/ecpg/ecpglib/po/es.po
M src/interfaces/ecpg/ecpglib/po/fr.po
M src/interfaces/ecpg/ecpglib/po/ru.po
M src/interfaces/ecpg/ecpglib/po/uk.po
M src/interfaces/ecpg/preproc/po/cs.po
M src/interfaces/ecpg/preproc/po/de.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/fr.po
M src/interfaces/ecpg/preproc/po/ko.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/ecpg/preproc/po/uk.po
M src/interfaces/libpq/nls.mk
M src/interfaces/libpq/po/cs.po
M src/interfaces/libpq/po/de.po
A src/interfaces/libpq/po/el.po
M src/interfaces/libpq/po/es.po
M src/interfaces/libpq/po/fr.po
M src/interfaces/libpq/po/ja.po
M src/interfaces/libpq/po/ko.po
M src/interfaces/libpq/po/ru.po
M src/interfaces/libpq/po/sv.po
M src/interfaces/libpq/po/uk.po
M src/interfaces/libpq/po/zh_CN.po
M src/pl/plperl/nls.mk
M src/pl/plperl/po/es.po
M src/pl/plperl/po/ru.po
A src/pl/plperl/po/uk.po
M src/pl/plpgsql/src/po/cs.po
M src/pl/plpgsql/src/po/de.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpgsql/src/po/fr.po
M src/pl/plpgsql/src/po/ko.po
M src/pl/plpgsql/src/po/ru.po
M src/pl/plpgsql/src/po/sv.po
M src/pl/plpgsql/src/po/uk.po
M src/pl/plpython/po/es.po
M src/pl/plpython/po/ru.po
M src/pl/plpython/po/uk.po
M src/pl/tcl/po/es.po
M src/pl/tcl/po/ru.po
M src/pl/tcl/po/uk.po
Emit dummy statements for probes.d probes when disabled
commit : fa8fbadb934b4727a7aeff074995e799f4685a75
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 May 2021 11:36:26 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 May 2021 11:36:26 +0200
When building without --enable-dtrace, emit dummy
do {} while (0)
statements for the stubbed-out TRACE_POSTGRESQL_foo() macros
instead of empty macros that totally elide the original probe
statement.
This fixes the
warning: suggest braces around empty body in an ‘if’ statement [-Wempty-body]
introduced by b94409a02f.
Author: Craig Ringer <craig.ringer@2ndquadrant.com>
Discussion: https://www.postgresql.org/message-id/flat/20210504221531.cfvpmmdfsou6eitb%40alap3.anarazel.de
M src/backend/utils/Gen_dummy_probes.pl
M src/backend/utils/Gen_dummy_probes.sed
Remove unused function arguments
commit : 3c554100307f4e57c0881e205dbdbc173bb84d56
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 May 2021 10:02:33 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 May 2021 10:02:33 +0200
Was present in original commit
198b3716dba68544b55cb97bd120738a86d5df2d but apparently never used.
M src/interfaces/libpq/fe-trace.c
Fix typos in operatorcmds.c
commit : 829daab4bbe356a2f9ae0b2ee0fc98bc2279d754
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 10 May 2021 15:45:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 10 May 2021 15:45:54 +0900
Author: Kyotaro Horiguchi, Justin Pryzby
Discussion: https://postgr.es/m/20210428.173633.1525059946206239295.horikyota.ntt@gmail.com
M src/backend/commands/operatorcmds.c
doc: first draft of the PG 14 release notes
commit : dc0260861063b125d297c0f3caca359feb381c6a
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 10 May 2021 01:58:59 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 10 May 2021 01:58:59 -0400
M doc/src/sgml/release-14.sgml
Fix generation of ./INSTALL for the distribution tarball
commit : 45aa88fe1d4028ea50ba7d26d390223b6ef78acc
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 10 May 2021 14:34:07 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 10 May 2021 14:34:07 +0900
"make dist", in charge of creating a distribution tarball, failed when
attempting to generate ./INSTALL as a new reference added to
guc-default-toast-compression on the documentation for the installation
details was not getting translated properly to plain text. Like all the
other link references on this page, this adds a new entry to
standalone-profile.xsl to allow the generation of ./INSTALL to finish
properly.
Oversight in 02a93e7, per buildfarm member guaibasaurus.
M doc/src/sgml/standalone-profile.xsl
Revert recovery prefetching feature.
commit : c2dc19342e05e081dc13b296787baa38352681ef
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 10 May 2021 16:00:53 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 10 May 2021 16:00:53 +1200
This set of commits has some bugs with known fixes, but at this late
stage in the release cycle it seems best to revert and resubmit next
time, along with some new automated test coverage for this whole area.
Commits reverted:
dc88460c: Doc: Review for "Optionally prefetch referenced data in recovery."
1d257577: Optionally prefetch referenced data in recovery.
f003d9f8: Add circular WAL decoding buffer.
323cbe7c: Remove read_page callback from XLogReader.
Remove the new GUC group WAL_RECOVERY recently added by a55a9847, as the
corresponding section of config.sgml is now reverted.
Discussion: https://postgr.es/m/CAOuzzgrn7iKnFRsB4MHp3UisEQAGgZMbk_ViTN4HV4-Ksq8zCg%40mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/Makefile
M src/backend/access/transam/generic_xlog.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xlog.c
D src/backend/access/transam/xlogprefetch.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogutils.c
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/logicalfuncs.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/walsender.c
M src/backend/storage/freespace/freespace.c
M src/backend/storage/ipc/ipci.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_waldump/pg_waldump.c
M src/include/access/xlog.h
D src/include/access/xlogprefetch.h
M src/include/access/xlogreader.h
M src/include/access/xlogutils.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/replication/logical.h
M src/include/utils/guc.h
M src/include/utils/guc_tables.h
M src/test/regress/expected/rules.out
M src/tools/pgindent/typedefs.list
Add more TAP tests for pg_dump with attribute compression
commit : 63db0ac3f9e6bae313da67f640c95c0045b7f0ee
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 10 May 2021 11:12:07 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 10 May 2021 11:12:07 +0900
Some relations with LZ4 used as the toast compression methods have been
left around in the main regression test suite to stress pg_upgrade, but
pg_dump, that includes tests much more picky in terms of output
generated, had no actual coverage with this feature.
Similarly to collations, tests only working with LZ4 are tracked with an
additional flag, and this uses TestLib::check_pg_config() to check if
the build supports LZ4 or not. This stresses more scenarios with
tables, materialized views and pg_dump --no-toast-compression.
Author: Dilip Kumar
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/CAFiTN-twgPmohG7qj1HXhySq16h123y5OowsQR+5h1YeZE9fmQ@mail.gmail.com
M src/bin/pg_dump/t/002_pg_dump.pl
doc: Fix some gaps with the documentation related to LZ4
commit : 02a93e7ef9612788081ef07ea1bbd0a8cc99ae99
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 10 May 2021 09:30:35 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 10 May 2021 09:30:35 +0900
The upstream project is officially named "LZ4", and the documentation
was confused with the option value that can be used with DDLs supporting
this option, and the project name.
Documentation related to the configure option --with-lz4 was missing, so
add something for that.
Author: Dilip Kumar, Michael Paquier
Reviewed-by: Justin Pryzby
Discussion: https://postgr.es/m/YJaOZQDXBVySq+Cc@paquier.xyz
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/installation.sgml
Improve comments about USE_VALGRIND in pg_config_manual.h.
commit : 8dc3d68cbe676deb5e74d1b1b565f57fffaf107e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 May 2021 19:33:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 May 2021 19:33:24 -0400
These comments left the impression that USE_VALGRIND isn't really
essential for valgrind testing. But that's wrong, as I learned
the hard way today.
Discussion: https://postgr.es/m/512778.1620588546@sss.pgh.pa.us
M src/include/pg_config_manual.h
Move memory accounting Asserts for Result Cache code
commit : 92c4c269d24d016c19858a21347ff25a7de1f486
author : David Rowley <drowley@postgresql.org>
date : Sun, 9 May 2021 11:37:18 +1200
committer: David Rowley <drowley@postgresql.org>
date : Sun, 9 May 2021 11:37:18 +1200
In 9eacee2e6, I included some code to verify the cache's memory tracking
is correct by counting up the number of entries and the memory they use
each time we evict something from the cache. Those values are then
compared to the expected values using Assert. The problem is that this
requires looping over the entire cache hash table each time we evict an
entry from the cache. That can be pretty expensive, as noted by Pavel
Stehule.
Here we move this memory accounting checking code so that we only verify
it on cassert builds once when shutting down the Result Cache node.
Aside from the performance increase, this has two distinct advantages:
1) We do the memory checks at the last possible moment before destroying
the cache. This means we'll now catch accounting problems that might
sneak in after a cache eviction.
2) We now do the memory Assert checks when there were no cache evictions.
This increases the coverage.
One small disadvantage is that we'll now miss any memory tracking issues
that somehow managed to resolve themselves by the end of execution.
However, it seems to me that such a memory tracking problem would be quite
unlikely, and likely somewhat less harmful if one were to exist.
In passing, adjust the loop over the hash table to use the standard
simplehash.h method of iteration.
Reported-by: Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRAzgoSkdEiqrKbT=7yG9FA5fjUAP3jmJywuDqYq6Ki5ug@mail.gmail.com
M src/backend/executor/nodeResultCache.c
Sync guc.c and postgresql.conf.sample with the SGML docs.
commit : a55a98477b690dedb9b4368d7e5710c8e7fa534e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 8 May 2021 12:13:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 8 May 2021 12:13:33 -0400
It seems that various people have moved GUCs around in the config.sgml
listing without bothering to make the code agree. Ensure that the
config_group codes assigned to GUCs match where they are listed in
config.sgml. Likewise ensure that postgresql.conf.sample lists GUCs
in the same sub-section and same ordering as they appear in config.sgml.
(I've got some doubts about some of these choices, but for the purposes
of this patch, we'll treat config.sgml as gospel.)
Notably, this requires adding a WAL_RECOVERY config_group value,
because 1d257577e didn't. As long as we're renumbering that enum
anyway, let's take out the values corresponding to major groups
that are divided into sub-groups. No GUC should be assigned to the
major group itself, so those values just create a temptation to
do the wrong thing, while adding work for translators.
In passing, adjust the short_desc strings for PRESET_OPTIONS GUCs
to uniformly use the phrasing "Shows XYZ.", removing the impression
some of these strings left that you can set the value.
While some of these errors are old, no back-patch, as changing the
contents of the pg_settings view in stable branches seems more likely
to be seen as a compatibility break than anything helpful.
Bharath Rupireddy, Justin Pryzby, Tom Lane
Discussion: https://postgr.es/m/16997-ff16127f6e0d1390@postgresql.org
Discussion: https://postgr.es/m/20210413123139.GE6091@telsasoft.com
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/utils/guc_tables.h
Doc: copy-editing for debug_invalidate_system_caches_always description.
commit : f9b809e7fbe36cd3fe1ce33edb277288a31da386
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 8 May 2021 11:33:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 8 May 2021 11:33:13 -0400
I came to fix "useful only useful", but the more I looked at the text
the more things I thought could be improved.
M doc/src/sgml/config.sgml
Fix incorrect error code for CREATE/ALTER TABLE COMPRESSION
commit : 9681f2160dcbe2a02fd2e2db2322ea204eff6562
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 8 May 2021 10:18:05 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 8 May 2021 10:18:05 +0900
Specifying an incorrect value for the compression method of an attribute
caused ERRCODE_FEATURE_NOT_SUPPORTED to be raised as error. Use instead
ERRCODE_INVALID_PARAMETER_VALUE to be more consistent.
Author: Dilip Kumar
Discussion: https://postgr.es/m/CAFiTN-vH84fE-8C4zGZw4v0Wyh4Y2v=5JWg2fGE5+LPaDvz1GQ@mail.gmail.com
M src/backend/commands/tablecmds.c
Copy the INSERT query in postgres_fdw
commit : c6a01d924939306e95c8deafd09352be6a955648
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 7 May 2021 22:29:43 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 7 May 2021 22:29:43 +0200
When executing the INSERT with batching, we may need to rebuild the
query when the batch size changes, in which case we pfree the current
string. We must not release the original string, stored in fdw_private,
because that may be needed in EXPLAIN ANALYZE. So make copy of the SQL,
but only for INSERT queries.
Reported-by: Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRCL_Rjw-MCR6J7VX9OF7MR6PA5K8qUbrMvprW_e-aHkfQ%40mail.gmail.com
M contrib/postgres_fdw/postgres_fdw.c
Add a README and Makefile recipe for Gen_dummy_probes.pl
commit : 8292c0675a793a5afd0a8eedbeb0db7abfb844f3
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 7 May 2021 14:27:18 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 7 May 2021 14:27:18 -0400
Discussion: https://postgr.es/m/20210506035602.3akutfvvojngj3nb@alap3.anarazel.de
M src/backend/utils/Makefile
A src/backend/utils/README.Gen_dummy_probes
Fix typo
commit : 9f989a8581cc37879d493a5a78b0f01ec0e3245a
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 May 2021 17:47:22 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 May 2021 17:47:22 +0200
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/create_table.out
M src/test/regress/sql/create_table.sql
AlterSubscription_refresh: avoid stomping on global variable
commit : 4e8c0f1a0d0d095a749a329a216c88a340a455b6
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 7 May 2021 11:46:37 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 7 May 2021 11:46:37 -0400
This patch replaces use of the global "wrconn" variable in
AlterSubscription_refresh with a local variable of the same name, making
it consistent with other functions in subscriptioncmds.c (e.g.
DropSubscription).
The global wrconn is only meant to be used for logical apply/tablesync worker.
Abusing it this way is known to cause trouble if an apply worker
manages to do a subscription refresh, such as reported by Jeremy Finzel
and diagnosed by Andres Freund back in November 2020, at
https://www.postgresql.org/message-id/20201111215820.qihhrz7fayu6myfi@alap3.anarazel.de
Backpatch to 10. In branch master, also move the connection establishment
to occur outside the PG_TRY block; this way we can remove a test for NULL in
PG_FINALLY, and it also makes the code more consistent with similar code in
the same file.
Author: Peter Smith <peter.b.smith@fujitsu.com>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Japin Li <japinli@hotmail.com>
Discussion: https://postgr.es/m/CAHut+Pu7Jv9L2BOEx_Z0UtJxfDevQSAUW2mJqWU+CtmDrEZVAg@mail.gmail.com
M src/backend/commands/subscriptioncmds.c
Remove extraneous newlines added by perl copyright patch
commit : 8b82de0164c13eb3b113a525dc7eda7887f5238b
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 7 May 2021 11:37:37 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 7 May 2021 11:37:37 -0400
M contrib/intarray/bench/bench.pl
M contrib/intarray/bench/create_test.pl
M contrib/seg/seg-validate.pl
M contrib/seg/sort-segments.pl
M src/interfaces/libpq/test/regress.pl
M src/pl/plperl/plperl_opmask.pl
M src/test/locale/sort-test.pl
M src/test/perl/PostgresNode.pm
M src/test/perl/RecursiveCopy.pm
M src/test/perl/TestLib.pm
M src/tools/git_changelog
M src/tools/msvc/pgbison.pl
M src/tools/msvc/pgflex.pl
M src/tools/msvc/vcregress.pl
M src/tools/pginclude/pgcheckdefines
M src/tools/pgindent/pgindent
Add a copyright notice to perl files lacking one.
commit : 8fa6e6919c1aaa6f74c74e16452aaf0b5f3b4cd5
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 7 May 2021 10:56:14 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 7 May 2021 10:56:14 -0400
M contrib/amcheck/t/001_verify_heapam.pl
M contrib/auto_explain/t/001_auto_explain.pl
M contrib/bloom/t/001_wal.pl
M contrib/intarray/bench/bench.pl
M contrib/intarray/bench/create_test.pl
M contrib/oid2name/t/001_basic.pl
M contrib/seg/seg-validate.pl
M contrib/seg/sort-segments.pl
M contrib/test_decoding/t/001_repl_stats.pl
M contrib/vacuumlo/t/001_basic.pl
M src/bin/initdb/t/001_initdb.pl
M src/bin/pg_amcheck/t/001_basic.pl
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/bin/pg_amcheck/t/005_opclass_damage.pl
M src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_basebackup/t/020_pg_receivewal.pl
M src/bin/pg_basebackup/t/030_pg_recvlogical.pl
M src/bin/pg_checksums/t/001_basic.pl
M src/bin/pg_checksums/t/002_actions.pl
M src/bin/pg_config/t/001_pg_config.pl
M src/bin/pg_controldata/t/001_pg_controldata.pl
M src/bin/pg_ctl/t/001_start_stop.pl
M src/bin/pg_ctl/t/002_status.pl
M src/bin/pg_ctl/t/003_promote.pl
M src/bin/pg_ctl/t/004_logrotate.pl
M src/bin/pg_dump/t/001_basic.pl
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/pg_dump/t/003_pg_dump_with_server.pl
M src/bin/pg_dump/t/010_dump_connstr.pl
M src/bin/pg_resetwal/t/001_basic.pl
M src/bin/pg_resetwal/t/002_corrupted.pl
M src/bin/pg_rewind/t/001_basic.pl
M src/bin/pg_rewind/t/002_databases.pl
M src/bin/pg_rewind/t/003_extrafiles.pl
M src/bin/pg_rewind/t/004_pg_xlog_symlink.pl
M src/bin/pg_rewind/t/005_same_timeline.pl
M src/bin/pg_rewind/t/006_options.pl
M src/bin/pg_rewind/t/007_standby_source.pl
M src/bin/pg_rewind/t/008_min_recovery_point.pl
M src/bin/pg_rewind/t/RewindTest.pm
M src/bin/pg_test_fsync/t/001_basic.pl
M src/bin/pg_test_timing/t/001_basic.pl
M src/bin/pg_verifybackup/t/001_basic.pl
M src/bin/pg_verifybackup/t/002_algorithm.pl
M src/bin/pg_verifybackup/t/003_corruption.pl
M src/bin/pg_verifybackup/t/004_options.pl
M src/bin/pg_verifybackup/t/005_bad_manifest.pl
M src/bin/pg_verifybackup/t/006_encoding.pl
M src/bin/pg_verifybackup/t/007_wal.pl
M src/bin/pg_waldump/t/001_basic.pl
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/pgbench/t/002_pgbench_no_server.pl
M src/bin/psql/t/010_tab_completion.pl
M src/bin/scripts/t/010_clusterdb.pl
M src/bin/scripts/t/011_clusterdb_all.pl
M src/bin/scripts/t/020_createdb.pl
M src/bin/scripts/t/040_createuser.pl
M src/bin/scripts/t/050_dropdb.pl
M src/bin/scripts/t/070_dropuser.pl
M src/bin/scripts/t/080_pg_isready.pl
M src/bin/scripts/t/090_reindexdb.pl
M src/bin/scripts/t/091_reindexdb_all.pl
M src/bin/scripts/t/100_vacuumdb.pl
M src/bin/scripts/t/101_vacuumdb_all.pl
M src/bin/scripts/t/102_vacuumdb_stages.pl
M src/bin/scripts/t/200_connstr.pl
M src/interfaces/libpq/test/regress.pl
M src/pl/plperl/plc_perlboot.pl
M src/pl/plperl/plc_trusted.pl
M src/pl/plperl/plperl_opmask.pl
M src/pl/plperl/text2macro.pl
M src/test/authentication/t/001_password.pl
M src/test/authentication/t/002_saslprep.pl
M src/test/kerberos/t/001_auth.pl
M src/test/ldap/t/001_auth.pl
M src/test/locale/sort-test.pl
M src/test/modules/brin/t/01_workitems.pl
M src/test/modules/commit_ts/t/001_base.pl
M src/test/modules/commit_ts/t/002_standby.pl
M src/test/modules/commit_ts/t/003_standby_2.pl
M src/test/modules/commit_ts/t/004_restart.pl
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
M src/test/modules/ssl_passphrase_callback/t/001_testfunc.pl
M src/test/modules/test_misc/t/001_constraint_validation.pl
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/perl/PostgresNode.pm
M src/test/perl/RecursiveCopy.pm
M src/test/perl/SimpleTee.pm
M src/test/perl/TestLib.pm
M src/test/recovery/t/001_stream_rep.pl
M src/test/recovery/t/002_archiving.pl
M src/test/recovery/t/003_recovery_targets.pl
M src/test/recovery/t/004_timeline_switch.pl
M src/test/recovery/t/005_replay_delay.pl
M src/test/recovery/t/006_logical_decoding.pl
M src/test/recovery/t/007_sync_rep.pl
M src/test/recovery/t/008_fsm_truncation.pl
M src/test/recovery/t/009_twophase.pl
M src/test/recovery/t/010_logical_decoding_timelines.pl
M src/test/recovery/t/011_crash_recovery.pl
M src/test/recovery/t/012_subtransactions.pl
M src/test/recovery/t/013_crash_restart.pl
M src/test/recovery/t/014_unlogged_reinit.pl
M src/test/recovery/t/015_promotion_pages.pl
M src/test/recovery/t/016_min_consistency.pl
M src/test/recovery/t/017_shm.pl
M src/test/recovery/t/018_wal_optimize.pl
M src/test/recovery/t/019_replslot_limit.pl
M src/test/recovery/t/020_archive_status.pl
M src/test/recovery/t/021_row_visibility.pl
M src/test/recovery/t/022_crash_temp_files.pl
M src/test/recovery/t/023_pitr_prepared_xact.pl
M src/test/recovery/t/024_archive_recovery.pl
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
M src/test/ssl/t/SSLServer.pm
M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/002_types.pl
M src/test/subscription/t/003_constraints.pl
M src/test/subscription/t/004_sync.pl
M src/test/subscription/t/005_encoding.pl
M src/test/subscription/t/006_rewrite.pl
M src/test/subscription/t/007_ddl.pl
M src/test/subscription/t/008_diff_schema.pl
M src/test/subscription/t/009_matviews.pl
M src/test/subscription/t/010_truncate.pl
M src/test/subscription/t/011_generated.pl
M src/test/subscription/t/012_collation.pl
M src/test/subscription/t/013_partition.pl
M src/test/subscription/t/014_binary.pl
M src/test/subscription/t/015_stream.pl
M src/test/subscription/t/016_stream_subxact.pl
M src/test/subscription/t/017_stream_ddl.pl
M src/test/subscription/t/018_stream_subxact_abort.pl
M src/test/subscription/t/019_stream_subxact_ddl_abort.pl
M src/test/subscription/t/020_messages.pl
M src/test/subscription/t/100_bugs.pl
M src/tools/git_changelog
M src/tools/msvc/Install.pm
M src/tools/msvc/MSBuildProject.pm
M src/tools/msvc/Mkvcbuild.pm
M src/tools/msvc/Project.pm
M src/tools/msvc/Solution.pm
M src/tools/msvc/VSObjectFactory.pm
M src/tools/msvc/build.pl
M src/tools/msvc/config_default.pl
M src/tools/msvc/dummylib/Win32.pm
M src/tools/msvc/dummylib/Win32/Registry.pm
M src/tools/msvc/dummylib/Win32API/File.pm
M src/tools/msvc/gendef.pl
M src/tools/msvc/install.pl
M src/tools/msvc/mkvcbuild.pl
M src/tools/msvc/pgbison.pl
M src/tools/msvc/pgflex.pl
M src/tools/msvc/vcregress.pl
M src/tools/pginclude/pgcheckdefines
M src/tools/pgindent/pgindent
Mention statistics objects in maintenance.sgml
commit : 44f90ad092f95fe19bebb51465193bc63849c15f
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 7 May 2021 14:02:22 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 7 May 2021 14:02:22 +0200
The docs mentioned expression indexes as a way to improve selectivity
estimates for functions, but we have a second option to improve that by
creating extended statistics. So mention that too.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210505210947.GA27406%40telsasoft.com
M doc/src/sgml/maintenance.sgml
Fix typos in comments about extended statistics
commit : 93f9af138795a7d12366187de95f4961fb07ed98
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 7 May 2021 13:57:29 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 7 May 2021 13:57:29 +0200
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210505210947.GA27406%40telsasoft.com
M src/backend/parser/parse_utilcmd.c
M src/backend/statistics/extended_stats.c
Make pg_get_statisticsobjdef_expressions return NULL
commit : 8d4b311d2494ca592e30aed03b29854d864eb846
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 7 May 2021 13:56:32 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 7 May 2021 13:56:32 +0200
The usual behavior for functions in ruleutils.c is to return NULL when
the object does not exist. pg_get_statisticsobjdef_expressions raised an
error instead, so correct that.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210505210947.GA27406%40telsasoft.com
M src/backend/utils/adt/ruleutils.c
Doc: Update notes about libc collation versions.
commit : b65431ca5e12a475ba7cf68afb63edb070c2ce08
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 7 May 2021 21:47:08 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 7 May 2021 21:47:08 +1200
The per-index collation version tracking feature was reverted, but we
still have the ability to ask Windows (352f6f2d) and FreeBSD
(ca051d8b) for collation versions to store in pg_collation.collversion.
So, from the reverted patch, take a few words of documentation about
libc on all three supported OSes to replace the pre-existing note that
mentioned only glibc.
Discussion: https://postgr.es/m/CA%2BhUKGLhj5t1fcjqAu8iD9B3ixJtsTNqyCCD4V0aTO9kAKAjjA%40mail.gmail.com
M doc/src/sgml/ref/alter_collation.sgml
Revert per-index collation version tracking feature.
commit : ec48314708262d8ea6cdcb83f803fc83dd89e721
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 7 May 2021 20:17:42 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 7 May 2021 20:17:42 +1200
Design problems were discovered in the handling of composite types and
record types that would cause some relevant versions not to be recorded.
Misgivings were also expressed about the use of the pg_depend catalog
for this purpose. We're out of time for this release so we'll revert
and try again.
Commits reverted:
1bf946bd: Doc: Document known problem with Windows collation versions.
cf002008: Remove no-longer-relevant test case.
ef387bed: Fix bogus collation-version-recording logic.
0fb0a050: Hide internal error for pg_collation_actual_version(<bad OID>).
ff942057: Suppress "warning: variable 'collcollate' set but not used".
d50e3b1f: Fix assertion in collation version lookup.
f24b1569: Rethink extraction of collation dependencies.
257836a7: Track collation versions for indexes.
cd6f479e: Add pg_depend.refobjversion.
7d1297df: Remove pg_collation.collversion.
Discussion: https://postgr.es/m/CA%2BhUKGLhj5t1fcjqAu8iD9B3ixJtsTNqyCCD4V0aTO9kAKAjjA%40mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/charset.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/alter_collation.sgml
M doc/src/sgml/ref/alter_index.sgml
M doc/src/sgml/ref/create_collation.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/reindex.sgml
M src/backend/catalog/dependency.c
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/pg_collation.c
M src/backend/catalog/pg_constraint.c
M src/backend/catalog/pg_depend.c
M src/backend/catalog/pg_type.c
M src/backend/commands/collationcmds.c
M src/backend/commands/statscmds.c
M src/backend/commands/tablecmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/optimizer/util/plancat.c
M src/backend/parser/gram.y
M src/backend/tcop/utility.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_upgrade_support.c
M src/backend/utils/cache/relcache.c
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_upgrade/dump.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/psql/tab-complete.c
M src/include/catalog/catversion.h
M src/include/catalog/dependency.h
M src/include/catalog/index.h
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_type.h
M src/include/commands/collationcmds.h
M src/include/nodes/parsenodes.h
M src/include/utils/pg_locale.h
M src/include/utils/rel.h
M src/test/Makefile
M src/test/locale/.gitignore
M src/test/locale/Makefile
D src/test/locale/t/001_index.pl
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/expected/collate.linux.utf8.out
M src/test/regress/expected/create_index.out
M src/test/regress/expected/misc_sanity.out
M src/test/regress/sql/collate.icu.utf8.sql
M src/test/regress/sql/collate.linux.utf8.sql
M src/test/regress/sql/create_index.sql
M src/tools/pgindent/typedefs.list
Remove redundant variable
commit : a288d94c91e345ebeb10ac30f247270c8c8e380a
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 6 May 2021 17:28:36 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 6 May 2021 17:28:36 -0400
Author: Amul Sul <sulamul@gmail.com>
Reviewed-by: Jeevan Ladhe <jeevan.ladhe@enterprisedb.com>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/CAAJ_b94HaNcrPVREUuB9-qUn2uB+gfcoX3FG_Vx0S6aFse+yhw@mail.gmail.com
M src/backend/parser/parse_utilcmd.c
Document lock level used by ALTER TABLE VALIDATE CONSTRAINT
commit : 469116389e18dbf6be0bd555bc2055a26be91a48
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 6 May 2021 17:17:57 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 6 May 2021 17:17:57 -0400
Backpatch all the way back to 9.6.
Author: Simon Riggs <simon.riggs@enterprisedb.com>
Discussion: https://postgr.es/m/CANbhV-EwxvdhHuOLdfG2ciYrHOHXV=mm6=fD5aMhqcH09Li3Tg@mail.gmail.com
M doc/src/sgml/ref/alter_table.sgml
Improve documentation on DETACH PARTITION lock levels
commit : db6e1aeb952e9aed26ba2a56b4145293c72b8068
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 6 May 2021 16:42:30 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 6 May 2021 16:42:30 -0400
This was forgotten in 71f4c8c6f74b.
Reported-by: Pavel Luzanov <p.luzanov@postgrespro.ru>
Author: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/0688e7c3-8bc8-a3e4-9d8e-3bcbbf3e1f4d@postgrespro.ru
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/alter_table.sgml
Remove overzealous VACUUM visibility map assertion.
commit : c9787385db47ba423d845b34d58e158551c6335d
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 6 May 2021 13:17:39 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 6 May 2021 13:17:39 -0700
The all_visible_according_to_vm variable's value is inherently prone to
becoming invalidated concurrently, since it is set before we even
acquire a lock on a related heap page buffer.
Oversight in commit 7136bf34, which added the assertion in passing.
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Reported-By: Tang <tanghy.fnst@fujitsu.com>
Diagnosed-By:: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAD21AoDzgc8_MYrA5m1fyydomw_eVKtQiYh7sfDK4KEhdMsf_g@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Track detached partitions more accurately in partdescs
commit : 3fe773b149755977d2ffde2afd89557b39d0afd9
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 6 May 2021 12:47:30 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 6 May 2021 12:47:30 -0400
In d6b8d29419df I (Álvaro) was sloppy about recording whether a
partition descripor does or does not include detached partitions, when
the snapshot checking does not see the pg_inherits row marked detached.
In that case no partition was omitted, yet in the relcache entry we were
saving the partdesc as omitting partitions. Flip that (so we save it as
a partdesc not omitting partitions, which indeed it doesn't), which
hopefully makes the code easier to reason about.
Author: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/CA+HiwqE7GxGU4VdzwZzfiz+Ont5SsopoFkgtrZGEdPqWRL+biA@mail.gmail.com
M src/backend/partitioning/partdesc.c
M src/include/utils/rel.h
Doc: trivial wording adjustment.
commit : c38cadc0907a8d071b043b2b32b83efa09db38ea
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 6 May 2021 09:59:11 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 6 May 2021 09:59:11 -0400
Improve self-referential foreign key example, per suggestion
from David Johnston.
Discussion: https://postgr.es/m/CAKFQuwZTke7+HUn4YUGqu2+gAPi4Cy18TXMrg_Z5nADkxfPNMw@mail.gmail.com
M doc/src/sgml/ddl.sgml
Additional doc fixes for configurable TOAST compression.
commit : 448b02c00515ba9d6683a8a97fe4305604d80028
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 6 May 2021 08:27:20 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 6 May 2021 08:27:20 -0400
The grammar changes in commit bbe0a81db69bd10bd166907c3701492a29aca294
allow SET COMPRESSION to be used with ALTER MATERIALIZED VIEW as
well as with ALTER TABLE, so update those docs to say that it works.
Also, update the documentation for the pg_column_compression()
to explain that it will return NULL when there's no relevant value.
Patch by me, per concerns from Michael Paquier.
Discussion: http://postgr.es/m/CA+Tgmob9h5u4iNL9KM0drZgkY-JL4oCVW0dWrMqtLPQ1zHkquA@mail.gmail.com
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/alter_materialized_view.sgml
docs: Clarify how ALTER TABLE .. SET COMPRESSION works.
commit : 2d0f662402635d591cac9f1daae5e81e7c4374fc
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 6 May 2021 08:22:45 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 6 May 2021 08:22:45 -0400
Justin Pryzby, per a complaint from Michael Paquier. Reviewed by
Dilip Kumar and by me.
Discussion: http://postgr.es/m/20210429040132.GF27406@telsasoft.com
M doc/src/sgml/ref/alter_table.sgml
Update replication statistics after every stream/spill.
commit : 592f00f8dec68038301467a904ac514eddabf6cd
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 6 May 2021 11:21:26 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 6 May 2021 11:21:26 +0530
Currently, replication slot statistics are updated at prepare, commit, and
rollback. Now, if the transaction is interrupted the stats might not get
updated. Fixed this by updating replication statistics after every
stream/spill.
In passing update the docs to change the description of some of the slot
stats.
Author: Vignesh C, Sawada Masahiko
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/20210319185247.ldebgpdaxsowiflw@alap3.anarazel.de
M doc/src/sgml/monitoring.sgml
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h
jit: Fix warning reported by gcc-11 caused by dubious function signature.
commit : 7f2e10baa2482494dbcf70e0ae6f0469771e0b4c
author : Andres Freund <andres@anarazel.de>
date : Wed, 5 May 2021 22:07:40 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 5 May 2021 22:07:40 -0700
Reported-By: Erik Rijkers <er@xs4all.nl>
Discussion: https://postgr.es/m/833107370.1313189.1619647621213@webmailclassic.xs4all.nl
Backpatch: 13, where b059d2f45685 introduced the issue.
M src/backend/jit/llvm/llvmjit_expr.c
Doc: update RELEASE_CHANGES checklist.
commit : e8ce68b0b9ae2757c6153a88bf869904d2d5ac0b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 5 May 2021 23:10:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 5 May 2021 23:10:33 -0400
Update checklist to reflect current practice:
* The platform-specific FAQ files are long gone.
* We've never routinely updated the libbind code we borrowed, either,
and there seems no reason to start now.
* Explain current practice of running pgindent twice per cycle.
Discussion: https://postgr.es/m/4038398.1620238684@sss.pgh.pa.us
M src/tools/RELEASE_CHANGES
Tighten the concurrent abort check during decoding.
commit : 2ce353fc19024d62e59ad99850d7592ebc9abecf
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 6 May 2021 08:26:42 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 6 May 2021 08:26:42 +0530
During decoding of an in-progress or prepared transaction, we detect
concurrent abort with an error code ERRCODE_TRANSACTION_ROLLBACK. That is
not sufficient because a callback can decide to throw that error code
at other times as well.
Reported-by: Tom Lane
Author: Amit Kapila
Reviewed-by: Dilip Kumar
Discussion: https://postgr.es/m/CAA4eK1KCjPRS4aZHB48QMM4J8XOC1+TD8jo-4Yu84E+MjwqVhA@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
Remove unused argument of ATAddForeignConstraint
commit : c250062df42ffd3e252471f6205bfb6cbef67b7b
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 5 May 2021 12:27:39 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 5 May 2021 12:27:39 -0400
Commit 0325d7a5957b made this unused but forgot to remove it. Do so now.
Author: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/209c99fe-b9a2-94f4-cd68-a8304186a09e@lab.ntt.co.jp
M src/backend/commands/tablecmds.c
Have ALTER CONSTRAINT recurse on partitioned tables
commit : 6f70d7ca1d1937a9f7b79eff6fb18ed1bb2a4c47
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 5 May 2021 12:14:21 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 5 May 2021 12:14:21 -0400
When ALTER TABLE .. ALTER CONSTRAINT changes deferrability properties
changed in a partitioned table, we failed to propagate those changes
correctly to partitions and to triggers. Repair by adding a recursion
mechanism to affect all derived constraints and all derived triggers.
(In particular, recurse to partitions even if their respective parents
are already in the desired state: it is possible for the partitions to
have been altered individually.) Because foreign keys involve tables in
two sides, we cannot use the standard ALTER TABLE recursion mechanism,
so we invent our own by following pg_constraint.conparentid down.
When ALTER TABLE .. ALTER CONSTRAINT is invoked on the derived
pg_constraint object that's automaticaly created in a partition as a
result of a constraint added to its parent, raise an error instead of
pretending to work and then failing to modify all the affected triggers.
Before this commit such a command would be allowed but failed to affect
all triggers, so it would silently misbehave. (Restoring dumps of
existing databases is not affected, because pg_dump does not produce
anything for such a derived constraint anyway.)
Add some tests for the case.
Backpatch to 11, where foreign key support was added to partitioned
tables by commit 3de241dba86f. (A related change is commit f56f8f8da6af
in pg12 which added support for FKs *referencing* partitioned tables;
this is what forces us to use an ad-hoc recursion mechanism for this.)
Diagnosed by Tom Lane from bug report from Ron L Johnson. As of this
writing, no reviews were offered.
Discussion: https://postgr.es/m/75fe0761-a291-86a9-c8d8-4906da077469@gmail.com
Discussion: https://postgr.es/m/3144850.1607369633@sss.pgh.pa.us
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
Doc: improve and centralize the documentation for OID alias types.
commit : f33a178a34809a2bae7a5f4c00984d87771f4204
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 5 May 2021 11:26:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 5 May 2021 11:26:48 -0400
Previously, a lot of information about type regclass existed only
in the discussion of the sequence functions. Maybe that made sense
in the beginning, because I think originally those were the only
functions taking regclass. But it doesn't make sense anymore.
Move that material to the "Object Identifier Types" section in
datatype.sgml, generalize it to talk about the other reg* types
as well, and add more examples.
Per bug #16991 from Federico Caselli.
Discussion: https://postgr.es/m/16991-bcaeaafa17e0a723@postgresql.org
M doc/src/sgml/datatype.sgml
M doc/src/sgml/func.sgml
GUC description improvements for clarity
commit : 38f36aad8c55c8f91e3fb8720fae1847c8fa0552
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 5 May 2021 08:18:22 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 5 May 2021 08:18:22 +0200
M src/backend/utils/misc/guc.c
Disable cache clobber to avoid breaking postgres_fdw termination test.
commit : 1273a15bf91fa322915e32d3b6dc6ec916397268
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 4 May 2021 13:36:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 4 May 2021 13:36:26 -0400
Commit 93f414614 improved a pre-existing test case so that it would
show whether or not termination of the "remote" worker process happened.
This soon exposed that, when debug_invalidate_system_caches_always
(nee CLOBBER_CACHE_ALWAYS) is enabled, no such termination occurs.
That's because cache invalidation forces postgres_fdw connections
to be dropped at end of transaction, so that there's no worker to
terminate. There's a race condition as to whether the worker will
manage to get out of the BackendStatusArray before we look, but at
least on buildfarm member hyrax, it's failed twice in two attempts.
Rather than re-lobotomizing the test, let's fix this by transiently
disabling debug_invalidate_system_caches_always. (Hooray for that
being just a GUC nowadays, rather than a compile-time option.)
If this proves not to be enough to make the test stable, we can
do the other thing instead.
Discussion: https://postgr.es/m/3854538.1620081771@sss.pgh.pa.us
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Fix OID passed to object-alter hook during ALTER CONSTRAINT
commit : e798d095da3a4a4bb5c50bb3dff886f07ef52f55
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 4 May 2021 10:09:12 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 4 May 2021 10:09:12 -0400
The OID of the constraint is used instead of the OID of the trigger --
an easy mistake to make. Apparently the object-alter hooks are not very
well tested :-(
Backpatch to 12, where this typo was introduced by 578b229718e8
Discussion: https://postgr.es/m/20210503231633.GA6994@alvherre.pgsql
M src/backend/commands/tablecmds.c
doc: Fix typos
commit : c98a6d7887ea6588b4e9797903182312a2b46f67
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 4 May 2021 15:45:13 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 4 May 2021 15:45:13 +0200
M doc/src/sgml/ecpg.sgml
pg_dump: Fix dump of generated columns in partitions
commit : feb270d1005f3d7b3705dec9e04c9a205750ea97
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 4 May 2021 14:03:54 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 4 May 2021 14:03:54 +0200
The previous fix for dumping of inherited generated columns
(0bf83648a52df96f7c8677edbbdf141bfa0cf32b) must not be applied to
partitions, since, unlike normal inherited tables, they are always
dumped separately and reattached.
Reported-by: Santosh Udupi <email@hitha.net>
Discussion: https://www.postgresql.org/message-id/flat/CACLRvHZ4a-%2BSM_159%2BtcrHdEqxFrG%3DW4gwTRnwf7Oj0UNj5R2A%40mail.gmail.com
M src/bin/pg_dump/common.c
Fix ALTER TABLE / INHERIT with generated columns
commit : a970edbed306354b0079bdcdc2fc74312122ad89
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 4 May 2021 11:45:37 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 4 May 2021 11:45:37 +0200
When running ALTER TABLE t2 INHERIT t1, we must check that columns in
t2 that correspond to a generated column in t1 are also generated and
have the same generation expression. Otherwise, this would allow
creating setups that a normal CREATE TABLE sequence would not allow.
Discussion: https://www.postgresql.org/message-id/22de27f6-7096-8d96-4619-7b882932ca25@2ndquadrant.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/generated.out
M src/test/regress/sql/generated.sql
Remove mention of the version number from pg_trgm docs
commit : ae9492a61bbf575e2862cf9323c7f02806382093
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 4 May 2021 03:56:16 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 4 May 2021 03:56:16 +0300
We don't usually mention the version number in similar situations. So, neither
mention it here.
Reported-by: Bruce Momjian
Discussion: https://postgr.es/m/20210503234914.GO6180%40momjian.us
M doc/src/sgml/pgtrgm.sgml
Update query_id computation
commit : f7a97b6ec31f3f57a6154d0039c4de81ad517064
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 3 May 2021 14:59:30 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 3 May 2021 14:59:30 -0400
Properly fix:
- the "ONLY" in FROM [ONLY] isn't hashed
- the agglevelsup field in GROUPING isn't hashed
- WITH TIES not being hashed (new in PG 13)
- "DISTINCT" in "GROUP BY [DISTINCT]" isn't hashed (new in PG 14)
Reported-by: Julien Rouhaud
Discussion: https://postgr.es/m/20210425081119.ulyzxqz23ueh3wuj@nol
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
M src/backend/utils/misc/queryjumble.c
doc: Add index entry for "multirange type"
commit : 5df6aeab42279eaea8e9ff92744645b155c85b03
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 3 May 2021 20:14:03 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 3 May 2021 20:14:03 +0200
Before now, looking up "multirange" in the index only led to the
multirange() function. To make this more useful, also add an entry
pointing to the range types section.
M doc/src/sgml/func.sgml
M doc/src/sgml/rangetypes.sgml
amcheck: Improve some confusing reports about TOAST problems.
commit : 50529e5b4e39ad80a637ba0905277f9691eb4a15
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 3 May 2021 12:32:05 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 3 May 2021 12:32:05 -0400
Don't phrase reports in terms of the number of tuples thus-far
returned by the index scan, but rather in terms of the chunk_seq
values found inside the tuples.
Patch by me, reviewed by Mark Dilger.
Discussion: http://postgr.es/m/CA+TgmoZUONCkdcdR778EKuE+f1r5Obieu63db2OgMPHaEvEPTQ@mail.gmail.com
M contrib/amcheck/verify_heapam.c
Fix performance issue in new regex match-all detection code.
commit : f68970e33f4dc48094c24c78c452ad730ae9ae12
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 May 2021 11:42:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 May 2021 11:42:31 -0400
Commit 824bf7190 introduced a new search of the NFAs generated by
regex compilation. I failed to think hard about the performance
characteristics of that search, with the predictable outcome
that it's bad: weird regexes can trigger exponential search time.
Worse, there's no check-for-interrupt in that code, so you can't
even cancel the query if this happens.
Fix by introducing memo-ization of the search results, so that any one
NFA state need be examined in detail just once. This potentially uses
a lot of memory, but we can bound the memory usage by putting a limit
on the number of states for which we'll try to prove match-all-ness.
That is sane because we already have a limit (DUPINF) on the maximum
finite string length that a matchall regex can match; and patterns
that involve much more than DUPINF states would probably exceed that
limit anyway.
Also, rearrange the logic so that we check the basic is-the-graph-
all-RAINBOW-arcs property before we start the recursive search to
determine path lengths. This will ensure that we fall out quickly
whenever the NFA couldn't possibly be matchall.
Also stick in a check-for-interrupt, just in case these measures
don't completely eliminate the risk of slowness.
Discussion: https://postgr.es/m/3483895.1619898362@sss.pgh.pa.us
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/test/regress/expected/regex.out
M src/test/regress/sql/regex.sql
Prevent lwlock dtrace probes from unnecessary work
commit : b94409a02f6122d77b5154e481c0819fed6b4c95
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 3 May 2021 12:11:33 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 3 May 2021 12:11:33 +0200
If dtrace is compiled in but disabled, the lwlock dtrace probes still
evaluate their arguments. Since PostgreSQL 13, T_NAME(lock) does
nontrivial work, so it should be avoided if not needed. To fix, make
these calls conditional on the *_ENABLED() macro corresponding to each
probe.
Reviewed-by: Craig Ringer <craig.ringer@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/CAGRY4nwxKUS_RvXFW-ugrZBYxPFFM5kjwKT5O+0+Stuga5b4+Q@mail.gmail.com
M src/backend/storage/lmgr/lwlock.c
Remove unused function argument
commit : c285babf8f44d86b7fd1e73e9e4f94456b825bfb
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 3 May 2021 09:05:58 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 3 May 2021 09:05:58 +0200
became unused by 04942bffd0aa9bd0d143d99b473342eb9ecee88b
M src/backend/rewrite/rewriteHandler.c
libpq: Refactor some error messages for easier translation
commit : ced12b73a9bc76b887cb7137df3d56222e2b5263
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 3 May 2021 08:51:30 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 3 May 2021 08:51:30 +0200
M src/interfaces/libpq/fe-connect.c
Factor out system call names from error messages
commit : 853c8c75571558f4b474eeac3ef9e6fcf9be62ba
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 3 May 2021 07:27:31 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 3 May 2021 07:27:31 +0200
One more that ought to have been part of
82c3cd974131d7fa1cfcd07cebfb04fffe26ee35.
M src/interfaces/libpq/fe-connect.c
Fix the computation of slot stats for 'total_bytes'.
commit : 205f466282be11ec97506f73341e47b72e0aee5d
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 3 May 2021 07:22:08 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 3 May 2021 07:22:08 +0530
Previously, we were using the size of all the changes present in
ReorderBuffer to compute total_bytes after decoding a transaction and that
can lead to counting some of the transactions' changes more than once. Fix
it by using the size of the changes decoded for a transaction to compute
'total_bytes'.
Author: Sawada Masahiko
Reviewed-by: Vignesh C, Amit Kapila
Discussion: https://postgr.es/m/20210319185247.ldebgpdaxsowiflw@alap3.anarazel.de
M src/backend/replication/logical/reorderbuffer.c
Make websearch_to_tsquery() parse text in quotes as a single token
commit : eb086056fec44516efdd5db71244a079fed65c7f
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 3 May 2021 03:58:03 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 3 May 2021 03:58:03 +0300
websearch_to_tsquery() splits text in quotes into tokens and connects them with
phrase operator on its own. However, that leads to surprising results when the
token contains no words.
For instance, websearch_to_tsquery('"aaa: bbb"') is 'aaa <2> bbb', because
it is equivalent of to_tsquery(E'aaa <-> \':\' <-> bbb'). But
websearch_to_tsquery('"aaa: bbb"') has to be 'aaa <-> bbb' in order to match
to_tsvector('aaa: bbb').
Since 0c4f355c6a, we anyway connect lexemes of complex tokens with phrase
operators. Thus, let's just websearch_to_tsquery() parse text in quotes as
a single token. Therefore, websearch_to_tsquery() should process the quoted
text in the same way phraseto_tsquery() does. This solution is what we exactly
need and also simplifies the code.
This commit is an incompatible change, so we don't backpatch it.
Reported-by: Valentin Gatien-Baron
Discussion: https://postgr.es/m/CA%2B0DEqiZs7gdOd4ikmg%3D0UWG%2BSwWOLxPsk_JW-sx9WNOyrb0KQ%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Tom Lane, Zhihong Yu
M src/backend/utils/adt/tsquery.c
M src/test/regress/expected/tsearch.out
M src/test/regress/sql/tsearch.sql
Revert use singular for -1 (commits 9ee7d533da and 5da9868ed9
commit : 651d005e76bc0b9542615f609b4d0d946035dc58
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 1 May 2021 10:42:44 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 1 May 2021 10:42:44 -0400
Turns out you can specify negative values using plurals:
https://english.stackexchange.com/questions/9735/is-1-followed-by-a-singular-or-plural-noun
so the previous code was correct enough, and consistent with other usage
in our code. Also add comment in the two places where this could be
confused.
Reported-by: Noah Misch
Diagnosed-by: 20210425115726.GA2353095@rfd.leadboat.com
M contrib/dblink/expected/dblink.out
M src/backend/utils/adt/datetime.c
M src/interfaces/ecpg/pgtypeslib/interval.c
M src/interfaces/libpq/fe-print.c
M src/test/regress/expected/interval.out
Doc: add an example of a self-referential foreign key to ddl.sgml.
commit : e6f9539dc32473793c03cbe95bc099ee0a199c73
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Apr 2021 15:37:56 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Apr 2021 15:37:56 -0400
While we've always allowed such cases, the documentation didn't
say you could do it.
Discussion: https://postgr.es/m/161969805833.690.13680986983883602407@wrigleys.postgresql.org
M doc/src/sgml/ddl.sgml
Doc: update libpq's documentation for PQfn().
commit : 386e64ea5abf346d887c21ea8869317838ba19b5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Apr 2021 15:10:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Apr 2021 15:10:06 -0400
Mention specifically that you can't call aggregates, window functions,
or procedures this way (the inability to call SRFs was already
mentioned).
Also, the claim that PQfn doesn't support NULL arguments or results
has been a lie since we invented protocol 3.0. Not sure why this
text was never updated for that, but do it now.
Discussion: https://postgr.es/m/2039442.1615317309@sss.pgh.pa.us
M doc/src/sgml/libpq.sgml
Disallow calling anything but plain functions via the fastpath API.
commit : 2efcd502e56a528f75ec8e88c02a287ad3457d77
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Apr 2021 14:10:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Apr 2021 14:10:26 -0400
Reject aggregates, window functions, and procedures. Aggregates
failed anyway, though with a somewhat obscure error message.
Window functions would hit an Assert or null-pointer dereference.
Procedures seemed to work as long as you didn't try to do
transaction control, but (a) transaction control is sort of the
point of a procedure, and (b) it's not entirely clear that no
bugs lurk in that path. Given the lack of testing of this area,
it seems safest to be conservative in what we support.
Also reject proretset functions, as the fastpath protocol can't
support returning a set.
Also remove an easily-triggered assertion that the given OID
isn't 0; the subsequent lookups can handle that case themselves.
Per report from Theodor-Arsenij Larionov-Trichkin.
Back-patch to all supported branches. (The procedure angle
only applies in v11+, of course.)
Discussion: https://postgr.es/m/2039442.1615317309@sss.pgh.pa.us
M src/backend/tcop/fastpath.c
Fix the bugs in selecting the transaction for streaming.
commit : ee4ba01dbbc31daa083f434ecd603a80bbe50501
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 30 Apr 2021 10:49:52 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 30 Apr 2021 10:49:52 +0530
There were two problems:
a. We were always selecting the next available txn instead of selecting it
when it is larger than the previous transaction.
b. We were selecting the transactions which haven't made any changes to
the database (base snapshot is not set). Later it was hitting an Assert
because we don't decode such transactions and the changes in txn remain as
it is. It is better not to choose such transactions for streaming in the
first place.
Reported-by: Haiying Tang
Author: Dilip Kumar
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/OS0PR01MB61133B94E63177040F7ECDA1FB429@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/backend/replication/logical/reorderbuffer.c
Adjust EXPLAIN output for parallel Result Cache plans
commit : 3c80e96dffd4df7f66fffa5f265cbd87becb7ef5
author : David Rowley <drowley@postgresql.org>
date : Fri, 30 Apr 2021 14:46:42 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 30 Apr 2021 14:46:42 +1200
Here we adjust the EXPLAIN ANALYZE output for Result Cache so that we
don't show any Result Cache stats for parallel workers who don't
contribute anything to Result Cache plan nodes.
I originally had ideas that workers who don't help could still have their
Result Cache stats displayed. The idea with that was so that I could
write some parallel Result Cache regression tests that show the EXPLAIN
ANALYZE output. However, I realized a little too late that such tests
would just not be possible to have run in a stable way on the buildfarm.
With that knowledge, before 9eacee2e6 went in, I had removed all of the
tests that were showing the EXPLAIN ANALYZE output of a parallel Result
Cache plan, however, I forgot to put back the code that adjusts the
EXPLAIN output to hide the Result Cache stats for parallel workers who
were not fast enough to help out before query execution was over. All
other nodes behave this way and so should Result Cache.
Additionally, with this change, it now seems safe enough to remove the SET
force_parallel_mode = off that I had added to the regression tests.
Also, perform some cleanup in the partition_prune tests. I had adjusted
the explain_parallel_append() function to sanitize the Result Cache
EXPLAIN ANALYZE output. However, since I didn't actually include any
parallel Result Cache tests that show their EXPLAIN ANALYZE output, that
code does nothing and can be removed.
In passing, move the setting of memPeakKb into the scope where it's used.
Reported-by: Amit Khandekar
Author: David Rowley, Amit Khandekar
Discussion: https://postgr.es/m/CAJ3gD9d8SkfY95GpM1zmsOtX2-Ogx5q-WLsf8f0ykEb0hCRK3w@mail.gmail.com
M src/backend/commands/explain.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/resultcache.out
M src/test/regress/sql/partition_prune.sql
M src/test/regress/sql/resultcache.sql
Another try to fix the test case added by commit f5fc2f5b23.
commit : 51ef9173030cc196c6633ae82b7b32f404b0f768
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 30 Apr 2021 07:55:42 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 30 Apr 2021 07:55:42 +0530
As per analysis, it appears that the 'drop slot' message from the previous
test and 'create slot' message of the new test are either missed or not
yet delivered to the stats collector due to which we will still see the
stats from the old slot. This can happen rarely which could be the reason
that we are seeing some failures in the buildfarm randomly. To avoid that
we are using a different slot name for the tests in
test_decoding/sql/stats.sql.
Reported-by: Tom Lane based on buildfarm reports
Author: Sawada Masahiko
Reviewed-by: Amit Kapila, Vignesh C
Discussion: https://postgr.es/m/20210319185247.ldebgpdaxsowiflw@alap3.anarazel.de
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
Improve wording of some pg_upgrade failure reports.
commit : c9c37ae03fea0c8ad467392ddf03940b61974935
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Apr 2021 15:40:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Apr 2021 15:40:34 -0400
Don't advocate dropping a whole table when dropping a column would
serve. While at it, try to make the layout of these messages a
bit cleaner and more consistent.
Per suggestion from Daniel Gustafsson. No back-patch, as we generally
don't like to churn translatable messages in released branches.
Discussion: https://postgr.es/m/2798740.1619622555@sss.pgh.pa.us
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/version.c
Fix some more omissions in pg_upgrade's tests for non-upgradable types.
commit : 57c081de0afcd01bf47c46f015bf8886b01c2c21
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Apr 2021 15:24:37 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Apr 2021 15:24:37 -0400
Commits 29aeda6e4 et al closed up some oversights involving not checking
for non-upgradable types within container types, such as arrays and
ranges. However, I only looked at version.c, failing to notice that
there were substantially-equivalent tests in check.c. (The division
of responsibility between those files is less than clear...)
In addition, because genbki.pl does not guarantee that auto-generated
rowtype OIDs will hold still across versions, we need to consider that
the composite type associated with a system catalog or view is
non-upgradable. It seems unlikely that someone would have a user
column declared that way, but if they did, trying to read it in another
PG version would likely draw "no such pg_type OID" failures, thanks
to the type OID embedded in composite Datums.
To support the composite and reg*-type cases, extend the recursive
query that does the search to allow any base query that returns
a column of pg_type OIDs, rather than limiting it to exactly one
starting type.
As before, back-patch to all supported branches.
Discussion: https://postgr.es/m/2798740.1619622555@sss.pgh.pa.us
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/version.c
Improve documentation for default_tablespace on partitioned tables
commit : 94b9cb722552c37da78c8320bac1d5b55e34def6
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 29 Apr 2021 11:31:24 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 29 Apr 2021 11:31:24 -0400
Backpatch to 12, where 87259588d0ab introduced the current behavior.
Per note from Justin Pryzby.
Co-authored-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/20210416143135.GI3315@telsasoft.com
M doc/src/sgml/config.sgml
psql: Fix line continuation prompts for unbalanced parentheses
commit : d9a9f4b4b92ad39e3c4e6600dc61d5603ddd6e24
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 29 Apr 2021 09:04:31 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 29 Apr 2021 09:04:31 +0200
This was broken by a silly mistake in
e717a9a18b2e34c9c40e5259ad4d31cd7e420750.
Reported-by: Jeff Janes <jeff.janes@gmail.com>
Author: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/CAMkU=1zKGWEJdBbYKw7Tn7cJmYR_UjgdcXTPDqJj=dNwCETBCQ@mail.gmail.com
M src/fe_utils/psqlscan.l
pg_hba.conf.sample: Reword connection type section
commit : 3a948ea0a2ced719f26e725b030558f2e4ab1d8e
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 29 Apr 2021 06:51:42 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 29 Apr 2021 06:51:42 +0200
Improve the wording in the connection type section of
pg_hba.conf.sample a bit. After the hostgssenc part was added on, the
whole thing became a bit wordy, and it's also a bit inaccurate for
example in that the current wording for "host" appears to say that it
does not apply to GSS-encrypted connections.
Discussion: https://www.postgresql.org/message-id/flat/fc06dcc5-513f-e944-cd07-ba51dd7c6916%40enterprisedb.com
M src/backend/libpq/pg_hba.conf.sample
doc: Fix description of target_session_attrs=prefer-standby
commit : 2977f244bc7e64e046364122be3fef08aa6efef9
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 29 Apr 2021 11:44:24 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 29 Apr 2021 11:44:24 +0900
If no standbys can be found in the set of connection points provided,
the fallback behavior is "any", and not "all" that does not exist.
Author: Greg Nancarrow
Reviewed-by: Laurenz Albe
Discussion: https://postgr.es/m/CAJcOf-fDaCv8qCpWH7k5Yh6zFxZeUwZowu4sCWQ2zFx4CdkHpA@mail.gmail.com
M doc/src/sgml/libpq.sgml
Add heuristic incoming-message-size limits in the server.
commit : 9626325da5e8e23ff90091bc96535495d350f06e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Apr 2021 15:50:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Apr 2021 15:50:42 -0400
We had a report of confusing server behavior caused by a client bug
that sent junk to the server: the server thought the junk was a
very long message length and waited patiently for data that would
never come. We can reduce the risk of that by being less trusting
about message lengths.
For a long time, libpq has had a heuristic rule that it wouldn't
believe large message size words, except for a small number of
message types that are expected to be (potentially) long. This
provides some defense against loss of message-boundary sync and
other corrupted-data cases. The server does something similar,
except that up to now it only limited the lengths of messages
received during the connection authentication phase. Let's
do the same as in libpq and put restrictions on the allowed
length of all messages, while distinguishing between message
types that are expected to be long and those that aren't.
I used a limit of 10000 bytes for non-long messages. (libpq's
corresponding limit is 30000 bytes, but given the asymmetry of
the FE/BE protocol, there's no good reason why the numbers should
be the same.) Experimentation suggests that this is at least a
factor of 10, maybe a factor of 100, more than we really need;
but plenty of daylight seems desirable to avoid false positives.
In any case we can adjust the limit based on beta-test results.
For long messages, set a limit of MaxAllocSize - 1, which is the
most that we can absorb into the StringInfo buffer that the message
is collected in. This just serves to make sure that a bogus message
size is reported as such, rather than as a confusing gripe about
not being able to enlarge a string buffer.
While at it, make sure that non-mainline code paths (such as
COPY FROM STDIN) are as paranoid as SocketBackend is, and validate
the message type code before believing the message length.
This provides an additional guard against getting stuck on corrupted
input.
Discussion: https://postgr.es/m/2003757.1619373089@sss.pgh.pa.us
M src/backend/commands/copyfromparse.c
M src/backend/libpq/auth.c
M src/backend/libpq/pqcomm.c
M src/backend/replication/walsender.c
M src/backend/tcop/postgres.c
M src/include/libpq/libpq.h
Allow a partdesc-omitting-partitions to be cached
commit : d6b8d29419df0efad57f95c80b871745d1b55da6
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 28 Apr 2021 15:44:35 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 28 Apr 2021 15:44:35 -0400
Makes partition descriptor acquisition faster during the transient
period in which a partition is in the process of being detached.
This also adds the restriction that only one partition can be in
pending-detach state for a partitioned table.
While at it, return find_inheritance_children() API to what it was
before 71f4c8c6f74b, and create a separate
find_inheritance_children_extended() that returns detailed info about
detached partitions.
(This incidentally fixes a bug in 8aba9322511 whereby a memory context
holding a transient partdesc is reparented to a NULL PortalContext,
leading to permanent leak of that memory. The fix is to no longer rely
on reparenting contexts to PortalContext. Reported by Amit Langote.)
Per gripe from Amit Langote
Discussion: https://postgr.es/m/CA+HiwqFgpP1LxJZOBYGt9rpvTjXXkg5qG2+Xch2Z1Q7KrqZR1A@mail.gmail.com
M doc/src/sgml/ref/alter_table.sgml
M src/backend/catalog/pg_inherits.c
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/partitioning/partdesc.c
M src/backend/utils/cache/relcache.c
M src/include/catalog/pg_inherits.h
M src/include/utils/rel.h
M src/test/isolation/expected/detach-partition-concurrently-3.out
M src/test/isolation/specs/detach-partition-concurrently-3.spec
Doc: fix discussion of how to get real Julian Dates.
commit : c93f8f3b8d3bc780892e2bf11192fbdd136fddfe
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Apr 2021 10:03:28 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Apr 2021 10:03:28 -0400
Somehow I'd convinced myself that rotating to UTC-12 was the way
to do this, but upon further review, it's definitely UTC+12.
Discussion: https://postgr.es/m/1197050.1619123213@sss.pgh.pa.us
M doc/src/sgml/datetime.sgml
Fix use-after-release issue with pg_identify_object_as_address()
commit : f93f0b5b25068807051edb2f3510614b69bb79ff
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Apr 2021 11:58:08 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Apr 2021 11:58:08 +0900
Spotted by buildfarm member prion, with -DRELCACHE_FORCE_RELEASE.
Introduced in f7aab36.
Discussion: https://postgr.es/m/2759018.1619577848@sss.pgh.pa.us
Backpatch-through: 9.6
M src/backend/catalog/objectaddress.c
Fix pg_identify_object_as_address() with event triggers
commit : f7aab36d61fd2fdbd949d5880247e8cae9ee4be0
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Apr 2021 11:17:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Apr 2021 11:17:58 +0900
Attempting to use this function with event triggers failed, as, since
its introduction in a676201, this code has never associated an object
name with event triggers. This addresses the failure by adding the
event trigger name to the set defining its object address.
Note that regression tests are added within event_trigger and not
object_address to avoid issues with concurrent connections in parallel
schedules.
Author: Joel Jacobson
Discussion: https://postgr.es/m/3c905e77-a026-46ae-8835-c3f6cd1d24c8@www.fastmail.com
Backpatch-through: 9.6
M src/backend/catalog/objectaddress.c
M src/test/regress/expected/event_trigger.out
M src/test/regress/sql/event_trigger.sql
Improve logic in PostgresVersion.pm
commit : fa26eba221a9e837493df47d0255ce615129e9a8
author : Andrew Dunstan <andrew@dunslane.net>
date : Tue, 27 Apr 2021 08:21:15 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Tue, 27 Apr 2021 08:21:15 -0400
Handle the situation where perl swaps the order of operands of
the comparison operator. See `perldoc overload` for details:
The third argument is set to TRUE if (and only if) the two
operands have been swapped. Perl may do this to ensure that the
first argument ($self) is an object implementing the overloaded
operation, in line with general object calling conventions.
M src/test/perl/PostgresVersion.pm
doc: Review for "Allow TRUNCATE command to truncate foreign tables".
commit : 0c8f40863acb94963df9fd6a4369eb71efe9a93b
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 27 Apr 2021 18:39:30 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 27 Apr 2021 18:39:30 +0900
Typos, corrections and language improvements in the docs.
Author: Justin Pryzby, Fujii Masao
Reviewed-by: Bharath Rupireddy, Justin Pryzby, Fujii Masao
Discussion: https://postgr.es/m/20210411041658.GB14564@telsasoft.com
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/ref/truncate.sgml
Don't pass "ONLY" options specified in TRUNCATE to foreign data wrapper.
commit : 8e9ea08bae93a754d5075b7bc9c0b2bc71958bfd
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 27 Apr 2021 14:41:27 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 27 Apr 2021 14:41:27 +0900
Commit 8ff1c94649 allowed TRUNCATE command to truncate foreign tables.
Previously the information about "ONLY" options specified in TRUNCATE
command were passed to the foreign data wrapper. Then postgres_fdw
constructed the TRUNCATE command to issue the remote server and
included "ONLY" options in it based on the passed information.
On the other hand, "ONLY" options specified in SELECT, UPDATE or DELETE
have no effect when accessing or modifying the remote table, i.e.,
are not passed to the foreign data wrapper. So it's inconsistent to
make only TRUNCATE command pass the "ONLY" options to the foreign data
wrapper. Therefore this commit changes the TRUNCATE command so that
it doesn't pass the "ONLY" options to the foreign data wrapper,
for the consistency with other statements. Also this commit changes
postgres_fdw so that it always doesn't include "ONLY" options in
the TRUNCATE command that it constructs.
Author: Fujii Masao
Reviewed-by: Bharath Rupireddy, Kyotaro Horiguchi, Justin Pryzby, Zhihong Yu
Discussion: https://postgr.es/m/551ed8c1-f531-818b-664a-2cecdab99cd8@oss.nttdata.com
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/postgres-fdw.sgml
M src/backend/commands/tablecmds.c
M src/backend/replication/logical/worker.c
M src/include/commands/tablecmds.h
M src/include/foreign/fdwapi.h
Use HTAB for replication slot statistics.
commit : 3fa17d37716f978f80dfcdab4e7c73f3a24e7a48
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 27 Apr 2021 09:09:11 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 27 Apr 2021 09:09:11 +0530
Previously, we used to use the array of size max_replication_slots to
store stats for replication slots. But that had two problems in the cases
where a message for dropping a slot gets lost: 1) the stats for the new
slot are not recorded if the array is full and 2) writing beyond the end
of the array if the user reduces the max_replication_slots.
This commit uses HTAB for replication slot statistics, resolving both
problems. Now, pgstat_vacuum_stat() search for all the dead replication
slots in stats hashtable and tell the collector to remove them. To avoid
showing the stats for the already-dropped slots, pg_stat_replication_slots
view searches slot stats by the slot name taken from pg_replication_slots.
Also, we send a message for creating a slot at slot creation, initializing
the stats. This reduces the possibility that the stats are accumulated
into the old slot stats when a message for dropping a slot gets lost.
Reported-by: Andres Freund
Author: Sawada Masahiko, test case by Vignesh C
Reviewed-by: Amit Kapila, Vignesh C, Dilip Kumar
Discussion: https://postgr.es/m/20210319185247.ldebgpdaxsowiflw@alap3.anarazel.de
M contrib/test_decoding/t/001_repl_stats.pl
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/replication/slot.h
M src/test/regress/expected/rules.out
M src/tools/pgindent/typedefs.list
Fix Logical Replication of Truncate in synchronous commit mode.
commit : e7eea52b2d61917fbbdac7f3f895e4ef636e935b
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 27 Apr 2021 08:28:26 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 27 Apr 2021 08:28:26 +0530
The Truncate operation acquires an exclusive lock on the target relation
and indexes. It then waits for logical replication of the operation to
finish at commit. Now because we are acquiring the shared lock on the
target index to get index attributes in pgoutput while sending the
changes for the Truncate operation, it leads to a deadlock.
Actually, we don't need to acquire a lock on the target index as we build
the cache entry using a historic snapshot and all the later changes are
absorbed while decoding WAL. So, we wrote a special purpose function for
logical replication to get a bitmap of replica identity attribute numbers
where we get that information without locking the target index.
We decided not to backpatch this as there doesn't seem to be any field
complaint about this issue since it was introduced in commit 5dfd1e5a in
v11.
Reported-by: Haiying Tang
Author: Takamichi Osumi, test case by Li Japin
Reviewed-by: Amit Kapila, Ajin Cherian
Discussion: https://postgr.es/m/OS0PR01MB6113C2499C7DC70EE55ADB82FB759@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/backend/replication/logical/proto.c
M src/backend/utils/cache/relcache.c
M src/include/utils/relcache.h
M src/test/subscription/t/010_truncate.pl
psql: tab-complete ALTER ... DETACH CONCURRENTLY / FINALIZE
commit : 6dd1042eda0acdabaab352c19b783288de62b587
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 22 Apr 2021 16:37:46 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 22 Apr 2021 16:37:46 -0400
New keywords per 71f4c8c6f74b.
Discussion: https://postgr.es/m/20210422204035.GA25929@alvherre.pgsql
M src/bin/psql/tab-complete.c
Remove rewriteTargetListIU's expansion of view targetlists in UPDATE.
commit : 04942bffd0aa9bd0d143d99b473342eb9ecee88b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 26 Apr 2021 13:58:00 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 26 Apr 2021 13:58:00 -0400
Commit 2ec993a7c, which added triggers on views, modified the rewriter
to add dummy entries like "SET x = x" for all columns that weren't
actually being updated by the user in any UPDATE directed at a view.
That was needed at the time to produce a complete "NEW" row to pass
to the trigger. Later it was found to cause problems for ordinary
updatable views, so commit cab5dc5da restricted it to happen only for
trigger-updatable views. But in the wake of commit 86dc90056, we
really don't need it at all. nodeModifyTable.c populates the trigger
"OLD" row from the whole-row variable that is generated for the view,
and then it computes the "NEW" row using that old row and the UPDATE
targetlist. So there is no need for the UPDATE tlist to have dummy
entries, any more than it needs them for regular tables or other
types of views.
(The comments for rewriteTargetListIU suggest that we must do this
for correct expansion of NEW references in rules, but I now think
that that was just lazy comment editing in 2ec993a7c. If we didn't
need it for rules on views before there were triggers, we don't need
it after that.)
This essentially propagates 86dc90056's decision that we don't need
dummy column updates into the view case. Aside from making the
different cases more uniform and hence possibly forestalling future
bugs, it ought to save a little bit of rewriter/planner effort.
Discussion: https://postgr.es/m/2181213.1619397634@sss.pgh.pa.us
M src/backend/rewrite/rewriteHandler.c
Doc: document EXTRACT(JULIAN ...), improve Julian Date explanation.
commit : 79a5928ebcb726b7061bf265b5c6990e835e8c4f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 26 Apr 2021 11:50:35 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 26 Apr 2021 11:50:35 -0400
For some reason, the "julian" option for extract()/date_part() has
never gotten listed in the manual. Also, while Appendix B mentioned
in passing that we don't conform to the usual astronomical definition
that a Julian date starts at noon UTC, it was kind of vague about what
we do instead. Clarify that, and add an example showing how to get
the astronomical definition if you want it.
It's been like this for ages, so back-patch to all supported branches.
Discussion: https://postgr.es/m/1197050.1619123213@sss.pgh.pa.us
M doc/src/sgml/datetime.sgml
M doc/src/sgml/func.sgml
Fix pg_upgrade test on Cygwin
commit : 38c9a5938ac5e1409b42677fee970a12632852ee
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 26 Apr 2021 12:10:46 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 26 Apr 2021 12:10:46 +0200
The verification of permissions doesn't succeed on Cygwin, because the
required feature is not implemented for Cygwin at the moment. So skip
this part of the test, like MinGW already does.
M src/bin/pg_upgrade/test.sh
Add more tests with triggers on partitions for logical replication
commit : 2ecfeda3e916f2a1123f818018d9d35908a499ac
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 26 Apr 2021 15:22:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 26 Apr 2021 15:22:48 +0900
The tuple routing logic used by a logical replication worker can fire
triggers on relations part of a partition tree, but there was no test
coverage in this area. The existing script 003_constraints.pl included
something, but nothing when a tuple is applied across partitioned tables
on a subscriber.
Author: Amit Langote
Discussion: https://postgr.es/m/OS0PR01MB611383FA0FE92EB9DE21946AFB769@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/test/subscription/t/013_partition.pl
Avoid sending prepare multiple times while decoding.
commit : f25a4584c6f56f3407f8f8734c78f2a87e4b77e8
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 26 Apr 2021 11:27:44 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 26 Apr 2021 11:27:44 +0530
We send the prepare for the concurrently aborted xacts so that later when
rollback prepared is decoded and sent, the downstream should be able to
rollback such a xact. For 'streaming' case (when we send changes for
in-progress transactions), we were sending prepare twice when concurrent
abort was detected.
Author: Peter Smith
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/f82133c6-6055-b400-7922-97dae9f2b50b@enterprisedb.com
M src/backend/replication/logical/reorderbuffer.c
Remove unused function argument
commit : 3cbea581c76e86d51b8f2babf116e643847e7712
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 26 Apr 2021 07:05:21 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 26 Apr 2021 07:05:21 +0200
This was already unused in the initial commit
257836a75585934cc05ed7a80bccf8190d41e056. Apparently, it was used in
an earlier proposed patch version.
M src/bin/pg_dump/pg_dump.c
Fix typo in reorderbuffer.c.
commit : 6d2e87a077b3c2394e4adb8eb226b3dcfe3f3346
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 26 Apr 2021 08:42:46 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 26 Apr 2021 08:42:46 +0530
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+PtvzuYY0zu=dVRK_WVz5WGos1+otZWgEWqjha1ncoSRag@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
Update comments for rewriteTargetListIU().
commit : 08a986966524e522914b96e4398a4bebf942b298
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 25 Apr 2021 18:02:03 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 25 Apr 2021 18:02:03 -0400
This function's behavior for UPDATE on a trigger-updatable view was
justified by analogy to what preptlist.c used to do for UPDATE on
regular tables. Since preptlist.c hasn't done that since 86dc90056,
that argument is no longer sensible, let alone convincing. I think
we do still need it to act that way, so update the comment to explain
why.
M src/backend/rewrite/rewriteHandler.c
Make a test endure log_error_verbosity=verbose.
commit : 59773da2b132ced19c84ff1f69b53b5cf95fd69e
author : Noah Misch <noah@leadboat.com>
date : Sun, 25 Apr 2021 01:08:05 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 25 Apr 2021 01:08:05 -0700
M src/test/recovery/t/024_archive_recovery.pl
Provide pg_amcheck with an --install-missing option
commit : b859d94c638968ccbb517ac7e151bdd94ed7c16a
author : Andrew Dunstan <andrew@dunslane.net>
date : Sat, 24 Apr 2021 10:13:07 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sat, 24 Apr 2021 10:13:07 -0400
This will install amcheck in the database if not present. The default
schema is for the extension is pg_catalog, but this can be overridden by
providing a value for the option.
Mark Dilger, slightly editorialized by me.
(rather divergent)
Discussion: https://postgr.es/m/bdc0f7c2-09e3-ee57-8471-569dfb509234@dunslane.net
M doc/src/sgml/ref/pg_amcheck.sgml
M src/bin/pg_amcheck/pg_amcheck.c
Teach PostgresVersion all the ways to mark non-release code
commit : aa271209f6d995488fc5cba9731415f974823990
author : Andrew Dunstan <andrew@dunslane.net>
date : Sat, 24 Apr 2021 09:37:20 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sat, 24 Apr 2021 09:37:20 -0400
As well as 'devel' version_stamp.pl provides for 'alphaN'
'betaN' and 'rcN', so teach PostgresVersion about those.
Also stash the version string instead of trying to reconstruct it during
stringification.
Discussion: https://postgr.es/m/YIHlw5nSgAHs4dK1@paquier.xyz
M src/test/perl/PostgresVersion.pm
Fix come comments in execMain.c
commit : 9b5558e7ad4706bbd53947e5b4d7c06e150390a5
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 24 Apr 2021 15:07:04 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 24 Apr 2021 15:07:04 +0900
1375422 has refactored this area of the executor code, and some comments
went out-of-sync.
Author: Yukun Wang
Reviewed-by: Amul Sul
Discussion: https://postgr.es/m/OS0PR01MB60033394FCAEF79B98F078F5B4459@OS0PR01MB6003.jpnprd01.prod.outlook.com
M src/backend/executor/execMain.c
Doc: Remove extraneous whitespaces with some tags
commit : 0d0049c58b4970183a102fc1f7dc1e9bef2a7149
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 24 Apr 2021 10:44:13 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 24 Apr 2021 10:44:13 +0900
Author: Justin Pryzby
Discussion: https://postgr.es/m/20210423184338.GL7256@telsasoft.com
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/ref/pg_rewind.sgml
M doc/src/sgml/runtime.sgml
Add some forgotten LSN_FORMAT_ARGS() in xlogreader.c
commit : 4aba61b87026b43fb37fc8e9ec5d9ae208e07b6b
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 24 Apr 2021 09:09:02 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 24 Apr 2021 09:09:02 +0900
6f6f284 has introduced a specific macro to make printf()-ing of LSNs
easier. This takes care of what looks like the remaining code paths
that did not get the call.
Author: Michael Paquier
Reviewed-by: Kyotaro Horiguchi, Tom Lane
Discussion: https://postgr.es/m/YIJS9x6K8ruizN7j@paquier.xyz
M src/backend/access/transam/xlogreader.c
amcheck: MAXALIGN() nbtree special area offset.
commit : bb3ecc8c961896ecb2ad3d5ba705c2877b933945
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 23 Apr 2021 15:37:03 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 23 Apr 2021 15:37:03 -0700
This isn't strictly necessary, but in theory it might matter if in the
future the width of the nbtree special area changes -- its total size
might not be an even number of MAXALIGN() quantums, even with padding.
PageInit() MAXALIGN()s all special area offsets, but amcheck uses the
offset to perform initial basic validation of line pointers, so we don't
rely on the offset from the page header.
The real reason to do this is to set a good example for new code that
adds amcheck coverage for other index AMs.
Reported-By: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Discussion: https://postgr.es/m/CALj2ACUMqTR9nErh99FbOBmzCXE9=gXNqhBiwYOhejJJS1LXqQ@mail.gmail.com
M contrib/amcheck/verify_nbtree.c
Factor out system call names from error messages
commit : 82c3cd974131d7fa1cfcd07cebfb04fffe26ee35
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 23 Apr 2021 14:18:11 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 23 Apr 2021 14:18:11 +0200
Instead, put them in via a format placeholder. This reduces the
number of distinct translatable messages and also reduces the chances
of typos during translation. We already did this for the system call
arguments in a number of cases, so this is just the same thing taken a
bit further.
Discussion: https://www.postgresql.org/message-id/flat/92d6f545-5102-65d8-3c87-489f71ea0a37%40enterprisedb.com
M src/backend/libpq/pqcomm.c
M src/backend/postmaster/pgstat.c
M src/backend/storage/ipc/latch.c
M src/bin/pg_basebackup/pg_recvlogical.c
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_dump/parallel.c
M src/bin/pg_upgrade/parallel.c
M src/common/exec.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-misc.c
Use correct format placeholder for WSAGetLastError()
commit : 9486844f301e265a3ad11b612decaba2462c3c15
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 23 Apr 2021 13:27:01 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 23 Apr 2021 13:27:01 +0200
Some code thought this was unsigned, but it's signed int.
M src/backend/libpq/pqcomm.c
M src/backend/port/win32/socket.c
M src/backend/storage/ipc/latch.c
M src/interfaces/libpq/fe-connect.c
Mark multirange_constructor0() and multirange_constructor2() strict
commit : 6bbcff096f932a1fe43ac3208c5c8b0acac29cda
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 23 Apr 2021 12:57:33 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 23 Apr 2021 12:57:33 +0300
These functions shouldn't receive null arguments: multirange_constructor0()
doesn't have any arguments while multirange_constructor2() has a single array
argument, which is never null.
But mark them strict anyway for the sake of uniformity.
Also, make checks for null arguments use elog() instead of ereport() as these
errors should normally be never thrown. And adjust corresponding comments.
Catversion is bumped.
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/0f783a96-8d67-9e71-996b-f34a7352eeef%40enterprisedb.com
M src/backend/commands/typecmds.c
M src/backend/utils/adt/multirangetypes.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Reorder COMPRESSION option in gram.y and parsenodes.h into alphabetical order.
commit : 3f20d5f37086e548c32ddb9d6ae09c2e1ce300ce
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 23 Apr 2021 19:10:24 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 23 Apr 2021 19:10:24 +0900
Commit bbe0a81db6 introduced "INCLUDING COMPRESSION" option
in CREATE TABLE command, but previously TableLikeOption in gram.y and
parsenodes.h didn't classify this new option in alphabetical order
with the rest.
Author: Fujii Masao
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/YHerAixOhfR1ryXa@paquier.xyz
M src/backend/parser/gram.y
M src/include/nodes/parsenodes.h
Mention that toplevel is part of pg_stat_statements key.
commit : 7531fcb1fcf5b3ea2f49959a3f095c083e3fc4c4
author : Magnus Hagander <magnus@hagander.net>
date : Fri, 23 Apr 2021 11:41:36 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Fri, 23 Apr 2021 11:41:36 +0200
While at it, also document that toplevel is always true if
pg_stat_statements.track is set to top.
Author: Julien Rouhaud
Reported-By: Fujii Masao
Discussion: https://postgr.es/m/a878d5ea-64a7-485e-5d2f-177618ebc52d@oss.nttdata.com
M doc/src/sgml/pgstatstatements.sgml
pg_amcheck: Use logging functions
commit : add5fad78aac8da96aeeb730155d35b16ff9b55a
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 23 Apr 2021 09:55:23 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 23 Apr 2021 09:55:23 +0200
This was already mostly done, but some error messages were printed the
long way.
M src/bin/pg_amcheck/pg_amcheck.c
doc: Fix typos
commit : 9bd563aa9f0694994a6640946a7ee3dc0431d507
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 23 Apr 2021 09:28:44 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 23 Apr 2021 09:28:44 +0200
Author: Justin Pryzby <pryzby@telsasoft.com>
M doc/src/sgml/func.sgml
doc: Fix obsolete description about pg_basebackup.
commit : eaec48b3c54eec222d64468b57af80ee4ddf76a9
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 23 Apr 2021 15:45:46 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 23 Apr 2021 15:45:46 +0900
Previously it was documented that if using "-X none" option there was
no guarantee that all required WAL files were archived at the end of
pg_basebackup when taking a backup from the standby. But this limitation
was removed by commit 52f8a59dd9. Now, even when taking a backup
from the standby, pg_basebackup can wait for all required WAL files
to be archived. Therefore this commit removes such obsolete
description from the docs.
Also this commit adds new description about the limitation when
taking a backup from the standby, into the docs. The limitation is that
pg_basebackup cannot force the standbfy to switch to a new WAL file
at the end of backup, which may cause pg_basebackup to wait a long
time for the last required WAL file to be switched and archived,
especially when write activity on the primary is low.
Back-patch to v10 where the issue was introduced.
Reported-by: Kyotaro Horiguchi
Author: Kyotaro Horiguchi, Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/20210420.133235.1342729068750553399.horikyota.ntt@gmail.com
M doc/src/sgml/ref/pg_basebackup.sgml
Fix incorrect format placeholder
commit : 7776a23a4bdeb7215e4f8ddea5989cb143becc12
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 23 Apr 2021 07:21:13 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 23 Apr 2021 07:21:13 +0200
M src/backend/utils/misc/guc-file.l
Fix some comments in fmgr.c
commit : 45c0c5f70eb5e22d31be8bb9a8b4d9c66a3e9b37
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 23 Apr 2021 13:34:02 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 23 Apr 2021 13:34:02 +0900
Oversight in 2a0faed.
Author: Hou Zhijie
Discussion: https://postgr.es/m/OS0PR01MB5716405E2464D85E6DB6DC0794469@OS0PR01MB5716.jpnprd01.prod.outlook.com
M src/backend/utils/fmgr/fmgr.c
Remove use of [U]INT64_FORMAT in some translatable strings
commit : 62aa2bb293148c13851484e63db4835e3c53147f
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 23 Apr 2021 13:25:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 23 Apr 2021 13:25:49 +0900
%lld with (long long), or %llu with (unsigned long long) are more
adapted. This is similar to 3286065.
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20210421.200000.1462448394029407895.horikyota.ntt@gmail.com
M contrib/pg_prewarm/pg_prewarm.c
M src/backend/access/transam/xlogprefetch.c
M src/bin/pgbench/pgbench.c
Minor code cleanup in asynchronous execution support.
commit : bb684c82f73316b88f9bcb321128a4347b5206fe
author : Etsuro Fujita <efujita@postgresql.org>
date : Fri, 23 Apr 2021 12:00:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Fri, 23 Apr 2021 12:00:00 +0900
This is cleanup for commit 27e1f1456:
* ExecAppendAsyncEventWait(), which was modified a bit further by commit
a8af856d3, duplicated the same nevents calculation. Simplify the code
a little bit to avoid the duplication. Update comments there.
* Add an assertion to ExecAppendAsyncRequest().
* Update a comment about merging the async_capable options from input
relations in merge_fdw_options(), per complaint from Kyotaro Horiguchi.
* Add a comment for fetch_more_data_begin().
Author: Etsuro Fujita
Discussion: https://postgr.es/m/CAPmGK1637W30Wx3MnrReewhafn6F_0J76mrJGoFXFnpPq4QfvA%40mail.gmail.com
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/executor/nodeAppend.c
Don't crash on reference to an un-available system column.
commit : d479d00285255d422a2b38f1cfaa35808968a08c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Apr 2021 17:30:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Apr 2021 17:30:42 -0400
Adopt a more consistent policy about what slot-type-specific
getsysattr functions should do when system attributes are not
available. To wit, they should all throw the same user-oriented
error, rather than variously crashing or emitting developer-oriented
messages.
This closes a identifiable problem in commits a71cfc56b and
3fb93103a (in v13 and v12), so back-patch into those branches,
along with a test case to try to ensure we don't break it again.
It is not known that any of the former crash cases are reachable
in HEAD, but this seems like a good safety improvement in any case.
Discussion: https://postgr.es/m/141051591267657@mail.yandex.ru
M src/backend/executor/execTuples.c
M src/test/regress/expected/update.out
M src/test/regress/sql/update.sql
Fix some trailing whitespace in documentation files
commit : 197d33ccbe888fc84ae4e49bb241e88ea3c81f15
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 22 Apr 2021 22:47:57 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 22 Apr 2021 22:47:57 +0200
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/ref/reindex.sgml
M doc/src/sgml/ref/vacuum.sgml
M doc/src/sgml/wal.sgml
Fix uninitialized memory bug
commit : 43b55ec4bc3bc06596d966391f16defe016310ec
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 22 Apr 2021 16:04:48 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 22 Apr 2021 16:04:48 -0400
Have interested callers of find_inheritance_children set the
detached_exist value to false prior to calling it, so that that routine
only has to set it true in the rare cases where it is necessary. Don't
touch it otherwise.
Per buildfarm member thorntail (which reported a UBSan failure here).
M src/backend/catalog/pg_inherits.c
M src/backend/partitioning/partdesc.c
doc: mention <link> can be inside of <command>, but not <xref>
commit : 84f15ccd4c25c4ffc4de6ed82f7658a3a199a1d7
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 22 Apr 2021 16:01:17 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 22 Apr 2021 16:01:17 -0400
This was discussed in commit 9081bddbd.
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/flat/87o8pco34z.fsf@wibble.ilmari.org
M doc/src/sgml/README.links
Make PostgresVersion code a bit more robust and simple.
commit : 502dc6df8f6eeba06812ce09488efc7e684f5ec9
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 22 Apr 2021 15:25:37 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 22 Apr 2021 15:25:37 -0400
per gripe from Alvaro Herrera.
M src/test/perl/PostgresVersion.pm
Fix relcache inconsistency hazard in partition detach
commit : 8aba9322511f718f12b618470d8c07f0ee5f0700
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 22 Apr 2021 15:13:25 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 22 Apr 2021 15:13:25 -0400
During queries coming from ri_triggers.c, we need to omit partitions
that are marked pending detach -- otherwise, the RI query is tricked
into allowing a row into the referencing table whose corresponding row
is in the detached partition. Which is bogus: once the detach operation
completes, the row becomes an orphan.
However, the code was not doing that in repeatable-read transactions,
because relcache kept a copy of the partition descriptor that included
the partition, and used it in the RI query. This commit changes the
partdesc cache code to only keep descriptors that aren't dependent on
a snapshot (namely: those where no detached partition exist, and those
where detached partitions are included). When a partdesc-without-
detached-partitions is requested, we create one afresh each time; also,
those partdescs are stored in PortalContext instead of
CacheMemoryContext.
find_inheritance_children gets a new output *detached_exist boolean,
which indicates whether any partition marked pending-detach is found.
Its "include_detached" input flag is changed to "omit_detached", because
that name captures desired the semantics more naturally.
CreatePartitionDirectory() and RelationGetPartitionDesc() arguments are
identically renamed.
This was noticed because a buildfarm member that runs with relcache
clobbering, which would not keep the improperly cached partdesc, broke
one test, which led us to realize that the expected output of that test
was bogus. This commit also corrects that expected output.
Author: Amit Langote <amitlangote09@gmail.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/3269784.1617215412@sss.pgh.pa.us
M src/backend/catalog/heap.c
M src/backend/catalog/pg_inherits.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/executor/execPartition.c
M src/backend/optimizer/util/plancat.c
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partdesc.c
M src/include/catalog/pg_inherits.h
M src/include/partitioning/partdesc.h
M src/test/isolation/expected/detach-partition-concurrently-4.out
Doc: document the tie-breaking behavior of the round() function.
commit : 82b13dbc4d4b46f71ca95ce1cc15c425deff5957
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Apr 2021 14:47:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Apr 2021 14:47:26 -0400
Back-patch to v13; the table layout in older branches is unfriendly
to adding such details.
Laurenz Albe
Discussion: https://postgr.es/m/161881920775.685.12293798764864559341@wrigleys.postgresql.org
M doc/src/sgml/func.sgml
Make PostgresNode version aware
commit : 4c4eaf3d19201c5e2d9efebc590903dfaba0d3e5
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 22 Apr 2021 10:56:28 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 22 Apr 2021 10:56:28 -0400
A new PostgresVersion object type is created and this is used in
PostgresNode using the output of `pg_config --version` and the result
stored in the PostgresNode object. This object can be compared to other
PostgresVersion objects, or to a number or string.
PostgresNode is currently believed to be compatible with versions down
to release 12, so PostgresNode will issue a warning if used with a
version prior to that.
No attempt has been made to deal with incompatibilities in older
versions - that remains work to be undertaken in a subsequent
development cycle.
Based on code from Mark Dilger and Jehan-Guillaume de Rorthais.
Discussion: https://postgr.es/m/a80421c0-3d7e-def1-bcfe-24777f15e344@dunslane.net
M src/test/perl/PostgresNode.pm
A src/test/perl/PostgresVersion.pm
Fix relation leak for subscribers firing triggers in logical replication
commit : f3b141c482552a57866c72919007d6481cd59ee3
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 22 Apr 2021 12:48:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 22 Apr 2021 12:48:54 +0900
Creating a trigger on a relation to which an apply operation is
triggered would cause a relation leak once the change gets committed,
as the executor would miss that the relation needs to be closed
beforehand. This issue got introduced with the refactoring done in
1375422c, where it becomes necessary to track relations within
es_opened_result_relations to make sure that they are closed.
We have discussed using ExecInitResultRelation() coupled with
ExecCloseResultRelations() for the relations in need of tracking by the
apply operations in the subscribers, which would simplify greatly the
opening and closing of indexes, but this requires a larger rework and
reorganization of the worker code, particularly for the tuple routing
part. And that's not really welcome post feature freeze. So, for now,
settle down to the same solution as TRUNCATE which is to fill in
es_opened_result_relations with the relation opened, to make sure that
ExecGetTriggerResultRel() finds them and that they get closed.
The code is lightly refactored so as a relation is not registered three
times for each DML code path, making the whole a bit easier to follow.
Reported-by: Tang Haiying, Shi Yu, Hou Zhijie
Author: Amit Langote, Masahiko Sawada, Hou Zhijie
Reviewed-by: Amit Kapila, Michael Paquier
Discussion: https://postgr.es/m/OS0PR01MB611383FA0FE92EB9DE21946AFB769@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/backend/replication/logical/worker.c
doc: Move parallel_leader_participation to its correct category
commit : 1599e7b375127cac81b539d2c69d3faf7598509b
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 22 Apr 2021 09:47:43 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 22 Apr 2021 09:47:43 +0900
parallel_leader_participation got introduced in e5253fd, where it was
listed under RESOURCES_ASYNCHRONOUS in guc.c, but the documentation
did not reflect that and listed it with the other planner-related
options. This commit fixes this inconsistency as the parameter is
intended to be an asynchronous one.
While on it, reorganize a bit the section dedicated to asynchronous
parameters, backend_flush_after being moved first to do better in terms
of alphabetical order of the options listed.
Reported-by: Yanliang Lei
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/16972-42d4b0c15aa1d5f5@postgresql.org
M doc/src/sgml/config.sgml
Add comment about extract_autovac_opts not holding lock
commit : 7c298c6573a0f181963ddcb40c850fa9c7da0ada
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 21 Apr 2021 18:36:12 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 21 Apr 2021 18:36:12 -0400
Per observation from Tom Lane.
Discussion: https://postgr.es/m/1901125.1617904665@sss.pgh.pa.us
M src/backend/postmaster/autovacuum.c
Don't add a redundant constraint when detaching a partition
commit : 7b357cc6ae553c0ecacdc11b2e5278b7bf477dba
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 21 Apr 2021 18:12:05 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 21 Apr 2021 18:12:05 -0400
On ALTER TABLE .. DETACH CONCURRENTLY, we add a new table constraint
that duplicates the partition constraint. But if the partition already
has another constraint that implies that one, then that's unnecessary.
We were already avoiding the addition of a duplicate constraint if there
was an exact 'equal' match -- this just improves the quality of the check.
Author: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20210410184226.GY6592@telsasoft.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
fix silly perl error in commit d064afc720
commit : e014d25deade08df082d2b37de45adb0c984f563
author : Andrew Dunstan <andrew@dunslane.net>
date : Wed, 21 Apr 2021 11:12:04 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Wed, 21 Apr 2021 11:12:04 -0400
M src/test/perl/PostgresNode.pm
Update config.guess and config.sub
commit : 26ac261ee4033710cad44f7924d53753129b60c7
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 21 Apr 2021 16:02:03 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 21 Apr 2021 16:02:03 +0200
M config/config.guess
M config/config.sub
Only ever test for non-127.0.0.1 addresses on Windows in PostgresNode
commit : d064afc7204b52cb78a83fea0e686693ce5ba00c
author : Andrew Dunstan <andrew@dunslane.net>
date : Wed, 21 Apr 2021 10:21:22 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Wed, 21 Apr 2021 10:21:22 -0400
This has been found to cause hangs where tcp usage is forced.
Alexey Kodratov
Discussion: https://postgr.es/m/82e271a9a11928337fcb5b5e57b423c0@postgrespro.ru
Backpatch to all live branches
M src/test/perl/PostgresNode.pm
Add DISTINCT to information schema usage views
commit : d84ffffe582b8e036a14c6bc2378df29167f3a00
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 21 Apr 2021 11:54:47 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 21 Apr 2021 11:54:47 +0200
Since pg_depend can contain duplicate entries, we need to eliminate
those in information schema views that build on pg_depend, using
DISTINCT. Some of the older views already did that correctly, but
some of the more recently added ones didn't. (In some of these views,
it might not be possible to reproduce the issue because of how the
implementation happens to deduplicate dependencies while recording
them, but it seems better to keep this consistent in all cases.)
M src/backend/catalog/information_schema.sql
M src/include/catalog/catversion.h
Use correct format placeholder for timeline IDs
commit : 39d0928a0e88426ee64189898565c40d4af9ad96
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 21 Apr 2021 08:26:18 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 21 Apr 2021 08:26:18 +0200
Should be %u rather than %d.
M src/bin/pg_rewind/pg_rewind.c
doc: Improve hyphenation consistency
commit : 544b28088f9d41750ccf193812da62bdfe4bd98a
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 21 Apr 2021 08:14:43 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 21 Apr 2021 08:14:43 +0200
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/alter_policy.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_policy.sgml
M doc/src/sgml/ref/drop_policy.sgml
M doc/src/sgml/rules.sgml
M src/backend/commands/copyto.c
M src/backend/commands/functioncmds.c
M src/backend/executor/execMain.c
M src/backend/optimizer/path/allpaths.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/rewrite/rowsecurity.c
M src/include/catalog/pg_authid.h
M src/test/regress/expected/rowsecurity.out
M src/test/regress/sql/rowsecurity.sql
Don't use INT64_FORMAT inside message strings
commit : 3286065651477c2060910dfb42b3cedbd79a7980
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Apr 2021 22:48:13 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Apr 2021 22:48:13 +0200
Use %lld and cast to long long int instead.
M src/bin/pg_rewind/libpq_source.c
Fix typo
commit : f0ec598b4323d8b29df5c67f2cd0000547a507ed
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Apr 2021 22:47:43 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Apr 2021 22:47:43 +0200
M src/backend/access/transam/xlogprefetch.c
doc: List compute_query_id in required config for pg_stat_statements
commit : 64087eb5def7786bd49e60eb5d984ec6e4a872a9
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 21 Apr 2021 12:02:41 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 21 Apr 2021 12:02:41 +0900
Not enabling compute_query_id would disable pg_stat_statements even if
the module is listed in shared_preload_libraries, so add it to the
minimum configuration set as listed in its documentation.
Author: Greg Nancarrow
Reviewed-by: Julien Rouhaud, Bharath Rupireddy
Discussion: https://postgr.es/m/CAJcOf-fXyb2QiDbwftD813UF70w-+BsK-03bFp1GrijXU9GQYQ@mail.gmail.com
M doc/src/sgml/pgstatstatements.sgml
Add CURRENT_ROLE to list of roles for tab completion of GRANT in psql
commit : 22b2dec31b2ef4dfee299290b16375e9fa6e6932
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 21 Apr 2021 10:34:43 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 21 Apr 2021 10:34:43 +0900
This compatibility has been added in 45b9805, but psql forgot the call.
Author: Wei Wang
Reviewed-by: Aleksander Alekseev
Discussion: https://postgr.es/m/OS3PR01MB6275935F62E161BCD393D6559E489@OS3PR01MB6275.jpnprd01.prod.outlook.com
M src/bin/psql/tab-complete.c
Improve WAL record descriptions for SP-GiST records.
commit : 783be78ca91166ac7f80c953f2bbc5af1f61c6cd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 20 Apr 2021 17:01:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 20 Apr 2021 17:01:43 -0400
While tracking down the bug fixed in the preceding commit, I got quite
annoyed by the low quality of spg_desc's output. Add missing fields,
try to make the formatting consistent.
M src/backend/access/rmgrdesc/spgdesc.c
Fix under-parenthesized XLogRecHasBlockRef() macro.
commit : 9e41148229192dccc4bcc40f53af588b73d8ffea
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 20 Apr 2021 16:58:30 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 20 Apr 2021 16:58:30 -0400
Commit f003d9f87 left this macro with inadequate (or, one could say,
too much) parenthesization. Which was catastrophic to the correctness
of calls such as "if (!XLogRecHasBlockRef(record, 1)) ...". There
are only a few of those, which perhaps explains why we didn't notice
immediately (with our general weakness of WAL replay testing being
another factor). I found it by debugging intermittent replay failures
like
2021-04-08 14:33:30.191 EDT [29463] PANIC: failed to locate backup block with ID 1
2021-04-08 14:33:30.191 EDT [29463] CONTEXT: WAL redo at 0/95D3438 for SPGist/ADD_NODE: off 1; blkref #0: rel 1663/16384/25998, blk 1
M src/include/access/xlogreader.h
Fix interaction of log_line_prefix's query_id and log_statement
commit : db01f797dd48f826c62e1b8eea70f11fe7ff3efc
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 20 Apr 2021 12:57:59 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 20 Apr 2021 12:57:59 -0400
log_statement is issued before query_id can be computed, so properly
clear the value, and document the interaction.
Reported-by: Fujii Masao, Michael Paquier
Discussion: https://postgr.es/m/YHPkU8hFi4no4NSw@paquier.xyz
Author: Julien Rouhaud
M doc/src/sgml/config.sgml
M src/backend/utils/activity/backend_status.c
adjust query id feature to use pg_stat_activity.query_id
commit : 9660834dd8bf5b093f7b49eef846666201d45a35
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 20 Apr 2021 12:22:26 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 20 Apr 2021 12:22:26 -0400
Previously, it was pg_stat_activity.queryid to match the
pg_stat_statements queryid column. This is an adjustment to patch
4f0b0966c8. This also adjusts some of the internal function calls to
match. Catversion bumped.
Reported-by: Álvaro Herrera, Julien Rouhaud
Discussion: https://postgr.es/m/20210408032704.GA7498@alvherre.pgsql
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/executor/execMain.c
M src/backend/executor/execParallel.c
M src/backend/parser/analyze.c
M src/backend/tcop/postgres.c
M src/backend/utils/activity/backend_status.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/error/elog.c
M src/backend/utils/misc/queryjumble.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/utils/backend_status.h
M src/test/regress/expected/rules.out
Rename find_em_expr_usable_for_sorting_rel.
commit : 7645376774c8532159f5f0f905e5e734d4ccbb18
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 20 Apr 2021 11:37:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 20 Apr 2021 11:37:36 -0400
I didn't particularly like this function name, as it fails to
express what's going on. Also, returning the sort expression
alone isn't too helpful --- typically, a caller would also
need some other fields of the EquivalenceMember. But the
sole caller really only needs a bool result, so let's make
it "bool relation_can_be_sorted_early()".
Discussion: https://postgr.es/m/91f3ec99-85a4-fa55-ea74-33f85a5c651f@swarm64.com
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/equivclass.c
M src/include/optimizer/paths.h
Fix planner failure in some cases of sorting by an aggregate.
commit : 375398244168add84a884347625d14581a421e71
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 20 Apr 2021 11:32:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 20 Apr 2021 11:32:02 -0400
An oversight introduced by the incremental-sort patches caused
"could not find pathkey item to sort" errors in some situations
where a sort key involves an aggregate or window function.
The basic problem here is that find_em_expr_usable_for_sorting_rel
isn't properly modeling what prepare_sort_from_pathkeys will do
later. Rather than hoping we can keep those functions in sync,
let's refactor so that they actually share the code for
identifying a suitable sort expression.
With this refactoring, tlist.c's tlist_member_ignore_relabel
is unused. I removed it in HEAD but left it in place in v13,
in case any extensions are using it.
Per report from Luc Vlaming. Back-patch to v13 where the
problem arose.
James Coleman and Tom Lane
Discussion: https://postgr.es/m/91f3ec99-85a4-fa55-ea74-33f85a5c651f@swarm64.com
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/tlist.c
M src/include/optimizer/paths.h
M src/include/optimizer/tlist.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Avoid unfortunate IPC::Run path caching in PostgresNode
commit : 95c3a1956ec9eac686c1b69b033dd79211b72343
author : Andrew Dunstan <andrew@dunslane.net>
date : Tue, 20 Apr 2021 10:14:16 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Tue, 20 Apr 2021 10:14:16 -0400
Commit b34ca595ab provided for installation-aware instances of
PostgresNode. However, it turns out that IPC::Run works against this by
caching the path to a binary and not consulting the path again, even if
it has changed. We work around this by calling Postgres binaries with
the installed path rather than just a bare name to be looked up in the
environment path, if there is an installed path. For the common case
where there is no installed path we continue to use the bare command
name.
Diagnosis and solution from Mark Dilger
Discussion: https://postgr.es/m/E8F512F8-B4D6-4514-BA8D-2E671439DA92@enterprisedb.com
M src/test/perl/PostgresNode.pm
Fix typo in comment
commit : 8b4b5669cde2b17bd6b5d68f584d97078f3296ac
author : Magnus Hagander <magnus@hagander.net>
date : Tue, 20 Apr 2021 14:35:16 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Tue, 20 Apr 2021 14:35:16 +0200
Author: Julien Rouhaud
Backpatch-through: 11
Discussion: https://postgr.es/m/20210420121659.odjueyd4rpilorn5@nol
M src/backend/lib/dshash.c
Document LP_DEAD accounting issues in VACUUM.
commit : 7136bf34f28892362144ae2e350714836a5c0c0c
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 19 Apr 2021 18:55:31 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 19 Apr 2021 18:55:31 -0700
Document VACUUM's soft assumption that any LP_DEAD items encountered
during pruning will become LP_UNUSED items before VACUUM finishes up.
This is integral to the accounting used by VACUUM to generate its final
report on the table to the stats collector. It also affects how VACUUM
determines which heap pages are truncatable. In both cases VACUUM is
concerned with the likely contents of the page in the near future, not
the current contents of the page.
This state of affairs created the false impression that VACUUM's dead
tuple accounting had significant difference with similar accounting used
during ANALYZE. There were and are no substantive differences, at least
when the soft assumption completely works out. This is far clearer now.
Also document cases where things don't quite work out for VACUUM's dead
tuple accounting. It's possible that a significant number of LP_DEAD
items will be left behind by VACUUM, and won't be recorded as remaining
dead tuples in VACUUM's statistics collector report. This behavior
dates back to commit a96c41fe, which taught VACUUM to run without index
and heap vacuuming at the user's request. The failsafe mechanism added
to VACUUM more recently by commit 1e55e7d1 takes the same approach to
dead tuple accounting.
Reported-By: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wz=Jmtu18PrsYq3EvvZJGOmZqSO2u3bvKpx9xJa5uhNp=Q@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Use correct format placeholder for pids
commit : 640b91c3ed24002b34c7226fb51ef9176fb72713
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Apr 2021 10:43:18 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Apr 2021 10:43:18 +0200
Should be signed, not unsigned.
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/bgworker.c
M src/backend/replication/logical/snapbuild.c
Fix test case added by commit f5fc2f5b23.
commit : c64dcc7fee5f8a7941a4fd098a969de1f457cc79
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 19 Apr 2021 09:02:47 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 19 Apr 2021 09:02:47 +0530
In the new test after resetting the stats, we were not waiting for the
stats message to be delivered. Also, we need to decode the results for
the new test, otherwise, it will show the old stats.
In passing,
a. Change docs added by commit f5fc2f5b23 as per suggestion by
Justin Pryzby.
b. Bump the PGSTAT_FILE_FORMAT_ID as commit f5fc2f5b23 changes the file
format of stats.
Reported-by: Tom Lane based on buildfarm reports
Author: Vignesh C, Justin Pryzby
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/20210319185247.ldebgpdaxsowiflw@alap3.anarazel.de
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
M doc/src/sgml/monitoring.sgml
M src/include/pgstat.h
Fix typos and grammar in comments and docs
commit : 7ef8b52cf079ef3ace4575f7b97c2d6f80463b4f
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 19 Apr 2021 11:32:30 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 19 Apr 2021 11:32:30 +0900
Author: Justin Pryzby
Discussion: https://postgr.es/m/20210416070310.GG3315@telsasoft.com
M contrib/amcheck/verify_heapam.c
M doc/src/sgml/brin.sgml
M doc/src/sgml/ecpg.sgml
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_bloom.c
M src/backend/access/brin/brin_minmax_multi.c
M src/backend/access/gist/gistbuild.c
M src/backend/access/index/genam.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/catalog/pg_type.c
M src/backend/commands/analyze.c
M src/backend/executor/nodeIncrementalSort.c
M src/backend/rewrite/rewriteSearchCycle.c
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/storage/ipc/procarray.c
M src/backend/tsearch/spell.c
M src/backend/utils/activity/backend_status.c
M src/backend/utils/adt/multirangetypes.c
M src/backend/utils/adt/selfuncs.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_waldump/pg_waldump.c
M src/common/hmac_openssl.c
M src/common/pg_lzcompress.c
M src/interfaces/ecpg/preproc/ecpg.c
M src/port/bsearch_arg.c
Replace magic constants for seek() calls in perl scripts
commit : c731f9187b5fd7038b04ba60703d3cace1806366
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 19 Apr 2021 10:15:35 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 19 Apr 2021 10:15:35 +0900
A couple of tests have been using 0 as magic constant while SEEK_SET can
be used instead. This makes the code easier to understand, and more
consistent with the changes done in 3c5b068.
Per discussion with Andrew Dunstan.
Discussion: https://postgr.es/m/YHrc24AgJQ6tQ1q0@paquier.xyz
M contrib/amcheck/t/001_verify_heapam.pl
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_checksums/t/002_actions.pl
Explain postmaster's treatment of SIGURG.
commit : 8e861eaae86eeaf5589963c9b1c7ce6d4c2acbb5
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 19 Apr 2021 10:22:31 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 19 Apr 2021 10:22:31 +1200
Add a few words of comment to explain why SIGURG doesn't follow the
dummy_handler pattern used for SIGUSR2, since that might otherwise
appear to be a bug.
Discussion: https://postgr.es/m/4006115.1618577212%40sss.pgh.pa.us
M src/backend/postmaster/postmaster.c
Add missing source files to nls.mk
commit : 4ed7f0599a8984d9ed967780a157d9b23d03fbb5
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 18 Apr 2021 16:11:58 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 18 Apr 2021 16:11:58 +0200
M src/bin/pg_amcheck/nls.mk
M src/bin/scripts/nls.mk
doc: Fix up spacing around verbatim DocBook elements
commit : f7c09706c14d0858d5a186f3cc769471cba41578
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 17 Apr 2021 14:14:26 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 17 Apr 2021 14:14:26 +0200
M doc/src/sgml/libpq.sgml
M doc/src/sgml/logicaldecoding.sgml
Use correct format placeholder for block numbers
commit : f59b58e2a1b7e4a48dee36cc61966759da0faedd
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 17 Apr 2021 09:40:50 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 17 Apr 2021 09:40:50 +0200
Should be %u rather than %d.
M src/backend/access/gist/gistbuild.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/replication/basebackup.c
Rethink extraction of collation dependencies.
commit : f24b156997059c257c697b825f022d115825091d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Apr 2021 22:23:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Apr 2021 22:23:46 -0400
As it stands, find_expr_references_walker() pays attention to leaf-node
collation fields while ignoring the input collations of actual function
and operator nodes. That seems exactly backwards from a semantic
standpoint, and it leads to reporting dependencies on collations that
really have nothing to do with the expression's behavior.
Hence, rewrite to look at function input collations instead. This
isn't completely perfect either; it fails to account for the behavior
of record_eq and its siblings. (The previous coding at least gave an
approximation of that, though I think it could be fooled pretty easily
into considering the columns of irrelevant composite types.) We may
be able to improve on this later, but for now this should satisfy the
buildfarm members that didn't like ef387bed8.
In passing fix some oversights in GetTypeCollations(), and get
rid of its duplicative de-duplications. (I'm worried that it's
still potentially O(N^2) or worse, but this makes it a little
better.)
Discussion: https://postgr.es/m/3564817.1618420687@sss.pgh.pa.us
M src/backend/catalog/dependency.c
M src/backend/catalog/pg_type.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Update dummy prosrc values.
commit : 8a2df442b652f83b1c189822737091b90f343965
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Apr 2021 19:01:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Apr 2021 19:01:22 -0400
Ooops, forgot to s/system_views.sql/system_functions.sql/g
in this part of 767982e36.
No need for an additional catversion bump, I think, since
these strings are gone by the time initdb finishes.
Discussion: https://postgr.es/m/3956760.1618529139@sss.pgh.pa.us
M src/include/catalog/pg_proc.dat
Convert built-in SQL-language functions to SQL-standard-body style.
commit : 767982e36298be4da44a063e36261e9cfdc0bf49
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Apr 2021 18:36:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Apr 2021 18:36:45 -0400
Adopt the new pre-parsed representation for all built-in and
information_schema SQL-language functions, except for a small
number that can't presently be converted because they have
polymorphic arguments.
This eliminates residual hazards around search-path safety of
these functions, and might provide some small performance benefits
by reducing parsing costs. It seems useful also to provide more
test coverage for the SQL-standard-body feature.
Discussion: https://postgr.es/m/3956760.1618529139@sss.pgh.pa.us
M src/backend/catalog/information_schema.sql
M src/backend/catalog/system_functions.sql
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Split function definitions out of system_views.sql into a new file.
commit : e80949372564c126c92aa7d64de483e04c0ef95e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Apr 2021 18:20:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Apr 2021 18:20:42 -0400
Invent system_functions.sql to carry the function definitions that
were formerly in system_views.sql. The function definitions were
already a quarter of the file and are about to be more, so it seems
appropriate to give them their own home.
In passing, fix an oversight in dfb75e478: it neglected to call
check_input() for system_constraints.sql.
Discussion: https://postgr.es/m/3956760.1618529139@sss.pgh.pa.us
M src/backend/catalog/Makefile
A src/backend/catalog/system_functions.sql
M src/backend/catalog/system_views.sql
M src/bin/initdb/initdb.c
Allow TestLib::slurp_file to skip contents, and use as needed
commit : 3c5b0685b921533f37622345beb0f8dd49200c01
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 16 Apr 2021 16:54:04 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 16 Apr 2021 16:54:04 -0400
In order to avoid getting old logfile contents certain functions in
PostgresNode were doing one of two things. On Windows it rotated the
logfile and restarted the server, while elsewhere it truncated the log
file. Both of these are unnecessary. We borrow from the buildfarm which
does this instead: note the size of the logfile before we start, and
then when fetching the logfile skip to that position before accumulating
contents. This is spelled differently on Windows but the effect is the
same. This is largely centralized in TestLib's slurp_file function,
which has a new optional parameter, the offset to skip to before
starting to reading the file. Code in the client becomes much neater.
Backpatch to all live branches.
Michael Paquier, slightly modified by me.
Discussion: https://postgr.es/m/YHajnhcMAI3++pJL@paquier.xyz
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
Fix bogus collation-version-recording logic.
commit : ef387bed87f2787b1012e68e9a33607a1074c123
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Apr 2021 12:26:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Apr 2021 12:26:50 -0400
recordMultipleDependencies had the wrong scope for its "version"
variable, allowing a version label to leak from the collation entry it
was meant for to subsequent non-collation entries. This is relatively
hard to trigger because of the OID-descending order that the inputs
will normally arrive in: subsequent non-collation items will tend to
be pinned. But it can be exhibited easily with a custom collation.
Also, don't special-case the default collation, but instead ignore
pinned-ness of a collation when we've found a version for it. This
avoids creating useless pg_depend entries, and removes a not-very-
future-proof assumption that C, POSIX, and DEFAULT are the only
pinned collations.
A small problem is that, because the default collation may or may
not have a version, the regression tests can't assume anything about
whether dependency entries will be made for it. This seems OK though
since it's now handled just the same as other collations, and we have
test cases for both versioned and unversioned collations.
Fixes oversights in commit 257836a75. Thanks to Julien Rouhaud
for review.
Discussion: https://postgr.es/m/3564817.1618420687@sss.pgh.pa.us
M src/backend/catalog/pg_depend.c
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Fix wrong units in two ExplainPropertyFloat calls.
commit : f90c708a048667befbf6bbe5f48ae9695cb89de4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Apr 2021 11:30:27 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Apr 2021 11:30:27 -0400
This is only a latent bug, since these calls are only reached for
non-text output formats, and currently none of those will print
the units. Still, we should get it right in case that ever changes.
Justin Pryzby
Discussion: https://postgr.es/m/20210415163846.GA3315@telsasoft.com
M src/backend/commands/explain.c
psql: Refine lexing of BEGIN...END blocks in CREATE FUNCTION statements
commit : 029c5ac03db72f1898ee17e417650a2e0764b239
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 16 Apr 2021 11:46:01 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 16 Apr 2021 11:46:01 +0200
Only track BEGIN...END blocks if they are in a CREATE [OR REPLACE]
{FUNCTION|PROCEDURE} statement. Ignore if in parentheses.
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://www.postgresql.org/message-id/cee01d26fe55bc086b3bcf10bfe4e8d450e2f608.camel@cybertec.at
M src/fe_utils/psqlscan.l
M src/include/fe_utils/psqlscan_int.h
psql: Small fixes for better translatability
commit : 25593d7d338232fb855ba563b325237de8f14091
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 16 Apr 2021 11:04:04 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 16 Apr 2021 11:04:04 +0200
M src/bin/psql/describe.c
M src/bin/psql/help.c
M src/test/regress/expected/psql.out
doc: Fix typo in example query of SQL/JSON
commit : 254a2164e5b216ecf98882f8bcb9e239ab2d432d
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 16 Apr 2021 16:56:12 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 16 Apr 2021 16:56:12 +0900
Author: Erik Rijkers
Discussion: https://postgr.es/m/1219476687.20432.1617452918468@webmailclassic.xs4all.nl
Backpatch-through: 12
M doc/src/sgml/json.sgml
Add information of total data processed to replication slot stats.
commit : f5fc2f5b23d1b1dff60f8ca5dc211161df47eda4
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 16 Apr 2021 07:34:43 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 16 Apr 2021 07:34:43 +0530
This adds the statistics about total transactions count and total
transaction data logically sent to the decoding output plugin from
ReorderBuffer. Users can query the pg_stat_replication_slots view to check
these stats.
Suggested-by: Andres Freund
Author: Vignesh C and Amit Kapila
Reviewed-by: Sawada Masahiko, Amit Kapila
Discussion: https://postgr.es/m/20210319185247.ldebgpdaxsowiflw@alap3.anarazel.de
M contrib/test_decoding/Makefile
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
A contrib/test_decoding/t/001_repl_stats.pl
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/replication/reorderbuffer.h
M src/test/regress/expected/rules.out
Doc: Document known problem with Windows collation versions.
commit : 1bf946bd43e545b86e567588b791311fe4e36a8c
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 16 Apr 2021 13:20:58 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 16 Apr 2021 13:20:58 +1200
Warn users that locales with traditional Windows NLS names like
"English_United States.1252" won't provide version information, and that
something like initdb --lc-collate=en-US would be needed to fix that
problem for the initial template databases.
Discussion: https://postgr.es/m/CA%2BhUKGJ_hk3rU%3D%3Dg2FpAMChb_4i%2BTJacpjjqFsinY-tRM3FBmA%40mail.gmail.com
M doc/src/sgml/charset.sgml
Provide query source text when parsing a SQL-standard function body.
commit : 409723365b2708acd3bdf2e830257504bdefac4b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 15 Apr 2021 17:24:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 15 Apr 2021 17:24:12 -0400
Without this, we lose error cursor positions, as shown in the
modified regression test result.
Discussion: https://postgr.es/m/2197698.1617984583@sss.pgh.pa.us
M src/backend/commands/functioncmds.c
M src/test/regress/expected/create_function_3.out
Revert "Cope with NULL query string in ExecInitParallelPlan()."
commit : 83efce7a1ebc5bae79617ddba12a64790141725c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 15 Apr 2021 17:17:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 15 Apr 2021 17:17:45 -0400
This reverts commit b3ee4c503872f3d0a5d6a7cbde48815f555af15b.
We don't need it in the wake of the preceding commit, which
added an upstream check that the querystring isn't null.
Discussion: https://postgr.es/m/2197698.1617984583@sss.pgh.pa.us
M src/backend/executor/execParallel.c
Undo decision to allow pg_proc.prosrc to be NULL.
commit : 1111b2668d89bfcb6f502789158b1233ab4217a6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 15 Apr 2021 17:17:20 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 15 Apr 2021 17:17:20 -0400
Commit e717a9a18 changed the longstanding rule that prosrc is NOT NULL
because when a SQL-language function is written in SQL-standard style,
we don't currently have anything useful to put there. This seems a poor
decision though, as it could easily have negative impacts on external
PLs (opening them to crashes they didn't use to have, for instance).
SQL-function-related code can just as easily test "is prosqlbody not
null" as "is prosrc null", so there's no real gain there either.
Hence, revert the NOT NULL marking removal and adjust related logic.
For now, we just put an empty string into prosrc for SQL-standard
functions. Maybe we'll have a better idea later, although the
history of things like pg_attrdef.adsrc suggests that it's not
easy to maintain a string equivalent of a node tree.
This also adds an assertion that queryDesc->sourceText != NULL
to standard_ExecutorStart. We'd been silently relying on that
for awhile, so let's make it less silent.
Also fix some overlooked documentation and test cases.
Discussion: https://postgr.es/m/2197698.1617984583@sss.pgh.pa.us
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/pg_proc.c
M src/backend/commands/functioncmds.c
M src/backend/executor/execMain.c
M src/backend/executor/functions.c
M src/backend/optimizer/util/clauses.c
M src/bin/pg_dump/pg_dump.c
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.h
M src/include/executor/functions.h
M src/test/regress/expected/create_function_3.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/sql/create_function_3.sql
M src/test/regress/sql/opr_sanity.sql
Stabilize recently-added information_schema test queries.
commit : 3157cbe974846729d49a1ee081944eee1839bdd8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 15 Apr 2021 16:31:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 15 Apr 2021 16:31:36 -0400
These queries could show unexpected entries if the core system,
or concurrently-running test scripts, created any functions that
would appear in the information_schema views. Restrict them
to showing functions belonging to this test's schema, as the
far-older nearby test case does.
Per experimentation with conversion of some built-in functions
to SQL-function-body style.
M src/test/regress/expected/create_function_3.out
M src/test/regress/sql/create_function_3.sql
Revert "psql: Show all query results by default"
commit : fae65629cec824738ee11bf60f757239906d64fa
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 15 Apr 2021 19:41:42 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 15 Apr 2021 19:41:42 +0200
This reverts commit 3a5130672296ed4e682403a77a9a3ad3d21cef75.
Per discussion, this patch had too many issues to resolve at this
point of the development cycle. We'll try again in the future.
Discussion: https://www.postgresql.org/message-id/flat/alpine.DEB.2.21.1904132231510.8961@lancre
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/common.c
M src/bin/psql/help.c
M src/bin/psql/settings.h
M src/bin/psql/startup.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/copyselect.out
M src/test/regress/expected/psql.out
M src/test/regress/expected/transactions.out
M src/test/regress/sql/copyselect.sql
M src/test/regress/sql/psql.sql
M src/test/regress/sql/transactions.sql
doc: Add missing COMPRESSION into CREATE TABLE synopsis.
commit : e2e2efca85b4857361780ed0c736c2a44edb458a
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 15 Apr 2021 23:15:19 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 15 Apr 2021 23:15:19 +0900
Commit bbe0a81db6 introduced "INCLUDING COMPRESSION" option
in CREATE TABLE command, but forgot to mention it in the
CREATE TABLE syntax synopsis.
Author: Fujii Masao
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/54d30e66-dbd6-5485-aaf6-a291ed55919d@oss.nttdata.com
M doc/src/sgml/ref/create_table.sgml
doc: Simplify example of HISTFILE for psql
commit : 1840d9f4c89998872a3b46473f8e9e5b6ff3c144
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 15 Apr 2021 16:45:34 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 15 Apr 2021 16:45:34 +0900
e4c7619 has added a space to the example used for HISTFILE in the docs
of psql before the variable DBNAME, as a workaround because variables
were not parsed the same way back then. This behavior has changed in
9.2, causing the example in the psql docs to result in the same history
file created with or without a space added before the DBNAME variable.
Let's just remove this space in the example, to reduce any confusion, as
the point of it is to prove that a per-database history file is easy to
set up, and that's easier to read this way.
Per discussion with Tom Lane.
Reported-by: Ludovic Kuty
Discussion: https://postgr.es/m/161830067409.691.16198363670687811485@wrigleys.postgresql.org
M doc/src/sgml/ref/psql-ref.sgml
pg_upgrade: Small fix for better translatability of help output
commit : cbae8774eb5c2f5552323ee18ca5286f9c8e5d06
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 15 Apr 2021 09:08:18 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 15 Apr 2021 09:08:18 +0200
M src/bin/pg_upgrade/option.c
amcheck: Use correct format placeholder for TOAST chunk numbers
commit : 59da8d9eb7255c3cb1c9f3b79d76b18b6a1c7da2
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 15 Apr 2021 08:58:03 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 15 Apr 2021 08:58:03 +0200
Several of these were already fixed in passing in
9acaf1a62197205b06a85afbfcaa7ffaac939ef3, but one was remaining
inconsistent.
M contrib/amcheck/verify_heapam.c
Tweak behavior of pg_dump --extension with configuration tables
commit : 344487e2db03f3cec13685a839dbc8a0e2a36750
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 15 Apr 2021 10:03:46 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 15 Apr 2021 10:03:46 +0900
6568cef, that introduced the option, had an inconsistent behavior when
it comes to configuration tables set up by pg_extension_config_dump, as
the data of all configuration tables would included in a dump even for
extensions not listed by a set of --extension switches.
The contents dumped changed depending on the schema where an extension
was installed when an extension was not listed. For example, an
extension installed under the public schema would have its configuration
data not dumped even when not listed with --extension, which was
inconsistent with the case of an extension installed on a non-public
schema, where the configuration would be dumped.
Per discussion with Noah, we have settled down to the simple rule of
dumping configuration data of an extension if it is listed in
--extension (default is unchanged and backward-compatible, to dump
everything on sight if there are no extensions directly listed). This
avoids some weird cases where the dumps depended on a --schema for one.
More tests are added to cover the gap, where we cross-check more
behaviors depending on --schema when an extension is not listed.
Reported-by: Noah Misch
Reviewed-by: Noah Misch
Discussion: https://postgr.es/m/20210404220802.GA728316@rfd.leadboat.com
M doc/src/sgml/ref/pg_dump.sgml
M src/bin/pg_dump/pg_dump.c
M src/test/modules/test_pg_dump/t/001_base.pl
Fix obsolete comments referencing JoinPathExtraData.extra_lateral_rels.
commit : e1623b7d86812ee6ab98a42f38b43a192c149988
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 14 Apr 2021 14:28:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 14 Apr 2021 14:28:17 -0400
That field went away in commit edca44b15, but it seems that
commit 45be99f8c re-introduced some comments mentioning it.
Noted by James Coleman, though this isn't exactly his
proposed new wording. Also thanks to Justin Pryzby for
software archaeology.
Discussion: https://postgr.es/m/CAAaqYe8fxZjq3na+XkNx4C78gDqykH-7dbnzygm9Qa9nuDTePg@mail.gmail.com
M src/backend/optimizer/path/joinpath.c
amcheck: Reword some messages and fix an alignment problem.
commit : 9acaf1a62197205b06a85afbfcaa7ffaac939ef3
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 14 Apr 2021 12:46:31 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 14 Apr 2021 12:46:31 -0400
We don't need to mention the attribute number in these messages, because
there's a dedicated column for that, but we should mention the toast
value ID, because that's really useful for any follow-up troubleshooting
the user wants to do. This also rewords some of the messages to hopefully
read a little better.
Also, use VARATT_EXTERNAL_GET_POINTER in case we're accessing a TOAST
pointer that isn't aligned on a platform that's fussy about alignment,
so that we don't crash while corruption-checking the user's data.
Mark Dilger, reviewed by me.
Discussion: http://postgr.es/m/7D3B9BF6-50D0-4C30-8506-1C1851C7F96F@enterprisedb.com
M contrib/amcheck/verify_heapam.c
M src/bin/pg_amcheck/t/004_verify_heapam.pl
Improve quoting in some error messages
commit : 07e5e66742333ab100a557e6e3f710e92fa1fd92
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 14 Apr 2021 09:10:57 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 14 Apr 2021 09:10:57 +0200
M contrib/amcheck/verify_nbtree.c
M src/backend/access/brin/brin.c
M src/backend/access/index/indexam.c
doc: Move force_parallel_mode to section for developer options
commit : ac725ee0f98c3fec703ffd9b070da629608e9a1e
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 14 Apr 2021 15:55:55 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 14 Apr 2021 15:55:55 +0900
This GUC has always been classified as a planner option since its
introduction in 7c944bd, and was listed in postgresql.conf.sample. As
this parameter exists for testing purposes, move it to the section
dedicated to developer parameters and hence remove it from
postgresql.conf.sample. This will avoid any temptation to play with it
on production servers for users that should never really have to touch
this parameter.
The general description used for developer options is reworded a bit, to
take into account the inclusion of force_parallel_mode, per a suggestion
from Tom Lane.
Per discussion between Tom Lane, Bruce Momjian, Justin Pryzby, Bharath
Rupireddy and me.
Author: Justin Pryzby, Tom Lane
Discussion: https://postgr.es/m/20210403152402.GA8049@momjian.us
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
Simplify tests of postgres_fdw terminating connections
commit : 93f41461449f917da20af4fa2973f8afe8e6ea6e
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 14 Apr 2021 14:23:53 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 14 Apr 2021 14:23:53 +0900
The tests introduced in 32a9c0b for connections broken and
re-established rely on pg_terminate_backend() for their logic. When
these were introduced, this function simply sent a signal to a backend
without waiting for the operation to complete, and the tests repeatedly
looked at pg_stat_activity to check if the operation was completed or
not. Since aaf0432, it is possible to define a timeout to make
pg_terminate_backend() wait for a certain duration, so make use of it,
with a timeout reasonably large enough (3min) to give enough room for
the tests to pass even on slow machines.
Some measurements show that the tests of postgres_fdw are much faster
with this change. For example, on my laptop, they now take 4s instead
of 6s.
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/CALj2ACXGY_EfGrMTjKjHy2zi-u1u9rdeioU_fro0T6Jo8t56KQ@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Use NameData datatype for slotname in stats.
commit : cca57c1d9bf7eeba5b81115e0b82651cf3d8e4ea
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 14 Apr 2021 08:55:03 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 14 Apr 2021 08:55:03 +0530
This will make it consistent with the other usage of slotname in the code.
In the passing, change pgstat_report_replslot signature to use a structure
rather than multiple parameters.
Reported-by: Andres Freund
Author: Vignesh C
Reviewed-by: Sawada Masahiko, Amit Kapila
Discussion: https://postgr.es/m/20210319185247.ldebgpdaxsowiflw@alap3.anarazel.de
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/pgstat.h
Initialize t_self and t_tableOid in statext_expressions_load
commit : 20661c15db8f430d4880ba685e6b12afa271c1ac
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 14 Apr 2021 00:46:12 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 14 Apr 2021 00:46:12 +0200
The function is building a fake heap tuple, but left some of the header
fields (tid and table OID) uninitialized. Per Coverity report.
Reported-by: Ranier Vilela
Discussion: https://postgr.es/m/CAEudQApj6h8tZ0-eP5Af5PKc5NG1YUc7=SdN_99YoHS51fKa0Q@mail.gmail.com
M src/backend/statistics/extended_stats.c
Don't truncate heap when VACUUM's failsafe is in effect.
commit : 60f1f09ff44308667ef6c72fbafd68235e55ae27
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 13 Apr 2021 12:58:31 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 13 Apr 2021 12:58:31 -0700
It seems like a good idea to bypass heap truncation when the wraparound
failsafe mechanism (which was added in commit 1e55e7d1) is in effect.
Deliberately don't bypass heap truncation in the INDEX_CLEANUP=off case,
even though it is similar to the failsafe case. There is already a
separate reloption (and related VACUUM parameter) for that.
Reported-By: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAD21AoDWRh6oTN5T8wa+cpZUVpHXET8BJ8Da7WHVHpwkPP6KLg@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Allow table-qualified variable names in ON CONFLICT ... WHERE.
commit : 6c0373ab77359c94b279c4e67c91aa623841af65
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Apr 2021 15:39:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Apr 2021 15:39:33 -0400
Previously you could only use unqualified variable names here.
While that's not a functional deficiency, since only the target
table can be referenced, it's a surprising inconsistency with the
rules for partial-index predicates, on which this syntax is
supposedly modeled.
The fix for that is no harder than passing addToRelNameSpace = true
to addNSItemToQuery. However, it's really pretty bogus for
transformOnConflictArbiter and transformOnConflictClause to be
messing with the namespace item for the target table at all.
It's not theirs to manage, it results in duplicative creations of
namespace items, and transformOnConflictClause wasn't even doing
it quite correctly (that coding resulted in two nsitems for the
target table, since it hadn't cleaned out the existing one).
Hence, make transformInsertStmt responsible for setting up the
target nsitem once for both these clauses and RETURNING.
Also, arrange for ON CONFLICT ... UPDATE's "excluded" pseudo-relation
to be added to the rangetable before we run transformOnConflictArbiter.
This produces a more helpful HINT if someone writes "excluded.col"
in the arbiter expression.
Per bug #16958 from Lukas Eder. Although I agree this is a bug,
the consequences are hardly severe, so no back-patch.
Discussion: https://postgr.es/m/16958-963f638020de271c@postgresql.org
M src/backend/parser/analyze.c
M src/backend/parser/parse_clause.c
M src/test/regress/expected/insert_conflict.out
M src/test/regress/sql/insert_conflict.sql
docs: Update TOAST storage docs for configurable compression.
commit : e8c435a824e123f43067ce6f69d66f14cfb8815e
author : Robert Haas <rhaas@postgresql.org>
date : Tue, 13 Apr 2021 14:56:12 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Tue, 13 Apr 2021 14:56:12 -0400
Mention that there are multiple TOAST compression methods and that the
compression method used is stored in a TOAST pointer along with the
other information that was stored there previously. Add a reference to
the documentation for default_toast_compression, where the supported
methods are listed, instead of duplicating that here.
I haven't tried to preserve the text claiming that pglz is "fairly
simple and very fast." I have no view on the veracity of the former
claim, but LZ4 seems to be faster (and to compress better) so it
seems better not to muddy the waters by talking about compression
speed as a strong point of PGLZ.
Patch by me, reviewed by Justin Pryzby.
Discussion: http://postgr.es/m/CA+Tgmoaw_YBwQhOS_hhEPPwFhfAnu+VCLs18EfGr9gQw1z4H-w@mail.gmail.com
M doc/src/sgml/storage.sgml
Fix some inappropriately-disallowed uses of ALTER ROLE/DATABASE SET.
commit : 69d5ca484b69771073380e234e5377b6d6a5ebaf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Apr 2021 15:10:18 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Apr 2021 15:10:18 -0400
Most GUC check hooks that inspect database state have special checks
that prevent them from throwing hard errors for state-dependent issues
when source == PGC_S_TEST. This allows, for example,
"ALTER DATABASE d SET default_text_search_config = foo" when the "foo"
configuration hasn't been created yet. Without this, we have problems
during dump/reload or pg_upgrade, because pg_dump has no idea about
possible dependencies of GUC values and can't ensure a safe restore
ordering.
However, check_role() and check_session_authorization() hadn't gotten
the memo about that, and would throw hard errors anyway. It's not
entirely clear what is the use-case for "ALTER ROLE x SET role = y",
but we've now heard two independent complaints about that bollixing
an upgrade, so apparently some people are doing it.
Hence, fix these two functions to act more like other check hooks
with similar needs. (But I did not change their insistence on
being inside a transaction, as it's still not apparent that setting
either GUC from the configuration file would be wise.)
Also fix check_temp_buffers, which had a different form of the disease
of making state-dependent checks without any exception for PGC_S_TEST.
A cursory survey of other GUC check hooks did not find any more issues
of this ilk. (There are a lot of interdependencies among
PGC_POSTMASTER and PGC_SIGHUP GUCs, which may be a bad idea, but
they're not relevant to the immediate concern because they can't be
set via ALTER ROLE/DATABASE.)
Per reports from Charlie Hornsby and Nathan Bossart. Back-patch
to all supported branches.
Discussion: https://postgr.es/m/HE1P189MB0523B31598B0C772C908088DB7709@HE1P189MB0523.EURP189.PROD.OUTLOOK.COM
Discussion: https://postgr.es/m/20160711223641.1426.86096@wrigleys.postgresql.org
M src/backend/commands/variable.c
M src/backend/utils/misc/guc.c
Redesign the caching done by get_cached_rowtype().
commit : c2db458c1036efae503ce5e451f8369e64c99541
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Apr 2021 13:37:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Apr 2021 13:37:07 -0400
Previously, get_cached_rowtype() cached a pointer to a reference-counted
tuple descriptor from the typcache, relying on the ExprContextCallback
mechanism to release the tupdesc refcount when the expression tree
using the tupdesc was destroyed. This worked fine when it was designed,
but the introduction of within-DO-block COMMITs broke it. The refcount
is logged in a transaction-lifespan resource owner, but plpgsql won't
destroy simple expressions made within the DO block (before its first
commit) until the DO block is exited. That results in a warning about
a leaked tupdesc refcount when the COMMIT destroys the original resource
owner, and then an error about the active resource owner not holding a
matching refcount when the expression is destroyed.
To fix, get rid of the need to have a shutdown callback at all, by
instead caching a pointer to the relevant typcache entry. Those
survive for the life of the backend, so we needn't worry about the
pointer becoming stale. (For registered RECORD types, we can still
cache a pointer to the tupdesc, knowing that it won't change for the
life of the backend.) This mechanism has been in use in plpgsql
and expandedrecord.c since commit 4b93f5799, and seems to work well.
This change requires modifying the ExprEvalStep structs used by the
relevant expression step types, which is slightly worrisome for
back-patching. However, there seems no good reason for extensions
to be familiar with the details of these particular sub-structs.
Per report from Rohit Bhogate. Back-patch to v11 where within-DO-block
COMMITs became a thing.
Discussion: https://postgr.es/m/CAAV6ZkQRCVBh8qAY+SZiHnz+U+FqAGBBDaDTjF2yiKa2nJSLKg@mail.gmail.com
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/include/executor/execExpr.h
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/sql/plpgsql_transaction.sql
Avoid improbable PANIC during heap_update.
commit : 34f581c39e97e2ea237255cf75cccebccc02d477
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Apr 2021 12:17:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Apr 2021 12:17:24 -0400
heap_update needs to clear any existing "all visible" flag on
the old tuple's page (and on the new page too, if different).
Per coding rules, to do this it must acquire pin on the appropriate
visibility-map page while not holding exclusive buffer lock;
which creates a race condition since someone else could set the
flag whenever we're not holding the buffer lock. The code is
supposed to handle that by re-checking the flag after acquiring
buffer lock and retrying if it became set. However, one code
path through heap_update itself, as well as one in its subroutine
RelationGetBufferForTuple, failed to do this. The end result,
in the unlikely event that a concurrent VACUUM did set the flag
while we're transiently not holding lock, is a non-recurring
"PANIC: wrong buffer passed to visibilitymap_clear" failure.
This has been seen a few times in the buildfarm since recent VACUUM
changes that added code paths that could set the all-visible flag
while holding only exclusive buffer lock. Previously, the flag
was (usually?) set only after doing LockBufferForCleanup, which
would insist on buffer pin count zero, thus preventing the flag
from becoming set partway through heap_update. However, it's
clear that it's heap_update not VACUUM that's at fault here.
What's less clear is whether there is any hazard from these bugs
in released branches. heap_update is certainly violating API
expectations, but if there is no code path that can set all-visible
without a cleanup lock then it's only a latent bug. That's not
100% certain though, besides which we should worry about extensions
or future back-patch fixes that could introduce such code paths.
I chose to back-patch to v12. Fixing RelationGetBufferForTuple
before that would require also back-patching portions of older
fixes (notably 0d1fe9f74), which is more code churn than seems
prudent to fix a hypothetical issue.
Discussion: https://postgr.es/m/2247102.1618008027@sss.pgh.pa.us
M src/backend/access/heap/heapam.c
M src/backend/access/heap/hio.c
doc: Fix typo in logicaldecoding.sgml.
commit : 5fe83adad9efd5e3929f0465b44e786dc23c7b55
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 13 Apr 2021 14:21:51 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 13 Apr 2021 14:21:51 +0900
Introduced in commit 0aa8a01d04.
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+Ps8rkVHKWyjg09Fb1PaVG5-EhoFTFJ9OZMF4HPzDSXfew@mail.gmail.com
M doc/src/sgml/logicaldecoding.sgml
Use "-I." in directories holding Bison parsers, for Oracle compilers.
commit : 455dbc010be53ac61fcb2da83b1e565f4c263449
author : Noah Misch <noah@leadboat.com>
date : Mon, 12 Apr 2021 19:24:41 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 12 Apr 2021 19:24:41 -0700
With the Oracle Developer Studio 12.6 compiler, #line directives alter
the current source file location for purposes of #include "..."
directives. Hence, a VPATH build failed with 'cannot find include file:
"specscanner.c"'. With two exceptions, parser-containing directories
already add "-I. -I$(srcdir)"; eliminate the exceptions. Back-patch to
9.6 (all supported versions).
M src/backend/utils/adt/Makefile
M src/test/isolation/Makefile
Port regress-python3-mangle.mk to Solaris "sed".
commit : c3556f6fac349b31da2fd00107469ce36fb37536
author : Noah Misch <noah@leadboat.com>
date : Mon, 12 Apr 2021 19:24:21 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 12 Apr 2021 19:24:21 -0700
It doesn't support "\(foo\)*" like a POSIX "sed" implementation does;
see the Autoconf manual. Back-patch to 9.6 (all supported versions).
M src/pl/plpython/regress-python3-mangle.mk
Fix potential SSI hazard in heap_update().
commit : b1df6b696b759f00ebbf02e6de64e259d4be5785
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 13 Apr 2021 12:34:25 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 13 Apr 2021 12:34:25 +1200
Commit 6f38d4dac38 failed to heed a warning about the stability of the
value pointed to by "otid". The caller is allowed to pass in a pointer to
newtup->t_self, which will be updated during the execution of the
function. Instead, the SSI check should use the value we copy into
oldtup.t_self near the top of the function.
Not a live bug, because newtup->t_self doesn't really get updated until
a bit later, but it was confusing and broke the rule established by the
comment.
Back-patch to 13.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/2689164.1618160085%40sss.pgh.pa.us
M src/backend/access/heap/heapam.c
Remove duplicated --no-sync switches in new tests of test_pg_dump
commit : 885a87641930778d9580fdf0656af24e3f52d276
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 13 Apr 2021 09:42:01 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 13 Apr 2021 09:42:01 +0900
These got introduced in 6568cef.
Reported-by: Noah Misch
Discussion: https://postgr.es/m/20210404220802.GA728316@rfd.leadboat.com
M src/test/modules/test_pg_dump/t/001_base.pl
Remove no-longer-relevant test case.
commit : cf0020080a3de20287217621da57bfd840e9a693
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Apr 2021 18:58:20 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Apr 2021 18:58:20 -0400
collate.icu.utf8.sql was exercising the recording of a collation
dependency for an enum comparison expression, but such an expression
should never have had any collation dependency in the first place.
After I fixed that in commit c402b02b9, the test started failing.
We don't need to test that scenario anymore, so just remove the
now-useless test steps.
(This test case is new in HEAD, so no need to back-patch.)
Discussion: https://postgr.es/m/3044030.1618261159@sss.pgh.pa.us
Discussion: https://postgr.es/m/HK0PR01MB22744393C474D503E16C8509F4709@HK0PR01MB2274.apcprd01.prod.exchangelabs.com
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Fix old bug with coercing the result of a COLLATE expression.
commit : c402b02b9fb53aee2a26876de90a8f95f9a9be92
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Apr 2021 14:37:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Apr 2021 14:37:22 -0400
There are hacks in parse_coerce.c to push down a requested coercion
to below any CollateExpr that may appear. However, we did that even
if the requested data type is non-collatable, leading to an invalid
expression tree in which CollateExpr is applied to a non-collatable
type. The fix is just to drop the CollateExpr altogether, reasoning
that it's useless.
This bug is ten years old, dating to the original addition of
COLLATE support. The lack of field complaints suggests that there
aren't a lot of user-visible consequences. We noticed the problem
because it would trigger an assertion in DefineVirtualRelation if
the invalid structure appears as an output column of a view; however,
in a non-assert build, you don't see a crash just a (subtly incorrect)
complaint about applying collation to a non-collatable type. I found
that by putting the incorrect structure further down in a view, I could
make a view definition that would fail dump/reload, per the added
regression test case. But CollateExpr doesn't do anything at run-time,
so this likely doesn't lead to any really exciting consequences.
Per report from Yulin Pei. Back-patch to all supported branches.
Discussion: https://postgr.es/m/HK0PR01MB22744393C474D503E16C8509F4709@HK0PR01MB2274.apcprd01.prod.exchangelabs.com
M src/backend/parser/parse_coerce.c
M src/test/regress/expected/collate.out
M src/test/regress/sql/collate.sql
pg_upgrade: Print OID using %u instead of %d
commit : 6787e53fe59eed19095c771a8d3323fb59420733
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 12 Apr 2021 20:29:24 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 12 Apr 2021 20:29:24 +0200
This could write wrong output into the cluster deletion script if a
database OID exceeds the signed 32-bit range.
M src/bin/pg_upgrade/check.c
pg_amcheck: Add basic NLS support
commit : fc0fefbfe0d7e805f6a3a4aaaad7127090fcca51
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 12 Apr 2021 19:04:33 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 12 Apr 2021 19:04:33 +0200
A src/bin/pg_amcheck/nls.mk
M src/bin/pg_amcheck/pg_amcheck.c
Fix files references in nls.mk
commit : 44c8a3d75997b08fa7645dac7ee5c7a918a235c7
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 12 Apr 2021 15:44:38 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 12 Apr 2021 15:44:38 +0200
broken by 37d2ff38031262a1778bc76a9c55fff7afbcf275
M src/bin/pg_rewind/nls.mk
Support tab-complete for TRUNCATE on foreign tables.
commit : 81e094bdfdd6cf6568cba2b25eea9876daceaacb
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 12 Apr 2021 21:34:23 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 12 Apr 2021 21:34:23 +0900
Commit 8ff1c94649 extended TRUNCATE command so that it can also truncate
foreign tables. But it forgot to support tab-complete for TRUNCATE on
foreign tables. That is, previously tab-complete for TRUNCATE displayed
only the names of regular tables.
This commit improves tab-complete for TRUNCATE so that it displays also
the names of foreign tables.
Author: Fujii Masao
Reviewed-by: Bharath Rupireddy
Discussion: https://postgr.es/m/551ed8c1-f531-818b-664a-2cecdab99cd8@oss.nttdata.com
M src/bin/psql/tab-complete.c
Move log_autovacuum_min_duration into its correct sections
commit : b094063cd16d22b2f065a432580bb3568b2d8a77
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 12 Apr 2021 13:53:17 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 12 Apr 2021 13:53:17 +0900
This GUC has already been classified as LOGGING_WHAT, but its location
in postgresql.conf.sample and the documentation did not reflect that, so
fix those inconsistencies.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20210404012546.GK6592@telsasoft.com
M doc/src/sgml/config.sgml
M src/backend/utils/misc/postgresql.conf.sample
doc: Update information of new messages for logical replication.
commit : 15c1a9d9cb7604472d4823f48b64cdc02c441194
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 12 Apr 2021 09:27:10 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 12 Apr 2021 09:27:10 +0530
Updated documentation for new messages added for streaming of in-progress
transactions, as well as changes made to the existing messages. It also
updates the information of protocol versions supported for logical
replication.
Author: Ajin Cherian
Reviewed-by: Amit Kapila, Peter Smith, Euler Taveira
Discussion: https://postgr.es/m/CAFPTHDYHN9m=MZZct-B=BYg_TETvv+kXvL9RD2DpaBS5pGxGYg@mail.gmail.com
M doc/src/sgml/protocol.sgml
Fix out-of-bound memory access for interval -> char conversion
commit : 7a3972597f6ed7a6976d81abb66c38a7a1c29058
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 12 Apr 2021 11:30:50 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 12 Apr 2021 11:30:50 +0900
Using Roman numbers (via "RM" or "rm") for a conversion to calculate a
number of months has never considered the case of negative numbers,
where a conversion could easily cause out-of-bound memory accesses. The
conversions in themselves were not completely consistent either, as
specifying 12 would result in NULL, but it should mean XII.
This commit reworks the conversion calculation to have a more
consistent behavior:
- If the number of months and years is 0, return NULL.
- If the number of months is positive, return the exact month number.
- If the number of months is negative, do a backward calculation, with
-1 meaning December, -2 November, etc.
Reported-by: Theodor Arsenij Larionov-Trichkin
Author: Julien Rouhaud
Discussion: https://postgr.es/m/16953-f255a18f8c51f1d5@postgresql.org
backpatch-through: 9.6
M src/backend/utils/adt/formatting.c
M src/test/regress/expected/timestamp.out
M src/test/regress/sql/timestamp.sql
Silence some Coverity warnings and improve code consistency.
commit : 6277435a8a89c59f716c111200c072d1454b8ff2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 11 Apr 2021 17:02:04 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 11 Apr 2021 17:02:04 -0400
Coverity complained about possible overflow in expressions like
intresult = tm->tm_sec * 1000000 + fsec;
on the grounds that the multiplication would happen in 32-bit
arithmetic before widening to the int64 result. I think these
are all false positives because of the limited possible range of
tm_sec; but nonetheless it seems silly to spell it like that when
nearby lines have the identical computation written with a 64-bit
constant.
... or more accurately, with an LL constant, which is not project
style. Make all of these use INT64CONST(), as we do elsewhere.
This is all new code from a2da77cdb, so no need for back-patch.
M src/backend/utils/adt/date.c
M src/backend/utils/adt/timestamp.c
Add macro PGWARNING, and make PGERROR available on all platforms.
commit : d7cff12c4c035b7cf12bb8454824f48f13018730
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 11 Apr 2021 13:22:56 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 11 Apr 2021 13:22:56 -0400
We'd previously noted the need for coping with Windows headers
that provide some other definition of macro "ERROR" than elog.h
does. It turns out that R also wants to define ERROR, and
WARNING too. PL/R has been working around this in a hacky way
that broke when we recently changed the numeric value of ERROR.
To let them have a more future-proof solution, provide an
alternate macro PGWARNING for WARNING, and make PGERROR visible
always, not only when #ifdef WIN32.
Discussion: https://postgr.es/m/CADK3HHK6iMChd1yoOqssxBn5Z14Zar8Ztr3G-N_fuG7F8YTP3w@mail.gmail.com
M src/include/utils/elog.h
Fix uninitialized variable from commit a4d75c86b.
commit : 9cb92334092fa75afc62a71243bbc1f4612ecfa4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 11 Apr 2021 11:46:32 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 11 Apr 2021 11:46:32 -0400
The path for *exprs != NIL would misbehave, and likely crash,
since pull_varattnos expects its last argument to be valid
at call.
Found by Coverity --- we have no coverage of this path in
the regression tests.
M src/backend/statistics/extended_stats.c
Avoid unnecessary table open/close in TRUNCATE command.
commit : 81a23dd87999ec9fb62554328c69c5b678612d56
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 12 Apr 2021 00:05:58 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 12 Apr 2021 00:05:58 +0900
ExecuteTruncate() filters out the duplicate tables specified
in the TRUNCATE command, for example in the case where "TRUNCATE foo, foo"
is executed. Such duplicate tables obviously don't need to be opened
and closed because they are skipped. But previously it always opened
the tables before checking whether they were duplicated ones or not,
and then closed them if they were. That is, the duplicated tables were
opened and closed unnecessarily.
This commit changes ExecuteTruncate() so that it opens the table
after it confirms that table is not duplicated one, which leads to
avoid unnecessary table open/close.
Do not back-patch because such unnecessary table open/close is not
a bug though it exists in older versions.
Author: Bharath Rupireddy
Reviewed-by: Amul Sul, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACUdBO_sXJTa08OZ0YT0qk7F_gAmRa9hT4dxRcgPS4nsZA@mail.gmail.com
M src/backend/commands/tablecmds.c
Remove COMMIT_TS_SETTS record.
commit : 08aa89b326261b669648df97d4f2a6edba22d26a
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 12 Apr 2021 00:00:18 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 12 Apr 2021 00:00:18 +0900
Commit 438fc4a39c prevented the WAL replay from writing
COMMIT_TS_SETTS record. By this change there is no code that
generates COMMIT_TS_SETTS record in PostgreSQL core.
Also we can think that there are no extensions using the record
because we've not received so far any complaints about the issue
that commit 438fc4a39c fixed. Therefore this commit removes
COMMIT_TS_SETTS record and its related code. Even without
this record, the timestamp required for commit timestamp feature
can be acquired from the COMMIT record.
Bump WAL page magic.
Reported-by: lx zou <zoulx1982@163.com>
Author: Fujii Masao
Reviewed-by: Alvaro Herrera
Discussion: https://postgr.es/m/16931-620d0f2fdc6108f1@postgresql.org
M src/backend/access/rmgrdesc/committsdesc.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xact.c
M src/include/access/commit_ts.h
M src/include/access/xlog_internal.h
Standardize pg_authid oid_symbol values.
commit : df5efaf4410f94cc1b69e8ade1d64dc92232ec1d
author : Noah Misch <noah@leadboat.com>
date : Sat, 10 Apr 2021 12:01:41 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 10 Apr 2021 12:01:41 -0700
Commit c9c41c7a337d3e2deb0b2a193e9ecfb865d8f52b used two different
naming patterns. Standardize on the majority pattern, which was the
only pattern in the last reviewed version of that commit.
M src/backend/catalog/aclchk.c
M src/backend/commands/user.c
M src/backend/utils/adt/acl.c
M src/include/catalog/pg_authid.dat
Improve behavior of date_bin with origin in the future
commit : 496e58bb0e5e939e6ed5839c92b05e3ab11b54bb
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 10 Apr 2021 19:33:46 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 10 Apr 2021 19:33:46 +0200
Currently, when the origin is after the input, the result is the
timestamp at the end of the bin, rather than the beginning as
expected. This puts the result consistently at the beginning of the
bin.
Author: John Naylor <john.naylor@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/CAFBsxsGjLDxQofRfH+d4KSAXxPf3MMevUG7s6EDfdBOvHLDLjw@mail.gmail.com
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/timestamp.out
M src/test/regress/sql/timestamp.sql
Fix failure of xlogprefetch.h to include all prerequisite headers.
commit : 99964c4ade468c35a3f6e248a2380a1ff67d9cd3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 10 Apr 2021 13:16:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 10 Apr 2021 13:16:08 -0400
Per cpluspluscheck.
M src/include/access/xlogprefetch.h
Doc: update documentation of check_function_bodies.
commit : 07b76833b15163c6574ea2c12d05d9a0800665e2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 10 Apr 2021 12:08:28 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 10 Apr 2021 12:08:28 -0400
Adjust docs and description string to note that check_function_bodies
applies to procedures too. (In hindsight it should have been named
check_routine_bodies, but it seems too late for that now.)
Daniel Westermann
Discussion: https://postgr.es/m/GV0P278MB04834A9EB9A74B036DC7CE49D2739@GV0P278MB0483.CHEP278.PROD.OUTLOOK.COM
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc.c
Improve slightly misleading comments in nodeFuncs.c
commit : 152d33bccec7176f50be225bdbedf2e6de214e54
author : David Rowley <drowley@postgresql.org>
date : Sat, 10 Apr 2021 19:19:45 +1200
committer: David Rowley <drowley@postgresql.org>
date : Sat, 10 Apr 2021 19:19:45 +1200
There were some comments in nodeFuncs.c that, depending on your
interpretation of the word "result", could lead you to believe that the
comments were badly copied and pasted from somewhere else. If you thought
of "result" as the return value of the function that the comment is
written in, then you'd be misled. However, if you'd correctly
interpreted "result" to mean the result type of the given node type,
you'd not have seen any issues.
Here we do a small cleanup to try to prevent any future
misinterpretations. Per wording suggestion from Tom Lane.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/CAApHDvp+Bw=2Qiu5=uXMKfC7gd0+B=4JvexVgGJU=am2g9a1CA@mail.gmail.com
M src/backend/nodes/nodeFuncs.c
doc: Fix man page whitespace issues
commit : 9cae39b8e6a53decb37cce22852bb2e6d0e3f5d3
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 9 Apr 2021 23:23:45 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 9 Apr 2021 23:23:45 +0200
Whitespace between tags is significant, and in some cases it creates
extra vertical space in man pages. The fix is to remove some newlines
in the markup.
M doc/src/sgml/ref/cluster.sgml
M doc/src/sgml/ref/create_extension.sgml
M doc/src/sgml/ref/create_procedure.sgml
Suppress length of Notice/Error msgs in PQtrace regress mode
commit : e7e341409a3d85aba4cf754ba9cf722a4d8e6676
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 9 Apr 2021 17:13:18 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 9 Apr 2021 17:13:18 -0400
A (relatively minor) annoyance of ErrorResponse/NoticeResponse messages
as printed by PQtrace() is that their length might vary when we move
error messages from one source file to another, one function to another,
or even when their location line numbers change number of digits.
To avoid having to adjust expected files for some tests, make the
regress mode of PQtrace() suppress the length word of NoticeResponse and
ErrorResponse messages.
Discussion: https://postgr.es/m/20210402023010.GA13563@alvherre.pgsql
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
M src/interfaces/libpq/fe-trace.c
M src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
M src/test/modules/libpq_pipeline/traces/transaction.trace
Make new GUC short descriptions more consistent.
commit : 846d35b2dc1bd4d09f5e3e5e5a2cc8efafeb600e
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 10 Apr 2021 08:41:07 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 10 Apr 2021 08:41:07 +1200
Reported-by: Daniel Westermann (DWE) <daniel.westermann@dbi-services.com>
Discussion: https://postgr.es/m/GV0P278MB0483490FEAC879DCA5ED583DD2739%40GV0P278MB0483.CHEP278.PROD.OUTLOOK.COM
M src/backend/utils/misc/guc.c
Doc: Review for "Optionally prefetch referenced data in recovery."
commit : dc88460c24ed71ba7464ef4749e5f25da1bf6652
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 10 Apr 2021 08:09:30 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 10 Apr 2021 08:09:30 +1200
Typos, corrections and language improvements in the docs, and a few in
code comments too.
Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/20210409033703.GP6592%40telsasoft.com
M doc/src/sgml/config.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/xlogprefetch.c
M src/backend/utils/misc/guc.c
doc: Additional documentation for date_bin
commit : 49fb4e6b249029e75ccc6b490d76f15cd53d553b
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 9 Apr 2021 21:55:08 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 9 Apr 2021 21:55:08 +0200
Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Author: John Naylor <john.naylor@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/CAFBsxsEEm1nuhZmfVQxvu_i3nDDEuvNJ_WMrDo9whFD_jusp-A@mail.gmail.com
M doc/src/sgml/func.sgml
Document ANALYZE storage parameters for partitioned tables
commit : 41badeaba8beee7648ebe7923a41c04f1f3cb302
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 9 Apr 2021 13:38:07 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 9 Apr 2021 13:38:07 -0400
Commit 0827e8af70f4 added parameters for autovacuum to support
partitioned tables, but didn't add any docs. Add them.
Discussion: https://postgr.es/m/20210408213051.GL6592@telsasoft.com
M doc/src/sgml/ref/create_table.sgml
Set pg_class.reltuples for partitioned tables
commit : 0e69f705cc1a3df273b38c9883fb5765991e04fe
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 9 Apr 2021 11:29:08 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 9 Apr 2021 11:29:08 -0400
When commit 0827e8af70f4 added auto-analyze support for partitioned
tables, it included code to obtain reltuples for the partitioned table
as a number of catalog accesses to read pg_class.reltuples for each
partition. That's not only very inefficient, but also problematic
because autovacuum doesn't hold any locks on any of those tables -- and
doesn't want to. Replace that code with a read of pg_class.reltuples
for the partitioned table, and make sure ANALYZE and TRUNCATE properly
maintain that value.
I found no code that would be affected by the change of relpages from
zero to non-zero for partitioned tables, and no other code that should
be maintaining it, but if there is, hopefully it'll be an easy fix.
Per buildfarm.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Zhihong Yu <zyu@yugabyte.com>
Discussion: https://postgr.es/m/1823909.1617862590@sss.pgh.pa.us
M src/backend/commands/analyze.c
M src/backend/commands/tablecmds.c
M src/backend/postmaster/autovacuum.c
Fix typo
commit : 1798d8f8b6fbb8ff922f640ff2d5dbd3e47064a2
author : Magnus Hagander <magnus@hagander.net>
date : Fri, 9 Apr 2021 12:40:14 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Fri, 9 Apr 2021 12:40:14 +0200
Author: Daniel Westermann
Backpatch-through: 9.6
Discussion: https://postgr.es/m/GV0P278MB0483A7AA85BAFCC06D90F453D2739@GV0P278MB0483.CHEP278.PROD.OUTLOOK.COM
M src/backend/utils/misc/guc.c
Fix typos and grammar in documentation and code comments
commit : 609b0652af00374b89411ea2613fd5bb92bca92c
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 9 Apr 2021 13:53:07 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 9 Apr 2021 13:53:07 +0900
Comment fixes are applied on HEAD, and documentation improvements are
applied on back-branches where needed.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20210408164008.GJ6592@telsasoft.com
Backpatch-through: 9.6
M doc/src/sgml/config.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/pgstatstatements.sgml
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/createuser.sgml
M doc/src/sgml/ref/declare.sgml
M doc/src/sgml/ref/pg_amcheck.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/wal.sgml
M src/backend/catalog/heap.c
M src/backend/commands/analyze.c
M src/backend/commands/cluster.c
M src/backend/commands/copyfrom.c
M src/backend/statistics/extended_stats.c
M src/backend/utils/adt/jsonfuncs.c
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/include/lib/sort_template.h
M src/include/utils/guc.h
Silence another _bt_check_unique compiler warning.
commit : 796092fb84c08162ae803e83a13aa8bd6d9b23d0
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 8 Apr 2021 12:54:31 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 8 Apr 2021 12:54:31 -0700
Per complaint from Tom Lane
Discussion: https://postgr.es/m/1922884.1617909599@sss.pgh.pa.us
M src/backend/access/nbtree/nbtinsert.c
Add support for tab-completion of type arguments in \df, \do.
commit : d1fcbde579d440c35023baa0de7ebf27f644a314
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 8 Apr 2021 15:38:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 8 Apr 2021 15:38:26 -0400
Oversight in commit a3027e1e7.
M src/bin/psql/tab-complete.c
Suppress uninitialized-variable warning.
commit : 01add89454d5dc289ed3126d5de03169bdeff41b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 8 Apr 2021 15:14:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 8 Apr 2021 15:14:26 -0400
Several buildfarm critters that don't usually produce such
warnings are complaining about e717a9a18. I think it's
actually safe, but move initialization to silence the warning.
M src/backend/catalog/pg_proc.c
Fixes for query_id feature
commit : 0f61727b75b93915ca9a9f20c996ed7020996a38
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 8 Apr 2021 11:16:01 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 8 Apr 2021 11:16:01 -0400
Ignore parallel workers in pg_stat_statements
Oversight in 4f0b0966c8 which exposed queryid in parallel workers.
Counters are aggregated by the main backend process so parallel workers
would report duplicated activity, and could also report activity for the
wrong entry as they are only aware of the top level queryid.
Fix thinko in pg_stat_get_activity when retrieving the queryid.
Remove unnecessary call to pgstat_report_queryid().
Reported-by: Amit Kapila, Andres Freund, Thomas Munro
Discussion: https://postgr.es/m/20210408051735.lfbdzun5zdlax5gd@alap3.anarazel.de p634GTSOqnDW86Owrn6qDAVosC5dJjXjp7BMfc5Gz1Q@mail.gmail.com
Author: Julien Rouhaud
M contrib/pg_stat_statements/pg_stat_statements.c
M src/backend/executor/execParallel.c
M src/backend/utils/adt/pgstatfuncs.c
Merge v1.10 of pg_stat_statements into v1.9
commit : 5844c23dc50508aefeb8183be45f4ee99e9dec17
author : Magnus Hagander <magnus@hagander.net>
date : Thu, 8 Apr 2021 15:15:17 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Thu, 8 Apr 2021 15:15:17 +0200
v1.9 is already new in this version of PostgreSQL, so turn it into just
one change.
Author: Julien Rohaud
Discussion: https://postgr.es/m/20210408120505.7zinijtdexbyghvb@nol
M contrib/pg_stat_statements/Makefile
M contrib/pg_stat_statements/pg_stat_statements–1.8–1.9.sql
D contrib/pg_stat_statements/pg_stat_statements–1.9–1.10.sql
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/pg_stat_statements.control
Remove duplicate typedef.
commit : 34399a670a1c559ef8a355ed25d090d73e400ad4
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 9 Apr 2021 00:36:59 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 9 Apr 2021 00:36:59 +1200
Thinko in commit 323cbe7c, per complaint from BF animal locust's older
GCC compiler.
M src/include/replication/logical.h
Allow TRUNCATE command to truncate foreign tables.
commit : 8ff1c94649f5c9184ac5f07981d8aea9dfd7ac19
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 8 Apr 2021 20:56:08 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 8 Apr 2021 20:56:08 +0900
This commit introduces new foreign data wrapper API for TRUNCATE.
It extends TRUNCATE command so that it accepts foreign tables as
the targets to truncate and invokes that API. Also it extends postgres_fdw
so that it can issue TRUNCATE command to foreign servers, by adding
new routine for that TRUNCATE API.
The information about options specified in TRUNCATE command, e.g.,
ONLY, CACADE, etc is passed to FDW via API. The list of foreign tables to
truncate is also passed to FDW. FDW truncates the foreign data sources
that the passed foreign tables specify, based on those information.
For example, postgres_fdw constructs TRUNCATE command using them
and issues it to the foreign server.
For performance, TRUNCATE command invokes the FDW routine for
TRUNCATE once per foreign server that foreign tables to truncate belong to.
Author: Kazutaka Onishi, Kohei KaiGai, slightly modified by Fujii Masao
Reviewed-by: Bharath Rupireddy, Michael Paquier, Zhihong Yu, Alvaro Herrera, Stephen Frost, Ashutosh Bapat, Amit Langote, Daniel Gustafsson, Ibrar Ahmed, Fujii Masao
Discussion: https://postgr.es/m/CAOP8fzb_gkReLput7OvOK+8NHgw-RKqNv59vem7=524krQTcWA@mail.gmail.com
Discussion: https://postgr.es/m/CAJuF6cMWDDqU-vn_knZgma+2GMaout68YUgn1uyDnexRhqqM5Q@mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/ref/truncate.sgml
M src/backend/commands/tablecmds.c
M src/backend/replication/logical/worker.c
M src/include/commands/tablecmds.h
M src/include/foreign/fdwapi.h
M src/include/utils/rel.h
M src/test/regress/expected/foreign_data.out
M src/tools/pgindent/typedefs.list
Speedup ScalarArrayOpExpr evaluation
commit : 50e17ad281b8d1c1b410c9833955bc80fbad4078
author : David Rowley <drowley@postgresql.org>
date : Thu, 8 Apr 2021 23:51:22 +1200
committer: David Rowley <drowley@postgresql.org>
date : Thu, 8 Apr 2021 23:51:22 +1200
ScalarArrayOpExprs with "useOr=true" and a set of Consts on the righthand
side have traditionally been evaluated by using a linear search over the
array. When these arrays contain large numbers of elements then this
linear search could become a significant part of execution time.
Here we add a new method of evaluating ScalarArrayOpExpr expressions to
allow them to be evaluated by first building a hash table containing each
element, then on subsequent evaluations, we just probe that hash table to
determine if there is a match.
The planner is in charge of determining when this optimization is possible
and it enables it by setting hashfuncid in the ScalarArrayOpExpr. The
executor will only perform the hash table evaluation when the hashfuncid
is set.
This means that not all cases are optimized. For example CHECK constraints
containing an IN clause won't go through the planner, so won't get the
hashfuncid set. We could maybe do something about that at some later
date. The reason we're not doing it now is from fear that we may slow
down cases where the expression is evaluated only once. Those cases can
be common, for example, a single row INSERT to a table with a CHECK
constraint containing an IN clause.
In the planner, we enable this when there are suitable hash functions for
the ScalarArrayOpExpr's operator and only when there is at least
MIN_ARRAY_SIZE_FOR_HASHED_SAOP elements in the array. The threshold is
currently set to 9.
Author: James Coleman, David Rowley
Reviewed-by: David Rowley, Tomas Vondra, Heikki Linnakangas
Discussion: https://postgr.es/m/CAAaqYe8x62+=wn0zvNKCj55tPpg-JBHzhZFFc6ANovdqFw7-dA@mail.gmail.com
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/jit/llvm/llvmjit_types.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/prep/prepqual.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/parse_oper.c
M src/backend/partitioning/partbounds.c
M src/include/catalog/catversion.h
M src/include/executor/execExpr.h
M src/include/nodes/primnodes.h
M src/include/optimizer/optimizer.h
M src/test/regress/expected/expressions.out
M src/test/regress/sql/expressions.sql
Optionally prefetch referenced data in recovery.
commit : 1d257577e08d3e598011d6850fd1025858de8c8c
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 8 Apr 2021 23:03:43 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 8 Apr 2021 23:03:43 +1200
Introduce a new GUC recovery_prefetch, disabled by default. When
enabled, look ahead in the WAL and try to initiate asynchronous reading
of referenced data blocks that are not yet cached in our buffer pool.
For now, this is done with posix_fadvise(), which has several caveats.
Better mechanisms will follow in later work on the I/O subsystem.
The GUC maintenance_io_concurrency is used to limit the number of
concurrent I/Os we allow ourselves to initiate, based on pessimistic
heuristics used to infer that I/Os have begun and completed.
The GUC wal_decode_buffer_size is used to limit the maximum distance we
are prepared to read ahead in the WAL to find uncached blocks.
Reviewed-by: Alvaro Herrera <alvherre@2ndquadrant.com> (parts)
Reviewed-by: Andres Freund <andres@anarazel.de> (parts)
Reviewed-by: Tomas Vondra <tomas.vondra@2ndquadrant.com> (parts)
Tested-by: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Tested-by: Jakub Wartak <Jakub.Wartak@tomtom.com>
Tested-by: Dmitry Dolgov <9erthalion6@gmail.com>
Tested-by: Sait Talha Nisanci <Sait.Nisanci@microsoft.com>
Discussion: https://postgr.es/m/CA%2BhUKGJ4VJN8ttxScUFM8dOKX0BrBiboo5uz1cq%3DAovOddfHpA%40mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/Makefile
M src/backend/access/transam/xlog.c
A src/backend/access/transam/xlogprefetch.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogutils.c
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/storage/freespace/freespace.c
M src/backend/storage/ipc/ipci.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/access/xlog.h
A src/include/access/xlogprefetch.h
M src/include/access/xlogreader.h
M src/include/access/xlogutils.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/utils/guc.h
M src/test/regress/expected/rules.out
M src/tools/pgindent/typedefs.list
Add circular WAL decoding buffer.
commit : f003d9f8721b3249e4aec8a1946034579d40d42c
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 8 Apr 2021 23:03:34 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 8 Apr 2021 23:03:34 +1200
Teach xlogreader.c to decode its output into a circular buffer, to
support optimizations based on looking ahead.
* XLogReadRecord() works as before, consuming records one by one, and
allowing them to be examined via the traditional XLogRecGetXXX()
macros.
* An alternative new interface XLogNextRecord() is added that returns
pointers to DecodedXLogRecord structs that can be examined directly.
* XLogReadAhead() provides a second cursor that lets you see
further ahead, as long as data is available and there is enough space
in the decoding buffer. This returns DecodedXLogRecord pointers to the
caller, but also adds them to a queue of records that will later be
consumed by XLogNextRecord()/XLogReadRecord().
The buffer's size is controlled with wal_decode_buffer_size. The buffer
could potentially be placed into shared memory, for future projects.
Large records that don't fit in the circular buffer are called
"oversized" and allocated separately with palloc().
Discussion: https://postgr.es/m/CA+hUKGJ4VJN8ttxScUFM8dOKX0BrBiboo5uz1cq=AovOddfHpA@mail.gmail.com
M src/backend/access/transam/generic_xlog.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogutils.c
M src/backend/replication/logical/decode.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_waldump/pg_waldump.c
M src/include/access/xlogreader.h
Remove read_page callback from XLogReader.
commit : 323cbe7c7ddcf18aaf24b7f6d682a45a61d4e31b
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 8 Apr 2021 23:03:23 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 8 Apr 2021 23:03:23 +1200
Previously, the XLogReader module would fetch new input data using a
callback function. Redesign the interface so that it tells the caller
to insert more data with a special return value instead. This API suits
later patches for prefetching, encryption and maybe other future
projects that would otherwise require continually extending the callback
interface.
As incidental cleanup work, move global variables readOff, readLen and
readSegNo inside XlogReaderState.
Author: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Author: Heikki Linnakangas <hlinnaka@iki.fi> (parts of earlier version)
Reviewed-by: Antonin Houska <ah@cybertec.at>
Reviewed-by: Alvaro Herrera <alvherre@2ndquadrant.com>
Reviewed-by: Takashi Menjo <takashi.menjo@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/20190418.210257.43726183.horiguchi.kyotaro%40lab.ntt.co.jp
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogutils.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/logicalfuncs.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/walsender.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_waldump/pg_waldump.c
M src/include/access/xlogreader.h
M src/include/access/xlogutils.h
M src/include/replication/logical.h
Cleanup partition pruning step generation
commit : 5ac9c4307337313bedeafc21dbbab93ba809241c
author : David Rowley <drowley@postgresql.org>
date : Thu, 8 Apr 2021 22:35:48 +1200
committer: David Rowley <drowley@postgresql.org>
date : Thu, 8 Apr 2021 22:35:48 +1200
There was some code in gen_prune_steps_from_opexps that needlessly
checked a list was not empty when it clearly had to contain at least one
item. This prompted a further cleanup operation in partprune.c.
Additionally, the previous code could end up adding additional needless
INTERSECT steps. However, those do not appear to be able to cause any
misbehavior.
gen_prune_steps_from_opexps is now no longer in charge of generating
combine pruning steps. Instead, gen_partprune_steps_internal, which
already does some combine step creation has been given the sole
responsibility of generating all combine steps. This means that when
we recursively call gen_partprune_steps_internal, since it always now adds
a combine step when it produces multiple steps, we can just pay attention
to the final step returned.
In passing, do quite a bit of work on the comments to try to more clearly
explain the role of both gen_partprune_steps_internal and
gen_prune_steps_from_opexps. This is fairly complex code so some extra
effort to give any new readers an overview of how things work seems like
a good idea.
Author: Amit Langote
Reported-by: Andy Fan
Reviewed-by: Kyotaro Horiguchi, Andy Fan, Ryan Lambert, David Rowley
Discussion: https://postgr.es/m/CAKU4AWqWoVii+bRTeBQmeVW+PznkdO8DfbwqNsu9Gj4ubt9A6w@mail.gmail.com
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partprune.c
M src/include/nodes/plannodes.h
Add ORDER BY to some regression test queries
commit : 7e3c54168d9ec058cb3c9d47f8105b1d32dc8ceb
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 8 Apr 2021 12:20:11 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 8 Apr 2021 12:20:11 +0200
Apparently, an unrelated patch introduced some variation on the build
farm.
Reported-by: Magnus Hagander <magnus@hagander.net>
M src/test/regress/expected/create_function_3.out
M src/test/regress/sql/create_function_3.sql
Add functions to wait for backend termination
commit : aaf043257205ec523f1ba09a3856464d17cf2281
author : Magnus Hagander <magnus@hagander.net>
date : Thu, 8 Apr 2021 11:32:14 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Thu, 8 Apr 2021 11:32:14 +0200
This adds a function, pg_wait_for_backend_termination(), and a new
timeout argument to pg_terminate_backend(), which will wait for the
backend to actually terminate (with or without signaling it to do so
depending on which function is called). The default behaviour of
pg_terminate_backend() remains being timeout=0 which does not waiting.
For pg_wait_for_backend_termination() the default wait is 5 seconds.
Author: Bharath Rupireddy
Reviewed-By: Fujii Masao, David Johnston, Muhammad Usama,
Hou Zhijie, Magnus Hagander
Discussion: https://postgr.es/m/CALj2ACUBpunmyhYZw-kXCYs5NM+h6oG_7Df_Tn4mLmmUQifkqA@mail.gmail.com
M doc/src/sgml/func.sgml
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/storage/ipc/signalfuncs.c
M src/backend/utils/activity/wait_event.c
M src/include/catalog/pg_proc.dat
M src/include/utils/wait_event.h
doc: Prefer explicit JOIN syntax over old implicit syntax in tutorial
commit : fb310f17812e7321599845a29af2f36c7f1191c3
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 8 Apr 2021 10:51:26 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 8 Apr 2021 10:51:26 +0200
Update src/tutorial/basics.source to match.
Author: Jürgen Purtz <juergen@purtz.de>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: "David G. Johnston" <david.g.johnston@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/158996922318.7035.10603922579567326239@wrigleys.postgresql.org
M doc/src/sgml/query.sgml
M src/tutorial/basics.source
Track identical top vs nested queries independently in pg_stat_statements
commit : 6b4d23feef6e334fb85af077f2857f62ab781848
author : Magnus Hagander <magnus@hagander.net>
date : Thu, 8 Apr 2021 10:23:10 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Thu, 8 Apr 2021 10:23:10 +0200
Changing pg_stat_statements.track between 'all' and 'top' would control
if pg_stat_statements tracked just top level statements or also
statements inside functions, but when tracking all it would not
differentiate between the two. Being table to differentiate this is
useful both to track where the actual query is coming from, and to see
if there are differences in executions between the two.
To do this, add a boolean to the hash key indicating if the statement
was top level or not.
Experience from the pg_stat_kcache module shows that in at least some
"reasonable worloads" only <5% of the queries show up both top level and
nested. Based on this, admittedly small, dataset, this patch does not
try to de-duplicate those query *texts*, and will just store one copy
for the top level and one for the nested.
Author: Julien Rohaud
Reviewed-By: Magnus Hagander, Masahiro Ikeda
Discussion: https://postgr.es/m/20201202040516.GA43757@nol
M contrib/pg_stat_statements/Makefile
M contrib/pg_stat_statements/expected/pg_stat_statements.out
A contrib/pg_stat_statements/pg_stat_statements–1.9–1.10.sql
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/pg_stat_statements.control
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
M doc/src/sgml/pgstatstatements.sgml
Update Unicode data to CLDR 39
commit : 2e0e0666790e48cec716d4947f89d067ef53490c
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 8 Apr 2021 08:28:03 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 8 Apr 2021 08:28:03 +0200
M contrib/unaccent/unaccent.rules
M src/Makefile.global.in
Provide ReadRecentBuffer() to re-pin buffers by ID.
commit : 2f27f8c511494cca9e0e9a4eeeb102fa9f193002
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 8 Apr 2021 17:48:37 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 8 Apr 2021 17:48:37 +1200
If you know the ID of a buffer that recently held a block that you would
like to pin, this function can be used check if it's still there. It
can be used to avoid a second lookup in the buffer mapping table after
PrefetchBuffer() reports a cache hit.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA+hUKGJ4VJN8ttxScUFM8dOKX0BrBiboo5uz1cq=AovOddfHpA@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
M src/include/storage/bufmgr.h
autovacuum: handle analyze for partitioned tables
commit : 0827e8af70f4653ba17ed773f123a60eadd9f9c9
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 8 Apr 2021 01:19:36 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 8 Apr 2021 01:19:36 -0400
Previously, autovacuum would completely ignore partitioned tables, which
is not good regarding analyze -- failing to analyze those tables means
poor plans may be chosen. Make autovacuum aware of those tables by
propagating "changes since analyze" counts from the leaf partitions up
the partitioning hierarchy.
This also introduces necessary reloptions support for partitioned tables
(autovacuum_enabled, autovacuum_analyze_scale_factor,
autovacuum_analyze_threshold). It's unclear how best to document this
aspect.
Author: Yuzuko Hosoya <yuzukohosoya@gmail.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/CAKkQ508_PwVgwJyBY=0Lmkz90j8CmWNPUxgHvCUwGhMrouz6UA@mail.gmail.com
M src/backend/access/common/reloptions.c
M src/backend/catalog/system_views.sql
M src/backend/commands/analyze.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/pgstat.c
M src/include/pgstat.h
M src/test/regress/expected/rules.out
Cope with NULL query string in ExecInitParallelPlan().
commit : b3ee4c503872f3d0a5d6a7cbde48815f555af15b
author : Andres Freund <andres@anarazel.de>
date : Wed, 7 Apr 2021 22:00:01 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 7 Apr 2021 22:00:01 -0700
It's far from clear that this is the right approach - but a good
portion of the buildfarm has been red for a few hours, on the last day
of the CF. And this fixes at least the obvious crash. So let's go with
that for now.
Discussion: https://postgr.es/m/20210407225806.majgznh4lk34hjvu%40alap3.anarazel.de
M src/backend/executor/execParallel.c
Fix typo in jsonfuncs.c.
commit : 8ffb003591ff02f59d92c36a5791307881863146
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 8 Apr 2021 10:24:00 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 8 Apr 2021 10:24:00 +0530
Author: Tatsuro Yamada
Discussion: https://postgr.es/m/7c166a60-2808-6b89-9524-feefc6233748@nttcom.co.jp_1
M src/backend/utils/adt/jsonfuncs.c
Repair find_inheritance_children with no active snapshot
commit : 4131f755d548f74eba56285dc674f1f26e4ed6b4
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 8 Apr 2021 00:46:14 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 8 Apr 2021 00:46:14 -0400
When working on a scan with only a catalog snapshot, we may not have an
ActiveSnapshot set. If we were to come across a detached partition,
that would cause a crash. Fix by only ignoring detached partitions when
there's an active snapshot.
M src/backend/catalog/pg_inherits.c
Allow psql's \df and \do commands to specify argument types.
commit : a3027e1e7f3d3a107ecd72d3b4d6333ea2aab6a5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Apr 2021 23:02:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Apr 2021 23:02:16 -0400
When dealing with overloaded function or operator names, having
to look through a long list of matches is tedious. Let's extend
these commands to allow specification of (input) argument types
to let such results be trimmed down. Each additional argument
is treated the same as the pattern argument of \dT and matched
against the appropriate argument's type name.
While at it, fix \dT (and these new options) to recognize the
usual notation of "foo[]" for "the array type over foo", and
to handle the special abbreviations allowed by the backend
grammar, such as "int" for "integer".
Greg Sabino Mullane, revised rather significantly by me
Discussion: https://postgr.es/m/CAKAnmmLF9Hhu02N+s7uAyLc5J1xZReg72HQUoiKhNiJV3_jACQ@mail.gmail.com
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/bin/psql/help.c
M src/fe_utils/string_utils.c
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
Add csvlog output for the new query_id value
commit : f57a2f5e03054ade221e554c70e628e1ffae1b66
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 7 Apr 2021 22:30:30 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 7 Apr 2021 22:30:30 -0400
This also adjusts the printf format for query id used by log_line_prefix
(%Q).
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210408005402.GG24239@momjian.us
Author: Julien Rouhaud, Bruce Momjian
M doc/src/sgml/config.sgml
M doc/src/sgml/file-fdw.sgml
M src/backend/utils/error/elog.c
Teach VACUUM to bypass unnecessary index vacuuming.
commit : 5100010ee4d5c8ef46619dbd1d17090c627e6d0a
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 7 Apr 2021 16:14:54 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 7 Apr 2021 16:14:54 -0700
VACUUM has never needed to call ambulkdelete() for each index in cases
where there are precisely zero TIDs in its dead_tuples array by the end
of its first pass over the heap (also its only pass over the heap in
this scenario). Index vacuuming is simply not required when this
happens. Index cleanup will still go ahead, but in practice most calls
to amvacuumcleanup() are usually no-ops when there were zero preceding
ambulkdelete() calls. In short, VACUUM has generally managed to avoid
index scans when there were clearly no index tuples to delete from
indexes. But cases with _close to_ no index tuples to delete were
another matter -- a round of ambulkdelete() calls took place (one per
index), each of which performed a full index scan.
VACUUM now behaves just as if there were zero index tuples to delete in
cases where there are in fact "virtually zero" such tuples. That is, it
can now bypass index vacuuming and heap vacuuming as an optimization
(though not index cleanup). Whether or not VACUUM bypasses indexes is
determined dynamically, based on the just-observed number of heap pages
in the table that have one or more LP_DEAD items (LP_DEAD items in heap
pages have a 1:1 correspondence with index tuples that still need to be
deleted from each index in the worst case).
We only skip index vacuuming when 2% or less of the table's pages have
one or more LP_DEAD items -- bypassing index vacuuming as an
optimization must not noticeably impede setting bits in the visibility
map. As a further condition, the dead_tuples array (i.e. VACUUM's array
of LP_DEAD item TIDs) must not exceed 32MB at the point that the first
pass over the heap finishes, which is also when the decision to bypass
is made. (The VACUUM must also have been able to fit all TIDs in its
maintenance_work_mem-bound dead_tuples space, though with a default
maintenance_work_mem setting it can't matter.)
This avoids surprising jumps in the duration and overhead of routine
vacuuming with workloads where successive VACUUM operations consistently
have almost zero dead index tuples. The number of LP_DEAD items may
well accumulate over multiple VACUUM operations, before finally the
threshold is crossed and VACUUM performs conventional index vacuuming.
Even then, the optimization will have avoided a great deal of largely
unnecessary index vacuuming.
In the future we may teach VACUUM to skip index vacuuming on a per-index
basis, using a much more sophisticated approach. For now we only
consider the extreme cases, where we can be quite confident that index
vacuuming just isn't worth it using simple heuristics.
Also log information about how many heap pages have one or more LP_DEAD
items when autovacuum logging is enabled.
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAD21AoD0SkE11fMw4jD4RENAwBMcw1wasVnwpJVw3tVqPOQgAw@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WzmkebqPd4MVGuPTOS9bMFvp9MDs5cRTCOsv1rQJ3jCbXw@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Fix regression test failure caused by commit 4f0b0966c8
commit : bc70728693bc2d28db7125e7a24d78ad7612f58c
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 7 Apr 2021 18:14:36 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 7 Apr 2021 18:14:36 -0400
The query originally used was too simple, cause explain_filter() to be
unable to remove JIT output text.
Reported-by: Tom Lane
Author: Julien Rouhaud
M src/test/regress/expected/explain.out
M src/test/regress/sql/explain.sql
Fix some failures with connection tests on Windows hosts
commit : c7578fa64019f27edc31261ea49066a4b2569a6c
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 8 Apr 2021 06:55:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 8 Apr 2021 06:55:00 +0900
The truncation of the log file, that this set of tests relies on to make
sure that a connection attempt matches with its expected backend log
pattern, fails, as reported by buildfarm member fairywren. Instead of a
truncation, do a rotation of the log file and restart the node. This
will ensure that the connection attempt data is unique for each test.
Discussion: https://postgr.es/m/YG05nCI8x8B+Ad3G@paquier.xyz
M src/test/perl/PostgresNode.pm
SQL-standard function body
commit : e717a9a18b2e34c9c40e5259ad4d31cd7e420750
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 7 Apr 2021 21:30:08 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 7 Apr 2021 21:30:08 +0200
This adds support for writing CREATE FUNCTION and CREATE PROCEDURE
statements for language SQL with a function body that conforms to the
SQL standard and is portable to other implementations.
Instead of the PostgreSQL-specific AS $$ string literal $$ syntax,
this allows writing out the SQL statements making up the body
unquoted, either as a single statement:
CREATE FUNCTION add(a integer, b integer) RETURNS integer
LANGUAGE SQL
RETURN a + b;
or as a block
CREATE PROCEDURE insert_data(a integer, b integer)
LANGUAGE SQL
BEGIN ATOMIC
INSERT INTO tbl VALUES (a);
INSERT INTO tbl VALUES (b);
END;
The function body is parsed at function definition time and stored as
expression nodes in a new pg_proc column prosqlbody. So at run time,
no further parsing is required.
However, this form does not support polymorphic arguments, because
there is no more parse analysis done at call time.
Dependencies between the function and the objects it uses are fully
tracked.
A new RETURN statement is introduced. This can only be used inside
function bodies. Internally, it is treated much like a SELECT
statement.
psql needs some new intelligence to keep track of function body
boundaries so that it doesn't send off statements when it sees
semicolons that are inside a function body.
Tested-by: Jaime Casanova <jcasanov@systemguards.com.ec>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/1c11f1eb-f00c-43b7-799d-2d44132c02d7@2ndquadrant.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/create_function.sgml
M doc/src/sgml/ref/create_procedure.sgml
M src/backend/catalog/pg_aggregate.c
M src/backend/catalog/pg_proc.c
M src/backend/commands/aggregatecmds.c
M src/backend/commands/functioncmds.c
M src/backend/commands/typecmds.c
M src/backend/executor/functions.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/tcop/postgres.c
M src/backend/utils/adt/ruleutils.c
M src/bin/pg_dump/pg_dump.c
M src/bin/psql/describe.c
M src/fe_utils/psqlscan.l
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_proc.h
M src/include/commands/defrem.h
M src/include/executor/functions.h
M src/include/fe_utils/psqlscan_int.h
M src/include/nodes/nodes.h
M src/include/nodes/parsenodes.h
M src/include/parser/kwlist.h
M src/include/tcop/tcopprot.h
M src/interfaces/ecpg/preproc/ecpg.addons
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/test/regress/expected/create_function_3.out
M src/test/regress/expected/create_procedure.out
M src/test/regress/sql/create_function_3.sql
M src/test/regress/sql/create_procedure.sql
Add wraparound failsafe to VACUUM.
commit : 1e55e7d1755cefbb44982fbacc7da461fa8684e6
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 7 Apr 2021 12:37:45 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 7 Apr 2021 12:37:45 -0700
Add a failsafe mechanism that is triggered by VACUUM when it notices
that the table's relfrozenxid and/or relminmxid are dangerously far in
the past. VACUUM checks the age of the table dynamically, at regular
intervals.
When the failsafe triggers, VACUUM takes extraordinary measures to
finish as quickly as possible so that relfrozenxid and/or relminmxid can
be advanced. VACUUM will stop applying any cost-based delay that may be
in effect. VACUUM will also bypass any further index vacuuming and heap
vacuuming -- it only completes whatever remaining pruning and freezing
is required. Bypassing index/heap vacuuming is enabled by commit
8523492d, which made it possible to dynamically trigger the mechanism
already used within VACUUM when it is run with INDEX_CLEANUP off.
It is expected that the failsafe will almost always trigger within an
autovacuum to prevent wraparound, long after the autovacuum began.
However, the failsafe mechanism can trigger in any VACUUM operation.
Even in a non-aggressive VACUUM, where we're likely to not advance
relfrozenxid, it still seems like a good idea to finish off remaining
pruning and freezing. An aggressive/anti-wraparound VACUUM will be
launched immediately afterwards. Note that the anti-wraparound VACUUM
that follows will itself trigger the failsafe, usually before it even
begins its first (and only) pass over the heap.
The failsafe is controlled by two new GUCs: vacuum_failsafe_age, and
vacuum_multixact_failsafe_age. There are no equivalent reloptions,
since that isn't expected to be useful. The GUCs have rather high
defaults (both default to 1.6 billion), and are expected to generally
only be used to make the failsafe trigger sooner/more frequently.
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAD21AoD0SkE11fMw4jD4RENAwBMcw1wasVnwpJVw3tVqPOQgAw@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WzmgH3ySGYeC-m-eOBsa2=sDwa292-CFghV4rESYo39FsQ@mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/vacuum.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/commands/vacuum.h
Make use of in-core query id added by commit 5fd9dfa5f5
commit : 4f0b0966c866ae9f0e15d7cc73ccf7ce4e1af84b
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 7 Apr 2021 14:03:56 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 7 Apr 2021 14:03:56 -0400
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
M contrib/pg_stat_statements/pg_stat_statements.c
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/ref/explain.sgml
M src/backend/catalog/system_views.sql
M src/backend/commands/explain.c
M src/backend/executor/execMain.c
M src/backend/executor/execParallel.c
M src/backend/parser/analyze.c
M src/backend/tcop/postgres.c
M src/backend/utils/activity/backend_status.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/error/elog.c
M src/backend/utils/misc/postgresql.conf.sample
M src/backend/utils/misc/queryjumble.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/utils/backend_status.h
M src/test/regress/expected/explain.out
M src/test/regress/expected/rules.out
M src/test/regress/sql/explain.sql
amcheck: fix multiple problems with TOAST pointer validation
commit : ec7ffb8096e8eb90f4c9230f7ba9487f0abe1a9f
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 7 Apr 2021 13:28:35 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 7 Apr 2021 13:28:35 -0400
First, don't perform database access while holding a buffer lock.
When checking a heap, we can validate that TOAST pointers are sane by
performing a scan on the TOAST index and looking up the chunks that
correspond to each value ID that appears in a TOAST poiner in the main
table. But, to do that while holding a buffer lock at least risks
causing other backends to wait uninterruptibly, and probably can cause
undetected and uninterruptible deadlocks. So, instead, make a list of
checks to perform while holding the lock, and then perform the checks
after releasing it.
Second, adjust things so that we don't try to follow TOAST pointers
for tuples that are already eligible to be pruned. The TOAST tuples
become eligible for pruning at the same time that the main tuple does,
so trying to check them may lead to spurious reports of corruption,
as observed in the buildfarm. The necessary infrastructure to decide
whether or not the tuple being checked is prunable was added by
commit 3b6c1259f9ca8e21860aaf24ec6735a8e5598ea0, but it wasn't
actually used for its intended purpose prior to this patch.
Mark Dilger, adjusted by me to avoid a memory leak.
Discussion: http://postgr.es/m/AC5479E4-6321-473D-AC92-5EC36299FBC2@enterprisedb.com
M contrib/amcheck/verify_heapam.c
M src/tools/pgindent/typedefs.list
Move pg_stat_statements query jumbling to core.
commit : 5fd9dfa5f50e4906c35133a414ebec5b6d518493
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 7 Apr 2021 13:06:47 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 7 Apr 2021 13:06:47 -0400
Add compute_query_id GUC to control whether a query identifier should be
computed by the core (off by default). It's thefore now possible to
disable core queryid computation and use pg_stat_statements with a
different algorithm to compute the query identifier by using a
third-party module.
To ensure that a single source of query identifier can be used and is
well defined, modules that calculate a query identifier should throw an
error if compute_query_id specified to compute a query id and if a query
idenfitier was already calculated.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/pg_stat_statements.conf
M doc/src/sgml/config.sgml
M doc/src/sgml/pgstatstatements.sgml
M src/backend/parser/analyze.c
M src/backend/tcop/postgres.c
M src/backend/utils/misc/Makefile
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
A src/backend/utils/misc/queryjumble.c
M src/include/parser/analyze.h
M src/include/utils/guc.h
A src/include/utils/queryjumble.h
Remove channel binding requirement from clientcert=verify-full test.
commit : a282ee68a070a8adc6e6d45e8e643769c587ecc3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Apr 2021 12:50:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Apr 2021 12:50:17 -0400
This fails on older OpenSSL versions that lack channel binding
support. Since that feature is not essential to this test case,
just remove it, instead of complicating matters. Per buildfarm.
Jacob Champion
Discussion: https://postgr.es/m/fa8dbbb58c20b1d1adf0082769f80d5466eaf485.camel@vmware.com
M src/test/ssl/t/002_scram.pl
Comment cleanup for a1115fa07.
commit : 0d46771eaaf77ad08555cf34e421234d5943edfa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Apr 2021 12:21:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Apr 2021 12:21:54 -0400
Amit Langote
Discussion: https://postgr.es/m/CA+HiwqEcawatEaUh1uTbZMEZTJeLzbroRTz9_X9Z5CFjTWJkhw@mail.gmail.com
M src/backend/executor/nodeModifyTable.c
amcheck: Remove duplicate XID/MXID bounds checks.
commit : 4573f6a9af6e232ba073392716a051ae2017d1e9
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 7 Apr 2021 12:11:44 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 7 Apr 2021 12:11:44 -0400
Commit 3b6c1259f9ca8e21860aaf24ec6735a8e5598ea0 resulted in the same
xmin and xmax bounds checking being performed in both check_tuple()
and check_tuple_visibility(). Remove the duplication.
While at it, adjust some code comments that were overlooked in that
commit.
Mark Dilger
Discussion: http://postgr.es/m/AC5479E4-6321-473D-AC92-5EC36299FBC2@enterprisedb.com
M contrib/amcheck/verify_heapam.c
Truncate line pointer array during VACUUM.
commit : 3c3b8a4b26891892bccf3d220580a7f413c0b9ca
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 7 Apr 2021 08:47:15 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 7 Apr 2021 08:47:15 -0700
Teach VACUUM to truncate the line pointer array of each heap page when a
contiguous group of LP_UNUSED line pointers appear at the end of the
array -- these unused and unreferenced items are excluded. This process
occurs during VACUUM's second pass over the heap, right after LP_DEAD
line pointers on the page (those encountered/pruned during the first
pass) are marked LP_UNUSED.
Truncation avoids line pointer bloat with certain workloads,
particularly those involving continual range DELETEs and bulk INSERTs
against the same table.
Also harden heapam code to check for an out-of-range page offset number
in places where we weren't already doing so.
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-By: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAEze2WjgaQc55Y5f5CQd3L=eS5CZcff2Obxp=O6pto8-f0hC4w@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-Wzn6a64PJM1Ggzm=uvx2otsopJMhFQj_g1rAj4GWr3ZSzw@mail.gmail.com
M src/backend/access/heap/heapam.c
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/storage/page/bufpage.c
M src/include/storage/bufpage.h
Tighten up allowed names for custom GUC parameters.
commit : 3db826bd55cd1df0dd8c3d811f8e5b936d7ba1e4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Apr 2021 11:22:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Apr 2021 11:22:22 -0400
Formerly we were pretty lax about what a custom GUC's name could
be; so long as it had at least one dot in it, we'd take it.
However, corner cases such as dashes or equal signs in the name
would cause various bits of functionality to misbehave. Rather
than trying to make the world perfectly safe for that, let's
just require that custom names look like "identifier.identifier",
where "identifier" means something that scan.l would accept
without double quotes.
Along the way, this patch refactors things slightly in guc.c
so that find_option() is responsible for reporting GUC-not-found
cases, allowing removal of duplicative code from its callers.
Per report from Hubert Depesz Lubaczewski. No back-patch,
since the consequences of the problem don't seem to warrant
changing behavior in stable branches.
Discussion: https://postgr.es/m/951335.1612910077@sss.pgh.pa.us
M src/backend/utils/misc/guc-file.l
M src/backend/utils/misc/guc.c
M src/test/regress/expected/guc.out
M src/test/regress/sql/guc.sql
Don't add non-existent pages to bitmap from BRIN
commit : 23607a8156d595522c232ff3933d77041d3adaa1
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 7 Apr 2021 15:58:35 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 7 Apr 2021 15:58:35 +0200
The code in bringetbitmap() simply added the whole matching page range
to the TID bitmap, as determined by pages_per_range, even if some of the
pages were beyond the end of the heap. The query then might fail with
an error like this:
ERROR: could not open file "base/20176/20228.2" (target block
262144): previous segment is only 131021 blocks
In this case, the relation has 262093 pages (131072 and 131021 pages),
but we're trying to acess block 262144, i.e. first block of the 3rd
segment. At that point _mdfd_getseg() notices the preceding segment is
incomplete, and fails.
Hitting this in practice is rather unlikely, because:
* Most indexes use power-of-two ranges, so segments and page ranges
align perfectly (segment end is also a page range end).
* The table size has to be just right, with the last segment being
almost full - less than one page range from full segment, so that the
last page range actually crosses the segment boundary.
* Prefetch has to be enabled. The regular page access checks that
pages are not beyond heap end, but prefetch does not. On older
releases (before 12) the execution stops after hitting the first
non-existent page, so the prefetch distance has to be sufficient
to reach the first page in the next segment to trigger the issue.
Since 12 it's enough to just have prefetch enabled, the prefetch
distance does not matter.
Fixed by not adding non-existent pages to the TID bitmap. Backpatch
all the way back to 9.6 (BRIN indexes were introduced in 9.5, but that
release is EOL).
Backpatch-through: 9.6
M src/backend/access/brin/brin.c
libpq: Set Server Name Indication (SNI) for SSL connections
commit : 5c55dc8b47338e72a4e598c155d2048d756fd10e
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 7 Apr 2021 15:11:41 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 7 Apr 2021 15:11:41 +0200
By default, have libpq set the TLS extension "Server Name Indication" (SNI).
This allows an SNI-aware SSL proxy to route connections. (This
requires a proxy that is aware of the PostgreSQL protocol, not just
any SSL proxy.)
In the future, this could also allow the server to use different SSL
certificates for different host specifications. (That would require
new server functionality. This would be the client-side functionality
for that.)
Since SNI makes the host name appear in cleartext in the network
traffic, this might be undesirable in some cases. Therefore, also add
a libpq connection option "sslsni" to turn it off.
Discussion: https://www.postgresql.org/message-id/flat/7289d5eb-62a5-a732-c3b9-438cee2cb709%40enterprisedb.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/libpq-int.h
Refactor hba_authname
commit : c1968426ba3de1fe37848863e35fff30261bf941
author : Magnus Hagander <magnus@hagander.net>
date : Wed, 7 Apr 2021 14:21:19 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Wed, 7 Apr 2021 14:21:19 +0200
The previous implementation (from 9afffcb833) had an unnecessary check
on the boundaries of the enum which trigtered compile warnings. To clean
it up, move the pre-existing static assert to a central location and
call that.
Reported-By: Erik Rijkers
Reviewed-By: Michael Paquier
Discussion: https://postgr.es/m/1056399262.13159.1617793249020@webmailclassic.xs4all.nl
M src/backend/libpq/auth.c
M src/backend/libpq/hba.c
M src/include/libpq/hba.h
doc: Improve wording
commit : 4560e0acdafd57f3ba109b98e15ac047798d960c
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 7 Apr 2021 13:52:26 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 7 Apr 2021 13:52:26 +0200
Discussion: https://www.postgresql.org/message-id/flat/161626776179.652.11944895442156126506%40wrigleys.postgresql.org
M doc/src/sgml/dml.sgml
Revert "Add sortsupport for gist_btree opclasses, for faster index builds."
commit : d92b1cdbab408d8f1299257125c9ae375f3ca644
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 7 Apr 2021 14:33:21 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 7 Apr 2021 14:33:21 +0300
This reverts commit 9f984ba6d23dc6eecebf479ab1d3f2e550a4e9be.
It was making the buildfarm unhappy, apparently setting client_min_messages
in a regression test produces different output if log_statement='all'.
Another issue is that I now suspect the bit sortsupport function was in
fact not correct to call byteacmp(). Revert to investigate both of those
issues.
M contrib/btree_gist/Makefile
M contrib/btree_gist/btree_bit.c
M contrib/btree_gist/btree_bytea.c
M contrib/btree_gist/btree_cash.c
M contrib/btree_gist/btree_date.c
M contrib/btree_gist/btree_enum.c
M contrib/btree_gist/btree_float4.c
M contrib/btree_gist/btree_float8.c
D contrib/btree_gist/btree_gist–1.6–1.7.sql
M contrib/btree_gist/btree_gist.control
M contrib/btree_gist/btree_gist.h
M contrib/btree_gist/btree_inet.c
M contrib/btree_gist/btree_int2.c
M contrib/btree_gist/btree_int4.c
M contrib/btree_gist/btree_int8.c
M contrib/btree_gist/btree_interval.c
M contrib/btree_gist/btree_macaddr.c
M contrib/btree_gist/btree_macaddr8.c
M contrib/btree_gist/btree_numeric.c
M contrib/btree_gist/btree_oid.c
M contrib/btree_gist/btree_text.c
M contrib/btree_gist/btree_time.c
M contrib/btree_gist/btree_ts.c
M contrib/btree_gist/btree_uuid.c
M contrib/btree_gist/expected/bit.out
M contrib/btree_gist/expected/bytea.out
M contrib/btree_gist/expected/cash.out
M contrib/btree_gist/expected/char.out
M contrib/btree_gist/expected/cidr.out
M contrib/btree_gist/expected/date.out
M contrib/btree_gist/expected/enum.out
M contrib/btree_gist/expected/float4.out
M contrib/btree_gist/expected/float8.out
M contrib/btree_gist/expected/inet.out
M contrib/btree_gist/expected/int2.out
M contrib/btree_gist/expected/int4.out
M contrib/btree_gist/expected/int8.out
M contrib/btree_gist/expected/interval.out
M contrib/btree_gist/expected/macaddr.out
M contrib/btree_gist/expected/macaddr8.out
M contrib/btree_gist/expected/numeric.out
M contrib/btree_gist/expected/oid.out
M contrib/btree_gist/expected/text.out
M contrib/btree_gist/expected/time.out
M contrib/btree_gist/expected/timestamp.out
M contrib/btree_gist/expected/timestamptz.out
M contrib/btree_gist/expected/timetz.out
M contrib/btree_gist/expected/uuid.out
M contrib/btree_gist/expected/varbit.out
M contrib/btree_gist/expected/varchar.out
M contrib/btree_gist/sql/bit.sql
M contrib/btree_gist/sql/bytea.sql
M contrib/btree_gist/sql/cash.sql
M contrib/btree_gist/sql/char.sql
M contrib/btree_gist/sql/cidr.sql
M contrib/btree_gist/sql/date.sql
M contrib/btree_gist/sql/enum.sql
M contrib/btree_gist/sql/float4.sql
M contrib/btree_gist/sql/float8.sql
M contrib/btree_gist/sql/inet.sql
M contrib/btree_gist/sql/int2.sql
M contrib/btree_gist/sql/int4.sql
M contrib/btree_gist/sql/int8.sql
M contrib/btree_gist/sql/interval.sql
M contrib/btree_gist/sql/macaddr.sql
M contrib/btree_gist/sql/macaddr8.sql
M contrib/btree_gist/sql/numeric.sql
M contrib/btree_gist/sql/oid.sql
M contrib/btree_gist/sql/text.sql
M contrib/btree_gist/sql/time.sql
M contrib/btree_gist/sql/timestamp.sql
M contrib/btree_gist/sql/timestamptz.sql
M contrib/btree_gist/sql/timetz.sql
M contrib/btree_gist/sql/uuid.sql
M contrib/btree_gist/sql/varbit.sql
M contrib/btree_gist/sql/varchar.sql
M src/backend/access/gist/gistbuild.c
Add sortsupport for gist_btree opclasses, for faster index builds.
commit : 9f984ba6d23dc6eecebf479ab1d3f2e550a4e9be
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 7 Apr 2021 13:22:05 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 7 Apr 2021 13:22:05 +0300
Commit 16fa9b2b30 introduced a faster way to build GiST indexes, by
sorting all the data. This commit adds the sortsupport functions needed
to make use of that feature for btree_gist.
Author: Andrey Borodin
Discussion: https://www.postgresql.org/message-id/2F3F7265-0D22-44DB-AD71-8554C743D943@yandex-team.ru
M contrib/btree_gist/Makefile
M contrib/btree_gist/btree_bit.c
M contrib/btree_gist/btree_bytea.c
M contrib/btree_gist/btree_cash.c
M contrib/btree_gist/btree_date.c
M contrib/btree_gist/btree_enum.c
M contrib/btree_gist/btree_float4.c
M contrib/btree_gist/btree_float8.c
A contrib/btree_gist/btree_gist–1.6–1.7.sql
M contrib/btree_gist/btree_gist.control
M contrib/btree_gist/btree_gist.h
M contrib/btree_gist/btree_inet.c
M contrib/btree_gist/btree_int2.c
M contrib/btree_gist/btree_int4.c
M contrib/btree_gist/btree_int8.c
M contrib/btree_gist/btree_interval.c
M contrib/btree_gist/btree_macaddr.c
M contrib/btree_gist/btree_macaddr8.c
M contrib/btree_gist/btree_numeric.c
M contrib/btree_gist/btree_oid.c
M contrib/btree_gist/btree_text.c
M contrib/btree_gist/btree_time.c
M contrib/btree_gist/btree_ts.c
M contrib/btree_gist/btree_uuid.c
M contrib/btree_gist/expected/bit.out
M contrib/btree_gist/expected/bytea.out
M contrib/btree_gist/expected/cash.out
M contrib/btree_gist/expected/char.out
M contrib/btree_gist/expected/cidr.out
M contrib/btree_gist/expected/date.out
M contrib/btree_gist/expected/enum.out
M contrib/btree_gist/expected/float4.out
M contrib/btree_gist/expected/float8.out
M contrib/btree_gist/expected/inet.out
M contrib/btree_gist/expected/int2.out
M contrib/btree_gist/expected/int4.out
M contrib/btree_gist/expected/int8.out
M contrib/btree_gist/expected/interval.out
M contrib/btree_gist/expected/macaddr.out
M contrib/btree_gist/expected/macaddr8.out
M contrib/btree_gist/expected/numeric.out
M contrib/btree_gist/expected/oid.out
M contrib/btree_gist/expected/text.out
M contrib/btree_gist/expected/time.out
M contrib/btree_gist/expected/timestamp.out
M contrib/btree_gist/expected/timestamptz.out
M contrib/btree_gist/expected/timetz.out
M contrib/btree_gist/expected/uuid.out
M contrib/btree_gist/expected/varbit.out
M contrib/btree_gist/expected/varchar.out
M contrib/btree_gist/sql/bit.sql
M contrib/btree_gist/sql/bytea.sql
M contrib/btree_gist/sql/cash.sql
M contrib/btree_gist/sql/char.sql
M contrib/btree_gist/sql/cidr.sql
M contrib/btree_gist/sql/date.sql
M contrib/btree_gist/sql/enum.sql
M contrib/btree_gist/sql/float4.sql
M contrib/btree_gist/sql/float8.sql
M contrib/btree_gist/sql/inet.sql
M contrib/btree_gist/sql/int2.sql
M contrib/btree_gist/sql/int4.sql
M contrib/btree_gist/sql/int8.sql
M contrib/btree_gist/sql/interval.sql
M contrib/btree_gist/sql/macaddr.sql
M contrib/btree_gist/sql/macaddr8.sql
M contrib/btree_gist/sql/numeric.sql
M contrib/btree_gist/sql/oid.sql
M contrib/btree_gist/sql/text.sql
M contrib/btree_gist/sql/time.sql
M contrib/btree_gist/sql/timestamp.sql
M contrib/btree_gist/sql/timestamptz.sql
M contrib/btree_gist/sql/timetz.sql
M contrib/btree_gist/sql/uuid.sql
M contrib/btree_gist/sql/varbit.sql
M contrib/btree_gist/sql/varchar.sql
M src/backend/access/gist/gistbuild.c
Fix use of cursor sensitivity terminology
commit : dd13ad9d39a1ba41cf329b6fe408b49be57c7b88
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 7 Apr 2021 07:49:27 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 7 Apr 2021 07:49:27 +0200
Documentation and comments in code and tests have been using the terms
sensitive/insensitive cursor incorrectly relative to the SQL standard.
(Cursor sensitivity is only relevant for changes made in the same
transaction as the cursor, not for concurrent changes in other
sessions.) Moreover, some of the behavior of PostgreSQL is incorrect
according to the SQL standard, confusing the issue further. (WHERE
CURRENT OF changes are not visible in insensitive cursors, but they
should be.)
This change corrects the terminology and removes the claim that
sensitive cursors are supported. It also adds a test case that checks
the insensitive behavior in a "correct" way, using a change command
not using WHERE CURRENT OF. Finally, it adds the ASENSITIVE cursor
option to select the default asensitive behavior, per SQL standard.
There are no changes to cursor behavior in this patch.
Discussion: https://www.postgresql.org/message-id/flat/96ee8b30-9889-9e1b-b053-90e10c050e85%40enterprisedb.com
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/ref/declare.sgml
M src/backend/catalog/sql_features.txt
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/bin/psql/tab-complete.c
M src/include/nodes/parsenodes.h
M src/include/parser/kwlist.h
M src/test/regress/expected/portals.out
M src/test/regress/sql/portals.sql
Message improvement
commit : 0b5e8245283eef67e88fb5380836cdc2c743d848
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 7 Apr 2021 07:37:30 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 7 Apr 2021 07:37:30 +0200
The previous wording contained a superfluous comma. Adjust phrasing
for grammatical correctness and clarity.
M contrib/test_decoding/expected/slot.out
M src/backend/replication/logical/logicalfuncs.c
M src/backend/replication/slotfuncs.c
Remove redundant memset(0) calls for page init of some index AMs
commit : 4c0239cb7a7775e3183cb575e62703d71bf3302d
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 7 Apr 2021 14:35:26 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 7 Apr 2021 14:35:26 +0900
Bloom, GIN, GiST and SP-GiST rely on PageInit() to initialize the
contents of a page, and this routine fills entirely a page with zeros
for a size of BLCKSZ, including the special space. Those index AMs have
been using an extra memset() call to fill with zeros the special page
space, or even the whole page, which is not necessary as PageInit()
already does this work, so let's remove them. GiST was not doing this
extra call, but has commented out a system call that did so since
6236991.
While on it, remove one MAXALIGN() for SP-GiST as PageInit() takes care
of that. This makes the whole page initialization logic more consistent
across all index AMs.
Author: Bharath Rupireddy
Reviewed-by: Vignesh C, Mahendra Singh Thalor
Discussion: https://postgr.es/m/CALj2ACViOo2qyaPT7krWm4LRyRTw9kOXt+g6PfNmYuGA=YHj9A@mail.gmail.com
M contrib/bloom/blinsert.c
M contrib/bloom/blutils.c
M src/backend/access/gin/ginutil.c
M src/backend/access/gist/gistutil.c
M src/backend/access/spgist/spgutils.c
Add some information about authenticated identity via log_connections
commit : 9afffcb833d3c5e59a328a2af674fac7e7334fc1
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 7 Apr 2021 10:16:39 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 7 Apr 2021 10:16:39 +0900
The "authenticated identity" is the string used by an authentication
method to identify a particular user. In many common cases, this is the
same as the PostgreSQL username, but for some third-party authentication
methods, the identifier in use may be shortened or otherwise translated
(e.g. through pg_ident user mappings) before the server stores it.
To help administrators see who has actually interacted with the system,
this commit adds the capability to store the original identity when
authentication succeeds within the backend's Port, and generates a log
entry when log_connections is enabled. The log entries generated look
something like this (where a local user named "foouser" is connecting to
the database as the database user called "admin"):
LOG: connection received: host=[local]
LOG: connection authenticated: identity="foouser" method=peer (/data/pg_hba.conf:88)
LOG: connection authorized: user=admin database=postgres application_name=psql
Port->authn_id is set according to the authentication method:
bsd: the PostgreSQL username (aka the local username)
cert: the client's Subject DN
gss: the user principal
ident: the remote username
ldap: the final bind DN
pam: the PostgreSQL username (aka PAM username)
password (and all pw-challenge methods): the PostgreSQL username
peer: the peer's pw_name
radius: the PostgreSQL username (aka the RADIUS username)
sspi: either the down-level (SAM-compatible) logon name, if
compat_realm=1, or the User Principal Name if compat_realm=0
The trust auth method does not set an authenticated identity. Neither
does clientcert=verify-full.
Port->authn_id could be used for other purposes, like a superuser-only
extra column in pg_stat_activity, but this is left as future work.
PostgresNode::connect_{ok,fails}() have been modified to let tests check
the backend log files for required or prohibited patterns, using the
new log_like and log_unlike parameters. This uses a method based on a
truncation of the existing server log file, like issues_sql_like().
Tests are added to the ldap, kerberos, authentication and SSL test
suites.
Author: Jacob Champion
Reviewed-by: Stephen Frost, Magnus Hagander, Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/c55788dd1773c521c862e8e0dddb367df51222be.camel@vmware.com
M doc/src/sgml/config.sgml
M src/backend/libpq/auth.c
M src/backend/libpq/hba.c
M src/include/libpq/hba.h
M src/include/libpq/libpq-be.h
M src/test/authentication/t/001_password.pl
M src/test/kerberos/t/001_auth.pl
M src/test/ldap/t/001_auth.pl
M src/test/perl/PostgresNode.pm
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
Fix test added by commit 9de9294b0c.
commit : 8ee9b662daa6d51b54d21ec274f22a218462ad2d
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 7 Apr 2021 07:42:36 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 7 Apr 2021 07:42:36 +0900
The buildfarm members "drongo" and "fairywren" reported that
the regression test (024_archive_recovery.pl) added by commit 9de9294b0c
failed. The cause of this failure is that the test calls $node->init()
without "allows_streaming => 1" and which doesn't add pg_hba.conf
entry for TCP/IP connection from pg_basebackup.
This commit fixes the issue by specifying "allows_streaming => 1"
when calling $node->init().
Author: Fujii Masao
Discussion: https://postgr.es/m/3cc3909d-f779-7a74-c201-f1f7f62c7497@oss.nttdata.com
M src/test/recovery/t/024_archive_recovery.pl
Postpone some more stuff out of ExecInitModifyTable.
commit : a1115fa0782378a8238045d238ae70cac36be8ae
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Apr 2021 18:13:05 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Apr 2021 18:13:05 -0400
Delay creation of the projections for INSERT and UPDATE tuples
until they're needed. This saves a pretty fair amount of work
when only some of the partitions are actually touched.
The logic associated with identifying junk columns in UPDATE/DELETE
is moved to another loop, allowing removal of one loop over the
target relations; but it didn't actually change at all.
Extracted from a larger patch, which seemed to me to be too messy
to push in one commit.
Amit Langote, reviewed at different times by Heikki Linnakangas and
myself
Discussion: https://postgr.es/m/CA+HiwqG7ZruBmmih3wPsBZ4s0H2EhywrnXEduckY5Hr3fWzPWA@mail.gmail.com
M src/backend/executor/execMain.c
M src/backend/executor/nodeModifyTable.c
M src/include/nodes/execnodes.h
Fix compiler warning for MSVC in libpq_pipeline.c
commit : 3b82d990ab784881153c0f127e4c1211e9b6065c
author : David Rowley <drowley@postgresql.org>
date : Wed, 7 Apr 2021 09:51:33 +1200
committer: David Rowley <drowley@postgresql.org>
date : Wed, 7 Apr 2021 09:51:33 +1200
DEBUG was already defined by the MSVC toolchain for "Debug" builds. On
these systems the unconditional #define DEBUG was causing a 'DEBUG': macro
redefinition warning.
Here we rename DEBUG to DEBUG_OUPUT and also get rid of the #define which
defined this constant. This appears to have been left in the code by
mistake.
Discussion: https://postgr.es/m/CAApHDvqTTgDm38s4HRj03nhzhzQ1oMOj-RXFUB1pE6Bj07jyuQ@mail.gmail.com
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Postpone some stuff out of ExecInitModifyTable.
commit : c5b7ba4e67aeb5d6f824b74f94114d99ed6e42b7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Apr 2021 15:56:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Apr 2021 15:56:55 -0400
Arrange to do some things on-demand, rather than immediately during
executor startup, because there's a fair chance of never having to do
them at all:
* Don't open result relations' indexes until needed.
* Don't initialize partition tuple routing, nor the child-to-root
tuple conversion map, until needed.
This wins in UPDATEs on partitioned tables when only some of the
partitions will actually receive updates; with larger partition
counts the savings is quite noticeable. Also, we can remove some
sketchy heuristics in ExecInitModifyTable about whether to set up
tuple routing.
Also, remove execPartition.c's private hash table tracking which
partitions were already opened by the ModifyTable node. Instead
use the hash added to ModifyTable itself by commit 86dc90056.
To allow lazy computation of the conversion maps, we now set
ri_RootResultRelInfo in all child ResultRelInfos. We formerly set it
only in some, not terribly well-defined, cases. This has user-visible
side effects in that now more error messages refer to the root
relation instead of some partition (and provide error data in the
root's column order, too). It looks to me like this is a strict
improvement in consistency, so I don't have a problem with the
output changes visible in this commit.
Extracted from a larger patch, which seemed to me to be too messy
to push in one commit.
Amit Langote, reviewed at different times by Heikki Linnakangas and
myself
Discussion: https://postgr.es/m/CA+HiwqG7ZruBmmih3wPsBZ4s0H2EhywrnXEduckY5Hr3fWzPWA@mail.gmail.com
M src/backend/commands/copyfrom.c
M src/backend/commands/trigger.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeModifyTable.c
M src/backend/replication/logical/worker.c
M src/include/executor/execPartition.h
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
M src/test/regress/expected/inherit.out
M src/test/regress/expected/privileges.out
M src/test/regress/expected/update.out
postgres_fdw: Allow partitions specified in LIMIT TO to be imported.
commit : a3740c48eb2f91663c7c06c948dfcfb6493d2588
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 7 Apr 2021 02:32:10 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 7 Apr 2021 02:32:10 +0900
Commit f49bcd4ef3 disallowed postgres_fdw to import table partitions.
Because all data can be accessed through the partitioned table which
is the root of the partitioning hierarchy, importing only partitioned
table should allow access to all the data without creating extra objects.
This is a reasonable default when importing a whole schema. But there
may be the case where users want to explicitly import one of
a partitioned tables' partitions.
For that use case, this commit allows postgres_fdw to import tables or
foreign tables which are partitions of some other table only when they
are explicitly specified in LIMIT TO clause. It doesn't change
the behavior that any partitions not specified in LIMIT TO are
automatically excluded in IMPORT FOREIGN SCHEMA command.
Author: Matthias van de Meent
Reviewed-by: Bernd Helmle, Amit Langote, Michael Paquier, Fujii Masao
Discussion: https://postgr.es/m/CAEze2Whwg4i=mzApMe+PXxCEfgoZmHGqdqQFW7J4bmj_5p6t1A@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Increment xactCompletionCount during subtransaction abort.
commit : 90c885cdab8bc5a5f12a243774fa0db51002a2fd
author : Andres Freund <andres@anarazel.de>
date : Tue, 6 Apr 2021 09:24:50 -0700
committer: Andres Freund <andres@anarazel.de>
date : Tue, 6 Apr 2021 09:24:50 -0700
Snapshot caching, introduced in 623a9ba79b, did not increment
xactCompletionCount during subtransaction abort. That could lead to an older
snapshot being reused. That is, at least as far as I can see, not a
correctness issue (for MVCC snapshots there's no difference between "in
progress" and "aborted"). The only difference between the old and new
snapshots would be a newer ->xmax.
While HeapTupleSatisfiesMVCC makes the same visibility determination, reusing
the old snapshot leads HeapTupleSatisfiesMVCC to not set
HEAP_XMIN_INVALID. Which subsequently causes the kill_prior_tuple optimization
to not kick in (via HeapTupleIsSurelyDead() returning false). The performance
effects of doing the same index-lookups over and over again is how the issue
was discovered...
Fix the issue by incrementing xactCompletionCount in
XidCacheRemoveRunningXids. It already acquires ProcArrayLock exclusively,
making that an easy proposition.
Add a test to ensure that kill_prior_tuple prevents index growth when it
involves aborted subtransaction of the current transaction.
Author: Andres Freund
Discussion: https://postgr.es/m/20210406043521.lopeo7bbigad3n6t@alap3.anarazel.de
Discussion: https://postgr.es/m/20210317055718.v6qs3ltzrformqoa%40alap3.anarazel.de
M src/backend/storage/ipc/procarray.c
A src/test/regress/expected/mvcc.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/mvcc.sql
Remove tupgone special case from vacuumlazy.c.
commit : 8523492d4e349c4714aa2ab0291be175a88cb4fc
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 6 Apr 2021 08:49:22 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 6 Apr 2021 08:49:22 -0700
Retry the call to heap_prune_page() in rare cases where there is
disagreement between the heap_prune_page() call and the call to
HeapTupleSatisfiesVacuum() that immediately follows. Disagreement is
possible when a concurrently-aborted transaction makes a tuple DEAD
during the tiny window between each step. This was the only case where
a tuple considered DEAD by VACUUM still had storage following pruning.
VACUUM's definition of dead tuples is now uniformly simple and
unambiguous: dead tuples from each page are always LP_DEAD line pointers
that were encountered just after we performed pruning (and just before
we considered freezing remaining items with tuple storage).
Eliminating the tupgone=true special case enables INDEX_CLEANUP=off
style skipping of index vacuuming that takes place based on flexible,
dynamic criteria. The INDEX_CLEANUP=off case had to know about skipping
indexes up-front before now, due to a subtle interaction with the
special case (see commit dd695979) -- this was a special case unto
itself. Now there are no special cases. And so now it won't matter
when or how we decide to skip index vacuuming: it won't affect how
pruning behaves, and it won't be affected by any of the implementation
details of pruning or freezing.
Also remove XLOG_HEAP2_CLEANUP_INFO records. These are no longer
necessary because we now rely entirely on heap pruning taking care of
recovery conflicts. There is no longer any need to generate recovery
conflicts for DEAD tuples that pruning just missed. This also means
that heap vacuuming now uses exactly the same strategy for recovery
conflicts as index vacuuming always has: REDO routines never need to
process a latestRemovedXid from the WAL record, since earlier REDO of
the WAL record from pruning is sufficient in all cases. The generic
XLOG_HEAP2_CLEAN record type is now split into two new record types to
reflect this new division (these are called XLOG_HEAP2_PRUNE and
XLOG_HEAP2_VACUUM).
Also stop acquiring a super-exclusive lock for heap pages when they're
vacuumed during VACUUM's second heap pass. A regular exclusive lock is
enough. This is correct because heap page vacuuming is now strictly a
matter of setting the LP_DEAD line pointers to LP_UNUSED. No other
backend can have a pointer to a tuple located in a pinned buffer that
can be invalidated by a concurrent heap page vacuum operation.
Heap vacuuming can now be thought of as conceptually similar to index
vacuuming and conceptually dissimilar to heap pruning. Heap pruning now
has sole responsibility for anything involving the logical contents of
the database (e.g., managing transaction status information, recovery
conflicts, considering what to do with HOT chains). Index vacuuming and
heap vacuuming are now only concerned with recycling garbage items from
physical data structures that back the logical database.
Bump XLOG_PAGE_MAGIC due to pruning and heap page vacuum WAL record
changes.
Credit for the idea of retrying pruning a page to avoid the tupgone case
goes to Andres Freund.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Andres Freund <andres@anarazel.de>
Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAH2-WznneCXTzuFmcwx_EyRQgfsfJAAsu+CsqRFmFXCAar=nJw@mail.gmail.com
M src/backend/access/gist/gistxlog.c
M src/backend/access/hash/hash_xlog.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/rmgrdesc/heapdesc.c
M src/backend/replication/logical/decode.c
M src/include/access/heapam.h
M src/include/access/heapam_xlog.h
M src/include/access/xlog_internal.h
M src/tools/pgindent/typedefs.list
Fix missing #include in nodeResultCache.h.
commit : 789d81de8a50d9a23cc1a3b8ea5d839246020689
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Apr 2021 11:23:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Apr 2021 11:23:48 -0400
Per cpluspluscheck.
M src/backend/executor/nodeResultCache.c
M src/include/executor/nodeResultCache.h
psql: Show all query results by default
commit : 3a5130672296ed4e682403a77a9a3ad3d21cef75
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 6 Apr 2021 16:58:10 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 6 Apr 2021 16:58:10 +0200
Previously, psql printed only the last result if a command string
returned multiple result sets. Now it prints all of them. The
previous behavior can be obtained by setting the psql variable
SHOW_ALL_RESULTS to off.
Author: Fabien COELHO <coelho@cri.ensmp.fr>
Reviewed-by: "Iwata, Aya" <iwata.aya@jp.fujitsu.com>
Reviewed-by: Daniel Verite <daniel@manitou-mail.org>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/alpine.DEB.2.21.1904132231510.8961@lancre
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/common.c
M src/bin/psql/help.c
M src/bin/psql/settings.h
M src/bin/psql/startup.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/copyselect.out
M src/test/regress/expected/psql.out
M src/test/regress/expected/transactions.out
M src/test/regress/sql/copyselect.sql
M src/test/regress/sql/psql.sql
M src/test/regress/sql/transactions.sql
Fix handling of clauses incompatible with extended statistics
commit : 518442c7f334f3b05ea28b7ef50f1b551cfcc23e
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 6 Apr 2021 16:12:37 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 6 Apr 2021 16:12:37 +0200
Handling of incompatible clauses while applying extended statistics was
a bit confused - while handling a mix of compatible and incompatible
clauses it sometimes incorrectly treated the incompatible clauses as
compatible, resulting in a crash.
Fixed by reworking the code applying the selected statistics object to
make it easier to understand, and adding a proper compatibility check.
Reported-by: David Rowley
Discussion: https://postgr.es/m/CAApHDvpYT10-nkSp8xXe-nbO3jmoaRyRFHbzh-RWMfAJynqgpQ%40mail.gmail.com
M src/backend/statistics/extended_stats.c
M src/backend/statistics/mcv.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Refactor lazy_scan_heap() loop.
commit : 7ab96cf6b312cfcd79cdc1a69c6bdb75de0ed30f
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 6 Apr 2021 07:49:39 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 6 Apr 2021 07:49:39 -0700
Add a lazy_scan_heap() subsidiary function that handles heap pruning and
tuple freezing: lazy_scan_prune(). This is a great deal cleaner. The
code that remains in lazy_scan_heap()'s per-block loop can now be
thought of as code that either comes before or after the call to
lazy_scan_prune(), which is now the clear focal point. This division is
enforced by the way in which we now manage state. lazy_scan_prune()
outputs state (using its own struct) that describes what to do with the
page following pruning and freezing (e.g., visibility map maintenance,
recording free space in the FSM). It doesn't get passed any special
instructional state from the preamble code, though.
Also cleanly separate the logic used by a VACUUM with INDEX_CLEANUP=off
from the logic used by single-heap-pass VACUUMs. The former case is now
structured as the omission of index and heap vacuuming by a two pass
VACUUM. The latter case goes back to being used only when the table
happens to have no indexes (just as it was before commit a96c41fe).
This structure is much more natural, since the whole point of
INDEX_CLEANUP=off is to skip the index and heap vacuuming that would
otherwise take place. The single-heap-pass case doesn't skip any useful
work, though -- it just does heap pruning and heap vacuuming together
when the table happens to have no indexes.
Both of these changes are preparation for an upcoming patch that
generalizes the mechanism used by INDEX_CLEANUP=off. The later patch
will allow VACUUM to give up on index and heap vacuuming dynamically, as
problems emerge (e.g., with wraparound), so that an affected VACUUM
operation can finish up as soon as possible.
Also fix a very old bug in single-pass VACUUM VERBOSE output. We were
reporting the number of tuples deleted via pruning as a direct
substitute for reporting the number of LP_DEAD items removed in a
function that deals with the second pass over the heap. But that
doesn't work at all -- they're two different things.
To fix, start tracking the total number of LP_DEAD items encountered
during pruning, and use that in the report instead. A single pass
VACUUM will always vacuum away whatever LP_DEAD items a heap page has
immediately after it is pruned, so the total number of LP_DEAD items
encountered during pruning equals the total number vacuumed-away.
(They are _not_ equal in the INDEX_CLEANUP=off case, but that's okay
because skipping index vacuuming is now a totally orthogonal concept to
one-pass VACUUM.)
Also stop reporting the count of LP_UNUSED items in VACUUM VERBOSE
output. This makes the output of VACUUM VERBOSE more consistent with
log_autovacuum's output (because it never showed information about
LP_UNUSED items). VACUUM VERBOSE reported LP_UNUSED items left behind
by the last VACUUM, and LP_UNUSED items created via pruning HOT chains
during the current VACUUM (it never included LP_UNUSED items left behind
by the current VACUUM's second pass over the heap). This makes it
useless as an indicator of line pointer bloat, which must have been the
original intention. (Like the first VACUUM VERBOSE issue, this issue was
arguably an oversight in commit 282d2a03, which added the heap-only
tuple optimization.)
Finally, stop reporting empty_pages in VACUUM VERBOSE output, and start
reporting pages_removed instead. This also makes the output of VACUUM
VERBOSE more consistent with log_autovacuum's output (which does not
show empty_pages, but does show pages_removed). An empty page isn't
meaningfully different to a page that is almost empty, or a page that is
empty but for only a small number of remaining LP_UNUSED items.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Robert Haas <robertmhaas@gmail.com>
Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAH2-WznneCXTzuFmcwx_EyRQgfsfJAAsu+CsqRFmFXCAar=nJw@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Clean up treatment of missing default and CHECK-constraint records.
commit : 091e22b2e673e3e8480abd68fbb827c5d6979615
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Apr 2021 10:34:37 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Apr 2021 10:34:37 -0400
Andrew Gierth reported that it's possible to crash the backend if no
pg_attrdef record is found to match an attribute that has atthasdef set.
AttrDefaultFetch warns about this situation, but then leaves behind
a relation tupdesc that has null "adbin" pointer(s), which most places
don't guard against.
We considered promoting the warning to an error, but throwing errors
during relcache load is pretty drastic: it effectively locks one out
of using the relation at all. What seems better is to leave the
load-time behavior as a warning, but then throw an error in any code
path that wants to use a default and can't find it. This confines
the error to a subset of INSERT/UPDATE operations on the table, and
in particular will at least allow a pg_dump to succeed.
Also, we should fix AttrDefaultFetch to not leave any null pointers
in the tupdesc, because that just creates an untested bug hazard.
While at it, apply the same philosophy of "warn at load, throw error
only upon use of the known-missing info" to CHECK constraints.
CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch,
but for reasons lost in the mists of time, it was throwing ERROR for
the same cases that AttrDefaultFetch treats as WARNING. Make the two
functions more nearly alike.
In passing, get rid of potentially-O(N^2) loops in equalTupleDesc
by making AttrDefaultFetch sort the entries after fetching them,
so that equalTupleDesc can assume that entries in two equal tupdescs
must be in matching order. (CheckConstraintFetch already was sorting
CHECK constraints, but equalTupleDesc hadn't been told about it.)
There's some argument for back-patching this, but with such a small
number of field reports, I'm content to fix it in HEAD.
Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
M src/backend/access/common/tupdesc.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execMain.c
M src/backend/parser/parse_utilcmd.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/utils/cache/relcache.c
Stop archive recovery if WAL generated with wal_level=minimal is found.
commit : 9de9294b0c4dac77edb80f029648afca79d14653
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 6 Apr 2021 22:56:51 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 6 Apr 2021 22:56:51 +0900
Previously if hot standby was enabled, archive recovery exited with
an error when it found WAL generated with wal_level=minimal.
But if hot standby was disabled, it just reported a warning and
continued in that case. Which could lead to data loss or errors
during normal operation. A warning was emitted, but users could
easily miss that and not notice this serious situation until
they encountered the actual errors.
To improve this situation, this commit changes archive recovery
so that it exits with FATAL error when it finds WAL generated with
wal_level=minimal whatever the setting of hot standby. This enables
users to notice the serious situation soon.
The FATAL error is thrown if archive recovery starts from a base
backup taken before wal_level is changed to minimal. When archive
recovery exits with the error, if users have a base backup taken
after setting wal_level to higher than minimal, they can recover
the database by starting archive recovery from that newer backup.
But note that if such backup doesn't exist, there is no easy way to
complete archive recovery, which may make the database server
unstartable and users may lose whole database. The commit adds
the note about this risk into the document.
Even in the case of unstartable database server, previously by just
disabling hot standby users could avoid the error during archive
recovery, forcibly start up the server and salvage data from it.
But note that this commit makes this procedure unavailable at all.
Author: Takamichi Osumi
Reviewed-by: Laurenz Albe, Kyotaro Horiguchi, David Steele, Fujii Masao
Discussion: https://postgr.es/m/OSBPR01MB4888CBE1DA08818FD2D90ED8EDF90@OSBPR01MB4888.jpnprd01.prod.outlook.com
M doc/src/sgml/config.sgml
M doc/src/sgml/perform.sgml
M src/backend/access/transam/xlog.c
A src/test/recovery/t/024_archive_recovery.pl
Mark test_enc_conversion() as STRICT.
commit : c4c393b3ec83ceb4b4d7f37cdd5302126377d069
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 6 Apr 2021 14:53:56 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 6 Apr 2021 14:53:56 +0300
Reported-by: Jaime Casanova, using SQLsmith
Discussion: https://www.postgresql.org/message-id/20210402235337.GA4082@ahch-to
M src/test/regress/input/create_function_1.source
M src/test/regress/output/create_function_1.source
pgbench: Function to generate random permutations.
commit : 6b258e3d688db14aadb58dde2a72939362310684
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 6 Apr 2021 11:50:42 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 6 Apr 2021 11:50:42 +0100
This adds a new function, permute(), that generates pseudorandom
permutations of arbitrary sizes. This can be used to randomly shuffle
a set of values to remove unwanted correlations. For example,
permuting the output from a non-uniform random distribution so that
all the most common values aren't collocated, allowing more realistic
tests to be performed.
Formerly, hash() was recommended for this purpose, but that suffers
from collisions that might alter the distribution, so recommend
permute() for this purpose instead.
Fabien Coelho and Hironobu Suzuki, with additional hacking be me.
Reviewed by Thomas Munro, Alvaro Herrera and Muhammad Usama.
Discussion: https://postgr.es/m/alpine.DEB.2.21.1807280944370.5142@lancre
M doc/src/sgml/ref/pgbench.sgml
M src/bin/pgbench/exprparse.y
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/pgbench.h
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/pgbench/t/002_pgbench_no_server.pl
Adjust input value to WaitEventSetWait() in ExecAppendAsyncEventWait().
commit : a8af856d3257138590788e40eb84049def147acf
author : Etsuro Fujita <efujita@postgresql.org>
date : Tue, 6 Apr 2021 19:15:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Tue, 6 Apr 2021 19:15:00 +0900
Adjust the number of events given to WaitEventSetWait() so that it
doesn't exceed the maximum number of events in the WaitEventSet given
to that function (set->nevents_space) in hopes of making the buildfarm
green.
Per valgrind failure report from Tom Lane and the buildfarm.
Author: Etsuro Fujita
Discussion: https://postgr.es/m/3411577.1617289776%40sss.pgh.pa.us
M src/backend/executor/nodeAppend.c
ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION
commit : 82ed7748b710e3ddce3f7ebc74af80fe4869492f
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 6 Apr 2021 10:44:26 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 6 Apr 2021 10:44:26 +0200
At present, if we want to update publications in a subscription, we
can use SET PUBLICATION. However, it requires supplying all
publications that exists and the new publications. If we want to add
new publications, it's inconvenient. The new syntax only supplies the
new publications. When the refresh is true, it only refreshes the new
publications.
Author: Japin Li <japinli@hotmail.com>
Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/MEYP282MB166939D0D6C480B7FBE7EFFBB6BC0@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
M doc/src/sgml/ref/alter_subscription.sgml
M src/backend/commands/subscriptioncmds.c
M src/backend/parser/gram.y
M src/bin/psql/tab-complete.c
M src/include/nodes/parsenodes.h
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
Fix the tests added by commit ac4645c015.
commit : 266b5673b4b6bed2e9ebfe73ca967f44d6dc0e6c
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 6 Apr 2021 14:58:52 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 6 Apr 2021 14:58:52 +0530
In the tests, after disabling the subscription, we were not waiting for
the replication connection to drop from the publisher. So when the test
was trying to use the same slot to fetch the messages via SQL API, it
sometimes gives an error that the replication slot is active for other
PID.
Per buildfarm.
M src/test/subscription/t/020_messages.pl
Fix compiler warning in fe-trace.c for MSVC
commit : 9bc9b4609a246ded5caf3f3d4c0013a002ba2323
author : David Rowley <drowley@postgresql.org>
date : Tue, 6 Apr 2021 18:33:40 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 6 Apr 2021 18:33:40 +1200
It seems that in MSVC timeval's tv_sec field is of type long.
localtime() takes a time_t pointer. Since long is 32-bit even on 64-bit
builds in MSVC, passing a long pointer instead of the correct time_t
pointer generated a compiler warning. Fix that.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/CAApHDvoRG25X_=ZCGSPb4KN_j2iu=G2uXsRSg8NBZeuhkOSETg@mail.gmail.com
M src/interfaces/libpq/fe-trace.c
Change return type of EXTRACT to numeric
commit : a2da77cdb4661826482ebf2ddba1f953bc74afe4
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 6 Apr 2021 07:17:13 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 6 Apr 2021 07:17:13 +0200
The previous implementation of EXTRACT mapped internally to
date_part(), which returned type double precision (since it was
implemented long before the numeric type existed). This can lead to
imprecise output in some cases, so returning numeric would be
preferrable. Changing the return type of an existing function is a
bit risky, so instead we do the following: We implement a new set of
functions, which are now called "extract", in parallel to the existing
date_part functions. They work the same way internally but use
numeric instead of float8. The EXTRACT construct is now mapped by the
parser to these new extract functions. That way, dumps of views
etc. from old versions (which would use date_part) continue to work
unchanged, but new uses will map to the new extract functions.
Additionally, the reverse compilation of EXTRACT now reproduces the
original syntax, using the new mechanism introduced in
40c24bfef92530bd846e111c1742c2a54441c62c.
The following minor changes of behavior result from the new
implementation:
- The column name from an isolated EXTRACT call is now "extract"
instead of "date_part".
- Extract from date now rejects inappropriate field names such as
HOUR. It was previously mapped internally to extract from
timestamp, so it would silently accept everything appropriate for
timestamp.
- Return values when extracting fields with possibly fractional
values, such as second and epoch, now have the full scale that the
value has internally (so, for example, '1.000000' instead of just
'1').
Reported-by: Petr Fedorov <petr.fedorov@phystech.edu>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/42b73d2d-da12-ba9f-570a-420e0cce19d9@phystech.edu
M doc/src/sgml/func.sgml
M src/backend/parser/gram.y
M src/backend/utils/adt/date.c
M src/backend/utils/adt/numeric.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/adt/timestamp.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/utils/numeric.h
M src/test/regress/expected/create_view.out
M src/test/regress/expected/date.out
M src/test/regress/expected/interval.out
M src/test/regress/expected/psql_crosstab.out
M src/test/regress/expected/time.out
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/expected/timetz.out
M src/test/regress/sql/date.sql
M src/test/regress/sql/interval.sql
M src/test/regress/sql/time.sql
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
M src/test/regress/sql/timetz.sql
Fix typo in pgstat.c.
commit : f5d94e405e17a49487672316610630be2f9d0bb7
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 6 Apr 2021 14:09:40 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 6 Apr 2021 14:09:40 +0900
Introduced by 9868167500.
Author: Vignesh C
Discussion: https://postgr.es/m/CALDaNm1DqgaLBAJrtGznKk1sR1mH-augmp7LfGvxWwTUhah+rg@mail.gmail.com
M src/backend/postmaster/pgstat.c
Add function to log the memory contexts of specified backend process.
commit : 43620e328617c1f41a2a54c8cee01723064e3ffa
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 6 Apr 2021 13:44:15 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 6 Apr 2021 13:44:15 +0900
Commit 3e98c0bafb added pg_backend_memory_contexts view to display
the memory contexts of the backend process. However its target process
is limited to the backend that is accessing to the view. So this is
not so convenient when investigating the local memory bloat of other
backend process. To improve this situation, this commit adds
pg_log_backend_memory_contexts() function that requests to log
the memory contexts of the specified backend process.
This information can be also collected by calling
MemoryContextStats(TopMemoryContext) via a debugger. But
this technique cannot be used in some environments because no debugger
is available there. So, pg_log_backend_memory_contexts() allows us to
see the memory contexts of specified backend more easily.
Only superusers are allowed to request to log the memory contexts
because allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its memory contexts at LOG_SERVER_ONLY level,
so that these memory contexts will appear in the server log but not
be sent to the client. It logs one message per memory context.
Because if it buffers all memory contexts into StringInfo to log them
as one message, which may require the buffer to be enlarged very much
and lead to OOM error since there can be a large number of memory
contexts in a backend.
When a backend process is consuming huge memory, logging all its
memory contexts might overrun available disk space. To prevent this,
now this patch limits the number of child contexts to log per parent
to 100. As with MemoryContextStats(), it supposes that practical cases
where the log gets long will typically be huge numbers of siblings
under the same parent context; while the additional debugging value
from seeing details about individual siblings beyond 100 will not be large.
There was another proposed patch to add the function to return
the memory contexts of specified backend as the result sets,
instead of logging them, in the discussion. However that patch is
not included in this commit because it had several issues to address.
Thanks to Tatsuhito Kasahara, Andres Freund, Tom Lane, Tomas Vondra,
Michael Paquier, Kyotaro Horiguchi and Zhihong Yu for the discussion.
Bump catalog version.
Author: Atsushi Torikoshi
Reviewed-by: Kyotaro Horiguchi, Zhihong Yu, Fujii Masao
Discussion: https://postgr.es/m/0271f440ac77f2a4180e0e56ebd944d1@oss.nttdata.com
M doc/src/sgml/func.sgml
M src/backend/storage/ipc/procsignal.c
M src/backend/tcop/postgres.c
M src/backend/utils/adt/mcxtfuncs.c
M src/backend/utils/init/globals.c
M src/backend/utils/mmgr/aset.c
M src/backend/utils/mmgr/generation.c
M src/backend/utils/mmgr/mcxt.c
M src/backend/utils/mmgr/slab.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/miscadmin.h
M src/include/nodes/memnodes.h
M src/include/storage/procsignal.h
M src/include/utils/memutils.h
M src/test/regress/expected/misc_functions.out
M src/test/regress/sql/misc_functions.sql
Fix some issues with SSL and Kerberos tests
commit : 5a71964a832febfee23cedc3bb354049d6ca78a7
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 6 Apr 2021 13:23:57 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 6 Apr 2021 13:23:57 +0900
The recent refactoring done in c50624c accidentally broke a portion of
the kerberos tests checking after a query, so add its functionality
back. Some inactive SSL tests had their arguments in an incorrect
order, which would cause them to fail if they were to run.
Author: Jacob Champion
Discussion: https://postgr.es/m/4f5b0b3dc0b6fe9ae6a34886b4d4000f61eb567e.camel@vmware.com
M src/test/kerberos/t/001_auth.pl
M src/test/ssl/t/001_ssltests.pl
Allow pgoutput to send logical decoding messages.
commit : ac4645c0157fc5fcef0af8ff571512aa284a2cec
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 6 Apr 2021 08:40:47 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 6 Apr 2021 08:40:47 +0530
The output plugin accepts a new parameter (messages) that controls if
logical decoding messages are written into the replication stream. It is
useful for those clients that use pgoutput as an output plugin and needs
to process messages that were written by pg_logical_emit_message().
Although logical streaming replication protocol supports logical
decoding messages now, logical replication does not use this feature yet.
Author: David Pirotte, Euler Taveira
Reviewed-by: Euler Taveira, Andres Freund, Ashutosh Bapat, Amit Kapila
Discussion: https://postgr.es/m/CADK3HHJ-+9SO7KuRLH=9Wa1rAo60Yreq1GFNkH_kd0=CdaWM+A@mail.gmail.com
M doc/src/sgml/protocol.sgml
M src/backend/replication/logical/proto.c
M src/backend/replication/logical/worker.c
M src/backend/replication/pgoutput/pgoutput.c
M src/include/replication/logicalproto.h
M src/include/replication/pgoutput.h
A src/test/subscription/t/020_messages.pl
Refactor function parse_output_parameters.
commit : 531737ddad214cb8a675953208e2f3a6b1be122b
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 6 Apr 2021 08:26:31 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 6 Apr 2021 08:26:31 +0530
Instead of using multiple parameters in parse_ouput_parameters function
signature, use the struct PGOutputData that encapsulates all pgoutput
options. It will be useful for future work where we need to add other
options in pgoutput.
Author: Euler Taveira
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CADK3HHJ-+9SO7KuRLH=9Wa1rAo60Yreq1GFNkH_kd0=CdaWM+A@mail.gmail.com
M src/backend/replication/pgoutput/pgoutput.c
M src/include/replication/pgoutput.h
Change PostgresNode::connect_fails() to never send down queries
commit : 6d41dd045ada28ee14182112fc4cf50fb3879d28
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 6 Apr 2021 09:53:06 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 6 Apr 2021 09:53:06 +0900
This type of failure is similar to what has been fixed in c757a3da,
where an authentication failure combined with psql pushing a command
down its communication pipe causes a test failure. This routine is
designed to fail, so sending a query has little sense anyway.
Per buildfarm members gaur and hoverfly, based on an analysis and fix
from Tom Lane.
Discussion: https://postgr.es/m/513200.1617634642@sss.pgh.pa.us
M src/test/perl/PostgresNode.pm
Allocate access strategy in parallel VACUUM workers.
commit : f6b8f19a084ce949522fcbc940dc116c034cfc47
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 5 Apr 2021 17:17:40 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 5 Apr 2021 17:17:40 -0700
Commit 49f49def took entirely the wrong approach to fixing this issue.
Just allocate a local buffer access strategy in each individual worker
instead of trying to propagate state. This state was never propagated
by parallel VACUUM in the first place.
It looks like the only reason that this worked following commit 40d964ec
was that it involved static global variables, which are initialized to 0
per the C standard.
A more comprehensive fix may be necessary, even on HEAD. This fix
should at least get the buildfarm green once again.
Thanks once again to Thomas Munro for continued off-list assistance with
the issue.
M src/backend/access/heap/vacuumlazy.c
Support INCLUDE'd columns in SP-GiST.
commit : 09c1c6ab4bc5764dd69c53ccfd43b2060b1fd090
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Apr 2021 18:41:09 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Apr 2021 18:41:09 -0400
Not much to say here: does what it says on the tin.
We steal a previously-always-zero bit from the nextOffset
field of leaf index tuples in order to track whether there
is a nulls bitmap. Otherwise it works about like included
columns in other index types.
Pavel Borisov, reviewed by Andrey Borodin and Anastasia Lubennikova,
and rather heavily editorialized on by me
Discussion: https://postgr.es/m/CALT9ZEFi-vMp4faht9f9Junb1nO3NOSjhpxTmbm1UGLMsLqiEQ@mail.gmail.com
M doc/src/sgml/indices.sgml
M doc/src/sgml/ref/create_index.sgml
M doc/src/sgml/spgist.sgml
M doc/src/sgml/xindex.sgml
M src/backend/access/common/indextuple.c
M src/backend/access/spgist/README
M src/backend/access/spgist/spgdoinsert.c
M src/backend/access/spgist/spginsert.c
M src/backend/access/spgist/spgscan.c
M src/backend/access/spgist/spgutils.c
M src/backend/access/spgist/spgvacuum.c
M src/backend/access/spgist/spgxlog.c
M src/include/access/itup.h
M src/include/access/spgist_private.h
M src/test/modules/spgist_name_ops/expected/spgist_name_ops.out
M src/test/modules/spgist_name_ops/sql/spgist_name_ops.sql
M src/test/regress/expected/amutils.out
M src/test/regress/expected/create_index_spgist.out
M src/test/regress/expected/index_including.out
M src/test/regress/sql/create_index_spgist.sql
M src/test/regress/sql/index_including.sql
Propagate parallel VACUUM's buffer access strategy.
commit : 49f49defe7c0a330cca084de5da14ccdfdafc6a3
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 5 Apr 2021 14:56:56 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 5 Apr 2021 14:56:56 -0700
Parallel VACUUM relied on global variable state from the leader process
being propagated to workers on fork(). Commit b4af70cb removed most
uses of global variables inside vacuumlazy.c, but did not account for
the buffer access strategy state.
To fix, propagate the state through shared memory instead.
Per buildfarm failures on elver, curculio, and morepork.
Many thanks to Thomas Munro for off-list assistance with this issue.
M src/backend/access/heap/vacuumlazy.c
Simplify state managed by VACUUM.
commit : b4af70cb210393c9c8f41643acf6b213e21178e7
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 5 Apr 2021 13:21:44 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 5 Apr 2021 13:21:44 -0700
Reorganize the state struct used by VACUUM -- group related items
together to make it easier to understand. Also stop relying on stack
variables inside lazy_scan_heap() -- move those into the state struct
instead. Doing things this way simplifies large groups of related
functions whose function signatures had a lot of unnecessary redundancy.
Switch over to using int64 for the struct fields used to count things
that are reported to the user via log_autovacuum and VACUUM VERBOSE
output. We were using double, but that doesn't seem to have any
advantages. Using int64 makes it possible to add assertions that verify
that the first pass over the heap (pruning) encounters precisely the
same number of LP_DEAD items that get deleted from indexes later on, in
the second pass over the heap. These assertions will be added in later
commits.
Finally, adjust the signatures of functions with IndexBulkDeleteResult
pointer arguments in cases where there was ambiguity about whether or
not the argument relates to a single index or all indexes. Functions
now use the idiom that both ambulkdelete() and amvacuumcleanup() have
always used (where appropriate): accept a mutable IndexBulkDeleteResult
pointer argument, and return a result IndexBulkDeleteResult pointer to
caller.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-By: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/CAH2-WzkeOSYwC6KNckbhk2b1aNnWum6Yyn0NKP9D-Hq1LGTDPw@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/index/indexam.c
M src/backend/commands/vacuum.c
M src/include/access/genam.h
M src/include/access/heapam.h
M src/include/access/tableam.h
Add pg_read_all_data and pg_write_all_data roles
commit : 6c3ffd697e2242f5497ea4b40fffc8f6f922ff60
author : Stephen Frost <sfrost@snowman.net>
date : Mon, 5 Apr 2021 13:42:52 -0400
committer: Stephen Frost <sfrost@snowman.net>
date : Mon, 5 Apr 2021 13:42:52 -0400
A commonly requested use-case is to have a role who can run an
unfettered pg_dump without having to explicitly GRANT that user access
to all tables, schemas, et al, without that role being a superuser.
This address that by adding a "pg_read_all_data" role which implicitly
gives any member of this role SELECT rights on all tables, views and
sequences, and USAGE rights on all schemas.
As there may be cases where it's also useful to have a role who has
write access to all objects, pg_write_all_data is also introduced and
gives users implicit INSERT, UPDATE and DELETE rights on all tables,
views and sequences.
These roles can not be logged into directly but instead should be
GRANT'd to a role which is able to log in. As noted in the
documentation, if RLS is being used then an administrator may (or may
not) wish to set BYPASSRLS on the login role which these predefined
roles are GRANT'd to.
Reviewed-by: Georgios Kokolatos
Discussion: https://postgr.es/m/20200828003023.GU29590@tamriel.snowman.net
M doc/src/sgml/user-manag.sgml
M src/backend/catalog/aclchk.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_authid.dat
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Shut down transaction tracking at startup process exit.
commit : ad8b674922eb70dc5cd02951dd82fe2c4c37c80a
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 6 Apr 2021 02:25:37 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 6 Apr 2021 02:25:37 +0900
Maxim Orlov reported that the shutdown of standby server could result in
the following assertion failure. The cause of this issue was that,
when the shutdown caused the startup process to exit, recovery-time
transaction tracking was not shut down even if it's already initialized,
and some locks the tracked transactions were holding could not be released.
At this situation, if other process was invoked and the PGPROC entry that
the startup process used was assigned to it, it found such unreleased locks
and caused the assertion failure, during the initialization of it.
TRAP: FailedAssertion("SHMQueueEmpty(&(MyProc->myProcLocks[i]))"
This commit fixes this issue by making the startup process shut down
transaction tracking and release all locks, at the exit of it.
Back-patch to all supported branches.
Reported-by: Maxim Orlov
Author: Fujii Masao
Reviewed-by: Maxim Orlov
Discussion: https://postgr.es/m/ad4ce692cc1d89a093b471ab1d969b0b@postgrespro.ru
M src/backend/postmaster/startup.c
M src/backend/storage/ipc/standby.c
Align some terms in arch-dev.sgml to glossary
commit : 6734e806958c4ebd253adb30b255983981818710
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 5 Apr 2021 11:45:17 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 5 Apr 2021 11:45:17 -0400
This mostly adds links to the glossary to the existing text, instead of
using <firstterm>. Heikki left this out of 29ad6595ef7f out of
stylistic concerns; these have since been addressed.
Author: Jürgen Purtz <juergen@purtz.de>
Discussion: https://postgr.es/m/67d7240f-8596-83fc-5e15-af06c128a0f5@purtz.de
M doc/src/sgml/arch-dev.sgml
Renumber cursor option flags
commit : a63dd8afe2b859b853d857cd8a0392ca1e04ab6c
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 5 Apr 2021 09:10:27 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 5 Apr 2021 09:10:27 +0200
Move the planner-control flags up so that there is more room for parse
options. Some pending patches need some room there, so do this
renumbering separately so that there is less potential for conflicts.
M src/include/nodes/parsenodes.h
Fix typo in collationcmds.c
commit : 9f6f1f9b8e61f9ce47e1936fc68c21a4a8d6722c
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 5 Apr 2021 11:18:12 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 5 Apr 2021 11:18:12 +0900
Introduced by 51e225d.
Author: Anton Voloshin
Discussion: https://postgr.es/m/05477da0-703c-7de7-998c-5879738e8f39@postgrespro.ru
M src/backend/commands/collationcmds.c
Refactor all TAP test suites doing connection checks
commit : c50624cdd248c13b4ba199f95e24c88d2cc8a097
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 5 Apr 2021 10:13:57 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 5 Apr 2021 10:13:57 +0900
This commit refactors more TAP tests to adapt with the recent
introduction of connect_ok() and connect_fails() in PostgresNode,
introduced by 0d1a3343. This changes the following test suites to use
the same code paths for connection checks:
- Kerberos
- LDAP
- SSL
- Authentication
Those routines are extended to be able to handle optional parameters
that are set depending on each suite's needs, as of:
- custom SQL query.
- expected stderr matching pattern.
- expected stdout matching pattern.
The new design is extensible with more parameters, and there are some
plans for those routines in the future with checks based on the contents
of the backend logs.
Author: Jacob Champion, Michael Paquier
Discussion: https://postgr.es/m/d17b919e27474abfa55d97786cb9cfadfe2b59e9.camel@vmware.com
M src/test/authentication/t/001_password.pl
M src/test/authentication/t/002_saslprep.pl
M src/test/kerberos/t/001_auth.pl
M src/test/ldap/t/001_auth.pl
M src/test/perl/PostgresNode.pm
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
Fix more confusion in SP-GiST.
commit : dfc843d465689d2c2af8b0e01c66c51ccaae2343
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 4 Apr 2021 17:57:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 4 Apr 2021 17:57:07 -0400
spg_box_quad_leaf_consistent unconditionally returned the leaf
datum as leafValue, even though in its usage for poly_ops that
value is of completely the wrong type.
In versions before 12, that was harmless because the core code did
nothing with leafValue in non-index-only scans ... but since commit
2a6368343, if we were doing a KNN-style scan, spgNewHeapItem would
unconditionally try to copy the value using the wrong datatype
parameters. Said copying is a waste of time and space if we're not
going to return the data, but it accidentally failed to fail until
I fixed the datatype confusion in ac9099fc1.
Hence, change spgNewHeapItem to not copy the datum unless we're
actually going to return it later. This saves cycles and dodges
the question of whether lossy opclasses are returning the right
type. Also change spg_box_quad_leaf_consistent to not return
data that might be of the wrong type, as insurance against
somebody introducing a similar bug into the core code in future.
It seems like a good idea to back-patch these two changes into
v12 and v13, although I'm afraid to change spgNewHeapItem's
mistaken idea of which datatype to use in those branches.
Per buildfarm results from ac9099fc1.
Discussion: https://postgr.es/m/3728741.1617381471@sss.pgh.pa.us
M src/backend/access/spgist/spgscan.c
M src/backend/utils/adt/geo_spgist.c
Fix confusion in SP-GiST between attribute type and leaf storage type.
commit : ac9099fc1dd460bffaafec19272159dd7bc86f5b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 4 Apr 2021 14:28:35 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 4 Apr 2021 14:28:35 -0400
According to the documentation, the attType passed to the opclass
config function (and also relied on by the core code) is the type
of the heap column or expression being indexed. But what was
actually being passed was the type stored for the index column.
This made no difference for user-defined SP-GiST opclasses,
because we weren't allowing the STORAGE clause of CREATE OPCLASS
to be used, so the two types would be the same. But it's silly
not to allow that, seeing that the built-in poly_ops opclass
has a different value for opckeytype than opcintype, and that if you
want to do lossy storage then the types must really be different.
(Thus, user-defined opclasses doing lossy storage had to lie about
what type is in the index.) Hence, remove the restriction, and make
sure that we use the input column type not opckeytype where relevant.
For reasons of backwards compatibility with existing user-defined
opclasses, we can't quite insist that the specified leafType match
the STORAGE clause; instead just add an amvalidate() warning if
they don't match.
Also fix some bugs that would only manifest when trying to return
index entries when attType is different from attLeafType. It's not
too surprising that these have not been reported, because the only
usual reason for such a difference is to store the leaf value
lossily, rendering index-only scans impossible.
Add a src/test/modules module to exercise cases where attType is
different from attLeafType and yet index-only scan is supported.
Discussion: https://postgr.es/m/3728741.1617381471@sss.pgh.pa.us
M doc/src/sgml/ref/create_opclass.sgml
M doc/src/sgml/spgist.sgml
M src/backend/access/spgist/spgscan.c
M src/backend/access/spgist/spgutils.c
M src/backend/access/spgist/spgvalidate.c
M src/include/access/spgist_private.h
M src/test/modules/Makefile
A src/test/modules/spgist_name_ops/.gitignore
A src/test/modules/spgist_name_ops/Makefile
A src/test/modules/spgist_name_ops/README
A src/test/modules/spgist_name_ops/expected/spgist_name_ops.out
A src/test/modules/spgist_name_ops/spgist_name_ops–1.0.sql
A src/test/modules/spgist_name_ops/spgist_name_ops.c
A src/test/modules/spgist_name_ops/spgist_name_ops.control
A src/test/modules/spgist_name_ops/sql/spgist_name_ops.sql
M src/test/regress/expected/rangetypes.out
M src/test/regress/sql/rangetypes.sql
Fix bug in brin_minmax_multi_union
commit : d9c5b9a9eeb9e3061ae139e0e564ce5358c94001
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 4 Apr 2021 19:36:10 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 4 Apr 2021 19:36:10 +0200
When calling sort_expanded_ranges() we need to remember the return
value, because the function sorts and also deduplicates the ranges. So
the number of ranges may decrease. brin_minmax_multi_union failed to do
that, which resulted in crashes due to bogus ranges (equal minval/maxval
but not marked as compacted).
Reported-by: Jaime Casanova
Discussion: https://postgr.es/m/20210404052550.GA4376%40ahch-to
M src/backend/access/brin/brin_minmax_multi.c
Add regression test for minmax-multi macaddr8 type
commit : 4908684ddab35135869efa2af6b49c4d67c422f9
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 4 Apr 2021 19:26:48 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 4 Apr 2021 19:26:48 +0200
The regression test for BRIN minmax-multi opclasses tested almost all
supported data types, with the exception of macaddr8. So this adds it.
M src/test/regress/expected/brin_multi.out
M src/test/regress/sql/brin_multi.sql
Fix order of parameters in BRIN minmax-multi calls
commit : 1dad2a5ea3d14dd205603c31cc94ec088183ab2a
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 4 Apr 2021 19:25:36 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 4 Apr 2021 19:25:36 +0200
The BRIN minmax-multi consistent function incorrectly assumed it can
lookup an operator, and then swap the arguments to get the commutator.
For example <(a,b) would be called as <(b,a) to get >(a,b). This works
when the arguments are of the same type, but with cross-type opclasses
this fails. We can't swap <(float4,float8) arguments, for example.
Fixed by passing arguments in the right order.
Discussion: https://postgr.es/m/CAJKUy5jLZFLCxyxfT%3DMfK5mtPfSzHA1rVLowR-j4RRsFVvKm7A%40mail.gmail.com
M src/backend/access/brin/brin_minmax_multi.c
Fix BRIN minmax-multi distance for inet type
commit : e1fbe1181c86247eaf8b4b142b81361ce4efcc66
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 4 Apr 2021 19:23:30 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 4 Apr 2021 19:23:30 +0200
The distance calculation ignored the mask, unlike the inet comparator,
which resulted in negative distance in some cases. Fixed by applying the
mask in brin_minmax_multi_distance_inet. I've considered simply calling
inetmi() to calculate the delta, but that does not consider mask either.
Reviewed-by: Zhihong Yu
Discussion: https://postgr.es/m/1a0a7b9d-9bda-e3a2-7fa4-88f15042a051%40enterprisedb.com
M src/backend/access/brin/brin_minmax_multi.c
Fix BRIN minmax-multi distance for timetz type
commit : 7262f2421a1e099a631356f7b80ad198e34e2a8a
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 4 Apr 2021 19:21:41 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 4 Apr 2021 19:21:41 +0200
The distance calculation ignored the time zone, so the result of (b-a)
might have ended negative even if (b > a). Fixed by considering the time
zone difference.
Reported-by: Jaime Casanova
Discussion: https://postgr.es/m/CAJKUy5jLZFLCxyxfT%3DMfK5mtPfSzHA1rVLowR-j4RRsFVvKm7A%40mail.gmail.com
M src/backend/access/brin/brin_minmax_multi.c
Fix BRIN minmax-multi distance for interval type
commit : 2b10e0e3c2ca14d732521479123e5d5e2094e143
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 4 Apr 2021 19:13:26 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 4 Apr 2021 19:13:26 +0200
The distance calculation for interval type was treating months as having
31 days, which is inconsistent with the interval comparator (using 30
days). Due to this it was possible to get negative distance (b-a) when
(a<b), trigerring an assert.
Fixed by adopting the same logic as interval_cmp_value.
Reported-by: Jaime Casanova
Discussion: https://postgr.es/m/CAJKUy5jKH0Xhneau2mNftNPtTy-BVgQfXc8zQkEvRvBHfeUThQ%40mail.gmail.com
M src/backend/access/brin/brin_minmax_multi.c
Improve psql's behavior when the editor is exited without saving.
commit : 55873a00e3c3349664e7215077dca74ccea08b4d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 3 Apr 2021 17:38:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 3 Apr 2021 17:38:31 -0400
When editing the previous query buffer, if the editor is exited
without modifying the temp file then clear the query buffer,
rather than re-loading (and probably re-executing) the previous
query buffer. This reduces the probability of accidentally
re-executing something you didn't intend to.
Similarly, in "\e file", if the file isn't actually modified
then don't load it into the query buffer. And in "\ef" and
"\ev", if no changes are made then clear the query buffer
instead of loading the function or view definition into it.
Cases where we fail to invoke the editor at all, or it returns
a nonzero status, are treated like the no-file-modification case.
Laurenz Albe, reviewed by Jacob Champion
Discussion: https://postgr.es/m/0ba3f2a658bac6546d9934ab6ba63a805d46a49b.camel@cybertec.at
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
Improve efficiency of wait event reporting, remove proc.h dependency.
commit : 225a22b19ed2960acc8e9c0b7ae53e0e5b0eac87
author : Andres Freund <andres@anarazel.de>
date : Sat, 3 Apr 2021 11:44:47 -0700
committer: Andres Freund <andres@anarazel.de>
date : Sat, 3 Apr 2021 11:44:47 -0700
pgstat_report_wait_start() and pgstat_report_wait_end() required two
conditional branches so far. One to check if MyProc is NULL, the other to
check if pgstat_track_activities is set. As wait events are used around
comparatively lightweight operations, and are inlined (reducing branch
predictor effectiveness), that's not great.
The dependency on MyProc has a second disadvantage: Low-level subsystems, like
storage/file/fd.c, report wait events, but architecturally it is preferable
for them to not depend on inter-process subsystems like proc.h (defining
PGPROC). After this change including pgstat.h (nor obviously its
sub-components like backend_status.h, wait_event.h, ...) does not pull in IPC
related headers anymore.
These goals, efficiency and abstraction, are achieved by having
pgstat_report_wait_start/end() not interact with MyProc, but instead a new
my_wait_event_info variable. At backend startup it points to a local variable,
removing the need to check for MyProc being NULL. During process
initialization my_wait_event_info is redirected to MyProc->wait_event_info. At
shutdown this is reversed. Because wait event reporting now does not need to
know about where the wait event is stored, it does not need to know about
PGPROC anymore.
The removal of the branch for checking pgstat_track_activities is simpler:
Don't check anymore. The cost due to the branch are often higher than the
store - and even if not, pgstat_track_activities is rarely disabled.
The main motivator to commit this work now is that removing the (indirect)
pgproc.h include from pgstat.h simplifies a patch to move statistics reporting
to shared memory (which still has a chance to get into 14).
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20210402194458.2vu324hkk2djq6ce@alap3.anarazel.de
M src/backend/storage/lmgr/proc.c
M src/backend/utils/activity/wait_event.c
M src/include/utils/wait_event.h
Split backend status and progress related functionality out of pgstat.c.
commit : e1025044cd4e7f33f7304aed54d5778b8a82cd5d
author : Andres Freund <andres@anarazel.de>
date : Sat, 3 Apr 2021 11:42:52 -0700
committer: Andres Freund <andres@anarazel.de>
date : Sat, 3 Apr 2021 11:42:52 -0700
Backend status (supporting pg_stat_activity) and command
progress (supporting pg_stat_progress*) related code is largely
independent from the rest of pgstat.[ch] (supporting views like
pg_stat_all_tables that accumulate data over time). See also
a333476b925.
This commit doesn't rename the function names to make the distinction
from the rest of pgstat_ clearer - that'd be more invasive and not
clearly beneficial. If we were to decide to do such a rename at some
point, it's better done separately from moving the code as well.
Robert's review was of an earlier version.
Reviewed-By: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/20210316195440.twxmlov24rr2nxrg@alap3.anarazel.de
M src/backend/bootstrap/bootstrap.c
M src/backend/postmaster/pgstat.c
M src/backend/utils/activity/Makefile
A src/backend/utils/activity/backend_progress.c
A src/backend/utils/activity/backend_status.c
M src/backend/utils/init/postinit.c
M src/backend/utils/misc/guc.c
M src/include/commands/progress.h
M src/include/pgstat.h
A src/include/utils/backend_progress.h
A src/include/utils/backend_status.h
Use more verbose matching patterns for errors in SSL TAP tests
commit : 8d3a4c3eae5367fba60ab77c159814defba784fe
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 3 Apr 2021 20:49:08 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 3 Apr 2021 20:49:08 +0900
The TAP tests of src/test/ssl/ have been using rather generic matching
patterns to check some failure scenarios, like "SSL error" or just
"FATAL". These have been introduced in 081bfc1.
Those messages are not wrong per se, but when working on the integration
of new SSL libraries it becomes hard to know if those errors are legit
or not, and existing scenarios may fail in incorrect ways. This commit
makes all those messages more verbose by adding the information
generated by OpenSSL. Fortunately, the same error messages are used for
all the versions supported on HEAD (checked that after running the tests
from 1.0.1 to 1.1.1), so the change is straight-forward.
Reported-by: Jacob Champion, Álvaro Herrera
Discussion: https://postgr.es/m/YGU3AxQh0zBMMW8m@paquier.xyz
M src/test/ssl/t/001_ssltests.pl
Refactor HMAC implementations
commit : e6bdfd9700ebfc7df811c97c2fc46d7e94e329a2
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 3 Apr 2021 17:30:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 3 Apr 2021 17:30:49 +0900
Similarly to the cryptohash implementations, this refactors the existing
HMAC code into a single set of APIs that can be plugged with any crypto
libraries PostgreSQL is built with (only OpenSSL currently). If there
is no such libraries, a fallback implementation is available. Those new
APIs are designed similarly to the existing cryptohash layer, so there
is no real new design here, with the same logic around buffer bound
checks and memory handling.
HMAC has a dependency on cryptohashes, so all the cryptohash types
supported by cryptohash{_openssl}.c can be used with HMAC. This
refactoring is an advantage mainly for SCRAM, that included its own
implementation of HMAC with SHA256 without relying on the existing
crypto libraries even if PostgreSQL was built with their support.
This code has been tested on Windows and Linux, with and without
OpenSSL, across all the versions supported on HEAD from 1.1.1 down to
1.0.1. I have also checked that the implementations are working fine
using some sample results, a custom extension of my own, and doing
cross-checks across different major versions with SCRAM with the client
and the backend.
Author: Michael Paquier
Reviewed-by: Bruce Momjian
Discussion: https://postgr.es/m/X9m0nkEJEzIPXjeZ@paquier.xyz
M configure
M configure.ac
M src/backend/libpq/auth-scram.c
M src/backend/utils/resowner/resowner.c
M src/common/Makefile
A src/common/hmac.c
A src/common/hmac_openssl.c
M src/common/scram-common.c
A src/include/common/hmac.h
M src/include/common/md5.h
M src/include/common/scram-common.h
M src/include/common/sha1.h
M src/include/pg_config.h.in
M src/include/utils/resowner_private.h
M src/interfaces/libpq/fe-auth-scram.c
M src/tools/msvc/Mkvcbuild.pm
M src/tools/msvc/Solution.pm
M src/tools/pgindent/typedefs.list
Do not rely on pgstat.h to indirectly include storage/ headers.
commit : 1d9c5d0ce2dcac05850401cf266a9df10a68de49
author : Andres Freund <andres@anarazel.de>
date : Fri, 2 Apr 2021 20:01:14 -0700
committer: Andres Freund <andres@anarazel.de>
date : Fri, 2 Apr 2021 20:01:14 -0700
An upcoming patch might remove the (now indirect) proc.h
include (which in turn includes other headers), and it's cleaner for
the modified files to include their dependencies directly anyway...
Discussion: https://postgr.es/m/20210402194458.2vu324hkk2djq6ce@alap3.anarazel.de
M contrib/pg_stat_statements/pg_stat_statements.c
M src/backend/postmaster/pgarch.c
M src/backend/postmaster/pgstat.c
M src/backend/replication/walreceiver.c
M src/backend/storage/lmgr/lmgr.c
M src/backend/utils/adt/misc.c
Split wait event related code from pgstat.[ch] into wait_event.[ch].
commit : a333476b925134f6185037eaff3424c07a9f466f
author : Andres Freund <andres@anarazel.de>
date : Fri, 2 Apr 2021 19:45:24 -0700
committer: Andres Freund <andres@anarazel.de>
date : Fri, 2 Apr 2021 19:45:24 -0700
The wait event related code is independent from the rest of the
pgstat.[ch] code, of nontrivial size and changes on a regular
basis. Put it into its own set of files.
As there doesn't seem to be a good pre-existing directory for code
like this, add src/backend/utils/activity.
Reviewed-By: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/20210316195440.twxmlov24rr2nxrg@alap3.anarazel.de
M src/backend/postmaster/pgstat.c
M src/backend/utils/Makefile
A src/backend/utils/activity/Makefile
A src/backend/utils/activity/wait_event.c
M src/include/pgstat.h
A src/include/utils/wait_event.h
Remove useless Asserts in Result Cache code
commit : 1267d9862fc6a4f8cdc0ca38d1988b61f39da585
author : David Rowley <drowley@postgresql.org>
date : Sat, 3 Apr 2021 10:41:43 +1300
committer: David Rowley <drowley@postgresql.org>
date : Sat, 3 Apr 2021 10:41:43 +1300
Testing if an unsigned variable is >= 0 is pretty pointless.
There's likely enough code in remove_cache_entry() to verify the cache
memory accounting is correct in assert enabled builds. These Asserts
were not adding much extra cover, even if they had been checking >= 0 on a
signed variable.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/20210402204734.6mo3nfacnljlicgn@alap3.anarazel.de
M src/backend/executor/nodeResultCache.c
Use macro MONTHS_PER_YEAR instead of '12' in /ecpg/pgtypeslib
commit : 84bc2b17523ef485f102be7f00f7affb88f00f18
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Apr 2021 16:42:29 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Apr 2021 16:42:29 -0400
All other places already use MONTHS_PER_YEAR appropriately.
Backpatch-through: 9.6
M src/interfaces/ecpg/pgtypeslib/interval.c
Detect POLLHUP/POLLRDHUP while running queries.
commit : c30f54ad732ca5c8762bb68bbe0f51de9137dd72
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 3 Apr 2021 08:52:30 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 3 Apr 2021 08:52:30 +1300
Provide a new GUC check_client_connection_interval that can be used to
check whether the client connection has gone away, while running very
long queries. It is disabled by default.
For now this uses a non-standard Linux extension (also adopted by at
least one other OS). POLLRDHUP is not defined by POSIX, and other OSes
don't have a reliable way to know if a connection was closed without
actually trying to read or write.
In future we might consider trying to send a no-op/heartbeat message
instead, but that could require protocol changes.
Author: Sergey Cherkashin <s.cherkashin@postgrespro.ru>
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Tatsuo Ishii <ishii@sraoss.co.jp>
Reviewed-by: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
Reviewed-by: Zhihong Yu <zyu@yugabyte.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Maksim Milyutin <milyutinma@gmail.com>
Reviewed-by: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (much earlier version)
Discussion: https://postgr.es/m/77def86b27e41f0efcba411460e929ae%40postgrespro.ru
M doc/src/sgml/config.sgml
M src/backend/libpq/pqcomm.c
M src/backend/tcop/postgres.c
M src/backend/utils/init/globals.c
M src/backend/utils/init/postinit.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/libpq/libpq.h
M src/include/miscadmin.h
M src/include/tcop/tcopprot.h
M src/include/utils/timeout.h
Clarify documentation of RESET ROLE
commit : 174edbe9f9c1538ab3347474e96d176223591cd1
author : Joe Conway <mail@joeconway.com>
date : Fri, 2 Apr 2021 13:48:42 -0400
committer: Joe Conway <mail@joeconway.com>
date : Fri, 2 Apr 2021 13:48:42 -0400
Command-line options, or previous "ALTER (ROLE|DATABASE) ...
SET ROLE ..." commands, can change the value of the default role
for a session. In the presence of one of these, RESET ROLE will
change the current user identifier to the default role rather
than the session user identifier. Fix the documentation to
reflect this reality. Backpatch to all supported versions.
Author: Nathan Bossart
Reviewed-By: Laurenz Albe, David G. Johnston, Joe Conway
Reported by: Nathan Bossart
Discussion: https://postgr.es/m/flat/925134DB-8212-4F60-8AB1-B1231D750CB4%40amazon.com
Backpatch-through: 9.6
M doc/src/sgml/ref/set_role.sgml
pg_checksums: Fix progress reporting.
commit : 2eb1fc8b1ae8b974007e85636fc7336a9b5d7222
author : Fujii Masao <fujii@postgresql.org>
date : Sat, 3 Apr 2021 00:07:00 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Sat, 3 Apr 2021 00:07:00 +0900
pg_checksums uses two counters, total size and current size,
to calculate the progress. Previously the progress that
pg_checksums reported could not reach 100% at the end.
The cause of this issue was that the sizes of only pages excluding
new ones in each file were counted as the current size
while the size of each file is counted as the total size.
That is, the total size of all new pages could be reported
as the difference between the total size and current size.
This commit fixes this issue by making pg_checksums count
the sizes of all pages including new ones in each file as
the current size.
Back-patch to v12 where progress reporting was added to pg_checksums.
Reported-by: Shinya Kato
Author: Shinya Kato
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/TYAPR01MB289656B1ACA0A5E7CAD07BE3C47A9@TYAPR01MB2896.jpnprd01.prod.outlook.com
M src/bin/pg_checksums/pg_checksums.c
Strip file names reported in error messages on Windows, too.
commit : 53aafdb9ff6a561c7dea0f428a7c168f2b7e0f16
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 2 Apr 2021 10:43:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 2 Apr 2021 10:43:54 -0400
Commit dd136052b established a policy that error message FILE items
should include only the base name of the reporting source file, for
uniformity and succinctness. We now observe that some Windows compilers
use backslashes in __FILE__ strings, so truncate at backslashes as well.
This is expected to fix some platform variation in the results of the
new libpq_pipeline test module.
Discussion: https://postgr.es/m/3650140.1617372290@sss.pgh.pa.us
M src/backend/utils/error/elog.c
Fix typo in 6d7a6feac4
commit : 1877c9ac3acc05cc787dd6392d073202f8c8ee21
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 2 Apr 2021 10:29:58 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 2 Apr 2021 10:29:58 -0400
Per gripe from Daniel Gustafsson
M src/test/ssl/Makefile
postgres_fdw: Add option to control whether to keep connections open.
commit : b1be3074ac719ce8073fba35d4c8b52fb4ddd0c3
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 2 Apr 2021 19:45:42 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 2 Apr 2021 19:45:42 +0900
This commit adds a new option keep_connections that controls
whether postgres_fdw keeps the connections to the foreign server open
so that the subsequent queries can re-use them. This option can only be
specified for a foreign server. The default is on. If set to off,
all connections to the foreign server will be discarded
at the end of transaction. Closed connections will be re-established
when they are necessary by future queries using a foreign table.
This option is useful, for example, when users want to prevent
the connections from eating up the foreign servers connections
capacity.
Author: Bharath Rupireddy
Reviewed-by: Alexey Kondratov, Vignesh C, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVvrp5=AVp2PupEm+nAC8S4buqR3fJMmaCoc7ftT0aD2A@mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Add support for NullIfExpr in eval_const_expressions
commit : 9c5f67fd6256246b2a788a8feb1d42b79dcd0448
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 2 Apr 2021 11:01:49 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 2 Apr 2021 11:01:49 +0200
Author: Hou Zhijie <houzj.fnst@cn.fujitsu.com>
Discussion: https://www.postgresql.org/message-id/flat/7ea5ce773bbc4eea9ff1a381acd3b102@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/optimizer/util/clauses.c
M src/test/regress/expected/case.out
M src/test/regress/sql/case.sql
Fix pgstat_report_replslot() to use proper data types for its arguments.
commit : 96bdb7e19de80a0c9521c5696455bca2a685c919
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 2 Apr 2021 17:27:31 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 2 Apr 2021 17:27:31 +0900
The caller of pgstat_report_replslot() passes int64 values to the function.
Also the function stores those values in PgStat_Counter (i.e., int64) fields
of PgStat_MsgReplSlot struct. But previously the function used "int" as
the data types of some arguments for those values, which could lead to
the overflow of values.
To avoid this risk, this commit fixes pgstat_report_replslot() to use
PgStat_Counter type for the arguments. Since they are the statistics counters,
PgStat_Counter, the data type used for counters, is used for them
instead of int64.
Reported-by: Vignesh C
Author: Vignesh C
Reviewed-by: Jeevan Ladhe, Fujii Masao
Discussion: https://postgr.es/m/CALDaNm080OpG=ZwOb0i8EyChH5SyHAMFWJCKaKTXmrfvJLbgaA@mail.gmail.com
M src/backend/postmaster/pgstat.c
M src/include/pgstat.h
doc: Clarify how to generate backup files with non-exclusive backups
commit : 6fb66c268df2de1112cac3cf0a6cf0a8b96ceaf0
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 2 Apr 2021 16:37:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 2 Apr 2021 16:37:00 +0900
The current instructions describing how to write the backup_label and
tablespace_map files are confusing. For example, opening a file in text
mode on Windows and copy-pasting the file's contents would result in a
failure at recovery because of the extra CRLF characters generated. The
documentation was not stating that clearly, and per discussion this is
not considered as a supported scenario.
This commit extends a bit the documentation to mention that it may be
required to open the file in binary mode before writing its data.
Reported-by: Wang Shenhao
Author: David Steele
Reviewed-by: Andrew Dunstan, Magnus Hagander
Discussion: https://postgr.es/m/8373f61426074f2cb6be92e02f838389@G08CNEXMBPEKD06.g08.fujitsu.local
Backpatch-through: 9.6
M doc/src/sgml/backup.sgml
Fix typos in comments.
commit : 98e5bd103f887326e381c509c2fbe879ba3ea2f3
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 2 Apr 2021 16:26:36 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 2 Apr 2021 16:26:36 +0900
Author: Masahiko Sawada
Discussion: https://postgr.es/m/CAD21AoA1YL7t0nzVSEySx6zOaE7xO3r0jyu8hkitGL2_XbaMxQ@mail.gmail.com
M contrib/amcheck/verify_heapam.c
Attempt to fix unstable Result Cache regression tests
commit : a4fac4ffe8f8d543a10ac7debf1157e34963ece3
author : David Rowley <drowley@postgresql.org>
date : Fri, 2 Apr 2021 15:25:38 +1300
committer: David Rowley <drowley@postgresql.org>
date : Fri, 2 Apr 2021 15:25:38 +1300
force_parallel_mode = regress is causing a few more problems than I
thought. It seems that both the leader and the single worker can
contribute to the execution. I had mistakenly thought that only the worker
process would do any work. Since it's not deterministic as to which
of the two processes will get a chance to work on the plan, it seems just
better to disable force_parallel_mode for these tests. At least doing
this seems better than changing to EXPLAIN only rather than EXPLAIN
ANALYZE.
Additionally, I overlooked the fact that the number of executions of the
sub-plan below a Result Cache will execute a varying number of times
depending on cache eviction. 32-bit machines will use less memory and
evict fewer tuples from the cache. That results in the subnode being
executed fewer times on 32-bit machines. Let's just blank out the number
of loops in each node.
M src/test/regress/expected/resultcache.out
M src/test/regress/sql/resultcache.sql
doc: mention that intervening major releases can be skipped
commit : 2bda93f813919b58225f5a0e282e10b98d7633d4
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 1 Apr 2021 21:17:24 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 1 Apr 2021 21:17:24 -0400
Also mention that you should read the intervening major releases notes.
This change was also applied to the website.
Discussion: https://postgr.es/m/20210330144949.GA8259@momjian.us
Backpatch-through: 9.6
M doc/src/sgml/runtime.sgml
Add Result Cache executor node (take 2)
commit : 9eacee2e62d89cab7b004f97c206c4fba4f1d745
author : David Rowley <drowley@postgresql.org>
date : Fri, 2 Apr 2021 14:10:56 +1300
committer: David Rowley <drowley@postgresql.org>
date : Fri, 2 Apr 2021 14:10:56 +1300
Here we add a new executor node type named "Result Cache". The planner
can include this node type in the plan to have the executor cache the
results from the inner side of parameterized nested loop joins. This
allows caching of tuples for sets of parameters so that in the event that
the node sees the same parameter values again, it can just return the
cached tuples instead of rescanning the inner side of the join all over
again. Internally, result cache uses a hash table in order to quickly
find tuples that have been previously cached.
For certain data sets, this can significantly improve the performance of
joins. The best cases for using this new node type are for join problems
where a large portion of the tuples from the inner side of the join have
no join partner on the outer side of the join. In such cases, hash join
would have to hash values that are never looked up, thus bloating the hash
table and possibly causing it to multi-batch. Merge joins would have to
skip over all of the unmatched rows. If we use a nested loop join with a
result cache, then we only cache tuples that have at least one join
partner on the outer side of the join. The benefits of using a
parameterized nested loop with a result cache increase when there are
fewer distinct values being looked up and the number of lookups of each
value is large. Also, hash probes to lookup the cache can be much faster
than the hash probe in a hash join as it's common that the result cache's
hash table is much smaller than the hash join's due to result cache only
caching useful tuples rather than all tuples from the inner side of the
join. This variation in hash probe performance is more significant when
the hash join's hash table no longer fits into the CPU's L3 cache, but the
result cache's hash table does. The apparent "random" access of hash
buckets with each hash probe can cause a poor L3 cache hit ratio for large
hash tables. Smaller hash tables generally perform better.
The hash table used for the cache limits itself to not exceeding work_mem
* hash_mem_multiplier in size. We maintain a dlist of keys for this cache
and when we're adding new tuples and realize we've exceeded the memory
budget, we evict cache entries starting with the least recently used ones
until we have enough memory to add the new tuples to the cache.
For parameterized nested loop joins, we now consider using one of these
result cache nodes in between the nested loop node and its inner node. We
determine when this might be useful based on cost, which is primarily
driven off of what the expected cache hit ratio will be. Estimating the
cache hit ratio relies on having good distinct estimates on the nested
loop's parameters.
For now, the planner will only consider using a result cache for
parameterized nested loop joins. This works for both normal joins and
also for LATERAL type joins to subqueries. It is possible to use this new
node for other uses in the future. For example, to cache results from
correlated subqueries. However, that's not done here due to some
difficulties obtaining a distinct estimation on the outer plan to
calculate the estimated cache hit ratio. Currently we plan the inner plan
before planning the outer plan so there is no good way to know if a result
cache would be useful or not since we can't estimate the number of times
the subplan will be called until the outer plan is generated.
The functionality being added here is newly introducing a dependency on
the return value of estimate_num_groups() during the join search.
Previously, during the join search, we only ever needed to perform
selectivity estimations. With this commit, we need to use
estimate_num_groups() in order to estimate what the hit ratio on the
result cache will be. In simple terms, if we expect 10 distinct values
and we expect 1000 outer rows, then we'll estimate the hit ratio to be
99%. Since cache hits are very cheap compared to scanning the underlying
nodes on the inner side of the nested loop join, then this will
significantly reduce the planner's cost for the join. However, it's
fairly easy to see here that things will go bad when estimate_num_groups()
incorrectly returns a value that's significantly lower than the actual
number of distinct values. If this happens then that may cause us to make
use of a nested loop join with a result cache instead of some other join
type, such as a merge or hash join. Our distinct estimations have been
known to be a source of trouble in the past, so the extra reliance on them
here could cause the planner to choose slower plans than it did previous
to having this feature. Distinct estimations are also fairly hard to
estimate accurately when several tables have been joined already or when a
WHERE clause filters out a set of values that are correlated to the
expressions we're estimating the number of distinct value for.
For now, the costing we perform during query planning for result caches
does put quite a bit of faith in the distinct estimations being accurate.
When these are accurate then we should generally see faster execution
times for plans containing a result cache. However, in the real world, we
may find that we need to either change the costings to put less trust in
the distinct estimations being accurate or perhaps even disable this
feature by default. There's always an element of risk when we teach the
query planner to do new tricks that it decides to use that new trick at
the wrong time and causes a regression. Users may opt to get the old
behavior by turning the feature off using the enable_resultcache GUC.
Currently, this is enabled by default. It remains to be seen if we'll
maintain that setting for the release.
Additionally, the name "Result Cache" is the best name I could think of
for this new node at the time I started writing the patch. Nobody seems
to strongly dislike the name. A few people did suggest other names but no
other name seemed to dominate in the brief discussion that there was about
names. Let's allow the beta period to see if the current name pleases
enough people. If there's some consensus on a better name, then we can
change it before the release. Please see the 2nd discussion link below
for the discussion on the "Result Cache" name.
Author: David Rowley
Reviewed-by: Andy Fan, Justin Pryzby, Zhihong Yu, Hou Zhijie
Tested-By: Konstantin Knizhnik
Discussion: https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com
Discussion: https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M src/backend/commands/explain.c
M src/backend/executor/Makefile
M src/backend/executor/execAmi.c
M src/backend/executor/execExpr.c
M src/backend/executor/execParallel.c
M src/backend/executor/execProcnode.c
A src/backend/executor/nodeResultCache.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/README
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/executor/executor.h
A src/include/executor/nodeResultCache.h
M src/include/lib/ilist.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/cost.h
M src/include/optimizer/pathnode.h
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/join.out
M src/test/regress/expected/partition_prune.out
A src/test/regress/expected/resultcache.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/sysviews.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/join.sql
M src/test/regress/sql/partition_prune.sql
A src/test/regress/sql/resultcache.sql
Improve stability of test with vacuum_truncate in reloptions.sql
commit : fe246d1c111d43fd60a1b0afff25ed09b7ae11eb
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 2 Apr 2021 09:44:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 2 Apr 2021 09:44:42 +0900
This test has been using a simple VACUUM with pg_relation_size() to
check if a relation gets physically truncated or not, but forgot the
fact that some concurrent activity, like checkpoint buffer writes, could
cause some pages to be skipped. The second test enabling
vacuum_truncate could fail, seeing a non-empty relation. The first test
would not have failed, but could finish by testing a behavior different
than the one aimed for. Both tests gain a FREEZE option, to make the
vacuums more aggressive and prevent page skips.
This is similar to the issues fixed in c2dc1a7.
Author: Arseny Sher
Reviewed-by: Masahiko Sawada
Discussion: https://postgr.es/m/87tuotr2hh.fsf@ars-thinkpad
backpatch-through: 12
M src/test/regress/expected/reloptions.out
M src/test/regress/sql/reloptions.sql
Rethink handling of pass-by-value leaf datums in SP-GiST.
commit : 1ebdec8c03294e55a9fdb6e676a9e8de680231cc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 1 Apr 2021 17:55:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 1 Apr 2021 17:55:14 -0400
The existing convention in SP-GiST is that any pass-by-value datatype
is stored in Datum representation, i.e. it's of width sizeof(Datum)
even when typlen is less than that. This is okay, or at least it's
too late to change it, for prefix datums and node-label datums in inner
(upper) tuples. But it's problematic for leaf datums, because we'd
prefer those to be stored in Postgres' standard on-disk representation
so that we can easily extend leaf tuples to carry additional "included"
columns.
I believe, however, that we can get away with just up and changing that.
This would be an unacceptable on-disk-format break, but there are two
big mitigating factors:
1. It seems quite unlikely that there are any SP-GiST opclasses out
there that use pass-by-value leaf datatypes. Certainly none of the
ones in core do, nor has codesearch.debian.net heard of any. Given
what SP-GiST is good for, it's hard to conceive of a use-case where
the leaf-level values would be both small and fixed-width. (As an
example, if you wanted to index text values with the leaf level being
just a byte, then every text string would have to be represented
with one level of inner tuple per preceding byte, which would be
horrendously space-inefficient and slow to access. You always want
to use as few inner-tuple levels as possible, leaving as much as
possible in the leaf values.)
2. Even granting that you have such an index, this change only
breaks things on big-endian machines. On little-endian, the high
order bytes of the Datum format will now just appear to be alignment
padding space.
So, change the code to store pass-by-value leaf datums in their
usual on-disk form. Inner-tuple datums are not touched.
This is extracted from a larger patch that intends to add support for
"included" columns. I'm committing it separately for visibility in
our commit logs.
Pavel Borisov and Tom Lane, reviewed by Andrey Borodin
Discussion: https://postgr.es/m/CALT9ZEFi-vMp4faht9f9Junb1nO3NOSjhpxTmbm1UGLMsLqiEQ@mail.gmail.com
M src/backend/access/spgist/spgdoinsert.c
M src/backend/access/spgist/spgutils.c
M src/include/access/spgist_private.h
Rename Default Roles to Predefined Roles
commit : c9c41c7a337d3e2deb0b2a193e9ecfb865d8f52b
author : Stephen Frost <sfrost@snowman.net>
date : Thu, 1 Apr 2021 15:32:06 -0400
committer: Stephen Frost <sfrost@snowman.net>
date : Thu, 1 Apr 2021 15:32:06 -0400
The term 'default roles' wasn't quite apt as these roles aren't able to
be modified or removed after installation, so rename them to be
'Predefined Roles' instead, adding an entry into the newly added
Obsolete Appendix to help users of current releases find the new
documentation.
Bruce Momjian and Stephen Frost
Discussion: https://postgr.es/m/157742545062.1149.11052653770497832538%40wrigleys.postgresql.org
and https://www.postgresql.org/message-id/20201120211304.GG16415@tamriel.snowman.net
M contrib/adminpack/adminpack.c
M contrib/file_fdw/file_fdw.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pgrowlocks/pgrowlocks.c
A doc/src/sgml/appendix-obsolete-default-roles.sgml
M doc/src/sgml/appendix-obsolete.sgml
M doc/src/sgml/file-fdw.sgml
M doc/src/sgml/filelist.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/ref/copy.sgml
M doc/src/sgml/user-manag.sgml
M src/backend/commands/copy.c
M src/backend/commands/user.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walsender.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/signalfuncs.c
M src/backend/utils/adt/acl.c
M src/backend/utils/adt/dbsize.c
M src/backend/utils/adt/genfile.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/misc/guc.c
M src/include/catalog/pg_authid.dat
Fix setvbuf()-induced crash in libpq_pipeline
commit : a68a894f0198aaeffa81b3027f135adcdaa8abf6
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 1 Apr 2021 16:25:46 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 1 Apr 2021 16:25:46 -0300
Windows doesn't like setvbuf(..., _IOLBF) and crashes if you use it,
which has been causing the libpq_pipeline failures all along ... and our
own port.h has known about it for a long time: it offers PG_IOLBF that's
defined to _IONBF on that platform. Follow its advice.
While at it, get rid of a bogus bitshift that used a constant of the
wrong size. Decorate the constant as LL to fix. While at it, remove a
pointless addition that only confused matters.
All as diagnosed by Tom Lane.
Discussion: https://postgr.es/m/3458958.1617302154@sss.pgh.pa.us
M src/test/modules/libpq_pipeline/libpq_pipeline.c
amcheck: Fix verify_heapam's tuple visibility checking rules.
commit : 3b6c1259f9ca8e21860aaf24ec6735a8e5598ea0
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 1 Apr 2021 13:36:28 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 1 Apr 2021 13:36:28 -0400
We now follow the order of checks from HeapTupleSatisfies* more
closely to avoid coming to erroneous conclusions.
Mark Dilger and Robert Haas
Discussion: http://postgr.es/m/CA+Tgmob6sii0yTvULYJ0Vq4w6ZBmj7zUhddL3b+SKDi9z9NA7Q@mail.gmail.com
M contrib/amcheck/verify_heapam.c
Fix pg_restore's misdesigned code for detecting archive file format.
commit : ec03f2df17a8ba5b431b34dd924e020a0be729f6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 1 Apr 2021 13:34:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 1 Apr 2021 13:34:16 -0400
Despite the clear comments pointing out that the duplicative code
segments in ReadHead() and _discoverArchiveFormat() needed to be
in sync, they were not: the latter did not bother to apply any of
the sanity checks in the former. We'd missed noticing this partly
because none of those checks would fail in scenarios we customarily
test, and partly because the oversight would be masked if both
segments execute, which they would in cases other than needing to
autodetect the format of a non-seekable stdin source. However,
in a case meeting all these requirements --- for example, trying
to read a newer-than-supported archive format from non-seekable
stdin --- pg_restore missed applying the version check and would
likely dump core or otherwise misbehave.
The whole thing is silly anyway, because there seems little reason
to duplicate the logic beyond the one-line verification that the
file starts with "PGDMP". There seems to have been an undocumented
assumption that multiple major formats (major enough to require
separate reader modules) would nonetheless share the first half-dozen
fields of the custom-format header. This seems unlikely, so let's
fix it by just nuking the duplicate logic in _discoverArchiveFormat().
Also get rid of the pointless attempt to seek back to the start of
the file after successful autodetection. That wastes cycles and
it means we have four behaviors to verify not two.
Per bug #16951 from Sergey Koposov. This has been broken for
decades, so back-patch to all supported versions.
Discussion: https://postgr.es/m/16951-a4dd68cf0de23048@postgresql.org
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_tar.c
Fix internal extract(timezone_minute) formulas
commit : 91e7c903291116bd081abe7d4a058d40a2a06e16
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 1 Apr 2021 16:12:53 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 1 Apr 2021 16:12:53 +0200
Through various refactorings over time, the extract(timezone_minute
from time with time zone) and extract(timezone_minute from timestamp
with time zone) implementations ended up with two different but
equally nonsensical formulas by using SECS_PER_MINUTE and
MINS_PER_HOUR interchangeably. Since those two are of course both the
same number, the formulas do work, but for readability, fix them to be
semantically correct.
M src/backend/utils/adt/date.c
M src/backend/utils/adt/timestamp.c
libpq_pipeline: Must strdup(optarg) to avoid crash
commit : dde1a35aee6266dc8105717275335c46cd2b3650
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 1 Apr 2021 10:04:45 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 1 Apr 2021 10:04:45 -0300
I forgot to strdup() when processing argv[]. Apparently many platforms
hide this mistake from users, but in those that don't you may get a
program crash. Repair.
Per buildfarm member drongo, which is the only one in all the buildfarm
manifesting a problem here.
While at it, move "numrows" processing out of the line of special cases,
and make it getopt's -r instead. (A similar thing could be done to
'conninfo', but current use of the program doesn't warrant spending time
on that -- nowhere else we use conninfo in so simplistic a manner.)
Discussion: https://postgr.es/m/20210401124850.GA19247@alvherre.pgsql
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
Do COPY FROM encoding conversion/verification in larger chunks.
commit : f82de5c46bdf8cd65812a7b04c9509c218e1545d
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 1 Apr 2021 12:23:40 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 1 Apr 2021 12:23:40 +0300
This gives a small performance gain, by reducing the number of calls
to the conversion/verification function, and letting it work with
larger inputs. Also, reorganizing the input pipeline makes it easier
to parallelize the input parsing: after the input has been converted
to the database encoding, the next stage of finding the newlines can
be done in parallel, because there cannot be any newline chars
"embedded" in multi-byte characters in the encodings that we support
as server encodings.
This changes behavior in one corner case: if client and server
encodings are the same single-byte encoding (e.g. latin1), previously
the input would not be checked for zero bytes ('\0'). Any fields
containing zero bytes would be truncated at the zero. But if encoding
conversion was needed, the conversion routine would throw an error on
the zero. After this commit, the input is always checked for zeros.
Reviewed-by: John Naylor
Discussion: https://www.postgresql.org/message-id/e7861509-3960-538a-9025-b75a61188e01%40iki.fi
M src/backend/commands/copyfrom.c
M src/backend/commands/copyfromparse.c
M src/include/commands/copyfrom_internal.h
M src/include/mb/pg_wchar.h
Add 'noError' argument to encoding conversion functions.
commit : ea1b99a6619cd9dcfd46b82ac0d926b0b80e0ae9
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 1 Apr 2021 11:45:22 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 1 Apr 2021 11:45:22 +0300
With the 'noError' argument, you can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions. Custom conversions are very rare, there are no
commonly used extensions that I know of that uses that feature. No other
objects can depend on conversions, so if you do have one, you can fairly
easily drop it before upgrading, and recreate it after the upgrade with
an updated version.
Add regression tests for built-in encoding conversions. This doesn't cover
every conversion, but it covers all the internal functions in conv.c that
are used to implement the conversions.
Reviewed-by: John Naylor
Discussion: https://www.postgresql.org/message-id/e7861509-3960-538a-9025-b75a61188e01%40iki.fi
M doc/src/sgml/ref/create_conversion.sgml
M src/backend/commands/conversioncmds.c
M src/backend/utils/error/elog.c
M src/backend/utils/mb/conv.c
M src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
M src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
M src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
M src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
M src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
M src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
M src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
M src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
M src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
M src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
M src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
M src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
M src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
M src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
M src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
M src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
M src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
M src/backend/utils/mb/mbutils.c
M src/bin/pg_upgrade/check.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/mb/pg_wchar.h
M src/test/regress/expected/conversion.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/input/create_function_1.source
M src/test/regress/output/create_function_1.source
M src/test/regress/regress.c
M src/test/regress/sql/conversion.sql
M src/test/regress/sql/opr_sanity.sql
Make extract(timetz) tests a bit more interesting
commit : e2639a767bfa1afebaf1877515a1187feb393443
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 1 Apr 2021 09:52:03 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 1 Apr 2021 09:52:03 +0200
Use a time zone offset with nonzero minutes to make the
timezone_minute test meaningful.
M src/test/regress/expected/timetz.out
M src/test/regress/sql/timetz.sql
doc: Clarify use of ACCESS EXCLUSIVE lock in various sections
commit : ffd3391ea94165fbb5adc9534894c62d41138505
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 1 Apr 2021 15:28:37 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 1 Apr 2021 15:28:37 +0900
Some sections of the documentation used "exclusive lock" to describe
that an ACCESS EXCLUSIVE lock is taken during a given operation. This
can be confusing to the reader as ACCESS SHARE is allowed with an
EXCLUSIVE lock is used, but that would not be the case with what is
described on those parts of the documentation.
Author: Greg Rychlewski
Discussion: https://postgr.es/m/CAKemG7VptD=7fNWckFMsMVZL_zzvgDO6v2yVmQ+ZiBfc_06kCQ@mail.gmail.com
Backpatch-through: 9.6
M doc/src/sgml/ddl.sgml
M doc/src/sgml/hstore.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/pgrowlocks.sgml
M doc/src/sgml/ref/drop_index.sgml
M doc/src/sgml/ref/reindex.sgml
M doc/src/sgml/ref/vacuum.sgml
Ensure to send a prepare after we detect concurrent abort during decoding.
commit : 4778826532a62fd6e4d3fdeef9532c943604c730
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 1 Apr 2021 07:57:34 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 1 Apr 2021 07:57:34 +0530
It is possible that while decoding a prepared transaction, it gets aborted
concurrently via a ROLLBACK PREPARED command. In that case, we were
skipping all the changes and directly sending Rollback Prepared when we
find the same in WAL. However, the downstream has no idea of the GID of
such a transaction. So, ensure to send prepare even when a concurrent
abort is detected.
Author: Ajin Cherian
Reviewed-by: Markus Wanner, Amit Kapila
Discussion: https://postgr.es/m/f82133c6-6055-b400-7922-97dae9f2b50b@enterprisedb.com
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/reorderbuffer.c
Move some client-specific routines from SSLServer to PostgresNode
commit : 0d1a33438d3a88938264e12e94c22818307d2f4d
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 1 Apr 2021 09:48:17 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 1 Apr 2021 09:48:17 +0900
test_connect_ok() and test_connect_fails() have always been part of the
SSL tests, and check if a connection to the backend should work or not,
and there are sanity checks done on specific error patterns dropped by
libpq if the connection fails.
This was fundamentally wrong on two aspects. First, SSLServer.pm works
mostly on setting up and changing the SSL configuration of a
PostgresNode, and has really nothing to do with the client. Second,
the situation became worse in light of b34ca595, where the SSL tests
would finish by using a psql command that may not come from the same
installation as the node set up.
This commit moves those client routines into PostgresNode, making easier
the refactoring of SSLServer to become more SSL-implementation aware.
This can also be reused by the ldap, kerberos and authentication test
suites for connection checks, and a follow-up patch should extend those
interfaces to match with backend log patterns.
Author: Michael Paquier
Reviewed-by: Andrew Dunstan, Daniel Gustafsson, Álvaro Herrera
Discussion: https://postgr.es/m/YGLKNBf9zyh6+WSt@paquier.xyz
M src/test/perl/PostgresNode.pm
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
M src/test/ssl/t/SSLServer.pm
Revert b6002a796
commit : 28b3e3905c982c42fb10ee800e6f881e9742c89d
author : David Rowley <drowley@postgresql.org>
date : Thu, 1 Apr 2021 13:33:23 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 1 Apr 2021 13:33:23 +1300
This removes "Add Result Cache executor node". It seems that something
weird is going on with the tracking of cache hits and misses as
highlighted by many buildfarm animals. It's not yet clear what the
problem is as other parts of the plan indicate that the cache did work
correctly, it's just the hits and misses that were being reported as 0.
This is especially a bad time to have the buildfarm so broken, so
reverting before too many more animals go red.
Discussion: https://postgr.es/m/CAApHDvq_hydhfovm4=izgWs+C5HqEeRScjMbOgbpC-jRAeK3Yw@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M src/backend/commands/explain.c
M src/backend/executor/Makefile
M src/backend/executor/execAmi.c
M src/backend/executor/execExpr.c
M src/backend/executor/execParallel.c
M src/backend/executor/execProcnode.c
D src/backend/executor/nodeResultCache.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/executor/executor.h
D src/include/executor/nodeResultCache.h
M src/include/lib/ilist.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/cost.h
M src/include/optimizer/pathnode.h
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/join.out
M src/test/regress/expected/partition_prune.out
D src/test/regress/expected/resultcache.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/sysviews.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/join.sql
M src/test/regress/sql/partition_prune.sql
D src/test/regress/sql/resultcache.sql
Add Result Cache executor node
commit : b6002a796dc0bfe721db5eaa54ba9d24fd9fd416
author : David Rowley <drowley@postgresql.org>
date : Thu, 1 Apr 2021 12:32:22 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 1 Apr 2021 12:32:22 +1300
Here we add a new executor node type named "Result Cache". The planner
can include this node type in the plan to have the executor cache the
results from the inner side of parameterized nested loop joins. This
allows caching of tuples for sets of parameters so that in the event that
the node sees the same parameter values again, it can just return the
cached tuples instead of rescanning the inner side of the join all over
again. Internally, result cache uses a hash table in order to quickly
find tuples that have been previously cached.
For certain data sets, this can significantly improve the performance of
joins. The best cases for using this new node type are for join problems
where a large portion of the tuples from the inner side of the join have
no join partner on the outer side of the join. In such cases, hash join
would have to hash values that are never looked up, thus bloating the hash
table and possibly causing it to multi-batch. Merge joins would have to
skip over all of the unmatched rows. If we use a nested loop join with a
result cache, then we only cache tuples that have at least one join
partner on the outer side of the join. The benefits of using a
parameterized nested loop with a result cache increase when there are
fewer distinct values being looked up and the number of lookups of each
value is large. Also, hash probes to lookup the cache can be much faster
than the hash probe in a hash join as it's common that the result cache's
hash table is much smaller than the hash join's due to result cache only
caching useful tuples rather than all tuples from the inner side of the
join. This variation in hash probe performance is more significant when
the hash join's hash table no longer fits into the CPU's L3 cache, but the
result cache's hash table does. The apparent "random" access of hash
buckets with each hash probe can cause a poor L3 cache hit ratio for large
hash tables. Smaller hash tables generally perform better.
The hash table used for the cache limits itself to not exceeding work_mem
* hash_mem_multiplier in size. We maintain a dlist of keys for this cache
and when we're adding new tuples and realize we've exceeded the memory
budget, we evict cache entries starting with the least recently used ones
until we have enough memory to add the new tuples to the cache.
For parameterized nested loop joins, we now consider using one of these
result cache nodes in between the nested loop node and its inner node. We
determine when this might be useful based on cost, which is primarily
driven off of what the expected cache hit ratio will be. Estimating the
cache hit ratio relies on having good distinct estimates on the nested
loop's parameters.
For now, the planner will only consider using a result cache for
parameterized nested loop joins. This works for both normal joins and
also for LATERAL type joins to subqueries. It is possible to use this new
node for other uses in the future. For example, to cache results from
correlated subqueries. However, that's not done here due to some
difficulties obtaining a distinct estimation on the outer plan to
calculate the estimated cache hit ratio. Currently we plan the inner plan
before planning the outer plan so there is no good way to know if a result
cache would be useful or not since we can't estimate the number of times
the subplan will be called until the outer plan is generated.
The functionality being added here is newly introducing a dependency on
the return value of estimate_num_groups() during the join search.
Previously, during the join search, we only ever needed to perform
selectivity estimations. With this commit, we need to use
estimate_num_groups() in order to estimate what the hit ratio on the
result cache will be. In simple terms, if we expect 10 distinct values
and we expect 1000 outer rows, then we'll estimate the hit ratio to be
99%. Since cache hits are very cheap compared to scanning the underlying
nodes on the inner side of the nested loop join, then this will
significantly reduce the planner's cost for the join. However, it's
fairly easy to see here that things will go bad when estimate_num_groups()
incorrectly returns a value that's significantly lower than the actual
number of distinct values. If this happens then that may cause us to make
use of a nested loop join with a result cache instead of some other join
type, such as a merge or hash join. Our distinct estimations have been
known to be a source of trouble in the past, so the extra reliance on them
here could cause the planner to choose slower plans than it did previous
to having this feature. Distinct estimations are also fairly hard to
estimate accurately when several tables have been joined already or when a
WHERE clause filters out a set of values that are correlated to the
expressions we're estimating the number of distinct value for.
For now, the costing we perform during query planning for result caches
does put quite a bit of faith in the distinct estimations being accurate.
When these are accurate then we should generally see faster execution
times for plans containing a result cache. However, in the real world, we
may find that we need to either change the costings to put less trust in
the distinct estimations being accurate or perhaps even disable this
feature by default. There's always an element of risk when we teach the
query planner to do new tricks that it decides to use that new trick at
the wrong time and causes a regression. Users may opt to get the old
behavior by turning the feature off using the enable_resultcache GUC.
Currently, this is enabled by default. It remains to be seen if we'll
maintain that setting for the release.
Additionally, the name "Result Cache" is the best name I could think of
for this new node at the time I started writing the patch. Nobody seems
to strongly dislike the name. A few people did suggest other names but no
other name seemed to dominate in the brief discussion that there was about
names. Let's allow the beta period to see if the current name pleases
enough people. If there's some consensus on a better name, then we can
change it before the release. Please see the 2nd discussion link below
for the discussion on the "Result Cache" name.
Author: David Rowley
Reviewed-by: Andy Fan, Justin Pryzby, Zhihong Yu
Tested-By: Konstantin Knizhnik
Discussion: https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com
Discussion: https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M src/backend/commands/explain.c
M src/backend/executor/Makefile
M src/backend/executor/execAmi.c
M src/backend/executor/execExpr.c
M src/backend/executor/execParallel.c
M src/backend/executor/execProcnode.c
A src/backend/executor/nodeResultCache.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/executor/executor.h
A src/include/executor/nodeResultCache.h
M src/include/lib/ilist.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/cost.h
M src/include/optimizer/pathnode.h
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/join.out
M src/test/regress/expected/partition_prune.out
A src/test/regress/expected/resultcache.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/sysviews.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/join.sql
M src/test/regress/sql/partition_prune.sql
A src/test/regress/sql/resultcache.sql
Remove setvbuf() call from PQtrace()
commit : 6ec578e60101c3c02533f99715945a0400fb3286
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 31 Mar 2021 20:11:51 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 31 Mar 2021 20:11:51 -0300
It's misplaced there -- it's not libpq's output stream to tweak in that
way. In particular, POSIX says that it has to be called before any
other operation on the file, so if a stream previously used by the
calling application, bad things may happen.
Put setvbuf() in libpq_pipeline for good measure.
Also, reduce fopen(..., "w+") to just fopen(..., "w") in
libpq_pipeline.c. It's not clear that this fixes anything, but we don't
use w+ anywhere.
Per complaints from Tom Lane.
Discussion: https://postgr.es/m/3337422.1617229905@sss.pgh.pa.us
M src/interfaces/libpq/fe-trace.c
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Initialize conn->Pfdebug to NULL when creating a connection
commit : aba24b51cc1b045a9810458b4bb15fee2c182948
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 31 Mar 2021 19:16:58 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 31 Mar 2021 19:16:58 -0300
Failing to do this can cause a crash, and I suspect is what has happened
with a buildfarm member reporting mysterious failures.
This is an ancient bug, but I'm not backpatching since evidently nobody
cares about PQtrace in older releases.
Discussion: https://postgr.es/m/3333908.1617227066@sss.pgh.pa.us
M src/interfaces/libpq/fe-connect.c
Disable force_parallel_mode in libpq_pipeline
commit : a6d3dea8e5e0c8a0df2f95d66b6c3903a4354ca0
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 31 Mar 2021 18:45:17 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 31 Mar 2021 18:45:17 -0300
Some buildfarm animals with force_parallel_mode=regress were failing
this test because the error is reported in a parallel worker quicker
than the rows that succeed.
Take the opportunity to move the SET of lc_messages out of the traced
section, because it's not very interesting.
Diagnosed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/3304521.1617221724@sss.pgh.pa.us
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace
M src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
M src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
M src/test/modules/libpq_pipeline/traces/prepared.trace
M src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
M src/test/modules/libpq_pipeline/traces/singlerow.trace
M src/test/modules/libpq_pipeline/traces/transaction.trace
Fix unportable use of isprint().
commit : 9e20406dd847d0f8c1cbd803786c6d0ad33bcbdd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 17:14:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 17:14:16 -0400
We must cast the arguments of <ctype.h> functions to unsigned
char to avoid problems where char is signed.
Speaking of which, considering that this *is* a <ctype.h> function,
it's rather remarkable that we aren't seeing more complaints about
not having included that header.
Per buildfarm.
M src/interfaces/libpq/fe-trace.c
Fix portability and safety issues in pqTraceFormatTimestamp.
commit : f1be740a991406d7885047beb971e1ff5dbe8b71
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 17:00:30 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 17:00:30 -0400
Remove confusion between time_t and pg_time_t; neither
gettimeofday() nor localtime() deal in the latter.
libpq indeed has no business using <pgtime.h> at all.
Use snprintf not sprintf, to ensure we can't overrun the
supplied buffer. (Unlikely, but let's be safe.)
Per buildfarm.
M src/interfaces/libpq/fe-trace.c
Silence compiler warning in non-assert builds.
commit : 8998e3cafa23632790787b8cc726998e84067259
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 16:50:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 16:50:45 -0400
Per buildfarm.
M contrib/postgres_fdw/postgres_fdw.c
Don't prematurely cram a value into a short int.
commit : c545e9524dcfcfce25c370f584b31562e8d7a4b7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 16:45:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 16:45:17 -0400
Since a4d75c86b, some buildfarm members have been warning that
Assert(attnum <= MaxAttrNumber);
is useless if attnum is an AttrNumber. I'm not certain how plausible
it is that the value coming out of the bitmap could actually exceed
MaxAttrNumber, but we seem to have thought that that was possible back
in 7300a6995. Revert the intermediate variable to int so that we have
the same overflow protection as before.
M src/backend/statistics/extended_stats.c
Add a docs section for obsoleted and renamed functions and settings
commit : 3b0c647bbfc52894d979976f1e6d60e40649bba7
author : Stephen Frost <sfrost@snowman.net>
date : Wed, 31 Mar 2021 16:23:25 -0400
committer: Stephen Frost <sfrost@snowman.net>
date : Wed, 31 Mar 2021 16:23:25 -0400
The new appendix groups information on renamed or removed settings,
commands, etc into an out-of-the-way part of the docs.
The original id elements are retained in each subsection to ensure that
the same filenames are produced for HTML docs. This prevents /current/
links on the web from breaking, and allows users of the web docs
to follow links from old version pages to info on the changes in the
new version. Prior to this change, a link to /current/ for renamed
sections like the recovery.conf docs would just 404. Similarly if
someone searched for recovery.conf they would find the pg11 docs,
but there would be no /12/ or /current/ link, so they couldn't easily
find out that it was removed in pg12 or how to adapt.
Index entries are also added so that there's a breadcrumb trail for
users to follow when they know the old name, but not what we changed it
to. So a user who is trying to find out how to set standby_mode in
PostgreSQL 12+, or where pg_resetxlog went, now has more chance of
finding that information.
Craig Ringer and Stephen Frost
Reviewed-by: Euler Taveira
Discussion: https://postgr.es/m/CAGRY4nzPNOyYQ_1-pWYToUVqQ0ThqP5jdURnJMZPm539fdizOg%40mail.gmail.com
Backpatch-through: 10
A doc/src/sgml/appendix-obsolete-pgreceivexlog.sgml
A doc/src/sgml/appendix-obsolete-pgresetxlog.sgml
A doc/src/sgml/appendix-obsolete-pgxlogdump.sgml
A doc/src/sgml/appendix-obsolete-recovery-config.sgml
A doc/src/sgml/appendix-obsolete.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/filelist.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/postgres.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
Suppress compiler warning in libpq_pipeline.c.
commit : 522d1a89f8d7ed45681988c60bd0a687332a4023
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 15:30:04 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 15:30:04 -0400
Some compilers seem to be concerned about the possibility that
recv_step is not any of the defined enum values. Silence
warnings about uninitialized cmdtag in a different way than
I did in 9fb9691a8.
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Improve style of some replication-related error messages.
commit : 6197db5340b8154adce1c6d07f6d3325547429c1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 15:25:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 15:25:53 -0400
Put the remote end's error message into the primary error string,
instead of relegating it to errdetail(). Although this could end up
being awkward if the remote sends us a really long error message,
it seems more in keeping with our message style guidelines, and more
helpful in situations where the errdetail could get dropped.
Peter Smith
Discussion: https://postgr.es/m/CAHut+Ps-Qv2yQceCwobQDP0aJOkfDzRFrOaR6+2Op2K=WHGeWg@mail.gmail.com
M src/backend/commands/subscriptioncmds.c
M src/backend/replication/logical/tablesync.c
Fix some libpq_pipeline test problems
commit : db973ffb3ca43e65a0bf15175a35184a53bf977d
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 31 Mar 2021 15:13:42 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 31 Mar 2021 15:13:42 -0300
Test pipeline_abort was not checking that it got the rows it expected in
one mode; make it do so. This doesn't fix the actual problem (no idea
what that is, yet) but at least it should make it more obvious rather
than being visible only as a difference in the trace output.
While at it, fix other infelicities in the test:
* I reversed the order of result vs. expected in like().
* The output traces from -t are being put in the log dir, which means
the buildfarm script uselessly captures them. Put them in a separate
dir tmp_check/traces instead, to avoid cluttering the buildfarm results.
* Test pipelined_insert was using too large a row count. Reduce that a
tad and add a filler column to make each insert a little bulkier, while
still keeping enough that a buffer is filled and we have to switch mode.
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
M src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
Fix has_column_privilege function corner case
commit : b12bd4869b5e64b742a69ca07915e2f77f85a9ae
author : Joe Conway <mail@joeconway.com>
date : Wed, 31 Mar 2021 13:55:25 -0400
committer: Joe Conway <mail@joeconway.com>
date : Wed, 31 Mar 2021 13:55:25 -0400
According to the comments, when an invalid or dropped column oid is passed
to has_column_privilege(), the intention has always been to return NULL.
However, when the caller had table level privilege the invalid/missing
column was never discovered, because table permissions were checked first.
Fix that by introducing extended versions of pg_attribute_acl(check|mask)
and pg_class_acl(check|mask) which take a new argument, is_missing. When
is_missing is NULL, the old behavior is preserved. But when is_missing is
passed by the caller, no ERROR is thrown for dropped or missing
columns/relations, and is_missing is flipped to true. This in turn allows
has_column_privilege to check for column privileges first, providing the
desired semantics.
Not backpatched since it is a user visible behavioral change with no previous
complaints, and the fix is a bit on the invasive side.
Author: Joe Conway
Reviewed-By: Tom Lane
Reported by: Ian Barwick
Discussion: https://postgr.es/m/flat/9b5f4311-157b-4164-7fe7-077b4fe8ed84%40joeconway.com
M src/backend/catalog/aclchk.c
M src/backend/utils/adt/acl.c
M src/include/utils/acl.h
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Rework planning and execution of UPDATE and DELETE.
commit : 86dc90056dfdbd9d1b891718d2e5614e3e432f35
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 11:52:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Mar 2021 11:52:34 -0400
This patch makes two closely related sets of changes:
1. For UPDATE, the subplan of the ModifyTable node now only delivers
the new values of the changed columns (i.e., the expressions computed
in the query's SET clause) plus row identity information such as CTID.
ModifyTable must re-fetch the original tuple to merge in the old
values of any unchanged columns. The core advantage of this is that
the changed columns are uniform across all tables of an inherited or
partitioned target relation, whereas the other columns might not be.
A secondary advantage, when the UPDATE involves joins, is that less
data needs to pass through the plan tree. The disadvantage of course
is an extra fetch of each tuple to be updated. However, that seems to
be very nearly free in context; even worst-case tests don't show it to
add more than a couple percent to the total query cost. At some point
it might be interesting to combine the re-fetch with the tuple access
that ModifyTable must do anyway to mark the old tuple dead; but that
would require a good deal of refactoring and it seems it wouldn't buy
all that much, so this patch doesn't attempt it.
2. For inherited UPDATE/DELETE, instead of generating a separate
subplan for each target relation, we now generate a single subplan
that is just exactly like a SELECT's plan, then stick ModifyTable
on top of that. To let ModifyTable know which target relation a
given incoming row refers to, a tableoid junk column is added to
the row identity information. This gets rid of the horrid hack
that was inheritance_planner(), eliminating O(N^2) planning cost
and memory consumption in cases where there were many unprunable
target relations.
Point 2 of course requires point 1, so that there is a uniform
definition of the non-junk columns to be returned by the subplan.
We can't insist on uniform definition of the row identity junk
columns however, if we want to keep the ability to have both
plain and foreign tables in a partitioning hierarchy. Since
it wouldn't scale very far to have every child table have its
own row identity column, this patch includes provisions to merge
similar row identity columns into one column of the subplan result.
In particular, we can merge the whole-row Vars typically used as
row identity by FDWs into one column by pretending they are type
RECORD. (It's still okay for the actual composite Datums to be
labeled with the table's rowtype OID, though.)
There is more that can be done to file down residual inefficiencies
in this patch, but it seems to be committable now.
FDW authors should note several API changes:
* The argument list for AddForeignUpdateTargets() has changed, and so
has the method it must use for adding junk columns to the query. Call
add_row_identity_var() instead of manipulating the parse tree directly.
You might want to reconsider exactly what you're adding, too.
* PlanDirectModify() must now work a little harder to find the
ForeignScan plan node; if the foreign table is part of a partitioning
hierarchy then the ForeignScan might not be the direct child of
ModifyTable. See postgres_fdw for sample code.
* To check whether a relation is a target relation, it's no
longer sufficient to compare its relid to root->parse->resultRelation.
Instead, check it against all_result_relids or leaf_result_relids,
as appropriate.
Amit Langote and Tom Lane
Discussion: https://postgr.es/m/CA+HiwqHpHdqdDn48yCEhynnniahH78rwcrv1rEX65-fsZGBOLQ@mail.gmail.com
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M doc/src/sgml/ddl.sgml
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/postgres-fdw.sgml
M src/backend/commands/copyfrom.c
M src/backend/commands/explain.c
M src/backend/commands/trigger.c
M src/backend/executor/README
M src/backend/executor/execExpr.c
M src/backend/executor/execJunk.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/planmain.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/optimizer/prep/preptlist.c
M src/backend/optimizer/util/appendinfo.c
M src/backend/optimizer/util/inherit.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/plancat.c
M src/backend/optimizer/util/relnode.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/utils/adt/ruleutils.c
M src/include/executor/executor.h
M src/include/foreign/fdwapi.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/nodes/primnodes.h
M src/include/optimizer/appendinfo.h
M src/include/optimizer/pathnode.h
M src/include/optimizer/prep.h
M src/include/rewrite/rewriteHandler.h
M src/test/regress/expected/inherit.out
M src/test/regress/expected/insert_conflict.out
M src/test/regress/expected/partition_join.out
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/rowsecurity.out
M src/test/regress/expected/updatable_views.out
M src/test/regress/expected/update.out
M src/test/regress/expected/with.out
Allow an alias to be attached to a JOIN ... USING
commit : 055fee7eb4dcc78e58672aef146334275e1cc40d
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 31 Mar 2021 17:09:24 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 31 Mar 2021 17:09:24 +0200
This allows something like
SELECT ... FROM t1 JOIN t2 USING (a, b, c) AS x
where x has the columns a, b, c and unlike a regular alias it does not
hide the range variables of the tables being joined t1 and t2.
Per SQL:2016 feature F404 "Range variable for common column names".
Reviewed-by: Vik Fearing <vik.fearing@2ndquadrant.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/454638cf-d563-ab76-a585-2564428062af@2ndquadrant.com
M doc/src/sgml/ref/select.sgml
M src/backend/catalog/sql_features.txt
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_relation.c
M src/backend/utils/adt/ruleutils.c
M src/include/nodes/parsenodes.h
M src/include/nodes/primnodes.h
M src/include/parser/parse_node.h
M src/include/parser/parse_relation.h
M src/test/regress/expected/create_view.out
M src/test/regress/expected/join.out
M src/test/regress/sql/create_view.sql
M src/test/regress/sql/join.sql
Add support for asynchronous execution.
commit : 27e1f14563cf982f1f4d71e21ef247866662a052
author : Etsuro Fujita <efujita@postgresql.org>
date : Wed, 31 Mar 2021 18:45:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Wed, 31 Mar 2021 18:45:00 +0900
This implements asynchronous execution, which runs multiple parts of a
non-parallel-aware Append concurrently rather than serially to improve
performance when possible. Currently, the only node type that can be
run concurrently is a ForeignScan that is an immediate child of such an
Append. In the case where such ForeignScans access data on different
remote servers, this would run those ForeignScans concurrently, and
overlap the remote operations to be performed simultaneously, so it'll
improve the performance especially when the operations involve
time-consuming ones such as remote join and remote aggregation.
We may extend this to other node types such as joins or aggregates over
ForeignScans in the future.
This also adds the support for postgres_fdw, which is enabled by the
table-level/server-level option "async_capable". The default is false.
Robert Haas, Kyotaro Horiguchi, Thomas Munro, and myself. This commit
is mostly based on the patch proposed by Robert Haas, but also uses
stuff from the patch proposed by Kyotaro Horiguchi and from the patch
proposed by Thomas Munro. Reviewed by Kyotaro Horiguchi, Konstantin
Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro, Justin Pryzby, and
others.
Discussion: https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com
Discussion: https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/postgres-fdw.sgml
M src/backend/commands/explain.c
M src/backend/executor/Makefile
M src/backend/executor/README
M src/backend/executor/execAmi.c
A src/backend/executor/execAsync.c
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeForeignscan.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/createplan.c
M src/backend/postmaster/pgstat.c
M src/backend/storage/ipc/latch.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
A src/include/executor/execAsync.h
M src/include/executor/nodeAppend.h
M src/include/executor/nodeForeignscan.h
M src/include/foreign/fdwapi.h
M src/include/nodes/execnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/cost.h
M src/include/pgstat.h
M src/include/storage/latch.h
M src/test/regress/expected/explain.out
M src/test/regress/expected/incremental_sort.out
M src/test/regress/expected/insert_conflict.out
M src/test/regress/expected/sysviews.out
Add p_names field to ParseNamespaceItem
commit : 66392d396508c91c2ec07a61568bf96acb663ad8
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 31 Mar 2021 10:52:37 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 31 Mar 2021 10:52:37 +0200
ParseNamespaceItem had a wired-in assumption that p_rte->eref
describes the table and column aliases exposed by the nsitem. This
relaxes this by creating a separate p_names field in an nsitem. This
is mainly preparation for a patch for JOIN USING aliases, but it saves
one indirection in common code paths, so it's possibly a win on its
own.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/785329.1616455091@sss.pgh.pa.us
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_relation.c
M src/include/parser/parse_node.h
Add errhint_plural() function and make use of it
commit : 91c5a8caaa61055959aa5fb68a00e5f690e39a34
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 31 Mar 2021 09:15:51 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 31 Mar 2021 09:15:51 +0200
Similar to existing errmsg_plural() and errdetail_plural(). Some
errhint() calls hadn't received the proper plural treatment yet.
M doc/src/sgml/sources.sgml
M src/backend/parser/parse_func.c
M src/backend/utils/error/elog.c
M src/include/utils/elog.h
M src/nls-global.mk
doc: Remove Cyrillic from unistr example
commit : 287d2a97c1de07486e4525c8ad06258f04bd6268
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 31 Mar 2021 08:20:35 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 31 Mar 2021 08:20:35 +0200
Not supported by PDF build right now, so let's do without it.
M doc/src/sgml/func.sgml
Remove extra semicolon in postgres_fdw tests.
commit : 13cb5bd84657ed49021fe6fc4ce46601c315c9a5
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 31 Mar 2021 10:36:39 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 31 Mar 2021 10:36:39 +0530
Author: Suraj Kharage
Reviewed-by: Bharath Rupireddy, Vignesh C
Discussion: https://postgr.es/m/CAF1DzPWRfxUeH-wShz7P_pK5Tx6M_nEK+TkS8gn5ngvg07Q5=g@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Doc: Use consistent terminology for tablesync slots.
commit : 9f45631766bd0c51a74102770737ba3b0561977e
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 31 Mar 2021 08:17:50 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 31 Mar 2021 08:17:50 +0530
At some places in the docs, we refer to them as tablesync slots and at other
places as table synchronization slots. For consistency, we refer to them as
table synchronization slots at all places.
Author: Peter Smith
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAHut+PvzYNKCeZ=kKBDkh3dw-r=2D3fk=nNc9SXSW=CZGk69xg@mail.gmail.com
M doc/src/sgml/ref/alter_subscription.sgml
Accept slightly-filled pages for tuples larger than fillfactor.
commit : 0ff8bbdee19a9db2794a95d439c946ab017d0acd
author : Noah Misch <noah@leadboat.com>
date : Tue, 30 Mar 2021 18:53:44 -0700
committer: Noah Misch <noah@leadboat.com>
date : Tue, 30 Mar 2021 18:53:44 -0700
We always inserted a larger-than-fillfactor tuple into a newly-extended
page, even when existing pages were empty or contained nothing but an
unused line pointer. This was unnecessary relation extension. Start
tolerating page usage up to 1/8 the maximum space that could be taken up
by line pointers. This is somewhat arbitrary, but it should allow more
cases to reuse pages. This has no effect on tables with fillfactor=100
(the default).
John Naylor and Floris van Nee. Reviewed by Matthias van de Meent.
Reported by Floris van Nee.
Discussion: https://postgr.es/m/6e263217180649339720afe2176c50aa@opammb0562.comp.optiver.com
M src/backend/access/heap/hio.c
M src/backend/access/heap/rewriteheap.c
M src/test/regress/expected/insert.out
M src/test/regress/sql/insert.sql
Fix comment in parsenodes.h
commit : 7ef64e7e72a65f191fc2f7d4bbe220f53dd8d5de
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 31 Mar 2021 09:35:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 31 Mar 2021 09:35:58 +0900
CreateStmt->inhRelations is a list of RangeVars, but a comment was
incorrect about that.
Author: Julien Rouhaud
Discussion: https://postgr.es/m/20210330123015.yzekhz5sweqbgxdr@nol
M src/include/nodes/parsenodes.h
Add support for --extension in pg_dump
commit : 6568cef26e0f40c25ae54b8e20aad8d1410a854b
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 31 Mar 2021 09:12:34 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 31 Mar 2021 09:12:34 +0900
When specified, only extensions matching the given pattern are included
in dumps. Similarly to --table and --schema, when --strict-names is
used, a perfect match is required. Also, like the two other options,
this new option offers no guarantee that dependent objects have been
dumped, so a restore may fail on a clean database.
Tests are added in test_pg_dump/, checking after a set of positive and
negative cases, with or without an extension's contents added to the
dump generated.
Author: Guillaume Lelarge
Reviewed-by: David Fetter, Tom Lane, Michael Paquier, Asif Rehman,
Julien Rouhaud
Discussion: https://postgr.es/m/CAECtzeXOt4cnMU5+XMZzxBPJ_wu76pNy6HZKPRBL-j7yj1E4+g@mail.gmail.com
M doc/src/sgml/ref/pg_dump.sgml
M src/bin/pg_dump/pg_dump.c
M src/test/modules/test_pg_dump/t/001_base.pl
Remove small inefficiency in ExecARDeleteTriggers/ExecARUpdateTriggers.
commit : 65158f497a7d7523ad438b2034d01a560fafe6bd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 30 Mar 2021 20:01:27 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 30 Mar 2021 20:01:27 -0400
Whilst poking at nodeModifyTable.c, I chanced to notice that while
its calls to ExecBR*Triggers and ExecIR*Triggers are protected by
tests to see if there are any relevant triggers to fire, its calls
to ExecAR*Triggers are not; the latter functions do the equivalent
tests themselves. This seems possibly reasonable given the more
complex conditions involved, but what's less reasonable is that
the ExecAR* functions aren't careful to do no work when there is
no work to be done. ExecARInsertTriggers gets this right, but the
other two will both force creation of a slot that the query may
have no use for. ExecARUpdateTriggers additionally performed a
usually-useless ExecClearTuple() on that slot. This is probably
all pretty microscopic in real workloads, but a cycle shaved is a
cycle earned.
M src/backend/commands/trigger.c
adjust dblink regression expected output for commit 5da9868ed9
commit : 9ee7d533dacf8594057ced2d016250f09056c284
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 30 Mar 2021 19:46:22 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 30 Mar 2021 19:46:22 -0400
Seems the -1/singular output is used in the dblink regression tests.
Reported-by: Álvaro Herrera
Discussion: https://postgr.es/m/20210330231506.GA10666@alvherre.pgsql
M contrib/dblink/expected/dblink.out
libpq_pipeline: add PQtrace() support and tests
commit : 7bebd0d00998a28449d83376f4bcdeec65d5eea6
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 30 Mar 2021 20:33:04 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 30 Mar 2021 20:33:04 -0300
The libpq_pipeline program recently introduced by commit acb7e4eb6b1c
is well equipped to test the PQtrace() functionality, so let's make it
do that.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20210327192812.GA25115@alvherre.pgsql
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
A src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace
A src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
A src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
A src/test/modules/libpq_pipeline/traces/prepared.trace
A src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
A src/test/modules/libpq_pipeline/traces/singlerow.trace
A src/test/modules/libpq_pipeline/traces/transaction.trace
Improve PQtrace() output format
commit : 198b3716dba68544b55cb97bd120738a86d5df2d
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 30 Mar 2021 20:12:34 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 30 Mar 2021 20:12:34 -0300
Transform the PQtrace output format from its ancient (and mostly
useless) byte-level output format to a logical-message-level output,
making it much more usable. This implementation allows the printing
code to be written (as it indeed was) by looking at the protocol
documentation, which gives more confidence that the three (docs, trace
code and actual code) actually match.
Author: 岩田 彩 (Aya Iwata) <iwata.aya@fujitsu.com>
Reviewed-by: 綱川 貴之 (Takayuki Tsunakawa) <tsunakawa.takay@fujitsu.com>
Reviewed-by: Kirk Jamison <k.jamison@fujitsu.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: 黒田 隼人 (Hayato Kuroda) <kuroda.hayato@fujitsu.com>
Reviewed-by: "Nagaura, Ryohei" <nagaura.ryohei@jp.fujitsu.com>
Reviewed-by: Ryo Matsumura <matsumura.ryo@fujitsu.com>
Reviewed-by: Greg Nancarrow <gregn4422@gmail.com>
Reviewed-by: Jim Doty <jdoty@pivotal.io>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/71E660EB361DF14299875B198D4CE5423DE3FBA4@g01jpexmbkw25
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/Makefile
M src/interfaces/libpq/exports.txt
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-misc.c
M src/interfaces/libpq/fe-protocol3.c
A src/interfaces/libpq/fe-trace.c
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
In messages, use singular nouns for -1, like we do for +1.
commit : 5da9868ed983f95cc1cff80dcd81252a513774f8
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 30 Mar 2021 18:34:27 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 30 Mar 2021 18:34:27 -0400
This outputs "-1 year", not "-1 years".
Reported-by: neverov.max@gmail.com
Bug: 16939
Discussion: https://postgr.es/m/16939-cceeb03fb72736ee@postgresql.org
M src/backend/utils/adt/datetime.c
M src/interfaces/ecpg/pgtypeslib/interval.c
M src/interfaces/libpq/fe-print.c
M src/test/regress/expected/interval.out
Add tests for date_part of epoch near upper bound of timestamp range
commit : 6131ffc43ff3d2f566e93f017e56a09e4e717318
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 30 Mar 2021 22:05:18 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 30 Mar 2021 22:05:18 +0200
This exercises a special case in the implementations of
date_part('epoch', timestamp[tz]) that was previously not tested.
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Use a WaitLatch for vacuum/autovacuum sleeping
commit : 4753ef37e0eda4ba0af614022d18fcbc5a946cc9
author : Stephen Frost <sfrost@snowman.net>
date : Tue, 30 Mar 2021 12:52:56 -0400
committer: Stephen Frost <sfrost@snowman.net>
date : Tue, 30 Mar 2021 12:52:56 -0400
Instead of using pg_usleep() in vacuum_delay_point(), use a WaitLatch.
This has the advantage that we will realize if the postmaster has been
killed since the last time we decided to sleep while vacuuming.
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/CAFh8B=kcdk8k-Y21RfXPu5dX=bgPqJ8TC3p_qxR_ygdBS=JN5w@mail.gmail.com
M src/backend/commands/vacuum.c
Further tweaking of pg_dump's handling of default_toast_compression.
commit : 54bb91c30e3964fd81059e6b02e377cc9dd2d64c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 30 Mar 2021 10:57:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 30 Mar 2021 10:57:57 -0400
As committed in bbe0a81db, pg_dump from a pre-v14 server effectively
acts as though you'd said --no-toast-compression. I think the right
thing is for it to act as though default_toast_compression is set to
"pglz", instead, so that the tables' toast compression behavior is
preserved. You can always get the other behavior, if you want that,
by giving the switch.
Discussion: https://postgr.es/m/1112852.1616609702@sss.pgh.pa.us
M src/bin/pg_dump/pg_dump.c
Allow estimate_num_groups() to pass back further details about the estimation
commit : ed934d4fa30f0f94e6f7125ad2154e6a58d1c7f7
author : David Rowley <drowley@postgresql.org>
date : Tue, 30 Mar 2021 20:52:46 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 30 Mar 2021 20:52:46 +1300
Here we add a new output parameter to estimate_num_groups() to allow it to
inform the caller of additional, possibly useful information about the
estimation.
The new output parameter is a struct that currently contains just a single
field with a set of flags. This was done rather than having the flags as
an output parameter to allow future fields to be added without having to
change the signature of the function at a later date when we want to pass
back further information that might not be suitable to store in the flags
field.
It seems reasonable that one day in the future that the planner would want
to know more about the estimation. For example, how many individual sets
of statistics was the estimation generated from? The planner may want to
take that into account if we ever want to consider risks as well as costs
when generating plans.
For now, there's only 1 flag we set in the flags field. This is to
indicate if the estimation fell back on using the hard-coded constants in
any part of the estimation. Callers may like to change their behavior if
this is set, and this gives them the ability to do so. Callers may pass
the flag pointer as NULL if they have no interest in obtaining any
additional information about the estimate.
We're not adding any actual usages of these flags here. Some follow-up
commits will make use of this feature. Additionally, we're also not
making any changes to add support for clauselist_selectivity() and
clauselist_selectivity_ext(). However, if this is required in the future
then the same struct being added here should be fine to use as a new
output argument for those functions too.
Author: David Rowley
Discussion: https://postgr.es/m/CAApHDvqQqpk=1W-G_ds7A9CsXX3BggWj_7okinzkLVhDubQzjA@mail.gmail.com
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/pathnode.c
M src/backend/utils/adt/selfuncs.c
M src/include/utils/selfuncs.h
Fix compiler warning in unistr function
commit : efd9d92bb39c74c2aded64fc08e2d601ce20c39d
author : David Rowley <drowley@postgresql.org>
date : Tue, 30 Mar 2021 20:28:09 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 30 Mar 2021 20:28:09 +1300
Some compilers are not aware that elog/ereport ERROR does not return.
M src/backend/utils/adt/varlena.c
Allow users of simplehash.h to perform direct deletions
commit : ff53d7b159b93ce9fc884897f9d96b97744781e2
author : David Rowley <drowley@postgresql.org>
date : Tue, 30 Mar 2021 19:56:50 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 30 Mar 2021 19:56:50 +1300
Previously simplehash.h only exposed a method to perform a hash table
delete using the hash table key. This meant that the delete function had
to perform a hash lookup in order to find the entry to delete. Here we
add a new function so that users of simplehash.h can perform a hash delete
directly using the entry pointer, thus saving the hash lookup.
An upcoming patch that uses simplehash.h already has performed the hash
lookup so already has the entry pointer. This change will allow the
code in that patch to perform the hash delete without the code in
simplehash.h having to perform an additional hash lookup.
Author: David Rowley
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/CAApHDvqFLXXge153WmPsjke5VGOSt7Ez0yD0c7eBXLfmWxs3Kw@mail.gmail.com
M src/include/lib/simplehash.h
Add upper boundary tests for timestamp and timestamptz types
commit : bc9f1afdebc98b490d0a00468d75e8e4d080afb0
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 30 Mar 2021 08:46:34 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 30 Mar 2021 08:46:34 +0200
The existing regression tests only tested the lower boundary of the
range supported by the timestamp and timestamptz types because "The
upper boundary differs between integer and float timestamps, so no
check". Since this is obsolete, add similar tests for the upper
boundary.
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Add a xid argument to the filter_prepare callback for output plugins.
commit : f64ea6dc5c8ccaec9a3d3d39695ca261febb6883
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 30 Mar 2021 10:34:43 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 30 Mar 2021 10:34:43 +0530
Along with gid, this provides a different way to identify the transaction.
The users that use xid in some way to prepare the transactions can use it
to filter prepare transactions. The later commands COMMIT PREPARED or
ROLLBACK PREPARED carries both identifiers, providing an output plugin the
choice of what to use.
Author: Markus Wanner
Reviewed-by: Vignesh C, Amit Kapila
Discussion: https://postgr.es/m/ee280000-7355-c4dc-e47b-2436e7be959c@enterprisedb.com
M contrib/test_decoding/test_decoding.c
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/logical.c
M src/include/replication/logical.h
M src/include/replication/output_plugin.h
Update obsolete comment.
commit : bc2797ebb14bae663da1ee7845774dd98716c0d0
author : Etsuro Fujita <efujita@postgresql.org>
date : Tue, 30 Mar 2021 13:00:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Tue, 30 Mar 2021 13:00:00 +0900
Back-patch to all supported branches.
Author: Etsuro Fujita
Discussion: https://postgr.es/m/CAPmGK17DwzaSf%2BB71dhL2apXdtG-OmD6u2AL9Cq2ZmAR0%2BzapQ%40mail.gmail.com
M contrib/postgres_fdw/postgres_fdw.c
psql: call clearerr() just before printing
commit : 8d645a116ef6e04bfb03e259149b8e163dbdf50c
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 29 Mar 2021 18:34:39 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 29 Mar 2021 18:34:39 -0300
We were never doing clearerr() on the output stream, which results in a
message being printed after each result once an EOF is seen:
could not print result table: Success
This message was added by commit b03436994bcc (in the pg13 era); before
that, the error indicator would never be examined. So backpatch only
that far back, even though the actual bug (to wit: the fact that the
error indicator is never cleared) is older.
M src/fe_utils/print.c
Adjust design of per-worker parallel seqscan data struct
commit : af527705edc3fd0b335264d17e0521c05edc5cca
author : David Rowley <drowley@postgresql.org>
date : Tue, 30 Mar 2021 10:17:09 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 30 Mar 2021 10:17:09 +1300
The design of the data structures which allow storage of the per-worker
memory during parallel seq scans were not ideal. The work done in
56788d215 required an additional data structure to allow workers to
remember the range of pages that had been allocated to them for
processing during a parallel seqscan. That commit added a void pointer
field to TableScanDescData to allow heapam to store the per-worker
allocation information. However putting the field there made very little
sense given that we have AM specific structs for that, e.g.
HeapScanDescData.
Here we remove the void pointer field from TableScanDescData and add a
dedicated field for this purpose to HeapScanDescData.
Previously we also allocated memory for this parallel per-worker data for
all scans, regardless if it was a parallel scan or not. This was just a
wasted allocation for non-parallel scans, so here we make the allocation
conditional on the scan being parallel.
Also, add previously missing pfree() to free the per-worker data in
heap_endscan().
Reported-by: Andres Freund
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/20210317023101.anvejcfotwka6gaa@alap3.anarazel.de
M src/backend/access/heap/heapam.c
M src/include/access/heapam.h
M src/include/access/relscan.h
Allow matching the DN of a client certificate for authentication
commit : 6d7a6feac48b1970c4cd127ee65d4c487acbb5e9
author : Andrew Dunstan <andrew@dunslane.net>
date : Mon, 29 Mar 2021 15:31:22 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Mon, 29 Mar 2021 15:31:22 -0400
Currently we only recognize the Common Name (CN) of a certificate's
subject to be matched against the user name. Thus certificates with
subjects '/OU=eng/CN=fred' and '/OU=sales/CN=fred' will have the same
connection rights. This patch provides an option to match the whole
Distinguished Name (DN) instead of just the CN. On any hba line using
client certificate identity, there is an option 'clientname' which can
have values of 'DN' or 'CN'. The default is 'CN', the current procedure.
The DN is matched against the RFC2253 formatted DN, which looks like
'CN=fred,OU=eng'.
This facility of probably best used in conjunction with an ident map.
Discussion: https://postgr.es/m/92e70110-9273-d93c-5913-0bccb6562740@dunslane.net
Reviewed-By: Michael Paquier, Daniel Gustafsson, Jacob Champion
M doc/src/sgml/client-auth.sgml
M src/backend/libpq/auth.c
M src/backend/libpq/be-secure-openssl.c
M src/backend/libpq/be-secure.c
M src/backend/libpq/hba.c
M src/include/libpq/hba.h
M src/include/libpq/libpq-be.h
M src/test/ssl/Makefile
A src/test/ssl/client-dn.config
A src/test/ssl/ssl/client-dn.crt
A src/test/ssl/ssl/client-dn.key
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/SSLServer.pm
Clean up date_part tests a bit
commit : efcc7572f532ea564fedc6359c2df43045ee7908
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 29 Mar 2021 17:53:30 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 29 Mar 2021 17:53:30 +0200
Some tests for timestamp and timestamptz were in the date.sql test
file. Move them to their appropriate files, or drop tests cases that
were already present there.
M src/test/regress/expected/date.out
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/date.sql
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Add unistr function
commit : f37fec837ce8bf7af408ba66d32099e5a0182402
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 28 Mar 2021 08:16:15 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 28 Mar 2021 08:16:15 +0200
This allows decoding a string with Unicode escape sequences. It is
similar to Unicode escape strings, but offers some more flexibility.
Author: Pavel Stehule <pavel.stehule@gmail.com>
Reviewed-by: Asif Rehman <asifr.rehman@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAFj8pRA5GnKT+gDVwbVRH2ep451H_myBt+NTz8RkYUARE9+qOQ@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/varlena.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
Reset standard_conforming_strings in strings test
commit : ebedd0c78fc51c293abe56e99a18c67af14da0c9
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 29 Mar 2021 08:40:39 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 29 Mar 2021 08:40:39 +0200
After some tests relating to standard_conforming_strings behavior, the
value was not reset to the default value. Therefore, the rest of the
tests in that file ran with the nondefault setting, which affected the
results of some tests. For clarity, reset the value and run the rest
of the tests with the default setting again.
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
PageAddItemExtended(): Add LP_UNUSED assertion.
commit : 30aaab26e52144097a1a5bbb0bb66ea1ebc0cb81
author : Peter Geoghegan <pg@bowt.ie>
date : Sun, 28 Mar 2021 20:10:02 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Sun, 28 Mar 2021 20:10:02 -0700
Assert that LP_UNUSED items have no storage. If it's worth having
defensive code in non-assert builds then it's worth having an assertion
as well.
M src/backend/storage/page/bufpage.c
Cache if PathTarget and RestrictInfos contain volatile functions
commit : f58b230ed0dba2a3d396794a2ec84541e321d92d
author : David Rowley <drowley@postgresql.org>
date : Mon, 29 Mar 2021 14:47:05 +1300
committer: David Rowley <drowley@postgresql.org>
date : Mon, 29 Mar 2021 14:47:05 +1300
Here we aim to reduce duplicate work done by contain_volatile_functions()
by caching whether PathTargets and RestrictInfos contain any volatile
functions the first time contain_volatile_functions() is called for them.
Any future calls for these nodes just use the cached value rather than
going to the trouble of recursively checking the sub-node all over again.
Thanks to Tom Lane for the idea.
Any locations in the code which make changes to a PathTarget or
RestrictInfo which could change the outcome of the volatility check must
change the cached value back to VOLATILITY_UNKNOWN again.
contain_volatile_functions() is the only code in charge of setting the
cache value to either VOLATILITY_VOLATILE or VOLATILITY_NOVOLATILE.
Some existing code does benefit from this additional caching, however,
this change is mainly aimed at an upcoming patch that must check for
volatility during the join search. Repeated volatility checks in that
case can become very expensive when the join search contains more than a
few relations.
Author: David Rowley
Discussion: https://postgr.es/m/3795226.1614059027@sss.pgh.pa.us
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/util/clauses.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/optimizer/util/tlist.c
M src/include/nodes/pathnodes.h
doc: Define TLS as an acronym
commit : b64654d6c450eb9fb04c6e3456915790510af482
author : Stephen Frost <sfrost@snowman.net>
date : Sun, 28 Mar 2021 11:27:59 -0400
committer: Stephen Frost <sfrost@snowman.net>
date : Sun, 28 Mar 2021 11:27:59 -0400
Commit c6763156589 added an acronym reference for "TLS" but the definition
was never added.
Author: Daniel Gustafsson
Reviewed-by: Michael Paquier
Backpatch-through: 9.6
Discussion: https://postgr.es/m/27109504-82DB-41A8-8E63-C0498314F5B0@yesql.se
M doc/src/sgml/acronyms.sgml
Stabilize stats_ext test with other collations
commit : 2a058e938c73bfb85bbc9fa93dea74788043ca6c
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 27 Mar 2021 18:26:52 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 27 Mar 2021 18:26:52 +0100
The tests used string concatenation to test statistics on expressions,
but that made the tests locale-dependent, e.g. because the ordering of
'11' and '1X' depends on the collation. This affected both the estimated
and actual row couts, breaking some of the tests.
Fixed by replacing the string concatenation with upper() function call,
so that the text values contain only digits.
Discussion: https://postgr.es/m/b650920b-2767-fbc3-c87a-cb8b5d693cbf%40enterprisedb.com
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Improve consistency of SQL code capitalization
commit : 8df2f371141ea267627364cd00e1791054d82d7e
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 27 Mar 2021 10:17:12 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 27 Mar 2021 10:17:12 +0100
M src/backend/catalog/system_views.sql
M src/bin/initdb/initdb.c
Extended statistics on expressions
commit : a4d75c86bf15220df22de0a92c819ecef9db3849
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 23:22:01 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 23:22:01 +0100
Allow defining extended statistics on expressions, not just just on
simple column references. With this commit, expressions are supported
by all existing extended statistics kinds, improving the same types of
estimates. A simple example may look like this:
CREATE TABLE t (a int);
CREATE STATISTICS s ON mod(a,10), mod(a,20) FROM t;
ANALYZE t;
The collected statistics are useful e.g. to estimate queries with those
expressions in WHERE or GROUP BY clauses:
SELECT * FROM t WHERE mod(a,10) = 0 AND mod(a,20) = 0;
SELECT 1 FROM t GROUP BY mod(a,10), mod(a,20);
This introduces new internal statistics kind 'e' (expressions) which is
built automatically when the statistics object definition includes any
expressions. This represents single-expression statistics, as if there
was an expression index (but without the index maintenance overhead).
The statistics is stored in pg_statistics_ext_data as an array of
composite types, which is possible thanks to 79f6a942bd.
CREATE STATISTICS allows building statistics on a single expression, in
which case in which case it's not possible to specify statistics kinds.
A new system view pg_stats_ext_exprs can be used to display expression
statistics, similarly to pg_stats and pg_stats_ext views.
ALTER TABLE ... ALTER COLUMN ... TYPE now treats indexes the same way it
treats indexes, i.e. it drops and recreates the statistics. This means
all statistics are reset, and we no longer try to preserve at least the
functional dependencies. This should not be a major issue in practice,
as the functional dependencies actually rely on per-column statistics,
which were always reset anyway.
Author: Tomas Vondra
Reviewed-by: Justin Pryzby, Dean Rasheed, Zhihong Yu
Discussion: https://postgr.es/m/ad7891d2-e90c-b446-9fe2-7419143847d7%40enterprisedb.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/create_statistics.sgml
M src/backend/catalog/Makefile
M src/backend/catalog/system_views.sql
M src/backend/commands/statscmds.c
M src/backend/commands/tablecmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/util/plancat.c
M src/backend/parser/gram.y
M src/backend/parser/parse_agg.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_func.c
M src/backend/parser/parse_utilcmd.c
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/statistics/mcv.c
M src/backend/statistics/mvdistinct.c
M src/backend/tcop/utility.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/adt/selfuncs.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/commands/defrem.h
M src/include/nodes/nodes.h
M src/include/nodes/parsenodes.h
M src/include/nodes/pathnodes.h
M src/include/parser/parse_node.h
M src/include/parser/parse_utilcmd.h
M src/include/statistics/extended_stats_internal.h
M src/include/statistics/statistics.h
M src/include/utils/ruleutils.h
M src/test/regress/expected/create_table_like.out
M src/test/regress/expected/oidjoins.out
M src/test/regress/expected/rules.out
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/create_table_like.sql
M src/test/regress/sql/stats_ext.sql
Reduce duration of stats_ext regression tests
commit : 98376c18f12e562421b5c77e619248e8b7aae3c6
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 23:00:41 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 23:00:41 +0100
The regression tests of extended statistics were taking a fair amount of
time, due to using fairly large data sets with a couple thousand rows.
So far this was fine, but with tests for statistics on expressions the
duration would get a bit excessive. So reduce the size of some of the
tests that will be used to test expressions, to keep the duration under
control. Done in a separate commit before adding the statistics on
expressions, to make it clear which estimates are expected to change.
Author: Tomas Vondra
Discussion: https://postgr.es/m/ad7891d2-e90c-b446-9fe2-7419143847d7%40enterprisedb.com
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Fix ndistinct estimates with system attributes
commit : 33e52ad9a32929a6d14dfd98a8440d57028f2e3e
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 22:34:53 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 22:34:53 +0100
When estimating the number of groups using extended statistics, the code
was discarding information about system attributes. This led to strange
situation that
SELECT 1 FROM t GROUP BY ctid;
could have produced higher estimate (equal to pg_class.reltuples) than
SELECT 1 FROM t GROUP BY a, b, ctid;
with extended statistics on (a,b). Fixed by retaining information about
the system attribute.
Backpatch all the way to 10, where extended statistics were introduced.
Author: Tomas Vondra
Backpatch-through: 10
M src/backend/utils/adt/selfuncs.c
M src/test/regress/expected/stats_ext.out
Add "pg_database_owner" default role.
commit : a14a0118a1fecf4066e53af52ed0f188607d0c4b
author : Noah Misch <noah@leadboat.com>
date : Fri, 26 Mar 2021 10:42:17 -0700
committer: Noah Misch <noah@leadboat.com>
date : Fri, 26 Mar 2021 10:42:17 -0700
Membership consists, implicitly, of the current database owner. Expect
use in template databases. Once pg_database_owner has rights within a
template, each owner of a database instantiated from that template will
exercise those rights.
Reviewed by John Naylor.
Discussion: https://postgr.es/m/20201228043148.GA1053024@rfd.leadboat.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/user-manag.sgml
M src/backend/catalog/information_schema.sql
M src/backend/commands/user.c
M src/backend/utils/adt/acl.c
M src/backend/utils/cache/catcache.c
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_authid.dat
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Merge similar algorithms into roles_is_member_of().
commit : f687bf61ed4dc75ec074c387f848147da2097e13
author : Noah Misch <noah@leadboat.com>
date : Fri, 26 Mar 2021 10:42:16 -0700
committer: Noah Misch <noah@leadboat.com>
date : Fri, 26 Mar 2021 10:42:16 -0700
The next commit would have complicated two or three algorithms, so take
this opportunity to consolidate. No functional changes.
Reviewed by John Naylor.
Discussion: https://postgr.es/m/20201228043148.GA1053024@rfd.leadboat.com
M src/backend/utils/adt/acl.c
Fix alignment in BRIN minmax-multi deserialization
commit : 73b96bad4af8fd113a36e4633dd3312001c115dc
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 16:33:19 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 16:33:19 +0100
The deserialization failed to ensure correct alignment, as it assumed it
can simply point into the serialized value. The serialization however
ignores alignment and copies just the significant bytes in order to make
the result as small as possible. This caused failures on systems that
are sensitive to mialigned addresses, like sparc, or with address
sanitizer enabled.
Fixed by copying the serialized data to ensure proper alignment. While
at it, fix an issue with serialization on big endian machines, using the
same store_att_byval/fetch_att trick as extended statistics.
Discussion: https://postgr.es/0c8c3304-d3dd-5e29-d5ac-b50589a23c8c%40enterprisedb.com
M src/backend/access/brin/brin_minmax_multi.c
BRIN minmax-multi indexes
commit : ab596105b55f1d7fbd5a66b66f65227d210b047d
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 13:54:29 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 13:54:29 +0100
Adds BRIN opclasses similar to the existing minmax, except that instead
of summarizing the page range into a single [min,max] range, the summary
consists of multiple ranges and/or points, allowing gaps. This allows
more efficient handling of data with poor correlation to physical
location within the table and/or outlier values, for which the regular
minmax opclassed tend to work poorly.
It's possible to specify the number of values kept for each page range,
either as a single point or an interval boundary.
CREATE TABLE t (a int);
CREATE INDEX ON t
USING brin (a int4_minmax_multi_ops(values_per_range=16));
When building the summary, the values are combined into intervals with
the goal to minimize the "covering" (sum of interval lengths), using a
support procedure computing distance between two values.
Bump catversion, due to various catalog changes.
Author: Tomas Vondra <tomas.vondra@postgresql.org>
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Reviewed-by: Sokolov Yura <y.sokolov@postgrespro.ru>
Reviewed-by: John Naylor <john.naylor@enterprisedb.com>
Discussion: https://postgr.es/m/c1138ead-7668-f0e1-0638-c3be3237e812@2ndquadrant.com
Discussion: https://postgr.es/m/5d78b774-7e9c-c94e-12cf-fef51cc89b1a%402ndquadrant.com
M doc/src/sgml/brin.sgml
M src/backend/access/brin/Makefile
A src/backend/access/brin/brin_minmax_multi.c
M src/backend/access/brin/brin_tuple.c
M src/include/access/brin_tuple.h
M src/include/access/transam.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_type.dat
A src/test/regress/expected/brin_multi.out
M src/test/regress/expected/psql.out
M src/test/regress/expected/type_sanity.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/brin_multi.sql
BRIN bloom indexes
commit : 77b88cd1bb9041a735f24072150cacfa06c699a3
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 13:35:29 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 13:35:29 +0100
Adds a BRIN opclass using a Bloom filter to summarize the range. Indexes
using the new opclasses allow only equality queries (similar to hash
indexes), but that works fine for data like UUID, MAC addresses etc. for
which range queries are not very common. This also means the indexes
work for data that is not well correlated to physical location within
the table, or perhaps even entirely random (which is a common issue with
existing BRIN minmax opclasses).
It's possible to specify opclass parameters with the usual Bloom filter
parameters, i.e. the desired false-positive rate and the expected number
of distinct values per page range.
CREATE TABLE t (a int);
CREATE INDEX ON t
USING brin (a int4_bloom_ops(false_positive_rate = 0.05,
n_distinct_per_range = 100));
The opclasses do not operate on the indexed values directly, but compute
a 32-bit hash first, and the Bloom filter is built on the hash value.
Collisions should not be a huge issue though, as the number of distinct
values in a page ranges is usually fairly small.
Bump catversion, due to various catalog changes.
Author: Tomas Vondra <tomas.vondra@postgresql.org>
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Reviewed-by: Sokolov Yura <y.sokolov@postgrespro.ru>
Reviewed-by: Nico Williams <nico@cryptonector.com>
Reviewed-by: John Naylor <john.naylor@enterprisedb.com>
Discussion: https://postgr.es/m/c1138ead-7668-f0e1-0638-c3be3237e812@2ndquadrant.com
Discussion: https://postgr.es/m/5d78b774-7e9c-c94e-12cf-fef51cc89b1a%402ndquadrant.com
M doc/src/sgml/brin.sgml
M src/backend/access/brin/Makefile
A src/backend/access/brin/brin_bloom.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_type.dat
A src/test/regress/expected/brin_bloom.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/expected/psql.out
M src/test/regress/expected/type_sanity.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/brin_bloom.sql
Support the old signature of BRIN consistent function
commit : a681e3c107aa97eb554f118935c4d2278892c3dd
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 13:17:56 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 13:17:56 +0100
Commit a1c649d889 changed the signature of the BRIN consistent function
by adding a new required parameter. Treating the parameter as optional,
which would make the change backwards incompatibile, was rejected with
the justification that there are few out-of-core extensions, so it's not
worth adding making the code more complex, and it's better to deal with
that in the extension.
But after further thought, that would be rather problematic, because
pg_upgrade simply dumps catalog contents and the same version of an
extension needs to work on both PostgreSQL versions. Supporting both
variants of the consistent function (with 3 or 4 arguments) makes that
possible.
The signature is not the only thing that changed, as commit 72ccf55cb9
moved handling of IS [NOT] NULL keys from the support procedures. But
this change is backward compatible - handling the keys in exension is
unnecessary, but harmless. The consistent function will do a bit of
unnecessary work, but it should be very cheap.
This also undoes most of the changes to the existing opclasses (minmax
and inclusion), making them use the old signature again. This should
make backpatching simpler.
Catversion bump, because of changes in pg_amproc.
Author: Tomas Vondra <tomas.vondra@postgresql.org>
Author: Nikita Glukhov <n.gluhov@postgrespro.ru>
Reviewed-by: Mark Dilger <hornschnorter@gmail.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Reviewed-by: Masahiko Sawada <masahiko.sawada@enterprisedb.com>
Reviewed-by: John Naylor <john.naylor@enterprisedb.com>
Discussion: https://postgr.es/m/c1138ead-7668-f0e1-0638-c3be3237e812@2ndquadrant.com
M doc/src/sgml/brin.sgml
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_inclusion.c
M src/backend/access/brin/brin_minmax.c
M src/backend/access/brin/brin_validate.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Remove unnecessary pg_amproc BRIN minmax entries
commit : a68dfa27d42fb7b7611fd1206d2356fc124ed390
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 13:04:13 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 13:04:13 +0100
The BRIN minmax opclasses included amproc entries with mismatching left
and right types, but those happen to be unnecessary. The opclasses only
need cross-type operators, not cross-type support procedures. Discovered
when trying to define equivalent BRIN operator families in an extension.
Catversion bump, because of pg_amproc changes.
Author: Tomas Vondra
Reviewed-by: Alvaro Herrera
Discussion: https://postgr.es/m/78c357ab-3395-8433-e7b3-b2cfcc9fdc23%40enterprisedb.com
M src/include/catalog/catversion.h
M src/include/catalog/pg_amproc.dat
Fix interaction of TOAST compression with expression indexes.
commit : 5db1fd7823a1a12e2bdad98abc8e102fd71ffbda
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 25 Mar 2021 19:55:32 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 25 Mar 2021 19:55:32 -0400
Before, trying to compress a value for insertion into an expression
index would crash.
Dilip Kumar, with some editing by me. Report by Jaime Casanova.
Discussion: http://postgr.es/m/CAJKUy5gcs0zGOp6JXU2mMVdthYhuQpFk=S3V8DOKT=LZC1L36Q@mail.gmail.com
M src/backend/access/brin/brin_tuple.c
M src/backend/access/common/indextuple.c
M src/backend/catalog/index.c
M src/test/regress/expected/compression.out
M src/test/regress/expected/compression_1.out
M src/test/regress/sql/compression.sql
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY
commit : 71f4c8c6f74ba021e55d35b1128d22fb8c6e1629
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 25 Mar 2021 18:00:28 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 25 Mar 2021 18:00:28 -0300
Allow a partition be detached from its partitioned table without
blocking concurrent queries, by running in two transactions and only
requiring ShareUpdateExclusive in the partitioned table.
Because it runs in two transactions, it cannot be used in a transaction
block. This is the main reason to use dedicated syntax: so that users
can choose to use the original mode if they need it. But also, it
doesn't work when a default partition exists (because an exclusive lock
would still need to be obtained on it, in order to change its partition
constraint.)
In case the second transaction is cancelled or a crash occurs, there's
ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final
steps.
The main trick to make this work is the addition of column
pg_inherits.inhdetachpending, initially false; can only be set true in
the first part of this command. Once that is committed, concurrent
transactions that use a PartitionDirectory will include or ignore
partitions so marked: in optimizer they are ignored if the row is marked
committed for the snapshot; in executor they are always included. As a
result, and because of the way PartitionDirectory caches partition
descriptors, queries that were planned before the detach will see the
rows in the detached partition and queries that are planned after the
detach, won't.
A CHECK constraint is created that duplicates the partition constraint.
This is probably not strictly necessary, and some users will prefer to
remove it afterwards, but if the partition is re-attached to a
partitioned table, the constraint needn't be rechecked.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/alter_table.sgml
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/partition.c
M src/backend/catalog/pg_inherits.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/executor/execPartition.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/optimizer/util/plancat.c
M src/backend/parser/gram.y
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partdesc.c
M src/backend/tcop/utility.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/cache/partcache.c
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/partition.h
M src/include/catalog/pg_inherits.h
M src/include/nodes/parsenodes.h
M src/include/parser/kwlist.h
M src/include/partitioning/partdesc.h
M src/include/utils/snapmgr.h
A src/test/isolation/expected/detach-partition-concurrently-1.out
A src/test/isolation/expected/detach-partition-concurrently-2.out
A src/test/isolation/expected/detach-partition-concurrently-3.out
A src/test/isolation/expected/detach-partition-concurrently-4.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/detach-partition-concurrently-1.spec
A src/test/isolation/specs/detach-partition-concurrently-2.spec
A src/test/isolation/specs/detach-partition-concurrently-3.spec
A src/test/isolation/specs/detach-partition-concurrently-4.spec
M src/test/modules/delay_execution/Makefile
A src/test/modules/delay_execution/expected/partition-removal-1.out
A src/test/modules/delay_execution/specs/partition-removal-1.spec
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
Document lock obtained during partition detach
commit : 650d623530c884c087c565f1d3b8cd76f8fe2b95
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 25 Mar 2021 16:30:22 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 25 Mar 2021 16:30:22 -0300
On partition detach, we acquire a SHARE lock on all tables that
reference the partitioned table that we're detaching a partition from,
but failed to document this fact. My oversight in commit f56f8f8da6af.
Repair. Backpatch to 12.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20210325180244.GA12738@alvherre.pgsql
M doc/src/sgml/ref/alter_table.sgml
Add comments for AlteredTableInfo->rel
commit : cc121d5596964f8aac93607e6f14607184558b16
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 25 Mar 2021 16:07:15 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 25 Mar 2021 16:07:15 -0300
The prior commit which introduced it was pretty squalid in terms of
code documentation, so add some comments.
M src/backend/commands/tablecmds.c
Let ALTER TABLE Phase 2 routines manage the relation pointer
commit : cd03c6e94b09ff402cbc3ce8da5587f09f0b5e58
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 25 Mar 2021 15:56:11 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 25 Mar 2021 15:56:11 -0300
Struct AlteredRelationInfo gains a new Relation member, to be used only
by Phase 2 (ATRewriteCatalogs); this allows ATExecCmd() subroutines open
and close the relation internally.
A future commit will use this facility to implement an ALTER TABLE
subcommand that closes and reopens the relation across transaction
boundaries.
(It is possible to keep the relation open past phase 2 to be used by
phase 3 instead of having to reopen it that point, but there are some
minor complications with that; it's not clear that there is much to be
won from doing that, though.)
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20200803234854.GA24158@alvherre.pgsql
M src/backend/commands/tablecmds.c
Rework HeapTupleHeader macros to reuse itemptr.h
commit : 4669cacbd4b4b1baa1b7f2ea53d461433a1b6276
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 22 Feb 2021 17:21:22 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 22 Feb 2021 17:21:22 -0300
The original definitions pointlessly disregarded existing ItemPointer
macros that do the same thing.
Reported-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20210222201557.GA32655@alvherre.pgsql
M src/include/access/htup_details.h
Remove StoreSingleInheritance reimplementation
commit : a24ae3d7b9efb3b113c0d53030aa99de0d41b40a
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 25 Mar 2021 10:47:38 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 25 Mar 2021 10:47:38 -0300
I introduced this duplicate code in commit 8b08f7d4820f for no good
reason. Remove it, and backpatch to 11 where it was introduced.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
M src/backend/commands/indexcmds.c
Trim some extra whitespace in parser file
commit : f2c7ce64ae9ba292c1846ae864cef1b8b37af1f3
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 25 Mar 2021 10:17:52 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 25 Mar 2021 10:17:52 +0100
M src/backend/parser/gram.y
Rename a parse node to be more general
commit : 91d1f2d302108f49006eedb8053522236dde77cc
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 25 Mar 2021 10:06:32 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 25 Mar 2021 10:06:32 +0100
A WHERE clause will be used for row filtering in logical replication.
We already have a similar node: 'WHERE (condition here)'. Let's
rename the node to a generic name and use it for row filtering too.
Author: Euler Taveira <euler.taveira@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/CAHE3wggb715X+mK_DitLXF25B=jE6xyNCH4YOwM860JR7HarGQ@mail.gmail.com
M src/backend/parser/gram.y
Sanitize the term "combo CID" in code comments
commit : a1999a01bb56c5f5451116abe61b892b2eec5e49
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 25 Mar 2021 16:08:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 25 Mar 2021 16:08:03 +0900
Combo CIDs were referred in the code comments using different terms
across various places of the code, so unify a bit the term used with
what is currently in use in some of the READMEs.
Author: "Hou, Zhijie"
Discussion: https://postgr.es/m/1d42865c91404f46af4562532fdbea31@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_visibility.c
M src/backend/executor/execMain.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/backend/utils/time/combocid.c
M src/include/access/htup_details.h
M src/test/regress/expected/combocid.out
M src/test/regress/sql/combocid.sql
Fix bug in WAL replay of COMMIT_TS_SETTS record.
commit : 438fc4a39c3905b7af88bb848bc5aeb1308a017d
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 25 Mar 2021 11:23:30 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 25 Mar 2021 11:23:30 +0900
Previously the WAL replay of COMMIT_TS_SETTS record called
TransactionTreeSetCommitTsData() with the argument write_xlog=true,
which generated and wrote new COMMIT_TS_SETTS record.
This should not be acceptable because it's during recovery.
This commit fixes the WAL replay of COMMIT_TS_SETTS record
so that it calls TransactionTreeSetCommitTsData() with write_xlog=false
and doesn't generate new WAL during recovery.
Back-patch to all supported branches.
Reported-by: lx zou <zoulx1982@163.com>
Author: Fujii Masao
Reviewed-by: Alvaro Herrera
Discussion: https://postgr.es/m/16931-620d0f2fdc6108f1@postgresql.org
M src/backend/access/transam/commit_ts.c
Improve connection denied error message during recovery.
commit : df9384492b89aac370ab9d12eb89375aeb38a1d4
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 25 Mar 2021 10:41:28 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 25 Mar 2021 10:41:28 +0900
Previously when an archive recovery or a standby was starting and
reached the consistent recovery state but hot_standby was configured
to off, the error message when a client connectted was "the database
system is starting up", which was needless confusing and not really
all that accurate either.
This commit improves the connection denied error message during
recovery, as follows, so that the users immediately know that their
servers are configured to deny those connections.
- If hot_standby is disabled, the error message "the database system
is not accepting connections" and the detail message "Hot standby
mode is disabled." are output when clients connect while an archive
recovery or a standby is running.
- If hot_standby is enabled, the error message "the database system
is not yet accepting connections" and the detail message
"Consistent recovery state has not been yet reached." are output
when clients connect until the consistent recovery state is reached
and postmaster starts accepting read only connections.
This commit doesn't change the connection denied error message of
"the database system is starting up" during normal server startup and
crash recovery. Because it's still suitable for those situations.
Author: James Coleman
Reviewed-by: Alvaro Herrera, Andres Freund, David Zhang, Tom Lane, Fujii Masao
Discussion: https://postgr.es/m/CAAaqYe8h5ES_B=F_zDT+Nj9XU7YEwNhKhHA2RE4CFhAQ93hfig@mail.gmail.com
M src/backend/postmaster/postmaster.c
M src/include/libpq/libpq-be.h
Allow for installation-aware instances of PostgresNode
commit : b34ca595abd697e716ce369ec1b58624bdd1c431
author : Andrew Dunstan <andrew@dunslane.net>
date : Wed, 24 Mar 2021 18:52:25 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Wed, 24 Mar 2021 18:52:25 -0400
Currently instances of PostgresNode find their Postgres executables in
the PATH of the caller. This modification allows for instances that know
the installation path they are supposed to use, and the module adjusts
the environment of methods that call Postgres executables appropriately.
This facility is activated by passing the installation path to the
constructor:
my $node = PostgresNode->get_new_node('mynode',
installation_path => '/path/to/installation');
This makes a number of things substantially easier, including
. testing third party modules
. testing different versions of postgres together
. testing different builds of postgres together
Discussion: https://postgr.es/m/a94c74f9-6b71-1957-7973-a734ea3cbef1@dunslane.net
Reviewed-By: Alvaro Herrera, Michael Paquier, Dagfinn Ilmari Mannsåker
M src/test/perl/PostgresNode.pm
Need to step forward in the loop to get to an end.
commit : 65c2ec6f30d9c0878a9ef83e0ec9a53e6b82d9d8
author : Michael Meskes <meskes@postgresql.org>
date : Wed, 24 Mar 2021 22:06:31 +0100
committer: Michael Meskes <meskes@postgresql.org>
date : Wed, 24 Mar 2021 22:06:31 +0100
M src/interfaces/ecpg/preproc/ecpg.c
Add DECLARE STATEMENT command to ECPG
commit : ad8305a43d1890768a613d3fb586b44f17360f29
author : Michael Meskes <meskes@postgresql.org>
date : Wed, 24 Mar 2021 20:48:20 +0100
committer: Michael Meskes <meskes@postgresql.org>
date : Wed, 24 Mar 2021 20:48:20 +0100
This command declares a SQL identifier for a SQL statement to be used in other
embedded SQL statements. The identifier is linked to a connection.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Shawn Wang <shawn.wang.pg@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/TY2PR01MB24438A52DB04E71D0E501452F5630@TY2PR01MB2443.jpnprd01.prod.outlook.com
M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/preproc/ecpg.addons
M src/interfaces/ecpg/preproc/ecpg.c
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/ecpg.type
M src/interfaces/ecpg/preproc/preproc_extern.h
M src/interfaces/ecpg/preproc/type.h
M src/interfaces/ecpg/test/ecpg_schedule
A src/interfaces/ecpg/test/expected/sql-declare.c
A src/interfaces/ecpg/test/expected/sql-declare.stderr
A src/interfaces/ecpg/test/expected/sql-declare.stdout
M src/interfaces/ecpg/test/sql/.gitignore
M src/interfaces/ecpg/test/sql/Makefile
A src/interfaces/ecpg/test/sql/declare.pgc
Fix stray double semicolons
commit : 37c99d304dcbf12ab581ff031f394af93b750895
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 24 Mar 2021 20:42:05 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 24 Mar 2021 20:42:05 +0100
Reported-by: John Naylor <john.naylor@enterprisedb.com>
M src/backend/access/brin/brin_minmax.c
M src/backend/utils/adt/timestamp.c
doc: Fix typo
commit : 5173e428928ce890be3e3d809b2d23d5f3c7da2f
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 24 Mar 2021 20:41:18 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 24 Mar 2021 20:41:18 +0100
Reported-by: Erik Rijkers <er@xs4all.nl>
M doc/src/sgml/func.sgml
Change checkpoint_completion_target default to 0.9
commit : bbcc4eb2e08fb6e4535c7f84b2c00f3ad508bb9b
author : Stephen Frost <sfrost@snowman.net>
date : Wed, 24 Mar 2021 13:07:51 -0400
committer: Stephen Frost <sfrost@snowman.net>
date : Wed, 24 Mar 2021 13:07:51 -0400
Common recommendations are that the checkpoint should be spread out as
much as possible, provided we avoid having it take too long. This
change updates the default to 0.9 (from 0.5) to match that
recommendation.
There was some debate about possibly removing the option entirely but it
seems there may be some corner-cases where having it set much lower to
try to force the checkpoint to be as fast as possible could result in
fewer periods of time of reduced performance due to kernel flushing.
General agreement is that the "spread more" is the preferred approach
though and those who need to tune away from that value are much less
common.
Reviewed-By: Michael Paquier, Peter Eisentraut, Tom Lane, David Steele,
Nathan Bossart
Discussion: https://postgr.es/m/20201207175329.GM16415%40tamriel.snowman.net
M doc/src/sgml/config.sgml
M doc/src/sgml/wal.sgml
M src/backend/postmaster/checkpointer.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/test/recovery/t/015_promotion_pages.pl
Tidy up more loose ends related to configurable TOAST compression.
commit : e5595de03ec6ce60afde980ae05e9353a1501fdf
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 24 Mar 2021 12:36:08 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 24 Mar 2021 12:36:08 -0400
Change the default_toast_compression GUC to be an enum rather than
a string. Earlier, uncommitted versions of the patch supported using
CREATE ACCESS METHOD to add new compression methods to a running
system, but that idea was dropped before commit. So, we can simplify
the GUC handling as well, which has the nice side effect of improving
the error messages.
While updating the documentation to reflect the new GUC type, also
move it back to the right place in the list. I moved this while
revising what became commit 24f0e395ac5892cd12e8914646fe921fac5ba23d,
but apparently the intended ordering is "alphabetical" rather than
"whatever Robert thinks looks nice."
Rejigger things to avoid having access/toast_compression.h depend on
utils/guc.h, so that we don't end up with every file that includes
it also depending on something largely unrelated. Move a few
inline functions back into the C source file partly to help reduce
dependencies and partly just to avoid clutter. A few very minor
cosmetic fixes.
Original patch by Justin Pryzby, but very heavily edited by me,
and reverse reviewed by him and also reviewed by by Tom Lane.
Discussion: http://postgr.es/m/CA+TgmoYp=GT_ztUCeZg2i4hkHAQv8o=-nVJ1-TKWTG1zQOmOpg@mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/access/common/toast_compression.c
M src/backend/utils/misc/guc.c
M src/include/access/toast_compression.h
M src/test/regress/expected/compression.out
M src/test/regress/expected/compression_1.out
Add date_bin function
commit : 49ab61f0bdc93984a8d36b602f6f2a15f09ebcc7
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 24 Mar 2021 16:16:14 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 24 Mar 2021 16:16:14 +0100
Similar to date_trunc, but allows binning by an arbitrary interval
rather than just full units.
Author: John Naylor <john.naylor@enterprisedb.com>
Reviewed-by: David Fetter <david@fetter.org>
Reviewed-by: Isaac Morland <isaac.morland@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Artur Zakirov <zaartur@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CACPNZCt4buQFRgy6DyjuZS-2aPDpccRkrJBmgUfwYc1KiaXYxg@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/timestamp.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Improve an error message
commit : 1509c6fc29c07d13c9a590fbd6f37c7576f58ba6
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 24 Mar 2021 08:02:06 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 24 Mar 2021 08:02:06 +0100
Make it the same as another nearby message.
M src/backend/replication/logical/tablesync.c
Revert "Enable parallel SELECT for "INSERT INTO ... SELECT ..."."
commit : 26acb54a1368bf3706294400abca85b15c9233a6
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 24 Mar 2021 11:10:12 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 24 Mar 2021 11:10:12 +0530
To allow inserts in parallel-mode this feature has to ensure that all the
constraints, triggers, etc. are parallel-safe for the partition hierarchy
which is costly and we need to find a better way to do that. Additionally,
we could have used existing cached information in some cases like indexes,
domains, etc. to determine the parallel-safety.
List of commits reverted, in reverse chronological order:
ed62d3737c Doc: Update description for parallel insert reloption.
c8f78b6161 Add a new GUC and a reloption to enable inserts in parallel-mode.
c5be48f092 Improve FK trigger parallel-safety check added by 05c8482f7f.
e2cda3c20a Fix use of relcache TriggerDesc field introduced by commit 05c8482f7f.
e4e87a32cc Fix valgrind issue in commit 05c8482f7f.
05c8482f7f Enable parallel SELECT for "INSERT INTO ... SELECT ...".
Discussion: https://postgr.es/m/E1lMiB9-0001c3-SY@gemulon.postgresql.org
M doc/src/sgml/config.sgml
M doc/src/sgml/parallel.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_table.sgml
M src/backend/access/common/reloptions.c
M src/backend/access/transam/xact.c
M src/backend/executor/execMain.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/util/clauses.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/psql/tab-complete.c
M src/include/access/xact.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/clauses.h
M src/include/optimizer/cost.h
M src/include/utils/rel.h
D src/test/regress/expected/insert_parallel.out
M src/test/regress/expected/sysviews.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
D src/test/regress/sql/insert_parallel.sql
M src/tools/pgindent/typedefs.list
Rename wait event WalrcvExit to WalReceiverExit.
commit : 84007043fc1b1be68dad5d0a78269347c12094b6
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 24 Mar 2021 10:37:54 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 24 Mar 2021 10:37:54 +0900
Commit de829ddf23 added wait event WalrcvExit. But its name is not
consistent with other wait events like WalReceiverMain or
WalReceiverWaitStart, etc. So this commit renames WalrcvExit to
WalReceiverExit.
Author: Fujii Masao
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/cced9995-8fa2-7b22-9d91-3f22a2b8c23c@oss.nttdata.com
M doc/src/sgml/monitoring.sgml
M src/backend/postmaster/pgstat.c
M src/backend/replication/walreceiverfuncs.c
M src/include/pgstat.h
Log when GetNewOidWithIndex() fails to find unused OID many times.
commit : 7fbcee1b2d5f1012c67942126881bd492e95077e
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 24 Mar 2021 10:36:56 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 24 Mar 2021 10:36:56 +0900
GetNewOidWithIndex() generates a new OID one by one until it finds
one not in the relation. If there are very long runs of consecutive
existing OIDs, GetNewOidWithIndex() needs to iterate many times
in the loop to find unused OID. Since TOAST table can have a large
number of entries and there can be such long runs of OIDs, there is
the case where it takes so many iterations to find new OID not in
TOAST table. Furthermore if all (i.e., 2^32) OIDs are already used,
GetNewOidWithIndex() enters something like busy loop and repeats
the iterations until at least one OID is marked as unused.
There are some reported troubles caused by a large number of
iterations in GetNewOidWithIndex(). For example, when inserting
a billion of records into the table, all the backends doing that
insertion operation got hang with 100% CPU usage at some point.
Previously there was no easy way to detect that GetNewOidWithIndex()
failed to find unused OID many times. So, for example, gdb full
backtrace of hanged backends needed to be taken, in order to
investigate that trouble. This is inconvenient and may not be
available in some production environments.
To provide easy way for that, this commit makes GetNewOidWithIndex()
log that it iterates more than GETNEWOID_LOG_THRESHOLD but have
not yet found OID unused in the relation. Also this commit makes
it repeat logging with exponentially increasing intervals until
it iterates more than GETNEWOID_LOG_MAX_INTERVAL, and makes it
finally repeat logging every GETNEWOID_LOG_MAX_INTERVAL unless
an unused OID is found. Those macro variables are used not to
fill up the server log with the similar messages.
In the discusion at pgsql-hackers, there was another idea to report
the lots of iterations in GetNewOidWithIndex() via wait event.
But since GetNewOidWithIndex() traverses indexes to find unused
OID and which will do I/O, acquire locks, etc, which will overwrite
the wait event and reset it to nothing once done. So that idea
doesn't work well, and we didn't adopt it.
Author: Tomohiro Hiramitsu
Reviewed-by: Tatsuhito Kasahara, Kyotaro Horiguchi, Tom Lane, Fujii Masao
Discussion: https://postgr.es/m/16722-93043fb459a41073@postgresql.org
M src/backend/catalog/catalog.c
Reword slightly logs generated for index stats in autovacuum
commit : 99dd75fb99baa9188971cf47779ed8d7a5e6eb29
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 24 Mar 2021 09:36:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 24 Mar 2021 09:36:03 +0900
Using "remain" is confusing, as it implies that the index file can
shrink. Instead, use "in total".
Per discussion with Peter Geoghegan.
Discussion: https://postgr.es/m/CAH2-WzkYgHZzpGOwR14CScJsjaQpvJrEkEfkh_=wGhzLb=yVdQ@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Allow composite types in catalog bootstrap
commit : 79f6a942bdb958fbd7ef6870d5bf2e3cefb65da5
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 24 Mar 2021 00:47:50 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 24 Mar 2021 00:47:50 +0100
When resolving types during catalog bootstrap, try to reload the pg_type
contents if a type is not found. That allows catalogs to contain
composite types, e.g. row types for other catalogs.
Author: Justin Pryzby
Reviewed-by: Dean Rasheed, Tomas Vondra
Discussion: https://postgr.es/m/ad7891d2-e90c-b446-9fe2-7419143847d7%40enterprisedb.com
M src/backend/bootstrap/bootstrap.c
Convert Typ from array to list in bootstrap
commit : e1a5e65703ce884529340819f6268d24f43ef8f7
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 24 Mar 2021 00:47:38 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 24 Mar 2021 00:47:38 +0100
It's a bit easier and more convenient to free and reload a List,
compared to a plain array. This will be helpful when allowing catalogs
to contain composite types.
Author: Justin Pryzby
Reviewed-by: Dean Rasheed, Tomas Vondra
Discussion: https://postgr.es/m/ad7891d2-e90c-b446-9fe2-7419143847d7%40enterprisedb.com
M src/backend/bootstrap/bootstrap.c
nbtree VACUUM: Cope with buggy opclasses.
commit : 5b861baa550a369e04bf67fbe83f3a5a8c742fb4
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 23 Mar 2021 16:09:51 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 23 Mar 2021 16:09:51 -0700
Teach nbtree VACUUM to press on with vacuuming in the event of a page
deletion attempt that fails to "re-find" a downlink for its child/target
page.
There is no good reason to treat this as an irrecoverable error. But
there is a good reason not to: pressing on at this point removes any
question of VACUUM not making progress solely due to misbehavior from
user-defined operator class code.
Discussion: https://postgr.es/m/CAH2-Wzma5G9CTtMjbrXTwOym+U=aWg-R7=-htySuztgoJLvZXg@mail.gmail.com
M src/backend/access/nbtree/nbtpage.c
Improve pg_amcheck's TAP test 003_check.pl.
commit : 87d90ac61fa113ffc886efcdb391c522c1982991
author : Robert Haas <rhaas@postgresql.org>
date : Tue, 23 Mar 2021 14:57:45 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Tue, 23 Mar 2021 14:57:45 -0400
Disable autovacuum, because we don't want it to run against
intentionally corrupted tables. Also, before corrupting the tables,
run pg_amcheck and ensure that it passes. Otherwise, if something
unexpected happens when we check the corrupted tables, it's not so
clear whether it would have also happened before we corrupted
them.
Mark Dilger
Discussion: http://postgr.es/m/AA5506CE-7D2A-42E4-A51D-358635E3722D@enterprisedb.com
M src/bin/pg_amcheck/t/003_check.pl
Fix psql's \connect command some more.
commit : ea80138545043c0cfcff8405b15626796f2695fe
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 23 Mar 2021 14:27:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 23 Mar 2021 14:27:50 -0400
Jasen Betts reported yet another unintended side effect of commit
85c54287a: reconnecting with "\c service=whatever" did not have the
expected results. The reason is that starting from the output of
PQconndefaults() effectively allows environment variables (such
as PGPORT) to override entries in the service file, whereas the
normal priority is the other way around.
Not using PQconndefaults at all would require yet a third main code
path in do_connect's parameter setup, so I don't really want to fix
it that way. But we can have the logic effectively ignore all the
default values for just a couple more lines of code.
This patch doesn't change the behavior for "\c -reuse-previous=on
service=whatever". That remains significantly different from before
85c54287a, because many more parameters will be re-used, and thus
not be possible for service entries to replace. But I think this
is (mostly?) intentional. In any case, since libpq does not report
where it got parameter values from, it's hard to do differently.
Per bug #16936 from Jasen Betts. As with the previous patches,
back-patch to all supported branches. (9.5 is unfortunately now
out of support, so this won't get fixed there.)
Discussion: https://postgr.es/m/16936-3f524322a53a29f0@postgresql.org
M src/bin/psql/command.c
Avoid possible crash while finishing up a heap rewrite.
commit : 9d523119fd38fd205cb9c8ea8e7cceeb54355818
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 23 Mar 2021 11:24:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 23 Mar 2021 11:24:16 -0400
end_heap_rewrite was not careful to ensure that the target relation
is open at the smgr level before performing its final smgrimmedsync.
In ordinary cases this is no problem, because it would have been
opened earlier during the rewrite. However a crash can be reproduced
by re-clustering an empty table with CLOBBER_CACHE_ALWAYS enabled.
Although that exact scenario does not crash in v13, I think that's
a chance result of unrelated planner changes, and the problem is
likely still reachable with other test cases. The true proximate
cause of this failure is commit c6b92041d, which replaced a call to
heap_sync (which was careful about opening smgr) with a direct call
to smgrimmedsync. Hence, back-patch to v13.
Amul Sul, per report from Neha Sharma; cosmetic changes
and test case by me.
Discussion: https://postgr.es/m/CANiYTQsU7yMFpQYnv=BrcRVqK_3U3mtAzAsJCaqtzsDHfsUbdQ@mail.gmail.com
M src/backend/access/heap/rewriteheap.c
M src/test/regress/expected/cluster.out
M src/test/regress/sql/cluster.sql
pgcrypto: Check for error return of px_cipher_decrypt()
commit : 22e1943f13b66df22ea4f8d15836411ba259263a
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 23 Mar 2021 11:35:12 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 23 Mar 2021 11:35:12 +0100
This has previously not been a problem (that anyone ever reported),
but in future OpenSSL versions (3.0.0), where legacy ciphers are/can
be disabled, this is the place where this is reported. So we need to
catch the error here, otherwise the higher-level functions would
return garbage. The nearby encryption code already handled errors
similarly.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://www.postgresql.org/message-id/9e9c431c-0adc-7a6d-9b1a-915de1ba3fe7@enterprisedb.com
M contrib/pgcrypto/px.c
Add bit_count SQL function
commit : a6715af1e72da289474011be1e2d536f991eda34
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 23 Mar 2021 08:45:51 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 23 Mar 2021 08:45:51 +0100
This function for bit and bytea counts the set bits in the bit or byte
string. Internally, we use the existing popcount functionality.
For the name, after some discussion, we settled on bit_count, which
also exists with this meaning in MySQL, Java, and Python.
Author: David Fetter <david@fetter.org>
Discussion: https://www.postgresql.org/message-id/flat/20201230105535.GJ13234@fetter.org
M doc/src/sgml/func.sgml
M src/backend/utils/adt/varbit.c
M src/backend/utils/adt/varlena.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/bit.out
M src/test/regress/expected/strings.out
M src/test/regress/sql/bit.sql
M src/test/regress/sql/strings.sql
Add per-index stats information in verbose logs of autovacuum
commit : 5aed6a1fc214913de9ac69c1717dc64a2483e16d
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 23 Mar 2021 13:25:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 23 Mar 2021 13:25:14 +0900
Once a relation's autovacuum is completed, the logs include more
information about this relation state if the threshold of
log_autovacuum_min_duration (or its relation option) is reached, with
for example contents about the statistics of the VACUUM operation for
the relation, WAL and system usage.
This commit adds more information about the statistics of the relation's
indexes, with one line of logs generated for each index. The index
stats were already calculated, but not printed in the context of
autovacuum yet. While on it, some refactoring is done to keep track of
the index statistics directly within LVRelStats, simplifying some
routines related to parallel VACUUMs.
Author: Masahiko Sawada
Reviewed-by: Michael Paquier, Euler Taveira
Discussion: https://postgr.es/m/CAD21AoAy6SxHiTivh5yAPJSUE4S=QRPpSZUdafOSz0R+fRcM6Q@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Fix dangling pointer reference in stream_cleanup_files.
commit : 4b82ed6eca41220e50d4712ab929c20030b30d35
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 23 Mar 2021 09:43:33 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 23 Mar 2021 09:43:33 +0530
We can't access the entry after it is removed from dynahash.
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+Ps-pL++f6CJwPx2+vUqXuew=Xt-9Bi-6kCyxn+Fwi2M7w@mail.gmail.com
M src/backend/replication/logical/worker.c
Use correct spelling of statistics kind
commit : a5f002ad9a2ddb501148a12281efbaacec6f6397
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 23 Mar 2021 04:45:26 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 23 Mar 2021 04:45:26 +0100
A couple error messages and comments used 'statistic kind', not the
correct 'statistics kind'. Fix and backpatch all the way back to 10,
where extended statistics were introduced.
Backpatch-through: 10
M doc/src/sgml/catalogs.sgml
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/statistics/mcv.c
M src/backend/statistics/mvdistinct.c
M src/include/nodes/pathnodes.h
Change the type of WalReceiverWaitStart wait event from Client to IPC.
commit : 1e3e8b51bda8ddd59984230f876f199c9ce3166a
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 23 Mar 2021 10:09:42 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 23 Mar 2021 10:09:42 +0900
Previously the type of this wait event was Client. But while this
wait event is being reported, walreceiver process is waiting for
the startup process to set initial data for streaming replication.
It's not waiting for any activity on a socket connected to a user
application or walsender. So this commit changes the type for
WalReceiverWaitStart wait event to IPC.
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/cdacc27c-37ff-f1a4-20e2-ce19933abfcc@oss.nttdata.com
M doc/src/sgml/monitoring.sgml
M src/backend/postmaster/pgstat.c
M src/include/pgstat.h
pg_waldump: Fix bug in per-record statistics.
commit : 51893c8463501fc9a38e39cc097773dbdfb9db82
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 23 Mar 2021 09:53:08 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 23 Mar 2021 09:53:08 +0900
pg_waldump --stats=record identifies a record by a combination
of the RmgrId and the four bits of the xl_info field of the record.
But XACT records use the first bit of those four bits for an optional
flag variable, and the following three bits for the opcode to
identify a record. So previously the same type of XACT record
could have different four bits (three bits are the same but the
first one bit is different), and which could cause
pg_waldump --stats=record to show two lines of per-record statistics
for the same XACT record. This is a bug.
This commit changes pg_waldump --stats=record so that it processes
only XACT record differently, i.e., filters the opcode out of xl_info
and uses a combination of the RmgrId and those three bits as
the identifier of a record, only for XACT record. For other records,
the four bits of the xl_info field are still used.
Back-patch to all supported branches.
Author: Kyotaro Horiguchi
Reviewed-by: Shinya Kato, Fujii Masao
Discussion: https://postgr.es/m/2020100913412132258847@highgo.ca
M src/bin/pg_waldump/pg_waldump.c
Add macro RelationIsPermanent() to report relation permanence
commit : 95d77149c53545a74e0c84717cf8f925b8f6d632
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 22 Mar 2021 20:22:48 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 22 Mar 2021 20:22:48 -0400
Previously, to check relation permanence, the Relation's Form_pg_class
structure member relpersistence was compared to the value
RELPERSISTENCE_PERMANENT ("p"). This commit adds the macro
RelationIsPermanent() and is used in appropirate places to simplify the
code. This matches other RelationIs* macros.
This macro will be used in more places in future cluster file encryption
patches.
Discussion: https://postgr.es/m/20210318153134.GH20766@tamriel.snowman.net
M src/backend/access/gist/gistutil.c
M src/backend/access/heap/heapam_handler.c
M src/backend/catalog/pg_publication.c
M src/backend/commands/tablecmds.c
M src/backend/optimizer/util/plancat.c
M src/backend/utils/cache/relcache.c
M src/include/utils/rel.h
M src/include/utils/snapmgr.h
Optimize allocations in bringetbitmap
commit : 8e4b332e88b8339408a3aa8c62bc93d96b67c808
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 23 Mar 2021 00:47:06 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 23 Mar 2021 00:47:06 +0100
The bringetbitmap function allocates memory for various purposes, which
may be quite expensive, depending on the number of scan keys. Instead of
allocating them separately, allocate one bit chunk of memory an carve it
into smaller pieces as needed - all the pieces have the same lifespan,
and it saves quite a bit of CPU and memory overhead.
Author: Tomas Vondra <tomas.vondra@postgresql.org>
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Mark Dilger <hornschnorter@gmail.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Reviewed-by: Masahiko Sawada <masahiko.sawada@enterprisedb.com>
Reviewed-by: John Naylor <john.naylor@enterprisedb.com>
Discussion: https://postgr.es/m/c1138ead-7668-f0e1-0638-c3be3237e812@2ndquadrant.com
M src/backend/access/brin/brin.c
Move IS [NOT] NULL handling from BRIN support functions
commit : 72ccf55cb99c6450dfb77f2f8f4a28b5c049ef7a
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 23 Mar 2021 00:45:33 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 23 Mar 2021 00:45:33 +0100
The handling of IS [NOT] NULL clauses is independent of an opclass, and
most of the code was exactly the same in both minmax and inclusion. So
instead move the code from support procedures to the AM.
This simplifies the code - especially the support procedures - quite a
bit, as they don't need to care about NULL values and flags at all. It
also means the IS [NOT] NULL clauses can be evaluated without invoking
the support procedure.
Author: Tomas Vondra <tomas.vondra@postgresql.org>
Author: Nikita Glukhov <n.gluhov@postgrespro.ru>
Reviewed-by: Nikita Glukhov <n.gluhov@postgrespro.ru>
Reviewed-by: Mark Dilger <hornschnorter@gmail.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Reviewed-by: Masahiko Sawada <masahiko.sawada@enterprisedb.com>
Reviewed-by: John Naylor <john.naylor@enterprisedb.com>
Discussion: https://postgr.es/m/c1138ead-7668-f0e1-0638-c3be3237e812@2ndquadrant.com
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_inclusion.c
M src/backend/access/brin/brin_minmax.c
M src/include/access/brin_internal.h
Pass all scan keys to BRIN consistent function at once
commit : a1c649d889bdf6e74e9382e1e28574d7071568de
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 23 Mar 2021 00:12:19 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 23 Mar 2021 00:12:19 +0100
This commit changes how we pass scan keys to BRIN consistent function.
Instead of passing them one by one, we now pass all scan keys for a
given attribute at once. That makes the consistent function a bit more
complex, as it has to loop through the keys, but it does allow more
elaborate opclasses that can use multiple keys to eliminate ranges much
more effectively.
The existing BRIN opclasses (minmax, inclusion) don't really benefit
from this change. The primary purpose is to allow future opclases to
benefit from seeing all keys at once.
This does change the BRIN API, because the signature of the consistent
function changes (a new parameter with number of scan keys). So this
breaks existing opclasses, and will require supporting two variants of
the code for different PostgreSQL versions. We've considered supporting
two variants of the consistent, but we've decided not to do that.
Firstly, there's another patch that moves handling of NULL values from
the opclass, which means the opclasses need to be updated anyway.
Secondly, we're not aware of any out-of-core BRIN opclasses, so it does
not seem worth the extra complexity.
Bump catversion, because of pg_proc changes.
Author: Tomas Vondra <tomas.vondra@postgresql.org>
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Mark Dilger <hornschnorter@gmail.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Reviewed-by: John Naylor <john.naylor@enterprisedb.com>
Reviewed-by: Nikita Glukhov <n.gluhov@postgrespro.ru>
Discussion: https://postgr.es/m/c1138ead-7668-f0e1-0638-c3be3237e812@2ndquadrant.com
M doc/src/sgml/brin.sgml
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_inclusion.c
M src/backend/access/brin/brin_minmax.c
M src/backend/access/brin/brin_validate.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Move bsearch_arg to src/port
commit : bfa2cee784125047771db2768fcf7f04d8bd6bb4
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 23 Mar 2021 00:11:20 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 23 Mar 2021 00:11:20 +0100
Until now the bsearch_arg function was used only in extended statistics
code, so it was defined in that code. But we already have qsort_arg in
src/port, so let's move it next to it.
M src/backend/statistics/extended_stats.c
M src/include/port.h
M src/include/statistics/extended_stats_internal.h
M src/port/Makefile
A src/port/bsearch_arg.c
M src/tools/msvc/Mkvcbuild.pm
Short-circuit slice requests that are for more than the object's size.
commit : 063dd37ebc7644e8db6419565b50dca019e69e86
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Mar 2021 14:01:20 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Mar 2021 14:01:20 -0400
substring(), and perhaps other callers, isn't careful to pass a
slice length that is no more than the datum's true size. Since
toast_decompress_datum_slice's children will palloc the requested
slice length, this can waste memory. Also, close study of the liblz4
documentation suggests that it is dependent on the caller to not ask
for more than the correct amount of decompressed data; this squares
with observed misbehavior with liblz4 1.8.3. Avoid these problems
by switching to the normal full-decompression code path if the
slice request is >= datum's decompressed size.
Tom Lane and Dilip Kumar
Discussion: https://postgr.es/m/507597.1616370729@sss.pgh.pa.us
M src/backend/access/common/detoast.c
M src/include/access/toast_internals.h
Mostly-cosmetic adjustments of TOAST-related macros.
commit : aeb1631ed207cef2d80e20f79eb52c72f03bca7d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Mar 2021 13:43:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Mar 2021 13:43:10 -0400
The authors of bbe0a81db hadn't quite got the idea that macros named
like SOMETHING_4B_C were only meant for internal endianness-related
details in postgres.h. Choose more legible names for macros that are
intended to be used elsewhere. Rearrange postgres.h a bit to clarify
the separation between those internal macros and ones intended for
wider use.
Also, avoid using the term "rawsize" for true decompressed size;
we've used "extsize" for that, because "rawsize" generally denotes
total Datum size including header. This choice seemed particularly
unfortunate in tests that were comparing one of these meanings to
the other.
This patch includes a couple of not-purely-cosmetic changes: be
sure that the shifts aligning compression methods are unsigned
(not critical today, but will be when compression method 2 exists),
and fix broken definition of VARATT_EXTERNAL_GET_COMPRESSION (now
VARATT_EXTERNAL_GET_COMPRESS_METHOD), whose callers worked only
accidentally.
Discussion: https://postgr.es/m/574197.1616428079@sss.pgh.pa.us
M src/backend/access/common/detoast.c
M src/backend/access/common/toast_compression.c
M src/backend/access/common/toast_internals.c
M src/include/access/toast_internals.h
M src/include/postgres.h
Remove useless configure probe for <lz4/lz4.h>.
commit : 2c75f8a612b207c7d36e5dc73317dc9ab6fb29d4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Mar 2021 11:20:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Mar 2021 11:20:44 -0400
This seems to have been just copied-and-pasted from some other
header checks. But our C code is entirely unprepared to support
such a header name, so it's only wasting cycles to look for it.
If we did need to support it, some #ifdefs would be required.
(A quick trawl at codesearch.debian.net finds some packages that
reference lz4/lz4.h; but they use *only* that spelling, and
appear to be intending to reference their own copy rather than
a system-level installation of liblz4. There's no evidence of
freestanding installations that require this spelling.)
Discussion: https://postgr.es/m/457962.1616362509@sss.pgh.pa.us
M configure
M configure.ac
M src/include/pg_config.h.in
M src/tools/msvc/Solution.pm
Error on invalid TOAST compression in CREATE or ALTER TABLE.
commit : a4d5284a10b5096585f1bbf1bf723954e9d6c2e0
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 22 Mar 2021 10:57:08 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 22 Mar 2021 10:57:08 -0400
The previous coding treated an invalid compression method name as
equivalent to the default, which is certainly not right.
Justin Pryzby
Discussion: http://postgr.es/m/20210321235544.GD4203@telsasoft.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/compression.out
M src/test/regress/expected/compression_1.out
M src/test/regress/sql/compression.sql
docs: Fix omissions related to configurable TOAST compression.
commit : 24f0e395ac5892cd12e8914646fe921fac5ba23d
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 22 Mar 2021 10:34:10 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 22 Mar 2021 10:34:10 -0400
Previously, the default_toast_compression GUC was not documented,
and neither was pg_dump's new --no-toast-compression option.
Justin Pryzby and Robert Haas
Discussion: http://postgr.es/m/20210321235544.GD4203@telsasoft.com
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/pg_dump.sgml
More code cleanup for configurable TOAST compression.
commit : 226e2be3876d0bda3dc33d16dfa0bed246b7b74f
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 22 Mar 2021 09:21:37 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 22 Mar 2021 09:21:37 -0400
Remove unused macro. Fix confusion about whether a TOAST compression
method is identified by an OID or a char.
Justin Pryzby
Discussion: http://postgr.es/m/20210321235544.GD4203@telsasoft.com
M src/backend/commands/tablecmds.c
M src/include/access/toast_compression.h
Fix concurrency issues with WAL segment recycling on Windows
commit : 909b449e00fc2f71e1a38569bbddbb6457d28485
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Mar 2021 14:02:26 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Mar 2021 14:02:26 +0900
This commit is mostly a revert of aaa3aed, that switched the routine
doing the internal renaming of recycled WAL segments to use on Windows a
combination of CreateHardLinkA() plus unlink() instead of rename(). As
reported by several users of Postgres 13, this is causing concurrency
issues when manipulating WAL segments, mostly in the shape of the
following error:
LOG: could not rename file "pg_wal/000000XX000000YY000000ZZ":
Permission denied
This moves back to a logic where a single rename() (well, pgrename() for
Windows) is used. This issue has proved to be hard to hit when I tested
it, facing it only once with an archive_command that was not able to do
its work, so it is environment-sensitive. The reporters of this issue
have been able to confirm that the situation improved once we switched
back to a single rename(). In order to check things, I have provided to
the reporters a patched build based on 13.2 with aaa3aed reverted, to
test if the error goes away, and an unpatched build of 13.2 to test if
the error still showed up (just to make sure that I did not mess up my
build process).
Extra thanks to Fujii Masao for pointing out what looked like the
culprit commit, and to all the reporters for taking the time to test
what I have sent them.
Reported-by: Andrus, Guy Burgess, Yaroslav Pashinsky, Thomas Trenz
Reviewed-by: Tom Lane, Andres Freund
Discussion: https://postgr.es/m/3861ff1e-0923-7838-e826-094cc9bef737@hot.ee
Discussion: https://postgr.es/m/16874-c3eecd319e36a2bf@postgresql.org
Discussion: https://postgr.es/m/095ccf8d-7f58-d928-427c-b17ace23cae6@burgess.co.nz
Discussion: https://postgr.es/m/16927-67c570d968c99567%40postgresql.org
Discussion: https://postgr.es/m/YFBcRbnBiPdGZvfW@paquier.xyz
Backpatch-through: 13
M src/backend/storage/file/fd.c
M src/include/pg_config_manual.h
pgbench: Improve error-handling in \sleep command.
commit : 8c6eda2d1c926be76baa79c28521275323bd26fd
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 22 Mar 2021 12:02:44 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 22 Mar 2021 12:02:44 +0900
This commit improves pgbench \sleep command so that it handles
the following three cases more properly.
(1) When only one argument was specified in \sleep command and
it's not a number, previously pgbench reported a confusing error
message like "unrecognized time unit, must be us, ms or s".
This commit fixes this so that more proper error message like
"invalid sleep time, must be an integer" is reported.
(2) When two arguments were specified in \sleep command and
the first argument was not a number, previously pgbench treated
that argument as the sleep time 0. No error was reported in this
case. This commit fixes this so that an error is thrown in this
case.
(3) When a variable was specified as the first argument in \sleep
command and the variable stored non-digit value, previously
pgbench treated that argument as the sleep time 0. No error
was reported in this case. This commit fixes this so that
an error is thrown in this case.
Author: Kota Miyake
Reviewed-by: Hayato Kuroda, Alvaro Herrera, Fujii Masao
Discussion: https://postgr.es/m/23b254daf20cec4332a2d9168505dbc9@oss.nttdata.com
M src/bin/pgbench/pgbench.c
Make a test endure log_error_verbosity=verbose.
commit : e3f4aec027891f794328050e62c9bbbe4ae02811
author : Noah Misch <noah@leadboat.com>
date : Sun, 21 Mar 2021 19:09:29 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 21 Mar 2021 19:09:29 -0700
Back-patch to v13, which introduced the test code in question.
M src/test/recovery/t/003_recovery_targets.pl
Fix new TAP test for 2PC transactions and PITRs on Windows
commit : 992d353a190c551db39bcab2dec0ecf14fbc7a40
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Mar 2021 09:51:05 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Mar 2021 09:51:05 +0900
The test added by 595b9cb forgot that on Windows it is necessary to set
up pg_hba.conf (see PostgresNode::set_replication_conf) with a specific
entry or base backups fail. Any node that requires to support
replication just needs to pass down allows_streaming at initialization.
This updates the test to do so. Simplify things a bit while on it.
Per buildfarm member fairywren. Any Windows hosts running this test
would have failed, and I have reproduced the problem as well.
Backpatch-through: 10
M src/test/recovery/t/023_pitr_prepared_xact.pl
Simplify TAP tests of kerberos with expected log file contents
commit : 11e1577a576fec6307aa0bfcde7333e63f907fa7
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Mar 2021 08:59:43 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Mar 2021 08:59:43 +0900
The TAP tests of kerberos rely on the logs generated by the backend to
check various connection scenarios. In order to make sure that a given
test does not overlap with the log contents generated by a previous
test, the test suite relied on a logic with the logging collector and a
rotation of the log files to ensure the uniqueness of the log generated
with a wait phase.
Parsing the log contents for expected patterns is a problem that has
been solved in a simpler way by PostgresNode::issues_sql_like() where
the log file is truncated before checking for the contents generated,
with the backend sending its output to a log file given by pg_ctl
instead. This commit switches the kerberos test suite to use such a
method, removing any wait phase and simplifying the whole logic,
resulting in less code. If a failure happens in the tests, the contents
of the logs are still showed to the user at the moment of the failure
thanks to like(), so this has no impact on debugging capabilities.
I have bumped into this issue while reviewing a different patch set
aiming at extending the kerberos test suite to check for multiple
log patterns instead of one now.
Author: Michael Paquier
Reviewed-by: Stephen Frost, Bharath Rupireddy
Discussion: https://postgr.es/m/YFXcq2vBTDGQVBNC@paquier.xyz
M src/test/kerberos/t/001_auth.pl
Fix timeline assignment in checkpoints with 2PC transactions
commit : 595b9cba2ab0cdd057e02d3c23f34a8bcfd90a2d
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Mar 2021 08:30:53 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Mar 2021 08:30:53 +0900
Any transactions found as still prepared by a checkpoint have their
state data read from the WAL records generated by PREPARE TRANSACTION
before being moved into their new location within pg_twophase/. While
reading such records, the WAL reader uses the callback
read_local_xlog_page() to read a page, that is shared across various
parts of the system. This callback, since 1148e22a, has introduced an
update of ThisTimeLineID when reading a record while in recovery, which
is potentially helpful in the context of cascading WAL senders.
This update of ThisTimeLineID interacts badly with the checkpointer if a
promotion happens while some 2PC data is read from its record, as, by
changing ThisTimeLineID, any follow-up WAL records would be written to
an timeline older than the promoted one. This results in consistency
issues. For instance, a subsequent server restart would cause a failure
in finding a valid checkpoint record, resulting in a PANIC, for
instance.
This commit changes the code reading the 2PC data to reset the timeline
once the 2PC record has been read, to prevent messing up with the static
state of the checkpointer. It would be tempting to do the same thing
directly in read_local_xlog_page(). However, based on the discussion
that has led to 1148e22a, users may rely on the updates of
ThisTimeLineID when a WAL record page is read in recovery, so changing
this callback could break some cases that are working currently.
A TAP test reproducing the issue is added, relying on a PITR to
precisely trigger a promotion with a prepared transaction still
tracked.
Per discussion with Heikki Linnakangas, Kyotaro Horiguchi, Fujii Masao
and myself.
Author: Soumyadeep Chakraborty, Jimmy Yih, Kevin Yeap
Discussion: https://postgr.es/m/CAE-ML+_EjH_fzfq1F3RJ1=XaaNG=-Jz-i3JqkNhXiLAsM3z-Ew@mail.gmail.com
Backpatch-through: 10
M src/backend/access/transam/twophase.c
A src/test/recovery/t/023_pitr_prepared_xact.pl
Fix assorted silliness in ATExecSetCompression().
commit : ac897c483485d3858ada23ca49650a0f2742a50f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Mar 2021 18:42:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Mar 2021 18:42:40 -0400
It's not okay to scribble directly on a syscache entry.
Nor to continue accessing said entry after releasing it.
Also get rid of not-used local variables.
Per valgrind testing.
M src/backend/commands/tablecmds.c
Recycle nbtree pages deleted during same VACUUM.
commit : 9dd963ae2534e9614f0abeccaafbd39f1b93ff8a
author : Peter Geoghegan <pg@bowt.ie>
date : Sun, 21 Mar 2021 15:25:39 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Sun, 21 Mar 2021 15:25:39 -0700
Maintain a simple array of metadata about pages that were deleted during
nbtree VACUUM's current btvacuumscan() call. Use this metadata at the
end of btvacuumscan() to attempt to place newly deleted pages in the FSM
without further delay. It might not yet be safe to place any of the
pages in the FSM by then (they may not be deemed recyclable), but we
have little to lose and plenty to gain by trying. In practice there is
a very good chance that this will work out when vacuuming larger
indexes, where scanning the index naturally takes quite a while.
This commit doesn't change the page recycling invariants; it merely
improves the efficiency of page recycling within the confines of the
existing design. Recycle safety is a part of nbtree's implementation of
what Lanin & Shasha call "the drain technique". The design happens to
use transaction IDs (they're stored in deleted pages), but that in
itself doesn't align the cutoff for recycle safety to any of the
XID-based cutoffs used by VACUUM (e.g., OldestXmin). All that matters
is whether or not _other_ backends might be able to observe various
inconsistencies in the tree structure (that they cannot just detect and
recover from by moving right). Recycle safety is purely a question of
maintaining the consistency (or the apparent consistency) of a physical
data structure.
Note that running a simple serial test case involving a large range
DELETE followed by a VACUUM VERBOSE will probably show that any newly
deleted nbtree pages are not yet reusable/recyclable. This is expected
in the absence of even one concurrent XID assignment. It is an old
implementation restriction. In practice it's unlikely to be the thing
that makes recycling remain unsafe, at least with larger indexes, where
recycling newly deleted pages during the same VACUUM actually matters.
An important high-level goal of this commit (as well as related recent
commits e5d8a999 and 9f3665fb) is to make expensive deferred cleanup
operations in index AMs rare in general. If index vacuuming frequently
depends on the next VACUUM operation finishing off work that the current
operation started, then the general behavior of index vacuuming is hard
to predict. This is relevant to ongoing work that adds a vacuumlazy.c
mechanism to skip index vacuuming in certain cases. Anything that makes
the real world behavior of index vacuuming simpler and more linear will
also make top-down modeling in vacuumlazy.c more robust.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wzk76_P=67iUscb1UN44-gyZL-KgpsXbSxq_bdcMa7Q+wQ@mail.gmail.com
M src/backend/access/nbtree/README
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/include/access/nbtree.h
Bring configure support for LZ4 up to snuff.
commit : 4d399a6fbeb720b34d33441330910b7d853f703d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Mar 2021 17:20:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Mar 2021 17:20:17 -0400
It's not okay to just shove the pkg_config results right into our
build flags, for a couple different reasons:
* This fails to maintain the separation between CPPFLAGS and CFLAGS,
as well as that between LDFLAGS and LIBS. (The CPPFLAGS angle is,
I believe, the reason for warning messages reported when building
with MacPorts' liblz4.)
* If pkg_config emits anything other than -I/-D/-L/-l switches,
it's highly unlikely that we want to absorb those. That'd be more
likely to break the build than do anything helpful. (Even the -D
case is questionable; but we're doing that for libxml2, so I kept it.)
Also, it's not okay to skip doing an AC_CHECK_LIB probe, as
evidenced by recent build failure on topminnow; that should
have been caught at configure time.
Model fixes for this on configure's libxml2 support.
It appears that somebody overlooked an autoheader run, too.
Discussion: https://postgr.es/m/20210119190720.GL8560@telsasoft.com
M configure
M configure.ac
M src/include/pg_config.h.in
M src/tools/msvc/Solution.pm
Make compression.sql regression test independent of default.
commit : fd1ac9a548966786cf7978e590be816c55936a50
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Mar 2021 16:26:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Mar 2021 16:26:44 -0400
This test will fail in "make installcheck" if the installation's
default_toast_compression setting is not 'pglz'. Make it robust
against that situation.
Dilip Kumar
Discussion: https://postgr.es/m/CAFiTN-t0w+Rc2U3S+y=7KWcLuOYNB5MfWeGdNa7+pg0UovVdcQ@mail.gmail.com
M src/test/regress/expected/compression.out
M src/test/regress/expected/compression_1.out
M src/test/regress/sql/compression.sql
Don't run recover crash_temp_files test in Windows perl
commit : ef823873840c9f341239e18633bdb0116d8d2738
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 21 Mar 2021 15:04:45 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 21 Mar 2021 15:04:45 -0400
This reverts commit 677271a3a125e294b33b891669f594a2c8cb36ce.
"Unbreak recovery test on Windows"
The test hangs on Windows, and attempts to remedy the problem have
proved fragile at best. So we simply disable the test on Windows perl.
(Msys perl seems perfectly happy).
Discussion: https://postgr.es/m/5b748470-7335-5439-e876-6a88c951e1c5@dunslane.net
M src/test/recovery/t/022_crash_temp_files.pl
Fix new memory leaks in libpq
commit : 2b526ed2e1cbaa54e5ad0c12d1294482f2757b17
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sun, 21 Mar 2021 14:55:27 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sun, 21 Mar 2021 14:55:27 -0300
My oversight in commit 9aa491abbf07.
Per coverity.
M src/interfaces/libpq/fe-exec.c
Unbreak recovery test on Windows
commit : 677271a3a125e294b33b891669f594a2c8cb36ce
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 21 Mar 2021 11:52:30 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 21 Mar 2021 11:52:30 -0400
On Windows we need to send explicit quit messages to psql or the TAP tests
can hang.
M src/test/recovery/t/022_crash_temp_files.pl
Suppress various new compiler warnings.
commit : 9fb9691a88ae8df9bc30e0f0f72de7c96e73e125
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Mar 2021 11:50:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Mar 2021 11:50:43 -0400
Compilers that don't understand that elog(ERROR) doesn't return
issued warnings here. In the cases in libpq_pipeline.c, we were
not exactly helping things by failing to mark pg_fatal() as noreturn.
Per buildfarm.
M src/backend/access/common/detoast.c
M src/backend/access/common/toast_compression.c
M src/include/access/toast_compression.h
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Move lwlock-release probe back where it belongs
commit : 96ae658e6238c5e69819fb1557c2c14a555506d8
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 21 Mar 2021 08:02:30 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 21 Mar 2021 08:02:30 +0100
The documentation specifically states that lwlock-release fires before
any released waiters have been awakened. It worked that way until
ab5194e6f617a9a9e7aadb3dd1cee948a42d0755, where is seems to have been
misplaced accidentally. Move it back where it belongs.
Author: Craig Ringer <craig.ringer@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/CAGRY4nwxKUS_RvXFW-ugrZBYxPFFM5kjwKT5O+0+Stuga5b4+Q@mail.gmail.com
M src/backend/storage/lmgr/lwlock.c
Use valid compression method in brin_form_tuple
commit : 882b2cdc08c4100e273f24742e2118be98708a07
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 21 Mar 2021 00:28:13 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 21 Mar 2021 00:28:13 +0100
When compressing the BRIN summary, we can't simply use the compression
method from the indexed attribute. The summary may use a different data
type, e.g. fixed-length attribute may have varlena summary, leading to
compression failures. For the built-in BRIN opclasses this happens to
work, because the summary uses the same data type as the attribute.
When the data types match, we can inherit use the compression method
specified for the attribute (it's copied into the index descriptor).
Otherwise we don't have much choice and have to use the default one.
Author: Tomas Vondra
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/e0367f27-392c-321a-7411-a58e1a7e4817%40enterprisedb.com
M src/backend/access/brin/brin_tuple.c
Fix up pg_dump's handling of per-attribute compression options.
commit : aa25d1089ac00bbc3f97d2efe8f54c3d4beed5d1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Mar 2021 15:01:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Mar 2021 15:01:10 -0400
The approach used in commit bbe0a81db would've been disastrous for
portability of dumps. Instead handle non-default compression options
in separate ALTER TABLE commands. This reduces chatter for the
common case where most columns are compressed the same way, and it
makes it possible to restore the dump to a server that lacks any
knowledge of per-attribute compression options (so long as you're
willing to ignore syntax errors from the ALTER TABLE commands).
There's a whole lot left to do to mop up after bbe0a81db, but
I'm fast-tracking this part because we need to see if it's
enough to make the buildfarm's cross-version-upgrade tests happy.
Justin Pryzby and Tom Lane
Discussion: https://postgr.es/m/20210119190720.GL8560@telsasoft.com
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
Fix memory leak when rejecting bogus DH parameters.
commit : e835e89a0fd267871e7fbddc39ad00ee3d0cb55c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Mar 2021 12:47:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Mar 2021 12:47:21 -0400
While back-patching e0e569e1d, I noted that there were some other
places where we ought to be applying DH_free(); namely, where we
load some DH parameters from a file and then reject them as not
being sufficiently secure. While it seems really unlikely that
anybody would hit these code paths in production, let alone do
so repeatedly, let's fix it for consistency.
Back-patch to v10 where this code was introduced.
Discussion: https://postgr.es/m/16160-18367e56e9a28264@postgresql.org
M src/backend/libpq/be-secure-openssl.c
Avoid leaking memory in RestoreGUCState(), and improve comments.
commit : f0c2a5bba6c566fad781802537eb17f2977702bc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 19 Mar 2021 23:03:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 19 Mar 2021 23:03:17 -0400
RestoreGUCState applied InitializeOneGUCOption to already-live
GUC entries, causing any malloc'd subsidiary data to be forgotten.
We do want the effect of resetting the GUC to its compiled-in
default, and InitializeOneGUCOption seems like the best way to do
that, so add code to free any existing subsidiary data beforehand.
The interaction between can_skip_gucvar, SerializeGUCState, and
RestoreGUCState is way more subtle than their opaque comments
would suggest to an unwary reader. Rewrite and enlarge the
comments to try to make it clearer what's happening.
Remove a long-obsolete assertion in read_nondefault_variables: the
behavior of set_config_option hasn't depended on IsInitProcessingMode
since f5d9698a8 installed a better way of controlling it.
Although this is fixing a clear memory leak, the leak is quite unlikely
to involve any large amount of data, and it can only happen once in the
lifetime of a worker process. So it seems unnecessary to take any
risk of back-patching.
Discussion: https://postgr.es/m/4105247.1616174862@sss.pgh.pa.us
M src/backend/utils/misc/guc.c
Provide recovery_init_sync_method=syncfs.
commit : 61752afb26404dfc99a535c7a53f7f04dc110263
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 20 Mar 2021 11:46:32 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 20 Mar 2021 11:46:32 +1300
Since commit 2ce439f3 we have opened every file in the data directory
and called fsync() at the start of crash recovery. This can be very
slow if there are many files, leading to field complaints of systems
taking minutes or even hours to begin crash recovery.
Provide an alternative method, for Linux only, where we call syncfs() on
every possibly different filesystem under the data directory. This is
equivalent, but avoids faulting in potentially many inodes from
potentially slow storage.
The new mode comes with some caveats, described in the documentation, so
the default value for the new setting is "fsync", preserving the older
behavior.
Reported-by: Michael Brown <michael.brown@discourse.org>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Paul Guo <guopa@vmware.com>
Reviewed-by: Bruce Momjian <bruce@momjian.us>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: David Steele <david@pgmasters.net>
Discussion: https://postgr.es/m/11bc2bb7-ecb5-3ad0-b39f-df632734cd81%40discourse.org
Discussion: https://postgr.es/m/CAEET0ZHGnbXmi8yF3ywsDZvb3m9CbdsGZgfTXscQ6agcbzcZAw%40mail.gmail.com
M configure
M configure.ac
M doc/src/sgml/config.sgml
M src/backend/storage/file/fd.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/pg_config.h.in
M src/include/storage/fd.h
M src/tools/msvc/Solution.pm
Use lfirst_int in cmp_list_len_contents_asc
commit : b822ae13ea93c18326d58d47829bbc66d36fae5c
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 19 Mar 2021 23:57:50 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 19 Mar 2021 23:57:50 +0100
The function added in be45be9c33 is comparing integer lists (IntList) by
length and contents, but there were two bugs. Firstly, it used intVal()
to extract the value, but that's for Value nodes, not for extracting int
values from IntList. Secondly, it called it directly on the ListCell,
without doing lfirst(). So just do lfirst_int() instead.
Interestingly enough, this did not cause any crashes on the buildfarm,
but valgrind rightfully complained about it.
Discussion: https://postgr.es/m/bf3805a8-d7d1-ae61-fece-761b7ff41ecc@postgresfriends.org
M src/backend/parser/parse_agg.c
Fix use-after-ReleaseSysCache problem in ATExecAlterColumnType.
commit : d00fbdc431192c3e429b3e91c43d364e5c7ba680
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 19 Mar 2021 17:17:48 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 19 Mar 2021 17:17:48 -0400
Introduced by commit bbe0a81db69bd10bd166907c3701492a29aca294.
Per buildfarm member prion.
M src/backend/commands/tablecmds.c
Allow configurable LZ4 TOAST compression.
commit : bbe0a81db69bd10bd166907c3701492a29aca294
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 19 Mar 2021 15:10:38 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 19 Mar 2021 15:10:38 -0400
There is now a per-column COMPRESSION option which can be set to pglz
(the default, and the only option in up until now) or lz4. Or, if you
like, you can set the new default_toast_compression GUC to lz4, and
then that will be the default for new table columns for which no value
is specified. We don't have lz4 support in the PostgreSQL code, so
to use lz4 compression, PostgreSQL must be built --with-lz4.
In general, TOAST compression means compression of individual column
values, not the whole tuple, and those values can either be compressed
inline within the tuple or compressed and then stored externally in
the TOAST table, so those properties also apply to this feature.
Prior to this commit, a TOAST pointer has two unused bits as part of
the va_extsize field, and a compessed datum has two unused bits as
part of the va_rawsize field. These bits are unused because the length
of a varlena is limited to 1GB; we now use them to indicate the
compression type that was used. This means we only have bit space for
2 more built-in compresison types, but we could work around that
problem, if necessary, by introducing a new vartag_external value for
any further types we end up wanting to add. Hopefully, it won't be
too important to offer a wide selection of algorithms here, since
each one we add not only takes more coding but also adds a build
dependency for every packager. Nevertheless, it seems worth doing
at least this much, because LZ4 gets better compression than PGLZ
with less CPU usage.
It's possible for LZ4-compressed datums to leak into composite type
values stored on disk, just as it is for PGLZ. It's also possible for
LZ4-compressed attributes to be copied into a different table via SQL
commands such as CREATE TABLE AS or INSERT .. SELECT. It would be
expensive to force such values to be decompressed, so PostgreSQL has
never done so. For the same reasons, we also don't force recompression
of already-compressed values even if the target table prefers a
different compression method than was used for the source data. These
architectural decisions are perhaps arguable but revisiting them is
well beyond the scope of what seemed possible to do as part of this
project. However, it's relatively cheap to recompress as part of
VACUUM FULL or CLUSTER, so this commit adjusts those commands to do
so, if the configured compression method of the table happens not to
match what was used for some column value stored therein.
Dilip Kumar. The original patches on which this work was based were
written by Ildus Kurbangaliev, and those were patches were based on
even earlier work by Nikita Glukhov, but the design has since changed
very substantially, since allow a potentially large number of
compression methods that could be added and dropped on a running
system proved too problematic given some of the architectural issues
mentioned above; the choice of which specific compression method to
add first is now different; and a lot of the code has been heavily
refactored. More recently, Justin Przyby helped quite a bit with
testing and reviewing and this version also includes some code
contributions from him. Other design input and review from Tomas
Vondra, Álvaro Herrera, Andres Freund, Oleg Bartunov, Alexander
Korotkov, and me.
Discussion: http://postgr.es/m/20170907194236.4cefce96%40wp.localdomain
Discussion: http://postgr.es/m/CAFiTN-uUpX3ck%3DK0mLEk-G_kUQY%3DSNOTeqdaNRR9FMdQrHKebw%40mail.gmail.com
M configure
M configure.ac
M contrib/amcheck/verify_heapam.c
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/psql-ref.sgml
M src/backend/access/brin/brin_tuple.c
M src/backend/access/common/Makefile
M src/backend/access/common/detoast.c
M src/backend/access/common/indextuple.c
A src/backend/access/common/toast_compression.c
M src/backend/access/common/toast_internals.c
M src/backend/access/common/tupdesc.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/table/toast_helper.c
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/toasting.c
M src/backend/commands/tablecmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_utilcmd.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/utils/adt/varlena.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/psql/describe.c
M src/bin/psql/help.c
M src/bin/psql/settings.h
M src/bin/psql/startup.c
M src/bin/psql/tab-complete.c
M src/include/access/detoast.h
A src/include/access/toast_compression.h
M src/include/access/toast_helper.h
M src/include/access/toast_internals.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_proc.dat
M src/include/nodes/parsenodes.h
M src/include/parser/kwlist.h
M src/include/pg_config.h.in
M src/include/postgres.h
A src/test/regress/expected/compression.out
A src/test/regress/expected/compression_1.out
M src/test/regress/parallel_schedule
M src/test/regress/pg_regress_main.c
M src/test/regress/serial_schedule
A src/test/regress/sql/compression.sql
M src/tools/msvc/Solution.pm
Fix race condition in remove_temp_files_after_crash TAP test
commit : e589c4890b05044a04207c2797e7c8af6693ea5f
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 19 Mar 2021 18:12:39 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 19 Mar 2021 18:12:39 +0100
The TAP test was written so that it was not waiting for the correct SQL
command, but for output from the preceding one. This resulted in race
conditions, allowing the commands to run in a different order, not block
as expected and so on. This fixes it by inverting the order of commands
where possible, so that observing the output guarantees the data was
inserted properly, and waiting for a lock to appear in pg_locks.
Discussion: https://postgr.es/m/CAH503wDKdYzyq7U-QJqGn%3DGm6XmoK%2B6_6xTJ-Yn5WSvoHLY1Ww%40mail.gmail.com
M src/test/recovery/t/022_crash_temp_files.pl
Blindly try to fix test script's tar invocation for MSYS.
commit : 27ab1981e7c9b8fcbcb143c5f6f706441a52bbc8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 22:43:03 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 22:43:03 -0400
Buildfarm member fairywren doesn't like the test case I added
in commit 081876d75. I'm guessing the reason is that I shouldn't
be using a perl2host-ified path in the tar command line.
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
Fix comments in postmaster.c.
commit : fd31214075cc740e43edc71ca1c385c8c53047b7
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 19 Mar 2021 11:28:54 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 19 Mar 2021 11:28:54 +0900
Commit 86c23a6eb2 changed the option to specify that postgres will
stop all other server processes by sending the signal SIGSTOP,
from -s to -T. But previously there were comments incorrectly
explaining that SIGSTOP behavior is set by -s option. This commit
fixes them.
Author: Kyotaro Horiguchi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/20210316.165141.1400441966284654043.horikyota.ntt@gmail.com
M src/backend/postmaster/postmaster.c
Don't leak malloc'd error string in libpqrcv_check_conninfo().
commit : 9bacdf9f536a3720976ae258238cb46c691ce9b2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 22:21:58 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 22:21:58 -0400
We leaked the error report from PQconninfoParse, when there was
one. It seems unlikely that real usage patterns would repeat
the failure often enough to create serious bloat, but let's
back-patch anyway to keep the code similar in all branches.
Found via valgrind testing.
Back-patch to v10 where this code was added.
Discussion: https://postgr.es/m/3816764.1616104288@sss.pgh.pa.us
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
Don't leak malloc'd strings when a GUC setting is rejected.
commit : 377b7a83007d277d32ef19f7c7590c8668d504cb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 22:09:41 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 22:09:41 -0400
Because guc.c prefers to keep all its string values in malloc'd
not palloc'd storage, it has to be more careful than usual to
avoid leaks. Error exits out of string GUC hook checks failed
to clear the proposed value string, and error exits out of
ProcessGUCArray() failed to clear the malloc'd results of
ParseLongOption().
Found via valgrind testing.
This problem is ancient, so back-patch to all supported branches.
Discussion: https://postgr.es/m/3816764.1616104288@sss.pgh.pa.us
M src/backend/utils/misc/guc.c
Don't leak compiled regex(es) when an ispell cache entry is dropped.
commit : d303849b059c3c315e5a8d4239016f8328f3296c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 21:44:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 21:44:42 -0400
The text search cache mechanisms assume that we can clean up
an invalidated dictionary cache entry simply by resetting the
associated long-lived memory context. However, that does not work
for ispell affixes that make use of regular expressions, because
the regex library deals in plain old malloc. Hence, we leaked
compiled regex(es) any time we dropped such a cache entry. That
could quickly add up, since even a fairly trivial regex can use up
tens of kB, and a large one can eat megabytes. Add a memory context
callback to ensure that a regex gets freed when its owning cache
entry is cleared.
Found via valgrind testing.
This problem is ancient, so back-patch to all supported branches.
Discussion: https://postgr.es/m/3816764.1616104288@sss.pgh.pa.us
M src/backend/tsearch/spell.c
M src/include/tsearch/dicts/spell.h
Don't run RelationInitTableAccessMethod in a long-lived context.
commit : 415ffdc2205e209b6a73fb42a3fdd6e57e16c7b2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 20:50:56 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 20:50:56 -0400
Some code paths in this function perform syscache lookups, which
can lead to table accesses and possibly leakage of cruft into
the caller's context. If said context is CacheMemoryContext,
we eventually will have visible bloat. But fixing this is no
harder than moving one memory context switch step. (The other
callers don't have a problem.)
Andres Freund and I independently found this via valgrind testing.
Back-patch to v12 where this code was added.
Discussion: https://postgr.es/m/20210317023101.anvejcfotwka6gaa@alap3.anarazel.de
Discussion: https://postgr.es/m/3816764.1616104288@sss.pgh.pa.us
M src/backend/utils/cache/relcache.c
Don't leak rd_statlist when a relcache entry is dropped.
commit : 28644fac10731e30e70b622986a6fbbeb5a5b2f9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 20:37:09 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 20:37:09 -0400
Although these lists are usually NIL, and even when not empty
are unlikely to be large, constant relcache update traffic could
eventually result in visible bloat of CacheMemoryContext.
Found via valgrind testing.
Back-patch to v10 where this field was added.
Discussion: https://postgr.es/m/3816764.1616104288@sss.pgh.pa.us
M src/backend/utils/cache/relcache.c
Fix TAP test for remove_temp_files_after_crash
commit : a16b2b960f0eec2fe367e86017b3c24ed688ba2b
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 19 Mar 2021 02:05:23 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 19 Mar 2021 02:05:23 +0100
The test included in cd91de0d17 had two simple flaws.
Firstly, the number of rows was low and on some platforms (e.g. 32-bit)
the sort did not require on-disk sort, so on those machines it was not
testing the automatic removal. The test was however failing, because
without any temporary files the base/pgsql_tmp directory was not even
created. Fixed by increasing the rowcount to 5000, which should be high
engough on any platform.
Secondly, the test used a simple sleep to wait for the temporary file to
be created. This is obviously problematic, because on slow machines (or
with valgrind, CLOBBER_CACHE_ALWAYS etc.) it may take a while to create
the temporary file. But we also want the tests run reasonably fast.
Fixed by instead relying on a UNIQUE constraint, blocking the query that
created the temporary file.
Author: Euler Taveira
Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/CAH503wDKdYzyq7U-QJqGn%3DGm6XmoK%2B6_6xTJ-Yn5WSvoHLY1Ww%40mail.gmail.com
M src/test/recovery/t/022_crash_temp_files.pl
Improve tab completion of IMPORT FOREIGN SCHEMA with \h in psql
commit : 5b2266e33fc74142d23685bdf54f64ad598fbdea
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 19 Mar 2021 09:18:41 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 19 Mar 2021 09:18:41 +0900
Only "IMPORT" was showing as result of the completion, while IMPORT
FOREIGN SCHEMA is the only command using this keyword in first
position. This changes the completion to show the full command name
instead of just "IMPORT".
Reviewed-by: Georgios Kokolatos, Julien Rouhaud
Discussion: https://postgr.es/m/YFL6JneBiuMWYyoh@paquier.xyz
M src/bin/psql/tab-complete.c
Fix misuse of foreach_delete_current().
commit : 1d581ce7129d7a33cd4ad27f8f246abfa1fd2db9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 19:24:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 19:24:22 -0400
Our coding convention requires this macro's result to be assigned
back to the original List variable. In this usage, since the
List could not become empty, there was no actual bug --- but
some compilers warned about it. Oversight in be45be9c3.
Discussion: https://postgr.es/m/35077b31-2d62-1e31-0e2e-ddb52d590b73@enterprisedb.com
M src/backend/parser/parse_agg.c
Implement GROUP BY DISTINCT
commit : be45be9c33a85e72cdaeb9967e9f6d2d00199e09
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 18 Mar 2021 17:45:38 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 18 Mar 2021 17:45:38 +0100
With grouping sets, it's possible that some of the grouping sets are
duplicate. This is especially common with CUBE and ROLLUP clauses. For
example GROUP BY CUBE (a,b), CUBE (b,c) is equivalent to
GROUP BY GROUPING SETS (
(a, b, c),
(a, b, c),
(a, b, c),
(a, b),
(a, b),
(a, b),
(a),
(a),
(a),
(c, a),
(c, a),
(c, a),
(c),
(b, c),
(b),
()
)
Some of the grouping sets are calculated multiple times, which is mostly
unnecessary. This commit implements a new GROUP BY DISTINCT feature, as
defined in the SQL standard, which eliminates the duplicate sets.
Author: Vik Fearing
Reviewed-by: Erik Rijkers, Georgios Kokolatos, Tomas Vondra
Discussion: https://postgr.es/m/bf3805a8-d7d1-ae61-fece-761b7ff41ecc@postgresfriends.org
M doc/src/sgml/queries.sgml
M doc/src/sgml/ref/select.sgml
M src/backend/catalog/sql_features.txt
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/list.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/plan/planner.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/parser/parse_agg.c
M src/backend/utils/adt/ruleutils.c
M src/include/nodes/parsenodes.h
M src/include/nodes/pg_list.h
M src/include/parser/parse_agg.h
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Remove temporary files after backend crash
commit : cd91de0d17952b5763466cfa663e98318f26d357
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 18 Mar 2021 16:05:03 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 18 Mar 2021 16:05:03 +0100
After a crash of a backend using temporary files, the files used to be
left behind, on the basis that it might be useful for debugging. But we
don't have any reports of anyone actually doing that, and it means the
disk usage may grow over time due to repeated backend failures (possibly
even hitting ENOSPC). So this behavior is a bit unfortunate, and fixing
it required either manual cleanup (deleting files, which is error-prone)
or restart of the instance (i.e. service disruption).
This implements automatic cleanup of temporary files, controled by a new
GUC remove_temp_files_after_crash. By default the files are removed, but
it can be disabled to restore the old behavior if needed.
Author: Euler Taveira
Reviewed-by: Tomas Vondra, Michael Paquier, Anastasia Lubennikova, Thomas Munro
Discussion: https://postgr.es/m/CAH503wDKdYzyq7U-QJqGn%3DGm6XmoK%2B6_6xTJ-Yn5WSvoHLY1Ww%40mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/postmaster/postmaster.c
M src/backend/storage/file/fd.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/postmaster/postmaster.h
A src/test/recovery/t/022_crash_temp_files.pl
Fix function name in error hint
commit : da18d829c28197efb04805a43f129f62650e50c8
author : Magnus Hagander <magnus@hagander.net>
date : Thu, 18 Mar 2021 11:17:42 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Thu, 18 Mar 2021 11:17:42 +0100
pg_read_file() is the function that's in core, pg_file_read() is in
adminpack. But when using pg_file_read() in adminpack it calls the *C*
level function pg_read_file() in core, which probably threw the original
author off. But the error hint should be about the SQL function.
Reported-By: Sergei Kornilov
Backpatch-through: 11
Discussion: https://postgr.es/m/373021616060475@mail.yandex.ru
M src/backend/utils/adt/genfile.c
Doc: Update description for parallel insert reloption.
commit : ed62d3737c1b823f796d974060b1d0295a3dd831
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 18 Mar 2021 15:34:55 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 18 Mar 2021 15:34:55 +0530
Commit c8f78b6161 added a new reloption to enable inserts in parallel-mode
but forgot to update at one of the places about the same in docs. In
passing, fix a typo in the same commit.
Reported-by: Justin Pryzby
Author: Justin Pryzby
Reviewed-by: "Hou, Zhijie", Amit Kapila
Discussion: https://postgr.es/m/20210318025228.GE11765@telsasoft.com
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_table.sgml
Add a new GUC and a reloption to enable inserts in parallel-mode.
commit : c8f78b616167bf8e24bc5dc69112c37755ed3058
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 18 Mar 2021 07:25:27 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 18 Mar 2021 07:25:27 +0530
Commit 05c8482f7f added the implementation of parallel SELECT for
"INSERT INTO ... SELECT ..." which may incur non-negligible overhead in
the additional parallel-safety checks that it performs, even when, in the
end, those checks determine that parallelism can't be used. This is
normally only ever a problem in the case of when the target table has a
large number of partitions.
A new GUC option "enable_parallel_insert" is added, to allow insert in
parallel-mode. The default is on.
In addition to the GUC option, the user may want a mechanism to allow
inserts in parallel-mode with finer granularity at table level. The new
table option "parallel_insert_enabled" allows this. The default is true.
Author: "Hou, Zhijie"
Reviewed-by: Greg Nancarrow, Amit Langote, Takayuki Tsunakawa, Amit Kapila
Discussion: https://postgr.es/m/CAA4eK1K-cW7svLC2D7DHoGHxdAdg3P37BLgebqBOC2ZLc9a6QQ%40mail.gmail.com
Discussion: https://postgr.es/m/CAJcOf-cXnB5cnMKqWEp2E2z7Mvcd04iLVmV=qpFJrR3AcrTS3g@mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_table.sgml
M src/backend/access/common/reloptions.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/util/clauses.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/psql/tab-complete.c
M src/include/optimizer/cost.h
M src/include/utils/rel.h
M src/test/regress/expected/insert_parallel.out
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/insert_parallel.sql
M src/tools/pgindent/typedefs.list
Fix memory lifetime issues of replication slot stats.
commit : 5f79580ad69f6e696365bdc63bc265f45bd77211
author : Andres Freund <andres@anarazel.de>
date : Wed, 17 Mar 2021 16:18:37 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 17 Mar 2021 16:18:37 -0700
When accessing replication slot stats, introduced in 98681675002d,
pgstat_read_statsfiles() reads the data into newly allocated
memory. Unfortunately the current memory context at that point is the
callers, leading to leaks and use-after-free dangers.
The fix is trivial, explicitly use pgStatLocalContext. There's some
potential for further improvements, but that's outside of the scope of
this bugfix.
No backpatch necessary, feature is only in HEAD.
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20210317230447.c7uc4g3vbs4wi32i@alap3.anarazel.de
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
M src/backend/postmaster/pgstat.c
Doc: remove duplicated step in RLS example.
commit : 70945649d734d16be22c3d1d90dd8c3d3c1e9d89
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Mar 2021 16:39:58 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Mar 2021 16:39:58 -0400
Seems to have been a copy-and-paste mistake in 093129c9d.
Per report from max1@inbox.ru.
Discussion: https://postgr.es/m/161591740692.24273.4202054598867879464@wrigleys.postgresql.org
M doc/src/sgml/ddl.sgml
Code review for server's handling of "tablespace map" files.
commit : 8620a7f6dbdf978e57cdb9f42802a0418656d863
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Mar 2021 16:18:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Mar 2021 16:18:46 -0400
While looking at Robert Foggia's report, I noticed a passel of
other issues in the same area:
* The scheme for backslash-quoting newlines in pathnames is just
wrong; it will misbehave if the last ordinary character in a pathname
is a backslash. I'm not sure why we're bothering to allow newlines
in tablespace paths, but if we're going to do it we should do it
without introducing other problems. Hence, backslashes themselves
have to be backslashed too.
* The author hadn't read the sscanf man page very carefully, because
this code would drop any leading whitespace from the path. (I doubt
that a tablespace path with leading whitespace could happen in
practice; but if we're bothering to allow newlines in the path, it
sure seems like leading whitespace is little less implausible.) Using
sscanf for the task of finding the first space is overkill anyway.
* While I'm not 100% sure what the rationale for escaping both \r and
\n is, if the idea is to allow Windows newlines in the file then this
code failed, because it'd throw an error if it saw \r followed by \n.
* There's no cross-check for an incomplete final line in the map file,
which would be a likely apparent symptom of the improper-escaping
bug.
On the generation end, aside from the escaping issue we have:
* If needtblspcmapfile is true then do_pg_start_backup will pass back
escaped strings in tablespaceinfo->path values, which no caller wants
or is prepared to deal with. I'm not sure if there's a live bug from
that, but it looks like there might be (given the dubious assumption
that anyone actually has newlines in their tablespace paths).
* It's not being very paranoid about the possibility of random stuff
in the pg_tblspc directory. IMO we should ignore anything without an
OID-like name.
The escaping rule change doesn't seem back-patchable: it'll require
doubling of backslashes in the tablespace_map file, which is basically
a basebackup format change. The odds of that causing trouble are
considerably more than the odds of the existing bug causing trouble.
The rest of this seems somewhat unlikely to cause problems too,
so no back-patch.
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogfuncs.c
M src/backend/replication/basebackup.c
M src/include/access/xlog.h
M src/include/replication/basebackup.h
Prevent buffer overrun in read_tablespace_map().
commit : a50e4fd028a1ece2b1a04df2c9ae6581783e9eef
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Mar 2021 16:10:37 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Mar 2021 16:10:37 -0400
Robert Foggia of Trustwave reported that read_tablespace_map()
fails to prevent an overrun of its on-stack input buffer.
Since the tablespace map file is presumed trustworthy, this does
not seem like an interesting security vulnerability, but still
we should fix it just in the name of robustness.
While here, document that pg_basebackup's --tablespace-mapping option
doesn't work with tar-format output, because it doesn't. To make it
work, we'd have to modify the tablespace_map file within the tarball
sent by the server, which might be possible but I'm not volunteering.
(Less-painful solutions would require changing the basebackup protocol
so that the source server could adjust the map. That's not very
appetizing either.)
M doc/src/sgml/ref/pg_basebackup.sgml
M src/backend/access/transam/xlog.c
Add end-to-end testing of pg_basebackup's tar-format output.
commit : 081876d75ea15c3bd2ee5ba64a794fd8ea46d794
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Mar 2021 14:52:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Mar 2021 14:52:55 -0400
The existing test script does run pg_basebackup with the -Ft option,
but it makes no real attempt to verify the sanity of the results.
We wouldn't know if the output is incompatible with standard "tar"
programs, nor if the server fails to start from the restored output.
Notably, this means that xlog.c's read_tablespace_map() is not being
meaningfully tested, since that code is used only in the tar-format
case. (We do have reasonable coverage of restoring from plain-format
output, though it's over in src/test/recovery not here.)
Hence, attempt to untar the output and start a server from it,
rather just hoping it's OK.
This test assumes that the local "tar" has the "-C directory"
switch. Although that's not promised by POSIX, my research
suggests that all non-extinct tar implementations have it.
Should the buildfarm's opinion differ, we can complicate the
test a bit to avoid requiring that.
Possibly this should be back-patched, but I'm unsure about
whether it could work on Windows before d66b23b03.
M src/bin/pg_basebackup/Makefile
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/test/perl/PostgresNode.pm
Doc: improve discussion of variable substitution in PL/pgSQL.
commit : c783e656d41816b0328cb4bff27f11b70200770e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Mar 2021 13:09:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Mar 2021 13:09:13 -0400
This was a bit disjointed, partly because of a not-well-considered
decision to document SQL commands that don't return result rows as
though they had nothing in common with commands that do. Rearrange
so that we have one discussion of variable substitution that clearly
applies to all types of SQL commands, and then handle the question
of processing command output separately. Clarify that EXPLAIN,
CREATE TABLE AS SELECT, and similar commands that incorporate an
optimizable statement will act like optimizable statements for the
purposes of variable substitution. Do a bunch of minor wordsmithing
in the same area.
David Johnston and Tom Lane, reviewed by Pavel Stehule and David
Steele
Discussion: https://postgr.es/m/CAKFQuwYvMKucM5fnZvHSo-ah4S=_n9gmKeu6EAo=_fTrohunqQ@mail.gmail.com
M doc/src/sgml/plpgsql.sgml
Revert "Fix race in Parallel Hash Join batch cleanup."
commit : 7f7f25f15edb6eacec58179ef5285e874aa4435b
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 18 Mar 2021 00:35:04 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 18 Mar 2021 00:35:04 +1300
This reverts commit 378802e3713c6c0fce31d2390c134cd5d7c30157.
This reverts commit 3b8981b6e1a2aea0f18384c803e21e9391de669a.
Discussion: https://postgr.es/m/CA%2BhUKGJmcqAE3MZeDCLLXa62cWM0AJbKmp2JrJYaJ86bz36LFA%40mail.gmail.com
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeHashjoin.c
M src/backend/postmaster/pgstat.c
M src/include/executor/hashjoin.h
M src/include/pgstat.h
Fix comment in indexing.c
commit : 9fd2952cf4920d563e9cea51634c5b364d57f71a
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 17 Mar 2021 18:07:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 17 Mar 2021 18:07:00 +0900
578b229, that removed support for WITH OIDS, has changed
CatalogTupleInsert() to not return an Oid, but one comment was still
mentioning that.
Author: Vik Fearing
Discussion: https://postgr.es/m/fef01975-ed10-3601-7b9e-80ecef72d00b@postgresfriends.org
M src/backend/catalog/indexing.c
Small error message improvement
commit : e1ae40f381d0582981b1e63856bd4b060cfe2d53
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 17 Mar 2021 08:17:33 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 17 Mar 2021 08:17:33 +0100
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_data.out
Update the names of Parallel Hash Join phases.
commit : 378802e3713c6c0fce31d2390c134cd5d7c30157
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 17 Mar 2021 18:24:45 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 17 Mar 2021 18:24:45 +1300
Commit 3048898e dropped -ING from some wait event names that correspond
to barrier phases. Update the phases' names to match.
While we're here making cosmetic changes, also rename "DONE" to "FREE".
That pairs better with "ALLOCATE", and describes the activity that
actually happens in that phase (as we do for the other phases) rather
than describing a state. The distinction is clearer after bugfix commit
3b8981b6 split the phase into two. As for the growth barriers, rename
their "ALLOCATE" phase to "REALLOCATE", which is probably a better
description of what happens then. Also improve the comments about
the phases a bit.
Discussion: https://postgr.es/m/CA%2BhUKG%2BMDpwF2Eo2LAvzd%3DpOh81wUTsrwU1uAwR-v6OGBB6%2B7g%40mail.gmail.com
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeHashjoin.c
M src/backend/postmaster/pgstat.c
M src/include/executor/hashjoin.h
M src/include/pgstat.h
Fix race in Parallel Hash Join batch cleanup.
commit : 3b8981b6e1a2aea0f18384c803e21e9391de669a
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 17 Mar 2021 17:46:39 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 17 Mar 2021 17:46:39 +1300
With very unlucky timing and parallel_leader_participation off, PHJ
could attempt to access per-batch state just as it was being freed.
There was code intended to prevent that by checking for a cleared
pointer, but it was buggy.
Fix, by introducing an extra barrier phase. The new phase
PHJ_BUILD_RUNNING means that it's safe to access the per-batch state to
find a batch to help with, and PHJ_BUILD_DONE means that it is too late.
The last to detach will free the array of per-batch state as before, but
now it will also atomically advance the phase at the same time, so that
late attachers can avoid the hazard, without the data race. This
mirrors the way per-batch hash tables are freed (see phases
PHJ_BATCH_PROBING and PHJ_BATCH_DONE).
Revealed by a one-off build farm failure, where BarrierAttach() failed a
sanity check assertion, because the memory had been clobbered by
dsa_free().
Back-patch to 11, where the code arrived.
Reported-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20200929061142.GA29096%40paquier.xyz
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeHashjoin.c
M src/include/executor/hashjoin.h
Fix transaction.sql tests in higher isolation levels.
commit : 37929599499fe5760a9dbab48a10a898879a0d44
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 17 Mar 2021 17:13:43 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 17 Mar 2021 17:13:43 +1300
It seems like a useful sanity check to be able to run "installcheck"
against a cluster running with default_transaction_level set to
serializable or repeatable read. Only one thing currently fails in
those configurations, so let's fix that.
No back-patch for now, because it fails in many other places in some of
the stable branches. We'd have to go back and fix those too if we
included this configuration in automated testing.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA%2BhUKGJUaHeK%3DHLATxF1JOKDjKJVrBKA-zmbPAebOM0Se2FQRg%40mail.gmail.com
M src/test/regress/expected/transactions.out
M src/test/regress/sql/transactions.sql
Fix race condition in drop subscription's handling of tablesync slots.
commit : 6b67d72b604cb913e39324b81b61ab194d94cba0
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 17 Mar 2021 08:15:12 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 17 Mar 2021 08:15:12 +0530
Commit ce0fdbfe97 made tablesync slots permanent and allow Drop
Subscription to drop such slots. However, it is possible that before
tablesync worker could get the acknowledgment of slot creation, drop
subscription stops it and that can lead to a dangling slot on the
publisher. Prevent cancel/die interrupts while creating a slot in the
tablesync worker.
Reported-by: Thomas Munro as per buildfarm
Author: Amit Kapila
Reviewed-by: Vignesh C, Takamichi Osumi
Discussion: https://postgr.es/m/CA+hUKGJG9dWpw1cOQ2nzWU8PHjm=PTraB+KgE5648K9nTfwvxg@mail.gmail.com
M src/backend/replication/logical/tablesync.c
Doc: Add a description of substream in pg_subscription.
commit : 7efeb214ad832fa96ea950d0906b1d2b96316d15
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 17 Mar 2021 07:40:23 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 17 Mar 2021 07:40:23 +0530
Commit 464824323e added a new column substream in pg_subscription but
forgot to update the docs.
Reported-by: Peter Smith
Author: Amit Kapila
Reviewed-by: Peter Smith
Discussion: https://postgr.es/m/CAHut+PuPGGASnh2Dy37VYODKULVQo-5oE=Shc6gwtRizDt==cA@mail.gmail.com
M doc/src/sgml/catalogs.sgml
Enable parallelism in REFRESH MATERIALIZED VIEW.
commit : 9e7ccd9ef64d05e87ceb1985d459bef9031205c0
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 17 Mar 2021 13:43:08 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 17 Mar 2021 13:43:08 +1300
Pass CURSOR_OPT_PARALLEL_OK to pg_plan_query() so that parallel plans
are considered when running the underlying SELECT query. This wasn't
done in commit e9baa5e9, which did this for CREATE MATERIALIZED VIEW,
because it wasn't yet known to be safe.
Since REFRESH always inserts into a freshly created table before later
merging or swapping the data into place with separate operations, we can
enable such plans here too.
Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Hou, Zhijie <houzj.fnst@cn.fujitsu.com>
Reviewed-by: Luc Vlaming <luc@swarm64.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CALj2ACXg-4hNKJC6nFnepRHYT4t5jJVstYvri%2BtKQHy7ydcr8A%40mail.gmail.com
M doc/src/sgml/parallel.sgml
M src/backend/commands/matview.c
M src/test/regress/expected/write_parallel.out
M src/test/regress/sql/write_parallel.sql
Fix comment about promising tuples.
commit : fbe4cb3bd49f9e524f53ef77c775c1bad4d0312a
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 16 Mar 2021 13:38:52 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 16 Mar 2021 13:38:52 -0700
Oversight in commit d168b666823, which added bottom-up index deletion.
M src/backend/access/heap/heapam.c
amcheck: Reduce debug message verbosity.
commit : 65445469d6de3670f3e027236613e093e119ec55
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 16 Mar 2021 13:11:17 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 16 Mar 2021 13:11:17 -0700
Empty sibling pages can occasionally be much more common than any other
event that we report on at elevel DEBUG1. Increase the elevel for
relevant cases to DEBUG2 to avoid overwhelming the user with relatively
insignificant details.
M contrib/amcheck/verify_nbtree.c
Avoid corner-case memory leak in SSL parameter processing.
commit : 4b12ab18c9d0735d760bf7667b158707b06e5df8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Mar 2021 16:02:49 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Mar 2021 16:02:49 -0400
After reading the root cert list from the ssl_ca_file, immediately
install it as client CA list of the new SSL context. That gives the
SSL context ownership of the list, so that SSL_CTX_free will free it.
This avoids a permanent memory leak if we fail further down in
be_tls_init(), which could happen if bogus CRL data is offered.
The leak could only amount to something if the CRL parameters get
broken after server start (else we'd just quit) and then the server
is SIGHUP'd many times without fixing the CRL data. That's rather
unlikely perhaps, but it seems worth fixing, if only because the
code is clearer this way.
While we're here, add some comments about the memory management
aspects of this logic.
Noted by Jelte Fennema and independently by Andres Freund.
Back-patch to v10; before commit de41869b6 it doesn't matter,
since we'd not re-execute this code during SIGHUP.
Discussion: https://postgr.es/m/16160-18367e56e9a28264@postgresql.org
M src/backend/libpq/be-secure-openssl.c
Fix a confusing amcheck corruption message.
commit : 4078ce65a0f7197180a9be2c6460ea4bf909bd98
author : Robert Haas <rhaas@postgresql.org>
date : Tue, 16 Mar 2021 15:42:20 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Tue, 16 Mar 2021 15:42:20 -0400
Don't complain about the last TOAST chunk number being different
from what we expected if there are no TOAST chunks at all.
In such a case, saying that the final chunk number is 0 is not
really accurate, and the fact the value is missing from the
TOAST table is reported separately anyway.
Mark Dilger
Discussion: http://postgr.es/m/AA5506CE-7D2A-42E4-A51D-358635E3722D@enterprisedb.com
M contrib/amcheck/verify_heapam.c
M src/bin/pg_amcheck/t/004_verify_heapam.pl
Use pre-fetching for ANALYZE
commit : c6fc50cb40285141fad401321ae21becbaea1c59
author : Stephen Frost <sfrost@snowman.net>
date : Tue, 16 Mar 2021 14:46:48 -0400
committer: Stephen Frost <sfrost@snowman.net>
date : Tue, 16 Mar 2021 14:46:48 -0400
When we have posix_fadvise() available, we can improve the performance
of an ANALYZE by quite a bit by using it to inform the kernel of the
blocks that we're going to be asking for. Similar to bitmap index
scans, the number of buffers pre-fetched is based off of the
maintenance_io_concurrency setting (for the particular tablespace or,
if not set, globally, via get_tablespace_maintenance_io_concurrency()).
Reviewed-By: Heikki Linnakangas, Tomas Vondra
Discussion: https://www.postgresql.org/message-id/VI1PR0701MB69603A433348EDCF783C6ECBF6EF0%40VI1PR0701MB6960.eurprd07.prod.outlook.com
M src/backend/commands/analyze.c
Improve logging of auto-vacuum and auto-analyze
commit : 94d13d474dc61800e8a17cc1959c55815b050ecd
author : Stephen Frost <sfrost@snowman.net>
date : Tue, 16 Mar 2021 14:46:48 -0400
committer: Stephen Frost <sfrost@snowman.net>
date : Tue, 16 Mar 2021 14:46:48 -0400
When logging auto-vacuum and auto-analyze activity, include the I/O
timing if track_io_timing is enabled. Also, for auto-analyze, add the
read rate and the dirty rate, similar to how that information has
historically been logged for auto-vacuum.
Stephen Frost and Jakub Wartak
Reviewed-By: Heikki Linnakangas, Tomas Vondra
Discussion: https://www.postgresql.org/message-id/VI1PR0701MB69603A433348EDCF783C6ECBF6EF0%40VI1PR0701MB6960.eurprd07.prod.outlook.com
M doc/src/sgml/config.sgml
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/analyze.c
Improve logging of bad parameter values in BIND messages.
commit : 1ea396362be1615e926ea69d666c770081a0d3ef
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Mar 2021 11:16:41 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Mar 2021 11:16:41 -0400
Since commit ba79cb5dc, values of bind parameters have been logged
during errors in extended query mode. However, we only did that after
we'd collected and converted all the parameter values, thus failing to
offer any useful localization of invalid-parameter problems. Add a
separate callback that's used during parameter collection, and have it
print the parameter number, along with the input string if text input
format is used.
Justin Pryzby and Tom Lane
Discussion: https://postgr.es/m/20210104170939.GH9712@telsasoft.com
Discussion: https://postgr.es/m/CANfkH5k-6nNt-4cSv1vPB80nq2BZCzhFVR5O4VznYbsX0wZmow@mail.gmail.com
M src/backend/tcop/postgres.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
(Blind) fix Perl splitting of strings at newlines
commit : 015061690c6526ff9f9f7af2940e1c1541654b68
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 16 Mar 2021 10:36:28 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 16 Mar 2021 10:36:28 -0300
I forgot that Windows represents newlines as \r\n, so splitting a string
at /\s/ creates additional empty strings. Let's rewrite that as /\s+/
to see if that avoids those. (There's precedent for using that pattern
on Windows in other scripts.)
Previously: 91bdf499b37b, 8ed428dc977f, 650b96707672.
Per buildfarm, via Tom Lane.
Discussion: https://postgr.es/m/3144460.1615860259@sss.pgh.pa.us
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
Add some basic tests for progress reporting of COPY
commit : 15639d5e8f6f278219681fec8a5668a92fb7e874
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 16 Mar 2021 09:55:43 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 16 Mar 2021 09:55:43 +0900
This tests some basic features for progress reporting of COPY, relying
on an INSERT trigger that gets fired when doing COPY FROM with a file or
stdin, checking for sizes, number of tuples processed, and number of
tuples excluded by a WHERE clause.
Author: Josef Šimánek, Matthias van de Meent
Reviewed-by: Michael Paquier, Justin Pryzby, Bharath Rupireddy, Tomas
Vondra
Discussion: https://postgr.es/m/CAEze2WiOcgdH4aQA8NtZq-4dgvnJzp8PohdeKchPkhMY-jWZXA@mail.gmail.com
M src/test/regress/input/copy.source
M src/test/regress/output/copy.source
Add libpq pipeline mode support to pgbench
commit : 9aa491abbf07ca8385a429385be8d68517384fdf
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 15 Mar 2021 18:33:03 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 15 Mar 2021 18:33:03 -0300
New metacommands \startpipeline and \endpipeline allow the user to run
queries in libpq pipeline mode.
Author: Daniel Vérité <daniel@manitou-mail.org>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/b4e34135-2bd9-4b8a-94ca-27d760da26d7@manitou-mail.org
M doc/src/sgml/ref/pgbench.sgml
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
Implement pipeline mode in libpq
commit : acb7e4eb6b1c614c68a62fb3a6a5bba1af0a2659
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 15 Mar 2021 18:13:42 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 15 Mar 2021 18:13:42 -0300
Pipeline mode in libpq lets an application avoid the Sync messages in
the FE/BE protocol that are implicit in the old libpq API after each
query. The application can then insert Sync at its leisure with a new
libpq function PQpipelineSync. This can lead to substantial reductions
in query latency.
Co-authored-by: Craig Ringer <craig.ringer@enterprisedb.com>
Co-authored-by: Matthieu Garrigues <matthieu.garrigues@gmail.com>
Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Aya Iwata <iwata.aya@jp.fujitsu.com>
Reviewed-by: Daniel Vérité <daniel@manitou-mail.org>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Kirk Jamison <k.jamison@fujitsu.com>
Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
Reviewed-by: Nikhil Sontakke <nikhils@2ndquadrant.com>
Reviewed-by: Vaishnavi Prabakaran <VaishnaviP@fast.au.fujitsu.com>
Reviewed-by: Zhihong Yu <zyu@yugabyte.com>
Discussion: https://postgr.es/m/CAMsr+YFUjJytRyV4J-16bEoiZyH=4nj+sQ7JP9ajwz=B4dMMZw@mail.gmail.com
Discussion: https://postgr.es/m/CAJkzx4T5E-2cQe3dtv2R78dYFvz+in8PY7A8MArvLhs_pg75gg@mail.gmail.com
M doc/src/sgml/libpq.sgml
M doc/src/sgml/lobj.sgml
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/bin/pg_amcheck/pg_amcheck.c
M src/interfaces/libpq/exports.txt
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
M src/test/modules/Makefile
A src/test/modules/libpq_pipeline/.gitignore
A src/test/modules/libpq_pipeline/Makefile
A src/test/modules/libpq_pipeline/README
A src/test/modules/libpq_pipeline/libpq_pipeline.c
A src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
M src/tools/msvc/Mkvcbuild.pm
M src/tools/pgindent/typedefs.list
Work around issues in MinGW-64's setjmp/longjmp support.
commit : 146cb3889c3ccb3fce198fe7464a1296a9e107c3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 15 Mar 2021 12:34:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 15 Mar 2021 12:34:17 -0400
It's hard to avoid the conclusion that there is something wrong with
setjmp/longjmp on MinGW-64, as we have seen failures come and go after
entirely-unrelated-looking changes in our own code. Other projects
such as Ruby have given up and started using gcc's setjmp/longjmp
builtins on that platform; this patch just follows that lead.
Note that this is a pretty fundamental ABI break for functions
containining either setjmp or longjmp, so we can't really consider
a back-patch.
Per reports from Regina Obe and Heath Lord, as well as recent failures
on buildfarm member walleye, and less-recent failures on fairywren.
Juan José Santamaría Flecha
Discussion: https://postgr.es/m/000401d716a0$1ed0fc70$5c72f550$@pcorp.us
Discussion: https://postgr.es/m/CA+BEBhvHhM-Bn628pf-LsjqRh3Ang7qCSBG0Ga+7KwhGqrNUPw@mail.gmail.com
Discussion: https://postgr.es/m/f1caef93-9640-022e-9211-bbe8755a56b0@2ndQuadrant.com
M src/include/c.h
Drop SERIALIZABLE workaround from parallel query tests.
commit : eeb60e45d82d5840b713a8741ae552238d57e8b9
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Mar 2021 23:27:08 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Mar 2021 23:27:08 +1300
SERIALIZABLE no longer inhibits parallelism, so we can drop some
outdated workarounds and comments from regression tests. The change
came in release 12, commit bb16aba5, but it's not really worth
back-patching.
Also fix a typo.
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGJUaHeK%3DHLATxF1JOKDjKJVrBKA-zmbPAebOM0Se2FQRg%40mail.gmail.com
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/explain.out
M src/test/regress/expected/insert_parallel.out
M src/test/regress/expected/select_parallel.out
M src/test/regress/expected/write_parallel.out
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/explain.sql
M src/test/regress/sql/insert_parallel.sql
M src/test/regress/sql/select_parallel.sql
M src/test/regress/sql/write_parallel.sql
Make archiver process an auxiliary process.
commit : d75288fb27b8fe0a926aaab7d75816f091ecdc27
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 15 Mar 2021 13:13:14 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 15 Mar 2021 13:13:14 +0900
This commit changes WAL archiver process so that it's treated as
an auxiliary process and can use shared memory. This is an infrastructure
patch required for upcoming shared-memory based stats collector patch
series. These patch series basically need any processes including archiver
that can report the statistics to access to shared memory. Since this patch
itself is useful to simplify the code and when users monitor the status of
archiver, it's committed separately in advance.
This commit simplifies the code for WAL archiving. For example, previously
backends need to signal to archiver via postmaster when they notify
archiver that there are some WAL files to archive. On the other hand,
this commit removes that signal to postmaster and enables backends to
notify archier directly using shared latch.
Also, as the side of this change, the information about archiver process
becomes viewable at pg_stat_activity view.
Author: Kyotaro Horiguchi
Reviewed-by: Andres Freund, Álvaro Herrera, Julien Rouhaud, Tomas Vondra, Arthur Zakirov, Fujii Masao
Discussion: https://postgr.es/m/20180629.173418.190173462.horiguchi.kyotaro@lab.ntt.co.jp
M doc/src/sgml/monitoring.sgml
M src/backend/access/transam/xlogarchive.c
M src/backend/bootstrap/bootstrap.c
M src/backend/postmaster/pgarch.c
M src/backend/postmaster/postmaster.c
M src/backend/storage/ipc/ipci.c
M src/include/miscadmin.h
M src/include/postmaster/pgarch.h
M src/include/storage/pmsignal.h
M src/include/storage/proc.h
M src/tools/pgindent/typedefs.list
Notice that heap page has dead items during VACUUM.
commit : 0ea71c93a06ddc38e0b72e48f1d512e5383a9c1b
author : Peter Geoghegan <pg@bowt.ie>
date : Sun, 14 Mar 2021 18:05:57 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Sun, 14 Mar 2021 18:05:57 -0700
Consistently set a flag variable that tracks whether the current heap
page has a dead item during lazy vacuum's heap scan. We missed the
common case where there is an preexisting (or even a new) LP_DEAD heap
line pointer.
Also make it clear that the variable might be affected by an existing
line pointer, say from an earlier opportunistic pruning operation. This
distinction is important because it's the main reason why we can't just
use the nearby tups_vacuumed variable instead.
No backpatch. In theory failing to set the page level flag variable had
no consequences. Currently it is only used to defensively check if a
page marked all visible has dead items, which should never happen anyway
(if it does then the table must be corrupt).
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Diagnosed-By: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAD21AoAtZb4+HJT_8RoOXvu4HM-Zd4HKS3YSMCH6+-W=bDyh-w@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Doc: add note about how to run the pg_amcheck regression tests.
commit : 58f57490facdec78119e5bab84229dbdc1d5ac6a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 13 Mar 2021 10:51:27 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 13 Mar 2021 10:51:27 -0500
It's not immediately obvious what you have to do to get "make
installcheck" to work here, so document that along the same lines
as we've used elsewhere.
A src/bin/pg_amcheck/README
In pg_amcheck tests, don't depend on perl's Q/q pack code.
commit : 945d2cb7d0255e296a55f3e9febb5dce6eaccc3e
author : Robert Haas <rhaas@postgresql.org>
date : Sat, 13 Mar 2021 10:55:33 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Sat, 13 Mar 2021 10:55:33 -0500
It does not work on all versions of perl across all platforms.
To avoid endian-ness issues, pick a new value for column a
that has the same upper 4 bytes as lower 4 bytes. Try to
make it something that isn't likely to occur anywhere nearby
in the page.
Discussion: http://postgr.es/m/29DA079B-0658-4E66-BDAA-0EFD7B64D9C6@enterprisedb.com
M src/bin/pg_amcheck/t/004_verify_heapam.pl
pg_amcheck: Keep trying to fix the tests.
commit : 9e294d0f34d6e3e4fecf6f190b48862988934cde
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 13 Mar 2021 00:06:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 13 Mar 2021 00:06:56 -0500
Fix another example of non-portable option ordering in the tests.
Oversight in 24189277f.
Mark Dilger
Discussion: https://postgr.es/m/C37D28BA-3BA3-4776-B812-17F05F3472D8@enterprisedb.com
M src/bin/pg_amcheck/t/003_check.pl
Fix new pthread code to respect --disable-thread-safety.
commit : de91c3b976cfacddacd45a9b52046264c0e44b11
author : Thomas Munro <tmunro@postgresql.org>
date : Sat, 13 Mar 2021 17:21:01 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sat, 13 Mar 2021 17:21:01 +1300
Don't try to compile src/port/pthread_barrier_wait.c if we opted out of
threads at configure time. Revealed by build farm member gaur, which
can't compile this code because of problems with its pthread
implementation. It shouldn't be trying to, because it's using
--disable-thread-safety.
Defect in commit 44bf3d50.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/2568537.1615603606%40sss.pgh.pa.us
M configure
M configure.ac
Improve FK trigger parallel-safety check added by 05c8482f7f.
commit : c5be48f092016b1caf597b2e21d588b56c88a23e
author : Amit Kapila <akapila@postgresql.org>
date : Sat, 13 Mar 2021 09:13:21 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Sat, 13 Mar 2021 09:13:21 +0530
Commit 05c8482f7f added special logic related to parallel-safety of FK
triggers. This is a bit of a hack and should have instead been done by
simply setting appropriate proparallel values on those trigger functions
themselves.
Suggested-by: Tom Lane
Author: Greg Nancarrow
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/2309260.1615485644@sss.pgh.pa.us
M src/backend/optimizer/util/clauses.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
pg_amcheck: Keep trying to fix the tests.
commit : b9164eab208342d685638fc90048ffaa2688cb47
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 21:59:56 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 21:59:56 -0500
Commit 24189277f6ff3169b15c7bc82926a372ca7f2dbf managed to remove
one of the two places where we were checking for a "no such user"
error while leaving the other one right next to it. So remove that
too. In fact, remove the entire test, because the whole point of
this test was to see which message we got on a failure.
M src/bin/pg_amcheck/t/002_nonesuch.pl
pg_amcheck: Try to fix still more test failures.
commit : 24189277f6ff3169b15c7bc82926a372ca7f2dbf
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 20:11:47 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 20:11:47 -0500
Avoid use of non-portable option ordering in command_checks_all().
The use of bare command line arguments before switches doesn't work
everywhere. Per buildfarm members drongo and hoverfly.
Avoid testing for the message "role \"%s\" does not exist", because
some buildfarm machines report a different error. fairywren complains
about "SSPI authentication failed for user \"%s\"", for example.
Mark Dilger
Discussion: http://postgr.es/m/9E76E46A-48B2-4869-BD0C-422204C1F767@enterprisedb.com
Discussion: http://postgr.es/m/F0A1FD70-A2F4-4528-8A03-8650CAEC0554%40enterprisedb.com
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/pg_amcheck/t/003_check.pl
Try to avoid apparent platform-dependency in IPC::Run
commit : f371a4cdba6dc805acd608cc63a7089b57cb4e9e
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 19:00:41 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 19:00:41 -0500
It's hard to believe, but buildfarm results from the new pg_amcheck
suggest that command_checks_all() perform shell expansion on some
machines but not others, apparently due to an underlying behavior
difference in IPC::Run. Let's see if we can work around that - and
confirm that it is the real problem - by passing '-S*' as a single
argument rather than '-S' and '*' as two separate ones.
Failures were observed on jacana and hoverfly.
Mark Dilger
Discussion: http://postgr.es/m/9E76E46A-48B2-4869-BD0C-422204C1F767@enterprisedb.com
M src/bin/pg_amcheck/t/002_nonesuch.pl
Fix portability issues in pg_amcheck's 004_verify_heapam.pl.
commit : 661125612706b1d0d5ed9f12d18908b08512a7eb
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 17:30:17 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 17:30:17 -0500
Test #12 overwrote a 1-byte varlena header to make it look like the
initial byte of a 4-byte varlena header, but the results were
endian-dependent. Also, the byte "abc" that followed the overwritten
byte would be interpreted differently depending on endian-ness.
Overwrite 4 bytes instead, in an endian-aware manner.
Test #13 accidentally managed to depend on TOAST_MAX_CHUNK_SIZE,
which varies slightly depending on MAXIMUM_ALIGNOF. That's not
the point anyway, so make the regexp insensitive to the expected
number of chunks.
Mark Dilger
Discussion: http://postgr.es/m/A80D68F6-E38F-482D-9522-E2FB6AAFE8A1@enterprisedb.com
M src/bin/pg_amcheck/t/004_verify_heapam.pl
Consolidate nbtree VACUUM metapage routines.
commit : 02b5940dbea17d07a1dbcba3cbe113cc8b70f228
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 12 Mar 2021 13:11:47 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 12 Mar 2021 13:11:47 -0800
Simplify _bt_vacuum_needs_cleanup() functions's signature (it only needs
a single 'rel' argument now), and move it next to its sibling function
in nbtpage.c.
I believe that _bt_vacuum_needs_cleanup() was originally located in
nbtree.c due to an include dependency issue. That's no longer an issue.
Follow-up to commit 9f3665fb.
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/include/access/nbtree.h
Move PG_USED_FOR_ASSERTS_ONLY before initializer.
commit : ac445955852fe6bc0e02e87a409f25ab6e0a82d6
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 15:04:10 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 15:04:10 -0500
Erik Rijkers reported a compile failure, and I think this is probably
the reason.
M src/bin/pg_amcheck/pg_amcheck.c
Adjust perl style.
commit : 7a1527c02c147a4107490662cb1872524c54a8ae
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 14:55:40 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 14:55:40 -0500
Per buildfarm member crake.
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/004_verify_heapam.pl
Try to fix compiler warnings.
commit : d60e61de4fb4a8e7ca88204c2c409e7380887d76
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 14:35:10 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 14:35:10 -0500
Per report from Peter Geoghegan.
Discussion: http://postgr.es/m/CAH2-WznpwULZ3uJ1_6WXvNMXYbOy8k8tYs3r=qSdGmZeRd6tDw@mail.gmail.com
M src/bin/pg_amcheck/pg_amcheck.c
Add pg_amcheck, a CLI for contrib/amcheck.
commit : 9706092839db2c8c93860674e426a917635438c3
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 13:00:01 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 12 Mar 2021 13:00:01 -0500
This makes it a lot easier to run the corruption checks that are
implemented by contrib/amcheck against lots of relations and get
the result in an easily understandable format. It has a wide variety
of options for choosing which relations to check and which checks
to perform, and it can run checks in parallel if you want.
Mark Dilger, reviewed by Peter Geoghegan and by me.
Discussion: http://postgr.es/m/12ED3DA8-25F0-4B68-937D-D907CFBF08E7@enterprisedb.com
Discussion: http://postgr.es/m/BA592F2D-F928-46FF-9516-2B827F067F57@enterprisedb.com
M doc/src/sgml/ref/allfiles.sgml
A doc/src/sgml/ref/pg_amcheck.sgml
M doc/src/sgml/reference.sgml
M src/bin/Makefile
A src/bin/pg_amcheck/.gitignore
A src/bin/pg_amcheck/Makefile
A src/bin/pg_amcheck/pg_amcheck.c
A src/bin/pg_amcheck/t/001_basic.pl
A src/bin/pg_amcheck/t/002_nonesuch.pl
A src/bin/pg_amcheck/t/003_check.pl
A src/bin/pg_amcheck/t/004_verify_heapam.pl
A src/bin/pg_amcheck/t/005_opclass_damage.pl
M src/tools/msvc/Install.pm
M src/tools/msvc/Mkvcbuild.pm
Fix race condition in psql \e's detection of file modification.
commit : 48d67fd897918c72e7cdf703d794056b88ed5725
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Mar 2021 12:20:15 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Mar 2021 12:20:15 -0500
psql's editing commands decide whether the user has edited the file
by checking for change of modification timestamp. This is probably
fine for a pre-existing file, but with a temporary file that is
created within the command, it's possible for a fast typist to
save-and-exit in less than the one-second granularity of stat(2)
timestamps. On Windows FAT filesystems the granularity is even
worse, 2 seconds, making the race a bit easier to hit.
To fix, try to set the temp file's mod time to be two seconds ago.
It's unlikely this would fail, but then again the race condition
itself is unlikely, so just ignore any error.
Also, we might as well check the file size as well as its mod time.
While this is a difficult bug to hit, it still seems worth
back-patching, to ensure that users' edits aren't lost.
Laurenz Albe, per gripe from Jacob Champion; based on fix suggestions
from Jacob and myself
Discussion: https://postgr.es/m/0ba3f2a658bac6546d9934ab6ba63a805d46a49b.camel@cybertec.at
M src/bin/psql/command.c
Forbid marking an identity column as nullable.
commit : f52c5d6749a61fc4e0908457c58f5069910d53a6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Mar 2021 11:08:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Mar 2021 11:08:42 -0500
GENERATED ALWAYS AS IDENTITY implies NOT NULL, but the code failed
to complain if you overrode that with "GENERATED ALWAYS AS IDENTITY
NULL". One might think the old behavior was a feature, but it was
inconsistent because the outcome varied depending on the order of
the clauses, so it seems to have been just an oversight.
Per bug #16913 from Pavel Boev. Back-patch to v10 where identity
columns were introduced.
Vik Fearing (minor tweaks by me)
Discussion: https://postgr.es/m/16913-3b5198410f67d8c6@postgresql.org
M doc/src/sgml/ref/create_table.sgml
M src/backend/parser/parse_utilcmd.c
M src/test/regress/expected/identity.out
M src/test/regress/sql/identity.sql
Specialize checkpointer sort functions.
commit : 1b88b8908e751271933c076234fa085cda251421
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 12 Mar 2021 23:56:02 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 12 Mar 2021 23:56:02 +1300
When sorting a potentially large number of dirty buffers, the
checkpointer can benefit from a faster sort routine. One reported
improvement on a large buffer pool system was 1.4s -> 0.6s.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGJ2-eaDqAum5bxhpMNhvuJmRDZxB_Tow0n-gse%2BHG0Yig%40mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
commit : 519e4c9ee21a656879123f4843f1d8d60cb71536
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 12 Mar 2021 15:42:08 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 12 Mar 2021 15:42:08 +0530
Reported-by: Thomas Munro
Author: Takayuki Tsunakawa
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/CA+hUKG+oPoFizjABt=GXZWTEHx3oev5rAe2scjW2r6F1rguo5w@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
Fix use of relcache TriggerDesc field introduced by commit 05c8482f7f.
commit : e2cda3c20a61c76e497fb2ebb6d2b2ae8c43c014
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 12 Mar 2021 15:14:41 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 12 Mar 2021 15:14:41 +0530
The commit added code which used a relcache TriggerDesc field across
another cache access, which it shouldn't because the relcache doesn't
guarantee it won't get moved.
Diagnosed-by: Tom Lane
Author: Greg Nancarrow
Reviewed-by: Hou Zhijie, Amit Kapila
Discussion: https://postgr.es/m/2309260.1615485644@sss.pgh.pa.us
M src/backend/optimizer/util/clauses.c
Poll postmaster less frequently in recovery.
commit : 57dcc2ef3320de3e1aef6f668601e6ad2bdf88c4
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 12 Mar 2021 19:08:52 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 12 Mar 2021 19:08:52 +1300
Since commits 9f095299 and f98b8476 we don't poll the postmaster
pipe at all during crash recovery on Linux and FreeBSD, but on other
operating systems we were still doing it for every WAL record. Do it
less frequently on operating systems where system calls are required, at
the cost of delaying exit a bit after postmaster death. This avoids
expensive system calls reported to slow down CPU-bound recovery by as
much as 10-30%.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CA%2BhUKGK1607VmtrDUHQXrsooU%3Dap4g4R2yaoByWOOA3m8xevUQ%40mail.gmail.com
Discussion: https://postgr.es/m/7261eb39-0369-f2f4-1bb5-62f3b6083b5e@iki.fi
M src/backend/postmaster/startup.c
Add condition variable for walreceiver shutdown.
commit : de829ddf23f69190efb4e0178704c4c4228e17cd
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 12 Mar 2021 19:07:27 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 12 Mar 2021 19:07:27 +1300
Use this new CV to wait for walreceiver shutdown without a sleep/poll
loop, while also benefiting from standard postmaster death handling.
Discussion: https://postgr.es/m/CA%2BhUKGK1607VmtrDUHQXrsooU%3Dap4g4R2yaoByWOOA3m8xevUQ%40mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/postmaster/pgstat.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walreceiverfuncs.c
M src/include/pgstat.h
M src/include/replication/walreceiver.h
Add condition variable for recovery resume.
commit : 600f2f50b7a57c8481276450c9019fa7b3656411
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 12 Mar 2021 19:03:52 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 12 Mar 2021 19:03:52 +1300
Replace a sleep loop with a CV, to get a fast reaction time when
recovery is resumed or the postmaster exits via standard infrastructure.
Unfortunately we still need to wake up every second to perform extra
polling during the recovery pause loop.
Discussion: https://postgr.es/m/CA%2BhUKGK1607VmtrDUHQXrsooU%3Dap4g4R2yaoByWOOA3m8xevUQ%40mail.gmail.com
M src/backend/access/transam/xlog.c
Send statistics collected during shutdown checkpoint to the stats collector.
commit : b82640df0062483431608b7e9e074255f03e6c02
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 12 Mar 2021 14:23:00 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 12 Mar 2021 14:23:00 +0900
When shutdown is requested, checkpointer performs checkpoint or
restartpoint, and updates the statistics, before it exits. But previously
checkpointer didn't send those statistics to the stats collector.
Shutdown checkpoint and restartpoint are treated as requested ones
instead of scheduled ones, so the number of them are counted in
pg_stat_bgwriter.checkpoints_req column.
Author: Masahiro Ikeda
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/0509ad67b585a5b86a83d445dfa75392@oss.nttdata.com
M src/backend/postmaster/checkpointer.c
Force to send remaining WAL stats to the stats collector at walwriter exit.
commit : 33394ee6f2433d3cc7785428a77cc9a813254df7
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 12 Mar 2021 13:29:59 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 12 Mar 2021 13:29:59 +0900
In walwriter's main loop, WAL stats message is only sent if enough time
has passed since last one was sent to reach PGSTAT_STAT_INTERVAL msecs.
This is necessary to avoid overloading to the stats collector. But this
can cause recent WAL stats to be unsent when walwriter exits.
To ensure that all the WAL stats are sent, this commit makes walwriter
force to send remaining WAL stats to the collector when it exits because
of shutdown request. Note that those remaining WAL stats can still be
unsent when walwriter exits with non-zero exit code (e.g., FATAL error).
This is OK because that walwriter exit leads to server crash and
subsequent recovery discards all the stats. So there is no need to send
remaining stats in that case.
Author: Masahiro Ikeda
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/0509ad67b585a5b86a83d445dfa75392@oss.nttdata.com
M src/backend/postmaster/walwriter.c
Minor modernization for README.barrier.
commit : 43c66624964aa1d2f519ad6be0c5ea8f170cf357
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 12 Mar 2021 15:24:28 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 12 Mar 2021 15:24:28 +1300
Itanium is very uncommon and being discontinued. ARM is everywhere.
Prefer ARM as an example of an architecture with weak memory ordering.
M src/backend/storage/lmgr/README.barrier
Save a few cycles during nbtree VACUUM.
commit : 7bb97211a5589265f3f88183ae9353639ab184c6
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 11 Mar 2021 14:18:23 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 11 Mar 2021 14:18:23 -0800
Avoid calling RelationGetNumberOfBlocks() unnecessarily in the common
case where there are no deleted but not yet recycled pages to recycle
during a cleanup-only nbtree VACUUM operation.
Follow-up to commit e5d8a999, which (among other things) taught the
"skip full scan" nbtree VACUUM mechanism to only trigger a full index
scan when the absolute number of deleted pages in the index is
considered excessive.
M src/backend/access/nbtree/nbtree.c
Add back vacuum_cleanup_index_scale_factor parameter.
commit : effdd3f3b633e88feaa675377075f02ecc99aee4
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 11 Mar 2021 12:42:46 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 11 Mar 2021 12:42:46 -0800
Commit 9f3665fb removed the vacuum_cleanup_index_scale_factor storage
parameter. However, that creates dump/reload hazards when moving across
major versions.
Add back the vacuum_cleanup_index_scale_factor parameter (though not the
GUC of the same name) purely to avoid problems when using tools like
pg_upgrade. The parameter remains disabled and undocumented.
No backpatch to Postgres 13, since vacuum_cleanup_index_scale_factor was
only disabled by REL_13_STABLE's version of master branch commit
9f3665fb in the first place -- the parameter already looks like this on
REL_13_STABLE.
Discussion: https://postgr.es/m/YEm/a3Ko3nKnBuVq@paquier.xyz
M src/backend/access/common/reloptions.c
M src/backend/access/nbtree/nbtutils.c
M src/include/access/nbtree.h
Be clear about whether a recovery pause has taken effect.
commit : 32fd2b57d7f64948e649fc205c43f007762ecaac
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 11 Mar 2021 14:52:32 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 11 Mar 2021 14:52:32 -0500
Previously, the code and documentation seem to have essentially
assumed than a call to pg_wal_replay_pause() would take place
immediately, but that's not the case, because we only check for a
pause in certain places. This means that a tool that uses this
function and then wants to do something else afterward that is
dependent on the pause having taken effect doesn't know how long it
needs to wait to be sure that no more WAL is going to be replayed.
To avoid that, add a new function pg_get_wal_replay_pause_state()
which returns either 'not paused', 'paused requested', or 'paused'.
After calling pg_wal_replay_pause() the status will immediate change
from 'not paused' to 'pause requested'; when the startup process
has noticed this, the status will change to 'pause'. For backward
compatibility, pg_is_wal_replay_paused() still exists and returns
the same thing as before: true if a pause has been requested,
whether or not it has taken effect yet; and false if not.
The documentation is updated to clarify.
To improve the changes that a pause request is quickly confirmed
effective, adjust things so that WaitForWALToBecomeAvailable will
swiftly reach a call to recoveryPausesHere() when a pause request
is made.
Dilip Kumar, reviewed by Simon Riggs, Kyotaro Horiguchi, Yugo Nagata,
Masahiko Sawada, and Bharath Rupireddy.
Discussion: http://postgr.es/m/CAFiTN-vcLLWEm8Zr%3DYK83rgYrT9pbC8VJCfa1kY9vL3AUPfu6g%40mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogfuncs.c
M src/include/access/xlog.h
M src/include/catalog/pg_proc.dat
Re-simplify management of inStart in pqParseInput3's subroutines.
commit : 51c54bb603098416dc6f9d9d46a3d14861f8fc38
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 11 Mar 2021 14:43:45 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 11 Mar 2021 14:43:45 -0500
Commit 92785dac2 copied some logic related to advancement of inStart
from pqParseInput3 into getRowDescriptions and getAnotherTuple,
because it wanted to allow user-defined row processor callbacks to
potentially longjmp out of the library, and inStart would have to be
updated before that happened to avoid an infinite loop. We later
decided that that API was impossibly fragile and reverted it, but
we didn't undo all of the related code changes, and this bit of
messiness survived. Undo it now so that there's just one place in
pqParseInput3's processing where inStart is advanced; this will
simplify addition of better tracing support.
getParamDescriptions had grown similar processing somewhere along
the way (not in 92785dac2; I didn't track down just when), but it's
actually buggy because its handling of corrupt-message cases seems to
have been copied from the v2 logic where we lacked a known message
length. The cases where we "goto not_enough_data" should not simply
return EOF, because then we won't consume the message, potentially
creating an infinite loop. That situation now represents a
definitively corrupt message, and we should report it as such.
Although no field reports of getParamDescriptions getting stuck in
a loop have been seen, it seems appropriate to back-patch that fix.
I chose to back-patch all of this to keep the logic looking more alike
in supported branches.
Discussion: https://postgr.es/m/2217283.1615411989@sss.pgh.pa.us
M src/interfaces/libpq/fe-protocol3.c
Refactor and generalize the ParallelSlot machinery.
commit : f71519e545a34ece0a27c8bb1a2b6e197d323163
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 11 Mar 2021 13:17:46 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 11 Mar 2021 13:17:46 -0500
Create a wrapper object, ParallelSlotArray, to encapsulate the
number of slots and the slot array itself, plus some other relevant
bits of information. This reduces the number of parameters we have
to pass around all over the place.
Allow for a ParallelSlotArray to contain slots connected to
different databases within a single cluster. The current clients
of this mechanism don't need this, but it is expected to be used
by future patches.
Defer connecting to databases until we actually need the connection
for something. This is a slight behavior change for vacuumdb and
reindexdb. If you specify a number of jobs that is larger than the
number of objects, the extra connections will now not be used.
But, on the other hand, if you specify a number of jobs that is
so large that it's going to fail, the failure would previously have
happened before any operations were actually started, and now it
won't.
Mark Dilger, reviewed by me.
Discussion: http://postgr.es/m/12ED3DA8-25F0-4B68-937D-D907CFBF08E7@enterprisedb.com
Discussion: http://postgr.es/m/BA592F2D-F928-46FF-9516-2B827F067F57@enterprisedb.com
M src/bin/scripts/reindexdb.c
M src/bin/scripts/vacuumdb.c
M src/fe_utils/parallel_slot.c
M src/include/fe_utils/parallel_slot.h
M src/tools/pgindent/typedefs.list
Set libcrypto callbacks for all connection threads in libpq
commit : 2c0cefcd18161549e9e8b103f46c0f65fca84d99
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 11 Mar 2021 17:14:25 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 11 Mar 2021 17:14:25 +0900
Based on an analysis of the OpenSSL code with Jacob, moving to EVP for
the cryptohash computations makes necessary the setup of the libcrypto
callbacks that were getting set only for SSL connections, but not for
connections without SSL. Not setting the callbacks makes the use of
threads potentially unsafe for connections calling cryptohashes during
authentication, like MD5 or SCRAM, if a failure happens during a
cryptohash computation. The logic setting the libssl and libcrypto
states is then split into two parts, both using the same locking, with
libcrypto being set up for SSL and non-SSL connections, while SSL
connections set any libssl state afterwards as needed.
Prior to this commit, only SSL connections would have set libcrypto
callbacks that are necessary to ensure a proper thread locking when
using multiple concurrent threads in libpq (ENABLE_THREAD_SAFETY). Note
that this is only required for OpenSSL 1.0.2 and 1.0.1 (oldest version
supported on HEAD), as 1.1.0 has its own internal locking and it has
dropped support for CRYPTO_set_locking_callback().
Tests with up to 300 threads with OpenSSL 1.0.1 and 1.0.2, mixing SSL
and non-SSL connection threads did not show any performance impact after
some micro-benchmarking. pgbench can be used here with -C and a
mostly-empty script (with one \set meta-command for example) to stress
authentication requests, and we have mixed that with some custom
programs for testing.
Reported-by: Jacob Champion
Author: Michael Paquier
Reviewed-by: Jacob Champion
Discussion: https://postgr.es/m/fd3ba610085f1ff54623478cf2f7adf5af193cbb.camel@vmware.com
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/fe-secure.c
M src/interfaces/libpq/libpq-int.h
Doc: B-Tree only has one additional parameter.
commit : 3f0daeb02f8dd605f89de9aa2349137c09cc7fb4
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 10 Mar 2021 22:10:36 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 10 Mar 2021 22:10:36 -0800
Oversight in commit 9f3665fb.
Backpatch: 13-, just like commit 9f3665fb.
M doc/src/sgml/ref/create_index.sgml
Improve comment for struct BufferDesc.
commit : 049d9b872db8a24b45709dbaed9a1051e92ed513
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 11 Mar 2021 15:58:05 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 11 Mar 2021 15:58:05 +1300
Add a note that per-buffer I/O condition variables currently live
outside the BufferDesc struct. Follow-up for commit d8725104.
Reported-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://postgr.es/m/20210311031118.hucytmrgwlktjxgq%40nol
M src/include/storage/buf_internals.h
tutorial: land height is "elevation", not "altitude"
commit : 2950ff32f31d073ea4be4f5f9b73249131f42bd7
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 10 Mar 2021 20:25:19 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 10 Mar 2021 20:25:19 -0500
This is a follow-on patch to 92c12e46d5. In that patch, we renamed
"altitude" to "elevation" in the docs, based on these details:
https://mapscaping.com/blogs/geo-candy/what-is-the-difference-between-elevation-relief-and-altitude
This renames the tutorial SQL files to match the documentation.
Reported-by: max1@inbox.ru
Discussion: https://postgr.es/m/161512392887.1046.3137472627109459518@wrigleys.postgresql.org
Backpatch-through: 9.6
M src/tutorial/advanced.source
VACUUM ANALYZE: Always update pg_class.reltuples.
commit : 5f8727f5a679452f7bbdd6966a1586934dcaa84f
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 10 Mar 2021 17:07:57 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 10 Mar 2021 17:07:57 -0800
vacuumlazy.c sometimes fails to update pg_class entries for each index
(to ensure that pg_class.reltuples is current), even though analyze.c
assumed that that must have happened during VACUUM ANALYZE. There are
at least a couple of reasons for this. For example, vacuumlazy.c could
fail to update pg_class when the index AM indicated that its statistics
are merely an estimate, per the contract for amvacuumcleanup() routines
established by commit e57345975cf back in 2006.
Stop assuming that pg_class must have been updated with accurate
statistics within VACUUM ANALYZE -- update pg_class for indexes at the
same time as the table relation in all cases. That way VACUUM ANALYZE
will never fail to keep pg_class.reltuples reasonably accurate.
The only downside of this approach (compared to the old approach) is
that it might inaccurately set pg_class.reltuples for indexes whose heap
relation ends up with the same inaccurate value anyway. This doesn't
seem too bad. We already consistently called vac_update_relstats() (to
update pg_class) for the heap/table relation twice during any VACUUM
ANALYZE -- once in vacuumlazy.c, and once in analyze.c. We now make
sure that we call vac_update_relstats() at least once (though often
twice) for each index.
This is follow up work to commit 9f3665fb, which dealt with issues in
btvacuumcleanup(). Technically this fixes an unrelated issue, though.
btvacuumcleanup() no longer provides an accurate num_index_tuples value
following commit 9f3665fb (when there was no btbulkdelete() call during
the VACUUM operation in question), but hashvacuumcleanup() has worked in
the same way for many years now.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAH2-WzknxdComjhqo4SUxVFk_Q1171GJO2ZgHZ1Y6pion6u8rA@mail.gmail.com
Backpatch: 13-, just like commit 9f3665fb.
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/analyze.c
Don't consider newly inserted tuples in nbtree VACUUM.
commit : 9f3665fbfc34b963933e51778c7feaa8134ac885
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 10 Mar 2021 16:27:01 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 10 Mar 2021 16:27:01 -0800
Remove the entire idea of "stale stats" within nbtree VACUUM (stop
caring about stats involving the number of inserted tuples). Also
remove the vacuum_cleanup_index_scale_factor GUC/param on the master
branch (though just disable them on postgres 13).
The vacuum_cleanup_index_scale_factor/stats interface made the nbtree AM
partially responsible for deciding when pg_class.reltuples stats needed
to be updated. This seems contrary to the spirit of the index AM API,
though -- it is not actually necessary for an index AM's bulk delete and
cleanup callbacks to provide accurate stats when it happens to be
inconvenient. The core code owns that. (Index AMs have the authority
to perform or not perform certain kinds of deferred cleanup based on
their own considerations, such as page deletion and recycling, but that
has little to do with pg_class.reltuples/num_index_tuples.)
This issue was fairly harmless until the introduction of the
autovacuum_vacuum_insert_threshold feature by commit b07642db, which had
an undesirable interaction with the vacuum_cleanup_index_scale_factor
mechanism: it made insert-driven autovacuums perform full index scans,
even though there is no real benefit to doing so. This has been tied to
a regression with an append-only insert benchmark [1].
Also have remaining cases that perform a full scan of an index during a
cleanup-only nbtree VACUUM indicate that the final tuple count is only
an estimate. This prevents vacuumlazy.c from setting the index's
pg_class.reltuples in those cases (it will now only update pg_class when
vacuumlazy.c had TIDs for nbtree to bulk delete). This arguably fixes
an oversight in deduplication-related bugfix commit 48e12913.
[1] https://smalldatum.blogspot.com/2021/01/insert-benchmark-postgres-is-still.html
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAD21AoA4WHthN5uU6+WScZ7+J_RcEjmcuH94qcoUPuB42ShXzg@mail.gmail.com
Backpatch: 13-, where autovacuum_vacuum_insert_threshold was added.
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/create_index.sgml
M src/backend/access/common/reloptions.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/rmgrdesc/nbtdesc.c
M src/backend/utils/init/globals.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/psql/tab-complete.c
M src/include/access/nbtree.h
M src/include/access/nbtxlog.h
M src/include/access/xlog_internal.h
M src/include/miscadmin.h
M src/test/regress/expected/btree_index.out
M src/test/regress/sql/btree_index.sql
C comments: improve description of GiST NSN and GistBuildLSN
commit : 845ac7f847a25505e91f30dca4e0330b25785ee0
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 10 Mar 2021 17:03:10 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 10 Mar 2021 17:03:10 -0500
GiST indexes are complex, so adding more details in the code might help
someone.
Discussion: https://postgr.es/m/20210302164021.GA364@momjian.us
M src/backend/access/gist/README
M src/include/access/gist.h
Replace buffer I/O locks with condition variables.
commit : d87251048a0f293ad20cc1fe26ce9f542de105e6
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 11 Mar 2021 10:05:58 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 11 Mar 2021 10:05:58 +1300
1. Backends waiting for buffer I/O are now interruptible.
2. If something goes wrong in a backend that is currently performing
I/O, waiting backends no longer wake up until that backend reaches
AbortBufferIO() and broadcasts on the CV. Previously, any waiters would
wake up (because the I/O lock was automatically released) and then
busy-loop until AbortBufferIO() cleared BM_IO_IN_PROGRESS.
3. LWLockMinimallyPadded is removed, as it would now be unused.
Author: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (earlier version, 2016)
Discussion: https://postgr.es/m/CA%2BhUKGJ8nBFrjLuCTuqKN0pd2PQOwj9b_jnsiGFFMDvUxahj_A%40mail.gmail.com
Discussion: https://postgr.es/m/CA+Tgmoaj2aPti0yho7FeEf2qt-JgQPRWb0gci_o1Hfr=C56Xng@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/postmaster/pgstat.c
M src/backend/storage/buffer/README
M src/backend/storage/buffer/buf_init.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/lmgr/lwlock.c
M src/include/pgstat.h
M src/include/storage/buf_internals.h
M src/include/storage/condition_variable.h
M src/include/storage/lwlock.h
M src/tools/pgindent/typedefs.list
Avoid creating duplicate cached plans for inherited FK constraints.
commit : c3ffe34863688115dd7878f118f2a123bafd8a26
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 10 Mar 2021 14:22:31 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 10 Mar 2021 14:22:31 -0500
When a foreign key constraint is applied to a partitioned table, each
leaf partition inherits a similar FK constraint. We were processing all
of those constraints independently, meaning that in large partitioning
trees we'd build up large collections of cached FK-checking query plans.
However, in all cases but one, the generated queries are actually
identical for all members of the inheritance tree (because, in most
cases, the query only mentions the topmost table of the other side of
the FK relationship). So we can share a single cached plan among all
the partitions, saving memory, not to mention time to build and maintain
the cached plans.
Keisuke Kuroda and Amit Langote
Discussion: https://postgr.es/m/cab4b85d-9292-967d-adf2-be0d803c3e23@nttcom.co.jp_1
M src/backend/utils/adt/ri_triggers.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
Doc: get rid of <foreignphrase> tags.
commit : b12436340adf27aa3d334c92579e6662dd3090ce
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 10 Mar 2021 12:38:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 10 Mar 2021 12:38:43 -0500
We italicized some, but not all, instances of "per se", "pro forma", and
"ad hoc". These phrases are widespread in formal registers of English,
so it"s debatable whether they even qualify as foreign. We could instead
try to be more consistent in the use of <foreignphrase>, but that"s
difficult to enforce, so let"s just remove the tags for those words.
The one case that seems to deserve the tag is "voilà". Instead of keeping
just one instance of the tag, change that to a more standard phrase.
John Naylor
Discussion: https://postgr.es/m/CAFBsxsHtWs_NsccAVgQ=tTUKkXHpHdkjZXtp_Cd9dGWyBDxfbQ@mail.gmail.com
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/create_role.sgml
M doc/src/sgml/rules.sgml
M doc/src/sgml/typeconv.sgml
Doc: improve introductory information about procedures.
commit : 227338b00d498d9e1c5705a1ab118585e5d57c87
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 10 Mar 2021 11:33:50 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 10 Mar 2021 11:33:50 -0500
Clarify the discussion in "User-Defined Procedures", by laying out
the key differences between functions and procedures in a bulleted
list. Notably, this avoids burying the lede about procedures being
able to do transaction control. Make the back-link in the CREATE
FUNCTION reference page more prominent, and add one in CREATE
PROCEDURE.
Per gripe from Guyren Howe. Thanks to David Johnston for discussion.
Discussion: https://postgr.es/m/BYAPR03MB4903C53A8BB7EFF5EA289674A6949@BYAPR03MB4903.namprd03.prod.outlook.com
M doc/src/sgml/ref/create_function.sgml
M doc/src/sgml/ref/create_procedure.sgml
M doc/src/sgml/xfunc.sgml
Doc: fix missing mention of procedure OUT parameters.
commit : 3ebc6d295705fec37dc8f57a4ece54b370f55f72
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 10 Mar 2021 10:59:48 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 10 Mar 2021 10:59:48 -0500
Small oversight in commit 2453ea142.
M doc/src/sgml/plpgsql.sgml
Add bound check before bsearch() for performance
commit : bbaf315309ed1194d451326cc8f4f59a45906408
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 10 Mar 2021 15:19:37 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 10 Mar 2021 15:19:37 +0100
In the current lazy vacuum implementation, some index AMs such as
btree indexes call lazy_tid_reaped() for each index tuple during
ambulkdelete to check if the index tuple points to the (collected)
garbage tuple. In that function, we simply call bsearch(), but we
should be able to know the result without bsearch() if the index tuple
points to the heap tuple that is out of range of the collected garbage
tuples. Therefore, add a simple bound check before resorting to
bsearch(). Testing has shown that this can give significant
performance benefits.
Author: Masahiko Sawada <masahiko.sawada@2ndquadrant.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+fd4k76j8jKzJzcx8UqEugvayaMSnQz0iLUt_XgBp-_-bd22A@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Fix another portability bug in recent pgbench commit.
commit : c427de427ac411039d5efd5d0dcc8a4e0c6da68d
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 10 Mar 2021 22:22:12 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 10 Mar 2021 22:22:12 +1300
Commit 547f04e7 produced errors on AIX/xlc while building plpython. The
new code appears to be incompatible with the hack installed by commit
a11cf433. Without access to an AIX system to check, my guess is that
_POSIX_C_SOURCE may be required for <time.h> to declare the things the
header needs to see, but plpython.h undefines it.
For now, to unbreak build farm animal hoverfly, just move the new
pg_time_usec_t support into pgbench.c. Perhaps later we could figure
out what to rearrange to put it back into a header for wider use.
Discussion: https://postgr.es/m/CA%2BhUKG%2BP%2BjcD%3Dx9%2BagyTdWtjpOT64MYiGic%2Bcbu_TD8CV%3D6A3w%40mail.gmail.com
M src/bin/pgbench/pgbench.c
M src/include/portability/instr_time.h
Try to fix portability bugs in recent pgbench commits.
commit : 68b34b2338f013cb025dea360b37a3b4fc930179
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 10 Mar 2021 20:18:15 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 10 Mar 2021 20:18:15 +1300
1. pg_time_usec_t needs to be printed with INT64_FORMAT, not %ld, or 32
bit systems complain, per lapwing.
2. Some Windows compilers didn't like a thread function not marked with
__stdcall, per whelk; let's see if this fixes the problem.
M src/bin/pgbench/pgbench.c
Small debug message tweak
commit : 1657b37d7cce5c35e4c1d500f0a2f3736a087d82
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 10 Mar 2021 08:16:38 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 10 Mar 2021 08:16:38 +0100
This makes the wording of the delete case match the update case.
M src/backend/replication/logical/worker.c
Move tablespace path re-creation from the makefiles to pg_regress
commit : 6c788d9f6aadb41d76a72d56149268371a7895ee
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 10 Mar 2021 14:50:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 10 Mar 2021 14:50:00 +0900
Moving this logic into pg_regress fixes a potential failure with
parallel tests when pg_upgrade and the main regression test suite both
trigger the makefile rule that cleaned up testtablespace/ under
src/test/regress. Even if pg_upgrade was triggering this rule, it has
no need to do so as it uses a different tablespace path. So if
pg_upgrade triggered the makefile rule for the tablespace setup while
the main regression test suite ran the tablespace cases, it would fail.
61be85a was a similar attempt at achieving that, but that broke cases
where the regression tests require to run under an Administrator
account, like with Appveyor.
Reported-by: Andres Freund, Kyotaro Horiguchi
Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/20201209012911.uk4d6nxcnkp7ehrx@alap3.anarazel.de
M src/bin/pg_upgrade/test.sh
M src/test/regress/GNUmakefile
M src/test/regress/pg_regress.c
M src/tools/msvc/vcregress.pl
pgbench: Synchronize client threads.
commit : aeb57af8e64000cc4288a7b8b8d7cf6040eae900
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 10 Mar 2021 16:17:34 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 10 Mar 2021 16:17:34 +1300
Wait until all pgbench threads are connected before benchmarking begins.
This fixes a problem where some connections could take a very long time
to be established because of lock contention from earlier connections,
making results unstable and bogus with high connection counts.
Author: Andres Freund <andres@anarazel.de>
Author: Fabien COELHO <coelho@cri.ensmp.fr>
Reviewed-by: Marina Polyakova <m.polyakova@postgrespro.ru>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/20200227180100.zyvjwzcpiokfsqm2%40alap3.anarazel.de
M src/bin/pgbench/pgbench.c
Add missing pthread_barrier_t.
commit : 44bf3d5083e151d772c5d6f656e3e162f573dced
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 10 Mar 2021 15:40:17 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 10 Mar 2021 15:40:17 +1300
Supply a simple implementation of the missing pthread_barrier_t type and
functions, for macOS.
Discussion: https://postgr.es/m/20200227180100.zyvjwzcpiokfsqm2%40alap3.anarazel.de
M configure
M configure.ac
M src/include/pg_config.h.in
A src/include/port/pg_pthread.h
A src/port/pthread_barrier_wait.c
M src/tools/msvc/Solution.pm
M src/tools/pgindent/typedefs.list
pgbench: Improve time logic.
commit : 547f04e7348b6ed992bd4a197d39661fe7c25097
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 10 Mar 2021 16:09:50 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 10 Mar 2021 16:09:50 +1300
Instead of instr_time (struct timespec) and the INSTR_XXX macros,
introduce pg_time_usec_t and use integer arithmetic. Don't include the
connection time in TPS unless using -C mode, but report it separately.
Author: Fabien COELHO <coelho@cri.ensmp.fr>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Discussion: https://postgr.es/m/20200227180100.zyvjwzcpiokfsqm2%40alap3.anarazel.de
M doc/src/sgml/ref/pgbench.sgml
M src/bin/pgbench/pgbench.c
M src/include/portability/instr_time.h
M src/tools/pgindent/typedefs.list
pgbench: Refactor thread portability support.
commit : b1d6a8f86813772b9198367a34c8ff8bff7fef9e
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 10 Mar 2021 15:39:08 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 10 Mar 2021 15:39:08 +1300
Instead of maintaining an incomplete emulation of POSIX threads for
Windows, let's use an extremely minimalist macro-based abstraction for
now. A later patch will extend this, without the need to supply more
complicated pthread emulation code. (There may be a need for a more
serious portable thread abstraction in later projects, but this is not
it.)
Minor incidental problems fixed: it wasn't OK to use (pthread_t) 0 as a
special value, it wasn't OK to compare thread_t values with ==, and we
incorrectly assumed that pthread functions set errno.
Discussion: https://postgr.es/m/20200227180100.zyvjwzcpiokfsqm2%40alap3.anarazel.de
M src/bin/pgbench/pgbench.c
Fix valgrind issue in commit 05c8482f7f.
commit : e4e87a32cc5559303068b148bd4aa6a39b8f44a1
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 10 Mar 2021 10:04:20 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 10 Mar 2021 10:04:20 +0530
Initialize other newly added variables in max_parallel_hazard_context via
is_parallel_safe() because we don't check the parallel-safety of target
relations in that function.
Reported-by: Tom Lane as per buildfarm
Author: Amit Kapila
Discussion: https://postgr.es/m/2060179.1615347455@sss.pgh.pa.us
M src/backend/optimizer/util/clauses.c
Enable parallel SELECT for "INSERT INTO ... SELECT ...".
commit : 05c8482f7f69a954fd65fce85f896e848fc48197
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 10 Mar 2021 07:38:58 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 10 Mar 2021 07:38:58 +0530
Parallel SELECT can't be utilized for INSERT in the following cases:
- INSERT statement uses the ON CONFLICT DO UPDATE clause
- Target table has a parallel-unsafe: trigger, index expression or
predicate, column default expression or check constraint
- Target table has a parallel-unsafe domain constraint on any column
- Target table is a partitioned table with a parallel-unsafe partition key
expression or support function
The planner is updated to perform additional parallel-safety checks for
the cases listed above, for determining whether it is safe to run INSERT
in parallel-mode with an underlying parallel SELECT. The planner will
consider using parallel SELECT for "INSERT INTO ... SELECT ...", provided
nothing unsafe is found from the additional parallel-safety checks, or
from the existing parallel-safety checks for SELECT.
While checking parallel-safety, we need to check it for all the partitions
on the table which can be costly especially when we decide not to use a
parallel plan. So, in a separate patch, we will introduce a GUC and or a
reloption to enable/disable parallelism for Insert statements.
Prior to entering parallel-mode for the execution of INSERT with parallel
SELECT, a TransactionId is acquired and assigned to the current
transaction state. This is necessary to prevent the INSERT from attempting
to assign the TransactionId whilst in parallel-mode, which is not allowed.
This approach has a disadvantage in that if the underlying SELECT does not
return any rows, then the TransactionId is not used, however that
shouldn't happen in practice in many cases.
Author: Greg Nancarrow, Amit Langote, Amit Kapila
Reviewed-by: Amit Langote, Hou Zhijie, Takayuki Tsunakawa, Antonin Houska, Bharath Rupireddy, Dilip Kumar, Vignesh C, Zhihong Yu, Amit Kapila
Tested-by: Tang, Haiying
Discussion: https://postgr.es/m/CAJcOf-cXnB5cnMKqWEp2E2z7Mvcd04iLVmV=qpFJrR3AcrTS3g@mail.gmail.com
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com
M doc/src/sgml/parallel.sgml
M src/backend/access/transam/xact.c
M src/backend/executor/execMain.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/util/clauses.c
M src/backend/utils/cache/plancache.c
M src/include/access/xact.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/clauses.h
A src/test/regress/expected/insert_parallel.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/insert_parallel.sql
Revert changes for SSL compression in libpq
commit : 0ba71107efeeccde9158f47118f95043afdca0bb
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 10 Mar 2021 09:35:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 10 Mar 2021 09:35:42 +0900
This partially reverts 096bbf7 and 9d2d457, undoing the libpq changes as
it could cause breakages in distributions that share one single libpq
version across multiple major versions of Postgres for extensions and
applications linking to that.
Note that the backend is unchanged here, and it still disables SSL
compression while simplifying the underlying catalogs that tracked if
compression was enabled or not for a SSL connection.
Per discussion with Tom Lane and Daniel Gustafsson.
Discussion: https://postgr.es/m/YEbq15JKJwIX+S6m@paquier.xyz
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/libpq.sgml
M src/bin/psql/command.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/libpq-int.h
M src/test/ssl/t/001_ssltests.pl
Fix vague comment in jsonb documentation
commit : 6540cc517dd452874a4e0fb268aee9b92e5136c6
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 9 Mar 2021 18:16:03 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 9 Mar 2021 18:16:03 +0300
The sample query fails because of an attempt to update the key of a numeric.
But the comment says it's just because of the missing object key. That's not
correct because jsonb subscription automatically adds missing keys.
Reported-by: Nikita Konev
M doc/src/sgml/json.sgml
libpq: Remove deprecated connection parameters authtype and tty
commit : 14d9b37607ad30c3848ea0f2955a78436eff1268
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 9 Mar 2021 15:01:22 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 9 Mar 2021 15:01:22 +0100
The authtype parameter was deprecated and made inactive in commit
d5bbe2aca55bc8, but the environment variable was left defined and thus
tested with a getenv call even though the value is of no use. Also,
if it would exist it would be copied but never freed as the cleanup
code had been removed.
tty was deprecated in commit cb7fb3ca958ec8bd5a14e7 but most of the
infrastructure around it remained in place.
Author: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/DDDF36F3-582A-4C02-8598-9B464CC42B34@yesql.se
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/libpq-int.h
M src/test/examples/testlibpq4.c
Switch back sslcompression to be a normal input field in libpq
commit : 096bbf7c934a4288c9e48a6ac8e91d8753ac1ccd
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 9 Mar 2021 19:52:36 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 9 Mar 2021 19:52:36 +0900
Per buildfarm member crake, any servers including a postgres_fdw server
with this option set would fail to do a pg_upgrade properly as the
option got hidden in f9264d1 by becoming a debug option, making the
restore of the FDW server fail.
This changes back the option in libpq to be visible, but still inactive
to fix this upgrade issue.
Discussion: https://postgr.es/m/YEbq15JKJwIX+S6m@paquier.xyz
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/interfaces/libpq/fe-connect.c
Track total amounts of times spent writing and syncing WAL data to disk.
commit : ff99918c625a84c91e7391db9032112ec8653623
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 9 Mar 2021 16:52:06 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 9 Mar 2021 16:52:06 +0900
This commit adds new GUC track_wal_io_timing. When this is enabled,
the total amounts of time XLogWrite writes and issue_xlog_fsync syncs
WAL data to disk are counted in pg_stat_wal. This information would be
useful to check how much WAL write and sync affect the performance.
Enabling track_wal_io_timing will make the server query the operating
system for the current time every time WAL is written or synced,
which may cause significant overhead on some platforms. To avoid such
additional overhead in the server with track_io_timing enabled,
this commit introduces track_wal_io_timing as a separate parameter from
track_io_timing.
Note that WAL write and sync activity by walreceiver has not been tracked yet.
This commit makes the server also track the numbers of times XLogWrite
writes and issue_xlog_fsync syncs WAL data to disk, in pg_stat_wal,
regardless of the setting of track_wal_io_timing. This counters can be
used to calculate the WAL write and sync time per request, for example.
Bump PGSTAT_FILE_FORMAT_ID.
Bump catalog version.
Author: Masahiro Ikeda
Reviewed-By: Japin Li, Hayato Kuroda, Masahiko Sawada, David Johnston, Fujii Masao
Discussion: https://postgr.es/m/0509ad67b585a5b86a83d445dfa75392@oss.nttdata.com
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/xlog.c
M src/backend/catalog/system_views.sql
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/walwriter.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/access/xlog.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
Add support for more progress reporting in COPY
commit : 9d2d45700928d49212fb7ed140feeaebe3a6014f
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 9 Mar 2021 14:21:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 9 Mar 2021 14:21:03 +0900
The command (TO or FROM), its type (file, pipe, program or callback),
and the number of tuples excluded by a WHERE clause in COPY FROM are
added to the progress reporting already available.
The column "lines_processed" is renamed to "tuples_processed" to
disambiguate the meaning of this column in the cases of CSV and BINARY
COPY and to be more consistent with the other catalog progress views.
Bump catalog version, again.
Author: Matthias van de Meent
Reviewed-by: Michael Paquier, Justin Pryzby, Bharath Rupireddy, Josef
Šimánek, Tomas Vondra
Discussion: https://postgr.es/m/CAEze2WiOcgdH4aQA8NtZq-4dgvnJzp8PohdeKchPkhMY-jWZXA@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/commands/copyfrom.c
M src/backend/commands/copyto.c
M src/include/catalog/catversion.h
M src/include/commands/progress.h
M src/test/regress/expected/rules.out
Remove support for SSL compression
commit : f9264d1524baa19e4a0528f033681ef16f61b137
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 9 Mar 2021 11:16:47 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 9 Mar 2021 11:16:47 +0900
PostgreSQL disabled compression as of e3bdb2d and the documentation
recommends against using it since. Additionally, SSL compression has
been disabled in OpenSSL since version 1.1.0, and was disabled in many
distributions long before that. The most recent TLS version, TLSv1.3,
disallows compression at the protocol level.
This commit removes the feature itself, removing support for the libpq
parameter sslcompression (parameter still listed for compatibility
reasons with existing connection strings, just ignored), and removes
the equivalent field in pg_stat_ssl and de facto PgBackendSSLStatus.
Note that, on top of removing the ability to activate compression by
configuration, compression is actively disabled in both frontend and
backend to avoid overrides from local configurations.
A TAP test is added for deprecated SSL parameters to check after
backwards compatibility.
Bump catalog version.
Author: Daniel Gustafsson
Reviewed-by: Peter Eisentraut, Magnus Hagander, Michael Paquier
Discussion: https://postgr.es/m/7E384D48-11C5-441B-9EC3-F7DB1F8518F6@yesql.se
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/libpq.sgml
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/libpq/be-secure-openssl.c
M src/backend/postmaster/pgstat.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/init/postinit.c
M src/bin/psql/command.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/libpq/libpq-be.h
M src/include/pgstat.h
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/libpq-int.h
M src/test/regress/expected/rules.out
M src/test/ssl/t/001_ssltests.pl
Complain if a function-in-FROM returns a set when it shouldn't.
commit : d4545dc19b8ea670bf62e06d22b0e4e6fcb45153
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Mar 2021 18:54:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Mar 2021 18:54:55 -0500
Throw a "function protocol violation" error if a function in FROM
tries to return a set though it wasn't marked proretset. Although
such cases work at the moment, it doesn't seem like something we
want to guarantee will keep working. Besides, there are other
negative consequences of not setting the proretset flag, such as
potentially bad plans.
No back-patch, since if there is any third-party code violating
this expectation, people wouldn't appreciate us breaking it in
a minor release.
Discussion: https://postgr.es/m/1636062.1615141782@sss.pgh.pa.us
M src/backend/executor/execSRF.c
Properly mark pg_stat_get_subscription() as returning a set.
commit : fed10d4eec79242688382d03ddca82007160ee6f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Mar 2021 18:47:23 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Mar 2021 18:47:23 -0500
The initial catalog data for this function failed to set proretset
or provide a prorows estimate. It accidentally worked anyway when
invoked in the FROM clause, because the executor isn't too picky
about this; but the planner didn't expect the function to return
multiple rows, which could lead to bad plans. Also the function
would fail if invoked in the SELECT list.
We can't easily back-patch this fix, but fortunately the bug's
consequences aren't awful in most cases. Getting this right is
mainly an exercise in future-proofing.
Discussion: https://postgr.es/m/1636062.1615141782@sss.pgh.pa.us
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Validate the OID argument of pg_import_system_collations().
commit : 5c06abb9b97b69513a3998cccf89556e73052e02
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Mar 2021 18:21:51 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Mar 2021 18:21:51 -0500
"SELECT pg_import_system_collations(0)" caused an assertion failure.
With a random nonzero argument --- or indeed with zero, in non-assert
builds --- it would happily make pg_collation entries with garbage
values of collnamespace. These are harmless as far as I can tell
(unless maybe the OID happens to become used for a schema, later on?).
In any case this isn't a security issue, since the function is
superuser-only. But it seems like a gotcha for unwary DBAs, so let's
add a check that the given OID belongs to some schema.
Back-patch to v10 where this function was introduced.
M src/backend/commands/collationcmds.c
Further tweak memory management for regex DFAs.
commit : 6c20bdb2a279086777a3595ab00bcf14671fc5a1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Mar 2021 16:32:29 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Mar 2021 16:32:29 -0500
Coverity is still unhappy after commit 190c79884, and after looking
closer I think it might be onto something. The callers of newdfa()
typically drop out if v->err has been set nonzero, which newdfa()
is faithfully doing if it fails. However, what if v->err was already
nonzero before we entered newdfa()? Then newdfa() could succeed and
the caller would promptly leak its result.
I don't think this scenario can actually happen, but the predicate
"v->err is always zero when newdfa() is called" seems difficult to be
entirely sure of; there's a good deal of code that potentially could
get that wrong.
It seems better to adjust the callers to directly check for a null
result instead of relying on ISERR() tests. This is slightly cheaper
than the previous coding anyway.
Lacking evidence that there's any real bug, no back-patch.
M src/backend/regex/rege_dfa.c
M src/backend/regex/regexec.c
Track replication origin progress for rollbacks.
commit : 8a812e5106c5db50039336288d376a188844e2cc
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 8 Mar 2021 07:54:03 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 8 Mar 2021 07:54:03 +0530
Commit 1eb6d6527a allowed to track replica origin replay progress for 2PC
but it was not complete. It misses to properly track the progress for
rollback prepared especially it missed updating the code for recovery.
Additionally, we need to allow tracking it on subscriber nodes where
wal_level might not be logical.
It is required to track decoding of 2PC which is committed in PG14
(a271a1b50e) and also nobody complained about this till now so not
backpatching it.
Author: Amit Kapila
Reviewed-by: Michael Paquier and Ajin Cherian
Discussion: https://postgr.es/m/CAA4eK1L-kHmMnSdrRW6UhRbCjR7cgh04c+6psY15qzT6ktcd+g@mail.gmail.com
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xact.c
Add bit_xor aggregate function
commit : f9a0392e1cf36179c984f655a6589734b8dd7468
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 6 Mar 2021 19:25:45 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 6 Mar 2021 19:25:45 +0100
This can be used as a checksum for unordered sets. bit_and and bit_or
already exist.
Author: Alexey Bashtanov <bashtanov@imap.cc>
Reviewed-by: Ibrar Ahmed <ibrar.ahmad@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/9d4582ae-ecfc-3a13-2238-6ab5a37c1f41@imap.cc
M doc/src/sgml/func.sgml
M src/include/catalog/catversion.h
M src/include/catalog/pg_aggregate.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql
Clarify that CREATEROLE roles are considered Administrators
commit : 30b26dfac7db2fc76dafaa8a6c459871bfcd100e
author : Magnus Hagander <magnus@hagander.net>
date : Sat, 6 Mar 2021 18:11:00 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Sat, 6 Mar 2021 18:11:00 +0100
Author: Michael Banck
Reviewed-By: Robert Treat
Discussion: https://postgr.es/m/2160a5071a7bb5339055b04a8cad81a822be9d8d.camel@credativ.de
M doc/src/sgml/user-manag.sgml
Add binary I/O capability for cube datatype.
commit : 1265a9c8f871cfa5aea4d607b1ab72b45b96dbb7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 6 Mar 2021 12:04:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 6 Mar 2021 12:04:00 -0500
We can adjust the not-yet-released cube--1.4--1.5.sql upgrade
rather than making a whole new version.
KaiGai Kohei
Discussion: https://postgr.es/m/CAOP8fzZO4y60QPTK=RGDXeVeVHV9tLHKOsh7voUOoUouVCPV8A@mail.gmail.com
M contrib/cube/cube–1.4–1.5.sql
M contrib/cube/cube.c
Improve docs on updatable views
commit : e045565dae4b2368ba812d60ef0fb7f927fe2646
author : Magnus Hagander <magnus@hagander.net>
date : Sat, 6 Mar 2021 17:36:48 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Sat, 6 Mar 2021 17:36:48 +0100
Introduce the options before going into details, and add a link to the
CREATE TRIGGER documentation.
Author: David Johnston
Reviewed-By: Anastasia Lubennikova
Discussion: https://postgr.es/m/CAKFQuwYLLRhheo0_Y4Jp=vJ_YDsz1KoRuTpX1A_bUxmHTmLe-A@mail.gmail.com
M doc/src/sgml/rules.sgml
pgbench: Simplify some port, host, user and dbname assignments
commit : f1516ad7b3a9645a316846fa7b2229163bc55907
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 6 Mar 2021 21:26:34 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 6 Mar 2021 21:26:34 +0900
Using pgbench in an environment with both PGPORT and PGUSER set would
have caused the generation of a debug log with an incorrect database
name due to an oversight in 412893b. Not specifying user, port and/or
database using the option switches, without their respective environment
variables, generated a log entry with empty strings, which was
rather useless.
This commit fixes this set of issues by simplifying the logic grabbing
the connection information, removing a set of getenv() calls that
emulated what libpq already does. The faulty debug log now directly
uses the information from the libpq connection, and it gets generated
after the connection to the backend is completed, not before it (in the
event of a failure libpq would complain with more information about the
connection attempt so the log is not really useful before anyway).
Author: Kota Miyake
Reviewed-by: Fujii Masao, Michael Paquier
Discussion: https://postgr.es/m/026b3ae6fc339a18394d053c32a4463d@oss.nttdata.com
M src/bin/pgbench/pgbench.c
Doc: remove obsolete entries in table of BRIN strategy numbers.
commit : 0ce4cd04da558178b0186057b721c50a00b7a945
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Mar 2021 10:59:23 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Mar 2021 10:59:23 -0500
See commit 591d282e8.
Noted by Justin Pryzby
Discussion: https://postgr.es/m/20201027032511.GF9241@telsasoft.com
M doc/src/sgml/brin.sgml
Remove deprecated containment operators for contrib types.
commit : 112d411fbeb56afd18c117e20b524a86afc9aba5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Mar 2021 10:45:41 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Mar 2021 10:45:41 -0500
Since PG 8.2, @ and ~ have been deprecated aliases for the containment
operators @> and <@. It seems like enough time has passed to actually
remove them, so do so.
This completes the project begun in commit 2f70fdb06. Note that in
the core types, the relation to the preferred operator names was
reversed from what it is in these contrib modules. The confusion
that induced was a large part of the reason for deprecation.
Justin Pryzby
Discussion: https://postgr.es/m/20201027032511.GF9241@telsasoft.com
M contrib/cube/Makefile
A contrib/cube/cube–1.4–1.5.sql
M contrib/cube/cube.control
M contrib/hstore/hstore–1.7–1.8.sql
M contrib/intarray/Makefile
A contrib/intarray/intarray–1.4–1.5.sql
M contrib/intarray/intarray.control
M contrib/seg/Makefile
A contrib/seg/seg–1.3–1.4.sql
M contrib/seg/seg.control
M doc/src/sgml/cube.sgml
M doc/src/sgml/hstore.sgml
M doc/src/sgml/intarray.sgml
M doc/src/sgml/seg.sgml
doc: Add backlinks to progress reporting documentation
commit : 8a8f4d8ede288c2a29105f4708e22ce7f3526149
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 5 Mar 2021 14:58:16 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 5 Mar 2021 14:58:16 +0900
Previously, the only place where progress reports were mentioned is in
the section for monitoring dedicated to its catalogs. This makes the
progress reporting more discoverable, by adding links from the pages of
the commands supporting progress reports to their related catalog
views.
Author: Matthias van de Meent
Reviewed-by: Justin Pryzby, Bharath Rupireddy, Josef Šimánek, Tomas
Vondra
Discussion: https://postgr.es/m/CAEze2WiOcgdH4aQA8NtZq-4dgvnJzp8PohdeKchPkhMY-jWZXA@mail.gmail.com
M doc/src/sgml/ref/analyze.sgml
M doc/src/sgml/ref/cluster.sgml
M doc/src/sgml/ref/copy.sgml
M doc/src/sgml/ref/create_index.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/reindex.sgml
M doc/src/sgml/ref/vacuum.sgml
Add support for PROVE_TESTS and PROVE_FLAGS in MSVC scripts
commit : 5bca69a76b3046a85c60c48271c1831fd5affa51
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 5 Mar 2021 10:12:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 5 Mar 2021 10:12:49 +0900
These can be set in buildenv.pl or a "set" command within a Windows
terminal. The MSVC script vcregress.pl parses the values available in
the environment to build the resulting prove commands, and the parsing
of PROVE_TESTS is able to handle name patterns in the same way as other
platforms.
Not specifying those environment values makes vcregress.pl fall back to
the previous default, with no extra flags for the prove command, and all
the tests run within t/.
Author: Michael Paquier
Reviewed-by: Juan José Santamaría Flecha, Julien Rouhaud
Discussion: https://postgr.es/m/YD9GigwHoL6lFY2y@paquier.xyz
M doc/src/sgml/install-windows.sgml
M src/tools/msvc/vcregress.pl
Close psql processes gracefully in recovery tests
commit : d3676a2e9f10a0972c6d6649235c1c7492cd6dea
author : Andrew Dunstan <andrew@dunslane.net>
date : Thu, 4 Mar 2021 13:13:10 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Thu, 4 Mar 2021 13:13:10 -0500
Under windows, psql processes need to be ended explicitly, or the TAP
tests hang. However, the recovery tests were doing this via
IPC::Run::kill_kill(), which causes other major problems on Windows.
We solve this by instead sending '\q' to psql so it quits of its own
accord, and then simply waiting for it. This means we can now run almost
all the recovery tests on all Windows platforms.
Discussion: https://postgr.es/m/20210301200715.tdjpuesfzebpffgn@alap3.anarazel.de
M src/test/recovery/t/011_crash_recovery.pl
M src/test/recovery/t/021_row_visibility.pl
pg_upgrade: Fix oversight in version checking
commit : 040af779382e8e4797242c49b93a5a8f9b79c370
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 4 Mar 2021 08:21:59 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 4 Mar 2021 08:21:59 +0100
Mistake in f06b1c598254f8adb2b7f51d6a7685618a7fb121: We should only
check the version of the binaries in the target installation. The
source installation can of course be of a different version.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://www.postgresql.org/message-id/flat/E1lHNKN-0005IC-V6%40gemulon.postgresql.org
M src/bin/pg_upgrade/exec.c
Remove redundant getenv() for PGUSER, in psql help.
commit : 4a4241e15b246663fc44b5e5355bc6d19c6d2aa6
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 4 Mar 2021 18:23:22 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 4 Mar 2021 18:23:22 +0900
Previously psql obtained the value of PGUSER twice to display
a default user in its help message.
Author: Kota Miyake
Reviewed-by: Nitin Jadhav, Fujii Masao
Discussion: https://postgr.es/m/2a3c612babdd6ed63a9d877bb575d793@oss.nttdata.com
M src/bin/psql/help.c
Avoid extra newline in errors received in FE protocol version 2.
commit : 85d94c575302644e7ddb4392c3bda572f981ec34
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 4 Mar 2021 10:56:33 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 4 Mar 2021 10:56:33 +0200
Contrary to what the comment said, the postmaster does in fact end all
its messages in a newline, since server version 7.2. Be tidy and don't
add an extra newline if the error message already has one.
Discussion: https://www.postgresql.org/message-id/CAFBsxsEdgMXc%2BtGfEy9Y41i%3D5pMMjKeH8t8vSAypR3ZnAoQnHg%40mail.gmail.com
M src/interfaces/libpq/fe-connect.c
Remove server and libpq support for old FE/BE protocol version 2.
commit : 3174d69fb96a66173224e60ec7053b988d5ed4d9
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 4 Mar 2021 10:45:55 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 4 Mar 2021 10:45:55 +0200
Protocol version 3 was introduced in PostgreSQL 7.4. There shouldn't be
many clients or servers left out there without version 3 support. But as
a courtesy, I kept just enough of the old protocol support that we can
still send the "unsupported protocol version" error in v2 format, so that
old clients can display the message properly. Likewise, libpq still
understands v2 ErrorResponse messages when establishing a connection.
The impetus to do this now is that I'm working on a patch to COPY
FROM, to always prefetch some data. We cannot do that safely with the
old protocol, because it requires parsing the input one byte at a time
to detect the end-of-copy marker.
Reviewed-by: Tom Lane, Alvaro Herrera, John Naylor
Discussion: https://www.postgresql.org/message-id/9ec25819-0a8a-d51a-17dc-4150bb3cca3b%40iki.fi
M doc/src/sgml/libpq.sgml
M src/backend/access/common/printtup.c
M src/backend/commands/async.c
M src/backend/commands/copyfrom.c
M src/backend/commands/copyfromparse.c
M src/backend/commands/copyto.c
M src/backend/libpq/auth.c
M src/backend/libpq/pqcomm.c
M src/backend/libpq/pqmq.c
M src/backend/postmaster/postmaster.c
M src/backend/tcop/dest.c
M src/backend/tcop/fastpath.c
M src/backend/tcop/postgres.c
M src/backend/utils/error/elog.c
M src/backend/utils/misc/guc.c
M src/bin/psql/common.c
M src/bin/psql/copy.c
M src/include/commands/copyfrom_internal.h
M src/include/libpq/libpq.h
M src/include/libpq/pqcomm.h
M src/include/tcop/fastpath.h
M src/include/utils/elog.h
M src/interfaces/libpq/Makefile
M src/interfaces/libpq/fe-auth.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-lobj.c
M src/interfaces/libpq/fe-misc.c
D src/interfaces/libpq/fe-protocol2.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
M src/interfaces/libpq/nls.mk
Add trim_array() function.
commit : 0a687c8f103d217ff1ca8c34a644b380d89bb0ad
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 3 Mar 2021 16:39:57 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 3 Mar 2021 16:39:57 -0500
This has been in the SQL spec since 2008. It's a pretty thin
wrapper around the array slice functionality, but the spec
says we should have it, so here it is.
Vik Fearing, reviewed by Dian Fay
Discussion: https://postgr.es/m/fc92ce17-9655-8ff1-c62a-4dc4c8ccd815@postgresfriends.org
M doc/src/sgml/func.sgml
M src/backend/catalog/sql_features.txt
M src/backend/utils/adt/arrayfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/arrays.out
M src/test/regress/sql/arrays.sql
Make test_target_session_attrs more robust against connection failure.
commit : 3769e11a31831fc2f3bd4c4a24b4f45c352fb8fb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 3 Mar 2021 13:51:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 3 Mar 2021 13:51:43 -0500
Feed the desired command to psql via "-c" not stdin, else Perl
may complain that it can't push stdin to an already-failed psql
process, as seen in intermittent buildfarm failures.
Make some minor cosmetic improvements while at it.
Before commit ee28cacf6 we had no tests here that expected failure
to connect, so there seems no need for a back-patch.
Discussion: https://postgr.es/m/CALDaNm2mo8YED=M2ZJKGf1U3F3mw6SaQuLXWCK8rZP6sECYcrA@mail.gmail.com
M src/test/recovery/t/001_stream_rep.pl
pg_upgrade: Check version of target cluster binaries
commit : f06b1c598254f8adb2b7f51d6a7685618a7fb121
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 3 Mar 2021 09:44:46 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 3 Mar 2021 09:44:46 +0100
This expands the binary validation in pg_upgrade with a version
check per binary to ensure that the target cluster installation
only contains binaries from the target version.
In order to reduce duplication, validate_exec is exported from
port.h and the local copy in pg_upgrade is removed.
Author: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://www.postgresql.org/message-id/flat/9328.1552952117@sss.pgh.pa.us
M src/bin/pg_upgrade/exec.c
M src/common/exec.c
M src/include/port.h
Clarify the usage of max_replication_slots on the subscriber side.
commit : 8af3c233e423e106a5121000ecd92abf57c11704
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 3 Mar 2021 12:01:56 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 3 Mar 2021 12:01:56 +0530
It was not clear in the docs that the max_replication_slots is also used
to track replication origins on the subscriber side.
Author: Paul Martinez
Reviewed-by: Amit Kapila
Backpatch-through: 10 where logical replication was introduced
Discussion: https://postgr.es/m/CACqFVBZgwCN_pHnW6dMNCrOS7tiHCw6Retf_=U2Vvj3aUSeATw@mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/logical-replication.sgml
Some copy-editing of GUC descriptions
commit : e527a99055154bb7fe6ed67f0417a67cf11fa056
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 3 Mar 2021 07:14:35 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 3 Mar 2021 07:14:35 +0100
M src/backend/utils/misc/guc.c
Silence perlcritic warning in commit ee28cacf6.
commit : d422a2a94b1e7a7bb25da9d5511fffff750c3f21
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Mar 2021 23:32:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Mar 2021 23:32:43 -0500
Per buildfarm; this fix is from Michael Paquier (vignesh C
proposed nearly the same).
Discussion: https://postgr.es/m/YD8IZ9OKfUf9X1eF@paquier.xyz
M src/test/recovery/t/001_stream_rep.pl
Use sort_template.h for qsort_tuple() and qsort_ssup().
commit : 8eda3eba3063d619214b59f7bc7ecfc09c21b93a
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 3 Mar 2021 16:57:42 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 3 Mar 2021 16:57:42 +1300
Replace the Perl code previously used to generate specialized sort
functions with sort_template.h.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CA%2BhUKGJ2-eaDqAum5bxhpMNhvuJmRDZxB_Tow0n-gse%2BHG0Yig%40mail.gmail.com
M src/backend/Makefile
D src/backend/utils/sort/.gitignore
M src/backend/utils/sort/Makefile
D src/backend/utils/sort/gen_qsort_tuple.pl
M src/backend/utils/sort/tuplesort.c
M src/tools/msvc/Solution.pm
M src/tools/msvc/clean.bat
Use sort_template.h for qsort() and qsort_arg().
commit : f374f4d66425060158d3c743dc6fb16cc0a2756a
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 3 Mar 2021 16:57:26 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 3 Mar 2021 16:57:26 +1300
Reduce duplication by using the new template.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CA%2BhUKGJ2-eaDqAum5bxhpMNhvuJmRDZxB_Tow0n-gse%2BHG0Yig%40mail.gmail.com
M src/port/qsort.c
M src/port/qsort_arg.c
Add sort_template.h for making sort functions.
commit : 0a1f1d3cac6baaa3744b89336673caba702f7628
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 3 Mar 2021 16:57:14 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 3 Mar 2021 16:57:14 +1300
Move our qsort implementation into a header that can be used to define
specialized functions for better performance and reduced duplication.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CA%2BhUKGJ2-eaDqAum5bxhpMNhvuJmRDZxB_Tow0n-gse%2BHG0Yig%40mail.gmail.com
A src/include/lib/sort_template.h
M src/tools/pgindent/typedefs.list
Add option to enable two_phase commits via pg_create_logical_replication_slot.
commit : 19890a064ebf53dedcefed0d8339ed3d449b06e6
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 3 Mar 2021 07:28:43 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 3 Mar 2021 07:28:43 +0530
Commit 0aa8a01d04 extends the output plugin API to allow decoding of
prepared xacts and allowed the user to enable/disable the two-phase option
via pg_logical_slot_get_changes(). This can lead to a problem such that
the first time when it gets changes via pg_logical_slot_get_changes()
without two_phase option enabled it will not get the prepared even though
prepare is after consistent snapshot. Now next time during getting changes,
if the two_phase option is enabled it can skip prepare because by that
time start decoding point has been moved. So the user will only get commit
prepared.
Allow to enable/disable this option at the create slot time and default
will be false. It will break the existing slots which is fine in a major
release.
Author: Ajin Cherian
Reviewed-by: Amit Kapila and Vignesh C
Discussion: https://postgr.es/m/d0f60d60-133d-bf8d-bd70-47784d8fabf3@enterprisedb.com
M contrib/test_decoding/expected/twophase.out
M contrib/test_decoding/expected/twophase_snapshot.out
M contrib/test_decoding/expected/twophase_stream.out
M contrib/test_decoding/specs/twophase_snapshot.spec
M contrib/test_decoding/sql/twophase.sql
M contrib/test_decoding/sql/twophase_stream.sql
M contrib/test_decoding/test_decoding.c
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/logicaldecoding.sgml
M src/backend/catalog/system_views.sql
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/walsender.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/nodes/replnodes.h
M src/include/replication/slot.h
M src/test/regress/expected/rules.out
Extend the abilities of libpq's target_session_attrs parameter.
commit : ee28cacf619f4d9c23af5a80e1171a5adae97381
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Mar 2021 20:17:45 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Mar 2021 20:17:45 -0500
In addition to the existing options of "any" and "read-write", we
now support "read-only", "primary", "standby", and "prefer-standby".
"read-write" retains its previous meaning of "transactions are
read-write by default", and "read-only" inverts that. The other
three modes test specifically for hot-standby status, which is not
quite the same thing. (Setting default_transaction_read_only on
a primary server renders it read-only to this logic, but not a
standby.)
Furthermore, if talking to a v14 or later server, no extra network
round trip is needed to detect the session's status; the GUC_REPORT
variables delivered by the server are enough. When talking to an
older server, a SHOW or SELECT query is issued to detect session
read-only-ness or server hot-standby state, as needed.
Haribabu Kommi, Greg Nancarrow, Vignesh C, Tom Lane; reviewed at
various times by Laurenz Albe, Takayuki Tsunakawa, Peter Smith.
Discussion: https://postgr.es/m/CAF3+xM+8-ztOkaV9gHiJ3wfgENTq97QcjXQt+rbFQ6F7oNzt9A@mail.gmail.com
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
M src/test/recovery/t/001_stream_rep.pl
Add --tablespace option to reindexdb
commit : 57e6db706e81fd2609fa385677e6ae72471822fe
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 3 Mar 2021 10:14:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 3 Mar 2021 10:14:21 +0900
This option provides REINDEX (TABLESPACE) for reindexdb, applying the
tablespace value given by the caller to all the REINDEX queries
generated.
While on it, this commit adds some tests for REINDEX TABLESPACE, with
and without CONCURRENTLY, when run on toast indexes and tables. Such
operations are not allowed, and toast relation names are not stable
enough to be part of the main regression test suite (even if using a PL
function with a TRY/CATCH logic, as CONCURRENTLY could not be tested).
Author: Michael Paquier
Reviewed-by: Mark Dilger, Daniel Gustafsson
Discussion: https://postgr.es/m/YDiaDMnzLICqeukl@paquier.xyz
M doc/src/sgml/ref/reindexdb.sgml
M src/bin/scripts/reindexdb.c
M src/bin/scripts/t/090_reindexdb.pl
nbtree page deletion: Add leaftopparent assertion.
commit : 5b2f2af3d9d57626b9052a05337e32ad1399093d
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 2 Mar 2021 14:06:07 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 2 Mar 2021 14:06:07 -0800
Add documenting assertion. This makes it easier to follow how we
maintain the top parent link in target subtree's half-dead/leaf level
page.
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtxlog.c
Fix nbtree page deletion error messages.
commit : 3d8d5787a358156edaa7782f0c88e231af974a01
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 2 Mar 2021 13:02:24 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 2 Mar 2021 13:02:24 -0800
Adjust some "can't happen" error messages that assumed that the page
deletion target page must be a half-dead page. This assumption was
wrong in the case of an internal target page. Simply refer to these
pages as the target page instead.
Internal pages are never marked half-dead. There is exactly one
half-dead page for each subtree undergoing deletion. The half-dead page
is also the target subtree's leaf-level page. This has been the case
since commit efada2b8, which totally overhauled nbtree page deletion.
M src/backend/access/nbtree/nbtpage.c
Mark default_transaction_read_only as GUC_REPORT.
commit : d16f8c8e416d288bd4734ed5f14076b62ec8d153
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Mar 2021 13:53:46 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Mar 2021 13:53:46 -0500
This allows clients to find out the setting at connection time without
having to expend a query round trip to do so; which is helpful when
trying to identify read/write servers. (One must also look at
in_hot_standby, but that's already GUC_REPORT, cf bf8a662c9.)
Modifying libpq to make use of this will come soon, but I felt it
cleaner to push the server change separately.
Haribabu Kommi, Greg Nancarrow, Vignesh C; reviewed at various times
by Laurenz Albe, Takayuki Tsunakawa, Peter Smith.
Discussion: https://postgr.es/m/CAF3+xM+8-ztOkaV9gHiJ3wfgENTq97QcjXQt+rbFQ6F7oNzt9A@mail.gmail.com
M doc/src/sgml/libpq.sgml
M doc/src/sgml/protocol.sgml
M src/backend/utils/misc/guc.c
Use native path separators to pg_ctl in initdb
commit : 75dbfe4ca70d5b7a83f61b0a66a0a14ad0b739ed
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 2 Mar 2021 15:39:34 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 2 Mar 2021 15:39:34 -0300
On Windows, CMD.EXE allegedly does not run a command that uses forward slashes,
so let's convert the path to use backslashes instead.
Backpatch to 10.
Author: Nitin Jadhav <nitinjadhavpostgres@gmail.com>
Reviewed-by: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
Discussion: https://postgr.es/m/CAMm1aWaNDuaPYFYMAqDeJrZmPtNvLcJRS++CcZWY8LT6KcoBZw@mail.gmail.com
M src/bin/initdb/initdb.c
Suppress unnecessary regex subre nodes in a couple more cases.
commit : 4604f83fdfe030a2f1984159ede5270c1d825310
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Mar 2021 12:14:14 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Mar 2021 12:14:14 -0500
This extends the changes made in commit cebc1d34e, teaching
parseqatom() to generate fewer or cheaper subre nodes in some edge
cases. The case of interest here is a quantified atom that is "messy"
only because it has greediness opposite to what preceded it (whereas
captures and backrefs are intrinsically messy). In this case we don't
need an iteration node, since we don't care where the sub-matches of
the quantifier are; and we might also not need a second concatenation
node. This seems of only marginal real-world use according to my
testing, but I wanted to get it in before wrapping up this series of
regex performance fixes.
Discussion: https://postgr.es/m/1340281.1613018383@sss.pgh.pa.us
M src/backend/regex/regcomp.c
Improve performance of regular expression back-references.
commit : 0c3405cf11a12da1a4278c6833f4d979fe06c866
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Mar 2021 11:55:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Mar 2021 11:55:12 -0500
In some cases, at the time that we're doing an NFA-based precheck
of whether a backref subexpression can match at a particular place
in the string, we already know which substring the referenced
subexpression matched. If so, we might as well forget about the NFA
and just compare the substring; this is faster and it gives an exact
rather than approximate answer.
In general, this optimization can help while we are prechecking within
the second child expression of a concat node, while the capture was
within the first child expression; then the substring was saved during
cdissect() of the first child and will be available to NFA checks done
while cdissect() recurses into the second child. It can help quite a
lot if the tree looks like
concat
/ \
capture concat
/ \
expensive stuff backref
as we will be able to avoid recursively dissecting the "expensive
stuff" before discovering that the backref isn't satisfied with a
particular midpoint that the lower concat node is testing. This
doesn't help if the concat tree is left-deep, as the capture node
won't get set soon enough (and it's hard to fix that without changing
the engine's match behavior). Fortunately, right-deep concat trees
are the common case.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/661609.1614560029@sss.pgh.pa.us
M src/backend/regex/rege_dfa.c
M src/backend/regex/regexec.c
Fix semantics of regular expression back-references.
commit : 4aea704a5bfd4b5894a268499369ccab89940c9c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Mar 2021 11:34:53 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Mar 2021 11:34:53 -0500
POSIX defines the behavior of back-references thus:
The back-reference expression '\n' shall match the same (possibly
empty) string of characters as was matched by a subexpression
enclosed between "\(" and "\)" preceding the '\n'.
As far as I can see, the back-reference is supposed to consider only
the data characters matched by the referenced subexpression. However,
because our engine copies the NFA constructed from the referenced
subexpression, it effectively enforces any constraints therein, too.
As an example, '(^.)\1' ought to match 'xx', or any other string
starting with two occurrences of the same character; but in our code
it does not, and indeed can't match anything, because the '^' anchor
constraint is included in the backref's copied NFA. If POSIX intended
that, you'd think they'd mention it. Perl for one doesn't act that
way, so it's hard to conclude that this isn't a bug.
Fix by modifying the backref's NFA immediately after it's copied from
the reference, replacing all constraint arcs by EMPTY arcs so that the
constraints are treated as automatically satisfied. This still allows
us to enforce matching rules that depend only on the data characters;
for example, in '(^\d+).*\1' the NFA matching step will still know
that the backref can only match strings of digits.
Perhaps surprisingly, this change does not affect the results of any
of a rather large corpus of real-world regexes. Nonetheless, I would
not consider back-patching it, since it's a clear compatibility break.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/661609.1614560029@sss.pgh.pa.us
M doc/src/sgml/func.sgml
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Fix duplicated test case in TAP tests of reindexdb
commit : c5530d8474a482d32c0d9bb099707d9a8e117f96
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 2 Mar 2021 13:18:06 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 2 Mar 2021 13:18:06 +0900
The same test for REINDEX (VERBOSE) was done twice, while it is clear
that the second test should use --concurrently. Issue introduced in
5dc92b8, for what looks like a copy-paste mistake.
Reviewed-by: Mark Dilger
Discussion: https://postgr.es/m/A7AE97EA-F4B0-4CAB-8FFF-3FECD31F9D63@enterprisedb.com
Backpatch-through: 12
M src/bin/scripts/t/090_reindexdb.pl
Simplify code to switch pg_class.relrowsecurity in tablecmds.c
commit : fabde52fab976ff782d77c80b8e2614f94f9dce1
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 2 Mar 2021 12:30:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 2 Mar 2021 12:30:21 +0900
The same code pattern was repeated twice to enable or disable ROW LEVEL
SECURITY with an ALTER TABLE command. This makes the code slightly
cleaner.
Author: Justin Pryzby
Reviewed-by: Zhihong Yu
Discussion: https://postgr.es/m/20210228211854.GC20769@telsasoft.com
M src/backend/commands/tablecmds.c
doc: Improve description of data checksums
commit : bd1b8d0ef2eb5f2a8765fbb3b74e0262c9983942
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 2 Mar 2021 10:50:13 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 2 Mar 2021 10:50:13 +0900
This partially reverts bcf2667 that got incorrectly merged, and this
improves the wording of the documentation that existed before that.
Per discussion with Justin Pryzby.
Discussion: https://postgr.es/m/20210301004647.GF20769@telsasoft.com
M doc/src/sgml/wal.sgml
doc: Mention archive_command failure handling on signals
commit : 8c1b6a186d420de863d1334c8917dcd047a7845a
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 2 Mar 2021 10:25:47 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 2 Mar 2021 10:25:47 +0900
The behavior is similar to restore_command, which was already documented
for the restore part, but not the archive part.
Author: Benoit Lobréau
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAPE8EZ7akCzc1hWohA4AcbmKtHh9rcWAB5MStOeZD2+9jC+hLQ@mail.gmail.com
M doc/src/sgml/backup.sgml
Improve reporting for syntax errors in multi-line JSON data.
commit : ffd3944ab9d481906137bc7d20f5325a2bd68acc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Mar 2021 16:44:17 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Mar 2021 16:44:17 -0500
Point to the specific line where the error was detected; the
previous code tended to include several preceding lines as well.
Avoid re-scanning the entire input to recompute which line that
was. Simplify the logic a bit. Add test cases.
Simon Riggs and Hamid Akhtar, reviewed by Daniel Gustafsson and myself
Discussion: https://postgr.es/m/CANbhV-EPBnXm3MF_TTWBwwqgn1a1Ghmep9VHfqmNBQ8BT0f+_g@mail.gmail.com
M src/backend/utils/adt/jsonfuncs.c
M src/common/jsonapi.c
M src/include/common/jsonapi.h
M src/test/regress/expected/json.out
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/json.sql
M src/test/regress/sql/jsonb.sql
Remove obsolete comment for WaitForProcSignalBarrier().
commit : bd69ddfcdbf650f11af4317f3f6686c012cf66d0
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 2 Mar 2021 09:15:15 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 2 Mar 2021 09:15:15 +1300
Commit 814f1d8b removed the behavior described.
Reported-by: Amit Kapila <amit.kapila16@gmail.com>
M src/backend/storage/ipc/procsignal.c
Fix recovery test hang in 021_row_visibility.pl on windows.
commit : 1e6e40447115ca7b4749d7d117b81b016ee5e2c2
author : Andres Freund <andres@anarazel.de>
date : Mon, 1 Mar 2021 09:52:15 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 1 Mar 2021 09:52:15 -0800
The psql processes were not explicitly killed (but would eventually
exit due postgres shutting down). For some reason windows perl doesn't
like that, resulting in errors like
Warning: unable to close filehandle GEN20 properly: Bad file descriptor during global destruction.
The test was introduced in d6734a897e3, so no backpatching necessary.
M src/test/recovery/t/021_row_visibility.pl
Allow condition variables to be used in interrupt code.
commit : f5a5773a9dc4185414fe538525e20d8512c2ba35
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 16:28:12 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 16:28:12 +1300
Adjust the condition variable sleep loop to work correctly when code
reached by its internal CHECK_FOR_INTERRUPTS() call interacts with
another condition variable.
There are no such cases currently, but a proposed patch would do this.
Discussion: https://postgr.es/m/CA+hUKGLdemy2gBm80kz20GTe6hNVwoErE8KwcJk6-U56oStjtg@mail.gmail.com
M src/backend/storage/lmgr/condition_variable.c
Use condition variables for ProcSignalBarriers.
commit : 814f1d8bc395bb9b4c0ae090ceea60adfbeb8e49
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 16:26:22 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 16:26:22 +1300
Instead of a poll/sleep loop, use a condition variable for precise
wake-up whenever a backend's pss_barrierGeneration advances.
Discussion: https://postgr.es/m/CA+hUKGLdemy2gBm80kz20GTe6hNVwoErE8KwcJk6-U56oStjtg@mail.gmail.com
M src/backend/storage/ipc/procsignal.c
Avoid repeated decoding of prepared transactions after a restart.
commit : 8bdb1332eb51837c15a10a972c179b84f654279e
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 1 Mar 2021 09:11:18 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 1 Mar 2021 09:11:18 +0530
In commit a271a1b50e, we allowed decoding at prepare time and the prepare
was decoded again if there is a restart after decoding it. It was done
that way because we can't distinguish between the cases where we have not
decoded the prepare because it was prior to consistent snapshot or we have
decoded it earlier but restarted. To distinguish between these two cases,
we have introduced an initial_consistent_point at the slot level which is
an LSN at which we found a consistent point at the time of slot creation.
This is also the point where we have exported a snapshot for the initial
copy. So, prepare transaction prior to this point are sent along with
commit prepared.
This commit bumps SNAPBUILD_VERSION because of change in SnapBuild. It
will break existing slots which is fine in a major release.
Author: Ajin Cherian, based on idea by Andres Freund
Reviewed-by: Amit Kapila and Vignesh C
Discussion: https://postgr.es/m/d0f60d60-133d-bf8d-bd70-47784d8fabf3@enterprisedb.com
M contrib/test_decoding/expected/twophase.out
M contrib/test_decoding/expected/twophase_stream.out
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/include/replication/reorderbuffer.h
M src/include/replication/slot.h
M src/include/replication/snapbuild.h
Use FeBeWaitSet for walsender.c.
commit : 6230912f23904aa6cb2a1f948ca9b08235b4f54a
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 15:17:28 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 15:17:28 +1300
This avoids the need to set up and tear down a fresh WaitEventSet every
time we need need to wait. We have to add an explicit exit on
postmaster exit (FeBeWaitSet isn't set up to do that automatically), so
move the code to do that into a new function to avoid repetition.
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> (earlier version)
Discussion: https://postgr.es/m/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw%40mail.gmail.com
M src/backend/replication/walsender.c
Introduce symbolic names for FeBeWaitSet positions.
commit : a042ba2ba7bb44f191f6f9398da27f04a958a0db
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 15:16:56 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 15:16:56 +1300
Previously we used 0 and 1 to refer to the socket and latch in far flung
parts of the tree, without any explanation. Also use PGINVALID_SOCKET
rather than -1 in a couple of places that didn't already do that.
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw%40mail.gmail.com
M src/backend/libpq/be-secure.c
M src/backend/libpq/pqcomm.c
M src/backend/utils/init/miscinit.c
M src/include/libpq/libpq.h
Update docs of logical replication for commit ce0fdbfe97.
commit : cf54e04b9e29a13c5e7c2386c5e2ff785e830414
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 1 Mar 2021 08:23:41 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 1 Mar 2021 08:23:41 +0530
Forgot to update the logical replication configuration settings page.
After commit ce0fdbfe97, table synchronization workers also started using
replication origins to track the progress and the same should be reflected
in docs.
Author: Amit Kapila
Discussion: https://postgr.es/m/CAA4eK1KkbppndxxRKbaT2sXrLkdPwy44F4pjEZ0EDrVjD9MPjQ@mail.gmail.com
M doc/src/sgml/logical-replication.sgml
Update the docs and comments for decoding of prepared xacts.
commit : b4e3dc7fd420cdc2287f30a638c4affd071f01b2
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 1 Mar 2021 08:14:33 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 1 Mar 2021 08:14:33 +0530
Commit a271a1b50e introduced decoding at prepare time in ReorderBuffer.
This can lead to deadlock for out-of-core logical replication solutions
that uses this feature to build distributed 2PC in case such transactions
lock [user] catalog tables exclusively. They need to inform users to not
have locks on catalog tables (via explicit LOCK command) in such
transactions.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/20210222222847.tpnb6eg3yiykzpky@alap3.anarazel.de
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/decode.c
Use EVFILT_SIGNAL for kqueue latches.
commit : 6148656a0be1c6245fbcfcbbeb87541f1b173162
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 12:28:43 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 12:28:43 +1300
Cut down on system calls and other overheads by waiting for SIGURG
explicitly with kqueue instead of using a signal handler and self-pipe.
Affects *BSD and macOS systems.
This leaves only the poll implementation with a signal handler and the
traditional self-pipe trick.
Discussion: https://postgr.es/m/CA+hUKGJjxPDpzBE0a3hyUywBvaZuC89yx3jK9RFZgfv_KHU7gg@mail.gmail.com
M src/backend/storage/ipc/latch.c
Use signalfd(2) for epoll latches.
commit : 6a2a70a02018d6362f9841cc2f499cc45405e86b
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 12:06:09 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 12:06:09 +1300
Cut down on system calls and other overheads by reading from a signalfd
instead of using a signal handler and self-pipe. Affects Linux sytems,
and possibly others including illumos that implement the Linux epoll and
signalfd interfaces.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA+hUKGJjxPDpzBE0a3hyUywBvaZuC89yx3jK9RFZgfv_KHU7gg@mail.gmail.com
M src/backend/libpq/pqsignal.c
M src/backend/storage/ipc/latch.c
M src/backend/utils/init/miscinit.c
Use SIGURG rather than SIGUSR1 for latches.
commit : 83709a0d5a46559db016c50ded1a95fd3b0d3be6
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 11:58:10 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 11:58:10 +1300
Traditionally, SIGUSR1 has been overloaded for ad-hoc signals,
procsignal.c signals and latch.c wakeups. Move that last use over to a
new dedicated signal. SIGURG is normally used to report out-of-band
socket data, but PostgreSQL doesn't use that facility.
The signal handler is now installed in all postmaster children by
InitializeLatchSupport(). Those wishing to disconnect from it should
call ShutdownLatchSupport().
Future patches will use this separation of signals to avoid the need for
a signal handler on some operating systems.
Discussion: https://postgr.es/m/CA+hUKGJjxPDpzBE0a3hyUywBvaZuC89yx3jK9RFZgfv_KHU7gg@mail.gmail.com
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/postmaster.c
M src/backend/storage/ipc/latch.c
M src/backend/storage/ipc/procsignal.c
M src/include/storage/latch.h
Optimize latches to send fewer signals.
commit : c8f3bc2401e7df7b79bae39dd3511c91f825b6a4
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 11:51:15 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 11:51:15 +1300
Don't send signals to processes that aren't sleeping.
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA+hUKGJjxPDpzBE0a3hyUywBvaZuC89yx3jK9RFZgfv_KHU7gg@mail.gmail.com
M src/backend/storage/ipc/latch.c
M src/include/storage/latch.h
Remove latch.c workaround for Linux < 2.6.27.
commit : d1b90995e8d41df7e59efe48e98f26cd66baba9b
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 10:44:21 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 1 Mar 2021 10:44:21 +1300
Commit 82ebbeb0 added a workaround for systems with no epoll_create1()
and EPOLL_CLOEXEC. Linux < 2.6.27 and glibc < 2.9 are long gone. Now
seems like a good time to drop the extra code, because otherwise we'd
have to add similar already-dead workaround code to new patches using
XXX_CLOEXEC flags that arrived in the same kernel release.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA%2BhUKGKL_%3DaO%3Dr30N%3Ds9VoDgTqHpRSzePRbA9dkYO7snc7HsxA%40mail.gmail.com
M src/backend/storage/ipc/latch.c
pgbench: Remove now-dead CState->ecnt
commit : 943eb478803cc2ed6f0b1f5df6f1b20c9ad6245d
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 28 Feb 2021 07:50:26 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 28 Feb 2021 07:50:26 +0900
The last use of ecnt was in 12788ae. It was getting incremented after a
backend error without any purpose since then, so let's get rid of it.
Author: Kota Miyake
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/786c3d9fbe067763d899e78c296f9f0f@oss.nttdata.com
M src/bin/pgbench/pgbench.c
Fix use-after-free bug with AfterTriggersTableData.storeslot
commit : 25936fd46c97039aad042ae8d46917d38d132fe4
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 27 Feb 2021 18:09:15 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 27 Feb 2021 18:09:15 -0300
AfterTriggerSaveEvent() wrongly allocates the slot in execution-span
memory context, whereas the correct thing is to allocate it in
a transaction-span context, because that's where the enclosing
AfterTriggersTableData instance belongs into.
Backpatch to 12 (the test back to 11, where it works well with no code
changes, and it's good to have to confirm that the case was previously
well supported); this bug seems introduced by commit ff11e7f4b9ae.
Reported-by: Bertrand Drouvot <bdrouvot@amazon.com>
Author: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/39a71864-b120-5a5c-8cc5-c632b6f16761@amazon.com
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Raise a timeout to 180s, in contrib/test_decoding.
commit : 388b959315205b0b65efb074ec84e1d7fad62402
author : Noah Misch <noah@leadboat.com>
date : Sat, 27 Feb 2021 07:02:56 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 27 Feb 2021 07:02:56 -0800
Per buildfarm member hornet. The test is new in v14, so no back-patch.
M contrib/test_decoding/expected/twophase.out
M contrib/test_decoding/sql/twophase.sql
Add missing TidRangeScan readfunc
commit : 977b2c08535f2a82ba7c310c88420cbbca1772e8
author : David Rowley <drowley@postgresql.org>
date : Sat, 27 Feb 2021 23:21:21 +1300
committer: David Rowley <drowley@postgresql.org>
date : Sat, 27 Feb 2021 23:21:21 +1300
Mistakenly forgotten in bb437f995
M src/backend/nodes/readfuncs.c
Add TID Range Scans to support efficient scanning ranges of TIDs
commit : bb437f995d47405ecd92cf66df71f7f7e40ed460
author : David Rowley <drowley@postgresql.org>
date : Sat, 27 Feb 2021 22:59:36 +1300
committer: David Rowley <drowley@postgresql.org>
date : Sat, 27 Feb 2021 22:59:36 +1300
This adds a new executor node named TID Range Scan. The query planner
will generate paths for TID Range scans when quals are discovered on base
relations which search for ranges on the table's ctid column. These
ranges may be open at either end. For example, WHERE ctid >= '(10,0)';
will return all tuples on page 10 and over.
To support this, two new optional callback functions have been added to
table AM. scan_set_tidrange is used to set the scan range to just the
given range of TIDs. scan_getnextslot_tidrange fetches the next tuple
in the given range.
For AMs were scanning ranges of TIDs would not make sense, these functions
can be set to NULL in the TableAmRoutine. The query planner won't
generate TID Range Scan Paths in that case.
Author: Edmund Horner, David Rowley
Reviewed-by: David Rowley, Tomas Vondra, Tom Lane, Andres Freund, Zhihong Yu
Discussion: https://postgr.es/m/CAMyN-kB-nFTkF=VA_JPwFNo08S0d-Yk0F741S2B7LDmYAi8eyA@mail.gmail.com
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/commands/explain.c
M src/backend/executor/Makefile
M src/backend/executor/execAmi.c
M src/backend/executor/execCurrent.c
M src/backend/executor/execProcnode.c
A src/backend/executor/nodeTidrangescan.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/README
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/tidpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/plancat.c
M src/backend/optimizer/util/relnode.c
M src/backend/storage/page/itemptr.c
M src/include/access/heapam.h
M src/include/access/relscan.h
M src/include/access/tableam.h
M src/include/catalog/pg_operator.dat
A src/include/executor/nodeTidrangescan.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/cost.h
M src/include/optimizer/pathnode.h
M src/include/storage/itemptr.h
A src/test/regress/expected/tidrangescan.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/tidrangescan.sql
Enhanced cycle mark values
commit : f4adc41c4f92cc91d507b19e397140c35bb9fd71
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 27 Feb 2021 08:11:14 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 27 Feb 2021 08:11:14 +0100
Per SQL:202x draft, in the CYCLE clause of a recursive query, the
cycle mark values can be of type boolean and can be omitted, in which
case they default to TRUE and FALSE.
Reviewed-by: Vik Fearing <vik@postgresfriends.org>
Discussion: https://www.postgresql.org/message-id/flat/db80ceee-6f97-9b4a-8ee8-3ba0c58e5be2@2ndquadrant.com
M doc/src/sgml/queries.sgml
M doc/src/sgml/ref/select.sgml
M src/backend/catalog/sql_features.txt
M src/backend/parser/gram.y
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Doc: further clarify libpq's description of connection string URIs.
commit : 4e90052c46c7751779ed83627676ed5e74ebe6d4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 26 Feb 2021 15:24:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 26 Feb 2021 15:24:00 -0500
Break the synopsis into named parts to make it less confusing.
Make more than zero effort at applying SGML markup. Do a bit
of copy-editing of nearby text.
The synopsis revision is by Alvaro Herrera and Paul Förster,
the rest is my fault. Back-patch to v10 where multi-host
connection strings appeared.
Discussion: https://postgr.es/m/6E752D6B-487C-463E-B6E2-C32E7FB007EA@gmail.com
M doc/src/sgml/libpq.sgml
Improve memory management in regex compiler.
commit : 0fc1af174cf7113445e116feb2813405b838a47d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 26 Feb 2021 13:52:10 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 26 Feb 2021 13:52:10 -0500
The previous logic here created a separate pool of arcs for each
state, so that the out-arcs of each state were physically stored
within it. Perhaps this choice was driven by trying to not include
a "from" pointer within each arc; but Spencer gave up on that idea
long ago, and it's hard to see what the value is now. The approach
turns out to be fairly disastrous in terms of memory consumption,
though. In the first place, NFAs built by this engine seem to have
about 4 arcs per state on average, with a majority having only one
or two out-arcs. So pre-allocating 10 out-arcs for each state is
already cause for a factor of two or more bloat. Worse, the NFA
optimization phase moves arcs around with abandon. In a large NFA,
some of the states will have hundreds of out-arcs, so towards the
end of the optimization phase we have a significant number of states
whose arc pools have room for hundreds of arcs each, even though only
a few of those arcs are in use. We have seen real-world regexes in
which this effect bloats the memory requirement by 25X or even more.
Hence, get rid of the per-state arc pools in favor of a single arc
pool for the whole NFA, with variable-sized allocation batches
instead of always asking for 10 at a time. While we're at it,
let's batch the allocations of state structs too, to further reduce
the malloc traffic.
This incidentally allows moveouts() to be optimized in a similar
way to moveins(): when moving an arc to another state, it's now
valid to just re-link the same arc struct into a different outchain,
where before the code invariants required us to make a physically
new arc and then free the old one.
These changes reduce the regex compiler's typical space consumption
for average-size regexes by about a factor of two, and much more for
large or complicated regexes. In a large test set of real-world
regexes, we formerly had half a dozen cases that failed with "regular
expression too complex" due to exceeding the REG_MAX_COMPILE_SPACE
limit (about 150MB); we would have had to raise that limit to
something close to 400MB to make them work with the old code. Now,
none of those cases need more than 13MB to compile. Furthermore,
the test set is about 10% faster overall due to less malloc traffic.
Discussion: https://postgr.es/m/168861.1614298592@sss.pgh.pa.us
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/include/regex/regguts.h
Extend a test case a little
commit : b3a9e9897ec702d56602b26a8cdc0950f23b29dc
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 26 Feb 2021 09:11:15 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 26 Feb 2021 09:11:15 +0100
This will possibly help a subsequent patch by making sure the notice
messages are distinct so that it's clear that they come out in the
right order.
Author: Fabien COELHO <coelho@cri.ensmp.fr>
Discussion: https://www.postgresql.org/message-id/alpine.DEB.2.21.1904240654120.3407%40lancre
M src/test/regress/expected/copydml.out
M src/test/regress/sql/copydml.sql
doc: Improve {archive,restore}_command for compressed logs
commit : 329784e11849ff691f0157f3b27c50f652bce76a
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 26 Feb 2021 14:39:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 26 Feb 2021 14:39:03 +0900
The commands mentioned in the docs with gzip and gunzip did not prefix
the archives with ".gz" and used inconsistent paths for the archives,
which can be confusing.
Reported-by: Philipp Gramzow
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/161397938841.15451.13129264141285167267@wrigleys.postgresql.org
M doc/src/sgml/backup.sgml
Revert "pg_collation_actual_version() -> pg_collation_current_version()."
commit : 8556267b2b1b8e1c26037c4c25cf390ee5afb5d9
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 26 Feb 2021 15:29:27 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 26 Feb 2021 15:29:27 +1300
This reverts commit 9cf184cc0599b6e65e7e5ecd9d91cd42e278bcd8. Name
change less well received than anticipated.
Discussion: https://postgr.es/m/afcfb97e-88a1-a540-db95-6c573b93bc2b%40eisentraut.org
M doc/src/sgml/func.sgml
M src/backend/commands/collationcmds.c
M src/backend/utils/adt/pg_locale.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Fix list-manipulation bug in WITH RECURSIVE processing.
commit : 80ca8464fe02296c8efefd53746e6d6a3f456d1e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 25 Feb 2021 20:47:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 25 Feb 2021 20:47:32 -0500
makeDependencyGraphWalker and checkWellFormedRecursionWalker
thought they could hold onto a pointer to a list's first
cons cell while the list was modified by recursive calls.
That was okay when the cons cell was actually separately
palloc'd ... but since commit 1cff1b95a, it's quite unsafe,
leading to core dumps or incorrect complaints of faulty
WITH nesting.
In the field this'd require at least a seven-deep WITH nest
to cause an issue, but enabling DEBUG_LIST_MEMORY_USAGE
allows the bug to be seen with lesser nesting depths.
Per bug #16801 from Alexander Lakhin. Back-patch to v13.
Michael Paquier and Tom Lane
Discussion: https://postgr.es/m/16801-393c7922143eaa4d@postgresql.org
M src/backend/parser/parse_cte.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
VACUUM VERBOSE: Count "newly deleted" index pages.
commit : 2376361839091b0dcdcc0b77f938b809b5f21646
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 25 Feb 2021 14:32:18 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 25 Feb 2021 14:32:18 -0800
Teach VACUUM VERBOSE to report on pages deleted by the _current_ VACUUM
operation -- these are newly deleted pages. VACUUM VERBOSE continues to
report on the total number of deleted pages in the entire index (no
change there). The former is a subset of the latter.
The distinction between each category of deleted index page only arises
with index AMs where page deletion is supported and is decoupled from
page recycling for performance reasons.
This is follow-up work to commit e5d8a999, which made nbtree store
64-bit XIDs (not 32-bit XIDs) in pages at the point at which they're
deleted. Note that the btm_last_cleanup_num_delpages metapage field
added by that commit usually gets set to pages_newly_deleted. The
exceptions (the scenarios in which they're not equal) all seem to be
tricky cases for the implementation (of page deletion and recycling) in
general.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX%3Dd5u-tRYhi-F4wnV2uN2zHpMUXw%40mail.gmail.com
M src/backend/access/gin/ginvacuum.c
M src/backend/access/gist/gistvacuum.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/spgist/spgvacuum.c
M src/include/access/genam.h
M src/include/access/nbtree.h
Doc: remove src/backend/regex/re_syntax.n.
commit : 301ed8812e3f8b102b85e1f5a59e95990ed9a868
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 25 Feb 2021 13:33:27 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 25 Feb 2021 13:33:27 -0500
We aren't publishing this file as documentation, and it's been
much more haphazardly maintained than the real docs in func.sgml,
so let's just drop it. I think the only reason I included it in
commit 7bcc6d98f was that the Berkeley-era sources had had a man
page in this directory.
Discussion: https://postgr.es/m/4099447.1614186542@sss.pgh.pa.us
D src/backend/regex/re_syntax.n
Change regex \D and \W shorthands to always match newlines.
commit : 7dc13a0f0805a353cea0455ed95701322b39d4dd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 25 Feb 2021 13:29:06 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 25 Feb 2021 13:29:06 -0500
Newline is certainly not a digit, nor a word character, so it is
sensible that it should match these complemented character classes.
Previously, \D and \W acted that way by default, but in
newline-sensitive mode ('n' or 'p' flag) they did not match newlines.
This behavior was previously forced because explicit complemented
character classes don't match newlines in newline-sensitive mode;
but as of the previous commit that implementation constraint no
longer exists. It seems useful to change this because the primary
real-world use for newline-sensitive mode seems to be to match the
default behavior of other regex engines such as Perl and Javascript
... and their default behavior is that these match newlines.
The old behavior can be kept by writing an explicit complemented
character class, i.e. [^[:digit:]] or [^[:word:]]. (This means
that \D and \W are not exactly equivalent to those strings, but
they weren't anyway.)
Discussion: https://postgr.es/m/3220564.1613859619@sss.pgh.pa.us
M doc/src/sgml/func.sgml
M src/backend/regex/re_syntax.n
M src/backend/regex/regcomp.c
M src/test/modules/test_regex/expected/test_regex.out
Allow complemented character class escapes within regex brackets.
commit : 2a0af7fe460eb46f9af996075972bf7c2e3f211d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 25 Feb 2021 13:00:40 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 25 Feb 2021 13:00:40 -0500
The complement-class escapes \D, \S, \W are now allowed within
bracket expressions. There is no semantic difficulty with doing
that, but the rather hokey macro-expansion-based implementation
previously used here couldn't cope.
Also, invent "word" as an allowed character class name, thus "\w"
is now equivalent to "[[:word:]]" outside brackets, or "[:word:]"
within brackets. POSIX allows such implementation-specific
extensions, and the same name is used in e.g. bash.
One surprising compatibility issue this raises is that constructs
such as "[\w-_]" are now disallowed, as our documentation has always
said they should be: character classes can't be endpoints of a range.
Previously, because \w was just a macro for "[:alnum:]_", such a
construct was read as "[[:alnum:]_-_]", so it was accepted so long as
the character after "-" was numerically greater than or equal to "_".
Some implementation cleanup along the way:
* Remove the lexnest() hack, and in consequence clean up wordchrs()
to not interact with the lexer.
* Fix colorcomplement() to not be O(N^2) in the number of colors
involved.
* Get rid of useless-as-far-as-I-can-see calls of element()
on single-character character element names in brackpart().
element() always maps these to the character itself, and things
would be quite broken if it didn't --- should "[a]" match something
different than "a" does? Besides, the shortcut path in brackpart()
wasn't doing this anyway, making it even more inconsistent.
Discussion: https://postgr.es/m/2845172.1613674385@sss.pgh.pa.us
Discussion: https://postgr.es/m/3220564.1613859619@sss.pgh.pa.us
M doc/src/sgml/func.sgml
M src/backend/regex/re_syntax.n
M src/backend/regex/regc_color.c
M src/backend/regex/regc_lex.c
M src/backend/regex/regc_locale.c
M src/backend/regex/regc_pg_locale.c
M src/backend/regex/regcomp.c
M src/include/regex/regguts.h
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Improve tab-completion for TRUNCATE.
commit : 6b40d9bdbdc9f873868b0ddecacd9a307fc8ee26
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 25 Feb 2021 18:20:57 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 25 Feb 2021 18:20:57 +0900
Author: Kota Miyake
Reviewed-by: Muhammad Usama
Discussion: https://postgr.es/m/f5d30053d00dcafda3280c9e267ecb0f@oss.nttdata.com
M src/bin/psql/tab-complete.c
doc: Mention PGDATABASE as supported by pgbench
commit : a6f8dc47a0582e37c234c8e916377b46d2b03817
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 25 Feb 2021 16:06:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 25 Feb 2021 16:06:54 +0900
PGHOST, PGPORT and PGUSER were already mentioned, but not PGDATABASE.
Like 5aaa584, backpatch down to 12.
Reported-by: Christophe Courtois
Discussion: https://postgr.es/m/161399398648.21711.15387267201764682579@wrigleys.postgresql.org
Backpatch-through: 12
M doc/src/sgml/ref/pgbench.sgml
Use full 64-bit XIDs in deleted nbtree pages.
commit : e5d8a999030418a1b9e53d5f15ccaca7ed674877
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 24 Feb 2021 18:41:34 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 24 Feb 2021 18:41:34 -0800
Otherwise we risk "leaking" deleted pages by making them non-recyclable
indefinitely. Commit 6655a729 did the same thing for deleted pages in
GiST indexes. That work was used as a starting point here.
Stop storing an XID indicating the oldest bpto.xact across all deleted
though unrecycled pages in nbtree metapages. There is no longer any
reason to care about that condition/the oldest XID. It only ever made
sense when wraparound was something _bt_vacuum_needs_cleanup() had to
consider.
The btm_oldest_btpo_xact metapage field has been repurposed and renamed.
It is now btm_last_cleanup_num_delpages, which is used to remember how
many non-recycled deleted pages remain from the last VACUUM (in practice
its value is usually the precise number of pages that were _newly
deleted_ during the specific VACUUM operation that last set the field).
The general idea behind storing btm_last_cleanup_num_delpages is to use
it to give _some_ consideration to non-recycled deleted pages inside
_bt_vacuum_needs_cleanup() -- though never too much. We only really
need to avoid leaving a truly excessive number of deleted pages in an
unrecycled state forever. We only do this to cover certain narrow cases
where no other factor makes VACUUM do a full scan, and yet the index
continues to grow (and so actually misses out on recycling existing
deleted pages).
These metapage changes result in a clear user-visible benefit: We no
longer trigger full index scans during VACUUM operations solely due to
the presence of only 1 or 2 known deleted (though unrecycled) blocks
from a very large index. All that matters now is keeping the costs and
benefits in balance over time.
Fix an issue that has been around since commit 857f9c36, which added the
"skip full scan of index" mechanism (i.e. the _bt_vacuum_needs_cleanup()
logic). The accuracy of btm_last_cleanup_num_heap_tuples accidentally
hinged upon _when_ the source value gets stored. We now always store
btm_last_cleanup_num_heap_tuples in btvacuumcleanup(). This fixes the
issue because IndexVacuumInfo.num_heap_tuples (the source field) is
expected to accurately indicate the state of the table _after_ the
VACUUM completes inside btvacuumcleanup().
A backpatchable fix cannot easily be extracted from this commit. A
targeted fix for the issue will follow in a later commit, though that
won't happen today.
I (pgeoghegan) have chosen to remove any mention of deleted pages in the
documentation of the vacuum_cleanup_index_scale_factor GUC/param, since
the presence of deleted (though unrecycled) pages is no longer of much
concern to users. The vacuum_cleanup_index_scale_factor description in
the docs now seems rather unclear in any case, and it should probably be
rewritten in the near future. Perhaps some passing mention of page
deletion will be added back at the same time.
Bump XLOG_PAGE_MAGIC due to nbtree WAL records using full XIDs now.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX=d5u-tRYhi-F4wnV2uN2zHpMUXw@mail.gmail.com
M contrib/amcheck/verify_nbtree.c
M contrib/pageinspect/btreefuncs.c
M contrib/pageinspect/expected/btree.out
M contrib/pageinspect/pageinspect–1.8–1.9.sql
M contrib/pgstattuple/pgstatindex.c
M doc/src/sgml/config.sgml
M doc/src/sgml/pageinspect.sgml
M src/backend/access/gist/gistxlog.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/rmgrdesc/nbtdesc.c
M src/backend/storage/ipc/standby.c
M src/include/access/nbtree.h
M src/include/access/nbtxlog.h
M src/include/access/xlog_internal.h
M src/include/storage/standby.h
Fix relcache reference leak introduced by ce0fdbfe97.
commit : 8a4f9522d0c7d197c7eaa62cc72fb1bf9b90f05b
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 25 Feb 2021 07:48:24 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 25 Feb 2021 07:48:24 +0530
Author: Sawada Masahiko
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAD21AoA7ZEfsOXQ9HQqMv3QYGsEm2H5Wk5ic5S=mvzDf-3a3SA@mail.gmail.com
M src/backend/catalog/pg_subscription.c
Fix some typos, grammar and style in docs and comments
commit : bcf2667bf62d72faced64cb60ffbd2e599a0ebe8
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 24 Feb 2021 16:13:17 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 24 Feb 2021 16:13:17 +0900
The portions fixing the documentation are backpatched where needed.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20210210235557.GQ20012@telsasoft.com
backpatch-through: 9.6
M contrib/pageinspect/heapfuncs.c
M doc/src/sgml/charset.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/pageinspect.sgml
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/alter_subscription.sgml
M doc/src/sgml/ref/create_type.sgml
M doc/src/sgml/ref/drop_index.sgml
M doc/src/sgml/rules.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/common/heaptuple.c
M src/backend/access/transam/xlogutils.c
M src/backend/commands/amcmds.c
M src/backend/executor/nodeAgg.c
M src/backend/optimizer/prep/prepagg.c
M src/backend/storage/ipc/procarray.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/adt/selfuncs.c
M src/backend/utils/cache/catcache.c
M src/bin/pg_rewind/libpq_source.c
M src/bin/pgbench/pgbench.c
M src/include/pg_config_manual.h
M src/interfaces/ecpg/include/sqlda-native.h
M src/test/regress/expected/geometry.out
M src/test/regress/expected/tuplesort.out
M src/test/regress/sql/geometry.sql
M src/test/regress/sql/tuplesort.sql
Message style fix
commit : 8ec8fe0f31712c62b761da9ef6d32214e08340d1
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 24 Feb 2021 07:00:49 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 24 Feb 2021 07:00:49 +0100
Don't quote type name placeholders.
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/hash_part.out
doc: Improve description of wal_receiver_status_interval
commit : c82d59d64e127a5e743f9cf16537f4c7c120ce7b
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 24 Feb 2021 11:15:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 24 Feb 2021 11:15:58 +0900
This parameter description was previously confusing, telling that a
value of 0 disabled completely status updates. This is not true as
there are cases where an update is sent while ignoring this parameter
value. The documentation is improved to outline the difference of
treatment for scheduled status messages and when these are forced.
Reported-by: Dmitriy Kuzmin
Author: Michael Paquier
Reviewed-by: Euler Taveira
Discussion: https://postgr.es/m/161346024420.3455.1345266601055047937@wrigleys.postgresql.org
M doc/src/sgml/config.sgml
Fix confusion in comments about generate_gather_paths
commit : 5a65eacfdc69ee8c9d3ed92d838bc53628fbb7c6
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 23 Feb 2021 20:04:47 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 23 Feb 2021 20:04:47 -0300
d2d8a229bc58 introduced a new function generate_useful_gather_paths to
be used as a replacement for generate_gather_paths, but forgot to update
a couple of places that referenced the older function.
This is possibly not 100% complete (ref. create_ordered_paths), but it's
better than not changing anything.
Author: "Hou, Zhijie" <houzj.fnst@cn.fujitsu.com>
Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://postgr.es/m/4ce1d5116fe746a699a6d29858c6a39a@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/plan/planner.c
Reinstate HEAP_XMAX_LOCK_ONLY|HEAP_KEYS_UPDATED as allowed
commit : 8deb6b38dc4c7a7fd4719ee45e4b00d62b27dffe
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 23 Feb 2021 17:30:21 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 23 Feb 2021 17:30:21 -0300
Commit 866e24d47db1 added an assert that HEAP_XMAX_LOCK_ONLY and
HEAP_KEYS_UPDATED cannot appear together, on the faulty assumption that
the latter necessarily referred to an update and not a tuple lock; but
that's wrong, because SELECT FOR UPDATE can use precisely that
combination, as evidenced by the amcheck test case added here.
Remove the Assert(), and also patch amcheck's verify_heapam.c to not
complain if the combination is found. Also, out of overabundance of
caution, update (across all branches) README.tuplock to be more explicit
about this.
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Mahendra Singh Thalor <mahi6run@gmail.com>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Discussion: https://postgr.es/m/20210124061758.GA11756@nol
M contrib/amcheck/t/001_verify_heapam.pl
M contrib/amcheck/verify_heapam.c
M src/backend/access/heap/README.tuplock
M src/backend/access/heap/hio.c
Suppress compiler warning in new regex match-all detection code.
commit : 3db05e76f92846d4b54d7de251b0875cf1e23aa4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 23 Feb 2021 13:55:11 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 23 Feb 2021 13:55:11 -0500
gcc 10 is smart enough to notice that control could reach this
"hasmatch[depth]" assignment with depth < 0, but not smart enough
to know that that would require a badly broken NFA graph. Change
the assert() to a plain runtime test to shut it up.
Per report from Andres Freund.
Discussion: https://postgr.es/m/20210223173437.b3ywijygsy6q42gq@alap3.anarazel.de
M src/backend/regex/regc_nfa.c
VACUUM: ignore indexing operations with CONCURRENTLY
commit : d9d076222f5b94a85e0e318339cfc44b8f26022d
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 23 Feb 2021 12:15:09 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 23 Feb 2021 12:15:09 -0300
As envisioned in commit c98763bf51bf, it is possible for VACUUM to
ignore certain transactions that are executing CREATE INDEX CONCURRENTLY
and REINDEX CONCURRENTLY for the purposes of computing Xmin; that's
because we know those transactions are not going to examine any other
tables, and are not going to execute anything else in the same
transaction. (Only operations on "safe" indexes can be ignored: those
on indexes that are neither partial nor expressional).
This is extremely useful in cases where CIC/RC can run for a very long
time, because that used to be a significant headache for concurrent
vacuuming of other tables.
Reviewed-by: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/20210115133858.GA18931@alvherre.pgsql
M doc/src/sgml/ref/create_index.sgml
M doc/src/sgml/ref/reindex.sgml
M src/backend/storage/ipc/procarray.c
M src/backend/utils/misc/guc.c
Simplify printing of LSNs
commit : 6f6f284c7ee44264eb3e128e2bf54d9276711d11
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 23 Feb 2021 10:14:38 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 23 Feb 2021 10:14:38 +0100
Add a macro LSN_FORMAT_ARGS for use in printf-style printing of LSNs.
Convert all applicable code to use it.
Reviewed-by: Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/CAExHW5ub5NaTELZ3hJUCE6amuvqAtsSxc7O+uK7y4t9Rrk23cw@mail.gmail.com
M contrib/amcheck/verify_nbtree.c
M contrib/pageinspect/rawpage.c
M src/backend/access/heap/rewriteheap.c
M src/backend/access/rmgrdesc/replorigindesc.c
M src/backend/access/rmgrdesc/xactdesc.c
M src/backend/access/rmgrdesc/xlogdesc.c
M src/backend/access/transam/timeline.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogutils.c
M src/backend/replication/backup_manifest.c
M src/backend/replication/basebackup.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/origin.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/syncrep.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walsender.c
M src/backend/storage/ipc/standby.c
M src/backend/utils/adt/pg_lsn.c
M src/bin/pg_basebackup/pg_receivewal.c
M src/bin/pg_basebackup/pg_recvlogical.c
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_controldata/pg_controldata.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/bin/pg_waldump/pg_waldump.c
M src/include/access/xlogdefs.h
Fix an oversight in ReorderBufferFinishPrepared.
commit : ade89ba5f408e6034db7cc8a2c9b7858f5a214c4
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 23 Feb 2021 09:47:41 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 23 Feb 2021 09:47:41 +0530
We don't have anything to decode in a transaction if ReorderBufferTXN
doesn't exist by the time we decode the commit prepared. So don't create a
new ReorderBufferTXN here. This is an oversight in commit a271a1b5.
Reported-by: Markus Wanner
Discussion: https://postgr.es/m/dbec82e2-dbd7-95a2-c6b6-e488cbbdf853@bluegap.ch
M src/backend/replication/logical/reorderbuffer.c
Change the error message for logical replication authentication failure.
commit : bc617a7b1cada2ec5f8b9d45a4312e49fc1766ad
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 23 Feb 2021 09:11:22 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 23 Feb 2021 09:11:22 +0530
The authentication failure error message wasn't distinguishing whether
it is a physical replication or logical replication connection failure and
was giving incomplete information on what led to failure in case of logical
replication connection.
Author: Paul Martinez and Amit Kapila
Reviewed-by: Euler Taveira and Amit Kapila
Discussion: https://postgr.es/m/CACqFVBYahrAi2OPdJfUA3YCvn3QMzzxZdw0ibSJ8wouWeDtiyQ@mail.gmail.com
M doc/src/sgml/client-auth.sgml
M src/backend/libpq/auth.c
Remove pointless HeapTupleHeaderIndicatesMovedPartitions calls
commit : 0f5505a8815aa4eb03ca61245a5a51ff9adda2f4
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 22 Feb 2021 16:51:44 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 22 Feb 2021 16:51:44 -0300
Pavan Deolasee recently noted that a few of the
HeapTupleHeaderIndicatesMovedPartitions calls added by commit
5db6df0c0117 are useless, since they are done after comparing t_self
with t_ctid. But because t_self can never be set to the magical values
that indicate that the tuple moved partition, this can never succeed: if
the first test fails (so we know t_self equals t_ctid), necessarily the
second test will also fail.
So these checks can be removed and no harm is done. There's no bug
here, just a code legibility issue.
Reported-by: Pavan Deolasee <pavan.deolasee@gmail.com>
Discussion: https://postgr.es/m/20200929164411.GA15497@alvherre.pgsql
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_visibility.c
Fix typo
commit : 6a03369a71d4a7dc5b8d928aab775ddd28b72494
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 22 Feb 2021 11:34:05 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 22 Feb 2021 11:34:05 -0300
M src/backend/storage/ipc/procarray.c
Fix docs build for website styles
commit : d22d0fa937616a3112f69ebd6a6ed4f039162441
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 22 Feb 2021 13:00:54 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 22 Feb 2021 13:00:54 +0100
Building the docs with STYLE=website referenced a stylesheet that long
longer exists on the website, since we changed it to use versioned
references.
To make it less likely for this to happen again, point to a single
stylesheet on the website which will in turn import the required one.
That puts the process entirely within the scope of the website
repository, so next time a version is switched that's the only place
changes have to be made, making them less likely to be missed.
Per (off-list) discussion with Peter Geoghegan and Jonathan Katz.
M doc/src/sgml/stylesheet.xsl
Tab-complete CREATE COLLATION.
commit : 5bc09a74719dfeb6c4cebb311b81385c508459a8
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 22 Feb 2021 23:17:31 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 22 Feb 2021 23:17:31 +1300
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20210117215940.GE8560%40telsasoft.com
M src/bin/psql/tab-complete.c
Refactor get_collation_current_version().
commit : beb4480c853a60ec43bd3f1a71252890dd234f10
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 22 Feb 2021 23:03:27 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 22 Feb 2021 23:03:27 +1300
The code paths for three different OSes finished up with three different
ways of excluding C[.xxx] and POSIX from consideration. Merge them.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20210117215940.GE8560%40telsasoft.com
M src/backend/utils/adt/pg_locale.c
pg_collation_actual_version() -> pg_collation_current_version().
commit : 9cf184cc0599b6e65e7e5ecd9d91cd42e278bcd8
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 22 Feb 2021 23:02:52 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 22 Feb 2021 23:02:52 +1300
The new name seems a bit more natural.
Discussion: https://postgr.es/m/20210117215940.GE8560%40telsasoft.com
M doc/src/sgml/func.sgml
M src/backend/commands/collationcmds.c
M src/backend/utils/adt/pg_locale.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Hide internal error for pg_collation_actual_version(<bad OID>).
commit : 0fb0a0503bfc125764c8dba4f515058145dc7f8b
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 22 Feb 2021 23:01:20 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 22 Feb 2021 23:01:20 +1300
Instead of an unsightly internal "cache lookup failed" message, just
return NULL for bad OIDs, as is the convention for other similar things.
Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20210117215940.GE8560%40telsasoft.com
M src/backend/catalog/index.c
M src/backend/catalog/pg_depend.c
M src/backend/commands/collationcmds.c
M src/backend/utils/adt/pg_locale.c
M src/include/utils/pg_locale.h
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Initialize atomic variable waitStart in PGPROC, at postmaster startup.
commit : f05ed5a5cfa55878baa77a1e39d68cb09793b477
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 22 Feb 2021 18:25:00 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 22 Feb 2021 18:25:00 +0900
Commit 46d6e5f567 added the atomic variable "waitStart" into PGPROC struct,
to store the time at which wait for lock acquisition started. Previously
this variable was initialized every time each backend started. Instead,
this commit makes postmaster initialize it at the startup, to ensure that
the variable should be initialized before any use of it.
This commit also moves the code to initialize "waitStart" variable for
prepare transaction, from TwoPhaseGetDummyProc() to MarkAsPreparingGuts().
Because MarkAsPreparingGuts() is more proper place to do that since
it initializes other PGPROC variables.
Author: Fujii Masao
Reviewed-by: Atsushi Torikoshi
Discussion: https://postgr.es/m/1df88660-6f08-cc6e-b7e2-f85296a2bdab@oss.nttdata.com
M src/backend/access/transam/twophase.c
M src/backend/storage/lmgr/proc.c
Improve new hash partition bound check error messages
commit : efbfb642414b61db1567a7a902ace3f307d7564a
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 22 Feb 2021 08:06:45 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 22 Feb 2021 08:06:45 +0100
For the error message "every hash partition modulus must be a factor
of the next larger modulus", add a detail message that shows the
particular numbers and existing partition involved. Also comment the
code more.
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/bb9d60b4-aadb-607a-1a9d-fdc3434dddcd%40enterprisedb.com
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/create_table.out
Use pgstat_progress_update_multi_param() where possible
commit : 92942642788c9d73e4c090ee0a77603f7afbc1d7
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Feb 2021 14:21:40 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Feb 2021 14:21:40 +0900
This commit changes one code path in REINDEX INDEX and one code path
in CREATE INDEX CONCURRENTLY to report the progress of each operation
using pgstat_progress_update_multi_param() rather than
multiple calls to pgstat_progress_update_param(). This has the
advantage to make the progress report more consistent to the end-user
without impacting the amount of information provided.
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/CALj2ACV5zW7GxD8D_tyO==bcj6ZktQchEKWKPBOAGKiLhAQo=w@mail.gmail.com
M src/backend/catalog/index.c
M src/backend/commands/indexcmds.c
Remove outdated reference to RAID spindles.
commit : db8374d804f2dd35a0f934984d1d3686cc71f545
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 22 Feb 2021 14:37:28 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 22 Feb 2021 14:37:28 +1300
Commit b09ff536 left behind some outdated advice in the long_desc field
of the GUC "effective_io_concurrency". Remove it.
Back-patch to 13.
Reported-by: Andrew Gierth <andrew@tao11.riddles.org.uk>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGJyyWqFBxL9gEj-qtjBThGjhAOBE8GBnF8MUJOJ3vrfag%40mail.gmail.com
M src/backend/utils/misc/guc.c
Simplify memory management for regex DFAs a little.
commit : 190c79884aae540c92f017701726ed69265e2dab
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Feb 2021 20:29:11 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Feb 2021 20:29:11 -0500
Coverity complained that functions in regexec.c might leak DFA
storage. It's wrong, but this logic is confusing enough that it's
not so surprising Coverity couldn't make sense of it. Rewrite
in hopes of making it more legible to humans as well as machines.
M src/backend/regex/rege_dfa.c
M src/backend/regex/regexec.c
Fix invalid array access in trgm_regexp.c.
commit : 6ee479abfc27a18c37fe77140d16d3ac31f4ac31
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Feb 2021 19:46:46 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 21 Feb 2021 19:46:46 -0500
Brown-paper-bag bug in 08c0d6ad6: I missed one place that needed
to guard against RAINBOW arc colors. Remarkably, nothing noticed
the invalid array access except buildfarm member thorntail.
Thanks to Noah Misch for assistance with tracking this down.
M contrib/pg_trgm/trgm_regexp.c
Avoid generating extra subre tree nodes for capturing parentheses.
commit : ea1268f6301cc7adce571cc9c5ebe8d9342a2ef4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Feb 2021 19:26:41 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Feb 2021 19:26:41 -0500
Previously, each pair of capturing parentheses gave rise to a separate
subre tree node, whose only function was to identify that we ought to
capture the match details for this particular sub-expression. In
most cases we don't really need that, since we can perfectly well
put a "capture this" annotation on the child node that does the real
matching work. As with the two preceding commits, the main value
of this is to avoid generating and optimizing an NFA for a tree node
that's not really pulling its weight.
The chosen data representation only allows one capture annotation
per subre node. In the legal-per-spec, but seemingly not very useful,
case where there are multiple capturing parens around the exact same
bit of the regex (i.e. "((xyz))"), wrap the child node in N-1 capture
nodes that act the same as before. We could work harder at that but
I'll refrain, pending some evidence that such cases are worth troubling
over.
In passing, improve the comments in regex.h to say what all the
different re_info bits mean. Some of them were pretty obvious
but others not so much, so reverse-engineer some documentation.
This is part of a patch series that in total reduces the regex engine's
runtime by about a factor of four on a large corpus of real-world regexes.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/1340281.1613018383@sss.pgh.pa.us
M src/backend/regex/README
M src/backend/regex/regcomp.c
M src/backend/regex/rege_dfa.c
M src/backend/regex/regexec.c
M src/include/regex/regex.h
M src/include/regex/regguts.h
Convert regex engine's subre tree from binary to N-ary style.
commit : 581043089472816061a7fd381f40572191dfa48f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Feb 2021 19:07:45 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Feb 2021 19:07:45 -0500
Instead of having left and right child links in subre structs,
have a single child link plus a sibling link. Multiple children
of a tree node are now reached by chasing the sibling chain.
The beneficiary of this is alternation tree nodes. A regular
expression with N (>1) branches is now represented by one alternation
node with N children, rather than a tree that includes N alternation
nodes as well as N children. While the old representation didn't
really cost anything extra at execution time, it was pretty horrid
for compilation purposes, because each of the alternation nodes had
its own NFA, which we were too stupid not to separately optimize.
(To make matters worse, all of those NFAs described the entire
alternation pattern, not just the portion of it that one might
expect from the tree structure.)
We continue to require concatenation nodes to have exactly two
children. This data structure is now prepared to support more,
but the executor's logic would need some careful redesign, and
it's not clear that a lot of benefit could be had.
This is part of a patch series that in total reduces the regex engine's
runtime by about a factor of four on a large corpus of real-world regexes.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/1340281.1613018383@sss.pgh.pa.us
M src/backend/regex/README
M src/backend/regex/regcomp.c
M src/backend/regex/regexec.c
M src/include/regex/regguts.h
Fix regex engine to suppress useless concatenation sub-REs.
commit : cebc1d34e5207c37871708f91be65dd839760b5f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Feb 2021 18:45:29 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Feb 2021 18:45:29 -0500
The comment for parsebranch() claims that it avoids generating
unnecessary concatenation nodes in the "subre" tree, but it missed
some significant cases. Once we've decided that a given atom is
"messy" and can't be bundled with the preceding atom(s) of the
current regex branch, parseqatom() always generated two new concat
nodes, one to concat the messy atom to what follows it in the branch,
and an upper node to concatenate the preceding part of the branch
to that one. But one or both of these could be unnecessary, if the
messy atom is the first, last, or only one in the branch. Improve
the code to suppress such useless concat nodes, along with the
no-op child nodes representing empty chunks of a branch.
Reducing the number of subre tree nodes offers significant savings
not only at execution but during compilation, because each subre node
has its own NFA that has to be separately optimized. (Maybe someday
we'll figure out how to share the optimization work across multiple
tree nodes, but it doesn't look easy.) Eliminating upper tree nodes
is especially useful because they tend to have larger NFAs.
This is part of a patch series that in total reduces the regex engine's
runtime by about a factor of four on a large corpus of real-world regexes.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/1340281.1613018383@sss.pgh.pa.us
M src/backend/regex/regcomp.c
Recognize "match-all" NFAs within the regex engine.
commit : 824bf71902db4a2067b8d64583c9d88bb264c44b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Feb 2021 18:31:19 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Feb 2021 18:31:19 -0500
This builds on the previous "rainbow" patch to detect NFAs that will
match any string, though possibly with constraints on the string length.
This definition is chosen to match constructs such as ".*", ".+", and
".{1,100}". Recognizing such an NFA after the optimization pass is
fairly cheap, since we basically just have to verify that all arcs
are RAINBOW arcs and count the number of steps to the end state.
(Well, there's a bit of complication with pseudo-color arcs for string
boundary conditions, but not much.)
Once we have these markings, the regex executor functions longest(),
shortest(), and matchuntil() don't have to expend per-character work
to determine whether a given substring satisfies such an NFA; they
just need to check its length against the bounds. Since some matching
problems require O(N) invocations of these functions, we've reduced
the runtime for an N-character string from O(N^2) to O(N). Of course,
this is no help for non-matchall sub-patterns, but those usually have
constraints that allow us to avoid needing O(N) substring checks in the
first place. It's precisely the unconstrained "match-all" cases that
cause the most headaches.
This is part of a patch series that in total reduces the regex engine's
runtime by about a factor of four on a large corpus of real-world regexes.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/1340281.1613018383@sss.pgh.pa.us
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/backend/regex/rege_dfa.c
M src/backend/regex/regprefix.c
M src/include/regex/regguts.h
Invent "rainbow" arcs within the regex engine.
commit : 08c0d6ad65f7c161add82ae906efb90dbd7f653d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Feb 2021 18:11:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Feb 2021 18:11:56 -0500
Some regular expression constructs, most notably the "." match-anything
metacharacter, produce a sheaf of parallel NFA arcs covering all
possible colors (that is, character equivalence classes). We can make
a noticeable improvement in the space and time needed to process large
regexes by replacing such cases with a single arc bearing the special
color code "RAINBOW". This requires only minor additional complication
in places such as pull() and push().
Callers of pg_reg_getoutarcs() must now be prepared for the possibility
of seeing a RAINBOW arc. For the one known user, contrib/pg_trgm,
that's a net benefit since it cuts the number of arcs to be dealt with,
and the handling isn't any different than for other colors that contain
too many characters to be dealt with individually.
This is part of a patch series that in total reduces the regex engine's
runtime by about a factor of four on a large corpus of real-world regexes.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/1340281.1613018383@sss.pgh.pa.us
M contrib/pg_trgm/trgm_regexp.c
M src/backend/regex/README
M src/backend/regex/regc_color.c
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/backend/regex/rege_dfa.c
M src/backend/regex/regexport.c
M src/backend/regex/regprefix.c
M src/include/regex/regexport.h
M src/include/regex/regguts.h
doc: Mention that partitions_{done,total} is 0 for REINDEX progress reports
commit : 17661188336c8cbb1783808912096932c57893a3
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 20 Feb 2021 10:25:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 20 Feb 2021 10:25:14 +0900
REINDEX has recently gained support for partitions, so it can be
confusing to see those fields not being set. Making useful reports for
for such relations is more complicated than it looks with the current
set of columns available in pg_stat_progress_create_index, and this
touches equally REINDEX DATABASE/SYSTEM/SCHEMA. This commit documents
that those two columns are not touched during a REINDEX.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210216064214.GI28165@telsasoft.com
M doc/src/sgml/monitoring.sgml
Fix inconsistent configure data for --with-ssl
commit : a899ec1cb23eb3c4a94648f18a9408e5316d32a4
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 20 Feb 2021 10:17:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 20 Feb 2021 10:17:10 +0900
This inconsistency was showing up after an autoreconf.
Reported-by: Antonin Houska
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/47255.1613716807@antos
M configure.ac
M src/include/pg_config.h.in
Fix psql's ON_ERROR_ROLLBACK so that it handles COMMIT AND CHAIN.
commit : fe06819f105ccea52c12d418c8dbaaaa54377e96
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 19 Feb 2021 22:01:25 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 19 Feb 2021 22:01:25 +0900
When ON_ERROR_ROLLBACK is enabled, psql releases a temporary savepoint
if it's idle in a valid transaction block after executing a query. But psql
doesn't do that after RELEASE or ROLLBACK is executed because a temporary
savepoint has already been destroyed in that case.
This commit changes psql's ON_ERROR_ROLLBACK so that it doesn't release
a temporary savepoint also when COMMIT AND CHAIN is executed. A temporary
savepoint doesn't need to be released in that case because
COMMIT AND CHAIN also destroys any savepoints defined within the transaction
to commit. Otherwise psql tries to release the savepoint that
COMMIT AND CHAIN has already destroyed and cause an error
"ERROR: savepoint "pg_psql_temporary_savepoint" does not exist".
Back-patch to v12 where transaction chaining was added.
Reported-by: Arthur Nascimento
Author: Arthur Nascimento
Reviewed-by: Fujii Masao, Vik Fearing
Discussion: https://postgr.es/m/16867-3475744069228158@postgresql.org
M src/bin/psql/common.c
Fix bug in COMMIT AND CHAIN command.
commit : 8a55cb5ba9655ffb1cf0a3042aaa6f5eef8c5a85
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 19 Feb 2021 21:57:52 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 19 Feb 2021 21:57:52 +0900
This commit fixes COMMIT AND CHAIN command so that it starts new transaction
immediately even if savepoints are defined within the transaction to commit.
Previously COMMIT AND CHAIN command did not in that case because
commit 280a408b48 forgot to make CommitTransactionCommand() handle
a transaction chaining when the transaction state was TBLOCK_SUBCOMMIT.
Also this commit adds the regression test for COMMIT AND CHAIN command
when savepoints are defined.
Back-patch to v12 where transaction chaining was added.
Reported-by: Arthur Nascimento
Author: Fujii Masao
Reviewed-by: Arthur Nascimento, Vik Fearing
Discussion: https://postgr.es/m/16867-3475744069228158@postgresql.org
M src/backend/access/transam/xact.c
M src/test/regress/expected/transactions.out
M src/test/regress/sql/transactions.sql
Update snowball
commit : 678d0e239b67174f349a401ea7dcecabb3c5b137
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 19 Feb 2021 07:57:42 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 19 Feb 2021 07:57:42 +0100
Update to snowball tag v2.1.0. Major changes are new stemmers for
Armenian, Serbian, and Yiddish.
M doc/src/sgml/textsearch.sgml
M src/backend/snowball/Makefile
M src/backend/snowball/README
M src/backend/snowball/dict_snowball.c
M src/backend/snowball/libstemmer/api.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_basque.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_catalan.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_danish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_dutch.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_english.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_finnish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_french.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_german.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_indonesian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_irish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_italian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_norwegian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_porter.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_portuguese.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_spanish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_swedish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_2_hungarian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_2_romanian.c
M src/backend/snowball/libstemmer/stem_KOI8_R_russian.c
M src/backend/snowball/libstemmer/stem_UTF_8_arabic.c
A src/backend/snowball/libstemmer/stem_UTF_8_armenian.c
M src/backend/snowball/libstemmer/stem_UTF_8_basque.c
M src/backend/snowball/libstemmer/stem_UTF_8_catalan.c
M src/backend/snowball/libstemmer/stem_UTF_8_danish.c
M src/backend/snowball/libstemmer/stem_UTF_8_dutch.c
M src/backend/snowball/libstemmer/stem_UTF_8_english.c
M src/backend/snowball/libstemmer/stem_UTF_8_finnish.c
M src/backend/snowball/libstemmer/stem_UTF_8_french.c
M src/backend/snowball/libstemmer/stem_UTF_8_german.c
M src/backend/snowball/libstemmer/stem_UTF_8_greek.c
M src/backend/snowball/libstemmer/stem_UTF_8_hindi.c
M src/backend/snowball/libstemmer/stem_UTF_8_hungarian.c
M src/backend/snowball/libstemmer/stem_UTF_8_indonesian.c
M src/backend/snowball/libstemmer/stem_UTF_8_irish.c
M src/backend/snowball/libstemmer/stem_UTF_8_italian.c
M src/backend/snowball/libstemmer/stem_UTF_8_lithuanian.c
M src/backend/snowball/libstemmer/stem_UTF_8_nepali.c
M src/backend/snowball/libstemmer/stem_UTF_8_norwegian.c
M src/backend/snowball/libstemmer/stem_UTF_8_porter.c
M src/backend/snowball/libstemmer/stem_UTF_8_portuguese.c
M src/backend/snowball/libstemmer/stem_UTF_8_romanian.c
M src/backend/snowball/libstemmer/stem_UTF_8_russian.c
A src/backend/snowball/libstemmer/stem_UTF_8_serbian.c
M src/backend/snowball/libstemmer/stem_UTF_8_spanish.c
M src/backend/snowball/libstemmer/stem_UTF_8_swedish.c
M src/backend/snowball/libstemmer/stem_UTF_8_tamil.c
M src/backend/snowball/libstemmer/stem_UTF_8_turkish.c
A src/backend/snowball/libstemmer/stem_UTF_8_yiddish.c
M src/backend/snowball/libstemmer/utilities.c
M src/bin/initdb/initdb.c
M src/include/catalog/catversion.h
M src/include/snowball/libstemmer/api.h
M src/include/snowball/libstemmer/header.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_basque.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_catalan.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_danish.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_dutch.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_english.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_finnish.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_french.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_german.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_indonesian.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_irish.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_italian.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_norwegian.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_porter.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_portuguese.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_spanish.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_swedish.h
M src/include/snowball/libstemmer/stem_ISO_8859_2_hungarian.h
M src/include/snowball/libstemmer/stem_ISO_8859_2_romanian.h
M src/include/snowball/libstemmer/stem_KOI8_R_russian.h
M src/include/snowball/libstemmer/stem_UTF_8_arabic.h
A src/include/snowball/libstemmer/stem_UTF_8_armenian.h
M src/include/snowball/libstemmer/stem_UTF_8_basque.h
M src/include/snowball/libstemmer/stem_UTF_8_catalan.h
M src/include/snowball/libstemmer/stem_UTF_8_danish.h
M src/include/snowball/libstemmer/stem_UTF_8_dutch.h
M src/include/snowball/libstemmer/stem_UTF_8_english.h
M src/include/snowball/libstemmer/stem_UTF_8_finnish.h
M src/include/snowball/libstemmer/stem_UTF_8_french.h
M src/include/snowball/libstemmer/stem_UTF_8_german.h
M src/include/snowball/libstemmer/stem_UTF_8_greek.h
M src/include/snowball/libstemmer/stem_UTF_8_hindi.h
M src/include/snowball/libstemmer/stem_UTF_8_hungarian.h
M src/include/snowball/libstemmer/stem_UTF_8_indonesian.h
M src/include/snowball/libstemmer/stem_UTF_8_irish.h
M src/include/snowball/libstemmer/stem_UTF_8_italian.h
M src/include/snowball/libstemmer/stem_UTF_8_lithuanian.h
M src/include/snowball/libstemmer/stem_UTF_8_nepali.h
M src/include/snowball/libstemmer/stem_UTF_8_norwegian.h
M src/include/snowball/libstemmer/stem_UTF_8_porter.h
M src/include/snowball/libstemmer/stem_UTF_8_portuguese.h
M src/include/snowball/libstemmer/stem_UTF_8_romanian.h
M src/include/snowball/libstemmer/stem_UTF_8_russian.h
A src/include/snowball/libstemmer/stem_UTF_8_serbian.h
M src/include/snowball/libstemmer/stem_UTF_8_spanish.h
M src/include/snowball/libstemmer/stem_UTF_8_swedish.h
M src/include/snowball/libstemmer/stem_UTF_8_tamil.h
M src/include/snowball/libstemmer/stem_UTF_8_turkish.h
A src/include/snowball/libstemmer/stem_UTF_8_yiddish.h
Add nbtree README section on page recycling.
commit : b071a311495bbf42ddf2466a556d033df8f0f5e7
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 18 Feb 2021 21:16:33 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 18 Feb 2021 21:16:33 -0800
Consolidate discussion of how VACUUM places pages in the FSM for
recycling by adding a new section that comes after discussion of page
deletion. This structure reflects the fact that page recycling is
explicitly decoupled from page deletion in Lanin & Shasha's paper. Page
recycling in nbtree is an implementation of what the paper calls "the
drain technique".
This decoupling is an important concept for nbtree VACUUM. Searchers
have to detect and recover from concurrent page deletions, but they will
never have to reason about concurrent page recycling. Recycling can
almost always be thought of as a low level garbage collection operation
that asynchronously frees the physical space that backs a logical tree
node. Almost all code need only concern itself with logical tree nodes.
(Note that "logical tree node" is not currently a term of art in the
nbtree code -- this all works implicitly.)
This is preparation for an upcoming patch that teaches nbtree VACUUM to
remember the details of pages that it deletes on the fly, in local
memory. This enables the same VACUUM operation to consider placing its
own deleted pages in the FSM later on, when it reaches the end of
btvacuumscan().
M src/backend/access/nbtree/README
Fix another ancient bug in parsing of BRE-mode regular expressions.
commit : b5a66e7353ba65c11c5fc6a79b72213bde8dbe44
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Feb 2021 22:38:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Feb 2021 22:38:55 -0500
While poking at the regex code, I happened to notice that the bug
squashed in commit afcc8772e had a sibling: next() failed to return
a specific value associated with the '}' token for a "\{m,n\}"
quantifier when parsing in basic RE mode. Again, this could result
in treating the quantifier as non-greedy, which it never should be in
basic mode. For that to happen, the last character before "\}" that
sets "nextvalue" would have to set it to zero, or it'd have to have
accidentally been zero from the start. The failure can be provoked
repeatably with, for example, a bound ending in digit "0".
Like the previous patch, back-patch all the way.
M src/backend/regex/regc_lex.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Fix "invalid spinlock number: 0" error in pg_stat_wal_receiver.
commit : 614b7f18b3bda738f352a8732cf749eb5fa56dae
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 18 Feb 2021 23:22:23 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 18 Feb 2021 23:22:23 +0900
Commit 2c8dd05d6c added the atomic variable writtenUpto into
walreceiver's shared memory information. It's initialized only
when walreceiver started up but could be read via pg_stat_wal_receiver
view anytime, i.e., even before it's initialized. In the server built
with --disable-atomics and --disable-spinlocks, this uninitialized
atomic variable read could cause "invalid spinlock number: 0" error.
This commit changed writtenUpto so that it's initialized at
the postmaster startup, to avoid the uninitialized variable read
via pg_stat_wal_receiver and fix the error.
Also this commit moved the read of writtenUpto after the release
of spinlock protecting walreceiver's shared variables. This is
necessary to prevent new spinlock from being taken by atomic
variable read while holding another spinlock, and to shorten
the spinlock duration. This change leads writtenUpto not to be
consistent with the other walreceiver's shared variables protected
by a spinlock. But this is OK because writtenUpto should not be
used for data integrity checks.
Back-patch to v13 where commit 2c8dd05d6c introduced the bug.
Author: Fujii Masao
Reviewed-by: Michael Paquier, Thomas Munro, Andres Freund
Discussion: https://postgr.es/m/7ef8708c-5b6b-edd3-2cf2-7783f1c7c175@oss.nttdata.com
M src/backend/replication/walreceiver.c
M src/backend/replication/walreceiverfuncs.c
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/sysviews.sql
Add tests for bytea LIKE operator
commit : eb42110d952f8d1ad4049b8f2491e9dfba75ffed
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 18 Feb 2021 08:27:05 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 18 Feb 2021 08:27:05 +0100
Add test coverage for the following operations, which were previously
not tested at all:
bytea LIKE bytea (bytealike)
bytea NOT LIKE bytea (byteanlike)
ESCAPE clause for the above (like_escape_bytea)
also
name NOT ILIKE text (nameicnlike)
Discussion: https://www.postgresql.org/message-id/flat/4d13563a-2c8d-fd91-20d5-e71b7a4eaa87%40enterprisedb.com
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
Allow specifying CRL directory
commit : f5465fade90827534fbd0b795d18dc62e56939e9
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 18 Feb 2021 07:59:10 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 18 Feb 2021 07:59:10 +0100
Add another method to specify CRLs, hashed directory method, for both
server and client side. This offers a means for server or libpq to
load only CRLs that are required to verify a certificate. The CRL
directory is specifed by separate GUC variables or connection options
ssl_crl_dir and sslcrldir, alongside the existing ssl_crl_file and
sslcrl, so both methods can be used at the same time.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/20200731.173911.904649928639357911.horikyota.ntt@gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M doc/src/sgml/config.sgml
M doc/src/sgml/libpq.sgml
M src/backend/libpq/be-secure-openssl.c
M src/backend/libpq/be-secure.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/libpq/libpq.h
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/libpq-int.h
M src/test/ssl/Makefile
A src/test/ssl/ssl/client-crldir/9bb9e3c3.r0
A src/test/ssl/ssl/root+client-crldir/9bb9e3c3.r0
A src/test/ssl/ssl/root+client-crldir/a3d11bff.r0
A src/test/ssl/ssl/root+server-crldir/a3d11bff.r0
A src/test/ssl/ssl/root+server-crldir/a836cc2d.r0
A src/test/ssl/ssl/server-crldir/a836cc2d.r0
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/SSLServer.pm
nbtree README: move VACUUM linear scan section.
commit : 128dd901a5c87e11c6a8cbe227a806cdc3afd10d
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 17 Feb 2021 21:13:15 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 17 Feb 2021 21:13:15 -0800
Discuss VACUUM's linear scan after discussion of tuple deletion by
VACUUM, but before discussion of page deletion by VACUUM. This
progression is a lot more natural.
Also tweak the wording a little. It seems unnecessary to talk about how
it worked prior to PostgreSQL 8.2.
M src/backend/access/nbtree/README
Fix tuple routing to initialize batching only for inserts
commit : 927f453a941061e3d5884bab206581c34784e45b
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 18 Feb 2021 00:02:00 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 18 Feb 2021 00:02:00 +0100
A cross-partition update on a partitioned table is implemented as a
delete followed by an insert. With foreign partitions, this was however
causing issues, because the FDW and core may disagree on when to enable
batching. postgres_fdw was only allowing batching for plain inserts
(CMD_INSERT) while core was trying to batch the insert component of the
cross-partition update. Fix by restricting core to apply batching only
to plain CMD_INSERT queries.
It's possible to allow batching for cross-partition updates, but that
will require more extensive changes, so better to leave that for a
separate patch.
Author: Amit Langote
Reviewed-by: Tomas Vondra, Takayuki Tsunakawa
Discussion: https://postgr.es/m/20200628151002.7x5laxwpgvkyiu3q@development
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/executor/execPartition.c
Fix pointer type in ExecForeignBatchInsert SGML docs
commit : c15283ff429bf318f161bf84768795843b22696d
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 18 Feb 2021 00:01:21 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 18 Feb 2021 00:01:21 +0100
Reported-by: Ian Barwick
Discussion: https://postgr.es/m/20200628151002.7x5laxwpgvkyiu3q@development
M doc/src/sgml/fdwhandler.sgml
Make some minor improvements in the regex code.
commit : 4e703d67193df0431c0740044d662d1feade73aa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Feb 2021 12:24:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Feb 2021 12:24:12 -0500
Push some hopefully-uncontroversial bits extracted from an upcoming
patch series, to remove non-relevant clutter from the main patches.
In compact(), return immediately after setting REG_ASSERT error;
continuing the loop would just lead to assertion failure below.
(Ask me how I know.)
In parseqatom(), remove assertion that moresubs() did its job.
When moresubs actually did its job, this is redundant with that
function's final assert; but when it failed on OOM, this is an
assertion crash. We could avoid the crash by adding a NOERR()
check before the assertion, but it seems better to subtract code
than add it. (Note that there's a NOERR exit a few lines further
down, and nothing else between here and there requires moresubs
to have succeeded. So we don't really need an extra error exit.)
This is a live bug in assert-enabled builds, but given the very
low likelihood of OOM in moresub's tiny allocation, I don't think
it's worth back-patching.
On the other hand, it seems worthwhile to add an assertion that
our intended v->subs[subno] target is still null by the time we
are ready to insert into it, since there's a recursion in between.
In pg_regexec, ensure we fflush any debug output on the way out,
and try to make MDEBUG messages more uniform and helpful. (In
particular, ensure that all of them are prefixed with the subre's
id number, so one can match up entry and exit reports.)
Add some test cases in test_regex to improve coverage of lookahead
and lookbehind constraints. Adding these now is mainly to establish
that this is indeed the existing behavior.
Discussion: https://postgr.es/m/1340281.1613018383@sss.pgh.pa.us
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/backend/regex/regexec.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Routine usage information schema tables
commit : f40c6969d0eddfc6de786006bd1048961a65a0eb
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 17 Feb 2021 17:53:18 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 17 Feb 2021 17:53:18 +0100
Several information schema views track dependencies between
functions/procedures and objects used by them. These had not been
implemented so far because PostgreSQL doesn't track objects used in a
function body. However, formally, these also show dependencies used
in parameter default expressions, which PostgreSQL does support and
track. So for the sake of completeness, we might as well add these.
If dependency tracking for function bodies is ever implemented, these
views will automatically work correctly.
Reviewed-by: Erik Rijkers <er@xs4all.nl>
Discussion: https://www.postgresql.org/message-id/flat/ac80fc74-e387-8950-9a31-2560778fc1e3%40enterprisedb.com
M doc/src/sgml/information_schema.sgml
M src/backend/catalog/information_schema.sql
M src/backend/catalog/sql_features.txt
M src/include/catalog/catversion.h
M src/test/regress/expected/create_function_3.out
M src/test/regress/sql/create_function_3.sql
Fix typo
commit : a29f30780f34d7706fcd398dea1d6882d184d17a
author : Magnus Hagander <magnus@hagander.net>
date : Wed, 17 Feb 2021 13:53:26 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Wed, 17 Feb 2021 13:53:26 +0100
Author: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/0CF087FC-BEAD-4010-8BB9-3CDD74DC9060@yesql.se
M doc/src/sgml/ref/create_table.sgml
Use errmsg_internal for debug messages
commit : 0e392fcc0d5ab73c81d9284e7dda5acbb7ad6d21
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 17 Feb 2021 11:24:46 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 17 Feb 2021 11:24:46 +0100
An inconsistent set of debug-level messages was not using
errmsg_internal(), thus uselessly exposing the messages to translation
work. Fix those.
M contrib/amcheck/verify_nbtree.c
M contrib/pg_prewarm/autoprewarm.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/varsup.c
M src/backend/access/transam/xlog.c
M src/backend/catalog/dependency.c
M src/backend/catalog/index.c
M src/backend/commands/indexcmds.c
M src/backend/commands/subscriptioncmds.c
M src/backend/commands/tablecmds.c
M src/backend/jit/llvm/llvmjit.c
M src/backend/libpq/be-secure.c
M src/backend/parser/parse_utilcmd.c
M src/backend/partitioning/partbounds.c
M src/backend/port/win32_shmem.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/postmaster.c
M src/backend/postmaster/syslogger.c
M src/backend/replication/logical/launcher.c
M src/backend/replication/syncrep.c
M src/backend/replication/walsender.c
M src/backend/storage/lmgr/predicate.c
M src/backend/storage/lmgr/proc.c
M src/backend/storage/smgr/md.c
M src/backend/storage/sync/sync.c
M src/backend/tcop/postgres.c
M src/backend/utils/init/miscinit.c
Add psql completion for [ NO ] DEPENDS ON EXTENSION
commit : e6b8e83b9f012cfbb3a2f96a7d2e74aebd299d4e
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 17 Feb 2021 11:50:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 17 Feb 2021 11:50:58 +0900
ALTER INDEX was able to handle that already. This adds tab completion
for all the remaining commands that support this grammar:
- ALTER FUNCTION
- ALTER PROCEDURE
- ALTER ROUTINE
- ALTER TRIGGER
- ALTER MATERIALIZED VIEW
Author: Ian Lawrence Barwick
Discussion: https://postgr.es/m/CAB8KJ=iypYudXuMOAMOP4BpkaYbXxk=a2cdJppX0e9mJXWtuig@mail.gmail.com
M src/bin/psql/tab-complete.c
Convert tsginidx.c's GIN indexing logic to fully ternary operation.
commit : 38bb3aef354ca98ff88cb37337995039a3b5135f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Feb 2021 12:07:14 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Feb 2021 12:07:14 -0500
Commit 2f2007fbb did this partially, but there were two remaining
warts. checkcondition_gin handled some uncertain cases by setting
the out-of-band recheck flag, some by returning TS_MAYBE, and some
by doing both. Meanwhile, TS_execute arbitrarily converted a
TS_MAYBE result to TS_YES. Thus, if checkcondition_gin chose to
only return TS_MAYBE, the outcome would be TS_YES with no recheck
flag, potentially resulting in wrong query outputs.
The case where this'd happen is if there were GIN_MAYBE entries
in the indexscan results passed to gin_tsquery_[tri]consistent,
which so far as I can see would only happen if the tidbitmap used
to accumulate indexscan results grew large enough to become lossy.
I initially thought of fixing this by ensuring we always set the
recheck flag as well as returning TS_MAYBE in uncertain cases.
But that errs in the other direction, potentially forcing rechecks
of rows that provably match the query (since the recheck flag
remains set even if TS_execute later finds that the answer must be
TS_YES). Instead, let's get rid of the out-of-band recheck flag
altogether and rely on returning TS_MAYBE. This requires exporting
a version of TS_execute that will actually return the full ternary
result of the evaluation ... but we likely should have done that
to start with.
Unfortunately it doesn't seem practical to add a regression test case
that covers this: the amount of data needed to cause the GIN bitmap to
become lossy results in a longer runtime than I think we want to have
in the tests. (I'm wondering about allowing smaller work_mem settings
to ameliorate that, but it'd be a matter for a separate patch.)
Per bug #16865 from Dimitri Nüscheler. Back-patch to v13 where
the faulty commit came in.
Discussion: https://postgr.es/m/16865-4ffdc3e682e6d75b@postgresql.org
M src/backend/utils/adt/tsginidx.c
M src/backend/utils/adt/tsvector_op.c
M src/include/tsearch/ts_utils.h
Remove the unnecessary PrepareWrite in pgoutput.
commit : f672df5fdd22dac14c98d0a0bf5bbaa6ab17f8a5
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 16 Feb 2021 07:26:50 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 16 Feb 2021 07:26:50 +0530
This issue exists from the inception of this code (PG-10) but got exposed
by the recent commit ce0fdbfe97 where we are using origins in tablesync
workers. The problem was that we were sometimes sending the prepare_write
('w') message but then the actual message was not being sent and on the
subscriber side, we always expect a message after prepare_write message
which led to this bug.
I refrained from backpatching this because there is no way in the core
code to hit this prior to commit ce0fdbfe97 and we haven't received any
complaints so far.
Reported-by: Erik Rijkers
Author: Amit Kapila and Vignesh C
Tested-by: Erik Rijkers
Discussion: https://postgr.es/m/1295168140.139428.1613133237154@webmailclassic.xs4all.nl
M src/backend/replication/pgoutput/pgoutput.c
M src/test/subscription/t/100_bugs.pl
Fix heap_page_prune() parameter order confusion introduced in dc7420c2c92.
commit : 8001cb77ee6cb4f32632850d41f00206a86bac3e
author : Andres Freund <andres@anarazel.de>
date : Mon, 15 Feb 2021 17:12:12 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 15 Feb 2021 17:12:12 -0800
Both luckily and unluckily the passed values meant the same for all
types. Luckily because that meant my confusion caused no harm,
unluckily because otherwise the compiler might have warned...
In passing, synchronize parameter names between definition and
declaration.
Reported-By: Peter Geoghegan <pg@bowt.ie>
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CAH2-Wz=L=nBoepQdH9b5Qd0nMvepFT2CnT6sjWvvpOXa=K8HVQ@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
M src/include/access/heapam.h
Remove backwards compat ugliness in snapbuild.c.
commit : a975ff4980d60f8cbd8d8cbcff70182ea53e787a
author : Andres Freund <andres@anarazel.de>
date : Mon, 15 Feb 2021 16:57:47 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 15 Feb 2021 16:57:47 -0800
In 955a684e040 we fixed a bug in initial snapshot creation. In the
course of which several members of struct SnapBuild were obsoleted. As
SnapBuild is serialized to disk we couldn't change the memory layout.
Unfortunately I subsequently forgot about removing the backward compat
gunk, but luckily Heikki just reminded me.
This commit bumps SNAPBUILD_VERSION, therefore breaking existing
slots (which is fine in a major release).
Author: Andres Freund
Reminded-By: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/c94be044-818f-15e3-1ad3-7a7ae2dfed0a@iki.fi
M src/backend/replication/logical/snapbuild.c
Simplify loop logic in nodeIncrementalSort.c.
commit : 0e5290312851557ee24e3d6103baf14d6066695c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 15 Feb 2021 10:17:58 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 15 Feb 2021 10:17:58 -0500
The inner loop in switchToPresortedPrefixMode() can be implemented
as a conventional integer-counter for() loop, removing a couple of
redundant boolean state variables. The old logic here was a remnant
of earlier development, but as things now stand there's no reason
for extra complexity.
Also, annotate the test case added by 82e0e2930 to explain why it
manages to hit the corner case fixed in that commit, and add an
EXPLAIN to verify that it's creating an incremental-sort plan.
Back-patch to v13, like the previous patch.
James Coleman and Tom Lane
Discussion: https://postgr.es/m/16846-ae49f51ac379a4cb@postgresql.org
M src/backend/executor/nodeIncrementalSort.c
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Make ExecGetInsertedCols() and friends more robust and improve comments.
commit : 54e51dcde03e5c746e8de6243c69fafdc8d0ec7a
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 15 Feb 2021 09:28:08 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 15 Feb 2021 09:28:08 +0200
If ExecGetInsertedCols(), ExecGetUpdatedCols() or ExecGetExtraUpdatedCols()
were called with a ResultRelInfo that's not in the range table and isn't a
partition routing target, the functions would dereference a NULL pointer,
relinfo->ri_RootResultRelInfo. Such ResultRelInfos are created when firing
RI triggers in tables that are not modified directly. None of the current
callers of these functions pass such relations, so this isn't a live bug,
but let's make them more robust.
Also update comment in ResultRelInfo; after commit 6214e2b228,
ri_RangeTableIndex is zero for ResultRelInfos created for partition tuple
routing.
Noted by Coverity. Backpatch down to v11, like commit 6214e2b228.
Reviewed-by: Tom Lane, Amit Langote
M src/backend/executor/execUtils.c
M src/include/nodes/execnodes.h
Display the time when the process started waiting for the lock, in pg_locks, take 2
commit : 46d6e5f567906389c31c4fb3a2653da1885c18ee
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 15 Feb 2021 15:13:37 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 15 Feb 2021 15:13:37 +0900
This commit adds new column "waitstart" into pg_locks view. This column
reports the time when the server process started waiting for the lock
if the lock is not held. This information is useful, for example, when
examining the amount of time to wait on a lock by subtracting
"waitstart" in pg_locks from the current time, and identify the lock
that the processes are waiting for very long.
This feature uses the current time obtained for the deadlock timeout
timer as "waitstart" (i.e., the time when this process started waiting
for the lock). Since getting the current time newly can cause overhead,
we reuse the already-obtained time to avoid that overhead.
Note that "waitstart" is updated without holding the lock table's
partition lock, to avoid the overhead by additional lock acquisition.
This can cause "waitstart" in pg_locks to become NULL for a very short
period of time after the wait started even though "granted" is false.
This is OK in practice because we can assume that users are likely to
look at "waitstart" when waiting for the lock for a long time.
The first attempt of this patch (commit 3b733fcd04) caused the buildfarm
member "rorqual" (built with --disable-atomics --disable-spinlocks) to report
the failure of the regression test. It was reverted by commit 890d2182a2.
The cause of this failure was that the atomic variable for "waitstart"
in the dummy process entry created at the end of prepare transaction was
not initialized. This second attempt fixes that issue.
Bump catalog version.
Author: Atsushi Torikoshi
Reviewed-by: Ian Lawrence Barwick, Robert Haas, Justin Pryzby, Fujii Masao
Discussion: https://postgr.es/m/a96013dc51cdc56b2a2b84fa8a16a993@oss.nttdata.com
M contrib/amcheck/expected/check_btree.out
M doc/src/sgml/catalogs.sgml
M src/backend/access/transam/twophase.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/adt/lockfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/storage/lock.h
M src/include/storage/proc.h
M src/test/regress/expected/rules.out
Add "LP_DEAD item?" column to GiST pageinspect functions
commit : 9e596b65f430fcb942685b41860b323398a88873
author : Peter Geoghegan <pg@bowt.ie>
date : Sun, 14 Feb 2021 20:11:11 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Sun, 14 Feb 2021 20:11:11 -0800
This brings gist_page_items() and gist_page_items_bytea() in line with
nbtree's bt_page_items() function.
Minor follow-up to commit 756ab291, which added the GiST functions.
Author: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/E0794687-7315-4C29-A9C7-EC54D448596D@yandex-team.ru
M contrib/pageinspect/expected/gist.out
M contrib/pageinspect/gistfuncs.c
M contrib/pageinspect/pageinspect–1.8–1.9.sql
M doc/src/sgml/pageinspect.sgml
Avoid misinterpreting GiST pages in pageinspect.
commit : fa41cf8f183ac5d702e91da567e9b3375c632081
author : Peter Geoghegan <pg@bowt.ie>
date : Sun, 14 Feb 2021 19:43:25 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Sun, 14 Feb 2021 19:43:25 -0800
GistPageSetDeleted() sets pd_lower when deleting a page, and sets the
page contents to a GISTDeletedPageContents. Avoid treating deleted GiST
pages as regular slotted pages within pageinspect.
Oversight in commit 756ab291.
Author: Andrey Borodin <x4mmm@yandex-team.ru>
M contrib/pageinspect/gistfuncs.c
Adjust lazy_scan_heap() accounting comments.
commit : 7cde6b13a9b630e2f04d91e2f17dedc2afee21c6
author : Peter Geoghegan <pg@bowt.ie>
date : Sun, 14 Feb 2021 19:28:37 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Sun, 14 Feb 2021 19:28:37 -0800
Explain which particular LP_DEAD line pointers get accounted for by the
tups_vacuumed variable.
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/vacuumlazy.c
Default to wal_sync_method=fdatasync on FreeBSD.
commit : f900a79ecdc1864a6ead72c97c34a41012227eaf
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Feb 2021 15:43:39 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Feb 2021 15:43:39 +1300
FreeBSD 13 gained O_DSYNC, which would normally cause wal_sync_method to
choose open_datasync as its default value. That may not be a good
choice for all systems, and performs worse than fdatasync in some
scenarios. Let's preserve the existing default behavior for now.
Like commit 576477e73c4, which did the same for Linux, back-patch to all
supported releases.
Discussion: https://postgr.es/m/CA%2BhUKGLsAMXBQrCxCXoW-JsUYmdOL8ALYvaX%3DCrHqWxm-nWbGA%40mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/utils/misc/postgresql.conf.sample
M src/include/port/freebsd.h
Use pg_pwrite() in pg_test_fsync.
commit : 2c8b42b50df6cc5ba6987c400a857d6bbfdf0300
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Feb 2021 15:08:04 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Feb 2021 15:08:04 +1300
For consistency with the PostgreSQL behavior this test program is
intended to simulate, use pwrite() instead of lseek() + write().
Also fix the final "non-sync" test, which was opening and closing the
file for every write.
Discussion: https://postgr.es/m/CA%2BhUKGJjjid2BJsvjMALBTduo1ogdx2SPYaTQL3wAy8y2hc4nw%40mail.gmail.com
M src/bin/pg_test_fsync/pg_test_fsync.c
Fix the warnings introduced in commit ce0fdbfe97.
commit : d9b0767becf5f41e4f001d8381e6a89941efa5b2
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 15 Feb 2021 07:28:02 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 15 Feb 2021 07:28:02 +0530
Author: Amit Kapila
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/1610789.1613170207@sss.pgh.pa.us
M src/backend/commands/subscriptioncmds.c
M src/backend/replication/logical/tablesync.c
M src/include/replication/slot.h
M src/include/replication/worker_internal.h
Hold interrupts while running dsm_detach() callbacks.
commit : 637668fb1d17ad789e392a40ff09694ff1aabffb
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Feb 2021 13:32:58 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Feb 2021 13:32:58 +1300
While cleaning up after a parallel query or parallel index creation that
created temporary files, we could be interrupted by a statement timeout.
The error handling path would then fail to clean up the files when it
ran dsm_detach() again, because the callback was already popped off the
list. Prevent this hazard by holding interrupts while the cleanup code
runs.
Thanks to Heikki Linnakangas for this suggestion, and also to Kyotaro
Horiguchi, Masahiko Sawada, Justin Pryzby and Tom Lane for discussion of
this and earlier ideas on how to fix the problem.
Back-patch to all supported releases.
Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/20191212180506.GR2082@telsasoft.com
M src/backend/storage/ipc/dsm.c
Add result size as argument of pg_cryptohash_final() for overflow checks
commit : b83dcf792869fb4a9270d17c961eab75f51c44e4
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 15 Feb 2021 10:18:34 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 15 Feb 2021 10:18:34 +0900
With its current design, a careless use of pg_cryptohash_final() could
would result in an out-of-bound write in memory as the size of the
destination buffer to store the result digest is not known to the
cryptohash internals, without the caller knowing about that. This
commit adds a new argument to pg_cryptohash_final() to allow such sanity
checks, and implements such defenses.
The internals of SCRAM for HMAC could be tightened a bit more, but as
everything is based on SCRAM_KEY_LEN with uses particular to this code
there is no need to complicate its interface more than necessary, and
this comes back to the refactoring of HMAC in core. Except that, this
minimizes the uses of the existing DIGEST_LENGTH variables, relying
instead on sizeof() for the result sizes. In ossp-uuid, this also makes
the code more defensive, as it already relied on dce_uuid_t being at
least the size of a MD5 digest.
This is in philosophy similar to cfc40d3 for base64.c and aef8948 for
hex.c.
Reported-by: Ranier Vilela
Author: Michael Paquier, Ranier Vilela
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAEudQAoqEGmcff3J4sTSV-R_16Monuz-UpJFbf_dnVH=APr02Q@mail.gmail.com
M contrib/pgcrypto/internal-sha2.c
M contrib/pgcrypto/internal.c
M contrib/uuid-ossp/uuid-ossp.c
M src/backend/libpq/auth-scram.c
M src/backend/replication/backup_manifest.c
M src/backend/utils/adt/cryptohashfuncs.c
M src/bin/pg_verifybackup/parse_manifest.c
M src/common/checksum_helper.c
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
M src/common/md5_common.c
M src/common/scram-common.c
M src/include/common/cryptohash.h
Minor fixes to improve regex debugging code.
commit : 2dd6733108f2bea07b0a3469e768bd900c0808b3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 14 Feb 2021 19:53:28 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 14 Feb 2021 19:53:28 -0500
When REG_DEBUG is defined, ensure that an un-filled "struct cnfa"
is all-zeroes, not just that it has nstates == 0. This is mainly
so that looking at "struct subre" structs in gdb doesn't distract
one with a lot of garbage fields during regex compilation.
Adjust some places that print debug output to have suitable fflush
calls afterwards.
In passing, correct an erroneous ancient comment: the concatenation
subre-s created by parsebranch() have op == '.' not ','.
Noted while fooling around with some regex performance improvements.
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/include/regex/regguts.h
ReadNewTransactionId() -> ReadNextTransactionId().
commit : c7ecd6af010e2ac8c5530f3985e97f24531bfa5f
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Feb 2021 13:03:10 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Feb 2021 13:03:10 +1300
The new name conveys the effect better, is more consistent with similar
functions ReadNextMultiXactId(), ReadNextFullTransactionId(), and
matches the name of the variable that it reads.
Reported-by: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WzmVR4SakBXQUdhhPpMf1aYvZCnna5%3DHKa7DAgEmBAg%2B8g%40mail.gmail.com
M src/backend/access/gin/ginvacuum.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/xact.c
M src/backend/commands/vacuum.c
M src/backend/postmaster/autovacuum.c
M src/include/access/transam.h
README/C-comment: document GiST's NSN value
commit : 8facf1ea00b7a0c08c755a0392212b83e04ae28a
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 13 Feb 2021 13:50:49 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 13 Feb 2021 13:50:49 -0500
M src/backend/access/gist/README
M src/include/access/gist.h
doc: Mention NO DEPENDS ON EXTENSION in its supported ALTER commands
commit : 8063d0f6f56e53edd991f53aadc8cb7f8d3fdd8f
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 13 Feb 2021 16:06:11 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 13 Feb 2021 16:06:11 +0900
This grammar flavor has been added by 5fc7039.
Author: Ian Lawrence Barwick
Discussion: https://postgr.es/m/CAB8KJ=ii6JScodxkA6-DO8bjatsMYU3OcewnL0mdN9geR+tTaw@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/ref/alter_index.sgml
M doc/src/sgml/ref/alter_materialized_view.sgml
M doc/src/sgml/ref/alter_procedure.sgml
M doc/src/sgml/ref/alter_routine.sgml
Tweak compiler version cutoff for no_sanitize("alignment") support.
commit : ad2ad698ac16f36a9784bc98f312cf5b5266c6aa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Feb 2021 17:33:14 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Feb 2021 17:33:14 -0500
Buildfarm results show that gcc up through 7.x produces annoying
warnings for this construct (and, presumably, wouldn't do the right
thing anyway). clang seems okay with the cutoff we have, though.
Discussion: https://postgr.es/m/CAPpHfdsne3%3DT%3DfMNU45PtxdhSL_J2PjLTeS8rwKnJzUR4YNd4w%40mail.gmail.com
Discussion: https://postgr.es/m/475514.1612745257%40sss.pgh.pa.us
M src/include/c.h
Avoid divide-by-zero in regex_selectivity() with long fixed prefix.
commit : ae4867ec749671dadb5eb49a1158ded9b8ca0b29
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Feb 2021 16:26:47 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Feb 2021 16:26:47 -0500
Given a regex pattern with a very long fixed prefix (approaching 500
characters), the result of pow(FIXED_CHAR_SEL, fixed_prefix_len) can
underflow to zero. Typically the preceding selectivity calculation
would have underflowed as well, so that we compute 0/0 and get NaN.
In released branches this leads to an assertion failure later on.
That doesn't happen in HEAD, for reasons I've not explored yet,
but it's surely still a bug.
To fix, just skip the division when the pow() result is zero, so
that we'll (most likely) return a zero selectivity estimate. In
the edge cases where "sel" didn't yet underflow, perhaps this
isn't desirable, but I'm not sure that the case is worth spending
a lot of effort on. The results of regex_selectivity_sub() are
barely worth the electrons they're written on anyway :-(
Per report from Alexander Lakhin. Back-patch to all supported versions.
Discussion: https://postgr.es/m/6de0a0c3-ada9-cd0c-3e4e-2fa9964b41e3@gmail.com
M src/backend/utils/adt/like_support.c
pg_attribute_no_sanitize_alignment() macro
commit : 993bdb9f935a751935a03c80d30857150ba2b645
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 12 Feb 2021 17:14:33 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 12 Feb 2021 17:14:33 +0300
Modern gcc and clang compilers offer alignment sanitizers, which help to detect
pointer misalignment. However, our codebase already contains x86-specific
crc32 computation code, which uses unalignment access. Thankfully, those
compilers also support the attribute, which disables alignment sanitizers at
the function level. This commit adds pg_attribute_no_sanitize_alignment(),
which wraps this attribute, and applies it to pg_comp_crc32c_sse42() function.
Discussion: https://postgr.es/m/CAPpHfdsne3%3DT%3DfMNU45PtxdhSL_J2PjLTeS8rwKnJzUR4YNd4w%40mail.gmail.com
Discussion: https://postgr.es/m/475514.1612745257%40sss.pgh.pa.us
Author: Alexander Korotkov, revised by Tom Lane
Reviewed-by: Tom Lane
M src/include/c.h
M src/port/pg_crc32c_sse42.c
Fix Subscription test added by commit ce0fdbfe97.
commit : c8b21b0378141948a2dc0ddde6f57407321787ea
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 12 Feb 2021 10:11:16 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 12 Feb 2021 10:11:16 +0530
We want to test the variants of Alter Subscription that are not allowed in
the transaction block but for that, we don't need to create a subscription
that tries to connect to the publisher. As such, there is no problem with
this test but it is good to allow such tests to run with
wal_level = minimal and max_wal_senders = 0 so as to keep them consistent
with other tests.
Reported by buildfarm.
Author: Amit Kapila
Reviewed-by: Ajin Cherian
Discussion: https://postgr.es/m/CAA4eK1Lw0V+e1JPGHDq=+hVACv=14H8sR+2eJ1k3PEgwKmU-jQ@mail.gmail.com
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
Allow multiple xacts during table sync in logical replication.
commit : ce0fdbfe9722867b7fad4d3ede9b6a6bfc51fb4e
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 12 Feb 2021 07:41:51 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 12 Feb 2021 07:41:51 +0530
For the initial table data synchronization in logical replication, we use
a single transaction to copy the entire table and then synchronize the
position in the stream with the main apply worker.
There are multiple downsides of this approach: (a) We have to perform the
entire copy operation again if there is any error (network breakdown,
error in the database operation, etc.) while we synchronize the WAL
position between tablesync worker and apply worker; this will be onerous
especially for large copies, (b) Using a single transaction in the
synchronization-phase (where we can receive WAL from multiple
transactions) will have the risk of exceeding the CID limit, (c) The slot
will hold the WAL till the entire sync is complete because we never commit
till the end.
This patch solves all the above downsides by allowing multiple
transactions during the tablesync phase. The initial copy is done in a
single transaction and after that, we commit each transaction as we
receive. To allow recovery after any error or crash, we use a permanent
slot and origin to track the progress. The slot and origin will be removed
once we finish the synchronization of the table. We also remove slot and
origin of tablesync workers if the user performs DROP SUBSCRIPTION .. or
ALTER SUBSCRIPTION .. REFERESH and some of the table syncs are still not
finished.
The commands ALTER SUBSCRIPTION ... REFRESH PUBLICATION and
ALTER SUBSCRIPTION ... SET PUBLICATION ... with refresh option as true
cannot be executed inside a transaction block because they can now drop
the slots for which we have no provision to rollback.
This will also open up the path for logical replication of 2PC
transactions on the subscriber side. Previously, we can't do that because
of the requirement of maintaining a single transaction in tablesync
workers.
Bump catalog version due to change of state in the catalog
(pg_subscription_rel).
Author: Peter Smith, Amit Kapila, and Takamichi Osumi
Reviewed-by: Ajin Cherian, Petr Jelinek, Hou Zhijie and Amit Kapila
Discussion: https://postgr.es/m/CAA4eK1KHJxaZS-fod-0fey=0tq3=Gkn4ho=8N4-5HWiCfu0H1A@mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/ref/alter_subscription.sgml
M doc/src/sgml/ref/drop_subscription.sgml
M src/backend/access/transam/xact.c
M src/backend/catalog/pg_subscription.c
M src/backend/commands/subscriptioncmds.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/launcher.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/backend/tcop/utility.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_subscription_rel.h
M src/include/commands/subscriptioncmds.h
M src/include/replication/logicallauncher.h
M src/include/replication/slot.h
M src/include/replication/walreceiver.h
M src/include/replication/worker_internal.h
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
M src/test/subscription/t/004_sync.pl
M src/tools/pgindent/typedefs.list
Remove obsolete IndexBulkDeleteResult stats field.
commit : 3063eb17593c3ad498ce4e89db3358862ea2dbb6
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 11 Feb 2021 16:49:41 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 11 Feb 2021 16:49:41 -0800
The pages_removed field is no longer used for anything. It hasn't been
possible for an index to physically shrink since old-style VACUUM FULL
was removed by commit 0a469c87.
M src/backend/access/gist/gistvacuum.c
M src/include/access/genam.h
Remove dead code in ECPGconnect(), and improve documentation.
commit : 62535cae9723afc48173ba1be65f1c7491813fc2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 11 Feb 2021 15:05:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 11 Feb 2021 15:05:55 -0500
The stanza in ECPGconnect() that intended to allow specification of a
Unix socket directory path in place of a port has never executed since
it was committed, nearly two decades ago; the preceding strrchr()
already found the last colon so there cannot be another one. The lack
of complaints about that is doubtless related to the fact that no
user-facing documentation suggested it was possible.
Rather than try to fix that up, let's just remove the unreachable
code, and instead document the way that does work to write a socket
directory path, namely specifying it as a "host" option.
In support of that, make another pass at clarifying the syntax
documentation for ECPG connection targets, particularly documenting
which things are parsed as identifiers and where to use double quotes.
Rearrange some things that seemed poorly ordered, and fix a couple of
minor doc errors.
Kyotaro Horiguchi, per gripe from Shenhao Wang
(docs changes mostly by me)
Discussion: https://postgr.es/m/ae52a416bbbf459c96bab30b3038e06c@G08CNEXMBPEKD06.g08.fujitsu.local
M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/ecpglib/connect.c
Simplify jsonfuncs.c code by using strtoint() not strtol().
commit : 69036aafb9a8f425fb489125b5075ba7719d20d0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 11 Feb 2021 12:49:22 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 11 Feb 2021 12:49:22 -0500
Explicitly testing for INT_MIN and INT_MAX isn't particularly good
style; it's tedious and may draw useless compiler warnings on
machines where int and long are the same width. We invented
strtoint() precisely for this usage, so use that instead.
While here, remove gratuitous variations in the way the tests for
did-strtoint-succeed were spelled. Also, avoid attempting to
negate INT_MIN; that would probably work given that the result
is implicitly cast to uint32, but I think it's nominally undefined
behavior.
Per gripe from Ranier Vilela, though this isn't his proposed patch.
Discussion: https://postgr.es/m/CAEudQAqge3QfzoBRhe59QrB_5g+NmQUj2QpzqZ9Nc7QepXGAEw@mail.gmail.com
M src/backend/utils/adt/jsonfuncs.c
Remove no-longer-used RTE argument of markVarForSelectPriv().
commit : d4c746516b8b4eb0bf993e3729ccc04d1febdb1e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 11 Feb 2021 11:23:25 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 11 Feb 2021 11:23:25 -0500
In the wake of c028faf2a, this is no longer needed. I left it
out of that patch since the API change would be undesirable in
a released branch; but there's no reason not to do it in HEAD.
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_target.c
M src/include/parser/parse_relation.h
Fix copy-paste error with SHA256 digest length in checksum_helper.c
commit : 42d74e0c443760af91875af99d2655c9eb4c40df
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 11 Feb 2021 19:16:11 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 11 Feb 2021 19:16:11 +0900
Issue introduced by 87ae969, noticed while working on the area. While
on it, fix some grammar in the surrounding static assertions.
M src/common/checksum_helper.c
Add test case for abbrev(cidr)
commit : 69273c8f88d7e6295ed7b38114969fa3ea03eff2
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 11 Feb 2021 09:56:14 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 11 Feb 2021 09:56:14 +0100
This will in particular add some good test coverage for
inet_cidr_ntop.c, which was previously completely uncovered.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/cb0c4662-4596-dab4-7f64-839c5e8582c8%40enterprisedb.com
M src/test/regress/expected/inet.out
M src/test/regress/sql/inet.sql
Remove extra Success message at the end of initdb
commit : e7f42914854926c2afbb89b9cd0e381fd90766be
author : Magnus Hagander <magnus@hagander.net>
date : Wed, 10 Feb 2021 18:21:55 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Wed, 10 Feb 2021 18:21:55 +0100
This was accidentally included in e09155bd62 and is redundant with the
lines right above it.
Reported-By: Peter Eisentraut
Discussion: https://postgr.es/m/455845d1-441d-cc40-d2a7-b47f4e422489@2ndquadrant.com
M src/bin/initdb/initdb.c
pg_dump: Add const decorations
commit : 64990081504661ff5c04dbf20cc4252be66ab149
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 10 Feb 2021 13:08:13 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 10 Feb 2021 13:08:13 +0100
Add const decorations to the *info arguments of the dump* functions,
to clarify that they don't modify that argument. Many other nearby
functions modify their arguments, so this can help clarify these
different APIs a bit.
Discussion: https://www.postgresql.org/message-id/flat/012d3030-9a2c-99a1-ed2d-988978b5632f%40enterprisedb.com
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
Fix lack of message pluralization
commit : 4ad5611055c8381d6a9b54fc11838418a1f14bc9
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 10 Feb 2021 10:47:29 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 10 Feb 2021 10:47:29 +0100
M src/backend/access/transam/xlogfuncs.c
Fix ORDER BY clause in new regression test of REINDEX CONCURRENTLY
commit : 4f4726005082c07c004506a1ddfca29896375206
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 10 Feb 2021 16:59:04 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 10 Feb 2021 16:59:04 +0900
Oversight in bd12080.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210210065805.GG20012@telsasoft.com
Backpatch-through: 12
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Simplify code related to compilation of SSL and OpenSSL
commit : 092b785fad3de3f81355a4b2420aa39a1bc0ccd5
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 10 Feb 2021 15:28:19 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 10 Feb 2021 15:28:19 +0900
This commit makes more generic some comments and code related to the
compilation with OpenSSL and SSL in general to ease the addition of more
SSL implementations in the future. In libpq, some OpenSSL-only code is
moved under USE_OPENSSL and not USE_SSL.
While on it, make a comment more consistent in libpq-fe.h.
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/5382CB4A-9CF3-4145-BA46-C802615935E0@yesql.se
M src/backend/libpq/hba.c
M src/include/libpq/libpq-be.h
M src/include/pg_config_manual.h
M src/interfaces/libpq/fe-secure.c
M src/interfaces/libpq/libpq-fe.h
Preserve pg_attribute.attstattarget across REINDEX CONCURRENTLY
commit : bd12080980297dbc8ae926a3bd5b2ef9cc47932b
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 10 Feb 2021 13:06:48 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 10 Feb 2021 13:06:48 +0900
For an index, attstattarget can be updated using ALTER INDEX SET
STATISTICS. This data was lost on the new index after REINDEX
CONCURRENTLY.
The update of this field is done when the old and new indexes are
swapped to make the fix back-patchable. Another approach we could look
after in the long-term is to change index_create() to pass the wanted
values of attstattarget when creating the new relation, but, as this
would cause an ABI breakage this can be done only on HEAD.
Reported-by: Ronan Dunklau
Author: Michael Paquier
Reviewed-by: Ronan Dunklau, Tomas Vondra
Discussion: https://postgr.es/m/16628084.uLZWGnKmhe@laptop-ronand
Backpatch-through: 12
M src/backend/catalog/index.c
M src/backend/utils/cache/lsyscache.c
M src/include/utils/lsyscache.h
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Make pg_replication_origin_drop safe against concurrent drops.
commit : cd142e032ebd50ec7974b3633269477c2c72f1cc
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 10 Feb 2021 07:17:09 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 10 Feb 2021 07:17:09 +0530
Currently, we get the origin id from the name and then drop the origin by
taking ExclusiveLock on ReplicationOriginRelationId. So, two concurrent
sessions can get the id from the name at the same time and then when they
try to drop the origin, one of the sessions will get the either
"tuple concurrently deleted" or "cache lookup failed for replication
origin ..".
To prevent this race condition we do the entire operation under lock. This
obviates the need for replorigin_drop() API and we have removed it so if
any extension authors are using it they need to instead use
replorigin_drop_by_name. See it's usage in pg_replication_origin_drop().
Author: Peter Smith
Reviewed-by: Amit Kapila, Euler Taveira, Petr Jelinek, and Alvaro
Herrera
Discussion: https://www.postgresql.org/message-id/CAHut%2BPuW8DWV5fskkMWWMqzt-x7RPcNQOtJQBp6SdwyRghCk7A%40mail.gmail.com
M src/backend/commands/subscriptioncmds.c
M src/backend/replication/logical/origin.c
M src/include/replication/origin.h
Fix obsolete FSM remarks in nbtree README.
commit : 31c7fb41e26bf03dae231c7165a1a16388b2e366
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 9 Feb 2021 11:36:51 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 9 Feb 2021 11:36:51 -0800
The free space map has used a dedicated relation fork rather than shared
memory segments for over a decade.
M src/backend/access/nbtree/README
Revert "Display the time when the process started waiting for the lock, in pg_locks."
commit : 890d2182a2c425aaa80f9bf9f7116d31e0c6538e
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 9 Feb 2021 18:30:40 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 9 Feb 2021 18:30:40 +0900
This reverts commit 3b733fcd04195399db56f73f0616b4f5c6828e18.
Per buildfarm members prion and rorqual.
M contrib/amcheck/expected/check_btree.out
M doc/src/sgml/catalogs.sgml
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/adt/lockfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/storage/lock.h
M src/include/storage/proc.h
M src/test/regress/expected/rules.out
Display the time when the process started waiting for the lock, in pg_locks.
commit : 3b733fcd04195399db56f73f0616b4f5c6828e18
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 9 Feb 2021 18:10:19 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 9 Feb 2021 18:10:19 +0900
This commit adds new column "waitstart" into pg_locks view. This column
reports the time when the server process started waiting for the lock
if the lock is not held. This information is useful, for example, when
examining the amount of time to wait on a lock by subtracting
"waitstart" in pg_locks from the current time, and identify the lock
that the processes are waiting for very long.
This feature uses the current time obtained for the deadlock timeout
timer as "waitstart" (i.e., the time when this process started waiting
for the lock). Since getting the current time newly can cause overhead,
we reuse the already-obtained time to avoid that overhead.
Note that "waitstart" is updated without holding the lock table's
partition lock, to avoid the overhead by additional lock acquisition.
This can cause "waitstart" in pg_locks to become NULL for a very short
period of time after the wait started even though "granted" is false.
This is OK in practice because we can assume that users are likely to
look at "waitstart" when waiting for the lock for a long time.
Bump catalog version.
Author: Atsushi Torikoshi
Reviewed-by: Ian Lawrence Barwick, Robert Haas, Justin Pryzby, Fujii Masao
Discussion: https://postgr.es/m/a96013dc51cdc56b2a2b84fa8a16a993@oss.nttdata.com
M contrib/amcheck/expected/check_btree.out
M doc/src/sgml/catalogs.sgml
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/adt/lockfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/storage/lock.h
M src/include/storage/proc.h
M src/test/regress/expected/rules.out
Add option PROCESS_TOAST to VACUUM
commit : 7cb3048f38e26b39dd5fd412ed8a4981b6809b35
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 9 Feb 2021 14:13:57 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 9 Feb 2021 14:13:57 +0900
This option controls if toast tables associated with a relation are
vacuumed or not when running a manual VACUUM. It was already possible
to trigger a manual VACUUM on a toast relation without processing its
main relation, but a manual vacuum on a main relation always forced a
vacuum on its toast table. This is useful in scenarios where the level
of bloat or transaction age of the main and toast relations differs a
lot.
This option is an extension of the existing VACOPT_SKIPTOAST that was
used by autovacuum to control if toast relations should be skipped or
not. This internal flag is renamed to VACOPT_PROCESS_TOAST for
consistency with the new option.
A new option switch, called --no-process-toast, is added to vacuumdb.
Author: Nathan Bossart
Reviewed-by: Kirk Jamison, Michael Paquier, Justin Pryzby
Discussion: https://postgr.es/m/BA8951E9-1524-48C5-94AF-73B1F0D7857F@amazon.com
M doc/src/sgml/ref/vacuum.sgml
M doc/src/sgml/ref/vacuumdb.sgml
M src/backend/commands/vacuum.c
M src/backend/postmaster/autovacuum.c
M src/bin/psql/tab-complete.c
M src/bin/scripts/t/100_vacuumdb.pl
M src/bin/scripts/vacuumdb.c
M src/include/commands/vacuum.h
M src/test/regress/expected/vacuum.out
M src/test/regress/sql/vacuum.sql
Correct pgstattuple B-Tree page comments.
commit : 5fd590021d268190e4c8f377370c7e7f1e7d9229
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 8 Feb 2021 15:20:08 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 8 Feb 2021 15:20:08 -0800
M contrib/pgstattuple/pgstattuple.c
Fix mishandling of column-level SELECT privileges for join aliases.
commit : c028faf2a62ce476c8d689a33b12471dc98b80f2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Feb 2021 10:14:09 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Feb 2021 10:14:09 -0500
scanNSItemForColumn, expandNSItemAttrs, and ExpandSingleTable would
pass the wrong RTE to markVarForSelectPriv when dealing with a join
ParseNamespaceItem: they'd pass the join RTE, when what we need to
mark is the base table that the join column came from. The end
result was to not fill the base table's selectedCols bitmap correctly,
resulting in an understatement of the set of columns that are read
by the query. The executor would still insist on there being at
least one selectable column; but with a correctly crafted query,
a user having SELECT privilege on just one column of a table would
nonetheless be allowed to read all its columns.
To fix, make markRTEForSelectPriv fetch the correct RTE for itself,
ignoring the possibly-mismatched RTE passed by the caller. Later,
we'll get rid of some now-unused RTE arguments, but that risks
API breaks so we won't do it in released branches.
This problem was introduced by commit 9ce77d75c, so back-patch
to v13 where that came in. Thanks to Sven Klemm for reporting
the problem.
Security: CVE-2021-20229
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_target.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Fix permission checks on constraint violation errors on partitions.
commit : 6214e2b2280462cbc3aa1986e350e167651b3905
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 8 Feb 2021 11:01:51 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 8 Feb 2021 11:01:51 +0200
If a cross-partition UPDATE violates a constraint on the target partition,
and the columns in the new partition are in different physical order than
in the parent, the error message can reveal columns that the user does not
have SELECT permission on. A similar bug was fixed earlier in commit
804b6b6db4.
The cause of the bug is that the callers of the
ExecBuildSlotValueDescription() function got confused when constructing
the list of modified columns. If the tuple was routed from a parent, we
converted the tuple to the parent's format, but the list of modified
columns was grabbed directly from the child's RTE entry.
ExecUpdateLockMode() had a similar issue. That lead to confusion on which
columns are key columns, leading to wrong tuple lock being taken on tables
referenced by foreign keys, when a row is updated with INSERT ON CONFLICT
UPDATE. A new isolation test is added for that corner case.
With this patch, the ri_RangeTableIndex field is no longer set for
partitions that don't have an entry in the range table. Previously, it was
set to the RTE entry of the parent relation, but that was confusing.
NOTE: This modifies the ResultRelInfo struct, replacing the
ri_PartitionRoot field with ri_RootResultRelInfo. That's a bit risky to
backpatch, because it breaks any extensions accessing the field. The
change that ri_RangeTableIndex is not set for partitions could potentially
break extensions, too. The ResultRelInfos are visible to FDWs at least,
and this patch required small changes to postgres_fdw. Nevertheless, this
seem like the least bad option. I don't think these fields widely used in
extensions; I don't think there are FDWs out there that uses the FDW
"direct update" API, other than postgres_fdw. If there is, you will get a
compilation error, so hopefully it is caught quickly.
Backpatch to 11, where support for both cross-partition UPDATEs, and unique
indexes on partitioned tables, were added.
Reviewed-by: Amit Langote
Security: CVE-2021-3393
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/access/common/tupconvert.c
M src/backend/commands/copyfrom.c
M src/backend/commands/explain.c
M src/backend/commands/trigger.c
M src/backend/executor/execIndexing.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeModifyTable.c
M src/include/access/tupconvert.h
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
A src/test/isolation/expected/tuplelock-partition.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/tuplelock-partition.spec
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Rename removable xid function for consistency.
commit : 617fffee8a6f350ff03069e2843ecd039ea06ccc
author : Peter Geoghegan <pg@bowt.ie>
date : Sun, 7 Feb 2021 10:11:14 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Sun, 7 Feb 2021 10:11:14 -0800
GlobalVisIsRemovableFullXid() is now GlobalVisCheckRemovableFullXid().
This is consistent with the general convention for FullTransactionId
equivalents of functions that deal with TransactionId values. It now
matches the nearby GlobalVisCheckRemovableXid() function, which performs
the same check for callers that use TransactionId values.
Oversight in commit dc7420c2c92.
Discussion: https://postgr.es/m/CAH2-Wzmes12jFNDcVgpU89Vp=r6uLFrE-MT0fjSWGsE70UiNaA@mail.gmail.com
M src/backend/access/gist/gistutil.c
M src/backend/access/gist/gistxlog.c
M src/backend/storage/ipc/procarray.c
M src/include/utils/snapmgr.h
Revert "Propagate CTE property flags when copying a CTE list into a rule."
commit : d1d2979852538d7021cc809a40ef127d59747697
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 7 Feb 2021 12:54:08 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 7 Feb 2021 12:54:08 -0500
This reverts commit ed290896335414c6c069b9ccae1f3dcdd2fac6ba and
equivalent back-branch commits. The issue is subtler than I thought,
and it's far from new, so just before a release deadline is no time
to be fooling with it. We'll consider what to do at a bit more
leisure.
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Docs: fix pg_wal_lsn_diff manual.
commit : 04fd3eeba5be52369fa296fb001d1e52af6e166d
author : Tatsuo Ishii <ishii@postgresql.org>
date : Sun, 7 Feb 2021 13:43:50 +0900
committer: Tatsuo Ishii <ishii@postgresql.org>
date : Sun, 7 Feb 2021 13:43:50 +0900
The manual did not mention whether its return value is (first arg -
second arg) or (second arg - first arg). The order matters because the
return value could have a sign. Fix the manual so that it mentions the
function returns (first arg - second arg).
Patch reviewed by Tom Lane.
Back-patch through v13. Older version's doc format is difficult to add
more description.
Discussion: https://postgr.es/m/flat/20210206.151125.960423226279810864.t-ishii%40sraoss.co.jp
M doc/src/sgml/func.sgml
Propagate CTE property flags when copying a CTE list into a rule.
commit : ed290896335414c6c069b9ccae1f3dcdd2fac6ba
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 6 Feb 2021 19:28:39 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 6 Feb 2021 19:28:39 -0500
rewriteRuleAction() neglected this step, although it was careful to
propagate other similar flags such as hasSubLinks or hasRowSecurity.
Omitting to transfer hasRecursive is just cosmetic at the moment,
but omitting hasModifyingCTE is a live bug, since the executor
certainly looks at that.
The proposed test case only fails back to v10, but since the executor
examines hasModifyingCTE in 9.x as well, I suspect that a test case
could be devised that fails in older branches. Given the nearness
of the release deadline, though, I'm not going to spend time looking
for a better test.
Report and patch by Greg Nancarrow, cosmetic changes by me
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Disallow converting an inheritance child table to a view.
commit : dd705a039f6cd41921529fa4e971d70b224be052
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 6 Feb 2021 15:17:01 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 6 Feb 2021 15:17:01 -0500
Generally, members of inheritance trees must be plain tables (or,
in more recent versions, foreign tables). ALTER TABLE INHERIT
rejects creating an inheritance relationship that has a view at
either end. When DefineQueryRewrite attempts to convert a relation
to a view, it already had checks prohibiting doing so for partitioning
parents or children as well as traditional-inheritance parents ...
but it neglected to check that a traditional-inheritance child wasn't
being converted. Since the planner assumes that any inheritance
child is a table, this led to making plans that tried to do a physical
scan on a view, causing failures (or even crashes, in recent versions).
One could imagine trying to support such a case by expanding the view
normally, but since the rewriter runs before the planner does
inheritance expansion, it would take some very fundamental refactoring
to make that possible. There are probably a lot of other parts of the
system that don't cope well with such a situation, too. For now,
just forbid it.
Per bug #16856 from Yang Lin. Back-patch to all supported branches.
(In versions before v10, this includes back-patching the portion of
commit 501ed02cf that added has_superclass(). Perhaps the lack of
that infrastructure partially explains the missing check.)
Discussion: https://postgr.es/m/16856-0363e05c6e1612fd@postgresql.org
M src/backend/catalog/pg_inherits.c
M src/backend/rewrite/rewriteDefine.c
M src/test/regress/expected/rules.out
M src/test/regress/sql/rules.sql
Clarify some comments around SharedRecoveryState in xlog.c
commit : f7400823c3bd6ce03c2fe1bec5b7066bad146809
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 6 Feb 2021 10:27:55 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 6 Feb 2021 10:27:55 +0900
SharedRecoveryState has been switched from a boolean to an enum as of
commit 4e87c48, but some comments still referred to it as a boolean.
Author: Amul Sul
Reviewed-by: Dilip Kumar, Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAAJ_b97Hf+1SXnm8jySpO+Fhm+-VKFAAce1T_cupUYtnE3Nxig
M src/backend/access/transam/xlog.c
Generalize parallel slot result handling.
commit : 418611c84d004f45d92bcaa3f8e100385d96cd41
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 5 Feb 2021 16:08:45 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 5 Feb 2021 16:08:45 -0500
Instead of having a hard-coded behavior that we ignore missing
tables and report all other errors, let the caller decide what
to do by setting a callback.
Mark Dilger, reviewed and somewhat revised by me. The larger patch
series of which this is a part has also had review from Peter
Geoghegan, Andres Freund, Álvaro Herrera, Michael Paquier, and Amul
Sul, but I don't know whether any of them have reviewed this bit
specifically.
Discussion: http://postgr.es/m/12ED3DA8-25F0-4B68-937D-D907CFBF08E7@enterprisedb.com
Discussion: http://postgr.es/m/5F743835-3399-419C-8324-2D424237E999@enterprisedb.com
Discussion: http://postgr.es/m/70655DF3-33CE-4527-9A4D-DDEB582B6BA0@enterprisedb.com
M src/bin/scripts/reindexdb.c
M src/bin/scripts/vacuumdb.c
M src/fe_utils/parallel_slot.c
M src/include/fe_utils/parallel_slot.h
Move some code from src/bin/scripts to src/fe_utils to permit reuse.
commit : e955bd4b6c2bcdbd253837f6cf4c7520b98e69d4
author : Robert Haas <rhaas@postgresql.org>
date : Fri, 5 Feb 2021 13:33:38 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Fri, 5 Feb 2021 13:33:38 -0500
The parallel slots infrastructure (which implements client-side
multiplexing of server connections doing similar things, not
threading or multiple processes or anything like that) are moved from
src/bin/scripts/scripts_parallel.c to src/fe_utils/parallel_slot.c.
The functions consumeQueryResult() and processQueryResult() which were
previously part of src/bin/scripts/common.c are now moved into that
file as well, becoming static helper functions. This might need to be
changed in the future, but currently they're not used for anything
else.
Some other functions from src/bin/scripts/common.c are moved to to
src/fe_utils and are split up among several files. connectDatabase(),
connectMaintenanceDatabase(), and disconnectDatabase() are moved to
connect_utils.c. executeQuery(), executeCommand(), and
executeMaintenanceCommand() are move to query_utils.c.
handle_help_version_opts() is moved to option_utils.c.
Mark Dilger, reviewed by me. The larger patch series of which this is
a part has also had review from Peter Geoghegan, Andres Freund, Álvaro
Herrera, Michael Paquier, and Amul Sul, but I don't know whether any
of them have reviewed this bit specifically.
Discussion: http://postgr.es/m/12ED3DA8-25F0-4B68-937D-D907CFBF08E7@enterprisedb.com
Discussion: http://postgr.es/m/5F743835-3399-419C-8324-2D424237E999@enterprisedb.com
Discussion: http://postgr.es/m/70655DF3-33CE-4527-9A4D-DDEB582B6BA0@enterprisedb.com
M src/bin/scripts/Makefile
M src/bin/scripts/clusterdb.c
M src/bin/scripts/common.c
M src/bin/scripts/common.h
M src/bin/scripts/createdb.c
M src/bin/scripts/createuser.c
M src/bin/scripts/dropdb.c
M src/bin/scripts/dropuser.c
M src/bin/scripts/nls.mk
M src/bin/scripts/pg_isready.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/vacuumdb.c
M src/fe_utils/Makefile
A src/fe_utils/connect_utils.c
A src/fe_utils/option_utils.c
R080 src/bin/scripts/scripts_parallel.c src/fe_utils/parallel_slot.c
A src/fe_utils/query_utils.c
A src/include/fe_utils/connect_utils.h
A src/include/fe_utils/option_utils.h
R082 src/bin/scripts/scripts_parallel.h src/include/fe_utils/parallel_slot.h
A src/include/fe_utils/query_utils.h
M src/tools/msvc/Mkvcbuild.pm
Fix backslash-escaping multibyte chars in COPY FROM.
commit : c444472af5c202067a9ecb0ff8df7370fb1ea8f4
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 5 Feb 2021 11:14:56 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 5 Feb 2021 11:14:56 +0200
If a multi-byte character is escaped with a backslash in TEXT mode input,
and the encoding is one of the client-only encodings where the bytes after
the first one can have an ASCII byte "embedded" in the char, we didn't
skip the character correctly. After a backslash, we only skipped the first
byte of the next character, so if it was a multi-byte character, we would
try to process its second byte as if it was a separate character. If it
was one of the characters with special meaning, like '\n', '\r', or
another '\\', that would cause trouble.
One such exmple is the byte sequence '\x5ca45c2e666f6f' in Big5 encoding.
That's supposed to be [backslash][two-byte character][.][f][o][o], but
because the second byte of the two-byte character is 0x5c, we incorrectly
treat it as another backslash. And because the next character is a dot, we
parse it as end-of-copy marker, and throw an "end-of-copy marker corrupt"
error.
Backpatch to all supported versions.
Reviewed-by: John Naylor, Kyotaro Horiguchi
Discussion: https://www.postgresql.org/message-id/a897f84f-8dca-8798-3139-07da5bb38728%40iki.fi
M src/backend/commands/copyfromparse.c
postgres_fdw: Fix assertion in estimate_path_cost_size().
commit : 5e7fa189ee92d5ecf42a295c336625d71bfe876d
author : Etsuro Fujita <efujita@postgresql.org>
date : Fri, 5 Feb 2021 15:30:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Fri, 5 Feb 2021 15:30:00 +0900
Commit 08d2d58a2 added an assertion assuming that the retrieved_rows
estimate for a foreign relation, which is re-used to cost pre-sorted
foreign paths with local stats, is set to at least one row in
estimate_path_cost_size(), which isn't correct because if the relation
is a foreign table with tuples=0, the estimate would be set to 0 there
when not using remote estimates.
Per bug #16807 from Alexander Lakhin. Back-patch to v13 where the
aforementioned commit went in.
Author: Etsuro Fujita
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/16807-9fe4e08fbaa5c7ce%40postgresql.org
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
Fix bug in HashAgg's selective-column-spilling logic.
commit : 0ff865fbe50e82f17df8a9280fa01faf270b7f3f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 4 Feb 2021 23:01:33 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 4 Feb 2021 23:01:33 -0500
Commit 230230223 taught nodeAgg.c that, when spilling tuples from
memory in an oversized hash aggregation, it only needed to spill
input columns referenced in the node's tlist and quals. Unfortunately,
that's wrong: we also have to save the grouping columns. The error
is masked in common cases because the grouping columns also appear
in the tlist, but that's not necessarily true. The main category
of plans where it's not true seem to come from semijoins ("WHERE
outercol IN (SELECT innercol FROM innertable)") where the innercol
needs an implicit promotion to make it comparable to the outercol.
The grouping column will be "innercol::promotedtype", but that
expression appears nowhere in the Agg node's own tlist and quals;
only the bare "innercol" is found in the tlist.
I spent quite a bit of time looking for a suitable regression test
case for this, without much success. If the number of distinct
values of the innercol is large enough to make spilling happen,
the planner tends to prefer a non-HashAgg plan, at least for
problem sizes that are reasonable to use in the regression tests.
So, no new regression test. However, this patch does demonstrably
fix the originally-reported test case.
Per report from s.p.e (at) gmx-topmail.de. Backpatch to v13
where the troublesome code came in.
Discussion: https://postgr.es/m/trinity-1c565d44-159f-488b-a518-caf13883134f-1611835701633@3c-app-gmx-bap78
M src/backend/executor/nodeAgg.c
Tab-complete CREATE DATABASE ... LOCALE.
commit : e1c02d92aee30328316c309c3c2f305d77f231a2
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 5 Feb 2021 15:30:56 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 5 Feb 2021 15:30:56 +1300
Author: Ian Lawrence Barwick <barwick@gmail.com>
Discussion: https://postgr.es/m/CAB8KJ%3Dh0XO2CB4QbLBc1Tm9Bg5wzSGQtT-eunaCmrghJp4nqdA%40mail.gmail.com
M src/bin/psql/tab-complete.c
Fix YA incremental sort bug.
commit : 82e0e29308dedbc6000e73329beb112ae7e1ad39
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 4 Feb 2021 19:12:09 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 4 Feb 2021 19:12:09 -0500
switchToPresortedPrefixMode() did the wrong thing if it detected
a batch boundary just at the last tuple of a fullsort group.
The initially-reported symptom was a "retrieved too many tuples in a
bounded sort" error, but the test case added here just silently gives
the wrong answer without this patch.
I (tgl) am not really happy about committing this patch without review
from the incremental-sort authors, but they seem AWOL and we are hard
against a release deadline. This does demonstrably make some cases
better, anyway.
Per bug #16846 from Yoran Heling. Back-patch to v13 where incremental
sort was introduced.
Neil Chen
Discussion: https://postgr.es/m/16846-ae49f51ac379a4cb@postgresql.org
M src/backend/executor/nodeIncrementalSort.c
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Harden nbtree page deletion.
commit : c34787f910585f82320f78b0afd53a6a170aa229
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 4 Feb 2021 15:42:36 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 4 Feb 2021 15:42:36 -0800
Add some additional defensive checks in the second phase of index
deletion to detect and report index corruption during VACUUM, and to
avoid having VACUUM become stuck in more cases. The code is still not
robust in the presence of a circular chain of sibling links, though it's
not clear whether that really matters. This is follow-up work to commit
3a01f68e.
The new defensive checks rely on the assumption that there can be no
more than one VACUUM operation running for an index at any given time.
Remove an old comment suggesting that multiple concurrent VACUUMs need
to be considered here. This concern now seems highly unlikely to have
any real validity, since we clearly rely on the same assumption in
several other places. For example, there are much more recent comments
that appear in the same function (added by commit efada2b8e92) that make
the same assumption.
Also add a CHECK_FOR_INTERRUPTS() to the relevant code path. Contrary
to comments added by commit 3a01f68e, it is actually possible to handle
interrupts here, at least in the common case where processing takes
place at the leaf level. We only hold a pin on leafbuf/target page when
stepping right at the leaf level.
No backpatch due to the lack of complaints following hardening added to
the same area by commit 3a01f68e.
M src/backend/access/nbtree/nbtpage.c
Fix small error in COPY FROM progress reporting.
commit : 2f86ab305e7fbc7b84960079551cf9cafd29684f
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 4 Feb 2021 17:40:33 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 4 Feb 2021 17:40:33 +0200
The # of bytes processed was accumulated slightly incorrectly. After
loading more data to the input buffer, we added the number of bytes in
the buffer to the sum. But in case of multi-byte characters or escapes,
there can be a few unprocessed bytes left over from previous load in the
buffer. Those bytes got counted twice.
M src/backend/commands/copyfromparse.c
Refactor Windows error message for easier translation
commit : 3c78e0569ca04f4c92f0adcd74471398bb7b2e55
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 4 Feb 2021 13:31:13 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 4 Feb 2021 13:31:13 +0100
In the error messages referring to the user right "Lock pages in
memory", this is a term from the Windows OS, so it should be
translated in accordance with the OS localization. Refactor the error
messages so this is easier and clearer. Also fix the capitalization
to match the existing capitalization in the OS.
M doc/src/sgml/config.sgml
M src/backend/port/win32_shmem.c
Ensure unlinking of old index file with REINDEX (TABLESPACE)
commit : 5128483d064038702f535aced2cbaa43256db214
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 4 Feb 2021 17:16:47 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 4 Feb 2021 17:16:47 +0900
The original versions of the patch included this part, but a mismerge
from my side has made this piece go missing. Oversight in c5b28604.
M src/backend/catalog/index.c
Clarify comment in tablesync.c
commit : fc749bc7041cb77b5f6b58c129ad2616a3f7ab4f
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 4 Feb 2021 16:02:31 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 4 Feb 2021 16:02:31 +0900
Author: Peter Smith
Reviewed-by: Amit Kapila, Michael Paquier, Euler Taveira
Discussion: https://postgr.es/m/CAHut+Pt9_T6pWar0FLtPsygNmme8HPWPdGUyZ_8mE1Yvjdf0ZA@mail.gmail.com
M src/backend/replication/logical/tablesync.c
Add TABLESPACE option to REINDEX
commit : c5b286047cd698021e57a527215b48865fd4ad4e
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 4 Feb 2021 14:34:20 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 4 Feb 2021 14:34:20 +0900
This patch adds the possibility to move indexes to a new tablespace
while rebuilding them. Both the concurrent and the non-concurrent cases
are supported, and the following set of restrictions apply:
- When using TABLESPACE with a REINDEX command that targets a
partitioned table or index, all the indexes of the leaf partitions are
moved to the new tablespace. The tablespace references of the non-leaf,
partitioned tables in pg_class.reltablespace are not changed. This
requires an extra ALTER TABLE SET TABLESPACE.
- Any index on a toast table rebuilt as part of a parent table is kept
in its original tablespace.
- The operation is forbidden on system catalogs, including trying to
directly move a toast relation with REINDEX. This results in an error
if doing REINDEX on a single object. REINDEX SCHEMA, DATABASE and
SYSTEM skip system relations when TABLESPACE is used.
Author: Alexey Kondratov, Michael Paquier, Justin Pryzby
Reviewed-by: Álvaro Herrera, Michael Paquier
Discussion: https://postgr.es/m/8a8f5f73-00d3-55f8-7583-1375ca8f6a91@postgrespro.ru
M doc/src/sgml/ref/reindex.sgml
M src/backend/catalog/index.c
M src/backend/commands/indexcmds.c
M src/bin/psql/tab-complete.c
M src/include/catalog/index.h
M src/test/regress/input/tablespace.source
M src/test/regress/output/tablespace.source
Avoid crash when rolling back within a prepared statement.
commit : 9624321ec502f4e4f4722290b358694049447f95
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 3 Feb 2021 19:38:29 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 3 Feb 2021 19:38:29 -0500
If a portal is used to run a prepared CALL or DO statement that
contains a ROLLBACK, PortalRunMulti fails because the portal's
statement list gets cleared by the rollback. (Since the grammar
doesn't allow CALL/DO in PREPARE, the only easy way to get to this is
via extended query protocol, which treats all inputs as prepared
statements.) It's difficult to avoid resetting the portal early
because of resource-management issues, so work around this by teaching
PortalRunMulti to be wary of portal->stmts having suddenly become NIL.
The crash has only been seen to occur in v13 and HEAD (as a
consequence of commit 1cff1b95a having added an extra touch of
portal->stmts). But even before that, the code involved touching a
List that the portal no longer has any claim on. In the test case at
hand, the List will still exist because of another refcount on the
cached plan; but I'm far from convinced that it's impossible for the
cached plan to have been dropped by the time control gets back to
PortalRunMulti. Hence, backpatch to v11 where nested transactions
were added.
Thomas Munro and Tom Lane, per bug #16811 from James Inform
Discussion: https://postgr.es/m/16811-c1b599b2c6c2d622@postgresql.org
M src/backend/tcop/pquery.c
Factor pattern-construction logic out of processSQLNamePattern.
commit : 2c8726c4b0a496608919d1f78a5abc8c9b6e0868
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 3 Feb 2021 13:19:41 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 3 Feb 2021 13:19:41 -0500
The logic for converting the shell-glob-like syntax supported by
utilities like psql and pg_dump to regular expression is
extracted into a new function patternToSQLRegex. The existing
function processSQLNamePattern now uses this function as a
subroutine.
patternToSQLRegex is a little more general than what is required
by processSQLNamePattern. That function is only interested in
patterns that can have up to 2 parts, a schema and a relation;
but patternToSQLRegex can limit the maximum number of parts to
between 1 and 3, so that patterns can look like either
"database.schema.relation", "schema.relation", or "relation"
depending on how it's invoked and what the user specifies.
processSQLNamePattern only passes two buffers, so works exactly
the same as before, always interpreting the pattern as either
a "schema.relation" pattern or a "relation" pattern. But,
future callers can use this function in other ways.
Mark Dilger, reviewed by me. The larger patch series of which this is
a part has also had review from Peter Geoghegan, Andres Freund, Álvaro
Herrera, Michael Paquier, and Amul Sul, but I don't know whether
any of them have reviewed this bit specifically.
Discussion: http://postgr.es/m/12ED3DA8-25F0-4B68-937D-D907CFBF08E7@enterprisedb.com
Discussion: http://postgr.es/m/5F743835-3399-419C-8324-2D424237E999@enterprisedb.com
Discussion: http://postgr.es/m/70655DF3-33CE-4527-9A4D-DDEB582B6BA0@enterprisedb.com
M src/fe_utils/string_utils.c
M src/include/fe_utils/string_utils.h
Remove special BKI_LOOKUP magic for namespace and role OIDs.
commit : ba0faf81c65ac99dd42ce192f3257d4d2231ea50
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 3 Feb 2021 12:01:48 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 3 Feb 2021 12:01:48 -0500
Now that commit 62f34097c attached BKI_LOOKUP annotation to all the
namespace and role OID columns in the catalogs, there's no real reason
to have the magic PGNSP and PGUID symbols. Get rid of them in favor
of implementing those lookups according to genbki.pl's normal pattern.
This means that in the catalog headers, BKI_DEFAULT(PGNSP) becomes
BKI_DEFAULT(pg_catalog), which seems a lot more transparent.
BKI_DEFAULT(PGUID) becomes BKI_DEFAULT(POSTGRES), which is perhaps
less so; but you can look into pg_authid.dat to discover that
POSTGRES is the nonce name for the bootstrap superuser.
This change also means that if we ever need cross-references in the
initial catalog data to any of the other built-in roles besides
POSTGRES, or to some other built-in schema besides pg_catalog,
we can just do it.
No catversion bump here, as there's no actual change in the contents
of postgres.bki.
Discussion: https://postgr.es/m/3240355.1612129197@sss.pgh.pa.us
M doc/src/sgml/bki.sgml
M src/backend/catalog/genbki.pl
M src/include/catalog/pg_authid.dat
M src/include/catalog/pg_class.h
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_conversion.h
M src/include/catalog/pg_database.h
M src/include/catalog/pg_language.h
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_opclass.h
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_opfamily.h
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_ts_config.h
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_ts_parser.h
M src/include/catalog/pg_ts_template.h
M src/include/catalog/pg_type.h
pg_dump: Fix dumping of inherited generated columns
commit : 0bf83648a52df96f7c8677edbbdf141bfa0cf32b
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 3 Feb 2021 11:27:13 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 3 Feb 2021 11:27:13 +0100
Generation expressions of generated columns are always inherited, so
there is no need to set them separately in child tables, and there is
no syntax to do so either. The code previously used the code paths
for the handling of default values, for which different rules apply;
in particular it might want to set a default value explicitly for an
inherited column. This resulted in unrestorable dumps. For generated
columns, just skip them in inherited tables.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/15830.1575468847%40sss.pgh.pa.us
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/t/002_pg_dump.pl
Retire findoidjoins.
commit : ef3d4613c0204ab2b87ffa7e8e9551d74f932816
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Feb 2021 17:21:37 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Feb 2021 17:21:37 -0500
In the wake of commit 62f34097c, we no longer need this tool.
Discussion: https://postgr.es/m/3240355.1612129197@sss.pgh.pa.us
M src/tools/RELEASE_CHANGES
D src/tools/findoidjoins/.gitignore
D src/tools/findoidjoins/Makefile
D src/tools/findoidjoins/README
D src/tools/findoidjoins/findoidjoins.c
D src/tools/findoidjoins/make_oidjoins_check
Build in some knowledge about foreign-key relationships in the catalogs.
commit : 62f34097c88433ef1f3de604714fe7e7024f2fdf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Feb 2021 17:11:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Feb 2021 17:11:55 -0500
This follows in the spirit of commit dfb75e478, which created primary
key and uniqueness constraints to improve the visibility of constraints
imposed on the system catalogs. While our catalogs contain many
foreign-key-like relationships, they don't quite follow SQL semantics,
in that the convention for an omitted reference is to write zero not
NULL. Plus, we have some cases in which there are arrays each of whose
elements is supposed to be an FK reference; SQL has no way to model that.
So we can't create actual foreign key constraints to describe the
situation. Nonetheless, we can collect and use knowledge about these
relationships.
This patch therefore adds annotations to the catalog header files to
declare foreign-key relationships. (The BKI_LOOKUP annotations cover
simple cases, but we weren't previously distinguishing which such
columns are allowed to contain zeroes; we also need new markings for
multi-column FK references.) Then, Catalog.pm and genbki.pl are
taught to collect this information into a table in a new generated
header "system_fk_info.h". The only user of that at the moment is
a new SQL function pg_get_catalog_foreign_keys(), which exposes the
table to SQL. The oidjoins regression test is rewritten to use
pg_get_catalog_foreign_keys() to find out which columns to check.
Aside from removing the need for manual maintenance of that test
script, this allows it to cover numerous relationships that were not
checked by the old implementation based on findoidjoins. (As of this
commit, 217 relationships are checked by the test, versus 181 before.)
Discussion: https://postgr.es/m/3240355.1612129197@sss.pgh.pa.us
M doc/src/sgml/bki.sgml
M doc/src/sgml/func.sgml
M src/backend/catalog/.gitignore
M src/backend/catalog/Catalog.pm
M src/backend/catalog/Makefile
M src/backend/catalog/genbki.pl
M src/backend/utils/adt/misc.c
M src/include/Makefile
M src/include/catalog/.gitignore
M src/include/catalog/catversion.h
M src/include/catalog/genbki.h
M src/include/catalog/pg_aggregate.h
M src/include/catalog/pg_amop.h
M src/include/catalog/pg_attrdef.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_auth_members.h
M src/include/catalog/pg_cast.h
M src/include/catalog/pg_class.h
M src/include/catalog/pg_collation.dat
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_conversion.h
M src/include/catalog/pg_database.h
M src/include/catalog/pg_db_role_setting.h
M src/include/catalog/pg_default_acl.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_description.h
M src/include/catalog/pg_enum.h
M src/include/catalog/pg_event_trigger.h
M src/include/catalog/pg_extension.h
M src/include/catalog/pg_foreign_data_wrapper.h
M src/include/catalog/pg_foreign_server.h
M src/include/catalog/pg_foreign_table.h
M src/include/catalog/pg_index.h
M src/include/catalog/pg_inherits.h
M src/include/catalog/pg_init_privs.h
M src/include/catalog/pg_language.h
M src/include/catalog/pg_largeobject.h
M src/include/catalog/pg_largeobject_metadata.h
M src/include/catalog/pg_namespace.dat
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_opclass.h
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_opfamily.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_policy.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_publication.h
M src/include/catalog/pg_publication_rel.h
M src/include/catalog/pg_range.h
M src/include/catalog/pg_rewrite.h
M src/include/catalog/pg_seclabel.h
M src/include/catalog/pg_sequence.h
M src/include/catalog/pg_shdepend.h
M src/include/catalog/pg_shdescription.h
M src/include/catalog/pg_shseclabel.h
M src/include/catalog/pg_statistic.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/catalog/pg_subscription.h
M src/include/catalog/pg_subscription_rel.h
M src/include/catalog/pg_tablespace.dat
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_transform.h
M src/include/catalog/pg_trigger.h
M src/include/catalog/pg_ts_config.h
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_ts_parser.h
M src/include/catalog/pg_ts_template.h
M src/include/catalog/pg_type.h
M src/include/catalog/pg_user_mapping.h
M src/test/regress/expected/oidjoins.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
M src/test/regress/sql/oidjoins.sql
M src/tools/msvc/Solution.pm
M src/tools/msvc/clean.bat
Doc: consistently identify OID catalog columns that can be zero.
commit : 479331406e8403cc2e75d1082f8c613e7669c113
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Feb 2021 16:15:29 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Feb 2021 16:15:29 -0500
Not all OID-reference columns that can contain zeroes (indicating
"no reference") were explicitly marked in catalogs.sgml. Fix that,
and try to make the style a little more consistent while at it ---
for example, consistently write "zero" not "0" for these cases.
Joel Jacobson and Tom Lane
Discussion: https://postgr.es/m/4ed9a372-7bf9-479a-926c-ae8e774717a8@www.fastmail.com
M doc/src/sgml/catalogs.sgml
Remove extra increment of plpgsql's statement counter for FOR loops.
commit : dfcc46fe3030b0114b7a5715d5fa40819561c04b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Feb 2021 14:35:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Feb 2021 14:35:12 -0500
This left gaps in the internal statement numbering, which is not
terribly harmful (else we'd have noticed sooner), but it's not
great either.
Oversight in bbd5c207b; backpatch to v12 where that came in.
Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRDXyQaJmpotNTQVc-t-WxdWZC35V2PnmwOaV1-taidFWA@mail.gmail.com
M src/pl/plpgsql/src/pl_gram.y
Fix ancient memory leak in contrib/auto_explain.
commit : 5c0f7cc5442108e113d4fb88c952329b467e2c6a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Feb 2021 13:49:08 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Feb 2021 13:49:08 -0500
The ExecutorEnd hook is invoked in a context that could be quite
long-lived, not the executor's own per-query context as I think
we were sort of assuming. Thus, any cruft generated while producing
the EXPLAIN output could accumulate over multiple queries. This can
result in spectacular leakage if log_nested_statements is on, and
even without that I'm surprised nobody complained before.
To fix, just switch into the executor's context so that anything we
allocate will be released when standard_ExecutorEnd frees the executor
state. We might as well nuke the code's retail pfree of the explain
output string, too; that's laughably inadequate to the need.
Japin Li, per report from Jeff Janes. This bug is old, so
back-patch to all supported branches.
Discussion: https://postgr.es/m/CAMkU=1wCVtbeRn0s9gt12KwQ7PLXovbpM8eg25SYocKW3BT4hg@mail.gmail.com
M contrib/auto_explain/auto_explain.c
Improve confusing variable names
commit : 1d71f3c83c113849fe3aa60cb2d2c12729485e97
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 2 Feb 2021 09:20:22 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 2 Feb 2021 09:20:22 +0100
The prototype calls the second argument of
pgstat_progress_update_multi_param() "index", and some callers name
their local variable that way. But when the surrounding code deals
with index relations, this is confusing, and in at least one case
shadowed another variable that is referring to an index relation.
Adjust those call sites to have clearer local variable naming, similar
to existing callers in indexcmds.c.
M src/backend/access/nbtree/nbtsort.c
M src/backend/catalog/index.c
Remove unused column atttypmod from initial tablesync query
commit : 4ad31bb2ef2517b6e49ea9e8f01aaed250d4df38
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 2 Feb 2021 13:59:23 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 2 Feb 2021 13:59:23 +0900
The initial tablesync done by logical replication used a query to fetch
the information of a relation's columns that included atttypmod, but it
was left unused. This was added by 7c4f524.
Author: Euler Taveira
Reviewed-by: Önder Kalacı, Amit Langote, Japin Li
Discussion: https://postgr.es/m/CAHE3wggb715X+mK_DitLXF25B=jE6xyNCH4YOwM860JR7HarGQ@mail.gmail.com
M src/backend/replication/logical/tablesync.c
Doc: work a little harder on the initial examples for regex matching.
commit : 9522085ac917af66dba29939af328ae67300f10a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Feb 2021 16:38:52 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Feb 2021 16:38:52 -0500
Writing unnecessary '.*' at start and end of a POSIX regex doesn't
do much except confuse the reader about whether that might be
necessary after all. Make the examples in table 9.16 a tad more
realistic, and try to turn the next group of examples into something
self-contained.
Per gripe from rmzgrimes. Back-patch to v13 because it's easy.
Discussion: https://postgr.es/m/161215841824.14653.8969016349304314299@wrigleys.postgresql.org
M doc/src/sgml/func.sgml
Remove [Merge]AppendPath.partitioned_rels.
commit : f003a7522bfa11177dc52c65eb97273a1057dfba
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Feb 2021 14:43:54 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Feb 2021 14:43:54 -0500
It turns out that the calculation of [Merge]AppendPath.partitioned_rels
in allpaths.c is faulty and sometimes omits relevant non-leaf partitions,
allowing an assertion added by commit a929e17e5a8 to trigger. Rather
than fix that, it seems better to get rid of those fields altogether.
We don't really need the info until create_plan time, and calculating
it once for the selected plan should be cheaper than calculating it
for each append path we consider.
The preceding two commits did away with all use of the partitioned_rels
values; this commit just mechanically removes the fields and the code
that calculated them.
Discussion: https://postgr.es/m/87sg8tqhsl.fsf@aurora.ydns.eu
Discussion: https://postgr.es/m/CAJKUy5gCXDSmFs2c=R+VGgn7FiYcLCsEFEuDNNLGfoha=pBE_g@mail.gmail.com
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/joinrels.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/pathnode.c
M src/backend/partitioning/partprune.c
M src/include/nodes/pathnodes.h
M src/include/optimizer/pathnode.h
M src/include/partitioning/partprune.h
Remove incidental dependencies on partitioned_rels lists.
commit : 5076f88bc985a7728eea337cbabae0e034b064b1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Feb 2021 14:34:59 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Feb 2021 14:34:59 -0500
It turns out that the calculation of [Merge]AppendPath.partitioned_rels
in allpaths.c is faulty and sometimes omits relevant non-leaf partitions,
allowing an assertion added by commit a929e17e5a8 to trigger. Rather
than fix that, it seems better to get rid of those fields altogether.
We don't really need the info until create_plan time, and calculating
it once for the selected plan should be cheaper than calculating it
for each append path we consider.
This patch undoes a couple of very minor uses of the partitioned_rels
values.
createplan.c was testing for nil-ness to optimize away the preparatory
work for make_partition_pruneinfo(). That is worth doing if the check
is nigh free, but it's not worth going to any great lengths to avoid.
create_append_path() was testing for nil-ness as part of deciding how
to set up ParamPathInfo for an AppendPath. I replaced that with a
check for the appendrel's parent rel being partitioned. That's not
quite the same thing but should cover most cases. If we note any
interesting loss of optimizations, we can dumb this down to just
always use the more expensive method when the parent is a baserel.
Discussion: https://postgr.es/m/87sg8tqhsl.fsf@aurora.ydns.eu
Discussion: https://postgr.es/m/CAJKUy5gCXDSmFs2c=R+VGgn7FiYcLCsEFEuDNNLGfoha=pBE_g@mail.gmail.com
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/pathnode.c
Revise make_partition_pruneinfo to not use its partitioned_rels input.
commit : fb2d645dd53ff571572d830e830fc8c368063802
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Feb 2021 14:05:51 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Feb 2021 14:05:51 -0500
It turns out that the calculation of [Merge]AppendPath.partitioned_rels
in allpaths.c is faulty and sometimes omits relevant non-leaf partitions,
allowing an assertion added by commit a929e17e5a8 to trigger. Rather
than fix that, it seems better to get rid of those fields altogether.
We don't really need the info until create_plan time, and calculating
it once for the selected plan should be cheaper than calculating it
for each append path we consider.
As a first step, teach make_partition_pruneinfo to collect the relevant
partitioned tables for itself. It's not hard to do so by traversing
from child tables up to parents using the AppendRelInfo links.
While here, make some minor stylistic improvements; mainly, don't use
the "Relids" alias for bitmapsets that are not identities of any
relation considered by the planner. Try to document the logic better,
too.
No backpatch, as there does not seem to be a live problem before
a929e17e5a8. Also no new regression test; the code where the bug
was will be gone at the end of this patch series, so it seems a
bit pointless to memorialize the issue.
Tom Lane and David Rowley, per reports from Andreas Seltenreich
and Jaime Casanova.
Discussion: https://postgr.es/m/87sg8tqhsl.fsf@aurora.ydns.eu
Discussion: https://postgr.es/m/CAJKUy5gCXDSmFs2c=R+VGgn7FiYcLCsEFEuDNNLGfoha=pBE_g@mail.gmail.com
M src/backend/partitioning/partprune.c
SEARCH and CYCLE clauses
commit : 3696a600e2292d43c00949ddf0352e4ebb487e5b
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 1 Feb 2021 13:54:59 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 1 Feb 2021 13:54:59 +0100
This adds the SQL standard feature that adds the SEARCH and CYCLE
clauses to recursive queries to be able to do produce breadth- or
depth-first search orders and detect cycles. These clauses can be
rewritten into queries using existing syntax, and that is what this
patch does in the rewriter.
Reviewed-by: Vik Fearing <vik@postgresfriends.org>
Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/db80ceee-6f97-9b4a-8ee8-3ba0c58e5be2@2ndquadrant.com
M doc/src/sgml/queries.sgml
M doc/src/sgml/ref/select.sgml
M src/backend/catalog/dependency.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/parser/parse_agg.c
M src/backend/parser/parse_cte.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_func.c
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_target.c
M src/backend/rewrite/Makefile
M src/backend/rewrite/rewriteHandler.c
A src/backend/rewrite/rewriteSearchCycle.c
M src/backend/utils/adt/ruleutils.c
M src/include/nodes/nodes.h
M src/include/nodes/parsenodes.h
M src/include/parser/analyze.h
M src/include/parser/kwlist.h
M src/include/parser/parse_node.h
A src/include/rewrite/rewriteSearchCycle.h
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Get rid of unnecessary memory allocation in jsonb_subscript_assign()
commit : bb513b364b4fe31574574c8d0afbb2255268b321
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 1 Feb 2021 14:06:02 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 1 Feb 2021 14:06:02 +0300
Current code allocates memory for JsonbValue, but it could be placed locally.
M src/backend/utils/adt/jsonbsubs.c
Introduce --with-ssl={openssl} as a configure option
commit : fe61df7f82aa6e0879476146dbe1da9c89b4946b
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 1 Feb 2021 19:19:44 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 1 Feb 2021 19:19:44 +0900
This is a replacement for the existing --with-openssl, extending the
logic to make easier the addition of new SSL libraries. The grammar is
chosen to be similar to --with-uuid, where multiple values can be
chosen, with "openssl" as the only supported value for now.
The original switch, --with-openssl, is kept for compatibility.
Author: Daniel Gustafsson, Michael Paquier
Reviewed-by: Jacob Champion
Discussion: https://postgr.es/m/FAB21FC8-0F62-434F-AA78-6BD9336D630A@yesql.se
M configure
M configure.ac
M contrib/Makefile
M contrib/pgcrypto/Makefile
M doc/src/sgml/installation.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/sslinfo.sgml
M src/Makefile.global.in
M src/backend/libpq/Makefile
M src/backend/libpq/hba.c
M src/common/Makefile
M src/include/pg_config.h.in
M src/interfaces/libpq/Makefile
M src/test/Makefile
M src/test/modules/Makefile
M src/test/modules/ssl_passphrase_callback/Makefile
M src/test/modules/ssl_passphrase_callback/t/001_testfunc.pl
M src/test/ssl/Makefile
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
M src/tools/msvc/Solution.pm
M src/tools/msvc/config_default.pl
Fix portability issue in new jsonbsubs code.
commit : 7c5d57caed4d8af705d0cc3131d0d8ed72b7a41d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Feb 2021 02:03:59 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Feb 2021 02:03:59 -0500
On machines where sizeof(Datum) > sizeof(Oid) (that is, any 64-bit
platform), the previous coding would compute a misaligned
workspace->index pointer if nupper is odd. Architectures where
misaligned access is a hard no-no would then fail. This appears
to explain why thorntail is unhappy but other buildfarm members
are not.
M src/backend/utils/adt/jsonbsubs.c
Throw error when assigning jsonb scalar instead of a composite object
commit : aa6e46daf5304e8d9e66fefc1a5bd77622ec6402
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 31 Jan 2021 23:51:06 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 31 Jan 2021 23:51:06 +0300
During the jsonb subscripting assignment, the provided path might assume an
object or an array where the source jsonb has a scalar value. Initial
subscripting assignment logic will skip such an update operation with no
message shown. This commit makes it throw an error to indicate this type
of situation.
Discussion: https://postgr.es/m/CA%2Bq6zcV8qvGcDXurwwgUbwACV86Th7G80pnubg42e-p9gsSf%3Dg%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcX3mdxGCgdThzuySwH-ApyHHM-G4oB1R0fn0j2hZqqkLQ%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcVDuGBv%3DM0FqBYX8DPebS3F_0KQ6OVFobGJPM507_SZ_w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcVovR%2BXY4mfk-7oNk-rF91gH0PebnNfuUjuuDsyHjOcVA%40mail.gmail.com
Author: Dmitry Dolgov
Reviewed-by: Tom Lane, Arthur Zakirov, Pavel Stehule, Dian M Fay
Reviewed-by: Andrew Dunstan, Chapman Flack, Merlin Moncure, Peter Geoghegan
Reviewed-by: Alvaro Herrera, Jim Nasby, Josh Berkus, Victor Wagner
Reviewed-by: Aleksander Alekseev, Robert Haas, Oleg Bartunov
M doc/src/sgml/json.sgml
M src/backend/utils/adt/jsonfuncs.c
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql
Filling array gaps during jsonb subscripting
commit : 81fcc72e66222357f9bccce3eeda62eb2cb29849
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 31 Jan 2021 23:51:01 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 31 Jan 2021 23:51:01 +0300
This commit introduces two new flags for jsonb assignment:
* JB_PATH_FILL_GAPS: Appending array elements on the specified position, gaps
are filled with nulls (similar to the JavaScript behavior). This mode also
instructs to create the whole path in a jsonb object if some part of the
path (more than just the last element) is not present.
* JB_PATH_CONSISTENT_POSITION: Assigning keeps array positions consistent by
preventing prepending of elements.
Both flags are used only in jsonb subscripting assignment.
Initially proposed by Nikita Glukhov based on polymorphic subscripting
patch, but transformed into an independent change.
Discussion: https://postgr.es/m/CA%2Bq6zcV8qvGcDXurwwgUbwACV86Th7G80pnubg42e-p9gsSf%3Dg%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcX3mdxGCgdThzuySwH-ApyHHM-G4oB1R0fn0j2hZqqkLQ%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcVDuGBv%3DM0FqBYX8DPebS3F_0KQ6OVFobGJPM507_SZ_w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcVovR%2BXY4mfk-7oNk-rF91gH0PebnNfuUjuuDsyHjOcVA%40mail.gmail.com
Author: Dmitry Dolgov
Reviewed-by: Tom Lane, Arthur Zakirov, Pavel Stehule, Dian M Fay
Reviewed-by: Andrew Dunstan, Chapman Flack, Merlin Moncure, Peter Geoghegan
Reviewed-by: Alvaro Herrera, Jim Nasby, Josh Berkus, Victor Wagner
Reviewed-by: Aleksander Alekseev, Robert Haas, Oleg Bartunov
M doc/src/sgml/json.sgml
M src/backend/utils/adt/jsonfuncs.c
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql
Implementation of subscripting for jsonb
commit : 676887a3b0b8e3c0348ac3f82ab0d16e9a24bd43
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 31 Jan 2021 23:50:40 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 31 Jan 2021 23:50:40 +0300
Subscripting for jsonb does not support slices, does not have a limit for the
number of subscripts, and an assignment expects a replace value to have jsonb
type. There is also one functional difference between assignment via
subscripting and assignment via jsonb_set(). When an original jsonb container
is NULL, the subscripting replaces it with an empty jsonb and proceeds with
an assignment.
For the sake of code reuse, we rearrange some parts of jsonb functionality
to allow the usage of the same functions for jsonb_set and assign subscripting
operation.
The original idea belongs to Oleg Bartunov.
Catversion is bumped.
Discussion: https://postgr.es/m/CA%2Bq6zcV8qvGcDXurwwgUbwACV86Th7G80pnubg42e-p9gsSf%3Dg%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcX3mdxGCgdThzuySwH-ApyHHM-G4oB1R0fn0j2hZqqkLQ%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcVDuGBv%3DM0FqBYX8DPebS3F_0KQ6OVFobGJPM507_SZ_w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcVovR%2BXY4mfk-7oNk-rF91gH0PebnNfuUjuuDsyHjOcVA%40mail.gmail.com
Author: Dmitry Dolgov
Reviewed-by: Tom Lane, Arthur Zakirov, Pavel Stehule, Dian M Fay
Reviewed-by: Andrew Dunstan, Chapman Flack, Merlin Moncure, Peter Geoghegan
Reviewed-by: Alvaro Herrera, Jim Nasby, Josh Berkus, Victor Wagner
Reviewed-by: Aleksander Alekseev, Robert Haas, Oleg Bartunov
M doc/src/sgml/json.sgml
M src/backend/utils/adt/Makefile
M src/backend/utils/adt/jsonb_util.c
A src/backend/utils/adt/jsonbsubs.c
M src/backend/utils/adt/jsonfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_type.dat
M src/include/utils/jsonb.h
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql
M src/tools/pgindent/typedefs.list
Remove unused _bt_delitems_delete() argument.
commit : dc43492e46c7145a476cb8ca6200fc8eefe673ef
author : Peter Geoghegan <pg@bowt.ie>
date : Sun, 31 Jan 2021 10:10:55 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Sun, 31 Jan 2021 10:10:55 -0800
The latestRemovedXid values used by nbtree deletion operations are
determined by _bt_delitems_delete()'s caller, so there is no reason to
pass a separate heapRel argument.
Oversight in commit d168b666823.
M src/backend/access/nbtree/nbtpage.c
Fix parsing of complex morphs to tsquery
commit : 0c4f355c6a5fd437f71349f2f3d5d491382572b7
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 31 Jan 2021 20:14:29 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 31 Jan 2021 20:14:29 +0300
When to_tsquery() or websearch_to_tsquery() meet a complex morph containing
multiple words residing adjacent position, these words are connected
with OP_AND operator. That leads to surprising results. For instace,
both websearch_to_tsquery('"pg_class pg"') and to_tsquery('pg_class <-> pg')
produce '( pg & class ) <-> pg' tsquery. This tsquery requires
'pg' and 'class' words to reside on the same position and doesn't match
to to_tsvector('pg_class pg'). It appears to be ridiculous behavior, which
needs to be fixed.
This commit makes to_tsquery() or websearch_to_tsquery() connect words
residing adjacent position with OP_PHRASE. Therefore, now those words are
normally chained with other OP_PHRASE operator. The examples of above now
produces 'pg <-> class <-> pg' tsquery, which matches to
to_tsvector('pg_class pg').
Another effect of this commit is that complex morph word positions now need to
match the tsvector even if there is no surrounding OP_PHRASE. This behavior
change generally looks like an improvement but making this commit not
backpatchable.
Reported-by: Barry Pederson
Bug: #16592
Discussion: https://postgr.es/m/16592-70b110ff9731c07d@postgresql.org
Discussion: https://postgr.es/m/CAPpHfdv0EzVhf6CWfB1_TTZqXV_2Sn-jSY3zSd7ePH%3D-%2B1V2DQ%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Tom Lane, Neil Chen
M src/backend/tsearch/to_tsany.c
M src/test/regress/expected/tsearch.out
M src/test/regress/sql/tsearch.sql
Add primary keys and unique constraints to system catalogs
commit : dfb75e478cacb33d277432e0df5e2f9a2a2b52d9
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 30 Jan 2021 19:14:31 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 30 Jan 2021 19:14:31 +0100
For those system catalogs that have a unique indexes, make a primary
key and unique constraint, using ALTER TABLE ... PRIMARY KEY/UNIQUE
USING INDEX.
This can be helpful for GUI tools that look for a primary key, and it
might in the future allow declaring foreign keys, for making schema
diagrams.
The constraint creation statements are automatically created by
genbki.pl from DECLARE_UNIQUE_INDEX directives. To specify which one
of the available unique indexes is the primary key, use the new
directive DECLARE_UNIQUE_INDEX_PKEY instead. By convention, we
usually make a catalog's OID column its primary key, if it has one.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/dc5f44d9-5ec1-a596-0251-dadadcdede98@2ndquadrant.com
M src/backend/catalog/.gitignore
M src/backend/catalog/Catalog.pm
M src/backend/catalog/Makefile
M src/backend/catalog/genbki.pl
M src/bin/initdb/initdb.c
M src/include/catalog/genbki.h
M src/include/catalog/pg_aggregate.h
M src/include/catalog/pg_am.h
M src/include/catalog/pg_amop.h
M src/include/catalog/pg_amproc.h
M src/include/catalog/pg_attrdef.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_auth_members.h
M src/include/catalog/pg_authid.h
M src/include/catalog/pg_cast.h
M src/include/catalog/pg_class.h
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_conversion.h
M src/include/catalog/pg_database.h
M src/include/catalog/pg_db_role_setting.h
M src/include/catalog/pg_default_acl.h
M src/include/catalog/pg_description.h
M src/include/catalog/pg_enum.h
M src/include/catalog/pg_event_trigger.h
M src/include/catalog/pg_extension.h
M src/include/catalog/pg_foreign_data_wrapper.h
M src/include/catalog/pg_foreign_server.h
M src/include/catalog/pg_foreign_table.h
M src/include/catalog/pg_index.h
M src/include/catalog/pg_inherits.h
M src/include/catalog/pg_init_privs.h
M src/include/catalog/pg_language.h
M src/include/catalog/pg_largeobject.h
M src/include/catalog/pg_largeobject_metadata.h
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_opclass.h
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_opfamily.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_policy.h
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_publication.h
M src/include/catalog/pg_publication_rel.h
M src/include/catalog/pg_range.h
M src/include/catalog/pg_replication_origin.h
M src/include/catalog/pg_rewrite.h
M src/include/catalog/pg_seclabel.h
M src/include/catalog/pg_sequence.h
M src/include/catalog/pg_shdescription.h
M src/include/catalog/pg_shseclabel.h
M src/include/catalog/pg_statistic.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/catalog/pg_subscription.h
M src/include/catalog/pg_subscription_rel.h
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_transform.h
M src/include/catalog/pg_trigger.h
M src/include/catalog/pg_ts_config.h
M src/include/catalog/pg_ts_config_map.h
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_ts_parser.h
M src/include/catalog/pg_ts_template.h
M src/include/catalog/pg_type.h
M src/include/catalog/pg_user_mapping.h
M src/test/modules/unsafe_tests/expected/alter_system_table.out
M src/test/modules/unsafe_tests/sql/alter_system_table.sql
M src/test/regress/expected/misc_sanity.out
M src/test/regress/sql/misc_sanity.sql
M src/tools/msvc/clean.bat
doc: Clarify status of SELECT INTO on reference page
commit : 65330622441d7ee08f768c4326825ae903f2595a
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 30 Jan 2021 11:05:15 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 30 Jan 2021 11:05:15 +0100
The documentation as well as source code comments weren't entirely
clear whether SELECT INTO was truly deprecated (thus in theory
destined to be removed eventually), or just a less recommended
variant. After discussion, it appears that other implementations also
use SELECT INTO in direct SQL in a way similar to PostgreSQL, so it
seems worth keeping it for compatibility. Update the language in the
documentation to that effect.
Discussion: https://www.postgresql.org/message-id/flat/96dc0df3-e13a-a85d-d045-d6e2c85218da%40enterprisedb.com
M doc/src/sgml/ref/select_into.sgml
Allow GRANTED BY clause in normal GRANT and REVOKE statements
commit : 6aaaa76bb47db11cd6f567eafa3d1ee81ca59556
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 30 Jan 2021 09:41:44 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 30 Jan 2021 09:41:44 +0100
The SQL standard allows a GRANTED BY clause on GRANT and
REVOKE (privilege) statements that can specify CURRENT_USER or
CURRENT_ROLE. In PostgreSQL, both of these are the default behavior.
Since we already have all the parsing support for this for the
GRANT (role) statement, we might as well add basic support for this
for the privilege variant as well. This allows us to check off SQL
feature T332. In the future, perhaps more interesting things could be
done with this, too.
Reviewed-by: Simon Riggs <simon@2ndquadrant.com>
Discussion: https://www.postgresql.org/message-id/flat/f2feac44-b4c5-f38f-3699-2851d6a76dc9@2ndquadrant.com
M doc/src/sgml/ref/grant.sgml
M doc/src/sgml/ref/revoke.sgml
M src/backend/catalog/aclchk.c
M src/backend/catalog/sql_features.txt
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/parser/gram.y
M src/include/nodes/parsenodes.h
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Revive "snapshot too old" with wal_level=minimal and SET TABLESPACE.
commit : 7da83415e5bc01bba0093f34f4f612b70c70b678
author : Noah Misch <noah@leadboat.com>
date : Sat, 30 Jan 2021 00:12:18 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 30 Jan 2021 00:12:18 -0800
Given a permanent relation rewritten in the current transaction, the
old_snapshot_threshold mechanism assumed the relation had never been
subject to early pruning. Hence, a query could fail to report "snapshot
too old" when the rewrite followed an early truncation. ALTER TABLE SET
TABLESPACE is probably the only rewrite mechanism capable of exposing
this bug. REINDEX sets indcheckxmin, avoiding the problem. CLUSTER has
zeroed page LSNs since before old_snapshot_threshold existed, so
old_snapshot_threshold has never cooperated with it. ALTER TABLE
... SET DATA TYPE makes the table look empty to every past snapshot,
which is strictly worse. Back-patch to v13, where commit
c6b92041d38512a4176ed76ad06f713d2e6c01a8 broke this.
Kyotaro Horiguchi and Noah Misch
Discussion: https://postgr.es/m/20210113.160705.2225256954956139776.horikyota.ntt@gmail.com
M src/backend/utils/time/snapmgr.c
M src/include/utils/snapmgr.h
Fix error with CREATE PUBLICATION, wal_level=minimal, and new tables.
commit : 360bd2321b1ad9c47466bc485ee5eb2f4515372e
author : Noah Misch <noah@leadboat.com>
date : Sat, 30 Jan 2021 00:11:38 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 30 Jan 2021 00:11:38 -0800
CREATE PUBLICATION has failed spuriously when applied to a permanent
relation created or rewritten in the current transaction. Make the same
change to another site having the same semantic intent; the second
instance has no user-visible consequences. Back-patch to v13, where
commit c6b92041d38512a4176ed76ad06f713d2e6c01a8 broke this.
Kyotaro Horiguchi
Discussion: https://postgr.es/m/20210113.160705.2225256954956139776.horikyota.ntt@gmail.com
M src/backend/catalog/pg_publication.c
M src/backend/optimizer/util/plancat.c
M src/test/subscription/t/001_rep_changes.pl
Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions.
commit : 8a54e12a38d1545d249f1402f66c8cde2837d97c
author : Noah Misch <noah@leadboat.com>
date : Sat, 30 Jan 2021 00:00:27 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 30 Jan 2021 00:00:27 -0800
In a cluster having used CREATE INDEX CONCURRENTLY while having enabled
prepared transactions, queries that use the resulting index can silently
fail to find rows. Fix this for future CREATE INDEX CONCURRENTLY by
making it wait for prepared transactions like it waits for ordinary
transactions. This expands the VirtualTransactionId structure domain to
admit prepared transactions. It may be necessary to reindex to recover
from past occurrences. Back-patch to 9.5 (all supported versions).
Andrey Borodin, reviewed (in earlier versions) by Tom Lane and Michael
Paquier.
Discussion: https://postgr.es/m/2E712143-97F7-4890-B470-4A35142ABC82@yandex-team.ru
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lock.c
M src/include/storage/lock.h
M src/test/isolation/Makefile
M src/test/isolation/README
A src/test/isolation/expected/prepared-transactions-cic.out
A src/test/isolation/specs/prepared-transactions-cic.spec
postgres_fdw: Fix tests for CLOBBER_CACHE_ALWAYS.
commit : f77717b2985aa529a185e6988de26b885ca10ddb
author : Fujii Masao <fujii@postgresql.org>
date : Sat, 30 Jan 2021 10:12:22 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Sat, 30 Jan 2021 10:12:22 +0900
The regression tests added in commits 708d165ddb and 411ae64997 caused
buildfarm failures when CLOBBER_CACHE_ALWAYS was enabled.
This commit stabilizes those tests.
The foreign server connections established by postgres_fdw behaves
differently depending on whether CLOBBER_CACHE_ALWAYS is enabled or not.
If it's not enabled, those connections are cached. On the other hand,
if it's enabled, when the connections are established outside transaction
block, they are not cached (i.e., they are immediately closed at the end of
query that established them). So the subsequent postgres_fdw_get_connections()
cannot list those connections and postgres_fdw_disconnect() cannot close them
(because they are already closed).
When the connections are established inside transaction block, they are
cached whether CLOBBER_CACHE_ALWAYS was enabled or not. But if it's enabled,
they are immediately marked as invalid, otherwise not. This causes the
subsequent postgres_fdw_get_connections() to return different result in
"valid" column depending on whether CLOBBER_CACHE_ALWAYS was enabled or not.
This commit prevents the above differences of behavior from
affecting the regression tests.
Per buildfarm failure on trilobite.
Original patch by Bharath Rupireddy. I (Fujii Masao) extracted
the regression test fix from that and revised it a bit.
Reported-by: Tom Lane
Author: Bharath Rupireddy
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/2688508.1611865371@sss.pgh.pa.us
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Doc: improve cross-references for SET/SHOW.
commit : f743a2bbd4e511ad58b6ce01f81841e5e1611474
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 29 Jan 2021 10:46:14 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 29 Jan 2021 10:46:14 -0500
The corresponding functions set_config and current_setting were
mostly not hyperlinked. Clarify their descriptions a tad, too.
Discussion: https://postgr.es/m/161183356250.4077.687338658090583892@wrigleys.postgresql.org
M doc/src/sgml/config.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/set.sgml
M doc/src/sgml/ref/show.sgml
Document behavior of the .** jsonpath accessor in the lax mode
commit : b41645460af563cfd4e4f57f354058cf69ef3b14
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 29 Jan 2021 15:27:55 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Fri, 29 Jan 2021 15:27:55 +0300
When the .** jsonpath accessor handles the array, it selects both array and
each of its elements. When using lax mode, subsequent accessors automatically
unwrap arrays. So, the content of each array element may be selected twice.
Even though this behavior is counterintuitive, it's correct because everything
works as designed. This commit documents it.
Backpatch to 12 where the jsonpath language was introduced.
Reported-by: Thomas Kellerer
Bug: #16828
Discussion: https://postgr.es/m/16828-2b0229babfad2d8c%40postgresql.org
Discussion: https://postgr.es/m/CAPpHfdtS-nNidT%3DEqZbAYOPcnNOWh_sd6skVdu2CAQUGdvpT8Q%40mail.gmail.com
Author: Alexandex Korotkov, revised by Tom Lane
Reviewed-by: Alvaro Herrera, Thomas Kellerer, Tom Lane
Backpatch-through: 12
M doc/src/sgml/func.sgml
Fix typo
commit : 2592be8be5aed6448a1e2b386fffd75ad69eb6c5
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 29 Jan 2021 09:43:21 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 29 Jan 2021 09:43:21 +0100
M src/bin/pg_dump/pg_dump.c
doc: Improve wording of section for repslot statistics
commit : 2a5862f013647dd185de3e1b496fd4f614d12f0e
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 29 Jan 2021 14:24:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 29 Jan 2021 14:24:49 +0900
This documentation has been added in 9868167, so no backpatch is
needed.
Author: Justin Pryzby, Michael Paquier
Discussion: https://postgr.es/m/20201222041153.GK30237@telsasoft.com
M doc/src/sgml/monitoring.sgml
Adjust comments of CheckRelationTableSpaceMove() and SetRelationTableSpace()
commit : 24843297a96d7be16cc3f4b090aacfc6e5e6839e
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 29 Jan 2021 13:59:18 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 29 Jan 2021 13:59:18 +0900
4c9c359, that introduced those two functions, has been overoptimistic on
the point that only ShareUpdateExclusiveLock would be required when
moving a relation to a new tablespace. AccessExclusiveLock is a
requirement, but ShareUpdateExclusiveLock may be used under specific
conditions like REINDEX CONCURRENTLY where waits on past transactions
make the operation safe even with a lower-level lock. The current code
does only the former, so update the existing comments to reflect that.
Once a REINDEX (TABLESPACE) is introduced, those comments would require
an extra refresh to mention their new use case.
While on it, fix an incorrect variable name.
Per discussion with Álvaro Herrera.
Discussion: https://postgr.es/m/20210127140741.GA14174@alvherre.pgsql
M src/backend/commands/tablecmds.c
Remove documentation of waiting restore_command.
commit : 5c6d184213bb9feac0800eee1a16769bec9e8d58
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 29 Jan 2021 14:16:29 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 29 Jan 2021 14:16:29 +1300
Following the removal of pg_standby, also remove the documentation
section that describes how to write your own "waiting restore_command"
along the same lines.
Discussion: https://postgr.es/m/20201029024412.GP5380%40telsasoft.com
Reviewed-by: Michael Paquier <michael@paquier.xyz>
M doc/src/sgml/high-availability.sgml
Retire pg_standby.
commit : 514b411a2b5226167add9ab139d3a96dbe98035d
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 29 Jan 2021 13:21:53 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 29 Jan 2021 13:21:53 +1300
pg_standby was useful more than a decade ago, but now it is obsolete.
It has been proposed that we retire it many times. Now seems like a
good time to finally do it, because "waiting restore commands"
are incompatible with a proposed recovery prefetching feature.
Discussion: https://postgr.es/m/20201029024412.GP5380%40telsasoft.com
Author: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
M contrib/Makefile
D contrib/pg_standby/.gitignore
D contrib/pg_standby/Makefile
D contrib/pg_standby/pg_standby.c
M doc/src/sgml/contrib.sgml
M doc/src/sgml/filelist.sgml
M doc/src/sgml/high-availability.sgml
D doc/src/sgml/pgstandby.sgml
M doc/src/sgml/ref/pgarchivecleanup.sgml
M src/backend/access/transam/xlog.c
M src/tools/msvc/Mkvcbuild.pm
Silence another gcc 11 warning.
commit : 1046dbedde2fc3fe55f007ff3255ab65ab98f858
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Jan 2021 17:18:23 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Jan 2021 17:18:23 -0500
Per buildfarm and local experimentation, bleeding-edge gcc isn't
convinced that the MemSet in reorder_function_arguments() is safe.
Shut it up by adding an explicit check that pronargs isn't negative,
and by changing MemSet to memset. (It appears that either change is
enough to quiet the warning at -O2, but let's do both to be sure.)
M src/backend/optimizer/util/clauses.c
Remove bogus restriction from BEFORE UPDATE triggers
commit : 6f5c8a8ec23f8ab00da4d2b77bfc8af2a578c4d3
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 28 Jan 2021 16:56:07 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 28 Jan 2021 16:56:07 -0300
In trying to protect the user from inconsistent behavior, commit
487e9861d0cf "Enable BEFORE row-level triggers for partitioned tables"
tried to prevent BEFORE UPDATE FOR EACH ROW triggers from moving the row
from one partition to another. However, it turns out that the
restriction is wrong in two ways: first, it fails spuriously, preventing
valid situations from working, as in bug #16794; and second, they don't
protect from any misbehavior, because tuple routing would cope anyway.
Fix by removing that restriction.
We keep the same restriction on BEFORE INSERT FOR EACH ROW triggers,
though. It is valid and useful there. In the future we could remove it
by having tuple reroute work for inserts as it does for updates.
Backpatch to 13.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reported-by: Phillip Menke <pg@pmenke.de>
Discussion: https://postgr.es/m/16794-350a655580fbb9ae@postgresql.org
M doc/src/sgml/ddl.sgml
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Fix hash partition pruning with asymmetric partition sets.
commit : 1d9351a87c9a9e82c7091aab03d9299982670ce0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Jan 2021 13:41:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Jan 2021 13:41:55 -0500
perform_pruning_combine_step() was not taught about the number of
partition indexes used in hash partitioning; more embarrassingly,
get_matching_hash_bounds() also had it wrong. These errors are masked
in the common case where all the partitions have the same modulus
and no partition is missing. However, with missing or unequal-size
partitions, we could erroneously prune some partitions that need
to be scanned, leading to silently wrong query answers.
While a minimal-footprint fix for this could be to export
get_partition_bound_num_indexes and make the incorrect functions use it,
I'm of the opinion that that function should never have existed in the
first place. It's not reasonable data structure design that
PartitionBoundInfoData lacks any explicit record of the length of
its indexes[] array. Perhaps that was all right when it could always
be assumed equal to ndatums, but something should have been done about
it as soon as that stopped being true. Putting in an explicit
"nindexes" field makes both partition_bounds_equal() and
partition_bounds_copy() simpler, safer, and faster than before,
and removes explicit knowledge of the number-of-partition-indexes
rules from some other places too.
This change also makes get_hash_partition_greatest_modulus obsolete.
I left that in place in case any external code uses it, but no core
code does anymore.
Per bug #16840 from Michał Albrycht. Back-patch to v11 where the
hash partitioning code came in. (In the back branches, add the new
field at the end of PartitionBoundInfoData to minimize ABI risks.)
Discussion: https://postgr.es/m/16840-571a22976f829ad4@postgresql.org
M src/backend/executor/execPartition.c
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partprune.c
M src/include/partitioning/partbounds.h
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Make ecpg's rjulmdy() and rmdyjul() agree with their declarations.
commit : 1b242f42ba1e732b202f35265ab1a1614ce7d859
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Jan 2021 11:17:13 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Jan 2021 11:17:13 -0500
We had "short *mdy" in the extern declarations, but "short mdy[3]"
in the actual function definitions. Per C99 these are equivalent,
but recent versions of gcc have started to issue warnings about
the inconsistency. Clean it up before the warnings get any more
widespread.
Back-patch, in case anyone wants to build older PG versions with
bleeding-edge compilers.
Discussion: https://postgr.es/m/2401575.1611764534@sss.pgh.pa.us
M src/interfaces/ecpg/compatlib/informix.c
pgbench: Remove dead code
commit : 6819b9042fe69154ff3fd2337c5028038d3cfab4
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 28 Jan 2021 12:50:40 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 28 Jan 2021 12:50:40 -0300
doConnect() never returns connections in state CONNECTION_BAD, so
checking for that is pointless. Remove the code that does.
This code has been dead since ba708ea3dc84, 20 years ago.
Discussion: https://postgr.es/m/20210126195224.GA20361@alvherre.pgsql
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
M src/bin/pgbench/pgbench.c
Remove gratuitous uses of deprecated SELECT INTO
commit : b034ef9b376dbe712caa076541d6a750f37d85ce
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 28 Jan 2021 14:01:41 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 28 Jan 2021 14:01:41 +0100
CREATE TABLE AS has been preferred over SELECT INTO (outside of ecpg
and PL/pgSQL) for a long time. There were still a few uses of SELECT
INTO in tests and documentation, some old, some more recent. This
changes them to CREATE TABLE AS. Some occurrences in the tests remain
where they are specifically testing SELECT INTO parsing or similar.
Discussion: https://www.postgresql.org/message-id/flat/96dc0df3-e13a-a85d-d045-d6e2c85218da%40enterprisedb.com
M contrib/sepgsql/expected/label.out
M contrib/sepgsql/sql/label.sql
M doc/src/sgml/hstore.sgml
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_checksums/t/002_actions.pl
M src/test/regress/expected/create_index.out
M src/test/regress/expected/create_misc.out
M src/test/regress/expected/random.out
M src/test/regress/expected/select_implicit.out
M src/test/regress/expected/select_implicit_1.out
M src/test/regress/expected/select_implicit_2.out
M src/test/regress/sql/create_index.sql
M src/test/regress/sql/create_misc.sql
M src/test/regress/sql/random.sql
M src/test/regress/sql/select_implicit.sql
Add direct conversion routines between EUC_TW and Big5.
commit : 6c5576075b0f93f2235ac8a82290fe3b6e82300d
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 28 Jan 2021 14:53:03 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 28 Jan 2021 14:53:03 +0200
Conversions between EUC_TW and Big5 were previously implemented by
converting the whole input to MIC first, and then from MIC to the target
encoding. Implement functions to convert directly between the two.
The reason to do this now is that I'm working on a patch that will change
the conversion function signature so that if the input is invalid, we
convert as much as we can and return the number of bytes successfully
converted. That's not possible if we use an intermediary format, because
if an error happens in the intermediary -> final conversion, we lose track
of the location of the invalid character in the original input. Avoiding
the intermediate step makes the conversions faster, too.
Reviewed-by: John Naylor
Discussion: https://www.postgresql.org/message-id/b9e3167f-f84b-7aa4-5738-be578a4db924%40iki.fi
M src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
Add mbverifystr() functions specific to each encoding.
commit : b80e10638e36b9d2f0b39170c613837af2ca2aac
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 28 Jan 2021 14:40:07 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 28 Jan 2021 14:40:07 +0200
This makes pg_verify_mbstr() function faster, by allowing more efficient
encoding-specific implementations. All the implementations included in
this commit are pretty naive, they just call the same encoding-specific
verifychar functions that were used previously, but that already gives a
performance boost because the tight character-at-a-time loop is simpler.
Reviewed-by: John Naylor
Discussion: https://www.postgresql.org/message-id/e7861509-3960-538a-9025-b75a61188e01@iki.fi
M src/backend/commands/extension.c
M src/backend/utils/mb/conv.c
M src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
M src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
M src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
M src/backend/utils/mb/mbutils.c
M src/common/wchar.c
M src/include/mb/pg_wchar.h
Don't add bailout adjustment for non-strict deserialize calls.
commit : a3367aa3c4552170004c92369681730d85e384c6
author : Andrew Gierth <rhodiumtoad@postgresql.org>
date : Thu, 28 Jan 2021 10:53:10 +0000
committer: Andrew Gierth <rhodiumtoad@postgresql.org>
date : Thu, 28 Jan 2021 10:53:10 +0000
When building aggregate expression steps, strict checks need a bailout
jump for when a null value is encountered, so there is a list of steps
that require later adjustment. Adding entries to that list for steps
that aren't actually strict would be harmless, except that there is an
Assert which catches them. This leads to spurious errors on asserts
builds, for data sets that trigger parallel aggregation of an
aggregate with a non-strict deserialization function (no such
aggregates exist in the core system).
Repair by not adding the adjustment entry when it's not needed.
Backpatch back to 11 where the code was introduced.
Per a report from Darafei (Komzpa) of the PostGIS project; analysis
and patch by me.
Discussion: https://postgr.es/m/87mty7peb3.fsf@news-spur.riddles.org.uk
M src/backend/executor/execExpr.c
Fix crash of pg_stat_statements_info() without library loaded
commit : bca96ddab57c7c64ce442b0f9f35558741c7f8bd
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 28 Jan 2021 16:22:34 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 28 Jan 2021 16:22:34 +0900
Other code paths are already protected against this case, and _PG_init()
warns about that in pg_stat_statements.c. While on it, I have checked
the other extensions of the tree but did not notice any holes.
Oversight in 9fbc3f3.
Author: Jaime Casanova
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAJKUy5gF4=_=qhJ1VX_tSGFfjKHb9BvzhRYWSApJD=Bfwp2SBw@mail.gmail.com
M contrib/pg_stat_statements/pg_stat_statements.c
Refactor SQL functions of SHA-2 in cryptohashfuncs.c
commit : f854c69a5b36ba7aa85bee9e9590c3e517970156
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 28 Jan 2021 16:13:26 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 28 Jan 2021 16:13:26 +0900
The same code pattern was repeated four times when compiling a SHA-2
hash. This refactoring has the advantage to issue a compilation warning
if a new value is added to pg_cryptohash_type, so as anybody doing an
addition in this area would need to consider if support for a new SQL
function is needed or not.
Author: Sehrope Sarkuni, Michael Paquier
Discussion: https://postgr.es/m/YA7DvLRn2xnTgsMc@paquier.xyz
M src/backend/utils/adt/cryptohashfuncs.c
Reduce the default value of vacuum_cost_page_miss.
commit : e19594c5c059d2e071b67d87ae84f569a52d2e32
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 27 Jan 2021 15:11:13 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 27 Jan 2021 15:11:13 -0800
When commit f425b605 introduced cost based vacuum delays back in 2004,
the defaults reflected then-current trends in hardware, as well as
certain historical limitations in PostgreSQL. There have been enormous
improvements in both areas since that time. The cost limit GUC defaults
finally became much more representative of current trends following
commit cbccac37, which decreased autovacuum_vacuum_cost_delay's default
by 10x for PostgreSQL 12 (it went from 20ms to only 2ms).
The relative costs have shifted too. This should also be accounted for
by the defaults. More specifically, the relative importance of avoiding
dirtying pages within VACUUM has greatly increased, primarily due to
main memory capacity scaling and trends in flash storage. Within
Postgres itself, improvements like sequential access during index
vacuuming (at least in nbtree and GiST indexes) have also been
contributing factors.
To reflect all this, decrease the default of vacuum_cost_page_miss to 2.
Since the default of vacuum_cost_page_dirty remains 20, dirtying a page
is now considered 10x "costlier" than a page miss by default.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WzmLPFnkWT8xMjmcsm7YS3+_Qi3iRWAb2+_Bc8UhVyHfuA@mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/utils/init/globals.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
In TrimCLOG(), don't reset XactCtl->shared->latest_page_number.
commit : 69059d3b2f0754c8e661ba479f7121e6631cdf4a
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 27 Jan 2021 15:52:34 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 27 Jan 2021 15:52:34 -0500
Since the CLOG page number is not recorded directly in the checkpoint
record, we have to use ShmemVariableCache->nextXid to figure out the
latest CLOG page number at the start of recovery. However, as recovery
progresses, replay of CLOG/EXTEND records will update our notion of
the latest page number, and we should rely on that being accurate
rather than recomputing the value based on an updated notion of
nextXid. ShmemVariableCache->nextXid is only an approximation
during recovery anyway, whereas CLOG/EXTEND records are an
authoritative representation of how the SLRU has been updated.
Commit 0fcc2decd485a61321a3220d8f76cb108b082009 makes this
simplification possible, as before that change clog_redo() might
have injected a bogus value here, and we'd want to get rid of
that before entering normal running.
Patch by me, reviewed by Heikki Linnakangas.
Discussion: http://postgr.es/m/CA+TgmoZYig9+AQodhF5sRXuKkJ=RgFDugLr3XX_dz_F-p=TwTg@mail.gmail.com
M src/backend/access/transam/clog.c
In clog_redo(), don't set XactCtl->shared->latest_page_number.
commit : 0fcc2decd485a61321a3220d8f76cb108b082009
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 27 Jan 2021 13:07:41 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 27 Jan 2021 13:07:41 -0500
The comment is no longer accurate, and hasn't been entirely accurate
since Hot Standby was introduced. The original idea here was that
StartupCLOG() wouldn't be called until the end of recovery and
therefore this value would be uninitialized when this code is reached,
but Hot Standby made that true only when hot_standby=off, and commit
1f113abdf87cd085dee3927960bb4f70442b7250 means that this value is now
always initialized before replay even starts.
The original purpose of this code was to bypass the sanity check
in SimpleLruTruncate(), which will no longer occur: now, if something
is wrong, that sanity check might trip during recovery. That's
probably a good thing, because in the current code base
latest_page_number should always be initialized and therefore we
expect that the sanity check should pass. If it doesn't, something
has gone wrong, and complaining about it is appropriate.
Patch by me, reviewed by Heikki Linnakangas.
Discussion: http://postgr.es/m/CA+TgmoZYig9+AQodhF5sRXuKkJ=RgFDugLr3XX_dz_F-p=TwTg@mail.gmail.com
M src/backend/access/transam/clog.c
Doc: improve documentation for UNNEST().
commit : 662affcfe9e816584e3d8602b3b4005236931bbb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 27 Jan 2021 12:50:17 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 27 Jan 2021 12:50:17 -0500
Per a user question, spell out that UNNEST() returns array elements
in storage order; also provide an example to clarify the behavior for
multi-dimensional arrays.
While here, also clarify the SELECT reference page's description of
WITH ORDINALITY. These details were already given in 7.2.1.4, but
a reference page should not omit details.
Back-patch to v13; there's not room in the table in older versions.
Discussion: https://postgr.es/m/FF1FB31F-0507-4F18-9559-2DE6E07E3B43@gmail.com
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/select.sgml
Move StartupCLOG() calls to just after we initialize ShmemVariableCache.
commit : 1f113abdf87cd085dee3927960bb4f70442b7250
author : Robert Haas <rhaas@postgresql.org>
date : Wed, 27 Jan 2021 12:20:46 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Wed, 27 Jan 2021 12:20:46 -0500
Previously, the hot_standby=off code path did this at end of recovery,
while the hot_standby=on code path did it at the beginning of recovery.
It's better to do this in only one place because (a) it's simpler,
(b) StartupCLOG() is trivial so trying to postpone the work isn't
useful, and (c) this will make it possible to simplify some other
logic.
Patch by me, reviewed by Heikki Linnakangas.
Discussion: http://postgr.es/m/CA+TgmoZYig9+AQodhF5sRXuKkJ=RgFDugLr3XX_dz_F-p=TwTg@mail.gmail.com
M src/backend/access/transam/xlog.c
Fix GiST index deletion assert issue.
commit : e42b3c3bd6a9c6233ac4c8a2e9b040367ba2f97c
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 26 Jan 2021 23:24:37 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 26 Jan 2021 23:24:37 -0800
Avoid calling heap_index_delete_tuples() with an empty deltids array to
avoid an assertion failure.
This issue was arguably an oversight in commit b5f58cf2, though the
failing assert itself was added by my recent commit d168b666. No
backpatch, though, since the oversight is harmless in the back branches.
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Jaime Casanova <jcasanov@systemguards.com.ec>
Discussion: https://postgr.es/m/CAJKUy5jscES84n3puE=sYngyF+zpb4wv8UMtuLnLPv5z=6yyNw@mail.gmail.com
M src/backend/access/gist/gist.c
M src/backend/access/index/genam.c
doc: Remove reference to views for TRUNCATE privilege
commit : 32bef758296142ce0fff944c3478d63fdefdaf33
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 27 Jan 2021 13:40:33 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 27 Jan 2021 13:40:33 +0900
The page about privilege rights mentioned that TRUNCATE could be applied
to views or even other relation types. This is confusing as this
command can be used only on tables and on partitioned tables.
Oversight in afc4a78.
Reported-by: Harisai Hari
Reviewed-by: Laurenz Albe
Discussion: https://postgr.es/m/161157636877.14625.15340884663716426087@wrigleys.postgresql.org
Backpatch-through: 12
M doc/src/sgml/ddl.sgml
Refactor code in tablecmds.c to check and process tablespace moves
commit : 4c9c359d38ff1e2de388eedd860785be6a49201c
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 27 Jan 2021 11:54:16 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 27 Jan 2021 11:54:16 +0900
Two code paths of tablecmds.c (for relations with storage and without
storage) use the same logic to check if the move of a relation to a
new tablespace is allowed or not and to update pg_class.reltablespace
and pg_class.relfilenode.
A potential TABLESPACE clause for REINDEX, CLUSTER and VACUUM FULL needs
similar checks to make sure that nothing is moved around in illegal ways
(no mapped relations, shared relations only in pg_global, no move of
temp tables owned by other backends).
This reorganizes the existing code of ALTER TABLE so as all this logic
is controlled by two new routines that can be reused for the other
commands able to move relations across tablespaces, limiting the number
of code paths in need of the same protections. This also removes some
code that was duplicated for tables with and without storage for ALTER
TABLE.
Author: Alexey Kondratov, Michael Paquier
Discussion: https://postgr.es/m/YA+9mAMWYLXJMVPL@paquier.xyz
M src/backend/commands/tablecmds.c
M src/include/commands/tablecmds.h
Rethink recently-added SPI interfaces.
commit : d5a83d79c9f9b660a6a5a77afafe146d3c8c6f46
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 26 Jan 2021 16:37:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 26 Jan 2021 16:37:12 -0500
SPI_execute_with_receiver and SPI_cursor_parse_open_with_paramlist are
new in v14 (cf. commit 2f48ede08). Before they can get out the door,
let's change their APIs to follow the practice recently established by
SPI_prepare_extended etc: shove all optional arguments into a struct
that callers are supposed to pre-zero. The hope is to allow future
addition of more options without either API breakage or a continuing
proliferation of new SPI entry points. With that in mind, choose
slightly more generic names for them: SPI_execute_extended and
SPI_cursor_parse_open respectively.
Discussion: https://postgr.es/m/CAFj8pRCLPdDAETvR7Po7gC5y_ibkn_-bOzbeJb39WHms01194Q@mail.gmail.com
M doc/src/sgml/spi.sgml
M src/backend/executor/spi.c
M src/include/executor/spi.h
M src/pl/plpgsql/src/pl_exec.c
Suppress compiler warnings from commit ee895a655.
commit : 7292fd8f1c781278021407276474d9188845113d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 26 Jan 2021 13:58:18 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 26 Jan 2021 13:58:18 -0500
For obscure reasons, some buildfarm members are now generating
complaints about plpgsql_call_handler's "retval" variable possibly
being used uninitialized. It seems no less safe than it was before
that commit, but these complaints are (mostly?) new. I trust that
initializing the variable where it's declared will be enough to
shut that up.
I also notice that some compilers are warning about setjmp clobber
of the same variable, which is maybe a bit more defensible. Mark
it volatile to silence that.
Also, rearrange the logic to give procedure_resowner a single
point of initialization, in hopes of silencing some setjmp-clobber
warnings about that. (Marking it volatile would serve too, but
its sibling variables are depending on single assignment, so let's
stick with that method.)
Discussion: https://postgr.es/m/E1l4F1z-0000cN-Lx@gemulon.postgresql.org
M src/pl/plpgsql/src/pl_handler.c
Code review for psql's helpSQL() function.
commit : f76a85000bba2f1b1c926cbbe525e47b246215f1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 26 Jan 2021 13:04:52 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 26 Jan 2021 13:04:52 -0500
The loops to identify word boundaries could access past the end of
the input string. Likely that would never result in an actual
crash, but it makes valgrind unhappy.
The logic to try different numbers of words didn't work when the
input has two words but we only have a match to the first, eg
"\h with select". (We must "continue" the pass loop, not "break".)
The logic to compute nl_count was bizarrely managed, and in at
least two code paths could end up calling PageOutput with
nl_count = 0, resulting in failing to paginate output that should
have been fed to the pager. Also, in v12 and up, the nl_count
calculation hadn't been updated to account for the addition of a URL.
The PQExpBuffer holding the command syntax details wasn't freed,
resulting in a session-lifespan memory leak.
While here, improve some comments, choose a more descriptive name
for a variable, fix inconsistent datatype choice for another variable.
Per bug #16837 from Alexander Lakhin. This code is very old,
so back-patch to all supported branches.
Kyotaro Horiguchi and Tom Lane
Discussion: https://postgr.es/m/16837-479bcd56040c71b3@postgresql.org
M src/bin/psql/help.c
Fix memory leak when deallocating prepared statement in postgres_fdw
commit : 7b4c660466dd8a1d25ca316ac02099a7cf0e5896
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 26 Jan 2021 18:43:01 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 26 Jan 2021 18:43:01 +0900
The leak is minor, so no backpatch is done. Oversight in 21734d2.
Reported-by: Tom Lane
M contrib/postgres_fdw/postgres_fdw.c
postgres_fdw: Fix test failure with -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
commit : 0c3fc09fe359a6dc46f1870ceccf60ec60396bc9
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 26 Jan 2021 17:16:52 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 26 Jan 2021 17:16:52 +0900
The roles created by regression test should have names starting with
"regress_", and the test introduced in commit 411ae64997 did not do that.
Per buildfarm member longfin.
Discussion: https://postgr.es/m/73fc5ae4-3c54-1262-4533-f8c547de2e60@oss.nttdata.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
postgres_fdw: Stabilize regression test for postgres_fdw_disconnect_all().
commit : 6adc5376dca4ef8b7d591c0ee7338cb9ff660216
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 26 Jan 2021 16:36:21 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 26 Jan 2021 16:36:21 +0900
The regression test added in commit 411ae64997 caused buildfarm failures.
The cause of them was that the order of warning messages output in the test
was not stable. To fix this, this commit sets client_min_messages to ERROR
temporarily when performing the test generating those warnings.
Per buildfarm failures.
Discussion: https://postgr.es/m/2147113.1611644754@sss.pgh.pa.us
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
postgres_fdw: Add functions to discard cached connections.
commit : 411ae64997dc3a42d19eda6721c581841ce2cb82
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 26 Jan 2021 03:54:46 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 26 Jan 2021 03:54:46 +0900
This commit introduces two new functions postgres_fdw_disconnect()
and postgres_fdw_disconnect_all(). The former function discards
the cached connections to the specified foreign server. The latter discards
all the cached connections. If the connection is used in the current
transaction, it's not closed and a warning message is emitted.
For example, these functions are useful when users want to explicitly
close the foreign server connections that are no longer necessary and
then to prevent them from eating up the foreign servers connections
capacity.
Author: Bharath Rupireddy, tweaked a bit by Fujii Masao
Reviewed-by: Alexey Kondratov, Zhijie Hou, Zhihong Yu, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVvrp5=AVp2PupEm+nAC8S4buqR3fJMmaCoc7ftT0aD2A@mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw–1.0–1.1.sql
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Improve performance of repeated CALLs within plpgsql procedures.
commit : ee895a655ce4341546facd6f23e3e8f2931b96bf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Jan 2021 22:28:29 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Jan 2021 22:28:29 -0500
This patch essentially is cleaning up technical debt left behind
by the original implementation of plpgsql procedures, particularly
commit d92bc83c4. That patch (or more precisely, follow-on patches
fixing its worst bugs) forced us to re-plan CALL and DO statements
each time through, if we're in a non-atomic context. That wasn't
for any fundamental reason, but just because use of a saved plan
requires having a ResourceOwner to hold a reference count for the
plan, and we had no suitable resowner at hand, nor would the
available APIs support using one if we did. While it's not that
expensive to create a "plan" for CALL/DO, the cycles do add up
in repeated executions.
This patch therefore makes the following API changes:
* GetCachedPlan/ReleaseCachedPlan are modified to let the caller
specify which resowner to use to pin the plan, rather than forcing
use of CurrentResourceOwner.
* spi.c gains a "SPI_execute_plan_extended" entry point that lets
callers say which resowner to use to pin the plan. This borrows the
idea of an options struct from the recently added SPI_prepare_extended,
hopefully allowing future options to be added without more API breaks.
This supersedes SPI_execute_plan_with_paramlist (which I've marked
deprecated) as well as SPI_execute_plan_with_receiver (which is new
in v14, so I just took it out altogether).
* I also took the opportunity to remove the crude hack of letting
plpgsql reach into SPI private data structures to mark SPI plans as
"no_snapshot". It's better to treat that as an option of
SPI_prepare_extended.
Now, when running a non-atomic procedure or DO block that contains
any CALL or DO commands, plpgsql creates a ResourceOwner that
will be used to pin the plans of the CALL/DO commands. (In an
atomic context, we just use CurrentResourceOwner, as before.)
Having done this, we can just save CALL/DO plans normally,
whether or not they are used across transaction boundaries.
This seems to be good for something like 2X speedup of a CALL
of a trivial procedure with a few simple argument expressions.
By restricting the creation of an extra ResourceOwner like this,
there's essentially zero penalty in cases that can't benefit.
Pavel Stehule, with some further hacking by me
Discussion: https://postgr.es/m/CAFj8pRCLPdDAETvR7Po7gC5y_ibkn_-bOzbeJb39WHms01194Q@mail.gmail.com
M doc/src/sgml/spi.sgml
M src/backend/commands/prepare.c
M src/backend/executor/spi.c
M src/backend/tcop/postgres.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/mmgr/portalmem.c
M src/backend/utils/resowner/resowner.c
M src/include/executor/spi.h
M src/include/executor/spi_priv.h
M src/include/utils/plancache.h
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/pl_handler.c
M src/pl/plpgsql/src/plpgsql.h
Fix two typos in snapbuild.c.
commit : 55ef8555f0c1207bac25050e7297bbb969c84233
author : Andres Freund <andres@anarazel.de>
date : Mon, 25 Jan 2021 12:15:10 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 25 Jan 2021 12:15:10 -0800
Reported-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/c94be044-818f-15e3-1ad3-7a7ae2dfed0a@iki.fi
M src/backend/replication/logical/snapbuild.c
Don't clobber the calling user's credentials cache in Kerberos test.
commit : 881933f194221abcce07fb134ebe8685e5bb58dd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Jan 2021 14:53:13 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Jan 2021 14:53:13 -0500
Embarrassing oversight in this test script, which fortunately is not
run by default.
Report and patch by Jacob Champion.
Discussion: https://postgr.es/m/1fcb175bafef6560f47a8c31229fa7c938486b8d.camel@vmware.com
M src/test/kerberos/t/001_auth.pl
Fix broken ruleutils support for function TRANSFORM clauses.
commit : 07d46fceb4254b00e79f3d06419cbae13b0ecb5a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Jan 2021 13:03:11 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Jan 2021 13:03:11 -0500
I chanced to notice that this dumped core due to a faulty Assert.
To add insult to injury, the output has been misformatted since v11.
Obviously we need some regression testing here.
Discussion: https://postgr.es/m/d1cc628c-3953-4209-957b-29427acc38c8@www.fastmail.com
M contrib/bool_plperl/expected/bool_plperl.out
M contrib/bool_plperl/expected/bool_plperlu.out
M contrib/bool_plperl/sql/bool_plperl.sql
M contrib/bool_plperl/sql/bool_plperlu.sql
M contrib/hstore_plpython/expected/hstore_plpython.out
M contrib/hstore_plpython/sql/hstore_plpython.sql
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/fmgr/funcapi.c
Remove CheckpointLock.
commit : d18e75664a2fda2e4d5cc433d68e37fc0e9499f2
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 25 Jan 2021 12:34:00 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 25 Jan 2021 12:34:00 -0500
Up until now, we've held this lock when performing a checkpoint or
restartpoint, but commit 076a055acf3c55314de267c62b03191586d79cf6 back
in 2004 and commit 7e48b77b1cebb9a43f9fdd6b17128a0ba36132f9 from 2009,
taken together, have removed all need for this. In the present code,
there's only ever one process entitled to attempt a checkpoint: either
the checkpointer, during normal operation, or the postmaster, during
single-user operation. So, we don't need the lock.
One possible concern in making this change is that it means that
a substantial amount of code where HOLD_INTERRUPTS() was previously
in effect due to the preceding LWLockAcquire() will now be
running without that. This could mean that ProcessInterrupts()
gets called in places from which it didn't before. However, this
seems unlikely to do very much, because the checkpointer doesn't
have any signal mapped to die(), so it's not clear how,
for example, ProcDiePending = true could happen in the first
place. Similarly with ClientConnectionLost and recovery conflicts.
Also, if there are any such problems, we might want to fix them
rather than reverting this, since running lots of code with
interrupt handling suspended is generally bad.
Patch by me, per an inquiry by Amul Sul. Review by Tom Lane
and Michael Paquier.
Discussion: http://postgr.es/m/CAAJ_b97XnBBfYeSREDJorFsyoD1sHgqnNuCi=02mNQBUMnA=FA@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/access/heap/rewriteheap.c
M src/backend/access/transam/xlog.c
M src/backend/replication/logical/origin.c
M src/backend/storage/lmgr/lwlocknames.txt
Doc: improve documentation of pg_proc.protrftypes.
commit : 951862eda57e5dc8f78c97b3c30fe2032a5562b8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Jan 2021 11:20:17 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Jan 2021 11:20:17 -0500
Add a "references" link pointing to pg_type, as we have for other arrays
of type OIDs. Wordsmith the explanation a bit.
Joel Jacobson, additional editing by me
Discussion: https://postgr.es/m/d1cc628c-3953-4209-957b-29427acc38c8@www.fastmail.com
M doc/src/sgml/catalogs.sgml
Remove duplicate include
commit : ecc4b1318c92594abdf9bd2933fb6feaa9d56229
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 25 Jan 2021 08:55:43 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 25 Jan 2021 08:55:43 +0100
Reported-by: Ashutosh Sharma <ashu.coek88@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAE9k0PkORqHHGKY54-sFyDpP90yAf%2B05Auc4fs9EAn4J%2BuBeUQ%40mail.gmail.com
M src/interfaces/ecpg/test/expected/preproc-define.c
M src/interfaces/ecpg/test/expected/preproc-define.stderr
M src/interfaces/ecpg/test/preproc/define.pgc
Fix hypothetical bug in heap backward scans
commit : 16dfe253e31f75b60e93acc0c2b5bbf19936c074
author : David Rowley <drowley@postgresql.org>
date : Mon, 25 Jan 2021 19:52:18 +1300
committer: David Rowley <drowley@postgresql.org>
date : Mon, 25 Jan 2021 19:52:18 +1300
Both heapgettup() and heapgettup_pagemode() incorrectly set the first page
to scan in a backward scan in which the number of pages to scan was
specified by heap_setscanlimits(). The code incorrectly started the scan
at the end of the relation when startBlk was 0, or otherwise at
startBlk - 1, neither of which is correct when only scanning a subset of
pages.
The fix here checks if heap_setscanlimits() has changed the number of
pages to scan and if so we set the first page to scan as the final page in
the specified range during backward scans.
Proper adjustment of this code was forgotten when heap_setscanlimits() was
added in 7516f5259 back in 9.5. However, practice, nowhere in core code
performs backward scans after having used heap_setscanlimits(), yet, it is
possible an extension uses the heap functions in this way, hence
backpatch.
An upcoming patch does use heap_setscanlimits() with backward scans, so
this must be fixed before that can go in.
Author: David Rowley
Discussion: https://postgr.es/m/CAApHDvpGc9h0_oVD2CtgBcxCS1N-qDYZSeBRnUh+0CWJA9cMaA@mail.gmail.com
Backpatch-through: 9.5, all supported versions
M src/backend/access/heap/heapam.c
Fix ALTER PUBLICATION...DROP TABLE behavior.
commit : 40ab64c1ec1cb9bd73695f519cf66ddbb97d8144
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 25 Jan 2021 07:39:29 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 25 Jan 2021 07:39:29 +0530
Commit 69bd60672 fixed the initialization of streamed transactions for
RelationSyncEntry. It forgot to initialize the publication actions while
invalidating the RelationSyncEntry due to which even though the relation
is dropped from a particular publication we still publish its changes. Fix
it by initializing pubactions when entry got invalidated.
Author: Japin Li and Bharath Rupireddy
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CALj2ACV+0UFpcZs5czYgBpujM9p0Hg1qdOZai_43OU7bqHU_xw@mail.gmail.com
M src/backend/replication/pgoutput/pgoutput.c
M src/test/subscription/t/001_rep_changes.pl
Make storage/standby.h compile standalone again.
commit : a4b03de589c1df0845e9732da203f505f2eedb6d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 24 Jan 2021 18:08:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 24 Jan 2021 18:08:55 -0500
This file has failed headerscheck/cpluspluscheck verification since
commit 0650ff230, as a result of referencing typedef TimestampTz
without including the appropriate header.
M src/include/storage/standby.h
Update time zone data files to tzdata release 2021a.
commit : c7edf4ac246b67073563354c2808c78868cbac36
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 24 Jan 2021 16:29:47 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 24 Jan 2021 16:29:47 -0500
DST law changes in Russia (Volgograd zone) and South Sudan.
Historical corrections for Australia, Bahamas, Belize, Bermuda,
Ghana, Israel, Kenya, Nigeria, Palestine, Seychelles, and Vanuatu.
Notably, the Australia/Currie zone has been corrected to the point
where it is identical to Australia/Hobart.
M src/timezone/Makefile
M src/timezone/data/tzdata.zi
Add a simple test for contrib/auto_explain.
commit : 0c1e8845f28bd07ad381c8b0d6701575d967b88e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 24 Jan 2021 14:59:33 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 24 Jan 2021 14:59:33 -0500
This module formerly had zero test coverage.
Discussion: https://postgr.es/m/1445881.1611441692@sss.pgh.pa.us
A contrib/auto_explain/.gitignore
M contrib/auto_explain/Makefile
A contrib/auto_explain/t/001_auto_explain.pl
Remove make_diff set of tools
commit : 8a337b0ed21c654521da137159b8b9327a5cb971
author : Magnus Hagander <magnus@hagander.net>
date : Sun, 24 Jan 2021 14:19:00 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Sun, 24 Jan 2021 14:19:00 +0100
These are mostly obsoleted by the switch to git, and it's easier to
remove them than to update the incorrect documentation.
Discussion: https://postgr.es/m/CABUevEwmASMn4WRJ6RagBx43sj10ctfMHcMA_-7KA3pDYmwpJw@mail.gmail.com
D src/tools/make_diff/README
D src/tools/make_diff/cporig
D src/tools/make_diff/difforig
D src/tools/make_diff/rmorig
Doc: clean up contrib/pageinspect's GIST function documentation.
commit : 7e57255f6189380d545e1df6a6b38827b213e3da
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 23 Jan 2021 22:40:46 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 23 Jan 2021 22:40:46 -0500
I came to fix the overwidth-PDF-page warnings seen in the buildfarm,
but stayed long enough to copy-edit some nearby text.
M doc/src/sgml/pageinspect.sgml
Fix COPY FREEZE with CLOBBER_CACHE_ALWAYS
commit : 39b66a91bdebb00af71a2c6218412ecfc89a0e13
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 24 Jan 2021 00:24:50 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 24 Jan 2021 00:24:50 +0100
This adds code omitted from commit 7db0cd2145 by accident, which had
two consequences. Firstly, only rows inserted by heap_multi_insert were
frozen as expected when running COPY FREEZE, while heap_insert left
rows unfrozen. That however includes rows in TOAST tables, so a lot of
data might have been left unfrozen. Secondly, page might have been left
partially empty after relcache invalidation.
This addresses both of those issues.
Discussion: https://postgr.es/m/CABOikdN-ptGv0mZntrK2Q8OtfUuAjqaYMGmkdU1dCKFtUxVLrg@mail.gmail.com
M src/backend/access/heap/heapam.c
M src/backend/access/heap/hio.c
Doc: update example connection-failure messages in the documentation.
commit : 183bbd1b6d4376f1b04c02b7a20b55019f6d84f4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 23 Jan 2021 15:50:51 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 23 Jan 2021 15:50:51 -0500
Now that the dust has more or less settled on 52a10224e and follow-ons,
make sure the examples in the documentation are up-to-date.
M doc/src/sgml/runtime.sgml
M doc/src/sgml/start.sgml
Update ecpg's connect-test1 for connection-failure message changes.
commit : 68d1c339417ea2e8df42e11db57472fcdbfe6a55
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 23 Jan 2021 15:08:39 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 23 Jan 2021 15:08:39 -0500
I should have updated this in commits 52a10224e and follow-ons,
but I missed it because it's not run by default, and none of the
buildfarm runs it either. Maybe we should try to improve that
situation.
Discussion: https://postgr.es/m/CAH2-Wz=j9SRW=s5BV4-3k+=tr4N3A03in+gTuVA09vNF+-iHjA@mail.gmail.com
M src/interfaces/ecpg/test/expected/connect-test1-minGW32.stderr
M src/interfaces/ecpg/test/expected/connect-test1.stderr
Introduce SHA1 implementations in the cryptohash infrastructure
commit : a8ed6bb8f4cf259b95c1bff5da09a8f4c79dca46
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 23 Jan 2021 11:33:04 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 23 Jan 2021 11:33:04 +0900
With this commit, SHA1 goes through the implementation provided by
OpenSSL via EVP when building the backend with it, and uses as fallback
implementation KAME which was located in pgcrypto and already shaped for
an integration with a set of init, update and final routines.
Structures and routines have been renamed to make things consistent with
the fallback implementations of MD5 and SHA2.
uuid-ossp has used for ages a shortcut with pgcrypto to fetch a copy of
SHA1 if needed. This was built depending on the build options within
./configure, so this cleans up some code and removes the build
dependency between pgcrypto and uuid-ossp.
Note that this will help with the refactoring of HMAC, as pgcrypto
offers the option to use MD5, SHA1 or SHA2, so only the second option
was missing to make that possible.
Author: Michael Paquier
Reviewed-by: Heikki Linnakangas
Discussion: https://postgr.es/m/X9HXKTgrvJvYO7Oh@paquier.xyz
M configure
M configure.ac
M contrib/pgcrypto/Makefile
M contrib/pgcrypto/internal.c
D contrib/pgcrypto/sha1.c
M contrib/uuid-ossp/.gitignore
M contrib/uuid-ossp/Makefile
M contrib/uuid-ossp/uuid-ossp.c
M src/Makefile.global.in
M src/common/Makefile
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
A src/common/sha1.c
R071 contrib/pgcrypto/sha1.h src/common/sha1_int.h
M src/include/common/cryptohash.h
A src/include/common/sha1.h
M src/tools/msvc/Mkvcbuild.pm
M src/tools/pgindent/typedefs.list
Suppress bison warning in ecpg grammar.
commit : 3fc81ce459e1696f7e5e5b3b8229409413bf64b4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Jan 2021 19:25:39 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Jan 2021 19:25:39 -0500
opt_distinct_clause is only used in PLpgSQL_Expr, which ecpg
ignores, so it needs to ignore opt_distinct_clause too.
My oversight in 7cd9765f9; reported by Bruce Momjian.
Discussion: https://postgr.es/m/E1l33wr-0005sJ-9n@gemulon.postgresql.org
M src/interfaces/ecpg/preproc/parse.pl
Doc: improve directions for building on macOS.
commit : 50bebc1ae1804e0ddf86b667122d3f8b22fa19b7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Jan 2021 18:58:25 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Jan 2021 18:58:25 -0500
In light of recent discussions, we should instruct people to
install Apple's command line tools; installing Xcode is secondary.
Also, fix sample command for finding out the default sysroot,
as we now know that the command originally recommended can give
a result that doesn't match your OS version.
Also document the workaround to use if you really don't want
configure to select a sysroot at all.
Discussion: https://postgr.es/m/20210119111625.20435-1-james.hilliard1@gmail.com
M doc/src/sgml/installation.sgml
Avoid redundantly prefixing PQerrorMessage for a connection failure.
commit : 58cd8dca3de0b3c7d378a412eca1f7289b5e4978
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Jan 2021 16:52:31 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Jan 2021 16:52:31 -0500
libpq's error messages for connection failures pretty well stand on
their own, especially since commits 52a10224e/27a48e5a1. Prefixing
them with 'could not connect to database "foo"' or the like is just
redundant, and perhaps even misleading if the specific database name
isn't relevant to the failure. (When it is, we trust that the
backend's error message will include the DB name.) Indeed, psql
hasn't used any such prefix in a long time. So, make all our other
programs and documentation examples agree with psql's practice.
Discussion: https://postgr.es/m/1094524.1611266589@sss.pgh.pa.us
M contrib/oid2name/oid2name.c
M contrib/vacuumlo/vacuumlo.c
M doc/src/sgml/libpq.sgml
M doc/src/sgml/lobj.sgml
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/pg_upgrade/server.c
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/scripts/common.c
M src/interfaces/ecpg/ecpglib/connect.c
M src/interfaces/ecpg/test/expected/connect-test5.stderr
M src/test/examples/testlibpq.c
M src/test/examples/testlibpq2.c
M src/test/examples/testlibpq3.c
M src/test/examples/testlibpq4.c
M src/test/examples/testlo.c
M src/test/examples/testlo64.c
M src/test/isolation/isolationtester.c
M src/tools/findoidjoins/findoidjoins.c
Re-allow DISTINCT in pl/pgsql expressions.
commit : 7cd9765f9bd3397b8d4d0f507021ef848b6d48d2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Jan 2021 16:26:22 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Jan 2021 16:26:22 -0500
I'd omitted this from the grammar in commit c9d529848, figuring that
it wasn't worth supporting. However we already have one complaint,
so it seems that judgment was wrong. It doesn't require a huge
amount of code, so add it back. (I'm still drawing the line at
UNION/INTERSECT/EXCEPT though: those'd require an unreasonable
amount of grammar refactoring, and the single-result-row restriction
makes them near useless anyway.)
Also rethink the documentation: this behavior is a property of
all pl/pgsql expressions, not just assignments.
Discussion: https://postgr.es/m/20210122134106.e94c5cd7@mail.verfriemelt.org
M doc/src/sgml/plpgsql.sgml
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
Doc: remove misleading claim in documentation of PQreset().
commit : ab66645628d38a7047996b294b213f3048f7ef9b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Jan 2021 11:29:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 22 Jan 2021 11:29:43 -0500
This text claimed that the reconnection would occur "to the same
server", but there is no such guarantee in the code, nor would
insisting on that be an improvement.
Back-patch to v10 where multi-host connection strings were added.
Discussion: https://postgr.es/m/1095901.1611268376@sss.pgh.pa.us
M doc/src/sgml/libpq.sgml
Remove reference to ftp servers from documentation
commit : 0a9ae44288d122c12aa0862b032ae892942da452
author : Magnus Hagander <magnus@hagander.net>
date : Fri, 22 Jan 2021 12:49:53 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Fri, 22 Jan 2021 12:49:53 +0100
It's been a long time since we used ftp, but there was a single
reference left in the docs.
Author: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/6880D602-7286-46EC-8A03-14E3248FEC7A@yesql.se
M doc/src/sgml/install-windows.sgml
Remove bogus tracepoint
commit : 09418bed67a7d5d5a8c9c3070f4c9048455a5389
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 22 Jan 2021 11:58:21 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 22 Jan 2021 11:58:21 +0100
Calls to LWLockWaitForVar() fired the TRACE_POSTGRESQL_LWLOCK_ACQUIRE
tracepoint, but LWLockWaitForVar() never actually acquires the LWLock.
(Probably a copy/paste bug in 68a2e52bbaf.) Remove it.
Author: Craig Ringer <craig.ringer@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/CAGRY4nxJo+-HCC2i5H93ttSZ4gZO-FSddCwvkb-qAfQ1zdXd1w@mail.gmail.com
M src/backend/storage/lmgr/lwlock.c
doc: Copy-edit the "Overview of PostgreSQL Internals" chapter
commit : 29ad6595ef7f568ca11dd9219c0d23048bdda513
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 22 Jan 2021 11:10:42 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 22 Jan 2021 11:10:42 +0200
Rephrase a few sentences to be more concise.
Refer to the postmaster process as "postmaster", not "postgres". This
originally said "postmaster process", but was changed to "postgres
process" in commit 5266f221a2, when we merged the "postmaster" and
"postgres" commands, and "postmaster" became just a symlink. That was a
case of overzealous search & replace, because the process is still called
"postmaster".
Author: Erik Rijkers and Jürgen Purtz
Discussion: https://www.postgresql.org/message-id/aa31f359-1168-ded5-53d0-0ed228bfe097%40iki.fi
M doc/src/sgml/arch-dev.sgml
Move SSL information callback earlier to capture more information
commit : af0e79c8f4f4c3c2306855045c0d02a6be6485f0
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 22 Jan 2021 09:26:27 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 22 Jan 2021 09:26:27 +0900
The callback for retrieving state change information during connection
setup was only installed when the connection was mostly set up, and
thus didn't provide much information and missed all the details related
to the handshake.
This also extends the callback with SSL_state_string_long() to print
more information about the state change within the SSL object handled.
While there, fix some comments which were incorrectly referring to the
callback and its previous location in fe-secure.c.
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/232CF476-94E1-42F1-9408-719E2AEC5491@yesql.se
M src/backend/libpq/be-secure-openssl.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/fe-secure.c
Improve new wording of libpq's connection failure messages.
commit : 27a48e5a16ff2227ddf44ee717d9bcd89d22a7aa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 21 Jan 2021 16:10:18 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 21 Jan 2021 16:10:18 -0500
"connection to server so-and-so failed:" seems clearer than the
previous wording "could not connect to so-and-so:" (introduced by
52a10224e), because the latter suggests a network-level connection
failure. We're now prefixing this string to all types of connection
failures, for instance authentication failures; so we need wording
that doesn't imply a low-level error.
Per discussion with Robert Haas.
Discussion: https://postgr.es/m/CA+TgmobssJ6rS22dspWnu-oDxXevGmhMD8VcRBjmj-b9UDqRjw@mail.gmail.com
M src/bin/pg_dump/t/002_pg_dump.pl
M src/interfaces/ecpg/test/expected/connect-test5.stderr
M src/interfaces/ecpg/test/pg_regress_ecpg.c
M src/interfaces/libpq/fe-connect.c
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
commit : 55dc86eca70b1dc18a79c141b3567efed910329d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 21 Jan 2021 15:37:23 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 21 Jan 2021 15:37:23 -0500
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/optimizer/path/clausesel.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/path/pathkeys.c
M src/backend/optimizer/path/tidpath.c
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/optimizer/util/clauses.c
M src/backend/optimizer/util/inherit.c
M src/backend/optimizer/util/orclauses.c
M src/backend/optimizer/util/placeholder.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/optimizer/util/var.c
M src/backend/utils/adt/selfuncs.c
M src/include/optimizer/clauses.h
M src/include/optimizer/optimizer.h
M src/include/optimizer/paths.h
M src/include/optimizer/planmain.h
M src/include/optimizer/restrictinfo.h
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Fix initialization of FDW batching in ExecInitModifyTable
commit : 920f853dc948b98a5dc96580c4ee011a302e33e4
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 21 Jan 2021 03:23:24 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 21 Jan 2021 03:23:24 +0100
ExecInitModifyTable has to initialize batching for all result relations,
not just the first one. Furthermore, when junk filters were necessary,
the pointer pointed past the mtstate->resultRelInfo array.
Per reports from multiple non-x86 animals (florican, locust, ...).
Discussion: https://postgr.es/m/20200628151002.7x5laxwpgvkyiu3q@development
M src/backend/executor/nodeModifyTable.c
Switch "cl /?" to "cl /help" in MSVC scripts for platform detection
commit : 733d670073efd2c3a9df07c225006668009ab793
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 21 Jan 2021 10:56:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 21 Jan 2021 10:56:03 +0900
"cl /?" produces a different output if run on a real or a virtual drive
(this can be set with a simple subst command), causing an error in the
MSVC scripts if building on a virtual drive because the platform to use
cannot be detected.
"cl /help", on the contrary, produces a consistent output if used on a
real or virtual drive. Changing to "/help" allows the compilation to
work with a virtual drive as long as the top of the code repository is
part of the drive, without impacting the build on real drives.
Reported-by: Robert Grange
Author: Juan José Santamaría Flecha
Discussion: https://postgr.es/m/16825-c4f104bcebc67034@postgresql.org
M src/tools/msvc/Solution.pm
Implement support for bulk inserts in postgres_fdw
commit : b663a4136331de6c7364226e3dbf7c88bfee7145
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 20 Jan 2021 23:05:46 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 20 Jan 2021 23:05:46 +0100
Extends the FDW API to allow batching inserts into foreign tables. That
is usually much more efficient than inserting individual rows, due to
high latency for each round-trip to the foreign server.
It was possible to implement something similar in the regular FDW API,
but it was inconvenient and there were issues with reporting the number
of actually inserted rows etc. This extends the FDW API with two new
functions:
* GetForeignModifyBatchSize - allows the FDW picking optimal batch size
* ExecForeignBatchInsert - inserts a batch of rows at once
Currently, only INSERT queries support batching. Support for DELETE and
UPDATE may be added in the future.
This also implements batching for postgres_fdw. The batch size may be
specified using "batch_size" option both at the server and table level.
The initial patch version was written by me, but it was rewritten and
improved in many ways by Takayuki Tsunakawa.
Author: Takayuki Tsunakawa
Reviewed-by: Tomas Vondra, Amit Langote
Discussion: https://postgr.es/m/20200628151002.7x5laxwpgvkyiu3q@development
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/postgres-fdw.sgml
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/backend/nodes/list.c
M src/include/foreign/fdwapi.h
M src/include/nodes/execnodes.h
M src/include/nodes/pg_list.h
psql \dX: list extended statistics objects
commit : ad600bba0422dde4b73fbd61049ff2a3847b068a
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 20 Jan 2021 22:56:06 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 20 Jan 2021 22:56:06 +0100
The new command lists extended statistics objects. All past releases
with extended statistics are supported.
This is a simplified version of commit 891a1d0bca, which had to be
reverted due to not considering pg_statistic_ext_data is not accessible
by regular users. Fields requiring access to this catalog were removed.
It's possible to add them, but it'll require changes to core.
Author: Tatsuro Yamada
Reviewed-by: Julien Rouhaud, Alvaro Herrera, Tomas Vondra, Noriyoshi Shinoda
Discussion: https://postgr.es/m/c027a541-5856-75a5-0868-341301e1624b%40nttcom.co.jp_1
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/bin/psql/help.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Further tweaking of PG_SYSROOT heuristics for macOS.
commit : 9d23c15a034ba163ae1045b945954e8e1bcfc72a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Jan 2021 12:07:23 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Jan 2021 12:07:23 -0500
It emerges that in some phases of the moon (perhaps to do with
directory entry order?), xcrun will report that the SDK path is
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
which is normally a symlink to a version-numbered sibling directory.
Our heuristic to skip non-version-numbered pathnames was rejecting
that, which is the wrong thing to do. We'd still like to end up
with a version-numbered PG_SYSROOT value, but we can have that by
dereferencing the symlink.
Like the previous fix, back-patch to all supported versions.
Discussion: https://postgr.es/m/522433.1611089678@sss.pgh.pa.us
M src/template/darwin
Disable vacuum page skipping in selected test cases.
commit : c2dc1a79767a0f947e1145f82eb65dfe4360d25f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Jan 2021 11:49:29 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Jan 2021 11:49:29 -0500
By default VACUUM will skip pages that it can't immediately get
exclusive access to, which means that even activities as harmless
and unpredictable as checkpoint buffer writes might prevent a page
from being processed. Ordinarily this is no big deal, but we have
a small number of test cases that examine the results of VACUUM's
processing and therefore will fail if the page of interest is skipped.
This seems to be the explanation for some rare buildfarm failures.
To fix, add the DISABLE_PAGE_SKIPPING option to the VACUUM commands
in tests where this could be an issue.
In passing, remove a duplicated query in pageinspect/sql/page.sql.
Back-patch as necessary (some of these cases are as old as v10).
Discussion: https://postgr.es/m/413923.1611006484@sss.pgh.pa.us
M contrib/amcheck/expected/check_heap.out
M contrib/amcheck/sql/check_heap.sql
M contrib/amcheck/t/001_verify_heapam.pl
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/sql/page.sql
M contrib/pg_visibility/expected/pg_visibility.out
M contrib/pg_visibility/sql/pg_visibility.sql
Fix bug in detecting concurrent page splits in GiST insert
commit : 6b4d3046f422c2682365924b515c7588d5a3e651
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 20 Jan 2021 11:58:03 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 20 Jan 2021 11:58:03 +0200
In commit 9eb5607e699, I got the condition on checking for split or
deleted page wrong: I used && instead of ||. The comment correctly said
"concurrent split _or_ deletion".
As a result, GiST insertion could miss a concurrent split, and insert to
wrong page. Duncan Sands demonstrated this with a test script that did a
lot of concurrent inserts.
Backpatch to v12, where this was introduced. REINDEX is required to fix
indexes that were affected by this bug.
Backpatch-through: 12
Reported-by: Duncan Sands
Discussion: https://www.postgresql.org/message-id/a9690483-6c6c-3c82-c8ba-dc1a40848f11%40deepbluecap.com
M src/backend/access/gist/gist.c
Fix sample output of EXPLAIN ANALYZE.
commit : 679744cf1b0d0569d16b4dd2d020f9095ea3d53b
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 20 Jan 2021 22:31:26 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 20 Jan 2021 22:31:26 +1300
Since commit f0f13a3a08b2757997410f3a1c38bdc22973c525, we estimate
ModifyTable paths without a RETURNING clause differently. Update an
example from the manual that showed the old behavior.
Author: Takayuki Tsunakawa <tsunakawa.takay@fujitsu.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/TYAPR01MB29905674F41693BBA9DA28CAFEA20%40TYAPR01MB2990.jpnprd01.prod.outlook.com
M doc/src/sgml/perform.sgml
Add regression test for DROP OWNED BY with default ACLs
commit : a36dc04d424a6bfa03ee2cf75c85a6b7f9697e70
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 20 Jan 2021 13:28:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 20 Jan 2021 13:28:10 +0900
DROP OWNED BY has a specific code path to remove ACLs stored in
pg_default_acl when cleaning up shared dependencies that had no
coverage with the existing tests. This issue has been found while
digging into the bug fixed by 21378e1.
As ALTER DEFAULT PRIVILEGES impacts the ACLs of all objects created
while the default permissions are visible, the test uses a transaction
rollback to isolate the test and avoid any impact with other sessions
running in parallel.
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/YAbQ1OD+3ip4lRv8@paquier.xyz
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Fix ALTER DEFAULT PRIVILEGES with duplicated objects
commit : 21378e1fefedcaed3d855ae7aa772555295d05d6
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 20 Jan 2021 11:38:17 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 20 Jan 2021 11:38:17 +0900
Specifying duplicated objects in this command would lead to unique
constraint violations in pg_default_acl or "tuple already updated by
self" errors. Similarly to GRANT/REVOKE, increment the command ID after
each subcommand processing to allow this case to work transparently.
A regression test is added by tweaking one of the existing queries of
privileges.sql to stress this case.
Reported-by: Andrus
Author: Michael Paquier
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/ae2a7dc1-9d71-8cba-3bb9-e4cb7eb1f44e@hot.ee
Backpatch-through: 9.5
M src/backend/catalog/aclchk.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Remove faulty support for MergeAppend plan with WHERE CURRENT OF.
commit : a0efda88a679edaee9855628cb05b2ab00d80a15
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Jan 2021 13:25:33 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Jan 2021 13:25:33 -0500
Somebody extended search_plan_tree() to treat MergeAppend exactly
like Append, which is 100% wrong, because unlike Append we can't
assume that only one input node is actively returning tuples.
Hence a cursor using a MergeAppend across a UNION ALL or inheritance
tree could falsely match a WHERE CURRENT OF query at a row that
isn't actually the cursor's current output row, but coincidentally
has the same TID (in a different table) as the current output row.
Delete the faulty code; this means that such a case will now return
an error like 'cursor "foo" is not a simply updatable scan of table
"bar"', instead of silently misbehaving. Users should not find that
surprising though, as the same cursor query could have failed that way
already depending on the chosen plan. (It would fail like that if the
sort were done with an explicit Sort node instead of MergeAppend.)
Expand the clearly-inadequate commentary to be more explicit about
what this code is doing, in hopes of forestalling future mistakes.
It's been like this for awhile, so back-patch to all supported
branches.
Discussion: https://postgr.es/m/482865.1611075182@sss.pgh.pa.us
M src/backend/executor/execCurrent.c
pageinspect: Change block number arguments to bigint
commit : f18aa1b203930ed28cfe42e82d3418ae6277576d
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 19 Jan 2021 10:28:05 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 19 Jan 2021 10:28:05 +0100
Block numbers are 32-bit unsigned integers. Therefore, the smallest
SQL integer type that they can fit in is bigint. However, in the
pageinspect module, most input and output parameters dealing with
block numbers were declared as int. The behavior with block numbers
larger than a signed 32-bit integer was therefore dubious. Change
these arguments to type bigint and add some more explicit error
checking on the block range.
(Other contrib modules appear to do this correctly already.)
Since we are changing argument types of existing functions, in order
to not misbehave if the binary is updated before the extension is
updated, we need to create new C symbols for the entry points, similar
to how it's done in other extensions as well.
Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
M contrib/pageinspect/Makefile
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/btreefuncs.c
M contrib/pageinspect/expected/btree.out
M contrib/pageinspect/expected/gin.out
M contrib/pageinspect/expected/hash.out
A contrib/pageinspect/expected/oldextversions.out
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/hashfuncs.c
M contrib/pageinspect/pageinspect–1.8–1.9.sql
M contrib/pageinspect/pageinspect.h
M contrib/pageinspect/rawpage.c
M contrib/pageinspect/sql/btree.sql
M contrib/pageinspect/sql/gin.sql
M contrib/pageinspect/sql/hash.sql
A contrib/pageinspect/sql/oldextversions.sql
M contrib/pageinspect/sql/page.sql
M doc/src/sgml/pageinspect.sgml
doc: Add note about the server name of postgres_fdw_get_connections() returns.
commit : ee79a548e746da9a99df0cac70a3ddc095f2829a
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 19 Jan 2021 00:56:10 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 19 Jan 2021 00:56:10 +0900
Previously the document didn't mention the case where
postgres_fdw_get_connections() returns NULL in server_name column.
Users might be confused about why NULL was returned.
This commit adds the note that, in postgres_fdw_get_connections(),
the server name of an invalid connection will be NULL if the server is dropped.
Suggested-by: Zhijie Hou
Author: Bharath Rupireddy
Reviewed-by: Zhijie Hou, Fujii Masao
Discussion: https://postgr.es/m/e7ddd14e96444fce88e47a709c196537@G08CNEXMBPEKD05.g08.fujitsu.local
M doc/src/sgml/postgres-fdw.sgml
pgindent worker.c.
commit : ed43677e20369040ca4e50c698010c39d5ac0f47
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 19 Jan 2021 08:10:13 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 19 Jan 2021 08:10:13 +0530
This is a leftover from commit 0926e96c49. Changing this separately
because this file is being modified for upcoming patch logical replication
of 2PC.
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+Ps+EgG8KzcmAyAgBUi_vuTps6o9ZA8DG6SdnO0-YuOhPQ@mail.gmail.com
M src/backend/replication/logical/worker.c
doc: adjust alignment of doc file list for "pg_waldump.sgml"
commit : 9e7dbe3369cd8f5b0136c53b817471002505f934
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 18 Jan 2021 18:48:25 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 18 Jan 2021 18:48:25 -0500
Backpatch-through: 10
M doc/src/sgml/ref/allfiles.sgml
Avoid crash with WHERE CURRENT OF and a custom scan plan.
commit : 60661bbf2dca0c1e744c948c74bccab98933c45b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 18 Jan 2021 18:32:30 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 18 Jan 2021 18:32:30 -0500
execCurrent.c's search_plan_tree() assumed that ForeignScanStates
and CustomScanStates necessarily have a valid ss_currentRelation.
This is demonstrably untrue for postgres_fdw's remote join and
remote aggregation plans, and non-leaf custom scans might not have
an identifiable scan relation either. Avoid crashing by ignoring
such nodes when the field is null.
This solution will lead to errors like 'cursor "foo" is not a
simply updatable scan of table "bar"' in cases where maybe we
could have allowed WHERE CURRENT OF to work. That's not an issue
for postgres_fdw's usages, since joins or aggregations would render
WHERE CURRENT OF invalid anyway. But an otherwise-transparent
upper level custom scan node might find this annoying. When and if
someone cares to expend work on such a scenario, we could invent a
custom-scan-provider callback to determine what's safe.
Report and patch by David Geier, commentary by me. It's been like
this for awhile, so back-patch to all supported branches.
Discussion: https://postgr.es/m/0253344d-9bdd-11c4-7f0d-d88c02cd7991@swarm64.com
M src/backend/executor/execCurrent.c
Narrow the scope of a local variable.
commit : 3fd80c728dc36fbd250ca3019c2f5fa2567f1a75
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 18 Jan 2021 15:55:01 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 18 Jan 2021 15:55:01 -0500
This is better style and more symmetrical with the other if-branch.
This likely should have been included in 9de77b545 (which created
the opportunity), but it was overlooked.
Japin Li
Discussion: https://postgr.es/m/MEYP282MB16699FA4A7CD57EB250E871FB6A40@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
M src/backend/replication/logical/proto.c
Add bytea equivalents of ltrim() and rtrim().
commit : a6cf3df4ebdcbc7857910a67f259705645383e9f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 18 Jan 2021 15:11:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 18 Jan 2021 15:11:32 -0500
We had bytea btrim() already, but for some reason not the other two.
Joel Jacobson
Discussion: https://postgr.es/m/d10cd5cd-a901-42f1-b832-763ac6f7ff3a@www.fastmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/oracle_compat.c
M src/backend/utils/adt/ruleutils.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/create_view.out
M src/test/regress/expected/strings.out
M src/test/regress/sql/create_view.sql
M src/test/regress/sql/strings.sql
Allow for error or refusal while absorbing a ProcSignalBarrier.
commit : a3ed4d1efe9f3a1765c504ba4a224f1f393b5b14
author : Robert Haas <rhaas@postgresql.org>
date : Mon, 18 Jan 2021 12:09:52 -0500
committer: Robert Haas <rhaas@postgresql.org>
date : Mon, 18 Jan 2021 12:09:52 -0500
Previously, the per-barrier-type functions tasked with absorbing
them were expected to always succeed and never throw an error.
However, that's a bit inconvenient. Further study has revealed that
there are realistic cases where it might not be possible to absorb
a ProcSignalBarrier without terminating the transaction, or even
the whole backend. Similarly, for some barrier types, there might
be other reasons where it's not reasonably possible to absorb the
barrier at certain points in the code, so provide a way for a
per-barrier-type function to reject absorbing the barrier.
Unfortunately, there's still no committed code making use of this
infrastructure; hopefully, we'll get there. :-(
Patch by me, reviewed by Andres Freund and Amul Sul.
Discussion: http://postgr.es/m/20200908182005.xya7wetdh3pndzim@alap3.anarazel.de
Discussion: http://postgr.es/m/CA+Tgmob56Pk1-5aTJdVPCWFHon7me4M96ENpGe9n_R4JUjjhZA@mail.gmail.com
M src/backend/storage/ipc/procsignal.c
Bump PGSTAT_FILE_FORMAT_ID
commit : b2f87b46690ab8cd3ee7c77226fcf3f5bb713e5b
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 18 Jan 2021 17:51:49 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 18 Jan 2021 17:51:49 +0100
This was missed in 960869da08
Reported-By: Laurenz Albe
Discussion: https://postgr.es/m/4f0aacc5fe1b4bfafa32b36ecd97469fae526a75.camel@cybertec.at
M src/include/pgstat.h
Check for BuildIndexValueDescription returning NULL in gist_page_items
commit : 5d1e5c8b758770186b005a1c3888b05e37af79c5
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 18 Jan 2021 14:48:43 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 18 Jan 2021 14:48:43 +0200
Per Coverity. BuildIndexValueDescription() cannot actually return NULL in
this instance, because it only returns NULL if the user doesn't have the
required privileges, and this function can only be used by superuser. But
better safe than sorry.
M contrib/pageinspect/gistfuncs.c
Pause recovery for insufficient parameter settings
commit : 15251c0a60be76eedee74ac0e94b433f9acca5af
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 18 Jan 2021 08:49:10 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 18 Jan 2021 08:49:10 +0100
When certain parameters are changed on a physical replication primary,
this is communicated to standbys using the XLOG_PARAMETER_CHANGE WAL
record. The standby then checks whether its own settings are at least
as big as the ones on the primary. If not, the standby shuts down
with a fatal error.
This patch changes this behavior for hot standbys to pause recovery at
that point instead. That allows read traffic on the standby to
continue while database administrators figure out next steps. When
recovery is unpaused, the server shuts down (as before). The idea is
to fix the parameters while recovery is paused and then restart when
there is a maintenance window.
Reviewed-by: Sergei Kornilov <sk@zsrv.org>
Discussion: https://www.postgresql.org/message-id/flat/4ad69a4c-cc9b-0dfe-0352-8b1b0cd36c7b@2ndquadrant.com
M doc/src/sgml/high-availability.sgml
M src/backend/access/transam/xlog.c
postgres_fdw: Add function to list cached connections to foreign servers.
commit : 708d165ddb92c54077a372acf6417e258dcb5fef
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 18 Jan 2021 15:11:08 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 18 Jan 2021 15:11:08 +0900
This commit adds function postgres_fdw_get_connections() to return
the foreign server names of all the open connections that postgres_fdw
established from the local session to the foreign servers. This function
also returns whether each connection is valid or not.
This function is useful when checking all the open foreign server connections.
If we found some connection to drop, from the result of function, probably
we can explicitly close them by the function that upcoming commit will add.
This commit bumps the version of postgres_fdw to 1.1 since it adds
new function.
Author: Bharath Rupireddy, tweaked by Fujii Masao
Reviewed-by: Zhijie Hou, Alexey Kondratov, Zhihong Yu, Fujii Masao
Discussion: https://postgr.es/m/2d5cb0b3-a6e8-9bbb-953f-879f47128faa@oss.nttdata.com
M contrib/postgres_fdw/Makefile
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
A contrib/postgres_fdw/postgres_fdw–1.0–1.1.sql
M contrib/postgres_fdw/postgres_fdw.control
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Refactor option handling of CLUSTER, REINDEX and VACUUM
commit : a3dc926009be833ea505eebd77ce4b72fe708b18
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 18 Jan 2021 14:03:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 18 Jan 2021 14:03:10 +0900
This continues the work done in b5913f6. All the options of those
commands are changed to use hex values rather than enums to reduce the
risk of compatibility bugs when introducing new options. Each option
set is moved into a new structure that can be extended with more
non-boolean options (this was already the case of VACUUM). The code of
REINDEX is restructured so as manual REINDEX commands go through a
single routine from utility.c, like VACUUM, to ease the allocation
handling of option parameters when a command needs to go through
multiple transactions.
This can be used as a base infrastructure for future patches related to
those commands, including reindex filtering and tablespace support.
Per discussion with people mentioned below, as well as Alvaro Herrera
and Peter Eisentraut.
Author: Michael Paquier, Justin Pryzby
Reviewed-by: Alexey Kondratov, Justin Pryzby
Discussion: https://postgr.es/m/X8riynBLwxAD9uKk@paquier.xyz
M src/backend/catalog/index.c
M src/backend/commands/cluster.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/vacuum.c
M src/backend/tcop/utility.c
M src/include/catalog/index.h
M src/include/commands/cluster.h
M src/include/commands/defrem.h
M src/include/commands/vacuum.h
M src/tools/pgindent/typedefs.list
pageinspect: Fix relcache leak in gist_page_items().
commit : 04eb75e783ba49ca2e0e75088d6590b64be8ed4d
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 18 Jan 2021 00:46:03 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 18 Jan 2021 00:46:03 +0200
The gist_page_items() function opened the index relation on first call and
closed it on the last call. But there's no guarantee that the function is
run to completion, leading to a relcache leak and warning at the end of
the transaction. To fix, refactor the function to return all the rows in
one call, as a tuplestore.
Reported-by: Tom Lane
Discussion: https://www.postgresql.org/message-id/234863.1610916631%40sss.pgh.pa.us
M contrib/pageinspect/gistfuncs.c
Set PD_ALL_VISIBLE and visibility map bits in COPY FREEZE
commit : 7db0cd2145f2bce84cac92402e205e4d2b045bf2
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 17 Jan 2021 22:11:39 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 17 Jan 2021 22:11:39 +0100
Make sure COPY FREEZE marks the pages as PD_ALL_VISIBLE and updates the
visibility map. Until now we only marked individual tuples as frozen,
but page-level flags were not updated, so the first VACUUM after the
COPY FREEZE had to rewrite the whole table.
This is a fairly old patch, and multiple people worked on it. The first
version was written by Jeff Janes, and then reworked by Pavan Deolasee
and Anastasia Lubennikova.
Author: Anastasia Lubennikova, Pavan Deolasee, Jeff Janes
Reviewed-by: Kuntal Ghosh, Jeff Janes, Tomas Vondra, Masahiko Sawada,
Andres Freund, Ibrar Ahmed, Robert Haas, Tatsuro Ishii,
Darafei Praliaskouski
Discussion: https://postgr.es/m/CABOikdN-ptGv0mZntrK2Q8OtfUuAjqaYMGmkdU1dCKFtUxVLrg@mail.gmail.com
Discussion: https://postgr.es/m/CAMkU%3D1w3osJJ2FneELhhNRLxfZitDgp9FPHee08NT2FQFmz_pQ%40mail.gmail.com
M contrib/pg_visibility/expected/pg_visibility.out
M contrib/pg_visibility/sql/pg_visibility.sql
M src/backend/access/heap/heapam.c
M src/backend/access/heap/hio.c
M src/include/access/heapam_xlog.h
Add missing array-enlargement logic to test_regex.c.
commit : 0c7d3bb99f72d66ec6ac63aee4c5fe6d683eee86
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 17 Jan 2021 12:53:48 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 17 Jan 2021 12:53:48 -0500
The stanza to report a "partial" match could overrun the initially
allocated output array, so it needs its own copy of the array-resizing
logic that's in the main loop. I overlooked the need for this in
ca8217c10.
Per report from Alexander Lakhin.
Discussion: https://postgr.es/m/3206aace-50db-e02a-bbea-76d5cdaa2cb6@gmail.com
M src/test/modules/test_regex/test_regex.c
Add documentation chapter about checksums
commit : cf621d9d84db1e6edaff8ffa26bad93fdce5f830
author : Magnus Hagander <magnus@hagander.net>
date : Sun, 17 Jan 2021 15:31:23 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Sun, 17 Jan 2021 15:31:23 +0100
Data checksums did not have a longer discussion in the docs,
this adds a short section with an overview.
Extracted from the larger patch for on-line enabling of checksums, which
has many more authors and reviewers.
Author: Daniel Gustafsson
Reviewed-By: Magnus Hagander, Michael Banck (and others through the big patch)
Discussion: https://postgr.es/m/5ff49fa4.1c69fb81.658f3.04ac@mx.google.com
M doc/src/sgml/amcheck.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/wal.sgml
Revert "psql \dX: list extended statistics objects"
commit : 1db0d173a2201119f297ea35edfb41579893dd8c
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 17 Jan 2021 15:11:14 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 17 Jan 2021 15:11:14 +0100
Reverts 891a1d0bca, because the new psql command \dX only worked for
users users who can read pg_statistic_ext_data catalog, and most regular
users lack that privilege (the catalog may contain sensitive user data).
Reported-by: Noriyoshi Shinoda
Discussion: https://postgr.es/m/c027a541-5856-75a5-0868-341301e1624b%40nttcom.co.jp_1
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/bin/psql/help.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Add --no-instructions parameter to initdb
commit : e09155bd62f0ac5817cb3d736eb35adf4200549e
author : Magnus Hagander <magnus@hagander.net>
date : Sun, 17 Jan 2021 14:28:17 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Sun, 17 Jan 2021 14:28:17 +0100
Specifying this parameter removes the informational messages about how
to start the server. This is intended for use by wrappers in different
packaging systems, where those instructions would most likely be wrong
anyway, but the other output from initdb would still be useful (and thus
just redirecting everything to /dev/null would be bad).
Author: Magnus Hagander
Reviewed-By: Peter Eisentraut
Discusion: https://postgr.es/m/CABUevEzo4t5bmTXF0_B9WzmuWpVbMpkNZZiGvzV8NZa-=fPqeQ@mail.gmail.com
M doc/src/sgml/ref/initdb.sgml
M src/bin/initdb/initdb.c
Add pg_stat_database counters for sessions and session time
commit : 960869da0803427d14335bba24393f414b476e2c
author : Magnus Hagander <magnus@hagander.net>
date : Sun, 17 Jan 2021 13:34:09 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Sun, 17 Jan 2021 13:34:09 +0100
This add counters for number of sessions, the different kind of session
termination types, and timers for how much time is spent in active vs
idle in a database to pg_stat_database.
Internally this also renames the parameter "force" to disconnect. This
was the only use-case for the parameter before, so repurposing it to
this mroe narrow usecase makes things cleaner than inventing something
new.
Author: Laurenz Albe
Reviewed-By: Magnus Hagander, Soumyadeep Chakraborty, Masahiro Ikeda
Discussion: https://postgr.es/m/b07e1f9953701b90c66ed368656f2aef40cac4fb.camel@cybertec.at
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/tcop/postgres.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/error/elog.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
psql \dX: list extended statistics objects
commit : 891a1d0bca262ca78564e0fea1eaa5ae544ea5ee
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 17 Jan 2021 00:16:25 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 17 Jan 2021 00:16:25 +0100
The new command lists extended statistics objects, possibly with their
sizes. All past releases with extended statistics are supported.
Author: Tatsuro Yamada
Reviewed-by: Julien Rouhaud, Alvaro Herrera, Tomas Vondra
Discussion: https://postgr.es/m/c027a541-5856-75a5-0868-341301e1624b%40nttcom.co.jp_1
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/bin/psql/help.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Documenation fixups for replication protocol.
commit : a32d98351e1ff33c5a65db4ed8e2def70a807bfa
author : Jeff Davis <jdavis@postgresql.org>
date : Sat, 16 Jan 2021 14:40:12 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Sat, 16 Jan 2021 14:40:12 -0800
There is no CopyResponse message; it should be CopyOutResponse.
Also, if there is no WAL to stream, the server does not immediately
send a CommandComplete; it's a historical timeline, so it will send a
response tuple first.
Discussion: https://postgr.es/m/0a2c985ebcaa1acd385350aeba561b6509187394.camel@j-davis.com
M doc/src/sgml/protocol.sgml
Fix pg_dump for GRANT OPTION among initial privileges.
commit : f713ff7c646e5912e08089de74dacdfaaac3d03b
author : Noah Misch <noah@leadboat.com>
date : Sat, 16 Jan 2021 12:21:35 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 16 Jan 2021 12:21:35 -0800
The context is an object that no longer bears some aclitem that it bore
initially. (A user issued REVOKE or GRANT statements upon the object.)
pg_dump is forming SQL to reproduce the object ACL. Since initdb
creates no ACL bearing GRANT OPTION, reaching this bug requires an
extension where the creation script establishes such an ACL. No PGXN
extension does that. If an installation did reach the bug, pg_dump
would have omitted a semicolon, causing a REVOKE and the next SQL
statement to fail. Separately, since the affected code exists to
eliminate an entire aclitem, it wants plain REVOKE, not REVOKE GRANT
OPTION FOR. Back-patch to 9.6, where commit
23f34fa4ba358671adab16773e79c17c92cbc870 first appeared.
Discussion: https://postgr.es/m/20210109102423.GA160022@rfd.leadboat.com
M src/bin/pg_dump/dumputils.c
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/modules/test_pg_dump/test_pg_dump–1.0.sql
Prevent excess SimpleLruTruncate() deletion.
commit : 6db992833c04c0322f7f34a486adece01651f929
author : Noah Misch <noah@leadboat.com>
date : Sat, 16 Jan 2021 12:21:35 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 16 Jan 2021 12:21:35 -0800
Every core SLRU wraps around. With the exception of pg_notify, the wrap
point can fall in the middle of a page. Account for this in the
PagePrecedes callback specification and in SimpleLruTruncate()'s use of
said callback. Update each callback implementation to fit the new
specification. This changes SerialPagePrecedesLogically() from the
style of asyncQueuePagePrecedes() to the style of CLOGPagePrecedes().
(Whereas pg_clog and pg_serial share a key space, pg_serial is nothing
like pg_notify.) The bug fixed here has the same symptoms and user
followup steps as 592a589a04bd456410b853d86bd05faa9432cbbb. Back-patch
to 9.5 (all supported versions).
Reviewed by Andrey Borodin and (in earlier versions) by Tom Lane.
Discussion: https://postgr.es/m/20190202083822.GC32531@gust.leadboat.com
M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/commands/async.c
M src/backend/storage/lmgr/predicate.c
M src/include/access/slru.h
Remove unnecessary pstrdup in fetch_table_list.
commit : c95765f47673b16ed36acbfe98e1242e3c3822a3
author : Amit Kapila <akapila@postgresql.org>
date : Sat, 16 Jan 2021 10:15:32 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Sat, 16 Jan 2021 10:15:32 +0530
The result of TextDatumGetCString is already palloc'ed so we don't need to
allocate memory for it again. We decide not to backpatch it as there
doesn't seem to be any case where it can create a meaningful leak.
Author: Zhijie Hou
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/229fed2eb8c54c71a96ccb99e516eb12@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/commands/subscriptioncmds.c
Disallow CREATE STATISTICS on system catalogs
commit : c9a0dc34865f2f82d8e186f534d9b605afaa4d11
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 15 Jan 2021 23:24:19 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 15 Jan 2021 23:24:19 +0100
Add a check that CREATE STATISTICS does not add extended statistics on
system catalogs, similarly to indexes etc. It can be overriden using
the allow_system_table_mods GUC.
This bug exists since 7b504eb282c, adding the extended statistics, so
backpatch all the way back to PostgreSQL 10.
Author: Tomas Vondra
Reported-by: Dean Rasheed
Backpatch-through: 10
Discussion: https://postgr.es/m/CAEZATCXAPrrOKwEsyZKQ4uzzJQWBCt6QAvOcgqRGdWwT1zb%2BrQ%40mail.gmail.com
M src/backend/commands/statscmds.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Improve our heuristic for selecting PG_SYSROOT on macOS.
commit : 4823621db312a0597c40686c4c94d47428889fef
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 15 Jan 2021 11:28:51 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 15 Jan 2021 11:28:51 -0500
In cases where Xcode is newer than the underlying macOS version,
asking xcodebuild for the SDK path will produce a pointer to the
SDK shipped with Xcode, which may end up building code that does
not work on the underlying macOS version. It appears that in
such cases, xcodebuild's answer also fails to match the default
behavior of Apple's compiler: assuming one has installed Xcode's
"command line tools", there will be an SDK for the OS's own version
in /Library/Developer/CommandLineTools, and the compiler will
default to using that. This is all pretty poorly documented,
but experimentation suggests that "xcrun --show-sdk-path" gives
the sysroot path that the compiler is actually using, at least
in some cases. Hence, try that first, but revert to xcodebuild
if xcrun fails (in very old Xcode, it is missing or lacks the
--show-sdk-path switch).
Also, "xcrun --show-sdk-path" may give a path that is valid but lacks
any OS version identifier. We don't really want that, since most
of the motivation for wiring -isysroot into the build flags at all
is to ensure that all parts of a PG installation are built against
the same SDK, even when considering extensions built later and/or on
a different machine. Insist on finding "N.N" in the directory name
before accepting the result. (Adding "--sdk macosx" to the xcrun
call seems to produce the same answer as xcodebuild, but usually
more quickly because it's cached, so we also try that as a fallback.)
The core reason why we don't want to use Xcode's default SDK in cases
like this is that Apple's technology for introducing new syscalls
does not play nice with Autoconf: for example, configure will think
that preadv/pwritev exist when using a Big Sur SDK, even when building
on an older macOS version where they don't exist. It'd be nice to
have a better solution to that problem, but this patch doesn't attempt
to fix that.
Per report from Sergey Shinderuk. Back-patch to all supported versions.
Discussion: https://postgr.es/m/ed3b8e5d-0da8-6ebd-fd1c-e0ac80a4b204@postgrespro.ru
M src/template/darwin
Avoid spurious wait in concurrent reindex
commit : f9900df5f94936067e6fa24a9df609863eb08da2
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 15 Jan 2021 10:31:42 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 15 Jan 2021 10:31:42 -0300
This is like commit c98763bf51bf, but for REINDEX CONCURRENTLY. To wit:
this flags indicates that the current process is safe to ignore for the
purposes of waiting for other snapshots, when doing CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY. This helps two processes doing
either of those things not deadlock, and also avoids spurious waits.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Hamid Akhtar <hamid.akhtar@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/20201130195439.GA24598@alvherre.pgsql
M src/backend/commands/indexcmds.c
M src/include/storage/proc.h
Fix calculation of how much shared memory is required to store a TOC.
commit : 2ad78a87f018260d4474eee63187e1cc73c9b976
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 15 Jan 2021 12:44:17 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 15 Jan 2021 12:44:17 +0900
Commit ac883ac453 refactored shm_toc_estimate() but changed its calculation
of shared memory size for TOC incorrectly. Previously this could cause too
large memory to be allocated.
Back-patch to v11 where the bug was introduced.
Author: Takayuki Tsunakawa
Discussion: https://postgr.es/m/TYAPR01MB2990BFB73170E2C4921E2C4DFEA80@TYAPR01MB2990.jpnprd01.prod.outlook.com
M src/backend/storage/ipc/shm_toc.c
Remove PG_SHA*_DIGEST_STRING_LENGTH from sha2.h
commit : ccf4e277a4de120a2f08db7e45399d87e1176bda
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 15 Jan 2021 11:46:34 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 15 Jan 2021 11:46:34 +0900
The last reference to those variables has been removed in aef8948, so
this cleans up a bit the code.
Discussion: https://postgr.es/m/X//ggAqmTtt+3t7X@paquier.xyz
M src/include/common/sha2.h
Fix O(N^2) stat() calls when recycling WAL segments
commit : 5ae1572993ae8bf1f6c33a933915c07cc9bc0add
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 15 Jan 2021 10:33:13 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 15 Jan 2021 10:33:13 +0900
The counter tracking the last segment number recycled was getting
initialized when recycling one single segment, while it should be used
across a full cycle of segments recycled to prevent useless checks
related to entries already recycled.
This performance issue has been introduced by b2a5545, and it was first
implemented in 61b86142.
No backpatch is done per the lack of field complaints.
Reported-by: Andres Freund, Thomas Munro
Author: Michael Paquier
Reviewed-By: Andres Freund
Discussion: https://postgr.es/m/20170621211016.eln6cxxp3jrv7m4m@alap3.anarazel.de
Discussion: https://postgr.es/m/CA+hUKG+DRiF9z1_MU4fWq+RfJMxP7zjoptfcmuCFPeO4JM2iVg@mail.gmail.com
M src/backend/access/transam/xlog.c
postgres_fdw: Save foreign server OID in connection cache entry.
commit : 5e5f4fcd89c082bba0239e8db1552834b4905c34
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 15 Jan 2021 10:30:19 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 15 Jan 2021 10:30:19 +0900
The foreign server OID stored in the connection cache entry is used as
a lookup key to directly get the server name.
Previously since the connection cache entry did not have the server OID,
postgres_fdw had to get the server OID at first from user mapping before
getting the server name. So if the corresponding user mapping was dropped,
postgres_fdw could raise the error "cache lookup failed for user mapping"
while looking up user mapping and fail to get the server name even though
the server had not been dropped yet.
Author: Bharath Rupireddy
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVRZPUB7ZwqLn-6DY8C_UmPs6084gSpHA92YBv++1AJXA@mail.gmail.com
M contrib/postgres_fdw/connection.c
pg_dump: label PUBLICATION TABLE ArchiveEntries with an owner.
commit : 8e396a773b80c72e5d5a0ca9755dffe043c97a05
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 14 Jan 2021 16:19:38 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 14 Jan 2021 16:19:38 -0500
This is the same fix as commit 9eabfe300 applied to INDEX ATTACH
entries, but for table-to-publication attachments. As in that
case, even though the backend doesn't record "ownership" of the
attachment, we still ought to label it in the dump archive with
the role name that should run the ALTER PUBLICATION command.
The existing behavior causes the ALTER to be done by the original
role that started the restore; that will usually work fine, but
there may be corner cases where it fails.
The bulk of the patch is concerned with changing struct
PublicationRelInfo to include a pointer to the associated
PublicationInfo object, so that we can get the owner's name
out of that when the time comes. While at it, I rewrote
getPublicationTables() to do just one query of pg_publication_rel,
not one per table.
Back-patch to v10 where this code was introduced.
Discussion: https://postgr.es/m/1165710.1610473242@sss.pgh.pa.us
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
Prevent drop of tablespaces used by partitioned relations
commit : ebfe2dbd6b624e2a428e14b7ee9322cc096f63f7
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 14 Jan 2021 15:32:14 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 14 Jan 2021 15:32:14 -0300
When a tablespace is used in a partitioned relation (per commits
ca4103025dfe in pg12 for tables and 33e6c34c3267 in pg11 for indexes),
it is possible to drop the tablespace, potentially causing various
problems. One such was reported in bug #16577, where a rewriting ALTER
TABLE causes a server crash.
Protect against this by using pg_shdepend to keep track of tablespaces
when used for relations that don't keep physical files; we now abort a
tablespace if we see that the tablespace is referenced from any
partitioned relations.
Backpatch this to 11, where this problem has been latent all along. We
don't try to create pg_shdepend entries for existing partitioned
indexes/tables, but any ones that are modified going forward will be
protected.
Note slight behavior change: when trying to drop a tablespace that
contains both regular tables as well as partitioned ones, you'd
previously get ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE and now you'll
get ERRCODE_DEPENDENT_OBJECTS_STILL_EXIST. Arguably, the latter is more
correct.
It is possible to add protecting pg_shdepend entries for existing
tables/indexes, by doing
ALTER TABLE ONLY some_partitioned_table SET TABLESPACE pg_default;
ALTER TABLE ONLY some_partitioned_table SET TABLESPACE original_tablespace;
for each partitioned table/index that is not in the database default
tablespace. Because these partitioned objects do not have storage, no
file needs to be actually moved, so it shouldn't take more time than
what's required to acquire locks.
This query can be used to search for such relations:
SELECT ... FROM pg_class WHERE relkind IN ('p', 'I') AND reltablespace <> 0
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/16577-881633a9f9894fd5@postgresql.org
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/heap.c
M src/backend/catalog/pg_shdepend.c
M src/backend/commands/tablecmds.c
M src/backend/commands/tablespace.c
M src/include/catalog/dependency.h
M src/test/regress/input/tablespace.source
M src/test/regress/output/tablespace.source
Stabilize timeline switch regression test.
commit : 424d7a9b277c0da5ec638bf6344cda899a2e544a
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 14 Jan 2021 14:37:01 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 14 Jan 2021 14:37:01 +0900
Commit fef5b47f6b added the regression test to check whether a standby is
able to follow a primary on a newer timeline when WAL archiving is enabled.
But the buildfarm member florican reported that this test failed because
the requested WAL segment was removed and replication failed. This is a
timing issue. Since neither replication slot is used nor wal_keep_size is set
in the test, checkpoint could remove the WAL segment that's still necessary
for replication.
This commit stabilizes the test by setting wal_keep_size.
Back-patch to v13 where the regression test that this commit stabilizes
was added.
Author: Fujii Masao
Discussion: https://postgr.es/m/X//PsenxcC50jDzX@paquier.xyz
M src/test/recovery/t/004_timeline_switch.pl
Improve tab-completion for CLOSE, DECLARE, FETCH and MOVE.
commit : 3f238b882c276a59f5d98224850e5aee2a3fec8c
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 14 Jan 2021 15:41:22 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 14 Jan 2021 15:41:22 +0900
This commit makes CLOSE, FETCH and MOVE commands tab-complete the list of
cursors. Also this commit makes DECLARE command tab-complete the options.
Author: Shinya Kato, Sawada Masahiko, tweaked by Fujii Masao
Reviewed-by: Shinya Kato, Sawada Masahiko, Fujii Masao
Discussion: https://postgr.es/m/b0e4c5c53ef84c5395524f5056fc71f0@MP-MSGSS-MBX001.msg.nttdata.co.jp
M src/bin/psql/tab-complete.c
Minor header cleanup for the new iovec code.
commit : fb29ab26b38f1647423bf6aa2991e5fdf9060f08
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 14 Jan 2021 18:09:32 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 14 Jan 2021 18:09:32 +1300
Remove redundant function declaration and improve header comment in
pg_iovec.h. Move the new declaration in fd.h next to a group of more
similar functions.
M src/include/port/pg_iovec.h
M src/include/storage/fd.h
Ensure that a standby is able to follow a primary on a newer timeline.
commit : fef5b47f6bfc9bfec619bb2e6e66b027e7ff21a3
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 14 Jan 2021 12:27:11 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 14 Jan 2021 12:27:11 +0900
Commit 709d003fbd refactored WAL-reading code, but accidentally caused
WalSndSegmentOpen() to fail to follow a timeline switch while reading from
a historic timeline. This issue caused a standby to fail to follow a primary
on a newer timeline when WAL archiving is enabled.
If there is a timeline switch within the segment, WalSndSegmentOpen() should
read from the WAL segment belonging to the new timeline. But previously
since it failed to follow a timeline switch, it tried to read the WAL segment
with old timeline. When WAL archiving is enabled, that WAL segment with
old timeline doesn't exist because it's renamed to .partial. This leads
a primary to have tried to read non-existent WAL segment, and which caused
replication to faill with the error "ERROR: requested WAL segment ... has
already been removed".
This commit fixes WalSndSegmentOpen() so that it's able to follow a timeline
switch, to ensure that a standby is able to follow a primary on a newer
timeline even when WAL archiving is enabled.
This commit also adds the regression test to check whether a standby is
able to follow a primary on a newer timeline when WAL archiving is enabled.
Back-patch to v13 where the bug was introduced.
Reported-by: Kyotaro Horiguchi
Author: Kyotaro Horiguchi, tweaked by Fujii Masao
Reviewed-by: Alvaro Herrera, Fujii Masao
Discussion: https://postgr.es/m/20201209.174314.282492377848029776.horikyota.ntt@gmail.com
M src/backend/replication/walsender.c
M src/test/recovery/t/004_timeline_switch.pl
Rework refactoring of hex and encoding routines
commit : aef8948f38d9f3aa58bf8c2d4c6f62a7a456a9d1
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Jan 2021 11:13:24 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 14 Jan 2021 11:13:24 +0900
This commit addresses some issues with c3826f83 that moved the hex
decoding routine to src/common/:
- The decoding function lacked overflow checks, so when used for
security-related features it was an open door to out-of-bound writes if
not carefully used that could remain undetected. Like the base64
routines already in src/common/ used by SCRAM, this routine is reworked
to check for overflows by having the size of the destination buffer
passed as argument, with overflows checked before doing any writes.
- The encoding routine was missing. This is moved to src/common/ and
it gains the same overflow checks as the decoding part.
On failure, the hex routines of src/common/ issue an error as per the
discussion done to make them usable by frontend tools, but not by shared
libraries. Note that this is why ECPG is left out of this commit, and
it still includes a duplicated logic doing hex encoding and decoding.
While on it, this commit uses better variable names for the source and
destination buffers in the existing escape and base64 routines in
encode.c and it makes them more robust to overflow detection. The
previous core code issued a FATAL after doing out-of-bound writes if
going through the SQL functions, which would be enough to detect
problems when working on changes that impacted this area of the
code. Instead, an error is issued before doing an out-of-bound write.
The hex routines were being directly called for bytea conversions and
backup manifests without such sanity checks. The current calls happen
to not have any problems, but careless uses of such APIs could easily
lead to CVE-class bugs.
Author: Bruce Momjian, Michael Paquier
Reviewed-by: Sehrope Sarkuni
Discussion: https://postgr.es/m/20201231003557.GB22199@momjian.us
M src/backend/replication/backup_manifest.c
M src/backend/utils/adt/encode.c
M src/backend/utils/adt/varlena.c
M src/common/Makefile
A src/common/hex.c
D src/common/hex_decode.c
A src/include/common/hex.h
D src/include/common/hex_decode.h
M src/include/utils/builtins.h
M src/tools/msvc/Mkvcbuild.pm
Move our p{read,write}v replacements into their own files.
commit : 0d56acfbaa799553c0c6ea350fd6e68d81025994
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 14 Jan 2021 11:10:24 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 14 Jan 2021 11:10:24 +1300
macOS's ranlib issued a warning about an empty pread.o file with the
previous arrangement, on systems new enough to require no replacement
functions. Let's go back to using configure's AC_REPLACE_FUNCS system
to build and include each .o in the library only if it's needed, which
requires moving the *v() functions to their own files.
Also move the _with_retry() wrapper to a more permanent home.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/1283127.1610554395%40sss.pgh.pa.us
M configure
M configure.ac
M src/backend/storage/file/fd.c
M src/include/storage/fd.h
M src/port/Makefile
M src/port/pread.c
A src/port/preadv.c
M src/port/pwrite.c
A src/port/pwritev.c
M src/tools/msvc/Mkvcbuild.pm
Mark inet_server_addr() and inet_server_port() as parallel-restricted.
commit : 5a6f9bce8dabd371bdb4e3db5dda436f7f0a680f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 16:23:15 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 16:23:15 -0500
These need to be PR because they access the MyProcPort data structure,
which doesn't get copied to parallel workers. The very similar
functions inet_client_addr() and inet_client_port() are already
marked PR, but somebody missed these.
Although this is a pre-existing bug, we can't readily fix it in the back
branches since we can't force initdb. Given the small usage of these
two functions, and the even smaller likelihood that they'd get pushed to
a parallel worker anyway, it doesn't seem worth the trouble to suggest
that DBAs should fix it manually.
Masahiko Sawada
Discussion: https://postgr.es/m/CAD21AoAT4aHP0Uxq91qpD7NL009tnUYQe-b14R3MnSVOjtE71g@mail.gmail.com
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Run reformat-dat-files to declutter the catalog data files.
commit : 8b411b8ff41566a1aa601d1f05aeebbebbdb4a54
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 16:14:38 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 16:14:38 -0500
Things had gotten pretty messy here, apparently mostly but not
entirely the fault of the multirange patch. No functional changes.
M src/include/catalog/pg_aggregate.dat
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_cast.dat
M src/include/catalog/pg_collation.dat
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_range.dat
M src/include/catalog/pg_type.dat
Doc, more or less: uncomment tutorial example that was fixed long ago.
commit : dce62490818170b6479dfe08a28aae4bcdf7cc2d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 15:59:57 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 15:59:57 -0500
Reverts a portion of commit 344190b7e. Apparently, back in the
twentieth century we had some issues with multi-statement SQL
functions, but they've worked fine for a long time.
Daniel Westermann
Discussion: https://postgr.es/m/GVAP278MB04242DCBF5E31F528D53FA18D2A90@GVAP278MB0424.CHEP278.PROD.OUTLOOK.COM
M src/tutorial/funcs.source
Call out vacuum considerations in create index docs
commit : 93c39f987e9c019cd28c450ece8a621b2d8ce28a
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 13 Jan 2021 17:55:41 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 13 Jan 2021 17:55:41 -0300
Backpatch to pg12, which is as far as it goes without conflicts.
Author: James Coleman <jtc331@gmail.com>
Reviewed-by: "David G. Johnston" <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/CAAaqYe9oEfbz7AxXq7OX+FFVi5w5p1e_Of8ON8ZnKO9QqBfmjg@mail.gmail.com
M doc/src/sgml/ref/create_index.sgml
M doc/src/sgml/ref/reindex.sgml
Disallow a digit as the first character of a variable name in pgbench.
commit : c21ea4d53e9404279273da800daa49b7b9a5e81e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 14:52:49 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 14:52:49 -0500
The point of this restriction is to avoid trying to substitute variables
into timestamp literal values, which may contain strings like '12:34'.
There is a good deal more that should be done to reduce pgbench's
tendency to substitute where it shouldn't. But this is sufficient to
solve the case complained of by Jaime Soler, and it's simple enough
to back-patch.
Back-patch to v11; before commit 9d36a3866, pgbench had a slightly
different definition of what a variable name is, and anyway it seems
unwise to change long-stable branches for this.
Fabien Coelho
Discussion: https://postgr.es/m/alpine.DEB.2.22.394.2006291740420.805678@pseudo
M doc/src/sgml/ref/pgbench.sgml
M src/bin/pgbench/pgbench.c
Fix test failure with wal_level=minimal.
commit : 5abca4b1cd7193d6a5c8235c97eb240312190bcb
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 13 Jan 2021 20:57:21 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 13 Jan 2021 20:57:21 +0200
The newly-added gist pageinspect test prints the LSNs of GiST pages,
expecting them all to be 1 (GistBuildLSN). But with wal_level=minimal,
they got updated by the whole-relation WAL-logging at commit. Fix by
wrapping the problematic tests in the same transaction with the CREATE
INDEX.
Per buildfarm failure on thorntail.
Discussion: https://www.postgresql.org/message-id/3B4F97E5-40FB-4142-8CAA-B301CDFBF982%40iki.fi
M contrib/pageinspect/expected/gist.out
M contrib/pageinspect/sql/gist.sql
Doc: clarify behavior of back-half options in pg_dump.
commit : 06ed235adeb621a73cafd6ab35fa2405b3177329
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 13:30:04 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 13:30:04 -0500
Options that change how the archive data is converted to SQL text
are ignored when dumping to archive formats. The documentation
previously said "not meaningful", which is not helpful.
Discussion: https://postgr.es/m/161052021249.12228.9598689907884726185@wrigleys.postgresql.org
M doc/src/sgml/ref/pg_dump.sgml
Enhance nbtree index tuple deletion.
commit : d168b666823b6e0bcf60ed19ce24fb5fb91b8ccf
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 13 Jan 2021 09:21:32 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 13 Jan 2021 09:21:32 -0800
Teach nbtree and heapam to cooperate in order to eagerly remove
duplicate tuples representing dead MVCC versions. This is "bottom-up
deletion". Each bottom-up deletion pass is triggered lazily in response
to a flood of versions on an nbtree leaf page. This usually involves a
"logically unchanged index" hint (these are produced by the executor
mechanism added by commit 9dc718bd).
The immediate goal of bottom-up index deletion is to avoid "unnecessary"
page splits caused entirely by version duplicates. It naturally has an
even more useful effect, though: it acts as a backstop against
accumulating an excessive number of index tuple versions for any given
_logical row_. Bottom-up index deletion complements what we might now
call "top-down index deletion": index vacuuming performed by VACUUM.
Bottom-up index deletion responds to the immediate local needs of
queries, while leaving it up to autovacuum to perform infrequent clean
sweeps of the index. The overall effect is to avoid certain
pathological performance issues related to "version churn" from UPDATEs.
The previous tableam interface used by index AMs to perform tuple
deletion (the table_compute_xid_horizon_for_tuples() function) has been
replaced with a new interface that supports certain new requirements.
Many (perhaps all) of the capabilities added to nbtree by this commit
could also be extended to other index AMs. That is left as work for a
later commit.
Extend deletion of LP_DEAD-marked index tuples in nbtree by adding logic
to consider extra index tuples (that are not LP_DEAD-marked) for
deletion in passing. This increases the number of index tuples deleted
significantly in many cases. The LP_DEAD deletion process (which is now
called "simple deletion" to clearly distinguish it from bottom-up
deletion) won't usually need to visit any extra table blocks to check
these extra tuples. We have to visit the same table blocks anyway to
generate a latestRemovedXid value (at least in the common case where the
index deletion operation's WAL record needs such a value).
Testing has shown that the "extra tuples" simple deletion enhancement
increases the number of index tuples deleted with almost any workload
that has LP_DEAD bits set in leaf pages. That is, it almost never fails
to delete at least a few extra index tuples. It helps most of all in
cases that happen to naturally have a lot of delete-safe tuples. It's
not uncommon for an individual deletion operation to end up deleting an
order of magnitude more index tuples compared to the old naive approach
(e.g., custom instrumentation of the patch shows that this happens
fairly often when the regression tests are run).
Add a further enhancement that augments simple deletion and bottom-up
deletion in indexes that make use of deduplication: Teach nbtree's
_bt_delitems_delete() function to support granular TID deletion in
posting list tuples. It is now possible to delete individual TIDs from
posting list tuples provided the TIDs have a tableam block number of a
table block that gets visited as part of the deletion process (visiting
the table block can be triggered directly or indirectly). Setting the
LP_DEAD bit of a posting list tuple is still an all-or-nothing thing,
but that matters much less now that deletion only needs to start out
with the right _general_ idea about which index tuples are deletable.
Bump XLOG_PAGE_MAGIC because xl_btree_delete changed.
No bump in BTREE_VERSION, since there are no changes to the on-disk
representation of nbtree indexes. Indexes built on PostgreSQL 12 or
PostgreSQL 13 will automatically benefit from bottom-up index deletion
(i.e. no reindexing required) following a pg_upgrade. The enhancement
to simple deletion is available with all B-Tree indexes following a
pg_upgrade, no matter what PostgreSQL version the user upgrades from.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-By: Victor Yegorov <vyegorov@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wzm+maE3apHB8NOtmM=p-DO65j2V5GzAWCOEEuy3JZgb2g@mail.gmail.com
M doc/src/sgml/btree.sgml
M doc/src/sgml/ref/create_index.sgml
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/index/genam.c
M src/backend/access/nbtree/README
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/rmgrdesc/nbtdesc.c
M src/backend/access/table/tableamapi.c
M src/include/access/heapam.h
M src/include/access/nbtree.h
M src/include/access/nbtxlog.h
M src/include/access/tableam.h
M src/include/access/xlog_internal.h
Pass down "logically unchanged index" hint.
commit : 9dc718bdf2b1a574481a45624d42b674332e2903
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 13 Jan 2021 08:11:00 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 13 Jan 2021 08:11:00 -0800
Add an executor aminsert() hint mechanism that informs index AMs that
the incoming index tuple (the tuple that accompanies the hint) is not
being inserted by execution of an SQL statement that logically modifies
any of the index's key columns.
The hint is received by indexes when an UPDATE takes place that does not
apply an optimization like heapam's HOT (though only for indexes where
all key columns are logically unchanged). Any index tuple that receives
the hint on insert is expected to be a duplicate of at least one
existing older version that is needed for the same logical row. Related
versions will typically be stored on the same index page, at least
within index AMs that apply the hint.
Recognizing the difference between MVCC version churn duplicates and
true logical row duplicates at the index AM level can help with cleanup
of garbage index tuples. Cleanup can intelligently target tuples that
are likely to be garbage, without wasting too many cycles on less
promising tuples/pages (index pages with little or no version churn).
This is infrastructure for an upcoming commit that will teach nbtree to
perform bottom-up index deletion. No index AM actually applies the hint
just yet.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Victor Yegorov <vyegorov@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wz=CEKFa74EScx_hFVshCOn6AA5T-ajFASTdzipdkLTNQQ@mail.gmail.com
M contrib/bloom/blinsert.c
M contrib/bloom/bloom.h
M doc/src/sgml/indexam.sgml
M src/backend/access/brin/brin.c
M src/backend/access/common/toast_internals.c
M src/backend/access/gin/gininsert.c
M src/backend/access/gist/gist.c
M src/backend/access/hash/hash.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/index/indexam.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/spgist/spginsert.c
M src/backend/catalog/indexing.c
M src/backend/commands/constraint.c
M src/backend/commands/copyfrom.c
M src/backend/commands/trigger.c
M src/backend/executor/execIndexing.c
M src/backend/executor/execMain.c
M src/backend/executor/execReplication.c
M src/backend/executor/nodeModifyTable.c
M src/backend/replication/logical/worker.c
M src/include/access/amapi.h
M src/include/access/brin_internal.h
M src/include/access/genam.h
M src/include/access/gin_private.h
M src/include/access/gist_private.h
M src/include/access/hash.h
M src/include/access/nbtree.h
M src/include/access/spgist.h
M src/include/executor/executor.h
M src/test/modules/dummy_index_am/dummy_index_am.c
Log long wait time on recovery conflict when it's resolved.
commit : 39b03690b529935a3c33024ee68f08e2d347cf4f
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 13 Jan 2021 22:59:17 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 13 Jan 2021 22:59:17 +0900
This is a follow-up of the work done in commit 0650ff2303. This commit
extends log_recovery_conflict_waits so that a log message is produced
also when recovery conflict has already been resolved after deadlock_timeout
passes, i.e., when the startup process finishes waiting for recovery
conflict after deadlock_timeout. This is useful in investigating how long
recovery conflicts prevented the recovery from applying WAL.
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Bertrand Drouvot
Discussion: https://postgr.es/m/9a60178c-a853-1440-2cdc-c3af916cff59@amazon.com
M doc/src/sgml/config.sgml
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/proc.c
M src/include/storage/standby.h
Fix portability issues in the new gist pageinspect test.
commit : 6ecaaf810b8b9e3132f00549e630400f38a7ca37
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 13 Jan 2021 12:32:54 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 13 Jan 2021 12:32:54 +0200
1. The raw bytea representation of the point-type keys used in the test
depends on endianess. Remove the raw key_data column from the test.
2. The items stored on non-leftmost gist page depends on how many items
git on the other pages. This showed up as a failure on 32-bit i386
systems. To fix, only test the gist_page_items() function on the
leftmost leaf page.
Per Andrey Borodin and the buildfarm.
Discussion: https://www.postgresql.org/message-id/9FCEC1DC-86FB-4A57-88EF-DD13663B36AF%40yandex-team.ru
M contrib/pageinspect/expected/gist.out
M contrib/pageinspect/sql/gist.sql
Remove incorrect markup
commit : e6eeb8d799c703eb829612ddcb4fd5ba82d3e880
author : Magnus Hagander <magnus@hagander.net>
date : Wed, 13 Jan 2021 11:07:37 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Wed, 13 Jan 2021 11:07:37 +0100
Seems 737d69ffc3c made a copy/paste or automation error resulting in two
extra right-parenthesis.
Reported-By: Michael Vastola
Backpatch-through: 13
Discussion: https://postgr.es/m/161051035421.12224.1741822783166533529@wrigleys.postgresql.org
M doc/src/sgml/func.sgml
Add functions to 'pageinspect' to inspect GiST indexes.
commit : 756ab29124d7850d4392e2227b67b69b61576cd6
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 13 Jan 2021 10:33:33 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 13 Jan 2021 10:33:33 +0200
Author: Andrey Borodin and me
Discussion: https://www.postgresql.org/message-id/3E4F9093-A1B5-4DF8-A292-0B48692E3954%40yandex-team.ru
M contrib/pageinspect/Makefile
A contrib/pageinspect/expected/gist.out
A contrib/pageinspect/gistfuncs.c
A contrib/pageinspect/pageinspect–1.8–1.9.sql
M contrib/pageinspect/pageinspect.control
A contrib/pageinspect/sql/gist.sql
M doc/src/sgml/pageinspect.sgml
Don't use elog() in src/port/pwrite.c.
commit : df10ac625c1672edf839ff59cfcac9dcc097515c
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 13 Jan 2021 19:11:09 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 13 Jan 2021 19:11:09 +1300
Nothing broke because of this oversight yet, but it would fail to link
if we tried to use pg_pwrite() in frontend code on a system that lacks
pwrite(). Use an assertion instead. Also pgindent while here.
Discussion: https://postgr.es/m/CA%2BhUKGL57RvoQsS35TVPnQoPYqbtBixsdRhynB8NpcUKpHTTtg%40mail.gmail.com
M src/port/pwrite.c
Fix memory leak in SnapBuildSerialize.
commit : ee1b38f65948cb09ecf3c39b58bd88aabc950e7c
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 13 Jan 2021 08:19:50 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 13 Jan 2021 08:19:50 +0530
The memory for the snapshot was leaked while serializing it to disk during
logical decoding. This memory will be freed only once walsender stops
streaming the changes. This can lead to a huge memory increase when master
logs Standby Snapshot too frequently say when the user is trying to create
many replication slots.
Reported-by: funnyxj.fxj@alibaba-inc.com
Diagnosed-by: funnyxj.fxj@alibaba-inc.com
Author: Amit Kapila
Backpatch-through: 9.5
Discussion: https://postgr.es/m/033ab54c-6393-42ee-8ec9-2b399b5d8cde.funnyxj.fxj@alibaba-inc.com
M src/backend/replication/logical/snapbuild.c
Optimize DropRelFileNodesAllBuffers() for recovery.
commit : bea449c635c0e68e21610593594c1e5d52842cdd
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 13 Jan 2021 07:46:11 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 13 Jan 2021 07:46:11 +0530
Similar to commit d6ad34f341, this patch optimizes
DropRelFileNodesAllBuffers() by avoiding the complete buffer pool scan and
instead find the buffers to be invalidated by doing lookups in the
BufMapping table.
This optimization helps operations where the relation files need to be
removed like Truncate, Drop, Abort of Create Table, etc.
Author: Kirk Jamison
Reviewed-by: Kyotaro Horiguchi, Takayuki Tsunakawa, and Amit Kapila
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/OSBPR01MB3207DCA7EC725FDD661B3EDAEF660@OSBPR01MB3207.jpnprd01.prod.outlook.com
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/smgr/smgr.c
M src/include/storage/bufmgr.h
Fix routine name in comment of catcache.c
commit : fce7d0e6efbef304e81846c75eddf73099628d10
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 13 Jan 2021 10:32:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 13 Jan 2021 10:32:21 +0900
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/CALj2ACUDXLAkf_XxQO9tAUtnTNGi3Lmd8fANd+vBJbcHn1HoWA@mail.gmail.com
M src/backend/utils/cache/catcache.c
Invent struct ReindexIndexInfo
commit : c6c4b37395b22f7e649234453a0aafe74d61b7e6
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 12 Jan 2021 17:04:49 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 12 Jan 2021 17:04:49 -0300
This struct is used by ReindexRelationConcurrently to keep track of the
relations to process. This saves having to obtain some data repeatedly,
and has future uses as well.
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Hamid Akhtar <hamid.akhtar@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/20201130195439.GA24598@alvherre.pgsql
M src/backend/commands/indexcmds.c
M src/tools/pgindent/typedefs.list
pg_dump: label INDEX ATTACH ArchiveEntries with an owner.
commit : 9eabfe300a22ad3d776dc293265e15379790bd9a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 12 Jan 2021 13:37:38 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 12 Jan 2021 13:37:38 -0500
Although a partitioned index's attachment to its parent doesn't
have separate ownership, the ArchiveEntry for it needs to be
marked with an owner anyway, to ensure that the ALTER command
is run by the appropriate role when restoring with
--use-set-session-authorization. Without this, the ALTER will
be run by the role that started the restore session, which will
usually work but it's formally the wrong thing.
Back-patch to v11 where this type of ArchiveEntry was added.
In HEAD, add equivalent commentary to the just-added TABLE ATTACH
case, which I'd made do the right thing already.
Discussion: https://postgr.es/m/1094034.1610418498@sss.pgh.pa.us
M src/bin/pg_dump/pg_dump.c
Doc: fix description of privileges needed for ALTER PUBLICATION.
commit : cc865c0f319fde22540625e02863f42e9853b3e4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 12 Jan 2021 12:52:14 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 12 Jan 2021 12:52:14 -0500
Adding a table to a publication requires ownership of the table
(in addition to ownership of the publication). This was mentioned
nowhere.
M doc/src/sgml/ref/alter_publication.sgml
Fix thinko in comment
commit : a3e51a36b77ab073df88860cc21b2a6094d699d4
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 12 Jan 2021 11:48:45 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 12 Jan 2021 11:48:45 -0300
This comment has been wrong since its introduction in commit
2c03216d8311.
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAD21AoAzz6qipFJBbGEaHmyWxvvNDp8httbwLR9tUQWaTjUs2Q@mail.gmail.com
M src/backend/access/transam/xlogreader.c
Fix relation descriptor leak.
commit : 044aa9e70e552d9adb4f6429a1b0e96dde946a92
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 12 Jan 2021 08:19:39 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 12 Jan 2021 08:19:39 +0530
We missed closing the relation descriptor while sending changes via the
root of partitioned relations during logical replication.
Author: Amit Langote and Mark Zhao
Reviewed-by: Amit Kapila and Ashutosh Bapat
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/tencent_41FEA657C206F19AB4F406BE9252A0F69C06@qq.com
Discussion: https://postgr.es/m/tencent_6E296D2F7D70AFC90D83353B69187C3AA507@qq.com
M src/backend/replication/pgoutput/pgoutput.c
Optimize DropRelFileNodeBuffers() for recovery.
commit : d6ad34f3410f101f9909c0918a49d6ce86fa216c
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 12 Jan 2021 07:45:40 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 12 Jan 2021 07:45:40 +0530
The recovery path of DropRelFileNodeBuffers() is optimized so that
scanning of the whole buffer pool can be avoided when the number of
blocks to be truncated in a relation is below a certain threshold. For
such cases, we find the buffers by doing lookups in BufMapping table.
This improves the performance by more than 100 times in many cases
when several small tables (tested with 1000 relations) are truncated
and where the server is configured with a large value of shared
buffers (greater than equal to 100GB).
This optimization helps cases (a) when vacuum or autovacuum truncated off
any of the empty pages at the end of a relation, or (b) when the relation is
truncated in the same transaction in which it was created.
This commit introduces a new API smgrnblocks_cached which returns a cached
value for the number of blocks in a relation fork. This helps us to determine
the exact size of relation which is required to apply this optimization. The
exact size is required to ensure that we don't leave any buffer for the
relation being dropped as otherwise the background writer or checkpointer
can lead to a PANIC error while flushing buffers corresponding to files that
don't exist.
Author: Kirk Jamison based on ideas by Amit Kapila
Reviewed-by: Kyotaro Horiguchi, Takayuki Tsunakawa, and Amit Kapila
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/OSBPR01MB3207DCA7EC725FDD661B3EDAEF660@OSBPR01MB3207.jpnprd01.prod.outlook.com
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/smgr/smgr.c
M src/include/storage/bufmgr.h
M src/include/storage/smgr.h
Dump ALTER TABLE ... ATTACH PARTITION as a separate ArchiveEntry.
commit : 9a4c0e36fbd671b5e7426a5a0670bdd7ba2714a0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 21:09:03 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 21:09:03 -0500
Previously, we emitted the ATTACH PARTITION command as part of
the child table's ArchiveEntry. This was a poor choice since it
complicates restoring the partition as a standalone table; you have
to ignore the error from the ATTACH, which isn't even an option when
restoring direct-to-database with pg_restore. (pg_restore will issue
the whole ArchiveEntry as one PQexec, so that any error rolls back
the table creation as well.) Hence, separate it out as its own
ArchiveEntry, as indeed we already did for index ATTACH PARTITION
commands.
Justin Pryzby
Discussion: https://postgr.es/m/20201023052940.GE9241@telsasoft.com
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/pg_dump_sort.c
Make pg_dump's table of object-type priorities more maintainable.
commit : d5ab79d815783fe60062cefc423b54e82fbb92ff
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 19:58:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 19:58:07 -0500
Wedging a new object type into this table has historically required
manually renumbering a lot of existing entries. (Although it appears
that some people got lazy and re-used the priority level of an
existing object type, even if it wasn't particularly related.)
We can let the compiler do the counting by inventing an enum type that
lists the desired priority levels in order. Now, if you want to add
or remove a priority level, that's a one-liner.
This patch is not purely cosmetic, because I split apart the priorities
of DO_COLLATION and DO_TRANSFORM, as well as those of DO_ACCESS_METHOD
and DO_OPERATOR, which look to me to have been merged out of expediency
rather than because it was a good idea. Shell types continue to be
sorted interchangeably with full types, and opclasses interchangeably
with opfamilies.
M src/bin/pg_dump/pg_dump_sort.c
Fix function prototypes in dependency.h.
commit : f315205f3fafd6f6c7c479f480289fcf45700310
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 12 Jan 2021 10:55:35 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 12 Jan 2021 10:55:35 +1300
Commit 257836a7 accidentally deleted a couple of
redundant-but-conventional "extern" keywords on function prototypes.
Put them back.
Reported-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
M src/include/catalog/dependency.h
Rethink SQLSTATE code for ERRCODE_IDLE_SESSION_TIMEOUT.
commit : 4edf96846a02693e4416478b3302e5133d2e8e01
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 14:53:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 14:53:42 -0500
Move it to class 57 (Operator Intervention), which seems like a
better choice given that from the client's standpoint it behaves
a heck of a lot like, e.g., ERRCODE_ADMIN_SHUTDOWN.
In a green field I'd put ERRCODE_IDLE_IN_TRANSACTION_SESSION_TIMEOUT
here as well. But that's been around for a few years, so it's
probably too late to change its SQLSTATE code.
Discussion: https://postgr.es/m/763A0689-F189-459E-946F-F0EC4458980B@hotmail.com
M src/backend/utils/errcodes.txt
Try next host after a "cannot connect now" failure.
commit : c1d589571c497a952d7fbe40d9828655859d746f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 14:12:31 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 14:12:31 -0500
If a server returns ERRCODE_CANNOT_CONNECT_NOW, try the next host,
if multiple host names have been provided. This allows dealing
gracefully with standby servers that might not be in hot standby mode
yet.
In the wake of the preceding commit, it might be plausible to retry
many more error cases than we do now, but I (tgl) am hesitant to
move too aggressively on that --- it's not clear it'd be desirable
for cases such as bad-password, for example. But this case seems
safe enough.
Hubert Zhang, reviewed by Takayuki Tsunakawa
Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
M src/interfaces/libpq/fe-connect.c
Uniformly identify the target host in libpq connection failure reports.
commit : 52a10224e3cc1d706ba9800695f97cb163b747d5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 14:03:39 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 14:03:39 -0500
Prefix "could not connect to host-or-socket-path:" to all connection
failure cases that occur after the socket() call, and remove the
ad-hoc server identity data that was appended to a few of these
messages. This should produce much more intelligible error reports
in multiple-target-host situations, especially for error cases that
are off the beaten track to any degree (because none of those provided
any server identity info).
As an example of the change, formerly a connection attempt with a bad
port number such as "psql -p 12345 -h localhost,/tmp" might produce
psql: error: could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 12345?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 12345?
could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.12345"?
Now it looks like
psql: error: could not connect to host "localhost" (::1), port 12345: Connection refused
Is the server running on that host and accepting TCP/IP connections?
could not connect to host "localhost" (127.0.0.1), port 12345: Connection refused
Is the server running on that host and accepting TCP/IP connections?
could not connect to socket "/tmp/.s.PGSQL.12345": No such file or directory
Is the server running locally and accepting connections on that socket?
This requires adjusting a couple of regression tests to allow for
variation in the contents of a connection failure message.
Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
M src/bin/pg_dump/t/002_pg_dump.pl
M src/interfaces/ecpg/test/expected/connect-test5.stderr
M src/interfaces/libpq/fe-connect.c
Allow pg_regress.c wrappers to postprocess test result files.
commit : 800d93f314b0f7c10193e48b259f87800cb85d84
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 13:43:19 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 13:43:19 -0500
Add an optional callback to regression_main() that, if provided,
is invoked on each test output file before we try to compare it
to the expected-result file.
The main and isolation test programs don't need this (yet).
In pg_regress_ecpg, add a filter that eliminates target-host
details from "could not connect" error reports. This filter
doesn't do anything as of this commit, but it will be needed
by the next one.
In the long run we might want to provide some more general,
perhaps pattern-based, filtering mechanism for test output.
For now, this will solve the immediate problem.
Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
M src/interfaces/ecpg/test/pg_regress_ecpg.c
M src/test/isolation/isolation_main.c
M src/test/regress/pg_regress.c
M src/test/regress/pg_regress.h
M src/test/regress/pg_regress_main.c
In libpq, always append new error messages to conn->errorMessage.
commit : ffa2e4670123124b92f037d335a1e844c3782d3f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 13:12:09 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 11 Jan 2021 13:12:09 -0500
Previously, we had an undisciplined mish-mash of printfPQExpBuffer and
appendPQExpBuffer calls to report errors within libpq. This commit
establishes a uniform rule that appendPQExpBuffer[Str] should be used.
conn->errorMessage is reset only at the start of an application request,
and then accumulates messages till we're done. We can remove no less
than three different ad-hoc mechanisms that were used to get the effect
of concatenation of error messages within a sequence of operations.
Although this makes things quite a bit cleaner conceptually, the main
reason to do it is to make the world safer for the multiple-target-host
feature that was added awhile back. Previously, there were many cases
in which an error occurring during an individual host connection attempt
would wipe out the record of what had happened during previous attempts.
(The reporting is still inadequate, in that it can be hard to tell which
host got the failure, but that seems like a matter for a separate commit.)
Currently, lo_import and lo_export contain exceptions to the "never
use printfPQExpBuffer" rule. If we changed them, we'd risk reporting
an incidental lo_close failure before the actual read or write
failure, which would be confusing, not least because lo_close happened
after the main failure. We could improve this by inventing an
internal version of lo_close that doesn't reset the errorMessage; but
we'd also need a version of PQfn() that does that, and it didn't quite
seem worth the trouble for now.
Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
M src/interfaces/libpq/fe-auth-scram.c
M src/interfaces/libpq/fe-auth.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-gssapi-common.c
M src/interfaces/libpq/fe-lobj.c
M src/interfaces/libpq/fe-misc.c
M src/interfaces/libpq/fe-protocol2.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/fe-secure-common.c
M src/interfaces/libpq/fe-secure-gssapi.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/fe-secure.c
M src/interfaces/libpq/libpq-int.h
Use vectored I/O to fill new WAL segments.
commit : ce6a71fa5300cf00adf32c9daee302c523609709
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 11 Jan 2021 14:41:39 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 11 Jan 2021 14:41:39 +1300
Instead of making many block-sized write() calls to fill a new WAL file
with zeroes, make a smaller number of pwritev() calls (or various
emulations). The actual number depends on the OS's IOV_MAX, which
PG_IOV_MAX currently caps at 32. That means we'll write 256kB per call
on typical systems. We may want to tune the number later with more
experience.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGJA%2Bu-220VONeoREBXJ9P3S94Y7J%2BkqCnTYmahvZJwM%3Dg%40mail.gmail.com
M src/backend/access/transam/xlog.c
Provide pg_preadv() and pg_pwritev().
commit : 13a021f3e8c99915b3cc0cb2021a948d9c71ff32
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 11 Jan 2021 14:37:13 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 11 Jan 2021 14:37:13 +1300
Provide synchronous vectored file I/O routines. These map to preadv()
and pwritev(), with fallback implementations for systems that don't have
them. Also provide a wrapper pg_pwritev_with_retry() that automatically
retries on short writes.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGJA%2Bu-220VONeoREBXJ9P3S94Y7J%2BkqCnTYmahvZJwM%3Dg%40mail.gmail.com
M configure
M configure.ac
M src/include/pg_config.h.in
M src/include/port.h
A src/include/port/pg_iovec.h
M src/port/Makefile
M src/port/pread.c
M src/port/pwrite.c
M src/tools/msvc/Solution.pm
doc: expand description of how non-SELECT queries are processed
commit : 01334c92fa09dc496a444a4f206854ef37247258
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 9 Jan 2021 12:11:16 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 9 Jan 2021 12:11:16 -0500
The previous description of how the executor processes non-SELECT
queries was very dense, causing lack of clarity. This expanded text
spells it out more simply.
Reported-by: fotis.koutoupas@gmail.com
Discussion: https://postgr.es/m/160912275508.676.17469511338925622905@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/arch-dev.sgml
Fix the test for decoding of two-phase transactions.
commit : e33d004900f76c35759293fdedd4861b198fbf5b
author : Amit Kapila <akapila@postgresql.org>
date : Sat, 9 Jan 2021 15:45:38 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Sat, 9 Jan 2021 15:45:38 +0530
Commit 5a3574d7b3 added the test for decoding of two-phase transactions
during the build of a consistent snapshot. The test forgot to skip empty
xacts which can lead to decoding of extra empty transactions due to
background activity by autovacuum.
Per report by buildfarm.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/363512.1610171267@sss.pgh.pa.us
M contrib/test_decoding/expected/twophase_snapshot.out
M contrib/test_decoding/specs/twophase_snapshot.spec
Fix plpgsql tests for debug_invalidate_system_caches_always.
commit : 39d4a153105f0693d93f593a23e5144e2bd031ef
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 8 Jan 2021 18:12:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 8 Jan 2021 18:12:07 -0500
Commit c9d529848 resulted in having a couple more places where
the error context stack for a failure varies depending on
debug_invalidate_system_caches_always (nee CLOBBER_CACHE_ALWAYS).
This is not very surprising, since we have to re-parse cached
plans if the plan cache is clobbered. Stabilize the expected
test output by hiding the context stack in these places,
as we've done elsewhere in this test script.
(Another idea worth considering, now that we have
debug_invalidate_system_caches_always, is to force it to zero for
these test cases. That seems like it'd risk reducing the coverage
of cache-clobber testing, which might or might not be worth being
able to verify that we get the expected error output in normal
cases. For the moment I just stuck with the existing technique.)
In passing, update comments that referred to CLOBBER_CACHE_ALWAYS.
Per buildfarm member hyrax.
M src/pl/plpgsql/src/expected/plpgsql_cache.out
M src/pl/plpgsql/src/expected/plpgsql_cache_1.out
M src/pl/plpgsql/src/expected/plpgsql_record.out
M src/pl/plpgsql/src/sql/plpgsql_cache.sql
M src/pl/plpgsql/src/sql/plpgsql_record.sql
Fix ancient bug in parsing of BRE-mode regular expressions.
commit : afcc8772edcec687d87b6f762ca6113229af7291
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 8 Jan 2021 12:16:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 8 Jan 2021 12:16:00 -0500
brenext(), when parsing a '*' quantifier, forgot to return any "value"
for the token; per the equivalent case in next(), it should return
value 1 to indicate that greedy rather than non-greedy behavior is
wanted. The result is that the compiled regexp could behave like 'x*?'
rather than the intended 'x*', if we were unlucky enough to have
a zero in v->nextvalue at this point. That seems to happen with some
reliability if we have '.*' at the beginning of a BRE-mode regexp,
although that depends on the initial contents of a stack-allocated
struct, so it's not guaranteed to fail.
Found by Alexander Lakhin using valgrind testing. This bug seems
to be aboriginal in Spencer's code, so back-patch all the way.
Discussion: https://postgr.es/m/16814-6c5e3edd2bdf0d50@postgresql.org
M src/backend/regex/regc_lex.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Fix and simplify some code related to cryptohashes
commit : 15b824da97afb45f47e51b6b5b7e5eca09e5d03d
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 8 Jan 2021 10:37:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 8 Jan 2021 10:37:03 +0900
This commit addresses two issues:
- In pgcrypto, MD5 computation called pg_cryptohash_{init,update,final}
without checking for the result status.
- Simplify pg_checksum_raw_context to use only one variable for all the
SHA2 options available in checksum manifests.
Reported-by: Heikki Linnakangas
Discussion: https://postgr.es/m/f62f26bb-47a5-8411-46e5-4350823e06a5@iki.fi
M contrib/pgcrypto/internal.c
M src/common/checksum_helper.c
M src/include/common/checksum_helper.h
Adjust createdb TAP tests to work on recent OpenBSD.
commit : 9ffe2278372d7549547176c23564a5b3404d072e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 7 Jan 2021 20:36:09 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 7 Jan 2021 20:36:09 -0500
We found last February that the error-case tests added by commit
008cf0409 failed on OpenBSD, because that platform doesn't really
check locale names. At the time it seemed that that was only an issue
for LC_CTYPE, but testing on a more recent version of OpenBSD shows
that it's now equally lax about LC_COLLATE.
Rather than dropping the LC_COLLATE test too, put back LC_CTYPE
(reverting c4b0edb07), and adjust these tests to accept the different
error message that we get if setlocale() doesn't reject a bogus locale
name. The point of these tests is not really what the backend does
with the locale name, but to show that createdb quotes funny locale
names safely; so we're not losing test reliability this way.
Back-patch as appropriate.
Discussion: https://postgr.es/m/231373.1610058324@sss.pgh.pa.us
M src/bin/scripts/t/020_createdb.pl
Further second thoughts about idle_session_timeout patch.
commit : b8d0cda53377515ac61357ec4a60e85ca873f486
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 7 Jan 2021 11:45:08 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 7 Jan 2021 11:45:08 -0500
On reflection, the order of operations in PostgresMain() is wrong.
These timeouts ought to be shut down before, not after, we do the
post-command-read CHECK_FOR_INTERRUPTS, to guarantee that any
timeout error will be detected there rather than at some ill-defined
later point (possibly after having wasted a lot of work).
This is really an error in the original idle_in_transaction_timeout
patch, so back-patch to 9.6 where that was introduced.
M src/backend/tcop/postgres.c
Minor fixes in COPY progress docs
commit : ebb5457cfa514972847a2d03b5b4fd46f69bdc9b
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 7 Jan 2021 17:31:58 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 7 Jan 2021 17:31:58 +0100
Author: Justin Pryzby
Discussion: https://postgr.es/m/CAFp7Qwr6_FmRM6pCO0x_a0mymOfX_Gg+FEKet4XaTGSW=LitKQ@mail.gmail.com
M doc/src/sgml/monitoring.sgml
Add GUC to log long wait times on recovery conflicts.
commit : 0650ff23038bc3eb8d8fd851744db837d921e285
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 8 Jan 2021 00:47:03 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 8 Jan 2021 00:47:03 +0900
This commit adds GUC log_recovery_conflict_waits that controls whether
a log message is produced when the startup process is waiting longer than
deadlock_timeout for recovery conflicts. This is useful in determining
if recovery conflicts prevent the recovery from applying WAL.
Note that currently a log message is produced only when recovery conflict
has not been resolved yet even after deadlock_timeout passes, i.e.,
only when the startup process is still waiting for recovery conflict
even after deadlock_timeout.
Author: Bertrand Drouvot, Masahiko Sawada
Reviewed-by: Alvaro Herrera, Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/9a60178c-a853-1440-2cdc-c3af916cff59@amazon.com
M doc/src/sgml/config.sgml
M doc/src/sgml/high-availability.sgml
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/storage/standby.h
Fix bogus link in test comments.
commit : f7a1a805cb178653ea2a6c8991ad73b035af953e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Jan 2021 22:09:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Jan 2021 22:09:00 -0500
I apparently copied-and-pasted the wrong link in commit ca8217c10.
Point it where it was meant to go.
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Improve commentary in timeout.c.
commit : 9486e7b666fd113f043d5f091fd42bc1ef72acd8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Jan 2021 22:02:35 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Jan 2021 22:02:35 -0500
On re-reading I realized that I'd missed one race condition in the new
timeout code. It's safe, but add a comment explaining it.
Discussion: https://postgr.es/m/CA+hUKG+o6pbuHBJSGnud=TadsuXySWA7CCcPgCt2QE9F6_4iHQ@mail.gmail.com
M src/backend/utils/misc/timeout.c
Fix allocation logic of cryptohash context data with OpenSSL
commit : 55fe26a4b580b17d721c5accb842cc6a08295273
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 7 Jan 2021 10:21:02 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 7 Jan 2021 10:21:02 +0900
The allocation of the cryptohash context data when building with OpenSSL
was happening in the memory context of the caller of
pg_cryptohash_create(), which could lead to issues with resowner cleanup
if cascading resources are cleaned up on an error. Like other
facilities using resowners, move the base allocation to TopMemoryContext
to ensure a correct cleanup on failure.
The resulting code gets simpler with this commit as the context data is
now hold by a unique opaque pointer, so as there is only one single
allocation done in TopMemoryContext.
After discussion, also change the cryptohash subroutines to return an
error if the caller provides NULL for the context data to ease error
detection on OOM.
Author: Heikki Linnakangas
Discussion: https://postgr.es/m/X9xbuEoiU3dlImfa@paquier.xyz
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
M src/include/common/cryptohash.h
M src/tools/pgindent/typedefs.list
Add idle_session_timeout.
commit : 9877374bef76ef03923f6aa8b955f2dbcbe6c2c7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Jan 2021 18:28:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Jan 2021 18:28:42 -0500
This GUC variable works much like idle_in_transaction_session_timeout,
in that it kills sessions that have waited too long for a new client
query. But it applies when we're not in a transaction, rather than
when we are.
Li Japin, reviewed by David Johnston and Hayato Kuroda, some
fixes by me
Discussion: https://postgr.es/m/763A0689-F189-459E-946F-F0EC4458980B@hotmail.com
M doc/src/sgml/config.sgml
M src/backend/storage/lmgr/proc.c
M src/backend/tcop/postgres.c
M src/backend/utils/errcodes.txt
M src/backend/utils/init/globals.c
M src/backend/utils/init/postinit.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/miscadmin.h
M src/include/storage/proc.h
M src/include/utils/timeout.h
Improve timeout.c's handling of repeated timeout set/cancel.
commit : 09cf1d52267644cdbdb734294012cf1228745aaa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Jan 2021 18:23:03 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Jan 2021 18:23:03 -0500
A very common usage pattern is that we set a timeout that we don't
expect to reach, cancel it after a little bit, and later repeat.
With the original implementation of timeout.c, this results in one
setitimer() call per timeout set or cancel. We can do a lot better
by being lazy about changing the timeout interrupt request, namely:
(1) never cancel the outstanding interrupt, even when we have no
active timeout events;
(2) if we need to set an interrupt, but there already is one pending
at or before the required time, leave it alone. When the interrupt
happens, the signal handler will reschedule it at whatever time is
then needed.
For example, with a one-second setting for statement_timeout, this
method results in having to interact with the kernel only a little
more than once a second, no matter how many statements we execute
in between. The mainline code might never call setitimer() at all
after the first time, while each time the signal handler fires,
it sees that the then-pending request is most of a second away,
and that's when it sets the next interrupt request for. Each
mainline timeout-set request after that will observe that the time
it wants is past the pending interrupt request time, and do nothing.
This also works pretty well for cases where a few different timeout
lengths are in use, as long as none of them are very short. But
that describes our usage well.
Idea and original patch by Thomas Munro; I fixed a race condition
and improved the comments.
Discussion: https://postgr.es/m/CA+hUKG+o6pbuHBJSGnud=TadsuXySWA7CCcPgCt2QE9F6_4iHQ@mail.gmail.com
M src/backend/utils/misc/timeout.c
Report progress of COPY commands
commit : 8a4f618e7ae3cb11b0b37d0f06f05c8ff905833f
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 6 Jan 2021 21:46:26 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 6 Jan 2021 21:46:26 +0100
This commit introduces a view pg_stat_progress_copy, reporting progress
of COPY commands. This allows rough estimates how far a running COPY
progressed, with the caveat that the total number of bytes may not be
available in some cases (e.g. when the input comes from the client).
Author: Josef Šimánek
Reviewed-by: Fujii Masao, Bharath Rupireddy, Vignesh C, Matthias van de Meent
Discussion: https://postgr.es/m/CAFp7QwqMGEi4OyyaLEK9DR0+E+oK3UtA4bEjDVCa4bNkwUY2PQ@mail.gmail.com
Discussion: https://postgr.es/m/CAFp7Qwr6_FmRM6pCO0x_a0mymOfX_Gg+FEKet4XaTGSW=LitKQ@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/commands/copyfrom.c
M src/backend/commands/copyfromparse.c
M src/backend/commands/copyto.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/commands/copyfrom_internal.h
M src/include/commands/progress.h
M src/include/pgstat.h
M src/test/regress/expected/rules.out
Add a test module for the regular expression package.
commit : ca8217c10138fa3ffe1e7d1def2484fd0eb78226
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Jan 2021 10:51:14 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 6 Jan 2021 10:51:14 -0500
This module provides a function test_regex() that is functionally
rather like regexp_matches(), but with additional debugging-oriented
options and additional output. The debug options are somewhat obscure;
they are chosen to match the API of the test harness that Henry Spencer
wrote way-back-when for use in Tcl. With this, we can import all the
test cases that Spencer wrote originally, even for regex functionality
that we don't currently expose in Postgres. This seems necessary
because we can no longer rely on Tcl to act as upstream and verify
any fixes or improvements that we make.
In addition to Spencer's tests, I added a few for lookbehind
constraints (which we added in 2015, and Tcl still hasn't absorbed)
that are modeled on his tests for lookahead constraints. After looking
at code coverage reports, I also threw in a couple of tests to more
fully exercise our "high colormap" logic.
According to my testing, this brings the check-world coverage
for src/backend/regex/ from 71.1% to 86.7% of lines.
(coverage.postgresql.org shows a slightly different number,
which I think is because it measures a non-assert build.)
Discussion: https://postgr.es/m/2873268.1609732164@sss.pgh.pa.us
M src/test/modules/Makefile
A src/test/modules/test_regex/.gitignore
A src/test/modules/test_regex/Makefile
A src/test/modules/test_regex/README
A src/test/modules/test_regex/expected/test_regex.out
A src/test/modules/test_regex/expected/test_regex_utf8.out
A src/test/modules/test_regex/expected/test_regex_utf8_1.out
A src/test/modules/test_regex/sql/test_regex.sql
A src/test/modules/test_regex/sql/test_regex_utf8.sql
A src/test/modules/test_regex/test_regex–1.0.sql
A src/test/modules/test_regex/test_regex.c
A src/test/modules/test_regex/test_regex.control
Replace CLOBBER_CACHE_ALWAYS with run-time GUC
commit : 4656e3d66893f286767285cf74dabb3877068e49
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 6 Jan 2021 10:15:19 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 6 Jan 2021 10:15:19 +0100
Forced cache invalidation (CLOBBER_CACHE_ALWAYS) has been impractical
to use for testing in PostgreSQL because it's so slow and because it's
toggled on/off only at build time. It is helpful when hunting bugs in
any code that uses the sycache/relcache because causes cache
invalidations to be injected whenever it would be possible for an
invalidation to occur, whether or not one was really pending.
Address this by providing run-time control over cache clobber
behaviour using the new debug_invalidate_system_caches_always GUC.
Support is not compiled in at all unless assertions are enabled or
CLOBBER_CACHE_ENABLED is explicitly defined at compile time. It
defaults to 0 if compiled in, so it has negligible effect on assert
build performance by default.
When support is compiled in, test code can now set
debug_invalidate_system_caches_always=1 locally to a backend to test
specific queries, functions, extensions, etc. Or tests can toggle it
globally for a specific test case while retaining normal performance
during test setup and teardown.
For backwards compatibility with existing test harnesses and scripts,
debug_invalidate_system_caches_always defaults to 1 if
CLOBBER_CACHE_ALWAYS is defined, and to 3 if CLOBBER_CACHE_RECURSIVE
is defined.
CLOBBER_CACHE_ENABLED is now visible in pg_config_manual.h, as is the
related RECOVER_RELATION_BUILD_MEMORY setting for the relcache.
Author: Craig Ringer <craig.ringer@2ndquadrant.com>
Discussion: https://www.postgresql.org/message-id/flat/CAMsr+YF=+ctXBZj3ywmvKNUjWpxmuTuUKuv-rgbHGX5i5pLstQ@mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/regress.sgml
M src/backend/utils/adt/lockfuncs.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/misc/guc.c
M src/include/pg_config_manual.h
M src/include/utils/inval.h
Detect the deadlocks between backends and the startup process.
commit : 8900b5a9d59a645b3485f5b046c4c7871b2c4026
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 6 Jan 2021 12:39:18 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 6 Jan 2021 12:39:18 +0900
The deadlocks that the recovery conflict on lock is involved in can
happen between hot-standby backends and the startup process.
If a backend takes an access exclusive lock on the table and which
finally triggers the deadlock, that deadlock can be detected
as expected. On the other hand, previously, if the startup process
took an access exclusive lock and which finally triggered the deadlock,
that deadlock could not be detected and could remain even after
deadlock_timeout passed. This is a bug.
The cause of this bug was that the code for handling the recovery
conflict on lock didn't take care of deadlock case at all. It assumed
that deadlocks involving the startup process and backends were able
to be detected by the deadlock detector invoked within backends.
But this assumption was incorrect. The startup process also should
have invoked the deadlock detector if necessary.
To fix this bug, this commit makes the startup process invoke
the deadlock detector if deadlock_timeout is reached while handling
the recovery conflict on lock. Specifically, in that case, the startup
process requests all the backends holding the conflicting locks to
check themselves for deadlocks.
Back-patch to v9.6. v9.5 has also this bug, but per discussion we decided
not to back-patch the fix to v9.5. Because v9.5 doesn't have some
infrastructure codes (e.g., 37c54863cf) that this bug fix patch depends on.
We can apply those codes for the back-patch, but since the next minor
version release is the final one for v9.5, it's risky to do that. If we
unexpectedly introduce new bug to v9.5 by the back-patch, there is no
chance to fix that. We determined that the back-patch to v9.5 would give
more risk than gain.
Author: Fujii Masao
Reviewed-by: Bertrand Drouvot, Masahiko Sawada, Kyotaro Horiguchi
Discussion: https://postgr.es/m/4041d6b6-cf24-a120-36fa-1294220f8243@oss.nttdata.com
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/proc.c
M src/backend/tcop/postgres.c
M src/include/storage/procarray.h
Fix typos in decode.c and logical.c.
commit : e02e840ff7787d4798fca9f6a5b8b3657f45cf0c
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 6 Jan 2021 08:56:19 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 6 Jan 2021 08:56:19 +0530
Per report by Ajin Cherian in email:
https://postgr.es/m/CAFPTHDYnRKDvzgDxoMn_CKqXA-D0MtrbyJvfvjBsO4G=UHDXkg@mail.gmail.com
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/logical.c
doc: Fix description about default behavior of recovery_target_timeline.
commit : 25dde5835772a58356383bf1112c6c2a1a37df0e
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 6 Jan 2021 11:58:23 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 6 Jan 2021 11:58:23 +0900
The default value of recovery_target_timeline was changed in v12,
but the description about the default behavior of that was not updated.
Back-patch to v12 where the default behavior of recovery_target_timeline
was changed.
Author: Benoit Lobréau
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAPE8EZ7c3aruEmM24GYkj8y8WmHKD1m9TtPtgCF0nQ3zw4LCkQ@mail.gmail.com
M doc/src/sgml/backup.sgml
Promote --data-checksums to the common set of options in initdb --help
commit : bc08f7971c03550ede43af43e73d4a47a7935c69
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 6 Jan 2021 10:52:26 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 6 Jan 2021 10:52:26 +0900
This was previously part of the section dedicated to less common
options, but it is an option commonly used these days.
Author: Michael Banck
Reviewed-by: Stephen Frost, Michael Paquier
Discussion: https://postgr.es/m/d7938aca4d4ea8e8c72c33bd75efe9f8218fe390.camel@credativ.de
M src/bin/initdb/initdb.c
Revert unstable test cases from commit 7d80441d2.
commit : 14d49f483d4c8a5a356e25d5e5ff5726ca43abff
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 5 Jan 2021 19:03:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 5 Jan 2021 19:03:56 -0500
I momentarily forgot that the "owner" column wouldn't be stable
in the buildfarm. Oh well, these tests weren't very valuable
anyway.
Discussion: https://postgr.es/m/20201130165436.GX24052@telsasoft.com
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
Allow psql's \dt and \di to show TOAST tables and their indexes.
commit : 7d80441d2c8de5cd5d593e302bd14e8b19ee92d4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 5 Jan 2021 18:41:50 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 5 Jan 2021 18:41:50 -0500
Formerly, TOAST objects were unconditionally suppressed, but since
\d is able to print them it's not very clear why these variants
should not. Instead, use the same rules as for system catalogs:
they can be seen if you write the 'S' modifier or a table name
pattern. (In practice, since hardly anybody would keep pg_toast
in their search_path, it's really down to whether you use a pattern
that can match pg_toast.*.)
No docs change seems necessary because the docs already say that
this happens for "system objects"; we're just classifying TOAST
tables as being that.
Justin Pryzby, reviewed by Laurenz Albe
Discussion: https://postgr.es/m/20201130165436.GX24052@telsasoft.com
M src/bin/psql/describe.c
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
Introduce a new GUC_REPORT setting "in_hot_standby".
commit : bf8a662c9afad6fd07b42cdc5e71416c51f75d31
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 5 Jan 2021 16:18:01 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 5 Jan 2021 16:18:01 -0500
Aside from being queriable via SHOW, this value is sent to the client
immediately at session startup, and again later on if the server gets
promoted to primary during the session. The immediate report will be
used in an upcoming patch to avoid an extra round trip when trying to
connect to a primary server.
Haribabu Kommi, Greg Nancarrow, Tom Lane; reviewed at various times
by Laurenz Albe, Takayuki Tsunakawa, Peter Smith.
Discussion: https://postgr.es/m/CAF3+xM+8-ztOkaV9gHiJ3wfgENTq97QcjXQt+rbFQ6F7oNzt9A@mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/protocol.sgml
M src/backend/utils/misc/check_guc
M src/backend/utils/misc/guc.c
doc: improve NLS instruction wording
commit : 47b2ed1d06c1218eee8869c8eca8bf002b9d43a6
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 5 Jan 2021 14:26:37 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 5 Jan 2021 14:26:37 -0500
Reported-by: "Tang, Haiying"
Discussion: https://postgr.es/m/bbbccf7a3c2d436e85d45869d612fd6b@G08CNEXMBPEKD05.g08.fujitsu.local
Author: "Tang, Haiying"
Backpatch-through: 9.5
M doc/src/sgml/nls.sgml
Add an explicit cast to double when using fabs().
commit : fead67c24ada8c6a4b661dec6f159dca1447e3d8
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 5 Jan 2021 11:52:42 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 5 Jan 2021 11:52:42 +0000
Commit bc43b7c2c0 used fabs() directly on an int variable, which
apparently requires an explicit cast on some platforms.
Per buildfarm.
M src/backend/utils/adt/numeric.c
Fix numeric_power() when the exponent is INT_MIN.
commit : bc43b7c2c06c32264efe79d0b86abd41236f1d5b
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 5 Jan 2021 11:15:28 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 5 Jan 2021 11:15:28 +0000
In power_var_int(), the computation of the number of significant
digits to use in the computation used log(Abs(exp)), which isn't safe
because Abs(exp) returns INT_MIN when exp is INT_MIN. Use fabs()
instead of Abs(), so that the exponent is cast to a double before the
absolute value is taken.
Back-patch to 9.6, where this was introduced (by 7d9a4737c2).
Discussion: https://postgr.es/m/CAEZATCVd6pMkz=BrZEgBKyqqJrt2xghr=fNc8+Z=5xC6cgWrWA@mail.gmail.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Test decoding of two-phase transactions during the build of a consistent snapshot.
commit : 5a3574d7b33480de51692962e6fd25d00c05718a
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 5 Jan 2021 15:05:22 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 5 Jan 2021 15:05:22 +0530
Commit a271a1b50e added the capability to allow decoding at prepare time.
This adds an isolation testcase to test that decoding happens at commit
time when the consistent snapshot state is reached after prepare but
before commit prepared.
Author: Ajin Cherian
Reviewed-by: Amit Kapila
Discussion:
https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru
https://postgr.es/m/CAMGcDxeqEpWj3fTXwqhSwBdXd2RS9jzwWscO-XbeCfso6ts3+Q@mail.gmail.com
M contrib/test_decoding/Makefile
A contrib/test_decoding/expected/twophase_snapshot.out
A contrib/test_decoding/specs/twophase_snapshot.spec
doc: Document how to run regression tests with custom server settings
commit : 854434c50a35e7cb2ae22588c80ebf36e889197a
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 5 Jan 2021 08:17:40 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 5 Jan 2021 08:17:40 +0100
Author: Craig Ringer <craig.ringer@2ndquadrant.com>
Discussion: https://www.postgresql.org/message-id/flat/CAMsr+YF=+ctXBZj3ywmvKNUjWpxmuTuUKuv-rgbHGX5i5pLstQ@mail.gmail.com
M doc/src/sgml/regress.sgml
Standardize one aspect of rmgr desc output.
commit : 83e3239ee77089f76fad022eeef91c83cdf9181d
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 4 Jan 2021 19:46:11 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 4 Jan 2021 19:46:11 -0800
Bring heap and hash rmgr desc output in line with nbtree and GiST desc
output by using the name latestRemovedXid for all fields that output the
contents of the latestRemovedXid field from the WAL record's C struct
(stop using local variants).
This seems like a clear improvement because latestRemovedXid is a symbol
name that already appears across many different source files, and so is
probably much more recognizable.
Discussion: https://postgr.es/m/CAH2-Wzkt_Rs4VqPSCk87nyjPAAEmWL8STU9zgET_83EF5YfrLw@mail.gmail.com
M src/backend/access/rmgrdesc/hashdesc.c
M src/backend/access/rmgrdesc/heapdesc.c
Fix typo in origin.c.
commit : cd357c76296e987298e4005b8053adda9e3fb417
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 5 Jan 2021 08:05:08 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 5 Jan 2021 08:05:08 +0530
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+PsReyuvww_Fn1NN_Vsv0wBP1bnzuhzRFr_2=y1nNZrG7w@mail.gmail.com
M src/backend/replication/logical/origin.c
Fix typo in reorderbuffer.c.
commit : 9da2224ea2bb9801afc29bff6a325bf796868bdc
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 5 Jan 2021 07:56:40 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 5 Jan 2021 07:56:40 +0530
Author: Zhijie Hou
Reviewed-by: Sawada Masahiko
Discussion: https://postgr.es/m/ba88bb58aaf14284abca16aec04bf279@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/replication/logical/reorderbuffer.c
Replace remaining uses of "whitelist".
commit : 034510c820cd75e0410332d92b4967ef9b844936
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 5 Jan 2021 14:00:16 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 5 Jan 2021 14:00:16 +1300
Instead describe the action that the list effects, or just use "list"
where the meaning is obvious from context.
Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://postgr.es/m/20200615182235.x7lch5n6kcjq4aue%40alap3.anarazel.de
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/shippable.c
M src/backend/access/hash/hashvalidate.c
M src/backend/utils/adt/lockfuncs.c
M src/tools/pginclude/README
pgindent: whitelist/blacklist -> additional/excluded.
commit : fe05b6b620066aec313c43b6b4d6c169d0a346f7
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 5 Jan 2021 13:27:06 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 5 Jan 2021 13:27:06 +1300
Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://postgr.es/m/20200615182235.x7lch5n6kcjq4aue%40alap3.anarazel.de
M src/tools/pgindent/pgindent
Rename "enum blacklist" to "uncommitted enums".
commit : c0d4f6d897492727d4812679e6a94d12edbe016f
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 5 Jan 2021 12:06:15 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 5 Jan 2021 12:06:15 +1300
We agreed to remove this terminology and use something more descriptive.
Discussion: https://postgr.es/m/20200615182235.x7lch5n6kcjq4aue%40alap3.anarazel.de
M src/backend/access/transam/parallel.c
M src/backend/catalog/pg_enum.c
M src/backend/utils/adt/enum.c
M src/include/catalog/pg_enum.h
Fix integer-overflow corner cases in substring() functions.
commit : 4bd3fad80e5c3bd107583dd9d32d4a47c045a3ec
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Jan 2021 18:32:40 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Jan 2021 18:32:40 -0500
If the substring start index and length overflow when added together,
substring() misbehaved, either throwing a bogus "negative substring
length" error on a case that should succeed, or failing to complain that
a negative length is negative (and instead returning the whole string,
in most cases). Unsurprisingly, the text, bytea, and bit variants of
the function all had this issue. Rearrange the logic to ensure that
negative lengths are always rejected, and add an overflow check to
handle the other case.
Also install similar guards into detoast_attr_slice() (nee
heap_tuple_untoast_attr_slice()), since it's far from clear that
no other code paths leading to that function could pass it values
that would overflow.
Patch by myself and Pavel Stehule, per bug #16804 from Rafi Shamim.
Back-patch to v11. While these bugs are old, the common/int.h
infrastructure for overflow-detecting arithmetic didn't exist before
commit 4d6ad3125, and it doesn't seem like these misbehaviors are bad
enough to justify developing a standalone fix for the older branches.
Discussion: https://postgr.es/m/16804-f4eeeb6c11ba71d4@postgresql.org
M src/backend/access/common/detoast.c
M src/backend/utils/adt/varbit.c
M src/backend/utils/adt/varlena.c
M src/test/regress/expected/bit.out
M src/test/regress/expected/strings.out
M src/test/regress/sql/bit.sql
M src/test/regress/sql/strings.sql
Remove unused function prototypes.
commit : 87c23d36a3bc81e57b813f13c403f74a67ff33a9
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 5 Jan 2021 11:40:03 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 5 Jan 2021 11:40:03 +1300
Cleanup for commit dee663f7.
Reported-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://postgr.es/m/CA+hUKGLJ=84YT+NvhkEEDAuUtVHMfQ9i-N7k_o50JmQ6Rpj_OQ@mail.gmail.com
M src/include/access/clog.h
M src/include/access/commit_ts.h
M src/include/access/multixact.h
M src/include/access/subtrans.h
Rethink the "read/write parameter" mechanism in pl/pgsql.
commit : 1c1cbe279b3c6e8038c8f8079402f069bb4cea4c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Jan 2021 12:39:27 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Jan 2021 12:39:27 -0500
Performance issues with the preceding patch to re-implement array
element assignment within pl/pgsql led me to realize that the read/write
parameter mechanism is misdesigned. Instead of requiring the assignment
source expression to be such that *all* its references to the target
variable could be passed as R/W, we really want to identify *one*
reference to the target variable to be passed as R/W, allowing any other
ones to be passed read/only as they would be by default. As long as the
R/W reference is a direct argument to the top-level (hence last to be
executed) function in the expression, there is no harm in R/O references
being passed to other lower parts of the expression. Nor is there any
use-case for more than one argument of the top-level function being R/W.
Hence, rewrite that logic to identify one single Param that references
the target variable, and make only that Param pass a read/write
reference, not any other Params referencing the target variable.
Discussion: https://postgr.es/m/4165684.1607707277@sss.pgh.pa.us
M src/backend/utils/adt/arrayfuncs.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
Remove PLPGSQL_DTYPE_ARRAYELEM datum type within pl/pgsql.
commit : 1788828d33516809fa2d842bf6e273d78e21d957
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Jan 2021 12:14:37 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Jan 2021 12:14:37 -0500
In the wake of the previous commit, we don't really need this anymore,
since array assignment is primarily handled by the core code.
The only way that that code could still be reached is that a GET
DIAGNOSTICS target variable could be an array element. But that
doesn't seem like a particularly essential feature. I'd added it
in commit 55caaaeba, but just because it was easy not because
anyone had actually asked for it. Hence, revert that patch and
then remove the now-unreachable stuff. (If we really had to,
we could probably reimplement GET DIAGNOSTICS using the new
assignment machinery; but the cost/benefit ratio looks very poor,
and it'd likely be a bit slower.)
Note that PLPGSQL_DTYPE_RECFIELD remains. It's possible that we
could get rid of that too, but maintaining the existing behaviors
for RECORD-type variables seems like it might be difficult. Since
there's not any functional limitation in those code paths as there
was in the ARRAYELEM code, I've not pursued the idea.
Discussion: https://postgr.es/m/4165684.1607707277@sss.pgh.pa.us
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_funcs.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
M src/test/regress/expected/plpgsql.out
M src/test/regress/sql/plpgsql.sql
Re-implement pl/pgsql's expression and assignment parsing.
commit : c9d5298485b78a37923a23f9af9aa0ade06762db
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Jan 2021 11:52:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Jan 2021 11:52:00 -0500
Invent new RawParseModes that allow the core grammar to handle
pl/pgsql expressions and assignments directly, and thereby get rid
of a lot of hackery in pl/pgsql's parser. This moves a good deal
of knowledge about pl/pgsql into the core code: notably, we have to
invent a CoercionContext that matches pl/pgsql's (rather dubious)
historical behavior for assignment coercions. That's getting away
from the original idea of pl/pgsql as an arm's-length extension of
the core, but really we crossed that bridge a long time ago.
The main advantage of doing this is that we can now use the core
parser to generate FieldStore and/or SubscriptingRef nodes to handle
assignments to pl/pgsql variables that are records or arrays. That
fixes a number of cases that had never been implemented in pl/pgsql
assignment, such as nested records and array slicing, and it allows
pl/pgsql assignment to support the datatype-specific subscripting
behaviors introduced in commit c7aba7c14.
There are cosmetic benefits too: when a syntax error occurs in a
pl/pgsql expression, the error report no longer includes the confusing
"SELECT" keyword that used to get prefixed to the expression text.
Also, there seem to be some small speed gains.
Discussion: https://postgr.es/m/4165684.1607707277@sss.pgh.pa.us
M contrib/hstore/expected/hstore.out
M contrib/hstore/sql/hstore.sql
M doc/src/sgml/plpgsql.sgml
M src/backend/commands/functioncmds.c
M src/backend/executor/spi.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/parser/parse_coerce.c
M src/backend/parser/parse_target.c
M src/backend/parser/parser.c
M src/backend/tcop/utility.c
M src/include/nodes/nodes.h
M src/include/nodes/parsenodes.h
M src/include/nodes/primnodes.h
M src/include/parser/parse_target.h
M src/include/parser/parser.h
M src/interfaces/ecpg/preproc/parse.pl
M src/pl/plpgsql/src/Makefile
A src/pl/plpgsql/src/expected/plpgsql_array.out
M src/pl/plpgsql/src/expected/plpgsql_record.out
M src/pl/plpgsql/src/expected/plpgsql_varprops.out
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
A src/pl/plpgsql/src/sql/plpgsql_array.sql
M src/pl/plpgsql/src/sql/plpgsql_record.sql
M src/test/regress/expected/plpgsql.out
Add the ability for the core grammar to have more than one parse target.
commit : 844fe9f159a948377907a63d0ef3fb16dc51ce50
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Jan 2021 11:03:22 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Jan 2021 11:03:22 -0500
This patch essentially allows gram.y to implement a family of related
syntax trees, rather than necessarily always parsing a list of SQL
statements. raw_parser() gains a new argument, enum RawParseMode,
to say what to do. As proof of concept, add a mode that just parses
a TypeName without any other decoration, and use that to greatly
simplify typeStringToTypeName().
In addition, invent a new SPI entry point SPI_prepare_extended() to
allow SPI users (particularly plpgsql) to get at this new functionality.
In hopes of making this the last variant of SPI_prepare(), set up its
additional arguments as a struct rather than direct arguments, and
promise that future additions to the struct can default to zero.
SPI_prepare_cursor() and SPI_prepare_params() can perhaps go away at
some point.
Discussion: https://postgr.es/m/4165684.1607707277@sss.pgh.pa.us
M doc/src/sgml/spi.sgml
M src/backend/commands/tablecmds.c
M src/backend/executor/spi.c
M src/backend/parser/gram.y
M src/backend/parser/parse_coerce.c
M src/backend/parser/parse_type.c
M src/backend/parser/parser.c
M src/backend/tcop/postgres.c
M src/include/executor/spi.h
M src/include/executor/spi_priv.h
M src/include/parser/parser.h
M src/interfaces/ecpg/preproc/parse.pl
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
Simplify some comments in xml.c
commit : b49154b3b7a45523ce4081fdae8d65049342fcec
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 4 Jan 2021 19:47:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 4 Jan 2021 19:47:58 +0900
Author: Justin Pryzby
Discussion: https://postgr.es/m/X/Ff7jfnvJUab013@paquier.xyz
M src/backend/utils/adt/xml.c
Allow decoding at prepare time in ReorderBuffer.
commit : a271a1b50e9bec07e2ef3a05e38e7285113e4ce6
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 4 Jan 2021 08:34:50 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 4 Jan 2021 08:34:50 +0530
This patch allows PREPARE-time decoding of two-phase transactions (if the
output plugin supports this capability), in which case the transactions
are replayed at PREPARE and then committed later when COMMIT PREPARED
arrives.
Now that we decode the changes before the commit, the concurrent aborts
may cause failures when the output plugin consults catalogs (both system
and user-defined).
We detect such failures with a special sqlerrcode
ERRCODE_TRANSACTION_ROLLBACK introduced by commit 7259736a6e and stop
decoding the remaining changes. Then we rollback the changes when rollback
prepared is encountered.
Author: Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Peter Smith, Sawada Masahiko, Arseny Sher, and Dilip Kumar
Tested-by: Takamichi Osumi
Discussion:
https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru
https://postgr.es/m/CAMGcDxeqEpWj3fTXwqhSwBdXd2RS9jzwWscO-XbeCfso6ts3+Q@mail.gmail.com
M contrib/test_decoding/Makefile
A contrib/test_decoding/expected/twophase.out
A contrib/test_decoding/expected/twophase_stream.out
A contrib/test_decoding/sql/twophase.sql
A contrib/test_decoding/sql/twophase_stream.sql
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/include/replication/reorderbuffer.h
Update copyright for 2021
commit : ca3b37487be333a1d241dab1bbdd17a211a88f43
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 2 Jan 2021 13:06:25 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 2 Jan 2021 13:06:25 -0500
Backpatch-through: 9.5
M COPYRIGHT
M config/thread_test.c
M configure
M configure.ac
M contrib/adminpack/adminpack.c
M contrib/amcheck/verify_heapam.c
M contrib/amcheck/verify_nbtree.c
M contrib/auth_delay/auth_delay.c
M contrib/auto_explain/auto_explain.c
M contrib/bloom/blcost.c
M contrib/bloom/blinsert.c
M contrib/bloom/bloom.h
M contrib/bloom/blscan.c
M contrib/bloom/blutils.c
M contrib/bloom/blvacuum.c
M contrib/bloom/blvalidate.c
M contrib/dblink/dblink.c
M contrib/dict_int/dict_int.c
M contrib/dict_xsyn/dict_xsyn.c
M contrib/file_fdw/file_fdw.c
M contrib/fuzzystrmatch/fuzzystrmatch.c
M contrib/hstore/hstore_subs.c
M contrib/intarray/_int_selfuncs.c
M contrib/isn/isn.c
M contrib/isn/isn.h
M contrib/old_snapshot/time_mapping.c
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/fsmfuncs.c
M contrib/pageinspect/ginfuncs.c
M contrib/pageinspect/hashfuncs.c
M contrib/pageinspect/heapfuncs.c
M contrib/pageinspect/pageinspect.h
M contrib/pageinspect/rawpage.c
M contrib/passwordcheck/passwordcheck.c
M contrib/pg_prewarm/autoprewarm.c
M contrib/pg_prewarm/pg_prewarm.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_surgery/heap_surgery.c
M contrib/pg_surgery/pg_surgery–1.0.sql
M contrib/pg_trgm/trgm_regexp.c
M contrib/pg_visibility/pg_visibility.c
M contrib/pgcrypto/imath.c
M contrib/pgstattuple/pgstatapprox.c
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/shippable.c
M contrib/sepgsql/database.c
M contrib/sepgsql/dml.c
M contrib/sepgsql/hooks.c
M contrib/sepgsql/label.c
M contrib/sepgsql/launcher
M contrib/sepgsql/proc.c
M contrib/sepgsql/relation.c
M contrib/sepgsql/schema.c
M contrib/sepgsql/selinux.c
M contrib/sepgsql/sepgsql.h
M contrib/sepgsql/uavc.c
M contrib/tablefunc/tablefunc.c
M contrib/tablefunc/tablefunc.h
M contrib/tcn/tcn.c
M contrib/test_decoding/test_decoding.c
M contrib/tsm_system_rows/tsm_system_rows.c
M contrib/tsm_system_time/tsm_system_time.c
M contrib/unaccent/unaccent.c
M contrib/uuid-ossp/uuid-ossp.c
M contrib/vacuumlo/vacuumlo.c
M doc/src/sgml/generate-errcodes-table.pl
M doc/src/sgml/generate-keywords-table.pl
M doc/src/sgml/legal.sgml
M doc/src/sgml/lobj.sgml
M src/backend/Makefile
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_inclusion.c
M src/backend/access/brin/brin_minmax.c
M src/backend/access/brin/brin_pageops.c
M src/backend/access/brin/brin_revmap.c
M src/backend/access/brin/brin_tuple.c
M src/backend/access/brin/brin_validate.c
M src/backend/access/brin/brin_xlog.c
M src/backend/access/common/attmap.c
M src/backend/access/common/bufmask.c
M src/backend/access/common/detoast.c
M src/backend/access/common/heaptuple.c
M src/backend/access/common/indextuple.c
M src/backend/access/common/printsimple.c
M src/backend/access/common/printtup.c
M src/backend/access/common/relation.c
M src/backend/access/common/reloptions.c
M src/backend/access/common/scankey.c
M src/backend/access/common/session.c
M src/backend/access/common/syncscan.c
M src/backend/access/common/toast_internals.c
M src/backend/access/common/tupconvert.c
M src/backend/access/common/tupdesc.c
M src/backend/access/gin/ginarrayproc.c
M src/backend/access/gin/ginbtree.c
M src/backend/access/gin/ginbulk.c
M src/backend/access/gin/gindatapage.c
M src/backend/access/gin/ginentrypage.c
M src/backend/access/gin/ginfast.c
M src/backend/access/gin/ginget.c
M src/backend/access/gin/gininsert.c
M src/backend/access/gin/ginlogic.c
M src/backend/access/gin/ginpostinglist.c
M src/backend/access/gin/ginscan.c
M src/backend/access/gin/ginutil.c
M src/backend/access/gin/ginvacuum.c
M src/backend/access/gin/ginvalidate.c
M src/backend/access/gin/ginxlog.c
M src/backend/access/gist/gist.c
M src/backend/access/gist/gistbuild.c
M src/backend/access/gist/gistbuildbuffers.c
M src/backend/access/gist/gistget.c
M src/backend/access/gist/gistproc.c
M src/backend/access/gist/gistscan.c
M src/backend/access/gist/gistsplit.c
M src/backend/access/gist/gistutil.c
M src/backend/access/gist/gistvacuum.c
M src/backend/access/gist/gistvalidate.c
M src/backend/access/gist/gistxlog.c
M src/backend/access/hash/hash.c
M src/backend/access/hash/hash_xlog.c
M src/backend/access/hash/hashfunc.c
M src/backend/access/hash/hashinsert.c
M src/backend/access/hash/hashovfl.c
M src/backend/access/hash/hashpage.c
M src/backend/access/hash/hashsearch.c
M src/backend/access/hash/hashsort.c
M src/backend/access/hash/hashutil.c
M src/backend/access/hash/hashvalidate.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/heap/heapam_visibility.c
M src/backend/access/heap/heaptoast.c
M src/backend/access/heap/hio.c
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/rewriteheap.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/heap/visibilitymap.c
M src/backend/access/index/amapi.c
M src/backend/access/index/amvalidate.c
M src/backend/access/index/genam.c
M src/backend/access/index/indexam.c
M src/backend/access/nbtree/nbtcompare.c
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/access/nbtree/nbtsplitloc.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/access/nbtree/nbtvalidate.c
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/rmgrdesc/brindesc.c
M src/backend/access/rmgrdesc/clogdesc.c
M src/backend/access/rmgrdesc/committsdesc.c
M src/backend/access/rmgrdesc/dbasedesc.c
M src/backend/access/rmgrdesc/genericdesc.c
M src/backend/access/rmgrdesc/gindesc.c
M src/backend/access/rmgrdesc/gistdesc.c
M src/backend/access/rmgrdesc/hashdesc.c
M src/backend/access/rmgrdesc/heapdesc.c
M src/backend/access/rmgrdesc/logicalmsgdesc.c
M src/backend/access/rmgrdesc/mxactdesc.c
M src/backend/access/rmgrdesc/nbtdesc.c
M src/backend/access/rmgrdesc/relmapdesc.c
M src/backend/access/rmgrdesc/replorigindesc.c
M src/backend/access/rmgrdesc/seqdesc.c
M src/backend/access/rmgrdesc/smgrdesc.c
M src/backend/access/rmgrdesc/spgdesc.c
M src/backend/access/rmgrdesc/standbydesc.c
M src/backend/access/rmgrdesc/tblspcdesc.c
M src/backend/access/rmgrdesc/xactdesc.c
M src/backend/access/rmgrdesc/xlogdesc.c
M src/backend/access/spgist/spgdoinsert.c
M src/backend/access/spgist/spginsert.c
M src/backend/access/spgist/spgkdtreeproc.c
M src/backend/access/spgist/spgproc.c
M src/backend/access/spgist/spgquadtreeproc.c
M src/backend/access/spgist/spgscan.c
M src/backend/access/spgist/spgtextproc.c
M src/backend/access/spgist/spgutils.c
M src/backend/access/spgist/spgvacuum.c
M src/backend/access/spgist/spgvalidate.c
M src/backend/access/spgist/spgxlog.c
M src/backend/access/table/table.c
M src/backend/access/table/tableam.c
M src/backend/access/table/tableamapi.c
M src/backend/access/table/toast_helper.c
M src/backend/access/tablesample/bernoulli.c
M src/backend/access/tablesample/system.c
M src/backend/access/tablesample/tablesample.c
M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/generic_xlog.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/parallel.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/access/transam/timeline.c
M src/backend/access/transam/transam.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/twophase_rmgr.c
M src/backend/access/transam/varsup.c
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogarchive.c
M src/backend/access/transam/xlogfuncs.c
M src/backend/access/transam/xloginsert.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogutils.c
M src/backend/bootstrap/bootparse.y
M src/backend/bootstrap/bootscanner.l
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/Catalog.pm
M src/backend/catalog/Makefile
M src/backend/catalog/aclchk.c
M src/backend/catalog/catalog.c
M src/backend/catalog/dependency.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/indexing.c
M src/backend/catalog/information_schema.sql
M src/backend/catalog/namespace.c
M src/backend/catalog/objectaccess.c
M src/backend/catalog/objectaddress.c
M src/backend/catalog/partition.c
M src/backend/catalog/pg_aggregate.c
M src/backend/catalog/pg_cast.c
M src/backend/catalog/pg_collation.c
M src/backend/catalog/pg_constraint.c
M src/backend/catalog/pg_conversion.c
M src/backend/catalog/pg_db_role_setting.c
M src/backend/catalog/pg_depend.c
M src/backend/catalog/pg_enum.c
M src/backend/catalog/pg_inherits.c
M src/backend/catalog/pg_largeobject.c
M src/backend/catalog/pg_namespace.c
M src/backend/catalog/pg_operator.c
M src/backend/catalog/pg_proc.c
M src/backend/catalog/pg_publication.c
M src/backend/catalog/pg_range.c
M src/backend/catalog/pg_shdepend.c
M src/backend/catalog/pg_subscription.c
M src/backend/catalog/pg_type.c
M src/backend/catalog/storage.c
M src/backend/catalog/system_views.sql
M src/backend/catalog/toasting.c
M src/backend/commands/aggregatecmds.c
M src/backend/commands/alter.c
M src/backend/commands/amcmds.c
M src/backend/commands/analyze.c
M src/backend/commands/async.c
M src/backend/commands/cluster.c
M src/backend/commands/collationcmds.c
M src/backend/commands/comment.c
M src/backend/commands/constraint.c
M src/backend/commands/conversioncmds.c
M src/backend/commands/copy.c
M src/backend/commands/copyfrom.c
M src/backend/commands/copyfromparse.c
M src/backend/commands/copyto.c
M src/backend/commands/createas.c
M src/backend/commands/dbcommands.c
M src/backend/commands/define.c
M src/backend/commands/discard.c
M src/backend/commands/dropcmds.c
M src/backend/commands/event_trigger.c
M src/backend/commands/explain.c
M src/backend/commands/extension.c
M src/backend/commands/foreigncmds.c
M src/backend/commands/functioncmds.c
M src/backend/commands/indexcmds.c
M src/backend/commands/lockcmds.c
M src/backend/commands/matview.c
M src/backend/commands/opclasscmds.c
M src/backend/commands/operatorcmds.c
M src/backend/commands/policy.c
M src/backend/commands/portalcmds.c
M src/backend/commands/prepare.c
M src/backend/commands/proclang.c
M src/backend/commands/publicationcmds.c
M src/backend/commands/schemacmds.c
M src/backend/commands/seclabel.c
M src/backend/commands/sequence.c
M src/backend/commands/statscmds.c
M src/backend/commands/subscriptioncmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/tablespace.c
M src/backend/commands/trigger.c
M src/backend/commands/tsearchcmds.c
M src/backend/commands/typecmds.c
M src/backend/commands/user.c
M src/backend/commands/vacuum.c
M src/backend/commands/variable.c
M src/backend/commands/view.c
M src/backend/executor/execAmi.c
M src/backend/executor/execCurrent.c
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/execGrouping.c
M src/backend/executor/execIndexing.c
M src/backend/executor/execJunk.c
M src/backend/executor/execMain.c
M src/backend/executor/execParallel.c
M src/backend/executor/execPartition.c
M src/backend/executor/execProcnode.c
M src/backend/executor/execReplication.c
M src/backend/executor/execSRF.c
M src/backend/executor/execScan.c
M src/backend/executor/execTuples.c
M src/backend/executor/execUtils.c
M src/backend/executor/functions.c
M src/backend/executor/instrument.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeBitmapAnd.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/backend/executor/nodeBitmapIndexscan.c
M src/backend/executor/nodeBitmapOr.c
M src/backend/executor/nodeCtescan.c
M src/backend/executor/nodeCustom.c
M src/backend/executor/nodeForeignscan.c
M src/backend/executor/nodeFunctionscan.c
M src/backend/executor/nodeGather.c
M src/backend/executor/nodeGatherMerge.c
M src/backend/executor/nodeGroup.c
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeHashjoin.c
M src/backend/executor/nodeIncrementalSort.c
M src/backend/executor/nodeIndexonlyscan.c
M src/backend/executor/nodeIndexscan.c
M src/backend/executor/nodeLimit.c
M src/backend/executor/nodeLockRows.c
M src/backend/executor/nodeMaterial.c
M src/backend/executor/nodeMergeAppend.c
M src/backend/executor/nodeMergejoin.c
M src/backend/executor/nodeModifyTable.c
M src/backend/executor/nodeNamedtuplestorescan.c
M src/backend/executor/nodeNestloop.c
M src/backend/executor/nodeProjectSet.c
M src/backend/executor/nodeRecursiveunion.c
M src/backend/executor/nodeResult.c
M src/backend/executor/nodeSamplescan.c
M src/backend/executor/nodeSeqscan.c
M src/backend/executor/nodeSetOp.c
M src/backend/executor/nodeSort.c
M src/backend/executor/nodeSubplan.c
M src/backend/executor/nodeSubqueryscan.c
M src/backend/executor/nodeTableFuncscan.c
M src/backend/executor/nodeTidscan.c
M src/backend/executor/nodeUnique.c
M src/backend/executor/nodeValuesscan.c
M src/backend/executor/nodeWindowAgg.c
M src/backend/executor/nodeWorktablescan.c
M src/backend/executor/spi.c
M src/backend/executor/tqueue.c
M src/backend/executor/tstoreReceiver.c
M src/backend/foreign/foreign.c
M src/backend/jit/jit.c
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_deform.c
M src/backend/jit/llvm/llvmjit_error.cpp
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/jit/llvm/llvmjit_inline.cpp
M src/backend/jit/llvm/llvmjit_types.c
M src/backend/jit/llvm/llvmjit_wrap.cpp
M src/backend/lib/binaryheap.c
M src/backend/lib/bipartite_match.c
M src/backend/lib/bloomfilter.c
M src/backend/lib/dshash.c
M src/backend/lib/hyperloglog.c
M src/backend/lib/ilist.c
M src/backend/lib/integerset.c
M src/backend/lib/knapsack.c
M src/backend/lib/pairingheap.c
M src/backend/lib/rbtree.c
M src/backend/libpq/auth-scram.c
M src/backend/libpq/auth.c
M src/backend/libpq/be-fsstubs.c
M src/backend/libpq/be-gssapi-common.c
M src/backend/libpq/be-secure-common.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/libpq/be-secure-openssl.c
M src/backend/libpq/be-secure.c
M src/backend/libpq/crypt.c
M src/backend/libpq/hba.c
M src/backend/libpq/ifaddr.c
M src/backend/libpq/pqcomm.c
M src/backend/libpq/pqformat.c
M src/backend/libpq/pqmq.c
M src/backend/libpq/pqsignal.c
M src/backend/main/main.c
M src/backend/nodes/bitmapset.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/extensible.c
M src/backend/nodes/list.c
M src/backend/nodes/makefuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/nodes.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/params.c
M src/backend/nodes/print.c
M src/backend/nodes/read.c
M src/backend/nodes/readfuncs.c
M src/backend/nodes/tidbitmap.c
M src/backend/nodes/value.c
M src/backend/optimizer/geqo/geqo_copy.c
M src/backend/optimizer/geqo/geqo_eval.c
M src/backend/optimizer/geqo/geqo_main.c
M src/backend/optimizer/geqo/geqo_misc.c
M src/backend/optimizer/geqo/geqo_pool.c
M src/backend/optimizer/geqo/geqo_random.c
M src/backend/optimizer/geqo/geqo_selection.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/clausesel.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/path/joinrels.c
M src/backend/optimizer/path/pathkeys.c
M src/backend/optimizer/path/tidpath.c
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/planagg.c
M src/backend/optimizer/plan/planmain.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepagg.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/optimizer/prep/prepqual.c
M src/backend/optimizer/prep/preptlist.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/appendinfo.c
M src/backend/optimizer/util/clauses.c
M src/backend/optimizer/util/inherit.c
M src/backend/optimizer/util/joininfo.c
M src/backend/optimizer/util/orclauses.c
M src/backend/optimizer/util/paramassign.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/placeholder.c
M src/backend/optimizer/util/plancat.c
M src/backend/optimizer/util/predtest.c
M src/backend/optimizer/util/relnode.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/optimizer/util/tlist.c
M src/backend/optimizer/util/var.c
M src/backend/parser/analyze.c
M src/backend/parser/check_keywords.pl
M src/backend/parser/gram.y
M src/backend/parser/parse_agg.c
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_coerce.c
M src/backend/parser/parse_collate.c
M src/backend/parser/parse_cte.c
M src/backend/parser/parse_enr.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_func.c
M src/backend/parser/parse_node.c
M src/backend/parser/parse_oper.c
M src/backend/parser/parse_param.c
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_target.c
M src/backend/parser/parse_type.c
M src/backend/parser/parse_utilcmd.c
M src/backend/parser/parser.c
M src/backend/parser/scan.l
M src/backend/parser/scansup.c
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partdesc.c
M src/backend/partitioning/partprune.c
M src/backend/port/atomics.c
M src/backend/port/posix_sema.c
M src/backend/port/sysv_sema.c
M src/backend/port/sysv_shmem.c
M src/backend/port/tas/sunstudio_sparc.s
M src/backend/port/tas/sunstudio_x86.s
M src/backend/port/win32/crashdump.c
M src/backend/port/win32/signal.c
M src/backend/port/win32/socket.c
M src/backend/port/win32/timer.c
M src/backend/port/win32_sema.c
M src/backend/port/win32_shmem.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/bgwriter.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/fork_process.c
M src/backend/postmaster/interrupt.c
M src/backend/postmaster/pgarch.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/postmaster/startup.c
M src/backend/postmaster/syslogger.c
M src/backend/postmaster/walwriter.c
M src/backend/regex/regc_pg_locale.c
M src/backend/regex/regexport.c
M src/backend/regex/regprefix.c
M src/backend/replication/backup_manifest.c
M src/backend/replication/basebackup.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/launcher.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/logicalfuncs.c
M src/backend/replication/logical/message.c
M src/backend/replication/logical/origin.c
M src/backend/replication/logical/proto.c
M src/backend/replication/logical/relation.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/replication/repl_gram.y
M src/backend/replication/repl_scanner.l
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/syncrep.c
M src/backend/replication/syncrep_gram.y
M src/backend/replication/syncrep_scanner.l
M src/backend/replication/walreceiver.c
M src/backend/replication/walreceiverfuncs.c
M src/backend/replication/walsender.c
M src/backend/rewrite/rewriteDefine.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/rewrite/rewriteManip.c
M src/backend/rewrite/rewriteRemove.c
M src/backend/rewrite/rewriteSupport.c
M src/backend/rewrite/rowsecurity.c
M src/backend/snowball/dict_snowball.c
M src/backend/snowball/snowball.sql.in
M src/backend/snowball/snowball_func.sql.in
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/statistics/mcv.c
M src/backend/statistics/mvdistinct.c
M src/backend/storage/buffer/buf_init.c
M src/backend/storage/buffer/buf_table.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/freelist.c
M src/backend/storage/buffer/localbuf.c
M src/backend/storage/file/buffile.c
M src/backend/storage/file/copydir.c
M src/backend/storage/file/fd.c
M src/backend/storage/file/reinit.c
M src/backend/storage/file/sharedfileset.c
M src/backend/storage/freespace/freespace.c
M src/backend/storage/freespace/fsmpage.c
M src/backend/storage/freespace/indexfsm.c
M src/backend/storage/ipc/barrier.c
M src/backend/storage/ipc/dsm.c
M src/backend/storage/ipc/dsm_impl.c
M src/backend/storage/ipc/ipc.c
M src/backend/storage/ipc/ipci.c
M src/backend/storage/ipc/latch.c
M src/backend/storage/ipc/pmsignal.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/procsignal.c
M src/backend/storage/ipc/shm_mq.c
M src/backend/storage/ipc/shm_toc.c
M src/backend/storage/ipc/shmem.c
M src/backend/storage/ipc/shmqueue.c
M src/backend/storage/ipc/signalfuncs.c
M src/backend/storage/ipc/sinval.c
M src/backend/storage/ipc/sinvaladt.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/large_object/inv_api.c
M src/backend/storage/lmgr/condition_variable.c
M src/backend/storage/lmgr/deadlock.c
M src/backend/storage/lmgr/generate-lwlocknames.pl
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/lwlock.c
M src/backend/storage/lmgr/predicate.c
M src/backend/storage/lmgr/proc.c
M src/backend/storage/lmgr/s_lock.c
M src/backend/storage/lmgr/spin.c
M src/backend/storage/page/bufpage.c
M src/backend/storage/page/checksum.c
M src/backend/storage/page/itemptr.c
M src/backend/storage/smgr/md.c
M src/backend/storage/smgr/smgr.c
M src/backend/storage/sync/sync.c
M src/backend/tcop/cmdtag.c
M src/backend/tcop/dest.c
M src/backend/tcop/fastpath.c
M src/backend/tcop/postgres.c
M src/backend/tcop/pquery.c
M src/backend/tcop/utility.c
M src/backend/tsearch/Makefile
M src/backend/tsearch/dict.c
M src/backend/tsearch/dict_ispell.c
M src/backend/tsearch/dict_simple.c
M src/backend/tsearch/dict_synonym.c
M src/backend/tsearch/dict_thesaurus.c
M src/backend/tsearch/regis.c
M src/backend/tsearch/spell.c
M src/backend/tsearch/to_tsany.c
M src/backend/tsearch/ts_locale.c
M src/backend/tsearch/ts_parse.c
M src/backend/tsearch/ts_selfuncs.c
M src/backend/tsearch/ts_typanalyze.c
M src/backend/tsearch/ts_utils.c
M src/backend/tsearch/wparser.c
M src/backend/tsearch/wparser_def.c
M src/backend/utils/Gen_dummy_probes.pl
M src/backend/utils/Gen_dummy_probes.sed
M src/backend/utils/Gen_fmgrtab.pl
M src/backend/utils/Makefile
M src/backend/utils/adt/acl.c
M src/backend/utils/adt/amutils.c
M src/backend/utils/adt/array_expanded.c
M src/backend/utils/adt/array_selfuncs.c
M src/backend/utils/adt/array_typanalyze.c
M src/backend/utils/adt/array_userfuncs.c
M src/backend/utils/adt/arrayfuncs.c
M src/backend/utils/adt/arraysubs.c
M src/backend/utils/adt/arrayutils.c
M src/backend/utils/adt/ascii.c
M src/backend/utils/adt/bool.c
M src/backend/utils/adt/char.c
M src/backend/utils/adt/cryptohashfuncs.c
M src/backend/utils/adt/date.c
M src/backend/utils/adt/datetime.c
M src/backend/utils/adt/datum.c
M src/backend/utils/adt/dbsize.c
M src/backend/utils/adt/domains.c
M src/backend/utils/adt/encode.c
M src/backend/utils/adt/enum.c
M src/backend/utils/adt/expandeddatum.c
M src/backend/utils/adt/expandedrecord.c
M src/backend/utils/adt/float.c
M src/backend/utils/adt/format_type.c
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/genfile.c
M src/backend/utils/adt/geo_ops.c
M src/backend/utils/adt/geo_selfuncs.c
M src/backend/utils/adt/geo_spgist.c
M src/backend/utils/adt/int.c
M src/backend/utils/adt/int8.c
M src/backend/utils/adt/json.c
M src/backend/utils/adt/jsonb.c
M src/backend/utils/adt/jsonb_gin.c
M src/backend/utils/adt/jsonb_op.c
M src/backend/utils/adt/jsonb_util.c
M src/backend/utils/adt/jsonfuncs.c
M src/backend/utils/adt/jsonpath.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/adt/jsonpath_gram.y
M src/backend/utils/adt/jsonpath_scan.l
M src/backend/utils/adt/levenshtein.c
M src/backend/utils/adt/like.c
M src/backend/utils/adt/like_match.c
M src/backend/utils/adt/like_support.c
M src/backend/utils/adt/lockfuncs.c
M src/backend/utils/adt/mac.c
M src/backend/utils/adt/mac8.c
M src/backend/utils/adt/mcxtfuncs.c
M src/backend/utils/adt/misc.c
M src/backend/utils/adt/multirangetypes.c
M src/backend/utils/adt/multirangetypes_selfuncs.c
M src/backend/utils/adt/name.c
M src/backend/utils/adt/network_gist.c
M src/backend/utils/adt/network_selfuncs.c
M src/backend/utils/adt/network_spgist.c
M src/backend/utils/adt/numeric.c
M src/backend/utils/adt/numutils.c
M src/backend/utils/adt/oid.c
M src/backend/utils/adt/oracle_compat.c
M src/backend/utils/adt/orderedsetaggs.c
M src/backend/utils/adt/partitionfuncs.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_lsn.c
M src/backend/utils/adt/pg_upgrade_support.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/adt/pseudotypes.c
M src/backend/utils/adt/quote.c
M src/backend/utils/adt/rangetypes.c
M src/backend/utils/adt/rangetypes_gist.c
M src/backend/utils/adt/rangetypes_selfuncs.c
M src/backend/utils/adt/rangetypes_spgist.c
M src/backend/utils/adt/rangetypes_typanalyze.c
M src/backend/utils/adt/regexp.c
M src/backend/utils/adt/regproc.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/adt/rowtypes.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/adt/selfuncs.c
M src/backend/utils/adt/tid.c
M src/backend/utils/adt/timestamp.c
M src/backend/utils/adt/trigfuncs.c
M src/backend/utils/adt/tsginidx.c
M src/backend/utils/adt/tsgistidx.c
M src/backend/utils/adt/tsquery.c
M src/backend/utils/adt/tsquery_cleanup.c
M src/backend/utils/adt/tsquery_gist.c
M src/backend/utils/adt/tsquery_op.c
M src/backend/utils/adt/tsquery_rewrite.c
M src/backend/utils/adt/tsquery_util.c
M src/backend/utils/adt/tsrank.c
M src/backend/utils/adt/tsvector.c
M src/backend/utils/adt/tsvector_op.c
M src/backend/utils/adt/tsvector_parser.c
M src/backend/utils/adt/uuid.c
M src/backend/utils/adt/varbit.c
M src/backend/utils/adt/varchar.c
M src/backend/utils/adt/varlena.c
M src/backend/utils/adt/version.c
M src/backend/utils/adt/windowfuncs.c
M src/backend/utils/adt/xid.c
M src/backend/utils/adt/xid8funcs.c
M src/backend/utils/adt/xml.c
M src/backend/utils/cache/attoptcache.c
M src/backend/utils/cache/catcache.c
M src/backend/utils/cache/evtcache.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/lsyscache.c
M src/backend/utils/cache/partcache.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/cache/relfilenodemap.c
M src/backend/utils/cache/relmapper.c
M src/backend/utils/cache/spccache.c
M src/backend/utils/cache/syscache.c
M src/backend/utils/cache/ts_cache.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/errcodes.txt
M src/backend/utils/error/assert.c
M src/backend/utils/error/elog.c
M src/backend/utils/fmgr/dfmgr.c
M src/backend/utils/fmgr/fmgr.c
M src/backend/utils/fmgr/funcapi.c
M src/backend/utils/generate-errcodes.pl
M src/backend/utils/hash/dynahash.c
M src/backend/utils/hash/pg_crc.c
M src/backend/utils/init/globals.c
M src/backend/utils/init/miscinit.c
M src/backend/utils/init/postinit.c
M src/backend/utils/mb/Unicode/Makefile
M src/backend/utils/mb/Unicode/UCS_to_BIG5.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_CN.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_JIS_2004.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_JP.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_KR.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_TW.pl
M src/backend/utils/mb/Unicode/UCS_to_GB18030.pl
M src/backend/utils/mb/Unicode/UCS_to_JOHAB.pl
M src/backend/utils/mb/Unicode/UCS_to_SHIFT_JIS_2004.pl
M src/backend/utils/mb/Unicode/UCS_to_SJIS.pl
M src/backend/utils/mb/Unicode/UCS_to_UHC.pl
M src/backend/utils/mb/Unicode/UCS_to_most.pl
M src/backend/utils/mb/Unicode/convutils.pm
M src/backend/utils/mb/conv.c
M src/backend/utils/mb/conversion_procs/Makefile
M src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
M src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
M src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
M src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
M src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
M src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
M src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
M src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
M src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
M src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
M src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
M src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
M src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
M src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
M src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
M src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
M src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
M src/backend/utils/mb/mbutils.c
M src/backend/utils/mb/stringinfo_mb.c
M src/backend/utils/misc/guc-file.l
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/help_config.c
M src/backend/utils/misc/pg_config.c
M src/backend/utils/misc/pg_controldata.c
M src/backend/utils/misc/pg_rusage.c
M src/backend/utils/misc/ps_status.c
M src/backend/utils/misc/queryenvironment.c
M src/backend/utils/misc/rls.c
M src/backend/utils/misc/sampling.c
M src/backend/utils/misc/superuser.c
M src/backend/utils/misc/timeout.c
M src/backend/utils/misc/tzparser.c
M src/backend/utils/mmgr/aset.c
M src/backend/utils/mmgr/dsa.c
M src/backend/utils/mmgr/freepage.c
M src/backend/utils/mmgr/generation.c
M src/backend/utils/mmgr/mcxt.c
M src/backend/utils/mmgr/memdebug.c
M src/backend/utils/mmgr/portalmem.c
M src/backend/utils/mmgr/slab.c
M src/backend/utils/probes.d
M src/backend/utils/resowner/resowner.c
M src/backend/utils/sort/logtape.c
M src/backend/utils/sort/sharedtuplestore.c
M src/backend/utils/sort/sortsupport.c
M src/backend/utils/sort/tuplesort.c
M src/backend/utils/sort/tuplestore.c
M src/backend/utils/time/combocid.c
M src/backend/utils/time/snapmgr.c
M src/bin/Makefile
M src/bin/initdb/Makefile
M src/bin/initdb/findtimezone.c
M src/bin/initdb/initdb.c
M src/bin/pg_basebackup/Makefile
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_basebackup/pg_receivewal.c
M src/bin/pg_basebackup/pg_recvlogical.c
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/receivelog.h
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_basebackup/streamutil.h
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_basebackup/walmethods.h
M src/bin/pg_checksums/Makefile
M src/bin/pg_checksums/pg_checksums.c
M src/bin/pg_config/Makefile
M src/bin/pg_config/pg_config.c
M src/bin/pg_controldata/Makefile
M src/bin/pg_ctl/Makefile
M src/bin/pg_ctl/pg_ctl.c
M src/bin/pg_dump/Makefile
M src/bin/pg_dump/common.c
M src/bin/pg_dump/compress_io.c
M src/bin/pg_dump/compress_io.h
M src/bin/pg_dump/dumputils.c
M src/bin/pg_dump/dumputils.h
M src/bin/pg_dump/parallel.c
M src/bin/pg_dump/parallel.h
M src/bin/pg_dump/pg_backup_directory.c
M src/bin/pg_dump/pg_backup_utils.c
M src/bin/pg_dump/pg_backup_utils.h
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/pg_dump_sort.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_resetwal/Makefile
M src/bin/pg_resetwal/pg_resetwal.c
M src/bin/pg_rewind/Makefile
M src/bin/pg_rewind/datapagemap.c
M src/bin/pg_rewind/datapagemap.h
M src/bin/pg_rewind/file_ops.c
M src/bin/pg_rewind/file_ops.h
M src/bin/pg_rewind/filemap.c
M src/bin/pg_rewind/filemap.h
M src/bin/pg_rewind/libpq_source.c
M src/bin/pg_rewind/local_source.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/pg_rewind.h
M src/bin/pg_rewind/rewind_source.h
M src/bin/pg_rewind/timeline.c
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/dump.c
M src/bin/pg_upgrade/exec.c
M src/bin/pg_upgrade/file.c
M src/bin/pg_upgrade/function.c
M src/bin/pg_upgrade/info.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/parallel.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/relfilenode.c
M src/bin/pg_upgrade/server.c
M src/bin/pg_upgrade/tablespace.c
M src/bin/pg_upgrade/test.sh
M src/bin/pg_upgrade/util.c
M src/bin/pg_upgrade/version.c
M src/bin/pg_verifybackup/parse_manifest.c
M src/bin/pg_verifybackup/parse_manifest.h
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/bin/pg_waldump/compat.c
M src/bin/pg_waldump/pg_waldump.c
M src/bin/pgbench/exprparse.y
M src/bin/pgbench/exprscan.l
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/pgbench.h
M src/bin/pgevent/Makefile
M src/bin/psql/Makefile
M src/bin/psql/command.c
M src/bin/psql/command.h
M src/bin/psql/common.c
M src/bin/psql/common.h
M src/bin/psql/copy.c
M src/bin/psql/copy.h
M src/bin/psql/create_help.pl
M src/bin/psql/crosstabview.c
M src/bin/psql/crosstabview.h
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/bin/psql/help.c
M src/bin/psql/help.h
M src/bin/psql/input.c
M src/bin/psql/input.h
M src/bin/psql/large_obj.c
M src/bin/psql/large_obj.h
M src/bin/psql/mainloop.c
M src/bin/psql/mainloop.h
M src/bin/psql/prompt.c
M src/bin/psql/prompt.h
M src/bin/psql/psqlscanslash.h
M src/bin/psql/psqlscanslash.l
M src/bin/psql/settings.h
M src/bin/psql/startup.c
M src/bin/psql/stringutils.c
M src/bin/psql/stringutils.h
M src/bin/psql/tab-complete.c
M src/bin/psql/tab-complete.h
M src/bin/psql/variables.c
M src/bin/psql/variables.h
M src/bin/scripts/Makefile
M src/bin/scripts/clusterdb.c
M src/bin/scripts/common.c
M src/bin/scripts/common.h
M src/bin/scripts/createdb.c
M src/bin/scripts/createuser.c
M src/bin/scripts/dropdb.c
M src/bin/scripts/dropuser.c
M src/bin/scripts/pg_isready.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/scripts_parallel.c
M src/bin/scripts/scripts_parallel.h
M src/bin/scripts/vacuumdb.c
M src/common/archive.c
M src/common/base64.c
M src/common/checksum_helper.c
M src/common/config_info.c
M src/common/controldata_utils.c
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
M src/common/d2s.c
M src/common/d2s_full_table.h
M src/common/d2s_intrinsics.h
M src/common/encnames.c
M src/common/exec.c
M src/common/f2s.c
M src/common/fe_memutils.c
M src/common/file_perm.c
M src/common/file_utils.c
M src/common/hashfn.c
M src/common/hex_decode.c
M src/common/ip.c
M src/common/jsonapi.c
M src/common/keywords.c
M src/common/kwlookup.c
M src/common/link-canary.c
M src/common/logging.c
M src/common/md5.c
M src/common/md5_common.c
M src/common/md5_int.h
M src/common/pg_get_line.c
M src/common/pg_lzcompress.c
M src/common/pgfnames.c
M src/common/protocol_openssl.c
M src/common/psprintf.c
M src/common/relpath.c
M src/common/restricted_token.c
M src/common/rmtree.c
M src/common/ryu_common.h
M src/common/saslprep.c
M src/common/scram-common.c
M src/common/sha2.c
M src/common/sha2_int.h
M src/common/sprompt.c
M src/common/string.c
M src/common/stringinfo.c
M src/common/unicode/generate-norm_test_table.pl
M src/common/unicode/generate-unicode_combining_table.pl
M src/common/unicode/generate-unicode_norm_table.pl
M src/common/unicode/generate-unicode_normprops_table.pl
M src/common/unicode/norm_test.c
M src/common/unicode_norm.c
M src/common/username.c
M src/common/wait_error.c
M src/common/wchar.c
M src/fe_utils/Makefile
M src/fe_utils/archive.c
M src/fe_utils/cancel.c
M src/fe_utils/conditional.c
M src/fe_utils/mbprint.c
M src/fe_utils/print.c
M src/fe_utils/psqlscan.l
M src/fe_utils/recovery_gen.c
M src/fe_utils/simple_list.c
M src/fe_utils/string_utils.c
M src/include/access/amapi.h
M src/include/access/amvalidate.h
M src/include/access/attmap.h
M src/include/access/attnum.h
M src/include/access/brin.h
M src/include/access/brin_internal.h
M src/include/access/brin_page.h
M src/include/access/brin_pageops.h
M src/include/access/brin_revmap.h
M src/include/access/brin_tuple.h
M src/include/access/brin_xlog.h
M src/include/access/bufmask.h
M src/include/access/clog.h
M src/include/access/commit_ts.h
M src/include/access/detoast.h
M src/include/access/genam.h
M src/include/access/generic_xlog.h
M src/include/access/gin.h
M src/include/access/gin_private.h
M src/include/access/ginblock.h
M src/include/access/ginxlog.h
M src/include/access/gist.h
M src/include/access/gist_private.h
M src/include/access/gistscan.h
M src/include/access/gistxlog.h
M src/include/access/hash.h
M src/include/access/hash_xlog.h
M src/include/access/heapam.h
M src/include/access/heapam_xlog.h
M src/include/access/heaptoast.h
M src/include/access/hio.h
M src/include/access/htup.h
M src/include/access/htup_details.h
M src/include/access/itup.h
M src/include/access/multixact.h
M src/include/access/nbtree.h
M src/include/access/nbtxlog.h
M src/include/access/parallel.h
M src/include/access/printsimple.h
M src/include/access/printtup.h
M src/include/access/relation.h
M src/include/access/reloptions.h
M src/include/access/relscan.h
M src/include/access/rewriteheap.h
M src/include/access/rmgrlist.h
M src/include/access/sdir.h
M src/include/access/session.h
M src/include/access/skey.h
M src/include/access/slru.h
M src/include/access/spgist.h
M src/include/access/spgist_private.h
M src/include/access/spgxlog.h
M src/include/access/stratnum.h
M src/include/access/subtrans.h
M src/include/access/syncscan.h
M src/include/access/sysattr.h
M src/include/access/table.h
M src/include/access/tableam.h
M src/include/access/timeline.h
M src/include/access/toast_helper.h
M src/include/access/toast_internals.h
M src/include/access/transam.h
M src/include/access/tsmapi.h
M src/include/access/tupconvert.h
M src/include/access/tupdesc.h
M src/include/access/tupdesc_details.h
M src/include/access/tupmacs.h
M src/include/access/twophase.h
M src/include/access/twophase_rmgr.h
M src/include/access/valid.h
M src/include/access/visibilitymap.h
M src/include/access/xact.h
M src/include/access/xlog.h
M src/include/access/xlog_internal.h
M src/include/access/xlogarchive.h
M src/include/access/xlogdefs.h
M src/include/access/xloginsert.h
M src/include/access/xlogreader.h
M src/include/access/xlogrecord.h
M src/include/access/xlogutils.h
M src/include/bootstrap/bootstrap.h
M src/include/c.h
M src/include/catalog/Makefile
M src/include/catalog/binary_upgrade.h
M src/include/catalog/catalog.h
M src/include/catalog/catversion.h
M src/include/catalog/dependency.h
M src/include/catalog/duplicate_oids
M src/include/catalog/genbki.h
M src/include/catalog/heap.h
M src/include/catalog/index.h
M src/include/catalog/indexing.h
M src/include/catalog/namespace.h
M src/include/catalog/objectaccess.h
M src/include/catalog/objectaddress.h
M src/include/catalog/partition.h
M src/include/catalog/pg_aggregate.dat
M src/include/catalog/pg_aggregate.h
M src/include/catalog/pg_am.dat
M src/include/catalog/pg_am.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amop.h
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_amproc.h
M src/include/catalog/pg_attrdef.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_auth_members.h
M src/include/catalog/pg_authid.dat
M src/include/catalog/pg_authid.h
M src/include/catalog/pg_cast.dat
M src/include/catalog/pg_cast.h
M src/include/catalog/pg_class.dat
M src/include/catalog/pg_class.h
M src/include/catalog/pg_collation.dat
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_control.h
M src/include/catalog/pg_conversion.dat
M src/include/catalog/pg_conversion.h
M src/include/catalog/pg_database.dat
M src/include/catalog/pg_database.h
M src/include/catalog/pg_db_role_setting.h
M src/include/catalog/pg_default_acl.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_description.h
M src/include/catalog/pg_enum.h
M src/include/catalog/pg_event_trigger.h
M src/include/catalog/pg_extension.h
M src/include/catalog/pg_foreign_data_wrapper.h
M src/include/catalog/pg_foreign_server.h
M src/include/catalog/pg_foreign_table.h
M src/include/catalog/pg_index.h
M src/include/catalog/pg_inherits.h
M src/include/catalog/pg_init_privs.h
M src/include/catalog/pg_language.dat
M src/include/catalog/pg_language.h
M src/include/catalog/pg_largeobject.h
M src/include/catalog/pg_largeobject_metadata.h
M src/include/catalog/pg_namespace.dat
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_opclass.h
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_opfamily.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_policy.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_publication.h
M src/include/catalog/pg_publication_rel.h
M src/include/catalog/pg_range.dat
M src/include/catalog/pg_range.h
M src/include/catalog/pg_replication_origin.h
M src/include/catalog/pg_rewrite.h
M src/include/catalog/pg_seclabel.h
M src/include/catalog/pg_sequence.h
M src/include/catalog/pg_shdepend.h
M src/include/catalog/pg_shdescription.h
M src/include/catalog/pg_shseclabel.h
M src/include/catalog/pg_statistic.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/catalog/pg_subscription.h
M src/include/catalog/pg_subscription_rel.h
M src/include/catalog/pg_tablespace.dat
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_transform.h
M src/include/catalog/pg_trigger.h
M src/include/catalog/pg_ts_config.dat
M src/include/catalog/pg_ts_config.h
M src/include/catalog/pg_ts_config_map.dat
M src/include/catalog/pg_ts_config_map.h
M src/include/catalog/pg_ts_dict.dat
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_ts_parser.dat
M src/include/catalog/pg_ts_parser.h
M src/include/catalog/pg_ts_template.dat
M src/include/catalog/pg_ts_template.h
M src/include/catalog/pg_type.dat
M src/include/catalog/pg_type.h
M src/include/catalog/pg_user_mapping.h
M src/include/catalog/reformat_dat_file.pl
M src/include/catalog/renumber_oids.pl
M src/include/catalog/storage.h
M src/include/catalog/storage_xlog.h
M src/include/catalog/toasting.h
M src/include/catalog/unused_oids
M src/include/commands/alter.h
M src/include/commands/async.h
M src/include/commands/cluster.h
M src/include/commands/collationcmds.h
M src/include/commands/comment.h
M src/include/commands/conversioncmds.h
M src/include/commands/copy.h
M src/include/commands/copyfrom_internal.h
M src/include/commands/createas.h
M src/include/commands/dbcommands.h
M src/include/commands/dbcommands_xlog.h
M src/include/commands/defrem.h
M src/include/commands/discard.h
M src/include/commands/event_trigger.h
M src/include/commands/explain.h
M src/include/commands/extension.h
M src/include/commands/lockcmds.h
M src/include/commands/matview.h
M src/include/commands/policy.h
M src/include/commands/portalcmds.h
M src/include/commands/prepare.h
M src/include/commands/proclang.h
M src/include/commands/progress.h
M src/include/commands/publicationcmds.h
M src/include/commands/schemacmds.h
M src/include/commands/seclabel.h
M src/include/commands/sequence.h
M src/include/commands/subscriptioncmds.h
M src/include/commands/tablecmds.h
M src/include/commands/tablespace.h
M src/include/commands/trigger.h
M src/include/commands/typecmds.h
M src/include/commands/vacuum.h
M src/include/commands/variable.h
M src/include/commands/view.h
M src/include/common/archive.h
M src/include/common/base64.h
M src/include/common/checksum_helper.h
M src/include/common/config_info.h
M src/include/common/connect.h
M src/include/common/controldata_utils.h
M src/include/common/cryptohash.h
M src/include/common/fe_memutils.h
M src/include/common/file_perm.h
M src/include/common/file_utils.h
M src/include/common/hashfn.h
M src/include/common/hex_decode.h
M src/include/common/int.h
M src/include/common/int128.h
M src/include/common/ip.h
M src/include/common/jsonapi.h
M src/include/common/keywords.h
M src/include/common/kwlookup.h
M src/include/common/link-canary.h
M src/include/common/logging.h
M src/include/common/md5.h
M src/include/common/openssl.h
M src/include/common/relpath.h
M src/include/common/restricted_token.h
M src/include/common/saslprep.h
M src/include/common/scram-common.h
M src/include/common/sha2.h
M src/include/common/shortest_dec.h
M src/include/common/string.h
M src/include/common/unicode_norm.h
M src/include/common/unicode_norm_hashfunc.h
M src/include/common/unicode_norm_table.h
M src/include/common/username.h
M src/include/datatype/timestamp.h
M src/include/executor/execExpr.h
M src/include/executor/execParallel.h
M src/include/executor/execPartition.h
M src/include/executor/execdebug.h
M src/include/executor/execdesc.h
M src/include/executor/executor.h
M src/include/executor/functions.h
M src/include/executor/hashjoin.h
M src/include/executor/instrument.h
M src/include/executor/nodeAgg.h
M src/include/executor/nodeAppend.h
M src/include/executor/nodeBitmapAnd.h
M src/include/executor/nodeBitmapHeapscan.h
M src/include/executor/nodeBitmapIndexscan.h
M src/include/executor/nodeBitmapOr.h
M src/include/executor/nodeCtescan.h
M src/include/executor/nodeCustom.h
M src/include/executor/nodeForeignscan.h
M src/include/executor/nodeFunctionscan.h
M src/include/executor/nodeGather.h
M src/include/executor/nodeGatherMerge.h
M src/include/executor/nodeGroup.h
M src/include/executor/nodeHash.h
M src/include/executor/nodeHashjoin.h
M src/include/executor/nodeIncrementalSort.h
M src/include/executor/nodeIndexonlyscan.h
M src/include/executor/nodeIndexscan.h
M src/include/executor/nodeLimit.h
M src/include/executor/nodeLockRows.h
M src/include/executor/nodeMaterial.h
M src/include/executor/nodeMergeAppend.h
M src/include/executor/nodeMergejoin.h
M src/include/executor/nodeModifyTable.h
M src/include/executor/nodeNamedtuplestorescan.h
M src/include/executor/nodeNestloop.h
M src/include/executor/nodeProjectSet.h
M src/include/executor/nodeRecursiveunion.h
M src/include/executor/nodeResult.h
M src/include/executor/nodeSamplescan.h
M src/include/executor/nodeSeqscan.h
M src/include/executor/nodeSetOp.h
M src/include/executor/nodeSort.h
M src/include/executor/nodeSubplan.h
M src/include/executor/nodeSubqueryscan.h
M src/include/executor/nodeTableFuncscan.h
M src/include/executor/nodeTidscan.h
M src/include/executor/nodeUnique.h
M src/include/executor/nodeValuesscan.h
M src/include/executor/nodeWindowAgg.h
M src/include/executor/nodeWorktablescan.h
M src/include/executor/spi.h
M src/include/executor/spi_priv.h
M src/include/executor/tablefunc.h
M src/include/executor/tqueue.h
M src/include/executor/tstoreReceiver.h
M src/include/executor/tuptable.h
M src/include/fe_utils/archive.h
M src/include/fe_utils/cancel.h
M src/include/fe_utils/conditional.h
M src/include/fe_utils/mbprint.h
M src/include/fe_utils/print.h
M src/include/fe_utils/psqlscan.h
M src/include/fe_utils/psqlscan_int.h
M src/include/fe_utils/recovery_gen.h
M src/include/fe_utils/simple_list.h
M src/include/fe_utils/string_utils.h
M src/include/fmgr.h
M src/include/foreign/fdwapi.h
M src/include/foreign/foreign.h
M src/include/funcapi.h
M src/include/getaddrinfo.h
M src/include/getopt_long.h
M src/include/jit/jit.h
M src/include/jit/llvmjit.h
M src/include/jit/llvmjit_emit.h
M src/include/lib/binaryheap.h
M src/include/lib/bipartite_match.h
M src/include/lib/bloomfilter.h
M src/include/lib/dshash.h
M src/include/lib/hyperloglog.h
M src/include/lib/ilist.h
M src/include/lib/integerset.h
M src/include/lib/knapsack.h
M src/include/lib/pairingheap.h
M src/include/lib/qunique.h
M src/include/lib/rbtree.h
M src/include/lib/stringinfo.h
M src/include/libpq/auth.h
M src/include/libpq/be-fsstubs.h
M src/include/libpq/be-gssapi-common.h
M src/include/libpq/crypt.h
M src/include/libpq/ifaddr.h
M src/include/libpq/libpq-be.h
M src/include/libpq/libpq-fs.h
M src/include/libpq/libpq.h
M src/include/libpq/pqcomm.h
M src/include/libpq/pqformat.h
M src/include/libpq/pqmq.h
M src/include/libpq/pqsignal.h
M src/include/libpq/scram.h
M src/include/mb/pg_wchar.h
M src/include/mb/stringinfo_mb.h
M src/include/miscadmin.h
M src/include/nodes/bitmapset.h
M src/include/nodes/execnodes.h
M src/include/nodes/extensible.h
M src/include/nodes/lockoptions.h
M src/include/nodes/makefuncs.h
M src/include/nodes/memnodes.h
M src/include/nodes/nodeFuncs.h
M src/include/nodes/nodes.h
M src/include/nodes/params.h
M src/include/nodes/parsenodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/pg_list.h
M src/include/nodes/plannodes.h
M src/include/nodes/primnodes.h
M src/include/nodes/print.h
M src/include/nodes/readfuncs.h
M src/include/nodes/replnodes.h
M src/include/nodes/subscripting.h
M src/include/nodes/supportnodes.h
M src/include/nodes/tidbitmap.h
M src/include/nodes/value.h
M src/include/optimizer/appendinfo.h
M src/include/optimizer/clauses.h
M src/include/optimizer/cost.h
M src/include/optimizer/geqo.h
M src/include/optimizer/geqo_copy.h
M src/include/optimizer/geqo_gene.h
M src/include/optimizer/geqo_misc.h
M src/include/optimizer/geqo_mutation.h
M src/include/optimizer/geqo_pool.h
M src/include/optimizer/geqo_random.h
M src/include/optimizer/geqo_recombination.h
M src/include/optimizer/geqo_selection.h
M src/include/optimizer/inherit.h
M src/include/optimizer/joininfo.h
M src/include/optimizer/optimizer.h
M src/include/optimizer/orclauses.h
M src/include/optimizer/paramassign.h
M src/include/optimizer/pathnode.h
M src/include/optimizer/paths.h
M src/include/optimizer/placeholder.h
M src/include/optimizer/plancat.h
M src/include/optimizer/planmain.h
M src/include/optimizer/planner.h
M src/include/optimizer/prep.h
M src/include/optimizer/restrictinfo.h
M src/include/optimizer/subselect.h
M src/include/optimizer/tlist.h
M src/include/parser/analyze.h
M src/include/parser/gramparse.h
M src/include/parser/kwlist.h
M src/include/parser/parse_agg.h
M src/include/parser/parse_clause.h
M src/include/parser/parse_coerce.h
M src/include/parser/parse_collate.h
M src/include/parser/parse_cte.h
M src/include/parser/parse_enr.h
M src/include/parser/parse_expr.h
M src/include/parser/parse_func.h
M src/include/parser/parse_node.h
M src/include/parser/parse_oper.h
M src/include/parser/parse_param.h
M src/include/parser/parse_relation.h
M src/include/parser/parse_target.h
M src/include/parser/parse_type.h
M src/include/parser/parse_utilcmd.h
M src/include/parser/parser.h
M src/include/parser/parsetree.h
M src/include/parser/scanner.h
M src/include/parser/scansup.h
M src/include/partitioning/partbounds.h
M src/include/partitioning/partdefs.h
M src/include/partitioning/partdesc.h
M src/include/partitioning/partprune.h
M src/include/pg_config_manual.h
M src/include/pg_getopt.h
M src/include/pg_trace.h
M src/include/pgstat.h
M src/include/pgtar.h
M src/include/pgtime.h
M src/include/port.h
M src/include/port/atomics.h
M src/include/port/atomics/arch-arm.h
M src/include/port/atomics/arch-hppa.h
M src/include/port/atomics/arch-ia64.h
M src/include/port/atomics/arch-ppc.h
M src/include/port/atomics/arch-x86.h
M src/include/port/atomics/fallback.h
M src/include/port/atomics/generic-acc.h
M src/include/port/atomics/generic-gcc.h
M src/include/port/atomics/generic-msvc.h
M src/include/port/atomics/generic-sunpro.h
M src/include/port/atomics/generic.h
M src/include/port/pg_bitutils.h
M src/include/port/pg_bswap.h
M src/include/port/pg_crc32c.h
M src/include/port/win32_port.h
M src/include/portability/instr_time.h
M src/include/portability/mem.h
M src/include/postgres.h
M src/include/postgres_fe.h
M src/include/postmaster/autovacuum.h
M src/include/postmaster/bgworker.h
M src/include/postmaster/bgworker_internals.h
M src/include/postmaster/bgwriter.h
M src/include/postmaster/fork_process.h
M src/include/postmaster/interrupt.h
M src/include/postmaster/pgarch.h
M src/include/postmaster/postmaster.h
M src/include/postmaster/startup.h
M src/include/postmaster/syslogger.h
M src/include/postmaster/walwriter.h
M src/include/regex/regexport.h
M src/include/replication/backup_manifest.h
M src/include/replication/basebackup.h
M src/include/replication/decode.h
M src/include/replication/logical.h
M src/include/replication/logicallauncher.h
M src/include/replication/logicalproto.h
M src/include/replication/logicalrelation.h
M src/include/replication/logicalworker.h
M src/include/replication/message.h
M src/include/replication/origin.h
M src/include/replication/output_plugin.h
M src/include/replication/pgoutput.h
M src/include/replication/reorderbuffer.h
M src/include/replication/slot.h
M src/include/replication/snapbuild.h
M src/include/replication/syncrep.h
M src/include/replication/walreceiver.h
M src/include/replication/walsender.h
M src/include/replication/walsender_private.h
M src/include/replication/worker_internal.h
M src/include/rewrite/prs2lock.h
M src/include/rewrite/rewriteDefine.h
M src/include/rewrite/rewriteHandler.h
M src/include/rewrite/rewriteManip.h
M src/include/rewrite/rewriteRemove.h
M src/include/rewrite/rewriteSupport.h
M src/include/rewrite/rowsecurity.h
M src/include/rusagestub.h
M src/include/snowball/header.h
M src/include/statistics/extended_stats_internal.h
M src/include/statistics/statistics.h
M src/include/storage/backendid.h
M src/include/storage/barrier.h
M src/include/storage/block.h
M src/include/storage/buf.h
M src/include/storage/buf_internals.h
M src/include/storage/buffile.h
M src/include/storage/bufmgr.h
M src/include/storage/bufpage.h
M src/include/storage/checksum.h
M src/include/storage/checksum_impl.h
M src/include/storage/condition_variable.h
M src/include/storage/copydir.h
M src/include/storage/dsm.h
M src/include/storage/dsm_impl.h
M src/include/storage/fd.h
M src/include/storage/freespace.h
M src/include/storage/fsm_internals.h
M src/include/storage/indexfsm.h
M src/include/storage/ipc.h
M src/include/storage/item.h
M src/include/storage/itemid.h
M src/include/storage/itemptr.h
M src/include/storage/large_object.h
M src/include/storage/latch.h
M src/include/storage/lmgr.h
M src/include/storage/lock.h
M src/include/storage/lockdefs.h
M src/include/storage/lwlock.h
M src/include/storage/md.h
M src/include/storage/off.h
M src/include/storage/pg_sema.h
M src/include/storage/pg_shmem.h
M src/include/storage/pmsignal.h
M src/include/storage/predicate.h
M src/include/storage/predicate_internals.h
M src/include/storage/proc.h
M src/include/storage/procarray.h
M src/include/storage/proclist.h
M src/include/storage/proclist_types.h
M src/include/storage/procsignal.h
M src/include/storage/reinit.h
M src/include/storage/relfilenode.h
M src/include/storage/s_lock.h
M src/include/storage/sharedfileset.h
M src/include/storage/shm_mq.h
M src/include/storage/shm_toc.h
M src/include/storage/shmem.h
M src/include/storage/sinval.h
M src/include/storage/sinvaladt.h
M src/include/storage/smgr.h
M src/include/storage/spin.h
M src/include/storage/standby.h
M src/include/storage/standbydefs.h
M src/include/storage/sync.h
M src/include/tcop/cmdtag.h
M src/include/tcop/cmdtaglist.h
M src/include/tcop/deparse_utility.h
M src/include/tcop/dest.h
M src/include/tcop/fastpath.h
M src/include/tcop/pquery.h
M src/include/tcop/tcopprot.h
M src/include/tcop/utility.h
M src/include/tsearch/dicts/regis.h
M src/include/tsearch/dicts/spell.h
M src/include/tsearch/ts_cache.h
M src/include/tsearch/ts_locale.h
M src/include/tsearch/ts_public.h
M src/include/tsearch/ts_type.h
M src/include/tsearch/ts_utils.h
M src/include/utils/acl.h
M src/include/utils/aclchk_internal.h
M src/include/utils/array.h
M src/include/utils/arrayaccess.h
M src/include/utils/ascii.h
M src/include/utils/attoptcache.h
M src/include/utils/builtins.h
M src/include/utils/bytea.h
M src/include/utils/catcache.h
M src/include/utils/combocid.h
M src/include/utils/date.h
M src/include/utils/datetime.h
M src/include/utils/datum.h
M src/include/utils/dsa.h
M src/include/utils/dynahash.h
M src/include/utils/elog.h
M src/include/utils/evtcache.h
M src/include/utils/expandeddatum.h
M src/include/utils/expandedrecord.h
M src/include/utils/float.h
M src/include/utils/fmgrtab.h
M src/include/utils/formatting.h
M src/include/utils/freepage.h
M src/include/utils/geo_decls.h
M src/include/utils/guc.h
M src/include/utils/guc_tables.h
M src/include/utils/help_config.h
M src/include/utils/hsearch.h
M src/include/utils/index_selfuncs.h
M src/include/utils/inet.h
M src/include/utils/int8.h
M src/include/utils/inval.h
M src/include/utils/json.h
M src/include/utils/jsonb.h
M src/include/utils/jsonfuncs.h
M src/include/utils/jsonpath.h
M src/include/utils/logtape.h
M src/include/utils/lsyscache.h
M src/include/utils/memdebug.h
M src/include/utils/memutils.h
M src/include/utils/multirangetypes.h
M src/include/utils/numeric.h
M src/include/utils/old_snapshot.h
M src/include/utils/palloc.h
M src/include/utils/partcache.h
M src/include/utils/pg_crc.h
M src/include/utils/pg_locale.h
M src/include/utils/pg_lsn.h
M src/include/utils/pg_rusage.h
M src/include/utils/pidfile.h
M src/include/utils/plancache.h
M src/include/utils/portal.h
M src/include/utils/queryenvironment.h
M src/include/utils/rangetypes.h
M src/include/utils/regproc.h
M src/include/utils/rel.h
M src/include/utils/relcache.h
M src/include/utils/relfilenodemap.h
M src/include/utils/relmapper.h
M src/include/utils/relptr.h
M src/include/utils/reltrigger.h
M src/include/utils/resowner.h
M src/include/utils/resowner_private.h
M src/include/utils/rls.h
M src/include/utils/ruleutils.h
M src/include/utils/sampling.h
M src/include/utils/selfuncs.h
M src/include/utils/sharedtuplestore.h
M src/include/utils/snapmgr.h
M src/include/utils/snapshot.h
M src/include/utils/sortsupport.h
M src/include/utils/spccache.h
M src/include/utils/syscache.h
M src/include/utils/timeout.h
M src/include/utils/timestamp.h
M src/include/utils/tuplesort.h
M src/include/utils/tuplestore.h
M src/include/utils/typcache.h
M src/include/utils/tzparser.h
M src/include/utils/uuid.h
M src/include/utils/varbit.h
M src/include/utils/varlena.h
M src/include/utils/xid8.h
M src/include/utils/xml.h
M src/include/windowapi.h
M src/interfaces/ecpg/compatlib/Makefile
M src/interfaces/ecpg/ecpglib/Makefile
M src/interfaces/ecpg/pgtypeslib/Makefile
M src/interfaces/ecpg/preproc/Makefile
M src/interfaces/ecpg/preproc/c_kwlist.h
M src/interfaces/ecpg/preproc/check_rules.pl
M src/interfaces/ecpg/preproc/ecpg.c
M src/interfaces/ecpg/preproc/ecpg_kwlist.h
M src/interfaces/ecpg/preproc/keywords.c
M src/interfaces/ecpg/preproc/parse.pl
M src/interfaces/ecpg/preproc/parser.c
M src/interfaces/ecpg/preproc/pgc.l
M src/interfaces/ecpg/test/pg_regress_ecpg.c
M src/interfaces/libpq/Makefile
M src/interfaces/libpq/fe-auth-scram.c
M src/interfaces/libpq/fe-auth.c
M src/interfaces/libpq/fe-auth.h
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-gssapi-common.c
M src/interfaces/libpq/fe-gssapi-common.h
M src/interfaces/libpq/fe-lobj.c
M src/interfaces/libpq/fe-misc.c
M src/interfaces/libpq/fe-print.c
M src/interfaces/libpq/fe-protocol2.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/fe-secure-common.c
M src/interfaces/libpq/fe-secure-common.h
M src/interfaces/libpq/fe-secure-gssapi.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/fe-secure.c
M src/interfaces/libpq/legacy-pqsignal.c
M src/interfaces/libpq/libpq-events.c
M src/interfaces/libpq/libpq-events.h
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
M src/interfaces/libpq/pqexpbuffer.c
M src/interfaces/libpq/pqexpbuffer.h
M src/interfaces/libpq/pthread-win32.c
M src/interfaces/libpq/test/uri-regress.c
M src/interfaces/libpq/win32.c
M src/pl/plperl/plperl.h
M src/pl/plpgsql/src/generate-plerrcodes.pl
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_funcs.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/pl_handler.c
M src/pl/plpgsql/src/pl_reserved_kwlist.h
M src/pl/plpgsql/src/pl_scanner.c
M src/pl/plpgsql/src/pl_unreserved_kwlist.h
M src/pl/plpgsql/src/plpgsql.h
M src/pl/plpython/generate-spiexceptions.pl
M src/pl/plpython/plpython.h
M src/pl/tcl/generate-pltclerrcodes.pl
M src/port/chklocale.c
M src/port/dirent.c
M src/port/dirmod.c
M src/port/dlopen.c
M src/port/erand48.c
M src/port/explicit_bzero.c
M src/port/fls.c
M src/port/getaddrinfo.c
M src/port/getpeereid.c
M src/port/getrusage.c
M src/port/kill.c
M src/port/link.c
M src/port/mkdtemp.c
M src/port/noblock.c
M src/port/open.c
M src/port/path.c
M src/port/pg_bitutils.c
M src/port/pg_crc32c_armv8.c
M src/port/pg_crc32c_armv8_choose.c
M src/port/pg_crc32c_sb8.c
M src/port/pg_crc32c_sse42.c
M src/port/pg_crc32c_sse42_choose.c
M src/port/pg_strong_random.c
M src/port/pgcheckdir.c
M src/port/pgsleep.c
M src/port/pgstrcasecmp.c
M src/port/pgstrsignal.c
M src/port/pqsignal.c
M src/port/pread.c
M src/port/pwrite.c
M src/port/quotes.c
M src/port/random.c
M src/port/setenv.c
M src/port/snprintf.c
M src/port/srandom.c
M src/port/strerror.c
M src/port/strlcpy.c
M src/port/strnlen.c
M src/port/strtof.c
M src/port/system.c
M src/port/thread.c
M src/port/unsetenv.c
M src/port/win32env.c
M src/port/win32error.c
M src/port/win32security.c
M src/port/win32setlocale.c
M src/port/win32stat.c
M src/port/win32ver.rc
M src/test/authentication/Makefile
M src/test/examples/testlo.c
M src/test/examples/testlo64.c
M src/test/isolation/isolation_main.c
M src/test/isolation/isolationtester.h
M src/test/isolation/specparse.y
M src/test/isolation/specscanner.l
M src/test/kerberos/Makefile
M src/test/ldap/Makefile
M src/test/modules/delay_execution/delay_execution.c
M src/test/modules/dummy_index_am/dummy_index_am.c
M src/test/modules/dummy_seclabel/dummy_seclabel.c
M src/test/modules/plsample/plsample.c
M src/test/modules/test_bloomfilter/test_bloomfilter.c
M src/test/modules/test_ddl_deparse/test_ddl_deparse.c
M src/test/modules/test_ginpostinglist/test_ginpostinglist.c
M src/test/modules/test_integerset/test_integerset.c
M src/test/modules/test_parser/test_parser.c
M src/test/modules/test_predtest/test_predtest.c
M src/test/modules/test_rbtree/test_rbtree.c
M src/test/modules/test_rls_hooks/test_rls_hooks.c
M src/test/modules/test_rls_hooks/test_rls_hooks.h
M src/test/modules/test_shm_mq/setup.c
M src/test/modules/test_shm_mq/test.c
M src/test/modules/test_shm_mq/test_shm_mq.h
M src/test/modules/test_shm_mq/worker.c
M src/test/modules/worker_spi/worker_spi.c
M src/test/perl/Makefile
M src/test/recovery/Makefile
M src/test/regress/GNUmakefile
M src/test/regress/pg_regress.c
M src/test/regress/pg_regress.h
M src/test/regress/pg_regress_main.c
M src/test/regress/regress.c
M src/test/ssl/Makefile
M src/test/subscription/Makefile
M src/timezone/pgtz.c
M src/timezone/pgtz.h
M src/tools/PerfectHash.pm
M src/tools/check_bison_recursion.pl
M src/tools/copyright.pl
M src/tools/findoidjoins/Makefile
M src/tools/findoidjoins/findoidjoins.c
M src/tools/fix-old-flex-code.pl
M src/tools/gen_keywordlist.pl
M src/tools/ifaddrs/Makefile
M src/tools/pginclude/cpluspluscheck
M src/tools/pginclude/headerscheck
M src/tools/testint128.c
M src/tools/version_stamp.pl
M src/tools/win32tzlist.pl
M src/tutorial/complex.source
M src/tutorial/syscat.source
Doc: improve explanation of EXTRACT(EPOCH) for timestamp without tz.
commit : 4d3f03f42227bb351c2021a9ccea2fff9c023cfc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 1 Jan 2021 15:51:09 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 1 Jan 2021 15:51:09 -0500
Try to be clearer about what computation is actually happening here.
Per bug #16797 from Dana Burd.
Discussion: https://postgr.es/m/16797-f264b0b980b53b8b@postgresql.org
M doc/src/sgml/func.sgml
Get heap page max offset with buffer lock held.
commit : 32d6287d2eef6b6a4dde07e0513f3e4f321792ad
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Dec 2020 17:21:42 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Dec 2020 17:21:42 -0800
On further reflection it seems better to call PageGetMaxOffsetNumber()
after acquiring a buffer lock on the page. This shouldn't really
matter, but doing it this way is cleaner.
Follow-up to commit 42288174.
Backpatch: 12-, just like commit 42288174
M src/backend/access/heap/heapam.c
Fix index deletion latestRemovedXid bug.
commit : 422881744997417944634a7f84af7a66a608de9a
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Dec 2020 16:29:05 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Dec 2020 16:29:05 -0800
The logic for determining the latest removed XID for the purposes of
generating recovery conflicts in REDO routines was subtly broken. It
failed to follow links from HOT chains, and so failed to consider all
relevant heap tuple headers in some cases.
To fix, expand the loop that deals with LP_REDIRECT line pointers to
also deal with HOT chains. The new version of the loop is loosely based
on a similar loop from heap_prune_chain().
The impact of this bug is probably quite limited, since the horizon code
necessarily deals with heap tuples that are pointed to by LP_DEAD-set
index tuples. The process of setting LP_DEAD index tuples (e.g. within
the kill_prior_tuple mechanism) is highly correlated with opportunistic
pruning of pointed-to heap tuples. Plus the question of generating a
recovery conflict usually comes up some time after index tuple LP_DEAD
bits were initially set, unlike heap pruning, where a latestRemovedXid
is generated at the point of the pruning operation (heap pruning has no
deferred "would-be page split" style processing that produces conflicts
lazily).
Only backpatch to Postgres 12, the first version where this logic runs
during original execution (following commit 558a9165e08). The index
latestRemovedXid mechanism has had the same bug since it first appeared
over 10 years ago (in commit a760893d), but backpatching to all
supported versions now seems like a bad idea on balance. Running the
new improved code during recovery seems risky, especially given the lack
of complaints from the field.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-Wz=Eib393+HHcERK_9MtgNS7Ew1HY=RDC_g6GL46zM5C6Q@mail.gmail.com
Backpatch: 12-
M src/backend/access/heap/heapam.c
M src/backend/storage/ipc/standby.c
Doc: spell out comparison behaviors for the date/time types.
commit : 319f4d54e82d15d4a0c3f4cc1328c40dba024b5c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Dec 2020 17:48:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Dec 2020 17:48:43 -0500
The behavior of cross-type comparisons among date/time data types was
not really explained anywhere. You could probably infer it if you
recognized the applicability of comments elsewhere about datatype
conversions, but it seems worthy of explicit documentation.
Per bug #16797 from Dana Burd.
Discussion: https://postgr.es/m/16797-f264b0b980b53b8b@postgresql.org
M doc/src/sgml/func.sgml
More fixups for pg_upgrade cross-version tests.
commit : 091866724cb3ee7251fa56e2517248c4b7796ca8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Dec 2020 14:15:41 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Dec 2020 14:15:41 -0500
Commit 7ca37fb04 removed regress_putenv from the regress.so library,
so reloading a SQL function dependent on that would not work.
Fix similarly to 52202bb39.
Per buildfarm.
M src/bin/pg_upgrade/test.sh
Refactor multirange_in()
commit : 16d531a30a120d13cc3b460fba6570024a1fcfa8
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 30 Dec 2020 21:11:31 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 30 Dec 2020 21:11:31 +0300
This commit preserves the logic of multirange_in() but makes it more clear
what's going on. Also, this commit fixes the compiler warning spotted by the
buildfarm.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/2246043.1609290699%40sss.pgh.pa.us
M src/backend/utils/adt/multirangetypes.c
Use setenv() in preference to putenv().
commit : 7ca37fb0406bc2cbbd864a2ffdbdb4479e338c0c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Dec 2020 12:55:59 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Dec 2020 12:55:59 -0500
Since at least 2001 we've used putenv() and avoided setenv(), on the
grounds that the latter was unportable and not in POSIX. However,
POSIX added it that same year, and by now the situation has reversed:
setenv() is probably more portable than putenv(), since POSIX now
treats the latter as not being a core function. And setenv() has
cleaner semantics too. So, let's reverse that old policy.
This commit adds a simple src/port/ implementation of setenv() for
any stragglers (we have one in the buildfarm, but I'd not be surprised
if that code is never used in the field). More importantly, extend
win32env.c to also support setenv(). Then, replace usages of putenv()
with setenv(), and get rid of some ad-hoc implementations of setenv()
wannabees.
Also, adjust our src/port/ implementation of unsetenv() to follow the
POSIX spec that it returns an error indicator, rather than returning
void as per the ancient BSD convention. I don't feel a need to make
all the call sites check for errors, but the portability stub ought
to match real-world practice.
Discussion: https://postgr.es/m/2065122.1609212051@sss.pgh.pa.us
M configure
M configure.ac
M contrib/dblink/input/paths.source
M contrib/dblink/output/paths.source
M src/backend/utils/adt/pg_locale.c
M src/bin/initdb/initdb.c
M src/bin/pg_ctl/pg_ctl.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/util.c
M src/bin/psql/command.c
M src/common/exec.c
M src/common/restricted_token.c
M src/include/pg_config.h.in
M src/include/port.h
M src/include/port/win32_port.h
M src/interfaces/ecpg/test/pg_regress_ecpg.c
A src/port/setenv.c
M src/port/unsetenv.c
M src/port/win32env.c
M src/test/isolation/isolation_main.c
M src/test/regress/pg_regress.c
M src/test/regress/pg_regress_main.c
M src/test/regress/regress.c
M src/tools/msvc/Solution.pm
Fix selectivity estimation @> (anymultirange, anyrange) operator
commit : 62097a4cc8c725fa86d3170396a8f30609acd0d3
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 30 Dec 2020 20:19:15 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 30 Dec 2020 20:19:15 +0300
Attempt to get selectivity estimation for @> (anymultirange, anyrange) operator
caused an error in buildfarm, because this operator was missed in switch()
of calc_hist_selectivity(). Fix that and also make regression tests reliably
check that selectivity estimation for (multi)ranges doesn't fall. Previously,
whether we test selectivity estimation for (multi)ranges depended on
whether autovacuum managed to gather concurrently to the test.
Reported-by: Michael Paquier
Discussion: https://postgr.es/m/X%2BwmgjRItuvHNBeV%40paquier.xyz
M src/backend/utils/adt/multirangetypes_selfuncs.c
M src/test/regress/expected/multirangetypes.out
M src/test/regress/expected/rangetypes.out
M src/test/regress/sql/multirangetypes.sql
M src/test/regress/sql/rangetypes.sql
Fix up usage of krb_server_keyfile GUC parameter.
commit : 860fe27ee1e2a4a1c36c2f874c37656533cccce9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Dec 2020 11:38:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Dec 2020 11:38:42 -0500
secure_open_gssapi() installed the krb_server_keyfile setting as
KRB5_KTNAME unconditionally, so long as it's not empty. However,
pg_GSS_recvauth() only installed it if KRB5_KTNAME wasn't set already,
leading to a troubling inconsistency: in theory, clients could see
different sets of server principal names depending on whether they
use GSSAPI encryption. Always using krb_server_keyfile seems like
the right thing, so make both places do that. Also fix up
secure_open_gssapi()'s lack of a check for setenv() failure ---
it's unlikely, surely, but security-critical actions are no place
to be sloppy.
Also improve the associated documentation.
This patch does nothing about secure_open_gssapi()'s use of setenv(),
and indeed causes pg_GSS_recvauth() to use it too. That's nominally
against project portability rules, but since this code is only built
with --with-gssapi, I do not feel a need to do something about this
in the back branches. A fix will be forthcoming for HEAD though.
Back-patch to v12 where GSSAPI encryption was introduced. The
dubious behavior in pg_GSS_recvauth() goes back further, but it
didn't have anything to be inconsistent with, so let it be.
Discussion: https://postgr.es/m/2187460.1609263156@sss.pgh.pa.us
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/config.sgml
M src/backend/libpq/auth.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/utils/misc/postgresql.conf.sample
Sanitize IF NOT EXISTS in EXPLAIN for CTAS and matviews
commit : e665769e6d1e84b6650f53ed297058fc11152f7f
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 30 Dec 2020 21:23:24 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 30 Dec 2020 21:23:24 +0900
IF NOT EXISTS was ignored when specified in an EXPLAIN query for CREATE
MATERIALIZED VIEW or CREATE TABLE AS. Hence, if this clause was
specified, the caller would get a failure if the relation already
exists instead of a success with a NOTICE message.
This commit makes the behavior of IF NOT EXISTS in EXPLAIN consistent
with the non-EXPLAIN'd DDL queries, preventing a failure with IF NOT
EXISTS if the relation to-be-created already exists. The skip is done
before the SELECT query used for the relation is planned or executed,
and a "dummy" plan is generated instead depending on the format used by
EXPLAIN.
Author: Bharath Rupireddy
Reviewed-by: Zhijie Hou, Michael Paquier
Discussion: https://postgr.es/m/CALj2ACVa3oJ9O_wcGd+FtHWZds04dEKcakxphGz5POVgD4wC7Q@mail.gmail.com
M src/backend/commands/createas.c
M src/backend/commands/explain.c
M src/include/commands/createas.h
M src/test/regress/expected/matview.out
M src/test/regress/expected/select_into.out
M src/test/regress/sql/matview.sql
M src/test/regress/sql/select_into.sql
Extend the output plugin API to allow decoding of prepared xacts.
commit : 0aa8a01d04c8fe200b7a106878eebc3d0af9105c
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 30 Dec 2020 16:17:26 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 30 Dec 2020 16:17:26 +0530
This adds six methods to the output plugin API, adding support for
streaming changes of two-phase transactions at prepare time.
* begin_prepare
* filter_prepare
* prepare
* commit_prepared
* rollback_prepared
* stream_prepare
Most of this is a simple extension of the existing methods, with the
semantic difference that the transaction is not yet committed and maybe
aborted later.
Until now two-phase transactions were translated into regular transactions
on the subscriber, and the GID was not forwarded to it. None of the
two-phase commands were communicated to the subscriber.
This patch provides the infrastructure for logical decoding plugins to be
informed of two-phase commands Like PREPARE TRANSACTION, COMMIT PREPARED
and ROLLBACK PREPARED commands with the corresponding GID.
This also extends the 'test_decoding' plugin, implementing these new
methods.
This commit simply adds these new APIs and the upcoming patch to "allow
the decoding at prepare time in ReorderBuffer" will use these APIs.
Author: Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Peter Smith, Sawada Masahiko, and Dilip Kumar
Discussion:
https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru
https://postgr.es/m/CAMGcDxeqEpWj3fTXwqhSwBdXd2RS9jzwWscO-XbeCfso6ts3+Q@mail.gmail.com
M contrib/test_decoding/test_decoding.c
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/logical.c
M src/include/replication/logical.h
M src/include/replication/output_plugin.h
M src/include/replication/reorderbuffer.h
M src/tools/pgindent/typedefs.list
In pg_upgrade cross-version test, handle postfix operators.
commit : fa744697c79189a661f802d9a979d959b4454df0
author : Noah Misch <noah@leadboat.com>
date : Wed, 30 Dec 2020 01:43:43 -0800
committer: Noah Misch <noah@leadboat.com>
date : Wed, 30 Dec 2020 01:43:43 -0800
Commit 1ed6b895634ce0dc5fd4bd040e87252b32182cba eliminated support for
them, so drop them from regression databases before upgrading. This is
necessary but not sufficient for testing v13 -> v14 upgrades.
Discussion: https://postgr.es/m/449144.1600439950@sss.pgh.pa.us
M src/bin/pg_upgrade/test.sh
In pg_upgrade cross-version test, handle lack of oldstyle_length().
commit : 52202bb396b1e96c43bfd767d6e434b1c6fd2ae1
author : Noah Misch <noah@leadboat.com>
date : Wed, 30 Dec 2020 01:43:43 -0800
committer: Noah Misch <noah@leadboat.com>
date : Wed, 30 Dec 2020 01:43:43 -0800
This suffices for testing v12 -> v13; some other version pairs need more
changes. Back-patch to v10, which removed the function.
M src/bin/pg_upgrade/test.sh
Remove references to libpq_srcdir in adminpack and old_snapshot
commit : 107a2d4204ff4bf4ce05e3525f0d94fc0bd497ff
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 30 Dec 2020 12:38:16 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 30 Dec 2020 12:38:16 +0900
Those two modules included references to libpq's source path, without
using anything from libpq. Some copy-pastos done when each module was
created are likely at the origin of those useless references (aecf5ee
for old_snapshot, fe59e56 for adminpack).
Reviewed-by: Tom Lane, David Rowley
Discussion: https://postgr.es/m/X+LQpfLyk7jgzUki@paquier.xyz
M contrib/adminpack/Makefile
M contrib/old_snapshot/Makefile
Doc: fix up PDF build warnings from over-width table columns.
commit : f20dc2c8dd50a5c738d535205d5d44bff82d3f75
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Dec 2020 20:44:03 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Dec 2020 20:44:03 -0500
Addition of multirange info to tables 8.27 and 65.1 made them start
throwing "exceed the available area" warnings in PDF docs builds.
For 8.27, twiddling the existing column width hints was enough to
fix this. For 65.1, I twiddled the widths a little, but to really
fix it I had to insert a space after each comma in the table, to
allow a line break to occur there. (This seemed easier to read
and maintain than the alternative of inserting &zwsp; entities.)
Per buildfarm.
M doc/src/sgml/datatype.sgml
M doc/src/sgml/gist.sgml
Suppress log spam from multiple reports of SIGQUIT shutdown.
commit : 1f9158ba48122fa232db955a2ee324eec1848ba9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Dec 2020 18:02:38 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Dec 2020 18:02:38 -0500
When the postmaster sends SIGQUIT to its children, there's no real
need for all the children to log that fact; the postmaster already
made a log entry about it, so adding perhaps dozens or hundreds of
child-process log entries adds nothing of value. So, let's introduce
a new ereport level to specify "WARNING, but never send to log" and
use that for these messages.
Such a change wouldn't have been desirable before commit 7e784d1dc,
because if someone manually SIGQUIT's a backend, we *do* want to log
that. But now we can tell the difference between a signal that was
issued by the postmaster and one that was not with reasonable
certainty.
While we're here, also clear error_context_stack before ereport'ing,
to prevent error callbacks from being invoked in the signal-handler
context. This should reduce the odds of getting hung up while trying
to notify the client.
Per a suggestion from Andres Freund.
Discussion: https://postgr.es/m/20201225230331.hru3u6obyy6j53tk@alap3.anarazel.de
M src/backend/tcop/postgres.c
M src/backend/utils/error/elog.c
M src/include/utils/elog.h
Add support of multirange matching to the existing range GiST indexes
commit : db6335b5b1d6654b0e3104f36817800d127c1c91
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Dec 2020 23:36:43 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Dec 2020 23:36:43 +0300
6df7a9698b has introduced a set of operators between ranges and multiranges.
Existing GiST indexes for ranges could easily support majority of them.
This commit adds support for new operators to the existing range GiST indexes.
New operators resides the same strategy numbers as existing ones. Appropriate
check function is determined using the subtype.
Catversion is bumped.
M doc/src/sgml/gist.sgml
M doc/src/sgml/rangetypes.sgml
M src/backend/utils/adt/multirangetypes.c
M src/backend/utils/adt/rangetypes_gist.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_proc.dat
M src/include/utils/multirangetypes.h
M src/test/regress/expected/multirangetypes.out
M src/test/regress/expected/rangetypes.out
M src/test/regress/sql/multirangetypes.sql
M src/test/regress/sql/rangetypes.sql
Improve the signature of internal multirange functions
commit : d1d61a8b23b604faf797695eeacaa5da4fe64762
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Dec 2020 23:35:38 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Dec 2020 23:35:38 +0300
There is a set of *_internal() functions exposed in
include/utils/multirangetypes.h. This commit improves the signatures of these
functions in two ways.
* Add const qualifies where applicable.
* Replace multirange typecache argument with range typecache argument.
Multirange typecache was used solely to find the range typecache. At the
same time, range typecache is easier for the caller to find.
M src/backend/utils/adt/multirangetypes.c
M src/include/utils/multirangetypes.h
Implement operators for checking if the range contains a multirange
commit : 4d7684cc754f312aee468abb83ca4f7da94b30a3
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Dec 2020 23:35:33 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Dec 2020 23:35:33 +0300
We have operators for checking if the multirange contains a range but don't
have the opposite. This commit improves completeness of the operator set by
adding two new operators: @> (anyrange,anymultirange) and
<@(anymultirange,anyrange).
Catversion is bumped.
M doc/src/sgml/func.sgml
M src/backend/utils/adt/multirangetypes.c
M src/backend/utils/adt/multirangetypes_selfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_proc.dat
M src/include/utils/multirangetypes.h
M src/test/regress/expected/multirangetypes.out
M src/test/regress/sql/multirangetypes.sql
Fix bugs in comparison functions for multirange_bsearch_match()
commit : a5b81b6f0006ea0b502780ce7f73d295a225842c
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Dec 2020 23:35:26 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Dec 2020 23:35:26 +0300
Two functions multirange_range_overlaps_bsearch_comparison() and
multirange_range_contains_bsearch_comparison() contain bugs of returning -1
instead of 1. This commit fixes these bugs and adds corresponding regression
tests.
M src/backend/utils/adt/multirangetypes.c
M src/test/regress/expected/multirangetypes.out
M src/test/regress/sql/multirangetypes.sql
doc: Improve description of min_dynamic_shared_memory
commit : 1b3433e25f81d6fc4f231887ab965e1ea1bcb47e
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 29 Dec 2020 16:49:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 29 Dec 2020 16:49:14 +0900
While on it, fix one oversight in 90fbf7c, that introduced a reference
to an incorrect value for the compression level of pg_dump.
Author: Justin Pryzby
Reviewed-by: Thomas Munro, Michael Paquier
Discussion: https://postgr.es/m/CA+hUKGJRTLWWPcQfjm_xaOk98M8aROK903X92O0x-4vLJPWrrA@mail.gmail.com
M doc/src/sgml/config.sgml
M src/bin/pg_dump/pg_backup_archiver.h
Improve log messages related to pg_hba.conf not matching a connection.
commit : 3995c424984e991b1069a2869af972dc07574c0b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 17:58:58 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 17:58:58 -0500
Include details on whether GSS encryption has been activated;
since we added "hostgssenc" type HBA entries, that's relevant info.
Kyotaro Horiguchi and Tom Lane. Back-patch to v12 where
GSS encryption was introduced.
Discussion: https://postgr.es/m/e5b0b6ed05764324a2f3fe7acfc766d5@smhi.se
M src/backend/libpq/auth.c
Fix assorted issues in backend's GSSAPI encryption support.
commit : 622ae4621ece72a9f64b5602c74d7aaf373c1631
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 17:44:17 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 17:44:17 -0500
Unrecoverable errors detected by GSSAPI encryption can't just be
reported with elog(ERROR) or elog(FATAL), because attempting to
send the error report to the client is likely to lead to infinite
recursion or loss of protocol sync. Instead make this code do what
the SSL encryption code has long done, which is to just report any
such failure to the server log (with elevel COMMERROR), then pretend
we've lost the connection by returning errno = ECONNRESET.
Along the way, fix confusion about whether message translation is done
by pg_GSS_error() or its callers (the latter should do it), and make
the backend version of that function work more like the frontend
version.
Avoid allocating the port->gss struct until it's needed; we surely
don't need to allocate it in the postmaster.
Improve logging of "connection authorized" messages with GSS enabled.
(As part of this, I back-patched the code changes from dc11f31a1.)
Make BackendStatusShmemSize() account for the GSS-related space that
will be allocated by CreateSharedBackendStatus(). This omission
could possibly cause out-of-shared-memory problems with very high
max_connections settings.
Remove arbitrary, pointless restriction that only GSS authentication
can be used on a GSS-encrypted connection.
Improve documentation; notably, document the fact that libpq now
prefers GSS encryption over SSL encryption if both are possible.
Per report from Mikael Gustavsson. Back-patch to v12 where
this code was introduced.
Discussion: https://postgr.es/m/e5b0b6ed05764324a2f3fe7acfc766d5@smhi.se
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/runtime.sgml
M src/backend/libpq/auth.c
M src/backend/libpq/be-gssapi-common.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/libpq/be-secure.c
M src/backend/libpq/hba.c
M src/backend/libpq/pqcomm.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/utils/init/postinit.c
M src/include/libpq/be-gssapi-common.h
M src/include/libpq/libpq-be.h
Fix bugs in libpq's GSSAPI encryption support.
commit : ff6ce9a3a691a96e8e47ed449bc51c5a178e6931
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 15:43:44 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 15:43:44 -0500
The critical issue fixed here is that if a GSSAPI-encrypted connection
is successfully made, pqsecure_open_gss() cleared conn->allow_ssl_try,
as an admittedly-hacky way of preventing us from then trying to tunnel
SSL encryption over the already-encrypted connection. The problem
with that is that if we abandon the GSSAPI connection because of a
failure during authentication, we would not attempt SSL encryption
in the next try with the same server. This can lead to unexpected
connection failure, or silently getting a non-encrypted connection
where an encrypted one is expected.
Fortunately, we'd only manage to make a GSSAPI-encrypted connection
if both client and server hold valid tickets in the same Kerberos
infrastructure, which is a relatively uncommon environment.
Nonetheless this is a very nasty bug with potential security
consequences. To fix, don't reset the flag, instead adding a
check for conn->gssenc being already true when deciding whether
to try to initiate SSL.
While here, fix some lesser issues in libpq's GSSAPI code:
* Use the need_new_connection stanza when dropping an attempted
GSSAPI connection, instead of partially duplicating that code.
The consequences of this are pretty minor: AFAICS it could only
lead to auth_req_received or password_needed remaining set when
they shouldn't, which is not too harmful.
* Fix pg_GSS_error() to not repeat the "mprefix" it's given multiple
times, and to notice any failure return from gss_display_status().
* Avoid gratuitous dependency on NI_MAXHOST in
pg_GSS_load_servicename().
Per report from Mikael Gustavsson. Back-patch to v12 where
this code was introduced.
Discussion: https://postgr.es/m/e5b0b6ed05764324a2f3fe7acfc766d5@smhi.se
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-gssapi-common.c
M src/interfaces/libpq/fe-secure-gssapi.c
Expose the default for channel_binding in PQconndefaults().
commit : cf61b0734c61d93c62827fe4e44fa2162a533b8e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 12:13:40 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 12:13:40 -0500
If there's a static default value for a connection option,
it should be shown in the PQconninfoOptions array.
Daniele Varrazzo
Discussion: https://postgr.es/m/CA+mi_8Zo8Rgn7p+6ZRY7QdDu+23ukT9AvoHNyPbgKACxwgGhZA@mail.gmail.com
M src/interfaces/libpq/fe-connect.c
Further fix thinko in plpgsql memory leak fix.
commit : 5f2e09bcccd771629fb7a2885f8c468ae0f7fb33
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 11:55:23 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 11:55:23 -0500
There's a second call of get_eval_mcontext() that should also be
get_stmt_mcontext(). This is actually dead code, since no
interesting allocations happen before switching back to the
original context, but we should keep it in sync with the other
call to forestall possible future bugs.
Discussion: https://postgr.es/m/f075f7be-c654-9aa8-3ffc-e9214622f02a@enterprisedb.com
M src/pl/plpgsql/src/pl_exec.c
Fix thinko in plpgsql memory leak fix.
commit : ea80d8d9437e80de6506dbfe3765d834653312bf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 11:41:25 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 11:41:25 -0500
Commit a6b1f5365 intended to place the transient "target" list of
a CALL statement in the function's statement-lifespan context,
but I fat-fingered that and used get_eval_mcontext() instead of
get_stmt_mcontext(). The eval_mcontext belongs to the "simple
expression" infrastructure, which is destroyed at transaction end.
The net effect is that a CALL in a procedure to another procedure
that has OUT or INOUT parameters would fail if the called procedure
did a COMMIT.
Per report from Peter Eisentraut. Back-patch to v11, like the
prior patch.
Discussion: https://postgr.es/m/f075f7be-c654-9aa8-3ffc-e9214622f02a@enterprisedb.com
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_call.sql
Fix inconsistent code with shared invalidations of snapshots
commit : 643428c54b95d472c6d949c3c3c11f347e371a2b
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 28 Dec 2020 22:16:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 28 Dec 2020 22:16:49 +0900
The code in charge of processing a single invalidation message has been
using since 568d413 the structure for relation mapping messages. This
had fortunately no consequence as both locate the database ID at the
same location, but it could become a problem in the future if this area
of the code changes.
Author: Konstantin Knizhnik
Discussion: https://postgr.es/m/8044c223-4d3a-2cdb-42bf-29940840ce94@postgrespro.ru
Backpatch-through: 9.5
M src/backend/utils/cache/inval.c
postgres_fdw: Fix connection leak.
commit : e3ebcca843a4703938b9f40a4811f43c4b315753
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 28 Dec 2020 19:56:13 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 28 Dec 2020 19:56:13 +0900
In postgres_fdw, the cached connections to foreign servers will not be
closed until the local session exits if the user mappings or foreign servers
that those connections depend on are dropped. Those connections can be
leaked.
To fix that connection leak issue, after a change to a pg_foreign_server
or pg_user_mapping catalog entry, this commit makes postgres_fdw close
the connections depending on that entry immediately if current
transaction has not used those connections yet. Otherwise, mark those
connections as invalid and then close them at the end of current transaction,
since they cannot be closed in the midst of the transaction using them.
Closed connections will be remade at the next opportunity if necessary.
Back-patch to all supported branches.
Author: Bharath Rupireddy
Reviewed-by: Zhihong Yu, Zhijie Hou, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVNcGH_6qLY-4_tXz8JLvA+4yeBThRfxMz7Oxbk1aHcpQ@mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Revert "Add key management system" (978f869b99) & later commits
commit : 3187ef7c46c5b884267a88f2d6119c9a05f1bbba
author : Bruce Momjian <bruce@momjian.us>
date : Sun, 27 Dec 2020 21:37:42 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sun, 27 Dec 2020 21:37:42 -0500
The patch needs test cases, reorganization, and cfbot testing.
Technically reverts commits 5c31afc49d..e35b2bad1a (exclusive/inclusive)
and 08db7c63f3..ccbe34139b.
Reported-by: Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/E1ktAAG-0002V2-VB@gemulon.postgresql.org
M doc/src/sgml/config.sgml
D doc/src/sgml/database-encryption.sgml
M doc/src/sgml/filelist.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/postgres.sgml
M doc/src/sgml/ref/allfiles.sgml
M doc/src/sgml/ref/initdb.sgml
D doc/src/sgml/ref/pg_alterckey.sgml
M doc/src/sgml/ref/pg_ctl-ref.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/postgres-ref.sgml
M doc/src/sgml/reference.sgml
M doc/src/sgml/storage.sgml
M src/backend/Makefile
M src/backend/access/transam/xlog.c
M src/backend/bootstrap/bootstrap.c
D src/backend/crypto/Makefile
D src/backend/crypto/ckey_aws.sh.sample
D src/backend/crypto/ckey_direct.sh.sample
D src/backend/crypto/ckey_passphrase.sh.sample
D src/backend/crypto/ckey_piv_nopin.sh.sample
D src/backend/crypto/ckey_piv_pin.sh.sample
D src/backend/crypto/kmgr.c
D src/backend/crypto/ssl_passphrase.sh.sample
M src/backend/libpq/be-secure-common.c
M src/backend/main/main.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/replication/basebackup.c
M src/backend/storage/ipc/ipci.c
M src/backend/storage/lmgr/lwlocknames.txt
M src/backend/tcop/postgres.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/pg_controldata.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/Makefile
M src/bin/initdb/initdb.c
D src/bin/pg_alterckey/.gitignore
D src/bin/pg_alterckey/Makefile
D src/bin/pg_alterckey/pg_alterckey.c
M src/bin/pg_controldata/pg_controldata.c
M src/bin/pg_ctl/pg_ctl.c
M src/bin/pg_resetwal/pg_resetwal.c
M src/bin/pg_rewind/filemap.c
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/file.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/server.c
M src/common/Makefile
D src/common/cipher.c
D src/common/cipher_openssl.c
D src/common/kmgr_utils.c
M src/include/catalog/pg_control.h
D src/include/common/cipher.h
D src/include/common/kmgr_utils.h
D src/include/crypto/kmgr.h
M src/include/pgstat.h
M src/include/postmaster/postmaster.h
M src/include/utils/guc_tables.h
M src/tools/msvc/Mkvcbuild.pm
Second attempt to stabilize 05c02589.
commit : facad31474ac6dace3894ebc7c45dc3cc829422e
author : Jeff Davis <jdavis@postgresql.org>
date : Sun, 27 Dec 2020 12:09:00 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Sun, 27 Dec 2020 12:09:00 -0800
Removing the EXPLAIN test to stabilize the buildfarm. The execution
test should still be effective to catch the bug even if the plan is
slightly different on different platforms.
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Stabilize test introduced in 05c02589, per buildfarm.
commit : fa0fdf0510df1a21f42ac9f232f77a79b8577152
author : Jeff Davis <jdavis@postgresql.org>
date : Sun, 27 Dec 2020 09:47:23 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Sun, 27 Dec 2020 09:47:23 -0800
In passing, make the capitalization match the rest of the file.
Reported-by: Tom Lane
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Fix bug #16784 in Disk-based Hash Aggregation.
commit : 05c0258966b502fae1bd63dcbe74d52f5c6f6948
author : Jeff Davis <jdavis@postgresql.org>
date : Sat, 26 Dec 2020 17:25:30 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Sat, 26 Dec 2020 17:25:30 -0800
Before processing tuples, agg_refill_hash_table() was setting all
pergroup pointers to NULL to signal to advance_aggregates() that it
should not attempt to advance groups that had spilled.
The problem was that it also set the pergroups for sorted grouping
sets to NULL, which caused rescanning to fail.
Instead, change agg_refill_hash_table() to only set the pergroups for
hashed grouping sets to NULL; and when compiling the expression, pass
doSort=false.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/16784-7ff169bf2c3d1588%40postgresql.org
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
initdb: document that -K requires an argument
commit : ccbe34139b7d9059f4af3405b2411f653cc2ce7c
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 26 Dec 2020 10:00:05 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 26 Dec 2020 10:00:05 -0500
Reported-by: "Shinoda, Noriyoshi"
Discussion: https://postgr.es/m/TU4PR8401MB1152E92B4D44C81E496D6032EEDB0@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM
Author: "Shinoda, Noriyoshi"
Backpatch-through: msater
M doc/src/sgml/ref/initdb.sgml
M src/bin/initdb/initdb.c
auth commands: list specific commands to install in Makefile
commit : ba6725df364cd30e068ccd962fa426d9b7b44066
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 26 Dec 2020 09:25:05 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 26 Dec 2020 09:25:05 -0500
Previously I used Makefile functions.
Backpatch-through: master
M src/backend/Makefile
pg_alterckey: remove TAP check rules from Makefile
commit : e174a6f1937248886e124cdb68408e727aeea278
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 26 Dec 2020 09:03:12 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 26 Dec 2020 09:03:12 -0500
Reported-by: Pavel Stehule, Michael Paquier
Discussion: https://postgr.es/m/CAFj8pRBRNo4co5bqCx4BLx1ZZ45Z_T-opPxA+u7SLp7gAtBpNA@mail.gmail.com
Backpatch-through: master
M src/bin/pg_alterckey/Makefile
Add scripts for retrieving the cluster file encryption key
commit : d7602afa2ef6d8b2018103dccd89e75b4985ac06
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 26 Dec 2020 01:19:09 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 26 Dec 2020 01:19:09 -0500
Scripts are passphrase, direct, AWS, and two Yubikey ones.
Backpatch-through: master
M src/backend/Makefile
A src/backend/crypto/ckey_aws.sh.sample
A src/backend/crypto/ckey_direct.sh.sample
A src/backend/crypto/ckey_passphrase.sh.sample
A src/backend/crypto/ckey_piv_nopin.sh.sample
A src/backend/crypto/ckey_piv_pin.sh.sample
A src/backend/crypto/ssl_passphrase.sh.sample
doc: fix SGML markup for pg_alterckey from commit 62afb42a7f
commit : 3d4843babce087ae8dc734aeec36db084001a0e4
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 26 Dec 2020 01:10:24 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 26 Dec 2020 01:10:24 -0500
Backpatch-through: master
M doc/src/sgml/ref/pg_alterckey.sgml
M doc/src/sgml/reference.sgml
pg_alterckey: adjust doc build and Win32 sleep/open build fails
commit : 82f8c45be551a2940e3cbaf08aa7a521bdac8331
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 22:47:16 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 22:47:16 -0500
Fix for commit 62afb42a7f.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/1252111.1608953815@sss.pgh.pa.us
Backpatch-through: master
M doc/src/sgml/ref/allfiles.sgml
R100 doc/src/sgml/ref/pg_alterkey.sgml doc/src/sgml/ref/pg_alterckey.sgml
M src/bin/pg_alterckey/pg_alterckey.c
Fix function call typo in frontend Win32 code, commit 978f869b99
commit : 7705f8ca03713457bf7cde0b859a1ab45e4f268b
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 20:49:50 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 20:49:50 -0500
Reported-by: buildfarm member walleye
Backpatch-through: master
M src/common/kmgr_utils.c
Allow ssl_passphrase_command to prompt the terminal
commit : 300e430c76644b483f70bc833d6251a6a7fb84a6
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 20:41:06 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 20:41:06 -0500
Previously the command could not access the terminal for a passphrase.
Backpatch-through: master
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/pg_ctl-ref.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M src/backend/libpq/be-secure-common.c
Add pg_alterckey utility to change the cluster key
commit : 62afb42a7f9f533efc6c19f462c3a848fa4ddb63
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 20:24:53 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 20:24:53 -0500
This can change the key that encrypts the data encryption keys used for
cluster file encryption.
Discussion: https://postgr.es/m/20201202213814.GG20285@momjian.us
Backpatch-through: master
A doc/src/sgml/ref/pg_alterkey.sgml
M src/bin/Makefile
A src/bin/pg_alterckey/.gitignore
A src/bin/pg_alterckey/Makefile
A src/bin/pg_alterckey/pg_alterckey.c
remove missing reference to crypto test from patch 978f869b99
commit : f234899353f8998bdbd265125ce4a505a312d910
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 15:13:16 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 15:13:16 -0500
Reported-by: Tom Lane
Discussion: https://postgr.es/m/1205031.1608925990@sss.pgh.pa.us
Backpatch-through: master
M src/test/Makefile
Really fix the dummy implementations in cipher.c.
commit : 0848cf4f553bda5a9f044c56b3bec19929ee24e6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 25 Dec 2020 14:44:17 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 25 Dec 2020 14:44:17 -0500
945083b2f wasn't enough to silence compiler warnings.
M src/common/cipher.c
fix no-return function call in cipher.c from commit 978f869b99
commit : 8e59813e2287db90044537b1093c0869020ffa4a
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 14:40:46 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 14:40:46 -0500
Reported-by: buildfarm member sifaka
Backpatch-through: master
M src/common/cipher.c
fixes docs and missing initdb help option for commit 978f869b99
commit : 26d60f2a6c1c93838a49ac4d685652b1bc92f547
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 14:00:12 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 14:00:12 -0500
Reported-by: Erik Rijkers
Discussion: https://postgr.es/m/a27e7bb60fc4c4a1fe960f7b055ba822@xs4all.nl
Backpatch-through: master
M doc/src/sgml/database-encryption.sgml
M src/bin/initdb/initdb.c
Invalidate acl.c caches when pg_authid changes.
commit : 08db7c63f34e6d9a402af81596e303f5b38d19b0
author : Noah Misch <noah@leadboat.com>
date : Fri, 25 Dec 2020 10:41:59 -0800
committer: Noah Misch <noah@leadboat.com>
date : Fri, 25 Dec 2020 10:41:59 -0800
This makes existing sessions reflect "ALTER ROLE ... [NO]INHERIT" as
quickly as they have been reflecting "GRANT role_name". Back-patch to
9.5 (all supported versions).
Reviewed by Nathan Bossart.
Discussion: https://postgr.es/m/20201221095028.GB3777719@rfd.leadboat.com
M src/backend/utils/adt/acl.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
remove uint128 requirement from patch 978f869b99 (CFE)
commit : e35b2bad1a10a8eef9c1ffb563847b9c9df0cfce
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 11:35:59 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 11:35:59 -0500
Used char[16] instead.
Reported-by: buildfarm member florican
Backpatch-through: master
M src/common/kmgr_utils.c
M src/include/common/kmgr_utils.h
Fix return value and const declaration from commit 978f869b99
commit : 945083b2f7e6c19c8921c655cac6778acb1e3e03
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 11:00:32 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 11:00:32 -0500
This fixes the non-OpenSSL compile case.
Reported-by: buildfarm member sifaka
Backpatch-through: master
M src/common/cipher.c
Add key management system
commit : 978f869b992f9fca343e99d6fdb71073c76e869a
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 10:19:44 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 25 Dec 2020 10:19:44 -0500
This adds a key management system that stores (currently) two data
encryption keys of length 128, 192, or 256 bits. The data keys are
AES256 encrypted using a key encryption key, and validated via GCM
cipher mode. A command to obtain the key encryption key must be
specified at initdb time, and will be run at every database server
start. New parameters allow a file descriptor open to the terminal to
be passed. pg_upgrade support has also been added.
Discussion: https://postgr.es/m/CA+fd4k7q5o6Nc_AaX6BcYM9yqTbC6_pnH-6nSD=54Zp6NBQTCQ@mail.gmail.com
Discussion: https://postgr.es/m/20201202213814.GG20285@momjian.us
Author: Masahiko Sawada, me, Stephen Frost
M doc/src/sgml/config.sgml
A doc/src/sgml/database-encryption.sgml
M doc/src/sgml/filelist.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/postgres.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/ref/pg_ctl-ref.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/postgres-ref.sgml
M doc/src/sgml/storage.sgml
M src/backend/Makefile
M src/backend/access/transam/xlog.c
M src/backend/bootstrap/bootstrap.c
A src/backend/crypto/Makefile
A src/backend/crypto/kmgr.c
M src/backend/main/main.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/replication/basebackup.c
M src/backend/storage/ipc/ipci.c
M src/backend/storage/lmgr/lwlocknames.txt
M src/backend/tcop/postgres.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/pg_controldata.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/initdb/initdb.c
M src/bin/pg_controldata/pg_controldata.c
M src/bin/pg_ctl/pg_ctl.c
M src/bin/pg_resetwal/pg_resetwal.c
M src/bin/pg_rewind/filemap.c
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/file.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/server.c
M src/common/Makefile
A src/common/cipher.c
A src/common/cipher_openssl.c
A src/common/kmgr_utils.c
M src/include/catalog/pg_control.h
A src/include/common/cipher.h
A src/include/common/kmgr_utils.h
A src/include/crypto/kmgr.h
M src/include/pgstat.h
M src/include/postmaster/postmaster.h
M src/include/utils/guc_tables.h
M src/test/Makefile
M src/tools/msvc/Mkvcbuild.pm
Avoid time-of-day-dependent failure in log rotation test.
commit : 5c31afc49d0b62b357218b6f8b01782509ef8acd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Dec 2020 21:37:46 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Dec 2020 21:37:46 -0500
Buildfarm members pogona and petalura have shown a failure when
pg_ctl/t/004_logrotate.pl starts just before local midnight.
The default rotate-at-midnight behavior occurs just before the
Perl script examines current_logfiles, so it figures that the
rotation it's already requested has occurred ... but in reality,
that rotation happens just after it looks, so the expected new
log data goes into a different file than the one it's examining.
In HEAD, src/test/kerberos/t/001_auth.pl has acquired similar code
that evidently has a related failure mode. Besides being quite new,
few buildfarm critters run that test, so it's unsurprising that
we've not yet seen a failure there.
Fix both cases by setting log_rotation_age = 0 so that no time-based
rotation can occur. Also absorb 004_logrotate.pl's decision to
set lc_messages = 'C' into the kerberos test, in hopes that it will
work in non-English prevailing locales.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=pogona&dt=2020-12-24%2022%3A10%3A04
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=petalura&dt=2020-02-01%2022%3A20%3A04
M src/bin/pg_ctl/t/004_logrotate.pl
M src/test/kerberos/t/001_auth.pl
revert removal of hex_decode() from ecpg from commit c3826f831e
commit : 558a6e8e21367dae9a294291337be10a8a4afd0a
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 24 Dec 2020 18:21:37 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 24 Dec 2020 18:21:37 -0500
ecpglib on certain platforms can't handle the pg_log_fatal calls from
libraries. This was reported by the buildfarm. It needs a refactoring
and return value change if it is later removed.
Backpatch-through: master
M src/interfaces/ecpg/ecpglib/data.c
move hex_decode() to /common so it can be called from frontend
commit : c3826f831e6e63e13a749fd3ab9fd7106707b549
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 24 Dec 2020 17:25:48 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 24 Dec 2020 17:25:48 -0500
This allows removal of a copy of hex_decode() from ecpg, and will be
used by the soon-to-be added pg_alterckey command.
Backpatch-through: master
M src/backend/utils/adt/encode.c
M src/backend/utils/adt/varlena.c
M src/common/Makefile
A src/common/hex_decode.c
A src/include/common/hex_decode.h
M src/include/utils/builtins.h
M src/interfaces/ecpg/ecpglib/data.c
M src/tools/msvc/Mkvcbuild.pm
Fix race condition between shutdown and unstarted background workers.
commit : 7519bd16d1a2a2007b3604f6740611c0b75a6cd0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Dec 2020 17:00:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Dec 2020 17:00:43 -0500
If a database shutdown (smart or fast) is commanded between the time
some process decides to request a new background worker and the time
that the postmaster can launch that worker, then nothing happens
because the postmaster won't launch any bgworkers once it's exited
PM_RUN state. This is fine ... unless the requesting process is
waiting for that worker to finish (or even for it to start); in that
case the requestor is stuck, and only manual intervention will get us
to the point of being able to shut down.
To fix, cancel pending requests for workers when the postmaster sends
shutdown (SIGTERM) signals, and similarly cancel any new requests that
arrive after that point. (We can optimize things slightly by only
doing the cancellation for workers that have waiters.) To fit within
the existing bgworker APIs, the "cancel" is made to look like the
worker was started and immediately stopped, causing deregistration of
the bgworker entry. Waiting processes would have to deal with
premature worker exit anyway, so this should introduce no bugs that
weren't there before. We do have a side effect that registration
records for restartable bgworkers might disappear when theoretically
they should have remained in place; but since we're shutting down,
that shouldn't matter.
Back-patch to v10. There might be value in putting this into 9.6
as well, but the management of bgworkers is a bit different there
(notably see 8ff518699) and I'm not convinced it's worth the effort
to validate the patch for that branch.
Discussion: https://postgr.es/m/661570.1608673226@sss.pgh.pa.us
M contrib/pg_prewarm/autoprewarm.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/postmaster.c
M src/include/postmaster/bgworker_internals.h
Improve client error messages for immediate-stop situations.
commit : 7e784d1dc191be24480a6b31a4ddc8e0e52be24d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Dec 2020 12:58:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Dec 2020 12:58:32 -0500
Up to now, if the DBA issued "pg_ctl stop -m immediate", the message
sent to clients was the same as for a crash-and-restart situation.
This is confusing, not least because the message claims that the
database will soon be up again, something we have no business
predicting.
Improve things so that we can generate distinct messages for the two
cases (and also recognize an ad-hoc SIGQUIT, should somebody try that).
To do that, add a field to pmsignal.c's shared memory data structure
that the postmaster sets just before broadcasting SIGQUIT to its
children. No interlocking seems to be necessary; the intervening
signal-sending and signal-receipt should sufficiently serialize accesses
to the field. Hence, this isn't any riskier than the existing usages
of pmsignal.c.
We might in future extend this idea to improve other
postmaster-to-children signal scenarios, although none of them
currently seem to be as badly overloaded as SIGQUIT.
Discussion: https://postgr.es/m/559291.1608587013@sss.pgh.pa.us
M src/backend/postmaster/postmaster.c
M src/backend/storage/ipc/pmsignal.c
M src/backend/tcop/postgres.c
M src/include/storage/pmsignal.h
Fix typos and grammar in docs and comments
commit : 90fbf7c57df601c7e0b43ae7cf71f0f69908a7cc
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 24 Dec 2020 17:05:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 24 Dec 2020 17:05:49 +0900
This fixes several areas of the documentation and some comments in
matters of style, grammar, or even format.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20201222041153.GK30237@telsasoft.com
M contrib/pg_stat_statements/pg_stat_statements.c
M doc/src/sgml/amcheck.sgml
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/cube.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/pgstatstatements.sgml
M doc/src/sgml/rangetypes.sgml
M doc/src/sgml/ref/create_type.sgml
M doc/src/sgml/ref/explain.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_verifybackup.sgml
M doc/src/sgml/ref/prepare.sgml
M doc/src/sgml/sources.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/xlog.c
M src/backend/partitioning/partprune.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_directory.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_upgrade/check.c
M src/tools/msvc/README
dummy commit
commit : 6ecf488d91ceb93a8be907a24c5d8a90358534ed
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 25 Nov 2020 12:29:21 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 25 Nov 2020 12:29:21 -0500
M doc/src/sgml/plperl.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/plpython.sgml
M doc/src/sgml/pltcl.sgml
M doc/src/sgml/spi.sgml
Fix portability issues with parsing of recovery_target_xid
commit : 6db27037b99358018353b8748719cfa9de484988
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 23 Dec 2020 12:51:22 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 23 Dec 2020 12:51:22 +0900
The parsing of this parameter has been using strtoul(), which is not
portable across platforms. On most Unix platforms, unsigned long has a
size of 64 bits, while on Windows it is 32 bits. It is common in
recovery scenarios to rely on the output of txid_current() or even the
newer pg_current_xact_id() to get a transaction ID for setting up
recovery_target_xid. The value returned by those functions includes the
epoch in the computed result, which would cause strtoul() to fail where
unsigned long has a size of 32 bits once the epoch is incremented.
WAL records and 2PC data include only information about 32-bit XIDs and
it is not possible to have XIDs across more than one epoch, so
discarding the high bits from the transaction ID set has no impact on
recovery. On the contrary, the use of strtoul() prevents a consistent
behavior across platforms depending on the size of unsigned long.
This commit changes the parsing of recovery_target_xid to use
pg_strtouint64() instead, available down to 9.6. There is one TAP test
stressing recovery with recovery_target_xid, where a tweak based on
pg_reset{xlog,wal} is added to bump the XID epoch so as this change gets
tested, as per an idea from Alexander Lakhin.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/16780-107fd0c0385b1035@postgresql.org
Backpatch-through: 9.6
M src/backend/utils/misc/guc.c
M src/test/recovery/t/003_recovery_targets.pl
Improve autoprewarm's handling of early-shutdown scenarios.
commit : ff769831e081a4e69147019128a849ced43ec802
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Dec 2020 13:23:49 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Dec 2020 13:23:49 -0500
Bad things happen if the DBA issues "pg_ctl stop -m fast" before
autoprewarm finishes loading its list of blocks to prewarm.
The current worker process successfully terminates early, but
(if this wasn't the last database with blocks to prewarm) the
leader process will just try to launch another worker for the
next database. Since the postmaster is now in PM_WAIT_BACKENDS
state, it ignores the launch request, and the leader just sits
until it's killed manually.
This is mostly the fault of our half-baked design for launching
background workers, but a proper fix for that is likely to be
too invasive to be back-patchable. To ameliorate the situation,
fix apw_load_buffers() to check whether SIGTERM has arrived
just before trying to launch another worker. That leaves us with
only a very narrow window in each worker launch where SIGTERM
could occur between the launch request and successful worker start.
Another issue is that if the leader process does manage to exit,
it unconditionally rewrites autoprewarm.blocks with only the
blocks currently in shared buffers, thus forgetting any blocks
that we hadn't reached yet while prewarming. This seems quite
unhelpful, since the next database start will then not have the
expected prewarming benefit. Fix it to not modify the file if
we shut down before the initial load attempt is complete.
Per bug #16785 from John Thompson. Back-patch to v11 where
the autoprewarm code was introduced.
Discussion: https://postgr.es/m/16785-c0207d8c67fb5f25@postgresql.org
M contrib/pg_prewarm/autoprewarm.c
Increase timeout in 021_row_visibility.pl.
commit : 08dde1b3dc78e90e01f3af8e89ab27ed37cd8963
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Dec 2020 11:10:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Dec 2020 11:10:12 -0500
Commit 7b28913bc figured 30 seconds is long enough for anybody,
but in contexts like valgrind runs, it isn't necessarily.
M src/test/recovery/t/021_row_visibility.pl
Improve find_em_expr_usable_for_sorting_rel comment
commit : 1ca2eb103113d64930cf1120ba2beaaebd1076f4
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 22 Dec 2020 02:00:39 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 22 Dec 2020 02:00:39 +0100
Clarify the relationship between find_em_expr_usable_for_sorting_rel and
prepare_sort_from_pathkeys, i.e. what restrictions need to be shared
between those two places.
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs%3DhC0mSksZC%3DH5M8LSchj5e5OxpTAg%40mail.gmail.com
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/plan/createplan.c
Don't search for volatile expr in find_em_expr_usable_for_sorting_rel
commit : 9aff4dc01fbb9e126efafd839ae8541362328f1a
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 19:37:14 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 19:37:14 +0100
While prepare_sort_from_pathkeys has to be concerned about matching up
a volatile expression to the proper tlist entry, we don't need to do
that in find_em_expr_usable_for_sorting_rel becausee such a sort will
have to be postponed anyway.
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs%3DhC0mSksZC%3DH5M8LSchj5e5OxpTAg%40mail.gmail.com
M src/backend/optimizer/path/equivclass.c
Disallow SRFs when considering sorts below Gather Merge
commit : fac1b470a9f7e846534620b78ea4cf122ed432b2
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 18:58:32 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 18:58:32 +0100
While we do allow SRFs in ORDER BY, scan/join processing should not
consider such cases - such sorts should only happen via final Sort atop
a ProjectSet. So make sure we don't try adding such sorts below Gather
Merge, just like we do for expressions that are volatile and/or not
parallel safe.
Backpatch to PostgreSQL 13, where this code was introduced as part of
the Incremental Sort patch.
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs=hC0mSksZC=H5M8LSchj5e5OxpTAg@mail.gmail.com
Discussion: https://postgr.es/m/295524.1606246314%40sss.pgh.pa.us
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/util/tlist.c
M src/include/optimizer/optimizer.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Remove "invalid concatenation of jsonb objects" error case.
commit : ff5d5611c01f60525c30b2c3ebc16d05edb7956d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Dec 2020 13:11:29 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Dec 2020 13:11:29 -0500
The jsonb || jsonb operator arbitrarily rejected certain combinations
of scalar and non-scalar inputs, while being willing to concatenate
other combinations. This was of course quite undocumented. Rather
than trying to document it, let's just remove the restriction,
creating a uniform rule that unless we are handling an object-to-object
concatenation, non-array inputs are converted to one-element arrays,
resulting in an array-to-array concatenation. (This does not change
the behavior for any case that didn't throw an error before.)
Per complaint from Joel Jacobson. Back-patch to all supported branches.
Discussion: https://postgr.es/m/163099.1608312033@sss.pgh.pa.us
M doc/src/sgml/func.sgml
M src/backend/utils/adt/jsonfuncs.c
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql
Check parallel safety in generate_useful_gather_paths
commit : 86b7cca72d4d0a4e043fac0a2cdd56218ff2f258
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 18:29:46 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 18:29:46 +0100
Commit ebb7ae839d ensured we ignore pathkeys with volatile expressions
when considering adding a sort below a Gather Merge. Turns out we need
to care about parallel safety of the pathkeys too, otherwise we might
try sorting e.g. on results of a correlated subquery (as demonstrated
by a report from Luis Roberto).
Initial investigation by Tom Lane, patch by James Coleman. Backpatch
to 13, where the code was instroduced (as part of Incremental Sort).
Reported-by: Luis Roberto
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/622580997.37108180.1604080457319.JavaMail.zimbra%40siscobra.com.br
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs=hC0mSksZC=H5M8LSchj5e5OxpTAg@mail.gmail.com
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/equivclass.c
M src/include/optimizer/paths.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Consider unsorted paths in generate_useful_gather_paths
commit : f4a3c0b06250ddc8ae09b59b87cf68e9bc0d7ca1
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 18:09:57 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 18:09:57 +0100
generate_useful_gather_paths used to skip unsorted paths (without any
pathkeys), but that is unnecessary - the later code actually can handle
such paths just fine by adding a Sort node. This is clearly a thinko,
preventing construction of useful plans.
Backpatch to 13, where Incremental Sort was introduced.
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs=hC0mSksZC=H5M8LSchj5e5OxpTAg@mail.gmail.com
M src/backend/optimizer/path/allpaths.c
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Fix compiler warning in multirange_constructor0()
commit : 29f8f546767ebb4253f6dc37815e2d95fff9acd0
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 21 Dec 2020 14:25:32 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 21 Dec 2020 14:25:32 +0300
Discussion: https://postgr.es/m/X%2BBP8XE0UpIB6Yvh%40paquier.xyz
Author: Michael Paquier
M src/backend/utils/adt/multirangetypes.c
Refactor logic to check for ASCII-only characters in string
commit : 93e8ff8701a65a70ea8826bdde7fdbbd9c285477
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 21 Dec 2020 09:37:11 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 21 Dec 2020 09:37:11 +0900
The same logic was present for collation commands, SASLprep and
pgcrypto, so this removes some code.
Author: Michael Paquier
Reviewed-by: Stephen Frost, Heikki Linnakangas
Discussion: https://postgr.es/m/X9womIn6rne6Gud2@paquier.xyz
M contrib/pgcrypto/pgp-pgsql.c
M src/backend/commands/collationcmds.c
M src/common/saslprep.c
M src/common/string.c
M src/include/common/string.h
Fix typalign in rangetypes statistics
commit : 4e1ee79e3182256d9c8ddbc1ce9c4e8419c611ff
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 21 Dec 2020 00:26:13 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 21 Dec 2020 00:26:13 +0300
6df7a9698b introduces multirange types, whose typanalyze function shares
infrastructure with range types typanalyze function. Since 6df7a9698b,
information about type gathered by statistics is filled from typcache.
But typalign is mistakenly always set to double. This commit fixes this
oversight.
M src/backend/utils/adt/rangetypes_typanalyze.c
Doc: fix description of how to use src/tutorial files.
commit : eea1e08cfc95bb7e1b7e9f515e73cf75adf147e3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Dec 2020 15:28:22 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Dec 2020 15:28:22 -0500
The separate "cd" command before invoking psql made sense (or at least
I thought so) when it was added in commit ed1939332. But 4e3a61635
removed the supporting text that explained when to use it, making it
just confusing. So drop it.
Also switch from four-dot to three-dot filler for the unsupplied
part of the path, since at least one person has read the four-dot
filler as a typo for "../..". And fix these/those inconsistency.
Discussion: https://postgr.es/m/160837647714.673.5195186835607800484@wrigleys.postgresql.org
M doc/src/sgml/query.sgml
Doc: improve description of pgbench script weights.
commit : 1990ce186ee64f24dcff885a87cea04e3dfd8166
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Dec 2020 13:37:25 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Dec 2020 13:37:25 -0500
Point out the workaround to be used if you want to write a script
file name that includes "@". Clean up the text a little.
Fabien Coelho, additional wordsmithing by me
Discussion: https://postgr.es/m/1c4e81550d214741827a03292222db8d@G08CNEXMBPEKD06.g08.fujitsu.local
M doc/src/sgml/ref/pgbench.sgml
Avoid memcpy() with same source and destination in pgstat_recv_replslot.
commit : ed6329cfa975a13c58c44676464585cda5d317b3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Dec 2020 12:38:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Dec 2020 12:38:32 -0500
Same type of issue as in commit 53d4f5fef and earlier fixes; also
found by apparently-more-picky-than-the-buildfarm valgrind testing.
This one is an oversight in commit 986816750. Since that's new in
HEAD, no need for a back-patch.
M src/backend/postmaster/pgstat.c
Fix compiler warning introduced in 6df7a9698b
commit : 11072e86939d2f5ca0a97c709b46f29ea8b57590
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 20 Dec 2020 16:27:01 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 20 Dec 2020 16:27:01 +0300
M src/backend/commands/typecmds.c
Fixes for pg_dump.c regarding multiranges
commit : 8344d72cccae699e13884a5705b91dc1c4747c03
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 20 Dec 2020 08:14:35 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 20 Dec 2020 08:14:35 +0300
This commit fixes two wrong version number checks and one wrong check for null.
M src/bin/pg_dump/pg_dump.c
Multirange datatypes
commit : 6df7a9698bb036610c1e8c6d375e1be38cb26d5f
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 20 Dec 2020 07:20:33 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 20 Dec 2020 07:20:33 +0300
Multiranges are basically sorted arrays of non-overlapping ranges with
set-theoretic operations defined over them.
Since v14, each range type automatically gets a corresponding multirange
datatype. There are both manual and automatic mechanisms for naming multirange
types. Once can specify multirange type name using multirange_type_name
attribute in CREATE TYPE. Otherwise, a multirange type name is generated
automatically. If the range type name contains "range" then we change that to
"multirange". Otherwise, we add "_multirange" to the end.
Implementation of multiranges comes with a space-efficient internal
representation format, which evades extra paddings and duplicated storage of
oids. Altogether this format allows fetching a particular range by its index
in O(n).
Statistic gathering and selectivity estimation are implemented for multiranges.
For this purpose, stored multirange is approximated as union range without gaps.
This field will likely need improvements in the future.
Catversion is bumped.
Discussion: https://postgr.es/m/CALNJ-vSUpQ_Y%3DjXvTxt1VYFztaBSsWVXeF1y6gTYQ4bOiWDLgQ%40mail.gmail.com
Discussion: https://postgr.es/m/a0b8026459d1e6167933be2104a6174e7d40d0ab.camel%40j-davis.com#fe7218c83b08068bfffb0c5293eceda0
Author: Paul Jungwirth, revised by me
Reviewed-by: David Fetter, Corey Huinker, Jeff Davis, Pavel Stehule
Reviewed-by: Alvaro Herrera, Tom Lane, Isaac Morland, David G. Johnston
Reviewed-by: Zhihong Yu, Alexander Korotkov
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/rangetypes.sgml
M doc/src/sgml/ref/create_type.sgml
M src/backend/catalog/pg_range.c
M src/backend/catalog/pg_type.c
M src/backend/commands/typecmds.c
M src/backend/executor/functions.c
M src/backend/parser/parse_coerce.c
M src/backend/utils/adt/Makefile
A src/backend/utils/adt/multirangetypes.c
A src/backend/utils/adt/multirangetypes_selfuncs.c
M src/backend/utils/adt/pg_upgrade_support.c
M src/backend/utils/adt/pseudotypes.c
M src/backend/utils/adt/rangetypes.c
M src/backend/utils/adt/rangetypes_typanalyze.c
M src/backend/utils/cache/lsyscache.c
M src/backend/utils/cache/syscache.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/fmgr/funcapi.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/include/access/tupmacs.h
M src/include/catalog/binary_upgrade.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_aggregate.dat
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_cast.dat
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_range.dat
M src/include/catalog/pg_range.h
M src/include/catalog/pg_type.dat
M src/include/catalog/pg_type.h
M src/include/commands/typecmds.h
M src/include/utils/lsyscache.h
A src/include/utils/multirangetypes.h
M src/include/utils/rangetypes.h
M src/include/utils/selfuncs.h
M src/include/utils/syscache.h
M src/include/utils/typcache.h
M src/pl/plpgsql/src/pl_comp.c
M src/test/regress/expected/dependency.out
M src/test/regress/expected/hash_func.out
A src/test/regress/expected/multirangetypes.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/expected/plpgsql.out
M src/test/regress/expected/polymorphism.out
M src/test/regress/expected/rangefuncs.out
M src/test/regress/expected/rangetypes.out
M src/test/regress/expected/sanity_check.out
M src/test/regress/expected/type_sanity.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
M src/test/regress/sql/hash_func.sql
A src/test/regress/sql/multirangetypes.sql
M src/test/regress/sql/opr_sanity.sql
M src/test/regress/sql/polymorphism.sql
M src/test/regress/sql/rangetypes.sql
M src/test/regress/sql/type_sanity.sql
M src/tools/pgindent/typedefs.list
Remove now-useless ALWAYS_SUBDIRS entry in src/test/Makefile.
commit : 08b01d4dd982b491a2f9641804b368185b8f4c53
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 19 Dec 2020 17:58:30 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 19 Dec 2020 17:58:30 -0500
Commit 257836a75 added the "locale" subdirectory to SUBDIRS,
but neglected to remove it from ALWAYS_SUBDIRS. This oversight
had no functional effect because the filter-out function would
remove it anyway. Still, it's confusing to readers to list a
subdirectory in both places, especially because it makes the
associated comment into a partial lie.
M src/test/Makefile
Update comment atop of ReorderBufferQueueMessage().
commit : 20659fd8e5ecb3bcb3e337581c4bd7123d490ddc
author : Amit Kapila <akapila@postgresql.org>
date : Sat, 19 Dec 2020 10:08:46 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Sat, 19 Dec 2020 10:08:46 +0530
The comments atop of this function describes behaviour in case of a
transactional WAL message only, but it accepts both transactional and
non-transactional WAL messages. Update the comments to describe
behaviour in case of non-transactional WAL message as well.
Ashutosh Bapat, rephrased by Amit Kapila
Discussion: https://postgr.es/m/CAGEoWWTTzNzHOi8bj0wfAo1siGi-YEh6wqH1oaz4DrkTJ6HbTQ@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
Add a couple of missed .gitignore entries.
commit : 8afca702ecbf0730b6175afda0cecbbf0a1967e7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Dec 2020 16:24:49 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Dec 2020 16:24:49 -0500
Any subdirectory that's ignoring /output_iso/ should also
ignore /tmp_check_iso/, which could be left behind by a
failed pg_isolation_regress_check run.
I think these have been wrong for awhile, but it doesn't
seem important to fix in back branches.
M src/test/modules/brin/.gitignore
M src/test/modules/snapshot_too_old/.gitignore
Avoid memcpy() with same source and destination during relmapper init.
commit : 53d4f5fef04653fc495ae385a9c2f78c77e5c7d9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Dec 2020 15:46:44 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Dec 2020 15:46:44 -0500
A narrow reading of the C standard says that memcpy(x,x,n) is undefined,
although it's hard to envision an implementation that would really
misbehave. However, analysis tools such as valgrind might whine about
this; accordingly, let's band-aid relmapper.c to not do it.
See also 5b630501e, d3f4e8a8a, ad7b48ea0, and other similar fixes.
Apparently, none of those folk tried valgrinding initdb? This has been
like this for long enough that I'm surprised it hasn't been reported
before.
Back-patch, just in case anybody wants to use a back branch on a platform
that complains about this; we back-patched those earlier fixes too.
Discussion: https://postgr.es/m/161790.1608310142@sss.pgh.pa.us
M src/backend/utils/cache/relmapper.c
pg_stat_statements: Track time at which all statistics were last reset.
commit : 2e0fedf0362cc964c4dae42258455b6391051e70
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 18 Dec 2020 10:49:58 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 18 Dec 2020 10:49:58 +0900
This commit adds "stats_reset" column into the pg_stat_statements_info
view. This column indicates the time at which all statistics in the
pg_stat_statements view were last reset.
Per discussion, this commit also changes pg_stat_statements_info code
so that "dealloc" column is reset at the same time as "stats_reset" is reset,
i.e., whenever all pg_stat_statements entries are removed, for the sake
of consistency. Previously "dealloc" was reset only when
pg_stat_statements_reset(0, 0, 0) is called and was not reset when
pg_stat_statements_reset() with non-zero value argument discards all
entries. This was confusing.
Author: Naoki Nakamichi, Yuki Seino
Reviewed-by: Yuki Seino, Kyotaro Horiguchi, Li Japin, Fujii Masao
Discussion: https://postgr.es/m/c102cf3180d0ee73c1c5a0f7f8558322@oss.nttdata.com
M contrib/pg_stat_statements/pg_stat_statements–1.8–1.9.sql
M contrib/pg_stat_statements/pg_stat_statements.c
M doc/src/sgml/pgstatstatements.sgml
Revert "Get rid of the dedicated latch for signaling the startup process".
commit : 00f690a239932e477f25120d19b08aacdc30deb7
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 17 Dec 2020 18:06:51 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 17 Dec 2020 18:06:51 +0900
Revert ac22929a26, as well as the followup fix 113d3591b8. Because it broke
the assumption that the startup process waiting for the recovery conflict
on buffer pin should be waken up only by buffer unpin or the timeout enabled
in ResolveRecoveryConflictWithBufferPin(). It caused, for example,
SIGHUP signal handler or walreceiver process to wake that startup process
up unnecessarily frequently.
Additionally, add the comments about why that dedicated latch that
the reverted patch tried to get rid of should not be removed.
Thanks to Kyotaro Horiguchi for the discussion.
Author: Fujii Masao
Discussion: https://postgr.es/m/d8c0c608-021b-3c73-fffd-3240829ee986@oss.nttdata.com
M src/backend/access/transam/xlog.c
M src/backend/postmaster/startup.c
M src/backend/storage/ipc/standby.c
Fix varchar_2.out to match reality in cs_CZ locale.
commit : 88e014c149cc396fb218b08eda17c47d5b33e94f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Dec 2020 21:42:08 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Dec 2020 21:42:08 -0500
Seems to be a copy-and-pasteo in c06d6aa4c. Per buildfarm.
M src/test/regress/expected/varchar_2.out
doc: Fix explanation related to pg_shmem_allocations
commit : bce641a2af7199f07ec2b0a84f8fec0f26b2adc0
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 16 Dec 2020 10:36:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 16 Dec 2020 10:36:03 +0900
Offsets are shown as NULL only for anonymous allocations.
Author: Benoit Lobréau
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAPE8EZ5Lnoyqoz7aZpvQM0E8sW+hw+k6G2NULe+m4arFRrA1aA@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/catalogs.sgml
doc: clarify COPY TO for partitioning/inheritance
commit : 02c767b0fe241416d7b7ee3c107dcaf379d95381
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 15 Dec 2020 19:20:15 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 15 Dec 2020 19:20:15 -0500
It was not clear how COPY TO behaved with partitioning/inheritance
because the paragraphs were so far apart. Also reword to simplify.
Discussion: https://postgr.es/m/20201203211723.GR24052@telsasoft.com
Author: Justin Pryzby
Backpatch-through: 10
M doc/src/sgml/ref/copy.sgml
Remove obsolete btrescan() comment.
commit : 41ddc27f66fd398abe96e98f3e892a36270db9aa
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 15 Dec 2020 15:55:07 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 15 Dec 2020 15:55:07 -0800
"Ordering stuff" refered to a _bt_first() call to _bt_orderkeys().
However, the _bt_orderkeys() function was renamed to
_bt_preprocess_keys() by commit fa5c8a055a0.
_bt_preprocess_keys() is directly referenced just after the removed
comment already, which seems sufficient.
M src/backend/access/nbtree/nbtree.c
Remove useless variable stores
commit : a18422a3adf79815fecb50a029b63a8e85ca9d21
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 15 Dec 2020 19:51:16 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 15 Dec 2020 19:51:16 -0300
Mistakenly introduced in 4cbe3ac3e867; bug repaired in 148e632c0541 but
the stores were accidentally.
M src/backend/executor/nodeModifyTable.c
Error out when Gather Merge input is not sorted
commit : 6bc27698324a3eadb8a4a5a79d96e16687be6141
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 15 Dec 2020 23:13:38 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 15 Dec 2020 23:13:38 +0100
To build Gather Merge path, the input needs to be sufficiently sorted.
Ensuring this is the responsibility of the code constructing the paths,
but create_gather_merge_plan tried to handle unsorted paths by adding
an explicit Sort. In light of the recent issues related to Incremental
Sort, this is rather fragile. Some of the expressions may be volatile
or parallel unsafe, in which case we can't add the Sort here.
We could do more checks and add the Sort in at least some cases, but
it seems cleaner to just error out and make it clear this is a bug in
code constructing those paths.
Author: James Coleman
Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs%3DhC0mSksZC%3DH5M8LSchj5e5OxpTAg%40mail.gmail.com
Discussion: https://postgr.es/m/CAJGNTeNaxpXgBVcRhJX%2B2vSbq%2BF2kJqGBcvompmpvXb7pq%2BoFA%40mail.gmail.com
M src/backend/optimizer/plan/createplan.c
Clean up ancient test style
commit : c06d6aa4c35177655e0fd4acaca888a73c3f9845
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 15 Dec 2020 21:54:06 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 15 Dec 2020 21:54:06 +0100
Many older tests where written in a style like
SELECT '' AS two, i.* FROM INT2_TBL
where the first column indicated the number of expected result rows.
This has gotten increasingly out of date, as the test data fixtures
have expanded, so a lot of these were wrong and misleading. Moreover,
this style isn't really necessary, since the psql output already shows
the number of result rows.
To clean this up, remove all those extra columns.
Discussion: https://www.postgresql.org/message-id/flat/1a25312b-2686-380d-3c67-7a69094a999f%40enterprisedb.com
M src/test/regress/expected/boolean.out
M src/test/regress/expected/box.out
M src/test/regress/expected/case.out
M src/test/regress/expected/char.out
M src/test/regress/expected/char_1.out
M src/test/regress/expected/char_2.out
M src/test/regress/expected/circle.out
M src/test/regress/expected/float4-misrounded-input.out
M src/test/regress/expected/float4.out
M src/test/regress/expected/float8.out
M src/test/regress/expected/geometry.out
M src/test/regress/expected/horology.out
M src/test/regress/expected/inet.out
M src/test/regress/expected/int2.out
M src/test/regress/expected/int4.out
M src/test/regress/expected/int8.out
M src/test/regress/expected/interval.out
M src/test/regress/expected/join.out
M src/test/regress/expected/name.out
M src/test/regress/expected/numeric.out
M src/test/regress/expected/numerology.out
M src/test/regress/expected/oid.out
M src/test/regress/expected/path.out
M src/test/regress/expected/point.out
M src/test/regress/expected/polygon.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/text.out
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/expected/varchar.out
M src/test/regress/expected/varchar_1.out
M src/test/regress/expected/varchar_2.out
M src/test/regress/input/constraints.source
M src/test/regress/output/constraints.source
M src/test/regress/sql/boolean.sql
M src/test/regress/sql/box.sql
M src/test/regress/sql/case.sql
M src/test/regress/sql/char.sql
M src/test/regress/sql/circle.sql
M src/test/regress/sql/float4.sql
M src/test/regress/sql/float8.sql
M src/test/regress/sql/geometry.sql
M src/test/regress/sql/horology.sql
M src/test/regress/sql/inet.sql
M src/test/regress/sql/int2.sql
M src/test/regress/sql/int4.sql
M src/test/regress/sql/int8.sql
M src/test/regress/sql/interval.sql
M src/test/regress/sql/join.sql
M src/test/regress/sql/name.sql
M src/test/regress/sql/numeric.sql
M src/test/regress/sql/numerology.sql
M src/test/regress/sql/oid.sql
M src/test/regress/sql/path.sql
M src/test/regress/sql/point.sql
M src/test/regress/sql/polygon.sql
M src/test/regress/sql/subselect.sql
M src/test/regress/sql/text.sql
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
M src/test/regress/sql/varchar.sql
Improve hash_create()'s API for some added robustness.
commit : b3817f5f774663d55931dd4fab9c5a94a15ae7ab
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Dec 2020 11:38:53 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Dec 2020 11:38:53 -0500
Invent a new flag bit HASH_STRINGS to specify C-string hashing, which
was formerly the default; and add assertions insisting that exactly
one of the bits HASH_STRINGS, HASH_BLOBS, and HASH_FUNCTION be set.
This is in hopes of preventing recurrences of the type of oversight
fixed in commit a1b8aa1e4 (i.e., mistakenly omitting HASH_BLOBS).
Also, when HASH_STRINGS is specified, insist that the keysize be
more than 8 bytes. This is a heuristic, but it should catch
accidental use of HASH_STRINGS for integer or pointer keys.
(Nearly all existing use-cases set the keysize to NAMEDATALEN or
more, so there's little reason to think this restriction should
be problematic.)
Tweak hash_create() to insist that the HASH_ELEM flag be set, and
remove the defaults it had for keysize and entrysize. Since those
defaults were undocumented and basically useless, no callers
omitted HASH_ELEM anyway.
Also, remove memset's zeroing the HASHCTL parameter struct from
those callers that had one. This has never been really necessary,
and while it wasn't a bad coding convention it was confusing that
some callers did it and some did not. We might as well save a few
cycles by standardizing on "not".
Also improve the documentation for hash_create().
In passing, improve reinit.c's usage of a hash table by storing
the key as a binary Oid rather than a string; and, since that's
a temporary hash table, allocate it in CurrentMemoryContext for
neatness.
Discussion: https://postgr.es/m/590625.1607878171@sss.pgh.pa.us
M contrib/dblink/dblink.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/shippable.c
M contrib/tablefunc/tablefunc.c
M src/backend/access/gist/gistbuildbuffers.c
M src/backend/access/hash/hashpage.c
M src/backend/access/heap/rewriteheap.c
M src/backend/access/transam/xlogutils.c
M src/backend/catalog/pg_enum.c
M src/backend/catalog/pg_inherits.c
M src/backend/commands/async.c
M src/backend/commands/prepare.c
M src/backend/commands/sequence.c
M src/backend/executor/execPartition.c
M src/backend/nodes/extensible.c
M src/backend/optimizer/util/predtest.c
M src/backend/optimizer/util/relnode.c
M src/backend/parser/parse_oper.c
M src/backend/partitioning/partdesc.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/relation.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/backend/storage/file/reinit.c
M src/backend/storage/ipc/shmem.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/lwlock.c
M src/backend/storage/lmgr/predicate.c
M src/backend/storage/smgr/smgr.c
M src/backend/storage/sync/sync.c
M src/backend/tsearch/ts_typanalyze.c
M src/backend/utils/adt/array_typanalyze.c
M src/backend/utils/adt/jsonfuncs.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/cache/attoptcache.c
M src/backend/utils/cache/evtcache.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/cache/relfilenodemap.c
M src/backend/utils/cache/spccache.c
M src/backend/utils/cache/ts_cache.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/fmgr/dfmgr.c
M src/backend/utils/fmgr/fmgr.c
M src/backend/utils/hash/dynahash.c
M src/backend/utils/mmgr/portalmem.c
M src/backend/utils/time/combocid.c
M src/include/utils/hsearch.h
M src/pl/plperl/plperl.c
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpython/plpy_plpymodule.c
M src/pl/plpython/plpy_procedure.c
M src/pl/tcl/pltcl.c
M src/timezone/pgtz.c
Revert "Cannot use WL_SOCKET_WRITEABLE without WL_SOCKET_READABLE."
commit : a58db3aa10e62e4228aa409ba006014fa07a8ca2
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 14 Dec 2020 23:47:30 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 14 Dec 2020 23:47:30 -0800
This reverts commit 3a9e64aa0d96c8ffb6c682b082d0f72b1d373327.
Commit 4bad60e3 fixed the root of the problem that 3a9e64aa worked
around.
This enables proper pipelining of commands after terminating
replication, eliminating an undocumented limitation.
Discussion: https://postgr.es/m/3d57bc29-4459-578b-79cb-7641baf53c57%40iki.fi
Backpatch-through: 9.5
M src/backend/replication/walsender.c
Improve some code around cryptohash functions
commit : 9b584953e7bf91e342af87ef44606acd6206cd1c
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 14 Dec 2020 12:38:13 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 14 Dec 2020 12:38:13 +0900
This adjusts some code related to recent changes for cryptohash
functions:
- Add a variable in md5.h to track down the size of a computed result,
moved from pgcrypto. Note that pg_md5_hash() assumed a result of this
size already.
- Call explicit_bzero() on the hashed data when freeing the context for
fallback implementations. For MD5, particularly, it would be annoying
to leave some non-zeroed data around.
- Clean up some code related to recent changes of uuid-ossp. .gitignore
still included md5.c and a comment was incorrect.
Discussion: https://postgr.es/m/X9HXKTgrvJvYO7Oh@paquier.xyz
M contrib/pgcrypto/internal.c
M contrib/uuid-ossp/.gitignore
M contrib/uuid-ossp/uuid-ossp.c
M src/common/cryptohash.c
M src/common/md5_common.c
M src/include/common/md5.h
Add some checkpoint/restartpoint status to ps display
commit : df9274adf3096feafbbde2562311c8ab80405267
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 14 Dec 2020 11:53:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 14 Dec 2020 11:53:58 +0900
This is done for end-of-recovery and shutdown checkpoints/restartpoints
(end-of-recovery restartpoints don't exist) rather than all types of
checkpoints, in cases where it may not be possible to rely on
pg_stat_activity to get a status from the startup or checkpointer
processes.
For example, at the end of a crash recovery, this is useful to know if a
checkpoint is running in the startup process, while previously the ps
display may only show some information about "recovering" something,
that can be confusing while a checkpoint runs.
Author: Justin Pryzby
Reviewed-by: Nathan Bossart, Kirk Jamison, Fujii Masao, Michael Paquier
Discussion: https://postgr.es/m/20200818225238.GP17022@telsasoft.com
M src/backend/access/transam/xlog.c
Use HASH_BLOBS for xidhash.
commit : a1b8aa1e4eec520ed8f11c3d134a7a866358d39a
author : Noah Misch <noah@leadboat.com>
date : Sat, 12 Dec 2020 21:38:36 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 12 Dec 2020 21:38:36 -0800
This caused BufFile errors on buildfarm member sungazer, and SIGSEGV was
possible. Conditions for reaching those symptoms were more frequent on
big-endian systems.
Discussion: https://postgr.es/m/20201129214441.GA691200@rfd.leadboat.com
M src/backend/replication/logical/worker.c
M src/test/perl/PostgresNode.pm
M src/test/subscription/t/015_stream.pl
Correct behavior descriptions in comments, and correct a test name.
commit : 73aae4522b35125c4f9a506842a69336ec5d0f1b
author : Noah Misch <noah@leadboat.com>
date : Sat, 12 Dec 2020 20:12:25 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 12 Dec 2020 20:12:25 -0800
M src/backend/replication/logical/worker.c
M src/backend/replication/walreceiver.c
M src/test/perl/PostgresNode.pm
M src/test/recovery/t/020_archive_status.pl
initdb: complete getopt_long alphabetization
commit : d6abfdf84ef7c01d98d04f486fb467a4aec83804
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 12 Dec 2020 12:59:09 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 12 Dec 2020 12:59:09 -0500
Backpatch-through: 9.5
M src/bin/initdb/initdb.c
initdb: properly alphabetize getopt_long options in C string
commit : 39f3a9d2ffcaafeb8ff25e0dbe9f28a657efb48e
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 12 Dec 2020 12:51:16 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 12 Dec 2020 12:51:16 -0500
Backpatch-through: 9.5
M src/bin/initdb/initdb.c
Allow subscripting of hstore values.
commit : 0ec5f7e78231a621a1d96c4bfedc4a1849a6c6cc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 11 Dec 2020 18:58:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 11 Dec 2020 18:58:07 -0500
This is basically a finger exercise to prove that it's possible for
an extension module to add subscripting ability. Subscripted fetch
from an hstore is not different from the existing "hstore -> text"
operator. Subscripted update does seem to be a little easier to
use than the traditional update method using hstore concatenation,
but it's not a fundamentally new ability.
However, there may be some value in the code as sample code, since
it shows what's basically the minimum-complexity way to implement
subscripting when one needn't consider nested container objects.
Discussion: https://postgr.es/m/3724341.1607551174@sss.pgh.pa.us
M contrib/hstore/Makefile
M contrib/hstore/expected/hstore.out
A contrib/hstore/hstore–1.7–1.8.sql
M contrib/hstore/hstore.control
A contrib/hstore/hstore_subs.c
M contrib/hstore/sql/hstore.sql
M doc/src/sgml/hstore.sgml
M doc/src/sgml/ref/create_type.sgml
Allow ALTER TYPE to update an existing type's typsubscript value.
commit : 8c15a297452e970d68529ee2ce6bd94d84598409
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 11 Dec 2020 18:07:02 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 11 Dec 2020 18:07:02 -0500
This is essential if we'd like to allow existing extension data types
to support subscripting in future, since dropping and recreating the
type isn't a practical thing for an extension upgrade script, and
direct manipulation of pg_type isn't a great answer either.
There was some discussion about also allowing alteration of typelem,
but it's less clear whether that's a good idea or not, so for now
I forebore.
Discussion: https://postgr.es/m/3724341.1607551174@sss.pgh.pa.us
M doc/src/sgml/ref/alter_type.sgml
M src/backend/commands/typecmds.c
M src/test/regress/expected/create_type.out
M src/test/regress/sql/create_type.sql
Provide an error cursor for "can't subscript" error messages.
commit : 653aa603f501aa6e4865105a928cd13082ee7152
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 11 Dec 2020 17:54:10 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 11 Dec 2020 17:54:10 -0500
Commit c7aba7c14 didn't add this, but after more fooling with the
feature I feel that it'd be useful. To make this possible, refactor
getSubscriptingRoutines() so that the caller is responsible for
throwing any error. (In clauses.c, I just chose to make the
most conservative assumption rather than throwing an error. We don't
expect failures there anyway really, so the code space for an error
message would be a poor investment.)
M src/backend/executor/execExpr.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/parse_node.c
M src/backend/utils/cache/lsyscache.c
M src/test/regress/expected/arrays.out
pg_dump: Don't use enums for defining bit mask values
commit : d2a2808eb444986d2fe716a48e21993329142f3d
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 11 Dec 2020 19:15:30 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 11 Dec 2020 19:15:30 +0100
This usage would mean that values of the enum type are potentially not
one of the enum values. Use macros instead, like everywhere else.
Discussion: https://www.postgresql.org/message-id/14dde730-1d34-260e-fa9d-7664df2d6313@enterprisedb.com
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_utils.h
Fix compilation of uuid-ossp
commit : 525e60b7429925d09fce1b5aa0bc2f23cfe6dd18
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 10 Dec 2020 12:49:43 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 10 Dec 2020 12:49:43 +0900
This module had a dependency on pgcrypto's md5.c that got removed by
b67b57a. Instead of the code from pgcrypto, this code can just use the
new cryptohash routines for MD5 as a drop-in replacement, so let's just
do this switch. This has also the merit to simplify a bit the
compilation of uuid-ossp.
This requires --with-uuid to be reproduced, and I have used e2fs as a
way to reproduce the failure, then test this commit.
Per reports from buildfarm members longfin, florican and sifaka.
Discussion: https://postgr.es/m/X9GToVd3QmWeNvj8@paquier.xyz
M configure
M configure.ac
M contrib/uuid-ossp/Makefile
M contrib/uuid-ossp/uuid-ossp.c
Refactor MD5 implementations according to new cryptohash infrastructure
commit : b67b57a966af0c4a9547ac6fff334d3c256d9c2a
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 10 Dec 2020 11:59:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 10 Dec 2020 11:59:10 +0900
This commit heavily reorganizes the MD5 implementations that exist in
the tree in various aspects.
First, MD5 is added to the list of options available in cryptohash.c and
cryptohash_openssl.c. This means that if building with OpenSSL, EVP is
used for MD5 instead of the fallback implementation that Postgres had
for ages. With the recent refactoring work for cryptohash functions,
this change is straight-forward. If not building with OpenSSL, a
fallback implementation internal to src/common/ is used.
Second, this reduces the number of MD5 implementations present in the
tree from two to one, by moving the KAME implementation from pgcrypto to
src/common/, and by removing the implementation that existed in
src/common/. KAME was already structured with an init/update/final set
of routines by pgcrypto (see original pgcrypto/md5.h) for compatibility
with OpenSSL, so moving it to src/common/ has proved to be a
straight-forward move, requiring no actual manipulation of the internals
of each routine. Some benchmarking has not shown any performance gap
between both implementations.
Similarly to the fallback implementation used for SHA2, the fallback
implementation of MD5 is moved to src/common/md5.c with an internal
header called md5_int.h for the init, update and final routines. This
gets then consumed by cryptohash.c.
The original routines used for MD5-hashed passwords are moved to a
separate file called md5_common.c, also in src/common/, aimed at being
shared between all MD5 implementations as utility routines to keep
compatibility with any code relying on them.
Like the SHA2 changes, this commit had its round of tests on both Linux
and Windows, across all versions of OpenSSL supported on HEAD, with and
even without OpenSSL.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/20201106073434.GA4961@paquier.xyz
M contrib/pgcrypto/Makefile
M contrib/pgcrypto/internal.c
D contrib/pgcrypto/md5.c
M src/common/Makefile
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
M src/common/md5.c
A src/common/md5_common.c
R063 contrib/pgcrypto/md5.h src/common/md5_int.h
M src/include/common/cryptohash.h
M src/include/common/md5.h
M src/tools/msvc/Mkvcbuild.pm
M src/tools/pgindent/typedefs.list
Support subscripting of arbitrary types, not only arrays.
commit : c7aba7c14efdbd9fc1bb44b4cb83bedee0c6a6fc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Dec 2020 12:40:37 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Dec 2020 12:40:37 -0500
This patch generalizes the subscripting infrastructure so that any
data type can be subscripted, if it provides a handler function to
define what that means. Traditional variable-length (varlena) arrays
all use array_subscript_handler(), while the existing fixed-length
types that support subscripting use raw_array_subscript_handler().
It's expected that other types that want to use subscripting notation
will define their own handlers. (This patch provides no such new
features, though; it only lays the foundation for them.)
To do this, move the parser's semantic processing of subscripts
(including coercion to whatever data type is required) into a
method callback supplied by the handler. On the execution side,
replace the ExecEvalSubscriptingRef* layer of functions with direct
calls to callback-supplied execution routines. (Thus, essentially
no new run-time overhead should be caused by this patch. Indeed,
there is room to remove some overhead by supplying specialized
execution routines. This patch does a little bit in that line,
but more could be done.)
Additional work is required here and there to remove formerly
hard-wired assumptions about the result type, collation, etc
of a SubscriptingRef expression node; and to remove assumptions
that the subscript values must be integers.
One useful side-effect of this is that we now have a less squishy
mechanism for identifying whether a data type is a "true" array:
instead of wiring in weird rules about typlen, we can look to see
if pg_type.typsubscript == F_ARRAY_SUBSCRIPT_HANDLER. For this
to be bulletproof, we have to forbid user-defined types from using
that handler directly; but there seems no good reason for them to
do so.
This patch also removes assumptions that the number of subscripts
is limited to MAXDIM (6), or indeed has any hard-wired limit.
That limit still applies to types handled by array_subscript_handler
or raw_array_subscript_handler, but to discourage other dependencies
on this constant, I've moved it from c.h to utils/array.h.
Dmitry Dolgov, reviewed at various times by Tom Lane, Arthur Zakirov,
Peter Eisentraut, Pavel Stehule
Discussion: https://postgr.es/m/CA+q6zcVDuGBv=M0FqBYX8DPebS3F_0KQ6OVFobGJPM507_SZ_w@mail.gmail.com
Discussion: https://postgr.es/m/CA+q6zcVovR+XY4mfk-7oNk-rF91gH0PebnNfuUjuuDsyHjOcVA@mail.gmail.com
M contrib/postgres_fdw/deparse.c
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/create_type.sgml
M src/backend/catalog/aclchk.c
M src/backend/catalog/dependency.c
M src/backend/catalog/heap.c
M src/backend/catalog/pg_type.c
M src/backend/commands/typecmds.c
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/jit/llvm/llvmjit_types.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/parse_coerce.c
M src/backend/parser/parse_collate.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_node.c
M src/backend/parser/parse_target.c
M src/backend/utils/adt/Makefile
M src/backend/utils/adt/arrayfuncs.c
A src/backend/utils/adt/arraysubs.c
M src/backend/utils/adt/format_type.c
M src/backend/utils/adt/jsonfuncs.c
M src/backend/utils/cache/lsyscache.c
M src/backend/utils/cache/typcache.c
M src/bin/pg_dump/pg_dump.c
M src/include/c.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_type.dat
M src/include/catalog/pg_type.h
M src/include/executor/execExpr.h
M src/include/nodes/primnodes.h
A src/include/nodes/subscripting.h
M src/include/parser/parse_node.h
M src/include/utils/array.h
M src/include/utils/lsyscache.h
M src/include/utils/typcache.h
M src/pl/plperl/plperl.c
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpython/plpy_typeio.c
M src/test/regress/expected/arrays.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/expected/type_sanity.out
M src/test/regress/sql/arrays.sql
M src/test/regress/sql/opr_sanity.sql
M src/test/regress/sql/type_sanity.sql
Change get_constraint_index() to use pg_constraint.conindid
commit : 8b069ef5dca97cd737a5fd64c420df3cd61ec1c9
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 9 Dec 2020 15:12:05 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 9 Dec 2020 15:12:05 +0100
It was still using a scan of pg_depend instead of using the conindid
column that has been added since.
Since it is now just a catalog lookup wrapper and not related to
pg_depend, move from pg_depend.c to lsyscache.c.
Reviewed-by: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/4688d55c-9a2e-9a5a-d166-5f24fe0bf8db%40enterprisedb.com
M src/backend/catalog/pg_depend.c
M src/backend/commands/tablecmds.c
M src/backend/optimizer/util/plancat.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/cache/lsyscache.c
M src/include/catalog/dependency.h
M src/include/utils/lsyscache.h
Simplify code for getting a unicode codepoint's canonical class.
commit : 16c302f51235eaec05a1f85a11c1df04ef3a6785
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 9 Dec 2020 13:24:38 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 9 Dec 2020 13:24:38 +0900
Three places of unicode_norm.c use a similar logic for getting the
combining class from a codepoint. Commit 2991ac5 has added the function
get_canonical_class() for this purpose, but it was only called by the
backend. This commit refactors the code to use this function in all
the places where the combining class is retrieved from a given
codepoint.
Author: John Naylor
Discussion: https://postgr.es/m/CAFBsxsHUV7s7YrOm6hFz-Jq8Sc7K_yxTkfNZxsDV-DuM-k-gwg@mail.gmail.com
M src/common/unicode_norm.c
jit: Reference function pointer types via llvmjit_types.c.
commit : df99ddc70b971a991c5111a33f2f08bd7945d5c2
author : Andres Freund <andres@anarazel.de>
date : Mon, 7 Dec 2020 13:16:55 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 7 Dec 2020 13:16:55 -0800
It is error prone (see 5da871bfa1b) and verbose to manually create function
types. Add a helper that can reference a function pointer type via
llvmjit_types.c and and convert existing instances of manual creation.
Author: Andres Freund <andres@anarazel.de>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20201207212142.wz5tnbk2jsaqzogb@alap3.anarazel.de
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/jit/llvm/llvmjit_types.c
M src/include/jit/llvmjit.h
Teach contain_leaked_vars that assignment SubscriptingRefs are leaky.
commit : 62ee70331336161cb44733b6c3e0811696d962aa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 17:50:54 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 17:50:54 -0500
array_get_element and array_get_slice qualify as leakproof, since
they will silently return NULL for bogus subscripts. But
array_set_element and array_set_slice throw errors for such cases,
making them clearly not leakproof. contain_leaked_vars was evidently
written with only the former case in mind, as it gave the wrong answer
for assignment SubscriptingRefs (nee ArrayRefs).
This would be a live security bug, were it not that assignment
SubscriptingRefs can only occur in INSERT and UPDATE target lists,
while we only care about leakproofness for qual expressions; so the
wrong answer can't occur in practice. Still, that's a rather shaky
answer for a security-related question; and maybe in future somebody
will want to ask about leakproofness of a tlist. So it seems wise to
fix and even back-patch this correction.
(We would need some change here anyway for the upcoming
generic-subscripting patch, since extensions might make different
tradeoffs about whether to throw errors. Commit 558d77f20 attempted
to lay groundwork for that by asking check_functions_in_node whether a
SubscriptingRef contains leaky functions; but that idea fails now that
the implementation methods of a SubscriptingRef are not SQL-visible
functions that could be marked leakproof or not.)
Back-patch to 9.6. While 9.5 has the same issue, the code's a bit
different. It seems quite unlikely that we'd introduce any actual bug
in the short time 9.5 has left to live, so the work/risk/reward balance
isn't attractive for changing 9.5.
Discussion: https://postgr.es/m/3143742.1607368115@sss.pgh.pa.us
M src/backend/optimizer/util/clauses.c
Remove operator_precedence_warning.
commit : a676386b58bf7cd2df81baa43eb1713d3a2ec055
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 16:29:52 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 16:29:52 -0500
This GUC was always intended as a temporary solution to help with
finding 9.4-to-9.5 migration issues. Now that all pre-9.5 branches
are out of support, and 9.5 will be too before v14 is released,
it seems like it's okay to drop it. Doing so allows removal of
several hundred lines of poorly-tested code in parse_expr.c,
which have been a fertile source of bugs when people did use this.
Discussion: https://postgr.es/m/2234320.1607117945@sss.pgh.pa.us
M doc/src/sgml/config.sgml
M doc/src/sgml/syntax.sgml
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_target.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/nodes/parsenodes.h
M src/include/parser/parse_expr.h
Improve estimation of ANDs under ORs using extended statistics.
commit : 4f5760d4afa9423fe4d38e4cbec48bf5e793e7e5
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 8 Dec 2020 20:10:11 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 8 Dec 2020 20:10:11 +0000
Formerly, extended statistics only handled clauses that were
RestrictInfos. However, the restrictinfo machinery doesn't create
sub-AND RestrictInfos for AND clauses underneath OR clauses.
Therefore teach extended statistics to handle bare AND clauses,
looking for compatible RestrictInfo clauses underneath them.
Dean Rasheed, reviewed by Tomas Vondra.
Discussion: https://postgr.es/m/CAEZATCW=J65GUFm50RcPv-iASnS2mTXQbr=CfBvWRVhFLJ_fWA@mail.gmail.com
M src/backend/optimizer/path/clausesel.c
M src/backend/statistics/extended_stats.c
M src/test/regress/expected/stats_ext.out
Improve estimation of OR clauses using multiple extended statistics.
commit : 88b0898fe35a5a0325fca21bd4f3ed6dffb364c1
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 8 Dec 2020 19:39:24 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 8 Dec 2020 19:39:24 +0000
When estimating an OR clause using multiple extended statistics
objects, treat the estimates for each set of clauses for each
statistics object as independent of one another. The overlap estimates
produced for each statistics object do not apply to clauses covered by
other statistics objects.
Dean Rasheed, reviewed by Tomas Vondra.
Discussion: https://postgr.es/m/CAEZATCW=J65GUFm50RcPv-iASnS2mTXQbr=CfBvWRVhFLJ_fWA@mail.gmail.com
M src/backend/statistics/extended_stats.c
M src/test/regress/expected/stats_ext.out
Doc: clarify that CREATE TABLE discards redundant unique constraints.
commit : f2a69b352de1dffc534c4835010e736018aa94de
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 13:09:47 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 13:09:47 -0500
The SQL standard says that redundant unique constraints are disallowed,
but we long ago decided that throwing an error would be too
user-unfriendly, so we just drop redundant ones. The docs weren't very
clear about that though, as this behavior was only explained for PRIMARY
KEY vs UNIQUE, not UNIQUE vs UNIQUE.
While here, I couldn't resist doing some copy-editing and markup-fixing
on the adjacent text about INCLUDE options.
Per bug #16767 from Matthias vd Meent.
Discussion: https://postgr.es/m/16767-1714a2056ca516d0@postgresql.org
M doc/src/sgml/ref/create_table.sgml
Doc: explain that the string types can't store \0 (ASCII NUL).
commit : 9a2641911aeaa7f6523dc2a465088051d4c85901
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 12:06:19 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 12:06:19 -0500
This restriction was mentioned in connection with string literals,
but it wasn't made clear that it's a general restriction not just
a syntactic limitation in query strings.
Per unsigned documentation comment.
Discussion: https://postgr.es/m/160720552914.710.16625261471128631268@wrigleys.postgresql.org
M doc/src/sgml/datatype.sgml
Speed up rechecking if relation needs to be vacuumed or analyze in autovacuum.
commit : e2ac3fed3b1c3281281eb530c220634030cd8084
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 8 Dec 2020 23:59:39 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 8 Dec 2020 23:59:39 +0900
After autovacuum collects the relations to vacuum or analyze, it rechecks
whether each relation still needs to be vacuumed or analyzed before actually
doing that. Previously this recheck could be a significant overhead
especially when there were a very large number of relations. This was
because each recheck forced the statistics to be refreshed, and the refresh
of the statistics for a very large number of relations could cause heavy
overhead. There was the report that this issue caused autovacuum workers
to have gotten “stuck” in a tight loop of table_recheck_autovac() that
rechecks whether a relation needs to be vacuumed or analyzed.
This commit speeds up the recheck by making autovacuum worker reuse
the previously-read statistics for the recheck if possible. Then if that
"stale" statistics says that a relation still needs to be vacuumed or analyzed,
autovacuum refreshes the statistics and does the recheck again.
The benchmark shows that the more relations exist and autovacuum workers
are running concurrently, the more this change reduces the autovacuum
execution time. For example, when there are 20,000 tables and 10 autovacuum
workers are running, the benchmark showed that the change improved
the performance of autovacuum more than three times. On the other hand,
even when there are only 1000 tables and only a single autovacuum worker
is running, the benchmark didn't show any big performance regression by
the change.
Firstly POC patch was proposed by Jim Nasby. As the result of discussion,
we used Tatsuhito Kasahara's version of the patch using the approach
suggested by Tom Lane.
Reported-by: Jim Nasby
Author: Tatsuhito Kasahara
Reviewed-by: Masahiko Sawada, Fujii Masao
Discussion: https://postgr.es/m/3FC6C2F2-8A47-44C0-B997-28830B5716D0@amazon.com
M src/backend/postmaster/autovacuum.c
Bump catversion for pg_stat_wal changes.
commit : 4e43ee88c28b725bb63e27609c1a717138fc7f39
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 8 Dec 2020 21:05:27 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 8 Dec 2020 21:05:27 +0900
Oversight in 01469241b2.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/20201207185614.zzf63vggm5r4sozg@alap3.anarazel.de
M src/include/catalog/catversion.h
pgcrypto: Detect errors with EVP calls from OpenSSL
commit : 28d1601ad988790c3c53d7ffd24ef6d2366d4457
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Dec 2020 15:22:12 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Dec 2020 15:22:12 +0900
The following routines are called within pgcrypto when handling digests
but there were no checks for failures:
- EVP_MD_CTX_size (can fail with -1 as of 3.0.0)
- EVP_MD_CTX_block_size (can fail with -1 as of 3.0.0)
- EVP_DigestInit_ex
- EVP_DigestUpdate
- EVP_DigestFinal_ex
A set of elog(ERROR) is added by this commit to detect such failures,
that should never happen except in the event of a processing failure
internal to OpenSSL.
Note that it would be possible to use ERR_reason_error_string() to get
more context about such errors, but these refer mainly to the internals
of OpenSSL, so it is not really obvious how useful that would be. This
is left out for simplicity.
Per report from Coverity. Thanks to Tom Lane for the discussion.
Backpatch-through: 9.5
M contrib/pgcrypto/openssl.c
jit: Correct parameter type for generated expression evaluation functions.
commit : 5da871bfa1ba41768ecd7786293d9b81dcf1b667
author : Andres Freund <andres@anarazel.de>
date : Mon, 7 Dec 2020 18:21:06 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 7 Dec 2020 18:21:06 -0800
clang only uses the 'i1' type for scalar booleans, not for pointers to
booleans (as the pointer might be pointing into a larger memory
allocation). Therefore a pointer-to-bool needs to the "storage" boolean.
There's no known case of wrong code generation due to this, but it seems quite
possible that it could cause problems (see e.g. 72559438f92).
Author: Andres Freund
Discussion: https://postgr.es/m/20201207212142.wz5tnbk2jsaqzogb@alap3.anarazel.de
Backpatch: 11-, where jit support was added
M src/backend/jit/llvm/llvmjit_expr.c
jit: configure: Explicitly reference 'native' component.
commit : 9543f0861b1d9b566be88edae21f24fb1377f45c
author : Andres Freund <andres@anarazel.de>
date : Mon, 7 Dec 2020 18:12:23 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 7 Dec 2020 18:12:23 -0800
Until recently 'native' was implicitly included via 'orcjit', but a change
included in LLVM 11 (not yet released) removed a number of such indirect
component references.
Reported-By: Fabien COELHO <coelho@cri.ensmp.fr>
Reported-By: Andres Freund <andres@anarazel.de>
Reported-By: Thomas Munro <thomas.munro@gmail.com>
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20201201064949.mex6kvi2kygby3ni@alap3.anarazel.de
Backpatch: 11-, where jit support was added
M config/llvm.m4
M configure
Avoid using tuple from syscache for update of pg_database.datfrozenxid
commit : 947789f1f5fb61daf663f26325cbe7cad8197d58
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Dec 2020 12:13:19 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Dec 2020 12:13:19 +0900
pg_database.datfrozenxid gets updated using an in-place update at the
end of vacuum or autovacuum. Since 96cdeae, as pg_database has a toast
relation, it is possible for a pg_database tuple to have toast values
if there is a large set of ACLs in place. In such a case, the in-place
update would fail because of the flattening of the toast values done for
the catcache entry fetched. Instead of using a copy from the catcache,
this changes the logic to fetch the copy of the tuple by directly
scanning pg_database.
Per the lack of complaints on the matter, no backpatch is done. Note
that before 96cdeae, attempting to insert such a tuple to pg_database
would cause a "row is too big" error, so the end-of-vacuum problem was
not reachable.
Author: Ashwin Agrawal, Junfeng Yang
Discussion: https://postgr.es/m/DM5PR0501MB38800D9E4605BCA72DD35557CCE10@DM5PR0501MB3880.namprd05.prod.outlook.com
M src/backend/access/heap/heapam.c
M src/backend/commands/vacuum.c
Add a couple of regression test cases related to array subscripting.
commit : 0a665bbc43c5a678331fb1b1f44274500eba6563
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Dec 2020 11:10:21 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Dec 2020 11:10:21 -0500
Exercise some error cases that were never reached in the existing
regression tests. This is partly for code-coverage reasons, and
partly to memorialize the current behavior in advance of planned
changes for generic subscripting.
Also, I noticed that type_sanity's check to verify that all standard
types have array types was never extended when we added arrays for
all system catalog rowtypes (f7f70d5e2), nor when we added arrays
over domain types (c12d570fa). So do that. Also, since the query's
expected output isn't empty, it seems like a good idea to add an
ORDER BY to make sure the result stays stable.
M src/test/regress/expected/arrays.out
M src/test/regress/expected/rowtypes.out
M src/test/regress/expected/type_sanity.out
M src/test/regress/sql/arrays.sql
M src/test/regress/sql/rowtypes.sql
M src/test/regress/sql/type_sanity.sql
Fix more race conditions in the newly-added pg_rewind test.
commit : 6ba581cf1104261f506377823a25795f0849ce62
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 7 Dec 2020 14:44:34 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 7 Dec 2020 14:44:34 +0200
pg_rewind looks at the control file to check what timeline a server is on.
But promotion doesn't immediately write a checkpoint, it merely writes
an end-of-recovery WAL record. If pg_rewind runs immediately after
promotion, before the checkpoint has completed, it will think think that
the server is still on the earlier timeline. We ran into this issue a long
time ago already, see commit 484a848a73f.
It's a bit bogus that pg_rewind doesn't determine the timeline correctly
until the end-of-recovery checkpoint has completed. We probably should
fix that. But for now work around it by waiting for the checkpoint
to complete before running pg_rewind, like we did in commit 484a848a73f.
In the passing, tidy up the new test a little bit. Rerder the INSERTs so
that the comments make more sense, remove a spurious CHECKPOINT call after
pg_rewind has already run, and add --debug option, so that if this fails
again, we'll have more data.
Per buildfarm failure at https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=rorqual&dt=2020-12-06%2018%3A32%3A19&stg=pg_rewind-check.
Backpatch to all supported versions.
Discussion: https://www.postgresql.org/message-id/1713707e-e318-761c-d287-5b6a4aa807e8@iki.fi
M src/bin/pg_rewind/t/008_min_recovery_point.pl
pg_dump: Reorganize dumpBaseType()
commit : 04732962462ba99cf8f8bcf6ac83932867cc96a8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Dec 2020 22:37:40 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Dec 2020 22:37:40 -0500
Along the same lines as ed2c7f65b and daa9fe8a5, reduce code duplication
by having just one copy of the parts of the query that are the same
across all server versions; and make the conditionals control the
smallest possible amount of code. This is in preparation for adding
another dumpable field to pg_type.
M src/bin/pg_dump/pg_dump.c
Fix fd leak in pg_verifybackup
commit : 51c388987734cb318c8fa875f382ca75a34d0010
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Dec 2020 09:29:32 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Dec 2020 09:29:32 +0900
An error code path newly-introduced by 87ae969 forgot to close a file
descriptor when verifying a file's checksum.
Per report from Coverity, via Tom Lane.
M src/bin/pg_verifybackup/pg_verifybackup.c
Fix missed step in removal of useless RESULT RTEs in the planner.
commit : e98c900993e89ad9278cdfbf0ba5495381a1faac
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Dec 2020 16:16:13 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Dec 2020 16:16:13 -0500
Commit 4be058fe9 forgot that the append_rel_list would already be
populated at the time we remove useless result RTEs, and it might contain
PlaceHolderVars that need to be adjusted like the ones in the main parse
tree. This could lead to "no relation entry for relid N" failures later
on, when the planner tries to do something with an unadjusted PHV.
Per report from Tom Ellis. Back-patch to v12 where the bug came in.
Discussion: https://postgr.es/m/20201205173056.GF30712@cloudinit-builder
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Fix race conditions in newly-added test.
commit : 36a4ac20fcf31361bd42b63b1b3390b28827a69e
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 4 Dec 2020 18:20:18 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 4 Dec 2020 18:20:18 +0200
Buildfarm has been failing sporadically on the new test. I was able to
reproduce this by adding a random 0-10 s delay in the walreceiver, just
before it connects to the primary. There's a race condition where node_3
is promoted before it has fully caught up with node_1, leading to diverged
timelines. When node_1 is later reconfigured as standby following node_3,
it fails to catch up:
LOG: primary server contains no more WAL on requested timeline 1
LOG: new timeline 2 forked off current database system timeline 1 before current recovery point 0/30000A0
That's the situation where you'd need to use pg_rewind, but in this case
it happens already when we are just setting up the actual pg_rewind
scenario we want to test, so change the test so that it waits until
node_3 is connected and fully caught up before promoting it, so that you
get a clean, controlled failover.
Also rewrite some of the comments, for clarity. The existing comments
detailed what each step in the test did, but didn't give a good overview
of the situation the steps were trying to create.
For reasons I don't understand, the test setup had to be written slightly
differently in 9.6 and 9.5 than in later versions. The 9.5/9.6 version
needed node 1 to be reinitialized from backup, whereas in later versions
it could be shut down and reconfigured to be a standby. But even 9.5 should
support "clean switchover", where primary makes sure that pending WAL is
replicated to standby on shutdown. It would be nice to figure out what's
going on there, but that's independent of pg_rewind and the scenario that
this test tests.
Discussion: https://www.postgresql.org/message-id/b0a3b95b-82d2-6089-6892-40570f8c5e60%40iki.fi
M src/bin/pg_rewind/t/008_min_recovery_point.pl
Convert elog(LOG) calls to ereport() where appropriate
commit : eb93f3a0b633ad6afb0f37391b87f460c4b0663b
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 4 Dec 2020 14:25:23 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 4 Dec 2020 14:25:23 +0100
User-visible log messages should go through ereport(), so they are
subject to translation. Many remaining elog(LOG) calls are really
debugging calls.
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://www.postgresql.org/message-id/flat/92d6f545-5102-65d8-3c87-489f71ea0a37%40enterprisedb.com
M src/backend/access/gist/gist.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/transam/xlog.c
M src/backend/libpq/auth.c
M src/backend/libpq/hba.c
M src/backend/libpq/pqcomm.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/replication/logical/origin.c
M src/backend/storage/file/fd.c
M src/backend/utils/misc/guc.c
Remove unnecessary grammar symbols
commit : a6964bc1bb0793e20636ccb573cd2a5ad3ef7667
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 4 Dec 2020 11:09:29 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 4 Dec 2020 11:09:29 +0100
Instead of publication_name_list, we can use name_list. We already
refer to publications everywhere else by the 'name' or 'name_list'
symbols, so this only improves consistency.
Reviewed-by: https://www.postgresql.org/message-id/flat/3e3ccddb-41bd-ecd8-29fe-195e34d9886f%40enterprisedb.com
Discussion: Tom Lane <tgl@sss.pgh.pa.us>
M src/backend/parser/gram.y
Remove incorrect assertion in reorderbuffer.c.
commit : 8ae4ef4fb0e0331f02c4615182600546c8e5c4d4
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 4 Dec 2020 13:54:50 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 4 Dec 2020 13:54:50 +0530
We start recording changes in ReorderBufferTXN even before we reach
SNAPBUILD_CONSISTENT state so that if the commit is encountered after
reaching that we should be able to send the changes of the entire transaction.
Now, while recording changes if the reorder buffer memory has exceeded
logical_decoding_work_mem then we can start streaming if it is allowed and
we haven't yet streamed that data. However, we must not allow streaming to
start unless the snapshot has reached SNAPBUILD_CONSISTENT state.
In passing, improve the comments atop ReorderBufferResetTXN to mention the
case when we need to continue streaming after getting an error.
Author: Amit Kapila
Reviewed-by: Dilip Kumar
Discussion: https://postgr.es/m/CAA4eK1KoOH0byboyYY40NBcC7Fe812trwTa+WY3jQF7WQWZbQg@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
Rename cryptohashes.c to cryptohashfuncs.c
commit : bd94a9c04e04bb3b626e88981a50fcca2bd99d60
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 4 Dec 2020 12:58:44 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 4 Dec 2020 12:58:44 +0900
87ae969 has created two new files called cryptohash{_openssl}.c in
src/common/, whose names overlap with the existing backend file called
cryptohashes.c dedicated to the SQL wrappers for SHA2 and MD5. This
file is renamed to cryptohashfuncs.c to be more consistent with the
surroundings and reduce the confusion with the new cryptohash interface
of src/common/.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/X8hHhaQgbMbW+aGU@paquier.xyz
M src/backend/utils/adt/Makefile
R098 src/backend/utils/adt/cryptohashes.c src/backend/utils/adt/cryptohashfuncs.c
Change SHA2 implementation based on OpenSSL to use EVP digest routines
commit : 4f48a6fbe2b28d8281dbbfa2d334fa2ed8472734
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 4 Dec 2020 10:49:23 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 4 Dec 2020 10:49:23 +0900
The use of low-level hash routines is not recommended by upstream
OpenSSL since 2000, and pgcrypto already switched to EVP as of 5ff4a67.
This takes advantage of the refactoring done in 87ae969 that has
introduced the allocation and free routines for cryptographic hashes.
Since 1.1.0, OpenSSL does not publish the contents of the cryptohash
contexts, forcing any consumers to rely on OpenSSL for all allocations.
Hence, the resource owner callback mechanism gains a new set of routines
to track and free cryptohash contexts when using OpenSSL, preventing any
risks of leaks in the backend. Nothing is needed in the frontend thanks
to the refactoring of 87ae969, and the resowner knowledge is isolated
into cryptohash_openssl.c.
Note that this also fixes a failure with SCRAM authentication when using
FIPS in OpenSSL, but as there have been few complaints about this
problem and as this causes an ABI breakage, no backpatch is done.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson, Heikki Linnakangas
Discussion: https://postgr.es/m/20200924025314.GE7405@paquier.xyz
Discussion: https://postgr.es/m/20180911030250.GA27115@paquier.xyz
M src/backend/replication/basebackup.c
M src/backend/utils/resowner/resowner.c
M src/common/cryptohash_openssl.c
M src/include/utils/resowner_private.h
M src/tools/pgindent/typedefs.list
doc: remove unnecessary blank before command option text
commit : 3f8971d92e767acf6e3d6e27c4cab7bfd88b71f4
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 3 Dec 2020 11:33:24 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 3 Dec 2020 11:33:24 -0500
Backpatch-through: 11
M doc/src/sgml/ref/pg_checksums.sgml
M doc/src/sgml/ref/pg_controldata.sgml
M doc/src/sgml/ref/pg_resetwal.sgml
docs: list single-letter options first in command-line summary
commit : a659e789b743f1a4b977a1ffc7402ab618a9f68c
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 3 Dec 2020 10:28:58 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 3 Dec 2020 10:28:58 -0500
In a few places, the long-version options were listed before the
single-letter ones in the command summary of a few commands. This
didn't match other commands, and didn't match the option ordering later
in the same reference page.
Backpatch-through: 9.5
M doc/src/sgml/ref/pg_controldata.sgml
M doc/src/sgml/ref/pg_resetwal.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
Fix pg_rewind bugs when rewinding a standby server.
commit : 2b4f3130382fe2f8705863e4d38589d4d69cd695
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 3 Dec 2020 15:57:48 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 3 Dec 2020 15:57:48 +0200
If the target is a standby server, its WAL doesn't end at the last
checkpoint record, but at minRecoveryPoint. We must scan all the
WAL from the last common checkpoint all the way up to minRecoveryPoint
for modified pages, and also consider that portion when determining
whether the server needs rewinding.
Backpatch to all supported versions.
Author: Ian Barwick and me
Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
A src/bin/pg_rewind/t/008_min_recovery_point.pl
Small code simplifications
commit : 6114040711affa2b0bcf47fa2791187daf8455fb
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Dec 2020 11:35:20 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 3 Dec 2020 11:35:20 +0100
strVal() can be used in a couple of places instead of coding the same
thing by hand.
M src/backend/foreign/foreign.c
M src/backend/parser/gram.y
Improve estimation of OR clauses using extended statistics.
commit : 25a9e54d2db31b8031c2d8166114b187e8347098
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 3 Dec 2020 10:03:49 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 3 Dec 2020 10:03:49 +0000
Formerly we only applied extended statistics to an OR clause as part
of the clauselist_selectivity() code path for an OR clause appearing
in an implicitly-ANDed list of clauses. This meant that it could only
use extended statistics if all sub-clauses of the OR clause were
covered by a single extended statistics object.
Instead, teach clause_selectivity() how to apply extended statistics
to an OR clause by handling its ORed list of sub-clauses in a similar
manner to an implicitly-ANDed list of sub-clauses, but with different
combination rules. This allows one or more extended statistics objects
to be used to estimate all or part of the list of sub-clauses. Any
remaining sub-clauses are then treated as if they are independent.
Additionally, to avoid double-application of extended statistics, this
introduces "extended" versions of clause_selectivity() and
clauselist_selectivity(), which include an option to ignore extended
statistics. This replaces the old clauselist_selectivity_simple()
function which failed to completely ignore extended statistics when
called from the extended statistics code.
A known limitation of the current infrastructure is that an AND clause
under an OR clause is not treated as compatible with extended
statistics (because we don't build RestrictInfos for such sub-AND
clauses). Thus, for example, "(a=1 AND b=1) OR (a=2 AND b=2)" will
currently be treated as two independent AND clauses (each of which may
be estimated using extended statistics), but extended statistics will
not currently be used to account for any possible overlap between
those clauses. Improving that is left as a task for the future.
Original patch by Tomas Vondra, with additional improvements by me.
Discussion: https://postgr.es/m/20200113230008.g67iyk4cs3xbnjju@development
M src/backend/optimizer/path/clausesel.c
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/statistics/mcv.c
M src/include/optimizer/optimizer.h
M src/include/statistics/extended_stats_internal.h
M src/include/statistics/statistics.h
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Refactor CLUSTER and REINDEX grammar to use DefElem for option lists
commit : b5913f6120792465f4394b93c15c2e2ac0c08376
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 3 Dec 2020 10:13:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 3 Dec 2020 10:13:21 +0900
This changes CLUSTER and REINDEX so as a parenthesized grammar becomes
possible for options, while unifying the grammar parsing rules for
option lists with the existing ones.
This is a follow-up of the work done in 873ea9e for VACUUM, ANALYZE and
EXPLAIN. This benefits REINDEX for a potential backend-side filtering
for collatable-sensitive indexes and TABLESPACE, while CLUSTER would
benefit from the latter.
Author: Alexey Kondratov, Justin Pryzby
Discussion: https://postgr.es/m/8a8f5f73-00d3-55f8-7583-1375ca8f6a91@postgrespro.ru
M doc/src/sgml/ref/cluster.sgml
M doc/src/sgml/ref/reindex.sgml
M src/backend/commands/cluster.c
M src/backend/commands/indexcmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/parser/gram.y
M src/backend/tcop/utility.c
M src/bin/psql/tab-complete.c
M src/include/catalog/index.h
M src/include/commands/cluster.h
M src/include/commands/defrem.h
M src/include/nodes/parsenodes.h
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Add GSS information to connection authorized log message
commit : dc11f31a1a891f8aa8890644e837556bcc5a75e7
author : Stephen Frost <sfrost@snowman.net>
date : Wed, 2 Dec 2020 14:41:53 -0500
committer: Stephen Frost <sfrost@snowman.net>
date : Wed, 2 Dec 2020 14:41:53 -0500
GSS information (if used) such as if the connection was authorized using
GSS or if it was encrypted using GSS, and perhaps most importantly, what
the GSS principal used for the authentication was, is extremely useful
but wasn't being included in the connection authorized log message.
Therefore, add to the connection authorized log message that
information, in a similar manner to how we log SSL information when SSL
is used for a connection.
Author: Vignesh C
Reviewed-by: Bharath Rupireddy
Discussion: https://www.postgresql.org/message-id/CALDaNm2N1385_Ltoo%3DS7VGT-ESu_bRQa-sC1wg6ikrM2L2Z49w%40mail.gmail.com
M src/backend/utils/init/postinit.c
M src/test/kerberos/t/001_auth.pl
Track total number of WAL records, FPIs and bytes generated in the cluster.
commit : 01469241b2ff6f17fc0b3f20f01559145ffab29e
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 2 Dec 2020 13:00:15 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 2 Dec 2020 13:00:15 +0900
Commit 6b466bf5f2 allowed pg_stat_statements to track the number of
WAL records, full page images and bytes that each statement generated.
Similarly this commit allows us to track the cluster-wide WAL statistics
counters.
New columns wal_records, wal_fpi and wal_bytes are added into the
pg_stat_wal view, and reports the total number of WAL records,
full page images and bytes generated in the , respectively.
Author: Masahiro Ikeda
Reviewed-by: Amit Kapila, Movead Li, Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/35ef960128b90bfae3b3fdf60a3a860f@oss.nttdata.com
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
Fix compilation warnings in cryptohash_openssl.c
commit : 91624c2ff8809145880383b0fa84be0ee98f55b5
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 2 Dec 2020 12:31:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 2 Dec 2020 12:31:10 +0900
These showed up with -O2. Oversight in 87ae969.
Author: Fujii Masao
Discussion: https://postgr.es/m/cee3df00-566a-400c-1252-67c3701f918a@oss.nttdata.com
M src/common/cryptohash_openssl.c
Allow restore_command parameter to be changed with reload.
commit : 942305a36365433eff3c1937945758f2dbf1662b
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 2 Dec 2020 11:00:15 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 2 Dec 2020 11:00:15 +0900
This commit changes restore_command from PGC_POSTMASTER to PGC_SIGHUP.
As the side effect of this commit, restore_command can be reset to
empty during archive recovery. In this setting, archive recovery
tries to replay only WAL files available in pg_wal directory. This is
the same behavior as when the command that always fails is specified
in restore_command.
Note that restore_command still must be specified (not empty) when
starting archive recovery, even after applying this commit. This is
necessary as the safeguard to prevent users from forgetting to
specify restore_command and starting archive recovery.
Thanks to Peter Eisentraut, Michael Paquier, Andres Freund,
Robert Haas and Anastasia Lubennikova for discussion.
Author: Sergei Kornilov
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/2317771549527294@sas2-985f744271ca.qloud-c.yandex.net
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
Move SHA2 routines to a new generic API layer for crypto hashes
commit : 87ae9691d25379785f8c0f81b06a14818cfd8c56
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 2 Dec 2020 10:37:20 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 2 Dec 2020 10:37:20 +0900
Two new routines to allocate a hash context and to free it are created,
as these become necessary for the goal behind this refactoring: switch
the all cryptohash implementations for OpenSSL to use EVP (for FIPS and
also because upstream does not recommend the use of low-level cryptohash
functions for 20 years). Note that OpenSSL hides the internals of
cryptohash contexts since 1.1.0, so it is necessary to leave the
allocation to OpenSSL itself, explaining the need for those two new
routines. This part is going to require more work to properly track
hash contexts with resource owners, but this not introduced here.
Still, this refactoring makes the move possible.
This reduces the number of routines for all SHA2 implementations from
twelve (SHA{224,256,386,512} with init, update and final calls) to five
(create, free, init, update and final calls) by incorporating the hash
type directly into the hash context data.
The new cryptohash routines are moved to a new file, called cryptohash.c
for the fallback implementations, with SHA2 specifics becoming a part
internal to src/common/. OpenSSL specifics are part of
cryptohash_openssl.c. This infrastructure is usable for more hash
types, like MD5 or HMAC.
Any code paths using the internal SHA2 routines are adapted to report
correctly errors, which are most of the changes of this commit. The
zones mostly impacted are checksum manifests, libpq and SCRAM.
Note that e21cbb4 was a first attempt to switch SHA2 to EVP, but it
lacked the refactoring needed for libpq, as done here.
This patch has been tested on Linux and Windows, with and without
OpenSSL, and down to 1.0.1, the oldest version supported on HEAD.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/20200924025314.GE7405@paquier.xyz
M contrib/pgcrypto/internal-sha2.c
M src/backend/libpq/auth-scram.c
M src/backend/replication/backup_manifest.c
M src/backend/replication/basebackup.c
M src/backend/utils/adt/cryptohashes.c
M src/bin/pg_verifybackup/parse_manifest.c
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/common/Makefile
M src/common/checksum_helper.c
A src/common/cryptohash.c
A src/common/cryptohash_openssl.c
M src/common/scram-common.c
M src/common/sha2.c
A src/common/sha2_int.h
D src/common/sha2_openssl.c
M src/include/common/checksum_helper.h
A src/include/common/cryptohash.h
M src/include/common/scram-common.h
M src/include/common/sha2.h
M src/include/replication/backup_manifest.h
M src/interfaces/libpq/fe-auth-scram.c
M src/tools/msvc/Mkvcbuild.pm
M src/tools/pgindent/typedefs.list
pg_checksums: data_checksum_version is unsigned so use %u not %d
commit : 888671a8cda5896d36d3ec523e201ab6a11e1855
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 1 Dec 2020 20:27:06 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 1 Dec 2020 20:27:06 -0500
While the previous behavior didn't generate a warning, we might as well
use an accurate *printf specification.
Backpatch-through: 12
M src/bin/pg_checksums/pg_checksums.c
Ensure that expandTableLikeClause() re-examines the same table.
commit : f7f83a55bf6051818a0e4387d718867ecfa8561b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Dec 2020 14:02:27 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Dec 2020 14:02:27 -0500
As it stood, expandTableLikeClause() re-did the same relation_openrv
call that transformTableLikeClause() had done. However there are
scenarios where this would not find the same table as expected.
We hold lock on the LIKE source table, so it can't be renamed or
dropped, but another table could appear before it in the search path.
This explains the odd behavior reported in bug #16758 when cloning a
table as a temp table of the same name. This case worked as expected
before commit 502898192 introduced the need to open the source table
twice, so we should fix it.
To make really sure we get the same table, let's re-open it by OID not
name. That requires adding an OID field to struct TableLikeClause,
which is a little nervous-making from an ABI standpoint, but as long
as it's at the end I don't think there's any serious risk.
Per bug #16758 from Marc Boeren. Like the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/16758-840e84a6cfab276d@postgresql.org
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_utilcmd.c
M src/include/nodes/parsenodes.h
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Avoid memcpy() with a NULL source pointer and count == 0
commit : 677f74e5bb8360122ebf2f473d7169ed65ce4dba
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 1 Dec 2020 11:46:56 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 1 Dec 2020 11:46:56 -0300
When memcpy() is called on a pointer, the compiler is entitled to assume
that the pointer is not null, which can lead to optimizing nearby code
in potentially undesirable ways. We still want such optimizations
(gcc's -fdelete-null-pointer-checks) in cases where they're valid.
Related: commit 13bba02271dc.
Backpatch to pg11, where this particular instance appeared.
Reported-by: Ranier Vilela <ranier.vf@gmail.com>
Reported-by: Zhihong Yu <zyu@yugabyte.com>
Discussion: https://postgr.es/m/CAEudQApUndmQkr5fLrCKXQ7+ib44i7S+Kk93pyVThS85PnG3bQ@mail.gmail.com
Discussion: https://postgr.es/m/CALNJ-vSdhwSM5f4tnNn1cdLHvXMVe_S+V3nR5GwNrmCPNB2VtQ@mail.gmail.com
M src/backend/commands/indexcmds.c
docs: ulink all references to RFC's
commit : 415dc2009683904f337a1837b6b4eb7f31c4dc55
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 1 Dec 2020 14:36:30 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 1 Dec 2020 14:36:30 +0200
Make sure that the first mentions of RFC's are ulinked to their ietf.org
entry, and subsequent ones are marked as acronyms. This makes references
to RFC's consistent across the documentation.
Author: Daniel Gustafsson
Discussion: https://www.postgresql.org/message-id/2C697878-4D01-4F06-8312-2FEDE931E973%40yesql.se
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/charset.sgml
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/json.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/textsearch.sgml
M doc/src/sgml/uuid-ossp.sgml
doc: Add additional index entries for progress reporting views.
commit : 0a4db67b5ed05c4013ea968930af36853f088404
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 1 Dec 2020 17:19:23 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 1 Dec 2020 17:19:23 +0900
In the docs, the index entries for progress reporting views link to
the "Viewing Statistics" section, but previously they did not link to
the dedicated section (e.g., "ANALYZE Progress Reporting") for
each view. This was inconvenient when finding the section describing
the detailed information about each view, from the index.
This commit adds additional index entries linking to those dedicated
sections.
Author: Fujii Masao
Reviewed-by: Shinya Kato
Discussion: https://postgr.es/m/e49c2768-65d2-188a-5424-270fa29ccc84@oss.nttdata.com
M doc/src/sgml/monitoring.sgml
doc: Remove more notes about compatibilities with past versions
commit : 8a17f44c1e7a9f3d2a9da97dc3eba4184a2a453c
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 1 Dec 2020 16:32:26 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 1 Dec 2020 16:32:26 +0900
This is a follow-up of the work done in fa42c2e, that did not include
all the fixes previously agreed on. The contents removed here can be
confusing to the reader as they refer to rather old server versions.
Author: Stephen Frost, Tom Lane, Heikki Linnakangas, Yaroslav Schekin
Discussion: https://postgr.es/m/CAB8KJ=jYHgnxLLZSNJz7gBTck4TxomngCmGkw3nEMSNF0yL6wA@mail.gmail.com
Discussion: https://postgr.es/m/1599765595731-0.post@n3.nabble.com
M doc/src/sgml/func.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/ref/select.sgml
Use truncate(2) where appropriate.
commit : 57faaf376e1961fa48866c6e5d6926463c6671b1
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 1 Dec 2020 15:34:57 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 1 Dec 2020 15:34:57 +1300
When truncating files by name, use truncate(2). Windows hasn't got it,
so keep our previous coding based on ftruncate(2) as a fallback.
Discussion: https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org
M src/backend/storage/file/fd.c
M src/backend/storage/smgr/md.c
M src/include/storage/fd.h
Free disk space for dropped relations on commit.
commit : 9f35f943732eaf9121a107f54ea043925e9a5d7b
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 1 Dec 2020 13:21:03 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 1 Dec 2020 13:21:03 +1300
When committing a transaction that dropped a relation, we previously
truncated only the first segment file to free up disk space (the one
that won't be unlinked until the next checkpoint).
Truncate higher numbered segments too, even though we unlink them on
commit. This frees the disk space immediately, even if other backends
have open file descriptors and might take a long time to get around to
handling shared invalidation events and closing them. Also extend the
same behavior to the first segment, in recovery.
Back-patch to all supported releases.
Bug: #16663
Reported-by: Denis Patron <denis.patron@previnet.it>
Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com>
Reviewed-by: Neil Chen <carpenter.nail.cz@gmail.com>
Reviewed-by: David Zhang <david.zhang@highgo.ca>
Discussion: https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org
M src/backend/storage/smgr/md.c
Fix missing outfuncs.c support for IncrementalSortPath.
commit : 8286223f3d820c39f2d5f14222f7ccde53bdf502
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 16:32:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 16:32:56 -0500
For debugging purposes, Path nodes are supposed to have outfuncs
support, but this was overlooked in the original incremental sort patch.
While at it, clean up a couple other minor oversights, as well as
bizarre choice of return type for create_incremental_sort_path().
(All the existing callers just cast it to "Path *" immediately, so
they don't care, but some future caller might care.)
outfuncs.c fix by Zhijie Hou, the rest by me
Discussion: https://postgr.es/m/324c4d81d8134117972a5b1f6cdf9560@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/README
M src/backend/optimizer/util/pathnode.c
M src/include/nodes/pathnodes.h
M src/include/optimizer/pathnode.h
Document concurrent indexes waiting on each other
commit : 58ebe967f8a1e34a824d6a0a35728027272041c6
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 30 Nov 2020 18:24:55 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 30 Nov 2020 18:24:55 -0300
Because regular CREATE INDEX commands are independent, and there's no
logical data dependency, it's not immediately obvious that transactions
held by concurrent index builds on one table will block the second phase
of concurrent index creation on an unrelated table, so document this
caveat.
Backpatch this all the way back. In branch master, mention that only
some indexes are involved.
Author: James Coleman <jtc331@gmail.com>
Reviewed-by: David Johnston <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/CAAaqYe994=PUrn8CJZ4UEo_S-FfRr_3ogERyhtdgHAb2WG_Ufg@mail.gmail.com
M doc/src/sgml/ref/create_index.sgml
Remove configure-time probe for DocBook DTD.
commit : 4823c4f6ac770ebd49b63b0ce64ecfe82987af57
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 15:24:13 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 15:24:13 -0500
Checking for DocBook being installed was valuable when we were on the
OpenSP docs toolchain, because that was rather hard to get installed
fully. Nowadays, as long as you have xmllint and xsltproc installed,
you're good, because those programs will fetch the DocBook files off
the net at need. Moreover, testing this at configure time means that
a network access may well occur whether or not you have any interest
in building the docs later. That can be slow (typically 2 or 3
seconds, though much higher delays have been reported), and it seems
not very nice to be doing an off-machine access without warning, too.
Hence, drop the PGAC_CHECK_DOCBOOK probe, and adjust related
documentation. Without that macro, there's not much left of
config/docbook.m4 at all, so I just removed it.
Back-patch to v11, where we started to use xmllint in the
PGAC_CHECK_DOCBOOK probe.
Discussion: https://postgr.es/m/E2EE6B76-2D96-408A-B961-CAE47D1A86F0@yesql.se
Discussion: https://postgr.es/m/A55A7FC9-FA60-47FE-98B5-139CDC57CE6E@gmail.com
M aclocal.m4
D config/docbook.m4
M configure
M configure.ac
M doc/src/sgml/docguide.sgml
Prevent parallel index build in a standalone backend.
commit : 275b3411d9189b3974687766db06727d64d22979
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 14:38:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 14:38:00 -0500
This can't work if there's no postmaster, and indeed the code got an
assertion failure trying. There should be a check on IsUnderPostmaster
gating the use of parallelism, as the planner has for ordinary
parallel queries.
Commit 40d964ec9 got this right, so follow its model of checking
IsUnderPostmaster at the same place where we check for
max_parallel_maintenance_workers == 0. In general, new code
implementing parallel utility operations should do the same.
Report and patch by Yulin Pei, cosmetically adjusted by me.
Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/HK0PR01MB22747D839F77142D7E76A45DF4F50@HK0PR01MB2274.apcprd01.prod.exchangelabs.com
M src/backend/optimizer/plan/planner.c
Fix miscomputation of direct_lateral_relids for join relations.
commit : b1738ff6ab73203cbbc02d7fb82941dbc061d301
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 12:22:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 12:22:43 -0500
If a PlaceHolderVar is to be evaluated at a join relation, but
its value is only needed there and not at higher levels, we neglected
to update the joinrel's direct_lateral_relids to include the PHV's
source rel. This causes problems because join_is_legal() then won't
allow joining the joinrel to the PHV's source rel at all, leading
to "failed to build any N-way joins" planner failures.
Per report from Andreas Seltenreich. Back-patch to 9.5
where the problem originated.
Discussion: https://postgr.es/m/87blfgqa4t.fsf@aurora.ydns.eu
M src/backend/optimizer/util/placeholder.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Refactor parsing rules for option lists of EXPLAIN, VACUUM and ANALYZE
commit : 873ea9ee692e7829614f913685db540b17998ba6
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Nov 2020 20:27:37 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Nov 2020 20:27:37 +0900
Those three commands have been using the same grammar rules to handle a
a list of parenthesized options. This refactors the code so as they use
the same parsing rules, shaving some code. A future commit will make
use of those option parsing rules for more utility commands, like
REINDEX and CLUSTER.
Author: Alexey Kondratov, Justin Pryzby
Discussion: https://postgr.es/m/8a8f5f73-00d3-55f8-7583-1375ca8f6a91@postgrespro.ru
M src/backend/parser/gram.y
Remove leftover comments, left behind by removal of WITH OIDS.
commit : 2bc588798bdd0cdaa8f6cb6713ba8c8cc039fcb1
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 30 Nov 2020 10:26:43 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 30 Nov 2020 10:26:43 +0200
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqGaRoF3XrhPW-Y7P%2BG7bKo84Z_h%3DkQHvMh-80%3Dav3wmOw%40mail.gmail.com
M contrib/file_fdw/file_fdw.c
M src/backend/commands/copyfromparse.c
Fix typo in comment.
commit : 6742e14959a3033d946ab3d67f5ce4c99367d332
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 30 Nov 2020 12:54:31 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 30 Nov 2020 12:54:31 +0900
Author: Haiying Tang <tanghy.fnst@cn.fujitsu.com>
Discussion: https://postgr.es/m/48a0928ac94b497d9c40acf1de394c15@G08CNEXMBPEKD05.g08.fujitsu.local
M src/include/optimizer/cost.h
Improve log message about termination of background workers.
commit : 98e2d58d66c81d9f1e49813c1acd4d7312089f4a
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 30 Nov 2020 11:05:19 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 30 Nov 2020 11:05:19 +0900
Previously the shutdown of a background worker that uses die() as
SIGTERM signal handler produced the log message "terminating
connection due to administrator command". This log message was
confusing because a background worker is not a connection.
This commit improves that log message to "terminating background
worker XXX due to administrator command" (XXX is replaced with
the name of the background worker). This is the same log message
as another SIGTERM signal handler bgworker_die() for a background
worker reports.
Author: Bharath Rupireddy
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/3f292fbb-f155-9a01-7cb2-7ccc9007ab3f@oss.nttdata.com
M src/backend/tcop/postgres.c
Fix recently-introduced breakage in psql's \connect command.
commit : 7e5e1bba034ee02245e8a3833aa1f6ea7253b584
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Nov 2020 15:22:04 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Nov 2020 15:22:04 -0500
Through my misreading of what the existing code actually did,
commits 85c54287a et al. broke psql's behavior for the case where
"\c connstring" provides a password in the connstring. We should
use that password in such a case, but as of 85c54287a we ignored it
(and instead, prompted for a password).
Commit 94929f1cf fixed that in HEAD, but since I thought it was
cleaning up a longstanding misbehavior and not one I'd just created,
I didn't back-patch it.
Hence, back-patch the portions of 94929f1cf having to do with
password management. In addition to fixing the introduced bug,
this means that "\c -reuse-previous=on connstring" will allow
re-use of an existing connection's password if the connstring
doesn't change user/host/port. That didn't happen before, but
it seems like a bug fix, and anyway I'm loath to have significant
differences in this code across versions.
Also fix an error with the same root cause about whether or not to
override a connstring's setting of client_encoding. As of 85c54287a
we always did so; restore the previous behavior of overriding only
when stdin/stdout are a terminal and there's no environment setting
of PGCLIENTENCODING. (I find that definition a bit surprising, but
right now doesn't seem like the time to revisit it.)
Per bug #16746 from Krzysztof Gradek. As with the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/16746-44b30e2edf4335d4@postgresql.org
M src/bin/psql/command.c
Doc: clarify behavior of PQconnectdbParams().
commit : d5e2bdf7dda712a60234f08e036ec9de28085a2d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Nov 2020 13:58:30 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Nov 2020 13:58:30 -0500
The documentation omitted the critical tidbit that a keyword-array entry
is simply ignored if its corresponding value-array entry is NULL or an
empty string; it will *not* override any previously-obtained value for
the parameter. (See conninfo_array_parse().) I'd supposed that would
force the setting back to default, which is what led me into bug #16746;
but it doesn't.
While here, I couldn't resist the temptation to do some copy-editing,
both in the description of PQconnectdbParams() and in the section
about connection URI syntax.
Discussion: https://postgr.es/m/931505.1606618746@sss.pgh.pa.us
M doc/src/sgml/libpq.sgml
Retry initial slurp_file("current_logfiles"), in test 004_logrotate.pl.
commit : 0f89ca083bbf71641cf61b4b951d6b30a2fd978b
author : Noah Misch <noah@leadboat.com>
date : Sat, 28 Nov 2020 21:52:27 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 28 Nov 2020 21:52:27 -0800
Buildfarm member topminnow failed when the test script attempted this
before the syslogger would have created the file. Back-patch to v12,
which introduced the test.
M src/bin/pg_ctl/t/004_logrotate.pl
Clean up after tests in src/test/locale/.
commit : b90a7fe15f78b1e2513cbcb27a33f4adc47bcd3d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Nov 2020 16:08:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Nov 2020 16:08:36 -0500
Oversight in 257836a75, which added these tests.
M src/test/locale/Makefile
Fix a recently-introduced race condition in LISTEN/NOTIFY handling.
commit : 9c83b54a9ccdb111ce693ada2309475197c19d70
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Nov 2020 14:03:40 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Nov 2020 14:03:40 -0500
Commit 566372b3d fixed some race conditions involving concurrent
SimpleLruTruncate calls, but it introduced new ones in async.c.
A newly-listening backend could attempt to read Notify SLRU pages that
were in process of being truncated, possibly causing an error. Also,
the QUEUE_TAIL pointer could become set to a value that's not equal to
the queue position of any backend. While that's fairly harmless in
v13 and up (thanks to commit 51004c717), in older branches it resulted
in near-permanent disabling of the queue truncation logic, so that
continued use of NOTIFY led to queue-fill warnings and eventual
inability to send any more notifies. (A server restart is enough to
make that go away, but it's still pretty unpleasant.)
The core of the problem is confusion about whether QUEUE_TAIL
represents the "logical" tail of the queue (i.e., the oldest
still-interesting data) or the "physical" tail (the oldest data we've
not yet truncated away). To fix, split that into two variables.
QUEUE_TAIL regains its definition as the logical tail, and we
introduce a new variable to track the oldest un-truncated page.
Per report from Mikael Gustavsson. Like the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/1b8561412e8a4f038d7a491c8b922788@smhi.se
M src/backend/commands/async.c
Fix CLUSTER progress reporting of number of blocks scanned.
commit : 3df51ca8b39f08ef19a77b9776f2547c86b70c49
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 27 Nov 2020 20:16:44 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 27 Nov 2020 20:16:44 +0900
Previously pg_stat_progress_cluster view reported the current block
number in heap scan as the number of heap blocks scanned (i.e.,
heap_blks_scanned). This reported number could be incorrect when
synchronize_seqscans is enabled, because it allowed the heap scan to
start at block in middle. This could result in wraparounds in the
heap_blks_scanned column when the heap scan wrapped around.
This commit fixes the bug by calculating the number of blocks from
the block that the heap scan starts at to the current block in scan,
and reporting that number in the heap_blks_scanned column.
Also, in pg_stat_progress_cluster view, previously heap_blks_scanned
could not reach heap_blks_total at the end of heap scan phase
if the last pages scanned were empty. This commit fixes the bug by
manually updating heap_blks_scanned to the same value as
heap_blks_total when the heap scan phase finishes.
Back-patch to v12 where pg_stat_progress_cluster view was introduced.
Reported-by: Matthias van de Meent
Author: Matthias van de Meent
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAEze2WjCBWSGkVfYag001Rc4+-nNLDpWM7QbyD6yPvuhKs-gYQ@mail.gmail.com
M src/backend/access/heap/heapam_handler.c
Use standard SIGTERM signal handler die() in test_shm_mq worker.
commit : ef848f4ac5a4bd072c65867186268775acfb4298
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 27 Nov 2020 15:45:01 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 27 Nov 2020 15:45:01 +0900
Previously test_shm_mq worker used the stripped-down version of die()
as the SIGTERM signal handler. This commit makes it use die(), instead,
to simplify the code.
In terms of the code, the difference between die() and the stripped-down
version previously used is whether the signal handler directly may call
ProcessInterrupts() or not. But this difference doesn't exist in
a background worker because, in bgworker, DoingCommandRead flag will
never be true and die() will never call ProcessInterrupts() directly.
Therefore test_shm_mq worker can safely use die(), like other bgworker
proceses (e.g., logical replication apply launcher or autoprewarm worker)
currently do.
Thanks to Craig Ringer for the report and investigation of the issue.
Author: Bharath Rupireddy
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAGRY4nxsAe_1k_9g5b47orA0S011iBoHsXHFMH7cg7HV0O1bwQ@mail.gmail.com
M src/test/modules/test_shm_mq/worker.c
Use standard SIGHUP and SIGTERM signal handlers in worker_spi.
commit : 2a0847720ae6fa4180bfbe404e4e6bbf2f737708
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 27 Nov 2020 15:11:19 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 27 Nov 2020 15:11:19 +0900
Previously worker_spi used its custom signal handlers for SIGHUP and
SIGTERM. This commit makes worker_spi use the standard signal handlers,
to simplify the code.
Note that die() is used as the standard SIGTERM signal handler in
worker_spi instead of SignalHandlerForShutdownRequest() or bgworker_die().
Previously the exit handling was only able to exit from within the main loop,
and not from within the backend code it calls. This is why die() needs to
be used here, so worker_spi can respond to SIGTERM signal while it's
executing a query.
Maybe we can say that it's a bug that worker_spi could not respond to
SIGTERM during query execution. But since worker_spi is a just example
of the background worker code, we don't do the back-patch.
Thanks to Craig Ringer for the report and investigation of the issue.
Author: Bharath Rupireddy
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CALj2ACXDEZhAFOTDcqO9cFSRvrFLyYOnPKrcA1UG4uZn9hUAVg@mail.gmail.com
Discussion: https://postgr.es/m/CAGRY4nxsAe_1k_9g5b47orA0S011iBoHsXHFMH7cg7HV0O1bwQ@mail.gmail.com
M src/test/modules/worker_spi/worker_spi.c
Fix replication of in-progress transactions in tablesync worker.
commit : 0926e96c493443644ba8e96b5d96d013a9ffaf64
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 27 Nov 2020 07:43:34 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 27 Nov 2020 07:43:34 +0530
Tablesync worker runs under a single transaction but in streaming mode, we
were committing the transaction on stream_stop, stream_abort, and
stream_commit. We need to avoid committing the transaction in a streaming
mode in tablesync worker.
In passing move the call to process_syncing_tables in
apply_handle_stream_commit after clean up of stream files. This will
allow clean up of files to happen before the exit of tablesync worker
which would otherwise be handled by one of the proc exit routines.
Author: Dilip Kumar
Reviewed-by: Amit Kapila and Peter Smith
Tested-by: Peter Smith
Discussion: https://postgr.es/m/CAHut+Pt4PyKQCwqzQ=EFF=bpKKJD7XKt_S23F6L20ayQNxg77A@mail.gmail.com
M src/backend/replication/logical/worker.c
Restore lock level to update statusFlags
commit : dcfff74fb16622898a9d1df33e530f477caab095
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 26 Nov 2020 12:30:48 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 26 Nov 2020 12:30:48 -0300
Reverts 27838981be9d (some comments are kept). Per discussion, it does
not seem safe to relax the lock level used for this; in order for it to
be safe, there would have to be memory barriers between the point we set
the flag and the point we set the trasaction Xid, which perhaps would
not be so bad; but there would also have to be barriers at the readers'
side, which from a performance perspective might be bad.
Now maybe this analysis is wrong and it *is* safe for some reason, but
proof of that is not trivial.
Discussion: https://postgr.es/m/20201118190928.vnztes7c2sldu43a@alap3.anarazel.de
M src/backend/commands/vacuum.c
M src/backend/replication/logical/logical.c
M src/backend/storage/ipc/procarray.c
M src/include/storage/proc.h
pg_stat_statements: Track number of times pgss entries were deallocated.
commit : 9fbc3f318d039c3e1e8614c38e40843cf8fcffde
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 26 Nov 2020 21:18:05 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 26 Nov 2020 21:18:05 +0900
If more distinct statements than pg_stat_statements.max are observed,
pg_stat_statements entries about the least-executed statements are
deallocated. This commit enables us to track the total number of times
those entries were deallocated. That number can be viewed in the
pg_stat_statements_info view that this commit adds. It's useful when
tuning pg_stat_statements.max parameter. If it's high, i.e., the entries
are deallocated very frequently, which might cause the performance
regression and we can increase pg_stat_statements.max to avoid those
frequent deallocations.
The pg_stat_statements_info view is intended to display the statistics
of pg_stat_statements module itself. Currently it has only one column
"dealloc" indicating the number of times entries were deallocated.
But an upcoming patch will add other columns (for example, the time
at which pg_stat_statements statistics were last reset) into the view.
Author: Katsuragi Yuta, Yuki Seino
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/0d9f1107772cf5c3f954e985464c7298@oss.nttdata.com
M contrib/pg_stat_statements/Makefile
M contrib/pg_stat_statements/expected/pg_stat_statements.out
A contrib/pg_stat_statements/pg_stat_statements–1.8–1.9.sql
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/pg_stat_statements.control
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
M doc/src/sgml/pgstatstatements.sgml
M src/tools/pgindent/typedefs.list
doc: Add description about re-analysis and re-planning of a prepared statement.
commit : 4a36eab79a193700b7b65baf6c09c795c90c02c6
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 26 Nov 2020 16:17:10 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 26 Nov 2020 16:17:10 +0900
A prepared statement is re-analyzed and re-planned whenever database
objects used in the statement have undergone definitional changes or
the planner statistics of them have been updated. The former has been
documented from before, but the latter was not previously. This commit
adds the description about the latter case into the docs.
Author: Atsushi Torikoshi
Reviewed-by: Andy Fan, Fujii Masao
Discussion: https://postgr.es/m/3ac82f4817c9fe274a905c8a38d87bd9@oss.nttdata.com
M doc/src/sgml/ref/prepare.sgml
Use Enums for logical replication message types at more places.
commit : f3a8f73ec274bf8cc6066ea315f1a5a022214b02
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 26 Nov 2020 09:21:14 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 26 Nov 2020 09:21:14 +0530
Commit 644f0d7cc9 added logical replication message type enums to use
instead of character literals but some char substitutions were overlooked.
Author: Peter Smith
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAHut+PsTG=Vrv8hgrvOnAvCNR21jhqMdPk2n0a1uJPoW0p+UfQ@mail.gmail.com
M src/backend/replication/logical/worker.c
Avoid spurious waits in concurrent indexing
commit : c98763bf51bf610b3ee7e209fc76c3ff9a6b3163
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 25 Nov 2020 18:21:08 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 25 Nov 2020 18:21:08 -0300
In the various waiting phases of CREATE INDEX CONCURRENTLY (CIC) and
REINDEX CONCURRENTLY (RC), we wait for other processes to release their
snapshots; this is necessary in general for correctness. However,
processes doing CIC in other tables cannot possibly affect CIC or RC
done in "this" table, so we don't need to wait for those. This commit
adds a flag in MyProc->statusFlags to indicate that the current process
is doing CIC, so that other processes doing CIC or RC can ignore it when
waiting.
Note that this logic is only valid if the index does not access other
tables. For simplicity we avoid setting the flag if the index has a
column that's an expression, or has a WHERE predicate. (It is possible
to have expressional or partial indexes that do not access other tables,
but figuring that out would require more work.)
This flag can potentially also be used by processes doing REINDEX
CONCURRENTLY to be skipped; and by VACUUM to ignore processes in CIC or
RC for the purposes of computing an Xmin. That's left for future
commits.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Author: Dimitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20200810233815.GA18970@alvherre.pgsql
M src/backend/commands/indexcmds.c
M src/include/storage/proc.h
In psql's \d commands, don't truncate attribute default values.
commit : 314fb9baeacb3426a5d9a26132aae8d828cc0ad7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 25 Nov 2020 16:19:25 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 25 Nov 2020 16:19:25 -0500
Historically, psql has truncated the text of a column's default
expression at 128 characters. This is unlike any other behavior
in describe.c, and it's become particularly confusing now that
the limit is only applied to the expression proper and not to
the "generated always as (...) stored" text that may get wrapped
around it.
Excavation in our git history suggests that the original motivation
for this limit was not really to limit the display width (as I'd long
supposed), but to make it safe to use a fixed-width output buffer to
store the result. That implementation restriction is long gone of
course, but the limit remained. Let's just get rid of it.
While here, rearrange the logic about when to free the output string
so that it's not so dependent on unstated assumptions about the
possible values of attidentity and attgenerated.
Per bug #16743 from David Turon. Back-patch to v12 where GENERATED
came in. (Arguably we could take it back further, but I'm hesitant
to change the behavior of long-stable branches for this.)
Discussion: https://postgr.es/m/16743-7b1bacc4af76e7ad@postgresql.org
M src/bin/psql/describe.c
Doc: minor improvements for section 11.2 "Index Types".
commit : 85b4ba73423b480902206ca04330c1cbea371c3c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 25 Nov 2020 14:04:28 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 25 Nov 2020 14:04:28 -0500
Break the per-index-type discussions into <sect2>'s so as to make
them more visually separate and easier to find. Improve the markup,
and make a couple of small wording adjustments.
This also fixes one stray reference to the now-deprecated point
operators <^ and >^.
Dagfinn Ilmari Mannsåker, reviewed by David Johnston and Jürgen Purtz
Discussion: https://postgr.es/m/877dukhvzg.fsf@wibble.ilmari.org
M doc/src/sgml/indices.sgml
Avoid spamming the client with multiple ParameterStatus messages.
commit : 2432b1a04087edc2fd9536c7c9aa4ca03fd1b363
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 25 Nov 2020 11:40:44 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 25 Nov 2020 11:40:44 -0500
Up to now, we sent a ParameterStatus message to the client immediately
upon any change in the active value of any GUC_REPORT variable. This
was only barely okay when the feature was designed; now that we have
things like function SET clauses, there are very plausible use-cases
where a GUC_REPORT variable might change many times within a query
--- and even end up back at its original value, perhaps. Fortunately
most of our GUC_REPORT variables are unlikely to be changed often;
but there are proposals in play to enlarge that set, or even make it
user-configurable.
Hence, let's fix things to not generate more than one ParameterStatus
message per variable per query, and to not send any message at all
unless the end-of-query value is different from what we last reported.
Discussion: https://postgr.es/m/5708.1601145259@sss.pgh.pa.us
M src/backend/tcop/postgres.c
M src/backend/utils/misc/guc.c
M src/include/utils/guc.h
M src/include/utils/guc_tables.h
tablefunc: Reject negative number of tuples passed to normal_rand()
commit : f73999262ed6c40d9a7c3d7cccec7143d4d15287
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Nov 2020 15:30:18 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Nov 2020 15:30:18 +0100
The function converted the first argument i.e. the number of tuples to
return into an unsigned integer which turns out to be huge number when
a negative value is passed. This causes the function to take much
longer time to execute. Instead, reject a negative value.
(If someone really wants to generate many more result rows, they
should consider adding a bigint or numeric variant.)
While at it, improve SQL test to test the number of tuples returned by
this function.
Author: Ashutosh Bapat <ashutosh.bapat@2ndquadrant.com>
Discussion: https://www.postgresql.org/message-id/CAG-ACPW3PUUmSnM6cLa9Rw4BEC5cEMKjX8Gogc8gvQcT3cYA1A@mail.gmail.com
M contrib/tablefunc/expected/tablefunc.out
M contrib/tablefunc/sql/tablefunc.sql
M contrib/tablefunc/tablefunc.c
doc: Fix typos
commit : 2fbd786c3446b1bc90f396d3fa5f4614b0a57d89
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Nov 2020 09:49:00 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Nov 2020 09:49:00 +0100
Author: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/20201121194105.GO24784@telsasoft.com
M doc/src/sgml/contrib.sgml
Make error hint from bind() failure more accurate
commit : d5d91acdccae6b322a5634be2c66d62fc6b9a43d
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Nov 2020 08:14:23 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Nov 2020 08:14:23 +0100
The hint "Is another postmaster already running ..." should only be
printed for errors that are really about something else already using
the address. In other cases it is misleading. So only show that hint
if errno == EADDRINUSE.
Also, since Unix-domain sockets in the file-system namespace never
report EADDRINUSE for an existing file (they would just overwrite it),
the part of the hint saying "If not, remove socket file \"%s\" and
retry." can never happen, so remove it. Unix-domain sockets in the
abstract namespace can report EADDRINUSE, but in that case there is no
file to remove, so the hint doesn't work there either.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/6dee8574-b0ad-fc49-9c8c-2edc796f0033@2ndquadrant.com
M src/backend/libpq/pqcomm.c
Add support for abstract Unix-domain sockets
commit : c9f0624bc2f544baacafa38e3797d5323401d039
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Nov 2020 08:14:23 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Nov 2020 08:14:23 +0100
This is a variant of the normal Unix-domain sockets that don't use the
file system but a separate "abstract" namespace. At the user
interface, such sockets are represented by names starting with "@".
Supported on Linux and Windows right now.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/6dee8574-b0ad-fc49-9c8c-2edc796f0033@2ndquadrant.com
M doc/src/sgml/config.sgml
M doc/src/sgml/libpq.sgml
M src/backend/libpq/pqcomm.c
M src/bin/psql/command.c
M src/bin/psql/prompt.c
M src/common/ip.c
M src/include/libpq/pqcomm.h
M src/interfaces/libpq/fe-connect.c
Fix WaitLatch(NULL) on Windows.
commit : a7e65dc88b6f088fc2fcf5a660d866de644b1300
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 25 Nov 2020 17:44:15 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 25 Nov 2020 17:44:15 +1300
Further to commit 733fa9aa, on Windows when a latch is triggered but we
aren't currently waiting for it, we need to locate the latch's HANDLE
rather than calling ResetEvent(NULL).
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reported-by: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/CAEudQArTPi1YBc%2Bn1fo0Asy3QBFhVjp_QgyKG-8yksVn%2ByRTiw%40mail.gmail.com
M src/backend/storage/ipc/latch.c
Remove obsolete comment atop ri_PlanCheck.
commit : 805b816305b62cb3e2670418ced8f7184a2d29c7
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 25 Nov 2020 09:14:45 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 25 Nov 2020 09:14:45 +0530
Commit 5b7ba75f7f removed the unused parameter but forgot to update the
nearby comments.
Author: Li Japin
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/0E2F62A2-B2F1-4052-83AE-F0BEC8A75789@hotmail.com
M src/backend/utils/adt/ri_triggers.c
Stop gap fix for __attribute__((cold)) compiler bug in MinGW 8.1
commit : 687f6163447514c23b44a981540987013529a9ec
author : David Rowley <drowley@postgresql.org>
date : Wed, 25 Nov 2020 16:33:43 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 25 Nov 2020 16:33:43 +1300
The buildfarm animal walleye, running MinGW 8.1 has been having problems
ever since 697e1d02f and 913ec71d6 went in. This appears to be a bug in
assembler which was fixed in a later version.
For now, in order to get that animal running green again, let's just
define pg_attribute_cold and pg_attribute_hot to be empty macros on that
compiler. Hopefully, we can get the support of the owner of the animal to
upgrade to a less buggy compiler and revert this at a later date.
Discussion: https://postgr.es/m/286560.1606233316@sss.pgh.pa.us
M src/include/c.h
Remove catalog function currtid()
commit : 7b94e999606e2e2e10d68d544d49fc5a5d5785ac
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 25 Nov 2020 12:18:26 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 25 Nov 2020 12:18:26 +0900
currtid() and currtid2() are an undocumented set of functions whose sole
known user is the Postgres ODBC driver, able to retrieve the latest TID
version for a tuple given by the caller of those functions.
As used by Postgres ODBC, currtid() is a shortcut able to retrieve the
last TID loaded into a backend by passing an OID of 0 (magic value)
after a tuple insertion. This is removed in this commit, as it became
obsolete after the driver began using "RETURNING ctid" with inserts, a
clause supported since Postgres 8.2 (using RETURNING is better for
performance anyway as it reduces the number of round-trips to the
backend).
currtid2() is still used by the driver, so this remains around for now.
Note that this function is kept in its original shape for backward
compatibility reasons.
Per discussion with many people, including Andres Freund, Peter
Eisentraut, Álvaro Herrera, Hiroshi Inoue, Tom Lane and myself.
Bump catalog version.
Discussion: https://postgr.es/m/20200603021448.GB89559@paquier.xyz
M src/backend/executor/nodeModifyTable.c
M src/backend/utils/adt/tid.c
M src/include/access/heapam.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/tid.out
M src/test/regress/sql/tid.sql
Properly check index mark/restore in ExecSupportsMarkRestore.
commit : 660b89928d18386de7755565c008439ae75d1218
author : Andrew Gierth <rhodiumtoad@postgresql.org>
date : Tue, 24 Nov 2020 20:58:32 +0000
committer: Andrew Gierth <rhodiumtoad@postgresql.org>
date : Tue, 24 Nov 2020 20:58:32 +0000
Previously this code assumed that all IndexScan nodes supported
mark/restore, which is not true since it depends on optional index AM
support functions. This could lead to errors about missing support
functions in rare edge cases of mergejoins with no sort keys, where an
unordered non-btree index scan was placed on the inner path without a
protecting Materialize node. (Normally, the fact that merge join
requires ordered input would avoid this error.)
Backpatch all the way since this bug is ancient.
Per report from Eugen Konkov on irc.
Discussion: https://postgr.es/m/87o8jn50be.fsf@news-spur.riddles.org.uk
M src/backend/executor/execAmi.c
M src/backend/optimizer/util/plancat.c
M src/include/nodes/pathnodes.h
Tidy up definitions of pg_attribute_hot and pg_attribute_cold
commit : b0727ae99b7056ca88e9708c014a86f6e611d938
author : David Rowley <drowley@postgresql.org>
date : Wed, 25 Nov 2020 10:52:50 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 25 Nov 2020 10:52:50 +1300
1fa22a43a was a quick fix for portability problem I introduced in
697e1d02f. 1fa22a43a adds a few more cases to the preprocessor logic than
I'd have liked. Andres Freund and Dagfinn Ilmari Mannsåker suggested a
better way to do this.
In passing, also adjust the only current usage of these macros so that the
macro comes before the function's return type in the declaration of the
function. This now matches what the definition of the function does.
Discussion: https://postgr.es/m/20200625163553.lt6wocbjhklp5pl4@alap3.anarazel.de
Discussion: https://postgr.es/m/87pn43bmok.fsf@wibble.ilmari.org
M src/include/c.h
M src/include/utils/elog.h
Put "inline" marker on declarations of inline functions.
commit : ec05bafdbbf474bf0a1416772da31f9f1f27fa1e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 24 Nov 2020 15:43:01 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 24 Nov 2020 15:43:01 -0500
I'm having a hard time telling whether the letter of the C standard
requires this, but we do have a couple of buildfarm members that
throw warnings when this is not done. Oversight in c532d15dd.
M src/backend/commands/copyfromparse.c
Fix expected output: the order of agg permission checks changed.
commit : 8818ad5b1557d42cd58e5196fac7084a2389bdad
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 24 Nov 2020 12:50:16 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 24 Nov 2020 12:50:16 +0200
Commit 0a2bc5d61e changed the order that permissions on the final and
transition functions of an aggregate are checked in. That shows up as a
difference in the order the LOG messages in this sepgsql regression test
are printed. Adjust the expected output.
Per buildfarm failure in rhinoceros.
M contrib/sepgsql/expected/misc.out
Move per-agg and per-trans duplicate finding to the planner.
commit : 0a2bc5d61e713e3fe72438f020eea5fcc90b0f0b
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 24 Nov 2020 10:45:00 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 24 Nov 2020 10:45:00 +0200
This has the advantage that the cost estimates for aggregates can count
the number of calls to transition and final functions correctly.
Bump catalog version, because views can contain Aggrefs.
Reviewed-by: Andres Freund
Discussion: https://www.postgresql.org/message-id/b2e3536b-1dbc-8303-c97e-89cb0b4a9a48%40iki.fi
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/nodeAgg.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planagg.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/prep/Makefile
A src/backend/optimizer/prep/prepagg.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/parse_func.c
M src/backend/utils/adt/selfuncs.c
M src/include/catalog/catversion.h
M src/include/executor/execExpr.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/primnodes.h
M src/include/optimizer/clauses.h
M src/include/optimizer/prep.h
M src/include/utils/selfuncs.h
M src/test/regress/expected/partition_aggregate.out
M src/test/regress/sql/partition_aggregate.sql
M src/tools/pgindent/typedefs.list
doc: Get rid of unnecessary space character from some index items.
commit : e522024bd8dd28a0f13dcccfd39170698f45c939
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 24 Nov 2020 17:00:16 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 24 Nov 2020 17:00:16 +0900
Previously some index items have " ," (i.e., space + comma) in the docs
as follows. Since the space character before the comma is unnecessary,
this commit gets rid of that for the sake of consistency with other
index items.
parallel_leader_participation configuration parameter , Other Planner Options
Author: Fujii Masao
Reviewed-by: Euler Taveira
Discussion: https://postgr.es/m/e87b4ddf-1498-2850-bf55-519df3928fd4@oss.nttdata.com
M doc/src/sgml/config.sgml
M doc/src/sgml/pgtrgm.sgml
Use macros instead of hardcoded offsets for LWLock initialization
commit : d03d7549b29236e300aceac0c22173cf19acc675
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 24 Nov 2020 12:39:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 24 Nov 2020 12:39:58 +0900
This makes the code slightly easier to follow, as the initialization
relies on an offset that overlapped with an equivalent set of macros
defined, which are used in other places already.
Author: Japin Li
Discussion: https://postgr.es/m/MEYP282MB1669FB410006758402F2C3A2B6E00@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
M src/backend/storage/lmgr/lwlock.c
Centralize logic for skipping useless ereport/elog calls.
commit : 789b938bf2b8e38d0894261eae6bc84bbbb4714e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Nov 2020 19:04:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Nov 2020 19:04:07 -0500
While ereport() and elog() themselves are quite cheap when the
error message level is too low to be printed, some places need to do
substantial work before they can call those macros at all. To allow
optimizing away such setup work when nothing is to be printed, make
elog.c export a new function message_level_is_interesting(elevel)
that reports whether ereport/elog will do anything. Make use of that
in various places that had ad-hoc direct tests of log_min_messages etc.
Also teach ProcSleep to use it to avoid some work. (There may well
be other places that could usefully use this; I didn't search hard.)
Within elog.c, refactor a little bit to avoid having duplicate copies
of the policy-setting logic. When that code was written, we weren't
relying on the availability of inline functions; so it had some
duplications in the name of efficiency, which I got rid of.
Alvaro Herrera and Tom Lane
Discussion: https://postgr.es/m/129515.1606166429@sss.pgh.pa.us
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlogutils.c
M src/backend/catalog/dependency.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walsender.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/error/elog.c
M src/include/utils/elog.h
Fix unportable usage of __has_attribute
commit : 1fa22a43a56e1fe44c7bb3a3d5ef31be5bcac41d
author : David Rowley <drowley@postgresql.org>
date : Tue, 24 Nov 2020 13:09:35 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 24 Nov 2020 13:09:35 +1300
This should fix the breakages caused by 697e1d02f, which seems to break
the build for GCC version < 5.
It seems, according to the GCC manual that __has_attribute is a "special
operator" and must be tested without any other conditions in the
preprocessor test.
Per recommendation from the GCC manual via Greg Nancarrow
Reported-by: Greg Nancarrow
Discussion: https://postgr.es/m/CAJcOf-euSu8fhC10v476o9dqnjqKysVs1_vRms-_fvajpZ3kFw@mail.gmail.com
M src/include/c.h
Improve compiler code layout in elog/ereport ERROR calls
commit : 913ec71d682e99852cc188a7edbdccd02d42b4b3
author : David Rowley <drowley@postgresql.org>
date : Tue, 24 Nov 2020 12:04:42 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 24 Nov 2020 12:04:42 +1300
Here we use a bit of preprocessor trickery to coax supporting compilers
into laying out their generated code so that the code that's in the same
branch as elog(ERROR)/ereport(ERROR) calls is moved away from the hot
path. Effectively, this reduces the size of the hot code meaning that it
can sit on fewer cache lines.
Performance improvements of between 10-15% have been seen on highly CPU
bound workloads using pgbench's TPC-b benchmark.
What's achieved here is very similar to putting the error condition inside
an unlikely() macro. For example;
if (unlikely(x < 0))
elog(ERROR, "invalid x value");
now there's no need to make use of unlikely() here as the common macro
used by elog and ereport will now see that elevel is >= ERROR and make use
of a pg_attribute_cold marked version of errstart().
When elevel < ERROR or if it cannot be determined to be constant, the
original behavior is maintained.
Author: David Rowley
Reviewed-by: Andres Freund, Peter Eisentraut
Discussion: https://postgr.es/m/CAApHDvrVpasrEzLL2er7p9iwZFZ%3DJj6WisePcFeunwfrV0js_A%40mail.gmail.com
M src/backend/utils/error/elog.c
M src/include/utils/elog.h
Define pg_attribute_cold and pg_attribute_hot macros
commit : 697e1d02f53f985992a4f479b0b08fca5c272c6c
author : David Rowley <drowley@postgresql.org>
date : Tue, 24 Nov 2020 11:29:28 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 24 Nov 2020 11:29:28 +1300
For compilers supporting __has_attribute and __has_attribute (hot/cold).
__has_attribute is supported on gcc >= 5, clang >= 2.9 and icc >= 17.
A followup commit will implement some usages of these macros.
Author: David Rowley
Reviewed-by: Andres Freund, Peter Eisentraut
Discussion: https://postgr.es/m/CAApHDvrVpasrEzLL2er7p9iwZFZ%3DJj6WisePcFeunwfrV0js_A%40mail.gmail.com
M src/include/c.h
Remove unnecessary #include.
commit : 3b9b01f75d6e2d7bf9e0bf8ec958ce420aa037c2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Nov 2020 17:00:01 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Nov 2020 17:00:01 -0500
Justin Pryzby
Discussion: https://postgr.es/m/20201123205505.GJ24052@telsasoft.com
M src/include/optimizer/clauses.h
Don't hold ProcArrayLock longer than needed in rare cases
commit : 450c8230b1f7eed1e927c44fb710d80502bea1d2
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 23 Nov 2020 18:55:23 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 23 Nov 2020 18:55:23 -0300
While cancelling an autovacuum worker, we hold ProcArrayLock while
formatting a debugging log string. We can make this shorter by saving
the data we need to produce the message and doing the formatting outside
the locked region.
This isn't terribly critical, as it only occurs pretty rarely: when a
backend runs deadlock detection and it happens to be blocked by a
autovacuum running autovacuum. Still, there's no need to cause a hiccup
in ProcArrayLock processing, which can be very high-traffic in some
cases.
While at it, rework code so that we only print the string when it is
really going to be used, as suggested by Michael Paquier.
Discussion: https://postgr.es/m/20201118214127.GA3179@alvherre.pgsql
Reviewed-by: Michael Paquier <michael@paquier.xyz>
M src/backend/storage/lmgr/proc.c
Rename the "point is strictly above/below point" comparison operators.
commit : 0cc99327888840f2bf572303b68438e4caf62de9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Nov 2020 11:38:37 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Nov 2020 11:38:37 -0500
Historically these were called >^ and <^, but that is inconsistent
with the similar box, polygon, and circle operators, which are named
|>> and <<| respectively. Worse, the >^ and <^ names are used for
*not* strict above/below tests for the box type.
Hence, invent new operators following the more common naming. The
old operators remain available for now, and are still accepted by
the relevant index opclasses too. But there's a deprecation notice,
so maybe we can get rid of them someday.
Emre Hasegeli, reviewed by Pavel Borisov
Discussion: https://postgr.es/m/24348.1587444160@sss.pgh.pa.us
M doc/src/sgml/func.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/spgist.sgml
M src/backend/access/gist/gistproc.c
M src/backend/access/spgist/spgkdtreeproc.c
M src/backend/access/spgist/spgquadtreeproc.c
M src/include/access/stratnum.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_operator.dat
M src/test/regress/expected/create_index.out
M src/test/regress/expected/create_index_spgist.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/expected/point.out
M src/test/regress/sql/create_index.sql
M src/test/regress/sql/create_index_spgist.sql
M src/test/regress/sql/point.sql
Improve wording of two error messages related to generated columns.
commit : d36228a9fcdccd57a7dc332572eb9837c7c301e6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Nov 2020 11:15:03 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Nov 2020 11:15:03 -0500
Clarify that you can "insert" into a generated column as long as what
you're inserting is a DEFAULT placeholder.
Also, use ERRCODE_GENERATED_ALWAYS in place of ERRCODE_SYNTAX_ERROR;
there doesn't seem to be any reason to use the less specific errcode.
Discussion: https://postgr.es/m/9q0sgcr416t.fsf@gmx.us
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/generated.out
M src/test/regress/expected/identity.out
M src/test/regress/expected/updatable_views.out
Make some sanity-check elogs more verbose
commit : fe051291550ab88267ce3104e9833925bf757393
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 23 Nov 2020 13:10:03 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 23 Nov 2020 13:10:03 -0300
A few sanity checks in funcapi.c were not mentioning all the possible
clauses for failure, confusing developers who fat-fingered catalog data
additions. Make the errors more detailed to avoid wasting time in
pinpointing mistakes.
Per complaint from Craig Ringer.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAMsr+YH7Kd87A3cU5m_wKo46HPQ46zFv5wesFNL0YWxkGhGv3g@mail.gmail.com
M src/backend/utils/fmgr/funcapi.c
Fix a few comments that referred to copy.c.
commit : 68b1a4877ef54f5ee6e05c64876655dd13d2c73b
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 23 Nov 2020 11:36:13 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 23 Nov 2020 11:36:13 +0200
Missed these in the previous commit.
M src/backend/libpq/pqcomm.c
M src/include/commands/trigger.h
M src/include/nodes/execnodes.h
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Split copy.c into four files.
commit : c532d15dddff14b01fe9ef1d465013cb8ef186df
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 23 Nov 2020 10:50:50 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 23 Nov 2020 10:50:50 +0200
Copy.c has grown really large. Split it into more manageable parts:
- copy.c now contains only a few functions that are common to COPY FROM
and COPY TO.
- copyto.c contains code for COPY TO.
- copyfrom.c contains code for initializing COPY FROM, and inserting the
tuples to the correct table.
- copyfromparse.c contains code for reading from the client/file/program,
and parsing the input text/CSV/binary format into tuples.
All of these parts are fairly complicated, and fairly independent of each
other. There is a patch being discussed to implement parallel COPY FROM,
which will add a lot of new code to the COPY FROM path, and another patch
which would allow INSERTs to use the same multi-insert machinery as COPY
FROM, both of which will require refactoring that code. With those two
patches, there's going to be a lot of code churn in copy.c anyway, so now
seems like a good time to do this refactoring.
The CopyStateData struct is also split. All the formatting options, like
FORMAT, QUOTE, ESCAPE, are put in a new CopyFormatOption struct, which
is used by both COPY FROM and TO. Other state data are kept in separate
CopyFromStateData and CopyToStateData structs.
Reviewed-by: Soumyadeep Chakraborty, Erik Rijkers, Vignesh C, Andres Freund
Discussion: https://www.postgresql.org/message-id/8e15b560-f387-7acc-ac90-763986617bfb%40iki.fi
M contrib/file_fdw/file_fdw.c
M src/backend/commands/Makefile
M src/backend/commands/copy.c
A src/backend/commands/copyfrom.c
A src/backend/commands/copyfromparse.c
A src/backend/commands/copyto.c
M src/backend/replication/logical/tablesync.c
M src/include/commands/copy.h
A src/include/commands/copyfrom_internal.h
Allow a multi-row INSERT to specify DEFAULTs for a generated column.
commit : 17958972fe3bb03454a4b53756b29d65dc285efa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 22 Nov 2020 15:48:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 22 Nov 2020 15:48:32 -0500
One can say "INSERT INTO tab(generated_col) VALUES (DEFAULT)" and not
draw an error. But the equivalent case for a multi-row VALUES list
always threw an error, even if one properly said DEFAULT in each row.
Fix that. While here, improve the test cases for nearby logic about
OVERRIDING SYSTEM/USER values.
Dean Rasheed
Discussion: https://postgr.es/m/9q0sgcr416t.fsf@gmx.us
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/generated.out
M src/test/regress/expected/identity.out
M src/test/regress/sql/generated.sql
M src/test/regress/sql/identity.sql
In geo_ops.c, represent infinite slope as Infinity, not DBL_MAX.
commit : 9fe649ea295f00baf6d0f0c1f9b0cb1298f64fb9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 21 Nov 2020 17:24:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 21 Nov 2020 17:24:07 -0500
Since we're assuming IEEE floats these days, there seems little
reason not to do this. It has the advantage that when the slope is
computed as infinite due to the presence of Inf coordinates, we get
saner behavior than before from line_construct(), and thence also
in some dependent operations such as finding the closest point.
Also fix line_construct() to special-case slope zero. The previous
coding got the right answer in most cases, but it could compute
C as NaN when the point has Inf coordinates.
Discussion: https://postgr.es/m/CAGf+fX70rWFOk5cd00uMfa__0yP+vtQg5ck7c2Onb-Yczp0URA@mail.gmail.com
M src/backend/utils/adt/geo_ops.c
M src/test/regress/expected/geometry.out
Fix FPeq() and friends to get the right answers for infinities.
commit : 8597a48d01b6cc0b09ff626253ac93c67e5516d5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 21 Nov 2020 16:46:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 21 Nov 2020 16:46:43 -0500
"FPeq(infinity, infinity)" returned false, on account of getting NaN
when it subtracts the two inputs. Fix that by adding a separate
check for exact equality.
FPle() and FPge() similarly got the wrong answer for two like-signed
infinities. In those cases, we can just rearrange the comparisons
to avoid potentially subtracting infinities.
While the sibling functions FPne() etc accidentally gave the right
answers even with the internal NaN results, it seems best to make
similar adjustments to them to avoid depending on this.
FPeq() has to be converted to an inline function to avoid double
evaluations of its arguments, and I did the same for the others
just for consistency.
In passing, make the handling of NaN cases in line_eq() and
point_eq_point() simpler and easier to reason about, and perhaps
faster.
This results in just one visible regression test change: slope()
now gives DBL_MAX for two inputs of (inf,1e300), which is consistent
with what it does for (1e300,inf), so that seems like a bug fix.
Discussion: https://postgr.es/m/CAGf+fX70rWFOk5cd00uMfa__0yP+vtQg5ck7c2Onb-Yczp0URA@mail.gmail.com
M src/backend/utils/adt/geo_ops.c
M src/include/utils/geo_decls.h
M src/test/regress/expected/geometry.out
Extend the geometric regression test cases a little.
commit : a45272b25d6fc8f96793623545fc1f836ac39d94
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 21 Nov 2020 16:34:22 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 21 Nov 2020 16:34:22 -0500
Add another edge-case value to "point_tbl", and add a test for
the line(point, point) function.
Some of the behaviors exposed here are wrong, but the idea of
committing this separately is to memorialize what we were getting,
and to allow easier inspection of the behavior changes caused by
upcoming patches.
Kyotaro Horiguchi (line() test added by me)
Discussion: https://postgr.es/m/CAGf+fX70rWFOk5cd00uMfa__0yP+vtQg5ck7c2Onb-Yczp0URA@mail.gmail.com
M src/test/regress/expected/create_index.out
M src/test/regress/expected/geometry.out
M src/test/regress/expected/point.out
M src/test/regress/sql/geometry.sql
M src/test/regress/sql/point.sql
Remove INSERT privilege check at table creation of CTAS and matview
commit : 878f3a19c6c8ff197e4a33f51d921a4abafcc494
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 21 Nov 2020 19:45:30 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 21 Nov 2020 19:45:30 +0900
As per discussion with Peter Eisentraunt, the SQL standard specifies
that any tuple insertion done as part of CREATE TABLE AS happens without
any extra ACL check, so it makes little sense to keep a check for INSERT
privileges when using WITH DATA. Materialized views are not part of the
standard, but similarly, this check can be confusing as this refers to
an access check on a table created within the same command as the one
that would insert data into this table.
This commit removes the INSERT privilege check for WITH DATA, the
default, that 846005e removed partially, but only for WITH NO DATA.
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/d049c272-9a47-d783-46b0-46665b011598@enterprisedb.com
M doc/src/sgml/ref/create_materialized_view.sgml
M doc/src/sgml/ref/create_table_as.sgml
M src/backend/commands/createas.c
M src/test/regress/expected/matview.out
M src/test/regress/expected/select_into.out
M src/test/regress/sql/matview.sql
M src/test/regress/sql/select_into.sql
doc: Improve tableoid description
commit : a47834db0fb70256e676862f1d96374a9e114d82
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 21 Nov 2020 08:26:20 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 21 Nov 2020 08:26:20 +0100
Mention that it's useful for determining table names for partitioned
tables as well as for those in inheritance hierarchies.
Author: Ian Lawrence Barwick <barwick@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAB8KJ=gFmBXP=P9htziOj+WM5PDAK4qc7iGQta+8kUh306kQnw@mail.gmail.com
M doc/src/sgml/ddl.sgml
Make pg_rewind test case more stable.
commit : c71f9a094b32770dcd34b9ba4909435e34583747
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 20 Nov 2020 16:08:58 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 20 Nov 2020 16:08:58 +0200
If replication is exceptionally slow for some reason, pg_rewind might run
before the test row has been replicated. Add an explicit wait for it.
Reported-by: Andres Freund
Discussion: https://www.postgresql.org/message-id/20201120003811.iknhqwatitw2vvxf%40alap3.anarazel.de
M src/bin/pg_rewind/t/007_standby_source.pl
Remove ability to independently select random number generator
commit : 16f96c74d48e65da23d28665103e2c4c9d3414cc
author : Magnus Hagander <magnus@hagander.net>
date : Fri, 20 Nov 2020 12:26:57 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Fri, 20 Nov 2020 12:26:57 +0100
Remove the ability to select random number generator independently from
SSL library. Instead, use the random number generator from the SSL
library (today only OpenSSL supported) if one is configured. If no SSL
library is configured, use the platform default (which means use
CryptoAPI on Win32 and /dev/urandom on Linux).
This also restructures pg_strong_random.c to have three clearly separate
sections, one for each implementation, with two functions in each,
instead of a scattered set of ifdefs throughout the whole file.
Author: Daniel Gustafsson, Magnus Hagander, Michael Paquier
Discussion: https://postgr.es/m/632623.1605460616@sss.pgh.pa.us
M configure
M configure.ac
M src/include/pg_config.h.in
M src/port/pg_strong_random.c
M src/tools/msvc/Solution.pm
Replace a macro by a function
commit : b5acf10cfc63ed1e0bc4eef466c0f4670a725ef3
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 20 Nov 2020 11:25:25 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 20 Nov 2020 11:25:25 +0100
Using a macro is ugly and not justified here.
Discussion: https://www.postgresql.org/message-id/flat/4ad69a4c-cc9b-0dfe-0352-8b1b0cd36c7b@2ndquadrant.com
M src/backend/access/transam/xlog.c
Add collation versions for FreeBSD.
commit : ca051d8b101d3205f1c5faf0d1da8a523ffc95a1
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 20 Nov 2020 21:49:57 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 20 Nov 2020 21:49:57 +1300
On FreeBSD 13, use querylocale() to read the current version of libc
collations. Similar to commits 352f6f2d for Windows and d5ac14f9 for
GNU/Linux.
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
M doc/src/sgml/charset.sgml
M src/backend/utils/adt/pg_locale.c
Emit log when restore_command succeeds but archived file faills to be restored.
commit : a4ef0329c27156dca81714854599e7d82becb17f
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 20 Nov 2020 15:42:47 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 20 Nov 2020 15:42:47 +0900
Previously, when restore_command claimed to succeed but failed to restore
the file with the right name, for example, due to mis-configuration of
restore_command, no log message was reported. Then the recovery failed
later with an error message not directly related to the issue.
This commit changes the recovery so that a log message is emitted in
this error case. This would enable us to investigate what happened in
this case more easily.
Author: Jeff Janes, Fujii Masao
Reviewed-by: Pavel Borisov, Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAMkU=1xkFs3Omp4JR4wMYWdam_KLuj6LXnTYfU8u3T0h=PLLMQ@mail.gmail.com
M src/backend/access/transam/xlogarchive.c
On macOS, use -isysroot in link steps as well as compile steps.
commit : 49407dc32a2931550e4ff1dea314b6a25afdfc35
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 20 Nov 2020 00:07:09 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 20 Nov 2020 00:07:09 -0500
We previously put the -isysroot switch only into CPPFLAGS, theorizing
that it was only needed to find the right copies of include files.
However, it seems that we also need to use it while linking programs,
to find the right stub ".tbd" files for libraries. We got away
without that up to now, but apparently that was mostly luck. It may
also be that failures are only observed when the Xcode version is
noticeably out of sync with the host macOS version; the case that's
prompting action right now is that builds fail when using latest Xcode
(12.2) on macOS Catalina, even though it's fine on Big Sur.
Hence, add -isysroot to LDFLAGS as well. (It seems that the more
common practice is to put it in CFLAGS, whence it'd be included at
both compile and link steps. However, we can't mess with CFLAGS in
the platform template file without confusing configure's logic for
choosing default CFLAGS.)
This should be back-patched, but first let's see if the buildfarm
likes it on HEAD.
Report and patch by James Hilliard (some cosmetic mods by me)
Discussion: https://postgr.es/m/20201120003314.20560-1-james.hilliard1@gmail.com
M configure
M configure.ac
M src/template/darwin
Remove undocumented IS [NOT] OF syntax.
commit : 926fa801ac9eb54c5275472271ec63a059904698
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Nov 2020 17:39:39 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Nov 2020 17:39:39 -0500
This feature was added a long time ago, in 7c1e67bd5 and eb121ba2c,
but never documented in any user-facing way. (Documentation added
in 6126d3e70 was commented out almost immediately, in 8272fc3f7.)
That's because, while this syntax is defined by SQL:99, our
implementation is only vaguely related to the standard's semantics.
The standard appears to intend a run-time not parse-time test, and
it definitely intends that the test should understand subtype
relationships.
No one has stepped up to fix that in the intervening years, but
people keep coming across the code and asking why it's not documented.
Let's just get rid of it: if anyone ever wants to make it work per
spec, they can easily recover whatever parts of this code are still
of value from our git history.
If there's anyone out there who's actually using this despite its
undocumented status, they can switch to using pg_typeof() instead,
eg. "pg_typeof(something) = 'mytype'::regtype". That gives
essentially the same semantics as what our IS OF code did.
(We didn't have that function last time this was discussed, or
we would have ripped out IS OF then.)
Discussion: https://postgr.es/m/CAKFQuwZ2pTc-DSkOiTfjauqLYkNREeNZvWmeg12Q-_69D+sYZA@mail.gmail.com
Discussion: https://postgr.es/m/BAY20-F23E9F2B4DAB3E4E88D3623F99B0@phx.gbl
Discussion: https://postgr.es/m/3E7CF81D.1000203@joeconway.com
M doc/src/sgml/func.sgml
M src/backend/catalog/sql_features.txt
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_expr.c
M src/include/nodes/parsenodes.h
M src/test/regress/expected/arrays.out
M src/test/regress/expected/domain.out
M src/test/regress/expected/with.out
M src/test/regress/sql/arrays.sql
M src/test/regress/sql/domain.sql
M src/test/regress/sql/with.sql
Further fixes for CREATE TABLE LIKE: cope with self-referential FKs.
commit : 97390fe8a6e96a153e59b0180f4303acaeb75b84
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Nov 2020 15:03:17 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Nov 2020 15:03:17 -0500
Commit 502898192 was too careless about the order of execution of the
additional ALTER TABLE operations generated by expandTableLikeClause.
It just stuck them all at the end, which seems okay for most purposes.
But it falls down in the case where LIKE is importing a primary key
or unique index and the outer CREATE TABLE includes a FOREIGN KEY
constraint that needs to depend on that index. Weird as that is,
it used to work, so we ought to keep it working.
To fix, make parse_utilcmd.c insert LIKE clauses between index-creation
and FK-creation commands in the transformed list of commands, and change
utility.c so that the commands generated by expandTableLikeClause are
executed immediately not at the end. One could imagine scenarios where
this wouldn't work either; but currently expandTableLikeClause only
makes column default expressions, CHECK constraints, and indexes, and
this ordering seems fine for those.
Per bug #16730 from Sofoklis Papasofokli. Like the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/16730-b902f7e6e0276b30@postgresql.org
M src/backend/parser/parse_utilcmd.c
M src/backend/tcop/utility.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Rename object in test to avoid conflict
commit : afaccbba787d5f1470c44ddd61c9ddaaad19e27e
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 19 Nov 2020 11:34:54 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 19 Nov 2020 11:34:54 +0100
In 01e658fa74cb7e3292448f6663b549135958003b, the hash_func test
creates a type t1, but apparently a test running in parallel might
also use that name, depending on timing. Rename the type to avoid the
issue.
M src/test/regress/expected/hash_func.out
M src/test/regress/sql/hash_func.sql
Hash support for row types
commit : 01e658fa74cb7e3292448f6663b549135958003b
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 19 Nov 2020 09:24:37 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 19 Nov 2020 09:24:37 +0100
Add hash functions for the record type as well as a hash operator
family and operator class for the record type. This enables all the
hash functionality for the record type such as hash-based plans for
UNION/INTERSECT/EXCEPT DISTINCT, recursive queries using UNION
DISTINCT, hash joins, and hash partitioning.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/38eccd35-4e2d-6767-1b3c-dada1eac3124%402ndquadrant.com
M doc/src/sgml/queries.sgml
M src/backend/utils/adt/rowtypes.c
M src/backend/utils/cache/lsyscache.c
M src/backend/utils/cache/typcache.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/hash_func.out
M src/test/regress/expected/join.out
M src/test/regress/expected/union.out
M src/test/regress/expected/with.out
M src/test/regress/sql/hash_func.sql
M src/test/regress/sql/join.sql
M src/test/regress/sql/union.sql
M src/test/regress/sql/with.sql
Add BarrierArriveAndDetachExceptLast().
commit : 7888b0999488511e4266f2134053fa3a6505a155
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 19 Nov 2020 18:09:50 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 19 Nov 2020 18:09:50 +1300
Provide a way for one process to continue the remaining phases of a
(previously) parallel computation alone. Later patches will use this to
extend Parallel Hash Join.
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKG%2BA6ftXPz4oe92%2Bx8Er%2BxpGZqto70-Q_ERwRaSyA%3DafNg%40mail.gmail.com
M src/backend/storage/ipc/barrier.c
M src/include/storage/barrier.h
Improve failure detection with array parsing in pg_dump
commit : 13b58f8934e6252868231c3493d49b8c2b363e5d
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 19 Nov 2020 10:36:08 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 19 Nov 2020 10:36:08 +0900
Similarly to 3636efa, the checks done in pg_dump when parsing array
values from catalogs have been too lax. Under memory pressure, it could
be possible, though very unlikely, to finish with dumps that miss some
data like:
- Statistics for indexes
- Run-time configuration of functions
- Configuration of extensions
- Publication list for a subscription
No backpatch is done as this is not going to be a problem in practice.
For example, if an OOM causes an array parsing to fail, a follow-up code
path of pg_dump would most likely complain with an allocation failure
due to the memory pressure.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/20201111061319.GE2276@paquier.xyz
M src/bin/pg_dump/pg_dump.c
Relax lock level for setting PGPROC->statusFlags
commit : 27838981be9de35d54ffcdc6cc06b5d5ea9e0cee
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 18 Nov 2020 13:24:22 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 18 Nov 2020 13:24:22 -0300
We don't actually need a lock to set PGPROC->statusFlags itself; what we
do need is a shared lock on either XidGenLock or ProcArrayLock in order to
ensure MyProc->pgxactoff keeps still while we modify the mirror array in
ProcGlobal->statusFlags. Some places were using an exclusive lock for
that, which is excessive. Relax those to use shared lock only.
procarray.c has a couple of places with somewhat brittle assumptions
about PGPROC changes: ProcArrayEndTransaction uses only shared lock, so
it's permissible to change MyProc only. On the other hand,
ProcArrayEndTransactionInternal also changes other procs, so it must
hold exclusive lock. Add asserts to ensure those assumptions continue
to hold.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20201117155501.GA13805@alvherre.pgsql
M src/backend/commands/vacuum.c
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/lmgr/deadlock.c
M src/include/storage/proc.h
Skip allocating hash table in EXPLAIN-only mode.
commit : 2cccb627f10d5dd4c21806fa251e4fb61eadd28c
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 18 Nov 2020 12:39:15 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 18 Nov 2020 12:39:15 +0200
Author: Alexey Bashtanov
Discussion: https://www.postgresql.org/message-id/36823f65-050d-ae24-aa4d-a37726998240%40imap.cc
M src/backend/executor/nodeAgg.c
docs: Clarify that signal functions have no feedback.
commit : dfab7f2f94812c6c3a62cfde0ec7f9e447132e64
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 18 Nov 2020 10:28:50 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 18 Nov 2020 10:28:50 +0200
Bug #16652 complains that pg_reload_conf() returned true, even though
the configuration file contained errors. That's the way pg_reload_conf()
works, by design, but the documentation wasn't very clear on it. Clarify
that a 'true' return value only means that the signal was sent
successfully. Also add links to the system views that can be used to
check the configuration files for errors.
David G. Johnston, with some rewording by me.
Discussion: https://www.postgresql.org/message-id/CAKFQuwax6GxhUQEes0D045UtXG-fBraM39_6UMd5JyR5K1HWCQ%40mail.gmail.com
M doc/src/sgml/func.sgml
Add more tests for hashing and hash-based plans
commit : 6dd8b00807871bfd0c3ce8e18fd1f6ba1a71fbf9
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 18 Nov 2020 07:58:37 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 18 Nov 2020 07:58:37 +0100
- Test hashing of an array of a non-hashable element type.
- Test UNION [DISTINCT] with hash- and sort-based plans. (Previously,
only INTERSECT and EXCEPT where tested there.)
- Test UNION [DISTINCT] with a non-hashable column type. This
currently reverts to a sort-based plan even if enable_hashagg is on.
- Test UNION/INTERSECT/EXCEPT hash- and sort-based plans with arrays
as column types. Also test an array with a non-hashable element
type.
- Test UNION/INTERSECT/EXCEPT similarly with row types as column
types. Currently, this uses only sort-based plans because there is
no hashing support for row types.
- Add a test case that shows that recursive queries using UNION
[DISTINCT] require hashable column types.
- Add a currently failing test that uses UNION DISTINCT in a
cycle-detection use case using row types as column types.
Discussion: https://www.postgresql.org/message-id/flat/38eccd35-4e2d-6767-1b3c-dada1eac3124%402ndquadrant.com
M src/test/regress/expected/hash_func.out
M src/test/regress/expected/union.out
M src/test/regress/expected/with.out
M src/test/regress/sql/hash_func.sql
M src/test/regress/sql/union.sql
M src/test/regress/sql/with.sql
Add tab completion for CREATE [OR REPLACE] TRIGGER in psql
commit : bf0aa7c4b83bcf3116c5a3c191bbc677ab3beb59
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 18 Nov 2020 14:01:53 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 18 Nov 2020 14:01:53 +0900
92bf7e2 has added support for this grammar.
Author: Noriyoshi Shinoda
Discussion: https://postgr.es/m/TU4PR8401MB115244623CF4724DCA0D507FEEE30@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM
M src/bin/psql/tab-complete.c
Deprecate nbtree's BTP_HAS_GARBAGE flag.
commit : cf2acaf4dcb5e20204dcec4d698cb4478af533e7
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 17 Nov 2020 09:45:56 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 17 Nov 2020 09:45:56 -0800
Streamline handling of the various strategies that we have to avoid a
page split in nbtinsert.c. When it looks like a leaf page is about to
overflow, we now perform deleting LP_DEAD items and deduplication in one
central place. This greatly simplifies _bt_findinsertloc().
This has an independently useful consequence: nbtree no longer relies on
the BTP_HAS_GARBAGE page level flag/hint for anything important. We
still set and unset the flag in the same way as before, but it's no
longer treated as a gating condition when considering if we should check
for already-set LP_DEAD bits. This happens at the point where the page
looks like it might have to be split anyway, so simply checking the
LP_DEAD bits in passing is practically free. This avoids missing
LP_DEAD bits just because the page-level hint is unset, which is
probably reasonably common (e.g. it happens when VACUUM unsets the
page-level flag without actually removing index tuples whose LP_DEAD-bit
was set recently, after the VACUUM operation began but before it reached
the leaf page in question).
Note that this isn't a big behavioral change compared to PostgreSQL 13.
We were already checking for set LP_DEAD bits regardless of whether the
BTP_HAS_GARBAGE page level flag was set before we considered doing a
deduplication pass. This commit only goes slightly further by doing the
same check for all indexes, even indexes where deduplication won't be
performed.
We don't completely remove the BTP_HAS_GARBAGE flag. We still rely on
it as a gating condition with pg_upgrade'd indexes from before B-tree
version 4/PostgreSQL 12. That makes sense because we sometimes have to
make a choice among pages full of duplicates when inserting a tuple with
pre version 4 indexes. It probably still pays to avoid accessing the
line pointer array of a page there, since it won't yet be clear whether
we'll insert on to the page in question at all, let alone split it as a
result.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Victor Yegorov <vyegorov@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wz%3DYpc1PDdk8OVJDChGJBjT06%3DA0Mbv9HyTLCsOknGcUFg%40mail.gmail.com
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/access/nbtree/nbtxlog.c
M src/include/access/nbtree.h
indexcmds.c: reorder function prototypes
commit : 7684b6fbed3a0770a0d8fdcbb5cf8b61394de691
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 17 Nov 2020 11:37:36 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 17 Nov 2020 11:37:36 -0300
... out of an overabundance of neatnikism, perhaps.
M src/backend/commands/indexcmds.c
nbtree: Rename nbtinsert.c variables for consistency.
commit : a034f8b60c6e9afd75f42dc420a76cb9d3875fcb
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 17 Nov 2020 09:01:14 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 17 Nov 2020 09:01:14 -0800
Stop naming special area/opaque pointer variables 'lpageop' in contexts
where it doesn't make sense. This is a holdover from a time when logic
that performs tasks that are now spread across _bt_insertonpg(),
_bt_findinsertloc(), and _bt_split() was more centralized. 'lpageop'
denotes "left page", which doesn't make sense outside of contexts in
which there isn't also a right page.
Also acquire page flag variables up front within _bt_insertonpg(). This
makes it closer to _bt_split() following refactoring commit bc3087b626d.
This allows the page split and retail insert paths to both make use of
the same variables.
M src/backend/access/nbtree/nbtinsert.c
Fix 'skip-empty-xacts' option in test_decoding for streaming mode.
commit : 9653f24ad8307f393de51e0a64d9b10a49efa6e3
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 17 Nov 2020 12:14:53 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 17 Nov 2020 12:14:53 +0530
In streaming mode, the transaction can be decoded in multiple streams and
those streams can be interleaved with streams of other transactions. So,
we can't remember the transaction's write status in the logical decoding
context because that might get changed due to some other transactions and
lead to wrong answers for 'skip-empty-xacts' option. We decided to keep
each transaction's write status in the ReorderBufferTxn to avoid
interleaved streams changing the status of some unrelated transactions.
Diagnosed-by: Amit Kapila
Author: Dilip Kumar
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAA4eK1LR7=XNM_TLmpZMFuV8ZQpoxkem--NZJYf8YXmesbvwLA@mail.gmail.com
M contrib/test_decoding/expected/concurrent_stream.out
M contrib/test_decoding/specs/concurrent_stream.spec
M contrib/test_decoding/test_decoding.c
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h
M src/tools/pgindent/typedefs.list
Don't Insert() a VFD entry until it's fully built.
commit : 2bd49b493a52f0a21bc3fc51c355963f99ac1a4e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Nov 2020 20:32:35 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Nov 2020 20:32:35 -0500
Otherwise, if FDDEBUG is enabled, the debugging output fails because
it tries to read the fileName, which isn't set up yet (and should in
fact always be NULL).
AFAICT, this has been wrong since Berkeley. Before 96bf88d52,
it would accidentally fail to crash on platforms where snprintf()
is forgiving about being passed a NULL pointer for %s; but the
file name intended to be included in the debug output wouldn't
ever have shown up.
Report and fix by Greg Nancarrow. Although this is only visibly
broken in custom-made builds, it still seems worth back-patching
to all supported branches, as the FDDEBUG code is pretty useless
as it stands.
Discussion: https://postgr.es/m/CAJcOf-cUDgm9qYtC_B6XrC6MktMPNRby2p61EtSGZKnfotMArw@mail.gmail.com
M src/backend/storage/file/fd.c
Rename PGPROC->vacuumFlags to statusFlags
commit : cd9c1b3e197a9b53b840dcc87eb41b04d601a5f9
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 16 Nov 2020 19:42:55 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 16 Nov 2020 19:42:55 -0300
With more flags associated to a PGPROC entry that are not related to
vacuum (currently existing or planned), the name "statusFlags" describes
its purpose better.
(The same is done to the mirroring PROC_HDR->vacuumFlags.)
No functional changes in this commit.
This was suggested first by Hari Babu Kommi in [1] and then by Michael
Paquier at [2].
[1] https://postgr.es/m/CAJrrPGcsDC-oy1AhqH0JkXYa0Z2AgbuXzHPpByLoBGMxfOZMEQ@mail.gmail.com
[2] https://postgr.es/m/20200820060929.GB3730@paquier.xyz
Author: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20201116182446.qcg3o6szo2zookyr@localhost
M src/backend/access/transam/twophase.c
M src/backend/commands/vacuum.c
M src/backend/postmaster/autovacuum.c
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/lmgr/deadlock.c
M src/backend/storage/lmgr/proc.c
M src/include/storage/proc.h
Do not return NULL for error cases in satisfies_hash_partition().
commit : 4025e6c46620048804467d2ad29d31aa9ba50387
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Nov 2020 16:39:59 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Nov 2020 16:39:59 -0500
Since this function is used as a CHECK constraint condition,
returning NULL is tantamount to returning TRUE, which would have the
effect of letting in a row that doesn't satisfy the hash condition.
Admittedly, the cases for which this is done should be unreachable
in practice, but that doesn't make it any less a bad idea. It also
seems like a dartboard was used to decide which error cases should
throw errors as opposed to returning NULL.
For the checks for NULL input values, I just switched it to returning
false. There's some argument that an error would be better; but the
case really should be can't-happen in a generated hash constraint,
so it's likely not worth more code for.
For the parent-relation-open-failure case, it seems like we might
as well let relation_open throw an error, instead of having an
impossible-to-diagnose constraint failure.
Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/24067.1605134819@sss.pgh.pa.us
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/hash_part.out
Use "true" not "TRUE" in one ICU function call.
commit : ad84ecc98d7e2ad81567094b8a6910b5078927a7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Nov 2020 15:16:39 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Nov 2020 15:16:39 -0500
This was evidently missed in commit 6337865f3, which generally did
s/TRUE/true/ everywhere. It escaped notice up to now because ICU
versions before ICU 68 provided definitions of "TRUE" and "FALSE"
regardless. With ICU 68, it fails to compile.
Per report from Condor. Back-patch to v11 where 6337865f3 came in.
(I've not tested v10, where this call originated, but I imagine
it's fine since we defined TRUE in c.h back then.)
Discussion: https://postgr.es/m/7a6f3336165bfe3ca66abcda7966f9d0@stz-bg.com
M src/backend/commands/collationcmds.c
doc: update bgwriter description
commit : 62b50ad698f8a789f92177ad72639f99d94d7d78
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 16 Nov 2020 13:13:43 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 16 Nov 2020 13:13:43 -0500
This clarifies exactly what the bgwriter does, which should help with
tuning.
Reported-by: Chris Wilson
Discussion: https://postgr.es/m/160399562040.7809.7335281028960123489@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
doc: clarify how to find pg_type_d.h in the install tree
commit : 8ad06bcbe136360296a804da7d7631a78fd15e70
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 16 Nov 2020 12:36:17 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 16 Nov 2020 12:36:17 -0500
Followup to patch 152ed04799.
Reported-by: Alvaro Herrera
Discussion: https://postgr.es/m/20201112202900.GA28098@alvherre.pgsql
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
Remove unused and deprecated strategy numbers from BRIN code
commit : d93ccdea1d447ab7e8c48797153ce58b051f1a37
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 16 Nov 2020 17:25:41 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 16 Nov 2020 17:25:41 +0100
These were dead code.
Discussion: https://www.postgresql.org/message-id/flat/20201027032511.GF9241@telsasoft.com
M src/backend/access/brin/brin_inclusion.c
doc: improve wording of the need for analyze of exp. indexes
commit : 481f9dc3dc2fc779af3ab2d58216a1c2b970b20f
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 16 Nov 2020 10:26:17 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 16 Nov 2020 10:26:17 -0500
This is a followup commit on 3370207986.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20201112211143.GL30691@telsasoft.com
Backpatch-through: 9.5
M doc/src/sgml/ref/create_index.sgml
Fix typo
commit : bcbd7713324798d5a74ca640c6cf6c439bda22e4
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 16 Nov 2020 10:54:11 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 16 Nov 2020 10:54:11 -0300
Introduced in 90fdc259866e; backpatch to 12.
Author: Erik Rijkers <er@xs4all.nl>
Discussion: https://postgr.es/m/e92b3fba98a0c0f7afc0a2a37e765954@xs4all.nl
M doc/src/sgml/ref/create_index.sgml
Normalize comment in empty grammar rules
commit : 5664b7be5b2c2e756372e7dde7c187a11a5ab701
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 16 Nov 2020 11:54:52 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 16 Nov 2020 11:54:52 +0100
Change lower case /* empty */ to /* EMPTY */ for consistency with the
majority.
Discussion: https://www.postgresql.org/message-id/flat/e9eed669-e32d-6919-fed4-acc0daea857b%40enterprisedb.com
M src/backend/parser/gram.y
Remove code handling removed deprecated containment operators
commit : 591d282e8d3e0448ec1339c6b066e10953f040a2
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 16 Nov 2020 11:45:00 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 16 Nov 2020 11:45:00 +0100
This removes the code that was there for handling the operators
removed by 2f70fdb0644c32c4154236c2b5c241bec92eac5e.
Discussion: https://www.postgresql.org/message-id/flat/20201027032511.GF9241@telsasoft.com
M src/backend/access/gist/gistproc.c
Make the standby server promptly handle interrupt signals.
commit : 2945a488a3426838223a637cd326ec71fff61954
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 16 Nov 2020 18:27:51 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 16 Nov 2020 18:27:51 +0900
This commit changes the startup process in the standby server so that
it handles the interrupt signals after waiting for wal_retrieve_retry_interval
on the latch and resetting it, before entering another wait on the latch.
This change causes the standby server to promptly handle interrupt signals.
Otherwise, previously, there was the case where the standby needs to
wait extra five seconds to shutdown when the shutdown request arrived
while the startup process was waiting for wal_retrieve_retry_interval
on the latch.
Author: Fujii Masao, but implementation idea is from Soumyadeep Chakraborty
Reviewed-by: Soumyadeep Chakraborty
Discussion: https://postgr.es/m/9d7e6ab0-8a53-ddb9-63cd-289bcb25fe0e@oss.nttdata.com
M src/backend/access/transam/xlog.c
Reword 'simple comparison' => 'inequality' in pgtrgm.sgml
commit : 7adb8feb0f69fd9c79078c6cea984bb4bff1c841
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 16 Nov 2020 09:09:07 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 16 Nov 2020 09:09:07 +0300
Discussion: https://postgr.es/m/CAMkU%3D1zxfA8_MGBW6sJMj54p8nPoe4bMb5LoG-rMYZVPq4j08Q%40mail.gmail.com
Author: Jeff Janes
M doc/src/sgml/pgtrgm.sgml
Relax INSERT privilege requirement for CTAS and matviews WITH NO DATA
commit : 846005e4f3829c3eafe1f8441b80ff90657d0a29
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 16 Nov 2020 11:52:40 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 16 Nov 2020 11:52:40 +0900
When specified, WITH NO DATA does not insert any data into the relation
created, so skip checking for the insert permissions. With WITH DATA or
WITH NO DATA, it is always required for the user to have CREATE
privileges on the schema targeted for the relation.
Note that plain CREATE TABLE AS or CREATE MATERIALIZED VIEW queries have
begun to work accidentally without INSERT privilege checks as of
874fe3ae, while using EXECUTE or EXPLAIN ANALYZE would fail with the ACL
check, so this makes the behavior for all the command flavors consistent
with each other. This is arguably a bug fix, but there have been no
complaints about the current behavior either so stable branches are not
changed.
While on it, document properly the privileges requirements for each
commands with more tests for all the scenarios possible, and avoid a
useless bulk-insert allocation when using WITH NO DATA.
Author: Bharath Rupireddy
Reviewed-by: Anastasia Lubennikova, Michael Paquier
Discussion: https://postgr.es/m/CALj2ACWc3N8j0_9nMPz9wcAUnVcdKHzFdDZJ3hVFNEbqtcyG9w@mail.gmail.com
M doc/src/sgml/ref/create_materialized_view.sgml
M doc/src/sgml/ref/create_table_as.sgml
M src/backend/commands/createas.c
M src/test/regress/expected/matview.out
M src/test/regress/expected/select_into.out
M src/test/regress/sql/matview.sql
M src/test/regress/sql/select_into.sql
Fix fuzzy thinking about amcanmulticol versus amcaninclude.
commit : 29d29d652f0be47dc42fa9d667dee5b8e1baa18a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 15 Nov 2020 16:10:48 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 15 Nov 2020 16:10:48 -0500
These flags should be independent: in particular an index AM should
be able to say that it supports include columns without necessarily
supporting multiple key columns. The included-columns patch got
this wrong, possibly aided by the fact that it didn't bother to
update the documentation.
While here, clarify some text about amcanreturn, which was a little
vague about what should happen when amcanreturn reports that only
some of the index columns are returnable.
Noted while reviewing the SP-GiST included-columns patch, which
quite incorrectly (and unsafely) changed SP-GiST to claim
amcanmulticol = true as a workaround for this bug.
Backpatch to v11 where included columns were introduced.
M doc/src/sgml/indexam.sgml
M src/backend/commands/indexcmds.c
nbtree: Demote incomplete split "can't happen" error.
commit : 46cf3c72c36fef8c623b4b97d8720321d8ad7507
author : Peter Geoghegan <pg@bowt.ie>
date : Sun, 15 Nov 2020 11:53:37 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Sun, 15 Nov 2020 11:53:37 -0800
Only a basic logic bug in a _bt_insertonpg() caller could lead to a
violation of this invariant (index corruption won't do it). A "can't
happen" error seems inappropriate (it is arbitrary at best).
Demote the error to a simple assertion. This matches similar nearby
sanity checks.
M src/backend/access/nbtree/nbtinsert.c
Suppress "warning: variable 'collcollate' set but not used".
commit : ff94205787244ace8ce63e3d9eb98338c59ac215
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 15 Nov 2020 12:39:49 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 15 Nov 2020 12:39:49 -0500
Buildfarm members that lack both HAVE_LOCALE_T and USE_ICU have been
complaining about pg_newlocale_from_collation's collcollate variable.
This is evidently fallout from commit 7d1297df0, which removed the
only usage outside those two #ifdef'd code paths. Mark the variable
pg_attribute_unused(), like its sibling collctype, which has been that
way for a long time.
M src/backend/utils/adt/pg_locale.c
Fix timing issue in pg_rewind test.
commit : 39f9f04b57844143c7f510cc901501057103bfe4
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sun, 15 Nov 2020 17:09:31 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sun, 15 Nov 2020 17:09:31 +0200
The test inserts a row in primary server, waits until the insertion has
been replicated to a cascaded standby, and checks that it's visible there
by querying the cascaded standby. In order for that to work reliably, the
test needs to wait until the insertion WAL record has been fully replayed.
This should fix the occasional buildfarm failures. Diagnosis by Tom Lane.
Discussion: https://www.postgresql.org/message-id/606796.1605424022@sss.pgh.pa.us
M src/bin/pg_rewind/t/007_standby_source.pl
Add missing 'the' to pgtrgm.sgml
commit : b7edb5d96e7c70fc4e522e4ca1f893e815fa5502
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 15 Nov 2020 14:01:22 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 15 Nov 2020 14:01:22 +0300
Author: Erik Rijkers
Discussion: https://postgr.es/m/159fba265fe7c37334640fdc0444cc4b%40xs4all.nl
M doc/src/sgml/pgtrgm.sgml
Fix alphabetic ordering in typedefs.list
commit : 8afa3795a7a29f018769f16f6af936439847bd66
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 15 Nov 2020 08:56:31 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 15 Nov 2020 08:56:31 +0300
M src/tools/pgindent/typedefs.list
Handle equality operator in contrib/pg_trgm
commit : 935f6666502250abde8615bc7805a6e5aa05a066
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 15 Nov 2020 08:52:12 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 15 Nov 2020 08:52:12 +0300
Obviously, in order to equality operator be satisfiable, target string must
contain all the trigrams of the search string. On this base, we implement
equality operator in GiST/GIN indexes with recheck.
Discussion: https://postgr.es/m/CAOBaU_YWwtT7tdggtROacjdOdeYHCz-tmSwuC-j-TOG-g97J0w%40mail.gmail.com
Author: Julien Rouhaud
Reviewed-by: Tom Lane, Alexander Korotkov, Georgios Kokolatos, Erik Rijkers
M contrib/pg_trgm/Makefile
M contrib/pg_trgm/expected/pg_trgm.out
A contrib/pg_trgm/pg_trgm–1.5–1.6.sql
M contrib/pg_trgm/pg_trgm.control
M contrib/pg_trgm/sql/pg_trgm.sql
M contrib/pg_trgm/trgm.h
M contrib/pg_trgm/trgm_gin.c
M contrib/pg_trgm/trgm_gist.c
M doc/src/sgml/pgtrgm.sgml
Provide the OR REPLACE option for CREATE TRIGGER.
commit : 92bf7e2d027466d750b4ac5b026f6f4ac29be881
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 14 Nov 2020 17:05:34 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 14 Nov 2020 17:05:34 -0500
This is mostly straightforward. However, we disallow replacing
constraint triggers or changing the is-constraint property; perhaps
that can be added later, but the complexity versus benefit tradeoff
doesn't look very good.
Also, no special thought is taken here for whether replacing an
existing trigger should result in changes to queued-but-not-fired
trigger actions. We just document that if you're surprised by the
results, too bad, don't do that. (Note that any such pending trigger
activity would have to be within the current session.)
Takamichi Osumi, reviewed at various times by Surafel Temesgen,
Peter Smith, and myself
Discussion: https://postgr.es/m/0DDF369B45A1B44B8A687ED43F06557C010BC362@G01JPEXMBYT03
M doc/src/sgml/ref/create_trigger.sgml
M src/backend/catalog/index.c
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/parser/gram.y
M src/include/nodes/parsenodes.h
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Doc: improve partitioning discussion in ddl.sgml.
commit : dbca94510c9e564708a10a1259f6f1d8ad09862c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 14 Nov 2020 13:09:53 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 14 Nov 2020 13:09:53 -0500
This started with the intent to explain that range upper bounds
are exclusive, which previously you could only find out by reading
the CREATE TABLE man page. But I soon found that section 5.11
really could stand a fair amount of editorial attention. It's
apparently been revised several times without much concern for
overall flow, nor careful copy-editing.
Back-patch to v11, which is as far as the patch goes easily.
Per gripe from Edson Richter. Thanks to David Johnston for review.
Discussion: https://postgr.es/m/DM6PR13MB3988736CF8F5DC5720440231CFE60@DM6PR13MB3988.namprd13.prod.outlook.com
M doc/src/sgml/ddl.sgml
Fix some typos
commit : 788dd0b839fc9f2c85caf86014105afdb60ed5e3
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 14 Nov 2020 11:43:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 14 Nov 2020 11:43:10 +0900
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/C36ADFDF-D09A-4EE5-B186-CB46C3653F4C@yesql.se
M contrib/amcheck/verify_heapam.c
M contrib/sepgsql/selinux.c
M src/backend/catalog/objectaddress.c
M src/backend/catalog/pg_type.c
M src/backend/storage/ipc/procarray.c
M src/backend/utils/mb/Unicode/UCS_to_EUC_JP.pl
M src/include/access/gist.h
M src/include/replication/logicalproto.h
Support negative indexes in split_part().
commit : ec0294fb2c8287fd673c57701cdcf6c6396b2f60
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 13 Nov 2020 13:49:48 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 13 Nov 2020 13:49:48 -0500
This provides a handy way to get, say, the last field of the string.
Use of a negative index in this way has precedent in the nearby
left() and right() functions.
The implementation scans the string twice when N < -1, but it seems
likely that N = -1 will be the huge majority of actual use cases,
so I'm not really excited about adding complexity to avoid that.
Nikhil Benesch, reviewed by Jacob Champion; cosmetic tweakage by me
Discussion: https://postgr.es/m/cbb7f861-6162-3a51-9823-97bc3aa0b638@gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/varlena.c
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
Remove another test that doesn't work on Windows.
commit : 3bf44303b954269558c30458df8d02fb99c40c67
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 13 Nov 2020 13:41:58 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 13 Nov 2020 13:41:58 +0200
Apparently double-quotes are not allowed in filenames on Windows, either.
Per buildfarm.
M src/bin/pg_rewind/t/003_extrafiles.pl
doc: clarify where to find pg_type_d.h (PG 11+) and pg_type.h
commit : 152ed04799e31e9abe6c2ed8afc7405b892b7d86
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 12 Nov 2020 15:13:02 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 12 Nov 2020 15:13:02 -0500
These files are in compiled directories and install directories.
Reported-by: e.indrupskaya@postgrespro.ru
Discussion: https://postgr.es/m/160379609706.24746.7506163279454026608@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
docs: mention that expression indexes need analyze
commit : 337020798674bcdfefe93e2277d98458da7266cd
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 12 Nov 2020 15:00:44 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 12 Nov 2020 15:00:44 -0500
Expression indexes can't benefit from pre-computed statistics on
columns.
Reported-by: Nikolay Samokhvalov
Discussion: https://postgr.es/m/CANNMO++5rw9RDA=p40iMVbMNPaW6O=S0AFzTU=KpYHRpCd1voA@mail.gmail.com
Author: Nikolay Samokhvalov, modified
Backpatch-through: 9.5
M doc/src/sgml/ref/create_index.sgml
change wire protocol data type for history file content
commit : 66a8f090485e3e897a4804121fdbe856cba72d70
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 12 Nov 2020 14:08:59 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 12 Nov 2020 14:08:59 -0500
This was marked as BYTEA, but is more like TEXT, which is how we already
pass the history timeline file name. Internally, we don't do any
encoding or bytea escape handling, but TEXT seems closest. This should
cause no behavioral change.
Reported-by: Brar Piening
Discussion: https://postgr.es/m/6a1b9cd9-17e3-df67-be55-86102af6bdf5@gmx.de
Backpatch-through: master
M doc/src/sgml/protocol.sgml
M src/backend/replication/walsender.c
Remove tests that don't work on Windows.
commit : 951dfa34f41bb851602216d3ada5f80135370e4f
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 12 Nov 2020 19:18:34 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 12 Nov 2020 19:18:34 +0200
On Windows, a filename cannot contain backslashes, because a backslash
is used directory separator. Remove tests I added in commit 9c4f5192f
that tried to do that. We could perhaps use a SKIP block to only skip
them on Windows, but I'm not sure how exactly to formulate that, so just
remove the tests to make the buildfarm green again.
Per buildfarm.
M src/bin/pg_rewind/t/003_extrafiles.pl
pg_trgm: fix crash in 2-item picksplit
commit : 3836d4b643369310dc2f98ad311baeac307590d7
author : Andrew Gierth <rhodiumtoad@postgresql.org>
date : Thu, 12 Nov 2020 14:34:37 +0000
committer: Andrew Gierth <rhodiumtoad@postgresql.org>
date : Thu, 12 Nov 2020 14:34:37 +0000
Whether from size overflow in gistSplit or from secondary splits,
picksplit is (rarely) called with exactly two items to split.
Formerly, due to special-case handling of the last item, this would
lead to access to an uninitialized cache entry; prior to PG 13 this
might have been harmless or at worst led to an incorrect union datum,
but in 13 onwards it can cause a backend crash from using an
uninitialized pointer.
Repair by removing the special case, which was deemed not to have been
appropriate anyway. Backpatch all the way, because this bug has
existed since pg_trgm was added.
Per report on IRC from user "ftzdomino". Analysis and testing by me,
patch from Alexander Korotkov.
Discussion: https://postgr.es/m/87k0usfdxg.fsf@news-spur.riddles.org.uk
M contrib/pg_trgm/trgm_gist.c
Allow pg_rewind to use a standby server as the source system.
commit : 9c4f5192f69ed16c99e0d079f0b5faebd7bad212
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 12 Nov 2020 14:52:24 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 12 Nov 2020 14:52:24 +0200
Using a hot standby server as the source has not been possible, because
pg_rewind creates a temporary table in the source system, to hold the
list of file ranges that need to be fetched. Refactor it to queue up the
file fetch requests in pg_rewind's memory, so that the temporary table
is no longer needed.
Also update the logic to compute 'minRecoveryPoint' correctly, when the
source is a standby server.
Reviewed-by: Kyotaro Horiguchi, Soumyadeep Chakraborty
Discussion: https://www.postgresql.org/message-id/0c5b3783-af52-3ee5-f8fa-6e794061f70d%40iki.fi
M doc/src/sgml/ref/pg_rewind.sgml
M src/bin/pg_rewind/libpq_source.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/t/003_extrafiles.pl
A src/bin/pg_rewind/t/007_standby_source.pl
Clean up optional rules in grammar
commit : 1b2b19f7584b7c0025aa40862cd38c79d340be7d
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 12 Nov 2020 07:48:36 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 12 Nov 2020 07:48:36 +0100
Various rules for optional keywords contained unnecessary rules and
type declarations. Remove those, thus making the output a tiny bit
smaller.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/e9eed669-e32d-6919-fed4-acc0daea857b%40enterprisedb.com
M src/backend/parser/gram.y
Fix typo in contrib/pg_trgm/pg_trgm--1.4--1.5.sql
commit : 3e8ec5b1408a15fb41cada2635163a598d8607d1
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 12 Nov 2020 08:55:09 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 12 Nov 2020 08:55:09 +0300
Backpatch-through: 13
M contrib/pg_trgm/pg_trgm–1.4–1.5.sql
Use standard SIGHUP and SIGTERM handlers in walreceiver.
commit : 1a2ae7c50fb4ab17a8d38e755fc871ebc3a5845d
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 12 Nov 2020 13:25:23 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 12 Nov 2020 13:25:23 +0900
Commit 1e53fe0e70 changed background processes so that they use
standard SIGHUP handler. Like that, this commit makes walreceiver
use standard SIGHUP and SIGTERM handlers, to simplify the code.
As the side effect of this commit, walreceiver can wake up and process
the configuration files promptly when receiving SIGHUP. Because the
standard SIGHUP handler sets the latch. On the other hand, previously
there could be a time lag between the receipt of SIGHUP and
the process of configuration files since the dedicated handler didn't
set the latch.
Author: Bharath Rupireddy, tweaked by Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACXPorUqePswDtOeM_s82v9RW32E1fYmOPZ5NuE+TWKj_A@mail.gmail.com
M src/backend/replication/walreceiver.c
Fix name of the macro for getting signature length trgm_gist.c
commit : 8ca8208ace4e67a9149946814d35d7cf18ff60d0
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 12 Nov 2020 06:19:16 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 12 Nov 2020 06:19:16 +0300
911e702077 has introduced the opclass parameters including signature length
for a set of GiST opclasses. Due to copy-pasting, macro for getting the
signature length in trgm_gist.c was named LTREE_GET_ASIGLEN(). Fix that by
renaming this macro to just GET_SIGLEN().
Backpatch-through: 13
M contrib/pg_trgm/trgm_gist.c
pg_stat_statements: track number of rows processed by REFRESH MATERIALIZED VIEW.
commit : b62e6056a05c60ce9edf93e87e1487ae50245a04
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 12 Nov 2020 11:26:55 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 12 Nov 2020 11:26:55 +0900
Commit 6023b7ea71 allowed pg_stat_statements to track the number
of rows retrieved or affected by some utility commands including
CREATE MATERIALIZED VIEW. However it did not track the rowcount
of REFRESH MATERIALIZED VIEW. This commit allows pg_stat_statements
to track that.
To track that, this commit changes the query completion for
REFRESH MATERIALIZED VIEW so that it saves the rowcount. But note that
the rowcount is still not displayed in the command completion tag output.
That is, the display_rowcount flag of CMDTAG_REFRESH_MATERIALIZED_VIEW
command tag is left false in cmdtaglist.h. Otherwise, the change of
completion tag output might break applications using it.
Author: Katsuragi Yuta, Seino Yuki
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/71f6bc72f8bbaa06e701f8bd2562c347@oss.nttdata.com
Discussion: https://postgr.es/m/aadbfba9-e4bb-9531-6b3a-d13c31c8f4fe@oss.nttdata.com
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
M src/backend/commands/matview.c
Remove useless SHA256 initialization when not using backup manifests
commit : 03f9cd93eaa5e3385b075bb1a17d2adde3e23edd
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 12 Nov 2020 10:56:33 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 12 Nov 2020 10:56:33 +0900
Attempting to take a base backup with Postgres linking to a build of
OpenSSL with FIPS enabled currently fails with or even without a backup
manifest requested because of this mandatory SHA256 initialization used
for the manifest file itself. However, there is no need to do this
initialization at all if backup manifests are not needed because there
is no data to append to the manifest.
Note that being able to use backup manifests with OpenSSL+FIPS requires
a switch of the SHA2 implementation to use EVP, which would cause an ABI
breakage so this cannot be backpatched to 13 as it has been already
released, but at least avoiding this SHA256 initialization gives users
the possibility to take a base backup even when specifying --no-manifest
with pg_basebackup.
Author: Michael Paquier
Discussion: https://postgr.es/m/20201110020014.GE1887@paquier.xyz
Backpatch-through: 13
M src/backend/replication/backup_manifest.c
Remove duplicate code in brin_memtuple_initialize
commit : 42c63ab6e2c39c2fbfc70fa659b0cf5692178b77
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 11 Nov 2020 18:37:36 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 11 Nov 2020 18:37:36 +0100
Commit 8bf74967dab moved some of the code from brin_new_memtuple to
brin_memtuple_initialize, but this resulted in some of the code being
duplicate. Fix by removing the duplicate lines and backpatch to 10.
Author: Tomas Vondra
Backpatch-through: 10
Discussion: https://postgr.es/m/5eb50c97-9a8e-b691-8c40-1b2a55611c4c%40enterprisedb.com
M src/backend/access/brin/brin_tuple.c
Fix some stray whitespace in parser files
commit : 0af302af40669d9bc6de7a202e4723fee39d9376
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 11 Nov 2020 17:37:18 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 11 Nov 2020 17:37:18 +0100
M src/backend/bootstrap/bootscanner.l
M src/backend/parser/gram.y
M src/backend/utils/adt/jsonpath_gram.y
M src/interfaces/ecpg/preproc/ecpg.trailer
Remove line missed in previous commit
commit : 4e3ffd5981536f93787ae6d81bc179da0acc423e
author : Magnus Hagander <magnus@hagander.net>
date : Wed, 11 Nov 2020 16:51:36 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Wed, 11 Nov 2020 16:51:36 +0100
M src/tools/msvc/vcregress.pl
Remove vacuumdb --analyze-in-stages from pg_upgrade tests
commit : 1e12a495b4749c7dc8e38e347554f2b24297f4ba
author : Magnus Hagander <magnus@hagander.net>
date : Wed, 11 Nov 2020 16:47:13 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Wed, 11 Nov 2020 16:47:13 +0100
This step was only there to test the script when we generated those, but
commit 8f113698b6 removed those scripts, so it's not needed anymore.
Reported-By: Peter Eisentraut
Discussion: https://postgr.es/m/ea403f46-2b33-a7de-618e-9cab35a698c8@enterprisedb.com
M src/bin/pg_upgrade/test.sh
M src/tools/msvc/vcregress.pl
Add pg_nodiscard decorations to some functions
commit : a378ba49a5cfec073d885f4943e1c7e4b83abd3b
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 17 Oct 2020 08:38:39 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 17 Oct 2020 08:38:39 +0200
Especially for the list API such as lappend() forgetting to assign the
return value is a common problem.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/e3753562-99cd-b65f-5aca-687dfd1ec2fc@2ndquadrant.com
M src/include/nodes/pg_list.h
M src/include/utils/palloc.h
Add pg_nodiscard function declaration specifier
commit : c617aea6acdd9057a14d91ff294905b1b5509348
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 17 Oct 2020 08:38:39 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 17 Oct 2020 08:38:39 +0200
pg_nodiscard means the compiler should warn if the result of a
function call is ignored. The name "nodiscard" is chosen in alignment
with (possibly future) C and C++ standards. It maps to the GCC
attribute warn_unused_result.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/e3753562-99cd-b65f-5aca-687dfd1ec2fc@2ndquadrant.com
M src/include/c.h
Fix cases of discarding result from list API functions
commit : c77f6f50e4187bed38d1d36ae16b0c248e286d15
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 17 Oct 2020 08:38:39 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 17 Oct 2020 08:38:39 +0200
Two cases violated list APIs by throwing away the return value. While
the code was technically correct, it relied on internal knowledge of
the list implementation, and the code wasn't really gaining anything
that way. It is planned to make this a compiler warning in the
future, so just fix these cases by assigning the return value
properly.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/e3753562-99cd-b65f-5aca-687dfd1ec2fc@2ndquadrant.com
M src/backend/commands/lockcmds.c
M src/backend/parser/analyze.c
Fix and simplify some usages of TimestampDifference().
commit : ec29427ce2a451e7fef7a22de6db8147d8a80994
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 Nov 2020 22:51:18 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 Nov 2020 22:51:18 -0500
Introduce TimestampDifferenceMilliseconds() to simplify callers
that would rather have the difference in milliseconds, instead of
the select()-oriented seconds-and-microseconds format. This gets
rid of at least one integer division per call, and it eliminates
some apparently-easy-to-mess-up arithmetic.
Two of these call sites were in fact wrong:
* pg_prewarm's autoprewarm_main() forgot to multiply the seconds
by 1000, thus ending up with a delay 1000X shorter than intended.
That doesn't quite make it a busy-wait, but close.
* postgres_fdw's pgfdw_get_cleanup_result() thought it needed to compute
microseconds not milliseconds, thus ending up with a delay 1000X longer
than intended. Somebody along the way had noticed this problem but
misdiagnosed the cause, and imposed an ad-hoc 60-second limit rather
than fixing the units. This was relatively harmless in context, because
we don't care that much about exactly how long this delay is; still,
it's wrong.
There are a few more callers of TimestampDifference() that don't
have a direct need for seconds-and-microseconds, but can't use
TimestampDifferenceMilliseconds() either because they do need
microsecond precision or because they might possibly deal with
intervals long enough to overflow 32-bit milliseconds. It might be
worth inventing another API to improve that, but that seems outside
the scope of this patch; so those callers are untouched here.
Given the fact that we are fixing some bugs, and the likelihood
that future patches might want to back-patch code that uses this
new API, back-patch to all supported branches.
Alexey Kondratov and Tom Lane
Discussion: https://postgr.es/m/3b1c053a21c07c1ed5e00be3b2b855ef@postgrespro.ru
M contrib/pg_prewarm/autoprewarm.c
M contrib/postgres_fdw/connection.c
M src/backend/access/transam/xlog.c
M src/backend/replication/walreceiverfuncs.c
M src/backend/replication/walsender.c
M src/backend/utils/adt/timestamp.c
M src/include/utils/timestamp.h
doc: fix spelling "connction" to "connection"
commit : b8b6a0124b21edd4aed24217a8e5ecf621ccd661
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 10 Nov 2020 19:18:35 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 10 Nov 2020 19:18:35 -0500
Was wrong in commit 1a9388bd0f.
Reported-by: Tom Lane, Justin Pryzby
Discussion: https://postgr.es/m/20201102063333.GE22691@telsasoft.com
Backpatch-through: 9.5
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_receivewal.sgml
Work around cross-version-upgrade issues created by commit 9e38c2bb5.
commit : 97f73a978fc1aca59c6ad765548ce0096d95a923
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 Nov 2020 18:32:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 Nov 2020 18:32:36 -0500
Summarily changing the STYPE of regression-test aggregates that
depend on array_append or array_cat is an issue for the buildfarm's
cross-version-upgrade tests, because those aggregates (as defined
in the back branches) now won't load into HEAD. Although this seems
like only a minimal risk for genuine user-defined aggregates, we
need to do something for the buildfarm. Hence, adjust the aggregate
definitions, in both HEAD and the back branches.
Discussion: https://postgr.es/m/1401824.1604537031@sss.pgh.pa.us
Discussion: https://postgr.es/m/E1kaQ2c-0005lx-Eg@gemulon.postgresql.org
M src/test/regress/expected/polymorphism.out
M src/test/regress/sql/polymorphism.sql
pg_rewind: Fix thinko in parsing target WAL.
commit : 72d172743e52f31bb874e3bbc07544b30bf0bb51
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 10 Nov 2020 19:25:46 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 10 Nov 2020 19:25:46 +0200
It's entirely possible to see WAL for a relation that doesn't exist in
the target anymore. That happens when the relation was dropped later.
The refactoring in commit eb00f1d4b broke that case, by sanity-checking
the file type in the target before checking the flag forwhether it
exists there at all.
I noticed this during manual testing. Modify the 001_basic.pl test so
that it covers this case.
M src/bin/pg_rewind/filemap.c
M src/bin/pg_rewind/t/001_basic.pl
Fix out of date comment
commit : 3f16cb505d1d734674da2a2cbf2104ceae22f9b4
author : Magnus Hagander <magnus@hagander.net>
date : Tue, 10 Nov 2020 13:14:09 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Tue, 10 Nov 2020 13:14:09 +0100
M src/backend/postmaster/postmaster.c
Remove -o option to postmaster
commit : d2e4bf688e16f95d180b68b7cbb7881b12434dba
author : Magnus Hagander <magnus@hagander.net>
date : Tue, 10 Nov 2020 13:08:21 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Tue, 10 Nov 2020 13:08:21 +0100
This option was declared obsolete many years ago.
Reviewed-By: Tom Lane
Discussion: https://postgr.es/m/CABUevEyOE=9CQwZm2j=vwP5+6OLCSoxn9pBjK8gyRdkTzMfqtQ@mail.gmail.com
M doc/src/sgml/ref/postgres-ref.sgml
M src/backend/main/main.c
M src/backend/postmaster/postmaster.c
M src/backend/tcop/postgres.c
jit: Add support for LLVM 12.
commit : 6c57f2ed16e1281067e3dde7e006e2d1cab72116
author : Andres Freund <andres@anarazel.de>
date : Mon, 9 Nov 2020 20:01:33 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 9 Nov 2020 20:01:33 -0800
LLVM 12, to be released in a few months, made some breaking changes to
the Orc JIT interface. OrcV2 eventually will make it easier to support
features like concurrent JIT compilation, but this commit only allows
to compile against LLVM 12.
This commit is a bit bigger than desirable. That partially is because
the V2 interface is more granular than V1 interface, but also because
I chose to make some minor changes to < LLVM 12 code to keep the code
somewhat readable.
The LLVM 12 support will need to be backpatched. I plan to do so after
the patch stewed on the buildfarm for a few days.
Author: Andres Freund
Discussion: https://postgr.es/m/20201016011244.pmyvr3ee2gbzplq4@alap3.anarazel.de
M src/backend/jit/llvm/llvmjit.c
M src/tools/pgindent/typedefs.list
Doc: clarify data type behavior of COALESCE and NULLIF.
commit : 24b83a5082541bdb1b333b7fcbe92f194128595c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 12:02:24 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 12:02:24 -0500
After studying the code, NULLIF is a lot more subtle than you might
have guessed.
Discussion: https://postgr.es/m/160486028730.25500.15740897403028593550@wrigleys.postgresql.org
M doc/src/sgml/func.sgml
M doc/src/sgml/typeconv.sgml
Remove ineffective heapam CHECK_FOR_INTERRUPTS().
commit : 180cf876d457c260291af070d883c5db5033cb3a
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 9 Nov 2020 09:00:12 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 9 Nov 2020 09:00:12 -0800
Remove a CHECK_FOR_INTERRUPTS() call that could never actually handle an
interrupt. We always have a heap page buffer lock at this point.
Having a useless CHECK_FOR_INTERRUPTS() call is harmless but misleading.
It is probably possible to work around the immediate problem by moving
the CHECK_FOR_INTERRUPTS() to before the heap page buffer lock is
acquired. That isn't enough to make the function responsive to
interrupts, though. The index AM caller will still hold an exclusive
buffer lock of its own.
M src/backend/access/heap/heapam.c
Ignore attempts to \gset into specially treated variables.
commit : 098fb00799ffb026ff12c64bd21635f963cfc609
author : Noah Misch <noah@leadboat.com>
date : Mon, 9 Nov 2020 07:32:09 -0800
committer: Noah Misch <noah@leadboat.com>
date : Mon, 9 Nov 2020 07:32:09 -0800
If an interactive psql session used \gset when querying a compromised
server, the attacker could execute arbitrary code as the operating
system account running psql. Using a prefix not found among specially
treated variables, e.g. every lowercase string, precluded the attack.
Fix by issuing a warning and setting no variable for the column in
question. Users wanting the old behavior can use a prefix and then a
meta-command like "\set HISTSIZE :prefix_HISTSIZE". Back-patch to 9.5
(all supported versions).
Reviewed by Robert Haas. Reported by Nick Cleaton.
Security: CVE-2020-25696
M src/bin/psql/common.c
M src/bin/psql/variables.c
M src/bin/psql/variables.h
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
In security-restricted operations, block enqueue of at-commit user code.
commit : 0c3185e963d9f9dd0608214f7d732b84aa0888fe
author : Noah Misch <noah@leadboat.com>
date : Mon, 9 Nov 2020 07:32:09 -0800
committer: Noah Misch <noah@leadboat.com>
date : Mon, 9 Nov 2020 07:32:09 -0800
Specifically, this blocks DECLARE ... WITH HOLD and firing of deferred
triggers within index expressions and materialized view queries. An
attacker having permission to create non-temp objects in at least one
schema could execute arbitrary SQL functions under the identity of the
bootstrap superuser. One can work around the vulnerability by disabling
autovacuum and not manually running ANALYZE, CLUSTER, REINDEX, CREATE
INDEX, VACUUM FULL, or REFRESH MATERIALIZED VIEW. (Don't restore from
pg_dump, since it runs some of those commands.) Plain VACUUM (without
FULL) is safe, and all commands are fine when a trusted user owns the
target object. Performance may degrade quickly under this workaround,
however. Back-patch to 9.5 (all supported versions).
Reviewed by Robert Haas. Reported by Etienne Stalmans.
Security: CVE-2020-25695
M contrib/postgres_fdw/connection.c
M src/backend/access/transam/xact.c
M src/backend/commands/portalcmds.c
M src/backend/commands/trigger.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Remove analyze_new_cluster script from pg_upgrade
commit : 8f113698b63b15a4e0a4b15d3ee37238c1d1821d
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 9 Nov 2020 12:14:59 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 9 Nov 2020 12:14:59 +0100
Since this script just runs vacuumdb anyway, remove the script and
replace the instructions to run it with instructions to run vacuumdb
directly.
Reviewed-By: Michael Paquier
Discussion: https://postgr.es/m/CABUevEwg5LDFzthhxzSj7sZGMiVsZe0VVNbzzwTQOHJ=rN7+5A@mail.gmail.com
M src/bin/pg_upgrade/.gitignore
M src/bin/pg_upgrade/Makefile
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/test.sh
M src/tools/msvc/vcregress.pl
Remove incorrect %s in string
commit : 7e84dd212040529de1811c7c97c996cc4e392015
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 9 Nov 2020 10:36:49 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 9 Nov 2020 10:36:49 +0100
Appears to have been a copy/paste error in the original commit that
moved the messages to fe_utils/.
Author: Tang, Haiying <tanghy.fnst@cn.fujitsu.com>
Backpatch-through: 13
Discussion: https://postgr.es/m/3321cbcea76d4d2c8320a05c19b9304a@G08CNEXMBPEKD05.g08.fujitsu.local
M src/fe_utils/cancel.c
doc: Add note about pg_settings and customized options into catalogs.sgml.
commit : ef60de67ebde6dfd1ea09b4d08166736bf05698c
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 9 Nov 2020 15:10:26 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 9 Nov 2020 15:10:26 +0900
The pg_settings view does not display customized options until
the extension module that defines them has been loaded. This commit
add the note about that behavior, into the docs.
Author: John Naylor
Reviewed-by: Tom Lane, Fujii Masao
Discussion: https://postgr.es/m/CAFBsxsGsBZsG=cLM0Op5HFb2Ks6SzJrOc_eRO_jcKSNuqFRKnQ@mail.gmail.com
M doc/src/sgml/catalogs.sgml
Fix parsePGArray() error checking in pg_dump.
commit : 3636efa11945af64479995609762b2626c6c319a
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 9 Nov 2020 16:01:51 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 9 Nov 2020 16:01:51 +1300
Coverity complained about a defect in commit 257836a7:
Calling "parsePGArray" without checking return value (as is
done elsewhere 11 out of 13 times).
Fix, and also check for empty strings explicitly (NULL as represented by
PQgetvalue()). That worked correctly before only because parsePGArray()
happens to set *nitems = 0 when it fails on an empty string. Also
convert a sanity check assertion to an error to be more paranoid, and
pgindent a nearby line.
Reported-by: Michael Paquier <michael@paquier.xyz>
M src/bin/pg_dump/pg_dump.c
In INSERT/UPDATE, use the table's real tuple descriptor as target.
commit : 8b39345a9d9abd1c371f8458301a227d0abb93b2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Nov 2020 13:08:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Nov 2020 13:08:36 -0500
This back-patches commit 20d3fe900 into the v12 and v13 branches.
At the time I thought that commit was not fixing any observable
bug, but Bertrand Drouvot showed otherwise: adding a dropped column
to the previously-considered scenario crashes v12 and v13, unless the
dropped column happens to be an integer. That is, of course, because
the tupdesc we derive from the plan output tlist fails to describe
the dropped column accurately, so that we'll do the wrong thing with
a tuple in which that column isn't NULL.
There is no bug in pre-v12 branches because they already did use
the table's real tuple descriptor for any trigger-returned tuple.
It seems that this set of bugs can be blamed on the changes that
removed es_trig_tuple_slot, though I've not attempted to pin that
down precisely.
Although there's no code change needed in HEAD, update the test case
to include a dropped column there too.
Discussion: https://postgr.es/m/db5d97c8-f48a-51e2-7b08-b73d5434d425@amazon.com
Discussion: https://postgr.es/m/16644-5da7ef98a7ac4545@postgresql.org
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Fix assertion in collation version lookup.
commit : d50e3b1f8d763d51080ca4f6cbaa6818a6472292
author : Thomas Munro <tmunro@postgresql.org>
date : Sun, 8 Nov 2020 20:43:45 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Sun, 8 Nov 2020 20:43:45 +1300
Commit 257836a7 included an assertion that a version lookup routine is
not trying to look up "C" or "POSIX", but that case is reachable with
the user-facing SQL function pg_collation_actual_version(). Remove the
assertion.
M src/backend/utils/adt/pg_locale.c
Fix test for error message change
commit : 8cff66d309b90e03f1e76494375960f35eff6424
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 8 Nov 2020 07:48:18 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 8 Nov 2020 07:48:18 +0100
fix for 6be725e701611660b36642de9ff1d665a1eb24f5
M src/test/ssl/t/002_scram.pl
Improve nbtree README's LP_DEAD section.
commit : 5a2f154a2ecaf545000a3ff3cdbadc76ae1df30a
author : Peter Geoghegan <pg@bowt.ie>
date : Sat, 7 Nov 2020 18:51:12 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Sat, 7 Nov 2020 18:51:12 -0800
The description of how LP_DEAD bit setting by index scans works
following commit 2ed5b87f was rather unclear. Clean that up a bit.
Also refer to LP_DEAD bit setting within _bt_check_unique() at the start
of the same section. This mechanism may actually be more important than
the generic kill_prior_tuple mechanism that the section focuses on, so
it at least deserves to be mentioned in passing.
M src/backend/access/nbtree/README
Message style improvements
commit : 52eec1c53aa6a7df1683fba79078793f1d0eba42
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 7 Nov 2020 19:33:43 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 7 Nov 2020 19:33:43 -0300
* Avoid pointlessly highlighting that an index vacuum was executed by a
parallel worker; user doesn't care.
* Don't give the impression that a non-concurrent reindex of an invalid
index on a TOAST table would work, because it wouldn't.
* Add a "translator:" comment for a mysterious message.
Discussion: https://postgr.es/m/20201107034943.GA16596@alvherre.pgsql
Reviewed-by: Michael Paquier <michael@paquier.xyz>
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/indexcmds.c
M src/backend/libpq/be-secure-openssl.c
Fix redundant error messages in client tools
commit : 6be725e701611660b36642de9ff1d665a1eb24f5
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Nov 2020 22:15:52 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Nov 2020 22:15:52 +0100
A few client tools duplicate error messages already provided by libpq.
Discussion: https://www.postgresql.org/message-id/flat/3e937641-88a1-e697-612e-99bba4b8e5e4%40enterprisedb.com
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/psql/startup.c
Avoid re-using output variables in new ecpg test case.
commit : eed4356fad84b0fd6e3caa49c7006f401159ac9a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Nov 2020 16:25:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Nov 2020 16:25:42 -0500
The buildfarm thinks this leads to memory stomps, though annoyingly
I can't duplicate that here. The existing code in strings.pgc is
doing something that doesn't seem to be sanctioned at all really
by the documentation, but I'm disinclined to try to make that nicer
right now. Let's just declare some more output variables in hopes
of working around it.
M src/interfaces/ecpg/test/expected/preproc-strings.c
M src/interfaces/ecpg/test/expected/preproc-strings.stderr
M src/interfaces/ecpg/test/preproc/strings.h
M src/interfaces/ecpg/test/preproc/strings.pgc
Fix ecpg's mishandling of B'...' and X'...' literals.
commit : 1e3868ab3bef5cfa0f4d44a6937a880be7a3a482
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Nov 2020 15:03:44 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Nov 2020 15:03:44 -0500
These were broken in multiple ways:
* The xbstart and xhstart lexer actions neglected to set
"state_before_str_start" before transitioning to the xb/xh states,
thus possibly resulting in "internal error: unreachable state" later.
* The test for valid string contents at the end of xb state was flat out
wrong, as it accounted incorrectly for the "b" prefix that the xbstart
action had injected. Meanwhile, the xh state had no such check at all.
* The generated literal value failed to include any quote marks.
* The grammar did the wrong thing anyway, typically ignoring the
literal value and emitting something else, since BCONST and XCONST
tokens were handled randomly differently from SCONST tokens.
The first of these problems is evidently an oversight in commit
7f380c59f, but the others seem to be very ancient. The lack of
complaints shows that ECPG users aren't using these syntaxes much
(although I do vaguely remember one previous complaint).
As written, this patch is dependent on 7f380c59f, so it can't go
back further than v13. Given the shortage of complaints, I'm not
excited about adapting the patch to prior branches.
Report and patch by Shenhao Wang (test case adjusted by me)
Discussion: https://postgr.es/m/d6402f1bacb74ecba22ef715dbba17fd@G08CNEXMBPEKD06.g08.fujitsu.local
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/ecpg.type
M src/interfaces/ecpg/preproc/parse.pl
M src/interfaces/ecpg/preproc/pgc.l
M src/interfaces/ecpg/test/expected/preproc-strings.c
M src/interfaces/ecpg/test/expected/preproc-strings.stderr
M src/interfaces/ecpg/test/expected/preproc-strings.stdout
M src/interfaces/ecpg/test/preproc/strings.pgc
Move catalog index declarations
commit : bdc4edbea6fc847f806e1e7118d730e159512bfc
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Nov 2020 12:11:40 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Nov 2020 12:11:40 +0100
Move the system catalog index declarations from catalog/indexing.h to
the respective parent tables' catalog/pg_*.h files. The original
reason for having it split was that the old genbki system produced the
output in the order of the catalog files it read, so all the indexing
stuff needed to come separately. But this is no longer the case, and
keeping it together makes more sense.
Reviewed-by: John Naylor <john.naylor@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/c7cc82d6-f976-75d6-2e3e-b03d2cab26bb@2ndquadrant.com
M contrib/dblink/dblink.c
M contrib/sepgsql/database.c
M contrib/sepgsql/label.c
M contrib/sepgsql/proc.c
M contrib/sepgsql/relation.c
M contrib/sepgsql/schema.c
M doc/src/sgml/bki.sgml
M src/backend/catalog/Makefile
M src/backend/catalog/catalog.c
M src/backend/catalog/objectaddress.c
M src/backend/commands/aggregatecmds.c
M src/backend/commands/tablecmds.c
M src/backend/partitioning/partdesc.c
M src/backend/replication/logical/message.c
M src/backend/rewrite/rewriteDefine.c
M src/backend/utils/adt/enum.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/cache/evtcache.c
M src/backend/utils/cache/relfilenodemap.c
M src/backend/utils/cache/syscache.c
M src/backend/utils/cache/ts_cache.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/init/postinit.c
M src/include/catalog/duplicate_oids
M src/include/catalog/genbki.h
M src/include/catalog/indexing.h
M src/include/catalog/pg_aggregate.h
M src/include/catalog/pg_am.h
M src/include/catalog/pg_amop.h
M src/include/catalog/pg_amproc.h
M src/include/catalog/pg_attrdef.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_auth_members.h
M src/include/catalog/pg_authid.h
M src/include/catalog/pg_cast.h
M src/include/catalog/pg_class.h
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_conversion.h
M src/include/catalog/pg_database.h
M src/include/catalog/pg_db_role_setting.h
M src/include/catalog/pg_default_acl.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_description.h
M src/include/catalog/pg_enum.h
M src/include/catalog/pg_event_trigger.h
M src/include/catalog/pg_extension.h
M src/include/catalog/pg_foreign_data_wrapper.h
M src/include/catalog/pg_foreign_server.h
M src/include/catalog/pg_foreign_table.h
M src/include/catalog/pg_index.h
M src/include/catalog/pg_inherits.h
M src/include/catalog/pg_init_privs.h
M src/include/catalog/pg_language.h
M src/include/catalog/pg_largeobject.h
M src/include/catalog/pg_largeobject_metadata.h
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_opclass.h
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_opfamily.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_policy.h
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_publication.h
M src/include/catalog/pg_publication_rel.h
M src/include/catalog/pg_range.h
M src/include/catalog/pg_replication_origin.h
M src/include/catalog/pg_rewrite.h
M src/include/catalog/pg_seclabel.h
M src/include/catalog/pg_sequence.h
M src/include/catalog/pg_shdepend.h
M src/include/catalog/pg_shdescription.h
M src/include/catalog/pg_shseclabel.h
M src/include/catalog/pg_statistic.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/catalog/pg_subscription.h
M src/include/catalog/pg_subscription_rel.h
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_transform.h
M src/include/catalog/pg_trigger.h
M src/include/catalog/pg_ts_config.h
M src/include/catalog/pg_ts_config_map.h
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_ts_parser.h
M src/include/catalog/pg_ts_template.h
M src/include/catalog/pg_type.h
M src/include/catalog/pg_user_mapping.h
M src/include/catalog/renumber_oids.pl
M src/include/catalog/unused_oids
M src/tools/msvc/Solution.pm
Move catalog toast table declarations
commit : b4c9695e79f79d39d55853acac4c1493ac70a529
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Nov 2020 12:11:40 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Nov 2020 12:11:40 +0100
Move the system catalog toast table declarations from
catalog/toasting.h to the respective parent tables' catalog/pg_*.h
files. The original reason for having it split was that the old
genbki system produced the output in the order of the catalog files it
read, so all the toasting stuff needed to come separately. But this
is no longer the case, and keeping it together makes more sense.
Reviewed-by: John Naylor <john.naylor@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/c7cc82d6-f976-75d6-2e3e-b03d2cab26bb@2ndquadrant.com
M doc/src/sgml/bki.sgml
M src/backend/catalog/Makefile
M src/backend/catalog/catalog.c
M src/backend/catalog/toasting.c
M src/include/catalog/duplicate_oids
M src/include/catalog/genbki.h
M src/include/catalog/pg_aggregate.h
M src/include/catalog/pg_attrdef.h
M src/include/catalog/pg_authid.h
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_database.h
M src/include/catalog/pg_db_role_setting.h
M src/include/catalog/pg_default_acl.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_description.h
M src/include/catalog/pg_event_trigger.h
M src/include/catalog/pg_extension.h
M src/include/catalog/pg_foreign_data_wrapper.h
M src/include/catalog/pg_foreign_server.h
M src/include/catalog/pg_foreign_table.h
M src/include/catalog/pg_init_privs.h
M src/include/catalog/pg_language.h
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_policy.h
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_replication_origin.h
M src/include/catalog/pg_rewrite.h
M src/include/catalog/pg_seclabel.h
M src/include/catalog/pg_shdescription.h
M src/include/catalog/pg_shseclabel.h
M src/include/catalog/pg_statistic.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/catalog/pg_subscription.h
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_trigger.h
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_type.h
M src/include/catalog/pg_user_mapping.h
M src/include/catalog/renumber_oids.pl
M src/include/catalog/toasting.h
M src/include/catalog/unused_oids
M src/tools/msvc/Solution.pm
Plug memory leak in index_get_partition
commit : 623644f02cbde7ad3812b201bd36213a206c3341
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 6 Nov 2020 22:52:16 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 6 Nov 2020 22:52:16 -0300
The list of indexes was being leaked when asked for an index that
doesn't have an index partition in the table partition. Not a common
case admittedly --and in most cases where it occurs, caller throws an
error anyway-- but worth fixing for cleanliness and in case any
third-party code is calling this function.
While at it, remove use of lfirst_oid() to obtain a value we already
have.
Author: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20201105203606.GF22691@telsasoft.com
M src/backend/catalog/partition.c
Add GUC_LIST_INPUT and GUC_LIST_QUOTE to unix_socket_directories
commit : a05dbf477b0ef173adb1ae5d004cbeb0cf400b66
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 7 Nov 2020 10:30:22 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 7 Nov 2020 10:30:22 +0900
This should have been done in the initial commit that made
unix_socket_directories a list as of c9b0cbe. This change allows to
support correctly the case of ALTER SYSTEM, where it is possible to
specify multiple paths as a list, like the following pattern where
flattening is applied to each item:
ALTER SYSTEM SET unix_socket_directories = '/path1', '/path2';
Any parameters specified in postgresql.conf are parsed the same way, so
there is no compatibility change. pg_dump has a hardcoded list of
parameters marked with GUC_LIST_QUOTE, that gets its routine update.
These are reordered alphabetically for clarity.
Author: Ian Lawrence Barwick
Reviewed-by: Peter Eisentraunt, Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/CAB8KJ=iMOtNY6_sUwV=LQVCJ2zgYHBDyNzVfvE5GN3WQ3v9kQg@mail.gmail.com
M src/backend/utils/misc/guc.c
M src/bin/pg_dump/dumputils.c
Fix minor issues with new unicode {de,re}composition code
commit : ceaeac54f75f0117a7b3caa2f8ecba5cfac18ee5
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 7 Nov 2020 10:15:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 7 Nov 2020 10:15:58 +0900
The table generation script would incorrectly complain in the
recomposition sorting when matching code points. This would not have
caused the generation of an incorrect table. Note that this condition
is not reachable yet, but could have been reached with future updates.
pg_bswap.h does not need to be included in the frontend.x
Author: John Naylor
Discussion: https://postgr.es/m/CAFBsxsGWmExpvv=61vtDKCs7+kBbhkwBDL2Ph9CacziFKnV_yw@mail.gmail.com
M src/common/unicode/generate-unicode_norm_table.pl
M src/common/unicode_norm.c
Properly detoast data in brin_form_tuple
commit : 7577dd84807a808f295dbfb0191540750ff98002
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 7 Nov 2020 00:39:19 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 7 Nov 2020 00:39:19 +0100
brin_form_tuple failed to consider the values may be toasted, inserting
the toast pointer into the index. This may easily result in index
corruption, as the toast data may be deleted and cleaned up by vacuum.
The cleanup however does not care about indexes, leaving invalid toast
pointers behind, which triggers errors like this:
ERROR: missing chunk number 0 for toast value 16433 in pg_toast_16426
A less severe consequence are inconsistent failures due to the index row
being too large, depending on whether brin_form_tuple operated on plain
or toasted version of the row. For example
CREATE TABLE t (val TEXT);
INSERT INTO t VALUES ('... long value ...')
CREATE INDEX idx ON t USING brin (val);
would likely succeed, as the row would likely include toast pointer.
Switching the order of INSERT and CREATE INDEX would likely fail:
ERROR: index row size 8712 exceeds maximum 8152 for index "idx"
because this happens before the row values are toasted.
The bug exists since PostgreSQL 9.5 where BRIN indexes were introduced.
So backpatch all the way back.
Author: Tomas Vondra
Reviewed-by: Alvaro Herrera
Backpatch-through: 9.5
Discussion: https://postgr.es/m/20201001184133.oq5uq75sb45pu3aw@development
Discussion: https://postgr.es/m/20201104010544.zexj52mlldagzowv%40development
M src/backend/access/brin/brin_tuple.c
M src/test/regress/expected/brin.out
M src/test/regress/sql/brin.sql
Revert "Accept relations of any kind in LOCK TABLE".
commit : eeda7f6338095701cfe1ba3da37070508efe019e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 16:17:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 16:17:56 -0500
Revert 59ab4ac32, as well as the followup fix 33862cb9c, in all
branches. We need to think a bit harder about what the behavior
of LOCK TABLE on views should be, and there's no time for that
before next week's releases. We'll take another crack at this
later.
Discussion: https://postgr.es/m/16703-e348f58aab3cf6cc@postgresql.org
M doc/src/sgml/ref/lock.sgml
M src/backend/commands/lockcmds.c
M src/test/regress/expected/lock.out
M src/test/regress/sql/lock.sql
Revert "pg_dump: Lock all relations, not just plain tables".
commit : d3adaabaf7d555ec8bb1d83c43f72e79f1bf0b7d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 15:48:04 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 15:48:04 -0500
Revert 403a3d91c, as well as the followup fix 7f4235032, in all
branches. We need to think a bit harder about what the behavior
of LOCK TABLE on views should be, and there's no time for that
before next week's releases. We'll take another crack at this
later.
Discussion: https://postgr.es/m/16703-e348f58aab3cf6cc@postgresql.org
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_backup_db.h
M src/bin/pg_dump/pg_dump.c
pg_prewarm: make autoprewarm leader use standard SIGHUP and SIGTERM handlers.
commit : 53f614f1302d56e5fbb98826c7eee3c715411184
author : Fujii Masao <fujii@postgresql.org>
date : Sat, 7 Nov 2020 02:08:06 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Sat, 7 Nov 2020 02:08:06 +0900
Commit 1e53fe0e70 changed background processes so that they use
standard SIGHUP handler. Like that, this commit makes autoprewarm leader
process also use standard SIGHUP and SIGTERM handlers, to simplify the code.
Author: Bharath Rupireddy
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACXPorUqePswDtOeM_s82v9RW32E1fYmOPZ5NuE+TWKj_A@mail.gmail.com
M contrib/pg_prewarm/autoprewarm.c
Add pg_strong_random_init function to initialize random number generator
commit : 5ee180a3947060b98284a935f26f92c71d698f7c
author : Magnus Hagander <magnus@hagander.net>
date : Fri, 6 Nov 2020 13:21:28 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Fri, 6 Nov 2020 13:21:28 +0100
Currently only OpenSSL requires this initialization, but in the future
other SSL implementations are likely to need it as well. Abstracting
this functionality out into a separate function makes this cleaner and
more clear, and also removes the dependency on OpenSSL headers from
fork_process.c.
OpenSSL is special in that we need to initialize this random number
generator even if we're not going to use it directly, until we drop
support for everything prior to OpenSSL 1.1.1. (And of course also if we
actually use it). All other implementations are left empty at this time,
but more are expected to be added in the future.
Author: Daniel Gustafsson <daniel@yesql.se>, Michael Paquier <michael@paquier.xyz>
Reviewed-By: Magnus Hagander <magnus@hagander.net>
Discussion: https://postgr.es/m/F6291C3C-747C-4C93-BCE0-28BB420B1FF5@yesql.se
M src/backend/postmaster/fork_process.c
M src/include/port.h
M src/port/pg_strong_random.c
Use strlcpy instead of memcpy for copying the slot name in pgstat.c.
commit : 4f841ce3f7f4d429a3a275f82745d63c78cde4b2
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 6 Nov 2020 08:12:48 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 6 Nov 2020 08:12:48 +0530
There is no outright bug here but it is better to be consistent with the
usage at other places in the same file. In the passing, fix a wrong
assertion in pgstat_recv_replslot.
Author: Kyotaro Horiguchi
Reviewed-by: Sawada Masahiko and Amit Kapila
Discussion: https://postgr.es/m/20201104.175523.1704166915688949637.horikyota.ntt@gmail.com
M src/backend/postmaster/pgstat.c
Fix wal_consistency_checking nbtree bug.
commit : efc5dcfd8ad4e1df633025d8a91b64cd44d93f42
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 5 Nov 2020 15:01:40 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 5 Nov 2020 15:01:40 -0800
wal_consistency_checking indicated an inconsistency in certain cases
involving nbtree page deletion. The underlying issue is that there was
a minor difference between the page image produced after a REDO routine
ran and the corresponding page image following original execution.
This harmless inconsistency has been around forever. We more or less
expect total consistency among even deleted nbtree pages these days,
though, so this won't do anymore.
To fix, tweak the REDO routine to match original execution.
Oversight in commit f47b5e13.
M src/backend/access/nbtree/nbtxlog.c
Don't throw an error for LOCK TABLE on a self-referential view.
commit : 5b7bfc39726ff9f6c52dd73e337c34e74e0d1f39
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 5 Nov 2020 11:44:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 5 Nov 2020 11:44:32 -0500
LOCK TABLE has complained about "infinite recursion" when applied
to a self-referential view, ever since we made it recurse into views
in v11. However, that breaks pg_dump's new assumption that it's
okay to lock every relation. There doesn't seem to be any good
reason to throw an error: if we just abandon the recursion, we've
still satisfied the requirement of locking every referenced relation.
Per bug #16703 from Andrew Bille (via Alexander Lakhin).
Discussion: https://postgr.es/m/16703-e348f58aab3cf6cc@postgresql.org
M src/backend/commands/lockcmds.c
M src/test/regress/expected/lock.out
M src/test/regress/sql/lock.sql
Fix nbtree cleanup-only VACUUM stats inaccuracies.
commit : 48e1291342dd7771cf8c67aa1d7ec1f394b95dd8
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 4 Nov 2020 18:42:27 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 4 Nov 2020 18:42:27 -0800
Logic for counting heap TIDs from posting list tuples (added by commit
0d861bbb) was faulty. It didn't count any TIDs/index tuples in the
event of no callback being set. This meant that we incorrectly counted
no index tuples in clean-up only VACUUMs, which could lead to
pg_class.reltuples being spuriously set to 0 in affected indexes.
To fix, go back to counting items from the page in cases where there is
no callback. This approach isn't very accurate, but it works well
enough in practice while avoiding the expense of accessing every index
tuple during cleanup-only VACUUMs.
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>
https://postgr.es/m/20201023174451.69e358f1@firost
Backpatch: 13-, where nbtree deduplication was introduced
M src/backend/access/nbtree/nbtree.c
Fix unlinking of SLRU segments.
commit : c732c3f8c122009de373cff9b44b5cf0992ba1bf
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 5 Nov 2020 13:45:32 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 5 Nov 2020 13:45:32 +1300
Commit dee663f7 intended to drop any queued up fsync requests before
unlinking segment files, but missed a code path. Fix, by centralizing
the forget-and-unlink code into a single function.
Reported-by: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Discussion: https://postgr.es/m/20201104013205.icogbi773przyny5%40development
M src/backend/access/transam/slru.c
Remove underflow error in float division with infinite divisor.
commit : fac83dbd6fe1ac3d4125bfa39f287f95bffe6cda
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 4 Nov 2020 18:11:15 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 4 Nov 2020 18:11:15 -0500
float4_div and float8_div correctly produced zero for zero divided
by infinity, but threw an underflow error for nonzero finite values
divided by infinity. This seems wrong; at the very least it's
inconsistent with the behavior recently implemented for numeric
infinities. Remove the error and allow zero to be returned.
This patch also removes a useless isinf() test from the overflow
checks in these functions (non-Inf divided by Inf can't produce Inf).
Extracted from a larger patch; this seems significant outside the
context of geometric operators, so it deserves its own commit.
Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAGf+fX70rWFOk5cd00uMfa__0yP+vtQg5ck7c2Onb-Yczp0URA@mail.gmail.com
M src/include/utils/float.h
M src/test/regress/expected/float4-misrounded-input.out
M src/test/regress/expected/float4.out
M src/test/regress/expected/float8.out
M src/test/regress/sql/float4.sql
M src/test/regress/sql/float8.sql
Declare assorted array functions using anycompatible not anyelement.
commit : 9e38c2bb5093ceb0c04d6315ccd8975bd17add66
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 4 Nov 2020 16:09:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 4 Nov 2020 16:09:55 -0500
Convert array_append, array_prepend, array_cat, array_position,
array_positions, array_remove, array_replace, and width_bucket
to use anycompatiblearray. This is a simple extension of commit
5c292e6b9 to hit some other places where there's a pretty obvious
gain in usability from doing so.
Ideally we'd also modify other functions taking multiple old-style
polymorphic arguments. But most of the remainder are tied into one
or more operator classes, making any such change a much larger can of
worms than I desire to open right now.
Discussion: https://postgr.es/m/77675130-89da-dab1-51dd-492c93dcf5d1@postgresfriends.org
M doc/src/sgml/func.sgml
M doc/src/sgml/xaggr.sgml
M src/include/catalog/catversion.h
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/arrays.out
M src/test/regress/expected/polymorphism.out
M src/test/regress/sql/arrays.sql
M src/test/regress/sql/polymorphism.sql
Declare lead() and lag() using anycompatible not anyelement.
commit : 5c292e6b90433c760a3e15027646c7b94afd0cdd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 4 Nov 2020 15:08:37 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 4 Nov 2020 15:08:37 -0500
This allows use of a "default" expression that doesn't slavishly
match the data column's type. Formerly you got something like
"function lag(numeric, integer, integer) does not exist", which
is not just unhelpful but actively misleading.
The SQL spec suggests that the default should be coerced to the data
column's type, but this implementation instead chooses the common
supertype, which seems at least as reasonable.
(Note: I took the opportunity to run "make reformat-dat-files" on
pg_proc.dat, so this commit includes some cosmetic changes to
recently-added entries that aren't related to lead/lag.)
Vik Fearing
Discussion: https://postgr.es/m/77675130-89da-dab1-51dd-492c93dcf5d1@postgresfriends.org
M doc/src/sgml/func.sgml
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/window.out
M src/test/regress/sql/window.sql
Improve our ability to regurgitate SQL-syntax function calls.
commit : 40c24bfef92530bd846e111c1742c2a54441c62c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 4 Nov 2020 12:34:50 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 4 Nov 2020 12:34:50 -0500
The SQL spec calls out nonstandard syntax for certain function calls,
for example substring() with numeric position info is supposed to be
spelled "SUBSTRING(string FROM start FOR count)". We accept many
of these things, but up to now would not print them in the same format,
instead simplifying down to "substring"(string, start, count).
That's long annoyed me because it creates an interoperability
problem: we're gratuitously injecting Postgres-specific syntax into
what might otherwise be a perfectly spec-compliant view definition.
However, the real reason for addressing it right now is to support
a planned change in the semantics of EXTRACT() a/k/a date_part().
When we switch that to returning numeric, we'll have the parser
translate EXTRACT() to some new function name (might as well be
"extract" if you ask me) and then teach ruleutils.c to reverse-list
that per SQL spec. In this way existing calls to date_part() will
continue to have the old semantics.
To implement this, invent a new CoercionForm value COERCE_SQL_SYNTAX,
and make the parser insert that rather than COERCE_EXPLICIT_CALL when
the input has SQL-spec decoration. (But if the input has the form of
a plain function call, continue to mark it COERCE_EXPLICIT_CALL, even
if it's calling one of these functions.) Then ruleutils.c recognizes
COERCE_SQL_SYNTAX as a cue to emit SQL call syntax. It can know
which decoration to emit using hard-wired knowledge about the
functions that could be called this way. (While this solution isn't
extensible without manual additions, neither is the grammar, so this
doesn't seem unmaintainable.) Notice that this solution will
reverse-list a function call with SQL decoration only if it was
entered that way; so dump-and-reload will not by itself produce any
changes in the appearance of views.
This requires adding a CoercionForm field to struct FuncCall.
(I couldn't resist the temptation to rearrange that struct's
field order a tad while I was at it.) FuncCall doesn't appear
in stored rules, so that change isn't a reason for a catversion
bump, but I did one anyway because the new enum value for
CoercionForm fields could confuse old backend code.
Possible future work:
* Perhaps CoercionForm should now be renamed to DisplayForm,
or something like that, to reflect its more general meaning.
This'd require touching a couple hundred places, so it's not
clear it's worth the code churn.
* The SQLValueFunction node type, which was invented partly for
the same goal of improving SQL-compatibility of view output,
could perhaps be replaced with regular function calls marked
with COERCE_SQL_SYNTAX. It's unclear if this would be a net
code savings, however.
Discussion: https://postgr.es/m/42b73d2d-da12-ba9f-570a-420e0cce19d9@phystech.edu
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/makefuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_func.c
M src/backend/parser/parse_utilcmd.c
M src/backend/utils/adt/ruleutils.c
M src/include/catalog/catversion.h
M src/include/nodes/makefuncs.h
M src/include/nodes/parsenodes.h
M src/include/nodes/primnodes.h
M src/test/modules/test_rls_hooks/test_rls_hooks.c
M src/test/regress/expected/create_view.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/create_view.sql
Remove useless entries for aggregate functions from fmgrtab.c.
commit : f21636e5d5b8394ed076e18ddc5f4ba710c69c99
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 4 Nov 2020 11:25:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 4 Nov 2020 11:25:56 -0500
Gen_fmgrtab.pl treated aggregate functions the same as other built-in
functions, which is wasteful because there is no real need to have
entries for them in the fmgr_builtins[] table. Suppressing those
entries saves about 3KB in the compiled table on my machine; which
is not a lot but it's not nothing either, considering that that
table is pretty "hot". The only outside code change needed is
that ExecInitWindowAgg() can't be allowed to call fmgr_info_cxt()
on a plain aggregate function. But that saves a few cycles anyway.
Having done that, the aggregate_dummy() function is unreferenced
and might as well be dropped. Using "aggregate_dummy" as the prosrc
value for an aggregate is now just a documentation convention not
something that matters. There was some discussion of using NULL
instead to save a few bytes in pg_proc, but we'd have to remove
prosrc's BKI_FORCE_NOT_NULL marking which doesn't seem a great idea.
Anyway, it's possible there's client-side code that expects to
see "aggregate_dummy" there, so I'm loath to change it without a
strong reason.
Discussion: https://postgr.es/m/533989.1604263665@sss.pgh.pa.us
M src/backend/catalog/pg_aggregate.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeWindowAgg.c
M src/backend/utils/Gen_fmgrtab.pl
Fix segmentation fault that commit ac22929a26 caused.
commit : 113d3591b859fb8dc191bc0599d1ad62d91f1aa4
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 4 Nov 2020 21:49:00 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 4 Nov 2020 21:49:00 +0900
Commit ac22929a26 changed recoveryWakeupLatch so that it's reset to
NULL at the end of recovery. This change could cause a segmentation fault
in the buildfarm member 'elver'.
Previously the latch was reset to NULL after calling ShutdownWalRcv().
But there could be a window between ShutdownWalRcv() and the actual
exit of walreceiver. If walreceiver set the latch during that window,
the segmentation fault could happen.
To fix the issue, this commit changes walreceiver so that it sets
the latch only when the latch has not been reset to NULL yet.
Author: Fujii Masao
Discussion: https://postgr.es/m/5c1f8a85-747c-7bf9-241e-dd467d8a3586@iki.fi
M src/backend/access/transam/xlog.c
Enable hash partitioning of text arrays
commit : 560564d3addcb6c54e13c1ca8bd11afafbb7ba11
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 4 Nov 2020 07:47:06 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 4 Nov 2020 07:47:06 +0100
hash_array_extended() needs to pass PG_GET_COLLATION() to the hash
function of the element type. Otherwise, the hash function of a
collation-aware data type such as text will error out, since the
introduction of nondeterministic collation made hash functions require
a collation, too.
The consequence of this is that before this change, hash partitioning
using an array over text in the partition key would not work.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/32c1fdae-95c6-5dc6-058a-a90330a3b621%40enterprisedb.com
M src/backend/utils/adt/arrayfuncs.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
pg_rewind: Refactor the abstraction to fetch from local/libpq source.
commit : 37d2ff38031262a1778bc76a9c55fff7afbcf275
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 4 Nov 2020 11:21:18 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 4 Nov 2020 11:21:18 +0200
This makes the abstraction of a "source" server more clear, by introducing
a common abstract class, borrowing the object-oriented programming term,
that represents all the operations that can be done on the source server.
There are two implementations of it, one for fetching via libpq, and
another to fetch from a local directory. This adds some code, but makes it
easier to understand what's going on.
The copy_executeFileMap() and libpq_executeFileMap() functions contained
basically the same logic, just calling different functions to fetch the
source files. Refactor so that the common logic is in one place, in a new
function called perform_rewind().
Reviewed-by: Kyotaro Horiguchi, Soumyadeep Chakraborty
Discussion: https://www.postgresql.org/message-id/0c5b3783-af52-3ee5-f8fa-6e794061f70d%40iki.fi
M src/bin/pg_rewind/Makefile
D src/bin/pg_rewind/copy_fetch.c
D src/bin/pg_rewind/fetch.c
D src/bin/pg_rewind/fetch.h
M src/bin/pg_rewind/file_ops.c
M src/bin/pg_rewind/file_ops.h
R065 src/bin/pg_rewind/libpq_fetch.c src/bin/pg_rewind/libpq_source.c
A src/bin/pg_rewind/local_source.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/pg_rewind.h
A src/bin/pg_rewind/rewind_source.h
M src/tools/pgindent/typedefs.list
pg_rewind: Replace the hybrid list+array data structure with simplehash.
commit : f81e97d0475cd4bc597adc23b665bd84fbf79a0d
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 4 Nov 2020 11:21:14 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 4 Nov 2020 11:21:14 +0200
Now that simplehash can be used in frontend code, let's make use of it.
Reviewed-by: Kyotaro Horiguchi, Soumyadeep Chakraborty
Discussion: https://www.postgresql.org/message-id/0c5b3783-af52-3ee5-f8fa-6e794061f70d%40iki.fi
M src/bin/pg_rewind/copy_fetch.c
M src/bin/pg_rewind/fetch.c
M src/bin/pg_rewind/fetch.h
M src/bin/pg_rewind/filemap.c
M src/bin/pg_rewind/filemap.h
M src/bin/pg_rewind/libpq_fetch.c
M src/bin/pg_rewind/pg_rewind.c
Refactor pg_rewind for more clear decision making.
commit : eb00f1d4bf96bdba236bcc089f3ae94db9b7c603
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 4 Nov 2020 11:21:09 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 4 Nov 2020 11:21:09 +0200
Deciding what to do with each file is now a separate step after all the
necessary information has been gathered. It is more clear that way.
Previously, the decision-making was divided between process_source_file()
and process_target_file(), and it was a bit hard to piece together what
the overall rules were.
Reviewed-by: Kyotaro Horiguchi, Soumyadeep Chakraborty
Discussion: https://www.postgresql.org/message-id/0c5b3783-af52-3ee5-f8fa-6e794061f70d%40iki.fi
M src/bin/pg_rewind/copy_fetch.c
M src/bin/pg_rewind/file_ops.c
M src/bin/pg_rewind/filemap.c
M src/bin/pg_rewind/filemap.h
M src/bin/pg_rewind/libpq_fetch.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
pg_rewind: Move syncTargetDirectory() to file_ops.c
commit : ffb4e27e9c5ea87f9fecb7036dfc7cc1f38169b6
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 4 Nov 2020 10:38:39 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 4 Nov 2020 10:38:39 +0200
For consistency. All the other low-level functions that operate on the
target directory are in file_ops.c.
Reviewed-by: Michael Paquier
Discussion: https://www.postgresql.org/message-id/0c5b3783-af52-3ee5-f8fa-6e794061f70d%40iki.fi
M src/bin/pg_rewind/file_ops.c
M src/bin/pg_rewind/file_ops.h
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/pg_rewind.h
Get rid of the dedicated latch for signaling the startup process.
commit : ac22929a2613e122708bd0172508ac863c51c1cc
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 4 Nov 2020 16:41:29 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 4 Nov 2020 16:41:29 +0900
This commit gets rid of the dedicated latch for signaling the startup
process in favor of using its procLatch, since that comports better
with possible generic signal handlers using that latch.
Commit 1e53fe0e70 changed background processes so that they use standard
SIGHUP handler. Like that, this commit also makes the startup process use
standard SIGHUP handler to simplify the code.
Author: Fujii Masao
Reviewed-by: Bharath Rupireddy, Michael Paquier
Discussion: https://postgr.es/m/CALj2ACXPorUqePswDtOeM_s82v9RW32E1fYmOPZ5NuE+TWKj_A@mail.gmail.com
M src/backend/access/transam/xlog.c
M src/backend/postmaster/startup.c
Use standard SIGHUP handler in syslogger.
commit : 02d332297f9bfe63476d53a439cc8f3b05bb5e67
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 4 Nov 2020 14:48:02 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 4 Nov 2020 14:48:02 +0900
Commit 1e53fe0e70 changed background processes so that they use
standard SIGHUP handler. Like that, this commit makes syslogger use
standard SIGHUP handler to simplify the code.
Author: Bharath Rupireddy
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CALj2ACXPorUqePswDtOeM_s82v9RW32E1fYmOPZ5NuE+TWKj_A@mail.gmail.com
M src/backend/postmaster/syslogger.c
Tolerate version lookup failure for old style Windows locale names.
commit : 9f12a3b95dd56c897f1aa3d756d8fb419e84a187
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 4 Nov 2020 14:58:34 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 4 Nov 2020 14:58:34 +1300
Accept that we can't get versions for such locale names for now. Users
will need to specify the newer language tag format to enable the
collation versioning feature. It's not clear that we can do automatic
conversion from the old style to the new style reliably enough for this
purpose.
Unfortunately, this means that collation versioning probably won't work
for the default collation unless you provide something like en-US at
initdb or CREATE DATABASE time (though, for reasons not yet understood,
it does seem to work on some systems). It'd be nice to find a better
solution, or document this quirk if we settle on it, but this should
unbreak the 3 failing build farm animals in the meantime.
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
M src/backend/utils/adt/pg_locale.c
Revert pg_relation_check_pages()
commit : e152506adef4bc503ea7b8ebb4fedc0b8eebda81
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 4 Nov 2020 10:21:46 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 4 Nov 2020 10:21:46 +0900
This reverts the following set of commits, following complaints about
the lack of portability of the central part of the code in bufmgr.c as
well as the use of partition mapping locks during page reads:
c780a7a9
f2b88396
b787d4ce
ce7f772c
60a51c6b
Per discussion with Andres Freund, Robert Haas and myself.
Bump catalog version.
Discussion: https://postgr.es/m/20201029181729.2nrub47u7yqncsv7@alap3.anarazel.de
M doc/src/sgml/func.sgml
M src/backend/catalog/system_views.sql
M src/backend/storage/buffer/bufmgr.c
M src/backend/utils/adt/Makefile
D src/backend/utils/adt/pagefuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/storage/bufmgr.h
D src/test/recovery/t/022_page_check.pl
D src/test/regress/expected/pagefuncs.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
D src/test/regress/sql/pagefuncs.sql
M src/tools/pgindent/typedefs.list
Use INT64_FORMAT to print int64 variables in sort debug
commit : 90851d1d26f54ccb4d7b1bc49449138113d6ec83
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 3 Nov 2020 20:43:12 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 3 Nov 2020 20:43:12 +0100
Commit 6ee3b5fb99 cleaned up most of the long/int64 confusion related to
incremental sort, but the sort debug messages were still using %ld for
int64 variables. So fix that.
Author: Haiying Tang
Backpatch-through: 13, where the incremental sort code was added
Discussion: https://postgr.es/m/4250be9d350c4992abb722a76e288aef%40G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/executor/nodeIncrementalSort.c
Fix get_useful_pathkeys_for_relation for volatile expressions
commit : ebb7ae839d033d0f279670e249f54646a08b8c48
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 3 Nov 2020 20:07:23 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 3 Nov 2020 20:07:23 +0100
When considering Incremental Sort below a Gather Merge, we need to be
a bit more careful when matching pathkeys to EC members. It's not enough
to find a member whose Vars are all in the current relation's target;
volatile expressions in particular need to be contained in the target,
otherwise it's too early to use the pathkey.
Reported-by: Jaime Casanova
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13, where the incremental sort code was added
Discussion: https://postgr.es/m/CAJGNTeNaxpXgBVcRhJX%2B2vSbq%2BF2kJqGBcvompmpvXb7pq%2BoFA%40mail.gmail.com
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/equivclass.c
M src/include/optimizer/paths.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Guard against core dump from uninitialized subplan.
commit : 92f87182f2c617fd420832972b6d0ae4527301c8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Nov 2020 16:16:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Nov 2020 16:16:36 -0500
If the planner erroneously puts a non-parallel-safe SubPlan into
a parallelized portion of the query tree, nodeSubplan.c will fail
in the worker processes because it finds a null in es_subplanstates,
which it's unable to cope with. It seems worth a test-and-elog to
make that an error case rather than a core dump case.
This probably should have been included in commit 16ebab688, which
was responsible for allowing nulls to appear in es_subplanstates
to begin with. So, back-patch to v10 where that came in.
Discussion: https://postgr.es/m/924226.1604422326@sss.pgh.pa.us
M src/backend/executor/nodeSubplan.c
Improve error messages around REPLICATION and BYPASSRLS properties.
commit : 17fb60387ce3fdc2bbb13d9b67bed0e4da77e173
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Nov 2020 15:49:05 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Nov 2020 15:49:05 -0500
Clarify wording as per suggestion from Wolfgang Walther.
No back-patch; this doesn't seem worth thrashing translatable
strings in the back branches.
Tom Lane and Stephen Frost
Discussion: https://postgr.es/m/a5548a9f-89ee-3167-129d-162b5985fcf8@technowledgy.de
M src/backend/commands/user.c
Allow users with BYPASSRLS to alter their own passwords.
commit : d907bd0543aa63e59653d7345840bed0f8b3a83b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Nov 2020 15:41:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Nov 2020 15:41:32 -0500
The intention in commit 491c029db was to require superuserness to
change the BYPASSRLS property, but the actual effect of the coding
in AlterRole() was to require superuserness to change anything at all
about a BYPASSRLS role. Other properties of a BYPASSRLS role should
be changeable under the same rules as for a normal role, though.
Fix that, and also take care of some documentation omissions related
to BYPASSRLS and REPLICATION role properties.
Tom Lane and Stephen Frost, per bug report from Wolfgang Walther.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/a5548a9f-89ee-3167-129d-162b5985fcf8@technowledgy.de
M doc/src/sgml/ref/alter_role.sgml
M doc/src/sgml/ref/create_role.sgml
M src/backend/commands/user.c
Disallow ALTER TABLE ONLY / DROP EXPRESSION
commit : bf797a8d9768239f5e3204b013044274b2c7c24a
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 3 Nov 2020 15:14:50 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 3 Nov 2020 15:14:50 +0100
The current implementation cannot handle this correctly, so just
forbid it for now.
GENERATED clauses must be attached to the column definition and cannot
be added later like DEFAULT, so if a child table has a generation
expression that the parent does not have, the child column will
necessarily be an attlocal column. So to implement ALTER TABLE ONLY /
DROP EXPRESSION, we'd need extra code to update attislocal of the
direct child tables, somewhat similar to how DROP COLUMN does it, so
that the resulting state can be properly dumped and restored.
Discussion: https://www.postgresql.org/message-id/flat/15830.1575468847%40sss.pgh.pa.us
M src/backend/commands/tablecmds.c
M src/test/regress/expected/generated.out
M src/test/regress/sql/generated.sql
Remove deprecated containment operators for built-in types
commit : 2f70fdb0644c32c4154236c2b5c241bec92eac5e
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 3 Nov 2020 10:32:20 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 3 Nov 2020 10:32:20 +0100
Remove old containment operators @ and ~ for built-in geometry data
types. These have been deprecated; use <@ and @> instead.
(Some contrib modules still contain the same deprecated operators.
That will be dealt with separately.)
Author: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/flat/20201027032511.GF9241@telsasoft.com
M doc/src/sgml/func.sgml
M src/include/catalog/catversion.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_operator.dat
M src/test/regress/expected/create_am.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/sql/create_am.sql
Use the non-deprecated TG_TABLE_MAME in test trigger
commit : 44a184cb686866b10d63695db344195c239f9374
author : Magnus Hagander <magnus@hagander.net>
date : Tue, 3 Nov 2020 10:19:55 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Tue, 3 Nov 2020 10:19:55 +0100
Commit 3a9ae3d2068 (back in 2006) deprecated TG_RELNAME
in favor of TG_TABLE_NAME, but the existing usage in test
cases has remained till today. Change to use TG_TABLE_NAME
instead (TG_RELNAME is still covered by a test case).
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Improve error handling in backend OpenSSL implementation
commit : 13cfa02f77936895bff6ffabf1fde5d47fd1df07
author : Magnus Hagander <magnus@hagander.net>
date : Tue, 3 Nov 2020 09:55:51 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Tue, 3 Nov 2020 09:55:51 +0100
Commit d94c36a45ab introduced error handling to sslinfo to handle
OpenSSL errors gracefully. This ports this errorhandling to the
backend TLS implementation.
Author: Daniel Gustafsson <daniel@yesql.se>
M src/backend/libpq/be-secure-openssl.c
Use be_tls_* API for SSL information in sslinfo
commit : 5d1833f414973595411779b86e085fb05907a805
author : Magnus Hagander <magnus@hagander.net>
date : Tue, 3 Nov 2020 09:47:36 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Tue, 3 Nov 2020 09:47:36 +0100
sslinfo was passing the Port->ssl member directly to OpenSSL in order
to extract information regarding the connection. This breaks the API
provided by the backend TLS implementation, as well as duplicates code
for no benefit. Rewrite to make use of the backend API as much as
possible.
Author: Daniel Gustafsson <daniel@yesql.se>
M contrib/sslinfo/sslinfo.c
M doc/src/sgml/sslinfo.sgml
Remove use of deprecated containment operators in tests
commit : 3165426e54df02a6199c0a216546e5095e875a0a
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 3 Nov 2020 09:03:22 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 3 Nov 2020 09:03:22 +0100
Switch @ to <@ and ~ to @> in gist-related tests. The old operator
names have been deprecated and will eventually (possibly soon) be
removed.
Author: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/flat/20201027032511.GF9241@telsasoft.com
M src/test/regress/expected/create_am.out
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_am.sql
M src/test/regress/sql/create_index.sql
Fix typos.
commit : 8c2d8f6cc4848cf9276dff445bb0f2f664083eca
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 3 Nov 2020 08:38:27 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 3 Nov 2020 08:38:27 +0530
Author: Hou Zhijie
Discussion: https://postgr.es/m/855a9421839d402b8b351d273c89a8f8@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/storage/ipc/procarray.c
M src/include/access/xlogreader.h
Fix unportable use of getnameinfo() in pg_hba_file_rules view.
commit : 0a4b34031279d938c2e59df8df7159d6c11e39b5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Nov 2020 21:11:50 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Nov 2020 21:11:50 -0500
fill_hba_line() thought it could get away with passing sizeof(struct
sockaddr_storage) rather than the actual addrlen previously returned
by getaddrinfo(). While that appears to work on many platforms,
it does not work on FreeBSD 11: you get back a failure, which leads
to the view showing NULL for the address and netmask columns in all
rows. The POSIX spec for getnameinfo() is pretty clearly on
FreeBSD's side here: you should pass the actual address length.
So it seems plausible that there are other platforms where this
coding also fails, and we just hadn't noticed.
Also, IMO the fact that getnameinfo() failure leads to a NULL output
is pretty bogus in itself. Our pg_getnameinfo_all() wrapper is
careful to emit "???" on failure, and we should use that in such
cases. NULL should only be emitted in rows that don't have IP
addresses.
Per bug #16695 from Peter Vandivier. Back-patch to v10 where this
code was added.
Discussion: https://postgr.es/m/16695-a665558e2f630be7@postgresql.org
M src/backend/libpq/hba.c
M src/include/libpq/hba.h
Remove special checks for pg_rewrite.ev_qual and ev_action being NULL.
commit : e1339bfc7a2fd4629e1c3f8f919ddd05b4745e13
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Nov 2020 14:34:34 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Nov 2020 14:34:34 -0500
make_ruledef() and make_viewdef() were coded to cope with possible
null-ness of these columns, but they've been marked BKI_FORCE_NOT_NULL
for some time. So there's not really any need to do more than what
we do for the other columns of pg_rewrite, i.e. just Assert that
we got non-null results.
(There is a school of thought that says Asserts aren't the thing
to do to check for corrupt data, but surely here is not the place
to start if we want such a policy.)
Also, remove long-dead-if-indeed-it-ever-wasn't-dead handling of
an empty actions list in make_ruledef(). That's an error case
and should be treated as such. (DO INSTEAD NOTHING is represented
by a CMD_NOTHING Query, not an empty list; cf transformRuleStmt.)
Kyotaro Horiguchi, some changes by me
Discussion: https://postgr.es/m/CAEudQApoA=tMTic6xEPYP_hsNZ8XtToVThK_0x7D_aFQYowq3w@mail.gmail.com
M src/backend/utils/adt/ruleutils.c
Rethink the generation rule for fmgroids.h macros.
commit : 8e1f37c07aafd4bb7aa6e1e1982010af11f8b5c7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Nov 2020 11:57:28 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Nov 2020 11:57:28 -0500
Traditionally, the names of fmgroids.h macros for pg_proc OIDs
have been constructed from the prosrc field. But sometimes the
same C function underlies multiple pg_proc entries, forcing us
to make an arbitrary choice of which OID to reference; the other
entries are then not namable via fmgroids.h. Moreover, we could
not have macros at all for pg_proc entries that aren't for
C-coded functions.
Instead, use the proname field, and append the proargtypes field
(replacing inter-argument spaces with underscores) if proname is
not unique. Special-casing unique entries such as F_OIDEQ removes
the need to change a lot of code. Indeed, I can only find two
places in the tree that need to be adjusted; while this changes
quite a few existing entries in fmgroids.h, few of them are
referenced from C code.
With this patch, all entries in pg_proc.dat have macros in fmgroids.h.
Discussion: https://postgr.es/m/472274.1604258384@sss.pgh.pa.us
M src/backend/optimizer/util/clauses.c
M src/backend/utils/Gen_fmgrtab.pl
M src/backend/utils/adt/ruleutils.c
Second thoughts on TOAST decompression.
commit : fd2997565c6f66837440dd57f5e52b56aa964d14
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Nov 2020 11:25:18 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Nov 2020 11:25:18 -0500
On detecting a corrupted match tag, pglz_decompress() should just
summarily return -1. Breaking out of the loop, as I did in dfc797730,
doesn't quite guarantee that will happen. Also, we can use
unlikely() on that check, just in case it helps.
Backpatch to v13, like the previous patch.
M src/common/pg_lzcompress.c
Use PG_GETARG_TRANSACTIONID where appropriate
commit : dd26a0ad760b11237d7ea9cda8ccccc1826c2c64
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 2 Nov 2020 16:48:22 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 2 Nov 2020 16:48:22 +0100
Some places were using PG_GETARG_UINT32 where PG_GETARG_TRANSACTIONID
would be more appropriate. (Of course, they are the same internally,
so there is no externally visible effect.) To do that, export
PG_GETARG_TRANSACTIONID outside of xid.c. We also export
PG_RETURN_TRANSACTIONID for symmetry, even though there are currently
no external users.
Author: Ashutosh Bapat <ashutosh.bapat@2ndquadrant.com>
Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/multixact.c
M src/backend/utils/adt/xid.c
M src/include/fmgr.h
Clarify temporary table name shadowing in CREATE TABLE docs
commit : 5b3dca096f8e504b73812f68716fb68dd1176d6d
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 2 Nov 2020 15:00:24 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 2 Nov 2020 15:00:24 +0100
Author: David Johnston
M doc/src/sgml/ref/create_table.sgml
Track collation versions for indexes.
commit : 257836a75585934cc05ed7a80bccf8190d41e056
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 2 Nov 2020 19:50:45 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 2 Nov 2020 19:50:45 +1300
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. When accessing the index later, warn
that the index may be corrupted if the current version doesn't match.
Thanks to Douglas Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for very helpful
discussion.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com> (earlier versions)
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/charset.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/alter_index.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/reindex.sgml
M src/backend/catalog/dependency.c
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/pg_constraint.c
M src/backend/catalog/pg_depend.c
M src/backend/catalog/pg_type.c
M src/backend/commands/collationcmds.c
M src/backend/commands/tablecmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/optimizer/util/plancat.c
M src/backend/parser/gram.y
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_upgrade_support.c
M src/backend/utils/cache/relcache.c
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_upgrade/dump.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/psql/tab-complete.c
M src/include/catalog/catversion.h
M src/include/catalog/dependency.h
M src/include/catalog/index.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_type.h
M src/include/nodes/parsenodes.h
M src/include/utils/pg_locale.h
M src/include/utils/rel.h
M src/test/Makefile
M src/test/locale/.gitignore
M src/test/locale/Makefile
A src/test/locale/t/001_index.pl
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/expected/create_index.out
M src/test/regress/sql/collate.icu.utf8.sql
M src/tools/pgindent/typedefs.list
Add pg_depend.refobjversion.
commit : cd6f479e79f3a33ef7a919c6b6c0c498c790f154
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 2 Nov 2020 19:40:49 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 2 Nov 2020 19:40:49 +1300
Provide a place for the version of referenced database objects to be
recorded. A follow-up commit will use this to record dependencies on
collation versions for indexes, but similar ideas for other kinds of
objects have also been mooted.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/dependency.c
M src/backend/catalog/pg_depend.c
M src/include/catalog/catversion.h
M src/include/catalog/dependency.h
M src/include/catalog/pg_depend.h
M src/include/catalog/toasting.h
M src/test/regress/expected/misc_sanity.out
Remove pg_collation.collversion.
commit : 7d1297df0830725d4434ba7dbf71a9e8221ad49e
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 2 Nov 2020 19:36:09 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 2 Nov 2020 19:36:09 +1300
This model couldn't be extended to cover the default collation, and
didn't have any information about the affected database objects when the
version changed. Remove, in preparation for a follow-up commit that
will add a new mechanism.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/alter_collation.sgml
M doc/src/sgml/ref/create_collation.sgml
M src/backend/catalog/pg_collation.c
M src/backend/commands/collationcmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/parser/gram.y
M src/backend/tcop/utility.c
M src/backend/utils/adt/pg_locale.c
M src/bin/pg_dump/pg_dump.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_collation.dat
M src/include/catalog/pg_collation.h
M src/include/catalog/toasting.h
M src/include/commands/collationcmds.h
M src/include/nodes/parsenodes.h
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/expected/collate.linux.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
M src/test/regress/sql/collate.linux.utf8.sql
doc: Mention UNION/ORDER BY etc. keywords in section headers.
commit : 8ef2a5afdf8ec9e4c8b28a7042c9508eb6161671
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 2 Nov 2020 12:51:46 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 2 Nov 2020 12:51:46 +0200
Most of the section and sub-section headers in the Queries chapter have
the keywords literally stated, but neither "Sorting Rows" nor "Combining
Rows" did. There's no rule that they must be, but it seems like a good
practice. The keywords will ring a bell to anyone with with even a little
bit of SQL experience.
David G. Johnston, per suggestion by bilge@scriptfusion.com
Discussion: https://www.postgresql.org/message-id/159981394174.31338.7014519396749859167%40wrigleys.postgresql.org
M doc/src/sgml/queries.sgml
Fix unstable partition_prune regression tests
commit : 90d8f1b1826ce076a502a43fe7c88423b46c6349
author : David Rowley <drowley@postgresql.org>
date : Mon, 2 Nov 2020 19:59:02 +1300
committer: David Rowley <drowley@postgresql.org>
date : Mon, 2 Nov 2020 19:59:02 +1300
This was broken recently by a929e17e5. I'd failed to remember that
parallel tests should have their EXPLAIN output run through the
explain_parallel_append function so that the output is stable when
parallel workers fail to start.
fairywren was first to notice.
Reported-by: Michael Paquier
Discussion: https://postgr.es/m/20201102062951.GB15770@paquier.xyz
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Fix some grammar and typos in comments and docs
commit : 8a15e735be00f156a7227741c0ce88702e6de099
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 2 Nov 2020 15:14:41 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 2 Nov 2020 15:14:41 +0900
The documentation fixes are backpatched down to where they apply.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20201031020801.GD3080@telsasoft.com
Backpatch-through: 9.6
M contrib/amcheck/verify_heapam.c
M contrib/pgcrypto/pgp-compress.c
M doc/src/sgml/auto-explain.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/pg_rewind.sgml
M src/backend/access/heap/pruneheap.c
M src/backend/catalog/namespace.c
M src/backend/catalog/pg_namespace.c
M src/backend/commands/copy.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execExpr.c
M src/backend/executor/nodeIncrementalSort.c
M src/backend/executor/nodeLimit.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/partitioning/partbounds.c
M src/backend/postmaster/interrupt.c
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/utils/adt/varlena.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pgbench/pgbench.c
M src/interfaces/libpq/fe-connect.c
M src/test/modules/dummy_index_am/dummy_index_am.c
Use Enum for top level logical replication message types.
commit : 644f0d7cc9c2cb270746f2024c706554e0fbec82
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 2 Nov 2020 08:18:18 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 2 Nov 2020 08:18:18 +0530
Logical replication protocol uses a single byte character to identify a
message type in logical replication protocol. The code uses string
literals for the same. Use Enum so that
1. All the string literals used can be found at a single place. This
makes it easy to add more types without the risk of conflicts.
2. It's easy to locate the code handling a given message type.
3. When used with switch statements, it is easy to identify the missing
cases using -Wswitch.
Author: Ashutosh Bapat
Reviewed-by: Kyotaro Horiguchi, Andres Freund, Peter Smith and Amit Kapila
Discussion: https://postgr.es/m/CAExHW5uPzQ7L0oAd_ENyvaiYMOPgkrAoJpE+ZY5-obdcVT6NPg@mail.gmail.com
M src/backend/replication/logical/proto.c
M src/backend/replication/logical/worker.c
M src/include/replication/logicalproto.h
Allow run-time pruning on nested Append/MergeAppend nodes
commit : a929e17e5a8c9b751b66002c8a89fdebdacfe194
author : David Rowley <drowley@postgresql.org>
date : Mon, 2 Nov 2020 13:46:56 +1300
committer: David Rowley <drowley@postgresql.org>
date : Mon, 2 Nov 2020 13:46:56 +1300
Previously we only tagged on the required information to allow the
executor to perform run-time partition pruning for Append/MergeAppend
nodes belonging to base relations. It was thought that nested
Append/MergeAppend nodes were just about always pulled up into the
top-level Append/MergeAppend and that making the run-time pruning info for
any sub Append/MergeAppend nodes was a waste of time. However, that was
likely badly thought through.
Some examples of cases we're unable to pullup nested Append/MergeAppends
are: 1) Parallel Append nodes with a mix of parallel and non-parallel
paths into a Parallel Append. 2) When planning an ordered Append scan a
sub-partition which is unordered may require a nested MergeAppend path to
ensure sub-partitions don't mix up the order of tuples being fed into the
top-level Append.
Unfortunately, it was not just as simple as removing the lines in
createplan.c which were purposefully not building the run-time pruning
info for anything but RELOPT_BASEREL relations. The code in
add_paths_to_append_rel() was far too sloppy about which partitioned_rels
it included for the Append/MergeAppend paths. The original code there
would always assume accumulate_append_subpath() would pull each sub-Append
and sub-MergeAppend path into the top-level path. While it does not
appear that there were any actual bugs caused by having the additional
partitioned table RT indexes recorded, what it did mean is that later in
planning, when we built the run-time pruning info that we wasted effort
and built PartitionedRelPruneInfos for partitioned tables that we had no
subpaths for the executor to run-time prune.
Here we tighten that up so that partitioned_rels only ever contains the RT
index for partitioned tables which actually have subpaths in the given
Append/MergeAppend. We can now Assert that every PartitionedRelPruneInfo
has a non-empty present_parts. That should allow us to catch any weird
corner cases that have been missed.
In passing, it seems there is no longer a good reason to have the
AppendPath and MergeAppendPath's partitioned_rel fields a List of IntList.
We can simply have a List of Relids instead. This is more compact in
memory and faster to add new members to. We still know which is the root
level partition as these always have a lower relid than their children.
Previously this field was used for more things, but run-time partition
pruning now remains the only user of it and it has no need for a List of
IntLists.
Here we also get rid of the RelOptInfo partitioned_child_rels field. This
is what was previously used to (sometimes incorrectly) set the
Append/MergeAppend path's partitioned_rels field. That was the only usage
of that field, so we can happily just remove it.
I also couldn't resist changing some nearby code to make use of the newly
added for_each_from macro so we can skip the first element in the list
without checking if the current item was the first one on each
iteration.
A bug report from Andreas Kretschmer prompted all this work, however,
after some consideration, I'm not personally classing this as a bug fix.
So no backpatch. In Andreas' test case, it just wasn't that clear that
there was a nested Append since the top-level Append just had a single
sub-path which was pulled up a level, per 8edd0e794.
Author: David Rowley
Reviewed-by: Amit Langote
Discussion: https://postgr.es/m/flat/CAApHDvqSchs%2BubdybcfFaSPB%2B%2BEA7kqMaoqajtP0GtZvzOOR3g%40mail.gmail.com
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/relnode.c
M src/backend/partitioning/partprune.c
M src/include/nodes/pathnodes.h
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Fix two issues in TOAST decompression.
commit : dfc797730fc7a07c0e6bd636ad1a564aecab3161
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Nov 2020 18:38:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Nov 2020 18:38:42 -0500
pglz_maximum_compressed_size() potentially underestimated the amount
of compressed data required to produce N bytes of decompressed data;
this is a fault in commit 11a078cf8.
Separately from that, pglz_decompress() failed to protect itself
against corrupt compressed data, particularly off == 0 in a match
tag. Commit c60e520f6 turned such a situation into an infinite loop,
where before it'd just have resulted in garbage output.
The combination of these two bugs seems like it may explain bug #16694
from Tom Vijlbrief, though it's impossible to be quite sure without
direct inspection of the failing session. (One needs to assume that
the pglz_maximum_compressed_size() bug caused us to fail to fetch the
second byte of a match tag, and what happened to be there instead was
a zero. The reported infinite loop is hard to explain without off == 0,
though.)
Aside from fixing the bugs, rewrite associated comments for more
clarity.
Back-patch to v13 where both these commits landed.
Discussion: https://postgr.es/m/16694-f107871e499ec114@postgresql.org
M src/common/pg_lzcompress.c
Avoid null pointer dereference if error result lacks SQLSTATE.
commit : 7f4235032f0d75ea1ad29b192d57fee3d8fe533e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Nov 2020 11:26:16 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Nov 2020 11:26:16 -0500
Although error results received from the backend should always have
a SQLSTATE field, ones generated by libpq won't, making this code
vulnerable to a crash after, say, untimely loss of connection.
Noted by Coverity.
Oversight in commit 403a3d91c. Back-patch to 9.5, as that was.
M src/bin/pg_dump/pg_backup_db.c
Preserve index data in pg_statistic across REINDEX CONCURRENTLY
commit : b17ff07aa3eb142d2cde2ea00e4a4e8f63686f96
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 1 Nov 2020 21:22:07 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 1 Nov 2020 21:22:07 +0900
Statistics associated to an index got lost after running REINDEX
CONCURRENTLY, while the non-concurrent case preserves these correctly.
The concurrent and non-concurrent operations need to be consistent for
the end-user, and missing statistics would force to wait for a new
analyze to happen, which could take some time depending on the activity
of the existing autovacuum workers. This issue is fixed by copying any
existing entries in pg_statistic associated to the old index to the new
one. Note that this copy is already done with the data of the index in
the stats collector.
Reported-by: Fabrízio de Royes Mello
Author: Michael Paquier, Fabrízio de Royes Mello
Reviewed-by: Justin Pryzby
Discussion: https://postgr.es/m/CAFcNs+qpFPmiHd1oTXvcPdvAHicJDA9qBUSujgAhUMJyUMb+SA@mail.gmail.com
Backpatch-through: 12
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/include/catalog/heap.h
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Add error code for encryption failure in pgcrypto
commit : aecaa04418f39c32adb3dbf91c4aa7f6e175f01c
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 1 Nov 2020 19:22:59 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 1 Nov 2020 19:22:59 +0900
PXE_DECRYPT_FAILED exists already for decryption errors, and an
equivalent for encryption did not exist. There is one code path that
deals with such failures for OpenSSL but it used PXE_ERR_GENERIC, which
was inconsistent. This switches this code path to use the new error
PXE_ENCRYPT_FAILED instead of PXE_ERR_GENERIC, making the code used for
encryption more consistent with the decryption.
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/03049139-CB7A-436E-B71B-42696D3E2EF7@yesql.se
M contrib/pgcrypto/openssl.c
M contrib/pgcrypto/px.c
M contrib/pgcrypto/px.h
Set debug_query_string in worker_spi.
commit : d2246cde825e4e1a85408390c66367b85b51a233
author : Noah Misch <noah@leadboat.com>
date : Sat, 31 Oct 2020 08:47:02 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 31 Oct 2020 08:47:02 -0700
This makes elog.c emit the string, which is good practice for a
background worker that executes SQL strings.
Reviewed by Tom Lane.
Discussion: https://postgr.es/m/20201014022636.GA1962668@rfd.leadboat.com
M src/test/modules/worker_spi/worker_spi.c
Reproduce debug_query_string==NULL on parallel workers.
commit : f90e80b9138355a51d2d5b5b63e1f89c4ba53325
author : Noah Misch <noah@leadboat.com>
date : Sat, 31 Oct 2020 08:43:28 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 31 Oct 2020 08:43:28 -0700
Certain background workers initiate parallel queries while
debug_query_string==NULL, at which point they attempted strlen(NULL) and
died to SIGSEGV. Older debug_query_string observers allow NULL, so do
likewise in these newer ones. Back-patch to v11, where commit
7de4a1bcc56f494acbd0d6e70781df877dc8ecb5 introduced the first of these.
Discussion: https://postgr.es/m/20201014022636.GA1962668@rfd.leadboat.com
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtsort.c
Fix assertion failure in check_new_partition_bound().
commit : 970c05057593c2f5919a69b43fd917c4fa86f51c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Oct 2020 17:00:59 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Oct 2020 17:00:59 -0400
Commit 6b2c4e59d was overly confident about not being able to see
a negative cmpval result from partition_range_bsearch(). Adjust
the code to cope with that.
Report and patch by Amul Sul; some additional cosmetic changes by me
Discussion: https://postgr.es/m/CAAJ_b97WCO=EyVA7fKzc86kKfojHXLU04_zs7-7+yVzm=-1QkQ@mail.gmail.com
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/create_table.out
M src/test/regress/sql/create_table.sql
Fix missing validation for the new GiST sortsupport functions.
commit : 6f0bc5e1daf09686c526aa161da5336f7c94f4eb
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 30 Oct 2020 19:30:19 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 30 Oct 2020 19:30:19 +0200
Because of this, if you tried to create an operator family with the new
sortsupport function, you got an error:
ERROR: support function number 11 is invalid for access method gist
We missed this in commit 16fa9b2b30 that added the sortsupport function,
because it only added sortsupport to a built-in operator family.
Author: Andrey Borodin
Discussion: https://www.postgresql.org/message-id/3520A18A-5C38-4697-A2E3-F3BDE3496CD5%40yandex-team.ru
M src/backend/access/gist/gistvalidate.c
Doc: clarify description for pg_constraint.convalidated.
commit : b401fa206d446f224ec4760f21e0a351816c97b3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Oct 2020 10:38:49 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Oct 2020 10:38:49 -0400
Jimmy Angelakos
Discussion: https://postgr.es/m/CABgVKCW_zPnvFXn24FTF0299_yU6+1p6JRUc0xpiZFWEXH1_jg@mail.gmail.com
M doc/src/sgml/catalogs.sgml
Stabilize timetz test across DST transitions.
commit : 4a071afbd056282746a5bc9362e87f579a56402d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Oct 2020 15:28:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Oct 2020 15:28:14 -0400
The timetz test cases I added in commit a9632830b were unintentionally
sensitive to whether or not DST is active in the PST8PDT time zone.
Thus, they'll start failing this coming weekend, as reported by
Bernhard M. Wiedemann in bug #16689. Fortunately, DST-awareness is
not significant to the purpose of these test cases, so we can just
force them all to PDT (DST hours) to preserve stability of the
results.
Back-patch to v10, as the prior patch was.
Discussion: https://postgr.es/m/16689-57701daa23b377bf@postgresql.org
M src/test/regress/expected/timetz.out
M src/test/regress/sql/timetz.sql
Don't use custom OID symbols in pg_type.dat, either.
commit : f90149e6285aaae6b48559afce1bd638ee26c33e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Oct 2020 13:33:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Oct 2020 13:33:38 -0400
On the same reasoning as in commit 36b931214, forbid using custom
oid_symbol macros in pg_type as well as pg_proc, so that we always
rely on the predictable macro names generated by genbki.pl.
We do continue to grant grandfather status to the names CASHOID and
LSNOID, although those are now considered deprecated aliases for the
preferred names MONEYOID and PG_LSNOID. This is because there's
likely to be client-side code using the old names, and this bout of
neatnik-ism doesn't quite seem worth breaking client code.
There might be a case for grandfathering EVTTRIGGEROID, too, since
externally-maintained PLs may reference that symbol. But renaming
such references to EVENT_TRIGGEROID doesn't seem like a particularly
heavy lift --- we make far more significant backend API changes in
every major release. For now I didn't add that, but we could
reconsider if there's pushback.
The other names changed here seem pretty unlikely to have any outside
uses. Again, we could add alias macros if there are complaints, but
for now I didn't.
As before, no need for a catversion bump.
John Naylor
Discussion: https://postgr.es/m/CAFBsxsHpCbjfoddNGpnnnY5pHwckWfiYkMYSF74PmP1su0+ZOw@mail.gmail.com
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/genbki.pl
M src/backend/commands/event_trigger.c
M src/backend/utils/misc/pg_controldata.c
M src/fe_utils/print.c
M src/include/catalog/pg_type.dat
M src/include/catalog/pg_type.h
M src/interfaces/ecpg/ecpglib/execute.c
M src/pl/plperl/plperl.c
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_handler.c
M src/pl/plpython/plpy_procedure.c
M src/pl/tcl/pltcl.c
Fix wrong data table horizon computation during backend startup.
commit : 1c7675a7a4265064a2c8e1ed02b2c042c2521664
author : Andres Freund <andres@anarazel.de>
date : Wed, 28 Oct 2020 21:48:38 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 28 Oct 2020 21:48:38 -0700
When ComputeXidHorizons() was called before MyDatabaseOid is set,
e.g. because a dead row in a shared relation is encountered during
InitPostgres(), the horizon for normal tables was computed too
aggressively, ignoring all backends connected to a database.
During subsequent pruning in a data table the too aggressive horizon
could end up still being used, possibly leading to still needed tuples
being removed. Not good.
This is a bug in dc7420c2c92, which the test added in 94bc27b5768 made
visible, if run with force_parallel_mode set to regress. In that case
the bug is reliably triggered, because "pruning_query" is run in a
parallel worker and the start of that parallel worker is likely to
encounter a dead row in pg_database.
The fix is trivial: Compute a more pessimistic data table horizon if
MyDatabaseId is not yet known.
Author: Andres Freund
Discussion: https://postgr.es/m/20201029040030.p4osrmaywhqaesd4@alap3.anarazel.de
M src/backend/storage/ipc/procarray.c
Track statistics for streaming of changes from ReorderBuffer.
commit : 8e90ec5580d5345fef31005d7cc2215ba2125070
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 29 Oct 2020 09:11:51 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 29 Oct 2020 09:11:51 +0530
This adds the statistics about transactions streamed to the decoding
output plugin from ReorderBuffer. Users can query the
pg_stat_replication_slots view to check these stats and call
pg_stat_reset_replication_slot to reset the stats of a particular slot.
Users can pass NULL in pg_stat_reset_replication_slot to reset stats of
all the slots.
Commit 9868167500 has added the basic infrastructure to capture the stats
of slot and this commit extends the statistics collector to track
additional information about slots.
Bump the catversion as we have added new columns in the catalog entry.
Author: Ajin Cherian and Amit Kapila
Reviewed-by: Sawada Masahiko and Dilip Kumar
Discussion: https://postgr.es/m/CAA4eK1+chpEomLzgSoky-D31qev19AmECNiEAietPQUGEFhtVA@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/slot.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/replication/reorderbuffer.h
M src/test/regress/expected/rules.out
Centralize horizon determination for temp tables, fixing bug due to skew.
commit : 94bc27b57680b4e757577e3f5b65dc32f96d33c1
author : Andres Freund <andres@anarazel.de>
date : Wed, 28 Oct 2020 17:53:41 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 28 Oct 2020 17:53:41 -0700
This fixes a bug in the edge case where, for a temp table, heap_page_prune()
can end up with a different horizon than heap_vacuum_rel(). Which can trigger
errors like "ERROR: cannot freeze committed xmax ...".
The bug was introduced due to interaction of a7212be8b9e "Set cutoff xmin more
aggressively when vacuuming a temporary table." with dc7420c2c92 "snapshot
scalability: Don't compute global horizons while building snapshots.".
The problem is caused by lazy_scan_heap() assuming that the only reason its
HeapTupleSatisfiesVacuum() call would return HEAPTUPLE_DEAD is if the tuple is
a HOT tuple, or if the tuple's inserting transaction has aborted since the
heap_page_prune() call. But after a7212be8b9e that was also possible in other
cases for temp tables, because heap_page_prune() uses a different visibility
test after dc7420c2c92.
The fix is fairly simple: Move the special case logic for temp tables from
vacuum_set_xid_limits() to the infrastructure introduced in dc7420c2c92. That
ensures that the horizon used for pruning is at least as aggressive as the one
used by lazy_scan_heap(). The concrete horizon used for temp tables is
slightly different than the logic in dc7420c2c92, but should always be as
aggressive as before (see comments).
A significant benefit to centralizing the logic procarray.c is that now the
more aggressive horizons for temp tables does not just apply to VACUUM but
also to e.g. HOT pruning and the nbtree killtuples logic.
Because isTopLevel is not needed by vacuum_set_xid_limits() anymore, I
undid the the related changes from a7212be8b9e.
This commit also adds an isolation test ensuring that the more aggressive
vacuuming and pruning of temp tables keeps working.
Debugged-By: Amit Kapila <amit.kapila16@gmail.com>
Debugged-By: Tom Lane <tgl@sss.pgh.pa.us>
Debugged-By: Ashutosh Sharma <ashu.coek88@gmail.com>
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20201014203103.72oke6hqywcyhx7s@alap3.anarazel.de
Discussion: https://postgr.es/m/20201015083735.derdzysdtqdvxshp@alap3.anarazel.de
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/cluster.c
M src/backend/commands/vacuum.c
M src/backend/storage/ipc/procarray.c
M src/include/commands/cluster.h
M src/include/commands/vacuum.h
A src/test/isolation/expected/horizons.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/horizons.spec
Fix incorrect placement of pfree() in pg_relation_check_pages()
commit : 60a51c6b32960822d3987ea7d2816c65bdbcb314
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 29 Oct 2020 09:17:34 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 29 Oct 2020 09:17:34 +0900
This would cause the function to crash when more than one page is
considered as broken and reported in the SRF.
Reported-by: Noriyoshi Shinoda
Discussion: https://postgr.es/m/TU4PR8401MB11523D42C315AAF822E74275EE170@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM
M src/backend/utils/adt/pagefuncs.c
Doc: clean up pg_relation_check_pages() documentation.
commit : b787d4ce6d910080065025bcd5f968544997271f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 17:03:05 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 17:03:05 -0400
Commit f2b883969 did not get the memo about the new formatting
style for tables documenting built-in functions. I noticed because
of a PDF build warning about an overwidth table.
M doc/src/sgml/func.sgml
Doc: clean up verify_heapam() documentation.
commit : 4c49d8fc15eeb1dc69b0ddb2d986a1884a5d7f5f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 16:31:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 16:31:40 -0400
I started with the intention of just suppressing a PDF build warning
by removing the example output, but ended up doing more: correcting
factual errors in the function's signature, moving a bunch of
generalized handwaving into the "Using amcheck Effectively" section
which seemed a better place for it, and improving wording and markup
a little bit.
Discussion: https://postgr.es/m/732904.1603728748@sss.pgh.pa.us
M doc/src/sgml/amcheck.sgml
Use mode "r" for popen() in psql's evaluate_backtick().
commit : 66f8687a8ff867f656de81e367314604d29dbd59
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 14:35:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 14:35:53 -0400
In almost all other places, we use plain "r" or "w" mode in popen()
calls (the exceptions being for COPY data). This one has been
overlooked (possibly because it's buried in a ".l" flex file?),
but it's using PG_BINARY_R.
Kensuke Okamura complained in bug #16688 that we fail to strip \r
when stripping the trailing newline from a backtick result string.
That's true enough, but we'd also fail to convert embedded \r\n
cleanly, which also seems undesirable. Fixing the popen() mode
seems like the best way to deal with this.
It's been like this for a long time, so back-patch to all supported
branches.
Discussion: https://postgr.es/m/16688-c649c7b69cd7e6f8@postgresql.org
M src/bin/psql/psqlscanslash.l
Calculate extraUpdatedCols in query rewriter, not parser.
commit : ad77039fad0f4128b0e4a05ddbf5dbc3ab5f3fa4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 13:47:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 13:47:02 -0400
It's unsafe to do this at parse time because addition of generated
columns to a table would not invalidate stored rules containing
UPDATEs on the table ... but there might now be dependent generated
columns that were not there when the rule was made. This also fixes
an oversight that rewriteTargetView failed to update extraUpdatedCols
when transforming an UPDATE on an updatable view. (Since the new
calculation is downstream of that, rewriteTargetView doesn't actually
need to do anything; but before, there was a demonstrable bug there.)
In v13 and HEAD, this leads to easily-visible bugs because (since
commit c6679e4fc) we won't recalculate generated columns that aren't
listed in extraUpdatedCols. In v12 this bitmap is mostly just used
for trigger-firing decisions, so you'd only notice a problem if a
trigger cared whether a generated column had been updated.
I'd complained about this back in May, but then forgot about it
until bug #16671 from Michael Paul Killian revived the issue.
Back-patch to v12 where this field was introduced. If existing
stored rules contain any extraUpdatedCols values, they'll be
ignored because the rewriter will overwrite them, so the bug will
be fixed even for existing rules. (But note that if someone were
to update to 13.1 or 12.5, store some rules with UPDATEs on tables
having generated columns, and then downgrade to a prior minor version,
they might observe issues similar to what this patch fixes. That
seems unlikely enough to not be worth going to a lot of effort to fix.)
Discussion: https://postgr.es/m/10206.1588964727@sss.pgh.pa.us
Discussion: https://postgr.es/m/16671-2fa55851859fb166@postgresql.org
M src/backend/optimizer/plan/setrefs.c
M src/backend/parser/analyze.c
M src/backend/replication/logical/worker.c
M src/backend/rewrite/rewriteHandler.c
M src/include/nodes/parsenodes.h
M src/include/parser/analyze.h
M src/include/rewrite/rewriteHandler.h
M src/test/regress/expected/updatable_views.out
M src/test/regress/sql/updatable_views.sql
Don't use custom OID symbols in pg_proc.dat.
commit : 36b93121436cbbf357974144068c23bac75154fa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 12:18:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 12:18:45 -0400
We have a perfectly good convention for OID macros for built-in functions
already, so making custom symbols is just introducing unnecessary
deviation from the convention. Remove the one case that had snuck in,
and add an error check in genbki.pl to discourage future instances.
Although this touches pg_proc.dat, there's no need for a catversion
bump since the actual catalog data isn't changed.
John Naylor
Discussion: https://postgr.es/m/CAFBsxsHpCbjfoddNGpnnnY5pHwckWfiYkMYSF74PmP1su0+ZOw@mail.gmail.com
M src/backend/catalog/genbki.pl
M src/backend/utils/cache/relcache.c
M src/include/catalog/pg_proc.dat
Fix foreign-key selectivity estimation in the presence of constants.
commit : ad1c36b0709e47cdb3cc4abd6c939fe64279b63f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 11:15:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 11:15:47 -0400
get_foreign_key_join_selectivity() looks for join clauses that equate
the two sides of the FK constraint. However, if we have a query like
"WHERE fktab.a = pktab.a and fktab.a = 1", it won't find any such join
clause, because equivclass.c replaces the given clauses with "fktab.a
= 1 and pktab.a = 1", which can be enforced at the scan level, leaving
nothing to be done for column "a" at the join level.
We can fix that expectation without much trouble, but then a new problem
arises: applying the foreign-key-based selectivity rule produces a
rowcount underestimate, because we're effectively double-counting the
selectivity of the "fktab.a = 1" clause. So we have to cancel that
selectivity out of the estimate.
To fix, refactor process_implied_equality() so that it can pass back the
new RestrictInfo to its callers in equivclass.c, allowing the generated
"fktab.a = 1" clause to be saved in the EquivalenceClass's ec_derives
list. Then it's not much trouble to dig out the relevant RestrictInfo
when we need to adjust an FK selectivity estimate. (While at it, we
can also remove the expensive use of initialize_mergeclause_eclasses()
to set up the new RestrictInfo's left_ec and right_ec pointers.
The equivclass.c code can set those basically for free.)
This seems like clearly a bug fix, but I'm hesitant to back-patch it,
first because there's some API/ABI risk for extensions and second because
we're usually loath to destabilize plan choices in stable branches.
Per report from Sigrid Ehrenreich.
Discussion: https://postgr.es/m/1019549.1603770457@sss.pgh.pa.us
Discussion: https://postgr.es/m/AM6PR02MB5287A0ADD936C1FA80973E72AB190@AM6PR02MB5287.eurprd02.prod.outlook.com
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/util/plancat.c
M src/include/nodes/pathnodes.h
M src/include/optimizer/paths.h
M src/include/optimizer/planmain.h
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Use correct GetDatum() in pg_relation_check_pages()
commit : ce7f772c5e6066e0bbafea5759e652c9757c8e6b
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Oct 2020 13:59:18 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Oct 2020 13:59:18 +0900
UInt32GetDatum() was getting used, while the result needs
Int64GetDatum(). Oversight in f2b8839.
Per buildfarm member florican.
Discussion: https://postgr.es/m/1226629.1603859189@sss.pgh.pa.us
M src/backend/utils/adt/pagefuncs.c
Add pg_relation_check_pages() to check on-disk pages of a relation
commit : f2b883969557f4572cdfa87e1a40083d2b1272e7
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Oct 2020 12:15:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Oct 2020 12:15:00 +0900
This makes use of CheckBuffer() introduced in c780a7a, adding a SQL
wrapper able to do checks for all the pages of a relation. By default,
all the fork types of a relation are checked, and it is possible to
check only a given relation fork. Note that if the relation given in
input has no physical storage or is temporary, then no errors are
generated, allowing full-database checks when coupled with a simple scan
of pg_class for example. This is not limited to clusters with data
checksums enabled, as clusters without data checksums can still apply
checks on pages using the page headers or for the case of a page full of
zeros.
This function returns a set of tuples consisting of:
- The physical file where a broken page has been detected (without the
segment number as that can be AM-dependent, which can be guessed from
the block number for heap). A relative path from PGPATH is used.
- The block number of the broken page.
By default, only superusers have an access to this function but
execution rights can be granted to other users.
The feature introduced here is still minimal, and more improvements
could be done, like:
- Addition of a start and end block number to run checks on a range
of blocks, which would apply only if one fork type is checked.
- Addition of some progress reporting.
- Throttling, with configuration parameters in function input or
potentially some cost-based GUCs.
Regression tests are added for positive cases in the main regression
test suite, and TAP tests are added for cases involving the emulation of
page corruptions.
Bump catalog version.
Author: Julien Rouhaud, Michael Paquier
Reviewed-by: Masahiko Sawada, Justin Pryzby
Discussion: https://postgr.es/m/CAOBaU_aVvMjQn=ge5qPiJOPMmOj5=ii3st5Q0Y+WuLML5sR17w@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/catalog/system_views.sql
M src/backend/utils/adt/Makefile
A src/backend/utils/adt/pagefuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
A src/test/recovery/t/022_page_check.pl
A src/test/regress/expected/pagefuncs.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/pagefuncs.sql
M src/tools/pgindent/typedefs.list
Add CheckBuffer() to check on-disk pages without shared buffer loading
commit : c780a7a90a8e7b074405ea2007e34f94e227e695
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Oct 2020 11:12:46 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Oct 2020 11:12:46 +0900
CheckBuffer() is designed to be a concurrent-safe function able to run
sanity checks on a relation page without loading it into the shared
buffers. The operation is done using a lock on the partition involved
in the shared buffer mapping hashtable and an I/O lock for the buffer
itself, preventing the risk of false positives due to any concurrent
activity.
The primary use of this function is the detection of on-disk corruptions
for relation pages. If a page is found in shared buffers, the on-disk
page is checked if not dirty (a follow-up checkpoint would flush a valid
version of the page if dirty anyway), as it could be possible that a
page was present for a long time in shared buffers with its on-disk
version corrupted. Such a scenario could lead to a corrupted cluster if
a host is plugged off for example. If the page is not found in shared
buffers, its on-disk state is checked. PageIsVerifiedExtended() is used
to apply the same sanity checks as when a page gets loaded into shared
buffers.
This function will be used by an upcoming patch able to check the state
of on-disk relation pages using a SQL function.
Author: Julien Rouhaud, Michael Paquier
Reviewed-by: Masahiko Sawada
Discussion: https://postgr.es/m/CAOBaU_aVvMjQn=ge5qPiJOPMmOj5=ii3st5Q0Y+WuLML5sR17w@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
M src/include/storage/bufmgr.h
Minor improvements in description of spilled counters in pg_stat_replication_slots view.
commit : 9e0f87a4955ede0769d2f9a93171145b6ddab901
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 28 Oct 2020 07:28:51 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 28 Oct 2020 07:28:51 +0530
Per a suggestion by Justin Pryzby.
Discussion: https://postgr.es/m/CA+fd4k5_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com
M doc/src/sgml/monitoring.sgml
Doc: improve explanation of how to use our code coverage infrastructure.
commit : 8d132b2850d4c95ed5666a09b295dea645bbc884
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 27 Oct 2020 14:31:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 27 Oct 2020 14:31:12 -0400
The reference to running "make coverage" in a subdirectory was a
bit obscure, so clarify what happens when you do that. Do a little
desultory copy-editing, too.
Per a question from Peter Smith.
Discussion: https://postgr.es/m/CAHut+Pu0r3AjRSyu5E0v2-zRj8r24OSrkWs3fEBxOuaw1i8DKA@mail.gmail.com
M doc/src/sgml/regress.sgml
Makefile comment: remove reference to tools/thread/thread_test
commit : 8e5d1aef8eed26977739b662cddcea2de5efe834
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 27 Oct 2020 14:00:38 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 27 Oct 2020 14:00:38 -0400
You can't compile thread_test alone anymore, and the location moved too.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/1062278.1603819969@sss.pgh.pa.us
Backpatch-through: 9.5
M src/template/netbsd
pg_dump: Lock all relations, not just plain tables
commit : 403a3d91c841beabf3efd7bffddb47a2bce4481f
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Oct 2020 14:31:37 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Oct 2020 14:31:37 -0300
Now that LOCK TABLE can take any relation type, acquire lock on all
relations that are to be dumped. This prevents schema changes or
deadlock errors that could cause a dump to fail after expending much
effort. The server is tested to have the capability and the feature
disabled if it doesn't, so that a patched pg_dump doesn't fail when
connecting to an unpatched server.
Backpatch to 9.5.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Wells Oliver <wells.oliver@gmail.com>
Discussion: https://postgr.es/m/20201021200659.GA32358@alvherre.pgsql
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_backup_db.h
M src/bin/pg_dump/pg_dump.c
Add select_common_typmod()
commit : f893e68d761adbee7f888109b1adf76151e3e17a
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 27 Oct 2020 17:39:23 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 27 Oct 2020 17:39:23 +0100
This accompanies select_common_type() and select_common_collation().
Typmods were previously combined using hand-coded logic in several
places. The logic in select_common_typmod() isn't very exciting, but
it makes the code more compact and readable in a few locations, and in
the future we can perhaps do more complicated things if desired.
As a small enhancement, the type unification of the direct and
aggregate arguments of hypothetical-set aggregates now unifies the
typmod as well using this new function, instead of just dropping it.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/97df3af9-8b5e-fb7f-a029-3eb7e80d7af9@2ndquadrant.com
M src/backend/parser/analyze.c
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_coerce.c
M src/backend/parser/parse_func.c
M src/include/parser/parse_coerce.h
Accept relations of any kind in LOCK TABLE
commit : 59ab4ac32460a6a93b665f4e487d7ff64979ba4d
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Oct 2020 13:49:19 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Oct 2020 13:49:19 -0300
The restriction that only tables and views can be locked by LOCK TABLE
is quite arbitrary, since the underlying mechanism can lock any relation
type. Drop the restriction so that programs such as pg_dump can lock
all relations they're interested in, preventing schema changes that
could cause a dump to fail after expending much effort.
Backpatch to 9.5.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Wells Oliver <wells.oliver@gmail.com>
Discussion: https://postgr.es/m/20201021200659.GA32358@alvherre.pgsql
M doc/src/sgml/ref/lock.sgml
M src/backend/commands/lockcmds.c
M src/test/regress/expected/lock.out
M src/test/regress/sql/lock.sql
docs: remove reference to src/tools/thread
commit : 4066b642909176aede03b6c64d29734ce2a34716
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 27 Oct 2020 12:43:11 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 27 Oct 2020 12:43:11 -0400
This directory and the ability to build the thread test independently
were removed in commit 8a2121185b.
Reported-by: e.indrupskaya@postgrespro.ru
Discussion: https://postgr.es/m/160379609706.24746.7506163279454026608@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
Fix enum errdetail to mention bytes, not chars
commit : 0525572860335d050a1bea194a5278c8833304d1
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 27 Oct 2020 11:50:18 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 27 Oct 2020 11:50:18 +0100
The enum label length is in terms of bytes, not charactes.
Author: Ian Lawrence Barwick <barwick@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAB8KJ=itZEJ7C9BacTHSYgeUysH4xx8wDiOnyppnSLyn6-g+Bw@mail.gmail.com
M src/backend/catalog/pg_enum.c
M src/test/regress/expected/enum.out
Make procedure OUT parameters work with JDBC
commit : 9213462c539e6412fe0498a7f8e20b662e15c4ec
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 27 Oct 2020 08:58:48 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 27 Oct 2020 08:58:48 +0100
The JDBC driver sends OUT parameters with type void. This makes sense
when calling a function, so that the parameters are ignored in
ParseFuncOrColumn(). For a procedure call we want to treat them as
unknown.
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://www.postgresql.org/message-id/flat/d7e49540-ea92-b4e2-5fff-42036102f968%402ndquadrant.com
M src/backend/parser/parse_param.c
doc: Fix order of protocol messages in listing
commit : 8fed2eadb8558185d162cc7fd09192c2e10b915f
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 27 Oct 2020 08:43:35 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 27 Oct 2020 08:43:35 +0100
Move GSSENCRequest to the correct alphabetical position.
M doc/src/sgml/protocol.sgml
Update description of spilled counters in pg_stat_replication_slots view.
commit : 2f0760c9ff827bb3d23ee327e6b46038000c7ef9
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 27 Oct 2020 08:22:39 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 27 Oct 2020 08:22:39 +0530
This is to make the description of spilled counters clear.
Author: Amit Kapila
Reviewed-by: Sawada Masahiko
Discussion: https://postgr.es/m/CA+fd4k5_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com
M doc/src/sgml/monitoring.sgml
doc: simplify wording of function error affects
commit : 2f17fe431860fc434084d99d8b789e6ec7e6977b
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Oct 2020 22:38:11 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Oct 2020 22:38:11 -0400
Reported-by: bob.henkel@gmail.com
Discussion: https://postgr.es/m/160324449781.693.8298142858847611071@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/plpgsql.sgml
doc: make blooms docs match reality
commit : e9661f2b0f3310d8bacc3f0802b309f599300cfb
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Oct 2020 19:17:05 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Oct 2020 19:17:05 -0400
Parallel execution changed the way bloom queries are executed, so update
the EXPLAIN output, and restructure the docs to be clearer and more
accurate.
Reported-by: Daniel Westermann
Discussion: https://postgr.es/m/ZR0P278MB0122119FAE78721A694C30C8D2340@ZR0P278MB0122.CHEP278.PROD.OUTLOOK.COM
Author: Daniel Westermann and me
Backpatch-through: 9.6
M doc/src/sgml/bloom.sgml
In INSERT/UPDATE, use the table's real tuple descriptor as target.
commit : 20d3fe9009ddbbbb3da3a2da298f922054b43f8c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 26 Oct 2020 11:36:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 26 Oct 2020 11:36:53 -0400
Previously, ExecInitModifyTable relied on ExecInitJunkFilter,
and thence ExecCleanTypeFromTL, to build the target descriptor from
the query tlist. While we just checked (in ExecCheckPlanOutput)
that the tlist produces compatible output, this is not a great
substitute for the relation's actual tuple descriptor that's
available from the relcache. For one thing, dropped columns will
not be correctly marked attisdropped; it's a bit surprising that
we've gotten away with that this long. But the real reason for
being concerned with this is that using the table's descriptor means
that the slot will have correct attrmissing data, allowing us to
revert the klugy fix of commit ba9f18abd. (This commit undoes
that one's changes in trigger.c, but keeps the new test case.)
Thus we can solve the bogus-trigger-tuple problem with fewer cycles
rather than more.
No back-patch, since this doesn't fix any additional bug, and it
seems somewhat more likely to have unforeseen side effects than
ba9f18abd's narrow fix.
Discussion: https://postgr.es/m/16644-5da7ef98a7ac4545@postgresql.org
M src/backend/commands/trigger.c
M src/backend/executor/execJunk.c
M src/backend/executor/nodeModifyTable.c
M src/include/executor/executor.h
docs: Remove notes about incompatibilies with very old versions.
commit : fa42c2ecb0f6e89f74bc1cc37b56a1d43e45d513
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 26 Oct 2020 09:07:14 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 26 Oct 2020 09:07:14 +0200
These are old enough that they'll cause more confusion and distraction
to new readers, than they could help anyone upgrade from very old
servers.
Discussion: https://www.postgresql.org/message-id/fd93f1c5-7818-a02c-01e5-1075ac0d4def%40iki.fi
M doc/src/sgml/func.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/pgfreespacemap.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
Extend PageIsVerified() to handle more custom options
commit : d401c5769ef6aeef0a28c147f3fb5afedcd59984
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 26 Oct 2020 09:55:28 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 26 Oct 2020 09:55:28 +0900
This is useful for checks of relation pages without having to load the
pages into the shared buffers, and two cases can make use of that: page
verification in base backups and the online, lock-safe, flavor.
Compatibility is kept with past versions using a macro that calls the
new extended routine with the set of options compatible with the
original version.
Extracted from a larger patch by the same author.
Author: Anastasia Lubennikova
Reviewed-by: Michael Paquier, Julien Rouhaud
Discussion: https://postgr.es/m/608f3476-0598-2514-2c03-e05c7d2b0cbd@postgrespro.ru
M src/backend/catalog/storage.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/page/bufpage.c
M src/include/storage/bufpage.h
Fix corner case for a BEFORE ROW UPDATE trigger returning OLD.
commit : ba9f18abd3650e385e9a35df7145a7c38af17e92
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 25 Oct 2020 13:57:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 25 Oct 2020 13:57:46 -0400
If the old row has any "missing" attributes that are supposed to
be retrieved from an associated tuple descriptor, the wrong things
happened because the trigger result is shoved directly into an
executor slot that lacks the missing-attribute data. Notably,
CHECK-constraint verification would incorrectly see those columns
as NULL, and so would RETURNING-list evaluation.
Band-aid around this by forcibly expanding the tuple before passing
it to the trigger function. (IMO it was a fundamental misdesign to
put the missing-attribute data into tuple constraints, which so
much of the system considers to be optional. But we're probably
stuck with that now, and will have to continue to apply band-aids
as we find other places with similar issues.)
Back-patch to v12. v11 would also have the issue, except that
commit 920311ab1 already applied a similar band-aid. That forced
expansion in more cases than seem really necessary, though, so
this isn't a directly equivalent fix.
Amit Langote, with some cosmetic changes by me
Discussion: https://postgr.es/m/16644-5da7ef98a7ac4545@postgresql.org
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Fix incorrect parameter name in a function header comment
commit : e83c9f913c6197586af8ac53c1d3652db15a3c91
author : David Rowley <drowley@postgresql.org>
date : Sun, 25 Oct 2020 22:39:00 +1300
committer: David Rowley <drowley@postgresql.org>
date : Sun, 25 Oct 2020 22:39:00 +1300
Author: Zhijie Hou
Discussion: https://postgr.es/m/14cd74ea00204cc8a7ea5d738ac82cd1@G08CNEXMBPEKD05.g08.fujitsu.local
Backpatch-through: 12, where the mistake was introduced
M src/backend/utils/cache/lsyscache.c
Fix ancient bug in ecpg's pthread_once() emulation for Windows.
commit : 21d36747d4fafe16539a0c55ebb91a01e4053e3c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 24 Oct 2020 13:12:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 24 Oct 2020 13:12:08 -0400
We must not set the "done" flag until after we've executed the
initialization function. Otherwise, other threads can fall through
the initial unlocked test before initialization is really complete.
This has been seen to cause rare failures of ecpg's thread/descriptor
test, and it could presumably cause other sorts of misbehavior in
threaded ECPG-using applications, since ecpglib relies on
pthread_once() in several places.
Diagnosis and patch by me, based on investigation by Alexander Lakhin.
Back-patch to all supported branches (the bug dates to 2007).
Discussion: https://postgr.es/m/16685-d6cd241872c101d3@postgresql.org
M src/interfaces/ecpg/ecpglib/misc.c
Fix issue with --enable-coverage and the new unicode {de,re}composition code
commit : 2771fcee18be0b86d2e008add20f73d175e06e90
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 24 Oct 2020 14:20:38 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 24 Oct 2020 14:20:38 +0900
genhtml has been generating the following warning with this new code:
WARNING: function data mismatch at /path/src/common/unicode_norm.c:102
HTML coverage reports care about the uniqueness of functions defined in
source files, ignoring any assumptions around CFLAGS. 783f0cc
introduced a duplicated definition of get_code_entry(), leading to a
warning and potentially some incorrect data generated in the reports.
This refactors the code so as the code has only one function
declaration, fixing the warning.
Oversight in 783f0cc.
Reported-by: Tom Lane
Author: Michael Paquier
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/207789.1603469272@sss.pgh.pa.us
M src/common/unicode_norm.c
Add tab completion for ALTER TABLE .. FORCE ROW LEVEL SECURITY in psql
commit : 0b46e82c06b0c4b0dc6a94a890d23945ebf720fd
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 24 Oct 2020 10:29:55 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 24 Oct 2020 10:29:55 +0900
This completes both the FORCE and NO FORCE options, NO INHERIT needing a
small adjustment.
Author: Li Japin
Discussion: https://postgr.es/m/15B10F9F-5847-4F5E-BD66-8E25AA473C95@hotmail.com
M src/bin/psql/tab-complete.c
Fix more portability issues in new amcheck code.
commit : 321633e17b07968e68ca5341429e2c8bbf15c331
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 23 Oct 2020 19:08:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 23 Oct 2020 19:08:01 -0400
verify_heapam() wasn't being careful to sanity-check tuple line
pointers before using them, resulting in SIGBUS on alignment-picky
architectures. Fix that, add some more test coverage.
Mark Dilger, some tweaking by me
Discussion: https://postgr.es/m/30B8E99A-2D9C-48D4-A55C-741C9D5F1563@enterprisedb.com
M contrib/amcheck/t/001_verify_heapam.pl
M contrib/amcheck/verify_heapam.c
Allow psql to re-use connection parameters after a connection loss.
commit : 1b62d0fb3e50ede570d0d4e4a2be69d5645b48a7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 23 Oct 2020 17:07:15 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 23 Oct 2020 17:07:15 -0400
Instead of immediately PQfinish'ing a dead connection, save it aside
so that we can still extract its parameters for \connect attempts.
(This works because PQconninfo doesn't care whether the PGconn is in
CONNECTION_BAD state.) This allows developers to reconnect with
just \c after a database crash and restart.
It's tempting to use the same approach instead of closing the old
connection after a failed non-interactive \connect command. However,
that would not be very safe: consider a script containing
\c db1 user1 live_server
\c db2 user2 dead_server
\c db3
The script would be expecting to connect to db3 at dead_server, but
if we re-use parameters from the first connection then it might
successfully connect to db3 at live_server. This'd defeat the goal
of not letting a script accidentally execute commands against the
wrong database.
Discussion: https://postgr.es/m/38464.1603394584@sss.pgh.pa.us
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/common.c
M src/bin/psql/describe.c
M src/bin/psql/settings.h
M src/bin/psql/startup.c
Fix portability issues in new amcheck test.
commit : 860593ec3bd15e8969effdfcb5cbd98c561dd722
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 23 Oct 2020 14:02:00 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 23 Oct 2020 14:02:00 -0400
The tests added by commit 866e24d47 failed on big-endian machines
due to lack of attention to endianness considerations. Fix that.
While here, improve a few small cosmetic things, such as running
it through perltidy.
Mark Dilger
Discussion: https://postgr.es/m/30B8E99A-2D9C-48D4-A55C-741C9D5F1563@enterprisedb.com
M contrib/amcheck/t/001_verify_heapam.pl
Fix broken XML formatting in EXPLAIN output for incremental sorts.
commit : 87a174c0e77eed0bec7d53ef6d470d60335f6444
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 23 Oct 2020 11:32:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 23 Oct 2020 11:32:33 -0400
The ExplainCloseGroup arguments for incremental sort usage data
didn't match the corresponding ExplainOpenGroup. This only matters
for XML-format output, which is probably why we'd not noticed.
Daniel Gustafsson, per bug #16683 from Frits Jalvingh
Discussion: https://postgr.es/m/16683-8005033324ad34e9@postgresql.org
M src/backend/commands/explain.c
doc: Fix order of protocol messages in listing
commit : 83d727e5b23c43f3fb7221963ddec24277c1126f
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 23 Oct 2020 13:01:39 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 23 Oct 2020 13:01:39 +0200
The order of AuthenticationGSSContinue and AuthenticationSSPI was
swapped, based on the other Authentication* protocol messages being
listed in subcode order.
M doc/src/sgml/protocol.sgml
doc: Remove reference to pre-8.2 pg_dump behaviour
commit : 902b57c9bf27b3ea2963ef832cddb38b1203130e
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 23 Oct 2020 11:49:59 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 23 Oct 2020 11:49:59 +0300
The behavioural change in the -t/--table option happened around 15 years
ago and there seems little point in keeping it around.
Author: Ian Barwick
Discussion: https://www.postgresql.org/message-id/CAB8KJ%3Dh-XALik4M7gv-pX48%3D%2BSPWexfaYwa%2ByTnPwD3DxceXrg%40mail.gmail.com
M doc/src/sgml/ref/pg_dump.sgml
Fix initialization of es_result_relations in EvalPlanQualStart().
commit : 22b73d3cb0b5bb4c141421f98dd67f091dda3c20
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 23 Oct 2020 09:30:08 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 23 Oct 2020 09:30:08 +0300
Thinko in commit 1375422c782. EvalPlanQualStart() was mistakenly
resetting the parent EState's es_result_relations, when it should
initialize the field in the child EPQ EState it just created.
That was clearly wrong, but it didn't cause any ill effects, because
es_result_relations is currently not used after the ExecInit* phase.
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqFEuq8AAAmxXsTDVZ1r38cHbfYuiPQx_%3DYyKe2DC-6q4A%40mail.gmail.com
M src/backend/executor/execMain.c
Improve performance of Unicode {de,re}composition in the backend
commit : 783f0cc64dcc05e3d112a06b1cd181e5a1ca9099
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 23 Oct 2020 11:05:46 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 23 Oct 2020 11:05:46 +0900
This replaces the existing binary search with two perfect hash functions
for the composition and the decomposition in the backend code, at the
cost of slightly-larger binaries there (35kB in libpgcommon_srv.a). Per
the measurements done, this improves the speed of the recomposition and
decomposition by up to 30~40 times for the NFC and NFKC conversions,
while all other operations get at least 40% faster. This is not as
"good" as what libicu has, but it closes the gap a lot as per the
feedback from Daniel Verite.
The decomposition table remains the same, getting used for the binary
search in the frontend code, where we care more about the size of the
libraries like libpq over performance as this gets involved only in code
paths related to the SCRAM authentication. In consequence, note that
the perfect hash function for the recomposition needs to use a new
inverse lookup array back to to the existing decomposition table.
The size of all frontend deliverables remains unchanged, even with
--enable-debug, including libpq.
Author: John Naylor
Reviewed-by: Michael Paquier, Tom Lane
Discussion: https://postgr.es/m/CAFBsxsHUuMFCt6-pU+oG-F1==CmEp8wR+O+bRouXWu6i8kXuqA@mail.gmail.com
M src/common/unicode/Makefile
M src/common/unicode/generate-unicode_norm_table.pl
M src/common/unicode_norm.c
A src/include/common/unicode_norm_hashfunc.h
M src/tools/pgindent/exclude_file_patterns
Update time zone data files to tzdata release 2020d.
commit : 7d6d6bce43c60bb7b77237e2cc6ab845646b911f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 21:23:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 21:23:47 -0400
DST law changes in Palestine, with a whopping 120 hours' notice.
Also some historical corrections for Palestine.
M src/timezone/data/tzdata.zi
Sync our copy of the timezone library with IANA release tzcode2020d.
commit : c5054da0d7ee59a93ca6d7f7fc669ef0210dbad3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 21:15:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 21:15:22 -0400
There's no functional change at all here, but I'm curious to see
whether this change successfully shuts up Coverity's warning about
a useless strcmp(), which appeared with the previous update.
Discussion: http://mm.icann.org/pipermail/tz/2020-October/029370.html
M src/timezone/README
M src/timezone/zic.c
Add documentation and tests for quote marks in ECPG literal queries.
commit : c16a1bbcf498f0aa053a3e55008f57d7f67357dd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 18:29:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 18:29:40 -0400
ECPG's PREPARE ... FROM and EXECUTE IMMEDIATE can optionally take
the target query as a simple literal, rather than the more usual
string-variable reference. This was previously documented as
being a C string literal, but that's a lie in one critical respect:
you can't write a data double quote as \" in such literals. That's
because the lexer is in SQL mode at this point, so it'll parse
double-quoted strings as SQL identifiers, within which backslash
is not special, so \" ends the literal.
I looked into making this work as documented, but getting the lexer
to switch behaviors at just the right point is somewhere between
very difficult and impossible. It's not really worth the trouble,
because these cases are next to useless: if you have a fixed SQL
statement to execute or prepare, you might as well write it as
a direct EXEC SQL, saving the messiness of converting it into
a string literal and gaining the opportunity for compile-time
SQL syntax checking.
Instead, let's just document (and test) the workaround of writing
a double quote as an octal escape (\042) in such cases.
There's no code behavioral change here, so in principle this could
be back-patched, but it's such a niche case I doubt it's worth
the trouble.
Per report from 1250kv.
Discussion: https://postgr.es/m/673825.1603223178@sss.pgh.pa.us
M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/preproc/pgc.l
M src/interfaces/ecpg/test/expected/sql-execute.c
M src/interfaces/ecpg/test/expected/sql-execute.stderr
M src/interfaces/ecpg/test/sql/execute.pgc
Avoid premature de-doubling of quote marks in ECPG strings.
commit : 3dfb1942d9b8748b93094a430289e2f7f3b3ae0d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 17:34:32 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 17:34:32 -0400
If you write the literal 'abc''def' in an EXEC SQL command, that will
come out the other end as 'abc'def', triggering a syntax error in the
backend. Likewise, "abc""def" is reduced to "abc"def" which is wrong
syntax for a quoted identifier.
The cause is that the lexer thinks it should emit just one quote
mark, whereas what it really should do is keep the string as-is.
Add some docs and test cases, too.
Although this seems clearly a bug, I fear users wouldn't appreciate
changing it in minor releases. Some may well be working around it
by applying an extra doubling of affected quotes, as for example
sql/dyntest.pgc has been doing.
Per investigation of a report from 1250kv, although this isn't
exactly what he/she was on about.
Discussion: https://postgr.es/m/673825.1603223178@sss.pgh.pa.us
M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/preproc/pgc.l
M src/interfaces/ecpg/test/expected/preproc-strings.c
M src/interfaces/ecpg/test/expected/preproc-strings.stderr
M src/interfaces/ecpg/test/expected/preproc-strings.stdout
M src/interfaces/ecpg/test/preproc/strings.pgc
M src/interfaces/ecpg/test/sql/dyntest.pgc
Try to avoid a compiler warning about using fxid uninitialized.
commit : 8bb0c9770e80fa79f15b27af8f3c3f2833225aa3
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 22 Oct 2020 16:13:58 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 22 Oct 2020 16:13:58 -0400
Mark Dilger, with a couple of stray semicolons removed by me.
Discussion: http://postgr.es/m/2A7DA1A8-C4AA-43DF-A985-3CA52F4DC775@enterprisedb.com
M contrib/amcheck/verify_heapam.c
Clean up some unpleasant behaviors in psql's \connect command.
commit : 94929f1cf6cb3ea070d0919d1303379b525a72a3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 14:04:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 14:04:21 -0400
The check for whether to complain about not having an old connection
to get parameters from was seriously out of date: it had not been
rethought when we invented connstrings, nor when we invented the
-reuse-previous option. Replace it with a check that throws an
error if reuse-previous is active and we lack an old connection to
reuse. While that doesn't move the goalposts very far in terms of
easing reconnection after a server crash, at least it's consistent.
If the user specifies a connstring plus additional parameters
(which is invalid per the documentation), the extra parameters were
silently ignored. That seems like it could be really confusing,
so let's throw a syntax error instead.
Teach the connstring code path to re-use the old connection's password
in the same cases as the old-style-syntax code path would, ie if we
are reusing parameters and the values of username, host/hostaddr, and
port are not being changed. Document this behavior, too, since it was
unmentioned before. Also simplify the implementation a bit, giving
rise to two new and useful properties: if there's a "password=xxx" in
the connstring, we'll use it not ignore it, and by default (i.e.,
except with --no-password) we will prompt for a password if the
re-used password or connstring password doesn't work. The previous
code just failed if the re-used password didn't work.
Given the paucity of field complaints about these issues, I don't
think that they rise to the level of back-patchable bug fixes,
and in any case they might represent undesirable behavior changes
in minor releases. So no back-patch.
Discussion: https://postgr.es/m/235210.1603321144@sss.pgh.pa.us
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
Extend amcheck to check heap pages.
commit : 866e24d47db1743dfcff5bd595b57e3a143f2cb1
author : Robert Haas <rhaas@postgresql.org>
date : Thu, 22 Oct 2020 08:44:18 -0400
committer: Robert Haas <rhaas@postgresql.org>
date : Thu, 22 Oct 2020 08:44:18 -0400
Mark Dilger, reviewed by Peter Geoghegan, Andres Freund, Álvaro Herrera,
Michael Paquier, Amul Sul, and by me. Some last-minute cosmetic
revisions by me.
Discussion: http://postgr.es/m/12ED3DA8-25F0-4B68-937D-D907CFBF08E7@enterprisedb.com
M contrib/amcheck/Makefile
A contrib/amcheck/amcheck–1.2–1.3.sql
M contrib/amcheck/amcheck.control
A contrib/amcheck/expected/check_heap.out
A contrib/amcheck/sql/check_heap.sql
A contrib/amcheck/t/001_verify_heapam.pl
A contrib/amcheck/verify_heapam.c
M doc/src/sgml/amcheck.sgml
M src/backend/access/heap/hio.c
M src/backend/access/transam/multixact.c
M src/include/access/multixact.h
M src/tools/pgindent/typedefs.list
Use croak instead of die in Perl code when appropriate
commit : f8721bd752790859df747905bc44fb5ad8dbf07d
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 22 Oct 2020 13:29:39 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 22 Oct 2020 13:29:39 +0200
M src/test/perl/TestLib.pm
Optimize a few list_delete_ptr calls
commit : e7c2b95d37a2b9c01367f7ccc55703555b39c81c
author : David Rowley <drowley@postgresql.org>
date : Thu, 22 Oct 2020 14:36:32 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 22 Oct 2020 14:36:32 +1300
There is a handful of places where we called list_delete_ptr() to remove
some element from a List. In many of these places we know, or with very
little additional effort know the index of the ListCell that we need to
remove.
Here we change all of those places to instead either use one of;
list_delete_nth_cell(), foreach_delete_current() or list_delete_last().
Each of these saves from having to iterate over the list to search for the
element to remove by its pointer value.
There are some small performance gains to be had by doing this, but in the
general case, none of these lists are likely to be very large, so the
lookup was probably never that expensive anyway. However, some of the
calls are in fairly hot code paths, e.g process_equivalence(). So any
small gains there are useful.
Author: Zhijie Hou and David Rowley
Discussion: https://postgr.es/m/b3517353ec7c4f87aa560678fbb1034b@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/path/joinpath.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_utilcmd.c
M src/backend/rewrite/rewriteHandler.c
Fix connection string handling in psql's \connect command.
commit : 85c54287af56fe351b53913ea2b81e9d6145f964
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 21 Oct 2020 16:18:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 21 Oct 2020 16:18:40 -0400
psql's \connect claims to be able to re-use previous connection
parameters, but in fact it only re-uses the database name, user name,
host name (and possibly hostaddr, depending on version), and port.
This is problematic for assorted use cases. Notably, pg_dump[all]
emits "\connect databasename" commands which we would like to have
re-use all other parameters. If such a script is loaded in a psql run
that initially had "-d connstring" with some non-default parameters,
those other parameters would be lost, potentially causing connection
failure. (Thus, this is the same kind of bug addressed in commits
a45bc8a4f and 8e5793ab6, although the details are much different.)
To fix, redesign do_connect() so that it pulls out all properties
of the old PGconn using PQconninfo(), and then replaces individual
properties in that array. In the case where we don't wish to re-use
anything, get libpq's default settings using PQconndefaults() and
replace entries in that, so that we don't need different code paths
for the two cases.
This does result in an additional behavioral change for cases where
the original connection parameters allowed multiple hosts, say
"psql -h host1,host2", and the \connect request allows re-use of the
host setting. Because the previous coding relied on PQhost(), it
would only permit reconnection to the same host originally selected.
Although one can think of scenarios where that's a good thing, there
are others where it is not. Moreover, that behavior doesn't seem to
meet the principle of least surprise, nor was it documented; nor is
it even clear it was intended, since that coding long pre-dates the
addition of multi-host support to libpq. Hence, this patch is content
to drop it and re-use the host list as given.
Per Peter Eisentraut's comments on bug #16604. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
Use fast checkpoint in PostgresNode::backup()
commit : 831611b11c34ed0066633864f42ff67a10286aee
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 21 Oct 2020 14:37:26 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 21 Oct 2020 14:37:26 -0300
Should cause tests to be a bit faster
M src/test/perl/PostgresNode.pm
Remove the option to build thread_test.c outside configure.
commit : 8a2121185b26d7eb439860c7a96a08c0aa3c8508
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 21 Oct 2020 12:08:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 21 Oct 2020 12:08:48 -0400
Theoretically one could go into src/test/thread and build/run this
program there. In practice, that hasn't worked since 96bf88d52,
and probably much longer on some platforms (likely including just
the sort of hoary leftovers where this test might be of interest).
While it wouldn't be too hard to repair the breakage, the fact that
nobody has noticed for two years shows that there is zero usefulness
in maintaining this build pathway. Let's get rid of it and decree
that thread_test.c is *only* meant to be built/used in configure.
Given that decision, it makes sense to put thread_test.c under config/
and get rid of src/test/thread altogether, so that's what I did.
In passing, update src/test/README, which had been ignored by some
not-so-recent additions of subdirectories.
Discussion: https://postgr.es/m/227659.1603041612@sss.pgh.pa.us
R093 src/test/thread/thread_test.c config/thread_test.c
M configure
M configure.ac
M src/Makefile
M src/port/thread.c
M src/test/Makefile
M src/test/README
D src/test/thread/.gitignore
D src/test/thread/Makefile
D src/test/thread/README
Remove obsolete ifdefs
commit : 555eb1a4f0114e1c4017093fe840045dde1ed9ea
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 21 Oct 2020 14:20:50 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 21 Oct 2020 14:20:50 +0200
Commit 8dace66e0735ca39b779922d02c24ea2686e6521 added #ifdefs for a
number of errno symbols because they were not present on Windows.
Later, commit 125ad539a275db5ab8f4647828b80a16d02eabd2 added
replacement #defines for some of those symbols. So some of the
changes from the first commit are made dead code by the second commit
and can now be removed.
Discussion: https://www.postgresql.org/message-id/flat/6dee8574-b0ad-fc49-9c8c-2edc796f0033@2ndquadrant.com
M src/port/strerror.c
Fix -Wcast-function-type warnings on Windows/MinGW
commit : 8a58347a3c10126ce9eb38872a46c55e91faa174
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 21 Oct 2020 08:17:51 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 21 Oct 2020 08:17:51 +0200
After de8feb1f3a23465b5737e8a8c160e8ca62f61339, some warnings remained
that were only visible when using GCC on Windows. Fix those as well.
Note that the ecpg test source files don't use the full pg_config.h,
so we can't use pg_funcptr_t there but have to do it the long way.
M src/backend/libpq/auth.c
M src/backend/port/win32/crashdump.c
M src/bin/pg_ctl/pg_ctl.c
M src/common/restricted_token.c
M src/interfaces/ecpg/test/expected/thread-thread.c
M src/interfaces/ecpg/test/expected/thread-thread_implicit.c
M src/interfaces/ecpg/test/thread/thread.pgc
M src/interfaces/ecpg/test/thread/thread_implicit.pgc
M src/port/getaddrinfo.c
M src/port/win32env.c
M src/port/win32stat.c
Review format of code generated by PerfectHash.pm
commit : 19ae53c92d5f5bdfb971d560a562e84c5f65c8b0
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 21 Oct 2020 09:22:27 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 21 Oct 2020 09:22:27 +0900
80f8eb7 has added to the normalization quick check headers some code
generated by PerfectHash.pm that is incompatible with the settings of
gitattributes for this repository, as whitespaces followed a set of tabs
for the first element of a line in the table. Instead of adding a new
exception to gitattributes, rework the format generated so as a right
padding with spaces is used instead of a left padding. This keeps the
table generated in a readable shape with its set of columns, making
unnecessary an update of gitattributes.
Reported-by: Peter Eisentraut
Author: John Naylor
Discussion: https://postgr.es/m/d601b3b5-a3c7-5457-2f84-3d6513d690fc@2ndquadrant.com
M src/include/common/unicode_normprops_table.h
M src/tools/PerfectHash.pm
Fix ALTER TABLE .. ENABLE/DISABLE TRIGGER recursion
commit : bbb927b4db9b3b449ccd0f76c1296de382a2f0c1
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 20 Oct 2020 19:22:09 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 20 Oct 2020 19:22:09 -0300
More precisely, correctly handle the ONLY flag indicating not to
recurse. This was implemented in 86f575948c77 by recursing in
trigger.c, but that's the wrong place; use ATSimpleRecursion instead,
which behaves properly. However, because legacy inheritance has never
recursed in that situation, make sure to do that only for new-style
partitioning.
I noticed this problem while testing a fix for another bug in the
vicinity.
This has been wrong all along, so backpatch to 11.
Discussion: https://postgr.es/m/20201016235925.GA29829@alvherre.pgsql
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Change the attribute name in pg_stat_replication_slots view.
commit : 03d51b776d88badbeb4452c1ab452feb78dce36a
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 20 Oct 2020 10:24:36 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 20 Oct 2020 10:24:36 +0530
Change the attribute 'name' to 'slot_name' in pg_stat_replication_slots
view to make it clear and that way we will be consistent with the other
places like pg_stat_wal_receiver view where we display the same attribute.
In the passing, fix the typo in one of the macros in the related code.
Bump the catversion as we have modified the name in the catalog as well.
Reported-by: Noriyoshi Shinoda
Author: Noriyoshi Shinoda
Reviewed-by: Sawada Masahiko and Amit Kapila
Discussion: https://postgr.es/m/CA+fd4k5_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/rules.out
Fix connection string handling in src/bin/scripts/ programs.
commit : 8e5793ab60bba65ffaa0f2237b39c9580d8972c7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 19:03:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 19:03:46 -0400
When told to process all databases, clusterdb, reindexdb, and vacuumdb
would reconnect by replacing their --maintenance-db parameter with the
name of the target database. If that parameter is a connstring (which
has been allowed for a long time, though we failed to document that
before this patch), we'd lose any other options it might specify, for
example SSL or GSS parameters, possibly resulting in failure to connect.
Thus, this is the same bug as commit a45bc8a4f fixed in pg_dump and
pg_restore. We can fix it in the same way, by using libpq's rules for
handling multiple "dbname" parameters to add the target database name
separately. I chose to apply the same refactoring approach as in that
patch, with a struct to handle the command line parameters that need to
be passed through to connectDatabase. (Maybe someday we can unify the
very similar functions here and in pg_dump/pg_restore.)
Per Peter Eisentraut's comments on bug #16604. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
M doc/src/sgml/ref/clusterdb.sgml
M doc/src/sgml/ref/createdb.sgml
M doc/src/sgml/ref/dropdb.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
M src/bin/scripts/clusterdb.c
M src/bin/scripts/common.c
M src/bin/scripts/common.h
M src/bin/scripts/createdb.c
M src/bin/scripts/createuser.c
M src/bin/scripts/dropdb.c
M src/bin/scripts/dropuser.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/scripts_parallel.c
M src/bin/scripts/scripts_parallel.h
M src/bin/scripts/vacuumdb.c
Fix list-munging bug that broke SQL function result coercions.
commit : c8ab9701791e22f7a8e1badf362654db179c9703
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 14:33:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 14:33:01 -0400
Since commit 913bbd88d, check_sql_fn_retval() can either insert type
coercion steps in-line in the Query that produces the SQL function's
results, or generate a new top-level Query to perform the coercions,
if modifying the Query's output in-place wouldn't be safe. However,
it appears that the latter case has never actually worked, because
the code tried to inject the new Query back into the query list it was
passed ... which is not the list that will be used for later processing
when we execute the SQL function "normally" (without inlining it).
So we ended up with no coercion happening at run-time, leading to
wrong results or crashes depending on the datatypes involved.
While the regression tests look like they cover this area well enough,
through a huge bit of bad luck all the test cases that exercise the
separate-Query path were checking either inline-able cases (which
accidentally didn't have the bug) or cases that are no-ops at runtime
(e.g., varchar to text), so that the failure to perform the coercion
wasn't obvious. The fact that the cases that don't work weren't
allowed at all before v13 probably contributed to not noticing the
problem sooner, too.
To fix, get rid of the separate "flat" list of Query nodes and instead
pass the real two-level list that is going to be used later. I chose
to make the same change in check_sql_fn_statements(), although that has
no actual bug, just so that we don't need that data structure at all.
This is an API change, as evidenced by the adjustments needed to
callers outside functions.c. That's a bit scary to be doing in a
released branch, but so far as I can tell from a quick search,
there are no outside callers of these functions (and they are
sufficiently specific to our semantics for SQL-language functions that
it's not apparent why any extension would need to call them). In any
case, v13 already changed the API of check_sql_fn_retval() compared to
prior branches.
Per report from pinker. Back-patch to v13 where this code came in.
Discussion: https://postgr.es/m/1603050466566-0.post@n3.nabble.com
M src/backend/catalog/pg_proc.c
M src/backend/executor/functions.c
M src/backend/optimizer/util/clauses.c
M src/include/executor/functions.h
M src/test/regress/expected/rangefuncs.out
M src/test/regress/sql/rangefuncs.sql
Misc documentation fixes.
commit : c5f42daa6077a4c309c5280a47d0e114c12dc572
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 19:28:54 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 19:28:54 +0300
- Misc grammar and punctuation fixes.
- Stylistic cleanup: use spaces between function arguments and JSON fields
in examples. For example "foo(a,b)" -> "foo(a, b)". Add semicolon after
last END in a few PL/pgSQL examples that were missing them.
- Make sentence that talked about "..." and ".." operators more clear,
by avoiding to end the sentence with "..". That makes it look the same
as "..."
- Fix syntax description for HAVING: HAVING conditions cannot be repeated
Patch by Justin Pryzby, per Yaroslav Schekin's report. Backpatch to all
supported versions, to the extent that the patch applies easily.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/dblink.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/hstore.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/isn.sgml
M doc/src/sgml/ltree.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/parallel.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/select.sgml
M doc/src/sgml/ref/select_into.sgml
M doc/src/sgml/rules.sgml
M doc/src/sgml/seg.sgml
M doc/src/sgml/textsearch.sgml
Fix TRUNCATE doc: ALTER SEQUENCE RESTART is now transactional.
commit : 2a972e0165c9b34da4cff3001a2dad8e5a9a5a2b
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 19:02:25 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 19:02:25 +0300
ALTER SEQUENCE RESTART was made transactional in commit 3d79013b97.
Backpatch to v10, where that was introduced.
Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/ref/truncate.sgml
Fix output of tsquery example in docs.
commit : c0bc4c682ee2a84dee2b562db3d17d173a6abc06
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 18:50:33 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 18:50:33 +0300
The output for this query changed in commit 4e2477b7b8. Backport to 9.6
like that commit.
Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/textsearch.sgml
Fix doc for full text search distance operator.
commit : 1a64c7636f24096155f7aa15d7164eba3cdef075
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 17:58:38 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 17:58:38 +0300
Commit 028350f619 changed its behavior from "at most" to "exactly", but
forgot to update the documentation. Backpatch to 9.6.
Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/textsearch.sgml
Update link for pllua
commit : b4d5b458e6fb4c861f92888753deea6a5005a68d
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 19 Oct 2020 13:47:09 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 19 Oct 2020 13:47:09 +0200
Author: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/A05874AE-8771-4C61-A24E-0B6249B8F3C2@yesql.se
M doc/src/sgml/external-projects.sgml
Remove PartitionRoutingInfo struct.
commit : fb5883da86154c3126264bfd97b0cd6f293bcebd
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 14:11:57 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 14:11:57 +0300
The extra indirection neeeded to access its members via its enclosing
ResultRelInfo seems pointless. Move all the fields from
PartitionRoutingInfo to ResultRelInfo.
Author: Amit Langote
Reviewed-by: Alvaro Herrera
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqFViT47Zbr_ASBejiK7iDG8%3DQ1swQ-tjM6caRPQ67pT%3Dw%40mail.gmail.com
M src/backend/commands/copy.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/backend/replication/logical/worker.c
M src/include/executor/execPartition.h
M src/include/nodes/execnodes.h
Revise child-to-root tuple conversion map management.
commit : 6973533650c04653d9018f61c1ac99ecb11094bd
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 14:11:54 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 14:11:54 +0300
Store the tuple conversion map to convert a tuple from a child table's
format to the root format in a new ri_ChildToRootMap field in
ResultRelInfo. It is initialized if transition tuple capture for FOR
STATEMENT triggers or INSERT tuple routing on a partitioned table is
needed. Previously, ModifyTable kept the maps in the per-subplan
ModifyTableState->mt_per_subplan_tupconv_maps array, or when tuple
routing was used, in
ResultRelInfo->ri_Partitioninfo->pi_PartitionToRootMap. The new field
replaces both of those.
Now that the child-to-root tuple conversion map is always available in
ResultRelInfo (when needed), remove the TransitionCaptureState.tcs_map
field. The callers of Exec*Trigger() functions no longer need to set or
save it, which is much less confusing and bug-prone. Also, as a future
optimization, this will allow us to delay creating the map for a given
result relation until the relation is actually processed during
execution.
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqHtCWLdK-LO%3DNEsvOdHx%2B7yv4mE_zYK0i3BH7dXb-wxog%40mail.gmail.com
M src/backend/commands/copy.c
M src/backend/commands/trigger.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/include/commands/trigger.h
M src/include/executor/execPartition.h
M src/include/nodes/execnodes.h
Clean up code to resolve the "root target relation" in nodeModifyTable.c
commit : f49b85d783f6781138f33bbe5f6e98da86907d84
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 14:11:44 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 14:11:44 +0300
When executing DDL on a partitioned table or on a table with inheritance
children, statement-level triggers must be fired against the table given
in the original statement. The code to look that up was a bit messy and
duplicative. Commit 501ed02cf6 added a helper function,
getASTriggerResultRelInfo() (later renamed to getTargetResultRelInfo())
for it, but for some reason it was only used when firing AFTER STATEMENT
triggers and the code to fire BEFORE STATEMENT triggers duplicated the
logic.
Determine the target relation in ExecInitModifyTable(), and set it always
in ModifyTableState. Code that used to call getTargetResultRelInfo() can
now use ModifyTableState->rootResultRelInfo directly.
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqFViT47Zbr_ASBejiK7iDG8%3DQ1swQ-tjM6caRPQ67pT%3Dw%40mail.gmail.com
M src/backend/executor/nodeModifyTable.c
M src/include/nodes/execnodes.h
Avoid invalid alloc size error in shm_mq
commit : 26ec6b5948a73d0e07ed9435ee4554594acdf34f
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Oct 2020 08:52:25 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Oct 2020 08:52:25 +0200
In shm_mq_receive(), a huge payload could trigger an unjustified
"invalid memory alloc request size" error due to the way the buffer
size is increased.
Add error checks (documenting the upper limit) and avoid the error by
limiting the allocation size to MaxAllocSize.
Author: Markus Wanner <markus.wanner@2ndquadrant.com>
Discussion: https://www.postgresql.org/message-id/flat/3bb363e7-ac04-0ac4-9fe8-db1148755bfa%402ndquadrant.com
M src/backend/storage/ipc/shm_mq.c
Improve whitespace
commit : afa0d53d4de6d2f43fbc0a76c88dceb22ba65e86
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Oct 2020 08:32:53 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Oct 2020 08:32:53 +0200
The SQL file was using a mix of tabs and spaces inconsistently.
Convert all to spaces and adjust indentation accordingly.
M src/test/regress/expected/hash_func.out
M src/test/regress/sql/hash_func.sql
Change the docs for PARALLEL option of Vacuum.
commit : 560d260d7852dc54a8c587c1b388843e8c433bc8
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 19 Oct 2020 09:13:17 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 19 Oct 2020 09:13:17 +0530
The rules to choose the number of parallel workers to perform parallel
vacuum operation were not clearly specified.
Reported-by: Peter Eisentraut
Author: Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/36aa8aea-61b7-eb3c-263b-648e0cb117b7@2ndquadrant.com
M doc/src/sgml/ref/vacuum.sgml
Fix potential memory leak in pgcrypto
commit : ca2a12c935f75fb56c3b14527d6f2ff6f549ea85
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 19 Oct 2020 09:36:56 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 19 Oct 2020 09:36:56 +0900
When allocating a EVP context, it would have been possible to leak some
memory allocated directly by OpenSSL, that PostgreSQL lost track of if
the initialization of the context allocated failed. The cleanup can be
done with EVP_MD_CTX_destroy().
Note that EVP APIs exist since OpenSSL 0.9.7 and we have in the tree
equivalent implementations for older versions since ce9b75d (code
removed with 9b7cd59a as of 10~). However, in 9.5 and 9.6, the existing
code makes use of EVP_MD_CTX_destroy() and EVP_MD_CTX_create() without
an equivalent implementation when building the tree with OpenSSL 0.9.6
or older, meaning that this code is in reality broken with such versions
since it got introduced in e2838c5. As we have heard no complains about
that, it does not seem worth bothering with in 9.5 and 9.6, so I have
left that out for simplicity.
Author: Michael Paquier
Discussion: https://postgr.es/m/20201015072212.GC2305@paquier.xyz
Backpatch-through: 9.5
M contrib/pgcrypto/openssl.c
Prevent overly large and NaN row estimates in relations
commit : a90c950fc7fd8796daa8c7948e7046bceb272894
author : David Rowley <drowley@postgresql.org>
date : Mon, 19 Oct 2020 10:53:52 +1300
committer: David Rowley <drowley@postgresql.org>
date : Mon, 19 Oct 2020 10:53:52 +1300
Given a query with enough joins, it was possible that the query planner,
after multiplying the row estimates with the join selectivity that the
estimated number of rows would exceed the limits of the double data type
and become infinite.
To give an indication on how extreme a case is required to hit this, the
particular example case reported required 379 joins to a table without any
statistics, which resulted in the 1.0/DEFAULT_NUM_DISTINCT being used for
the join selectivity. This eventually caused the row estimates to go
infinite and resulted in an assert failure in initial_cost_mergejoin()
where the infinite row estimated was multiplied by an outerstartsel of 0.0
resulting in NaN. The failing assert verified that NaN <= Inf, which is
false.
To get around this we use clamp_row_est() to cap row estimates at a
maximum of 1e100. This value is thought to be low enough that costs
derived from it would remain within the bounds of what the double type can
represent.
Aside from fixing the failing Assert, this also has the added benefit of
making it so add_path() will still receive proper numerical values as
costs which will allow it to make more sane choices when determining the
cheaper path in extreme cases such as the one described above.
Additionally, we also get rid of the isnan() checks in the join costing
functions. The actual case which originally triggered those checks to be
added in the first place never made it to the mailing lists. It seems
likely that the new code being added to clamp_row_est() will result in
those becoming checks redundant, so just remove them.
The fairly harmless assert failure problem does also exist in the
backbranches, however, a more minimalistic fix will be applied there.
Reported-by: Onder Kalaci
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/DM6PR21MB1211FF360183BCA901B27F04D80B0@DM6PR21MB1211.namprd21.prod.outlook.com
M src/backend/optimizer/path/costsize.c
Update the Winsock API version requested by libpq.
commit : d5a9a661fcd2f5db037274157f931863a52004fd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 18 Oct 2020 12:56:43 -0400
committer: Tom Lane