Stamp 14.0.
commit : 86a4dc1e6f29d1992a2afa3fac1a0b0a6e84568c
author : Tom Lane <[email protected]>
date : Mon, 27 Sep 2021 16:57:41 -0400
committer: Tom Lane <[email protected]>
date : Mon, 27 Sep 2021 16:57:41 -0400
M configure
M configure.ac
Translation updates
commit : 9f535e4aeafc4e427355760178ecc89055f96942
author : Peter Eisentraut <[email protected]>
date : Mon, 27 Sep 2021 09:22:27 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 27 Sep 2021 09:22:27 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 941ca560d0b36a8bace8432b06302ca003829f42
M src/bin/psql/po/de.po
Update list of acknowledgments in release notes
commit : 3cc51338438acb38ea14cebe6fd979f00ad5bebf
author : Peter Eisentraut <[email protected]>
date : Mon, 27 Sep 2021 08:59:03 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 27 Sep 2021 08:59:03 +0200
current through e8b39cebdaf042dfeeb31d2f48f0fe7b33886210
M doc/src/sgml/release-14.sgml
Fix typos in docs
commit : e8b39cebdaf042dfeeb31d2f48f0fe7b33886210
author : Michael Paquier <[email protected]>
date : Sun, 26 Sep 2021 19:18:23 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 26 Sep 2021 19:18:23 +0900
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M doc/src/sgml/datatype.sgml
Doc: final(?) updates for 14.0 release notes.
commit : 765f677f364100072160e7af37288eb1df2ff355
author : Tom Lane <[email protected]>
date : Sat, 25 Sep 2021 11:36:43 -0400
committer: Tom Lane <[email protected]>
date : Sat, 25 Sep 2021 11:36:43 -0400
Add the customary short list of major features. Set release date.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/release-14.sgml
Doc: extend warnings about collation-mismatch hazards in postgres_fdw.
commit : 02c4e3533926f4e7398611b6349d7c438b86c63b
author : Tom Lane <[email protected]>
date : Sat, 25 Sep 2021 10:53:54 -0400
committer: Tom Lane <[email protected]>
date : Sat, 25 Sep 2021 10:53:54 -0400
Be a little more vocal about the risks of remote collations not
matching local ones. Actually fixing these risks seems hard,
and I've given up on the idea that it might be back-patchable.
So the best we can do for the back branches is add documentation.
Per discussion of bug #16583 from Jiří Fejfar.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/postgres-fdw.sgml
Add alternative output for OpenSSL 3 without legacy loaded
commit : 6d0001aabf2a49180a236d9c2a7ecdf24e0cdb37
author : Daniel Gustafsson <[email protected]>
date : Sat, 25 Sep 2021 11:27:28 +0200
committer: Daniel Gustafsson <[email protected]>
date : Sat, 25 Sep 2021 11:27:28 +0200
OpenSSL 3 introduced the concept of providers to support modularization,
and moved the outdated ciphers to the new legacy provider. In case it's
not loaded in the users openssl.cnf file there will be a lot of regress
test failures, so add alternative outputs covering those.
Also document the need to load the legacy provider in order to use older
ciphers with OpenSSL-enabled pgcrypto.
This will be backpatched to all supported version once there is sufficient
testing in the buildfarm of OpenSSL 3.
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
A contrib/pgcrypto/expected/blowfish_1.out
A contrib/pgcrypto/expected/cast5_1.out
A contrib/pgcrypto/expected/des_1.out
A contrib/pgcrypto/expected/pgp-decrypt_1.out
A contrib/pgcrypto/expected/pgp-pubkey-decrypt_1.out
M doc/src/sgml/pgcrypto.sgml
Disable OpenSSL EVP digest padding in pgcrypto
commit : 4fa2b15e1c9cae79afe17c14591074111b0d4093
author : Daniel Gustafsson <[email protected]>
date : Sat, 25 Sep 2021 11:27:20 +0200
committer: Daniel Gustafsson <[email protected]>
date : Sat, 25 Sep 2021 11:27:20 +0200
The PX layer in pgcrypto is handling digest padding on its own uniformly
for all backend implementations. Starting with OpenSSL 3.0.0, DecryptUpdate
doesn't flush the last block in case padding is enabled so explicitly
disable it as we don't use it.
This will be backpatched to all supported version once there is sufficient
testing in the buildfarm of OpenSSL 3.
Reviewed-by: Peter Eisentraut, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M contrib/pgcrypto/openssl.c
doc: Improve description of index vacuuming with GUCs
commit : bfe1ead94488986008771c0d295c2725bab952cb
author : Michael Paquier <[email protected]>
date : Sat, 25 Sep 2021 15:11:52 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 25 Sep 2021 15:11:52 +0900
Index vacuums may happen multiple times depending on the number of dead
tuples stored, as of maintenance_work_mem for a manual VACUUM. For
autovacuum, this is controlled by autovacuum_work_mem instead, if set.
The documentation mentioned the former, but not the latter in the
context of autovacuum.
Reported-by: Nikolai Berkoff
Author: Laurenz Albe, Euler Taveira
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M doc/src/sgml/monitoring.sgml
doc: Add missing markup in CREATE EVENT TRIGGER page
commit : 3eea9139ee46124746e942293d57e3fb1d3c0e09
author : Michael Paquier <[email protected]>
date : Sat, 25 Sep 2021 14:48:09 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 25 Sep 2021 14:48:09 +0900
Reported-by: rir
Discussion: https://postgr.es/m/20210924183658.3syyitp3yuxjv2fp@localhost
Backpatch-through: 9.6
M doc/src/sgml/ref/create_event_trigger.sgml
Add missing $Test::Builder::Level settings
commit : e536a2683439c3dd4ea6d339a878fa4430e3174d
author : Peter Eisentraut <[email protected]>
date : Thu, 23 Sep 2021 22:49:20 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 23 Sep 2021 22:49:20 +0200
One of these was accidentally removed by c50624c. The others are
added by analogy.
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/test/authentication/t/001_password.pl
M src/test/authentication/t/002_saslprep.pl
M src/test/kerberos/t/001_auth.pl
M src/test/ldap/t/001_auth.pl
Split macros from visibilitymap.h into a separate header
commit : 7186f07189baf069c54718315b81e65d87f4c0c1
author : Alexander Korotkov <[email protected]>
date : Thu, 23 Sep 2021 19:59:03 +0300
committer: Alexander Korotkov <[email protected]>
date : Thu, 23 Sep 2021 19:59:03 +0300
That allows to include just visibilitymapdefs.h from file.c, and in turn,
remove include of postgres.h from relcache.h.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/20210913232614.czafiubr435l6egi%40alap3.anarazel.de
Author: Alexander Korotkov
Reviewed-by: Andres Freund, Tom Lane, Alvaro Herrera
Backpatch-through: 13
M src/bin/pg_upgrade/file.c
M src/include/access/visibilitymap.h
A src/include/access/visibilitymapdefs.h
M src/include/utils/relcache.h
Release memory allocated by dependency_degree
commit : abb2f9144ba1b7ac806f3779f53ae2f6174cd2d9
author : Tomas Vondra <[email protected]>
date : Tue, 21 Sep 2021 01:13:11 +0200
committer: Tomas Vondra <[email protected]>
date : Tue, 21 Sep 2021 01:13:11 +0200
Calculating degree of a functional dependency may allocate a lot of
memory - we have released mot of the explicitly allocated memory, but
e.g. detoasted varlena values were left behind. That may be an issue,
because we consider a lot of dependencies (all combinations), and the
detoasting may happen for each one again.
Fixed by calling dependency_degree() in a dedicated context, and
resetting it after each call. We only need the calculated dependency
degree, so we don't need to copy anything.
Backpatch to PostgreSQL 10, where extended statistics were introduced.
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com
M src/backend/statistics/dependencies.c
Free memory after building each statistics object
commit : bb7628e55eda6f450f0f824d11c9a34b11f6bb12
author : Tomas Vondra <[email protected]>
date : Tue, 21 Sep 2021 01:14:11 +0200
committer: Tomas Vondra <[email protected]>
date : Tue, 21 Sep 2021 01:14:11 +0200
Until now, all extended statistics on a given relation were built in the
same memory context, without resetting. Some of the memory was released
explicitly, but not all of it - for example memory allocated while
detoasting values is hard to free. This is how it worked since extended
statistics were introduced in PostgreSQL 10, but adding support for
extended stats on expressions made the issue somewhat worse as it
increases the number of statistics to build.
Fixed by adding a memory context which gets reset after building each
statistics object (all the statistics kinds included in it). Resetting
it after building each statistics kind would be even better, but it
would require more invasive changes and copying of results, making it
harder to backpatch.
Backpatch to PostgreSQL 10, where extended statistics were introduced.
Author: Justin Pryzby
Reported-by: Justin Pryzby
Reviewed-by: Tomas Vondra
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com
M src/backend/statistics/extended_stats.c
Invalidate all partitions for a partitioned table in publication.
commit : 9eff8593265929d3a1fcdee375bd0a801c12b367
author : Amit Kapila <[email protected]>
date : Wed, 22 Sep 2021 08:13:37 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 22 Sep 2021 08:13:37 +0530
Updates/Deletes on a partition were allowed even without replica identity
after the parent table was added to a publication. This would later lead
to an error on subscribers. The reason was that we were not invalidating
the partition's relcache and the publication information for partitions
was not getting rebuilt. Similarly, we were not invalidating the
partitions' relcache after dropping a partitioned table from a publication
which will prohibit Updates/Deletes on its partition without replica
identity even without any publication.
Reported-by: Haiying Tang
Author: Hou Zhijie and Vignesh C
Reviewed-by: Vignesh C and Amit Kapila
Backpatch-through: 13
Discussion: https://postgr.es/m/OS0PR01MB6113D77F583C922F1CEAA1C3FBD29@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/backend/catalog/pg_publication.c
M src/backend/commands/publicationcmds.c
M src/include/catalog/pg_publication.h
M src/include/commands/publicationcmds.h
M src/test/regress/expected/publication.out
M src/test/regress/sql/publication.sql
Fix "single value strategy" index deletion issue.
commit : e665129c4727004e7a7c12c86d077abc750b3307
author : Peter Geoghegan <[email protected]>
date : Tue, 21 Sep 2021 18:57:31 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 21 Sep 2021 18:57:31 -0700
It is not appropriate for deduplication to apply single value strategy
when triggered by a bottom-up index deletion pass. This wastes cycles
because later bottom-up deletion passes will overinterpret older
duplicate tuples that deduplication actually just skipped over "by
design". It also makes bottom-up deletion much less effective for low
cardinality indexes that happen to cross a meaningless "index has single
key value per leaf page" threshold.
To fix, slightly narrow the conditions under which deduplication's
single value strategy is considered. We already avoided the strategy
for a unique index, since our high level goal must just be to buy time
for VACUUM to run (not to buy space). We'll now also avoid it when we
just had a bottom-up pass that reported failure. The two cases share
the same high level goal, and already overlapped significantly, so this
approach is quite natural.
Oversight in commit d168b666, which added bottom-up index deletion.
Author: Peter Geoghegan <[email protected]>
Discussion: https://postgr.es/m/CAH2-WznaOvM+Gyj-JQ0X=JxoMDxctDTYjiEuETdAGbF5EUc3MA@mail.gmail.com
Backpatch: 14-, where bottom-up deletion was introduced.
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtinsert.c
M src/include/access/nbtree.h
Fix places in TestLib.pm in need of adaptation to the output of Msys perl
commit : 90251ff199858844fa450ba9614092c06c67fc4f
author : Michael Paquier <[email protected]>
date : Wed, 22 Sep 2021 08:43:00 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 22 Sep 2021 08:43:00 +0900
Contrary to the output of native perl, Msys perl generates outputs with
CRLFs characters. There are already places in the TAP code where CRLFs
(\r\n) are automatically converted to LF (\n) on Msys, but we missed a
couple of places when running commands and using their output for
comparison, that would lead to failures.
This problem has been found thanks to the test added in 5adb067 using
TestLib::command_checks_all(), but after a closer look more code paths
were missing a filter.
This is backpatched all the way down to prevent any surprises if a new
test is introduced in stable branches.
Reviewed-by: Andrew Dunstan, Álvaro Herrera
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/test/perl/TestLib.pm
Fix misevaluation of STABLE parameters in CALL within plpgsql.
commit : 2ad5f963e1306aa6f4cf96076a230e96529d2237
author : Tom Lane <[email protected]>
date : Tue, 21 Sep 2021 19:06:33 -0400
committer: Tom Lane <[email protected]>
date : Tue, 21 Sep 2021 19:06:33 -0400
Before commit 84f5c2908, a STABLE function in a plpgsql CALL
statement's argument list would see an up-to-date snapshot,
because exec_stmt_call would push a new snapshot. I got rid of
that because the possibility of the snapshot disappearing within
COMMIT made it too hard to manage a snapshot across the CALL
statement. That's fine so far as the procedure itself goes,
but I forgot to think about the possibility of STABLE functions
within the CALL argument list. As things now stand, those'll
be executed with the Portal's snapshot as ActiveSnapshot,
keeping them from seeing updates more recent than Portal startup.
(VOLATILE functions don't have a problem because they take their
own snapshots; which indeed is also why the procedure itself
doesn't have a problem. There are no STABLE procedures.)
We can fix this by pushing a new snapshot transiently within
ExecuteCallStmt itself. Popping the snapshot before we get
into the procedure proper eliminates the management problem.
The possibly-useless extra snapshot-grab is slightly annoying,
but it's no worse than what happened before 84f5c2908.
Per bug #17199 from Alexander Nawratil. Back-patch to v11,
like the previous patch.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/functioncmds.c
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/sql/plpgsql_transaction.sql
Document XLOG_INCLUDE_XID a little better
commit : c1d1ae1db23796e4d1b04f5c087944722cf1446a
author : Alvaro Herrera <[email protected]>
date : Tue, 21 Sep 2021 19:47:53 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 21 Sep 2021 19:47:53 -0300
I noticed that commit 0bead9af484c left this flag undocumented in
XLogSetRecordFlags, which led me to discover that the flag doesn't
actually do what the one comment on it said it does. Improve the
situation by adding some more comments.
Backpatch to 14, where the aforementioned commit appears.
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xloginsert.c
M src/include/access/xlog.h
M src/include/access/xlogrecord.h
Stamp 14rc1.
commit : 8aed7f7a2ec46ad60c84c3c97300210850c3fbc8
author : Tom Lane <[email protected]>
date : Mon, 20 Sep 2021 17:33:01 -0400
committer: Tom Lane <[email protected]>
date : Mon, 20 Sep 2021 17:33:01 -0400
M configure
M configure.ac
Remove overzealous index deletion assertion.
commit : 955a6a893498a5d3af544d9c0b1c292b19ec1690
author : Peter Geoghegan <[email protected]>
date : Mon, 20 Sep 2021 14:26:24 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 20 Sep 2021 14:26:24 -0700
A broken HOT chain is not an unexpected condition, even when the offset
number points past the end of the page's line pointer array.
heap_prune_chain() does not (and never has) treated this condition as
unexpected, so derivative code in heap_index_delete_tuples() shouldn't
do so either.
Oversight in commit 4228817449.
The assertion can probably only fail on Postgres 14 and master. Earlier
releases don't have commit 3c3b8a4b, which taught VACUUM to truncate the
line pointer array of heap pages. Backpatch all the same, just to be
consistent.
Author: Peter Geoghegan <[email protected]>
Reported-By: Alexander Lakhin <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch: 12-, just like commit 4228817449.
M src/backend/access/heap/heapam.c
Translation updates
commit : 3e8525aab86e78593844c9b395be40938582ebaf
author : Peter Eisentraut <[email protected]>
date : Mon, 20 Sep 2021 16:23:13 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 20 Sep 2021 16:23:13 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 10b675b81a3a04bac460cb049e0b7b6e17fb4795
M src/backend/nls.mk
M src/backend/po/de.po
M src/backend/po/fr.po
D src/backend/po/id.po
M src/backend/po/ja.po
D src/backend/po/pl.po
D src/backend/po/pt_BR.po
M src/backend/po/ru.po
D src/backend/po/tr.po
M src/backend/po/uk.po
M src/bin/initdb/nls.mk
M src/bin/initdb/po/el.po
M src/bin/initdb/po/fr.po
D src/bin/initdb/po/he.po
D src/bin/initdb/po/it.po
M src/bin/initdb/po/ja.po
D src/bin/initdb/po/pl.po
D src/bin/initdb/po/pt_BR.po
M src/bin/initdb/po/ru.po
M src/bin/initdb/po/tr.po
M src/bin/initdb/po/uk.po
D src/bin/initdb/po/vi.po
M src/bin/initdb/po/zh_CN.po
M src/bin/pg_amcheck/nls.mk
M src/bin/pg_amcheck/po/de.po
M src/bin/pg_amcheck/po/el.po
M src/bin/pg_amcheck/po/fr.po
M src/bin/pg_amcheck/po/ja.po
A src/bin/pg_amcheck/po/ru.po
M src/bin/pg_amcheck/po/uk.po
M src/bin/pg_amcheck/po/zh_CN.po
M src/bin/pg_archivecleanup/nls.mk
M src/bin/pg_archivecleanup/po/el.po
M src/bin/pg_archivecleanup/po/ja.po
D src/bin/pg_archivecleanup/po/pl.po
M src/bin/pg_archivecleanup/po/ru.po
M src/bin/pg_archivecleanup/po/tr.po
M src/bin/pg_archivecleanup/po/uk.po
D src/bin/pg_archivecleanup/po/vi.po
M src/bin/pg_archivecleanup/po/zh_CN.po
M src/bin/pg_basebackup/nls.mk
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_basebackup/po/fr.po
D src/bin/pg_basebackup/po/he.po
D src/bin/pg_basebackup/po/it.po
M src/bin/pg_basebackup/po/ja.po
D src/bin/pg_basebackup/po/pl.po
D src/bin/pg_basebackup/po/pt_BR.po
M src/bin/pg_basebackup/po/ru.po
M src/bin/pg_basebackup/po/uk.po
D src/bin/pg_basebackup/po/vi.po
M src/bin/pg_basebackup/po/zh_CN.po
M src/bin/pg_checksums/po/el.po
M src/bin/pg_checksums/po/ja.po
M src/bin/pg_checksums/po/ru.po
M src/bin/pg_checksums/po/uk.po
M src/bin/pg_checksums/po/zh_CN.po
M src/bin/pg_config/nls.mk
M src/bin/pg_config/po/el.po
M src/bin/pg_config/po/fr.po
M src/bin/pg_config/po/ja.po
D src/bin/pg_config/po/nb.po
D src/bin/pg_config/po/ro.po
M src/bin/pg_config/po/ru.po
D src/bin/pg_config/po/ta.po
M src/bin/pg_config/po/tr.po
M src/bin/pg_config/po/uk.po
M src/bin/pg_config/po/zh_CN.po
D src/bin/pg_config/po/zh_TW.po
M src/bin/pg_controldata/nls.mk
M src/bin/pg_controldata/po/el.po
M src/bin/pg_controldata/po/fr.po
M src/bin/pg_controldata/po/ja.po
D src/bin/pg_controldata/po/pl.po
D src/bin/pg_controldata/po/pt_BR.po
M src/bin/pg_controldata/po/ru.po
M src/bin/pg_controldata/po/uk.po
D src/bin/pg_controldata/po/vi.po
M src/bin/pg_controldata/po/zh_CN.po
M src/bin/pg_ctl/nls.mk
M src/bin/pg_ctl/po/cs.po
M src/bin/pg_ctl/po/el.po
M src/bin/pg_ctl/po/fr.po
D src/bin/pg_ctl/po/he.po
M src/bin/pg_ctl/po/ja.po
D src/bin/pg_ctl/po/pl.po
D src/bin/pg_ctl/po/pt_BR.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_ctl/po/uk.po
M src/bin/pg_ctl/po/zh_CN.po
M src/bin/pg_dump/nls.mk
M src/bin/pg_dump/po/cs.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_dump/po/el.po
M src/bin/pg_dump/po/fr.po
D src/bin/pg_dump/po/he.po
D src/bin/pg_dump/po/it.po
M src/bin/pg_dump/po/ja.po
D src/bin/pg_dump/po/pl.po
D src/bin/pg_dump/po/pt_BR.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/uk.po
M src/bin/pg_dump/po/zh_CN.po
M src/bin/pg_resetwal/nls.mk
M src/bin/pg_resetwal/po/el.po
M src/bin/pg_resetwal/po/fr.po
D src/bin/pg_resetwal/po/it.po
M src/bin/pg_resetwal/po/ja.po
D src/bin/pg_resetwal/po/pl.po
D src/bin/pg_resetwal/po/pt_BR.po
M src/bin/pg_resetwal/po/ru.po
M src/bin/pg_resetwal/po/uk.po
M src/bin/pg_resetwal/po/zh_CN.po
M src/bin/pg_rewind/nls.mk
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/el.po
M src/bin/pg_rewind/po/fr.po
D src/bin/pg_rewind/po/it.po
M src/bin/pg_rewind/po/ja.po
D src/bin/pg_rewind/po/ko.po
D src/bin/pg_rewind/po/pl.po
D src/bin/pg_rewind/po/pt_BR.po
M src/bin/pg_rewind/po/ru.po
D src/bin/pg_rewind/po/tr.po
M src/bin/pg_rewind/po/uk.po
M src/bin/pg_rewind/po/zh_CN.po
M src/bin/pg_test_fsync/po/el.po
M src/bin/pg_test_fsync/po/ja.po
M src/bin/pg_test_fsync/po/ru.po
M src/bin/pg_test_fsync/po/uk.po
M src/bin/pg_test_fsync/po/zh_CN.po
M src/bin/pg_test_timing/nls.mk
D src/bin/pg_test_timing/po/cs.po
M src/bin/pg_test_timing/po/el.po
M src/bin/pg_test_timing/po/ja.po
D src/bin/pg_test_timing/po/ko.po
D src/bin/pg_test_timing/po/pl.po
M src/bin/pg_test_timing/po/ru.po
D src/bin/pg_test_timing/po/sv.po
D src/bin/pg_test_timing/po/tr.po
M src/bin/pg_test_timing/po/uk.po
D src/bin/pg_test_timing/po/vi.po
M src/bin/pg_test_timing/po/zh_CN.po
M src/bin/pg_upgrade/po/cs.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/fr.po
M src/bin/pg_upgrade/po/ja.po
M src/bin/pg_upgrade/po/ru.po
M src/bin/pg_upgrade/po/tr.po
M src/bin/pg_upgrade/po/uk.po
M src/bin/pg_upgrade/po/zh_CN.po
M src/bin/pg_verifybackup/po/el.po
M src/bin/pg_verifybackup/po/fr.po
M src/bin/pg_verifybackup/po/ja.po
M src/bin/pg_verifybackup/po/ru.po
M src/bin/pg_verifybackup/po/uk.po
M src/bin/pg_verifybackup/po/zh_CN.po
M src/bin/pg_waldump/nls.mk
M src/bin/pg_waldump/po/cs.po
M src/bin/pg_waldump/po/el.po
M src/bin/pg_waldump/po/es.po
M src/bin/pg_waldump/po/ja.po
M src/bin/pg_waldump/po/ru.po
M src/bin/pg_waldump/po/tr.po
M src/bin/pg_waldump/po/uk.po
D src/bin/pg_waldump/po/vi.po
M src/bin/pg_waldump/po/zh_CN.po
M src/bin/psql/nls.mk
M src/bin/psql/po/de.po
M src/bin/psql/po/el.po
M src/bin/psql/po/fr.po
D src/bin/psql/po/he.po
M src/bin/psql/po/ja.po
D src/bin/psql/po/pl.po
D src/bin/psql/po/pt_BR.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/uk.po
M src/bin/psql/po/zh_CN.po
D src/bin/psql/po/zh_TW.po
M src/bin/scripts/nls.mk
M src/bin/scripts/po/cs.po
M src/bin/scripts/po/de.po
A src/bin/scripts/po/el.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/fr.po
D src/bin/scripts/po/he.po
D src/bin/scripts/po/it.po
M src/bin/scripts/po/ja.po
D src/bin/scripts/po/pl.po
D src/bin/scripts/po/pt_BR.po
M src/bin/scripts/po/ru.po
M src/bin/scripts/po/tr.po
M src/bin/scripts/po/uk.po
M src/bin/scripts/po/zh_CN.po
M src/interfaces/ecpg/ecpglib/nls.mk
A src/interfaces/ecpg/ecpglib/po/el.po
M src/interfaces/ecpg/ecpglib/po/ja.po
M src/interfaces/ecpg/ecpglib/po/ru.po
M src/interfaces/ecpg/ecpglib/po/uk.po
M src/interfaces/ecpg/ecpglib/po/zh_CN.po
M src/interfaces/ecpg/preproc/nls.mk
M src/interfaces/ecpg/preproc/po/cs.po
M src/interfaces/ecpg/preproc/po/de.po
A src/interfaces/ecpg/preproc/po/el.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/fr.po
M src/interfaces/ecpg/preproc/po/ja.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/ecpg/preproc/po/tr.po
M src/interfaces/ecpg/preproc/po/uk.po
M src/interfaces/ecpg/preproc/po/zh_CN.po
M src/interfaces/libpq/nls.mk
M src/interfaces/libpq/po/el.po
M src/interfaces/libpq/po/fr.po
D src/interfaces/libpq/po/he.po
D src/interfaces/libpq/po/it.po
M src/interfaces/libpq/po/ja.po
D src/interfaces/libpq/po/pl.po
D src/interfaces/libpq/po/pt_BR.po
M src/interfaces/libpq/po/ru.po
D src/interfaces/libpq/po/tr.po
M src/interfaces/libpq/po/uk.po
M src/interfaces/libpq/po/zh_CN.po
D src/interfaces/libpq/po/zh_TW.po
M src/pl/plperl/nls.mk
M src/pl/plperl/po/el.po
M src/pl/plperl/po/fr.po
M src/pl/plperl/po/ja.po
M src/pl/plperl/po/ru.po
M src/pl/plperl/po/uk.po
M src/pl/plperl/po/zh_CN.po
D src/pl/plperl/po/zh_TW.po
M src/pl/plpgsql/src/nls.mk
M src/pl/plpgsql/src/po/de.po
A src/pl/plpgsql/src/po/el.po
M src/pl/plpgsql/src/po/fr.po
M src/pl/plpgsql/src/po/ja.po
D src/pl/plpgsql/src/po/ro.po
M src/pl/plpgsql/src/po/ru.po
M src/pl/plpgsql/src/po/uk.po
M src/pl/plpgsql/src/po/zh_CN.po
D src/pl/plpgsql/src/po/zh_TW.po
M src/pl/plpython/po/el.po
M src/pl/plpython/po/fr.po
M src/pl/plpython/po/ja.po
M src/pl/plpython/po/ru.po
M src/pl/plpython/po/uk.po
M src/pl/plpython/po/zh_CN.po
M src/pl/tcl/nls.mk
M src/pl/tcl/po/el.po
M src/pl/tcl/po/fr.po
M src/pl/tcl/po/ja.po
D src/pl/tcl/po/pt_BR.po
D src/pl/tcl/po/ro.po
M src/pl/tcl/po/ru.po
M src/pl/tcl/po/uk.po
M src/pl/tcl/po/zh_CN.po
D src/pl/tcl/po/zh_TW.po
doc: Replace characters that the PDF build cannot handle
commit : 89a967e9b5262db1cea62a5725dd7bd7489d240d
author : Peter Eisentraut <[email protected]>
date : Mon, 20 Sep 2021 10:05:46 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 20 Sep 2021 10:05:46 +0200
A few characters in the acknowledgments list cannot be handled by the
PDF build, so replace with a similar ASCII character.
M doc/src/sgml/release-14.sgml
Update list of acknowledgments in release notes
commit : 3609b453cd655060562f8750ce0ce4bdddf43a57
author : Peter Eisentraut <[email protected]>
date : Mon, 20 Sep 2021 09:18:17 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 20 Sep 2021 09:18:17 +0200
current through 66061077155d68463ec00604ba7d6f0ae69716e8
M doc/src/sgml/release-14.sgml
Disallow extended statistics on system columns
commit : 66061077155d68463ec00604ba7d6f0ae69716e8
author : Tomas Vondra <[email protected]>
date : Mon, 20 Sep 2021 00:34:57 +0200
committer: Tomas Vondra <[email protected]>
date : Mon, 20 Sep 2021 00:34:57 +0200
Since introduction of extended statistics, we've disallowed references
to system columns. So for example
CREATE STATISTICS s ON ctid FROM t;
would fail. But with extended statistics on expressions, it was possible
to work around this limitation quite easily
CREATE STATISTICS s ON (ctid::text) FROM t;
This is an oversight in a4d75c86bf, fixed by adding a simple check.
Backpatch to PostgreSQL 14, where support for extended statistics on
expressions was introduced.
Backpatch-through: 14
Discussion: https://postgr.es/m/20210816013255.GS10479%40telsasoft.com
M src/backend/commands/statscmds.c
Doc: further tweaking of v14 release notes.
commit : 2a9a34c3100d8c0311c63baaa435be38a7d7aae5
author : Tom Lane <[email protected]>
date : Sun, 19 Sep 2021 12:10:34 -0400
committer: Tom Lane <[email protected]>
date : Sun, 19 Sep 2021 12:10:34 -0400
A recent question reminded me that the notes' description of
commit 86dc90056 rather undersold its benefits.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/release-14.sgml
Doc: fix typos.
commit : 55abb37506e9af4bbc4d8410ac8469b9d38e84e7
author : Tom Lane <[email protected]>
date : Sun, 19 Sep 2021 11:36:53 -0400
committer: Tom Lane <[email protected]>
date : Sun, 19 Sep 2021 11:36:53 -0400
"PGcon" should be "PGconn". Noted by D. Frey.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/lobj.sgml
Doc: copy-editing for v14 release notes.
commit : f8dacf125db42547579629be1936fa613bad8070
author : Tom Lane <[email protected]>
date : Sat, 18 Sep 2021 17:09:46 -0400
committer: Tom Lane <[email protected]>
date : Sat, 18 Sep 2021 17:09:46 -0400
Improve various item descriptions. Rearrange some things into
(IMO) more logical order. Fix missing markup and dubious
choices of link destinations. Drop a couple of items that
were later back-patched or otherwise don't seem to need
to be documented here.
M doc/src/sgml/release-14.sgml
Doc: update v14 release notes through today.
commit : 68b6ed42e69f920c67902e33d4a648bd39e0a19b
author : Tom Lane <[email protected]>
date : Sat, 18 Sep 2021 13:46:07 -0400
committer: Tom Lane <[email protected]>
date : Sat, 18 Sep 2021 13:46:07 -0400
Account for recent commits, notably reversion of 0827e8af7.
Strip trailing spaces.
M doc/src/sgml/release-14.sgml
pageinspect: Make page deletion elog less chatty.
commit : 55934416d2c50cdc9fba044a97fb14da1c779589
author : Peter Geoghegan <[email protected]>
date : Fri, 17 Sep 2021 14:19:50 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 17 Sep 2021 14:19:50 -0700
An elog that reports the value of a transaction ID stored on a deleted
nbtree page was added by commit e5d8a999, which taught page deletion to
store full 64-bit XIDs. It seems very chatty on further reflection, so
lower its elevel from NOTICE to DEBUG2.
Author: Peter Geoghegan <[email protected]>
Backpatch: 14-, just like the nbtree XID enhancement.
M contrib/pageinspect/btreefuncs.c
Fix pull_varnos to cope with translated PlaceHolderVars.
commit : 4d5b4483db1c6370861ca968edd753ab4dc03f9d
author : Tom Lane <[email protected]>
date : Fri, 17 Sep 2021 15:41:16 -0400
committer: Tom Lane <[email protected]>
date : Fri, 17 Sep 2021 15:41:16 -0400
Commit 55dc86eca changed pull_varnos to use (if possible) the associated
ph_eval_at for a PlaceHolderVar. I missed a fine point though: we might
be looking at a PHV in the quals or tlist of a child appendrel, in which
case we need to compute a ph_eval_at value that's been translated in the
same way that the PHV itself has been (cf. adjust_appendrel_attrs).
Fortunately, enough info is available in the PlaceHolderInfo to make
such translation possible without additional outside data, so we don't
need another round of uglification of planner APIs. This is a little
bit complicated, but since it's a hard-to-hit corner case, I'm not much
worried about adding cycles here.
Per report from Jaime Casanova. Back-patch to v12, like the previous
commit.
Discussion: https://postgr.es/m/20210915230959.GB17635@ahch-to
M src/backend/optimizer/util/var.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Fix EXPLAIN to handle SEARCH BREADTH FIRST queries.
commit : 388726753b638fb9938883bdd057b2ffe6f950f5
author : Tom Lane <[email protected]>
date : Thu, 16 Sep 2021 10:45:42 -0400
committer: Tom Lane <[email protected]>
date : Thu, 16 Sep 2021 10:45:42 -0400
The rewriter transformation for SEARCH BREADTH FIRST produces a
FieldSelect on a Var of type RECORD, where the Var references the
recursive union's worktable output. EXPLAIN VERBOSE failed to handle
this case, because it only expected such Vars to appear in CteScans
not WorkTableScans. Fix that, and add some test cases exercising
EXPLAIN on SEARCH and CYCLE queries.
In principle this oversight is an old bug, but it seems that the
case is unreachable without SEARCH BREADTH FIRST, because the
parser fails when attempting to create such a reference manually.
So for today I'll just patch HEAD/v14. Someday we might find that
the code portion of this patch needs to be back-patched further.
Per report from Atsushi Torikoshi.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Message style improvements
commit : f46dc96fcc652d5c29343e0eb4fa8e86a9468c36
author : Peter Eisentraut <[email protected]>
date : Thu, 16 Sep 2021 14:48:52 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 16 Sep 2021 14:48:52 +0200
M src/backend/commands/typecmds.c
M src/backend/libpq/auth.c
M src/backend/libpq/be-secure-openssl.c
M src/backend/utils/adt/jsonbsubs.c
M src/backend/utils/adt/multirangetypes.c
M src/test/regress/expected/jsonb.out
Fix performance regression from session statistics.
commit : 7890a423470b3a763aa22faf69a44cb6a2df8f0e
author : Andres Freund <[email protected]>
date : Thu, 16 Sep 2021 02:02:40 -0700
committer: Andres Freund <[email protected]>
date : Thu, 16 Sep 2021 02:02:40 -0700
Session statistics, as introduced by 960869da08, had several shortcomings:
- an additional GetCurrentTimestamp() call that also impaired the accuracy of
the data collected
This can be avoided by passing the current timestamp we already have in
pgstat_report_stat().
- an additional statistics UDP packet sent every 500ms
This is solved by adding the new statistics to PgStat_MsgTabstat.
This is conceptually ugly, because session statistics are not
table statistics. But the struct already contains data unrelated
to tables, so there is not much damage done.
Connection and disconnection are reported in separate messages, which
reduces the number of additional messages to two messages per session and a
slight increase in PgStat_MsgTabstat size (but the same number of table
stats fit).
- Session time computation could overflow on systems where long is 32 bit.
Reported-By: Andres Freund <[email protected]>
Author: Andres Freund <[email protected]>
Author: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/20210801205501.nyxzxoelqoo4x2qc%40alap3.anarazel.de
Backpatch: 14-, where the feature was introduced.
M src/backend/postmaster/pgstat.c
M src/backend/tcop/postgres.c
M src/backend/utils/activity/backend_status.c
M src/include/pgstat.h
M src/tools/pgindent/typedefs.list
Fix variable shadowing in procarray.c.
commit : 92a8d7610ea0f440a80328ced342b782180a3f43
author : Fujii Masao <[email protected]>
date : Thu, 16 Sep 2021 13:06:21 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 16 Sep 2021 13:06:21 +0900
ProcArrayGroupClearXid function has a parameter named "proc",
but the same name was used for its local variables. This commit fixes
this variable shadowing, to improve code readability.
Back-patch to all supported versions, to make future back-patching
easy though this patch is classified as refactoring only.
Reported-by: Ranier Vilela
Author: Ranier Vilela, Aleksander Alekseev
https://postgr.es/m/CAEudQAqyoTZC670xWi6w-Oe2_Bk1bfu2JzXz6xRfiOUzm7xbyQ@mail.gmail.com
M src/backend/storage/ipc/procarray.c
Use int instead of size_t in procarray.c.
commit : fe8821ca7da4bc1f72688cece0b7f2c3076d813d
author : Fujii Masao <[email protected]>
date : Thu, 16 Sep 2021 12:52:30 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 16 Sep 2021 12:52:30 +0900
All size_t variables declared in procarray.c are actually int ones.
Let's use int instead of size_t for those variables. Which would
reduce Wsign-compare compiler warnings.
Back-patch to v14 where commit 941697c3c1 added size_t variables
in procarray.c, to make future back-patching easy though
this patch is classified as refactoring only.
Reported-by: Ranier Vilela
Author: Ranier Vilela, Aleksander Alekseev
https://postgr.es/m/CAEudQAqyoTZC670xWi6w-Oe2_Bk1bfu2JzXz6xRfiOUzm7xbyQ@mail.gmail.com
M src/backend/storage/ipc/procarray.c
Disallow LISTEN in background workers.
commit : d84d62b6225b4af86cd9837b71f6fd2cf33fe80c
author : Tom Lane <[email protected]>
date : Wed, 15 Sep 2021 12:31:56 -0400
committer: Tom Lane <[email protected]>
date : Wed, 15 Sep 2021 12:31:56 -0400
It's possible to execute user-defined SQL in some background processes;
for example, logical replication workers can fire triggers. This opens
the possibility that someone would try to execute LISTEN in such a
context. But since only regular backends ever call
ProcessNotifyInterrupt, no messages would actually be received, and
thus the registered listener would simply prevent the message queue
from being cleaned. Eventually NOTIFY would stop working, which is bad.
Perhaps someday somebody will invent infrastructure to make listening
in a background worker actually useful. In the meantime, forbid it.
Back-patch to v13, which is where we introduced the MyBackendType
variable. It'd be a lot harder to implement the check without that,
and it doesn't seem worth the trouble.
Discussion: https://postgr.es/m/[email protected]
M src/backend/tcop/utility.c
Fix hash_array
commit : 9b2fd490577bc957429f337cfd72869eb8ef08c9
author : Peter Eisentraut <[email protected]>
date : Wed, 15 Sep 2021 11:59:34 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 15 Sep 2021 11:59:34 +0200
Commit 054adca641ac1279dc8d9b74fda41948ac35e9a9 neglected to
initialize the type_id field of the synthesized type cache entry, so
it would make a new one on every call.
Also, better use the per-function memory context for this; otherwise
it leaks memory.
Discussion: https://www.postgresql.org/message-id/flat/17158-8a2ba823982537a4%40postgresql.org
M src/backend/utils/adt/arrayfuncs.c
doc: Clarify refresh options for DROP PUBLICATION
commit : 6e6adbddbce1391ffb676ce9779f24c25a6a9f09
author : Daniel Gustafsson <[email protected]>
date : Wed, 15 Sep 2021 09:54:45 +0200
committer: Daniel Gustafsson <[email protected]>
date : Wed, 15 Sep 2021 09:54:45 +0200
The available refresh options are specified as refresh_options under
REFRESH PUBLICATION, and DROP PUBLICATION itself has an option named
refresh. Clarify what we mean by refresh options to avoid confusion.
Backpatch through v14 where ALTER SUBSCRIPTION ... DROP PUBLICATION
was introduced.
Author: Masahiko Sawada <[email protected]>
Reviewed-by: Amit Kapila <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Peter Smith <[email protected]>
Discussion: https://postgr.es/m/CAD21AoCm1wJ3A8Q9EmBjRbShYkJ+o+Oa_z9O0hvwhvhUa2BSyg@mail.gmail.com
Backpatch-through: 14
M doc/src/sgml/ref/alter_subscription.sgml
Send NOTIFY signals during CommitTransaction.
commit : 0eff10a008447bc6f6594547e2fc04756849eaed
author : Tom Lane <[email protected]>
date : Tue, 14 Sep 2021 17:18:25 -0400
committer: Tom Lane <[email protected]>
date : Tue, 14 Sep 2021 17:18:25 -0400
Formerly, we sent signals for outgoing NOTIFY messages within
ProcessCompletedNotifies, which was also responsible for sending
relevant ones of those messages to our connected client. It therefore
had to run during the main-loop processing that occurs just before
going idle. This arrangement had two big disadvantages:
* Now that procedures allow intra-command COMMITs, it would be
useful to send NOTIFYs to other sessions immediately at COMMIT
(though, for reasons of wire-protocol stability, we still shouldn't
forward them to our client until end of command).
* Background processes such as replication workers would not send
NOTIFYs at all, since they never execute the client communication
loop. We've had requests to allow triggers running in replication
workers to send NOTIFYs, so that's a problem.
To fix these things, move transmission of outgoing NOTIFY signals
into AtCommit_Notify, where it will happen during CommitTransaction.
Also move the possible call of asyncQueueAdvanceTail there, to
ensure we don't bloat the async SLRU if a background worker sends
many NOTIFYs with no one listening.
We can also drop the call of asyncQueueReadAllNotifications,
allowing ProcessCompletedNotifies to go away entirely. That's
because commit 790026972 added a call of ProcessNotifyInterrupt
adjacent to PostgresMain's call of ProcessCompletedNotifies,
and that does its own call of asyncQueueReadAllNotifications,
meaning that we were uselessly doing two such calls (inside two
separate transactions) whenever inbound notify signals coincided
with an outbound notify. We need only set notifyInterruptPending
to ensure that ProcessNotifyInterrupt runs, and we're done.
The existing documentation suggests that custom background workers
should call ProcessCompletedNotifies if they want to send NOTIFY
messages. To avoid an ABI break in the back branches, reduce it
to an empty routine rather than removing it entirely. Removal
will occur in v15.
Although the problems mentioned above have existed for awhile,
I don't feel comfortable back-patching this any further than v13.
There was quite a bit of churn in adjacent code between 12 and 13.
At minimum we'd have to also backpatch 51004c717, and a good deal
of other adjustment would also be needed, so the benefit-to-risk
ratio doesn't look attractive.
Per bug #15293 from Michael Powers (and similar gripes from others).
Artur Zakirov and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/bgworker.sgml
M src/backend/access/transam/xact.c
M src/backend/commands/async.c
M src/backend/tcop/postgres.c
M src/include/commands/async.h
Fix planner error with multiple copies of an AlternativeSubPlan.
commit : 29aa0ce361d16a79e4ebf7561cbb16ed1d0e2211
author : Tom Lane <[email protected]>
date : Tue, 14 Sep 2021 15:11:21 -0400
committer: Tom Lane <[email protected]>
date : Tue, 14 Sep 2021 15:11:21 -0400
It's possible for us to copy an AlternativeSubPlan expression node
into multiple places, for example the scan quals of several
partition children. Then it's possible that we choose a different
one of the alternatives as optimal in each place. Commit 41efb8340
failed to consider this scenario, so its attempt to remove "unused"
subplans could remove subplans that were still used elsewhere.
Fix by delaying the removal logic until we've examined all the
AlternativeSubPlans in a given query level. (This does assume that
AlternativeSubPlans couldn't get copied to other query levels, but
for the foreseeable future that's fine; cf qual_is_pushdown_safe.)
Per report from Rajkumar Raghuwanshi. Back-patch to v14
where the faulty logic came in.
Discussion: https://postgr.es/m/CAKcux6==O3NNZC3bZ2prRYv3cjm3_Zw1GfzmOjEVqYN4jub2+Q@mail.gmail.com
M src/backend/optimizer/plan/setrefs.c
M src/include/nodes/pathnodes.h
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
jit: Do not try to shut down LLVM state in case of LLVM triggered errors.
commit : 4e86887e0922f20add67e2473c7baae8c7f05d5e
author : Andres Freund <[email protected]>
date : Mon, 13 Sep 2021 18:07:19 -0700
committer: Andres Freund <[email protected]>
date : Mon, 13 Sep 2021 18:07:19 -0700
If an allocation failed within LLVM it is not safe to call back into LLVM as
LLVM is not generally safe against exceptions / stack-unwinding. Thus errors
while in LLVM code are promoted to FATAL. However llvm_shutdown() did call
back into LLVM even in such cases, while llvm_release_context() was careful
not to do so.
We cannot generally skip shutting down LLVM, as that can break profiling. But
it's OK to do so if there was an error from within LLVM.
Reported-By: Jelte Fennema <[email protected]>
Author: Andres Freund <[email protected]>
Author: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/AM5PR83MB0178C52CCA0A8DEA0207DC14F7FF9@AM5PR83MB0178.EURPRD83.prod.outlook.com
Backpatch: 11-, where jit was introduced
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_error.cpp
M src/include/jit/llvmjit.h
Fix potential for compiler warning in GlobalVisTestFor().
commit : 0d0bbee5e3ee2eabe3a1f6081cbc4226520764c0
author : Andres Freund <[email protected]>
date : Mon, 13 Sep 2021 16:50:10 -0700
committer: Andres Freund <[email protected]>
date : Mon, 13 Sep 2021 16:50:10 -0700
In d9d8aa9bb9a I added a defensive NULL assignment to protect against a
not-too-smart compiler warning about unitialized variable use after the
switch. Unfortunately I only did so on master and forgot to adjust that for
14.
Stephen noticed that there actually is a compiler warning :(.
Reported-By: Stephen Frost <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/procarray.c
Clear conn->errorMessage at successful completion of PQconnectdb().
commit : 896a0c44f93b0b449fdf7e6c3973deab5fdfed4f
author : Tom Lane <[email protected]>
date : Mon, 13 Sep 2021 16:53:11 -0400
committer: Tom Lane <[email protected]>
date : Mon, 13 Sep 2021 16:53:11 -0400
Commits ffa2e4670 and 52a10224e caused libpq's connection-establishment
functions to usually leave a nonempty string in the connection's
errorMessage buffer, even after a successful connection. While that
was intentional on my part, more sober reflection says that it wasn't
a great idea: the string would be a bit confusing. Also this broke at
least one application that checked for connection success by examining
the errorMessage, instead of using PQstatus() as documented. Let's
clear the buffer at success exit, restoring the pre-v14 behavior.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-connect.c
Fix EXIT out of outermost block in plpgsql.
commit : 4ffd3fe4d6ccc9ff36271e5a21ea3b065621b982
author : Tom Lane <[email protected]>
date : Mon, 13 Sep 2021 12:42:03 -0400
committer: Tom Lane <[email protected]>
date : Mon, 13 Sep 2021 12:42:03 -0400
Ordinarily, using EXIT this way would draw "control reached end of
function without RETURN". However, if the function is one where we
don't require an explicit RETURN (such as a DO block), that should
not happen. It did anyway, because add_dummy_return() neglected to
account for the case.
Per report from Herwig Goemans. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/pl/plpgsql/src/expected/plpgsql_control.out
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/sql/plpgsql_control.sql
doc: fix PG 14 release note typo
commit : 10faeb7ed5049afb70d26561e9991d52c363bd8e
author : Bruce Momjian <[email protected]>
date : Mon, 13 Sep 2021 10:42:16 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 13 Sep 2021 10:42:16 -0400
Reported-by: Robert Haas
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
Doc: Remove type information for import_generated in postgres-fdw.sgml.
commit : 749945c2425b7c83d95b0359733f4839049fdcb8
author : Etsuro Fujita <[email protected]>
date : Mon, 13 Sep 2021 17:30:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Mon, 13 Sep 2021 17:30:00 +0900
The type information for FDW options is only added to HEAD; remove this
from back branches. Oversight in commit aa769f80e.
Apply the patch to v12, v13, and v14.
Discussion: https://postgr.es/m/CAPmGK14z92twaKwRoccHbbh5Va5vbRDZcTYYTx50+0JTQ8xx_g@mail.gmail.com
M doc/src/sgml/postgres-fdw.sgml
Fix reorder buffer memory accounting for toast changes.
commit : f5e0ff4631b5402950db69c989c88c4c6295504b
author : Amit Kapila <[email protected]>
date : Mon, 13 Sep 2021 10:35:00 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 13 Sep 2021 10:35:00 +0530
While processing toast changes in logical decoding, we rejigger the
tuple change to point to in-memory toast tuples instead to on-disk toast
tuples. And, to make sure the memory accounting is correct, we were
subtracting the old change size and then after re-computing the new tuple,
re-adding its size at the end. Now, if there is any error before we add
the new size, we will release the changes and that will update the
accounting info (subtracting the size from the counters). And we were
underflowing there which leads to an assertion failure in assert enabled
builds and wrong memory accounting in reorder buffer otherwise.
Author: Bertrand Drouvot
Reviewed-by: Amit Kapila
Backpatch-through: 13, where memory accounting was introduced
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/reorderbuffer.c
Fix error handling with threads on OOM in ECPG connection logic
commit : cc057fb315e24b78552f5b7ac05792418042e71b
author : Michael Paquier <[email protected]>
date : Mon, 13 Sep 2021 13:24:04 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 13 Sep 2021 13:24:04 +0900
An out-of-memory failure happening when allocating the structures to
store the connection parameter keywords and values would mess up with
the set of connections saved, as on failure the pthread mutex would
still be hold with the new connection object listed but free()'d.
Rather than just unlocking the mutex, which would leave the static list
of connections into an inconsistent state, move the allocation for the
structures of the connection parameters before beginning the test
manipulation. This ensures that the list of connections and the
connection mutex remain consistent all the time in this code path.
This error is unlikely going to happen, but this could mess up badly
with ECPG clients in surprising ways, so backpatch all the way down.
Reported-by: ryancaicse
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/interfaces/ecpg/ecpglib/connect.c
Make pg_regexec() robust against out-of-range search_start.
commit : b33283cbd336adbf982c21aac1399130c8ffaaa9
author : Tom Lane <[email protected]>
date : Sat, 11 Sep 2021 15:19:31 -0400
committer: Tom Lane <[email protected]>
date : Sat, 11 Sep 2021 15:19:31 -0400
If search_start is greater than the length of the string, we should just
return REG_NOMATCH immediately. (Note that the equality case should
*not* be rejected, since the pattern might be able to match zero
characters.) This guards various internal assumptions that the min of a
range of string positions is not more than the max. Violation of those
assumptions could allow an attempt to fetch string[search_start-1],
possibly causing a crash.
Jaime Casanova pointed out that this situation is reachable with the
new regexp_xxx functions that accept a user-specified start position.
I don't believe it's reachable via any in-core call site in v14 and
below. However, extensions could possibly call pg_regexec with an
out-of-range search_start, so let's back-patch the fix anyway.
Discussion: https://postgr.es/m/20210911180357.GA6870@ahch-to
M src/backend/regex/regexec.c
Fix some anomalies with NO SCROLL cursors.
commit : d844cd75a6764a1dd2d8b3410144df9ebf8a0f04
author : Tom Lane <[email protected]>
date : Fri, 10 Sep 2021 13:18:32 -0400
committer: Tom Lane <[email protected]>
date : Fri, 10 Sep 2021 13:18:32 -0400
We have long forbidden fetching backwards from a NO SCROLL cursor,
but the prohibition didn't extend to cases in which we rewind the
query altogether and then re-fetch forwards. I think the reason is
that this logic was mainly meant to protect plan nodes that can't
be run in the reverse direction. However, re-reading the query output
is problematic if the query is volatile (which includes SELECT FOR
UPDATE, not just queries with volatile functions): the re-read can
produce different results, which confuses the cursor navigation logic
completely. Another reason for disliking this approach is that some
code paths will either fetch backwards or rewind-and-fetch-forwards
depending on the distance to the target row; so that seemingly
identical use-cases may or may not draw the "cursor can only scan
forward" error. Hence, let's clean things up by disallowing rewind
as well as fetch-backwards in a NO SCROLL cursor.
Ordinarily we'd only make such a definitional change in HEAD, but
there is a third reason to consider this change now. Commit ba2c6d6ce
created some new user-visible anomalies for non-scrollable cursors
WITH HOLD, in that navigation in the cursor result got confused if the
cursor had been partially read before committing. The only good way
to resolve those anomalies is to forbid rewinding such a cursor, which
allows removal of the incorrect cursor state manipulations that
ba2c6d6ce added to PersistHoldablePortal.
To minimize the behavioral change in the back branches (including
v14), refuse to rewind a NO SCROLL cursor only when it has a holdStore,
ie has been held over from a previous transaction due to WITH HOLD.
This should avoid breaking most applications that have been sloppy
about whether to declare cursors as scrollable. We'll enforce the
prohibition across-the-board beginning in v15.
Back-patch to v11, as ba2c6d6ce was.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/portalcmds.c
M src/backend/tcop/pquery.c
M src/test/regress/expected/portals.out
M src/test/regress/sql/portals.sql
Avoid fetching from an already-terminated plan.
commit : b7056c0a25e545f4c1efcc1e2a6118e4626e377b
author : Tom Lane <[email protected]>
date : Thu, 9 Sep 2021 13:36:31 -0400
committer: Tom Lane <[email protected]>
date : Thu, 9 Sep 2021 13:36:31 -0400
Some plan node types don't react well to being called again after
they've already returned NULL. PortalRunSelect() has long dealt
with this by calling the executor with NoMovementScanDirection
if it sees that we've already run the portal to the end. However,
commit ba2c6d6ce overlooked this point, so that persisting an
already-fully-fetched cursor would fail if it had such a plan.
Per report from Tomas Barton. Back-patch to v11, as the faulty
commit was. (I've omitted a test case because the type of plan
that causes a problem isn't all that stable.)
Discussion: https://postgr.es/m/CAPV2KRjd=ErgVGbvO2Ty20tKTEZZr6cYsYLxgN_W3eAo9pf5sw@mail.gmail.com
M src/backend/commands/portalcmds.c
pgbench: Stop counting skipped transactions as soon as timer is exceeded.
commit : b27d0cd3147e2a0215253f61944c392a30265db8
author : Fujii Masao <[email protected]>
date : Fri, 10 Sep 2021 01:28:17 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 10 Sep 2021 01:28:17 +0900
When throttling is used, transactions that lag behind schedule by
more than the latency limit are counted and reported as skipped.
Previously, there was the case where pgbench counted all skipped
transactions even if the timer specified in -T option was exceeded.
This could take a very long time to do that especially when unrealistically
high rate setting in -R option caused quite a lot of transactions that
lagged behind schedule. This could prevent pgbench from ending
immediately, and so pgbench could look like it got stuck to users.
To fix the issue, this commit changes pgbench so that it stops counting
skipped transactions as soon as the timer is exceeded. The timer can
make pgbench end soon even when there are lots of skipped transactions
that have not been counted yet.
Note that there is no guarantee that all skipped transactions are
counted under -T though there is under -t. This is OK in practice
because it's very unlikely to happen with realistic setting. Also this is
not the issue that this commit newly introdues. There used to be
the case where pgbench ended without counting all skipped
transactions since before.
Back-patch to v14. Per discussion, we decided not to bother
back-patch to the stable branches because it's hard to imagine
the issue happens in practice (with realistic setting).
Author: Yugo Nagata, Fabien COELHO
Reviewed-by: Greg Sabino Mullane, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/bin/pgbench/pgbench.c
Check for relation length overrun soon enough.
commit : 7430c774209cd98bbc33076cc3c07497c1086d9a
author : Tom Lane <[email protected]>
date : Thu, 9 Sep 2021 11:45:48 -0400
committer: Tom Lane <[email protected]>
date : Thu, 9 Sep 2021 11:45:48 -0400
We don't allow relations to exceed 2^32-1 blocks, because block
numbers are 32 bits and the last possible block number is reserved
to mean InvalidBlockNumber. There is a check for this in mdextend,
but that's really way too late, because the smgr API requires us to
create a buffer for the block-to-be-added, and we do not want to
have any buffer with blocknum InvalidBlockNumber. (Such a case
can trigger assertions in bufmgr.c, plus I think it might confuse
ReadBuffer's logic for data-past-EOF later on.) So put the check
into ReadBuffer.
Per report from Christoph Berg. It's been like this forever,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/smgr/md.c
Fix issue with WAL archiving in standby.
commit : b5ec22bf5e8f19964f7c7d7ef357ff947a38c7fc
author : Fujii Masao <[email protected]>
date : Thu, 9 Sep 2021 23:58:05 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 9 Sep 2021 23:58:05 +0900
Previously, walreceiver always closed the currently-opened WAL segment
and created its archive notification file, after it finished writing
the current segment up and received any WAL data that should be
written into the next segment. If walreceiver exited just before
any WAL data in the next segment arrived at standby, it did not
create the archive notification file of the current segment
even though that's known completed. This behavior could cause
WAL archiving of the segment to be delayed until subsequent
restartpoints or checkpoints created its notification file.
To fix the issue, this commit changes walreceiver so that it creates
an archive notification file of a current WAL segment immediately
if that's known completed before receiving next WAL data.
Back-patch to all supported branches.
Reported-by: Kyotaro Horiguchi
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/walreceiver.c
Avoid useless malloc/free traffic around getFormattedTypeName().
commit : 52c300df323e8ebb3d0613baa4711179d9f342f3
author : Tom Lane <[email protected]>
date : Wed, 8 Sep 2021 15:09:42 -0400
committer: Tom Lane <[email protected]>
date : Wed, 8 Sep 2021 15:09:42 -0400
Coverity complained that one caller of getFormattedTypeName() failed
to free the returned string. Which is true, but rather than fixing
that one, let's get rid of this tedious and error-prone requirement.
Now that getFormattedTypeName() caches its result, strdup'ing that
result and expecting the caller to free it accomplishes little except
to waste cycles. We do create a leak in the case where getTypes didn't
make a TypeInfo for the type, but that basically shouldn't ever happen.
Back-patch, as commit 6c450a861 was. This isn't a particularly
interesting bug fix, but the API change seems like a hazard for
future back-patching activity if we don't back-patch it.
M src/bin/pg_dump/pg_dump.c
Fix misleading comments about TOAST access macros.
commit : 67948a433e18a8561daf89dcc53e591a3fac659a
author : Tom Lane <[email protected]>
date : Wed, 8 Sep 2021 14:11:35 -0400
committer: Tom Lane <[email protected]>
date : Wed, 8 Sep 2021 14:11:35 -0400
Seems to have been my error in commit aeb1631ed.
Noted by Christoph Berg.
Discussion: https://postgr.es/m/[email protected]
M src/include/postgres.h
Fix rewriter to set hasModifyingCTE correctly on rewritten queries.
commit : 03d01d746b9338dd1568d37d26b344928f82c5ff
author : Tom Lane <[email protected]>
date : Wed, 8 Sep 2021 12:05:43 -0400
committer: Tom Lane <[email protected]>
date : Wed, 8 Sep 2021 12:05:43 -0400
If we copy data-modifying CTEs from the original query to a replacement
query (from a DO INSTEAD rule), we must set hasModifyingCTE properly
in the replacement query. Failure to do this can cause various
unpleasantness, such as unsafe usage of parallel plans. The code also
neglected to propagate hasRecursive, though that's only cosmetic at
the moment.
A difficulty arises if the rule action is an INSERT...SELECT. We
attach the original query's RTEs and CTEs to the sub-SELECT Query, but
data-modifying CTEs are only allowed to appear in the topmost Query.
For the moment, throw an error in such cases. It would probably be
possible to avoid this error by attaching the CTEs to the top INSERT
Query instead; but that would require a bunch of new code to adjust
ctelevelsup references. Given the narrowness of the use-case, and
the need to back-patch this fix, it does not seem worth the trouble
for now. We can revisit this if we get field complaints.
Per report from Greg Nancarrow. Back-patch to all supported branches.
(The test case added here does not fail before v10, but there are
plenty of places checking top-level hasModifyingCTE in 9.6, so I have
no doubt that this code change is necessary there too.)
Greg Nancarrow and Tom Lane
Discussion: https://postgr.es/m/CAJcOf-f68DT=26YAMz_i0+Au3TcLO5oiHY5=fL6Sfuits6r+_w@mail.gmail.com
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Disable anonymous record hash support except in special cases
commit : 054adca641ac1279dc8d9b74fda41948ac35e9a9
author : Peter Eisentraut <[email protected]>
date : Wed, 8 Sep 2021 09:25:46 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 8 Sep 2021 09:25:46 +0200
Commit 01e658fa74 added hash support for row types. This also added
support for hashing anonymous record types, using the same approach
that the type cache uses for comparison support for record types: It
just reports that it works, but it might fail at run time if a
component type doesn't actually support the operation. We get away
with that for comparison because most types support that. But some
types don't support hashing, so the current state can result in
failures at run time where the planner chooses hashing over sorting,
whereas that previously worked if only sorting was an option.
We do, however, want the record hashing support for path tracking in
recursive unions, and the SEARCH and CYCLE clauses built on that. In
that case, hashing is the only plan option. So enable that, this
commit implements the following approach: The type cache does not
report that hashing is available for the record type. This undoes
that part of 01e658fa74. Instead, callers that require hashing no
matter what can override that result themselves. This patch only
touches the callers to make the aforementioned recursive query cases
work, namely the parse analysis of unions, as well as the hash_array()
function.
Reported-by: Sait Talha Nisanci <[email protected]>
Bug: #17158
Discussion: https://www.postgresql.org/message-id/flat/17158-8a2ba823982537a4%40postgresql.org
M src/backend/parser/analyze.c
M src/backend/rewrite/rewriteSearchCycle.c
M src/backend/utils/adt/arrayfuncs.c
M src/backend/utils/cache/typcache.c
M src/include/parser/analyze.h
M src/test/regress/expected/union.out
M src/test/regress/sql/union.sql
Invalidate relcache for publications defined for all tables.
commit : 8db27fbc11974ef491fec97be3365277ed4fab20
author : Amit Kapila <[email protected]>
date : Wed, 8 Sep 2021 09:58:28 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 8 Sep 2021 09:58:28 +0530
Updates/Deletes on a relation were allowed even without replica identity
after we define the publication for all tables. This would later lead to
an error on subscribers. The reason was that for such publications we were
not invalidating the relcache and the publication information for
relations was not getting rebuilt. Similarly, we were not invalidating the
relcache after dropping of such publications which will prohibit
Updates/Deletes without replica identity even without any publication.
Author: Vignesh C and Hou Zhijie
Reviewed-by: Hou Zhijie, Kyotaro Horiguchi, Amit Kapila
Backpatch-through: 10, where it was introduced
Discussion: https://postgr.es/m/CALDaNm0pF6zeWqCA8TCe2sDuwFAy8fCqba=nHampCKag-qLixg@mail.gmail.com
M src/backend/catalog/dependency.c
M src/backend/commands/publicationcmds.c
M src/include/commands/publicationcmds.h
M src/test/regress/expected/publication.out
M src/test/regress/sql/publication.sql
Consistently use read-only instead of "read only"
commit : b7fd291042a846b04439f122cb81a41d3cd2e8de
author : Magnus Hagander <[email protected]>
date : Tue, 7 Sep 2021 21:59:25 +0200
committer: Magnus Hagander <[email protected]>
date : Tue, 7 Sep 2021 21:59:25 +0200
This affects one message and some documentation that used the format
"read only", unlike everything else that used read-only.
Backpatch-through: 14
Discussion: https://postgr.es/m/CABUevExuxKwn0YM3+wdSeQSvK6CRrJ-hewocGVX3R4-xVX4eMw@mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/mvcc.sgml
M src/backend/postmaster/postmaster.c
Finish reverting 3eda9fc09fd6b9a1aec2d0113c633c69c3214b4d.
commit : 8b895374cd9cf6989bcee5b6f70f65f2d3520224
author : Tom Lane <[email protected]>
date : Tue, 7 Sep 2021 10:52:25 -0400
committer: Tom Lane <[email protected]>
date : Tue, 7 Sep 2021 10:52:25 -0400
Commit 67c33a114 should have set v14's catversion back to what it was
before 3eda9fc09, to avoid forcing a useless pg_upgrade cycle on users
of 14beta3. Do that now.
Discussion: https://postgr.es/m/[email protected]
M src/include/catalog/catversion.h
Fix missing words in comment.
commit : e66add755df2eae5e874dc96743f3faaf4bced83
author : Heikki Linnakangas <[email protected]>
date : Tue, 7 Sep 2021 10:28:55 +0300
committer: Heikki Linnakangas <[email protected]>
date : Tue, 7 Sep 2021 10:28:55 +0300
Introduced by commit c3928b467a, backpatch to v14 like that one.
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA+HiwqFQgNLS6VGntMcuJV6erBFV425xA6wBVnY=41GK4zC0Bw@mail.gmail.com
M src/backend/executor/nodeForeignscan.c
AIX: Fix missing libpq symbols by respecting SHLIB_EXPORTS.
commit : 47d54b6ba2749f5da72b7ab612e53e7f7b45b819
author : Noah Misch <[email protected]>
date : Mon, 6 Sep 2021 11:27:59 -0700
committer: Noah Misch <[email protected]>
date : Mon, 6 Sep 2021 11:27:59 -0700
We make each AIX shared library export all globals found in .o files
that originate in the library. That doesn't include symbols acquired by
-lpgcommon_shlib. That is good on average, but it became a problem for
libpq when commit e6afa8918c461c1dd80c5063a950518fa4e950cd moved five
official libpq API symbols into src/common. Fix this by implementing
the SHLIB_EXPORTS mechanism for AIX, so affected libraries export the
same symbols that they export on Linux. This reintroduces symbols
pg_encoding_to_char, pg_utf_mblen, pg_char_to_encoding,
pg_valid_server_encoding, and pg_valid_server_encoding_id. Back-patch
to v13, where the aforementioned commit first appeared. While a minor
release is usually the wrong time to add or remove symbol exports in
libpq or libecpg, we should expect users to want each documented symbol.
Tony Reix
Discussion: https://postgr.es/m/PR3PR02MB6396742E2FC3E77D37A920BC86C79@PR3PR02MB6396.eurprd02.prod.outlook.com
M src/Makefile.shlib
M src/port/README
Fix bogus timetz_zone() results for DYNTZ abbreviations.
commit : 599c73a91a0471465a84f12fe6a2e7236a825721
author : Tom Lane <[email protected]>
date : Mon, 6 Sep 2021 11:29:52 -0400
committer: Tom Lane <[email protected]>
date : Mon, 6 Sep 2021 11:29:52 -0400
timetz_zone() delivered completely wrong answers if the zone was
specified by a dynamic TZ abbreviation, because it failed to account
for the difference between the POSIX conventions for field values in
struct pg_tm and the conventions used in PG-specific datetime code.
As a stopgap fix, just adjust the tm_year and tm_mon fields to match
PG conventions. This is fixed in a different way in HEAD (388e71af8)
but I don't want to back-patch the change of reference point.
Discussion: https://postgr.es/m/CAJ7c6TOMG8zSNEZtCn5SPe+cCk3Lfxb71ZaQwT2F4T7PJ_t=KA@mail.gmail.com
M src/backend/utils/adt/date.c
Fix pkg-config files for static linking
commit : 1e9afc868eb5a3b2f50629c2724d9fcdbe0afee2
author : Peter Eisentraut <[email protected]>
date : Mon, 6 Sep 2021 09:41:03 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 6 Sep 2021 09:41:03 +0200
Since ea53100d5 (PostgreSQL 12), the shipped pkg-config files have
been broken for statically linking libpq because libpgcommon and
libpgport are missing. This patch adds those two missing private
dependencies (in a non-hardcoded way).
Reported-by: Filip Gospodinov <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/Makefile.shlib
Further portability tweaks for float4/float8 hash functions.
commit : 718978d9daa7128e0f5e4e06c0fd402cdd63704f
author : Tom Lane <[email protected]>
date : Sat, 4 Sep 2021 16:29:08 -0400
committer: Tom Lane <[email protected]>
date : Sat, 4 Sep 2021 16:29:08 -0400
Attempting to make hashfloat4() look as much as possible like
hashfloat8(), I'd figured I could replace NaNs with get_float4_nan()
before widening to float8. However, results from protosciurus
and topminnow show that on some platforms that produces a different
bit-pattern from get_float8_nan(), breaking the intent of ce773f230.
Rearrange so that we use the result of get_float8_nan() for all NaN
cases. As before, back-patch.
M src/backend/access/hash/hashfunc.c
Minor improvements for psql help output.
commit : 5edbcbd5b94c696526e18bbb490ab2e9f4282826
author : Tom Lane <[email protected]>
date : Sat, 4 Sep 2021 13:27:55 -0400
committer: Tom Lane <[email protected]>
date : Sat, 4 Sep 2021 13:27:55 -0400
Fix alphabetization of the output of "\?", and improve one description.
Update PageOutput counts where needed, fixing breakage from previous
patches.
Haiying Tang (PageOutput fixes by me)
Discussion: https://postgr.es/m/OS0PR01MB61136018064660F095CB57A8FB129@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/bin/psql/help.c
Revert "Avoid creating archive status ".ready" files too early"
commit : aa8bd0890bb29356a465d235dee59c1b08fa5fc5
author : Alvaro Herrera <[email protected]>
date : Sat, 4 Sep 2021 12:14:30 -0400
committer: Alvaro Herrera <[email protected]>
date : Sat, 4 Sep 2021 12:14:30 -0400
This reverts commit 515e3d84a0b5 and equivalent commits in back
branches. This solution to the problem has a number of problems, so
we'll try again with a different approach.
Per note from Andres Freund
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
M src/backend/postmaster/walwriter.c
M src/include/access/xlog.h
M src/include/access/xlogdefs.h
Remove arbitrary MAXPGPATH limit on command lengths in pg_ctl.
commit : 69d670e68ec9bd00b71ddc528274746790d7b6bd
author : Tom Lane <[email protected]>
date : Fri, 3 Sep 2021 21:04:44 -0400
committer: Tom Lane <[email protected]>
date : Fri, 3 Sep 2021 21:04:44 -0400
Replace fixed-length command buffers with psprintf() calls. We didn't
have anything as convenient as psprintf() when this code was written,
but now that we do, there's little reason for the limitation to
stand. Removing it eliminates some corner cases where (for example)
starting the postmaster with a whole lot of options fails.
Most individual file names that pg_ctl deals with are still restricted
to MAXPGPATH, but we've seldom had complaints about that limitation
so long as it only applies to one filename.
Back-patch to all supported branches.
Phil Krylov
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_ctl/pg_ctl.c
Disallow creating an ICU collation if the DB encoding won't support it.
commit : 2cc018ba8f80a5236faf2c58dfcd18fee581dbbc
author : Tom Lane <[email protected]>
date : Fri, 3 Sep 2021 16:38:55 -0400
committer: Tom Lane <[email protected]>
date : Fri, 3 Sep 2021 16:38:55 -0400
Previously this was allowed, but the collation effectively vanished
into the ether because of the way lookup_collation() works: you could
not use the collation, nor even drop it. Seems better to give an
error up front than to leave the user wondering why it doesn't work.
(Because this test is in DefineCollation not CreateCollation, it does
not prevent pg_import_system_collations from creating ICU collations,
regardless of the initially-chosen encoding.)
Per bug #17170 from Andrew Bille. Back-patch to v10 where ICU support
was added.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/collationcmds.c
Set the volatility of the timestamptz version of date_bin() back to immutable
commit : 67c33a114f38edbd66f68d7b2c0cb7b03611ec48
author : John Naylor <[email protected]>
date : Fri, 3 Sep 2021 13:38:15 -0400
committer: John Naylor <[email protected]>
date : Fri, 3 Sep 2021 13:38:15 -0400
543f36b43d was too hasty in thinking that the volatility of date_bin()
had to match date_trunc(), since only the latter references
session_timezone.
Bump catversion
Per feedback from Aleksander Alekseev
Backpatch to v14, as the former commit was
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Fix portability issue in tests from commit ce773f230.
commit : 08b96a2b5243957acd3f7c3105af8b5b2d0da3ef
author : Tom Lane <[email protected]>
date : Fri, 3 Sep 2021 10:01:02 -0400
committer: Tom Lane <[email protected]>
date : Fri, 3 Sep 2021 10:01:02 -0400
Modern POSIX seems to require strtod() to accept "-NaN", but there's
nothing about NaN in SUSv2, and some of our oldest buildfarm members
don't like it. Let's try writing it as -'NaN' instead; that seems
to produce the same result, at least on Intel hardware.
Per buildfarm.
M src/test/regress/expected/hash_func.out
M src/test/regress/sql/hash_func.sql
In count_usable_fds(), duplicate stderr not stdin.
commit : 6b54f12332a184bfd03524c012795678ddc992f4
author : Tom Lane <[email protected]>
date : Thu, 2 Sep 2021 18:53:10 -0400
committer: Tom Lane <[email protected]>
date : Thu, 2 Sep 2021 18:53:10 -0400
We had a complaint that the postmaster fails to start if the invoking
program closes stdin. That happens because count_usable_fds expects
to be able to dup(0), and if it can't, we conclude there are no free
FDs and go belly-up. So far as I can find, though, there is no other
place in the server that touches stdin, and it's not unreasonable to
expect that a daemon wouldn't use that file.
As a simple improvement, let's dup FD 2 (stderr) instead. Unlike stdin,
it *is* reasonable for us to expect that stderr be open; even if we are
configured not to touch it, common libraries such as libc might try to
write error messages there.
Per gripe from Mario Emmenlauer. Given the lack of previous complaints,
I'm not excited about pushing this into stable branches, but it seems
OK to squeeze it into v14.
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/file/fd.c
Fix float4/float8 hash functions to produce uniform results for NaNs.
commit : 23c6bc581dc90d08421a13494f135681504ee4e6
author : Tom Lane <[email protected]>
date : Thu, 2 Sep 2021 17:24:42 -0400
committer: Tom Lane <[email protected]>
date : Thu, 2 Sep 2021 17:24:42 -0400
The IEEE 754 standard allows a wide variety of bit patterns for NaNs,
of which at least two ("NaN" and "-NaN") are pretty easy to produce
from SQL on most machines. This is problematic because our btree
comparison functions deem all NaNs to be equal, but our float hash
functions know nothing about NaNs and will happily produce varying
hash codes for them. That causes unexpected results from queries
that hash a column containing different NaN values. It could also
produce unexpected lookup failures when using a hash index on a
float column, i.e. "WHERE x = 'NaN'" will not find all the rows
it should.
To fix, special-case NaN in the float hash functions, not too much
unlike the existing special case that forces zero and minus zero
to hash the same. I arranged for the most vanilla sort of NaN
(that coming from the C99 NAN constant) to still have the same
hash code as before, to reduce the risk to existing hash indexes.
I dithered about whether to back-patch this into stable branches,
but ultimately decided to do so. It's a clear improvement for
queries that hash internally. If there is anybody who has -NaN
in a hash index, they'd be well advised to re-index after applying
this patch ... but the misbehavior if they don't will not be much
worse than the misbehavior they had before.
Per bug #17172 from Ma Liangzhu.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/hash/hashfunc.c
M src/test/regress/expected/hash_func.out
M src/test/regress/sql/hash_func.sql
doc: Replace some uses of "which" by "that" in parallel.sgml
commit : 2c1981ec3c8e1c5a52a8e9513a832d362f4fb4c1
author : Michael Paquier <[email protected]>
date : Thu, 2 Sep 2021 11:35:56 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 2 Sep 2021 11:35:56 +0900
This makes the documentation more accurate grammatically.
Author: Elena Indrupskaya
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M doc/src/sgml/parallel.sgml
Doc: clarify how triggers relate to transactions.
commit : 95bc40f880a68dc092ca34c4813f2b27962f233d
author : Tom Lane <[email protected]>
date : Wed, 1 Sep 2021 17:24:59 -0400
committer: Tom Lane <[email protected]>
date : Wed, 1 Sep 2021 17:24:59 -0400
Laurenz Albe, per gripe from Nathan Long.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_trigger.sgml
M doc/src/sgml/trigger.sgml
Identify simple column references in extended statistics
commit : 50ba70a957f9e018495a111fc4b5e5eb2ea62044
author : Tomas Vondra <[email protected]>
date : Wed, 1 Sep 2021 17:41:54 +0200
committer: Tomas Vondra <[email protected]>
date : Wed, 1 Sep 2021 17:41:54 +0200
Until now, when defining extended statistics, everything except a plain
column reference was treated as complex expression. So for example "a"
was a column reference, but "(a)" would be an expression. In most cases
this does not matter much, but there were a couple strange consequences.
For example
CREATE STATISTICS s ON a FROM t;
would fail, because extended stats require at least two columns. But
CREATE STATISTICS s ON (a) FROM t;
would succeed, because that requirement does not apply to expressions.
Moreover, that statistics object is useless - the optimizer will always
use the regular statistics collected for attribute "a".
So do a bit more work to identify those expressions referencing a single
column, and translate them to a simple column reference. Backpatch to
14, where support for extended statistics on expressions was introduced.
Reported-by: Justin Pryzby
Backpatch-through: 14
Discussion: https://postgr.es/m/20210816013255.GS10479%40telsasoft.com
M src/backend/commands/statscmds.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
pgbench: Fix bug in measurement of disconnection delays.
commit : d760d942c73c9a161feefb7dc4a0004b9b4e7787
author : Fujii Masao <[email protected]>
date : Wed, 1 Sep 2021 17:05:13 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 1 Sep 2021 17:05:13 +0900
When -C/--connect option is specified, pgbench establishes and closes
the connection for each transaction. In this case pgbench needs to
measure the times taken for all those connections and disconnections,
to include the average connection time in the benchmark result.
But previously pgbench could not measure those disconnection delays.
To fix the bug, this commit makes pgbench measure the disconnection
delays whenever the connection is closed at the end of transaction,
if -C/--connect option is specified.
Back-patch to v14. Per discussion, we concluded not to back-patch to v13
or before because changing that behavior in stable branches would
surprise users rather than providing benefits.
Author: Yugo Nagata
Reviewed-by: Fabien COELHO, Tatsuo Ishii, Asif Rehman, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/bin/pgbench/pgbench.c
Fix the random test failure in 001_rep_changes.
commit : b7ad093d50e13aadfffb1662f53cd16a1c59e09d
author : Amit Kapila <[email protected]>
date : Wed, 1 Sep 2021 10:00:12 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 1 Sep 2021 10:00:12 +0530
The check to test whether the subscription workers were restarting after a
change in the subscription was failing. The reason was that the test was
assuming the walsender started before it reaches the 'streaming' state and
the walsender was exiting due to an error before that. Now, the walsender
was erroring out before reaching the 'streaming' state because it tries to
acquire the slot before the previous walsender has exited.
In passing, improve the die messages so that it is easier to investigate
the failures in the future if any.
Reported-by: Michael Paquier, as per buildfarm
Author: Ajin Cherian
Reviewed-by: Masahiko Sawada, Amit Kapila
Backpatch-through: 10, where this test was introduced
Discussion: https://postgr.es/m/[email protected]
M src/test/subscription/t/001_rep_changes.pl
VACUUM VERBOSE: Don't report "pages removed".
commit : 0d892cf73a13b3a32af438a059a168e711aa0a7f
author : Peter Geoghegan <[email protected]>
date : Tue, 31 Aug 2021 20:37:17 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 31 Aug 2021 20:37:17 -0700
It doesn't make any sense to report this information, since VACUUM
VERBOSE reports on heap relation truncation directly. This was an
oversight in commit 7ab96cf6, which made VACUUM VERBOSE output a little
more consistent with nearby autovacuum-specific log output. Adjust
comments that describe how this is supposed to work in passing.
Also bring truncation-related VACUUM VERBOSE output in line with the
convention established for VACUUM VERBOSE output by commit f4f4a649.
Author: Peter Geoghegan <[email protected]>
Backpatch: 14-, where VACUUM VERBOSE's output changed.
M src/backend/access/heap/vacuumlazy.c
Don't print extra parens around expressions in extended stats
commit : 4d1816ec26e877297608a850736afed962527d70
author : Tomas Vondra <[email protected]>
date : Wed, 1 Sep 2021 00:42:32 +0200
committer: Tomas Vondra <[email protected]>
date : Wed, 1 Sep 2021 00:42:32 +0200
The code printing expressions for extended statistics doubled the
parens, producing results like ((a+1)), which is unnecessary and not
consistent with how we print expressions elsewhere.
Fixed by tweaking the code to produce just a single set of parens.
Reported by Mark Dilger, fix by me. Backpatch to 14, where support for
extended statistics on expressions was added.
Reported-by: Mark Dilger
Discussion: https://postgr.es/m/20210122040101.GF27167%40telsasoft.com
M src/backend/utils/adt/ruleutils.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/test/regress/expected/create_table_like.out
M src/test/regress/expected/stats_ext.out
Mark the timestamptz variant of date_bin() as stable
commit : 3eda9fc09fd6b9a1aec2d0113c633c69c3214b4d
author : John Naylor <[email protected]>
date : Tue, 31 Aug 2021 14:15:22 -0400
committer: John Naylor <[email protected]>
date : Tue, 31 Aug 2021 14:15:22 -0400
Previously, it was immutable by lack of marking. This is not
correct, since the time zone could change.
Bump catversion
Discussion: https://www.postgresql.org/message-id/CAFBsxsG2UHk8mOWL0tca%3D_cg%2B_oA5mVRNLhDF0TBw980iOg5NQ%40mail.gmail.com
Backpatch to v14, when this function came in
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
In pg_dump, avoid doing per-table queries for RLS policies.
commit : a20a9f26cefcf4e35ba7bb3d9e8672cb4ce1cf32
author : Tom Lane <[email protected]>
date : Tue, 31 Aug 2021 15:04:05 -0400
committer: Tom Lane <[email protected]>
date : Tue, 31 Aug 2021 15:04:05 -0400
For no particularly good reason, getPolicies() queried pg_policy
separately for each table. We can collect all the policies in
a single query instead, and attach them to the correct TableInfo
objects using findTableByOid() lookups. On the regression
database, this reduces the number of queries substantially, and
provides a visible savings even when running against a local
server.
Per complaint from Hubert Depesz Lubaczewski. Since this is such
a simple fix and can have a visible performance benefit, back-patch
to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_dump.c
Cache the results of format_type() queries in pg_dump.
commit : 9407dbbcb5b587cbefc4af14d1612b49abcb143b
author : Tom Lane <[email protected]>
date : Tue, 31 Aug 2021 13:53:33 -0400
committer: Tom Lane <[email protected]>
date : Tue, 31 Aug 2021 13:53:33 -0400
There's long been a "TODO: there might be some value in caching
the results" annotation on pg_dump's getFormattedTypeName function;
but we hadn't gotten around to checking what it was costing us to
repetitively look up type names. It turns out that when dumping the
current regression database, about 10% of the total number of queries
issued are duplicative format_type() queries. However, Hubert Depesz
Lubaczewski reported a not-unusual case where these account for over
half of the queries issued by pg_dump. Individually these queries
aren't expensive, but when network lag is a factor, they add up to a
problem. We can very easily add some caching to getFormattedTypeName
to solve it.
Since this is such a simple fix and can have a visible performance
benefit, back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
Rename the role in stats_ext to have regress_ prefix
commit : 4090ff2a99b76b7bd51534fb0f7013aa646d1e24
author : Tomas Vondra <[email protected]>
date : Tue, 31 Aug 2021 19:21:29 +0200
committer: Tomas Vondra <[email protected]>
date : Tue, 31 Aug 2021 19:21:29 +0200
Commit 5be8ce82e8 added a new role to the stats_ext regression suite,
but the role name did not start with regress_ causing failures when
running with ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS. Fixed by
renaming the role to start with the expected regress_ prefix.
Backpatch-through: 10, same as the new regression test
Discussion: https://postgr.es/m/1F238937-7CC2-4703-A1B1-6DC225B8978A%40enterprisedb.com
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Fix lookup error in extended stats ownership check
commit : a371a5ba349d9c39e7754f7d0f195ac46ee87b3f
author : Tomas Vondra <[email protected]>
date : Tue, 31 Aug 2021 18:03:05 +0200
committer: Tomas Vondra <[email protected]>
date : Tue, 31 Aug 2021 18:03:05 +0200
When an ownership check on extended statistics object failed, the code
was calling aclcheck_error_type to report the failure, which is clearly
wrong, resulting in cache lookup errors. Fix by calling aclcheck_error.
This issue exists since the introduction of extended statistics, so
backpatch all the way back to PostgreSQL 10. It went unnoticed because
there were no tests triggering the error, so add one.
Reported-by: Mark Dilger
Backpatch-through: 10, where extended stats were introduced
Discussion: https://postgr.es/m/1F238937-7CC2-4703-A1B1-6DC225B8978A%40enterprisedb.com
M src/backend/catalog/objectaddress.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Fix missed lock acquisition while inlining new-style SQL functions.
commit : 983d7033df6e03134572eb75dd76603fb9245211
author : Tom Lane <[email protected]>
date : Tue, 31 Aug 2021 12:02:36 -0400
committer: Tom Lane <[email protected]>
date : Tue, 31 Aug 2021 12:02:36 -0400
When starting to use a query parsetree loaded from the catalogs,
we must begin by applying AcquireRewriteLocks(), to obtain the same
relation locks that the parser would have gotten if the query were
entered interactively, and to do some other cleanup such as dealing
with later-dropped columns. New-style SQL functions are just as
subject to this rule as other stored parsetrees; however, of the
places dealing with such functions, only init_sql_fcache had gotten
the memo. In particular, if we successfully inlined a new-style
set-returning SQL function that contained any relation references,
we'd either get an assertion failure or attempt to use those
relation(s) sans locks.
I also added AcquireRewriteLocks calls to fmgr_sql_validator and
print_function_sqlbody. Desultory experiments didn't demonstrate any
failures in those, but I suspect that I just didn't try hard enough.
Certainly we don't expect nearby code paths to operate without locks.
On the same logic of it-ought-to-have-the-same-effects-as-the-old-code,
call pg_rewrite_query() in fmgr_sql_validator, too. It's possible
that neither code path there needs to bother with rewriting, but
doing the analysis to prove that is beyond my goals for today.
Per bug #17161 from Alexander Lakhin.
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/pg_proc.c
M src/backend/optimizer/util/clauses.c
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/create_function_3.out
M src/test/regress/sql/create_function_3.sql
Report tuple address in data-corruption error message
commit : eae08e21653c356f0e9223d26379527d653b71ed
author : Alvaro Herrera <[email protected]>
date : Mon, 30 Aug 2021 16:29:12 -0400
committer: Alvaro Herrera <[email protected]>
date : Mon, 30 Aug 2021 16:29:12 -0400
Most data-corruption reports mention the location of the problem, but
this one failed to. Add it.
Backpatch all the way back. In 12 and older, also assign the
ERRCODE_DATA_CORRUPTED error code as was done in commit fd6ec93bf890 for
13 and later.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam_handler.c
psql: Fix name quoting on extended statistics
commit : ce6b662aae7abc8b533b0cfa8fff50a9001508b1
author : Alvaro Herrera <[email protected]>
date : Mon, 30 Aug 2021 14:01:29 -0400
committer: Alvaro Herrera <[email protected]>
date : Mon, 30 Aug 2021 14:01:29 -0400
Per our message style guidelines, for human consumption we quote
qualified names as a whole rather than each part separately; but commits
bc085205c8a4 introduced a deviation for extended statistics and
a4d75c86bf15 copied it. I don't agree with this policy applying to
names shown by psql, but that's a poor reason to deviate from the
practice only in two obscure corners, so make said corners use the same
style as everywhere else.
Backpatch to 14. The first of these is older, but I'm not sure we want
to destabilize the psql output in the older branches for such a small
thing.
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/describe.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/expected/stats_ext.out
pgbench: Avoid unnecessary measurement of connection delays.
commit : efe2382d5ade2c8d63f2aa1b48cc85691a2bfabd
author : Fujii Masao <[email protected]>
date : Mon, 30 Aug 2021 21:35:24 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 30 Aug 2021 21:35:24 +0900
Commit 547f04e734 changed pgbench so that it used the measurement result
of connection delays in its benchmark report only when -C/--connect option
is specified. But previously those delays were unnecessarily measured
even when that option is not specified. Which was a waste of cycles.
This commit improves pgbench so that it avoids such unnecessary measurement.
Back-patch to v14 where commit 547f04e734 first appeared.
Author: Yugo Nagata
Reviewed-by: Fabien COELHO, Asif Rehman, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/bin/pgbench/pgbench.c
Add list of acknowledgments to release notes
commit : 7af5c38eb9da5b0ae72c1dd3d847f43cd39d1f5a
author : Peter Eisentraut <[email protected]>
date : Mon, 30 Aug 2021 08:56:16 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 30 Aug 2021 08:56:16 +0200
This contains all individuals mentioned in the commit messages during
PostgreSQL 14 development.
current through ed740b06b18e1a23becd54c97ff229aba4c94349
M doc/src/sgml/release-14.sgml
Fix incorrect error code in StartupReplicationOrigin().
commit : 0a143c33f01257a0670af71205343b505f5bd89b
author : Amit Kapila <[email protected]>
date : Mon, 30 Aug 2021 09:22:28 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 30 Aug 2021 09:22:28 +0530
ERRCODE_CONFIGURATION_LIMIT_EXCEEDED was used for checksum failure, use
ERRCODE_DATA_CORRUPTED instead.
Reported-by: Tatsuhito Kasahara
Author: Tatsuhito Kasahara
Backpatch-through: 9.6, where it was introduced
Discussion: https://postgr.es/m/CAP0=ZVLHtYffs8SOWcFJWrBGoRzT9QQbk+_aP+E5AHLNXiOorA@mail.gmail.com
M src/backend/replication/logical/origin.c
Keep stats up to date for partitioned tables
commit : e1efc5b465c844969a0ed0d07e1364f3ce424d8c
author : Alvaro Herrera <[email protected]>
date : Sat, 28 Aug 2021 15:58:23 -0400
committer: Alvaro Herrera <[email protected]>
date : Sat, 28 Aug 2021 15:58:23 -0400
In the long-going saga for analyze on partitioned tables, one thing I
missed while reverting 0827e8af70f4 is the maintenance of analyze count
and last analyze time for partitioned tables. This is a mostly trivial
change that enables users assess the need for invoking manual ANALYZE on
partitioned tables.
This patch, posted by Justin and modified a bit by me (Álvaro), can be
mostly traced back to Hosoya-san, though any problems introduced with
the scissors are mine.
Backpatch to 14, in line with 6f8127b73901.
Co-authored-by: Yuzuko Hosoya <[email protected]>
Co-authored-by: Justin Pryzby <[email protected]>
Co-authored-by: Álvaro Herrera <[email protected]>
Reported-by: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/analyze.c
M src/backend/postmaster/pgstat.c
psql \dX: reference regclass with "pg_catalog." prefix
commit : 359bcf775550aa577c86ea30a6d071487fcca1ed
author : Alvaro Herrera <[email protected]>
date : Sat, 28 Aug 2021 12:04:15 -0400
committer: Alvaro Herrera <[email protected]>
date : Sat, 28 Aug 2021 12:04:15 -0400
Déjà vu of commit fc40ba1296a7, for another backslash command.
Strictly speaking this isn't a bug, but since all references to catalog
objects are schema-qualified, we might as well be consistent. The
omission first appeared in commit ad600bba0422 and replicated in
a4d75c86bf15; backpatch to 14.
Author: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/describe.c
psql \dP: reference regclass with "pg_catalog." prefix
commit : a32860b88f1ffd6ba5b8bc803aecf6bc0f87cf02
author : Alvaro Herrera <[email protected]>
date : Sat, 28 Aug 2021 11:45:47 -0400
committer: Alvaro Herrera <[email protected]>
date : Sat, 28 Aug 2021 11:45:47 -0400
Strictly speaking this isn't a bug, but since all references to catalog
objects are schema-qualified, we might as well be consistent. The
omission first appeared in commit 1c5d9270e339, so backpatch to 12.
Author: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/describe.c
Fix data loss in wal_level=minimal crash recovery of CREATE TABLESPACE.
commit : 5513c09c89993cb0b2c1a4222b15722bcf71844b
author : Noah Misch <[email protected]>
date : Fri, 27 Aug 2021 23:33:23 -0700
committer: Noah Misch <[email protected]>
date : Fri, 27 Aug 2021 23:33:23 -0700
If the system crashed between CREATE TABLESPACE and the next checkpoint,
the result could be some files in the tablespace unexpectedly containing
no rows. Affected files would be those for which the system did not
write WAL; see the wal_skip_threshold documentation. Before v13, a
different set of conditions governed the writing of WAL; see v12's
<sect2 id="populate-pitr">. (The v12 conditions were broader in some
ways and narrower in others.) Users may want to audit non-default
tablespaces for unexpected short files. The bug could have truncated an
index without affecting the associated table, and reindexing the index
would fix that particular problem.
This fixes the bug by making create_tablespace_directories() more like
TablespaceCreateDbspace(). create_tablespace_directories() was
recursively removing tablespace contents, reasoning that WAL redo would
recreate everything removed that way. That assumption holds for other
wal_level values. Under wal_level=minimal, the old approach could
delete files for which no other copy existed. Back-patch to 9.6 (all
supported versions).
Reviewed by Robert Haas and Prabhat Sahu. Reported by Robert Haas.
Discussion: https://postgr.es/m/CA+TgmoaLO9ncuwvr2nN-J4VEP5XyAcy=zKiHxQzBbFRxxGxm0w@mail.gmail.com
M src/backend/commands/tablespace.c
M src/test/recovery/t/018_wal_optimize.pl
Count SP-GiST index scans in pg_stat statistics.
commit : e84d4810cd47d83a6a864fa33db74bcc5570dd76
author : Tom Lane <[email protected]>
date : Fri, 27 Aug 2021 19:42:42 -0400
committer: Tom Lane <[email protected]>
date : Fri, 27 Aug 2021 19:42:42 -0400
Somehow, spgist overlooked the need to call pgstat_count_index_scan().
Hence, pg_stat_all_indexes.idx_scan and equivalent columns never
became nonzero for an SP-GiST index, although the related per-tuple
counters worked fine.
This fix works a bit differently from other index AMs, in that the
counter increment occurs in spgrescan not spggettuple/spggetbitmap.
It looks like this won't make the user-visible semantics noticeably
different, so I won't go to the trouble of introducing an is-this-
the-first-call flag just to make the counter bumps happen in the
same places.
Per bug #17163 from Christian Quest. Back-patch to all supported
versions.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/spgist/spgscan.c
Use maintenance_io_concurrency for ANALYZE prefetch
commit : 9efa998a6403c5fe973ce5801d09ffa63e706eb6
author : Stephen Frost <[email protected]>
date : Fri, 27 Aug 2021 19:23:11 -0400
committer: Stephen Frost <[email protected]>
date : Fri, 27 Aug 2021 19:23:11 -0400
When prefetching pages for ANALYZE, we should be using
maintenance_io_concurrenty (by calling
get_tablespace_maintenance_io_concurrency(), not
get_tablespace_io_concurrency()).
ANALYZE prefetching was introduced in c6fc50c, so back-patch to 14.
Backpatch-through: 14
Reported-By: Egor Rogov
Discussion: https://postgr.es/m/9beada99-34ce-8c95-fadb-451768d08c64%40postgrespro.ru
M src/backend/commands/analyze.c
docs: Add command tags for SQL commands
commit : 8f6c110349769e2b6375cd01e632199a104dc4a1
author : Stephen Frost <[email protected]>
date : Fri, 27 Aug 2021 18:25:34 -0400
committer: Stephen Frost <[email protected]>
date : Fri, 27 Aug 2021 18:25:34 -0400
Commit 6c3ffd6 added a couple new predefined roles but didn't properly
wrap the SQL commands mentioned in the description of those roles with
command tags, so add them now.
Backpatch-through: 14
Reported-by: Michael Banck
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/user-manag.sgml
docs: clarify bgw_restart_time documentation
commit : 652804ebde82b6bb04b427becbcb4f6bea02ff90
author : Daniel Gustafsson <[email protected]>
date : Fri, 27 Aug 2021 22:50:19 +0200
committer: Daniel Gustafsson <[email protected]>
date : Fri, 27 Aug 2021 22:50:19 +0200
Author: Dave Cramer <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/CADK3HHLZmqAQZ2ByPDQQ9yhGqax36kksq6sDkV0yYzsxw6ipvQ@mail.gmail.com
M doc/src/sgml/bgworker.sgml
track_io_timing logging: Don't special case 0 ms.
commit : 6a1095234e0fcfa5c2f618c7c841e8ffe3c6c712
author : Peter Geoghegan <[email protected]>
date : Fri, 27 Aug 2021 13:33:58 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 27 Aug 2021 13:33:58 -0700
Adjust track_io_timing related logging code added by commit 94d13d474d.
Make it consistent with other nearby autovacuum and autoanalyze logging
code by removing logic that suppressed zero millisecond outputs.
log_autovacuum_min_duration log output now reliably shows "read:" and
"write:" millisecond-based values in its report (when track_io_timing is
enabled).
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Stephen Frost <[email protected]>
Discussion: https://postgr.es/m/CAH2-WznW0FNxSVQMSRazAMYNfZ6DR_gr5WE78hc6E1CBkkJpzw@mail.gmail.com
Backpatch: 14-, where the track_io_timing logging was introduced.
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/analyze.c
Reorder log_autovacuum_min_duration log output.
commit : fd134f374ef7a8cc0b2d878a92dfe2c7088e084b
author : Peter Geoghegan <[email protected]>
date : Fri, 27 Aug 2021 13:08:39 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 27 Aug 2021 13:08:39 -0700
This order seems more natural. It starts with details that are
particular to heap and index data structures, and ends with system-level
costs incurred during the autovacuum worker's VACUUM/ANALYZE operation.
Author: Peter Geoghegan <[email protected]>
Discussion: https://postgr.es/m/CAH2-WzkzxK6ahA9xxsOftRtBX_R0swuHZsvo4QUbak1Bz7hb7Q@mail.gmail.com
Backpatch: 14-, which enhanced the log output in various ways.
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/analyze.c
Handle interaction of regexp's makesearch and MATCHALL more honestly.
commit : 3068e45799327298a3f4c22b03db2aa48e2ab0da
author : Tom Lane <[email protected]>
date : Fri, 27 Aug 2021 12:18:58 -0400
committer: Tom Lane <[email protected]>
date : Fri, 27 Aug 2021 12:18:58 -0400
Second thoughts about commit 824bf7190: we apply makesearch() to
an NFA after having determined whether it is a MATCHALL pattern.
Prepending ".*" doesn't make it non-MATCHALL, but it does change the
maximum possible match length, and makesearch() failed to update that.
This has no ill effects given the stylized usage of search NFAs, but
it seems like it's better to keep the data structure consistent. In
particular, fixing this allows more honest handling of the MATCHALL
check in matchuntil(): we can now assert that maxmatchall is infinity,
instead of lamely assuming that it should act that way.
In passing, improve the code in dump[c]nfa so that infinite maxmatchall
is printed as "inf" not a magic number.
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/backend/regex/rege_dfa.c
contrib/amcheck: Add heapam CHECK_FOR_INTERRUPTS().
commit : ba05dfe943046820312eaa20995ffc626936b8be
author : Peter Geoghegan <[email protected]>
date : Thu, 26 Aug 2021 18:42:18 -0700
committer: Peter Geoghegan <[email protected]>
date : Thu, 26 Aug 2021 18:42:18 -0700
Add a CHECK_FOR_INTERRUPTS() call to make heap relation verification
responsive to query cancellations.
Author: Mark Dilger <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wzk-9RtQgb2QiuLv8j2O0j9tSFKPmmch5nWSZhguUxvbrw%40mail.gmail.com
Backpatch: 14-, where amcheck heap verification was introduced.
M contrib/amcheck/verify_heapam.c
Remove redundant test.
commit : ed740b06b18e1a23becd54c97ff229aba4c94349
author : Tom Lane <[email protected]>
date : Wed, 25 Aug 2021 11:06:34 -0400
committer: Tom Lane <[email protected]>
date : Wed, 25 Aug 2021 11:06:34 -0400
The condition "context_start < context_end" is strictly weaker
than "context_end - context_start >= 50", so we don't need both.
Oversight in commit ffd3944ab, noted by tanghy.fnst.
In passing, line-wrap a nearby test to make it more readable.
Discussion: https://postgr.es/m/OS0PR01MB61137C4054774F44E3A9DC89FBC69@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/backend/utils/adt/jsonfuncs.c
Fix broken snapshot handling in parallel workers.
commit : 11c1239881b399d1fe87186f1f8a10a505d4a692
author : Robert Haas <[email protected]>
date : Wed, 25 Aug 2021 08:32:04 -0400
committer: Robert Haas <[email protected]>
date : Wed, 25 Aug 2021 08:32:04 -0400
Pengchengliu reported an assertion failure in a parallel woker while
performing a parallel scan using an overflowed snapshot. The proximate
cause is that TransactionXmin was set to an incorrect value. The
underlying cause is incorrect snapshot handling in parallel.c.
In particular, InitializeParallelDSM() was unconditionally calling
GetTransactionSnapshot(), because I (rhaas) mistakenly thought that
was always retrieving an existing snapshot whereas, at isolation
levels less than REPEATABLE READ, it's actually taking a new one. So
instead do this only at higher isolation levels where there actually
is a single snapshot for the whole transaction.
By itself, this is not a sufficient fix, because we still need to
guarantee that TransactionXmin gets set properly in the workers. The
easiest way to do that seems to be to install the leader's active
snapshot as the transaction snapshot if the leader did not serialize a
transaction snapshot. This doesn't affect the results of future
GetTrasnactionSnapshot() calls since those have to take a new snapshot
anyway; what we care about is the side effect of setting TransactionXmin.
Report by Pengchengliu. Patch by Greg Nancarrow, except for some comment
text which I supplied.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/parallel.c
Fix typo
commit : a599109e591426689d5f79041af45525b00ef7b2
author : Peter Eisentraut <[email protected]>
date : Wed, 25 Aug 2021 10:14:51 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 25 Aug 2021 10:14:51 +0200
M src/include/utils/typcache.h
Fix incorrect merge in ECPG code with DECLARE
commit : 8ab3452df8d5ff26cf52c089b986256b0c02d555
author : Michael Paquier <[email protected]>
date : Wed, 25 Aug 2021 15:16:55 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 25 Aug 2021 15:16:55 +0900
The same condition was repeated twice when comparing the connection used
by existing declared statement with the one coming from a fresh DECLARE
statement. This had no consequences, but let's keep the code clean.
Oversight in f576de1.
Author: Shenhao Wang
Discussion: https://postgr.es/m/OSBPR01MB42149653BC0AB0A49D23C1B8F2C69@OSBPR01MB4214.jpnprd01.prod.outlook.com
Backpatch-through: 14
M src/interfaces/ecpg/preproc/ecpg.header
Fix toast rewrites in logical decoding.
commit : 9d7a80ce0185bd0c38bd973638bb7e9a854cf9f8
author : Amit Kapila <[email protected]>
date : Wed, 25 Aug 2021 10:10:50 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 25 Aug 2021 10:10:50 +0530
Commit 325f2ec555 introduced pg_class.relwrite to skip operations on
tables created as part of a heap rewrite during DDL. It links such
transient heaps to the original relation OID via this new field in
pg_class but forgot to do anything about toast tables. So, logical
decoding was not able to skip operations on internally created toast
tables. This leads to an error when we tried to decode the WAL for the
next operation for which it appeared that there is a toast data where
actually it didn't have any toast data.
To fix this, we set pg_class.relwrite for internally created toast tables
as well which allowed skipping operations on them during logical decoding.
Author: Bertrand Drouvot
Reviewed-by: David Zhang, Amit Kapila
Backpatch-through: 11, where it was introduced
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/expected/toast.out
M contrib/test_decoding/sql/toast.sql
M src/backend/catalog/toasting.c
M src/backend/commands/cluster.c
M src/backend/commands/tablecmds.c
M src/include/catalog/toasting.h
M src/include/commands/tablecmds.h
Doc: Tweak function prototype indentation for consistency.
commit : 22583edee7f4c6be6894b03c5cd93fd02e2a826a
author : Etsuro Fujita <[email protected]>
date : Wed, 25 Aug 2021 13:00:01 +0900
committer: Etsuro Fujita <[email protected]>
date : Wed, 25 Aug 2021 13:00:01 +0900
M doc/src/sgml/fdwhandler.sgml
ecpg: Remove trailing period from error message.
commit : d207df71f4af7de8e3e865e2635ae3a6b3a98d51
author : Fujii Masao <[email protected]>
date : Wed, 25 Aug 2021 09:56:04 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 25 Aug 2021 09:56:04 +0900
This commit improves the ecpg's error message that commit f576de1db1 updated,
so that it gets rid of trailing period and uppercases the command name
in the error message.
Back-patch to v14 where the error message exists.
Author: Kyotaro Horiguchi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/ecpg/preproc/ecpg.header
Avoid using ambiguous word "positive" in error message.
commit : ec619102aa56d3356fb6550a343af01ad477e4a2
author : Fujii Masao <[email protected]>
date : Wed, 25 Aug 2021 11:46:25 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 25 Aug 2021 11:46:25 +0900
There are two identical error messages about valid value of modulus for
hash partition, in PostgreSQL source code. Commit 0e1275fb07 improved
only one of them so that ambiguous word "positive" was avoided there,
and forgot to improve the other. This commit improves the other.
Which would reduce translator burden.
Back-pach to v11 where the error message exists.
Author: Kyotaro Horiguchi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/parse_utilcmd.c
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/create_table.out
Improve error message about valid value for distance in phrase operator.
commit : 1d0567ec61fbc429a7717ab8d5027952c806a8fb
author : Fujii Masao <[email protected]>
date : Wed, 25 Aug 2021 11:43:56 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 25 Aug 2021 11:43:56 +0900
The distance in phrase operator must be an integer value between zero
and MAXENTRYPOS inclusive. But previously the error message about
its valid value included the information about its upper limit
but not lower limit (i.e., zero). This commit improves the error message
so that it also includes the information about its lower limit.
Back-patch to v9.6 where full-text phrase search was supported.
Author: Kyotaro Horiguchi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/tsquery.c
Fix regexp misbehavior with capturing parens inside "{0}".
commit : 244dd79923a16afcf5f9a8dea53fc3af2ac0f7db
author : Tom Lane <[email protected]>
date : Tue, 24 Aug 2021 16:37:27 -0400
committer: Tom Lane <[email protected]>
date : Tue, 24 Aug 2021 16:37:27 -0400
Regexps like "(.){0}...\1" drew an "invalid backreference number".
That's not unreasonable on its face, since the capture group will
never be matched if it's iterated zero times. However, other engines
such as Perl's don't complain about this, nor do we throw an error for
related cases such as "(.)|\1", even though that backref can never
succeed either. Also, if the zero-iterations case happens at runtime
rather than compile time --- say, "(x)*...\1" when there's no "x" to
be found --- that's not an error, we just deem the backref to not
match. Making this even less defensible, no error was thrown for
nested cases such as "((.)){0}...\2"; and to add insult to injury,
those cases could result in assertion failures instead. (It seems
that nothing especially bad happened in non-assert builds, though.)
Let's just fix it so that no error is thrown and instead the backref
is deemed to never match, so that compile-time detection of no
iterations behaves the same as run-time detection.
Per report from Mark Dilger. This appears to be an aboriginal error
in Spencer's library, so back-patch to all supported versions.
Pre-v14, it turns out to also be necessary to back-patch one aspect of
commits cb76fbd7e/00116dee5, namely to create capture-node subREs with
the begin/end states of their subexpressions, not the current lp/rp
of the outer parseqatom invocation. Otherwise delsub complains that
we're trying to disconnect a state from itself. This is a bit scary
but code examination shows that it's safe: in the pre-v14 code, if we
want to wrap iteration around the subexpression, the first thing we do
is overwrite the atom's begin/end fields with new states. So the
bogus values didn't survive long enough to be used for anything, except
if no iteration is required, in which case it doesn't matter.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regcomp.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
M src/test/regress/expected/regex.out
M src/test/regress/sql/regex.sql
Fix Alter Subscription's Add/Drop Publication behavior.
commit : 5cfcd46e9d297eac627bf3183272713112ac5f60
author : Amit Kapila <[email protected]>
date : Tue, 24 Aug 2021 08:38:11 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 24 Aug 2021 08:38:11 +0530
The current refresh behavior tries to just refresh added/dropped
publications but that leads to removing wrong tables from subscription. We
can't refresh just the dropped publication because it is quite possible
that some of the tables are removed from publication by that time and now
those will remain as part of the subscription. Also, there is a chance
that the tables that were part of the publication being dropped are also
part of another publication, so we can't remove those.
So, we decided that by default, add/drop commands will also act like
REFRESH PUBLICATION which means they will refresh all the publications. We
can keep the old behavior for "add publication" but it is better to be
consistent with "drop publication".
Author: Hou Zhijie
Reviewed-by: Masahiko Sawada, Amit Kapila
Backpatch-through: 14, where it was introduced
Discussion: https://postgr.es/m/OS0PR01MB5716935D4C2CC85A6143073F94EF9@OS0PR01MB5716.jpnprd01.prod.outlook.com
M doc/src/sgml/ref/alter_subscription.sgml
M src/backend/commands/subscriptioncmds.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
A src/test/subscription/t/021_alter_sub_pub.pl
Prevent regexp back-refs from sometimes matching when they shouldn't.
commit : 779557bd22895420b48eba409d2286f1dea08c06
author : Tom Lane <[email protected]>
date : Mon, 23 Aug 2021 17:41:07 -0400
committer: Tom Lane <[email protected]>
date : Mon, 23 Aug 2021 17:41:07 -0400
The recursion in cdissect() was careless about clearing match data
for capturing parentheses after rejecting a partial match. This
could allow a later back-reference to succeed when by rights it
should fail for lack of a defined referent.
To fix, think a little more rigorously about what the contract
between different levels of cdissect's recursion needs to be.
With the right spec, we can fix this using fewer rather than more
resets of the match data; the key decision being that a failed
sub-match is now explicitly responsible for clearing any matches
it may have set.
There are enough other cross-checks and optimizations in the code
that it's not especially easy to exhibit this problem; usually, the
match will fail as-expected. Plus, regexps that are even potentially
vulnerable are most likely user errors, since there's just not much
point in writing a back-ref that doesn't always have a referent.
These facts perhaps explain why the issue hasn't been detected,
even though it's almost certainly a couple of decades old.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regexec.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
M src/test/regress/expected/regex.out
M src/test/regress/sql/regex.sql
Avoid creating archive status ".ready" files too early
commit : e3fb6170e58c4325cd1e1eb22f96ef43c3b4152a
author : Alvaro Herrera <[email protected]>
date : Mon, 23 Aug 2021 15:50:35 -0400
committer: Alvaro Herrera <[email protected]>
date : Mon, 23 Aug 2021 15:50:35 -0400
WAL records may span multiple segments, but XLogWrite() does not
wait for the entire record to be written out to disk before
creating archive status files. Instead, as soon as the last WAL page of
the segment is written, the archive status file is created, and the
archiver may process it. If PostgreSQL crashes before it is able to
write and flush the rest of the record (in the next WAL segment), the
wrong version of the first segment file lingers in the archive, which
causes operations such as point-in-time restores to fail.
To fix this, keep track of records that span across segments and ensure
that segments are only marked ready-for-archival once such records have
been completely written to disk.
This has always been wrong, so backpatch all the way back.
Author: Nathan Bossart <[email protected]>
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Reviewed-by: Ryo Matsumura <[email protected]>
Reviewed-by: Andrey Borodin <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
M src/backend/postmaster/walwriter.c
M src/include/access/xlog.h
M src/include/access/xlogdefs.h
Fix backup manifests to generate correct WAL-Ranges across timelines
commit : 65b649fecb0c0bd2c2c1f075f94f45a74cdc41be
author : Michael Paquier <[email protected]>
date : Mon, 23 Aug 2021 11:09:54 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 23 Aug 2021 11:09:54 +0900
In a backup manifest, WAL-Ranges stores the range of WAL that is
required for the backup to be valid. pg_verifybackup would then
internally use pg_waldump for the checks based on this data.
When the timeline where the backup started was more than 1 with a
history file looked at for the manifest data generation, the calculation
of the WAL range for the first timeline to check was incorrect. The
previous logic used as start LSN the start position of the first
timeline, but it needs to use the start LSN of the backup. This would
cause failures with pg_verifybackup, or any tools making use of the
backup manifests.
This commit adds a test based on a logic using a self-promoted node,
making it rather cheap.
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/backend/replication/backup_manifest.c
M src/bin/pg_verifybackup/t/007_wal.pl
Improve error messages about misuse of SELECT INTO.
commit : 8f6a52196aceb11a78eb1d673a03ab3ab72da231
author : Tom Lane <[email protected]>
date : Sat, 21 Aug 2021 10:22:14 -0400
committer: Tom Lane <[email protected]>
date : Sat, 21 Aug 2021 10:22:14 -0400
Improve two places in plpgsql, and one in spi.c, where an error
message would confusingly tell you that you couldn't use a SELECT
query, when what you had written *was* a SELECT query. The actual
problem is that you can't use SELECT ... INTO in these contexts,
but the messages failed to make that apparent. Special-case
SELECT INTO to make these errors more helpful.
Also, fix the same spots in plpgsql, as well as several messages
in exec_eval_expr(), to not quote the entire complained-of query or
expression in the primary error message. That behavior very easily
led to violating our message style guideline about keeping the primary
error message short and single-line. Also, since the important part
of the message was after the inserted text, it could make the real
problem very hard to see. We can report the query or expression as
the first line of errcontext instead.
Per complaint from Roger Mason. Back-patch to v14, since (a) some
of these messages are new in v14 and (b) v14's translatable strings
are still somewhat in flux. The problem's older than that of
course, but I'm hesitant to change the behavior further back.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/spi.c
M src/pl/plpgsql/src/expected/plpgsql_array.out
M src/pl/plpgsql/src/pl_exec.c
Fix performance bug in regexp's citerdissect/creviterdissect.
commit : 57a2d4a1b3bb48ec422a2701f8df026123a98a27
author : Tom Lane <[email protected]>
date : Fri, 20 Aug 2021 14:19:04 -0400
committer: Tom Lane <[email protected]>
date : Fri, 20 Aug 2021 14:19:04 -0400
After detecting a sub-match "dissect" failure (i.e., a backref match
failure) in the i'th sub-match of an iteration node, we should proceed
by adjusting the attempted length of the i'th submatch. As coded,
though, these functions changed the attempted length of the *last*
sub-match, and only after exhausting all possibilities for that would
they back up to adjust the next-to-last sub-match, and then the
second-from-last, etc; all of which is wasted effort, since only
changing the start or length of the i'th sub-match can possibly make
it succeed. This oversight creates the possibility for exponentially
bad performance. Fortunately the problem is masked in most cases by
optimizations or constraints applied elsewhere; which explains why
we'd not noticed it before. But it is possible to reach the problem
with fairly simple, if contrived, regexps.
Oversight in my commit 173e29aa5. That's pretty ancient now,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regexec.c
Remove --quiet option from pg_amcheck
commit : 92ce7f527960ca672d2ad70e61442f4e5b3bb641
author : Daniel Gustafsson <[email protected]>
date : Fri, 20 Aug 2021 12:44:54 +0200
committer: Daniel Gustafsson <[email protected]>
date : Fri, 20 Aug 2021 12:44:54 +0200
Using --quiet in combination with --no-strict-names didn't work as
documented, a warning message was still emitted. Since the --quiet
flag was working in an unconventional way to other utilities, fix
by removing the functionality instead.
Backpatch through 14 where pg_amcheck was introduced.
Bug: 17148
Reported-by: Chen Jiaoqian <[email protected]>
Reviewed-by: Julien Rouhaud <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 14
M doc/src/sgml/ref/pg_amcheck.sgml
M src/bin/pg_amcheck/pg_amcheck.c
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/005_opclass_damage.pl
pg_amcheck: Fix block number parsing on command line
commit : 88cfcbb79f8064705362d8cfc0dff23d3c16195f
author : Peter Eisentraut <[email protected]>
date : Fri, 20 Aug 2021 07:48:22 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 20 Aug 2021 07:48:22 +0200
The previous code wouldn't handle higher block numbers on systems
where sizeof(long) == 4.
Reviewed-by: Mark Dilger <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/6a10a211-872b-3c4c-106b-909ae5fefa61%40enterprisedb.com
M src/bin/pg_amcheck/pg_amcheck.c
Avoid trying to lock OLD/NEW in a rule with FOR UPDATE.
commit : 4649003933522282f03643c5a50fd2eb2a31a783
author : Tom Lane <[email protected]>
date : Thu, 19 Aug 2021 12:12:35 -0400
committer: Tom Lane <[email protected]>
date : Thu, 19 Aug 2021 12:12:35 -0400
transformLockingClause neglected to exclude the pseudo-RTEs for
OLD/NEW when processing a rule's query. This led to odd errors
or even crashes later on. This bug is very ancient, but it's
not terribly surprising that nobody noticed, since the use-case
for SELECT FOR UPDATE in a non-view rule is somewhere between
thin and non-existent. Still, crashing is not OK.
Per bug #17151 from Zhiyong Wu. Thanks to Masahiko Sawada
for analysis of the problem.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/analyze.c
M src/include/nodes/parsenodes.h
M src/test/regress/expected/rules.out
M src/test/regress/sql/rules.sql
Unset MyBEEntry, making elog.c's call to pgstat_get_my_query_id() safe.
commit : 18914f24ec6e704e81a5528cf09f3d54b23ef12b
author : Andres Freund <[email protected]>
date : Thu, 19 Aug 2021 04:59:06 -0700
committer: Andres Freund <[email protected]>
date : Thu, 19 Aug 2021 04:59:06 -0700
Previously log messages late during shutdown could end up using either another
backend's PgBackendStatus (multi user) or segfault (single user) because
pgstat_get_my_query_id()'s check for !MyBEEntry didn't filter out use after
pgstat_beshutdown_hook().
This became a bug in 4f0b0966c86, but was a bit fishy before. But given
there's no known problematic cases before 14, it doesn't seem worth
backpatching further.
Also fixes a wrong filename in a comment, introduced in e1025044.
Reported-By: Andres Freund <[email protected]>
Reviewed-By: Julien Rouhaud <[email protected]>
Discussion: https://postgr.es/m/Julien Rouhaud <[email protected]>
Backpatch: 14-
M src/backend/utils/activity/backend_status.c
Fix typo in protocol.sgml.
commit : e1915646658def5dd87331ac77fb9d8d0abd763b
author : Amit Kapila <[email protected]>
date : Thu, 19 Aug 2021 09:05:54 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 19 Aug 2021 09:05:54 +0530
The 'Stream Stop' message is misspelled as 'Stream End' in the docs.
Author: Masahiko Sawada
Backpatch-through: 14, where it was introduced
Discussion: https://postgr.es/m/CAD21AoDeScrsHhLyEPYqN3sydg6PxAPVBboK=30xJfUVihNZDA@mail.gmail.com
M doc/src/sgml/protocol.sgml
Revert refactoring of hex code to src/common/
commit : 1900c140554efdcaa94134705e9be7ce1437be9c
author : Michael Paquier <[email protected]>
date : Thu, 19 Aug 2021 09:20:19 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 19 Aug 2021 09:20:19 +0900
This is a combined revert of the following commits:
- c3826f8, a refactoring piece that moved the hex decoding code to
src/common/. This code was cleaned up by aef8948, as it originally
included no overflow checks in the same way as the base64 routines in
src/common/ used by SCRAM, making it unsafe for its purpose.
- aef8948, a more advanced refactoring of the hex encoding/decoding code
to src/common/ that added sanity checks on the result buffer for hex
decoding and encoding. As reported by Hans Buschmann, those overflow
checks are expensive, and it is possible to see a performance drop in
the decoding/encoding of bytea or LOs the longer they are. Simple SQLs
working on large bytea values show a clear difference in perf profile.
- ccf4e27, a cleanup made possible by aef8948.
The reverts of all those commits bring back the performance of hex
decoding and encoding back to what it was in ~13. Fow now and
post-beta3, this is the simplest option.
Reported-by: Hans Buschmann
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 14
M src/backend/replication/backup_manifest.c
M src/backend/utils/adt/encode.c
M src/backend/utils/adt/varlena.c
M src/common/Makefile
D src/common/hex.c
D src/include/common/hex.h
M src/include/common/sha2.h
M src/include/utils/builtins.h
M src/tools/msvc/Mkvcbuild.pm
Fix check_agg_arguments' examination of aggregate FILTER clauses.
commit : 61f9dae2ce7680d51035e5bb8668cf5227346eac
author : Tom Lane <[email protected]>
date : Wed, 18 Aug 2021 18:12:51 -0400
committer: Tom Lane <[email protected]>
date : Wed, 18 Aug 2021 18:12:51 -0400
Recursion into the FILTER clause was mis-implemented, such that a
relevant Var or Aggref at the very top of the FILTER clause would
be ignored. (Of course, that'd have to be a plain boolean Var or
boolean-returning aggregate.) The consequence would be
mis-identification of the correct semantic level of the aggregate,
which could lead to not-per-spec query behavior. If the FILTER
expression is an aggregate, this could also lead to failure to issue
an expected "aggregate function calls cannot be nested" error, which
would likely result in a core dump later on, since the planner and
executor aren't expecting such cases to appear.
The root cause is that commit b560ec1b0 blindly copied some code
that assumed it's recursing into a List, and thus didn't examine the
top-level node. To forestall questions about why this call doesn't
look like the others, as well as possible future copy-and-paste
mistakes, let's change all three check_agg_arguments_walker calls in
check_agg_arguments, even though only the one for the filter clause
is really broken.
Per bug #17152 from Zhiyong Wu. This has been wrong since we
implemented FILTER, so back-patch to all supported versions.
(Testing suggests that pre-v11 branches manage to avoid crashing
in the bad-Aggref case, thanks to "redundant" checks in ExecInitAgg.
But I'm not sure how thorough that protection is, and anyway the
wrong-behavior issue remains, so fix 9.6 and 10 too.)
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/parse_agg.c
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql
Fix pg_amcheck --skip option parameter handling
commit : 5310c61ecc14f23d28429f055c968a97d5e8b39c
author : Daniel Gustafsson <[email protected]>
date : Wed, 18 Aug 2021 11:23:43 +0200
committer: Daniel Gustafsson <[email protected]>
date : Wed, 18 Aug 2021 11:23:43 +0200
The skip options set for all-visible and all-frozen were incorrect
as they used space rather than hyphen, causing a syntax error when
invoked. Also, the option for not skipping any pages at all, none,
was documented but not implemented.
Backpatch through 14 where pg_amcheck was introduced.
Bug: #17149
Reported-by: Chen Jiaoqian <[email protected]>
Reviewed-by: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 14
M src/bin/pg_amcheck/pg_amcheck.c
Prevent ALTER TYPE/DOMAIN/OPERATOR from changing extension membership.
commit : 8f51ee63df3a9022cfd07d7482b8f3f21ff8f46d
author : Tom Lane <[email protected]>
date : Tue, 17 Aug 2021 14:29:22 -0400
committer: Tom Lane <[email protected]>
date : Tue, 17 Aug 2021 14:29:22 -0400
If recordDependencyOnCurrentExtension is invoked on a pre-existing,
free-standing object during an extension update script, that object
will become owned by the extension. In our current code this is
possible in three cases:
* Replacing a "shell" type or operator.
* CREATE OR REPLACE overwriting an existing object.
* ALTER TYPE SET, ALTER DOMAIN SET, and ALTER OPERATOR SET.
The first of these cases is intentional behavior, as noted by the
existing comments for GenerateTypeDependencies. It seems like
appropriate behavior for CREATE OR REPLACE too; at least, the obvious
alternatives are not better. However, the fact that it happens during
ALTER is an artifact of trying to share code (GenerateTypeDependencies
and makeOperatorDependencies) between the CREATE and ALTER cases.
Since an extension script would be unlikely to ALTER an object that
didn't already belong to the extension, this behavior is not very
troubling for the direct target object ... but ALTER TYPE SET will
recurse to dependent domains, and it is very uncool for those to
become owned by the extension if they were not already.
Let's fix this by redefining the ALTER cases to never change extension
membership, full stop. We could minimize the behavioral change by
only changing the behavior when ALTER TYPE SET is recursing to a
domain, but that would complicate the code and it does not seem like
a better definition.
Per bug #17144 from Alex Kozhemyakin. Back-patch to v13 where ALTER
TYPE SET was added. (The other cases are older, but since they only
affect the directly-named object, there's not enough of a problem to
justify changing the behavior further back.)
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/pg_depend.c
M src/backend/catalog/pg_operator.c
M src/backend/catalog/pg_type.c
M src/backend/commands/operatorcmds.c
M src/backend/commands/typecmds.c
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_type.h
Improved ECPG warning as suggested by Michael Paquier and removed test case that triggers the warning during regression tests.
commit : e2d6da0708df2c2e244d8748c26ebc49c0963ed5
author : Michael Meskes <[email protected]>
date : Tue, 17 Aug 2021 14:58:33 +0200
committer: Michael Meskes <[email protected]>
date : Tue, 17 Aug 2021 14:58:33 +0200
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/test/expected/sql-declare.c
M src/interfaces/ecpg/test/expected/sql-declare.stderr
M src/interfaces/ecpg/test/expected/sql-declare.stdout
M src/interfaces/ecpg/test/sql/declare.pgc
Set type identifier on BIO
commit : b88377ad65dfa714e1bd1c041421446065c07c9c
author : Daniel Gustafsson <[email protected]>
date : Tue, 17 Aug 2021 14:27:37 +0200
committer: Daniel Gustafsson <[email protected]>
date : Tue, 17 Aug 2021 14:27:37 +0200
In OpenSSL there are two types of BIO's (I/O abstractions):
source/sink and filters. A source/sink BIO is a source and/or
sink of data, ie one acting on a socket or a file. A filter
BIO takes a stream of input from another BIO and transforms it.
In order for BIO_find_type() to be able to traverse the chain
of BIO's and correctly find all BIO's of a certain type they
shall have the type bit set accordingly, source/sink BIO's
(what PostgreSQL implements) use BIO_TYPE_SOURCE_SINK and
filter BIO's use BIO_TYPE_FILTER. In addition to these, file
descriptor based BIO's should have the descriptor bit set,
BIO_TYPE_DESCRIPTOR.
The PostgreSQL implementation didn't set the type bits, which
went unnoticed for a long time as it's only really relevant
for code auditing the OpenSSL installation, or doing similar
tasks. It is required by the API though, so this fixes it.
Backpatch through 9.6 as this has been wrong for a long time.
Author: Itamar Gafni
Discussion: https://postgr.es/m/SN6PR06MB39665EC10C34BB20956AE4578AF39@SN6PR06MB3966.namprd06.prod.outlook.com
Backpatch-through: 9.6
M src/backend/libpq/be-secure-openssl.c
M src/interfaces/libpq/fe-secure-openssl.c
doc: \123 and \x12 escapes in COPY are in database encoding.
commit : f90c05959ec3fb712cba889817e0a0689fa6fa89
author : Heikki Linnakangas <[email protected]>
date : Tue, 17 Aug 2021 10:00:06 +0300
committer: Heikki Linnakangas <[email protected]>
date : Tue, 17 Aug 2021 10:00:06 +0300
The backslash sequences, including \123 and \x12 escapes, are interpreted
after encoding conversion. The docs failed to mention that.
Backpatch to all supported versions.
Reported-by: Andreas Grob
Discussion: https://www.postgresql.org/message-id/17142-9181542ca1df75ab%40postgresql.org
M doc/src/sgml/ref/copy.sgml
Revert analyze support for partitioned tables
commit : b3d24cc0f0aa882ceec0a74a99f94166c6fc3247
author : Alvaro Herrera <[email protected]>
date : Mon, 16 Aug 2021 17:27:52 -0400
committer: Alvaro Herrera <[email protected]>
date : Mon, 16 Aug 2021 17:27:52 -0400
This reverts the following commits:
1b5617eb844cd2470a334c1d2eec66cf9b39c41a Describe (auto-)analyze behavior for partitioned tables
0e69f705cc1a3df273b38c9883fb5765991e04fe Set pg_class.reltuples for partitioned tables
41badeaba8beee7648ebe7923a41c04f1f3cb302 Document ANALYZE storage parameters for partitioned tables
0827e8af70f4653ba17ed773f123a60eadd9f9c9 autovacuum: handle analyze for partitioned tables
There are efficiency issues in this code when handling databases with
large numbers of partitions, and it doesn't look like there isn't any
trivial way to handle those. There are some other issues as well. It's
now too late in the cycle for nontrivial fixes, so we'll have to let
Postgres 14 users continue to manually deal with ANALYZE their
partitioned tables, and hopefully we can fix the issues for Postgres 15.
I kept [most of] be280cdad298 ("Don't reset relhasindex for partitioned
tables on ANALYZE") because while we added it due to 0827e8af70f4, it is
a good bugfix in its own right, since it affects manual analyze as well
as autovacuum-induced analyze, and there's no reason to revert it.
I retained the addition of relkind 'p' to tables included by
pg_stat_user_tables, because reverting that would require a catversion
bump.
Also, in pg14 only, I keep a struct member that was added to
PgStat_TabStatEntry to avoid breaking compatibility with existing stat
files.
Backpatch to 14.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/ref/analyze.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/pg_restore.sgml
M src/backend/access/common/reloptions.c
M src/backend/commands/analyze.c
M src/backend/commands/tablecmds.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/pgstat.c
M src/include/pgstat.h
Refresh apply delay on reload of recovery_min_apply_delay at recovery
commit : f83d80ea1e57927471848d901843ddc6fd5ecacc
author : Michael Paquier <[email protected]>
date : Mon, 16 Aug 2021 12:11:49 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 16 Aug 2021 12:11:49 +0900
This commit ensures that the wait interval in the replay delay loop
waiting for an amount of time defined by recovery_min_apply_delay is
correctly handled on reload, recalculating the delay if this GUC value
is updated, based on the timestamp of the commit record being replayed.
The previous behavior would be problematic for example with replay
still waiting even if the delay got reduced or just cancelled. If the
apply delay was increased to a larger value, the wait would have just
respected the old value set, finishing earlier.
Author: Soumyadeep Chakraborty, Ashwin Agrawal
Reviewed-by: Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/CAE-ML+93zfr-HLN8OuxF0BjpWJ17O5dv1eMvSE5jsj9jpnAXZA@mail.gmail.com
Backpatch-through: 9.6
M src/backend/access/transam/xlog.c
Add RISC-V spinlock support in s_lock.h.
commit : 4ffbd55d9346b6a8ba391c6df9a0f692c4b61ace
author : Tom Lane <[email protected]>
date : Fri, 13 Aug 2021 13:58:47 -0400
committer: Tom Lane <[email protected]>
date : Fri, 13 Aug 2021 13:58:47 -0400
Like the ARM case, just use gcc's __sync_lock_test_and_set();
that will compile into AMOSWAP.W.AQ which does what we need.
At some point it might be worth doing some work on atomic ops
for RISC-V, but this should be enough for a creditable port.
Back-patch to all supported branches, just in case somebody
wants to try them on RISC-V.
Marek Szuba
Discussion: https://postgr.es/m/[email protected]
M src/include/storage/s_lock.h
pg_amcheck: Message style and structuring improvements
commit : e546d1d052f220b077cf4d9fee8e6d5bf85911c5
author : Peter Eisentraut <[email protected]>
date : Fri, 13 Aug 2021 17:15:03 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 13 Aug 2021 17:15:03 +0200
M src/bin/pg_amcheck/pg_amcheck.c
M src/bin/pg_amcheck/t/004_verify_heapam.pl
Fix connection handling for DEALLOCATE and DESCRIBE statements
commit : a945f5527322d18d02990a9772a893f741e7d8df
author : Michael Meskes <[email protected]>
date : Fri, 13 Aug 2021 10:34:04 +0200
committer: Michael Meskes <[email protected]>
date : Fri, 13 Aug 2021 10:34:04 +0200
After binding a statement to a connection with DECLARE STATEMENT the connection
was still not used for DEALLOCATE and DESCRIBE statements. This patch fixes
that, adds a missing warning and cleans up the code.
Author: Hayato Kuroda
Reviewed-by: Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/TYAPR01MB5866BA57688DF2770E2F95C6F5069%40TYAPR01MB5866.jpnprd01.prod.outlook.com
M src/interfaces/ecpg/preproc/descriptor.c
M src/interfaces/ecpg/preproc/ecpg.addons
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/ecpg.type
M src/interfaces/ecpg/preproc/output.c
M src/interfaces/ecpg/preproc/type.h
M src/interfaces/ecpg/test/expected/sql-declare.c
M src/interfaces/ecpg/test/expected/sql-declare.stderr
M src/interfaces/ecpg/test/expected/sql-declare.stdout
M src/interfaces/ecpg/test/sql/declare.pgc
Fix sslsni connparam boolean check
commit : ffff00a3556734f859f375b8c76c89f1d2920bcd
author : Daniel Gustafsson <[email protected]>
date : Fri, 13 Aug 2021 10:32:16 +0200
committer: Daniel Gustafsson <[email protected]>
date : Fri, 13 Aug 2021 10:32:16 +0200
The check for sslsni only checked for existence of the parameter
but not for the actual value of the param. This meant that the
SNI extension was always turned on. Fix by inspecting the value
of sslsni and only activate the SNI extension iff sslsni has been
enabled. Also update the docs to be more in line with how other
boolean params are documented.
Backpatch to 14 where sslsni was first implemented.
Reviewed-by: Tom Lane
Backpatch-through: 14, where sslni was added
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-secure-openssl.c
Fix incorrect hash table resizing code in simplehash.h
commit : dc23c77d07af086574124ea5ca65acf9360b8691
author : David Rowley <[email protected]>
date : Fri, 13 Aug 2021 16:41:56 +1200
committer: David Rowley <[email protected]>
date : Fri, 13 Aug 2021 16:41:56 +1200
This fixes a bug in simplehash.h which caused an incorrect size mask to be
used when the hash table grew to SH_MAX_SIZE (2^32). The code was
incorrectly setting the size mask to 0 when the hash tables reached the
maximum possible number of buckets. This would result always trying to
use the 0th bucket causing an infinite loop of trying to grow the hash
table due to there being too many collisions.
Seemingly it's not that common for simplehash tables to ever grow this big
as this bug dates back to v10 and nobody seems to have noticed it before.
However, probably the most likely place that people would notice it would
be doing a large in-memory Hash Aggregate with something close to at least
2^31 groups.
After this fix, the code now works correctly with up to within 98% of 2^32
groups and will fail with the following error when trying to insert any
more items into the hash table:
ERROR: hash table size exceeded
However, the work_mem (or hash_mem_multiplier in newer versions) settings
will generally cause Hash Aggregates to spill to disk long before reaching
that many groups. The minimal test case I did took a work_mem setting of
over 192GB to hit the bug.
simplehash hash tables are used in a few other places such as Bitmap Index
Scans, however, again the size that the hash table can become there is
also limited to work_mem and it would take a relation of around 16TB
(2^31) pages and a very large work_mem setting to hit this. With smaller
work_mem values the table would become lossy and never grow large enough
to hit the problem.
Author: Yura Sokolov
Reviewed-by: David Rowley, Ranier Vilela
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 10, where simplehash.h was added
M src/include/lib/simplehash.h
Make EXEC_BACKEND more convenient on macOS.
commit : 8201b60565b8da465e75da4e4c6d2f01bf976a43
author : Thomas Munro <[email protected]>
date : Fri, 13 Aug 2021 10:38:22 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 13 Aug 2021 10:38:22 +1200
It's hard to disable ASLR on current macOS releases, for testing with
-DEXEC_BACKEND. You could already set the environment variable
PG_SHMEM_ADDR to something not likely to collide with mappings created
earlier in process startup. Let's also provide a default value that
works on current releases and architectures, for developer convenience.
As noted in the pre-existing comment, this is a horrible hack, but
-DEXEC_BACKEND is only used by Unix-based PostgreSQL developers for
testing some otherwise Windows-only code paths, so it seems excusable.
Back-patch to all supported branches.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/20210806032944.m4tz7j2w47mant26%40alap3.anarazel.de
M src/backend/port/sysv_shmem.c
Use appropriate tuple descriptor in FDW batching
commit : 886531f3f403cae67a4537f0a2a1edc730f6b793
author : Tomas Vondra <[email protected]>
date : Thu, 12 Aug 2021 21:32:53 +0200
committer: Tomas Vondra <[email protected]>
date : Thu, 12 Aug 2021 21:32:53 +0200
The FDW batching code was using the same tuple descriptor both for all
slots (regular and plan slots), but that's incorrect - the subplan may
use a different descriptor. Currently this is benign, because batching
is used only for INSERTs, and in that case the descriptors always match.
But that would change if we allow batching UPDATEs.
Fix by copying the appropriate tuple descriptor. Backpatch to 14, where
the FDW batching was implemented.
Author: Amit Langote
Backpatch-through: 14, where FDW batching was added
Discussion: https://postgr.es/m/CA%2BHiwqEWd5B0-e-RvixGGUrNvGkjH2s4m95%3DJcwUnyV%3Df0rAKQ%40mail.gmail.com
M src/backend/executor/nodeModifyTable.c
Fix segfault during EvalPlanQual with mix of local and foreign partitions.
commit : 6458ed18fe40ac91f29496128dd1a49c0ef2db5b
author : Heikki Linnakangas <[email protected]>
date : Thu, 12 Aug 2021 11:02:29 +0300
committer: Heikki Linnakangas <[email protected]>
date : Thu, 12 Aug 2021 11:02:29 +0300
It's not sensible to re-evaluate a direct-modify Foreign Update or Delete
during EvalPlanQual. However, ExecInitForeignScan() can still get called
if a table mixes local and foreign partitions. EvalPlanQualStart() left
the es_result_relations array uninitialized in the child EPQ EState, but
ExecInitForeignScan() still expected to find it. That caused a segfault.
Fix by skipping the es_result_relations lookup during EvalPlanQual
processing. To make things a bit more robust, also skip the
BeginDirectModify calls, and add a runtime check that ExecForeignScan()
is not called on direct-modify foreign scans during EvalPlanQual
processing.
This is new in v14, commit 1375422c782. Before that, EvalPlanQualStart()
copied the whole ResultRelInfo array to the EPQ EState. Backpatch to v14.
Report and diagnosis by Andrey Lepikhov.
Discussion: https://www.postgresql.org/message-id/cb2b808d-cbaa-4772-76ee-c8809bafcf3d%40postgrespro.ru
M src/backend/executor/nodeForeignscan.c
Fix failure of btree_gin indexscans with "char" type and </<= operators.
commit : a4957b5a72dc9e169b78604f67357bb3dc49d826
author : Tom Lane <[email protected]>
date : Tue, 10 Aug 2021 18:10:30 -0400
committer: Tom Lane <[email protected]>
date : Tue, 10 Aug 2021 18:10:30 -0400
As a result of confusion about whether the "char" type is signed or
unsigned, scans for index searches like "col < 'x'" or "col <= 'x'"
would start at the middle of the index not the left end, thus missing
many or all of the entries they should find. Fortunately, this
is not a symptom of index corruption. It's only the search logic
that is broken, and we can fix it without unpleasant side-effects.
Per report from Jason Kim. This has been wrong since btree_gin's
beginning, so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M contrib/btree_gin/btree_gin.c
M contrib/btree_gin/expected/char.out
Stamp 14beta3.
commit : 26f5391df63445692c88e4540dcbb558e567044c
author : Tom Lane <[email protected]>
date : Mon, 9 Aug 2021 16:47:06 -0400
committer: Tom Lane <[email protected]>
date : Mon, 9 Aug 2021 16:47:06 -0400
M configure
M configure.ac
Translation updates
commit : 7f7a9c20621f52867100fa3acfb4d453004ed924
author : Peter Eisentraut <[email protected]>
date : Mon, 9 Aug 2021 11:51:59 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 9 Aug 2021 11:51:59 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 1234b3cdae465246e534cc4114129f18d3c04c38
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/ja.po
M src/backend/po/uk.po
M src/bin/initdb/po/de.po
M src/bin/initdb/po/el.po
M src/bin/initdb/po/es.po
M src/bin/initdb/po/fr.po
M src/bin/initdb/po/ja.po
M src/bin/initdb/po/uk.po
M src/bin/pg_amcheck/nls.mk
M src/bin/pg_amcheck/po/de.po
M src/bin/pg_amcheck/po/el.po
A src/bin/pg_amcheck/po/ja.po
A src/bin/pg_amcheck/po/uk.po
M src/bin/pg_archivecleanup/po/el.po
M src/bin/pg_archivecleanup/po/es.po
M src/bin/pg_archivecleanup/po/ja.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_basebackup/po/ja.po
M src/bin/pg_basebackup/po/uk.po
M src/bin/pg_checksums/po/el.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_checksums/po/ja.po
M src/bin/pg_config/po/el.po
M src/bin/pg_config/po/es.po
M src/bin/pg_config/po/ja.po
M src/bin/pg_controldata/po/el.po
M src/bin/pg_controldata/po/es.po
M src/bin/pg_controldata/po/ja.po
M src/bin/pg_ctl/po/el.po
M src/bin/pg_ctl/po/es.po
M src/bin/pg_ctl/po/ja.po
M src/bin/pg_ctl/po/uk.po
M src/bin/pg_dump/po/el.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/ja.po
M src/bin/pg_dump/po/uk.po
M src/bin/pg_resetwal/nls.mk
M src/bin/pg_resetwal/po/de.po
A src/bin/pg_resetwal/po/el.po
M src/bin/pg_resetwal/po/es.po
M src/bin/pg_resetwal/po/ja.po
M src/bin/pg_rewind/nls.mk
A src/bin/pg_rewind/po/el.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/ja.po
M src/bin/pg_rewind/po/uk.po
M src/bin/pg_test_fsync/po/el.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_fsync/po/ja.po
M src/bin/pg_test_fsync/po/uk.po
M src/bin/pg_test_timing/po/el.po
M src/bin/pg_test_timing/po/ja.po
M src/bin/pg_test_timing/po/uk.po
M src/bin/pg_upgrade/po/de.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/ja.po
M src/bin/pg_upgrade/po/uk.po
M src/bin/pg_verifybackup/po/el.po
M src/bin/pg_verifybackup/po/es.po
M src/bin/pg_verifybackup/po/ja.po
M src/bin/pg_verifybackup/po/uk.po
M src/bin/pg_waldump/po/el.po
M src/bin/pg_waldump/po/es.po
M src/bin/pg_waldump/po/ja.po
M src/bin/psql/po/de.po
M src/bin/psql/po/el.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ja.po
M src/bin/psql/po/uk.po
M src/bin/scripts/po/de.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/ja.po
M src/bin/scripts/po/uk.po
M src/interfaces/ecpg/ecpglib/po/es.po
M src/interfaces/ecpg/ecpglib/po/ja.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/ja.po
M src/interfaces/ecpg/preproc/po/uk.po
M src/interfaces/libpq/po/de.po
M src/interfaces/libpq/po/el.po
M src/interfaces/libpq/po/es.po
M src/interfaces/libpq/po/uk.po
M src/pl/plperl/po/el.po
M src/pl/plperl/po/es.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpython/po/el.po
M src/pl/plpython/po/es.po
M src/pl/tcl/po/el.po
M src/pl/tcl/po/es.po
Doc: Fix misleading statement about VACUUM memory limits
commit : b5815dd00ad32b413070b088d083754d7ec42d37
author : David Rowley <[email protected]>
date : Mon, 9 Aug 2021 16:46:49 +1200
committer: David Rowley <[email protected]>
date : Mon, 9 Aug 2021 16:46:49 +1200
In ec34040af I added a mention that there was no point in setting
maintenance_work_limit to anything higher than 1GB for vacuum, but that
was incorrect as ginInsertCleanup() also looks at what
maintenance_work_mem is set to during VACUUM and that's not limited to
1GB.
Here I attempt to make it more clear that the limitation is only around
the number of dead tuple identifiers that we can collect during VACUUM.
I've also added a note to autovacuum_work_mem to mention this limitation.
I didn't do that in ec34040af as I'd had some wrong-headed ideas about
just limiting the maximum value for that GUC to 1GB.
Author: David Rowley
Discussion: https://postgr.es/m/CAApHDvpGwOAvunp-E-bN_rbAs3hmxMoasm5pzkYDbf36h73s7w@mail.gmail.com
Backpatch-through: 9.6, same as ec34040af
M doc/src/sgml/config.sgml
Use ExplainPropertyInteger for queryid in EXPLAIN
commit : c26552f4fc4fb72c64103ea877ea3c2a251856ad
author : David Rowley <[email protected]>
date : Mon, 9 Aug 2021 15:46:28 +1200
committer: David Rowley <[email protected]>
date : Mon, 9 Aug 2021 15:46:28 +1200
This saves a few lines of code. Also add a comment to mention why we use
ExplainPropertyInteger instead of ExplainPropertyUInteger given that
queryid is a uint64 type.
Author: David Rowley
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAApHDvqhSLYpSU_EqUdN39w9Uvb8ogmHV7_3YhJ0S3aScGBjsg@mail.gmail.com
Backpatch-through: 14, where this code was originally added
M src/backend/commands/explain.c
doc: mention pg_upgrade extension script
commit : 9268fc34526427a48bc27b5ac4727c7fb4cddf7a
author : Bruce Momjian <[email protected]>
date : Sun, 8 Aug 2021 21:05:46 -0400
committer: Bruce Momjian <[email protected]>
date : Sun, 8 Aug 2021 21:05:46 -0400
Since commit e462856a7a, pg_upgrade automatically creates a script to
update extensions, so mention that instead of ALTER EXTENSION.
Backpatch-through: 9.6
M doc/src/sgml/ref/pgupgrade.sgml
Doc: remove bogus <indexterm> items.
commit : c905e64d42fe0725ab7fe84b70a41b4c381aee07
author : Tom Lane <[email protected]>
date : Sun, 8 Aug 2021 15:35:30 -0400
committer: Tom Lane <[email protected]>
date : Sun, 8 Aug 2021 15:35:30 -0400
Copy-and-pasteo in 665c5855e, evidently. The 9.6 docs toolchain
whined about duplicate index entries, though our modern toolchain
doesn't. In any case, these GUCs surely are not about the
default settings of these values.
M doc/src/sgml/config.sgml
Rethink regexp engine's backref-related compilation state.
commit : 5227d99896726b3c889e388f14a3593e104e36ee
author : Tom Lane <[email protected]>
date : Sun, 8 Aug 2021 11:56:29 -0400
committer: Tom Lane <[email protected]>
date : Sun, 8 Aug 2021 11:56:29 -0400
I had committer's remorse almost immediately after pushing cb76fbd7e,
upon finding that removing capturing subexpressions' subREs from the
data structure broke my proposed patch for REG_NOSUB optimization.
Revert that data structure change. Instead, address the concern
about not changing capturing subREs' endpoints by not changing the
endpoints. We don't need to, because the point of that bit was just
to ensure that the atom has endpoints distinct from the outer state
pair that we're stringing the branch between. We already made
suitable states in the parenthesized-subexpression case, so the
additional ones were just useless overhead. This seems more
understandable than Spencer's original coding, and it ought to be
a shade faster too by saving a few state creations and arc changes.
(I actually see a couple percent improvement on Jacobson's web
corpus, though that's barely above the noise floor so I wouldn't
put much stock in that result.)
Also, fix the logic added by ea1268f63 to ensure that the subRE
recorded in v->subs[subno] is exactly the one with capno == subno.
Spencer's original coding recorded the child subRE of the capture
node, which is okay so far as having the right endpoint states is
concerned, but as of cb76fbd7e the capturing subRE itself always
has those endpoints too. I think the inconsistency is confusing
for the REG_NOSUB optimization.
As before, backpatch to v14.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regcomp.c
Make regexp engine's backref-related compilation state more bulletproof.
commit : 5e6ad63c6db79a5515e7a4117b5de053530fd7d9
author : Tom Lane <[email protected]>
date : Sat, 7 Aug 2021 22:27:02 -0400
committer: Tom Lane <[email protected]>
date : Sat, 7 Aug 2021 22:27:02 -0400
Up to now, we remembered the definition of a capturing parenthesis
subexpression by storing a pointer to the associated subRE node.
That was okay before, because that subRE didn't get modified anymore
while parsing the rest of the regexp. However, in the wake of
commit ea1268f63, that's no longer true: the outer invocation of
parseqatom() feels free to scribble on that subRE. This seems to
work anyway, because the states we jam into the child atom in the
"prepare a general-purpose state skeleton" stanza aren't really
semantically different from the original endpoints of the child atom.
But that would be mighty easy to break, and it's definitely not how
things worked before.
Between this and the issue fixed in the prior commit, it seems best
to get rid of this dependence on subRE nodes entirely. We don't need
the whole child subRE for future backrefs, only its starting and ending
NFA states; so let's just store pointers to those.
Also, in the corner case where we make an extra subRE to handle
immediately-nested capturing parentheses, it seems like it'd be smart
to have the extra subRE have the same begin/end states as the original
child subRE does (s/s2 not lp/rp). I think that linking it from lp to
rp might actually be semantically wrong, though since Spencer's original
code did it that way, I'm not totally certain. Using s/s2 is certainly
not wrong, in any case.
Per report from Mark Dilger. Back-patch to v14 where the problematic
patches came in.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regcomp.c
Fix use-after-free issue in regexp engine.
commit : f42ea8350db22725a251e98a5dafb4d2539c800f
author : Tom Lane <[email protected]>
date : Sat, 7 Aug 2021 22:05:27 -0400
committer: Tom Lane <[email protected]>
date : Sat, 7 Aug 2021 22:05:27 -0400
Commit cebc1d34e taught parseqatom() to optimize cases where a branch
contains only one, "messy", atom by getting rid of excess subRE nodes.
The way we really should do that is to keep the subRE built for the
"messy" child atom; but to avoid changing parseqatom's nominal API,
I made it delete that node after copying its fields to the outer subRE
made by parsebranch(). It seems that that actually worked at the time;
but it became dangerous after ea1268f63, because that later commit
allowed the lower invocation of parse() to return a subRE that was also
pointed to by some v->subs[] entry. This meant we could wind up with a
dangling pointer in v->subs[], allowing a later backref to misbehave,
but only if that subRE struct had been reused in between. So the damage
seems confined to cases like '((...))...(...\2'.
To fix, do what I should have done before and modify parseqatom's API
to make it possible for it to remove the caller's subRE instead of the
callee's. That's safer because we know that subRE isn't complete yet,
so noplace else will have a pointer to it.
Per report from Mark Dilger. Back-patch to v14 where the problematic
patches came in.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regcomp.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
pg_amcheck: Message style improvements
commit : 51b95fb257a24aa4186960be8abc277774466218
author : Peter Eisentraut <[email protected]>
date : Sat, 7 Aug 2021 20:34:49 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 7 Aug 2021 20:34:49 +0200
M src/bin/pg_amcheck/pg_amcheck.c
Really fix the ambiguity in REFRESH MATERIALIZED VIEW CONCURRENTLY.
commit : 2c915905e3e82b1db28dce8630f8d20b3db316f2
author : Tom Lane <[email protected]>
date : Sat, 7 Aug 2021 13:29:32 -0400
committer: Tom Lane <[email protected]>
date : Sat, 7 Aug 2021 13:29:32 -0400
Rather than trying to pick table aliases that won't conflict with
any possible user-defined matview column name, adjust the queries'
syntax so that the aliases are only used in places where they can't be
mistaken for column names. Mostly this consists of writing "alias.*"
not just "alias", which adds clarity for humans as well as machines.
We do have the issue that "SELECT alias.*" acts differently from
"SELECT alias", but we can use the same hack ruleutils.c uses for
whole-row variables in SELECT lists: write "alias.*::compositetype".
We might as well revert to the original aliases after doing this;
they're a bit easier to read.
Like 75d66d10e, back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/matview.c
M src/test/regress/expected/matview.out
M src/test/regress/sql/matview.sql
pg_amcheck: Add missing translation markers
commit : 9b0d71725eb2ca6ea0aaffdc38be599b90e7dc56
author : Peter Eisentraut <[email protected]>
date : Sat, 7 Aug 2021 13:36:59 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 7 Aug 2021 13:36:59 +0200
M src/bin/pg_amcheck/nls.mk
M src/bin/pg_amcheck/pg_amcheck.c
Message style improvements
commit : d954019f0a60bb989ef6fe8e478b764d0d5f301c
author : Peter Eisentraut <[email protected]>
date : Sat, 7 Aug 2021 12:09:22 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 7 Aug 2021 12:09:22 +0200
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/analyze.c
Don't elide casting to typmod -1.
commit : e5f6493e3584ea7eec1f992f87639e7f186ae03e
author : Tom Lane <[email protected]>
date : Fri, 6 Aug 2021 17:32:54 -0400
committer: Tom Lane <[email protected]>
date : Fri, 6 Aug 2021 17:32:54 -0400
Casting a value that's already of a type with a specific typmod
to an unspecified typmod doesn't do anything so far as run-time
behavior is concerned. However, it really ought to change the
exposed type of the expression to match. Up to now,
coerce_type_typmod hasn't bothered with that, which creates gotchas
in contexts such as recursive unions. If for example one side of
the union is numeric(18,3), but it needs to be plain numeric to
match the other side, there's no direct way to express that.
This is easy enough to fix, by inserting a RelabelType to update the
exposed type of the expression. However, it's a bit nervous-making
to change this behavior, because it's stood for a really long time.
(I strongly suspect that it's like this in part because the logic
pre-dates the introduction of RelabelType in 7.0. The commit log
message for 57b30e8e2 is interesting reading here.) As a compromise,
we'll sneak the change into 14beta3, and consider back-patching to
stable branches if no complaints emerge in the next three months.
Discussion: https://postgr.es/m/CABNQVagu3bZGqiTjb31a8D5Od3fUMs7Oh3gmZMQZVHZ=uWWWfQ@mail.gmail.com
M src/backend/parser/parse_coerce.c
M src/test/regress/expected/expressions.out
M src/test/regress/sql/expressions.sql
Adjust the integer overflow tests in the numeric code.
commit : 0325565702d8fd18ba66cdeaad4d7f43744525b2
author : Dean Rasheed <[email protected]>
date : Fri, 6 Aug 2021 21:30:25 +0100
committer: Dean Rasheed <[email protected]>
date : Fri, 6 Aug 2021 21:30:25 +0100
Formerly, the numeric code tested whether an integer value of a larger
type would fit in a smaller type by casting it to the smaller type and
then testing if the reverse conversion produced the original value.
That's perfectly fine, except that it caused a test failure on
buildfarm animal castoroides, most likely due to a compiler bug.
Instead, do these tests by comparing against PG_INT16/32_MIN/MAX. That
matches existing code in other places, such as int84(), which is more
widely tested, and so is less likely to go wrong.
While at it, add regression tests covering the numeric-to-int8/4/2
conversions, and adjust the recently added tests to the style of
434ddfb79a (on the v11 branch) to make failures easier to diagnose.
Per buildfarm via Tom Lane, reviewed by Tom Lane.
Discussion: https://postgr.es/m/2394813.1628179479%40sss.pgh.pa.us
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Add missing message punctuation
commit : c3a135b41eed3c719a0c34b8cc072835ff21b129
author : Peter Eisentraut <[email protected]>
date : Fri, 6 Aug 2021 22:11:02 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 6 Aug 2021 22:11:02 +0200
M src/backend/catalog/pg_type.c
M src/test/regress/expected/multirangetypes.out
Fix wording
commit : acd6b6e637fb670003d4ac2477212b4d51016f9a
author : Peter Eisentraut <[email protected]>
date : Fri, 6 Aug 2021 20:55:59 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 6 Aug 2021 20:55:59 +0200
M doc/src/sgml/rules.sgml
M src/backend/utils/adt/multirangetypes_selfuncs.c
M src/backend/utils/adt/rangetypes_selfuncs.c
M src/backend/utils/adt/rangetypes_spgist.c
M src/bin/pg_resetwal/pg_resetwal.c
Doc: remove commit 2945a488a from v14 release notes.
commit : 64b7a8353214a3fa84942366ffb66b36c7f3d2e3
author : Tom Lane <[email protected]>
date : Thu, 5 Aug 2021 19:09:24 -0400
committer: Tom Lane <[email protected]>
date : Thu, 5 Aug 2021 19:09:24 -0400
Now that this has been back-patched, it's no longer a new feature
for v14.
M doc/src/sgml/release-14.sgml
postgres_fdw: Fix issues with generated columns in foreign tables.
commit : 588d3f597c67fac6191658c3befc3b9120772905
author : Etsuro Fujita <[email protected]>
date : Thu, 5 Aug 2021 20:00:01 +0900
committer: Etsuro Fujita <[email protected]>
date : Thu, 5 Aug 2021 20:00:01 +0900
postgres_fdw imported generated columns from the remote tables as plain
columns, and caused failures like "ERROR: cannot insert a non-DEFAULT
value into column "foo"" when inserting into the foreign tables, as it
tried to insert values into the generated columns. To fix, we do the
following under the assumption that generated columns in a postgres_fdw
foreign table are defined so that they represent generated columns in
the underlying remote table:
* Send DEFAULT for the generated columns to the foreign server on insert
or update, not generated column values computed on the local server.
* Add to postgresImportForeignSchema() an option "import_generated" to
include column generated expressions in the definitions of foreign
tables imported from a foreign server. The option is true by default.
The assumption seems reasonable, because that would make a query of the
postgres_fdw foreign table return values for the generated columns that
are consistent with the generated expression.
While here, fix another issue in postgresImportForeignSchema(): it tried
to include column generated expressions as column default expressions in
the foreign table definitions when the import_default option was enabled.
Per bug #16631 from Daniel Cherniy. Back-patch to v12 where generated
columns were added.
Discussion: https://postgr.es/m/16631-e929fe9db0ffc7cf%40postgresql.org
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Fix division-by-zero error in to_char() with 'EEEE' format.
commit : ecbdbdfd9056a55e38f7922565c7ce46b1321907
author : Dean Rasheed <[email protected]>
date : Thu, 5 Aug 2021 09:27:35 +0100
committer: Dean Rasheed <[email protected]>
date : Thu, 5 Aug 2021 09:27:35 +0100
This fixes a long-standing bug when using to_char() to format a
numeric value in scientific notation -- if the value's exponent is
less than -NUMERIC_MAX_DISPLAY_SCALE-1 (-1001), it produced a
division-by-zero error.
The reason for this error was that get_str_from_var_sci() divides its
input by 10^exp, which it produced using power_var_int(). However, the
underflow test in power_var_int() causes it to return zero if the
result scale is too small. That's not a problem for power_var_int()'s
only other caller, power_var(), since that limits the rscale to 1000,
but in get_str_from_var_sci() the exponent can be much smaller,
requiring a much larger rscale. Fix by introducing a new function to
compute 10^exp directly, with no rscale limit. This also allows 10^exp
to be computed more efficiently, without any numeric multiplication,
division or rounding.
Discussion: https://postgr.es/m/CAEZATCWhojfH4whaqgUKBe8D5jNHB8ytzemL-PnRx+KCTyMXmg@mail.gmail.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
pgbench: When using pipelining only do PQconsumeInput() when necessary.
commit : fa604e0dd07a39ba34f93d06ded8243280dffdeb
author : Andres Freund <[email protected]>
date : Wed, 4 Aug 2021 19:19:44 -0700
committer: Andres Freund <[email protected]>
date : Wed, 4 Aug 2021 19:19:44 -0700
Up to now we did a PQconsumeInput() for each pipelined query, asking the OS
for more input - which it often won't have, as all results might already have
been sent. That turns out to have a noticeable performance impact.
Alvaro Herrera reviewed the idea to add the PQisBusy() check, but not this
concrete patch.
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch: 14, where libpq/pgbench pipelining was introduced.
M src/bin/pgbench/pgbench.c
Make vacuum_index_cleanup reloption RELOPT_TYPE_ENUM.
commit : e8086bd3ba0ab73a18ac2293dd14f488734126ec
author : Peter Geoghegan <[email protected]>
date : Tue, 3 Aug 2021 21:53:40 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 3 Aug 2021 21:53:40 -0700
Oversight in commit 3499df0d, which generalized the reloption as a way
of giving users a way to consistently avoid VACUUM's index bypass
optimization.
Per off-list report from Nikolay Shaplov.
Backpatch: 14-, where index cleanup reloption was extended.
M src/backend/access/common/reloptions.c
C comment: correct heading of extension query
commit : 3a0ba31a32f2def545399549c2ace6951f98e47c
author : Bruce Momjian <[email protected]>
date : Tue, 3 Aug 2021 12:26:08 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 3 Aug 2021 12:26:08 -0400
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/bin/pg_upgrade/version.c
doc: interval spill method for units greater than months
commit : 2c92bae3e944d7159611e3010c34fd91b696a27f
author : Bruce Momjian <[email protected]>
date : Tue, 3 Aug 2021 12:17:58 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 3 Aug 2021 12:17:58 -0400
Units are _truncated_ to months, but only in back branches since the
recent commit.
Reported-by: Bryn Llewellyn
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6 to 14
M doc/src/sgml/datatype.sgml
pg_upgrade: warn about extensions that need updating
commit : 4051a7775752bb6643b2fef45f30142b65726b1a
author : Bruce Momjian <[email protected]>
date : Tue, 3 Aug 2021 11:58:15 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 3 Aug 2021 11:58:15 -0400
Also create a script that can be run to update them.
Reported-by: Dave Cramer
Discussion: https://postgr.es/m/CADK3HHKawwbOcGwMGnDuAf3-U8YfvTcS8jqDv3UM=niijs3MMA@mail.gmail.com
Backpatch-through: 9.6
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/version.c
pg_upgrade: improve docs about extension upgrades
commit : 9f6215b4c9e1f208a520c255eff7c6c1dece93c9
author : Bruce Momjian <[email protected]>
date : Tue, 3 Aug 2021 11:27:33 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 3 Aug 2021 11:27:33 -0400
The previous wording was unclear about the steps needed to upgrade
extensions, and how to update them after pg_upgrade.
Reported-by: Dave Cramer
Discussion: https://postgr.es/m/CADK3HHKawwbOcGwMGnDuAf3-U8YfvTcS8jqDv3UM=niijs3MMA@mail.gmail.com
Backpatch-through: 9.6
M doc/src/sgml/ref/pgupgrade.sgml
doc: mention inheritance's tableoid can be used in partitioning
commit : c362570e83ab08e41e18437e158a8542dd8c3556
author : Bruce Momjian <[email protected]>
date : Tue, 3 Aug 2021 11:11:51 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 3 Aug 2021 11:11:51 -0400
Previously tableoid was not mentioned in the partition doc section. We
only had a link to the "all the normal rules" of inheritance section.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 10
M doc/src/sgml/ddl.sgml
doc: add example of using pg_dump with GNU split and gzip
commit : 924e1d0da958c82b79b225b43672532f9f36df9b
author : Bruce Momjian <[email protected]>
date : Tue, 3 Aug 2021 10:57:32 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 3 Aug 2021 10:57:32 -0400
This is only possible with GNU split, not other versions like BSD split.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M doc/src/sgml/backup.sgml
Fix oversight in commit 1ec7fca8592178281cd5cdada0f27a340fb813fc.
commit : fb234086fe81fb1848934b6e1f6382611fc9ad4f
author : Etsuro Fujita <[email protected]>
date : Mon, 2 Aug 2021 12:45:01 +0900
committer: Etsuro Fujita <[email protected]>
date : Mon, 2 Aug 2021 12:45:01 +0900
I failed to account for the possibility that when
ExecAppendAsyncEventWait() notifies multiple async-capable nodes using
postgres_fdw, a preceding node might invoke process_pending_request() to
process a pending asynchronous request made by a succeeding node. In
that case the succeeding node should produce a tuple to return to the
parent Append node from tuples fetched by process_pending_request() when
notified. Repair.
Per buildfarm via Michael Paquier. Back-patch to v14, like the previous
commit.
Thanks to Tom Lane for testing.
Discussion: https://postgr.es/m/YQP0UPT8KmPiHTMs%40paquier.xyz
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/executor/nodeAppend.c
Use elog, not Assert, to report failure to provide an outer snapshot.
commit : ec410c985e6d360f666e39be5609f3c4da5edc8f
author : Tom Lane <[email protected]>
date : Sat, 31 Jul 2021 11:50:14 -0400
committer: Tom Lane <[email protected]>
date : Sat, 31 Jul 2021 11:50:14 -0400
As of commit 84f5c2908, executing SQL commands (via SPI or otherwise)
requires having either an active Portal, or a caller-established
active snapshot. We were simply Assert'ing that that's the case.
But we've now had a couple different reports of people testing
extensions that didn't meet this requirement, and were confused by
the resulting crash. Let's convert the Assert to a test-and-elog,
in hopes of making the issue clearer for extension authors.
Per gripes from Liu Huailing and RekGRpth. Back-patch to v11,
like the prior commit.
Discussion: https://postgr.es/m/OSZPR01MB6215671E3C5956A034A080DFBEEC9@OSZPR01MB6215.jpnprd01.prod.outlook.com
Discussion: https://postgr.es/m/[email protected]
M src/backend/tcop/pquery.c
Fix corner-case errors and loss of precision in numeric_power().
commit : 0d6b87497e32a5b15be8fa3247b8fae48358bb1b
author : Dean Rasheed <[email protected]>
date : Sat, 31 Jul 2021 11:23:48 +0100
committer: Dean Rasheed <[email protected]>
date : Sat, 31 Jul 2021 11:23:48 +0100
This fixes a couple of related problems that arise when raising
numbers to very large powers.
Firstly, when raising a negative number to a very large integer power,
the result should be well-defined, but the previous code would only
cope if the exponent was small enough to go through power_var_int().
Otherwise it would throw an internal error, attempting to take the
logarithm of a negative number. Fix this by adding suitable handling
to the general case in power_var() to cope with negative bases,
checking for integer powers there.
Next, when raising a (positive or negative) number whose absolute
value is slightly less than 1 to a very large power, the result should
approach zero as the power is increased. However, in some cases, for
sufficiently large powers, this would lose all precision and return 1
instead of 0. This was due to the way that the local_rscale was being
calculated for the final full-precision calculation:
local_rscale = rscale + (int) val - ln_dweight + 8
The first two terms on the right hand side are meant to give the
number of significant digits required in the result ("val" being the
estimated result weight). However, this failed to account for the fact
that rscale is clipped to a maximum of NUMERIC_MAX_DISPLAY_SCALE
(1000), and the result weight might be less then -1000, causing their
sum to be negative, leading to a loss of precision. Fix this by
forcing the number of significant digits calculated to be nonnegative.
It's OK for it to be zero (when the result weight is less than -1000),
since the local_rscale value then includes a few extra digits to
ensure an accurate result.
Finally, add additional underflow checks to exp_var() and power_var(),
so that they consistently return zero for cases like this where the
result is indistinguishable from zero. Some paths through this code
already returned zero in such cases, but others were throwing overflow
errors.
Dean Rasheed, reviewed by Yugo Nagata.
Discussion: http://postgr.es/m/CAEZATCW6Dvq7+3wN3tt5jLj-FyOcUgT5xNoOqce5=6Su0bCR0w@mail.gmail.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Fix range check in ECPG numeric to int conversion
commit : f051b87ace6dda42c191b77858c8d099af0de46a
author : John Naylor <[email protected]>
date : Fri, 30 Jul 2021 13:50:23 -0400
committer: John Naylor <[email protected]>
date : Fri, 30 Jul 2021 13:50:23 -0400
The previous coding guarded against -INT_MAX instead of INT_MIN,
leading to -2147483648 being rejected as out of range.
Per bug #17128 from Kevin Sweet
Discussion: https://www.postgresql.org/message-id/flat/17128-55a8a879727a3e3a%40postgresql.org
Reviewed-by: Tom Lane
Backpatch to all supported branches
M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/pgtypeslib/numeric.c
M src/interfaces/ecpg/test/expected/pgtypeslib-num_test.c
M src/interfaces/ecpg/test/expected/pgtypeslib-num_test.stderr
M src/interfaces/ecpg/test/expected/pgtypeslib-num_test.stdout
M src/interfaces/ecpg/test/pgtypeslib/num_test.pgc
Update obsolete comment that still referred to CheckpointLock
commit : 99da905d5c4335d2df80f5cf05dad046a10d30fb
author : Heikki Linnakangas <[email protected]>
date : Fri, 30 Jul 2021 12:52:49 +0300
committer: Heikki Linnakangas <[email protected]>
date : Fri, 30 Jul 2021 12:52:49 +0300
CheckpointLock was removed in commit d18e75664a, and commit ce197e91d0
updated a leftover comment in CreateCheckPoint, but there was another
copy of it in CreateRestartPoint still.
M src/backend/access/transam/xlog.c
postgres_fdw: Fix handling of pending asynchronous requests.
commit : 1cf7fb376acd48710c1c2c5cd97d11cd82e71ae4
author : Etsuro Fujita <[email protected]>
date : Fri, 30 Jul 2021 17:00:01 +0900
committer: Etsuro Fujita <[email protected]>
date : Fri, 30 Jul 2021 17:00:01 +0900
A pending asynchronous request is handled by process_pending_request(),
which previously not only processed an in-progress remote query but
performed ExecForeignScan() to produce a tuple to return to the local
server asynchronously from the result of the remote query. But that led
to a server crash when executing a query or led to an "InstrStartNode
called twice in a row" or "InstrEndLoop called on running node" failure
when doing EXPLAIN ANALYZE of it, in cases where the plan tree for it
contained multiple async-capable nodes accessing the same
initplan/subplan that contained multiple async-capable nodes scanning
the same foreign tables as for the parent async-capable nodes, as
reported by Andrey Lepikhov. The reason is that the second step in
process_pending_request() invoked when executing the initplan/subplan
for one of the parent async-capable nodes caused recursive execution of
the initplan/subplan for another of the parent async-capable nodes.
To fix, split process_pending_request() into the two steps and postpone
the second step until ForeignAsyncConfigureWait() is called for each of
the pending asynchronous requests. Also, in ExecAppendAsyncEventWait()
we assumed that FDWs would register at least one wait event in a
WaitEventSet created there when they were called from
ForeignAsyncConfigureWait() in that function, but allow FDWs to register
zero wait events in the WaitEventSet; modify ExecAppendAsyncEventWait()
to just return in that case.
Oversight in commit 27e1f1456. Back-patch to v14 where that commit went
in.
Andrey Lepikhov and Etsuro Fujita
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/executor/nodeAppend.c
Remove unused argument in apply_handle_commit_internal().
commit : f4b939f1a3720209e1941b88e11e02ea5e671c1b
author : Amit Kapila <[email protected]>
date : Fri, 30 Jul 2021 08:21:59 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 30 Jul 2021 08:21:59 +0530
Oversight in commit 0926e96c49.
Author: Masahiko Sawada
Reviewed-By: Amit Kapila
Backpatch-through: 14, where it was introduced
Discussion: https://postgr.es/m/CAD21AoDeScrsHhLyEPYqN3sydg6PxAPVBboK=30xJfUVihNZDA@mail.gmail.com
M src/backend/replication/logical/worker.c
Close yet another race condition in replication slot test code
commit : f951f6f69c7e77acdd04776168126f00e269dcfb
author : Alvaro Herrera <[email protected]>
date : Thu, 29 Jul 2021 17:09:06 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 29 Jul 2021 17:09:06 -0400
Buildfarm shows that this test has a further failure mode when a
checkpoint starts earlier than expected, so we detect a "checkpoint
completed" line that's not the one we want. Change the config to try
and prevent this.
Per buildfarm
While at it, update one comment that was forgotten in commit
d18e75664a2f.
Author: Kyotaro Horiguchi <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
M src/test/recovery/t/019_replslot_limit.pl
docs: Fix bit_count example output
commit : c73bba23ef07f88ce3860a06bbf56180e00b185d
author : Daniel Gustafsson <[email protected]>
date : Thu, 29 Jul 2021 21:39:40 +0200
committer: Daniel Gustafsson <[email protected]>
date : Thu, 29 Jul 2021 21:39:40 +0200
The returnvalue for the bit_count(::bytea) example was assuming a
non-default value of standard_conforming_strings. This was fixed
in the tests in commit ebedd0c78.
Author: [email protected]
Discussion: https://postgr.es/m/OSZPR01MB6551FFAC1088C82C3D799BE0FAEB9@OSZPR01MB6551.jpnprd01.prod.outlook.com
Backpatch-through: 14
M doc/src/sgml/func.sgml
Improve libpq's handling of OOM during error message construction.
commit : 43f1d2ab361ccbd5ba8b53578fd4bcea9a6344a6
author : Tom Lane <[email protected]>
date : Thu, 29 Jul 2021 13:33:31 -0400
committer: Tom Lane <[email protected]>
date : Thu, 29 Jul 2021 13:33:31 -0400
Commit ffa2e4670 changed libpq so that multiple error reports
occurring during one operation (a connection attempt or query)
are accumulated in conn->errorMessage, where before new ones
usually replaced any prior error. At least in theory, that makes
us more vulnerable to running out of memory for the errorMessage
buffer. If it did happen, the user would be left with just an
empty-string error report, which is pretty unhelpful.
We can improve this by relying on pqexpbuffer.c's existing "broken
buffer" convention to track whether we've hit OOM for the current
operation's error string, and then substituting a constant "out of
memory" string in the small number of places where the errorMessage
is read out.
While at it, apply the same method to similar OOM cases in
pqInternalNotice and pqGetErrorNotice3.
Back-patch to v14 where ffa2e4670 came in. In principle this could
go back further; but in view of the lack of field reports, the
hazard seems negligible in older branches.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-int.h
Avoid calling TestLib::perl2host on a symlinked directory
commit : c42d3d04d7d03782ae179bf92ea0eb9f2fa8f409
author : Andrew Dunstan <[email protected]>
date : Thu, 29 Jul 2021 12:15:03 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 29 Jul 2021 12:15:03 -0400
Certain versions of msys2/Windows have been observed to resolve symlinks
in perl2host rather than just follow them. This defeats using a
symlinked shorter path to a longer path, and makes certain tests fail.
We therefore call perl2host on the parent directory of the symlink and
thereafter just use that result.
Apply to release 14 where the problem has been observed.
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
Make TestLib::perl2host more consistent and robust
commit : 68011e17d098da66070a2d648a609625241f73f6
author : Andrew Dunstan <[email protected]>
date : Thu, 29 Jul 2021 12:15:03 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 29 Jul 2021 12:15:03 -0400
Sometimes cygpath has been observed to return a path with a trailing
slash. That can cause problems, Also, make "cygpath" usage
consistent with "pwd -W" with respect to the use of forward slashes.
Backpatch to release 14 where the current code was introduced.
M src/test/perl/TestLib.pm
Add missing exit() in pg_verifybackup when failing to find pg_waldump
commit : 67445deb7eca32d25721dffb228b009bfbe415d5
author : Michael Paquier <[email protected]>
date : Thu, 29 Jul 2021 10:59:56 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 29 Jul 2021 10:59:56 +0900
pg_verifybackup needs by default pg_waldump to check after a range of
WAL segments required for a backup, except if --no-parse-wal is
specified. The code checked for the presence of the binary pg_waldump
in an installation and reported an error, but it forgot to properly
exit(). This could lead to confusing errors reported.
Reviewed-by: Robert Haas, Fabien Coelho
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/bin/pg_verifybackup/pg_verifybackup.c
Update minimum recovery point on truncation during WAL replay of abort record.
commit : f2a3d7404e5d4aa17dbdf7299b1f0d548fe59b9d
author : Fujii Masao <[email protected]>
date : Thu, 29 Jul 2021 01:30:02 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 29 Jul 2021 01:30:02 +0900
If a file is truncated, we must update minRecoveryPoint. Once a file is
truncated, there's no going back; it would not be safe to stop recovery
at a point earlier than that anymore.
Commit 7bffc9b7bf changed xact_redo_commit() so that it updates
minRecoveryPoint on truncation, but forgot to change xact_redo_abort().
Back-patch to all supported versions.
Reported-by: [email protected]
Author: Fujii Masao
Reviewed-by: Heikki Linnakangas
Discussion: https://postgr.es/m/b029fce3-4fac-4265-968e-16f36ff4d075.mengjuan.cmj@alibaba-inc.com
M src/backend/access/transam/xact.c
Disallow negative strides in date_bin()
commit : fc0d9b8c224ff6b84113cefdca1ba9dde879d863
author : John Naylor <[email protected]>
date : Wed, 28 Jul 2021 11:22:58 -0400
committer: John Naylor <[email protected]>
date : Wed, 28 Jul 2021 11:22:58 -0400
It's not clear what the semantics of negative strides would be, so throw
an error instead.
Per report from Bauyrzhan Sakhariyev
Reviewed-by: Tom Lane, Michael Paquier
Discussion: https://www.postgresql.org/message-id/CAKpL73vZmLuFVuwF26FJ%2BNk11PVHhAnQRoREFcA03x7znRoFvA%40mail.gmail.com
Backpatch to v14
M doc/src/sgml/func.sgml
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Doc: Clarify lock levels taken during ATTACH PARTITION
commit : 5a8a9be08307a1c06fbed4510667de6b4f40fe56
author : David Rowley <[email protected]>
date : Wed, 28 Jul 2021 15:02:12 +1200
committer: David Rowley <[email protected]>
date : Wed, 28 Jul 2021 15:02:12 +1200
It wasn't all that clear which lock levels, if any, would be held on the
DEFAULT partition during an ATTACH PARTITION operation.
Also, clarify which locks will be taken if the DEFAULT partition or the
table being attached are themselves partitioned tables.
Here I'm only backpatching to v12 as before then we obtained an ACCESS
EXCLUSIVE lock on the partitioned table. It seems much less relevant to
mention which locks are taken on other tables when the partitioned table
itself is locked with an ACCESS EXCLUSIVE lock.
Author: Matthias van de Meent, David Rowley
Discussion: https://postgr.es/m/CAEze2WiTB6iwrV8W_J=fnrnZ7fowW3qu-8iQ8zCHP3FiQ6+o-A@mail.gmail.com
Backpatch-through: 12
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/alter_table.sgml
Set pg_setting.pending_restart when pertinent config lines are removed
commit : ad3b40eb26a30d376e8448eb1170421501f0fc6b
author : Alvaro Herrera <[email protected]>
date : Tue, 27 Jul 2021 15:44:12 -0400
committer: Alvaro Herrera <[email protected]>
date : Tue, 27 Jul 2021 15:44:12 -0400
This changes the behavior of examining the pg_file_settings view after
changing a config option that requires restart. The user needs to know
that any change of such options does not take effect until a restart,
and this worked correctly if the line is edited without removing it.
However, for the case where the line is removed altogether, the flag
doesn't get set, because a flag was only set in set_config_option, but
that's not called for lines removed. Repair.
(Ref.: commits 62d16c7fc561 and a486e35706ea)
Author: Álvaro Herrera <[email protected]>
Reviewed-by: Daniel Gustafsson <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/misc/guc-file.l
Fix bugs in polymorphic-argument resolution for multiranges.
commit : b7ea0e8c41f1e512923267a57cd08df63115b783
author : Tom Lane <[email protected]>
date : Tue, 27 Jul 2021 15:01:49 -0400
committer: Tom Lane <[email protected]>
date : Tue, 27 Jul 2021 15:01:49 -0400
We failed to deal with an UNKNOWN-type input for
anycompatiblemultirange; that should throw an error indicating
that we don't know how to resolve the multirange type.
We also failed to infer the type of an anycompatiblerange output
from an anycompatiblemultirange input or vice versa.
Per bug #17066 from Alexander Lakhin. Back-patch to v14
where multiranges were added.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/parse_coerce.c
M src/test/regress/expected/polymorphism.out
M src/test/regress/expected/rangefuncs.out
M src/test/regress/sql/polymorphism.sql
Avoid using ambiguous word "non-negative" in error messages.
commit : fd90f6ba7a75f51d14dac27219bb3755b893e251
author : Fujii Masao <[email protected]>
date : Wed, 28 Jul 2021 01:20:16 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 28 Jul 2021 01:20:16 +0900
The error messages using the word "non-negative" are confusing
because it's ambiguous about whether it accepts zero or not.
This commit improves those error messages by replacing it with
less ambiguous word like "greater than zero" or
"greater than or equal to zero".
Also this commit added the note about the word "non-negative" to
the error message style guide, to help writing the new error messages.
When postgres_fdw option fetch_size was set to zero, previously
the error message "fetch_size requires a non-negative integer value"
was reported. This error message was outright buggy. Therefore
back-patch to all supported versions where such buggy error message
could be thrown.
Reported-by: Hou Zhijie
Author: Bharath Rupireddy
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/OS0PR01MB5716415335A06B489F1B3A8194569@OS0PR01MB5716.jpnprd01.prod.outlook.com
M contrib/postgres_fdw/option.c
M doc/src/sgml/sources.sgml
M src/backend/partitioning/partbounds.c
M src/backend/utils/adt/tsquery_op.c
M src/test/modules/test_shm_mq/test.c
M src/test/regress/expected/hash_part.out
doc: for various substring funcs, document if only first match
commit : 5a353a2d43eeab7554a6986b04b0a9d98ad55c92
author : Bruce Momjian <[email protected]>
date : Mon, 26 Jul 2021 22:54:35 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 26 Jul 2021 22:54:35 -0400
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M doc/src/sgml/func.sgml
pg_resetxlog: add option to set oldest xid & use by pg_upgrade
commit : 695b4a113abca380fd7aed26cefa039e4e8cd569
author : Bruce Momjian <[email protected]>
date : Mon, 26 Jul 2021 22:38:14 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 26 Jul 2021 22:38:14 -0400
Add pg_resetxlog -u option to set the oldest xid in pg_control.
Previously -x set this value be -2 billion less than the -x value.
However, this causes the server to immediately scan all relation's
relfrozenxid so it can advance pg_control's oldest xid to be inside the
autovacuum_freeze_max_age range, which is inefficient and might disrupt
diagnostic recovery. pg_upgrade will use this option to better create
the new cluster to match the old cluster.
Reported-by: Jason Harvey, Floris Van Nee
Discussion: https://postgr.es/m/[email protected], [email protected]
Author: Bertrand Drouvot
Backpatch-through: 9.6
M doc/src/sgml/ref/pg_resetwal.sgml
M src/bin/pg_resetwal/pg_resetwal.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/pg_upgrade.h
psql \dX: check schema when listing statistics objects
commit : 611e42444b121370f561d860552beb7d28dacbf8
author : Tomas Vondra <[email protected]>
date : Mon, 26 Jul 2021 17:12:28 +0200
committer: Tomas Vondra <[email protected]>
date : Mon, 26 Jul 2021 17:12:28 +0200
Commit ad600bba04 added psql command \dX listing extended statistics
objects, but it failed to consider search_path when selecting the
elements so some of the returned elements might be invisible.
The visibility was already considered for tab completion (added by
commit d99d58cdc8), so adding it to the query is fairly simple.
Reported and fix by Justin Pryzby, regression tests by me. Backpatch
to PostgreSQL 14, where \dX was introduced.
Batchpatch-through: 14
Author: Justin Pryzby
Reviewed-by: Tatsuro Yamada
Discussion: https://postgr.es/m/c027a541-5856-75a5-0868-341301e1624b%40nttcom.co.jp_1
M src/bin/psql/describe.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
doc: Fix command example to run regression tests with PGOPTIONS
commit : f2a37ddbb10177fd731109df4a63d10150a91c0d
author : Michael Paquier <[email protected]>
date : Mon, 26 Jul 2021 16:27:01 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 26 Jul 2021 16:27:01 +0900
The documentation mentioned the use of log_checkpoints, that cannot be
used in this context. This commit replaces log_checkpoints with
force_parallel_mode, a developer option useful to perform checks related
to parallelism.
Oversight in 854434c.
Author: Haiying Tang
Discussion: https://postgr.es/m/OS0PR01MB6113954B883ACEB2DDC973F2FBE59@OS0PR01MB6113.jpnprd01.prod.outlook.com
Backpatch-through: 14
M doc/src/sgml/regress.sgml
Harden pg_stat_statements tests against CLOBBER_CACHE_ALWAYS.
commit : e6d9544681b3f5c66a16c72bfd8dca450ec7536a
author : Tom Lane <[email protected]>
date : Sun, 25 Jul 2021 23:25:15 -0400
committer: Tom Lane <[email protected]>
date : Sun, 25 Jul 2021 23:25:15 -0400
Turns out the buildfarm hasn't been testing this, which will soon change.
Julien Rouhaud, per report from me
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
Fix incorrect comment for get_agg_clause_costs
commit : dece64a941457686184654968ac1ded58b7d4ee8
author : David Rowley <[email protected]>
date : Mon, 26 Jul 2021 14:56:09 +1200
committer: David Rowley <[email protected]>
date : Mon, 26 Jul 2021 14:56:09 +1200
Adjust the header comment in get_agg_clause_costs so that it matches what
the function currently does. No recursive searching has been done ever
since 0a2bc5d61. It also does not determine the aggtranstype like the
comment claimed. That's all done in preprocess_aggref().
preprocess_aggref also now determines the numOrderedAggs, so remove the
mention that get_agg_clause_costs also calculates "counts".
Normally, since this is just an adjustment of a comment it might not be
worth back-patching, but since this code is new to PG14 and that version
is still in beta, then it seems worth having the comments match.
Discussion: https://postgr.es/m/CAApHDvrrGrTJFPELrjx0CnDtz9B7Jy2XYW3Z2BKifAWLSaJYwQ@mail.gmail.com
Backpatch-though: 14
M src/backend/optimizer/prep/prepagg.c
Fix a couple of memory leaks in src/bin/pg_basebackup/
commit : b0d28671964d61416248789e3cee4508ac6281b4
author : Michael Paquier <[email protected]>
date : Mon, 26 Jul 2021 11:14:08 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 26 Jul 2021 11:14:08 +0900
These have been introduced by 7fbe0c8, and could happen for
pg_basebackup and pg_receivewal.
Per report from Coverity for the ones in walmethods.c, I have spotted
the ones in receivelog.c after more review.
Backpatch-through: 10
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_basebackup/walmethods.h
Get rid of artificial restriction on hash table sizes on Windows.
commit : b154ee63bb659ce280d486db6bbbe77ddec105c5
author : Tom Lane <[email protected]>
date : Sun, 25 Jul 2021 14:02:27 -0400
committer: Tom Lane <[email protected]>
date : Sun, 25 Jul 2021 14:02:27 -0400
The point of introducing the hash_mem_multiplier GUC was to let users
reproduce the old behavior of hash aggregation, i.e. that it could use
more than work_mem at need. However, the implementation failed to get
the job done on Win64, where work_mem is clamped to 2GB to protect
various places that calculate memory sizes using "long int". As
written, the same clamp was applied to hash_mem. This resulted in
severe performance regressions for queries requiring a bit more than
2GB for hash aggregation, as they now spill to disk and there's no
way to stop that.
Getting rid of the work_mem restriction seems like a good idea, but
it's a big job and could not conceivably be back-patched. However,
there's only a fairly small number of places that are concerned with
the hash_mem value, and it turns out to be possible to remove the
restriction there without too much code churn or any ABI breaks.
So, let's do that for now to fix the regression, and leave the
larger task for another day.
This patch does introduce a bit more infrastructure that should help
with the larger task, namely pg_bitutils.h support for working with
size_t values.
Per gripe from Laurent Hasson. Back-patch to v13 where the
behavior change came in.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/MN2PR15MB25601E80A9B6D1BA6F592B1985E39@MN2PR15MB2560.namprd15.prod.outlook.com
M src/backend/executor/execGrouping.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeMemoize.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/pathnode.c
M src/include/miscadmin.h
M src/include/port/pg_bitutils.h
Deduplicate choice of horizon for a relation procarray.c.
commit : 3d0a4636aa4c976e971c05c77e162fc70c61f40b
author : Andres Freund <[email protected]>
date : Sat, 24 Jul 2021 20:14:03 -0700
committer: Andres Freund <[email protected]>
date : Sat, 24 Jul 2021 20:14:03 -0700
5a1e1d83022 was a minimal bug fix for dc7420c2c92. To avoid future bugs of
that kind, deduplicate the choice of a relation's horizon into a new helper,
GlobalVisHorizonKindForRel().
As the code in question was only introduced in dc7420c2c92 it seems worth
backpatching this change as well, otherwise 14 will look different from all
other branches.
A different approach to this was suggested by Matthias van de Meent.
Author: Andres Freund
Discussion: https://postgr.es/m/[email protected]
Backpatch: 14, like 5a1e1d83022
M src/backend/storage/ipc/procarray.c
Fix check for conflicting session- vs transaction-level locks.
commit : 712ba6b8b73870fa9e3336df8d88f4bc5f824112
author : Tom Lane <[email protected]>
date : Sat, 24 Jul 2021 18:35:52 -0400
committer: Tom Lane <[email protected]>
date : Sat, 24 Jul 2021 18:35:52 -0400
We have an implementation restriction that PREPARE TRANSACTION can't
handle cases where both session-lifespan and transaction-lifespan locks
are held on the same lockable object. (That's because we'd otherwise
need to acquire a new PROCLOCK entry during post-prepare cleanup, which
is an operation that might fail. The situation can only arise with odd
usages of advisory locks, so removing the restriction is probably not
worth the amount of effort it would take.) AtPrepare_Locks attempted
to enforce this, but its logic was many bricks shy of a load, because
it only detected cases where the session and transaction locks had the
same lockmode. Locks of different modes on the same object would lead
to the rather unhelpful message "PANIC: we seem to have dropped a bit
somewhere".
To fix, build a transient hashtable with one entry per locktag,
not one per locktag + mode, and use that to detect conflicts.
Per bug #17122 from Alexander Pyhalov. This bug is ancient,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/lmgr/lock.c
M src/test/regress/expected/prepared_xacts.out
M src/test/regress/expected/prepared_xacts_1.out
M src/test/regress/sql/prepared_xacts.sql
Make printf("%s", NULL) print "(null)" instead of crashing.
commit : 89ad14cd7870bce199448c527e3130c36f80d2bf
author : Tom Lane <[email protected]>
date : Sat, 24 Jul 2021 13:41:17 -0400
committer: Tom Lane <[email protected]>
date : Sat, 24 Jul 2021 13:41:17 -0400
We previously took a hard-line attitude that callers should never print
a null string pointer, and doing so is worthy of an assertion failure
or crash. However, we've long since flushed out any easy-to-find bugs
of that nature. What remains is a lot of code that perhaps could fail
that way in hard-to-reach corner cases. For example, in something as
simple as
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_OBJECT),
errmsg("constraint \"%s\" for table \"%s\" does not exist",
conname, get_rel_name(relid))));
one must wonder whether it's completely guaranteed that get_rel_name
cannot return NULL in this context. If such a situation did occur,
the existing policy converts what might be a pretty minor bug into
a server crash condition. This is not good for robustness.
Hence, let's follow the lead of glibc and print "(null)" instead
of failing. We should, of course, still consider it a bug if that
behavior is reachable in ordinary use; but crashing seems less
desirable than not crashing.
This fix works across-the-board in v12 and up, where we always use
src/port/snprintf.c. Before that, on most platforms we're at the mercy
of the local libc, but it appears that Solaris 10 is the only supported
platform where we'd still get a crash. Most other platforms such as
*BSD, macOS, and Solaris 11 have adopted glibc's behavior at some
point. (AIX and HPUX just print "" not "(null)", but that's close
enough.) I've not checked what Windows' native printf would do, but
it doesn't matter because we've long used snprintf.c on that platform.
In v12 and up, also const-ify related code so that we're not casting
away const on the constant string. This is just neatnik-ism, since
next to no compilers will warn about that.
Discussion: https://postgr.es/m/[email protected]
M src/port/snprintf.c
Remove configure-time thread safety checking (thread_test.c).
commit : d5e913a81cbab063c53b0a06dc8894da89390676
author : Tom Lane <[email protected]>
date : Sat, 24 Jul 2021 12:16:40 -0400
committer: Tom Lane <[email protected]>
date : Sat, 24 Jul 2021 12:16:40 -0400
This testing was useful when it was written, nigh twenty years ago,
but it seems fairly pointless for any platform built in the last
dozen or more years. (Compare also the comments at 8a2121185.)
Also we now have reports that the test program itself fails under
ThreadSanitizer. Rather than invest effort in fixing it, let's
just drop it, and assume that the few people who still care
already know they need to use --disable-thread-safety.
Back-patch into v14, for consistency with 8a2121185.
Discussion: https://postgr.es/m/CADhDkKzPSiNvA3Hyq+wSR_icuPmazG0cFe=YnC3U-CFcYLc8Xw@mail.gmail.com
D config/thread_test.c
M configure
M configure.ac
Fix division by zero error in date_bin
commit : 81322fc409743f9ef169cb7bd89b0d0113a0aaa1
author : John Naylor <[email protected]>
date : Thu, 22 Jul 2021 17:34:19 -0400
committer: John Naylor <[email protected]>
date : Thu, 22 Jul 2021 17:34:19 -0400
Bauyrzhan Sakhariyev, via Github
Backpatch to v14
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
jit: Don't inline functions that access thread-locals.
commit : b1c1b7af573bbc9fe9039c82ffb7d3a3c378fe4a
author : Thomas Munro <[email protected]>
date : Thu, 22 Jul 2021 14:11:17 +1200
committer: Thomas Munro <[email protected]>
date : Thu, 22 Jul 2021 14:11:17 +1200
Code inlined by LLVM can crash or fail with "Relocation type not
implemented yet!" if it tries to access thread local variables. Don't
inline such code.
Back-patch to 11, where LLVM arrived. Bug #16696.
Author: Dmitry Marakasov <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/jit/llvm/llvmjit_inline.cpp
Doc: improve documentation about exponentiation operator.
commit : 227c4e57d6a70961e015ed4185facfc638afd048
author : Tom Lane <[email protected]>
date : Wed, 21 Jul 2021 18:03:33 -0400
committer: Tom Lane <[email protected]>
date : Wed, 21 Jul 2021 18:03:33 -0400
Now that we're not having to wedge this into the straitjacket of
the old operator table format, we can add another example to
clarify the point about left-to-right associativity.
Per suggestion from mdione at grulic.org.ar.
https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
Document "B" and "us" as accepted units in postgres.conf.sample
commit : 65cc77c9847022f895a12012085606d8553c5fff
author : John Naylor <[email protected]>
date : Wed, 21 Jul 2021 10:17:07 -0400
committer: John Naylor <[email protected]>
date : Wed, 21 Jul 2021 10:17:07 -0400
In postgresql.conf, memory and file size GUCs can be specified with "B"
(bytes) as of b06d8e58b. Likewise, time GUCs can be specified with "us"
(microseconds) as of caf626b2c. Update postgres.conf.sample to reflect
that fact.
Pavel Luzanov
Backpatch to v12, which is the earliest version that allows both of
these units. A separate commit will document the "B" case for v11.
Discussion: https://www.postgresql.org/message-id/flat/f10d16fc-8fa0-1b3c-7371-cb3a35a13b7a%40postgrespro.ru
M src/backend/utils/misc/postgresql.conf.sample
Add missing check of noError parameter in euc_tw_and_big5.c
commit : e5cebe1ae8895fd9793344e70f58714e0a1742a4
author : John Naylor <[email protected]>
date : Wed, 21 Jul 2021 09:09:32 -0400
committer: John Naylor <[email protected]>
date : Wed, 21 Jul 2021 09:09:32 -0400
Oversight in ea1b99a66
Yukun Wang
Backpatch to v14 where this parameter was introduced
Discussion: https://www.postgresql.org/message-id/flat/OS0PR01MB6003FCEFF0201EF21685FD33B4E39%40OS0PR01MB6003.jpnprd01.prod.outlook.com
M src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
doc: Document that only superusers can use pg_import_system_collations().
commit : 9c1d56a9b08f87303749e60f9df8cf5c26c279eb
author : Fujii Masao <[email protected]>
date : Wed, 21 Jul 2021 13:52:37 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 21 Jul 2021 13:52:37 +0900
Back-patch to v10 where pg_import_system_collations() was added.
Author: Atsushi Torikoshi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
doc: PG 14 relnote adjustments
commit : f8d1333dec312f96673efc6314a0f1df99e38abc
author : Bruce Momjian <[email protected]>
date : Tue, 20 Jul 2021 19:37:04 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 20 Jul 2021 19:37:04 -0400
Reported-by: Elena Indrupskaya
Discussion: https://postgr.es/m/[email protected]
Author: Elena Indrupskaya
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
Fix corner-case uninitialized-variable issues in plpgsql.
commit : 899564e010abf216a90c48d9a1ce3728b8b00266
author : Tom Lane <[email protected]>
date : Tue, 20 Jul 2021 13:01:48 -0400
committer: Tom Lane <[email protected]>
date : Tue, 20 Jul 2021 13:01:48 -0400
If an error was raised during our initial attempt to check whether
a successfully-compiled expression is "simple", subsequent calls of
exec_stmt_execsql would suppose that stmt->mod_stmt was already computed
when it had not been. This could lead to assertion failures in debug
builds; in production builds the effect would typically be to act as
if INTO STRICT had been specified even when it had not been. Of course
that only matters if the subsequent attempt to execute the expression
succeeds, so that the problem can only be reached by fixing a failure
in some referenced, inline-able SQL function and then retrying the
calling plpgsql function in the same session.
(There might be even-more-obscure ways to change the expression's
behavior without changing the plpgsql function, but that one seems
like the only one people would be likely to hit in practice.)
The most foolproof way to fix this would be to arrange for
exec_prepare_plan to not set expr->plan until we've finished the
subsidiary simple-expression check. But it seems hard to do that
without creating reference-count leak issues. So settle for documenting
the hazard in a comment and fixing exec_stmt_execsql to test separately
for whether it's computed stmt->mod_stmt. (That adds a test-and-branch
per execution, but hopefully that's negligible in context.) In v11 and
up, also fix exec_stmt_call which had a variant of the same issue.
Per bug #17113 from Alexander Lakhin. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/pl/plpgsql/src/expected/plpgsql_simple.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
M src/pl/plpgsql/src/sql/plpgsql_simple.sql
Fix some issues with WAL segment opening for pg_receivewal --compress
commit : 3a0d2d0cbaf349a1aba928bb7b14edeaffab5212
author : Michael Paquier <[email protected]>
date : Tue, 20 Jul 2021 12:12:47 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 20 Jul 2021 12:12:47 +0900
The logic handling the opening of new WAL segments was fuzzy when using
--compress if a partial, non-compressed, segment with the same base name
existed in the repository storing those files. In this case, using
--compress would cause the code to first check for the existence and the
size of a non-compressed segment, followed by the opening of a new
compressed, partial, segment. The code was accidentally working
correctly on most platforms as the buildfarm has proved, except
bowerbird where gzflush() could fail in this code path. It is wrong
anyway to take the code path used pre-padding when creating a new
partial, non-compressed, segment, so let's fix it.
Note that this issue exists when users mix successive runs of
pg_receivewal with or without compression, as discovered with the tests
introduced by ffc9dda.
While on it, this refactors the code so as code paths that need to know
about the ".gz" suffix are down from four to one in walmethods.c, easing
a bit the introduction of new compression methods. This addresses a
second issue where log messages generated for an unexpected failure
would not show the compressed segment name involved, which was
confusing, printing instead the name of the non-compressed equivalent.
Reported-by: Georgios Kokolatos
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 10
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_basebackup/walmethods.h
Doc: vacuum_multixact_failsafe_age is multixact-based.
commit : e1cdf61726e8d70971cf29d98c7ab5806875ac0e
author : Peter Geoghegan <[email protected]>
date : Mon, 19 Jul 2021 17:20:24 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 19 Jul 2021 17:20:24 -0700
Oversight in commit 1e55e7d1, which added a wraparound failsafe
mechanism to VACUUM.
Backpatch: 14-, where VACUUM failsafe was introduced.
M doc/src/sgml/config.sgml
vacuumdb: Correct comment about --force-index-cleanup.
commit : 9a3d41a26fbb00c67b1ac00306b21ab83f171e08
author : Peter Geoghegan <[email protected]>
date : Mon, 19 Jul 2021 17:06:47 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 19 Jul 2021 17:06:47 -0700
Commit 3499df0d added a comment that incorrectly suggested that
--force-index-cleanup did not appear in the same major version as the
similar --no-index-cleanup option. In fact, both options are new to
PostgreSQL 14.
Backpatch: 14-, where both options were introduced.
M src/bin/scripts/vacuumdb.c
Make new replication slot test code even less racy
commit : 1e8751380836056fdf546433dcf0328bc18c9313
author : Alvaro Herrera <[email protected]>
date : Mon, 19 Jul 2021 17:21:07 -0400
committer: Alvaro Herrera <[email protected]>
date : Mon, 19 Jul 2021 17:21:07 -0400
Further fix the test code in ead9e51e8236, this time by waiting until
the checkpoint has completed before moving on; this ensures that the
WAL segment removal has already happened when we create the next slot.
Author: Kyotaro Horiguchi <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/019_replslot_limit.pl
Don't allow to set replication slot_name as ''.
commit : 40295d158fd4d462c55e6debae19dcd43ab530a6
author : Amit Kapila <[email protected]>
date : Mon, 19 Jul 2021 10:54:21 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 19 Jul 2021 10:54:21 +0530
We don't allow to create replication slot_name as an empty string ('') via
SQL API pg_create_logical_replication_slot() but it is allowed to be set
via Alter Subscription command. This will lead to apply worker repeatedly
keep trying to stream data via slot_name '' and the user is not allowed to
create the slot with that name.
Author: Japin Li
Reviewed-By: Ranier Vilela, Amit Kapila
Backpatch-through: 10, where it was introduced
Discussion: https://postgr.es/m/MEYP282MB1669CBD98E721C77CA696499B61A9@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
M src/backend/commands/subscriptioncmds.c
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
doc: Mention CASCADE/RESTRICT for DROP STATISTICS
commit : 6d0dc1a7da7942915c8160caa379c58dfd1488b8
author : Michael Paquier <[email protected]>
date : Mon, 19 Jul 2021 12:39:50 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 19 Jul 2021 12:39:50 +0900
This grammar has no effect as there are no dependencies on statistics,
but it is supported by the parser. This is more consistent with the
other DROP commands.
Author: Vignesh C
Discussion: https://postgr.es/m/CALDaNm1LA=yNmzcSfy+0oe6CEAgsxXRf_-UutE3ZncFi8QkFNQ@mail.gmail.com
Backpatch-through: 10
M doc/src/sgml/ref/drop_statistics.sgml
Support for unnest(multirange)
commit : 244ad5415557812a6ac4dc5d6e2ae908361d82c3
author : Alexander Korotkov <[email protected]>
date : Sun, 18 Jul 2021 21:07:24 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 18 Jul 2021 21:07:24 +0300
It has been spotted that multiranges lack of ability to decompose them into
individual ranges. Subscription and proper expanded object representation
require substantial work, and it's too late for v14. This commit
provides the implementation of unnest(multirange), which is quite trivial.
unnest(multirange) is defined as a polymorphic procedure.
Catversion is bumped.
Reported-by: Jonathan S. Katz
Discussion: https://postgr.es/m/flat/60258efe-bd7e-4886-82e1-196e0cac5433%40postgresql.org
Author: Alexander Korotkov
Reviewed-by: Justin Pryzby, Jonathan S. Katz, Zhihong Yu, Tom Lane
Reviewed-by: Alvaro Herrera
M doc/src/sgml/func.sgml
M src/backend/utils/adt/multirangetypes.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/multirangetypes.out
M src/test/regress/sql/multirangetypes.sql
Make new replication slot test code less racy
commit : d8f3b021c618bf58c19f88fb86476793fd650336
author : Alvaro Herrera <[email protected]>
date : Sat, 17 Jul 2021 13:19:17 -0400
committer: Alvaro Herrera <[email protected]>
date : Sat, 17 Jul 2021 13:19:17 -0400
The new test code added in ead9e51e8236 is racy -- it hinges on
shared-memory state, which changes before the WARNING message is logged.
Put it the other way around.
Backpatch to 13.
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/019_replslot_limit.pl
Doc: document the current-transaction-modes GUCs.
commit : 4466b38ec6ffd4ddea135c5baff978b7b43f8257
author : Tom Lane <[email protected]>
date : Sat, 17 Jul 2021 11:52:54 -0400
committer: Tom Lane <[email protected]>
date : Sat, 17 Jul 2021 11:52:54 -0400
We had documentation of default_transaction_isolation et al,
but for some reason not of transaction_isolation et al.
AFAICS this is just an ancient oversight, so repair.
Per bug #17077 from Yanliang Lei.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/set_transaction.sgml
Fix pg_dump for disabled triggers on partitioned tables
commit : 3c5b7c6286215de41ba0a327d065876ff8bc26b2
author : Alvaro Herrera <[email protected]>
date : Fri, 16 Jul 2021 17:29:22 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 16 Jul 2021 17:29:22 -0400
pg_dump failed to preserve the 'enabled' flag (which can be not only
disabled, but also REPLICA or ALWAYS) for partitions which had it
changed from their respective parents. Attempt to handle that by
including a definition for such triggers in the dump, but replace the
standard CREATE TRIGGER line with an ALTER TRIGGER line.
Backpatch to 11, where these triggers can exist. In branches 11 and 12,
pick up a few test lines from commit b9b408c48724 to verify that
pg_upgrade is okay with these arrangements.
Co-authored-by: Justin Pryzby <[email protected]>
Co-authored-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/test/regress/expected/sanity_check.out
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Preserve firing-on state when cloning row triggers to partitions
commit : eef92de11e50837e4a0d02fc7269fdba7c97e583
author : Alvaro Herrera <[email protected]>
date : Fri, 16 Jul 2021 13:01:43 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 16 Jul 2021 13:01:43 -0400
When triggers are cloned from partitioned tables to their partitions,
the 'tgenabled' flag (origin/replica/always/disable) was not propagated.
Make it so that the flag on the trigger on partition is initially set to
the same value as on the partitioned table.
Add a test case to verify the behavior.
Backpatch to 11, where this appeared in commit 86f575948c77.
Author: Álvaro Herrera <[email protected]>
Reported-by: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/include/commands/trigger.h
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Advance old-segment horizon properly after slot invalidation
commit : e5bcbb10707b844471c67d5e5fd8226d1891ee97
author : Alvaro Herrera <[email protected]>
date : Fri, 16 Jul 2021 12:07:30 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 16 Jul 2021 12:07:30 -0400
When some slots are invalidated due to the max_slot_wal_keep_size limit,
the old segment horizon should move forward to stay within the limit.
However, in commit c6550776394e we forgot to call KeepLogSeg again to
recompute the horizon after invalidating replication slots. In cases
where other slots remained, the limits would be recomputed eventually
for other reasons, but if all slots were invalidated, the limits would
not move at all afterwards. Repair.
Backpatch to 13 where the feature was introduced.
Author: Kyotaro Horiguchi <[email protected]>
Reported-by: Marcin Krupowicz <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
M src/backend/replication/slot.c
M src/include/replication/slot.h
M src/test/recovery/t/019_replslot_limit.pl
doc: Spell checking
commit : 525115a332af0fd479503550fd4f82551b797aea
author : Peter Eisentraut <[email protected]>
date : Fri, 16 Jul 2021 10:35:38 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 16 Jul 2021 10:35:38 +0200
M doc/src/sgml/bki.sgml
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/json.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/pgstatstatements.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/release-14.sgml
M doc/src/sgml/wal.sgml
docs: fix inconsistencies in markup and case
commit : 9ffe128a6d616675b559a14a89f8b42a30084323
author : Daniel Gustafsson <[email protected]>
date : Thu, 15 Jul 2021 23:22:59 +0200
committer: Daniel Gustafsson <[email protected]>
date : Thu, 15 Jul 2021 23:22:59 +0200
Ensure to properly mark up function parameters in text with <parameter>,
avoid using <acronym> for terms which aren't acronyms and properly place
the ", and" in a value list. The acronym removal is a follow-up to commit
fb72a7b8c3 which removed it for minmax-multi. In passing, also fix an
incorrectly cased word.
Author: Ekaterina Kiryanova <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: v14
M doc/src/sgml/amcheck.sgml
M doc/src/sgml/brin.sgml
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
Ensure HAVE_DECL_XXX macros in MSVC builds match those in Unix.
commit : 081e86bd9e7e00af3cbf9bcdec8277f02d22e6b4
author : Tom Lane <[email protected]>
date : Thu, 15 Jul 2021 11:00:43 -0400
committer: Tom Lane <[email protected]>
date : Thu, 15 Jul 2021 11:00:43 -0400
Autoconf's AC_CHECK_DECLS() always defines HAVE_DECL_whatever
as 1 or 0, but some of the entries in msvc/Solution.pm showed
such symbols as "undef" instead of 0. Fix that for consistency.
There's no live bug in current usages AFAICS, but it's not hard
to imagine one creeping in if more-complex #if tests get added.
Back-patch to v13, which is as far back as Solution.pm contains
this data. The inconsistency still exists in the manually-filled
pg_config_ext.h.win32 files of older branches; but as long as the
problem is only latent, it doesn't seem worth the trouble to
clean things up there.
Discussion: https://postgr.es/m/[email protected]
M src/tools/msvc/Solution.pm
Fix small inconsistencies in catalog definition of multirange operators
commit : 4d39d4e639b44503c9950f61ec882113385db7ec
author : Alexander Korotkov <[email protected]>
date : Thu, 15 Jul 2021 14:18:19 +0300
committer: Alexander Korotkov <[email protected]>
date : Thu, 15 Jul 2021 14:18:19 +0300
This commit fixes the description of a couple of multirange operators and
oprjoin for another multirange operator. The change of oprjoin is more
cosmetic since both old and new functions return the same constant.
These cosmetic changes don't worth catalog incompatibility between 14beta2
and 14beta3. So, catversion isn't bumped.
Discussion: https://postgr.es/m/CAPpHfdv9OZEuZDqOQoUKpXhq%3Dmc-qa4gKCPmcgG5Vvesu7%3Ds1w%40mail.gmail.com
Backpatch-throgh: 14
M src/include/catalog/pg_operator.dat
Remove unnecessary assertion in postmaster.c
commit : b90063511a3ee41d037b2da0d4e0b3bc399140c5
author : Michael Paquier <[email protected]>
date : Thu, 15 Jul 2021 15:00:52 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 15 Jul 2021 15:00:52 +0900
A code path asserted that the archiver was dead, but a check made that
impossible to happen.
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/CALj2ACW=CYE1ars+2XyPTEPq0wQvru4c0dPZ=Nrn3EqNBkksvQ@mail.gmail.com
Backpatch-throgh: 14
M src/backend/postmaster/postmaster.c
Clarify description of pg_stat_statements columns
commit : 3b57d5af7435d0d21bdec392dc1ce7c13871df8b
author : Magnus Hagander <[email protected]>
date : Wed, 14 Jul 2021 11:04:15 +0200
committer: Magnus Hagander <[email protected]>
date : Wed, 14 Jul 2021 11:04:15 +0200
Reported-By: Peter Eisentraut
Backpatch-through: 14
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/pgstatstatements.sgml
Fix unexpected error messages for various flavors of ALTER TABLE
commit : 0c83eb2e0edb42f54a37fdeac85fb80eb5de2cca
author : Michael Paquier <[email protected]>
date : Wed, 14 Jul 2021 17:15:01 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 14 Jul 2021 17:15:01 +0900
Some commands of ALTER TABLE could fail with the following error:
ERROR: "tab" is of the wrong type
This error is unexpected, as all the code paths leading to
ATWrongRelkindError() should use a supported set of relkinds to generate
correct error messages. This commit closes the gap with such mistakes,
by adding all the missing relkind combinations. Tests are added to
check all the problems found. Note that some combinations are not used,
but these are left around as it could have an impact on applications
relying on this code.
2ed532e has done a much larger refactoring on HEAD to make such error
messages easier to manage in the long-term, so nothing is needed there.
Author: Kyotaro Horiguchi
Reviewed-by: Peter Eisentraut, Ahsan Hadi, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/foreign_data.out
M src/test/regress/sql/alter_table.sql
M src/test/regress/sql/foreign_data.sql
Fix lack of message pluralization
commit : b4842a8d085e970ff24667954c0d861e911e16c1
author : Peter Eisentraut <[email protected]>
date : Wed, 14 Jul 2021 09:15:14 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 14 Jul 2021 09:15:14 +0200
M src/backend/storage/ipc/signalfuncs.c
Change the name of the Result Cache node to Memoize
commit : 47ca4836441d1c24f75a94d43af8bd72d4c8d057
author : David Rowley <[email protected]>
date : Wed, 14 Jul 2021 12:45:00 +1200
committer: David Rowley <[email protected]>
date : Wed, 14 Jul 2021 12:45:00 +1200
"Result Cache" was never a great name for this node, but nobody managed
to come up with another name that anyone liked enough. That was until
David Johnston mentioned "Node Memoization", which Tom Lane revised to
just "Memoize". People seem to like "Memoize", so let's do the rename.
Reviewed-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 14, where Result Cache was introduced
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M doc/src/sgml/release-14.sgml
M src/backend/commands/explain.c
M src/backend/executor/Makefile
M src/backend/executor/execAmi.c
M src/backend/executor/execParallel.c
M src/backend/executor/execProcnode.c
R065 src/backend/executor/nodeResultCache.c src/backend/executor/nodeMemoize.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/README
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/pathnode.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
A src/include/executor/nodeMemoize.h
D src/include/executor/nodeResultCache.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/cost.h
M src/include/optimizer/pathnode.h
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/join.out
R088 src/test/regress/expected/resultcache.out src/test/regress/expected/memoize.out
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/sysviews.out
M src/test/regress/parallel_schedule
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/join.sql
R088 src/test/regress/sql/resultcache.sql src/test/regress/sql/memoize.sql
M src/test/regress/sql/partition_prune.sql
M src/tools/pgindent/typedefs.list
Rename debug_invalidate_system_caches_always to debug_discard_caches.
commit : 6201fa3c166fe2383dd44a9dd5082bc748c2937a
author : Tom Lane <[email protected]>
date : Tue, 13 Jul 2021 15:01:01 -0400
committer: Tom Lane <[email protected]>
date : Tue, 13 Jul 2021 15:01:01 -0400
The name introduced by commit 4656e3d66 was agreed to be unreasonably
long. To match this change, rename initdb's recently-added
--clobber-cache option to --discard-caches.
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/regress.sgml
M doc/src/sgml/release-14.sgml
M src/backend/utils/adt/lockfuncs.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/misc/guc.c
M src/bin/initdb/initdb.c
M src/include/pg_config_manual.h
M src/include/utils/inval.h
M src/pl/plpgsql/src/expected/plpgsql_cache.out
M src/pl/plpgsql/src/expected/plpgsql_cache_1.out
M src/pl/plpgsql/src/expected/plpgsql_record.out
M src/pl/plpgsql/src/sql/plpgsql_cache.sql
M src/pl/plpgsql/src/sql/plpgsql_record.sql
M src/test/isolation/specs/deadlock-soft-2.spec
Robustify tuplesort's free_sort_tuple function
commit : a92709fed63428a711f36270a2b4bee039399d60
author : David Rowley <[email protected]>
date : Tue, 13 Jul 2021 13:27:44 +1200
committer: David Rowley <[email protected]>
date : Tue, 13 Jul 2021 13:27:44 +1200
41469253e went to the trouble of removing a theoretical bug from
free_sort_tuple by checking if the tuple was NULL before freeing it. Let's
make this a little more robust by also setting the tuple to NULL so that
should we be called again we won't end up doing a pfree on the already
pfree'd tuple. Per advice from Tom Lane.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6, same as 41469253e
M src/backend/utils/sort/tuplesort.c
Fix theoretical bug in tuplesort
commit : a3b8d91ccd7964e5bec1066c9e598277294efd33
author : David Rowley <[email protected]>
date : Tue, 13 Jul 2021 12:42:04 +1200
committer: David Rowley <[email protected]>
date : Tue, 13 Jul 2021 12:42:04 +1200
This fixes a theoretical bug in tuplesort.c which, if a bounded sort was
used in combination with a byval Datum sort (tuplesort_begin_datum), when
switching the sort to a bounded heap in make_bounded_heap(), we'd call
free_sort_tuple(). The problem was that when sorting Datums of a byval
type, the tuple is NULL and free_sort_tuple() would free the memory for it
regardless of that. This would result in a crash.
Here we fix that simply by adding a check to see if the tuple is NULL
before trying to disassociate and free any memory belonging to it.
The reason this bug is only theoretical is that nowhere in the current
code base do we do tuplesort_set_bound() when performing a Datum sort.
However, let's backpatch a fix for this as if any extension uses the code
in this way then it's likely to cause problems.
Author: Ronan Dunklau
Discussion: https://postgr.es/m/CAApHDvpdoqNC5FjDb3KUTSMs5dg6f+XxH4Bg_dVcLi8UYAG3EQ@mail.gmail.com
Backpatch-through: 9.6, oldest supported version
M src/backend/utils/sort/tuplesort.c
Probe for preadv/pwritev in a more macOS-friendly way.
commit : e75da4f1b6c63523021e550f1a92c67d2b8a92fb
author : Tom Lane <[email protected]>
date : Mon, 12 Jul 2021 19:17:35 -0400
committer: Tom Lane <[email protected]>
date : Mon, 12 Jul 2021 19:17:35 -0400
Apple's mechanism for dealing with functions that are available
in only some OS versions confuses AC_CHECK_FUNCS, and therefore
AC_REPLACE_FUNCS. We can use AC_CHECK_DECLS instead, so long as
we enable -Werror=unguarded-availability-new. This allows people
compiling for macOS to control whether or not preadv/pwritev are
used by setting MACOSX_DEPLOYMENT_TARGET, rather than supplying
a back-rev SDK. (Of course, the latter still works, too.)
James Hilliard
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
M src/include/pg_config.h.in
M src/include/port/pg_iovec.h
M src/tools/msvc/Solution.pm
doc: Fix typo in function prototype
commit : 9b450b1f1e5bbeac414bbb8e6a2f08fa40f470ff
author : Peter Eisentraut <[email protected]>
date : Mon, 12 Jul 2021 22:07:35 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 12 Jul 2021 22:07:35 +0200
M doc/src/sgml/indexam.sgml
Remove dead assignment to local variable.
commit : 233280803cb6b3fe6b905abc1a28913855f40dd6
author : Heikki Linnakangas <[email protected]>
date : Mon, 12 Jul 2021 11:13:33 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 12 Jul 2021 11:13:33 +0300
This should have been removed in commit 7e30c186da, which split the loop
into two. Only the first loop uses the 'from' variable; updating it in
the second loop is bogus. It was never read after the first loop, so this
was harmless and surely optimized away by the compiler, but let's be tidy.
Backpatch to all supported versions.
Author: Ranier Vilela
Discussion: https://www.postgresql.org/message-id/CAEudQAoWq%2BAL3BnELHu7gms2GN07k-np6yLbukGaxJ1vY-zeiQ%40mail.gmail.com
M src/backend/access/nbtree/nbtxlog.c
doc: Make release note entries more accurate
commit : 166fcf4a6364769501dd8562db927abefdd310b2
author : Peter Eisentraut <[email protected]>
date : Mon, 12 Jul 2021 08:45:34 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 12 Jul 2021 08:45:34 +0200
Two tsquery-related release note entries had inaccuracies in the
before and after examples. Clean that up.
M doc/src/sgml/release-14.sgml
Revert "Fix issues with Windows' stat() for files pending on deletion"
commit : 5e60237ad1b4e2d159aec48ffd9914a633d2d6e0
author : Michael Paquier <[email protected]>
date : Mon, 12 Jul 2021 14:46:40 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 12 Jul 2021 14:46:40 +0900
This reverts commit 54fb8c7, as per the issues reported by fairywren
when it comes to MinGW because of the lack of microsoft_native_stat()
there. Using just stat() for MSVC is not sufficient to take care of the
concurrency problems with files pending on deletion. It may be possible
to paint some __MINGW64__ in the code to switch to a different
implementation of stat() in this build context, but I am not sure either
if relying on the implementation of stat() in MinGW to take care of the
problems we are trying to fix is enough or not. So this needs more
study.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 14
M src/port/open.c
M src/port/win32stat.c
Fix issues with Windows' stat() for files pending on deletion
commit : de1510e2f5a1826890b206253016ebfc592c2f0a
author : Michael Paquier <[email protected]>
date : Mon, 12 Jul 2021 13:02:45 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 12 Jul 2021 13:02:45 +0900
The code introduced by bed9075 to enhance the stat() implementation on
Windows for file sizes larger than 4GB fails to properly detect files
pending for deletion with its method based on NtQueryInformationFile()
or GetFileInformationByHandleEx(), as proved by Alexander Lakhin in a
custom TAP test of his own.
The method used in the implementation of open() to sleep and loop when
when failing on ERROR_ACCESS_DENIED (EACCES) is showing much more
stability, so switch to this method. This could still lead to issues if
the permission problem stays around for much longer than the timeout of
1 second used, but that should (hopefully) never happen in
performance-critical paths. Still, there could be a point in increasing
the timeouts for the sake of machines that handle heavy loads.
Note that WIN32's open() now uses microsoft_native_stat() as it should
be similar to stat() when working around issues with concurrent file
deletions.
I have spent some time testing this patch with pgbench in combination
of the SQL functions from genfile.c, as well as running the TAP test
provided on the thread with MSVC builds, and this looks much more
stable than the previous method.
Author: Alexander Lakhin
Reviewed-by: Tom Lane, Michael Paquier, Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 14
M src/port/open.c
M src/port/win32stat.c
Lock the extension during ALTER EXTENSION ADD/DROP.
commit : 69dfc36fd54d1620209d5d40f3c68006d96b0046
author : Tom Lane <[email protected]>
date : Sun, 11 Jul 2021 12:54:24 -0400
committer: Tom Lane <[email protected]>
date : Sun, 11 Jul 2021 12:54:24 -0400
Although we were careful to lock the object being added or dropped,
we failed to get any sort of lock on the extension itself. This
allowed the ALTER to proceed in parallel with a DROP EXTENSION,
which is problematic for a couple of reasons. If both commands
succeeded we'd be left with a dangling link in pg_depend, which
would cause problems later. Also, if the ALTER failed for some
reason, it might try to print the extension's name, and that could
result in a crash or (in older branches) a silly error message
complaining about extension "(null)".
Per bug #17098 from Alexander Lakhin. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/extension.c
Fix pgbench timestamp bugs.
commit : 5614a0f78eaaa51c07bd753ca854de238c730b84
author : Thomas Munro <[email protected]>
date : Sun, 11 Jul 2021 19:50:55 +1200
committer: Thomas Munro <[email protected]>
date : Sun, 11 Jul 2021 19:50:55 +1200
Commit 547f04e changed pgbench to use microsecond accounting, but
introduced a couple of logging and aggregation bugs:
1. We print Unix epoch timestamps so that you can correlate them with
other logs, but these were inadvertently changed to use a
system-dependent reference epoch. Compute Unix timestamps, and begin
aggregation intervals on the boundaries of whole Unix epoch seconds, as
before.
2. The user-supplied aggregation interval needed to be scaled.
Back-patch to 14.
Author: Fabien COELHO <[email protected]>
Author: Yugo NAGATA <[email protected]>
Author: Kyotaro Horiguchi <[email protected]>
Reported-by: YoungHwan Joo <[email protected]>
Reported-by: Gregory Smith <[email protected]>
Discussion: https://postgr.es/m/CAF7igB1r6wRfSCEAB-iZBKxkowWY6%2BdFF2jObSdd9%2BiVK%2BvHJg%40mail.gmail.com
Discussion: https://postgr.es/m/CAHLJuCW_8Vpcr0%3Dt6O_gozrg3wXXWXZXDioYJd3NhvKriqgpfQ%40mail.gmail.com
M src/bin/pgbench/pgbench.c
Fix assign_record_type_typmod().
commit : 10a07973cf72b32f32f0e05ada16c84580496d93
author : Jeff Davis <[email protected]>
date : Fri, 9 Jul 2021 14:15:48 -0700
committer: Jeff Davis <[email protected]>
date : Fri, 9 Jul 2021 14:15:48 -0700
If an error occurred in the wrong place, it was possible to leave an
unintialized entry in the hash table, leading to a crash. Fixed.
Also, be more careful about the order of operations so that an
allocation error doesn't leak memory in CacheMemoryContext or
unnecessarily advance NextRecordTypmod.
Backpatch through version 11. Earlier versions (prior to 35ea75632a5)
do not exhibit the problem, because an uninitialized hash entry
contains a valid empty list.
Author: Sait Talha Nisanci <[email protected]>
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/HE1PR8303MB009069D476225B9A9E194B8891779@HE1PR8303MB0090.EURPRD83.prod.outlook.com
Backpatch-through: 11
M src/backend/utils/cache/typcache.c
Fix busted test for ldap_initialize.
commit : ebc346e5bb72b2a68258f1be8791f2b90191cd38
author : Tom Lane <[email protected]>
date : Sat, 10 Jul 2021 13:19:31 -0400
committer: Tom Lane <[email protected]>
date : Sat, 10 Jul 2021 13:19:31 -0400
Sigh ... I was expecting AC_CHECK_LIB to do something it didn't,
namely update LIBS. This led to not finding ldap_initialize.
Fix by moving the probe for ldap_initialize. In some sense this
is more correct anyway, since (at least for now) we care about
whether ldap_initialize exists in libldap not libldap_r.
Per buildfarm member elver and local testing.
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
Fix numeric_mul() overflow due to too many digits after decimal point.
commit : 06883d58ff29cf4fb8c32fd13ce9947796b9fb0f
author : Dean Rasheed <[email protected]>
date : Sat, 10 Jul 2021 12:45:00 +0100
committer: Dean Rasheed <[email protected]>
date : Sat, 10 Jul 2021 12:45:00 +0100
This fixes an overflow error when using the numeric * operator if the
result has more than 16383 digits after the decimal point by rounding
the result. Overflow errors should only occur if the result has too
many digits *before* the decimal point.
Discussion: https://postgr.es/m/CAEZATCUmeFWCrq2dNzZpRj5+6LfN85jYiDoqm+ucSXhb9U2TbA@mail.gmail.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Un-break AIX build, take 2.
commit : 9ffad7ae7ffc632ff8d3822916c431b7e4a4df38
author : Tom Lane <[email protected]>
date : Fri, 9 Jul 2021 16:59:07 -0400
committer: Tom Lane <[email protected]>
date : Fri, 9 Jul 2021 16:59:07 -0400
I incorrectly diagnosed the reason why hoverfly is unhappy.
Looking closer, it appears that it fails to link libldap
unless libssl is also present; so the problem was my
idea of clearing LIBS before making the check. Revert
to essentially the original coding, except that instead
of failing when libldap_r isn't there, use libldap.
Per buildfarm member hoverfly.
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
libpq: Fix sending queries in pipeline aborted state
commit : 1beaa654da61ee66857a3c5125682b629f62fdf9
author : Alvaro Herrera <[email protected]>
date : Fri, 9 Jul 2021 15:57:59 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 9 Jul 2021 15:57:59 -0400
When sending queries in pipeline mode, we were careless about leaving
the connection in the right state so that PQgetResult would behave
correctly; trying to read further results after sending a query after
having read a result with an error would sometimes hang. Fix by
ensuring internal libpq state is changed properly. All the state
changes were being done by the callers of pqAppendCmdQueueEntry(); it
would have become too repetitious to have this logic in each of them, so
instead put it all in that function and relieve callers of the
responsibility.
Add a test to verify this case. Without the code fix, this new test
hangs sometimes.
Also, document that PQisBusy() would return false when no queries are
pending result. This is not intuitively obvious, and NULL would be
obtained by calling PQgetResult() at that point, which is confusing.
Wording by Boris Kolpackov.
In passing, fix bogus use of "false" to mean "0", per Ranier Vilela.
Backpatch to 14.
Author: Álvaro Herrera <[email protected]>
Reported-by: Boris Kolpackov <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-exec.c
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Un-break AIX build.
commit : 7f2eca6f9db6e2227b87d03204e201ee6d759b5b
author : Tom Lane <[email protected]>
date : Fri, 9 Jul 2021 14:15:41 -0400
committer: Tom Lane <[email protected]>
date : Fri, 9 Jul 2021 14:15:41 -0400
In commit d0a02bdb8, I'd supposed that uniformly probing for
ldap_bind would make the intent clearer. However, that seems
not to work on AIX, for obscure reasons (maybe it's a macro
there?). Revert to the former behavior of probing
ldap_simple_bind for thread-safe cases and ldap_bind otherwise.
Per buildfarm member hoverfly.
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
Avoid creating a RESULT RTE that's marked LATERAL.
commit : 1d98fdaed89c00465ef68fa2804967ea27b03abc
author : Tom Lane <[email protected]>
date : Fri, 9 Jul 2021 13:38:24 -0400
committer: Tom Lane <[email protected]>
date : Fri, 9 Jul 2021 13:38:24 -0400
Commit 7266d0997 added code to pull up simple constant function
results, converting the RTE_FUNCTION RTE to a dummy RTE_RESULT
RTE since it no longer need be scanned. But I forgot to clear
the LATERAL flag if the RTE has it set. If the function reduced
to a constant, it surely contains no lateral references so this
simplification is logically OK. It's needed because various other
places will Assert that RESULT RTEs aren't LATERAL.
Per bug #17097 from Yaoguang Chen. Back-patch to v13 where the
faulty code came in.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Update configure's probe for libldap to work with OpenLDAP 2.5.
commit : 5620ec83362d08b9f86c90c97c0a70031c4d0b2c
author : Tom Lane <[email protected]>
date : Fri, 9 Jul 2021 12:38:55 -0400
committer: Tom Lane <[email protected]>
date : Fri, 9 Jul 2021 12:38:55 -0400
The separate libldap_r is gone and libldap itself is now always
thread-safe. Unfortunately there seems no easy way to tell by
inspection whether libldap is thread-safe, so we have to take
it on faith that libldap is thread-safe if there's no libldap_r.
That should be okay, as it appears that libldap_r was a standard
part of the installation going back at least 20 years.
Report and patch by Adrian Ho. Back-patch to all supported
branches, since people might try to build any of them with
a newer OpenLDAP.
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
M src/include/pg_config.h.in
M src/tools/msvc/Solution.pm
Reject cases where a query in WITH rewrites to just NOTIFY.
commit : 39b6e85f135f7a1dcf43c0551d7d10e8c57b7fce
author : Tom Lane <[email protected]>
date : Fri, 9 Jul 2021 11:02:26 -0400
committer: Tom Lane <[email protected]>
date : Fri, 9 Jul 2021 11:02:26 -0400
Since the executor can't cope with a utility statement appearing
as a node of a plan tree, we can't support cases where a rewrite
rule inserts a NOTIFY into an INSERT/UPDATE/DELETE command appearing
in a WITH clause of a larger query. (One can imagine ways around
that, but it'd be a new feature not a bug fix, and so far there's
been no demand for it.) RewriteQuery checked for this, but it
missed the case where the DML command rewrites to *only* a NOTIFY.
That'd lead to crashes later on in planning. Add the missed check,
and improve the level of testing of this area.
Per bug #17094 from Yaoguang Chen. It's been busted since WITH
was introduced, so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Remove more obsolete comments about semaphores.
commit : 8d48a3436dd83f7d6e3cde2d76f91f49f27ba16d
author : Thomas Munro <[email protected]>
date : Fri, 9 Jul 2021 17:51:48 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 9 Jul 2021 17:51:48 +1200
Commit 6753333f stopped using semaphores as the sleep/wake mechanism for
heavyweight locks, but some obsolete references to that scheme remained
in comments. As with similar commit 25b93a29, back-patch all the way.
Reviewed-by: Daniel Gustafsson <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGLafjB1uzXcy%3D%3D2L3cy7rjHkqOVn7qRYGBjk%3D%3DtMJE7Yg%40mail.gmail.com
M src/backend/storage/lmgr/lwlock.c
Fix incorrect return value in pg_size_pretty(bigint)
commit : 6de3a21bbc0601b22cd465311d10978acca7cc0b
author : David Rowley <[email protected]>
date : Fri, 9 Jul 2021 14:04:40 +1200
committer: David Rowley <[email protected]>
date : Fri, 9 Jul 2021 14:04:40 +1200
Due to how pg_size_pretty(bigint) was implemented, it's possible that when
given a negative number of bytes that the returning value would not match
the equivalent positive return value when given the equivalent positive
number of bytes. This was due to two separate issues.
1. The function used bit shifting to convert the number of bytes into
larger units. The rounding performed by bit shifting is not the same as
dividing. For example -3 >> 1 = -2, but -3 / 2 = -1. These two
operations are only equivalent with positive numbers.
2. The half_rounded() macro rounded towards positive infinity. This meant
that negative numbers rounded towards zero and positive numbers rounded
away from zero.
Here we fix #1 by dividing the values instead of bit shifting. We fix #2
by adjusting the half_rounded macro always to round away from zero.
Additionally, adjust the pg_size_pretty(numeric) function to be more
explicit that it's using division rather than bit shifting. A casual
observer might have believed bit shifting was used due to a static
function being named numeric_shift_right. However, that function was
calculating the divisor from the number of bits and performed division.
Here we make that more clear. This change is just cosmetic and does not
affect the return value of the numeric version of the function.
Here we also add a set of regression tests both versions of
pg_size_pretty() which test the values directly before and after the
function switches to the next unit.
This bug was introduced in 8a1fab36a. Prior to that negative values were
always displayed in bytes.
Author: Dean Rasheed, David Rowley
Discussion: https://postgr.es/m/CAEZATCXnNW4HsmZnxhfezR5FuiGgp+mkY4AzcL5eRGO4fuadWg@mail.gmail.com
Backpatch-through: 9.6, where the bug was introduced.
M src/backend/utils/adt/dbsize.c
M src/test/regress/expected/dbsize.out
M src/test/regress/sql/dbsize.sql
doc: minor PG 14 relnote changes
commit : 9893d5124b1a0a2d84fb8038bde3bf11491abf7a
author : Bruce Momjian <[email protected]>
date : Thu, 8 Jul 2021 18:59:19 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 8 Jul 2021 18:59:19 -0400
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
doc: PG 14 relnote, move system view items to the proper sect.
commit : 6f566f0e9b1aab41eea1ca1991e04e3d8a79c198
author : Bruce Momjian <[email protected]>
date : Thu, 8 Jul 2021 13:08:09 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 8 Jul 2021 13:08:09 -0400
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
doc: improve PG 14 relnote item about adding btree pages to FSM
commit : 049d3617a3bedb2a1d97d1621f36c861f52269a7
author : Bruce Momjian <[email protected]>
date : Thu, 8 Jul 2021 12:27:30 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 8 Jul 2021 12:27:30 -0400
Wording confirmed by Peter Geoghegan.
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
Fix crash in postgres_fdw for provably-empty remote UPDATE/DELETE.
commit : 30a35bca3f9306f87ca8db095ddac2ba62156934
author : Tom Lane <[email protected]>
date : Wed, 7 Jul 2021 15:21:25 -0400
committer: Tom Lane <[email protected]>
date : Wed, 7 Jul 2021 15:21:25 -0400
In 86dc90056, I'd written find_modifytable_subplan with the assumption
that if the immediate child of a ModifyTable is a Result, it must be
a projecting Result with a subplan. However, if the UPDATE or DELETE
has a provably-constant-false WHERE clause, that's not so: we'll
generate a dummy subplan with a childless Result. Add the missing
null-check so we don't crash on such cases.
Per report from Alexander Pyhalov.
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
doc: Fix description about pg_stat_statements.track_planning.
commit : e48f2afee631be42739e50fbefd758503e8dcf82
author : Fujii Masao <[email protected]>
date : Wed, 7 Jul 2021 21:54:47 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 7 Jul 2021 21:54:47 +0900
This commit fixes wrong wording like "a fewer kinds"
in the description about track_planning option.
Back-patch to v13 where pg_stat_statements.track_planning was added.
Author: Justin Pryzby
Reviewed-by: Julien Rouhaud, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/pgstatstatements.sgml
postgres_fdw: Tighten up allowed values for batch_size, fetch_size options.
commit : 4173477b3841749ce491c77b54a5940f6f3e9eb6
author : Fujii Masao <[email protected]>
date : Wed, 7 Jul 2021 11:13:40 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 7 Jul 2021 11:13:40 +0900
Previously the values such as '100$%$#$#', '9,223,372,' were accepted and
treated as valid integers for postgres_fdw options batch_size and fetch_size.
Whereas this is not the case with fdw_startup_cost and fdw_tuple_cost options
for which an error is thrown. This was because endptr was not used
while converting strings to integers using strtol.
This commit changes the logic so that it uses parse_int function
instead of strtol as it serves the purpose by returning false in case
if it is unable to convert the string to integer. Note that
this function also rounds off the values such as '100.456' to 100 and
'100.567' or '100.678' to 101.
While on this, use parse_real for fdw_startup_cost and fdw_tuple_cost options.
Since parse_int and parse_real are being used for reloptions and GUCs,
it is more appropriate to use in postgres_fdw rather than using strtol
and strtod directly.
Back-patch to v14.
Author: Bharath Rupireddy
Reviewed-by: Ashutosh Bapat, Tom Lane, Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVMO6wY5Pc4oe1OCgUOAtdjHuFsBDw8R5uoYR86eWFQDA@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Avoid doing catalog lookups in postgres_fdw's conversion_error_callback.
commit : 86d4914210e9e7f8f8c32defacc71744d0222feb
author : Tom Lane <[email protected]>
date : Tue, 6 Jul 2021 12:36:12 -0400
committer: Tom Lane <[email protected]>
date : Tue, 6 Jul 2021 12:36:12 -0400
As in 50371df26, this is a bad idea since the callback can't really
know what error is being thrown and thus whether or not it is safe
to attempt catalog accesses. Rather than pushing said accesses into
the mainline code where they'd usually be a waste of cycles, we can
look at the query's rangetable instead.
This change does mean that we'll be printing query aliases (if any
were used) rather than the table or column's true name. But that
doesn't seem like a bad thing: it's certainly a more useful definition
in self-join cases, for instance. In any case, it seems unlikely that
any applications would be depending on this detail, so it seems safe
to change.
Patch by me. Original complaint by Andres Freund; Bharath Rupireddy
noted the connection to conversion_error_callback.
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
Doc: add info about timestamps with fractional-minute UTC offsets.
commit : 94911ec28f01d1babe61053977ca9f057983cb21
author : Tom Lane <[email protected]>
date : Tue, 6 Jul 2021 10:34:51 -0400
committer: Tom Lane <[email protected]>
date : Tue, 6 Jul 2021 10:34:51 -0400
Our code has supported fractional-minute UTC offsets for ages, but
there was no mention of the possibility in the main docs, and only
a very indirect reference in Appendix B. Improve that.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/datatype.sgml
Reduce overhead of cache-clobber testing in LookupOpclassInfo().
commit : 07f1e06964adeec6d79bb24f3b1d50666694213b
author : Tom Lane <[email protected]>
date : Mon, 5 Jul 2021 16:51:57 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Jul 2021 16:51:57 -0400
Commit 03ffc4d6d added logic to bypass all caching behavior in
LookupOpclassInfo when CLOBBER_CACHE_ALWAYS is enabled. It doesn't
look like I stopped to think much about what that would cost, but
recent investigation shows that the cost is enormous: it roughly
doubles the time needed for cache-clobber test runs.
There does seem to be value in this behavior when trying to test
the opclass-cache loading logic itself, but for other purposes the
cost is excessive. Hence, let's back off to doing this only when
debug_invalidate_system_caches_always is at least 3; or in older
branches, when CLOBBER_CACHE_RECURSIVELY is defined.
While here, clean up some other minor issues in LookupOpclassInfo.
Re-order the code so we aren't left with broken cache entries (leading
to later core dumps) in the unlikely case that we suffer OOM while
trying to allocate space for a new entry. (That seems to be my
oversight in 03ffc4d6d.) Also, in >= v13, stop allocating one array
entry too many. That's evidently left over from sloppy reversion in
851b14b0c.
Back-patch to all supported branches, mainly to reduce the runtime
of cache-clobbering buildfarm animals.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/cache/relcache.c
Rethink blocking annotations in detach-partition-concurrently-[34].
commit : 9fa6fe466c9d2eaba4fdd8091203ee61e74d71bf
author : Tom Lane <[email protected]>
date : Mon, 5 Jul 2021 14:34:47 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Jul 2021 14:34:47 -0400
In 741d7f104, I tried to make the reports from canceled steps come out
after the pg_cancel_backend() steps, since that was the most common
ordering before. However, that doesn't ensure that a canceled step
doesn't report even later, as shown in a recent failure on buildfarm
member idiacanthus. Rather than complicating things even more with
additional annotations, let's just force the cancel's effect to be
reported first. It's not *that* unnatural-looking.
Back-patch to v14 where these test cases appeared.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=idiacanthus&dt=2021-07-02%2001%3A40%3A04
M src/test/isolation/expected/detach-partition-concurrently-3.out
M src/test/isolation/expected/detach-partition-concurrently-4.out
M src/test/isolation/specs/detach-partition-concurrently-3.spec
M src/test/isolation/specs/detach-partition-concurrently-4.spec
doc: Fix quoting markup
commit : 1479c5afdce2c57928dc1322bd71106858a0770f
author : Peter Eisentraut <[email protected]>
date : Mon, 5 Jul 2021 08:26:00 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 5 Jul 2021 08:26:00 +0200
M doc/src/sgml/appendix-obsolete-default-roles.sgml
Doc: Hash Indexes.
commit : f3690fbdeac43ab8547c212062ec515a55dd118c
author : Amit Kapila <[email protected]>
date : Mon, 5 Jul 2021 09:38:17 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 5 Jul 2021 09:38:17 +0530
A new chapter for Hash Indexes, designed to help users understand how
they work and when to use them.
Backpatch-through 10 where we have made hash indexes durable.
Author: Simon Riggs
Reviewed-By: Justin Pryzby, Amit Kapila
Discussion: https://postgr.es/m/CANbhV-HRjNPYgHo--P1ewBrFJ-GpZPb9_25P7=Wgu7s7hy_sLQ@mail.gmail.com
M doc/src/sgml/filelist.sgml
A doc/src/sgml/hash.sgml
M doc/src/sgml/postgres.sgml
doc: Mention requirement to --enable-tap-tests on section for TAP tests
commit : 95e2f925f1443f49b31e861a8d66dabca5528aaa
author : Michael Paquier <[email protected]>
date : Sun, 4 Jul 2021 20:59:10 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 4 Jul 2021 20:59:10 +0900
Author: Greg Sabino Mullane
Discussion: https://postgr.es/m/CAKAnmmJYH2FBn_+Vwd2FD5SaKn8hjhAXOCHpZc6n4wXaUaW_SA@mail.gmail.com
Backpatch-through: 9.6
M doc/src/sgml/regress.sgml
Doc: mention that VACUUM can't utilize over 1GB of RAM
commit : b88e5fe60eef5decd3ccfd73566559dc6136cb61
author : David Rowley <[email protected]>
date : Sun, 4 Jul 2021 22:29:16 +1200
committer: David Rowley <[email protected]>
date : Sun, 4 Jul 2021 22:29:16 +1200
Document that setting maintenance_work_mem to values over 1GB has no
effect on VACUUM.
Reported-by: Martín Marqués
Author: Laurenz Albe
Discussion: https://postgr.es/m/CABeG9LsZ2ozUMcqtqWu_-GiFKB17ih3p8wBHXcpfnHqhCnsc7A%40mail.gmail.com
Backpatch-through: 9.6, oldest supported release
M doc/src/sgml/config.sgml
doc: adjust "cities" example to be consistent with other SQL
commit : e033e260b9d0fcba969e5314be967429ade1b9ec
author : Bruce Momjian <[email protected]>
date : Fri, 2 Jul 2021 20:42:46 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 2 Jul 2021 20:42:46 -0400
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M doc/src/sgml/advanced.sgml
docs: clarify new aggressive vacuum mode for multi-xacts
commit : d54092955690e29656186ff5d4d315a3c56ccb37
author : Bruce Momjian <[email protected]>
date : Fri, 2 Jul 2021 18:00:30 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 2 Jul 2021 18:00:30 -0400
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 14
M doc/src/sgml/maintenance.sgml
Don't try to print data type names in slot_store_error_callback().
commit : 63a952167092bd2d457691202d71ace943e1b06a
author : Tom Lane <[email protected]>
date : Fri, 2 Jul 2021 16:04:54 -0400
committer: Tom Lane <[email protected]>
date : Fri, 2 Jul 2021 16:04:54 -0400
The existing code tried to do syscache lookups in an already-failed
transaction, which is problematic to say the least. After some
consideration of alternatives, the best fix seems to be to just drop
type names from the error message altogether. The table and column
names seem like sufficient localization. If the user is unsure what
types are involved, she can check the local and remote table
definitions.
Having done that, we can also discard the LogicalRepTypMap hash
table, which had no other use. Arguably, LOGICAL_REP_MSG_TYPE
replication messages are now obsolete as well; but we should
probably keep them in case some other use emerges. (The complexity
of removing something from the replication protocol would likely
outweigh any savings anyhow.)
Masahiko Sawada and Bharath Rupireddy, per complaint from Andres
Freund. Back-patch to v10 where this code originated.
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/relation.c
M src/backend/replication/logical/worker.c
M src/include/replication/logicalrelation.h
doc: PG 14 relnotes, mention CONCURRENTLY improvements
commit : 56d467a07de1c203cfe78fea69cd8326fa297e23
author : Bruce Momjian <[email protected]>
date : Fri, 2 Jul 2021 14:46:31 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 2 Jul 2021 14:46:31 -0400
Add items for vacuum not having to wait for CONCURRENTLY, and
CONCURRENTLY not having to wait for other CONCURRENTLY operations.
Reported-by: Simon Riggs
Discussion: https://postgr.es/m/CANbhV-EMM4nf7Ys-Yae_kY25dXT_3eiOXke2+yw44jgy+4jNsA@mail.gmail.com
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
doc: update PG 14 release notes
commit : a3a681f8590d9476c8ce94172316d3d339375fe6
author : Bruce Momjian <[email protected]>
date : Thu, 1 Jul 2021 20:33:32 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 1 Jul 2021 20:33:32 -0400
Mostly addition of <literal> tags
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
doc: Remove inappropriate <acronym> tags
commit : fb72a7b8c384e25d752b22a69e77d8227af65b4b
author : Peter Eisentraut <[email protected]>
date : Thu, 1 Jul 2021 22:23:37 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 1 Jul 2021 22:23:37 +0200
M doc/src/sgml/brin.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/release-14.sgml
add missing tag from commit b8c4261e5e
commit : 1da2ea0cccc7f9ce5eed3d6e62cba785e82a6a0d
author : Andrew Dunstan <[email protected]>
date : Thu, 1 Jul 2021 15:38:06 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 1 Jul 2021 15:38:06 -0400
M GNUmakefile.in
M doc/src/sgml/installation.sgml
doc: Clean up title case use
commit : 46bbe5d777722938496f177706cc87ff64512def
author : Peter Eisentraut <[email protected]>
date : Thu, 1 Jul 2021 21:27:39 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 1 Jul 2021 21:27:39 +0200
M doc/src/sgml/appendix-obsolete-default-roles.sgml
M doc/src/sgml/btree.sgml
M doc/src/sgml/logicaldecoding.sgml
Add new make targets world-bin and install-world-bin
commit : 100e9ae53f210c9e82671f1cc554aca945c4c180
author : Andrew Dunstan <[email protected]>
date : Thu, 1 Jul 2021 14:21:09 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 1 Jul 2021 14:21:09 -0400
These are the same as world and install-world respectively, but without
building or installing the documentation. There are many reasons for
wanting to be able to do this, including speed, lack of documentation
building tools, and wanting to build other formats of the documentation.
Plans for simplifying the buildfarm client code include using these
targets.
Backpatch to all live branches.
Discussion: https://postgr.es/m/[email protected]
M GNUmakefile.in
M doc/src/sgml/installation.sgml
Add --clobber-cache option to initdb, for CCA testing.
commit : d0477080174b227e6f5cbe0dd10c4d79abb7f3e6
author : Tom Lane <[email protected]>
date : Thu, 1 Jul 2021 13:33:05 -0400
committer: Tom Lane <[email protected]>
date : Thu, 1 Jul 2021 13:33:05 -0400
Commit 4656e3d66 replaced the "#define CLOBBER_CACHE_ALWAYS"
testing mechanism with a GUC, which has been a great help for
doing cache-clobber testing in more efficient ways; but there
is a gap in the implementation. The only way to do cache-clobber
testing during an initdb run is to use the old method with #define,
because one can't set the GUC from outside. Improve this by
adding a switch to initdb for the purpose.
(Perhaps someday we should let initdb pass through arbitrary
"-c NAME=VALUE" switches. Quoting difficulties dissuaded me
from attempting that right now, though.)
Back-patch to v14 where 4656e3d66 came in.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/initdb.sgml
M src/bin/initdb/initdb.c
Don't reset relhasindex for partitioned tables on ANALYZE
commit : be280cdad2985749e558212b0a5c8bdf9abb4e6a
author : Alvaro Herrera <[email protected]>
date : Thu, 1 Jul 2021 12:56:30 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 1 Jul 2021 12:56:30 -0400
Commit 0e69f705cc1a introduced code to analyze partitioned table;
however, that code fails to preserve pg_class.relhasindex correctly.
Fix by observing whether any indexes exist rather than accidentally
falling through to assuming none do.
Backpatch to 14.
Author: Alexander Pyhalov <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Reviewed-by: Zhihong Yu <[email protected]>
Discussion: https://postgr.es/m/CALNJ-vS1R3Qoe5t4tbzxrkpBtzRbPq1dDcW4RmA_a+oqweF30w@mail.gmail.com
M src/backend/commands/analyze.c
M src/test/regress/expected/vacuum.out
M src/test/regress/sql/vacuum.sql
Fix prove_installcheck to use correct paths when used with PGXS
commit : c4774ce339beff4a8968ceecb86bbbfa3335ed2b
author : Andrew Dunstan <[email protected]>
date : Thu, 1 Jul 2021 08:29:10 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 1 Jul 2021 08:29:10 -0400
The prove_installcheck recipe in src/Makefile.global.in was emitting
bogus paths for a couple of elements when used with PGXS. Here we create
a separate recipe for the PGXS case that does it correctly. We also take
the opportunity to make the make the file more readable by breaking up
the prove_installcheck and prove_check recipes across several lines, and
to remove the setting for REGRESS_SHLIB to src/test/recovery/Makefile,
which is the only set of tests that actually need it.
Backpatch to all live branches
Discussion: https://postgr.es/m/[email protected]
M src/Makefile.global.in
M src/test/recovery/Makefile
Replace magic constants used in pg_stat_get_replication_slot().
commit : a9cb00a965c6ac0cd434733fae0948218702e501
author : Amit Kapila <[email protected]>
date : Wed, 30 Jun 2021 11:40:06 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 30 Jun 2021 11:40:06 +0530
A few variables have been using 10 as a magic constant while
PG_STAT_GET_REPLICATION_SLOT_COLS can be used instead.
Author: Masahiko Sawada
Reviewed-By: Amit Kapila
Backpatch-through: 14, where it was introduced
Discussion: https://postgr.es/m/CAD21AoBvqODDfmD17DkEuPCvV2KbruukXQ2Vwrv5Xi-TsAsTJA@mail.gmail.com
M src/backend/utils/adt/pgstatfuncs.c
Allow streaming the changes after speculative aborts.
commit : dfceed30abc191c097557357dd6db56e875bb7e1
author : Amit Kapila <[email protected]>
date : Wed, 30 Jun 2021 09:48:07 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 30 Jun 2021 09:48:07 +0530
Until now, we didn't allow to stream the changes in logical replication
till we receive speculative confirm or the next DML change record after
speculative inserts. The reason was that we never use to process
speculative aborts but after commit 4daa140a2f it is possible to process
them so we can allow streaming once we receive speculative abort after
speculative insertion.
We decided to backpatch to 14 where the feature for streaming in progress
transactions have been introduced as this is a minor change and makes that
functionality better.
Author: Amit Kapila
Reviewed-By: Dilip Kumar
Backpatch-through: 14
Discussion: https://postgr.es/m/CAA4eK1KdqmTCtrBR6oFfGELrLLbDLDedL6zACcsUOQuTJBj1vw@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
Fix incorrect PITR message for transaction ROLLBACK PREPARED
commit : 607a3a43bca573a02570f374fd98c82a0c2e12a1
author : Michael Paquier <[email protected]>
date : Wed, 30 Jun 2021 11:49:10 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 30 Jun 2021 11:49:10 +0900
Reaching PITR on such a transaction would cause the generation of a LOG
message mentioning a transaction committed, not aborted.
Oversight in 4f1b890.
Author: Simon Riggs
Discussion: https://postgr.es/m/CANbhV-GJ6KijeCgdOrxqMCQ+C8QiK657EMhCy4csjrPcEUFv_Q@mail.gmail.com
Backpatch-through: 9.6
M src/backend/access/transam/xlog.c
Fixes for multirange selectivity estimation
commit : 322e82b77ef4acb9697c6e4259292f5671cb85bb
author : Alexander Korotkov <[email protected]>
date : Tue, 29 Jun 2021 23:18:09 +0300
committer: Alexander Korotkov <[email protected]>
date : Tue, 29 Jun 2021 23:18:09 +0300
* Fix enumeration of the multirange operators in calc_multirangesel() and
calc_multirangesel() switches.
* Add more regression tests for matching to empty ranges/multiranges.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/c5269c65-f967-77c5-ff7c-15e621c47f6a%40gmail.com
Author: Alexander Korotkov
Backpatch-through: 14, where multiranges were introduced
M src/backend/utils/adt/multirangetypes_selfuncs.c
M src/test/regress/expected/multirangetypes.out
M src/test/regress/sql/multirangetypes.sql
Fix libpq state machine in pipeline mode
commit : 690339fcd58759d213523f87071f992c4402e980
author : Alvaro Herrera <[email protected]>
date : Tue, 29 Jun 2021 15:01:29 -0400
committer: Alvaro Herrera <[email protected]>
date : Tue, 29 Jun 2021 15:01:29 -0400
The original coding required that PQpipelineSync had been called before
the first call to PQgetResult, and failure to do that would result in an
unexpected NULL result being returned. Fix by setting the right state
when a query is sent, rather than leaving it unchanged and having
PQpipelineSync apply the necessary state change.
A new test case to verify the behavior is added, which relies on the new
PQsendFlushRequest() function added by commit a7192326c74d.
Backpatch to 14, where pipeline mode was added.
Reported-by: Boris Kolpackov <[email protected]>
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-exec.c
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
A src/test/modules/libpq_pipeline/traces/nosync.trace
Add PQsendFlushRequest to libpq
commit : 69cf1d5429d48d030d71d5dd4931084d422413cb
author : Alvaro Herrera <[email protected]>
date : Tue, 29 Jun 2021 14:37:39 -0400
committer: Alvaro Herrera <[email protected]>
date : Tue, 29 Jun 2021 14:37:39 -0400
This new libpq function allows the application to send an 'H' message,
which instructs the server to flush its outgoing buffer.
This hasn't been needed so far because the Sync message already requests
a buffer; and I failed to realize that this was needed in pipeline mode
because PQpipelineSync also causes the buffer to be flushed. However,
sometimes it is useful to request a flush without establishing a
synchronization point.
Backpatch to 14, where pipeline mode was introduced in libpq.
Reported-by: Boris Kolpackov <[email protected]>
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/exports.txt
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/libpq-fe.h
Fix bogus logic for reporting which hash partition conflicts.
commit : f8b51464c265696c1eab1c896bddc797beb9a13c
author : Tom Lane <[email protected]>
date : Tue, 29 Jun 2021 14:34:31 -0400
committer: Tom Lane <[email protected]>
date : Tue, 29 Jun 2021 14:34:31 -0400
Commit efbfb6424 added logic for reporting exactly which existing
partition conflicts when complaining that a new hash partition's
modulus isn't compatible with the existing ones. However, it
misunderstood the partitioning data structure, and would select
the wrong partition in some cases, or crash outright due to fetching
a bogus table OID in other cases.
Per bug #17076 from Alexander Lakhin. Fix by Amit Langote;
some further work on the code comments by me.
Discussion: https://postgr.es/m/[email protected]
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/create_table.out
M src/test/regress/sql/create_table.sql
doc: Improve PG14 release notes
commit : caa0f07d2d4b69cd53b5bf7c847d9d65eadbbfee
author : Bruce Momjian <[email protected]>
date : Mon, 28 Jun 2021 20:58:47 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 28 Jun 2021 20:58:47 -0400
Mostly markup improvements.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 14 only
M doc/src/sgml/release-14.sgml
Don't use abort(3) in libpq's fe-print.c.
commit : cf1f545bf281d3dcb7b44e6b7f21a9369024fbf0
author : Tom Lane <[email protected]>
date : Mon, 28 Jun 2021 14:17:42 -0400
committer: Tom Lane <[email protected]>
date : Mon, 28 Jun 2021 14:17:42 -0400
Causing a core dump on out-of-memory seems pretty unfriendly,
and surely is far outside the expected behavior of a general-purpose
library. Just print an error message (as we did already) and return.
These functions unfortunately don't have an error return convention,
but code using them is probably just looking for a quick-n-dirty
print method and wouldn't bother to check anyway.
Although these functions are semi-deprecated, it still seems
appropriate to back-patch this. In passing, also back-patch
b90e6cef1, just to reduce cosmetic differences between the
branches.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-print.c
Don't depend on -fwrapv semantics in pgbench's random() function.
commit : 203c5aaaba56715168c1e80a45d4929120c9281b
author : Tom Lane <[email protected]>
date : Mon, 28 Jun 2021 12:40:37 -0400
committer: Tom Lane <[email protected]>
date : Mon, 28 Jun 2021 12:40:37 -0400
Instead use the common/int.h functions to check for integer overflow
in a more C-standard-compliant fashion. This is motivated by recent
failures on buildfarm member moonjelly, where it appears that
development-tip gcc is optimizing without regard to the -fwrapv
switch. Presumably that's a gcc bug that will be fixed soon, but
we might as well install cleaner coding here rather than wait.
(This does not address the question of whether we'll ever be able
to get rid of using -fwrapv. Testing shows that this spot is the
only place where doing so creates visible regression test failures,
but unfortunately that proves very little.)
Back-patch to v12. The common/int.h functions exist in v11, but
that branch doesn't use them in any client-side code. I judge
that this case isn't interesting enough in the real world to take
even a small risk of issues from being the first such use.
Tom Lane and Fabien Coelho
Discussion: https://postgr.es/m/[email protected]
M src/bin/pgbench/pgbench.c
Pre branch pgindent / pgperltidy run
commit : e1c1c30f635390b6a3ae4993e8cac213a33e6e3f
author : Andrew Dunstan <[email protected]>
date : Mon, 28 Jun 2021 11:05:54 -0400
committer: Andrew Dunstan <[email protected]>
date : Mon, 28 Jun 2021 11:05:54 -0400
Along the way make a slight adjustment to
src/include/utils/queryjumble.h to avoid an unused typedef.
M src/backend/access/heap/hio.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/heap.c
M src/backend/executor/nodeModifyTable.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/storage/ipc/procarray.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/include/nodes/execnodes.h
M src/include/utils/queryjumble.h
M src/test/perl/PostgresNode.pm
M src/test/recovery/t/005_replay_delay.pl
M src/test/recovery/t/025_stuck_on_old_timeline.pl
M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/010_truncate.pl
M src/test/subscription/t/013_partition.pl
M src/test/subscription/t/020_messages.pl
M src/tools/pgindent/typedefs.list
Message style improvements
commit : c31833779d5a4775d7220be4b9143bec66c9a9fd
author : Peter Eisentraut <[email protected]>
date : Mon, 28 Jun 2021 08:36:44 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 28 Jun 2021 08:36:44 +0200
M src/backend/access/common/toast_compression.c
M src/backend/catalog/catalog.c
M src/backend/catalog/pg_inherits.c
M src/backend/commands/tablecmds.c
M src/backend/postmaster/pgstat.c
M src/backend/storage/file/fd.c
M src/backend/storage/ipc/standby.c
M src/backend/tcop/fastpath.c
M src/backend/utils/adt/multirangetypes.c
M src/test/regress/expected/compression_1.out
M src/test/regress/expected/multirangetypes.out
Improve RelationGetIdentityKeyBitmap().
commit : ee3fdb8f3465b3a5937a7fe647b7b6584a600647
author : Amit Kapila <[email protected]>
date : Mon, 28 Jun 2021 10:56:53 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 28 Jun 2021 10:56:53 +0530
We were using RelationGetIndexList() to update the relation's replica
identity index but instead, we can directly use RelationGetReplicaIndex()
which uses the same functionality. This is a minor code readability
improvement.
Author: Japin Li
Reviewed-By: Takamichi Osumi, Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/cache/relcache.c
Fix race condition in TransactionGroupUpdateXidStatus().
commit : b786304c2904a4e444fe740bbc2e0b69efacc19d
author : Amit Kapila <[email protected]>
date : Mon, 28 Jun 2021 09:29:38 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 28 Jun 2021 09:29:38 +0530
When we cannot immediately acquire XactSLRULock in exclusive mode at
commit time, we add ourselves to a list of processes that need their XIDs
status update. We do this if the clog page where we need to update the
current transaction status is the same as the group leader's clog page,
otherwise, we allow the caller to clear it by itself. Now, when we can't
add ourselves to any group, we were not clearing the current proc if it
has already become a member of some group which was leading to an
assertion failure when the same proc was assigned to another backend after
the current backend exits.
Reported-by: Alexander Lakhin
Bug: 17072
Author: Amit Kapila
Tested-By: Alexander Lakhin
Backpatch-through: 11, where it was introduced
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/clog.c
Change recovery_init_sync_method to PGC_SIGHUP.
commit : 34a8b64b4e5f0cd818e5cc7f98846de57938ea57
author : Thomas Munro <[email protected]>
date : Mon, 28 Jun 2021 15:17:43 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 28 Jun 2021 15:17:43 +1200
The setting has no effect except during startup. It's still nice to be
able to change it dynamically, which is expected to be pretty useful to
an admin following crash recovery when restarting the cluster is not so
appealing.
Per discussions following commits 2941138e6 and 61752afb2.
Author: Justin Pryzby <[email protected]>
Reviewed-by: Fujii Masao <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Discussion: https://postgr.es/m/20210529192321.GM2082%40telsasoft.com
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
Fix variable initialization with ALTER SUBSCRIPTION DROP PUBLICATION
commit : 79718c1c6c007c27e9c1b8e92bd96d17067606fa
author : Michael Paquier <[email protected]>
date : Mon, 28 Jun 2021 12:11:18 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 28 Jun 2021 12:11:18 +0900
copy_data is not a supported option with this sub-command of ALTER
SUBSCRIPTION, which would not make a variable related to it initialized
after parsing the option set in DefElems. A refresh could then refer to
it.
Author: Ranier Vilela
Reviewed-by: Peter Smith
Discussion: https://postgr.es/m/CAEudQAp5P8nr=ze2Gv=BMj=DJFZnrvendZCZcC-fos3QiDe2sg@mail.gmail.com
M src/backend/commands/subscriptioncmds.c
Add test for CREATE INDEX CONCURRENTLY with not-so-immutable predicate
commit : 09a69f6e23369847cf11cd03c999a0342d47bbcc
author : Michael Paquier <[email protected]>
date : Mon, 28 Jun 2021 11:17:05 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 28 Jun 2021 11:17:05 +0900
83158f7 has improved index_set_state_flags() so as it is possible to use
transactional updates when updating pg_index state flags, but there was
not really a test case which stressed directly the possibility it fixed.
This commit adds such a test, using a predicate that looks valid in
appearance but calls a stable function.
Author: Andrey Lepikhov
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Remove memory leaks in isolationtester.
commit : 642c0697c96b9c6ba5d194653a329f7820565f01
author : Tom Lane <[email protected]>
date : Sun, 27 Jun 2021 12:45:04 -0400
committer: Tom Lane <[email protected]>
date : Sun, 27 Jun 2021 12:45:04 -0400
specscanner.l leaked a kilobyte of memory per token of the spec file.
Apparently somebody thought that the introductory code block would be
executed once; but it's once per yylex() call.
A couple of functions in isolationtester.c leaked small amounts of
memory due to not bothering to free one-time allocations. Might
as well improve these so that valgrind gives this program a clean
bill of health. Also get rid of an ugly static variable.
Coverity complained about one of the one-time leaks, which led me
to try valgrind'ing isolationtester, which led to discovery of the
larger leak.
M src/test/isolation/isolationtester.c
M src/test/isolation/specscanner.l
Error message refactoring
commit : c302a6139072fed9410204fa9e751d95930e05ff
author : Peter Eisentraut <[email protected]>
date : Sun, 27 Jun 2021 09:41:16 +0200
committer: Peter Eisentraut <[email protected]>
date : Sun, 27 Jun 2021 09:41:16 +0200
Take some untranslatable things out of the message and replace by
format placeholders, to reduce translatable strings and reduce
translation mistakes.
M src/backend/replication/logical/logical.c
M src/backend/statistics/extended_stats.c
M src/backend/utils/adt/numeric.c
Doc: update v14 release notes for revert of commit c0cb87fbb.
commit : dcffc9ba8a1e0ab1b0a57e9b9d38e3dc9960f83f
author : Tom Lane <[email protected]>
date : Sat, 26 Jun 2021 15:45:16 -0400
committer: Tom Lane <[email protected]>
date : Sat, 26 Jun 2021 15:45:16 -0400
M doc/src/sgml/release-14.sgml
Remove undesirable libpq dependency on stringinfo.c.
commit : 8ec00dc5cd70e0e579e9fbf8661bc46f5ccd8078
author : Tom Lane <[email protected]>
date : Sat, 26 Jun 2021 14:20:17 -0400
committer: Tom Lane <[email protected]>
date : Sat, 26 Jun 2021 14:20:17 -0400
Commit c0cb87fbb unwisely introduced a dependency on the StringInfo
machinery in fe-connect.c. We must not use that in libpq, because
it will do a summary exit(1) if it hits OOM, and that is not
appropriate behavior for a general-purpose library. The goal of
allowing arbitrary line lengths in service files doesn't seem like
it's worth a lot of effort, so revert back to the previous method
of using a stack-allocated buffer and failing on buffer overflow.
This isn't an exact revert though. I kept that patch's refactoring
to have a single exit path, as that seems cleaner than having each
error path know what to do to clean up. Also, I made the fixed-size
buffer 1024 bytes not 256, just to push off the need for an expandable
buffer some more.
There is more to do here; in particular the lack of any mechanical
check for this type of mistake now seems pretty hazardous. But this
fix gets us back to the level of robustness we had in v13, anyway.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-connect.c
Remove non-existing variable reference in MSVC's Solution.pm
commit : d5a2c413fcdd187dc16c4fab16610af7d4849cc1
author : Michael Paquier <[email protected]>
date : Sat, 26 Jun 2021 13:52:48 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 26 Jun 2021 13:52:48 +0900
The version string is grabbed from PACKAGE_VERSION in pg_config.h in the
MSVC build since 8f4fb4c6, but an error message referenced a variable
that existed before that. This had no consequences except if one messes
up enough with the version number of the build.
Author: Anton Voloshin
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/tools/msvc/Solution.pm
Remove some useless logs from the TAP tests of pgbench
commit : 704e1dbd9aa29a0b46c356f1803ad55cbdef2c20
author : Michael Paquier <[email protected]>
date : Sat, 26 Jun 2021 12:39:54 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 26 Jun 2021 12:39:54 +0900
002_pgbench_no_server was printing some array pointers instead of the
actual contents of those arrays for the expected outputs of stdout and
stderr for a tested command. This does not add any new information that
can help with debugging as the test names allow to track failure
locations, if any.
This commit simply removes those logs as the rest of the printed
information is redundant with command_checks_all().
Per discussion with Andrew Dunstan and Álvaro Herrera.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M src/bin/pgbench/t/002_pgbench_no_server.pl
Remove unnecessary failure cases in RemoveRoleFromObjectPolicy().
commit : 5a0f1c8c0193f0dd7fba50c22d96781fa2414007
author : Tom Lane <[email protected]>
date : Fri, 25 Jun 2021 13:59:38 -0400
committer: Tom Lane <[email protected]>
date : Fri, 25 Jun 2021 13:59:38 -0400
It's not really necessary for this function to open or lock the
relation associated with the pg_policy entry it's modifying. The
error checks it's making on the rel are if anything counterproductive
(e.g., if we don't want to allow installation of policies on system
catalogs, here is not the place to prevent that). In particular, it
seems just wrong to insist on an ownership check. That has the net
effect of forcing people to use superuser for DROP OWNED BY, which
surely is not an effect we want. Also there is no point in rebuilding
the dependencies of the policy expressions, which aren't being
changed. Lastly, locking the table also seems counterproductive; it's
not helping to prevent race conditions, since we failed to re-read the
pg_policy row after acquiring the lock. That means that concurrent
DDL would likely result in "tuple concurrently updated/deleted"
errors; which is the same behavior this code will produce, with less
overhead.
Per discussion of bug #17062. Back-patch to all supported versions,
as the failure cases this eliminates seem just as undesirable in 9.6
as in HEAD.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/policy.c
Doc: remove commit f560209c6 from v14 release notes.
commit : 8a80562d732c0da1ddcc9fb88dfb976f4b846577
author : Tom Lane <[email protected]>
date : Fri, 25 Jun 2021 11:18:15 -0400
committer: Tom Lane <[email protected]>
date : Fri, 25 Jun 2021 11:18:15 -0400
Now that this has been back-patched, it's no longer a new feature
for v14.
M doc/src/sgml/release-14.sgml
Cleanup some code related to pgbench log checks in TAP tests
commit : 38ff135d9466c35b506b1049fedef73047907be0
author : Michael Paquier <[email protected]>
date : Fri, 25 Jun 2021 20:15:24 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 25 Jun 2021 20:15:24 +0900
This fixes a couple of problems within the so-said code of this commit
subject:
- Replace the use of open() with slurp_file(), fixing an issue reported
by buildfarm member fairywren whose perl installation keep around CRLF
characters, causing the matching patterns for the logs to fail.
- Remove the eval block, which is not really necessary.
This set of issues has come into light after fixing a different issue
with c13585fe, and this is wrong since this code has been introduced.
Reported-by: Andrew Dunstan, and buildfarm member fairywren
Author: Michael Paquier
Reviewed-by: Andrew Dunstan
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M src/bin/pgbench/t/001_pgbench_with_server.pl
Put option listing back into alphabetical order
commit : 3af10943ce21450e299b3915b9cad47cd90369e9
author : Peter Eisentraut <[email protected]>
date : Fri, 25 Jun 2021 11:40:06 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 25 Jun 2021 11:40:06 +0200
M doc/src/sgml/ref/vacuumdb.sgml
M src/bin/scripts/vacuumdb.c
Fixes in ALTER SUBSCRIPTION DROP PUBLICATION code
commit : e59d428f34297cd0eb67e3b4e4b8c8bc58504921
author : Peter Eisentraut <[email protected]>
date : Fri, 25 Jun 2021 09:51:24 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 25 Jun 2021 09:51:24 +0200
ALTER SUBSCRIPTION DROP PUBLICATION does not actually support
copy_data option, so remove it from tab completion.
Also, reword the error message that is thrown when all the
publications from a subscription are specified to be dropped.
Also, made few doc and cosmetic adjustments.
Author: Vignesh C <[email protected]>
Reviewed-by: Bharath Rupireddy <[email protected]>
Reviewed-by: Japin Li <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CALDaNm21RwsDzs4xj14ApteAF7auyyomHNnp+NEL-sH8m-jMvQ@mail.gmail.com
M doc/src/sgml/ref/alter_subscription.sgml
M src/backend/commands/subscriptioncmds.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/subscription.out
doc: Change reloption data type spelling for consistency
commit : 63e6d05bf34da58eef7cd1ed461b9c8315177a38
author : Peter Eisentraut <[email protected]>
date : Fri, 25 Jun 2021 08:11:10 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 25 Jun 2021 08:11:10 +0200
Use "floating point" rather than "float4", like everywhere else in
this context.
Author: [email protected]
Discussion: https://www.postgresql.org/message-id/flat/TYAPR01MB28965989AF84B57FC351B97BC40F9@TYAPR01MB2896.jpnprd01.prod.outlook.com
M doc/src/sgml/ref/create_table.sgml
Remove redundant variable pageSize in gistinitpage
commit : a60c4c5c1a99746485123ae93fbd3e58c78e5d62
author : Peter Eisentraut <[email protected]>
date : Fri, 25 Jun 2021 07:55:34 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 25 Jun 2021 07:55:34 +0200
In gistinitpage, pageSize variable looks redundant, instead just
pass BLCKSZ. This will be consistent with its peers BloomInitPage,
brin_page_init and SpGistInitPage.
Author: Bharath Rupireddy <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CALj2ACWj=V1k5591eeZK2sOg2FYuBUp6azFO8tMkBtGfXf8PMQ@mail.gmail.com
M src/backend/access/gist/gistutil.c
Doc: Fix minor formatting issue in logicaldecoding.sgml.
commit : 847c62ee76cbc237b3a204ca94b1b12811d698e3
author : Amit Kapila <[email protected]>
date : Fri, 25 Jun 2021 08:22:44 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 25 Jun 2021 08:22:44 +0530
Author: Guillaume Lelarge
Discussion: https://www.postgresql.org/message-id/CAECtzeXf3_oZoU6mgFCOy5+pDZ5n4XtH0Da4a5n_KacraVWiHQ@mail.gmail.com
M doc/src/sgml/logicaldecoding.sgml
doc: Add acronyms for MITM and SNI
commit : 15ff5401d1719aaf6c9a47e5abea517cc2bcbaf1
author : Michael Paquier <[email protected]>
date : Fri, 25 Jun 2021 11:29:03 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 25 Jun 2021 11:29:03 +0900
This adds MITM and SNI as acronyms, as the documentation already had
them marked up with <acronym>.
While on it, make sure to spell man-in-the-middle with dashes
consistently, and add acronyms for those new terms where appropriate.
Author: Daniel Gustafsson
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/acronyms.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/libpq.sgml
Add more debugging information with log checks in TAP tests of pgbench
commit : 87b2124dfa0782a697ea7b90aff15a6f6117a720
author : Michael Paquier <[email protected]>
date : Fri, 25 Jun 2021 09:56:44 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 25 Jun 2021 09:56:44 +0900
fairywren is not happy with the pattern checks introduced by c13585f.
I am not sure if this outlines a bug in pgbench or if the regex patterns
used in the tests are too restrictive for this buildfarm member's
environment. This adds more debugging information to show the log
entries that do not match with the expected pattern, to help in finding
out what's happening. That seems like a good addition in the long-term
anyway as that may not be the only issue in this area.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pgbench/t/001_pgbench_with_server.pl
doc: Move remove_temp_files_after_crash to section for developer options
commit : 797b0fc0b078c7b4c46ef9f2d9e47aa2d98c6c63
author : Michael Paquier <[email protected]>
date : Fri, 25 Jun 2021 08:40:16 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 25 Jun 2021 08:40:16 +0900
The main goal of this option is to allow inspecting temporary files for
debugging purposes, so moving the parameter there is natural.
Oversight in cd91de0.
Reported-by: Justin Pryzby
Author: Euler Taveira
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
Prepare for forthcoming LLVM 13 API change.
commit : 9b4e4cfe66ff133717c1b8ba3c2725d525c3e67c
author : Thomas Munro <[email protected]>
date : Fri, 25 Jun 2021 09:55:26 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 25 Jun 2021 09:55:26 +1200
LLVM 13 (due out in September) has changed the semantics of
LLVMOrcAbsoluteSymbols(), so we need to bump some reference counts to
avoid a double-free that causes crashes and bad query results.
A proactive change seems necessary to avoid having a window of time
where our respective latest releases would interact badly. It's
possible that the situation could change before then, though.
Thanks to Fabien Coelho for monitoring bleeding edge LLVM and Andres
Freund for tracking down the change.
Back-patch to 11, where the JIT code arrived.
Discussion: https://postgr.es/m/CA%2BhUKGLEy8mgtN7BNp0ooFAjUedDTJj5dME7NxLU-m91b85siA%40mail.gmail.com
M src/backend/jit/llvm/llvmjit.c
Fix pattern matching logic for logs in TAP tests of pgbench
commit : c13585fe9e55813cf9feac67fe7b65d3a78fff92
author : Michael Paquier <[email protected]>
date : Fri, 25 Jun 2021 06:52:36 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 25 Jun 2021 06:52:36 +0900
The logic checking for the format of per-thread logs used grep() with
directly "$re", which would cause the test to consider all the logs as
a match without caring about their format at all. Using "/$re/" makes
grep() perform a regex test, which is what we want here.
While on it, improve some of the tests to be more picky with the
patterns expected and add more comments to describe the tests.
Issue discovered while digging into a separate patch.
Author: Fabien Coelho, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M src/bin/pgbench/t/001_pgbench_with_server.pl
Further stabilize postgres_fdw test.
commit : 802177090992511c610804da54a4603d4f50c594
author : Tom Lane <[email protected]>
date : Thu, 24 Jun 2021 15:02:13 -0400
committer: Tom Lane <[email protected]>
date : Thu, 24 Jun 2021 15:02:13 -0400
The queries involving ft1_nopw don't stably return the same row
anymore. I surmise that an autovacuum hitting "S 1"."T 1"
right after the updates introduced by f61db909d/5843659d0 freed
some space, changing where subsequent insertions get stored.
It's only by good luck that these results were stable before,
though, since a LIMIT without ORDER BY isn't well defined,
and it's not like we've ever treated that table as append-only
in this test script.
Since we only really care whether these commands succeed or not,
just replace "SELECT *" with "SELECT 1".
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake&dt=2021-06-23%2019%3A52%3A08
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Another fix to relmapper race condition.
commit : 9b8ed0f52bffceaccf6fa650ffe482e7da20fbf2
author : Heikki Linnakangas <[email protected]>
date : Thu, 24 Jun 2021 11:19:03 +0300
committer: Heikki Linnakangas <[email protected]>
date : Thu, 24 Jun 2021 11:19:03 +0300
In previous commit, I missed that relmap_redo() was also not acquiring the
RelationMappingLock. Thanks to Thomas Munro for pointing that out.
Backpatch-through: 9.6, like previous commit.
Discussion: https://www.postgresql.org/message-id/CA%2BhUKGLev%3DPpOSaL3WRZgOvgk217et%2BbxeJcRr4eR-NttP1F6Q%40mail.gmail.com
M src/backend/utils/cache/relmapper.c
Prevent race condition while reading relmapper file.
commit : b6d8d2073f228d9f7198f1f9826d8f6ab643c219
author : Heikki Linnakangas <[email protected]>
date : Thu, 24 Jun 2021 10:45:23 +0300
committer: Heikki Linnakangas <[email protected]>
date : Thu, 24 Jun 2021 10:45:23 +0300
Contrary to the comment here, POSIX does not guarantee atomicity of a
read(), if another process calls write() concurrently. Or at least Linux
does not. Add locking to load_relmap_file() to avoid the race condition.
Fixes bug #17064. Thanks to Alexander Lakhin for the report and test case.
Backpatch-through: 9.6, all supported versions.
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/backend/utils/cache/relmapper.c
Doc: Update logical replication message formats.
commit : f08722cf83ab1fabee948a4e5754bf6236ad700b
author : Amit Kapila <[email protected]>
date : Thu, 24 Jun 2021 11:51:58 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 24 Jun 2021 11:51:58 +0530
Commits 9de77b5453 and ac4645c015 missed to update the logical replication
message formats section in the docs.
Author: Brar Piening
Reviewed-by: Amit Kapila
Discussion: https://www.postgresql.org/message-id/[email protected]
M doc/src/sgml/protocol.sgml
Doc: Update caveats in synchronous logical replication.
commit : c66fb78ebb4f473bb4fd8773de3cda9339e14f81
author : Amit Kapila <[email protected]>
date : Thu, 24 Jun 2021 09:13:46 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 24 Jun 2021 09:13:46 +0530
Reported-by: Simon Riggs
Author: Takamichi Osumi
Reviewed-by: Amit Kapila
Backpatch-through: 9.6
Discussion: https://www.postgresql.org/message-id/[email protected]
M doc/src/sgml/logicaldecoding.sgml
Allow non-quoted identifiers as isolation test session/step names.
commit : a443c1b2d6a646cf90a8afc193c07ed12a2bf045
author : Tom Lane <[email protected]>
date : Wed, 23 Jun 2021 18:41:39 -0400
committer: Tom Lane <[email protected]>
date : Wed, 23 Jun 2021 18:41:39 -0400
For no obvious reason, isolationtester has always insisted that
session and step names be written with double quotes. This is
fairly tedious and does little for test readability, especially
since the names that people actually choose almost always look
like normal identifiers. Hence, let's tweak the lexer to allow
SQL-like identifiers not only double-quoted strings.
(They're SQL-like, not exactly SQL, because I didn't add any
case-folding logic. Also there's no provision for U&"..." names,
not that anyone's likely to care.)
There is one incompatibility introduced by this change: if you write
"foo""bar" with no space, that used to be taken as two identifiers,
but now it's just one identifier with an embedded quote mark.
I converted all the src/test/isolation/ specfiles to remove
unnecessary double quotes, but stopped there because my
eyes were glazing over already.
Like 741d7f104, back-patch to all supported branches, so that this
isn't a stumbling block for back-patching isolation test changes.
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/specs/oldest_xmin.spec
M src/test/isolation/README
M src/test/isolation/specparse.y
M src/test/isolation/specs/aborted-keyrevoke.spec
M src/test/isolation/specs/alter-table-1.spec
M src/test/isolation/specs/alter-table-2.spec
M src/test/isolation/specs/alter-table-3.spec
M src/test/isolation/specs/alter-table-4.spec
M src/test/isolation/specs/async-notify.spec
M src/test/isolation/specs/classroom-scheduling.spec
M src/test/isolation/specs/create-trigger.spec
M src/test/isolation/specs/deadlock-hard.spec
M src/test/isolation/specs/deadlock-parallel.spec
M src/test/isolation/specs/deadlock-simple.spec
M src/test/isolation/specs/deadlock-soft-2.spec
M src/test/isolation/specs/deadlock-soft.spec
M src/test/isolation/specs/delete-abort-savept-2.spec
M src/test/isolation/specs/delete-abort-savept.spec
M src/test/isolation/specs/detach-partition-concurrently-1.spec
M src/test/isolation/specs/detach-partition-concurrently-2.spec
M src/test/isolation/specs/detach-partition-concurrently-3.spec
M src/test/isolation/specs/detach-partition-concurrently-4.spec
M src/test/isolation/specs/drop-index-concurrently-1.spec
M src/test/isolation/specs/eval-plan-qual-trigger.spec
M src/test/isolation/specs/eval-plan-qual.spec
M src/test/isolation/specs/fk-contention.spec
M src/test/isolation/specs/fk-deadlock.spec
M src/test/isolation/specs/fk-deadlock2.spec
M src/test/isolation/specs/fk-partitioned-1.spec
M src/test/isolation/specs/fk-partitioned-2.spec
M src/test/isolation/specs/freeze-the-dead.spec
M src/test/isolation/specs/horizons.spec
M src/test/isolation/specs/index-only-scan.spec
M src/test/isolation/specs/inherit-temp.spec
M src/test/isolation/specs/insert-conflict-do-nothing-2.spec
M src/test/isolation/specs/insert-conflict-do-nothing.spec
M src/test/isolation/specs/insert-conflict-do-update-2.spec
M src/test/isolation/specs/insert-conflict-do-update-3.spec
M src/test/isolation/specs/insert-conflict-do-update.spec
M src/test/isolation/specs/insert-conflict-specconflict.spec
M src/test/isolation/specs/lock-committed-keyupdate.spec
M src/test/isolation/specs/lock-committed-update.spec
M src/test/isolation/specs/lock-update-delete.spec
M src/test/isolation/specs/lock-update-traversal.spec
M src/test/isolation/specs/multiple-cic.spec
M src/test/isolation/specs/multiple-row-versions.spec
M src/test/isolation/specs/multixact-no-deadlock.spec
M src/test/isolation/specs/multixact-no-forget.spec
M src/test/isolation/specs/nowait-2.spec
M src/test/isolation/specs/nowait-3.spec
M src/test/isolation/specs/nowait-4.spec
M src/test/isolation/specs/nowait-5.spec
M src/test/isolation/specs/nowait.spec
M src/test/isolation/specs/partial-index.spec
M src/test/isolation/specs/partition-concurrent-attach.spec
M src/test/isolation/specs/partition-key-update-1.spec
M src/test/isolation/specs/partition-key-update-2.spec
M src/test/isolation/specs/partition-key-update-3.spec
M src/test/isolation/specs/partition-key-update-4.spec
M src/test/isolation/specs/plpgsql-toast.spec
M src/test/isolation/specs/predicate-gin.spec
M src/test/isolation/specs/predicate-gist.spec
M src/test/isolation/specs/predicate-hash.spec
M src/test/isolation/specs/predicate-lock-hot-tuple.spec
M src/test/isolation/specs/prepared-transactions-cic.spec
M src/test/isolation/specs/prepared-transactions.spec
M src/test/isolation/specs/project-manager.spec
M src/test/isolation/specs/propagate-lock-delete.spec
M src/test/isolation/specs/read-only-anomaly-2.spec
M src/test/isolation/specs/read-only-anomaly-3.spec
M src/test/isolation/specs/read-only-anomaly.spec
M src/test/isolation/specs/read-write-unique-2.spec
M src/test/isolation/specs/read-write-unique-3.spec
M src/test/isolation/specs/read-write-unique-4.spec
M src/test/isolation/specs/read-write-unique.spec
M src/test/isolation/specs/receipt-report.spec
M src/test/isolation/specs/referential-integrity.spec
M src/test/isolation/specs/reindex-concurrently.spec
M src/test/isolation/specs/reindex-schema.spec
M src/test/isolation/specs/ri-trigger.spec
M src/test/isolation/specs/sequence-ddl.spec
M src/test/isolation/specs/serializable-parallel-2.spec
M src/test/isolation/specs/serializable-parallel.spec
M src/test/isolation/specs/simple-write-skew.spec
M src/test/isolation/specs/skip-locked-2.spec
M src/test/isolation/specs/skip-locked-3.spec
M src/test/isolation/specs/skip-locked-4.spec
M src/test/isolation/specs/skip-locked.spec
M src/test/isolation/specs/temporal-range-integrity.spec
M src/test/isolation/specs/timeouts.spec
M src/test/isolation/specs/total-cash.spec
M src/test/isolation/specs/truncate-conflict.spec
M src/test/isolation/specs/tuplelock-conflict.spec
M src/test/isolation/specs/tuplelock-partition.spec
M src/test/isolation/specs/tuplelock-update.spec
M src/test/isolation/specs/tuplelock-upgrade-no-deadlock.spec
M src/test/isolation/specs/two-ids.spec
M src/test/isolation/specs/update-conflict-out.spec
M src/test/isolation/specs/update-locked-tuple.spec
M src/test/isolation/specs/vacuum-concurrent-drop.spec
M src/test/isolation/specs/vacuum-conflict.spec
M src/test/isolation/specs/vacuum-reltuples.spec
M src/test/isolation/specs/vacuum-skip-locked.spec
M src/test/isolation/specscanner.l
Doc: fix confusion about LEAKPROOF in syntax summaries.
commit : 2031e1668e5577e64cfed29da69a34903d5a5227
author : Tom Lane <[email protected]>
date : Wed, 23 Jun 2021 14:27:13 -0400
committer: Tom Lane <[email protected]>
date : Wed, 23 Jun 2021 14:27:13 -0400
The syntax summaries for CREATE FUNCTION and allied commands
made it look like LEAKPROOF is an alternative to
IMMUTABLE/STABLE/VOLATILE, when of course it is an orthogonal
option. Improve that.
Per gripe from aazamrafeeque0. Thanks to David Johnston for
suggestions.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/alter_function.sgml
M doc/src/sgml/ref/alter_routine.sgml
M doc/src/sgml/ref/create_function.sgml
Don't assume GSSAPI result strings are null-terminated.
commit : 126cdaf47af275f76b2f2ddb023bfdc6f018ae30
author : Tom Lane <[email protected]>
date : Wed, 23 Jun 2021 14:01:32 -0400
committer: Tom Lane <[email protected]>
date : Wed, 23 Jun 2021 14:01:32 -0400
Our uses of gss_display_status() and gss_display_name() assumed
that the gss_buffer_desc strings returned by those functions are
null-terminated. It appears that they generally are, given the
lack of field complaints up to now. However, the available
documentation does not promise this, and some man pages
for gss_display_status() show examples that rely on the
gss_buffer_desc.length field instead of expecting null
termination. Also, we now have a report that on some
implementations, clang's address sanitizer is of the opinion
that the byte after the specified length is undefined.
Hence, change the code to rely on the length field instead.
This might well be cosmetic rather than fixing any real bug, but
it's hard to be sure, so back-patch to all supported branches.
While here, also back-patch the v12 changes that made pg_GSS_error
deal honestly with multiple messages available from
gss_display_status.
Per report from Sudheer H R.
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/auth.c
M src/backend/libpq/be-gssapi-common.c
M src/interfaces/libpq/fe-gssapi-common.c
Improve display of query results in isolation tests.
commit : 4a054069a36032a59afceb07f3b837f09ab1a2e9
author : Tom Lane <[email protected]>
date : Wed, 23 Jun 2021 11:12:31 -0400
committer: Tom Lane <[email protected]>
date : Wed, 23 Jun 2021 11:12:31 -0400
Previously, isolationtester displayed SQL query results using some
ad-hoc code that clearly hadn't had much effort expended on it.
Field values longer than 14 characters weren't separated from
the next field, and usually caused misalignment of the columns
too. Also there was no visual separation of a query's result
from subsequent isolationtester output. This made test result
files confusing and hard to read.
To improve matters, let's use libpq's PQprint() function. Although
that's long since unused by psql, it's still plenty good enough
for the purpose here.
Like 741d7f104, back-patch to all supported branches, so that this
isn't a stumbling block for back-patching isolation test changes.
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/expected/concurrent_ddl_dml.out
M contrib/test_decoding/expected/concurrent_stream.out
M contrib/test_decoding/expected/delayed_startup.out
M contrib/test_decoding/expected/mxact.out
M contrib/test_decoding/expected/oldest_xmin.out
M contrib/test_decoding/expected/ondisk_startup.out
M contrib/test_decoding/expected/snapshot_transfer.out
M contrib/test_decoding/expected/subxact_without_top.out
M contrib/test_decoding/expected/twophase_snapshot.out
M src/test/isolation/expected/aborted-keyrevoke.out
M src/test/isolation/expected/alter-table-1.out
M src/test/isolation/expected/alter-table-2.out
M src/test/isolation/expected/alter-table-3.out
M src/test/isolation/expected/alter-table-4.out
M src/test/isolation/expected/async-notify.out
M src/test/isolation/expected/classroom-scheduling.out
M src/test/isolation/expected/create-trigger.out
M src/test/isolation/expected/deadlock-parallel.out
M src/test/isolation/expected/delete-abort-savept-2.out
M src/test/isolation/expected/delete-abort-savept.out
M src/test/isolation/expected/detach-partition-concurrently-1.out
M src/test/isolation/expected/detach-partition-concurrently-2.out
M src/test/isolation/expected/detach-partition-concurrently-3.out
M src/test/isolation/expected/detach-partition-concurrently-4.out
M src/test/isolation/expected/drop-index-concurrently-1.out
M src/test/isolation/expected/drop-index-concurrently-1_2.out
M src/test/isolation/expected/eval-plan-qual-trigger.out
M src/test/isolation/expected/eval-plan-qual.out
M src/test/isolation/expected/fk-partitioned-2.out
M src/test/isolation/expected/freeze-the-dead.out
M src/test/isolation/expected/horizons.out
M src/test/isolation/expected/inherit-temp.out
M src/test/isolation/expected/insert-conflict-do-nothing-2.out
M src/test/isolation/expected/insert-conflict-do-nothing.out
M src/test/isolation/expected/insert-conflict-do-update-2.out
M src/test/isolation/expected/insert-conflict-do-update-3.out
M src/test/isolation/expected/insert-conflict-do-update.out
M src/test/isolation/expected/insert-conflict-specconflict.out
M src/test/isolation/expected/lock-committed-keyupdate.out
M src/test/isolation/expected/lock-committed-update.out
M src/test/isolation/expected/lock-update-delete.out
M src/test/isolation/expected/lock-update-delete_1.out
M src/test/isolation/expected/lock-update-traversal.out
M src/test/isolation/expected/multiple-cic.out
M src/test/isolation/expected/multiple-row-versions.out
M src/test/isolation/expected/multixact-no-deadlock.out
M src/test/isolation/expected/multixact-no-forget.out
M src/test/isolation/expected/multixact-no-forget_1.out
M src/test/isolation/expected/nowait-2.out
M src/test/isolation/expected/nowait-3.out
M src/test/isolation/expected/nowait-4.out
M src/test/isolation/expected/nowait-4_1.out
M src/test/isolation/expected/nowait-5.out
M src/test/isolation/expected/nowait.out
M src/test/isolation/expected/partial-index.out
M src/test/isolation/expected/partition-concurrent-attach.out
M src/test/isolation/expected/partition-key-update-2.out
M src/test/isolation/expected/partition-key-update-3.out
M src/test/isolation/expected/partition-key-update-4.out
M src/test/isolation/expected/plpgsql-toast.out
M src/test/isolation/expected/predicate-gin.out
M src/test/isolation/expected/predicate-gist.out
M src/test/isolation/expected/predicate-hash.out
M src/test/isolation/expected/predicate-lock-hot-tuple.out
M src/test/isolation/expected/prepared-transactions-cic.out
M src/test/isolation/expected/prepared-transactions.out
M src/test/isolation/expected/project-manager.out
M src/test/isolation/expected/read-only-anomaly-2.out
M src/test/isolation/expected/read-only-anomaly-3.out
M src/test/isolation/expected/read-only-anomaly.out
M src/test/isolation/expected/read-write-unique-2.out
M src/test/isolation/expected/read-write-unique-3.out
M src/test/isolation/expected/read-write-unique-4.out
M src/test/isolation/expected/read-write-unique.out
M src/test/isolation/expected/receipt-report.out
M src/test/isolation/expected/referential-integrity.out
M src/test/isolation/expected/reindex-concurrently.out
M src/test/isolation/expected/ri-trigger.out
M src/test/isolation/expected/sequence-ddl.out
M src/test/isolation/expected/serializable-parallel-2.out
M src/test/isolation/expected/serializable-parallel.out
M src/test/isolation/expected/skip-locked-2.out
M src/test/isolation/expected/skip-locked-3.out
M src/test/isolation/expected/skip-locked-4.out
M src/test/isolation/expected/skip-locked-4_1.out
M src/test/isolation/expected/skip-locked.out
M src/test/isolation/expected/temporal-range-integrity.out
M src/test/isolation/expected/timeouts.out
M src/test/isolation/expected/total-cash.out
M src/test/isolation/expected/truncate-conflict.out
M src/test/isolation/expected/tuplelock-conflict.out
M src/test/isolation/expected/tuplelock-partition.out
M src/test/isolation/expected/tuplelock-update.out
M src/test/isolation/expected/tuplelock-upgrade-no-deadlock.out
M src/test/isolation/expected/two-ids.out
M src/test/isolation/expected/update-conflict-out.out
M src/test/isolation/expected/vacuum-reltuples.out
M src/test/isolation/isolationtester.c
M src/test/modules/brin/expected/summarization-and-inprogress-insertion.out
M src/test/modules/delay_execution/expected/partition-addition.out
M src/test/modules/delay_execution/expected/partition-removal-1.out
M src/test/modules/snapshot_too_old/expected/sto_using_cursor.out
M src/test/modules/snapshot_too_old/expected/sto_using_hash_index.out
M src/test/modules/snapshot_too_old/expected/sto_using_select.out
Add test case for obsoleting slot with active walsender, take 2
commit : 24043c27b46f873211177e3460ab09dc011802a5
author : Alvaro Herrera <[email protected]>
date : Wed, 23 Jun 2021 09:53:18 -0400
committer: Alvaro Herrera <[email protected]>
date : Wed, 23 Jun 2021 09:53:18 -0400
The code to signal a running walsender when its reserved WAL size grows
too large is completely uncovered before this commit; this adds coverage
for that case.
This test involves sending SIGSTOP to walsender and walreceiver, then
advancing enough WAL for a checkpoint to trigger, then sending SIGCONT.
There's no precedent for STOP signalling in Perl tests, and my reading
of relevant manpages says it's likely to fail on Windows. Because of
this, this test is always skipped on that platform.
This version fixes a couple of rarely hit race conditions in the
previous attempt 09126984a263; most notably, both LOG string searches
are loops, not just the second one; we acquire the start-of-log position
before STOP-signalling; and reference the correct process name in the
test description. All per Tom Lane.
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/019_replslot_limit.pl
Use annotations to reduce instability of isolation-test results.
commit : 741d7f1047fe52da7ced6fa9cea661ce9320c8d4
author : Tom Lane <[email protected]>
date : Tue, 22 Jun 2021 21:43:12 -0400
committer: Tom Lane <[email protected]>
date : Tue, 22 Jun 2021 21:43:12 -0400
We've long contended with isolation test results that aren't entirely
stable. Some test scripts insert long delays to try to force stable
results, which is not terribly desirable; but other erratic failure
modes remain, causing unrepeatable buildfarm failures. I've spent a
fair amount of time trying to solve this by improving the server-side
support code, without much success: that way is fundamentally unable
to cope with diffs that stem from chance ordering of arrival of
messages from different server processes.
We can improve matters on the client side, however, by annotating
the test scripts themselves to show the desired reporting order
of events that might occur in different orders. This patch adds
three types of annotations to deal with (a) test steps that might or
might not complete their waits before the isolationtester can see them
waiting; (b) test steps in different sessions that can legitimately
complete in either order; and (c) NOTIFY messages that might arrive
before or after the completion of a step in another session. We might
need more annotation types later, but this seems to be enough to deal
with the instabilities we've seen in the buildfarm. It also lets us
get rid of all the long delays that were previously used, cutting more
than a minute off the runtime of the isolation tests.
Back-patch to all supported branches, because the buildfarm
instabilities affect all the branches, and because it seems desirable
to keep isolationtester's capabilities the same across all branches
to simplify possible future back-patching of tests.
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/expected/concurrent_ddl_dml.out
M src/test/isolation/README
M src/test/isolation/expected/alter-table-3.out
M src/test/isolation/expected/alter-table-4.out
M src/test/isolation/expected/deadlock-hard.out
M src/test/isolation/expected/deadlock-simple.out
M src/test/isolation/expected/detach-partition-concurrently-3.out
M src/test/isolation/expected/detach-partition-concurrently-4.out
M src/test/isolation/expected/eval-plan-qual-trigger.out
M src/test/isolation/expected/eval-plan-qual.out
M src/test/isolation/expected/fk-deadlock2_1.out
M src/test/isolation/expected/fk-deadlock2_2.out
M src/test/isolation/expected/fk-deadlock_1.out
M src/test/isolation/expected/fk-partitioned-1.out
M src/test/isolation/expected/fk-partitioned-2.out
M src/test/isolation/expected/insert-conflict-do-nothing-2.out
M src/test/isolation/expected/insert-conflict-specconflict.out
M src/test/isolation/expected/lock-committed-keyupdate.out
M src/test/isolation/expected/lock-update-delete_1.out
M src/test/isolation/expected/multiple-cic.out
D src/test/isolation/expected/multiple-cic_1.out
M src/test/isolation/expected/multixact-no-forget_1.out
M src/test/isolation/expected/nowait-4.out
M src/test/isolation/expected/nowait-4_1.out
M src/test/isolation/expected/nowait-5.out
M src/test/isolation/expected/partition-concurrent-attach.out
M src/test/isolation/expected/partition-key-update-1.out
M src/test/isolation/expected/partition-key-update-3.out
M src/test/isolation/expected/propagate-lock-delete.out
M src/test/isolation/expected/read-write-unique-2.out
M src/test/isolation/expected/read-write-unique-3.out
M src/test/isolation/expected/read-write-unique-4.out
M src/test/isolation/expected/read-write-unique.out
M src/test/isolation/expected/sequence-ddl.out
M src/test/isolation/expected/skip-locked-4_1.out
M src/test/isolation/expected/timeouts.out
M src/test/isolation/expected/tuplelock-update.out
M src/test/isolation/isolationtester.c
M src/test/isolation/isolationtester.h
M src/test/isolation/specparse.y
M src/test/isolation/specs/deadlock-hard.spec
M src/test/isolation/specs/deadlock-soft-2.spec
M src/test/isolation/specs/detach-partition-concurrently-3.spec
M src/test/isolation/specs/detach-partition-concurrently-4.spec
M src/test/isolation/specs/insert-conflict-specconflict.spec
M src/test/isolation/specs/multiple-cic.spec
M src/test/isolation/specs/timeouts.spec
M src/test/isolation/specs/tuplelock-update.spec
M src/test/isolation/specscanner.l
Restore the portal-level snapshot for simple expressions, too.
commit : d102aafb6259a6a412803d4b1d8c4f00aa17f67e
author : Tom Lane <[email protected]>
date : Tue, 22 Jun 2021 17:48:39 -0400
committer: Tom Lane <[email protected]>
date : Tue, 22 Jun 2021 17:48:39 -0400
Commits 84f5c2908 et al missed the need to cover plpgsql's "simple
expression" code path. If the first thing we execute after a
COMMIT/ROLLBACK is one of those, rather than a full-fledged SPI command,
we must explicitly do EnsurePortalSnapshotExists() to make sure we have
an outer snapshot. Note that it wouldn't be good enough to just push a
snapshot for the duration of the expression execution: what comes back
might be toasted, so we'd better have a snapshot protecting it.
The test case demonstrating this fact cheats a bit by marking a SQL
function immutable even though it fetches from a table. That's
nothing that users haven't been seen to do, though.
Per report from Jim Nasby. Back-patch to v11, like the previous fix.
Discussion: https://postgr.es/m/[email protected]
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_transaction.sql
Add list of ignorable pgindent commits for git-blame.
commit : 8e638845ff6bb255ad1dea15831089a234535391
author : Peter Geoghegan <[email protected]>
date : Tue, 22 Jun 2021 09:06:32 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 22 Jun 2021 09:06:32 -0700
Add a .git-blame-ignore-revs file with a list of pgindent, pgperlyidy,
and reformat-dat-files commit hashes. Postgres hackers that configure
git to use the ignore file will get git-blame output that avoids
attributing line changes to the ignored indent commits. This makes
git-blame output much easier to work with in practice.
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wz=cVh3GHTP6SdLU-Gnmt2zRdF8vZkcrFdSzXQ=WhbWm9Q@mail.gmail.com
A .git-blame-ignore-revs
M src/tools/RELEASE_CHANGES
M src/tools/pgindent/README
Stamp 14beta2.
commit : bafad2c5b261a1449bed60ebac9e7918ebcc42b4
author : Joe Conway <[email protected]>
date : Mon, 21 Jun 2021 17:07:55 -0400
committer: Joe Conway <[email protected]>
date : Mon, 21 Jun 2021 17:07:55 -0400
M configure
M configure.ac
Use correct horizon when vacuuming catalog relations.
commit : 5a1e1d83022b976ebdec5cfa8f255c4278b75b8e
author : Andres Freund <[email protected]>
date : Mon, 21 Jun 2021 05:13:46 -0700
committer: Andres Freund <[email protected]>
date : Mon, 21 Jun 2021 05:13:46 -0700
In dc7420c2c92 I (Andres) accidentally used
RelationIsAccessibleInLogicalDecoding() as the sole condition to use the
non-shared catalog horizon in GetOldestNonRemovableTransactionId(). That is
incorrect, as RelationIsAccessibleInLogicalDecoding() checks whether wal_level
is logical.
The correct check, as done e.g. in GlobalVisTestFor(), is to check
IsCatalogRelation() and RelationIsAccessibleInLogicalDecoding().
The observed misbehavior of this bug was that there could be an endless loop
in lazy_scan_prune(), because the horizons used in heap_page_prune() and the
individual tuple liveliness checks did not match. Likely there are other
potential consequences as well.
A later commit will unify the determination which horizon has to be used, and
add additional assertions to make it easier to catch a bug like this.
Reported-By: Justin Pryzby <[email protected]>
Diagnosed-By: Matthias van de Meent <[email protected]>
Author: Matthias van de Meent <[email protected]>
Discussion: https://postgr.es/m/CAEze2Wg32Y9+WJfw=aofkRx1ZRFt_Ev6bNPc4PSaz7PjSFtZgQ@mail.gmail.com
M src/backend/storage/ipc/procarray.c
Fix assert failure in expand_grouping_sets
commit : 8d29d45d9b3cab95a866efbcdd9138b3d76741b3
author : David Rowley <[email protected]>
date : Mon, 21 Jun 2021 23:11:23 +1200
committer: David Rowley <[email protected]>
date : Mon, 21 Jun 2021 23:11:23 +1200
linitial_node() fails in assert enabled builds if the given pointer is
not of the specified type. Here the type is IntList. The code thought
it should be expecting List, but it was wrong.
In the existing tests which run this code the initial list element is
always NIL. Since linitial_node() allows NULL, we didn't trigger any
assert failures in the existing regression tests.
There is still some discussion as to whether we need a few more tests in
this area, but for now, since beta2 is looming, fix the bug first.
Bug: #17067
Discussion: https://postgr.es/m/[email protected]
Reported-by: Yaoguang Chen
M src/backend/parser/parse_agg.c
Translation updates
commit : a7bb0ce58f56ee8907c3f49c52d99f502536c796
author : Peter Eisentraut <[email protected]>
date : Mon, 21 Jun 2021 12:32:14 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 21 Jun 2021 12:32:14 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 70796ae860c444c764bb591c885f22cac1c168ec
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/bin/initdb/po/el.po
M src/bin/initdb/po/es.po
M src/bin/pg_amcheck/nls.mk
A src/bin/pg_amcheck/po/el.po
A src/bin/pg_amcheck/po/es.po
M src/bin/pg_amcheck/po/fr.po
A src/bin/pg_amcheck/po/zh_CN.po
M src/bin/pg_archivecleanup/po/el.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_checksums/po/el.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_config/po/el.po
M src/bin/pg_config/po/es.po
M src/bin/pg_controldata/nls.mk
A src/bin/pg_controldata/po/el.po
M src/bin/pg_ctl/po/el.po
M src/bin/pg_ctl/po/es.po
M src/bin/pg_dump/po/el.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/zh_CN.po
M src/bin/pg_test_fsync/po/el.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_timing/po/el.po
M src/bin/pg_test_timing/po/es.po
M src/bin/pg_test_timing/po/zh_CN.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_verifybackup/nls.mk
A src/bin/pg_verifybackup/po/el.po
M src/bin/pg_verifybackup/po/es.po
M src/bin/pg_waldump/nls.mk
A src/bin/pg_waldump/po/el.po
M src/bin/psql/po/el.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/fr.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/fr.po
M src/interfaces/libpq/po/el.po
M src/interfaces/libpq/po/es.po
M src/interfaces/libpq/po/fr.po
M src/pl/plperl/nls.mk
A src/pl/plperl/po/el.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpython/nls.mk
A src/pl/plpython/po/el.po
M src/pl/tcl/po/el.po
Finish rename of PQtraceSetFlags() to PQsetTraceFlags().
commit : 047a259e35b9dde2dad5fd0e5d5d784bb327b848
author : Noah Misch <[email protected]>
date : Mon, 21 Jun 2021 02:48:11 -0700
committer: Noah Misch <[email protected]>
date : Mon, 21 Jun 2021 02:48:11 -0700
Jie Zhang
Discussion: https://postgr.es/m/TYWPR01MB767844835390EDD8DB276D75F90A9@TYWPR01MB7678.jpnprd01.prod.outlook.com
M doc/src/sgml/libpq.sgml
amcheck: Fix code comments
commit : 97b7134186490b36e01efc9d2feaf7038c666457
author : Peter Eisentraut <[email protected]>
date : Mon, 21 Jun 2021 11:17:49 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 21 Jun 2021 11:17:49 +0200
Code comments were claiming that verify_heapam() was checking
privileges on the relation it was operating on, but it didn't actually
do that. Perhaps earlier versions of the patch did that, but now the
access is regulated by privileges on the function. Remove the wrong
comments.
M contrib/amcheck/verify_heapam.c
doc: adjust PG 14 relnotes to be current
commit : 69a58bfe4ab05567a8fab8bdce7f3095ed06b99c
author : Bruce Momjian <[email protected]>
date : Mon, 21 Jun 2021 01:09:32 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 21 Jun 2021 01:09:32 -0400
M doc/src/sgml/release-14.sgml
doc: add mention of +4GB windows file handling in PG14 relnotes
commit : 90855908b751d40f67352fa0252e0fcdaa7e317b
author : Bruce Momjian <[email protected]>
date : Sun, 20 Jun 2021 23:53:00 -0400
committer: Bruce Momjian <[email protected]>
date : Sun, 20 Jun 2021 23:53:00 -0400
Reported-by: Masahiko Sawada
Discussion: https://postgr.es/m/CAD21AoCTHyouoGv-xt1qNjjvPbGMErLi0AJncByTvr66Nq7j8g@mail.gmail.com
M doc/src/sgml/release-14.sgml
Remove overzealous VACUUM failsafe assertions.
commit : e8f201ab82be234b2f103234cf9f262f9afeaeba
author : Peter Geoghegan <[email protected]>
date : Sun, 20 Jun 2021 18:14:00 -0700
committer: Peter Geoghegan <[email protected]>
date : Sun, 20 Jun 2021 18:14:00 -0700
The failsafe can trigger when index processing is already disabled.
This can happen when VACUUM's INDEX_CLEANUP parameter is "off" and the
failsafe happens to trigger. Remove assertions that assume that index
processing is directly tied to the failsafe.
Oversight in commit c242baa4, which made it possible for the failsafe to
trigger in a two-pass strategy VACUUM that has yet to make its first
call to lazy_vacuum_all_indexes().
M src/backend/access/heap/vacuumlazy.c
Revert "Add test case for obsoleting slot with active walsender"
commit : 96795176810b979a2bf1f4563bdcb161679d5b95
author : Alvaro Herrera <[email protected]>
date : Sun, 20 Jun 2021 12:28:08 -0400
committer: Alvaro Herrera <[email protected]>
date : Sun, 20 Jun 2021 12:28:08 -0400
This reverts commit 09126984a263; the test case added there failed once
in circumstances that remain mysterious. It seems better to remove the
test for now so that 14beta2 doesn't have random failures built in.
M src/test/recovery/t/019_replslot_limit.pl
Stabilize test case added by commit f61db909d.
commit : 5843659d091bfb6f2c60e010ea1fd00e55ee6ada
author : Tom Lane <[email protected]>
date : Sun, 20 Jun 2021 11:48:44 -0400
committer: Tom Lane <[email protected]>
date : Sun, 20 Jun 2021 11:48:44 -0400
Buildfarm members ayu and tern have sometimes shown a different
plan than expected for this query. I'd been unable to reproduce
that before today, but I finally realized what is happening.
If there is a concurrent open transaction (probably an autovacuum
run in the buildfarm, but this can also be arranged manually),
then the index entries for the rows removed by the DELETE a few
lines up are not killed promptly, causing a change in the planner's
estimate of the extremal value of ft2.c1, which moves the rowcount
estimate for "c1 > 1100" by enough to change the join plan from
nestloop to hash.
To fix, change the query condition to "c1 > 1000", causing the
hash plan to be preferred whether or not a concurrent open
transaction exists. Since this UPDATE is tailored to be a no-op,
nothing else changes.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=ayu&dt=2021-06-09%2022%3A45%3A48
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=ayu&dt=2021-06-13%2022%3A38%3A18
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tern&dt=2021-06-20%2004%3A55%3A36
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Provide feature-test macros for libpq features added in v14.
commit : 6991e774e0304f5ef488cf1ae4fa79578b6ae3d5
author : Tom Lane <[email protected]>
date : Sat, 19 Jun 2021 11:44:39 -0400
committer: Tom Lane <[email protected]>
date : Sat, 19 Jun 2021 11:44:39 -0400
We had a request to provide a way to test at compile time for the
availability of the new pipeline features. More generally, it
seems like a good idea to provide a way to test via #ifdef for
all new libpq API features. People have been using the version
from pg_config.h for that; but that's more likely to represent the
server version than the libpq version, in the increasingly-common
scenario where they're different. It's safer if libpq-fe.h itself
is the source of truth about what features it offers.
Hence, establish a policy that starting in v14 we'll add a suitable
feature-is-present macro to libpq-fe.h when we add new API there.
(There doesn't seem to be much point in applying this policy
retroactively, but it's not too late for v14.)
Tom Lane and Alvaro Herrera, per suggestion from Boris Kolpackov.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/libpq-fe.h
Handle no replica identity index case in RelationGetIdentityKeyBitmap.
commit : 2731ce1bd550d08f3fdd7bcb1497af4b95170976
author : Amit Kapila <[email protected]>
date : Sat, 19 Jun 2021 11:30:33 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 19 Jun 2021 11:30:33 +0530
Commit e7eea52b2d has introduced a new function
RelationGetIdentityKeyBitmap which omits to handle the case where there is
no replica identity index on a relation.
Author: Mark Dilger
Reviewed-by: Takamichi Osumi, Amit Kapila
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/backend/utils/cache/relcache.c
M src/test/subscription/t/001_rep_changes.pl
Support disabling index bypassing by VACUUM.
commit : 3499df0dee8c4ea51d264a674df5b5e31991319a
author : Peter Geoghegan <[email protected]>
date : Fri, 18 Jun 2021 20:04:07 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 18 Jun 2021 20:04:07 -0700
Generalize the INDEX_CLEANUP VACUUM parameter (and the corresponding
reloption): make it into a ternary style boolean parameter. It now
exposes a third option, "auto". The "auto" option (which is now the
default) enables the "bypass index vacuuming" optimization added by
commit 1e55e7d1.
"VACUUM (INDEX_CLEANUP TRUE)" is redefined to once again make VACUUM
simply do any required index vacuuming, regardless of how few dead
tuples are encountered during the first scan of the target heap relation
(unless there are exactly zero). This gives users a way of opting out
of the "bypass index vacuuming" optimization, if for whatever reason
that proves necessary. It is also expected to be used by PostgreSQL
developers as a testing option from time to time.
"VACUUM (INDEX_CLEANUP FALSE)" does the same thing as it always has: it
forcibly disables both index vacuuming and index cleanup. It's not
expected to be used much in PostgreSQL 14. The failsafe mechanism added
by commit 1e55e7d1 addresses the same problem in a simpler way.
INDEX_CLEANUP can now be thought of as a testing and compatibility
option.
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Masahiko Sawada <[email protected]>
Reviewed-By: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/CAH2-WznrBoCST4_Gxh_G9hA8NzGUbeBGnOUC8FcXcrhqsv6OHQ@mail.gmail.com
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/vacuum.sgml
M doc/src/sgml/ref/vacuumdb.sgml
M src/backend/access/common/reloptions.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/vacuum.c
M src/backend/postmaster/autovacuum.c
M src/bin/psql/tab-complete.c
M src/bin/scripts/vacuumdb.c
M src/include/commands/vacuum.h
M src/include/utils/rel.h
M src/test/regress/expected/vacuum.out
M src/test/regress/sql/vacuum.sql
Add test case for obsoleting slot with active walsender
commit : 09126984a2631db8dd6299f23954e0dede69735f
author : Alvaro Herrera <[email protected]>
date : Fri, 18 Jun 2021 18:42:00 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 18 Jun 2021 18:42:00 -0400
The code to signal a running walsender when its reserved WAL size grows
too large is completely uncovered before this commit; this adds coverage
for that case.
This test involves sending SIGSTOP to walsender and walreceiver and
running a checkpoint while advancing WAL, then sending SIGCONT. There's
no precedent for this coding in Perl tests, and my reading of relevant
manpages says it's likely to fail on Windows. Because of this, this
test is always skipped on that platform.
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/019_replslot_limit.pl
Fix misbehavior of DROP OWNED BY with duplicate polroles entries.
commit : d21fca084356946664bfce19d66d2df2bb873cbd
author : Tom Lane <[email protected]>
date : Fri, 18 Jun 2021 18:00:09 -0400
committer: Tom Lane <[email protected]>
date : Fri, 18 Jun 2021 18:00:09 -0400
Ordinarily, a pg_policy.polroles array wouldn't list the same role
more than once; but CREATE POLICY does not prevent that. If we
perform DROP OWNED BY on a role that is listed more than once,
RemoveRoleFromObjectPolicy either suffered an assertion failure
or encountered a tuple-updated-by-self error. Rewrite it to cope
correctly with duplicate entries, and add a CommandCounterIncrement
call to prevent the other problem.
Per discussion, there's other cleanup that ought to happen here,
but this seems like the minimum essential fix.
Per bug #17062 from Alexander Lakhin. It's been broken all along,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/policy.c
M src/test/regress/expected/rowsecurity.out
M src/test/regress/sql/rowsecurity.sql
Improve version reporting in pgbench.
commit : 84bee9610965331d5110971d8de390a5bbe2effc
author : Tom Lane <[email protected]>
date : Fri, 18 Jun 2021 17:05:23 -0400
committer: Tom Lane <[email protected]>
date : Fri, 18 Jun 2021 17:05:23 -0400
Commit 547f04e73 caused pgbench to start printing its version number,
which seems like a fine idea, but it needs a bit more work:
* Print the server version number too, when different.
* Print the PG_VERSION string, not some reconstructed approximation.
This patch copies psql's well-tested code for the same purpose.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
Centralize the logic for protective copying of utility statements.
commit : 7c337b6b527b7052e6a751f966d5734c56f668b5
author : Tom Lane <[email protected]>
date : Fri, 18 Jun 2021 11:22:58 -0400
committer: Tom Lane <[email protected]>
date : Fri, 18 Jun 2021 11:22:58 -0400
In the "simple Query" code path, it's fine for parse analysis or
execution of a utility statement to scribble on the statement's node
tree, since that'll just be thrown away afterwards. However it's
not fine if the node tree is in the plan cache, as then it'd be
corrupted for subsequent executions. Up to now we've dealt with
that by having individual utility-statement functions apply
copyObject() if they were going to modify the tree. But that's
prone to errors of omission. Bug #17053 from Charles Samborski
shows that CREATE/ALTER DOMAIN didn't get this memo, and can
crash if executed repeatedly from plan cache.
In the back branches, we'll just apply a narrow band-aid for that,
but in HEAD it seems prudent to have a more principled fix that
will close off the possibility of other similar bugs in future.
Hence, let's hoist the responsibility for doing copyObject up into
ProcessUtility from its children, thus ensuring that it happens for
all utility statement types.
Also, modify ProcessUtility's API so that its callers can tell it
whether a copy step is necessary. It turns out that in all cases,
the immediate caller knows whether the node tree is transient, so
this doesn't involve a huge amount of code thrashing. In this way,
while we lose a little bit in the execute-from-cache code path due
to sometimes copying node trees that wouldn't be mutated anyway,
we gain something in the simple-Query code path by not copying
throwaway node trees. Statements that are complex enough to be
expensive to copy are almost certainly ones that would have to be
copied anyway, so the loss in the cache code path shouldn't be much.
(Note that this whole problem applies only to utility statements.
Optimizable statements don't have the issue because we long ago made
the executor treat Plan trees as read-only. Perhaps someday we will
make utility statement execution act likewise, but I'm not holding
my breath.)
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/sepgsql/hooks.c
M src/backend/commands/copyto.c
M src/backend/commands/createas.c
M src/backend/commands/explain.c
M src/backend/commands/extension.c
M src/backend/commands/foreigncmds.c
M src/backend/commands/policy.c
M src/backend/commands/portalcmds.c
M src/backend/commands/prepare.c
M src/backend/commands/schemacmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/view.c
M src/backend/executor/functions.c
M src/backend/executor/spi.c
M src/backend/parser/parse_utilcmd.c
M src/backend/tcop/pquery.c
M src/backend/tcop/utility.c
M src/include/tcop/utility.h
Don't set a fast default for anything but a plain table
commit : 0a4efdc7ebf2584257b166c87e82797eb92815b5
author : Andrew Dunstan <[email protected]>
date : Fri, 18 Jun 2021 06:51:12 -0400
committer: Andrew Dunstan <[email protected]>
date : Fri, 18 Jun 2021 06:51:12 -0400
The fast default code added in Release 11 omitted to check that the
table a fast default was being added to was a plain table. Thus one
could be added to a foreign table, which predicably blows up. Here we
perform that check.
In addition, on the back branches, since some of these might have
escaped into the wild, if we encounter a missing value for
an attribute of something other than a plain table we ignore it.
Fixes bug #17056
Backpatch to release 11,
Reviewed by: Andres Freund, Álvaro Herrera and Tom Lane
M src/backend/catalog/heap.c
M src/test/regress/expected/fast_default.out
M src/test/regress/sql/fast_default.sql
Make archiver process handle barrier events.
commit : 981524d2e3aa3f28d364c472e34f5386be846811
author : Fujii Masao <[email protected]>
date : Fri, 18 Jun 2021 17:57:09 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 18 Jun 2021 17:57:09 +0900
Commit d75288fb27 made WAL archiver process an auxiliary process.
An auxiliary process needs to handle barrier events but the commit
forgot to make archiver process do that.
Reported-by: Thomas Munro
Author: Fujii Masao
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/CA+hUKGLah2w1pWKHonZP_+EQw69=q56AHYwCgEN8GDzsRG_Hgw@mail.gmail.com
M src/backend/postmaster/pgarch.c
doc: Apply markup <productname> to OpenSSL more consistently
commit : f80979f659d39e238e95444e6752142799428078
author : Michael Paquier <[email protected]>
date : Fri, 18 Jun 2021 14:22:31 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 18 Jun 2021 14:22:31 +0900
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M doc/src/sgml/libpq.sgml
Tidy up GetMultiXactIdMembers()'s behavior on error
commit : d24c5658a80c8f5037e9e1948de311d3f3350f12
author : Heikki Linnakangas <[email protected]>
date : Thu, 17 Jun 2021 14:50:42 +0300
committer: Heikki Linnakangas <[email protected]>
date : Thu, 17 Jun 2021 14:50:42 +0300
One of the error paths left *members uninitialized. That's not a live
bug, because most callers don't look at *members when the function
returns -1, but let's be tidy. One caller, in heap_lock_tuple(), does
"if (members != NULL) pfree(members)", but AFAICS it never passes an
invalid 'multi' value so it should not reach that error case.
The callers are also a bit inconsistent in their expectations.
heap_lock_tuple() pfrees the 'members' array if it's not-NULL, others
pfree() it if "nmembers >= 0", and others if "nmembers > 0". That's
not a live bug either, because the function should never return 0, but
add an Assert for that to make it more clear. I left the callers alone
for now.
I also moved the line where we set *nmembers. It wasn't wrong before,
but I like to do that right next to the 'return' statement, to make it
clear that it's always set on return.
Also remove one unreachable return statement after ereport(ERROR), for
brevity and for consistency with the similar if-block right after it.
Author: Greg Nancarrow with the additional changes by me
Backpatch-through: 9.6, all supported versions
M src/backend/access/transam/multixact.c
Document a few caveats in synchronous logical replication.
commit : 3cb828dbe26087e7754f49f3cfe3ed036d5af439
author : Amit Kapila <[email protected]>
date : Thu, 17 Jun 2021 09:56:05 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 17 Jun 2021 09:56:05 +0530
In a synchronous logical setup, locking [user] catalog tables can cause
deadlock. This is because logical decoding of transactions can lock
catalog tables to access them so exclusively locking those in transactions
can lead to deadlock. To avoid this users must refrain from having
exclusive locks on catalog tables.
Author: Takamichi Osumi
Reviewed-by: Vignesh C, Amit Kapila
Backpatch-through: 9.6
Discussion: https://www.postgresql.org/message-id/20210222222847.tpnb6eg3yiykzpky%40alap3.anarazel.de
M doc/src/sgml/logicaldecoding.sgml
Fix plancache refcount leak after error in ExecuteQuery.
commit : 131ea3e908d3c97a2fe1ab25cce5046dd5cb905f
author : Tom Lane <[email protected]>
date : Wed, 16 Jun 2021 19:30:17 -0400
committer: Tom Lane <[email protected]>
date : Wed, 16 Jun 2021 19:30:17 -0400
When stuffing a plan from the plancache into a Portal, one is
not supposed to risk throwing an error between GetCachedPlan and
PortalDefineQuery; if that happens, the plan refcount incremented
by GetCachedPlan will be leaked. I managed to break this rule
while refactoring code in 9dbf2b7d7. There is no visible
consequence other than some memory leakage, and since nobody is
very likely to trigger the relevant error conditions many times
in a row, it's not surprising we haven't noticed. Nonetheless,
it's a bug, so rearrange the order of operations to remove the
hazard.
Noted on the way to looking for a better fix for bug #17053.
This mistake is pretty old, so back-patch to all supported
branches.
M src/backend/commands/prepare.c
Fix copying data into slots with FDW batching
commit : 99cea49d6525e30bc3768e4ffb95377e7584dea1
author : Tomas Vondra <[email protected]>
date : Wed, 16 Jun 2021 22:53:31 +0200
committer: Tomas Vondra <[email protected]>
date : Wed, 16 Jun 2021 22:53:31 +0200
Commit b676ac443b optimized handling of tuple slots with bulk inserts
into foreign tables, so that the slots are initialized only once and
reused for all batches. The data was however copied into the slots only
after the initialization, inserting duplicate values when the slot gets
reused. Fixed by moving the ExecCopySlot outside the init branch.
The existing postgres_fdw tests failed to catch this due to inserting
data into foreign tables without unique indexes, and then checking only
the number of inserted rows. This adds a new test with both a unique
index and a check of inserted values.
Reported-by: Alexander Pyhalov
Discussion: https://postgr.es/m/7a8cf8d56b3d18e5c0bccd6cd42d04ac%40postgrespro.ru
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/executor/nodeModifyTable.c
Improve SQLSTATE reporting in some replication-related code.
commit : 6b787d9e32005867ee3660d1ea20f447810a403d
author : Tom Lane <[email protected]>
date : Wed, 16 Jun 2021 11:52:05 -0400
committer: Tom Lane <[email protected]>
date : Wed, 16 Jun 2021 11:52:05 -0400
I started out with the goal of reporting ERRCODE_CONNECTION_FAILURE
when walrcv_connect() fails, but as I looked around I realized that
whoever wrote this code was of the opinion that errcodes are purely
optional. That's not my understanding of our project policy. Hence,
make sure that an errcode is provided in each ereport that (a) is
ERROR or higher level and (b) isn't arguably an internal logic error.
Also fix some very dubious existing errcode assignments.
While this is not per policy, it's also largely cosmetic, since few
of these cases could get reported to applications. So I don't
feel a need to back-patch.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/subscriptioncmds.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/backend/replication/walreceiver.c
Fix outdated comment that talked about seek position of WAL file.
commit : d0303bc8d2670d11c9df9220aa08a2c33529e100
author : Heikki Linnakangas <[email protected]>
date : Wed, 16 Jun 2021 12:34:32 +0300
committer: Heikki Linnakangas <[email protected]>
date : Wed, 16 Jun 2021 12:34:32 +0300
Since commit c24dcd0cfd, we have been using pg_pread() to read the WAL
file, which doesn't change the seek position (unless we fall back to
the implementation in src/port/pread.c). Update comment accordingly.
Backpatch-through: 12, where we started to use pg_pread()
M src/backend/access/transam/xlog.c
Update another variant expected-result file.
commit : d3c878499c9d639ff06e0664d06b8c731e30c2fc
author : Tom Lane <[email protected]>
date : Tue, 15 Jun 2021 16:11:45 -0400
committer: Tom Lane <[email protected]>
date : Tue, 15 Jun 2021 16:11:45 -0400
This should have been updated in 533e9c6b0, but it was overlooked.
Given the lack of complaints, I won't bother back-patching.
M src/test/isolation/expected/lock-update-delete_1.out
Remove another orphan expected-result file.
commit : f6352a0d4e437ac8bc266f77df22d064592056c9
author : Tom Lane <[email protected]>
date : Tue, 15 Jun 2021 16:09:14 -0400
committer: Tom Lane <[email protected]>
date : Tue, 15 Jun 2021 16:09:14 -0400
aborted-keyrevoke_2.out was apparently needed when it was added (in
commit 0ac5ad513) to handle the case of serializable transaction mode.
However, the output in serializable mode actually matches the regular
aborted-keyrevoke.out file, and AFAICT has done so for a long time.
There's no need to keep dragging this variant along.
D src/test/isolation/expected/aborted-keyrevoke_2.out
Further refinement of stuck_on_old_timeline recovery test
commit : 54a5ed22016940d7ad5060ed62d23473924756a1
author : Andrew Dunstan <[email protected]>
date : Tue, 15 Jun 2021 15:30:11 -0400
committer: Andrew Dunstan <[email protected]>
date : Tue, 15 Jun 2021 15:30:11 -0400
TestLib::perl2host can take a file argument as well as a directory
argument, so that code becomes substantially simpler. Also add comments
on why we're using forward slashes, and why we're setting
PERL_BADLANG=0.
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/025_stuck_on_old_timeline.pl
Revert 29854ee8d1 due to buildfarm failures
commit : 817bb0a7d1e02df4643d37304aed8574cf43f629
author : Alexander Korotkov <[email protected]>
date : Tue, 15 Jun 2021 21:43:17 +0300
committer: Alexander Korotkov <[email protected]>
date : Tue, 15 Jun 2021 21:43:17 +0300
Reported-by: Tom Lane
Discussion: https://postgr.es/m/CAPpHfdvcnw3x7jdV3r52p4%3D5S4WUxBCzcQKB3JukQHoicv1LSQ%40mail.gmail.com
M doc/src/sgml/func.sgml
M doc/src/sgml/rangetypes.sgml
M src/backend/commands/typecmds.c
M src/backend/utils/adt/multirangetypes.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_cast.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/multirangetypes.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/sql/multirangetypes.sql
M src/test/regress/sql/opr_sanity.sql
Remove unneeded field from VACUUM state.
commit : 958cfbcf2dd338e3179c2d8a35f48bde020eba60
author : Peter Geoghegan <[email protected]>
date : Tue, 15 Jun 2021 08:59:36 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 15 Jun 2021 08:59:36 -0700
Bugfix commit 5fc89376 effectively made the lock_waiter_detected field
from vacuumlazy.c's global state struct into private state owned by
lazy_truncate_heap(). Finish this off by replacing the struct field
with a local variable.
M src/backend/access/heap/vacuumlazy.c
Add missing type name "multirange" in docs chapter title
commit : ad2da246c69dd5ec55764d4b6066f3b0c0d24ca2
author : Alexander Korotkov <[email protected]>
date : Tue, 15 Jun 2021 16:06:32 +0300
committer: Alexander Korotkov <[email protected]>
date : Tue, 15 Jun 2021 16:06:32 +0300
Discussion: https://postgr.es/m/CAFj8pRDioOxiJgmgw9TqQqZ3CxnJC4P5B2Oospf2eMgAjJuewA%40mail.gmail.com
Author: Pavel Stehule, Alexander Korotkov
Reviewed-by: Justin Pryzby, Tom Lane
M doc/src/sgml/func.sgml
Support for unnest(multirange) and cast multirange as an array of ranges
commit : 29854ee8d1ca4a46adb7e84deb17e6fb18e531cc
author : Alexander Korotkov <[email protected]>
date : Tue, 15 Jun 2021 15:59:20 +0300
committer: Alexander Korotkov <[email protected]>
date : Tue, 15 Jun 2021 15:59:20 +0300
It has been spotted that multiranges lack of ability to decompose them into
individual ranges. Subscription and proper expanded object representation
require substantial work, and it's too late for v14. This commit
provides the implementation of unnest(multirange) and cast multirange as
an array of ranges, which is quite trivial.
unnest(multirange) is defined as a polymorphic procedure. The catalog
description of the cast underlying procedure is duplicated for each multirange
type because we don't have anyrangearray polymorphic type to use here.
Catversion is bumped.
Reported-by: Jonathan S. Katz
Discussion: https://postgr.es/m/flat/60258efe-bd7e-4886-82e1-196e0cac5433%40postgresql.org
Author: Alexander Korotkov
Reviewed-by: Justin Pryzby, Jonathan S. Katz, Zhihong Yu
M doc/src/sgml/func.sgml
M doc/src/sgml/rangetypes.sgml
M src/backend/commands/typecmds.c
M src/backend/utils/adt/multirangetypes.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_cast.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/multirangetypes.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/sql/multirangetypes.sql
M src/test/regress/sql/opr_sanity.sql
Fix decoding of speculative aborts.
commit : 4daa140a2f50e9a160bc180c3997ee13c7aabf9e
author : Amit Kapila <[email protected]>
date : Tue, 15 Jun 2021 08:28:36 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 15 Jun 2021 08:28:36 +0530
During decoding for speculative inserts, we were relying for cleaning
toast hash on confirmation records or next change records. But that
could lead to multiple problems (a) memory leak if there is neither a
confirmation record nor any other record after toast insertion for a
speculative insert in the transaction, (b) error and assertion failures
if the next operation is not an insert/update on the same table.
The fix is to start queuing spec abort change and clean up toast hash
and change record during its processing. Currently, we are queuing the
spec aborts for both toast and main table even though we perform cleanup
while processing the main table's spec abort record. Later, if we have a
way to distinguish between the spec abort record of toast and the main
table, we can avoid queuing the change for spec aborts of toast tables.
Reported-by: Ashutosh Bapat
Author: Dilip Kumar
Reviewed-by: Amit Kapila
Backpatch-through: 9.6, where it was introduced
Discussion: https://postgr.es/m/CAExHW5sPKF-Oovx_qZe4p5oM6Dvof7_P+XgsNAViug15Fm99jA@mail.gmail.com
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h
Update variant expected-result file.
commit : 0a1e80c5c4f094087257fc4284a87e0bc7bca591
author : Tom Lane <[email protected]>
date : Mon, 14 Jun 2021 21:58:26 -0400
committer: Tom Lane <[email protected]>
date : Mon, 14 Jun 2021 21:58:26 -0400
This should have been updated in d2d8a229b, but it was overlooked.
According to 31a877f18 which added it, this file is meant to show the
results you get under default_transaction_isolation = serializable.
We've largely lost track of that goal in other isolation tests, but
as long as we've got this one, it should be right.
Noted while fooling about with the isolationtester.
M src/test/isolation/expected/drop-index-concurrently-1_2.out
Remove orphaned expected-result file.
commit : ffbe9dec13599fa786ea6567df1c6a3f3ee3c673
author : Tom Lane <[email protected]>
date : Mon, 14 Jun 2021 21:28:21 -0400
committer: Tom Lane <[email protected]>
date : Mon, 14 Jun 2021 21:28:21 -0400
This should have been removed in 43e084197, which removed the
corresponding spec file. Noted while fooling about with the
isolationtester.
D src/test/isolation/expected/insert-conflict-toast_1.out
Remove pg_wait_for_backend_termination().
commit : 5f1df62a459b51ab3bb625a0ee5e655429254257
author : Noah Misch <[email protected]>
date : Mon, 14 Jun 2021 17:29:37 -0700
committer: Noah Misch <[email protected]>
date : Mon, 14 Jun 2021 17:29:37 -0700
It was unable to wait on a backend that had already left the procarray.
Users tolerant of that limitation can poll pg_stat_activity. Other
users can employ the "timeout" argument of pg_terminate_backend().
Reviewed by Bharath Rupireddy.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M doc/src/sgml/release-14.sgml
M src/backend/catalog/system_functions.sql
M src/backend/storage/ipc/signalfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Copy-edit text for the pg_terminate_backend() "timeout" parameter.
commit : 0aac73e6a2602696d23aa7a9686204965f9093dc
author : Noah Misch <[email protected]>
date : Mon, 14 Jun 2021 17:29:37 -0700
committer: Noah Misch <[email protected]>
date : Mon, 14 Jun 2021 17:29:37 -0700
Revert the pg_description entry to its v13 form, since those messages
usually remain shorter and don't discuss individual parameters. No
catversion bump, since pg_description content does not impair backend
compatibility or application compatibility.
Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/storage/ipc/signalfuncs.c
M src/include/catalog/pg_proc.dat
Fix logic bug in 1632ea43682f
commit : 33c509956704e1d918077b51ccd954f2e201a8f5
author : Alvaro Herrera <[email protected]>
date : Mon, 14 Jun 2021 16:31:12 -0400
committer: Alvaro Herrera <[email protected]>
date : Mon, 14 Jun 2021 16:31:12 -0400
I overlooked that one condition was logically inverted. The fix is a
little bit more involved than simply negating the condition, to make
the code easier to read.
Fix some outdated comments left by the same commit, while at it.
Author: Masahiko Sawada <[email protected]>
Author: Álvaro Herrera <[email protected]>
Reviewed-by: Amit Kapila <[email protected]>
Discussion: https://postgr.es/m/YMRlmB3/[email protected]
M src/backend/replication/slot.c
doc: PG 14 relnotes fixes
commit : 86b222b09071d3918c5c55b164d22b2236d3f872
author : Bruce Momjian <[email protected]>
date : Mon, 14 Jun 2021 16:14:04 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 14 Jun 2021 16:14:04 -0400
Items related to logical replication attribution and BRIN indexes
Reported-by: Tomas Vondra, John Naylor
Discussion: https://postgr.es/m/[email protected], CAFBsxsH21KnteYdk33F1oZu2O726NSD6_XBq51Tn0jytsA1AnA@mail.gmail.com
M doc/src/sgml/release-14.sgml
doc: PG 14 relnote updates
commit : a2559d4093725592a3fd5da8a4c7ac7c883115a0
author : Bruce Momjian <[email protected]>
date : Mon, 14 Jun 2021 16:03:18 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 14 Jun 2021 16:03:18 -0400
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/release-14.sgml
doc: add PG 14 relnote item about array function references
commit : 25dfb5a831a1b8a83a8a68453b4bbe38a5ef737e
author : Bruce Momjian <[email protected]>
date : Mon, 14 Jun 2021 12:49:05 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 14 Jun 2021 12:49:05 -0400
User-defined objects that reference some built-in array functions will
need to be recreated in PG 14.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/release-14.sgml
Improve handling of dropped objects in pg_event_trigger_ddl_commands()
commit : 2d689babe3cb50dcb29f6ed595a61d56e518c0d8
author : Michael Paquier <[email protected]>
date : Mon, 14 Jun 2021 14:57:22 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 14 Jun 2021 14:57:22 +0900
An object found as dropped when digging into the list of objects
returned by pg_event_trigger_ddl_commands() could cause a cache lookup
error, as the calls grabbing for the object address and the type name
would fail if the object was missing.
Those lookup errors could be seen with combinations of ALTER TABLE
sub-commands involving identity columns. The lookup logic is changed in
this code path to get a behavior similar to any other SQL-callable
function by ignoring objects that are not found, taking advantage of
2a10fdc. The back-branches are not changed, as they require this commit
that is too invasive for stable branches.
While on it, add test cases to exercise event triggers with identity
columns, and stress more cases with the event ddl_command_end for
relations.
Author: Sven Klemm, Aleksander Alekseev, Michael Paquier
Discussion: https://postgr.es/m/CAMCrgp2R1cEXU53iYKtW6yVEp2_yKUz+z=3-CTrYpPP+xryRtg@mail.gmail.com
M src/backend/commands/event_trigger.c
M src/test/regress/expected/event_trigger.out
M src/test/regress/sql/event_trigger.sql
Remove forced toast recompression in VACUUM FULL/CLUSTER
commit : dbab0c07e5ba1f19a991da2d72972a8fe9a41bda
author : Michael Paquier <[email protected]>
date : Mon, 14 Jun 2021 09:25:50 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 14 Jun 2021 09:25:50 +0900
The extra checks added by the recompression of toast data introduced in
bbe0a81 is proving to have a performance impact on VACUUM or CLUSTER
even if no recompression is done. This is more noticeable with more
toastable columns that contain non-NULL values.
Improvements could be done to make those extra checks less expensive,
but that's not material for 14 at this stage, and we are not sure either
if the code path of VACUUM FULL/CLUSTER is adapted for this job.
Per discussion with several people, including Andres Freund, Robert
Haas, Álvaro Herrera, Tom Lane and myself.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/alter_table.sgml
M src/backend/access/heap/heapam_handler.c
M src/test/regress/expected/compression.out
M src/test/regress/expected/compression_1.out
M src/test/regress/sql/compression.sql
Work around portability issue with newer versions of mktime().
commit : f807e3410fdfc29ced6590c7c2afa76637e001ad
author : Tom Lane <[email protected]>
date : Sun, 13 Jun 2021 14:32:42 -0400
committer: Tom Lane <[email protected]>
date : Sun, 13 Jun 2021 14:32:42 -0400
Recent glibc versions have made mktime() fail if tm_isdst is
inconsistent with the prevailing timezone; in particular it fails for
tm_isdst = 1 when the zone is UTC. (This seems wildly inconsistent
with the POSIX-mandated treatment of "incorrect" values for the other
fields of struct tm, so if you ask me it's a bug, but I bet they'll
say it's intentional.) This has been observed to cause cosmetic
problems when pg_restore'ing an archive created in a different
timezone.
To fix, do mktime() using the field values from the archive, and if
that fails try again with tm_isdst = -1. This will give a result
that's off by the UTC-offset difference from the original zone, but
that was true before, too. It's not terribly critical since we don't
do anything with the result except possibly print it. (Someday we
should flush this entire bit of logic and record a standard-format
timestamp in the archive instead. That's not okay for a back-patched
bug fix, though.)
Also, guard our only other use of mktime() by having initdb's
build_time_t() set tm_isdst = -1 not 0. This case could only have
an issue in zones that are DST year-round; but I think some do exist,
or could in future.
Per report from Wells Oliver. Back-patch to all supported
versions, since any of them might need to run with a newer glibc.
Discussion: https://postgr.es/m/CAOC+FBWDhDHO7G-i1_n_hjRzCnUeFO+H-Czi1y10mFhRWpBrew@mail.gmail.com
M src/bin/initdb/findtimezone.c
M src/bin/pg_dump/pg_backup_archiver.c
Further tweaks to stuck_on_old_timeline recovery test
commit : 9d97c3408319b43718e4b85bc694697db1af32c6
author : Andrew Dunstan <[email protected]>
date : Sun, 13 Jun 2021 07:10:41 -0400
committer: Andrew Dunstan <[email protected]>
date : Sun, 13 Jun 2021 07:10:41 -0400
Translate path slashes on target directory path. This was confusing old
branches, but is applied to all branches for the sake of uniformity.
Perl is perfectly able to understand paths with forward slashes.
Along the way, restore the previous archive_wait query, for the sake of
uniformity with other tests, per gripe from Tom Lane.
M src/test/recovery/t/025_stuck_on_old_timeline.pl
Ignore more environment variables in pg_regress.c
commit : a9e0b3b08fe38d5e31f03ea96859ff5e413d4a38
author : Michael Paquier <[email protected]>
date : Sun, 13 Jun 2021 20:07:39 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 13 Jun 2021 20:07:39 +0900
This is similar to the work done in 8279f68 for TestLib.pm, where
environment variables set may cause unwanted failures if using a
temporary installation with pg_regress. The list of variables reset is
adjusted in each stable branch depending on what is supported.
Comments are added to remember that the lists in TestLib.pm and
pg_regress.c had better be kept in sync.
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/test/perl/TestLib.pm
M src/test/regress/pg_regress.c
Restore robustness of TAP tests that wait for postmaster restart.
commit : f452aaf7d4a96cfcecd6c60ccd294ffe7b8ea088
author : Tom Lane <[email protected]>
date : Sat, 12 Jun 2021 15:12:10 -0400
committer: Tom Lane <[email protected]>
date : Sat, 12 Jun 2021 15:12:10 -0400
Several TAP tests use poll_query_until() to wait for the postmaster
to restart. They were checking to see if a trivial query
(e.g. "SELECT 1") succeeds. However, that's problematic in the wake
of commit 11e9caff8, because now that we feed said query to psql
via stdin, we risk IPC::Run whining about a SIGPIPE failure if psql
quits before reading the query. Hence, we can't use a nonempty
query in cases where we need to wait for connection failures to
stop happening.
Per the precedent of commits c757a3da0 and 6d41dd045, we can pass
"undef" as the query in such cases to ensure that IPC::Run has
nothing to write. However, then we have to say that the expected
output is empty, and this exposes a deficiency in poll_query_until:
if psql fails altogether and returns empty stdout, poll_query_until
will treat that as a success! That's because, contrary to its
documentation, it makes no actual check for psql failure, looking
neither at the exit status nor at stderr.
To fix that, adjust poll_query_until to insist on empty stderr as
well as a stdout match. (I experimented with checking exit status
instead, but it seems that psql often does exit(1) in cases that we
need to consider successes. That might be something to fix someday,
but it would be a non-back-patchable behavior change.)
Back-patch to v10. The test cases needing this exist only as far
back as v11, but it seems wise to keep poll_query_until's behavior
the same in v10, in case we back-patch another such test case in
future. (9.6 does not currently need this change, because in that
branch poll_query_until can't be told to accept empty stdout as
a success case.)
Per assorted buildfarm failures, mostly on hoverfly.
Discussion: https://postgr.es/m/CAA4eK1+zM6L4QSA1XMvXY_qqWwdUmqkOS1+hWvL8QcYEBGA1Uw@mail.gmail.com
M src/test/perl/PostgresNode.pm
M src/test/recovery/t/013_crash_restart.pl
M src/test/recovery/t/022_crash_temp_files.pl
Ensure pg_filenode_relation(0, 0) returns NULL.
commit : 1250aad42520fd5a3db68d6381997b7e1f9bb4aa
author : Tom Lane <[email protected]>
date : Sat, 12 Jun 2021 13:29:24 -0400
committer: Tom Lane <[email protected]>
date : Sat, 12 Jun 2021 13:29:24 -0400
Previously, a zero value for the relfilenode resulted in
a confusing error message about "unexpected duplicate".
This function returns NULL for other invalid relfilenode
values, so zero should be treated likewise.
It's been like this all along, so back-patch to all supported
branches.
Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/dbsize.c
Don't use Asserts to check for violations of replication protocol.
commit : fe6a20ce54cbbb6fcfe9f6675d563af836ae799a
author : Tom Lane <[email protected]>
date : Sat, 12 Jun 2021 12:59:15 -0400
committer: Tom Lane <[email protected]>
date : Sat, 12 Jun 2021 12:59:15 -0400
Using an Assert to check the validity of incoming messages is an
extremely poor decision. In a debug build, it should not be that easy
for a broken or malicious remote client to crash the logrep worker.
The consequences could be even worse in non-debug builds, which will
fail to make such checks at all, leading to who-knows-what misbehavior.
Hence, promote every Assert that could possibly be triggered by wrong
or out-of-order replication messages to a full test-and-ereport.
To avoid bloating the set of messages the translation team has to cope
with, establish a policy that replication protocol violation error
reports don't need to be translated. Hence, all the new messages here
use errmsg_internal(). A couple of old messages are changed likewise
for consistency.
Along the way, fix some non-idiomatic or outright wrong uses of
hash_search().
Most of these mistakes are new with the "streaming replication"
patch (commit 464824323), but a couple go back a long way.
Back-patch as appropriate.
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/worker.c
Fix new recovery test for use under msys
commit : c3652f976b7696a96a9c5606cc2d613af77e2e63
author : Andrew Dunstan <[email protected]>
date : Sat, 12 Jun 2021 08:37:16 -0400
committer: Andrew Dunstan <[email protected]>
date : Sat, 12 Jun 2021 08:37:16 -0400
Commit caba8f0d43 wasn't quite right for msys, as demonstrated by
several buildfarm animals, including jacana and fairywren. We need to
use the msys perl in the archive command, but call it in such a way that
Windows will understand the path. Furthermore, inside the copy script we
need to convert a Windows path to an msys path.
M src/test/recovery/t/025_stuck_on_old_timeline.pl
M src/test/recovery/t/cp_history_files
Simplify some code in getObjectTypeDescription()
commit : b56b83aa0d6e044cf38d553f7a87f4b84708cac6
author : Michael Paquier <[email protected]>
date : Sat, 12 Jun 2021 16:29:11 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 12 Jun 2021 16:29:11 +0900
This routine is designed to never return an empty description or NULL,
providing description fallbacks even if missing objects are accepted,
but it included a code path where this was considered possible. All the
callers of this routine already consider NULL as not possible, so
change a bit the code to map with the assumptions of the callers, and
add more comments close to the callers of this routine to outline the
behavior expected.
This code is new as of 2a10fdc, so no backpatch is needed.
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/objectaddress.c
Improve log pattern detection in recently-added TAP tests
commit : bfd96b7a3dc26a8384d4185d274573fb6a46b033
author : Michael Paquier <[email protected]>
date : Sat, 12 Jun 2021 15:29:48 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 12 Jun 2021 15:29:48 +0900
ab55d74 has introduced some tests with rows found as missing in logical
replication subscriptions for partitioned tables, relying on a logic
with a lookup of the logs generated, scanning the whole file. This
commit makes the logic more precise, by scanning the logs only from the
position before the key queries are run to the position where we check
for the logs. This will reduce the risk of issues with log patterns
overlapping with each other if those tests get more complicated in the
future.
Per discussion with Tom Lane.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/013_partition.pl
Improve and cleanup ProcArrayAdd(), ProcArrayRemove().
commit : d8e950d3ae7b33a2064a4fb39b7829334b0b47bc
author : Andres Freund <[email protected]>
date : Fri, 11 Jun 2021 21:22:21 -0700
committer: Andres Freund <[email protected]>
date : Fri, 11 Jun 2021 21:22:21 -0700
941697c3c1a changed ProcArrayAdd()/Remove() substantially. As reported by
zhanyi, I missed that due to the introduction of the PGPROC->pgxactoff
ProcArrayRemove() does not need to search for the position in
procArray->pgprocnos anymore - that's pgxactoff. Remove the search loop.
The removal of the search loop reduces assertion coverage a bit - but we can
easily do better than before by adding assertions to other loops over
pgprocnos elements.
Also do a bit of cleanup, mostly by reducing line lengths by introducing local
helper variables and adding newlines.
Author: zhanyi <[email protected]>
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/procarray.c
doc: remove extra right angle bracket in PG 14 relnotes
commit : d120e66fac87f768ea2289d2d923d0ee4262ec0f
author : Bruce Momjian <[email protected]>
date : Fri, 11 Jun 2021 21:01:52 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 11 Jun 2021 21:01:52 -0400
Reported-by: Justin Pryzby
M doc/src/sgml/release-14.sgml
docs: fix incorrect indenting in PG 14 relnotes
commit : 0725913982e5e06895a32a9eeea2c59a13978927
author : Bruce Momjian <[email protected]>
date : Fri, 11 Jun 2021 19:51:35 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 11 Jun 2021 19:51:35 -0400
M doc/src/sgml/release-14.sgml
Report sort phase progress in parallel btree build
commit : 5cc1cd502879d642da799e1fd12619d83d987369
author : Alvaro Herrera <[email protected]>
date : Fri, 11 Jun 2021 19:07:32 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 11 Jun 2021 19:07:32 -0400
We were already reporting it, but only after the parallel workers were
finished, which is visibly much later than what happens in a serial
build.
With this change we report it when the leader starts its own sort phase
when participating in the build (the normal case). Now this might
happen a little later than when the workers start their sorting phases,
but a) communicating the actual phase start from workers is likely to be
a hassle, and b) the sort phase start is pretty fuzzy anyway, since
sorting per se is actually initiated by tuplesort.c internally earlier
than tuplesort_performsort() is called.
Backpatch to pg12, where the progress reporting code for CREATE INDEX
went in.
Reported-by: Tomas Vondra <[email protected]>
Author: Matthias van de Meent <[email protected]>
Reviewed-by: Greg Nancarrow <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/1128176d-1eee-55d4-37ca-e63644422adb
M src/backend/access/nbtree/nbtsort.c
Fix multiple crasher bugs in partitioned-table replication logic.
commit : ab55d742eb7162c22ee60f1e15e07d2a60063c4e
author : Tom Lane <[email protected]>
date : Fri, 11 Jun 2021 16:12:36 -0400
committer: Tom Lane <[email protected]>
date : Fri, 11 Jun 2021 16:12:36 -0400
apply_handle_tuple_routing(), having detected and reported that
the tuple it needed to update didn't exist, tried to update that
tuple anyway, leading to a null-pointer dereference.
logicalrep_partition_open() failed to ensure that the
LogicalRepPartMapEntry it built for a partition was fully
independent of that for the partition root, leading to
trouble if the root entry was later freed or rebuilt.
Meanwhile, on the publisher's side, pgoutput_change() sometimes
attempted to apply execute_attr_map_tuple() to a NULL tuple.
The first of these was reported by Sergey Bernikov in bug #17055;
I found the other two while developing some test cases for this
sadly under-tested code.
Diagnosis and patch for the first issue by Amit Langote; patches
for the others by me; new test cases by me. Back-patch to v13
where this logic came in.
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/relation.c
M src/backend/replication/logical/worker.c
M src/backend/replication/pgoutput/pgoutput.c
M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/013_partition.pl
Add 'Portal Close' message to pipelined PQsendQuery()
commit : 4efcf47053eaf8dd88de2b1a89478df43d37d5c0
author : Alvaro Herrera <[email protected]>
date : Fri, 11 Jun 2021 16:05:50 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 11 Jun 2021 16:05:50 -0400
Commit acb7e4eb6b1c added a new implementation for PQsendQuery so that
it works in pipeline mode (by using extended query protocol), but it
behaves differently from the 'Q' message (in simple query protocol) used
by regular implementation: the new one doesn't close the unnamed portal.
Change the new code to have identical behavior to the old.
Reported-by: Yura Sokolov <[email protected]>
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-exec.c
M src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
Return ReplicationSlotAcquire API to its original form
commit : 1632ea43682fcea8836ea245771ae85b9e1bcd38
author : Alvaro Herrera <[email protected]>
date : Fri, 11 Jun 2021 15:48:26 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 11 Jun 2021 15:48:26 -0400
Per 96540f80f833; the awkward API introduced by c6550776394e is no
longer needed.
Author: Andres Freund <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/logicalfuncs.c
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/walsender.c
M src/include/replication/slot.h
Optimize creation of slots for FDW bulk inserts
commit : b676ac443b6a83558d4701b2dd9491c0b37e17c4
author : Tomas Vondra <[email protected]>
date : Fri, 11 Jun 2021 20:19:48 +0200
committer: Tomas Vondra <[email protected]>
date : Fri, 11 Jun 2021 20:19:48 +0200
Commit b663a41363 introduced bulk inserts for FDW, but the handling of
tuple slots turned out to be problematic for two reasons. Firstly, the
slots were re-created for each individual batch. Secondly, all slots
referenced the same tuple descriptor - with reasonably small batches
this is not an issue, but with large batches this triggers O(N^2)
behavior in the resource owner code.
These two issues work against each other - to reduce the number of times
a slot has to be created/dropped, larger batches are needed. However,
the larger the batch, the more expensive the resource owner gets. For
practical batch sizes (100 - 1000) this would not be a big problem, as
the benefits (latency savings) greatly exceed the resource owner costs.
But for extremely large batches it might be much worse, possibly even
losing with non-batching mode.
Fixed by initializing tuple slots only once (and reusing them across
batches) and by using a new tuple descriptor copy for each slot.
Discussion: https://postgr.es/m/ebbbcc7d-4286-8c28-0272-61b4753af761%40enterprisedb.com
M src/backend/executor/nodeModifyTable.c
M src/include/nodes/execnodes.h
Fix race condition in invalidating obsolete replication slots
commit : 96540f80f8334a3f0f4a13f0d42e4565d8fa9eb7
author : Alvaro Herrera <[email protected]>
date : Fri, 11 Jun 2021 12:16:14 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 11 Jun 2021 12:16:14 -0400
The code added to mark replication slots invalid in commit c6550776394e
had the race condition that a slot can be dropped or advanced
concurrently with checkpointer trying to invalidate it. Rewrite the
code to close those races.
The changes to ReplicationSlotAcquire's API added with c6550776394e are
not necessary anymore. To avoid an ABI break in released branches, this
commit leaves that unchanged; it'll be changed in a master-only commit
separately.
Backpatch to 13, where this code first appeared.
Reported-by: Andres Freund <[email protected]>
Author: Andres Freund <[email protected]>
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/slot.c
Improve psql tab completion for options of subcriptions and publications
commit : d08237b5b494f96e72220bcef36a14a642969f16
author : Michael Paquier <[email protected]>
date : Fri, 11 Jun 2021 15:46:18 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 11 Jun 2021 15:46:18 +0900
The list of options provided by the tab completion was outdated for the
following commands:
- ALTER SUBSCRIPTION
- CREATE SUBSCRIPTION
- ALTER PUBLICATION
- CREATE PUBLICATION
Author: Vignesh C
Reviewed-by: Bharath Rupireddy
Discussion: https://postgr.es/m/CALDaNm18oHDFu6SFCHE=ZbiO153Fx7E-L1MG0YyScbaDV--U+A@mail.gmail.com
M src/bin/psql/tab-complete.c
Change position of field "transformed" in struct CreateStatsStmt.
commit : 13a1ca160dcfc316c9f4005891a312f5a84c5ca2
author : Noah Misch <[email protected]>
date : Thu, 10 Jun 2021 21:56:14 -0700
committer: Noah Misch <[email protected]>
date : Thu, 10 Jun 2021 21:56:14 -0700
Resolve the disagreement with nodes/*funcs.c field order in favor of the
latter, which is better-aligned with the IndexStmt field order. This
field is new in v14.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/parse_utilcmd.c
M src/include/nodes/parsenodes.h
Rename PQtraceSetFlags() to PQsetTraceFlags().
commit : d0e750c0acaf31f60667b1635311bcef5ab38bbe
author : Noah Misch <[email protected]>
date : Thu, 10 Jun 2021 21:56:13 -0700
committer: Noah Misch <[email protected]>
date : Thu, 10 Jun 2021 21:56:13 -0700
We have a dozen PQset*() functions. PQresultSetInstanceData() and this
were the libpq setter functions having a different word order. Adopt
the majority word order.
Reviewed by Alvaro Herrera and Robert Haas, though this choice of name
was not unanimous.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/exports.txt
M src/interfaces/libpq/fe-trace.c
M src/interfaces/libpq/libpq-fe.h
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Use the correct article for abbreviations
commit : 04539e73faaaaa1c06c1407671910dceaffdfcd4
author : David Rowley <[email protected]>
date : Fri, 11 Jun 2021 13:38:04 +1200
committer: David Rowley <[email protected]>
date : Fri, 11 Jun 2021 13:38:04 +1200
We've accumulated quite a mix of instances of "an SQL" and "a SQL" in the
documents. It would be good to be a bit more consistent with these.
The most recent version of the SQL standard I looked at seems to prefer
"an SQL". That seems like a good lead to follow, so here we change all
instances of "a SQL" to become "an SQL". Most instances correctly use
"an SQL" already, so it also makes sense to use the dominant variation in
order to minimise churn.
Additionally, there were some other abbreviations that needed to be
adjusted. FSM, SSPI, SRF and a few others. Also fix some pronounceable,
abbreviations to use "a" instead of "an". For example, "a SASL" instead
of "an SASL".
Here I've only adjusted the documents and error messages. Many others
still exist in source code comments. Translator hint comments seem to be
the biggest culprit. It currently does not seem worth the churn to change
these.
Discussion: https://postgr.es/m/CAApHDvpML27UqFXnrYO1MJddsKVMQoiZisPvsAGhKE_tsKXquw%40mail.gmail.com
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/dblink.sgml
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/ltree.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/pageinspect.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/plpython.sgml
M doc/src/sgml/pltcl.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/alter_opfamily.sgml
M doc/src/sgml/ref/create_function.sgml
M doc/src/sgml/ref/create_opclass.sgml
M doc/src/sgml/ref/create_statistics.sgml
M doc/src/sgml/ref/create_trigger.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pgbench.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/tablefunc.sgml
M doc/src/sgml/tablesample-method.sgml
M doc/src/sgml/textsearch.sgml
M doc/src/sgml/trigger.sgml
M doc/src/sgml/typeconv.sgml
M doc/src/sgml/xfunc.sgml
M src/backend/executor/functions.c
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
Reconsider the handling of procedure OUT parameters.
commit : e56bce5d43789cce95d099554ae9593ada92b3b7
author : Tom Lane <[email protected]>
date : Thu, 10 Jun 2021 17:11:36 -0400
committer: Tom Lane <[email protected]>
date : Thu, 10 Jun 2021 17:11:36 -0400
Commit 2453ea142 redefined pg_proc.proargtypes to include the types of
OUT parameters, for procedures only. While that had some advantages
for implementing the SQL-spec behavior of DROP PROCEDURE, it was pretty
disastrous from a number of other perspectives. Notably, since the
primary key of pg_proc is name + proargtypes, this made it possible to
have multiple procedures with identical names + input arguments and
differing output argument types. That would make it impossible to call
any one of the procedures by writing just NULL (or "?", or any other
data-type-free notation) for the output argument(s). The change also
seems likely to cause grave confusion for client applications that
examine pg_proc and expect the traditional definition of proargtypes.
Hence, revert the definition of proargtypes to what it was, and
undo a number of complications that had been added to support that.
To support the SQL-spec behavior of DROP PROCEDURE, when there are
no argmode markers in the command's parameter list, we perform the
lookup both ways (that is, matching against both proargtypes and
proallargtypes), succeeding if we get just one unique match.
In principle this could result in ambiguous-function failures
that would not happen when using only one of the two rules.
However, overloading of procedure names is thought to be a pretty
rare usage, so this shouldn't cause many problems in practice.
Postgres-specific code such as pg_dump can defend against any
possibility of such failures by being careful to specify argmodes
for all procedure arguments.
This also fixes a few other bugs in the area of CALL statements
with named parameters, and improves the documentation a little.
catversion bump forced because the representation of procedures
with OUT arguments changes.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/ref/alter_extension.sgml
M doc/src/sgml/ref/alter_procedure.sgml
M doc/src/sgml/ref/call.sgml
M doc/src/sgml/ref/comment.sgml
M doc/src/sgml/ref/drop_procedure.sgml
M doc/src/sgml/ref/drop_routine.sgml
M doc/src/sgml/ref/security_label.sgml
M doc/src/sgml/xfunc.sgml
M src/backend/catalog/namespace.c
M src/backend/catalog/pg_aggregate.c
M src/backend/catalog/pg_proc.c
M src/backend/commands/functioncmds.c
M src/backend/executor/functions.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/parser/parse_func.c
M src/backend/parser/parse_oper.c
M src/backend/utils/adt/regproc.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/fmgr/funcapi.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/include/catalog/catversion.h
M src/include/catalog/namespace.h
M src/include/catalog/pg_proc.h
M src/include/funcapi.h
M src/include/nodes/parsenodes.h
M src/include/optimizer/optimizer.h
M src/include/parser/parse_func.h
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_call.sql
M src/pl/plpython/expected/plpython_call.out
M src/pl/plpython/plpy_procedure.c
M src/pl/plpython/sql/plpython_call.sql
M src/pl/tcl/expected/pltcl_call.out
M src/pl/tcl/sql/pltcl_call.sql
M src/test/regress/expected/create_procedure.out
M src/test/regress/sql/create_procedure.sql
Rearrange logrep worker's snapshot handling some more.
commit : 3a09d75b4f6cabc8331e228b6988dbfcd9afdfbe
author : Tom Lane <[email protected]>
date : Thu, 10 Jun 2021 12:27:27 -0400
committer: Tom Lane <[email protected]>
date : Thu, 10 Jun 2021 12:27:27 -0400
It turns out that worker.c's code path for TRUNCATE was also
careless about establishing a snapshot while executing user-defined
code, allowing the checks added by commit 84f5c2908 to fail when
a trigger is fired in that context.
We could just wrap Push/PopActiveSnapshot around the truncate call,
but it seems better to establish a policy of holding a snapshot
throughout execution of a replication step. To help with that and
possible future requirements, replace the previous ensure_transaction
calls with pairs of begin/end_replication_step calls.
Per report from Mark Dilger. Back-patch to v11, like the previous
changes.
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/worker.c
Shut down EvalPlanQual machinery when LockRows node reaches the end.
commit : bb4aed46a5aeb00d2f1d8b8160feed339f4eaf12
author : Tom Lane <[email protected]>
date : Thu, 10 Jun 2021 11:15:13 -0400
committer: Tom Lane <[email protected]>
date : Thu, 10 Jun 2021 11:15:13 -0400
Previously, we left the EPQ sub-executor alone until ExecEndLockRows.
This caused any buffer pins or other resources that it might hold to
remain held until ExecutorEnd, which in some code paths means that
they are held till the Portal is closed. That can cause user-visible
problems, such as blocking VACUUM; and it's unlike the behavior of
ordinary table-scanning nodes, which will have released all buffer
pins by the time they return an EOF indication.
We can make LockRows work more like other plan nodes by calling
EvalPlanQualEnd just before returning NULL. We still need to call it
in ExecEndLockRows in case the node was not run to completion, but in
the normal case the second call does nothing and costs little.
Per report from Yura Sokolov. In principle this is a longstanding
bug, but in view of the lack of other complaints and the low severity
of the consequences, I chose not to back-patch.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/nodeLockRows.c
Avoid ECPG test failures in some GSS-capable environments.
commit : 9bb5eecce645dd72853e3ed262bef7bf11cae566
author : Tom Lane <[email protected]>
date : Thu, 10 Jun 2021 10:45:31 -0400
committer: Tom Lane <[email protected]>
date : Thu, 10 Jun 2021 10:45:31 -0400
Buildfarm member hamerkop has been reporting that two cases in
connect/test5.pgc show different error messages than the test expects,
because since commit ffa2e4670 libpq's connection failure messages
are exposing the fact that a GSS-encrypted connection was attempted
and failed. That's pretty interesting information in itself, and
I certainly don't wish to shoot the messenger, but we need to do
something to stabilize the ECPG results.
For the second of these two failure cases, we can add the
gssencmode=disable option to prevent the discrepancy. However,
that solution is problematic for the first failure, because the only
unique thing about that case is that it's testing a completely-omitted
connection target; there's noplace to add the option without defeating
the point of the test case. After some thrashing around with
alternative fixes that turned out to have undesirable side-effects,
the most workable answer is just to give up and remove that test case.
Perhaps we can revert this later, if we figure out why the GSS code
is misbehaving in hamerkop's environment.
Thanks to Michael Paquier for exploration of alternatives.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/ecpg/test/connect/test5.pgc
M src/interfaces/ecpg/test/expected/connect-test5.c
M src/interfaces/ecpg/test/expected/connect-test5.stderr
Add some const decorations
commit : b29fa951ec519bdde153465e2caa6c0b7b3bcfc3
author : Peter Eisentraut <[email protected]>
date : Thu, 10 Jun 2021 16:21:48 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 10 Jun 2021 16:21:48 +0200
One of these functions is new in PostgreSQL 14; might as well start it
out right.
M src/backend/replication/logical/origin.c
M src/include/replication/origin.h
Adjust new test case to set wal_keep_size.
commit : 4dcb1d087aebc6fc2477ce4458ea82f548e2c1ee
author : Robert Haas <[email protected]>
date : Thu, 10 Jun 2021 09:08:30 -0400
committer: Robert Haas <[email protected]>
date : Thu, 10 Jun 2021 09:08:30 -0400
Per buildfarm member conchuela and Kyotaro Horiguchi, it's possible
for the WAL segment that the cascading standby needs to be removed
too quickly. Hopefully this will prevent that.
Kyotaro Horiguchi
Discussion: http://postgr.es/m/[email protected]
M src/test/recovery/t/025_stuck_on_old_timeline.pl
Fix an asssortment of typos in brin_minmax_multi.c and mcv.c
commit : 55ba5973d9144a552661cf1fa4cbd228a3799212
author : David Rowley <[email protected]>
date : Thu, 10 Jun 2021 20:13:44 +1200
committer: David Rowley <[email protected]>
date : Thu, 10 Jun 2021 20:13:44 +1200
Discussion: https://postgr.es/m/CAApHDvrbyJNOPBws4RUhXghZ7+TBjtdO-rznTsqZECuowNorXg@mail.gmail.com
M src/backend/access/brin/brin_minmax_multi.c
M src/backend/statistics/mcv.c
Fix corner case failure of new standby to follow new primary.
commit : caba8f0d43fb679c6f9643456080408a6bc370e8
author : Robert Haas <[email protected]>
date : Wed, 9 Jun 2021 16:17:00 -0400
committer: Robert Haas <[email protected]>
date : Wed, 9 Jun 2021 16:17:00 -0400
This only happens if (1) the new standby has no WAL available locally,
(2) the new standby is starting from the old timeline, (3) the promotion
happened in the WAL segment from which the new standby is starting,
(4) the timeline history file for the new timeline is available from
the archive but the WAL files for are not (i.e. this is a race),
(5) the WAL files for the new timeline are available via streaming,
and (6) recovery_target_timeline='latest'.
Commit ee994272ca50f70b53074f0febaec97e28f83c4e introduced this
logic and was an improvement over the previous code, but it mishandled
this case. If recovery_target_timeline='latest' and restore_command is
set, validateRecoveryParameters() can change recoveryTargetTLI to be
different from receiveTLI. If streaming is then tried afterward,
expectedTLEs gets initialized with the history of the wrong timeline.
It's supposed to be a list of entries explaining how to get to the
target timeline, but in this case it ends up with a list of entries
explaining how to get to the new standby's original timeline, which
isn't right.
Dilip Kumar and Robert Haas, reviewed by Kyotaro Horiguchi.
Discussion: http://postgr.es/m/CAFiTN-sE-jr=LB8jQuxeqikd-Ux+jHiXyh4YDiZMPedgQKup0g@mail.gmail.com
M src/backend/access/transam/xlog.c
A src/test/recovery/t/025_stuck_on_old_timeline.pl
A src/test/recovery/t/cp_history_files
Fix inconsistencies in psql --help=commands
commit : 845cad4d51cb12a34ea012dfe58af5ef490384fc
author : Michael Paquier <[email protected]>
date : Wed, 9 Jun 2021 16:24:52 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 9 Jun 2021 16:24:52 +0900
The set of subcommands supported by \dAp, \do and \dy was described
incorrectly in psql's --help. The documentation was already consistent
with the code.
Reported-by: inoas, from IRC
Author: Matthijs van der Vleuten
Reviewed-by: Neil Chen
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/bin/psql/help.c
Force NO SCROLL for plpgsql's implicit cursors.
commit : be90098907475f3cfff7dc6d590641b9c2dae081
author : Tom Lane <[email protected]>
date : Tue, 8 Jun 2021 18:40:06 -0400
committer: Tom Lane <[email protected]>
date : Tue, 8 Jun 2021 18:40:06 -0400
Further thought about bug #17050 suggests that it's a good idea
to use CURSOR_OPT_NO_SCROLL for the implicit cursor opened by
a plpgsql FOR-over-query loop. This ensures that, if somebody
commits inside the loop, PersistHoldablePortal won't try to
rewind and re-read the cursor. While we'd have selected NO_SCROLL
anyway if FOR UPDATE/SHARE appears in the query, there are other
hazards with volatile functions; and in any case, it's silly to
expend effort storing rows that we know for certain won't be needed.
(While here, improve the comment in exec_run_select, which was a bit
confused about the rationale for when we can use parallel mode.
Cursor operations aren't a hazard for nameless portals.)
This wasn't an issue until v11, which introduced the possibility
of persisting such cursors. Hence, back-patch to v11.
Per bug #17050 from Алексей Булгаков.
Discussion: https://postgr.es/m/[email protected]
M src/pl/plpgsql/src/pl_exec.c
Avoid misbehavior when persisting a non-stable cursor.
commit : ba2c6d6cec000f0aeaeda4d56a23a335f6164860
author : Tom Lane <[email protected]>
date : Tue, 8 Jun 2021 17:50:15 -0400
committer: Tom Lane <[email protected]>
date : Tue, 8 Jun 2021 17:50:15 -0400
PersistHoldablePortal has long assumed that it should store the
entire output of the query-to-be-persisted, which requires rewinding
and re-reading the output. This is problematic if the query is not
stable: we might get different row contents, or even a different
number of rows, which'd confuse the cursor state mightily.
In the case where the cursor is NO SCROLL, this is very easy to
solve: just store the remaining query output, without any rewinding,
and tweak the portal's cursor state to match. Aside from removing
the semantic problem, this could be significantly more efficient
than storing the whole output.
If the cursor is scrollable, there's not much we can do, but it
was already the case that scrolling a volatile query's result was
pretty unsafe. We can just document more clearly that getting
correct results from that is not guaranteed.
There are already prohibitions in place on using SCROLL with
FOR UPDATE/SHARE, which is one way for a SELECT query to have
non-stable results. We could imagine prohibiting SCROLL when
the query contains volatile functions, but that would be
expensive to enforce. Moreover, it could break applications
that work just fine, if they have functions that are in fact
stable but the user neglected to mark them so. So settle for
documenting the hazard.
While this problem has existed in some guise for a long time,
it got a lot worse in v11, which introduced the possibility
of persisting plpgsql cursors (perhaps implicit ones) even
when they violate the rules for what can be marked WITH HOLD.
Hence, I've chosen to back-patch to v11 but not further.
Per bug #17050 from Алексей Булгаков.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/ref/declare.sgml
M src/backend/commands/portalcmds.c
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/sql/plpgsql_transaction.sql
doc: update release note item about the v2 wire protocol
commit : 444302ed56273e4c4006a9be319e60fa12a90346
author : Bruce Momjian <[email protected]>
date : Tue, 8 Jun 2021 16:47:14 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 8 Jun 2021 16:47:14 -0400
Protocol v2 was last used in PG 7.3, not 7.2.
Reported-by: Tatsuo Ishii
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/release-14.sgml
Adjust batch size in postgres_fdw to not use too many parameters
commit : cb92703384e2bb3fa0a690e5dbb95ad333c2b44c
author : Tomas Vondra <[email protected]>
date : Tue, 8 Jun 2021 20:22:18 +0200
committer: Tomas Vondra <[email protected]>
date : Tue, 8 Jun 2021 20:22:18 +0200
The FE/BE protocol identifies parameters with an Int16 index, which
limits the maximum number of parameters per query to 65535. With
batching added to postges_fdw this limit is much easier to hit, as
the whole batch is essentially a single query, making this error much
easier to hit.
The failures are a bit unpredictable, because it also depends on the
number of columns in the query. So instead of just failing, this patch
tweaks the batch_size to not exceed the maximum number of parameters.
Reported-by: Hou Zhijie <[email protected]>
Reviewed-by: Bharath Rupireddy <[email protected]>
Discussion: https://postgr.es/m/OS0PR01MB571603973C0AC2874AD6BF2594299%40OS0PR01MB5716.jpnprd01.prod.outlook.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/libpq-fe.h
Fix pg_visibility regression failure with CLOBBER_CACHE_ALWAYS
commit : d1f0aa7696917213485c03b076b573497a535076
author : Tomas Vondra <[email protected]>
date : Tue, 8 Jun 2021 19:24:27 +0200
committer: Tomas Vondra <[email protected]>
date : Tue, 8 Jun 2021 19:24:27 +0200
Commit 8e03eb92e9 reverted a bit too much code, reintroducing one of the
issues fixed by 39b66a91bd - a page might have been left partially empty
after relcache invalidation.
Reported-By: Tom Lane
Author: Masahiko Sawada
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CAD21AoA%3D%3Df2VSw3c-Cp_y%3DWLKHMKc1D6s7g3YWsCOvgaYPpJcg%40mail.gmail.com
M src/backend/access/heap/hio.c
Don't crash on empty statements in SQL-standard function bodies.
commit : bfeede9fa464ab707eb5ac5704742bf78bd7ac9e
author : Tom Lane <[email protected]>
date : Tue, 8 Jun 2021 11:59:34 -0400
committer: Tom Lane <[email protected]>
date : Tue, 8 Jun 2021 11:59:34 -0400
gram.y should discard NULL pointers (empty statements) when
assembling a routine_body_stmt_list, as it does for other
sorts of statement lists.
Julien Rouhaud and Tom Lane, per report from Noah Misch.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/gram.y
M src/test/regress/expected/create_function_3.out
M src/test/regress/sql/create_function_3.sql
libpq: Fix SNI host handling
commit : 37e1cce4ddf0be362e3093cee55493aee41bc423
author : Peter Eisentraut <[email protected]>
date : Tue, 8 Jun 2021 15:37:54 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 8 Jun 2021 15:37:54 +0200
Fix handling of NULL host name (possibly by using hostaddr). It
previously crashed. Also, we should look at connhost, not pghost, to
handle multi-host specifications.
Also remove an unnecessary SSL_CTX_free().
Reported-by: Jacob Champion <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/interfaces/libpq/fe-secure-openssl.c
Doc: Further update documentation for asynchronous execution.
commit : eab81953682d5087295afb911c93f36cb1533bd9
author : Etsuro Fujita <[email protected]>
date : Tue, 8 Jun 2021 13:45:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Tue, 8 Jun 2021 13:45:00 +0900
Add a note about asynchronous execution by postgres_fdw when applied to
Append nodes that contain synchronous subplan(s) as well. Follow-up for
commit 27e1f1456.
Andrey Lepikhov and Etsuro Fujita
Discussion: https://postgr.es/m/58fa2aa5-07f5-80b5-59a1-fec8a349fee7%40postgrespro.ru
M doc/src/sgml/postgres-fdw.sgml
Reorder superuser check in pg_log_backend_memory_contexts()
commit : 4e47b02834827fa700627290fae02f89a450368c
author : Michael Paquier <[email protected]>
date : Tue, 8 Jun 2021 08:53:12 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 8 Jun 2021 08:53:12 +0900
The use of this function is limited to superusers and the code includes
a hardcoded check for that. However, the code would look for the PGPROC
entry to signal for the memory dump before checking if the user is a
superuser or not, which does not make sense if we know that an error
will be returned. Note that the code would let one know if a process
was a PostgreSQL process or not even for non-authorized users, which is
not the case now, but this avoids taking ProcArrayLock that will most
likely finish by being unnecessary.
Thanks to Julien Rouhaud and Tom Lane for the discussion.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/mcxtfuncs.c
Add _outTidRangePath()
commit : 3bb309be7533e153d86390642e8a2d054bbe82c8
author : Peter Eisentraut <[email protected]>
date : Mon, 7 Jun 2021 21:32:53 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 7 Jun 2021 21:32:53 +0200
We have outNode() coverage for all path nodes, but this one was
missed when it was added.
M src/backend/nodes/outfuncs.c
Stabilize contrib/seg regression test.
commit : d16ebfbff74b30c83a4669a1df318cacfa7e52ca
author : Tom Lane <[email protected]>
date : Mon, 7 Jun 2021 14:52:42 -0400
committer: Tom Lane <[email protected]>
date : Mon, 7 Jun 2021 14:52:42 -0400
If autovacuum comes along just after we fill table test_seg with
some data, it will update the stats to the point where we prefer
a plain indexscan over a bitmap scan, breaking the expected
output (as well as the point of the test case). To fix, just
force a bitmap scan to be chosen here.
This has evidently been wrong since commit de1d042f5. It's not
clear why we just recently saw any buildfarm failures due to it;
but prairiedog has failed twice on this test in the past week.
Hence, backpatch to v11 where this test case came in.
M contrib/seg/expected/seg.out
M contrib/seg/sql/seg.sql
Fix incautious handling of possibly-miscoded strings in client code.
commit : 42f94f56bf9559f0a3cf5f3ffde50655834694ee
author : Tom Lane <[email protected]>
date : Mon, 7 Jun 2021 14:15:25 -0400
committer: Tom Lane <[email protected]>
date : Mon, 7 Jun 2021 14:15:25 -0400
An incorrectly-encoded multibyte character near the end of a string
could cause various processing loops to run past the string's
terminating NUL, with results ranging from no detectable issue to
a program crash, depending on what happens to be in the following
memory.
This isn't an issue in the server, because we take care to verify
the encoding of strings before doing any interesting processing
on them. However, that lack of care leaked into client-side code
which shouldn't assume that anyone has validated the encoding of
its input.
Although this is certainly a bug worth fixing, the PG security team
elected not to regard it as a security issue, primarily because
any untrusted text should be sanitized by PQescapeLiteral or
the like before being incorporated into a SQL or psql command.
(If an app fails to do so, the same technique can be used to
cause SQL injection, with probably much more dire consequences
than a mere client-program crash.) Those functions were already
made proof against this class of problem, cf CVE-2006-2313.
To fix, invent PQmblenBounded() which is like PQmblen() except it
won't return more than the number of bytes remaining in the string.
In HEAD we can make this a new libpq function, as PQmblen() is.
It seems imprudent to change libpq's API in stable branches though,
so in the back branches define PQmblenBounded as a macro in the files
that need it. (Note that just changing PQmblen's behavior would not
be a good idea; notably, it would completely break the escaping
functions' defense against this exact problem. So we just want a
version for those callers that don't have any better way of handling
this issue.)
Per private report from houjingyi. Back-patch to all supported branches.
M src/bin/psql/common.c
M src/bin/psql/psqlscanslash.l
M src/bin/psql/stringutils.c
M src/bin/psql/tab-complete.c
M src/bin/scripts/common.c
M src/common/jsonapi.c
M src/common/wchar.c
M src/fe_utils/print.c
M src/fe_utils/string_utils.c
M src/include/mb/pg_wchar.h
M src/interfaces/libpq/exports.txt
M src/interfaces/libpq/fe-misc.c
M src/interfaces/libpq/fe-print.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-fe.h
Fix portability issue in test indirect_toast
commit : 68a6d8a87006ba727d9662ec84c7a3d9de402df0
author : Michael Paquier <[email protected]>
date : Mon, 7 Jun 2021 18:12:29 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 7 Jun 2021 18:12:29 +0900
When run on a server using default_toast_compression set to LZ4, this
test would fail because of a consistency issue with the order of the
tuples treated. LZ4 causes one tuple to be stored inline instead of
getting externalized. As the goal of this test is to check after data
stored externally, stick to pglz as the compression algorithm used, so
as all data of this test is stored the way it should.
Analyzed-by: Dilip Kumar
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/indirect_toast.out
M src/test/regress/sql/indirect_toast.sql
Remove two_phase variable from CreateReplicationSlotCmd struct.
commit : be57f21650d36449ec34a67b2d9af71126a663b3
author : Amit Kapila <[email protected]>
date : Mon, 7 Jun 2021 09:32:06 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 7 Jun 2021 09:32:06 +0530
Commit 19890a064e added the option to enable two_phase commits via
pg_create_logical_replication_slot but didn't extend the support of same
in replication protocol. However, by mistake, it added the two_phase
variable in CreateReplicationSlotCmd which is required only when we extend
the replication protocol.
Reported-by: Jeff Davis
Author: Ajin Cherian
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/walsender.c
M src/include/nodes/replnodes.h
Fix rescanning of async-aware Append nodes.
commit : f3baaf28a6da588987b94a05a725894805c3eae9
author : Etsuro Fujita <[email protected]>
date : Mon, 7 Jun 2021 12:45:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Mon, 7 Jun 2021 12:45:00 +0900
In cases where run-time pruning isn't required, the synchronous and
asynchronous subplans for an async-aware Append node determined using
classify_matching_subplans() should be re-used when rescanning the node,
but the previous code re-determined them using that function repeatedly
each time when rescanning the node, leading to incorrect results in a
normal build and an Assert failure in an Assert-enabled build as that
function doesn't assume that it's called repeatedly in such cases. Fix
the code as mentioned above.
My oversight in commit 27e1f1456.
While at it, initialize async-related pointers/variables to NULL/zero
explicitly in ExecInitAppend() and ExecReScanAppend(), just to be sure.
(The variables would have been set to zero before we get to the latter
function, but let's do so.)
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAPmGK16Q4B2_KY%2BJH7rb7wQbw54AUprp7TMekGTd2T1B62yysQ%40mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/executor/nodeAppend.c
Fix inconsistent equalfuncs.c behavior for FuncCall.funcformat.
commit : a65e9f3f1405b786673feec131879843432bf9a6
author : Tom Lane <[email protected]>
date : Sun, 6 Jun 2021 15:46:58 -0400
committer: Tom Lane <[email protected]>
date : Sun, 6 Jun 2021 15:46:58 -0400
Other equalfuncs.c checks on CoercionForm fields use
COMPARE_COERCIONFORM_FIELD (which makes them no-ops),
but commit 40c24bfef neglected to make _equalFuncCall
do likewise. Fix that.
This is only strictly correct if FuncCall.funcformat has
no semantic effect, instead just determining ruleutils.c
display formatting. 40c24bfef added a couple of checks
in parse analysis that could break that rule; but on closer
inspection, they're redundant, so just take them out again.
Per report from Noah Misch.
Discussion: https://postgr.es/m/[email protected]
M src/backend/nodes/equalfuncs.c
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_func.c
Add transformed flag to nodes/*funcs.c for CREATE STATISTICS
commit : d57ecebd128cdf2f4844a2ea4d35ff28d7d69be8
author : Tomas Vondra <[email protected]>
date : Sun, 6 Jun 2021 20:52:58 +0200
committer: Tomas Vondra <[email protected]>
date : Sun, 6 Jun 2021 20:52:58 +0200
Commit a4d75c86bf added a new flag, tracking if the statement was
processed by transformStatsStmt(), but failed to add this flag to
nodes/*funcs.c.
Catversion bump, due to adding a flag to copy/equal/out functions.
Reported-by: Noah Misch
Discussion: https://postgr.es/m/ad7891d2-e90c-b446-9fe2-7419143847d7%40enterprisedb.com
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/include/catalog/catversion.h
Standardize nodes/*funcs.c cosmetics for ForeignScan.resultRelation.
commit : a2dee328bbe5b1979bbc6a784deb86a336c9cd74
author : Noah Misch <[email protected]>
date : Sun, 6 Jun 2021 00:08:21 -0700
committer: Noah Misch <[email protected]>
date : Sun, 6 Jun 2021 00:08:21 -0700
catversion bump due to readfuncs.c field order change.
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/include/catalog/catversion.h
doc: Simplify COMMENT and SECURITY LABEL documentation
commit : 5c25fd650a774cc4f16ac9c635830d9bc5797f61
author : Peter Eisentraut <[email protected]>
date : Sat, 5 Jun 2021 09:08:40 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 5 Jun 2021 09:08:40 +0200
Just say that objects that reside in schemas can be schema-qualified.
Don't list all possible such objects. The existing lists weren't
complete anyway.
Discussion: https://www.postgresql.org/message-id/flat/b2ec2234-67fe-d861-5cea-f526cd18c086%40enterprisedb.com
M doc/src/sgml/ref/comment.sgml
M doc/src/sgml/ref/security_label.sgml
doc: Make terminology in glossary consistent
commit : 01ddd2f7e411ba434473faebf00f5b5af84c0f64
author : Peter Eisentraut <[email protected]>
date : Sat, 5 Jun 2021 09:01:29 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 5 Jun 2021 09:01:29 +0200
Use "reside in" rather than "belong to" for objects in a schema.
Previous use was a mix of the two.
Author: Alvaro Herrera <[email protected]>
Discussion: https://www.postgresql.org/message-id/[email protected]
M doc/src/sgml/glossary.sgml
gitattributes: Add new entry to silence whitespace error
commit : e6159885b78e9b91b2adc3161c5f827d081f2b13
author : Peter Eisentraut <[email protected]>
date : Sat, 5 Jun 2021 07:57:31 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 5 Jun 2021 07:57:31 +0200
M .gitattributes
Fix subtransaction test for Python 3.10
commit : 4a682d85a1408e48ac529295c329ba2c17a44724
author : Peter Eisentraut <[email protected]>
date : Sat, 5 Jun 2021 07:16:34 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 5 Jun 2021 07:16:34 +0200
Starting with Python 3.10, the stacktrace looks differently:
- PL/Python function "subtransaction_exit_subtransaction_in_with", line 3, in <module>
- s.__exit__(None, None, None)
+ PL/Python function "subtransaction_exit_subtransaction_in_with", line 2, in <module>
+ with plpy.subtransaction() as s:
Using try/except specifically makes the error look always the same.
(See https://github.com/python/cpython/pull/25719 for the discussion
of this change in Python.)
Author: Honza Horak <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/853083.1620749597%40sss.pgh.pa.us
RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1959080
M src/pl/plpython/expected/plpython_subtransaction.out
M src/pl/plpython/sql/plpython_subtransaction.sql
Fix postgres_fdw failure with whole-row Vars of type RECORD.
commit : f61db909dfb94f3411f8719916601a11a905b95e
author : Tom Lane <[email protected]>
date : Fri, 4 Jun 2021 20:07:08 -0400
committer: Tom Lane <[email protected]>
date : Fri, 4 Jun 2021 20:07:08 -0400
Commit 86dc90056 expects that FDWs can cope with whole-row Vars for
their tables, even if the Vars are marked with vartype RECORDOID.
Previously, whole-row Vars generated by the planner had vartype equal
to the relevant table's rowtype OID. (The point behind this change is
to enable sharing of resjunk columns across inheritance child tables.)
It turns out that postgres_fdw fails to cope with this, though through
bad fortune none of its test cases exposed that. Things mostly work,
but when we try to read back a value of such a Var, the expected
rowtype is not available to record_in(). Fortunately, it's not
difficult to hack up the tupdesc that controls this process to
substitute the foreign table's rowtype for RECORDOID. Thus we can
solve the runtime problem while still sharing the resjunk column
with other tables.
Per report from Alexander Pyhalov.
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
Doc: Fix some spelling mistakes
commit : 8f3c06c5d56fc0fa414bcf548860ac50a8fe5982
author : David Rowley <[email protected]>
date : Fri, 4 Jun 2021 23:39:40 +1200
committer: David Rowley <[email protected]>
date : Fri, 4 Jun 2021 23:39:40 +1200
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/ref/analyze.sgml
M doc/src/sgml/ref/pg_amcheck.sgml
Clean up some questionable usages of DatumGet* macros
commit : 8bdb36aab287f564eac534878bc0e1d873a4e3db
author : David Rowley <[email protected]>
date : Fri, 4 Jun 2021 22:42:17 +1200
committer: David Rowley <[email protected]>
date : Fri, 4 Jun 2021 22:42:17 +1200
This tidies up some questionable coding which made use of
DatumGetPointer() for Datums being passed into functions where the
parameter is expected to be a cstring. We saw no compiler warnings with
the old code as the Pointer type used in DatumGetPointer() happens to
be a char * rather than a void *. However, that's no excuse and we should
be using the correct macro for the job.
Here we also make use of OutputFunctionCall() rather than using
FunctionCall1() directly to call the type's output function.
OutputFunctionCall() is the standard way to do this. It casts the
returned value to a cstring for us.
In passing get rid of a duplicate call to strlen(). Most compilers will
likely optimize away the 2nd call, but there may be some that won't. In
any case, this just aligns the code to some other nearby code that already
does this.
Discussion: https://postgr.es/m/CAApHDvq1D=ehZ8hey8Hz67N+_Zth0GHO5wiVCfv1YcGPMXJq0A@mail.gmail.com
M src/backend/access/brin/brin_minmax_multi.c
Doc: fix bogus intarray index example.
commit : e4539386decae1c435767a69507cc7cbb11ac3ff
author : Tom Lane <[email protected]>
date : Thu, 3 Jun 2021 21:07:12 -0400
committer: Tom Lane <[email protected]>
date : Thu, 3 Jun 2021 21:07:12 -0400
The siglen parameter is provided by gist__intbig_ops not
gist__int_ops.
Simon Norris
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/intarray.sgml
doc: Add description for PGSSLCRLDIR
commit : 1e809db86b160e697a56bf47358f7733475840d3
author : Michael Paquier <[email protected]>
date : Fri, 4 Jun 2021 09:46:15 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 4 Jun 2021 09:46:15 +0900
This was missing in the section dedicated to the supported environment
variables of libpq. Oversight in f5465fa.
Reviewed-by: Daniel Gustafsson, Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
doc: Fix link reference for PGSSLMAXPROTOCOLVERSION
commit : 77e9d1b4884262fa09cd8d141c7eadad3affde8b
author : Michael Paquier <[email protected]>
date : Fri, 4 Jun 2021 09:33:14 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 4 Jun 2021 09:33:14 +0900
The link was pointing to the minimum protocol version. Incorrect as of
ff8ca5f.
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M doc/src/sgml/libpq.sgml
Adjust locations which have an incorrect copyright year
commit : 7fc26d11e370afe237631265714221364d7e7910
author : David Rowley <[email protected]>
date : Fri, 4 Jun 2021 12:19:50 +1200
committer: David Rowley <[email protected]>
date : Fri, 4 Jun 2021 12:19:50 +1200
A few patches committed after ca3b37487 mistakenly forgot to make the
copyright year 2021. Fix these.
Discussion: https://postgr.es/m/CAApHDvqyLmd9P2oBQYJ=DbrV8QwyPRdmXtCTFYPE08h+ip0UJw@mail.gmail.com
M contrib/pageinspect/gistfuncs.c
M src/backend/access/brin/brin_bloom.c
M src/backend/access/brin/brin_minmax_multi.c
M src/backend/rewrite/rewriteSearchCycle.c
M src/backend/utils/adt/jsonbsubs.c
M src/common/hex.c
M src/common/hmac.c
M src/common/hmac_openssl.c
M src/common/sha1.c
M src/common/sha1_int.h
M src/include/common/hex.h
M src/include/common/hmac.h
M src/include/common/sha1.h
M src/include/port/pg_iovec.h
M src/include/rewrite/rewriteSearchCycle.h
In PostgresNode.pm, don't pass SQL to psql on the command line
commit : 11e9caff82bc7326e2bc9782937cb03875050cc4
author : Andrew Dunstan <[email protected]>
date : Thu, 3 Jun 2021 16:08:33 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 3 Jun 2021 16:08:33 -0400
The Msys shell mangles certain patterns in its command line, so avoid
handing arbitrary SQL to psql on the command line and instead use
IPC::Run's redirection facility for stdin. This pattern is already
mostly whats used, but query_poll_until() was not doing the right thing.
Problem discovered on the buildfarm when a new TAP test failed on msys.
M src/test/perl/PostgresNode.pm
Fix incorrect permissions on pg_subscription.
commit : 3590680b85a8e51ef8df550e5a10dedd0d2dfd88
author : Tom Lane <[email protected]>
date : Thu, 3 Jun 2021 14:54:06 -0400
committer: Tom Lane <[email protected]>
date : Thu, 3 Jun 2021 14:54:06 -0400
The documented intent is for all columns except subconninfo to be
publicly readable. However, this has been overlooked twice.
subsynccommit has never been readable since it was introduced,
nor has the oid column (which is important for joining).
Given the lack of previous complaints, it's not clear that it's
worth doing anything about this in the back branches. But there's
still time to fix it inexpensively for v14.
Per report from Israel Barth (via Euler Taveira).
Patch by Euler Taveira, possibly-vain comment updates by me.
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/system_views.sql
M src/include/catalog/catversion.h
M src/include/catalog/pg_subscription.h
Reduce risks of conflicts in internal queries of REFRESH MATVIEW CONCURRENTLY
commit : 187682c3217375c9b70417bf3235790f639e8e7e
author : Michael Paquier <[email protected]>
date : Thu, 3 Jun 2021 15:28:24 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 3 Jun 2021 15:28:24 +0900
The internal SQL queries used by REFRESH MATERIALIZED VIEW CONCURRENTLY
include some aliases for its diff and temporary relations with
rather-generic names: diff, newdata, newdata2 and mv. Depending on the
queries used for the materialized view, using CONCURRENTLY could lead to
some internal failures if the matview query and those internal aliases
conflict.
Those names have been chosen in 841c29c8. This commit switches instead
to a naming pattern which is less likely going to cause conflicts, based
on an idea from Thomas Munro, by appending _$ to those aliases. This is
not perfect as those new names could still conflict, but at least it has
the advantage to keep the code readable and simple while reducing the
likelihood of conflicts to be close to zero.
Reported-by: Mathis Rudolf
Author: Bharath Rupireddy
Reviewed-by: Bernd Helmle, Thomas Munro, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/backend/commands/matview.c
doc: Group options in pg_amcheck reference page
commit : cb3cffe694b6041c1de47b12b225e05f664c7285
author : Peter Eisentraut <[email protected]>
date : Thu, 3 Jun 2021 06:55:04 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 3 Jun 2021 06:55:04 +0200
The previous arrangement was just one big list, and the internal order
was not all consistent either. Now arrange the options by group and
sort them, the way it's already done in the --help output and one
other reference pages. Also fix some ordering in the --help output.
M doc/src/sgml/ref/pg_amcheck.sgml
M src/bin/pg_amcheck/pg_amcheck.c
Standardize usages of appendStringInfo and appendPQExpBuffer
commit : f736e188ce70bda34f04c9f381b7c5141dc20dce
author : David Rowley <[email protected]>
date : Thu, 3 Jun 2021 16:38:03 +1200
committer: David Rowley <[email protected]>
date : Thu, 3 Jun 2021 16:38:03 +1200
Fix a few places that were using appendStringInfo() when they should have
been using appendStringInfoString(). Also some cases of
appendPQExpBuffer() that would have been better suited to use
appendPQExpBufferChar(), and finally, some places that used
appendPQExpBuffer() when appendPQExpBufferStr() would have suited better.
There are no bugs are being fixed here. The aim is just to make the code
use the most optimal function for the job.
All the code being changed here is new to PG14. It makes sense to fix
these before we branch for PG15. There are a few other places that we
could fix, but those cases are older code so fixing those seems less
worthwhile as it may cause unnecessary back-patching pain in the future.
Author: Hou Zhijie
Discussion: https://postgr.es/m/OS0PR01MB5716732158B1C4142C6FE375943D9@OS0PR01MB5716.jpnprd01.prod.outlook.com
M src/backend/access/brin/brin_minmax_multi.c
M src/backend/access/heap/vacuumlazy.c
M src/bin/pg_amcheck/pg_amcheck.c
M src/bin/psql/describe.c
Ignore more environment variables in TAP tests
commit : 8279f68a1b13d58a0c9869a60081009d8995e4df
author : Michael Paquier <[email protected]>
date : Thu, 3 Jun 2021 11:50:56 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 3 Jun 2021 11:50:56 +0900
Various environment variables were not getting reset in the TAP tests,
which would cause failures depending on the tests or the environment
variables involved. For example, PGSSL{MAX,MIN}PROTOCOLVERSION could
cause failures in the SSL tests. Even worse, a junk value of
PGCLIENTENCODING makes a server startup fail. The list of variables
reset is adjusted in each stable branch depending on what is supported.
While on it, simplify a bit the code per a suggestion from Andrew
Dunstan, using a list of variables instead of doing single deletions.
Reviewed-by: Andrew Dunstan, Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/test/perl/TestLib.pm
Re-allow custom GUC names that have more than two components.
commit : 2955c2be79b35fa369c83fa3b5f44661cb88afa9
author : Tom Lane <[email protected]>
date : Wed, 2 Jun 2021 18:50:15 -0400
committer: Tom Lane <[email protected]>
date : Wed, 2 Jun 2021 18:50:15 -0400
Commit 3db826bd5 disallowed this case, but it turns out that some
people are depending on it. Since the core grammar has allowed
it since 3dc37cd8d, it seems like this code should fall in line.
Per bug #17045 from Robert Sosinski.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/misc/guc.c
M src/test/regress/expected/guc.out
M src/test/regress/sql/guc.sql
Revert most of 39b66a91bd
commit : 8e03eb92e9ad54e2f1ed8b5a73617601f6262f81
author : Tomas Vondra <[email protected]>
date : Thu, 3 Jun 2021 00:06:42 +0200
committer: Tomas Vondra <[email protected]>
date : Thu, 3 Jun 2021 00:06:42 +0200
Reverts most of commit 39b66a91bd, which was found to cause significant
regression for REFRESH MATERIALIZED VIEW. This means only rows inserted
by heap_multi_insert will benefit from the optimization, implemented in
commit 7db0cd2145.
Reported-by: Masahiko Sawada
Discussion: https://postgr.es/m/CAD21AoA%3D%3Df2VSw3c-Cp_y%3DWLKHMKc1D6s7g3YWsCOvgaYPpJcg%40mail.gmail.com
M src/backend/access/heap/heapam.c
M src/backend/access/heap/hio.c
Fix planner's row-mark code for inheritance from a foreign table.
commit : 889592344c48d3965567f331b4ea89dfe6447bce
author : Tom Lane <[email protected]>
date : Wed, 2 Jun 2021 14:38:14 -0400
committer: Tom Lane <[email protected]>
date : Wed, 2 Jun 2021 14:38:14 -0400
Commit 428b260f8 broke planning of cases where row marks are needed
(SELECT FOR UPDATE, etc) and one of the query's tables is a foreign
table that has regular table(s) as inheritance children. We got the
reverse case right, but apparently were thinking that foreign tables
couldn't be inheritance parents. Not so; so we need to be able to
add a CTID junk column while adding a new child, not only a wholerow
junk column.
Back-patch to v12 where the faulty code came in.
Amit Langote
Discussion: https://postgr.es/m/CA+HiwqEmo3FV1LAQ4TVyS2h1WM=kMkZUmbNuZSCnfHvMcUcPeA@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/optimizer/util/inherit.c
Update plannodes.h's comments about PlanRowMark.
commit : 79c50ca57828e9f8375766b36cce1e2960eebf87
author : Tom Lane <[email protected]>
date : Wed, 2 Jun 2021 11:52:35 -0400
committer: Tom Lane <[email protected]>
date : Wed, 2 Jun 2021 11:52:35 -0400
The reference here to different physical column numbers in inherited
UPDATE/DELETE plans is obsolete as of 86dc90056; remove it. Also
rework the text about inheritance cases to make it clearer.
M src/include/nodes/plannodes.h
Teach tab-complete.c about recently-added CREATE TYPE options.
commit : 9e3b3ff2664dd0b349d2a6d6f047128cb3489cf2
author : Tom Lane <[email protected]>
date : Wed, 2 Jun 2021 10:44:16 -0400
committer: Tom Lane <[email protected]>
date : Wed, 2 Jun 2021 10:44:16 -0400
Commit c7aba7c14 missed adding SUBSCRIPT here,
and commit 6df7a9698 missed adding MULTIRANGE_TYPE_NAME.
Haiying Tang and Tom Lane
Discussion: https://postgr.es/m/OS0PR01MB6113F9EDA46FA53BAA5445BDFB3D9@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/bin/psql/tab-complete.c
Remove unnecessary use of Time::HiRes from 013_crash_restart.pl.
commit : df466d30c6caa02e2215595fd83ca70be3199cec
author : Fujii Masao <[email protected]>
date : Wed, 2 Jun 2021 12:20:15 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 2 Jun 2021 12:20:15 +0900
The regression test 013_crash_restart.pl included "use Time::HiRes qw(usleep)",
but the usleep was not used there.
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/013_crash_restart.pl
Add regression test for recovery pause.
commit : 6bbc5c5e96b08f6b8c7d28d10ed8dfe6c49dca30
author : Fujii Masao <[email protected]>
date : Wed, 2 Jun 2021 12:19:39 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 2 Jun 2021 12:19:39 +0900
Previously there was no regression test for recovery pause feature.
This commit adds the test that checks
- recovery can be paused or resumed expectedly
- pg_get_wal_replay_pause_state() reports the correct pause state
- the paused state ends and promotion continues if a promotion
is triggered while recovery is paused
Suggested-by: Michael Paquier
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Dilip Kumar
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/005_replay_delay.pl
Add Windows file version information to libpq_pipeline.exe.
commit : 42344ad0ed465ea988d8310d2f413d65329f55a8
author : Noah Misch <[email protected]>
date : Tue, 1 Jun 2021 18:04:15 -0700
committer: Noah Misch <[email protected]>
date : Tue, 1 Jun 2021 18:04:15 -0700
M src/test/modules/libpq_pipeline/Makefile
Fix missing gettimeofday() declaration.
commit : 49527a32ca97761d78efef732a4ac76a2fc086b2
author : Noah Misch <[email protected]>
date : Tue, 1 Jun 2021 18:04:14 -0700
committer: Noah Misch <[email protected]>
date : Tue, 1 Jun 2021 18:04:14 -0700
This avoids a warning under MinGW versions having gettimeofday(), per
buildfarm member walleye.
M src/interfaces/libpq/fe-trace.c
Reject SELECT ... GROUP BY GROUPING SETS (()) FOR UPDATE.
commit : 1103033aedc10295eb689a4b7158f21ef4c14a11
author : Tom Lane <[email protected]>
date : Tue, 1 Jun 2021 11:12:56 -0400
committer: Tom Lane <[email protected]>
date : Tue, 1 Jun 2021 11:12:56 -0400
This case should be disallowed, just as FOR UPDATE with a plain
GROUP BY is disallowed; FOR UPDATE only makes sense when each row
of the query result can be identified with a single table row.
However, we missed teaching CheckSelectLocking() to check
groupingSets as well as groupClause, so that it would allow
degenerate grouping sets. That resulted in a bad plan and
a null-pointer dereference in the executor.
Looking around for other instances of the same bug, the only one
I found was in examine_simple_variable(). That'd just lead to
silly estimates, but it should be fixed too.
Per private report from Yaoguang Chen.
Back-patch to all supported branches.
M src/backend/parser/analyze.c
M src/backend/utils/adt/selfuncs.c
M src/test/regress/expected/errors.out
M src/test/regress/sql/errors.sql
pgoutput: Fix memory leak due to RelationSyncEntry.map.
commit : eb89cb43a0d0e401e71b8e2345b5f5bc8b2755a1
author : Amit Kapila <[email protected]>
date : Tue, 1 Jun 2021 14:14:02 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 1 Jun 2021 14:14:02 +0530
Release memory allocated when creating the tuple-conversion map and its
component TupleDescs when its owning sync entry is invalidated.
TupleDescs must also be freed when no map is deemed necessary, to begin
with.
Reported-by: Andres Freund
Author: Amit Langote
Reviewed-by: Takamichi Osumi, Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/MEYP282MB166933B1AB02B4FE56E82453B64D9@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
M src/backend/replication/pgoutput/pgoutput.c
M src/test/subscription/t/013_partition.pl
Fix error handling in replacement pthread_barrier_init().
commit : a40646e30d85e51a76fb620822d4d921b6802157
author : Thomas Munro <[email protected]>
date : Tue, 1 Jun 2021 11:22:22 +1200
committer: Thomas Munro <[email protected]>
date : Tue, 1 Jun 2021 11:22:22 +1200
Commit 44bf3d50 incorrectly used an errno-style interface when supplying
missing pthread functionality (i.e. on macOS), but it should check for
and return error numbers directly.
M src/port/pthread_barrier_wait.c
Fix RADIUS error reporting in hba file parsing
commit : 7c544ecdad814ccda709cfb6aa7d62840c3a7486
author : Peter Eisentraut <[email protected]>
date : Mon, 31 May 2021 18:32:41 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 31 May 2021 18:32:41 +0200
The RADIUS-related checks in parse_hba_line() did not respect elevel
and did not fill in *err_msg. Also, verify_option_list_length()
pasted together error messages in an untranslatable way. To fix the
latter, remove the function and do the error checking inline. It's a
bit more verbose but only minimally longer, and it makes fixing the
first two issues straightforward.
Reviewed-by: Magnus Hagander <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/8381e425-8c23-99b3-15ec-3115001db1b2%40enterprisedb.com
M src/backend/libpq/hba.c
Fix mis-planning of repeated application of a projection.
commit : 6ee41a301e70fc8e4ad383bad22d695f66ccb0ac
author : Tom Lane <[email protected]>
date : Mon, 31 May 2021 12:03:00 -0400
committer: Tom Lane <[email protected]>
date : Mon, 31 May 2021 12:03:00 -0400
create_projection_plan contains a hidden assumption (here made
explicit by an Assert) that a projection-capable Path will yield a
projection-capable Plan. Unfortunately, that assumption is violated
only a few lines away, by create_projection_plan itself. This means
that two stacked ProjectionPaths can yield an outcome where we try to
jam the upper path's tlist into a non-projection-capable child node,
resulting in an invalid plan.
There isn't any good reason to have stacked ProjectionPaths; indeed the
whole concept is faulty, since the set of Vars/Aggs/etc needed by the
upper one wouldn't necessarily be available in the output of the lower
one, nor could the lower one create such values if they weren't
available from its input. Hence, we can fix this by adjusting
create_projection_path to strip any top-level ProjectionPath from the
subpath it's given. (This amounts to saying "oh, we changed our
minds about what we need to project here".)
The test case added here only fails in v13 and HEAD; before that, we
don't attempt to shove the Sort into the parallel part of the plan,
for reasons that aren't entirely clear to me. However, all the
directly-related code looks generally the same as far back as v11,
where the hazard was introduced (by d7c19e62a). So I've got no faith
that the same type of bug doesn't exist in v11 and v12, given the
right test case. Hence, back-patch the code changes, but not the
irrelevant test case, into those branches.
Per report from Bas Poot.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/pathnode.c
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql
Raise a timeout to 180s, in test 010_logical_decoding_timelines.pl.
commit : d03eeab886baa1be73f8fc2938fb842d25a71fe8
author : Noah Misch <[email protected]>
date : Mon, 31 May 2021 00:29:58 -0700
committer: Noah Misch <[email protected]>
date : Mon, 31 May 2021 00:29:58 -0700
Per buildfarm member hornet. Also, update Pod documentation showing the
lower value. Back-patch to v10, where the test first appeared.
M src/test/perl/PostgresNode.pm
M src/test/recovery/t/010_logical_decoding_timelines.pl
Improve some error wording with multirange type parsing
commit : 12cc956664f159e97be71e33f15ec5f42e46b24e
author : Michael Paquier <[email protected]>
date : Mon, 31 May 2021 11:35:00 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 31 May 2021 11:35:00 +0900
Braces were referred in some error messages as only brackets (not curly
brackets or curly braces), which can be confusing as other types of
brackets could be used.
While on it, add one test to check after the case of junk characters
detected after a right brace.
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/multirangetypes.c
M src/test/regress/expected/multirangetypes.out
M src/test/regress/sql/multirangetypes.sql
Doc: improve libpq service-file docs, avoid overspecifying pathnames.
commit : ba356a397de565c014384aa01a945aab7d50928c
author : Tom Lane <[email protected]>
date : Sat, 29 May 2021 14:27:37 -0400
committer: Tom Lane <[email protected]>
date : Sat, 29 May 2021 14:27:37 -0400
Clarify libpq.sgml's description of service file locations and
semantics. Avoid use of backtick'ed pg_config calls to describe
paths; that doesn't work on Windows, and even on Unix it's an
idiom that not all readers may be instantly familiar with.
Don't overspecify the locations of include files, instead writing
only as much as you'd use in #include directives. The previous text
in these places was incorrect for some installations, depending on
where "postgresql" is in the install path.
Our convention for referencing the user's home directory seems
to be "~", so change the one place that spelled it "$HOME".
install-windows.sgml follows the platform convention of spelling
file paths with "\", so change the one place that used "/".
Haiying Tang and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M doc/src/sgml/install-windows.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/pgbuffercache.sgml
Fix race condition when sharing tuple descriptors.
commit : b1d6538903504904db44679355ac109aeda01737
author : Thomas Munro <[email protected]>
date : Sat, 29 May 2021 14:48:15 +1200
committer: Thomas Munro <[email protected]>
date : Sat, 29 May 2021 14:48:15 +1200
Parallel query processes that called BlessTupleDesc() for identical
tuple descriptors at the same moment could crash. There was code to
handle that rare case, but it dereferenced a bogus DSA pointer. Repair.
Back-patch to 11, where commit cc5f8136 added support for sharing tuple
descriptors in parallel queries.
Reported-by: Eric Thinnes <[email protected]>
Discussion: https://postgr.es/m/99aaa2eb-e194-bf07-c29a-1a76b4f2bcf9%40gmx.de
M src/backend/utils/cache/typcache.c
fix syntax error
commit : d69fcb9caef1ac1f38241645d4fb9f7e0ce02a70
author : Andrew Dunstan <[email protected]>
date : Fri, 28 May 2021 09:35:11 -0400
committer: Andrew Dunstan <[email protected]>
date : Fri, 28 May 2021 09:35:11 -0400
M src/tools/msvc/Solution.pm
Report configured port in MSVC built pg_config
commit : fb424ae85f6b1e32e545f13902d3ba3429be44df
author : Andrew Dunstan <[email protected]>
date : Fri, 28 May 2021 09:26:30 -0400
committer: Andrew Dunstan <[email protected]>
date : Fri, 28 May 2021 09:26:30 -0400
This is a long standing omission, discovered when trying to write code
that relied on it.
Backpatch to all live branches.
M src/tools/msvc/Solution.pm
Fix VACUUM VERBOSE's LP_DEAD item pages output.
commit : 9afdea982420f9672b88e5c17d1ee8eec64105fc
author : Peter Geoghegan <[email protected]>
date : Thu, 27 May 2021 17:09:16 -0700
committer: Peter Geoghegan <[email protected]>
date : Thu, 27 May 2021 17:09:16 -0700
Oversight in commit 5100010e.
M src/backend/access/heap/vacuumlazy.c
Reduce the range of OIDs reserved for genbki.pl.
commit : a4390abecf0f5152cff864e82b67e5f6c8489698
author : Tom Lane <[email protected]>
date : Thu, 27 May 2021 15:55:08 -0400
committer: Tom Lane <[email protected]>
date : Thu, 27 May 2021 15:55:08 -0400
Commit ab596105b increased FirstBootstrapObjectId from 12000 to 13000,
but we've had some push-back about that. It's worrisome to reduce the
daylight between there and FirstNormalObjectId, because the number of
OIDs consumed during initdb for collation objects is hard to predict.
We can improve the situation by abandoning the assumption that these
OIDs must be globally unique. It should be sufficient for them to be
unique per-catalog. (Any code that's unhappy about that is broken
anyway, since no more than per-catalog uniqueness can be guaranteed
once the OID counter wraps around.) With that change, the largest OID
assigned during genbki.pl (starting from a base of 10000) is a bit
under 11000. This allows reverting FirstBootstrapObjectId to 12000
with reasonable confidence that that will be sufficient for many years
to come.
We are not, at this time, abandoning the expectation that
hand-assigned OIDs (below 10000) are globally unique. Someday that'll
likely be necessary, but the need seems years away still.
This is late for v14, but it seems worth doing it now so that
downstream software doesn't have to deal with the consequences of
a change in FirstBootstrapObjectId. In any case, we already
bought into forcing an initdb for beta2, so another catversion
bump won't hurt.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/bki.sgml
M src/backend/catalog/genbki.pl
M src/include/access/transam.h
M src/include/catalog/catversion.h
Rethink definition of pg_attribute.attcompression.
commit : e6241d8e030fbd2746b3ea3f44e728224298f35b
author : Tom Lane <[email protected]>
date : Thu, 27 May 2021 13:24:24 -0400
committer: Tom Lane <[email protected]>
date : Thu, 27 May 2021 13:24:24 -0400
Redefine '\0' (InvalidCompressionMethod) as meaning "if we need to
compress, use the current setting of default_toast_compression".
This allows '\0' to be a suitable default choice regardless of
datatype, greatly simplifying code paths that initialize tupledescs
and the like. It seems like a more user-friendly approach as well,
because now the default compression choice doesn't migrate into table
definitions, meaning that changing default_toast_compression is
usually sufficient to flip an installation's behavior; one needn't
tediously issue per-column ALTER SET COMPRESSION commands.
Along the way, fix a few minor bugs and documentation issues
with the per-column-compression feature. Adopt more robust
APIs for SetIndexStorageProperties and GetAttributeCompression.
Bump catversion because typical contents of attcompression will now
be different. We could get away without doing that, but it seems
better to ensure v14 installations all agree on this. (We already
forced initdb for beta2, anyway.)
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/storage.sgml
M src/backend/access/brin/brin_tuple.c
M src/backend/access/common/indextuple.c
M src/backend/access/common/toast_internals.c
M src/backend/access/common/tupdesc.c
M src/backend/access/heap/heapam_handler.c
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/heap.c
M src/backend/commands/tablecmds.c
M src/backend/parser/gram.y
M src/backend/utils/misc/guc.c
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_dump.c
M src/bin/psql/describe.c
M src/include/access/toast_compression.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_attribute.h
M src/test/regress/expected/compression.out
M src/test/regress/expected/compression_1.out
M src/test/regress/sql/compression.sql
Fix vpath build in libpq_pipeline test
commit : a717e5c771610cf8607f2423ab3ab6b5d30f44ea
author : Peter Eisentraut <[email protected]>
date : Thu, 27 May 2021 16:40:52 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 27 May 2021 16:40:52 +0200
The path needs to be set to refer to the build directory, not the
current directory, because that's actually the source directory at
that point.
fix for 6abc8c2596dbfcb24f9b4d954a1465b8015118c3
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
Add NO_INSTALL option to pgxs
commit : 6abc8c2596dbfcb24f9b4d954a1465b8015118c3
author : Peter Eisentraut <[email protected]>
date : Thu, 27 May 2021 13:58:13 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 27 May 2021 13:58:13 +0200
Apply in libpq_pipeline test makefile, so that the test file is not
installed into tmp_install.
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/cb9d16a6-760f-cd44-28d6-b091d5fb6ca7%40enterprisedb.com
M doc/src/sgml/extend.sgml
M src/makefiles/pgxs.mk
M src/test/modules/libpq_pipeline/Makefile
Fix MSVC scripts when building with GSSAPI/Kerberos
commit : 025110663448a8c877f4b591495f2e5d187d8936
author : Michael Paquier <[email protected]>
date : Thu, 27 May 2021 20:11:00 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 27 May 2021 20:11:00 +0900
The deliverables of upstream Kerberos on Windows are installed with
paths that do not match our MSVC scripts. First, the include folder was
named "inc/" in our scripts, but the upstream MSIs use "include/".
Second, the build would fail with 64-bit environments as the libraries
are named differently.
This commit adjusts the MSVC scripts to be compatible with the latest
installations of upstream, and I have checked that the compilation was
able to work with the 32-bit and 64-bit installations.
Special thanks to Kondo Yuta for the help in investigating the situation
in hamerkop, which had an incorrect configuration for the GSS
compilation.
Reported-by: Brian Ye
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/tools/msvc/Solution.pm
Replace run-time error check with assertion
commit : 388e75ad33489b77cfb9a8590a91e9287d8fb960
author : Peter Eisentraut <[email protected]>
date : Thu, 27 May 2021 09:52:12 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 27 May 2021 09:52:12 +0200
The error message was checking that the structures returned from the
parser matched expectations. That's something we usually use
assertions for, not a full user-facing error message. So replace that
with an assertion (hidden inside lfirst_node()).
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/452e9df8-ec89-e01b-b64a-8cc6ce830458%40enterprisedb.com
M src/backend/commands/statscmds.c
doc: Fix description of some GUCs in docs and postgresql.conf.sample
commit : 2941138e60fc711bd221b3264807f36cc079dfbb
author : Michael Paquier <[email protected]>
date : Thu, 27 May 2021 14:57:28 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 27 May 2021 14:57:28 +0900
The following parameters have been imprecise, or incorrect, about their
description (PGC_POSTMASTER or PGC_SIGHUP):
- autovacuum_work_mem (docs, as of 9.6~)
- huge_page_size (docs, as of 14~)
- max_logical_replication_workers (docs, as of 10~)
- max_sync_workers_per_subscription (docs, as of 10~)
- min_dynamic_shared_memory (docs, as of 14~)
- recovery_init_sync_method (postgresql.conf.sample, as of 14~)
- remove_temp_files_after_crash (docs, as of 14~)
- restart_after_crash (docs, as of 9.6~)
- ssl_min_protocol_version (docs, as of 12~)
- ssl_max_protocol_version (docs, as of 12~)
This commit adjusts the description of all these parameters to be more
consistent with the practice used for the others.
Revewed-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M doc/src/sgml/config.sgml
M src/backend/utils/misc/postgresql.conf.sample
Fix assertion during streaming of multi-insert toast changes.
commit : 6f4bdf81529fdaf6744875b0be99ecb9bfb3b7e0
author : Amit Kapila <[email protected]>
date : Thu, 27 May 2021 07:59:43 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 27 May 2021 07:59:43 +0530
While decoding the multi-insert WAL we can't clean the toast untill we get
the last insert of that WAL record. Now if we stream the changes before we
get the last change, the memory for toast chunks won't be released and we
expect the txn to have streamed all changes after streaming. This
restriction is mainly to ensure the correctness of streamed transactions
and it doesn't seem worth uplifting such a restriction just to allow this
case because anyway we will stream the transaction once such an insert is
complete.
Previously we were using two different flags (one for toast tuples and
another for speculative inserts) to indicate partial changes. Now instead
we replaced both of them with a single flag to indicate partial changes.
Reported-by: Pavan Deolasee
Author: Dilip Kumar
Reviewed-by: Pavan Deolasee, Amit Kapila
Discussion: https://postgr.es/m/CABOikdN-_858zojYN-2tNcHiVTw-nhxPwoQS4quExeweQfG1Ug@mail.gmail.com
M contrib/test_decoding/expected/stream.out
M contrib/test_decoding/sql/stream.sql
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h
Fix typo in heapam.c
commit : 190fa5a00a8f9ecee8eef2c8e26136b772b94e19
author : Michael Paquier <[email protected]>
date : Wed, 26 May 2021 19:53:03 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 26 May 2021 19:53:03 +0900
Author: Hou Zhijie
Discussion: https://postgr.es/m/OS0PR01MB571612191738540B27A8DE5894249@OS0PR01MB5716.jpnprd01.prod.outlook.com
M src/backend/access/heap/heapam.c
Make detach-partition-concurrently-4 less timing sensitive
commit : eb43bdbf5104c183412aac0fccf8e515e60d9212
author : Alvaro Herrera <[email protected]>
date : Tue, 25 May 2021 19:32:22 -0400
committer: Alvaro Herrera <[email protected]>
date : Tue, 25 May 2021 19:32:22 -0400
Same as 5e0b1aeb2dfe, for the companion test file.
This one seems lower probability (only two failures in a month of runs);
I was hardly able to reproduce a failure without a patch, so the fact
that I was also unable to reproduce one with it doesn't say anything.
We'll have to wait for further buildfarm results to see if we need any
further adjustments.
Discussion: https://postgr.es/m/[email protected]
M src/test/isolation/expected/detach-partition-concurrently-4.out
M src/test/isolation/specs/detach-partition-concurrently-4.spec
Fix use of uninitialized variable in inline_function().
commit : e30e3fdea873e4e9517c490232ea1d3bcef6c643
author : Tom Lane <[email protected]>
date : Tue, 25 May 2021 12:55:52 -0400
committer: Tom Lane <[email protected]>
date : Tue, 25 May 2021 12:55:52 -0400
Commit e717a9a18 introduced a code path that bypassed the call of
get_expr_result_type, which is not good because we need its rettupdesc
result to pass to check_sql_fn_retval. We'd failed to notice right
away because the code path in which check_sql_fn_retval uses that
argument is fairly hard to reach in this context. It's not impossible
though, and in any case inline_function would have no business
assuming that check_sql_fn_retval doesn't need that value.
To fix, move get_expr_result_type out of the if-block, which in
turn requires moving the construction of the dummy FuncExpr
out of it.
Per report from Ranier Vilela. (I'm bemused by the lack of any
compiler complaints...)
Discussion: https://postgr.es/m/CAEudQAqBqQpQ3HruWAGU_7WaMJ7tntpk0T8k_dVtNB46DqdBgw@mail.gmail.com
M src/backend/optimizer/util/clauses.c
Make detach-partition-concurrently-3 less timing-sensitive
commit : 5e0b1aeb2dfed4f1eb7ac5154c1573885a70db41
author : Alvaro Herrera <[email protected]>
date : Tue, 25 May 2021 12:53:29 -0400
committer: Alvaro Herrera <[email protected]>
date : Tue, 25 May 2021 12:53:29 -0400
This recently added test has shown to be too sensitive to timing when
sending a cancel to a session waiting for a lock.
We fix this by running a no-op query in the blocked session immediately
after the cancel; this avoids the session that sent the cancel sending
another query immediately before the cancel has been reported.
Idea by Noah Misch.
With that fix, we sometimes see that the cancel error report is shown
only relative to the step that is cancelled, instead of together with
the step that sends the cancel. To increase the probability that both
steps are shown togeter, add a 0.1s sleep to the cancel. In normal
conditions this appears sufficient to silence most failures, but we'll
see that the slower buildfarm members say about it.
Reported-by: Takamichi Osumi <[email protected]>
Discussion: https://postgr.es/m/OSBPR01MB4888C4ABA361C7E81094AC66ED269@OSBPR01MB4888.jpnprd01.prod.outlook.com
M src/test/isolation/expected/detach-partition-concurrently-3.out
M src/test/isolation/specs/detach-partition-concurrently-3.spec
postgresql.conf.sample: Make vertical spacing consistent
commit : 8673a37c85fef00dd5b9c04197538142bec10542
author : Peter Eisentraut <[email protected]>
date : Tue, 25 May 2021 11:49:54 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 25 May 2021 11:49:54 +0200
M src/backend/utils/misc/postgresql.conf.sample
Fix memory leak when de-toasting compressed values in VACUUM FULL/CLUSTER
commit : fb0f5f0172edf9f63c8f70ea9c1ec043b61c770e
author : Michael Paquier <[email protected]>
date : Tue, 25 May 2021 14:27:18 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 25 May 2021 14:27:18 +0900
VACUUM FULL and CLUSTER can be used to enforce the use of the existing
compression method of a toastable column if a value currently stored is
compressed with a method that does not match the column's defined
method. The code in charge of decompressing and recompressing toast
values at rewrite left around the detoasted values, causing an
accumulation of memory allocated in TopTransactionContext.
When processing large relations, this could cause the system to run out
of memory. The detoasted values are not needed once their tuple is
rewritten, and this commit ensures that the necessary cleanup happens.
Issue introduced by bbe0a81d. The comments of the area are reordered a
bit while on it.
Reported-by: Andres Freund
Analyzed-by: Andres Freund
Author: Michael Paquier
Reviewed-by: Dilip Kumar
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam_handler.c
Doc: Update logical decoding stats information.
commit : 0c6b92d9c6fb74255467573fde54f65139b26603
author : Amit Kapila <[email protected]>
date : Tue, 25 May 2021 10:51:45 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 25 May 2021 10:51:45 +0530
Add the information of pg_stat_replication_slots view along with other
system catalogs related to logical decoding.
Author: Vignesh C
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/logicaldecoding.sgml
Improve docs and error messages for parallel vacuum.
commit : 0734b0e983443882ec509ab4501c30ba9b706f5f
author : Amit Kapila <[email protected]>
date : Tue, 25 May 2021 09:26:53 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 25 May 2021 09:26:53 +0530
The error messages, docs, and one of the options were using
'parallel degree' to indicate parallelism used by vacuum command. We
normally use 'parallel workers' at other places so change it for parallel
vacuum accordingly.
Author: Bharath Rupireddy
Reviewed-by: Dilip Kumar, Amit Kapila
Backpatch-through: 13
Discussion: https://postgr.es/m/CALj2ACWz=PYrrFXVsEKb9J1aiX4raA+UBe02hdRp_zqDkrWUiw@mail.gmail.com
M doc/src/sgml/ref/vacuumdb.sgml
M src/backend/commands/vacuum.c
M src/bin/scripts/vacuumdb.c
M src/test/regress/expected/vacuum.out
Disallow SSL renegotiation
commit : 01e6f1a842f406170e5f717305e4a6cf0e84b3ee
author : Michael Paquier <[email protected]>
date : Tue, 25 May 2021 10:10:09 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 25 May 2021 10:10:09 +0900
SSL renegotiation is already disabled as of 48d23c72, however this does
not prevent the server to comply with a client willing to use
renegotiation. In the last couple of years, renegotiation had its set
of security issues and flaws (like the recent CVE-2021-3449), and it
could be possible to crash the backend with a client attempting
renegotiation.
This commit takes one extra step by disabling renegotiation in the
backend in the same way as SSL compression (f9264d15) or tickets
(97d3a0b0). OpenSSL 1.1.0h has added an option named
SSL_OP_NO_RENEGOTIATION able to achieve that. In older versions
there is an option called SSL3_FLAGS_NO_RENEGOTIATE_CIPHERS that
was undocumented, and could be set within the SSL object created when
the TLS connection opens, but I have decided not to use it, as it feels
trickier to rely on, and it is not official. Note that this option is
not usable in OpenSSL < 1.1.0h as the internal contents of the *SSL
object are hidden to applications.
SSL renegotiation concerns protocols up to TLSv1.2.
Per original report from Robert Haas, with a patch based on a suggestion
by Andres Freund.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/backend/libpq/be-secure-openssl.c
Fix setrefs.c code for Result Cache nodes
commit : cba5c70b956810c61b3778f7041f92fbb8065acb
author : David Rowley <[email protected]>
date : Tue, 25 May 2021 12:50:22 +1200
committer: David Rowley <[email protected]>
date : Tue, 25 May 2021 12:50:22 +1200
Result Cache, added in 9eacee2e6 neglected to properly adjust the plan
references in setrefs.c. This could lead to the following error during
EXPLAIN:
ERROR: cannot decompile join alias var in plan tree
Fix that.
Bug: 17030
Reported-by: Hans Buschmann
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/plan/setrefs.c
M src/test/regress/expected/join.out
Consider triggering VACUUM failsafe during scan.
commit : c242baa4a831ac2e7dcaec85feb410aefa3a996e
author : Peter Geoghegan <[email protected]>
date : Mon, 24 May 2021 17:14:02 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 24 May 2021 17:14:02 -0700
The wraparound failsafe mechanism added by commit 1e55e7d1 handled the
one-pass strategy case (i.e. the "table has no indexes" case) by adding
a dedicated failsafe check. This made up for the fact that the usual
one-pass checks inside lazy_vacuum_all_indexes() cannot ever be reached
during a one-pass strategy VACUUM.
This approach failed to account for two-pass VACUUMs that opt out of
index vacuuming up-front. The INDEX_CLEANUP off case in the only case
that works like that.
Fix this by performing a failsafe check every 4GB during the first scan
of the heap, regardless of the details of the VACUUM. This eliminates
the special case, and will make the failsafe trigger more reliably.
Author: Peter Geoghegan <[email protected]>
Reported-By: Andres Freund <[email protected]>
Reviewed-By: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/vacuumlazy.c
Doc: move some catalogs.sgml entries to the right place.
commit : 713a431c781fbfe1a22fae4991836077f0f4c513
author : Tom Lane <[email protected]>
date : Mon, 24 May 2021 18:03:47 -0400
committer: Tom Lane <[email protected]>
date : Mon, 24 May 2021 18:03:47 -0400
pg_statistic_ext_data.stxdexpr was listed under the wrong catalog,
as was pg_stats_ext.exprs. Also there was a bogus entry for
pg_statistic_ext_data.stxexprs. Apparently a merge failure in
commit a4d75c86b.
Guillaume Lelarge and Tom Lane
Discussion: https://postgr.es/m/CAECtzeUHw+w64eUFVeV_2FJviAw6oZ0wNLkmU843ZH4hAQfiWg@mail.gmail.com
M doc/src/sgml/catalogs.sgml
Add missing NULL check when building Result Cache paths
commit : 99c5852e20a0987eca1c38ba0c09329d4076b6a0
author : David Rowley <[email protected]>
date : Mon, 24 May 2021 12:37:11 +1200
committer: David Rowley <[email protected]>
date : Mon, 24 May 2021 12:37:11 +1200
Code added in 9e215378d to disable building of Result Cache paths when
not all join conditions are part of the parameterization of a unique
join failed to first check if the inner path's param_info was set before
checking the param_info's ppi_clauses.
Add a check for NULL values here and just bail on trying to build the
path if param_info is NULL. lateral_vars are not considered when
deciding if the join is unique, so we're not missing out on doing the
optimization when there are lateral_vars and no param_info.
Reported-by: Coverity, via Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/path/joinpath.c
Re-order pg_attribute columns to eliminate some padding space.
commit : f5024d8d7b04de2f5f4742ab433cc38160354861
author : Tom Lane <[email protected]>
date : Sun, 23 May 2021 12:12:09 -0400
committer: Tom Lane <[email protected]>
date : Sun, 23 May 2021 12:12:09 -0400
Now that attcompression is just a char, there's a lot of wasted
padding space after it. Move it into the group of char-wide
columns to save a net of 4 bytes per pg_attribute entry. While
we're at it, swap the order of attstorage and attalign to make for
a more logical grouping of these columns.
Also re-order actions in related code to match the new field ordering.
This patch also fixes one outright bug: equalTupleDescs() failed to
compare attcompression. That could, for example, cause relcache
reload to fail to adopt a new value following a change.
Michael Paquier and Tom Lane, per a gripe from Andres Freund.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M src/backend/access/common/tupdesc.c
M src/backend/access/spgist/spgutils.c
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/commands/tablecmds.c
M src/include/access/spgist_private.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_attribute.h
Be more verbose when the postmaster unexpectedly quits.
commit : bc2a389efb3b52d259cefd53c16cfa00742116f2
author : Tom Lane <[email protected]>
date : Sun, 23 May 2021 10:50:21 -0400
committer: Tom Lane <[email protected]>
date : Sun, 23 May 2021 10:50:21 -0400
Emit a LOG message when the postmaster stops because of a failure in
the startup process. There already is a similar message if we exit
for that reason during PM_STARTUP phase, so it seems inconsistent
that there was none if the startup process fails later on.
Also emit a LOG message when the postmaster stops after a crash
because restart_after_crash is disabled. This seems potentially
helpful in case DBAs (or developers) forget that that's set.
Also, it was the only remaining place where the postmaster would
do an abnormal exit without any comment as to why.
In passing, remove an unreachable call of ExitPostmaster(0).
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/postmaster.c
doc: word-wrap and indent PG 14 relnotes
commit : 8f73ed6b659464274eb9cc8358588b569960d0be
author : Bruce Momjian <[email protected]>
date : Sat, 22 May 2021 22:25:05 -0400
committer: Bruce Momjian <[email protected]>
date : Sat, 22 May 2021 22:25:05 -0400
M doc/src/sgml/release-14.sgml
Fix access to no-longer-open relcache entry in logical-rep worker.
commit : b39630fd41f25b414d0ea9b30804f4105f2a0aff
author : Tom Lane <[email protected]>
date : Sat, 22 May 2021 21:24:48 -0400
committer: Tom Lane <[email protected]>
date : Sat, 22 May 2021 21:24:48 -0400
If we redirected a replicated tuple operation into a partition child
table, and then tried to fire AFTER triggers for that event, the
relation cache entry for the child table was already closed. This has
no visible ill effects as long as the entry is still there and still
valid, but an unluckily-timed cache flush could result in a crash or
other misbehavior.
To fix, postpone the ExecCleanupTupleRouting call (which is what
closes the child table) until after we've fired triggers. This
requires a bit of refactoring so that the cleanup function can
have access to the necessary state.
In HEAD, I took the opportunity to simplify some of worker.c's
function APIs based on use of the new ApplyExecutionData struct.
However, it doesn't seem safe/practical to back-patch that aspect,
at least not without a lot of analysis of possible interactions
with a04daa97a.
In passing, add an Assert to afterTriggerInvokeEvents to catch
such cases. This seems worthwhile because we've grown a number
of fairly unstructured ways of calling AfterTriggerEndQuery.
Back-patch to v13, where worker.c grew the ability to deal with
partitioned target tables.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/trigger.c
M src/backend/replication/logical/worker.c
doc: PG 14 relnotes, adjust pg_{read|write}_all_data entry
commit : 7ce7d07e1c5fb33ee56bda235ae3d53f162f3bc0
author : Bruce Momjian <[email protected]>
date : Sat, 22 May 2021 20:17:58 -0400
committer: Bruce Momjian <[email protected]>
date : Sat, 22 May 2021 20:17:58 -0400
Reported-by: Stephen Frost
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/release-14.sgml
Update PG 14 relnotes for vacuum_cost_page_miss
commit : 8dcae7f0a3d6aba1afad1599ab18d259c417b4ee
author : Bruce Momjian <[email protected]>
date : Sat, 22 May 2021 19:24:23 -0400
committer: Bruce Momjian <[email protected]>
date : Sat, 22 May 2021 19:24:23 -0400
Reported-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-WzmgSnDX9WVoxRZxuKeCy2MzLO9Dmo4+go0RzNW0VBdhmw@mail.gmail.com
M doc/src/sgml/release-14.sgml
Remove plpgsql's special-case code paths for SET/RESET.
commit : 30168be8f75b95183abccf48f0da7a64a0cfbd9f
author : Tom Lane <[email protected]>
date : Sat, 22 May 2021 10:25:36 -0400
committer: Tom Lane <[email protected]>
date : Sat, 22 May 2021 10:25:36 -0400
In the wake of 84f5c2908, it's no longer necessary for plpgsql to
handle SET/RESET specially. The point of that was just to avoid
taking a new transaction snapshot prematurely, which the regular code
path through _SPI_execute_plan() now does just fine (in fact better,
since it now does the right thing for LOCK too). Hence, rip out a
few lines of code, going back to the old way of treating SET/RESET
as a generic SQL command. This essentially reverts all but the
test cases from b981275b6.
Discussion: https://postgr.es/m/[email protected]
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_funcs.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/pl_unreserved_kwlist.h
M src/pl/plpgsql/src/plpgsql.h
Fix planner's use of Result Cache with unique joins
commit : 9e215378d7fbb7d4615be917917c52f246cc6c61
author : David Rowley <[email protected]>
date : Sat, 22 May 2021 16:22:27 +1200
committer: David Rowley <[email protected]>
date : Sat, 22 May 2021 16:22:27 +1200
When the planner considered using a Result Cache node to cache results
from the inner side of a Nested Loop Join, it failed to consider that the
inner path's parameterization may not be the entire join condition. If
the join was marked as inner_unique then we may accidentally put the cache
in singlerow mode. This meant that entries would be marked as complete
after caching the first row. That was wrong as if only part of the join
condition was parameterized then the uniqueness of the unique join was not
guaranteed at the Result Cache's level. The uniqueness is only guaranteed
after Nested Loop applies the join filter. If subsequent rows were found,
this would lead to:
ERROR: cache entry already complete
This could have been fixed by only putting the cache in singlerow mode if
the entire join condition was parameterized. However, Nested Loop will
only read its inner side so far as the first matching row when the join is
unique, so that might mean we never get an opportunity to mark cache
entries as complete. Since non-complete cache entries are useless for
subsequent lookups, we just don't bother considering a Result Cache path
in this case.
In passing, remove the XXX comment that claimed the above ERROR might be
better suited to be an Assert. After there being an actual case which
triggered it, it seems better to keep it an ERROR.
Reported-by: David Christensen
Discussion: https://postgr.es/m/CAOxo6X+dy-V58iEPFgst8ahPKEU+38NZzUuc+a7wDBZd4TrHMQ@mail.gmail.com
M src/backend/executor/nodeResultCache.c
M src/backend/optimizer/path/joinpath.c
doc: complete adding XML markup to PG 14 relnotes
commit : 0cdaa05b40e9f28e5d6d58ccd06fe19f3cd920c9
author : Bruce Momjian <[email protected]>
date : Fri, 21 May 2021 20:51:53 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 21 May 2021 20:51:53 -0400
M doc/src/sgml/release-14.sgml
doc: more XML markup for PG 14 release notes
commit : 55370f8db96c8416940ad0b05be7a00a9f059a9f
author : Bruce Momjian <[email protected]>
date : Fri, 21 May 2021 16:16:56 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 21 May 2021 16:16:56 -0400
M doc/src/sgml/release-14.sgml
Disallow whole-row variables in GENERATED expressions.
commit : 4b10074453d182b5fc11a5667bab2ef8532ff3a6
author : Tom Lane <[email protected]>
date : Fri, 21 May 2021 15:12:08 -0400
committer: Tom Lane <[email protected]>
date : Fri, 21 May 2021 15:12:08 -0400
This was previously allowed, but I think that was just an oversight.
It's a clear violation of the rule that a generated column cannot
depend on itself or other generated columns. Moreover, because the
code was relying on the assumption that no such cross-references
exist, it was pretty easy to crash ALTER TABLE and perhaps other
places. Even if you managed not to crash, you got quite unstable,
implementation-dependent results.
Per report from Vitaly Ustinov.
Back-patch to v12 where GENERATED came in.
Discussion: https://postgr.es/m/CAM_DEiWR2DPT6U4xb-Ehigozzd3n3G37ZB1+867zbsEVtYoJww@mail.gmail.com
M src/backend/catalog/heap.c
M src/test/regress/expected/generated.out
M src/test/regress/sql/generated.sql
Fix usage of "tableoid" in GENERATED expressions.
commit : 2b0ee126bbf01cbfd657bd53c94f9284ba903ca2
author : Tom Lane <[email protected]>
date : Fri, 21 May 2021 15:02:06 -0400
committer: Tom Lane <[email protected]>
date : Fri, 21 May 2021 15:02:06 -0400
We consider this supported (though I've got my doubts that it's a
good idea, because tableoid is not immutable). However, several
code paths failed to fill the field in soon enough, causing such
a GENERATED expression to see zero or the wrong value. This
occurred when ALTER TABLE adds a new GENERATED column to a table
with existing rows, and during regular INSERT or UPDATE on a
foreign table with GENERATED columns.
Noted during investigation of a report from Vitaly Ustinov.
Back-patch to v12 where GENERATED came in.
Discussion: https://postgr.es/m/CAM_DEiWR2DPT6U4xb-Ehigozzd3n3G37ZB1+867zbsEVtYoJww@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/backend/executor/nodeModifyTable.c
M src/test/regress/expected/generated.out
M src/test/regress/sql/generated.sql
Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.
commit : 84f5c2908dad81e8622b0406beea580e40bb03ac
author : Tom Lane <[email protected]>
date : Fri, 21 May 2021 14:03:53 -0400
committer: Tom Lane <[email protected]>
date : Fri, 21 May 2021 14:03:53 -0400
COMMIT/ROLLBACK necessarily destroys all snapshots within the session.
The original implementation of intra-procedure transactions just
cavalierly did that, ignoring the fact that this left us executing in
a rather different environment than normal. In particular, it turns
out that handling of toasted datums depends rather critically on there
being an outer ActiveSnapshot: otherwise, when SPI or the core
executor pop whatever snapshot they used and return, it's unsafe to
dereference any toasted datums that may appear in the query result.
It's possible to demonstrate "no known snapshots" and "missing chunk
number N for toast value" errors as a result of this oversight.
Historically this outer snapshot has been held by the Portal code,
and that seems like a good plan to preserve. So add infrastructure
to pquery.c to allow re-establishing the Portal-owned snapshot if it's
not there anymore, and add enough bookkeeping support that we can tell
whether it is or not.
We can't, however, just re-establish the Portal snapshot as part of
COMMIT/ROLLBACK. As in normal transaction start, acquiring the first
snapshot should wait until after SET and LOCK commands. Hence, teach
spi.c about doing this at the right time. (Note that this patch
doesn't fix the problem for any PLs that try to run intra-procedure
transactions without using SPI to execute SQL commands.)
This makes SPI's no_snapshots parameter rather a misnomer, so in HEAD,
rename that to allow_nonatomic.
replication/logical/worker.c also needs some fixes, because it wasn't
careful to hold a snapshot open around AFTER trigger execution.
That code doesn't use a Portal, which I suspect someday we're gonna
have to fix. But for now, just rearrange the order of operations.
This includes back-patching the recent addition of finish_estate()
to centralize the cleanup logic there.
This also back-patches commit 2ecfeda3e into v13, to improve the
test coverage for worker.c (it was that test that exposed that
worker.c's snapshot management is wrong).
Per bug #15990 from Andreas Wicht. Back-patch to v11 where
intra-procedure COMMIT was added.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/spi.sgml
M src/backend/commands/functioncmds.c
M src/backend/executor/spi.c
M src/backend/replication/logical/worker.c
M src/backend/tcop/pquery.c
M src/backend/utils/mmgr/portalmem.c
M src/include/executor/spi.h
M src/include/tcop/pquery.h
M src/include/utils/portal.h
M src/pl/plpgsql/src/pl_exec.c
M src/test/isolation/expected/plpgsql-toast.out
M src/test/isolation/specs/plpgsql-toast.spec
Put some psql documentation pieces back into alphabetical order
commit : 124966c1a35b950210e12048e64533963960febd
author : Peter Eisentraut <[email protected]>
date : Fri, 21 May 2021 17:10:09 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 21 May 2021 17:10:09 +0200
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/help.c
Fix deadlock for multiple replicating truncates of the same table.
commit : 6d0eb38557155855539cd007f04736dc3b2ba16f
author : Amit Kapila <[email protected]>
date : Fri, 21 May 2021 07:54:27 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 21 May 2021 07:54:27 +0530
While applying the truncate change, the logical apply worker acquires
RowExclusiveLock on the relation being truncated. This allowed truncate on
the relation at a time by two apply workers which lead to a deadlock. The
reason was that one of the workers after updating the pg_class tuple tries
to acquire SHARE lock on the relation and started to wait for the second
worker which has acquired RowExclusiveLock on the relation. And when the
second worker tries to update the pg_class tuple, it starts to wait for
the first worker which leads to a deadlock. Fix it by acquiring
AccessExclusiveLock on the relation before applying the truncate change as
we do for normal truncate operation.
Author: Peter Smith, test case by Haiying Tang
Reviewed-by: Dilip Kumar, Amit Kapila
Backpatch-through: 11
Discussion: https://postgr.es/m/CAHut+PsNm43p0jM+idTvWwiGZPcP0hGrHMPK9TOAkc+a4UpUqw@mail.gmail.com
M src/backend/replication/logical/worker.c
M src/test/subscription/t/010_truncate.pl
Avoid detoasting failure after COMMIT inside a plpgsql FOR loop.
commit : f21fadafaf0fb5ea4c9622d915972651273d62ce
author : Tom Lane <[email protected]>
date : Thu, 20 May 2021 18:32:37 -0400
committer: Tom Lane <[email protected]>
date : Thu, 20 May 2021 18:32:37 -0400
exec_for_query() normally tries to prefetch a few rows at a time
from the query being iterated over, so as to reduce executor
entry/exit overhead. Unfortunately this is unsafe if we have
COMMIT or ROLLBACK within the loop, because there might be
TOAST references in the data that we prefetched but haven't
yet examined. Immediately after the COMMIT/ROLLBACK, we have
no snapshots in the session, meaning that VACUUM is at liberty
to remove recently-deleted TOAST rows.
This was originally reported as a case triggering the "no known
snapshots" error in init_toast_snapshot(), but even if you miss
hitting that, you can get "missing toast chunk", as illustrated
by the added isolation test case.
To fix, just disable prefetching in non-atomic contexts. Maybe
there will be performance complaints prompting us to work harder
later, but it's not clear at the moment that this really costs
much, and I doubt we'd want to back-patch any complicated fix.
In passing, adjust that error message in init_toast_snapshot()
to be a little clearer about the likely cause of the problem.
Patch by me, based on earlier investigation by Konstantin Knizhnik.
Per bug #15990 from Andreas Wicht. Back-patch to v11 where
intra-procedure COMMIT was added.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/common/toast_internals.c
M src/pl/plpgsql/src/pl_exec.c
M src/test/isolation/expected/plpgsql-toast.out
M src/test/isolation/specs/plpgsql-toast.spec
doc: change PG 14 relnotes as suggested by Justin Pryzby
commit : 4f586fe244a296d7c781de3f06c54755f2ae222b
author : Bruce Momjian <[email protected]>
date : Thu, 20 May 2021 15:50:46 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 20 May 2021 15:50:46 -0400
M doc/src/sgml/release-14.sgml
Install PostgresVersion.pm
commit : bdbb2ce7d51e93ca2ec68e25e2fafb271b34e72d
author : Andrew Dunstan <[email protected]>
date : Thu, 20 May 2021 15:11:17 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 20 May 2021 15:11:17 -0400
A lamentable oversight on my part meant that when PostgresVersion.pm was
added in commit 4c4eaf3d19 provision to install it was not added to the
Makefile, so it was not installed along with the other perl modules.
M src/test/perl/Makefile
Clean up cpluspluscheck violation.
commit : 6d59a218c38adf5b993200a804713df4982a0c75
author : Tom Lane <[email protected]>
date : Thu, 20 May 2021 13:03:08 -0400
committer: Tom Lane <[email protected]>
date : Thu, 20 May 2021 13:03:08 -0400
"typename" is a C++ keyword, so pg_upgrade.h fails to compile in C++.
Fortunately, there seems no likely reason for somebody to need to
do that. Nonetheless, it's project policy that all .h files should
pass cpluspluscheck, so rename the argument to fix that.
Oversight in 57c081de0; back-patch as that was. (The policy requiring
pg_upgrade.h to pass cpluspluscheck only goes back to v12, but it
seems best to keep this code looking the same in all branches.)
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/version.c
Use a more portable way to get the version string in PostgresNode
commit : 8bdd6f563aa2456de602e78991e6a9f61b8ec86d
author : Andrew Dunstan <[email protected]>
date : Thu, 20 May 2021 08:03:15 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 20 May 2021 08:03:15 -0400
Older versions of perl on Windows don't like the list form of pipe open,
and perlcritic doesn't like the string form of open, so we avoid both
with a simpler formulation using qx{}.
Per complaint from Amit Kapila.
M src/test/perl/PostgresNode.pm
Avoid creating testtablespace directories where not wanted.
commit : 413c1ef98e0c9c708c4a9a13a838a55b65b16a80
author : Tom Lane <[email protected]>
date : Wed, 19 May 2021 14:04:01 -0400
committer: Tom Lane <[email protected]>
date : Wed, 19 May 2021 14:04:01 -0400
Recently we refactored things so that pg_regress makes the
"testtablespace" subdirectory used by the core regression tests,
instead of doing that in the makefiles. That had the undesirable
side effect of making such a subdirectory in every directory that
has "input" or "output" test files. Since these subdirectories
remain empty, git doesn't complain about them, but nonetheless
they're clutter.
To fix, invent an explicit --make-testtablespace-dir switch,
so that pg_regress only makes the subdirectory when explicitly
told to.
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/GNUmakefile
M src/test/regress/pg_regress.c
M src/tools/msvc/vcregress.pl
doc: revert 1e7d53bd01 so libpq chapter number is accessable
commit : 4f7d1c30966cc02fd5eba2f0d51d1f53be07d457
author : Bruce Momjian <[email protected]>
date : Wed, 19 May 2021 11:22:21 -0400
committer: Bruce Momjian <[email protected]>
date : Wed, 19 May 2021 11:22:21 -0400
Fix PG 14 relnotes to use <link> instead of <xref>. This was discussed
in commit message 59fa7eb603.
M doc/src/sgml/libpq.sgml
M doc/src/sgml/release-14.sgml
doc: add xreflabel for libpq chapter, needed for PG 14 relnotes
commit : 1e7d53bd019e9d86ef1013308715622a2e400d3b
author : Bruce Momjian <[email protected]>
date : Wed, 19 May 2021 11:01:28 -0400
committer: Bruce Momjian <[email protected]>
date : Wed, 19 May 2021 11:01:28 -0400
M doc/src/sgml/libpq.sgml
Fix pgbench permute tests.
commit : 0f516d039d8023163e82fa51104052306068dd69
author : Dean Rasheed <[email protected]>
date : Wed, 19 May 2021 12:50:58 +0100
committer: Dean Rasheed <[email protected]>
date : Wed, 19 May 2021 12:50:58 +0100
One of the tests for the pgbench permute() function added by
6b258e3d68 fails on some 32-bit platforms, due to variations in the
floating point computations in getrand(). The remaining tests give
sufficient coverage, so just remove the failing test.
Reported by Christoph Berg. Analysis by Thomas Munro and Tom Lane.
Based on patch by Fabien Coelho.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pgbench/t/001_pgbench_with_server.pl
Make standby promotion reset the recovery pause state to 'not paused'.
commit : 167bd4804995afd654bd97ca9486acbece24377e
author : Fujii Masao <[email protected]>
date : Wed, 19 May 2021 13:48:19 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 19 May 2021 13:48:19 +0900
If a promotion is triggered while recovery is paused, the paused state ends
and promotion continues. But previously in that case
pg_get_wal_replay_pause_state() returned 'paused' wrongly while a promotion
was ongoing.
This commit changes a standby promotion so that it marks the recovery
pause state as 'not paused' when it's triggered, to fix the issue.
Author: Fujii Masao
Reviewed-by: Dilip Kumar, Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
Fix 020_messages.pl test.
commit : 0a442a408b40d2c6710de7e5397cb2e769d8c630
author : Amit Kapila <[email protected]>
date : Wed, 19 May 2021 08:54:46 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 19 May 2021 08:54:46 +0530
We were not waiting for a publisher to catch up with the subscriber after
creating a subscription. Now, it can happen that apply worker starts
replication even after we have disabled the subscription in the test. This
will make the test expect that there is no active slot whereas there
exists one. Fix this symptom by allowing the publisher to wait for
catching up with the subscription.
It is not a good idea to ensure if the slot is still active by checking
for walsender existence as we release the slot after we clean up the
walsender related memory. Fix that by checking the slot status in
pg_replication_slots.
Also, it is better to avoid repeated enabling/disabling of the
subscription.
Finally, we make autovacuum off for this test to avoid any empty
transaction appearing in the test while consuming changes.
Reported-by: as per buildfarm
Author: Vignesh C
Reviewed-by: Amit Kapila, Michael Paquier
Discussion: https://postgr.es/m/CAA4eK1+uW1UGDHDz-HWMHMen76mKP7NJebOTZN4uwbyMjaYVww@mail.gmail.com
M src/test/subscription/t/020_messages.pl
doc: partial completion of XML markup for PG 14 release notes
commit : 6a5bde7d4f96ef153578eaeb624ae12e48b46e85
author : Bruce Momjian <[email protected]>
date : Tue, 18 May 2021 23:21:47 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 18 May 2021 23:21:47 -0400
M doc/src/sgml/release-14.sgml
Fix issues in pg_stat_wal.
commit : d8735b8b4651f5ed50afc472e236a8e6120f07f2
author : Fujii Masao <[email protected]>
date : Wed, 19 May 2021 11:38:34 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 19 May 2021 11:38:34 +0900
1) Previously there were both pgstat_send_wal() and pgstat_report_wal()
in order to send WAL activity to the stats collector. With the former being
used by wal writer, the latter by most other processes. They were a bit
redundant and so this commit merges them into pgstat_send_wal() to
simplify the code.
2) Previously WAL global statistics counters were calculated and then
compared with zero-filled buffer in order to determine whether any WAL
activity has happened since the last submission. These calculation and
comparison were not cheap. This was regularly exercised even in read-only
workloads. This commit fixes the issue by making some WAL activity
counters directly be checked to determine if there's WAL activity stats
to send.
3) Previously pgstat_report_stat() did not check if there's WAL activity
stats to send as part of the "Don't expend a clock check if nothing to do"
check at the top. It's probably rare to have pending WAL stats without
also passing one of the other conditions, but for safely this commit
changes pgstat_report_stats() so that it checks also some WAL activity
counters at the top.
This commit also adds the comments about the design of WAL stats.
Reported-by: Andres Freund
Author: Masahiro Ikeda
Reviewed-by: Kyotaro Horiguchi, Atsushi Torikoshi, Andres Freund, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/pgstat.c
M src/include/executor/instrument.h
M src/include/pgstat.h
Add --no-toast-compression to pg_dumpall
commit : 694da1983e9569b2a2f96cd786ead6b8dba31f1d
author : Michael Paquier <[email protected]>
date : Wed, 19 May 2021 09:38:48 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 19 May 2021 09:38:48 +0900
This is an oversight from bbe0a81d, where the equivalent option exists
in pg_dump. This is useful to be able to reset the compression methods
cluster-wide when restoring the data based on default_toast_compression.
Reviewed-by: Daniel Gustafsson, Tom Lane
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/pg_dumpall.sgml
M src/bin/pg_dump/pg_dumpall.c
doc: add PG 14 rel item about vacuum_cleanup_index_scale_factor
commit : 2e7c17837064297f25c427d58154dce8d4287302
author : Bruce Momjian <[email protected]>
date : Tue, 18 May 2021 15:17:44 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 18 May 2021 15:17:44 -0400
M doc/src/sgml/release-14.sgml
Fix typo and outdated information in README.barrier
commit : 2ded19fa3a4dafbae80245710fa371d5163bdad4
author : David Rowley <[email protected]>
date : Tue, 18 May 2021 09:54:56 +1200
committer: David Rowley <[email protected]>
date : Tue, 18 May 2021 09:54:56 +1200
README.barrier didn't seem to get the memo when atomics were added. Fix
that.
Author: Tatsuo Ishii, David Rowley
Discussion: https://postgr.es/m/20210516.211133.2159010194908437625.t-ishii%40sraoss.co.jp
Backpatch-through: 9.6, oldest supported release
M src/backend/storage/lmgr/README.barrier
Stamp 14beta1.
commit : e4f9737fac77a5cb03a84d1f4038d300ffd28afd
author : Tom Lane <[email protected]>
date : Mon, 17 May 2021 16:11:18 -0400
committer: Tom Lane <[email protected]>
date : Mon, 17 May 2021 16:11:18 -0400
M configure
M configure.ac
Remove obsolete reference to winflex download
commit : cff8436f19e1c0c278f1ee96d450507fbd43f9ef
author : Magnus Hagander <[email protected]>
date : Mon, 17 May 2021 21:54:36 +0200
committer: Magnus Hagander <[email protected]>
date : Mon, 17 May 2021 21:54:36 +0200
We used to distribute a binary version of flex for windows on our
download site, but it hasn't been working for many years. The "old
documentation" referenced was also for versions that have been EOL for
many years. So, remove it.
Discussion: https://postgr.es/m/CABUevEwXLJpVpab62f7AFXNWQ5=U0kvErCLq4VEsikidLyzSQg@mail.gmail.com
M doc/src/sgml/install-windows.sgml
doc: PG 14 relnotes adjustments from Fujii Masao
commit : fe2fb9ebcae8445fdb3915ecf8402a3a887effc2
author : Bruce Momjian <[email protected]>
date : Mon, 17 May 2021 14:05:05 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 17 May 2021 14:05:05 -0400
M doc/src/sgml/release-14.sgml
Translation updates
commit : 6292b83074243db94df89271842bda0877cbc4ce
author : Peter Eisentraut <[email protected]>
date : Mon, 17 May 2021 14:30:27 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 17 May 2021 14:30:27 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 9bbd9c3714d0c76daaa806588b1fbf744aa60496
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/bin/initdb/po/es.po
M src/bin/pg_amcheck/nls.mk
A src/bin/pg_amcheck/po/de.po
M src/bin/pg_archivecleanup/nls.mk
A src/bin/pg_archivecleanup/po/el.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_checksums/nls.mk
A src/bin/pg_checksums/po/el.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_config/nls.mk
A src/bin/pg_config/po/el.po
M src/bin/pg_config/po/es.po
M src/bin/pg_ctl/po/es.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_test_fsync/nls.mk
A src/bin/pg_test_fsync/po/el.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_timing/nls.mk
A src/bin/pg_test_timing/po/el.po
M src/bin/pg_upgrade/po/de.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/fr.po
M src/bin/pg_verifybackup/po/es.po
M src/bin/psql/po/es.po
M src/bin/scripts/po/es.po
M src/interfaces/ecpg/preproc/po/de.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/libpq/po/es.po
M src/interfaces/libpq/po/fr.po
M src/pl/plpgsql/src/po/es.po
M src/pl/tcl/nls.mk
A src/pl/tcl/po/el.po
Fix wording in description of pg_stat_statements.toplevel
commit : f9e6d00df029144fd8f4ec70c52b5a1d2444f895
author : Magnus Hagander <[email protected]>
date : Mon, 17 May 2021 10:59:54 +0200
committer: Magnus Hagander <[email protected]>
date : Mon, 17 May 2021 10:59:54 +0200
Incorrect wording got applied in 7531fcb1fcf.
Reported-By: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/pgstatstatements.sgml
Doc: Update documentation for asynchronous execution.
commit : 15fcd33e0694428d0567a6796891b759bc91e6f9
author : Etsuro Fujita <[email protected]>
date : Mon, 17 May 2021 17:30:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Mon, 17 May 2021 17:30:00 +0900
Add a note of caution on the performance of asynchronous execution by
postgres_fdw. Follow-up for commit 27e1f1456.
Stephen Frost, a little bit expanded by me.
Discussion: https://postgr.es/m/20210506171224.GV20766%40tamriel.snowman.net
M doc/src/sgml/postgres-fdw.sgml
doc: update PG 14 relnotes from feedback by Tom, Alvaro, Julien
commit : 07af57dbad589bbef9d7178d9b1cb354412e823f
author : Bruce Momjian <[email protected]>
date : Sun, 16 May 2021 23:34:50 -0400
committer: Bruce Momjian <[email protected]>
date : Sun, 16 May 2021 23:34:50 -0400
M doc/src/sgml/release-14.sgml
doc: remove XML comments around compute_query_id PG14 rel text
commit : f39b21e6a25c7269f50a709aa874e321e6f84b20
author : Bruce Momjian <[email protected]>
date : Sat, 15 May 2021 17:30:45 -0400
committer: Bruce Momjian <[email protected]>
date : Sat, 15 May 2021 17:30:45 -0400
M doc/src/sgml/release-14.sgml
doc: update PG 14 release notes for compute_query_id change
commit : 6cb5346cb15d56e6ba8288b891c7098f0aecdadc
author : Bruce Momjian <[email protected]>
date : Sat, 15 May 2021 17:26:26 -0400
committer: Bruce Momjian <[email protected]>
date : Sat, 15 May 2021 17:26:26 -0400
Also remove ALTER TYPE ...SUBSCRIPT, and update for all current commits.
M doc/src/sgml/release-14.sgml
Unbreak EXEC_BACKEND build
commit : 354f32d01dedc2c86a05be298a62cdae9710d203
author : Alvaro Herrera <[email protected]>
date : Sat, 15 May 2021 15:17:15 -0400
committer: Alvaro Herrera <[email protected]>
date : Sat, 15 May 2021 15:17:15 -0400
Per buildfarm
M src/backend/postmaster/postmaster.c
Allow compute_query_id to be set to 'auto' and make it default
commit : cafde58b337e007cb6a719f5ab4dd6459d932a39
author : Alvaro Herrera <[email protected]>
date : Sat, 15 May 2021 14:13:09 -0400
committer: Alvaro Herrera <[email protected]>
date : Sat, 15 May 2021 14:13:09 -0400
Allowing only on/off meant that all either all existing configuration
guides would become obsolete if we disabled it by default, or that we
would have to accept a performance loss in the default config if we
enabled it by default. By allowing 'auto' as a middle ground, the
performance cost is only paid by those who enable pg_stat_statements and
similar modules.
I only edited the release notes to comment-out a paragraph that is now
factually wrong; further edits are probably needed to describe the
related change in more detail.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/20210513002623.eugftm4nk2lvvks3@nol
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/pg_stat_statements.conf
M doc/src/sgml/config.sgml
M doc/src/sgml/pgstatstatements.sgml
M doc/src/sgml/release-14.sgml
M src/backend/commands/explain.c
M src/backend/parser/analyze.c
M src/backend/postmaster/postmaster.c
M src/backend/tcop/postgres.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/backend/utils/misc/queryjumble.c
M src/include/utils/guc.h
M src/include/utils/queryjumble.h
Be more careful about barriers when releasing BackgroundWorkerSlots.
commit : 30d8bad494ad1f604295033e4f4de4b8f258fe74
author : Tom Lane <[email protected]>
date : Sat, 15 May 2021 12:21:06 -0400
committer: Tom Lane <[email protected]>
date : Sat, 15 May 2021 12:21:06 -0400
ForgetBackgroundWorker lacked any memory barrier at all, while
BackgroundWorkerStateChange had one but unaccountably did
additional manipulation of the slot after the barrier. AFAICS,
the rule must be that the barrier is immediately before setting
or clearing slot->in_use.
It looks like back in 9.6 when ForgetBackgroundWorker was first
written, there might have been some case for not needing a
barrier there, but I'm not very convinced of that --- the fact
that the load of bgw_notify_pid is in the caller doesn't seem
to guarantee no memory ordering problem. So patch 9.6 too.
It's likely that this doesn't fix any observable bug on Intel
hardware, but machines with weaker memory ordering rules could
have problems here.
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/bgworker.c
Harden nbtree deduplication posting split code.
commit : 8f72bbac3e4b1d1be9598e8edb9353fa5dc48138
author : Peter Geoghegan <[email protected]>
date : Fri, 14 May 2021 15:08:02 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 14 May 2021 15:08:02 -0700
Add a defensive "can't happen" error to code that handles nbtree posting
list splits (promote an existing assertion). This avoids a segfault in
the event of an insertion of a newitem that is somehow identical to an
existing non-pivot tuple in the index. An nbtree index should never
have two index tuples with identical TIDs.
This scenario is not particular unlikely in the event of any kind of
corruption that leaves the index in an inconsistent state relative to
the heap relation that is indexed. There are two known reports of
preventable hard crashes. Doing nothing seems unacceptable given the
general expectation that nbtree will cope reasonably well with corrupt
data.
Discussion: https://postgr.es/m/CAH2-Wz=Jr_d-dOYEEmwz0-ifojVNWho01eAqewfQXgKfoe114w@mail.gmail.com
Backpatch: 13-, where nbtree deduplication was introduced.
M src/backend/access/nbtree/nbtdedup.c
Prevent infinite insertion loops in spgdoinsert().
commit : c3c35a733c77b298d3cf7e7de2eeb4aea540a631
author : Tom Lane <[email protected]>
date : Fri, 14 May 2021 15:07:34 -0400
committer: Tom Lane <[email protected]>
date : Fri, 14 May 2021 15:07:34 -0400
Formerly we just relied on operator classes that assert longValuesOK
to eventually shorten the leaf value enough to fit on an index page.
That fails since the introduction of INCLUDE-column support (commit
09c1c6ab4), because the INCLUDE columns might alone take up more
than a page, meaning no amount of leaf-datum compaction will get
the job done. At least with spgtextproc.c, that leads to an infinite
loop, since spgtextproc.c won't throw an error for not being able
to shorten the leaf datum anymore.
To fix without breaking cases that would otherwise work, add logic
to spgdoinsert() to verify that the leaf tuple size is decreasing
after each "choose" step. Some opclasses might not decrease the
size on every single cycle, and in any case, alignment roundoff
of the tuple size could obscure small gains. Therefore, allow
up to 10 cycles without additional savings before throwing an
error. (Perhaps this number will need adjustment, but it seems
quite generous right now.)
As long as we've developed this logic, let's back-patch it.
The back branches don't have INCLUDE columns to worry about, but
this seems like a good defense against possible bugs in operator
classes. We already know that an infinite loop here is pretty
unpleasant, so having a defense seems to outweigh the risk of
breaking things. (Note that spgtextproc.c is actually the only
known opclass with longValuesOK support, so that this is all moot
for known non-core opclasses anyway.)
Per report from Dilip Kumar.
Discussion: https://postgr.es/m/CAFiTN-uxP_soPhVG840tRMQTBmtA_f_Y8N51G7DKYYqDh7XN-A@mail.gmail.com
M doc/src/sgml/spgist.sgml
M src/backend/access/spgist/spgdoinsert.c
M src/test/modules/spgist_name_ops/expected/spgist_name_ops.out
M src/test/modules/spgist_name_ops/sql/spgist_name_ops.sql
Fix query-cancel handling in spgdoinsert().
commit : eb7a6b9229432dcb791f4bf0c44fe97bab661134
author : Tom Lane <[email protected]>
date : Fri, 14 May 2021 13:26:55 -0400
committer: Tom Lane <[email protected]>
date : Fri, 14 May 2021 13:26:55 -0400
Knowing that a buggy opclass could cause an infinite insertion loop,
spgdoinsert() intended to allow its loop to be interrupted by query
cancel. However, that never actually worked, because in iterations
after the first, we'd be holding buffer lock(s) which would cause
InterruptHoldoffCount to be positive, preventing servicing of the
interrupt.
To fix, check if an interrupt is pending, and if so fall out of
the insertion loop and service the interrupt after we've released
the buffers. If it was indeed a query cancel, that's the end of
the matter. If it was a non-canceling interrupt reason, make use
of the existing provision to retry the whole insertion. (This isn't
as wasteful as it might seem, since any upper-level index tuples we
already created should be usable in the next attempt.)
While there's no known instance of such a bug in existing release
branches, it still seems like a good idea to back-patch this to
all supported branches, since the behavior is fairly nasty if a
loop does happen --- not only is it uncancelable, but it will
quickly consume memory to the point of an OOM failure. In any
case, this code is certainly not working as intended.
Per report from Dilip Kumar.
Discussion: https://postgr.es/m/CAFiTN-uxP_soPhVG840tRMQTBmtA_f_Y8N51G7DKYYqDh7XN-A@mail.gmail.com
M src/backend/access/spgist/spgdoinsert.c
Refactor CHECK_FOR_INTERRUPTS() to add flexibility.
commit : e47f93f981ccb70b4c4c5a0966e5fa0400e11a7e
author : Tom Lane <[email protected]>
date : Fri, 14 May 2021 12:54:26 -0400
committer: Tom Lane <[email protected]>
date : Fri, 14 May 2021 12:54:26 -0400
Split up CHECK_FOR_INTERRUPTS() to provide an additional macro
INTERRUPTS_PENDING_CONDITION(), which just tests whether an
interrupt is pending without attempting to service it. This is
useful in situations where the caller knows that interrupts are
blocked, and would like to find out if it's worth the trouble
to unblock them.
Also add INTERRUPTS_CAN_BE_PROCESSED(), which indicates whether
CHECK_FOR_INTERRUPTS() can be relied on to clear the pending interrupt.
This commit doesn't actually add any uses of the new macros,
but a follow-on bug fix will do so. Back-patch to all supported
branches to provide infrastructure for that fix.
Alvaro Herrera and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/tcop/postgres.c
M src/include/miscadmin.h
Describe (auto-)analyze behavior for partitioned tables
commit : 1b5617eb844cd2470a334c1d2eec66cf9b39c41a
author : Alvaro Herrera <[email protected]>
date : Fri, 14 May 2021 13:10:52 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 14 May 2021 13:10:52 -0400
This explains the new behavior introduced by 0827e8af70f4 as well as
preexisting.
Author: Justin Pryzby <[email protected]>
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/ref/analyze.sgml
M doc/src/sgml/ref/pg_restore.sgml
doc: update PG 14 release notes with recent feedback
commit : 5eb1b27d20670b378508391fab01a6871a86a8e9
author : Bruce Momjian <[email protected]>
date : Fri, 14 May 2021 13:01:03 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 14 May 2021 13:01:03 -0400
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/release-14.sgml
Message style improvements
commit : 09ae329957b739dfbaf722eb5624d0a71fdff3b4
author : Peter Eisentraut <[email protected]>
date : Fri, 14 May 2021 10:21:28 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 14 May 2021 10:21:28 +0200
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_rewind/pg_rewind.c
M src/interfaces/ecpg/preproc/ecpg.trailer
doc: PG 14 release notes, reorder items by significance
commit : 521d08a21a2b1ba7038ccc815b8bccc3c9be1351
author : Bruce Momjian <[email protected]>
date : Thu, 13 May 2021 21:16:34 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 13 May 2021 21:16:34 -0400
M doc/src/sgml/release-14.sgml
Convert misleading while loop into an if condition
commit : 6cb93beddd33d00e0ce2ee55edfa32cd2a935394
author : David Rowley <[email protected]>
date : Fri, 14 May 2021 12:26:11 +1200
committer: David Rowley <[email protected]>
date : Fri, 14 May 2021 12:26:11 +1200
This seems to be leftover from ea15e1867 and from when we used to evaluate
SRFs at each node.
Since there is an unconditional "return" at the end of the loop body, only
1 loop is ever possible, so we can just change this into an if condition.
There is no actual bug being fixed here so no back-patch. It seems fine to
just fix this anomaly in master only.
Author: Greg Nancarrow
Discussion: https://postgr.es/m/CAJcOf-d7T1q0az-D8evWXnsuBZjigT04WkV5hCAOEJQZRWy28w@mail.gmail.com
M src/backend/executor/nodeResult.c
Fix autovacuum log output heap truncation issue.
commit : fbe9b80610fe17ed27ee318bdc5ba06ed86b1a71
author : Peter Geoghegan <[email protected]>
date : Thu, 13 May 2021 16:07:17 -0700
committer: Peter Geoghegan <[email protected]>
date : Thu, 13 May 2021 16:07:17 -0700
The percentage of blocks from the table value reported by autovacuum log
output (following commit 5100010ee4d) should never exceed 100% because
it describes the state of the table back when lazy_vacuum() was called.
The value could nevertheless exceed 100% in the event of heap relation
truncation. We failed to compensate for how truncation affects
rel_pages.
Fix the faulty accounting by using the original rel_pages value instead
of the current/final rel_pages value.
Reported-By: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/vacuumlazy.c
doc: PG 14 release notes, adjust updates/deletes on partitions
commit : b2d0c7c96711843c6e47fce71335d43127f81647
author : Bruce Momjian <[email protected]>
date : Thu, 13 May 2021 11:45:43 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 13 May 2021 11:45:43 -0400
M doc/src/sgml/release-14.sgml
Improve documentation example for jsonpath like_regex operator
commit : 9b7286c2b394381c937559a98f35df64a92ffbac
author : Alexander Korotkov <[email protected]>
date : Thu, 13 May 2021 16:10:21 +0300
committer: Alexander Korotkov <[email protected]>
date : Thu, 13 May 2021 16:10:21 +0300
Make sample like_regex match string values of the root object instead of the
whole document. The corrected example seems to represent a more relevant
use case.
Backpatch to 12, when jsonpath was introduced.
Discussion: https://postgr.es/m/13440f8b-4c1f-5875-c8e3-f3f65606af2f%40xs4all.nl
Author: Erik Rijkers
Reviewed-by: Michael Paquier, Alexander Korotkov
Backpatch-through: 12
M doc/src/sgml/func.sgml
Prevent asynchronous execution of direct foreign-table modifications.
commit : a784859f4480ceaa05a00ca35311071ca33483d1
author : Etsuro Fujita <[email protected]>
date : Thu, 13 May 2021 20:00:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Thu, 13 May 2021 20:00:00 +0900
Commits 27e1f1456 and 86dc90056, which were independently discussed,
cause a crash when executing an inherited foreign UPDATE/DELETE query
with asynchronous execution enabled, where children of an Append node
that is the direct/indirect child of the ModifyTable node are rewritten
so as to modify foreign tables directly by postgresPlanDirectModify();
as in that case the direct modifications are executed asynchronously,
which is not currently supported by asynchronous execution. Fix by
disabling asynchronous execution of the direct modifications in that
function.
Author: Etsuro Fujita
Reviewed-by: Amit Langote
Discussion: https://postgr.es/m/CAPmGK158e9sJOfuWxfn%2B0ynrspXQU3JhNjSCbaoeSzMvnga%2Bbw%40mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
pg_amcheck: Message style and formatting improvements
commit : 5a73a9e3b5b24cf2dd90ab4a7ae3724b2c12a0cc
author : Peter Eisentraut <[email protected]>
date : Thu, 13 May 2021 08:09:53 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 13 May 2021 08:09:53 +0200
M src/bin/pg_amcheck/pg_amcheck.c
Fix tests for replication slots stats.
commit : fc69509131c33c298e39dd25d542374e86aa3295
author : Amit Kapila <[email protected]>
date : Thu, 13 May 2021 10:14:07 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 13 May 2021 10:14:07 +0530
Some of the tests were not considering that the slot's spill stats could be
received by the stats collector after we have reset the stats. Remove
those tests and don't check total bytes decoded and sent to output plugin
in the spilled stats test as we can send the spilled stats to the stats
collector before actually sending the changes to output plugin.
Reported-by: Tom Lane as per buildfarm
Author: Vignesh C, Sawada Masahiko
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
doc: update PG 14 release notes based on current feedback
commit : b35f827b68dc1e761e17f621fbf17c3ecd073cb0
author : Bruce Momjian <[email protected]>
date : Wed, 12 May 2021 23:34:35 -0400
committer: Bruce Momjian <[email protected]>
date : Wed, 12 May 2021 23:34:35 -0400
M doc/src/sgml/release-14.sgml
Make saner the tab completion of INSERT and DELETE in psql
commit : 1906cc07d90a8e58fd381dba43c1085e9231f236
author : Michael Paquier <[email protected]>
date : Thu, 13 May 2021 09:48:28 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 13 May 2021 09:48:28 +0900
When specified directly as DML queries, INSERT was not getting always
completed to "INSERT INTO", same for DELETE with "DELETE FROM". This
makes the completion behavior more consistent for both commands, saving
a few keystrokes.
Commands on policies, triggers, grant/revoke, etc. require only DELETE
as completion keyword.
Author: Haiying Tang
Reviewed-by: Dilip Kumar, Julien Rouhaud
Discussion: https://postgr.es/m/OS0PR01MB61135AE2B07CCD1AB8C6A0F6FB549@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/bin/psql/tab-complete.c
Rename the logical replication global "wrconn"
commit : db16c656478b815627a03bb0a31833391a733eb0
author : Alvaro Herrera <[email protected]>
date : Wed, 12 May 2021 19:13:54 -0400
committer: Alvaro Herrera <[email protected]>
date : Wed, 12 May 2021 19:13:54 -0400
The worker.c global wrconn is only meant to be used by logical apply/
tablesync workers, but there are other variables with the same name. To
reduce future confusion rename the global from "wrconn" to
"LogRepWorkerWalRcvConn".
While this is just cosmetic, it seems better to backpatch it all the way
back to 10 where this code appeared, to avoid future backpatching
issues.
Author: Peter Smith <[email protected]>
Discussion: https://postgr.es/m/CAHut+Pu7Jv9L2BOEx_Z0UtJxfDevQSAUW2mJqWU+CtmDrEZVAg@mail.gmail.com
M src/backend/replication/logical/launcher.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/include/replication/worker_internal.h
Double-space commands in system_constraints.sql/system_functions.sql.
commit : 7dde98728a2ef6d48ef397ee783dd130fdb34e6b
author : Tom Lane <[email protected]>
date : Wed, 12 May 2021 18:41:39 -0400
committer: Tom Lane <[email protected]>
date : Wed, 12 May 2021 18:41:39 -0400
Previously, any error reported by the backend while reading
system_constraints.sql would report the entire file, not just the
particular command it was working on. (Ask me how I know.) Likewise,
there were chunks of system_functions.sql that would be read as one
command, which would be annoying if anything failed there.
The issue for system_constraints.sql is an oversight in commit
dfb75e478. I didn't try to trace down where the poor formatting
in system_functions.sql started, but it's certainly contrary to
the advice at the head of that file.
M src/backend/catalog/genbki.pl
M src/backend/catalog/system_functions.sql
Doc: update bki.sgml's statements about OID ranges.
commit : 1f9b0e6938054515b8c9df545437c3d347eed683
author : Tom Lane <[email protected]>
date : Wed, 12 May 2021 17:41:07 -0400
committer: Tom Lane <[email protected]>
date : Wed, 12 May 2021 17:41:07 -0400
Commit ab596105b neglected to make the docs match the code.
M doc/src/sgml/bki.sgml
Do pre-release housekeeping on catalog data.
commit : 14472442861ca95cc9158518acdedf740c4bff55
author : Tom Lane <[email protected]>
date : Wed, 12 May 2021 13:36:06 -0400
committer: Tom Lane <[email protected]>
date : Wed, 12 May 2021 13:36:06 -0400
Run renumber_oids.pl to move high-numbered OIDs down, as per pre-beta
tasks specified by RELEASE_CHANGES. For reference, the command was
./renumber_oids.pl --first-mapped-oid 8000 --target-oid 6150
M src/include/catalog/catversion.h
M src/include/catalog/pg_authid.dat
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_type.dat
Initial pgindent and pgperltidy run for v14.
commit : def5b065ff22a16a80084587613599fe15627213
author : Tom Lane <[email protected]>
date : Wed, 12 May 2021 13:14:10 -0400
committer: Tom Lane <[email protected]>
date : Wed, 12 May 2021 13:14:10 -0400
Also "make reformat-dat-files".
The only change worthy of note is that pgindent messed up the formatting
of launcher.c's struct LogicalRepWorkerId, which led me to notice that
that struct wasn't used at all anymore, so I just took it out.
M contrib/amcheck/t/001_verify_heapam.pl
M contrib/amcheck/verify_heapam.c
M contrib/amcheck/verify_nbtree.c
M contrib/old_snapshot/time_mapping.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_bloom.c
M src/backend/access/brin/brin_minmax_multi.c
M src/backend/access/brin/brin_revmap.c
M src/backend/access/brin/brin_tuple.c
M src/backend/access/common/indextuple.c
M src/backend/access/common/toast_compression.c
M src/backend/access/common/toast_internals.c
M src/backend/access/common/tupconvert.c
M src/backend/access/gist/gistproc.c
M src/backend/access/gist/gistvalidate.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/heap/heapam_visibility.c
M src/backend/access/heap/hio.c
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/index/genam.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/varsup.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogfuncs.c
M src/backend/access/transam/xloginsert.c
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/Catalog.pm
M src/backend/catalog/aclchk.c
M src/backend/catalog/dependency.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/index.c
M src/backend/catalog/objectaddress.c
M src/backend/catalog/pg_inherits.c
M src/backend/catalog/pg_proc.c
M src/backend/catalog/pg_shdepend.c
M src/backend/catalog/pg_subscription.c
M src/backend/catalog/toasting.c
M src/backend/commands/analyze.c
M src/backend/commands/copyto.c
M src/backend/commands/explain.c
M src/backend/commands/extension.c
M src/backend/commands/indexcmds.c
M src/backend/commands/subscriptioncmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/commands/typecmds.c
M src/backend/commands/vacuum.c
M src/backend/executor/execAmi.c
M src/backend/executor/execAsync.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeGather.c
M src/backend/executor/nodeGatherMerge.c
M src/backend/executor/nodeIncrementalSort.c
M src/backend/executor/nodeModifyTable.c
M src/backend/jit/llvm/llvmjit.c
M src/backend/libpq/auth.c
M src/backend/libpq/be-secure-openssl.c
M src/backend/libpq/pqcomm.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/prep/preptlist.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/analyze.c
M src/backend/parser/parse_agg.c
M src/backend/parser/parse_cte.c
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_utilcmd.c
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partdesc.c
M src/backend/port/win32_shmem.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/postmaster/syslogger.c
M src/backend/replication/basebackup.c
M src/backend/replication/logical/launcher.c
M src/backend/replication/logical/origin.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/syncrep.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walsender.c
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/file/fd.c
M src/backend/storage/file/sharedfileset.c
M src/backend/storage/ipc/latch.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/procsignal.c
M src/backend/storage/ipc/signalfuncs.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/proc.c
M src/backend/storage/lmgr/spin.c
M src/backend/storage/page/bufpage.c
M src/backend/storage/sync/sync.c
M src/backend/tcop/postgres.c
M src/backend/utils/activity/backend_progress.c
M src/backend/utils/activity/backend_status.c
M src/backend/utils/activity/wait_event.c
M src/backend/utils/adt/acl.c
M src/backend/utils/adt/dbsize.c
M src/backend/utils/adt/genfile.c
M src/backend/utils/adt/lockfuncs.c
M src/backend/utils/adt/mcxtfuncs.c
M src/backend/utils/adt/name.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/rangetypes_typanalyze.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/adt/rowtypes.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/adt/selfuncs.c
M src/backend/utils/adt/timestamp.c
M src/backend/utils/adt/varlena.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/error/elog.c
M src/backend/utils/mb/Unicode/convutils.pm
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/queryjumble.c
M src/backend/utils/sort/logtape.c
M src/backend/utils/time/snapmgr.c
M src/bin/pg_amcheck/pg_amcheck.c
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/bin/pg_amcheck/t/005_opclass_damage.pl
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_tar.c
M src/bin/pg_dump/t/010_dump_connstr.pl
M src/bin/pg_rewind/t/001_basic.pl
M src/bin/pg_rewind/t/003_extrafiles.pl
M src/bin/pg_rewind/t/008_min_recovery_point.pl
M src/bin/pg_rewind/t/RewindTest.pm
M src/bin/pg_test_fsync/pg_test_fsync.c
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/exec.c
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/psql/tab-complete.c
M src/include/access/brin_tuple.h
M src/include/access/commit_ts.h
M src/include/access/nbtree.h
M src/include/access/relscan.h
M src/include/access/toast_compression.h
M src/include/access/transam.h
M src/include/access/xact.h
M src/include/catalog/pg_aggregate.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_type.dat
M src/include/catalog/renumber_oids.pl
M src/include/commands/copy.h
M src/include/commands/copyfrom_internal.h
M src/include/commands/defrem.h
M src/include/executor/execAsync.h
M src/include/executor/functions.h
M src/include/foreign/fdwapi.h
M src/include/lib/sort_template.h
M src/include/nodes/execnodes.h
M src/include/nodes/parsenodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/pg_config_manual.h
M src/include/pgstat.h
M src/include/storage/fd.h
M src/include/storage/proc.h
M src/include/utils/backend_progress.h
M src/include/utils/backend_status.h
M src/include/utils/builtins.h
M src/include/utils/selfuncs.h
M src/include/utils/wait_event.h
M src/interfaces/ecpg/preproc/ecpg.c
M src/interfaces/ecpg/preproc/parse.pl
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/fe-trace.c
M src/port/preadv.c
M src/test/authentication/t/001_password.pl
M src/test/kerberos/t/001_auth.pl
M src/test/modules/commit_ts/t/002_standby.pl
M src/test/modules/commit_ts/t/003_standby_2.pl
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
M src/test/perl/PostgresNode.pm
M src/test/perl/PostgresVersion.pm
M src/test/perl/TestLib.pm
M src/test/recovery/t/001_stream_rep.pl
M src/test/recovery/t/003_recovery_targets.pl
M src/test/recovery/t/007_sync_rep.pl
M src/test/recovery/t/011_crash_recovery.pl
M src/test/recovery/t/021_row_visibility.pl
M src/test/recovery/t/022_crash_temp_files.pl
M src/test/recovery/t/024_archive_recovery.pl
M src/test/regress/regress.c
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/SSLServer.pm
M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/004_sync.pl
M src/test/subscription/t/010_truncate.pl
M src/test/subscription/t/015_stream.pl
M src/test/subscription/t/016_stream_subxact.pl
M src/test/subscription/t/017_stream_ddl.pl
M src/test/subscription/t/018_stream_subxact_abort.pl
M src/test/subscription/t/019_stream_subxact_ddl_abort.pl
M src/test/subscription/t/020_messages.pl
M src/test/subscription/t/100_bugs.pl
M src/tools/msvc/Install.pm
M src/tools/msvc/Mkvcbuild.pm
M src/tools/msvc/Solution.pm
M src/tools/msvc/vcregress.pl
M src/tools/pgindent/typedefs.list
Simplify one use of ScanKey in pg_subscription.c
commit : e6ccd1ce1644d1b40b7981f8bc172394de524f99
author : Michael Paquier <[email protected]>
date : Wed, 12 May 2021 14:54:02 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 12 May 2021 14:54:02 +0900
The section of the code in charge of returning all the relations
associated to a subscription only need one ScanKey, but allocated two of
them. This code was introduced as a copy-paste from a different area on
the same file by 7c4f524, making the result confusing to follow.
Author: Peter Smith
Reviewed-by: Tom Lane, Julien Rouhaud, Bharath Rupireddy
Discussion: https://postgr.es/m/CAHut+PsLKe+rN3FjchoJsd76rx2aMsFTB7CTFxRgUP05p=kcpQ@mail.gmail.com
M src/backend/catalog/pg_subscription.c
Refactor some error messages for easier translation
commit : ec6e70c79fffe9292402ee602d3742a8c7d31bd2
author : Peter Eisentraut <[email protected]>
date : Wed, 12 May 2021 07:20:10 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 12 May 2021 07:20:10 +0200
M src/backend/access/common/toast_compression.c
M src/backend/parser/parse_coerce.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/xml.c
Fix EXPLAIN ANALYZE for async-capable nodes.
commit : a363bc6da96b14d27e1cae1bae97242eb6ade5e6
author : Etsuro Fujita <[email protected]>
date : Wed, 12 May 2021 14:00:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Wed, 12 May 2021 14:00:00 +0900
EXPLAIN ANALYZE for an async-capable ForeignScan node associated with
postgres_fdw is done just by using instrumentation for ExecProcNode()
called from the node's callbacks, causing the following problems:
1) If the remote table to scan is empty, the node is incorrectly
considered as "never executed" by the command even if the node is
executed, as ExecProcNode() isn't called from the node's callbacks at
all in that case.
2) The command fails to collect timings for things other than
ExecProcNode() done in the node, such as creating a cursor for the
node's remote query.
To fix these problems, add instrumentation for async-capable nodes, and
modify postgres_fdw accordingly.
My oversight in commit 27e1f1456.
While at it, update a comment for the AsyncRequest struct in execnodes.h
and the documentation for the ForeignAsyncRequest API in fdwhandler.sgml
to match the code in ExecAsyncAppendResponse() in nodeAppend.c, and fix
typos in comments in nodeAppend.c.
Per report from Andrey Lepikhov, though I didn't use his patch.
Reviewed-by: Andrey Lepikhov
Discussion: https://postgr.es/m/2eb662bb-105d-fc20-7412-2f027cc3ca72%40postgrespro.ru
M contrib/auto_explain/auto_explain.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/fdwhandler.sgml
M src/backend/executor/execAsync.c
M src/backend/executor/execMain.c
M src/backend/executor/execProcnode.c
M src/backend/executor/instrument.c
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeForeignscan.c
M src/include/executor/instrument.h
M src/include/nodes/execnodes.h
Reduce runtime of privileges.sql test under CLOBBER_CACHE_ALWAYS.
commit : e135743ef07ea59088d09c459f41ee2eaabe95c3
author : Tom Lane <[email protected]>
date : Tue, 11 May 2021 20:59:45 -0400
committer: Tom Lane <[email protected]>
date : Tue, 11 May 2021 20:59:45 -0400
Several queries in the privileges regression test cause the planner
to apply the plpgsql function "leak()" to every element of the
histogram for atest12.b. Since commit 0c882e52a increased the size
of that histogram to 10000 entries, the test invokes that function
over 100000 times, which takes an absolutely unreasonable amount of
time in clobber-cache-always mode.
However, there's no real reason why that has to be a plpgsql
function: for the purposes of this test, all that matters is that
it not be marked leakproof. So we can replace the plpgsql
implementation with a direct call of int4lt, which has the same
behavior and is orders of magnitude faster. This is expected to
cut several hours off the buildfarm cycle time for CCA animals.
It has some positive impact in normal builds too, though that's
probably lost in the noise.
Back-patch to v13 where 0c882e52a came in.
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Change data type of counters in BufferUsage and WalUsage from long to int64.
commit : d780d7c0882fe9a385102b292907baaceb505ed0
author : Fujii Masao <[email protected]>
date : Wed, 12 May 2021 09:56:34 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 12 May 2021 09:56:34 +0900
Previously long was used as the data type for some counters in BufferUsage
and WalUsage. But long is only four byte, e.g., on Windows, and it's entirely
possible to wrap a four byte counter. For example, emitting more than
four billion WAL records in one transaction isn't actually particularly rare.
To avoid the overflows of those counters, this commit changes the data type
of them from long to int64.
Suggested-by: Andres Freund
Author: Masahiro Ikeda
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/explain.c
M src/include/executor/instrument.h
Tweak generation of Gen_dummy_probes.pl
commit : 0bf62931cae0db1294937eb9190b183494af4cf8
author : Andrew Dunstan <[email protected]>
date : Tue, 11 May 2021 20:02:02 -0400
committer: Andrew Dunstan <[email protected]>
date : Tue, 11 May 2021 20:02:02 -0400
Use a static prolog file instead of generating the prolog from the
existing perl script. Also, support generation of the file in a vpath
build.
Discussion: https://postgr.es/m/[email protected]
A src/backend/utils/Gen_dummy_probes.pl.prolog
M src/backend/utils/Makefile
M src/backend/utils/README.Gen_dummy_probes
Fix vcregress.pl's ancient misspelling of --max-connections.
commit : 0b85fa93e4575183aa5a71ebe3c6bae8d97704ed
author : Tom Lane <[email protected]>
date : Tue, 11 May 2021 19:17:07 -0400
committer: Tom Lane <[email protected]>
date : Tue, 11 May 2021 19:17:07 -0400
I copied the existing spelling of "--max_connections", but
that's just wrong :-(. Evidently setting $ENV{MAX_CONNECTIONS}
has never actually worked in this script. Given the lack of
complaints, it's probably not worth back-patching a fix.
Per buildfarm.
Discussion: https://postgr.es/m/[email protected]
M src/tools/msvc/vcregress.pl
Get rid of the separate serial_schedule list of tests.
commit : 1df3555acc78dedc3ca25eb5e83649b3da1f298f
author : Tom Lane <[email protected]>
date : Tue, 11 May 2021 17:52:04 -0400
committer: Tom Lane <[email protected]>
date : Tue, 11 May 2021 17:52:04 -0400
Having to maintain two lists of regression test scripts has proven
annoyingly error-prone. We can achieve the effect of the
serial_schedule by running the parallel_schedule with
"--max_connections=1"; so do that and remove serial_schedule.
This causes cosmetic differences in the progress output, but it
doesn't seem worth restructuring pg_regress to avoid that.
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/GNUmakefile
D src/test/regress/serial_schedule
M src/tools/msvc/vcregress.pl
doc: update PG 14 release notes based on feedback
commit : 5b2d09beaffa915edd6e74fcf030b13844d3326f
author : Bruce Momjian <[email protected]>
date : Tue, 11 May 2021 17:40:44 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 11 May 2021 17:40:44 -0400
M doc/src/sgml/release-14.sgml
Replace opr_sanity test's binary_coercible() function with C code.
commit : 6303a5730914dfe6ef2709b28b225553315c573c
author : Tom Lane <[email protected]>
date : Tue, 11 May 2021 14:28:11 -0400
committer: Tom Lane <[email protected]>
date : Tue, 11 May 2021 14:28:11 -0400
opr_sanity's binary_coercible() function has always been meant
to match the parser's notion of binary coercibility, but it also
has always been a rather poor approximation of the parser's
real rules (as embodied in IsBinaryCoercible()). That hasn't
bit us so far, but it's predictable that it will eventually.
It also now emerges that implementing this check in plpgsql
performs absolutely horribly in clobber-cache-always testing.
(Perhaps we could do something about that, but I suspect it just
means that plpgsql is exploiting catalog caching to the hilt.)
Hence, let's replace binary_coercible() with a C shim that directly
invokes IsBinaryCoercible(), eliminating both the semantic hazard
and the performance issue.
Most of regress.c's C functions are declared in create_function_1,
but we can't simply move that to before opr_sanity/type_sanity
since those tests would complain about the resulting shell types.
I chose to split it into create_function_0 and create_function_1.
Since create_function_0 now runs as part of a parallel group while
create_function_1 doesn't, reduce the latter to create just those
functions that opr_sanity and type_sanity would whine about.
To make room for create_function_0 in the second parallel group
of tests, move tstypes to the third parallel group.
In passing, clean up some ordering deviations between
parallel_schedule and serial_schedule.
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/.gitignore
M src/test/regress/expected/conversion.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/expected/type_sanity.out
A src/test/regress/input/create_function_0.source
M src/test/regress/input/create_function_1.source
A src/test/regress/output/create_function_0.source
M src/test/regress/output/create_function_1.source
M src/test/regress/parallel_schedule
M src/test/regress/regress.c
M src/test/regress/serial_schedule
M src/test/regress/sql/.gitignore
M src/test/regress/sql/conversion.sql
M src/test/regress/sql/opr_sanity.sql
M src/test/regress/sql/type_sanity.sql
Fix typo
commit : 6d177e2813a2b4415539e2861b595583cc1a8f71
author : Peter Eisentraut <[email protected]>
date : Tue, 11 May 2021 09:06:49 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 11 May 2021 09:06:49 +0200
M src/backend/parser/parse_cte.c
M src/test/regress/expected/with.out
doc: update PG 14 release notes based on feedback so far
commit : ff51679220ce31091bfdbc96d2e90fc02ac6f329
author : Bruce Momjian <[email protected]>
date : Mon, 10 May 2021 23:56:32 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 10 May 2021 23:56:32 -0400
M doc/src/sgml/release-14.sgml
Doc: Remove outdated note about run-time partition pruning
commit : 1692d0c3a3fc7716d7d00e0d657248cb98bf4df8
author : David Rowley <[email protected]>
date : Tue, 11 May 2021 15:55:33 +1200
committer: David Rowley <[email protected]>
date : Tue, 11 May 2021 15:55:33 +1200
The note is no longer true as of 86dc90056, so remove it.
Author: Amit Langote
Discussion: https://postgr.es/m/CA+HiwqFxQn7Hz1wT+wYgnf_9SK0c4BwOOwFFT8jcSZwJrd8HEA@mail.gmail.com
M doc/src/sgml/ddl.sgml
Add support for LZ4 build in MSVC scripts
commit : 9ca40dcd4d0cad43d95a9a253fafaa9a9ba7de24
author : Michael Paquier <[email protected]>
date : Tue, 11 May 2021 10:43:05 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 11 May 2021 10:43:05 +0900
Since its introduction in bbe0a81, compression of table data supports
LZ4, but nothing had been done within the MSVC scripts to allow users to
build the code with this library.
This commit closes the gap by extending the MSVC scripts to be able to
build optionally with LZ4. Getting libraries that can be used for
compilation and execution is possible as LZ4 can be compiled down to
MSVC 2010 using its source tarball. MinGW may require extra efforts to
be able to work, and I have been able to test this only with MSVC, still
this is better than nothing to give users a way to test the feature on
Windows.
Author: Dilip Kumar
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/install-windows.sgml
M src/tools/msvc/Solution.pm
M src/tools/msvc/config_default.pl
Fix mishandling of resjunk columns in ON CONFLICT ... UPDATE tlists.
commit : 049e1e2edb06854d7cd9460c22516efaa165fbf8
author : Tom Lane <[email protected]>
date : Mon, 10 May 2021 11:02:29 -0400
committer: Tom Lane <[email protected]>
date : Mon, 10 May 2021 11:02:29 -0400
It's unusual to have any resjunk columns in an ON CONFLICT ... UPDATE
list, but it can happen when MULTIEXPR_SUBLINK SubPlans are present.
If it happens, the ON CONFLICT UPDATE code path would end up storing
tuples that include the values of the extra resjunk columns. That's
fairly harmless in the short run, but if new columns are added to
the table then the values would become accessible, possibly leading
to malfunctions if they don't match the datatypes of the new columns.
This had escaped notice through a confluence of missing sanity checks,
including
* There's no cross-check that a tuple presented to heap_insert or
heap_update matches the table rowtype. While it's difficult to
check that fully at reasonable cost, we can easily add assertions
that there aren't too many columns.
* The output-column-assignment cases in execExprInterp.c lacked
any sanity checks on the output column numbers, which seems like
an oversight considering there are plenty of assertion checks on
input column numbers. Add assertions there too.
* We failed to apply nodeModifyTable's ExecCheckPlanOutput() to
the ON CONFLICT UPDATE tlist. That wouldn't have caught this
specific error, since that function is chartered to ignore resjunk
columns; but it sure seems like a bad omission now that we've seen
this bug.
In HEAD, the right way to fix this is to make the processing of
ON CONFLICT UPDATE tlists work the same as regular UPDATE tlists
now do, that is don't add "SET x = x" entries, and use
ExecBuildUpdateProjection to evaluate the tlist and combine it with
old values of the not-set columns. This adds a little complication
to ExecBuildUpdateProjection, but allows removal of a comparable
amount of now-dead code from the planner.
In the back branches, the most expedient solution seems to be to
(a) use an output slot for the ON CONFLICT UPDATE projection that
actually matches the target table, and then (b) invent a variant of
ExecBuildProjectionInfo that can be told to not store values resulting
from resjunk columns, so it doesn't try to store into nonexistent
columns of the output slot. (We can't simply ignore the resjunk columns
altogether; they have to be evaluated for MULTIEXPR_SUBLINK to work.)
This works back to v10. In 9.6, projections work much differently and
we can't cheaply give them such an option. The 9.6 version of this
patch works by inserting a JunkFilter when it's necessary to get rid
of resjunk columns.
In addition, v11 and up have the reverse problem when trying to
perform ON CONFLICT UPDATE on a partitioned table. Through a
further oversight, adjust_partition_tlist() discarded resjunk columns
when re-ordering the ON CONFLICT UPDATE tlist to match a partition.
This accidentally prevented the storing-bogus-tuples problem, but
at the cost that MULTIEXPR_SUBLINK cases didn't work, typically
crashing if more than one row has to be updated. Fix by preserving
resjunk columns in that routine. (I failed to resist the temptation
to add more assertions there too, and to do some minor code
beautification.)
Per report from Andres Freund. Back-patch to all supported branches.
Security: CVE-2021-32028
M src/backend/access/heap/heapam.c
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/prep/preptlist.c
M src/include/executor/executor.h
M src/include/nodes/plannodes.h
M src/include/optimizer/prep.h
M src/test/regress/expected/update.out
M src/test/regress/sql/update.sql
Prevent integer overflows in array subscripting calculations.
commit : f02b9085ad2f6fefd9c5cdf85579cb9f0ff0f0ea
author : Tom Lane <[email protected]>
date : Mon, 10 May 2021 10:44:38 -0400
committer: Tom Lane <[email protected]>
date : Mon, 10 May 2021 10:44:38 -0400
While we were (mostly) careful about ensuring that the dimensions of
arrays aren't large enough to cause integer overflow, the lower bound
values were generally not checked. This allows situations where
lower_bound + dimension overflows an integer. It seems that that's
harmless so far as array reading is concerned, except that array
elements with subscripts notionally exceeding INT_MAX are inaccessible.
However, it confuses various array-assignment logic, resulting in a
potential for memory stomps.
Fix by adding checks that array lower bounds aren't large enough to
cause lower_bound + dimension to overflow. (Note: this results in
disallowing cases where the last subscript position would be exactly
INT_MAX. In principle we could probably allow that, but there's a lot
of code that computes lower_bound + dimension and would need adjustment.
It seems doubtful that it's worth the trouble/risk to allow it.)
Somewhat independently of that, array_set_element() was careless
about possible overflow when checking the subscript of a fixed-length
array, creating a different route to memory stomps. Fix that too.
Security: CVE-2021-32027
M src/backend/executor/execExprInterp.c
M src/backend/utils/adt/array_userfuncs.c
M src/backend/utils/adt/arrayfuncs.c
M src/backend/utils/adt/arrayutils.c
M src/include/utils/array.h
Translation updates
commit : 6206454bdac1ccd6f6ed9d811e1a1139e663a8b9
author : Peter Eisentraut <[email protected]>
date : Mon, 10 May 2021 14:36:21 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 10 May 2021 14:36:21 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 1c361d3ac016b61715d99f2055dee050397e3f13
M src/backend/nls.mk
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/backend/po/ja.po
M src/backend/po/ko.po
M src/backend/po/ru.po
M src/backend/po/sv.po
A src/backend/po/uk.po
M src/bin/initdb/nls.mk
M src/bin/initdb/po/cs.po
M src/bin/initdb/po/de.po
A src/bin/initdb/po/el.po
M src/bin/initdb/po/es.po
M src/bin/initdb/po/fr.po
M src/bin/initdb/po/ja.po
M src/bin/initdb/po/ko.po
M src/bin/initdb/po/ru.po
M src/bin/initdb/po/uk.po
M src/bin/pg_amcheck/nls.mk
A src/bin/pg_amcheck/po/fr.po
M src/bin/pg_archivecleanup/po/cs.po
M src/bin/pg_archivecleanup/po/es.po
M src/bin/pg_archivecleanup/po/fr.po
M src/bin/pg_archivecleanup/po/ja.po
M src/bin/pg_archivecleanup/po/ko.po
M src/bin/pg_archivecleanup/po/ru.po
M src/bin/pg_archivecleanup/po/uk.po
M src/bin/pg_basebackup/nls.mk
M src/bin/pg_basebackup/po/cs.po
M src/bin/pg_basebackup/po/de.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_basebackup/po/fr.po
M src/bin/pg_basebackup/po/ja.po
M src/bin/pg_basebackup/po/ko.po
M src/bin/pg_basebackup/po/ru.po
A src/bin/pg_basebackup/po/uk.po
M src/bin/pg_checksums/nls.mk
M src/bin/pg_checksums/po/cs.po
M src/bin/pg_checksums/po/de.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_checksums/po/fr.po
M src/bin/pg_checksums/po/ja.po
M src/bin/pg_checksums/po/ko.po
M src/bin/pg_checksums/po/ru.po
A src/bin/pg_checksums/po/uk.po
A src/bin/pg_checksums/po/zh_CN.po
M src/bin/pg_config/po/cs.po
M src/bin/pg_config/po/de.po
M src/bin/pg_config/po/es.po
M src/bin/pg_config/po/fr.po
M src/bin/pg_config/po/ja.po
M src/bin/pg_config/po/ko.po
M src/bin/pg_config/po/ru.po
M src/bin/pg_config/po/uk.po
M src/bin/pg_controldata/po/cs.po
M src/bin/pg_controldata/po/es.po
M src/bin/pg_controldata/po/ja.po
M src/bin/pg_controldata/po/ko.po
M src/bin/pg_controldata/po/ru.po
M src/bin/pg_controldata/po/uk.po
M src/bin/pg_ctl/nls.mk
M src/bin/pg_ctl/po/cs.po
M src/bin/pg_ctl/po/de.po
A src/bin/pg_ctl/po/el.po
M src/bin/pg_ctl/po/es.po
M src/bin/pg_ctl/po/fr.po
M src/bin/pg_ctl/po/ja.po
M src/bin/pg_ctl/po/ko.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_ctl/po/uk.po
M src/bin/pg_dump/nls.mk
M src/bin/pg_dump/po/cs.po
M src/bin/pg_dump/po/de.po
A src/bin/pg_dump/po/el.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_dump/po/ja.po
M src/bin/pg_dump/po/ko.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/sv.po
A src/bin/pg_dump/po/uk.po
M src/bin/pg_resetwal/nls.mk
M src/bin/pg_resetwal/po/cs.po
M src/bin/pg_resetwal/po/es.po
M src/bin/pg_resetwal/po/ja.po
M src/bin/pg_resetwal/po/ko.po
M src/bin/pg_resetwal/po/ru.po
A src/bin/pg_resetwal/po/uk.po
M src/bin/pg_rewind/nls.mk
M src/bin/pg_rewind/po/cs.po
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/ja.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_rewind/po/sv.po
A src/bin/pg_rewind/po/uk.po
M src/bin/pg_test_fsync/po/de.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_fsync/po/fr.po
M src/bin/pg_test_fsync/po/ru.po
M src/bin/pg_test_fsync/po/uk.po
M src/bin/pg_test_timing/po/de.po
M src/bin/pg_test_timing/po/fr.po
M src/bin/pg_upgrade/nls.mk
M src/bin/pg_upgrade/po/cs.po
M src/bin/pg_upgrade/po/de.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/fr.po
M src/bin/pg_upgrade/po/ja.po
M src/bin/pg_upgrade/po/ko.po
M src/bin/pg_upgrade/po/ru.po
M src/bin/pg_upgrade/po/sv.po
A src/bin/pg_upgrade/po/uk.po
M src/bin/pg_verifybackup/nls.mk
A src/bin/pg_verifybackup/po/de.po
A src/bin/pg_verifybackup/po/es.po
M src/bin/pg_verifybackup/po/fr.po
A src/bin/pg_verifybackup/po/ja.po
A src/bin/pg_verifybackup/po/ko.po
A src/bin/pg_verifybackup/po/ru.po
M src/bin/pg_verifybackup/po/sv.po
A src/bin/pg_verifybackup/po/uk.po
A src/bin/pg_verifybackup/po/zh_CN.po
M src/bin/pg_waldump/nls.mk
M src/bin/pg_waldump/po/cs.po
M src/bin/pg_waldump/po/es.po
M src/bin/pg_waldump/po/ja.po
M src/bin/pg_waldump/po/ru.po
A src/bin/pg_waldump/po/uk.po
M src/bin/pg_waldump/po/zh_CN.po
M src/bin/psql/nls.mk
M src/bin/psql/po/cs.po
M src/bin/psql/po/de.po
A src/bin/psql/po/el.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ja.po
M src/bin/psql/po/ko.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/sv.po
M src/bin/psql/po/uk.po
M src/bin/scripts/nls.mk
M src/bin/scripts/po/cs.po
M src/bin/scripts/po/de.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/fr.po
M src/bin/scripts/po/ja.po
M src/bin/scripts/po/ko.po
M src/bin/scripts/po/ru.po
M src/bin/scripts/po/sv.po
A src/bin/scripts/po/uk.po
M src/interfaces/ecpg/ecpglib/po/es.po
M src/interfaces/ecpg/ecpglib/po/fr.po
M src/interfaces/ecpg/ecpglib/po/ru.po
M src/interfaces/ecpg/ecpglib/po/uk.po
M src/interfaces/ecpg/preproc/po/cs.po
M src/interfaces/ecpg/preproc/po/de.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/fr.po
M src/interfaces/ecpg/preproc/po/ko.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/ecpg/preproc/po/uk.po
M src/interfaces/libpq/nls.mk
M src/interfaces/libpq/po/cs.po
M src/interfaces/libpq/po/de.po
A src/interfaces/libpq/po/el.po
M src/interfaces/libpq/po/es.po
M src/interfaces/libpq/po/fr.po
M src/interfaces/libpq/po/ja.po
M src/interfaces/libpq/po/ko.po
M src/interfaces/libpq/po/ru.po
M src/interfaces/libpq/po/sv.po
M src/interfaces/libpq/po/uk.po
M src/interfaces/libpq/po/zh_CN.po
M src/pl/plperl/nls.mk
M src/pl/plperl/po/es.po
M src/pl/plperl/po/ru.po
A src/pl/plperl/po/uk.po
M src/pl/plpgsql/src/po/cs.po
M src/pl/plpgsql/src/po/de.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpgsql/src/po/fr.po
M src/pl/plpgsql/src/po/ko.po
M src/pl/plpgsql/src/po/ru.po
M src/pl/plpgsql/src/po/sv.po
M src/pl/plpgsql/src/po/uk.po
M src/pl/plpython/po/es.po
M src/pl/plpython/po/ru.po
M src/pl/plpython/po/uk.po
M src/pl/tcl/po/es.po
M src/pl/tcl/po/ru.po
M src/pl/tcl/po/uk.po
Emit dummy statements for probes.d probes when disabled
commit : fa8fbadb934b4727a7aeff074995e799f4685a75
author : Peter Eisentraut <[email protected]>
date : Mon, 10 May 2021 11:36:26 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 10 May 2021 11:36:26 +0200
When building without --enable-dtrace, emit dummy
do {} while (0)
statements for the stubbed-out TRACE_POSTGRESQL_foo() macros
instead of empty macros that totally elide the original probe
statement.
This fixes the
warning: suggest braces around empty body in an ‘if’ statement [-Wempty-body]
introduced by b94409a02f.
Author: Craig Ringer <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/20210504221531.cfvpmmdfsou6eitb%40alap3.anarazel.de
M src/backend/utils/Gen_dummy_probes.pl
M src/backend/utils/Gen_dummy_probes.sed
Remove unused function arguments
commit : 3c554100307f4e57c0881e205dbdbc173bb84d56
author : Peter Eisentraut <[email protected]>
date : Mon, 10 May 2021 10:02:33 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 10 May 2021 10:02:33 +0200
Was present in original commit
198b3716dba68544b55cb97bd120738a86d5df2d but apparently never used.
M src/interfaces/libpq/fe-trace.c
Fix typos in operatorcmds.c
commit : 829daab4bbe356a2f9ae0b2ee0fc98bc2279d754
author : Michael Paquier <[email protected]>
date : Mon, 10 May 2021 15:45:54 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 10 May 2021 15:45:54 +0900
Author: Kyotaro Horiguchi, Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/operatorcmds.c
doc: first draft of the PG 14 release notes
commit : dc0260861063b125d297c0f3caca359feb381c6a
author : Bruce Momjian <[email protected]>
date : Mon, 10 May 2021 01:58:59 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 10 May 2021 01:58:59 -0400
M doc/src/sgml/release-14.sgml
Fix generation of ./INSTALL for the distribution tarball
commit : 45aa88fe1d4028ea50ba7d26d390223b6ef78acc
author : Michael Paquier <[email protected]>
date : Mon, 10 May 2021 14:34:07 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 10 May 2021 14:34:07 +0900
"make dist", in charge of creating a distribution tarball, failed when
attempting to generate ./INSTALL as a new reference added to
guc-default-toast-compression on the documentation for the installation
details was not getting translated properly to plain text. Like all the
other link references on this page, this adds a new entry to
standalone-profile.xsl to allow the generation of ./INSTALL to finish
properly.
Oversight in 02a93e7, per buildfarm member guaibasaurus.
M doc/src/sgml/standalone-profile.xsl
Revert recovery prefetching feature.
commit : c2dc19342e05e081dc13b296787baa38352681ef
author : Thomas Munro <[email protected]>
date : Mon, 10 May 2021 16:00:53 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 10 May 2021 16:00:53 +1200
This set of commits has some bugs with known fixes, but at this late
stage in the release cycle it seems best to revert and resubmit next
time, along with some new automated test coverage for this whole area.
Commits reverted:
dc88460c: Doc: Review for "Optionally prefetch referenced data in recovery."
1d257577: Optionally prefetch referenced data in recovery.
f003d9f8: Add circular WAL decoding buffer.
323cbe7c: Remove read_page callback from XLogReader.
Remove the new GUC group WAL_RECOVERY recently added by a55a9847, as the
corresponding section of config.sgml is now reverted.
Discussion: https://postgr.es/m/CAOuzzgrn7iKnFRsB4MHp3UisEQAGgZMbk_ViTN4HV4-Ksq8zCg%40mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/Makefile
M src/backend/access/transam/generic_xlog.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xlog.c
D src/backend/access/transam/xlogprefetch.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogutils.c
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/logicalfuncs.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/walsender.c
M src/backend/storage/freespace/freespace.c
M src/backend/storage/ipc/ipci.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_waldump/pg_waldump.c
M src/include/access/xlog.h
D src/include/access/xlogprefetch.h
M src/include/access/xlogreader.h
M src/include/access/xlogutils.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/replication/logical.h
M src/include/utils/guc.h
M src/include/utils/guc_tables.h
M src/test/regress/expected/rules.out
M src/tools/pgindent/typedefs.list
Add more TAP tests for pg_dump with attribute compression
commit : 63db0ac3f9e6bae313da67f640c95c0045b7f0ee
author : Michael Paquier <[email protected]>
date : Mon, 10 May 2021 11:12:07 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 10 May 2021 11:12:07 +0900
Some relations with LZ4 used as the toast compression methods have been
left around in the main regression test suite to stress pg_upgrade, but
pg_dump, that includes tests much more picky in terms of output
generated, had no actual coverage with this feature.
Similarly to collations, tests only working with LZ4 are tracked with an
additional flag, and this uses TestLib::check_pg_config() to check if
the build supports LZ4 or not. This stresses more scenarios with
tables, materialized views and pg_dump --no-toast-compression.
Author: Dilip Kumar
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/CAFiTN-twgPmohG7qj1HXhySq16h123y5OowsQR+5h1YeZE9fmQ@mail.gmail.com
M src/bin/pg_dump/t/002_pg_dump.pl
doc: Fix some gaps with the documentation related to LZ4
commit : 02a93e7ef9612788081ef07ea1bbd0a8cc99ae99
author : Michael Paquier <[email protected]>
date : Mon, 10 May 2021 09:30:35 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 10 May 2021 09:30:35 +0900
The upstream project is officially named "LZ4", and the documentation
was confused with the option value that can be used with DDLs supporting
this option, and the project name.
Documentation related to the configure option --with-lz4 was missing, so
add something for that.
Author: Dilip Kumar, Michael Paquier
Reviewed-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/installation.sgml
Improve comments about USE_VALGRIND in pg_config_manual.h.
commit : 8dc3d68cbe676deb5e74d1b1b565f57fffaf107e
author : Tom Lane <[email protected]>
date : Sun, 9 May 2021 19:33:24 -0400
committer: Tom Lane <[email protected]>
date : Sun, 9 May 2021 19:33:24 -0400
These comments left the impression that USE_VALGRIND isn't really
essential for valgrind testing. But that's wrong, as I learned
the hard way today.
Discussion: https://postgr.es/m/[email protected]
M src/include/pg_config_manual.h
Move memory accounting Asserts for Result Cache code
commit : 92c4c269d24d016c19858a21347ff25a7de1f486
author : David Rowley <[email protected]>
date : Sun, 9 May 2021 11:37:18 +1200
committer: David Rowley <[email protected]>
date : Sun, 9 May 2021 11:37:18 +1200
In 9eacee2e6, I included some code to verify the cache's memory tracking
is correct by counting up the number of entries and the memory they use
each time we evict something from the cache. Those values are then
compared to the expected values using Assert. The problem is that this
requires looping over the entire cache hash table each time we evict an
entry from the cache. That can be pretty expensive, as noted by Pavel
Stehule.
Here we move this memory accounting checking code so that we only verify
it on cassert builds once when shutting down the Result Cache node.
Aside from the performance increase, this has two distinct advantages:
1) We do the memory checks at the last possible moment before destroying
the cache. This means we'll now catch accounting problems that might
sneak in after a cache eviction.
2) We now do the memory Assert checks when there were no cache evictions.
This increases the coverage.
One small disadvantage is that we'll now miss any memory tracking issues
that somehow managed to resolve themselves by the end of execution.
However, it seems to me that such a memory tracking problem would be quite
unlikely, and likely somewhat less harmful if one were to exist.
In passing, adjust the loop over the hash table to use the standard
simplehash.h method of iteration.
Reported-by: Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRAzgoSkdEiqrKbT=7yG9FA5fjUAP3jmJywuDqYq6Ki5ug@mail.gmail.com
M src/backend/executor/nodeResultCache.c
Sync guc.c and postgresql.conf.sample with the SGML docs.
commit : a55a98477b690dedb9b4368d7e5710c8e7fa534e
author : Tom Lane <[email protected]>
date : Sat, 8 May 2021 12:13:33 -0400
committer: Tom Lane <[email protected]>
date : Sat, 8 May 2021 12:13:33 -0400
It seems that various people have moved GUCs around in the config.sgml
listing without bothering to make the code agree. Ensure that the
config_group codes assigned to GUCs match where they are listed in
config.sgml. Likewise ensure that postgresql.conf.sample lists GUCs
in the same sub-section and same ordering as they appear in config.sgml.
(I've got some doubts about some of these choices, but for the purposes
of this patch, we'll treat config.sgml as gospel.)
Notably, this requires adding a WAL_RECOVERY config_group value,
because 1d257577e didn't. As long as we're renumbering that enum
anyway, let's take out the values corresponding to major groups
that are divided into sub-groups. No GUC should be assigned to the
major group itself, so those values just create a temptation to
do the wrong thing, while adding work for translators.
In passing, adjust the short_desc strings for PRESET_OPTIONS GUCs
to uniformly use the phrasing "Shows XYZ.", removing the impression
some of these strings left that you can set the value.
While some of these errors are old, no back-patch, as changing the
contents of the pg_settings view in stable branches seems more likely
to be seen as a compatibility break than anything helpful.
Bharath Rupireddy, Justin Pryzby, Tom Lane
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/utils/guc_tables.h
Doc: copy-editing for debug_invalidate_system_caches_always description.
commit : f9b809e7fbe36cd3fe1ce33edb277288a31da386
author : Tom Lane <[email protected]>
date : Sat, 8 May 2021 11:33:13 -0400
committer: Tom Lane <[email protected]>
date : Sat, 8 May 2021 11:33:13 -0400
I came to fix "useful only useful", but the more I looked at the text
the more things I thought could be improved.
M doc/src/sgml/config.sgml
Fix incorrect error code for CREATE/ALTER TABLE COMPRESSION
commit : 9681f2160dcbe2a02fd2e2db2322ea204eff6562
author : Michael Paquier <[email protected]>
date : Sat, 8 May 2021 10:18:05 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 8 May 2021 10:18:05 +0900
Specifying an incorrect value for the compression method of an attribute
caused ERRCODE_FEATURE_NOT_SUPPORTED to be raised as error. Use instead
ERRCODE_INVALID_PARAMETER_VALUE to be more consistent.
Author: Dilip Kumar
Discussion: https://postgr.es/m/CAFiTN-vH84fE-8C4zGZw4v0Wyh4Y2v=5JWg2fGE5+LPaDvz1GQ@mail.gmail.com
M src/backend/commands/tablecmds.c
Copy the INSERT query in postgres_fdw
commit : c6a01d924939306e95c8deafd09352be6a955648
author : Tomas Vondra <[email protected]>
date : Fri, 7 May 2021 22:29:43 +0200
committer: Tomas Vondra <[email protected]>
date : Fri, 7 May 2021 22:29:43 +0200
When executing the INSERT with batching, we may need to rebuild the
query when the batch size changes, in which case we pfree the current
string. We must not release the original string, stored in fdw_private,
because that may be needed in EXPLAIN ANALYZE. So make copy of the SQL,
but only for INSERT queries.
Reported-by: Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRCL_Rjw-MCR6J7VX9OF7MR6PA5K8qUbrMvprW_e-aHkfQ%40mail.gmail.com
M contrib/postgres_fdw/postgres_fdw.c
Add a README and Makefile recipe for Gen_dummy_probes.pl
commit : 8292c0675a793a5afd0a8eedbeb0db7abfb844f3
author : Andrew Dunstan <[email protected]>
date : Fri, 7 May 2021 14:27:18 -0400
committer: Andrew Dunstan <[email protected]>
date : Fri, 7 May 2021 14:27:18 -0400
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/Makefile
A src/backend/utils/README.Gen_dummy_probes
Fix typo
commit : 9f989a8581cc37879d493a5a78b0f01ec0e3245a
author : Peter Eisentraut <[email protected]>
date : Fri, 7 May 2021 17:47:22 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 7 May 2021 17:47:22 +0200
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/create_table.out
M src/test/regress/sql/create_table.sql
AlterSubscription_refresh: avoid stomping on global variable
commit : 4e8c0f1a0d0d095a749a329a216c88a340a455b6
author : Alvaro Herrera <[email protected]>
date : Fri, 7 May 2021 11:46:37 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 7 May 2021 11:46:37 -0400
This patch replaces use of the global "wrconn" variable in
AlterSubscription_refresh with a local variable of the same name, making
it consistent with other functions in subscriptioncmds.c (e.g.
DropSubscription).
The global wrconn is only meant to be used for logical apply/tablesync worker.
Abusing it this way is known to cause trouble if an apply worker
manages to do a subscription refresh, such as reported by Jeremy Finzel
and diagnosed by Andres Freund back in November 2020, at
https://www.postgresql.org/message-id/[email protected]
Backpatch to 10. In branch master, also move the connection establishment
to occur outside the PG_TRY block; this way we can remove a test for NULL in
PG_FINALLY, and it also makes the code more consistent with similar code in
the same file.
Author: Peter Smith <[email protected]>
Reviewed-by: Bharath Rupireddy <[email protected]>
Reviewed-by: Japin Li <[email protected]>
Discussion: https://postgr.es/m/CAHut+Pu7Jv9L2BOEx_Z0UtJxfDevQSAUW2mJqWU+CtmDrEZVAg@mail.gmail.com
M src/backend/commands/subscriptioncmds.c
Remove extraneous newlines added by perl copyright patch
commit : 8b82de0164c13eb3b113a525dc7eda7887f5238b
author : Andrew Dunstan <[email protected]>
date : Fri, 7 May 2021 11:37:37 -0400
committer: Andrew Dunstan <[email protected]>
date : Fri, 7 May 2021 11:37:37 -0400
M contrib/intarray/bench/bench.pl
M contrib/intarray/bench/create_test.pl
M contrib/seg/seg-validate.pl
M contrib/seg/sort-segments.pl
M src/interfaces/libpq/test/regress.pl
M src/pl/plperl/plperl_opmask.pl
M src/test/locale/sort-test.pl
M src/test/perl/PostgresNode.pm
M src/test/perl/RecursiveCopy.pm
M src/test/perl/TestLib.pm
M src/tools/git_changelog
M src/tools/msvc/pgbison.pl
M src/tools/msvc/pgflex.pl
M src/tools/msvc/vcregress.pl
M src/tools/pginclude/pgcheckdefines
M src/tools/pgindent/pgindent
Add a copyright notice to perl files lacking one.
commit : 8fa6e6919c1aaa6f74c74e16452aaf0b5f3b4cd5
author : Andrew Dunstan <[email protected]>
date : Fri, 7 May 2021 10:56:14 -0400
committer: Andrew Dunstan <[email protected]>
date : Fri, 7 May 2021 10:56:14 -0400
M contrib/amcheck/t/001_verify_heapam.pl
M contrib/auto_explain/t/001_auto_explain.pl
M contrib/bloom/t/001_wal.pl
M contrib/intarray/bench/bench.pl
M contrib/intarray/bench/create_test.pl
M contrib/oid2name/t/001_basic.pl
M contrib/seg/seg-validate.pl
M contrib/seg/sort-segments.pl
M contrib/test_decoding/t/001_repl_stats.pl
M contrib/vacuumlo/t/001_basic.pl
M src/bin/initdb/t/001_initdb.pl
M src/bin/pg_amcheck/t/001_basic.pl
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/bin/pg_amcheck/t/005_opclass_damage.pl
M src/bin/pg_archivecleanup/t/010_pg_archivecleanup.pl
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_basebackup/t/020_pg_receivewal.pl
M src/bin/pg_basebackup/t/030_pg_recvlogical.pl
M src/bin/pg_checksums/t/001_basic.pl
M src/bin/pg_checksums/t/002_actions.pl
M src/bin/pg_config/t/001_pg_config.pl
M src/bin/pg_controldata/t/001_pg_controldata.pl
M src/bin/pg_ctl/t/001_start_stop.pl
M src/bin/pg_ctl/t/002_status.pl
M src/bin/pg_ctl/t/003_promote.pl
M src/bin/pg_ctl/t/004_logrotate.pl
M src/bin/pg_dump/t/001_basic.pl
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/pg_dump/t/003_pg_dump_with_server.pl
M src/bin/pg_dump/t/010_dump_connstr.pl
M src/bin/pg_resetwal/t/001_basic.pl
M src/bin/pg_resetwal/t/002_corrupted.pl
M src/bin/pg_rewind/t/001_basic.pl
M src/bin/pg_rewind/t/002_databases.pl
M src/bin/pg_rewind/t/003_extrafiles.pl
M src/bin/pg_rewind/t/004_pg_xlog_symlink.pl
M src/bin/pg_rewind/t/005_same_timeline.pl
M src/bin/pg_rewind/t/006_options.pl
M src/bin/pg_rewind/t/007_standby_source.pl
M src/bin/pg_rewind/t/008_min_recovery_point.pl
M src/bin/pg_rewind/t/RewindTest.pm
M src/bin/pg_test_fsync/t/001_basic.pl
M src/bin/pg_test_timing/t/001_basic.pl
M src/bin/pg_verifybackup/t/001_basic.pl
M src/bin/pg_verifybackup/t/002_algorithm.pl
M src/bin/pg_verifybackup/t/003_corruption.pl
M src/bin/pg_verifybackup/t/004_options.pl
M src/bin/pg_verifybackup/t/005_bad_manifest.pl
M src/bin/pg_verifybackup/t/006_encoding.pl
M src/bin/pg_verifybackup/t/007_wal.pl
M src/bin/pg_waldump/t/001_basic.pl
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/pgbench/t/002_pgbench_no_server.pl
M src/bin/psql/t/010_tab_completion.pl
M src/bin/scripts/t/010_clusterdb.pl
M src/bin/scripts/t/011_clusterdb_all.pl
M src/bin/scripts/t/020_createdb.pl
M src/bin/scripts/t/040_createuser.pl
M src/bin/scripts/t/050_dropdb.pl
M src/bin/scripts/t/070_dropuser.pl
M src/bin/scripts/t/080_pg_isready.pl
M src/bin/scripts/t/090_reindexdb.pl
M src/bin/scripts/t/091_reindexdb_all.pl
M src/bin/scripts/t/100_vacuumdb.pl
M src/bin/scripts/t/101_vacuumdb_all.pl
M src/bin/scripts/t/102_vacuumdb_stages.pl
M src/bin/scripts/t/200_connstr.pl
M src/interfaces/libpq/test/regress.pl
M src/pl/plperl/plc_perlboot.pl
M src/pl/plperl/plc_trusted.pl
M src/pl/plperl/plperl_opmask.pl
M src/pl/plperl/text2macro.pl
M src/test/authentication/t/001_password.pl
M src/test/authentication/t/002_saslprep.pl
M src/test/kerberos/t/001_auth.pl
M src/test/ldap/t/001_auth.pl
M src/test/locale/sort-test.pl
M src/test/modules/brin/t/01_workitems.pl
M src/test/modules/commit_ts/t/001_base.pl
M src/test/modules/commit_ts/t/002_standby.pl
M src/test/modules/commit_ts/t/003_standby_2.pl
M src/test/modules/commit_ts/t/004_restart.pl
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
M src/test/modules/ssl_passphrase_callback/t/001_testfunc.pl
M src/test/modules/test_misc/t/001_constraint_validation.pl
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/perl/PostgresNode.pm
M src/test/perl/RecursiveCopy.pm
M src/test/perl/SimpleTee.pm
M src/test/perl/TestLib.pm
M src/test/recovery/t/001_stream_rep.pl
M src/test/recovery/t/002_archiving.pl
M src/test/recovery/t/003_recovery_targets.pl
M src/test/recovery/t/004_timeline_switch.pl
M src/test/recovery/t/005_replay_delay.pl
M src/test/recovery/t/006_logical_decoding.pl
M src/test/recovery/t/007_sync_rep.pl
M src/test/recovery/t/008_fsm_truncation.pl
M src/test/recovery/t/009_twophase.pl
M src/test/recovery/t/010_logical_decoding_timelines.pl
M src/test/recovery/t/011_crash_recovery.pl
M src/test/recovery/t/012_subtransactions.pl
M src/test/recovery/t/013_crash_restart.pl
M src/test/recovery/t/014_unlogged_reinit.pl
M src/test/recovery/t/015_promotion_pages.pl
M src/test/recovery/t/016_min_consistency.pl
M src/test/recovery/t/017_shm.pl
M src/test/recovery/t/018_wal_optimize.pl
M src/test/recovery/t/019_replslot_limit.pl
M src/test/recovery/t/020_archive_status.pl
M src/test/recovery/t/021_row_visibility.pl
M src/test/recovery/t/022_crash_temp_files.pl
M src/test/recovery/t/023_pitr_prepared_xact.pl
M src/test/recovery/t/024_archive_recovery.pl
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
M src/test/ssl/t/SSLServer.pm
M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/002_types.pl
M src/test/subscription/t/003_constraints.pl
M src/test/subscription/t/004_sync.pl
M src/test/subscription/t/005_encoding.pl
M src/test/subscription/t/006_rewrite.pl
M src/test/subscription/t/007_ddl.pl
M src/test/subscription/t/008_diff_schema.pl
M src/test/subscription/t/009_matviews.pl
M src/test/subscription/t/010_truncate.pl
M src/test/subscription/t/011_generated.pl
M src/test/subscription/t/012_collation.pl
M src/test/subscription/t/013_partition.pl
M src/test/subscription/t/014_binary.pl
M src/test/subscription/t/015_stream.pl
M src/test/subscription/t/016_stream_subxact.pl
M src/test/subscription/t/017_stream_ddl.pl
M src/test/subscription/t/018_stream_subxact_abort.pl
M src/test/subscription/t/019_stream_subxact_ddl_abort.pl
M src/test/subscription/t/020_messages.pl
M src/test/subscription/t/100_bugs.pl
M src/tools/git_changelog
M src/tools/msvc/Install.pm
M src/tools/msvc/MSBuildProject.pm
M src/tools/msvc/Mkvcbuild.pm
M src/tools/msvc/Project.pm
M src/tools/msvc/Solution.pm
M src/tools/msvc/VSObjectFactory.pm
M src/tools/msvc/build.pl
M src/tools/msvc/config_default.pl
M src/tools/msvc/dummylib/Win32.pm
M src/tools/msvc/dummylib/Win32/Registry.pm
M src/tools/msvc/dummylib/Win32API/File.pm
M src/tools/msvc/gendef.pl
M src/tools/msvc/install.pl
M src/tools/msvc/mkvcbuild.pl
M src/tools/msvc/pgbison.pl
M src/tools/msvc/pgflex.pl
M src/tools/msvc/vcregress.pl
M src/tools/pginclude/pgcheckdefines
M src/tools/pgindent/pgindent
Mention statistics objects in maintenance.sgml
commit : 44f90ad092f95fe19bebb51465193bc63849c15f
author : Tomas Vondra <[email protected]>
date : Fri, 7 May 2021 14:02:22 +0200
committer: Tomas Vondra <[email protected]>
date : Fri, 7 May 2021 14:02:22 +0200
The docs mentioned expression indexes as a way to improve selectivity
estimates for functions, but we have a second option to improve that by
creating extended statistics. So mention that too.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210505210947.GA27406%40telsasoft.com
M doc/src/sgml/maintenance.sgml
Fix typos in comments about extended statistics
commit : 93f9af138795a7d12366187de95f4961fb07ed98
author : Tomas Vondra <[email protected]>
date : Fri, 7 May 2021 13:57:29 +0200
committer: Tomas Vondra <[email protected]>
date : Fri, 7 May 2021 13:57:29 +0200
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210505210947.GA27406%40telsasoft.com
M src/backend/parser/parse_utilcmd.c
M src/backend/statistics/extended_stats.c
Make pg_get_statisticsobjdef_expressions return NULL
commit : 8d4b311d2494ca592e30aed03b29854d864eb846
author : Tomas Vondra <[email protected]>
date : Fri, 7 May 2021 13:56:32 +0200
committer: Tomas Vondra <[email protected]>
date : Fri, 7 May 2021 13:56:32 +0200
The usual behavior for functions in ruleutils.c is to return NULL when
the object does not exist. pg_get_statisticsobjdef_expressions raised an
error instead, so correct that.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20210505210947.GA27406%40telsasoft.com
M src/backend/utils/adt/ruleutils.c
Doc: Update notes about libc collation versions.
commit : b65431ca5e12a475ba7cf68afb63edb070c2ce08
author : Thomas Munro <[email protected]>
date : Fri, 7 May 2021 21:47:08 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 7 May 2021 21:47:08 +1200
The per-index collation version tracking feature was reverted, but we
still have the ability to ask Windows (352f6f2d) and FreeBSD
(ca051d8b) for collation versions to store in pg_collation.collversion.
So, from the reverted patch, take a few words of documentation about
libc on all three supported OSes to replace the pre-existing note that
mentioned only glibc.
Discussion: https://postgr.es/m/CA%2BhUKGLhj5t1fcjqAu8iD9B3ixJtsTNqyCCD4V0aTO9kAKAjjA%40mail.gmail.com
M doc/src/sgml/ref/alter_collation.sgml
Revert per-index collation version tracking feature.
commit : ec48314708262d8ea6cdcb83f803fc83dd89e721
author : Thomas Munro <[email protected]>
date : Fri, 7 May 2021 20:17:42 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 7 May 2021 20:17:42 +1200
Design problems were discovered in the handling of composite types and
record types that would cause some relevant versions not to be recorded.
Misgivings were also expressed about the use of the pg_depend catalog
for this purpose. We're out of time for this release so we'll revert
and try again.
Commits reverted:
1bf946bd: Doc: Document known problem with Windows collation versions.
cf002008: Remove no-longer-relevant test case.
ef387bed: Fix bogus collation-version-recording logic.
0fb0a050: Hide internal error for pg_collation_actual_version(<bad OID>).
ff942057: Suppress "warning: variable 'collcollate' set but not used".
d50e3b1f: Fix assertion in collation version lookup.
f24b1569: Rethink extraction of collation dependencies.
257836a7: Track collation versions for indexes.
cd6f479e: Add pg_depend.refobjversion.
7d1297df: Remove pg_collation.collversion.
Discussion: https://postgr.es/m/CA%2BhUKGLhj5t1fcjqAu8iD9B3ixJtsTNqyCCD4V0aTO9kAKAjjA%40mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/charset.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/alter_collation.sgml
M doc/src/sgml/ref/alter_index.sgml
M doc/src/sgml/ref/create_collation.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/reindex.sgml
M src/backend/catalog/dependency.c
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/pg_collation.c
M src/backend/catalog/pg_constraint.c
M src/backend/catalog/pg_depend.c
M src/backend/catalog/pg_type.c
M src/backend/commands/collationcmds.c
M src/backend/commands/statscmds.c
M src/backend/commands/tablecmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/optimizer/util/plancat.c
M src/backend/parser/gram.y
M src/backend/tcop/utility.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_upgrade_support.c
M src/backend/utils/cache/relcache.c
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_upgrade/dump.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/psql/tab-complete.c
M src/include/catalog/catversion.h
M src/include/catalog/dependency.h
M src/include/catalog/index.h
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_type.h
M src/include/commands/collationcmds.h
M src/include/nodes/parsenodes.h
M src/include/utils/pg_locale.h
M src/include/utils/rel.h
M src/test/Makefile
M src/test/locale/.gitignore
M src/test/locale/Makefile
D src/test/locale/t/001_index.pl
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/expected/collate.linux.utf8.out
M src/test/regress/expected/create_index.out
M src/test/regress/expected/misc_sanity.out
M src/test/regress/sql/collate.icu.utf8.sql
M src/test/regress/sql/collate.linux.utf8.sql
M src/test/regress/sql/create_index.sql
M src/tools/pgindent/typedefs.list
Remove redundant variable
commit : a288d94c91e345ebeb10ac30f247270c8c8e380a
author : Alvaro Herrera <[email protected]>
date : Thu, 6 May 2021 17:28:36 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 6 May 2021 17:28:36 -0400
Author: Amul Sul <[email protected]>
Reviewed-by: Jeevan Ladhe <[email protected]>
Reviewed-by: Bharath Rupireddy <[email protected]>
Reviewed-by: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/CAAJ_b94HaNcrPVREUuB9-qUn2uB+gfcoX3FG_Vx0S6aFse+yhw@mail.gmail.com
M src/backend/parser/parse_utilcmd.c
Document lock level used by ALTER TABLE VALIDATE CONSTRAINT
commit : 469116389e18dbf6be0bd555bc2055a26be91a48
author : Alvaro Herrera <[email protected]>
date : Thu, 6 May 2021 17:17:57 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 6 May 2021 17:17:57 -0400
Backpatch all the way back to 9.6.
Author: Simon Riggs <[email protected]>
Discussion: https://postgr.es/m/CANbhV-EwxvdhHuOLdfG2ciYrHOHXV=mm6=fD5aMhqcH09Li3Tg@mail.gmail.com
M doc/src/sgml/ref/alter_table.sgml
Improve documentation on DETACH PARTITION lock levels
commit : db6e1aeb952e9aed26ba2a56b4145293c72b8068
author : Alvaro Herrera <[email protected]>
date : Thu, 6 May 2021 16:42:30 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 6 May 2021 16:42:30 -0400
This was forgotten in 71f4c8c6f74b.
Reported-by: Pavel Luzanov <[email protected]>
Author: Amit Langote <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/alter_table.sgml
Remove overzealous VACUUM visibility map assertion.
commit : c9787385db47ba423d845b34d58e158551c6335d
author : Peter Geoghegan <[email protected]>
date : Thu, 6 May 2021 13:17:39 -0700
committer: Peter Geoghegan <[email protected]>
date : Thu, 6 May 2021 13:17:39 -0700
The all_visible_according_to_vm variable's value is inherently prone to
becoming invalidated concurrently, since it is set before we even
acquire a lock on a related heap page buffer.
Oversight in commit 7136bf34, which added the assertion in passing.
Author: Masahiko Sawada <[email protected]>
Reported-By: Tang <[email protected]>
Diagnosed-By:: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/CAD21AoDzgc8_MYrA5m1fyydomw_eVKtQiYh7sfDK4KEhdMsf_g@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Track detached partitions more accurately in partdescs
commit : 3fe773b149755977d2ffde2afd89557b39d0afd9
author : Alvaro Herrera <[email protected]>
date : Thu, 6 May 2021 12:47:30 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 6 May 2021 12:47:30 -0400
In d6b8d29419df I (Álvaro) was sloppy about recording whether a
partition descripor does or does not include detached partitions, when
the snapshot checking does not see the pg_inherits row marked detached.
In that case no partition was omitted, yet in the relcache entry we were
saving the partdesc as omitting partitions. Flip that (so we save it as
a partdesc not omitting partitions, which indeed it doesn't), which
hopefully makes the code easier to reason about.
Author: Amit Langote <[email protected]>
Discussion: https://postgr.es/m/CA+HiwqE7GxGU4VdzwZzfiz+Ont5SsopoFkgtrZGEdPqWRL+biA@mail.gmail.com
M src/backend/partitioning/partdesc.c
M src/include/utils/rel.h
Doc: trivial wording adjustment.
commit : c38cadc0907a8d071b043b2b32b83efa09db38ea
author : Tom Lane <[email protected]>
date : Thu, 6 May 2021 09:59:11 -0400
committer: Tom Lane <[email protected]>
date : Thu, 6 May 2021 09:59:11 -0400
Improve self-referential foreign key example, per suggestion
from David Johnston.
Discussion: https://postgr.es/m/CAKFQuwZTke7+HUn4YUGqu2+gAPi4Cy18TXMrg_Z5nADkxfPNMw@mail.gmail.com
M doc/src/sgml/ddl.sgml
Additional doc fixes for configurable TOAST compression.
commit : 448b02c00515ba9d6683a8a97fe4305604d80028
author : Robert Haas <[email protected]>
date : Thu, 6 May 2021 08:27:20 -0400
committer: Robert Haas <[email protected]>
date : Thu, 6 May 2021 08:27:20 -0400
The grammar changes in commit bbe0a81db69bd10bd166907c3701492a29aca294
allow SET COMPRESSION to be used with ALTER MATERIALIZED VIEW as
well as with ALTER TABLE, so update those docs to say that it works.
Also, update the documentation for the pg_column_compression()
to explain that it will return NULL when there's no relevant value.
Patch by me, per concerns from Michael Paquier.
Discussion: http://postgr.es/m/CA+Tgmob9h5u4iNL9KM0drZgkY-JL4oCVW0dWrMqtLPQ1zHkquA@mail.gmail.com
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/alter_materialized_view.sgml
docs: Clarify how ALTER TABLE .. SET COMPRESSION works.
commit : 2d0f662402635d591cac9f1daae5e81e7c4374fc
author : Robert Haas <[email protected]>
date : Thu, 6 May 2021 08:22:45 -0400
committer: Robert Haas <[email protected]>
date : Thu, 6 May 2021 08:22:45 -0400
Justin Pryzby, per a complaint from Michael Paquier. Reviewed by
Dilip Kumar and by me.
Discussion: http://postgr.es/m/[email protected]
M doc/src/sgml/ref/alter_table.sgml
Update replication statistics after every stream/spill.
commit : 592f00f8dec68038301467a904ac514eddabf6cd
author : Amit Kapila <[email protected]>
date : Thu, 6 May 2021 11:21:26 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 6 May 2021 11:21:26 +0530
Currently, replication slot statistics are updated at prepare, commit, and
rollback. Now, if the transaction is interrupted the stats might not get
updated. Fixed this by updating replication statistics after every
stream/spill.
In passing update the docs to change the description of some of the slot
stats.
Author: Vignesh C, Sawada Masahiko
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h
jit: Fix warning reported by gcc-11 caused by dubious function signature.
commit : 7f2e10baa2482494dbcf70e0ae6f0469771e0b4c
author : Andres Freund <[email protected]>
date : Wed, 5 May 2021 22:07:40 -0700
committer: Andres Freund <[email protected]>
date : Wed, 5 May 2021 22:07:40 -0700
Reported-By: Erik Rijkers <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch: 13, where b059d2f45685 introduced the issue.
M src/backend/jit/llvm/llvmjit_expr.c
Doc: update RELEASE_CHANGES checklist.
commit : e8ce68b0b9ae2757c6153a88bf869904d2d5ac0b
author : Tom Lane <[email protected]>
date : Wed, 5 May 2021 23:10:33 -0400
committer: Tom Lane <[email protected]>
date : Wed, 5 May 2021 23:10:33 -0400
Update checklist to reflect current practice:
* The platform-specific FAQ files are long gone.
* We've never routinely updated the libbind code we borrowed, either,
and there seems no reason to start now.
* Explain current practice of running pgindent twice per cycle.
Discussion: https://postgr.es/m/[email protected]
M src/tools/RELEASE_CHANGES
Tighten the concurrent abort check during decoding.
commit : 2ce353fc19024d62e59ad99850d7592ebc9abecf
author : Amit Kapila <[email protected]>
date : Thu, 6 May 2021 08:26:42 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 6 May 2021 08:26:42 +0530
During decoding of an in-progress or prepared transaction, we detect
concurrent abort with an error code ERRCODE_TRANSACTION_ROLLBACK. That is
not sufficient because a callback can decide to throw that error code
at other times as well.
Reported-by: Tom Lane
Author: Amit Kapila
Reviewed-by: Dilip Kumar
Discussion: https://postgr.es/m/CAA4eK1KCjPRS4aZHB48QMM4J8XOC1+TD8jo-4Yu84E+MjwqVhA@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
Remove unused argument of ATAddForeignConstraint
commit : c250062df42ffd3e252471f6205bfb6cbef67b7b
author : Alvaro Herrera <[email protected]>
date : Wed, 5 May 2021 12:27:39 -0400
committer: Alvaro Herrera <[email protected]>
date : Wed, 5 May 2021 12:27:39 -0400
Commit 0325d7a5957b made this unused but forgot to remove it. Do so now.
Author: Amit Langote <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
Have ALTER CONSTRAINT recurse on partitioned tables
commit : 6f70d7ca1d1937a9f7b79eff6fb18ed1bb2a4c47
author : Alvaro Herrera <[email protected]>
date : Wed, 5 May 2021 12:14:21 -0400
committer: Alvaro Herrera <[email protected]>
date : Wed, 5 May 2021 12:14:21 -0400
When ALTER TABLE .. ALTER CONSTRAINT changes deferrability properties
changed in a partitioned table, we failed to propagate those changes
correctly to partitions and to triggers. Repair by adding a recursion
mechanism to affect all derived constraints and all derived triggers.
(In particular, recurse to partitions even if their respective parents
are already in the desired state: it is possible for the partitions to
have been altered individually.) Because foreign keys involve tables in
two sides, we cannot use the standard ALTER TABLE recursion mechanism,
so we invent our own by following pg_constraint.conparentid down.
When ALTER TABLE .. ALTER CONSTRAINT is invoked on the derived
pg_constraint object that's automaticaly created in a partition as a
result of a constraint added to its parent, raise an error instead of
pretending to work and then failing to modify all the affected triggers.
Before this commit such a command would be allowed but failed to affect
all triggers, so it would silently misbehave. (Restoring dumps of
existing databases is not affected, because pg_dump does not produce
anything for such a derived constraint anyway.)
Add some tests for the case.
Backpatch to 11, where foreign key support was added to partitioned
tables by commit 3de241dba86f. (A related change is commit f56f8f8da6af
in pg12 which added support for FKs *referencing* partitioned tables;
this is what forces us to use an ad-hoc recursion mechanism for this.)
Diagnosed by Tom Lane from bug report from Ron L Johnson. As of this
writing, no reviews were offered.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
Doc: improve and centralize the documentation for OID alias types.
commit : f33a178a34809a2bae7a5f4c00984d87771f4204
author : Tom Lane <[email protected]>
date : Wed, 5 May 2021 11:26:48 -0400
committer: Tom Lane <[email protected]>
date : Wed, 5 May 2021 11:26:48 -0400
Previously, a lot of information about type regclass existed only
in the discussion of the sequence functions. Maybe that made sense
in the beginning, because I think originally those were the only
functions taking regclass. But it doesn't make sense anymore.
Move that material to the "Object Identifier Types" section in
datatype.sgml, generalize it to talk about the other reg* types
as well, and add more examples.
Per bug #16991 from Federico Caselli.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/datatype.sgml
M doc/src/sgml/func.sgml
GUC description improvements for clarity
commit : 38f36aad8c55c8f91e3fb8720fae1847c8fa0552
author : Peter Eisentraut <[email protected]>
date : Wed, 5 May 2021 08:18:22 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 5 May 2021 08:18:22 +0200
M src/backend/utils/misc/guc.c
Disable cache clobber to avoid breaking postgres_fdw termination test.
commit : 1273a15bf91fa322915e32d3b6dc6ec916397268
author : Tom Lane <[email protected]>
date : Tue, 4 May 2021 13:36:26 -0400
committer: Tom Lane <[email protected]>
date : Tue, 4 May 2021 13:36:26 -0400
Commit 93f414614 improved a pre-existing test case so that it would
show whether or not termination of the "remote" worker process happened.
This soon exposed that, when debug_invalidate_system_caches_always
(nee CLOBBER_CACHE_ALWAYS) is enabled, no such termination occurs.
That's because cache invalidation forces postgres_fdw connections
to be dropped at end of transaction, so that there's no worker to
terminate. There's a race condition as to whether the worker will
manage to get out of the BackendStatusArray before we look, but at
least on buildfarm member hyrax, it's failed twice in two attempts.
Rather than re-lobotomizing the test, let's fix this by transiently
disabling debug_invalidate_system_caches_always. (Hooray for that
being just a GUC nowadays, rather than a compile-time option.)
If this proves not to be enough to make the test stable, we can
do the other thing instead.
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Fix OID passed to object-alter hook during ALTER CONSTRAINT
commit : e798d095da3a4a4bb5c50bb3dff886f07ef52f55
author : Alvaro Herrera <[email protected]>
date : Tue, 4 May 2021 10:09:12 -0400
committer: Alvaro Herrera <[email protected]>
date : Tue, 4 May 2021 10:09:12 -0400
The OID of the constraint is used instead of the OID of the trigger --
an easy mistake to make. Apparently the object-alter hooks are not very
well tested :-(
Backpatch to 12, where this typo was introduced by 578b229718e8
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
doc: Fix typos
commit : c98a6d7887ea6588b4e9797903182312a2b46f67
author : Peter Eisentraut <[email protected]>
date : Tue, 4 May 2021 15:45:13 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 4 May 2021 15:45:13 +0200
M doc/src/sgml/ecpg.sgml
pg_dump: Fix dump of generated columns in partitions
commit : feb270d1005f3d7b3705dec9e04c9a205750ea97
author : Peter Eisentraut <[email protected]>
date : Tue, 4 May 2021 14:03:54 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 4 May 2021 14:03:54 +0200
The previous fix for dumping of inherited generated columns
(0bf83648a52df96f7c8677edbbdf141bfa0cf32b) must not be applied to
partitions, since, unlike normal inherited tables, they are always
dumped separately and reattached.
Reported-by: Santosh Udupi <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CACLRvHZ4a-%2BSM_159%2BtcrHdEqxFrG%3DW4gwTRnwf7Oj0UNj5R2A%40mail.gmail.com
M src/bin/pg_dump/common.c
Fix ALTER TABLE / INHERIT with generated columns
commit : a970edbed306354b0079bdcdc2fc74312122ad89
author : Peter Eisentraut <[email protected]>
date : Tue, 4 May 2021 11:45:37 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 4 May 2021 11:45:37 +0200
When running ALTER TABLE t2 INHERIT t1, we must check that columns in
t2 that correspond to a generated column in t1 are also generated and
have the same generation expression. Otherwise, this would allow
creating setups that a normal CREATE TABLE sequence would not allow.
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/backend/commands/tablecmds.c
M src/test/regress/expected/generated.out
M src/test/regress/sql/generated.sql
Remove mention of the version number from pg_trgm docs
commit : ae9492a61bbf575e2862cf9323c7f02806382093
author : Alexander Korotkov <[email protected]>
date : Tue, 4 May 2021 03:56:16 +0300
committer: Alexander Korotkov <[email protected]>
date : Tue, 4 May 2021 03:56:16 +0300
We don't usually mention the version number in similar situations. So, neither
mention it here.
Reported-by: Bruce Momjian
Discussion: https://postgr.es/m/20210503234914.GO6180%40momjian.us
M doc/src/sgml/pgtrgm.sgml
Update query_id computation
commit : f7a97b6ec31f3f57a6154d0039c4de81ad517064
author : Bruce Momjian <[email protected]>
date : Mon, 3 May 2021 14:59:30 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 3 May 2021 14:59:30 -0400
Properly fix:
- the "ONLY" in FROM [ONLY] isn't hashed
- the agglevelsup field in GROUPING isn't hashed
- WITH TIES not being hashed (new in PG 13)
- "DISTINCT" in "GROUP BY [DISTINCT]" isn't hashed (new in PG 14)
Reported-by: Julien Rouhaud
Discussion: https://postgr.es/m/20210425081119.ulyzxqz23ueh3wuj@nol
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
M src/backend/utils/misc/queryjumble.c
doc: Add index entry for "multirange type"
commit : 5df6aeab42279eaea8e9ff92744645b155c85b03
author : Peter Eisentraut <[email protected]>
date : Mon, 3 May 2021 20:14:03 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 3 May 2021 20:14:03 +0200
Before now, looking up "multirange" in the index only led to the
multirange() function. To make this more useful, also add an entry
pointing to the range types section.
M doc/src/sgml/func.sgml
M doc/src/sgml/rangetypes.sgml
amcheck: Improve some confusing reports about TOAST problems.
commit : 50529e5b4e39ad80a637ba0905277f9691eb4a15
author : Robert Haas <[email protected]>
date : Mon, 3 May 2021 12:32:05 -0400
committer: Robert Haas <[email protected]>
date : Mon, 3 May 2021 12:32:05 -0400
Don't phrase reports in terms of the number of tuples thus-far
returned by the index scan, but rather in terms of the chunk_seq
values found inside the tuples.
Patch by me, reviewed by Mark Dilger.
Discussion: http://postgr.es/m/CA+TgmoZUONCkdcdR778EKuE+f1r5Obieu63db2OgMPHaEvEPTQ@mail.gmail.com
M contrib/amcheck/verify_heapam.c
Fix performance issue in new regex match-all detection code.
commit : f68970e33f4dc48094c24c78c452ad730ae9ae12
author : Tom Lane <[email protected]>
date : Mon, 3 May 2021 11:42:31 -0400
committer: Tom Lane <[email protected]>
date : Mon, 3 May 2021 11:42:31 -0400
Commit 824bf7190 introduced a new search of the NFAs generated by
regex compilation. I failed to think hard about the performance
characteristics of that search, with the predictable outcome
that it's bad: weird regexes can trigger exponential search time.
Worse, there's no check-for-interrupt in that code, so you can't
even cancel the query if this happens.
Fix by introducing memo-ization of the search results, so that any one
NFA state need be examined in detail just once. This potentially uses
a lot of memory, but we can bound the memory usage by putting a limit
on the number of states for which we'll try to prove match-all-ness.
That is sane because we already have a limit (DUPINF) on the maximum
finite string length that a matchall regex can match; and patterns
that involve much more than DUPINF states would probably exceed that
limit anyway.
Also, rearrange the logic so that we check the basic is-the-graph-
all-RAINBOW-arcs property before we start the recursive search to
determine path lengths. This will ensure that we fall out quickly
whenever the NFA couldn't possibly be matchall.
Also stick in a check-for-interrupt, just in case these measures
don't completely eliminate the risk of slowness.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/test/regress/expected/regex.out
M src/test/regress/sql/regex.sql
Prevent lwlock dtrace probes from unnecessary work
commit : b94409a02f6122d77b5154e481c0819fed6b4c95
author : Peter Eisentraut <[email protected]>
date : Mon, 3 May 2021 12:11:33 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 3 May 2021 12:11:33 +0200
If dtrace is compiled in but disabled, the lwlock dtrace probes still
evaluate their arguments. Since PostgreSQL 13, T_NAME(lock) does
nontrivial work, so it should be avoided if not needed. To fix, make
these calls conditional on the *_ENABLED() macro corresponding to each
probe.
Reviewed-by: Craig Ringer <[email protected]>
Discussion: https://www.postgresql.org/message-id/CAGRY4nwxKUS_RvXFW-ugrZBYxPFFM5kjwKT5O+0+Stuga5b4+Q@mail.gmail.com
M src/backend/storage/lmgr/lwlock.c
Remove unused function argument
commit : c285babf8f44d86b7fd1e73e9e4f94456b825bfb
author : Peter Eisentraut <[email protected]>
date : Mon, 3 May 2021 09:05:58 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 3 May 2021 09:05:58 +0200
became unused by 04942bffd0aa9bd0d143d99b473342eb9ecee88b
M src/backend/rewrite/rewriteHandler.c
libpq: Refactor some error messages for easier translation
commit : ced12b73a9bc76b887cb7137df3d56222e2b5263
author : Peter Eisentraut <[email protected]>
date : Mon, 3 May 2021 08:51:30 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 3 May 2021 08:51:30 +0200
M src/interfaces/libpq/fe-connect.c
Factor out system call names from error messages
commit : 853c8c75571558f4b474eeac3ef9e6fcf9be62ba
author : Peter Eisentraut <[email protected]>
date : Mon, 3 May 2021 07:27:31 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 3 May 2021 07:27:31 +0200
One more that ought to have been part of
82c3cd974131d7fa1cfcd07cebfb04fffe26ee35.
M src/interfaces/libpq/fe-connect.c
Fix the computation of slot stats for 'total_bytes'.
commit : 205f466282be11ec97506f73341e47b72e0aee5d
author : Amit Kapila <[email protected]>
date : Mon, 3 May 2021 07:22:08 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 3 May 2021 07:22:08 +0530
Previously, we were using the size of all the changes present in
ReorderBuffer to compute total_bytes after decoding a transaction and that
can lead to counting some of the transactions' changes more than once. Fix
it by using the size of the changes decoded for a transaction to compute
'total_bytes'.
Author: Sawada Masahiko
Reviewed-by: Vignesh C, Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/reorderbuffer.c
Make websearch_to_tsquery() parse text in quotes as a single token
commit : eb086056fec44516efdd5db71244a079fed65c7f
author : Alexander Korotkov <[email protected]>
date : Mon, 3 May 2021 03:58:03 +0300
committer: Alexander Korotkov <[email protected]>
date : Mon, 3 May 2021 03:58:03 +0300
websearch_to_tsquery() splits text in quotes into tokens and connects them with
phrase operator on its own. However, that leads to surprising results when the
token contains no words.
For instance, websearch_to_tsquery('"aaa: bbb"') is 'aaa <2> bbb', because
it is equivalent of to_tsquery(E'aaa <-> \':\' <-> bbb'). But
websearch_to_tsquery('"aaa: bbb"') has to be 'aaa <-> bbb' in order to match
to_tsvector('aaa: bbb').
Since 0c4f355c6a, we anyway connect lexemes of complex tokens with phrase
operators. Thus, let's just websearch_to_tsquery() parse text in quotes as
a single token. Therefore, websearch_to_tsquery() should process the quoted
text in the same way phraseto_tsquery() does. This solution is what we exactly
need and also simplifies the code.
This commit is an incompatible change, so we don't backpatch it.
Reported-by: Valentin Gatien-Baron
Discussion: https://postgr.es/m/CA%2B0DEqiZs7gdOd4ikmg%3D0UWG%2BSwWOLxPsk_JW-sx9WNOyrb0KQ%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Tom Lane, Zhihong Yu
M src/backend/utils/adt/tsquery.c
M src/test/regress/expected/tsearch.out
M src/test/regress/sql/tsearch.sql
Revert use singular for -1 (commits 9ee7d533da and 5da9868ed9
commit : 651d005e76bc0b9542615f609b4d0d946035dc58
author : Bruce Momjian <[email protected]>
date : Sat, 1 May 2021 10:42:44 -0400
committer: Bruce Momjian <[email protected]>
date : Sat, 1 May 2021 10:42:44 -0400
Turns out you can specify negative values using plurals:
https://english.stackexchange.com/questions/9735/is-1-followed-by-a-singular-or-plural-noun
so the previous code was correct enough, and consistent with other usage
in our code. Also add comment in the two places where this could be
confused.
Reported-by: Noah Misch
Diagnosed-by: [email protected]
M contrib/dblink/expected/dblink.out
M src/backend/utils/adt/datetime.c
M src/interfaces/ecpg/pgtypeslib/interval.c
M src/interfaces/libpq/fe-print.c
M src/test/regress/expected/interval.out
Doc: add an example of a self-referential foreign key to ddl.sgml.
commit : e6f9539dc32473793c03cbe95bc099ee0a199c73
author : Tom Lane <[email protected]>
date : Fri, 30 Apr 2021 15:37:56 -0400
committer: Tom Lane <[email protected]>
date : Fri, 30 Apr 2021 15:37:56 -0400
While we've always allowed such cases, the documentation didn't
say you could do it.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ddl.sgml
Doc: update libpq's documentation for PQfn().
commit : 386e64ea5abf346d887c21ea8869317838ba19b5
author : Tom Lane <[email protected]>
date : Fri, 30 Apr 2021 15:10:06 -0400
committer: Tom Lane <[email protected]>
date : Fri, 30 Apr 2021 15:10:06 -0400
Mention specifically that you can't call aggregates, window functions,
or procedures this way (the inability to call SRFs was already
mentioned).
Also, the claim that PQfn doesn't support NULL arguments or results
has been a lie since we invented protocol 3.0. Not sure why this
text was never updated for that, but do it now.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
Disallow calling anything but plain functions via the fastpath API.
commit : 2efcd502e56a528f75ec8e88c02a287ad3457d77
author : Tom Lane <[email protected]>
date : Fri, 30 Apr 2021 14:10:26 -0400
committer: Tom Lane <[email protected]>
date : Fri, 30 Apr 2021 14:10:26 -0400
Reject aggregates, window functions, and procedures. Aggregates
failed anyway, though with a somewhat obscure error message.
Window functions would hit an Assert or null-pointer dereference.
Procedures seemed to work as long as you didn't try to do
transaction control, but (a) transaction control is sort of the
point of a procedure, and (b) it's not entirely clear that no
bugs lurk in that path. Given the lack of testing of this area,
it seems safest to be conservative in what we support.
Also reject proretset functions, as the fastpath protocol can't
support returning a set.
Also remove an easily-triggered assertion that the given OID
isn't 0; the subsequent lookups can handle that case themselves.
Per report from Theodor-Arsenij Larionov-Trichkin.
Back-patch to all supported branches. (The procedure angle
only applies in v11+, of course.)
Discussion: https://postgr.es/m/[email protected]
M src/backend/tcop/fastpath.c
Fix the bugs in selecting the transaction for streaming.
commit : ee4ba01dbbc31daa083f434ecd603a80bbe50501
author : Amit Kapila <[email protected]>
date : Fri, 30 Apr 2021 10:49:52 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 30 Apr 2021 10:49:52 +0530
There were two problems:
a. We were always selecting the next available txn instead of selecting it
when it is larger than the previous transaction.
b. We were selecting the transactions which haven't made any changes to
the database (base snapshot is not set). Later it was hitting an Assert
because we don't decode such transactions and the changes in txn remain as
it is. It is better not to choose such transactions for streaming in the
first place.
Reported-by: Haiying Tang
Author: Dilip Kumar
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/OS0PR01MB61133B94E63177040F7ECDA1FB429@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/backend/replication/logical/reorderbuffer.c
Adjust EXPLAIN output for parallel Result Cache plans
commit : 3c80e96dffd4df7f66fffa5f265cbd87becb7ef5
author : David Rowley <[email protected]>
date : Fri, 30 Apr 2021 14:46:42 +1200
committer: David Rowley <[email protected]>
date : Fri, 30 Apr 2021 14:46:42 +1200
Here we adjust the EXPLAIN ANALYZE output for Result Cache so that we
don't show any Result Cache stats for parallel workers who don't
contribute anything to Result Cache plan nodes.
I originally had ideas that workers who don't help could still have their
Result Cache stats displayed. The idea with that was so that I could
write some parallel Result Cache regression tests that show the EXPLAIN
ANALYZE output. However, I realized a little too late that such tests
would just not be possible to have run in a stable way on the buildfarm.
With that knowledge, before 9eacee2e6 went in, I had removed all of the
tests that were showing the EXPLAIN ANALYZE output of a parallel Result
Cache plan, however, I forgot to put back the code that adjusts the
EXPLAIN output to hide the Result Cache stats for parallel workers who
were not fast enough to help out before query execution was over. All
other nodes behave this way and so should Result Cache.
Additionally, with this change, it now seems safe enough to remove the SET
force_parallel_mode = off that I had added to the regression tests.
Also, perform some cleanup in the partition_prune tests. I had adjusted
the explain_parallel_append() function to sanitize the Result Cache
EXPLAIN ANALYZE output. However, since I didn't actually include any
parallel Result Cache tests that show their EXPLAIN ANALYZE output, that
code does nothing and can be removed.
In passing, move the setting of memPeakKb into the scope where it's used.
Reported-by: Amit Khandekar
Author: David Rowley, Amit Khandekar
Discussion: https://postgr.es/m/CAJ3gD9d8SkfY95GpM1zmsOtX2-Ogx5q-WLsf8f0ykEb0hCRK3w@mail.gmail.com
M src/backend/commands/explain.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/resultcache.out
M src/test/regress/sql/partition_prune.sql
M src/test/regress/sql/resultcache.sql
Another try to fix the test case added by commit f5fc2f5b23.
commit : 51ef9173030cc196c6633ae82b7b32f404b0f768
author : Amit Kapila <[email protected]>
date : Fri, 30 Apr 2021 07:55:42 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 30 Apr 2021 07:55:42 +0530
As per analysis, it appears that the 'drop slot' message from the previous
test and 'create slot' message of the new test are either missed or not
yet delivered to the stats collector due to which we will still see the
stats from the old slot. This can happen rarely which could be the reason
that we are seeing some failures in the buildfarm randomly. To avoid that
we are using a different slot name for the tests in
test_decoding/sql/stats.sql.
Reported-by: Tom Lane based on buildfarm reports
Author: Sawada Masahiko
Reviewed-by: Amit Kapila, Vignesh C
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
Improve wording of some pg_upgrade failure reports.
commit : c9c37ae03fea0c8ad467392ddf03940b61974935
author : Tom Lane <[email protected]>
date : Thu, 29 Apr 2021 15:40:34 -0400
committer: Tom Lane <[email protected]>
date : Thu, 29 Apr 2021 15:40:34 -0400
Don't advocate dropping a whole table when dropping a column would
serve. While at it, try to make the layout of these messages a
bit cleaner and more consistent.
Per suggestion from Daniel Gustafsson. No back-patch, as we generally
don't like to churn translatable messages in released branches.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/version.c
Fix some more omissions in pg_upgrade's tests for non-upgradable types.
commit : 57c081de0afcd01bf47c46f015bf8886b01c2c21
author : Tom Lane <[email protected]>
date : Thu, 29 Apr 2021 15:24:37 -0400
committer: Tom Lane <[email protected]>
date : Thu, 29 Apr 2021 15:24:37 -0400
Commits 29aeda6e4 et al closed up some oversights involving not checking
for non-upgradable types within container types, such as arrays and
ranges. However, I only looked at version.c, failing to notice that
there were substantially-equivalent tests in check.c. (The division
of responsibility between those files is less than clear...)
In addition, because genbki.pl does not guarantee that auto-generated
rowtype OIDs will hold still across versions, we need to consider that
the composite type associated with a system catalog or view is
non-upgradable. It seems unlikely that someone would have a user
column declared that way, but if they did, trying to read it in another
PG version would likely draw "no such pg_type OID" failures, thanks
to the type OID embedded in composite Datums.
To support the composite and reg*-type cases, extend the recursive
query that does the search to allow any base query that returns
a column of pg_type OIDs, rather than limiting it to exactly one
starting type.
As before, back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/version.c
Improve documentation for default_tablespace on partitioned tables
commit : 94b9cb722552c37da78c8320bac1d5b55e34def6
author : Alvaro Herrera <[email protected]>
date : Thu, 29 Apr 2021 11:31:24 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 29 Apr 2021 11:31:24 -0400
Backpatch to 12, where 87259588d0ab introduced the current behavior.
Per note from Justin Pryzby.
Co-authored-by: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
psql: Fix line continuation prompts for unbalanced parentheses
commit : d9a9f4b4b92ad39e3c4e6600dc61d5603ddd6e24
author : Peter Eisentraut <[email protected]>
date : Thu, 29 Apr 2021 09:04:31 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 29 Apr 2021 09:04:31 +0200
This was broken by a silly mistake in
e717a9a18b2e34c9c40e5259ad4d31cd7e420750.
Reported-by: Jeff Janes <[email protected]>
Author: Justin Pryzby <[email protected]>
Discussion: https://www.postgresql.org/message-id/CAMkU=1zKGWEJdBbYKw7Tn7cJmYR_UjgdcXTPDqJj=dNwCETBCQ@mail.gmail.com
M src/fe_utils/psqlscan.l
pg_hba.conf.sample: Reword connection type section
commit : 3a948ea0a2ced719f26e725b030558f2e4ab1d8e
author : Peter Eisentraut <[email protected]>
date : Thu, 29 Apr 2021 06:51:42 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 29 Apr 2021 06:51:42 +0200
Improve the wording in the connection type section of
pg_hba.conf.sample a bit. After the hostgssenc part was added on, the
whole thing became a bit wordy, and it's also a bit inaccurate for
example in that the current wording for "host" appears to say that it
does not apply to GSS-encrypted connections.
Discussion: https://www.postgresql.org/message-id/flat/fc06dcc5-513f-e944-cd07-ba51dd7c6916%40enterprisedb.com
M src/backend/libpq/pg_hba.conf.sample
doc: Fix description of target_session_attrs=prefer-standby
commit : 2977f244bc7e64e046364122be3fef08aa6efef9
author : Michael Paquier <[email protected]>
date : Thu, 29 Apr 2021 11:44:24 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 29 Apr 2021 11:44:24 +0900
If no standbys can be found in the set of connection points provided,
the fallback behavior is "any", and not "all" that does not exist.
Author: Greg Nancarrow
Reviewed-by: Laurenz Albe
Discussion: https://postgr.es/m/CAJcOf-fDaCv8qCpWH7k5Yh6zFxZeUwZowu4sCWQ2zFx4CdkHpA@mail.gmail.com
M doc/src/sgml/libpq.sgml
Add heuristic incoming-message-size limits in the server.
commit : 9626325da5e8e23ff90091bc96535495d350f06e
author : Tom Lane <[email protected]>
date : Wed, 28 Apr 2021 15:50:42 -0400
committer: Tom Lane <[email protected]>
date : Wed, 28 Apr 2021 15:50:42 -0400
We had a report of confusing server behavior caused by a client bug
that sent junk to the server: the server thought the junk was a
very long message length and waited patiently for data that would
never come. We can reduce the risk of that by being less trusting
about message lengths.
For a long time, libpq has had a heuristic rule that it wouldn't
believe large message size words, except for a small number of
message types that are expected to be (potentially) long. This
provides some defense against loss of message-boundary sync and
other corrupted-data cases. The server does something similar,
except that up to now it only limited the lengths of messages
received during the connection authentication phase. Let's
do the same as in libpq and put restrictions on the allowed
length of all messages, while distinguishing between message
types that are expected to be long and those that aren't.
I used a limit of 10000 bytes for non-long messages. (libpq's
corresponding limit is 30000 bytes, but given the asymmetry of
the FE/BE protocol, there's no good reason why the numbers should
be the same.) Experimentation suggests that this is at least a
factor of 10, maybe a factor of 100, more than we really need;
but plenty of daylight seems desirable to avoid false positives.
In any case we can adjust the limit based on beta-test results.
For long messages, set a limit of MaxAllocSize - 1, which is the
most that we can absorb into the StringInfo buffer that the message
is collected in. This just serves to make sure that a bogus message
size is reported as such, rather than as a confusing gripe about
not being able to enlarge a string buffer.
While at it, make sure that non-mainline code paths (such as
COPY FROM STDIN) are as paranoid as SocketBackend is, and validate
the message type code before believing the message length.
This provides an additional guard against getting stuck on corrupted
input.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/copyfromparse.c
M src/backend/libpq/auth.c
M src/backend/libpq/pqcomm.c
M src/backend/replication/walsender.c
M src/backend/tcop/postgres.c
M src/include/libpq/libpq.h
Allow a partdesc-omitting-partitions to be cached
commit : d6b8d29419df0efad57f95c80b871745d1b55da6
author : Alvaro Herrera <[email protected]>
date : Wed, 28 Apr 2021 15:44:35 -0400
committer: Alvaro Herrera <[email protected]>
date : Wed, 28 Apr 2021 15:44:35 -0400
Makes partition descriptor acquisition faster during the transient
period in which a partition is in the process of being detached.
This also adds the restriction that only one partition can be in
pending-detach state for a partitioned table.
While at it, return find_inheritance_children() API to what it was
before 71f4c8c6f74b, and create a separate
find_inheritance_children_extended() that returns detailed info about
detached partitions.
(This incidentally fixes a bug in 8aba9322511 whereby a memory context
holding a transient partdesc is reparented to a NULL PortalContext,
leading to permanent leak of that memory. The fix is to no longer rely
on reparenting contexts to PortalContext. Reported by Amit Langote.)
Per gripe from Amit Langote
Discussion: https://postgr.es/m/CA+HiwqFgpP1LxJZOBYGt9rpvTjXXkg5qG2+Xch2Z1Q7KrqZR1A@mail.gmail.com
M doc/src/sgml/ref/alter_table.sgml
M src/backend/catalog/pg_inherits.c
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/partitioning/partdesc.c
M src/backend/utils/cache/relcache.c
M src/include/catalog/pg_inherits.h
M src/include/utils/rel.h
M src/test/isolation/expected/detach-partition-concurrently-3.out
M src/test/isolation/specs/detach-partition-concurrently-3.spec
Doc: fix discussion of how to get real Julian Dates.
commit : c93f8f3b8d3bc780892e2bf11192fbdd136fddfe
author : Tom Lane <[email protected]>
date : Wed, 28 Apr 2021 10:03:28 -0400
committer: Tom Lane <[email protected]>
date : Wed, 28 Apr 2021 10:03:28 -0400
Somehow I'd convinced myself that rotating to UTC-12 was the way
to do this, but upon further review, it's definitely UTC+12.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/datetime.sgml
Fix use-after-release issue with pg_identify_object_as_address()
commit : f93f0b5b25068807051edb2f3510614b69bb79ff
author : Michael Paquier <[email protected]>
date : Wed, 28 Apr 2021 11:58:08 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 28 Apr 2021 11:58:08 +0900
Spotted by buildfarm member prion, with -DRELCACHE_FORCE_RELEASE.
Introduced in f7aab36.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/backend/catalog/objectaddress.c
Fix pg_identify_object_as_address() with event triggers
commit : f7aab36d61fd2fdbd949d5880247e8cae9ee4be0
author : Michael Paquier <[email protected]>
date : Wed, 28 Apr 2021 11:17:58 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 28 Apr 2021 11:17:58 +0900
Attempting to use this function with event triggers failed, as, since
its introduction in a676201, this code has never associated an object
name with event triggers. This addresses the failure by adding the
event trigger name to the set defining its object address.
Note that regression tests are added within event_trigger and not
object_address to avoid issues with concurrent connections in parallel
schedules.
Author: Joel Jacobson
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/backend/catalog/objectaddress.c
M src/test/regress/expected/event_trigger.out
M src/test/regress/sql/event_trigger.sql
Improve logic in PostgresVersion.pm
commit : fa26eba221a9e837493df47d0255ce615129e9a8
author : Andrew Dunstan <[email protected]>
date : Tue, 27 Apr 2021 08:21:15 -0400
committer: Andrew Dunstan <[email protected]>
date : Tue, 27 Apr 2021 08:21:15 -0400
Handle the situation where perl swaps the order of operands of
the comparison operator. See `perldoc overload` for details:
The third argument is set to TRUE if (and only if) the two
operands have been swapped. Perl may do this to ensure that the
first argument ($self) is an object implementing the overloaded
operation, in line with general object calling conventions.
M src/test/perl/PostgresVersion.pm
doc: Review for "Allow TRUNCATE command to truncate foreign tables".
commit : 0c8f40863acb94963df9fd6a4369eb71efe9a93b
author : Fujii Masao <[email protected]>
date : Tue, 27 Apr 2021 18:39:30 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 27 Apr 2021 18:39:30 +0900
Typos, corrections and language improvements in the docs.
Author: Justin Pryzby, Fujii Masao
Reviewed-by: Bharath Rupireddy, Justin Pryzby, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/ref/truncate.sgml
Don't pass "ONLY" options specified in TRUNCATE to foreign data wrapper.
commit : 8e9ea08bae93a754d5075b7bc9c0b2bc71958bfd
author : Fujii Masao <[email protected]>
date : Tue, 27 Apr 2021 14:41:27 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 27 Apr 2021 14:41:27 +0900
Commit 8ff1c94649 allowed TRUNCATE command to truncate foreign tables.
Previously the information about "ONLY" options specified in TRUNCATE
command were passed to the foreign data wrapper. Then postgres_fdw
constructed the TRUNCATE command to issue the remote server and
included "ONLY" options in it based on the passed information.
On the other hand, "ONLY" options specified in SELECT, UPDATE or DELETE
have no effect when accessing or modifying the remote table, i.e.,
are not passed to the foreign data wrapper. So it's inconsistent to
make only TRUNCATE command pass the "ONLY" options to the foreign data
wrapper. Therefore this commit changes the TRUNCATE command so that
it doesn't pass the "ONLY" options to the foreign data wrapper,
for the consistency with other statements. Also this commit changes
postgres_fdw so that it always doesn't include "ONLY" options in
the TRUNCATE command that it constructs.
Author: Fujii Masao
Reviewed-by: Bharath Rupireddy, Kyotaro Horiguchi, Justin Pryzby, Zhihong Yu
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/postgres-fdw.sgml
M src/backend/commands/tablecmds.c
M src/backend/replication/logical/worker.c
M src/include/commands/tablecmds.h
M src/include/foreign/fdwapi.h
Use HTAB for replication slot statistics.
commit : 3fa17d37716f978f80dfcdab4e7c73f3a24e7a48
author : Amit Kapila <[email protected]>
date : Tue, 27 Apr 2021 09:09:11 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 27 Apr 2021 09:09:11 +0530
Previously, we used to use the array of size max_replication_slots to
store stats for replication slots. But that had two problems in the cases
where a message for dropping a slot gets lost: 1) the stats for the new
slot are not recorded if the array is full and 2) writing beyond the end
of the array if the user reduces the max_replication_slots.
This commit uses HTAB for replication slot statistics, resolving both
problems. Now, pgstat_vacuum_stat() search for all the dead replication
slots in stats hashtable and tell the collector to remove them. To avoid
showing the stats for the already-dropped slots, pg_stat_replication_slots
view searches slot stats by the slot name taken from pg_replication_slots.
Also, we send a message for creating a slot at slot creation, initializing
the stats. This reduces the possibility that the stats are accumulated
into the old slot stats when a message for dropping a slot gets lost.
Reported-by: Andres Freund
Author: Sawada Masahiko, test case by Vignesh C
Reviewed-by: Amit Kapila, Vignesh C, Dilip Kumar
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/t/001_repl_stats.pl
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/replication/slot.h
M src/test/regress/expected/rules.out
M src/tools/pgindent/typedefs.list
Fix Logical Replication of Truncate in synchronous commit mode.
commit : e7eea52b2d61917fbbdac7f3f895e4ef636e935b
author : Amit Kapila <[email protected]>
date : Tue, 27 Apr 2021 08:28:26 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 27 Apr 2021 08:28:26 +0530
The Truncate operation acquires an exclusive lock on the target relation
and indexes. It then waits for logical replication of the operation to
finish at commit. Now because we are acquiring the shared lock on the
target index to get index attributes in pgoutput while sending the
changes for the Truncate operation, it leads to a deadlock.
Actually, we don't need to acquire a lock on the target index as we build
the cache entry using a historic snapshot and all the later changes are
absorbed while decoding WAL. So, we wrote a special purpose function for
logical replication to get a bitmap of replica identity attribute numbers
where we get that information without locking the target index.
We decided not to backpatch this as there doesn't seem to be any field
complaint about this issue since it was introduced in commit 5dfd1e5a in
v11.
Reported-by: Haiying Tang
Author: Takamichi Osumi, test case by Li Japin
Reviewed-by: Amit Kapila, Ajin Cherian
Discussion: https://postgr.es/m/OS0PR01MB6113C2499C7DC70EE55ADB82FB759@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/backend/replication/logical/proto.c
M src/backend/utils/cache/relcache.c
M src/include/utils/relcache.h
M src/test/subscription/t/010_truncate.pl
psql: tab-complete ALTER ... DETACH CONCURRENTLY / FINALIZE
commit : 6dd1042eda0acdabaab352c19b783288de62b587
author : Alvaro Herrera <[email protected]>
date : Thu, 22 Apr 2021 16:37:46 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 22 Apr 2021 16:37:46 -0400
New keywords per 71f4c8c6f74b.
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/tab-complete.c
Remove rewriteTargetListIU's expansion of view targetlists in UPDATE.
commit : 04942bffd0aa9bd0d143d99b473342eb9ecee88b
author : Tom Lane <[email protected]>
date : Mon, 26 Apr 2021 13:58:00 -0400
committer: Tom Lane <[email protected]>
date : Mon, 26 Apr 2021 13:58:00 -0400
Commit 2ec993a7c, which added triggers on views, modified the rewriter
to add dummy entries like "SET x = x" for all columns that weren't
actually being updated by the user in any UPDATE directed at a view.
That was needed at the time to produce a complete "NEW" row to pass
to the trigger. Later it was found to cause problems for ordinary
updatable views, so commit cab5dc5da restricted it to happen only for
trigger-updatable views. But in the wake of commit 86dc90056, we
really don't need it at all. nodeModifyTable.c populates the trigger
"OLD" row from the whole-row variable that is generated for the view,
and then it computes the "NEW" row using that old row and the UPDATE
targetlist. So there is no need for the UPDATE tlist to have dummy
entries, any more than it needs them for regular tables or other
types of views.
(The comments for rewriteTargetListIU suggest that we must do this
for correct expansion of NEW references in rules, but I now think
that that was just lazy comment editing in 2ec993a7c. If we didn't
need it for rules on views before there were triggers, we don't need
it after that.)
This essentially propagates 86dc90056's decision that we don't need
dummy column updates into the view case. Aside from making the
different cases more uniform and hence possibly forestalling future
bugs, it ought to save a little bit of rewriter/planner effort.
Discussion: https://postgr.es/m/[email protected]
M src/backend/rewrite/rewriteHandler.c
Doc: document EXTRACT(JULIAN ...), improve Julian Date explanation.
commit : 79a5928ebcb726b7061bf265b5c6990e835e8c4f
author : Tom Lane <[email protected]>
date : Mon, 26 Apr 2021 11:50:35 -0400
committer: Tom Lane <[email protected]>
date : Mon, 26 Apr 2021 11:50:35 -0400
For some reason, the "julian" option for extract()/date_part() has
never gotten listed in the manual. Also, while Appendix B mentioned
in passing that we don't conform to the usual astronomical definition
that a Julian date starts at noon UTC, it was kind of vague about what
we do instead. Clarify that, and add an example showing how to get
the astronomical definition if you want it.
It's been like this for ages, so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/datetime.sgml
M doc/src/sgml/func.sgml
Fix pg_upgrade test on Cygwin
commit : 38c9a5938ac5e1409b42677fee970a12632852ee
author : Peter Eisentraut <[email protected]>
date : Mon, 26 Apr 2021 12:10:46 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 26 Apr 2021 12:10:46 +0200
The verification of permissions doesn't succeed on Cygwin, because the
required feature is not implemented for Cygwin at the moment. So skip
this part of the test, like MinGW already does.
M src/bin/pg_upgrade/test.sh
Add more tests with triggers on partitions for logical replication
commit : 2ecfeda3e916f2a1123f818018d9d35908a499ac
author : Michael Paquier <[email protected]>
date : Mon, 26 Apr 2021 15:22:48 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 26 Apr 2021 15:22:48 +0900
The tuple routing logic used by a logical replication worker can fire
triggers on relations part of a partition tree, but there was no test
coverage in this area. The existing script 003_constraints.pl included
something, but nothing when a tuple is applied across partitioned tables
on a subscriber.
Author: Amit Langote
Discussion: https://postgr.es/m/OS0PR01MB611383FA0FE92EB9DE21946AFB769@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/test/subscription/t/013_partition.pl
Avoid sending prepare multiple times while decoding.
commit : f25a4584c6f56f3407f8f8734c78f2a87e4b77e8
author : Amit Kapila <[email protected]>
date : Mon, 26 Apr 2021 11:27:44 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 26 Apr 2021 11:27:44 +0530
We send the prepare for the concurrently aborted xacts so that later when
rollback prepared is decoded and sent, the downstream should be able to
rollback such a xact. For 'streaming' case (when we send changes for
in-progress transactions), we were sending prepare twice when concurrent
abort was detected.
Author: Peter Smith
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/reorderbuffer.c
Remove unused function argument
commit : 3cbea581c76e86d51b8f2babf116e643847e7712
author : Peter Eisentraut <[email protected]>
date : Mon, 26 Apr 2021 07:05:21 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 26 Apr 2021 07:05:21 +0200
This was already unused in the initial commit
257836a75585934cc05ed7a80bccf8190d41e056. Apparently, it was used in
an earlier proposed patch version.
M src/bin/pg_dump/pg_dump.c
Fix typo in reorderbuffer.c.
commit : 6d2e87a077b3c2394e4adb8eb226b3dcfe3f3346
author : Amit Kapila <[email protected]>
date : Mon, 26 Apr 2021 08:42:46 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 26 Apr 2021 08:42:46 +0530
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+PtvzuYY0zu=dVRK_WVz5WGos1+otZWgEWqjha1ncoSRag@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
Update comments for rewriteTargetListIU().
commit : 08a986966524e522914b96e4398a4bebf942b298
author : Tom Lane <[email protected]>
date : Sun, 25 Apr 2021 18:02:03 -0400
committer: Tom Lane <[email protected]>
date : Sun, 25 Apr 2021 18:02:03 -0400
This function's behavior for UPDATE on a trigger-updatable view was
justified by analogy to what preptlist.c used to do for UPDATE on
regular tables. Since preptlist.c hasn't done that since 86dc90056,
that argument is no longer sensible, let alone convincing. I think
we do still need it to act that way, so update the comment to explain
why.
M src/backend/rewrite/rewriteHandler.c
Make a test endure log_error_verbosity=verbose.
commit : 59773da2b132ced19c84ff1f69b53b5cf95fd69e
author : Noah Misch <[email protected]>
date : Sun, 25 Apr 2021 01:08:05 -0700
committer: Noah Misch <[email protected]>
date : Sun, 25 Apr 2021 01:08:05 -0700
M src/test/recovery/t/024_archive_recovery.pl
Provide pg_amcheck with an --install-missing option
commit : b859d94c638968ccbb517ac7e151bdd94ed7c16a
author : Andrew Dunstan <[email protected]>
date : Sat, 24 Apr 2021 10:13:07 -0400
committer: Andrew Dunstan <[email protected]>
date : Sat, 24 Apr 2021 10:13:07 -0400
This will install amcheck in the database if not present. The default
schema is for the extension is pg_catalog, but this can be overridden by
providing a value for the option.
Mark Dilger, slightly editorialized by me.
(rather divergent)
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/pg_amcheck.sgml
M src/bin/pg_amcheck/pg_amcheck.c
Teach PostgresVersion all the ways to mark non-release code
commit : aa271209f6d995488fc5cba9731415f974823990
author : Andrew Dunstan <[email protected]>
date : Sat, 24 Apr 2021 09:37:20 -0400
committer: Andrew Dunstan <[email protected]>
date : Sat, 24 Apr 2021 09:37:20 -0400
As well as 'devel' version_stamp.pl provides for 'alphaN'
'betaN' and 'rcN', so teach PostgresVersion about those.
Also stash the version string instead of trying to reconstruct it during
stringification.
Discussion: https://postgr.es/m/[email protected]
M src/test/perl/PostgresVersion.pm
Fix come comments in execMain.c
commit : 9b5558e7ad4706bbd53947e5b4d7c06e150390a5
author : Michael Paquier <[email protected]>
date : Sat, 24 Apr 2021 15:07:04 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 24 Apr 2021 15:07:04 +0900
1375422 has refactored this area of the executor code, and some comments
went out-of-sync.
Author: Yukun Wang
Reviewed-by: Amul Sul
Discussion: https://postgr.es/m/OS0PR01MB60033394FCAEF79B98F078F5B4459@OS0PR01MB6003.jpnprd01.prod.outlook.com
M src/backend/executor/execMain.c
Doc: Remove extraneous whitespaces with some tags
commit : 0d0049c58b4970183a102fc1f7dc1e9bef2a7149
author : Michael Paquier <[email protected]>
date : Sat, 24 Apr 2021 10:44:13 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 24 Apr 2021 10:44:13 +0900
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/ref/pg_rewind.sgml
M doc/src/sgml/runtime.sgml
Add some forgotten LSN_FORMAT_ARGS() in xlogreader.c
commit : 4aba61b87026b43fb37fc8e9ec5d9ae208e07b6b
author : Michael Paquier <[email protected]>
date : Sat, 24 Apr 2021 09:09:02 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 24 Apr 2021 09:09:02 +0900
6f6f284 has introduced a specific macro to make printf()-ing of LSNs
easier. This takes care of what looks like the remaining code paths
that did not get the call.
Author: Michael Paquier
Reviewed-by: Kyotaro Horiguchi, Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlogreader.c
amcheck: MAXALIGN() nbtree special area offset.
commit : bb3ecc8c961896ecb2ad3d5ba705c2877b933945
author : Peter Geoghegan <[email protected]>
date : Fri, 23 Apr 2021 15:37:03 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 23 Apr 2021 15:37:03 -0700
This isn't strictly necessary, but in theory it might matter if in the
future the width of the nbtree special area changes -- its total size
might not be an even number of MAXALIGN() quantums, even with padding.
PageInit() MAXALIGN()s all special area offsets, but amcheck uses the
offset to perform initial basic validation of line pointers, so we don't
rely on the offset from the page header.
The real reason to do this is to set a good example for new code that
adds amcheck coverage for other index AMs.
Reported-By: Bharath Rupireddy <[email protected]>
Discussion: https://postgr.es/m/CALj2ACUMqTR9nErh99FbOBmzCXE9=gXNqhBiwYOhejJJS1LXqQ@mail.gmail.com
M contrib/amcheck/verify_nbtree.c
Factor out system call names from error messages
commit : 82c3cd974131d7fa1cfcd07cebfb04fffe26ee35
author : Peter Eisentraut <[email protected]>
date : Fri, 23 Apr 2021 14:18:11 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 23 Apr 2021 14:18:11 +0200
Instead, put them in via a format placeholder. This reduces the
number of distinct translatable messages and also reduces the chances
of typos during translation. We already did this for the system call
arguments in a number of cases, so this is just the same thing taken a
bit further.
Discussion: https://www.postgresql.org/message-id/flat/92d6f545-5102-65d8-3c87-489f71ea0a37%40enterprisedb.com
M src/backend/libpq/pqcomm.c
M src/backend/postmaster/pgstat.c
M src/backend/storage/ipc/latch.c
M src/bin/pg_basebackup/pg_recvlogical.c
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_dump/parallel.c
M src/bin/pg_upgrade/parallel.c
M src/common/exec.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-misc.c
Use correct format placeholder for WSAGetLastError()
commit : 9486844f301e265a3ad11b612decaba2462c3c15
author : Peter Eisentraut <[email protected]>
date : Fri, 23 Apr 2021 13:27:01 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 23 Apr 2021 13:27:01 +0200
Some code thought this was unsigned, but it's signed int.
M src/backend/libpq/pqcomm.c
M src/backend/port/win32/socket.c
M src/backend/storage/ipc/latch.c
M src/interfaces/libpq/fe-connect.c
Mark multirange_constructor0() and multirange_constructor2() strict
commit : 6bbcff096f932a1fe43ac3208c5c8b0acac29cda
author : Alexander Korotkov <[email protected]>
date : Fri, 23 Apr 2021 12:57:33 +0300
committer: Alexander Korotkov <[email protected]>
date : Fri, 23 Apr 2021 12:57:33 +0300
These functions shouldn't receive null arguments: multirange_constructor0()
doesn't have any arguments while multirange_constructor2() has a single array
argument, which is never null.
But mark them strict anyway for the sake of uniformity.
Also, make checks for null arguments use elog() instead of ereport() as these
errors should normally be never thrown. And adjust corresponding comments.
Catversion is bumped.
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/0f783a96-8d67-9e71-996b-f34a7352eeef%40enterprisedb.com
M src/backend/commands/typecmds.c
M src/backend/utils/adt/multirangetypes.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Reorder COMPRESSION option in gram.y and parsenodes.h into alphabetical order.
commit : 3f20d5f37086e548c32ddb9d6ae09c2e1ce300ce
author : Fujii Masao <[email protected]>
date : Fri, 23 Apr 2021 19:10:24 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 23 Apr 2021 19:10:24 +0900
Commit bbe0a81db6 introduced "INCLUDING COMPRESSION" option
in CREATE TABLE command, but previously TableLikeOption in gram.y and
parsenodes.h didn't classify this new option in alphabetical order
with the rest.
Author: Fujii Masao
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/gram.y
M src/include/nodes/parsenodes.h
Mention that toplevel is part of pg_stat_statements key.
commit : 7531fcb1fcf5b3ea2f49959a3f095c083e3fc4c4
author : Magnus Hagander <[email protected]>
date : Fri, 23 Apr 2021 11:41:36 +0200
committer: Magnus Hagander <[email protected]>
date : Fri, 23 Apr 2021 11:41:36 +0200
While at it, also document that toplevel is always true if
pg_stat_statements.track is set to top.
Author: Julien Rouhaud
Reported-By: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/pgstatstatements.sgml
pg_amcheck: Use logging functions
commit : add5fad78aac8da96aeeb730155d35b16ff9b55a
author : Peter Eisentraut <[email protected]>
date : Fri, 23 Apr 2021 09:55:23 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 23 Apr 2021 09:55:23 +0200
This was already mostly done, but some error messages were printed the
long way.
M src/bin/pg_amcheck/pg_amcheck.c
doc: Fix typos
commit : 9bd563aa9f0694994a6640946a7ee3dc0431d507
author : Peter Eisentraut <[email protected]>
date : Fri, 23 Apr 2021 09:28:44 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 23 Apr 2021 09:28:44 +0200
Author: Justin Pryzby <[email protected]>
M doc/src/sgml/func.sgml
doc: Fix obsolete description about pg_basebackup.
commit : eaec48b3c54eec222d64468b57af80ee4ddf76a9
author : Fujii Masao <[email protected]>
date : Fri, 23 Apr 2021 15:45:46 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 23 Apr 2021 15:45:46 +0900
Previously it was documented that if using "-X none" option there was
no guarantee that all required WAL files were archived at the end of
pg_basebackup when taking a backup from the standby. But this limitation
was removed by commit 52f8a59dd9. Now, even when taking a backup
from the standby, pg_basebackup can wait for all required WAL files
to be archived. Therefore this commit removes such obsolete
description from the docs.
Also this commit adds new description about the limitation when
taking a backup from the standby, into the docs. The limitation is that
pg_basebackup cannot force the standbfy to switch to a new WAL file
at the end of backup, which may cause pg_basebackup to wait a long
time for the last required WAL file to be switched and archived,
especially when write activity on the primary is low.
Back-patch to v10 where the issue was introduced.
Reported-by: Kyotaro Horiguchi
Author: Kyotaro Horiguchi, Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/pg_basebackup.sgml
Fix incorrect format placeholder
commit : 7776a23a4bdeb7215e4f8ddea5989cb143becc12
author : Peter Eisentraut <[email protected]>
date : Fri, 23 Apr 2021 07:21:13 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 23 Apr 2021 07:21:13 +0200
M src/backend/utils/misc/guc-file.l
Fix some comments in fmgr.c
commit : 45c0c5f70eb5e22d31be8bb9a8b4d9c66a3e9b37
author : Michael Paquier <[email protected]>
date : Fri, 23 Apr 2021 13:34:02 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 23 Apr 2021 13:34:02 +0900
Oversight in 2a0faed.
Author: Hou Zhijie
Discussion: https://postgr.es/m/OS0PR01MB5716405E2464D85E6DB6DC0794469@OS0PR01MB5716.jpnprd01.prod.outlook.com
M src/backend/utils/fmgr/fmgr.c
Remove use of [U]INT64_FORMAT in some translatable strings
commit : 62aa2bb293148c13851484e63db4835e3c53147f
author : Michael Paquier <[email protected]>
date : Fri, 23 Apr 2021 13:25:49 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 23 Apr 2021 13:25:49 +0900
%lld with (long long), or %llu with (unsigned long long) are more
adapted. This is similar to 3286065.
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_prewarm/pg_prewarm.c
M src/backend/access/transam/xlogprefetch.c
M src/bin/pgbench/pgbench.c
Minor code cleanup in asynchronous execution support.
commit : bb684c82f73316b88f9bcb321128a4347b5206fe
author : Etsuro Fujita <[email protected]>
date : Fri, 23 Apr 2021 12:00:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Fri, 23 Apr 2021 12:00:00 +0900
This is cleanup for commit 27e1f1456:
* ExecAppendAsyncEventWait(), which was modified a bit further by commit
a8af856d3, duplicated the same nevents calculation. Simplify the code
a little bit to avoid the duplication. Update comments there.
* Add an assertion to ExecAppendAsyncRequest().
* Update a comment about merging the async_capable options from input
relations in merge_fdw_options(), per complaint from Kyotaro Horiguchi.
* Add a comment for fetch_more_data_begin().
Author: Etsuro Fujita
Discussion: https://postgr.es/m/CAPmGK1637W30Wx3MnrReewhafn6F_0J76mrJGoFXFnpPq4QfvA%40mail.gmail.com
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/executor/nodeAppend.c
Don't crash on reference to an un-available system column.
commit : d479d00285255d422a2b38f1cfaa35808968a08c
author : Tom Lane <[email protected]>
date : Thu, 22 Apr 2021 17:30:42 -0400
committer: Tom Lane <[email protected]>
date : Thu, 22 Apr 2021 17:30:42 -0400
Adopt a more consistent policy about what slot-type-specific
getsysattr functions should do when system attributes are not
available. To wit, they should all throw the same user-oriented
error, rather than variously crashing or emitting developer-oriented
messages.
This closes a identifiable problem in commits a71cfc56b and
3fb93103a (in v13 and v12), so back-patch into those branches,
along with a test case to try to ensure we don't break it again.
It is not known that any of the former crash cases are reachable
in HEAD, but this seems like a good safety improvement in any case.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execTuples.c
M src/test/regress/expected/update.out
M src/test/regress/sql/update.sql
Fix some trailing whitespace in documentation files
commit : 197d33ccbe888fc84ae4e49bb241e88ea3c81f15
author : Peter Eisentraut <[email protected]>
date : Thu, 22 Apr 2021 22:47:57 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 22 Apr 2021 22:47:57 +0200
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/ref/reindex.sgml
M doc/src/sgml/ref/vacuum.sgml
M doc/src/sgml/wal.sgml
Fix uninitialized memory bug
commit : 43b55ec4bc3bc06596d966391f16defe016310ec
author : Alvaro Herrera <[email protected]>
date : Thu, 22 Apr 2021 16:04:48 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 22 Apr 2021 16:04:48 -0400
Have interested callers of find_inheritance_children set the
detached_exist value to false prior to calling it, so that that routine
only has to set it true in the rare cases where it is necessary. Don't
touch it otherwise.
Per buildfarm member thorntail (which reported a UBSan failure here).
M src/backend/catalog/pg_inherits.c
M src/backend/partitioning/partdesc.c
doc: mention <link> can be inside of <command>, but not <xref>
commit : 84f15ccd4c25c4ffc4de6ed82f7658a3a199a1d7
author : Bruce Momjian <[email protected]>
date : Thu, 22 Apr 2021 16:01:17 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 22 Apr 2021 16:01:17 -0400
This was discussed in commit 9081bddbd.
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/flat/[email protected]
M doc/src/sgml/README.links
Make PostgresVersion code a bit more robust and simple.
commit : 502dc6df8f6eeba06812ce09488efc7e684f5ec9
author : Andrew Dunstan <[email protected]>
date : Thu, 22 Apr 2021 15:25:37 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 22 Apr 2021 15:25:37 -0400
per gripe from Alvaro Herrera.
M src/test/perl/PostgresVersion.pm
Fix relcache inconsistency hazard in partition detach
commit : 8aba9322511f718f12b618470d8c07f0ee5f0700
author : Alvaro Herrera <[email protected]>
date : Thu, 22 Apr 2021 15:13:25 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 22 Apr 2021 15:13:25 -0400
During queries coming from ri_triggers.c, we need to omit partitions
that are marked pending detach -- otherwise, the RI query is tricked
into allowing a row into the referencing table whose corresponding row
is in the detached partition. Which is bogus: once the detach operation
completes, the row becomes an orphan.
However, the code was not doing that in repeatable-read transactions,
because relcache kept a copy of the partition descriptor that included
the partition, and used it in the RI query. This commit changes the
partdesc cache code to only keep descriptors that aren't dependent on
a snapshot (namely: those where no detached partition exist, and those
where detached partitions are included). When a partdesc-without-
detached-partitions is requested, we create one afresh each time; also,
those partdescs are stored in PortalContext instead of
CacheMemoryContext.
find_inheritance_children gets a new output *detached_exist boolean,
which indicates whether any partition marked pending-detach is found.
Its "include_detached" input flag is changed to "omit_detached", because
that name captures desired the semantics more naturally.
CreatePartitionDirectory() and RelationGetPartitionDesc() arguments are
identically renamed.
This was noticed because a buildfarm member that runs with relcache
clobbering, which would not keep the improperly cached partdesc, broke
one test, which led us to realize that the expected output of that test
was bogus. This commit also corrects that expected output.
Author: Amit Langote <[email protected]>
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/heap.c
M src/backend/catalog/pg_inherits.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/executor/execPartition.c
M src/backend/optimizer/util/plancat.c
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partdesc.c
M src/include/catalog/pg_inherits.h
M src/include/partitioning/partdesc.h
M src/test/isolation/expected/detach-partition-concurrently-4.out
Doc: document the tie-breaking behavior of the round() function.
commit : 82b13dbc4d4b46f71ca95ce1cc15c425deff5957
author : Tom Lane <[email protected]>
date : Thu, 22 Apr 2021 14:47:26 -0400
committer: Tom Lane <[email protected]>
date : Thu, 22 Apr 2021 14:47:26 -0400
Back-patch to v13; the table layout in older branches is unfriendly
to adding such details.
Laurenz Albe
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
Make PostgresNode version aware
commit : 4c4eaf3d19201c5e2d9efebc590903dfaba0d3e5
author : Andrew Dunstan <[email protected]>
date : Thu, 22 Apr 2021 10:56:28 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 22 Apr 2021 10:56:28 -0400
A new PostgresVersion object type is created and this is used in
PostgresNode using the output of `pg_config --version` and the result
stored in the PostgresNode object. This object can be compared to other
PostgresVersion objects, or to a number or string.
PostgresNode is currently believed to be compatible with versions down
to release 12, so PostgresNode will issue a warning if used with a
version prior to that.
No attempt has been made to deal with incompatibilities in older
versions - that remains work to be undertaken in a subsequent
development cycle.
Based on code from Mark Dilger and Jehan-Guillaume de Rorthais.
Discussion: https://postgr.es/m/[email protected]
M src/test/perl/PostgresNode.pm
A src/test/perl/PostgresVersion.pm
Fix relation leak for subscribers firing triggers in logical replication
commit : f3b141c482552a57866c72919007d6481cd59ee3
author : Michael Paquier <[email protected]>
date : Thu, 22 Apr 2021 12:48:54 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 22 Apr 2021 12:48:54 +0900
Creating a trigger on a relation to which an apply operation is
triggered would cause a relation leak once the change gets committed,
as the executor would miss that the relation needs to be closed
beforehand. This issue got introduced with the refactoring done in
1375422c, where it becomes necessary to track relations within
es_opened_result_relations to make sure that they are closed.
We have discussed using ExecInitResultRelation() coupled with
ExecCloseResultRelations() for the relations in need of tracking by the
apply operations in the subscribers, which would simplify greatly the
opening and closing of indexes, but this requires a larger rework and
reorganization of the worker code, particularly for the tuple routing
part. And that's not really welcome post feature freeze. So, for now,
settle down to the same solution as TRUNCATE which is to fill in
es_opened_result_relations with the relation opened, to make sure that
ExecGetTriggerResultRel() finds them and that they get closed.
The code is lightly refactored so as a relation is not registered three
times for each DML code path, making the whole a bit easier to follow.
Reported-by: Tang Haiying, Shi Yu, Hou Zhijie
Author: Amit Langote, Masahiko Sawada, Hou Zhijie
Reviewed-by: Amit Kapila, Michael Paquier
Discussion: https://postgr.es/m/OS0PR01MB611383FA0FE92EB9DE21946AFB769@OS0PR01MB6113.jpnprd01.prod.outlook.com
M src/backend/replication/logical/worker.c
doc: Move parallel_leader_participation to its correct category
commit : 1599e7b375127cac81b539d2c69d3faf7598509b
author : Michael Paquier <[email protected]>
date : Thu, 22 Apr 2021 09:47:43 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 22 Apr 2021 09:47:43 +0900
parallel_leader_participation got introduced in e5253fd, where it was
listed under RESOURCES_ASYNCHRONOUS in guc.c, but the documentation
did not reflect that and listed it with the other planner-related
options. This commit fixes this inconsistency as the parameter is
intended to be an asynchronous one.
While on it, reorganize a bit the section dedicated to asynchronous
parameters, backend_flush_after being moved first to do better in terms
of alphabetical order of the options listed.
Reported-by: Yanliang Lei
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
Add comment about extract_autovac_opts not holding lock
commit : 7c298c6573a0f181963ddcb40c850fa9c7da0ada
author : Alvaro Herrera <[email protected]>
date : Wed, 21 Apr 2021 18:36:12 -0400
committer: Alvaro Herrera <[email protected]>
date : Wed, 21 Apr 2021 18:36:12 -0400
Per observation from Tom Lane.
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/autovacuum.c
Don't add a redundant constraint when detaching a partition
commit : 7b357cc6ae553c0ecacdc11b2e5278b7bf477dba
author : Alvaro Herrera <[email protected]>
date : Wed, 21 Apr 2021 18:12:05 -0400
committer: Alvaro Herrera <[email protected]>
date : Wed, 21 Apr 2021 18:12:05 -0400
On ALTER TABLE .. DETACH CONCURRENTLY, we add a new table constraint
that duplicates the partition constraint. But if the partition already
has another constraint that implies that one, then that's unnecessary.
We were already avoiding the addition of a duplicate constraint if there
was an exact 'equal' match -- this just improves the quality of the check.
Author: Justin Pryzby <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
fix silly perl error in commit d064afc720
commit : e014d25deade08df082d2b37de45adb0c984f563
author : Andrew Dunstan <[email protected]>
date : Wed, 21 Apr 2021 11:12:04 -0400
committer: Andrew Dunstan <[email protected]>
date : Wed, 21 Apr 2021 11:12:04 -0400
M src/test/perl/PostgresNode.pm
Update config.guess and config.sub
commit : 26ac261ee4033710cad44f7924d53753129b60c7
author : Peter Eisentraut <[email protected]>
date : Wed, 21 Apr 2021 16:02:03 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 21 Apr 2021 16:02:03 +0200
M config/config.guess
M config/config.sub
Only ever test for non-127.0.0.1 addresses on Windows in PostgresNode
commit : d064afc7204b52cb78a83fea0e686693ce5ba00c
author : Andrew Dunstan <[email protected]>
date : Wed, 21 Apr 2021 10:21:22 -0400
committer: Andrew Dunstan <[email protected]>
date : Wed, 21 Apr 2021 10:21:22 -0400
This has been found to cause hangs where tcp usage is forced.
Alexey Kodratov
Discussion: https://postgr.es/m/[email protected]
Backpatch to all live branches
M src/test/perl/PostgresNode.pm
Add DISTINCT to information schema usage views
commit : d84ffffe582b8e036a14c6bc2378df29167f3a00
author : Peter Eisentraut <[email protected]>
date : Wed, 21 Apr 2021 11:54:47 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 21 Apr 2021 11:54:47 +0200
Since pg_depend can contain duplicate entries, we need to eliminate
those in information schema views that build on pg_depend, using
DISTINCT. Some of the older views already did that correctly, but
some of the more recently added ones didn't. (In some of these views,
it might not be possible to reproduce the issue because of how the
implementation happens to deduplicate dependencies while recording
them, but it seems better to keep this consistent in all cases.)
M src/backend/catalog/information_schema.sql
M src/include/catalog/catversion.h
Use correct format placeholder for timeline IDs
commit : 39d0928a0e88426ee64189898565c40d4af9ad96
author : Peter Eisentraut <[email protected]>
date : Wed, 21 Apr 2021 08:26:18 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 21 Apr 2021 08:26:18 +0200
Should be %u rather than %d.
M src/bin/pg_rewind/pg_rewind.c
doc: Improve hyphenation consistency
commit : 544b28088f9d41750ccf193812da62bdfe4bd98a
author : Peter Eisentraut <[email protected]>
date : Wed, 21 Apr 2021 08:14:43 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 21 Apr 2021 08:14:43 +0200
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/alter_policy.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_policy.sgml
M doc/src/sgml/ref/drop_policy.sgml
M doc/src/sgml/rules.sgml
M src/backend/commands/copyto.c
M src/backend/commands/functioncmds.c
M src/backend/executor/execMain.c
M src/backend/optimizer/path/allpaths.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/rewrite/rowsecurity.c
M src/include/catalog/pg_authid.h
M src/test/regress/expected/rowsecurity.out
M src/test/regress/sql/rowsecurity.sql
Don't use INT64_FORMAT inside message strings
commit : 3286065651477c2060910dfb42b3cedbd79a7980
author : Peter Eisentraut <[email protected]>
date : Mon, 19 Apr 2021 22:48:13 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 19 Apr 2021 22:48:13 +0200
Use %lld and cast to long long int instead.
M src/bin/pg_rewind/libpq_source.c
Fix typo
commit : f0ec598b4323d8b29df5c67f2cd0000547a507ed
author : Peter Eisentraut <[email protected]>
date : Mon, 19 Apr 2021 22:47:43 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 19 Apr 2021 22:47:43 +0200
M src/backend/access/transam/xlogprefetch.c
doc: List compute_query_id in required config for pg_stat_statements
commit : 64087eb5def7786bd49e60eb5d984ec6e4a872a9
author : Michael Paquier <[email protected]>
date : Wed, 21 Apr 2021 12:02:41 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 21 Apr 2021 12:02:41 +0900
Not enabling compute_query_id would disable pg_stat_statements even if
the module is listed in shared_preload_libraries, so add it to the
minimum configuration set as listed in its documentation.
Author: Greg Nancarrow
Reviewed-by: Julien Rouhaud, Bharath Rupireddy
Discussion: https://postgr.es/m/CAJcOf-fXyb2QiDbwftD813UF70w-+BsK-03bFp1GrijXU9GQYQ@mail.gmail.com
M doc/src/sgml/pgstatstatements.sgml
Add CURRENT_ROLE to list of roles for tab completion of GRANT in psql
commit : 22b2dec31b2ef4dfee299290b16375e9fa6e6932
author : Michael Paquier <[email protected]>
date : Wed, 21 Apr 2021 10:34:43 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 21 Apr 2021 10:34:43 +0900
This compatibility has been added in 45b9805, but psql forgot the call.
Author: Wei Wang
Reviewed-by: Aleksander Alekseev
Discussion: https://postgr.es/m/OS3PR01MB6275935F62E161BCD393D6559E489@OS3PR01MB6275.jpnprd01.prod.outlook.com
M src/bin/psql/tab-complete.c
Improve WAL record descriptions for SP-GiST records.
commit : 783be78ca91166ac7f80c953f2bbc5af1f61c6cd
author : Tom Lane <[email protected]>
date : Tue, 20 Apr 2021 17:01:43 -0400
committer: Tom Lane <[email protected]>
date : Tue, 20 Apr 2021 17:01:43 -0400
While tracking down the bug fixed in the preceding commit, I got quite
annoyed by the low quality of spg_desc's output. Add missing fields,
try to make the formatting consistent.
M src/backend/access/rmgrdesc/spgdesc.c
Fix under-parenthesized XLogRecHasBlockRef() macro.
commit : 9e41148229192dccc4bcc40f53af588b73d8ffea
author : Tom Lane <[email protected]>
date : Tue, 20 Apr 2021 16:58:30 -0400
committer: Tom Lane <[email protected]>
date : Tue, 20 Apr 2021 16:58:30 -0400
Commit f003d9f87 left this macro with inadequate (or, one could say,
too much) parenthesization. Which was catastrophic to the correctness
of calls such as "if (!XLogRecHasBlockRef(record, 1)) ...". There
are only a few of those, which perhaps explains why we didn't notice
immediately (with our general weakness of WAL replay testing being
another factor). I found it by debugging intermittent replay failures
like
2021-04-08 14:33:30.191 EDT [29463] PANIC: failed to locate backup block with ID 1
2021-04-08 14:33:30.191 EDT [29463] CONTEXT: WAL redo at 0/95D3438 for SPGist/ADD_NODE: off 1; blkref #0: rel 1663/16384/25998, blk 1
M src/include/access/xlogreader.h
Fix interaction of log_line_prefix's query_id and log_statement
commit : db01f797dd48f826c62e1b8eea70f11fe7ff3efc
author : Bruce Momjian <[email protected]>
date : Tue, 20 Apr 2021 12:57:59 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 20 Apr 2021 12:57:59 -0400
log_statement is issued before query_id can be computed, so properly
clear the value, and document the interaction.
Reported-by: Fujii Masao, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Author: Julien Rouhaud
M doc/src/sgml/config.sgml
M src/backend/utils/activity/backend_status.c
adjust query id feature to use pg_stat_activity.query_id
commit : 9660834dd8bf5b093f7b49eef846666201d45a35
author : Bruce Momjian <[email protected]>
date : Tue, 20 Apr 2021 12:22:26 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 20 Apr 2021 12:22:26 -0400
Previously, it was pg_stat_activity.queryid to match the
pg_stat_statements queryid column. This is an adjustment to patch
4f0b0966c8. This also adjusts some of the internal function calls to
match. Catversion bumped.
Reported-by: Álvaro Herrera, Julien Rouhaud
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/executor/execMain.c
M src/backend/executor/execParallel.c
M src/backend/parser/analyze.c
M src/backend/tcop/postgres.c
M src/backend/utils/activity/backend_status.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/error/elog.c
M src/backend/utils/misc/queryjumble.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/utils/backend_status.h
M src/test/regress/expected/rules.out
Rename find_em_expr_usable_for_sorting_rel.
commit : 7645376774c8532159f5f0f905e5e734d4ccbb18
author : Tom Lane <[email protected]>
date : Tue, 20 Apr 2021 11:37:36 -0400
committer: Tom Lane <[email protected]>
date : Tue, 20 Apr 2021 11:37:36 -0400
I didn't particularly like this function name, as it fails to
express what's going on. Also, returning the sort expression
alone isn't too helpful --- typically, a caller would also
need some other fields of the EquivalenceMember. But the
sole caller really only needs a bool result, so let's make
it "bool relation_can_be_sorted_early()".
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/equivclass.c
M src/include/optimizer/paths.h
Fix planner failure in some cases of sorting by an aggregate.
commit : 375398244168add84a884347625d14581a421e71
author : Tom Lane <[email protected]>
date : Tue, 20 Apr 2021 11:32:02 -0400
committer: Tom Lane <[email protected]>
date : Tue, 20 Apr 2021 11:32:02 -0400
An oversight introduced by the incremental-sort patches caused
"could not find pathkey item to sort" errors in some situations
where a sort key involves an aggregate or window function.
The basic problem here is that find_em_expr_usable_for_sorting_rel
isn't properly modeling what prepare_sort_from_pathkeys will do
later. Rather than hoping we can keep those functions in sync,
let's refactor so that they actually share the code for
identifying a suitable sort expression.
With this refactoring, tlist.c's tlist_member_ignore_relabel
is unused. I removed it in HEAD but left it in place in v13,
in case any extensions are using it.
Per report from Luc Vlaming. Back-patch to v13 where the
problem arose.
James Coleman and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/tlist.c
M src/include/optimizer/paths.h
M src/include/optimizer/tlist.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Avoid unfortunate IPC::Run path caching in PostgresNode
commit : 95c3a1956ec9eac686c1b69b033dd79211b72343
author : Andrew Dunstan <[email protected]>
date : Tue, 20 Apr 2021 10:14:16 -0400
committer: Andrew Dunstan <[email protected]>
date : Tue, 20 Apr 2021 10:14:16 -0400
Commit b34ca595ab provided for installation-aware instances of
PostgresNode. However, it turns out that IPC::Run works against this by
caching the path to a binary and not consulting the path again, even if
it has changed. We work around this by calling Postgres binaries with
the installed path rather than just a bare name to be looked up in the
environment path, if there is an installed path. For the common case
where there is no installed path we continue to use the bare command
name.
Diagnosis and solution from Mark Dilger
Discussion: https://postgr.es/m/[email protected]
M src/test/perl/PostgresNode.pm
Fix typo in comment
commit : 8b4b5669cde2b17bd6b5d68f584d97078f3296ac
author : Magnus Hagander <[email protected]>
date : Tue, 20 Apr 2021 14:35:16 +0200
committer: Magnus Hagander <[email protected]>
date : Tue, 20 Apr 2021 14:35:16 +0200
Author: Julien Rouhaud
Backpatch-through: 11
Discussion: https://postgr.es/m/20210420121659.odjueyd4rpilorn5@nol
M src/backend/lib/dshash.c
Document LP_DEAD accounting issues in VACUUM.
commit : 7136bf34f28892362144ae2e350714836a5c0c0c
author : Peter Geoghegan <[email protected]>
date : Mon, 19 Apr 2021 18:55:31 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 19 Apr 2021 18:55:31 -0700
Document VACUUM's soft assumption that any LP_DEAD items encountered
during pruning will become LP_UNUSED items before VACUUM finishes up.
This is integral to the accounting used by VACUUM to generate its final
report on the table to the stats collector. It also affects how VACUUM
determines which heap pages are truncatable. In both cases VACUUM is
concerned with the likely contents of the page in the near future, not
the current contents of the page.
This state of affairs created the false impression that VACUUM's dead
tuple accounting had significant difference with similar accounting used
during ANALYZE. There were and are no substantive differences, at least
when the soft assumption completely works out. This is far clearer now.
Also document cases where things don't quite work out for VACUUM's dead
tuple accounting. It's possible that a significant number of LP_DEAD
items will be left behind by VACUUM, and won't be recorded as remaining
dead tuples in VACUUM's statistics collector report. This behavior
dates back to commit a96c41fe, which taught VACUUM to run without index
and heap vacuuming at the user's request. The failsafe mechanism added
to VACUUM more recently by commit 1e55e7d1 takes the same approach to
dead tuple accounting.
Reported-By: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wz=Jmtu18PrsYq3EvvZJGOmZqSO2u3bvKpx9xJa5uhNp=Q@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Use correct format placeholder for pids
commit : 640b91c3ed24002b34c7226fb51ef9176fb72713
author : Peter Eisentraut <[email protected]>
date : Mon, 19 Apr 2021 10:43:18 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 19 Apr 2021 10:43:18 +0200
Should be signed, not unsigned.
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/bgworker.c
M src/backend/replication/logical/snapbuild.c
Fix test case added by commit f5fc2f5b23.
commit : c64dcc7fee5f8a7941a4fd098a969de1f457cc79
author : Amit Kapila <[email protected]>
date : Mon, 19 Apr 2021 09:02:47 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 19 Apr 2021 09:02:47 +0530
In the new test after resetting the stats, we were not waiting for the
stats message to be delivered. Also, we need to decode the results for
the new test, otherwise, it will show the old stats.
In passing,
a. Change docs added by commit f5fc2f5b23 as per suggestion by
Justin Pryzby.
b. Bump the PGSTAT_FILE_FORMAT_ID as commit f5fc2f5b23 changes the file
format of stats.
Reported-by: Tom Lane based on buildfarm reports
Author: Vignesh C, Justin Pryzby
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
M doc/src/sgml/monitoring.sgml
M src/include/pgstat.h
Fix typos and grammar in comments and docs
commit : 7ef8b52cf079ef3ace4575f7b97c2d6f80463b4f
author : Michael Paquier <[email protected]>
date : Mon, 19 Apr 2021 11:32:30 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 19 Apr 2021 11:32:30 +0900
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M contrib/amcheck/verify_heapam.c
M doc/src/sgml/brin.sgml
M doc/src/sgml/ecpg.sgml
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_bloom.c
M src/backend/access/brin/brin_minmax_multi.c
M src/backend/access/gist/gistbuild.c
M src/backend/access/index/genam.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/catalog/pg_type.c
M src/backend/commands/analyze.c
M src/backend/executor/nodeIncrementalSort.c
M src/backend/rewrite/rewriteSearchCycle.c
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/storage/ipc/procarray.c
M src/backend/tsearch/spell.c
M src/backend/utils/activity/backend_status.c
M src/backend/utils/adt/multirangetypes.c
M src/backend/utils/adt/selfuncs.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_waldump/pg_waldump.c
M src/common/hmac_openssl.c
M src/common/pg_lzcompress.c
M src/interfaces/ecpg/preproc/ecpg.c
M src/port/bsearch_arg.c
Replace magic constants for seek() calls in perl scripts
commit : c731f9187b5fd7038b04ba60703d3cace1806366
author : Michael Paquier <[email protected]>
date : Mon, 19 Apr 2021 10:15:35 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 19 Apr 2021 10:15:35 +0900
A couple of tests have been using 0 as magic constant while SEEK_SET can
be used instead. This makes the code easier to understand, and more
consistent with the changes done in 3c5b068.
Per discussion with Andrew Dunstan.
Discussion: https://postgr.es/m/[email protected]
M contrib/amcheck/t/001_verify_heapam.pl
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_checksums/t/002_actions.pl
Explain postmaster's treatment of SIGURG.
commit : 8e861eaae86eeaf5589963c9b1c7ce6d4c2acbb5
author : Thomas Munro <[email protected]>
date : Mon, 19 Apr 2021 10:22:31 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 19 Apr 2021 10:22:31 +1200
Add a few words of comment to explain why SIGURG doesn't follow the
dummy_handler pattern used for SIGUSR2, since that might otherwise
appear to be a bug.
Discussion: https://postgr.es/m/4006115.1618577212%40sss.pgh.pa.us
M src/backend/postmaster/postmaster.c
Add missing source files to nls.mk
commit : 4ed7f0599a8984d9ed967780a157d9b23d03fbb5
author : Peter Eisentraut <[email protected]>
date : Sun, 18 Apr 2021 16:11:58 +0200
committer: Peter Eisentraut <[email protected]>
date : Sun, 18 Apr 2021 16:11:58 +0200
M src/bin/pg_amcheck/nls.mk
M src/bin/scripts/nls.mk
doc: Fix up spacing around verbatim DocBook elements
commit : f7c09706c14d0858d5a186f3cc769471cba41578
author : Peter Eisentraut <[email protected]>
date : Sat, 17 Apr 2021 14:14:26 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 17 Apr 2021 14:14:26 +0200
M doc/src/sgml/libpq.sgml
M doc/src/sgml/logicaldecoding.sgml
Use correct format placeholder for block numbers
commit : f59b58e2a1b7e4a48dee36cc61966759da0faedd
author : Peter Eisentraut <[email protected]>
date : Sat, 17 Apr 2021 09:40:50 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 17 Apr 2021 09:40:50 +0200
Should be %u rather than %d.
M src/backend/access/gist/gistbuild.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/replication/basebackup.c
Rethink extraction of collation dependencies.
commit : f24b156997059c257c697b825f022d115825091d
author : Tom Lane <[email protected]>
date : Fri, 16 Apr 2021 22:23:46 -0400
committer: Tom Lane <[email protected]>
date : Fri, 16 Apr 2021 22:23:46 -0400
As it stands, find_expr_references_walker() pays attention to leaf-node
collation fields while ignoring the input collations of actual function
and operator nodes. That seems exactly backwards from a semantic
standpoint, and it leads to reporting dependencies on collations that
really have nothing to do with the expression's behavior.
Hence, rewrite to look at function input collations instead. This
isn't completely perfect either; it fails to account for the behavior
of record_eq and its siblings. (The previous coding at least gave an
approximation of that, though I think it could be fooled pretty easily
into considering the columns of irrelevant composite types.) We may
be able to improve on this later, but for now this should satisfy the
buildfarm members that didn't like ef387bed8.
In passing fix some oversights in GetTypeCollations(), and get
rid of its duplicative de-duplications. (I'm worried that it's
still potentially O(N^2) or worse, but this makes it a little
better.)
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/dependency.c
M src/backend/catalog/pg_type.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Update dummy prosrc values.
commit : 8a2df442b652f83b1c189822737091b90f343965
author : Tom Lane <[email protected]>
date : Fri, 16 Apr 2021 19:01:22 -0400
committer: Tom Lane <[email protected]>
date : Fri, 16 Apr 2021 19:01:22 -0400
Ooops, forgot to s/system_views.sql/system_functions.sql/g
in this part of 767982e36.
No need for an additional catversion bump, I think, since
these strings are gone by the time initdb finishes.
Discussion: https://postgr.es/m/[email protected]
M src/include/catalog/pg_proc.dat
Convert built-in SQL-language functions to SQL-standard-body style.
commit : 767982e36298be4da44a063e36261e9cfdc0bf49
author : Tom Lane <[email protected]>
date : Fri, 16 Apr 2021 18:36:45 -0400
committer: Tom Lane <[email protected]>
date : Fri, 16 Apr 2021 18:36:45 -0400
Adopt the new pre-parsed representation for all built-in and
information_schema SQL-language functions, except for a small
number that can't presently be converted because they have
polymorphic arguments.
This eliminates residual hazards around search-path safety of
these functions, and might provide some small performance benefits
by reducing parsing costs. It seems useful also to provide more
test coverage for the SQL-standard-body feature.
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/information_schema.sql
M src/backend/catalog/system_functions.sql
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Split function definitions out of system_views.sql into a new file.
commit : e80949372564c126c92aa7d64de483e04c0ef95e
author : Tom Lane <[email protected]>
date : Fri, 16 Apr 2021 18:20:42 -0400
committer: Tom Lane <[email protected]>
date : Fri, 16 Apr 2021 18:20:42 -0400
Invent system_functions.sql to carry the function definitions that
were formerly in system_views.sql. The function definitions were
already a quarter of the file and are about to be more, so it seems
appropriate to give them their own home.
In passing, fix an oversight in dfb75e478: it neglected to call
check_input() for system_constraints.sql.
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/Makefile
A src/backend/catalog/system_functions.sql
M src/backend/catalog/system_views.sql
M src/bin/initdb/initdb.c
Allow TestLib::slurp_file to skip contents, and use as needed
commit : 3c5b0685b921533f37622345beb0f8dd49200c01
author : Andrew Dunstan <[email protected]>
date : Fri, 16 Apr 2021 16:54:04 -0400
committer: Andrew Dunstan <[email protected]>
date : Fri, 16 Apr 2021 16:54:04 -0400
In order to avoid getting old logfile contents certain functions in
PostgresNode were doing one of two things. On Windows it rotated the
logfile and restarted the server, while elsewhere it truncated the log
file. Both of these are unnecessary. We borrow from the buildfarm which
does this instead: note the size of the logfile before we start, and
then when fetching the logfile skip to that position before accumulating
contents. This is spelled differently on Windows but the effect is the
same. This is largely centralized in TestLib's slurp_file function,
which has a new optional parameter, the offset to skip to before
starting to reading the file. Code in the client becomes much neater.
Backpatch to all live branches.
Michael Paquier, slightly modified by me.
Discussion: https://postgr.es/m/[email protected]
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
Fix bogus collation-version-recording logic.
commit : ef387bed87f2787b1012e68e9a33607a1074c123
author : Tom Lane <[email protected]>
date : Fri, 16 Apr 2021 12:26:50 -0400
committer: Tom Lane <[email protected]>
date : Fri, 16 Apr 2021 12:26:50 -0400
recordMultipleDependencies had the wrong scope for its "version"
variable, allowing a version label to leak from the collation entry it
was meant for to subsequent non-collation entries. This is relatively
hard to trigger because of the OID-descending order that the inputs
will normally arrive in: subsequent non-collation items will tend to
be pinned. But it can be exhibited easily with a custom collation.
Also, don't special-case the default collation, but instead ignore
pinned-ness of a collation when we've found a version for it. This
avoids creating useless pg_depend entries, and removes a not-very-
future-proof assumption that C, POSIX, and DEFAULT are the only
pinned collations.
A small problem is that, because the default collation may or may
not have a version, the regression tests can't assume anything about
whether dependency entries will be made for it. This seems OK though
since it's now handled just the same as other collations, and we have
test cases for both versioned and unversioned collations.
Fixes oversights in commit 257836a75. Thanks to Julien Rouhaud
for review.
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/pg_depend.c
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Fix wrong units in two ExplainPropertyFloat calls.
commit : f90c708a048667befbf6bbe5f48ae9695cb89de4
author : Tom Lane <[email protected]>
date : Fri, 16 Apr 2021 11:30:27 -0400
committer: Tom Lane <[email protected]>
date : Fri, 16 Apr 2021 11:30:27 -0400
This is only a latent bug, since these calls are only reached for
non-text output formats, and currently none of those will print
the units. Still, we should get it right in case that ever changes.
Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/explain.c
psql: Refine lexing of BEGIN...END blocks in CREATE FUNCTION statements
commit : 029c5ac03db72f1898ee17e417650a2e0764b239
author : Peter Eisentraut <[email protected]>
date : Fri, 16 Apr 2021 11:46:01 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 16 Apr 2021 11:46:01 +0200
Only track BEGIN...END blocks if they are in a CREATE [OR REPLACE]
{FUNCTION|PROCEDURE} statement. Ignore if in parentheses.
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/fe_utils/psqlscan.l
M src/include/fe_utils/psqlscan_int.h
psql: Small fixes for better translatability
commit : 25593d7d338232fb855ba563b325237de8f14091
author : Peter Eisentraut <[email protected]>
date : Fri, 16 Apr 2021 11:04:04 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 16 Apr 2021 11:04:04 +0200
M src/bin/psql/describe.c
M src/bin/psql/help.c
M src/test/regress/expected/psql.out
doc: Fix typo in example query of SQL/JSON
commit : 254a2164e5b216ecf98882f8bcb9e239ab2d432d
author : Michael Paquier <[email protected]>
date : Fri, 16 Apr 2021 16:56:12 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 16 Apr 2021 16:56:12 +0900
Author: Erik Rijkers
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M doc/src/sgml/json.sgml
Add information of total data processed to replication slot stats.
commit : f5fc2f5b23d1b1dff60f8ca5dc211161df47eda4
author : Amit Kapila <[email protected]>
date : Fri, 16 Apr 2021 07:34:43 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 16 Apr 2021 07:34:43 +0530
This adds the statistics about total transactions count and total
transaction data logically sent to the decoding output plugin from
ReorderBuffer. Users can query the pg_stat_replication_slots view to check
these stats.
Suggested-by: Andres Freund
Author: Vignesh C and Amit Kapila
Reviewed-by: Sawada Masahiko, Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/Makefile
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
A contrib/test_decoding/t/001_repl_stats.pl
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/replication/reorderbuffer.h
M src/test/regress/expected/rules.out
Doc: Document known problem with Windows collation versions.
commit : 1bf946bd43e545b86e567588b791311fe4e36a8c
author : Thomas Munro <[email protected]>
date : Fri, 16 Apr 2021 13:20:58 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 16 Apr 2021 13:20:58 +1200
Warn users that locales with traditional Windows NLS names like
"English_United States.1252" won't provide version information, and that
something like initdb --lc-collate=en-US would be needed to fix that
problem for the initial template databases.
Discussion: https://postgr.es/m/CA%2BhUKGJ_hk3rU%3D%3Dg2FpAMChb_4i%2BTJacpjjqFsinY-tRM3FBmA%40mail.gmail.com
M doc/src/sgml/charset.sgml
Provide query source text when parsing a SQL-standard function body.
commit : 409723365b2708acd3bdf2e830257504bdefac4b
author : Tom Lane <[email protected]>
date : Thu, 15 Apr 2021 17:24:12 -0400
committer: Tom Lane <[email protected]>
date : Thu, 15 Apr 2021 17:24:12 -0400
Without this, we lose error cursor positions, as shown in the
modified regression test result.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/functioncmds.c
M src/test/regress/expected/create_function_3.out
Revert "Cope with NULL query string in ExecInitParallelPlan()."
commit : 83efce7a1ebc5bae79617ddba12a64790141725c
author : Tom Lane <[email protected]>
date : Thu, 15 Apr 2021 17:17:45 -0400
committer: Tom Lane <[email protected]>
date : Thu, 15 Apr 2021 17:17:45 -0400
This reverts commit b3ee4c503872f3d0a5d6a7cbde48815f555af15b.
We don't need it in the wake of the preceding commit, which
added an upstream check that the querystring isn't null.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execParallel.c
Undo decision to allow pg_proc.prosrc to be NULL.
commit : 1111b2668d89bfcb6f502789158b1233ab4217a6
author : Tom Lane <[email protected]>
date : Thu, 15 Apr 2021 17:17:20 -0400
committer: Tom Lane <[email protected]>
date : Thu, 15 Apr 2021 17:17:20 -0400
Commit e717a9a18 changed the longstanding rule that prosrc is NOT NULL
because when a SQL-language function is written in SQL-standard style,
we don't currently have anything useful to put there. This seems a poor
decision though, as it could easily have negative impacts on external
PLs (opening them to crashes they didn't use to have, for instance).
SQL-function-related code can just as easily test "is prosqlbody not
null" as "is prosrc null", so there's no real gain there either.
Hence, revert the NOT NULL marking removal and adjust related logic.
For now, we just put an empty string into prosrc for SQL-standard
functions. Maybe we'll have a better idea later, although the
history of things like pg_attrdef.adsrc suggests that it's not
easy to maintain a string equivalent of a node tree.
This also adds an assertion that queryDesc->sourceText != NULL
to standard_ExecutorStart. We'd been silently relying on that
for awhile, so let's make it less silent.
Also fix some overlooked documentation and test cases.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/pg_proc.c
M src/backend/commands/functioncmds.c
M src/backend/executor/execMain.c
M src/backend/executor/functions.c
M src/backend/optimizer/util/clauses.c
M src/bin/pg_dump/pg_dump.c
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.h
M src/include/executor/functions.h
M src/test/regress/expected/create_function_3.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/sql/create_function_3.sql
M src/test/regress/sql/opr_sanity.sql
Stabilize recently-added information_schema test queries.
commit : 3157cbe974846729d49a1ee081944eee1839bdd8
author : Tom Lane <[email protected]>
date : Thu, 15 Apr 2021 16:31:36 -0400
committer: Tom Lane <[email protected]>
date : Thu, 15 Apr 2021 16:31:36 -0400
These queries could show unexpected entries if the core system,
or concurrently-running test scripts, created any functions that
would appear in the information_schema views. Restrict them
to showing functions belonging to this test's schema, as the
far-older nearby test case does.
Per experimentation with conversion of some built-in functions
to SQL-function-body style.
M src/test/regress/expected/create_function_3.out
M src/test/regress/sql/create_function_3.sql
Revert "psql: Show all query results by default"
commit : fae65629cec824738ee11bf60f757239906d64fa
author : Peter Eisentraut <[email protected]>
date : Thu, 15 Apr 2021 19:41:42 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 15 Apr 2021 19:41:42 +0200
This reverts commit 3a5130672296ed4e682403a77a9a3ad3d21cef75.
Per discussion, this patch had too many issues to resolve at this
point of the development cycle. We'll try again in the future.
Discussion: https://www.postgresql.org/message-id/flat/alpine.DEB.2.21.1904132231510.8961@lancre
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/common.c
M src/bin/psql/help.c
M src/bin/psql/settings.h
M src/bin/psql/startup.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/copyselect.out
M src/test/regress/expected/psql.out
M src/test/regress/expected/transactions.out
M src/test/regress/sql/copyselect.sql
M src/test/regress/sql/psql.sql
M src/test/regress/sql/transactions.sql
doc: Add missing COMPRESSION into CREATE TABLE synopsis.
commit : e2e2efca85b4857361780ed0c736c2a44edb458a
author : Fujii Masao <[email protected]>
date : Thu, 15 Apr 2021 23:15:19 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 15 Apr 2021 23:15:19 +0900
Commit bbe0a81db6 introduced "INCLUDING COMPRESSION" option
in CREATE TABLE command, but forgot to mention it in the
CREATE TABLE syntax synopsis.
Author: Fujii Masao
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_table.sgml
doc: Simplify example of HISTFILE for psql
commit : 1840d9f4c89998872a3b46473f8e9e5b6ff3c144
author : Michael Paquier <[email protected]>
date : Thu, 15 Apr 2021 16:45:34 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 15 Apr 2021 16:45:34 +0900
e4c7619 has added a space to the example used for HISTFILE in the docs
of psql before the variable DBNAME, as a workaround because variables
were not parsed the same way back then. This behavior has changed in
9.2, causing the example in the psql docs to result in the same history
file created with or without a space added before the DBNAME variable.
Let's just remove this space in the example, to reduce any confusion, as
the point of it is to prove that a per-database history file is easy to
set up, and that's easier to read this way.
Per discussion with Tom Lane.
Reported-by: Ludovic Kuty
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/psql-ref.sgml
pg_upgrade: Small fix for better translatability of help output
commit : cbae8774eb5c2f5552323ee18ca5286f9c8e5d06
author : Peter Eisentraut <[email protected]>
date : Thu, 15 Apr 2021 09:08:18 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 15 Apr 2021 09:08:18 +0200
M src/bin/pg_upgrade/option.c
amcheck: Use correct format placeholder for TOAST chunk numbers
commit : 59da8d9eb7255c3cb1c9f3b79d76b18b6a1c7da2
author : Peter Eisentraut <[email protected]>
date : Thu, 15 Apr 2021 08:58:03 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 15 Apr 2021 08:58:03 +0200
Several of these were already fixed in passing in
9acaf1a62197205b06a85afbfcaa7ffaac939ef3, but one was remaining
inconsistent.
M contrib/amcheck/verify_heapam.c
Tweak behavior of pg_dump --extension with configuration tables
commit : 344487e2db03f3cec13685a839dbc8a0e2a36750
author : Michael Paquier <[email protected]>
date : Thu, 15 Apr 2021 10:03:46 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 15 Apr 2021 10:03:46 +0900
6568cef, that introduced the option, had an inconsistent behavior when
it comes to configuration tables set up by pg_extension_config_dump, as
the data of all configuration tables would included in a dump even for
extensions not listed by a set of --extension switches.
The contents dumped changed depending on the schema where an extension
was installed when an extension was not listed. For example, an
extension installed under the public schema would have its configuration
data not dumped even when not listed with --extension, which was
inconsistent with the case of an extension installed on a non-public
schema, where the configuration would be dumped.
Per discussion with Noah, we have settled down to the simple rule of
dumping configuration data of an extension if it is listed in
--extension (default is unchanged and backward-compatible, to dump
everything on sight if there are no extensions directly listed). This
avoids some weird cases where the dumps depended on a --schema for one.
More tests are added to cover the gap, where we cross-check more
behaviors depending on --schema when an extension is not listed.
Reported-by: Noah Misch
Reviewed-by: Noah Misch
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/pg_dump.sgml
M src/bin/pg_dump/pg_dump.c
M src/test/modules/test_pg_dump/t/001_base.pl
Fix obsolete comments referencing JoinPathExtraData.extra_lateral_rels.
commit : e1623b7d86812ee6ab98a42f38b43a192c149988
author : Tom Lane <[email protected]>
date : Wed, 14 Apr 2021 14:28:17 -0400
committer: Tom Lane <[email protected]>
date : Wed, 14 Apr 2021 14:28:17 -0400
That field went away in commit edca44b15, but it seems that
commit 45be99f8c re-introduced some comments mentioning it.
Noted by James Coleman, though this isn't exactly his
proposed new wording. Also thanks to Justin Pryzby for
software archaeology.
Discussion: https://postgr.es/m/CAAaqYe8fxZjq3na+XkNx4C78gDqykH-7dbnzygm9Qa9nuDTePg@mail.gmail.com
M src/backend/optimizer/path/joinpath.c
amcheck: Reword some messages and fix an alignment problem.
commit : 9acaf1a62197205b06a85afbfcaa7ffaac939ef3
author : Robert Haas <[email protected]>
date : Wed, 14 Apr 2021 12:46:31 -0400
committer: Robert Haas <[email protected]>
date : Wed, 14 Apr 2021 12:46:31 -0400
We don't need to mention the attribute number in these messages, because
there's a dedicated column for that, but we should mention the toast
value ID, because that's really useful for any follow-up troubleshooting
the user wants to do. This also rewords some of the messages to hopefully
read a little better.
Also, use VARATT_EXTERNAL_GET_POINTER in case we're accessing a TOAST
pointer that isn't aligned on a platform that's fussy about alignment,
so that we don't crash while corruption-checking the user's data.
Mark Dilger, reviewed by me.
Discussion: http://postgr.es/m/[email protected]
M contrib/amcheck/verify_heapam.c
M src/bin/pg_amcheck/t/004_verify_heapam.pl
Improve quoting in some error messages
commit : 07e5e66742333ab100a557e6e3f710e92fa1fd92
author : Peter Eisentraut <[email protected]>
date : Wed, 14 Apr 2021 09:10:57 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 14 Apr 2021 09:10:57 +0200
M contrib/amcheck/verify_nbtree.c
M src/backend/access/brin/brin.c
M src/backend/access/index/indexam.c
doc: Move force_parallel_mode to section for developer options
commit : ac725ee0f98c3fec703ffd9b070da629608e9a1e
author : Michael Paquier <[email protected]>
date : Wed, 14 Apr 2021 15:55:55 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 14 Apr 2021 15:55:55 +0900
This GUC has always been classified as a planner option since its
introduction in 7c944bd, and was listed in postgresql.conf.sample. As
this parameter exists for testing purposes, move it to the section
dedicated to developer parameters and hence remove it from
postgresql.conf.sample. This will avoid any temptation to play with it
on production servers for users that should never really have to touch
this parameter.
The general description used for developer options is reworded a bit, to
take into account the inclusion of force_parallel_mode, per a suggestion
from Tom Lane.
Per discussion between Tom Lane, Bruce Momjian, Justin Pryzby, Bharath
Rupireddy and me.
Author: Justin Pryzby, Tom Lane
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
Simplify tests of postgres_fdw terminating connections
commit : 93f41461449f917da20af4fa2973f8afe8e6ea6e
author : Michael Paquier <[email protected]>
date : Wed, 14 Apr 2021 14:23:53 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 14 Apr 2021 14:23:53 +0900
The tests introduced in 32a9c0b for connections broken and
re-established rely on pg_terminate_backend() for their logic. When
these were introduced, this function simply sent a signal to a backend
without waiting for the operation to complete, and the tests repeatedly
looked at pg_stat_activity to check if the operation was completed or
not. Since aaf0432, it is possible to define a timeout to make
pg_terminate_backend() wait for a certain duration, so make use of it,
with a timeout reasonably large enough (3min) to give enough room for
the tests to pass even on slow machines.
Some measurements show that the tests of postgres_fdw are much faster
with this change. For example, on my laptop, they now take 4s instead
of 6s.
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/CALj2ACXGY_EfGrMTjKjHy2zi-u1u9rdeioU_fro0T6Jo8t56KQ@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Use NameData datatype for slotname in stats.
commit : cca57c1d9bf7eeba5b81115e0b82651cf3d8e4ea
author : Amit Kapila <[email protected]>
date : Wed, 14 Apr 2021 08:55:03 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 14 Apr 2021 08:55:03 +0530
This will make it consistent with the other usage of slotname in the code.
In the passing, change pgstat_report_replslot signature to use a structure
rather than multiple parameters.
Reported-by: Andres Freund
Author: Vignesh C
Reviewed-by: Sawada Masahiko, Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/pgstat.h
Initialize t_self and t_tableOid in statext_expressions_load
commit : 20661c15db8f430d4880ba685e6b12afa271c1ac
author : Tomas Vondra <[email protected]>
date : Wed, 14 Apr 2021 00:46:12 +0200
committer: Tomas Vondra <[email protected]>
date : Wed, 14 Apr 2021 00:46:12 +0200
The function is building a fake heap tuple, but left some of the header
fields (tid and table OID) uninitialized. Per Coverity report.
Reported-by: Ranier Vilela
Discussion: https://postgr.es/m/CAEudQApj6h8tZ0-eP5Af5PKc5NG1YUc7=SdN_99YoHS51fKa0Q@mail.gmail.com
M src/backend/statistics/extended_stats.c
Don't truncate heap when VACUUM's failsafe is in effect.
commit : 60f1f09ff44308667ef6c72fbafd68235e55ae27
author : Peter Geoghegan <[email protected]>
date : Tue, 13 Apr 2021 12:58:31 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 13 Apr 2021 12:58:31 -0700
It seems like a good idea to bypass heap truncation when the wraparound
failsafe mechanism (which was added in commit 1e55e7d1) is in effect.
Deliberately don't bypass heap truncation in the INDEX_CLEANUP=off case,
even though it is similar to the failsafe case. There is already a
separate reloption (and related VACUUM parameter) for that.
Reported-By: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/CAD21AoDWRh6oTN5T8wa+cpZUVpHXET8BJ8Da7WHVHpwkPP6KLg@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Allow table-qualified variable names in ON CONFLICT ... WHERE.
commit : 6c0373ab77359c94b279c4e67c91aa623841af65
author : Tom Lane <[email protected]>
date : Tue, 13 Apr 2021 15:39:33 -0400
committer: Tom Lane <[email protected]>
date : Tue, 13 Apr 2021 15:39:33 -0400
Previously you could only use unqualified variable names here.
While that's not a functional deficiency, since only the target
table can be referenced, it's a surprising inconsistency with the
rules for partial-index predicates, on which this syntax is
supposedly modeled.
The fix for that is no harder than passing addToRelNameSpace = true
to addNSItemToQuery. However, it's really pretty bogus for
transformOnConflictArbiter and transformOnConflictClause to be
messing with the namespace item for the target table at all.
It's not theirs to manage, it results in duplicative creations of
namespace items, and transformOnConflictClause wasn't even doing
it quite correctly (that coding resulted in two nsitems for the
target table, since it hadn't cleaned out the existing one).
Hence, make transformInsertStmt responsible for setting up the
target nsitem once for both these clauses and RETURNING.
Also, arrange for ON CONFLICT ... UPDATE's "excluded" pseudo-relation
to be added to the rangetable before we run transformOnConflictArbiter.
This produces a more helpful HINT if someone writes "excluded.col"
in the arbiter expression.
Per bug #16958 from Lukas Eder. Although I agree this is a bug,
the consequences are hardly severe, so no back-patch.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/analyze.c
M src/backend/parser/parse_clause.c
M src/test/regress/expected/insert_conflict.out
M src/test/regress/sql/insert_conflict.sql
docs: Update TOAST storage docs for configurable compression.
commit : e8c435a824e123f43067ce6f69d66f14cfb8815e
author : Robert Haas <[email protected]>
date : Tue, 13 Apr 2021 14:56:12 -0400
committer: Robert Haas <[email protected]>
date : Tue, 13 Apr 2021 14:56:12 -0400
Mention that there are multiple TOAST compression methods and that the
compression method used is stored in a TOAST pointer along with the
other information that was stored there previously. Add a reference to
the documentation for default_toast_compression, where the supported
methods are listed, instead of duplicating that here.
I haven't tried to preserve the text claiming that pglz is "fairly
simple and very fast." I have no view on the veracity of the former
claim, but LZ4 seems to be faster (and to compress better) so it
seems better not to muddy the waters by talking about compression
speed as a strong point of PGLZ.
Patch by me, reviewed by Justin Pryzby.
Discussion: http://postgr.es/m/CA+Tgmoaw_YBwQhOS_hhEPPwFhfAnu+VCLs18EfGr9gQw1z4H-w@mail.gmail.com
M doc/src/sgml/storage.sgml
Fix some inappropriately-disallowed uses of ALTER ROLE/DATABASE SET.
commit : 69d5ca484b69771073380e234e5377b6d6a5ebaf
author : Tom Lane <[email protected]>
date : Tue, 13 Apr 2021 15:10:18 -0400
committer: Tom Lane <[email protected]>
date : Tue, 13 Apr 2021 15:10:18 -0400
Most GUC check hooks that inspect database state have special checks
that prevent them from throwing hard errors for state-dependent issues
when source == PGC_S_TEST. This allows, for example,
"ALTER DATABASE d SET default_text_search_config = foo" when the "foo"
configuration hasn't been created yet. Without this, we have problems
during dump/reload or pg_upgrade, because pg_dump has no idea about
possible dependencies of GUC values and can't ensure a safe restore
ordering.
However, check_role() and check_session_authorization() hadn't gotten
the memo about that, and would throw hard errors anyway. It's not
entirely clear what is the use-case for "ALTER ROLE x SET role = y",
but we've now heard two independent complaints about that bollixing
an upgrade, so apparently some people are doing it.
Hence, fix these two functions to act more like other check hooks
with similar needs. (But I did not change their insistence on
being inside a transaction, as it's still not apparent that setting
either GUC from the configuration file would be wise.)
Also fix check_temp_buffers, which had a different form of the disease
of making state-dependent checks without any exception for PGC_S_TEST.
A cursory survey of other GUC check hooks did not find any more issues
of this ilk. (There are a lot of interdependencies among
PGC_POSTMASTER and PGC_SIGHUP GUCs, which may be a bad idea, but
they're not relevant to the immediate concern because they can't be
set via ALTER ROLE/DATABASE.)
Per reports from Charlie Hornsby and Nathan Bossart. Back-patch
to all supported branches.
Discussion: https://postgr.es/m/HE1P189MB0523B31598B0C772C908088DB7709@HE1P189MB0523.EURP189.PROD.OUTLOOK.COM
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/variable.c
M src/backend/utils/misc/guc.c
Redesign the caching done by get_cached_rowtype().
commit : c2db458c1036efae503ce5e451f8369e64c99541
author : Tom Lane <[email protected]>
date : Tue, 13 Apr 2021 13:37:07 -0400
committer: Tom Lane <[email protected]>
date : Tue, 13 Apr 2021 13:37:07 -0400
Previously, get_cached_rowtype() cached a pointer to a reference-counted
tuple descriptor from the typcache, relying on the ExprContextCallback
mechanism to release the tupdesc refcount when the expression tree
using the tupdesc was destroyed. This worked fine when it was designed,
but the introduction of within-DO-block COMMITs broke it. The refcount
is logged in a transaction-lifespan resource owner, but plpgsql won't
destroy simple expressions made within the DO block (before its first
commit) until the DO block is exited. That results in a warning about
a leaked tupdesc refcount when the COMMIT destroys the original resource
owner, and then an error about the active resource owner not holding a
matching refcount when the expression is destroyed.
To fix, get rid of the need to have a shutdown callback at all, by
instead caching a pointer to the relevant typcache entry. Those
survive for the life of the backend, so we needn't worry about the
pointer becoming stale. (For registered RECORD types, we can still
cache a pointer to the tupdesc, knowing that it won't change for the
life of the backend.) This mechanism has been in use in plpgsql
and expandedrecord.c since commit 4b93f5799, and seems to work well.
This change requires modifying the ExprEvalStep structs used by the
relevant expression step types, which is slightly worrisome for
back-patching. However, there seems no good reason for extensions
to be familiar with the details of these particular sub-structs.
Per report from Rohit Bhogate. Back-patch to v11 where within-DO-block
COMMITs became a thing.
Discussion: https://postgr.es/m/CAAV6ZkQRCVBh8qAY+SZiHnz+U+FqAGBBDaDTjF2yiKa2nJSLKg@mail.gmail.com
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/include/executor/execExpr.h
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/sql/plpgsql_transaction.sql
Avoid improbable PANIC during heap_update.
commit : 34f581c39e97e2ea237255cf75cccebccc02d477
author : Tom Lane <[email protected]>
date : Tue, 13 Apr 2021 12:17:24 -0400
committer: Tom Lane <[email protected]>
date : Tue, 13 Apr 2021 12:17:24 -0400
heap_update needs to clear any existing "all visible" flag on
the old tuple's page (and on the new page too, if different).
Per coding rules, to do this it must acquire pin on the appropriate
visibility-map page while not holding exclusive buffer lock;
which creates a race condition since someone else could set the
flag whenever we're not holding the buffer lock. The code is
supposed to handle that by re-checking the flag after acquiring
buffer lock and retrying if it became set. However, one code
path through heap_update itself, as well as one in its subroutine
RelationGetBufferForTuple, failed to do this. The end result,
in the unlikely event that a concurrent VACUUM did set the flag
while we're transiently not holding lock, is a non-recurring
"PANIC: wrong buffer passed to visibilitymap_clear" failure.
This has been seen a few times in the buildfarm since recent VACUUM
changes that added code paths that could set the all-visible flag
while holding only exclusive buffer lock. Previously, the flag
was (usually?) set only after doing LockBufferForCleanup, which
would insist on buffer pin count zero, thus preventing the flag
from becoming set partway through heap_update. However, it's
clear that it's heap_update not VACUUM that's at fault here.
What's less clear is whether there is any hazard from these bugs
in released branches. heap_update is certainly violating API
expectations, but if there is no code path that can set all-visible
without a cleanup lock then it's only a latent bug. That's not
100% certain though, besides which we should worry about extensions
or future back-patch fixes that could introduce such code paths.
I chose to back-patch to v12. Fixing RelationGetBufferForTuple
before that would require also back-patching portions of older
fixes (notably 0d1fe9f74), which is more code churn than seems
prudent to fix a hypothetical issue.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam.c
M src/backend/access/heap/hio.c
doc: Fix typo in logicaldecoding.sgml.
commit : 5fe83adad9efd5e3929f0465b44e786dc23c7b55
author : Fujii Masao <[email protected]>
date : Tue, 13 Apr 2021 14:21:51 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 13 Apr 2021 14:21:51 +0900
Introduced in commit 0aa8a01d04.
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+Ps8rkVHKWyjg09Fb1PaVG5-EhoFTFJ9OZMF4HPzDSXfew@mail.gmail.com
M doc/src/sgml/logicaldecoding.sgml
Use "-I." in directories holding Bison parsers, for Oracle compilers.
commit : 455dbc010be53ac61fcb2da83b1e565f4c263449
author : Noah Misch <[email protected]>
date : Mon, 12 Apr 2021 19:24:41 -0700
committer: Noah Misch <[email protected]>
date : Mon, 12 Apr 2021 19:24:41 -0700
With the Oracle Developer Studio 12.6 compiler, #line directives alter
the current source file location for purposes of #include "..."
directives. Hence, a VPATH build failed with 'cannot find include file:
"specscanner.c"'. With two exceptions, parser-containing directories
already add "-I. -I$(srcdir)"; eliminate the exceptions. Back-patch to
9.6 (all supported versions).
M src/backend/utils/adt/Makefile
M src/test/isolation/Makefile
Port regress-python3-mangle.mk to Solaris "sed".
commit : c3556f6fac349b31da2fd00107469ce36fb37536
author : Noah Misch <[email protected]>
date : Mon, 12 Apr 2021 19:24:21 -0700
committer: Noah Misch <[email protected]>
date : Mon, 12 Apr 2021 19:24:21 -0700
It doesn't support "\(foo\)*" like a POSIX "sed" implementation does;
see the Autoconf manual. Back-patch to 9.6 (all supported versions).
M src/pl/plpython/regress-python3-mangle.mk
Fix potential SSI hazard in heap_update().
commit : b1df6b696b759f00ebbf02e6de64e259d4be5785
author : Thomas Munro <[email protected]>
date : Tue, 13 Apr 2021 12:34:25 +1200
committer: Thomas Munro <[email protected]>
date : Tue, 13 Apr 2021 12:34:25 +1200
Commit 6f38d4dac38 failed to heed a warning about the stability of the
value pointed to by "otid". The caller is allowed to pass in a pointer to
newtup->t_self, which will be updated during the execution of the
function. Instead, the SSI check should use the value we copy into
oldtup.t_self near the top of the function.
Not a live bug, because newtup->t_self doesn't really get updated until
a bit later, but it was confusing and broke the rule established by the
comment.
Back-patch to 13.
Reported-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/2689164.1618160085%40sss.pgh.pa.us
M src/backend/access/heap/heapam.c
Remove duplicated --no-sync switches in new tests of test_pg_dump
commit : 885a87641930778d9580fdf0656af24e3f52d276
author : Michael Paquier <[email protected]>
date : Tue, 13 Apr 2021 09:42:01 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 13 Apr 2021 09:42:01 +0900
These got introduced in 6568cef.
Reported-by: Noah Misch
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/test_pg_dump/t/001_base.pl
Remove no-longer-relevant test case.
commit : cf0020080a3de20287217621da57bfd840e9a693
author : Tom Lane <[email protected]>
date : Mon, 12 Apr 2021 18:58:20 -0400
committer: Tom Lane <[email protected]>
date : Mon, 12 Apr 2021 18:58:20 -0400
collate.icu.utf8.sql was exercising the recording of a collation
dependency for an enum comparison expression, but such an expression
should never have had any collation dependency in the first place.
After I fixed that in commit c402b02b9, the test started failing.
We don't need to test that scenario anymore, so just remove the
now-useless test steps.
(This test case is new in HEAD, so no need to back-patch.)
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/HK0PR01MB22744393C474D503E16C8509F4709@HK0PR01MB2274.apcprd01.prod.exchangelabs.com
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Fix old bug with coercing the result of a COLLATE expression.
commit : c402b02b9fb53aee2a26876de90a8f95f9a9be92
author : Tom Lane <[email protected]>
date : Mon, 12 Apr 2021 14:37:22 -0400
committer: Tom Lane <[email protected]>
date : Mon, 12 Apr 2021 14:37:22 -0400
There are hacks in parse_coerce.c to push down a requested coercion
to below any CollateExpr that may appear. However, we did that even
if the requested data type is non-collatable, leading to an invalid
expression tree in which CollateExpr is applied to a non-collatable
type. The fix is just to drop the CollateExpr altogether, reasoning
that it's useless.
This bug is ten years old, dating to the original addition of
COLLATE support. The lack of field complaints suggests that there
aren't a lot of user-visible consequences. We noticed the problem
because it would trigger an assertion in DefineVirtualRelation if
the invalid structure appears as an output column of a view; however,
in a non-assert build, you don't see a crash just a (subtly incorrect)
complaint about applying collation to a non-collatable type. I found
that by putting the incorrect structure further down in a view, I could
make a view definition that would fail dump/reload, per the added
regression test case. But CollateExpr doesn't do anything at run-time,
so this likely doesn't lead to any really exciting consequences.
Per report from Yulin Pei. Back-patch to all supported branches.
Discussion: https://postgr.es/m/HK0PR01MB22744393C474D503E16C8509F4709@HK0PR01MB2274.apcprd01.prod.exchangelabs.com
M src/backend/parser/parse_coerce.c
M src/test/regress/expected/collate.out
M src/test/regress/sql/collate.sql
pg_upgrade: Print OID using %u instead of %d
commit : 6787e53fe59eed19095c771a8d3323fb59420733
author : Peter Eisentraut <[email protected]>
date : Mon, 12 Apr 2021 20:29:24 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 12 Apr 2021 20:29:24 +0200
This could write wrong output into the cluster deletion script if a
database OID exceeds the signed 32-bit range.
M src/bin/pg_upgrade/check.c
pg_amcheck: Add basic NLS support
commit : fc0fefbfe0d7e805f6a3a4aaaad7127090fcca51
author : Peter Eisentraut <[email protected]>
date : Mon, 12 Apr 2021 19:04:33 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 12 Apr 2021 19:04:33 +0200
A src/bin/pg_amcheck/nls.mk
M src/bin/pg_amcheck/pg_amcheck.c
Fix files references in nls.mk
commit : 44c8a3d75997b08fa7645dac7ee5c7a918a235c7
author : Peter Eisentraut <[email protected]>
date : Mon, 12 Apr 2021 15:44:38 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 12 Apr 2021 15:44:38 +0200
broken by 37d2ff38031262a1778bc76a9c55fff7afbcf275
M src/bin/pg_rewind/nls.mk
Support tab-complete for TRUNCATE on foreign tables.
commit : 81e094bdfdd6cf6568cba2b25eea9876daceaacb
author : Fujii Masao <[email protected]>
date : Mon, 12 Apr 2021 21:34:23 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 12 Apr 2021 21:34:23 +0900
Commit 8ff1c94649 extended TRUNCATE command so that it can also truncate
foreign tables. But it forgot to support tab-complete for TRUNCATE on
foreign tables. That is, previously tab-complete for TRUNCATE displayed
only the names of regular tables.
This commit improves tab-complete for TRUNCATE so that it displays also
the names of foreign tables.
Author: Fujii Masao
Reviewed-by: Bharath Rupireddy
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/tab-complete.c
Move log_autovacuum_min_duration into its correct sections
commit : b094063cd16d22b2f065a432580bb3568b2d8a77
author : Michael Paquier <[email protected]>
date : Mon, 12 Apr 2021 13:53:17 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 12 Apr 2021 13:53:17 +0900
This GUC has already been classified as LOGGING_WHAT, but its location
in postgresql.conf.sample and the documentation did not reflect that, so
fix those inconsistencies.
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M src/backend/utils/misc/postgresql.conf.sample
doc: Update information of new messages for logical replication.
commit : 15c1a9d9cb7604472d4823f48b64cdc02c441194
author : Amit Kapila <[email protected]>
date : Mon, 12 Apr 2021 09:27:10 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 12 Apr 2021 09:27:10 +0530
Updated documentation for new messages added for streaming of in-progress
transactions, as well as changes made to the existing messages. It also
updates the information of protocol versions supported for logical
replication.
Author: Ajin Cherian
Reviewed-by: Amit Kapila, Peter Smith, Euler Taveira
Discussion: https://postgr.es/m/CAFPTHDYHN9m=MZZct-B=BYg_TETvv+kXvL9RD2DpaBS5pGxGYg@mail.gmail.com
M doc/src/sgml/protocol.sgml
Fix out-of-bound memory access for interval -> char conversion
commit : 7a3972597f6ed7a6976d81abb66c38a7a1c29058
author : Michael Paquier <[email protected]>
date : Mon, 12 Apr 2021 11:30:50 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 12 Apr 2021 11:30:50 +0900
Using Roman numbers (via "RM" or "rm") for a conversion to calculate a
number of months has never considered the case of negative numbers,
where a conversion could easily cause out-of-bound memory accesses. The
conversions in themselves were not completely consistent either, as
specifying 12 would result in NULL, but it should mean XII.
This commit reworks the conversion calculation to have a more
consistent behavior:
- If the number of months and years is 0, return NULL.
- If the number of months is positive, return the exact month number.
- If the number of months is negative, do a backward calculation, with
-1 meaning December, -2 November, etc.
Reported-by: Theodor Arsenij Larionov-Trichkin
Author: Julien Rouhaud
Discussion: https://postgr.es/m/[email protected]
backpatch-through: 9.6
M src/backend/utils/adt/formatting.c
M src/test/regress/expected/timestamp.out
M src/test/regress/sql/timestamp.sql
Silence some Coverity warnings and improve code consistency.
commit : 6277435a8a89c59f716c111200c072d1454b8ff2
author : Tom Lane <[email protected]>
date : Sun, 11 Apr 2021 17:02:04 -0400
committer: Tom Lane <[email protected]>
date : Sun, 11 Apr 2021 17:02:04 -0400
Coverity complained about possible overflow in expressions like
intresult = tm->tm_sec * 1000000 + fsec;
on the grounds that the multiplication would happen in 32-bit
arithmetic before widening to the int64 result. I think these
are all false positives because of the limited possible range of
tm_sec; but nonetheless it seems silly to spell it like that when
nearby lines have the identical computation written with a 64-bit
constant.
... or more accurately, with an LL constant, which is not project
style. Make all of these use INT64CONST(), as we do elsewhere.
This is all new code from a2da77cdb, so no need for back-patch.
M src/backend/utils/adt/date.c
M src/backend/utils/adt/timestamp.c
Add macro PGWARNING, and make PGERROR available on all platforms.
commit : d7cff12c4c035b7cf12bb8454824f48f13018730
author : Tom Lane <[email protected]>
date : Sun, 11 Apr 2021 13:22:56 -0400
committer: Tom Lane <[email protected]>
date : Sun, 11 Apr 2021 13:22:56 -0400
We'd previously noted the need for coping with Windows headers
that provide some other definition of macro "ERROR" than elog.h
does. It turns out that R also wants to define ERROR, and
WARNING too. PL/R has been working around this in a hacky way
that broke when we recently changed the numeric value of ERROR.
To let them have a more future-proof solution, provide an
alternate macro PGWARNING for WARNING, and make PGERROR visible
always, not only when #ifdef WIN32.
Discussion: https://postgr.es/m/CADK3HHK6iMChd1yoOqssxBn5Z14Zar8Ztr3G-N_fuG7F8YTP3w@mail.gmail.com
M src/include/utils/elog.h
Fix uninitialized variable from commit a4d75c86b.
commit : 9cb92334092fa75afc62a71243bbc1f4612ecfa4
author : Tom Lane <[email protected]>
date : Sun, 11 Apr 2021 11:46:32 -0400
committer: Tom Lane <[email protected]>
date : Sun, 11 Apr 2021 11:46:32 -0400
The path for *exprs != NIL would misbehave, and likely crash,
since pull_varattnos expects its last argument to be valid
at call.
Found by Coverity --- we have no coverage of this path in
the regression tests.
M src/backend/statistics/extended_stats.c
Avoid unnecessary table open/close in TRUNCATE command.
commit : 81a23dd87999ec9fb62554328c69c5b678612d56
author : Fujii Masao <[email protected]>
date : Mon, 12 Apr 2021 00:05:58 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 12 Apr 2021 00:05:58 +0900
ExecuteTruncate() filters out the duplicate tables specified
in the TRUNCATE command, for example in the case where "TRUNCATE foo, foo"
is executed. Such duplicate tables obviously don't need to be opened
and closed because they are skipped. But previously it always opened
the tables before checking whether they were duplicated ones or not,
and then closed them if they were. That is, the duplicated tables were
opened and closed unnecessarily.
This commit changes ExecuteTruncate() so that it opens the table
after it confirms that table is not duplicated one, which leads to
avoid unnecessary table open/close.
Do not back-patch because such unnecessary table open/close is not
a bug though it exists in older versions.
Author: Bharath Rupireddy
Reviewed-by: Amul Sul, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACUdBO_sXJTa08OZ0YT0qk7F_gAmRa9hT4dxRcgPS4nsZA@mail.gmail.com
M src/backend/commands/tablecmds.c
Remove COMMIT_TS_SETTS record.
commit : 08aa89b326261b669648df97d4f2a6edba22d26a
author : Fujii Masao <[email protected]>
date : Mon, 12 Apr 2021 00:00:18 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 12 Apr 2021 00:00:18 +0900
Commit 438fc4a39c prevented the WAL replay from writing
COMMIT_TS_SETTS record. By this change there is no code that
generates COMMIT_TS_SETTS record in PostgreSQL core.
Also we can think that there are no extensions using the record
because we've not received so far any complaints about the issue
that commit 438fc4a39c fixed. Therefore this commit removes
COMMIT_TS_SETTS record and its related code. Even without
this record, the timestamp required for commit timestamp feature
can be acquired from the COMMIT record.
Bump WAL page magic.
Reported-by: lx zou <[email protected]>
Author: Fujii Masao
Reviewed-by: Alvaro Herrera
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/rmgrdesc/committsdesc.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xact.c
M src/include/access/commit_ts.h
M src/include/access/xlog_internal.h
Standardize pg_authid oid_symbol values.
commit : df5efaf4410f94cc1b69e8ade1d64dc92232ec1d
author : Noah Misch <[email protected]>
date : Sat, 10 Apr 2021 12:01:41 -0700
committer: Noah Misch <[email protected]>
date : Sat, 10 Apr 2021 12:01:41 -0700
Commit c9c41c7a337d3e2deb0b2a193e9ecfb865d8f52b used two different
naming patterns. Standardize on the majority pattern, which was the
only pattern in the last reviewed version of that commit.
M src/backend/catalog/aclchk.c
M src/backend/commands/user.c
M src/backend/utils/adt/acl.c
M src/include/catalog/pg_authid.dat
Improve behavior of date_bin with origin in the future
commit : 496e58bb0e5e939e6ed5839c92b05e3ab11b54bb
author : Peter Eisentraut <[email protected]>
date : Sat, 10 Apr 2021 19:33:46 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 10 Apr 2021 19:33:46 +0200
Currently, when the origin is after the input, the result is the
timestamp at the end of the bin, rather than the beginning as
expected. This puts the result consistently at the beginning of the
bin.
Author: John Naylor <[email protected]>
Discussion: https://www.postgresql.org/message-id/CAFBsxsGjLDxQofRfH+d4KSAXxPf3MMevUG7s6EDfdBOvHLDLjw@mail.gmail.com
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/timestamp.out
M src/test/regress/sql/timestamp.sql
Fix failure of xlogprefetch.h to include all prerequisite headers.
commit : 99964c4ade468c35a3f6e248a2380a1ff67d9cd3
author : Tom Lane <[email protected]>
date : Sat, 10 Apr 2021 13:16:08 -0400
committer: Tom Lane <[email protected]>
date : Sat, 10 Apr 2021 13:16:08 -0400
Per cpluspluscheck.
M src/include/access/xlogprefetch.h
Doc: update documentation of check_function_bodies.
commit : 07b76833b15163c6574ea2c12d05d9a0800665e2
author : Tom Lane <[email protected]>
date : Sat, 10 Apr 2021 12:08:28 -0400
committer: Tom Lane <[email protected]>
date : Sat, 10 Apr 2021 12:08:28 -0400
Adjust docs and description string to note that check_function_bodies
applies to procedures too. (In hindsight it should have been named
check_routine_bodies, but it seems too late for that now.)
Daniel Westermann
Discussion: https://postgr.es/m/GV0P278MB04834A9EB9A74B036DC7CE49D2739@GV0P278MB0483.CHEP278.PROD.OUTLOOK.COM
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc.c
Improve slightly misleading comments in nodeFuncs.c
commit : 152d33bccec7176f50be225bdbedf2e6de214e54
author : David Rowley <[email protected]>
date : Sat, 10 Apr 2021 19:19:45 +1200
committer: David Rowley <[email protected]>
date : Sat, 10 Apr 2021 19:19:45 +1200
There were some comments in nodeFuncs.c that, depending on your
interpretation of the word "result", could lead you to believe that the
comments were badly copied and pasted from somewhere else. If you thought
of "result" as the return value of the function that the comment is
written in, then you'd be misled. However, if you'd correctly
interpreted "result" to mean the result type of the given node type,
you'd not have seen any issues.
Here we do a small cleanup to try to prevent any future
misinterpretations. Per wording suggestion from Tom Lane.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/CAApHDvp+Bw=2Qiu5=uXMKfC7gd0+B=4JvexVgGJU=am2g9a1CA@mail.gmail.com
M src/backend/nodes/nodeFuncs.c
doc: Fix man page whitespace issues
commit : 9cae39b8e6a53decb37cce22852bb2e6d0e3f5d3
author : Peter Eisentraut <[email protected]>
date : Fri, 9 Apr 2021 23:23:45 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 9 Apr 2021 23:23:45 +0200
Whitespace between tags is significant, and in some cases it creates
extra vertical space in man pages. The fix is to remove some newlines
in the markup.
M doc/src/sgml/ref/cluster.sgml
M doc/src/sgml/ref/create_extension.sgml
M doc/src/sgml/ref/create_procedure.sgml
Suppress length of Notice/Error msgs in PQtrace regress mode
commit : e7e341409a3d85aba4cf754ba9cf722a4d8e6676
author : Alvaro Herrera <[email protected]>
date : Fri, 9 Apr 2021 17:13:18 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 9 Apr 2021 17:13:18 -0400
A (relatively minor) annoyance of ErrorResponse/NoticeResponse messages
as printed by PQtrace() is that their length might vary when we move
error messages from one source file to another, one function to another,
or even when their location line numbers change number of digits.
To avoid having to adjust expected files for some tests, make the
regress mode of PQtrace() suppress the length word of NoticeResponse and
ErrorResponse messages.
Discussion: https://postgr.es/m/[email protected]
Reviewed-by: Tom Lane <[email protected]>
M src/interfaces/libpq/fe-trace.c
M src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
M src/test/modules/libpq_pipeline/traces/transaction.trace
Make new GUC short descriptions more consistent.
commit : 846d35b2dc1bd4d09f5e3e5e5a2cc8efafeb600e
author : Thomas Munro <[email protected]>
date : Sat, 10 Apr 2021 08:41:07 +1200
committer: Thomas Munro <[email protected]>
date : Sat, 10 Apr 2021 08:41:07 +1200
Reported-by: Daniel Westermann (DWE) <[email protected]>
Discussion: https://postgr.es/m/GV0P278MB0483490FEAC879DCA5ED583DD2739%40GV0P278MB0483.CHEP278.PROD.OUTLOOK.COM
M src/backend/utils/misc/guc.c
Doc: Review for "Optionally prefetch referenced data in recovery."
commit : dc88460c24ed71ba7464ef4749e5f25da1bf6652
author : Thomas Munro <[email protected]>
date : Sat, 10 Apr 2021 08:09:30 +1200
committer: Thomas Munro <[email protected]>
date : Sat, 10 Apr 2021 08:09:30 +1200
Typos, corrections and language improvements in the docs, and a few in
code comments too.
Reported-by: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/20210409033703.GP6592%40telsasoft.com
M doc/src/sgml/config.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/xlogprefetch.c
M src/backend/utils/misc/guc.c
doc: Additional documentation for date_bin
commit : 49fb4e6b249029e75ccc6b490d76f15cd53d553b
author : Peter Eisentraut <[email protected]>
date : Fri, 9 Apr 2021 21:55:08 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 9 Apr 2021 21:55:08 +0200
Reported-by: Justin Pryzby <[email protected]>
Author: John Naylor <[email protected]>
Discussion: https://www.postgresql.org/message-id/CAFBsxsEEm1nuhZmfVQxvu_i3nDDEuvNJ_WMrDo9whFD_jusp-A@mail.gmail.com
M doc/src/sgml/func.sgml
Document ANALYZE storage parameters for partitioned tables
commit : 41badeaba8beee7648ebe7923a41c04f1f3cb302
author : Alvaro Herrera <[email protected]>
date : Fri, 9 Apr 2021 13:38:07 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 9 Apr 2021 13:38:07 -0400
Commit 0827e8af70f4 added parameters for autovacuum to support
partitioned tables, but didn't add any docs. Add them.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_table.sgml
Set pg_class.reltuples for partitioned tables
commit : 0e69f705cc1a3df273b38c9883fb5765991e04fe
author : Alvaro Herrera <[email protected]>
date : Fri, 9 Apr 2021 11:29:08 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 9 Apr 2021 11:29:08 -0400
When commit 0827e8af70f4 added auto-analyze support for partitioned
tables, it included code to obtain reltuples for the partitioned table
as a number of catalog accesses to read pg_class.reltuples for each
partition. That's not only very inefficient, but also problematic
because autovacuum doesn't hold any locks on any of those tables -- and
doesn't want to. Replace that code with a read of pg_class.reltuples
for the partitioned table, and make sure ANALYZE and TRUNCATE properly
maintain that value.
I found no code that would be affected by the change of relpages from
zero to non-zero for partitioned tables, and no other code that should
be maintaining it, but if there is, hopefully it'll be an easy fix.
Per buildfarm.
Author: Álvaro Herrera <[email protected]>
Reviewed-by: Zhihong Yu <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/analyze.c
M src/backend/commands/tablecmds.c
M src/backend/postmaster/autovacuum.c
Fix typo
commit : 1798d8f8b6fbb8ff922f640ff2d5dbd3e47064a2
author : Magnus Hagander <[email protected]>
date : Fri, 9 Apr 2021 12:40:14 +0200
committer: Magnus Hagander <[email protected]>
date : Fri, 9 Apr 2021 12:40:14 +0200
Author: Daniel Westermann
Backpatch-through: 9.6
Discussion: https://postgr.es/m/GV0P278MB0483A7AA85BAFCC06D90F453D2739@GV0P278MB0483.CHEP278.PROD.OUTLOOK.COM
M src/backend/utils/misc/guc.c
Fix typos and grammar in documentation and code comments
commit : 609b0652af00374b89411ea2613fd5bb92bca92c
author : Michael Paquier <[email protected]>
date : Fri, 9 Apr 2021 13:53:07 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 9 Apr 2021 13:53:07 +0900
Comment fixes are applied on HEAD, and documentation improvements are
applied on back-branches where needed.
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M doc/src/sgml/config.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/pgstatstatements.sgml
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/createuser.sgml
M doc/src/sgml/ref/declare.sgml
M doc/src/sgml/ref/pg_amcheck.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/wal.sgml
M src/backend/catalog/heap.c
M src/backend/commands/analyze.c
M src/backend/commands/cluster.c
M src/backend/commands/copyfrom.c
M src/backend/statistics/extended_stats.c
M src/backend/utils/adt/jsonfuncs.c
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/include/lib/sort_template.h
M src/include/utils/guc.h
Silence another _bt_check_unique compiler warning.
commit : 796092fb84c08162ae803e83a13aa8bd6d9b23d0
author : Peter Geoghegan <[email protected]>
date : Thu, 8 Apr 2021 12:54:31 -0700
committer: Peter Geoghegan <[email protected]>
date : Thu, 8 Apr 2021 12:54:31 -0700
Per complaint from Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/nbtree/nbtinsert.c
Add support for tab-completion of type arguments in \df, \do.
commit : d1fcbde579d440c35023baa0de7ebf27f644a314
author : Tom Lane <[email protected]>
date : Thu, 8 Apr 2021 15:38:26 -0400
committer: Tom Lane <[email protected]>
date : Thu, 8 Apr 2021 15:38:26 -0400
Oversight in commit a3027e1e7.
M src/bin/psql/tab-complete.c
Suppress uninitialized-variable warning.
commit : 01add89454d5dc289ed3126d5de03169bdeff41b
author : Tom Lane <[email protected]>
date : Thu, 8 Apr 2021 15:14:26 -0400
committer: Tom Lane <[email protected]>
date : Thu, 8 Apr 2021 15:14:26 -0400
Several buildfarm critters that don't usually produce such
warnings are complaining about e717a9a18. I think it's
actually safe, but move initialization to silence the warning.
M src/backend/catalog/pg_proc.c
Fixes for query_id feature
commit : 0f61727b75b93915ca9a9f20c996ed7020996a38
author : Bruce Momjian <[email protected]>
date : Thu, 8 Apr 2021 11:16:01 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 8 Apr 2021 11:16:01 -0400
Ignore parallel workers in pg_stat_statements
Oversight in 4f0b0966c8 which exposed queryid in parallel workers.
Counters are aggregated by the main backend process so parallel workers
would report duplicated activity, and could also report activity for the
wrong entry as they are only aware of the top level queryid.
Fix thinko in pg_stat_get_activity when retrieving the queryid.
Remove unnecessary call to pgstat_report_queryid().
Reported-by: Amit Kapila, Andres Freund, Thomas Munro
Discussion: https://postgr.es/m/[email protected] [email protected]
Author: Julien Rouhaud
M contrib/pg_stat_statements/pg_stat_statements.c
M src/backend/executor/execParallel.c
M src/backend/utils/adt/pgstatfuncs.c
Merge v1.10 of pg_stat_statements into v1.9
commit : 5844c23dc50508aefeb8183be45f4ee99e9dec17
author : Magnus Hagander <[email protected]>
date : Thu, 8 Apr 2021 15:15:17 +0200
committer: Magnus Hagander <[email protected]>
date : Thu, 8 Apr 2021 15:15:17 +0200
v1.9 is already new in this version of PostgreSQL, so turn it into just
one change.
Author: Julien Rohaud
Discussion: https://postgr.es/m/20210408120505.7zinijtdexbyghvb@nol
M contrib/pg_stat_statements/Makefile
M contrib/pg_stat_statements/pg_stat_statements–1.8–1.9.sql
D contrib/pg_stat_statements/pg_stat_statements–1.9–1.10.sql
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/pg_stat_statements.control
Remove duplicate typedef.
commit : 34399a670a1c559ef8a355ed25d090d73e400ad4
author : Thomas Munro <[email protected]>
date : Fri, 9 Apr 2021 00:36:59 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 9 Apr 2021 00:36:59 +1200
Thinko in commit 323cbe7c, per complaint from BF animal locust's older
GCC compiler.
M src/include/replication/logical.h
Allow TRUNCATE command to truncate foreign tables.
commit : 8ff1c94649f5c9184ac5f07981d8aea9dfd7ac19
author : Fujii Masao <[email protected]>
date : Thu, 8 Apr 2021 20:56:08 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 8 Apr 2021 20:56:08 +0900
This commit introduces new foreign data wrapper API for TRUNCATE.
It extends TRUNCATE command so that it accepts foreign tables as
the targets to truncate and invokes that API. Also it extends postgres_fdw
so that it can issue TRUNCATE command to foreign servers, by adding
new routine for that TRUNCATE API.
The information about options specified in TRUNCATE command, e.g.,
ONLY, CACADE, etc is passed to FDW via API. The list of foreign tables to
truncate is also passed to FDW. FDW truncates the foreign data sources
that the passed foreign tables specify, based on those information.
For example, postgres_fdw constructs TRUNCATE command using them
and issues it to the foreign server.
For performance, TRUNCATE command invokes the FDW routine for
TRUNCATE once per foreign server that foreign tables to truncate belong to.
Author: Kazutaka Onishi, Kohei KaiGai, slightly modified by Fujii Masao
Reviewed-by: Bharath Rupireddy, Michael Paquier, Zhihong Yu, Alvaro Herrera, Stephen Frost, Ashutosh Bapat, Amit Langote, Daniel Gustafsson, Ibrar Ahmed, Fujii Masao
Discussion: https://postgr.es/m/CAOP8fzb_gkReLput7OvOK+8NHgw-RKqNv59vem7=524krQTcWA@mail.gmail.com
Discussion: https://postgr.es/m/CAJuF6cMWDDqU-vn_knZgma+2GMaout68YUgn1uyDnexRhqqM5Q@mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/ref/truncate.sgml
M src/backend/commands/tablecmds.c
M src/backend/replication/logical/worker.c
M src/include/commands/tablecmds.h
M src/include/foreign/fdwapi.h
M src/include/utils/rel.h
M src/test/regress/expected/foreign_data.out
M src/tools/pgindent/typedefs.list
Speedup ScalarArrayOpExpr evaluation
commit : 50e17ad281b8d1c1b410c9833955bc80fbad4078
author : David Rowley <[email protected]>
date : Thu, 8 Apr 2021 23:51:22 +1200
committer: David Rowley <[email protected]>
date : Thu, 8 Apr 2021 23:51:22 +1200
ScalarArrayOpExprs with "useOr=true" and a set of Consts on the righthand
side have traditionally been evaluated by using a linear search over the
array. When these arrays contain large numbers of elements then this
linear search could become a significant part of execution time.
Here we add a new method of evaluating ScalarArrayOpExpr expressions to
allow them to be evaluated by first building a hash table containing each
element, then on subsequent evaluations, we just probe that hash table to
determine if there is a match.
The planner is in charge of determining when this optimization is possible
and it enables it by setting hashfuncid in the ScalarArrayOpExpr. The
executor will only perform the hash table evaluation when the hashfuncid
is set.
This means that not all cases are optimized. For example CHECK constraints
containing an IN clause won't go through the planner, so won't get the
hashfuncid set. We could maybe do something about that at some later
date. The reason we're not doing it now is from fear that we may slow
down cases where the expression is evaluated only once. Those cases can
be common, for example, a single row INSERT to a table with a CHECK
constraint containing an IN clause.
In the planner, we enable this when there are suitable hash functions for
the ScalarArrayOpExpr's operator and only when there is at least
MIN_ARRAY_SIZE_FOR_HASHED_SAOP elements in the array. The threshold is
currently set to 9.
Author: James Coleman, David Rowley
Reviewed-by: David Rowley, Tomas Vondra, Heikki Linnakangas
Discussion: https://postgr.es/m/CAAaqYe8x62+=wn0zvNKCj55tPpg-JBHzhZFFc6ANovdqFw7-dA@mail.gmail.com
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/jit/llvm/llvmjit_types.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/prep/prepqual.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/parse_oper.c
M src/backend/partitioning/partbounds.c
M src/include/catalog/catversion.h
M src/include/executor/execExpr.h
M src/include/nodes/primnodes.h
M src/include/optimizer/optimizer.h
M src/test/regress/expected/expressions.out
M src/test/regress/sql/expressions.sql
Optionally prefetch referenced data in recovery.
commit : 1d257577e08d3e598011d6850fd1025858de8c8c
author : Thomas Munro <[email protected]>
date : Thu, 8 Apr 2021 23:03:43 +1200
committer: Thomas Munro <[email protected]>
date : Thu, 8 Apr 2021 23:03:43 +1200
Introduce a new GUC recovery_prefetch, disabled by default. When
enabled, look ahead in the WAL and try to initiate asynchronous reading
of referenced data blocks that are not yet cached in our buffer pool.
For now, this is done with posix_fadvise(), which has several caveats.
Better mechanisms will follow in later work on the I/O subsystem.
The GUC maintenance_io_concurrency is used to limit the number of
concurrent I/Os we allow ourselves to initiate, based on pessimistic
heuristics used to infer that I/Os have begun and completed.
The GUC wal_decode_buffer_size is used to limit the maximum distance we
are prepared to read ahead in the WAL to find uncached blocks.
Reviewed-by: Alvaro Herrera <[email protected]> (parts)
Reviewed-by: Andres Freund <[email protected]> (parts)
Reviewed-by: Tomas Vondra <[email protected]> (parts)
Tested-by: Tomas Vondra <[email protected]>
Tested-by: Jakub Wartak <[email protected]>
Tested-by: Dmitry Dolgov <[email protected]>
Tested-by: Sait Talha Nisanci <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJ4VJN8ttxScUFM8dOKX0BrBiboo5uz1cq%3DAovOddfHpA%40mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/Makefile
M src/backend/access/transam/xlog.c
A src/backend/access/transam/xlogprefetch.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogutils.c
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/storage/freespace/freespace.c
M src/backend/storage/ipc/ipci.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/access/xlog.h
A src/include/access/xlogprefetch.h
M src/include/access/xlogreader.h
M src/include/access/xlogutils.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/utils/guc.h
M src/test/regress/expected/rules.out
M src/tools/pgindent/typedefs.list
Add circular WAL decoding buffer.
commit : f003d9f8721b3249e4aec8a1946034579d40d42c
author : Thomas Munro <[email protected]>
date : Thu, 8 Apr 2021 23:03:34 +1200
committer: Thomas Munro <[email protected]>
date : Thu, 8 Apr 2021 23:03:34 +1200
Teach xlogreader.c to decode its output into a circular buffer, to
support optimizations based on looking ahead.
* XLogReadRecord() works as before, consuming records one by one, and
allowing them to be examined via the traditional XLogRecGetXXX()
macros.
* An alternative new interface XLogNextRecord() is added that returns
pointers to DecodedXLogRecord structs that can be examined directly.
* XLogReadAhead() provides a second cursor that lets you see
further ahead, as long as data is available and there is enough space
in the decoding buffer. This returns DecodedXLogRecord pointers to the
caller, but also adds them to a queue of records that will later be
consumed by XLogNextRecord()/XLogReadRecord().
The buffer's size is controlled with wal_decode_buffer_size. The buffer
could potentially be placed into shared memory, for future projects.
Large records that don't fit in the circular buffer are called
"oversized" and allocated separately with palloc().
Discussion: https://postgr.es/m/CA+hUKGJ4VJN8ttxScUFM8dOKX0BrBiboo5uz1cq=AovOddfHpA@mail.gmail.com
M src/backend/access/transam/generic_xlog.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogutils.c
M src/backend/replication/logical/decode.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_waldump/pg_waldump.c
M src/include/access/xlogreader.h
Remove read_page callback from XLogReader.
commit : 323cbe7c7ddcf18aaf24b7f6d682a45a61d4e31b
author : Thomas Munro <[email protected]>
date : Thu, 8 Apr 2021 23:03:23 +1200
committer: Thomas Munro <[email protected]>
date : Thu, 8 Apr 2021 23:03:23 +1200
Previously, the XLogReader module would fetch new input data using a
callback function. Redesign the interface so that it tells the caller
to insert more data with a special return value instead. This API suits
later patches for prefetching, encryption and maybe other future
projects that would otherwise require continually extending the callback
interface.
As incidental cleanup work, move global variables readOff, readLen and
readSegNo inside XlogReaderState.
Author: Kyotaro HORIGUCHI <[email protected]>
Author: Heikki Linnakangas <[email protected]> (parts of earlier version)
Reviewed-by: Antonin Houska <[email protected]>
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Takashi Menjo <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Discussion: https://postgr.es/m/20190418.210257.43726183.horiguchi.kyotaro%40lab.ntt.co.jp
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogutils.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/logicalfuncs.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/walsender.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_waldump/pg_waldump.c
M src/include/access/xlogreader.h
M src/include/access/xlogutils.h
M src/include/replication/logical.h
Cleanup partition pruning step generation
commit : 5ac9c4307337313bedeafc21dbbab93ba809241c
author : David Rowley <[email protected]>
date : Thu, 8 Apr 2021 22:35:48 +1200
committer: David Rowley <[email protected]>
date : Thu, 8 Apr 2021 22:35:48 +1200
There was some code in gen_prune_steps_from_opexps that needlessly
checked a list was not empty when it clearly had to contain at least one
item. This prompted a further cleanup operation in partprune.c.
Additionally, the previous code could end up adding additional needless
INTERSECT steps. However, those do not appear to be able to cause any
misbehavior.
gen_prune_steps_from_opexps is now no longer in charge of generating
combine pruning steps. Instead, gen_partprune_steps_internal, which
already does some combine step creation has been given the sole
responsibility of generating all combine steps. This means that when
we recursively call gen_partprune_steps_internal, since it always now adds
a combine step when it produces multiple steps, we can just pay attention
to the final step returned.
In passing, do quite a bit of work on the comments to try to more clearly
explain the role of both gen_partprune_steps_internal and
gen_prune_steps_from_opexps. This is fairly complex code so some extra
effort to give any new readers an overview of how things work seems like
a good idea.
Author: Amit Langote
Reported-by: Andy Fan
Reviewed-by: Kyotaro Horiguchi, Andy Fan, Ryan Lambert, David Rowley
Discussion: https://postgr.es/m/CAKU4AWqWoVii+bRTeBQmeVW+PznkdO8DfbwqNsu9Gj4ubt9A6w@mail.gmail.com
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partprune.c
M src/include/nodes/plannodes.h
Add ORDER BY to some regression test queries
commit : 7e3c54168d9ec058cb3c9d47f8105b1d32dc8ceb
author : Peter Eisentraut <[email protected]>
date : Thu, 8 Apr 2021 12:20:11 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 8 Apr 2021 12:20:11 +0200
Apparently, an unrelated patch introduced some variation on the build
farm.
Reported-by: Magnus Hagander <[email protected]>
M src/test/regress/expected/create_function_3.out
M src/test/regress/sql/create_function_3.sql
Add functions to wait for backend termination
commit : aaf043257205ec523f1ba09a3856464d17cf2281
author : Magnus Hagander <[email protected]>
date : Thu, 8 Apr 2021 11:32:14 +0200
committer: Magnus Hagander <[email protected]>
date : Thu, 8 Apr 2021 11:32:14 +0200
This adds a function, pg_wait_for_backend_termination(), and a new
timeout argument to pg_terminate_backend(), which will wait for the
backend to actually terminate (with or without signaling it to do so
depending on which function is called). The default behaviour of
pg_terminate_backend() remains being timeout=0 which does not waiting.
For pg_wait_for_backend_termination() the default wait is 5 seconds.
Author: Bharath Rupireddy
Reviewed-By: Fujii Masao, David Johnston, Muhammad Usama,
Hou Zhijie, Magnus Hagander
Discussion: https://postgr.es/m/CALj2ACUBpunmyhYZw-kXCYs5NM+h6oG_7Df_Tn4mLmmUQifkqA@mail.gmail.com
M doc/src/sgml/func.sgml
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/storage/ipc/signalfuncs.c
M src/backend/utils/activity/wait_event.c
M src/include/catalog/pg_proc.dat
M src/include/utils/wait_event.h
doc: Prefer explicit JOIN syntax over old implicit syntax in tutorial
commit : fb310f17812e7321599845a29af2f36c7f1191c3
author : Peter Eisentraut <[email protected]>
date : Thu, 8 Apr 2021 10:51:26 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 8 Apr 2021 10:51:26 +0200
Update src/tutorial/basics.source to match.
Author: Jürgen Purtz <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Reviewed-by: "David G. Johnston" <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/query.sgml
M src/tutorial/basics.source
Track identical top vs nested queries independently in pg_stat_statements
commit : 6b4d23feef6e334fb85af077f2857f62ab781848
author : Magnus Hagander <[email protected]>
date : Thu, 8 Apr 2021 10:23:10 +0200
committer: Magnus Hagander <[email protected]>
date : Thu, 8 Apr 2021 10:23:10 +0200
Changing pg_stat_statements.track between 'all' and 'top' would control
if pg_stat_statements tracked just top level statements or also
statements inside functions, but when tracking all it would not
differentiate between the two. Being table to differentiate this is
useful both to track where the actual query is coming from, and to see
if there are differences in executions between the two.
To do this, add a boolean to the hash key indicating if the statement
was top level or not.
Experience from the pg_stat_kcache module shows that in at least some
"reasonable worloads" only <5% of the queries show up both top level and
nested. Based on this, admittedly small, dataset, this patch does not
try to de-duplicate those query *texts*, and will just store one copy
for the top level and one for the nested.
Author: Julien Rohaud
Reviewed-By: Magnus Hagander, Masahiro Ikeda
Discussion: https://postgr.es/m/20201202040516.GA43757@nol
M contrib/pg_stat_statements/Makefile
M contrib/pg_stat_statements/expected/pg_stat_statements.out
A contrib/pg_stat_statements/pg_stat_statements–1.9–1.10.sql
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/pg_stat_statements.control
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
M doc/src/sgml/pgstatstatements.sgml
Update Unicode data to CLDR 39
commit : 2e0e0666790e48cec716d4947f89d067ef53490c
author : Peter Eisentraut <[email protected]>
date : Thu, 8 Apr 2021 08:28:03 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 8 Apr 2021 08:28:03 +0200
M contrib/unaccent/unaccent.rules
M src/Makefile.global.in
Provide ReadRecentBuffer() to re-pin buffers by ID.
commit : 2f27f8c511494cca9e0e9a4eeeb102fa9f193002
author : Thomas Munro <[email protected]>
date : Thu, 8 Apr 2021 17:48:37 +1200
committer: Thomas Munro <[email protected]>
date : Thu, 8 Apr 2021 17:48:37 +1200
If you know the ID of a buffer that recently held a block that you would
like to pin, this function can be used check if it's still there. It
can be used to avoid a second lookup in the buffer mapping table after
PrefetchBuffer() reports a cache hit.
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CA+hUKGJ4VJN8ttxScUFM8dOKX0BrBiboo5uz1cq=AovOddfHpA@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
M src/include/storage/bufmgr.h
autovacuum: handle analyze for partitioned tables
commit : 0827e8af70f4653ba17ed773f123a60eadd9f9c9
author : Alvaro Herrera <[email protected]>
date : Thu, 8 Apr 2021 01:19:36 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 8 Apr 2021 01:19:36 -0400
Previously, autovacuum would completely ignore partitioned tables, which
is not good regarding analyze -- failing to analyze those tables means
poor plans may be chosen. Make autovacuum aware of those tables by
propagating "changes since analyze" counts from the leaf partitions up
the partitioning hierarchy.
This also introduces necessary reloptions support for partitioned tables
(autovacuum_enabled, autovacuum_analyze_scale_factor,
autovacuum_analyze_threshold). It's unclear how best to document this
aspect.
Author: Yuzuko Hosoya <[email protected]>
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Reviewed-by: Tomas Vondra <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/CAKkQ508_PwVgwJyBY=0Lmkz90j8CmWNPUxgHvCUwGhMrouz6UA@mail.gmail.com
M src/backend/access/common/reloptions.c
M src/backend/catalog/system_views.sql
M src/backend/commands/analyze.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/pgstat.c
M src/include/pgstat.h
M src/test/regress/expected/rules.out
Cope with NULL query string in ExecInitParallelPlan().
commit : b3ee4c503872f3d0a5d6a7cbde48815f555af15b
author : Andres Freund <[email protected]>
date : Wed, 7 Apr 2021 22:00:01 -0700
committer: Andres Freund <[email protected]>
date : Wed, 7 Apr 2021 22:00:01 -0700
It's far from clear that this is the right approach - but a good
portion of the buildfarm has been red for a few hours, on the last day
of the CF. And this fixes at least the obvious crash. So let's go with
that for now.
Discussion: https://postgr.es/m/20210407225806.majgznh4lk34hjvu%40alap3.anarazel.de
M src/backend/executor/execParallel.c
Fix typo in jsonfuncs.c.
commit : 8ffb003591ff02f59d92c36a5791307881863146
author : Amit Kapila <[email protected]>
date : Thu, 8 Apr 2021 10:24:00 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 8 Apr 2021 10:24:00 +0530
Author: Tatsuro Yamada
Discussion: https://postgr.es/m/[email protected]_1
M src/backend/utils/adt/jsonfuncs.c
Repair find_inheritance_children with no active snapshot
commit : 4131f755d548f74eba56285dc674f1f26e4ed6b4
author : Alvaro Herrera <[email protected]>
date : Thu, 8 Apr 2021 00:46:14 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 8 Apr 2021 00:46:14 -0400
When working on a scan with only a catalog snapshot, we may not have an
ActiveSnapshot set. If we were to come across a detached partition,
that would cause a crash. Fix by only ignoring detached partitions when
there's an active snapshot.
M src/backend/catalog/pg_inherits.c
Allow psql's \df and \do commands to specify argument types.
commit : a3027e1e7f3d3a107ecd72d3b4d6333ea2aab6a5
author : Tom Lane <[email protected]>
date : Wed, 7 Apr 2021 23:02:16 -0400
committer: Tom Lane <[email protected]>
date : Wed, 7 Apr 2021 23:02:16 -0400
When dealing with overloaded function or operator names, having
to look through a long list of matches is tedious. Let's extend
these commands to allow specification of (input) argument types
to let such results be trimmed down. Each additional argument
is treated the same as the pattern argument of \dT and matched
against the appropriate argument's type name.
While at it, fix \dT (and these new options) to recognize the
usual notation of "foo[]" for "the array type over foo", and
to handle the special abbreviations allowed by the backend
grammar, such as "int" for "integer".
Greg Sabino Mullane, revised rather significantly by me
Discussion: https://postgr.es/m/CAKAnmmLF9Hhu02N+s7uAyLc5J1xZReg72HQUoiKhNiJV3_jACQ@mail.gmail.com
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/bin/psql/help.c
M src/fe_utils/string_utils.c
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
Add csvlog output for the new query_id value
commit : f57a2f5e03054ade221e554c70e628e1ffae1b66
author : Bruce Momjian <[email protected]>
date : Wed, 7 Apr 2021 22:30:30 -0400
committer: Bruce Momjian <[email protected]>
date : Wed, 7 Apr 2021 22:30:30 -0400
This also adjusts the printf format for query id used by log_line_prefix
(%Q).
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Author: Julien Rouhaud, Bruce Momjian
M doc/src/sgml/config.sgml
M doc/src/sgml/file-fdw.sgml
M src/backend/utils/error/elog.c
Teach VACUUM to bypass unnecessary index vacuuming.
commit : 5100010ee4d5c8ef46619dbd1d17090c627e6d0a
author : Peter Geoghegan <[email protected]>
date : Wed, 7 Apr 2021 16:14:54 -0700
committer: Peter Geoghegan <[email protected]>
date : Wed, 7 Apr 2021 16:14:54 -0700
VACUUM has never needed to call ambulkdelete() for each index in cases
where there are precisely zero TIDs in its dead_tuples array by the end
of its first pass over the heap (also its only pass over the heap in
this scenario). Index vacuuming is simply not required when this
happens. Index cleanup will still go ahead, but in practice most calls
to amvacuumcleanup() are usually no-ops when there were zero preceding
ambulkdelete() calls. In short, VACUUM has generally managed to avoid
index scans when there were clearly no index tuples to delete from
indexes. But cases with _close to_ no index tuples to delete were
another matter -- a round of ambulkdelete() calls took place (one per
index), each of which performed a full index scan.
VACUUM now behaves just as if there were zero index tuples to delete in
cases where there are in fact "virtually zero" such tuples. That is, it
can now bypass index vacuuming and heap vacuuming as an optimization
(though not index cleanup). Whether or not VACUUM bypasses indexes is
determined dynamically, based on the just-observed number of heap pages
in the table that have one or more LP_DEAD items (LP_DEAD items in heap
pages have a 1:1 correspondence with index tuples that still need to be
deleted from each index in the worst case).
We only skip index vacuuming when 2% or less of the table's pages have
one or more LP_DEAD items -- bypassing index vacuuming as an
optimization must not noticeably impede setting bits in the visibility
map. As a further condition, the dead_tuples array (i.e. VACUUM's array
of LP_DEAD item TIDs) must not exceed 32MB at the point that the first
pass over the heap finishes, which is also when the decision to bypass
is made. (The VACUUM must also have been able to fit all TIDs in its
maintenance_work_mem-bound dead_tuples space, though with a default
maintenance_work_mem setting it can't matter.)
This avoids surprising jumps in the duration and overhead of routine
vacuuming with workloads where successive VACUUM operations consistently
have almost zero dead index tuples. The number of LP_DEAD items may
well accumulate over multiple VACUUM operations, before finally the
threshold is crossed and VACUUM performs conventional index vacuuming.
Even then, the optimization will have avoided a great deal of largely
unnecessary index vacuuming.
In the future we may teach VACUUM to skip index vacuuming on a per-index
basis, using a much more sophisticated approach. For now we only
consider the extreme cases, where we can be quite confident that index
vacuuming just isn't worth it using simple heuristics.
Also log information about how many heap pages have one or more LP_DEAD
items when autovacuum logging is enabled.
Author: Masahiko Sawada <[email protected]>
Author: Peter Geoghegan <[email protected]>
Discussion: https://postgr.es/m/CAD21AoD0SkE11fMw4jD4RENAwBMcw1wasVnwpJVw3tVqPOQgAw@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WzmkebqPd4MVGuPTOS9bMFvp9MDs5cRTCOsv1rQJ3jCbXw@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Fix regression test failure caused by commit 4f0b0966c8
commit : bc70728693bc2d28db7125e7a24d78ad7612f58c
author : Bruce Momjian <[email protected]>
date : Wed, 7 Apr 2021 18:14:36 -0400
committer: Bruce Momjian <[email protected]>
date : Wed, 7 Apr 2021 18:14:36 -0400
The query originally used was too simple, cause explain_filter() to be
unable to remove JIT output text.
Reported-by: Tom Lane
Author: Julien Rouhaud
M src/test/regress/expected/explain.out
M src/test/regress/sql/explain.sql
Fix some failures with connection tests on Windows hosts
commit : c7578fa64019f27edc31261ea49066a4b2569a6c
author : Michael Paquier <[email protected]>
date : Thu, 8 Apr 2021 06:55:00 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 8 Apr 2021 06:55:00 +0900
The truncation of the log file, that this set of tests relies on to make
sure that a connection attempt matches with its expected backend log
pattern, fails, as reported by buildfarm member fairywren. Instead of a
truncation, do a rotation of the log file and restart the node. This
will ensure that the connection attempt data is unique for each test.
Discussion: https://postgr.es/m/[email protected]
M src/test/perl/PostgresNode.pm
SQL-standard function body
commit : e717a9a18b2e34c9c40e5259ad4d31cd7e420750
author : Peter Eisentraut <[email protected]>
date : Wed, 7 Apr 2021 21:30:08 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 7 Apr 2021 21:30:08 +0200
This adds support for writing CREATE FUNCTION and CREATE PROCEDURE
statements for language SQL with a function body that conforms to the
SQL standard and is portable to other implementations.
Instead of the PostgreSQL-specific AS $$ string literal $$ syntax,
this allows writing out the SQL statements making up the body
unquoted, either as a single statement:
CREATE FUNCTION add(a integer, b integer) RETURNS integer
LANGUAGE SQL
RETURN a + b;
or as a block
CREATE PROCEDURE insert_data(a integer, b integer)
LANGUAGE SQL
BEGIN ATOMIC
INSERT INTO tbl VALUES (a);
INSERT INTO tbl VALUES (b);
END;
The function body is parsed at function definition time and stored as
expression nodes in a new pg_proc column prosqlbody. So at run time,
no further parsing is required.
However, this form does not support polymorphic arguments, because
there is no more parse analysis done at call time.
Dependencies between the function and the objects it uses are fully
tracked.
A new RETURN statement is introduced. This can only be used inside
function bodies. Internally, it is treated much like a SELECT
statement.
psql needs some new intelligence to keep track of function body
boundaries so that it doesn't send off statements when it sees
semicolons that are inside a function body.
Tested-by: Jaime Casanova <[email protected]>
Reviewed-by: Julien Rouhaud <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/create_function.sgml
M doc/src/sgml/ref/create_procedure.sgml
M src/backend/catalog/pg_aggregate.c
M src/backend/catalog/pg_proc.c
M src/backend/commands/aggregatecmds.c
M src/backend/commands/functioncmds.c
M src/backend/commands/typecmds.c
M src/backend/executor/functions.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/tcop/postgres.c
M src/backend/utils/adt/ruleutils.c
M src/bin/pg_dump/pg_dump.c
M src/bin/psql/describe.c
M src/fe_utils/psqlscan.l
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_proc.h
M src/include/commands/defrem.h
M src/include/executor/functions.h
M src/include/fe_utils/psqlscan_int.h
M src/include/nodes/nodes.h
M src/include/nodes/parsenodes.h
M src/include/parser/kwlist.h
M src/include/tcop/tcopprot.h
M src/interfaces/ecpg/preproc/ecpg.addons
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/test/regress/expected/create_function_3.out
M src/test/regress/expected/create_procedure.out
M src/test/regress/sql/create_function_3.sql
M src/test/regress/sql/create_procedure.sql
Add wraparound failsafe to VACUUM.
commit : 1e55e7d1755cefbb44982fbacc7da461fa8684e6
author : Peter Geoghegan <[email protected]>
date : Wed, 7 Apr 2021 12:37:45 -0700
committer: Peter Geoghegan <[email protected]>
date : Wed, 7 Apr 2021 12:37:45 -0700
Add a failsafe mechanism that is triggered by VACUUM when it notices
that the table's relfrozenxid and/or relminmxid are dangerously far in
the past. VACUUM checks the age of the table dynamically, at regular
intervals.
When the failsafe triggers, VACUUM takes extraordinary measures to
finish as quickly as possible so that relfrozenxid and/or relminmxid can
be advanced. VACUUM will stop applying any cost-based delay that may be
in effect. VACUUM will also bypass any further index vacuuming and heap
vacuuming -- it only completes whatever remaining pruning and freezing
is required. Bypassing index/heap vacuuming is enabled by commit
8523492d, which made it possible to dynamically trigger the mechanism
already used within VACUUM when it is run with INDEX_CLEANUP off.
It is expected that the failsafe will almost always trigger within an
autovacuum to prevent wraparound, long after the autovacuum began.
However, the failsafe mechanism can trigger in any VACUUM operation.
Even in a non-aggressive VACUUM, where we're likely to not advance
relfrozenxid, it still seems like a good idea to finish off remaining
pruning and freezing. An aggressive/anti-wraparound VACUUM will be
launched immediately afterwards. Note that the anti-wraparound VACUUM
that follows will itself trigger the failsafe, usually before it even
begins its first (and only) pass over the heap.
The failsafe is controlled by two new GUCs: vacuum_failsafe_age, and
vacuum_multixact_failsafe_age. There are no equivalent reloptions,
since that isn't expected to be useful. The GUCs have rather high
defaults (both default to 1.6 billion), and are expected to generally
only be used to make the failsafe trigger sooner/more frequently.
Author: Masahiko Sawada <[email protected]>
Author: Peter Geoghegan <[email protected]>
Discussion: https://postgr.es/m/CAD21AoD0SkE11fMw4jD4RENAwBMcw1wasVnwpJVw3tVqPOQgAw@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WzmgH3ySGYeC-m-eOBsa2=sDwa292-CFghV4rESYo39FsQ@mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/vacuum.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/commands/vacuum.h
Make use of in-core query id added by commit 5fd9dfa5f5
commit : 4f0b0966c866ae9f0e15d7cc73ccf7ce4e1af84b
author : Bruce Momjian <[email protected]>
date : Wed, 7 Apr 2021 14:03:56 -0400
committer: Bruce Momjian <[email protected]>
date : Wed, 7 Apr 2021 14:03:56 -0400
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
M contrib/pg_stat_statements/pg_stat_statements.c
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/ref/explain.sgml
M src/backend/catalog/system_views.sql
M src/backend/commands/explain.c
M src/backend/executor/execMain.c
M src/backend/executor/execParallel.c
M src/backend/parser/analyze.c
M src/backend/tcop/postgres.c
M src/backend/utils/activity/backend_status.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/error/elog.c
M src/backend/utils/misc/postgresql.conf.sample
M src/backend/utils/misc/queryjumble.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/utils/backend_status.h
M src/test/regress/expected/explain.out
M src/test/regress/expected/rules.out
M src/test/regress/sql/explain.sql
amcheck: fix multiple problems with TOAST pointer validation
commit : ec7ffb8096e8eb90f4c9230f7ba9487f0abe1a9f
author : Robert Haas <[email protected]>
date : Wed, 7 Apr 2021 13:28:35 -0400
committer: Robert Haas <[email protected]>
date : Wed, 7 Apr 2021 13:28:35 -0400
First, don't perform database access while holding a buffer lock.
When checking a heap, we can validate that TOAST pointers are sane by
performing a scan on the TOAST index and looking up the chunks that
correspond to each value ID that appears in a TOAST poiner in the main
table. But, to do that while holding a buffer lock at least risks
causing other backends to wait uninterruptibly, and probably can cause
undetected and uninterruptible deadlocks. So, instead, make a list of
checks to perform while holding the lock, and then perform the checks
after releasing it.
Second, adjust things so that we don't try to follow TOAST pointers
for tuples that are already eligible to be pruned. The TOAST tuples
become eligible for pruning at the same time that the main tuple does,
so trying to check them may lead to spurious reports of corruption,
as observed in the buildfarm. The necessary infrastructure to decide
whether or not the tuple being checked is prunable was added by
commit 3b6c1259f9ca8e21860aaf24ec6735a8e5598ea0, but it wasn't
actually used for its intended purpose prior to this patch.
Mark Dilger, adjusted by me to avoid a memory leak.
Discussion: http://postgr.es/m/[email protected]
M contrib/amcheck/verify_heapam.c
M src/tools/pgindent/typedefs.list
Move pg_stat_statements query jumbling to core.
commit : 5fd9dfa5f50e4906c35133a414ebec5b6d518493
author : Bruce Momjian <[email protected]>
date : Wed, 7 Apr 2021 13:06:47 -0400
committer: Bruce Momjian <[email protected]>
date : Wed, 7 Apr 2021 13:06:47 -0400
Add compute_query_id GUC to control whether a query identifier should be
computed by the core (off by default). It's thefore now possible to
disable core queryid computation and use pg_stat_statements with a
different algorithm to compute the query identifier by using a
third-party module.
To ensure that a single source of query identifier can be used and is
well defined, modules that calculate a query identifier should throw an
error if compute_query_id specified to compute a query id and if a query
idenfitier was already calculated.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/pg_stat_statements.conf
M doc/src/sgml/config.sgml
M doc/src/sgml/pgstatstatements.sgml
M src/backend/parser/analyze.c
M src/backend/tcop/postgres.c
M src/backend/utils/misc/Makefile
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
A src/backend/utils/misc/queryjumble.c
M src/include/parser/analyze.h
M src/include/utils/guc.h
A src/include/utils/queryjumble.h
Remove channel binding requirement from clientcert=verify-full test.
commit : a282ee68a070a8adc6e6d45e8e643769c587ecc3
author : Tom Lane <[email protected]>
date : Wed, 7 Apr 2021 12:50:17 -0400
committer: Tom Lane <[email protected]>
date : Wed, 7 Apr 2021 12:50:17 -0400
This fails on older OpenSSL versions that lack channel binding
support. Since that feature is not essential to this test case,
just remove it, instead of complicating matters. Per buildfarm.
Jacob Champion
Discussion: https://postgr.es/m/[email protected]
M src/test/ssl/t/002_scram.pl
Comment cleanup for a1115fa07.
commit : 0d46771eaaf77ad08555cf34e421234d5943edfa
author : Tom Lane <[email protected]>
date : Wed, 7 Apr 2021 12:21:54 -0400
committer: Tom Lane <[email protected]>
date : Wed, 7 Apr 2021 12:21:54 -0400
Amit Langote
Discussion: https://postgr.es/m/CA+HiwqEcawatEaUh1uTbZMEZTJeLzbroRTz9_X9Z5CFjTWJkhw@mail.gmail.com
M src/backend/executor/nodeModifyTable.c
amcheck: Remove duplicate XID/MXID bounds checks.
commit : 4573f6a9af6e232ba073392716a051ae2017d1e9
author : Robert Haas <[email protected]>
date : Wed, 7 Apr 2021 12:11:44 -0400
committer: Robert Haas <[email protected]>
date : Wed, 7 Apr 2021 12:11:44 -0400
Commit 3b6c1259f9ca8e21860aaf24ec6735a8e5598ea0 resulted in the same
xmin and xmax bounds checking being performed in both check_tuple()
and check_tuple_visibility(). Remove the duplication.
While at it, adjust some code comments that were overlooked in that
commit.
Mark Dilger
Discussion: http://postgr.es/m/[email protected]
M contrib/amcheck/verify_heapam.c
Truncate line pointer array during VACUUM.
commit : 3c3b8a4b26891892bccf3d220580a7f413c0b9ca
author : Peter Geoghegan <[email protected]>
date : Wed, 7 Apr 2021 08:47:15 -0700
committer: Peter Geoghegan <[email protected]>
date : Wed, 7 Apr 2021 08:47:15 -0700
Teach VACUUM to truncate the line pointer array of each heap page when a
contiguous group of LP_UNUSED line pointers appear at the end of the
array -- these unused and unreferenced items are excluded. This process
occurs during VACUUM's second pass over the heap, right after LP_DEAD
line pointers on the page (those encountered/pruned during the first
pass) are marked LP_UNUSED.
Truncation avoids line pointer bloat with certain workloads,
particularly those involving continual range DELETEs and bulk INSERTs
against the same table.
Also harden heapam code to check for an out-of-range page offset number
in places where we weren't already doing so.
Author: Matthias van de Meent <[email protected]>
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Masahiko Sawada <[email protected]>
Reviewed-By: Peter Geoghegan <[email protected]>
Discussion: https://postgr.es/m/CAEze2WjgaQc55Y5f5CQd3L=eS5CZcff2Obxp=O6pto8-f0hC4w@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-Wzn6a64PJM1Ggzm=uvx2otsopJMhFQj_g1rAj4GWr3ZSzw@mail.gmail.com
M src/backend/access/heap/heapam.c
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/storage/page/bufpage.c
M src/include/storage/bufpage.h
Tighten up allowed names for custom GUC parameters.
commit : 3db826bd55cd1df0dd8c3d811f8e5b936d7ba1e4
author : Tom Lane <[email protected]>
date : Wed, 7 Apr 2021 11:22:22 -0400
committer: Tom Lane <[email protected]>
date : Wed, 7 Apr 2021 11:22:22 -0400
Formerly we were pretty lax about what a custom GUC's name could
be; so long as it had at least one dot in it, we'd take it.
However, corner cases such as dashes or equal signs in the name
would cause various bits of functionality to misbehave. Rather
than trying to make the world perfectly safe for that, let's
just require that custom names look like "identifier.identifier",
where "identifier" means something that scan.l would accept
without double quotes.
Along the way, this patch refactors things slightly in guc.c
so that find_option() is responsible for reporting GUC-not-found
cases, allowing removal of duplicative code from its callers.
Per report from Hubert Depesz Lubaczewski. No back-patch,
since the consequences of the problem don't seem to warrant
changing behavior in stable branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/misc/guc-file.l
M src/backend/utils/misc/guc.c
M src/test/regress/expected/guc.out
M src/test/regress/sql/guc.sql
Don't add non-existent pages to bitmap from BRIN
commit : 23607a8156d595522c232ff3933d77041d3adaa1
author : Tomas Vondra <[email protected]>
date : Wed, 7 Apr 2021 15:58:35 +0200
committer: Tomas Vondra <[email protected]>
date : Wed, 7 Apr 2021 15:58:35 +0200
The code in bringetbitmap() simply added the whole matching page range
to the TID bitmap, as determined by pages_per_range, even if some of the
pages were beyond the end of the heap. The query then might fail with
an error like this:
ERROR: could not open file "base/20176/20228.2" (target block
262144): previous segment is only 131021 blocks
In this case, the relation has 262093 pages (131072 and 131021 pages),
but we're trying to acess block 262144, i.e. first block of the 3rd
segment. At that point _mdfd_getseg() notices the preceding segment is
incomplete, and fails.
Hitting this in practice is rather unlikely, because:
* Most indexes use power-of-two ranges, so segments and page ranges
align perfectly (segment end is also a page range end).
* The table size has to be just right, with the last segment being
almost full - less than one page range from full segment, so that the
last page range actually crosses the segment boundary.
* Prefetch has to be enabled. The regular page access checks that
pages are not beyond heap end, but prefetch does not. On older
releases (before 12) the execution stops after hitting the first
non-existent page, so the prefetch distance has to be sufficient
to reach the first page in the next segment to trigger the issue.
Since 12 it's enough to just have prefetch enabled, the prefetch
distance does not matter.
Fixed by not adding non-existent pages to the TID bitmap. Backpatch
all the way back to 9.6 (BRIN indexes were introduced in 9.5, but that
release is EOL).
Backpatch-through: 9.6
M src/backend/access/brin/brin.c
libpq: Set Server Name Indication (SNI) for SSL connections
commit : 5c55dc8b47338e72a4e598c155d2048d756fd10e
author : Peter Eisentraut <[email protected]>
date : Wed, 7 Apr 2021 15:11:41 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 7 Apr 2021 15:11:41 +0200
By default, have libpq set the TLS extension "Server Name Indication" (SNI).
This allows an SNI-aware SSL proxy to route connections. (This
requires a proxy that is aware of the PostgreSQL protocol, not just
any SSL proxy.)
In the future, this could also allow the server to use different SSL
certificates for different host specifications. (That would require
new server functionality. This would be the client-side functionality
for that.)
Since SNI makes the host name appear in cleartext in the network
traffic, this might be undesirable in some cases. Therefore, also add
a libpq connection option "sslsni" to turn it off.
Discussion: https://www.postgresql.org/message-id/flat/7289d5eb-62a5-a732-c3b9-438cee2cb709%40enterprisedb.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/libpq-int.h
Refactor hba_authname
commit : c1968426ba3de1fe37848863e35fff30261bf941
author : Magnus Hagander <[email protected]>
date : Wed, 7 Apr 2021 14:21:19 +0200
committer: Magnus Hagander <[email protected]>
date : Wed, 7 Apr 2021 14:21:19 +0200
The previous implementation (from 9afffcb833) had an unnecessary check
on the boundaries of the enum which trigtered compile warnings. To clean
it up, move the pre-existing static assert to a central location and
call that.
Reported-By: Erik Rijkers
Reviewed-By: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/auth.c
M src/backend/libpq/hba.c
M src/include/libpq/hba.h
doc: Improve wording
commit : 4560e0acdafd57f3ba109b98e15ac047798d960c
author : Peter Eisentraut <[email protected]>
date : Wed, 7 Apr 2021 13:52:26 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 7 Apr 2021 13:52:26 +0200
Discussion: https://www.postgresql.org/message-id/flat/161626776179.652.11944895442156126506%40wrigleys.postgresql.org
M doc/src/sgml/dml.sgml
Revert "Add sortsupport for gist_btree opclasses, for faster index builds."
commit : d92b1cdbab408d8f1299257125c9ae375f3ca644
author : Heikki Linnakangas <[email protected]>
date : Wed, 7 Apr 2021 14:33:21 +0300
committer: Heikki Linnakangas <[email protected]>
date : Wed, 7 Apr 2021 14:33:21 +0300
This reverts commit 9f984ba6d23dc6eecebf479ab1d3f2e550a4e9be.
It was making the buildfarm unhappy, apparently setting client_min_messages
in a regression test produces different output if log_statement='all'.
Another issue is that I now suspect the bit sortsupport function was in
fact not correct to call byteacmp(). Revert to investigate both of those
issues.
M contrib/btree_gist/Makefile
M contrib/btree_gist/btree_bit.c
M contrib/btree_gist/btree_bytea.c
M contrib/btree_gist/btree_cash.c
M contrib/btree_gist/btree_date.c
M contrib/btree_gist/btree_enum.c
M contrib/btree_gist/btree_float4.c
M contrib/btree_gist/btree_float8.c
D contrib/btree_gist/btree_gist–1.6–1.7.sql
M contrib/btree_gist/btree_gist.control
M contrib/btree_gist/btree_gist.h
M contrib/btree_gist/btree_inet.c
M contrib/btree_gist/btree_int2.c
M contrib/btree_gist/btree_int4.c
M contrib/btree_gist/btree_int8.c
M contrib/btree_gist/btree_interval.c
M contrib/btree_gist/btree_macaddr.c
M contrib/btree_gist/btree_macaddr8.c
M contrib/btree_gist/btree_numeric.c
M contrib/btree_gist/btree_oid.c
M contrib/btree_gist/btree_text.c
M contrib/btree_gist/btree_time.c
M contrib/btree_gist/btree_ts.c
M contrib/btree_gist/btree_uuid.c
M contrib/btree_gist/expected/bit.out
M contrib/btree_gist/expected/bytea.out
M contrib/btree_gist/expected/cash.out
M contrib/btree_gist/expected/char.out
M contrib/btree_gist/expected/cidr.out
M contrib/btree_gist/expected/date.out
M contrib/btree_gist/expected/enum.out
M contrib/btree_gist/expected/float4.out
M contrib/btree_gist/expected/float8.out
M contrib/btree_gist/expected/inet.out
M contrib/btree_gist/expected/int2.out
M contrib/btree_gist/expected/int4.out
M contrib/btree_gist/expected/int8.out
M contrib/btree_gist/expected/interval.out
M contrib/btree_gist/expected/macaddr.out
M contrib/btree_gist/expected/macaddr8.out
M contrib/btree_gist/expected/numeric.out
M contrib/btree_gist/expected/oid.out
M contrib/btree_gist/expected/text.out
M contrib/btree_gist/expected/time.out
M contrib/btree_gist/expected/timestamp.out
M contrib/btree_gist/expected/timestamptz.out
M contrib/btree_gist/expected/timetz.out
M contrib/btree_gist/expected/uuid.out
M contrib/btree_gist/expected/varbit.out
M contrib/btree_gist/expected/varchar.out
M contrib/btree_gist/sql/bit.sql
M contrib/btree_gist/sql/bytea.sql
M contrib/btree_gist/sql/cash.sql
M contrib/btree_gist/sql/char.sql
M contrib/btree_gist/sql/cidr.sql
M contrib/btree_gist/sql/date.sql
M contrib/btree_gist/sql/enum.sql
M contrib/btree_gist/sql/float4.sql
M contrib/btree_gist/sql/float8.sql
M contrib/btree_gist/sql/inet.sql
M contrib/btree_gist/sql/int2.sql
M contrib/btree_gist/sql/int4.sql
M contrib/btree_gist/sql/int8.sql
M contrib/btree_gist/sql/interval.sql
M contrib/btree_gist/sql/macaddr.sql
M contrib/btree_gist/sql/macaddr8.sql
M contrib/btree_gist/sql/numeric.sql
M contrib/btree_gist/sql/oid.sql
M contrib/btree_gist/sql/text.sql
M contrib/btree_gist/sql/time.sql
M contrib/btree_gist/sql/timestamp.sql
M contrib/btree_gist/sql/timestamptz.sql
M contrib/btree_gist/sql/timetz.sql
M contrib/btree_gist/sql/uuid.sql
M contrib/btree_gist/sql/varbit.sql
M contrib/btree_gist/sql/varchar.sql
M src/backend/access/gist/gistbuild.c
Add sortsupport for gist_btree opclasses, for faster index builds.
commit : 9f984ba6d23dc6eecebf479ab1d3f2e550a4e9be
author : Heikki Linnakangas <[email protected]>
date : Wed, 7 Apr 2021 13:22:05 +0300
committer: Heikki Linnakangas <[email protected]>
date : Wed, 7 Apr 2021 13:22:05 +0300
Commit 16fa9b2b30 introduced a faster way to build GiST indexes, by
sorting all the data. This commit adds the sortsupport functions needed
to make use of that feature for btree_gist.
Author: Andrey Borodin
Discussion: https://www.postgresql.org/message-id/[email protected]
M contrib/btree_gist/Makefile
M contrib/btree_gist/btree_bit.c
M contrib/btree_gist/btree_bytea.c
M contrib/btree_gist/btree_cash.c
M contrib/btree_gist/btree_date.c
M contrib/btree_gist/btree_enum.c
M contrib/btree_gist/btree_float4.c
M contrib/btree_gist/btree_float8.c
A contrib/btree_gist/btree_gist–1.6–1.7.sql
M contrib/btree_gist/btree_gist.control
M contrib/btree_gist/btree_gist.h
M contrib/btree_gist/btree_inet.c
M contrib/btree_gist/btree_int2.c
M contrib/btree_gist/btree_int4.c
M contrib/btree_gist/btree_int8.c
M contrib/btree_gist/btree_interval.c
M contrib/btree_gist/btree_macaddr.c
M contrib/btree_gist/btree_macaddr8.c
M contrib/btree_gist/btree_numeric.c
M contrib/btree_gist/btree_oid.c
M contrib/btree_gist/btree_text.c
M contrib/btree_gist/btree_time.c
M contrib/btree_gist/btree_ts.c
M contrib/btree_gist/btree_uuid.c
M contrib/btree_gist/expected/bit.out
M contrib/btree_gist/expected/bytea.out
M contrib/btree_gist/expected/cash.out
M contrib/btree_gist/expected/char.out
M contrib/btree_gist/expected/cidr.out
M contrib/btree_gist/expected/date.out
M contrib/btree_gist/expected/enum.out
M contrib/btree_gist/expected/float4.out
M contrib/btree_gist/expected/float8.out
M contrib/btree_gist/expected/inet.out
M contrib/btree_gist/expected/int2.out
M contrib/btree_gist/expected/int4.out
M contrib/btree_gist/expected/int8.out
M contrib/btree_gist/expected/interval.out
M contrib/btree_gist/expected/macaddr.out
M contrib/btree_gist/expected/macaddr8.out
M contrib/btree_gist/expected/numeric.out
M contrib/btree_gist/expected/oid.out
M contrib/btree_gist/expected/text.out
M contrib/btree_gist/expected/time.out
M contrib/btree_gist/expected/timestamp.out
M contrib/btree_gist/expected/timestamptz.out
M contrib/btree_gist/expected/timetz.out
M contrib/btree_gist/expected/uuid.out
M contrib/btree_gist/expected/varbit.out
M contrib/btree_gist/expected/varchar.out
M contrib/btree_gist/sql/bit.sql
M contrib/btree_gist/sql/bytea.sql
M contrib/btree_gist/sql/cash.sql
M contrib/btree_gist/sql/char.sql
M contrib/btree_gist/sql/cidr.sql
M contrib/btree_gist/sql/date.sql
M contrib/btree_gist/sql/enum.sql
M contrib/btree_gist/sql/float4.sql
M contrib/btree_gist/sql/float8.sql
M contrib/btree_gist/sql/inet.sql
M contrib/btree_gist/sql/int2.sql
M contrib/btree_gist/sql/int4.sql
M contrib/btree_gist/sql/int8.sql
M contrib/btree_gist/sql/interval.sql
M contrib/btree_gist/sql/macaddr.sql
M contrib/btree_gist/sql/macaddr8.sql
M contrib/btree_gist/sql/numeric.sql
M contrib/btree_gist/sql/oid.sql
M contrib/btree_gist/sql/text.sql
M contrib/btree_gist/sql/time.sql
M contrib/btree_gist/sql/timestamp.sql
M contrib/btree_gist/sql/timestamptz.sql
M contrib/btree_gist/sql/timetz.sql
M contrib/btree_gist/sql/uuid.sql
M contrib/btree_gist/sql/varbit.sql
M contrib/btree_gist/sql/varchar.sql
M src/backend/access/gist/gistbuild.c
Fix use of cursor sensitivity terminology
commit : dd13ad9d39a1ba41cf329b6fe408b49be57c7b88
author : Peter Eisentraut <[email protected]>
date : Wed, 7 Apr 2021 07:49:27 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 7 Apr 2021 07:49:27 +0200
Documentation and comments in code and tests have been using the terms
sensitive/insensitive cursor incorrectly relative to the SQL standard.
(Cursor sensitivity is only relevant for changes made in the same
transaction as the cursor, not for concurrent changes in other
sessions.) Moreover, some of the behavior of PostgreSQL is incorrect
according to the SQL standard, confusing the issue further. (WHERE
CURRENT OF changes are not visible in insensitive cursors, but they
should be.)
This change corrects the terminology and removes the claim that
sensitive cursors are supported. It also adds a test case that checks
the insensitive behavior in a "correct" way, using a change command
not using WHERE CURRENT OF. Finally, it adds the ASENSITIVE cursor
option to select the default asensitive behavior, per SQL standard.
There are no changes to cursor behavior in this patch.
Discussion: https://www.postgresql.org/message-id/flat/96ee8b30-9889-9e1b-b053-90e10c050e85%40enterprisedb.com
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/ref/declare.sgml
M src/backend/catalog/sql_features.txt
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/bin/psql/tab-complete.c
M src/include/nodes/parsenodes.h
M src/include/parser/kwlist.h
M src/test/regress/expected/portals.out
M src/test/regress/sql/portals.sql
Message improvement
commit : 0b5e8245283eef67e88fb5380836cdc2c743d848
author : Peter Eisentraut <[email protected]>
date : Wed, 7 Apr 2021 07:37:30 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 7 Apr 2021 07:37:30 +0200
The previous wording contained a superfluous comma. Adjust phrasing
for grammatical correctness and clarity.
M contrib/test_decoding/expected/slot.out
M src/backend/replication/logical/logicalfuncs.c
M src/backend/replication/slotfuncs.c
Remove redundant memset(0) calls for page init of some index AMs
commit : 4c0239cb7a7775e3183cb575e62703d71bf3302d
author : Michael Paquier <[email protected]>
date : Wed, 7 Apr 2021 14:35:26 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 7 Apr 2021 14:35:26 +0900
Bloom, GIN, GiST and SP-GiST rely on PageInit() to initialize the
contents of a page, and this routine fills entirely a page with zeros
for a size of BLCKSZ, including the special space. Those index AMs have
been using an extra memset() call to fill with zeros the special page
space, or even the whole page, which is not necessary as PageInit()
already does this work, so let's remove them. GiST was not doing this
extra call, but has commented out a system call that did so since
6236991.
While on it, remove one MAXALIGN() for SP-GiST as PageInit() takes care
of that. This makes the whole page initialization logic more consistent
across all index AMs.
Author: Bharath Rupireddy
Reviewed-by: Vignesh C, Mahendra Singh Thalor
Discussion: https://postgr.es/m/CALj2ACViOo2qyaPT7krWm4LRyRTw9kOXt+g6PfNmYuGA=YHj9A@mail.gmail.com
M contrib/bloom/blinsert.c
M contrib/bloom/blutils.c
M src/backend/access/gin/ginutil.c
M src/backend/access/gist/gistutil.c
M src/backend/access/spgist/spgutils.c
Add some information about authenticated identity via log_connections
commit : 9afffcb833d3c5e59a328a2af674fac7e7334fc1
author : Michael Paquier <[email protected]>
date : Wed, 7 Apr 2021 10:16:39 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 7 Apr 2021 10:16:39 +0900
The "authenticated identity" is the string used by an authentication
method to identify a particular user. In many common cases, this is the
same as the PostgreSQL username, but for some third-party authentication
methods, the identifier in use may be shortened or otherwise translated
(e.g. through pg_ident user mappings) before the server stores it.
To help administrators see who has actually interacted with the system,
this commit adds the capability to store the original identity when
authentication succeeds within the backend's Port, and generates a log
entry when log_connections is enabled. The log entries generated look
something like this (where a local user named "foouser" is connecting to
the database as the database user called "admin"):
LOG: connection received: host=[local]
LOG: connection authenticated: identity="foouser" method=peer (/data/pg_hba.conf:88)
LOG: connection authorized: user=admin database=postgres application_name=psql
Port->authn_id is set according to the authentication method:
bsd: the PostgreSQL username (aka the local username)
cert: the client's Subject DN
gss: the user principal
ident: the remote username
ldap: the final bind DN
pam: the PostgreSQL username (aka PAM username)
password (and all pw-challenge methods): the PostgreSQL username
peer: the peer's pw_name
radius: the PostgreSQL username (aka the RADIUS username)
sspi: either the down-level (SAM-compatible) logon name, if
compat_realm=1, or the User Principal Name if compat_realm=0
The trust auth method does not set an authenticated identity. Neither
does clientcert=verify-full.
Port->authn_id could be used for other purposes, like a superuser-only
extra column in pg_stat_activity, but this is left as future work.
PostgresNode::connect_{ok,fails}() have been modified to let tests check
the backend log files for required or prohibited patterns, using the
new log_like and log_unlike parameters. This uses a method based on a
truncation of the existing server log file, like issues_sql_like().
Tests are added to the ldap, kerberos, authentication and SSL test
suites.
Author: Jacob Champion
Reviewed-by: Stephen Frost, Magnus Hagander, Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M src/backend/libpq/auth.c
M src/backend/libpq/hba.c
M src/include/libpq/hba.h
M src/include/libpq/libpq-be.h
M src/test/authentication/t/001_password.pl
M src/test/kerberos/t/001_auth.pl
M src/test/ldap/t/001_auth.pl
M src/test/perl/PostgresNode.pm
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
Fix test added by commit 9de9294b0c.
commit : 8ee9b662daa6d51b54d21ec274f22a218462ad2d
author : Fujii Masao <[email protected]>
date : Wed, 7 Apr 2021 07:42:36 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 7 Apr 2021 07:42:36 +0900
The buildfarm members "drongo" and "fairywren" reported that
the regression test (024_archive_recovery.pl) added by commit 9de9294b0c
failed. The cause of this failure is that the test calls $node->init()
without "allows_streaming => 1" and which doesn't add pg_hba.conf
entry for TCP/IP connection from pg_basebackup.
This commit fixes the issue by specifying "allows_streaming => 1"
when calling $node->init().
Author: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/024_archive_recovery.pl
Postpone some more stuff out of ExecInitModifyTable.
commit : a1115fa0782378a8238045d238ae70cac36be8ae
author : Tom Lane <[email protected]>
date : Tue, 6 Apr 2021 18:13:05 -0400
committer: Tom Lane <[email protected]>
date : Tue, 6 Apr 2021 18:13:05 -0400
Delay creation of the projections for INSERT and UPDATE tuples
until they're needed. This saves a pretty fair amount of work
when only some of the partitions are actually touched.
The logic associated with identifying junk columns in UPDATE/DELETE
is moved to another loop, allowing removal of one loop over the
target relations; but it didn't actually change at all.
Extracted from a larger patch, which seemed to me to be too messy
to push in one commit.
Amit Langote, reviewed at different times by Heikki Linnakangas and
myself
Discussion: https://postgr.es/m/CA+HiwqG7ZruBmmih3wPsBZ4s0H2EhywrnXEduckY5Hr3fWzPWA@mail.gmail.com
M src/backend/executor/execMain.c
M src/backend/executor/nodeModifyTable.c
M src/include/nodes/execnodes.h
Fix compiler warning for MSVC in libpq_pipeline.c
commit : 3b82d990ab784881153c0f127e4c1211e9b6065c
author : David Rowley <[email protected]>
date : Wed, 7 Apr 2021 09:51:33 +1200
committer: David Rowley <[email protected]>
date : Wed, 7 Apr 2021 09:51:33 +1200
DEBUG was already defined by the MSVC toolchain for "Debug" builds. On
these systems the unconditional #define DEBUG was causing a 'DEBUG': macro
redefinition warning.
Here we rename DEBUG to DEBUG_OUPUT and also get rid of the #define which
defined this constant. This appears to have been left in the code by
mistake.
Discussion: https://postgr.es/m/CAApHDvqTTgDm38s4HRj03nhzhzQ1oMOj-RXFUB1pE6Bj07jyuQ@mail.gmail.com
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Postpone some stuff out of ExecInitModifyTable.
commit : c5b7ba4e67aeb5d6f824b74f94114d99ed6e42b7
author : Tom Lane <[email protected]>
date : Tue, 6 Apr 2021 15:56:55 -0400
committer: Tom Lane <[email protected]>
date : Tue, 6 Apr 2021 15:56:55 -0400
Arrange to do some things on-demand, rather than immediately during
executor startup, because there's a fair chance of never having to do
them at all:
* Don't open result relations' indexes until needed.
* Don't initialize partition tuple routing, nor the child-to-root
tuple conversion map, until needed.
This wins in UPDATEs on partitioned tables when only some of the
partitions will actually receive updates; with larger partition
counts the savings is quite noticeable. Also, we can remove some
sketchy heuristics in ExecInitModifyTable about whether to set up
tuple routing.
Also, remove execPartition.c's private hash table tracking which
partitions were already opened by the ModifyTable node. Instead
use the hash added to ModifyTable itself by commit 86dc90056.
To allow lazy computation of the conversion maps, we now set
ri_RootResultRelInfo in all child ResultRelInfos. We formerly set it
only in some, not terribly well-defined, cases. This has user-visible
side effects in that now more error messages refer to the root
relation instead of some partition (and provide error data in the
root's column order, too). It looks to me like this is a strict
improvement in consistency, so I don't have a problem with the
output changes visible in this commit.
Extracted from a larger patch, which seemed to me to be too messy
to push in one commit.
Amit Langote, reviewed at different times by Heikki Linnakangas and
myself
Discussion: https://postgr.es/m/CA+HiwqG7ZruBmmih3wPsBZ4s0H2EhywrnXEduckY5Hr3fWzPWA@mail.gmail.com
M src/backend/commands/copyfrom.c
M src/backend/commands/trigger.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeModifyTable.c
M src/backend/replication/logical/worker.c
M src/include/executor/execPartition.h
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
M src/test/regress/expected/inherit.out
M src/test/regress/expected/privileges.out
M src/test/regress/expected/update.out
postgres_fdw: Allow partitions specified in LIMIT TO to be imported.
commit : a3740c48eb2f91663c7c06c948dfcfb6493d2588
author : Fujii Masao <[email protected]>
date : Wed, 7 Apr 2021 02:32:10 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 7 Apr 2021 02:32:10 +0900
Commit f49bcd4ef3 disallowed postgres_fdw to import table partitions.
Because all data can be accessed through the partitioned table which
is the root of the partitioning hierarchy, importing only partitioned
table should allow access to all the data without creating extra objects.
This is a reasonable default when importing a whole schema. But there
may be the case where users want to explicitly import one of
a partitioned tables' partitions.
For that use case, this commit allows postgres_fdw to import tables or
foreign tables which are partitions of some other table only when they
are explicitly specified in LIMIT TO clause. It doesn't change
the behavior that any partitions not specified in LIMIT TO are
automatically excluded in IMPORT FOREIGN SCHEMA command.
Author: Matthias van de Meent
Reviewed-by: Bernd Helmle, Amit Langote, Michael Paquier, Fujii Masao
Discussion: https://postgr.es/m/CAEze2Whwg4i=mzApMe+PXxCEfgoZmHGqdqQFW7J4bmj_5p6t1A@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Increment xactCompletionCount during subtransaction abort.
commit : 90c885cdab8bc5a5f12a243774fa0db51002a2fd
author : Andres Freund <[email protected]>
date : Tue, 6 Apr 2021 09:24:50 -0700
committer: Andres Freund <[email protected]>
date : Tue, 6 Apr 2021 09:24:50 -0700
Snapshot caching, introduced in 623a9ba79b, did not increment
xactCompletionCount during subtransaction abort. That could lead to an older
snapshot being reused. That is, at least as far as I can see, not a
correctness issue (for MVCC snapshots there's no difference between "in
progress" and "aborted"). The only difference between the old and new
snapshots would be a newer ->xmax.
While HeapTupleSatisfiesMVCC makes the same visibility determination, reusing
the old snapshot leads HeapTupleSatisfiesMVCC to not set
HEAP_XMIN_INVALID. Which subsequently causes the kill_prior_tuple optimization
to not kick in (via HeapTupleIsSurelyDead() returning false). The performance
effects of doing the same index-lookups over and over again is how the issue
was discovered...
Fix the issue by incrementing xactCompletionCount in
XidCacheRemoveRunningXids. It already acquires ProcArrayLock exclusively,
making that an easy proposition.
Add a test to ensure that kill_prior_tuple prevents index growth when it
involves aborted subtransaction of the current transaction.
Author: Andres Freund
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/20210317055718.v6qs3ltzrformqoa%40alap3.anarazel.de
M src/backend/storage/ipc/procarray.c
A src/test/regress/expected/mvcc.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/mvcc.sql
Remove tupgone special case from vacuumlazy.c.
commit : 8523492d4e349c4714aa2ab0291be175a88cb4fc
author : Peter Geoghegan <[email protected]>
date : Tue, 6 Apr 2021 08:49:22 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 6 Apr 2021 08:49:22 -0700
Retry the call to heap_prune_page() in rare cases where there is
disagreement between the heap_prune_page() call and the call to
HeapTupleSatisfiesVacuum() that immediately follows. Disagreement is
possible when a concurrently-aborted transaction makes a tuple DEAD
during the tiny window between each step. This was the only case where
a tuple considered DEAD by VACUUM still had storage following pruning.
VACUUM's definition of dead tuples is now uniformly simple and
unambiguous: dead tuples from each page are always LP_DEAD line pointers
that were encountered just after we performed pruning (and just before
we considered freezing remaining items with tuple storage).
Eliminating the tupgone=true special case enables INDEX_CLEANUP=off
style skipping of index vacuuming that takes place based on flexible,
dynamic criteria. The INDEX_CLEANUP=off case had to know about skipping
indexes up-front before now, due to a subtle interaction with the
special case (see commit dd695979) -- this was a special case unto
itself. Now there are no special cases. And so now it won't matter
when or how we decide to skip index vacuuming: it won't affect how
pruning behaves, and it won't be affected by any of the implementation
details of pruning or freezing.
Also remove XLOG_HEAP2_CLEANUP_INFO records. These are no longer
necessary because we now rely entirely on heap pruning taking care of
recovery conflicts. There is no longer any need to generate recovery
conflicts for DEAD tuples that pruning just missed. This also means
that heap vacuuming now uses exactly the same strategy for recovery
conflicts as index vacuuming always has: REDO routines never need to
process a latestRemovedXid from the WAL record, since earlier REDO of
the WAL record from pruning is sufficient in all cases. The generic
XLOG_HEAP2_CLEAN record type is now split into two new record types to
reflect this new division (these are called XLOG_HEAP2_PRUNE and
XLOG_HEAP2_VACUUM).
Also stop acquiring a super-exclusive lock for heap pages when they're
vacuumed during VACUUM's second heap pass. A regular exclusive lock is
enough. This is correct because heap page vacuuming is now strictly a
matter of setting the LP_DEAD line pointers to LP_UNUSED. No other
backend can have a pointer to a tuple located in a pinned buffer that
can be invalidated by a concurrent heap page vacuum operation.
Heap vacuuming can now be thought of as conceptually similar to index
vacuuming and conceptually dissimilar to heap pruning. Heap pruning now
has sole responsibility for anything involving the logical contents of
the database (e.g., managing transaction status information, recovery
conflicts, considering what to do with HOT chains). Index vacuuming and
heap vacuuming are now only concerned with recycling garbage items from
physical data structures that back the logical database.
Bump XLOG_PAGE_MAGIC due to pruning and heap page vacuum WAL record
changes.
Credit for the idea of retrying pruning a page to avoid the tupgone case
goes to Andres Freund.
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Andres Freund <[email protected]>
Reviewed-By: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/CAH2-WznneCXTzuFmcwx_EyRQgfsfJAAsu+CsqRFmFXCAar=nJw@mail.gmail.com
M src/backend/access/gist/gistxlog.c
M src/backend/access/hash/hash_xlog.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/rmgrdesc/heapdesc.c
M src/backend/replication/logical/decode.c
M src/include/access/heapam.h
M src/include/access/heapam_xlog.h
M src/include/access/xlog_internal.h
M src/tools/pgindent/typedefs.list
Fix missing #include in nodeResultCache.h.
commit : 789d81de8a50d9a23cc1a3b8ea5d839246020689
author : Tom Lane <[email protected]>
date : Tue, 6 Apr 2021 11:23:48 -0400
committer: Tom Lane <[email protected]>
date : Tue, 6 Apr 2021 11:23:48 -0400
Per cpluspluscheck.
M src/backend/executor/nodeResultCache.c
M src/include/executor/nodeResultCache.h
psql: Show all query results by default
commit : 3a5130672296ed4e682403a77a9a3ad3d21cef75
author : Peter Eisentraut <[email protected]>
date : Tue, 6 Apr 2021 16:58:10 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 6 Apr 2021 16:58:10 +0200
Previously, psql printed only the last result if a command string
returned multiple result sets. Now it prints all of them. The
previous behavior can be obtained by setting the psql variable
SHOW_ALL_RESULTS to off.
Author: Fabien COELHO <[email protected]>
Reviewed-by: "Iwata, Aya" <[email protected]>
Reviewed-by: Daniel Verite <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Reviewed-by: vignesh C <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/alpine.DEB.2.21.1904132231510.8961@lancre
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/common.c
M src/bin/psql/help.c
M src/bin/psql/settings.h
M src/bin/psql/startup.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/copyselect.out
M src/test/regress/expected/psql.out
M src/test/regress/expected/transactions.out
M src/test/regress/sql/copyselect.sql
M src/test/regress/sql/psql.sql
M src/test/regress/sql/transactions.sql
Fix handling of clauses incompatible with extended statistics
commit : 518442c7f334f3b05ea28b7ef50f1b551cfcc23e
author : Tomas Vondra <[email protected]>
date : Tue, 6 Apr 2021 16:12:37 +0200
committer: Tomas Vondra <[email protected]>
date : Tue, 6 Apr 2021 16:12:37 +0200
Handling of incompatible clauses while applying extended statistics was
a bit confused - while handling a mix of compatible and incompatible
clauses it sometimes incorrectly treated the incompatible clauses as
compatible, resulting in a crash.
Fixed by reworking the code applying the selected statistics object to
make it easier to understand, and adding a proper compatibility check.
Reported-by: David Rowley
Discussion: https://postgr.es/m/CAApHDvpYT10-nkSp8xXe-nbO3jmoaRyRFHbzh-RWMfAJynqgpQ%40mail.gmail.com
M src/backend/statistics/extended_stats.c
M src/backend/statistics/mcv.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Refactor lazy_scan_heap() loop.
commit : 7ab96cf6b312cfcd79cdc1a69c6bdb75de0ed30f
author : Peter Geoghegan <[email protected]>
date : Tue, 6 Apr 2021 07:49:39 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 6 Apr 2021 07:49:39 -0700
Add a lazy_scan_heap() subsidiary function that handles heap pruning and
tuple freezing: lazy_scan_prune(). This is a great deal cleaner. The
code that remains in lazy_scan_heap()'s per-block loop can now be
thought of as code that either comes before or after the call to
lazy_scan_prune(), which is now the clear focal point. This division is
enforced by the way in which we now manage state. lazy_scan_prune()
outputs state (using its own struct) that describes what to do with the
page following pruning and freezing (e.g., visibility map maintenance,
recording free space in the FSM). It doesn't get passed any special
instructional state from the preamble code, though.
Also cleanly separate the logic used by a VACUUM with INDEX_CLEANUP=off
from the logic used by single-heap-pass VACUUMs. The former case is now
structured as the omission of index and heap vacuuming by a two pass
VACUUM. The latter case goes back to being used only when the table
happens to have no indexes (just as it was before commit a96c41fe).
This structure is much more natural, since the whole point of
INDEX_CLEANUP=off is to skip the index and heap vacuuming that would
otherwise take place. The single-heap-pass case doesn't skip any useful
work, though -- it just does heap pruning and heap vacuuming together
when the table happens to have no indexes.
Both of these changes are preparation for an upcoming patch that
generalizes the mechanism used by INDEX_CLEANUP=off. The later patch
will allow VACUUM to give up on index and heap vacuuming dynamically, as
problems emerge (e.g., with wraparound), so that an affected VACUUM
operation can finish up as soon as possible.
Also fix a very old bug in single-pass VACUUM VERBOSE output. We were
reporting the number of tuples deleted via pruning as a direct
substitute for reporting the number of LP_DEAD items removed in a
function that deals with the second pass over the heap. But that
doesn't work at all -- they're two different things.
To fix, start tracking the total number of LP_DEAD items encountered
during pruning, and use that in the report instead. A single pass
VACUUM will always vacuum away whatever LP_DEAD items a heap page has
immediately after it is pruned, so the total number of LP_DEAD items
encountered during pruning equals the total number vacuumed-away.
(They are _not_ equal in the INDEX_CLEANUP=off case, but that's okay
because skipping index vacuuming is now a totally orthogonal concept to
one-pass VACUUM.)
Also stop reporting the count of LP_UNUSED items in VACUUM VERBOSE
output. This makes the output of VACUUM VERBOSE more consistent with
log_autovacuum's output (because it never showed information about
LP_UNUSED items). VACUUM VERBOSE reported LP_UNUSED items left behind
by the last VACUUM, and LP_UNUSED items created via pruning HOT chains
during the current VACUUM (it never included LP_UNUSED items left behind
by the current VACUUM's second pass over the heap). This makes it
useless as an indicator of line pointer bloat, which must have been the
original intention. (Like the first VACUUM VERBOSE issue, this issue was
arguably an oversight in commit 282d2a03, which added the heap-only
tuple optimization.)
Finally, stop reporting empty_pages in VACUUM VERBOSE output, and start
reporting pages_removed instead. This also makes the output of VACUUM
VERBOSE more consistent with log_autovacuum's output (which does not
show empty_pages, but does show pages_removed). An empty page isn't
meaningfully different to a page that is almost empty, or a page that is
empty but for only a small number of remaining LP_UNUSED items.
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Robert Haas <[email protected]>
Reviewed-By: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/CAH2-WznneCXTzuFmcwx_EyRQgfsfJAAsu+CsqRFmFXCAar=nJw@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Clean up treatment of missing default and CHECK-constraint records.
commit : 091e22b2e673e3e8480abd68fbb827c5d6979615
author : Tom Lane <[email protected]>
date : Tue, 6 Apr 2021 10:34:37 -0400
committer: Tom Lane <[email protected]>
date : Tue, 6 Apr 2021 10:34:37 -0400
Andrew Gierth reported that it's possible to crash the backend if no
pg_attrdef record is found to match an attribute that has atthasdef set.
AttrDefaultFetch warns about this situation, but then leaves behind
a relation tupdesc that has null "adbin" pointer(s), which most places
don't guard against.
We considered promoting the warning to an error, but throwing errors
during relcache load is pretty drastic: it effectively locks one out
of using the relation at all. What seems better is to leave the
load-time behavior as a warning, but then throw an error in any code
path that wants to use a default and can't find it. This confines
the error to a subset of INSERT/UPDATE operations on the table, and
in particular will at least allow a pg_dump to succeed.
Also, we should fix AttrDefaultFetch to not leave any null pointers
in the tupdesc, because that just creates an untested bug hazard.
While at it, apply the same philosophy of "warn at load, throw error
only upon use of the known-missing info" to CHECK constraints.
CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch,
but for reasons lost in the mists of time, it was throwing ERROR for
the same cases that AttrDefaultFetch treats as WARNING. Make the two
functions more nearly alike.
In passing, get rid of potentially-O(N^2) loops in equalTupleDesc
by making AttrDefaultFetch sort the entries after fetching them,
so that equalTupleDesc can assume that entries in two equal tupdescs
must be in matching order. (CheckConstraintFetch already was sorting
CHECK constraints, but equalTupleDesc hadn't been told about it.)
There's some argument for back-patching this, but with such a small
number of field reports, I'm content to fix it in HEAD.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/common/tupdesc.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execMain.c
M src/backend/parser/parse_utilcmd.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/utils/cache/relcache.c
Stop archive recovery if WAL generated with wal_level=minimal is found.
commit : 9de9294b0c4dac77edb80f029648afca79d14653
author : Fujii Masao <[email protected]>
date : Tue, 6 Apr 2021 22:56:51 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 6 Apr 2021 22:56:51 +0900
Previously if hot standby was enabled, archive recovery exited with
an error when it found WAL generated with wal_level=minimal.
But if hot standby was disabled, it just reported a warning and
continued in that case. Which could lead to data loss or errors
during normal operation. A warning was emitted, but users could
easily miss that and not notice this serious situation until
they encountered the actual errors.
To improve this situation, this commit changes archive recovery
so that it exits with FATAL error when it finds WAL generated with
wal_level=minimal whatever the setting of hot standby. This enables
users to notice the serious situation soon.
The FATAL error is thrown if archive recovery starts from a base
backup taken before wal_level is changed to minimal. When archive
recovery exits with the error, if users have a base backup taken
after setting wal_level to higher than minimal, they can recover
the database by starting archive recovery from that newer backup.
But note that if such backup doesn't exist, there is no easy way to
complete archive recovery, which may make the database server
unstartable and users may lose whole database. The commit adds
the note about this risk into the document.
Even in the case of unstartable database server, previously by just
disabling hot standby users could avoid the error during archive
recovery, forcibly start up the server and salvage data from it.
But note that this commit makes this procedure unavailable at all.
Author: Takamichi Osumi
Reviewed-by: Laurenz Albe, Kyotaro Horiguchi, David Steele, Fujii Masao
Discussion: https://postgr.es/m/OSBPR01MB4888CBE1DA08818FD2D90ED8EDF90@OSBPR01MB4888.jpnprd01.prod.outlook.com
M doc/src/sgml/config.sgml
M doc/src/sgml/perform.sgml
M src/backend/access/transam/xlog.c
A src/test/recovery/t/024_archive_recovery.pl
Mark test_enc_conversion() as STRICT.
commit : c4c393b3ec83ceb4b4d7f37cdd5302126377d069
author : Heikki Linnakangas <[email protected]>
date : Tue, 6 Apr 2021 14:53:56 +0300
committer: Heikki Linnakangas <[email protected]>
date : Tue, 6 Apr 2021 14:53:56 +0300
Reported-by: Jaime Casanova, using SQLsmith
Discussion: https://www.postgresql.org/message-id/20210402235337.GA4082@ahch-to
M src/test/regress/input/create_function_1.source
M src/test/regress/output/create_function_1.source
pgbench: Function to generate random permutations.
commit : 6b258e3d688db14aadb58dde2a72939362310684
author : Dean Rasheed <[email protected]>
date : Tue, 6 Apr 2021 11:50:42 +0100
committer: Dean Rasheed <[email protected]>
date : Tue, 6 Apr 2021 11:50:42 +0100
This adds a new function, permute(), that generates pseudorandom
permutations of arbitrary sizes. This can be used to randomly shuffle
a set of values to remove unwanted correlations. For example,
permuting the output from a non-uniform random distribution so that
all the most common values aren't collocated, allowing more realistic
tests to be performed.
Formerly, hash() was recommended for this purpose, but that suffers
from collisions that might alter the distribution, so recommend
permute() for this purpose instead.
Fabien Coelho and Hironobu Suzuki, with additional hacking be me.
Reviewed by Thomas Munro, Alvaro Herrera and Muhammad Usama.
Discussion: https://postgr.es/m/alpine.DEB.2.21.1807280944370.5142@lancre
M doc/src/sgml/ref/pgbench.sgml
M src/bin/pgbench/exprparse.y
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/pgbench.h
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/pgbench/t/002_pgbench_no_server.pl
Adjust input value to WaitEventSetWait() in ExecAppendAsyncEventWait().
commit : a8af856d3257138590788e40eb84049def147acf
author : Etsuro Fujita <[email protected]>
date : Tue, 6 Apr 2021 19:15:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Tue, 6 Apr 2021 19:15:00 +0900
Adjust the number of events given to WaitEventSetWait() so that it
doesn't exceed the maximum number of events in the WaitEventSet given
to that function (set->nevents_space) in hopes of making the buildfarm
green.
Per valgrind failure report from Tom Lane and the buildfarm.
Author: Etsuro Fujita
Discussion: https://postgr.es/m/3411577.1617289776%40sss.pgh.pa.us
M src/backend/executor/nodeAppend.c
ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION
commit : 82ed7748b710e3ddce3f7ebc74af80fe4869492f
author : Peter Eisentraut <[email protected]>
date : Tue, 6 Apr 2021 10:44:26 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 6 Apr 2021 10:44:26 +0200
At present, if we want to update publications in a subscription, we
can use SET PUBLICATION. However, it requires supplying all
publications that exists and the new publications. If we want to add
new publications, it's inconvenient. The new syntax only supplies the
new publications. When the refresh is true, it only refreshes the new
publications.
Author: Japin Li <[email protected]>
Author: Bharath Rupireddy <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/MEYP282MB166939D0D6C480B7FBE7EFFBB6BC0@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
M doc/src/sgml/ref/alter_subscription.sgml
M src/backend/commands/subscriptioncmds.c
M src/backend/parser/gram.y
M src/bin/psql/tab-complete.c
M src/include/nodes/parsenodes.h
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
Fix the tests added by commit ac4645c015.
commit : 266b5673b4b6bed2e9ebfe73ca967f44d6dc0e6c
author : Amit Kapila <[email protected]>
date : Tue, 6 Apr 2021 14:58:52 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 6 Apr 2021 14:58:52 +0530
In the tests, after disabling the subscription, we were not waiting for
the replication connection to drop from the publisher. So when the test
was trying to use the same slot to fetch the messages via SQL API, it
sometimes gives an error that the replication slot is active for other
PID.
Per buildfarm.
M src/test/subscription/t/020_messages.pl
Fix compiler warning in fe-trace.c for MSVC
commit : 9bc9b4609a246ded5caf3f3d4c0013a002ba2323
author : David Rowley <[email protected]>
date : Tue, 6 Apr 2021 18:33:40 +1200
committer: David Rowley <[email protected]>
date : Tue, 6 Apr 2021 18:33:40 +1200
It seems that in MSVC timeval's tv_sec field is of type long.
localtime() takes a time_t pointer. Since long is 32-bit even on 64-bit
builds in MSVC, passing a long pointer instead of the correct time_t
pointer generated a compiler warning. Fix that.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/CAApHDvoRG25X_=ZCGSPb4KN_j2iu=G2uXsRSg8NBZeuhkOSETg@mail.gmail.com
M src/interfaces/libpq/fe-trace.c
Change return type of EXTRACT to numeric
commit : a2da77cdb4661826482ebf2ddba1f953bc74afe4
author : Peter Eisentraut <[email protected]>
date : Tue, 6 Apr 2021 07:17:13 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 6 Apr 2021 07:17:13 +0200
The previous implementation of EXTRACT mapped internally to
date_part(), which returned type double precision (since it was
implemented long before the numeric type existed). This can lead to
imprecise output in some cases, so returning numeric would be
preferrable. Changing the return type of an existing function is a
bit risky, so instead we do the following: We implement a new set of
functions, which are now called "extract", in parallel to the existing
date_part functions. They work the same way internally but use
numeric instead of float8. The EXTRACT construct is now mapped by the
parser to these new extract functions. That way, dumps of views
etc. from old versions (which would use date_part) continue to work
unchanged, but new uses will map to the new extract functions.
Additionally, the reverse compilation of EXTRACT now reproduces the
original syntax, using the new mechanism introduced in
40c24bfef92530bd846e111c1742c2a54441c62c.
The following minor changes of behavior result from the new
implementation:
- The column name from an isolated EXTRACT call is now "extract"
instead of "date_part".
- Extract from date now rejects inappropriate field names such as
HOUR. It was previously mapped internally to extract from
timestamp, so it would silently accept everything appropriate for
timestamp.
- Return values when extracting fields with possibly fractional
values, such as second and epoch, now have the full scale that the
value has internally (so, for example, '1.000000' instead of just
'1').
Reported-by: Petr Fedorov <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/func.sgml
M src/backend/parser/gram.y
M src/backend/utils/adt/date.c
M src/backend/utils/adt/numeric.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/adt/timestamp.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/utils/numeric.h
M src/test/regress/expected/create_view.out
M src/test/regress/expected/date.out
M src/test/regress/expected/interval.out
M src/test/regress/expected/psql_crosstab.out
M src/test/regress/expected/time.out
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/expected/timetz.out
M src/test/regress/sql/date.sql
M src/test/regress/sql/interval.sql
M src/test/regress/sql/time.sql
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
M src/test/regress/sql/timetz.sql
Fix typo in pgstat.c.
commit : f5d94e405e17a49487672316610630be2f9d0bb7
author : Fujii Masao <[email protected]>
date : Tue, 6 Apr 2021 14:09:40 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 6 Apr 2021 14:09:40 +0900
Introduced by 9868167500.
Author: Vignesh C
Discussion: https://postgr.es/m/CALDaNm1DqgaLBAJrtGznKk1sR1mH-augmp7LfGvxWwTUhah+rg@mail.gmail.com
M src/backend/postmaster/pgstat.c
Add function to log the memory contexts of specified backend process.
commit : 43620e328617c1f41a2a54c8cee01723064e3ffa
author : Fujii Masao <[email protected]>
date : Tue, 6 Apr 2021 13:44:15 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 6 Apr 2021 13:44:15 +0900
Commit 3e98c0bafb added pg_backend_memory_contexts view to display
the memory contexts of the backend process. However its target process
is limited to the backend that is accessing to the view. So this is
not so convenient when investigating the local memory bloat of other
backend process. To improve this situation, this commit adds
pg_log_backend_memory_contexts() function that requests to log
the memory contexts of the specified backend process.
This information can be also collected by calling
MemoryContextStats(TopMemoryContext) via a debugger. But
this technique cannot be used in some environments because no debugger
is available there. So, pg_log_backend_memory_contexts() allows us to
see the memory contexts of specified backend more easily.
Only superusers are allowed to request to log the memory contexts
because allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its memory contexts at LOG_SERVER_ONLY level,
so that these memory contexts will appear in the server log but not
be sent to the client. It logs one message per memory context.
Because if it buffers all memory contexts into StringInfo to log them
as one message, which may require the buffer to be enlarged very much
and lead to OOM error since there can be a large number of memory
contexts in a backend.
When a backend process is consuming huge memory, logging all its
memory contexts might overrun available disk space. To prevent this,
now this patch limits the number of child contexts to log per parent
to 100. As with MemoryContextStats(), it supposes that practical cases
where the log gets long will typically be huge numbers of siblings
under the same parent context; while the additional debugging value
from seeing details about individual siblings beyond 100 will not be large.
There was another proposed patch to add the function to return
the memory contexts of specified backend as the result sets,
instead of logging them, in the discussion. However that patch is
not included in this commit because it had several issues to address.
Thanks to Tatsuhito Kasahara, Andres Freund, Tom Lane, Tomas Vondra,
Michael Paquier, Kyotaro Horiguchi and Zhihong Yu for the discussion.
Bump catalog version.
Author: Atsushi Torikoshi
Reviewed-by: Kyotaro Horiguchi, Zhihong Yu, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/storage/ipc/procsignal.c
M src/backend/tcop/postgres.c
M src/backend/utils/adt/mcxtfuncs.c
M src/backend/utils/init/globals.c
M src/backend/utils/mmgr/aset.c
M src/backend/utils/mmgr/generation.c
M src/backend/utils/mmgr/mcxt.c
M src/backend/utils/mmgr/slab.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/miscadmin.h
M src/include/nodes/memnodes.h
M src/include/storage/procsignal.h
M src/include/utils/memutils.h
M src/test/regress/expected/misc_functions.out
M src/test/regress/sql/misc_functions.sql
Fix some issues with SSL and Kerberos tests
commit : 5a71964a832febfee23cedc3bb354049d6ca78a7
author : Michael Paquier <[email protected]>
date : Tue, 6 Apr 2021 13:23:57 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 6 Apr 2021 13:23:57 +0900
The recent refactoring done in c50624c accidentally broke a portion of
the kerberos tests checking after a query, so add its functionality
back. Some inactive SSL tests had their arguments in an incorrect
order, which would cause them to fail if they were to run.
Author: Jacob Champion
Discussion: https://postgr.es/m/[email protected]
M src/test/kerberos/t/001_auth.pl
M src/test/ssl/t/001_ssltests.pl
Allow pgoutput to send logical decoding messages.
commit : ac4645c0157fc5fcef0af8ff571512aa284a2cec
author : Amit Kapila <[email protected]>
date : Tue, 6 Apr 2021 08:40:47 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 6 Apr 2021 08:40:47 +0530
The output plugin accepts a new parameter (messages) that controls if
logical decoding messages are written into the replication stream. It is
useful for those clients that use pgoutput as an output plugin and needs
to process messages that were written by pg_logical_emit_message().
Although logical streaming replication protocol supports logical
decoding messages now, logical replication does not use this feature yet.
Author: David Pirotte, Euler Taveira
Reviewed-by: Euler Taveira, Andres Freund, Ashutosh Bapat, Amit Kapila
Discussion: https://postgr.es/m/CADK3HHJ-+9SO7KuRLH=9Wa1rAo60Yreq1GFNkH_kd0=CdaWM+A@mail.gmail.com
M doc/src/sgml/protocol.sgml
M src/backend/replication/logical/proto.c
M src/backend/replication/logical/worker.c
M src/backend/replication/pgoutput/pgoutput.c
M src/include/replication/logicalproto.h
M src/include/replication/pgoutput.h
A src/test/subscription/t/020_messages.pl
Refactor function parse_output_parameters.
commit : 531737ddad214cb8a675953208e2f3a6b1be122b
author : Amit Kapila <[email protected]>
date : Tue, 6 Apr 2021 08:26:31 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 6 Apr 2021 08:26:31 +0530
Instead of using multiple parameters in parse_ouput_parameters function
signature, use the struct PGOutputData that encapsulates all pgoutput
options. It will be useful for future work where we need to add other
options in pgoutput.
Author: Euler Taveira
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CADK3HHJ-+9SO7KuRLH=9Wa1rAo60Yreq1GFNkH_kd0=CdaWM+A@mail.gmail.com
M src/backend/replication/pgoutput/pgoutput.c
M src/include/replication/pgoutput.h
Change PostgresNode::connect_fails() to never send down queries
commit : 6d41dd045ada28ee14182112fc4cf50fb3879d28
author : Michael Paquier <[email protected]>
date : Tue, 6 Apr 2021 09:53:06 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 6 Apr 2021 09:53:06 +0900
This type of failure is similar to what has been fixed in c757a3da,
where an authentication failure combined with psql pushing a command
down its communication pipe causes a test failure. This routine is
designed to fail, so sending a query has little sense anyway.
Per buildfarm members gaur and hoverfly, based on an analysis and fix
from Tom Lane.
Discussion: https://postgr.es/m/[email protected]
M src/test/perl/PostgresNode.pm
Allocate access strategy in parallel VACUUM workers.
commit : f6b8f19a084ce949522fcbc940dc116c034cfc47
author : Peter Geoghegan <[email protected]>
date : Mon, 5 Apr 2021 17:17:40 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 5 Apr 2021 17:17:40 -0700
Commit 49f49def took entirely the wrong approach to fixing this issue.
Just allocate a local buffer access strategy in each individual worker
instead of trying to propagate state. This state was never propagated
by parallel VACUUM in the first place.
It looks like the only reason that this worked following commit 40d964ec
was that it involved static global variables, which are initialized to 0
per the C standard.
A more comprehensive fix may be necessary, even on HEAD. This fix
should at least get the buildfarm green once again.
Thanks once again to Thomas Munro for continued off-list assistance with
the issue.
M src/backend/access/heap/vacuumlazy.c
Support INCLUDE'd columns in SP-GiST.
commit : 09c1c6ab4bc5764dd69c53ccfd43b2060b1fd090
author : Tom Lane <[email protected]>
date : Mon, 5 Apr 2021 18:41:09 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Apr 2021 18:41:09 -0400
Not much to say here: does what it says on the tin.
We steal a previously-always-zero bit from the nextOffset
field of leaf index tuples in order to track whether there
is a nulls bitmap. Otherwise it works about like included
columns in other index types.
Pavel Borisov, reviewed by Andrey Borodin and Anastasia Lubennikova,
and rather heavily editorialized on by me
Discussion: https://postgr.es/m/CALT9ZEFi-vMp4faht9f9Junb1nO3NOSjhpxTmbm1UGLMsLqiEQ@mail.gmail.com
M doc/src/sgml/indices.sgml
M doc/src/sgml/ref/create_index.sgml
M doc/src/sgml/spgist.sgml
M doc/src/sgml/xindex.sgml
M src/backend/access/common/indextuple.c
M src/backend/access/spgist/README
M src/backend/access/spgist/spgdoinsert.c
M src/backend/access/spgist/spginsert.c
M src/backend/access/spgist/spgscan.c
M src/backend/access/spgist/spgutils.c
M src/backend/access/spgist/spgvacuum.c
M src/backend/access/spgist/spgxlog.c
M src/include/access/itup.h
M src/include/access/spgist_private.h
M src/test/modules/spgist_name_ops/expected/spgist_name_ops.out
M src/test/modules/spgist_name_ops/sql/spgist_name_ops.sql
M src/test/regress/expected/amutils.out
M src/test/regress/expected/create_index_spgist.out
M src/test/regress/expected/index_including.out
M src/test/regress/sql/create_index_spgist.sql
M src/test/regress/sql/index_including.sql
Propagate parallel VACUUM's buffer access strategy.
commit : 49f49defe7c0a330cca084de5da14ccdfdafc6a3
author : Peter Geoghegan <[email protected]>
date : Mon, 5 Apr 2021 14:56:56 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 5 Apr 2021 14:56:56 -0700
Parallel VACUUM relied on global variable state from the leader process
being propagated to workers on fork(). Commit b4af70cb removed most
uses of global variables inside vacuumlazy.c, but did not account for
the buffer access strategy state.
To fix, propagate the state through shared memory instead.
Per buildfarm failures on elver, curculio, and morepork.
Many thanks to Thomas Munro for off-list assistance with this issue.
M src/backend/access/heap/vacuumlazy.c
Simplify state managed by VACUUM.
commit : b4af70cb210393c9c8f41643acf6b213e21178e7
author : Peter Geoghegan <[email protected]>
date : Mon, 5 Apr 2021 13:21:44 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 5 Apr 2021 13:21:44 -0700
Reorganize the state struct used by VACUUM -- group related items
together to make it easier to understand. Also stop relying on stack
variables inside lazy_scan_heap() -- move those into the state struct
instead. Doing things this way simplifies large groups of related
functions whose function signatures had a lot of unnecessary redundancy.
Switch over to using int64 for the struct fields used to count things
that are reported to the user via log_autovacuum and VACUUM VERBOSE
output. We were using double, but that doesn't seem to have any
advantages. Using int64 makes it possible to add assertions that verify
that the first pass over the heap (pruning) encounters precisely the
same number of LP_DEAD items that get deleted from indexes later on, in
the second pass over the heap. These assertions will be added in later
commits.
Finally, adjust the signatures of functions with IndexBulkDeleteResult
pointer arguments in cases where there was ambiguity about whether or
not the argument relates to a single index or all indexes. Functions
now use the idiom that both ambulkdelete() and amvacuumcleanup() have
always used (where appropriate): accept a mutable IndexBulkDeleteResult
pointer argument, and return a result IndexBulkDeleteResult pointer to
caller.
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Masahiko Sawada <[email protected]>
Reviewed-By: Robert Haas <[email protected]>
Discussion: https://postgr.es/m/CAH2-WzkeOSYwC6KNckbhk2b1aNnWum6Yyn0NKP9D-Hq1LGTDPw@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/index/indexam.c
M src/backend/commands/vacuum.c
M src/include/access/genam.h
M src/include/access/heapam.h
M src/include/access/tableam.h
Add pg_read_all_data and pg_write_all_data roles
commit : 6c3ffd697e2242f5497ea4b40fffc8f6f922ff60
author : Stephen Frost <[email protected]>
date : Mon, 5 Apr 2021 13:42:52 -0400
committer: Stephen Frost <[email protected]>
date : Mon, 5 Apr 2021 13:42:52 -0400
A commonly requested use-case is to have a role who can run an
unfettered pg_dump without having to explicitly GRANT that user access
to all tables, schemas, et al, without that role being a superuser.
This address that by adding a "pg_read_all_data" role which implicitly
gives any member of this role SELECT rights on all tables, views and
sequences, and USAGE rights on all schemas.
As there may be cases where it's also useful to have a role who has
write access to all objects, pg_write_all_data is also introduced and
gives users implicit INSERT, UPDATE and DELETE rights on all tables,
views and sequences.
These roles can not be logged into directly but instead should be
GRANT'd to a role which is able to log in. As noted in the
documentation, if RLS is being used then an administrator may (or may
not) wish to set BYPASSRLS on the login role which these predefined
roles are GRANT'd to.
Reviewed-by: Georgios Kokolatos
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/user-manag.sgml
M src/backend/catalog/aclchk.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_authid.dat
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Shut down transaction tracking at startup process exit.
commit : ad8b674922eb70dc5cd02951dd82fe2c4c37c80a
author : Fujii Masao <[email protected]>
date : Tue, 6 Apr 2021 02:25:37 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 6 Apr 2021 02:25:37 +0900
Maxim Orlov reported that the shutdown of standby server could result in
the following assertion failure. The cause of this issue was that,
when the shutdown caused the startup process to exit, recovery-time
transaction tracking was not shut down even if it's already initialized,
and some locks the tracked transactions were holding could not be released.
At this situation, if other process was invoked and the PGPROC entry that
the startup process used was assigned to it, it found such unreleased locks
and caused the assertion failure, during the initialization of it.
TRAP: FailedAssertion("SHMQueueEmpty(&(MyProc->myProcLocks[i]))"
This commit fixes this issue by making the startup process shut down
transaction tracking and release all locks, at the exit of it.
Back-patch to all supported branches.
Reported-by: Maxim Orlov
Author: Fujii Masao
Reviewed-by: Maxim Orlov
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/startup.c
M src/backend/storage/ipc/standby.c
Align some terms in arch-dev.sgml to glossary
commit : 6734e806958c4ebd253adb30b255983981818710
author : Alvaro Herrera <[email protected]>
date : Mon, 5 Apr 2021 11:45:17 -0400
committer: Alvaro Herrera <[email protected]>
date : Mon, 5 Apr 2021 11:45:17 -0400
This mostly adds links to the glossary to the existing text, instead of
using <firstterm>. Heikki left this out of 29ad6595ef7f out of
stylistic concerns; these have since been addressed.
Author: Jürgen Purtz <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/arch-dev.sgml
Renumber cursor option flags
commit : a63dd8afe2b859b853d857cd8a0392ca1e04ab6c
author : Peter Eisentraut <[email protected]>
date : Mon, 5 Apr 2021 09:10:27 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 5 Apr 2021 09:10:27 +0200
Move the planner-control flags up so that there is more room for parse
options. Some pending patches need some room there, so do this
renumbering separately so that there is less potential for conflicts.
M src/include/nodes/parsenodes.h
Fix typo in collationcmds.c
commit : 9f6f1f9b8e61f9ce47e1936fc68c21a4a8d6722c
author : Michael Paquier <[email protected]>
date : Mon, 5 Apr 2021 11:18:12 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 5 Apr 2021 11:18:12 +0900
Introduced by 51e225d.
Author: Anton Voloshin
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/collationcmds.c
Refactor all TAP test suites doing connection checks
commit : c50624cdd248c13b4ba199f95e24c88d2cc8a097
author : Michael Paquier <[email protected]>
date : Mon, 5 Apr 2021 10:13:57 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 5 Apr 2021 10:13:57 +0900
This commit refactors more TAP tests to adapt with the recent
introduction of connect_ok() and connect_fails() in PostgresNode,
introduced by 0d1a3343. This changes the following test suites to use
the same code paths for connection checks:
- Kerberos
- LDAP
- SSL
- Authentication
Those routines are extended to be able to handle optional parameters
that are set depending on each suite's needs, as of:
- custom SQL query.
- expected stderr matching pattern.
- expected stdout matching pattern.
The new design is extensible with more parameters, and there are some
plans for those routines in the future with checks based on the contents
of the backend logs.
Author: Jacob Champion, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/test/authentication/t/001_password.pl
M src/test/authentication/t/002_saslprep.pl
M src/test/kerberos/t/001_auth.pl
M src/test/ldap/t/001_auth.pl
M src/test/perl/PostgresNode.pm
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
Fix more confusion in SP-GiST.
commit : dfc843d465689d2c2af8b0e01c66c51ccaae2343
author : Tom Lane <[email protected]>
date : Sun, 4 Apr 2021 17:57:07 -0400
committer: Tom Lane <[email protected]>
date : Sun, 4 Apr 2021 17:57:07 -0400
spg_box_quad_leaf_consistent unconditionally returned the leaf
datum as leafValue, even though in its usage for poly_ops that
value is of completely the wrong type.
In versions before 12, that was harmless because the core code did
nothing with leafValue in non-index-only scans ... but since commit
2a6368343, if we were doing a KNN-style scan, spgNewHeapItem would
unconditionally try to copy the value using the wrong datatype
parameters. Said copying is a waste of time and space if we're not
going to return the data, but it accidentally failed to fail until
I fixed the datatype confusion in ac9099fc1.
Hence, change spgNewHeapItem to not copy the datum unless we're
actually going to return it later. This saves cycles and dodges
the question of whether lossy opclasses are returning the right
type. Also change spg_box_quad_leaf_consistent to not return
data that might be of the wrong type, as insurance against
somebody introducing a similar bug into the core code in future.
It seems like a good idea to back-patch these two changes into
v12 and v13, although I'm afraid to change spgNewHeapItem's
mistaken idea of which datatype to use in those branches.
Per buildfarm results from ac9099fc1.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/spgist/spgscan.c
M src/backend/utils/adt/geo_spgist.c
Fix confusion in SP-GiST between attribute type and leaf storage type.
commit : ac9099fc1dd460bffaafec19272159dd7bc86f5b
author : Tom Lane <[email protected]>
date : Sun, 4 Apr 2021 14:28:35 -0400
committer: Tom Lane <[email protected]>
date : Sun, 4 Apr 2021 14:28:35 -0400
According to the documentation, the attType passed to the opclass
config function (and also relied on by the core code) is the type
of the heap column or expression being indexed. But what was
actually being passed was the type stored for the index column.
This made no difference for user-defined SP-GiST opclasses,
because we weren't allowing the STORAGE clause of CREATE OPCLASS
to be used, so the two types would be the same. But it's silly
not to allow that, seeing that the built-in poly_ops opclass
has a different value for opckeytype than opcintype, and that if you
want to do lossy storage then the types must really be different.
(Thus, user-defined opclasses doing lossy storage had to lie about
what type is in the index.) Hence, remove the restriction, and make
sure that we use the input column type not opckeytype where relevant.
For reasons of backwards compatibility with existing user-defined
opclasses, we can't quite insist that the specified leafType match
the STORAGE clause; instead just add an amvalidate() warning if
they don't match.
Also fix some bugs that would only manifest when trying to return
index entries when attType is different from attLeafType. It's not
too surprising that these have not been reported, because the only
usual reason for such a difference is to store the leaf value
lossily, rendering index-only scans impossible.
Add a src/test/modules module to exercise cases where attType is
different from attLeafType and yet index-only scan is supported.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_opclass.sgml
M doc/src/sgml/spgist.sgml
M src/backend/access/spgist/spgscan.c
M src/backend/access/spgist/spgutils.c
M src/backend/access/spgist/spgvalidate.c
M src/include/access/spgist_private.h
M src/test/modules/Makefile
A src/test/modules/spgist_name_ops/.gitignore
A src/test/modules/spgist_name_ops/Makefile
A src/test/modules/spgist_name_ops/README
A src/test/modules/spgist_name_ops/expected/spgist_name_ops.out
A src/test/modules/spgist_name_ops/spgist_name_ops–1.0.sql
A src/test/modules/spgist_name_ops/spgist_name_ops.c
A src/test/modules/spgist_name_ops/spgist_name_ops.control
A src/test/modules/spgist_name_ops/sql/spgist_name_ops.sql
M src/test/regress/expected/rangetypes.out
M src/test/regress/sql/rangetypes.sql
Fix bug in brin_minmax_multi_union
commit : d9c5b9a9eeb9e3061ae139e0e564ce5358c94001
author : Tomas Vondra <[email protected]>
date : Sun, 4 Apr 2021 19:36:10 +0200
committer: Tomas Vondra <[email protected]>
date : Sun, 4 Apr 2021 19:36:10 +0200
When calling sort_expanded_ranges() we need to remember the return
value, because the function sorts and also deduplicates the ranges. So
the number of ranges may decrease. brin_minmax_multi_union failed to do
that, which resulted in crashes due to bogus ranges (equal minval/maxval
but not marked as compacted).
Reported-by: Jaime Casanova
Discussion: https://postgr.es/m/20210404052550.GA4376%40ahch-to
M src/backend/access/brin/brin_minmax_multi.c
Add regression test for minmax-multi macaddr8 type
commit : 4908684ddab35135869efa2af6b49c4d67c422f9
author : Tomas Vondra <[email protected]>
date : Sun, 4 Apr 2021 19:26:48 +0200
committer: Tomas Vondra <[email protected]>
date : Sun, 4 Apr 2021 19:26:48 +0200
The regression test for BRIN minmax-multi opclasses tested almost all
supported data types, with the exception of macaddr8. So this adds it.
M src/test/regress/expected/brin_multi.out
M src/test/regress/sql/brin_multi.sql
Fix order of parameters in BRIN minmax-multi calls
commit : 1dad2a5ea3d14dd205603c31cc94ec088183ab2a
author : Tomas Vondra <[email protected]>
date : Sun, 4 Apr 2021 19:25:36 +0200
committer: Tomas Vondra <[email protected]>
date : Sun, 4 Apr 2021 19:25:36 +0200
The BRIN minmax-multi consistent function incorrectly assumed it can
lookup an operator, and then swap the arguments to get the commutator.
For example <(a,b) would be called as <(b,a) to get >(a,b). This works
when the arguments are of the same type, but with cross-type opclasses
this fails. We can't swap <(float4,float8) arguments, for example.
Fixed by passing arguments in the right order.
Discussion: https://postgr.es/m/CAJKUy5jLZFLCxyxfT%3DMfK5mtPfSzHA1rVLowR-j4RRsFVvKm7A%40mail.gmail.com
M src/backend/access/brin/brin_minmax_multi.c
Fix BRIN minmax-multi distance for inet type
commit : e1fbe1181c86247eaf8b4b142b81361ce4efcc66
author : Tomas Vondra <[email protected]>
date : Sun, 4 Apr 2021 19:23:30 +0200
committer: Tomas Vondra <[email protected]>
date : Sun, 4 Apr 2021 19:23:30 +0200
The distance calculation ignored the mask, unlike the inet comparator,
which resulted in negative distance in some cases. Fixed by applying the
mask in brin_minmax_multi_distance_inet. I've considered simply calling
inetmi() to calculate the delta, but that does not consider mask either.
Reviewed-by: Zhihong Yu
Discussion: https://postgr.es/m/1a0a7b9d-9bda-e3a2-7fa4-88f15042a051%40enterprisedb.com
M src/backend/access/brin/brin_minmax_multi.c
Fix BRIN minmax-multi distance for timetz type
commit : 7262f2421a1e099a631356f7b80ad198e34e2a8a
author : Tomas Vondra <[email protected]>
date : Sun, 4 Apr 2021 19:21:41 +0200
committer: Tomas Vondra <[email protected]>
date : Sun, 4 Apr 2021 19:21:41 +0200
The distance calculation ignored the time zone, so the result of (b-a)
might have ended negative even if (b > a). Fixed by considering the time
zone difference.
Reported-by: Jaime Casanova
Discussion: https://postgr.es/m/CAJKUy5jLZFLCxyxfT%3DMfK5mtPfSzHA1rVLowR-j4RRsFVvKm7A%40mail.gmail.com
M src/backend/access/brin/brin_minmax_multi.c
Fix BRIN minmax-multi distance for interval type
commit : 2b10e0e3c2ca14d732521479123e5d5e2094e143
author : Tomas Vondra <[email protected]>
date : Sun, 4 Apr 2021 19:13:26 +0200
committer: Tomas Vondra <[email protected]>
date : Sun, 4 Apr 2021 19:13:26 +0200
The distance calculation for interval type was treating months as having
31 days, which is inconsistent with the interval comparator (using 30
days). Due to this it was possible to get negative distance (b-a) when
(a<b), trigerring an assert.
Fixed by adopting the same logic as interval_cmp_value.
Reported-by: Jaime Casanova
Discussion: https://postgr.es/m/CAJKUy5jKH0Xhneau2mNftNPtTy-BVgQfXc8zQkEvRvBHfeUThQ%40mail.gmail.com
M src/backend/access/brin/brin_minmax_multi.c
Improve psql's behavior when the editor is exited without saving.
commit : 55873a00e3c3349664e7215077dca74ccea08b4d
author : Tom Lane <[email protected]>
date : Sat, 3 Apr 2021 17:38:31 -0400
committer: Tom Lane <[email protected]>
date : Sat, 3 Apr 2021 17:38:31 -0400
When editing the previous query buffer, if the editor is exited
without modifying the temp file then clear the query buffer,
rather than re-loading (and probably re-executing) the previous
query buffer. This reduces the probability of accidentally
re-executing something you didn't intend to.
Similarly, in "\e file", if the file isn't actually modified
then don't load it into the query buffer. And in "\ef" and
"\ev", if no changes are made then clear the query buffer
instead of loading the function or view definition into it.
Cases where we fail to invoke the editor at all, or it returns
a nonzero status, are treated like the no-file-modification case.
Laurenz Albe, reviewed by Jacob Champion
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
Improve efficiency of wait event reporting, remove proc.h dependency.
commit : 225a22b19ed2960acc8e9c0b7ae53e0e5b0eac87
author : Andres Freund <[email protected]>
date : Sat, 3 Apr 2021 11:44:47 -0700
committer: Andres Freund <[email protected]>
date : Sat, 3 Apr 2021 11:44:47 -0700
pgstat_report_wait_start() and pgstat_report_wait_end() required two
conditional branches so far. One to check if MyProc is NULL, the other to
check if pgstat_track_activities is set. As wait events are used around
comparatively lightweight operations, and are inlined (reducing branch
predictor effectiveness), that's not great.
The dependency on MyProc has a second disadvantage: Low-level subsystems, like
storage/file/fd.c, report wait events, but architecturally it is preferable
for them to not depend on inter-process subsystems like proc.h (defining
PGPROC). After this change including pgstat.h (nor obviously its
sub-components like backend_status.h, wait_event.h, ...) does not pull in IPC
related headers anymore.
These goals, efficiency and abstraction, are achieved by having
pgstat_report_wait_start/end() not interact with MyProc, but instead a new
my_wait_event_info variable. At backend startup it points to a local variable,
removing the need to check for MyProc being NULL. During process
initialization my_wait_event_info is redirected to MyProc->wait_event_info. At
shutdown this is reversed. Because wait event reporting now does not need to
know about where the wait event is stored, it does not need to know about
PGPROC anymore.
The removal of the branch for checking pgstat_track_activities is simpler:
Don't check anymore. The cost due to the branch are often higher than the
store - and even if not, pgstat_track_activities is rarely disabled.
The main motivator to commit this work now is that removing the (indirect)
pgproc.h include from pgstat.h simplifies a patch to move statistics reporting
to shared memory (which still has a chance to get into 14).
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/lmgr/proc.c
M src/backend/utils/activity/wait_event.c
M src/include/utils/wait_event.h
Split backend status and progress related functionality out of pgstat.c.
commit : e1025044cd4e7f33f7304aed54d5778b8a82cd5d
author : Andres Freund <[email protected]>
date : Sat, 3 Apr 2021 11:42:52 -0700
committer: Andres Freund <[email protected]>
date : Sat, 3 Apr 2021 11:42:52 -0700
Backend status (supporting pg_stat_activity) and command
progress (supporting pg_stat_progress*) related code is largely
independent from the rest of pgstat.[ch] (supporting views like
pg_stat_all_tables that accumulate data over time). See also
a333476b925.
This commit doesn't rename the function names to make the distinction
from the rest of pgstat_ clearer - that'd be more invasive and not
clearly beneficial. If we were to decide to do such a rename at some
point, it's better done separately from moving the code as well.
Robert's review was of an earlier version.
Reviewed-By: Robert Haas <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/bootstrap/bootstrap.c
M src/backend/postmaster/pgstat.c
M src/backend/utils/activity/Makefile
A src/backend/utils/activity/backend_progress.c
A src/backend/utils/activity/backend_status.c
M src/backend/utils/init/postinit.c
M src/backend/utils/misc/guc.c
M src/include/commands/progress.h
M src/include/pgstat.h
A src/include/utils/backend_progress.h
A src/include/utils/backend_status.h
Use more verbose matching patterns for errors in SSL TAP tests
commit : 8d3a4c3eae5367fba60ab77c159814defba784fe
author : Michael Paquier <[email protected]>
date : Sat, 3 Apr 2021 20:49:08 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 3 Apr 2021 20:49:08 +0900
The TAP tests of src/test/ssl/ have been using rather generic matching
patterns to check some failure scenarios, like "SSL error" or just
"FATAL". These have been introduced in 081bfc1.
Those messages are not wrong per se, but when working on the integration
of new SSL libraries it becomes hard to know if those errors are legit
or not, and existing scenarios may fail in incorrect ways. This commit
makes all those messages more verbose by adding the information
generated by OpenSSL. Fortunately, the same error messages are used for
all the versions supported on HEAD (checked that after running the tests
from 1.0.1 to 1.1.1), so the change is straight-forward.
Reported-by: Jacob Champion, Álvaro Herrera
Discussion: https://postgr.es/m/[email protected]
M src/test/ssl/t/001_ssltests.pl
Refactor HMAC implementations
commit : e6bdfd9700ebfc7df811c97c2fc46d7e94e329a2
author : Michael Paquier <[email protected]>
date : Sat, 3 Apr 2021 17:30:49 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 3 Apr 2021 17:30:49 +0900
Similarly to the cryptohash implementations, this refactors the existing
HMAC code into a single set of APIs that can be plugged with any crypto
libraries PostgreSQL is built with (only OpenSSL currently). If there
is no such libraries, a fallback implementation is available. Those new
APIs are designed similarly to the existing cryptohash layer, so there
is no real new design here, with the same logic around buffer bound
checks and memory handling.
HMAC has a dependency on cryptohashes, so all the cryptohash types
supported by cryptohash{_openssl}.c can be used with HMAC. This
refactoring is an advantage mainly for SCRAM, that included its own
implementation of HMAC with SHA256 without relying on the existing
crypto libraries even if PostgreSQL was built with their support.
This code has been tested on Windows and Linux, with and without
OpenSSL, across all the versions supported on HEAD from 1.1.1 down to
1.0.1. I have also checked that the implementations are working fine
using some sample results, a custom extension of my own, and doing
cross-checks across different major versions with SCRAM with the client
and the backend.
Author: Michael Paquier
Reviewed-by: Bruce Momjian
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
M src/backend/libpq/auth-scram.c
M src/backend/utils/resowner/resowner.c
M src/common/Makefile
A src/common/hmac.c
A src/common/hmac_openssl.c
M src/common/scram-common.c
A src/include/common/hmac.h
M src/include/common/md5.h
M src/include/common/scram-common.h
M src/include/common/sha1.h
M src/include/pg_config.h.in
M src/include/utils/resowner_private.h
M src/interfaces/libpq/fe-auth-scram.c
M src/tools/msvc/Mkvcbuild.pm
M src/tools/msvc/Solution.pm
M src/tools/pgindent/typedefs.list
Do not rely on pgstat.h to indirectly include storage/ headers.
commit : 1d9c5d0ce2dcac05850401cf266a9df10a68de49
author : Andres Freund <[email protected]>
date : Fri, 2 Apr 2021 20:01:14 -0700
committer: Andres Freund <[email protected]>
date : Fri, 2 Apr 2021 20:01:14 -0700
An upcoming patch might remove the (now indirect) proc.h
include (which in turn includes other headers), and it's cleaner for
the modified files to include their dependencies directly anyway...
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_stat_statements/pg_stat_statements.c
M src/backend/postmaster/pgarch.c
M src/backend/postmaster/pgstat.c
M src/backend/replication/walreceiver.c
M src/backend/storage/lmgr/lmgr.c
M src/backend/utils/adt/misc.c
Split wait event related code from pgstat.[ch] into wait_event.[ch].
commit : a333476b925134f6185037eaff3424c07a9f466f
author : Andres Freund <[email protected]>
date : Fri, 2 Apr 2021 19:45:24 -0700
committer: Andres Freund <[email protected]>
date : Fri, 2 Apr 2021 19:45:24 -0700
The wait event related code is independent from the rest of the
pgstat.[ch] code, of nontrivial size and changes on a regular
basis. Put it into its own set of files.
As there doesn't seem to be a good pre-existing directory for code
like this, add src/backend/utils/activity.
Reviewed-By: Robert Haas <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/pgstat.c
M src/backend/utils/Makefile
A src/backend/utils/activity/Makefile
A src/backend/utils/activity/wait_event.c
M src/include/pgstat.h
A src/include/utils/wait_event.h
Remove useless Asserts in Result Cache code
commit : 1267d9862fc6a4f8cdc0ca38d1988b61f39da585
author : David Rowley <[email protected]>
date : Sat, 3 Apr 2021 10:41:43 +1300
committer: David Rowley <[email protected]>
date : Sat, 3 Apr 2021 10:41:43 +1300
Testing if an unsigned variable is >= 0 is pretty pointless.
There's likely enough code in remove_cache_entry() to verify the cache
memory accounting is correct in assert enabled builds. These Asserts
were not adding much extra cover, even if they had been checking >= 0 on a
signed variable.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/nodeResultCache.c
Use macro MONTHS_PER_YEAR instead of '12' in /ecpg/pgtypeslib
commit : 84bc2b17523ef485f102be7f00f7affb88f00f18
author : Bruce Momjian <[email protected]>
date : Fri, 2 Apr 2021 16:42:29 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 2 Apr 2021 16:42:29 -0400
All other places already use MONTHS_PER_YEAR appropriately.
Backpatch-through: 9.6
M src/interfaces/ecpg/pgtypeslib/interval.c
Detect POLLHUP/POLLRDHUP while running queries.
commit : c30f54ad732ca5c8762bb68bbe0f51de9137dd72
author : Thomas Munro <[email protected]>
date : Sat, 3 Apr 2021 08:52:30 +1300
committer: Thomas Munro <[email protected]>
date : Sat, 3 Apr 2021 08:52:30 +1300
Provide a new GUC check_client_connection_interval that can be used to
check whether the client connection has gone away, while running very
long queries. It is disabled by default.
For now this uses a non-standard Linux extension (also adopted by at
least one other OS). POLLRDHUP is not defined by POSIX, and other OSes
don't have a reliable way to know if a connection was closed without
actually trying to read or write.
In future we might consider trying to send a no-op/heartbeat message
instead, but that could require protocol changes.
Author: Sergey Cherkashin <[email protected]>
Author: Thomas Munro <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Reviewed-by: Tatsuo Ishii <[email protected]>
Reviewed-by: Konstantin Knizhnik <[email protected]>
Reviewed-by: Zhihong Yu <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Maksim Milyutin <[email protected]>
Reviewed-by: Tsunakawa, Takayuki/綱川 貴之 <[email protected]>
Reviewed-by: Tom Lane <[email protected]> (much earlier version)
Discussion: https://postgr.es/m/77def86b27e41f0efcba411460e929ae%40postgrespro.ru
M doc/src/sgml/config.sgml
M src/backend/libpq/pqcomm.c
M src/backend/tcop/postgres.c
M src/backend/utils/init/globals.c
M src/backend/utils/init/postinit.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/libpq/libpq.h
M src/include/miscadmin.h
M src/include/tcop/tcopprot.h
M src/include/utils/timeout.h
Clarify documentation of RESET ROLE
commit : 174edbe9f9c1538ab3347474e96d176223591cd1
author : Joe Conway <[email protected]>
date : Fri, 2 Apr 2021 13:48:42 -0400
committer: Joe Conway <[email protected]>
date : Fri, 2 Apr 2021 13:48:42 -0400
Command-line options, or previous "ALTER (ROLE|DATABASE) ...
SET ROLE ..." commands, can change the value of the default role
for a session. In the presence of one of these, RESET ROLE will
change the current user identifier to the default role rather
than the session user identifier. Fix the documentation to
reflect this reality. Backpatch to all supported versions.
Author: Nathan Bossart
Reviewed-By: Laurenz Albe, David G. Johnston, Joe Conway
Reported by: Nathan Bossart
Discussion: https://postgr.es/m/flat/925134DB-8212-4F60-8AB1-B1231D750CB4%40amazon.com
Backpatch-through: 9.6
M doc/src/sgml/ref/set_role.sgml
pg_checksums: Fix progress reporting.
commit : 2eb1fc8b1ae8b974007e85636fc7336a9b5d7222
author : Fujii Masao <[email protected]>
date : Sat, 3 Apr 2021 00:07:00 +0900
committer: Fujii Masao <[email protected]>
date : Sat, 3 Apr 2021 00:07:00 +0900
pg_checksums uses two counters, total size and current size,
to calculate the progress. Previously the progress that
pg_checksums reported could not reach 100% at the end.
The cause of this issue was that the sizes of only pages excluding
new ones in each file were counted as the current size
while the size of each file is counted as the total size.
That is, the total size of all new pages could be reported
as the difference between the total size and current size.
This commit fixes this issue by making pg_checksums count
the sizes of all pages including new ones in each file as
the current size.
Back-patch to v12 where progress reporting was added to pg_checksums.
Reported-by: Shinya Kato
Author: Shinya Kato
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/TYAPR01MB289656B1ACA0A5E7CAD07BE3C47A9@TYAPR01MB2896.jpnprd01.prod.outlook.com
M src/bin/pg_checksums/pg_checksums.c
Strip file names reported in error messages on Windows, too.
commit : 53aafdb9ff6a561c7dea0f428a7c168f2b7e0f16
author : Tom Lane <[email protected]>
date : Fri, 2 Apr 2021 10:43:54 -0400
committer: Tom Lane <[email protected]>
date : Fri, 2 Apr 2021 10:43:54 -0400
Commit dd136052b established a policy that error message FILE items
should include only the base name of the reporting source file, for
uniformity and succinctness. We now observe that some Windows compilers
use backslashes in __FILE__ strings, so truncate at backslashes as well.
This is expected to fix some platform variation in the results of the
new libpq_pipeline test module.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/error/elog.c
Fix typo in 6d7a6feac4
commit : 1877c9ac3acc05cc787dd6392d073202f8c8ee21
author : Andrew Dunstan <[email protected]>
date : Fri, 2 Apr 2021 10:29:58 -0400
committer: Andrew Dunstan <[email protected]>
date : Fri, 2 Apr 2021 10:29:58 -0400
Per gripe from Daniel Gustafsson
M src/test/ssl/Makefile
postgres_fdw: Add option to control whether to keep connections open.
commit : b1be3074ac719ce8073fba35d4c8b52fb4ddd0c3
author : Fujii Masao <[email protected]>
date : Fri, 2 Apr 2021 19:45:42 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 2 Apr 2021 19:45:42 +0900
This commit adds a new option keep_connections that controls
whether postgres_fdw keeps the connections to the foreign server open
so that the subsequent queries can re-use them. This option can only be
specified for a foreign server. The default is on. If set to off,
all connections to the foreign server will be discarded
at the end of transaction. Closed connections will be re-established
when they are necessary by future queries using a foreign table.
This option is useful, for example, when users want to prevent
the connections from eating up the foreign servers connections
capacity.
Author: Bharath Rupireddy
Reviewed-by: Alexey Kondratov, Vignesh C, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVvrp5=AVp2PupEm+nAC8S4buqR3fJMmaCoc7ftT0aD2A@mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Add support for NullIfExpr in eval_const_expressions
commit : 9c5f67fd6256246b2a788a8feb1d42b79dcd0448
author : Peter Eisentraut <[email protected]>
date : Fri, 2 Apr 2021 11:01:49 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 2 Apr 2021 11:01:49 +0200
Author: Hou Zhijie <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/7ea5ce773bbc4eea9ff1a381acd3b102@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/optimizer/util/clauses.c
M src/test/regress/expected/case.out
M src/test/regress/sql/case.sql
Fix pgstat_report_replslot() to use proper data types for its arguments.
commit : 96bdb7e19de80a0c9521c5696455bca2a685c919
author : Fujii Masao <[email protected]>
date : Fri, 2 Apr 2021 17:27:31 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 2 Apr 2021 17:27:31 +0900
The caller of pgstat_report_replslot() passes int64 values to the function.
Also the function stores those values in PgStat_Counter (i.e., int64) fields
of PgStat_MsgReplSlot struct. But previously the function used "int" as
the data types of some arguments for those values, which could lead to
the overflow of values.
To avoid this risk, this commit fixes pgstat_report_replslot() to use
PgStat_Counter type for the arguments. Since they are the statistics counters,
PgStat_Counter, the data type used for counters, is used for them
instead of int64.
Reported-by: Vignesh C
Author: Vignesh C
Reviewed-by: Jeevan Ladhe, Fujii Masao
Discussion: https://postgr.es/m/CALDaNm080OpG=ZwOb0i8EyChH5SyHAMFWJCKaKTXmrfvJLbgaA@mail.gmail.com
M src/backend/postmaster/pgstat.c
M src/include/pgstat.h
doc: Clarify how to generate backup files with non-exclusive backups
commit : 6fb66c268df2de1112cac3cf0a6cf0a8b96ceaf0
author : Michael Paquier <[email protected]>
date : Fri, 2 Apr 2021 16:37:00 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 2 Apr 2021 16:37:00 +0900
The current instructions describing how to write the backup_label and
tablespace_map files are confusing. For example, opening a file in text
mode on Windows and copy-pasting the file's contents would result in a
failure at recovery because of the extra CRLF characters generated. The
documentation was not stating that clearly, and per discussion this is
not considered as a supported scenario.
This commit extends a bit the documentation to mention that it may be
required to open the file in binary mode before writing its data.
Reported-by: Wang Shenhao
Author: David Steele
Reviewed-by: Andrew Dunstan, Magnus Hagander
Discussion: https://postgr.es/m/8373f61426074f2cb6be92e02f838389@G08CNEXMBPEKD06.g08.fujitsu.local
Backpatch-through: 9.6
M doc/src/sgml/backup.sgml
Fix typos in comments.
commit : 98e5bd103f887326e381c509c2fbe879ba3ea2f3
author : Fujii Masao <[email protected]>
date : Fri, 2 Apr 2021 16:26:36 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 2 Apr 2021 16:26:36 +0900
Author: Masahiko Sawada
Discussion: https://postgr.es/m/CAD21AoA1YL7t0nzVSEySx6zOaE7xO3r0jyu8hkitGL2_XbaMxQ@mail.gmail.com
M contrib/amcheck/verify_heapam.c
Attempt to fix unstable Result Cache regression tests
commit : a4fac4ffe8f8d543a10ac7debf1157e34963ece3
author : David Rowley <[email protected]>
date : Fri, 2 Apr 2021 15:25:38 +1300
committer: David Rowley <[email protected]>
date : Fri, 2 Apr 2021 15:25:38 +1300
force_parallel_mode = regress is causing a few more problems than I
thought. It seems that both the leader and the single worker can
contribute to the execution. I had mistakenly thought that only the worker
process would do any work. Since it's not deterministic as to which
of the two processes will get a chance to work on the plan, it seems just
better to disable force_parallel_mode for these tests. At least doing
this seems better than changing to EXPLAIN only rather than EXPLAIN
ANALYZE.
Additionally, I overlooked the fact that the number of executions of the
sub-plan below a Result Cache will execute a varying number of times
depending on cache eviction. 32-bit machines will use less memory and
evict fewer tuples from the cache. That results in the subnode being
executed fewer times on 32-bit machines. Let's just blank out the number
of loops in each node.
M src/test/regress/expected/resultcache.out
M src/test/regress/sql/resultcache.sql
doc: mention that intervening major releases can be skipped
commit : 2bda93f813919b58225f5a0e282e10b98d7633d4
author : Bruce Momjian <[email protected]>
date : Thu, 1 Apr 2021 21:17:24 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 1 Apr 2021 21:17:24 -0400
Also mention that you should read the intervening major releases notes.
This change was also applied to the website.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M doc/src/sgml/runtime.sgml
Add Result Cache executor node (take 2)
commit : 9eacee2e62d89cab7b004f97c206c4fba4f1d745
author : David Rowley <[email protected]>
date : Fri, 2 Apr 2021 14:10:56 +1300
committer: David Rowley <[email protected]>
date : Fri, 2 Apr 2021 14:10:56 +1300
Here we add a new executor node type named "Result Cache". The planner
can include this node type in the plan to have the executor cache the
results from the inner side of parameterized nested loop joins. This
allows caching of tuples for sets of parameters so that in the event that
the node sees the same parameter values again, it can just return the
cached tuples instead of rescanning the inner side of the join all over
again. Internally, result cache uses a hash table in order to quickly
find tuples that have been previously cached.
For certain data sets, this can significantly improve the performance of
joins. The best cases for using this new node type are for join problems
where a large portion of the tuples from the inner side of the join have
no join partner on the outer side of the join. In such cases, hash join
would have to hash values that are never looked up, thus bloating the hash
table and possibly causing it to multi-batch. Merge joins would have to
skip over all of the unmatched rows. If we use a nested loop join with a
result cache, then we only cache tuples that have at least one join
partner on the outer side of the join. The benefits of using a
parameterized nested loop with a result cache increase when there are
fewer distinct values being looked up and the number of lookups of each
value is large. Also, hash probes to lookup the cache can be much faster
than the hash probe in a hash join as it's common that the result cache's
hash table is much smaller than the hash join's due to result cache only
caching useful tuples rather than all tuples from the inner side of the
join. This variation in hash probe performance is more significant when
the hash join's hash table no longer fits into the CPU's L3 cache, but the
result cache's hash table does. The apparent "random" access of hash
buckets with each hash probe can cause a poor L3 cache hit ratio for large
hash tables. Smaller hash tables generally perform better.
The hash table used for the cache limits itself to not exceeding work_mem
* hash_mem_multiplier in size. We maintain a dlist of keys for this cache
and when we're adding new tuples and realize we've exceeded the memory
budget, we evict cache entries starting with the least recently used ones
until we have enough memory to add the new tuples to the cache.
For parameterized nested loop joins, we now consider using one of these
result cache nodes in between the nested loop node and its inner node. We
determine when this might be useful based on cost, which is primarily
driven off of what the expected cache hit ratio will be. Estimating the
cache hit ratio relies on having good distinct estimates on the nested
loop's parameters.
For now, the planner will only consider using a result cache for
parameterized nested loop joins. This works for both normal joins and
also for LATERAL type joins to subqueries. It is possible to use this new
node for other uses in the future. For example, to cache results from
correlated subqueries. However, that's not done here due to some
difficulties obtaining a distinct estimation on the outer plan to
calculate the estimated cache hit ratio. Currently we plan the inner plan
before planning the outer plan so there is no good way to know if a result
cache would be useful or not since we can't estimate the number of times
the subplan will be called until the outer plan is generated.
The functionality being added here is newly introducing a dependency on
the return value of estimate_num_groups() during the join search.
Previously, during the join search, we only ever needed to perform
selectivity estimations. With this commit, we need to use
estimate_num_groups() in order to estimate what the hit ratio on the
result cache will be. In simple terms, if we expect 10 distinct values
and we expect 1000 outer rows, then we'll estimate the hit ratio to be
99%. Since cache hits are very cheap compared to scanning the underlying
nodes on the inner side of the nested loop join, then this will
significantly reduce the planner's cost for the join. However, it's
fairly easy to see here that things will go bad when estimate_num_groups()
incorrectly returns a value that's significantly lower than the actual
number of distinct values. If this happens then that may cause us to make
use of a nested loop join with a result cache instead of some other join
type, such as a merge or hash join. Our distinct estimations have been
known to be a source of trouble in the past, so the extra reliance on them
here could cause the planner to choose slower plans than it did previous
to having this feature. Distinct estimations are also fairly hard to
estimate accurately when several tables have been joined already or when a
WHERE clause filters out a set of values that are correlated to the
expressions we're estimating the number of distinct value for.
For now, the costing we perform during query planning for result caches
does put quite a bit of faith in the distinct estimations being accurate.
When these are accurate then we should generally see faster execution
times for plans containing a result cache. However, in the real world, we
may find that we need to either change the costings to put less trust in
the distinct estimations being accurate or perhaps even disable this
feature by default. There's always an element of risk when we teach the
query planner to do new tricks that it decides to use that new trick at
the wrong time and causes a regression. Users may opt to get the old
behavior by turning the feature off using the enable_resultcache GUC.
Currently, this is enabled by default. It remains to be seen if we'll
maintain that setting for the release.
Additionally, the name "Result Cache" is the best name I could think of
for this new node at the time I started writing the patch. Nobody seems
to strongly dislike the name. A few people did suggest other names but no
other name seemed to dominate in the brief discussion that there was about
names. Let's allow the beta period to see if the current name pleases
enough people. If there's some consensus on a better name, then we can
change it before the release. Please see the 2nd discussion link below
for the discussion on the "Result Cache" name.
Author: David Rowley
Reviewed-by: Andy Fan, Justin Pryzby, Zhihong Yu, Hou Zhijie
Tested-By: Konstantin Knizhnik
Discussion: https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com
Discussion: https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M src/backend/commands/explain.c
M src/backend/executor/Makefile
M src/backend/executor/execAmi.c
M src/backend/executor/execExpr.c
M src/backend/executor/execParallel.c
M src/backend/executor/execProcnode.c
A src/backend/executor/nodeResultCache.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/README
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/executor/executor.h
A src/include/executor/nodeResultCache.h
M src/include/lib/ilist.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/cost.h
M src/include/optimizer/pathnode.h
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/join.out
M src/test/regress/expected/partition_prune.out
A src/test/regress/expected/resultcache.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/sysviews.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/join.sql
M src/test/regress/sql/partition_prune.sql
A src/test/regress/sql/resultcache.sql
Improve stability of test with vacuum_truncate in reloptions.sql
commit : fe246d1c111d43fd60a1b0afff25ed09b7ae11eb
author : Michael Paquier <[email protected]>
date : Fri, 2 Apr 2021 09:44:42 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 2 Apr 2021 09:44:42 +0900
This test has been using a simple VACUUM with pg_relation_size() to
check if a relation gets physically truncated or not, but forgot the
fact that some concurrent activity, like checkpoint buffer writes, could
cause some pages to be skipped. The second test enabling
vacuum_truncate could fail, seeing a non-empty relation. The first test
would not have failed, but could finish by testing a behavior different
than the one aimed for. Both tests gain a FREEZE option, to make the
vacuums more aggressive and prevent page skips.
This is similar to the issues fixed in c2dc1a7.
Author: Arseny Sher
Reviewed-by: Masahiko Sawada
Discussion: https://postgr.es/m/87tuotr2hh.fsf@ars-thinkpad
backpatch-through: 12
M src/test/regress/expected/reloptions.out
M src/test/regress/sql/reloptions.sql
Rethink handling of pass-by-value leaf datums in SP-GiST.
commit : 1ebdec8c03294e55a9fdb6e676a9e8de680231cc
author : Tom Lane <[email protected]>
date : Thu, 1 Apr 2021 17:55:14 -0400
committer: Tom Lane <[email protected]>
date : Thu, 1 Apr 2021 17:55:14 -0400
The existing convention in SP-GiST is that any pass-by-value datatype
is stored in Datum representation, i.e. it's of width sizeof(Datum)
even when typlen is less than that. This is okay, or at least it's
too late to change it, for prefix datums and node-label datums in inner
(upper) tuples. But it's problematic for leaf datums, because we'd
prefer those to be stored in Postgres' standard on-disk representation
so that we can easily extend leaf tuples to carry additional "included"
columns.
I believe, however, that we can get away with just up and changing that.
This would be an unacceptable on-disk-format break, but there are two
big mitigating factors:
1. It seems quite unlikely that there are any SP-GiST opclasses out
there that use pass-by-value leaf datatypes. Certainly none of the
ones in core do, nor has codesearch.debian.net heard of any. Given
what SP-GiST is good for, it's hard to conceive of a use-case where
the leaf-level values would be both small and fixed-width. (As an
example, if you wanted to index text values with the leaf level being
just a byte, then every text string would have to be represented
with one level of inner tuple per preceding byte, which would be
horrendously space-inefficient and slow to access. You always want
to use as few inner-tuple levels as possible, leaving as much as
possible in the leaf values.)
2. Even granting that you have such an index, this change only
breaks things on big-endian machines. On little-endian, the high
order bytes of the Datum format will now just appear to be alignment
padding space.
So, change the code to store pass-by-value leaf datums in their
usual on-disk form. Inner-tuple datums are not touched.
This is extracted from a larger patch that intends to add support for
"included" columns. I'm committing it separately for visibility in
our commit logs.
Pavel Borisov and Tom Lane, reviewed by Andrey Borodin
Discussion: https://postgr.es/m/CALT9ZEFi-vMp4faht9f9Junb1nO3NOSjhpxTmbm1UGLMsLqiEQ@mail.gmail.com
M src/backend/access/spgist/spgdoinsert.c
M src/backend/access/spgist/spgutils.c
M src/include/access/spgist_private.h
Rename Default Roles to Predefined Roles
commit : c9c41c7a337d3e2deb0b2a193e9ecfb865d8f52b
author : Stephen Frost <[email protected]>
date : Thu, 1 Apr 2021 15:32:06 -0400
committer: Stephen Frost <[email protected]>
date : Thu, 1 Apr 2021 15:32:06 -0400
The term 'default roles' wasn't quite apt as these roles aren't able to
be modified or removed after installation, so rename them to be
'Predefined Roles' instead, adding an entry into the newly added
Obsolete Appendix to help users of current releases find the new
documentation.
Bruce Momjian and Stephen Frost
Discussion: https://postgr.es/m/157742545062.1149.11052653770497832538%40wrigleys.postgresql.org
and https://www.postgresql.org/message-id/[email protected]
M contrib/adminpack/adminpack.c
M contrib/file_fdw/file_fdw.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pgrowlocks/pgrowlocks.c
A doc/src/sgml/appendix-obsolete-default-roles.sgml
M doc/src/sgml/appendix-obsolete.sgml
M doc/src/sgml/file-fdw.sgml
M doc/src/sgml/filelist.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/ref/copy.sgml
M doc/src/sgml/user-manag.sgml
M src/backend/commands/copy.c
M src/backend/commands/user.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walsender.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/signalfuncs.c
M src/backend/utils/adt/acl.c
M src/backend/utils/adt/dbsize.c
M src/backend/utils/adt/genfile.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/misc/guc.c
M src/include/catalog/pg_authid.dat
Fix setvbuf()-induced crash in libpq_pipeline
commit : a68a894f0198aaeffa81b3027f135adcdaa8abf6
author : Alvaro Herrera <[email protected]>
date : Thu, 1 Apr 2021 16:25:46 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 1 Apr 2021 16:25:46 -0300
Windows doesn't like setvbuf(..., _IOLBF) and crashes if you use it,
which has been causing the libpq_pipeline failures all along ... and our
own port.h has known about it for a long time: it offers PG_IOLBF that's
defined to _IONBF on that platform. Follow its advice.
While at it, get rid of a bogus bitshift that used a constant of the
wrong size. Decorate the constant as LL to fix. While at it, remove a
pointless addition that only confused matters.
All as diagnosed by Tom Lane.
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/libpq_pipeline/libpq_pipeline.c
amcheck: Fix verify_heapam's tuple visibility checking rules.
commit : 3b6c1259f9ca8e21860aaf24ec6735a8e5598ea0
author : Robert Haas <[email protected]>
date : Thu, 1 Apr 2021 13:36:28 -0400
committer: Robert Haas <[email protected]>
date : Thu, 1 Apr 2021 13:36:28 -0400
We now follow the order of checks from HeapTupleSatisfies* more
closely to avoid coming to erroneous conclusions.
Mark Dilger and Robert Haas
Discussion: http://postgr.es/m/CA+Tgmob6sii0yTvULYJ0Vq4w6ZBmj7zUhddL3b+SKDi9z9NA7Q@mail.gmail.com
M contrib/amcheck/verify_heapam.c
Fix pg_restore's misdesigned code for detecting archive file format.
commit : ec03f2df17a8ba5b431b34dd924e020a0be729f6
author : Tom Lane <[email protected]>
date : Thu, 1 Apr 2021 13:34:16 -0400
committer: Tom Lane <[email protected]>
date : Thu, 1 Apr 2021 13:34:16 -0400
Despite the clear comments pointing out that the duplicative code
segments in ReadHead() and _discoverArchiveFormat() needed to be
in sync, they were not: the latter did not bother to apply any of
the sanity checks in the former. We'd missed noticing this partly
because none of those checks would fail in scenarios we customarily
test, and partly because the oversight would be masked if both
segments execute, which they would in cases other than needing to
autodetect the format of a non-seekable stdin source. However,
in a case meeting all these requirements --- for example, trying
to read a newer-than-supported archive format from non-seekable
stdin --- pg_restore missed applying the version check and would
likely dump core or otherwise misbehave.
The whole thing is silly anyway, because there seems little reason
to duplicate the logic beyond the one-line verification that the
file starts with "PGDMP". There seems to have been an undocumented
assumption that multiple major formats (major enough to require
separate reader modules) would nonetheless share the first half-dozen
fields of the custom-format header. This seems unlikely, so let's
fix it by just nuking the duplicate logic in _discoverArchiveFormat().
Also get rid of the pointless attempt to seek back to the start of
the file after successful autodetection. That wastes cycles and
it means we have four behaviors to verify not two.
Per bug #16951 from Sergey Koposov. This has been broken for
decades, so back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_tar.c
Fix internal extract(timezone_minute) formulas
commit : 91e7c903291116bd081abe7d4a058d40a2a06e16
author : Peter Eisentraut <[email protected]>
date : Thu, 1 Apr 2021 16:12:53 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 1 Apr 2021 16:12:53 +0200
Through various refactorings over time, the extract(timezone_minute
from time with time zone) and extract(timezone_minute from timestamp
with time zone) implementations ended up with two different but
equally nonsensical formulas by using SECS_PER_MINUTE and
MINS_PER_HOUR interchangeably. Since those two are of course both the
same number, the formulas do work, but for readability, fix them to be
semantically correct.
M src/backend/utils/adt/date.c
M src/backend/utils/adt/timestamp.c
libpq_pipeline: Must strdup(optarg) to avoid crash
commit : dde1a35aee6266dc8105717275335c46cd2b3650
author : Alvaro Herrera <[email protected]>
date : Thu, 1 Apr 2021 10:04:45 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 1 Apr 2021 10:04:45 -0300
I forgot to strdup() when processing argv[]. Apparently many platforms
hide this mistake from users, but in those that don't you may get a
program crash. Repair.
Per buildfarm member drongo, which is the only one in all the buildfarm
manifesting a problem here.
While at it, move "numrows" processing out of the line of special cases,
and make it getopt's -r instead. (A similar thing could be done to
'conninfo', but current use of the program doesn't warrant spending time
on that -- nowhere else we use conninfo in so simplistic a manner.)
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
Do COPY FROM encoding conversion/verification in larger chunks.
commit : f82de5c46bdf8cd65812a7b04c9509c218e1545d
author : Heikki Linnakangas <[email protected]>
date : Thu, 1 Apr 2021 12:23:40 +0300
committer: Heikki Linnakangas <[email protected]>
date : Thu, 1 Apr 2021 12:23:40 +0300
This gives a small performance gain, by reducing the number of calls
to the conversion/verification function, and letting it work with
larger inputs. Also, reorganizing the input pipeline makes it easier
to parallelize the input parsing: after the input has been converted
to the database encoding, the next stage of finding the newlines can
be done in parallel, because there cannot be any newline chars
"embedded" in multi-byte characters in the encodings that we support
as server encodings.
This changes behavior in one corner case: if client and server
encodings are the same single-byte encoding (e.g. latin1), previously
the input would not be checked for zero bytes ('\0'). Any fields
containing zero bytes would be truncated at the zero. But if encoding
conversion was needed, the conversion routine would throw an error on
the zero. After this commit, the input is always checked for zeros.
Reviewed-by: John Naylor
Discussion: https://www.postgresql.org/message-id/e7861509-3960-538a-9025-b75a61188e01%40iki.fi
M src/backend/commands/copyfrom.c
M src/backend/commands/copyfromparse.c
M src/include/commands/copyfrom_internal.h
M src/include/mb/pg_wchar.h
Add 'noError' argument to encoding conversion functions.
commit : ea1b99a6619cd9dcfd46b82ac0d926b0b80e0ae9
author : Heikki Linnakangas <[email protected]>
date : Thu, 1 Apr 2021 11:45:22 +0300
committer: Heikki Linnakangas <[email protected]>
date : Thu, 1 Apr 2021 11:45:22 +0300
With the 'noError' argument, you can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions. Custom conversions are very rare, there are no
commonly used extensions that I know of that uses that feature. No other
objects can depend on conversions, so if you do have one, you can fairly
easily drop it before upgrading, and recreate it after the upgrade with
an updated version.
Add regression tests for built-in encoding conversions. This doesn't cover
every conversion, but it covers all the internal functions in conv.c that
are used to implement the conversions.
Reviewed-by: John Naylor
Discussion: https://www.postgresql.org/message-id/e7861509-3960-538a-9025-b75a61188e01%40iki.fi
M doc/src/sgml/ref/create_conversion.sgml
M src/backend/commands/conversioncmds.c
M src/backend/utils/error/elog.c
M src/backend/utils/mb/conv.c
M src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
M src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
M src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
M src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
M src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
M src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
M src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
M src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
M src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
M src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
M src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
M src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
M src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
M src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
M src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
M src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
M src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
M src/backend/utils/mb/mbutils.c
M src/bin/pg_upgrade/check.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/mb/pg_wchar.h
M src/test/regress/expected/conversion.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/input/create_function_1.source
M src/test/regress/output/create_function_1.source
M src/test/regress/regress.c
M src/test/regress/sql/conversion.sql
M src/test/regress/sql/opr_sanity.sql
Make extract(timetz) tests a bit more interesting
commit : e2639a767bfa1afebaf1877515a1187feb393443
author : Peter Eisentraut <[email protected]>
date : Thu, 1 Apr 2021 09:52:03 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 1 Apr 2021 09:52:03 +0200
Use a time zone offset with nonzero minutes to make the
timezone_minute test meaningful.
M src/test/regress/expected/timetz.out
M src/test/regress/sql/timetz.sql
doc: Clarify use of ACCESS EXCLUSIVE lock in various sections
commit : ffd3391ea94165fbb5adc9534894c62d41138505
author : Michael Paquier <[email protected]>
date : Thu, 1 Apr 2021 15:28:37 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 1 Apr 2021 15:28:37 +0900
Some sections of the documentation used "exclusive lock" to describe
that an ACCESS EXCLUSIVE lock is taken during a given operation. This
can be confusing to the reader as ACCESS SHARE is allowed with an
EXCLUSIVE lock is used, but that would not be the case with what is
described on those parts of the documentation.
Author: Greg Rychlewski
Discussion: https://postgr.es/m/CAKemG7VptD=7fNWckFMsMVZL_zzvgDO6v2yVmQ+ZiBfc_06kCQ@mail.gmail.com
Backpatch-through: 9.6
M doc/src/sgml/ddl.sgml
M doc/src/sgml/hstore.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/pgrowlocks.sgml
M doc/src/sgml/ref/drop_index.sgml
M doc/src/sgml/ref/reindex.sgml
M doc/src/sgml/ref/vacuum.sgml
Ensure to send a prepare after we detect concurrent abort during decoding.
commit : 4778826532a62fd6e4d3fdeef9532c943604c730
author : Amit Kapila <[email protected]>
date : Thu, 1 Apr 2021 07:57:34 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 1 Apr 2021 07:57:34 +0530
It is possible that while decoding a prepared transaction, it gets aborted
concurrently via a ROLLBACK PREPARED command. In that case, we were
skipping all the changes and directly sending Rollback Prepared when we
find the same in WAL. However, the downstream has no idea of the GID of
such a transaction. So, ensure to send prepare even when a concurrent
abort is detected.
Author: Ajin Cherian
Reviewed-by: Markus Wanner, Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/reorderbuffer.c
Move some client-specific routines from SSLServer to PostgresNode
commit : 0d1a33438d3a88938264e12e94c22818307d2f4d
author : Michael Paquier <[email protected]>
date : Thu, 1 Apr 2021 09:48:17 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 1 Apr 2021 09:48:17 +0900
test_connect_ok() and test_connect_fails() have always been part of the
SSL tests, and check if a connection to the backend should work or not,
and there are sanity checks done on specific error patterns dropped by
libpq if the connection fails.
This was fundamentally wrong on two aspects. First, SSLServer.pm works
mostly on setting up and changing the SSL configuration of a
PostgresNode, and has really nothing to do with the client. Second,
the situation became worse in light of b34ca595, where the SSL tests
would finish by using a psql command that may not come from the same
installation as the node set up.
This commit moves those client routines into PostgresNode, making easier
the refactoring of SSLServer to become more SSL-implementation aware.
This can also be reused by the ldap, kerberos and authentication test
suites for connection checks, and a follow-up patch should extend those
interfaces to match with backend log patterns.
Author: Michael Paquier
Reviewed-by: Andrew Dunstan, Daniel Gustafsson, Álvaro Herrera
Discussion: https://postgr.es/m/[email protected]
M src/test/perl/PostgresNode.pm
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
M src/test/ssl/t/SSLServer.pm
Revert b6002a796
commit : 28b3e3905c982c42fb10ee800e6f881e9742c89d
author : David Rowley <[email protected]>
date : Thu, 1 Apr 2021 13:33:23 +1300
committer: David Rowley <[email protected]>
date : Thu, 1 Apr 2021 13:33:23 +1300
This removes "Add Result Cache executor node". It seems that something
weird is going on with the tracking of cache hits and misses as
highlighted by many buildfarm animals. It's not yet clear what the
problem is as other parts of the plan indicate that the cache did work
correctly, it's just the hits and misses that were being reported as 0.
This is especially a bad time to have the buildfarm so broken, so
reverting before too many more animals go red.
Discussion: https://postgr.es/m/CAApHDvq_hydhfovm4=izgWs+C5HqEeRScjMbOgbpC-jRAeK3Yw@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M src/backend/commands/explain.c
M src/backend/executor/Makefile
M src/backend/executor/execAmi.c
M src/backend/executor/execExpr.c
M src/backend/executor/execParallel.c
M src/backend/executor/execProcnode.c
D src/backend/executor/nodeResultCache.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/executor/executor.h
D src/include/executor/nodeResultCache.h
M src/include/lib/ilist.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/cost.h
M src/include/optimizer/pathnode.h
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/join.out
M src/test/regress/expected/partition_prune.out
D src/test/regress/expected/resultcache.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/sysviews.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/join.sql
M src/test/regress/sql/partition_prune.sql
D src/test/regress/sql/resultcache.sql
Add Result Cache executor node
commit : b6002a796dc0bfe721db5eaa54ba9d24fd9fd416
author : David Rowley <[email protected]>
date : Thu, 1 Apr 2021 12:32:22 +1300
committer: David Rowley <[email protected]>
date : Thu, 1 Apr 2021 12:32:22 +1300
Here we add a new executor node type named "Result Cache". The planner
can include this node type in the plan to have the executor cache the
results from the inner side of parameterized nested loop joins. This
allows caching of tuples for sets of parameters so that in the event that
the node sees the same parameter values again, it can just return the
cached tuples instead of rescanning the inner side of the join all over
again. Internally, result cache uses a hash table in order to quickly
find tuples that have been previously cached.
For certain data sets, this can significantly improve the performance of
joins. The best cases for using this new node type are for join problems
where a large portion of the tuples from the inner side of the join have
no join partner on the outer side of the join. In such cases, hash join
would have to hash values that are never looked up, thus bloating the hash
table and possibly causing it to multi-batch. Merge joins would have to
skip over all of the unmatched rows. If we use a nested loop join with a
result cache, then we only cache tuples that have at least one join
partner on the outer side of the join. The benefits of using a
parameterized nested loop with a result cache increase when there are
fewer distinct values being looked up and the number of lookups of each
value is large. Also, hash probes to lookup the cache can be much faster
than the hash probe in a hash join as it's common that the result cache's
hash table is much smaller than the hash join's due to result cache only
caching useful tuples rather than all tuples from the inner side of the
join. This variation in hash probe performance is more significant when
the hash join's hash table no longer fits into the CPU's L3 cache, but the
result cache's hash table does. The apparent "random" access of hash
buckets with each hash probe can cause a poor L3 cache hit ratio for large
hash tables. Smaller hash tables generally perform better.
The hash table used for the cache limits itself to not exceeding work_mem
* hash_mem_multiplier in size. We maintain a dlist of keys for this cache
and when we're adding new tuples and realize we've exceeded the memory
budget, we evict cache entries starting with the least recently used ones
until we have enough memory to add the new tuples to the cache.
For parameterized nested loop joins, we now consider using one of these
result cache nodes in between the nested loop node and its inner node. We
determine when this might be useful based on cost, which is primarily
driven off of what the expected cache hit ratio will be. Estimating the
cache hit ratio relies on having good distinct estimates on the nested
loop's parameters.
For now, the planner will only consider using a result cache for
parameterized nested loop joins. This works for both normal joins and
also for LATERAL type joins to subqueries. It is possible to use this new
node for other uses in the future. For example, to cache results from
correlated subqueries. However, that's not done here due to some
difficulties obtaining a distinct estimation on the outer plan to
calculate the estimated cache hit ratio. Currently we plan the inner plan
before planning the outer plan so there is no good way to know if a result
cache would be useful or not since we can't estimate the number of times
the subplan will be called until the outer plan is generated.
The functionality being added here is newly introducing a dependency on
the return value of estimate_num_groups() during the join search.
Previously, during the join search, we only ever needed to perform
selectivity estimations. With this commit, we need to use
estimate_num_groups() in order to estimate what the hit ratio on the
result cache will be. In simple terms, if we expect 10 distinct values
and we expect 1000 outer rows, then we'll estimate the hit ratio to be
99%. Since cache hits are very cheap compared to scanning the underlying
nodes on the inner side of the nested loop join, then this will
significantly reduce the planner's cost for the join. However, it's
fairly easy to see here that things will go bad when estimate_num_groups()
incorrectly returns a value that's significantly lower than the actual
number of distinct values. If this happens then that may cause us to make
use of a nested loop join with a result cache instead of some other join
type, such as a merge or hash join. Our distinct estimations have been
known to be a source of trouble in the past, so the extra reliance on them
here could cause the planner to choose slower plans than it did previous
to having this feature. Distinct estimations are also fairly hard to
estimate accurately when several tables have been joined already or when a
WHERE clause filters out a set of values that are correlated to the
expressions we're estimating the number of distinct value for.
For now, the costing we perform during query planning for result caches
does put quite a bit of faith in the distinct estimations being accurate.
When these are accurate then we should generally see faster execution
times for plans containing a result cache. However, in the real world, we
may find that we need to either change the costings to put less trust in
the distinct estimations being accurate or perhaps even disable this
feature by default. There's always an element of risk when we teach the
query planner to do new tricks that it decides to use that new trick at
the wrong time and causes a regression. Users may opt to get the old
behavior by turning the feature off using the enable_resultcache GUC.
Currently, this is enabled by default. It remains to be seen if we'll
maintain that setting for the release.
Additionally, the name "Result Cache" is the best name I could think of
for this new node at the time I started writing the patch. Nobody seems
to strongly dislike the name. A few people did suggest other names but no
other name seemed to dominate in the brief discussion that there was about
names. Let's allow the beta period to see if the current name pleases
enough people. If there's some consensus on a better name, then we can
change it before the release. Please see the 2nd discussion link below
for the discussion on the "Result Cache" name.
Author: David Rowley
Reviewed-by: Andy Fan, Justin Pryzby, Zhihong Yu
Tested-By: Konstantin Knizhnik
Discussion: https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com
Discussion: https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M src/backend/commands/explain.c
M src/backend/executor/Makefile
M src/backend/executor/execAmi.c
M src/backend/executor/execExpr.c
M src/backend/executor/execParallel.c
M src/backend/executor/execProcnode.c
A src/backend/executor/nodeResultCache.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/executor/executor.h
A src/include/executor/nodeResultCache.h
M src/include/lib/ilist.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/cost.h
M src/include/optimizer/pathnode.h
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/join.out
M src/test/regress/expected/partition_prune.out
A src/test/regress/expected/resultcache.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/sysviews.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/join.sql
M src/test/regress/sql/partition_prune.sql
A src/test/regress/sql/resultcache.sql
Remove setvbuf() call from PQtrace()
commit : 6ec578e60101c3c02533f99715945a0400fb3286
author : Alvaro Herrera <[email protected]>
date : Wed, 31 Mar 2021 20:11:51 -0300
committer: Alvaro Herrera <[email protected]>
date : Wed, 31 Mar 2021 20:11:51 -0300
It's misplaced there -- it's not libpq's output stream to tweak in that
way. In particular, POSIX says that it has to be called before any
other operation on the file, so if a stream previously used by the
calling application, bad things may happen.
Put setvbuf() in libpq_pipeline for good measure.
Also, reduce fopen(..., "w+") to just fopen(..., "w") in
libpq_pipeline.c. It's not clear that this fixes anything, but we don't
use w+ anywhere.
Per complaints from Tom Lane.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-trace.c
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Initialize conn->Pfdebug to NULL when creating a connection
commit : aba24b51cc1b045a9810458b4bb15fee2c182948
author : Alvaro Herrera <[email protected]>
date : Wed, 31 Mar 2021 19:16:58 -0300
committer: Alvaro Herrera <[email protected]>
date : Wed, 31 Mar 2021 19:16:58 -0300
Failing to do this can cause a crash, and I suspect is what has happened
with a buildfarm member reporting mysterious failures.
This is an ancient bug, but I'm not backpatching since evidently nobody
cares about PQtrace in older releases.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-connect.c
Disable force_parallel_mode in libpq_pipeline
commit : a6d3dea8e5e0c8a0df2f95d66b6c3903a4354ca0
author : Alvaro Herrera <[email protected]>
date : Wed, 31 Mar 2021 18:45:17 -0300
committer: Alvaro Herrera <[email protected]>
date : Wed, 31 Mar 2021 18:45:17 -0300
Some buildfarm animals with force_parallel_mode=regress were failing
this test because the error is reported in a parallel worker quicker
than the rows that succeed.
Take the opportunity to move the SET of lc_messages out of the traced
section, because it's not very interesting.
Diagnosed-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace
M src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
M src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
M src/test/modules/libpq_pipeline/traces/prepared.trace
M src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
M src/test/modules/libpq_pipeline/traces/singlerow.trace
M src/test/modules/libpq_pipeline/traces/transaction.trace
Fix unportable use of isprint().
commit : 9e20406dd847d0f8c1cbd803786c6d0ad33bcbdd
author : Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 17:14:16 -0400
committer: Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 17:14:16 -0400
We must cast the arguments of <ctype.h> functions to unsigned
char to avoid problems where char is signed.
Speaking of which, considering that this *is* a <ctype.h> function,
it's rather remarkable that we aren't seeing more complaints about
not having included that header.
Per buildfarm.
M src/interfaces/libpq/fe-trace.c
Fix portability and safety issues in pqTraceFormatTimestamp.
commit : f1be740a991406d7885047beb971e1ff5dbe8b71
author : Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 17:00:30 -0400
committer: Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 17:00:30 -0400
Remove confusion between time_t and pg_time_t; neither
gettimeofday() nor localtime() deal in the latter.
libpq indeed has no business using <pgtime.h> at all.
Use snprintf not sprintf, to ensure we can't overrun the
supplied buffer. (Unlikely, but let's be safe.)
Per buildfarm.
M src/interfaces/libpq/fe-trace.c
Silence compiler warning in non-assert builds.
commit : 8998e3cafa23632790787b8cc726998e84067259
author : Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 16:50:45 -0400
committer: Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 16:50:45 -0400
Per buildfarm.
M contrib/postgres_fdw/postgres_fdw.c
Don't prematurely cram a value into a short int.
commit : c545e9524dcfcfce25c370f584b31562e8d7a4b7
author : Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 16:45:17 -0400
committer: Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 16:45:17 -0400
Since a4d75c86b, some buildfarm members have been warning that
Assert(attnum <= MaxAttrNumber);
is useless if attnum is an AttrNumber. I'm not certain how plausible
it is that the value coming out of the bitmap could actually exceed
MaxAttrNumber, but we seem to have thought that that was possible back
in 7300a6995. Revert the intermediate variable to int so that we have
the same overflow protection as before.
M src/backend/statistics/extended_stats.c
Add a docs section for obsoleted and renamed functions and settings
commit : 3b0c647bbfc52894d979976f1e6d60e40649bba7
author : Stephen Frost <[email protected]>
date : Wed, 31 Mar 2021 16:23:25 -0400
committer: Stephen Frost <[email protected]>
date : Wed, 31 Mar 2021 16:23:25 -0400
The new appendix groups information on renamed or removed settings,
commands, etc into an out-of-the-way part of the docs.
The original id elements are retained in each subsection to ensure that
the same filenames are produced for HTML docs. This prevents /current/
links on the web from breaking, and allows users of the web docs
to follow links from old version pages to info on the changes in the
new version. Prior to this change, a link to /current/ for renamed
sections like the recovery.conf docs would just 404. Similarly if
someone searched for recovery.conf they would find the pg11 docs,
but there would be no /12/ or /current/ link, so they couldn't easily
find out that it was removed in pg12 or how to adapt.
Index entries are also added so that there's a breadcrumb trail for
users to follow when they know the old name, but not what we changed it
to. So a user who is trying to find out how to set standby_mode in
PostgreSQL 12+, or where pg_resetxlog went, now has more chance of
finding that information.
Craig Ringer and Stephen Frost
Reviewed-by: Euler Taveira
Discussion: https://postgr.es/m/CAGRY4nzPNOyYQ_1-pWYToUVqQ0ThqP5jdURnJMZPm539fdizOg%40mail.gmail.com
Backpatch-through: 10
A doc/src/sgml/appendix-obsolete-pgreceivexlog.sgml
A doc/src/sgml/appendix-obsolete-pgresetxlog.sgml
A doc/src/sgml/appendix-obsolete-pgxlogdump.sgml
A doc/src/sgml/appendix-obsolete-recovery-config.sgml
A doc/src/sgml/appendix-obsolete.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/filelist.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/postgres.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
Suppress compiler warning in libpq_pipeline.c.
commit : 522d1a89f8d7ed45681988c60bd0a687332a4023
author : Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 15:30:04 -0400
committer: Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 15:30:04 -0400
Some compilers seem to be concerned about the possibility that
recv_step is not any of the defined enum values. Silence
warnings about uninitialized cmdtag in a different way than
I did in 9fb9691a8.
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Improve style of some replication-related error messages.
commit : 6197db5340b8154adce1c6d07f6d3325547429c1
author : Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 15:25:53 -0400
committer: Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 15:25:53 -0400
Put the remote end's error message into the primary error string,
instead of relegating it to errdetail(). Although this could end up
being awkward if the remote sends us a really long error message,
it seems more in keeping with our message style guidelines, and more
helpful in situations where the errdetail could get dropped.
Peter Smith
Discussion: https://postgr.es/m/CAHut+Ps-Qv2yQceCwobQDP0aJOkfDzRFrOaR6+2Op2K=WHGeWg@mail.gmail.com
M src/backend/commands/subscriptioncmds.c
M src/backend/replication/logical/tablesync.c
Fix some libpq_pipeline test problems
commit : db973ffb3ca43e65a0bf15175a35184a53bf977d
author : Alvaro Herrera <[email protected]>
date : Wed, 31 Mar 2021 15:13:42 -0300
committer: Alvaro Herrera <[email protected]>
date : Wed, 31 Mar 2021 15:13:42 -0300
Test pipeline_abort was not checking that it got the rows it expected in
one mode; make it do so. This doesn't fix the actual problem (no idea
what that is, yet) but at least it should make it more obvious rather
than being visible only as a difference in the trace output.
While at it, fix other infelicities in the test:
* I reversed the order of result vs. expected in like().
* The output traces from -t are being put in the log dir, which means
the buildfarm script uselessly captures them. Put them in a separate
dir tmp_check/traces instead, to avoid cluttering the buildfarm results.
* Test pipelined_insert was using too large a row count. Reduce that a
tad and add a filler column to make each insert a little bulkier, while
still keeping enough that a buffer is filled and we have to switch mode.
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
M src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
Fix has_column_privilege function corner case
commit : b12bd4869b5e64b742a69ca07915e2f77f85a9ae
author : Joe Conway <[email protected]>
date : Wed, 31 Mar 2021 13:55:25 -0400
committer: Joe Conway <[email protected]>
date : Wed, 31 Mar 2021 13:55:25 -0400
According to the comments, when an invalid or dropped column oid is passed
to has_column_privilege(), the intention has always been to return NULL.
However, when the caller had table level privilege the invalid/missing
column was never discovered, because table permissions were checked first.
Fix that by introducing extended versions of pg_attribute_acl(check|mask)
and pg_class_acl(check|mask) which take a new argument, is_missing. When
is_missing is NULL, the old behavior is preserved. But when is_missing is
passed by the caller, no ERROR is thrown for dropped or missing
columns/relations, and is_missing is flipped to true. This in turn allows
has_column_privilege to check for column privileges first, providing the
desired semantics.
Not backpatched since it is a user visible behavioral change with no previous
complaints, and the fix is a bit on the invasive side.
Author: Joe Conway
Reviewed-By: Tom Lane
Reported by: Ian Barwick
Discussion: https://postgr.es/m/flat/9b5f4311-157b-4164-7fe7-077b4fe8ed84%40joeconway.com
M src/backend/catalog/aclchk.c
M src/backend/utils/adt/acl.c
M src/include/utils/acl.h
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Rework planning and execution of UPDATE and DELETE.
commit : 86dc90056dfdbd9d1b891718d2e5614e3e432f35
author : Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 11:52:34 -0400
committer: Tom Lane <[email protected]>
date : Wed, 31 Mar 2021 11:52:34 -0400
This patch makes two closely related sets of changes:
1. For UPDATE, the subplan of the ModifyTable node now only delivers
the new values of the changed columns (i.e., the expressions computed
in the query's SET clause) plus row identity information such as CTID.
ModifyTable must re-fetch the original tuple to merge in the old
values of any unchanged columns. The core advantage of this is that
the changed columns are uniform across all tables of an inherited or
partitioned target relation, whereas the other columns might not be.
A secondary advantage, when the UPDATE involves joins, is that less
data needs to pass through the plan tree. The disadvantage of course
is an extra fetch of each tuple to be updated. However, that seems to
be very nearly free in context; even worst-case tests don't show it to
add more than a couple percent to the total query cost. At some point
it might be interesting to combine the re-fetch with the tuple access
that ModifyTable must do anyway to mark the old tuple dead; but that
would require a good deal of refactoring and it seems it wouldn't buy
all that much, so this patch doesn't attempt it.
2. For inherited UPDATE/DELETE, instead of generating a separate
subplan for each target relation, we now generate a single subplan
that is just exactly like a SELECT's plan, then stick ModifyTable
on top of that. To let ModifyTable know which target relation a
given incoming row refers to, a tableoid junk column is added to
the row identity information. This gets rid of the horrid hack
that was inheritance_planner(), eliminating O(N^2) planning cost
and memory consumption in cases where there were many unprunable
target relations.
Point 2 of course requires point 1, so that there is a uniform
definition of the non-junk columns to be returned by the subplan.
We can't insist on uniform definition of the row identity junk
columns however, if we want to keep the ability to have both
plain and foreign tables in a partitioning hierarchy. Since
it wouldn't scale very far to have every child table have its
own row identity column, this patch includes provisions to merge
similar row identity columns into one column of the subplan result.
In particular, we can merge the whole-row Vars typically used as
row identity by FDWs into one column by pretending they are type
RECORD. (It's still okay for the actual composite Datums to be
labeled with the table's rowtype OID, though.)
There is more that can be done to file down residual inefficiencies
in this patch, but it seems to be committable now.
FDW authors should note several API changes:
* The argument list for AddForeignUpdateTargets() has changed, and so
has the method it must use for adding junk columns to the query. Call
add_row_identity_var() instead of manipulating the parse tree directly.
You might want to reconsider exactly what you're adding, too.
* PlanDirectModify() must now work a little harder to find the
ForeignScan plan node; if the foreign table is part of a partitioning
hierarchy then the ForeignScan might not be the direct child of
ModifyTable. See postgres_fdw for sample code.
* To check whether a relation is a target relation, it's no
longer sufficient to compare its relid to root->parse->resultRelation.
Instead, check it against all_result_relids or leaf_result_relids,
as appropriate.
Amit Langote and Tom Lane
Discussion: https://postgr.es/m/CA+HiwqHpHdqdDn48yCEhynnniahH78rwcrv1rEX65-fsZGBOLQ@mail.gmail.com
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M doc/src/sgml/ddl.sgml
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/postgres-fdw.sgml
M src/backend/commands/copyfrom.c
M src/backend/commands/explain.c
M src/backend/commands/trigger.c
M src/backend/executor/README
M src/backend/executor/execExpr.c
M src/backend/executor/execJunk.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/planmain.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/optimizer/prep/preptlist.c
M src/backend/optimizer/util/appendinfo.c
M src/backend/optimizer/util/inherit.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/plancat.c
M src/backend/optimizer/util/relnode.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/utils/adt/ruleutils.c
M src/include/executor/executor.h
M src/include/foreign/fdwapi.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/nodes/primnodes.h
M src/include/optimizer/appendinfo.h
M src/include/optimizer/pathnode.h
M src/include/optimizer/prep.h
M src/include/rewrite/rewriteHandler.h
M src/test/regress/expected/inherit.out
M src/test/regress/expected/insert_conflict.out
M src/test/regress/expected/partition_join.out
M src/test/regress/expected/partition_prune.out
M src/test/regress/expected/rowsecurity.out
M src/test/regress/expected/updatable_views.out
M src/test/regress/expected/update.out
M src/test/regress/expected/with.out
Allow an alias to be attached to a JOIN ... USING
commit : 055fee7eb4dcc78e58672aef146334275e1cc40d
author : Peter Eisentraut <[email protected]>
date : Wed, 31 Mar 2021 17:09:24 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 31 Mar 2021 17:09:24 +0200
This allows something like
SELECT ... FROM t1 JOIN t2 USING (a, b, c) AS x
where x has the columns a, b, c and unlike a regular alias it does not
hide the range variables of the tables being joined t1 and t2.
Per SQL:2016 feature F404 "Range variable for common column names".
Reviewed-by: Vik Fearing <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/ref/select.sgml
M src/backend/catalog/sql_features.txt
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_relation.c
M src/backend/utils/adt/ruleutils.c
M src/include/nodes/parsenodes.h
M src/include/nodes/primnodes.h
M src/include/parser/parse_node.h
M src/include/parser/parse_relation.h
M src/test/regress/expected/create_view.out
M src/test/regress/expected/join.out
M src/test/regress/sql/create_view.sql
M src/test/regress/sql/join.sql
Add support for asynchronous execution.
commit : 27e1f14563cf982f1f4d71e21ef247866662a052
author : Etsuro Fujita <[email protected]>
date : Wed, 31 Mar 2021 18:45:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Wed, 31 Mar 2021 18:45:00 +0900
This implements asynchronous execution, which runs multiple parts of a
non-parallel-aware Append concurrently rather than serially to improve
performance when possible. Currently, the only node type that can be
run concurrently is a ForeignScan that is an immediate child of such an
Append. In the case where such ForeignScans access data on different
remote servers, this would run those ForeignScans concurrently, and
overlap the remote operations to be performed simultaneously, so it'll
improve the performance especially when the operations involve
time-consuming ones such as remote join and remote aggregation.
We may extend this to other node types such as joins or aggregates over
ForeignScans in the future.
This also adds the support for postgres_fdw, which is enabled by the
table-level/server-level option "async_capable". The default is false.
Robert Haas, Kyotaro Horiguchi, Thomas Munro, and myself. This commit
is mostly based on the patch proposed by Robert Haas, but also uses
stuff from the patch proposed by Kyotaro Horiguchi and from the patch
proposed by Thomas Munro. Reviewed by Kyotaro Horiguchi, Konstantin
Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro, Justin Pryzby, and
others.
Discussion: https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com
Discussion: https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/postgres-fdw.sgml
M src/backend/commands/explain.c
M src/backend/executor/Makefile
M src/backend/executor/README
M src/backend/executor/execAmi.c
A src/backend/executor/execAsync.c
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeForeignscan.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/createplan.c
M src/backend/postmaster/pgstat.c
M src/backend/storage/ipc/latch.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
A src/include/executor/execAsync.h
M src/include/executor/nodeAppend.h
M src/include/executor/nodeForeignscan.h
M src/include/foreign/fdwapi.h
M src/include/nodes/execnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/cost.h
M src/include/pgstat.h
M src/include/storage/latch.h
M src/test/regress/expected/explain.out
M src/test/regress/expected/incremental_sort.out
M src/test/regress/expected/insert_conflict.out
M src/test/regress/expected/sysviews.out
Add p_names field to ParseNamespaceItem
commit : 66392d396508c91c2ec07a61568bf96acb663ad8
author : Peter Eisentraut <[email protected]>
date : Wed, 31 Mar 2021 10:52:37 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 31 Mar 2021 10:52:37 +0200
ParseNamespaceItem had a wired-in assumption that p_rte->eref
describes the table and column aliases exposed by the nsitem. This
relaxes this by creating a separate p_names field in an nsitem. This
is mainly preparation for a patch for JOIN USING aliases, but it saves
one indirection in common code paths, so it's possibly a win on its
own.
Author: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_relation.c
M src/include/parser/parse_node.h
Add errhint_plural() function and make use of it
commit : 91c5a8caaa61055959aa5fb68a00e5f690e39a34
author : Peter Eisentraut <[email protected]>
date : Wed, 31 Mar 2021 09:15:51 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 31 Mar 2021 09:15:51 +0200
Similar to existing errmsg_plural() and errdetail_plural(). Some
errhint() calls hadn't received the proper plural treatment yet.
M doc/src/sgml/sources.sgml
M src/backend/parser/parse_func.c
M src/backend/utils/error/elog.c
M src/include/utils/elog.h
M src/nls-global.mk
doc: Remove Cyrillic from unistr example
commit : 287d2a97c1de07486e4525c8ad06258f04bd6268
author : Peter Eisentraut <[email protected]>
date : Wed, 31 Mar 2021 08:20:35 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 31 Mar 2021 08:20:35 +0200
Not supported by PDF build right now, so let's do without it.
M doc/src/sgml/func.sgml
Remove extra semicolon in postgres_fdw tests.
commit : 13cb5bd84657ed49021fe6fc4ce46601c315c9a5
author : Amit Kapila <[email protected]>
date : Wed, 31 Mar 2021 10:36:39 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 31 Mar 2021 10:36:39 +0530
Author: Suraj Kharage
Reviewed-by: Bharath Rupireddy, Vignesh C
Discussion: https://postgr.es/m/CAF1DzPWRfxUeH-wShz7P_pK5Tx6M_nEK+TkS8gn5ngvg07Q5=g@mail.gmail.com
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Doc: Use consistent terminology for tablesync slots.
commit : 9f45631766bd0c51a74102770737ba3b0561977e
author : Amit Kapila <[email protected]>
date : Wed, 31 Mar 2021 08:17:50 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 31 Mar 2021 08:17:50 +0530
At some places in the docs, we refer to them as tablesync slots and at other
places as table synchronization slots. For consistency, we refer to them as
table synchronization slots at all places.
Author: Peter Smith
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAHut+PvzYNKCeZ=kKBDkh3dw-r=2D3fk=nNc9SXSW=CZGk69xg@mail.gmail.com
M doc/src/sgml/ref/alter_subscription.sgml
Accept slightly-filled pages for tuples larger than fillfactor.
commit : 0ff8bbdee19a9db2794a95d439c946ab017d0acd
author : Noah Misch <[email protected]>
date : Tue, 30 Mar 2021 18:53:44 -0700
committer: Noah Misch <[email protected]>
date : Tue, 30 Mar 2021 18:53:44 -0700
We always inserted a larger-than-fillfactor tuple into a newly-extended
page, even when existing pages were empty or contained nothing but an
unused line pointer. This was unnecessary relation extension. Start
tolerating page usage up to 1/8 the maximum space that could be taken up
by line pointers. This is somewhat arbitrary, but it should allow more
cases to reuse pages. This has no effect on tables with fillfactor=100
(the default).
John Naylor and Floris van Nee. Reviewed by Matthias van de Meent.
Reported by Floris van Nee.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/hio.c
M src/backend/access/heap/rewriteheap.c
M src/test/regress/expected/insert.out
M src/test/regress/sql/insert.sql
Fix comment in parsenodes.h
commit : 7ef64e7e72a65f191fc2f7d4bbe220f53dd8d5de
author : Michael Paquier <[email protected]>
date : Wed, 31 Mar 2021 09:35:58 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 31 Mar 2021 09:35:58 +0900
CreateStmt->inhRelations is a list of RangeVars, but a comment was
incorrect about that.
Author: Julien Rouhaud
Discussion: https://postgr.es/m/20210330123015.yzekhz5sweqbgxdr@nol
M src/include/nodes/parsenodes.h
Add support for --extension in pg_dump
commit : 6568cef26e0f40c25ae54b8e20aad8d1410a854b
author : Michael Paquier <[email protected]>
date : Wed, 31 Mar 2021 09:12:34 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 31 Mar 2021 09:12:34 +0900
When specified, only extensions matching the given pattern are included
in dumps. Similarly to --table and --schema, when --strict-names is
used, a perfect match is required. Also, like the two other options,
this new option offers no guarantee that dependent objects have been
dumped, so a restore may fail on a clean database.
Tests are added in test_pg_dump/, checking after a set of positive and
negative cases, with or without an extension's contents added to the
dump generated.
Author: Guillaume Lelarge
Reviewed-by: David Fetter, Tom Lane, Michael Paquier, Asif Rehman,
Julien Rouhaud
Discussion: https://postgr.es/m/CAECtzeXOt4cnMU5+XMZzxBPJ_wu76pNy6HZKPRBL-j7yj1E4+g@mail.gmail.com
M doc/src/sgml/ref/pg_dump.sgml
M src/bin/pg_dump/pg_dump.c
M src/test/modules/test_pg_dump/t/001_base.pl
Remove small inefficiency in ExecARDeleteTriggers/ExecARUpdateTriggers.
commit : 65158f497a7d7523ad438b2034d01a560fafe6bd
author : Tom Lane <[email protected]>
date : Tue, 30 Mar 2021 20:01:27 -0400
committer: Tom Lane <[email protected]>
date : Tue, 30 Mar 2021 20:01:27 -0400
Whilst poking at nodeModifyTable.c, I chanced to notice that while
its calls to ExecBR*Triggers and ExecIR*Triggers are protected by
tests to see if there are any relevant triggers to fire, its calls
to ExecAR*Triggers are not; the latter functions do the equivalent
tests themselves. This seems possibly reasonable given the more
complex conditions involved, but what's less reasonable is that
the ExecAR* functions aren't careful to do no work when there is
no work to be done. ExecARInsertTriggers gets this right, but the
other two will both force creation of a slot that the query may
have no use for. ExecARUpdateTriggers additionally performed a
usually-useless ExecClearTuple() on that slot. This is probably
all pretty microscopic in real workloads, but a cycle shaved is a
cycle earned.
M src/backend/commands/trigger.c
adjust dblink regression expected output for commit 5da9868ed9
commit : 9ee7d533dacf8594057ced2d016250f09056c284
author : Bruce Momjian <[email protected]>
date : Tue, 30 Mar 2021 19:46:22 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 30 Mar 2021 19:46:22 -0400
Seems the -1/singular output is used in the dblink regression tests.
Reported-by: Álvaro Herrera
Discussion: https://postgr.es/m/[email protected]
M contrib/dblink/expected/dblink.out
libpq_pipeline: add PQtrace() support and tests
commit : 7bebd0d00998a28449d83376f4bcdeec65d5eea6
author : Alvaro Herrera <[email protected]>
date : Tue, 30 Mar 2021 20:33:04 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 30 Mar 2021 20:33:04 -0300
The libpq_pipeline program recently introduced by commit acb7e4eb6b1c
is well equipped to test the PQtrace() functionality, so let's make it
do that.
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/libpq_pipeline/libpq_pipeline.c
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
A src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace
A src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
A src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
A src/test/modules/libpq_pipeline/traces/prepared.trace
A src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
A src/test/modules/libpq_pipeline/traces/singlerow.trace
A src/test/modules/libpq_pipeline/traces/transaction.trace
Improve PQtrace() output format
commit : 198b3716dba68544b55cb97bd120738a86d5df2d
author : Alvaro Herrera <[email protected]>
date : Tue, 30 Mar 2021 20:12:34 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 30 Mar 2021 20:12:34 -0300
Transform the PQtrace output format from its ancient (and mostly
useless) byte-level output format to a logical-message-level output,
making it much more usable. This implementation allows the printing
code to be written (as it indeed was) by looking at the protocol
documentation, which gives more confidence that the three (docs, trace
code and actual code) actually match.
Author: 岩田 彩 (Aya Iwata) <[email protected]>
Reviewed-by: 綱川 貴之 (Takayuki Tsunakawa) <[email protected]>
Reviewed-by: Kirk Jamison <[email protected]>
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Reviewed-by: 黒田 隼人 (Hayato Kuroda) <[email protected]>
Reviewed-by: "Nagaura, Ryohei" <[email protected]>
Reviewed-by: Ryo Matsumura <[email protected]>
Reviewed-by: Greg Nancarrow <[email protected]>
Reviewed-by: Jim Doty <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/71E660EB361DF14299875B198D4CE5423DE3FBA4@g01jpexmbkw25
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/Makefile
M src/interfaces/libpq/exports.txt
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-misc.c
M src/interfaces/libpq/fe-protocol3.c
A src/interfaces/libpq/fe-trace.c
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
In messages, use singular nouns for -1, like we do for +1.
commit : 5da9868ed983f95cc1cff80dcd81252a513774f8
author : Bruce Momjian <[email protected]>
date : Tue, 30 Mar 2021 18:34:27 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 30 Mar 2021 18:34:27 -0400
This outputs "-1 year", not "-1 years".
Reported-by: [email protected]
Bug: 16939
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/datetime.c
M src/interfaces/ecpg/pgtypeslib/interval.c
M src/interfaces/libpq/fe-print.c
M src/test/regress/expected/interval.out
Add tests for date_part of epoch near upper bound of timestamp range
commit : 6131ffc43ff3d2f566e93f017e56a09e4e717318
author : Peter Eisentraut <[email protected]>
date : Tue, 30 Mar 2021 22:05:18 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 30 Mar 2021 22:05:18 +0200
This exercises a special case in the implementations of
date_part('epoch', timestamp[tz]) that was previously not tested.
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Use a WaitLatch for vacuum/autovacuum sleeping
commit : 4753ef37e0eda4ba0af614022d18fcbc5a946cc9
author : Stephen Frost <[email protected]>
date : Tue, 30 Mar 2021 12:52:56 -0400
committer: Stephen Frost <[email protected]>
date : Tue, 30 Mar 2021 12:52:56 -0400
Instead of using pg_usleep() in vacuum_delay_point(), use a WaitLatch.
This has the advantage that we will realize if the postmaster has been
killed since the last time we decided to sleep while vacuuming.
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/CAFh8B=kcdk8k-Y21RfXPu5dX=bgPqJ8TC3p_qxR_ygdBS=JN5w@mail.gmail.com
M src/backend/commands/vacuum.c
Further tweaking of pg_dump's handling of default_toast_compression.
commit : 54bb91c30e3964fd81059e6b02e377cc9dd2d64c
author : Tom Lane <[email protected]>
date : Tue, 30 Mar 2021 10:57:57 -0400
committer: Tom Lane <[email protected]>
date : Tue, 30 Mar 2021 10:57:57 -0400
As committed in bbe0a81db, pg_dump from a pre-v14 server effectively
acts as though you'd said --no-toast-compression. I think the right
thing is for it to act as though default_toast_compression is set to
"pglz", instead, so that the tables' toast compression behavior is
preserved. You can always get the other behavior, if you want that,
by giving the switch.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_dump.c
Allow estimate_num_groups() to pass back further details about the estimation
commit : ed934d4fa30f0f94e6f7125ad2154e6a58d1c7f7
author : David Rowley <[email protected]>
date : Tue, 30 Mar 2021 20:52:46 +1300
committer: David Rowley <[email protected]>
date : Tue, 30 Mar 2021 20:52:46 +1300
Here we add a new output parameter to estimate_num_groups() to allow it to
inform the caller of additional, possibly useful information about the
estimation.
The new output parameter is a struct that currently contains just a single
field with a set of flags. This was done rather than having the flags as
an output parameter to allow future fields to be added without having to
change the signature of the function at a later date when we want to pass
back further information that might not be suitable to store in the flags
field.
It seems reasonable that one day in the future that the planner would want
to know more about the estimation. For example, how many individual sets
of statistics was the estimation generated from? The planner may want to
take that into account if we ever want to consider risks as well as costs
when generating plans.
For now, there's only 1 flag we set in the flags field. This is to
indicate if the estimation fell back on using the hard-coded constants in
any part of the estimation. Callers may like to change their behavior if
this is set, and this gives them the ability to do so. Callers may pass
the flag pointer as NULL if they have no interest in obtaining any
additional information about the estimate.
We're not adding any actual usages of these flags here. Some follow-up
commits will make use of this feature. Additionally, we're also not
making any changes to add support for clauselist_selectivity() and
clauselist_selectivity_ext(). However, if this is required in the future
then the same struct being added here should be fine to use as a new
output argument for those functions too.
Author: David Rowley
Discussion: https://postgr.es/m/CAApHDvqQqpk=1W-G_ds7A9CsXX3BggWj_7okinzkLVhDubQzjA@mail.gmail.com
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/pathnode.c
M src/backend/utils/adt/selfuncs.c
M src/include/utils/selfuncs.h
Fix compiler warning in unistr function
commit : efd9d92bb39c74c2aded64fc08e2d601ce20c39d
author : David Rowley <[email protected]>
date : Tue, 30 Mar 2021 20:28:09 +1300
committer: David Rowley <[email protected]>
date : Tue, 30 Mar 2021 20:28:09 +1300
Some compilers are not aware that elog/ereport ERROR does not return.
M src/backend/utils/adt/varlena.c
Allow users of simplehash.h to perform direct deletions
commit : ff53d7b159b93ce9fc884897f9d96b97744781e2
author : David Rowley <[email protected]>
date : Tue, 30 Mar 2021 19:56:50 +1300
committer: David Rowley <[email protected]>
date : Tue, 30 Mar 2021 19:56:50 +1300
Previously simplehash.h only exposed a method to perform a hash table
delete using the hash table key. This meant that the delete function had
to perform a hash lookup in order to find the entry to delete. Here we
add a new function so that users of simplehash.h can perform a hash delete
directly using the entry pointer, thus saving the hash lookup.
An upcoming patch that uses simplehash.h already has performed the hash
lookup so already has the entry pointer. This change will allow the
code in that patch to perform the hash delete without the code in
simplehash.h having to perform an additional hash lookup.
Author: David Rowley
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/CAApHDvqFLXXge153WmPsjke5VGOSt7Ez0yD0c7eBXLfmWxs3Kw@mail.gmail.com
M src/include/lib/simplehash.h
Add upper boundary tests for timestamp and timestamptz types
commit : bc9f1afdebc98b490d0a00468d75e8e4d080afb0
author : Peter Eisentraut <[email protected]>
date : Tue, 30 Mar 2021 08:46:34 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 30 Mar 2021 08:46:34 +0200
The existing regression tests only tested the lower boundary of the
range supported by the timestamp and timestamptz types because "The
upper boundary differs between integer and float timestamps, so no
check". Since this is obsolete, add similar tests for the upper
boundary.
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Add a xid argument to the filter_prepare callback for output plugins.
commit : f64ea6dc5c8ccaec9a3d3d39695ca261febb6883
author : Amit Kapila <[email protected]>
date : Tue, 30 Mar 2021 10:34:43 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 30 Mar 2021 10:34:43 +0530
Along with gid, this provides a different way to identify the transaction.
The users that use xid in some way to prepare the transactions can use it
to filter prepare transactions. The later commands COMMIT PREPARED or
ROLLBACK PREPARED carries both identifiers, providing an output plugin the
choice of what to use.
Author: Markus Wanner
Reviewed-by: Vignesh C, Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/test_decoding.c
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/logical.c
M src/include/replication/logical.h
M src/include/replication/output_plugin.h
Update obsolete comment.
commit : bc2797ebb14bae663da1ee7845774dd98716c0d0
author : Etsuro Fujita <[email protected]>
date : Tue, 30 Mar 2021 13:00:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Tue, 30 Mar 2021 13:00:00 +0900
Back-patch to all supported branches.
Author: Etsuro Fujita
Discussion: https://postgr.es/m/CAPmGK17DwzaSf%2BB71dhL2apXdtG-OmD6u2AL9Cq2ZmAR0%2BzapQ%40mail.gmail.com
M contrib/postgres_fdw/postgres_fdw.c
psql: call clearerr() just before printing
commit : 8d645a116ef6e04bfb03e259149b8e163dbdf50c
author : Alvaro Herrera <[email protected]>
date : Mon, 29 Mar 2021 18:34:39 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 29 Mar 2021 18:34:39 -0300
We were never doing clearerr() on the output stream, which results in a
message being printed after each result once an EOF is seen:
could not print result table: Success
This message was added by commit b03436994bcc (in the pg13 era); before
that, the error indicator would never be examined. So backpatch only
that far back, even though the actual bug (to wit: the fact that the
error indicator is never cleared) is older.
M src/fe_utils/print.c
Adjust design of per-worker parallel seqscan data struct
commit : af527705edc3fd0b335264d17e0521c05edc5cca
author : David Rowley <[email protected]>
date : Tue, 30 Mar 2021 10:17:09 +1300
committer: David Rowley <[email protected]>
date : Tue, 30 Mar 2021 10:17:09 +1300
The design of the data structures which allow storage of the per-worker
memory during parallel seq scans were not ideal. The work done in
56788d215 required an additional data structure to allow workers to
remember the range of pages that had been allocated to them for
processing during a parallel seqscan. That commit added a void pointer
field to TableScanDescData to allow heapam to store the per-worker
allocation information. However putting the field there made very little
sense given that we have AM specific structs for that, e.g.
HeapScanDescData.
Here we remove the void pointer field from TableScanDescData and add a
dedicated field for this purpose to HeapScanDescData.
Previously we also allocated memory for this parallel per-worker data for
all scans, regardless if it was a parallel scan or not. This was just a
wasted allocation for non-parallel scans, so here we make the allocation
conditional on the scan being parallel.
Also, add previously missing pfree() to free the per-worker data in
heap_endscan().
Reported-by: Andres Freund
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam.c
M src/include/access/heapam.h
M src/include/access/relscan.h
Allow matching the DN of a client certificate for authentication
commit : 6d7a6feac48b1970c4cd127ee65d4c487acbb5e9
author : Andrew Dunstan <[email protected]>
date : Mon, 29 Mar 2021 15:31:22 -0400
committer: Andrew Dunstan <[email protected]>
date : Mon, 29 Mar 2021 15:31:22 -0400
Currently we only recognize the Common Name (CN) of a certificate's
subject to be matched against the user name. Thus certificates with
subjects '/OU=eng/CN=fred' and '/OU=sales/CN=fred' will have the same
connection rights. This patch provides an option to match the whole
Distinguished Name (DN) instead of just the CN. On any hba line using
client certificate identity, there is an option 'clientname' which can
have values of 'DN' or 'CN'. The default is 'CN', the current procedure.
The DN is matched against the RFC2253 formatted DN, which looks like
'CN=fred,OU=eng'.
This facility of probably best used in conjunction with an ident map.
Discussion: https://postgr.es/m/[email protected]
Reviewed-By: Michael Paquier, Daniel Gustafsson, Jacob Champion
M doc/src/sgml/client-auth.sgml
M src/backend/libpq/auth.c
M src/backend/libpq/be-secure-openssl.c
M src/backend/libpq/be-secure.c
M src/backend/libpq/hba.c
M src/include/libpq/hba.h
M src/include/libpq/libpq-be.h
M src/test/ssl/Makefile
A src/test/ssl/client-dn.config
A src/test/ssl/ssl/client-dn.crt
A src/test/ssl/ssl/client-dn.key
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/SSLServer.pm
Clean up date_part tests a bit
commit : efcc7572f532ea564fedc6359c2df43045ee7908
author : Peter Eisentraut <[email protected]>
date : Mon, 29 Mar 2021 17:53:30 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 29 Mar 2021 17:53:30 +0200
Some tests for timestamp and timestamptz were in the date.sql test
file. Move them to their appropriate files, or drop tests cases that
were already present there.
M src/test/regress/expected/date.out
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/date.sql
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Add unistr function
commit : f37fec837ce8bf7af408ba66d32099e5a0182402
author : Peter Eisentraut <[email protected]>
date : Sun, 28 Mar 2021 08:16:15 +0200
committer: Peter Eisentraut <[email protected]>
date : Sun, 28 Mar 2021 08:16:15 +0200
This allows decoding a string with Unicode escape sequences. It is
similar to Unicode escape strings, but offers some more flexibility.
Author: Pavel Stehule <[email protected]>
Reviewed-by: Asif Rehman <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CAFj8pRA5GnKT+gDVwbVRH2ep451H_myBt+NTz8RkYUARE9+qOQ@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/varlena.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
Reset standard_conforming_strings in strings test
commit : ebedd0c78fc51c293abe56e99a18c67af14da0c9
author : Peter Eisentraut <[email protected]>
date : Mon, 29 Mar 2021 08:40:39 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 29 Mar 2021 08:40:39 +0200
After some tests relating to standard_conforming_strings behavior, the
value was not reset to the default value. Therefore, the rest of the
tests in that file ran with the nondefault setting, which affected the
results of some tests. For clarity, reset the value and run the rest
of the tests with the default setting again.
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
PageAddItemExtended(): Add LP_UNUSED assertion.
commit : 30aaab26e52144097a1a5bbb0bb66ea1ebc0cb81
author : Peter Geoghegan <[email protected]>
date : Sun, 28 Mar 2021 20:10:02 -0700
committer: Peter Geoghegan <[email protected]>
date : Sun, 28 Mar 2021 20:10:02 -0700
Assert that LP_UNUSED items have no storage. If it's worth having
defensive code in non-assert builds then it's worth having an assertion
as well.
M src/backend/storage/page/bufpage.c
Cache if PathTarget and RestrictInfos contain volatile functions
commit : f58b230ed0dba2a3d396794a2ec84541e321d92d
author : David Rowley <[email protected]>
date : Mon, 29 Mar 2021 14:47:05 +1300
committer: David Rowley <[email protected]>
date : Mon, 29 Mar 2021 14:47:05 +1300
Here we aim to reduce duplicate work done by contain_volatile_functions()
by caching whether PathTargets and RestrictInfos contain any volatile
functions the first time contain_volatile_functions() is called for them.
Any future calls for these nodes just use the cached value rather than
going to the trouble of recursively checking the sub-node all over again.
Thanks to Tom Lane for the idea.
Any locations in the code which make changes to a PathTarget or
RestrictInfo which could change the outcome of the volatility check must
change the cached value back to VOLATILITY_UNKNOWN again.
contain_volatile_functions() is the only code in charge of setting the
cache value to either VOLATILITY_VOLATILE or VOLATILITY_NOVOLATILE.
Some existing code does benefit from this additional caching, however,
this change is mainly aimed at an upcoming patch that must check for
volatility during the join search. Repeated volatility checks in that
case can become very expensive when the join search contains more than a
few relations.
Author: David Rowley
Discussion: https://postgr.es/m/[email protected]
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/util/clauses.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/optimizer/util/tlist.c
M src/include/nodes/pathnodes.h
doc: Define TLS as an acronym
commit : b64654d6c450eb9fb04c6e3456915790510af482
author : Stephen Frost <[email protected]>
date : Sun, 28 Mar 2021 11:27:59 -0400
committer: Stephen Frost <[email protected]>
date : Sun, 28 Mar 2021 11:27:59 -0400
Commit c6763156589 added an acronym reference for "TLS" but the definition
was never added.
Author: Daniel Gustafsson
Reviewed-by: Michael Paquier
Backpatch-through: 9.6
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/acronyms.sgml
Stabilize stats_ext test with other collations
commit : 2a058e938c73bfb85bbc9fa93dea74788043ca6c
author : Tomas Vondra <[email protected]>
date : Sat, 27 Mar 2021 18:26:52 +0100
committer: Tomas Vondra <[email protected]>
date : Sat, 27 Mar 2021 18:26:52 +0100
The tests used string concatenation to test statistics on expressions,
but that made the tests locale-dependent, e.g. because the ordering of
'11' and '1X' depends on the collation. This affected both the estimated
and actual row couts, breaking some of the tests.
Fixed by replacing the string concatenation with upper() function call,
so that the text values contain only digits.
Discussion: https://postgr.es/m/b650920b-2767-fbc3-c87a-cb8b5d693cbf%40enterprisedb.com
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Improve consistency of SQL code capitalization
commit : 8df2f371141ea267627364cd00e1791054d82d7e
author : Peter Eisentraut <[email protected]>
date : Sat, 27 Mar 2021 10:17:12 +0100
committer: Peter Eisentraut <[email protected]>
date : Sat, 27 Mar 2021 10:17:12 +0100
M src/backend/catalog/system_views.sql
M src/bin/initdb/initdb.c
Extended statistics on expressions
commit : a4d75c86bf15220df22de0a92c819ecef9db3849
author : Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 23:22:01 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 23:22:01 +0100
Allow defining extended statistics on expressions, not just just on
simple column references. With this commit, expressions are supported
by all existing extended statistics kinds, improving the same types of
estimates. A simple example may look like this:
CREATE TABLE t (a int);
CREATE STATISTICS s ON mod(a,10), mod(a,20) FROM t;
ANALYZE t;
The collected statistics are useful e.g. to estimate queries with those
expressions in WHERE or GROUP BY clauses:
SELECT * FROM t WHERE mod(a,10) = 0 AND mod(a,20) = 0;
SELECT 1 FROM t GROUP BY mod(a,10), mod(a,20);
This introduces new internal statistics kind 'e' (expressions) which is
built automatically when the statistics object definition includes any
expressions. This represents single-expression statistics, as if there
was an expression index (but without the index maintenance overhead).
The statistics is stored in pg_statistics_ext_data as an array of
composite types, which is possible thanks to 79f6a942bd.
CREATE STATISTICS allows building statistics on a single expression, in
which case in which case it's not possible to specify statistics kinds.
A new system view pg_stats_ext_exprs can be used to display expression
statistics, similarly to pg_stats and pg_stats_ext views.
ALTER TABLE ... ALTER COLUMN ... TYPE now treats indexes the same way it
treats indexes, i.e. it drops and recreates the statistics. This means
all statistics are reset, and we no longer try to preserve at least the
functional dependencies. This should not be a major issue in practice,
as the functional dependencies actually rely on per-column statistics,
which were always reset anyway.
Author: Tomas Vondra
Reviewed-by: Justin Pryzby, Dean Rasheed, Zhihong Yu
Discussion: https://postgr.es/m/ad7891d2-e90c-b446-9fe2-7419143847d7%40enterprisedb.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/create_statistics.sgml
M src/backend/catalog/Makefile
M src/backend/catalog/system_views.sql
M src/backend/commands/statscmds.c
M src/backend/commands/tablecmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/util/plancat.c
M src/backend/parser/gram.y
M src/backend/parser/parse_agg.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_func.c
M src/backend/parser/parse_utilcmd.c
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/statistics/mcv.c
M src/backend/statistics/mvdistinct.c
M src/backend/tcop/utility.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/adt/selfuncs.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/commands/defrem.h
M src/include/nodes/nodes.h
M src/include/nodes/parsenodes.h
M src/include/nodes/pathnodes.h
M src/include/parser/parse_node.h
M src/include/parser/parse_utilcmd.h
M src/include/statistics/extended_stats_internal.h
M src/include/statistics/statistics.h
M src/include/utils/ruleutils.h
M src/test/regress/expected/create_table_like.out
M src/test/regress/expected/oidjoins.out
M src/test/regress/expected/rules.out
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/create_table_like.sql
M src/test/regress/sql/stats_ext.sql
Reduce duration of stats_ext regression tests
commit : 98376c18f12e562421b5c77e619248e8b7aae3c6
author : Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 23:00:41 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 23:00:41 +0100
The regression tests of extended statistics were taking a fair amount of
time, due to using fairly large data sets with a couple thousand rows.
So far this was fine, but with tests for statistics on expressions the
duration would get a bit excessive. So reduce the size of some of the
tests that will be used to test expressions, to keep the duration under
control. Done in a separate commit before adding the statistics on
expressions, to make it clear which estimates are expected to change.
Author: Tomas Vondra
Discussion: https://postgr.es/m/ad7891d2-e90c-b446-9fe2-7419143847d7%40enterprisedb.com
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Fix ndistinct estimates with system attributes
commit : 33e52ad9a32929a6d14dfd98a8440d57028f2e3e
author : Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 22:34:53 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 22:34:53 +0100
When estimating the number of groups using extended statistics, the code
was discarding information about system attributes. This led to strange
situation that
SELECT 1 FROM t GROUP BY ctid;
could have produced higher estimate (equal to pg_class.reltuples) than
SELECT 1 FROM t GROUP BY a, b, ctid;
with extended statistics on (a,b). Fixed by retaining information about
the system attribute.
Backpatch all the way to 10, where extended statistics were introduced.
Author: Tomas Vondra
Backpatch-through: 10
M src/backend/utils/adt/selfuncs.c
M src/test/regress/expected/stats_ext.out
Add "pg_database_owner" default role.
commit : a14a0118a1fecf4066e53af52ed0f188607d0c4b
author : Noah Misch <[email protected]>
date : Fri, 26 Mar 2021 10:42:17 -0700
committer: Noah Misch <[email protected]>
date : Fri, 26 Mar 2021 10:42:17 -0700
Membership consists, implicitly, of the current database owner. Expect
use in template databases. Once pg_database_owner has rights within a
template, each owner of a database instantiated from that template will
exercise those rights.
Reviewed by John Naylor.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/user-manag.sgml
M src/backend/catalog/information_schema.sql
M src/backend/commands/user.c
M src/backend/utils/adt/acl.c
M src/backend/utils/cache/catcache.c
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_authid.dat
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Merge similar algorithms into roles_is_member_of().
commit : f687bf61ed4dc75ec074c387f848147da2097e13
author : Noah Misch <[email protected]>
date : Fri, 26 Mar 2021 10:42:16 -0700
committer: Noah Misch <[email protected]>
date : Fri, 26 Mar 2021 10:42:16 -0700
The next commit would have complicated two or three algorithms, so take
this opportunity to consolidate. No functional changes.
Reviewed by John Naylor.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/acl.c
Fix alignment in BRIN minmax-multi deserialization
commit : 73b96bad4af8fd113a36e4633dd3312001c115dc
author : Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 16:33:19 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 16:33:19 +0100
The deserialization failed to ensure correct alignment, as it assumed it
can simply point into the serialized value. The serialization however
ignores alignment and copies just the significant bytes in order to make
the result as small as possible. This caused failures on systems that
are sensitive to mialigned addresses, like sparc, or with address
sanitizer enabled.
Fixed by copying the serialized data to ensure proper alignment. While
at it, fix an issue with serialization on big endian machines, using the
same store_att_byval/fetch_att trick as extended statistics.
Discussion: https://postgr.es/0c8c3304-d3dd-5e29-d5ac-b50589a23c8c%40enterprisedb.com
M src/backend/access/brin/brin_minmax_multi.c
BRIN minmax-multi indexes
commit : ab596105b55f1d7fbd5a66b66f65227d210b047d
author : Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 13:54:29 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 13:54:29 +0100
Adds BRIN opclasses similar to the existing minmax, except that instead
of summarizing the page range into a single [min,max] range, the summary
consists of multiple ranges and/or points, allowing gaps. This allows
more efficient handling of data with poor correlation to physical
location within the table and/or outlier values, for which the regular
minmax opclassed tend to work poorly.
It's possible to specify the number of values kept for each page range,
either as a single point or an interval boundary.
CREATE TABLE t (a int);
CREATE INDEX ON t
USING brin (a int4_minmax_multi_ops(values_per_range=16));
When building the summary, the values are combined into intervals with
the goal to minimize the "covering" (sum of interval lengths), using a
support procedure computing distance between two values.
Bump catversion, due to various catalog changes.
Author: Tomas Vondra <[email protected]>
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Alexander Korotkov <[email protected]>
Reviewed-by: Sokolov Yura <[email protected]>
Reviewed-by: John Naylor <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/5d78b774-7e9c-c94e-12cf-fef51cc89b1a%402ndquadrant.com
M doc/src/sgml/brin.sgml
M src/backend/access/brin/Makefile
A src/backend/access/brin/brin_minmax_multi.c
M src/backend/access/brin/brin_tuple.c
M src/include/access/brin_tuple.h
M src/include/access/transam.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_type.dat
A src/test/regress/expected/brin_multi.out
M src/test/regress/expected/psql.out
M src/test/regress/expected/type_sanity.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/brin_multi.sql
BRIN bloom indexes
commit : 77b88cd1bb9041a735f24072150cacfa06c699a3
author : Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 13:35:29 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 13:35:29 +0100
Adds a BRIN opclass using a Bloom filter to summarize the range. Indexes
using the new opclasses allow only equality queries (similar to hash
indexes), but that works fine for data like UUID, MAC addresses etc. for
which range queries are not very common. This also means the indexes
work for data that is not well correlated to physical location within
the table, or perhaps even entirely random (which is a common issue with
existing BRIN minmax opclasses).
It's possible to specify opclass parameters with the usual Bloom filter
parameters, i.e. the desired false-positive rate and the expected number
of distinct values per page range.
CREATE TABLE t (a int);
CREATE INDEX ON t
USING brin (a int4_bloom_ops(false_positive_rate = 0.05,
n_distinct_per_range = 100));
The opclasses do not operate on the indexed values directly, but compute
a 32-bit hash first, and the Bloom filter is built on the hash value.
Collisions should not be a huge issue though, as the number of distinct
values in a page ranges is usually fairly small.
Bump catversion, due to various catalog changes.
Author: Tomas Vondra <[email protected]>
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Alexander Korotkov <[email protected]>
Reviewed-by: Sokolov Yura <[email protected]>
Reviewed-by: Nico Williams <[email protected]>
Reviewed-by: John Naylor <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/5d78b774-7e9c-c94e-12cf-fef51cc89b1a%402ndquadrant.com
M doc/src/sgml/brin.sgml
M src/backend/access/brin/Makefile
A src/backend/access/brin/brin_bloom.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_type.dat
A src/test/regress/expected/brin_bloom.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/expected/psql.out
M src/test/regress/expected/type_sanity.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/brin_bloom.sql
Support the old signature of BRIN consistent function
commit : a681e3c107aa97eb554f118935c4d2278892c3dd
author : Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 13:17:56 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 13:17:56 +0100
Commit a1c649d889 changed the signature of the BRIN consistent function
by adding a new required parameter. Treating the parameter as optional,
which would make the change backwards incompatibile, was rejected with
the justification that there are few out-of-core extensions, so it's not
worth adding making the code more complex, and it's better to deal with
that in the extension.
But after further thought, that would be rather problematic, because
pg_upgrade simply dumps catalog contents and the same version of an
extension needs to work on both PostgreSQL versions. Supporting both
variants of the consistent function (with 3 or 4 arguments) makes that
possible.
The signature is not the only thing that changed, as commit 72ccf55cb9
moved handling of IS [NOT] NULL keys from the support procedures. But
this change is backward compatible - handling the keys in exension is
unnecessary, but harmless. The consistent function will do a bit of
unnecessary work, but it should be very cheap.
This also undoes most of the changes to the existing opclasses (minmax
and inclusion), making them use the old signature again. This should
make backpatching simpler.
Catversion bump, because of changes in pg_amproc.
Author: Tomas Vondra <[email protected]>
Author: Nikita Glukhov <[email protected]>
Reviewed-by: Mark Dilger <[email protected]>
Reviewed-by: Alexander Korotkov <[email protected]>
Reviewed-by: Masahiko Sawada <[email protected]>
Reviewed-by: John Naylor <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/brin.sgml
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_inclusion.c
M src/backend/access/brin/brin_minmax.c
M src/backend/access/brin/brin_validate.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Remove unnecessary pg_amproc BRIN minmax entries
commit : a68dfa27d42fb7b7611fd1206d2356fc124ed390
author : Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 13:04:13 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 26 Mar 2021 13:04:13 +0100
The BRIN minmax opclasses included amproc entries with mismatching left
and right types, but those happen to be unnecessary. The opclasses only
need cross-type operators, not cross-type support procedures. Discovered
when trying to define equivalent BRIN operator families in an extension.
Catversion bump, because of pg_amproc changes.
Author: Tomas Vondra
Reviewed-by: Alvaro Herrera
Discussion: https://postgr.es/m/78c357ab-3395-8433-e7b3-b2cfcc9fdc23%40enterprisedb.com
M src/include/catalog/catversion.h
M src/include/catalog/pg_amproc.dat
Fix interaction of TOAST compression with expression indexes.
commit : 5db1fd7823a1a12e2bdad98abc8e102fd71ffbda
author : Robert Haas <[email protected]>
date : Thu, 25 Mar 2021 19:55:32 -0400
committer: Robert Haas <[email protected]>
date : Thu, 25 Mar 2021 19:55:32 -0400
Before, trying to compress a value for insertion into an expression
index would crash.
Dilip Kumar, with some editing by me. Report by Jaime Casanova.
Discussion: http://postgr.es/m/CAJKUy5gcs0zGOp6JXU2mMVdthYhuQpFk=S3V8DOKT=LZC1L36Q@mail.gmail.com
M src/backend/access/brin/brin_tuple.c
M src/backend/access/common/indextuple.c
M src/backend/catalog/index.c
M src/test/regress/expected/compression.out
M src/test/regress/expected/compression_1.out
M src/test/regress/sql/compression.sql
ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY
commit : 71f4c8c6f74ba021e55d35b1128d22fb8c6e1629
author : Alvaro Herrera <[email protected]>
date : Thu, 25 Mar 2021 18:00:28 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 25 Mar 2021 18:00:28 -0300
Allow a partition be detached from its partitioned table without
blocking concurrent queries, by running in two transactions and only
requiring ShareUpdateExclusive in the partitioned table.
Because it runs in two transactions, it cannot be used in a transaction
block. This is the main reason to use dedicated syntax: so that users
can choose to use the original mode if they need it. But also, it
doesn't work when a default partition exists (because an exclusive lock
would still need to be obtained on it, in order to change its partition
constraint.)
In case the second transaction is cancelled or a crash occurs, there's
ALTER TABLE .. DETACH PARTITION .. FINALIZE, which executes the final
steps.
The main trick to make this work is the addition of column
pg_inherits.inhdetachpending, initially false; can only be set true in
the first part of this command. Once that is committed, concurrent
transactions that use a PartitionDirectory will include or ignore
partitions so marked: in optimizer they are ignored if the row is marked
committed for the snapshot; in executor they are always included. As a
result, and because of the way PartitionDirectory caches partition
descriptors, queries that were planned before the detach will see the
rows in the detached partition and queries that are planned after the
detach, won't.
A CHECK constraint is created that duplicates the partition constraint.
This is probably not strictly necessary, and some users will prefer to
remove it afterwards, but if the partition is re-attached to a
partitioned table, the constraint needn't be rechecked.
Author: Álvaro Herrera <[email protected]>
Reviewed-by: Amit Langote <[email protected]>
Reviewed-by: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/alter_table.sgml
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/partition.c
M src/backend/catalog/pg_inherits.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/executor/execPartition.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/optimizer/util/plancat.c
M src/backend/parser/gram.y
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partdesc.c
M src/backend/tcop/utility.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/cache/partcache.c
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/partition.h
M src/include/catalog/pg_inherits.h
M src/include/nodes/parsenodes.h
M src/include/parser/kwlist.h
M src/include/partitioning/partdesc.h
M src/include/utils/snapmgr.h
A src/test/isolation/expected/detach-partition-concurrently-1.out
A src/test/isolation/expected/detach-partition-concurrently-2.out
A src/test/isolation/expected/detach-partition-concurrently-3.out
A src/test/isolation/expected/detach-partition-concurrently-4.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/detach-partition-concurrently-1.spec
A src/test/isolation/specs/detach-partition-concurrently-2.spec
A src/test/isolation/specs/detach-partition-concurrently-3.spec
A src/test/isolation/specs/detach-partition-concurrently-4.spec
M src/test/modules/delay_execution/Makefile
A src/test/modules/delay_execution/expected/partition-removal-1.out
A src/test/modules/delay_execution/specs/partition-removal-1.spec
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
Document lock obtained during partition detach
commit : 650d623530c884c087c565f1d3b8cd76f8fe2b95
author : Alvaro Herrera <[email protected]>
date : Thu, 25 Mar 2021 16:30:22 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 25 Mar 2021 16:30:22 -0300
On partition detach, we acquire a SHARE lock on all tables that
reference the partitioned table that we're detaching a partition from,
but failed to document this fact. My oversight in commit f56f8f8da6af.
Repair. Backpatch to 12.
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/alter_table.sgml
Add comments for AlteredTableInfo->rel
commit : cc121d5596964f8aac93607e6f14607184558b16
author : Alvaro Herrera <[email protected]>
date : Thu, 25 Mar 2021 16:07:15 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 25 Mar 2021 16:07:15 -0300
The prior commit which introduced it was pretty squalid in terms of
code documentation, so add some comments.
M src/backend/commands/tablecmds.c
Let ALTER TABLE Phase 2 routines manage the relation pointer
commit : cd03c6e94b09ff402cbc3ce8da5587f09f0b5e58
author : Alvaro Herrera <[email protected]>
date : Thu, 25 Mar 2021 15:56:11 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 25 Mar 2021 15:56:11 -0300
Struct AlteredRelationInfo gains a new Relation member, to be used only
by Phase 2 (ATRewriteCatalogs); this allows ATExecCmd() subroutines open
and close the relation internally.
A future commit will use this facility to implement an ALTER TABLE
subcommand that closes and reopens the relation across transaction
boundaries.
(It is possible to keep the relation open past phase 2 to be used by
phase 3 instead of having to reopen it that point, but there are some
minor complications with that; it's not clear that there is much to be
won from doing that, though.)
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
Rework HeapTupleHeader macros to reuse itemptr.h
commit : 4669cacbd4b4b1baa1b7f2ea53d461433a1b6276
author : Alvaro Herrera <[email protected]>
date : Mon, 22 Feb 2021 17:21:22 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 22 Feb 2021 17:21:22 -0300
The original definitions pointlessly disregarded existing ItemPointer
macros that do the same thing.
Reported-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/include/access/htup_details.h
Remove StoreSingleInheritance reimplementation
commit : a24ae3d7b9efb3b113c0d53030aa99de0d41b40a
author : Alvaro Herrera <[email protected]>
date : Thu, 25 Mar 2021 10:47:38 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 25 Mar 2021 10:47:38 -0300
I introduced this duplicate code in commit 8b08f7d4820f for no good
reason. Remove it, and backpatch to 11 where it was introduced.
Author: Álvaro Herrera <[email protected]>
M src/backend/commands/indexcmds.c
Trim some extra whitespace in parser file
commit : f2c7ce64ae9ba292c1846ae864cef1b8b37af1f3
author : Peter Eisentraut <[email protected]>
date : Thu, 25 Mar 2021 10:17:52 +0100
committer: Peter Eisentraut <[email protected]>
date : Thu, 25 Mar 2021 10:17:52 +0100
M src/backend/parser/gram.y
Rename a parse node to be more general
commit : 91d1f2d302108f49006eedb8053522236dde77cc
author : Peter Eisentraut <[email protected]>
date : Thu, 25 Mar 2021 10:06:32 +0100
committer: Peter Eisentraut <[email protected]>
date : Thu, 25 Mar 2021 10:06:32 +0100
A WHERE clause will be used for row filtering in logical replication.
We already have a similar node: 'WHERE (condition here)'. Let's
rename the node to a generic name and use it for row filtering too.
Author: Euler Taveira <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CAHE3wggb715X+mK_DitLXF25B=jE6xyNCH4YOwM860JR7HarGQ@mail.gmail.com
M src/backend/parser/gram.y
Sanitize the term "combo CID" in code comments
commit : a1999a01bb56c5f5451116abe61b892b2eec5e49
author : Michael Paquier <[email protected]>
date : Thu, 25 Mar 2021 16:08:03 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 25 Mar 2021 16:08:03 +0900
Combo CIDs were referred in the code comments using different terms
across various places of the code, so unify a bit the term used with
what is currently in use in some of the READMEs.
Author: "Hou, Zhijie"
Discussion: https://postgr.es/m/1d42865c91404f46af4562532fdbea31@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_visibility.c
M src/backend/executor/execMain.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/backend/utils/time/combocid.c
M src/include/access/htup_details.h
M src/test/regress/expected/combocid.out
M src/test/regress/sql/combocid.sql
Fix bug in WAL replay of COMMIT_TS_SETTS record.
commit : 438fc4a39c3905b7af88bb848bc5aeb1308a017d
author : Fujii Masao <[email protected]>
date : Thu, 25 Mar 2021 11:23:30 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 25 Mar 2021 11:23:30 +0900
Previously the WAL replay of COMMIT_TS_SETTS record called
TransactionTreeSetCommitTsData() with the argument write_xlog=true,
which generated and wrote new COMMIT_TS_SETTS record.
This should not be acceptable because it's during recovery.
This commit fixes the WAL replay of COMMIT_TS_SETTS record
so that it calls TransactionTreeSetCommitTsData() with write_xlog=false
and doesn't generate new WAL during recovery.
Back-patch to all supported branches.
Reported-by: lx zou <[email protected]>
Author: Fujii Masao
Reviewed-by: Alvaro Herrera
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/commit_ts.c
Improve connection denied error message during recovery.
commit : df9384492b89aac370ab9d12eb89375aeb38a1d4
author : Fujii Masao <[email protected]>
date : Thu, 25 Mar 2021 10:41:28 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 25 Mar 2021 10:41:28 +0900
Previously when an archive recovery or a standby was starting and
reached the consistent recovery state but hot_standby was configured
to off, the error message when a client connectted was "the database
system is starting up", which was needless confusing and not really
all that accurate either.
This commit improves the connection denied error message during
recovery, as follows, so that the users immediately know that their
servers are configured to deny those connections.
- If hot_standby is disabled, the error message "the database system
is not accepting connections" and the detail message "Hot standby
mode is disabled." are output when clients connect while an archive
recovery or a standby is running.
- If hot_standby is enabled, the error message "the database system
is not yet accepting connections" and the detail message
"Consistent recovery state has not been yet reached." are output
when clients connect until the consistent recovery state is reached
and postmaster starts accepting read only connections.
This commit doesn't change the connection denied error message of
"the database system is starting up" during normal server startup and
crash recovery. Because it's still suitable for those situations.
Author: James Coleman
Reviewed-by: Alvaro Herrera, Andres Freund, David Zhang, Tom Lane, Fujii Masao
Discussion: https://postgr.es/m/CAAaqYe8h5ES_B=F_zDT+Nj9XU7YEwNhKhHA2RE4CFhAQ93hfig@mail.gmail.com
M src/backend/postmaster/postmaster.c
M src/include/libpq/libpq-be.h
Allow for installation-aware instances of PostgresNode
commit : b34ca595abd697e716ce369ec1b58624bdd1c431
author : Andrew Dunstan <[email protected]>
date : Wed, 24 Mar 2021 18:52:25 -0400
committer: Andrew Dunstan <[email protected]>
date : Wed, 24 Mar 2021 18:52:25 -0400
Currently instances of PostgresNode find their Postgres executables in
the PATH of the caller. This modification allows for instances that know
the installation path they are supposed to use, and the module adjusts
the environment of methods that call Postgres executables appropriately.
This facility is activated by passing the installation path to the
constructor:
my $node = PostgresNode->get_new_node('mynode',
installation_path => '/path/to/installation');
This makes a number of things substantially easier, including
. testing third party modules
. testing different versions of postgres together
. testing different builds of postgres together
Discussion: https://postgr.es/m/[email protected]
Reviewed-By: Alvaro Herrera, Michael Paquier, Dagfinn Ilmari Mannsåker
M src/test/perl/PostgresNode.pm
Need to step forward in the loop to get to an end.
commit : 65c2ec6f30d9c0878a9ef83e0ec9a53e6b82d9d8
author : Michael Meskes <[email protected]>
date : Wed, 24 Mar 2021 22:06:31 +0100
committer: Michael Meskes <[email protected]>
date : Wed, 24 Mar 2021 22:06:31 +0100
M src/interfaces/ecpg/preproc/ecpg.c
Add DECLARE STATEMENT command to ECPG
commit : ad8305a43d1890768a613d3fb586b44f17360f29
author : Michael Meskes <[email protected]>
date : Wed, 24 Mar 2021 20:48:20 +0100
committer: Michael Meskes <[email protected]>
date : Wed, 24 Mar 2021 20:48:20 +0100
This command declares a SQL identifier for a SQL statement to be used in other
embedded SQL statements. The identifier is linked to a connection.
Author: Hayato Kuroda <[email protected]>
Reviewed-by: Shawn Wang <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/TY2PR01MB24438A52DB04E71D0E501452F5630@TY2PR01MB2443.jpnprd01.prod.outlook.com
M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/preproc/ecpg.addons
M src/interfaces/ecpg/preproc/ecpg.c
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/ecpg.type
M src/interfaces/ecpg/preproc/preproc_extern.h
M src/interfaces/ecpg/preproc/type.h
M src/interfaces/ecpg/test/ecpg_schedule
A src/interfaces/ecpg/test/expected/sql-declare.c
A src/interfaces/ecpg/test/expected/sql-declare.stderr
A src/interfaces/ecpg/test/expected/sql-declare.stdout
M src/interfaces/ecpg/test/sql/.gitignore
M src/interfaces/ecpg/test/sql/Makefile
A src/interfaces/ecpg/test/sql/declare.pgc
Fix stray double semicolons
commit : 37c99d304dcbf12ab581ff031f394af93b750895
author : Peter Eisentraut <[email protected]>
date : Wed, 24 Mar 2021 20:42:05 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 24 Mar 2021 20:42:05 +0100
Reported-by: John Naylor <[email protected]>
M src/backend/access/brin/brin_minmax.c
M src/backend/utils/adt/timestamp.c
doc: Fix typo
commit : 5173e428928ce890be3e3d809b2d23d5f3c7da2f
author : Peter Eisentraut <[email protected]>
date : Wed, 24 Mar 2021 20:41:18 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 24 Mar 2021 20:41:18 +0100
Reported-by: Erik Rijkers <[email protected]>
M doc/src/sgml/func.sgml
Change checkpoint_completion_target default to 0.9
commit : bbcc4eb2e08fb6e4535c7f84b2c00f3ad508bb9b
author : Stephen Frost <[email protected]>
date : Wed, 24 Mar 2021 13:07:51 -0400
committer: Stephen Frost <[email protected]>
date : Wed, 24 Mar 2021 13:07:51 -0400
Common recommendations are that the checkpoint should be spread out as
much as possible, provided we avoid having it take too long. This
change updates the default to 0.9 (from 0.5) to match that
recommendation.
There was some debate about possibly removing the option entirely but it
seems there may be some corner-cases where having it set much lower to
try to force the checkpoint to be as fast as possible could result in
fewer periods of time of reduced performance due to kernel flushing.
General agreement is that the "spread more" is the preferred approach
though and those who need to tune away from that value are much less
common.
Reviewed-By: Michael Paquier, Peter Eisentraut, Tom Lane, David Steele,
Nathan Bossart
Discussion: https://postgr.es/m/20201207175329.GM16415%40tamriel.snowman.net
M doc/src/sgml/config.sgml
M doc/src/sgml/wal.sgml
M src/backend/postmaster/checkpointer.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/test/recovery/t/015_promotion_pages.pl
Tidy up more loose ends related to configurable TOAST compression.
commit : e5595de03ec6ce60afde980ae05e9353a1501fdf
author : Robert Haas <[email protected]>
date : Wed, 24 Mar 2021 12:36:08 -0400
committer: Robert Haas <[email protected]>
date : Wed, 24 Mar 2021 12:36:08 -0400
Change the default_toast_compression GUC to be an enum rather than
a string. Earlier, uncommitted versions of the patch supported using
CREATE ACCESS METHOD to add new compression methods to a running
system, but that idea was dropped before commit. So, we can simplify
the GUC handling as well, which has the nice side effect of improving
the error messages.
While updating the documentation to reflect the new GUC type, also
move it back to the right place in the list. I moved this while
revising what became commit 24f0e395ac5892cd12e8914646fe921fac5ba23d,
but apparently the intended ordering is "alphabetical" rather than
"whatever Robert thinks looks nice."
Rejigger things to avoid having access/toast_compression.h depend on
utils/guc.h, so that we don't end up with every file that includes
it also depending on something largely unrelated. Move a few
inline functions back into the C source file partly to help reduce
dependencies and partly just to avoid clutter. A few very minor
cosmetic fixes.
Original patch by Justin Pryzby, but very heavily edited by me,
and reverse reviewed by him and also reviewed by by Tom Lane.
Discussion: http://postgr.es/m/CA+TgmoYp=GT_ztUCeZg2i4hkHAQv8o=-nVJ1-TKWTG1zQOmOpg@mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/access/common/toast_compression.c
M src/backend/utils/misc/guc.c
M src/include/access/toast_compression.h
M src/test/regress/expected/compression.out
M src/test/regress/expected/compression_1.out
Add date_bin function
commit : 49ab61f0bdc93984a8d36b602f6f2a15f09ebcc7
author : Peter Eisentraut <[email protected]>
date : Wed, 24 Mar 2021 16:16:14 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 24 Mar 2021 16:16:14 +0100
Similar to date_trunc, but allows binning by an arbitrary interval
rather than just full units.
Author: John Naylor <[email protected]>
Reviewed-by: David Fetter <[email protected]>
Reviewed-by: Isaac Morland <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Reviewed-by: Artur Zakirov <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CACPNZCt4buQFRgy6DyjuZS-2aPDpccRkrJBmgUfwYc1KiaXYxg@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/timestamp.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Improve an error message
commit : 1509c6fc29c07d13c9a590fbd6f37c7576f58ba6
author : Peter Eisentraut <[email protected]>
date : Wed, 24 Mar 2021 08:02:06 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 24 Mar 2021 08:02:06 +0100
Make it the same as another nearby message.
M src/backend/replication/logical/tablesync.c
Revert "Enable parallel SELECT for "INSERT INTO ... SELECT ..."."
commit : 26acb54a1368bf3706294400abca85b15c9233a6
author : Amit Kapila <[email protected]>
date : Wed, 24 Mar 2021 11:10:12 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 24 Mar 2021 11:10:12 +0530
To allow inserts in parallel-mode this feature has to ensure that all the
constraints, triggers, etc. are parallel-safe for the partition hierarchy
which is costly and we need to find a better way to do that. Additionally,
we could have used existing cached information in some cases like indexes,
domains, etc. to determine the parallel-safety.
List of commits reverted, in reverse chronological order:
ed62d3737c Doc: Update description for parallel insert reloption.
c8f78b6161 Add a new GUC and a reloption to enable inserts in parallel-mode.
c5be48f092 Improve FK trigger parallel-safety check added by 05c8482f7f.
e2cda3c20a Fix use of relcache TriggerDesc field introduced by commit 05c8482f7f.
e4e87a32cc Fix valgrind issue in commit 05c8482f7f.
05c8482f7f Enable parallel SELECT for "INSERT INTO ... SELECT ...".
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M doc/src/sgml/parallel.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_table.sgml
M src/backend/access/common/reloptions.c
M src/backend/access/transam/xact.c
M src/backend/executor/execMain.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/util/clauses.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/psql/tab-complete.c
M src/include/access/xact.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/clauses.h
M src/include/optimizer/cost.h
M src/include/utils/rel.h
D src/test/regress/expected/insert_parallel.out
M src/test/regress/expected/sysviews.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
D src/test/regress/sql/insert_parallel.sql
M src/tools/pgindent/typedefs.list
Rename wait event WalrcvExit to WalReceiverExit.
commit : 84007043fc1b1be68dad5d0a78269347c12094b6
author : Fujii Masao <[email protected]>
date : Wed, 24 Mar 2021 10:37:54 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 24 Mar 2021 10:37:54 +0900
Commit de829ddf23 added wait event WalrcvExit. But its name is not
consistent with other wait events like WalReceiverMain or
WalReceiverWaitStart, etc. So this commit renames WalrcvExit to
WalReceiverExit.
Author: Fujii Masao
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
M src/backend/postmaster/pgstat.c
M src/backend/replication/walreceiverfuncs.c
M src/include/pgstat.h
Log when GetNewOidWithIndex() fails to find unused OID many times.
commit : 7fbcee1b2d5f1012c67942126881bd492e95077e
author : Fujii Masao <[email protected]>
date : Wed, 24 Mar 2021 10:36:56 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 24 Mar 2021 10:36:56 +0900
GetNewOidWithIndex() generates a new OID one by one until it finds
one not in the relation. If there are very long runs of consecutive
existing OIDs, GetNewOidWithIndex() needs to iterate many times
in the loop to find unused OID. Since TOAST table can have a large
number of entries and there can be such long runs of OIDs, there is
the case where it takes so many iterations to find new OID not in
TOAST table. Furthermore if all (i.e., 2^32) OIDs are already used,
GetNewOidWithIndex() enters something like busy loop and repeats
the iterations until at least one OID is marked as unused.
There are some reported troubles caused by a large number of
iterations in GetNewOidWithIndex(). For example, when inserting
a billion of records into the table, all the backends doing that
insertion operation got hang with 100% CPU usage at some point.
Previously there was no easy way to detect that GetNewOidWithIndex()
failed to find unused OID many times. So, for example, gdb full
backtrace of hanged backends needed to be taken, in order to
investigate that trouble. This is inconvenient and may not be
available in some production environments.
To provide easy way for that, this commit makes GetNewOidWithIndex()
log that it iterates more than GETNEWOID_LOG_THRESHOLD but have
not yet found OID unused in the relation. Also this commit makes
it repeat logging with exponentially increasing intervals until
it iterates more than GETNEWOID_LOG_MAX_INTERVAL, and makes it
finally repeat logging every GETNEWOID_LOG_MAX_INTERVAL unless
an unused OID is found. Those macro variables are used not to
fill up the server log with the similar messages.
In the discusion at pgsql-hackers, there was another idea to report
the lots of iterations in GetNewOidWithIndex() via wait event.
But since GetNewOidWithIndex() traverses indexes to find unused
OID and which will do I/O, acquire locks, etc, which will overwrite
the wait event and reset it to nothing once done. So that idea
doesn't work well, and we didn't adopt it.
Author: Tomohiro Hiramitsu
Reviewed-by: Tatsuhito Kasahara, Kyotaro Horiguchi, Tom Lane, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/catalog.c
Reword slightly logs generated for index stats in autovacuum
commit : 99dd75fb99baa9188971cf47779ed8d7a5e6eb29
author : Michael Paquier <[email protected]>
date : Wed, 24 Mar 2021 09:36:03 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 24 Mar 2021 09:36:03 +0900
Using "remain" is confusing, as it implies that the index file can
shrink. Instead, use "in total".
Per discussion with Peter Geoghegan.
Discussion: https://postgr.es/m/CAH2-WzkYgHZzpGOwR14CScJsjaQpvJrEkEfkh_=wGhzLb=yVdQ@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Allow composite types in catalog bootstrap
commit : 79f6a942bdb958fbd7ef6870d5bf2e3cefb65da5
author : Tomas Vondra <[email protected]>
date : Wed, 24 Mar 2021 00:47:50 +0100
committer: Tomas Vondra <[email protected]>
date : Wed, 24 Mar 2021 00:47:50 +0100
When resolving types during catalog bootstrap, try to reload the pg_type
contents if a type is not found. That allows catalogs to contain
composite types, e.g. row types for other catalogs.
Author: Justin Pryzby
Reviewed-by: Dean Rasheed, Tomas Vondra
Discussion: https://postgr.es/m/ad7891d2-e90c-b446-9fe2-7419143847d7%40enterprisedb.com
M src/backend/bootstrap/bootstrap.c
Convert Typ from array to list in bootstrap
commit : e1a5e65703ce884529340819f6268d24f43ef8f7
author : Tomas Vondra <[email protected]>
date : Wed, 24 Mar 2021 00:47:38 +0100
committer: Tomas Vondra <[email protected]>
date : Wed, 24 Mar 2021 00:47:38 +0100
It's a bit easier and more convenient to free and reload a List,
compared to a plain array. This will be helpful when allowing catalogs
to contain composite types.
Author: Justin Pryzby
Reviewed-by: Dean Rasheed, Tomas Vondra
Discussion: https://postgr.es/m/ad7891d2-e90c-b446-9fe2-7419143847d7%40enterprisedb.com
M src/backend/bootstrap/bootstrap.c
nbtree VACUUM: Cope with buggy opclasses.
commit : 5b861baa550a369e04bf67fbe83f3a5a8c742fb4
author : Peter Geoghegan <[email protected]>
date : Tue, 23 Mar 2021 16:09:51 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 23 Mar 2021 16:09:51 -0700
Teach nbtree VACUUM to press on with vacuuming in the event of a page
deletion attempt that fails to "re-find" a downlink for its child/target
page.
There is no good reason to treat this as an irrecoverable error. But
there is a good reason not to: pressing on at this point removes any
question of VACUUM not making progress solely due to misbehavior from
user-defined operator class code.
Discussion: https://postgr.es/m/CAH2-Wzma5G9CTtMjbrXTwOym+U=aWg-R7=-htySuztgoJLvZXg@mail.gmail.com
M src/backend/access/nbtree/nbtpage.c
Improve pg_amcheck's TAP test 003_check.pl.
commit : 87d90ac61fa113ffc886efcdb391c522c1982991
author : Robert Haas <[email protected]>
date : Tue, 23 Mar 2021 14:57:45 -0400
committer: Robert Haas <[email protected]>
date : Tue, 23 Mar 2021 14:57:45 -0400
Disable autovacuum, because we don't want it to run against
intentionally corrupted tables. Also, before corrupting the tables,
run pg_amcheck and ensure that it passes. Otherwise, if something
unexpected happens when we check the corrupted tables, it's not so
clear whether it would have also happened before we corrupted
them.
Mark Dilger
Discussion: http://postgr.es/m/[email protected]
M src/bin/pg_amcheck/t/003_check.pl
Fix psql's \connect command some more.
commit : ea80138545043c0cfcff8405b15626796f2695fe
author : Tom Lane <[email protected]>
date : Tue, 23 Mar 2021 14:27:50 -0400
committer: Tom Lane <[email protected]>
date : Tue, 23 Mar 2021 14:27:50 -0400
Jasen Betts reported yet another unintended side effect of commit
85c54287a: reconnecting with "\c service=whatever" did not have the
expected results. The reason is that starting from the output of
PQconndefaults() effectively allows environment variables (such
as PGPORT) to override entries in the service file, whereas the
normal priority is the other way around.
Not using PQconndefaults at all would require yet a third main code
path in do_connect's parameter setup, so I don't really want to fix
it that way. But we can have the logic effectively ignore all the
default values for just a couple more lines of code.
This patch doesn't change the behavior for "\c -reuse-previous=on
service=whatever". That remains significantly different from before
85c54287a, because many more parameters will be re-used, and thus
not be possible for service entries to replace. But I think this
is (mostly?) intentional. In any case, since libpq does not report
where it got parameter values from, it's hard to do differently.
Per bug #16936 from Jasen Betts. As with the previous patches,
back-patch to all supported branches. (9.5 is unfortunately now
out of support, so this won't get fixed there.)
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/command.c
Avoid possible crash while finishing up a heap rewrite.
commit : 9d523119fd38fd205cb9c8ea8e7cceeb54355818
author : Tom Lane <[email protected]>
date : Tue, 23 Mar 2021 11:24:16 -0400
committer: Tom Lane <[email protected]>
date : Tue, 23 Mar 2021 11:24:16 -0400
end_heap_rewrite was not careful to ensure that the target relation
is open at the smgr level before performing its final smgrimmedsync.
In ordinary cases this is no problem, because it would have been
opened earlier during the rewrite. However a crash can be reproduced
by re-clustering an empty table with CLOBBER_CACHE_ALWAYS enabled.
Although that exact scenario does not crash in v13, I think that's
a chance result of unrelated planner changes, and the problem is
likely still reachable with other test cases. The true proximate
cause of this failure is commit c6b92041d, which replaced a call to
heap_sync (which was careful about opening smgr) with a direct call
to smgrimmedsync. Hence, back-patch to v13.
Amul Sul, per report from Neha Sharma; cosmetic changes
and test case by me.
Discussion: https://postgr.es/m/CANiYTQsU7yMFpQYnv=BrcRVqK_3U3mtAzAsJCaqtzsDHfsUbdQ@mail.gmail.com
M src/backend/access/heap/rewriteheap.c
M src/test/regress/expected/cluster.out
M src/test/regress/sql/cluster.sql
pgcrypto: Check for error return of px_cipher_decrypt()
commit : 22e1943f13b66df22ea4f8d15836411ba259263a
author : Peter Eisentraut <[email protected]>
date : Tue, 23 Mar 2021 11:35:12 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 23 Mar 2021 11:35:12 +0100
This has previously not been a problem (that anyone ever reported),
but in future OpenSSL versions (3.0.0), where legacy ciphers are/can
be disabled, this is the place where this is reported. So we need to
catch the error here, otherwise the higher-level functions would
return garbage. The nearby encryption code already handled errors
similarly.
Reviewed-by: Daniel Gustafsson <[email protected]>
Discussion: https://www.postgresql.org/message-id/[email protected]
M contrib/pgcrypto/px.c
Add bit_count SQL function
commit : a6715af1e72da289474011be1e2d536f991eda34
author : Peter Eisentraut <[email protected]>
date : Tue, 23 Mar 2021 08:45:51 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 23 Mar 2021 08:45:51 +0100
This function for bit and bytea counts the set bits in the bit or byte
string. Internally, we use the existing popcount functionality.
For the name, after some discussion, we settled on bit_count, which
also exists with this meaning in MySQL, Java, and Python.
Author: David Fetter <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/func.sgml
M src/backend/utils/adt/varbit.c
M src/backend/utils/adt/varlena.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/bit.out
M src/test/regress/expected/strings.out
M src/test/regress/sql/bit.sql
M src/test/regress/sql/strings.sql
Add per-index stats information in verbose logs of autovacuum
commit : 5aed6a1fc214913de9ac69c1717dc64a2483e16d
author : Michael Paquier <[email protected]>
date : Tue, 23 Mar 2021 13:25:14 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 23 Mar 2021 13:25:14 +0900
Once a relation's autovacuum is completed, the logs include more
information about this relation state if the threshold of
log_autovacuum_min_duration (or its relation option) is reached, with
for example contents about the statistics of the VACUUM operation for
the relation, WAL and system usage.
This commit adds more information about the statistics of the relation's
indexes, with one line of logs generated for each index. The index
stats were already calculated, but not printed in the context of
autovacuum yet. While on it, some refactoring is done to keep track of
the index statistics directly within LVRelStats, simplifying some
routines related to parallel VACUUMs.
Author: Masahiko Sawada
Reviewed-by: Michael Paquier, Euler Taveira
Discussion: https://postgr.es/m/CAD21AoAy6SxHiTivh5yAPJSUE4S=QRPpSZUdafOSz0R+fRcM6Q@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Fix dangling pointer reference in stream_cleanup_files.
commit : 4b82ed6eca41220e50d4712ab929c20030b30d35
author : Amit Kapila <[email protected]>
date : Tue, 23 Mar 2021 09:43:33 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 23 Mar 2021 09:43:33 +0530
We can't access the entry after it is removed from dynahash.
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+Ps-pL++f6CJwPx2+vUqXuew=Xt-9Bi-6kCyxn+Fwi2M7w@mail.gmail.com
M src/backend/replication/logical/worker.c
Use correct spelling of statistics kind
commit : a5f002ad9a2ddb501148a12281efbaacec6f6397
author : Tomas Vondra <[email protected]>
date : Tue, 23 Mar 2021 04:45:26 +0100
committer: Tomas Vondra <[email protected]>
date : Tue, 23 Mar 2021 04:45:26 +0100
A couple error messages and comments used 'statistic kind', not the
correct 'statistics kind'. Fix and backpatch all the way back to 10,
where extended statistics were introduced.
Backpatch-through: 10
M doc/src/sgml/catalogs.sgml
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/statistics/mcv.c
M src/backend/statistics/mvdistinct.c
M src/include/nodes/pathnodes.h
Change the type of WalReceiverWaitStart wait event from Client to IPC.
commit : 1e3e8b51bda8ddd59984230f876f199c9ce3166a
author : Fujii Masao <[email protected]>
date : Tue, 23 Mar 2021 10:09:42 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 23 Mar 2021 10:09:42 +0900
Previously the type of this wait event was Client. But while this
wait event is being reported, walreceiver process is waiting for
the startup process to set initial data for streaming replication.
It's not waiting for any activity on a socket connected to a user
application or walsender. So this commit changes the type for
WalReceiverWaitStart wait event to IPC.
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
M src/backend/postmaster/pgstat.c
M src/include/pgstat.h
pg_waldump: Fix bug in per-record statistics.
commit : 51893c8463501fc9a38e39cc097773dbdfb9db82
author : Fujii Masao <[email protected]>
date : Tue, 23 Mar 2021 09:53:08 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 23 Mar 2021 09:53:08 +0900
pg_waldump --stats=record identifies a record by a combination
of the RmgrId and the four bits of the xl_info field of the record.
But XACT records use the first bit of those four bits for an optional
flag variable, and the following three bits for the opcode to
identify a record. So previously the same type of XACT record
could have different four bits (three bits are the same but the
first one bit is different), and which could cause
pg_waldump --stats=record to show two lines of per-record statistics
for the same XACT record. This is a bug.
This commit changes pg_waldump --stats=record so that it processes
only XACT record differently, i.e., filters the opcode out of xl_info
and uses a combination of the RmgrId and those three bits as
the identifier of a record, only for XACT record. For other records,
the four bits of the xl_info field are still used.
Back-patch to all supported branches.
Author: Kyotaro Horiguchi
Reviewed-by: Shinya Kato, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_waldump/pg_waldump.c
Add macro RelationIsPermanent() to report relation permanence
commit : 95d77149c53545a74e0c84717cf8f925b8f6d632
author : Bruce Momjian <[email protected]>
date : Mon, 22 Mar 2021 20:22:48 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 22 Mar 2021 20:22:48 -0400
Previously, to check relation permanence, the Relation's Form_pg_class
structure member relpersistence was compared to the value
RELPERSISTENCE_PERMANENT ("p"). This commit adds the macro
RelationIsPermanent() and is used in appropirate places to simplify the
code. This matches other RelationIs* macros.
This macro will be used in more places in future cluster file encryption
patches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/gist/gistutil.c
M src/backend/access/heap/heapam_handler.c
M src/backend/catalog/pg_publication.c
M src/backend/commands/tablecmds.c
M src/backend/optimizer/util/plancat.c
M src/backend/utils/cache/relcache.c
M src/include/utils/rel.h
M src/include/utils/snapmgr.h
Optimize allocations in bringetbitmap
commit : 8e4b332e88b8339408a3aa8c62bc93d96b67c808
author : Tomas Vondra <[email protected]>
date : Tue, 23 Mar 2021 00:47:06 +0100
committer: Tomas Vondra <[email protected]>
date : Tue, 23 Mar 2021 00:47:06 +0100
The bringetbitmap function allocates memory for various purposes, which
may be quite expensive, depending on the number of scan keys. Instead of
allocating them separately, allocate one bit chunk of memory an carve it
into smaller pieces as needed - all the pieces have the same lifespan,
and it saves quite a bit of CPU and memory overhead.
Author: Tomas Vondra <[email protected]>
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Mark Dilger <[email protected]>
Reviewed-by: Alexander Korotkov <[email protected]>
Reviewed-by: Masahiko Sawada <[email protected]>
Reviewed-by: John Naylor <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/brin/brin.c
Move IS [NOT] NULL handling from BRIN support functions
commit : 72ccf55cb99c6450dfb77f2f8f4a28b5c049ef7a
author : Tomas Vondra <[email protected]>
date : Tue, 23 Mar 2021 00:45:33 +0100
committer: Tomas Vondra <[email protected]>
date : Tue, 23 Mar 2021 00:45:33 +0100
The handling of IS [NOT] NULL clauses is independent of an opclass, and
most of the code was exactly the same in both minmax and inclusion. So
instead move the code from support procedures to the AM.
This simplifies the code - especially the support procedures - quite a
bit, as they don't need to care about NULL values and flags at all. It
also means the IS [NOT] NULL clauses can be evaluated without invoking
the support procedure.
Author: Tomas Vondra <[email protected]>
Author: Nikita Glukhov <[email protected]>
Reviewed-by: Nikita Glukhov <[email protected]>
Reviewed-by: Mark Dilger <[email protected]>
Reviewed-by: Alexander Korotkov <[email protected]>
Reviewed-by: Masahiko Sawada <[email protected]>
Reviewed-by: John Naylor <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_inclusion.c
M src/backend/access/brin/brin_minmax.c
M src/include/access/brin_internal.h
Pass all scan keys to BRIN consistent function at once
commit : a1c649d889bdf6e74e9382e1e28574d7071568de
author : Tomas Vondra <[email protected]>
date : Tue, 23 Mar 2021 00:12:19 +0100
committer: Tomas Vondra <[email protected]>
date : Tue, 23 Mar 2021 00:12:19 +0100
This commit changes how we pass scan keys to BRIN consistent function.
Instead of passing them one by one, we now pass all scan keys for a
given attribute at once. That makes the consistent function a bit more
complex, as it has to loop through the keys, but it does allow more
elaborate opclasses that can use multiple keys to eliminate ranges much
more effectively.
The existing BRIN opclasses (minmax, inclusion) don't really benefit
from this change. The primary purpose is to allow future opclases to
benefit from seeing all keys at once.
This does change the BRIN API, because the signature of the consistent
function changes (a new parameter with number of scan keys). So this
breaks existing opclasses, and will require supporting two variants of
the code for different PostgreSQL versions. We've considered supporting
two variants of the consistent, but we've decided not to do that.
Firstly, there's another patch that moves handling of NULL values from
the opclass, which means the opclasses need to be updated anyway.
Secondly, we're not aware of any out-of-core BRIN opclasses, so it does
not seem worth the extra complexity.
Bump catversion, because of pg_proc changes.
Author: Tomas Vondra <[email protected]>
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Mark Dilger <[email protected]>
Reviewed-by: Alexander Korotkov <[email protected]>
Reviewed-by: John Naylor <[email protected]>
Reviewed-by: Nikita Glukhov <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/brin.sgml
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_inclusion.c
M src/backend/access/brin/brin_minmax.c
M src/backend/access/brin/brin_validate.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Move bsearch_arg to src/port
commit : bfa2cee784125047771db2768fcf7f04d8bd6bb4
author : Tomas Vondra <[email protected]>
date : Tue, 23 Mar 2021 00:11:20 +0100
committer: Tomas Vondra <[email protected]>
date : Tue, 23 Mar 2021 00:11:20 +0100
Until now the bsearch_arg function was used only in extended statistics
code, so it was defined in that code. But we already have qsort_arg in
src/port, so let's move it next to it.
M src/backend/statistics/extended_stats.c
M src/include/port.h
M src/include/statistics/extended_stats_internal.h
M src/port/Makefile
A src/port/bsearch_arg.c
M src/tools/msvc/Mkvcbuild.pm
Short-circuit slice requests that are for more than the object's size.
commit : 063dd37ebc7644e8db6419565b50dca019e69e86
author : Tom Lane <[email protected]>
date : Mon, 22 Mar 2021 14:01:20 -0400
committer: Tom Lane <[email protected]>
date : Mon, 22 Mar 2021 14:01:20 -0400
substring(), and perhaps other callers, isn't careful to pass a
slice length that is no more than the datum's true size. Since
toast_decompress_datum_slice's children will palloc the requested
slice length, this can waste memory. Also, close study of the liblz4
documentation suggests that it is dependent on the caller to not ask
for more than the correct amount of decompressed data; this squares
with observed misbehavior with liblz4 1.8.3. Avoid these problems
by switching to the normal full-decompression code path if the
slice request is >= datum's decompressed size.
Tom Lane and Dilip Kumar
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/common/detoast.c
M src/include/access/toast_internals.h
Mostly-cosmetic adjustments of TOAST-related macros.
commit : aeb1631ed207cef2d80e20f79eb52c72f03bca7d
author : Tom Lane <[email protected]>
date : Mon, 22 Mar 2021 13:43:10 -0400
committer: Tom Lane <[email protected]>
date : Mon, 22 Mar 2021 13:43:10 -0400
The authors of bbe0a81db hadn't quite got the idea that macros named
like SOMETHING_4B_C were only meant for internal endianness-related
details in postgres.h. Choose more legible names for macros that are
intended to be used elsewhere. Rearrange postgres.h a bit to clarify
the separation between those internal macros and ones intended for
wider use.
Also, avoid using the term "rawsize" for true decompressed size;
we've used "extsize" for that, because "rawsize" generally denotes
total Datum size including header. This choice seemed particularly
unfortunate in tests that were comparing one of these meanings to
the other.
This patch includes a couple of not-purely-cosmetic changes: be
sure that the shifts aligning compression methods are unsigned
(not critical today, but will be when compression method 2 exists),
and fix broken definition of VARATT_EXTERNAL_GET_COMPRESSION (now
VARATT_EXTERNAL_GET_COMPRESS_METHOD), whose callers worked only
accidentally.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/common/detoast.c
M src/backend/access/common/toast_compression.c
M src/backend/access/common/toast_internals.c
M src/include/access/toast_internals.h
M src/include/postgres.h
Remove useless configure probe for <lz4/lz4.h>.
commit : 2c75f8a612b207c7d36e5dc73317dc9ab6fb29d4
author : Tom Lane <[email protected]>
date : Mon, 22 Mar 2021 11:20:44 -0400
committer: Tom Lane <[email protected]>
date : Mon, 22 Mar 2021 11:20:44 -0400
This seems to have been just copied-and-pasted from some other
header checks. But our C code is entirely unprepared to support
such a header name, so it's only wasting cycles to look for it.
If we did need to support it, some #ifdefs would be required.
(A quick trawl at codesearch.debian.net finds some packages that
reference lz4/lz4.h; but they use *only* that spelling, and
appear to be intending to reference their own copy rather than
a system-level installation of liblz4. There's no evidence of
freestanding installations that require this spelling.)
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
M src/include/pg_config.h.in
M src/tools/msvc/Solution.pm
Error on invalid TOAST compression in CREATE or ALTER TABLE.
commit : a4d5284a10b5096585f1bbf1bf723954e9d6c2e0
author : Robert Haas <[email protected]>
date : Mon, 22 Mar 2021 10:57:08 -0400
committer: Robert Haas <[email protected]>
date : Mon, 22 Mar 2021 10:57:08 -0400
The previous coding treated an invalid compression method name as
equivalent to the default, which is certainly not right.
Justin Pryzby
Discussion: http://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/test/regress/expected/compression.out
M src/test/regress/expected/compression_1.out
M src/test/regress/sql/compression.sql
docs: Fix omissions related to configurable TOAST compression.
commit : 24f0e395ac5892cd12e8914646fe921fac5ba23d
author : Robert Haas <[email protected]>
date : Mon, 22 Mar 2021 10:34:10 -0400
committer: Robert Haas <[email protected]>
date : Mon, 22 Mar 2021 10:34:10 -0400
Previously, the default_toast_compression GUC was not documented,
and neither was pg_dump's new --no-toast-compression option.
Justin Pryzby and Robert Haas
Discussion: http://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/pg_dump.sgml
More code cleanup for configurable TOAST compression.
commit : 226e2be3876d0bda3dc33d16dfa0bed246b7b74f
author : Robert Haas <[email protected]>
date : Mon, 22 Mar 2021 09:21:37 -0400
committer: Robert Haas <[email protected]>
date : Mon, 22 Mar 2021 09:21:37 -0400
Remove unused macro. Fix confusion about whether a TOAST compression
method is identified by an OID or a char.
Justin Pryzby
Discussion: http://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/include/access/toast_compression.h
Fix concurrency issues with WAL segment recycling on Windows
commit : 909b449e00fc2f71e1a38569bbddbb6457d28485
author : Michael Paquier <[email protected]>
date : Mon, 22 Mar 2021 14:02:26 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 22 Mar 2021 14:02:26 +0900
This commit is mostly a revert of aaa3aed, that switched the routine
doing the internal renaming of recycled WAL segments to use on Windows a
combination of CreateHardLinkA() plus unlink() instead of rename(). As
reported by several users of Postgres 13, this is causing concurrency
issues when manipulating WAL segments, mostly in the shape of the
following error:
LOG: could not rename file "pg_wal/000000XX000000YY000000ZZ":
Permission denied
This moves back to a logic where a single rename() (well, pgrename() for
Windows) is used. This issue has proved to be hard to hit when I tested
it, facing it only once with an archive_command that was not able to do
its work, so it is environment-sensitive. The reporters of this issue
have been able to confirm that the situation improved once we switched
back to a single rename(). In order to check things, I have provided to
the reporters a patched build based on 13.2 with aaa3aed reverted, to
test if the error goes away, and an unpatched build of 13.2 to test if
the error still showed up (just to make sure that I did not mess up my
build process).
Extra thanks to Fujii Masao for pointing out what looked like the
culprit commit, and to all the reporters for taking the time to test
what I have sent them.
Reported-by: Andrus, Guy Burgess, Yaroslav Pashinsky, Thomas Trenz
Reviewed-by: Tom Lane, Andres Freund
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/16927-67c570d968c99567%40postgresql.org
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/backend/storage/file/fd.c
M src/include/pg_config_manual.h
pgbench: Improve error-handling in \sleep command.
commit : 8c6eda2d1c926be76baa79c28521275323bd26fd
author : Fujii Masao <[email protected]>
date : Mon, 22 Mar 2021 12:02:44 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 22 Mar 2021 12:02:44 +0900
This commit improves pgbench \sleep command so that it handles
the following three cases more properly.
(1) When only one argument was specified in \sleep command and
it's not a number, previously pgbench reported a confusing error
message like "unrecognized time unit, must be us, ms or s".
This commit fixes this so that more proper error message like
"invalid sleep time, must be an integer" is reported.
(2) When two arguments were specified in \sleep command and
the first argument was not a number, previously pgbench treated
that argument as the sleep time 0. No error was reported in this
case. This commit fixes this so that an error is thrown in this
case.
(3) When a variable was specified as the first argument in \sleep
command and the variable stored non-digit value, previously
pgbench treated that argument as the sleep time 0. No error
was reported in this case. This commit fixes this so that
an error is thrown in this case.
Author: Kota Miyake
Reviewed-by: Hayato Kuroda, Alvaro Herrera, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/bin/pgbench/pgbench.c
Make a test endure log_error_verbosity=verbose.
commit : e3f4aec027891f794328050e62c9bbbe4ae02811
author : Noah Misch <[email protected]>
date : Sun, 21 Mar 2021 19:09:29 -0700
committer: Noah Misch <[email protected]>
date : Sun, 21 Mar 2021 19:09:29 -0700
Back-patch to v13, which introduced the test code in question.
M src/test/recovery/t/003_recovery_targets.pl
Fix new TAP test for 2PC transactions and PITRs on Windows
commit : 992d353a190c551db39bcab2dec0ecf14fbc7a40
author : Michael Paquier <[email protected]>
date : Mon, 22 Mar 2021 09:51:05 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 22 Mar 2021 09:51:05 +0900
The test added by 595b9cb forgot that on Windows it is necessary to set
up pg_hba.conf (see PostgresNode::set_replication_conf) with a specific
entry or base backups fail. Any node that requires to support
replication just needs to pass down allows_streaming at initialization.
This updates the test to do so. Simplify things a bit while on it.
Per buildfarm member fairywren. Any Windows hosts running this test
would have failed, and I have reproduced the problem as well.
Backpatch-through: 10
M src/test/recovery/t/023_pitr_prepared_xact.pl
Simplify TAP tests of kerberos with expected log file contents
commit : 11e1577a576fec6307aa0bfcde7333e63f907fa7
author : Michael Paquier <[email protected]>
date : Mon, 22 Mar 2021 08:59:43 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 22 Mar 2021 08:59:43 +0900
The TAP tests of kerberos rely on the logs generated by the backend to
check various connection scenarios. In order to make sure that a given
test does not overlap with the log contents generated by a previous
test, the test suite relied on a logic with the logging collector and a
rotation of the log files to ensure the uniqueness of the log generated
with a wait phase.
Parsing the log contents for expected patterns is a problem that has
been solved in a simpler way by PostgresNode::issues_sql_like() where
the log file is truncated before checking for the contents generated,
with the backend sending its output to a log file given by pg_ctl
instead. This commit switches the kerberos test suite to use such a
method, removing any wait phase and simplifying the whole logic,
resulting in less code. If a failure happens in the tests, the contents
of the logs are still showed to the user at the moment of the failure
thanks to like(), so this has no impact on debugging capabilities.
I have bumped into this issue while reviewing a different patch set
aiming at extending the kerberos test suite to check for multiple
log patterns instead of one now.
Author: Michael Paquier
Reviewed-by: Stephen Frost, Bharath Rupireddy
Discussion: https://postgr.es/m/[email protected]
M src/test/kerberos/t/001_auth.pl
Fix timeline assignment in checkpoints with 2PC transactions
commit : 595b9cba2ab0cdd057e02d3c23f34a8bcfd90a2d
author : Michael Paquier <[email protected]>
date : Mon, 22 Mar 2021 08:30:53 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 22 Mar 2021 08:30:53 +0900
Any transactions found as still prepared by a checkpoint have their
state data read from the WAL records generated by PREPARE TRANSACTION
before being moved into their new location within pg_twophase/. While
reading such records, the WAL reader uses the callback
read_local_xlog_page() to read a page, that is shared across various
parts of the system. This callback, since 1148e22a, has introduced an
update of ThisTimeLineID when reading a record while in recovery, which
is potentially helpful in the context of cascading WAL senders.
This update of ThisTimeLineID interacts badly with the checkpointer if a
promotion happens while some 2PC data is read from its record, as, by
changing ThisTimeLineID, any follow-up WAL records would be written to
an timeline older than the promoted one. This results in consistency
issues. For instance, a subsequent server restart would cause a failure
in finding a valid checkpoint record, resulting in a PANIC, for
instance.
This commit changes the code reading the 2PC data to reset the timeline
once the 2PC record has been read, to prevent messing up with the static
state of the checkpointer. It would be tempting to do the same thing
directly in read_local_xlog_page(). However, based on the discussion
that has led to 1148e22a, users may rely on the updates of
ThisTimeLineID when a WAL record page is read in recovery, so changing
this callback could break some cases that are working currently.
A TAP test reproducing the issue is added, relying on a PITR to
precisely trigger a promotion with a prepared transaction still
tracked.
Per discussion with Heikki Linnakangas, Kyotaro Horiguchi, Fujii Masao
and myself.
Author: Soumyadeep Chakraborty, Jimmy Yih, Kevin Yeap
Discussion: https://postgr.es/m/CAE-ML+_EjH_fzfq1F3RJ1=XaaNG=-Jz-i3JqkNhXiLAsM3z-Ew@mail.gmail.com
Backpatch-through: 10
M src/backend/access/transam/twophase.c
A src/test/recovery/t/023_pitr_prepared_xact.pl
Fix assorted silliness in ATExecSetCompression().
commit : ac897c483485d3858ada23ca49650a0f2742a50f
author : Tom Lane <[email protected]>
date : Sun, 21 Mar 2021 18:42:40 -0400
committer: Tom Lane <[email protected]>
date : Sun, 21 Mar 2021 18:42:40 -0400
It's not okay to scribble directly on a syscache entry.
Nor to continue accessing said entry after releasing it.
Also get rid of not-used local variables.
Per valgrind testing.
M src/backend/commands/tablecmds.c
Recycle nbtree pages deleted during same VACUUM.
commit : 9dd963ae2534e9614f0abeccaafbd39f1b93ff8a
author : Peter Geoghegan <[email protected]>
date : Sun, 21 Mar 2021 15:25:39 -0700
committer: Peter Geoghegan <[email protected]>
date : Sun, 21 Mar 2021 15:25:39 -0700
Maintain a simple array of metadata about pages that were deleted during
nbtree VACUUM's current btvacuumscan() call. Use this metadata at the
end of btvacuumscan() to attempt to place newly deleted pages in the FSM
without further delay. It might not yet be safe to place any of the
pages in the FSM by then (they may not be deemed recyclable), but we
have little to lose and plenty to gain by trying. In practice there is
a very good chance that this will work out when vacuuming larger
indexes, where scanning the index naturally takes quite a while.
This commit doesn't change the page recycling invariants; it merely
improves the efficiency of page recycling within the confines of the
existing design. Recycle safety is a part of nbtree's implementation of
what Lanin & Shasha call "the drain technique". The design happens to
use transaction IDs (they're stored in deleted pages), but that in
itself doesn't align the cutoff for recycle safety to any of the
XID-based cutoffs used by VACUUM (e.g., OldestXmin). All that matters
is whether or not _other_ backends might be able to observe various
inconsistencies in the tree structure (that they cannot just detect and
recover from by moving right). Recycle safety is purely a question of
maintaining the consistency (or the apparent consistency) of a physical
data structure.
Note that running a simple serial test case involving a large range
DELETE followed by a VACUUM VERBOSE will probably show that any newly
deleted nbtree pages are not yet reusable/recyclable. This is expected
in the absence of even one concurrent XID assignment. It is an old
implementation restriction. In practice it's unlikely to be the thing
that makes recycling remain unsafe, at least with larger indexes, where
recycling newly deleted pages during the same VACUUM actually matters.
An important high-level goal of this commit (as well as related recent
commits e5d8a999 and 9f3665fb) is to make expensive deferred cleanup
operations in index AMs rare in general. If index vacuuming frequently
depends on the next VACUUM operation finishing off work that the current
operation started, then the general behavior of index vacuuming is hard
to predict. This is relevant to ongoing work that adds a vacuumlazy.c
mechanism to skip index vacuuming in certain cases. Anything that makes
the real world behavior of index vacuuming simpler and more linear will
also make top-down modeling in vacuumlazy.c more robust.
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wzk76_P=67iUscb1UN44-gyZL-KgpsXbSxq_bdcMa7Q+wQ@mail.gmail.com
M src/backend/access/nbtree/README
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/include/access/nbtree.h
Bring configure support for LZ4 up to snuff.
commit : 4d399a6fbeb720b34d33441330910b7d853f703d
author : Tom Lane <[email protected]>
date : Sun, 21 Mar 2021 17:20:17 -0400
committer: Tom Lane <[email protected]>
date : Sun, 21 Mar 2021 17:20:17 -0400
It's not okay to just shove the pkg_config results right into our
build flags, for a couple different reasons:
* This fails to maintain the separation between CPPFLAGS and CFLAGS,
as well as that between LDFLAGS and LIBS. (The CPPFLAGS angle is,
I believe, the reason for warning messages reported when building
with MacPorts' liblz4.)
* If pkg_config emits anything other than -I/-D/-L/-l switches,
it's highly unlikely that we want to absorb those. That'd be more
likely to break the build than do anything helpful. (Even the -D
case is questionable; but we're doing that for libxml2, so I kept it.)
Also, it's not okay to skip doing an AC_CHECK_LIB probe, as
evidenced by recent build failure on topminnow; that should
have been caught at configure time.
Model fixes for this on configure's libxml2 support.
It appears that somebody overlooked an autoheader run, too.
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
M src/include/pg_config.h.in
M src/tools/msvc/Solution.pm
Make compression.sql regression test independent of default.
commit : fd1ac9a548966786cf7978e590be816c55936a50
author : Tom Lane <[email protected]>
date : Sun, 21 Mar 2021 16:26:44 -0400
committer: Tom Lane <[email protected]>
date : Sun, 21 Mar 2021 16:26:44 -0400
This test will fail in "make installcheck" if the installation's
default_toast_compression setting is not 'pglz'. Make it robust
against that situation.
Dilip Kumar
Discussion: https://postgr.es/m/CAFiTN-t0w+Rc2U3S+y=7KWcLuOYNB5MfWeGdNa7+pg0UovVdcQ@mail.gmail.com
M src/test/regress/expected/compression.out
M src/test/regress/expected/compression_1.out
M src/test/regress/sql/compression.sql
Don't run recover crash_temp_files test in Windows perl
commit : ef823873840c9f341239e18633bdb0116d8d2738
author : Andrew Dunstan <[email protected]>
date : Sun, 21 Mar 2021 15:04:45 -0400
committer: Andrew Dunstan <[email protected]>
date : Sun, 21 Mar 2021 15:04:45 -0400
This reverts commit 677271a3a125e294b33b891669f594a2c8cb36ce.
"Unbreak recovery test on Windows"
The test hangs on Windows, and attempts to remedy the problem have
proved fragile at best. So we simply disable the test on Windows perl.
(Msys perl seems perfectly happy).
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/022_crash_temp_files.pl
Fix new memory leaks in libpq
commit : 2b526ed2e1cbaa54e5ad0c12d1294482f2757b17
author : Alvaro Herrera <[email protected]>
date : Sun, 21 Mar 2021 14:55:27 -0300
committer: Alvaro Herrera <[email protected]>
date : Sun, 21 Mar 2021 14:55:27 -0300
My oversight in commit 9aa491abbf07.
Per coverity.
M src/interfaces/libpq/fe-exec.c
Unbreak recovery test on Windows
commit : 677271a3a125e294b33b891669f594a2c8cb36ce
author : Andrew Dunstan <[email protected]>
date : Sun, 21 Mar 2021 11:52:30 -0400
committer: Andrew Dunstan <[email protected]>
date : Sun, 21 Mar 2021 11:52:30 -0400
On Windows we need to send explicit quit messages to psql or the TAP tests
can hang.
M src/test/recovery/t/022_crash_temp_files.pl
Suppress various new compiler warnings.
commit : 9fb9691a88ae8df9bc30e0f0f72de7c96e73e125
author : Tom Lane <[email protected]>
date : Sun, 21 Mar 2021 11:50:43 -0400
committer: Tom Lane <[email protected]>
date : Sun, 21 Mar 2021 11:50:43 -0400
Compilers that don't understand that elog(ERROR) doesn't return
issued warnings here. In the cases in libpq_pipeline.c, we were
not exactly helping things by failing to mark pg_fatal() as noreturn.
Per buildfarm.
M src/backend/access/common/detoast.c
M src/backend/access/common/toast_compression.c
M src/include/access/toast_compression.h
M src/test/modules/libpq_pipeline/libpq_pipeline.c
Move lwlock-release probe back where it belongs
commit : 96ae658e6238c5e69819fb1557c2c14a555506d8
author : Peter Eisentraut <[email protected]>
date : Sun, 21 Mar 2021 08:02:30 +0100
committer: Peter Eisentraut <[email protected]>
date : Sun, 21 Mar 2021 08:02:30 +0100
The documentation specifically states that lwlock-release fires before
any released waiters have been awakened. It worked that way until
ab5194e6f617a9a9e7aadb3dd1cee948a42d0755, where is seems to have been
misplaced accidentally. Move it back where it belongs.
Author: Craig Ringer <[email protected]>
Discussion: https://www.postgresql.org/message-id/CAGRY4nwxKUS_RvXFW-ugrZBYxPFFM5kjwKT5O+0+Stuga5b4+Q@mail.gmail.com
M src/backend/storage/lmgr/lwlock.c
Use valid compression method in brin_form_tuple
commit : 882b2cdc08c4100e273f24742e2118be98708a07
author : Tomas Vondra <[email protected]>
date : Sun, 21 Mar 2021 00:28:13 +0100
committer: Tomas Vondra <[email protected]>
date : Sun, 21 Mar 2021 00:28:13 +0100
When compressing the BRIN summary, we can't simply use the compression
method from the indexed attribute. The summary may use a different data
type, e.g. fixed-length attribute may have varlena summary, leading to
compression failures. For the built-in BRIN opclasses this happens to
work, because the summary uses the same data type as the attribute.
When the data types match, we can inherit use the compression method
specified for the attribute (it's copied into the index descriptor).
Otherwise we don't have much choice and have to use the default one.
Author: Tomas Vondra
Reviewed-by: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/e0367f27-392c-321a-7411-a58e1a7e4817%40enterprisedb.com
M src/backend/access/brin/brin_tuple.c
Fix up pg_dump's handling of per-attribute compression options.
commit : aa25d1089ac00bbc3f97d2efe8f54c3d4beed5d1
author : Tom Lane <[email protected]>
date : Sat, 20 Mar 2021 15:01:10 -0400
committer: Tom Lane <[email protected]>
date : Sat, 20 Mar 2021 15:01:10 -0400
The approach used in commit bbe0a81db would've been disastrous for
portability of dumps. Instead handle non-default compression options
in separate ALTER TABLE commands. This reduces chatter for the
common case where most columns are compressed the same way, and it
makes it possible to restore the dump to a server that lacks any
knowledge of per-attribute compression options (so long as you're
willing to ignore syntax errors from the ALTER TABLE commands).
There's a whole lot left to do to mop up after bbe0a81db, but
I'm fast-tracking this part because we need to see if it's
enough to make the buildfarm's cross-version-upgrade tests happy.
Justin Pryzby and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
Fix memory leak when rejecting bogus DH parameters.
commit : e835e89a0fd267871e7fbddc39ad00ee3d0cb55c
author : Tom Lane <[email protected]>
date : Sat, 20 Mar 2021 12:47:21 -0400
committer: Tom Lane <[email protected]>
date : Sat, 20 Mar 2021 12:47:21 -0400
While back-patching e0e569e1d, I noted that there were some other
places where we ought to be applying DH_free(); namely, where we
load some DH parameters from a file and then reject them as not
being sufficiently secure. While it seems really unlikely that
anybody would hit these code paths in production, let alone do
so repeatedly, let's fix it for consistency.
Back-patch to v10 where this code was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/be-secure-openssl.c
Avoid leaking memory in RestoreGUCState(), and improve comments.
commit : f0c2a5bba6c566fad781802537eb17f2977702bc
author : Tom Lane <[email protected]>
date : Fri, 19 Mar 2021 23:03:17 -0400
committer: Tom Lane <[email protected]>
date : Fri, 19 Mar 2021 23:03:17 -0400
RestoreGUCState applied InitializeOneGUCOption to already-live
GUC entries, causing any malloc'd subsidiary data to be forgotten.
We do want the effect of resetting the GUC to its compiled-in
default, and InitializeOneGUCOption seems like the best way to do
that, so add code to free any existing subsidiary data beforehand.
The interaction between can_skip_gucvar, SerializeGUCState, and
RestoreGUCState is way more subtle than their opaque comments
would suggest to an unwary reader. Rewrite and enlarge the
comments to try to make it clearer what's happening.
Remove a long-obsolete assertion in read_nondefault_variables: the
behavior of set_config_option hasn't depended on IsInitProcessingMode
since f5d9698a8 installed a better way of controlling it.
Although this is fixing a clear memory leak, the leak is quite unlikely
to involve any large amount of data, and it can only happen once in the
lifetime of a worker process. So it seems unnecessary to take any
risk of back-patching.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/misc/guc.c
Provide recovery_init_sync_method=syncfs.
commit : 61752afb26404dfc99a535c7a53f7f04dc110263
author : Thomas Munro <[email protected]>
date : Sat, 20 Mar 2021 11:46:32 +1300
committer: Thomas Munro <[email protected]>
date : Sat, 20 Mar 2021 11:46:32 +1300
Since commit 2ce439f3 we have opened every file in the data directory
and called fsync() at the start of crash recovery. This can be very
slow if there are many files, leading to field complaints of systems
taking minutes or even hours to begin crash recovery.
Provide an alternative method, for Linux only, where we call syncfs() on
every possibly different filesystem under the data directory. This is
equivalent, but avoids faulting in potentially many inodes from
potentially slow storage.
The new mode comes with some caveats, described in the documentation, so
the default value for the new setting is "fsync", preserving the older
behavior.
Reported-by: Michael Brown <[email protected]>
Reviewed-by: Fujii Masao <[email protected]>
Reviewed-by: Paul Guo <[email protected]>
Reviewed-by: Bruce Momjian <[email protected]>
Reviewed-by: Justin Pryzby <[email protected]>
Reviewed-by: David Steele <[email protected]>
Discussion: https://postgr.es/m/11bc2bb7-ecb5-3ad0-b39f-df632734cd81%40discourse.org
Discussion: https://postgr.es/m/CAEET0ZHGnbXmi8yF3ywsDZvb3m9CbdsGZgfTXscQ6agcbzcZAw%40mail.gmail.com
M configure
M configure.ac
M doc/src/sgml/config.sgml
M src/backend/storage/file/fd.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/pg_config.h.in
M src/include/storage/fd.h
M src/tools/msvc/Solution.pm
Use lfirst_int in cmp_list_len_contents_asc
commit : b822ae13ea93c18326d58d47829bbc66d36fae5c
author : Tomas Vondra <[email protected]>
date : Fri, 19 Mar 2021 23:57:50 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 19 Mar 2021 23:57:50 +0100
The function added in be45be9c33 is comparing integer lists (IntList) by
length and contents, but there were two bugs. Firstly, it used intVal()
to extract the value, but that's for Value nodes, not for extracting int
values from IntList. Secondly, it called it directly on the ListCell,
without doing lfirst(). So just do lfirst_int() instead.
Interestingly enough, this did not cause any crashes on the buildfarm,
but valgrind rightfully complained about it.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/parse_agg.c
Fix use-after-ReleaseSysCache problem in ATExecAlterColumnType.
commit : d00fbdc431192c3e429b3e91c43d364e5c7ba680
author : Robert Haas <[email protected]>
date : Fri, 19 Mar 2021 17:17:48 -0400
committer: Robert Haas <[email protected]>
date : Fri, 19 Mar 2021 17:17:48 -0400
Introduced by commit bbe0a81db69bd10bd166907c3701492a29aca294.
Per buildfarm member prion.
M src/backend/commands/tablecmds.c
Allow configurable LZ4 TOAST compression.
commit : bbe0a81db69bd10bd166907c3701492a29aca294
author : Robert Haas <[email protected]>
date : Fri, 19 Mar 2021 15:10:38 -0400
committer: Robert Haas <[email protected]>
date : Fri, 19 Mar 2021 15:10:38 -0400
There is now a per-column COMPRESSION option which can be set to pglz
(the default, and the only option in up until now) or lz4. Or, if you
like, you can set the new default_toast_compression GUC to lz4, and
then that will be the default for new table columns for which no value
is specified. We don't have lz4 support in the PostgreSQL code, so
to use lz4 compression, PostgreSQL must be built --with-lz4.
In general, TOAST compression means compression of individual column
values, not the whole tuple, and those values can either be compressed
inline within the tuple or compressed and then stored externally in
the TOAST table, so those properties also apply to this feature.
Prior to this commit, a TOAST pointer has two unused bits as part of
the va_extsize field, and a compessed datum has two unused bits as
part of the va_rawsize field. These bits are unused because the length
of a varlena is limited to 1GB; we now use them to indicate the
compression type that was used. This means we only have bit space for
2 more built-in compresison types, but we could work around that
problem, if necessary, by introducing a new vartag_external value for
any further types we end up wanting to add. Hopefully, it won't be
too important to offer a wide selection of algorithms here, since
each one we add not only takes more coding but also adds a build
dependency for every packager. Nevertheless, it seems worth doing
at least this much, because LZ4 gets better compression than PGLZ
with less CPU usage.
It's possible for LZ4-compressed datums to leak into composite type
values stored on disk, just as it is for PGLZ. It's also possible for
LZ4-compressed attributes to be copied into a different table via SQL
commands such as CREATE TABLE AS or INSERT .. SELECT. It would be
expensive to force such values to be decompressed, so PostgreSQL has
never done so. For the same reasons, we also don't force recompression
of already-compressed values even if the target table prefers a
different compression method than was used for the source data. These
architectural decisions are perhaps arguable but revisiting them is
well beyond the scope of what seemed possible to do as part of this
project. However, it's relatively cheap to recompress as part of
VACUUM FULL or CLUSTER, so this commit adjusts those commands to do
so, if the configured compression method of the table happens not to
match what was used for some column value stored therein.
Dilip Kumar. The original patches on which this work was based were
written by Ildus Kurbangaliev, and those were patches were based on
even earlier work by Nikita Glukhov, but the design has since changed
very substantially, since allow a potentially large number of
compression methods that could be added and dropped on a running
system proved too problematic given some of the architectural issues
mentioned above; the choice of which specific compression method to
add first is now different; and a lot of the code has been heavily
refactored. More recently, Justin Przyby helped quite a bit with
testing and reviewing and this version also includes some code
contributions from him. Other design input and review from Tomas
Vondra, Álvaro Herrera, Andres Freund, Oleg Bartunov, Alexander
Korotkov, and me.
Discussion: http://postgr.es/m/20170907194236.4cefce96%40wp.localdomain
Discussion: http://postgr.es/m/CAFiTN-uUpX3ck%3DK0mLEk-G_kUQY%3DSNOTeqdaNRR9FMdQrHKebw%40mail.gmail.com
M configure
M configure.ac
M contrib/amcheck/verify_heapam.c
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/psql-ref.sgml
M src/backend/access/brin/brin_tuple.c
M src/backend/access/common/Makefile
M src/backend/access/common/detoast.c
M src/backend/access/common/indextuple.c
A src/backend/access/common/toast_compression.c
M src/backend/access/common/toast_internals.c
M src/backend/access/common/tupdesc.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/table/toast_helper.c
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/toasting.c
M src/backend/commands/tablecmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_utilcmd.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/utils/adt/varlena.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/pg_amcheck/t/004_verify_heapam.pl
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/psql/describe.c
M src/bin/psql/help.c
M src/bin/psql/settings.h
M src/bin/psql/startup.c
M src/bin/psql/tab-complete.c
M src/include/access/detoast.h
A src/include/access/toast_compression.h
M src/include/access/toast_helper.h
M src/include/access/toast_internals.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_proc.dat
M src/include/nodes/parsenodes.h
M src/include/parser/kwlist.h
M src/include/pg_config.h.in
M src/include/postgres.h
A src/test/regress/expected/compression.out
A src/test/regress/expected/compression_1.out
M src/test/regress/parallel_schedule
M src/test/regress/pg_regress_main.c
M src/test/regress/serial_schedule
A src/test/regress/sql/compression.sql
M src/tools/msvc/Solution.pm
Fix race condition in remove_temp_files_after_crash TAP test
commit : e589c4890b05044a04207c2797e7c8af6693ea5f
author : Tomas Vondra <[email protected]>
date : Fri, 19 Mar 2021 18:12:39 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 19 Mar 2021 18:12:39 +0100
The TAP test was written so that it was not waiting for the correct SQL
command, but for output from the preceding one. This resulted in race
conditions, allowing the commands to run in a different order, not block
as expected and so on. This fixes it by inverting the order of commands
where possible, so that observing the output guarantees the data was
inserted properly, and waiting for a lock to appear in pg_locks.
Discussion: https://postgr.es/m/CAH503wDKdYzyq7U-QJqGn%3DGm6XmoK%2B6_6xTJ-Yn5WSvoHLY1Ww%40mail.gmail.com
M src/test/recovery/t/022_crash_temp_files.pl
Blindly try to fix test script's tar invocation for MSYS.
commit : 27ab1981e7c9b8fcbcb143c5f6f706441a52bbc8
author : Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 22:43:03 -0400
committer: Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 22:43:03 -0400
Buildfarm member fairywren doesn't like the test case I added
in commit 081876d75. I'm guessing the reason is that I shouldn't
be using a perl2host-ified path in the tar command line.
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
Fix comments in postmaster.c.
commit : fd31214075cc740e43edc71ca1c385c8c53047b7
author : Fujii Masao <[email protected]>
date : Fri, 19 Mar 2021 11:28:54 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 19 Mar 2021 11:28:54 +0900
Commit 86c23a6eb2 changed the option to specify that postgres will
stop all other server processes by sending the signal SIGSTOP,
from -s to -T. But previously there were comments incorrectly
explaining that SIGSTOP behavior is set by -s option. This commit
fixes them.
Author: Kyotaro Horiguchi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/postmaster.c
Don't leak malloc'd error string in libpqrcv_check_conninfo().
commit : 9bacdf9f536a3720976ae258238cb46c691ce9b2
author : Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 22:21:58 -0400
committer: Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 22:21:58 -0400
We leaked the error report from PQconninfoParse, when there was
one. It seems unlikely that real usage patterns would repeat
the failure often enough to create serious bloat, but let's
back-patch anyway to keep the code similar in all branches.
Found via valgrind testing.
Back-patch to v10 where this code was added.
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
Don't leak malloc'd strings when a GUC setting is rejected.
commit : 377b7a83007d277d32ef19f7c7590c8668d504cb
author : Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 22:09:41 -0400
committer: Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 22:09:41 -0400
Because guc.c prefers to keep all its string values in malloc'd
not palloc'd storage, it has to be more careful than usual to
avoid leaks. Error exits out of string GUC hook checks failed
to clear the proposed value string, and error exits out of
ProcessGUCArray() failed to clear the malloc'd results of
ParseLongOption().
Found via valgrind testing.
This problem is ancient, so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/misc/guc.c
Don't leak compiled regex(es) when an ispell cache entry is dropped.
commit : d303849b059c3c315e5a8d4239016f8328f3296c
author : Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 21:44:42 -0400
committer: Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 21:44:42 -0400
The text search cache mechanisms assume that we can clean up
an invalidated dictionary cache entry simply by resetting the
associated long-lived memory context. However, that does not work
for ispell affixes that make use of regular expressions, because
the regex library deals in plain old malloc. Hence, we leaked
compiled regex(es) any time we dropped such a cache entry. That
could quickly add up, since even a fairly trivial regex can use up
tens of kB, and a large one can eat megabytes. Add a memory context
callback to ensure that a regex gets freed when its owning cache
entry is cleared.
Found via valgrind testing.
This problem is ancient, so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/tsearch/spell.c
M src/include/tsearch/dicts/spell.h
Don't run RelationInitTableAccessMethod in a long-lived context.
commit : 415ffdc2205e209b6a73fb42a3fdd6e57e16c7b2
author : Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 20:50:56 -0400
committer: Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 20:50:56 -0400
Some code paths in this function perform syscache lookups, which
can lead to table accesses and possibly leakage of cruft into
the caller's context. If said context is CacheMemoryContext,
we eventually will have visible bloat. But fixing this is no
harder than moving one memory context switch step. (The other
callers don't have a problem.)
Andres Freund and I independently found this via valgrind testing.
Back-patch to v12 where this code was added.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/cache/relcache.c
Don't leak rd_statlist when a relcache entry is dropped.
commit : 28644fac10731e30e70b622986a6fbbeb5a5b2f9
author : Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 20:37:09 -0400
committer: Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 20:37:09 -0400
Although these lists are usually NIL, and even when not empty
are unlikely to be large, constant relcache update traffic could
eventually result in visible bloat of CacheMemoryContext.
Found via valgrind testing.
Back-patch to v10 where this field was added.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/cache/relcache.c
Fix TAP test for remove_temp_files_after_crash
commit : a16b2b960f0eec2fe367e86017b3c24ed688ba2b
author : Tomas Vondra <[email protected]>
date : Fri, 19 Mar 2021 02:05:23 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 19 Mar 2021 02:05:23 +0100
The test included in cd91de0d17 had two simple flaws.
Firstly, the number of rows was low and on some platforms (e.g. 32-bit)
the sort did not require on-disk sort, so on those machines it was not
testing the automatic removal. The test was however failing, because
without any temporary files the base/pgsql_tmp directory was not even
created. Fixed by increasing the rowcount to 5000, which should be high
engough on any platform.
Secondly, the test used a simple sleep to wait for the temporary file to
be created. This is obviously problematic, because on slow machines (or
with valgrind, CLOBBER_CACHE_ALWAYS etc.) it may take a while to create
the temporary file. But we also want the tests run reasonably fast.
Fixed by instead relying on a UNIQUE constraint, blocking the query that
created the temporary file.
Author: Euler Taveira
Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/CAH503wDKdYzyq7U-QJqGn%3DGm6XmoK%2B6_6xTJ-Yn5WSvoHLY1Ww%40mail.gmail.com
M src/test/recovery/t/022_crash_temp_files.pl
Improve tab completion of IMPORT FOREIGN SCHEMA with \h in psql
commit : 5b2266e33fc74142d23685bdf54f64ad598fbdea
author : Michael Paquier <[email protected]>
date : Fri, 19 Mar 2021 09:18:41 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 19 Mar 2021 09:18:41 +0900
Only "IMPORT" was showing as result of the completion, while IMPORT
FOREIGN SCHEMA is the only command using this keyword in first
position. This changes the completion to show the full command name
instead of just "IMPORT".
Reviewed-by: Georgios Kokolatos, Julien Rouhaud
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/tab-complete.c
Fix misuse of foreach_delete_current().
commit : 1d581ce7129d7a33cd4ad27f8f246abfa1fd2db9
author : Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 19:24:22 -0400
committer: Tom Lane <[email protected]>
date : Thu, 18 Mar 2021 19:24:22 -0400
Our coding convention requires this macro's result to be assigned
back to the original List variable. In this usage, since the
List could not become empty, there was no actual bug --- but
some compilers warned about it. Oversight in be45be9c3.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/parse_agg.c
Implement GROUP BY DISTINCT
commit : be45be9c33a85e72cdaeb9967e9f6d2d00199e09
author : Tomas Vondra <[email protected]>
date : Thu, 18 Mar 2021 17:45:38 +0100
committer: Tomas Vondra <[email protected]>
date : Thu, 18 Mar 2021 17:45:38 +0100
With grouping sets, it's possible that some of the grouping sets are
duplicate. This is especially common with CUBE and ROLLUP clauses. For
example GROUP BY CUBE (a,b), CUBE (b,c) is equivalent to
GROUP BY GROUPING SETS (
(a, b, c),
(a, b, c),
(a, b, c),
(a, b),
(a, b),
(a, b),
(a),
(a),
(a),
(c, a),
(c, a),
(c, a),
(c),
(b, c),
(b),
()
)
Some of the grouping sets are calculated multiple times, which is mostly
unnecessary. This commit implements a new GROUP BY DISTINCT feature, as
defined in the SQL standard, which eliminates the duplicate sets.
Author: Vik Fearing
Reviewed-by: Erik Rijkers, Georgios Kokolatos, Tomas Vondra
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/queries.sgml
M doc/src/sgml/ref/select.sgml
M src/backend/catalog/sql_features.txt
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/list.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/plan/planner.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/parser/parse_agg.c
M src/backend/utils/adt/ruleutils.c
M src/include/nodes/parsenodes.h
M src/include/nodes/pg_list.h
M src/include/parser/parse_agg.h
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Remove temporary files after backend crash
commit : cd91de0d17952b5763466cfa663e98318f26d357
author : Tomas Vondra <[email protected]>
date : Thu, 18 Mar 2021 16:05:03 +0100
committer: Tomas Vondra <[email protected]>
date : Thu, 18 Mar 2021 16:05:03 +0100
After a crash of a backend using temporary files, the files used to be
left behind, on the basis that it might be useful for debugging. But we
don't have any reports of anyone actually doing that, and it means the
disk usage may grow over time due to repeated backend failures (possibly
even hitting ENOSPC). So this behavior is a bit unfortunate, and fixing
it required either manual cleanup (deleting files, which is error-prone)
or restart of the instance (i.e. service disruption).
This implements automatic cleanup of temporary files, controled by a new
GUC remove_temp_files_after_crash. By default the files are removed, but
it can be disabled to restore the old behavior if needed.
Author: Euler Taveira
Reviewed-by: Tomas Vondra, Michael Paquier, Anastasia Lubennikova, Thomas Munro
Discussion: https://postgr.es/m/CAH503wDKdYzyq7U-QJqGn%3DGm6XmoK%2B6_6xTJ-Yn5WSvoHLY1Ww%40mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/postmaster/postmaster.c
M src/backend/storage/file/fd.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/postmaster/postmaster.h
A src/test/recovery/t/022_crash_temp_files.pl
Fix function name in error hint
commit : da18d829c28197efb04805a43f129f62650e50c8
author : Magnus Hagander <[email protected]>
date : Thu, 18 Mar 2021 11:17:42 +0100
committer: Magnus Hagander <[email protected]>
date : Thu, 18 Mar 2021 11:17:42 +0100
pg_read_file() is the function that's in core, pg_file_read() is in
adminpack. But when using pg_file_read() in adminpack it calls the *C*
level function pg_read_file() in core, which probably threw the original
author off. But the error hint should be about the SQL function.
Reported-By: Sergei Kornilov
Backpatch-through: 11
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/genfile.c
Doc: Update description for parallel insert reloption.
commit : ed62d3737c1b823f796d974060b1d0295a3dd831
author : Amit Kapila <[email protected]>
date : Thu, 18 Mar 2021 15:34:55 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 18 Mar 2021 15:34:55 +0530
Commit c8f78b6161 added a new reloption to enable inserts in parallel-mode
but forgot to update at one of the places about the same in docs. In
passing, fix a typo in the same commit.
Reported-by: Justin Pryzby
Author: Justin Pryzby
Reviewed-by: "Hou, Zhijie", Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_table.sgml
Add a new GUC and a reloption to enable inserts in parallel-mode.
commit : c8f78b616167bf8e24bc5dc69112c37755ed3058
author : Amit Kapila <[email protected]>
date : Thu, 18 Mar 2021 07:25:27 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 18 Mar 2021 07:25:27 +0530
Commit 05c8482f7f added the implementation of parallel SELECT for
"INSERT INTO ... SELECT ..." which may incur non-negligible overhead in
the additional parallel-safety checks that it performs, even when, in the
end, those checks determine that parallelism can't be used. This is
normally only ever a problem in the case of when the target table has a
large number of partitions.
A new GUC option "enable_parallel_insert" is added, to allow insert in
parallel-mode. The default is on.
In addition to the GUC option, the user may want a mechanism to allow
inserts in parallel-mode with finer granularity at table level. The new
table option "parallel_insert_enabled" allows this. The default is true.
Author: "Hou, Zhijie"
Reviewed-by: Greg Nancarrow, Amit Langote, Takayuki Tsunakawa, Amit Kapila
Discussion: https://postgr.es/m/CAA4eK1K-cW7svLC2D7DHoGHxdAdg3P37BLgebqBOC2ZLc9a6QQ%40mail.gmail.com
Discussion: https://postgr.es/m/CAJcOf-cXnB5cnMKqWEp2E2z7Mvcd04iLVmV=qpFJrR3AcrTS3g@mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_table.sgml
M src/backend/access/common/reloptions.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/util/clauses.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/psql/tab-complete.c
M src/include/optimizer/cost.h
M src/include/utils/rel.h
M src/test/regress/expected/insert_parallel.out
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/insert_parallel.sql
M src/tools/pgindent/typedefs.list
Fix memory lifetime issues of replication slot stats.
commit : 5f79580ad69f6e696365bdc63bc265f45bd77211
author : Andres Freund <[email protected]>
date : Wed, 17 Mar 2021 16:18:37 -0700
committer: Andres Freund <[email protected]>
date : Wed, 17 Mar 2021 16:18:37 -0700
When accessing replication slot stats, introduced in 98681675002d,
pgstat_read_statsfiles() reads the data into newly allocated
memory. Unfortunately the current memory context at that point is the
callers, leading to leaks and use-after-free dangers.
The fix is trivial, explicitly use pgStatLocalContext. There's some
potential for further improvements, but that's outside of the scope of
this bugfix.
No backpatch necessary, feature is only in HEAD.
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
M src/backend/postmaster/pgstat.c
Doc: remove duplicated step in RLS example.
commit : 70945649d734d16be22c3d1d90dd8c3d3c1e9d89
author : Tom Lane <[email protected]>
date : Wed, 17 Mar 2021 16:39:58 -0400
committer: Tom Lane <[email protected]>
date : Wed, 17 Mar 2021 16:39:58 -0400
Seems to have been a copy-and-paste mistake in 093129c9d.
Per report from [email protected].
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ddl.sgml
Code review for server's handling of "tablespace map" files.
commit : 8620a7f6dbdf978e57cdb9f42802a0418656d863
author : Tom Lane <[email protected]>
date : Wed, 17 Mar 2021 16:18:46 -0400
committer: Tom Lane <[email protected]>
date : Wed, 17 Mar 2021 16:18:46 -0400
While looking at Robert Foggia's report, I noticed a passel of
other issues in the same area:
* The scheme for backslash-quoting newlines in pathnames is just
wrong; it will misbehave if the last ordinary character in a pathname
is a backslash. I'm not sure why we're bothering to allow newlines
in tablespace paths, but if we're going to do it we should do it
without introducing other problems. Hence, backslashes themselves
have to be backslashed too.
* The author hadn't read the sscanf man page very carefully, because
this code would drop any leading whitespace from the path. (I doubt
that a tablespace path with leading whitespace could happen in
practice; but if we're bothering to allow newlines in the path, it
sure seems like leading whitespace is little less implausible.) Using
sscanf for the task of finding the first space is overkill anyway.
* While I'm not 100% sure what the rationale for escaping both \r and
\n is, if the idea is to allow Windows newlines in the file then this
code failed, because it'd throw an error if it saw \r followed by \n.
* There's no cross-check for an incomplete final line in the map file,
which would be a likely apparent symptom of the improper-escaping
bug.
On the generation end, aside from the escaping issue we have:
* If needtblspcmapfile is true then do_pg_start_backup will pass back
escaped strings in tablespaceinfo->path values, which no caller wants
or is prepared to deal with. I'm not sure if there's a live bug from
that, but it looks like there might be (given the dubious assumption
that anyone actually has newlines in their tablespace paths).
* It's not being very paranoid about the possibility of random stuff
in the pg_tblspc directory. IMO we should ignore anything without an
OID-like name.
The escaping rule change doesn't seem back-patchable: it'll require
doubling of backslashes in the tablespace_map file, which is basically
a basebackup format change. The odds of that causing trouble are
considerably more than the odds of the existing bug causing trouble.
The rest of this seems somewhat unlikely to cause problems too,
so no back-patch.
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogfuncs.c
M src/backend/replication/basebackup.c
M src/include/access/xlog.h
M src/include/replication/basebackup.h
Prevent buffer overrun in read_tablespace_map().
commit : a50e4fd028a1ece2b1a04df2c9ae6581783e9eef
author : Tom Lane <[email protected]>
date : Wed, 17 Mar 2021 16:10:37 -0400
committer: Tom Lane <[email protected]>
date : Wed, 17 Mar 2021 16:10:37 -0400
Robert Foggia of Trustwave reported that read_tablespace_map()
fails to prevent an overrun of its on-stack input buffer.
Since the tablespace map file is presumed trustworthy, this does
not seem like an interesting security vulnerability, but still
we should fix it just in the name of robustness.
While here, document that pg_basebackup's --tablespace-mapping option
doesn't work with tar-format output, because it doesn't. To make it
work, we'd have to modify the tablespace_map file within the tarball
sent by the server, which might be possible but I'm not volunteering.
(Less-painful solutions would require changing the basebackup protocol
so that the source server could adjust the map. That's not very
appetizing either.)
M doc/src/sgml/ref/pg_basebackup.sgml
M src/backend/access/transam/xlog.c
Add end-to-end testing of pg_basebackup's tar-format output.
commit : 081876d75ea15c3bd2ee5ba64a794fd8ea46d794
author : Tom Lane <[email protected]>
date : Wed, 17 Mar 2021 14:52:55 -0400
committer: Tom Lane <[email protected]>
date : Wed, 17 Mar 2021 14:52:55 -0400
The existing test script does run pg_basebackup with the -Ft option,
but it makes no real attempt to verify the sanity of the results.
We wouldn't know if the output is incompatible with standard "tar"
programs, nor if the server fails to start from the restored output.
Notably, this means that xlog.c's read_tablespace_map() is not being
meaningfully tested, since that code is used only in the tar-format
case. (We do have reasonable coverage of restoring from plain-format
output, though it's over in src/test/recovery not here.)
Hence, attempt to untar the output and start a server from it,
rather just hoping it's OK.
This test assumes that the local "tar" has the "-C directory"
switch. Although that's not promised by POSIX, my research
suggests that all non-extinct tar implementations have it.
Should the buildfarm's opinion differ, we can complicate the
test a bit to avoid requiring that.
Possibly this should be back-patched, but I'm unsure about
whether it could work on Windows before d66b23b03.
M src/bin/pg_basebackup/Makefile
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/test/perl/PostgresNode.pm
Doc: improve discussion of variable substitution in PL/pgSQL.
commit : c783e656d41816b0328cb4bff27f11b70200770e
author : Tom Lane <[email protected]>
date : Wed, 17 Mar 2021 13:09:13 -0400
committer: Tom Lane <[email protected]>
date : Wed, 17 Mar 2021 13:09:13 -0400
This was a bit disjointed, partly because of a not-well-considered
decision to document SQL commands that don't return result rows as
though they had nothing in common with commands that do. Rearrange
so that we have one discussion of variable substitution that clearly
applies to all types of SQL commands, and then handle the question
of processing command output separately. Clarify that EXPLAIN,
CREATE TABLE AS SELECT, and similar commands that incorporate an
optimizable statement will act like optimizable statements for the
purposes of variable substitution. Do a bunch of minor wordsmithing
in the same area.
David Johnston and Tom Lane, reviewed by Pavel Stehule and David
Steele
Discussion: https://postgr.es/m/CAKFQuwYvMKucM5fnZvHSo-ah4S=_n9gmKeu6EAo=_fTrohunqQ@mail.gmail.com
M doc/src/sgml/plpgsql.sgml
Revert "Fix race in Parallel Hash Join batch cleanup."
commit : 7f7f25f15edb6eacec58179ef5285e874aa4435b
author : Thomas Munro <[email protected]>
date : Thu, 18 Mar 2021 00:35:04 +1300
committer: Thomas Munro <[email protected]>
date : Thu, 18 Mar 2021 00:35:04 +1300
This reverts commit 378802e3713c6c0fce31d2390c134cd5d7c30157.
This reverts commit 3b8981b6e1a2aea0f18384c803e21e9391de669a.
Discussion: https://postgr.es/m/CA%2BhUKGJmcqAE3MZeDCLLXa62cWM0AJbKmp2JrJYaJ86bz36LFA%40mail.gmail.com
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeHashjoin.c
M src/backend/postmaster/pgstat.c
M src/include/executor/hashjoin.h
M src/include/pgstat.h
Fix comment in indexing.c
commit : 9fd2952cf4920d563e9cea51634c5b364d57f71a
author : Michael Paquier <[email protected]>
date : Wed, 17 Mar 2021 18:07:00 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 17 Mar 2021 18:07:00 +0900
578b229, that removed support for WITH OIDS, has changed
CatalogTupleInsert() to not return an Oid, but one comment was still
mentioning that.
Author: Vik Fearing
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/indexing.c
Small error message improvement
commit : e1ae40f381d0582981b1e63856bd4b060cfe2d53
author : Peter Eisentraut <[email protected]>
date : Wed, 17 Mar 2021 08:17:33 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 17 Mar 2021 08:17:33 +0100
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_data.out
Update the names of Parallel Hash Join phases.
commit : 378802e3713c6c0fce31d2390c134cd5d7c30157
author : Thomas Munro <[email protected]>
date : Wed, 17 Mar 2021 18:24:45 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 17 Mar 2021 18:24:45 +1300
Commit 3048898e dropped -ING from some wait event names that correspond
to barrier phases. Update the phases' names to match.
While we're here making cosmetic changes, also rename "DONE" to "FREE".
That pairs better with "ALLOCATE", and describes the activity that
actually happens in that phase (as we do for the other phases) rather
than describing a state. The distinction is clearer after bugfix commit
3b8981b6 split the phase into two. As for the growth barriers, rename
their "ALLOCATE" phase to "REALLOCATE", which is probably a better
description of what happens then. Also improve the comments about
the phases a bit.
Discussion: https://postgr.es/m/CA%2BhUKG%2BMDpwF2Eo2LAvzd%3DpOh81wUTsrwU1uAwR-v6OGBB6%2B7g%40mail.gmail.com
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeHashjoin.c
M src/backend/postmaster/pgstat.c
M src/include/executor/hashjoin.h
M src/include/pgstat.h
Fix race in Parallel Hash Join batch cleanup.
commit : 3b8981b6e1a2aea0f18384c803e21e9391de669a
author : Thomas Munro <[email protected]>
date : Wed, 17 Mar 2021 17:46:39 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 17 Mar 2021 17:46:39 +1300
With very unlucky timing and parallel_leader_participation off, PHJ
could attempt to access per-batch state just as it was being freed.
There was code intended to prevent that by checking for a cleared
pointer, but it was buggy.
Fix, by introducing an extra barrier phase. The new phase
PHJ_BUILD_RUNNING means that it's safe to access the per-batch state to
find a batch to help with, and PHJ_BUILD_DONE means that it is too late.
The last to detach will free the array of per-batch state as before, but
now it will also atomically advance the phase at the same time, so that
late attachers can avoid the hazard, without the data race. This
mirrors the way per-batch hash tables are freed (see phases
PHJ_BATCH_PROBING and PHJ_BATCH_DONE).
Revealed by a one-off build farm failure, where BarrierAttach() failed a
sanity check assertion, because the memory had been clobbered by
dsa_free().
Back-patch to 11, where the code arrived.
Reported-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/20200929061142.GA29096%40paquier.xyz
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeHashjoin.c
M src/include/executor/hashjoin.h
Fix transaction.sql tests in higher isolation levels.
commit : 37929599499fe5760a9dbab48a10a898879a0d44
author : Thomas Munro <[email protected]>
date : Wed, 17 Mar 2021 17:13:43 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 17 Mar 2021 17:13:43 +1300
It seems like a useful sanity check to be able to run "installcheck"
against a cluster running with default_transaction_level set to
serializable or repeatable read. Only one thing currently fails in
those configurations, so let's fix that.
No back-patch for now, because it fails in many other places in some of
the stable branches. We'd have to go back and fix those too if we
included this configuration in automated testing.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJUaHeK%3DHLATxF1JOKDjKJVrBKA-zmbPAebOM0Se2FQRg%40mail.gmail.com
M src/test/regress/expected/transactions.out
M src/test/regress/sql/transactions.sql
Fix race condition in drop subscription's handling of tablesync slots.
commit : 6b67d72b604cb913e39324b81b61ab194d94cba0
author : Amit Kapila <[email protected]>
date : Wed, 17 Mar 2021 08:15:12 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 17 Mar 2021 08:15:12 +0530
Commit ce0fdbfe97 made tablesync slots permanent and allow Drop
Subscription to drop such slots. However, it is possible that before
tablesync worker could get the acknowledgment of slot creation, drop
subscription stops it and that can lead to a dangling slot on the
publisher. Prevent cancel/die interrupts while creating a slot in the
tablesync worker.
Reported-by: Thomas Munro as per buildfarm
Author: Amit Kapila
Reviewed-by: Vignesh C, Takamichi Osumi
Discussion: https://postgr.es/m/CA+hUKGJG9dWpw1cOQ2nzWU8PHjm=PTraB+KgE5648K9nTfwvxg@mail.gmail.com
M src/backend/replication/logical/tablesync.c
Doc: Add a description of substream in pg_subscription.
commit : 7efeb214ad832fa96ea950d0906b1d2b96316d15
author : Amit Kapila <[email protected]>
date : Wed, 17 Mar 2021 07:40:23 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 17 Mar 2021 07:40:23 +0530
Commit 464824323e added a new column substream in pg_subscription but
forgot to update the docs.
Reported-by: Peter Smith
Author: Amit Kapila
Reviewed-by: Peter Smith
Discussion: https://postgr.es/m/CAHut+PuPGGASnh2Dy37VYODKULVQo-5oE=Shc6gwtRizDt==cA@mail.gmail.com
M doc/src/sgml/catalogs.sgml
Enable parallelism in REFRESH MATERIALIZED VIEW.
commit : 9e7ccd9ef64d05e87ceb1985d459bef9031205c0
author : Thomas Munro <[email protected]>
date : Wed, 17 Mar 2021 13:43:08 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 17 Mar 2021 13:43:08 +1300
Pass CURSOR_OPT_PARALLEL_OK to pg_plan_query() so that parallel plans
are considered when running the underlying SELECT query. This wasn't
done in commit e9baa5e9, which did this for CREATE MATERIALIZED VIEW,
because it wasn't yet known to be safe.
Since REFRESH always inserts into a freshly created table before later
merging or swapping the data into place with separate operations, we can
enable such plans here too.
Author: Bharath Rupireddy <[email protected]>
Reviewed-by: Hou, Zhijie <[email protected]>
Reviewed-by: Luc Vlaming <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Discussion: https://postgr.es/m/CALj2ACXg-4hNKJC6nFnepRHYT4t5jJVstYvri%2BtKQHy7ydcr8A%40mail.gmail.com
M doc/src/sgml/parallel.sgml
M src/backend/commands/matview.c
M src/test/regress/expected/write_parallel.out
M src/test/regress/sql/write_parallel.sql
Fix comment about promising tuples.
commit : fbe4cb3bd49f9e524f53ef77c775c1bad4d0312a
author : Peter Geoghegan <[email protected]>
date : Tue, 16 Mar 2021 13:38:52 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 16 Mar 2021 13:38:52 -0700
Oversight in commit d168b666823, which added bottom-up index deletion.
M src/backend/access/heap/heapam.c
amcheck: Reduce debug message verbosity.
commit : 65445469d6de3670f3e027236613e093e119ec55
author : Peter Geoghegan <[email protected]>
date : Tue, 16 Mar 2021 13:11:17 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 16 Mar 2021 13:11:17 -0700
Empty sibling pages can occasionally be much more common than any other
event that we report on at elevel DEBUG1. Increase the elevel for
relevant cases to DEBUG2 to avoid overwhelming the user with relatively
insignificant details.
M contrib/amcheck/verify_nbtree.c
Avoid corner-case memory leak in SSL parameter processing.
commit : 4b12ab18c9d0735d760bf7667b158707b06e5df8
author : Tom Lane <[email protected]>
date : Tue, 16 Mar 2021 16:02:49 -0400
committer: Tom Lane <[email protected]>
date : Tue, 16 Mar 2021 16:02:49 -0400
After reading the root cert list from the ssl_ca_file, immediately
install it as client CA list of the new SSL context. That gives the
SSL context ownership of the list, so that SSL_CTX_free will free it.
This avoids a permanent memory leak if we fail further down in
be_tls_init(), which could happen if bogus CRL data is offered.
The leak could only amount to something if the CRL parameters get
broken after server start (else we'd just quit) and then the server
is SIGHUP'd many times without fixing the CRL data. That's rather
unlikely perhaps, but it seems worth fixing, if only because the
code is clearer this way.
While we're here, add some comments about the memory management
aspects of this logic.
Noted by Jelte Fennema and independently by Andres Freund.
Back-patch to v10; before commit de41869b6 it doesn't matter,
since we'd not re-execute this code during SIGHUP.
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/be-secure-openssl.c
Fix a confusing amcheck corruption message.
commit : 4078ce65a0f7197180a9be2c6460ea4bf909bd98
author : Robert Haas <[email protected]>
date : Tue, 16 Mar 2021 15:42:20 -0400
committer: Robert Haas <[email protected]>
date : Tue, 16 Mar 2021 15:42:20 -0400
Don't complain about the last TOAST chunk number being different
from what we expected if there are no TOAST chunks at all.
In such a case, saying that the final chunk number is 0 is not
really accurate, and the fact the value is missing from the
TOAST table is reported separately anyway.
Mark Dilger
Discussion: http://postgr.es/m/[email protected]
M contrib/amcheck/verify_heapam.c
M src/bin/pg_amcheck/t/004_verify_heapam.pl
Use pre-fetching for ANALYZE
commit : c6fc50cb40285141fad401321ae21becbaea1c59
author : Stephen Frost <[email protected]>
date : Tue, 16 Mar 2021 14:46:48 -0400
committer: Stephen Frost <[email protected]>
date : Tue, 16 Mar 2021 14:46:48 -0400
When we have posix_fadvise() available, we can improve the performance
of an ANALYZE by quite a bit by using it to inform the kernel of the
blocks that we're going to be asking for. Similar to bitmap index
scans, the number of buffers pre-fetched is based off of the
maintenance_io_concurrency setting (for the particular tablespace or,
if not set, globally, via get_tablespace_maintenance_io_concurrency()).
Reviewed-By: Heikki Linnakangas, Tomas Vondra
Discussion: https://www.postgresql.org/message-id/VI1PR0701MB69603A433348EDCF783C6ECBF6EF0%40VI1PR0701MB6960.eurprd07.prod.outlook.com
M src/backend/commands/analyze.c
Improve logging of auto-vacuum and auto-analyze
commit : 94d13d474dc61800e8a17cc1959c55815b050ecd
author : Stephen Frost <[email protected]>
date : Tue, 16 Mar 2021 14:46:48 -0400
committer: Stephen Frost <[email protected]>
date : Tue, 16 Mar 2021 14:46:48 -0400
When logging auto-vacuum and auto-analyze activity, include the I/O
timing if track_io_timing is enabled. Also, for auto-analyze, add the
read rate and the dirty rate, similar to how that information has
historically been logged for auto-vacuum.
Stephen Frost and Jakub Wartak
Reviewed-By: Heikki Linnakangas, Tomas Vondra
Discussion: https://www.postgresql.org/message-id/VI1PR0701MB69603A433348EDCF783C6ECBF6EF0%40VI1PR0701MB6960.eurprd07.prod.outlook.com
M doc/src/sgml/config.sgml
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/analyze.c
Improve logging of bad parameter values in BIND messages.
commit : 1ea396362be1615e926ea69d666c770081a0d3ef
author : Tom Lane <[email protected]>
date : Tue, 16 Mar 2021 11:16:41 -0400
committer: Tom Lane <[email protected]>
date : Tue, 16 Mar 2021 11:16:41 -0400
Since commit ba79cb5dc, values of bind parameters have been logged
during errors in extended query mode. However, we only did that after
we'd collected and converted all the parameter values, thus failing to
offer any useful localization of invalid-parameter problems. Add a
separate callback that's used during parameter collection, and have it
print the parameter number, along with the input string if text input
format is used.
Justin Pryzby and Tom Lane
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CANfkH5k-6nNt-4cSv1vPB80nq2BZCzhFVR5O4VznYbsX0wZmow@mail.gmail.com
M src/backend/tcop/postgres.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
(Blind) fix Perl splitting of strings at newlines
commit : 015061690c6526ff9f9f7af2940e1c1541654b68
author : Alvaro Herrera <[email protected]>
date : Tue, 16 Mar 2021 10:36:28 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 16 Mar 2021 10:36:28 -0300
I forgot that Windows represents newlines as \r\n, so splitting a string
at /\s/ creates additional empty strings. Let's rewrite that as /\s+/
to see if that avoids those. (There's precedent for using that pattern
on Windows in other scripts.)
Previously: 91bdf499b37b, 8ed428dc977f, 650b96707672.
Per buildfarm, via Tom Lane.
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
Add some basic tests for progress reporting of COPY
commit : 15639d5e8f6f278219681fec8a5668a92fb7e874
author : Michael Paquier <[email protected]>
date : Tue, 16 Mar 2021 09:55:43 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 16 Mar 2021 09:55:43 +0900
This tests some basic features for progress reporting of COPY, relying
on an INSERT trigger that gets fired when doing COPY FROM with a file or
stdin, checking for sizes, number of tuples processed, and number of
tuples excluded by a WHERE clause.
Author: Josef Šimánek, Matthias van de Meent
Reviewed-by: Michael Paquier, Justin Pryzby, Bharath Rupireddy, Tomas
Vondra
Discussion: https://postgr.es/m/CAEze2WiOcgdH4aQA8NtZq-4dgvnJzp8PohdeKchPkhMY-jWZXA@mail.gmail.com
M src/test/regress/input/copy.source
M src/test/regress/output/copy.source
Add libpq pipeline mode support to pgbench
commit : 9aa491abbf07ca8385a429385be8d68517384fdf
author : Alvaro Herrera <[email protected]>
date : Mon, 15 Mar 2021 18:33:03 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 15 Mar 2021 18:33:03 -0300
New metacommands \startpipeline and \endpipeline allow the user to run
queries in libpq pipeline mode.
Author: Daniel Vérité <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/pgbench.sgml
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
Implement pipeline mode in libpq
commit : acb7e4eb6b1c614c68a62fb3a6a5bba1af0a2659
author : Alvaro Herrera <[email protected]>
date : Mon, 15 Mar 2021 18:13:42 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 15 Mar 2021 18:13:42 -0300
Pipeline mode in libpq lets an application avoid the Sync messages in
the FE/BE protocol that are implicit in the old libpq API after each
query. The application can then insert Sync at its leisure with a new
libpq function PQpipelineSync. This can lead to substantial reductions
in query latency.
Co-authored-by: Craig Ringer <[email protected]>
Co-authored-by: Matthieu Garrigues <[email protected]>
Co-authored-by: Álvaro Herrera <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Aya Iwata <[email protected]>
Reviewed-by: Daniel Vérité <[email protected]>
Reviewed-by: David G. Johnston <[email protected]>
Reviewed-by: Justin Pryzby <[email protected]>
Reviewed-by: Kirk Jamison <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Reviewed-by: Nikhil Sontakke <[email protected]>
Reviewed-by: Vaishnavi Prabakaran <[email protected]>
Reviewed-by: Zhihong Yu <[email protected]>
Discussion: https://postgr.es/m/CAMsr+YFUjJytRyV4J-16bEoiZyH=4nj+sQ7JP9ajwz=B4dMMZw@mail.gmail.com
Discussion: https://postgr.es/m/CAJkzx4T5E-2cQe3dtv2R78dYFvz+in8PY7A8MArvLhs_pg75gg@mail.gmail.com
M doc/src/sgml/libpq.sgml
M doc/src/sgml/lobj.sgml
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/bin/pg_amcheck/pg_amcheck.c
M src/interfaces/libpq/exports.txt
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
M src/test/modules/Makefile
A src/test/modules/libpq_pipeline/.gitignore
A src/test/modules/libpq_pipeline/Makefile
A src/test/modules/libpq_pipeline/README
A src/test/modules/libpq_pipeline/libpq_pipeline.c
A src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
M src/tools/msvc/Mkvcbuild.pm
M src/tools/pgindent/typedefs.list
Work around issues in MinGW-64's setjmp/longjmp support.
commit : 146cb3889c3ccb3fce198fe7464a1296a9e107c3
author : Tom Lane <[email protected]>
date : Mon, 15 Mar 2021 12:34:17 -0400
committer: Tom Lane <[email protected]>
date : Mon, 15 Mar 2021 12:34:17 -0400
It's hard to avoid the conclusion that there is something wrong with
setjmp/longjmp on MinGW-64, as we have seen failures come and go after
entirely-unrelated-looking changes in our own code. Other projects
such as Ruby have given up and started using gcc's setjmp/longjmp
builtins on that platform; this patch just follows that lead.
Note that this is a pretty fundamental ABI break for functions
containining either setjmp or longjmp, so we can't really consider
a back-patch.
Per reports from Regina Obe and Heath Lord, as well as recent failures
on buildfarm member walleye, and less-recent failures on fairywren.
Juan José Santamaría Flecha
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CA+BEBhvHhM-Bn628pf-LsjqRh3Ang7qCSBG0Ga+7KwhGqrNUPw@mail.gmail.com
Discussion: https://postgr.es/m/[email protected]
M src/include/c.h
Drop SERIALIZABLE workaround from parallel query tests.
commit : eeb60e45d82d5840b713a8741ae552238d57e8b9
author : Thomas Munro <[email protected]>
date : Mon, 15 Mar 2021 23:27:08 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 15 Mar 2021 23:27:08 +1300
SERIALIZABLE no longer inhibits parallelism, so we can drop some
outdated workarounds and comments from regression tests. The change
came in release 12, commit bb16aba5, but it's not really worth
back-patching.
Also fix a typo.
Reviewed-by: Bharath Rupireddy <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJUaHeK%3DHLATxF1JOKDjKJVrBKA-zmbPAebOM0Se2FQRg%40mail.gmail.com
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/explain.out
M src/test/regress/expected/insert_parallel.out
M src/test/regress/expected/select_parallel.out
M src/test/regress/expected/write_parallel.out
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/explain.sql
M src/test/regress/sql/insert_parallel.sql
M src/test/regress/sql/select_parallel.sql
M src/test/regress/sql/write_parallel.sql
Make archiver process an auxiliary process.
commit : d75288fb27b8fe0a926aaab7d75816f091ecdc27
author : Fujii Masao <[email protected]>
date : Mon, 15 Mar 2021 13:13:14 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 15 Mar 2021 13:13:14 +0900
This commit changes WAL archiver process so that it's treated as
an auxiliary process and can use shared memory. This is an infrastructure
patch required for upcoming shared-memory based stats collector patch
series. These patch series basically need any processes including archiver
that can report the statistics to access to shared memory. Since this patch
itself is useful to simplify the code and when users monitor the status of
archiver, it's committed separately in advance.
This commit simplifies the code for WAL archiving. For example, previously
backends need to signal to archiver via postmaster when they notify
archiver that there are some WAL files to archive. On the other hand,
this commit removes that signal to postmaster and enables backends to
notify archier directly using shared latch.
Also, as the side of this change, the information about archiver process
becomes viewable at pg_stat_activity view.
Author: Kyotaro Horiguchi
Reviewed-by: Andres Freund, Álvaro Herrera, Julien Rouhaud, Tomas Vondra, Arthur Zakirov, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
M src/backend/access/transam/xlogarchive.c
M src/backend/bootstrap/bootstrap.c
M src/backend/postmaster/pgarch.c
M src/backend/postmaster/postmaster.c
M src/backend/storage/ipc/ipci.c
M src/include/miscadmin.h
M src/include/postmaster/pgarch.h
M src/include/storage/pmsignal.h
M src/include/storage/proc.h
M src/tools/pgindent/typedefs.list
Notice that heap page has dead items during VACUUM.
commit : 0ea71c93a06ddc38e0b72e48f1d512e5383a9c1b
author : Peter Geoghegan <[email protected]>
date : Sun, 14 Mar 2021 18:05:57 -0700
committer: Peter Geoghegan <[email protected]>
date : Sun, 14 Mar 2021 18:05:57 -0700
Consistently set a flag variable that tracks whether the current heap
page has a dead item during lazy vacuum's heap scan. We missed the
common case where there is an preexisting (or even a new) LP_DEAD heap
line pointer.
Also make it clear that the variable might be affected by an existing
line pointer, say from an earlier opportunistic pruning operation. This
distinction is important because it's the main reason why we can't just
use the nearby tups_vacuumed variable instead.
No backpatch. In theory failing to set the page level flag variable had
no consequences. Currently it is only used to defensively check if a
page marked all visible has dead items, which should never happen anyway
(if it does then the table must be corrupt).
Author: Masahiko Sawada <[email protected]>
Diagnosed-By: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/CAD21AoAtZb4+HJT_8RoOXvu4HM-Zd4HKS3YSMCH6+-W=bDyh-w@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Doc: add note about how to run the pg_amcheck regression tests.
commit : 58f57490facdec78119e5bab84229dbdc1d5ac6a
author : Tom Lane <[email protected]>
date : Sat, 13 Mar 2021 10:51:27 -0500
committer: Tom Lane <[email protected]>
date : Sat, 13 Mar 2021 10:51:27 -0500
It's not immediately obvious what you have to do to get "make
installcheck" to work here, so document that along the same lines
as we've used elsewhere.
A src/bin/pg_amcheck/README
In pg_amcheck tests, don't depend on perl's Q/q pack code.
commit : 945d2cb7d0255e296a55f3e9febb5dce6eaccc3e
author : Robert Haas <[email protected]>
date : Sat, 13 Mar 2021 10:55:33 -0500
committer: Robert Haas <[email protected]>
date : Sat, 13 Mar 2021 10:55:33 -0500
It does not work on all versions of perl across all platforms.
To avoid endian-ness issues, pick a new value for column a
that has the same upper 4 bytes as lower 4 bytes. Try to
make it something that isn't likely to occur anywhere nearby
in the page.
Discussion: http://postgr.es/m/[email protected]
M src/bin/pg_amcheck/t/004_verify_heapam.pl
pg_amcheck: Keep trying to fix the tests.
commit : 9e294d0f34d6e3e4fecf6f190b48862988934cde
author : Tom Lane <[email protected]>
date : Sat, 13 Mar 2021 00:06:56 -0500
committer: Tom Lane <[email protected]>
date : Sat, 13 Mar 2021 00:06:56 -0500
Fix another example of non-portable option ordering in the tests.
Oversight in 24189277f.
Mark Dilger
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_amcheck/t/003_check.pl
Fix new pthread code to respect --disable-thread-safety.
commit : de91c3b976cfacddacd45a9b52046264c0e44b11
author : Thomas Munro <[email protected]>
date : Sat, 13 Mar 2021 17:21:01 +1300
committer: Thomas Munro <[email protected]>
date : Sat, 13 Mar 2021 17:21:01 +1300
Don't try to compile src/port/pthread_barrier_wait.c if we opted out of
threads at configure time. Revealed by build farm member gaur, which
can't compile this code because of problems with its pthread
implementation. It shouldn't be trying to, because it's using
--disable-thread-safety.
Defect in commit 44bf3d50.
Reported-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/2568537.1615603606%40sss.pgh.pa.us
M configure
M configure.ac
Improve FK trigger parallel-safety check added by 05c8482f7f.
commit : c5be48f092016b1caf597b2e21d588b56c88a23e
author : Amit Kapila <[email protected]>
date : Sat, 13 Mar 2021 09:13:21 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 13 Mar 2021 09:13:21 +0530
Commit 05c8482f7f added special logic related to parallel-safety of FK
triggers. This is a bit of a hack and should have instead been done by
simply setting appropriate proparallel values on those trigger functions
themselves.
Suggested-by: Tom Lane
Author: Greg Nancarrow
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/util/clauses.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
pg_amcheck: Keep trying to fix the tests.
commit : b9164eab208342d685638fc90048ffaa2688cb47
author : Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 21:59:56 -0500
committer: Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 21:59:56 -0500
Commit 24189277f6ff3169b15c7bc82926a372ca7f2dbf managed to remove
one of the two places where we were checking for a "no such user"
error while leaving the other one right next to it. So remove that
too. In fact, remove the entire test, because the whole point of
this test was to see which message we got on a failure.
M src/bin/pg_amcheck/t/002_nonesuch.pl
pg_amcheck: Try to fix still more test failures.
commit : 24189277f6ff3169b15c7bc82926a372ca7f2dbf
author : Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 20:11:47 -0500
committer: Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 20:11:47 -0500
Avoid use of non-portable option ordering in command_checks_all().
The use of bare command line arguments before switches doesn't work
everywhere. Per buildfarm members drongo and hoverfly.
Avoid testing for the message "role \"%s\" does not exist", because
some buildfarm machines report a different error. fairywren complains
about "SSPI authentication failed for user \"%s\"", for example.
Mark Dilger
Discussion: http://postgr.es/m/[email protected]
Discussion: http://postgr.es/m/F0A1FD70-A2F4-4528-8A03-8650CAEC0554%40enterprisedb.com
M src/bin/pg_amcheck/t/002_nonesuch.pl
M src/bin/pg_amcheck/t/003_check.pl
Try to avoid apparent platform-dependency in IPC::Run
commit : f371a4cdba6dc805acd608cc63a7089b57cb4e9e
author : Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 19:00:41 -0500
committer: Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 19:00:41 -0500
It's hard to believe, but buildfarm results from the new pg_amcheck
suggest that command_checks_all() perform shell expansion on some
machines but not others, apparently due to an underlying behavior
difference in IPC::Run. Let's see if we can work around that - and
confirm that it is the real problem - by passing '-S*' as a single
argument rather than '-S' and '*' as two separate ones.
Failures were observed on jacana and hoverfly.
Mark Dilger
Discussion: http://postgr.es/m/[email protected]
M src/bin/pg_amcheck/t/002_nonesuch.pl
Fix portability issues in pg_amcheck's 004_verify_heapam.pl.
commit : 661125612706b1d0d5ed9f12d18908b08512a7eb
author : Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 17:30:17 -0500
committer: Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 17:30:17 -0500
Test #12 overwrote a 1-byte varlena header to make it look like the
initial byte of a 4-byte varlena header, but the results were
endian-dependent. Also, the byte "abc" that followed the overwritten
byte would be interpreted differently depending on endian-ness.
Overwrite 4 bytes instead, in an endian-aware manner.
Test #13 accidentally managed to depend on TOAST_MAX_CHUNK_SIZE,
which varies slightly depending on MAXIMUM_ALIGNOF. That's not
the point anyway, so make the regexp insensitive to the expected
number of chunks.
Mark Dilger
Discussion: http://postgr.es/m/[email protected]
M src/bin/pg_amcheck/t/004_verify_heapam.pl
Consolidate nbtree VACUUM metapage routines.
commit : 02b5940dbea17d07a1dbcba3cbe113cc8b70f228
author : Peter Geoghegan <[email protected]>
date : Fri, 12 Mar 2021 13:11:47 -0800
committer: Peter Geoghegan <[email protected]>
date : Fri, 12 Mar 2021 13:11:47 -0800
Simplify _bt_vacuum_needs_cleanup() functions's signature (it only needs
a single 'rel' argument now), and move it next to its sibling function
in nbtpage.c.
I believe that _bt_vacuum_needs_cleanup() was originally located in
nbtree.c due to an include dependency issue. That's no longer an issue.
Follow-up to commit 9f3665fb.
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/include/access/nbtree.h
Move PG_USED_FOR_ASSERTS_ONLY before initializer.
commit : ac445955852fe6bc0e02e87a409f25ab6e0a82d6
author : Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 15:04:10 -0500
committer: Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 15:04:10 -0500
Erik Rijkers reported a compile failure, and I think this is probably
the reason.
M src/bin/pg_amcheck/pg_amcheck.c
Adjust perl style.
commit : 7a1527c02c147a4107490662cb1872524c54a8ae
author : Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 14:55:40 -0500
committer: Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 14:55:40 -0500
Per buildfarm member crake.
M src/bin/pg_amcheck/t/003_check.pl
M src/bin/pg_amcheck/t/004_verify_heapam.pl
Try to fix compiler warnings.
commit : d60e61de4fb4a8e7ca88204c2c409e7380887d76
author : Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 14:35:10 -0500
committer: Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 14:35:10 -0500
Per report from Peter Geoghegan.
Discussion: http://postgr.es/m/CAH2-WznpwULZ3uJ1_6WXvNMXYbOy8k8tYs3r=qSdGmZeRd6tDw@mail.gmail.com
M src/bin/pg_amcheck/pg_amcheck.c
Add pg_amcheck, a CLI for contrib/amcheck.
commit : 9706092839db2c8c93860674e426a917635438c3
author : Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 13:00:01 -0500
committer: Robert Haas <[email protected]>
date : Fri, 12 Mar 2021 13:00:01 -0500
This makes it a lot easier to run the corruption checks that are
implemented by contrib/amcheck against lots of relations and get
the result in an easily understandable format. It has a wide variety
of options for choosing which relations to check and which checks
to perform, and it can run checks in parallel if you want.
Mark Dilger, reviewed by Peter Geoghegan and by me.
Discussion: http://postgr.es/m/[email protected]
Discussion: http://postgr.es/m/[email protected]
M doc/src/sgml/ref/allfiles.sgml
A doc/src/sgml/ref/pg_amcheck.sgml
M doc/src/sgml/reference.sgml
M src/bin/Makefile
A src/bin/pg_amcheck/.gitignore
A src/bin/pg_amcheck/Makefile
A src/bin/pg_amcheck/pg_amcheck.c
A src/bin/pg_amcheck/t/001_basic.pl
A src/bin/pg_amcheck/t/002_nonesuch.pl
A src/bin/pg_amcheck/t/003_check.pl
A src/bin/pg_amcheck/t/004_verify_heapam.pl
A src/bin/pg_amcheck/t/005_opclass_damage.pl
M src/tools/msvc/Install.pm
M src/tools/msvc/Mkvcbuild.pm
Fix race condition in psql \e's detection of file modification.
commit : 48d67fd897918c72e7cdf703d794056b88ed5725
author : Tom Lane <[email protected]>
date : Fri, 12 Mar 2021 12:20:15 -0500
committer: Tom Lane <[email protected]>
date : Fri, 12 Mar 2021 12:20:15 -0500
psql's editing commands decide whether the user has edited the file
by checking for change of modification timestamp. This is probably
fine for a pre-existing file, but with a temporary file that is
created within the command, it's possible for a fast typist to
save-and-exit in less than the one-second granularity of stat(2)
timestamps. On Windows FAT filesystems the granularity is even
worse, 2 seconds, making the race a bit easier to hit.
To fix, try to set the temp file's mod time to be two seconds ago.
It's unlikely this would fail, but then again the race condition
itself is unlikely, so just ignore any error.
Also, we might as well check the file size as well as its mod time.
While this is a difficult bug to hit, it still seems worth
back-patching, to ensure that users' edits aren't lost.
Laurenz Albe, per gripe from Jacob Champion; based on fix suggestions
from Jacob and myself
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/command.c
Forbid marking an identity column as nullable.
commit : f52c5d6749a61fc4e0908457c58f5069910d53a6
author : Tom Lane <[email protected]>
date : Fri, 12 Mar 2021 11:08:42 -0500
committer: Tom Lane <[email protected]>
date : Fri, 12 Mar 2021 11:08:42 -0500
GENERATED ALWAYS AS IDENTITY implies NOT NULL, but the code failed
to complain if you overrode that with "GENERATED ALWAYS AS IDENTITY
NULL". One might think the old behavior was a feature, but it was
inconsistent because the outcome varied depending on the order of
the clauses, so it seems to have been just an oversight.
Per bug #16913 from Pavel Boev. Back-patch to v10 where identity
columns were introduced.
Vik Fearing (minor tweaks by me)
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_table.sgml
M src/backend/parser/parse_utilcmd.c
M src/test/regress/expected/identity.out
M src/test/regress/sql/identity.sql
Specialize checkpointer sort functions.
commit : 1b88b8908e751271933c076234fa085cda251421
author : Thomas Munro <[email protected]>
date : Fri, 12 Mar 2021 23:56:02 +1300
committer: Thomas Munro <[email protected]>
date : Fri, 12 Mar 2021 23:56:02 +1300
When sorting a potentially large number of dirty buffers, the
checkpointer can benefit from a faster sort routine. One reported
improvement on a large buffer pool system was 1.4s -> 0.6s.
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJ2-eaDqAum5bxhpMNhvuJmRDZxB_Tow0n-gse%2BHG0Yig%40mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
commit : 519e4c9ee21a656879123f4843f1d8d60cb71536
author : Amit Kapila <[email protected]>
date : Fri, 12 Mar 2021 15:42:08 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 12 Mar 2021 15:42:08 +0530
Reported-by: Thomas Munro
Author: Takayuki Tsunakawa
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/CA+hUKG+oPoFizjABt=GXZWTEHx3oev5rAe2scjW2r6F1rguo5w@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
Fix use of relcache TriggerDesc field introduced by commit 05c8482f7f.
commit : e2cda3c20a61c76e497fb2ebb6d2b2ae8c43c014
author : Amit Kapila <[email protected]>
date : Fri, 12 Mar 2021 15:14:41 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 12 Mar 2021 15:14:41 +0530
The commit added code which used a relcache TriggerDesc field across
another cache access, which it shouldn't because the relcache doesn't
guarantee it won't get moved.
Diagnosed-by: Tom Lane
Author: Greg Nancarrow
Reviewed-by: Hou Zhijie, Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/util/clauses.c
Poll postmaster less frequently in recovery.
commit : 57dcc2ef3320de3e1aef6f668601e6ad2bdf88c4
author : Thomas Munro <[email protected]>
date : Fri, 12 Mar 2021 19:08:52 +1300
committer: Thomas Munro <[email protected]>
date : Fri, 12 Mar 2021 19:08:52 +1300
Since commits 9f095299 and f98b8476 we don't poll the postmaster
pipe at all during crash recovery on Linux and FreeBSD, but on other
operating systems we were still doing it for every WAL record. Do it
less frequently on operating systems where system calls are required, at
the cost of delaying exit a bit after postmaster death. This avoids
expensive system calls reported to slow down CPU-bound recovery by as
much as 10-30%.
Reviewed-by: Heikki Linnakangas <[email protected]>
Reviewed-by: Fujii Masao <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGK1607VmtrDUHQXrsooU%3Dap4g4R2yaoByWOOA3m8xevUQ%40mail.gmail.com
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/startup.c
Add condition variable for walreceiver shutdown.
commit : de829ddf23f69190efb4e0178704c4c4228e17cd
author : Thomas Munro <[email protected]>
date : Fri, 12 Mar 2021 19:07:27 +1300
committer: Thomas Munro <[email protected]>
date : Fri, 12 Mar 2021 19:07:27 +1300
Use this new CV to wait for walreceiver shutdown without a sleep/poll
loop, while also benefiting from standard postmaster death handling.
Discussion: https://postgr.es/m/CA%2BhUKGK1607VmtrDUHQXrsooU%3Dap4g4R2yaoByWOOA3m8xevUQ%40mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/postmaster/pgstat.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walreceiverfuncs.c
M src/include/pgstat.h
M src/include/replication/walreceiver.h
Add condition variable for recovery resume.
commit : 600f2f50b7a57c8481276450c9019fa7b3656411
author : Thomas Munro <[email protected]>
date : Fri, 12 Mar 2021 19:03:52 +1300
committer: Thomas Munro <[email protected]>
date : Fri, 12 Mar 2021 19:03:52 +1300
Replace a sleep loop with a CV, to get a fast reaction time when
recovery is resumed or the postmaster exits via standard infrastructure.
Unfortunately we still need to wake up every second to perform extra
polling during the recovery pause loop.
Discussion: https://postgr.es/m/CA%2BhUKGK1607VmtrDUHQXrsooU%3Dap4g4R2yaoByWOOA3m8xevUQ%40mail.gmail.com
M src/backend/access/transam/xlog.c
Send statistics collected during shutdown checkpoint to the stats collector.
commit : b82640df0062483431608b7e9e074255f03e6c02
author : Fujii Masao <[email protected]>
date : Fri, 12 Mar 2021 14:23:00 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 12 Mar 2021 14:23:00 +0900
When shutdown is requested, checkpointer performs checkpoint or
restartpoint, and updates the statistics, before it exits. But previously
checkpointer didn't send those statistics to the stats collector.
Shutdown checkpoint and restartpoint are treated as requested ones
instead of scheduled ones, so the number of them are counted in
pg_stat_bgwriter.checkpoints_req column.
Author: Masahiro Ikeda
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/checkpointer.c
Force to send remaining WAL stats to the stats collector at walwriter exit.
commit : 33394ee6f2433d3cc7785428a77cc9a813254df7
author : Fujii Masao <[email protected]>
date : Fri, 12 Mar 2021 13:29:59 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 12 Mar 2021 13:29:59 +0900
In walwriter's main loop, WAL stats message is only sent if enough time
has passed since last one was sent to reach PGSTAT_STAT_INTERVAL msecs.
This is necessary to avoid overloading to the stats collector. But this
can cause recent WAL stats to be unsent when walwriter exits.
To ensure that all the WAL stats are sent, this commit makes walwriter
force to send remaining WAL stats to the collector when it exits because
of shutdown request. Note that those remaining WAL stats can still be
unsent when walwriter exits with non-zero exit code (e.g., FATAL error).
This is OK because that walwriter exit leads to server crash and
subsequent recovery discards all the stats. So there is no need to send
remaining stats in that case.
Author: Masahiro Ikeda
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/walwriter.c
Minor modernization for README.barrier.
commit : 43c66624964aa1d2f519ad6be0c5ea8f170cf357
author : Thomas Munro <[email protected]>
date : Fri, 12 Mar 2021 15:24:28 +1300
committer: Thomas Munro <[email protected]>
date : Fri, 12 Mar 2021 15:24:28 +1300
Itanium is very uncommon and being discontinued. ARM is everywhere.
Prefer ARM as an example of an architecture with weak memory ordering.
M src/backend/storage/lmgr/README.barrier
Save a few cycles during nbtree VACUUM.
commit : 7bb97211a5589265f3f88183ae9353639ab184c6
author : Peter Geoghegan <[email protected]>
date : Thu, 11 Mar 2021 14:18:23 -0800
committer: Peter Geoghegan <[email protected]>
date : Thu, 11 Mar 2021 14:18:23 -0800
Avoid calling RelationGetNumberOfBlocks() unnecessarily in the common
case where there are no deleted but not yet recycled pages to recycle
during a cleanup-only nbtree VACUUM operation.
Follow-up to commit e5d8a999, which (among other things) taught the
"skip full scan" nbtree VACUUM mechanism to only trigger a full index
scan when the absolute number of deleted pages in the index is
considered excessive.
M src/backend/access/nbtree/nbtree.c
Add back vacuum_cleanup_index_scale_factor parameter.
commit : effdd3f3b633e88feaa675377075f02ecc99aee4
author : Peter Geoghegan <[email protected]>
date : Thu, 11 Mar 2021 12:42:46 -0800
committer: Peter Geoghegan <[email protected]>
date : Thu, 11 Mar 2021 12:42:46 -0800
Commit 9f3665fb removed the vacuum_cleanup_index_scale_factor storage
parameter. However, that creates dump/reload hazards when moving across
major versions.
Add back the vacuum_cleanup_index_scale_factor parameter (though not the
GUC of the same name) purely to avoid problems when using tools like
pg_upgrade. The parameter remains disabled and undocumented.
No backpatch to Postgres 13, since vacuum_cleanup_index_scale_factor was
only disabled by REL_13_STABLE's version of master branch commit
9f3665fb in the first place -- the parameter already looks like this on
REL_13_STABLE.
Discussion: https://postgr.es/m/YEm/[email protected]
M src/backend/access/common/reloptions.c
M src/backend/access/nbtree/nbtutils.c
M src/include/access/nbtree.h
Be clear about whether a recovery pause has taken effect.
commit : 32fd2b57d7f64948e649fc205c43f007762ecaac
author : Robert Haas <[email protected]>
date : Thu, 11 Mar 2021 14:52:32 -0500
committer: Robert Haas <[email protected]>
date : Thu, 11 Mar 2021 14:52:32 -0500
Previously, the code and documentation seem to have essentially
assumed than a call to pg_wal_replay_pause() would take place
immediately, but that's not the case, because we only check for a
pause in certain places. This means that a tool that uses this
function and then wants to do something else afterward that is
dependent on the pause having taken effect doesn't know how long it
needs to wait to be sure that no more WAL is going to be replayed.
To avoid that, add a new function pg_get_wal_replay_pause_state()
which returns either 'not paused', 'paused requested', or 'paused'.
After calling pg_wal_replay_pause() the status will immediate change
from 'not paused' to 'pause requested'; when the startup process
has noticed this, the status will change to 'pause'. For backward
compatibility, pg_is_wal_replay_paused() still exists and returns
the same thing as before: true if a pause has been requested,
whether or not it has taken effect yet; and false if not.
The documentation is updated to clarify.
To improve the changes that a pause request is quickly confirmed
effective, adjust things so that WaitForWALToBecomeAvailable will
swiftly reach a call to recoveryPausesHere() when a pause request
is made.
Dilip Kumar, reviewed by Simon Riggs, Kyotaro Horiguchi, Yugo Nagata,
Masahiko Sawada, and Bharath Rupireddy.
Discussion: http://postgr.es/m/CAFiTN-vcLLWEm8Zr%3DYK83rgYrT9pbC8VJCfa1kY9vL3AUPfu6g%40mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogfuncs.c
M src/include/access/xlog.h
M src/include/catalog/pg_proc.dat
Re-simplify management of inStart in pqParseInput3's subroutines.
commit : 51c54bb603098416dc6f9d9d46a3d14861f8fc38
author : Tom Lane <[email protected]>
date : Thu, 11 Mar 2021 14:43:45 -0500
committer: Tom Lane <[email protected]>
date : Thu, 11 Mar 2021 14:43:45 -0500
Commit 92785dac2 copied some logic related to advancement of inStart
from pqParseInput3 into getRowDescriptions and getAnotherTuple,
because it wanted to allow user-defined row processor callbacks to
potentially longjmp out of the library, and inStart would have to be
updated before that happened to avoid an infinite loop. We later
decided that that API was impossibly fragile and reverted it, but
we didn't undo all of the related code changes, and this bit of
messiness survived. Undo it now so that there's just one place in
pqParseInput3's processing where inStart is advanced; this will
simplify addition of better tracing support.
getParamDescriptions had grown similar processing somewhere along
the way (not in 92785dac2; I didn't track down just when), but it's
actually buggy because its handling of corrupt-message cases seems to
have been copied from the v2 logic where we lacked a known message
length. The cases where we "goto not_enough_data" should not simply
return EOF, because then we won't consume the message, potentially
creating an infinite loop. That situation now represents a
definitively corrupt message, and we should report it as such.
Although no field reports of getParamDescriptions getting stuck in
a loop have been seen, it seems appropriate to back-patch that fix.
I chose to back-patch all of this to keep the logic looking more alike
in supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-protocol3.c
Refactor and generalize the ParallelSlot machinery.
commit : f71519e545a34ece0a27c8bb1a2b6e197d323163
author : Robert Haas <[email protected]>
date : Thu, 11 Mar 2021 13:17:46 -0500
committer: Robert Haas <[email protected]>
date : Thu, 11 Mar 2021 13:17:46 -0500
Create a wrapper object, ParallelSlotArray, to encapsulate the
number of slots and the slot array itself, plus some other relevant
bits of information. This reduces the number of parameters we have
to pass around all over the place.
Allow for a ParallelSlotArray to contain slots connected to
different databases within a single cluster. The current clients
of this mechanism don't need this, but it is expected to be used
by future patches.
Defer connecting to databases until we actually need the connection
for something. This is a slight behavior change for vacuumdb and
reindexdb. If you specify a number of jobs that is larger than the
number of objects, the extra connections will now not be used.
But, on the other hand, if you specify a number of jobs that is
so large that it's going to fail, the failure would previously have
happened before any operations were actually started, and now it
won't.
Mark Dilger, reviewed by me.
Discussion: http://postgr.es/m/[email protected]
Discussion: http://postgr.es/m/[email protected]
M src/bin/scripts/reindexdb.c
M src/bin/scripts/vacuumdb.c
M src/fe_utils/parallel_slot.c
M src/include/fe_utils/parallel_slot.h
M src/tools/pgindent/typedefs.list
Set libcrypto callbacks for all connection threads in libpq
commit : 2c0cefcd18161549e9e8b103f46c0f65fca84d99
author : Michael Paquier <[email protected]>
date : Thu, 11 Mar 2021 17:14:25 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 11 Mar 2021 17:14:25 +0900
Based on an analysis of the OpenSSL code with Jacob, moving to EVP for
the cryptohash computations makes necessary the setup of the libcrypto
callbacks that were getting set only for SSL connections, but not for
connections without SSL. Not setting the callbacks makes the use of
threads potentially unsafe for connections calling cryptohashes during
authentication, like MD5 or SCRAM, if a failure happens during a
cryptohash computation. The logic setting the libssl and libcrypto
states is then split into two parts, both using the same locking, with
libcrypto being set up for SSL and non-SSL connections, while SSL
connections set any libssl state afterwards as needed.
Prior to this commit, only SSL connections would have set libcrypto
callbacks that are necessary to ensure a proper thread locking when
using multiple concurrent threads in libpq (ENABLE_THREAD_SAFETY). Note
that this is only required for OpenSSL 1.0.2 and 1.0.1 (oldest version
supported on HEAD), as 1.1.0 has its own internal locking and it has
dropped support for CRYPTO_set_locking_callback().
Tests with up to 300 threads with OpenSSL 1.0.1 and 1.0.2, mixing SSL
and non-SSL connection threads did not show any performance impact after
some micro-benchmarking. pgbench can be used here with -C and a
mostly-empty script (with one \set meta-command for example) to stress
authentication requests, and we have mixed that with some custom
programs for testing.
Reported-by: Jacob Champion
Author: Michael Paquier
Reviewed-by: Jacob Champion
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/fe-secure.c
M src/interfaces/libpq/libpq-int.h
Doc: B-Tree only has one additional parameter.
commit : 3f0daeb02f8dd605f89de9aa2349137c09cc7fb4
author : Peter Geoghegan <[email protected]>
date : Wed, 10 Mar 2021 22:10:36 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 10 Mar 2021 22:10:36 -0800
Oversight in commit 9f3665fb.
Backpatch: 13-, just like commit 9f3665fb.
M doc/src/sgml/ref/create_index.sgml
Improve comment for struct BufferDesc.
commit : 049d9b872db8a24b45709dbaed9a1051e92ed513
author : Thomas Munro <[email protected]>
date : Thu, 11 Mar 2021 15:58:05 +1300
committer: Thomas Munro <[email protected]>
date : Thu, 11 Mar 2021 15:58:05 +1300
Add a note that per-buffer I/O condition variables currently live
outside the BufferDesc struct. Follow-up for commit d8725104.
Reported-by: Julien Rouhaud <[email protected]>
Discussion: https://postgr.es/m/20210311031118.hucytmrgwlktjxgq%40nol
M src/include/storage/buf_internals.h
tutorial: land height is "elevation", not "altitude"
commit : 2950ff32f31d073ea4be4f5f9b73249131f42bd7
author : Bruce Momjian <[email protected]>
date : Wed, 10 Mar 2021 20:25:19 -0500
committer: Bruce Momjian <[email protected]>
date : Wed, 10 Mar 2021 20:25:19 -0500
This is a follow-on patch to 92c12e46d5. In that patch, we renamed
"altitude" to "elevation" in the docs, based on these details:
https://mapscaping.com/blogs/geo-candy/what-is-the-difference-between-elevation-relief-and-altitude
This renames the tutorial SQL files to match the documentation.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/tutorial/advanced.source
VACUUM ANALYZE: Always update pg_class.reltuples.
commit : 5f8727f5a679452f7bbdd6966a1586934dcaa84f
author : Peter Geoghegan <[email protected]>
date : Wed, 10 Mar 2021 17:07:57 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 10 Mar 2021 17:07:57 -0800
vacuumlazy.c sometimes fails to update pg_class entries for each index
(to ensure that pg_class.reltuples is current), even though analyze.c
assumed that that must have happened during VACUUM ANALYZE. There are
at least a couple of reasons for this. For example, vacuumlazy.c could
fail to update pg_class when the index AM indicated that its statistics
are merely an estimate, per the contract for amvacuumcleanup() routines
established by commit e57345975cf back in 2006.
Stop assuming that pg_class must have been updated with accurate
statistics within VACUUM ANALYZE -- update pg_class for indexes at the
same time as the table relation in all cases. That way VACUUM ANALYZE
will never fail to keep pg_class.reltuples reasonably accurate.
The only downside of this approach (compared to the old approach) is
that it might inaccurately set pg_class.reltuples for indexes whose heap
relation ends up with the same inaccurate value anyway. This doesn't
seem too bad. We already consistently called vac_update_relstats() (to
update pg_class) for the heap/table relation twice during any VACUUM
ANALYZE -- once in vacuumlazy.c, and once in analyze.c. We now make
sure that we call vac_update_relstats() at least once (though often
twice) for each index.
This is follow up work to commit 9f3665fb, which dealt with issues in
btvacuumcleanup(). Technically this fixes an unrelated issue, though.
btvacuumcleanup() no longer provides an accurate num_index_tuples value
following commit 9f3665fb (when there was no btbulkdelete() call during
the VACUUM operation in question), but hashvacuumcleanup() has worked in
the same way for many years now.
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/CAH2-WzknxdComjhqo4SUxVFk_Q1171GJO2ZgHZ1Y6pion6u8rA@mail.gmail.com
Backpatch: 13-, just like commit 9f3665fb.
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/analyze.c
Don't consider newly inserted tuples in nbtree VACUUM.
commit : 9f3665fbfc34b963933e51778c7feaa8134ac885
author : Peter Geoghegan <[email protected]>
date : Wed, 10 Mar 2021 16:27:01 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 10 Mar 2021 16:27:01 -0800
Remove the entire idea of "stale stats" within nbtree VACUUM (stop
caring about stats involving the number of inserted tuples). Also
remove the vacuum_cleanup_index_scale_factor GUC/param on the master
branch (though just disable them on postgres 13).
The vacuum_cleanup_index_scale_factor/stats interface made the nbtree AM
partially responsible for deciding when pg_class.reltuples stats needed
to be updated. This seems contrary to the spirit of the index AM API,
though -- it is not actually necessary for an index AM's bulk delete and
cleanup callbacks to provide accurate stats when it happens to be
inconvenient. The core code owns that. (Index AMs have the authority
to perform or not perform certain kinds of deferred cleanup based on
their own considerations, such as page deletion and recycling, but that
has little to do with pg_class.reltuples/num_index_tuples.)
This issue was fairly harmless until the introduction of the
autovacuum_vacuum_insert_threshold feature by commit b07642db, which had
an undesirable interaction with the vacuum_cleanup_index_scale_factor
mechanism: it made insert-driven autovacuums perform full index scans,
even though there is no real benefit to doing so. This has been tied to
a regression with an append-only insert benchmark [1].
Also have remaining cases that perform a full scan of an index during a
cleanup-only nbtree VACUUM indicate that the final tuple count is only
an estimate. This prevents vacuumlazy.c from setting the index's
pg_class.reltuples in those cases (it will now only update pg_class when
vacuumlazy.c had TIDs for nbtree to bulk delete). This arguably fixes
an oversight in deduplication-related bugfix commit 48e12913.
[1] https://smalldatum.blogspot.com/2021/01/insert-benchmark-postgres-is-still.html
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/CAD21AoA4WHthN5uU6+WScZ7+J_RcEjmcuH94qcoUPuB42ShXzg@mail.gmail.com
Backpatch: 13-, where autovacuum_vacuum_insert_threshold was added.
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/create_index.sgml
M src/backend/access/common/reloptions.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/rmgrdesc/nbtdesc.c
M src/backend/utils/init/globals.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/psql/tab-complete.c
M src/include/access/nbtree.h
M src/include/access/nbtxlog.h
M src/include/access/xlog_internal.h
M src/include/miscadmin.h
M src/test/regress/expected/btree_index.out
M src/test/regress/sql/btree_index.sql
C comments: improve description of GiST NSN and GistBuildLSN
commit : 845ac7f847a25505e91f30dca4e0330b25785ee0
author : Bruce Momjian <[email protected]>
date : Wed, 10 Mar 2021 17:03:10 -0500
committer: Bruce Momjian <[email protected]>
date : Wed, 10 Mar 2021 17:03:10 -0500
GiST indexes are complex, so adding more details in the code might help
someone.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/gist/README
M src/include/access/gist.h
Replace buffer I/O locks with condition variables.
commit : d87251048a0f293ad20cc1fe26ce9f542de105e6
author : Thomas Munro <[email protected]>
date : Thu, 11 Mar 2021 10:05:58 +1300
committer: Thomas Munro <[email protected]>
date : Thu, 11 Mar 2021 10:05:58 +1300
1. Backends waiting for buffer I/O are now interruptible.
2. If something goes wrong in a backend that is currently performing
I/O, waiting backends no longer wake up until that backend reaches
AbortBufferIO() and broadcasts on the CV. Previously, any waiters would
wake up (because the I/O lock was automatically released) and then
busy-loop until AbortBufferIO() cleared BM_IO_IN_PROGRESS.
3. LWLockMinimallyPadded is removed, as it would now be unused.
Author: Robert Haas <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Reviewed-by: Julien Rouhaud <[email protected]>
Reviewed-by: Tom Lane <[email protected]> (earlier version, 2016)
Discussion: https://postgr.es/m/CA%2BhUKGJ8nBFrjLuCTuqKN0pd2PQOwj9b_jnsiGFFMDvUxahj_A%40mail.gmail.com
Discussion: https://postgr.es/m/CA+Tgmoaj2aPti0yho7FeEf2qt-JgQPRWb0gci_o1Hfr=C56Xng@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/postmaster/pgstat.c
M src/backend/storage/buffer/README
M src/backend/storage/buffer/buf_init.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/lmgr/lwlock.c
M src/include/pgstat.h
M src/include/storage/buf_internals.h
M src/include/storage/condition_variable.h
M src/include/storage/lwlock.h
M src/tools/pgindent/typedefs.list
Avoid creating duplicate cached plans for inherited FK constraints.
commit : c3ffe34863688115dd7878f118f2a123bafd8a26
author : Tom Lane <[email protected]>
date : Wed, 10 Mar 2021 14:22:31 -0500
committer: Tom Lane <[email protected]>
date : Wed, 10 Mar 2021 14:22:31 -0500
When a foreign key constraint is applied to a partitioned table, each
leaf partition inherits a similar FK constraint. We were processing all
of those constraints independently, meaning that in large partitioning
trees we'd build up large collections of cached FK-checking query plans.
However, in all cases but one, the generated queries are actually
identical for all members of the inheritance tree (because, in most
cases, the query only mentions the topmost table of the other side of
the FK relationship). So we can share a single cached plan among all
the partitions, saving memory, not to mention time to build and maintain
the cached plans.
Keisuke Kuroda and Amit Langote
Discussion: https://postgr.es/m/[email protected]_1
M src/backend/utils/adt/ri_triggers.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
Doc: get rid of <foreignphrase> tags.
commit : b12436340adf27aa3d334c92579e6662dd3090ce
author : Tom Lane <[email protected]>
date : Wed, 10 Mar 2021 12:38:43 -0500
committer: Tom Lane <[email protected]>
date : Wed, 10 Mar 2021 12:38:43 -0500
We italicized some, but not all, instances of "per se", "pro forma", and
"ad hoc". These phrases are widespread in formal registers of English,
so it"s debatable whether they even qualify as foreign. We could instead
try to be more consistent in the use of <foreignphrase>, but that"s
difficult to enforce, so let"s just remove the tags for those words.
The one case that seems to deserve the tag is "voilà". Instead of keeping
just one instance of the tag, change that to a more standard phrase.
John Naylor
Discussion: https://postgr.es/m/CAFBsxsHtWs_NsccAVgQ=tTUKkXHpHdkjZXtp_Cd9dGWyBDxfbQ@mail.gmail.com
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/create_role.sgml
M doc/src/sgml/rules.sgml
M doc/src/sgml/typeconv.sgml
Doc: improve introductory information about procedures.
commit : 227338b00d498d9e1c5705a1ab118585e5d57c87
author : Tom Lane <[email protected]>
date : Wed, 10 Mar 2021 11:33:50 -0500
committer: Tom Lane <[email protected]>
date : Wed, 10 Mar 2021 11:33:50 -0500
Clarify the discussion in "User-Defined Procedures", by laying out
the key differences between functions and procedures in a bulleted
list. Notably, this avoids burying the lede about procedures being
able to do transaction control. Make the back-link in the CREATE
FUNCTION reference page more prominent, and add one in CREATE
PROCEDURE.
Per gripe from Guyren Howe. Thanks to David Johnston for discussion.
Discussion: https://postgr.es/m/BYAPR03MB4903C53A8BB7EFF5EA289674A6949@BYAPR03MB4903.namprd03.prod.outlook.com
M doc/src/sgml/ref/create_function.sgml
M doc/src/sgml/ref/create_procedure.sgml
M doc/src/sgml/xfunc.sgml
Doc: fix missing mention of procedure OUT parameters.
commit : 3ebc6d295705fec37dc8f57a4ece54b370f55f72
author : Tom Lane <[email protected]>
date : Wed, 10 Mar 2021 10:59:48 -0500
committer: Tom Lane <[email protected]>
date : Wed, 10 Mar 2021 10:59:48 -0500
Small oversight in commit 2453ea142.
M doc/src/sgml/plpgsql.sgml
Add bound check before bsearch() for performance
commit : bbaf315309ed1194d451326cc8f4f59a45906408
author : Peter Eisentraut <[email protected]>
date : Wed, 10 Mar 2021 15:19:37 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 10 Mar 2021 15:19:37 +0100
In the current lazy vacuum implementation, some index AMs such as
btree indexes call lazy_tid_reaped() for each index tuple during
ambulkdelete to check if the index tuple points to the (collected)
garbage tuple. In that function, we simply call bsearch(), but we
should be able to know the result without bsearch() if the index tuple
points to the heap tuple that is out of range of the collected garbage
tuples. Therefore, add a simple bound check before resorting to
bsearch(). Testing has shown that this can give significant
performance benefits.
Author: Masahiko Sawada <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CA+fd4k76j8jKzJzcx8UqEugvayaMSnQz0iLUt_XgBp-_-bd22A@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Fix another portability bug in recent pgbench commit.
commit : c427de427ac411039d5efd5d0dcc8a4e0c6da68d
author : Thomas Munro <[email protected]>
date : Wed, 10 Mar 2021 22:22:12 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 10 Mar 2021 22:22:12 +1300
Commit 547f04e7 produced errors on AIX/xlc while building plpython. The
new code appears to be incompatible with the hack installed by commit
a11cf433. Without access to an AIX system to check, my guess is that
_POSIX_C_SOURCE may be required for <time.h> to declare the things the
header needs to see, but plpython.h undefines it.
For now, to unbreak build farm animal hoverfly, just move the new
pg_time_usec_t support into pgbench.c. Perhaps later we could figure
out what to rearrange to put it back into a header for wider use.
Discussion: https://postgr.es/m/CA%2BhUKG%2BP%2BjcD%3Dx9%2BagyTdWtjpOT64MYiGic%2Bcbu_TD8CV%3D6A3w%40mail.gmail.com
M src/bin/pgbench/pgbench.c
M src/include/portability/instr_time.h
Try to fix portability bugs in recent pgbench commits.
commit : 68b34b2338f013cb025dea360b37a3b4fc930179
author : Thomas Munro <[email protected]>
date : Wed, 10 Mar 2021 20:18:15 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 10 Mar 2021 20:18:15 +1300
1. pg_time_usec_t needs to be printed with INT64_FORMAT, not %ld, or 32
bit systems complain, per lapwing.
2. Some Windows compilers didn't like a thread function not marked with
__stdcall, per whelk; let's see if this fixes the problem.
M src/bin/pgbench/pgbench.c
Small debug message tweak
commit : 1657b37d7cce5c35e4c1d500f0a2f3736a087d82
author : Peter Eisentraut <[email protected]>
date : Wed, 10 Mar 2021 08:16:38 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 10 Mar 2021 08:16:38 +0100
This makes the wording of the delete case match the update case.
M src/backend/replication/logical/worker.c
Move tablespace path re-creation from the makefiles to pg_regress
commit : 6c788d9f6aadb41d76a72d56149268371a7895ee
author : Michael Paquier <[email protected]>
date : Wed, 10 Mar 2021 14:50:00 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 10 Mar 2021 14:50:00 +0900
Moving this logic into pg_regress fixes a potential failure with
parallel tests when pg_upgrade and the main regression test suite both
trigger the makefile rule that cleaned up testtablespace/ under
src/test/regress. Even if pg_upgrade was triggering this rule, it has
no need to do so as it uses a different tablespace path. So if
pg_upgrade triggered the makefile rule for the tablespace setup while
the main regression test suite ran the tablespace cases, it would fail.
61be85a was a similar attempt at achieving that, but that broke cases
where the regression tests require to run under an Administrator
account, like with Appveyor.
Reported-by: Andres Freund, Kyotaro Horiguchi
Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_upgrade/test.sh
M src/test/regress/GNUmakefile
M src/test/regress/pg_regress.c
M src/tools/msvc/vcregress.pl
pgbench: Synchronize client threads.
commit : aeb57af8e64000cc4288a7b8b8d7cf6040eae900
author : Thomas Munro <[email protected]>
date : Wed, 10 Mar 2021 16:17:34 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 10 Mar 2021 16:17:34 +1300
Wait until all pgbench threads are connected before benchmarking begins.
This fixes a problem where some connections could take a very long time
to be established because of lock contention from earlier connections,
making results unstable and bogus with high connection counts.
Author: Andres Freund <[email protected]>
Author: Fabien COELHO <[email protected]>
Reviewed-by: Marina Polyakova <[email protected]>
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Reviewed-by: Hayato Kuroda <[email protected]>
Reviewed-by: David Rowley <[email protected]>
Discussion: https://postgr.es/m/20200227180100.zyvjwzcpiokfsqm2%40alap3.anarazel.de
M src/bin/pgbench/pgbench.c
Add missing pthread_barrier_t.
commit : 44bf3d5083e151d772c5d6f656e3e162f573dced
author : Thomas Munro <[email protected]>
date : Wed, 10 Mar 2021 15:40:17 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 10 Mar 2021 15:40:17 +1300
Supply a simple implementation of the missing pthread_barrier_t type and
functions, for macOS.
Discussion: https://postgr.es/m/20200227180100.zyvjwzcpiokfsqm2%40alap3.anarazel.de
M configure
M configure.ac
M src/include/pg_config.h.in
A src/include/port/pg_pthread.h
A src/port/pthread_barrier_wait.c
M src/tools/msvc/Solution.pm
M src/tools/pgindent/typedefs.list
pgbench: Improve time logic.
commit : 547f04e7348b6ed992bd4a197d39661fe7c25097
author : Thomas Munro <[email protected]>
date : Wed, 10 Mar 2021 16:09:50 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 10 Mar 2021 16:09:50 +1300
Instead of instr_time (struct timespec) and the INSTR_XXX macros,
introduce pg_time_usec_t and use integer arithmetic. Don't include the
connection time in TPS unless using -C mode, but report it separately.
Author: Fabien COELHO <[email protected]>
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Reviewed-by: Hayato Kuroda <[email protected]>
Discussion: https://postgr.es/m/20200227180100.zyvjwzcpiokfsqm2%40alap3.anarazel.de
M doc/src/sgml/ref/pgbench.sgml
M src/bin/pgbench/pgbench.c
M src/include/portability/instr_time.h
M src/tools/pgindent/typedefs.list
pgbench: Refactor thread portability support.
commit : b1d6a8f86813772b9198367a34c8ff8bff7fef9e
author : Thomas Munro <[email protected]>
date : Wed, 10 Mar 2021 15:39:08 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 10 Mar 2021 15:39:08 +1300
Instead of maintaining an incomplete emulation of POSIX threads for
Windows, let's use an extremely minimalist macro-based abstraction for
now. A later patch will extend this, without the need to supply more
complicated pthread emulation code. (There may be a need for a more
serious portable thread abstraction in later projects, but this is not
it.)
Minor incidental problems fixed: it wasn't OK to use (pthread_t) 0 as a
special value, it wasn't OK to compare thread_t values with ==, and we
incorrectly assumed that pthread functions set errno.
Discussion: https://postgr.es/m/20200227180100.zyvjwzcpiokfsqm2%40alap3.anarazel.de
M src/bin/pgbench/pgbench.c
Fix valgrind issue in commit 05c8482f7f.
commit : e4e87a32cc5559303068b148bd4aa6a39b8f44a1
author : Amit Kapila <[email protected]>
date : Wed, 10 Mar 2021 10:04:20 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 10 Mar 2021 10:04:20 +0530
Initialize other newly added variables in max_parallel_hazard_context via
is_parallel_safe() because we don't check the parallel-safety of target
relations in that function.
Reported-by: Tom Lane as per buildfarm
Author: Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/util/clauses.c
Enable parallel SELECT for "INSERT INTO ... SELECT ...".
commit : 05c8482f7f69a954fd65fce85f896e848fc48197
author : Amit Kapila <[email protected]>
date : Wed, 10 Mar 2021 07:38:58 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 10 Mar 2021 07:38:58 +0530
Parallel SELECT can't be utilized for INSERT in the following cases:
- INSERT statement uses the ON CONFLICT DO UPDATE clause
- Target table has a parallel-unsafe: trigger, index expression or
predicate, column default expression or check constraint
- Target table has a parallel-unsafe domain constraint on any column
- Target table is a partitioned table with a parallel-unsafe partition key
expression or support function
The planner is updated to perform additional parallel-safety checks for
the cases listed above, for determining whether it is safe to run INSERT
in parallel-mode with an underlying parallel SELECT. The planner will
consider using parallel SELECT for "INSERT INTO ... SELECT ...", provided
nothing unsafe is found from the additional parallel-safety checks, or
from the existing parallel-safety checks for SELECT.
While checking parallel-safety, we need to check it for all the partitions
on the table which can be costly especially when we decide not to use a
parallel plan. So, in a separate patch, we will introduce a GUC and or a
reloption to enable/disable parallelism for Insert statements.
Prior to entering parallel-mode for the execution of INSERT with parallel
SELECT, a TransactionId is acquired and assigned to the current
transaction state. This is necessary to prevent the INSERT from attempting
to assign the TransactionId whilst in parallel-mode, which is not allowed.
This approach has a disadvantage in that if the underlying SELECT does not
return any rows, then the TransactionId is not used, however that
shouldn't happen in practice in many cases.
Author: Greg Nancarrow, Amit Langote, Amit Kapila
Reviewed-by: Amit Langote, Hou Zhijie, Takayuki Tsunakawa, Antonin Houska, Bharath Rupireddy, Dilip Kumar, Vignesh C, Zhihong Yu, Amit Kapila
Tested-by: Tang, Haiying
Discussion: https://postgr.es/m/CAJcOf-cXnB5cnMKqWEp2E2z7Mvcd04iLVmV=qpFJrR3AcrTS3g@mail.gmail.com
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com
M doc/src/sgml/parallel.sgml
M src/backend/access/transam/xact.c
M src/backend/executor/execMain.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/util/clauses.c
M src/backend/utils/cache/plancache.c
M src/include/access/xact.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/clauses.h
A src/test/regress/expected/insert_parallel.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/insert_parallel.sql
Revert changes for SSL compression in libpq
commit : 0ba71107efeeccde9158f47118f95043afdca0bb
author : Michael Paquier <[email protected]>
date : Wed, 10 Mar 2021 09:35:42 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 10 Mar 2021 09:35:42 +0900
This partially reverts 096bbf7 and 9d2d457, undoing the libpq changes as
it could cause breakages in distributions that share one single libpq
version across multiple major versions of Postgres for extensions and
applications linking to that.
Note that the backend is unchanged here, and it still disables SSL
compression while simplifying the underlying catalogs that tracked if
compression was enabled or not for a SSL connection.
Per discussion with Tom Lane and Daniel Gustafsson.
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/libpq.sgml
M src/bin/psql/command.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/libpq-int.h
M src/test/ssl/t/001_ssltests.pl
Fix vague comment in jsonb documentation
commit : 6540cc517dd452874a4e0fb268aee9b92e5136c6
author : Alexander Korotkov <[email protected]>
date : Tue, 9 Mar 2021 18:16:03 +0300
committer: Alexander Korotkov <[email protected]>
date : Tue, 9 Mar 2021 18:16:03 +0300
The sample query fails because of an attempt to update the key of a numeric.
But the comment says it's just because of the missing object key. That's not
correct because jsonb subscription automatically adds missing keys.
Reported-by: Nikita Konev
M doc/src/sgml/json.sgml
libpq: Remove deprecated connection parameters authtype and tty
commit : 14d9b37607ad30c3848ea0f2955a78436eff1268
author : Peter Eisentraut <[email protected]>
date : Tue, 9 Mar 2021 15:01:22 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 9 Mar 2021 15:01:22 +0100
The authtype parameter was deprecated and made inactive in commit
d5bbe2aca55bc8, but the environment variable was left defined and thus
tested with a getenv call even though the value is of no use. Also,
if it would exist it would be copied but never freed as the cleanup
code had been removed.
tty was deprecated in commit cb7fb3ca958ec8bd5a14e7 but most of the
infrastructure around it remained in place.
Author: Daniel Gustafsson <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/libpq-int.h
M src/test/examples/testlibpq4.c
Switch back sslcompression to be a normal input field in libpq
commit : 096bbf7c934a4288c9e48a6ac8e91d8753ac1ccd
author : Michael Paquier <[email protected]>
date : Tue, 9 Mar 2021 19:52:36 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 9 Mar 2021 19:52:36 +0900
Per buildfarm member crake, any servers including a postgres_fdw server
with this option set would fail to do a pg_upgrade properly as the
option got hidden in f9264d1 by becoming a debug option, making the
restore of the FDW server fail.
This changes back the option in libpq to be visible, but still inactive
to fix this upgrade issue.
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/interfaces/libpq/fe-connect.c
Track total amounts of times spent writing and syncing WAL data to disk.
commit : ff99918c625a84c91e7391db9032112ec8653623
author : Fujii Masao <[email protected]>
date : Tue, 9 Mar 2021 16:52:06 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 9 Mar 2021 16:52:06 +0900
This commit adds new GUC track_wal_io_timing. When this is enabled,
the total amounts of time XLogWrite writes and issue_xlog_fsync syncs
WAL data to disk are counted in pg_stat_wal. This information would be
useful to check how much WAL write and sync affect the performance.
Enabling track_wal_io_timing will make the server query the operating
system for the current time every time WAL is written or synced,
which may cause significant overhead on some platforms. To avoid such
additional overhead in the server with track_io_timing enabled,
this commit introduces track_wal_io_timing as a separate parameter from
track_io_timing.
Note that WAL write and sync activity by walreceiver has not been tracked yet.
This commit makes the server also track the numbers of times XLogWrite
writes and issue_xlog_fsync syncs WAL data to disk, in pg_stat_wal,
regardless of the setting of track_wal_io_timing. This counters can be
used to calculate the WAL write and sync time per request, for example.
Bump PGSTAT_FILE_FORMAT_ID.
Bump catalog version.
Author: Masahiro Ikeda
Reviewed-By: Japin Li, Hayato Kuroda, Masahiko Sawada, David Johnston, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/xlog.c
M src/backend/catalog/system_views.sql
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/walwriter.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/access/xlog.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
Add support for more progress reporting in COPY
commit : 9d2d45700928d49212fb7ed140feeaebe3a6014f
author : Michael Paquier <[email protected]>
date : Tue, 9 Mar 2021 14:21:03 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 9 Mar 2021 14:21:03 +0900
The command (TO or FROM), its type (file, pipe, program or callback),
and the number of tuples excluded by a WHERE clause in COPY FROM are
added to the progress reporting already available.
The column "lines_processed" is renamed to "tuples_processed" to
disambiguate the meaning of this column in the cases of CSV and BINARY
COPY and to be more consistent with the other catalog progress views.
Bump catalog version, again.
Author: Matthias van de Meent
Reviewed-by: Michael Paquier, Justin Pryzby, Bharath Rupireddy, Josef
Šimánek, Tomas Vondra
Discussion: https://postgr.es/m/CAEze2WiOcgdH4aQA8NtZq-4dgvnJzp8PohdeKchPkhMY-jWZXA@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/commands/copyfrom.c
M src/backend/commands/copyto.c
M src/include/catalog/catversion.h
M src/include/commands/progress.h
M src/test/regress/expected/rules.out
Remove support for SSL compression
commit : f9264d1524baa19e4a0528f033681ef16f61b137
author : Michael Paquier <[email protected]>
date : Tue, 9 Mar 2021 11:16:47 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 9 Mar 2021 11:16:47 +0900
PostgreSQL disabled compression as of e3bdb2d and the documentation
recommends against using it since. Additionally, SSL compression has
been disabled in OpenSSL since version 1.1.0, and was disabled in many
distributions long before that. The most recent TLS version, TLSv1.3,
disallows compression at the protocol level.
This commit removes the feature itself, removing support for the libpq
parameter sslcompression (parameter still listed for compatibility
reasons with existing connection strings, just ignored), and removes
the equivalent field in pg_stat_ssl and de facto PgBackendSSLStatus.
Note that, on top of removing the ability to activate compression by
configuration, compression is actively disabled in both frontend and
backend to avoid overrides from local configurations.
A TAP test is added for deprecated SSL parameters to check after
backwards compatibility.
Bump catalog version.
Author: Daniel Gustafsson
Reviewed-by: Peter Eisentraut, Magnus Hagander, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/libpq.sgml
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/libpq/be-secure-openssl.c
M src/backend/postmaster/pgstat.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/init/postinit.c
M src/bin/psql/command.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/libpq/libpq-be.h
M src/include/pgstat.h
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/libpq-int.h
M src/test/regress/expected/rules.out
M src/test/ssl/t/001_ssltests.pl
Complain if a function-in-FROM returns a set when it shouldn't.
commit : d4545dc19b8ea670bf62e06d22b0e4e6fcb45153
author : Tom Lane <[email protected]>
date : Mon, 8 Mar 2021 18:54:55 -0500
committer: Tom Lane <[email protected]>
date : Mon, 8 Mar 2021 18:54:55 -0500
Throw a "function protocol violation" error if a function in FROM
tries to return a set though it wasn't marked proretset. Although
such cases work at the moment, it doesn't seem like something we
want to guarantee will keep working. Besides, there are other
negative consequences of not setting the proretset flag, such as
potentially bad plans.
No back-patch, since if there is any third-party code violating
this expectation, people wouldn't appreciate us breaking it in
a minor release.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execSRF.c
Properly mark pg_stat_get_subscription() as returning a set.
commit : fed10d4eec79242688382d03ddca82007160ee6f
author : Tom Lane <[email protected]>
date : Mon, 8 Mar 2021 18:47:23 -0500
committer: Tom Lane <[email protected]>
date : Mon, 8 Mar 2021 18:47:23 -0500
The initial catalog data for this function failed to set proretset
or provide a prorows estimate. It accidentally worked anyway when
invoked in the FROM clause, because the executor isn't too picky
about this; but the planner didn't expect the function to return
multiple rows, which could lead to bad plans. Also the function
would fail if invoked in the SELECT list.
We can't easily back-patch this fix, but fortunately the bug's
consequences aren't awful in most cases. Getting this right is
mainly an exercise in future-proofing.
Discussion: https://postgr.es/m/[email protected]
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Validate the OID argument of pg_import_system_collations().
commit : 5c06abb9b97b69513a3998cccf89556e73052e02
author : Tom Lane <[email protected]>
date : Mon, 8 Mar 2021 18:21:51 -0500
committer: Tom Lane <[email protected]>
date : Mon, 8 Mar 2021 18:21:51 -0500
"SELECT pg_import_system_collations(0)" caused an assertion failure.
With a random nonzero argument --- or indeed with zero, in non-assert
builds --- it would happily make pg_collation entries with garbage
values of collnamespace. These are harmless as far as I can tell
(unless maybe the OID happens to become used for a schema, later on?).
In any case this isn't a security issue, since the function is
superuser-only. But it seems like a gotcha for unwary DBAs, so let's
add a check that the given OID belongs to some schema.
Back-patch to v10 where this function was introduced.
M src/backend/commands/collationcmds.c
Further tweak memory management for regex DFAs.
commit : 6c20bdb2a279086777a3595ab00bcf14671fc5a1
author : Tom Lane <[email protected]>
date : Mon, 8 Mar 2021 16:32:29 -0500
committer: Tom Lane <[email protected]>
date : Mon, 8 Mar 2021 16:32:29 -0500
Coverity is still unhappy after commit 190c79884, and after looking
closer I think it might be onto something. The callers of newdfa()
typically drop out if v->err has been set nonzero, which newdfa()
is faithfully doing if it fails. However, what if v->err was already
nonzero before we entered newdfa()? Then newdfa() could succeed and
the caller would promptly leak its result.
I don't think this scenario can actually happen, but the predicate
"v->err is always zero when newdfa() is called" seems difficult to be
entirely sure of; there's a good deal of code that potentially could
get that wrong.
It seems better to adjust the callers to directly check for a null
result instead of relying on ISERR() tests. This is slightly cheaper
than the previous coding anyway.
Lacking evidence that there's any real bug, no back-patch.
M src/backend/regex/rege_dfa.c
M src/backend/regex/regexec.c
Track replication origin progress for rollbacks.
commit : 8a812e5106c5db50039336288d376a188844e2cc
author : Amit Kapila <[email protected]>
date : Mon, 8 Mar 2021 07:54:03 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 8 Mar 2021 07:54:03 +0530
Commit 1eb6d6527a allowed to track replica origin replay progress for 2PC
but it was not complete. It misses to properly track the progress for
rollback prepared especially it missed updating the code for recovery.
Additionally, we need to allow tracking it on subscriber nodes where
wal_level might not be logical.
It is required to track decoding of 2PC which is committed in PG14
(a271a1b50e) and also nobody complained about this till now so not
backpatching it.
Author: Amit Kapila
Reviewed-by: Michael Paquier and Ajin Cherian
Discussion: https://postgr.es/m/CAA4eK1L-kHmMnSdrRW6UhRbCjR7cgh04c+6psY15qzT6ktcd+g@mail.gmail.com
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xact.c
Add bit_xor aggregate function
commit : f9a0392e1cf36179c984f655a6589734b8dd7468
author : Peter Eisentraut <[email protected]>
date : Sat, 6 Mar 2021 19:25:45 +0100
committer: Peter Eisentraut <[email protected]>
date : Sat, 6 Mar 2021 19:25:45 +0100
This can be used as a checksum for unordered sets. bit_and and bit_or
already exist.
Author: Alexey Bashtanov <[email protected]>
Reviewed-by: Ibrar Ahmed <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/func.sgml
M src/include/catalog/catversion.h
M src/include/catalog/pg_aggregate.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql
Clarify that CREATEROLE roles are considered Administrators
commit : 30b26dfac7db2fc76dafaa8a6c459871bfcd100e
author : Magnus Hagander <[email protected]>
date : Sat, 6 Mar 2021 18:11:00 +0100
committer: Magnus Hagander <[email protected]>
date : Sat, 6 Mar 2021 18:11:00 +0100
Author: Michael Banck
Reviewed-By: Robert Treat
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/user-manag.sgml
Add binary I/O capability for cube datatype.
commit : 1265a9c8f871cfa5aea4d607b1ab72b45b96dbb7
author : Tom Lane <[email protected]>
date : Sat, 6 Mar 2021 12:04:00 -0500
committer: Tom Lane <[email protected]>
date : Sat, 6 Mar 2021 12:04:00 -0500
We can adjust the not-yet-released cube--1.4--1.5.sql upgrade
rather than making a whole new version.
KaiGai Kohei
Discussion: https://postgr.es/m/CAOP8fzZO4y60QPTK=RGDXeVeVHV9tLHKOsh7voUOoUouVCPV8A@mail.gmail.com
M contrib/cube/cube–1.4–1.5.sql
M contrib/cube/cube.c
Improve docs on updatable views
commit : e045565dae4b2368ba812d60ef0fb7f927fe2646
author : Magnus Hagander <[email protected]>
date : Sat, 6 Mar 2021 17:36:48 +0100
committer: Magnus Hagander <[email protected]>
date : Sat, 6 Mar 2021 17:36:48 +0100
Introduce the options before going into details, and add a link to the
CREATE TRIGGER documentation.
Author: David Johnston
Reviewed-By: Anastasia Lubennikova
Discussion: https://postgr.es/m/CAKFQuwYLLRhheo0_Y4Jp=vJ_YDsz1KoRuTpX1A_bUxmHTmLe-A@mail.gmail.com
M doc/src/sgml/rules.sgml
pgbench: Simplify some port, host, user and dbname assignments
commit : f1516ad7b3a9645a316846fa7b2229163bc55907
author : Michael Paquier <[email protected]>
date : Sat, 6 Mar 2021 21:26:34 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 6 Mar 2021 21:26:34 +0900
Using pgbench in an environment with both PGPORT and PGUSER set would
have caused the generation of a debug log with an incorrect database
name due to an oversight in 412893b. Not specifying user, port and/or
database using the option switches, without their respective environment
variables, generated a log entry with empty strings, which was
rather useless.
This commit fixes this set of issues by simplifying the logic grabbing
the connection information, removing a set of getenv() calls that
emulated what libpq already does. The faulty debug log now directly
uses the information from the libpq connection, and it gets generated
after the connection to the backend is completed, not before it (in the
event of a failure libpq would complain with more information about the
connection attempt so the log is not really useful before anyway).
Author: Kota Miyake
Reviewed-by: Fujii Masao, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/bin/pgbench/pgbench.c
Doc: remove obsolete entries in table of BRIN strategy numbers.
commit : 0ce4cd04da558178b0186057b721c50a00b7a945
author : Tom Lane <[email protected]>
date : Fri, 5 Mar 2021 10:59:23 -0500
committer: Tom Lane <[email protected]>
date : Fri, 5 Mar 2021 10:59:23 -0500
See commit 591d282e8.
Noted by Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/brin.sgml
Remove deprecated containment operators for contrib types.
commit : 112d411fbeb56afd18c117e20b524a86afc9aba5
author : Tom Lane <[email protected]>
date : Fri, 5 Mar 2021 10:45:41 -0500
committer: Tom Lane <[email protected]>
date : Fri, 5 Mar 2021 10:45:41 -0500
Since PG 8.2, @ and ~ have been deprecated aliases for the containment
operators @> and <@. It seems like enough time has passed to actually
remove them, so do so.
This completes the project begun in commit 2f70fdb06. Note that in
the core types, the relation to the preferred operator names was
reversed from what it is in these contrib modules. The confusion
that induced was a large part of the reason for deprecation.
Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M contrib/cube/Makefile
A contrib/cube/cube–1.4–1.5.sql
M contrib/cube/cube.control
M contrib/hstore/hstore–1.7–1.8.sql
M contrib/intarray/Makefile
A contrib/intarray/intarray–1.4–1.5.sql
M contrib/intarray/intarray.control
M contrib/seg/Makefile
A contrib/seg/seg–1.3–1.4.sql
M contrib/seg/seg.control
M doc/src/sgml/cube.sgml
M doc/src/sgml/hstore.sgml
M doc/src/sgml/intarray.sgml
M doc/src/sgml/seg.sgml
doc: Add backlinks to progress reporting documentation
commit : 8a8f4d8ede288c2a29105f4708e22ce7f3526149
author : Michael Paquier <[email protected]>
date : Fri, 5 Mar 2021 14:58:16 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 5 Mar 2021 14:58:16 +0900
Previously, the only place where progress reports were mentioned is in
the section for monitoring dedicated to its catalogs. This makes the
progress reporting more discoverable, by adding links from the pages of
the commands supporting progress reports to their related catalog
views.
Author: Matthias van de Meent
Reviewed-by: Justin Pryzby, Bharath Rupireddy, Josef Šimánek, Tomas
Vondra
Discussion: https://postgr.es/m/CAEze2WiOcgdH4aQA8NtZq-4dgvnJzp8PohdeKchPkhMY-jWZXA@mail.gmail.com
M doc/src/sgml/ref/analyze.sgml
M doc/src/sgml/ref/cluster.sgml
M doc/src/sgml/ref/copy.sgml
M doc/src/sgml/ref/create_index.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/reindex.sgml
M doc/src/sgml/ref/vacuum.sgml
Add support for PROVE_TESTS and PROVE_FLAGS in MSVC scripts
commit : 5bca69a76b3046a85c60c48271c1831fd5affa51
author : Michael Paquier <[email protected]>
date : Fri, 5 Mar 2021 10:12:49 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 5 Mar 2021 10:12:49 +0900
These can be set in buildenv.pl or a "set" command within a Windows
terminal. The MSVC script vcregress.pl parses the values available in
the environment to build the resulting prove commands, and the parsing
of PROVE_TESTS is able to handle name patterns in the same way as other
platforms.
Not specifying those environment values makes vcregress.pl fall back to
the previous default, with no extra flags for the prove command, and all
the tests run within t/.
Author: Michael Paquier
Reviewed-by: Juan José Santamaría Flecha, Julien Rouhaud
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/install-windows.sgml
M src/tools/msvc/vcregress.pl
Close psql processes gracefully in recovery tests
commit : d3676a2e9f10a0972c6d6649235c1c7492cd6dea
author : Andrew Dunstan <[email protected]>
date : Thu, 4 Mar 2021 13:13:10 -0500
committer: Andrew Dunstan <[email protected]>
date : Thu, 4 Mar 2021 13:13:10 -0500
Under windows, psql processes need to be ended explicitly, or the TAP
tests hang. However, the recovery tests were doing this via
IPC::Run::kill_kill(), which causes other major problems on Windows.
We solve this by instead sending '\q' to psql so it quits of its own
accord, and then simply waiting for it. This means we can now run almost
all the recovery tests on all Windows platforms.
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/011_crash_recovery.pl
M src/test/recovery/t/021_row_visibility.pl
pg_upgrade: Fix oversight in version checking
commit : 040af779382e8e4797242c49b93a5a8f9b79c370
author : Peter Eisentraut <[email protected]>
date : Thu, 4 Mar 2021 08:21:59 +0100
committer: Peter Eisentraut <[email protected]>
date : Thu, 4 Mar 2021 08:21:59 +0100
Mistake in f06b1c598254f8adb2b7f51d6a7685618a7fb121: We should only
check the version of the binaries in the target installation. The
source installation can of course be of a different version.
Reviewed-by: Daniel Gustafsson <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/E1lHNKN-0005IC-V6%40gemulon.postgresql.org
M src/bin/pg_upgrade/exec.c
Remove redundant getenv() for PGUSER, in psql help.
commit : 4a4241e15b246663fc44b5e5355bc6d19c6d2aa6
author : Fujii Masao <[email protected]>
date : Thu, 4 Mar 2021 18:23:22 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 4 Mar 2021 18:23:22 +0900
Previously psql obtained the value of PGUSER twice to display
a default user in its help message.
Author: Kota Miyake
Reviewed-by: Nitin Jadhav, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/help.c
Avoid extra newline in errors received in FE protocol version 2.
commit : 85d94c575302644e7ddb4392c3bda572f981ec34
author : Heikki Linnakangas <[email protected]>
date : Thu, 4 Mar 2021 10:56:33 +0200
committer: Heikki Linnakangas <[email protected]>
date : Thu, 4 Mar 2021 10:56:33 +0200
Contrary to what the comment said, the postmaster does in fact end all
its messages in a newline, since server version 7.2. Be tidy and don't
add an extra newline if the error message already has one.
Discussion: https://www.postgresql.org/message-id/CAFBsxsEdgMXc%2BtGfEy9Y41i%3D5pMMjKeH8t8vSAypR3ZnAoQnHg%40mail.gmail.com
M src/interfaces/libpq/fe-connect.c
Remove server and libpq support for old FE/BE protocol version 2.
commit : 3174d69fb96a66173224e60ec7053b988d5ed4d9
author : Heikki Linnakangas <[email protected]>
date : Thu, 4 Mar 2021 10:45:55 +0200
committer: Heikki Linnakangas <[email protected]>
date : Thu, 4 Mar 2021 10:45:55 +0200
Protocol version 3 was introduced in PostgreSQL 7.4. There shouldn't be
many clients or servers left out there without version 3 support. But as
a courtesy, I kept just enough of the old protocol support that we can
still send the "unsupported protocol version" error in v2 format, so that
old clients can display the message properly. Likewise, libpq still
understands v2 ErrorResponse messages when establishing a connection.
The impetus to do this now is that I'm working on a patch to COPY
FROM, to always prefetch some data. We cannot do that safely with the
old protocol, because it requires parsing the input one byte at a time
to detect the end-of-copy marker.
Reviewed-by: Tom Lane, Alvaro Herrera, John Naylor
Discussion: https://www.postgresql.org/message-id/9ec25819-0a8a-d51a-17dc-4150bb3cca3b%40iki.fi
M doc/src/sgml/libpq.sgml
M src/backend/access/common/printtup.c
M src/backend/commands/async.c
M src/backend/commands/copyfrom.c
M src/backend/commands/copyfromparse.c
M src/backend/commands/copyto.c
M src/backend/libpq/auth.c
M src/backend/libpq/pqcomm.c
M src/backend/libpq/pqmq.c
M src/backend/postmaster/postmaster.c
M src/backend/tcop/dest.c
M src/backend/tcop/fastpath.c
M src/backend/tcop/postgres.c
M src/backend/utils/error/elog.c
M src/backend/utils/misc/guc.c
M src/bin/psql/common.c
M src/bin/psql/copy.c
M src/include/commands/copyfrom_internal.h
M src/include/libpq/libpq.h
M src/include/libpq/pqcomm.h
M src/include/tcop/fastpath.h
M src/include/utils/elog.h
M src/interfaces/libpq/Makefile
M src/interfaces/libpq/fe-auth.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-lobj.c
M src/interfaces/libpq/fe-misc.c
D src/interfaces/libpq/fe-protocol2.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
M src/interfaces/libpq/nls.mk
Add trim_array() function.
commit : 0a687c8f103d217ff1ca8c34a644b380d89bb0ad
author : Tom Lane <[email protected]>
date : Wed, 3 Mar 2021 16:39:57 -0500
committer: Tom Lane <[email protected]>
date : Wed, 3 Mar 2021 16:39:57 -0500
This has been in the SQL spec since 2008. It's a pretty thin
wrapper around the array slice functionality, but the spec
says we should have it, so here it is.
Vik Fearing, reviewed by Dian Fay
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/catalog/sql_features.txt
M src/backend/utils/adt/arrayfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/arrays.out
M src/test/regress/sql/arrays.sql
Make test_target_session_attrs more robust against connection failure.
commit : 3769e11a31831fc2f3bd4c4a24b4f45c352fb8fb
author : Tom Lane <[email protected]>
date : Wed, 3 Mar 2021 13:51:43 -0500
committer: Tom Lane <[email protected]>
date : Wed, 3 Mar 2021 13:51:43 -0500
Feed the desired command to psql via "-c" not stdin, else Perl
may complain that it can't push stdin to an already-failed psql
process, as seen in intermittent buildfarm failures.
Make some minor cosmetic improvements while at it.
Before commit ee28cacf6 we had no tests here that expected failure
to connect, so there seems no need for a back-patch.
Discussion: https://postgr.es/m/CALDaNm2mo8YED=M2ZJKGf1U3F3mw6SaQuLXWCK8rZP6sECYcrA@mail.gmail.com
M src/test/recovery/t/001_stream_rep.pl
pg_upgrade: Check version of target cluster binaries
commit : f06b1c598254f8adb2b7f51d6a7685618a7fb121
author : Peter Eisentraut <[email protected]>
date : Wed, 3 Mar 2021 09:44:46 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 3 Mar 2021 09:44:46 +0100
This expands the binary validation in pg_upgrade with a version
check per binary to ensure that the target cluster installation
only contains binaries from the target version.
In order to reduce duplication, validate_exec is exported from
port.h and the local copy in pg_upgrade is removed.
Author: Daniel Gustafsson <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/bin/pg_upgrade/exec.c
M src/common/exec.c
M src/include/port.h
Clarify the usage of max_replication_slots on the subscriber side.
commit : 8af3c233e423e106a5121000ecd92abf57c11704
author : Amit Kapila <[email protected]>
date : Wed, 3 Mar 2021 12:01:56 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 3 Mar 2021 12:01:56 +0530
It was not clear in the docs that the max_replication_slots is also used
to track replication origins on the subscriber side.
Author: Paul Martinez
Reviewed-by: Amit Kapila
Backpatch-through: 10 where logical replication was introduced
Discussion: https://postgr.es/m/CACqFVBZgwCN_pHnW6dMNCrOS7tiHCw6Retf_=U2Vvj3aUSeATw@mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/logical-replication.sgml
Some copy-editing of GUC descriptions
commit : e527a99055154bb7fe6ed67f0417a67cf11fa056
author : Peter Eisentraut <[email protected]>
date : Wed, 3 Mar 2021 07:14:35 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 3 Mar 2021 07:14:35 +0100
M src/backend/utils/misc/guc.c
Silence perlcritic warning in commit ee28cacf6.
commit : d422a2a94b1e7a7bb25da9d5511fffff750c3f21
author : Tom Lane <[email protected]>
date : Tue, 2 Mar 2021 23:32:43 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Mar 2021 23:32:43 -0500
Per buildfarm; this fix is from Michael Paquier (vignesh C
proposed nearly the same).
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/001_stream_rep.pl
Use sort_template.h for qsort_tuple() and qsort_ssup().
commit : 8eda3eba3063d619214b59f7bc7ecfc09c21b93a
author : Thomas Munro <[email protected]>
date : Wed, 3 Mar 2021 16:57:42 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 3 Mar 2021 16:57:42 +1300
Replace the Perl code previously used to generate specialized sort
functions with sort_template.h.
Reviewed-by: Daniel Gustafsson <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJ2-eaDqAum5bxhpMNhvuJmRDZxB_Tow0n-gse%2BHG0Yig%40mail.gmail.com
M src/backend/Makefile
D src/backend/utils/sort/.gitignore
M src/backend/utils/sort/Makefile
D src/backend/utils/sort/gen_qsort_tuple.pl
M src/backend/utils/sort/tuplesort.c
M src/tools/msvc/Solution.pm
M src/tools/msvc/clean.bat
Use sort_template.h for qsort() and qsort_arg().
commit : f374f4d66425060158d3c743dc6fb16cc0a2756a
author : Thomas Munro <[email protected]>
date : Wed, 3 Mar 2021 16:57:26 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 3 Mar 2021 16:57:26 +1300
Reduce duplication by using the new template.
Reviewed-by: Daniel Gustafsson <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJ2-eaDqAum5bxhpMNhvuJmRDZxB_Tow0n-gse%2BHG0Yig%40mail.gmail.com
M src/port/qsort.c
M src/port/qsort_arg.c
Add sort_template.h for making sort functions.
commit : 0a1f1d3cac6baaa3744b89336673caba702f7628
author : Thomas Munro <[email protected]>
date : Wed, 3 Mar 2021 16:57:14 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 3 Mar 2021 16:57:14 +1300
Move our qsort implementation into a header that can be used to define
specialized functions for better performance and reduced duplication.
Reviewed-by: Daniel Gustafsson <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJ2-eaDqAum5bxhpMNhvuJmRDZxB_Tow0n-gse%2BHG0Yig%40mail.gmail.com
A src/include/lib/sort_template.h
M src/tools/pgindent/typedefs.list
Add option to enable two_phase commits via pg_create_logical_replication_slot.
commit : 19890a064ebf53dedcefed0d8339ed3d449b06e6
author : Amit Kapila <[email protected]>
date : Wed, 3 Mar 2021 07:28:43 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 3 Mar 2021 07:28:43 +0530
Commit 0aa8a01d04 extends the output plugin API to allow decoding of
prepared xacts and allowed the user to enable/disable the two-phase option
via pg_logical_slot_get_changes(). This can lead to a problem such that
the first time when it gets changes via pg_logical_slot_get_changes()
without two_phase option enabled it will not get the prepared even though
prepare is after consistent snapshot. Now next time during getting changes,
if the two_phase option is enabled it can skip prepare because by that
time start decoding point has been moved. So the user will only get commit
prepared.
Allow to enable/disable this option at the create slot time and default
will be false. It will break the existing slots which is fine in a major
release.
Author: Ajin Cherian
Reviewed-by: Amit Kapila and Vignesh C
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/expected/twophase.out
M contrib/test_decoding/expected/twophase_snapshot.out
M contrib/test_decoding/expected/twophase_stream.out
M contrib/test_decoding/specs/twophase_snapshot.spec
M contrib/test_decoding/sql/twophase.sql
M contrib/test_decoding/sql/twophase_stream.sql
M contrib/test_decoding/test_decoding.c
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/logicaldecoding.sgml
M src/backend/catalog/system_views.sql
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/walsender.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/nodes/replnodes.h
M src/include/replication/slot.h
M src/test/regress/expected/rules.out
Extend the abilities of libpq's target_session_attrs parameter.
commit : ee28cacf619f4d9c23af5a80e1171a5adae97381
author : Tom Lane <[email protected]>
date : Tue, 2 Mar 2021 20:17:45 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Mar 2021 20:17:45 -0500
In addition to the existing options of "any" and "read-write", we
now support "read-only", "primary", "standby", and "prefer-standby".
"read-write" retains its previous meaning of "transactions are
read-write by default", and "read-only" inverts that. The other
three modes test specifically for hot-standby status, which is not
quite the same thing. (Setting default_transaction_read_only on
a primary server renders it read-only to this logic, but not a
standby.)
Furthermore, if talking to a v14 or later server, no extra network
round trip is needed to detect the session's status; the GUC_REPORT
variables delivered by the server are enough. When talking to an
older server, a SHOW or SELECT query is issued to detect session
read-only-ness or server hot-standby state, as needed.
Haribabu Kommi, Greg Nancarrow, Vignesh C, Tom Lane; reviewed at
various times by Laurenz Albe, Takayuki Tsunakawa, Peter Smith.
Discussion: https://postgr.es/m/CAF3+xM+8-ztOkaV9gHiJ3wfgENTq97QcjXQt+rbFQ6F7oNzt9A@mail.gmail.com
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
M src/test/recovery/t/001_stream_rep.pl
Add --tablespace option to reindexdb
commit : 57e6db706e81fd2609fa385677e6ae72471822fe
author : Michael Paquier <[email protected]>
date : Wed, 3 Mar 2021 10:14:21 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 3 Mar 2021 10:14:21 +0900
This option provides REINDEX (TABLESPACE) for reindexdb, applying the
tablespace value given by the caller to all the REINDEX queries
generated.
While on it, this commit adds some tests for REINDEX TABLESPACE, with
and without CONCURRENTLY, when run on toast indexes and tables. Such
operations are not allowed, and toast relation names are not stable
enough to be part of the main regression test suite (even if using a PL
function with a TRY/CATCH logic, as CONCURRENTLY could not be tested).
Author: Michael Paquier
Reviewed-by: Mark Dilger, Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/reindexdb.sgml
M src/bin/scripts/reindexdb.c
M src/bin/scripts/t/090_reindexdb.pl
nbtree page deletion: Add leaftopparent assertion.
commit : 5b2f2af3d9d57626b9052a05337e32ad1399093d
author : Peter Geoghegan <[email protected]>
date : Tue, 2 Mar 2021 14:06:07 -0800
committer: Peter Geoghegan <[email protected]>
date : Tue, 2 Mar 2021 14:06:07 -0800
Add documenting assertion. This makes it easier to follow how we
maintain the top parent link in target subtree's half-dead/leaf level
page.
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtxlog.c
Fix nbtree page deletion error messages.
commit : 3d8d5787a358156edaa7782f0c88e231af974a01
author : Peter Geoghegan <[email protected]>
date : Tue, 2 Mar 2021 13:02:24 -0800
committer: Peter Geoghegan <[email protected]>
date : Tue, 2 Mar 2021 13:02:24 -0800
Adjust some "can't happen" error messages that assumed that the page
deletion target page must be a half-dead page. This assumption was
wrong in the case of an internal target page. Simply refer to these
pages as the target page instead.
Internal pages are never marked half-dead. There is exactly one
half-dead page for each subtree undergoing deletion. The half-dead page
is also the target subtree's leaf-level page. This has been the case
since commit efada2b8, which totally overhauled nbtree page deletion.
M src/backend/access/nbtree/nbtpage.c
Mark default_transaction_read_only as GUC_REPORT.
commit : d16f8c8e416d288bd4734ed5f14076b62ec8d153
author : Tom Lane <[email protected]>
date : Tue, 2 Mar 2021 13:53:46 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Mar 2021 13:53:46 -0500
This allows clients to find out the setting at connection time without
having to expend a query round trip to do so; which is helpful when
trying to identify read/write servers. (One must also look at
in_hot_standby, but that's already GUC_REPORT, cf bf8a662c9.)
Modifying libpq to make use of this will come soon, but I felt it
cleaner to push the server change separately.
Haribabu Kommi, Greg Nancarrow, Vignesh C; reviewed at various times
by Laurenz Albe, Takayuki Tsunakawa, Peter Smith.
Discussion: https://postgr.es/m/CAF3+xM+8-ztOkaV9gHiJ3wfgENTq97QcjXQt+rbFQ6F7oNzt9A@mail.gmail.com
M doc/src/sgml/libpq.sgml
M doc/src/sgml/protocol.sgml
M src/backend/utils/misc/guc.c
Use native path separators to pg_ctl in initdb
commit : 75dbfe4ca70d5b7a83f61b0a66a0a14ad0b739ed
author : Alvaro Herrera <[email protected]>
date : Tue, 2 Mar 2021 15:39:34 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 2 Mar 2021 15:39:34 -0300
On Windows, CMD.EXE allegedly does not run a command that uses forward slashes,
so let's convert the path to use backslashes instead.
Backpatch to 10.
Author: Nitin Jadhav <[email protected]>
Reviewed-by: Juan José Santamaría Flecha <[email protected]>
Discussion: https://postgr.es/m/CAMm1aWaNDuaPYFYMAqDeJrZmPtNvLcJRS++CcZWY8LT6KcoBZw@mail.gmail.com
M src/bin/initdb/initdb.c
Suppress unnecessary regex subre nodes in a couple more cases.
commit : 4604f83fdfe030a2f1984159ede5270c1d825310
author : Tom Lane <[email protected]>
date : Tue, 2 Mar 2021 12:14:14 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Mar 2021 12:14:14 -0500
This extends the changes made in commit cebc1d34e, teaching
parseqatom() to generate fewer or cheaper subre nodes in some edge
cases. The case of interest here is a quantified atom that is "messy"
only because it has greediness opposite to what preceded it (whereas
captures and backrefs are intrinsically messy). In this case we don't
need an iteration node, since we don't care where the sub-matches of
the quantifier are; and we might also not need a second concatenation
node. This seems of only marginal real-world use according to my
testing, but I wanted to get it in before wrapping up this series of
regex performance fixes.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regcomp.c
Improve performance of regular expression back-references.
commit : 0c3405cf11a12da1a4278c6833f4d979fe06c866
author : Tom Lane <[email protected]>
date : Tue, 2 Mar 2021 11:55:12 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Mar 2021 11:55:12 -0500
In some cases, at the time that we're doing an NFA-based precheck
of whether a backref subexpression can match at a particular place
in the string, we already know which substring the referenced
subexpression matched. If so, we might as well forget about the NFA
and just compare the substring; this is faster and it gives an exact
rather than approximate answer.
In general, this optimization can help while we are prechecking within
the second child expression of a concat node, while the capture was
within the first child expression; then the substring was saved during
cdissect() of the first child and will be available to NFA checks done
while cdissect() recurses into the second child. It can help quite a
lot if the tree looks like
concat
/ \
capture concat
/ \
expensive stuff backref
as we will be able to avoid recursively dissecting the "expensive
stuff" before discovering that the backref isn't satisfied with a
particular midpoint that the lower concat node is testing. This
doesn't help if the concat tree is left-deep, as the capture node
won't get set soon enough (and it's hard to fix that without changing
the engine's match behavior). Fortunately, right-deep concat trees
are the common case.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/rege_dfa.c
M src/backend/regex/regexec.c
Fix semantics of regular expression back-references.
commit : 4aea704a5bfd4b5894a268499369ccab89940c9c
author : Tom Lane <[email protected]>
date : Tue, 2 Mar 2021 11:34:53 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Mar 2021 11:34:53 -0500
POSIX defines the behavior of back-references thus:
The back-reference expression '\n' shall match the same (possibly
empty) string of characters as was matched by a subexpression
enclosed between "\(" and "\)" preceding the '\n'.
As far as I can see, the back-reference is supposed to consider only
the data characters matched by the referenced subexpression. However,
because our engine copies the NFA constructed from the referenced
subexpression, it effectively enforces any constraints therein, too.
As an example, '(^.)\1' ought to match 'xx', or any other string
starting with two occurrences of the same character; but in our code
it does not, and indeed can't match anything, because the '^' anchor
constraint is included in the backref's copied NFA. If POSIX intended
that, you'd think they'd mention it. Perl for one doesn't act that
way, so it's hard to conclude that this isn't a bug.
Fix by modifying the backref's NFA immediately after it's copied from
the reference, replacing all constraint arcs by EMPTY arcs so that the
constraints are treated as automatically satisfied. This still allows
us to enforce matching rules that depend only on the data characters;
for example, in '(^\d+).*\1' the NFA matching step will still know
that the backref can only match strings of digits.
Perhaps surprisingly, this change does not affect the results of any
of a rather large corpus of real-world regexes. Nonetheless, I would
not consider back-patching it, since it's a clear compatibility break.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Fix duplicated test case in TAP tests of reindexdb
commit : c5530d8474a482d32c0d9bb099707d9a8e117f96
author : Michael Paquier <[email protected]>
date : Tue, 2 Mar 2021 13:18:06 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 2 Mar 2021 13:18:06 +0900
The same test for REINDEX (VERBOSE) was done twice, while it is clear
that the second test should use --concurrently. Issue introduced in
5dc92b8, for what looks like a copy-paste mistake.
Reviewed-by: Mark Dilger
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M src/bin/scripts/t/090_reindexdb.pl
Simplify code to switch pg_class.relrowsecurity in tablecmds.c
commit : fabde52fab976ff782d77c80b8e2614f94f9dce1
author : Michael Paquier <[email protected]>
date : Tue, 2 Mar 2021 12:30:21 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 2 Mar 2021 12:30:21 +0900
The same code pattern was repeated twice to enable or disable ROW LEVEL
SECURITY with an ALTER TABLE command. This makes the code slightly
cleaner.
Author: Justin Pryzby
Reviewed-by: Zhihong Yu
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
doc: Improve description of data checksums
commit : bd1b8d0ef2eb5f2a8765fbb3b74e0262c9983942
author : Michael Paquier <[email protected]>
date : Tue, 2 Mar 2021 10:50:13 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 2 Mar 2021 10:50:13 +0900
This partially reverts bcf2667 that got incorrectly merged, and this
improves the wording of the documentation that existed before that.
Per discussion with Justin Pryzby.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/wal.sgml
doc: Mention archive_command failure handling on signals
commit : 8c1b6a186d420de863d1334c8917dcd047a7845a
author : Michael Paquier <[email protected]>
date : Tue, 2 Mar 2021 10:25:47 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 2 Mar 2021 10:25:47 +0900
The behavior is similar to restore_command, which was already documented
for the restore part, but not the archive part.
Author: Benoit Lobréau
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAPE8EZ7akCzc1hWohA4AcbmKtHh9rcWAB5MStOeZD2+9jC+hLQ@mail.gmail.com
M doc/src/sgml/backup.sgml
Improve reporting for syntax errors in multi-line JSON data.
commit : ffd3944ab9d481906137bc7d20f5325a2bd68acc
author : Tom Lane <[email protected]>
date : Mon, 1 Mar 2021 16:44:17 -0500
committer: Tom Lane <[email protected]>
date : Mon, 1 Mar 2021 16:44:17 -0500
Point to the specific line where the error was detected; the
previous code tended to include several preceding lines as well.
Avoid re-scanning the entire input to recompute which line that
was. Simplify the logic a bit. Add test cases.
Simon Riggs and Hamid Akhtar, reviewed by Daniel Gustafsson and myself
Discussion: https://postgr.es/m/CANbhV-EPBnXm3MF_TTWBwwqgn1a1Ghmep9VHfqmNBQ8BT0f+_g@mail.gmail.com
M src/backend/utils/adt/jsonfuncs.c
M src/common/jsonapi.c
M src/include/common/jsonapi.h
M src/test/regress/expected/json.out
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/json.sql
M src/test/regress/sql/jsonb.sql
Remove obsolete comment for WaitForProcSignalBarrier().
commit : bd69ddfcdbf650f11af4317f3f6686c012cf66d0
author : Thomas Munro <[email protected]>
date : Tue, 2 Mar 2021 09:15:15 +1300
committer: Thomas Munro <[email protected]>
date : Tue, 2 Mar 2021 09:15:15 +1300
Commit 814f1d8b removed the behavior described.
Reported-by: Amit Kapila <[email protected]>
M src/backend/storage/ipc/procsignal.c
Fix recovery test hang in 021_row_visibility.pl on windows.
commit : 1e6e40447115ca7b4749d7d117b81b016ee5e2c2
author : Andres Freund <[email protected]>
date : Mon, 1 Mar 2021 09:52:15 -0800
committer: Andres Freund <[email protected]>
date : Mon, 1 Mar 2021 09:52:15 -0800
The psql processes were not explicitly killed (but would eventually
exit due postgres shutting down). For some reason windows perl doesn't
like that, resulting in errors like
Warning: unable to close filehandle GEN20 properly: Bad file descriptor during global destruction.
The test was introduced in d6734a897e3, so no backpatching necessary.
M src/test/recovery/t/021_row_visibility.pl
Allow condition variables to be used in interrupt code.
commit : f5a5773a9dc4185414fe538525e20d8512c2ba35
author : Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 16:28:12 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 16:28:12 +1300
Adjust the condition variable sleep loop to work correctly when code
reached by its internal CHECK_FOR_INTERRUPTS() call interacts with
another condition variable.
There are no such cases currently, but a proposed patch would do this.
Discussion: https://postgr.es/m/CA+hUKGLdemy2gBm80kz20GTe6hNVwoErE8KwcJk6-U56oStjtg@mail.gmail.com
M src/backend/storage/lmgr/condition_variable.c
Use condition variables for ProcSignalBarriers.
commit : 814f1d8bc395bb9b4c0ae090ceea60adfbeb8e49
author : Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 16:26:22 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 16:26:22 +1300
Instead of a poll/sleep loop, use a condition variable for precise
wake-up whenever a backend's pss_barrierGeneration advances.
Discussion: https://postgr.es/m/CA+hUKGLdemy2gBm80kz20GTe6hNVwoErE8KwcJk6-U56oStjtg@mail.gmail.com
M src/backend/storage/ipc/procsignal.c
Avoid repeated decoding of prepared transactions after a restart.
commit : 8bdb1332eb51837c15a10a972c179b84f654279e
author : Amit Kapila <[email protected]>
date : Mon, 1 Mar 2021 09:11:18 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 1 Mar 2021 09:11:18 +0530
In commit a271a1b50e, we allowed decoding at prepare time and the prepare
was decoded again if there is a restart after decoding it. It was done
that way because we can't distinguish between the cases where we have not
decoded the prepare because it was prior to consistent snapshot or we have
decoded it earlier but restarted. To distinguish between these two cases,
we have introduced an initial_consistent_point at the slot level which is
an LSN at which we found a consistent point at the time of slot creation.
This is also the point where we have exported a snapshot for the initial
copy. So, prepare transaction prior to this point are sent along with
commit prepared.
This commit bumps SNAPBUILD_VERSION because of change in SnapBuild. It
will break existing slots which is fine in a major release.
Author: Ajin Cherian, based on idea by Andres Freund
Reviewed-by: Amit Kapila and Vignesh C
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/expected/twophase.out
M contrib/test_decoding/expected/twophase_stream.out
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/include/replication/reorderbuffer.h
M src/include/replication/slot.h
M src/include/replication/snapbuild.h
Use FeBeWaitSet for walsender.c.
commit : 6230912f23904aa6cb2a1f948ca9b08235b4f54a
author : Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 15:17:28 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 15:17:28 +1300
This avoids the need to set up and tear down a fresh WaitEventSet every
time we need need to wait. We have to add an explicit exit on
postmaster exit (FeBeWaitSet isn't set up to do that automatically), so
move the code to do that into a new function to avoid repetition.
Reviewed-by: Kyotaro Horiguchi <[email protected]> (earlier version)
Discussion: https://postgr.es/m/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw%40mail.gmail.com
M src/backend/replication/walsender.c
Introduce symbolic names for FeBeWaitSet positions.
commit : a042ba2ba7bb44f191f6f9398da27f04a958a0db
author : Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 15:16:56 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 15:16:56 +1300
Previously we used 0 and 1 to refer to the socket and latch in far flung
parts of the tree, without any explanation. Also use PGINVALID_SOCKET
rather than -1 in a couple of places that didn't already do that.
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw%40mail.gmail.com
M src/backend/libpq/be-secure.c
M src/backend/libpq/pqcomm.c
M src/backend/utils/init/miscinit.c
M src/include/libpq/libpq.h
Update docs of logical replication for commit ce0fdbfe97.
commit : cf54e04b9e29a13c5e7c2386c5e2ff785e830414
author : Amit Kapila <[email protected]>
date : Mon, 1 Mar 2021 08:23:41 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 1 Mar 2021 08:23:41 +0530
Forgot to update the logical replication configuration settings page.
After commit ce0fdbfe97, table synchronization workers also started using
replication origins to track the progress and the same should be reflected
in docs.
Author: Amit Kapila
Discussion: https://postgr.es/m/CAA4eK1KkbppndxxRKbaT2sXrLkdPwy44F4pjEZ0EDrVjD9MPjQ@mail.gmail.com
M doc/src/sgml/logical-replication.sgml
Update the docs and comments for decoding of prepared xacts.
commit : b4e3dc7fd420cdc2287f30a638c4affd071f01b2
author : Amit Kapila <[email protected]>
date : Mon, 1 Mar 2021 08:14:33 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 1 Mar 2021 08:14:33 +0530
Commit a271a1b50e introduced decoding at prepare time in ReorderBuffer.
This can lead to deadlock for out-of-core logical replication solutions
that uses this feature to build distributed 2PC in case such transactions
lock [user] catalog tables exclusively. They need to inform users to not
have locks on catalog tables (via explicit LOCK command) in such
transactions.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/decode.c
Use EVFILT_SIGNAL for kqueue latches.
commit : 6148656a0be1c6245fbcfcbbeb87541f1b173162
author : Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 12:28:43 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 12:28:43 +1300
Cut down on system calls and other overheads by waiting for SIGURG
explicitly with kqueue instead of using a signal handler and self-pipe.
Affects *BSD and macOS systems.
This leaves only the poll implementation with a signal handler and the
traditional self-pipe trick.
Discussion: https://postgr.es/m/CA+hUKGJjxPDpzBE0a3hyUywBvaZuC89yx3jK9RFZgfv_KHU7gg@mail.gmail.com
M src/backend/storage/ipc/latch.c
Use signalfd(2) for epoll latches.
commit : 6a2a70a02018d6362f9841cc2f499cc45405e86b
author : Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 12:06:09 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 12:06:09 +1300
Cut down on system calls and other overheads by reading from a signalfd
instead of using a signal handler and self-pipe. Affects Linux sytems,
and possibly others including illumos that implement the Linux epoll and
signalfd interfaces.
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CA+hUKGJjxPDpzBE0a3hyUywBvaZuC89yx3jK9RFZgfv_KHU7gg@mail.gmail.com
M src/backend/libpq/pqsignal.c
M src/backend/storage/ipc/latch.c
M src/backend/utils/init/miscinit.c
Use SIGURG rather than SIGUSR1 for latches.
commit : 83709a0d5a46559db016c50ded1a95fd3b0d3be6
author : Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 11:58:10 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 11:58:10 +1300
Traditionally, SIGUSR1 has been overloaded for ad-hoc signals,
procsignal.c signals and latch.c wakeups. Move that last use over to a
new dedicated signal. SIGURG is normally used to report out-of-band
socket data, but PostgreSQL doesn't use that facility.
The signal handler is now installed in all postmaster children by
InitializeLatchSupport(). Those wishing to disconnect from it should
call ShutdownLatchSupport().
Future patches will use this separation of signals to avoid the need for
a signal handler on some operating systems.
Discussion: https://postgr.es/m/CA+hUKGJjxPDpzBE0a3hyUywBvaZuC89yx3jK9RFZgfv_KHU7gg@mail.gmail.com
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/postmaster.c
M src/backend/storage/ipc/latch.c
M src/backend/storage/ipc/procsignal.c
M src/include/storage/latch.h
Optimize latches to send fewer signals.
commit : c8f3bc2401e7df7b79bae39dd3511c91f825b6a4
author : Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 11:51:15 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 11:51:15 +1300
Don't send signals to processes that aren't sleeping.
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CA+hUKGJjxPDpzBE0a3hyUywBvaZuC89yx3jK9RFZgfv_KHU7gg@mail.gmail.com
M src/backend/storage/ipc/latch.c
M src/include/storage/latch.h
Remove latch.c workaround for Linux < 2.6.27.
commit : d1b90995e8d41df7e59efe48e98f26cd66baba9b
author : Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 10:44:21 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 1 Mar 2021 10:44:21 +1300
Commit 82ebbeb0 added a workaround for systems with no epoll_create1()
and EPOLL_CLOEXEC. Linux < 2.6.27 and glibc < 2.9 are long gone. Now
seems like a good time to drop the extra code, because otherwise we'd
have to add similar already-dead workaround code to new patches using
XXX_CLOEXEC flags that arrived in the same kernel release.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGKL_%3DaO%3Dr30N%3Ds9VoDgTqHpRSzePRbA9dkYO7snc7HsxA%40mail.gmail.com
M src/backend/storage/ipc/latch.c
pgbench: Remove now-dead CState->ecnt
commit : 943eb478803cc2ed6f0b1f5df6f1b20c9ad6245d
author : Michael Paquier <[email protected]>
date : Sun, 28 Feb 2021 07:50:26 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 28 Feb 2021 07:50:26 +0900
The last use of ecnt was in 12788ae. It was getting incremented after a
backend error without any purpose since then, so let's get rid of it.
Author: Kota Miyake
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/[email protected]
M src/bin/pgbench/pgbench.c
Fix use-after-free bug with AfterTriggersTableData.storeslot
commit : 25936fd46c97039aad042ae8d46917d38d132fe4
author : Alvaro Herrera <[email protected]>
date : Sat, 27 Feb 2021 18:09:15 -0300
committer: Alvaro Herrera <[email protected]>
date : Sat, 27 Feb 2021 18:09:15 -0300
AfterTriggerSaveEvent() wrongly allocates the slot in execution-span
memory context, whereas the correct thing is to allocate it in
a transaction-span context, because that's where the enclosing
AfterTriggersTableData instance belongs into.
Backpatch to 12 (the test back to 11, where it works well with no code
changes, and it's good to have to confirm that the case was previously
well supported); this bug seems introduced by commit ff11e7f4b9ae.
Reported-by: Bertrand Drouvot <[email protected]>
Author: Amit Langote <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Raise a timeout to 180s, in contrib/test_decoding.
commit : 388b959315205b0b65efb074ec84e1d7fad62402
author : Noah Misch <[email protected]>
date : Sat, 27 Feb 2021 07:02:56 -0800
committer: Noah Misch <[email protected]>
date : Sat, 27 Feb 2021 07:02:56 -0800
Per buildfarm member hornet. The test is new in v14, so no back-patch.
M contrib/test_decoding/expected/twophase.out
M contrib/test_decoding/sql/twophase.sql
Add missing TidRangeScan readfunc
commit : 977b2c08535f2a82ba7c310c88420cbbca1772e8
author : David Rowley <[email protected]>
date : Sat, 27 Feb 2021 23:21:21 +1300
committer: David Rowley <[email protected]>
date : Sat, 27 Feb 2021 23:21:21 +1300
Mistakenly forgotten in bb437f995
M src/backend/nodes/readfuncs.c
Add TID Range Scans to support efficient scanning ranges of TIDs
commit : bb437f995d47405ecd92cf66df71f7f7e40ed460
author : David Rowley <[email protected]>
date : Sat, 27 Feb 2021 22:59:36 +1300
committer: David Rowley <[email protected]>
date : Sat, 27 Feb 2021 22:59:36 +1300
This adds a new executor node named TID Range Scan. The query planner
will generate paths for TID Range scans when quals are discovered on base
relations which search for ranges on the table's ctid column. These
ranges may be open at either end. For example, WHERE ctid >= '(10,0)';
will return all tuples on page 10 and over.
To support this, two new optional callback functions have been added to
table AM. scan_set_tidrange is used to set the scan range to just the
given range of TIDs. scan_getnextslot_tidrange fetches the next tuple
in the given range.
For AMs were scanning ranges of TIDs would not make sense, these functions
can be set to NULL in the TableAmRoutine. The query planner won't
generate TID Range Scan Paths in that case.
Author: Edmund Horner, David Rowley
Reviewed-by: David Rowley, Tomas Vondra, Tom Lane, Andres Freund, Zhihong Yu
Discussion: https://postgr.es/m/CAMyN-kB-nFTkF=VA_JPwFNo08S0d-Yk0F741S2B7LDmYAi8eyA@mail.gmail.com
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/commands/explain.c
M src/backend/executor/Makefile
M src/backend/executor/execAmi.c
M src/backend/executor/execCurrent.c
M src/backend/executor/execProcnode.c
A src/backend/executor/nodeTidrangescan.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/README
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/tidpath.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/plancat.c
M src/backend/optimizer/util/relnode.c
M src/backend/storage/page/itemptr.c
M src/include/access/heapam.h
M src/include/access/relscan.h
M src/include/access/tableam.h
M src/include/catalog/pg_operator.dat
A src/include/executor/nodeTidrangescan.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
M src/include/optimizer/cost.h
M src/include/optimizer/pathnode.h
M src/include/storage/itemptr.h
A src/test/regress/expected/tidrangescan.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/tidrangescan.sql
Enhanced cycle mark values
commit : f4adc41c4f92cc91d507b19e397140c35bb9fd71
author : Peter Eisentraut <[email protected]>
date : Sat, 27 Feb 2021 08:11:14 +0100
committer: Peter Eisentraut <[email protected]>
date : Sat, 27 Feb 2021 08:11:14 +0100
Per SQL:202x draft, in the CYCLE clause of a recursive query, the
cycle mark values can be of type boolean and can be omitted, in which
case they default to TRUE and FALSE.
Reviewed-by: Vik Fearing <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/queries.sgml
M doc/src/sgml/ref/select.sgml
M src/backend/catalog/sql_features.txt
M src/backend/parser/gram.y
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Doc: further clarify libpq's description of connection string URIs.
commit : 4e90052c46c7751779ed83627676ed5e74ebe6d4
author : Tom Lane <[email protected]>
date : Fri, 26 Feb 2021 15:24:00 -0500
committer: Tom Lane <[email protected]>
date : Fri, 26 Feb 2021 15:24:00 -0500
Break the synopsis into named parts to make it less confusing.
Make more than zero effort at applying SGML markup. Do a bit
of copy-editing of nearby text.
The synopsis revision is by Alvaro Herrera and Paul Förster,
the rest is my fault. Back-patch to v10 where multi-host
connection strings appeared.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
Improve memory management in regex compiler.
commit : 0fc1af174cf7113445e116feb2813405b838a47d
author : Tom Lane <[email protected]>
date : Fri, 26 Feb 2021 13:52:10 -0500
committer: Tom Lane <[email protected]>
date : Fri, 26 Feb 2021 13:52:10 -0500
The previous logic here created a separate pool of arcs for each
state, so that the out-arcs of each state were physically stored
within it. Perhaps this choice was driven by trying to not include
a "from" pointer within each arc; but Spencer gave up on that idea
long ago, and it's hard to see what the value is now. The approach
turns out to be fairly disastrous in terms of memory consumption,
though. In the first place, NFAs built by this engine seem to have
about 4 arcs per state on average, with a majority having only one
or two out-arcs. So pre-allocating 10 out-arcs for each state is
already cause for a factor of two or more bloat. Worse, the NFA
optimization phase moves arcs around with abandon. In a large NFA,
some of the states will have hundreds of out-arcs, so towards the
end of the optimization phase we have a significant number of states
whose arc pools have room for hundreds of arcs each, even though only
a few of those arcs are in use. We have seen real-world regexes in
which this effect bloats the memory requirement by 25X or even more.
Hence, get rid of the per-state arc pools in favor of a single arc
pool for the whole NFA, with variable-sized allocation batches
instead of always asking for 10 at a time. While we're at it,
let's batch the allocations of state structs too, to further reduce
the malloc traffic.
This incidentally allows moveouts() to be optimized in a similar
way to moveins(): when moving an arc to another state, it's now
valid to just re-link the same arc struct into a different outchain,
where before the code invariants required us to make a physically
new arc and then free the old one.
These changes reduce the regex compiler's typical space consumption
for average-size regexes by about a factor of two, and much more for
large or complicated regexes. In a large test set of real-world
regexes, we formerly had half a dozen cases that failed with "regular
expression too complex" due to exceeding the REG_MAX_COMPILE_SPACE
limit (about 150MB); we would have had to raise that limit to
something close to 400MB to make them work with the old code. Now,
none of those cases need more than 13MB to compile. Furthermore,
the test set is about 10% faster overall due to less malloc traffic.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/include/regex/regguts.h
Extend a test case a little
commit : b3a9e9897ec702d56602b26a8cdc0950f23b29dc
author : Peter Eisentraut <[email protected]>
date : Fri, 26 Feb 2021 09:11:15 +0100
committer: Peter Eisentraut <[email protected]>
date : Fri, 26 Feb 2021 09:11:15 +0100
This will possibly help a subsequent patch by making sure the notice
messages are distinct so that it's clear that they come out in the
right order.
Author: Fabien COELHO <[email protected]>
Discussion: https://www.postgresql.org/message-id/alpine.DEB.2.21.1904240654120.3407%40lancre
M src/test/regress/expected/copydml.out
M src/test/regress/sql/copydml.sql
doc: Improve {archive,restore}_command for compressed logs
commit : 329784e11849ff691f0157f3b27c50f652bce76a
author : Michael Paquier <[email protected]>
date : Fri, 26 Feb 2021 14:39:03 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 26 Feb 2021 14:39:03 +0900
The commands mentioned in the docs with gzip and gunzip did not prefix
the archives with ".gz" and used inconsistent paths for the archives,
which can be confusing.
Reported-by: Philipp Gramzow
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/backup.sgml
Revert "pg_collation_actual_version() -> pg_collation_current_version()."
commit : 8556267b2b1b8e1c26037c4c25cf390ee5afb5d9
author : Thomas Munro <[email protected]>
date : Fri, 26 Feb 2021 15:29:27 +1300
committer: Thomas Munro <[email protected]>
date : Fri, 26 Feb 2021 15:29:27 +1300
This reverts commit 9cf184cc0599b6e65e7e5ecd9d91cd42e278bcd8. Name
change less well received than anticipated.
Discussion: https://postgr.es/m/afcfb97e-88a1-a540-db95-6c573b93bc2b%40eisentraut.org
M doc/src/sgml/func.sgml
M src/backend/commands/collationcmds.c
M src/backend/utils/adt/pg_locale.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Fix list-manipulation bug in WITH RECURSIVE processing.
commit : 80ca8464fe02296c8efefd53746e6d6a3f456d1e
author : Tom Lane <[email protected]>
date : Thu, 25 Feb 2021 20:47:32 -0500
committer: Tom Lane <[email protected]>
date : Thu, 25 Feb 2021 20:47:32 -0500
makeDependencyGraphWalker and checkWellFormedRecursionWalker
thought they could hold onto a pointer to a list's first
cons cell while the list was modified by recursive calls.
That was okay when the cons cell was actually separately
palloc'd ... but since commit 1cff1b95a, it's quite unsafe,
leading to core dumps or incorrect complaints of faulty
WITH nesting.
In the field this'd require at least a seven-deep WITH nest
to cause an issue, but enabling DEBUG_LIST_MEMORY_USAGE
allows the bug to be seen with lesser nesting depths.
Per bug #16801 from Alexander Lakhin. Back-patch to v13.
Michael Paquier and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/parse_cte.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
VACUUM VERBOSE: Count "newly deleted" index pages.
commit : 2376361839091b0dcdcc0b77f938b809b5f21646
author : Peter Geoghegan <[email protected]>
date : Thu, 25 Feb 2021 14:32:18 -0800
committer: Peter Geoghegan <[email protected]>
date : Thu, 25 Feb 2021 14:32:18 -0800
Teach VACUUM VERBOSE to report on pages deleted by the _current_ VACUUM
operation -- these are newly deleted pages. VACUUM VERBOSE continues to
report on the total number of deleted pages in the entire index (no
change there). The former is a subset of the latter.
The distinction between each category of deleted index page only arises
with index AMs where page deletion is supported and is decoupled from
page recycling for performance reasons.
This is follow-up work to commit e5d8a999, which made nbtree store
64-bit XIDs (not 32-bit XIDs) in pages at the point at which they're
deleted. Note that the btm_last_cleanup_num_delpages metapage field
added by that commit usually gets set to pages_newly_deleted. The
exceptions (the scenarios in which they're not equal) all seem to be
tricky cases for the implementation (of page deletion and recycling) in
general.
Author: Peter Geoghegan <[email protected]>
Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX%3Dd5u-tRYhi-F4wnV2uN2zHpMUXw%40mail.gmail.com
M src/backend/access/gin/ginvacuum.c
M src/backend/access/gist/gistvacuum.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/spgist/spgvacuum.c
M src/include/access/genam.h
M src/include/access/nbtree.h
Doc: remove src/backend/regex/re_syntax.n.
commit : 301ed8812e3f8b102b85e1f5a59e95990ed9a868
author : Tom Lane <[email protected]>
date : Thu, 25 Feb 2021 13:33:27 -0500
committer: Tom Lane <[email protected]>
date : Thu, 25 Feb 2021 13:33:27 -0500
We aren't publishing this file as documentation, and it's been
much more haphazardly maintained than the real docs in func.sgml,
so let's just drop it. I think the only reason I included it in
commit 7bcc6d98f was that the Berkeley-era sources had had a man
page in this directory.
Discussion: https://postgr.es/m/[email protected]
D src/backend/regex/re_syntax.n
Change regex \D and \W shorthands to always match newlines.
commit : 7dc13a0f0805a353cea0455ed95701322b39d4dd
author : Tom Lane <[email protected]>
date : Thu, 25 Feb 2021 13:29:06 -0500
committer: Tom Lane <[email protected]>
date : Thu, 25 Feb 2021 13:29:06 -0500
Newline is certainly not a digit, nor a word character, so it is
sensible that it should match these complemented character classes.
Previously, \D and \W acted that way by default, but in
newline-sensitive mode ('n' or 'p' flag) they did not match newlines.
This behavior was previously forced because explicit complemented
character classes don't match newlines in newline-sensitive mode;
but as of the previous commit that implementation constraint no
longer exists. It seems useful to change this because the primary
real-world use for newline-sensitive mode seems to be to match the
default behavior of other regex engines such as Perl and Javascript
... and their default behavior is that these match newlines.
The old behavior can be kept by writing an explicit complemented
character class, i.e. [^[:digit:]] or [^[:word:]]. (This means
that \D and \W are not exactly equivalent to those strings, but
they weren't anyway.)
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/regex/re_syntax.n
M src/backend/regex/regcomp.c
M src/test/modules/test_regex/expected/test_regex.out
Allow complemented character class escapes within regex brackets.
commit : 2a0af7fe460eb46f9af996075972bf7c2e3f211d
author : Tom Lane <[email protected]>
date : Thu, 25 Feb 2021 13:00:40 -0500
committer: Tom Lane <[email protected]>
date : Thu, 25 Feb 2021 13:00:40 -0500
The complement-class escapes \D, \S, \W are now allowed within
bracket expressions. There is no semantic difficulty with doing
that, but the rather hokey macro-expansion-based implementation
previously used here couldn't cope.
Also, invent "word" as an allowed character class name, thus "\w"
is now equivalent to "[[:word:]]" outside brackets, or "[:word:]"
within brackets. POSIX allows such implementation-specific
extensions, and the same name is used in e.g. bash.
One surprising compatibility issue this raises is that constructs
such as "[\w-_]" are now disallowed, as our documentation has always
said they should be: character classes can't be endpoints of a range.
Previously, because \w was just a macro for "[:alnum:]_", such a
construct was read as "[[:alnum:]_-_]", so it was accepted so long as
the character after "-" was numerically greater than or equal to "_".
Some implementation cleanup along the way:
* Remove the lexnest() hack, and in consequence clean up wordchrs()
to not interact with the lexer.
* Fix colorcomplement() to not be O(N^2) in the number of colors
involved.
* Get rid of useless-as-far-as-I-can-see calls of element()
on single-character character element names in brackpart().
element() always maps these to the character itself, and things
would be quite broken if it didn't --- should "[a]" match something
different than "a" does? Besides, the shortcut path in brackpart()
wasn't doing this anyway, making it even more inconsistent.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/regex/re_syntax.n
M src/backend/regex/regc_color.c
M src/backend/regex/regc_lex.c
M src/backend/regex/regc_locale.c
M src/backend/regex/regc_pg_locale.c
M src/backend/regex/regcomp.c
M src/include/regex/regguts.h
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Improve tab-completion for TRUNCATE.
commit : 6b40d9bdbdc9f873868b0ddecacd9a307fc8ee26
author : Fujii Masao <[email protected]>
date : Thu, 25 Feb 2021 18:20:57 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 25 Feb 2021 18:20:57 +0900
Author: Kota Miyake
Reviewed-by: Muhammad Usama
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/tab-complete.c
doc: Mention PGDATABASE as supported by pgbench
commit : a6f8dc47a0582e37c234c8e916377b46d2b03817
author : Michael Paquier <[email protected]>
date : Thu, 25 Feb 2021 16:06:54 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 25 Feb 2021 16:06:54 +0900
PGHOST, PGPORT and PGUSER were already mentioned, but not PGDATABASE.
Like 5aaa584, backpatch down to 12.
Reported-by: Christophe Courtois
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M doc/src/sgml/ref/pgbench.sgml
Use full 64-bit XIDs in deleted nbtree pages.
commit : e5d8a999030418a1b9e53d5f15ccaca7ed674877
author : Peter Geoghegan <[email protected]>
date : Wed, 24 Feb 2021 18:41:34 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 24 Feb 2021 18:41:34 -0800
Otherwise we risk "leaking" deleted pages by making them non-recyclable
indefinitely. Commit 6655a729 did the same thing for deleted pages in
GiST indexes. That work was used as a starting point here.
Stop storing an XID indicating the oldest bpto.xact across all deleted
though unrecycled pages in nbtree metapages. There is no longer any
reason to care about that condition/the oldest XID. It only ever made
sense when wraparound was something _bt_vacuum_needs_cleanup() had to
consider.
The btm_oldest_btpo_xact metapage field has been repurposed and renamed.
It is now btm_last_cleanup_num_delpages, which is used to remember how
many non-recycled deleted pages remain from the last VACUUM (in practice
its value is usually the precise number of pages that were _newly
deleted_ during the specific VACUUM operation that last set the field).
The general idea behind storing btm_last_cleanup_num_delpages is to use
it to give _some_ consideration to non-recycled deleted pages inside
_bt_vacuum_needs_cleanup() -- though never too much. We only really
need to avoid leaving a truly excessive number of deleted pages in an
unrecycled state forever. We only do this to cover certain narrow cases
where no other factor makes VACUUM do a full scan, and yet the index
continues to grow (and so actually misses out on recycling existing
deleted pages).
These metapage changes result in a clear user-visible benefit: We no
longer trigger full index scans during VACUUM operations solely due to
the presence of only 1 or 2 known deleted (though unrecycled) blocks
from a very large index. All that matters now is keeping the costs and
benefits in balance over time.
Fix an issue that has been around since commit 857f9c36, which added the
"skip full scan of index" mechanism (i.e. the _bt_vacuum_needs_cleanup()
logic). The accuracy of btm_last_cleanup_num_heap_tuples accidentally
hinged upon _when_ the source value gets stored. We now always store
btm_last_cleanup_num_heap_tuples in btvacuumcleanup(). This fixes the
issue because IndexVacuumInfo.num_heap_tuples (the source field) is
expected to accurately indicate the state of the table _after_ the
VACUUM completes inside btvacuumcleanup().
A backpatchable fix cannot easily be extracted from this commit. A
targeted fix for the issue will follow in a later commit, though that
won't happen today.
I (pgeoghegan) have chosen to remove any mention of deleted pages in the
documentation of the vacuum_cleanup_index_scale_factor GUC/param, since
the presence of deleted (though unrecycled) pages is no longer of much
concern to users. The vacuum_cleanup_index_scale_factor description in
the docs now seems rather unclear in any case, and it should probably be
rewritten in the near future. Perhaps some passing mention of page
deletion will be added back at the same time.
Bump XLOG_PAGE_MAGIC due to nbtree WAL records using full XIDs now.
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX=d5u-tRYhi-F4wnV2uN2zHpMUXw@mail.gmail.com
M contrib/amcheck/verify_nbtree.c
M contrib/pageinspect/btreefuncs.c
M contrib/pageinspect/expected/btree.out
M contrib/pageinspect/pageinspect–1.8–1.9.sql
M contrib/pgstattuple/pgstatindex.c
M doc/src/sgml/config.sgml
M doc/src/sgml/pageinspect.sgml
M src/backend/access/gist/gistxlog.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/rmgrdesc/nbtdesc.c
M src/backend/storage/ipc/standby.c
M src/include/access/nbtree.h
M src/include/access/nbtxlog.h
M src/include/access/xlog_internal.h
M src/include/storage/standby.h
Fix relcache reference leak introduced by ce0fdbfe97.
commit : 8a4f9522d0c7d197c7eaa62cc72fb1bf9b90f05b
author : Amit Kapila <[email protected]>
date : Thu, 25 Feb 2021 07:48:24 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 25 Feb 2021 07:48:24 +0530
Author: Sawada Masahiko
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAD21AoA7ZEfsOXQ9HQqMv3QYGsEm2H5Wk5ic5S=mvzDf-3a3SA@mail.gmail.com
M src/backend/catalog/pg_subscription.c
Fix some typos, grammar and style in docs and comments
commit : bcf2667bf62d72faced64cb60ffbd2e599a0ebe8
author : Michael Paquier <[email protected]>
date : Wed, 24 Feb 2021 16:13:17 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 24 Feb 2021 16:13:17 +0900
The portions fixing the documentation are backpatched where needed.
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
backpatch-through: 9.6
M contrib/pageinspect/heapfuncs.c
M doc/src/sgml/charset.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/pageinspect.sgml
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/alter_subscription.sgml
M doc/src/sgml/ref/create_type.sgml
M doc/src/sgml/ref/drop_index.sgml
M doc/src/sgml/rules.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/common/heaptuple.c
M src/backend/access/transam/xlogutils.c
M src/backend/commands/amcmds.c
M src/backend/executor/nodeAgg.c
M src/backend/optimizer/prep/prepagg.c
M src/backend/storage/ipc/procarray.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/adt/selfuncs.c
M src/backend/utils/cache/catcache.c
M src/bin/pg_rewind/libpq_source.c
M src/bin/pgbench/pgbench.c
M src/include/pg_config_manual.h
M src/interfaces/ecpg/include/sqlda-native.h
M src/test/regress/expected/geometry.out
M src/test/regress/expected/tuplesort.out
M src/test/regress/sql/geometry.sql
M src/test/regress/sql/tuplesort.sql
Message style fix
commit : 8ec8fe0f31712c62b761da9ef6d32214e08340d1
author : Peter Eisentraut <[email protected]>
date : Wed, 24 Feb 2021 07:00:49 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 24 Feb 2021 07:00:49 +0100
Don't quote type name placeholders.
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/hash_part.out
doc: Improve description of wal_receiver_status_interval
commit : c82d59d64e127a5e743f9cf16537f4c7c120ce7b
author : Michael Paquier <[email protected]>
date : Wed, 24 Feb 2021 11:15:58 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 24 Feb 2021 11:15:58 +0900
This parameter description was previously confusing, telling that a
value of 0 disabled completely status updates. This is not true as
there are cases where an update is sent while ignoring this parameter
value. The documentation is improved to outline the difference of
treatment for scheduled status messages and when these are forced.
Reported-by: Dmitriy Kuzmin
Author: Michael Paquier
Reviewed-by: Euler Taveira
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
Fix confusion in comments about generate_gather_paths
commit : 5a65eacfdc69ee8c9d3ed92d838bc53628fbb7c6
author : Alvaro Herrera <[email protected]>
date : Tue, 23 Feb 2021 20:04:47 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 23 Feb 2021 20:04:47 -0300
d2d8a229bc58 introduced a new function generate_useful_gather_paths to
be used as a replacement for generate_gather_paths, but forgot to update
a couple of places that referenced the older function.
This is possibly not 100% complete (ref. create_ordered_paths), but it's
better than not changing anything.
Author: "Hou, Zhijie" <[email protected]>
Reviewed-by: Tomas Vondra <[email protected]>
Discussion: https://postgr.es/m/4ce1d5116fe746a699a6d29858c6a39a@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/plan/planner.c
Reinstate HEAP_XMAX_LOCK_ONLY|HEAP_KEYS_UPDATED as allowed
commit : 8deb6b38dc4c7a7fd4719ee45e4b00d62b27dffe
author : Alvaro Herrera <[email protected]>
date : Tue, 23 Feb 2021 17:30:21 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 23 Feb 2021 17:30:21 -0300
Commit 866e24d47db1 added an assert that HEAP_XMAX_LOCK_ONLY and
HEAP_KEYS_UPDATED cannot appear together, on the faulty assumption that
the latter necessarily referred to an update and not a tuple lock; but
that's wrong, because SELECT FOR UPDATE can use precisely that
combination, as evidenced by the amcheck test case added here.
Remove the Assert(), and also patch amcheck's verify_heapam.c to not
complain if the combination is found. Also, out of overabundance of
caution, update (across all branches) README.tuplock to be more explicit
about this.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Mahendra Singh Thalor <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>
Discussion: https://postgr.es/m/20210124061758.GA11756@nol
M contrib/amcheck/t/001_verify_heapam.pl
M contrib/amcheck/verify_heapam.c
M src/backend/access/heap/README.tuplock
M src/backend/access/heap/hio.c
Suppress compiler warning in new regex match-all detection code.
commit : 3db05e76f92846d4b54d7de251b0875cf1e23aa4
author : Tom Lane <[email protected]>
date : Tue, 23 Feb 2021 13:55:11 -0500
committer: Tom Lane <[email protected]>
date : Tue, 23 Feb 2021 13:55:11 -0500
gcc 10 is smart enough to notice that control could reach this
"hasmatch[depth]" assignment with depth < 0, but not smart enough
to know that that would require a badly broken NFA graph. Change
the assert() to a plain runtime test to shut it up.
Per report from Andres Freund.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regc_nfa.c
VACUUM: ignore indexing operations with CONCURRENTLY
commit : d9d076222f5b94a85e0e318339cfc44b8f26022d
author : Alvaro Herrera <[email protected]>
date : Tue, 23 Feb 2021 12:15:09 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 23 Feb 2021 12:15:09 -0300
As envisioned in commit c98763bf51bf, it is possible for VACUUM to
ignore certain transactions that are executing CREATE INDEX CONCURRENTLY
and REINDEX CONCURRENTLY for the purposes of computing Xmin; that's
because we know those transactions are not going to examine any other
tables, and are not going to execute anything else in the same
transaction. (Only operations on "safe" indexes can be ignored: those
on indexes that are neither partial nor expressional).
This is extremely useful in cases where CIC/RC can run for a very long
time, because that used to be a significant headache for concurrent
vacuuming of other tables.
Reviewed-by: Matthias van de Meent <[email protected]>
Reviewed-by: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_index.sgml
M doc/src/sgml/ref/reindex.sgml
M src/backend/storage/ipc/procarray.c
M src/backend/utils/misc/guc.c
Simplify printing of LSNs
commit : 6f6f284c7ee44264eb3e128e2bf54d9276711d11
author : Peter Eisentraut <[email protected]>
date : Tue, 23 Feb 2021 10:14:38 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 23 Feb 2021 10:14:38 +0100
Add a macro LSN_FORMAT_ARGS for use in printf-style printing of LSNs.
Convert all applicable code to use it.
Reviewed-by: Ashutosh Bapat <[email protected]>
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CAExHW5ub5NaTELZ3hJUCE6amuvqAtsSxc7O+uK7y4t9Rrk23cw@mail.gmail.com
M contrib/amcheck/verify_nbtree.c
M contrib/pageinspect/rawpage.c
M src/backend/access/heap/rewriteheap.c
M src/backend/access/rmgrdesc/replorigindesc.c
M src/backend/access/rmgrdesc/xactdesc.c
M src/backend/access/rmgrdesc/xlogdesc.c
M src/backend/access/transam/timeline.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogutils.c
M src/backend/replication/backup_manifest.c
M src/backend/replication/basebackup.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/origin.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/syncrep.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walsender.c
M src/backend/storage/ipc/standby.c
M src/backend/utils/adt/pg_lsn.c
M src/bin/pg_basebackup/pg_receivewal.c
M src/bin/pg_basebackup/pg_recvlogical.c
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_controldata/pg_controldata.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/bin/pg_waldump/pg_waldump.c
M src/include/access/xlogdefs.h
Fix an oversight in ReorderBufferFinishPrepared.
commit : ade89ba5f408e6034db7cc8a2c9b7858f5a214c4
author : Amit Kapila <[email protected]>
date : Tue, 23 Feb 2021 09:47:41 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 23 Feb 2021 09:47:41 +0530
We don't have anything to decode in a transaction if ReorderBufferTXN
doesn't exist by the time we decode the commit prepared. So don't create a
new ReorderBufferTXN here. This is an oversight in commit a271a1b5.
Reported-by: Markus Wanner
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/reorderbuffer.c
Change the error message for logical replication authentication failure.
commit : bc617a7b1cada2ec5f8b9d45a4312e49fc1766ad
author : Amit Kapila <[email protected]>
date : Tue, 23 Feb 2021 09:11:22 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 23 Feb 2021 09:11:22 +0530
The authentication failure error message wasn't distinguishing whether
it is a physical replication or logical replication connection failure and
was giving incomplete information on what led to failure in case of logical
replication connection.
Author: Paul Martinez and Amit Kapila
Reviewed-by: Euler Taveira and Amit Kapila
Discussion: https://postgr.es/m/CACqFVBYahrAi2OPdJfUA3YCvn3QMzzxZdw0ibSJ8wouWeDtiyQ@mail.gmail.com
M doc/src/sgml/client-auth.sgml
M src/backend/libpq/auth.c
Remove pointless HeapTupleHeaderIndicatesMovedPartitions calls
commit : 0f5505a8815aa4eb03ca61245a5a51ff9adda2f4
author : Alvaro Herrera <[email protected]>
date : Mon, 22 Feb 2021 16:51:44 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 22 Feb 2021 16:51:44 -0300
Pavan Deolasee recently noted that a few of the
HeapTupleHeaderIndicatesMovedPartitions calls added by commit
5db6df0c0117 are useless, since they are done after comparing t_self
with t_ctid. But because t_self can never be set to the magical values
that indicate that the tuple moved partition, this can never succeed: if
the first test fails (so we know t_self equals t_ctid), necessarily the
second test will also fail.
So these checks can be removed and no harm is done. There's no bug
here, just a code legibility issue.
Reported-by: Pavan Deolasee <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_visibility.c
Fix typo
commit : 6a03369a71d4a7dc5b8d928aab775ddd28b72494
author : Alvaro Herrera <[email protected]>
date : Mon, 22 Feb 2021 11:34:05 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 22 Feb 2021 11:34:05 -0300
M src/backend/storage/ipc/procarray.c
Fix docs build for website styles
commit : d22d0fa937616a3112f69ebd6a6ed4f039162441
author : Magnus Hagander <[email protected]>
date : Mon, 22 Feb 2021 13:00:54 +0100
committer: Magnus Hagander <[email protected]>
date : Mon, 22 Feb 2021 13:00:54 +0100
Building the docs with STYLE=website referenced a stylesheet that long
longer exists on the website, since we changed it to use versioned
references.
To make it less likely for this to happen again, point to a single
stylesheet on the website which will in turn import the required one.
That puts the process entirely within the scope of the website
repository, so next time a version is switched that's the only place
changes have to be made, making them less likely to be missed.
Per (off-list) discussion with Peter Geoghegan and Jonathan Katz.
M doc/src/sgml/stylesheet.xsl
Tab-complete CREATE COLLATION.
commit : 5bc09a74719dfeb6c4cebb311b81385c508459a8
author : Thomas Munro <[email protected]>
date : Mon, 22 Feb 2021 23:17:31 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 22 Feb 2021 23:17:31 +1300
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/20210117215940.GE8560%40telsasoft.com
M src/bin/psql/tab-complete.c
Refactor get_collation_current_version().
commit : beb4480c853a60ec43bd3f1a71252890dd234f10
author : Thomas Munro <[email protected]>
date : Mon, 22 Feb 2021 23:03:27 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 22 Feb 2021 23:03:27 +1300
The code paths for three different OSes finished up with three different
ways of excluding C[.xxx] and POSIX from consideration. Merge them.
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/20210117215940.GE8560%40telsasoft.com
M src/backend/utils/adt/pg_locale.c
pg_collation_actual_version() -> pg_collation_current_version().
commit : 9cf184cc0599b6e65e7e5ecd9d91cd42e278bcd8
author : Thomas Munro <[email protected]>
date : Mon, 22 Feb 2021 23:02:52 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 22 Feb 2021 23:02:52 +1300
The new name seems a bit more natural.
Discussion: https://postgr.es/m/20210117215940.GE8560%40telsasoft.com
M doc/src/sgml/func.sgml
M src/backend/commands/collationcmds.c
M src/backend/utils/adt/pg_locale.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Hide internal error for pg_collation_actual_version(<bad OID>).
commit : 0fb0a0503bfc125764c8dba4f515058145dc7f8b
author : Thomas Munro <[email protected]>
date : Mon, 22 Feb 2021 23:01:20 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 22 Feb 2021 23:01:20 +1300
Instead of an unsightly internal "cache lookup failed" message, just
return NULL for bad OIDs, as is the convention for other similar things.
Reported-by: Justin Pryzby <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/20210117215940.GE8560%40telsasoft.com
M src/backend/catalog/index.c
M src/backend/catalog/pg_depend.c
M src/backend/commands/collationcmds.c
M src/backend/utils/adt/pg_locale.c
M src/include/utils/pg_locale.h
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Initialize atomic variable waitStart in PGPROC, at postmaster startup.
commit : f05ed5a5cfa55878baa77a1e39d68cb09793b477
author : Fujii Masao <[email protected]>
date : Mon, 22 Feb 2021 18:25:00 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 22 Feb 2021 18:25:00 +0900
Commit 46d6e5f567 added the atomic variable "waitStart" into PGPROC struct,
to store the time at which wait for lock acquisition started. Previously
this variable was initialized every time each backend started. Instead,
this commit makes postmaster initialize it at the startup, to ensure that
the variable should be initialized before any use of it.
This commit also moves the code to initialize "waitStart" variable for
prepare transaction, from TwoPhaseGetDummyProc() to MarkAsPreparingGuts().
Because MarkAsPreparingGuts() is more proper place to do that since
it initializes other PGPROC variables.
Author: Fujii Masao
Reviewed-by: Atsushi Torikoshi
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/twophase.c
M src/backend/storage/lmgr/proc.c
Improve new hash partition bound check error messages
commit : efbfb642414b61db1567a7a902ace3f307d7564a
author : Peter Eisentraut <[email protected]>
date : Mon, 22 Feb 2021 08:06:45 +0100
committer: Peter Eisentraut <[email protected]>
date : Mon, 22 Feb 2021 08:06:45 +0100
For the error message "every hash partition modulus must be a factor
of the next larger modulus", add a detail message that shows the
particular numbers and existing partition involved. Also comment the
code more.
Reviewed-by: Amit Langote <[email protected]>
Reviewed-by: Heikki Linnakangas <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/bb9d60b4-aadb-607a-1a9d-fdc3434dddcd%40enterprisedb.com
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/create_table.out
Use pgstat_progress_update_multi_param() where possible
commit : 92942642788c9d73e4c090ee0a77603f7afbc1d7
author : Michael Paquier <[email protected]>
date : Mon, 22 Feb 2021 14:21:40 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 22 Feb 2021 14:21:40 +0900
This commit changes one code path in REINDEX INDEX and one code path
in CREATE INDEX CONCURRENTLY to report the progress of each operation
using pgstat_progress_update_multi_param() rather than
multiple calls to pgstat_progress_update_param(). This has the
advantage to make the progress report more consistent to the end-user
without impacting the amount of information provided.
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/CALj2ACV5zW7GxD8D_tyO==bcj6ZktQchEKWKPBOAGKiLhAQo=w@mail.gmail.com
M src/backend/catalog/index.c
M src/backend/commands/indexcmds.c
Remove outdated reference to RAID spindles.
commit : db8374d804f2dd35a0f934984d1d3686cc71f545
author : Thomas Munro <[email protected]>
date : Mon, 22 Feb 2021 14:37:28 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 22 Feb 2021 14:37:28 +1300
Commit b09ff536 left behind some outdated advice in the long_desc field
of the GUC "effective_io_concurrency". Remove it.
Back-patch to 13.
Reported-by: Andrew Gierth <[email protected]>
Reviewed-by: Julien Rouhaud <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJyyWqFBxL9gEj-qtjBThGjhAOBE8GBnF8MUJOJ3vrfag%40mail.gmail.com
M src/backend/utils/misc/guc.c
Simplify memory management for regex DFAs a little.
commit : 190c79884aae540c92f017701726ed69265e2dab
author : Tom Lane <[email protected]>
date : Sun, 21 Feb 2021 20:29:11 -0500
committer: Tom Lane <[email protected]>
date : Sun, 21 Feb 2021 20:29:11 -0500
Coverity complained that functions in regexec.c might leak DFA
storage. It's wrong, but this logic is confusing enough that it's
not so surprising Coverity couldn't make sense of it. Rewrite
in hopes of making it more legible to humans as well as machines.
M src/backend/regex/rege_dfa.c
M src/backend/regex/regexec.c
Fix invalid array access in trgm_regexp.c.
commit : 6ee479abfc27a18c37fe77140d16d3ac31f4ac31
author : Tom Lane <[email protected]>
date : Sun, 21 Feb 2021 19:46:46 -0500
committer: Tom Lane <[email protected]>
date : Sun, 21 Feb 2021 19:46:46 -0500
Brown-paper-bag bug in 08c0d6ad6: I missed one place that needed
to guard against RAINBOW arc colors. Remarkably, nothing noticed
the invalid array access except buildfarm member thorntail.
Thanks to Noah Misch for assistance with tracking this down.
M contrib/pg_trgm/trgm_regexp.c
Avoid generating extra subre tree nodes for capturing parentheses.
commit : ea1268f6301cc7adce571cc9c5ebe8d9342a2ef4
author : Tom Lane <[email protected]>
date : Sat, 20 Feb 2021 19:26:41 -0500
committer: Tom Lane <[email protected]>
date : Sat, 20 Feb 2021 19:26:41 -0500
Previously, each pair of capturing parentheses gave rise to a separate
subre tree node, whose only function was to identify that we ought to
capture the match details for this particular sub-expression. In
most cases we don't really need that, since we can perfectly well
put a "capture this" annotation on the child node that does the real
matching work. As with the two preceding commits, the main value
of this is to avoid generating and optimizing an NFA for a tree node
that's not really pulling its weight.
The chosen data representation only allows one capture annotation
per subre node. In the legal-per-spec, but seemingly not very useful,
case where there are multiple capturing parens around the exact same
bit of the regex (i.e. "((xyz))"), wrap the child node in N-1 capture
nodes that act the same as before. We could work harder at that but
I'll refrain, pending some evidence that such cases are worth troubling
over.
In passing, improve the comments in regex.h to say what all the
different re_info bits mean. Some of them were pretty obvious
but others not so much, so reverse-engineer some documentation.
This is part of a patch series that in total reduces the regex engine's
runtime by about a factor of four on a large corpus of real-world regexes.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/README
M src/backend/regex/regcomp.c
M src/backend/regex/rege_dfa.c
M src/backend/regex/regexec.c
M src/include/regex/regex.h
M src/include/regex/regguts.h
Convert regex engine's subre tree from binary to N-ary style.
commit : 581043089472816061a7fd381f40572191dfa48f
author : Tom Lane <[email protected]>
date : Sat, 20 Feb 2021 19:07:45 -0500
committer: Tom Lane <[email protected]>
date : Sat, 20 Feb 2021 19:07:45 -0500
Instead of having left and right child links in subre structs,
have a single child link plus a sibling link. Multiple children
of a tree node are now reached by chasing the sibling chain.
The beneficiary of this is alternation tree nodes. A regular
expression with N (>1) branches is now represented by one alternation
node with N children, rather than a tree that includes N alternation
nodes as well as N children. While the old representation didn't
really cost anything extra at execution time, it was pretty horrid
for compilation purposes, because each of the alternation nodes had
its own NFA, which we were too stupid not to separately optimize.
(To make matters worse, all of those NFAs described the entire
alternation pattern, not just the portion of it that one might
expect from the tree structure.)
We continue to require concatenation nodes to have exactly two
children. This data structure is now prepared to support more,
but the executor's logic would need some careful redesign, and
it's not clear that a lot of benefit could be had.
This is part of a patch series that in total reduces the regex engine's
runtime by about a factor of four on a large corpus of real-world regexes.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/README
M src/backend/regex/regcomp.c
M src/backend/regex/regexec.c
M src/include/regex/regguts.h
Fix regex engine to suppress useless concatenation sub-REs.
commit : cebc1d34e5207c37871708f91be65dd839760b5f
author : Tom Lane <[email protected]>
date : Sat, 20 Feb 2021 18:45:29 -0500
committer: Tom Lane <[email protected]>
date : Sat, 20 Feb 2021 18:45:29 -0500
The comment for parsebranch() claims that it avoids generating
unnecessary concatenation nodes in the "subre" tree, but it missed
some significant cases. Once we've decided that a given atom is
"messy" and can't be bundled with the preceding atom(s) of the
current regex branch, parseqatom() always generated two new concat
nodes, one to concat the messy atom to what follows it in the branch,
and an upper node to concatenate the preceding part of the branch
to that one. But one or both of these could be unnecessary, if the
messy atom is the first, last, or only one in the branch. Improve
the code to suppress such useless concat nodes, along with the
no-op child nodes representing empty chunks of a branch.
Reducing the number of subre tree nodes offers significant savings
not only at execution but during compilation, because each subre node
has its own NFA that has to be separately optimized. (Maybe someday
we'll figure out how to share the optimization work across multiple
tree nodes, but it doesn't look easy.) Eliminating upper tree nodes
is especially useful because they tend to have larger NFAs.
This is part of a patch series that in total reduces the regex engine's
runtime by about a factor of four on a large corpus of real-world regexes.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regcomp.c
Recognize "match-all" NFAs within the regex engine.
commit : 824bf71902db4a2067b8d64583c9d88bb264c44b
author : Tom Lane <[email protected]>
date : Sat, 20 Feb 2021 18:31:19 -0500
committer: Tom Lane <[email protected]>
date : Sat, 20 Feb 2021 18:31:19 -0500
This builds on the previous "rainbow" patch to detect NFAs that will
match any string, though possibly with constraints on the string length.
This definition is chosen to match constructs such as ".*", ".+", and
".{1,100}". Recognizing such an NFA after the optimization pass is
fairly cheap, since we basically just have to verify that all arcs
are RAINBOW arcs and count the number of steps to the end state.
(Well, there's a bit of complication with pseudo-color arcs for string
boundary conditions, but not much.)
Once we have these markings, the regex executor functions longest(),
shortest(), and matchuntil() don't have to expend per-character work
to determine whether a given substring satisfies such an NFA; they
just need to check its length against the bounds. Since some matching
problems require O(N) invocations of these functions, we've reduced
the runtime for an N-character string from O(N^2) to O(N). Of course,
this is no help for non-matchall sub-patterns, but those usually have
constraints that allow us to avoid needing O(N) substring checks in the
first place. It's precisely the unconstrained "match-all" cases that
cause the most headaches.
This is part of a patch series that in total reduces the regex engine's
runtime by about a factor of four on a large corpus of real-world regexes.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/backend/regex/rege_dfa.c
M src/backend/regex/regprefix.c
M src/include/regex/regguts.h
Invent "rainbow" arcs within the regex engine.
commit : 08c0d6ad65f7c161add82ae906efb90dbd7f653d
author : Tom Lane <[email protected]>
date : Sat, 20 Feb 2021 18:11:56 -0500
committer: Tom Lane <[email protected]>
date : Sat, 20 Feb 2021 18:11:56 -0500
Some regular expression constructs, most notably the "." match-anything
metacharacter, produce a sheaf of parallel NFA arcs covering all
possible colors (that is, character equivalence classes). We can make
a noticeable improvement in the space and time needed to process large
regexes by replacing such cases with a single arc bearing the special
color code "RAINBOW". This requires only minor additional complication
in places such as pull() and push().
Callers of pg_reg_getoutarcs() must now be prepared for the possibility
of seeing a RAINBOW arc. For the one known user, contrib/pg_trgm,
that's a net benefit since it cuts the number of arcs to be dealt with,
and the handling isn't any different than for other colors that contain
too many characters to be dealt with individually.
This is part of a patch series that in total reduces the regex engine's
runtime by about a factor of four on a large corpus of real-world regexes.
Patch by me, reviewed by Joel Jacobson
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_trgm/trgm_regexp.c
M src/backend/regex/README
M src/backend/regex/regc_color.c
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/backend/regex/rege_dfa.c
M src/backend/regex/regexport.c
M src/backend/regex/regprefix.c
M src/include/regex/regexport.h
M src/include/regex/regguts.h
doc: Mention that partitions_{done,total} is 0 for REINDEX progress reports
commit : 17661188336c8cbb1783808912096932c57893a3
author : Michael Paquier <[email protected]>
date : Sat, 20 Feb 2021 10:25:14 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 20 Feb 2021 10:25:14 +0900
REINDEX has recently gained support for partitions, so it can be
confusing to see those fields not being set. Making useful reports for
for such relations is more complicated than it looks with the current
set of columns available in pg_stat_progress_create_index, and this
touches equally REINDEX DATABASE/SYSTEM/SCHEMA. This commit documents
that those two columns are not touched during a REINDEX.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
Fix inconsistent configure data for --with-ssl
commit : a899ec1cb23eb3c4a94648f18a9408e5316d32a4
author : Michael Paquier <[email protected]>
date : Sat, 20 Feb 2021 10:17:10 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 20 Feb 2021 10:17:10 +0900
This inconsistency was showing up after an autoreconf.
Reported-by: Antonin Houska
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/47255.1613716807@antos
M configure.ac
M src/include/pg_config.h.in
Fix psql's ON_ERROR_ROLLBACK so that it handles COMMIT AND CHAIN.
commit : fe06819f105ccea52c12d418c8dbaaaa54377e96
author : Fujii Masao <[email protected]>
date : Fri, 19 Feb 2021 22:01:25 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 19 Feb 2021 22:01:25 +0900
When ON_ERROR_ROLLBACK is enabled, psql releases a temporary savepoint
if it's idle in a valid transaction block after executing a query. But psql
doesn't do that after RELEASE or ROLLBACK is executed because a temporary
savepoint has already been destroyed in that case.
This commit changes psql's ON_ERROR_ROLLBACK so that it doesn't release
a temporary savepoint also when COMMIT AND CHAIN is executed. A temporary
savepoint doesn't need to be released in that case because
COMMIT AND CHAIN also destroys any savepoints defined within the transaction
to commit. Otherwise psql tries to release the savepoint that
COMMIT AND CHAIN has already destroyed and cause an error
"ERROR: savepoint "pg_psql_temporary_savepoint" does not exist".
Back-patch to v12 where transaction chaining was added.
Reported-by: Arthur Nascimento
Author: Arthur Nascimento
Reviewed-by: Fujii Masao, Vik Fearing
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/common.c
Fix bug in COMMIT AND CHAIN command.
commit : 8a55cb5ba9655ffb1cf0a3042aaa6f5eef8c5a85
author : Fujii Masao <[email protected]>
date : Fri, 19 Feb 2021 21:57:52 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 19 Feb 2021 21:57:52 +0900
This commit fixes COMMIT AND CHAIN command so that it starts new transaction
immediately even if savepoints are defined within the transaction to commit.
Previously COMMIT AND CHAIN command did not in that case because
commit 280a408b48 forgot to make CommitTransactionCommand() handle
a transaction chaining when the transaction state was TBLOCK_SUBCOMMIT.
Also this commit adds the regression test for COMMIT AND CHAIN command
when savepoints are defined.
Back-patch to v12 where transaction chaining was added.
Reported-by: Arthur Nascimento
Author: Fujii Masao
Reviewed-by: Arthur Nascimento, Vik Fearing
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xact.c
M src/test/regress/expected/transactions.out
M src/test/regress/sql/transactions.sql
Update snowball
commit : 678d0e239b67174f349a401ea7dcecabb3c5b137
author : Peter Eisentraut <[email protected]>
date : Fri, 19 Feb 2021 07:57:42 +0100
committer: Peter Eisentraut <[email protected]>
date : Fri, 19 Feb 2021 07:57:42 +0100
Update to snowball tag v2.1.0. Major changes are new stemmers for
Armenian, Serbian, and Yiddish.
M doc/src/sgml/textsearch.sgml
M src/backend/snowball/Makefile
M src/backend/snowball/README
M src/backend/snowball/dict_snowball.c
M src/backend/snowball/libstemmer/api.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_basque.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_catalan.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_danish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_dutch.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_english.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_finnish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_french.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_german.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_indonesian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_irish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_italian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_norwegian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_porter.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_portuguese.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_spanish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_swedish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_2_hungarian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_2_romanian.c
M src/backend/snowball/libstemmer/stem_KOI8_R_russian.c
M src/backend/snowball/libstemmer/stem_UTF_8_arabic.c
A src/backend/snowball/libstemmer/stem_UTF_8_armenian.c
M src/backend/snowball/libstemmer/stem_UTF_8_basque.c
M src/backend/snowball/libstemmer/stem_UTF_8_catalan.c
M src/backend/snowball/libstemmer/stem_UTF_8_danish.c
M src/backend/snowball/libstemmer/stem_UTF_8_dutch.c
M src/backend/snowball/libstemmer/stem_UTF_8_english.c
M src/backend/snowball/libstemmer/stem_UTF_8_finnish.c
M src/backend/snowball/libstemmer/stem_UTF_8_french.c
M src/backend/snowball/libstemmer/stem_UTF_8_german.c
M src/backend/snowball/libstemmer/stem_UTF_8_greek.c
M src/backend/snowball/libstemmer/stem_UTF_8_hindi.c
M src/backend/snowball/libstemmer/stem_UTF_8_hungarian.c
M src/backend/snowball/libstemmer/stem_UTF_8_indonesian.c
M src/backend/snowball/libstemmer/stem_UTF_8_irish.c
M src/backend/snowball/libstemmer/stem_UTF_8_italian.c
M src/backend/snowball/libstemmer/stem_UTF_8_lithuanian.c
M src/backend/snowball/libstemmer/stem_UTF_8_nepali.c
M src/backend/snowball/libstemmer/stem_UTF_8_norwegian.c
M src/backend/snowball/libstemmer/stem_UTF_8_porter.c
M src/backend/snowball/libstemmer/stem_UTF_8_portuguese.c
M src/backend/snowball/libstemmer/stem_UTF_8_romanian.c
M src/backend/snowball/libstemmer/stem_UTF_8_russian.c
A src/backend/snowball/libstemmer/stem_UTF_8_serbian.c
M src/backend/snowball/libstemmer/stem_UTF_8_spanish.c
M src/backend/snowball/libstemmer/stem_UTF_8_swedish.c
M src/backend/snowball/libstemmer/stem_UTF_8_tamil.c
M src/backend/snowball/libstemmer/stem_UTF_8_turkish.c
A src/backend/snowball/libstemmer/stem_UTF_8_yiddish.c
M src/backend/snowball/libstemmer/utilities.c
M src/bin/initdb/initdb.c
M src/include/catalog/catversion.h
M src/include/snowball/libstemmer/api.h
M src/include/snowball/libstemmer/header.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_basque.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_catalan.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_danish.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_dutch.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_english.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_finnish.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_french.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_german.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_indonesian.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_irish.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_italian.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_norwegian.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_porter.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_portuguese.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_spanish.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_swedish.h
M src/include/snowball/libstemmer/stem_ISO_8859_2_hungarian.h
M src/include/snowball/libstemmer/stem_ISO_8859_2_romanian.h
M src/include/snowball/libstemmer/stem_KOI8_R_russian.h
M src/include/snowball/libstemmer/stem_UTF_8_arabic.h
A src/include/snowball/libstemmer/stem_UTF_8_armenian.h
M src/include/snowball/libstemmer/stem_UTF_8_basque.h
M src/include/snowball/libstemmer/stem_UTF_8_catalan.h
M src/include/snowball/libstemmer/stem_UTF_8_danish.h
M src/include/snowball/libstemmer/stem_UTF_8_dutch.h
M src/include/snowball/libstemmer/stem_UTF_8_english.h
M src/include/snowball/libstemmer/stem_UTF_8_finnish.h
M src/include/snowball/libstemmer/stem_UTF_8_french.h
M src/include/snowball/libstemmer/stem_UTF_8_german.h
M src/include/snowball/libstemmer/stem_UTF_8_greek.h
M src/include/snowball/libstemmer/stem_UTF_8_hindi.h
M src/include/snowball/libstemmer/stem_UTF_8_hungarian.h
M src/include/snowball/libstemmer/stem_UTF_8_indonesian.h
M src/include/snowball/libstemmer/stem_UTF_8_irish.h
M src/include/snowball/libstemmer/stem_UTF_8_italian.h
M src/include/snowball/libstemmer/stem_UTF_8_lithuanian.h
M src/include/snowball/libstemmer/stem_UTF_8_nepali.h
M src/include/snowball/libstemmer/stem_UTF_8_norwegian.h
M src/include/snowball/libstemmer/stem_UTF_8_porter.h
M src/include/snowball/libstemmer/stem_UTF_8_portuguese.h
M src/include/snowball/libstemmer/stem_UTF_8_romanian.h
M src/include/snowball/libstemmer/stem_UTF_8_russian.h
A src/include/snowball/libstemmer/stem_UTF_8_serbian.h
M src/include/snowball/libstemmer/stem_UTF_8_spanish.h
M src/include/snowball/libstemmer/stem_UTF_8_swedish.h
M src/include/snowball/libstemmer/stem_UTF_8_tamil.h
M src/include/snowball/libstemmer/stem_UTF_8_turkish.h
A src/include/snowball/libstemmer/stem_UTF_8_yiddish.h
Add nbtree README section on page recycling.
commit : b071a311495bbf42ddf2466a556d033df8f0f5e7
author : Peter Geoghegan <[email protected]>
date : Thu, 18 Feb 2021 21:16:33 -0800
committer: Peter Geoghegan <[email protected]>
date : Thu, 18 Feb 2021 21:16:33 -0800
Consolidate discussion of how VACUUM places pages in the FSM for
recycling by adding a new section that comes after discussion of page
deletion. This structure reflects the fact that page recycling is
explicitly decoupled from page deletion in Lanin & Shasha's paper. Page
recycling in nbtree is an implementation of what the paper calls "the
drain technique".
This decoupling is an important concept for nbtree VACUUM. Searchers
have to detect and recover from concurrent page deletions, but they will
never have to reason about concurrent page recycling. Recycling can
almost always be thought of as a low level garbage collection operation
that asynchronously frees the physical space that backs a logical tree
node. Almost all code need only concern itself with logical tree nodes.
(Note that "logical tree node" is not currently a term of art in the
nbtree code -- this all works implicitly.)
This is preparation for an upcoming patch that teaches nbtree VACUUM to
remember the details of pages that it deletes on the fly, in local
memory. This enables the same VACUUM operation to consider placing its
own deleted pages in the FSM later on, when it reaches the end of
btvacuumscan().
M src/backend/access/nbtree/README
Fix another ancient bug in parsing of BRE-mode regular expressions.
commit : b5a66e7353ba65c11c5fc6a79b72213bde8dbe44
author : Tom Lane <[email protected]>
date : Thu, 18 Feb 2021 22:38:55 -0500
committer: Tom Lane <[email protected]>
date : Thu, 18 Feb 2021 22:38:55 -0500
While poking at the regex code, I happened to notice that the bug
squashed in commit afcc8772e had a sibling: next() failed to return
a specific value associated with the '}' token for a "\{m,n\}"
quantifier when parsing in basic RE mode. Again, this could result
in treating the quantifier as non-greedy, which it never should be in
basic mode. For that to happen, the last character before "\}" that
sets "nextvalue" would have to set it to zero, or it'd have to have
accidentally been zero from the start. The failure can be provoked
repeatably with, for example, a bound ending in digit "0".
Like the previous patch, back-patch all the way.
M src/backend/regex/regc_lex.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Fix "invalid spinlock number: 0" error in pg_stat_wal_receiver.
commit : 614b7f18b3bda738f352a8732cf749eb5fa56dae
author : Fujii Masao <[email protected]>
date : Thu, 18 Feb 2021 23:22:23 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 18 Feb 2021 23:22:23 +0900
Commit 2c8dd05d6c added the atomic variable writtenUpto into
walreceiver's shared memory information. It's initialized only
when walreceiver started up but could be read via pg_stat_wal_receiver
view anytime, i.e., even before it's initialized. In the server built
with --disable-atomics and --disable-spinlocks, this uninitialized
atomic variable read could cause "invalid spinlock number: 0" error.
This commit changed writtenUpto so that it's initialized at
the postmaster startup, to avoid the uninitialized variable read
via pg_stat_wal_receiver and fix the error.
Also this commit moved the read of writtenUpto after the release
of spinlock protecting walreceiver's shared variables. This is
necessary to prevent new spinlock from being taken by atomic
variable read while holding another spinlock, and to shorten
the spinlock duration. This change leads writtenUpto not to be
consistent with the other walreceiver's shared variables protected
by a spinlock. But this is OK because writtenUpto should not be
used for data integrity checks.
Back-patch to v13 where commit 2c8dd05d6c introduced the bug.
Author: Fujii Masao
Reviewed-by: Michael Paquier, Thomas Munro, Andres Freund
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/walreceiver.c
M src/backend/replication/walreceiverfuncs.c
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/sysviews.sql
Add tests for bytea LIKE operator
commit : eb42110d952f8d1ad4049b8f2491e9dfba75ffed
author : Peter Eisentraut <[email protected]>
date : Thu, 18 Feb 2021 08:27:05 +0100
committer: Peter Eisentraut <[email protected]>
date : Thu, 18 Feb 2021 08:27:05 +0100
Add test coverage for the following operations, which were previously
not tested at all:
bytea LIKE bytea (bytealike)
bytea NOT LIKE bytea (byteanlike)
ESCAPE clause for the above (like_escape_bytea)
also
name NOT ILIKE text (nameicnlike)
Discussion: https://www.postgresql.org/message-id/flat/4d13563a-2c8d-fd91-20d5-e71b7a4eaa87%40enterprisedb.com
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
Allow specifying CRL directory
commit : f5465fade90827534fbd0b795d18dc62e56939e9
author : Peter Eisentraut <[email protected]>
date : Thu, 18 Feb 2021 07:59:10 +0100
committer: Peter Eisentraut <[email protected]>
date : Thu, 18 Feb 2021 07:59:10 +0100
Add another method to specify CRLs, hashed directory method, for both
server and client side. This offers a means for server or libpq to
load only CRLs that are required to verify a certificate. The CRL
directory is specifed by separate GUC variables or connection options
ssl_crl_dir and sslcrldir, alongside the existing ssl_crl_file and
sslcrl, so both methods can be used at the same time.
Author: Kyotaro Horiguchi <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M doc/src/sgml/config.sgml
M doc/src/sgml/libpq.sgml
M src/backend/libpq/be-secure-openssl.c
M src/backend/libpq/be-secure.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/libpq/libpq.h
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/libpq-int.h
M src/test/ssl/Makefile
A src/test/ssl/ssl/client-crldir/9bb9e3c3.r0
A src/test/ssl/ssl/root+client-crldir/9bb9e3c3.r0
A src/test/ssl/ssl/root+client-crldir/a3d11bff.r0
A src/test/ssl/ssl/root+server-crldir/a3d11bff.r0
A src/test/ssl/ssl/root+server-crldir/a836cc2d.r0
A src/test/ssl/ssl/server-crldir/a836cc2d.r0
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/SSLServer.pm
nbtree README: move VACUUM linear scan section.
commit : 128dd901a5c87e11c6a8cbe227a806cdc3afd10d
author : Peter Geoghegan <[email protected]>
date : Wed, 17 Feb 2021 21:13:15 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 17 Feb 2021 21:13:15 -0800
Discuss VACUUM's linear scan after discussion of tuple deletion by
VACUUM, but before discussion of page deletion by VACUUM. This
progression is a lot more natural.
Also tweak the wording a little. It seems unnecessary to talk about how
it worked prior to PostgreSQL 8.2.
M src/backend/access/nbtree/README
Fix tuple routing to initialize batching only for inserts
commit : 927f453a941061e3d5884bab206581c34784e45b
author : Tomas Vondra <[email protected]>
date : Thu, 18 Feb 2021 00:02:00 +0100
committer: Tomas Vondra <[email protected]>
date : Thu, 18 Feb 2021 00:02:00 +0100
A cross-partition update on a partitioned table is implemented as a
delete followed by an insert. With foreign partitions, this was however
causing issues, because the FDW and core may disagree on when to enable
batching. postgres_fdw was only allowing batching for plain inserts
(CMD_INSERT) while core was trying to batch the insert component of the
cross-partition update. Fix by restricting core to apply batching only
to plain CMD_INSERT queries.
It's possible to allow batching for cross-partition updates, but that
will require more extensive changes, so better to leave that for a
separate patch.
Author: Amit Langote
Reviewed-by: Tomas Vondra, Takayuki Tsunakawa
Discussion: https://postgr.es/m/20200628151002.7x5laxwpgvkyiu3q@development
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/executor/execPartition.c
Fix pointer type in ExecForeignBatchInsert SGML docs
commit : c15283ff429bf318f161bf84768795843b22696d
author : Tomas Vondra <[email protected]>
date : Thu, 18 Feb 2021 00:01:21 +0100
committer: Tomas Vondra <[email protected]>
date : Thu, 18 Feb 2021 00:01:21 +0100
Reported-by: Ian Barwick
Discussion: https://postgr.es/m/20200628151002.7x5laxwpgvkyiu3q@development
M doc/src/sgml/fdwhandler.sgml
Make some minor improvements in the regex code.
commit : 4e703d67193df0431c0740044d662d1feade73aa
author : Tom Lane <[email protected]>
date : Wed, 17 Feb 2021 12:24:12 -0500
committer: Tom Lane <[email protected]>
date : Wed, 17 Feb 2021 12:24:12 -0500
Push some hopefully-uncontroversial bits extracted from an upcoming
patch series, to remove non-relevant clutter from the main patches.
In compact(), return immediately after setting REG_ASSERT error;
continuing the loop would just lead to assertion failure below.
(Ask me how I know.)
In parseqatom(), remove assertion that moresubs() did its job.
When moresubs actually did its job, this is redundant with that
function's final assert; but when it failed on OOM, this is an
assertion crash. We could avoid the crash by adding a NOERR()
check before the assertion, but it seems better to subtract code
than add it. (Note that there's a NOERR exit a few lines further
down, and nothing else between here and there requires moresubs
to have succeeded. So we don't really need an extra error exit.)
This is a live bug in assert-enabled builds, but given the very
low likelihood of OOM in moresub's tiny allocation, I don't think
it's worth back-patching.
On the other hand, it seems worthwhile to add an assertion that
our intended v->subs[subno] target is still null by the time we
are ready to insert into it, since there's a recursion in between.
In pg_regexec, ensure we fflush any debug output on the way out,
and try to make MDEBUG messages more uniform and helpful. (In
particular, ensure that all of them are prefixed with the subre's
id number, so one can match up entry and exit reports.)
Add some test cases in test_regex to improve coverage of lookahead
and lookbehind constraints. Adding these now is mainly to establish
that this is indeed the existing behavior.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/backend/regex/regexec.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Routine usage information schema tables
commit : f40c6969d0eddfc6de786006bd1048961a65a0eb
author : Peter Eisentraut <[email protected]>
date : Wed, 17 Feb 2021 17:53:18 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 17 Feb 2021 17:53:18 +0100
Several information schema views track dependencies between
functions/procedures and objects used by them. These had not been
implemented so far because PostgreSQL doesn't track objects used in a
function body. However, formally, these also show dependencies used
in parameter default expressions, which PostgreSQL does support and
track. So for the sake of completeness, we might as well add these.
If dependency tracking for function bodies is ever implemented, these
views will automatically work correctly.
Reviewed-by: Erik Rijkers <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/ac80fc74-e387-8950-9a31-2560778fc1e3%40enterprisedb.com
M doc/src/sgml/information_schema.sgml
M src/backend/catalog/information_schema.sql
M src/backend/catalog/sql_features.txt
M src/include/catalog/catversion.h
M src/test/regress/expected/create_function_3.out
M src/test/regress/sql/create_function_3.sql
Fix typo
commit : a29f30780f34d7706fcd398dea1d6882d184d17a
author : Magnus Hagander <[email protected]>
date : Wed, 17 Feb 2021 13:53:26 +0100
committer: Magnus Hagander <[email protected]>
date : Wed, 17 Feb 2021 13:53:26 +0100
Author: Daniel Gustafsson <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_table.sgml
Use errmsg_internal for debug messages
commit : 0e392fcc0d5ab73c81d9284e7dda5acbb7ad6d21
author : Peter Eisentraut <[email protected]>
date : Wed, 17 Feb 2021 11:24:46 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 17 Feb 2021 11:24:46 +0100
An inconsistent set of debug-level messages was not using
errmsg_internal(), thus uselessly exposing the messages to translation
work. Fix those.
M contrib/amcheck/verify_nbtree.c
M contrib/pg_prewarm/autoprewarm.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/varsup.c
M src/backend/access/transam/xlog.c
M src/backend/catalog/dependency.c
M src/backend/catalog/index.c
M src/backend/commands/indexcmds.c
M src/backend/commands/subscriptioncmds.c
M src/backend/commands/tablecmds.c
M src/backend/jit/llvm/llvmjit.c
M src/backend/libpq/be-secure.c
M src/backend/parser/parse_utilcmd.c
M src/backend/partitioning/partbounds.c
M src/backend/port/win32_shmem.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/postmaster.c
M src/backend/postmaster/syslogger.c
M src/backend/replication/logical/launcher.c
M src/backend/replication/syncrep.c
M src/backend/replication/walsender.c
M src/backend/storage/lmgr/predicate.c
M src/backend/storage/lmgr/proc.c
M src/backend/storage/smgr/md.c
M src/backend/storage/sync/sync.c
M src/backend/tcop/postgres.c
M src/backend/utils/init/miscinit.c
Add psql completion for [ NO ] DEPENDS ON EXTENSION
commit : e6b8e83b9f012cfbb3a2f96a7d2e74aebd299d4e
author : Michael Paquier <[email protected]>
date : Wed, 17 Feb 2021 11:50:58 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 17 Feb 2021 11:50:58 +0900
ALTER INDEX was able to handle that already. This adds tab completion
for all the remaining commands that support this grammar:
- ALTER FUNCTION
- ALTER PROCEDURE
- ALTER ROUTINE
- ALTER TRIGGER
- ALTER MATERIALIZED VIEW
Author: Ian Lawrence Barwick
Discussion: https://postgr.es/m/CAB8KJ=iypYudXuMOAMOP4BpkaYbXxk=a2cdJppX0e9mJXWtuig@mail.gmail.com
M src/bin/psql/tab-complete.c
Convert tsginidx.c's GIN indexing logic to fully ternary operation.
commit : 38bb3aef354ca98ff88cb37337995039a3b5135f
author : Tom Lane <[email protected]>
date : Tue, 16 Feb 2021 12:07:14 -0500
committer: Tom Lane <[email protected]>
date : Tue, 16 Feb 2021 12:07:14 -0500
Commit 2f2007fbb did this partially, but there were two remaining
warts. checkcondition_gin handled some uncertain cases by setting
the out-of-band recheck flag, some by returning TS_MAYBE, and some
by doing both. Meanwhile, TS_execute arbitrarily converted a
TS_MAYBE result to TS_YES. Thus, if checkcondition_gin chose to
only return TS_MAYBE, the outcome would be TS_YES with no recheck
flag, potentially resulting in wrong query outputs.
The case where this'd happen is if there were GIN_MAYBE entries
in the indexscan results passed to gin_tsquery_[tri]consistent,
which so far as I can see would only happen if the tidbitmap used
to accumulate indexscan results grew large enough to become lossy.
I initially thought of fixing this by ensuring we always set the
recheck flag as well as returning TS_MAYBE in uncertain cases.
But that errs in the other direction, potentially forcing rechecks
of rows that provably match the query (since the recheck flag
remains set even if TS_execute later finds that the answer must be
TS_YES). Instead, let's get rid of the out-of-band recheck flag
altogether and rely on returning TS_MAYBE. This requires exporting
a version of TS_execute that will actually return the full ternary
result of the evaluation ... but we likely should have done that
to start with.
Unfortunately it doesn't seem practical to add a regression test case
that covers this: the amount of data needed to cause the GIN bitmap to
become lossy results in a longer runtime than I think we want to have
in the tests. (I'm wondering about allowing smaller work_mem settings
to ameliorate that, but it'd be a matter for a separate patch.)
Per bug #16865 from Dimitri Nüscheler. Back-patch to v13 where
the faulty commit came in.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/tsginidx.c
M src/backend/utils/adt/tsvector_op.c
M src/include/tsearch/ts_utils.h
Remove the unnecessary PrepareWrite in pgoutput.
commit : f672df5fdd22dac14c98d0a0bf5bbaa6ab17f8a5
author : Amit Kapila <[email protected]>
date : Tue, 16 Feb 2021 07:26:50 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 16 Feb 2021 07:26:50 +0530
This issue exists from the inception of this code (PG-10) but got exposed
by the recent commit ce0fdbfe97 where we are using origins in tablesync
workers. The problem was that we were sometimes sending the prepare_write
('w') message but then the actual message was not being sent and on the
subscriber side, we always expect a message after prepare_write message
which led to this bug.
I refrained from backpatching this because there is no way in the core
code to hit this prior to commit ce0fdbfe97 and we haven't received any
complaints so far.
Reported-by: Erik Rijkers
Author: Amit Kapila and Vignesh C
Tested-by: Erik Rijkers
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/pgoutput/pgoutput.c
M src/test/subscription/t/100_bugs.pl
Fix heap_page_prune() parameter order confusion introduced in dc7420c2c92.
commit : 8001cb77ee6cb4f32632850d41f00206a86bac3e
author : Andres Freund <[email protected]>
date : Mon, 15 Feb 2021 17:12:12 -0800
committer: Andres Freund <[email protected]>
date : Mon, 15 Feb 2021 17:12:12 -0800
Both luckily and unluckily the passed values meant the same for all
types. Luckily because that meant my confusion caused no harm,
unluckily because otherwise the compiler might have warned...
In passing, synchronize parameter names between definition and
declaration.
Reported-By: Peter Geoghegan <[email protected]>
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wz=L=nBoepQdH9b5Qd0nMvepFT2CnT6sjWvvpOXa=K8HVQ@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
M src/include/access/heapam.h
Remove backwards compat ugliness in snapbuild.c.
commit : a975ff4980d60f8cbd8d8cbcff70182ea53e787a
author : Andres Freund <[email protected]>
date : Mon, 15 Feb 2021 16:57:47 -0800
committer: Andres Freund <[email protected]>
date : Mon, 15 Feb 2021 16:57:47 -0800
In 955a684e040 we fixed a bug in initial snapshot creation. In the
course of which several members of struct SnapBuild were obsoleted. As
SnapBuild is serialized to disk we couldn't change the memory layout.
Unfortunately I subsequently forgot about removing the backward compat
gunk, but luckily Heikki just reminded me.
This commit bumps SNAPBUILD_VERSION, therefore breaking existing
slots (which is fine in a major release).
Author: Andres Freund
Reminded-By: Heikki Linnakangas <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/snapbuild.c
Simplify loop logic in nodeIncrementalSort.c.
commit : 0e5290312851557ee24e3d6103baf14d6066695c
author : Tom Lane <[email protected]>
date : Mon, 15 Feb 2021 10:17:58 -0500
committer: Tom Lane <[email protected]>
date : Mon, 15 Feb 2021 10:17:58 -0500
The inner loop in switchToPresortedPrefixMode() can be implemented
as a conventional integer-counter for() loop, removing a couple of
redundant boolean state variables. The old logic here was a remnant
of earlier development, but as things now stand there's no reason
for extra complexity.
Also, annotate the test case added by 82e0e2930 to explain why it
manages to hit the corner case fixed in that commit, and add an
EXPLAIN to verify that it's creating an incremental-sort plan.
Back-patch to v13, like the previous patch.
James Coleman and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/nodeIncrementalSort.c
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Make ExecGetInsertedCols() and friends more robust and improve comments.
commit : 54e51dcde03e5c746e8de6243c69fafdc8d0ec7a
author : Heikki Linnakangas <[email protected]>
date : Mon, 15 Feb 2021 09:28:08 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 15 Feb 2021 09:28:08 +0200
If ExecGetInsertedCols(), ExecGetUpdatedCols() or ExecGetExtraUpdatedCols()
were called with a ResultRelInfo that's not in the range table and isn't a
partition routing target, the functions would dereference a NULL pointer,
relinfo->ri_RootResultRelInfo. Such ResultRelInfos are created when firing
RI triggers in tables that are not modified directly. None of the current
callers of these functions pass such relations, so this isn't a live bug,
but let's make them more robust.
Also update comment in ResultRelInfo; after commit 6214e2b228,
ri_RangeTableIndex is zero for ResultRelInfos created for partition tuple
routing.
Noted by Coverity. Backpatch down to v11, like commit 6214e2b228.
Reviewed-by: Tom Lane, Amit Langote
M src/backend/executor/execUtils.c
M src/include/nodes/execnodes.h
Display the time when the process started waiting for the lock, in pg_locks, take 2
commit : 46d6e5f567906389c31c4fb3a2653da1885c18ee
author : Fujii Masao <[email protected]>
date : Mon, 15 Feb 2021 15:13:37 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 15 Feb 2021 15:13:37 +0900
This commit adds new column "waitstart" into pg_locks view. This column
reports the time when the server process started waiting for the lock
if the lock is not held. This information is useful, for example, when
examining the amount of time to wait on a lock by subtracting
"waitstart" in pg_locks from the current time, and identify the lock
that the processes are waiting for very long.
This feature uses the current time obtained for the deadlock timeout
timer as "waitstart" (i.e., the time when this process started waiting
for the lock). Since getting the current time newly can cause overhead,
we reuse the already-obtained time to avoid that overhead.
Note that "waitstart" is updated without holding the lock table's
partition lock, to avoid the overhead by additional lock acquisition.
This can cause "waitstart" in pg_locks to become NULL for a very short
period of time after the wait started even though "granted" is false.
This is OK in practice because we can assume that users are likely to
look at "waitstart" when waiting for the lock for a long time.
The first attempt of this patch (commit 3b733fcd04) caused the buildfarm
member "rorqual" (built with --disable-atomics --disable-spinlocks) to report
the failure of the regression test. It was reverted by commit 890d2182a2.
The cause of this failure was that the atomic variable for "waitstart"
in the dummy process entry created at the end of prepare transaction was
not initialized. This second attempt fixes that issue.
Bump catalog version.
Author: Atsushi Torikoshi
Reviewed-by: Ian Lawrence Barwick, Robert Haas, Justin Pryzby, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M contrib/amcheck/expected/check_btree.out
M doc/src/sgml/catalogs.sgml
M src/backend/access/transam/twophase.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/adt/lockfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/storage/lock.h
M src/include/storage/proc.h
M src/test/regress/expected/rules.out
Add "LP_DEAD item?" column to GiST pageinspect functions
commit : 9e596b65f430fcb942685b41860b323398a88873
author : Peter Geoghegan <[email protected]>
date : Sun, 14 Feb 2021 20:11:11 -0800
committer: Peter Geoghegan <[email protected]>
date : Sun, 14 Feb 2021 20:11:11 -0800
This brings gist_page_items() and gist_page_items_bytea() in line with
nbtree's bt_page_items() function.
Minor follow-up to commit 756ab291, which added the GiST functions.
Author: Andrey Borodin <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M contrib/pageinspect/expected/gist.out
M contrib/pageinspect/gistfuncs.c
M contrib/pageinspect/pageinspect–1.8–1.9.sql
M doc/src/sgml/pageinspect.sgml
Avoid misinterpreting GiST pages in pageinspect.
commit : fa41cf8f183ac5d702e91da567e9b3375c632081
author : Peter Geoghegan <[email protected]>
date : Sun, 14 Feb 2021 19:43:25 -0800
committer: Peter Geoghegan <[email protected]>
date : Sun, 14 Feb 2021 19:43:25 -0800
GistPageSetDeleted() sets pd_lower when deleting a page, and sets the
page contents to a GISTDeletedPageContents. Avoid treating deleted GiST
pages as regular slotted pages within pageinspect.
Oversight in commit 756ab291.
Author: Andrey Borodin <[email protected]>
M contrib/pageinspect/gistfuncs.c
Adjust lazy_scan_heap() accounting comments.
commit : 7cde6b13a9b630e2f04d91e2f17dedc2afee21c6
author : Peter Geoghegan <[email protected]>
date : Sun, 14 Feb 2021 19:28:37 -0800
committer: Peter Geoghegan <[email protected]>
date : Sun, 14 Feb 2021 19:28:37 -0800
Explain which particular LP_DEAD line pointers get accounted for by the
tups_vacuumed variable.
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/vacuumlazy.c
Default to wal_sync_method=fdatasync on FreeBSD.
commit : f900a79ecdc1864a6ead72c97c34a41012227eaf
author : Thomas Munro <[email protected]>
date : Mon, 15 Feb 2021 15:43:39 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 15 Feb 2021 15:43:39 +1300
FreeBSD 13 gained O_DSYNC, which would normally cause wal_sync_method to
choose open_datasync as its default value. That may not be a good
choice for all systems, and performs worse than fdatasync in some
scenarios. Let's preserve the existing default behavior for now.
Like commit 576477e73c4, which did the same for Linux, back-patch to all
supported releases.
Discussion: https://postgr.es/m/CA%2BhUKGLsAMXBQrCxCXoW-JsUYmdOL8ALYvaX%3DCrHqWxm-nWbGA%40mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/utils/misc/postgresql.conf.sample
M src/include/port/freebsd.h
Use pg_pwrite() in pg_test_fsync.
commit : 2c8b42b50df6cc5ba6987c400a857d6bbfdf0300
author : Thomas Munro <[email protected]>
date : Mon, 15 Feb 2021 15:08:04 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 15 Feb 2021 15:08:04 +1300
For consistency with the PostgreSQL behavior this test program is
intended to simulate, use pwrite() instead of lseek() + write().
Also fix the final "non-sync" test, which was opening and closing the
file for every write.
Discussion: https://postgr.es/m/CA%2BhUKGJjjid2BJsvjMALBTduo1ogdx2SPYaTQL3wAy8y2hc4nw%40mail.gmail.com
M src/bin/pg_test_fsync/pg_test_fsync.c
Fix the warnings introduced in commit ce0fdbfe97.
commit : d9b0767becf5f41e4f001d8381e6a89941efa5b2
author : Amit Kapila <[email protected]>
date : Mon, 15 Feb 2021 07:28:02 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 15 Feb 2021 07:28:02 +0530
Author: Amit Kapila
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/subscriptioncmds.c
M src/backend/replication/logical/tablesync.c
M src/include/replication/slot.h
M src/include/replication/worker_internal.h
Hold interrupts while running dsm_detach() callbacks.
commit : 637668fb1d17ad789e392a40ff09694ff1aabffb
author : Thomas Munro <[email protected]>
date : Mon, 15 Feb 2021 13:32:58 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 15 Feb 2021 13:32:58 +1300
While cleaning up after a parallel query or parallel index creation that
created temporary files, we could be interrupted by a statement timeout.
The error handling path would then fail to clean up the files when it
ran dsm_detach() again, because the callback was already popped off the
list. Prevent this hazard by holding interrupts while the cleanup code
runs.
Thanks to Heikki Linnakangas for this suggestion, and also to Kyotaro
Horiguchi, Masahiko Sawada, Justin Pryzby and Tom Lane for discussion of
this and earlier ideas on how to fix the problem.
Back-patch to all supported releases.
Reported-by: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/dsm.c
Add result size as argument of pg_cryptohash_final() for overflow checks
commit : b83dcf792869fb4a9270d17c961eab75f51c44e4
author : Michael Paquier <[email protected]>
date : Mon, 15 Feb 2021 10:18:34 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 15 Feb 2021 10:18:34 +0900
With its current design, a careless use of pg_cryptohash_final() could
would result in an out-of-bound write in memory as the size of the
destination buffer to store the result digest is not known to the
cryptohash internals, without the caller knowing about that. This
commit adds a new argument to pg_cryptohash_final() to allow such sanity
checks, and implements such defenses.
The internals of SCRAM for HMAC could be tightened a bit more, but as
everything is based on SCRAM_KEY_LEN with uses particular to this code
there is no need to complicate its interface more than necessary, and
this comes back to the refactoring of HMAC in core. Except that, this
minimizes the uses of the existing DIGEST_LENGTH variables, relying
instead on sizeof() for the result sizes. In ossp-uuid, this also makes
the code more defensive, as it already relied on dce_uuid_t being at
least the size of a MD5 digest.
This is in philosophy similar to cfc40d3 for base64.c and aef8948 for
hex.c.
Reported-by: Ranier Vilela
Author: Michael Paquier, Ranier Vilela
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAEudQAoqEGmcff3J4sTSV-R_16Monuz-UpJFbf_dnVH=APr02Q@mail.gmail.com
M contrib/pgcrypto/internal-sha2.c
M contrib/pgcrypto/internal.c
M contrib/uuid-ossp/uuid-ossp.c
M src/backend/libpq/auth-scram.c
M src/backend/replication/backup_manifest.c
M src/backend/utils/adt/cryptohashfuncs.c
M src/bin/pg_verifybackup/parse_manifest.c
M src/common/checksum_helper.c
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
M src/common/md5_common.c
M src/common/scram-common.c
M src/include/common/cryptohash.h
Minor fixes to improve regex debugging code.
commit : 2dd6733108f2bea07b0a3469e768bd900c0808b3
author : Tom Lane <[email protected]>
date : Sun, 14 Feb 2021 19:53:28 -0500
committer: Tom Lane <[email protected]>
date : Sun, 14 Feb 2021 19:53:28 -0500
When REG_DEBUG is defined, ensure that an un-filled "struct cnfa"
is all-zeroes, not just that it has nstates == 0. This is mainly
so that looking at "struct subre" structs in gdb doesn't distract
one with a lot of garbage fields during regex compilation.
Adjust some places that print debug output to have suitable fflush
calls afterwards.
In passing, correct an erroneous ancient comment: the concatenation
subre-s created by parsebranch() have op == '.' not ','.
Noted while fooling around with some regex performance improvements.
M src/backend/regex/regc_nfa.c
M src/backend/regex/regcomp.c
M src/include/regex/regguts.h
ReadNewTransactionId() -> ReadNextTransactionId().
commit : c7ecd6af010e2ac8c5530f3985e97f24531bfa5f
author : Thomas Munro <[email protected]>
date : Mon, 15 Feb 2021 13:03:10 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 15 Feb 2021 13:03:10 +1300
The new name conveys the effect better, is more consistent with similar
functions ReadNextMultiXactId(), ReadNextFullTransactionId(), and
matches the name of the variable that it reads.
Reported-by: Peter Geoghegan <[email protected]>
Discussion: https://postgr.es/m/CAH2-WzmVR4SakBXQUdhhPpMf1aYvZCnna5%3DHKa7DAgEmBAg%2B8g%40mail.gmail.com
M src/backend/access/gin/ginvacuum.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/xact.c
M src/backend/commands/vacuum.c
M src/backend/postmaster/autovacuum.c
M src/include/access/transam.h
README/C-comment: document GiST's NSN value
commit : 8facf1ea00b7a0c08c755a0392212b83e04ae28a
author : Bruce Momjian <[email protected]>
date : Sat, 13 Feb 2021 13:50:49 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 13 Feb 2021 13:50:49 -0500
M src/backend/access/gist/README
M src/include/access/gist.h
doc: Mention NO DEPENDS ON EXTENSION in its supported ALTER commands
commit : 8063d0f6f56e53edd991f53aadc8cb7f8d3fdd8f
author : Michael Paquier <[email protected]>
date : Sat, 13 Feb 2021 16:06:11 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 13 Feb 2021 16:06:11 +0900
This grammar flavor has been added by 5fc7039.
Author: Ian Lawrence Barwick
Discussion: https://postgr.es/m/CAB8KJ=ii6JScodxkA6-DO8bjatsMYU3OcewnL0mdN9geR+tTaw@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/ref/alter_index.sgml
M doc/src/sgml/ref/alter_materialized_view.sgml
M doc/src/sgml/ref/alter_procedure.sgml
M doc/src/sgml/ref/alter_routine.sgml
Tweak compiler version cutoff for no_sanitize("alignment") support.
commit : ad2ad698ac16f36a9784bc98f312cf5b5266c6aa
author : Tom Lane <[email protected]>
date : Fri, 12 Feb 2021 17:33:14 -0500
committer: Tom Lane <[email protected]>
date : Fri, 12 Feb 2021 17:33:14 -0500
Buildfarm results show that gcc up through 7.x produces annoying
warnings for this construct (and, presumably, wouldn't do the right
thing anyway). clang seems okay with the cutoff we have, though.
Discussion: https://postgr.es/m/CAPpHfdsne3%3DT%3DfMNU45PtxdhSL_J2PjLTeS8rwKnJzUR4YNd4w%40mail.gmail.com
Discussion: https://postgr.es/m/475514.1612745257%40sss.pgh.pa.us
M src/include/c.h
Avoid divide-by-zero in regex_selectivity() with long fixed prefix.
commit : ae4867ec749671dadb5eb49a1158ded9b8ca0b29
author : Tom Lane <[email protected]>
date : Fri, 12 Feb 2021 16:26:47 -0500
committer: Tom Lane <[email protected]>
date : Fri, 12 Feb 2021 16:26:47 -0500
Given a regex pattern with a very long fixed prefix (approaching 500
characters), the result of pow(FIXED_CHAR_SEL, fixed_prefix_len) can
underflow to zero. Typically the preceding selectivity calculation
would have underflowed as well, so that we compute 0/0 and get NaN.
In released branches this leads to an assertion failure later on.
That doesn't happen in HEAD, for reasons I've not explored yet,
but it's surely still a bug.
To fix, just skip the division when the pow() result is zero, so
that we'll (most likely) return a zero selectivity estimate. In
the edge cases where "sel" didn't yet underflow, perhaps this
isn't desirable, but I'm not sure that the case is worth spending
a lot of effort on. The results of regex_selectivity_sub() are
barely worth the electrons they're written on anyway :-(
Per report from Alexander Lakhin. Back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/like_support.c
pg_attribute_no_sanitize_alignment() macro
commit : 993bdb9f935a751935a03c80d30857150ba2b645
author : Alexander Korotkov <[email protected]>
date : Fri, 12 Feb 2021 17:14:33 +0300
committer: Alexander Korotkov <[email protected]>
date : Fri, 12 Feb 2021 17:14:33 +0300
Modern gcc and clang compilers offer alignment sanitizers, which help to detect
pointer misalignment. However, our codebase already contains x86-specific
crc32 computation code, which uses unalignment access. Thankfully, those
compilers also support the attribute, which disables alignment sanitizers at
the function level. This commit adds pg_attribute_no_sanitize_alignment(),
which wraps this attribute, and applies it to pg_comp_crc32c_sse42() function.
Discussion: https://postgr.es/m/CAPpHfdsne3%3DT%3DfMNU45PtxdhSL_J2PjLTeS8rwKnJzUR4YNd4w%40mail.gmail.com
Discussion: https://postgr.es/m/475514.1612745257%40sss.pgh.pa.us
Author: Alexander Korotkov, revised by Tom Lane
Reviewed-by: Tom Lane
M src/include/c.h
M src/port/pg_crc32c_sse42.c
Fix Subscription test added by commit ce0fdbfe97.
commit : c8b21b0378141948a2dc0ddde6f57407321787ea
author : Amit Kapila <[email protected]>
date : Fri, 12 Feb 2021 10:11:16 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 12 Feb 2021 10:11:16 +0530
We want to test the variants of Alter Subscription that are not allowed in
the transaction block but for that, we don't need to create a subscription
that tries to connect to the publisher. As such, there is no problem with
this test but it is good to allow such tests to run with
wal_level = minimal and max_wal_senders = 0 so as to keep them consistent
with other tests.
Reported by buildfarm.
Author: Amit Kapila
Reviewed-by: Ajin Cherian
Discussion: https://postgr.es/m/CAA4eK1Lw0V+e1JPGHDq=+hVACv=14H8sR+2eJ1k3PEgwKmU-jQ@mail.gmail.com
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
Allow multiple xacts during table sync in logical replication.
commit : ce0fdbfe9722867b7fad4d3ede9b6a6bfc51fb4e
author : Amit Kapila <[email protected]>
date : Fri, 12 Feb 2021 07:41:51 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 12 Feb 2021 07:41:51 +0530
For the initial table data synchronization in logical replication, we use
a single transaction to copy the entire table and then synchronize the
position in the stream with the main apply worker.
There are multiple downsides of this approach: (a) We have to perform the
entire copy operation again if there is any error (network breakdown,
error in the database operation, etc.) while we synchronize the WAL
position between tablesync worker and apply worker; this will be onerous
especially for large copies, (b) Using a single transaction in the
synchronization-phase (where we can receive WAL from multiple
transactions) will have the risk of exceeding the CID limit, (c) The slot
will hold the WAL till the entire sync is complete because we never commit
till the end.
This patch solves all the above downsides by allowing multiple
transactions during the tablesync phase. The initial copy is done in a
single transaction and after that, we commit each transaction as we
receive. To allow recovery after any error or crash, we use a permanent
slot and origin to track the progress. The slot and origin will be removed
once we finish the synchronization of the table. We also remove slot and
origin of tablesync workers if the user performs DROP SUBSCRIPTION .. or
ALTER SUBSCRIPTION .. REFERESH and some of the table syncs are still not
finished.
The commands ALTER SUBSCRIPTION ... REFRESH PUBLICATION and
ALTER SUBSCRIPTION ... SET PUBLICATION ... with refresh option as true
cannot be executed inside a transaction block because they can now drop
the slots for which we have no provision to rollback.
This will also open up the path for logical replication of 2PC
transactions on the subscriber side. Previously, we can't do that because
of the requirement of maintaining a single transaction in tablesync
workers.
Bump catalog version due to change of state in the catalog
(pg_subscription_rel).
Author: Peter Smith, Amit Kapila, and Takamichi Osumi
Reviewed-by: Ajin Cherian, Petr Jelinek, Hou Zhijie and Amit Kapila
Discussion: https://postgr.es/m/CAA4eK1KHJxaZS-fod-0fey=0tq3=Gkn4ho=8N4-5HWiCfu0H1A@mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/ref/alter_subscription.sgml
M doc/src/sgml/ref/drop_subscription.sgml
M src/backend/access/transam/xact.c
M src/backend/catalog/pg_subscription.c
M src/backend/commands/subscriptioncmds.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/launcher.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/backend/tcop/utility.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_subscription_rel.h
M src/include/commands/subscriptioncmds.h
M src/include/replication/logicallauncher.h
M src/include/replication/slot.h
M src/include/replication/walreceiver.h
M src/include/replication/worker_internal.h
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
M src/test/subscription/t/004_sync.pl
M src/tools/pgindent/typedefs.list
Remove obsolete IndexBulkDeleteResult stats field.
commit : 3063eb17593c3ad498ce4e89db3358862ea2dbb6
author : Peter Geoghegan <[email protected]>
date : Thu, 11 Feb 2021 16:49:41 -0800
committer: Peter Geoghegan <[email protected]>
date : Thu, 11 Feb 2021 16:49:41 -0800
The pages_removed field is no longer used for anything. It hasn't been
possible for an index to physically shrink since old-style VACUUM FULL
was removed by commit 0a469c87.
M src/backend/access/gist/gistvacuum.c
M src/include/access/genam.h
Remove dead code in ECPGconnect(), and improve documentation.
commit : 62535cae9723afc48173ba1be65f1c7491813fc2
author : Tom Lane <[email protected]>
date : Thu, 11 Feb 2021 15:05:55 -0500
committer: Tom Lane <[email protected]>
date : Thu, 11 Feb 2021 15:05:55 -0500
The stanza in ECPGconnect() that intended to allow specification of a
Unix socket directory path in place of a port has never executed since
it was committed, nearly two decades ago; the preceding strrchr()
already found the last colon so there cannot be another one. The lack
of complaints about that is doubtless related to the fact that no
user-facing documentation suggested it was possible.
Rather than try to fix that up, let's just remove the unreachable
code, and instead document the way that does work to write a socket
directory path, namely specifying it as a "host" option.
In support of that, make another pass at clarifying the syntax
documentation for ECPG connection targets, particularly documenting
which things are parsed as identifiers and where to use double quotes.
Rearrange some things that seemed poorly ordered, and fix a couple of
minor doc errors.
Kyotaro Horiguchi, per gripe from Shenhao Wang
(docs changes mostly by me)
Discussion: https://postgr.es/m/ae52a416bbbf459c96bab30b3038e06c@G08CNEXMBPEKD06.g08.fujitsu.local
M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/ecpglib/connect.c
Simplify jsonfuncs.c code by using strtoint() not strtol().
commit : 69036aafb9a8f425fb489125b5075ba7719d20d0
author : Tom Lane <[email protected]>
date : Thu, 11 Feb 2021 12:49:22 -0500
committer: Tom Lane <[email protected]>
date : Thu, 11 Feb 2021 12:49:22 -0500
Explicitly testing for INT_MIN and INT_MAX isn't particularly good
style; it's tedious and may draw useless compiler warnings on
machines where int and long are the same width. We invented
strtoint() precisely for this usage, so use that instead.
While here, remove gratuitous variations in the way the tests for
did-strtoint-succeed were spelled. Also, avoid attempting to
negate INT_MIN; that would probably work given that the result
is implicitly cast to uint32, but I think it's nominally undefined
behavior.
Per gripe from Ranier Vilela, though this isn't his proposed patch.
Discussion: https://postgr.es/m/CAEudQAqge3QfzoBRhe59QrB_5g+NmQUj2QpzqZ9Nc7QepXGAEw@mail.gmail.com
M src/backend/utils/adt/jsonfuncs.c
Remove no-longer-used RTE argument of markVarForSelectPriv().
commit : d4c746516b8b4eb0bf993e3729ccc04d1febdb1e
author : Tom Lane <[email protected]>
date : Thu, 11 Feb 2021 11:23:25 -0500
committer: Tom Lane <[email protected]>
date : Thu, 11 Feb 2021 11:23:25 -0500
In the wake of c028faf2a, this is no longer needed. I left it
out of that patch since the API change would be undesirable in
a released branch; but there's no reason not to do it in HEAD.
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_target.c
M src/include/parser/parse_relation.h
Fix copy-paste error with SHA256 digest length in checksum_helper.c
commit : 42d74e0c443760af91875af99d2655c9eb4c40df
author : Michael Paquier <[email protected]>
date : Thu, 11 Feb 2021 19:16:11 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 11 Feb 2021 19:16:11 +0900
Issue introduced by 87ae969, noticed while working on the area. While
on it, fix some grammar in the surrounding static assertions.
M src/common/checksum_helper.c
Add test case for abbrev(cidr)
commit : 69273c8f88d7e6295ed7b38114969fa3ea03eff2
author : Peter Eisentraut <[email protected]>
date : Thu, 11 Feb 2021 09:56:14 +0100
committer: Peter Eisentraut <[email protected]>
date : Thu, 11 Feb 2021 09:56:14 +0100
This will in particular add some good test coverage for
inet_cidr_ntop.c, which was previously completely uncovered.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/cb0c4662-4596-dab4-7f64-839c5e8582c8%40enterprisedb.com
M src/test/regress/expected/inet.out
M src/test/regress/sql/inet.sql
Remove extra Success message at the end of initdb
commit : e7f42914854926c2afbb89b9cd0e381fd90766be
author : Magnus Hagander <[email protected]>
date : Wed, 10 Feb 2021 18:21:55 +0100
committer: Magnus Hagander <[email protected]>
date : Wed, 10 Feb 2021 18:21:55 +0100
This was accidentally included in e09155bd62 and is redundant with the
lines right above it.
Reported-By: Peter Eisentraut
Discussion: https://postgr.es/m/[email protected]
M src/bin/initdb/initdb.c
pg_dump: Add const decorations
commit : 64990081504661ff5c04dbf20cc4252be66ab149
author : Peter Eisentraut <[email protected]>
date : Wed, 10 Feb 2021 13:08:13 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 10 Feb 2021 13:08:13 +0100
Add const decorations to the *info arguments of the dump* functions,
to clarify that they don't modify that argument. Many other nearby
functions modify their arguments, so this can help clarify these
different APIs a bit.
Discussion: https://www.postgresql.org/message-id/flat/012d3030-9a2c-99a1-ed2d-988978b5632f%40enterprisedb.com
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
Fix lack of message pluralization
commit : 4ad5611055c8381d6a9b54fc11838418a1f14bc9
author : Peter Eisentraut <[email protected]>
date : Wed, 10 Feb 2021 10:47:29 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 10 Feb 2021 10:47:29 +0100
M src/backend/access/transam/xlogfuncs.c
Fix ORDER BY clause in new regression test of REINDEX CONCURRENTLY
commit : 4f4726005082c07c004506a1ddfca29896375206
author : Michael Paquier <[email protected]>
date : Wed, 10 Feb 2021 16:59:04 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 10 Feb 2021 16:59:04 +0900
Oversight in bd12080.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Simplify code related to compilation of SSL and OpenSSL
commit : 092b785fad3de3f81355a4b2420aa39a1bc0ccd5
author : Michael Paquier <[email protected]>
date : Wed, 10 Feb 2021 15:28:19 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 10 Feb 2021 15:28:19 +0900
This commit makes more generic some comments and code related to the
compilation with OpenSSL and SSL in general to ease the addition of more
SSL implementations in the future. In libpq, some OpenSSL-only code is
moved under USE_OPENSSL and not USE_SSL.
While on it, make a comment more consistent in libpq-fe.h.
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/hba.c
M src/include/libpq/libpq-be.h
M src/include/pg_config_manual.h
M src/interfaces/libpq/fe-secure.c
M src/interfaces/libpq/libpq-fe.h
Preserve pg_attribute.attstattarget across REINDEX CONCURRENTLY
commit : bd12080980297dbc8ae926a3bd5b2ef9cc47932b
author : Michael Paquier <[email protected]>
date : Wed, 10 Feb 2021 13:06:48 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 10 Feb 2021 13:06:48 +0900
For an index, attstattarget can be updated using ALTER INDEX SET
STATISTICS. This data was lost on the new index after REINDEX
CONCURRENTLY.
The update of this field is done when the old and new indexes are
swapped to make the fix back-patchable. Another approach we could look
after in the long-term is to change index_create() to pass the wanted
values of attstattarget when creating the new relation, but, as this
would cause an ABI breakage this can be done only on HEAD.
Reported-by: Ronan Dunklau
Author: Michael Paquier
Reviewed-by: Ronan Dunklau, Tomas Vondra
Discussion: https://postgr.es/m/16628084.uLZWGnKmhe@laptop-ronand
Backpatch-through: 12
M src/backend/catalog/index.c
M src/backend/utils/cache/lsyscache.c
M src/include/utils/lsyscache.h
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Make pg_replication_origin_drop safe against concurrent drops.
commit : cd142e032ebd50ec7974b3633269477c2c72f1cc
author : Amit Kapila <[email protected]>
date : Wed, 10 Feb 2021 07:17:09 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 10 Feb 2021 07:17:09 +0530
Currently, we get the origin id from the name and then drop the origin by
taking ExclusiveLock on ReplicationOriginRelationId. So, two concurrent
sessions can get the id from the name at the same time and then when they
try to drop the origin, one of the sessions will get the either
"tuple concurrently deleted" or "cache lookup failed for replication
origin ..".
To prevent this race condition we do the entire operation under lock. This
obviates the need for replorigin_drop() API and we have removed it so if
any extension authors are using it they need to instead use
replorigin_drop_by_name. See it's usage in pg_replication_origin_drop().
Author: Peter Smith
Reviewed-by: Amit Kapila, Euler Taveira, Petr Jelinek, and Alvaro
Herrera
Discussion: https://www.postgresql.org/message-id/CAHut%2BPuW8DWV5fskkMWWMqzt-x7RPcNQOtJQBp6SdwyRghCk7A%40mail.gmail.com
M src/backend/commands/subscriptioncmds.c
M src/backend/replication/logical/origin.c
M src/include/replication/origin.h
Fix obsolete FSM remarks in nbtree README.
commit : 31c7fb41e26bf03dae231c7165a1a16388b2e366
author : Peter Geoghegan <[email protected]>
date : Tue, 9 Feb 2021 11:36:51 -0800
committer: Peter Geoghegan <[email protected]>
date : Tue, 9 Feb 2021 11:36:51 -0800
The free space map has used a dedicated relation fork rather than shared
memory segments for over a decade.
M src/backend/access/nbtree/README
Revert "Display the time when the process started waiting for the lock, in pg_locks."
commit : 890d2182a2c425aaa80f9bf9f7116d31e0c6538e
author : Fujii Masao <[email protected]>
date : Tue, 9 Feb 2021 18:30:40 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 9 Feb 2021 18:30:40 +0900
This reverts commit 3b733fcd04195399db56f73f0616b4f5c6828e18.
Per buildfarm members prion and rorqual.
M contrib/amcheck/expected/check_btree.out
M doc/src/sgml/catalogs.sgml
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/adt/lockfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/storage/lock.h
M src/include/storage/proc.h
M src/test/regress/expected/rules.out
Display the time when the process started waiting for the lock, in pg_locks.
commit : 3b733fcd04195399db56f73f0616b4f5c6828e18
author : Fujii Masao <[email protected]>
date : Tue, 9 Feb 2021 18:10:19 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 9 Feb 2021 18:10:19 +0900
This commit adds new column "waitstart" into pg_locks view. This column
reports the time when the server process started waiting for the lock
if the lock is not held. This information is useful, for example, when
examining the amount of time to wait on a lock by subtracting
"waitstart" in pg_locks from the current time, and identify the lock
that the processes are waiting for very long.
This feature uses the current time obtained for the deadlock timeout
timer as "waitstart" (i.e., the time when this process started waiting
for the lock). Since getting the current time newly can cause overhead,
we reuse the already-obtained time to avoid that overhead.
Note that "waitstart" is updated without holding the lock table's
partition lock, to avoid the overhead by additional lock acquisition.
This can cause "waitstart" in pg_locks to become NULL for a very short
period of time after the wait started even though "granted" is false.
This is OK in practice because we can assume that users are likely to
look at "waitstart" when waiting for the lock for a long time.
Bump catalog version.
Author: Atsushi Torikoshi
Reviewed-by: Ian Lawrence Barwick, Robert Haas, Justin Pryzby, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M contrib/amcheck/expected/check_btree.out
M doc/src/sgml/catalogs.sgml
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/adt/lockfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/storage/lock.h
M src/include/storage/proc.h
M src/test/regress/expected/rules.out
Add option PROCESS_TOAST to VACUUM
commit : 7cb3048f38e26b39dd5fd412ed8a4981b6809b35
author : Michael Paquier <[email protected]>
date : Tue, 9 Feb 2021 14:13:57 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 9 Feb 2021 14:13:57 +0900
This option controls if toast tables associated with a relation are
vacuumed or not when running a manual VACUUM. It was already possible
to trigger a manual VACUUM on a toast relation without processing its
main relation, but a manual vacuum on a main relation always forced a
vacuum on its toast table. This is useful in scenarios where the level
of bloat or transaction age of the main and toast relations differs a
lot.
This option is an extension of the existing VACOPT_SKIPTOAST that was
used by autovacuum to control if toast relations should be skipped or
not. This internal flag is renamed to VACOPT_PROCESS_TOAST for
consistency with the new option.
A new option switch, called --no-process-toast, is added to vacuumdb.
Author: Nathan Bossart
Reviewed-by: Kirk Jamison, Michael Paquier, Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/vacuum.sgml
M doc/src/sgml/ref/vacuumdb.sgml
M src/backend/commands/vacuum.c
M src/backend/postmaster/autovacuum.c
M src/bin/psql/tab-complete.c
M src/bin/scripts/t/100_vacuumdb.pl
M src/bin/scripts/vacuumdb.c
M src/include/commands/vacuum.h
M src/test/regress/expected/vacuum.out
M src/test/regress/sql/vacuum.sql
Correct pgstattuple B-Tree page comments.
commit : 5fd590021d268190e4c8f377370c7e7f1e7d9229
author : Peter Geoghegan <[email protected]>
date : Mon, 8 Feb 2021 15:20:08 -0800
committer: Peter Geoghegan <[email protected]>
date : Mon, 8 Feb 2021 15:20:08 -0800
M contrib/pgstattuple/pgstattuple.c
Fix mishandling of column-level SELECT privileges for join aliases.
commit : c028faf2a62ce476c8d689a33b12471dc98b80f2
author : Tom Lane <[email protected]>
date : Mon, 8 Feb 2021 10:14:09 -0500
committer: Tom Lane <[email protected]>
date : Mon, 8 Feb 2021 10:14:09 -0500
scanNSItemForColumn, expandNSItemAttrs, and ExpandSingleTable would
pass the wrong RTE to markVarForSelectPriv when dealing with a join
ParseNamespaceItem: they'd pass the join RTE, when what we need to
mark is the base table that the join column came from. The end
result was to not fill the base table's selectedCols bitmap correctly,
resulting in an understatement of the set of columns that are read
by the query. The executor would still insist on there being at
least one selectable column; but with a correctly crafted query,
a user having SELECT privilege on just one column of a table would
nonetheless be allowed to read all its columns.
To fix, make markRTEForSelectPriv fetch the correct RTE for itself,
ignoring the possibly-mismatched RTE passed by the caller. Later,
we'll get rid of some now-unused RTE arguments, but that risks
API breaks so we won't do it in released branches.
This problem was introduced by commit 9ce77d75c, so back-patch
to v13 where that came in. Thanks to Sven Klemm for reporting
the problem.
Security: CVE-2021-20229
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_target.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Fix permission checks on constraint violation errors on partitions.
commit : 6214e2b2280462cbc3aa1986e350e167651b3905
author : Heikki Linnakangas <[email protected]>
date : Mon, 8 Feb 2021 11:01:51 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 8 Feb 2021 11:01:51 +0200
If a cross-partition UPDATE violates a constraint on the target partition,
and the columns in the new partition are in different physical order than
in the parent, the error message can reveal columns that the user does not
have SELECT permission on. A similar bug was fixed earlier in commit
804b6b6db4.
The cause of the bug is that the callers of the
ExecBuildSlotValueDescription() function got confused when constructing
the list of modified columns. If the tuple was routed from a parent, we
converted the tuple to the parent's format, but the list of modified
columns was grabbed directly from the child's RTE entry.
ExecUpdateLockMode() had a similar issue. That lead to confusion on which
columns are key columns, leading to wrong tuple lock being taken on tables
referenced by foreign keys, when a row is updated with INSERT ON CONFLICT
UPDATE. A new isolation test is added for that corner case.
With this patch, the ri_RangeTableIndex field is no longer set for
partitions that don't have an entry in the range table. Previously, it was
set to the RTE entry of the parent relation, but that was confusing.
NOTE: This modifies the ResultRelInfo struct, replacing the
ri_PartitionRoot field with ri_RootResultRelInfo. That's a bit risky to
backpatch, because it breaks any extensions accessing the field. The
change that ri_RangeTableIndex is not set for partitions could potentially
break extensions, too. The ResultRelInfos are visible to FDWs at least,
and this patch required small changes to postgres_fdw. Nevertheless, this
seem like the least bad option. I don't think these fields widely used in
extensions; I don't think there are FDWs out there that uses the FDW
"direct update" API, other than postgres_fdw. If there is, you will get a
compilation error, so hopefully it is caught quickly.
Backpatch to 11, where support for both cross-partition UPDATEs, and unique
indexes on partitioned tables, were added.
Reviewed-by: Amit Langote
Security: CVE-2021-3393
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/access/common/tupconvert.c
M src/backend/commands/copyfrom.c
M src/backend/commands/explain.c
M src/backend/commands/trigger.c
M src/backend/executor/execIndexing.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeModifyTable.c
M src/include/access/tupconvert.h
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
A src/test/isolation/expected/tuplelock-partition.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/tuplelock-partition.spec
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Rename removable xid function for consistency.
commit : 617fffee8a6f350ff03069e2843ecd039ea06ccc
author : Peter Geoghegan <[email protected]>
date : Sun, 7 Feb 2021 10:11:14 -0800
committer: Peter Geoghegan <[email protected]>
date : Sun, 7 Feb 2021 10:11:14 -0800
GlobalVisIsRemovableFullXid() is now GlobalVisCheckRemovableFullXid().
This is consistent with the general convention for FullTransactionId
equivalents of functions that deal with TransactionId values. It now
matches the nearby GlobalVisCheckRemovableXid() function, which performs
the same check for callers that use TransactionId values.
Oversight in commit dc7420c2c92.
Discussion: https://postgr.es/m/CAH2-Wzmes12jFNDcVgpU89Vp=r6uLFrE-MT0fjSWGsE70UiNaA@mail.gmail.com
M src/backend/access/gist/gistutil.c
M src/backend/access/gist/gistxlog.c
M src/backend/storage/ipc/procarray.c
M src/include/utils/snapmgr.h
Revert "Propagate CTE property flags when copying a CTE list into a rule."
commit : d1d2979852538d7021cc809a40ef127d59747697
author : Tom Lane <[email protected]>
date : Sun, 7 Feb 2021 12:54:08 -0500
committer: Tom Lane <[email protected]>
date : Sun, 7 Feb 2021 12:54:08 -0500
This reverts commit ed290896335414c6c069b9ccae1f3dcdd2fac6ba and
equivalent back-branch commits. The issue is subtler than I thought,
and it's far from new, so just before a release deadline is no time
to be fooling with it. We'll consider what to do at a bit more
leisure.
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Docs: fix pg_wal_lsn_diff manual.
commit : 04fd3eeba5be52369fa296fb001d1e52af6e166d
author : Tatsuo Ishii <[email protected]>
date : Sun, 7 Feb 2021 13:43:50 +0900
committer: Tatsuo Ishii <[email protected]>
date : Sun, 7 Feb 2021 13:43:50 +0900
The manual did not mention whether its return value is (first arg -
second arg) or (second arg - first arg). The order matters because the
return value could have a sign. Fix the manual so that it mentions the
function returns (first arg - second arg).
Patch reviewed by Tom Lane.
Back-patch through v13. Older version's doc format is difficult to add
more description.
Discussion: https://postgr.es/m/flat/20210206.151125.960423226279810864.t-ishii%40sraoss.co.jp
M doc/src/sgml/func.sgml
Propagate CTE property flags when copying a CTE list into a rule.
commit : ed290896335414c6c069b9ccae1f3dcdd2fac6ba
author : Tom Lane <[email protected]>
date : Sat, 6 Feb 2021 19:28:39 -0500
committer: Tom Lane <[email protected]>
date : Sat, 6 Feb 2021 19:28:39 -0500
rewriteRuleAction() neglected this step, although it was careful to
propagate other similar flags such as hasSubLinks or hasRowSecurity.
Omitting to transfer hasRecursive is just cosmetic at the moment,
but omitting hasModifyingCTE is a live bug, since the executor
certainly looks at that.
The proposed test case only fails back to v10, but since the executor
examines hasModifyingCTE in 9.x as well, I suspect that a test case
could be devised that fails in older branches. Given the nearness
of the release deadline, though, I'm not going to spend time looking
for a better test.
Report and patch by Greg Nancarrow, cosmetic changes by me
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Disallow converting an inheritance child table to a view.
commit : dd705a039f6cd41921529fa4e971d70b224be052
author : Tom Lane <[email protected]>
date : Sat, 6 Feb 2021 15:17:01 -0500
committer: Tom Lane <[email protected]>
date : Sat, 6 Feb 2021 15:17:01 -0500
Generally, members of inheritance trees must be plain tables (or,
in more recent versions, foreign tables). ALTER TABLE INHERIT
rejects creating an inheritance relationship that has a view at
either end. When DefineQueryRewrite attempts to convert a relation
to a view, it already had checks prohibiting doing so for partitioning
parents or children as well as traditional-inheritance parents ...
but it neglected to check that a traditional-inheritance child wasn't
being converted. Since the planner assumes that any inheritance
child is a table, this led to making plans that tried to do a physical
scan on a view, causing failures (or even crashes, in recent versions).
One could imagine trying to support such a case by expanding the view
normally, but since the rewriter runs before the planner does
inheritance expansion, it would take some very fundamental refactoring
to make that possible. There are probably a lot of other parts of the
system that don't cope well with such a situation, too. For now,
just forbid it.
Per bug #16856 from Yang Lin. Back-patch to all supported branches.
(In versions before v10, this includes back-patching the portion of
commit 501ed02cf that added has_superclass(). Perhaps the lack of
that infrastructure partially explains the missing check.)
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/pg_inherits.c
M src/backend/rewrite/rewriteDefine.c
M src/test/regress/expected/rules.out
M src/test/regress/sql/rules.sql
Clarify some comments around SharedRecoveryState in xlog.c
commit : f7400823c3bd6ce03c2fe1bec5b7066bad146809
author : Michael Paquier <[email protected]>
date : Sat, 6 Feb 2021 10:27:55 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 6 Feb 2021 10:27:55 +0900
SharedRecoveryState has been switched from a boolean to an enum as of
commit 4e87c48, but some comments still referred to it as a boolean.
Author: Amul Sul
Reviewed-by: Dilip Kumar, Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAAJ_b97Hf+1SXnm8jySpO+Fhm+-VKFAAce1T_cupUYtnE3Nxig
M src/backend/access/transam/xlog.c
Generalize parallel slot result handling.
commit : 418611c84d004f45d92bcaa3f8e100385d96cd41
author : Robert Haas <[email protected]>
date : Fri, 5 Feb 2021 16:08:45 -0500
committer: Robert Haas <[email protected]>
date : Fri, 5 Feb 2021 16:08:45 -0500
Instead of having a hard-coded behavior that we ignore missing
tables and report all other errors, let the caller decide what
to do by setting a callback.
Mark Dilger, reviewed and somewhat revised by me. The larger patch
series of which this is a part has also had review from Peter
Geoghegan, Andres Freund, Álvaro Herrera, Michael Paquier, and Amul
Sul, but I don't know whether any of them have reviewed this bit
specifically.
Discussion: http://postgr.es/m/[email protected]
Discussion: http://postgr.es/m/[email protected]
Discussion: http://postgr.es/m/[email protected]
M src/bin/scripts/reindexdb.c
M src/bin/scripts/vacuumdb.c
M src/fe_utils/parallel_slot.c
M src/include/fe_utils/parallel_slot.h
Move some code from src/bin/scripts to src/fe_utils to permit reuse.
commit : e955bd4b6c2bcdbd253837f6cf4c7520b98e69d4
author : Robert Haas <[email protected]>
date : Fri, 5 Feb 2021 13:33:38 -0500
committer: Robert Haas <[email protected]>
date : Fri, 5 Feb 2021 13:33:38 -0500
The parallel slots infrastructure (which implements client-side
multiplexing of server connections doing similar things, not
threading or multiple processes or anything like that) are moved from
src/bin/scripts/scripts_parallel.c to src/fe_utils/parallel_slot.c.
The functions consumeQueryResult() and processQueryResult() which were
previously part of src/bin/scripts/common.c are now moved into that
file as well, becoming static helper functions. This might need to be
changed in the future, but currently they're not used for anything
else.
Some other functions from src/bin/scripts/common.c are moved to to
src/fe_utils and are split up among several files. connectDatabase(),
connectMaintenanceDatabase(), and disconnectDatabase() are moved to
connect_utils.c. executeQuery(), executeCommand(), and
executeMaintenanceCommand() are move to query_utils.c.
handle_help_version_opts() is moved to option_utils.c.
Mark Dilger, reviewed by me. The larger patch series of which this is
a part has also had review from Peter Geoghegan, Andres Freund, Álvaro
Herrera, Michael Paquier, and Amul Sul, but I don't know whether any
of them have reviewed this bit specifically.
Discussion: http://postgr.es/m/[email protected]
Discussion: http://postgr.es/m/[email protected]
Discussion: http://postgr.es/m/[email protected]
M src/bin/scripts/Makefile
M src/bin/scripts/clusterdb.c
M src/bin/scripts/common.c
M src/bin/scripts/common.h
M src/bin/scripts/createdb.c
M src/bin/scripts/createuser.c
M src/bin/scripts/dropdb.c
M src/bin/scripts/dropuser.c
M src/bin/scripts/nls.mk
M src/bin/scripts/pg_isready.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/vacuumdb.c
M src/fe_utils/Makefile
A src/fe_utils/connect_utils.c
A src/fe_utils/option_utils.c
R080 src/bin/scripts/scripts_parallel.c src/fe_utils/parallel_slot.c
A src/fe_utils/query_utils.c
A src/include/fe_utils/connect_utils.h
A src/include/fe_utils/option_utils.h
R082 src/bin/scripts/scripts_parallel.h src/include/fe_utils/parallel_slot.h
A src/include/fe_utils/query_utils.h
M src/tools/msvc/Mkvcbuild.pm
Fix backslash-escaping multibyte chars in COPY FROM.
commit : c444472af5c202067a9ecb0ff8df7370fb1ea8f4
author : Heikki Linnakangas <[email protected]>
date : Fri, 5 Feb 2021 11:14:56 +0200
committer: Heikki Linnakangas <[email protected]>
date : Fri, 5 Feb 2021 11:14:56 +0200
If a multi-byte character is escaped with a backslash in TEXT mode input,
and the encoding is one of the client-only encodings where the bytes after
the first one can have an ASCII byte "embedded" in the char, we didn't
skip the character correctly. After a backslash, we only skipped the first
byte of the next character, so if it was a multi-byte character, we would
try to process its second byte as if it was a separate character. If it
was one of the characters with special meaning, like '\n', '\r', or
another '\\', that would cause trouble.
One such exmple is the byte sequence '\x5ca45c2e666f6f' in Big5 encoding.
That's supposed to be [backslash][two-byte character][.][f][o][o], but
because the second byte of the two-byte character is 0x5c, we incorrectly
treat it as another backslash. And because the next character is a dot, we
parse it as end-of-copy marker, and throw an "end-of-copy marker corrupt"
error.
Backpatch to all supported versions.
Reviewed-by: John Naylor, Kyotaro Horiguchi
Discussion: https://www.postgresql.org/message-id/a897f84f-8dca-8798-3139-07da5bb38728%40iki.fi
M src/backend/commands/copyfromparse.c
postgres_fdw: Fix assertion in estimate_path_cost_size().
commit : 5e7fa189ee92d5ecf42a295c336625d71bfe876d
author : Etsuro Fujita <[email protected]>
date : Fri, 5 Feb 2021 15:30:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Fri, 5 Feb 2021 15:30:00 +0900
Commit 08d2d58a2 added an assertion assuming that the retrieved_rows
estimate for a foreign relation, which is re-used to cost pre-sorted
foreign paths with local stats, is set to at least one row in
estimate_path_cost_size(), which isn't correct because if the relation
is a foreign table with tuples=0, the estimate would be set to 0 there
when not using remote estimates.
Per bug #16807 from Alexander Lakhin. Back-patch to v13 where the
aforementioned commit went in.
Author: Etsuro Fujita
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/16807-9fe4e08fbaa5c7ce%40postgresql.org
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
Fix bug in HashAgg's selective-column-spilling logic.
commit : 0ff865fbe50e82f17df8a9280fa01faf270b7f3f
author : Tom Lane <[email protected]>
date : Thu, 4 Feb 2021 23:01:33 -0500
committer: Tom Lane <[email protected]>
date : Thu, 4 Feb 2021 23:01:33 -0500
Commit 230230223 taught nodeAgg.c that, when spilling tuples from
memory in an oversized hash aggregation, it only needed to spill
input columns referenced in the node's tlist and quals. Unfortunately,
that's wrong: we also have to save the grouping columns. The error
is masked in common cases because the grouping columns also appear
in the tlist, but that's not necessarily true. The main category
of plans where it's not true seem to come from semijoins ("WHERE
outercol IN (SELECT innercol FROM innertable)") where the innercol
needs an implicit promotion to make it comparable to the outercol.
The grouping column will be "innercol::promotedtype", but that
expression appears nowhere in the Agg node's own tlist and quals;
only the bare "innercol" is found in the tlist.
I spent quite a bit of time looking for a suitable regression test
case for this, without much success. If the number of distinct
values of the innercol is large enough to make spilling happen,
the planner tends to prefer a non-HashAgg plan, at least for
problem sizes that are reasonable to use in the regression tests.
So, no new regression test. However, this patch does demonstrably
fix the originally-reported test case.
Per report from s.p.e (at) gmx-topmail.de. Backpatch to v13
where the troublesome code came in.
Discussion: https://postgr.es/m/trinity-1c565d44-159f-488b-a518-caf13883134f-1611835701633@3c-app-gmx-bap78
M src/backend/executor/nodeAgg.c
Tab-complete CREATE DATABASE ... LOCALE.
commit : e1c02d92aee30328316c309c3c2f305d77f231a2
author : Thomas Munro <[email protected]>
date : Fri, 5 Feb 2021 15:30:56 +1300
committer: Thomas Munro <[email protected]>
date : Fri, 5 Feb 2021 15:30:56 +1300
Author: Ian Lawrence Barwick <[email protected]>
Discussion: https://postgr.es/m/CAB8KJ%3Dh0XO2CB4QbLBc1Tm9Bg5wzSGQtT-eunaCmrghJp4nqdA%40mail.gmail.com
M src/bin/psql/tab-complete.c
Fix YA incremental sort bug.
commit : 82e0e29308dedbc6000e73329beb112ae7e1ad39
author : Tom Lane <[email protected]>
date : Thu, 4 Feb 2021 19:12:09 -0500
committer: Tom Lane <[email protected]>
date : Thu, 4 Feb 2021 19:12:09 -0500
switchToPresortedPrefixMode() did the wrong thing if it detected
a batch boundary just at the last tuple of a fullsort group.
The initially-reported symptom was a "retrieved too many tuples in a
bounded sort" error, but the test case added here just silently gives
the wrong answer without this patch.
I (tgl) am not really happy about committing this patch without review
from the incremental-sort authors, but they seem AWOL and we are hard
against a release deadline. This does demonstrably make some cases
better, anyway.
Per bug #16846 from Yoran Heling. Back-patch to v13 where incremental
sort was introduced.
Neil Chen
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/nodeIncrementalSort.c
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Harden nbtree page deletion.
commit : c34787f910585f82320f78b0afd53a6a170aa229
author : Peter Geoghegan <[email protected]>
date : Thu, 4 Feb 2021 15:42:36 -0800
committer: Peter Geoghegan <[email protected]>
date : Thu, 4 Feb 2021 15:42:36 -0800
Add some additional defensive checks in the second phase of index
deletion to detect and report index corruption during VACUUM, and to
avoid having VACUUM become stuck in more cases. The code is still not
robust in the presence of a circular chain of sibling links, though it's
not clear whether that really matters. This is follow-up work to commit
3a01f68e.
The new defensive checks rely on the assumption that there can be no
more than one VACUUM operation running for an index at any given time.
Remove an old comment suggesting that multiple concurrent VACUUMs need
to be considered here. This concern now seems highly unlikely to have
any real validity, since we clearly rely on the same assumption in
several other places. For example, there are much more recent comments
that appear in the same function (added by commit efada2b8e92) that make
the same assumption.
Also add a CHECK_FOR_INTERRUPTS() to the relevant code path. Contrary
to comments added by commit 3a01f68e, it is actually possible to handle
interrupts here, at least in the common case where processing takes
place at the leaf level. We only hold a pin on leafbuf/target page when
stepping right at the leaf level.
No backpatch due to the lack of complaints following hardening added to
the same area by commit 3a01f68e.
M src/backend/access/nbtree/nbtpage.c
Fix small error in COPY FROM progress reporting.
commit : 2f86ab305e7fbc7b84960079551cf9cafd29684f
author : Heikki Linnakangas <[email protected]>
date : Thu, 4 Feb 2021 17:40:33 +0200
committer: Heikki Linnakangas <[email protected]>
date : Thu, 4 Feb 2021 17:40:33 +0200
The # of bytes processed was accumulated slightly incorrectly. After
loading more data to the input buffer, we added the number of bytes in
the buffer to the sum. But in case of multi-byte characters or escapes,
there can be a few unprocessed bytes left over from previous load in the
buffer. Those bytes got counted twice.
M src/backend/commands/copyfromparse.c
Refactor Windows error message for easier translation
commit : 3c78e0569ca04f4c92f0adcd74471398bb7b2e55
author : Peter Eisentraut <[email protected]>
date : Thu, 4 Feb 2021 13:31:13 +0100
committer: Peter Eisentraut <[email protected]>
date : Thu, 4 Feb 2021 13:31:13 +0100
In the error messages referring to the user right "Lock pages in
memory", this is a term from the Windows OS, so it should be
translated in accordance with the OS localization. Refactor the error
messages so this is easier and clearer. Also fix the capitalization
to match the existing capitalization in the OS.
M doc/src/sgml/config.sgml
M src/backend/port/win32_shmem.c
Ensure unlinking of old index file with REINDEX (TABLESPACE)
commit : 5128483d064038702f535aced2cbaa43256db214
author : Michael Paquier <[email protected]>
date : Thu, 4 Feb 2021 17:16:47 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 4 Feb 2021 17:16:47 +0900
The original versions of the patch included this part, but a mismerge
from my side has made this piece go missing. Oversight in c5b28604.
M src/backend/catalog/index.c
Clarify comment in tablesync.c
commit : fc749bc7041cb77b5f6b58c129ad2616a3f7ab4f
author : Michael Paquier <[email protected]>
date : Thu, 4 Feb 2021 16:02:31 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 4 Feb 2021 16:02:31 +0900
Author: Peter Smith
Reviewed-by: Amit Kapila, Michael Paquier, Euler Taveira
Discussion: https://postgr.es/m/CAHut+Pt9_T6pWar0FLtPsygNmme8HPWPdGUyZ_8mE1Yvjdf0ZA@mail.gmail.com
M src/backend/replication/logical/tablesync.c
Add TABLESPACE option to REINDEX
commit : c5b286047cd698021e57a527215b48865fd4ad4e
author : Michael Paquier <[email protected]>
date : Thu, 4 Feb 2021 14:34:20 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 4 Feb 2021 14:34:20 +0900
This patch adds the possibility to move indexes to a new tablespace
while rebuilding them. Both the concurrent and the non-concurrent cases
are supported, and the following set of restrictions apply:
- When using TABLESPACE with a REINDEX command that targets a
partitioned table or index, all the indexes of the leaf partitions are
moved to the new tablespace. The tablespace references of the non-leaf,
partitioned tables in pg_class.reltablespace are not changed. This
requires an extra ALTER TABLE SET TABLESPACE.
- Any index on a toast table rebuilt as part of a parent table is kept
in its original tablespace.
- The operation is forbidden on system catalogs, including trying to
directly move a toast relation with REINDEX. This results in an error
if doing REINDEX on a single object. REINDEX SCHEMA, DATABASE and
SYSTEM skip system relations when TABLESPACE is used.
Author: Alexey Kondratov, Michael Paquier, Justin Pryzby
Reviewed-by: Álvaro Herrera, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/reindex.sgml
M src/backend/catalog/index.c
M src/backend/commands/indexcmds.c
M src/bin/psql/tab-complete.c
M src/include/catalog/index.h
M src/test/regress/input/tablespace.source
M src/test/regress/output/tablespace.source
Avoid crash when rolling back within a prepared statement.
commit : 9624321ec502f4e4f4722290b358694049447f95
author : Tom Lane <[email protected]>
date : Wed, 3 Feb 2021 19:38:29 -0500
committer: Tom Lane <[email protected]>
date : Wed, 3 Feb 2021 19:38:29 -0500
If a portal is used to run a prepared CALL or DO statement that
contains a ROLLBACK, PortalRunMulti fails because the portal's
statement list gets cleared by the rollback. (Since the grammar
doesn't allow CALL/DO in PREPARE, the only easy way to get to this is
via extended query protocol, which treats all inputs as prepared
statements.) It's difficult to avoid resetting the portal early
because of resource-management issues, so work around this by teaching
PortalRunMulti to be wary of portal->stmts having suddenly become NIL.
The crash has only been seen to occur in v13 and HEAD (as a
consequence of commit 1cff1b95a having added an extra touch of
portal->stmts). But even before that, the code involved touching a
List that the portal no longer has any claim on. In the test case at
hand, the List will still exist because of another refcount on the
cached plan; but I'm far from convinced that it's impossible for the
cached plan to have been dropped by the time control gets back to
PortalRunMulti. Hence, backpatch to v11 where nested transactions
were added.
Thomas Munro and Tom Lane, per bug #16811 from James Inform
Discussion: https://postgr.es/m/[email protected]
M src/backend/tcop/pquery.c
Factor pattern-construction logic out of processSQLNamePattern.
commit : 2c8726c4b0a496608919d1f78a5abc8c9b6e0868
author : Robert Haas <[email protected]>
date : Wed, 3 Feb 2021 13:19:41 -0500
committer: Robert Haas <[email protected]>
date : Wed, 3 Feb 2021 13:19:41 -0500
The logic for converting the shell-glob-like syntax supported by
utilities like psql and pg_dump to regular expression is
extracted into a new function patternToSQLRegex. The existing
function processSQLNamePattern now uses this function as a
subroutine.
patternToSQLRegex is a little more general than what is required
by processSQLNamePattern. That function is only interested in
patterns that can have up to 2 parts, a schema and a relation;
but patternToSQLRegex can limit the maximum number of parts to
between 1 and 3, so that patterns can look like either
"database.schema.relation", "schema.relation", or "relation"
depending on how it's invoked and what the user specifies.
processSQLNamePattern only passes two buffers, so works exactly
the same as before, always interpreting the pattern as either
a "schema.relation" pattern or a "relation" pattern. But,
future callers can use this function in other ways.
Mark Dilger, reviewed by me. The larger patch series of which this is
a part has also had review from Peter Geoghegan, Andres Freund, Álvaro
Herrera, Michael Paquier, and Amul Sul, but I don't know whether
any of them have reviewed this bit specifically.
Discussion: http://postgr.es/m/[email protected]
Discussion: http://postgr.es/m/[email protected]
Discussion: http://postgr.es/m/[email protected]
M src/fe_utils/string_utils.c
M src/include/fe_utils/string_utils.h
Remove special BKI_LOOKUP magic for namespace and role OIDs.
commit : ba0faf81c65ac99dd42ce192f3257d4d2231ea50
author : Tom Lane <[email protected]>
date : Wed, 3 Feb 2021 12:01:48 -0500
committer: Tom Lane <[email protected]>
date : Wed, 3 Feb 2021 12:01:48 -0500
Now that commit 62f34097c attached BKI_LOOKUP annotation to all the
namespace and role OID columns in the catalogs, there's no real reason
to have the magic PGNSP and PGUID symbols. Get rid of them in favor
of implementing those lookups according to genbki.pl's normal pattern.
This means that in the catalog headers, BKI_DEFAULT(PGNSP) becomes
BKI_DEFAULT(pg_catalog), which seems a lot more transparent.
BKI_DEFAULT(PGUID) becomes BKI_DEFAULT(POSTGRES), which is perhaps
less so; but you can look into pg_authid.dat to discover that
POSTGRES is the nonce name for the bootstrap superuser.
This change also means that if we ever need cross-references in the
initial catalog data to any of the other built-in roles besides
POSTGRES, or to some other built-in schema besides pg_catalog,
we can just do it.
No catversion bump here, as there's no actual change in the contents
of postgres.bki.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/bki.sgml
M src/backend/catalog/genbki.pl
M src/include/catalog/pg_authid.dat
M src/include/catalog/pg_class.h
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_conversion.h
M src/include/catalog/pg_database.h
M src/include/catalog/pg_language.h
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_opclass.h
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_opfamily.h
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_ts_config.h
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_ts_parser.h
M src/include/catalog/pg_ts_template.h
M src/include/catalog/pg_type.h
pg_dump: Fix dumping of inherited generated columns
commit : 0bf83648a52df96f7c8677edbbdf141bfa0cf32b
author : Peter Eisentraut <[email protected]>
date : Wed, 3 Feb 2021 11:27:13 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 3 Feb 2021 11:27:13 +0100
Generation expressions of generated columns are always inherited, so
there is no need to set them separately in child tables, and there is
no syntax to do so either. The code previously used the code paths
for the handling of default values, for which different rules apply;
in particular it might want to set a default value explicitly for an
inherited column. This resulted in unrestorable dumps. For generated
columns, just skip them in inherited tables.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/15830.1575468847%40sss.pgh.pa.us
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/t/002_pg_dump.pl
Retire findoidjoins.
commit : ef3d4613c0204ab2b87ffa7e8e9551d74f932816
author : Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 17:21:37 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 17:21:37 -0500
In the wake of commit 62f34097c, we no longer need this tool.
Discussion: https://postgr.es/m/[email protected]
M src/tools/RELEASE_CHANGES
D src/tools/findoidjoins/.gitignore
D src/tools/findoidjoins/Makefile
D src/tools/findoidjoins/README
D src/tools/findoidjoins/findoidjoins.c
D src/tools/findoidjoins/make_oidjoins_check
Build in some knowledge about foreign-key relationships in the catalogs.
commit : 62f34097c88433ef1f3de604714fe7e7024f2fdf
author : Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 17:11:55 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 17:11:55 -0500
This follows in the spirit of commit dfb75e478, which created primary
key and uniqueness constraints to improve the visibility of constraints
imposed on the system catalogs. While our catalogs contain many
foreign-key-like relationships, they don't quite follow SQL semantics,
in that the convention for an omitted reference is to write zero not
NULL. Plus, we have some cases in which there are arrays each of whose
elements is supposed to be an FK reference; SQL has no way to model that.
So we can't create actual foreign key constraints to describe the
situation. Nonetheless, we can collect and use knowledge about these
relationships.
This patch therefore adds annotations to the catalog header files to
declare foreign-key relationships. (The BKI_LOOKUP annotations cover
simple cases, but we weren't previously distinguishing which such
columns are allowed to contain zeroes; we also need new markings for
multi-column FK references.) Then, Catalog.pm and genbki.pl are
taught to collect this information into a table in a new generated
header "system_fk_info.h". The only user of that at the moment is
a new SQL function pg_get_catalog_foreign_keys(), which exposes the
table to SQL. The oidjoins regression test is rewritten to use
pg_get_catalog_foreign_keys() to find out which columns to check.
Aside from removing the need for manual maintenance of that test
script, this allows it to cover numerous relationships that were not
checked by the old implementation based on findoidjoins. (As of this
commit, 217 relationships are checked by the test, versus 181 before.)
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/bki.sgml
M doc/src/sgml/func.sgml
M src/backend/catalog/.gitignore
M src/backend/catalog/Catalog.pm
M src/backend/catalog/Makefile
M src/backend/catalog/genbki.pl
M src/backend/utils/adt/misc.c
M src/include/Makefile
M src/include/catalog/.gitignore
M src/include/catalog/catversion.h
M src/include/catalog/genbki.h
M src/include/catalog/pg_aggregate.h
M src/include/catalog/pg_amop.h
M src/include/catalog/pg_attrdef.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_auth_members.h
M src/include/catalog/pg_cast.h
M src/include/catalog/pg_class.h
M src/include/catalog/pg_collation.dat
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_conversion.h
M src/include/catalog/pg_database.h
M src/include/catalog/pg_db_role_setting.h
M src/include/catalog/pg_default_acl.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_description.h
M src/include/catalog/pg_enum.h
M src/include/catalog/pg_event_trigger.h
M src/include/catalog/pg_extension.h
M src/include/catalog/pg_foreign_data_wrapper.h
M src/include/catalog/pg_foreign_server.h
M src/include/catalog/pg_foreign_table.h
M src/include/catalog/pg_index.h
M src/include/catalog/pg_inherits.h
M src/include/catalog/pg_init_privs.h
M src/include/catalog/pg_language.h
M src/include/catalog/pg_largeobject.h
M src/include/catalog/pg_largeobject_metadata.h
M src/include/catalog/pg_namespace.dat
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_opclass.h
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_opfamily.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_policy.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_publication.h
M src/include/catalog/pg_publication_rel.h
M src/include/catalog/pg_range.h
M src/include/catalog/pg_rewrite.h
M src/include/catalog/pg_seclabel.h
M src/include/catalog/pg_sequence.h
M src/include/catalog/pg_shdepend.h
M src/include/catalog/pg_shdescription.h
M src/include/catalog/pg_shseclabel.h
M src/include/catalog/pg_statistic.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/catalog/pg_subscription.h
M src/include/catalog/pg_subscription_rel.h
M src/include/catalog/pg_tablespace.dat
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_transform.h
M src/include/catalog/pg_trigger.h
M src/include/catalog/pg_ts_config.h
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_ts_parser.h
M src/include/catalog/pg_ts_template.h
M src/include/catalog/pg_type.h
M src/include/catalog/pg_user_mapping.h
M src/test/regress/expected/oidjoins.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
M src/test/regress/sql/oidjoins.sql
M src/tools/msvc/Solution.pm
M src/tools/msvc/clean.bat
Doc: consistently identify OID catalog columns that can be zero.
commit : 479331406e8403cc2e75d1082f8c613e7669c113
author : Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 16:15:29 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 16:15:29 -0500
Not all OID-reference columns that can contain zeroes (indicating
"no reference") were explicitly marked in catalogs.sgml. Fix that,
and try to make the style a little more consistent while at it ---
for example, consistently write "zero" not "0" for these cases.
Joel Jacobson and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
Remove extra increment of plpgsql's statement counter for FOR loops.
commit : dfcc46fe3030b0114b7a5715d5fa40819561c04b
author : Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 14:35:12 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 14:35:12 -0500
This left gaps in the internal statement numbering, which is not
terribly harmful (else we'd have noticed sooner), but it's not
great either.
Oversight in bbd5c207b; backpatch to v12 where that came in.
Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRDXyQaJmpotNTQVc-t-WxdWZC35V2PnmwOaV1-taidFWA@mail.gmail.com
M src/pl/plpgsql/src/pl_gram.y
Fix ancient memory leak in contrib/auto_explain.
commit : 5c0f7cc5442108e113d4fb88c952329b467e2c6a
author : Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 13:49:08 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 13:49:08 -0500
The ExecutorEnd hook is invoked in a context that could be quite
long-lived, not the executor's own per-query context as I think
we were sort of assuming. Thus, any cruft generated while producing
the EXPLAIN output could accumulate over multiple queries. This can
result in spectacular leakage if log_nested_statements is on, and
even without that I'm surprised nobody complained before.
To fix, just switch into the executor's context so that anything we
allocate will be released when standard_ExecutorEnd frees the executor
state. We might as well nuke the code's retail pfree of the explain
output string, too; that's laughably inadequate to the need.
Japin Li, per report from Jeff Janes. This bug is old, so
back-patch to all supported branches.
Discussion: https://postgr.es/m/CAMkU=1wCVtbeRn0s9gt12KwQ7PLXovbpM8eg25SYocKW3BT4hg@mail.gmail.com
M contrib/auto_explain/auto_explain.c
Improve confusing variable names
commit : 1d71f3c83c113849fe3aa60cb2d2c12729485e97
author : Peter Eisentraut <[email protected]>
date : Tue, 2 Feb 2021 09:20:22 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 2 Feb 2021 09:20:22 +0100
The prototype calls the second argument of
pgstat_progress_update_multi_param() "index", and some callers name
their local variable that way. But when the surrounding code deals
with index relations, this is confusing, and in at least one case
shadowed another variable that is referring to an index relation.
Adjust those call sites to have clearer local variable naming, similar
to existing callers in indexcmds.c.
M src/backend/access/nbtree/nbtsort.c
M src/backend/catalog/index.c
Remove unused column atttypmod from initial tablesync query
commit : 4ad31bb2ef2517b6e49ea9e8f01aaed250d4df38
author : Michael Paquier <[email protected]>
date : Tue, 2 Feb 2021 13:59:23 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 2 Feb 2021 13:59:23 +0900
The initial tablesync done by logical replication used a query to fetch
the information of a relation's columns that included atttypmod, but it
was left unused. This was added by 7c4f524.
Author: Euler Taveira
Reviewed-by: Önder Kalacı, Amit Langote, Japin Li
Discussion: https://postgr.es/m/CAHE3wggb715X+mK_DitLXF25B=jE6xyNCH4YOwM860JR7HarGQ@mail.gmail.com
M src/backend/replication/logical/tablesync.c
Doc: work a little harder on the initial examples for regex matching.
commit : 9522085ac917af66dba29939af328ae67300f10a
author : Tom Lane <[email protected]>
date : Mon, 1 Feb 2021 16:38:52 -0500
committer: Tom Lane <[email protected]>
date : Mon, 1 Feb 2021 16:38:52 -0500
Writing unnecessary '.*' at start and end of a POSIX regex doesn't
do much except confuse the reader about whether that might be
necessary after all. Make the examples in table 9.16 a tad more
realistic, and try to turn the next group of examples into something
self-contained.
Per gripe from rmzgrimes. Back-patch to v13 because it's easy.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
Remove [Merge]AppendPath.partitioned_rels.
commit : f003a7522bfa11177dc52c65eb97273a1057dfba
author : Tom Lane <[email protected]>
date : Mon, 1 Feb 2021 14:43:54 -0500
committer: Tom Lane <[email protected]>
date : Mon, 1 Feb 2021 14:43:54 -0500
It turns out that the calculation of [Merge]AppendPath.partitioned_rels
in allpaths.c is faulty and sometimes omits relevant non-leaf partitions,
allowing an assertion added by commit a929e17e5a8 to trigger. Rather
than fix that, it seems better to get rid of those fields altogether.
We don't really need the info until create_plan time, and calculating
it once for the selected plan should be cheaper than calculating it
for each append path we consider.
The preceding two commits did away with all use of the partitioned_rels
values; this commit just mechanically removes the fields and the code
that calculated them.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CAJKUy5gCXDSmFs2c=R+VGgn7FiYcLCsEFEuDNNLGfoha=pBE_g@mail.gmail.com
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/joinrels.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/pathnode.c
M src/backend/partitioning/partprune.c
M src/include/nodes/pathnodes.h
M src/include/optimizer/pathnode.h
M src/include/partitioning/partprune.h
Remove incidental dependencies on partitioned_rels lists.
commit : 5076f88bc985a7728eea337cbabae0e034b064b1
author : Tom Lane <[email protected]>
date : Mon, 1 Feb 2021 14:34:59 -0500
committer: Tom Lane <[email protected]>
date : Mon, 1 Feb 2021 14:34:59 -0500
It turns out that the calculation of [Merge]AppendPath.partitioned_rels
in allpaths.c is faulty and sometimes omits relevant non-leaf partitions,
allowing an assertion added by commit a929e17e5a8 to trigger. Rather
than fix that, it seems better to get rid of those fields altogether.
We don't really need the info until create_plan time, and calculating
it once for the selected plan should be cheaper than calculating it
for each append path we consider.
This patch undoes a couple of very minor uses of the partitioned_rels
values.
createplan.c was testing for nil-ness to optimize away the preparatory
work for make_partition_pruneinfo(). That is worth doing if the check
is nigh free, but it's not worth going to any great lengths to avoid.
create_append_path() was testing for nil-ness as part of deciding how
to set up ParamPathInfo for an AppendPath. I replaced that with a
check for the appendrel's parent rel being partitioned. That's not
quite the same thing but should cover most cases. If we note any
interesting loss of optimizations, we can dumb this down to just
always use the more expensive method when the parent is a baserel.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CAJKUy5gCXDSmFs2c=R+VGgn7FiYcLCsEFEuDNNLGfoha=pBE_g@mail.gmail.com
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/pathnode.c
Revise make_partition_pruneinfo to not use its partitioned_rels input.
commit : fb2d645dd53ff571572d830e830fc8c368063802
author : Tom Lane <[email protected]>
date : Mon, 1 Feb 2021 14:05:51 -0500
committer: Tom Lane <[email protected]>
date : Mon, 1 Feb 2021 14:05:51 -0500
It turns out that the calculation of [Merge]AppendPath.partitioned_rels
in allpaths.c is faulty and sometimes omits relevant non-leaf partitions,
allowing an assertion added by commit a929e17e5a8 to trigger. Rather
than fix that, it seems better to get rid of those fields altogether.
We don't really need the info until create_plan time, and calculating
it once for the selected plan should be cheaper than calculating it
for each append path we consider.
As a first step, teach make_partition_pruneinfo to collect the relevant
partitioned tables for itself. It's not hard to do so by traversing
from child tables up to parents using the AppendRelInfo links.
While here, make some minor stylistic improvements; mainly, don't use
the "Relids" alias for bitmapsets that are not identities of any
relation considered by the planner. Try to document the logic better,
too.
No backpatch, as there does not seem to be a live problem before
a929e17e5a8. Also no new regression test; the code where the bug
was will be gone at the end of this patch series, so it seems a
bit pointless to memorialize the issue.
Tom Lane and David Rowley, per reports from Andreas Seltenreich
and Jaime Casanova.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CAJKUy5gCXDSmFs2c=R+VGgn7FiYcLCsEFEuDNNLGfoha=pBE_g@mail.gmail.com
M src/backend/partitioning/partprune.c
SEARCH and CYCLE clauses
commit : 3696a600e2292d43c00949ddf0352e4ebb487e5b
author : Peter Eisentraut <[email protected]>
date : Mon, 1 Feb 2021 13:54:59 +0100
committer: Peter Eisentraut <[email protected]>
date : Mon, 1 Feb 2021 13:54:59 +0100
This adds the SQL standard feature that adds the SEARCH and CYCLE
clauses to recursive queries to be able to do produce breadth- or
depth-first search orders and detect cycles. These clauses can be
rewritten into queries using existing syntax, and that is what this
patch does in the rewriter.
Reviewed-by: Vik Fearing <[email protected]>
Reviewed-by: Pavel Stehule <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/queries.sgml
M doc/src/sgml/ref/select.sgml
M src/backend/catalog/dependency.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/parser/parse_agg.c
M src/backend/parser/parse_cte.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_func.c
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_target.c
M src/backend/rewrite/Makefile
M src/backend/rewrite/rewriteHandler.c
A src/backend/rewrite/rewriteSearchCycle.c
M src/backend/utils/adt/ruleutils.c
M src/include/nodes/nodes.h
M src/include/nodes/parsenodes.h
M src/include/parser/analyze.h
M src/include/parser/kwlist.h
M src/include/parser/parse_node.h
A src/include/rewrite/rewriteSearchCycle.h
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Get rid of unnecessary memory allocation in jsonb_subscript_assign()
commit : bb513b364b4fe31574574c8d0afbb2255268b321
author : Alexander Korotkov <[email protected]>
date : Mon, 1 Feb 2021 14:06:02 +0300
committer: Alexander Korotkov <[email protected]>
date : Mon, 1 Feb 2021 14:06:02 +0300
Current code allocates memory for JsonbValue, but it could be placed locally.
M src/backend/utils/adt/jsonbsubs.c
Introduce --with-ssl={openssl} as a configure option
commit : fe61df7f82aa6e0879476146dbe1da9c89b4946b
author : Michael Paquier <[email protected]>
date : Mon, 1 Feb 2021 19:19:44 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 1 Feb 2021 19:19:44 +0900
This is a replacement for the existing --with-openssl, extending the
logic to make easier the addition of new SSL libraries. The grammar is
chosen to be similar to --with-uuid, where multiple values can be
chosen, with "openssl" as the only supported value for now.
The original switch, --with-openssl, is kept for compatibility.
Author: Daniel Gustafsson, Michael Paquier
Reviewed-by: Jacob Champion
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
M contrib/Makefile
M contrib/pgcrypto/Makefile
M doc/src/sgml/installation.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/sslinfo.sgml
M src/Makefile.global.in
M src/backend/libpq/Makefile
M src/backend/libpq/hba.c
M src/common/Makefile
M src/include/pg_config.h.in
M src/interfaces/libpq/Makefile
M src/test/Makefile
M src/test/modules/Makefile
M src/test/modules/ssl_passphrase_callback/Makefile
M src/test/modules/ssl_passphrase_callback/t/001_testfunc.pl
M src/test/ssl/Makefile
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
M src/tools/msvc/Solution.pm
M src/tools/msvc/config_default.pl
Fix portability issue in new jsonbsubs code.
commit : 7c5d57caed4d8af705d0cc3131d0d8ed72b7a41d
author : Tom Lane <[email protected]>
date : Mon, 1 Feb 2021 02:03:59 -0500
committer: Tom Lane <[email protected]>
date : Mon, 1 Feb 2021 02:03:59 -0500
On machines where sizeof(Datum) > sizeof(Oid) (that is, any 64-bit
platform), the previous coding would compute a misaligned
workspace->index pointer if nupper is odd. Architectures where
misaligned access is a hard no-no would then fail. This appears
to explain why thorntail is unhappy but other buildfarm members
are not.
M src/backend/utils/adt/jsonbsubs.c
Throw error when assigning jsonb scalar instead of a composite object
commit : aa6e46daf5304e8d9e66fefc1a5bd77622ec6402
author : Alexander Korotkov <[email protected]>
date : Sun, 31 Jan 2021 23:51:06 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 31 Jan 2021 23:51:06 +0300
During the jsonb subscripting assignment, the provided path might assume an
object or an array where the source jsonb has a scalar value. Initial
subscripting assignment logic will skip such an update operation with no
message shown. This commit makes it throw an error to indicate this type
of situation.
Discussion: https://postgr.es/m/CA%2Bq6zcV8qvGcDXurwwgUbwACV86Th7G80pnubg42e-p9gsSf%3Dg%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcX3mdxGCgdThzuySwH-ApyHHM-G4oB1R0fn0j2hZqqkLQ%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcVDuGBv%3DM0FqBYX8DPebS3F_0KQ6OVFobGJPM507_SZ_w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcVovR%2BXY4mfk-7oNk-rF91gH0PebnNfuUjuuDsyHjOcVA%40mail.gmail.com
Author: Dmitry Dolgov
Reviewed-by: Tom Lane, Arthur Zakirov, Pavel Stehule, Dian M Fay
Reviewed-by: Andrew Dunstan, Chapman Flack, Merlin Moncure, Peter Geoghegan
Reviewed-by: Alvaro Herrera, Jim Nasby, Josh Berkus, Victor Wagner
Reviewed-by: Aleksander Alekseev, Robert Haas, Oleg Bartunov
M doc/src/sgml/json.sgml
M src/backend/utils/adt/jsonfuncs.c
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql
Filling array gaps during jsonb subscripting
commit : 81fcc72e66222357f9bccce3eeda62eb2cb29849
author : Alexander Korotkov <[email protected]>
date : Sun, 31 Jan 2021 23:51:01 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 31 Jan 2021 23:51:01 +0300
This commit introduces two new flags for jsonb assignment:
* JB_PATH_FILL_GAPS: Appending array elements on the specified position, gaps
are filled with nulls (similar to the JavaScript behavior). This mode also
instructs to create the whole path in a jsonb object if some part of the
path (more than just the last element) is not present.
* JB_PATH_CONSISTENT_POSITION: Assigning keeps array positions consistent by
preventing prepending of elements.
Both flags are used only in jsonb subscripting assignment.
Initially proposed by Nikita Glukhov based on polymorphic subscripting
patch, but transformed into an independent change.
Discussion: https://postgr.es/m/CA%2Bq6zcV8qvGcDXurwwgUbwACV86Th7G80pnubg42e-p9gsSf%3Dg%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcX3mdxGCgdThzuySwH-ApyHHM-G4oB1R0fn0j2hZqqkLQ%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcVDuGBv%3DM0FqBYX8DPebS3F_0KQ6OVFobGJPM507_SZ_w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcVovR%2BXY4mfk-7oNk-rF91gH0PebnNfuUjuuDsyHjOcVA%40mail.gmail.com
Author: Dmitry Dolgov
Reviewed-by: Tom Lane, Arthur Zakirov, Pavel Stehule, Dian M Fay
Reviewed-by: Andrew Dunstan, Chapman Flack, Merlin Moncure, Peter Geoghegan
Reviewed-by: Alvaro Herrera, Jim Nasby, Josh Berkus, Victor Wagner
Reviewed-by: Aleksander Alekseev, Robert Haas, Oleg Bartunov
M doc/src/sgml/json.sgml
M src/backend/utils/adt/jsonfuncs.c
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql
Implementation of subscripting for jsonb
commit : 676887a3b0b8e3c0348ac3f82ab0d16e9a24bd43
author : Alexander Korotkov <[email protected]>
date : Sun, 31 Jan 2021 23:50:40 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 31 Jan 2021 23:50:40 +0300
Subscripting for jsonb does not support slices, does not have a limit for the
number of subscripts, and an assignment expects a replace value to have jsonb
type. There is also one functional difference between assignment via
subscripting and assignment via jsonb_set(). When an original jsonb container
is NULL, the subscripting replaces it with an empty jsonb and proceeds with
an assignment.
For the sake of code reuse, we rearrange some parts of jsonb functionality
to allow the usage of the same functions for jsonb_set and assign subscripting
operation.
The original idea belongs to Oleg Bartunov.
Catversion is bumped.
Discussion: https://postgr.es/m/CA%2Bq6zcV8qvGcDXurwwgUbwACV86Th7G80pnubg42e-p9gsSf%3Dg%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcX3mdxGCgdThzuySwH-ApyHHM-G4oB1R0fn0j2hZqqkLQ%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcVDuGBv%3DM0FqBYX8DPebS3F_0KQ6OVFobGJPM507_SZ_w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2Bq6zcVovR%2BXY4mfk-7oNk-rF91gH0PebnNfuUjuuDsyHjOcVA%40mail.gmail.com
Author: Dmitry Dolgov
Reviewed-by: Tom Lane, Arthur Zakirov, Pavel Stehule, Dian M Fay
Reviewed-by: Andrew Dunstan, Chapman Flack, Merlin Moncure, Peter Geoghegan
Reviewed-by: Alvaro Herrera, Jim Nasby, Josh Berkus, Victor Wagner
Reviewed-by: Aleksander Alekseev, Robert Haas, Oleg Bartunov
M doc/src/sgml/json.sgml
M src/backend/utils/adt/Makefile
M src/backend/utils/adt/jsonb_util.c
A src/backend/utils/adt/jsonbsubs.c
M src/backend/utils/adt/jsonfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_type.dat
M src/include/utils/jsonb.h
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql
M src/tools/pgindent/typedefs.list
Remove unused _bt_delitems_delete() argument.
commit : dc43492e46c7145a476cb8ca6200fc8eefe673ef
author : Peter Geoghegan <[email protected]>
date : Sun, 31 Jan 2021 10:10:55 -0800
committer: Peter Geoghegan <[email protected]>
date : Sun, 31 Jan 2021 10:10:55 -0800
The latestRemovedXid values used by nbtree deletion operations are
determined by _bt_delitems_delete()'s caller, so there is no reason to
pass a separate heapRel argument.
Oversight in commit d168b666823.
M src/backend/access/nbtree/nbtpage.c
Fix parsing of complex morphs to tsquery
commit : 0c4f355c6a5fd437f71349f2f3d5d491382572b7
author : Alexander Korotkov <[email protected]>
date : Sun, 31 Jan 2021 20:14:29 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 31 Jan 2021 20:14:29 +0300
When to_tsquery() or websearch_to_tsquery() meet a complex morph containing
multiple words residing adjacent position, these words are connected
with OP_AND operator. That leads to surprising results. For instace,
both websearch_to_tsquery('"pg_class pg"') and to_tsquery('pg_class <-> pg')
produce '( pg & class ) <-> pg' tsquery. This tsquery requires
'pg' and 'class' words to reside on the same position and doesn't match
to to_tsvector('pg_class pg'). It appears to be ridiculous behavior, which
needs to be fixed.
This commit makes to_tsquery() or websearch_to_tsquery() connect words
residing adjacent position with OP_PHRASE. Therefore, now those words are
normally chained with other OP_PHRASE operator. The examples of above now
produces 'pg <-> class <-> pg' tsquery, which matches to
to_tsvector('pg_class pg').
Another effect of this commit is that complex morph word positions now need to
match the tsvector even if there is no surrounding OP_PHRASE. This behavior
change generally looks like an improvement but making this commit not
backpatchable.
Reported-by: Barry Pederson
Bug: #16592
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CAPpHfdv0EzVhf6CWfB1_TTZqXV_2Sn-jSY3zSd7ePH%3D-%2B1V2DQ%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Tom Lane, Neil Chen
M src/backend/tsearch/to_tsany.c
M src/test/regress/expected/tsearch.out
M src/test/regress/sql/tsearch.sql
Add primary keys and unique constraints to system catalogs
commit : dfb75e478cacb33d277432e0df5e2f9a2a2b52d9
author : Peter Eisentraut <[email protected]>
date : Sat, 30 Jan 2021 19:14:31 +0100
committer: Peter Eisentraut <[email protected]>
date : Sat, 30 Jan 2021 19:14:31 +0100
For those system catalogs that have a unique indexes, make a primary
key and unique constraint, using ALTER TABLE ... PRIMARY KEY/UNIQUE
USING INDEX.
This can be helpful for GUI tools that look for a primary key, and it
might in the future allow declaring foreign keys, for making schema
diagrams.
The constraint creation statements are automatically created by
genbki.pl from DECLARE_UNIQUE_INDEX directives. To specify which one
of the available unique indexes is the primary key, use the new
directive DECLARE_UNIQUE_INDEX_PKEY instead. By convention, we
usually make a catalog's OID column its primary key, if it has one.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/backend/catalog/.gitignore
M src/backend/catalog/Catalog.pm
M src/backend/catalog/Makefile
M src/backend/catalog/genbki.pl
M src/bin/initdb/initdb.c
M src/include/catalog/genbki.h
M src/include/catalog/pg_aggregate.h
M src/include/catalog/pg_am.h
M src/include/catalog/pg_amop.h
M src/include/catalog/pg_amproc.h
M src/include/catalog/pg_attrdef.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_auth_members.h
M src/include/catalog/pg_authid.h
M src/include/catalog/pg_cast.h
M src/include/catalog/pg_class.h
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_conversion.h
M src/include/catalog/pg_database.h
M src/include/catalog/pg_db_role_setting.h
M src/include/catalog/pg_default_acl.h
M src/include/catalog/pg_description.h
M src/include/catalog/pg_enum.h
M src/include/catalog/pg_event_trigger.h
M src/include/catalog/pg_extension.h
M src/include/catalog/pg_foreign_data_wrapper.h
M src/include/catalog/pg_foreign_server.h
M src/include/catalog/pg_foreign_table.h
M src/include/catalog/pg_index.h
M src/include/catalog/pg_inherits.h
M src/include/catalog/pg_init_privs.h
M src/include/catalog/pg_language.h
M src/include/catalog/pg_largeobject.h
M src/include/catalog/pg_largeobject_metadata.h
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_opclass.h
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_opfamily.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_policy.h
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_publication.h
M src/include/catalog/pg_publication_rel.h
M src/include/catalog/pg_range.h
M src/include/catalog/pg_replication_origin.h
M src/include/catalog/pg_rewrite.h
M src/include/catalog/pg_seclabel.h
M src/include/catalog/pg_sequence.h
M src/include/catalog/pg_shdescription.h
M src/include/catalog/pg_shseclabel.h
M src/include/catalog/pg_statistic.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/catalog/pg_subscription.h
M src/include/catalog/pg_subscription_rel.h
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_transform.h
M src/include/catalog/pg_trigger.h
M src/include/catalog/pg_ts_config.h
M src/include/catalog/pg_ts_config_map.h
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_ts_parser.h
M src/include/catalog/pg_ts_template.h
M src/include/catalog/pg_type.h
M src/include/catalog/pg_user_mapping.h
M src/test/modules/unsafe_tests/expected/alter_system_table.out
M src/test/modules/unsafe_tests/sql/alter_system_table.sql
M src/test/regress/expected/misc_sanity.out
M src/test/regress/sql/misc_sanity.sql
M src/tools/msvc/clean.bat
doc: Clarify status of SELECT INTO on reference page
commit : 65330622441d7ee08f768c4326825ae903f2595a
author : Peter Eisentraut <[email protected]>
date : Sat, 30 Jan 2021 11:05:15 +0100
committer: Peter Eisentraut <[email protected]>
date : Sat, 30 Jan 2021 11:05:15 +0100
The documentation as well as source code comments weren't entirely
clear whether SELECT INTO was truly deprecated (thus in theory
destined to be removed eventually), or just a less recommended
variant. After discussion, it appears that other implementations also
use SELECT INTO in direct SQL in a way similar to PostgreSQL, so it
seems worth keeping it for compatibility. Update the language in the
documentation to that effect.
Discussion: https://www.postgresql.org/message-id/flat/96dc0df3-e13a-a85d-d045-d6e2c85218da%40enterprisedb.com
M doc/src/sgml/ref/select_into.sgml
Allow GRANTED BY clause in normal GRANT and REVOKE statements
commit : 6aaaa76bb47db11cd6f567eafa3d1ee81ca59556
author : Peter Eisentraut <[email protected]>
date : Sat, 30 Jan 2021 09:41:44 +0100
committer: Peter Eisentraut <[email protected]>
date : Sat, 30 Jan 2021 09:41:44 +0100
The SQL standard allows a GRANTED BY clause on GRANT and
REVOKE (privilege) statements that can specify CURRENT_USER or
CURRENT_ROLE. In PostgreSQL, both of these are the default behavior.
Since we already have all the parsing support for this for the
GRANT (role) statement, we might as well add basic support for this
for the privilege variant as well. This allows us to check off SQL
feature T332. In the future, perhaps more interesting things could be
done with this, too.
Reviewed-by: Simon Riggs <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/ref/grant.sgml
M doc/src/sgml/ref/revoke.sgml
M src/backend/catalog/aclchk.c
M src/backend/catalog/sql_features.txt
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/parser/gram.y
M src/include/nodes/parsenodes.h
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Revive "snapshot too old" with wal_level=minimal and SET TABLESPACE.
commit : 7da83415e5bc01bba0093f34f4f612b70c70b678
author : Noah Misch <[email protected]>
date : Sat, 30 Jan 2021 00:12:18 -0800
committer: Noah Misch <[email protected]>
date : Sat, 30 Jan 2021 00:12:18 -0800
Given a permanent relation rewritten in the current transaction, the
old_snapshot_threshold mechanism assumed the relation had never been
subject to early pruning. Hence, a query could fail to report "snapshot
too old" when the rewrite followed an early truncation. ALTER TABLE SET
TABLESPACE is probably the only rewrite mechanism capable of exposing
this bug. REINDEX sets indcheckxmin, avoiding the problem. CLUSTER has
zeroed page LSNs since before old_snapshot_threshold existed, so
old_snapshot_threshold has never cooperated with it. ALTER TABLE
... SET DATA TYPE makes the table look empty to every past snapshot,
which is strictly worse. Back-patch to v13, where commit
c6b92041d38512a4176ed76ad06f713d2e6c01a8 broke this.
Kyotaro Horiguchi and Noah Misch
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/time/snapmgr.c
M src/include/utils/snapmgr.h
Fix error with CREATE PUBLICATION, wal_level=minimal, and new tables.
commit : 360bd2321b1ad9c47466bc485ee5eb2f4515372e
author : Noah Misch <[email protected]>
date : Sat, 30 Jan 2021 00:11:38 -0800
committer: Noah Misch <[email protected]>
date : Sat, 30 Jan 2021 00:11:38 -0800
CREATE PUBLICATION has failed spuriously when applied to a permanent
relation created or rewritten in the current transaction. Make the same
change to another site having the same semantic intent; the second
instance has no user-visible consequences. Back-patch to v13, where
commit c6b92041d38512a4176ed76ad06f713d2e6c01a8 broke this.
Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/pg_publication.c
M src/backend/optimizer/util/plancat.c
M src/test/subscription/t/001_rep_changes.pl
Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions.
commit : 8a54e12a38d1545d249f1402f66c8cde2837d97c
author : Noah Misch <[email protected]>
date : Sat, 30 Jan 2021 00:00:27 -0800
committer: Noah Misch <[email protected]>
date : Sat, 30 Jan 2021 00:00:27 -0800
In a cluster having used CREATE INDEX CONCURRENTLY while having enabled
prepared transactions, queries that use the resulting index can silently
fail to find rows. Fix this for future CREATE INDEX CONCURRENTLY by
making it wait for prepared transactions like it waits for ordinary
transactions. This expands the VirtualTransactionId structure domain to
admit prepared transactions. It may be necessary to reindex to recover
from past occurrences. Back-patch to 9.5 (all supported versions).
Andrey Borodin, reviewed (in earlier versions) by Tom Lane and Michael
Paquier.
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lock.c
M src/include/storage/lock.h
M src/test/isolation/Makefile
M src/test/isolation/README
A src/test/isolation/expected/prepared-transactions-cic.out
A src/test/isolation/specs/prepared-transactions-cic.spec
postgres_fdw: Fix tests for CLOBBER_CACHE_ALWAYS.
commit : f77717b2985aa529a185e6988de26b885ca10ddb
author : Fujii Masao <[email protected]>
date : Sat, 30 Jan 2021 10:12:22 +0900
committer: Fujii Masao <[email protected]>
date : Sat, 30 Jan 2021 10:12:22 +0900
The regression tests added in commits 708d165ddb and 411ae64997 caused
buildfarm failures when CLOBBER_CACHE_ALWAYS was enabled.
This commit stabilizes those tests.
The foreign server connections established by postgres_fdw behaves
differently depending on whether CLOBBER_CACHE_ALWAYS is enabled or not.
If it's not enabled, those connections are cached. On the other hand,
if it's enabled, when the connections are established outside transaction
block, they are not cached (i.e., they are immediately closed at the end of
query that established them). So the subsequent postgres_fdw_get_connections()
cannot list those connections and postgres_fdw_disconnect() cannot close them
(because they are already closed).
When the connections are established inside transaction block, they are
cached whether CLOBBER_CACHE_ALWAYS was enabled or not. But if it's enabled,
they are immediately marked as invalid, otherwise not. This causes the
subsequent postgres_fdw_get_connections() to return different result in
"valid" column depending on whether CLOBBER_CACHE_ALWAYS was enabled or not.
This commit prevents the above differences of behavior from
affecting the regression tests.
Per buildfarm failure on trilobite.
Original patch by Bharath Rupireddy. I (Fujii Masao) extracted
the regression test fix from that and revised it a bit.
Reported-by: Tom Lane
Author: Bharath Rupireddy
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Doc: improve cross-references for SET/SHOW.
commit : f743a2bbd4e511ad58b6ce01f81841e5e1611474
author : Tom Lane <[email protected]>
date : Fri, 29 Jan 2021 10:46:14 -0500
committer: Tom Lane <[email protected]>
date : Fri, 29 Jan 2021 10:46:14 -0500
The corresponding functions set_config and current_setting were
mostly not hyperlinked. Clarify their descriptions a tad, too.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/set.sgml
M doc/src/sgml/ref/show.sgml
Document behavior of the .** jsonpath accessor in the lax mode
commit : b41645460af563cfd4e4f57f354058cf69ef3b14
author : Alexander Korotkov <[email protected]>
date : Fri, 29 Jan 2021 15:27:55 +0300
committer: Alexander Korotkov <[email protected]>
date : Fri, 29 Jan 2021 15:27:55 +0300
When the .** jsonpath accessor handles the array, it selects both array and
each of its elements. When using lax mode, subsequent accessors automatically
unwrap arrays. So, the content of each array element may be selected twice.
Even though this behavior is counterintuitive, it's correct because everything
works as designed. This commit documents it.
Backpatch to 12 where the jsonpath language was introduced.
Reported-by: Thomas Kellerer
Bug: #16828
Discussion: https://postgr.es/m/16828-2b0229babfad2d8c%40postgresql.org
Discussion: https://postgr.es/m/CAPpHfdtS-nNidT%3DEqZbAYOPcnNOWh_sd6skVdu2CAQUGdvpT8Q%40mail.gmail.com
Author: Alexandex Korotkov, revised by Tom Lane
Reviewed-by: Alvaro Herrera, Thomas Kellerer, Tom Lane
Backpatch-through: 12
M doc/src/sgml/func.sgml
Fix typo
commit : 2592be8be5aed6448a1e2b386fffd75ad69eb6c5
author : Peter Eisentraut <[email protected]>
date : Fri, 29 Jan 2021 09:43:21 +0100
committer: Peter Eisentraut <[email protected]>
date : Fri, 29 Jan 2021 09:43:21 +0100
M src/bin/pg_dump/pg_dump.c
doc: Improve wording of section for repslot statistics
commit : 2a5862f013647dd185de3e1b496fd4f614d12f0e
author : Michael Paquier <[email protected]>
date : Fri, 29 Jan 2021 14:24:49 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 29 Jan 2021 14:24:49 +0900
This documentation has been added in 9868167, so no backpatch is
needed.
Author: Justin Pryzby, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
Adjust comments of CheckRelationTableSpaceMove() and SetRelationTableSpace()
commit : 24843297a96d7be16cc3f4b090aacfc6e5e6839e
author : Michael Paquier <[email protected]>
date : Fri, 29 Jan 2021 13:59:18 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 29 Jan 2021 13:59:18 +0900
4c9c359, that introduced those two functions, has been overoptimistic on
the point that only ShareUpdateExclusiveLock would be required when
moving a relation to a new tablespace. AccessExclusiveLock is a
requirement, but ShareUpdateExclusiveLock may be used under specific
conditions like REINDEX CONCURRENTLY where waits on past transactions
make the operation safe even with a lower-level lock. The current code
does only the former, so update the existing comments to reflect that.
Once a REINDEX (TABLESPACE) is introduced, those comments would require
an extra refresh to mention their new use case.
While on it, fix an incorrect variable name.
Per discussion with Álvaro Herrera.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
Remove documentation of waiting restore_command.
commit : 5c6d184213bb9feac0800eee1a16769bec9e8d58
author : Thomas Munro <[email protected]>
date : Fri, 29 Jan 2021 14:16:29 +1300
committer: Thomas Munro <[email protected]>
date : Fri, 29 Jan 2021 14:16:29 +1300
Following the removal of pg_standby, also remove the documentation
section that describes how to write your own "waiting restore_command"
along the same lines.
Discussion: https://postgr.es/m/20201029024412.GP5380%40telsasoft.com
Reviewed-by: Michael Paquier <[email protected]>
M doc/src/sgml/high-availability.sgml
Retire pg_standby.
commit : 514b411a2b5226167add9ab139d3a96dbe98035d
author : Thomas Munro <[email protected]>
date : Fri, 29 Jan 2021 13:21:53 +1300
committer: Thomas Munro <[email protected]>
date : Fri, 29 Jan 2021 13:21:53 +1300
pg_standby was useful more than a decade ago, but now it is obsolete.
It has been proposed that we retire it many times. Now seems like a
good time to finally do it, because "waiting restore commands"
are incompatible with a proposed recovery prefetching feature.
Discussion: https://postgr.es/m/20201029024412.GP5380%40telsasoft.com
Author: Justin Pryzby <[email protected]>
Reviewed-by: Heikki Linnakangas <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Reviewed-by: Fujii Masao <[email protected]>
M contrib/Makefile
D contrib/pg_standby/.gitignore
D contrib/pg_standby/Makefile
D contrib/pg_standby/pg_standby.c
M doc/src/sgml/contrib.sgml
M doc/src/sgml/filelist.sgml
M doc/src/sgml/high-availability.sgml
D doc/src/sgml/pgstandby.sgml
M doc/src/sgml/ref/pgarchivecleanup.sgml
M src/backend/access/transam/xlog.c
M src/tools/msvc/Mkvcbuild.pm
Silence another gcc 11 warning.
commit : 1046dbedde2fc3fe55f007ff3255ab65ab98f858
author : Tom Lane <[email protected]>
date : Thu, 28 Jan 2021 17:18:23 -0500
committer: Tom Lane <[email protected]>
date : Thu, 28 Jan 2021 17:18:23 -0500
Per buildfarm and local experimentation, bleeding-edge gcc isn't
convinced that the MemSet in reorder_function_arguments() is safe.
Shut it up by adding an explicit check that pronargs isn't negative,
and by changing MemSet to memset. (It appears that either change is
enough to quiet the warning at -O2, but let's do both to be sure.)
M src/backend/optimizer/util/clauses.c
Remove bogus restriction from BEFORE UPDATE triggers
commit : 6f5c8a8ec23f8ab00da4d2b77bfc8af2a578c4d3
author : Alvaro Herrera <[email protected]>
date : Thu, 28 Jan 2021 16:56:07 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 28 Jan 2021 16:56:07 -0300
In trying to protect the user from inconsistent behavior, commit
487e9861d0cf "Enable BEFORE row-level triggers for partitioned tables"
tried to prevent BEFORE UPDATE FOR EACH ROW triggers from moving the row
from one partition to another. However, it turns out that the
restriction is wrong in two ways: first, it fails spuriously, preventing
valid situations from working, as in bug #16794; and second, they don't
protect from any misbehavior, because tuple routing would cope anyway.
Fix by removing that restriction.
We keep the same restriction on BEFORE INSERT FOR EACH ROW triggers,
though. It is valid and useful there. In the future we could remove it
by having tuple reroute work for inserts as it does for updates.
Backpatch to 13.
Author: Álvaro Herrera <[email protected]>
Reported-by: Phillip Menke <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ddl.sgml
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Fix hash partition pruning with asymmetric partition sets.
commit : 1d9351a87c9a9e82c7091aab03d9299982670ce0
author : Tom Lane <[email protected]>
date : Thu, 28 Jan 2021 13:41:55 -0500
committer: Tom Lane <[email protected]>
date : Thu, 28 Jan 2021 13:41:55 -0500
perform_pruning_combine_step() was not taught about the number of
partition indexes used in hash partitioning; more embarrassingly,
get_matching_hash_bounds() also had it wrong. These errors are masked
in the common case where all the partitions have the same modulus
and no partition is missing. However, with missing or unequal-size
partitions, we could erroneously prune some partitions that need
to be scanned, leading to silently wrong query answers.
While a minimal-footprint fix for this could be to export
get_partition_bound_num_indexes and make the incorrect functions use it,
I'm of the opinion that that function should never have existed in the
first place. It's not reasonable data structure design that
PartitionBoundInfoData lacks any explicit record of the length of
its indexes[] array. Perhaps that was all right when it could always
be assumed equal to ndatums, but something should have been done about
it as soon as that stopped being true. Putting in an explicit
"nindexes" field makes both partition_bounds_equal() and
partition_bounds_copy() simpler, safer, and faster than before,
and removes explicit knowledge of the number-of-partition-indexes
rules from some other places too.
This change also makes get_hash_partition_greatest_modulus obsolete.
I left that in place in case any external code uses it, but no core
code does anymore.
Per bug #16840 from Michał Albrycht. Back-patch to v11 where the
hash partitioning code came in. (In the back branches, add the new
field at the end of PartitionBoundInfoData to minimize ABI risks.)
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execPartition.c
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partprune.c
M src/include/partitioning/partbounds.h
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Make ecpg's rjulmdy() and rmdyjul() agree with their declarations.
commit : 1b242f42ba1e732b202f35265ab1a1614ce7d859
author : Tom Lane <[email protected]>
date : Thu, 28 Jan 2021 11:17:13 -0500
committer: Tom Lane <[email protected]>
date : Thu, 28 Jan 2021 11:17:13 -0500
We had "short *mdy" in the extern declarations, but "short mdy[3]"
in the actual function definitions. Per C99 these are equivalent,
but recent versions of gcc have started to issue warnings about
the inconsistency. Clean it up before the warnings get any more
widespread.
Back-patch, in case anyone wants to build older PG versions with
bleeding-edge compilers.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/ecpg/compatlib/informix.c
pgbench: Remove dead code
commit : 6819b9042fe69154ff3fd2337c5028038d3cfab4
author : Alvaro Herrera <[email protected]>
date : Thu, 28 Jan 2021 12:50:40 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 28 Jan 2021 12:50:40 -0300
doConnect() never returns connections in state CONNECTION_BAD, so
checking for that is pointless. Remove the code that does.
This code has been dead since ba708ea3dc84, 20 years ago.
Discussion: https://postgr.es/m/[email protected]
Reviewed-by: Tom Lane <[email protected]>
M src/bin/pgbench/pgbench.c
Remove gratuitous uses of deprecated SELECT INTO
commit : b034ef9b376dbe712caa076541d6a750f37d85ce
author : Peter Eisentraut <[email protected]>
date : Thu, 28 Jan 2021 14:01:41 +0100
committer: Peter Eisentraut <[email protected]>
date : Thu, 28 Jan 2021 14:01:41 +0100
CREATE TABLE AS has been preferred over SELECT INTO (outside of ecpg
and PL/pgSQL) for a long time. There were still a few uses of SELECT
INTO in tests and documentation, some old, some more recent. This
changes them to CREATE TABLE AS. Some occurrences in the tests remain
where they are specifically testing SELECT INTO parsing or similar.
Discussion: https://www.postgresql.org/message-id/flat/96dc0df3-e13a-a85d-d045-d6e2c85218da%40enterprisedb.com
M contrib/sepgsql/expected/label.out
M contrib/sepgsql/sql/label.sql
M doc/src/sgml/hstore.sgml
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_checksums/t/002_actions.pl
M src/test/regress/expected/create_index.out
M src/test/regress/expected/create_misc.out
M src/test/regress/expected/random.out
M src/test/regress/expected/select_implicit.out
M src/test/regress/expected/select_implicit_1.out
M src/test/regress/expected/select_implicit_2.out
M src/test/regress/sql/create_index.sql
M src/test/regress/sql/create_misc.sql
M src/test/regress/sql/random.sql
M src/test/regress/sql/select_implicit.sql
Add direct conversion routines between EUC_TW and Big5.
commit : 6c5576075b0f93f2235ac8a82290fe3b6e82300d
author : Heikki Linnakangas <[email protected]>
date : Thu, 28 Jan 2021 14:53:03 +0200
committer: Heikki Linnakangas <[email protected]>
date : Thu, 28 Jan 2021 14:53:03 +0200
Conversions between EUC_TW and Big5 were previously implemented by
converting the whole input to MIC first, and then from MIC to the target
encoding. Implement functions to convert directly between the two.
The reason to do this now is that I'm working on a patch that will change
the conversion function signature so that if the input is invalid, we
convert as much as we can and return the number of bytes successfully
converted. That's not possible if we use an intermediary format, because
if an error happens in the intermediary -> final conversion, we lose track
of the location of the invalid character in the original input. Avoiding
the intermediate step makes the conversions faster, too.
Reviewed-by: John Naylor
Discussion: https://www.postgresql.org/message-id/b9e3167f-f84b-7aa4-5738-be578a4db924%40iki.fi
M src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
Add mbverifystr() functions specific to each encoding.
commit : b80e10638e36b9d2f0b39170c613837af2ca2aac
author : Heikki Linnakangas <[email protected]>
date : Thu, 28 Jan 2021 14:40:07 +0200
committer: Heikki Linnakangas <[email protected]>
date : Thu, 28 Jan 2021 14:40:07 +0200
This makes pg_verify_mbstr() function faster, by allowing more efficient
encoding-specific implementations. All the implementations included in
this commit are pretty naive, they just call the same encoding-specific
verifychar functions that were used previously, but that already gives a
performance boost because the tight character-at-a-time loop is simpler.
Reviewed-by: John Naylor
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/backend/commands/extension.c
M src/backend/utils/mb/conv.c
M src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
M src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
M src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
M src/backend/utils/mb/mbutils.c
M src/common/wchar.c
M src/include/mb/pg_wchar.h
Don't add bailout adjustment for non-strict deserialize calls.
commit : a3367aa3c4552170004c92369681730d85e384c6
author : Andrew Gierth <[email protected]>
date : Thu, 28 Jan 2021 10:53:10 +0000
committer: Andrew Gierth <[email protected]>
date : Thu, 28 Jan 2021 10:53:10 +0000
When building aggregate expression steps, strict checks need a bailout
jump for when a null value is encountered, so there is a list of steps
that require later adjustment. Adding entries to that list for steps
that aren't actually strict would be harmless, except that there is an
Assert which catches them. This leads to spurious errors on asserts
builds, for data sets that trigger parallel aggregation of an
aggregate with a non-strict deserialization function (no such
aggregates exist in the core system).
Repair by not adding the adjustment entry when it's not needed.
Backpatch back to 11 where the code was introduced.
Per a report from Darafei (Komzpa) of the PostGIS project; analysis
and patch by me.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execExpr.c
Fix crash of pg_stat_statements_info() without library loaded
commit : bca96ddab57c7c64ce442b0f9f35558741c7f8bd
author : Michael Paquier <[email protected]>
date : Thu, 28 Jan 2021 16:22:34 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 28 Jan 2021 16:22:34 +0900
Other code paths are already protected against this case, and _PG_init()
warns about that in pg_stat_statements.c. While on it, I have checked
the other extensions of the tree but did not notice any holes.
Oversight in 9fbc3f3.
Author: Jaime Casanova
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAJKUy5gF4=_=qhJ1VX_tSGFfjKHb9BvzhRYWSApJD=Bfwp2SBw@mail.gmail.com
M contrib/pg_stat_statements/pg_stat_statements.c
Refactor SQL functions of SHA-2 in cryptohashfuncs.c
commit : f854c69a5b36ba7aa85bee9e9590c3e517970156
author : Michael Paquier <[email protected]>
date : Thu, 28 Jan 2021 16:13:26 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 28 Jan 2021 16:13:26 +0900
The same code pattern was repeated four times when compiling a SHA-2
hash. This refactoring has the advantage to issue a compilation warning
if a new value is added to pg_cryptohash_type, so as anybody doing an
addition in this area would need to consider if support for a new SQL
function is needed or not.
Author: Sehrope Sarkuni, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/cryptohashfuncs.c
Reduce the default value of vacuum_cost_page_miss.
commit : e19594c5c059d2e071b67d87ae84f569a52d2e32
author : Peter Geoghegan <[email protected]>
date : Wed, 27 Jan 2021 15:11:13 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 27 Jan 2021 15:11:13 -0800
When commit f425b605 introduced cost based vacuum delays back in 2004,
the defaults reflected then-current trends in hardware, as well as
certain historical limitations in PostgreSQL. There have been enormous
improvements in both areas since that time. The cost limit GUC defaults
finally became much more representative of current trends following
commit cbccac37, which decreased autovacuum_vacuum_cost_delay's default
by 10x for PostgreSQL 12 (it went from 20ms to only 2ms).
The relative costs have shifted too. This should also be accounted for
by the defaults. More specifically, the relative importance of avoiding
dirtying pages within VACUUM has greatly increased, primarily due to
main memory capacity scaling and trends in flash storage. Within
Postgres itself, improvements like sequential access during index
vacuuming (at least in nbtree and GiST indexes) have also been
contributing factors.
To reflect all this, decrease the default of vacuum_cost_page_miss to 2.
Since the default of vacuum_cost_page_dirty remains 20, dirtying a page
is now considered 10x "costlier" than a page miss by default.
Author: Peter Geoghegan <[email protected]>
Discussion: https://postgr.es/m/CAH2-WzmLPFnkWT8xMjmcsm7YS3+_Qi3iRWAb2+_Bc8UhVyHfuA@mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/utils/init/globals.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
In TrimCLOG(), don't reset XactCtl->shared->latest_page_number.
commit : 69059d3b2f0754c8e661ba479f7121e6631cdf4a
author : Robert Haas <[email protected]>
date : Wed, 27 Jan 2021 15:52:34 -0500
committer: Robert Haas <[email protected]>
date : Wed, 27 Jan 2021 15:52:34 -0500
Since the CLOG page number is not recorded directly in the checkpoint
record, we have to use ShmemVariableCache->nextXid to figure out the
latest CLOG page number at the start of recovery. However, as recovery
progresses, replay of CLOG/EXTEND records will update our notion of
the latest page number, and we should rely on that being accurate
rather than recomputing the value based on an updated notion of
nextXid. ShmemVariableCache->nextXid is only an approximation
during recovery anyway, whereas CLOG/EXTEND records are an
authoritative representation of how the SLRU has been updated.
Commit 0fcc2decd485a61321a3220d8f76cb108b082009 makes this
simplification possible, as before that change clog_redo() might
have injected a bogus value here, and we'd want to get rid of
that before entering normal running.
Patch by me, reviewed by Heikki Linnakangas.
Discussion: http://postgr.es/m/CA+TgmoZYig9+AQodhF5sRXuKkJ=RgFDugLr3XX_dz_F-p=TwTg@mail.gmail.com
M src/backend/access/transam/clog.c
In clog_redo(), don't set XactCtl->shared->latest_page_number.
commit : 0fcc2decd485a61321a3220d8f76cb108b082009
author : Robert Haas <[email protected]>
date : Wed, 27 Jan 2021 13:07:41 -0500
committer: Robert Haas <[email protected]>
date : Wed, 27 Jan 2021 13:07:41 -0500
The comment is no longer accurate, and hasn't been entirely accurate
since Hot Standby was introduced. The original idea here was that
StartupCLOG() wouldn't be called until the end of recovery and
therefore this value would be uninitialized when this code is reached,
but Hot Standby made that true only when hot_standby=off, and commit
1f113abdf87cd085dee3927960bb4f70442b7250 means that this value is now
always initialized before replay even starts.
The original purpose of this code was to bypass the sanity check
in SimpleLruTruncate(), which will no longer occur: now, if something
is wrong, that sanity check might trip during recovery. That's
probably a good thing, because in the current code base
latest_page_number should always be initialized and therefore we
expect that the sanity check should pass. If it doesn't, something
has gone wrong, and complaining about it is appropriate.
Patch by me, reviewed by Heikki Linnakangas.
Discussion: http://postgr.es/m/CA+TgmoZYig9+AQodhF5sRXuKkJ=RgFDugLr3XX_dz_F-p=TwTg@mail.gmail.com
M src/backend/access/transam/clog.c
Doc: improve documentation for UNNEST().
commit : 662affcfe9e816584e3d8602b3b4005236931bbb
author : Tom Lane <[email protected]>
date : Wed, 27 Jan 2021 12:50:17 -0500
committer: Tom Lane <[email protected]>
date : Wed, 27 Jan 2021 12:50:17 -0500
Per a user question, spell out that UNNEST() returns array elements
in storage order; also provide an example to clarify the behavior for
multi-dimensional arrays.
While here, also clarify the SELECT reference page's description of
WITH ORDINALITY. These details were already given in 7.2.1.4, but
a reference page should not omit details.
Back-patch to v13; there's not room in the table in older versions.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/select.sgml
Move StartupCLOG() calls to just after we initialize ShmemVariableCache.
commit : 1f113abdf87cd085dee3927960bb4f70442b7250
author : Robert Haas <[email protected]>
date : Wed, 27 Jan 2021 12:20:46 -0500
committer: Robert Haas <[email protected]>
date : Wed, 27 Jan 2021 12:20:46 -0500
Previously, the hot_standby=off code path did this at end of recovery,
while the hot_standby=on code path did it at the beginning of recovery.
It's better to do this in only one place because (a) it's simpler,
(b) StartupCLOG() is trivial so trying to postpone the work isn't
useful, and (c) this will make it possible to simplify some other
logic.
Patch by me, reviewed by Heikki Linnakangas.
Discussion: http://postgr.es/m/CA+TgmoZYig9+AQodhF5sRXuKkJ=RgFDugLr3XX_dz_F-p=TwTg@mail.gmail.com
M src/backend/access/transam/xlog.c
Fix GiST index deletion assert issue.
commit : e42b3c3bd6a9c6233ac4c8a2e9b040367ba2f97c
author : Peter Geoghegan <[email protected]>
date : Tue, 26 Jan 2021 23:24:37 -0800
committer: Peter Geoghegan <[email protected]>
date : Tue, 26 Jan 2021 23:24:37 -0800
Avoid calling heap_index_delete_tuples() with an empty deltids array to
avoid an assertion failure.
This issue was arguably an oversight in commit b5f58cf2, though the
failing assert itself was added by my recent commit d168b666. No
backpatch, though, since the oversight is harmless in the back branches.
Author: Peter Geoghegan <[email protected]>
Reported-By: Jaime Casanova <[email protected]>
Discussion: https://postgr.es/m/CAJKUy5jscES84n3puE=sYngyF+zpb4wv8UMtuLnLPv5z=6yyNw@mail.gmail.com
M src/backend/access/gist/gist.c
M src/backend/access/index/genam.c
doc: Remove reference to views for TRUNCATE privilege
commit : 32bef758296142ce0fff944c3478d63fdefdaf33
author : Michael Paquier <[email protected]>
date : Wed, 27 Jan 2021 13:40:33 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 27 Jan 2021 13:40:33 +0900
The page about privilege rights mentioned that TRUNCATE could be applied
to views or even other relation types. This is confusing as this
command can be used only on tables and on partitioned tables.
Oversight in afc4a78.
Reported-by: Harisai Hari
Reviewed-by: Laurenz Albe
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M doc/src/sgml/ddl.sgml
Refactor code in tablecmds.c to check and process tablespace moves
commit : 4c9c359d38ff1e2de388eedd860785be6a49201c
author : Michael Paquier <[email protected]>
date : Wed, 27 Jan 2021 11:54:16 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 27 Jan 2021 11:54:16 +0900
Two code paths of tablecmds.c (for relations with storage and without
storage) use the same logic to check if the move of a relation to a
new tablespace is allowed or not and to update pg_class.reltablespace
and pg_class.relfilenode.
A potential TABLESPACE clause for REINDEX, CLUSTER and VACUUM FULL needs
similar checks to make sure that nothing is moved around in illegal ways
(no mapped relations, shared relations only in pg_global, no move of
temp tables owned by other backends).
This reorganizes the existing code of ALTER TABLE so as all this logic
is controlled by two new routines that can be reused for the other
commands able to move relations across tablespaces, limiting the number
of code paths in need of the same protections. This also removes some
code that was duplicated for tables with and without storage for ALTER
TABLE.
Author: Alexey Kondratov, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/include/commands/tablecmds.h
Rethink recently-added SPI interfaces.
commit : d5a83d79c9f9b660a6a5a77afafe146d3c8c6f46
author : Tom Lane <[email protected]>
date : Tue, 26 Jan 2021 16:37:12 -0500
committer: Tom Lane <[email protected]>
date : Tue, 26 Jan 2021 16:37:12 -0500
SPI_execute_with_receiver and SPI_cursor_parse_open_with_paramlist are
new in v14 (cf. commit 2f48ede08). Before they can get out the door,
let's change their APIs to follow the practice recently established by
SPI_prepare_extended etc: shove all optional arguments into a struct
that callers are supposed to pre-zero. The hope is to allow future
addition of more options without either API breakage or a continuing
proliferation of new SPI entry points. With that in mind, choose
slightly more generic names for them: SPI_execute_extended and
SPI_cursor_parse_open respectively.
Discussion: https://postgr.es/m/CAFj8pRCLPdDAETvR7Po7gC5y_ibkn_-bOzbeJb39WHms01194Q@mail.gmail.com
M doc/src/sgml/spi.sgml
M src/backend/executor/spi.c
M src/include/executor/spi.h
M src/pl/plpgsql/src/pl_exec.c
Suppress compiler warnings from commit ee895a655.
commit : 7292fd8f1c781278021407276474d9188845113d
author : Tom Lane <[email protected]>
date : Tue, 26 Jan 2021 13:58:18 -0500
committer: Tom Lane <[email protected]>
date : Tue, 26 Jan 2021 13:58:18 -0500
For obscure reasons, some buildfarm members are now generating
complaints about plpgsql_call_handler's "retval" variable possibly
being used uninitialized. It seems no less safe than it was before
that commit, but these complaints are (mostly?) new. I trust that
initializing the variable where it's declared will be enough to
shut that up.
I also notice that some compilers are warning about setjmp clobber
of the same variable, which is maybe a bit more defensible. Mark
it volatile to silence that.
Also, rearrange the logic to give procedure_resowner a single
point of initialization, in hopes of silencing some setjmp-clobber
warnings about that. (Marking it volatile would serve too, but
its sibling variables are depending on single assignment, so let's
stick with that method.)
Discussion: https://postgr.es/m/[email protected]
M src/pl/plpgsql/src/pl_handler.c
Code review for psql's helpSQL() function.
commit : f76a85000bba2f1b1c926cbbe525e47b246215f1
author : Tom Lane <[email protected]>
date : Tue, 26 Jan 2021 13:04:52 -0500
committer: Tom Lane <[email protected]>
date : Tue, 26 Jan 2021 13:04:52 -0500
The loops to identify word boundaries could access past the end of
the input string. Likely that would never result in an actual
crash, but it makes valgrind unhappy.
The logic to try different numbers of words didn't work when the
input has two words but we only have a match to the first, eg
"\h with select". (We must "continue" the pass loop, not "break".)
The logic to compute nl_count was bizarrely managed, and in at
least two code paths could end up calling PageOutput with
nl_count = 0, resulting in failing to paginate output that should
have been fed to the pager. Also, in v12 and up, the nl_count
calculation hadn't been updated to account for the addition of a URL.
The PQExpBuffer holding the command syntax details wasn't freed,
resulting in a session-lifespan memory leak.
While here, improve some comments, choose a more descriptive name
for a variable, fix inconsistent datatype choice for another variable.
Per bug #16837 from Alexander Lakhin. This code is very old,
so back-patch to all supported branches.
Kyotaro Horiguchi and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/help.c
Fix memory leak when deallocating prepared statement in postgres_fdw
commit : 7b4c660466dd8a1d25ca316ac02099a7cf0e5896
author : Michael Paquier <[email protected]>
date : Tue, 26 Jan 2021 18:43:01 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 26 Jan 2021 18:43:01 +0900
The leak is minor, so no backpatch is done. Oversight in 21734d2.
Reported-by: Tom Lane
M contrib/postgres_fdw/postgres_fdw.c
postgres_fdw: Fix test failure with -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
commit : 0c3fc09fe359a6dc46f1870ceccf60ec60396bc9
author : Fujii Masao <[email protected]>
date : Tue, 26 Jan 2021 17:16:52 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 26 Jan 2021 17:16:52 +0900
The roles created by regression test should have names starting with
"regress_", and the test introduced in commit 411ae64997 did not do that.
Per buildfarm member longfin.
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
postgres_fdw: Stabilize regression test for postgres_fdw_disconnect_all().
commit : 6adc5376dca4ef8b7d591c0ee7338cb9ff660216
author : Fujii Masao <[email protected]>
date : Tue, 26 Jan 2021 16:36:21 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 26 Jan 2021 16:36:21 +0900
The regression test added in commit 411ae64997 caused buildfarm failures.
The cause of them was that the order of warning messages output in the test
was not stable. To fix this, this commit sets client_min_messages to ERROR
temporarily when performing the test generating those warnings.
Per buildfarm failures.
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
postgres_fdw: Add functions to discard cached connections.
commit : 411ae64997dc3a42d19eda6721c581841ce2cb82
author : Fujii Masao <[email protected]>
date : Tue, 26 Jan 2021 03:54:46 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 26 Jan 2021 03:54:46 +0900
This commit introduces two new functions postgres_fdw_disconnect()
and postgres_fdw_disconnect_all(). The former function discards
the cached connections to the specified foreign server. The latter discards
all the cached connections. If the connection is used in the current
transaction, it's not closed and a warning message is emitted.
For example, these functions are useful when users want to explicitly
close the foreign server connections that are no longer necessary and
then to prevent them from eating up the foreign servers connections
capacity.
Author: Bharath Rupireddy, tweaked a bit by Fujii Masao
Reviewed-by: Alexey Kondratov, Zhijie Hou, Zhihong Yu, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVvrp5=AVp2PupEm+nAC8S4buqR3fJMmaCoc7ftT0aD2A@mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw–1.0–1.1.sql
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Improve performance of repeated CALLs within plpgsql procedures.
commit : ee895a655ce4341546facd6f23e3e8f2931b96bf
author : Tom Lane <[email protected]>
date : Mon, 25 Jan 2021 22:28:29 -0500
committer: Tom Lane <[email protected]>
date : Mon, 25 Jan 2021 22:28:29 -0500
This patch essentially is cleaning up technical debt left behind
by the original implementation of plpgsql procedures, particularly
commit d92bc83c4. That patch (or more precisely, follow-on patches
fixing its worst bugs) forced us to re-plan CALL and DO statements
each time through, if we're in a non-atomic context. That wasn't
for any fundamental reason, but just because use of a saved plan
requires having a ResourceOwner to hold a reference count for the
plan, and we had no suitable resowner at hand, nor would the
available APIs support using one if we did. While it's not that
expensive to create a "plan" for CALL/DO, the cycles do add up
in repeated executions.
This patch therefore makes the following API changes:
* GetCachedPlan/ReleaseCachedPlan are modified to let the caller
specify which resowner to use to pin the plan, rather than forcing
use of CurrentResourceOwner.
* spi.c gains a "SPI_execute_plan_extended" entry point that lets
callers say which resowner to use to pin the plan. This borrows the
idea of an options struct from the recently added SPI_prepare_extended,
hopefully allowing future options to be added without more API breaks.
This supersedes SPI_execute_plan_with_paramlist (which I've marked
deprecated) as well as SPI_execute_plan_with_receiver (which is new
in v14, so I just took it out altogether).
* I also took the opportunity to remove the crude hack of letting
plpgsql reach into SPI private data structures to mark SPI plans as
"no_snapshot". It's better to treat that as an option of
SPI_prepare_extended.
Now, when running a non-atomic procedure or DO block that contains
any CALL or DO commands, plpgsql creates a ResourceOwner that
will be used to pin the plans of the CALL/DO commands. (In an
atomic context, we just use CurrentResourceOwner, as before.)
Having done this, we can just save CALL/DO plans normally,
whether or not they are used across transaction boundaries.
This seems to be good for something like 2X speedup of a CALL
of a trivial procedure with a few simple argument expressions.
By restricting the creation of an extra ResourceOwner like this,
there's essentially zero penalty in cases that can't benefit.
Pavel Stehule, with some further hacking by me
Discussion: https://postgr.es/m/CAFj8pRCLPdDAETvR7Po7gC5y_ibkn_-bOzbeJb39WHms01194Q@mail.gmail.com
M doc/src/sgml/spi.sgml
M src/backend/commands/prepare.c
M src/backend/executor/spi.c
M src/backend/tcop/postgres.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/mmgr/portalmem.c
M src/backend/utils/resowner/resowner.c
M src/include/executor/spi.h
M src/include/executor/spi_priv.h
M src/include/utils/plancache.h
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/pl_handler.c
M src/pl/plpgsql/src/plpgsql.h
Fix two typos in snapbuild.c.
commit : 55ef8555f0c1207bac25050e7297bbb969c84233
author : Andres Freund <[email protected]>
date : Mon, 25 Jan 2021 12:15:10 -0800
committer: Andres Freund <[email protected]>
date : Mon, 25 Jan 2021 12:15:10 -0800
Reported-by: Heikki Linnakangas <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/snapbuild.c
Don't clobber the calling user's credentials cache in Kerberos test.
commit : 881933f194221abcce07fb134ebe8685e5bb58dd
author : Tom Lane <[email protected]>
date : Mon, 25 Jan 2021 14:53:13 -0500
committer: Tom Lane <[email protected]>
date : Mon, 25 Jan 2021 14:53:13 -0500
Embarrassing oversight in this test script, which fortunately is not
run by default.
Report and patch by Jacob Champion.
Discussion: https://postgr.es/m/[email protected]
M src/test/kerberos/t/001_auth.pl
Fix broken ruleutils support for function TRANSFORM clauses.
commit : 07d46fceb4254b00e79f3d06419cbae13b0ecb5a
author : Tom Lane <[email protected]>
date : Mon, 25 Jan 2021 13:03:11 -0500
committer: Tom Lane <[email protected]>
date : Mon, 25 Jan 2021 13:03:11 -0500
I chanced to notice that this dumped core due to a faulty Assert.
To add insult to injury, the output has been misformatted since v11.
Obviously we need some regression testing here.
Discussion: https://postgr.es/m/[email protected]
M contrib/bool_plperl/expected/bool_plperl.out
M contrib/bool_plperl/expected/bool_plperlu.out
M contrib/bool_plperl/sql/bool_plperl.sql
M contrib/bool_plperl/sql/bool_plperlu.sql
M contrib/hstore_plpython/expected/hstore_plpython.out
M contrib/hstore_plpython/sql/hstore_plpython.sql
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/fmgr/funcapi.c
Remove CheckpointLock.
commit : d18e75664a2fda2e4d5cc433d68e37fc0e9499f2
author : Robert Haas <[email protected]>
date : Mon, 25 Jan 2021 12:34:00 -0500
committer: Robert Haas <[email protected]>
date : Mon, 25 Jan 2021 12:34:00 -0500
Up until now, we've held this lock when performing a checkpoint or
restartpoint, but commit 076a055acf3c55314de267c62b03191586d79cf6 back
in 2004 and commit 7e48b77b1cebb9a43f9fdd6b17128a0ba36132f9 from 2009,
taken together, have removed all need for this. In the present code,
there's only ever one process entitled to attempt a checkpoint: either
the checkpointer, during normal operation, or the postmaster, during
single-user operation. So, we don't need the lock.
One possible concern in making this change is that it means that
a substantial amount of code where HOLD_INTERRUPTS() was previously
in effect due to the preceding LWLockAcquire() will now be
running without that. This could mean that ProcessInterrupts()
gets called in places from which it didn't before. However, this
seems unlikely to do very much, because the checkpointer doesn't
have any signal mapped to die(), so it's not clear how,
for example, ProcDiePending = true could happen in the first
place. Similarly with ClientConnectionLost and recovery conflicts.
Also, if there are any such problems, we might want to fix them
rather than reverting this, since running lots of code with
interrupt handling suspended is generally bad.
Patch by me, per an inquiry by Amul Sul. Review by Tom Lane
and Michael Paquier.
Discussion: http://postgr.es/m/CAAJ_b97XnBBfYeSREDJorFsyoD1sHgqnNuCi=02mNQBUMnA=FA@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/access/heap/rewriteheap.c
M src/backend/access/transam/xlog.c
M src/backend/replication/logical/origin.c
M src/backend/storage/lmgr/lwlocknames.txt
Doc: improve documentation of pg_proc.protrftypes.
commit : 951862eda57e5dc8f78c97b3c30fe2032a5562b8
author : Tom Lane <[email protected]>
date : Mon, 25 Jan 2021 11:20:17 -0500
committer: Tom Lane <[email protected]>
date : Mon, 25 Jan 2021 11:20:17 -0500
Add a "references" link pointing to pg_type, as we have for other arrays
of type OIDs. Wordsmith the explanation a bit.
Joel Jacobson, additional editing by me
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
Remove duplicate include
commit : ecc4b1318c92594abdf9bd2933fb6feaa9d56229
author : Peter Eisentraut <[email protected]>
date : Mon, 25 Jan 2021 08:55:43 +0100
committer: Peter Eisentraut <[email protected]>
date : Mon, 25 Jan 2021 08:55:43 +0100
Reported-by: Ashutosh Sharma <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CAE9k0PkORqHHGKY54-sFyDpP90yAf%2B05Auc4fs9EAn4J%2BuBeUQ%40mail.gmail.com
M src/interfaces/ecpg/test/expected/preproc-define.c
M src/interfaces/ecpg/test/expected/preproc-define.stderr
M src/interfaces/ecpg/test/preproc/define.pgc
Fix hypothetical bug in heap backward scans
commit : 16dfe253e31f75b60e93acc0c2b5bbf19936c074
author : David Rowley <[email protected]>
date : Mon, 25 Jan 2021 19:52:18 +1300
committer: David Rowley <[email protected]>
date : Mon, 25 Jan 2021 19:52:18 +1300
Both heapgettup() and heapgettup_pagemode() incorrectly set the first page
to scan in a backward scan in which the number of pages to scan was
specified by heap_setscanlimits(). The code incorrectly started the scan
at the end of the relation when startBlk was 0, or otherwise at
startBlk - 1, neither of which is correct when only scanning a subset of
pages.
The fix here checks if heap_setscanlimits() has changed the number of
pages to scan and if so we set the first page to scan as the final page in
the specified range during backward scans.
Proper adjustment of this code was forgotten when heap_setscanlimits() was
added in 7516f5259 back in 9.5. However, practice, nowhere in core code
performs backward scans after having used heap_setscanlimits(), yet, it is
possible an extension uses the heap functions in this way, hence
backpatch.
An upcoming patch does use heap_setscanlimits() with backward scans, so
this must be fixed before that can go in.
Author: David Rowley
Discussion: https://postgr.es/m/CAApHDvpGc9h0_oVD2CtgBcxCS1N-qDYZSeBRnUh+0CWJA9cMaA@mail.gmail.com
Backpatch-through: 9.5, all supported versions
M src/backend/access/heap/heapam.c
Fix ALTER PUBLICATION...DROP TABLE behavior.
commit : 40ab64c1ec1cb9bd73695f519cf66ddbb97d8144
author : Amit Kapila <[email protected]>
date : Mon, 25 Jan 2021 07:39:29 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 25 Jan 2021 07:39:29 +0530
Commit 69bd60672 fixed the initialization of streamed transactions for
RelationSyncEntry. It forgot to initialize the publication actions while
invalidating the RelationSyncEntry due to which even though the relation
is dropped from a particular publication we still publish its changes. Fix
it by initializing pubactions when entry got invalidated.
Author: Japin Li and Bharath Rupireddy
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CALj2ACV+0UFpcZs5czYgBpujM9p0Hg1qdOZai_43OU7bqHU_xw@mail.gmail.com
M src/backend/replication/pgoutput/pgoutput.c
M src/test/subscription/t/001_rep_changes.pl
Make storage/standby.h compile standalone again.
commit : a4b03de589c1df0845e9732da203f505f2eedb6d
author : Tom Lane <[email protected]>
date : Sun, 24 Jan 2021 18:08:55 -0500
committer: Tom Lane <[email protected]>
date : Sun, 24 Jan 2021 18:08:55 -0500
This file has failed headerscheck/cpluspluscheck verification since
commit 0650ff230, as a result of referencing typedef TimestampTz
without including the appropriate header.
M src/include/storage/standby.h
Update time zone data files to tzdata release 2021a.
commit : c7edf4ac246b67073563354c2808c78868cbac36
author : Tom Lane <[email protected]>
date : Sun, 24 Jan 2021 16:29:47 -0500
committer: Tom Lane <[email protected]>
date : Sun, 24 Jan 2021 16:29:47 -0500
DST law changes in Russia (Volgograd zone) and South Sudan.
Historical corrections for Australia, Bahamas, Belize, Bermuda,
Ghana, Israel, Kenya, Nigeria, Palestine, Seychelles, and Vanuatu.
Notably, the Australia/Currie zone has been corrected to the point
where it is identical to Australia/Hobart.
M src/timezone/Makefile
M src/timezone/data/tzdata.zi
Add a simple test for contrib/auto_explain.
commit : 0c1e8845f28bd07ad381c8b0d6701575d967b88e
author : Tom Lane <[email protected]>
date : Sun, 24 Jan 2021 14:59:33 -0500
committer: Tom Lane <[email protected]>
date : Sun, 24 Jan 2021 14:59:33 -0500
This module formerly had zero test coverage.
Discussion: https://postgr.es/m/[email protected]
A contrib/auto_explain/.gitignore
M contrib/auto_explain/Makefile
A contrib/auto_explain/t/001_auto_explain.pl
Remove make_diff set of tools
commit : 8a337b0ed21c654521da137159b8b9327a5cb971
author : Magnus Hagander <[email protected]>
date : Sun, 24 Jan 2021 14:19:00 +0100
committer: Magnus Hagander <[email protected]>
date : Sun, 24 Jan 2021 14:19:00 +0100
These are mostly obsoleted by the switch to git, and it's easier to
remove them than to update the incorrect documentation.
Discussion: https://postgr.es/m/CABUevEwmASMn4WRJ6RagBx43sj10ctfMHcMA_-7KA3pDYmwpJw@mail.gmail.com
D src/tools/make_diff/README
D src/tools/make_diff/cporig
D src/tools/make_diff/difforig
D src/tools/make_diff/rmorig
Doc: clean up contrib/pageinspect's GIST function documentation.
commit : 7e57255f6189380d545e1df6a6b38827b213e3da
author : Tom Lane <[email protected]>
date : Sat, 23 Jan 2021 22:40:46 -0500
committer: Tom Lane <[email protected]>
date : Sat, 23 Jan 2021 22:40:46 -0500
I came to fix the overwidth-PDF-page warnings seen in the buildfarm,
but stayed long enough to copy-edit some nearby text.
M doc/src/sgml/pageinspect.sgml
Fix COPY FREEZE with CLOBBER_CACHE_ALWAYS
commit : 39b66a91bdebb00af71a2c6218412ecfc89a0e13
author : Tomas Vondra <[email protected]>
date : Sun, 24 Jan 2021 00:24:50 +0100
committer: Tomas Vondra <[email protected]>
date : Sun, 24 Jan 2021 00:24:50 +0100
This adds code omitted from commit 7db0cd2145 by accident, which had
two consequences. Firstly, only rows inserted by heap_multi_insert were
frozen as expected when running COPY FREEZE, while heap_insert left
rows unfrozen. That however includes rows in TOAST tables, so a lot of
data might have been left unfrozen. Secondly, page might have been left
partially empty after relcache invalidation.
This addresses both of those issues.
Discussion: https://postgr.es/m/CABOikdN-ptGv0mZntrK2Q8OtfUuAjqaYMGmkdU1dCKFtUxVLrg@mail.gmail.com
M src/backend/access/heap/heapam.c
M src/backend/access/heap/hio.c
Doc: update example connection-failure messages in the documentation.
commit : 183bbd1b6d4376f1b04c02b7a20b55019f6d84f4
author : Tom Lane <[email protected]>
date : Sat, 23 Jan 2021 15:50:51 -0500
committer: Tom Lane <[email protected]>
date : Sat, 23 Jan 2021 15:50:51 -0500
Now that the dust has more or less settled on 52a10224e and follow-ons,
make sure the examples in the documentation are up-to-date.
M doc/src/sgml/runtime.sgml
M doc/src/sgml/start.sgml
Update ecpg's connect-test1 for connection-failure message changes.
commit : 68d1c339417ea2e8df42e11db57472fcdbfe6a55
author : Tom Lane <[email protected]>
date : Sat, 23 Jan 2021 15:08:39 -0500
committer: Tom Lane <[email protected]>
date : Sat, 23 Jan 2021 15:08:39 -0500
I should have updated this in commits 52a10224e and follow-ons,
but I missed it because it's not run by default, and none of the
buildfarm runs it either. Maybe we should try to improve that
situation.
Discussion: https://postgr.es/m/CAH2-Wz=j9SRW=s5BV4-3k+=tr4N3A03in+gTuVA09vNF+-iHjA@mail.gmail.com
M src/interfaces/ecpg/test/expected/connect-test1-minGW32.stderr
M src/interfaces/ecpg/test/expected/connect-test1.stderr
Introduce SHA1 implementations in the cryptohash infrastructure
commit : a8ed6bb8f4cf259b95c1bff5da09a8f4c79dca46
author : Michael Paquier <[email protected]>
date : Sat, 23 Jan 2021 11:33:04 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 23 Jan 2021 11:33:04 +0900
With this commit, SHA1 goes through the implementation provided by
OpenSSL via EVP when building the backend with it, and uses as fallback
implementation KAME which was located in pgcrypto and already shaped for
an integration with a set of init, update and final routines.
Structures and routines have been renamed to make things consistent with
the fallback implementations of MD5 and SHA2.
uuid-ossp has used for ages a shortcut with pgcrypto to fetch a copy of
SHA1 if needed. This was built depending on the build options within
./configure, so this cleans up some code and removes the build
dependency between pgcrypto and uuid-ossp.
Note that this will help with the refactoring of HMAC, as pgcrypto
offers the option to use MD5, SHA1 or SHA2, so only the second option
was missing to make that possible.
Author: Michael Paquier
Reviewed-by: Heikki Linnakangas
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
M contrib/pgcrypto/Makefile
M contrib/pgcrypto/internal.c
D contrib/pgcrypto/sha1.c
M contrib/uuid-ossp/.gitignore
M contrib/uuid-ossp/Makefile
M contrib/uuid-ossp/uuid-ossp.c
M src/Makefile.global.in
M src/common/Makefile
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
A src/common/sha1.c
R071 contrib/pgcrypto/sha1.h src/common/sha1_int.h
M src/include/common/cryptohash.h
A src/include/common/sha1.h
M src/tools/msvc/Mkvcbuild.pm
M src/tools/pgindent/typedefs.list
Suppress bison warning in ecpg grammar.
commit : 3fc81ce459e1696f7e5e5b3b8229409413bf64b4
author : Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 19:25:39 -0500
committer: Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 19:25:39 -0500
opt_distinct_clause is only used in PLpgSQL_Expr, which ecpg
ignores, so it needs to ignore opt_distinct_clause too.
My oversight in 7cd9765f9; reported by Bruce Momjian.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/ecpg/preproc/parse.pl
Doc: improve directions for building on macOS.
commit : 50bebc1ae1804e0ddf86b667122d3f8b22fa19b7
author : Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 18:58:25 -0500
committer: Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 18:58:25 -0500
In light of recent discussions, we should instruct people to
install Apple's command line tools; installing Xcode is secondary.
Also, fix sample command for finding out the default sysroot,
as we now know that the command originally recommended can give
a result that doesn't match your OS version.
Also document the workaround to use if you really don't want
configure to select a sysroot at all.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/installation.sgml
Avoid redundantly prefixing PQerrorMessage for a connection failure.
commit : 58cd8dca3de0b3c7d378a412eca1f7289b5e4978
author : Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 16:52:31 -0500
committer: Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 16:52:31 -0500
libpq's error messages for connection failures pretty well stand on
their own, especially since commits 52a10224e/27a48e5a1. Prefixing
them with 'could not connect to database "foo"' or the like is just
redundant, and perhaps even misleading if the specific database name
isn't relevant to the failure. (When it is, we trust that the
backend's error message will include the DB name.) Indeed, psql
hasn't used any such prefix in a long time. So, make all our other
programs and documentation examples agree with psql's practice.
Discussion: https://postgr.es/m/[email protected]
M contrib/oid2name/oid2name.c
M contrib/vacuumlo/vacuumlo.c
M doc/src/sgml/libpq.sgml
M doc/src/sgml/lobj.sgml
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/bin/pg_upgrade/server.c
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/scripts/common.c
M src/interfaces/ecpg/ecpglib/connect.c
M src/interfaces/ecpg/test/expected/connect-test5.stderr
M src/test/examples/testlibpq.c
M src/test/examples/testlibpq2.c
M src/test/examples/testlibpq3.c
M src/test/examples/testlibpq4.c
M src/test/examples/testlo.c
M src/test/examples/testlo64.c
M src/test/isolation/isolationtester.c
M src/tools/findoidjoins/findoidjoins.c
Re-allow DISTINCT in pl/pgsql expressions.
commit : 7cd9765f9bd3397b8d4d0f507021ef848b6d48d2
author : Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 16:26:22 -0500
committer: Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 16:26:22 -0500
I'd omitted this from the grammar in commit c9d529848, figuring that
it wasn't worth supporting. However we already have one complaint,
so it seems that judgment was wrong. It doesn't require a huge
amount of code, so add it back. (I'm still drawing the line at
UNION/INTERSECT/EXCEPT though: those'd require an unreasonable
amount of grammar refactoring, and the single-result-row restriction
makes them near useless anyway.)
Also rethink the documentation: this behavior is a property of
all pl/pgsql expressions, not just assignments.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/plpgsql.sgml
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
Doc: remove misleading claim in documentation of PQreset().
commit : ab66645628d38a7047996b294b213f3048f7ef9b
author : Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 11:29:43 -0500
committer: Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 11:29:43 -0500
This text claimed that the reconnection would occur "to the same
server", but there is no such guarantee in the code, nor would
insisting on that be an improvement.
Back-patch to v10 where multi-host connection strings were added.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
Remove reference to ftp servers from documentation
commit : 0a9ae44288d122c12aa0862b032ae892942da452
author : Magnus Hagander <[email protected]>
date : Fri, 22 Jan 2021 12:49:53 +0100
committer: Magnus Hagander <[email protected]>
date : Fri, 22 Jan 2021 12:49:53 +0100
It's been a long time since we used ftp, but there was a single
reference left in the docs.
Author: Daniel Gustafsson <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/install-windows.sgml
Remove bogus tracepoint
commit : 09418bed67a7d5d5a8c9c3070f4c9048455a5389
author : Peter Eisentraut <[email protected]>
date : Fri, 22 Jan 2021 11:58:21 +0100
committer: Peter Eisentraut <[email protected]>
date : Fri, 22 Jan 2021 11:58:21 +0100
Calls to LWLockWaitForVar() fired the TRACE_POSTGRESQL_LWLOCK_ACQUIRE
tracepoint, but LWLockWaitForVar() never actually acquires the LWLock.
(Probably a copy/paste bug in 68a2e52bbaf.) Remove it.
Author: Craig Ringer <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CAGRY4nxJo+-HCC2i5H93ttSZ4gZO-FSddCwvkb-qAfQ1zdXd1w@mail.gmail.com
M src/backend/storage/lmgr/lwlock.c
doc: Copy-edit the "Overview of PostgreSQL Internals" chapter
commit : 29ad6595ef7f568ca11dd9219c0d23048bdda513
author : Heikki Linnakangas <[email protected]>
date : Fri, 22 Jan 2021 11:10:42 +0200
committer: Heikki Linnakangas <[email protected]>
date : Fri, 22 Jan 2021 11:10:42 +0200
Rephrase a few sentences to be more concise.
Refer to the postmaster process as "postmaster", not "postgres". This
originally said "postmaster process", but was changed to "postgres
process" in commit 5266f221a2, when we merged the "postmaster" and
"postgres" commands, and "postmaster" became just a symlink. That was a
case of overzealous search & replace, because the process is still called
"postmaster".
Author: Erik Rijkers and Jürgen Purtz
Discussion: https://www.postgresql.org/message-id/aa31f359-1168-ded5-53d0-0ed228bfe097%40iki.fi
M doc/src/sgml/arch-dev.sgml
Move SSL information callback earlier to capture more information
commit : af0e79c8f4f4c3c2306855045c0d02a6be6485f0
author : Michael Paquier <[email protected]>
date : Fri, 22 Jan 2021 09:26:27 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 22 Jan 2021 09:26:27 +0900
The callback for retrieving state change information during connection
setup was only installed when the connection was mostly set up, and
thus didn't provide much information and missed all the details related
to the handshake.
This also extends the callback with SSL_state_string_long() to print
more information about the state change within the SSL object handled.
While there, fix some comments which were incorrectly referring to the
callback and its previous location in fe-secure.c.
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/be-secure-openssl.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/fe-secure.c
Improve new wording of libpq's connection failure messages.
commit : 27a48e5a16ff2227ddf44ee717d9bcd89d22a7aa
author : Tom Lane <[email protected]>
date : Thu, 21 Jan 2021 16:10:18 -0500
committer: Tom Lane <[email protected]>
date : Thu, 21 Jan 2021 16:10:18 -0500
"connection to server so-and-so failed:" seems clearer than the
previous wording "could not connect to so-and-so:" (introduced by
52a10224e), because the latter suggests a network-level connection
failure. We're now prefixing this string to all types of connection
failures, for instance authentication failures; so we need wording
that doesn't imply a low-level error.
Per discussion with Robert Haas.
Discussion: https://postgr.es/m/CA+TgmobssJ6rS22dspWnu-oDxXevGmhMD8VcRBjmj-b9UDqRjw@mail.gmail.com
M src/bin/pg_dump/t/002_pg_dump.pl
M src/interfaces/ecpg/test/expected/connect-test5.stderr
M src/interfaces/ecpg/test/pg_regress_ecpg.c
M src/interfaces/libpq/fe-connect.c
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
commit : 55dc86eca70b1dc18a79c141b3567efed910329d
author : Tom Lane <[email protected]>
date : Thu, 21 Jan 2021 15:37:23 -0500
committer: Tom Lane <[email protected]>
date : Thu, 21 Jan 2021 15:37:23 -0500
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/optimizer/path/clausesel.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/path/pathkeys.c
M src/backend/optimizer/path/tidpath.c
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/optimizer/util/clauses.c
M src/backend/optimizer/util/inherit.c
M src/backend/optimizer/util/orclauses.c
M src/backend/optimizer/util/placeholder.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/optimizer/util/var.c
M src/backend/utils/adt/selfuncs.c
M src/include/optimizer/clauses.h
M src/include/optimizer/optimizer.h
M src/include/optimizer/paths.h
M src/include/optimizer/planmain.h
M src/include/optimizer/restrictinfo.h
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Fix initialization of FDW batching in ExecInitModifyTable
commit : 920f853dc948b98a5dc96580c4ee011a302e33e4
author : Tomas Vondra <[email protected]>
date : Thu, 21 Jan 2021 03:23:24 +0100
committer: Tomas Vondra <[email protected]>
date : Thu, 21 Jan 2021 03:23:24 +0100
ExecInitModifyTable has to initialize batching for all result relations,
not just the first one. Furthermore, when junk filters were necessary,
the pointer pointed past the mtstate->resultRelInfo array.
Per reports from multiple non-x86 animals (florican, locust, ...).
Discussion: https://postgr.es/m/20200628151002.7x5laxwpgvkyiu3q@development
M src/backend/executor/nodeModifyTable.c
Switch "cl /?" to "cl /help" in MSVC scripts for platform detection
commit : 733d670073efd2c3a9df07c225006668009ab793
author : Michael Paquier <[email protected]>
date : Thu, 21 Jan 2021 10:56:03 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 21 Jan 2021 10:56:03 +0900
"cl /?" produces a different output if run on a real or a virtual drive
(this can be set with a simple subst command), causing an error in the
MSVC scripts if building on a virtual drive because the platform to use
cannot be detected.
"cl /help", on the contrary, produces a consistent output if used on a
real or virtual drive. Changing to "/help" allows the compilation to
work with a virtual drive as long as the top of the code repository is
part of the drive, without impacting the build on real drives.
Reported-by: Robert Grange
Author: Juan José Santamaría Flecha
Discussion: https://postgr.es/m/[email protected]
M src/tools/msvc/Solution.pm
Implement support for bulk inserts in postgres_fdw
commit : b663a4136331de6c7364226e3dbf7c88bfee7145
author : Tomas Vondra <[email protected]>
date : Wed, 20 Jan 2021 23:05:46 +0100
committer: Tomas Vondra <[email protected]>
date : Wed, 20 Jan 2021 23:05:46 +0100
Extends the FDW API to allow batching inserts into foreign tables. That
is usually much more efficient than inserting individual rows, due to
high latency for each round-trip to the foreign server.
It was possible to implement something similar in the regular FDW API,
but it was inconvenient and there were issues with reporting the number
of actually inserted rows etc. This extends the FDW API with two new
functions:
* GetForeignModifyBatchSize - allows the FDW picking optimal batch size
* ExecForeignBatchInsert - inserts a batch of rows at once
Currently, only INSERT queries support batching. Support for DELETE and
UPDATE may be added in the future.
This also implements batching for postgres_fdw. The batch size may be
specified using "batch_size" option both at the server and table level.
The initial patch version was written by me, but it was rewritten and
improved in many ways by Takayuki Tsunakawa.
Author: Takayuki Tsunakawa
Reviewed-by: Tomas Vondra, Amit Langote
Discussion: https://postgr.es/m/20200628151002.7x5laxwpgvkyiu3q@development
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/postgres-fdw.sgml
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/backend/nodes/list.c
M src/include/foreign/fdwapi.h
M src/include/nodes/execnodes.h
M src/include/nodes/pg_list.h
psql \dX: list extended statistics objects
commit : ad600bba0422dde4b73fbd61049ff2a3847b068a
author : Tomas Vondra <[email protected]>
date : Wed, 20 Jan 2021 22:56:06 +0100
committer: Tomas Vondra <[email protected]>
date : Wed, 20 Jan 2021 22:56:06 +0100
The new command lists extended statistics objects. All past releases
with extended statistics are supported.
This is a simplified version of commit 891a1d0bca, which had to be
reverted due to not considering pg_statistic_ext_data is not accessible
by regular users. Fields requiring access to this catalog were removed.
It's possible to add them, but it'll require changes to core.
Author: Tatsuro Yamada
Reviewed-by: Julien Rouhaud, Alvaro Herrera, Tomas Vondra, Noriyoshi Shinoda
Discussion: https://postgr.es/m/c027a541-5856-75a5-0868-341301e1624b%40nttcom.co.jp_1
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/bin/psql/help.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Further tweaking of PG_SYSROOT heuristics for macOS.
commit : 9d23c15a034ba163ae1045b945954e8e1bcfc72a
author : Tom Lane <[email protected]>
date : Wed, 20 Jan 2021 12:07:23 -0500
committer: Tom Lane <[email protected]>
date : Wed, 20 Jan 2021 12:07:23 -0500
It emerges that in some phases of the moon (perhaps to do with
directory entry order?), xcrun will report that the SDK path is
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
which is normally a symlink to a version-numbered sibling directory.
Our heuristic to skip non-version-numbered pathnames was rejecting
that, which is the wrong thing to do. We'd still like to end up
with a version-numbered PG_SYSROOT value, but we can have that by
dereferencing the symlink.
Like the previous fix, back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]
M src/template/darwin
Disable vacuum page skipping in selected test cases.
commit : c2dc1a79767a0f947e1145f82eb65dfe4360d25f
author : Tom Lane <[email protected]>
date : Wed, 20 Jan 2021 11:49:29 -0500
committer: Tom Lane <[email protected]>
date : Wed, 20 Jan 2021 11:49:29 -0500
By default VACUUM will skip pages that it can't immediately get
exclusive access to, which means that even activities as harmless
and unpredictable as checkpoint buffer writes might prevent a page
from being processed. Ordinarily this is no big deal, but we have
a small number of test cases that examine the results of VACUUM's
processing and therefore will fail if the page of interest is skipped.
This seems to be the explanation for some rare buildfarm failures.
To fix, add the DISABLE_PAGE_SKIPPING option to the VACUUM commands
in tests where this could be an issue.
In passing, remove a duplicated query in pageinspect/sql/page.sql.
Back-patch as necessary (some of these cases are as old as v10).
Discussion: https://postgr.es/m/[email protected]
M contrib/amcheck/expected/check_heap.out
M contrib/amcheck/sql/check_heap.sql
M contrib/amcheck/t/001_verify_heapam.pl
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/sql/page.sql
M contrib/pg_visibility/expected/pg_visibility.out
M contrib/pg_visibility/sql/pg_visibility.sql
Fix bug in detecting concurrent page splits in GiST insert
commit : 6b4d3046f422c2682365924b515c7588d5a3e651
author : Heikki Linnakangas <[email protected]>
date : Wed, 20 Jan 2021 11:58:03 +0200
committer: Heikki Linnakangas <[email protected]>
date : Wed, 20 Jan 2021 11:58:03 +0200
In commit 9eb5607e699, I got the condition on checking for split or
deleted page wrong: I used && instead of ||. The comment correctly said
"concurrent split _or_ deletion".
As a result, GiST insertion could miss a concurrent split, and insert to
wrong page. Duncan Sands demonstrated this with a test script that did a
lot of concurrent inserts.
Backpatch to v12, where this was introduced. REINDEX is required to fix
indexes that were affected by this bug.
Backpatch-through: 12
Reported-by: Duncan Sands
Discussion: https://www.postgresql.org/message-id/a9690483-6c6c-3c82-c8ba-dc1a40848f11%40deepbluecap.com
M src/backend/access/gist/gist.c
Fix sample output of EXPLAIN ANALYZE.
commit : 679744cf1b0d0569d16b4dd2d020f9095ea3d53b
author : Thomas Munro <[email protected]>
date : Wed, 20 Jan 2021 22:31:26 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 20 Jan 2021 22:31:26 +1300
Since commit f0f13a3a08b2757997410f3a1c38bdc22973c525, we estimate
ModifyTable paths without a RETURNING clause differently. Update an
example from the manual that showed the old behavior.
Author: Takayuki Tsunakawa <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/TYAPR01MB29905674F41693BBA9DA28CAFEA20%40TYAPR01MB2990.jpnprd01.prod.outlook.com
M doc/src/sgml/perform.sgml
Add regression test for DROP OWNED BY with default ACLs
commit : a36dc04d424a6bfa03ee2cf75c85a6b7f9697e70
author : Michael Paquier <[email protected]>
date : Wed, 20 Jan 2021 13:28:10 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 20 Jan 2021 13:28:10 +0900
DROP OWNED BY has a specific code path to remove ACLs stored in
pg_default_acl when cleaning up shared dependencies that had no
coverage with the existing tests. This issue has been found while
digging into the bug fixed by 21378e1.
As ALTER DEFAULT PRIVILEGES impacts the ACLs of all objects created
while the default permissions are visible, the test uses a transaction
rollback to isolate the test and avoid any impact with other sessions
running in parallel.
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Fix ALTER DEFAULT PRIVILEGES with duplicated objects
commit : 21378e1fefedcaed3d855ae7aa772555295d05d6
author : Michael Paquier <[email protected]>
date : Wed, 20 Jan 2021 11:38:17 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 20 Jan 2021 11:38:17 +0900
Specifying duplicated objects in this command would lead to unique
constraint violations in pg_default_acl or "tuple already updated by
self" errors. Similarly to GRANT/REVOKE, increment the command ID after
each subcommand processing to allow this case to work transparently.
A regression test is added by tweaking one of the existing queries of
privileges.sql to stress this case.
Reported-by: Andrus
Author: Michael Paquier
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M src/backend/catalog/aclchk.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Remove faulty support for MergeAppend plan with WHERE CURRENT OF.
commit : a0efda88a679edaee9855628cb05b2ab00d80a15
author : Tom Lane <[email protected]>
date : Tue, 19 Jan 2021 13:25:33 -0500
committer: Tom Lane <[email protected]>
date : Tue, 19 Jan 2021 13:25:33 -0500
Somebody extended search_plan_tree() to treat MergeAppend exactly
like Append, which is 100% wrong, because unlike Append we can't
assume that only one input node is actively returning tuples.
Hence a cursor using a MergeAppend across a UNION ALL or inheritance
tree could falsely match a WHERE CURRENT OF query at a row that
isn't actually the cursor's current output row, but coincidentally
has the same TID (in a different table) as the current output row.
Delete the faulty code; this means that such a case will now return
an error like 'cursor "foo" is not a simply updatable scan of table
"bar"', instead of silently misbehaving. Users should not find that
surprising though, as the same cursor query could have failed that way
already depending on the chosen plan. (It would fail like that if the
sort were done with an explicit Sort node instead of MergeAppend.)
Expand the clearly-inadequate commentary to be more explicit about
what this code is doing, in hopes of forestalling future mistakes.
It's been like this for awhile, so back-patch to all supported
branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execCurrent.c
pageinspect: Change block number arguments to bigint
commit : f18aa1b203930ed28cfe42e82d3418ae6277576d
author : Peter Eisentraut <[email protected]>
date : Tue, 19 Jan 2021 10:28:05 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 19 Jan 2021 10:28:05 +0100
Block numbers are 32-bit unsigned integers. Therefore, the smallest
SQL integer type that they can fit in is bigint. However, in the
pageinspect module, most input and output parameters dealing with
block numbers were declared as int. The behavior with block numbers
larger than a signed 32-bit integer was therefore dubious. Change
these arguments to type bigint and add some more explicit error
checking on the block range.
(Other contrib modules appear to do this correctly already.)
Since we are changing argument types of existing functions, in order
to not misbehave if the binary is updated before the extension is
updated, we need to create new C symbols for the entry points, similar
to how it's done in other extensions as well.
Reported-by: Ashutosh Bapat <[email protected]>
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
M contrib/pageinspect/Makefile
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/btreefuncs.c
M contrib/pageinspect/expected/btree.out
M contrib/pageinspect/expected/gin.out
M contrib/pageinspect/expected/hash.out
A contrib/pageinspect/expected/oldextversions.out
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/hashfuncs.c
M contrib/pageinspect/pageinspect–1.8–1.9.sql
M contrib/pageinspect/pageinspect.h
M contrib/pageinspect/rawpage.c
M contrib/pageinspect/sql/btree.sql
M contrib/pageinspect/sql/gin.sql
M contrib/pageinspect/sql/hash.sql
A contrib/pageinspect/sql/oldextversions.sql
M contrib/pageinspect/sql/page.sql
M doc/src/sgml/pageinspect.sgml
doc: Add note about the server name of postgres_fdw_get_connections() returns.
commit : ee79a548e746da9a99df0cac70a3ddc095f2829a
author : Fujii Masao <[email protected]>
date : Tue, 19 Jan 2021 00:56:10 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 19 Jan 2021 00:56:10 +0900
Previously the document didn't mention the case where
postgres_fdw_get_connections() returns NULL in server_name column.
Users might be confused about why NULL was returned.
This commit adds the note that, in postgres_fdw_get_connections(),
the server name of an invalid connection will be NULL if the server is dropped.
Suggested-by: Zhijie Hou
Author: Bharath Rupireddy
Reviewed-by: Zhijie Hou, Fujii Masao
Discussion: https://postgr.es/m/e7ddd14e96444fce88e47a709c196537@G08CNEXMBPEKD05.g08.fujitsu.local
M doc/src/sgml/postgres-fdw.sgml
pgindent worker.c.
commit : ed43677e20369040ca4e50c698010c39d5ac0f47
author : Amit Kapila <[email protected]>
date : Tue, 19 Jan 2021 08:10:13 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 19 Jan 2021 08:10:13 +0530
This is a leftover from commit 0926e96c49. Changing this separately
because this file is being modified for upcoming patch logical replication
of 2PC.
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+Ps+EgG8KzcmAyAgBUi_vuTps6o9ZA8DG6SdnO0-YuOhPQ@mail.gmail.com
M src/backend/replication/logical/worker.c
doc: adjust alignment of doc file list for "pg_waldump.sgml"
commit : 9e7dbe3369cd8f5b0136c53b817471002505f934
author : Bruce Momjian <[email protected]>
date : Mon, 18 Jan 2021 18:48:25 -0500
committer: Bruce Momjian <[email protected]>
date : Mon, 18 Jan 2021 18:48:25 -0500
Backpatch-through: 10
M doc/src/sgml/ref/allfiles.sgml
Avoid crash with WHERE CURRENT OF and a custom scan plan.
commit : 60661bbf2dca0c1e744c948c74bccab98933c45b
author : Tom Lane <[email protected]>
date : Mon, 18 Jan 2021 18:32:30 -0500
committer: Tom Lane <[email protected]>
date : Mon, 18 Jan 2021 18:32:30 -0500
execCurrent.c's search_plan_tree() assumed that ForeignScanStates
and CustomScanStates necessarily have a valid ss_currentRelation.
This is demonstrably untrue for postgres_fdw's remote join and
remote aggregation plans, and non-leaf custom scans might not have
an identifiable scan relation either. Avoid crashing by ignoring
such nodes when the field is null.
This solution will lead to errors like 'cursor "foo" is not a
simply updatable scan of table "bar"' in cases where maybe we
could have allowed WHERE CURRENT OF to work. That's not an issue
for postgres_fdw's usages, since joins or aggregations would render
WHERE CURRENT OF invalid anyway. But an otherwise-transparent
upper level custom scan node might find this annoying. When and if
someone cares to expend work on such a scenario, we could invent a
custom-scan-provider callback to determine what's safe.
Report and patch by David Geier, commentary by me. It's been like
this for awhile, so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execCurrent.c
Narrow the scope of a local variable.
commit : 3fd80c728dc36fbd250ca3019c2f5fa2567f1a75
author : Tom Lane <[email protected]>
date : Mon, 18 Jan 2021 15:55:01 -0500
committer: Tom Lane <[email protected]>
date : Mon, 18 Jan 2021 15:55:01 -0500
This is better style and more symmetrical with the other if-branch.
This likely should have been included in 9de77b545 (which created
the opportunity), but it was overlooked.
Japin Li
Discussion: https://postgr.es/m/MEYP282MB16699FA4A7CD57EB250E871FB6A40@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
M src/backend/replication/logical/proto.c
Add bytea equivalents of ltrim() and rtrim().
commit : a6cf3df4ebdcbc7857910a67f259705645383e9f
author : Tom Lane <[email protected]>
date : Mon, 18 Jan 2021 15:11:32 -0500
committer: Tom Lane <[email protected]>
date : Mon, 18 Jan 2021 15:11:32 -0500
We had bytea btrim() already, but for some reason not the other two.
Joel Jacobson
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/utils/adt/oracle_compat.c
M src/backend/utils/adt/ruleutils.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/create_view.out
M src/test/regress/expected/strings.out
M src/test/regress/sql/create_view.sql
M src/test/regress/sql/strings.sql
Allow for error or refusal while absorbing a ProcSignalBarrier.
commit : a3ed4d1efe9f3a1765c504ba4a224f1f393b5b14
author : Robert Haas <[email protected]>
date : Mon, 18 Jan 2021 12:09:52 -0500
committer: Robert Haas <[email protected]>
date : Mon, 18 Jan 2021 12:09:52 -0500
Previously, the per-barrier-type functions tasked with absorbing
them were expected to always succeed and never throw an error.
However, that's a bit inconvenient. Further study has revealed that
there are realistic cases where it might not be possible to absorb
a ProcSignalBarrier without terminating the transaction, or even
the whole backend. Similarly, for some barrier types, there might
be other reasons where it's not reasonably possible to absorb the
barrier at certain points in the code, so provide a way for a
per-barrier-type function to reject absorbing the barrier.
Unfortunately, there's still no committed code making use of this
infrastructure; hopefully, we'll get there. :-(
Patch by me, reviewed by Andres Freund and Amul Sul.
Discussion: http://postgr.es/m/[email protected]
Discussion: http://postgr.es/m/CA+Tgmob56Pk1-5aTJdVPCWFHon7me4M96ENpGe9n_R4JUjjhZA@mail.gmail.com
M src/backend/storage/ipc/procsignal.c
Bump PGSTAT_FILE_FORMAT_ID
commit : b2f87b46690ab8cd3ee7c77226fcf3f5bb713e5b
author : Magnus Hagander <[email protected]>
date : Mon, 18 Jan 2021 17:51:49 +0100
committer: Magnus Hagander <[email protected]>
date : Mon, 18 Jan 2021 17:51:49 +0100
This was missed in 960869da08
Reported-By: Laurenz Albe
Discussion: https://postgr.es/m/[email protected]
M src/include/pgstat.h
Check for BuildIndexValueDescription returning NULL in gist_page_items
commit : 5d1e5c8b758770186b005a1c3888b05e37af79c5
author : Heikki Linnakangas <[email protected]>
date : Mon, 18 Jan 2021 14:48:43 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 18 Jan 2021 14:48:43 +0200
Per Coverity. BuildIndexValueDescription() cannot actually return NULL in
this instance, because it only returns NULL if the user doesn't have the
required privileges, and this function can only be used by superuser. But
better safe than sorry.
M contrib/pageinspect/gistfuncs.c
Pause recovery for insufficient parameter settings
commit : 15251c0a60be76eedee74ac0e94b433f9acca5af
author : Peter Eisentraut <[email protected]>
date : Mon, 18 Jan 2021 08:49:10 +0100
committer: Peter Eisentraut <[email protected]>
date : Mon, 18 Jan 2021 08:49:10 +0100
When certain parameters are changed on a physical replication primary,
this is communicated to standbys using the XLOG_PARAMETER_CHANGE WAL
record. The standby then checks whether its own settings are at least
as big as the ones on the primary. If not, the standby shuts down
with a fatal error.
This patch changes this behavior for hot standbys to pause recovery at
that point instead. That allows read traffic on the standby to
continue while database administrators figure out next steps. When
recovery is unpaused, the server shuts down (as before). The idea is
to fix the parameters while recovery is paused and then restart when
there is a maintenance window.
Reviewed-by: Sergei Kornilov <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/high-availability.sgml
M src/backend/access/transam/xlog.c
postgres_fdw: Add function to list cached connections to foreign servers.
commit : 708d165ddb92c54077a372acf6417e258dcb5fef
author : Fujii Masao <[email protected]>
date : Mon, 18 Jan 2021 15:11:08 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 18 Jan 2021 15:11:08 +0900
This commit adds function postgres_fdw_get_connections() to return
the foreign server names of all the open connections that postgres_fdw
established from the local session to the foreign servers. This function
also returns whether each connection is valid or not.
This function is useful when checking all the open foreign server connections.
If we found some connection to drop, from the result of function, probably
we can explicitly close them by the function that upcoming commit will add.
This commit bumps the version of postgres_fdw to 1.1 since it adds
new function.
Author: Bharath Rupireddy, tweaked by Fujii Masao
Reviewed-by: Zhijie Hou, Alexey Kondratov, Zhihong Yu, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/Makefile
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
A contrib/postgres_fdw/postgres_fdw–1.0–1.1.sql
M contrib/postgres_fdw/postgres_fdw.control
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml
Refactor option handling of CLUSTER, REINDEX and VACUUM
commit : a3dc926009be833ea505eebd77ce4b72fe708b18
author : Michael Paquier <[email protected]>
date : Mon, 18 Jan 2021 14:03:10 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 18 Jan 2021 14:03:10 +0900
This continues the work done in b5913f6. All the options of those
commands are changed to use hex values rather than enums to reduce the
risk of compatibility bugs when introducing new options. Each option
set is moved into a new structure that can be extended with more
non-boolean options (this was already the case of VACUUM). The code of
REINDEX is restructured so as manual REINDEX commands go through a
single routine from utility.c, like VACUUM, to ease the allocation
handling of option parameters when a command needs to go through
multiple transactions.
This can be used as a base infrastructure for future patches related to
those commands, including reindex filtering and tablespace support.
Per discussion with people mentioned below, as well as Alvaro Herrera
and Peter Eisentraut.
Author: Michael Paquier, Justin Pryzby
Reviewed-by: Alexey Kondratov, Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/index.c
M src/backend/commands/cluster.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/vacuum.c
M src/backend/tcop/utility.c
M src/include/catalog/index.h
M src/include/commands/cluster.h
M src/include/commands/defrem.h
M src/include/commands/vacuum.h
M src/tools/pgindent/typedefs.list
pageinspect: Fix relcache leak in gist_page_items().
commit : 04eb75e783ba49ca2e0e75088d6590b64be8ed4d
author : Heikki Linnakangas <[email protected]>
date : Mon, 18 Jan 2021 00:46:03 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 18 Jan 2021 00:46:03 +0200
The gist_page_items() function opened the index relation on first call and
closed it on the last call. But there's no guarantee that the function is
run to completion, leading to a relcache leak and warning at the end of
the transaction. To fix, refactor the function to return all the rows in
one call, as a tuplestore.
Reported-by: Tom Lane
Discussion: https://www.postgresql.org/message-id/234863.1610916631%40sss.pgh.pa.us
M contrib/pageinspect/gistfuncs.c
Set PD_ALL_VISIBLE and visibility map bits in COPY FREEZE
commit : 7db0cd2145f2bce84cac92402e205e4d2b045bf2
author : Tomas Vondra <[email protected]>
date : Sun, 17 Jan 2021 22:11:39 +0100
committer: Tomas Vondra <[email protected]>
date : Sun, 17 Jan 2021 22:11:39 +0100
Make sure COPY FREEZE marks the pages as PD_ALL_VISIBLE and updates the
visibility map. Until now we only marked individual tuples as frozen,
but page-level flags were not updated, so the first VACUUM after the
COPY FREEZE had to rewrite the whole table.
This is a fairly old patch, and multiple people worked on it. The first
version was written by Jeff Janes, and then reworked by Pavan Deolasee
and Anastasia Lubennikova.
Author: Anastasia Lubennikova, Pavan Deolasee, Jeff Janes
Reviewed-by: Kuntal Ghosh, Jeff Janes, Tomas Vondra, Masahiko Sawada,
Andres Freund, Ibrar Ahmed, Robert Haas, Tatsuro Ishii,
Darafei Praliaskouski
Discussion: https://postgr.es/m/CABOikdN-ptGv0mZntrK2Q8OtfUuAjqaYMGmkdU1dCKFtUxVLrg@mail.gmail.com
Discussion: https://postgr.es/m/CAMkU%3D1w3osJJ2FneELhhNRLxfZitDgp9FPHee08NT2FQFmz_pQ%40mail.gmail.com
M contrib/pg_visibility/expected/pg_visibility.out
M contrib/pg_visibility/sql/pg_visibility.sql
M src/backend/access/heap/heapam.c
M src/backend/access/heap/hio.c
M src/include/access/heapam_xlog.h
Add missing array-enlargement logic to test_regex.c.
commit : 0c7d3bb99f72d66ec6ac63aee4c5fe6d683eee86
author : Tom Lane <[email protected]>
date : Sun, 17 Jan 2021 12:53:48 -0500
committer: Tom Lane <[email protected]>
date : Sun, 17 Jan 2021 12:53:48 -0500
The stanza to report a "partial" match could overrun the initially
allocated output array, so it needs its own copy of the array-resizing
logic that's in the main loop. I overlooked the need for this in
ca8217c10.
Per report from Alexander Lakhin.
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/test_regex/test_regex.c
Add documentation chapter about checksums
commit : cf621d9d84db1e6edaff8ffa26bad93fdce5f830
author : Magnus Hagander <[email protected]>
date : Sun, 17 Jan 2021 15:31:23 +0100
committer: Magnus Hagander <[email protected]>
date : Sun, 17 Jan 2021 15:31:23 +0100
Data checksums did not have a longer discussion in the docs,
this adds a short section with an overview.
Extracted from the larger patch for on-line enabling of checksums, which
has many more authors and reviewers.
Author: Daniel Gustafsson
Reviewed-By: Magnus Hagander, Michael Banck (and others through the big patch)
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/amcheck.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/wal.sgml
Revert "psql \dX: list extended statistics objects"
commit : 1db0d173a2201119f297ea35edfb41579893dd8c
author : Tomas Vondra <[email protected]>
date : Sun, 17 Jan 2021 15:11:14 +0100
committer: Tomas Vondra <[email protected]>
date : Sun, 17 Jan 2021 15:11:14 +0100
Reverts 891a1d0bca, because the new psql command \dX only worked for
users users who can read pg_statistic_ext_data catalog, and most regular
users lack that privilege (the catalog may contain sensitive user data).
Reported-by: Noriyoshi Shinoda
Discussion: https://postgr.es/m/c027a541-5856-75a5-0868-341301e1624b%40nttcom.co.jp_1
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/bin/psql/help.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Add --no-instructions parameter to initdb
commit : e09155bd62f0ac5817cb3d736eb35adf4200549e
author : Magnus Hagander <[email protected]>
date : Sun, 17 Jan 2021 14:28:17 +0100
committer: Magnus Hagander <[email protected]>
date : Sun, 17 Jan 2021 14:28:17 +0100
Specifying this parameter removes the informational messages about how
to start the server. This is intended for use by wrappers in different
packaging systems, where those instructions would most likely be wrong
anyway, but the other output from initdb would still be useful (and thus
just redirecting everything to /dev/null would be bad).
Author: Magnus Hagander
Reviewed-By: Peter Eisentraut
Discusion: https://postgr.es/m/CABUevEzo4t5bmTXF0_B9WzmuWpVbMpkNZZiGvzV8NZa-=fPqeQ@mail.gmail.com
M doc/src/sgml/ref/initdb.sgml
M src/bin/initdb/initdb.c
Add pg_stat_database counters for sessions and session time
commit : 960869da0803427d14335bba24393f414b476e2c
author : Magnus Hagander <[email protected]>
date : Sun, 17 Jan 2021 13:34:09 +0100
committer: Magnus Hagander <[email protected]>
date : Sun, 17 Jan 2021 13:34:09 +0100
This add counters for number of sessions, the different kind of session
termination types, and timers for how much time is spent in active vs
idle in a database to pg_stat_database.
Internally this also renames the parameter "force" to disconnect. This
was the only use-case for the parameter before, so repurposing it to
this mroe narrow usecase makes things cleaner than inventing something
new.
Author: Laurenz Albe
Reviewed-By: Magnus Hagander, Soumyadeep Chakraborty, Masahiro Ikeda
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/tcop/postgres.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/error/elog.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
psql \dX: list extended statistics objects
commit : 891a1d0bca262ca78564e0fea1eaa5ae544ea5ee
author : Tomas Vondra <[email protected]>
date : Sun, 17 Jan 2021 00:16:25 +0100
committer: Tomas Vondra <[email protected]>
date : Sun, 17 Jan 2021 00:16:25 +0100
The new command lists extended statistics objects, possibly with their
sizes. All past releases with extended statistics are supported.
Author: Tatsuro Yamada
Reviewed-by: Julien Rouhaud, Alvaro Herrera, Tomas Vondra
Discussion: https://postgr.es/m/c027a541-5856-75a5-0868-341301e1624b%40nttcom.co.jp_1
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/bin/psql/help.c
M src/bin/psql/tab-complete.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Documenation fixups for replication protocol.
commit : a32d98351e1ff33c5a65db4ed8e2def70a807bfa
author : Jeff Davis <[email protected]>
date : Sat, 16 Jan 2021 14:40:12 -0800
committer: Jeff Davis <[email protected]>
date : Sat, 16 Jan 2021 14:40:12 -0800
There is no CopyResponse message; it should be CopyOutResponse.
Also, if there is no WAL to stream, the server does not immediately
send a CommandComplete; it's a historical timeline, so it will send a
response tuple first.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/protocol.sgml
Fix pg_dump for GRANT OPTION among initial privileges.
commit : f713ff7c646e5912e08089de74dacdfaaac3d03b
author : Noah Misch <[email protected]>
date : Sat, 16 Jan 2021 12:21:35 -0800
committer: Noah Misch <[email protected]>
date : Sat, 16 Jan 2021 12:21:35 -0800
The context is an object that no longer bears some aclitem that it bore
initially. (A user issued REVOKE or GRANT statements upon the object.)
pg_dump is forming SQL to reproduce the object ACL. Since initdb
creates no ACL bearing GRANT OPTION, reaching this bug requires an
extension where the creation script establishes such an ACL. No PGXN
extension does that. If an installation did reach the bug, pg_dump
would have omitted a semicolon, causing a REVOKE and the next SQL
statement to fail. Separately, since the affected code exists to
eliminate an entire aclitem, it wants plain REVOKE, not REVOKE GRANT
OPTION FOR. Back-patch to 9.6, where commit
23f34fa4ba358671adab16773e79c17c92cbc870 first appeared.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/dumputils.c
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/modules/test_pg_dump/test_pg_dump–1.0.sql
Prevent excess SimpleLruTruncate() deletion.
commit : 6db992833c04c0322f7f34a486adece01651f929
author : Noah Misch <[email protected]>
date : Sat, 16 Jan 2021 12:21:35 -0800
committer: Noah Misch <[email protected]>
date : Sat, 16 Jan 2021 12:21:35 -0800
Every core SLRU wraps around. With the exception of pg_notify, the wrap
point can fall in the middle of a page. Account for this in the
PagePrecedes callback specification and in SimpleLruTruncate()'s use of
said callback. Update each callback implementation to fit the new
specification. This changes SerialPagePrecedesLogically() from the
style of asyncQueuePagePrecedes() to the style of CLOGPagePrecedes().
(Whereas pg_clog and pg_serial share a key space, pg_serial is nothing
like pg_notify.) The bug fixed here has the same symptoms and user
followup steps as 592a589a04bd456410b853d86bd05faa9432cbbb. Back-patch
to 9.5 (all supported versions).
Reviewed by Andrey Borodin and (in earlier versions) by Tom Lane.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/commands/async.c
M src/backend/storage/lmgr/predicate.c
M src/include/access/slru.h
Remove unnecessary pstrdup in fetch_table_list.
commit : c95765f47673b16ed36acbfe98e1242e3c3822a3
author : Amit Kapila <[email protected]>
date : Sat, 16 Jan 2021 10:15:32 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 16 Jan 2021 10:15:32 +0530
The result of TextDatumGetCString is already palloc'ed so we don't need to
allocate memory for it again. We decide not to backpatch it as there
doesn't seem to be any case where it can create a meaningful leak.
Author: Zhijie Hou
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/229fed2eb8c54c71a96ccb99e516eb12@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/commands/subscriptioncmds.c
Disallow CREATE STATISTICS on system catalogs
commit : c9a0dc34865f2f82d8e186f534d9b605afaa4d11
author : Tomas Vondra <[email protected]>
date : Fri, 15 Jan 2021 23:24:19 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 15 Jan 2021 23:24:19 +0100
Add a check that CREATE STATISTICS does not add extended statistics on
system catalogs, similarly to indexes etc. It can be overriden using
the allow_system_table_mods GUC.
This bug exists since 7b504eb282c, adding the extended statistics, so
backpatch all the way back to PostgreSQL 10.
Author: Tomas Vondra
Reported-by: Dean Rasheed
Backpatch-through: 10
Discussion: https://postgr.es/m/CAEZATCXAPrrOKwEsyZKQ4uzzJQWBCt6QAvOcgqRGdWwT1zb%2BrQ%40mail.gmail.com
M src/backend/commands/statscmds.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Improve our heuristic for selecting PG_SYSROOT on macOS.
commit : 4823621db312a0597c40686c4c94d47428889fef
author : Tom Lane <[email protected]>
date : Fri, 15 Jan 2021 11:28:51 -0500
committer: Tom Lane <[email protected]>
date : Fri, 15 Jan 2021 11:28:51 -0500
In cases where Xcode is newer than the underlying macOS version,
asking xcodebuild for the SDK path will produce a pointer to the
SDK shipped with Xcode, which may end up building code that does
not work on the underlying macOS version. It appears that in
such cases, xcodebuild's answer also fails to match the default
behavior of Apple's compiler: assuming one has installed Xcode's
"command line tools", there will be an SDK for the OS's own version
in /Library/Developer/CommandLineTools, and the compiler will
default to using that. This is all pretty poorly documented,
but experimentation suggests that "xcrun --show-sdk-path" gives
the sysroot path that the compiler is actually using, at least
in some cases. Hence, try that first, but revert to xcodebuild
if xcrun fails (in very old Xcode, it is missing or lacks the
--show-sdk-path switch).
Also, "xcrun --show-sdk-path" may give a path that is valid but lacks
any OS version identifier. We don't really want that, since most
of the motivation for wiring -isysroot into the build flags at all
is to ensure that all parts of a PG installation are built against
the same SDK, even when considering extensions built later and/or on
a different machine. Insist on finding "N.N" in the directory name
before accepting the result. (Adding "--sdk macosx" to the xcrun
call seems to produce the same answer as xcodebuild, but usually
more quickly because it's cached, so we also try that as a fallback.)
The core reason why we don't want to use Xcode's default SDK in cases
like this is that Apple's technology for introducing new syscalls
does not play nice with Autoconf: for example, configure will think
that preadv/pwritev exist when using a Big Sur SDK, even when building
on an older macOS version where they don't exist. It'd be nice to
have a better solution to that problem, but this patch doesn't attempt
to fix that.
Per report from Sergey Shinderuk. Back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]
M src/template/darwin
Avoid spurious wait in concurrent reindex
commit : f9900df5f94936067e6fa24a9df609863eb08da2
author : Alvaro Herrera <[email protected]>
date : Fri, 15 Jan 2021 10:31:42 -0300
committer: Alvaro Herrera <[email protected]>
date : Fri, 15 Jan 2021 10:31:42 -0300
This is like commit c98763bf51bf, but for REINDEX CONCURRENTLY. To wit:
this flags indicates that the current process is safe to ignore for the
purposes of waiting for other snapshots, when doing CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY. This helps two processes doing
either of those things not deadlock, and also avoids spurious waits.
Author: Álvaro Herrera <[email protected]>
Reviewed-by: Dmitry Dolgov <[email protected]>
Reviewed-by: Hamid Akhtar <[email protected]>
Reviewed-by: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/indexcmds.c
M src/include/storage/proc.h
Fix calculation of how much shared memory is required to store a TOC.
commit : 2ad78a87f018260d4474eee63187e1cc73c9b976
author : Fujii Masao <[email protected]>
date : Fri, 15 Jan 2021 12:44:17 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 15 Jan 2021 12:44:17 +0900
Commit ac883ac453 refactored shm_toc_estimate() but changed its calculation
of shared memory size for TOC incorrectly. Previously this could cause too
large memory to be allocated.
Back-patch to v11 where the bug was introduced.
Author: Takayuki Tsunakawa
Discussion: https://postgr.es/m/TYAPR01MB2990BFB73170E2C4921E2C4DFEA80@TYAPR01MB2990.jpnprd01.prod.outlook.com
M src/backend/storage/ipc/shm_toc.c
Remove PG_SHA*_DIGEST_STRING_LENGTH from sha2.h
commit : ccf4e277a4de120a2f08db7e45399d87e1176bda
author : Michael Paquier <[email protected]>
date : Fri, 15 Jan 2021 11:46:34 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 15 Jan 2021 11:46:34 +0900
The last reference to those variables has been removed in aef8948, so
this cleans up a bit the code.
Discussion: https://postgr.es/m/X//[email protected]
M src/include/common/sha2.h
Fix O(N^2) stat() calls when recycling WAL segments
commit : 5ae1572993ae8bf1f6c33a933915c07cc9bc0add
author : Michael Paquier <[email protected]>
date : Fri, 15 Jan 2021 10:33:13 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 15 Jan 2021 10:33:13 +0900
The counter tracking the last segment number recycled was getting
initialized when recycling one single segment, while it should be used
across a full cycle of segments recycled to prevent useless checks
related to entries already recycled.
This performance issue has been introduced by b2a5545, and it was first
implemented in 61b86142.
No backpatch is done per the lack of field complaints.
Reported-by: Andres Freund, Thomas Munro
Author: Michael Paquier
Reviewed-By: Andres Freund
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CA+hUKG+DRiF9z1_MU4fWq+RfJMxP7zjoptfcmuCFPeO4JM2iVg@mail.gmail.com
M src/backend/access/transam/xlog.c
postgres_fdw: Save foreign server OID in connection cache entry.
commit : 5e5f4fcd89c082bba0239e8db1552834b4905c34
author : Fujii Masao <[email protected]>
date : Fri, 15 Jan 2021 10:30:19 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 15 Jan 2021 10:30:19 +0900
The foreign server OID stored in the connection cache entry is used as
a lookup key to directly get the server name.
Previously since the connection cache entry did not have the server OID,
postgres_fdw had to get the server OID at first from user mapping before
getting the server name. So if the corresponding user mapping was dropped,
postgres_fdw could raise the error "cache lookup failed for user mapping"
while looking up user mapping and fail to get the server name even though
the server had not been dropped yet.
Author: Bharath Rupireddy
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVRZPUB7ZwqLn-6DY8C_UmPs6084gSpHA92YBv++1AJXA@mail.gmail.com
M contrib/postgres_fdw/connection.c
pg_dump: label PUBLICATION TABLE ArchiveEntries with an owner.
commit : 8e396a773b80c72e5d5a0ca9755dffe043c97a05
author : Tom Lane <[email protected]>
date : Thu, 14 Jan 2021 16:19:38 -0500
committer: Tom Lane <[email protected]>
date : Thu, 14 Jan 2021 16:19:38 -0500
This is the same fix as commit 9eabfe300 applied to INDEX ATTACH
entries, but for table-to-publication attachments. As in that
case, even though the backend doesn't record "ownership" of the
attachment, we still ought to label it in the dump archive with
the role name that should run the ALTER PUBLICATION command.
The existing behavior causes the ALTER to be done by the original
role that started the restore; that will usually work fine, but
there may be corner cases where it fails.
The bulk of the patch is concerned with changing struct
PublicationRelInfo to include a pointer to the associated
PublicationInfo object, so that we can get the owner's name
out of that when the time comes. While at it, I rewrote
getPublicationTables() to do just one query of pg_publication_rel,
not one per table.
Back-patch to v10 where this code was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
Prevent drop of tablespaces used by partitioned relations
commit : ebfe2dbd6b624e2a428e14b7ee9322cc096f63f7
author : Alvaro Herrera <[email protected]>
date : Thu, 14 Jan 2021 15:32:14 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 14 Jan 2021 15:32:14 -0300
When a tablespace is used in a partitioned relation (per commits
ca4103025dfe in pg12 for tables and 33e6c34c3267 in pg11 for indexes),
it is possible to drop the tablespace, potentially causing various
problems. One such was reported in bug #16577, where a rewriting ALTER
TABLE causes a server crash.
Protect against this by using pg_shdepend to keep track of tablespaces
when used for relations that don't keep physical files; we now abort a
tablespace if we see that the tablespace is referenced from any
partitioned relations.
Backpatch this to 11, where this problem has been latent all along. We
don't try to create pg_shdepend entries for existing partitioned
indexes/tables, but any ones that are modified going forward will be
protected.
Note slight behavior change: when trying to drop a tablespace that
contains both regular tables as well as partitioned ones, you'd
previously get ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE and now you'll
get ERRCODE_DEPENDENT_OBJECTS_STILL_EXIST. Arguably, the latter is more
correct.
It is possible to add protecting pg_shdepend entries for existing
tables/indexes, by doing
ALTER TABLE ONLY some_partitioned_table SET TABLESPACE pg_default;
ALTER TABLE ONLY some_partitioned_table SET TABLESPACE original_tablespace;
for each partitioned table/index that is not in the database default
tablespace. Because these partitioned objects do not have storage, no
file needs to be actually moved, so it shouldn't take more time than
what's required to acquire locks.
This query can be used to search for such relations:
SELECT ... FROM pg_class WHERE relkind IN ('p', 'I') AND reltablespace <> 0
Reported-by: Alexander Lakhin <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Author: Álvaro Herrera <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/heap.c
M src/backend/catalog/pg_shdepend.c
M src/backend/commands/tablecmds.c
M src/backend/commands/tablespace.c
M src/include/catalog/dependency.h
M src/test/regress/input/tablespace.source
M src/test/regress/output/tablespace.source
Stabilize timeline switch regression test.
commit : 424d7a9b277c0da5ec638bf6344cda899a2e544a
author : Fujii Masao <[email protected]>
date : Thu, 14 Jan 2021 14:37:01 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 14 Jan 2021 14:37:01 +0900
Commit fef5b47f6b added the regression test to check whether a standby is
able to follow a primary on a newer timeline when WAL archiving is enabled.
But the buildfarm member florican reported that this test failed because
the requested WAL segment was removed and replication failed. This is a
timing issue. Since neither replication slot is used nor wal_keep_size is set
in the test, checkpoint could remove the WAL segment that's still necessary
for replication.
This commit stabilizes the test by setting wal_keep_size.
Back-patch to v13 where the regression test that this commit stabilizes
was added.
Author: Fujii Masao
Discussion: https://postgr.es/m/X//[email protected]
M src/test/recovery/t/004_timeline_switch.pl
Improve tab-completion for CLOSE, DECLARE, FETCH and MOVE.
commit : 3f238b882c276a59f5d98224850e5aee2a3fec8c
author : Fujii Masao <[email protected]>
date : Thu, 14 Jan 2021 15:41:22 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 14 Jan 2021 15:41:22 +0900
This commit makes CLOSE, FETCH and MOVE commands tab-complete the list of
cursors. Also this commit makes DECLARE command tab-complete the options.
Author: Shinya Kato, Sawada Masahiko, tweaked by Fujii Masao
Reviewed-by: Shinya Kato, Sawada Masahiko, Fujii Masao
Discussion: https://postgr.es/m/b0e4c5c53ef84c5395524f5056fc71f0@MP-MSGSS-MBX001.msg.nttdata.co.jp
M src/bin/psql/tab-complete.c
Minor header cleanup for the new iovec code.
commit : fb29ab26b38f1647423bf6aa2991e5fdf9060f08
author : Thomas Munro <[email protected]>
date : Thu, 14 Jan 2021 18:09:32 +1300
committer: Thomas Munro <[email protected]>
date : Thu, 14 Jan 2021 18:09:32 +1300
Remove redundant function declaration and improve header comment in
pg_iovec.h. Move the new declaration in fd.h next to a group of more
similar functions.
M src/include/port/pg_iovec.h
M src/include/storage/fd.h
Ensure that a standby is able to follow a primary on a newer timeline.
commit : fef5b47f6bfc9bfec619bb2e6e66b027e7ff21a3
author : Fujii Masao <[email protected]>
date : Thu, 14 Jan 2021 12:27:11 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 14 Jan 2021 12:27:11 +0900
Commit 709d003fbd refactored WAL-reading code, but accidentally caused
WalSndSegmentOpen() to fail to follow a timeline switch while reading from
a historic timeline. This issue caused a standby to fail to follow a primary
on a newer timeline when WAL archiving is enabled.
If there is a timeline switch within the segment, WalSndSegmentOpen() should
read from the WAL segment belonging to the new timeline. But previously
since it failed to follow a timeline switch, it tried to read the WAL segment
with old timeline. When WAL archiving is enabled, that WAL segment with
old timeline doesn't exist because it's renamed to .partial. This leads
a primary to have tried to read non-existent WAL segment, and which caused
replication to faill with the error "ERROR: requested WAL segment ... has
already been removed".
This commit fixes WalSndSegmentOpen() so that it's able to follow a timeline
switch, to ensure that a standby is able to follow a primary on a newer
timeline even when WAL archiving is enabled.
This commit also adds the regression test to check whether a standby is
able to follow a primary on a newer timeline when WAL archiving is enabled.
Back-patch to v13 where the bug was introduced.
Reported-by: Kyotaro Horiguchi
Author: Kyotaro Horiguchi, tweaked by Fujii Masao
Reviewed-by: Alvaro Herrera, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/walsender.c
M src/test/recovery/t/004_timeline_switch.pl
Rework refactoring of hex and encoding routines
commit : aef8948f38d9f3aa58bf8c2d4c6f62a7a456a9d1
author : Michael Paquier <[email protected]>
date : Thu, 14 Jan 2021 11:13:24 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 14 Jan 2021 11:13:24 +0900
This commit addresses some issues with c3826f83 that moved the hex
decoding routine to src/common/:
- The decoding function lacked overflow checks, so when used for
security-related features it was an open door to out-of-bound writes if
not carefully used that could remain undetected. Like the base64
routines already in src/common/ used by SCRAM, this routine is reworked
to check for overflows by having the size of the destination buffer
passed as argument, with overflows checked before doing any writes.
- The encoding routine was missing. This is moved to src/common/ and
it gains the same overflow checks as the decoding part.
On failure, the hex routines of src/common/ issue an error as per the
discussion done to make them usable by frontend tools, but not by shared
libraries. Note that this is why ECPG is left out of this commit, and
it still includes a duplicated logic doing hex encoding and decoding.
While on it, this commit uses better variable names for the source and
destination buffers in the existing escape and base64 routines in
encode.c and it makes them more robust to overflow detection. The
previous core code issued a FATAL after doing out-of-bound writes if
going through the SQL functions, which would be enough to detect
problems when working on changes that impacted this area of the
code. Instead, an error is issued before doing an out-of-bound write.
The hex routines were being directly called for bytea conversions and
backup manifests without such sanity checks. The current calls happen
to not have any problems, but careless uses of such APIs could easily
lead to CVE-class bugs.
Author: Bruce Momjian, Michael Paquier
Reviewed-by: Sehrope Sarkuni
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/backup_manifest.c
M src/backend/utils/adt/encode.c
M src/backend/utils/adt/varlena.c
M src/common/Makefile
A src/common/hex.c
D src/common/hex_decode.c
A src/include/common/hex.h
D src/include/common/hex_decode.h
M src/include/utils/builtins.h
M src/tools/msvc/Mkvcbuild.pm
Move our p{read,write}v replacements into their own files.
commit : 0d56acfbaa799553c0c6ea350fd6e68d81025994
author : Thomas Munro <[email protected]>
date : Thu, 14 Jan 2021 11:10:24 +1300
committer: Thomas Munro <[email protected]>
date : Thu, 14 Jan 2021 11:10:24 +1300
macOS's ranlib issued a warning about an empty pread.o file with the
previous arrangement, on systems new enough to require no replacement
functions. Let's go back to using configure's AC_REPLACE_FUNCS system
to build and include each .o in the library only if it's needed, which
requires moving the *v() functions to their own files.
Also move the _with_retry() wrapper to a more permanent home.
Reported-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/1283127.1610554395%40sss.pgh.pa.us
M configure
M configure.ac
M src/backend/storage/file/fd.c
M src/include/storage/fd.h
M src/port/Makefile
M src/port/pread.c
A src/port/preadv.c
M src/port/pwrite.c
A src/port/pwritev.c
M src/tools/msvc/Mkvcbuild.pm
Mark inet_server_addr() and inet_server_port() as parallel-restricted.
commit : 5a6f9bce8dabd371bdb4e3db5dda436f7f0a680f
author : Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 16:23:15 -0500
committer: Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 16:23:15 -0500
These need to be PR because they access the MyProcPort data structure,
which doesn't get copied to parallel workers. The very similar
functions inet_client_addr() and inet_client_port() are already
marked PR, but somebody missed these.
Although this is a pre-existing bug, we can't readily fix it in the back
branches since we can't force initdb. Given the small usage of these
two functions, and the even smaller likelihood that they'd get pushed to
a parallel worker anyway, it doesn't seem worth the trouble to suggest
that DBAs should fix it manually.
Masahiko Sawada
Discussion: https://postgr.es/m/CAD21AoAT4aHP0Uxq91qpD7NL009tnUYQe-b14R3MnSVOjtE71g@mail.gmail.com
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
Run reformat-dat-files to declutter the catalog data files.
commit : 8b411b8ff41566a1aa601d1f05aeebbebbdb4a54
author : Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 16:14:38 -0500
committer: Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 16:14:38 -0500
Things had gotten pretty messy here, apparently mostly but not
entirely the fault of the multirange patch. No functional changes.
M src/include/catalog/pg_aggregate.dat
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_cast.dat
M src/include/catalog/pg_collation.dat
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_range.dat
M src/include/catalog/pg_type.dat
Doc, more or less: uncomment tutorial example that was fixed long ago.
commit : dce62490818170b6479dfe08a28aae4bcdf7cc2d
author : Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 15:59:57 -0500
committer: Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 15:59:57 -0500
Reverts a portion of commit 344190b7e. Apparently, back in the
twentieth century we had some issues with multi-statement SQL
functions, but they've worked fine for a long time.
Daniel Westermann
Discussion: https://postgr.es/m/GVAP278MB04242DCBF5E31F528D53FA18D2A90@GVAP278MB0424.CHEP278.PROD.OUTLOOK.COM
M src/tutorial/funcs.source
Call out vacuum considerations in create index docs
commit : 93c39f987e9c019cd28c450ece8a621b2d8ce28a
author : Alvaro Herrera <[email protected]>
date : Wed, 13 Jan 2021 17:55:41 -0300
committer: Alvaro Herrera <[email protected]>
date : Wed, 13 Jan 2021 17:55:41 -0300
Backpatch to pg12, which is as far as it goes without conflicts.
Author: James Coleman <[email protected]>
Reviewed-by: "David G. Johnston" <[email protected]>
Discussion: https://postgr.es/m/CAAaqYe9oEfbz7AxXq7OX+FFVi5w5p1e_Of8ON8ZnKO9QqBfmjg@mail.gmail.com
M doc/src/sgml/ref/create_index.sgml
M doc/src/sgml/ref/reindex.sgml
Disallow a digit as the first character of a variable name in pgbench.
commit : c21ea4d53e9404279273da800daa49b7b9a5e81e
author : Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 14:52:49 -0500
committer: Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 14:52:49 -0500
The point of this restriction is to avoid trying to substitute variables
into timestamp literal values, which may contain strings like '12:34'.
There is a good deal more that should be done to reduce pgbench's
tendency to substitute where it shouldn't. But this is sufficient to
solve the case complained of by Jaime Soler, and it's simple enough
to back-patch.
Back-patch to v11; before commit 9d36a3866, pgbench had a slightly
different definition of what a variable name is, and anyway it seems
unwise to change long-stable branches for this.
Fabien Coelho
Discussion: https://postgr.es/m/alpine.DEB.2.22.394.2006291740420.805678@pseudo
M doc/src/sgml/ref/pgbench.sgml
M src/bin/pgbench/pgbench.c
Fix test failure with wal_level=minimal.
commit : 5abca4b1cd7193d6a5c8235c97eb240312190bcb
author : Heikki Linnakangas <[email protected]>
date : Wed, 13 Jan 2021 20:57:21 +0200
committer: Heikki Linnakangas <[email protected]>
date : Wed, 13 Jan 2021 20:57:21 +0200
The newly-added gist pageinspect test prints the LSNs of GiST pages,
expecting them all to be 1 (GistBuildLSN). But with wal_level=minimal,
they got updated by the whole-relation WAL-logging at commit. Fix by
wrapping the problematic tests in the same transaction with the CREATE
INDEX.
Per buildfarm failure on thorntail.
Discussion: https://www.postgresql.org/message-id/3B4F97E5-40FB-4142-8CAA-B301CDFBF982%40iki.fi
M contrib/pageinspect/expected/gist.out
M contrib/pageinspect/sql/gist.sql
Doc: clarify behavior of back-half options in pg_dump.
commit : 06ed235adeb621a73cafd6ab35fa2405b3177329
author : Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 13:30:04 -0500
committer: Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 13:30:04 -0500
Options that change how the archive data is converted to SQL text
are ignored when dumping to archive formats. The documentation
previously said "not meaningful", which is not helpful.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/pg_dump.sgml
Enhance nbtree index tuple deletion.
commit : d168b666823b6e0bcf60ed19ce24fb5fb91b8ccf
author : Peter Geoghegan <[email protected]>
date : Wed, 13 Jan 2021 09:21:32 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 13 Jan 2021 09:21:32 -0800
Teach nbtree and heapam to cooperate in order to eagerly remove
duplicate tuples representing dead MVCC versions. This is "bottom-up
deletion". Each bottom-up deletion pass is triggered lazily in response
to a flood of versions on an nbtree leaf page. This usually involves a
"logically unchanged index" hint (these are produced by the executor
mechanism added by commit 9dc718bd).
The immediate goal of bottom-up index deletion is to avoid "unnecessary"
page splits caused entirely by version duplicates. It naturally has an
even more useful effect, though: it acts as a backstop against
accumulating an excessive number of index tuple versions for any given
_logical row_. Bottom-up index deletion complements what we might now
call "top-down index deletion": index vacuuming performed by VACUUM.
Bottom-up index deletion responds to the immediate local needs of
queries, while leaving it up to autovacuum to perform infrequent clean
sweeps of the index. The overall effect is to avoid certain
pathological performance issues related to "version churn" from UPDATEs.
The previous tableam interface used by index AMs to perform tuple
deletion (the table_compute_xid_horizon_for_tuples() function) has been
replaced with a new interface that supports certain new requirements.
Many (perhaps all) of the capabilities added to nbtree by this commit
could also be extended to other index AMs. That is left as work for a
later commit.
Extend deletion of LP_DEAD-marked index tuples in nbtree by adding logic
to consider extra index tuples (that are not LP_DEAD-marked) for
deletion in passing. This increases the number of index tuples deleted
significantly in many cases. The LP_DEAD deletion process (which is now
called "simple deletion" to clearly distinguish it from bottom-up
deletion) won't usually need to visit any extra table blocks to check
these extra tuples. We have to visit the same table blocks anyway to
generate a latestRemovedXid value (at least in the common case where the
index deletion operation's WAL record needs such a value).
Testing has shown that the "extra tuples" simple deletion enhancement
increases the number of index tuples deleted with almost any workload
that has LP_DEAD bits set in leaf pages. That is, it almost never fails
to delete at least a few extra index tuples. It helps most of all in
cases that happen to naturally have a lot of delete-safe tuples. It's
not uncommon for an individual deletion operation to end up deleting an
order of magnitude more index tuples compared to the old naive approach
(e.g., custom instrumentation of the patch shows that this happens
fairly often when the regression tests are run).
Add a further enhancement that augments simple deletion and bottom-up
deletion in indexes that make use of deduplication: Teach nbtree's
_bt_delitems_delete() function to support granular TID deletion in
posting list tuples. It is now possible to delete individual TIDs from
posting list tuples provided the TIDs have a tableam block number of a
table block that gets visited as part of the deletion process (visiting
the table block can be triggered directly or indirectly). Setting the
LP_DEAD bit of a posting list tuple is still an all-or-nothing thing,
but that matters much less now that deletion only needs to start out
with the right _general_ idea about which index tuples are deletable.
Bump XLOG_PAGE_MAGIC because xl_btree_delete changed.
No bump in BTREE_VERSION, since there are no changes to the on-disk
representation of nbtree indexes. Indexes built on PostgreSQL 12 or
PostgreSQL 13 will automatically benefit from bottom-up index deletion
(i.e. no reindexing required) following a pg_upgrade. The enhancement
to simple deletion is available with all B-Tree indexes following a
pg_upgrade, no matter what PostgreSQL version the user upgrades from.
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Heikki Linnakangas <[email protected]>
Reviewed-By: Victor Yegorov <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wzm+maE3apHB8NOtmM=p-DO65j2V5GzAWCOEEuy3JZgb2g@mail.gmail.com
M doc/src/sgml/btree.sgml
M doc/src/sgml/ref/create_index.sgml
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/index/genam.c
M src/backend/access/nbtree/README
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/rmgrdesc/nbtdesc.c
M src/backend/access/table/tableamapi.c
M src/include/access/heapam.h
M src/include/access/nbtree.h
M src/include/access/nbtxlog.h
M src/include/access/tableam.h
M src/include/access/xlog_internal.h
Pass down "logically unchanged index" hint.
commit : 9dc718bdf2b1a574481a45624d42b674332e2903
author : Peter Geoghegan <[email protected]>
date : Wed, 13 Jan 2021 08:11:00 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 13 Jan 2021 08:11:00 -0800
Add an executor aminsert() hint mechanism that informs index AMs that
the incoming index tuple (the tuple that accompanies the hint) is not
being inserted by execution of an SQL statement that logically modifies
any of the index's key columns.
The hint is received by indexes when an UPDATE takes place that does not
apply an optimization like heapam's HOT (though only for indexes where
all key columns are logically unchanged). Any index tuple that receives
the hint on insert is expected to be a duplicate of at least one
existing older version that is needed for the same logical row. Related
versions will typically be stored on the same index page, at least
within index AMs that apply the hint.
Recognizing the difference between MVCC version churn duplicates and
true logical row duplicates at the index AM level can help with cleanup
of garbage index tuples. Cleanup can intelligently target tuples that
are likely to be garbage, without wasting too many cycles on less
promising tuples/pages (index pages with little or no version churn).
This is infrastructure for an upcoming commit that will teach nbtree to
perform bottom-up index deletion. No index AM actually applies the hint
just yet.
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Victor Yegorov <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wz=CEKFa74EScx_hFVshCOn6AA5T-ajFASTdzipdkLTNQQ@mail.gmail.com
M contrib/bloom/blinsert.c
M contrib/bloom/bloom.h
M doc/src/sgml/indexam.sgml
M src/backend/access/brin/brin.c
M src/backend/access/common/toast_internals.c
M src/backend/access/gin/gininsert.c
M src/backend/access/gist/gist.c
M src/backend/access/hash/hash.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/index/indexam.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/spgist/spginsert.c
M src/backend/catalog/indexing.c
M src/backend/commands/constraint.c
M src/backend/commands/copyfrom.c
M src/backend/commands/trigger.c
M src/backend/executor/execIndexing.c
M src/backend/executor/execMain.c
M src/backend/executor/execReplication.c
M src/backend/executor/nodeModifyTable.c
M src/backend/replication/logical/worker.c
M src/include/access/amapi.h
M src/include/access/brin_internal.h
M src/include/access/genam.h
M src/include/access/gin_private.h
M src/include/access/gist_private.h
M src/include/access/hash.h
M src/include/access/nbtree.h
M src/include/access/spgist.h
M src/include/executor/executor.h
M src/test/modules/dummy_index_am/dummy_index_am.c
Log long wait time on recovery conflict when it's resolved.
commit : 39b03690b529935a3c33024ee68f08e2d347cf4f
author : Fujii Masao <[email protected]>
date : Wed, 13 Jan 2021 22:59:17 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 13 Jan 2021 22:59:17 +0900
This is a follow-up of the work done in commit 0650ff2303. This commit
extends log_recovery_conflict_waits so that a log message is produced
also when recovery conflict has already been resolved after deadlock_timeout
passes, i.e., when the startup process finishes waiting for recovery
conflict after deadlock_timeout. This is useful in investigating how long
recovery conflicts prevented the recovery from applying WAL.
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Bertrand Drouvot
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/proc.c
M src/include/storage/standby.h
Fix portability issues in the new gist pageinspect test.
commit : 6ecaaf810b8b9e3132f00549e630400f38a7ca37
author : Heikki Linnakangas <[email protected]>
date : Wed, 13 Jan 2021 12:32:54 +0200
committer: Heikki Linnakangas <[email protected]>
date : Wed, 13 Jan 2021 12:32:54 +0200
1. The raw bytea representation of the point-type keys used in the test
depends on endianess. Remove the raw key_data column from the test.
2. The items stored on non-leftmost gist page depends on how many items
git on the other pages. This showed up as a failure on 32-bit i386
systems. To fix, only test the gist_page_items() function on the
leftmost leaf page.
Per Andrey Borodin and the buildfarm.
Discussion: https://www.postgresql.org/message-id/9FCEC1DC-86FB-4A57-88EF-DD13663B36AF%40yandex-team.ru
M contrib/pageinspect/expected/gist.out
M contrib/pageinspect/sql/gist.sql
Remove incorrect markup
commit : e6eeb8d799c703eb829612ddcb4fd5ba82d3e880
author : Magnus Hagander <[email protected]>
date : Wed, 13 Jan 2021 11:07:37 +0100
committer: Magnus Hagander <[email protected]>
date : Wed, 13 Jan 2021 11:07:37 +0100
Seems 737d69ffc3c made a copy/paste or automation error resulting in two
extra right-parenthesis.
Reported-By: Michael Vastola
Backpatch-through: 13
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
Add functions to 'pageinspect' to inspect GiST indexes.
commit : 756ab29124d7850d4392e2227b67b69b61576cd6
author : Heikki Linnakangas <[email protected]>
date : Wed, 13 Jan 2021 10:33:33 +0200
committer: Heikki Linnakangas <[email protected]>
date : Wed, 13 Jan 2021 10:33:33 +0200
Author: Andrey Borodin and me
Discussion: https://www.postgresql.org/message-id/3E4F9093-A1B5-4DF8-A292-0B48692E3954%40yandex-team.ru
M contrib/pageinspect/Makefile
A contrib/pageinspect/expected/gist.out
A contrib/pageinspect/gistfuncs.c
A contrib/pageinspect/pageinspect–1.8–1.9.sql
M contrib/pageinspect/pageinspect.control
A contrib/pageinspect/sql/gist.sql
M doc/src/sgml/pageinspect.sgml
Don't use elog() in src/port/pwrite.c.
commit : df10ac625c1672edf839ff59cfcac9dcc097515c
author : Thomas Munro <[email protected]>
date : Wed, 13 Jan 2021 19:11:09 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 13 Jan 2021 19:11:09 +1300
Nothing broke because of this oversight yet, but it would fail to link
if we tried to use pg_pwrite() in frontend code on a system that lacks
pwrite(). Use an assertion instead. Also pgindent while here.
Discussion: https://postgr.es/m/CA%2BhUKGL57RvoQsS35TVPnQoPYqbtBixsdRhynB8NpcUKpHTTtg%40mail.gmail.com
M src/port/pwrite.c
Fix memory leak in SnapBuildSerialize.
commit : ee1b38f65948cb09ecf3c39b58bd88aabc950e7c
author : Amit Kapila <[email protected]>
date : Wed, 13 Jan 2021 08:19:50 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 13 Jan 2021 08:19:50 +0530
The memory for the snapshot was leaked while serializing it to disk during
logical decoding. This memory will be freed only once walsender stops
streaming the changes. This can lead to a huge memory increase when master
logs Standby Snapshot too frequently say when the user is trying to create
many replication slots.
Reported-by: [email protected]
Diagnosed-by: [email protected]
Author: Amit Kapila
Backpatch-through: 9.5
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/snapbuild.c
Optimize DropRelFileNodesAllBuffers() for recovery.
commit : bea449c635c0e68e21610593594c1e5d52842cdd
author : Amit Kapila <[email protected]>
date : Wed, 13 Jan 2021 07:46:11 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 13 Jan 2021 07:46:11 +0530
Similar to commit d6ad34f341, this patch optimizes
DropRelFileNodesAllBuffers() by avoiding the complete buffer pool scan and
instead find the buffers to be invalidated by doing lookups in the
BufMapping table.
This optimization helps operations where the relation files need to be
removed like Truncate, Drop, Abort of Create Table, etc.
Author: Kirk Jamison
Reviewed-by: Kyotaro Horiguchi, Takayuki Tsunakawa, and Amit Kapila
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/OSBPR01MB3207DCA7EC725FDD661B3EDAEF660@OSBPR01MB3207.jpnprd01.prod.outlook.com
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/smgr/smgr.c
M src/include/storage/bufmgr.h
Fix routine name in comment of catcache.c
commit : fce7d0e6efbef304e81846c75eddf73099628d10
author : Michael Paquier <[email protected]>
date : Wed, 13 Jan 2021 10:32:21 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 13 Jan 2021 10:32:21 +0900
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/CALj2ACUDXLAkf_XxQO9tAUtnTNGi3Lmd8fANd+vBJbcHn1HoWA@mail.gmail.com
M src/backend/utils/cache/catcache.c
Invent struct ReindexIndexInfo
commit : c6c4b37395b22f7e649234453a0aafe74d61b7e6
author : Alvaro Herrera <[email protected]>
date : Tue, 12 Jan 2021 17:04:49 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 12 Jan 2021 17:04:49 -0300
This struct is used by ReindexRelationConcurrently to keep track of the
relations to process. This saves having to obtain some data repeatedly,
and has future uses as well.
Reviewed-by: Dmitry Dolgov <[email protected]>
Reviewed-by: Hamid Akhtar <[email protected]>
Reviewed-by: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/indexcmds.c
M src/tools/pgindent/typedefs.list
pg_dump: label INDEX ATTACH ArchiveEntries with an owner.
commit : 9eabfe300a22ad3d776dc293265e15379790bd9a
author : Tom Lane <[email protected]>
date : Tue, 12 Jan 2021 13:37:38 -0500
committer: Tom Lane <[email protected]>
date : Tue, 12 Jan 2021 13:37:38 -0500
Although a partitioned index's attachment to its parent doesn't
have separate ownership, the ArchiveEntry for it needs to be
marked with an owner anyway, to ensure that the ALTER command
is run by the appropriate role when restoring with
--use-set-session-authorization. Without this, the ALTER will
be run by the role that started the restore session, which will
usually work but it's formally the wrong thing.
Back-patch to v11 where this type of ArchiveEntry was added.
In HEAD, add equivalent commentary to the just-added TABLE ATTACH
case, which I'd made do the right thing already.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_dump.c
Doc: fix description of privileges needed for ALTER PUBLICATION.
commit : cc865c0f319fde22540625e02863f42e9853b3e4
author : Tom Lane <[email protected]>
date : Tue, 12 Jan 2021 12:52:14 -0500
committer: Tom Lane <[email protected]>
date : Tue, 12 Jan 2021 12:52:14 -0500
Adding a table to a publication requires ownership of the table
(in addition to ownership of the publication). This was mentioned
nowhere.
M doc/src/sgml/ref/alter_publication.sgml
Fix thinko in comment
commit : a3e51a36b77ab073df88860cc21b2a6094d699d4
author : Alvaro Herrera <[email protected]>
date : Tue, 12 Jan 2021 11:48:45 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 12 Jan 2021 11:48:45 -0300
This comment has been wrong since its introduction in commit
2c03216d8311.
Author: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/CAD21AoAzz6qipFJBbGEaHmyWxvvNDp8httbwLR9tUQWaTjUs2Q@mail.gmail.com
M src/backend/access/transam/xlogreader.c
Fix relation descriptor leak.
commit : 044aa9e70e552d9adb4f6429a1b0e96dde946a92
author : Amit Kapila <[email protected]>
date : Tue, 12 Jan 2021 08:19:39 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 12 Jan 2021 08:19:39 +0530
We missed closing the relation descriptor while sending changes via the
root of partitioned relations during logical replication.
Author: Amit Langote and Mark Zhao
Reviewed-by: Amit Kapila and Ashutosh Bapat
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/pgoutput/pgoutput.c
Optimize DropRelFileNodeBuffers() for recovery.
commit : d6ad34f3410f101f9909c0918a49d6ce86fa216c
author : Amit Kapila <[email protected]>
date : Tue, 12 Jan 2021 07:45:40 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 12 Jan 2021 07:45:40 +0530
The recovery path of DropRelFileNodeBuffers() is optimized so that
scanning of the whole buffer pool can be avoided when the number of
blocks to be truncated in a relation is below a certain threshold. For
such cases, we find the buffers by doing lookups in BufMapping table.
This improves the performance by more than 100 times in many cases
when several small tables (tested with 1000 relations) are truncated
and where the server is configured with a large value of shared
buffers (greater than equal to 100GB).
This optimization helps cases (a) when vacuum or autovacuum truncated off
any of the empty pages at the end of a relation, or (b) when the relation is
truncated in the same transaction in which it was created.
This commit introduces a new API smgrnblocks_cached which returns a cached
value for the number of blocks in a relation fork. This helps us to determine
the exact size of relation which is required to apply this optimization. The
exact size is required to ensure that we don't leave any buffer for the
relation being dropped as otherwise the background writer or checkpointer
can lead to a PANIC error while flushing buffers corresponding to files that
don't exist.
Author: Kirk Jamison based on ideas by Amit Kapila
Reviewed-by: Kyotaro Horiguchi, Takayuki Tsunakawa, and Amit Kapila
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/OSBPR01MB3207DCA7EC725FDD661B3EDAEF660@OSBPR01MB3207.jpnprd01.prod.outlook.com
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/smgr/smgr.c
M src/include/storage/bufmgr.h
M src/include/storage/smgr.h
Dump ALTER TABLE ... ATTACH PARTITION as a separate ArchiveEntry.
commit : 9a4c0e36fbd671b5e7426a5a0670bdd7ba2714a0
author : Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 21:09:03 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 21:09:03 -0500
Previously, we emitted the ATTACH PARTITION command as part of
the child table's ArchiveEntry. This was a poor choice since it
complicates restoring the partition as a standalone table; you have
to ignore the error from the ATTACH, which isn't even an option when
restoring direct-to-database with pg_restore. (pg_restore will issue
the whole ArchiveEntry as one PQexec, so that any error rolls back
the table creation as well.) Hence, separate it out as its own
ArchiveEntry, as indeed we already did for index ATTACH PARTITION
commands.
Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/pg_dump_sort.c
Make pg_dump's table of object-type priorities more maintainable.
commit : d5ab79d815783fe60062cefc423b54e82fbb92ff
author : Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 19:58:07 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 19:58:07 -0500
Wedging a new object type into this table has historically required
manually renumbering a lot of existing entries. (Although it appears
that some people got lazy and re-used the priority level of an
existing object type, even if it wasn't particularly related.)
We can let the compiler do the counting by inventing an enum type that
lists the desired priority levels in order. Now, if you want to add
or remove a priority level, that's a one-liner.
This patch is not purely cosmetic, because I split apart the priorities
of DO_COLLATION and DO_TRANSFORM, as well as those of DO_ACCESS_METHOD
and DO_OPERATOR, which look to me to have been merged out of expediency
rather than because it was a good idea. Shell types continue to be
sorted interchangeably with full types, and opclasses interchangeably
with opfamilies.
M src/bin/pg_dump/pg_dump_sort.c
Fix function prototypes in dependency.h.
commit : f315205f3fafd6f6c7c479f480289fcf45700310
author : Thomas Munro <[email protected]>
date : Tue, 12 Jan 2021 10:55:35 +1300
committer: Thomas Munro <[email protected]>
date : Tue, 12 Jan 2021 10:55:35 +1300
Commit 257836a7 accidentally deleted a couple of
redundant-but-conventional "extern" keywords on function prototypes.
Put them back.
Reported-by: Alvaro Herrera <[email protected]>
M src/include/catalog/dependency.h
Rethink SQLSTATE code for ERRCODE_IDLE_SESSION_TIMEOUT.
commit : 4edf96846a02693e4416478b3302e5133d2e8e01
author : Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 14:53:42 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 14:53:42 -0500
Move it to class 57 (Operator Intervention), which seems like a
better choice given that from the client's standpoint it behaves
a heck of a lot like, e.g., ERRCODE_ADMIN_SHUTDOWN.
In a green field I'd put ERRCODE_IDLE_IN_TRANSACTION_SESSION_TIMEOUT
here as well. But that's been around for a few years, so it's
probably too late to change its SQLSTATE code.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/errcodes.txt
Try next host after a "cannot connect now" failure.
commit : c1d589571c497a952d7fbe40d9828655859d746f
author : Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 14:12:31 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 14:12:31 -0500
If a server returns ERRCODE_CANNOT_CONNECT_NOW, try the next host,
if multiple host names have been provided. This allows dealing
gracefully with standby servers that might not be in hot standby mode
yet.
In the wake of the preceding commit, it might be plausible to retry
many more error cases than we do now, but I (tgl) am hesitant to
move too aggressively on that --- it's not clear it'd be desirable
for cases such as bad-password, for example. But this case seems
safe enough.
Hubert Zhang, reviewed by Takayuki Tsunakawa
Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
M src/interfaces/libpq/fe-connect.c
Uniformly identify the target host in libpq connection failure reports.
commit : 52a10224e3cc1d706ba9800695f97cb163b747d5
author : Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 14:03:39 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 14:03:39 -0500
Prefix "could not connect to host-or-socket-path:" to all connection
failure cases that occur after the socket() call, and remove the
ad-hoc server identity data that was appended to a few of these
messages. This should produce much more intelligible error reports
in multiple-target-host situations, especially for error cases that
are off the beaten track to any degree (because none of those provided
any server identity info).
As an example of the change, formerly a connection attempt with a bad
port number such as "psql -p 12345 -h localhost,/tmp" might produce
psql: error: could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 12345?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 12345?
could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.12345"?
Now it looks like
psql: error: could not connect to host "localhost" (::1), port 12345: Connection refused
Is the server running on that host and accepting TCP/IP connections?
could not connect to host "localhost" (127.0.0.1), port 12345: Connection refused
Is the server running on that host and accepting TCP/IP connections?
could not connect to socket "/tmp/.s.PGSQL.12345": No such file or directory
Is the server running locally and accepting connections on that socket?
This requires adjusting a couple of regression tests to allow for
variation in the contents of a connection failure message.
Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
M src/bin/pg_dump/t/002_pg_dump.pl
M src/interfaces/ecpg/test/expected/connect-test5.stderr
M src/interfaces/libpq/fe-connect.c
Allow pg_regress.c wrappers to postprocess test result files.
commit : 800d93f314b0f7c10193e48b259f87800cb85d84
author : Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 13:43:19 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 13:43:19 -0500
Add an optional callback to regression_main() that, if provided,
is invoked on each test output file before we try to compare it
to the expected-result file.
The main and isolation test programs don't need this (yet).
In pg_regress_ecpg, add a filter that eliminates target-host
details from "could not connect" error reports. This filter
doesn't do anything as of this commit, but it will be needed
by the next one.
In the long run we might want to provide some more general,
perhaps pattern-based, filtering mechanism for test output.
For now, this will solve the immediate problem.
Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
M src/interfaces/ecpg/test/pg_regress_ecpg.c
M src/test/isolation/isolation_main.c
M src/test/regress/pg_regress.c
M src/test/regress/pg_regress.h
M src/test/regress/pg_regress_main.c
In libpq, always append new error messages to conn->errorMessage.
commit : ffa2e4670123124b92f037d335a1e844c3782d3f
author : Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 13:12:09 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Jan 2021 13:12:09 -0500
Previously, we had an undisciplined mish-mash of printfPQExpBuffer and
appendPQExpBuffer calls to report errors within libpq. This commit
establishes a uniform rule that appendPQExpBuffer[Str] should be used.
conn->errorMessage is reset only at the start of an application request,
and then accumulates messages till we're done. We can remove no less
than three different ad-hoc mechanisms that were used to get the effect
of concatenation of error messages within a sequence of operations.
Although this makes things quite a bit cleaner conceptually, the main
reason to do it is to make the world safer for the multiple-target-host
feature that was added awhile back. Previously, there were many cases
in which an error occurring during an individual host connection attempt
would wipe out the record of what had happened during previous attempts.
(The reporting is still inadequate, in that it can be hard to tell which
host got the failure, but that seems like a matter for a separate commit.)
Currently, lo_import and lo_export contain exceptions to the "never
use printfPQExpBuffer" rule. If we changed them, we'd risk reporting
an incidental lo_close failure before the actual read or write
failure, which would be confusing, not least because lo_close happened
after the main failure. We could improve this by inventing an
internal version of lo_close that doesn't reset the errorMessage; but
we'd also need a version of PQfn() that does that, and it didn't quite
seem worth the trouble for now.
Discussion: https://postgr.es/m/BN6PR05MB3492948E4FD76C156E747E8BC9160@BN6PR05MB3492.namprd05.prod.outlook.com
M src/interfaces/libpq/fe-auth-scram.c
M src/interfaces/libpq/fe-auth.c
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-gssapi-common.c
M src/interfaces/libpq/fe-lobj.c
M src/interfaces/libpq/fe-misc.c
M src/interfaces/libpq/fe-protocol2.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/fe-secure-common.c
M src/interfaces/libpq/fe-secure-gssapi.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/fe-secure.c
M src/interfaces/libpq/libpq-int.h
Use vectored I/O to fill new WAL segments.
commit : ce6a71fa5300cf00adf32c9daee302c523609709
author : Thomas Munro <[email protected]>
date : Mon, 11 Jan 2021 14:41:39 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 11 Jan 2021 14:41:39 +1300
Instead of making many block-sized write() calls to fill a new WAL file
with zeroes, make a smaller number of pwritev() calls (or various
emulations). The actual number depends on the OS's IOV_MAX, which
PG_IOV_MAX currently caps at 32. That means we'll write 256kB per call
on typical systems. We may want to tune the number later with more
experience.
Reviewed-by: Tom Lane <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJA%2Bu-220VONeoREBXJ9P3S94Y7J%2BkqCnTYmahvZJwM%3Dg%40mail.gmail.com
M src/backend/access/transam/xlog.c
Provide pg_preadv() and pg_pwritev().
commit : 13a021f3e8c99915b3cc0cb2021a948d9c71ff32
author : Thomas Munro <[email protected]>
date : Mon, 11 Jan 2021 14:37:13 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 11 Jan 2021 14:37:13 +1300
Provide synchronous vectored file I/O routines. These map to preadv()
and pwritev(), with fallback implementations for systems that don't have
them. Also provide a wrapper pg_pwritev_with_retry() that automatically
retries on short writes.
Reviewed-by: Tom Lane <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJA%2Bu-220VONeoREBXJ9P3S94Y7J%2BkqCnTYmahvZJwM%3Dg%40mail.gmail.com
M configure
M configure.ac
M src/include/pg_config.h.in
M src/include/port.h
A src/include/port/pg_iovec.h
M src/port/Makefile
M src/port/pread.c
M src/port/pwrite.c
M src/tools/msvc/Solution.pm
doc: expand description of how non-SELECT queries are processed
commit : 01334c92fa09dc496a444a4f206854ef37247258
author : Bruce Momjian <[email protected]>
date : Sat, 9 Jan 2021 12:11:16 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 9 Jan 2021 12:11:16 -0500
The previous description of how the executor processes non-SELECT
queries was very dense, causing lack of clarity. This expanded text
spells it out more simply.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/arch-dev.sgml
Fix the test for decoding of two-phase transactions.
commit : e33d004900f76c35759293fdedd4861b198fbf5b
author : Amit Kapila <[email protected]>
date : Sat, 9 Jan 2021 15:45:38 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 9 Jan 2021 15:45:38 +0530
Commit 5a3574d7b3 added the test for decoding of two-phase transactions
during the build of a consistent snapshot. The test forgot to skip empty
xacts which can lead to decoding of extra empty transactions due to
background activity by autovacuum.
Per report by buildfarm.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/expected/twophase_snapshot.out
M contrib/test_decoding/specs/twophase_snapshot.spec
Fix plpgsql tests for debug_invalidate_system_caches_always.
commit : 39d4a153105f0693d93f593a23e5144e2bd031ef
author : Tom Lane <[email protected]>
date : Fri, 8 Jan 2021 18:12:07 -0500
committer: Tom Lane <[email protected]>
date : Fri, 8 Jan 2021 18:12:07 -0500
Commit c9d529848 resulted in having a couple more places where
the error context stack for a failure varies depending on
debug_invalidate_system_caches_always (nee CLOBBER_CACHE_ALWAYS).
This is not very surprising, since we have to re-parse cached
plans if the plan cache is clobbered. Stabilize the expected
test output by hiding the context stack in these places,
as we've done elsewhere in this test script.
(Another idea worth considering, now that we have
debug_invalidate_system_caches_always, is to force it to zero for
these test cases. That seems like it'd risk reducing the coverage
of cache-clobber testing, which might or might not be worth being
able to verify that we get the expected error output in normal
cases. For the moment I just stuck with the existing technique.)
In passing, update comments that referred to CLOBBER_CACHE_ALWAYS.
Per buildfarm member hyrax.
M src/pl/plpgsql/src/expected/plpgsql_cache.out
M src/pl/plpgsql/src/expected/plpgsql_cache_1.out
M src/pl/plpgsql/src/expected/plpgsql_record.out
M src/pl/plpgsql/src/sql/plpgsql_cache.sql
M src/pl/plpgsql/src/sql/plpgsql_record.sql
Fix ancient bug in parsing of BRE-mode regular expressions.
commit : afcc8772edcec687d87b6f762ca6113229af7291
author : Tom Lane <[email protected]>
date : Fri, 8 Jan 2021 12:16:00 -0500
committer: Tom Lane <[email protected]>
date : Fri, 8 Jan 2021 12:16:00 -0500
brenext(), when parsing a '*' quantifier, forgot to return any "value"
for the token; per the equivalent case in next(), it should return
value 1 to indicate that greedy rather than non-greedy behavior is
wanted. The result is that the compiled regexp could behave like 'x*?'
rather than the intended 'x*', if we were unlucky enough to have
a zero in v->nextvalue at this point. That seems to happen with some
reliability if we have '.*' at the beginning of a BRE-mode regexp,
although that depends on the initial contents of a stack-allocated
struct, so it's not guaranteed to fail.
Found by Alexander Lakhin using valgrind testing. This bug seems
to be aboriginal in Spencer's code, so back-patch all the way.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regc_lex.c
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Fix and simplify some code related to cryptohashes
commit : 15b824da97afb45f47e51b6b5b7e5eca09e5d03d
author : Michael Paquier <[email protected]>
date : Fri, 8 Jan 2021 10:37:03 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 8 Jan 2021 10:37:03 +0900
This commit addresses two issues:
- In pgcrypto, MD5 computation called pg_cryptohash_{init,update,final}
without checking for the result status.
- Simplify pg_checksum_raw_context to use only one variable for all the
SHA2 options available in checksum manifests.
Reported-by: Heikki Linnakangas
Discussion: https://postgr.es/m/[email protected]
M contrib/pgcrypto/internal.c
M src/common/checksum_helper.c
M src/include/common/checksum_helper.h
Adjust createdb TAP tests to work on recent OpenBSD.
commit : 9ffe2278372d7549547176c23564a5b3404d072e
author : Tom Lane <[email protected]>
date : Thu, 7 Jan 2021 20:36:09 -0500
committer: Tom Lane <[email protected]>
date : Thu, 7 Jan 2021 20:36:09 -0500
We found last February that the error-case tests added by commit
008cf0409 failed on OpenBSD, because that platform doesn't really
check locale names. At the time it seemed that that was only an issue
for LC_CTYPE, but testing on a more recent version of OpenBSD shows
that it's now equally lax about LC_COLLATE.
Rather than dropping the LC_COLLATE test too, put back LC_CTYPE
(reverting c4b0edb07), and adjust these tests to accept the different
error message that we get if setlocale() doesn't reject a bogus locale
name. The point of these tests is not really what the backend does
with the locale name, but to show that createdb quotes funny locale
names safely; so we're not losing test reliability this way.
Back-patch as appropriate.
Discussion: https://postgr.es/m/[email protected]
M src/bin/scripts/t/020_createdb.pl
Further second thoughts about idle_session_timeout patch.
commit : b8d0cda53377515ac61357ec4a60e85ca873f486
author : Tom Lane <[email protected]>
date : Thu, 7 Jan 2021 11:45:08 -0500
committer: Tom Lane <[email protected]>
date : Thu, 7 Jan 2021 11:45:08 -0500
On reflection, the order of operations in PostgresMain() is wrong.
These timeouts ought to be shut down before, not after, we do the
post-command-read CHECK_FOR_INTERRUPTS, to guarantee that any
timeout error will be detected there rather than at some ill-defined
later point (possibly after having wasted a lot of work).
This is really an error in the original idle_in_transaction_timeout
patch, so back-patch to 9.6 where that was introduced.
M src/backend/tcop/postgres.c
Minor fixes in COPY progress docs
commit : ebb5457cfa514972847a2d03b5b4fd46f69bdc9b
author : Tomas Vondra <[email protected]>
date : Thu, 7 Jan 2021 17:31:58 +0100
committer: Tomas Vondra <[email protected]>
date : Thu, 7 Jan 2021 17:31:58 +0100
Author: Justin Pryzby
Discussion: https://postgr.es/m/CAFp7Qwr6_FmRM6pCO0x_a0mymOfX_Gg+FEKet4XaTGSW=LitKQ@mail.gmail.com
M doc/src/sgml/monitoring.sgml
Add GUC to log long wait times on recovery conflicts.
commit : 0650ff23038bc3eb8d8fd851744db837d921e285
author : Fujii Masao <[email protected]>
date : Fri, 8 Jan 2021 00:47:03 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 8 Jan 2021 00:47:03 +0900
This commit adds GUC log_recovery_conflict_waits that controls whether
a log message is produced when the startup process is waiting longer than
deadlock_timeout for recovery conflicts. This is useful in determining
if recovery conflicts prevent the recovery from applying WAL.
Note that currently a log message is produced only when recovery conflict
has not been resolved yet even after deadlock_timeout passes, i.e.,
only when the startup process is still waiting for recovery conflict
even after deadlock_timeout.
Author: Bertrand Drouvot, Masahiko Sawada
Reviewed-by: Alvaro Herrera, Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M doc/src/sgml/high-availability.sgml
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/storage/standby.h
Fix bogus link in test comments.
commit : f7a1a805cb178653ea2a6c8991ad73b035af953e
author : Tom Lane <[email protected]>
date : Wed, 6 Jan 2021 22:09:00 -0500
committer: Tom Lane <[email protected]>
date : Wed, 6 Jan 2021 22:09:00 -0500
I apparently copied-and-pasted the wrong link in commit ca8217c10.
Point it where it was meant to go.
M src/test/modules/test_regex/expected/test_regex.out
M src/test/modules/test_regex/sql/test_regex.sql
Improve commentary in timeout.c.
commit : 9486e7b666fd113f043d5f091fd42bc1ef72acd8
author : Tom Lane <[email protected]>
date : Wed, 6 Jan 2021 22:02:35 -0500
committer: Tom Lane <[email protected]>
date : Wed, 6 Jan 2021 22:02:35 -0500
On re-reading I realized that I'd missed one race condition in the new
timeout code. It's safe, but add a comment explaining it.
Discussion: https://postgr.es/m/CA+hUKG+o6pbuHBJSGnud=TadsuXySWA7CCcPgCt2QE9F6_4iHQ@mail.gmail.com
M src/backend/utils/misc/timeout.c
Fix allocation logic of cryptohash context data with OpenSSL
commit : 55fe26a4b580b17d721c5accb842cc6a08295273
author : Michael Paquier <[email protected]>
date : Thu, 7 Jan 2021 10:21:02 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 7 Jan 2021 10:21:02 +0900
The allocation of the cryptohash context data when building with OpenSSL
was happening in the memory context of the caller of
pg_cryptohash_create(), which could lead to issues with resowner cleanup
if cascading resources are cleaned up on an error. Like other
facilities using resowners, move the base allocation to TopMemoryContext
to ensure a correct cleanup on failure.
The resulting code gets simpler with this commit as the context data is
now hold by a unique opaque pointer, so as there is only one single
allocation done in TopMemoryContext.
After discussion, also change the cryptohash subroutines to return an
error if the caller provides NULL for the context data to ease error
detection on OOM.
Author: Heikki Linnakangas
Discussion: https://postgr.es/m/[email protected]
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
M src/include/common/cryptohash.h
M src/tools/pgindent/typedefs.list
Add idle_session_timeout.
commit : 9877374bef76ef03923f6aa8b955f2dbcbe6c2c7
author : Tom Lane <[email protected]>
date : Wed, 6 Jan 2021 18:28:42 -0500
committer: Tom Lane <[email protected]>
date : Wed, 6 Jan 2021 18:28:42 -0500
This GUC variable works much like idle_in_transaction_session_timeout,
in that it kills sessions that have waited too long for a new client
query. But it applies when we're not in a transaction, rather than
when we are.
Li Japin, reviewed by David Johnston and Hayato Kuroda, some
fixes by me
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M src/backend/storage/lmgr/proc.c
M src/backend/tcop/postgres.c
M src/backend/utils/errcodes.txt
M src/backend/utils/init/globals.c
M src/backend/utils/init/postinit.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/miscadmin.h
M src/include/storage/proc.h
M src/include/utils/timeout.h
Improve timeout.c's handling of repeated timeout set/cancel.
commit : 09cf1d52267644cdbdb734294012cf1228745aaa
author : Tom Lane <[email protected]>
date : Wed, 6 Jan 2021 18:23:03 -0500
committer: Tom Lane <[email protected]>
date : Wed, 6 Jan 2021 18:23:03 -0500
A very common usage pattern is that we set a timeout that we don't
expect to reach, cancel it after a little bit, and later repeat.
With the original implementation of timeout.c, this results in one
setitimer() call per timeout set or cancel. We can do a lot better
by being lazy about changing the timeout interrupt request, namely:
(1) never cancel the outstanding interrupt, even when we have no
active timeout events;
(2) if we need to set an interrupt, but there already is one pending
at or before the required time, leave it alone. When the interrupt
happens, the signal handler will reschedule it at whatever time is
then needed.
For example, with a one-second setting for statement_timeout, this
method results in having to interact with the kernel only a little
more than once a second, no matter how many statements we execute
in between. The mainline code might never call setitimer() at all
after the first time, while each time the signal handler fires,
it sees that the then-pending request is most of a second away,
and that's when it sets the next interrupt request for. Each
mainline timeout-set request after that will observe that the time
it wants is past the pending interrupt request time, and do nothing.
This also works pretty well for cases where a few different timeout
lengths are in use, as long as none of them are very short. But
that describes our usage well.
Idea and original patch by Thomas Munro; I fixed a race condition
and improved the comments.
Discussion: https://postgr.es/m/CA+hUKG+o6pbuHBJSGnud=TadsuXySWA7CCcPgCt2QE9F6_4iHQ@mail.gmail.com
M src/backend/utils/misc/timeout.c
Report progress of COPY commands
commit : 8a4f618e7ae3cb11b0b37d0f06f05c8ff905833f
author : Tomas Vondra <[email protected]>
date : Wed, 6 Jan 2021 21:46:26 +0100
committer: Tomas Vondra <[email protected]>
date : Wed, 6 Jan 2021 21:46:26 +0100
This commit introduces a view pg_stat_progress_copy, reporting progress
of COPY commands. This allows rough estimates how far a running COPY
progressed, with the caveat that the total number of bytes may not be
available in some cases (e.g. when the input comes from the client).
Author: Josef Šimánek
Reviewed-by: Fujii Masao, Bharath Rupireddy, Vignesh C, Matthias van de Meent
Discussion: https://postgr.es/m/CAFp7QwqMGEi4OyyaLEK9DR0+E+oK3UtA4bEjDVCa4bNkwUY2PQ@mail.gmail.com
Discussion: https://postgr.es/m/CAFp7Qwr6_FmRM6pCO0x_a0mymOfX_Gg+FEKet4XaTGSW=LitKQ@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/commands/copyfrom.c
M src/backend/commands/copyfromparse.c
M src/backend/commands/copyto.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/commands/copyfrom_internal.h
M src/include/commands/progress.h
M src/include/pgstat.h
M src/test/regress/expected/rules.out
Add a test module for the regular expression package.
commit : ca8217c10138fa3ffe1e7d1def2484fd0eb78226
author : Tom Lane <[email protected]>
date : Wed, 6 Jan 2021 10:51:14 -0500
committer: Tom Lane <[email protected]>
date : Wed, 6 Jan 2021 10:51:14 -0500
This module provides a function test_regex() that is functionally
rather like regexp_matches(), but with additional debugging-oriented
options and additional output. The debug options are somewhat obscure;
they are chosen to match the API of the test harness that Henry Spencer
wrote way-back-when for use in Tcl. With this, we can import all the
test cases that Spencer wrote originally, even for regex functionality
that we don't currently expose in Postgres. This seems necessary
because we can no longer rely on Tcl to act as upstream and verify
any fixes or improvements that we make.
In addition to Spencer's tests, I added a few for lookbehind
constraints (which we added in 2015, and Tcl still hasn't absorbed)
that are modeled on his tests for lookahead constraints. After looking
at code coverage reports, I also threw in a couple of tests to more
fully exercise our "high colormap" logic.
According to my testing, this brings the check-world coverage
for src/backend/regex/ from 71.1% to 86.7% of lines.
(coverage.postgresql.org shows a slightly different number,
which I think is because it measures a non-assert build.)
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/Makefile
A src/test/modules/test_regex/.gitignore
A src/test/modules/test_regex/Makefile
A src/test/modules/test_regex/README
A src/test/modules/test_regex/expected/test_regex.out
A src/test/modules/test_regex/expected/test_regex_utf8.out
A src/test/modules/test_regex/expected/test_regex_utf8_1.out
A src/test/modules/test_regex/sql/test_regex.sql
A src/test/modules/test_regex/sql/test_regex_utf8.sql
A src/test/modules/test_regex/test_regex–1.0.sql
A src/test/modules/test_regex/test_regex.c
A src/test/modules/test_regex/test_regex.control
Replace CLOBBER_CACHE_ALWAYS with run-time GUC
commit : 4656e3d66893f286767285cf74dabb3877068e49
author : Peter Eisentraut <[email protected]>
date : Wed, 6 Jan 2021 10:15:19 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 6 Jan 2021 10:15:19 +0100
Forced cache invalidation (CLOBBER_CACHE_ALWAYS) has been impractical
to use for testing in PostgreSQL because it's so slow and because it's
toggled on/off only at build time. It is helpful when hunting bugs in
any code that uses the sycache/relcache because causes cache
invalidations to be injected whenever it would be possible for an
invalidation to occur, whether or not one was really pending.
Address this by providing run-time control over cache clobber
behaviour using the new debug_invalidate_system_caches_always GUC.
Support is not compiled in at all unless assertions are enabled or
CLOBBER_CACHE_ENABLED is explicitly defined at compile time. It
defaults to 0 if compiled in, so it has negligible effect on assert
build performance by default.
When support is compiled in, test code can now set
debug_invalidate_system_caches_always=1 locally to a backend to test
specific queries, functions, extensions, etc. Or tests can toggle it
globally for a specific test case while retaining normal performance
during test setup and teardown.
For backwards compatibility with existing test harnesses and scripts,
debug_invalidate_system_caches_always defaults to 1 if
CLOBBER_CACHE_ALWAYS is defined, and to 3 if CLOBBER_CACHE_RECURSIVE
is defined.
CLOBBER_CACHE_ENABLED is now visible in pg_config_manual.h, as is the
related RECOVER_RELATION_BUILD_MEMORY setting for the relcache.
Author: Craig Ringer <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CAMsr+YF=+ctXBZj3ywmvKNUjWpxmuTuUKuv-rgbHGX5i5pLstQ@mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/regress.sgml
M src/backend/utils/adt/lockfuncs.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/misc/guc.c
M src/include/pg_config_manual.h
M src/include/utils/inval.h
Detect the deadlocks between backends and the startup process.
commit : 8900b5a9d59a645b3485f5b046c4c7871b2c4026
author : Fujii Masao <[email protected]>
date : Wed, 6 Jan 2021 12:39:18 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 6 Jan 2021 12:39:18 +0900
The deadlocks that the recovery conflict on lock is involved in can
happen between hot-standby backends and the startup process.
If a backend takes an access exclusive lock on the table and which
finally triggers the deadlock, that deadlock can be detected
as expected. On the other hand, previously, if the startup process
took an access exclusive lock and which finally triggered the deadlock,
that deadlock could not be detected and could remain even after
deadlock_timeout passed. This is a bug.
The cause of this bug was that the code for handling the recovery
conflict on lock didn't take care of deadlock case at all. It assumed
that deadlocks involving the startup process and backends were able
to be detected by the deadlock detector invoked within backends.
But this assumption was incorrect. The startup process also should
have invoked the deadlock detector if necessary.
To fix this bug, this commit makes the startup process invoke
the deadlock detector if deadlock_timeout is reached while handling
the recovery conflict on lock. Specifically, in that case, the startup
process requests all the backends holding the conflicting locks to
check themselves for deadlocks.
Back-patch to v9.6. v9.5 has also this bug, but per discussion we decided
not to back-patch the fix to v9.5. Because v9.5 doesn't have some
infrastructure codes (e.g., 37c54863cf) that this bug fix patch depends on.
We can apply those codes for the back-patch, but since the next minor
version release is the final one for v9.5, it's risky to do that. If we
unexpectedly introduce new bug to v9.5 by the back-patch, there is no
chance to fix that. We determined that the back-patch to v9.5 would give
more risk than gain.
Author: Fujii Masao
Reviewed-by: Bertrand Drouvot, Masahiko Sawada, Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/proc.c
M src/backend/tcop/postgres.c
M src/include/storage/procarray.h
Fix typos in decode.c and logical.c.
commit : e02e840ff7787d4798fca9f6a5b8b3657f45cf0c
author : Amit Kapila <[email protected]>
date : Wed, 6 Jan 2021 08:56:19 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 6 Jan 2021 08:56:19 +0530
Per report by Ajin Cherian in email:
https://postgr.es/m/CAFPTHDYnRKDvzgDxoMn_CKqXA-D0MtrbyJvfvjBsO4G=UHDXkg@mail.gmail.com
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/logical.c
doc: Fix description about default behavior of recovery_target_timeline.
commit : 25dde5835772a58356383bf1112c6c2a1a37df0e
author : Fujii Masao <[email protected]>
date : Wed, 6 Jan 2021 11:58:23 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 6 Jan 2021 11:58:23 +0900
The default value of recovery_target_timeline was changed in v12,
but the description about the default behavior of that was not updated.
Back-patch to v12 where the default behavior of recovery_target_timeline
was changed.
Author: Benoit Lobréau
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAPE8EZ7c3aruEmM24GYkj8y8WmHKD1m9TtPtgCF0nQ3zw4LCkQ@mail.gmail.com
M doc/src/sgml/backup.sgml
Promote --data-checksums to the common set of options in initdb --help
commit : bc08f7971c03550ede43af43e73d4a47a7935c69
author : Michael Paquier <[email protected]>
date : Wed, 6 Jan 2021 10:52:26 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 6 Jan 2021 10:52:26 +0900
This was previously part of the section dedicated to less common
options, but it is an option commonly used these days.
Author: Michael Banck
Reviewed-by: Stephen Frost, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/bin/initdb/initdb.c
Revert unstable test cases from commit 7d80441d2.
commit : 14d49f483d4c8a5a356e25d5e5ff5726ca43abff
author : Tom Lane <[email protected]>
date : Tue, 5 Jan 2021 19:03:56 -0500
committer: Tom Lane <[email protected]>
date : Tue, 5 Jan 2021 19:03:56 -0500
I momentarily forgot that the "owner" column wouldn't be stable
in the buildfarm. Oh well, these tests weren't very valuable
anyway.
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
Allow psql's \dt and \di to show TOAST tables and their indexes.
commit : 7d80441d2c8de5cd5d593e302bd14e8b19ee92d4
author : Tom Lane <[email protected]>
date : Tue, 5 Jan 2021 18:41:50 -0500
committer: Tom Lane <[email protected]>
date : Tue, 5 Jan 2021 18:41:50 -0500
Formerly, TOAST objects were unconditionally suppressed, but since
\d is able to print them it's not very clear why these variants
should not. Instead, use the same rules as for system catalogs:
they can be seen if you write the 'S' modifier or a table name
pattern. (In practice, since hardly anybody would keep pg_toast
in their search_path, it's really down to whether you use a pattern
that can match pg_toast.*.)
No docs change seems necessary because the docs already say that
this happens for "system objects"; we're just classifying TOAST
tables as being that.
Justin Pryzby, reviewed by Laurenz Albe
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/describe.c
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
Introduce a new GUC_REPORT setting "in_hot_standby".
commit : bf8a662c9afad6fd07b42cdc5e71416c51f75d31
author : Tom Lane <[email protected]>
date : Tue, 5 Jan 2021 16:18:01 -0500
committer: Tom Lane <[email protected]>
date : Tue, 5 Jan 2021 16:18:01 -0500
Aside from being queriable via SHOW, this value is sent to the client
immediately at session startup, and again later on if the server gets
promoted to primary during the session. The immediate report will be
used in an upcoming patch to avoid an extra round trip when trying to
connect to a primary server.
Haribabu Kommi, Greg Nancarrow, Tom Lane; reviewed at various times
by Laurenz Albe, Takayuki Tsunakawa, Peter Smith.
Discussion: https://postgr.es/m/CAF3+xM+8-ztOkaV9gHiJ3wfgENTq97QcjXQt+rbFQ6F7oNzt9A@mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/protocol.sgml
M src/backend/utils/misc/check_guc
M src/backend/utils/misc/guc.c
doc: improve NLS instruction wording
commit : 47b2ed1d06c1218eee8869c8eca8bf002b9d43a6
author : Bruce Momjian <[email protected]>
date : Tue, 5 Jan 2021 14:26:37 -0500
committer: Bruce Momjian <[email protected]>
date : Tue, 5 Jan 2021 14:26:37 -0500
Reported-by: "Tang, Haiying"
Discussion: https://postgr.es/m/bbbccf7a3c2d436e85d45869d612fd6b@G08CNEXMBPEKD05.g08.fujitsu.local
Author: "Tang, Haiying"
Backpatch-through: 9.5
M doc/src/sgml/nls.sgml
Add an explicit cast to double when using fabs().
commit : fead67c24ada8c6a4b661dec6f159dca1447e3d8
author : Dean Rasheed <[email protected]>
date : Tue, 5 Jan 2021 11:52:42 +0000
committer: Dean Rasheed <[email protected]>
date : Tue, 5 Jan 2021 11:52:42 +0000
Commit bc43b7c2c0 used fabs() directly on an int variable, which
apparently requires an explicit cast on some platforms.
Per buildfarm.
M src/backend/utils/adt/numeric.c
Fix numeric_power() when the exponent is INT_MIN.
commit : bc43b7c2c06c32264efe79d0b86abd41236f1d5b
author : Dean Rasheed <[email protected]>
date : Tue, 5 Jan 2021 11:15:28 +0000
committer: Dean Rasheed <[email protected]>
date : Tue, 5 Jan 2021 11:15:28 +0000
In power_var_int(), the computation of the number of significant
digits to use in the computation used log(Abs(exp)), which isn't safe
because Abs(exp) returns INT_MIN when exp is INT_MIN. Use fabs()
instead of Abs(), so that the exponent is cast to a double before the
absolute value is taken.
Back-patch to 9.6, where this was introduced (by 7d9a4737c2).
Discussion: https://postgr.es/m/CAEZATCVd6pMkz=BrZEgBKyqqJrt2xghr=fNc8+Z=5xC6cgWrWA@mail.gmail.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Test decoding of two-phase transactions during the build of a consistent snapshot.
commit : 5a3574d7b33480de51692962e6fd25d00c05718a
author : Amit Kapila <[email protected]>
date : Tue, 5 Jan 2021 15:05:22 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 5 Jan 2021 15:05:22 +0530
Commit a271a1b50e added the capability to allow decoding at prepare time.
This adds an isolation testcase to test that decoding happens at commit
time when the consistent snapshot state is reached after prepare but
before commit prepared.
Author: Ajin Cherian
Reviewed-by: Amit Kapila
Discussion:
https://postgr.es/m/[email protected]
https://postgr.es/m/CAMGcDxeqEpWj3fTXwqhSwBdXd2RS9jzwWscO-XbeCfso6ts3+Q@mail.gmail.com
M contrib/test_decoding/Makefile
A contrib/test_decoding/expected/twophase_snapshot.out
A contrib/test_decoding/specs/twophase_snapshot.spec
doc: Document how to run regression tests with custom server settings
commit : 854434c50a35e7cb2ae22588c80ebf36e889197a
author : Peter Eisentraut <[email protected]>
date : Tue, 5 Jan 2021 08:17:40 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 5 Jan 2021 08:17:40 +0100
Author: Craig Ringer <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CAMsr+YF=+ctXBZj3ywmvKNUjWpxmuTuUKuv-rgbHGX5i5pLstQ@mail.gmail.com
M doc/src/sgml/regress.sgml
Standardize one aspect of rmgr desc output.
commit : 83e3239ee77089f76fad022eeef91c83cdf9181d
author : Peter Geoghegan <[email protected]>
date : Mon, 4 Jan 2021 19:46:11 -0800
committer: Peter Geoghegan <[email protected]>
date : Mon, 4 Jan 2021 19:46:11 -0800
Bring heap and hash rmgr desc output in line with nbtree and GiST desc
output by using the name latestRemovedXid for all fields that output the
contents of the latestRemovedXid field from the WAL record's C struct
(stop using local variants).
This seems like a clear improvement because latestRemovedXid is a symbol
name that already appears across many different source files, and so is
probably much more recognizable.
Discussion: https://postgr.es/m/CAH2-Wzkt_Rs4VqPSCk87nyjPAAEmWL8STU9zgET_83EF5YfrLw@mail.gmail.com
M src/backend/access/rmgrdesc/hashdesc.c
M src/backend/access/rmgrdesc/heapdesc.c
Fix typo in origin.c.
commit : cd357c76296e987298e4005b8053adda9e3fb417
author : Amit Kapila <[email protected]>
date : Tue, 5 Jan 2021 08:05:08 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 5 Jan 2021 08:05:08 +0530
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+PsReyuvww_Fn1NN_Vsv0wBP1bnzuhzRFr_2=y1nNZrG7w@mail.gmail.com
M src/backend/replication/logical/origin.c
Fix typo in reorderbuffer.c.
commit : 9da2224ea2bb9801afc29bff6a325bf796868bdc
author : Amit Kapila <[email protected]>
date : Tue, 5 Jan 2021 07:56:40 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 5 Jan 2021 07:56:40 +0530
Author: Zhijie Hou
Reviewed-by: Sawada Masahiko
Discussion: https://postgr.es/m/ba88bb58aaf14284abca16aec04bf279@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/replication/logical/reorderbuffer.c
Replace remaining uses of "whitelist".
commit : 034510c820cd75e0410332d92b4967ef9b844936
author : Thomas Munro <[email protected]>
date : Tue, 5 Jan 2021 14:00:16 +1300
committer: Thomas Munro <[email protected]>
date : Tue, 5 Jan 2021 14:00:16 +1300
Instead describe the action that the list effects, or just use "list"
where the meaning is obvious from context.
Author: Dagfinn Ilmari Mannsåker <[email protected]>
Discussion: https://postgr.es/m/20200615182235.x7lch5n6kcjq4aue%40alap3.anarazel.de
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/shippable.c
M src/backend/access/hash/hashvalidate.c
M src/backend/utils/adt/lockfuncs.c
M src/tools/pginclude/README
pgindent: whitelist/blacklist -> additional/excluded.
commit : fe05b6b620066aec313c43b6b4d6c169d0a346f7
author : Thomas Munro <[email protected]>
date : Tue, 5 Jan 2021 13:27:06 +1300
committer: Thomas Munro <[email protected]>
date : Tue, 5 Jan 2021 13:27:06 +1300
Author: Dagfinn Ilmari Mannsåker <[email protected]>
Discussion: https://postgr.es/m/20200615182235.x7lch5n6kcjq4aue%40alap3.anarazel.de
M src/tools/pgindent/pgindent
Rename "enum blacklist" to "uncommitted enums".
commit : c0d4f6d897492727d4812679e6a94d12edbe016f
author : Thomas Munro <[email protected]>
date : Tue, 5 Jan 2021 12:06:15 +1300
committer: Thomas Munro <[email protected]>
date : Tue, 5 Jan 2021 12:06:15 +1300
We agreed to remove this terminology and use something more descriptive.
Discussion: https://postgr.es/m/20200615182235.x7lch5n6kcjq4aue%40alap3.anarazel.de
M src/backend/access/transam/parallel.c
M src/backend/catalog/pg_enum.c
M src/backend/utils/adt/enum.c
M src/include/catalog/pg_enum.h
Fix integer-overflow corner cases in substring() functions.
commit : 4bd3fad80e5c3bd107583dd9d32d4a47c045a3ec
author : Tom Lane <[email protected]>
date : Mon, 4 Jan 2021 18:32:40 -0500
committer: Tom Lane <[email protected]>
date : Mon, 4 Jan 2021 18:32:40 -0500
If the substring start index and length overflow when added together,
substring() misbehaved, either throwing a bogus "negative substring
length" error on a case that should succeed, or failing to complain that
a negative length is negative (and instead returning the whole string,
in most cases). Unsurprisingly, the text, bytea, and bit variants of
the function all had this issue. Rearrange the logic to ensure that
negative lengths are always rejected, and add an overflow check to
handle the other case.
Also install similar guards into detoast_attr_slice() (nee
heap_tuple_untoast_attr_slice()), since it's far from clear that
no other code paths leading to that function could pass it values
that would overflow.
Patch by myself and Pavel Stehule, per bug #16804 from Rafi Shamim.
Back-patch to v11. While these bugs are old, the common/int.h
infrastructure for overflow-detecting arithmetic didn't exist before
commit 4d6ad3125, and it doesn't seem like these misbehaviors are bad
enough to justify developing a standalone fix for the older branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/common/detoast.c
M src/backend/utils/adt/varbit.c
M src/backend/utils/adt/varlena.c
M src/test/regress/expected/bit.out
M src/test/regress/expected/strings.out
M src/test/regress/sql/bit.sql
M src/test/regress/sql/strings.sql
Remove unused function prototypes.
commit : 87c23d36a3bc81e57b813f13c403f74a67ff33a9
author : Thomas Munro <[email protected]>
date : Tue, 5 Jan 2021 11:40:03 +1300
committer: Thomas Munro <[email protected]>
date : Tue, 5 Jan 2021 11:40:03 +1300
Cleanup for commit dee663f7.
Reported-by: Tomas Vondra <[email protected]>
Discussion: https://postgr.es/m/CA+hUKGLJ=84YT+NvhkEEDAuUtVHMfQ9i-N7k_o50JmQ6Rpj_OQ@mail.gmail.com
M src/include/access/clog.h
M src/include/access/commit_ts.h
M src/include/access/multixact.h
M src/include/access/subtrans.h
Rethink the "read/write parameter" mechanism in pl/pgsql.
commit : 1c1cbe279b3c6e8038c8f8079402f069bb4cea4c
author : Tom Lane <[email protected]>
date : Mon, 4 Jan 2021 12:39:27 -0500
committer: Tom Lane <[email protected]>
date : Mon, 4 Jan 2021 12:39:27 -0500
Performance issues with the preceding patch to re-implement array
element assignment within pl/pgsql led me to realize that the read/write
parameter mechanism is misdesigned. Instead of requiring the assignment
source expression to be such that *all* its references to the target
variable could be passed as R/W, we really want to identify *one*
reference to the target variable to be passed as R/W, allowing any other
ones to be passed read/only as they would be by default. As long as the
R/W reference is a direct argument to the top-level (hence last to be
executed) function in the expression, there is no harm in R/O references
being passed to other lower parts of the expression. Nor is there any
use-case for more than one argument of the top-level function being R/W.
Hence, rewrite that logic to identify one single Param that references
the target variable, and make only that Param pass a read/write
reference, not any other Params referencing the target variable.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/arrayfuncs.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
Remove PLPGSQL_DTYPE_ARRAYELEM datum type within pl/pgsql.
commit : 1788828d33516809fa2d842bf6e273d78e21d957
author : Tom Lane <[email protected]>
date : Mon, 4 Jan 2021 12:14:37 -0500
committer: Tom Lane <[email protected]>
date : Mon, 4 Jan 2021 12:14:37 -0500
In the wake of the previous commit, we don't really need this anymore,
since array assignment is primarily handled by the core code.
The only way that that code could still be reached is that a GET
DIAGNOSTICS target variable could be an array element. But that
doesn't seem like a particularly essential feature. I'd added it
in commit 55caaaeba, but just because it was easy not because
anyone had actually asked for it. Hence, revert that patch and
then remove the now-unreachable stuff. (If we really had to,
we could probably reimplement GET DIAGNOSTICS using the new
assignment machinery; but the cost/benefit ratio looks very poor,
and it'd likely be a bit slower.)
Note that PLPGSQL_DTYPE_RECFIELD remains. It's possible that we
could get rid of that too, but maintaining the existing behaviors
for RECORD-type variables seems like it might be difficult. Since
there's not any functional limitation in those code paths as there
was in the ARRAYELEM code, I've not pursued the idea.
Discussion: https://postgr.es/m/[email protected]
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_funcs.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
M src/test/regress/expected/plpgsql.out
M src/test/regress/sql/plpgsql.sql
Re-implement pl/pgsql's expression and assignment parsing.
commit : c9d5298485b78a37923a23f9af9aa0ade06762db
author : Tom Lane <[email protected]>
date : Mon, 4 Jan 2021 11:52:00 -0500
committer: Tom Lane <[email protected]>
date : Mon, 4 Jan 2021 11:52:00 -0500
Invent new RawParseModes that allow the core grammar to handle
pl/pgsql expressions and assignments directly, and thereby get rid
of a lot of hackery in pl/pgsql's parser. This moves a good deal
of knowledge about pl/pgsql into the core code: notably, we have to
invent a CoercionContext that matches pl/pgsql's (rather dubious)
historical behavior for assignment coercions. That's getting away
from the original idea of pl/pgsql as an arm's-length extension of
the core, but really we crossed that bridge a long time ago.
The main advantage of doing this is that we can now use the core
parser to generate FieldStore and/or SubscriptingRef nodes to handle
assignments to pl/pgsql variables that are records or arrays. That
fixes a number of cases that had never been implemented in pl/pgsql
assignment, such as nested records and array slicing, and it allows
pl/pgsql assignment to support the datatype-specific subscripting
behaviors introduced in commit c7aba7c14.
There are cosmetic benefits too: when a syntax error occurs in a
pl/pgsql expression, the error report no longer includes the confusing
"SELECT" keyword that used to get prefixed to the expression text.
Also, there seem to be some small speed gains.
Discussion: https://postgr.es/m/[email protected]
M contrib/hstore/expected/hstore.out
M contrib/hstore/sql/hstore.sql
M doc/src/sgml/plpgsql.sgml
M src/backend/commands/functioncmds.c
M src/backend/executor/spi.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/parser/parse_coerce.c
M src/backend/parser/parse_target.c
M src/backend/parser/parser.c
M src/backend/tcop/utility.c
M src/include/nodes/nodes.h
M src/include/nodes/parsenodes.h
M src/include/nodes/primnodes.h
M src/include/parser/parse_target.h
M src/include/parser/parser.h
M src/interfaces/ecpg/preproc/parse.pl
M src/pl/plpgsql/src/Makefile
A src/pl/plpgsql/src/expected/plpgsql_array.out
M src/pl/plpgsql/src/expected/plpgsql_record.out
M src/pl/plpgsql/src/expected/plpgsql_varprops.out
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
A src/pl/plpgsql/src/sql/plpgsql_array.sql
M src/pl/plpgsql/src/sql/plpgsql_record.sql
M src/test/regress/expected/plpgsql.out
Add the ability for the core grammar to have more than one parse target.
commit : 844fe9f159a948377907a63d0ef3fb16dc51ce50
author : Tom Lane <[email protected]>
date : Mon, 4 Jan 2021 11:03:22 -0500
committer: Tom Lane <[email protected]>
date : Mon, 4 Jan 2021 11:03:22 -0500
This patch essentially allows gram.y to implement a family of related
syntax trees, rather than necessarily always parsing a list of SQL
statements. raw_parser() gains a new argument, enum RawParseMode,
to say what to do. As proof of concept, add a mode that just parses
a TypeName without any other decoration, and use that to greatly
simplify typeStringToTypeName().
In addition, invent a new SPI entry point SPI_prepare_extended() to
allow SPI users (particularly plpgsql) to get at this new functionality.
In hopes of making this the last variant of SPI_prepare(), set up its
additional arguments as a struct rather than direct arguments, and
promise that future additions to the struct can default to zero.
SPI_prepare_cursor() and SPI_prepare_params() can perhaps go away at
some point.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/spi.sgml
M src/backend/commands/tablecmds.c
M src/backend/executor/spi.c
M src/backend/parser/gram.y
M src/backend/parser/parse_coerce.c
M src/backend/parser/parse_type.c
M src/backend/parser/parser.c
M src/backend/tcop/postgres.c
M src/include/executor/spi.h
M src/include/executor/spi_priv.h
M src/include/parser/parser.h
M src/interfaces/ecpg/preproc/parse.pl
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
Simplify some comments in xml.c
commit : b49154b3b7a45523ce4081fdae8d65049342fcec
author : Michael Paquier <[email protected]>
date : Mon, 4 Jan 2021 19:47:58 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 4 Jan 2021 19:47:58 +0900
Author: Justin Pryzby
Discussion: https://postgr.es/m/X/[email protected]
M src/backend/utils/adt/xml.c
Allow decoding at prepare time in ReorderBuffer.
commit : a271a1b50e9bec07e2ef3a05e38e7285113e4ce6
author : Amit Kapila <[email protected]>
date : Mon, 4 Jan 2021 08:34:50 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 4 Jan 2021 08:34:50 +0530
This patch allows PREPARE-time decoding of two-phase transactions (if the
output plugin supports this capability), in which case the transactions
are replayed at PREPARE and then committed later when COMMIT PREPARED
arrives.
Now that we decode the changes before the commit, the concurrent aborts
may cause failures when the output plugin consults catalogs (both system
and user-defined).
We detect such failures with a special sqlerrcode
ERRCODE_TRANSACTION_ROLLBACK introduced by commit 7259736a6e and stop
decoding the remaining changes. Then we rollback the changes when rollback
prepared is encountered.
Author: Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Peter Smith, Sawada Masahiko, Arseny Sher, and Dilip Kumar
Tested-by: Takamichi Osumi
Discussion:
https://postgr.es/m/[email protected]
https://postgr.es/m/CAMGcDxeqEpWj3fTXwqhSwBdXd2RS9jzwWscO-XbeCfso6ts3+Q@mail.gmail.com
M contrib/test_decoding/Makefile
A contrib/test_decoding/expected/twophase.out
A contrib/test_decoding/expected/twophase_stream.out
A contrib/test_decoding/sql/twophase.sql
A contrib/test_decoding/sql/twophase_stream.sql
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/include/replication/reorderbuffer.h
Update copyright for 2021
commit : ca3b37487be333a1d241dab1bbdd17a211a88f43
author : Bruce Momjian <[email protected]>
date : Sat, 2 Jan 2021 13:06:25 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 2 Jan 2021 13:06:25 -0500
Backpatch-through: 9.5
M COPYRIGHT
M config/thread_test.c
M configure
M configure.ac
M contrib/adminpack/adminpack.c
M contrib/amcheck/verify_heapam.c
M contrib/amcheck/verify_nbtree.c
M contrib/auth_delay/auth_delay.c
M contrib/auto_explain/auto_explain.c
M contrib/bloom/blcost.c
M contrib/bloom/blinsert.c
M contrib/bloom/bloom.h
M contrib/bloom/blscan.c
M contrib/bloom/blutils.c
M contrib/bloom/blvacuum.c
M contrib/bloom/blvalidate.c
M contrib/dblink/dblink.c
M contrib/dict_int/dict_int.c
M contrib/dict_xsyn/dict_xsyn.c
M contrib/file_fdw/file_fdw.c
M contrib/fuzzystrmatch/fuzzystrmatch.c
M contrib/hstore/hstore_subs.c
M contrib/intarray/_int_selfuncs.c
M contrib/isn/isn.c
M contrib/isn/isn.h
M contrib/old_snapshot/time_mapping.c
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/fsmfuncs.c
M contrib/pageinspect/ginfuncs.c
M contrib/pageinspect/hashfuncs.c
M contrib/pageinspect/heapfuncs.c
M contrib/pageinspect/pageinspect.h
M contrib/pageinspect/rawpage.c
M contrib/passwordcheck/passwordcheck.c
M contrib/pg_prewarm/autoprewarm.c
M contrib/pg_prewarm/pg_prewarm.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_surgery/heap_surgery.c
M contrib/pg_surgery/pg_surgery–1.0.sql
M contrib/pg_trgm/trgm_regexp.c
M contrib/pg_visibility/pg_visibility.c
M contrib/pgcrypto/imath.c
M contrib/pgstattuple/pgstatapprox.c
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/option.c
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/postgres_fdw.h
M contrib/postgres_fdw/shippable.c
M contrib/sepgsql/database.c
M contrib/sepgsql/dml.c
M contrib/sepgsql/hooks.c
M contrib/sepgsql/label.c
M contrib/sepgsql/launcher
M contrib/sepgsql/proc.c
M contrib/sepgsql/relation.c
M contrib/sepgsql/schema.c
M contrib/sepgsql/selinux.c
M contrib/sepgsql/sepgsql.h
M contrib/sepgsql/uavc.c
M contrib/tablefunc/tablefunc.c
M contrib/tablefunc/tablefunc.h
M contrib/tcn/tcn.c
M contrib/test_decoding/test_decoding.c
M contrib/tsm_system_rows/tsm_system_rows.c
M contrib/tsm_system_time/tsm_system_time.c
M contrib/unaccent/unaccent.c
M contrib/uuid-ossp/uuid-ossp.c
M contrib/vacuumlo/vacuumlo.c
M doc/src/sgml/generate-errcodes-table.pl
M doc/src/sgml/generate-keywords-table.pl
M doc/src/sgml/legal.sgml
M doc/src/sgml/lobj.sgml
M src/backend/Makefile
M src/backend/access/brin/brin.c
M src/backend/access/brin/brin_inclusion.c
M src/backend/access/brin/brin_minmax.c
M src/backend/access/brin/brin_pageops.c
M src/backend/access/brin/brin_revmap.c
M src/backend/access/brin/brin_tuple.c
M src/backend/access/brin/brin_validate.c
M src/backend/access/brin/brin_xlog.c
M src/backend/access/common/attmap.c
M src/backend/access/common/bufmask.c
M src/backend/access/common/detoast.c
M src/backend/access/common/heaptuple.c
M src/backend/access/common/indextuple.c
M src/backend/access/common/printsimple.c
M src/backend/access/common/printtup.c
M src/backend/access/common/relation.c
M src/backend/access/common/reloptions.c
M src/backend/access/common/scankey.c
M src/backend/access/common/session.c
M src/backend/access/common/syncscan.c
M src/backend/access/common/toast_internals.c
M src/backend/access/common/tupconvert.c
M src/backend/access/common/tupdesc.c
M src/backend/access/gin/ginarrayproc.c
M src/backend/access/gin/ginbtree.c
M src/backend/access/gin/ginbulk.c
M src/backend/access/gin/gindatapage.c
M src/backend/access/gin/ginentrypage.c
M src/backend/access/gin/ginfast.c
M src/backend/access/gin/ginget.c
M src/backend/access/gin/gininsert.c
M src/backend/access/gin/ginlogic.c
M src/backend/access/gin/ginpostinglist.c
M src/backend/access/gin/ginscan.c
M src/backend/access/gin/ginutil.c
M src/backend/access/gin/ginvacuum.c
M src/backend/access/gin/ginvalidate.c
M src/backend/access/gin/ginxlog.c
M src/backend/access/gist/gist.c
M src/backend/access/gist/gistbuild.c
M src/backend/access/gist/gistbuildbuffers.c
M src/backend/access/gist/gistget.c
M src/backend/access/gist/gistproc.c
M src/backend/access/gist/gistscan.c
M src/backend/access/gist/gistsplit.c
M src/backend/access/gist/gistutil.c
M src/backend/access/gist/gistvacuum.c
M src/backend/access/gist/gistvalidate.c
M src/backend/access/gist/gistxlog.c
M src/backend/access/hash/hash.c
M src/backend/access/hash/hash_xlog.c
M src/backend/access/hash/hashfunc.c
M src/backend/access/hash/hashinsert.c
M src/backend/access/hash/hashovfl.c
M src/backend/access/hash/hashpage.c
M src/backend/access/hash/hashsearch.c
M src/backend/access/hash/hashsort.c
M src/backend/access/hash/hashutil.c
M src/backend/access/hash/hashvalidate.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/heap/heapam_visibility.c
M src/backend/access/heap/heaptoast.c
M src/backend/access/heap/hio.c
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/rewriteheap.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/heap/visibilitymap.c
M src/backend/access/index/amapi.c
M src/backend/access/index/amvalidate.c
M src/backend/access/index/genam.c
M src/backend/access/index/indexam.c
M src/backend/access/nbtree/nbtcompare.c
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/access/nbtree/nbtsplitloc.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/access/nbtree/nbtvalidate.c
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/rmgrdesc/brindesc.c
M src/backend/access/rmgrdesc/clogdesc.c
M src/backend/access/rmgrdesc/committsdesc.c
M src/backend/access/rmgrdesc/dbasedesc.c
M src/backend/access/rmgrdesc/genericdesc.c
M src/backend/access/rmgrdesc/gindesc.c
M src/backend/access/rmgrdesc/gistdesc.c
M src/backend/access/rmgrdesc/hashdesc.c
M src/backend/access/rmgrdesc/heapdesc.c
M src/backend/access/rmgrdesc/logicalmsgdesc.c
M src/backend/access/rmgrdesc/mxactdesc.c
M src/backend/access/rmgrdesc/nbtdesc.c
M src/backend/access/rmgrdesc/relmapdesc.c
M src/backend/access/rmgrdesc/replorigindesc.c
M src/backend/access/rmgrdesc/seqdesc.c
M src/backend/access/rmgrdesc/smgrdesc.c
M src/backend/access/rmgrdesc/spgdesc.c
M src/backend/access/rmgrdesc/standbydesc.c
M src/backend/access/rmgrdesc/tblspcdesc.c
M src/backend/access/rmgrdesc/xactdesc.c
M src/backend/access/rmgrdesc/xlogdesc.c
M src/backend/access/spgist/spgdoinsert.c
M src/backend/access/spgist/spginsert.c
M src/backend/access/spgist/spgkdtreeproc.c
M src/backend/access/spgist/spgproc.c
M src/backend/access/spgist/spgquadtreeproc.c
M src/backend/access/spgist/spgscan.c
M src/backend/access/spgist/spgtextproc.c
M src/backend/access/spgist/spgutils.c
M src/backend/access/spgist/spgvacuum.c
M src/backend/access/spgist/spgvalidate.c
M src/backend/access/spgist/spgxlog.c
M src/backend/access/table/table.c
M src/backend/access/table/tableam.c
M src/backend/access/table/tableamapi.c
M src/backend/access/table/toast_helper.c
M src/backend/access/tablesample/bernoulli.c
M src/backend/access/tablesample/system.c
M src/backend/access/tablesample/tablesample.c
M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/generic_xlog.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/parallel.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/access/transam/timeline.c
M src/backend/access/transam/transam.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/twophase_rmgr.c
M src/backend/access/transam/varsup.c
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogarchive.c
M src/backend/access/transam/xlogfuncs.c
M src/backend/access/transam/xloginsert.c
M src/backend/access/transam/xlogreader.c
M src/backend/access/transam/xlogutils.c
M src/backend/bootstrap/bootparse.y
M src/backend/bootstrap/bootscanner.l
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/Catalog.pm
M src/backend/catalog/Makefile
M src/backend/catalog/aclchk.c
M src/backend/catalog/catalog.c
M src/backend/catalog/dependency.c
M src/backend/catalog/genbki.pl
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/indexing.c
M src/backend/catalog/information_schema.sql
M src/backend/catalog/namespace.c
M src/backend/catalog/objectaccess.c
M src/backend/catalog/objectaddress.c
M src/backend/catalog/partition.c
M src/backend/catalog/pg_aggregate.c
M src/backend/catalog/pg_cast.c
M src/backend/catalog/pg_collation.c
M src/backend/catalog/pg_constraint.c
M src/backend/catalog/pg_conversion.c
M src/backend/catalog/pg_db_role_setting.c
M src/backend/catalog/pg_depend.c
M src/backend/catalog/pg_enum.c
M src/backend/catalog/pg_inherits.c
M src/backend/catalog/pg_largeobject.c
M src/backend/catalog/pg_namespace.c
M src/backend/catalog/pg_operator.c
M src/backend/catalog/pg_proc.c
M src/backend/catalog/pg_publication.c
M src/backend/catalog/pg_range.c
M src/backend/catalog/pg_shdepend.c
M src/backend/catalog/pg_subscription.c
M src/backend/catalog/pg_type.c
M src/backend/catalog/storage.c
M src/backend/catalog/system_views.sql
M src/backend/catalog/toasting.c
M src/backend/commands/aggregatecmds.c
M src/backend/commands/alter.c
M src/backend/commands/amcmds.c
M src/backend/commands/analyze.c
M src/backend/commands/async.c
M src/backend/commands/cluster.c
M src/backend/commands/collationcmds.c
M src/backend/commands/comment.c
M src/backend/commands/constraint.c
M src/backend/commands/conversioncmds.c
M src/backend/commands/copy.c
M src/backend/commands/copyfrom.c
M src/backend/commands/copyfromparse.c
M src/backend/commands/copyto.c
M src/backend/commands/createas.c
M src/backend/commands/dbcommands.c
M src/backend/commands/define.c
M src/backend/commands/discard.c
M src/backend/commands/dropcmds.c
M src/backend/commands/event_trigger.c
M src/backend/commands/explain.c
M src/backend/commands/extension.c
M src/backend/commands/foreigncmds.c
M src/backend/commands/functioncmds.c
M src/backend/commands/indexcmds.c
M src/backend/commands/lockcmds.c
M src/backend/commands/matview.c
M src/backend/commands/opclasscmds.c
M src/backend/commands/operatorcmds.c
M src/backend/commands/policy.c
M src/backend/commands/portalcmds.c
M src/backend/commands/prepare.c
M src/backend/commands/proclang.c
M src/backend/commands/publicationcmds.c
M src/backend/commands/schemacmds.c
M src/backend/commands/seclabel.c
M src/backend/commands/sequence.c
M src/backend/commands/statscmds.c
M src/backend/commands/subscriptioncmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/tablespace.c
M src/backend/commands/trigger.c
M src/backend/commands/tsearchcmds.c
M src/backend/commands/typecmds.c
M src/backend/commands/user.c
M src/backend/commands/vacuum.c
M src/backend/commands/variable.c
M src/backend/commands/view.c
M src/backend/executor/execAmi.c
M src/backend/executor/execCurrent.c
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/execGrouping.c
M src/backend/executor/execIndexing.c
M src/backend/executor/execJunk.c
M src/backend/executor/execMain.c
M src/backend/executor/execParallel.c
M src/backend/executor/execPartition.c
M src/backend/executor/execProcnode.c
M src/backend/executor/execReplication.c
M src/backend/executor/execSRF.c
M src/backend/executor/execScan.c
M src/backend/executor/execTuples.c
M src/backend/executor/execUtils.c
M src/backend/executor/functions.c
M src/backend/executor/instrument.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeAppend.c
M src/backend/executor/nodeBitmapAnd.c
M src/backend/executor/nodeBitmapHeapscan.c
M src/backend/executor/nodeBitmapIndexscan.c
M src/backend/executor/nodeBitmapOr.c
M src/backend/executor/nodeCtescan.c
M src/backend/executor/nodeCustom.c
M src/backend/executor/nodeForeignscan.c
M src/backend/executor/nodeFunctionscan.c
M src/backend/executor/nodeGather.c
M src/backend/executor/nodeGatherMerge.c
M src/backend/executor/nodeGroup.c
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeHashjoin.c
M src/backend/executor/nodeIncrementalSort.c
M src/backend/executor/nodeIndexonlyscan.c
M src/backend/executor/nodeIndexscan.c
M src/backend/executor/nodeLimit.c
M src/backend/executor/nodeLockRows.c
M src/backend/executor/nodeMaterial.c
M src/backend/executor/nodeMergeAppend.c
M src/backend/executor/nodeMergejoin.c
M src/backend/executor/nodeModifyTable.c
M src/backend/executor/nodeNamedtuplestorescan.c
M src/backend/executor/nodeNestloop.c
M src/backend/executor/nodeProjectSet.c
M src/backend/executor/nodeRecursiveunion.c
M src/backend/executor/nodeResult.c
M src/backend/executor/nodeSamplescan.c
M src/backend/executor/nodeSeqscan.c
M src/backend/executor/nodeSetOp.c
M src/backend/executor/nodeSort.c
M src/backend/executor/nodeSubplan.c
M src/backend/executor/nodeSubqueryscan.c
M src/backend/executor/nodeTableFuncscan.c
M src/backend/executor/nodeTidscan.c
M src/backend/executor/nodeUnique.c
M src/backend/executor/nodeValuesscan.c
M src/backend/executor/nodeWindowAgg.c
M src/backend/executor/nodeWorktablescan.c
M src/backend/executor/spi.c
M src/backend/executor/tqueue.c
M src/backend/executor/tstoreReceiver.c
M src/backend/foreign/foreign.c
M src/backend/jit/jit.c
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_deform.c
M src/backend/jit/llvm/llvmjit_error.cpp
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/jit/llvm/llvmjit_inline.cpp
M src/backend/jit/llvm/llvmjit_types.c
M src/backend/jit/llvm/llvmjit_wrap.cpp
M src/backend/lib/binaryheap.c
M src/backend/lib/bipartite_match.c
M src/backend/lib/bloomfilter.c
M src/backend/lib/dshash.c
M src/backend/lib/hyperloglog.c
M src/backend/lib/ilist.c
M src/backend/lib/integerset.c
M src/backend/lib/knapsack.c
M src/backend/lib/pairingheap.c
M src/backend/lib/rbtree.c
M src/backend/libpq/auth-scram.c
M src/backend/libpq/auth.c
M src/backend/libpq/be-fsstubs.c
M src/backend/libpq/be-gssapi-common.c
M src/backend/libpq/be-secure-common.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/libpq/be-secure-openssl.c
M src/backend/libpq/be-secure.c
M src/backend/libpq/crypt.c
M src/backend/libpq/hba.c
M src/backend/libpq/ifaddr.c
M src/backend/libpq/pqcomm.c
M src/backend/libpq/pqformat.c
M src/backend/libpq/pqmq.c
M src/backend/libpq/pqsignal.c
M src/backend/main/main.c
M src/backend/nodes/bitmapset.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/extensible.c
M src/backend/nodes/list.c
M src/backend/nodes/makefuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/nodes.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/params.c
M src/backend/nodes/print.c
M src/backend/nodes/read.c
M src/backend/nodes/readfuncs.c
M src/backend/nodes/tidbitmap.c
M src/backend/nodes/value.c
M src/backend/optimizer/geqo/geqo_copy.c
M src/backend/optimizer/geqo/geqo_eval.c
M src/backend/optimizer/geqo/geqo_main.c
M src/backend/optimizer/geqo/geqo_misc.c
M src/backend/optimizer/geqo/geqo_pool.c
M src/backend/optimizer/geqo/geqo_random.c
M src/backend/optimizer/geqo/geqo_selection.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/clausesel.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/path/joinrels.c
M src/backend/optimizer/path/pathkeys.c
M src/backend/optimizer/path/tidpath.c
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/planagg.c
M src/backend/optimizer/plan/planmain.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepagg.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/optimizer/prep/prepqual.c
M src/backend/optimizer/prep/preptlist.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/appendinfo.c
M src/backend/optimizer/util/clauses.c
M src/backend/optimizer/util/inherit.c
M src/backend/optimizer/util/joininfo.c
M src/backend/optimizer/util/orclauses.c
M src/backend/optimizer/util/paramassign.c
M src/backend/optimizer/util/pathnode.c
M src/backend/optimizer/util/placeholder.c
M src/backend/optimizer/util/plancat.c
M src/backend/optimizer/util/predtest.c
M src/backend/optimizer/util/relnode.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/optimizer/util/tlist.c
M src/backend/optimizer/util/var.c
M src/backend/parser/analyze.c
M src/backend/parser/check_keywords.pl
M src/backend/parser/gram.y
M src/backend/parser/parse_agg.c
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_coerce.c
M src/backend/parser/parse_collate.c
M src/backend/parser/parse_cte.c
M src/backend/parser/parse_enr.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_func.c
M src/backend/parser/parse_node.c
M src/backend/parser/parse_oper.c
M src/backend/parser/parse_param.c
M src/backend/parser/parse_relation.c
M src/backend/parser/parse_target.c
M src/backend/parser/parse_type.c
M src/backend/parser/parse_utilcmd.c
M src/backend/parser/parser.c
M src/backend/parser/scan.l
M src/backend/parser/scansup.c
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partdesc.c
M src/backend/partitioning/partprune.c
M src/backend/port/atomics.c
M src/backend/port/posix_sema.c
M src/backend/port/sysv_sema.c
M src/backend/port/sysv_shmem.c
M src/backend/port/tas/sunstudio_sparc.s
M src/backend/port/tas/sunstudio_x86.s
M src/backend/port/win32/crashdump.c
M src/backend/port/win32/signal.c
M src/backend/port/win32/socket.c
M src/backend/port/win32/timer.c
M src/backend/port/win32_sema.c
M src/backend/port/win32_shmem.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/bgwriter.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/fork_process.c
M src/backend/postmaster/interrupt.c
M src/backend/postmaster/pgarch.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/postmaster/startup.c
M src/backend/postmaster/syslogger.c
M src/backend/postmaster/walwriter.c
M src/backend/regex/regc_pg_locale.c
M src/backend/regex/regexport.c
M src/backend/regex/regprefix.c
M src/backend/replication/backup_manifest.c
M src/backend/replication/basebackup.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/launcher.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/logicalfuncs.c
M src/backend/replication/logical/message.c
M src/backend/replication/logical/origin.c
M src/backend/replication/logical/proto.c
M src/backend/replication/logical/relation.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/replication/repl_gram.y
M src/backend/replication/repl_scanner.l
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/backend/replication/syncrep.c
M src/backend/replication/syncrep_gram.y
M src/backend/replication/syncrep_scanner.l
M src/backend/replication/walreceiver.c
M src/backend/replication/walreceiverfuncs.c
M src/backend/replication/walsender.c
M src/backend/rewrite/rewriteDefine.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/rewrite/rewriteManip.c
M src/backend/rewrite/rewriteRemove.c
M src/backend/rewrite/rewriteSupport.c
M src/backend/rewrite/rowsecurity.c
M src/backend/snowball/dict_snowball.c
M src/backend/snowball/snowball.sql.in
M src/backend/snowball/snowball_func.sql.in
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/statistics/mcv.c
M src/backend/statistics/mvdistinct.c
M src/backend/storage/buffer/buf_init.c
M src/backend/storage/buffer/buf_table.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/freelist.c
M src/backend/storage/buffer/localbuf.c
M src/backend/storage/file/buffile.c
M src/backend/storage/file/copydir.c
M src/backend/storage/file/fd.c
M src/backend/storage/file/reinit.c
M src/backend/storage/file/sharedfileset.c
M src/backend/storage/freespace/freespace.c
M src/backend/storage/freespace/fsmpage.c
M src/backend/storage/freespace/indexfsm.c
M src/backend/storage/ipc/barrier.c
M src/backend/storage/ipc/dsm.c
M src/backend/storage/ipc/dsm_impl.c
M src/backend/storage/ipc/ipc.c
M src/backend/storage/ipc/ipci.c
M src/backend/storage/ipc/latch.c
M src/backend/storage/ipc/pmsignal.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/procsignal.c
M src/backend/storage/ipc/shm_mq.c
M src/backend/storage/ipc/shm_toc.c
M src/backend/storage/ipc/shmem.c
M src/backend/storage/ipc/shmqueue.c
M src/backend/storage/ipc/signalfuncs.c
M src/backend/storage/ipc/sinval.c
M src/backend/storage/ipc/sinvaladt.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/large_object/inv_api.c
M src/backend/storage/lmgr/condition_variable.c
M src/backend/storage/lmgr/deadlock.c
M src/backend/storage/lmgr/generate-lwlocknames.pl
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/lwlock.c
M src/backend/storage/lmgr/predicate.c
M src/backend/storage/lmgr/proc.c
M src/backend/storage/lmgr/s_lock.c
M src/backend/storage/lmgr/spin.c
M src/backend/storage/page/bufpage.c
M src/backend/storage/page/checksum.c
M src/backend/storage/page/itemptr.c
M src/backend/storage/smgr/md.c
M src/backend/storage/smgr/smgr.c
M src/backend/storage/sync/sync.c
M src/backend/tcop/cmdtag.c
M src/backend/tcop/dest.c
M src/backend/tcop/fastpath.c
M src/backend/tcop/postgres.c
M src/backend/tcop/pquery.c
M src/backend/tcop/utility.c
M src/backend/tsearch/Makefile
M src/backend/tsearch/dict.c
M src/backend/tsearch/dict_ispell.c
M src/backend/tsearch/dict_simple.c
M src/backend/tsearch/dict_synonym.c
M src/backend/tsearch/dict_thesaurus.c
M src/backend/tsearch/regis.c
M src/backend/tsearch/spell.c
M src/backend/tsearch/to_tsany.c
M src/backend/tsearch/ts_locale.c
M src/backend/tsearch/ts_parse.c
M src/backend/tsearch/ts_selfuncs.c
M src/backend/tsearch/ts_typanalyze.c
M src/backend/tsearch/ts_utils.c
M src/backend/tsearch/wparser.c
M src/backend/tsearch/wparser_def.c
M src/backend/utils/Gen_dummy_probes.pl
M src/backend/utils/Gen_dummy_probes.sed
M src/backend/utils/Gen_fmgrtab.pl
M src/backend/utils/Makefile
M src/backend/utils/adt/acl.c
M src/backend/utils/adt/amutils.c
M src/backend/utils/adt/array_expanded.c
M src/backend/utils/adt/array_selfuncs.c
M src/backend/utils/adt/array_typanalyze.c
M src/backend/utils/adt/array_userfuncs.c
M src/backend/utils/adt/arrayfuncs.c
M src/backend/utils/adt/arraysubs.c
M src/backend/utils/adt/arrayutils.c
M src/backend/utils/adt/ascii.c
M src/backend/utils/adt/bool.c
M src/backend/utils/adt/char.c
M src/backend/utils/adt/cryptohashfuncs.c
M src/backend/utils/adt/date.c
M src/backend/utils/adt/datetime.c
M src/backend/utils/adt/datum.c
M src/backend/utils/adt/dbsize.c
M src/backend/utils/adt/domains.c
M src/backend/utils/adt/encode.c
M src/backend/utils/adt/enum.c
M src/backend/utils/adt/expandeddatum.c
M src/backend/utils/adt/expandedrecord.c
M src/backend/utils/adt/float.c
M src/backend/utils/adt/format_type.c
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/genfile.c
M src/backend/utils/adt/geo_ops.c
M src/backend/utils/adt/geo_selfuncs.c
M src/backend/utils/adt/geo_spgist.c
M src/backend/utils/adt/int.c
M src/backend/utils/adt/int8.c
M src/backend/utils/adt/json.c
M src/backend/utils/adt/jsonb.c
M src/backend/utils/adt/jsonb_gin.c
M src/backend/utils/adt/jsonb_op.c
M src/backend/utils/adt/jsonb_util.c
M src/backend/utils/adt/jsonfuncs.c
M src/backend/utils/adt/jsonpath.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/adt/jsonpath_gram.y
M src/backend/utils/adt/jsonpath_scan.l
M src/backend/utils/adt/levenshtein.c
M src/backend/utils/adt/like.c
M src/backend/utils/adt/like_match.c
M src/backend/utils/adt/like_support.c
M src/backend/utils/adt/lockfuncs.c
M src/backend/utils/adt/mac.c
M src/backend/utils/adt/mac8.c
M src/backend/utils/adt/mcxtfuncs.c
M src/backend/utils/adt/misc.c
M src/backend/utils/adt/multirangetypes.c
M src/backend/utils/adt/multirangetypes_selfuncs.c
M src/backend/utils/adt/name.c
M src/backend/utils/adt/network_gist.c
M src/backend/utils/adt/network_selfuncs.c
M src/backend/utils/adt/network_spgist.c
M src/backend/utils/adt/numeric.c
M src/backend/utils/adt/numutils.c
M src/backend/utils/adt/oid.c
M src/backend/utils/adt/oracle_compat.c
M src/backend/utils/adt/orderedsetaggs.c
M src/backend/utils/adt/partitionfuncs.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_lsn.c
M src/backend/utils/adt/pg_upgrade_support.c
M src/backend/utils/adt/pgstatfuncs.c
M src/backend/utils/adt/pseudotypes.c
M src/backend/utils/adt/quote.c
M src/backend/utils/adt/rangetypes.c
M src/backend/utils/adt/rangetypes_gist.c
M src/backend/utils/adt/rangetypes_selfuncs.c
M src/backend/utils/adt/rangetypes_spgist.c
M src/backend/utils/adt/rangetypes_typanalyze.c
M src/backend/utils/adt/regexp.c
M src/backend/utils/adt/regproc.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/adt/rowtypes.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/adt/selfuncs.c
M src/backend/utils/adt/tid.c
M src/backend/utils/adt/timestamp.c
M src/backend/utils/adt/trigfuncs.c
M src/backend/utils/adt/tsginidx.c
M src/backend/utils/adt/tsgistidx.c
M src/backend/utils/adt/tsquery.c
M src/backend/utils/adt/tsquery_cleanup.c
M src/backend/utils/adt/tsquery_gist.c
M src/backend/utils/adt/tsquery_op.c
M src/backend/utils/adt/tsquery_rewrite.c
M src/backend/utils/adt/tsquery_util.c
M src/backend/utils/adt/tsrank.c
M src/backend/utils/adt/tsvector.c
M src/backend/utils/adt/tsvector_op.c
M src/backend/utils/adt/tsvector_parser.c
M src/backend/utils/adt/uuid.c
M src/backend/utils/adt/varbit.c
M src/backend/utils/adt/varchar.c
M src/backend/utils/adt/varlena.c
M src/backend/utils/adt/version.c
M src/backend/utils/adt/windowfuncs.c
M src/backend/utils/adt/xid.c
M src/backend/utils/adt/xid8funcs.c
M src/backend/utils/adt/xml.c
M src/backend/utils/cache/attoptcache.c
M src/backend/utils/cache/catcache.c
M src/backend/utils/cache/evtcache.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/lsyscache.c
M src/backend/utils/cache/partcache.c
M src/backend/utils/cache/plancache.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/cache/relfilenodemap.c
M src/backend/utils/cache/relmapper.c
M src/backend/utils/cache/spccache.c
M src/backend/utils/cache/syscache.c
M src/backend/utils/cache/ts_cache.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/errcodes.txt
M src/backend/utils/error/assert.c
M src/backend/utils/error/elog.c
M src/backend/utils/fmgr/dfmgr.c
M src/backend/utils/fmgr/fmgr.c
M src/backend/utils/fmgr/funcapi.c
M src/backend/utils/generate-errcodes.pl
M src/backend/utils/hash/dynahash.c
M src/backend/utils/hash/pg_crc.c
M src/backend/utils/init/globals.c
M src/backend/utils/init/miscinit.c
M src/backend/utils/init/postinit.c
M src/backend/utils/mb/Unicode/Makefile
M src/backend/utils/mb/Unicode/UCS_to_BIG5.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_CN.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_JIS_2004.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_JP.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_KR.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_TW.pl
M src/backend/utils/mb/Unicode/UCS_to_GB18030.pl
M src/backend/utils/mb/Unicode/UCS_to_JOHAB.pl
M src/backend/utils/mb/Unicode/UCS_to_SHIFT_JIS_2004.pl
M src/backend/utils/mb/Unicode/UCS_to_SJIS.pl
M src/backend/utils/mb/Unicode/UCS_to_UHC.pl
M src/backend/utils/mb/Unicode/UCS_to_most.pl
M src/backend/utils/mb/Unicode/convutils.pm
M src/backend/utils/mb/conv.c
M src/backend/utils/mb/conversion_procs/Makefile
M src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
M src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
M src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
M src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
M src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
M src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
M src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
M src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
M src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
M src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
M src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
M src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
M src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
M src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
M src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
M src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
M src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
M src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
M src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
M src/backend/utils/mb/mbutils.c
M src/backend/utils/mb/stringinfo_mb.c
M src/backend/utils/misc/guc-file.l
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/help_config.c
M src/backend/utils/misc/pg_config.c
M src/backend/utils/misc/pg_controldata.c
M src/backend/utils/misc/pg_rusage.c
M src/backend/utils/misc/ps_status.c
M src/backend/utils/misc/queryenvironment.c
M src/backend/utils/misc/rls.c
M src/backend/utils/misc/sampling.c
M src/backend/utils/misc/superuser.c
M src/backend/utils/misc/timeout.c
M src/backend/utils/misc/tzparser.c
M src/backend/utils/mmgr/aset.c
M src/backend/utils/mmgr/dsa.c
M src/backend/utils/mmgr/freepage.c
M src/backend/utils/mmgr/generation.c
M src/backend/utils/mmgr/mcxt.c
M src/backend/utils/mmgr/memdebug.c
M src/backend/utils/mmgr/portalmem.c
M src/backend/utils/mmgr/slab.c
M src/backend/utils/probes.d
M src/backend/utils/resowner/resowner.c
M src/backend/utils/sort/logtape.c
M src/backend/utils/sort/sharedtuplestore.c
M src/backend/utils/sort/sortsupport.c
M src/backend/utils/sort/tuplesort.c
M src/backend/utils/sort/tuplestore.c
M src/backend/utils/time/combocid.c
M src/backend/utils/time/snapmgr.c
M src/bin/Makefile
M src/bin/initdb/Makefile
M src/bin/initdb/findtimezone.c
M src/bin/initdb/initdb.c
M src/bin/pg_basebackup/Makefile
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_basebackup/pg_receivewal.c
M src/bin/pg_basebackup/pg_recvlogical.c
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/receivelog.h
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_basebackup/streamutil.h
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_basebackup/walmethods.h
M src/bin/pg_checksums/Makefile
M src/bin/pg_checksums/pg_checksums.c
M src/bin/pg_config/Makefile
M src/bin/pg_config/pg_config.c
M src/bin/pg_controldata/Makefile
M src/bin/pg_ctl/Makefile
M src/bin/pg_ctl/pg_ctl.c
M src/bin/pg_dump/Makefile
M src/bin/pg_dump/common.c
M src/bin/pg_dump/compress_io.c
M src/bin/pg_dump/compress_io.h
M src/bin/pg_dump/dumputils.c
M src/bin/pg_dump/dumputils.h
M src/bin/pg_dump/parallel.c
M src/bin/pg_dump/parallel.h
M src/bin/pg_dump/pg_backup_directory.c
M src/bin/pg_dump/pg_backup_utils.c
M src/bin/pg_dump/pg_backup_utils.h
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/pg_dump_sort.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_resetwal/Makefile
M src/bin/pg_resetwal/pg_resetwal.c
M src/bin/pg_rewind/Makefile
M src/bin/pg_rewind/datapagemap.c
M src/bin/pg_rewind/datapagemap.h
M src/bin/pg_rewind/file_ops.c
M src/bin/pg_rewind/file_ops.h
M src/bin/pg_rewind/filemap.c
M src/bin/pg_rewind/filemap.h
M src/bin/pg_rewind/libpq_source.c
M src/bin/pg_rewind/local_source.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/pg_rewind.h
M src/bin/pg_rewind/rewind_source.h
M src/bin/pg_rewind/timeline.c
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/dump.c
M src/bin/pg_upgrade/exec.c
M src/bin/pg_upgrade/file.c
M src/bin/pg_upgrade/function.c
M src/bin/pg_upgrade/info.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/parallel.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/relfilenode.c
M src/bin/pg_upgrade/server.c
M src/bin/pg_upgrade/tablespace.c
M src/bin/pg_upgrade/test.sh
M src/bin/pg_upgrade/util.c
M src/bin/pg_upgrade/version.c
M src/bin/pg_verifybackup/parse_manifest.c
M src/bin/pg_verifybackup/parse_manifest.h
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/bin/pg_waldump/compat.c
M src/bin/pg_waldump/pg_waldump.c
M src/bin/pgbench/exprparse.y
M src/bin/pgbench/exprscan.l
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/pgbench.h
M src/bin/pgevent/Makefile
M src/bin/psql/Makefile
M src/bin/psql/command.c
M src/bin/psql/command.h
M src/bin/psql/common.c
M src/bin/psql/common.h
M src/bin/psql/copy.c
M src/bin/psql/copy.h
M src/bin/psql/create_help.pl
M src/bin/psql/crosstabview.c
M src/bin/psql/crosstabview.h
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/bin/psql/help.c
M src/bin/psql/help.h
M src/bin/psql/input.c
M src/bin/psql/input.h
M src/bin/psql/large_obj.c
M src/bin/psql/large_obj.h
M src/bin/psql/mainloop.c
M src/bin/psql/mainloop.h
M src/bin/psql/prompt.c
M src/bin/psql/prompt.h
M src/bin/psql/psqlscanslash.h
M src/bin/psql/psqlscanslash.l
M src/bin/psql/settings.h
M src/bin/psql/startup.c
M src/bin/psql/stringutils.c
M src/bin/psql/stringutils.h
M src/bin/psql/tab-complete.c
M src/bin/psql/tab-complete.h
M src/bin/psql/variables.c
M src/bin/psql/variables.h
M src/bin/scripts/Makefile
M src/bin/scripts/clusterdb.c
M src/bin/scripts/common.c
M src/bin/scripts/common.h
M src/bin/scripts/createdb.c
M src/bin/scripts/createuser.c
M src/bin/scripts/dropdb.c
M src/bin/scripts/dropuser.c
M src/bin/scripts/pg_isready.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/scripts_parallel.c
M src/bin/scripts/scripts_parallel.h
M src/bin/scripts/vacuumdb.c
M src/common/archive.c
M src/common/base64.c
M src/common/checksum_helper.c
M src/common/config_info.c
M src/common/controldata_utils.c
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
M src/common/d2s.c
M src/common/d2s_full_table.h
M src/common/d2s_intrinsics.h
M src/common/encnames.c
M src/common/exec.c
M src/common/f2s.c
M src/common/fe_memutils.c
M src/common/file_perm.c
M src/common/file_utils.c
M src/common/hashfn.c
M src/common/hex_decode.c
M src/common/ip.c
M src/common/jsonapi.c
M src/common/keywords.c
M src/common/kwlookup.c
M src/common/link-canary.c
M src/common/logging.c
M src/common/md5.c
M src/common/md5_common.c
M src/common/md5_int.h
M src/common/pg_get_line.c
M src/common/pg_lzcompress.c
M src/common/pgfnames.c
M src/common/protocol_openssl.c
M src/common/psprintf.c
M src/common/relpath.c
M src/common/restricted_token.c
M src/common/rmtree.c
M src/common/ryu_common.h
M src/common/saslprep.c
M src/common/scram-common.c
M src/common/sha2.c
M src/common/sha2_int.h
M src/common/sprompt.c
M src/common/string.c
M src/common/stringinfo.c
M src/common/unicode/generate-norm_test_table.pl
M src/common/unicode/generate-unicode_combining_table.pl
M src/common/unicode/generate-unicode_norm_table.pl
M src/common/unicode/generate-unicode_normprops_table.pl
M src/common/unicode/norm_test.c
M src/common/unicode_norm.c
M src/common/username.c
M src/common/wait_error.c
M src/common/wchar.c
M src/fe_utils/Makefile
M src/fe_utils/archive.c
M src/fe_utils/cancel.c
M src/fe_utils/conditional.c
M src/fe_utils/mbprint.c
M src/fe_utils/print.c
M src/fe_utils/psqlscan.l
M src/fe_utils/recovery_gen.c
M src/fe_utils/simple_list.c
M src/fe_utils/string_utils.c
M src/include/access/amapi.h
M src/include/access/amvalidate.h
M src/include/access/attmap.h
M src/include/access/attnum.h
M src/include/access/brin.h
M src/include/access/brin_internal.h
M src/include/access/brin_page.h
M src/include/access/brin_pageops.h
M src/include/access/brin_revmap.h
M src/include/access/brin_tuple.h
M src/include/access/brin_xlog.h
M src/include/access/bufmask.h
M src/include/access/clog.h
M src/include/access/commit_ts.h
M src/include/access/detoast.h
M src/include/access/genam.h
M src/include/access/generic_xlog.h
M src/include/access/gin.h
M src/include/access/gin_private.h
M src/include/access/ginblock.h
M src/include/access/ginxlog.h
M src/include/access/gist.h
M src/include/access/gist_private.h
M src/include/access/gistscan.h
M src/include/access/gistxlog.h
M src/include/access/hash.h
M src/include/access/hash_xlog.h
M src/include/access/heapam.h
M src/include/access/heapam_xlog.h
M src/include/access/heaptoast.h
M src/include/access/hio.h
M src/include/access/htup.h
M src/include/access/htup_details.h
M src/include/access/itup.h
M src/include/access/multixact.h
M src/include/access/nbtree.h
M src/include/access/nbtxlog.h
M src/include/access/parallel.h
M src/include/access/printsimple.h
M src/include/access/printtup.h
M src/include/access/relation.h
M src/include/access/reloptions.h
M src/include/access/relscan.h
M src/include/access/rewriteheap.h
M src/include/access/rmgrlist.h
M src/include/access/sdir.h
M src/include/access/session.h
M src/include/access/skey.h
M src/include/access/slru.h
M src/include/access/spgist.h
M src/include/access/spgist_private.h
M src/include/access/spgxlog.h
M src/include/access/stratnum.h
M src/include/access/subtrans.h
M src/include/access/syncscan.h
M src/include/access/sysattr.h
M src/include/access/table.h
M src/include/access/tableam.h
M src/include/access/timeline.h
M src/include/access/toast_helper.h
M src/include/access/toast_internals.h
M src/include/access/transam.h
M src/include/access/tsmapi.h
M src/include/access/tupconvert.h
M src/include/access/tupdesc.h
M src/include/access/tupdesc_details.h
M src/include/access/tupmacs.h
M src/include/access/twophase.h
M src/include/access/twophase_rmgr.h
M src/include/access/valid.h
M src/include/access/visibilitymap.h
M src/include/access/xact.h
M src/include/access/xlog.h
M src/include/access/xlog_internal.h
M src/include/access/xlogarchive.h
M src/include/access/xlogdefs.h
M src/include/access/xloginsert.h
M src/include/access/xlogreader.h
M src/include/access/xlogrecord.h
M src/include/access/xlogutils.h
M src/include/bootstrap/bootstrap.h
M src/include/c.h
M src/include/catalog/Makefile
M src/include/catalog/binary_upgrade.h
M src/include/catalog/catalog.h
M src/include/catalog/catversion.h
M src/include/catalog/dependency.h
M src/include/catalog/duplicate_oids
M src/include/catalog/genbki.h
M src/include/catalog/heap.h
M src/include/catalog/index.h
M src/include/catalog/indexing.h
M src/include/catalog/namespace.h
M src/include/catalog/objectaccess.h
M src/include/catalog/objectaddress.h
M src/include/catalog/partition.h
M src/include/catalog/pg_aggregate.dat
M src/include/catalog/pg_aggregate.h
M src/include/catalog/pg_am.dat
M src/include/catalog/pg_am.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amop.h
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_amproc.h
M src/include/catalog/pg_attrdef.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_auth_members.h
M src/include/catalog/pg_authid.dat
M src/include/catalog/pg_authid.h
M src/include/catalog/pg_cast.dat
M src/include/catalog/pg_cast.h
M src/include/catalog/pg_class.dat
M src/include/catalog/pg_class.h
M src/include/catalog/pg_collation.dat
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_control.h
M src/include/catalog/pg_conversion.dat
M src/include/catalog/pg_conversion.h
M src/include/catalog/pg_database.dat
M src/include/catalog/pg_database.h
M src/include/catalog/pg_db_role_setting.h
M src/include/catalog/pg_default_acl.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_description.h
M src/include/catalog/pg_enum.h
M src/include/catalog/pg_event_trigger.h
M src/include/catalog/pg_extension.h
M src/include/catalog/pg_foreign_data_wrapper.h
M src/include/catalog/pg_foreign_server.h
M src/include/catalog/pg_foreign_table.h
M src/include/catalog/pg_index.h
M src/include/catalog/pg_inherits.h
M src/include/catalog/pg_init_privs.h
M src/include/catalog/pg_language.dat
M src/include/catalog/pg_language.h
M src/include/catalog/pg_largeobject.h
M src/include/catalog/pg_largeobject_metadata.h
M src/include/catalog/pg_namespace.dat
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_opclass.h
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_opfamily.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_policy.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_publication.h
M src/include/catalog/pg_publication_rel.h
M src/include/catalog/pg_range.dat
M src/include/catalog/pg_range.h
M src/include/catalog/pg_replication_origin.h
M src/include/catalog/pg_rewrite.h
M src/include/catalog/pg_seclabel.h
M src/include/catalog/pg_sequence.h
M src/include/catalog/pg_shdepend.h
M src/include/catalog/pg_shdescription.h
M src/include/catalog/pg_shseclabel.h
M src/include/catalog/pg_statistic.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/catalog/pg_subscription.h
M src/include/catalog/pg_subscription_rel.h
M src/include/catalog/pg_tablespace.dat
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_transform.h
M src/include/catalog/pg_trigger.h
M src/include/catalog/pg_ts_config.dat
M src/include/catalog/pg_ts_config.h
M src/include/catalog/pg_ts_config_map.dat
M src/include/catalog/pg_ts_config_map.h
M src/include/catalog/pg_ts_dict.dat
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_ts_parser.dat
M src/include/catalog/pg_ts_parser.h
M src/include/catalog/pg_ts_template.dat
M src/include/catalog/pg_ts_template.h
M src/include/catalog/pg_type.dat
M src/include/catalog/pg_type.h
M src/include/catalog/pg_user_mapping.h
M src/include/catalog/reformat_dat_file.pl
M src/include/catalog/renumber_oids.pl
M src/include/catalog/storage.h
M src/include/catalog/storage_xlog.h
M src/include/catalog/toasting.h
M src/include/catalog/unused_oids
M src/include/commands/alter.h
M src/include/commands/async.h
M src/include/commands/cluster.h
M src/include/commands/collationcmds.h
M src/include/commands/comment.h
M src/include/commands/conversioncmds.h
M src/include/commands/copy.h
M src/include/commands/copyfrom_internal.h
M src/include/commands/createas.h
M src/include/commands/dbcommands.h
M src/include/commands/dbcommands_xlog.h
M src/include/commands/defrem.h
M src/include/commands/discard.h
M src/include/commands/event_trigger.h
M src/include/commands/explain.h
M src/include/commands/extension.h
M src/include/commands/lockcmds.h
M src/include/commands/matview.h
M src/include/commands/policy.h
M src/include/commands/portalcmds.h
M src/include/commands/prepare.h
M src/include/commands/proclang.h
M src/include/commands/progress.h
M src/include/commands/publicationcmds.h
M src/include/commands/schemacmds.h
M src/include/commands/seclabel.h
M src/include/commands/sequence.h
M src/include/commands/subscriptioncmds.h
M src/include/commands/tablecmds.h
M src/include/commands/tablespace.h
M src/include/commands/trigger.h
M src/include/commands/typecmds.h
M src/include/commands/vacuum.h
M src/include/commands/variable.h
M src/include/commands/view.h
M src/include/common/archive.h
M src/include/common/base64.h
M src/include/common/checksum_helper.h
M src/include/common/config_info.h
M src/include/common/connect.h
M src/include/common/controldata_utils.h
M src/include/common/cryptohash.h
M src/include/common/fe_memutils.h
M src/include/common/file_perm.h
M src/include/common/file_utils.h
M src/include/common/hashfn.h
M src/include/common/hex_decode.h
M src/include/common/int.h
M src/include/common/int128.h
M src/include/common/ip.h
M src/include/common/jsonapi.h
M src/include/common/keywords.h
M src/include/common/kwlookup.h
M src/include/common/link-canary.h
M src/include/common/logging.h
M src/include/common/md5.h
M src/include/common/openssl.h
M src/include/common/relpath.h
M src/include/common/restricted_token.h
M src/include/common/saslprep.h
M src/include/common/scram-common.h
M src/include/common/sha2.h
M src/include/common/shortest_dec.h
M src/include/common/string.h
M src/include/common/unicode_norm.h
M src/include/common/unicode_norm_hashfunc.h
M src/include/common/unicode_norm_table.h
M src/include/common/username.h
M src/include/datatype/timestamp.h
M src/include/executor/execExpr.h
M src/include/executor/execParallel.h
M src/include/executor/execPartition.h
M src/include/executor/execdebug.h
M src/include/executor/execdesc.h
M src/include/executor/executor.h
M src/include/executor/functions.h
M src/include/executor/hashjoin.h
M src/include/executor/instrument.h
M src/include/executor/nodeAgg.h
M src/include/executor/nodeAppend.h
M src/include/executor/nodeBitmapAnd.h
M src/include/executor/nodeBitmapHeapscan.h
M src/include/executor/nodeBitmapIndexscan.h
M src/include/executor/nodeBitmapOr.h
M src/include/executor/nodeCtescan.h
M src/include/executor/nodeCustom.h
M src/include/executor/nodeForeignscan.h
M src/include/executor/nodeFunctionscan.h
M src/include/executor/nodeGather.h
M src/include/executor/nodeGatherMerge.h
M src/include/executor/nodeGroup.h
M src/include/executor/nodeHash.h
M src/include/executor/nodeHashjoin.h
M src/include/executor/nodeIncrementalSort.h
M src/include/executor/nodeIndexonlyscan.h
M src/include/executor/nodeIndexscan.h
M src/include/executor/nodeLimit.h
M src/include/executor/nodeLockRows.h
M src/include/executor/nodeMaterial.h
M src/include/executor/nodeMergeAppend.h
M src/include/executor/nodeMergejoin.h
M src/include/executor/nodeModifyTable.h
M src/include/executor/nodeNamedtuplestorescan.h
M src/include/executor/nodeNestloop.h
M src/include/executor/nodeProjectSet.h
M src/include/executor/nodeRecursiveunion.h
M src/include/executor/nodeResult.h
M src/include/executor/nodeSamplescan.h
M src/include/executor/nodeSeqscan.h
M src/include/executor/nodeSetOp.h
M src/include/executor/nodeSort.h
M src/include/executor/nodeSubplan.h
M src/include/executor/nodeSubqueryscan.h
M src/include/executor/nodeTableFuncscan.h
M src/include/executor/nodeTidscan.h
M src/include/executor/nodeUnique.h
M src/include/executor/nodeValuesscan.h
M src/include/executor/nodeWindowAgg.h
M src/include/executor/nodeWorktablescan.h
M src/include/executor/spi.h
M src/include/executor/spi_priv.h
M src/include/executor/tablefunc.h
M src/include/executor/tqueue.h
M src/include/executor/tstoreReceiver.h
M src/include/executor/tuptable.h
M src/include/fe_utils/archive.h
M src/include/fe_utils/cancel.h
M src/include/fe_utils/conditional.h
M src/include/fe_utils/mbprint.h
M src/include/fe_utils/print.h
M src/include/fe_utils/psqlscan.h
M src/include/fe_utils/psqlscan_int.h
M src/include/fe_utils/recovery_gen.h
M src/include/fe_utils/simple_list.h
M src/include/fe_utils/string_utils.h
M src/include/fmgr.h
M src/include/foreign/fdwapi.h
M src/include/foreign/foreign.h
M src/include/funcapi.h
M src/include/getaddrinfo.h
M src/include/getopt_long.h
M src/include/jit/jit.h
M src/include/jit/llvmjit.h
M src/include/jit/llvmjit_emit.h
M src/include/lib/binaryheap.h
M src/include/lib/bipartite_match.h
M src/include/lib/bloomfilter.h
M src/include/lib/dshash.h
M src/include/lib/hyperloglog.h
M src/include/lib/ilist.h
M src/include/lib/integerset.h
M src/include/lib/knapsack.h
M src/include/lib/pairingheap.h
M src/include/lib/qunique.h
M src/include/lib/rbtree.h
M src/include/lib/stringinfo.h
M src/include/libpq/auth.h
M src/include/libpq/be-fsstubs.h
M src/include/libpq/be-gssapi-common.h
M src/include/libpq/crypt.h
M src/include/libpq/ifaddr.h
M src/include/libpq/libpq-be.h
M src/include/libpq/libpq-fs.h
M src/include/libpq/libpq.h
M src/include/libpq/pqcomm.h
M src/include/libpq/pqformat.h
M src/include/libpq/pqmq.h
M src/include/libpq/pqsignal.h
M src/include/libpq/scram.h
M src/include/mb/pg_wchar.h
M src/include/mb/stringinfo_mb.h
M src/include/miscadmin.h
M src/include/nodes/bitmapset.h
M src/include/nodes/execnodes.h
M src/include/nodes/extensible.h
M src/include/nodes/lockoptions.h
M src/include/nodes/makefuncs.h
M src/include/nodes/memnodes.h
M src/include/nodes/nodeFuncs.h
M src/include/nodes/nodes.h
M src/include/nodes/params.h
M src/include/nodes/parsenodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/pg_list.h
M src/include/nodes/plannodes.h
M src/include/nodes/primnodes.h
M src/include/nodes/print.h
M src/include/nodes/readfuncs.h
M src/include/nodes/replnodes.h
M src/include/nodes/subscripting.h
M src/include/nodes/supportnodes.h
M src/include/nodes/tidbitmap.h
M src/include/nodes/value.h
M src/include/optimizer/appendinfo.h
M src/include/optimizer/clauses.h
M src/include/optimizer/cost.h
M src/include/optimizer/geqo.h
M src/include/optimizer/geqo_copy.h
M src/include/optimizer/geqo_gene.h
M src/include/optimizer/geqo_misc.h
M src/include/optimizer/geqo_mutation.h
M src/include/optimizer/geqo_pool.h
M src/include/optimizer/geqo_random.h
M src/include/optimizer/geqo_recombination.h
M src/include/optimizer/geqo_selection.h
M src/include/optimizer/inherit.h
M src/include/optimizer/joininfo.h
M src/include/optimizer/optimizer.h
M src/include/optimizer/orclauses.h
M src/include/optimizer/paramassign.h
M src/include/optimizer/pathnode.h
M src/include/optimizer/paths.h
M src/include/optimizer/placeholder.h
M src/include/optimizer/plancat.h
M src/include/optimizer/planmain.h
M src/include/optimizer/planner.h
M src/include/optimizer/prep.h
M src/include/optimizer/restrictinfo.h
M src/include/optimizer/subselect.h
M src/include/optimizer/tlist.h
M src/include/parser/analyze.h
M src/include/parser/gramparse.h
M src/include/parser/kwlist.h
M src/include/parser/parse_agg.h
M src/include/parser/parse_clause.h
M src/include/parser/parse_coerce.h
M src/include/parser/parse_collate.h
M src/include/parser/parse_cte.h
M src/include/parser/parse_enr.h
M src/include/parser/parse_expr.h
M src/include/parser/parse_func.h
M src/include/parser/parse_node.h
M src/include/parser/parse_oper.h
M src/include/parser/parse_param.h
M src/include/parser/parse_relation.h
M src/include/parser/parse_target.h
M src/include/parser/parse_type.h
M src/include/parser/parse_utilcmd.h
M src/include/parser/parser.h
M src/include/parser/parsetree.h
M src/include/parser/scanner.h
M src/include/parser/scansup.h
M src/include/partitioning/partbounds.h
M src/include/partitioning/partdefs.h
M src/include/partitioning/partdesc.h
M src/include/partitioning/partprune.h
M src/include/pg_config_manual.h
M src/include/pg_getopt.h
M src/include/pg_trace.h
M src/include/pgstat.h
M src/include/pgtar.h
M src/include/pgtime.h
M src/include/port.h
M src/include/port/atomics.h
M src/include/port/atomics/arch-arm.h
M src/include/port/atomics/arch-hppa.h
M src/include/port/atomics/arch-ia64.h
M src/include/port/atomics/arch-ppc.h
M src/include/port/atomics/arch-x86.h
M src/include/port/atomics/fallback.h
M src/include/port/atomics/generic-acc.h
M src/include/port/atomics/generic-gcc.h
M src/include/port/atomics/generic-msvc.h
M src/include/port/atomics/generic-sunpro.h
M src/include/port/atomics/generic.h
M src/include/port/pg_bitutils.h
M src/include/port/pg_bswap.h
M src/include/port/pg_crc32c.h
M src/include/port/win32_port.h
M src/include/portability/instr_time.h
M src/include/portability/mem.h
M src/include/postgres.h
M src/include/postgres_fe.h
M src/include/postmaster/autovacuum.h
M src/include/postmaster/bgworker.h
M src/include/postmaster/bgworker_internals.h
M src/include/postmaster/bgwriter.h
M src/include/postmaster/fork_process.h
M src/include/postmaster/interrupt.h
M src/include/postmaster/pgarch.h
M src/include/postmaster/postmaster.h
M src/include/postmaster/startup.h
M src/include/postmaster/syslogger.h
M src/include/postmaster/walwriter.h
M src/include/regex/regexport.h
M src/include/replication/backup_manifest.h
M src/include/replication/basebackup.h
M src/include/replication/decode.h
M src/include/replication/logical.h
M src/include/replication/logicallauncher.h
M src/include/replication/logicalproto.h
M src/include/replication/logicalrelation.h
M src/include/replication/logicalworker.h
M src/include/replication/message.h
M src/include/replication/origin.h
M src/include/replication/output_plugin.h
M src/include/replication/pgoutput.h
M src/include/replication/reorderbuffer.h
M src/include/replication/slot.h
M src/include/replication/snapbuild.h
M src/include/replication/syncrep.h
M src/include/replication/walreceiver.h
M src/include/replication/walsender.h
M src/include/replication/walsender_private.h
M src/include/replication/worker_internal.h
M src/include/rewrite/prs2lock.h
M src/include/rewrite/rewriteDefine.h
M src/include/rewrite/rewriteHandler.h
M src/include/rewrite/rewriteManip.h
M src/include/rewrite/rewriteRemove.h
M src/include/rewrite/rewriteSupport.h
M src/include/rewrite/rowsecurity.h
M src/include/rusagestub.h
M src/include/snowball/header.h
M src/include/statistics/extended_stats_internal.h
M src/include/statistics/statistics.h
M src/include/storage/backendid.h
M src/include/storage/barrier.h
M src/include/storage/block.h
M src/include/storage/buf.h
M src/include/storage/buf_internals.h
M src/include/storage/buffile.h
M src/include/storage/bufmgr.h
M src/include/storage/bufpage.h
M src/include/storage/checksum.h
M src/include/storage/checksum_impl.h
M src/include/storage/condition_variable.h
M src/include/storage/copydir.h
M src/include/storage/dsm.h
M src/include/storage/dsm_impl.h
M src/include/storage/fd.h
M src/include/storage/freespace.h
M src/include/storage/fsm_internals.h
M src/include/storage/indexfsm.h
M src/include/storage/ipc.h
M src/include/storage/item.h
M src/include/storage/itemid.h
M src/include/storage/itemptr.h
M src/include/storage/large_object.h
M src/include/storage/latch.h
M src/include/storage/lmgr.h
M src/include/storage/lock.h
M src/include/storage/lockdefs.h
M src/include/storage/lwlock.h
M src/include/storage/md.h
M src/include/storage/off.h
M src/include/storage/pg_sema.h
M src/include/storage/pg_shmem.h
M src/include/storage/pmsignal.h
M src/include/storage/predicate.h
M src/include/storage/predicate_internals.h
M src/include/storage/proc.h
M src/include/storage/procarray.h
M src/include/storage/proclist.h
M src/include/storage/proclist_types.h
M src/include/storage/procsignal.h
M src/include/storage/reinit.h
M src/include/storage/relfilenode.h
M src/include/storage/s_lock.h
M src/include/storage/sharedfileset.h
M src/include/storage/shm_mq.h
M src/include/storage/shm_toc.h
M src/include/storage/shmem.h
M src/include/storage/sinval.h
M src/include/storage/sinvaladt.h
M src/include/storage/smgr.h
M src/include/storage/spin.h
M src/include/storage/standby.h
M src/include/storage/standbydefs.h
M src/include/storage/sync.h
M src/include/tcop/cmdtag.h
M src/include/tcop/cmdtaglist.h
M src/include/tcop/deparse_utility.h
M src/include/tcop/dest.h
M src/include/tcop/fastpath.h
M src/include/tcop/pquery.h
M src/include/tcop/tcopprot.h
M src/include/tcop/utility.h
M src/include/tsearch/dicts/regis.h
M src/include/tsearch/dicts/spell.h
M src/include/tsearch/ts_cache.h
M src/include/tsearch/ts_locale.h
M src/include/tsearch/ts_public.h
M src/include/tsearch/ts_type.h
M src/include/tsearch/ts_utils.h
M src/include/utils/acl.h
M src/include/utils/aclchk_internal.h
M src/include/utils/array.h
M src/include/utils/arrayaccess.h
M src/include/utils/ascii.h
M src/include/utils/attoptcache.h
M src/include/utils/builtins.h
M src/include/utils/bytea.h
M src/include/utils/catcache.h
M src/include/utils/combocid.h
M src/include/utils/date.h
M src/include/utils/datetime.h
M src/include/utils/datum.h
M src/include/utils/dsa.h
M src/include/utils/dynahash.h
M src/include/utils/elog.h
M src/include/utils/evtcache.h
M src/include/utils/expandeddatum.h
M src/include/utils/expandedrecord.h
M src/include/utils/float.h
M src/include/utils/fmgrtab.h
M src/include/utils/formatting.h
M src/include/utils/freepage.h
M src/include/utils/geo_decls.h
M src/include/utils/guc.h
M src/include/utils/guc_tables.h
M src/include/utils/help_config.h
M src/include/utils/hsearch.h
M src/include/utils/index_selfuncs.h
M src/include/utils/inet.h
M src/include/utils/int8.h
M src/include/utils/inval.h
M src/include/utils/json.h
M src/include/utils/jsonb.h
M src/include/utils/jsonfuncs.h
M src/include/utils/jsonpath.h
M src/include/utils/logtape.h
M src/include/utils/lsyscache.h
M src/include/utils/memdebug.h
M src/include/utils/memutils.h
M src/include/utils/multirangetypes.h
M src/include/utils/numeric.h
M src/include/utils/old_snapshot.h
M src/include/utils/palloc.h
M src/include/utils/partcache.h
M src/include/utils/pg_crc.h
M src/include/utils/pg_locale.h
M src/include/utils/pg_lsn.h
M src/include/utils/pg_rusage.h
M src/include/utils/pidfile.h
M src/include/utils/plancache.h
M src/include/utils/portal.h
M src/include/utils/queryenvironment.h
M src/include/utils/rangetypes.h
M src/include/utils/regproc.h
M src/include/utils/rel.h
M src/include/utils/relcache.h
M src/include/utils/relfilenodemap.h
M src/include/utils/relmapper.h
M src/include/utils/relptr.h
M src/include/utils/reltrigger.h
M src/include/utils/resowner.h
M src/include/utils/resowner_private.h
M src/include/utils/rls.h
M src/include/utils/ruleutils.h
M src/include/utils/sampling.h
M src/include/utils/selfuncs.h
M src/include/utils/sharedtuplestore.h
M src/include/utils/snapmgr.h
M src/include/utils/snapshot.h
M src/include/utils/sortsupport.h
M src/include/utils/spccache.h
M src/include/utils/syscache.h
M src/include/utils/timeout.h
M src/include/utils/timestamp.h
M src/include/utils/tuplesort.h
M src/include/utils/tuplestore.h
M src/include/utils/typcache.h
M src/include/utils/tzparser.h
M src/include/utils/uuid.h
M src/include/utils/varbit.h
M src/include/utils/varlena.h
M src/include/utils/xid8.h
M src/include/utils/xml.h
M src/include/windowapi.h
M src/interfaces/ecpg/compatlib/Makefile
M src/interfaces/ecpg/ecpglib/Makefile
M src/interfaces/ecpg/pgtypeslib/Makefile
M src/interfaces/ecpg/preproc/Makefile
M src/interfaces/ecpg/preproc/c_kwlist.h
M src/interfaces/ecpg/preproc/check_rules.pl
M src/interfaces/ecpg/preproc/ecpg.c
M src/interfaces/ecpg/preproc/ecpg_kwlist.h
M src/interfaces/ecpg/preproc/keywords.c
M src/interfaces/ecpg/preproc/parse.pl
M src/interfaces/ecpg/preproc/parser.c
M src/interfaces/ecpg/preproc/pgc.l
M src/interfaces/ecpg/test/pg_regress_ecpg.c
M src/interfaces/libpq/Makefile
M src/interfaces/libpq/fe-auth-scram.c
M src/interfaces/libpq/fe-auth.c
M src/interfaces/libpq/fe-auth.h
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-gssapi-common.c
M src/interfaces/libpq/fe-gssapi-common.h
M src/interfaces/libpq/fe-lobj.c
M src/interfaces/libpq/fe-misc.c
M src/interfaces/libpq/fe-print.c
M src/interfaces/libpq/fe-protocol2.c
M src/interfaces/libpq/fe-protocol3.c
M src/interfaces/libpq/fe-secure-common.c
M src/interfaces/libpq/fe-secure-common.h
M src/interfaces/libpq/fe-secure-gssapi.c
M src/interfaces/libpq/fe-secure-openssl.c
M src/interfaces/libpq/fe-secure.c
M src/interfaces/libpq/legacy-pqsignal.c
M src/interfaces/libpq/libpq-events.c
M src/interfaces/libpq/libpq-events.h
M src/interfaces/libpq/libpq-fe.h
M src/interfaces/libpq/libpq-int.h
M src/interfaces/libpq/pqexpbuffer.c
M src/interfaces/libpq/pqexpbuffer.h
M src/interfaces/libpq/pthread-win32.c
M src/interfaces/libpq/test/uri-regress.c
M src/interfaces/libpq/win32.c
M src/pl/plperl/plperl.h
M src/pl/plpgsql/src/generate-plerrcodes.pl
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_funcs.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/pl_handler.c
M src/pl/plpgsql/src/pl_reserved_kwlist.h
M src/pl/plpgsql/src/pl_scanner.c
M src/pl/plpgsql/src/pl_unreserved_kwlist.h
M src/pl/plpgsql/src/plpgsql.h
M src/pl/plpython/generate-spiexceptions.pl
M src/pl/plpython/plpython.h
M src/pl/tcl/generate-pltclerrcodes.pl
M src/port/chklocale.c
M src/port/dirent.c
M src/port/dirmod.c
M src/port/dlopen.c
M src/port/erand48.c
M src/port/explicit_bzero.c
M src/port/fls.c
M src/port/getaddrinfo.c
M src/port/getpeereid.c
M src/port/getrusage.c
M src/port/kill.c
M src/port/link.c
M src/port/mkdtemp.c
M src/port/noblock.c
M src/port/open.c
M src/port/path.c
M src/port/pg_bitutils.c
M src/port/pg_crc32c_armv8.c
M src/port/pg_crc32c_armv8_choose.c
M src/port/pg_crc32c_sb8.c
M src/port/pg_crc32c_sse42.c
M src/port/pg_crc32c_sse42_choose.c
M src/port/pg_strong_random.c
M src/port/pgcheckdir.c
M src/port/pgsleep.c
M src/port/pgstrcasecmp.c
M src/port/pgstrsignal.c
M src/port/pqsignal.c
M src/port/pread.c
M src/port/pwrite.c
M src/port/quotes.c
M src/port/random.c
M src/port/setenv.c
M src/port/snprintf.c
M src/port/srandom.c
M src/port/strerror.c
M src/port/strlcpy.c
M src/port/strnlen.c
M src/port/strtof.c
M src/port/system.c
M src/port/thread.c
M src/port/unsetenv.c
M src/port/win32env.c
M src/port/win32error.c
M src/port/win32security.c
M src/port/win32setlocale.c
M src/port/win32stat.c
M src/port/win32ver.rc
M src/test/authentication/Makefile
M src/test/examples/testlo.c
M src/test/examples/testlo64.c
M src/test/isolation/isolation_main.c
M src/test/isolation/isolationtester.h
M src/test/isolation/specparse.y
M src/test/isolation/specscanner.l
M src/test/kerberos/Makefile
M src/test/ldap/Makefile
M src/test/modules/delay_execution/delay_execution.c
M src/test/modules/dummy_index_am/dummy_index_am.c
M src/test/modules/dummy_seclabel/dummy_seclabel.c
M src/test/modules/plsample/plsample.c
M src/test/modules/test_bloomfilter/test_bloomfilter.c
M src/test/modules/test_ddl_deparse/test_ddl_deparse.c
M src/test/modules/test_ginpostinglist/test_ginpostinglist.c
M src/test/modules/test_integerset/test_integerset.c
M src/test/modules/test_parser/test_parser.c
M src/test/modules/test_predtest/test_predtest.c
M src/test/modules/test_rbtree/test_rbtree.c
M src/test/modules/test_rls_hooks/test_rls_hooks.c
M src/test/modules/test_rls_hooks/test_rls_hooks.h
M src/test/modules/test_shm_mq/setup.c
M src/test/modules/test_shm_mq/test.c
M src/test/modules/test_shm_mq/test_shm_mq.h
M src/test/modules/test_shm_mq/worker.c
M src/test/modules/worker_spi/worker_spi.c
M src/test/perl/Makefile
M src/test/recovery/Makefile
M src/test/regress/GNUmakefile
M src/test/regress/pg_regress.c
M src/test/regress/pg_regress.h
M src/test/regress/pg_regress_main.c
M src/test/regress/regress.c
M src/test/ssl/Makefile
M src/test/subscription/Makefile
M src/timezone/pgtz.c
M src/timezone/pgtz.h
M src/tools/PerfectHash.pm
M src/tools/check_bison_recursion.pl
M src/tools/copyright.pl
M src/tools/findoidjoins/Makefile
M src/tools/findoidjoins/findoidjoins.c
M src/tools/fix-old-flex-code.pl
M src/tools/gen_keywordlist.pl
M src/tools/ifaddrs/Makefile
M src/tools/pginclude/cpluspluscheck
M src/tools/pginclude/headerscheck
M src/tools/testint128.c
M src/tools/version_stamp.pl
M src/tools/win32tzlist.pl
M src/tutorial/complex.source
M src/tutorial/syscat.source
Doc: improve explanation of EXTRACT(EPOCH) for timestamp without tz.
commit : 4d3f03f42227bb351c2021a9ccea2fff9c023cfc
author : Tom Lane <[email protected]>
date : Fri, 1 Jan 2021 15:51:09 -0500
committer: Tom Lane <[email protected]>
date : Fri, 1 Jan 2021 15:51:09 -0500
Try to be clearer about what computation is actually happening here.
Per bug #16797 from Dana Burd.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
Get heap page max offset with buffer lock held.
commit : 32d6287d2eef6b6a4dde07e0513f3e4f321792ad
author : Peter Geoghegan <[email protected]>
date : Wed, 30 Dec 2020 17:21:42 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 30 Dec 2020 17:21:42 -0800
On further reflection it seems better to call PageGetMaxOffsetNumber()
after acquiring a buffer lock on the page. This shouldn't really
matter, but doing it this way is cleaner.
Follow-up to commit 42288174.
Backpatch: 12-, just like commit 42288174
M src/backend/access/heap/heapam.c
Fix index deletion latestRemovedXid bug.
commit : 422881744997417944634a7f84af7a66a608de9a
author : Peter Geoghegan <[email protected]>
date : Wed, 30 Dec 2020 16:29:05 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 30 Dec 2020 16:29:05 -0800
The logic for determining the latest removed XID for the purposes of
generating recovery conflicts in REDO routines was subtly broken. It
failed to follow links from HOT chains, and so failed to consider all
relevant heap tuple headers in some cases.
To fix, expand the loop that deals with LP_REDIRECT line pointers to
also deal with HOT chains. The new version of the loop is loosely based
on a similar loop from heap_prune_chain().
The impact of this bug is probably quite limited, since the horizon code
necessarily deals with heap tuples that are pointed to by LP_DEAD-set
index tuples. The process of setting LP_DEAD index tuples (e.g. within
the kill_prior_tuple mechanism) is highly correlated with opportunistic
pruning of pointed-to heap tuples. Plus the question of generating a
recovery conflict usually comes up some time after index tuple LP_DEAD
bits were initially set, unlike heap pruning, where a latestRemovedXid
is generated at the point of the pruning operation (heap pruning has no
deferred "would-be page split" style processing that produces conflicts
lazily).
Only backpatch to Postgres 12, the first version where this logic runs
during original execution (following commit 558a9165e08). The index
latestRemovedXid mechanism has had the same bug since it first appeared
over 10 years ago (in commit a760893d), but backpatching to all
supported versions now seems like a bad idea on balance. Running the
new improved code during recovery seems risky, especially given the lack
of complaints from the field.
Author: Peter Geoghegan <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wz=Eib393+HHcERK_9MtgNS7Ew1HY=RDC_g6GL46zM5C6Q@mail.gmail.com
Backpatch: 12-
M src/backend/access/heap/heapam.c
M src/backend/storage/ipc/standby.c
Doc: spell out comparison behaviors for the date/time types.
commit : 319f4d54e82d15d4a0c3f4cc1328c40dba024b5c
author : Tom Lane <[email protected]>
date : Wed, 30 Dec 2020 17:48:43 -0500
committer: Tom Lane <[email protected]>
date : Wed, 30 Dec 2020 17:48:43 -0500
The behavior of cross-type comparisons among date/time data types was
not really explained anywhere. You could probably infer it if you
recognized the applicability of comments elsewhere about datatype
conversions, but it seems worthy of explicit documentation.
Per bug #16797 from Dana Burd.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
More fixups for pg_upgrade cross-version tests.
commit : 091866724cb3ee7251fa56e2517248c4b7796ca8
author : Tom Lane <[email protected]>
date : Wed, 30 Dec 2020 14:15:41 -0500
committer: Tom Lane <[email protected]>
date : Wed, 30 Dec 2020 14:15:41 -0500
Commit 7ca37fb04 removed regress_putenv from the regress.so library,
so reloading a SQL function dependent on that would not work.
Fix similarly to 52202bb39.
Per buildfarm.
M src/bin/pg_upgrade/test.sh
Refactor multirange_in()
commit : 16d531a30a120d13cc3b460fba6570024a1fcfa8
author : Alexander Korotkov <[email protected]>
date : Wed, 30 Dec 2020 21:11:31 +0300
committer: Alexander Korotkov <[email protected]>
date : Wed, 30 Dec 2020 21:11:31 +0300
This commit preserves the logic of multirange_in() but makes it more clear
what's going on. Also, this commit fixes the compiler warning spotted by the
buildfarm.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/2246043.1609290699%40sss.pgh.pa.us
M src/backend/utils/adt/multirangetypes.c
Use setenv() in preference to putenv().
commit : 7ca37fb0406bc2cbbd864a2ffdbdb4479e338c0c
author : Tom Lane <[email protected]>
date : Wed, 30 Dec 2020 12:55:59 -0500
committer: Tom Lane <[email protected]>
date : Wed, 30 Dec 2020 12:55:59 -0500
Since at least 2001 we've used putenv() and avoided setenv(), on the
grounds that the latter was unportable and not in POSIX. However,
POSIX added it that same year, and by now the situation has reversed:
setenv() is probably more portable than putenv(), since POSIX now
treats the latter as not being a core function. And setenv() has
cleaner semantics too. So, let's reverse that old policy.
This commit adds a simple src/port/ implementation of setenv() for
any stragglers (we have one in the buildfarm, but I'd not be surprised
if that code is never used in the field). More importantly, extend
win32env.c to also support setenv(). Then, replace usages of putenv()
with setenv(), and get rid of some ad-hoc implementations of setenv()
wannabees.
Also, adjust our src/port/ implementation of unsetenv() to follow the
POSIX spec that it returns an error indicator, rather than returning
void as per the ancient BSD convention. I don't feel a need to make
all the call sites check for errors, but the portability stub ought
to match real-world practice.
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
M contrib/dblink/input/paths.source
M contrib/dblink/output/paths.source
M src/backend/utils/adt/pg_locale.c
M src/bin/initdb/initdb.c
M src/bin/pg_ctl/pg_ctl.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/util.c
M src/bin/psql/command.c
M src/common/exec.c
M src/common/restricted_token.c
M src/include/pg_config.h.in
M src/include/port.h
M src/include/port/win32_port.h
M src/interfaces/ecpg/test/pg_regress_ecpg.c
A src/port/setenv.c
M src/port/unsetenv.c
M src/port/win32env.c
M src/test/isolation/isolation_main.c
M src/test/regress/pg_regress.c
M src/test/regress/pg_regress_main.c
M src/test/regress/regress.c
M src/tools/msvc/Solution.pm
Fix selectivity estimation @> (anymultirange, anyrange) operator
commit : 62097a4cc8c725fa86d3170396a8f30609acd0d3
author : Alexander Korotkov <[email protected]>
date : Wed, 30 Dec 2020 20:19:15 +0300
committer: Alexander Korotkov <[email protected]>
date : Wed, 30 Dec 2020 20:19:15 +0300
Attempt to get selectivity estimation for @> (anymultirange, anyrange) operator
caused an error in buildfarm, because this operator was missed in switch()
of calc_hist_selectivity(). Fix that and also make regression tests reliably
check that selectivity estimation for (multi)ranges doesn't fall. Previously,
whether we test selectivity estimation for (multi)ranges depended on
whether autovacuum managed to gather concurrently to the test.
Reported-by: Michael Paquier
Discussion: https://postgr.es/m/X%2BwmgjRItuvHNBeV%40paquier.xyz
M src/backend/utils/adt/multirangetypes_selfuncs.c
M src/test/regress/expected/multirangetypes.out
M src/test/regress/expected/rangetypes.out
M src/test/regress/sql/multirangetypes.sql
M src/test/regress/sql/rangetypes.sql
Fix up usage of krb_server_keyfile GUC parameter.
commit : 860fe27ee1e2a4a1c36c2f874c37656533cccce9
author : Tom Lane <[email protected]>
date : Wed, 30 Dec 2020 11:38:42 -0500
committer: Tom Lane <[email protected]>
date : Wed, 30 Dec 2020 11:38:42 -0500
secure_open_gssapi() installed the krb_server_keyfile setting as
KRB5_KTNAME unconditionally, so long as it's not empty. However,
pg_GSS_recvauth() only installed it if KRB5_KTNAME wasn't set already,
leading to a troubling inconsistency: in theory, clients could see
different sets of server principal names depending on whether they
use GSSAPI encryption. Always using krb_server_keyfile seems like
the right thing, so make both places do that. Also fix up
secure_open_gssapi()'s lack of a check for setenv() failure ---
it's unlikely, surely, but security-critical actions are no place
to be sloppy.
Also improve the associated documentation.
This patch does nothing about secure_open_gssapi()'s use of setenv(),
and indeed causes pg_GSS_recvauth() to use it too. That's nominally
against project portability rules, but since this code is only built
with --with-gssapi, I do not feel a need to do something about this
in the back branches. A fix will be forthcoming for HEAD though.
Back-patch to v12 where GSSAPI encryption was introduced. The
dubious behavior in pg_GSS_recvauth() goes back further, but it
didn't have anything to be inconsistent with, so let it be.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/config.sgml
M src/backend/libpq/auth.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/utils/misc/postgresql.conf.sample
Sanitize IF NOT EXISTS in EXPLAIN for CTAS and matviews
commit : e665769e6d1e84b6650f53ed297058fc11152f7f
author : Michael Paquier <[email protected]>
date : Wed, 30 Dec 2020 21:23:24 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 30 Dec 2020 21:23:24 +0900
IF NOT EXISTS was ignored when specified in an EXPLAIN query for CREATE
MATERIALIZED VIEW or CREATE TABLE AS. Hence, if this clause was
specified, the caller would get a failure if the relation already
exists instead of a success with a NOTICE message.
This commit makes the behavior of IF NOT EXISTS in EXPLAIN consistent
with the non-EXPLAIN'd DDL queries, preventing a failure with IF NOT
EXISTS if the relation to-be-created already exists. The skip is done
before the SELECT query used for the relation is planned or executed,
and a "dummy" plan is generated instead depending on the format used by
EXPLAIN.
Author: Bharath Rupireddy
Reviewed-by: Zhijie Hou, Michael Paquier
Discussion: https://postgr.es/m/CALj2ACVa3oJ9O_wcGd+FtHWZds04dEKcakxphGz5POVgD4wC7Q@mail.gmail.com
M src/backend/commands/createas.c
M src/backend/commands/explain.c
M src/include/commands/createas.h
M src/test/regress/expected/matview.out
M src/test/regress/expected/select_into.out
M src/test/regress/sql/matview.sql
M src/test/regress/sql/select_into.sql
Extend the output plugin API to allow decoding of prepared xacts.
commit : 0aa8a01d04c8fe200b7a106878eebc3d0af9105c
author : Amit Kapila <[email protected]>
date : Wed, 30 Dec 2020 16:17:26 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 30 Dec 2020 16:17:26 +0530
This adds six methods to the output plugin API, adding support for
streaming changes of two-phase transactions at prepare time.
* begin_prepare
* filter_prepare
* prepare
* commit_prepared
* rollback_prepared
* stream_prepare
Most of this is a simple extension of the existing methods, with the
semantic difference that the transaction is not yet committed and maybe
aborted later.
Until now two-phase transactions were translated into regular transactions
on the subscriber, and the GID was not forwarded to it. None of the
two-phase commands were communicated to the subscriber.
This patch provides the infrastructure for logical decoding plugins to be
informed of two-phase commands Like PREPARE TRANSACTION, COMMIT PREPARED
and ROLLBACK PREPARED commands with the corresponding GID.
This also extends the 'test_decoding' plugin, implementing these new
methods.
This commit simply adds these new APIs and the upcoming patch to "allow
the decoding at prepare time in ReorderBuffer" will use these APIs.
Author: Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Peter Smith, Sawada Masahiko, and Dilip Kumar
Discussion:
https://postgr.es/m/[email protected]
https://postgr.es/m/CAMGcDxeqEpWj3fTXwqhSwBdXd2RS9jzwWscO-XbeCfso6ts3+Q@mail.gmail.com
M contrib/test_decoding/test_decoding.c
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/logical.c
M src/include/replication/logical.h
M src/include/replication/output_plugin.h
M src/include/replication/reorderbuffer.h
M src/tools/pgindent/typedefs.list
In pg_upgrade cross-version test, handle postfix operators.
commit : fa744697c79189a661f802d9a979d959b4454df0
author : Noah Misch <[email protected]>
date : Wed, 30 Dec 2020 01:43:43 -0800
committer: Noah Misch <[email protected]>
date : Wed, 30 Dec 2020 01:43:43 -0800
Commit 1ed6b895634ce0dc5fd4bd040e87252b32182cba eliminated support for
them, so drop them from regression databases before upgrading. This is
necessary but not sufficient for testing v13 -> v14 upgrades.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_upgrade/test.sh
In pg_upgrade cross-version test, handle lack of oldstyle_length().
commit : 52202bb396b1e96c43bfd767d6e434b1c6fd2ae1
author : Noah Misch <[email protected]>
date : Wed, 30 Dec 2020 01:43:43 -0800
committer: Noah Misch <[email protected]>
date : Wed, 30 Dec 2020 01:43:43 -0800
This suffices for testing v12 -> v13; some other version pairs need more
changes. Back-patch to v10, which removed the function.
M src/bin/pg_upgrade/test.sh
Remove references to libpq_srcdir in adminpack and old_snapshot
commit : 107a2d4204ff4bf4ce05e3525f0d94fc0bd497ff
author : Michael Paquier <[email protected]>
date : Wed, 30 Dec 2020 12:38:16 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 30 Dec 2020 12:38:16 +0900
Those two modules included references to libpq's source path, without
using anything from libpq. Some copy-pastos done when each module was
created are likely at the origin of those useless references (aecf5ee
for old_snapshot, fe59e56 for adminpack).
Reviewed-by: Tom Lane, David Rowley
Discussion: https://postgr.es/m/[email protected]
M contrib/adminpack/Makefile
M contrib/old_snapshot/Makefile
Doc: fix up PDF build warnings from over-width table columns.
commit : f20dc2c8dd50a5c738d535205d5d44bff82d3f75
author : Tom Lane <[email protected]>
date : Tue, 29 Dec 2020 20:44:03 -0500
committer: Tom Lane <[email protected]>
date : Tue, 29 Dec 2020 20:44:03 -0500
Addition of multirange info to tables 8.27 and 65.1 made them start
throwing "exceed the available area" warnings in PDF docs builds.
For 8.27, twiddling the existing column width hints was enough to
fix this. For 65.1, I twiddled the widths a little, but to really
fix it I had to insert a space after each comma in the table, to
allow a line break to occur there. (This seemed easier to read
and maintain than the alternative of inserting &zwsp; entities.)
Per buildfarm.
M doc/src/sgml/datatype.sgml
M doc/src/sgml/gist.sgml
Suppress log spam from multiple reports of SIGQUIT shutdown.
commit : 1f9158ba48122fa232db955a2ee324eec1848ba9
author : Tom Lane <[email protected]>
date : Tue, 29 Dec 2020 18:02:38 -0500
committer: Tom Lane <[email protected]>
date : Tue, 29 Dec 2020 18:02:38 -0500
When the postmaster sends SIGQUIT to its children, there's no real
need for all the children to log that fact; the postmaster already
made a log entry about it, so adding perhaps dozens or hundreds of
child-process log entries adds nothing of value. So, let's introduce
a new ereport level to specify "WARNING, but never send to log" and
use that for these messages.
Such a change wouldn't have been desirable before commit 7e784d1dc,
because if someone manually SIGQUIT's a backend, we *do* want to log
that. But now we can tell the difference between a signal that was
issued by the postmaster and one that was not with reasonable
certainty.
While we're here, also clear error_context_stack before ereport'ing,
to prevent error callbacks from being invoked in the signal-handler
context. This should reduce the odds of getting hung up while trying
to notify the client.
Per a suggestion from Andres Freund.
Discussion: https://postgr.es/m/[email protected]
M src/backend/tcop/postgres.c
M src/backend/utils/error/elog.c
M src/include/utils/elog.h
Add support of multirange matching to the existing range GiST indexes
commit : db6335b5b1d6654b0e3104f36817800d127c1c91
author : Alexander Korotkov <[email protected]>
date : Tue, 29 Dec 2020 23:36:43 +0300
committer: Alexander Korotkov <[email protected]>
date : Tue, 29 Dec 2020 23:36:43 +0300
6df7a9698b has introduced a set of operators between ranges and multiranges.
Existing GiST indexes for ranges could easily support majority of them.
This commit adds support for new operators to the existing range GiST indexes.
New operators resides the same strategy numbers as existing ones. Appropriate
check function is determined using the subtype.
Catversion is bumped.
M doc/src/sgml/gist.sgml
M doc/src/sgml/rangetypes.sgml
M src/backend/utils/adt/multirangetypes.c
M src/backend/utils/adt/rangetypes_gist.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_proc.dat
M src/include/utils/multirangetypes.h
M src/test/regress/expected/multirangetypes.out
M src/test/regress/expected/rangetypes.out
M src/test/regress/sql/multirangetypes.sql
M src/test/regress/sql/rangetypes.sql
Improve the signature of internal multirange functions
commit : d1d61a8b23b604faf797695eeacaa5da4fe64762
author : Alexander Korotkov <[email protected]>
date : Tue, 29 Dec 2020 23:35:38 +0300
committer: Alexander Korotkov <[email protected]>
date : Tue, 29 Dec 2020 23:35:38 +0300
There is a set of *_internal() functions exposed in
include/utils/multirangetypes.h. This commit improves the signatures of these
functions in two ways.
* Add const qualifies where applicable.
* Replace multirange typecache argument with range typecache argument.
Multirange typecache was used solely to find the range typecache. At the
same time, range typecache is easier for the caller to find.
M src/backend/utils/adt/multirangetypes.c
M src/include/utils/multirangetypes.h
Implement operators for checking if the range contains a multirange
commit : 4d7684cc754f312aee468abb83ca4f7da94b30a3
author : Alexander Korotkov <[email protected]>
date : Tue, 29 Dec 2020 23:35:33 +0300
committer: Alexander Korotkov <[email protected]>
date : Tue, 29 Dec 2020 23:35:33 +0300
We have operators for checking if the multirange contains a range but don't
have the opposite. This commit improves completeness of the operator set by
adding two new operators: @> (anyrange,anymultirange) and
<@(anymultirange,anyrange).
Catversion is bumped.
M doc/src/sgml/func.sgml
M src/backend/utils/adt/multirangetypes.c
M src/backend/utils/adt/multirangetypes_selfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_proc.dat
M src/include/utils/multirangetypes.h
M src/test/regress/expected/multirangetypes.out
M src/test/regress/sql/multirangetypes.sql
Fix bugs in comparison functions for multirange_bsearch_match()
commit : a5b81b6f0006ea0b502780ce7f73d295a225842c
author : Alexander Korotkov <[email protected]>
date : Tue, 29 Dec 2020 23:35:26 +0300
committer: Alexander Korotkov <[email protected]>
date : Tue, 29 Dec 2020 23:35:26 +0300
Two functions multirange_range_overlaps_bsearch_comparison() and
multirange_range_contains_bsearch_comparison() contain bugs of returning -1
instead of 1. This commit fixes these bugs and adds corresponding regression
tests.
M src/backend/utils/adt/multirangetypes.c
M src/test/regress/expected/multirangetypes.out
M src/test/regress/sql/multirangetypes.sql
doc: Improve description of min_dynamic_shared_memory
commit : 1b3433e25f81d6fc4f231887ab965e1ea1bcb47e
author : Michael Paquier <[email protected]>
date : Tue, 29 Dec 2020 16:49:14 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 29 Dec 2020 16:49:14 +0900
While on it, fix one oversight in 90fbf7c, that introduced a reference
to an incorrect value for the compression level of pg_dump.
Author: Justin Pryzby
Reviewed-by: Thomas Munro, Michael Paquier
Discussion: https://postgr.es/m/CA+hUKGJRTLWWPcQfjm_xaOk98M8aROK903X92O0x-4vLJPWrrA@mail.gmail.com
M doc/src/sgml/config.sgml
M src/bin/pg_dump/pg_backup_archiver.h
Improve log messages related to pg_hba.conf not matching a connection.
commit : 3995c424984e991b1069a2869af972dc07574c0b
author : Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 17:58:58 -0500
committer: Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 17:58:58 -0500
Include details on whether GSS encryption has been activated;
since we added "hostgssenc" type HBA entries, that's relevant info.
Kyotaro Horiguchi and Tom Lane. Back-patch to v12 where
GSS encryption was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/auth.c
Fix assorted issues in backend's GSSAPI encryption support.
commit : 622ae4621ece72a9f64b5602c74d7aaf373c1631
author : Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 17:44:17 -0500
committer: Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 17:44:17 -0500
Unrecoverable errors detected by GSSAPI encryption can't just be
reported with elog(ERROR) or elog(FATAL), because attempting to
send the error report to the client is likely to lead to infinite
recursion or loss of protocol sync. Instead make this code do what
the SSL encryption code has long done, which is to just report any
such failure to the server log (with elevel COMMERROR), then pretend
we've lost the connection by returning errno = ECONNRESET.
Along the way, fix confusion about whether message translation is done
by pg_GSS_error() or its callers (the latter should do it), and make
the backend version of that function work more like the frontend
version.
Avoid allocating the port->gss struct until it's needed; we surely
don't need to allocate it in the postmaster.
Improve logging of "connection authorized" messages with GSS enabled.
(As part of this, I back-patched the code changes from dc11f31a1.)
Make BackendStatusShmemSize() account for the GSS-related space that
will be allocated by CreateSharedBackendStatus(). This omission
could possibly cause out-of-shared-memory problems with very high
max_connections settings.
Remove arbitrary, pointless restriction that only GSS authentication
can be used on a GSS-encrypted connection.
Improve documentation; notably, document the fact that libpq now
prefers GSS encryption over SSL encryption if both are possible.
Per report from Mikael Gustavsson. Back-patch to v12 where
this code was introduced.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/runtime.sgml
M src/backend/libpq/auth.c
M src/backend/libpq/be-gssapi-common.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/libpq/be-secure.c
M src/backend/libpq/hba.c
M src/backend/libpq/pqcomm.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/utils/init/postinit.c
M src/include/libpq/be-gssapi-common.h
M src/include/libpq/libpq-be.h
Fix bugs in libpq's GSSAPI encryption support.
commit : ff6ce9a3a691a96e8e47ed449bc51c5a178e6931
author : Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 15:43:44 -0500
committer: Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 15:43:44 -0500
The critical issue fixed here is that if a GSSAPI-encrypted connection
is successfully made, pqsecure_open_gss() cleared conn->allow_ssl_try,
as an admittedly-hacky way of preventing us from then trying to tunnel
SSL encryption over the already-encrypted connection. The problem
with that is that if we abandon the GSSAPI connection because of a
failure during authentication, we would not attempt SSL encryption
in the next try with the same server. This can lead to unexpected
connection failure, or silently getting a non-encrypted connection
where an encrypted one is expected.
Fortunately, we'd only manage to make a GSSAPI-encrypted connection
if both client and server hold valid tickets in the same Kerberos
infrastructure, which is a relatively uncommon environment.
Nonetheless this is a very nasty bug with potential security
consequences. To fix, don't reset the flag, instead adding a
check for conn->gssenc being already true when deciding whether
to try to initiate SSL.
While here, fix some lesser issues in libpq's GSSAPI code:
* Use the need_new_connection stanza when dropping an attempted
GSSAPI connection, instead of partially duplicating that code.
The consequences of this are pretty minor: AFAICS it could only
lead to auth_req_received or password_needed remaining set when
they shouldn't, which is not too harmful.
* Fix pg_GSS_error() to not repeat the "mprefix" it's given multiple
times, and to notice any failure return from gss_display_status().
* Avoid gratuitous dependency on NI_MAXHOST in
pg_GSS_load_servicename().
Per report from Mikael Gustavsson. Back-patch to v12 where
this code was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-gssapi-common.c
M src/interfaces/libpq/fe-secure-gssapi.c
Expose the default for channel_binding in PQconndefaults().
commit : cf61b0734c61d93c62827fe4e44fa2162a533b8e
author : Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 12:13:40 -0500
committer: Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 12:13:40 -0500
If there's a static default value for a connection option,
it should be shown in the PQconninfoOptions array.
Daniele Varrazzo
Discussion: https://postgr.es/m/CA+mi_8Zo8Rgn7p+6ZRY7QdDu+23ukT9AvoHNyPbgKACxwgGhZA@mail.gmail.com
M src/interfaces/libpq/fe-connect.c
Further fix thinko in plpgsql memory leak fix.
commit : 5f2e09bcccd771629fb7a2885f8c468ae0f7fb33
author : Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 11:55:23 -0500
committer: Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 11:55:23 -0500
There's a second call of get_eval_mcontext() that should also be
get_stmt_mcontext(). This is actually dead code, since no
interesting allocations happen before switching back to the
original context, but we should keep it in sync with the other
call to forestall possible future bugs.
Discussion: https://postgr.es/m/[email protected]
M src/pl/plpgsql/src/pl_exec.c
Fix thinko in plpgsql memory leak fix.
commit : ea80d8d9437e80de6506dbfe3765d834653312bf
author : Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 11:41:25 -0500
committer: Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 11:41:25 -0500
Commit a6b1f5365 intended to place the transient "target" list of
a CALL statement in the function's statement-lifespan context,
but I fat-fingered that and used get_eval_mcontext() instead of
get_stmt_mcontext(). The eval_mcontext belongs to the "simple
expression" infrastructure, which is destroyed at transaction end.
The net effect is that a CALL in a procedure to another procedure
that has OUT or INOUT parameters would fail if the called procedure
did a COMMIT.
Per report from Peter Eisentraut. Back-patch to v11, like the
prior patch.
Discussion: https://postgr.es/m/[email protected]
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_call.sql
Fix inconsistent code with shared invalidations of snapshots
commit : 643428c54b95d472c6d949c3c3c11f347e371a2b
author : Michael Paquier <[email protected]>
date : Mon, 28 Dec 2020 22:16:49 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 28 Dec 2020 22:16:49 +0900
The code in charge of processing a single invalidation message has been
using since 568d413 the structure for relation mapping messages. This
had fortunately no consequence as both locate the database ID at the
same location, but it could become a problem in the future if this area
of the code changes.
Author: Konstantin Knizhnik
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M src/backend/utils/cache/inval.c
postgres_fdw: Fix connection leak.
commit : e3ebcca843a4703938b9f40a4811f43c4b315753
author : Fujii Masao <[email protected]>
date : Mon, 28 Dec 2020 19:56:13 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 28 Dec 2020 19:56:13 +0900
In postgres_fdw, the cached connections to foreign servers will not be
closed until the local session exits if the user mappings or foreign servers
that those connections depend on are dropped. Those connections can be
leaked.
To fix that connection leak issue, after a change to a pg_foreign_server
or pg_user_mapping catalog entry, this commit makes postgres_fdw close
the connections depending on that entry immediately if current
transaction has not used those connections yet. Otherwise, mark those
connections as invalid and then close them at the end of current transaction,
since they cannot be closed in the midst of the transaction using them.
Closed connections will be remade at the next opportunity if necessary.
Back-patch to all supported branches.
Author: Bharath Rupireddy
Reviewed-by: Zhihong Yu, Zhijie Hou, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVNcGH_6qLY-4_tXz8JLvA+4yeBThRfxMz7Oxbk1aHcpQ@mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Revert "Add key management system" (978f869b99) & later commits
commit : 3187ef7c46c5b884267a88f2d6119c9a05f1bbba
author : Bruce Momjian <[email protected]>
date : Sun, 27 Dec 2020 21:37:42 -0500
committer: Bruce Momjian <[email protected]>
date : Sun, 27 Dec 2020 21:37:42 -0500
The patch needs test cases, reorganization, and cfbot testing.
Technically reverts commits 5c31afc49d..e35b2bad1a (exclusive/inclusive)
and 08db7c63f3..ccbe34139b.
Reported-by: Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
D doc/src/sgml/database-encryption.sgml
M doc/src/sgml/filelist.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/postgres.sgml
M doc/src/sgml/ref/allfiles.sgml
M doc/src/sgml/ref/initdb.sgml
D doc/src/sgml/ref/pg_alterckey.sgml
M doc/src/sgml/ref/pg_ctl-ref.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/postgres-ref.sgml
M doc/src/sgml/reference.sgml
M doc/src/sgml/storage.sgml
M src/backend/Makefile
M src/backend/access/transam/xlog.c
M src/backend/bootstrap/bootstrap.c
D src/backend/crypto/Makefile
D src/backend/crypto/ckey_aws.sh.sample
D src/backend/crypto/ckey_direct.sh.sample
D src/backend/crypto/ckey_passphrase.sh.sample
D src/backend/crypto/ckey_piv_nopin.sh.sample
D src/backend/crypto/ckey_piv_pin.sh.sample
D src/backend/crypto/kmgr.c
D src/backend/crypto/ssl_passphrase.sh.sample
M src/backend/libpq/be-secure-common.c
M src/backend/main/main.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/replication/basebackup.c
M src/backend/storage/ipc/ipci.c
M src/backend/storage/lmgr/lwlocknames.txt
M src/backend/tcop/postgres.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/pg_controldata.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/Makefile
M src/bin/initdb/initdb.c
D src/bin/pg_alterckey/.gitignore
D src/bin/pg_alterckey/Makefile
D src/bin/pg_alterckey/pg_alterckey.c
M src/bin/pg_controldata/pg_controldata.c
M src/bin/pg_ctl/pg_ctl.c
M src/bin/pg_resetwal/pg_resetwal.c
M src/bin/pg_rewind/filemap.c
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/file.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/server.c
M src/common/Makefile
D src/common/cipher.c
D src/common/cipher_openssl.c
D src/common/kmgr_utils.c
M src/include/catalog/pg_control.h
D src/include/common/cipher.h
D src/include/common/kmgr_utils.h
D src/include/crypto/kmgr.h
M src/include/pgstat.h
M src/include/postmaster/postmaster.h
M src/include/utils/guc_tables.h
M src/tools/msvc/Mkvcbuild.pm
Second attempt to stabilize 05c02589.
commit : facad31474ac6dace3894ebc7c45dc3cc829422e
author : Jeff Davis <[email protected]>
date : Sun, 27 Dec 2020 12:09:00 -0800
committer: Jeff Davis <[email protected]>
date : Sun, 27 Dec 2020 12:09:00 -0800
Removing the EXPLAIN test to stabilize the buildfarm. The execution
test should still be effective to catch the bug even if the plan is
slightly different on different platforms.
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Stabilize test introduced in 05c02589, per buildfarm.
commit : fa0fdf0510df1a21f42ac9f232f77a79b8577152
author : Jeff Davis <[email protected]>
date : Sun, 27 Dec 2020 09:47:23 -0800
committer: Jeff Davis <[email protected]>
date : Sun, 27 Dec 2020 09:47:23 -0800
In passing, make the capitalization match the rest of the file.
Reported-by: Tom Lane
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Fix bug #16784 in Disk-based Hash Aggregation.
commit : 05c0258966b502fae1bd63dcbe74d52f5c6f6948
author : Jeff Davis <[email protected]>
date : Sat, 26 Dec 2020 17:25:30 -0800
committer: Jeff Davis <[email protected]>
date : Sat, 26 Dec 2020 17:25:30 -0800
Before processing tuples, agg_refill_hash_table() was setting all
pergroup pointers to NULL to signal to advance_aggregates() that it
should not attempt to advance groups that had spilled.
The problem was that it also set the pergroups for sorted grouping
sets to NULL, which caused rescanning to fail.
Instead, change agg_refill_hash_table() to only set the pergroups for
hashed grouping sets to NULL; and when compiling the expression, pass
doSort=false.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/16784-7ff169bf2c3d1588%40postgresql.org
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
initdb: document that -K requires an argument
commit : ccbe34139b7d9059f4af3405b2411f653cc2ce7c
author : Bruce Momjian <[email protected]>
date : Sat, 26 Dec 2020 10:00:05 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 26 Dec 2020 10:00:05 -0500
Reported-by: "Shinoda, Noriyoshi"
Discussion: https://postgr.es/m/TU4PR8401MB1152E92B4D44C81E496D6032EEDB0@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM
Author: "Shinoda, Noriyoshi"
Backpatch-through: msater
M doc/src/sgml/ref/initdb.sgml
M src/bin/initdb/initdb.c
auth commands: list specific commands to install in Makefile
commit : ba6725df364cd30e068ccd962fa426d9b7b44066
author : Bruce Momjian <[email protected]>
date : Sat, 26 Dec 2020 09:25:05 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 26 Dec 2020 09:25:05 -0500
Previously I used Makefile functions.
Backpatch-through: master
M src/backend/Makefile
pg_alterckey: remove TAP check rules from Makefile
commit : e174a6f1937248886e124cdb68408e727aeea278
author : Bruce Momjian <[email protected]>
date : Sat, 26 Dec 2020 09:03:12 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 26 Dec 2020 09:03:12 -0500
Reported-by: Pavel Stehule, Michael Paquier
Discussion: https://postgr.es/m/CAFj8pRBRNo4co5bqCx4BLx1ZZ45Z_T-opPxA+u7SLp7gAtBpNA@mail.gmail.com
Backpatch-through: master
M src/bin/pg_alterckey/Makefile
Add scripts for retrieving the cluster file encryption key
commit : d7602afa2ef6d8b2018103dccd89e75b4985ac06
author : Bruce Momjian <[email protected]>
date : Sat, 26 Dec 2020 01:19:09 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 26 Dec 2020 01:19:09 -0500
Scripts are passphrase, direct, AWS, and two Yubikey ones.
Backpatch-through: master
M src/backend/Makefile
A src/backend/crypto/ckey_aws.sh.sample
A src/backend/crypto/ckey_direct.sh.sample
A src/backend/crypto/ckey_passphrase.sh.sample
A src/backend/crypto/ckey_piv_nopin.sh.sample
A src/backend/crypto/ckey_piv_pin.sh.sample
A src/backend/crypto/ssl_passphrase.sh.sample
doc: fix SGML markup for pg_alterckey from commit 62afb42a7f
commit : 3d4843babce087ae8dc734aeec36db084001a0e4
author : Bruce Momjian <[email protected]>
date : Sat, 26 Dec 2020 01:10:24 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 26 Dec 2020 01:10:24 -0500
Backpatch-through: master
M doc/src/sgml/ref/pg_alterckey.sgml
M doc/src/sgml/reference.sgml
pg_alterckey: adjust doc build and Win32 sleep/open build fails
commit : 82f8c45be551a2940e3cbaf08aa7a521bdac8331
author : Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 22:47:16 -0500
committer: Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 22:47:16 -0500
Fix for commit 62afb42a7f.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: master
M doc/src/sgml/ref/allfiles.sgml
R100 doc/src/sgml/ref/pg_alterkey.sgml doc/src/sgml/ref/pg_alterckey.sgml
M src/bin/pg_alterckey/pg_alterckey.c
Fix function call typo in frontend Win32 code, commit 978f869b99
commit : 7705f8ca03713457bf7cde0b859a1ab45e4f268b
author : Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 20:49:50 -0500
committer: Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 20:49:50 -0500
Reported-by: buildfarm member walleye
Backpatch-through: master
M src/common/kmgr_utils.c
Allow ssl_passphrase_command to prompt the terminal
commit : 300e430c76644b483f70bc833d6251a6a7fb84a6
author : Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 20:41:06 -0500
committer: Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 20:41:06 -0500
Previously the command could not access the terminal for a passphrase.
Backpatch-through: master
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/pg_ctl-ref.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M src/backend/libpq/be-secure-common.c
Add pg_alterckey utility to change the cluster key
commit : 62afb42a7f9f533efc6c19f462c3a848fa4ddb63
author : Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 20:24:53 -0500
committer: Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 20:24:53 -0500
This can change the key that encrypts the data encryption keys used for
cluster file encryption.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: master
A doc/src/sgml/ref/pg_alterkey.sgml
M src/bin/Makefile
A src/bin/pg_alterckey/.gitignore
A src/bin/pg_alterckey/Makefile
A src/bin/pg_alterckey/pg_alterckey.c
remove missing reference to crypto test from patch 978f869b99
commit : f234899353f8998bdbd265125ce4a505a312d910
author : Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 15:13:16 -0500
committer: Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 15:13:16 -0500
Reported-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: master
M src/test/Makefile
Really fix the dummy implementations in cipher.c.
commit : 0848cf4f553bda5a9f044c56b3bec19929ee24e6
author : Tom Lane <[email protected]>
date : Fri, 25 Dec 2020 14:44:17 -0500
committer: Tom Lane <[email protected]>
date : Fri, 25 Dec 2020 14:44:17 -0500
945083b2f wasn't enough to silence compiler warnings.
M src/common/cipher.c
fix no-return function call in cipher.c from commit 978f869b99
commit : 8e59813e2287db90044537b1093c0869020ffa4a
author : Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 14:40:46 -0500
committer: Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 14:40:46 -0500
Reported-by: buildfarm member sifaka
Backpatch-through: master
M src/common/cipher.c
fixes docs and missing initdb help option for commit 978f869b99
commit : 26d60f2a6c1c93838a49ac4d685652b1bc92f547
author : Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 14:00:12 -0500
committer: Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 14:00:12 -0500
Reported-by: Erik Rijkers
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: master
M doc/src/sgml/database-encryption.sgml
M src/bin/initdb/initdb.c
Invalidate acl.c caches when pg_authid changes.
commit : 08db7c63f34e6d9a402af81596e303f5b38d19b0
author : Noah Misch <[email protected]>
date : Fri, 25 Dec 2020 10:41:59 -0800
committer: Noah Misch <[email protected]>
date : Fri, 25 Dec 2020 10:41:59 -0800
This makes existing sessions reflect "ALTER ROLE ... [NO]INHERIT" as
quickly as they have been reflecting "GRANT role_name". Back-patch to
9.5 (all supported versions).
Reviewed by Nathan Bossart.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/acl.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
remove uint128 requirement from patch 978f869b99 (CFE)
commit : e35b2bad1a10a8eef9c1ffb563847b9c9df0cfce
author : Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 11:35:59 -0500
committer: Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 11:35:59 -0500
Used char[16] instead.
Reported-by: buildfarm member florican
Backpatch-through: master
M src/common/kmgr_utils.c
M src/include/common/kmgr_utils.h
Fix return value and const declaration from commit 978f869b99
commit : 945083b2f7e6c19c8921c655cac6778acb1e3e03
author : Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 11:00:32 -0500
committer: Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 11:00:32 -0500
This fixes the non-OpenSSL compile case.
Reported-by: buildfarm member sifaka
Backpatch-through: master
M src/common/cipher.c
Add key management system
commit : 978f869b992f9fca343e99d6fdb71073c76e869a
author : Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 10:19:44 -0500
committer: Bruce Momjian <[email protected]>
date : Fri, 25 Dec 2020 10:19:44 -0500
This adds a key management system that stores (currently) two data
encryption keys of length 128, 192, or 256 bits. The data keys are
AES256 encrypted using a key encryption key, and validated via GCM
cipher mode. A command to obtain the key encryption key must be
specified at initdb time, and will be run at every database server
start. New parameters allow a file descriptor open to the terminal to
be passed. pg_upgrade support has also been added.
Discussion: https://postgr.es/m/CA+fd4k7q5o6Nc_AaX6BcYM9yqTbC6_pnH-6nSD=54Zp6NBQTCQ@mail.gmail.com
Discussion: https://postgr.es/m/[email protected]
Author: Masahiko Sawada, me, Stephen Frost
M doc/src/sgml/config.sgml
A doc/src/sgml/database-encryption.sgml
M doc/src/sgml/filelist.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/postgres.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/ref/pg_ctl-ref.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/postgres-ref.sgml
M doc/src/sgml/storage.sgml
M src/backend/Makefile
M src/backend/access/transam/xlog.c
M src/backend/bootstrap/bootstrap.c
A src/backend/crypto/Makefile
A src/backend/crypto/kmgr.c
M src/backend/main/main.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/replication/basebackup.c
M src/backend/storage/ipc/ipci.c
M src/backend/storage/lmgr/lwlocknames.txt
M src/backend/tcop/postgres.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/pg_controldata.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/initdb/initdb.c
M src/bin/pg_controldata/pg_controldata.c
M src/bin/pg_ctl/pg_ctl.c
M src/bin/pg_resetwal/pg_resetwal.c
M src/bin/pg_rewind/filemap.c
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/file.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/server.c
M src/common/Makefile
A src/common/cipher.c
A src/common/cipher_openssl.c
A src/common/kmgr_utils.c
M src/include/catalog/pg_control.h
A src/include/common/cipher.h
A src/include/common/kmgr_utils.h
A src/include/crypto/kmgr.h
M src/include/pgstat.h
M src/include/postmaster/postmaster.h
M src/include/utils/guc_tables.h
M src/test/Makefile
M src/tools/msvc/Mkvcbuild.pm
Avoid time-of-day-dependent failure in log rotation test.
commit : 5c31afc49d0b62b357218b6f8b01782509ef8acd
author : Tom Lane <[email protected]>
date : Thu, 24 Dec 2020 21:37:46 -0500
committer: Tom Lane <[email protected]>
date : Thu, 24 Dec 2020 21:37:46 -0500
Buildfarm members pogona and petalura have shown a failure when
pg_ctl/t/004_logrotate.pl starts just before local midnight.
The default rotate-at-midnight behavior occurs just before the
Perl script examines current_logfiles, so it figures that the
rotation it's already requested has occurred ... but in reality,
that rotation happens just after it looks, so the expected new
log data goes into a different file than the one it's examining.
In HEAD, src/test/kerberos/t/001_auth.pl has acquired similar code
that evidently has a related failure mode. Besides being quite new,
few buildfarm critters run that test, so it's unsurprising that
we've not yet seen a failure there.
Fix both cases by setting log_rotation_age = 0 so that no time-based
rotation can occur. Also absorb 004_logrotate.pl's decision to
set lc_messages = 'C' into the kerberos test, in hopes that it will
work in non-English prevailing locales.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=pogona&dt=2020-12-24%2022%3A10%3A04
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=petalura&dt=2020-02-01%2022%3A20%3A04
M src/bin/pg_ctl/t/004_logrotate.pl
M src/test/kerberos/t/001_auth.pl
revert removal of hex_decode() from ecpg from commit c3826f831e
commit : 558a6e8e21367dae9a294291337be10a8a4afd0a
author : Bruce Momjian <[email protected]>
date : Thu, 24 Dec 2020 18:21:37 -0500
committer: Bruce Momjian <[email protected]>
date : Thu, 24 Dec 2020 18:21:37 -0500
ecpglib on certain platforms can't handle the pg_log_fatal calls from
libraries. This was reported by the buildfarm. It needs a refactoring
and return value change if it is later removed.
Backpatch-through: master
M src/interfaces/ecpg/ecpglib/data.c
move hex_decode() to /common so it can be called from frontend
commit : c3826f831e6e63e13a749fd3ab9fd7106707b549
author : Bruce Momjian <[email protected]>
date : Thu, 24 Dec 2020 17:25:48 -0500
committer: Bruce Momjian <[email protected]>
date : Thu, 24 Dec 2020 17:25:48 -0500
This allows removal of a copy of hex_decode() from ecpg, and will be
used by the soon-to-be added pg_alterckey command.
Backpatch-through: master
M src/backend/utils/adt/encode.c
M src/backend/utils/adt/varlena.c
M src/common/Makefile
A src/common/hex_decode.c
A src/include/common/hex_decode.h
M src/include/utils/builtins.h
M src/interfaces/ecpg/ecpglib/data.c
M src/tools/msvc/Mkvcbuild.pm
Fix race condition between shutdown and unstarted background workers.
commit : 7519bd16d1a2a2007b3604f6740611c0b75a6cd0
author : Tom Lane <[email protected]>
date : Thu, 24 Dec 2020 17:00:43 -0500
committer: Tom Lane <[email protected]>
date : Thu, 24 Dec 2020 17:00:43 -0500
If a database shutdown (smart or fast) is commanded between the time
some process decides to request a new background worker and the time
that the postmaster can launch that worker, then nothing happens
because the postmaster won't launch any bgworkers once it's exited
PM_RUN state. This is fine ... unless the requesting process is
waiting for that worker to finish (or even for it to start); in that
case the requestor is stuck, and only manual intervention will get us
to the point of being able to shut down.
To fix, cancel pending requests for workers when the postmaster sends
shutdown (SIGTERM) signals, and similarly cancel any new requests that
arrive after that point. (We can optimize things slightly by only
doing the cancellation for workers that have waiters.) To fit within
the existing bgworker APIs, the "cancel" is made to look like the
worker was started and immediately stopped, causing deregistration of
the bgworker entry. Waiting processes would have to deal with
premature worker exit anyway, so this should introduce no bugs that
weren't there before. We do have a side effect that registration
records for restartable bgworkers might disappear when theoretically
they should have remained in place; but since we're shutting down,
that shouldn't matter.
Back-patch to v10. There might be value in putting this into 9.6
as well, but the management of bgworkers is a bit different there
(notably see 8ff518699) and I'm not convinced it's worth the effort
to validate the patch for that branch.
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_prewarm/autoprewarm.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/postmaster.c
M src/include/postmaster/bgworker_internals.h
Improve client error messages for immediate-stop situations.
commit : 7e784d1dc191be24480a6b31a4ddc8e0e52be24d
author : Tom Lane <[email protected]>
date : Thu, 24 Dec 2020 12:58:32 -0500
committer: Tom Lane <[email protected]>
date : Thu, 24 Dec 2020 12:58:32 -0500
Up to now, if the DBA issued "pg_ctl stop -m immediate", the message
sent to clients was the same as for a crash-and-restart situation.
This is confusing, not least because the message claims that the
database will soon be up again, something we have no business
predicting.
Improve things so that we can generate distinct messages for the two
cases (and also recognize an ad-hoc SIGQUIT, should somebody try that).
To do that, add a field to pmsignal.c's shared memory data structure
that the postmaster sets just before broadcasting SIGQUIT to its
children. No interlocking seems to be necessary; the intervening
signal-sending and signal-receipt should sufficiently serialize accesses
to the field. Hence, this isn't any riskier than the existing usages
of pmsignal.c.
We might in future extend this idea to improve other
postmaster-to-children signal scenarios, although none of them
currently seem to be as badly overloaded as SIGQUIT.
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/postmaster.c
M src/backend/storage/ipc/pmsignal.c
M src/backend/tcop/postgres.c
M src/include/storage/pmsignal.h
Fix typos and grammar in docs and comments
commit : 90fbf7c57df601c7e0b43ae7cf71f0f69908a7cc
author : Michael Paquier <[email protected]>
date : Thu, 24 Dec 2020 17:05:49 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 24 Dec 2020 17:05:49 +0900
This fixes several areas of the documentation and some comments in
matters of style, grammar, or even format.
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_stat_statements/pg_stat_statements.c
M doc/src/sgml/amcheck.sgml
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/cube.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/pgstatstatements.sgml
M doc/src/sgml/rangetypes.sgml
M doc/src/sgml/ref/create_type.sgml
M doc/src/sgml/ref/explain.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_verifybackup.sgml
M doc/src/sgml/ref/prepare.sgml
M doc/src/sgml/sources.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/xlog.c
M src/backend/partitioning/partprune.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_directory.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_upgrade/check.c
M src/tools/msvc/README
dummy commit
commit : 6ecf488d91ceb93a8be907a24c5d8a90358534ed
author : Bruce Momjian <[email protected]>
date : Wed, 25 Nov 2020 12:29:21 -0500
committer: Bruce Momjian <[email protected]>
date : Wed, 25 Nov 2020 12:29:21 -0500
M doc/src/sgml/plperl.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/plpython.sgml
M doc/src/sgml/pltcl.sgml
M doc/src/sgml/spi.sgml
Fix portability issues with parsing of recovery_target_xid
commit : 6db27037b99358018353b8748719cfa9de484988
author : Michael Paquier <[email protected]>
date : Wed, 23 Dec 2020 12:51:22 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 23 Dec 2020 12:51:22 +0900
The parsing of this parameter has been using strtoul(), which is not
portable across platforms. On most Unix platforms, unsigned long has a
size of 64 bits, while on Windows it is 32 bits. It is common in
recovery scenarios to rely on the output of txid_current() or even the
newer pg_current_xact_id() to get a transaction ID for setting up
recovery_target_xid. The value returned by those functions includes the
epoch in the computed result, which would cause strtoul() to fail where
unsigned long has a size of 32 bits once the epoch is incremented.
WAL records and 2PC data include only information about 32-bit XIDs and
it is not possible to have XIDs across more than one epoch, so
discarding the high bits from the transaction ID set has no impact on
recovery. On the contrary, the use of strtoul() prevents a consistent
behavior across platforms depending on the size of unsigned long.
This commit changes the parsing of recovery_target_xid to use
pg_strtouint64() instead, available down to 9.6. There is one TAP test
stressing recovery with recovery_target_xid, where a tweak based on
pg_reset{xlog,wal} is added to bump the XID epoch so as this change gets
tested, as per an idea from Alexander Lakhin.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/backend/utils/misc/guc.c
M src/test/recovery/t/003_recovery_targets.pl
Improve autoprewarm's handling of early-shutdown scenarios.
commit : ff769831e081a4e69147019128a849ced43ec802
author : Tom Lane <[email protected]>
date : Tue, 22 Dec 2020 13:23:49 -0500
committer: Tom Lane <[email protected]>
date : Tue, 22 Dec 2020 13:23:49 -0500
Bad things happen if the DBA issues "pg_ctl stop -m fast" before
autoprewarm finishes loading its list of blocks to prewarm.
The current worker process successfully terminates early, but
(if this wasn't the last database with blocks to prewarm) the
leader process will just try to launch another worker for the
next database. Since the postmaster is now in PM_WAIT_BACKENDS
state, it ignores the launch request, and the leader just sits
until it's killed manually.
This is mostly the fault of our half-baked design for launching
background workers, but a proper fix for that is likely to be
too invasive to be back-patchable. To ameliorate the situation,
fix apw_load_buffers() to check whether SIGTERM has arrived
just before trying to launch another worker. That leaves us with
only a very narrow window in each worker launch where SIGTERM
could occur between the launch request and successful worker start.
Another issue is that if the leader process does manage to exit,
it unconditionally rewrites autoprewarm.blocks with only the
blocks currently in shared buffers, thus forgetting any blocks
that we hadn't reached yet while prewarming. This seems quite
unhelpful, since the next database start will then not have the
expected prewarming benefit. Fix it to not modify the file if
we shut down before the initial load attempt is complete.
Per bug #16785 from John Thompson. Back-patch to v11 where
the autoprewarm code was introduced.
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_prewarm/autoprewarm.c
Increase timeout in 021_row_visibility.pl.
commit : 08dde1b3dc78e90e01f3af8e89ab27ed37cd8963
author : Tom Lane <[email protected]>
date : Tue, 22 Dec 2020 11:10:12 -0500
committer: Tom Lane <[email protected]>
date : Tue, 22 Dec 2020 11:10:12 -0500
Commit 7b28913bc figured 30 seconds is long enough for anybody,
but in contexts like valgrind runs, it isn't necessarily.
M src/test/recovery/t/021_row_visibility.pl
Improve find_em_expr_usable_for_sorting_rel comment
commit : 1ca2eb103113d64930cf1120ba2beaaebd1076f4
author : Tomas Vondra <[email protected]>
date : Tue, 22 Dec 2020 02:00:39 +0100
committer: Tomas Vondra <[email protected]>
date : Tue, 22 Dec 2020 02:00:39 +0100
Clarify the relationship between find_em_expr_usable_for_sorting_rel and
prepare_sort_from_pathkeys, i.e. what restrictions need to be shared
between those two places.
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs%3DhC0mSksZC%3DH5M8LSchj5e5OxpTAg%40mail.gmail.com
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/plan/createplan.c
Don't search for volatile expr in find_em_expr_usable_for_sorting_rel
commit : 9aff4dc01fbb9e126efafd839ae8541362328f1a
author : Tomas Vondra <[email protected]>
date : Mon, 21 Dec 2020 19:37:14 +0100
committer: Tomas Vondra <[email protected]>
date : Mon, 21 Dec 2020 19:37:14 +0100
While prepare_sort_from_pathkeys has to be concerned about matching up
a volatile expression to the proper tlist entry, we don't need to do
that in find_em_expr_usable_for_sorting_rel becausee such a sort will
have to be postponed anyway.
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs%3DhC0mSksZC%3DH5M8LSchj5e5OxpTAg%40mail.gmail.com
M src/backend/optimizer/path/equivclass.c
Disallow SRFs when considering sorts below Gather Merge
commit : fac1b470a9f7e846534620b78ea4cf122ed432b2
author : Tomas Vondra <[email protected]>
date : Mon, 21 Dec 2020 18:58:32 +0100
committer: Tomas Vondra <[email protected]>
date : Mon, 21 Dec 2020 18:58:32 +0100
While we do allow SRFs in ORDER BY, scan/join processing should not
consider such cases - such sorts should only happen via final Sort atop
a ProjectSet. So make sure we don't try adding such sorts below Gather
Merge, just like we do for expressions that are volatile and/or not
parallel safe.
Backpatch to PostgreSQL 13, where this code was introduced as part of
the Incremental Sort patch.
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs=hC0mSksZC=H5M8LSchj5e5OxpTAg@mail.gmail.com
Discussion: https://postgr.es/m/295524.1606246314%40sss.pgh.pa.us
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/util/tlist.c
M src/include/optimizer/optimizer.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Remove "invalid concatenation of jsonb objects" error case.
commit : ff5d5611c01f60525c30b2c3ebc16d05edb7956d
author : Tom Lane <[email protected]>
date : Mon, 21 Dec 2020 13:11:29 -0500
committer: Tom Lane <[email protected]>
date : Mon, 21 Dec 2020 13:11:29 -0500
The jsonb || jsonb operator arbitrarily rejected certain combinations
of scalar and non-scalar inputs, while being willing to concatenate
other combinations. This was of course quite undocumented. Rather
than trying to document it, let's just remove the restriction,
creating a uniform rule that unless we are handling an object-to-object
concatenation, non-array inputs are converted to one-element arrays,
resulting in an array-to-array concatenation. (This does not change
the behavior for any case that didn't throw an error before.)
Per complaint from Joel Jacobson. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/utils/adt/jsonfuncs.c
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql
Check parallel safety in generate_useful_gather_paths
commit : 86b7cca72d4d0a4e043fac0a2cdd56218ff2f258
author : Tomas Vondra <[email protected]>
date : Mon, 21 Dec 2020 18:29:46 +0100
committer: Tomas Vondra <[email protected]>
date : Mon, 21 Dec 2020 18:29:46 +0100
Commit ebb7ae839d ensured we ignore pathkeys with volatile expressions
when considering adding a sort below a Gather Merge. Turns out we need
to care about parallel safety of the pathkeys too, otherwise we might
try sorting e.g. on results of a correlated subquery (as demonstrated
by a report from Luis Roberto).
Initial investigation by Tom Lane, patch by James Coleman. Backpatch
to 13, where the code was instroduced (as part of Incremental Sort).
Reported-by: Luis Roberto
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/622580997.37108180.1604080457319.JavaMail.zimbra%40siscobra.com.br
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs=hC0mSksZC=H5M8LSchj5e5OxpTAg@mail.gmail.com
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/equivclass.c
M src/include/optimizer/paths.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Consider unsorted paths in generate_useful_gather_paths
commit : f4a3c0b06250ddc8ae09b59b87cf68e9bc0d7ca1
author : Tomas Vondra <[email protected]>
date : Mon, 21 Dec 2020 18:09:57 +0100
committer: Tomas Vondra <[email protected]>
date : Mon, 21 Dec 2020 18:09:57 +0100
generate_useful_gather_paths used to skip unsorted paths (without any
pathkeys), but that is unnecessary - the later code actually can handle
such paths just fine by adding a Sort node. This is clearly a thinko,
preventing construction of useful plans.
Backpatch to 13, where Incremental Sort was introduced.
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs=hC0mSksZC=H5M8LSchj5e5OxpTAg@mail.gmail.com
M src/backend/optimizer/path/allpaths.c
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Fix compiler warning in multirange_constructor0()
commit : 29f8f546767ebb4253f6dc37815e2d95fff9acd0
author : Alexander Korotkov <[email protected]>
date : Mon, 21 Dec 2020 14:25:32 +0300
committer: Alexander Korotkov <[email protected]>
date : Mon, 21 Dec 2020 14:25:32 +0300
Discussion: https://postgr.es/m/X%2BBP8XE0UpIB6Yvh%40paquier.xyz
Author: Michael Paquier
M src/backend/utils/adt/multirangetypes.c
Refactor logic to check for ASCII-only characters in string
commit : 93e8ff8701a65a70ea8826bdde7fdbbd9c285477
author : Michael Paquier <[email protected]>
date : Mon, 21 Dec 2020 09:37:11 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 21 Dec 2020 09:37:11 +0900
The same logic was present for collation commands, SASLprep and
pgcrypto, so this removes some code.
Author: Michael Paquier
Reviewed-by: Stephen Frost, Heikki Linnakangas
Discussion: https://postgr.es/m/[email protected]
M contrib/pgcrypto/pgp-pgsql.c
M src/backend/commands/collationcmds.c
M src/common/saslprep.c
M src/common/string.c
M src/include/common/string.h
Fix typalign in rangetypes statistics
commit : 4e1ee79e3182256d9c8ddbc1ce9c4e8419c611ff
author : Alexander Korotkov <[email protected]>
date : Mon, 21 Dec 2020 00:26:13 +0300
committer: Alexander Korotkov <[email protected]>
date : Mon, 21 Dec 2020 00:26:13 +0300
6df7a9698b introduces multirange types, whose typanalyze function shares
infrastructure with range types typanalyze function. Since 6df7a9698b,
information about type gathered by statistics is filled from typcache.
But typalign is mistakenly always set to double. This commit fixes this
oversight.
M src/backend/utils/adt/rangetypes_typanalyze.c
Doc: fix description of how to use src/tutorial files.
commit : eea1e08cfc95bb7e1b7e9f515e73cf75adf147e3
author : Tom Lane <[email protected]>
date : Sun, 20 Dec 2020 15:28:22 -0500
committer: Tom Lane <[email protected]>
date : Sun, 20 Dec 2020 15:28:22 -0500
The separate "cd" command before invoking psql made sense (or at least
I thought so) when it was added in commit ed1939332. But 4e3a61635
removed the supporting text that explained when to use it, making it
just confusing. So drop it.
Also switch from four-dot to three-dot filler for the unsupplied
part of the path, since at least one person has read the four-dot
filler as a typo for "../..". And fix these/those inconsistency.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/query.sgml
Doc: improve description of pgbench script weights.
commit : 1990ce186ee64f24dcff885a87cea04e3dfd8166
author : Tom Lane <[email protected]>
date : Sun, 20 Dec 2020 13:37:25 -0500
committer: Tom Lane <[email protected]>
date : Sun, 20 Dec 2020 13:37:25 -0500
Point out the workaround to be used if you want to write a script
file name that includes "@". Clean up the text a little.
Fabien Coelho, additional wordsmithing by me
Discussion: https://postgr.es/m/1c4e81550d214741827a03292222db8d@G08CNEXMBPEKD06.g08.fujitsu.local
M doc/src/sgml/ref/pgbench.sgml
Avoid memcpy() with same source and destination in pgstat_recv_replslot.
commit : ed6329cfa975a13c58c44676464585cda5d317b3
author : Tom Lane <[email protected]>
date : Sun, 20 Dec 2020 12:38:32 -0500
committer: Tom Lane <[email protected]>
date : Sun, 20 Dec 2020 12:38:32 -0500
Same type of issue as in commit 53d4f5fef and earlier fixes; also
found by apparently-more-picky-than-the-buildfarm valgrind testing.
This one is an oversight in commit 986816750. Since that's new in
HEAD, no need for a back-patch.
M src/backend/postmaster/pgstat.c
Fix compiler warning introduced in 6df7a9698b
commit : 11072e86939d2f5ca0a97c709b46f29ea8b57590
author : Alexander Korotkov <[email protected]>
date : Sun, 20 Dec 2020 16:27:01 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 20 Dec 2020 16:27:01 +0300
M src/backend/commands/typecmds.c
Fixes for pg_dump.c regarding multiranges
commit : 8344d72cccae699e13884a5705b91dc1c4747c03
author : Alexander Korotkov <[email protected]>
date : Sun, 20 Dec 2020 08:14:35 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 20 Dec 2020 08:14:35 +0300
This commit fixes two wrong version number checks and one wrong check for null.
M src/bin/pg_dump/pg_dump.c
Multirange datatypes
commit : 6df7a9698bb036610c1e8c6d375e1be38cb26d5f
author : Alexander Korotkov <[email protected]>
date : Sun, 20 Dec 2020 07:20:33 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 20 Dec 2020 07:20:33 +0300
Multiranges are basically sorted arrays of non-overlapping ranges with
set-theoretic operations defined over them.
Since v14, each range type automatically gets a corresponding multirange
datatype. There are both manual and automatic mechanisms for naming multirange
types. Once can specify multirange type name using multirange_type_name
attribute in CREATE TYPE. Otherwise, a multirange type name is generated
automatically. If the range type name contains "range" then we change that to
"multirange". Otherwise, we add "_multirange" to the end.
Implementation of multiranges comes with a space-efficient internal
representation format, which evades extra paddings and duplicated storage of
oids. Altogether this format allows fetching a particular range by its index
in O(n).
Statistic gathering and selectivity estimation are implemented for multiranges.
For this purpose, stored multirange is approximated as union range without gaps.
This field will likely need improvements in the future.
Catversion is bumped.
Discussion: https://postgr.es/m/CALNJ-vSUpQ_Y%3DjXvTxt1VYFztaBSsWVXeF1y6gTYQ4bOiWDLgQ%40mail.gmail.com
Discussion: https://postgr.es/m/a0b8026459d1e6167933be2104a6174e7d40d0ab.camel%40j-davis.com#fe7218c83b08068bfffb0c5293eceda0
Author: Paul Jungwirth, revised by me
Reviewed-by: David Fetter, Corey Huinker, Jeff Davis, Pavel Stehule
Reviewed-by: Alvaro Herrera, Tom Lane, Isaac Morland, David G. Johnston
Reviewed-by: Zhihong Yu, Alexander Korotkov
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/rangetypes.sgml
M doc/src/sgml/ref/create_type.sgml
M src/backend/catalog/pg_range.c
M src/backend/catalog/pg_type.c
M src/backend/commands/typecmds.c
M src/backend/executor/functions.c
M src/backend/parser/parse_coerce.c
M src/backend/utils/adt/Makefile
A src/backend/utils/adt/multirangetypes.c
A src/backend/utils/adt/multirangetypes_selfuncs.c
M src/backend/utils/adt/pg_upgrade_support.c
M src/backend/utils/adt/pseudotypes.c
M src/backend/utils/adt/rangetypes.c
M src/backend/utils/adt/rangetypes_typanalyze.c
M src/backend/utils/cache/lsyscache.c
M src/backend/utils/cache/syscache.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/fmgr/funcapi.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/include/access/tupmacs.h
M src/include/catalog/binary_upgrade.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_aggregate.dat
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_cast.dat
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_range.dat
M src/include/catalog/pg_range.h
M src/include/catalog/pg_type.dat
M src/include/catalog/pg_type.h
M src/include/commands/typecmds.h
M src/include/utils/lsyscache.h
A src/include/utils/multirangetypes.h
M src/include/utils/rangetypes.h
M src/include/utils/selfuncs.h
M src/include/utils/syscache.h
M src/include/utils/typcache.h
M src/pl/plpgsql/src/pl_comp.c
M src/test/regress/expected/dependency.out
M src/test/regress/expected/hash_func.out
A src/test/regress/expected/multirangetypes.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/expected/plpgsql.out
M src/test/regress/expected/polymorphism.out
M src/test/regress/expected/rangefuncs.out
M src/test/regress/expected/rangetypes.out
M src/test/regress/expected/sanity_check.out
M src/test/regress/expected/type_sanity.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
M src/test/regress/sql/hash_func.sql
A src/test/regress/sql/multirangetypes.sql
M src/test/regress/sql/opr_sanity.sql
M src/test/regress/sql/polymorphism.sql
M src/test/regress/sql/rangetypes.sql
M src/test/regress/sql/type_sanity.sql
M src/tools/pgindent/typedefs.list
Remove now-useless ALWAYS_SUBDIRS entry in src/test/Makefile.
commit : 08b01d4dd982b491a2f9641804b368185b8f4c53
author : Tom Lane <[email protected]>
date : Sat, 19 Dec 2020 17:58:30 -0500
committer: Tom Lane <[email protected]>
date : Sat, 19 Dec 2020 17:58:30 -0500
Commit 257836a75 added the "locale" subdirectory to SUBDIRS,
but neglected to remove it from ALWAYS_SUBDIRS. This oversight
had no functional effect because the filter-out function would
remove it anyway. Still, it's confusing to readers to list a
subdirectory in both places, especially because it makes the
associated comment into a partial lie.
M src/test/Makefile
Update comment atop of ReorderBufferQueueMessage().
commit : 20659fd8e5ecb3bcb3e337581c4bd7123d490ddc
author : Amit Kapila <[email protected]>
date : Sat, 19 Dec 2020 10:08:46 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 19 Dec 2020 10:08:46 +0530
The comments atop of this function describes behaviour in case of a
transactional WAL message only, but it accepts both transactional and
non-transactional WAL messages. Update the comments to describe
behaviour in case of non-transactional WAL message as well.
Ashutosh Bapat, rephrased by Amit Kapila
Discussion: https://postgr.es/m/CAGEoWWTTzNzHOi8bj0wfAo1siGi-YEh6wqH1oaz4DrkTJ6HbTQ@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
Add a couple of missed .gitignore entries.
commit : 8afca702ecbf0730b6175afda0cecbbf0a1967e7
author : Tom Lane <[email protected]>
date : Fri, 18 Dec 2020 16:24:49 -0500
committer: Tom Lane <[email protected]>
date : Fri, 18 Dec 2020 16:24:49 -0500
Any subdirectory that's ignoring /output_iso/ should also
ignore /tmp_check_iso/, which could be left behind by a
failed pg_isolation_regress_check run.
I think these have been wrong for awhile, but it doesn't
seem important to fix in back branches.
M src/test/modules/brin/.gitignore
M src/test/modules/snapshot_too_old/.gitignore
Avoid memcpy() with same source and destination during relmapper init.
commit : 53d4f5fef04653fc495ae385a9c2f78c77e5c7d9
author : Tom Lane <[email protected]>
date : Fri, 18 Dec 2020 15:46:44 -0500
committer: Tom Lane <[email protected]>
date : Fri, 18 Dec 2020 15:46:44 -0500
A narrow reading of the C standard says that memcpy(x,x,n) is undefined,
although it's hard to envision an implementation that would really
misbehave. However, analysis tools such as valgrind might whine about
this; accordingly, let's band-aid relmapper.c to not do it.
See also 5b630501e, d3f4e8a8a, ad7b48ea0, and other similar fixes.
Apparently, none of those folk tried valgrinding initdb? This has been
like this for long enough that I'm surprised it hasn't been reported
before.
Back-patch, just in case anybody wants to use a back branch on a platform
that complains about this; we back-patched those earlier fixes too.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/cache/relmapper.c
pg_stat_statements: Track time at which all statistics were last reset.
commit : 2e0fedf0362cc964c4dae42258455b6391051e70
author : Fujii Masao <[email protected]>
date : Fri, 18 Dec 2020 10:49:58 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 18 Dec 2020 10:49:58 +0900
This commit adds "stats_reset" column into the pg_stat_statements_info
view. This column indicates the time at which all statistics in the
pg_stat_statements view were last reset.
Per discussion, this commit also changes pg_stat_statements_info code
so that "dealloc" column is reset at the same time as "stats_reset" is reset,
i.e., whenever all pg_stat_statements entries are removed, for the sake
of consistency. Previously "dealloc" was reset only when
pg_stat_statements_reset(0, 0, 0) is called and was not reset when
pg_stat_statements_reset() with non-zero value argument discards all
entries. This was confusing.
Author: Naoki Nakamichi, Yuki Seino
Reviewed-by: Yuki Seino, Kyotaro Horiguchi, Li Japin, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_stat_statements/pg_stat_statements–1.8–1.9.sql
M contrib/pg_stat_statements/pg_stat_statements.c
M doc/src/sgml/pgstatstatements.sgml
Revert "Get rid of the dedicated latch for signaling the startup process".
commit : 00f690a239932e477f25120d19b08aacdc30deb7
author : Fujii Masao <[email protected]>
date : Thu, 17 Dec 2020 18:06:51 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 17 Dec 2020 18:06:51 +0900
Revert ac22929a26, as well as the followup fix 113d3591b8. Because it broke
the assumption that the startup process waiting for the recovery conflict
on buffer pin should be waken up only by buffer unpin or the timeout enabled
in ResolveRecoveryConflictWithBufferPin(). It caused, for example,
SIGHUP signal handler or walreceiver process to wake that startup process
up unnecessarily frequently.
Additionally, add the comments about why that dedicated latch that
the reverted patch tried to get rid of should not be removed.
Thanks to Kyotaro Horiguchi for the discussion.
Author: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
M src/backend/postmaster/startup.c
M src/backend/storage/ipc/standby.c
Fix varchar_2.out to match reality in cs_CZ locale.
commit : 88e014c149cc396fb218b08eda17c47d5b33e94f
author : Tom Lane <[email protected]>
date : Tue, 15 Dec 2020 21:42:08 -0500
committer: Tom Lane <[email protected]>
date : Tue, 15 Dec 2020 21:42:08 -0500
Seems to be a copy-and-pasteo in c06d6aa4c. Per buildfarm.
M src/test/regress/expected/varchar_2.out
doc: Fix explanation related to pg_shmem_allocations
commit : bce641a2af7199f07ec2b0a84f8fec0f26b2adc0
author : Michael Paquier <[email protected]>
date : Wed, 16 Dec 2020 10:36:03 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 16 Dec 2020 10:36:03 +0900
Offsets are shown as NULL only for anonymous allocations.
Author: Benoit Lobréau
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAPE8EZ5Lnoyqoz7aZpvQM0E8sW+hw+k6G2NULe+m4arFRrA1aA@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/catalogs.sgml
doc: clarify COPY TO for partitioning/inheritance
commit : 02c767b0fe241416d7b7ee3c107dcaf379d95381
author : Bruce Momjian <[email protected]>
date : Tue, 15 Dec 2020 19:20:15 -0500
committer: Bruce Momjian <[email protected]>
date : Tue, 15 Dec 2020 19:20:15 -0500
It was not clear how COPY TO behaved with partitioning/inheritance
because the paragraphs were so far apart. Also reword to simplify.
Discussion: https://postgr.es/m/[email protected]
Author: Justin Pryzby
Backpatch-through: 10
M doc/src/sgml/ref/copy.sgml
Remove obsolete btrescan() comment.
commit : 41ddc27f66fd398abe96e98f3e892a36270db9aa
author : Peter Geoghegan <[email protected]>
date : Tue, 15 Dec 2020 15:55:07 -0800
committer: Peter Geoghegan <[email protected]>
date : Tue, 15 Dec 2020 15:55:07 -0800
"Ordering stuff" refered to a _bt_first() call to _bt_orderkeys().
However, the _bt_orderkeys() function was renamed to
_bt_preprocess_keys() by commit fa5c8a055a0.
_bt_preprocess_keys() is directly referenced just after the removed
comment already, which seems sufficient.
M src/backend/access/nbtree/nbtree.c
Remove useless variable stores
commit : a18422a3adf79815fecb50a029b63a8e85ca9d21
author : Alvaro Herrera <[email protected]>
date : Tue, 15 Dec 2020 19:51:16 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 15 Dec 2020 19:51:16 -0300
Mistakenly introduced in 4cbe3ac3e867; bug repaired in 148e632c0541 but
the stores were accidentally.
M src/backend/executor/nodeModifyTable.c
Error out when Gather Merge input is not sorted
commit : 6bc27698324a3eadb8a4a5a79d96e16687be6141
author : Tomas Vondra <[email protected]>
date : Tue, 15 Dec 2020 23:13:38 +0100
committer: Tomas Vondra <[email protected]>
date : Tue, 15 Dec 2020 23:13:38 +0100
To build Gather Merge path, the input needs to be sufficiently sorted.
Ensuring this is the responsibility of the code constructing the paths,
but create_gather_merge_plan tried to handle unsorted paths by adding
an explicit Sort. In light of the recent issues related to Incremental
Sort, this is rather fragile. Some of the expressions may be volatile
or parallel unsafe, in which case we can't add the Sort here.
We could do more checks and add the Sort in at least some cases, but
it seems cleaner to just error out and make it clear this is a bug in
code constructing those paths.
Author: James Coleman
Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs%3DhC0mSksZC%3DH5M8LSchj5e5OxpTAg%40mail.gmail.com
Discussion: https://postgr.es/m/CAJGNTeNaxpXgBVcRhJX%2B2vSbq%2BF2kJqGBcvompmpvXb7pq%2BoFA%40mail.gmail.com
M src/backend/optimizer/plan/createplan.c
Clean up ancient test style
commit : c06d6aa4c35177655e0fd4acaca888a73c3f9845
author : Peter Eisentraut <[email protected]>
date : Tue, 15 Dec 2020 21:54:06 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 15 Dec 2020 21:54:06 +0100
Many older tests where written in a style like
SELECT '' AS two, i.* FROM INT2_TBL
where the first column indicated the number of expected result rows.
This has gotten increasingly out of date, as the test data fixtures
have expanded, so a lot of these were wrong and misleading. Moreover,
this style isn't really necessary, since the psql output already shows
the number of result rows.
To clean this up, remove all those extra columns.
Discussion: https://www.postgresql.org/message-id/flat/1a25312b-2686-380d-3c67-7a69094a999f%40enterprisedb.com
M src/test/regress/expected/boolean.out
M src/test/regress/expected/box.out
M src/test/regress/expected/case.out
M src/test/regress/expected/char.out
M src/test/regress/expected/char_1.out
M src/test/regress/expected/char_2.out
M src/test/regress/expected/circle.out
M src/test/regress/expected/float4-misrounded-input.out
M src/test/regress/expected/float4.out
M src/test/regress/expected/float8.out
M src/test/regress/expected/geometry.out
M src/test/regress/expected/horology.out
M src/test/regress/expected/inet.out
M src/test/regress/expected/int2.out
M src/test/regress/expected/int4.out
M src/test/regress/expected/int8.out
M src/test/regress/expected/interval.out
M src/test/regress/expected/join.out
M src/test/regress/expected/name.out
M src/test/regress/expected/numeric.out
M src/test/regress/expected/numerology.out
M src/test/regress/expected/oid.out
M src/test/regress/expected/path.out
M src/test/regress/expected/point.out
M src/test/regress/expected/polygon.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/text.out
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/expected/varchar.out
M src/test/regress/expected/varchar_1.out
M src/test/regress/expected/varchar_2.out
M src/test/regress/input/constraints.source
M src/test/regress/output/constraints.source
M src/test/regress/sql/boolean.sql
M src/test/regress/sql/box.sql
M src/test/regress/sql/case.sql
M src/test/regress/sql/char.sql
M src/test/regress/sql/circle.sql
M src/test/regress/sql/float4.sql
M src/test/regress/sql/float8.sql
M src/test/regress/sql/geometry.sql
M src/test/regress/sql/horology.sql
M src/test/regress/sql/inet.sql
M src/test/regress/sql/int2.sql
M src/test/regress/sql/int4.sql
M src/test/regress/sql/int8.sql
M src/test/regress/sql/interval.sql
M src/test/regress/sql/join.sql
M src/test/regress/sql/name.sql
M src/test/regress/sql/numeric.sql
M src/test/regress/sql/numerology.sql
M src/test/regress/sql/oid.sql
M src/test/regress/sql/path.sql
M src/test/regress/sql/point.sql
M src/test/regress/sql/polygon.sql
M src/test/regress/sql/subselect.sql
M src/test/regress/sql/text.sql
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
M src/test/regress/sql/varchar.sql
Improve hash_create()'s API for some added robustness.
commit : b3817f5f774663d55931dd4fab9c5a94a15ae7ab
author : Tom Lane <[email protected]>
date : Tue, 15 Dec 2020 11:38:53 -0500
committer: Tom Lane <[email protected]>
date : Tue, 15 Dec 2020 11:38:53 -0500
Invent a new flag bit HASH_STRINGS to specify C-string hashing, which
was formerly the default; and add assertions insisting that exactly
one of the bits HASH_STRINGS, HASH_BLOBS, and HASH_FUNCTION be set.
This is in hopes of preventing recurrences of the type of oversight
fixed in commit a1b8aa1e4 (i.e., mistakenly omitting HASH_BLOBS).
Also, when HASH_STRINGS is specified, insist that the keysize be
more than 8 bytes. This is a heuristic, but it should catch
accidental use of HASH_STRINGS for integer or pointer keys.
(Nearly all existing use-cases set the keysize to NAMEDATALEN or
more, so there's little reason to think this restriction should
be problematic.)
Tweak hash_create() to insist that the HASH_ELEM flag be set, and
remove the defaults it had for keysize and entrysize. Since those
defaults were undocumented and basically useless, no callers
omitted HASH_ELEM anyway.
Also, remove memset's zeroing the HASHCTL parameter struct from
those callers that had one. This has never been really necessary,
and while it wasn't a bad coding convention it was confusing that
some callers did it and some did not. We might as well save a few
cycles by standardizing on "not".
Also improve the documentation for hash_create().
In passing, improve reinit.c's usage of a hash table by storing
the key as a binary Oid rather than a string; and, since that's
a temporary hash table, allocate it in CurrentMemoryContext for
neatness.
Discussion: https://postgr.es/m/[email protected]
M contrib/dblink/dblink.c
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/shippable.c
M contrib/tablefunc/tablefunc.c
M src/backend/access/gist/gistbuildbuffers.c
M src/backend/access/hash/hashpage.c
M src/backend/access/heap/rewriteheap.c
M src/backend/access/transam/xlogutils.c
M src/backend/catalog/pg_enum.c
M src/backend/catalog/pg_inherits.c
M src/backend/commands/async.c
M src/backend/commands/prepare.c
M src/backend/commands/sequence.c
M src/backend/executor/execPartition.c
M src/backend/nodes/extensible.c
M src/backend/optimizer/util/predtest.c
M src/backend/optimizer/util/relnode.c
M src/backend/parser/parse_oper.c
M src/backend/partitioning/partdesc.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/relation.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/pgoutput/pgoutput.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/buffer/localbuf.c
M src/backend/storage/file/reinit.c
M src/backend/storage/ipc/shmem.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/lwlock.c
M src/backend/storage/lmgr/predicate.c
M src/backend/storage/smgr/smgr.c
M src/backend/storage/sync/sync.c
M src/backend/tsearch/ts_typanalyze.c
M src/backend/utils/adt/array_typanalyze.c
M src/backend/utils/adt/jsonfuncs.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/cache/attoptcache.c
M src/backend/utils/cache/evtcache.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/cache/relfilenodemap.c
M src/backend/utils/cache/spccache.c
M src/backend/utils/cache/ts_cache.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/fmgr/dfmgr.c
M src/backend/utils/fmgr/fmgr.c
M src/backend/utils/hash/dynahash.c
M src/backend/utils/mmgr/portalmem.c
M src/backend/utils/time/combocid.c
M src/include/utils/hsearch.h
M src/pl/plperl/plperl.c
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpython/plpy_plpymodule.c
M src/pl/plpython/plpy_procedure.c
M src/pl/tcl/pltcl.c
M src/timezone/pgtz.c
Revert "Cannot use WL_SOCKET_WRITEABLE without WL_SOCKET_READABLE."
commit : a58db3aa10e62e4228aa409ba006014fa07a8ca2
author : Jeff Davis <[email protected]>
date : Mon, 14 Dec 2020 23:47:30 -0800
committer: Jeff Davis <[email protected]>
date : Mon, 14 Dec 2020 23:47:30 -0800
This reverts commit 3a9e64aa0d96c8ffb6c682b082d0f72b1d373327.
Commit 4bad60e3 fixed the root of the problem that 3a9e64aa worked
around.
This enables proper pipelining of commands after terminating
replication, eliminating an undocumented limitation.
Discussion: https://postgr.es/m/3d57bc29-4459-578b-79cb-7641baf53c57%40iki.fi
Backpatch-through: 9.5
M src/backend/replication/walsender.c
Improve some code around cryptohash functions
commit : 9b584953e7bf91e342af87ef44606acd6206cd1c
author : Michael Paquier <[email protected]>
date : Mon, 14 Dec 2020 12:38:13 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 14 Dec 2020 12:38:13 +0900
This adjusts some code related to recent changes for cryptohash
functions:
- Add a variable in md5.h to track down the size of a computed result,
moved from pgcrypto. Note that pg_md5_hash() assumed a result of this
size already.
- Call explicit_bzero() on the hashed data when freeing the context for
fallback implementations. For MD5, particularly, it would be annoying
to leave some non-zeroed data around.
- Clean up some code related to recent changes of uuid-ossp. .gitignore
still included md5.c and a comment was incorrect.
Discussion: https://postgr.es/m/[email protected]
M contrib/pgcrypto/internal.c
M contrib/uuid-ossp/.gitignore
M contrib/uuid-ossp/uuid-ossp.c
M src/common/cryptohash.c
M src/common/md5_common.c
M src/include/common/md5.h
Add some checkpoint/restartpoint status to ps display
commit : df9274adf3096feafbbde2562311c8ab80405267
author : Michael Paquier <[email protected]>
date : Mon, 14 Dec 2020 11:53:58 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 14 Dec 2020 11:53:58 +0900
This is done for end-of-recovery and shutdown checkpoints/restartpoints
(end-of-recovery restartpoints don't exist) rather than all types of
checkpoints, in cases where it may not be possible to rely on
pg_stat_activity to get a status from the startup or checkpointer
processes.
For example, at the end of a crash recovery, this is useful to know if a
checkpoint is running in the startup process, while previously the ps
display may only show some information about "recovering" something,
that can be confusing while a checkpoint runs.
Author: Justin Pryzby
Reviewed-by: Nathan Bossart, Kirk Jamison, Fujii Masao, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
Use HASH_BLOBS for xidhash.
commit : a1b8aa1e4eec520ed8f11c3d134a7a866358d39a
author : Noah Misch <[email protected]>
date : Sat, 12 Dec 2020 21:38:36 -0800
committer: Noah Misch <[email protected]>
date : Sat, 12 Dec 2020 21:38:36 -0800
This caused BufFile errors on buildfarm member sungazer, and SIGSEGV was
possible. Conditions for reaching those symptoms were more frequent on
big-endian systems.
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/worker.c
M src/test/perl/PostgresNode.pm
M src/test/subscription/t/015_stream.pl
Correct behavior descriptions in comments, and correct a test name.
commit : 73aae4522b35125c4f9a506842a69336ec5d0f1b
author : Noah Misch <[email protected]>
date : Sat, 12 Dec 2020 20:12:25 -0800
committer: Noah Misch <[email protected]>
date : Sat, 12 Dec 2020 20:12:25 -0800
M src/backend/replication/logical/worker.c
M src/backend/replication/walreceiver.c
M src/test/perl/PostgresNode.pm
M src/test/recovery/t/020_archive_status.pl
initdb: complete getopt_long alphabetization
commit : d6abfdf84ef7c01d98d04f486fb467a4aec83804
author : Bruce Momjian <[email protected]>
date : Sat, 12 Dec 2020 12:59:09 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 12 Dec 2020 12:59:09 -0500
Backpatch-through: 9.5
M src/bin/initdb/initdb.c
initdb: properly alphabetize getopt_long options in C string
commit : 39f3a9d2ffcaafeb8ff25e0dbe9f28a657efb48e
author : Bruce Momjian <[email protected]>
date : Sat, 12 Dec 2020 12:51:16 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 12 Dec 2020 12:51:16 -0500
Backpatch-through: 9.5
M src/bin/initdb/initdb.c
Allow subscripting of hstore values.
commit : 0ec5f7e78231a621a1d96c4bfedc4a1849a6c6cc
author : Tom Lane <[email protected]>
date : Fri, 11 Dec 2020 18:58:07 -0500
committer: Tom Lane <[email protected]>
date : Fri, 11 Dec 2020 18:58:07 -0500
This is basically a finger exercise to prove that it's possible for
an extension module to add subscripting ability. Subscripted fetch
from an hstore is not different from the existing "hstore -> text"
operator. Subscripted update does seem to be a little easier to
use than the traditional update method using hstore concatenation,
but it's not a fundamentally new ability.
However, there may be some value in the code as sample code, since
it shows what's basically the minimum-complexity way to implement
subscripting when one needn't consider nested container objects.
Discussion: https://postgr.es/m/[email protected]
M contrib/hstore/Makefile
M contrib/hstore/expected/hstore.out
A contrib/hstore/hstore–1.7–1.8.sql
M contrib/hstore/hstore.control
A contrib/hstore/hstore_subs.c
M contrib/hstore/sql/hstore.sql
M doc/src/sgml/hstore.sgml
M doc/src/sgml/ref/create_type.sgml
Allow ALTER TYPE to update an existing type's typsubscript value.
commit : 8c15a297452e970d68529ee2ce6bd94d84598409
author : Tom Lane <[email protected]>
date : Fri, 11 Dec 2020 18:07:02 -0500
committer: Tom Lane <[email protected]>
date : Fri, 11 Dec 2020 18:07:02 -0500
This is essential if we'd like to allow existing extension data types
to support subscripting in future, since dropping and recreating the
type isn't a practical thing for an extension upgrade script, and
direct manipulation of pg_type isn't a great answer either.
There was some discussion about also allowing alteration of typelem,
but it's less clear whether that's a good idea or not, so for now
I forebore.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/alter_type.sgml
M src/backend/commands/typecmds.c
M src/test/regress/expected/create_type.out
M src/test/regress/sql/create_type.sql
Provide an error cursor for "can't subscript" error messages.
commit : 653aa603f501aa6e4865105a928cd13082ee7152
author : Tom Lane <[email protected]>
date : Fri, 11 Dec 2020 17:54:10 -0500
committer: Tom Lane <[email protected]>
date : Fri, 11 Dec 2020 17:54:10 -0500
Commit c7aba7c14 didn't add this, but after more fooling with the
feature I feel that it'd be useful. To make this possible, refactor
getSubscriptingRoutines() so that the caller is responsible for
throwing any error. (In clauses.c, I just chose to make the
most conservative assumption rather than throwing an error. We don't
expect failures there anyway really, so the code space for an error
message would be a poor investment.)
M src/backend/executor/execExpr.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/parse_node.c
M src/backend/utils/cache/lsyscache.c
M src/test/regress/expected/arrays.out
pg_dump: Don't use enums for defining bit mask values
commit : d2a2808eb444986d2fe716a48e21993329142f3d
author : Peter Eisentraut <[email protected]>
date : Fri, 11 Dec 2020 19:15:30 +0100
committer: Peter Eisentraut <[email protected]>
date : Fri, 11 Dec 2020 19:15:30 +0100
This usage would mean that values of the enum type are potentially not
one of the enum values. Use macros instead, like everywhere else.
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_utils.h
Fix compilation of uuid-ossp
commit : 525e60b7429925d09fce1b5aa0bc2f23cfe6dd18
author : Michael Paquier <[email protected]>
date : Thu, 10 Dec 2020 12:49:43 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 10 Dec 2020 12:49:43 +0900
This module had a dependency on pgcrypto's md5.c that got removed by
b67b57a. Instead of the code from pgcrypto, this code can just use the
new cryptohash routines for MD5 as a drop-in replacement, so let's just
do this switch. This has also the merit to simplify a bit the
compilation of uuid-ossp.
This requires --with-uuid to be reproduced, and I have used e2fs as a
way to reproduce the failure, then test this commit.
Per reports from buildfarm members longfin, florican and sifaka.
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
M contrib/uuid-ossp/Makefile
M contrib/uuid-ossp/uuid-ossp.c
Refactor MD5 implementations according to new cryptohash infrastructure
commit : b67b57a966af0c4a9547ac6fff334d3c256d9c2a
author : Michael Paquier <[email protected]>
date : Thu, 10 Dec 2020 11:59:10 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 10 Dec 2020 11:59:10 +0900
This commit heavily reorganizes the MD5 implementations that exist in
the tree in various aspects.
First, MD5 is added to the list of options available in cryptohash.c and
cryptohash_openssl.c. This means that if building with OpenSSL, EVP is
used for MD5 instead of the fallback implementation that Postgres had
for ages. With the recent refactoring work for cryptohash functions,
this change is straight-forward. If not building with OpenSSL, a
fallback implementation internal to src/common/ is used.
Second, this reduces the number of MD5 implementations present in the
tree from two to one, by moving the KAME implementation from pgcrypto to
src/common/, and by removing the implementation that existed in
src/common/. KAME was already structured with an init/update/final set
of routines by pgcrypto (see original pgcrypto/md5.h) for compatibility
with OpenSSL, so moving it to src/common/ has proved to be a
straight-forward move, requiring no actual manipulation of the internals
of each routine. Some benchmarking has not shown any performance gap
between both implementations.
Similarly to the fallback implementation used for SHA2, the fallback
implementation of MD5 is moved to src/common/md5.c with an internal
header called md5_int.h for the init, update and final routines. This
gets then consumed by cryptohash.c.
The original routines used for MD5-hashed passwords are moved to a
separate file called md5_common.c, also in src/common/, aimed at being
shared between all MD5 implementations as utility routines to keep
compatibility with any code relying on them.
Like the SHA2 changes, this commit had its round of tests on both Linux
and Windows, across all versions of OpenSSL supported on HEAD, with and
even without OpenSSL.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M contrib/pgcrypto/Makefile
M contrib/pgcrypto/internal.c
D contrib/pgcrypto/md5.c
M src/common/Makefile
M src/common/cryptohash.c
M src/common/cryptohash_openssl.c
M src/common/md5.c
A src/common/md5_common.c
R063 contrib/pgcrypto/md5.h src/common/md5_int.h
M src/include/common/cryptohash.h
M src/include/common/md5.h
M src/tools/msvc/Mkvcbuild.pm
M src/tools/pgindent/typedefs.list
Support subscripting of arbitrary types, not only arrays.
commit : c7aba7c14efdbd9fc1bb44b4cb83bedee0c6a6fc
author : Tom Lane <[email protected]>
date : Wed, 9 Dec 2020 12:40:37 -0500
committer: Tom Lane <[email protected]>
date : Wed, 9 Dec 2020 12:40:37 -0500
This patch generalizes the subscripting infrastructure so that any
data type can be subscripted, if it provides a handler function to
define what that means. Traditional variable-length (varlena) arrays
all use array_subscript_handler(), while the existing fixed-length
types that support subscripting use raw_array_subscript_handler().
It's expected that other types that want to use subscripting notation
will define their own handlers. (This patch provides no such new
features, though; it only lays the foundation for them.)
To do this, move the parser's semantic processing of subscripts
(including coercion to whatever data type is required) into a
method callback supplied by the handler. On the execution side,
replace the ExecEvalSubscriptingRef* layer of functions with direct
calls to callback-supplied execution routines. (Thus, essentially
no new run-time overhead should be caused by this patch. Indeed,
there is room to remove some overhead by supplying specialized
execution routines. This patch does a little bit in that line,
but more could be done.)
Additional work is required here and there to remove formerly
hard-wired assumptions about the result type, collation, etc
of a SubscriptingRef expression node; and to remove assumptions
that the subscript values must be integers.
One useful side-effect of this is that we now have a less squishy
mechanism for identifying whether a data type is a "true" array:
instead of wiring in weird rules about typlen, we can look to see
if pg_type.typsubscript == F_ARRAY_SUBSCRIPT_HANDLER. For this
to be bulletproof, we have to forbid user-defined types from using
that handler directly; but there seems no good reason for them to
do so.
This patch also removes assumptions that the number of subscripts
is limited to MAXDIM (6), or indeed has any hard-wired limit.
That limit still applies to types handled by array_subscript_handler
or raw_array_subscript_handler, but to discourage other dependencies
on this constant, I've moved it from c.h to utils/array.h.
Dmitry Dolgov, reviewed at various times by Tom Lane, Arthur Zakirov,
Peter Eisentraut, Pavel Stehule
Discussion: https://postgr.es/m/CA+q6zcVDuGBv=M0FqBYX8DPebS3F_0KQ6OVFobGJPM507_SZ_w@mail.gmail.com
Discussion: https://postgr.es/m/CA+q6zcVovR+XY4mfk-7oNk-rF91gH0PebnNfuUjuuDsyHjOcVA@mail.gmail.com
M contrib/postgres_fdw/deparse.c
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/create_type.sgml
M src/backend/catalog/aclchk.c
M src/backend/catalog/dependency.c
M src/backend/catalog/heap.c
M src/backend/catalog/pg_type.c
M src/backend/commands/typecmds.c
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/jit/llvm/llvmjit_types.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/nodeFuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/parse_coerce.c
M src/backend/parser/parse_collate.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_node.c
M src/backend/parser/parse_target.c
M src/backend/utils/adt/Makefile
M src/backend/utils/adt/arrayfuncs.c
A src/backend/utils/adt/arraysubs.c
M src/backend/utils/adt/format_type.c
M src/backend/utils/adt/jsonfuncs.c
M src/backend/utils/cache/lsyscache.c
M src/backend/utils/cache/typcache.c
M src/bin/pg_dump/pg_dump.c
M src/include/c.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/catalog/pg_type.dat
M src/include/catalog/pg_type.h
M src/include/executor/execExpr.h
M src/include/nodes/primnodes.h
A src/include/nodes/subscripting.h
M src/include/parser/parse_node.h
M src/include/utils/array.h
M src/include/utils/lsyscache.h
M src/include/utils/typcache.h
M src/pl/plperl/plperl.c
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpython/plpy_typeio.c
M src/test/regress/expected/arrays.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/expected/type_sanity.out
M src/test/regress/sql/arrays.sql
M src/test/regress/sql/opr_sanity.sql
M src/test/regress/sql/type_sanity.sql
Change get_constraint_index() to use pg_constraint.conindid
commit : 8b069ef5dca97cd737a5fd64c420df3cd61ec1c9
author : Peter Eisentraut <[email protected]>
date : Wed, 9 Dec 2020 15:12:05 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 9 Dec 2020 15:12:05 +0100
It was still using a scan of pg_depend instead of using the conindid
column that has been added since.
Since it is now just a catalog lookup wrapper and not related to
pg_depend, move from pg_depend.c to lsyscache.c.
Reviewed-by: Matthias van de Meent <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/4688d55c-9a2e-9a5a-d166-5f24fe0bf8db%40enterprisedb.com
M src/backend/catalog/pg_depend.c
M src/backend/commands/tablecmds.c
M src/backend/optimizer/util/plancat.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/cache/lsyscache.c
M src/include/catalog/dependency.h
M src/include/utils/lsyscache.h
Simplify code for getting a unicode codepoint's canonical class.
commit : 16c302f51235eaec05a1f85a11c1df04ef3a6785
author : Michael Paquier <[email protected]>
date : Wed, 9 Dec 2020 13:24:38 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 9 Dec 2020 13:24:38 +0900
Three places of unicode_norm.c use a similar logic for getting the
combining class from a codepoint. Commit 2991ac5 has added the function
get_canonical_class() for this purpose, but it was only called by the
backend. This commit refactors the code to use this function in all
the places where the combining class is retrieved from a given
codepoint.
Author: John Naylor
Discussion: https://postgr.es/m/CAFBsxsHUV7s7YrOm6hFz-Jq8Sc7K_yxTkfNZxsDV-DuM-k-gwg@mail.gmail.com
M src/common/unicode_norm.c
jit: Reference function pointer types via llvmjit_types.c.
commit : df99ddc70b971a991c5111a33f2f08bd7945d5c2
author : Andres Freund <[email protected]>
date : Mon, 7 Dec 2020 13:16:55 -0800
committer: Andres Freund <[email protected]>
date : Mon, 7 Dec 2020 13:16:55 -0800
It is error prone (see 5da871bfa1b) and verbose to manually create function
types. Add a helper that can reference a function pointer type via
llvmjit_types.c and and convert existing instances of manual creation.
Author: Andres Freund <[email protected]>
Reviewed-By: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/jit/llvm/llvmjit_types.c
M src/include/jit/llvmjit.h
Teach contain_leaked_vars that assignment SubscriptingRefs are leaky.
commit : 62ee70331336161cb44733b6c3e0811696d962aa
author : Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 17:50:54 -0500
committer: Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 17:50:54 -0500
array_get_element and array_get_slice qualify as leakproof, since
they will silently return NULL for bogus subscripts. But
array_set_element and array_set_slice throw errors for such cases,
making them clearly not leakproof. contain_leaked_vars was evidently
written with only the former case in mind, as it gave the wrong answer
for assignment SubscriptingRefs (nee ArrayRefs).
This would be a live security bug, were it not that assignment
SubscriptingRefs can only occur in INSERT and UPDATE target lists,
while we only care about leakproofness for qual expressions; so the
wrong answer can't occur in practice. Still, that's a rather shaky
answer for a security-related question; and maybe in future somebody
will want to ask about leakproofness of a tlist. So it seems wise to
fix and even back-patch this correction.
(We would need some change here anyway for the upcoming
generic-subscripting patch, since extensions might make different
tradeoffs about whether to throw errors. Commit 558d77f20 attempted
to lay groundwork for that by asking check_functions_in_node whether a
SubscriptingRef contains leaky functions; but that idea fails now that
the implementation methods of a SubscriptingRef are not SQL-visible
functions that could be marked leakproof or not.)
Back-patch to 9.6. While 9.5 has the same issue, the code's a bit
different. It seems quite unlikely that we'd introduce any actual bug
in the short time 9.5 has left to live, so the work/risk/reward balance
isn't attractive for changing 9.5.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/util/clauses.c
Remove operator_precedence_warning.
commit : a676386b58bf7cd2df81baa43eb1713d3a2ec055
author : Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 16:29:52 -0500
committer: Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 16:29:52 -0500
This GUC was always intended as a temporary solution to help with
finding 9.4-to-9.5 migration issues. Now that all pre-9.5 branches
are out of support, and 9.5 will be too before v14 is released,
it seems like it's okay to drop it. Doing so allows removal of
several hundred lines of poorly-tested code in parse_expr.c,
which have been a fertile source of bugs when people did use this.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M doc/src/sgml/syntax.sgml
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_target.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/nodes/parsenodes.h
M src/include/parser/parse_expr.h
Improve estimation of ANDs under ORs using extended statistics.
commit : 4f5760d4afa9423fe4d38e4cbec48bf5e793e7e5
author : Dean Rasheed <[email protected]>
date : Tue, 8 Dec 2020 20:10:11 +0000
committer: Dean Rasheed <[email protected]>
date : Tue, 8 Dec 2020 20:10:11 +0000
Formerly, extended statistics only handled clauses that were
RestrictInfos. However, the restrictinfo machinery doesn't create
sub-AND RestrictInfos for AND clauses underneath OR clauses.
Therefore teach extended statistics to handle bare AND clauses,
looking for compatible RestrictInfo clauses underneath them.
Dean Rasheed, reviewed by Tomas Vondra.
Discussion: https://postgr.es/m/CAEZATCW=J65GUFm50RcPv-iASnS2mTXQbr=CfBvWRVhFLJ_fWA@mail.gmail.com
M src/backend/optimizer/path/clausesel.c
M src/backend/statistics/extended_stats.c
M src/test/regress/expected/stats_ext.out
Improve estimation of OR clauses using multiple extended statistics.
commit : 88b0898fe35a5a0325fca21bd4f3ed6dffb364c1
author : Dean Rasheed <[email protected]>
date : Tue, 8 Dec 2020 19:39:24 +0000
committer: Dean Rasheed <[email protected]>
date : Tue, 8 Dec 2020 19:39:24 +0000
When estimating an OR clause using multiple extended statistics
objects, treat the estimates for each set of clauses for each
statistics object as independent of one another. The overlap estimates
produced for each statistics object do not apply to clauses covered by
other statistics objects.
Dean Rasheed, reviewed by Tomas Vondra.
Discussion: https://postgr.es/m/CAEZATCW=J65GUFm50RcPv-iASnS2mTXQbr=CfBvWRVhFLJ_fWA@mail.gmail.com
M src/backend/statistics/extended_stats.c
M src/test/regress/expected/stats_ext.out
Doc: clarify that CREATE TABLE discards redundant unique constraints.
commit : f2a69b352de1dffc534c4835010e736018aa94de
author : Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 13:09:47 -0500
committer: Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 13:09:47 -0500
The SQL standard says that redundant unique constraints are disallowed,
but we long ago decided that throwing an error would be too
user-unfriendly, so we just drop redundant ones. The docs weren't very
clear about that though, as this behavior was only explained for PRIMARY
KEY vs UNIQUE, not UNIQUE vs UNIQUE.
While here, I couldn't resist doing some copy-editing and markup-fixing
on the adjacent text about INCLUDE options.
Per bug #16767 from Matthias vd Meent.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_table.sgml
Doc: explain that the string types can't store \0 (ASCII NUL).
commit : 9a2641911aeaa7f6523dc2a465088051d4c85901
author : Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 12:06:19 -0500
committer: Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 12:06:19 -0500
This restriction was mentioned in connection with string literals,
but it wasn't made clear that it's a general restriction not just
a syntactic limitation in query strings.
Per unsigned documentation comment.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/datatype.sgml
Speed up rechecking if relation needs to be vacuumed or analyze in autovacuum.
commit : e2ac3fed3b1c3281281eb530c220634030cd8084
author : Fujii Masao <[email protected]>
date : Tue, 8 Dec 2020 23:59:39 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 8 Dec 2020 23:59:39 +0900
After autovacuum collects the relations to vacuum or analyze, it rechecks
whether each relation still needs to be vacuumed or analyzed before actually
doing that. Previously this recheck could be a significant overhead
especially when there were a very large number of relations. This was
because each recheck forced the statistics to be refreshed, and the refresh
of the statistics for a very large number of relations could cause heavy
overhead. There was the report that this issue caused autovacuum workers
to have gotten “stuck” in a tight loop of table_recheck_autovac() that
rechecks whether a relation needs to be vacuumed or analyzed.
This commit speeds up the recheck by making autovacuum worker reuse
the previously-read statistics for the recheck if possible. Then if that
"stale" statistics says that a relation still needs to be vacuumed or analyzed,
autovacuum refreshes the statistics and does the recheck again.
The benchmark shows that the more relations exist and autovacuum workers
are running concurrently, the more this change reduces the autovacuum
execution time. For example, when there are 20,000 tables and 10 autovacuum
workers are running, the benchmark showed that the change improved
the performance of autovacuum more than three times. On the other hand,
even when there are only 1000 tables and only a single autovacuum worker
is running, the benchmark didn't show any big performance regression by
the change.
Firstly POC patch was proposed by Jim Nasby. As the result of discussion,
we used Tatsuhito Kasahara's version of the patch using the approach
suggested by Tom Lane.
Reported-by: Jim Nasby
Author: Tatsuhito Kasahara
Reviewed-by: Masahiko Sawada, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/autovacuum.c
Bump catversion for pg_stat_wal changes.
commit : 4e43ee88c28b725bb63e27609c1a717138fc7f39
author : Fujii Masao <[email protected]>
date : Tue, 8 Dec 2020 21:05:27 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 8 Dec 2020 21:05:27 +0900
Oversight in 01469241b2.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/[email protected]
M src/include/catalog/catversion.h
pgcrypto: Detect errors with EVP calls from OpenSSL
commit : 28d1601ad988790c3c53d7ffd24ef6d2366d4457
author : Michael Paquier <[email protected]>
date : Tue, 8 Dec 2020 15:22:12 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 8 Dec 2020 15:22:12 +0900
The following routines are called within pgcrypto when handling digests
but there were no checks for failures:
- EVP_MD_CTX_size (can fail with -1 as of 3.0.0)
- EVP_MD_CTX_block_size (can fail with -1 as of 3.0.0)
- EVP_DigestInit_ex
- EVP_DigestUpdate
- EVP_DigestFinal_ex
A set of elog(ERROR) is added by this commit to detect such failures,
that should never happen except in the event of a processing failure
internal to OpenSSL.
Note that it would be possible to use ERR_reason_error_string() to get
more context about such errors, but these refer mainly to the internals
of OpenSSL, so it is not really obvious how useful that would be. This
is left out for simplicity.
Per report from Coverity. Thanks to Tom Lane for the discussion.
Backpatch-through: 9.5
M contrib/pgcrypto/openssl.c
jit: Correct parameter type for generated expression evaluation functions.
commit : 5da871bfa1ba41768ecd7786293d9b81dcf1b667
author : Andres Freund <[email protected]>
date : Mon, 7 Dec 2020 18:21:06 -0800
committer: Andres Freund <[email protected]>
date : Mon, 7 Dec 2020 18:21:06 -0800
clang only uses the 'i1' type for scalar booleans, not for pointers to
booleans (as the pointer might be pointing into a larger memory
allocation). Therefore a pointer-to-bool needs to the "storage" boolean.
There's no known case of wrong code generation due to this, but it seems quite
possible that it could cause problems (see e.g. 72559438f92).
Author: Andres Freund
Discussion: https://postgr.es/m/[email protected]
Backpatch: 11-, where jit support was added
M src/backend/jit/llvm/llvmjit_expr.c
jit: configure: Explicitly reference 'native' component.
commit : 9543f0861b1d9b566be88edae21f24fb1377f45c
author : Andres Freund <[email protected]>
date : Mon, 7 Dec 2020 18:12:23 -0800
committer: Andres Freund <[email protected]>
date : Mon, 7 Dec 2020 18:12:23 -0800
Until recently 'native' was implicitly included via 'orcjit', but a change
included in LLVM 11 (not yet released) removed a number of such indirect
component references.
Reported-By: Fabien COELHO <[email protected]>
Reported-By: Andres Freund <[email protected]>
Reported-By: Thomas Munro <[email protected]>
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch: 11-, where jit support was added
M config/llvm.m4
M configure
Avoid using tuple from syscache for update of pg_database.datfrozenxid
commit : 947789f1f5fb61daf663f26325cbe7cad8197d58
author : Michael Paquier <[email protected]>
date : Tue, 8 Dec 2020 12:13:19 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 8 Dec 2020 12:13:19 +0900
pg_database.datfrozenxid gets updated using an in-place update at the
end of vacuum or autovacuum. Since 96cdeae, as pg_database has a toast
relation, it is possible for a pg_database tuple to have toast values
if there is a large set of ACLs in place. In such a case, the in-place
update would fail because of the flattening of the toast values done for
the catcache entry fetched. Instead of using a copy from the catcache,
this changes the logic to fetch the copy of the tuple by directly
scanning pg_database.
Per the lack of complaints on the matter, no backpatch is done. Note
that before 96cdeae, attempting to insert such a tuple to pg_database
would cause a "row is too big" error, so the end-of-vacuum problem was
not reachable.
Author: Ashwin Agrawal, Junfeng Yang
Discussion: https://postgr.es/m/DM5PR0501MB38800D9E4605BCA72DD35557CCE10@DM5PR0501MB3880.namprd05.prod.outlook.com
M src/backend/access/heap/heapam.c
M src/backend/commands/vacuum.c
Add a couple of regression test cases related to array subscripting.
commit : 0a665bbc43c5a678331fb1b1f44274500eba6563
author : Tom Lane <[email protected]>
date : Mon, 7 Dec 2020 11:10:21 -0500
committer: Tom Lane <[email protected]>
date : Mon, 7 Dec 2020 11:10:21 -0500
Exercise some error cases that were never reached in the existing
regression tests. This is partly for code-coverage reasons, and
partly to memorialize the current behavior in advance of planned
changes for generic subscripting.
Also, I noticed that type_sanity's check to verify that all standard
types have array types was never extended when we added arrays for
all system catalog rowtypes (f7f70d5e2), nor when we added arrays
over domain types (c12d570fa). So do that. Also, since the query's
expected output isn't empty, it seems like a good idea to add an
ORDER BY to make sure the result stays stable.
M src/test/regress/expected/arrays.out
M src/test/regress/expected/rowtypes.out
M src/test/regress/expected/type_sanity.out
M src/test/regress/sql/arrays.sql
M src/test/regress/sql/rowtypes.sql
M src/test/regress/sql/type_sanity.sql
Fix more race conditions in the newly-added pg_rewind test.
commit : 6ba581cf1104261f506377823a25795f0849ce62
author : Heikki Linnakangas <[email protected]>
date : Mon, 7 Dec 2020 14:44:34 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 7 Dec 2020 14:44:34 +0200
pg_rewind looks at the control file to check what timeline a server is on.
But promotion doesn't immediately write a checkpoint, it merely writes
an end-of-recovery WAL record. If pg_rewind runs immediately after
promotion, before the checkpoint has completed, it will think think that
the server is still on the earlier timeline. We ran into this issue a long
time ago already, see commit 484a848a73f.
It's a bit bogus that pg_rewind doesn't determine the timeline correctly
until the end-of-recovery checkpoint has completed. We probably should
fix that. But for now work around it by waiting for the checkpoint
to complete before running pg_rewind, like we did in commit 484a848a73f.
In the passing, tidy up the new test a little bit. Rerder the INSERTs so
that the comments make more sense, remove a spurious CHECKPOINT call after
pg_rewind has already run, and add --debug option, so that if this fails
again, we'll have more data.
Per buildfarm failure at https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=rorqual&dt=2020-12-06%2018%3A32%3A19&stg=pg_rewind-check.
Backpatch to all supported versions.
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/bin/pg_rewind/t/008_min_recovery_point.pl
pg_dump: Reorganize dumpBaseType()
commit : 04732962462ba99cf8f8bcf6ac83932867cc96a8
author : Tom Lane <[email protected]>
date : Sun, 6 Dec 2020 22:37:40 -0500
committer: Tom Lane <[email protected]>
date : Sun, 6 Dec 2020 22:37:40 -0500
Along the same lines as ed2c7f65b and daa9fe8a5, reduce code duplication
by having just one copy of the parts of the query that are the same
across all server versions; and make the conditionals control the
smallest possible amount of code. This is in preparation for adding
another dumpable field to pg_type.
M src/bin/pg_dump/pg_dump.c
Fix fd leak in pg_verifybackup
commit : 51c388987734cb318c8fa875f382ca75a34d0010
author : Michael Paquier <[email protected]>
date : Mon, 7 Dec 2020 09:29:32 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 7 Dec 2020 09:29:32 +0900
An error code path newly-introduced by 87ae969 forgot to close a file
descriptor when verifying a file's checksum.
Per report from Coverity, via Tom Lane.
M src/bin/pg_verifybackup/pg_verifybackup.c
Fix missed step in removal of useless RESULT RTEs in the planner.
commit : e98c900993e89ad9278cdfbf0ba5495381a1faac
author : Tom Lane <[email protected]>
date : Sat, 5 Dec 2020 16:16:13 -0500
committer: Tom Lane <[email protected]>
date : Sat, 5 Dec 2020 16:16:13 -0500
Commit 4be058fe9 forgot that the append_rel_list would already be
populated at the time we remove useless result RTEs, and it might contain
PlaceHolderVars that need to be adjusted like the ones in the main parse
tree. This could lead to "no relation entry for relid N" failures later
on, when the planner tries to do something with an unadjusted PHV.
Per report from Tom Ellis. Back-patch to v12 where the bug came in.
Discussion: https://postgr.es/m/20201205173056.GF30712@cloudinit-builder
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Fix race conditions in newly-added test.
commit : 36a4ac20fcf31361bd42b63b1b3390b28827a69e
author : Heikki Linnakangas <[email protected]>
date : Fri, 4 Dec 2020 18:20:18 +0200
committer: Heikki Linnakangas <[email protected]>
date : Fri, 4 Dec 2020 18:20:18 +0200
Buildfarm has been failing sporadically on the new test. I was able to
reproduce this by adding a random 0-10 s delay in the walreceiver, just
before it connects to the primary. There's a race condition where node_3
is promoted before it has fully caught up with node_1, leading to diverged
timelines. When node_1 is later reconfigured as standby following node_3,
it fails to catch up:
LOG: primary server contains no more WAL on requested timeline 1
LOG: new timeline 2 forked off current database system timeline 1 before current recovery point 0/30000A0
That's the situation where you'd need to use pg_rewind, but in this case
it happens already when we are just setting up the actual pg_rewind
scenario we want to test, so change the test so that it waits until
node_3 is connected and fully caught up before promoting it, so that you
get a clean, controlled failover.
Also rewrite some of the comments, for clarity. The existing comments
detailed what each step in the test did, but didn't give a good overview
of the situation the steps were trying to create.
For reasons I don't understand, the test setup had to be written slightly
differently in 9.6 and 9.5 than in later versions. The 9.5/9.6 version
needed node 1 to be reinitialized from backup, whereas in later versions
it could be shut down and reconfigured to be a standby. But even 9.5 should
support "clean switchover", where primary makes sure that pending WAL is
replicated to standby on shutdown. It would be nice to figure out what's
going on there, but that's independent of pg_rewind and the scenario that
this test tests.
Discussion: https://www.postgresql.org/message-id/b0a3b95b-82d2-6089-6892-40570f8c5e60%40iki.fi
M src/bin/pg_rewind/t/008_min_recovery_point.pl
Convert elog(LOG) calls to ereport() where appropriate
commit : eb93f3a0b633ad6afb0f37391b87f460c4b0663b
author : Peter Eisentraut <[email protected]>
date : Fri, 4 Dec 2020 14:25:23 +0100
committer: Peter Eisentraut <[email protected]>
date : Fri, 4 Dec 2020 14:25:23 +0100
User-visible log messages should go through ereport(), so they are
subject to translation. Many remaining elog(LOG) calls are really
debugging calls.
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Reviewed-by: Noah Misch <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/92d6f545-5102-65d8-3c87-489f71ea0a37%40enterprisedb.com
M src/backend/access/gist/gist.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/transam/xlog.c
M src/backend/libpq/auth.c
M src/backend/libpq/hba.c
M src/backend/libpq/pqcomm.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/replication/logical/origin.c
M src/backend/storage/file/fd.c
M src/backend/utils/misc/guc.c
Remove unnecessary grammar symbols
commit : a6964bc1bb0793e20636ccb573cd2a5ad3ef7667
author : Peter Eisentraut <[email protected]>
date : Fri, 4 Dec 2020 11:09:29 +0100
committer: Peter Eisentraut <[email protected]>
date : Fri, 4 Dec 2020 11:09:29 +0100
Instead of publication_name_list, we can use name_list. We already
refer to publications everywhere else by the 'name' or 'name_list'
symbols, so this only improves consistency.
Reviewed-by: https://www.postgresql.org/message-id/flat/3e3ccddb-41bd-ecd8-29fe-195e34d9886f%40enterprisedb.com
Discussion: Tom Lane <[email protected]>
M src/backend/parser/gram.y
Remove incorrect assertion in reorderbuffer.c.
commit : 8ae4ef4fb0e0331f02c4615182600546c8e5c4d4
author : Amit Kapila <[email protected]>
date : Fri, 4 Dec 2020 13:54:50 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 4 Dec 2020 13:54:50 +0530
We start recording changes in ReorderBufferTXN even before we reach
SNAPBUILD_CONSISTENT state so that if the commit is encountered after
reaching that we should be able to send the changes of the entire transaction.
Now, while recording changes if the reorder buffer memory has exceeded
logical_decoding_work_mem then we can start streaming if it is allowed and
we haven't yet streamed that data. However, we must not allow streaming to
start unless the snapshot has reached SNAPBUILD_CONSISTENT state.
In passing, improve the comments atop ReorderBufferResetTXN to mention the
case when we need to continue streaming after getting an error.
Author: Amit Kapila
Reviewed-by: Dilip Kumar
Discussion: https://postgr.es/m/CAA4eK1KoOH0byboyYY40NBcC7Fe812trwTa+WY3jQF7WQWZbQg@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
Rename cryptohashes.c to cryptohashfuncs.c
commit : bd94a9c04e04bb3b626e88981a50fcca2bd99d60
author : Michael Paquier <[email protected]>
date : Fri, 4 Dec 2020 12:58:44 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 4 Dec 2020 12:58:44 +0900
87ae969 has created two new files called cryptohash{_openssl}.c in
src/common/, whose names overlap with the existing backend file called
cryptohashes.c dedicated to the SQL wrappers for SHA2 and MD5. This
file is renamed to cryptohashfuncs.c to be more consistent with the
surroundings and reduce the confusion with the new cryptohash interface
of src/common/.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/Makefile
R098 src/backend/utils/adt/cryptohashes.c src/backend/utils/adt/cryptohashfuncs.c
Change SHA2 implementation based on OpenSSL to use EVP digest routines
commit : 4f48a6fbe2b28d8281dbbfa2d334fa2ed8472734
author : Michael Paquier <[email protected]>
date : Fri, 4 Dec 2020 10:49:23 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 4 Dec 2020 10:49:23 +0900
The use of low-level hash routines is not recommended by upstream
OpenSSL since 2000, and pgcrypto already switched to EVP as of 5ff4a67.
This takes advantage of the refactoring done in 87ae969 that has
introduced the allocation and free routines for cryptographic hashes.
Since 1.1.0, OpenSSL does not publish the contents of the cryptohash
contexts, forcing any consumers to rely on OpenSSL for all allocations.
Hence, the resource owner callback mechanism gains a new set of routines
to track and free cryptohash contexts when using OpenSSL, preventing any
risks of leaks in the backend. Nothing is needed in the frontend thanks
to the refactoring of 87ae969, and the resowner knowledge is isolated
into cryptohash_openssl.c.
Note that this also fixes a failure with SCRAM authentication when using
FIPS in OpenSSL, but as there have been few complaints about this
problem and as this causes an ABI breakage, no backpatch is done.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson, Heikki Linnakangas
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/basebackup.c
M src/backend/utils/resowner/resowner.c
M src/common/cryptohash_openssl.c
M src/include/utils/resowner_private.h
M src/tools/pgindent/typedefs.list
doc: remove unnecessary blank before command option text
commit : 3f8971d92e767acf6e3d6e27c4cab7bfd88b71f4
author : Bruce Momjian <[email protected]>
date : Thu, 3 Dec 2020 11:33:24 -0500
committer: Bruce Momjian <[email protected]>
date : Thu, 3 Dec 2020 11:33:24 -0500
Backpatch-through: 11
M doc/src/sgml/ref/pg_checksums.sgml
M doc/src/sgml/ref/pg_controldata.sgml
M doc/src/sgml/ref/pg_resetwal.sgml
docs: list single-letter options first in command-line summary
commit : a659e789b743f1a4b977a1ffc7402ab618a9f68c
author : Bruce Momjian <[email protected]>
date : Thu, 3 Dec 2020 10:28:58 -0500
committer: Bruce Momjian <[email protected]>
date : Thu, 3 Dec 2020 10:28:58 -0500
In a few places, the long-version options were listed before the
single-letter ones in the command summary of a few commands. This
didn't match other commands, and didn't match the option ordering later
in the same reference page.
Backpatch-through: 9.5
M doc/src/sgml/ref/pg_controldata.sgml
M doc/src/sgml/ref/pg_resetwal.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
Fix pg_rewind bugs when rewinding a standby server.
commit : 2b4f3130382fe2f8705863e4d38589d4d69cd695
author : Heikki Linnakangas <[email protected]>
date : Thu, 3 Dec 2020 15:57:48 +0200
committer: Heikki Linnakangas <[email protected]>
date : Thu, 3 Dec 2020 15:57:48 +0200
If the target is a standby server, its WAL doesn't end at the last
checkpoint record, but at minRecoveryPoint. We must scan all the
WAL from the last common checkpoint all the way up to minRecoveryPoint
for modified pages, and also consider that portion when determining
whether the server needs rewinding.
Backpatch to all supported versions.
Author: Ian Barwick and me
Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
A src/bin/pg_rewind/t/008_min_recovery_point.pl
Small code simplifications
commit : 6114040711affa2b0bcf47fa2791187daf8455fb
author : Peter Eisentraut <[email protected]>
date : Thu, 3 Dec 2020 11:35:20 +0100
committer: Peter Eisentraut <[email protected]>
date : Thu, 3 Dec 2020 11:35:20 +0100
strVal() can be used in a couple of places instead of coding the same
thing by hand.
M src/backend/foreign/foreign.c
M src/backend/parser/gram.y
Improve estimation of OR clauses using extended statistics.
commit : 25a9e54d2db31b8031c2d8166114b187e8347098
author : Dean Rasheed <[email protected]>
date : Thu, 3 Dec 2020 10:03:49 +0000
committer: Dean Rasheed <[email protected]>
date : Thu, 3 Dec 2020 10:03:49 +0000
Formerly we only applied extended statistics to an OR clause as part
of the clauselist_selectivity() code path for an OR clause appearing
in an implicitly-ANDed list of clauses. This meant that it could only
use extended statistics if all sub-clauses of the OR clause were
covered by a single extended statistics object.
Instead, teach clause_selectivity() how to apply extended statistics
to an OR clause by handling its ORed list of sub-clauses in a similar
manner to an implicitly-ANDed list of sub-clauses, but with different
combination rules. This allows one or more extended statistics objects
to be used to estimate all or part of the list of sub-clauses. Any
remaining sub-clauses are then treated as if they are independent.
Additionally, to avoid double-application of extended statistics, this
introduces "extended" versions of clause_selectivity() and
clauselist_selectivity(), which include an option to ignore extended
statistics. This replaces the old clauselist_selectivity_simple()
function which failed to completely ignore extended statistics when
called from the extended statistics code.
A known limitation of the current infrastructure is that an AND clause
under an OR clause is not treated as compatible with extended
statistics (because we don't build RestrictInfos for such sub-AND
clauses). Thus, for example, "(a=1 AND b=1) OR (a=2 AND b=2)" will
currently be treated as two independent AND clauses (each of which may
be estimated using extended statistics), but extended statistics will
not currently be used to account for any possible overlap between
those clauses. Improving that is left as a task for the future.
Original patch by Tomas Vondra, with additional improvements by me.
Discussion: https://postgr.es/m/20200113230008.g67iyk4cs3xbnjju@development
M src/backend/optimizer/path/clausesel.c
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/statistics/mcv.c
M src/include/optimizer/optimizer.h
M src/include/statistics/extended_stats_internal.h
M src/include/statistics/statistics.h
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Refactor CLUSTER and REINDEX grammar to use DefElem for option lists
commit : b5913f6120792465f4394b93c15c2e2ac0c08376
author : Michael Paquier <[email protected]>
date : Thu, 3 Dec 2020 10:13:21 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 3 Dec 2020 10:13:21 +0900
This changes CLUSTER and REINDEX so as a parenthesized grammar becomes
possible for options, while unifying the grammar parsing rules for
option lists with the existing ones.
This is a follow-up of the work done in 873ea9e for VACUUM, ANALYZE and
EXPLAIN. This benefits REINDEX for a potential backend-side filtering
for collatable-sensitive indexes and TABLESPACE, while CLUSTER would
benefit from the latter.
Author: Alexey Kondratov, Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/cluster.sgml
M doc/src/sgml/ref/reindex.sgml
M src/backend/commands/cluster.c
M src/backend/commands/indexcmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/parser/gram.y
M src/backend/tcop/utility.c
M src/bin/psql/tab-complete.c
M src/include/catalog/index.h
M src/include/commands/cluster.h
M src/include/commands/defrem.h
M src/include/nodes/parsenodes.h
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Add GSS information to connection authorized log message
commit : dc11f31a1a891f8aa8890644e837556bcc5a75e7
author : Stephen Frost <[email protected]>
date : Wed, 2 Dec 2020 14:41:53 -0500
committer: Stephen Frost <[email protected]>
date : Wed, 2 Dec 2020 14:41:53 -0500
GSS information (if used) such as if the connection was authorized using
GSS or if it was encrypted using GSS, and perhaps most importantly, what
the GSS principal used for the authentication was, is extremely useful
but wasn't being included in the connection authorized log message.
Therefore, add to the connection authorized log message that
information, in a similar manner to how we log SSL information when SSL
is used for a connection.
Author: Vignesh C
Reviewed-by: Bharath Rupireddy
Discussion: https://www.postgresql.org/message-id/CALDaNm2N1385_Ltoo%3DS7VGT-ESu_bRQa-sC1wg6ikrM2L2Z49w%40mail.gmail.com
M src/backend/utils/init/postinit.c
M src/test/kerberos/t/001_auth.pl
Track total number of WAL records, FPIs and bytes generated in the cluster.
commit : 01469241b2ff6f17fc0b3f20f01559145ffab29e
author : Fujii Masao <[email protected]>
date : Wed, 2 Dec 2020 13:00:15 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 2 Dec 2020 13:00:15 +0900
Commit 6b466bf5f2 allowed pg_stat_statements to track the number of
WAL records, full page images and bytes that each statement generated.
Similarly this commit allows us to track the cluster-wide WAL statistics
counters.
New columns wal_records, wal_fpi and wal_bytes are added into the
pg_stat_wal view, and reports the total number of WAL records,
full page images and bytes generated in the , respectively.
Author: Masahiro Ikeda
Reviewed-by: Amit Kapila, Movead Li, Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
Fix compilation warnings in cryptohash_openssl.c
commit : 91624c2ff8809145880383b0fa84be0ee98f55b5
author : Michael Paquier <[email protected]>
date : Wed, 2 Dec 2020 12:31:10 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 2 Dec 2020 12:31:10 +0900
These showed up with -O2. Oversight in 87ae969.
Author: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/common/cryptohash_openssl.c
Allow restore_command parameter to be changed with reload.
commit : 942305a36365433eff3c1937945758f2dbf1662b
author : Fujii Masao <[email protected]>
date : Wed, 2 Dec 2020 11:00:15 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 2 Dec 2020 11:00:15 +0900
This commit changes restore_command from PGC_POSTMASTER to PGC_SIGHUP.
As the side effect of this commit, restore_command can be reset to
empty during archive recovery. In this setting, archive recovery
tries to replay only WAL files available in pg_wal directory. This is
the same behavior as when the command that always fails is specified
in restore_command.
Note that restore_command still must be specified (not empty) when
starting archive recovery, even after applying this commit. This is
necessary as the safeguard to prevent users from forgetting to
specify restore_command and starting archive recovery.
Thanks to Peter Eisentraut, Michael Paquier, Andres Freund,
Robert Haas and Anastasia Lubennikova for discussion.
Author: Sergei Kornilov
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
Move SHA2 routines to a new generic API layer for crypto hashes
commit : 87ae9691d25379785f8c0f81b06a14818cfd8c56
author : Michael Paquier <[email protected]>
date : Wed, 2 Dec 2020 10:37:20 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 2 Dec 2020 10:37:20 +0900
Two new routines to allocate a hash context and to free it are created,
as these become necessary for the goal behind this refactoring: switch
the all cryptohash implementations for OpenSSL to use EVP (for FIPS and
also because upstream does not recommend the use of low-level cryptohash
functions for 20 years). Note that OpenSSL hides the internals of
cryptohash contexts since 1.1.0, so it is necessary to leave the
allocation to OpenSSL itself, explaining the need for those two new
routines. This part is going to require more work to properly track
hash contexts with resource owners, but this not introduced here.
Still, this refactoring makes the move possible.
This reduces the number of routines for all SHA2 implementations from
twelve (SHA{224,256,386,512} with init, update and final calls) to five
(create, free, init, update and final calls) by incorporating the hash
type directly into the hash context data.
The new cryptohash routines are moved to a new file, called cryptohash.c
for the fallback implementations, with SHA2 specifics becoming a part
internal to src/common/. OpenSSL specifics are part of
cryptohash_openssl.c. This infrastructure is usable for more hash
types, like MD5 or HMAC.
Any code paths using the internal SHA2 routines are adapted to report
correctly errors, which are most of the changes of this commit. The
zones mostly impacted are checksum manifests, libpq and SCRAM.
Note that e21cbb4 was a first attempt to switch SHA2 to EVP, but it
lacked the refactoring needed for libpq, as done here.
This patch has been tested on Linux and Windows, with and without
OpenSSL, and down to 1.0.1, the oldest version supported on HEAD.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M contrib/pgcrypto/internal-sha2.c
M src/backend/libpq/auth-scram.c
M src/backend/replication/backup_manifest.c
M src/backend/replication/basebackup.c
M src/backend/utils/adt/cryptohashes.c
M src/bin/pg_verifybackup/parse_manifest.c
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/common/Makefile
M src/common/checksum_helper.c
A src/common/cryptohash.c
A src/common/cryptohash_openssl.c
M src/common/scram-common.c
M src/common/sha2.c
A src/common/sha2_int.h
D src/common/sha2_openssl.c
M src/include/common/checksum_helper.h
A src/include/common/cryptohash.h
M src/include/common/scram-common.h
M src/include/common/sha2.h
M src/include/replication/backup_manifest.h
M src/interfaces/libpq/fe-auth-scram.c
M src/tools/msvc/Mkvcbuild.pm
M src/tools/pgindent/typedefs.list
pg_checksums: data_checksum_version is unsigned so use %u not %d
commit : 888671a8cda5896d36d3ec523e201ab6a11e1855
author : Bruce Momjian <[email protected]>
date : Tue, 1 Dec 2020 20:27:06 -0500
committer: Bruce Momjian <[email protected]>
date : Tue, 1 Dec 2020 20:27:06 -0500
While the previous behavior didn't generate a warning, we might as well
use an accurate *printf specification.
Backpatch-through: 12
M src/bin/pg_checksums/pg_checksums.c
Ensure that expandTableLikeClause() re-examines the same table.
commit : f7f83a55bf6051818a0e4387d718867ecfa8561b
author : Tom Lane <[email protected]>
date : Tue, 1 Dec 2020 14:02:27 -0500
committer: Tom Lane <[email protected]>
date : Tue, 1 Dec 2020 14:02:27 -0500
As it stood, expandTableLikeClause() re-did the same relation_openrv
call that transformTableLikeClause() had done. However there are
scenarios where this would not find the same table as expected.
We hold lock on the LIKE source table, so it can't be renamed or
dropped, but another table could appear before it in the search path.
This explains the odd behavior reported in bug #16758 when cloning a
table as a temp table of the same name. This case worked as expected
before commit 502898192 introduced the need to open the source table
twice, so we should fix it.
To make really sure we get the same table, let's re-open it by OID not
name. That requires adding an OID field to struct TableLikeClause,
which is a little nervous-making from an ABI standpoint, but as long
as it's at the end I don't think there's any serious risk.
Per bug #16758 from Marc Boeren. Like the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_utilcmd.c
M src/include/nodes/parsenodes.h
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Avoid memcpy() with a NULL source pointer and count == 0
commit : 677f74e5bb8360122ebf2f473d7169ed65ce4dba
author : Alvaro Herrera <[email protected]>
date : Tue, 1 Dec 2020 11:46:56 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 1 Dec 2020 11:46:56 -0300
When memcpy() is called on a pointer, the compiler is entitled to assume
that the pointer is not null, which can lead to optimizing nearby code
in potentially undesirable ways. We still want such optimizations
(gcc's -fdelete-null-pointer-checks) in cases where they're valid.
Related: commit 13bba02271dc.
Backpatch to pg11, where this particular instance appeared.
Reported-by: Ranier Vilela <[email protected]>
Reported-by: Zhihong Yu <[email protected]>
Discussion: https://postgr.es/m/CAEudQApUndmQkr5fLrCKXQ7+ib44i7S+Kk93pyVThS85PnG3bQ@mail.gmail.com
Discussion: https://postgr.es/m/CALNJ-vSdhwSM5f4tnNn1cdLHvXMVe_S+V3nR5GwNrmCPNB2VtQ@mail.gmail.com
M src/backend/commands/indexcmds.c
docs: ulink all references to RFC's
commit : 415dc2009683904f337a1837b6b4eb7f31c4dc55
author : Heikki Linnakangas <[email protected]>
date : Tue, 1 Dec 2020 14:36:30 +0200
committer: Heikki Linnakangas <[email protected]>
date : Tue, 1 Dec 2020 14:36:30 +0200
Make sure that the first mentions of RFC's are ulinked to their ietf.org
entry, and subsequent ones are marked as acronyms. This makes references
to RFC's consistent across the documentation.
Author: Daniel Gustafsson
Discussion: https://www.postgresql.org/message-id/2C697878-4D01-4F06-8312-2FEDE931E973%40yesql.se
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/charset.sgml
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/json.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/textsearch.sgml
M doc/src/sgml/uuid-ossp.sgml
doc: Add additional index entries for progress reporting views.
commit : 0a4db67b5ed05c4013ea968930af36853f088404
author : Fujii Masao <[email protected]>
date : Tue, 1 Dec 2020 17:19:23 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 1 Dec 2020 17:19:23 +0900
In the docs, the index entries for progress reporting views link to
the "Viewing Statistics" section, but previously they did not link to
the dedicated section (e.g., "ANALYZE Progress Reporting") for
each view. This was inconvenient when finding the section describing
the detailed information about each view, from the index.
This commit adds additional index entries linking to those dedicated
sections.
Author: Fujii Masao
Reviewed-by: Shinya Kato
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
doc: Remove more notes about compatibilities with past versions
commit : 8a17f44c1e7a9f3d2a9da97dc3eba4184a2a453c
author : Michael Paquier <[email protected]>
date : Tue, 1 Dec 2020 16:32:26 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 1 Dec 2020 16:32:26 +0900
This is a follow-up of the work done in fa42c2e, that did not include
all the fixes previously agreed on. The contents removed here can be
confusing to the reader as they refer to rather old server versions.
Author: Stephen Frost, Tom Lane, Heikki Linnakangas, Yaroslav Schekin
Discussion: https://postgr.es/m/CAB8KJ=jYHgnxLLZSNJz7gBTck4TxomngCmGkw3nEMSNF0yL6wA@mail.gmail.com
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/ref/select.sgml
Use truncate(2) where appropriate.
commit : 57faaf376e1961fa48866c6e5d6926463c6671b1
author : Thomas Munro <[email protected]>
date : Tue, 1 Dec 2020 15:34:57 +1300
committer: Thomas Munro <[email protected]>
date : Tue, 1 Dec 2020 15:34:57 +1300
When truncating files by name, use truncate(2). Windows hasn't got it,
so keep our previous coding based on ftruncate(2) as a fallback.
Discussion: https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org
M src/backend/storage/file/fd.c
M src/backend/storage/smgr/md.c
M src/include/storage/fd.h
Free disk space for dropped relations on commit.
commit : 9f35f943732eaf9121a107f54ea043925e9a5d7b
author : Thomas Munro <[email protected]>
date : Tue, 1 Dec 2020 13:21:03 +1300
committer: Thomas Munro <[email protected]>
date : Tue, 1 Dec 2020 13:21:03 +1300
When committing a transaction that dropped a relation, we previously
truncated only the first segment file to free up disk space (the one
that won't be unlinked until the next checkpoint).
Truncate higher numbered segments too, even though we unlink them on
commit. This frees the disk space immediately, even if other backends
have open file descriptors and might take a long time to get around to
handling shared invalidation events and closing them. Also extend the
same behavior to the first segment, in recovery.
Back-patch to all supported releases.
Bug: #16663
Reported-by: Denis Patron <[email protected]>
Reviewed-by: Pavel Borisov <[email protected]>
Reviewed-by: Neil Chen <[email protected]>
Reviewed-by: David Zhang <[email protected]>
Discussion: https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org
M src/backend/storage/smgr/md.c
Fix missing outfuncs.c support for IncrementalSortPath.
commit : 8286223f3d820c39f2d5f14222f7ccde53bdf502
author : Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 16:32:56 -0500
committer: Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 16:32:56 -0500
For debugging purposes, Path nodes are supposed to have outfuncs
support, but this was overlooked in the original incremental sort patch.
While at it, clean up a couple other minor oversights, as well as
bizarre choice of return type for create_incremental_sort_path().
(All the existing callers just cast it to "Path *" immediately, so
they don't care, but some future caller might care.)
outfuncs.c fix by Zhijie Hou, the rest by me
Discussion: https://postgr.es/m/324c4d81d8134117972a5b1f6cdf9560@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/README
M src/backend/optimizer/util/pathnode.c
M src/include/nodes/pathnodes.h
M src/include/optimizer/pathnode.h
Document concurrent indexes waiting on each other
commit : 58ebe967f8a1e34a824d6a0a35728027272041c6
author : Alvaro Herrera <[email protected]>
date : Mon, 30 Nov 2020 18:24:55 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 30 Nov 2020 18:24:55 -0300
Because regular CREATE INDEX commands are independent, and there's no
logical data dependency, it's not immediately obvious that transactions
held by concurrent index builds on one table will block the second phase
of concurrent index creation on an unrelated table, so document this
caveat.
Backpatch this all the way back. In branch master, mention that only
some indexes are involved.
Author: James Coleman <[email protected]>
Reviewed-by: David Johnston <[email protected]>
Discussion: https://postgr.es/m/CAAaqYe994=PUrn8CJZ4UEo_S-FfRr_3ogERyhtdgHAb2WG_Ufg@mail.gmail.com
M doc/src/sgml/ref/create_index.sgml
Remove configure-time probe for DocBook DTD.
commit : 4823c4f6ac770ebd49b63b0ce64ecfe82987af57
author : Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 15:24:13 -0500
committer: Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 15:24:13 -0500
Checking for DocBook being installed was valuable when we were on the
OpenSP docs toolchain, because that was rather hard to get installed
fully. Nowadays, as long as you have xmllint and xsltproc installed,
you're good, because those programs will fetch the DocBook files off
the net at need. Moreover, testing this at configure time means that
a network access may well occur whether or not you have any interest
in building the docs later. That can be slow (typically 2 or 3
seconds, though much higher delays have been reported), and it seems
not very nice to be doing an off-machine access without warning, too.
Hence, drop the PGAC_CHECK_DOCBOOK probe, and adjust related
documentation. Without that macro, there's not much left of
config/docbook.m4 at all, so I just removed it.
Back-patch to v11, where we started to use xmllint in the
PGAC_CHECK_DOCBOOK probe.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M aclocal.m4
D config/docbook.m4
M configure
M configure.ac
M doc/src/sgml/docguide.sgml
Prevent parallel index build in a standalone backend.
commit : 275b3411d9189b3974687766db06727d64d22979
author : Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 14:38:00 -0500
committer: Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 14:38:00 -0500
This can't work if there's no postmaster, and indeed the code got an
assertion failure trying. There should be a check on IsUnderPostmaster
gating the use of parallelism, as the planner has for ordinary
parallel queries.
Commit 40d964ec9 got this right, so follow its model of checking
IsUnderPostmaster at the same place where we check for
max_parallel_maintenance_workers == 0. In general, new code
implementing parallel utility operations should do the same.
Report and patch by Yulin Pei, cosmetically adjusted by me.
Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/HK0PR01MB22747D839F77142D7E76A45DF4F50@HK0PR01MB2274.apcprd01.prod.exchangelabs.com
M src/backend/optimizer/plan/planner.c
Fix miscomputation of direct_lateral_relids for join relations.
commit : b1738ff6ab73203cbbc02d7fb82941dbc061d301
author : Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 12:22:43 -0500
committer: Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 12:22:43 -0500
If a PlaceHolderVar is to be evaluated at a join relation, but
its value is only needed there and not at higher levels, we neglected
to update the joinrel's direct_lateral_relids to include the PHV's
source rel. This causes problems because join_is_legal() then won't
allow joining the joinrel to the PHV's source rel at all, leading
to "failed to build any N-way joins" planner failures.
Per report from Andreas Seltenreich. Back-patch to 9.5
where the problem originated.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/util/placeholder.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Refactor parsing rules for option lists of EXPLAIN, VACUUM and ANALYZE
commit : 873ea9ee692e7829614f913685db540b17998ba6
author : Michael Paquier <[email protected]>
date : Mon, 30 Nov 2020 20:27:37 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 30 Nov 2020 20:27:37 +0900
Those three commands have been using the same grammar rules to handle a
a list of parenthesized options. This refactors the code so as they use
the same parsing rules, shaving some code. A future commit will make
use of those option parsing rules for more utility commands, like
REINDEX and CLUSTER.
Author: Alexey Kondratov, Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/gram.y
Remove leftover comments, left behind by removal of WITH OIDS.
commit : 2bc588798bdd0cdaa8f6cb6713ba8c8cc039fcb1
author : Heikki Linnakangas <[email protected]>
date : Mon, 30 Nov 2020 10:26:43 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 30 Nov 2020 10:26:43 +0200
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqGaRoF3XrhPW-Y7P%2BG7bKo84Z_h%3DkQHvMh-80%3Dav3wmOw%40mail.gmail.com
M contrib/file_fdw/file_fdw.c
M src/backend/commands/copyfromparse.c
Fix typo in comment.
commit : 6742e14959a3033d946ab3d67f5ce4c99367d332
author : Fujii Masao <[email protected]>
date : Mon, 30 Nov 2020 12:54:31 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 30 Nov 2020 12:54:31 +0900
Author: Haiying Tang <[email protected]>
Discussion: https://postgr.es/m/48a0928ac94b497d9c40acf1de394c15@G08CNEXMBPEKD05.g08.fujitsu.local
M src/include/optimizer/cost.h
Improve log message about termination of background workers.
commit : 98e2d58d66c81d9f1e49813c1acd4d7312089f4a
author : Fujii Masao <[email protected]>
date : Mon, 30 Nov 2020 11:05:19 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 30 Nov 2020 11:05:19 +0900
Previously the shutdown of a background worker that uses die() as
SIGTERM signal handler produced the log message "terminating
connection due to administrator command". This log message was
confusing because a background worker is not a connection.
This commit improves that log message to "terminating background
worker XXX due to administrator command" (XXX is replaced with
the name of the background worker). This is the same log message
as another SIGTERM signal handler bgworker_die() for a background
worker reports.
Author: Bharath Rupireddy
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/tcop/postgres.c
Fix recently-introduced breakage in psql's \connect command.
commit : 7e5e1bba034ee02245e8a3833aa1f6ea7253b584
author : Tom Lane <[email protected]>
date : Sun, 29 Nov 2020 15:22:04 -0500
committer: Tom Lane <[email protected]>
date : Sun, 29 Nov 2020 15:22:04 -0500
Through my misreading of what the existing code actually did,
commits 85c54287a et al. broke psql's behavior for the case where
"\c connstring" provides a password in the connstring. We should
use that password in such a case, but as of 85c54287a we ignored it
(and instead, prompted for a password).
Commit 94929f1cf fixed that in HEAD, but since I thought it was
cleaning up a longstanding misbehavior and not one I'd just created,
I didn't back-patch it.
Hence, back-patch the portions of 94929f1cf having to do with
password management. In addition to fixing the introduced bug,
this means that "\c -reuse-previous=on connstring" will allow
re-use of an existing connection's password if the connstring
doesn't change user/host/port. That didn't happen before, but
it seems like a bug fix, and anyway I'm loath to have significant
differences in this code across versions.
Also fix an error with the same root cause about whether or not to
override a connstring's setting of client_encoding. As of 85c54287a
we always did so; restore the previous behavior of overriding only
when stdin/stdout are a terminal and there's no environment setting
of PGCLIENTENCODING. (I find that definition a bit surprising, but
right now doesn't seem like the time to revisit it.)
Per bug #16746 from Krzysztof Gradek. As with the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/command.c
Doc: clarify behavior of PQconnectdbParams().
commit : d5e2bdf7dda712a60234f08e036ec9de28085a2d
author : Tom Lane <[email protected]>
date : Sun, 29 Nov 2020 13:58:30 -0500
committer: Tom Lane <[email protected]>
date : Sun, 29 Nov 2020 13:58:30 -0500
The documentation omitted the critical tidbit that a keyword-array entry
is simply ignored if its corresponding value-array entry is NULL or an
empty string; it will *not* override any previously-obtained value for
the parameter. (See conninfo_array_parse().) I'd supposed that would
force the setting back to default, which is what led me into bug #16746;
but it doesn't.
While here, I couldn't resist the temptation to do some copy-editing,
both in the description of PQconnectdbParams() and in the section
about connection URI syntax.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
Retry initial slurp_file("current_logfiles"), in test 004_logrotate.pl.
commit : 0f89ca083bbf71641cf61b4b951d6b30a2fd978b
author : Noah Misch <[email protected]>
date : Sat, 28 Nov 2020 21:52:27 -0800
committer: Noah Misch <[email protected]>
date : Sat, 28 Nov 2020 21:52:27 -0800
Buildfarm member topminnow failed when the test script attempted this
before the syslogger would have created the file. Back-patch to v12,
which introduced the test.
M src/bin/pg_ctl/t/004_logrotate.pl
Clean up after tests in src/test/locale/.
commit : b90a7fe15f78b1e2513cbcb27a33f4adc47bcd3d
author : Tom Lane <[email protected]>
date : Sat, 28 Nov 2020 16:08:36 -0500
committer: Tom Lane <[email protected]>
date : Sat, 28 Nov 2020 16:08:36 -0500
Oversight in 257836a75, which added these tests.
M src/test/locale/Makefile
Fix a recently-introduced race condition in LISTEN/NOTIFY handling.
commit : 9c83b54a9ccdb111ce693ada2309475197c19d70
author : Tom Lane <[email protected]>
date : Sat, 28 Nov 2020 14:03:40 -0500
committer: Tom Lane <[email protected]>
date : Sat, 28 Nov 2020 14:03:40 -0500
Commit 566372b3d fixed some race conditions involving concurrent
SimpleLruTruncate calls, but it introduced new ones in async.c.
A newly-listening backend could attempt to read Notify SLRU pages that
were in process of being truncated, possibly causing an error. Also,
the QUEUE_TAIL pointer could become set to a value that's not equal to
the queue position of any backend. While that's fairly harmless in
v13 and up (thanks to commit 51004c717), in older branches it resulted
in near-permanent disabling of the queue truncation logic, so that
continued use of NOTIFY led to queue-fill warnings and eventual
inability to send any more notifies. (A server restart is enough to
make that go away, but it's still pretty unpleasant.)
The core of the problem is confusion about whether QUEUE_TAIL
represents the "logical" tail of the queue (i.e., the oldest
still-interesting data) or the "physical" tail (the oldest data we've
not yet truncated away). To fix, split that into two variables.
QUEUE_TAIL regains its definition as the logical tail, and we
introduce a new variable to track the oldest un-truncated page.
Per report from Mikael Gustavsson. Like the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/async.c
Fix CLUSTER progress reporting of number of blocks scanned.
commit : 3df51ca8b39f08ef19a77b9776f2547c86b70c49
author : Fujii Masao <[email protected]>
date : Fri, 27 Nov 2020 20:16:44 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 27 Nov 2020 20:16:44 +0900
Previously pg_stat_progress_cluster view reported the current block
number in heap scan as the number of heap blocks scanned (i.e.,
heap_blks_scanned). This reported number could be incorrect when
synchronize_seqscans is enabled, because it allowed the heap scan to
start at block in middle. This could result in wraparounds in the
heap_blks_scanned column when the heap scan wrapped around.
This commit fixes the bug by calculating the number of blocks from
the block that the heap scan starts at to the current block in scan,
and reporting that number in the heap_blks_scanned column.
Also, in pg_stat_progress_cluster view, previously heap_blks_scanned
could not reach heap_blks_total at the end of heap scan phase
if the last pages scanned were empty. This commit fixes the bug by
manually updating heap_blks_scanned to the same value as
heap_blks_total when the heap scan phase finishes.
Back-patch to v12 where pg_stat_progress_cluster view was introduced.
Reported-by: Matthias van de Meent
Author: Matthias van de Meent
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAEze2WjCBWSGkVfYag001Rc4+-nNLDpWM7QbyD6yPvuhKs-gYQ@mail.gmail.com
M src/backend/access/heap/heapam_handler.c
Use standard SIGTERM signal handler die() in test_shm_mq worker.
commit : ef848f4ac5a4bd072c65867186268775acfb4298
author : Fujii Masao <[email protected]>
date : Fri, 27 Nov 2020 15:45:01 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 27 Nov 2020 15:45:01 +0900
Previously test_shm_mq worker used the stripped-down version of die()
as the SIGTERM signal handler. This commit makes it use die(), instead,
to simplify the code.
In terms of the code, the difference between die() and the stripped-down
version previously used is whether the signal handler directly may call
ProcessInterrupts() or not. But this difference doesn't exist in
a background worker because, in bgworker, DoingCommandRead flag will
never be true and die() will never call ProcessInterrupts() directly.
Therefore test_shm_mq worker can safely use die(), like other bgworker
proceses (e.g., logical replication apply launcher or autoprewarm worker)
currently do.
Thanks to Craig Ringer for the report and investigation of the issue.
Author: Bharath Rupireddy
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAGRY4nxsAe_1k_9g5b47orA0S011iBoHsXHFMH7cg7HV0O1bwQ@mail.gmail.com
M src/test/modules/test_shm_mq/worker.c
Use standard SIGHUP and SIGTERM signal handlers in worker_spi.
commit : 2a0847720ae6fa4180bfbe404e4e6bbf2f737708
author : Fujii Masao <[email protected]>
date : Fri, 27 Nov 2020 15:11:19 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 27 Nov 2020 15:11:19 +0900
Previously worker_spi used its custom signal handlers for SIGHUP and
SIGTERM. This commit makes worker_spi use the standard signal handlers,
to simplify the code.
Note that die() is used as the standard SIGTERM signal handler in
worker_spi instead of SignalHandlerForShutdownRequest() or bgworker_die().
Previously the exit handling was only able to exit from within the main loop,
and not from within the backend code it calls. This is why die() needs to
be used here, so worker_spi can respond to SIGTERM signal while it's
executing a query.
Maybe we can say that it's a bug that worker_spi could not respond to
SIGTERM during query execution. But since worker_spi is a just example
of the background worker code, we don't do the back-patch.
Thanks to Craig Ringer for the report and investigation of the issue.
Author: Bharath Rupireddy
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CALj2ACXDEZhAFOTDcqO9cFSRvrFLyYOnPKrcA1UG4uZn9hUAVg@mail.gmail.com
Discussion: https://postgr.es/m/CAGRY4nxsAe_1k_9g5b47orA0S011iBoHsXHFMH7cg7HV0O1bwQ@mail.gmail.com
M src/test/modules/worker_spi/worker_spi.c
Fix replication of in-progress transactions in tablesync worker.
commit : 0926e96c493443644ba8e96b5d96d013a9ffaf64
author : Amit Kapila <[email protected]>
date : Fri, 27 Nov 2020 07:43:34 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 27 Nov 2020 07:43:34 +0530
Tablesync worker runs under a single transaction but in streaming mode, we
were committing the transaction on stream_stop, stream_abort, and
stream_commit. We need to avoid committing the transaction in a streaming
mode in tablesync worker.
In passing move the call to process_syncing_tables in
apply_handle_stream_commit after clean up of stream files. This will
allow clean up of files to happen before the exit of tablesync worker
which would otherwise be handled by one of the proc exit routines.
Author: Dilip Kumar
Reviewed-by: Amit Kapila and Peter Smith
Tested-by: Peter Smith
Discussion: https://postgr.es/m/CAHut+Pt4PyKQCwqzQ=EFF=bpKKJD7XKt_S23F6L20ayQNxg77A@mail.gmail.com
M src/backend/replication/logical/worker.c
Restore lock level to update statusFlags
commit : dcfff74fb16622898a9d1df33e530f477caab095
author : Alvaro Herrera <[email protected]>
date : Thu, 26 Nov 2020 12:30:48 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 26 Nov 2020 12:30:48 -0300
Reverts 27838981be9d (some comments are kept). Per discussion, it does
not seem safe to relax the lock level used for this; in order for it to
be safe, there would have to be memory barriers between the point we set
the flag and the point we set the trasaction Xid, which perhaps would
not be so bad; but there would also have to be barriers at the readers'
side, which from a performance perspective might be bad.
Now maybe this analysis is wrong and it *is* safe for some reason, but
proof of that is not trivial.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/vacuum.c
M src/backend/replication/logical/logical.c
M src/backend/storage/ipc/procarray.c
M src/include/storage/proc.h
pg_stat_statements: Track number of times pgss entries were deallocated.
commit : 9fbc3f318d039c3e1e8614c38e40843cf8fcffde
author : Fujii Masao <[email protected]>
date : Thu, 26 Nov 2020 21:18:05 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 26 Nov 2020 21:18:05 +0900
If more distinct statements than pg_stat_statements.max are observed,
pg_stat_statements entries about the least-executed statements are
deallocated. This commit enables us to track the total number of times
those entries were deallocated. That number can be viewed in the
pg_stat_statements_info view that this commit adds. It's useful when
tuning pg_stat_statements.max parameter. If it's high, i.e., the entries
are deallocated very frequently, which might cause the performance
regression and we can increase pg_stat_statements.max to avoid those
frequent deallocations.
The pg_stat_statements_info view is intended to display the statistics
of pg_stat_statements module itself. Currently it has only one column
"dealloc" indicating the number of times entries were deallocated.
But an upcoming patch will add other columns (for example, the time
at which pg_stat_statements statistics were last reset) into the view.
Author: Katsuragi Yuta, Yuki Seino
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_stat_statements/Makefile
M contrib/pg_stat_statements/expected/pg_stat_statements.out
A contrib/pg_stat_statements/pg_stat_statements–1.8–1.9.sql
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/pg_stat_statements.control
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
M doc/src/sgml/pgstatstatements.sgml
M src/tools/pgindent/typedefs.list
doc: Add description about re-analysis and re-planning of a prepared statement.
commit : 4a36eab79a193700b7b65baf6c09c795c90c02c6
author : Fujii Masao <[email protected]>
date : Thu, 26 Nov 2020 16:17:10 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 26 Nov 2020 16:17:10 +0900
A prepared statement is re-analyzed and re-planned whenever database
objects used in the statement have undergone definitional changes or
the planner statistics of them have been updated. The former has been
documented from before, but the latter was not previously. This commit
adds the description about the latter case into the docs.
Author: Atsushi Torikoshi
Reviewed-by: Andy Fan, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/prepare.sgml
Use Enums for logical replication message types at more places.
commit : f3a8f73ec274bf8cc6066ea315f1a5a022214b02
author : Amit Kapila <[email protected]>
date : Thu, 26 Nov 2020 09:21:14 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 26 Nov 2020 09:21:14 +0530
Commit 644f0d7cc9 added logical replication message type enums to use
instead of character literals but some char substitutions were overlooked.
Author: Peter Smith
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAHut+PsTG=Vrv8hgrvOnAvCNR21jhqMdPk2n0a1uJPoW0p+UfQ@mail.gmail.com
M src/backend/replication/logical/worker.c
Avoid spurious waits in concurrent indexing
commit : c98763bf51bf610b3ee7e209fc76c3ff9a6b3163
author : Alvaro Herrera <[email protected]>
date : Wed, 25 Nov 2020 18:21:08 -0300
committer: Alvaro Herrera <[email protected]>
date : Wed, 25 Nov 2020 18:21:08 -0300
In the various waiting phases of CREATE INDEX CONCURRENTLY (CIC) and
REINDEX CONCURRENTLY (RC), we wait for other processes to release their
snapshots; this is necessary in general for correctness. However,
processes doing CIC in other tables cannot possibly affect CIC or RC
done in "this" table, so we don't need to wait for those. This commit
adds a flag in MyProc->statusFlags to indicate that the current process
is doing CIC, so that other processes doing CIC or RC can ignore it when
waiting.
Note that this logic is only valid if the index does not access other
tables. For simplicity we avoid setting the flag if the index has a
column that's an expression, or has a WHERE predicate. (It is possible
to have expressional or partial indexes that do not access other tables,
but figuring that out would require more work.)
This flag can potentially also be used by processes doing REINDEX
CONCURRENTLY to be skipped; and by VACUUM to ignore processes in CIC or
RC for the purposes of computing an Xmin. That's left for future
commits.
Author: Álvaro Herrera <[email protected]>
Author: Dimitry Dolgov <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/indexcmds.c
M src/include/storage/proc.h
In psql's \d commands, don't truncate attribute default values.
commit : 314fb9baeacb3426a5d9a26132aae8d828cc0ad7
author : Tom Lane <[email protected]>
date : Wed, 25 Nov 2020 16:19:25 -0500
committer: Tom Lane <[email protected]>
date : Wed, 25 Nov 2020 16:19:25 -0500
Historically, psql has truncated the text of a column's default
expression at 128 characters. This is unlike any other behavior
in describe.c, and it's become particularly confusing now that
the limit is only applied to the expression proper and not to
the "generated always as (...) stored" text that may get wrapped
around it.
Excavation in our git history suggests that the original motivation
for this limit was not really to limit the display width (as I'd long
supposed), but to make it safe to use a fixed-width output buffer to
store the result. That implementation restriction is long gone of
course, but the limit remained. Let's just get rid of it.
While here, rearrange the logic about when to free the output string
so that it's not so dependent on unstated assumptions about the
possible values of attidentity and attgenerated.
Per bug #16743 from David Turon. Back-patch to v12 where GENERATED
came in. (Arguably we could take it back further, but I'm hesitant
to change the behavior of long-stable branches for this.)
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/describe.c
Doc: minor improvements for section 11.2 "Index Types".
commit : 85b4ba73423b480902206ca04330c1cbea371c3c
author : Tom Lane <[email protected]>
date : Wed, 25 Nov 2020 14:04:28 -0500
committer: Tom Lane <[email protected]>
date : Wed, 25 Nov 2020 14:04:28 -0500
Break the per-index-type discussions into <sect2>'s so as to make
them more visually separate and easier to find. Improve the markup,
and make a couple of small wording adjustments.
This also fixes one stray reference to the now-deprecated point
operators <^ and >^.
Dagfinn Ilmari Mannsåker, reviewed by David Johnston and Jürgen Purtz
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/indices.sgml
Avoid spamming the client with multiple ParameterStatus messages.
commit : 2432b1a04087edc2fd9536c7c9aa4ca03fd1b363
author : Tom Lane <[email protected]>
date : Wed, 25 Nov 2020 11:40:44 -0500
committer: Tom Lane <[email protected]>
date : Wed, 25 Nov 2020 11:40:44 -0500
Up to now, we sent a ParameterStatus message to the client immediately
upon any change in the active value of any GUC_REPORT variable. This
was only barely okay when the feature was designed; now that we have
things like function SET clauses, there are very plausible use-cases
where a GUC_REPORT variable might change many times within a query
--- and even end up back at its original value, perhaps. Fortunately
most of our GUC_REPORT variables are unlikely to be changed often;
but there are proposals in play to enlarge that set, or even make it
user-configurable.
Hence, let's fix things to not generate more than one ParameterStatus
message per variable per query, and to not send any message at all
unless the end-of-query value is different from what we last reported.
Discussion: https://postgr.es/m/[email protected]
M src/backend/tcop/postgres.c
M src/backend/utils/misc/guc.c
M src/include/utils/guc.h
M src/include/utils/guc_tables.h
tablefunc: Reject negative number of tuples passed to normal_rand()
commit : f73999262ed6c40d9a7c3d7cccec7143d4d15287
author : Peter Eisentraut <[email protected]>
date : Wed, 25 Nov 2020 15:30:18 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 25 Nov 2020 15:30:18 +0100
The function converted the first argument i.e. the number of tuples to
return into an unsigned integer which turns out to be huge number when
a negative value is passed. This causes the function to take much
longer time to execute. Instead, reject a negative value.
(If someone really wants to generate many more result rows, they
should consider adding a bigint or numeric variant.)
While at it, improve SQL test to test the number of tuples returned by
this function.
Author: Ashutosh Bapat <[email protected]>
Discussion: https://www.postgresql.org/message-id/CAG-ACPW3PUUmSnM6cLa9Rw4BEC5cEMKjX8Gogc8gvQcT3cYA1A@mail.gmail.com
M contrib/tablefunc/expected/tablefunc.out
M contrib/tablefunc/sql/tablefunc.sql
M contrib/tablefunc/tablefunc.c
doc: Fix typos
commit : 2fbd786c3446b1bc90f396d3fa5f4614b0a57d89
author : Peter Eisentraut <[email protected]>
date : Wed, 25 Nov 2020 09:49:00 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 25 Nov 2020 09:49:00 +0100
Author: Justin Pryzby <[email protected]>
Discussion: https://www.postgresql.org/message-id/[email protected]
M doc/src/sgml/contrib.sgml
Make error hint from bind() failure more accurate
commit : d5d91acdccae6b322a5634be2c66d62fc6b9a43d
author : Peter Eisentraut <[email protected]>
date : Wed, 25 Nov 2020 08:14:23 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 25 Nov 2020 08:14:23 +0100
The hint "Is another postmaster already running ..." should only be
printed for errors that are really about something else already using
the address. In other cases it is misleading. So only show that hint
if errno == EADDRINUSE.
Also, since Unix-domain sockets in the file-system namespace never
report EADDRINUSE for an existing file (they would just overwrite it),
the part of the hint saying "If not, remove socket file \"%s\" and
retry." can never happen, so remove it. Unix-domain sockets in the
abstract namespace can report EADDRINUSE, but in that case there is no
file to remove, so the hint doesn't work there either.
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/backend/libpq/pqcomm.c
Add support for abstract Unix-domain sockets
commit : c9f0624bc2f544baacafa38e3797d5323401d039
author : Peter Eisentraut <[email protected]>
date : Wed, 25 Nov 2020 08:14:23 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 25 Nov 2020 08:14:23 +0100
This is a variant of the normal Unix-domain sockets that don't use the
file system but a separate "abstract" namespace. At the user
interface, such sockets are represented by names starting with "@".
Supported on Linux and Windows right now.
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/config.sgml
M doc/src/sgml/libpq.sgml
M src/backend/libpq/pqcomm.c
M src/bin/psql/command.c
M src/bin/psql/prompt.c
M src/common/ip.c
M src/include/libpq/pqcomm.h
M src/interfaces/libpq/fe-connect.c
Fix WaitLatch(NULL) on Windows.
commit : a7e65dc88b6f088fc2fcf5a660d866de644b1300
author : Thomas Munro <[email protected]>
date : Wed, 25 Nov 2020 17:44:15 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 25 Nov 2020 17:44:15 +1300
Further to commit 733fa9aa, on Windows when a latch is triggered but we
aren't currently waiting for it, we need to locate the latch's HANDLE
rather than calling ResetEvent(NULL).
Author: Kyotaro Horiguchi <[email protected]>
Reported-by: Ranier Vilela <[email protected]>
Discussion: https://postgr.es/m/CAEudQArTPi1YBc%2Bn1fo0Asy3QBFhVjp_QgyKG-8yksVn%2ByRTiw%40mail.gmail.com
M src/backend/storage/ipc/latch.c
Remove obsolete comment atop ri_PlanCheck.
commit : 805b816305b62cb3e2670418ced8f7184a2d29c7
author : Amit Kapila <[email protected]>
date : Wed, 25 Nov 2020 09:14:45 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 25 Nov 2020 09:14:45 +0530
Commit 5b7ba75f7f removed the unused parameter but forgot to update the
nearby comments.
Author: Li Japin
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/ri_triggers.c
Stop gap fix for __attribute__((cold)) compiler bug in MinGW 8.1
commit : 687f6163447514c23b44a981540987013529a9ec
author : David Rowley <[email protected]>
date : Wed, 25 Nov 2020 16:33:43 +1300
committer: David Rowley <[email protected]>
date : Wed, 25 Nov 2020 16:33:43 +1300
The buildfarm animal walleye, running MinGW 8.1 has been having problems
ever since 697e1d02f and 913ec71d6 went in. This appears to be a bug in
assembler which was fixed in a later version.
For now, in order to get that animal running green again, let's just
define pg_attribute_cold and pg_attribute_hot to be empty macros on that
compiler. Hopefully, we can get the support of the owner of the animal to
upgrade to a less buggy compiler and revert this at a later date.
Discussion: https://postgr.es/m/[email protected]
M src/include/c.h
Remove catalog function currtid()
commit : 7b94e999606e2e2e10d68d544d49fc5a5d5785ac
author : Michael Paquier <[email protected]>
date : Wed, 25 Nov 2020 12:18:26 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 25 Nov 2020 12:18:26 +0900
currtid() and currtid2() are an undocumented set of functions whose sole
known user is the Postgres ODBC driver, able to retrieve the latest TID
version for a tuple given by the caller of those functions.
As used by Postgres ODBC, currtid() is a shortcut able to retrieve the
last TID loaded into a backend by passing an OID of 0 (magic value)
after a tuple insertion. This is removed in this commit, as it became
obsolete after the driver began using "RETURNING ctid" with inserts, a
clause supported since Postgres 8.2 (using RETURNING is better for
performance anyway as it reduces the number of round-trips to the
backend).
currtid2() is still used by the driver, so this remains around for now.
Note that this function is kept in its original shape for backward
compatibility reasons.
Per discussion with many people, including Andres Freund, Peter
Eisentraut, Álvaro Herrera, Hiroshi Inoue, Tom Lane and myself.
Bump catalog version.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/nodeModifyTable.c
M src/backend/utils/adt/tid.c
M src/include/access/heapam.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/tid.out
M src/test/regress/sql/tid.sql
Properly check index mark/restore in ExecSupportsMarkRestore.
commit : 660b89928d18386de7755565c008439ae75d1218
author : Andrew Gierth <[email protected]>
date : Tue, 24 Nov 2020 20:58:32 +0000
committer: Andrew Gierth <[email protected]>
date : Tue, 24 Nov 2020 20:58:32 +0000
Previously this code assumed that all IndexScan nodes supported
mark/restore, which is not true since it depends on optional index AM
support functions. This could lead to errors about missing support
functions in rare edge cases of mergejoins with no sort keys, where an
unordered non-btree index scan was placed on the inner path without a
protecting Materialize node. (Normally, the fact that merge join
requires ordered input would avoid this error.)
Backpatch all the way since this bug is ancient.
Per report from Eugen Konkov on irc.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execAmi.c
M src/backend/optimizer/util/plancat.c
M src/include/nodes/pathnodes.h
Tidy up definitions of pg_attribute_hot and pg_attribute_cold
commit : b0727ae99b7056ca88e9708c014a86f6e611d938
author : David Rowley <[email protected]>
date : Wed, 25 Nov 2020 10:52:50 +1300
committer: David Rowley <[email protected]>
date : Wed, 25 Nov 2020 10:52:50 +1300
1fa22a43a was a quick fix for portability problem I introduced in
697e1d02f. 1fa22a43a adds a few more cases to the preprocessor logic than
I'd have liked. Andres Freund and Dagfinn Ilmari Mannsåker suggested a
better way to do this.
In passing, also adjust the only current usage of these macros so that the
macro comes before the function's return type in the declaration of the
function. This now matches what the definition of the function does.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/include/c.h
M src/include/utils/elog.h
Put "inline" marker on declarations of inline functions.
commit : ec05bafdbbf474bf0a1416772da31f9f1f27fa1e
author : Tom Lane <[email protected]>
date : Tue, 24 Nov 2020 15:43:01 -0500
committer: Tom Lane <[email protected]>
date : Tue, 24 Nov 2020 15:43:01 -0500
I'm having a hard time telling whether the letter of the C standard
requires this, but we do have a couple of buildfarm members that
throw warnings when this is not done. Oversight in c532d15dd.
M src/backend/commands/copyfromparse.c
Fix expected output: the order of agg permission checks changed.
commit : 8818ad5b1557d42cd58e5196fac7084a2389bdad
author : Heikki Linnakangas <[email protected]>
date : Tue, 24 Nov 2020 12:50:16 +0200
committer: Heikki Linnakangas <[email protected]>
date : Tue, 24 Nov 2020 12:50:16 +0200
Commit 0a2bc5d61e changed the order that permissions on the final and
transition functions of an aggregate are checked in. That shows up as a
difference in the order the LOG messages in this sepgsql regression test
are printed. Adjust the expected output.
Per buildfarm failure in rhinoceros.
M contrib/sepgsql/expected/misc.out
Move per-agg and per-trans duplicate finding to the planner.
commit : 0a2bc5d61e713e3fe72438f020eea5fcc90b0f0b
author : Heikki Linnakangas <[email protected]>
date : Tue, 24 Nov 2020 10:45:00 +0200
committer: Heikki Linnakangas <[email protected]>
date : Tue, 24 Nov 2020 10:45:00 +0200
This has the advantage that the cost estimates for aggregates can count
the number of calls to transition and final functions correctly.
Bump catalog version, because views can contain Aggrefs.
Reviewed-by: Andres Freund
Discussion: https://www.postgresql.org/message-id/b2e3536b-1dbc-8303-c97e-89cb0b4a9a48%40iki.fi
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/nodeAgg.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planagg.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/prep/Makefile
A src/backend/optimizer/prep/prepagg.c
M src/backend/optimizer/util/clauses.c
M src/backend/parser/parse_func.c
M src/backend/utils/adt/selfuncs.c
M src/include/catalog/catversion.h
M src/include/executor/execExpr.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/primnodes.h
M src/include/optimizer/clauses.h
M src/include/optimizer/prep.h
M src/include/utils/selfuncs.h
M src/test/regress/expected/partition_aggregate.out
M src/test/regress/sql/partition_aggregate.sql
M src/tools/pgindent/typedefs.list
doc: Get rid of unnecessary space character from some index items.
commit : e522024bd8dd28a0f13dcccfd39170698f45c939
author : Fujii Masao <[email protected]>
date : Tue, 24 Nov 2020 17:00:16 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 24 Nov 2020 17:00:16 +0900
Previously some index items have " ," (i.e., space + comma) in the docs
as follows. Since the space character before the comma is unnecessary,
this commit gets rid of that for the sake of consistency with other
index items.
parallel_leader_participation configuration parameter , Other Planner Options
Author: Fujii Masao
Reviewed-by: Euler Taveira
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M doc/src/sgml/pgtrgm.sgml
Use macros instead of hardcoded offsets for LWLock initialization
commit : d03d7549b29236e300aceac0c22173cf19acc675
author : Michael Paquier <[email protected]>
date : Tue, 24 Nov 2020 12:39:58 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 24 Nov 2020 12:39:58 +0900
This makes the code slightly easier to follow, as the initialization
relies on an offset that overlapped with an equivalent set of macros
defined, which are used in other places already.
Author: Japin Li
Discussion: https://postgr.es/m/MEYP282MB1669FB410006758402F2C3A2B6E00@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
M src/backend/storage/lmgr/lwlock.c
Centralize logic for skipping useless ereport/elog calls.
commit : 789b938bf2b8e38d0894261eae6bc84bbbb4714e
author : Tom Lane <[email protected]>
date : Mon, 23 Nov 2020 19:04:07 -0500
committer: Tom Lane <[email protected]>
date : Mon, 23 Nov 2020 19:04:07 -0500
While ereport() and elog() themselves are quite cheap when the
error message level is too low to be printed, some places need to do
substantial work before they can call those macros at all. To allow
optimizing away such setup work when nothing is to be printed, make
elog.c export a new function message_level_is_interesting(elevel)
that reports whether ereport/elog will do anything. Make use of that
in various places that had ad-hoc direct tests of log_min_messages etc.
Also teach ProcSleep to use it to avoid some work. (There may well
be other places that could usefully use this; I didn't search hard.)
Within elog.c, refactor a little bit to avoid having duplicate copies
of the policy-setting logic. When that code was written, we weren't
relying on the availability of inline functions; so it had some
duplications in the name of efficiency, which I got rid of.
Alvaro Herrera and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlogutils.c
M src/backend/catalog/dependency.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walsender.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/error/elog.c
M src/include/utils/elog.h
Fix unportable usage of __has_attribute
commit : 1fa22a43a56e1fe44c7bb3a3d5ef31be5bcac41d
author : David Rowley <[email protected]>
date : Tue, 24 Nov 2020 13:09:35 +1300
committer: David Rowley <[email protected]>
date : Tue, 24 Nov 2020 13:09:35 +1300
This should fix the breakages caused by 697e1d02f, which seems to break
the build for GCC version < 5.
It seems, according to the GCC manual that __has_attribute is a "special
operator" and must be tested without any other conditions in the
preprocessor test.
Per recommendation from the GCC manual via Greg Nancarrow
Reported-by: Greg Nancarrow
Discussion: https://postgr.es/m/CAJcOf-euSu8fhC10v476o9dqnjqKysVs1_vRms-_fvajpZ3kFw@mail.gmail.com
M src/include/c.h
Improve compiler code layout in elog/ereport ERROR calls
commit : 913ec71d682e99852cc188a7edbdccd02d42b4b3
author : David Rowley <[email protected]>
date : Tue, 24 Nov 2020 12:04:42 +1300
committer: David Rowley <[email protected]>
date : Tue, 24 Nov 2020 12:04:42 +1300
Here we use a bit of preprocessor trickery to coax supporting compilers
into laying out their generated code so that the code that's in the same
branch as elog(ERROR)/ereport(ERROR) calls is moved away from the hot
path. Effectively, this reduces the size of the hot code meaning that it
can sit on fewer cache lines.
Performance improvements of between 10-15% have been seen on highly CPU
bound workloads using pgbench's TPC-b benchmark.
What's achieved here is very similar to putting the error condition inside
an unlikely() macro. For example;
if (unlikely(x < 0))
elog(ERROR, "invalid x value");
now there's no need to make use of unlikely() here as the common macro
used by elog and ereport will now see that elevel is >= ERROR and make use
of a pg_attribute_cold marked version of errstart().
When elevel < ERROR or if it cannot be determined to be constant, the
original behavior is maintained.
Author: David Rowley
Reviewed-by: Andres Freund, Peter Eisentraut
Discussion: https://postgr.es/m/CAApHDvrVpasrEzLL2er7p9iwZFZ%3DJj6WisePcFeunwfrV0js_A%40mail.gmail.com
M src/backend/utils/error/elog.c
M src/include/utils/elog.h
Define pg_attribute_cold and pg_attribute_hot macros
commit : 697e1d02f53f985992a4f479b0b08fca5c272c6c
author : David Rowley <[email protected]>
date : Tue, 24 Nov 2020 11:29:28 +1300
committer: David Rowley <[email protected]>
date : Tue, 24 Nov 2020 11:29:28 +1300
For compilers supporting __has_attribute and __has_attribute (hot/cold).
__has_attribute is supported on gcc >= 5, clang >= 2.9 and icc >= 17.
A followup commit will implement some usages of these macros.
Author: David Rowley
Reviewed-by: Andres Freund, Peter Eisentraut
Discussion: https://postgr.es/m/CAApHDvrVpasrEzLL2er7p9iwZFZ%3DJj6WisePcFeunwfrV0js_A%40mail.gmail.com
M src/include/c.h
Remove unnecessary #include.
commit : 3b9b01f75d6e2d7bf9e0bf8ec958ce420aa037c2
author : Tom Lane <[email protected]>
date : Mon, 23 Nov 2020 17:00:01 -0500
committer: Tom Lane <[email protected]>
date : Mon, 23 Nov 2020 17:00:01 -0500
Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M src/include/optimizer/clauses.h
Don't hold ProcArrayLock longer than needed in rare cases
commit : 450c8230b1f7eed1e927c44fb710d80502bea1d2
author : Alvaro Herrera <[email protected]>
date : Mon, 23 Nov 2020 18:55:23 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 23 Nov 2020 18:55:23 -0300
While cancelling an autovacuum worker, we hold ProcArrayLock while
formatting a debugging log string. We can make this shorter by saving
the data we need to produce the message and doing the formatting outside
the locked region.
This isn't terribly critical, as it only occurs pretty rarely: when a
backend runs deadlock detection and it happens to be blocked by a
autovacuum running autovacuum. Still, there's no need to cause a hiccup
in ProcArrayLock processing, which can be very high-traffic in some
cases.
While at it, rework code so that we only print the string when it is
really going to be used, as suggested by Michael Paquier.
Discussion: https://postgr.es/m/[email protected]
Reviewed-by: Michael Paquier <[email protected]>
M src/backend/storage/lmgr/proc.c
Rename the "point is strictly above/below point" comparison operators.
commit : 0cc99327888840f2bf572303b68438e4caf62de9
author : Tom Lane <[email protected]>
date : Mon, 23 Nov 2020 11:38:37 -0500
committer: Tom Lane <[email protected]>
date : Mon, 23 Nov 2020 11:38:37 -0500
Historically these were called >^ and <^, but that is inconsistent
with the similar box, polygon, and circle operators, which are named
|>> and <<| respectively. Worse, the >^ and <^ names are used for
*not* strict above/below tests for the box type.
Hence, invent new operators following the more common naming. The
old operators remain available for now, and are still accepted by
the relevant index opclasses too. But there's a deprecation notice,
so maybe we can get rid of them someday.
Emre Hasegeli, reviewed by Pavel Borisov
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/spgist.sgml
M src/backend/access/gist/gistproc.c
M src/backend/access/spgist/spgkdtreeproc.c
M src/backend/access/spgist/spgquadtreeproc.c
M src/include/access/stratnum.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_operator.dat
M src/test/regress/expected/create_index.out
M src/test/regress/expected/create_index_spgist.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/expected/point.out
M src/test/regress/sql/create_index.sql
M src/test/regress/sql/create_index_spgist.sql
M src/test/regress/sql/point.sql
Improve wording of two error messages related to generated columns.
commit : d36228a9fcdccd57a7dc332572eb9837c7c301e6
author : Tom Lane <[email protected]>
date : Mon, 23 Nov 2020 11:15:03 -0500
committer: Tom Lane <[email protected]>
date : Mon, 23 Nov 2020 11:15:03 -0500
Clarify that you can "insert" into a generated column as long as what
you're inserting is a DEFAULT placeholder.
Also, use ERRCODE_GENERATED_ALWAYS in place of ERRCODE_SYNTAX_ERROR;
there doesn't seem to be any reason to use the less specific errcode.
Discussion: https://postgr.es/m/[email protected]
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/generated.out
M src/test/regress/expected/identity.out
M src/test/regress/expected/updatable_views.out
Make some sanity-check elogs more verbose
commit : fe051291550ab88267ce3104e9833925bf757393
author : Alvaro Herrera <[email protected]>
date : Mon, 23 Nov 2020 13:10:03 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 23 Nov 2020 13:10:03 -0300
A few sanity checks in funcapi.c were not mentioning all the possible
clauses for failure, confusing developers who fat-fingered catalog data
additions. Make the errors more detailed to avoid wasting time in
pinpointing mistakes.
Per complaint from Craig Ringer.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/CAMsr+YH7Kd87A3cU5m_wKo46HPQ46zFv5wesFNL0YWxkGhGv3g@mail.gmail.com
M src/backend/utils/fmgr/funcapi.c
Fix a few comments that referred to copy.c.
commit : 68b1a4877ef54f5ee6e05c64876655dd13d2c73b
author : Heikki Linnakangas <[email protected]>
date : Mon, 23 Nov 2020 11:36:13 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 23 Nov 2020 11:36:13 +0200
Missed these in the previous commit.
M src/backend/libpq/pqcomm.c
M src/include/commands/trigger.h
M src/include/nodes/execnodes.h
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Split copy.c into four files.
commit : c532d15dddff14b01fe9ef1d465013cb8ef186df
author : Heikki Linnakangas <[email protected]>
date : Mon, 23 Nov 2020 10:50:50 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 23 Nov 2020 10:50:50 +0200
Copy.c has grown really large. Split it into more manageable parts:
- copy.c now contains only a few functions that are common to COPY FROM
and COPY TO.
- copyto.c contains code for COPY TO.
- copyfrom.c contains code for initializing COPY FROM, and inserting the
tuples to the correct table.
- copyfromparse.c contains code for reading from the client/file/program,
and parsing the input text/CSV/binary format into tuples.
All of these parts are fairly complicated, and fairly independent of each
other. There is a patch being discussed to implement parallel COPY FROM,
which will add a lot of new code to the COPY FROM path, and another patch
which would allow INSERTs to use the same multi-insert machinery as COPY
FROM, both of which will require refactoring that code. With those two
patches, there's going to be a lot of code churn in copy.c anyway, so now
seems like a good time to do this refactoring.
The CopyStateData struct is also split. All the formatting options, like
FORMAT, QUOTE, ESCAPE, are put in a new CopyFormatOption struct, which
is used by both COPY FROM and TO. Other state data are kept in separate
CopyFromStateData and CopyToStateData structs.
Reviewed-by: Soumyadeep Chakraborty, Erik Rijkers, Vignesh C, Andres Freund
Discussion: https://www.postgresql.org/message-id/8e15b560-f387-7acc-ac90-763986617bfb%40iki.fi
M contrib/file_fdw/file_fdw.c
M src/backend/commands/Makefile
M src/backend/commands/copy.c
A src/backend/commands/copyfrom.c
A src/backend/commands/copyfromparse.c
A src/backend/commands/copyto.c
M src/backend/replication/logical/tablesync.c
M src/include/commands/copy.h
A src/include/commands/copyfrom_internal.h
Allow a multi-row INSERT to specify DEFAULTs for a generated column.
commit : 17958972fe3bb03454a4b53756b29d65dc285efa
author : Tom Lane <[email protected]>
date : Sun, 22 Nov 2020 15:48:32 -0500
committer: Tom Lane <[email protected]>
date : Sun, 22 Nov 2020 15:48:32 -0500
One can say "INSERT INTO tab(generated_col) VALUES (DEFAULT)" and not
draw an error. But the equivalent case for a multi-row VALUES list
always threw an error, even if one properly said DEFAULT in each row.
Fix that. While here, improve the test cases for nearby logic about
OVERRIDING SYSTEM/USER values.
Dean Rasheed
Discussion: https://postgr.es/m/[email protected]
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/generated.out
M src/test/regress/expected/identity.out
M src/test/regress/sql/generated.sql
M src/test/regress/sql/identity.sql
In geo_ops.c, represent infinite slope as Infinity, not DBL_MAX.
commit : 9fe649ea295f00baf6d0f0c1f9b0cb1298f64fb9
author : Tom Lane <[email protected]>
date : Sat, 21 Nov 2020 17:24:07 -0500
committer: Tom Lane <[email protected]>
date : Sat, 21 Nov 2020 17:24:07 -0500
Since we're assuming IEEE floats these days, there seems little
reason not to do this. It has the advantage that when the slope is
computed as infinite due to the presence of Inf coordinates, we get
saner behavior than before from line_construct(), and thence also
in some dependent operations such as finding the closest point.
Also fix line_construct() to special-case slope zero. The previous
coding got the right answer in most cases, but it could compute
C as NaN when the point has Inf coordinates.
Discussion: https://postgr.es/m/CAGf+fX70rWFOk5cd00uMfa__0yP+vtQg5ck7c2Onb-Yczp0URA@mail.gmail.com
M src/backend/utils/adt/geo_ops.c
M src/test/regress/expected/geometry.out
Fix FPeq() and friends to get the right answers for infinities.
commit : 8597a48d01b6cc0b09ff626253ac93c67e5516d5
author : Tom Lane <[email protected]>
date : Sat, 21 Nov 2020 16:46:43 -0500
committer: Tom Lane <[email protected]>
date : Sat, 21 Nov 2020 16:46:43 -0500
"FPeq(infinity, infinity)" returned false, on account of getting NaN
when it subtracts the two inputs. Fix that by adding a separate
check for exact equality.
FPle() and FPge() similarly got the wrong answer for two like-signed
infinities. In those cases, we can just rearrange the comparisons
to avoid potentially subtracting infinities.
While the sibling functions FPne() etc accidentally gave the right
answers even with the internal NaN results, it seems best to make
similar adjustments to them to avoid depending on this.
FPeq() has to be converted to an inline function to avoid double
evaluations of its arguments, and I did the same for the others
just for consistency.
In passing, make the handling of NaN cases in line_eq() and
point_eq_point() simpler and easier to reason about, and perhaps
faster.
This results in just one visible regression test change: slope()
now gives DBL_MAX for two inputs of (inf,1e300), which is consistent
with what it does for (1e300,inf), so that seems like a bug fix.
Discussion: https://postgr.es/m/CAGf+fX70rWFOk5cd00uMfa__0yP+vtQg5ck7c2Onb-Yczp0URA@mail.gmail.com
M src/backend/utils/adt/geo_ops.c
M src/include/utils/geo_decls.h
M src/test/regress/expected/geometry.out
Extend the geometric regression test cases a little.
commit : a45272b25d6fc8f96793623545fc1f836ac39d94
author : Tom Lane <[email protected]>
date : Sat, 21 Nov 2020 16:34:22 -0500
committer: Tom Lane <[email protected]>
date : Sat, 21 Nov 2020 16:34:22 -0500
Add another edge-case value to "point_tbl", and add a test for
the line(point, point) function.
Some of the behaviors exposed here are wrong, but the idea of
committing this separately is to memorialize what we were getting,
and to allow easier inspection of the behavior changes caused by
upcoming patches.
Kyotaro Horiguchi (line() test added by me)
Discussion: https://postgr.es/m/CAGf+fX70rWFOk5cd00uMfa__0yP+vtQg5ck7c2Onb-Yczp0URA@mail.gmail.com
M src/test/regress/expected/create_index.out
M src/test/regress/expected/geometry.out
M src/test/regress/expected/point.out
M src/test/regress/sql/geometry.sql
M src/test/regress/sql/point.sql
Remove INSERT privilege check at table creation of CTAS and matview
commit : 878f3a19c6c8ff197e4a33f51d921a4abafcc494
author : Michael Paquier <[email protected]>
date : Sat, 21 Nov 2020 19:45:30 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 21 Nov 2020 19:45:30 +0900
As per discussion with Peter Eisentraunt, the SQL standard specifies
that any tuple insertion done as part of CREATE TABLE AS happens without
any extra ACL check, so it makes little sense to keep a check for INSERT
privileges when using WITH DATA. Materialized views are not part of the
standard, but similarly, this check can be confusing as this refers to
an access check on a table created within the same command as the one
that would insert data into this table.
This commit removes the INSERT privilege check for WITH DATA, the
default, that 846005e removed partially, but only for WITH NO DATA.
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_materialized_view.sgml
M doc/src/sgml/ref/create_table_as.sgml
M src/backend/commands/createas.c
M src/test/regress/expected/matview.out
M src/test/regress/expected/select_into.out
M src/test/regress/sql/matview.sql
M src/test/regress/sql/select_into.sql
doc: Improve tableoid description
commit : a47834db0fb70256e676862f1d96374a9e114d82
author : Peter Eisentraut <[email protected]>
date : Sat, 21 Nov 2020 08:26:20 +0100
committer: Peter Eisentraut <[email protected]>
date : Sat, 21 Nov 2020 08:26:20 +0100
Mention that it's useful for determining table names for partitioned
tables as well as for those in inheritance hierarchies.
Author: Ian Lawrence Barwick <[email protected]>
Reviewed-by: Ashutosh Bapat <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CAB8KJ=gFmBXP=P9htziOj+WM5PDAK4qc7iGQta+8kUh306kQnw@mail.gmail.com
M doc/src/sgml/ddl.sgml
Make pg_rewind test case more stable.
commit : c71f9a094b32770dcd34b9ba4909435e34583747
author : Heikki Linnakangas <[email protected]>
date : Fri, 20 Nov 2020 16:08:58 +0200
committer: Heikki Linnakangas <[email protected]>
date : Fri, 20 Nov 2020 16:08:58 +0200
If replication is exceptionally slow for some reason, pg_rewind might run
before the test row has been replicated. Add an explicit wait for it.
Reported-by: Andres Freund
Discussion: https://www.postgresql.org/message-id/20201120003811.iknhqwatitw2vvxf%40alap3.anarazel.de
M src/bin/pg_rewind/t/007_standby_source.pl
Remove ability to independently select random number generator
commit : 16f96c74d48e65da23d28665103e2c4c9d3414cc
author : Magnus Hagander <[email protected]>
date : Fri, 20 Nov 2020 12:26:57 +0100
committer: Magnus Hagander <[email protected]>
date : Fri, 20 Nov 2020 12:26:57 +0100
Remove the ability to select random number generator independently from
SSL library. Instead, use the random number generator from the SSL
library (today only OpenSSL supported) if one is configured. If no SSL
library is configured, use the platform default (which means use
CryptoAPI on Win32 and /dev/urandom on Linux).
This also restructures pg_strong_random.c to have three clearly separate
sections, one for each implementation, with two functions in each,
instead of a scattered set of ifdefs throughout the whole file.
Author: Daniel Gustafsson, Magnus Hagander, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
M src/include/pg_config.h.in
M src/port/pg_strong_random.c
M src/tools/msvc/Solution.pm
Replace a macro by a function
commit : b5acf10cfc63ed1e0bc4eef466c0f4670a725ef3
author : Peter Eisentraut <[email protected]>
date : Fri, 20 Nov 2020 11:25:25 +0100
committer: Peter Eisentraut <[email protected]>
date : Fri, 20 Nov 2020 11:25:25 +0100
Using a macro is ugly and not justified here.
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/backend/access/transam/xlog.c
Add collation versions for FreeBSD.
commit : ca051d8b101d3205f1c5faf0d1da8a523ffc95a1
author : Thomas Munro <[email protected]>
date : Fri, 20 Nov 2020 21:49:57 +1300
committer: Thomas Munro <[email protected]>
date : Fri, 20 Nov 2020 21:49:57 +1300
On FreeBSD 13, use querylocale() to read the current version of libc
collations. Similar to commits 352f6f2d for Windows and d5ac14f9 for
GNU/Linux.
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
M doc/src/sgml/charset.sgml
M src/backend/utils/adt/pg_locale.c
Emit log when restore_command succeeds but archived file faills to be restored.
commit : a4ef0329c27156dca81714854599e7d82becb17f
author : Fujii Masao <[email protected]>
date : Fri, 20 Nov 2020 15:42:47 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 20 Nov 2020 15:42:47 +0900
Previously, when restore_command claimed to succeed but failed to restore
the file with the right name, for example, due to mis-configuration of
restore_command, no log message was reported. Then the recovery failed
later with an error message not directly related to the issue.
This commit changes the recovery so that a log message is emitted in
this error case. This would enable us to investigate what happened in
this case more easily.
Author: Jeff Janes, Fujii Masao
Reviewed-by: Pavel Borisov, Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAMkU=1xkFs3Omp4JR4wMYWdam_KLuj6LXnTYfU8u3T0h=PLLMQ@mail.gmail.com
M src/backend/access/transam/xlogarchive.c
On macOS, use -isysroot in link steps as well as compile steps.
commit : 49407dc32a2931550e4ff1dea314b6a25afdfc35
author : Tom Lane <[email protected]>
date : Fri, 20 Nov 2020 00:07:09 -0500
committer: Tom Lane <[email protected]>
date : Fri, 20 Nov 2020 00:07:09 -0500
We previously put the -isysroot switch only into CPPFLAGS, theorizing
that it was only needed to find the right copies of include files.
However, it seems that we also need to use it while linking programs,
to find the right stub ".tbd" files for libraries. We got away
without that up to now, but apparently that was mostly luck. It may
also be that failures are only observed when the Xcode version is
noticeably out of sync with the host macOS version; the case that's
prompting action right now is that builds fail when using latest Xcode
(12.2) on macOS Catalina, even though it's fine on Big Sur.
Hence, add -isysroot to LDFLAGS as well. (It seems that the more
common practice is to put it in CFLAGS, whence it'd be included at
both compile and link steps. However, we can't mess with CFLAGS in
the platform template file without confusing configure's logic for
choosing default CFLAGS.)
This should be back-patched, but first let's see if the buildfarm
likes it on HEAD.
Report and patch by James Hilliard (some cosmetic mods by me)
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
M src/template/darwin
Remove undocumented IS [NOT] OF syntax.
commit : 926fa801ac9eb54c5275472271ec63a059904698
author : Tom Lane <[email protected]>
date : Thu, 19 Nov 2020 17:39:39 -0500
committer: Tom Lane <[email protected]>
date : Thu, 19 Nov 2020 17:39:39 -0500
This feature was added a long time ago, in 7c1e67bd5 and eb121ba2c,
but never documented in any user-facing way. (Documentation added
in 6126d3e70 was commented out almost immediately, in 8272fc3f7.)
That's because, while this syntax is defined by SQL:99, our
implementation is only vaguely related to the standard's semantics.
The standard appears to intend a run-time not parse-time test, and
it definitely intends that the test should understand subtype
relationships.
No one has stepped up to fix that in the intervening years, but
people keep coming across the code and asking why it's not documented.
Let's just get rid of it: if anyone ever wants to make it work per
spec, they can easily recover whatever parts of this code are still
of value from our git history.
If there's anyone out there who's actually using this despite its
undocumented status, they can switch to using pg_typeof() instead,
eg. "pg_typeof(something) = 'mytype'::regtype". That gives
essentially the same semantics as what our IS OF code did.
(We didn't have that function last time this was discussed, or
we would have ripped out IS OF then.)
Discussion: https://postgr.es/m/CAKFQuwZ2pTc-DSkOiTfjauqLYkNREeNZvWmeg12Q-_69D+sYZA@mail.gmail.com
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/catalog/sql_features.txt
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_expr.c
M src/include/nodes/parsenodes.h
M src/test/regress/expected/arrays.out
M src/test/regress/expected/domain.out
M src/test/regress/expected/with.out
M src/test/regress/sql/arrays.sql
M src/test/regress/sql/domain.sql
M src/test/regress/sql/with.sql
Further fixes for CREATE TABLE LIKE: cope with self-referential FKs.
commit : 97390fe8a6e96a153e59b0180f4303acaeb75b84
author : Tom Lane <[email protected]>
date : Thu, 19 Nov 2020 15:03:17 -0500
committer: Tom Lane <[email protected]>
date : Thu, 19 Nov 2020 15:03:17 -0500
Commit 502898192 was too careless about the order of execution of the
additional ALTER TABLE operations generated by expandTableLikeClause.
It just stuck them all at the end, which seems okay for most purposes.
But it falls down in the case where LIKE is importing a primary key
or unique index and the outer CREATE TABLE includes a FOREIGN KEY
constraint that needs to depend on that index. Weird as that is,
it used to work, so we ought to keep it working.
To fix, make parse_utilcmd.c insert LIKE clauses between index-creation
and FK-creation commands in the transformed list of commands, and change
utility.c so that the commands generated by expandTableLikeClause are
executed immediately not at the end. One could imagine scenarios where
this wouldn't work either; but currently expandTableLikeClause only
makes column default expressions, CHECK constraints, and indexes, and
this ordering seems fine for those.
Per bug #16730 from Sofoklis Papasofokli. Like the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/parse_utilcmd.c
M src/backend/tcop/utility.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Rename object in test to avoid conflict
commit : afaccbba787d5f1470c44ddd61c9ddaaad19e27e
author : Peter Eisentraut <[email protected]>
date : Thu, 19 Nov 2020 11:34:54 +0100
committer: Peter Eisentraut <[email protected]>
date : Thu, 19 Nov 2020 11:34:54 +0100
In 01e658fa74cb7e3292448f6663b549135958003b, the hash_func test
creates a type t1, but apparently a test running in parallel might
also use that name, depending on timing. Rename the type to avoid the
issue.
M src/test/regress/expected/hash_func.out
M src/test/regress/sql/hash_func.sql
Hash support for row types
commit : 01e658fa74cb7e3292448f6663b549135958003b
author : Peter Eisentraut <[email protected]>
date : Thu, 19 Nov 2020 09:24:37 +0100
committer: Peter Eisentraut <[email protected]>
date : Thu, 19 Nov 2020 09:24:37 +0100
Add hash functions for the record type as well as a hash operator
family and operator class for the record type. This enables all the
hash functionality for the record type such as hash-based plans for
UNION/INTERSECT/EXCEPT DISTINCT, recursive queries using UNION
DISTINCT, hash joins, and hash partitioning.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/38eccd35-4e2d-6767-1b3c-dada1eac3124%402ndquadrant.com
M doc/src/sgml/queries.sgml
M src/backend/utils/adt/rowtypes.c
M src/backend/utils/cache/lsyscache.c
M src/backend/utils/cache/typcache.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_opclass.dat
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_opfamily.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/hash_func.out
M src/test/regress/expected/join.out
M src/test/regress/expected/union.out
M src/test/regress/expected/with.out
M src/test/regress/sql/hash_func.sql
M src/test/regress/sql/join.sql
M src/test/regress/sql/union.sql
M src/test/regress/sql/with.sql
Add BarrierArriveAndDetachExceptLast().
commit : 7888b0999488511e4266f2134053fa3a6505a155
author : Thomas Munro <[email protected]>
date : Thu, 19 Nov 2020 18:09:50 +1300
committer: Thomas Munro <[email protected]>
date : Thu, 19 Nov 2020 18:09:50 +1300
Provide a way for one process to continue the remaining phases of a
(previously) parallel computation alone. Later patches will use this to
extend Parallel Hash Join.
Author: Melanie Plageman <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKG%2BA6ftXPz4oe92%2Bx8Er%2BxpGZqto70-Q_ERwRaSyA%3DafNg%40mail.gmail.com
M src/backend/storage/ipc/barrier.c
M src/include/storage/barrier.h
Improve failure detection with array parsing in pg_dump
commit : 13b58f8934e6252868231c3493d49b8c2b363e5d
author : Michael Paquier <[email protected]>
date : Thu, 19 Nov 2020 10:36:08 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 19 Nov 2020 10:36:08 +0900
Similarly to 3636efa, the checks done in pg_dump when parsing array
values from catalogs have been too lax. Under memory pressure, it could
be possible, though very unlikely, to finish with dumps that miss some
data like:
- Statistics for indexes
- Run-time configuration of functions
- Configuration of extensions
- Publication list for a subscription
No backpatch is done as this is not going to be a problem in practice.
For example, if an OOM causes an array parsing to fail, a follow-up code
path of pg_dump would most likely complain with an allocation failure
due to the memory pressure.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_dump.c
Relax lock level for setting PGPROC->statusFlags
commit : 27838981be9de35d54ffcdc6cc06b5d5ea9e0cee
author : Alvaro Herrera <[email protected]>
date : Wed, 18 Nov 2020 13:24:22 -0300
committer: Alvaro Herrera <[email protected]>
date : Wed, 18 Nov 2020 13:24:22 -0300
We don't actually need a lock to set PGPROC->statusFlags itself; what we
do need is a shared lock on either XidGenLock or ProcArrayLock in order to
ensure MyProc->pgxactoff keeps still while we modify the mirror array in
ProcGlobal->statusFlags. Some places were using an exclusive lock for
that, which is excessive. Relax those to use shared lock only.
procarray.c has a couple of places with somewhat brittle assumptions
about PGPROC changes: ProcArrayEndTransaction uses only shared lock, so
it's permissible to change MyProc only. On the other hand,
ProcArrayEndTransactionInternal also changes other procs, so it must
hold exclusive lock. Add asserts to ensure those assumptions continue
to hold.
Author: Álvaro Herrera <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/vacuum.c
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/lmgr/deadlock.c
M src/include/storage/proc.h
Skip allocating hash table in EXPLAIN-only mode.
commit : 2cccb627f10d5dd4c21806fa251e4fb61eadd28c
author : Heikki Linnakangas <[email protected]>
date : Wed, 18 Nov 2020 12:39:15 +0200
committer: Heikki Linnakangas <[email protected]>
date : Wed, 18 Nov 2020 12:39:15 +0200
Author: Alexey Bashtanov
Discussion: https://www.postgresql.org/message-id/36823f65-050d-ae24-aa4d-a37726998240%40imap.cc
M src/backend/executor/nodeAgg.c
docs: Clarify that signal functions have no feedback.
commit : dfab7f2f94812c6c3a62cfde0ec7f9e447132e64
author : Heikki Linnakangas <[email protected]>
date : Wed, 18 Nov 2020 10:28:50 +0200
committer: Heikki Linnakangas <[email protected]>
date : Wed, 18 Nov 2020 10:28:50 +0200
Bug #16652 complains that pg_reload_conf() returned true, even though
the configuration file contained errors. That's the way pg_reload_conf()
works, by design, but the documentation wasn't very clear on it. Clarify
that a 'true' return value only means that the signal was sent
successfully. Also add links to the system views that can be used to
check the configuration files for errors.
David G. Johnston, with some rewording by me.
Discussion: https://www.postgresql.org/message-id/CAKFQuwax6GxhUQEes0D045UtXG-fBraM39_6UMd5JyR5K1HWCQ%40mail.gmail.com
M doc/src/sgml/func.sgml
Add more tests for hashing and hash-based plans
commit : 6dd8b00807871bfd0c3ce8e18fd1f6ba1a71fbf9
author : Peter Eisentraut <[email protected]>
date : Wed, 18 Nov 2020 07:58:37 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 18 Nov 2020 07:58:37 +0100
- Test hashing of an array of a non-hashable element type.
- Test UNION [DISTINCT] with hash- and sort-based plans. (Previously,
only INTERSECT and EXCEPT where tested there.)
- Test UNION [DISTINCT] with a non-hashable column type. This
currently reverts to a sort-based plan even if enable_hashagg is on.
- Test UNION/INTERSECT/EXCEPT hash- and sort-based plans with arrays
as column types. Also test an array with a non-hashable element
type.
- Test UNION/INTERSECT/EXCEPT similarly with row types as column
types. Currently, this uses only sort-based plans because there is
no hashing support for row types.
- Add a test case that shows that recursive queries using UNION
[DISTINCT] require hashable column types.
- Add a currently failing test that uses UNION DISTINCT in a
cycle-detection use case using row types as column types.
Discussion: https://www.postgresql.org/message-id/flat/38eccd35-4e2d-6767-1b3c-dada1eac3124%402ndquadrant.com
M src/test/regress/expected/hash_func.out
M src/test/regress/expected/union.out
M src/test/regress/expected/with.out
M src/test/regress/sql/hash_func.sql
M src/test/regress/sql/union.sql
M src/test/regress/sql/with.sql
Add tab completion for CREATE [OR REPLACE] TRIGGER in psql
commit : bf0aa7c4b83bcf3116c5a3c191bbc677ab3beb59
author : Michael Paquier <[email protected]>
date : Wed, 18 Nov 2020 14:01:53 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 18 Nov 2020 14:01:53 +0900
92bf7e2 has added support for this grammar.
Author: Noriyoshi Shinoda
Discussion: https://postgr.es/m/TU4PR8401MB115244623CF4724DCA0D507FEEE30@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM
M src/bin/psql/tab-complete.c
Deprecate nbtree's BTP_HAS_GARBAGE flag.
commit : cf2acaf4dcb5e20204dcec4d698cb4478af533e7
author : Peter Geoghegan <[email protected]>
date : Tue, 17 Nov 2020 09:45:56 -0800
committer: Peter Geoghegan <[email protected]>
date : Tue, 17 Nov 2020 09:45:56 -0800
Streamline handling of the various strategies that we have to avoid a
page split in nbtinsert.c. When it looks like a leaf page is about to
overflow, we now perform deleting LP_DEAD items and deduplication in one
central place. This greatly simplifies _bt_findinsertloc().
This has an independently useful consequence: nbtree no longer relies on
the BTP_HAS_GARBAGE page level flag/hint for anything important. We
still set and unset the flag in the same way as before, but it's no
longer treated as a gating condition when considering if we should check
for already-set LP_DEAD bits. This happens at the point where the page
looks like it might have to be split anyway, so simply checking the
LP_DEAD bits in passing is practically free. This avoids missing
LP_DEAD bits just because the page-level hint is unset, which is
probably reasonably common (e.g. it happens when VACUUM unsets the
page-level flag without actually removing index tuples whose LP_DEAD-bit
was set recently, after the VACUUM operation began but before it reached
the leaf page in question).
Note that this isn't a big behavioral change compared to PostgreSQL 13.
We were already checking for set LP_DEAD bits regardless of whether the
BTP_HAS_GARBAGE page level flag was set before we considered doing a
deduplication pass. This commit only goes slightly further by doing the
same check for all indexes, even indexes where deduplication won't be
performed.
We don't completely remove the BTP_HAS_GARBAGE flag. We still rely on
it as a gating condition with pg_upgrade'd indexes from before B-tree
version 4/PostgreSQL 12. That makes sense because we sometimes have to
make a choice among pages full of duplicates when inserting a tuple with
pre version 4 indexes. It probably still pays to avoid accessing the
line pointer array of a page there, since it won't yet be clear whether
we'll insert on to the page in question at all, let alone split it as a
result.
Author: Peter Geoghegan <[email protected]>
Reviewed-By: Victor Yegorov <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wz%3DYpc1PDdk8OVJDChGJBjT06%3DA0Mbv9HyTLCsOknGcUFg%40mail.gmail.com
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/access/nbtree/nbtxlog.c
M src/include/access/nbtree.h
indexcmds.c: reorder function prototypes
commit : 7684b6fbed3a0770a0d8fdcbb5cf8b61394de691
author : Alvaro Herrera <[email protected]>
date : Tue, 17 Nov 2020 11:37:36 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 17 Nov 2020 11:37:36 -0300
... out of an overabundance of neatnikism, perhaps.
M src/backend/commands/indexcmds.c
nbtree: Rename nbtinsert.c variables for consistency.
commit : a034f8b60c6e9afd75f42dc420a76cb9d3875fcb
author : Peter Geoghegan <[email protected]>
date : Tue, 17 Nov 2020 09:01:14 -0800
committer: Peter Geoghegan <[email protected]>
date : Tue, 17 Nov 2020 09:01:14 -0800
Stop naming special area/opaque pointer variables 'lpageop' in contexts
where it doesn't make sense. This is a holdover from a time when logic
that performs tasks that are now spread across _bt_insertonpg(),
_bt_findinsertloc(), and _bt_split() was more centralized. 'lpageop'
denotes "left page", which doesn't make sense outside of contexts in
which there isn't also a right page.
Also acquire page flag variables up front within _bt_insertonpg(). This
makes it closer to _bt_split() following refactoring commit bc3087b626d.
This allows the page split and retail insert paths to both make use of
the same variables.
M src/backend/access/nbtree/nbtinsert.c
Fix 'skip-empty-xacts' option in test_decoding for streaming mode.
commit : 9653f24ad8307f393de51e0a64d9b10a49efa6e3
author : Amit Kapila <[email protected]>
date : Tue, 17 Nov 2020 12:14:53 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 17 Nov 2020 12:14:53 +0530
In streaming mode, the transaction can be decoded in multiple streams and
those streams can be interleaved with streams of other transactions. So,
we can't remember the transaction's write status in the logical decoding
context because that might get changed due to some other transactions and
lead to wrong answers for 'skip-empty-xacts' option. We decided to keep
each transaction's write status in the ReorderBufferTxn to avoid
interleaved streams changing the status of some unrelated transactions.
Diagnosed-by: Amit Kapila
Author: Dilip Kumar
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAA4eK1LR7=XNM_TLmpZMFuV8ZQpoxkem--NZJYf8YXmesbvwLA@mail.gmail.com
M contrib/test_decoding/expected/concurrent_stream.out
M contrib/test_decoding/specs/concurrent_stream.spec
M contrib/test_decoding/test_decoding.c
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h
M src/tools/pgindent/typedefs.list
Don't Insert() a VFD entry until it's fully built.
commit : 2bd49b493a52f0a21bc3fc51c355963f99ac1a4e
author : Tom Lane <[email protected]>
date : Mon, 16 Nov 2020 20:32:35 -0500
committer: Tom Lane <[email protected]>
date : Mon, 16 Nov 2020 20:32:35 -0500
Otherwise, if FDDEBUG is enabled, the debugging output fails because
it tries to read the fileName, which isn't set up yet (and should in
fact always be NULL).
AFAICT, this has been wrong since Berkeley. Before 96bf88d52,
it would accidentally fail to crash on platforms where snprintf()
is forgiving about being passed a NULL pointer for %s; but the
file name intended to be included in the debug output wouldn't
ever have shown up.
Report and fix by Greg Nancarrow. Although this is only visibly
broken in custom-made builds, it still seems worth back-patching
to all supported branches, as the FDDEBUG code is pretty useless
as it stands.
Discussion: https://postgr.es/m/CAJcOf-cUDgm9qYtC_B6XrC6MktMPNRby2p61EtSGZKnfotMArw@mail.gmail.com
M src/backend/storage/file/fd.c
Rename PGPROC->vacuumFlags to statusFlags
commit : cd9c1b3e197a9b53b840dcc87eb41b04d601a5f9
author : Alvaro Herrera <[email protected]>
date : Mon, 16 Nov 2020 19:42:55 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 16 Nov 2020 19:42:55 -0300
With more flags associated to a PGPROC entry that are not related to
vacuum (currently existing or planned), the name "statusFlags" describes
its purpose better.
(The same is done to the mirroring PROC_HDR->vacuumFlags.)
No functional changes in this commit.
This was suggested first by Hari Babu Kommi in [1] and then by Michael
Paquier at [2].
[1] https://postgr.es/m/CAJrrPGcsDC-oy1AhqH0JkXYa0Z2AgbuXzHPpByLoBGMxfOZMEQ@mail.gmail.com
[2] https://postgr.es/m/[email protected]
Author: Dmitry Dolgov <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/20201116182446.qcg3o6szo2zookyr@localhost
M src/backend/access/transam/twophase.c
M src/backend/commands/vacuum.c
M src/backend/postmaster/autovacuum.c
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/lmgr/deadlock.c
M src/backend/storage/lmgr/proc.c
M src/include/storage/proc.h
Do not return NULL for error cases in satisfies_hash_partition().
commit : 4025e6c46620048804467d2ad29d31aa9ba50387
author : Tom Lane <[email protected]>
date : Mon, 16 Nov 2020 16:39:59 -0500
committer: Tom Lane <[email protected]>
date : Mon, 16 Nov 2020 16:39:59 -0500
Since this function is used as a CHECK constraint condition,
returning NULL is tantamount to returning TRUE, which would have the
effect of letting in a row that doesn't satisfy the hash condition.
Admittedly, the cases for which this is done should be unreachable
in practice, but that doesn't make it any less a bad idea. It also
seems like a dartboard was used to decide which error cases should
throw errors as opposed to returning NULL.
For the checks for NULL input values, I just switched it to returning
false. There's some argument that an error would be better; but the
case really should be can't-happen in a generated hash constraint,
so it's likely not worth more code for.
For the parent-relation-open-failure case, it seems like we might
as well let relation_open throw an error, instead of having an
impossible-to-diagnose constraint failure.
Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/[email protected]
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/hash_part.out
Use "true" not "TRUE" in one ICU function call.
commit : ad84ecc98d7e2ad81567094b8a6910b5078927a7
author : Tom Lane <[email protected]>
date : Mon, 16 Nov 2020 15:16:39 -0500
committer: Tom Lane <[email protected]>
date : Mon, 16 Nov 2020 15:16:39 -0500
This was evidently missed in commit 6337865f3, which generally did
s/TRUE/true/ everywhere. It escaped notice up to now because ICU
versions before ICU 68 provided definitions of "TRUE" and "FALSE"
regardless. With ICU 68, it fails to compile.
Per report from Condor. Back-patch to v11 where 6337865f3 came in.
(I've not tested v10, where this call originated, but I imagine
it's fine since we defined TRUE in c.h back then.)
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/collationcmds.c
doc: update bgwriter description
commit : 62b50ad698f8a789f92177ad72639f99d94d7d78
author : Bruce Momjian <[email protected]>
date : Mon, 16 Nov 2020 13:13:43 -0500
committer: Bruce Momjian <[email protected]>
date : Mon, 16 Nov 2020 13:13:43 -0500
This clarifies exactly what the bgwriter does, which should help with
tuning.
Reported-by: Chris Wilson
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
doc: clarify how to find pg_type_d.h in the install tree
commit : 8ad06bcbe136360296a804da7d7631a78fd15e70
author : Bruce Momjian <[email protected]>
date : Mon, 16 Nov 2020 12:36:17 -0500
committer: Bruce Momjian <[email protected]>
date : Mon, 16 Nov 2020 12:36:17 -0500
Followup to patch 152ed04799.
Reported-by: Alvaro Herrera
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
Remove unused and deprecated strategy numbers from BRIN code
commit : d93ccdea1d447ab7e8c48797153ce58b051f1a37
author : Peter Eisentraut <[email protected]>
date : Mon, 16 Nov 2020 17:25:41 +0100
committer: Peter Eisentraut <[email protected]>
date : Mon, 16 Nov 2020 17:25:41 +0100
These were dead code.
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/backend/access/brin/brin_inclusion.c
doc: improve wording of the need for analyze of exp. indexes
commit : 481f9dc3dc2fc779af3ab2d58216a1c2b970b20f
author : Bruce Momjian <[email protected]>
date : Mon, 16 Nov 2020 10:26:17 -0500
committer: Bruce Momjian <[email protected]>
date : Mon, 16 Nov 2020 10:26:17 -0500
This is a followup commit on 3370207986.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/ref/create_index.sgml
Fix typo
commit : bcbd7713324798d5a74ca640c6cf6c439bda22e4
author : Alvaro Herrera <[email protected]>
date : Mon, 16 Nov 2020 10:54:11 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 16 Nov 2020 10:54:11 -0300
Introduced in 90fdc259866e; backpatch to 12.
Author: Erik Rijkers <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_index.sgml
Normalize comment in empty grammar rules
commit : 5664b7be5b2c2e756372e7dde7c187a11a5ab701
author : Peter Eisentraut <[email protected]>
date : Mon, 16 Nov 2020 11:54:52 +0100
committer: Peter Eisentraut <[email protected]>
date : Mon, 16 Nov 2020 11:54:52 +0100
Change lower case /* empty */ to /* EMPTY */ for consistency with the
majority.
Discussion: https://www.postgresql.org/message-id/flat/e9eed669-e32d-6919-fed4-acc0daea857b%40enterprisedb.com
M src/backend/parser/gram.y
Remove code handling removed deprecated containment operators
commit : 591d282e8d3e0448ec1339c6b066e10953f040a2
author : Peter Eisentraut <[email protected]>
date : Mon, 16 Nov 2020 11:45:00 +0100
committer: Peter Eisentraut <[email protected]>
date : Mon, 16 Nov 2020 11:45:00 +0100
This removes the code that was there for handling the operators
removed by 2f70fdb0644c32c4154236c2b5c241bec92eac5e.
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/backend/access/gist/gistproc.c
Make the standby server promptly handle interrupt signals.
commit : 2945a488a3426838223a637cd326ec71fff61954
author : Fujii Masao <[email protected]>
date : Mon, 16 Nov 2020 18:27:51 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 16 Nov 2020 18:27:51 +0900
This commit changes the startup process in the standby server so that
it handles the interrupt signals after waiting for wal_retrieve_retry_interval
on the latch and resetting it, before entering another wait on the latch.
This change causes the standby server to promptly handle interrupt signals.
Otherwise, previously, there was the case where the standby needs to
wait extra five seconds to shutdown when the shutdown request arrived
while the startup process was waiting for wal_retrieve_retry_interval
on the latch.
Author: Fujii Masao, but implementation idea is from Soumyadeep Chakraborty
Reviewed-by: Soumyadeep Chakraborty
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
Reword 'simple comparison' => 'inequality' in pgtrgm.sgml
commit : 7adb8feb0f69fd9c79078c6cea984bb4bff1c841
author : Alexander Korotkov <[email protected]>
date : Mon, 16 Nov 2020 09:09:07 +0300
committer: Alexander Korotkov <[email protected]>
date : Mon, 16 Nov 2020 09:09:07 +0300
Discussion: https://postgr.es/m/CAMkU%3D1zxfA8_MGBW6sJMj54p8nPoe4bMb5LoG-rMYZVPq4j08Q%40mail.gmail.com
Author: Jeff Janes
M doc/src/sgml/pgtrgm.sgml
Relax INSERT privilege requirement for CTAS and matviews WITH NO DATA
commit : 846005e4f3829c3eafe1f8441b80ff90657d0a29
author : Michael Paquier <[email protected]>
date : Mon, 16 Nov 2020 11:52:40 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 16 Nov 2020 11:52:40 +0900
When specified, WITH NO DATA does not insert any data into the relation
created, so skip checking for the insert permissions. With WITH DATA or
WITH NO DATA, it is always required for the user to have CREATE
privileges on the schema targeted for the relation.
Note that plain CREATE TABLE AS or CREATE MATERIALIZED VIEW queries have
begun to work accidentally without INSERT privilege checks as of
874fe3ae, while using EXECUTE or EXPLAIN ANALYZE would fail with the ACL
check, so this makes the behavior for all the command flavors consistent
with each other. This is arguably a bug fix, but there have been no
complaints about the current behavior either so stable branches are not
changed.
While on it, document properly the privileges requirements for each
commands with more tests for all the scenarios possible, and avoid a
useless bulk-insert allocation when using WITH NO DATA.
Author: Bharath Rupireddy
Reviewed-by: Anastasia Lubennikova, Michael Paquier
Discussion: https://postgr.es/m/CALj2ACWc3N8j0_9nMPz9wcAUnVcdKHzFdDZJ3hVFNEbqtcyG9w@mail.gmail.com
M doc/src/sgml/ref/create_materialized_view.sgml
M doc/src/sgml/ref/create_table_as.sgml
M src/backend/commands/createas.c
M src/test/regress/expected/matview.out
M src/test/regress/expected/select_into.out
M src/test/regress/sql/matview.sql
M src/test/regress/sql/select_into.sql
Fix fuzzy thinking about amcanmulticol versus amcaninclude.
commit : 29d29d652f0be47dc42fa9d667dee5b8e1baa18a
author : Tom Lane <[email protected]>
date : Sun, 15 Nov 2020 16:10:48 -0500
committer: Tom Lane <[email protected]>
date : Sun, 15 Nov 2020 16:10:48 -0500
These flags should be independent: in particular an index AM should
be able to say that it supports include columns without necessarily
supporting multiple key columns. The included-columns patch got
this wrong, possibly aided by the fact that it didn't bother to
update the documentation.
While here, clarify some text about amcanreturn, which was a little
vague about what should happen when amcanreturn reports that only
some of the index columns are returnable.
Noted while reviewing the SP-GiST included-columns patch, which
quite incorrectly (and unsafely) changed SP-GiST to claim
amcanmulticol = true as a workaround for this bug.
Backpatch to v11 where included columns were introduced.
M doc/src/sgml/indexam.sgml
M src/backend/commands/indexcmds.c
nbtree: Demote incomplete split "can't happen" error.
commit : 46cf3c72c36fef8c623b4b97d8720321d8ad7507
author : Peter Geoghegan <[email protected]>
date : Sun, 15 Nov 2020 11:53:37 -0800
committer: Peter Geoghegan <[email protected]>
date : Sun, 15 Nov 2020 11:53:37 -0800
Only a basic logic bug in a _bt_insertonpg() caller could lead to a
violation of this invariant (index corruption won't do it). A "can't
happen" error seems inappropriate (it is arbitrary at best).
Demote the error to a simple assertion. This matches similar nearby
sanity checks.
M src/backend/access/nbtree/nbtinsert.c
Suppress "warning: variable 'collcollate' set but not used".
commit : ff94205787244ace8ce63e3d9eb98338c59ac215
author : Tom Lane <[email protected]>
date : Sun, 15 Nov 2020 12:39:49 -0500
committer: Tom Lane <[email protected]>
date : Sun, 15 Nov 2020 12:39:49 -0500
Buildfarm members that lack both HAVE_LOCALE_T and USE_ICU have been
complaining about pg_newlocale_from_collation's collcollate variable.
This is evidently fallout from commit 7d1297df0, which removed the
only usage outside those two #ifdef'd code paths. Mark the variable
pg_attribute_unused(), like its sibling collctype, which has been that
way for a long time.
M src/backend/utils/adt/pg_locale.c
Fix timing issue in pg_rewind test.
commit : 39f9f04b57844143c7f510cc901501057103bfe4
author : Heikki Linnakangas <[email protected]>
date : Sun, 15 Nov 2020 17:09:31 +0200
committer: Heikki Linnakangas <[email protected]>
date : Sun, 15 Nov 2020 17:09:31 +0200
The test inserts a row in primary server, waits until the insertion has
been replicated to a cascaded standby, and checks that it's visible there
by querying the cascaded standby. In order for that to work reliably, the
test needs to wait until the insertion WAL record has been fully replayed.
This should fix the occasional buildfarm failures. Diagnosis by Tom Lane.
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/bin/pg_rewind/t/007_standby_source.pl
Add missing 'the' to pgtrgm.sgml
commit : b7edb5d96e7c70fc4e522e4ca1f893e815fa5502
author : Alexander Korotkov <[email protected]>
date : Sun, 15 Nov 2020 14:01:22 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 15 Nov 2020 14:01:22 +0300
Author: Erik Rijkers
Discussion: https://postgr.es/m/159fba265fe7c37334640fdc0444cc4b%40xs4all.nl
M doc/src/sgml/pgtrgm.sgml
Fix alphabetic ordering in typedefs.list
commit : 8afa3795a7a29f018769f16f6af936439847bd66
author : Alexander Korotkov <[email protected]>
date : Sun, 15 Nov 2020 08:56:31 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 15 Nov 2020 08:56:31 +0300
M src/tools/pgindent/typedefs.list
Handle equality operator in contrib/pg_trgm
commit : 935f6666502250abde8615bc7805a6e5aa05a066
author : Alexander Korotkov <[email protected]>
date : Sun, 15 Nov 2020 08:52:12 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 15 Nov 2020 08:52:12 +0300
Obviously, in order to equality operator be satisfiable, target string must
contain all the trigrams of the search string. On this base, we implement
equality operator in GiST/GIN indexes with recheck.
Discussion: https://postgr.es/m/CAOBaU_YWwtT7tdggtROacjdOdeYHCz-tmSwuC-j-TOG-g97J0w%40mail.gmail.com
Author: Julien Rouhaud
Reviewed-by: Tom Lane, Alexander Korotkov, Georgios Kokolatos, Erik Rijkers
M contrib/pg_trgm/Makefile
M contrib/pg_trgm/expected/pg_trgm.out
A contrib/pg_trgm/pg_trgm–1.5–1.6.sql
M contrib/pg_trgm/pg_trgm.control
M contrib/pg_trgm/sql/pg_trgm.sql
M contrib/pg_trgm/trgm.h
M contrib/pg_trgm/trgm_gin.c
M contrib/pg_trgm/trgm_gist.c
M doc/src/sgml/pgtrgm.sgml
Provide the OR REPLACE option for CREATE TRIGGER.
commit : 92bf7e2d027466d750b4ac5b026f6f4ac29be881
author : Tom Lane <[email protected]>
date : Sat, 14 Nov 2020 17:05:34 -0500
committer: Tom Lane <[email protected]>
date : Sat, 14 Nov 2020 17:05:34 -0500
This is mostly straightforward. However, we disallow replacing
constraint triggers or changing the is-constraint property; perhaps
that can be added later, but the complexity versus benefit tradeoff
doesn't look very good.
Also, no special thought is taken here for whether replacing an
existing trigger should result in changes to queued-but-not-fired
trigger actions. We just document that if you're surprised by the
results, too bad, don't do that. (Note that any such pending trigger
activity would have to be within the current session.)
Takamichi Osumi, reviewed at various times by Surafel Temesgen,
Peter Smith, and myself
Discussion: https://postgr.es/m/0DDF369B45A1B44B8A687ED43F06557C010BC362@G01JPEXMBYT03
M doc/src/sgml/ref/create_trigger.sgml
M src/backend/catalog/index.c
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/parser/gram.y
M src/include/nodes/parsenodes.h
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Doc: improve partitioning discussion in ddl.sgml.
commit : dbca94510c9e564708a10a1259f6f1d8ad09862c
author : Tom Lane <[email protected]>
date : Sat, 14 Nov 2020 13:09:53 -0500
committer: Tom Lane <[email protected]>
date : Sat, 14 Nov 2020 13:09:53 -0500
This started with the intent to explain that range upper bounds
are exclusive, which previously you could only find out by reading
the CREATE TABLE man page. But I soon found that section 5.11
really could stand a fair amount of editorial attention. It's
apparently been revised several times without much concern for
overall flow, nor careful copy-editing.
Back-patch to v11, which is as far as the patch goes easily.
Per gripe from Edson Richter. Thanks to David Johnston for review.
Discussion: https://postgr.es/m/DM6PR13MB3988736CF8F5DC5720440231CFE60@DM6PR13MB3988.namprd13.prod.outlook.com
M doc/src/sgml/ddl.sgml
Fix some typos
commit : 788dd0b839fc9f2c85caf86014105afdb60ed5e3
author : Michael Paquier <[email protected]>
date : Sat, 14 Nov 2020 11:43:10 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 14 Nov 2020 11:43:10 +0900
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M contrib/amcheck/verify_heapam.c
M contrib/sepgsql/selinux.c
M src/backend/catalog/objectaddress.c
M src/backend/catalog/pg_type.c
M src/backend/storage/ipc/procarray.c
M src/backend/utils/mb/Unicode/UCS_to_EUC_JP.pl
M src/include/access/gist.h
M src/include/replication/logicalproto.h
Support negative indexes in split_part().
commit : ec0294fb2c8287fd673c57701cdcf6c6396b2f60
author : Tom Lane <[email protected]>
date : Fri, 13 Nov 2020 13:49:48 -0500
committer: Tom Lane <[email protected]>
date : Fri, 13 Nov 2020 13:49:48 -0500
This provides a handy way to get, say, the last field of the string.
Use of a negative index in this way has precedent in the nearby
left() and right() functions.
The implementation scans the string twice when N < -1, but it seems
likely that N = -1 will be the huge majority of actual use cases,
so I'm not really excited about adding complexity to avoid that.
Nikhil Benesch, reviewed by Jacob Champion; cosmetic tweakage by me
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/utils/adt/varlena.c
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
Remove another test that doesn't work on Windows.
commit : 3bf44303b954269558c30458df8d02fb99c40c67
author : Heikki Linnakangas <[email protected]>
date : Fri, 13 Nov 2020 13:41:58 +0200
committer: Heikki Linnakangas <[email protected]>
date : Fri, 13 Nov 2020 13:41:58 +0200
Apparently double-quotes are not allowed in filenames on Windows, either.
Per buildfarm.
M src/bin/pg_rewind/t/003_extrafiles.pl
doc: clarify where to find pg_type_d.h (PG 11+) and pg_type.h
commit : 152ed04799e31e9abe6c2ed8afc7405b892b7d86
author : Bruce Momjian <[email protected]>
date : Thu, 12 Nov 2020 15:13:02 -0500
committer: Bruce Momjian <[email protected]>
date : Thu, 12 Nov 2020 15:13:02 -0500
These files are in compiled directories and install directories.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
docs: mention that expression indexes need analyze
commit : 337020798674bcdfefe93e2277d98458da7266cd
author : Bruce Momjian <[email protected]>
date : Thu, 12 Nov 2020 15:00:44 -0500
committer: Bruce Momjian <[email protected]>
date : Thu, 12 Nov 2020 15:00:44 -0500
Expression indexes can't benefit from pre-computed statistics on
columns.
Reported-by: Nikolay Samokhvalov
Discussion: https://postgr.es/m/CANNMO++5rw9RDA=p40iMVbMNPaW6O=S0AFzTU=KpYHRpCd1voA@mail.gmail.com
Author: Nikolay Samokhvalov, modified
Backpatch-through: 9.5
M doc/src/sgml/ref/create_index.sgml
change wire protocol data type for history file content
commit : 66a8f090485e3e897a4804121fdbe856cba72d70
author : Bruce Momjian <[email protected]>
date : Thu, 12 Nov 2020 14:08:59 -0500
committer: Bruce Momjian <[email protected]>
date : Thu, 12 Nov 2020 14:08:59 -0500
This was marked as BYTEA, but is more like TEXT, which is how we already
pass the history timeline file name. Internally, we don't do any
encoding or bytea escape handling, but TEXT seems closest. This should
cause no behavioral change.
Reported-by: Brar Piening
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: master
M doc/src/sgml/protocol.sgml
M src/backend/replication/walsender.c
Remove tests that don't work on Windows.
commit : 951dfa34f41bb851602216d3ada5f80135370e4f
author : Heikki Linnakangas <[email protected]>
date : Thu, 12 Nov 2020 19:18:34 +0200
committer: Heikki Linnakangas <[email protected]>
date : Thu, 12 Nov 2020 19:18:34 +0200
On Windows, a filename cannot contain backslashes, because a backslash
is used directory separator. Remove tests I added in commit 9c4f5192f
that tried to do that. We could perhaps use a SKIP block to only skip
them on Windows, but I'm not sure how exactly to formulate that, so just
remove the tests to make the buildfarm green again.
Per buildfarm.
M src/bin/pg_rewind/t/003_extrafiles.pl
pg_trgm: fix crash in 2-item picksplit
commit : 3836d4b643369310dc2f98ad311baeac307590d7
author : Andrew Gierth <[email protected]>
date : Thu, 12 Nov 2020 14:34:37 +0000
committer: Andrew Gierth <[email protected]>
date : Thu, 12 Nov 2020 14:34:37 +0000
Whether from size overflow in gistSplit or from secondary splits,
picksplit is (rarely) called with exactly two items to split.
Formerly, due to special-case handling of the last item, this would
lead to access to an uninitialized cache entry; prior to PG 13 this
might have been harmless or at worst led to an incorrect union datum,
but in 13 onwards it can cause a backend crash from using an
uninitialized pointer.
Repair by removing the special case, which was deemed not to have been
appropriate anyway. Backpatch all the way, because this bug has
existed since pg_trgm was added.
Per report on IRC from user "ftzdomino". Analysis and testing by me,
patch from Alexander Korotkov.
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_trgm/trgm_gist.c
Allow pg_rewind to use a standby server as the source system.
commit : 9c4f5192f69ed16c99e0d079f0b5faebd7bad212
author : Heikki Linnakangas <[email protected]>
date : Thu, 12 Nov 2020 14:52:24 +0200
committer: Heikki Linnakangas <[email protected]>
date : Thu, 12 Nov 2020 14:52:24 +0200
Using a hot standby server as the source has not been possible, because
pg_rewind creates a temporary table in the source system, to hold the
list of file ranges that need to be fetched. Refactor it to queue up the
file fetch requests in pg_rewind's memory, so that the temporary table
is no longer needed.
Also update the logic to compute 'minRecoveryPoint' correctly, when the
source is a standby server.
Reviewed-by: Kyotaro Horiguchi, Soumyadeep Chakraborty
Discussion: https://www.postgresql.org/message-id/0c5b3783-af52-3ee5-f8fa-6e794061f70d%40iki.fi
M doc/src/sgml/ref/pg_rewind.sgml
M src/bin/pg_rewind/libpq_source.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/t/003_extrafiles.pl
A src/bin/pg_rewind/t/007_standby_source.pl
Clean up optional rules in grammar
commit : 1b2b19f7584b7c0025aa40862cd38c79d340be7d
author : Peter Eisentraut <[email protected]>
date : Thu, 12 Nov 2020 07:48:36 +0100
committer: Peter Eisentraut <[email protected]>
date : Thu, 12 Nov 2020 07:48:36 +0100
Various rules for optional keywords contained unnecessary rules and
type declarations. Remove those, thus making the output a tiny bit
smaller.
Reviewed-by: Heikki Linnakangas <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/e9eed669-e32d-6919-fed4-acc0daea857b%40enterprisedb.com
M src/backend/parser/gram.y
Fix typo in contrib/pg_trgm/pg_trgm--1.4--1.5.sql
commit : 3e8ec5b1408a15fb41cada2635163a598d8607d1
author : Alexander Korotkov <[email protected]>
date : Thu, 12 Nov 2020 08:55:09 +0300
committer: Alexander Korotkov <[email protected]>
date : Thu, 12 Nov 2020 08:55:09 +0300
Backpatch-through: 13
M contrib/pg_trgm/pg_trgm–1.4–1.5.sql
Use standard SIGHUP and SIGTERM handlers in walreceiver.
commit : 1a2ae7c50fb4ab17a8d38e755fc871ebc3a5845d
author : Fujii Masao <[email protected]>
date : Thu, 12 Nov 2020 13:25:23 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 12 Nov 2020 13:25:23 +0900
Commit 1e53fe0e70 changed background processes so that they use
standard SIGHUP handler. Like that, this commit makes walreceiver
use standard SIGHUP and SIGTERM handlers, to simplify the code.
As the side effect of this commit, walreceiver can wake up and process
the configuration files promptly when receiving SIGHUP. Because the
standard SIGHUP handler sets the latch. On the other hand, previously
there could be a time lag between the receipt of SIGHUP and
the process of configuration files since the dedicated handler didn't
set the latch.
Author: Bharath Rupireddy, tweaked by Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACXPorUqePswDtOeM_s82v9RW32E1fYmOPZ5NuE+TWKj_A@mail.gmail.com
M src/backend/replication/walreceiver.c
Fix name of the macro for getting signature length trgm_gist.c
commit : 8ca8208ace4e67a9149946814d35d7cf18ff60d0
author : Alexander Korotkov <[email protected]>
date : Thu, 12 Nov 2020 06:19:16 +0300
committer: Alexander Korotkov <[email protected]>
date : Thu, 12 Nov 2020 06:19:16 +0300
911e702077 has introduced the opclass parameters including signature length
for a set of GiST opclasses. Due to copy-pasting, macro for getting the
signature length in trgm_gist.c was named LTREE_GET_ASIGLEN(). Fix that by
renaming this macro to just GET_SIGLEN().
Backpatch-through: 13
M contrib/pg_trgm/trgm_gist.c
pg_stat_statements: track number of rows processed by REFRESH MATERIALIZED VIEW.
commit : b62e6056a05c60ce9edf93e87e1487ae50245a04
author : Fujii Masao <[email protected]>
date : Thu, 12 Nov 2020 11:26:55 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 12 Nov 2020 11:26:55 +0900
Commit 6023b7ea71 allowed pg_stat_statements to track the number
of rows retrieved or affected by some utility commands including
CREATE MATERIALIZED VIEW. However it did not track the rowcount
of REFRESH MATERIALIZED VIEW. This commit allows pg_stat_statements
to track that.
To track that, this commit changes the query completion for
REFRESH MATERIALIZED VIEW so that it saves the rowcount. But note that
the rowcount is still not displayed in the command completion tag output.
That is, the display_rowcount flag of CMDTAG_REFRESH_MATERIALIZED_VIEW
command tag is left false in cmdtaglist.h. Otherwise, the change of
completion tag output might break applications using it.
Author: Katsuragi Yuta, Seino Yuki
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
M src/backend/commands/matview.c
Remove useless SHA256 initialization when not using backup manifests
commit : 03f9cd93eaa5e3385b075bb1a17d2adde3e23edd
author : Michael Paquier <[email protected]>
date : Thu, 12 Nov 2020 10:56:33 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 12 Nov 2020 10:56:33 +0900
Attempting to take a base backup with Postgres linking to a build of
OpenSSL with FIPS enabled currently fails with or even without a backup
manifest requested because of this mandatory SHA256 initialization used
for the manifest file itself. However, there is no need to do this
initialization at all if backup manifests are not needed because there
is no data to append to the manifest.
Note that being able to use backup manifests with OpenSSL+FIPS requires
a switch of the SHA2 implementation to use EVP, which would cause an ABI
breakage so this cannot be backpatched to 13 as it has been already
released, but at least avoiding this SHA256 initialization gives users
the possibility to take a base backup even when specifying --no-manifest
with pg_basebackup.
Author: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/backend/replication/backup_manifest.c
Remove duplicate code in brin_memtuple_initialize
commit : 42c63ab6e2c39c2fbfc70fa659b0cf5692178b77
author : Tomas Vondra <[email protected]>
date : Wed, 11 Nov 2020 18:37:36 +0100
committer: Tomas Vondra <[email protected]>
date : Wed, 11 Nov 2020 18:37:36 +0100
Commit 8bf74967dab moved some of the code from brin_new_memtuple to
brin_memtuple_initialize, but this resulted in some of the code being
duplicate. Fix by removing the duplicate lines and backpatch to 10.
Author: Tomas Vondra
Backpatch-through: 10
Discussion: https://postgr.es/m/5eb50c97-9a8e-b691-8c40-1b2a55611c4c%40enterprisedb.com
M src/backend/access/brin/brin_tuple.c
Fix some stray whitespace in parser files
commit : 0af302af40669d9bc6de7a202e4723fee39d9376
author : Peter Eisentraut <[email protected]>
date : Wed, 11 Nov 2020 17:37:18 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 11 Nov 2020 17:37:18 +0100
M src/backend/bootstrap/bootscanner.l
M src/backend/parser/gram.y
M src/backend/utils/adt/jsonpath_gram.y
M src/interfaces/ecpg/preproc/ecpg.trailer
Remove line missed in previous commit
commit : 4e3ffd5981536f93787ae6d81bc179da0acc423e
author : Magnus Hagander <[email protected]>
date : Wed, 11 Nov 2020 16:51:36 +0100
committer: Magnus Hagander <[email protected]>
date : Wed, 11 Nov 2020 16:51:36 +0100
M src/tools/msvc/vcregress.pl
Remove vacuumdb --analyze-in-stages from pg_upgrade tests
commit : 1e12a495b4749c7dc8e38e347554f2b24297f4ba
author : Magnus Hagander <[email protected]>
date : Wed, 11 Nov 2020 16:47:13 +0100
committer: Magnus Hagander <[email protected]>
date : Wed, 11 Nov 2020 16:47:13 +0100
This step was only there to test the script when we generated those, but
commit 8f113698b6 removed those scripts, so it's not needed anymore.
Reported-By: Peter Eisentraut
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_upgrade/test.sh
M src/tools/msvc/vcregress.pl
Add pg_nodiscard decorations to some functions
commit : a378ba49a5cfec073d885f4943e1c7e4b83abd3b
author : Peter Eisentraut <[email protected]>
date : Sat, 17 Oct 2020 08:38:39 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 17 Oct 2020 08:38:39 +0200
Especially for the list API such as lappend() forgetting to assign the
return value is a common problem.
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/include/nodes/pg_list.h
M src/include/utils/palloc.h
Add pg_nodiscard function declaration specifier
commit : c617aea6acdd9057a14d91ff294905b1b5509348
author : Peter Eisentraut <[email protected]>
date : Sat, 17 Oct 2020 08:38:39 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 17 Oct 2020 08:38:39 +0200
pg_nodiscard means the compiler should warn if the result of a
function call is ignored. The name "nodiscard" is chosen in alignment
with (possibly future) C and C++ standards. It maps to the GCC
attribute warn_unused_result.
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/include/c.h
Fix cases of discarding result from list API functions
commit : c77f6f50e4187bed38d1d36ae16b0c248e286d15
author : Peter Eisentraut <[email protected]>
date : Sat, 17 Oct 2020 08:38:39 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 17 Oct 2020 08:38:39 +0200
Two cases violated list APIs by throwing away the return value. While
the code was technically correct, it relied on internal knowledge of
the list implementation, and the code wasn't really gaining anything
that way. It is planned to make this a compiler warning in the
future, so just fix these cases by assigning the return value
properly.
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/backend/commands/lockcmds.c
M src/backend/parser/analyze.c
Fix and simplify some usages of TimestampDifference().
commit : ec29427ce2a451e7fef7a22de6db8147d8a80994
author : Tom Lane <[email protected]>
date : Tue, 10 Nov 2020 22:51:18 -0500
committer: Tom Lane <[email protected]>
date : Tue, 10 Nov 2020 22:51:18 -0500
Introduce TimestampDifferenceMilliseconds() to simplify callers
that would rather have the difference in milliseconds, instead of
the select()-oriented seconds-and-microseconds format. This gets
rid of at least one integer division per call, and it eliminates
some apparently-easy-to-mess-up arithmetic.
Two of these call sites were in fact wrong:
* pg_prewarm's autoprewarm_main() forgot to multiply the seconds
by 1000, thus ending up with a delay 1000X shorter than intended.
That doesn't quite make it a busy-wait, but close.
* postgres_fdw's pgfdw_get_cleanup_result() thought it needed to compute
microseconds not milliseconds, thus ending up with a delay 1000X longer
than intended. Somebody along the way had noticed this problem but
misdiagnosed the cause, and imposed an ad-hoc 60-second limit rather
than fixing the units. This was relatively harmless in context, because
we don't care that much about exactly how long this delay is; still,
it's wrong.
There are a few more callers of TimestampDifference() that don't
have a direct need for seconds-and-microseconds, but can't use
TimestampDifferenceMilliseconds() either because they do need
microsecond precision or because they might possibly deal with
intervals long enough to overflow 32-bit milliseconds. It might be
worth inventing another API to improve that, but that seems outside
the scope of this patch; so those callers are untouched here.
Given the fact that we are fixing some bugs, and the likelihood
that future patches might want to back-patch code that uses this
new API, back-patch to all supported branches.
Alexey Kondratov and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_prewarm/autoprewarm.c
M contrib/postgres_fdw/connection.c
M src/backend/access/transam/xlog.c
M src/backend/replication/walreceiverfuncs.c
M src/backend/replication/walsender.c
M src/backend/utils/adt/timestamp.c
M src/include/utils/timestamp.h
doc: fix spelling "connction" to "connection"
commit : b8b6a0124b21edd4aed24217a8e5ecf621ccd661
author : Bruce Momjian <[email protected]>
date : Tue, 10 Nov 2020 19:18:35 -0500
committer: Bruce Momjian <[email protected]>
date : Tue, 10 Nov 2020 19:18:35 -0500
Was wrong in commit 1a9388bd0f.
Reported-by: Tom Lane, Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_receivewal.sgml
Work around cross-version-upgrade issues created by commit 9e38c2bb5.
commit : 97f73a978fc1aca59c6ad765548ce0096d95a923
author : Tom Lane <[email protected]>
date : Tue, 10 Nov 2020 18:32:36 -0500
committer: Tom Lane <[email protected]>
date : Tue, 10 Nov 2020 18:32:36 -0500
Summarily changing the STYPE of regression-test aggregates that
depend on array_append or array_cat is an issue for the buildfarm's
cross-version-upgrade tests, because those aggregates (as defined
in the back branches) now won't load into HEAD. Although this seems
like only a minimal risk for genuine user-defined aggregates, we
need to do something for the buildfarm. Hence, adjust the aggregate
definitions, in both HEAD and the back branches.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/polymorphism.out
M src/test/regress/sql/polymorphism.sql
pg_rewind: Fix thinko in parsing target WAL.
commit : 72d172743e52f31bb874e3bbc07544b30bf0bb51
author : Heikki Linnakangas <[email protected]>
date : Tue, 10 Nov 2020 19:25:46 +0200
committer: Heikki Linnakangas <[email protected]>
date : Tue, 10 Nov 2020 19:25:46 +0200
It's entirely possible to see WAL for a relation that doesn't exist in
the target anymore. That happens when the relation was dropped later.
The refactoring in commit eb00f1d4b broke that case, by sanity-checking
the file type in the target before checking the flag forwhether it
exists there at all.
I noticed this during manual testing. Modify the 001_basic.pl test so
that it covers this case.
M src/bin/pg_rewind/filemap.c
M src/bin/pg_rewind/t/001_basic.pl
Fix out of date comment
commit : 3f16cb505d1d734674da2a2cbf2104ceae22f9b4
author : Magnus Hagander <[email protected]>
date : Tue, 10 Nov 2020 13:14:09 +0100
committer: Magnus Hagander <[email protected]>
date : Tue, 10 Nov 2020 13:14:09 +0100
M src/backend/postmaster/postmaster.c
Remove -o option to postmaster
commit : d2e4bf688e16f95d180b68b7cbb7881b12434dba
author : Magnus Hagander <[email protected]>
date : Tue, 10 Nov 2020 13:08:21 +0100
committer: Magnus Hagander <[email protected]>
date : Tue, 10 Nov 2020 13:08:21 +0100
This option was declared obsolete many years ago.
Reviewed-By: Tom Lane
Discussion: https://postgr.es/m/CABUevEyOE=9CQwZm2j=vwP5+6OLCSoxn9pBjK8gyRdkTzMfqtQ@mail.gmail.com
M doc/src/sgml/ref/postgres-ref.sgml
M src/backend/main/main.c
M src/backend/postmaster/postmaster.c
M src/backend/tcop/postgres.c
jit: Add support for LLVM 12.
commit : 6c57f2ed16e1281067e3dde7e006e2d1cab72116
author : Andres Freund <[email protected]>
date : Mon, 9 Nov 2020 20:01:33 -0800
committer: Andres Freund <[email protected]>
date : Mon, 9 Nov 2020 20:01:33 -0800
LLVM 12, to be released in a few months, made some breaking changes to
the Orc JIT interface. OrcV2 eventually will make it easier to support
features like concurrent JIT compilation, but this commit only allows
to compile against LLVM 12.
This commit is a bit bigger than desirable. That partially is because
the V2 interface is more granular than V1 interface, but also because
I chose to make some minor changes to < LLVM 12 code to keep the code
somewhat readable.
The LLVM 12 support will need to be backpatched. I plan to do so after
the patch stewed on the buildfarm for a few days.
Author: Andres Freund
Discussion: https://postgr.es/m/[email protected]
M src/backend/jit/llvm/llvmjit.c
M src/tools/pgindent/typedefs.list
Doc: clarify data type behavior of COALESCE and NULLIF.
commit : 24b83a5082541bdb1b333b7fcbe92f194128595c
author : Tom Lane <[email protected]>
date : Mon, 9 Nov 2020 12:02:24 -0500
committer: Tom Lane <[email protected]>
date : Mon, 9 Nov 2020 12:02:24 -0500
After studying the code, NULLIF is a lot more subtle than you might
have guessed.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M doc/src/sgml/typeconv.sgml
Remove ineffective heapam CHECK_FOR_INTERRUPTS().
commit : 180cf876d457c260291af070d883c5db5033cb3a
author : Peter Geoghegan <[email protected]>
date : Mon, 9 Nov 2020 09:00:12 -0800
committer: Peter Geoghegan <[email protected]>
date : Mon, 9 Nov 2020 09:00:12 -0800
Remove a CHECK_FOR_INTERRUPTS() call that could never actually handle an
interrupt. We always have a heap page buffer lock at this point.
Having a useless CHECK_FOR_INTERRUPTS() call is harmless but misleading.
It is probably possible to work around the immediate problem by moving
the CHECK_FOR_INTERRUPTS() to before the heap page buffer lock is
acquired. That isn't enough to make the function responsive to
interrupts, though. The index AM caller will still hold an exclusive
buffer lock of its own.
M src/backend/access/heap/heapam.c
Ignore attempts to \gset into specially treated variables.
commit : 098fb00799ffb026ff12c64bd21635f963cfc609
author : Noah Misch <[email protected]>
date : Mon, 9 Nov 2020 07:32:09 -0800
committer: Noah Misch <[email protected]>
date : Mon, 9 Nov 2020 07:32:09 -0800
If an interactive psql session used \gset when querying a compromised
server, the attacker could execute arbitrary code as the operating
system account running psql. Using a prefix not found among specially
treated variables, e.g. every lowercase string, precluded the attack.
Fix by issuing a warning and setting no variable for the column in
question. Users wanting the old behavior can use a prefix and then a
meta-command like "\set HISTSIZE :prefix_HISTSIZE". Back-patch to 9.5
(all supported versions).
Reviewed by Robert Haas. Reported by Nick Cleaton.
Security: CVE-2020-25696
M src/bin/psql/common.c
M src/bin/psql/variables.c
M src/bin/psql/variables.h
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
In security-restricted operations, block enqueue of at-commit user code.
commit : 0c3185e963d9f9dd0608214f7d732b84aa0888fe
author : Noah Misch <[email protected]>
date : Mon, 9 Nov 2020 07:32:09 -0800
committer: Noah Misch <[email protected]>
date : Mon, 9 Nov 2020 07:32:09 -0800
Specifically, this blocks DECLARE ... WITH HOLD and firing of deferred
triggers within index expressions and materialized view queries. An
attacker having permission to create non-temp objects in at least one
schema could execute arbitrary SQL functions under the identity of the
bootstrap superuser. One can work around the vulnerability by disabling
autovacuum and not manually running ANALYZE, CLUSTER, REINDEX, CREATE
INDEX, VACUUM FULL, or REFRESH MATERIALIZED VIEW. (Don't restore from
pg_dump, since it runs some of those commands.) Plain VACUUM (without
FULL) is safe, and all commands are fine when a trusted user owns the
target object. Performance may degrade quickly under this workaround,
however. Back-patch to 9.5 (all supported versions).
Reviewed by Robert Haas. Reported by Etienne Stalmans.
Security: CVE-2020-25695
M contrib/postgres_fdw/connection.c
M src/backend/access/transam/xact.c
M src/backend/commands/portalcmds.c
M src/backend/commands/trigger.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Remove analyze_new_cluster script from pg_upgrade
commit : 8f113698b63b15a4e0a4b15d3ee37238c1d1821d
author : Magnus Hagander <[email protected]>
date : Mon, 9 Nov 2020 12:14:59 +0100
committer: Magnus Hagander <[email protected]>
date : Mon, 9 Nov 2020 12:14:59 +0100
Since this script just runs vacuumdb anyway, remove the script and
replace the instructions to run it with instructions to run vacuumdb
directly.
Reviewed-By: Michael Paquier
Discussion: https://postgr.es/m/CABUevEwg5LDFzthhxzSj7sZGMiVsZe0VVNbzzwTQOHJ=rN7+5A@mail.gmail.com
M src/bin/pg_upgrade/.gitignore
M src/bin/pg_upgrade/Makefile
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/test.sh
M src/tools/msvc/vcregress.pl
Remove incorrect %s in string
commit : 7e84dd212040529de1811c7c97c996cc4e392015
author : Magnus Hagander <[email protected]>
date : Mon, 9 Nov 2020 10:36:49 +0100
committer: Magnus Hagander <[email protected]>
date : Mon, 9 Nov 2020 10:36:49 +0100
Appears to have been a copy/paste error in the original commit that
moved the messages to fe_utils/.
Author: Tang, Haiying <[email protected]>
Backpatch-through: 13
Discussion: https://postgr.es/m/3321cbcea76d4d2c8320a05c19b9304a@G08CNEXMBPEKD05.g08.fujitsu.local
M src/fe_utils/cancel.c
doc: Add note about pg_settings and customized options into catalogs.sgml.
commit : ef60de67ebde6dfd1ea09b4d08166736bf05698c
author : Fujii Masao <[email protected]>
date : Mon, 9 Nov 2020 15:10:26 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 9 Nov 2020 15:10:26 +0900
The pg_settings view does not display customized options until
the extension module that defines them has been loaded. This commit
add the note about that behavior, into the docs.
Author: John Naylor
Reviewed-by: Tom Lane, Fujii Masao
Discussion: https://postgr.es/m/CAFBsxsGsBZsG=cLM0Op5HFb2Ks6SzJrOc_eRO_jcKSNuqFRKnQ@mail.gmail.com
M doc/src/sgml/catalogs.sgml
Fix parsePGArray() error checking in pg_dump.
commit : 3636efa11945af64479995609762b2626c6c319a
author : Thomas Munro <[email protected]>
date : Mon, 9 Nov 2020 16:01:51 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 9 Nov 2020 16:01:51 +1300
Coverity complained about a defect in commit 257836a7:
Calling "parsePGArray" without checking return value (as is
done elsewhere 11 out of 13 times).
Fix, and also check for empty strings explicitly (NULL as represented by
PQgetvalue()). That worked correctly before only because parsePGArray()
happens to set *nitems = 0 when it fails on an empty string. Also
convert a sanity check assertion to an error to be more paranoid, and
pgindent a nearby line.
Reported-by: Michael Paquier <[email protected]>
M src/bin/pg_dump/pg_dump.c
In INSERT/UPDATE, use the table's real tuple descriptor as target.
commit : 8b39345a9d9abd1c371f8458301a227d0abb93b2
author : Tom Lane <[email protected]>
date : Sun, 8 Nov 2020 13:08:36 -0500
committer: Tom Lane <[email protected]>
date : Sun, 8 Nov 2020 13:08:36 -0500
This back-patches commit 20d3fe900 into the v12 and v13 branches.
At the time I thought that commit was not fixing any observable
bug, but Bertrand Drouvot showed otherwise: adding a dropped column
to the previously-considered scenario crashes v12 and v13, unless the
dropped column happens to be an integer. That is, of course, because
the tupdesc we derive from the plan output tlist fails to describe
the dropped column accurately, so that we'll do the wrong thing with
a tuple in which that column isn't NULL.
There is no bug in pre-v12 branches because they already did use
the table's real tuple descriptor for any trigger-returned tuple.
It seems that this set of bugs can be blamed on the changes that
removed es_trig_tuple_slot, though I've not attempted to pin that
down precisely.
Although there's no code change needed in HEAD, update the test case
to include a dropped column there too.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Fix assertion in collation version lookup.
commit : d50e3b1f8d763d51080ca4f6cbaa6818a6472292
author : Thomas Munro <[email protected]>
date : Sun, 8 Nov 2020 20:43:45 +1300
committer: Thomas Munro <[email protected]>
date : Sun, 8 Nov 2020 20:43:45 +1300
Commit 257836a7 included an assertion that a version lookup routine is
not trying to look up "C" or "POSIX", but that case is reachable with
the user-facing SQL function pg_collation_actual_version(). Remove the
assertion.
M src/backend/utils/adt/pg_locale.c
Fix test for error message change
commit : 8cff66d309b90e03f1e76494375960f35eff6424
author : Peter Eisentraut <[email protected]>
date : Sun, 8 Nov 2020 07:48:18 +0100
committer: Peter Eisentraut <[email protected]>
date : Sun, 8 Nov 2020 07:48:18 +0100
fix for 6be725e701611660b36642de9ff1d665a1eb24f5
M src/test/ssl/t/002_scram.pl
Improve nbtree README's LP_DEAD section.
commit : 5a2f154a2ecaf545000a3ff3cdbadc76ae1df30a
author : Peter Geoghegan <[email protected]>
date : Sat, 7 Nov 2020 18:51:12 -0800
committer: Peter Geoghegan <[email protected]>
date : Sat, 7 Nov 2020 18:51:12 -0800
The description of how LP_DEAD bit setting by index scans works
following commit 2ed5b87f was rather unclear. Clean that up a bit.
Also refer to LP_DEAD bit setting within _bt_check_unique() at the start
of the same section. This mechanism may actually be more important than
the generic kill_prior_tuple mechanism that the section focuses on, so
it at least deserves to be mentioned in passing.
M src/backend/access/nbtree/README
Message style improvements
commit : 52eec1c53aa6a7df1683fba79078793f1d0eba42
author : Alvaro Herrera <[email protected]>
date : Sat, 7 Nov 2020 19:33:43 -0300
committer: Alvaro Herrera <[email protected]>
date : Sat, 7 Nov 2020 19:33:43 -0300
* Avoid pointlessly highlighting that an index vacuum was executed by a
parallel worker; user doesn't care.
* Don't give the impression that a non-concurrent reindex of an invalid
index on a TOAST table would work, because it wouldn't.
* Add a "translator:" comment for a mysterious message.
Discussion: https://postgr.es/m/[email protected]
Reviewed-by: Michael Paquier <[email protected]>
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/indexcmds.c
M src/backend/libpq/be-secure-openssl.c
Fix redundant error messages in client tools
commit : 6be725e701611660b36642de9ff1d665a1eb24f5
author : Peter Eisentraut <[email protected]>
date : Sat, 7 Nov 2020 22:15:52 +0100
committer: Peter Eisentraut <[email protected]>
date : Sat, 7 Nov 2020 22:15:52 +0100
A few client tools duplicate error messages already provided by libpq.
Discussion: https://www.postgresql.org/message-id/flat/3e937641-88a1-e697-612e-99bba4b8e5e4%40enterprisedb.com
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/psql/startup.c
Avoid re-using output variables in new ecpg test case.
commit : eed4356fad84b0fd6e3caa49c7006f401159ac9a
author : Tom Lane <[email protected]>
date : Sat, 7 Nov 2020 16:25:42 -0500
committer: Tom Lane <[email protected]>
date : Sat, 7 Nov 2020 16:25:42 -0500
The buildfarm thinks this leads to memory stomps, though annoyingly
I can't duplicate that here. The existing code in strings.pgc is
doing something that doesn't seem to be sanctioned at all really
by the documentation, but I'm disinclined to try to make that nicer
right now. Let's just declare some more output variables in hopes
of working around it.
M src/interfaces/ecpg/test/expected/preproc-strings.c
M src/interfaces/ecpg/test/expected/preproc-strings.stderr
M src/interfaces/ecpg/test/preproc/strings.h
M src/interfaces/ecpg/test/preproc/strings.pgc
Fix ecpg's mishandling of B'...' and X'...' literals.
commit : 1e3868ab3bef5cfa0f4d44a6937a880be7a3a482
author : Tom Lane <[email protected]>
date : Sat, 7 Nov 2020 15:03:44 -0500
committer: Tom Lane <[email protected]>
date : Sat, 7 Nov 2020 15:03:44 -0500
These were broken in multiple ways:
* The xbstart and xhstart lexer actions neglected to set
"state_before_str_start" before transitioning to the xb/xh states,
thus possibly resulting in "internal error: unreachable state" later.
* The test for valid string contents at the end of xb state was flat out
wrong, as it accounted incorrectly for the "b" prefix that the xbstart
action had injected. Meanwhile, the xh state had no such check at all.
* The generated literal value failed to include any quote marks.
* The grammar did the wrong thing anyway, typically ignoring the
literal value and emitting something else, since BCONST and XCONST
tokens were handled randomly differently from SCONST tokens.
The first of these problems is evidently an oversight in commit
7f380c59f, but the others seem to be very ancient. The lack of
complaints shows that ECPG users aren't using these syntaxes much
(although I do vaguely remember one previous complaint).
As written, this patch is dependent on 7f380c59f, so it can't go
back further than v13. Given the shortage of complaints, I'm not
excited about adapting the patch to prior branches.
Report and patch by Shenhao Wang (test case adjusted by me)
Discussion: https://postgr.es/m/d6402f1bacb74ecba22ef715dbba17fd@G08CNEXMBPEKD06.g08.fujitsu.local
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/ecpg.type
M src/interfaces/ecpg/preproc/parse.pl
M src/interfaces/ecpg/preproc/pgc.l
M src/interfaces/ecpg/test/expected/preproc-strings.c
M src/interfaces/ecpg/test/expected/preproc-strings.stderr
M src/interfaces/ecpg/test/expected/preproc-strings.stdout
M src/interfaces/ecpg/test/preproc/strings.pgc
Move catalog index declarations
commit : bdc4edbea6fc847f806e1e7118d730e159512bfc
author : Peter Eisentraut <[email protected]>
date : Sat, 7 Nov 2020 12:11:40 +0100
committer: Peter Eisentraut <[email protected]>
date : Sat, 7 Nov 2020 12:11:40 +0100
Move the system catalog index declarations from catalog/indexing.h to
the respective parent tables' catalog/pg_*.h files. The original
reason for having it split was that the old genbki system produced the
output in the order of the catalog files it read, so all the indexing
stuff needed to come separately. But this is no longer the case, and
keeping it together makes more sense.
Reviewed-by: John Naylor <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M contrib/dblink/dblink.c
M contrib/sepgsql/database.c
M contrib/sepgsql/label.c
M contrib/sepgsql/proc.c
M contrib/sepgsql/relation.c
M contrib/sepgsql/schema.c
M doc/src/sgml/bki.sgml
M src/backend/catalog/Makefile
M src/backend/catalog/catalog.c
M src/backend/catalog/objectaddress.c
M src/backend/commands/aggregatecmds.c
M src/backend/commands/tablecmds.c
M src/backend/partitioning/partdesc.c
M src/backend/replication/logical/message.c
M src/backend/rewrite/rewriteDefine.c
M src/backend/utils/adt/enum.c
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/cache/evtcache.c
M src/backend/utils/cache/relfilenodemap.c
M src/backend/utils/cache/syscache.c
M src/backend/utils/cache/ts_cache.c
M src/backend/utils/cache/typcache.c
M src/backend/utils/init/postinit.c
M src/include/catalog/duplicate_oids
M src/include/catalog/genbki.h
M src/include/catalog/indexing.h
M src/include/catalog/pg_aggregate.h
M src/include/catalog/pg_am.h
M src/include/catalog/pg_amop.h
M src/include/catalog/pg_amproc.h
M src/include/catalog/pg_attrdef.h
M src/include/catalog/pg_attribute.h
M src/include/catalog/pg_auth_members.h
M src/include/catalog/pg_authid.h
M src/include/catalog/pg_cast.h
M src/include/catalog/pg_class.h
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_conversion.h
M src/include/catalog/pg_database.h
M src/include/catalog/pg_db_role_setting.h
M src/include/catalog/pg_default_acl.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_description.h
M src/include/catalog/pg_enum.h
M src/include/catalog/pg_event_trigger.h
M src/include/catalog/pg_extension.h
M src/include/catalog/pg_foreign_data_wrapper.h
M src/include/catalog/pg_foreign_server.h
M src/include/catalog/pg_foreign_table.h
M src/include/catalog/pg_index.h
M src/include/catalog/pg_inherits.h
M src/include/catalog/pg_init_privs.h
M src/include/catalog/pg_language.h
M src/include/catalog/pg_largeobject.h
M src/include/catalog/pg_largeobject_metadata.h
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_opclass.h
M src/include/catalog/pg_operator.h
M src/include/catalog/pg_opfamily.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_policy.h
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_publication.h
M src/include/catalog/pg_publication_rel.h
M src/include/catalog/pg_range.h
M src/include/catalog/pg_replication_origin.h
M src/include/catalog/pg_rewrite.h
M src/include/catalog/pg_seclabel.h
M src/include/catalog/pg_sequence.h
M src/include/catalog/pg_shdepend.h
M src/include/catalog/pg_shdescription.h
M src/include/catalog/pg_shseclabel.h
M src/include/catalog/pg_statistic.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/catalog/pg_subscription.h
M src/include/catalog/pg_subscription_rel.h
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_transform.h
M src/include/catalog/pg_trigger.h
M src/include/catalog/pg_ts_config.h
M src/include/catalog/pg_ts_config_map.h
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_ts_parser.h
M src/include/catalog/pg_ts_template.h
M src/include/catalog/pg_type.h
M src/include/catalog/pg_user_mapping.h
M src/include/catalog/renumber_oids.pl
M src/include/catalog/unused_oids
M src/tools/msvc/Solution.pm
Move catalog toast table declarations
commit : b4c9695e79f79d39d55853acac4c1493ac70a529
author : Peter Eisentraut <[email protected]>
date : Sat, 7 Nov 2020 12:11:40 +0100
committer: Peter Eisentraut <[email protected]>
date : Sat, 7 Nov 2020 12:11:40 +0100
Move the system catalog toast table declarations from
catalog/toasting.h to the respective parent tables' catalog/pg_*.h
files. The original reason for having it split was that the old
genbki system produced the output in the order of the catalog files it
read, so all the toasting stuff needed to come separately. But this
is no longer the case, and keeping it together makes more sense.
Reviewed-by: John Naylor <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/bki.sgml
M src/backend/catalog/Makefile
M src/backend/catalog/catalog.c
M src/backend/catalog/toasting.c
M src/include/catalog/duplicate_oids
M src/include/catalog/genbki.h
M src/include/catalog/pg_aggregate.h
M src/include/catalog/pg_attrdef.h
M src/include/catalog/pg_authid.h
M src/include/catalog/pg_constraint.h
M src/include/catalog/pg_database.h
M src/include/catalog/pg_db_role_setting.h
M src/include/catalog/pg_default_acl.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_description.h
M src/include/catalog/pg_event_trigger.h
M src/include/catalog/pg_extension.h
M src/include/catalog/pg_foreign_data_wrapper.h
M src/include/catalog/pg_foreign_server.h
M src/include/catalog/pg_foreign_table.h
M src/include/catalog/pg_init_privs.h
M src/include/catalog/pg_language.h
M src/include/catalog/pg_namespace.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_policy.h
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_replication_origin.h
M src/include/catalog/pg_rewrite.h
M src/include/catalog/pg_seclabel.h
M src/include/catalog/pg_shdescription.h
M src/include/catalog/pg_shseclabel.h
M src/include/catalog/pg_statistic.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_statistic_ext_data.h
M src/include/catalog/pg_subscription.h
M src/include/catalog/pg_tablespace.h
M src/include/catalog/pg_trigger.h
M src/include/catalog/pg_ts_dict.h
M src/include/catalog/pg_type.h
M src/include/catalog/pg_user_mapping.h
M src/include/catalog/renumber_oids.pl
M src/include/catalog/toasting.h
M src/include/catalog/unused_oids
M src/tools/msvc/Solution.pm
Plug memory leak in index_get_partition
commit : 623644f02cbde7ad3812b201bd36213a206c3341
author : Alvaro Herrera <[email protected]>
date : Fri, 6 Nov 2020 22:52:16 -0300
committer: Alvaro Herrera <[email protected]>
date : Fri, 6 Nov 2020 22:52:16 -0300
The list of indexes was being leaked when asked for an index that
doesn't have an index partition in the table partition. Not a common
case admittedly --and in most cases where it occurs, caller throws an
error anyway-- but worth fixing for cleanliness and in case any
third-party code is calling this function.
While at it, remove use of lfirst_oid() to obtain a value we already
have.
Author: Justin Pryzby <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/partition.c
Add GUC_LIST_INPUT and GUC_LIST_QUOTE to unix_socket_directories
commit : a05dbf477b0ef173adb1ae5d004cbeb0cf400b66
author : Michael Paquier <[email protected]>
date : Sat, 7 Nov 2020 10:30:22 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 7 Nov 2020 10:30:22 +0900
This should have been done in the initial commit that made
unix_socket_directories a list as of c9b0cbe. This change allows to
support correctly the case of ALTER SYSTEM, where it is possible to
specify multiple paths as a list, like the following pattern where
flattening is applied to each item:
ALTER SYSTEM SET unix_socket_directories = '/path1', '/path2';
Any parameters specified in postgresql.conf are parsed the same way, so
there is no compatibility change. pg_dump has a hardcoded list of
parameters marked with GUC_LIST_QUOTE, that gets its routine update.
These are reordered alphabetically for clarity.
Author: Ian Lawrence Barwick
Reviewed-by: Peter Eisentraunt, Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/CAB8KJ=iMOtNY6_sUwV=LQVCJ2zgYHBDyNzVfvE5GN3WQ3v9kQg@mail.gmail.com
M src/backend/utils/misc/guc.c
M src/bin/pg_dump/dumputils.c
Fix minor issues with new unicode {de,re}composition code
commit : ceaeac54f75f0117a7b3caa2f8ecba5cfac18ee5
author : Michael Paquier <[email protected]>
date : Sat, 7 Nov 2020 10:15:58 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 7 Nov 2020 10:15:58 +0900
The table generation script would incorrectly complain in the
recomposition sorting when matching code points. This would not have
caused the generation of an incorrect table. Note that this condition
is not reachable yet, but could have been reached with future updates.
pg_bswap.h does not need to be included in the frontend.x
Author: John Naylor
Discussion: https://postgr.es/m/CAFBsxsGWmExpvv=61vtDKCs7+kBbhkwBDL2Ph9CacziFKnV_yw@mail.gmail.com
M src/common/unicode/generate-unicode_norm_table.pl
M src/common/unicode_norm.c
Properly detoast data in brin_form_tuple
commit : 7577dd84807a808f295dbfb0191540750ff98002
author : Tomas Vondra <[email protected]>
date : Sat, 7 Nov 2020 00:39:19 +0100
committer: Tomas Vondra <[email protected]>
date : Sat, 7 Nov 2020 00:39:19 +0100
brin_form_tuple failed to consider the values may be toasted, inserting
the toast pointer into the index. This may easily result in index
corruption, as the toast data may be deleted and cleaned up by vacuum.
The cleanup however does not care about indexes, leaving invalid toast
pointers behind, which triggers errors like this:
ERROR: missing chunk number 0 for toast value 16433 in pg_toast_16426
A less severe consequence are inconsistent failures due to the index row
being too large, depending on whether brin_form_tuple operated on plain
or toasted version of the row. For example
CREATE TABLE t (val TEXT);
INSERT INTO t VALUES ('... long value ...')
CREATE INDEX idx ON t USING brin (val);
would likely succeed, as the row would likely include toast pointer.
Switching the order of INSERT and CREATE INDEX would likely fail:
ERROR: index row size 8712 exceeds maximum 8152 for index "idx"
because this happens before the row values are toasted.
The bug exists since PostgreSQL 9.5 where BRIN indexes were introduced.
So backpatch all the way back.
Author: Tomas Vondra
Reviewed-by: Alvaro Herrera
Backpatch-through: 9.5
Discussion: https://postgr.es/m/20201001184133.oq5uq75sb45pu3aw@development
Discussion: https://postgr.es/m/20201104010544.zexj52mlldagzowv%40development
M src/backend/access/brin/brin_tuple.c
M src/test/regress/expected/brin.out
M src/test/regress/sql/brin.sql
Revert "Accept relations of any kind in LOCK TABLE".
commit : eeda7f6338095701cfe1ba3da37070508efe019e
author : Tom Lane <[email protected]>
date : Fri, 6 Nov 2020 16:17:56 -0500
committer: Tom Lane <[email protected]>
date : Fri, 6 Nov 2020 16:17:56 -0500
Revert 59ab4ac32, as well as the followup fix 33862cb9c, in all
branches. We need to think a bit harder about what the behavior
of LOCK TABLE on views should be, and there's no time for that
before next week's releases. We'll take another crack at this
later.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/lock.sgml
M src/backend/commands/lockcmds.c
M src/test/regress/expected/lock.out
M src/test/regress/sql/lock.sql
Revert "pg_dump: Lock all relations, not just plain tables".
commit : d3adaabaf7d555ec8bb1d83c43f72e79f1bf0b7d
author : Tom Lane <[email protected]>
date : Fri, 6 Nov 2020 15:48:04 -0500
committer: Tom Lane <[email protected]>
date : Fri, 6 Nov 2020 15:48:04 -0500
Revert 403a3d91c, as well as the followup fix 7f4235032, in all
branches. We need to think a bit harder about what the behavior
of LOCK TABLE on views should be, and there's no time for that
before next week's releases. We'll take another crack at this
later.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_backup_db.h
M src/bin/pg_dump/pg_dump.c
pg_prewarm: make autoprewarm leader use standard SIGHUP and SIGTERM handlers.
commit : 53f614f1302d56e5fbb98826c7eee3c715411184
author : Fujii Masao <[email protected]>
date : Sat, 7 Nov 2020 02:08:06 +0900
committer: Fujii Masao <[email protected]>
date : Sat, 7 Nov 2020 02:08:06 +0900
Commit 1e53fe0e70 changed background processes so that they use
standard SIGHUP handler. Like that, this commit makes autoprewarm leader
process also use standard SIGHUP and SIGTERM handlers, to simplify the code.
Author: Bharath Rupireddy
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACXPorUqePswDtOeM_s82v9RW32E1fYmOPZ5NuE+TWKj_A@mail.gmail.com
M contrib/pg_prewarm/autoprewarm.c
Add pg_strong_random_init function to initialize random number generator
commit : 5ee180a3947060b98284a935f26f92c71d698f7c
author : Magnus Hagander <[email protected]>
date : Fri, 6 Nov 2020 13:21:28 +0100
committer: Magnus Hagander <[email protected]>
date : Fri, 6 Nov 2020 13:21:28 +0100
Currently only OpenSSL requires this initialization, but in the future
other SSL implementations are likely to need it as well. Abstracting
this functionality out into a separate function makes this cleaner and
more clear, and also removes the dependency on OpenSSL headers from
fork_process.c.
OpenSSL is special in that we need to initialize this random number
generator even if we're not going to use it directly, until we drop
support for everything prior to OpenSSL 1.1.1. (And of course also if we
actually use it). All other implementations are left empty at this time,
but more are expected to be added in the future.
Author: Daniel Gustafsson <[email protected]>, Michael Paquier <[email protected]>
Reviewed-By: Magnus Hagander <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/fork_process.c
M src/include/port.h
M src/port/pg_strong_random.c
Use strlcpy instead of memcpy for copying the slot name in pgstat.c.
commit : 4f841ce3f7f4d429a3a275f82745d63c78cde4b2
author : Amit Kapila <[email protected]>
date : Fri, 6 Nov 2020 08:12:48 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 6 Nov 2020 08:12:48 +0530
There is no outright bug here but it is better to be consistent with the
usage at other places in the same file. In the passing, fix a wrong
assertion in pgstat_recv_replslot.
Author: Kyotaro Horiguchi
Reviewed-by: Sawada Masahiko and Amit Kapila
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/pgstat.c
Fix wal_consistency_checking nbtree bug.
commit : efc5dcfd8ad4e1df633025d8a91b64cd44d93f42
author : Peter Geoghegan <[email protected]>
date : Thu, 5 Nov 2020 15:01:40 -0800
committer: Peter Geoghegan <[email protected]>
date : Thu, 5 Nov 2020 15:01:40 -0800
wal_consistency_checking indicated an inconsistency in certain cases
involving nbtree page deletion. The underlying issue is that there was
a minor difference between the page image produced after a REDO routine
ran and the corresponding page image following original execution.
This harmless inconsistency has been around forever. We more or less
expect total consistency among even deleted nbtree pages these days,
though, so this won't do anymore.
To fix, tweak the REDO routine to match original execution.
Oversight in commit f47b5e13.
M src/backend/access/nbtree/nbtxlog.c
Don't throw an error for LOCK TABLE on a self-referential view.
commit : 5b7bfc39726ff9f6c52dd73e337c34e74e0d1f39
author : Tom Lane <[email protected]>
date : Thu, 5 Nov 2020 11:44:32 -0500
committer: Tom Lane <[email protected]>
date : Thu, 5 Nov 2020 11:44:32 -0500
LOCK TABLE has complained about "infinite recursion" when applied
to a self-referential view, ever since we made it recurse into views
in v11. However, that breaks pg_dump's new assumption that it's
okay to lock every relation. There doesn't seem to be any good
reason to throw an error: if we just abandon the recursion, we've
still satisfied the requirement of locking every referenced relation.
Per bug #16703 from Andrew Bille (via Alexander Lakhin).
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/lockcmds.c
M src/test/regress/expected/lock.out
M src/test/regress/sql/lock.sql
Fix nbtree cleanup-only VACUUM stats inaccuracies.
commit : 48e1291342dd7771cf8c67aa1d7ec1f394b95dd8
author : Peter Geoghegan <[email protected]>
date : Wed, 4 Nov 2020 18:42:27 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 4 Nov 2020 18:42:27 -0800
Logic for counting heap TIDs from posting list tuples (added by commit
0d861bbb) was faulty. It didn't count any TIDs/index tuples in the
event of no callback being set. This meant that we incorrectly counted
no index tuples in clean-up only VACUUMs, which could lead to
pg_class.reltuples being spuriously set to 0 in affected indexes.
To fix, go back to counting items from the page in cases where there is
no callback. This approach isn't very accurate, but it works well
enough in practice while avoiding the expense of accessing every index
tuple during cleanup-only VACUUMs.
Author: Peter Geoghegan <[email protected]>
Reported-By: Jehan-Guillaume de Rorthais <[email protected]>
https://postgr.es/m/20201023174451.69e358f1@firost
Backpatch: 13-, where nbtree deduplication was introduced
M src/backend/access/nbtree/nbtree.c
Fix unlinking of SLRU segments.
commit : c732c3f8c122009de373cff9b44b5cf0992ba1bf
author : Thomas Munro <[email protected]>
date : Thu, 5 Nov 2020 13:45:32 +1300
committer: Thomas Munro <[email protected]>
date : Thu, 5 Nov 2020 13:45:32 +1300
Commit dee663f7 intended to drop any queued up fsync requests before
unlinking segment files, but missed a code path. Fix, by centralizing
the forget-and-unlink code into a single function.
Reported-by: Tomas Vondra <[email protected]>
Discussion: https://postgr.es/m/20201104013205.icogbi773przyny5%40development
M src/backend/access/transam/slru.c
Remove underflow error in float division with infinite divisor.
commit : fac83dbd6fe1ac3d4125bfa39f287f95bffe6cda
author : Tom Lane <[email protected]>
date : Wed, 4 Nov 2020 18:11:15 -0500
committer: Tom Lane <[email protected]>
date : Wed, 4 Nov 2020 18:11:15 -0500
float4_div and float8_div correctly produced zero for zero divided
by infinity, but threw an underflow error for nonzero finite values
divided by infinity. This seems wrong; at the very least it's
inconsistent with the behavior recently implemented for numeric
infinities. Remove the error and allow zero to be returned.
This patch also removes a useless isinf() test from the overflow
checks in these functions (non-Inf divided by Inf can't produce Inf).
Extracted from a larger patch; this seems significant outside the
context of geometric operators, so it deserves its own commit.
Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAGf+fX70rWFOk5cd00uMfa__0yP+vtQg5ck7c2Onb-Yczp0URA@mail.gmail.com
M src/include/utils/float.h
M src/test/regress/expected/float4-misrounded-input.out
M src/test/regress/expected/float4.out
M src/test/regress/expected/float8.out
M src/test/regress/sql/float4.sql
M src/test/regress/sql/float8.sql
Declare assorted array functions using anycompatible not anyelement.
commit : 9e38c2bb5093ceb0c04d6315ccd8975bd17add66
author : Tom Lane <[email protected]>
date : Wed, 4 Nov 2020 16:09:55 -0500
committer: Tom Lane <[email protected]>
date : Wed, 4 Nov 2020 16:09:55 -0500
Convert array_append, array_prepend, array_cat, array_position,
array_positions, array_remove, array_replace, and width_bucket
to use anycompatiblearray. This is a simple extension of commit
5c292e6b9 to hit some other places where there's a pretty obvious
gain in usability from doing so.
Ideally we'd also modify other functions taking multiple old-style
polymorphic arguments. But most of the remainder are tied into one
or more operator classes, making any such change a much larger can of
worms than I desire to open right now.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M doc/src/sgml/xaggr.sgml
M src/include/catalog/catversion.h
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/arrays.out
M src/test/regress/expected/polymorphism.out
M src/test/regress/sql/arrays.sql
M src/test/regress/sql/polymorphism.sql
Declare lead() and lag() using anycompatible not anyelement.
commit : 5c292e6b90433c760a3e15027646c7b94afd0cdd
author : Tom Lane <[email protected]>
date : Wed, 4 Nov 2020 15:08:37 -0500
committer: Tom Lane <[email protected]>
date : Wed, 4 Nov 2020 15:08:37 -0500
This allows use of a "default" expression that doesn't slavishly
match the data column's type. Formerly you got something like
"function lag(numeric, integer, integer) does not exist", which
is not just unhelpful but actively misleading.
The SQL spec suggests that the default should be coerced to the data
column's type, but this implementation instead chooses the common
supertype, which seems at least as reasonable.
(Note: I took the opportunity to run "make reformat-dat-files" on
pg_proc.dat, so this commit includes some cosmetic changes to
recently-added entries that aren't related to lead/lag.)
Vik Fearing
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/window.out
M src/test/regress/sql/window.sql
Improve our ability to regurgitate SQL-syntax function calls.
commit : 40c24bfef92530bd846e111c1742c2a54441c62c
author : Tom Lane <[email protected]>
date : Wed, 4 Nov 2020 12:34:50 -0500
committer: Tom Lane <[email protected]>
date : Wed, 4 Nov 2020 12:34:50 -0500
The SQL spec calls out nonstandard syntax for certain function calls,
for example substring() with numeric position info is supposed to be
spelled "SUBSTRING(string FROM start FOR count)". We accept many
of these things, but up to now would not print them in the same format,
instead simplifying down to "substring"(string, start, count).
That's long annoyed me because it creates an interoperability
problem: we're gratuitously injecting Postgres-specific syntax into
what might otherwise be a perfectly spec-compliant view definition.
However, the real reason for addressing it right now is to support
a planned change in the semantics of EXTRACT() a/k/a date_part().
When we switch that to returning numeric, we'll have the parser
translate EXTRACT() to some new function name (might as well be
"extract" if you ask me) and then teach ruleutils.c to reverse-list
that per SQL spec. In this way existing calls to date_part() will
continue to have the old semantics.
To implement this, invent a new CoercionForm value COERCE_SQL_SYNTAX,
and make the parser insert that rather than COERCE_EXPLICIT_CALL when
the input has SQL-spec decoration. (But if the input has the form of
a plain function call, continue to mark it COERCE_EXPLICIT_CALL, even
if it's calling one of these functions.) Then ruleutils.c recognizes
COERCE_SQL_SYNTAX as a cue to emit SQL call syntax. It can know
which decoration to emit using hard-wired knowledge about the
functions that could be called this way. (While this solution isn't
extensible without manual additions, neither is the grammar, so this
doesn't seem unmaintainable.) Notice that this solution will
reverse-list a function call with SQL decoration only if it was
entered that way; so dump-and-reload will not by itself produce any
changes in the appearance of views.
This requires adding a CoercionForm field to struct FuncCall.
(I couldn't resist the temptation to rearrange that struct's
field order a tad while I was at it.) FuncCall doesn't appear
in stored rules, so that change isn't a reason for a catversion
bump, but I did one anyway because the new enum value for
CoercionForm fields could confuse old backend code.
Possible future work:
* Perhaps CoercionForm should now be renamed to DisplayForm,
or something like that, to reflect its more general meaning.
This'd require touching a couple hundred places, so it's not
clear it's worth the code churn.
* The SQLValueFunction node type, which was invented partly for
the same goal of improving SQL-compatibility of view output,
could perhaps be replaced with regular function calls marked
with COERCE_SQL_SYNTAX. It's unclear if this would be a net
code savings, however.
Discussion: https://postgr.es/m/[email protected]
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/makefuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_func.c
M src/backend/parser/parse_utilcmd.c
M src/backend/utils/adt/ruleutils.c
M src/include/catalog/catversion.h
M src/include/nodes/makefuncs.h
M src/include/nodes/parsenodes.h
M src/include/nodes/primnodes.h
M src/test/modules/test_rls_hooks/test_rls_hooks.c
M src/test/regress/expected/create_view.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/create_view.sql
Remove useless entries for aggregate functions from fmgrtab.c.
commit : f21636e5d5b8394ed076e18ddc5f4ba710c69c99
author : Tom Lane <[email protected]>
date : Wed, 4 Nov 2020 11:25:56 -0500
committer: Tom Lane <[email protected]>
date : Wed, 4 Nov 2020 11:25:56 -0500
Gen_fmgrtab.pl treated aggregate functions the same as other built-in
functions, which is wasteful because there is no real need to have
entries for them in the fmgr_builtins[] table. Suppressing those
entries saves about 3KB in the compiled table on my machine; which
is not a lot but it's not nothing either, considering that that
table is pretty "hot". The only outside code change needed is
that ExecInitWindowAgg() can't be allowed to call fmgr_info_cxt()
on a plain aggregate function. But that saves a few cycles anyway.
Having done that, the aggregate_dummy() function is unreferenced
and might as well be dropped. Using "aggregate_dummy" as the prosrc
value for an aggregate is now just a documentation convention not
something that matters. There was some discussion of using NULL
instead to save a few bytes in pg_proc, but we'd have to remove
prosrc's BKI_FORCE_NOT_NULL marking which doesn't seem a great idea.
Anyway, it's possible there's client-side code that expects to
see "aggregate_dummy" there, so I'm loath to change it without a
strong reason.
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/pg_aggregate.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeWindowAgg.c
M src/backend/utils/Gen_fmgrtab.pl
Fix segmentation fault that commit ac22929a26 caused.
commit : 113d3591b859fb8dc191bc0599d1ad62d91f1aa4
author : Fujii Masao <[email protected]>
date : Wed, 4 Nov 2020 21:49:00 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 4 Nov 2020 21:49:00 +0900
Commit ac22929a26 changed recoveryWakeupLatch so that it's reset to
NULL at the end of recovery. This change could cause a segmentation fault
in the buildfarm member 'elver'.
Previously the latch was reset to NULL after calling ShutdownWalRcv().
But there could be a window between ShutdownWalRcv() and the actual
exit of walreceiver. If walreceiver set the latch during that window,
the segmentation fault could happen.
To fix the issue, this commit changes walreceiver so that it sets
the latch only when the latch has not been reset to NULL yet.
Author: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
Enable hash partitioning of text arrays
commit : 560564d3addcb6c54e13c1ca8bd11afafbb7ba11
author : Peter Eisentraut <[email protected]>
date : Wed, 4 Nov 2020 07:47:06 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 4 Nov 2020 07:47:06 +0100
hash_array_extended() needs to pass PG_GET_COLLATION() to the hash
function of the element type. Otherwise, the hash function of a
collation-aware data type such as text will error out, since the
introduction of nondeterministic collation made hash functions require
a collation, too.
The consequence of this is that before this change, hash partitioning
using an array over text in the partition key would not work.
Reviewed-by: Heikki Linnakangas <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/32c1fdae-95c6-5dc6-058a-a90330a3b621%40enterprisedb.com
M src/backend/utils/adt/arrayfuncs.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
pg_rewind: Refactor the abstraction to fetch from local/libpq source.
commit : 37d2ff38031262a1778bc76a9c55fff7afbcf275
author : Heikki Linnakangas <[email protected]>
date : Wed, 4 Nov 2020 11:21:18 +0200
committer: Heikki Linnakangas <[email protected]>
date : Wed, 4 Nov 2020 11:21:18 +0200
This makes the abstraction of a "source" server more clear, by introducing
a common abstract class, borrowing the object-oriented programming term,
that represents all the operations that can be done on the source server.
There are two implementations of it, one for fetching via libpq, and
another to fetch from a local directory. This adds some code, but makes it
easier to understand what's going on.
The copy_executeFileMap() and libpq_executeFileMap() functions contained
basically the same logic, just calling different functions to fetch the
source files. Refactor so that the common logic is in one place, in a new
function called perform_rewind().
Reviewed-by: Kyotaro Horiguchi, Soumyadeep Chakraborty
Discussion: https://www.postgresql.org/message-id/0c5b3783-af52-3ee5-f8fa-6e794061f70d%40iki.fi
M src/bin/pg_rewind/Makefile
D src/bin/pg_rewind/copy_fetch.c
D src/bin/pg_rewind/fetch.c
D src/bin/pg_rewind/fetch.h
M src/bin/pg_rewind/file_ops.c
M src/bin/pg_rewind/file_ops.h
R065 src/bin/pg_rewind/libpq_fetch.c src/bin/pg_rewind/libpq_source.c
A src/bin/pg_rewind/local_source.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/pg_rewind.h
A src/bin/pg_rewind/rewind_source.h
M src/tools/pgindent/typedefs.list
pg_rewind: Replace the hybrid list+array data structure with simplehash.
commit : f81e97d0475cd4bc597adc23b665bd84fbf79a0d
author : Heikki Linnakangas <[email protected]>
date : Wed, 4 Nov 2020 11:21:14 +0200
committer: Heikki Linnakangas <[email protected]>
date : Wed, 4 Nov 2020 11:21:14 +0200
Now that simplehash can be used in frontend code, let's make use of it.
Reviewed-by: Kyotaro Horiguchi, Soumyadeep Chakraborty
Discussion: https://www.postgresql.org/message-id/0c5b3783-af52-3ee5-f8fa-6e794061f70d%40iki.fi
M src/bin/pg_rewind/copy_fetch.c
M src/bin/pg_rewind/fetch.c
M src/bin/pg_rewind/fetch.h
M src/bin/pg_rewind/filemap.c
M src/bin/pg_rewind/filemap.h
M src/bin/pg_rewind/libpq_fetch.c
M src/bin/pg_rewind/pg_rewind.c
Refactor pg_rewind for more clear decision making.
commit : eb00f1d4bf96bdba236bcc089f3ae94db9b7c603
author : Heikki Linnakangas <[email protected]>
date : Wed, 4 Nov 2020 11:21:09 +0200
committer: Heikki Linnakangas <[email protected]>
date : Wed, 4 Nov 2020 11:21:09 +0200
Deciding what to do with each file is now a separate step after all the
necessary information has been gathered. It is more clear that way.
Previously, the decision-making was divided between process_source_file()
and process_target_file(), and it was a bit hard to piece together what
the overall rules were.
Reviewed-by: Kyotaro Horiguchi, Soumyadeep Chakraborty
Discussion: https://www.postgresql.org/message-id/0c5b3783-af52-3ee5-f8fa-6e794061f70d%40iki.fi
M src/bin/pg_rewind/copy_fetch.c
M src/bin/pg_rewind/file_ops.c
M src/bin/pg_rewind/filemap.c
M src/bin/pg_rewind/filemap.h
M src/bin/pg_rewind/libpq_fetch.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
pg_rewind: Move syncTargetDirectory() to file_ops.c
commit : ffb4e27e9c5ea87f9fecb7036dfc7cc1f38169b6
author : Heikki Linnakangas <[email protected]>
date : Wed, 4 Nov 2020 10:38:39 +0200
committer: Heikki Linnakangas <[email protected]>
date : Wed, 4 Nov 2020 10:38:39 +0200
For consistency. All the other low-level functions that operate on the
target directory are in file_ops.c.
Reviewed-by: Michael Paquier
Discussion: https://www.postgresql.org/message-id/0c5b3783-af52-3ee5-f8fa-6e794061f70d%40iki.fi
M src/bin/pg_rewind/file_ops.c
M src/bin/pg_rewind/file_ops.h
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/pg_rewind.h
Get rid of the dedicated latch for signaling the startup process.
commit : ac22929a2613e122708bd0172508ac863c51c1cc
author : Fujii Masao <[email protected]>
date : Wed, 4 Nov 2020 16:41:29 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 4 Nov 2020 16:41:29 +0900
This commit gets rid of the dedicated latch for signaling the startup
process in favor of using its procLatch, since that comports better
with possible generic signal handlers using that latch.
Commit 1e53fe0e70 changed background processes so that they use standard
SIGHUP handler. Like that, this commit also makes the startup process use
standard SIGHUP handler to simplify the code.
Author: Fujii Masao
Reviewed-by: Bharath Rupireddy, Michael Paquier
Discussion: https://postgr.es/m/CALj2ACXPorUqePswDtOeM_s82v9RW32E1fYmOPZ5NuE+TWKj_A@mail.gmail.com
M src/backend/access/transam/xlog.c
M src/backend/postmaster/startup.c
Use standard SIGHUP handler in syslogger.
commit : 02d332297f9bfe63476d53a439cc8f3b05bb5e67
author : Fujii Masao <[email protected]>
date : Wed, 4 Nov 2020 14:48:02 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 4 Nov 2020 14:48:02 +0900
Commit 1e53fe0e70 changed background processes so that they use
standard SIGHUP handler. Like that, this commit makes syslogger use
standard SIGHUP handler to simplify the code.
Author: Bharath Rupireddy
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CALj2ACXPorUqePswDtOeM_s82v9RW32E1fYmOPZ5NuE+TWKj_A@mail.gmail.com
M src/backend/postmaster/syslogger.c
Tolerate version lookup failure for old style Windows locale names.
commit : 9f12a3b95dd56c897f1aa3d756d8fb419e84a187
author : Thomas Munro <[email protected]>
date : Wed, 4 Nov 2020 14:58:34 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 4 Nov 2020 14:58:34 +1300
Accept that we can't get versions for such locale names for now. Users
will need to specify the newer language tag format to enable the
collation versioning feature. It's not clear that we can do automatic
conversion from the old style to the new style reliably enough for this
purpose.
Unfortunately, this means that collation versioning probably won't work
for the default collation unless you provide something like en-US at
initdb or CREATE DATABASE time (though, for reasons not yet understood,
it does seem to work on some systems). It'd be nice to find a better
solution, or document this quirk if we settle on it, but this should
unbreak the 3 failing build farm animals in the meantime.
Reviewed-by: David Rowley <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
M src/backend/utils/adt/pg_locale.c
Revert pg_relation_check_pages()
commit : e152506adef4bc503ea7b8ebb4fedc0b8eebda81
author : Michael Paquier <[email protected]>
date : Wed, 4 Nov 2020 10:21:46 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 4 Nov 2020 10:21:46 +0900
This reverts the following set of commits, following complaints about
the lack of portability of the central part of the code in bufmgr.c as
well as the use of partition mapping locks during page reads:
c780a7a9
f2b88396
b787d4ce
ce7f772c
60a51c6b
Per discussion with Andres Freund, Robert Haas and myself.
Bump catalog version.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/catalog/system_views.sql
M src/backend/storage/buffer/bufmgr.c
M src/backend/utils/adt/Makefile
D src/backend/utils/adt/pagefuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/storage/bufmgr.h
D src/test/recovery/t/022_page_check.pl
D src/test/regress/expected/pagefuncs.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
D src/test/regress/sql/pagefuncs.sql
M src/tools/pgindent/typedefs.list
Use INT64_FORMAT to print int64 variables in sort debug
commit : 90851d1d26f54ccb4d7b1bc49449138113d6ec83
author : Tomas Vondra <[email protected]>
date : Tue, 3 Nov 2020 20:43:12 +0100
committer: Tomas Vondra <[email protected]>
date : Tue, 3 Nov 2020 20:43:12 +0100
Commit 6ee3b5fb99 cleaned up most of the long/int64 confusion related to
incremental sort, but the sort debug messages were still using %ld for
int64 variables. So fix that.
Author: Haiying Tang
Backpatch-through: 13, where the incremental sort code was added
Discussion: https://postgr.es/m/4250be9d350c4992abb722a76e288aef%40G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/executor/nodeIncrementalSort.c
Fix get_useful_pathkeys_for_relation for volatile expressions
commit : ebb7ae839d033d0f279670e249f54646a08b8c48
author : Tomas Vondra <[email protected]>
date : Tue, 3 Nov 2020 20:07:23 +0100
committer: Tomas Vondra <[email protected]>
date : Tue, 3 Nov 2020 20:07:23 +0100
When considering Incremental Sort below a Gather Merge, we need to be
a bit more careful when matching pathkeys to EC members. It's not enough
to find a member whose Vars are all in the current relation's target;
volatile expressions in particular need to be contained in the target,
otherwise it's too early to use the pathkey.
Reported-by: Jaime Casanova
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13, where the incremental sort code was added
Discussion: https://postgr.es/m/CAJGNTeNaxpXgBVcRhJX%2B2vSbq%2BF2kJqGBcvompmpvXb7pq%2BoFA%40mail.gmail.com
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/equivclass.c
M src/include/optimizer/paths.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Guard against core dump from uninitialized subplan.
commit : 92f87182f2c617fd420832972b6d0ae4527301c8
author : Tom Lane <[email protected]>
date : Tue, 3 Nov 2020 16:16:36 -0500
committer: Tom Lane <[email protected]>
date : Tue, 3 Nov 2020 16:16:36 -0500
If the planner erroneously puts a non-parallel-safe SubPlan into
a parallelized portion of the query tree, nodeSubplan.c will fail
in the worker processes because it finds a null in es_subplanstates,
which it's unable to cope with. It seems worth a test-and-elog to
make that an error case rather than a core dump case.
This probably should have been included in commit 16ebab688, which
was responsible for allowing nulls to appear in es_subplanstates
to begin with. So, back-patch to v10 where that came in.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/nodeSubplan.c
Improve error messages around REPLICATION and BYPASSRLS properties.
commit : 17fb60387ce3fdc2bbb13d9b67bed0e4da77e173
author : Tom Lane <[email protected]>
date : Tue, 3 Nov 2020 15:49:05 -0500
committer: Tom Lane <[email protected]>
date : Tue, 3 Nov 2020 15:49:05 -0500
Clarify wording as per suggestion from Wolfgang Walther.
No back-patch; this doesn't seem worth thrashing translatable
strings in the back branches.
Tom Lane and Stephen Frost
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/user.c
Allow users with BYPASSRLS to alter their own passwords.
commit : d907bd0543aa63e59653d7345840bed0f8b3a83b
author : Tom Lane <[email protected]>
date : Tue, 3 Nov 2020 15:41:32 -0500
committer: Tom Lane <[email protected]>
date : Tue, 3 Nov 2020 15:41:32 -0500
The intention in commit 491c029db was to require superuserness to
change the BYPASSRLS property, but the actual effect of the coding
in AlterRole() was to require superuserness to change anything at all
about a BYPASSRLS role. Other properties of a BYPASSRLS role should
be changeable under the same rules as for a normal role, though.
Fix that, and also take care of some documentation omissions related
to BYPASSRLS and REPLICATION role properties.
Tom Lane and Stephen Frost, per bug report from Wolfgang Walther.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/alter_role.sgml
M doc/src/sgml/ref/create_role.sgml
M src/backend/commands/user.c
Disallow ALTER TABLE ONLY / DROP EXPRESSION
commit : bf797a8d9768239f5e3204b013044274b2c7c24a
author : Peter Eisentraut <[email protected]>
date : Tue, 3 Nov 2020 15:14:50 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 3 Nov 2020 15:14:50 +0100
The current implementation cannot handle this correctly, so just
forbid it for now.
GENERATED clauses must be attached to the column definition and cannot
be added later like DEFAULT, so if a child table has a generation
expression that the parent does not have, the child column will
necessarily be an attlocal column. So to implement ALTER TABLE ONLY /
DROP EXPRESSION, we'd need extra code to update attislocal of the
direct child tables, somewhat similar to how DROP COLUMN does it, so
that the resulting state can be properly dumped and restored.
Discussion: https://www.postgresql.org/message-id/flat/15830.1575468847%40sss.pgh.pa.us
M src/backend/commands/tablecmds.c
M src/test/regress/expected/generated.out
M src/test/regress/sql/generated.sql
Remove deprecated containment operators for built-in types
commit : 2f70fdb0644c32c4154236c2b5c241bec92eac5e
author : Peter Eisentraut <[email protected]>
date : Tue, 3 Nov 2020 10:32:20 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 3 Nov 2020 10:32:20 +0100
Remove old containment operators @ and ~ for built-in geometry data
types. These have been deprecated; use <@ and @> instead.
(Some contrib modules still contain the same deprecated operators.
That will be dealt with separately.)
Author: Justin Pryzby <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/func.sgml
M src/include/catalog/catversion.h
M src/include/catalog/pg_amop.dat
M src/include/catalog/pg_operator.dat
M src/test/regress/expected/create_am.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/sql/create_am.sql
Use the non-deprecated TG_TABLE_MAME in test trigger
commit : 44a184cb686866b10d63695db344195c239f9374
author : Magnus Hagander <[email protected]>
date : Tue, 3 Nov 2020 10:19:55 +0100
committer: Magnus Hagander <[email protected]>
date : Tue, 3 Nov 2020 10:19:55 +0100
Commit 3a9ae3d2068 (back in 2006) deprecated TG_RELNAME
in favor of TG_TABLE_NAME, but the existing usage in test
cases has remained till today. Change to use TG_TABLE_NAME
instead (TG_RELNAME is still covered by a test case).
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Improve error handling in backend OpenSSL implementation
commit : 13cfa02f77936895bff6ffabf1fde5d47fd1df07
author : Magnus Hagander <[email protected]>
date : Tue, 3 Nov 2020 09:55:51 +0100
committer: Magnus Hagander <[email protected]>
date : Tue, 3 Nov 2020 09:55:51 +0100
Commit d94c36a45ab introduced error handling to sslinfo to handle
OpenSSL errors gracefully. This ports this errorhandling to the
backend TLS implementation.
Author: Daniel Gustafsson <[email protected]>
M src/backend/libpq/be-secure-openssl.c
Use be_tls_* API for SSL information in sslinfo
commit : 5d1833f414973595411779b86e085fb05907a805
author : Magnus Hagander <[email protected]>
date : Tue, 3 Nov 2020 09:47:36 +0100
committer: Magnus Hagander <[email protected]>
date : Tue, 3 Nov 2020 09:47:36 +0100
sslinfo was passing the Port->ssl member directly to OpenSSL in order
to extract information regarding the connection. This breaks the API
provided by the backend TLS implementation, as well as duplicates code
for no benefit. Rewrite to make use of the backend API as much as
possible.
Author: Daniel Gustafsson <[email protected]>
M contrib/sslinfo/sslinfo.c
M doc/src/sgml/sslinfo.sgml
Remove use of deprecated containment operators in tests
commit : 3165426e54df02a6199c0a216546e5095e875a0a
author : Peter Eisentraut <[email protected]>
date : Tue, 3 Nov 2020 09:03:22 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 3 Nov 2020 09:03:22 +0100
Switch @ to <@ and ~ to @> in gist-related tests. The old operator
names have been deprecated and will eventually (possibly soon) be
removed.
Author: Justin Pryzby <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/test/regress/expected/create_am.out
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_am.sql
M src/test/regress/sql/create_index.sql
Fix typos.
commit : 8c2d8f6cc4848cf9276dff445bb0f2f664083eca
author : Amit Kapila <[email protected]>
date : Tue, 3 Nov 2020 08:38:27 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 3 Nov 2020 08:38:27 +0530
Author: Hou Zhijie
Discussion: https://postgr.es/m/855a9421839d402b8b351d273c89a8f8@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/storage/ipc/procarray.c
M src/include/access/xlogreader.h
Fix unportable use of getnameinfo() in pg_hba_file_rules view.
commit : 0a4b34031279d938c2e59df8df7159d6c11e39b5
author : Tom Lane <[email protected]>
date : Mon, 2 Nov 2020 21:11:50 -0500
committer: Tom Lane <[email protected]>
date : Mon, 2 Nov 2020 21:11:50 -0500
fill_hba_line() thought it could get away with passing sizeof(struct
sockaddr_storage) rather than the actual addrlen previously returned
by getaddrinfo(). While that appears to work on many platforms,
it does not work on FreeBSD 11: you get back a failure, which leads
to the view showing NULL for the address and netmask columns in all
rows. The POSIX spec for getnameinfo() is pretty clearly on
FreeBSD's side here: you should pass the actual address length.
So it seems plausible that there are other platforms where this
coding also fails, and we just hadn't noticed.
Also, IMO the fact that getnameinfo() failure leads to a NULL output
is pretty bogus in itself. Our pg_getnameinfo_all() wrapper is
careful to emit "???" on failure, and we should use that in such
cases. NULL should only be emitted in rows that don't have IP
addresses.
Per bug #16695 from Peter Vandivier. Back-patch to v10 where this
code was added.
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/hba.c
M src/include/libpq/hba.h
Remove special checks for pg_rewrite.ev_qual and ev_action being NULL.
commit : e1339bfc7a2fd4629e1c3f8f919ddd05b4745e13
author : Tom Lane <[email protected]>
date : Mon, 2 Nov 2020 14:34:34 -0500
committer: Tom Lane <[email protected]>
date : Mon, 2 Nov 2020 14:34:34 -0500
make_ruledef() and make_viewdef() were coded to cope with possible
null-ness of these columns, but they've been marked BKI_FORCE_NOT_NULL
for some time. So there's not really any need to do more than what
we do for the other columns of pg_rewrite, i.e. just Assert that
we got non-null results.
(There is a school of thought that says Asserts aren't the thing
to do to check for corrupt data, but surely here is not the place
to start if we want such a policy.)
Also, remove long-dead-if-indeed-it-ever-wasn't-dead handling of
an empty actions list in make_ruledef(). That's an error case
and should be treated as such. (DO INSTEAD NOTHING is represented
by a CMD_NOTHING Query, not an empty list; cf transformRuleStmt.)
Kyotaro Horiguchi, some changes by me
Discussion: https://postgr.es/m/CAEudQApoA=tMTic6xEPYP_hsNZ8XtToVThK_0x7D_aFQYowq3w@mail.gmail.com
M src/backend/utils/adt/ruleutils.c
Rethink the generation rule for fmgroids.h macros.
commit : 8e1f37c07aafd4bb7aa6e1e1982010af11f8b5c7
author : Tom Lane <[email protected]>
date : Mon, 2 Nov 2020 11:57:28 -0500
committer: Tom Lane <[email protected]>
date : Mon, 2 Nov 2020 11:57:28 -0500
Traditionally, the names of fmgroids.h macros for pg_proc OIDs
have been constructed from the prosrc field. But sometimes the
same C function underlies multiple pg_proc entries, forcing us
to make an arbitrary choice of which OID to reference; the other
entries are then not namable via fmgroids.h. Moreover, we could
not have macros at all for pg_proc entries that aren't for
C-coded functions.
Instead, use the proname field, and append the proargtypes field
(replacing inter-argument spaces with underscores) if proname is
not unique. Special-casing unique entries such as F_OIDEQ removes
the need to change a lot of code. Indeed, I can only find two
places in the tree that need to be adjusted; while this changes
quite a few existing entries in fmgroids.h, few of them are
referenced from C code.
With this patch, all entries in pg_proc.dat have macros in fmgroids.h.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/util/clauses.c
M src/backend/utils/Gen_fmgrtab.pl
M src/backend/utils/adt/ruleutils.c
Second thoughts on TOAST decompression.
commit : fd2997565c6f66837440dd57f5e52b56aa964d14
author : Tom Lane <[email protected]>
date : Mon, 2 Nov 2020 11:25:18 -0500
committer: Tom Lane <[email protected]>
date : Mon, 2 Nov 2020 11:25:18 -0500
On detecting a corrupted match tag, pglz_decompress() should just
summarily return -1. Breaking out of the loop, as I did in dfc797730,
doesn't quite guarantee that will happen. Also, we can use
unlikely() on that check, just in case it helps.
Backpatch to v13, like the previous patch.
M src/common/pg_lzcompress.c
Use PG_GETARG_TRANSACTIONID where appropriate
commit : dd26a0ad760b11237d7ea9cda8ccccc1826c2c64
author : Peter Eisentraut <[email protected]>
date : Mon, 2 Nov 2020 16:48:22 +0100
committer: Peter Eisentraut <[email protected]>
date : Mon, 2 Nov 2020 16:48:22 +0100
Some places were using PG_GETARG_UINT32 where PG_GETARG_TRANSACTIONID
would be more appropriate. (Of course, they are the same internally,
so there is no externally visible effect.) To do that, export
PG_GETARG_TRANSACTIONID outside of xid.c. We also export
PG_RETURN_TRANSACTIONID for symmetry, even though there are currently
no external users.
Author: Ashutosh Bapat <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/multixact.c
M src/backend/utils/adt/xid.c
M src/include/fmgr.h
Clarify temporary table name shadowing in CREATE TABLE docs
commit : 5b3dca096f8e504b73812f68716fb68dd1176d6d
author : Magnus Hagander <[email protected]>
date : Mon, 2 Nov 2020 15:00:24 +0100
committer: Magnus Hagander <[email protected]>
date : Mon, 2 Nov 2020 15:00:24 +0100
Author: David Johnston
M doc/src/sgml/ref/create_table.sgml
Track collation versions for indexes.
commit : 257836a75585934cc05ed7a80bccf8190d41e056
author : Thomas Munro <[email protected]>
date : Mon, 2 Nov 2020 19:50:45 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 2 Nov 2020 19:50:45 +1300
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. When accessing the index later, warn
that the index may be corrupted if the current version doesn't match.
Thanks to Douglas Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for very helpful
discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]> (earlier versions)
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/charset.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/alter_index.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/reindex.sgml
M src/backend/catalog/dependency.c
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/pg_constraint.c
M src/backend/catalog/pg_depend.c
M src/backend/catalog/pg_type.c
M src/backend/commands/collationcmds.c
M src/backend/commands/tablecmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/optimizer/util/plancat.c
M src/backend/parser/gram.y
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/pg_upgrade_support.c
M src/backend/utils/cache/relcache.c
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_upgrade/dump.c
M src/bin/pg_upgrade/option.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/psql/tab-complete.c
M src/include/catalog/catversion.h
M src/include/catalog/dependency.h
M src/include/catalog/index.h
M src/include/catalog/pg_depend.h
M src/include/catalog/pg_type.h
M src/include/nodes/parsenodes.h
M src/include/utils/pg_locale.h
M src/include/utils/rel.h
M src/test/Makefile
M src/test/locale/.gitignore
M src/test/locale/Makefile
A src/test/locale/t/001_index.pl
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/expected/create_index.out
M src/test/regress/sql/collate.icu.utf8.sql
M src/tools/pgindent/typedefs.list
Add pg_depend.refobjversion.
commit : cd6f479e79f3a33ef7a919c6b6c0c498c790f154
author : Thomas Munro <[email protected]>
date : Mon, 2 Nov 2020 19:40:49 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 2 Nov 2020 19:40:49 +1300
Provide a place for the version of referenced database objects to be
recorded. A follow-up commit will use this to record dependencies on
collation versions for indexes, but similar ideas for other kinds of
objects have also been mooted.
Author: Thomas Munro <[email protected]>
Reviewed-by: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/dependency.c
M src/backend/catalog/pg_depend.c
M src/include/catalog/catversion.h
M src/include/catalog/dependency.h
M src/include/catalog/pg_depend.h
M src/include/catalog/toasting.h
M src/test/regress/expected/misc_sanity.out
Remove pg_collation.collversion.
commit : 7d1297df0830725d4434ba7dbf71a9e8221ad49e
author : Thomas Munro <[email protected]>
date : Mon, 2 Nov 2020 19:36:09 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 2 Nov 2020 19:36:09 +1300
This model couldn't be extended to cover the default collation, and
didn't have any information about the affected database objects when the
version changed. Remove, in preparation for a follow-up commit that
will add a new mechanism.
Author: Thomas Munro <[email protected]>
Reviewed-by: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/alter_collation.sgml
M doc/src/sgml/ref/create_collation.sgml
M src/backend/catalog/pg_collation.c
M src/backend/commands/collationcmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/parser/gram.y
M src/backend/tcop/utility.c
M src/backend/utils/adt/pg_locale.c
M src/bin/pg_dump/pg_dump.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_collation.dat
M src/include/catalog/pg_collation.h
M src/include/catalog/toasting.h
M src/include/commands/collationcmds.h
M src/include/nodes/parsenodes.h
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/expected/collate.linux.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
M src/test/regress/sql/collate.linux.utf8.sql
doc: Mention UNION/ORDER BY etc. keywords in section headers.
commit : 8ef2a5afdf8ec9e4c8b28a7042c9508eb6161671
author : Heikki Linnakangas <[email protected]>
date : Mon, 2 Nov 2020 12:51:46 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 2 Nov 2020 12:51:46 +0200
Most of the section and sub-section headers in the Queries chapter have
the keywords literally stated, but neither "Sorting Rows" nor "Combining
Rows" did. There's no rule that they must be, but it seems like a good
practice. The keywords will ring a bell to anyone with with even a little
bit of SQL experience.
David G. Johnston, per suggestion by [email protected]
Discussion: https://www.postgresql.org/message-id/159981394174.31338.7014519396749859167%40wrigleys.postgresql.org
M doc/src/sgml/queries.sgml
Fix unstable partition_prune regression tests
commit : 90d8f1b1826ce076a502a43fe7c88423b46c6349
author : David Rowley <[email protected]>
date : Mon, 2 Nov 2020 19:59:02 +1300
committer: David Rowley <[email protected]>
date : Mon, 2 Nov 2020 19:59:02 +1300
This was broken recently by a929e17e5. I'd failed to remember that
parallel tests should have their EXPLAIN output run through the
explain_parallel_append function so that the output is stable when
parallel workers fail to start.
fairywren was first to notice.
Reported-by: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Fix some grammar and typos in comments and docs
commit : 8a15e735be00f156a7227741c0ce88702e6de099
author : Michael Paquier <[email protected]>
date : Mon, 2 Nov 2020 15:14:41 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 2 Nov 2020 15:14:41 +0900
The documentation fixes are backpatched down to where they apply.
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M contrib/amcheck/verify_heapam.c
M contrib/pgcrypto/pgp-compress.c
M doc/src/sgml/auto-explain.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/pg_rewind.sgml
M src/backend/access/heap/pruneheap.c
M src/backend/catalog/namespace.c
M src/backend/catalog/pg_namespace.c
M src/backend/commands/copy.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execExpr.c
M src/backend/executor/nodeIncrementalSort.c
M src/backend/executor/nodeLimit.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/partitioning/partbounds.c
M src/backend/postmaster/interrupt.c
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/utils/adt/varlena.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pgbench/pgbench.c
M src/interfaces/libpq/fe-connect.c
M src/test/modules/dummy_index_am/dummy_index_am.c
Use Enum for top level logical replication message types.
commit : 644f0d7cc9c2cb270746f2024c706554e0fbec82
author : Amit Kapila <[email protected]>
date : Mon, 2 Nov 2020 08:18:18 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 2 Nov 2020 08:18:18 +0530
Logical replication protocol uses a single byte character to identify a
message type in logical replication protocol. The code uses string
literals for the same. Use Enum so that
1. All the string literals used can be found at a single place. This
makes it easy to add more types without the risk of conflicts.
2. It's easy to locate the code handling a given message type.
3. When used with switch statements, it is easy to identify the missing
cases using -Wswitch.
Author: Ashutosh Bapat
Reviewed-by: Kyotaro Horiguchi, Andres Freund, Peter Smith and Amit Kapila
Discussion: https://postgr.es/m/CAExHW5uPzQ7L0oAd_ENyvaiYMOPgkrAoJpE+ZY5-obdcVT6NPg@mail.gmail.com
M src/backend/replication/logical/proto.c
M src/backend/replication/logical/worker.c
M src/include/replication/logicalproto.h
Allow run-time pruning on nested Append/MergeAppend nodes
commit : a929e17e5a8c9b751b66002c8a89fdebdacfe194
author : David Rowley <[email protected]>
date : Mon, 2 Nov 2020 13:46:56 +1300
committer: David Rowley <[email protected]>
date : Mon, 2 Nov 2020 13:46:56 +1300
Previously we only tagged on the required information to allow the
executor to perform run-time partition pruning for Append/MergeAppend
nodes belonging to base relations. It was thought that nested
Append/MergeAppend nodes were just about always pulled up into the
top-level Append/MergeAppend and that making the run-time pruning info for
any sub Append/MergeAppend nodes was a waste of time. However, that was
likely badly thought through.
Some examples of cases we're unable to pullup nested Append/MergeAppends
are: 1) Parallel Append nodes with a mix of parallel and non-parallel
paths into a Parallel Append. 2) When planning an ordered Append scan a
sub-partition which is unordered may require a nested MergeAppend path to
ensure sub-partitions don't mix up the order of tuples being fed into the
top-level Append.
Unfortunately, it was not just as simple as removing the lines in
createplan.c which were purposefully not building the run-time pruning
info for anything but RELOPT_BASEREL relations. The code in
add_paths_to_append_rel() was far too sloppy about which partitioned_rels
it included for the Append/MergeAppend paths. The original code there
would always assume accumulate_append_subpath() would pull each sub-Append
and sub-MergeAppend path into the top-level path. While it does not
appear that there were any actual bugs caused by having the additional
partitioned table RT indexes recorded, what it did mean is that later in
planning, when we built the run-time pruning info that we wasted effort
and built PartitionedRelPruneInfos for partitioned tables that we had no
subpaths for the executor to run-time prune.
Here we tighten that up so that partitioned_rels only ever contains the RT
index for partitioned tables which actually have subpaths in the given
Append/MergeAppend. We can now Assert that every PartitionedRelPruneInfo
has a non-empty present_parts. That should allow us to catch any weird
corner cases that have been missed.
In passing, it seems there is no longer a good reason to have the
AppendPath and MergeAppendPath's partitioned_rel fields a List of IntList.
We can simply have a List of Relids instead. This is more compact in
memory and faster to add new members to. We still know which is the root
level partition as these always have a lower relid than their children.
Previously this field was used for more things, but run-time partition
pruning now remains the only user of it and it has no need for a List of
IntLists.
Here we also get rid of the RelOptInfo partitioned_child_rels field. This
is what was previously used to (sometimes incorrectly) set the
Append/MergeAppend path's partitioned_rels field. That was the only usage
of that field, so we can happily just remove it.
I also couldn't resist changing some nearby code to make use of the newly
added for_each_from macro so we can skip the first element in the list
without checking if the current item was the first one on each
iteration.
A bug report from Andreas Kretschmer prompted all this work, however,
after some consideration, I'm not personally classing this as a bug fix.
So no backpatch. In Andreas' test case, it just wasn't that clear that
there was a nested Append since the top-level Append just had a single
sub-path which was pulled up a level, per 8edd0e794.
Author: David Rowley
Reviewed-by: Amit Langote
Discussion: https://postgr.es/m/flat/CAApHDvqSchs%2BubdybcfFaSPB%2B%2BEA7kqMaoqajtP0GtZvzOOR3g%40mail.gmail.com
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/relnode.c
M src/backend/partitioning/partprune.c
M src/include/nodes/pathnodes.h
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Fix two issues in TOAST decompression.
commit : dfc797730fc7a07c0e6bd636ad1a564aecab3161
author : Tom Lane <[email protected]>
date : Sun, 1 Nov 2020 18:38:42 -0500
committer: Tom Lane <[email protected]>
date : Sun, 1 Nov 2020 18:38:42 -0500
pglz_maximum_compressed_size() potentially underestimated the amount
of compressed data required to produce N bytes of decompressed data;
this is a fault in commit 11a078cf8.
Separately from that, pglz_decompress() failed to protect itself
against corrupt compressed data, particularly off == 0 in a match
tag. Commit c60e520f6 turned such a situation into an infinite loop,
where before it'd just have resulted in garbage output.
The combination of these two bugs seems like it may explain bug #16694
from Tom Vijlbrief, though it's impossible to be quite sure without
direct inspection of the failing session. (One needs to assume that
the pglz_maximum_compressed_size() bug caused us to fail to fetch the
second byte of a match tag, and what happened to be there instead was
a zero. The reported infinite loop is hard to explain without off == 0,
though.)
Aside from fixing the bugs, rewrite associated comments for more
clarity.
Back-patch to v13 where both these commits landed.
Discussion: https://postgr.es/m/[email protected]
M src/common/pg_lzcompress.c
Avoid null pointer dereference if error result lacks SQLSTATE.
commit : 7f4235032f0d75ea1ad29b192d57fee3d8fe533e
author : Tom Lane <[email protected]>
date : Sun, 1 Nov 2020 11:26:16 -0500
committer: Tom Lane <[email protected]>
date : Sun, 1 Nov 2020 11:26:16 -0500
Although error results received from the backend should always have
a SQLSTATE field, ones generated by libpq won't, making this code
vulnerable to a crash after, say, untimely loss of connection.
Noted by Coverity.
Oversight in commit 403a3d91c. Back-patch to 9.5, as that was.
M src/bin/pg_dump/pg_backup_db.c
Preserve index data in pg_statistic across REINDEX CONCURRENTLY
commit : b17ff07aa3eb142d2cde2ea00e4a4e8f63686f96
author : Michael Paquier <[email protected]>
date : Sun, 1 Nov 2020 21:22:07 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 1 Nov 2020 21:22:07 +0900
Statistics associated to an index got lost after running REINDEX
CONCURRENTLY, while the non-concurrent case preserves these correctly.
The concurrent and non-concurrent operations need to be consistent for
the end-user, and missing statistics would force to wait for a new
analyze to happen, which could take some time depending on the activity
of the existing autovacuum workers. This issue is fixed by copying any
existing entries in pg_statistic associated to the old index to the new
one. Note that this copy is already done with the data of the index in
the stats collector.
Reported-by: Fabrízio de Royes Mello
Author: Michael Paquier, Fabrízio de Royes Mello
Reviewed-by: Justin Pryzby
Discussion: https://postgr.es/m/CAFcNs+qpFPmiHd1oTXvcPdvAHicJDA9qBUSujgAhUMJyUMb+SA@mail.gmail.com
Backpatch-through: 12
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/include/catalog/heap.h
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Add error code for encryption failure in pgcrypto
commit : aecaa04418f39c32adb3dbf91c4aa7f6e175f01c
author : Michael Paquier <[email protected]>
date : Sun, 1 Nov 2020 19:22:59 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 1 Nov 2020 19:22:59 +0900
PXE_DECRYPT_FAILED exists already for decryption errors, and an
equivalent for encryption did not exist. There is one code path that
deals with such failures for OpenSSL but it used PXE_ERR_GENERIC, which
was inconsistent. This switches this code path to use the new error
PXE_ENCRYPT_FAILED instead of PXE_ERR_GENERIC, making the code used for
encryption more consistent with the decryption.
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M contrib/pgcrypto/openssl.c
M contrib/pgcrypto/px.c
M contrib/pgcrypto/px.h
Set debug_query_string in worker_spi.
commit : d2246cde825e4e1a85408390c66367b85b51a233
author : Noah Misch <[email protected]>
date : Sat, 31 Oct 2020 08:47:02 -0700
committer: Noah Misch <[email protected]>
date : Sat, 31 Oct 2020 08:47:02 -0700
This makes elog.c emit the string, which is good practice for a
background worker that executes SQL strings.
Reviewed by Tom Lane.
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/worker_spi/worker_spi.c
Reproduce debug_query_string==NULL on parallel workers.
commit : f90e80b9138355a51d2d5b5b63e1f89c4ba53325
author : Noah Misch <[email protected]>
date : Sat, 31 Oct 2020 08:43:28 -0700
committer: Noah Misch <[email protected]>
date : Sat, 31 Oct 2020 08:43:28 -0700
Certain background workers initiate parallel queries while
debug_query_string==NULL, at which point they attempted strlen(NULL) and
died to SIGSEGV. Older debug_query_string observers allow NULL, so do
likewise in these newer ones. Back-patch to v11, where commit
7de4a1bcc56f494acbd0d6e70781df877dc8ecb5 introduced the first of these.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtsort.c
Fix assertion failure in check_new_partition_bound().
commit : 970c05057593c2f5919a69b43fd917c4fa86f51c
author : Tom Lane <[email protected]>
date : Fri, 30 Oct 2020 17:00:59 -0400
committer: Tom Lane <[email protected]>
date : Fri, 30 Oct 2020 17:00:59 -0400
Commit 6b2c4e59d was overly confident about not being able to see
a negative cmpval result from partition_range_bsearch(). Adjust
the code to cope with that.
Report and patch by Amul Sul; some additional cosmetic changes by me
Discussion: https://postgr.es/m/CAAJ_b97WCO=EyVA7fKzc86kKfojHXLU04_zs7-7+yVzm=-1QkQ@mail.gmail.com
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/create_table.out
M src/test/regress/sql/create_table.sql
Fix missing validation for the new GiST sortsupport functions.
commit : 6f0bc5e1daf09686c526aa161da5336f7c94f4eb
author : Heikki Linnakangas <[email protected]>
date : Fri, 30 Oct 2020 19:30:19 +0200
committer: Heikki Linnakangas <[email protected]>
date : Fri, 30 Oct 2020 19:30:19 +0200
Because of this, if you tried to create an operator family with the new
sortsupport function, you got an error:
ERROR: support function number 11 is invalid for access method gist
We missed this in commit 16fa9b2b30 that added the sortsupport function,
because it only added sortsupport to a built-in operator family.
Author: Andrey Borodin
Discussion: https://www.postgresql.org/message-id/3520A18A-5C38-4697-A2E3-F3BDE3496CD5%40yandex-team.ru
M src/backend/access/gist/gistvalidate.c
Doc: clarify description for pg_constraint.convalidated.
commit : b401fa206d446f224ec4760f21e0a351816c97b3
author : Tom Lane <[email protected]>
date : Fri, 30 Oct 2020 10:38:49 -0400
committer: Tom Lane <[email protected]>
date : Fri, 30 Oct 2020 10:38:49 -0400
Jimmy Angelakos
Discussion: https://postgr.es/m/CABgVKCW_zPnvFXn24FTF0299_yU6+1p6JRUc0xpiZFWEXH1_jg@mail.gmail.com
M doc/src/sgml/catalogs.sgml
Stabilize timetz test across DST transitions.
commit : 4a071afbd056282746a5bc9362e87f579a56402d
author : Tom Lane <[email protected]>
date : Thu, 29 Oct 2020 15:28:14 -0400
committer: Tom Lane <[email protected]>
date : Thu, 29 Oct 2020 15:28:14 -0400
The timetz test cases I added in commit a9632830b were unintentionally
sensitive to whether or not DST is active in the PST8PDT time zone.
Thus, they'll start failing this coming weekend, as reported by
Bernhard M. Wiedemann in bug #16689. Fortunately, DST-awareness is
not significant to the purpose of these test cases, so we can just
force them all to PDT (DST hours) to preserve stability of the
results.
Back-patch to v10, as the prior patch was.
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/timetz.out
M src/test/regress/sql/timetz.sql
Don't use custom OID symbols in pg_type.dat, either.
commit : f90149e6285aaae6b48559afce1bd638ee26c33e
author : Tom Lane <[email protected]>
date : Thu, 29 Oct 2020 13:33:38 -0400
committer: Tom Lane <[email protected]>
date : Thu, 29 Oct 2020 13:33:38 -0400
On the same reasoning as in commit 36b931214, forbid using custom
oid_symbol macros in pg_type as well as pg_proc, so that we always
rely on the predictable macro names generated by genbki.pl.
We do continue to grant grandfather status to the names CASHOID and
LSNOID, although those are now considered deprecated aliases for the
preferred names MONEYOID and PG_LSNOID. This is because there's
likely to be client-side code using the old names, and this bout of
neatnik-ism doesn't quite seem worth breaking client code.
There might be a case for grandfathering EVTTRIGGEROID, too, since
externally-maintained PLs may reference that symbol. But renaming
such references to EVENT_TRIGGEROID doesn't seem like a particularly
heavy lift --- we make far more significant backend API changes in
every major release. For now I didn't add that, but we could
reconsider if there's pushback.
The other names changed here seem pretty unlikely to have any outside
uses. Again, we could add alias macros if there are complaints, but
for now I didn't.
As before, no need for a catversion bump.
John Naylor
Discussion: https://postgr.es/m/CAFBsxsHpCbjfoddNGpnnnY5pHwckWfiYkMYSF74PmP1su0+ZOw@mail.gmail.com
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/genbki.pl
M src/backend/commands/event_trigger.c
M src/backend/utils/misc/pg_controldata.c
M src/fe_utils/print.c
M src/include/catalog/pg_type.dat
M src/include/catalog/pg_type.h
M src/interfaces/ecpg/ecpglib/execute.c
M src/pl/plperl/plperl.c
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/pl_handler.c
M src/pl/plpython/plpy_procedure.c
M src/pl/tcl/pltcl.c
Fix wrong data table horizon computation during backend startup.
commit : 1c7675a7a4265064a2c8e1ed02b2c042c2521664
author : Andres Freund <[email protected]>
date : Wed, 28 Oct 2020 21:48:38 -0700
committer: Andres Freund <[email protected]>
date : Wed, 28 Oct 2020 21:48:38 -0700
When ComputeXidHorizons() was called before MyDatabaseOid is set,
e.g. because a dead row in a shared relation is encountered during
InitPostgres(), the horizon for normal tables was computed too
aggressively, ignoring all backends connected to a database.
During subsequent pruning in a data table the too aggressive horizon
could end up still being used, possibly leading to still needed tuples
being removed. Not good.
This is a bug in dc7420c2c92, which the test added in 94bc27b5768 made
visible, if run with force_parallel_mode set to regress. In that case
the bug is reliably triggered, because "pruning_query" is run in a
parallel worker and the start of that parallel worker is likely to
encounter a dead row in pg_database.
The fix is trivial: Compute a more pessimistic data table horizon if
MyDatabaseId is not yet known.
Author: Andres Freund
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/procarray.c
Track statistics for streaming of changes from ReorderBuffer.
commit : 8e90ec5580d5345fef31005d7cc2215ba2125070
author : Amit Kapila <[email protected]>
date : Thu, 29 Oct 2020 09:11:51 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 29 Oct 2020 09:11:51 +0530
This adds the statistics about transactions streamed to the decoding
output plugin from ReorderBuffer. Users can query the
pg_stat_replication_slots view to check these stats and call
pg_stat_reset_replication_slot to reset the stats of a particular slot.
Users can pass NULL in pg_stat_reset_replication_slot to reset stats of
all the slots.
Commit 9868167500 has added the basic infrastructure to capture the stats
of slot and this commit extends the statistics collector to track
additional information about slots.
Bump the catversion as we have added new columns in the catalog entry.
Author: Ajin Cherian and Amit Kapila
Reviewed-by: Sawada Masahiko and Dilip Kumar
Discussion: https://postgr.es/m/CAA4eK1+chpEomLzgSoky-D31qev19AmECNiEAietPQUGEFhtVA@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/slot.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/replication/reorderbuffer.h
M src/test/regress/expected/rules.out
Centralize horizon determination for temp tables, fixing bug due to skew.
commit : 94bc27b57680b4e757577e3f5b65dc32f96d33c1
author : Andres Freund <[email protected]>
date : Wed, 28 Oct 2020 17:53:41 -0700
committer: Andres Freund <[email protected]>
date : Wed, 28 Oct 2020 17:53:41 -0700
This fixes a bug in the edge case where, for a temp table, heap_page_prune()
can end up with a different horizon than heap_vacuum_rel(). Which can trigger
errors like "ERROR: cannot freeze committed xmax ...".
The bug was introduced due to interaction of a7212be8b9e "Set cutoff xmin more
aggressively when vacuuming a temporary table." with dc7420c2c92 "snapshot
scalability: Don't compute global horizons while building snapshots.".
The problem is caused by lazy_scan_heap() assuming that the only reason its
HeapTupleSatisfiesVacuum() call would return HEAPTUPLE_DEAD is if the tuple is
a HOT tuple, or if the tuple's inserting transaction has aborted since the
heap_page_prune() call. But after a7212be8b9e that was also possible in other
cases for temp tables, because heap_page_prune() uses a different visibility
test after dc7420c2c92.
The fix is fairly simple: Move the special case logic for temp tables from
vacuum_set_xid_limits() to the infrastructure introduced in dc7420c2c92. That
ensures that the horizon used for pruning is at least as aggressive as the one
used by lazy_scan_heap(). The concrete horizon used for temp tables is
slightly different than the logic in dc7420c2c92, but should always be as
aggressive as before (see comments).
A significant benefit to centralizing the logic procarray.c is that now the
more aggressive horizons for temp tables does not just apply to VACUUM but
also to e.g. HOT pruning and the nbtree killtuples logic.
Because isTopLevel is not needed by vacuum_set_xid_limits() anymore, I
undid the the related changes from a7212be8b9e.
This commit also adds an isolation test ensuring that the more aggressive
vacuuming and pruning of temp tables keeps working.
Debugged-By: Amit Kapila <[email protected]>
Debugged-By: Tom Lane <[email protected]>
Debugged-By: Ashutosh Sharma <[email protected]>
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/cluster.c
M src/backend/commands/vacuum.c
M src/backend/storage/ipc/procarray.c
M src/include/commands/cluster.h
M src/include/commands/vacuum.h
A src/test/isolation/expected/horizons.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/horizons.spec
Fix incorrect placement of pfree() in pg_relation_check_pages()
commit : 60a51c6b32960822d3987ea7d2816c65bdbcb314
author : Michael Paquier <[email protected]>
date : Thu, 29 Oct 2020 09:17:34 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 29 Oct 2020 09:17:34 +0900
This would cause the function to crash when more than one page is
considered as broken and reported in the SRF.
Reported-by: Noriyoshi Shinoda
Discussion: https://postgr.es/m/TU4PR8401MB11523D42C315AAF822E74275EE170@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM
M src/backend/utils/adt/pagefuncs.c
Doc: clean up pg_relation_check_pages() documentation.
commit : b787d4ce6d910080065025bcd5f968544997271f
author : Tom Lane <[email protected]>
date : Wed, 28 Oct 2020 17:03:05 -0400
committer: Tom Lane <[email protected]>
date : Wed, 28 Oct 2020 17:03:05 -0400
Commit f2b883969 did not get the memo about the new formatting
style for tables documenting built-in functions. I noticed because
of a PDF build warning about an overwidth table.
M doc/src/sgml/func.sgml
Doc: clean up verify_heapam() documentation.
commit : 4c49d8fc15eeb1dc69b0ddb2d986a1884a5d7f5f
author : Tom Lane <[email protected]>
date : Wed, 28 Oct 2020 16:31:40 -0400
committer: Tom Lane <[email protected]>
date : Wed, 28 Oct 2020 16:31:40 -0400
I started with the intention of just suppressing a PDF build warning
by removing the example output, but ended up doing more: correcting
factual errors in the function's signature, moving a bunch of
generalized handwaving into the "Using amcheck Effectively" section
which seemed a better place for it, and improving wording and markup
a little bit.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/amcheck.sgml
Use mode "r" for popen() in psql's evaluate_backtick().
commit : 66f8687a8ff867f656de81e367314604d29dbd59
author : Tom Lane <[email protected]>
date : Wed, 28 Oct 2020 14:35:53 -0400
committer: Tom Lane <[email protected]>
date : Wed, 28 Oct 2020 14:35:53 -0400
In almost all other places, we use plain "r" or "w" mode in popen()
calls (the exceptions being for COPY data). This one has been
overlooked (possibly because it's buried in a ".l" flex file?),
but it's using PG_BINARY_R.
Kensuke Okamura complained in bug #16688 that we fail to strip \r
when stripping the trailing newline from a backtick result string.
That's true enough, but we'd also fail to convert embedded \r\n
cleanly, which also seems undesirable. Fixing the popen() mode
seems like the best way to deal with this.
It's been like this for a long time, so back-patch to all supported
branches.
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/psqlscanslash.l
Calculate extraUpdatedCols in query rewriter, not parser.
commit : ad77039fad0f4128b0e4a05ddbf5dbc3ab5f3fa4
author : Tom Lane <[email protected]>
date : Wed, 28 Oct 2020 13:47:02 -0400
committer: Tom Lane <[email protected]>
date : Wed, 28 Oct 2020 13:47:02 -0400
It's unsafe to do this at parse time because addition of generated
columns to a table would not invalidate stored rules containing
UPDATEs on the table ... but there might now be dependent generated
columns that were not there when the rule was made. This also fixes
an oversight that rewriteTargetView failed to update extraUpdatedCols
when transforming an UPDATE on an updatable view. (Since the new
calculation is downstream of that, rewriteTargetView doesn't actually
need to do anything; but before, there was a demonstrable bug there.)
In v13 and HEAD, this leads to easily-visible bugs because (since
commit c6679e4fc) we won't recalculate generated columns that aren't
listed in extraUpdatedCols. In v12 this bitmap is mostly just used
for trigger-firing decisions, so you'd only notice a problem if a
trigger cared whether a generated column had been updated.
I'd complained about this back in May, but then forgot about it
until bug #16671 from Michael Paul Killian revived the issue.
Back-patch to v12 where this field was introduced. If existing
stored rules contain any extraUpdatedCols values, they'll be
ignored because the rewriter will overwrite them, so the bug will
be fixed even for existing rules. (But note that if someone were
to update to 13.1 or 12.5, store some rules with UPDATEs on tables
having generated columns, and then downgrade to a prior minor version,
they might observe issues similar to what this patch fixes. That
seems unlikely enough to not be worth going to a lot of effort to fix.)
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/plan/setrefs.c
M src/backend/parser/analyze.c
M src/backend/replication/logical/worker.c
M src/backend/rewrite/rewriteHandler.c
M src/include/nodes/parsenodes.h
M src/include/parser/analyze.h
M src/include/rewrite/rewriteHandler.h
M src/test/regress/expected/updatable_views.out
M src/test/regress/sql/updatable_views.sql
Don't use custom OID symbols in pg_proc.dat.
commit : 36b93121436cbbf357974144068c23bac75154fa
author : Tom Lane <[email protected]>
date : Wed, 28 Oct 2020 12:18:45 -0400
committer: Tom Lane <[email protected]>
date : Wed, 28 Oct 2020 12:18:45 -0400
We have a perfectly good convention for OID macros for built-in functions
already, so making custom symbols is just introducing unnecessary
deviation from the convention. Remove the one case that had snuck in,
and add an error check in genbki.pl to discourage future instances.
Although this touches pg_proc.dat, there's no need for a catversion
bump since the actual catalog data isn't changed.
John Naylor
Discussion: https://postgr.es/m/CAFBsxsHpCbjfoddNGpnnnY5pHwckWfiYkMYSF74PmP1su0+ZOw@mail.gmail.com
M src/backend/catalog/genbki.pl
M src/backend/utils/cache/relcache.c
M src/include/catalog/pg_proc.dat
Fix foreign-key selectivity estimation in the presence of constants.
commit : ad1c36b0709e47cdb3cc4abd6c939fe64279b63f
author : Tom Lane <[email protected]>
date : Wed, 28 Oct 2020 11:15:47 -0400
committer: Tom Lane <[email protected]>
date : Wed, 28 Oct 2020 11:15:47 -0400
get_foreign_key_join_selectivity() looks for join clauses that equate
the two sides of the FK constraint. However, if we have a query like
"WHERE fktab.a = pktab.a and fktab.a = 1", it won't find any such join
clause, because equivclass.c replaces the given clauses with "fktab.a
= 1 and pktab.a = 1", which can be enforced at the scan level, leaving
nothing to be done for column "a" at the join level.
We can fix that expectation without much trouble, but then a new problem
arises: applying the foreign-key-based selectivity rule produces a
rowcount underestimate, because we're effectively double-counting the
selectivity of the "fktab.a = 1" clause. So we have to cancel that
selectivity out of the estimate.
To fix, refactor process_implied_equality() so that it can pass back the
new RestrictInfo to its callers in equivclass.c, allowing the generated
"fktab.a = 1" clause to be saved in the EquivalenceClass's ec_derives
list. Then it's not much trouble to dig out the relevant RestrictInfo
when we need to adjust an FK selectivity estimate. (While at it, we
can also remove the expensive use of initialize_mergeclause_eclasses()
to set up the new RestrictInfo's left_ec and right_ec pointers.
The equivclass.c code can set those basically for free.)
This seems like clearly a bug fix, but I'm hesitant to back-patch it,
first because there's some API/ABI risk for extensions and second because
we're usually loath to destabilize plan choices in stable branches.
Per report from Sigrid Ehrenreich.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/AM6PR02MB5287A0ADD936C1FA80973E72AB190@AM6PR02MB5287.eurprd02.prod.outlook.com
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/util/plancat.c
M src/include/nodes/pathnodes.h
M src/include/optimizer/paths.h
M src/include/optimizer/planmain.h
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Use correct GetDatum() in pg_relation_check_pages()
commit : ce7f772c5e6066e0bbafea5759e652c9757c8e6b
author : Michael Paquier <[email protected]>
date : Wed, 28 Oct 2020 13:59:18 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 28 Oct 2020 13:59:18 +0900
UInt32GetDatum() was getting used, while the result needs
Int64GetDatum(). Oversight in f2b8839.
Per buildfarm member florican.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/pagefuncs.c
Add pg_relation_check_pages() to check on-disk pages of a relation
commit : f2b883969557f4572cdfa87e1a40083d2b1272e7
author : Michael Paquier <[email protected]>
date : Wed, 28 Oct 2020 12:15:00 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 28 Oct 2020 12:15:00 +0900
This makes use of CheckBuffer() introduced in c780a7a, adding a SQL
wrapper able to do checks for all the pages of a relation. By default,
all the fork types of a relation are checked, and it is possible to
check only a given relation fork. Note that if the relation given in
input has no physical storage or is temporary, then no errors are
generated, allowing full-database checks when coupled with a simple scan
of pg_class for example. This is not limited to clusters with data
checksums enabled, as clusters without data checksums can still apply
checks on pages using the page headers or for the case of a page full of
zeros.
This function returns a set of tuples consisting of:
- The physical file where a broken page has been detected (without the
segment number as that can be AM-dependent, which can be guessed from
the block number for heap). A relative path from PGPATH is used.
- The block number of the broken page.
By default, only superusers have an access to this function but
execution rights can be granted to other users.
The feature introduced here is still minimal, and more improvements
could be done, like:
- Addition of a start and end block number to run checks on a range
of blocks, which would apply only if one fork type is checked.
- Addition of some progress reporting.
- Throttling, with configuration parameters in function input or
potentially some cost-based GUCs.
Regression tests are added for positive cases in the main regression
test suite, and TAP tests are added for cases involving the emulation of
page corruptions.
Bump catalog version.
Author: Julien Rouhaud, Michael Paquier
Reviewed-by: Masahiko Sawada, Justin Pryzby
Discussion: https://postgr.es/m/CAOBaU_aVvMjQn=ge5qPiJOPMmOj5=ii3st5Q0Y+WuLML5sR17w@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/catalog/system_views.sql
M src/backend/utils/adt/Makefile
A src/backend/utils/adt/pagefuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
A src/test/recovery/t/022_page_check.pl
A src/test/regress/expected/pagefuncs.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/pagefuncs.sql
M src/tools/pgindent/typedefs.list
Add CheckBuffer() to check on-disk pages without shared buffer loading
commit : c780a7a90a8e7b074405ea2007e34f94e227e695
author : Michael Paquier <[email protected]>
date : Wed, 28 Oct 2020 11:12:46 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 28 Oct 2020 11:12:46 +0900
CheckBuffer() is designed to be a concurrent-safe function able to run
sanity checks on a relation page without loading it into the shared
buffers. The operation is done using a lock on the partition involved
in the shared buffer mapping hashtable and an I/O lock for the buffer
itself, preventing the risk of false positives due to any concurrent
activity.
The primary use of this function is the detection of on-disk corruptions
for relation pages. If a page is found in shared buffers, the on-disk
page is checked if not dirty (a follow-up checkpoint would flush a valid
version of the page if dirty anyway), as it could be possible that a
page was present for a long time in shared buffers with its on-disk
version corrupted. Such a scenario could lead to a corrupted cluster if
a host is plugged off for example. If the page is not found in shared
buffers, its on-disk state is checked. PageIsVerifiedExtended() is used
to apply the same sanity checks as when a page gets loaded into shared
buffers.
This function will be used by an upcoming patch able to check the state
of on-disk relation pages using a SQL function.
Author: Julien Rouhaud, Michael Paquier
Reviewed-by: Masahiko Sawada
Discussion: https://postgr.es/m/CAOBaU_aVvMjQn=ge5qPiJOPMmOj5=ii3st5Q0Y+WuLML5sR17w@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
M src/include/storage/bufmgr.h
Minor improvements in description of spilled counters in pg_stat_replication_slots view.
commit : 9e0f87a4955ede0769d2f9a93171145b6ddab901
author : Amit Kapila <[email protected]>
date : Wed, 28 Oct 2020 07:28:51 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 28 Oct 2020 07:28:51 +0530
Per a suggestion by Justin Pryzby.
Discussion: https://postgr.es/m/CA+fd4k5_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com
M doc/src/sgml/monitoring.sgml
Doc: improve explanation of how to use our code coverage infrastructure.
commit : 8d132b2850d4c95ed5666a09b295dea645bbc884
author : Tom Lane <[email protected]>
date : Tue, 27 Oct 2020 14:31:12 -0400
committer: Tom Lane <[email protected]>
date : Tue, 27 Oct 2020 14:31:12 -0400
The reference to running "make coverage" in a subdirectory was a
bit obscure, so clarify what happens when you do that. Do a little
desultory copy-editing, too.
Per a question from Peter Smith.
Discussion: https://postgr.es/m/CAHut+Pu0r3AjRSyu5E0v2-zRj8r24OSrkWs3fEBxOuaw1i8DKA@mail.gmail.com
M doc/src/sgml/regress.sgml
Makefile comment: remove reference to tools/thread/thread_test
commit : 8e5d1aef8eed26977739b662cddcea2de5efe834
author : Bruce Momjian <[email protected]>
date : Tue, 27 Oct 2020 14:00:38 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 27 Oct 2020 14:00:38 -0400
You can't compile thread_test alone anymore, and the location moved too.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M src/template/netbsd
pg_dump: Lock all relations, not just plain tables
commit : 403a3d91c841beabf3efd7bffddb47a2bce4481f
author : Alvaro Herrera <[email protected]>
date : Tue, 27 Oct 2020 14:31:37 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 27 Oct 2020 14:31:37 -0300
Now that LOCK TABLE can take any relation type, acquire lock on all
relations that are to be dumped. This prevents schema changes or
deadlock errors that could cause a dump to fail after expending much
effort. The server is tested to have the capability and the feature
disabled if it doesn't, so that a patched pg_dump doesn't fail when
connecting to an unpatched server.
Backpatch to 9.5.
Author: Álvaro Herrera <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Reported-by: Wells Oliver <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_backup_db.h
M src/bin/pg_dump/pg_dump.c
Add select_common_typmod()
commit : f893e68d761adbee7f888109b1adf76151e3e17a
author : Peter Eisentraut <[email protected]>
date : Tue, 27 Oct 2020 17:39:23 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 27 Oct 2020 17:39:23 +0100
This accompanies select_common_type() and select_common_collation().
Typmods were previously combined using hand-coded logic in several
places. The logic in select_common_typmod() isn't very exciting, but
it makes the code more compact and readable in a few locations, and in
the future we can perhaps do more complicated things if desired.
As a small enhancement, the type unification of the direct and
aggregate arguments of hypothetical-set aggregates now unifies the
typmod as well using this new function, instead of just dropping it.
Reviewed-by: Heikki Linnakangas <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/backend/parser/analyze.c
M src/backend/parser/parse_clause.c
M src/backend/parser/parse_coerce.c
M src/backend/parser/parse_func.c
M src/include/parser/parse_coerce.h
Accept relations of any kind in LOCK TABLE
commit : 59ab4ac32460a6a93b665f4e487d7ff64979ba4d
author : Alvaro Herrera <[email protected]>
date : Tue, 27 Oct 2020 13:49:19 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 27 Oct 2020 13:49:19 -0300
The restriction that only tables and views can be locked by LOCK TABLE
is quite arbitrary, since the underlying mechanism can lock any relation
type. Drop the restriction so that programs such as pg_dump can lock
all relations they're interested in, preventing schema changes that
could cause a dump to fail after expending much effort.
Backpatch to 9.5.
Author: Álvaro Herrera <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Reported-by: Wells Oliver <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/lock.sgml
M src/backend/commands/lockcmds.c
M src/test/regress/expected/lock.out
M src/test/regress/sql/lock.sql
docs: remove reference to src/tools/thread
commit : 4066b642909176aede03b6c64d29734ce2a34716
author : Bruce Momjian <[email protected]>
date : Tue, 27 Oct 2020 12:43:11 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 27 Oct 2020 12:43:11 -0400
This directory and the ability to build the thread test independently
were removed in commit 8a2121185b.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
Fix enum errdetail to mention bytes, not chars
commit : 0525572860335d050a1bea194a5278c8833304d1
author : Peter Eisentraut <[email protected]>
date : Tue, 27 Oct 2020 11:50:18 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 27 Oct 2020 11:50:18 +0100
The enum label length is in terms of bytes, not charactes.
Author: Ian Lawrence Barwick <[email protected]>
Reviewed-by: Julien Rouhaud <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CAB8KJ=itZEJ7C9BacTHSYgeUysH4xx8wDiOnyppnSLyn6-g+Bw@mail.gmail.com
M src/backend/catalog/pg_enum.c
M src/test/regress/expected/enum.out
Make procedure OUT parameters work with JDBC
commit : 9213462c539e6412fe0498a7f8e20b662e15c4ec
author : Peter Eisentraut <[email protected]>
date : Tue, 27 Oct 2020 08:58:48 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 27 Oct 2020 08:58:48 +0100
The JDBC driver sends OUT parameters with type void. This makes sense
when calling a function, so that the parameters are ignored in
ParseFuncOrColumn(). For a procedure call we want to treat them as
unknown.
Reviewed-by: Andrew Dunstan <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/d7e49540-ea92-b4e2-5fff-42036102f968%402ndquadrant.com
M src/backend/parser/parse_param.c
doc: Fix order of protocol messages in listing
commit : 8fed2eadb8558185d162cc7fd09192c2e10b915f
author : Peter Eisentraut <[email protected]>
date : Tue, 27 Oct 2020 08:43:35 +0100
committer: Peter Eisentraut <[email protected]>
date : Tue, 27 Oct 2020 08:43:35 +0100
Move GSSENCRequest to the correct alphabetical position.
M doc/src/sgml/protocol.sgml
Update description of spilled counters in pg_stat_replication_slots view.
commit : 2f0760c9ff827bb3d23ee327e6b46038000c7ef9
author : Amit Kapila <[email protected]>
date : Tue, 27 Oct 2020 08:22:39 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 27 Oct 2020 08:22:39 +0530
This is to make the description of spilled counters clear.
Author: Amit Kapila
Reviewed-by: Sawada Masahiko
Discussion: https://postgr.es/m/CA+fd4k5_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com
M doc/src/sgml/monitoring.sgml
doc: simplify wording of function error affects
commit : 2f17fe431860fc434084d99d8b789e6ec7e6977b
author : Bruce Momjian <[email protected]>
date : Mon, 26 Oct 2020 22:38:11 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 26 Oct 2020 22:38:11 -0400
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/plpgsql.sgml
doc: make blooms docs match reality
commit : e9661f2b0f3310d8bacc3f0802b309f599300cfb
author : Bruce Momjian <[email protected]>
date : Mon, 26 Oct 2020 19:17:05 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 26 Oct 2020 19:17:05 -0400
Parallel execution changed the way bloom queries are executed, so update
the EXPLAIN output, and restructure the docs to be clearer and more
accurate.
Reported-by: Daniel Westermann
Discussion: https://postgr.es/m/ZR0P278MB0122119FAE78721A694C30C8D2340@ZR0P278MB0122.CHEP278.PROD.OUTLOOK.COM
Author: Daniel Westermann and me
Backpatch-through: 9.6
M doc/src/sgml/bloom.sgml
In INSERT/UPDATE, use the table's real tuple descriptor as target.
commit : 20d3fe9009ddbbbb3da3a2da298f922054b43f8c
author : Tom Lane <[email protected]>
date : Mon, 26 Oct 2020 11:36:53 -0400
committer: Tom Lane <[email protected]>
date : Mon, 26 Oct 2020 11:36:53 -0400
Previously, ExecInitModifyTable relied on ExecInitJunkFilter,
and thence ExecCleanTypeFromTL, to build the target descriptor from
the query tlist. While we just checked (in ExecCheckPlanOutput)
that the tlist produces compatible output, this is not a great
substitute for the relation's actual tuple descriptor that's
available from the relcache. For one thing, dropped columns will
not be correctly marked attisdropped; it's a bit surprising that
we've gotten away with that this long. But the real reason for
being concerned with this is that using the table's descriptor means
that the slot will have correct attrmissing data, allowing us to
revert the klugy fix of commit ba9f18abd. (This commit undoes
that one's changes in trigger.c, but keeps the new test case.)
Thus we can solve the bogus-trigger-tuple problem with fewer cycles
rather than more.
No back-patch, since this doesn't fix any additional bug, and it
seems somewhat more likely to have unforeseen side effects than
ba9f18abd's narrow fix.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/trigger.c
M src/backend/executor/execJunk.c
M src/backend/executor/nodeModifyTable.c
M src/include/executor/executor.h
docs: Remove notes about incompatibilies with very old versions.
commit : fa42c2ecb0f6e89f74bc1cc37b56a1d43e45d513
author : Heikki Linnakangas <[email protected]>
date : Mon, 26 Oct 2020 09:07:14 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 26 Oct 2020 09:07:14 +0200
These are old enough that they'll cause more confusion and distraction
to new readers, than they could help anyone upgrade from very old
servers.
Discussion: https://www.postgresql.org/message-id/fd93f1c5-7818-a02c-01e5-1075ac0d4def%40iki.fi
M doc/src/sgml/func.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/pgfreespacemap.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
Extend PageIsVerified() to handle more custom options
commit : d401c5769ef6aeef0a28c147f3fb5afedcd59984
author : Michael Paquier <[email protected]>
date : Mon, 26 Oct 2020 09:55:28 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 26 Oct 2020 09:55:28 +0900
This is useful for checks of relation pages without having to load the
pages into the shared buffers, and two cases can make use of that: page
verification in base backups and the online, lock-safe, flavor.
Compatibility is kept with past versions using a macro that calls the
new extended routine with the set of options compatible with the
original version.
Extracted from a larger patch by the same author.
Author: Anastasia Lubennikova
Reviewed-by: Michael Paquier, Julien Rouhaud
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/storage.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/page/bufpage.c
M src/include/storage/bufpage.h
Fix corner case for a BEFORE ROW UPDATE trigger returning OLD.
commit : ba9f18abd3650e385e9a35df7145a7c38af17e92
author : Tom Lane <[email protected]>
date : Sun, 25 Oct 2020 13:57:46 -0400
committer: Tom Lane <[email protected]>
date : Sun, 25 Oct 2020 13:57:46 -0400
If the old row has any "missing" attributes that are supposed to
be retrieved from an associated tuple descriptor, the wrong things
happened because the trigger result is shoved directly into an
executor slot that lacks the missing-attribute data. Notably,
CHECK-constraint verification would incorrectly see those columns
as NULL, and so would RETURNING-list evaluation.
Band-aid around this by forcibly expanding the tuple before passing
it to the trigger function. (IMO it was a fundamental misdesign to
put the missing-attribute data into tuple constraints, which so
much of the system considers to be optional. But we're probably
stuck with that now, and will have to continue to apply band-aids
as we find other places with similar issues.)
Back-patch to v12. v11 would also have the issue, except that
commit 920311ab1 already applied a similar band-aid. That forced
expansion in more cases than seem really necessary, though, so
this isn't a directly equivalent fix.
Amit Langote, with some cosmetic changes by me
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Fix incorrect parameter name in a function header comment
commit : e83c9f913c6197586af8ac53c1d3652db15a3c91
author : David Rowley <[email protected]>
date : Sun, 25 Oct 2020 22:39:00 +1300
committer: David Rowley <[email protected]>
date : Sun, 25 Oct 2020 22:39:00 +1300
Author: Zhijie Hou
Discussion: https://postgr.es/m/14cd74ea00204cc8a7ea5d738ac82cd1@G08CNEXMBPEKD05.g08.fujitsu.local
Backpatch-through: 12, where the mistake was introduced
M src/backend/utils/cache/lsyscache.c
Fix ancient bug in ecpg's pthread_once() emulation for Windows.
commit : 21d36747d4fafe16539a0c55ebb91a01e4053e3c
author : Tom Lane <[email protected]>
date : Sat, 24 Oct 2020 13:12:08 -0400
committer: Tom Lane <[email protected]>
date : Sat, 24 Oct 2020 13:12:08 -0400
We must not set the "done" flag until after we've executed the
initialization function. Otherwise, other threads can fall through
the initial unlocked test before initialization is really complete.
This has been seen to cause rare failures of ecpg's thread/descriptor
test, and it could presumably cause other sorts of misbehavior in
threaded ECPG-using applications, since ecpglib relies on
pthread_once() in several places.
Diagnosis and patch by me, based on investigation by Alexander Lakhin.
Back-patch to all supported branches (the bug dates to 2007).
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/ecpg/ecpglib/misc.c
Fix issue with --enable-coverage and the new unicode {de,re}composition code
commit : 2771fcee18be0b86d2e008add20f73d175e06e90
author : Michael Paquier <[email protected]>
date : Sat, 24 Oct 2020 14:20:38 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 24 Oct 2020 14:20:38 +0900
genhtml has been generating the following warning with this new code:
WARNING: function data mismatch at /path/src/common/unicode_norm.c:102
HTML coverage reports care about the uniqueness of functions defined in
source files, ignoring any assumptions around CFLAGS. 783f0cc
introduced a duplicated definition of get_code_entry(), leading to a
warning and potentially some incorrect data generated in the reports.
This refactors the code so as the code has only one function
declaration, fixing the warning.
Oversight in 783f0cc.
Reported-by: Tom Lane
Author: Michael Paquier
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/common/unicode_norm.c
Add tab completion for ALTER TABLE .. FORCE ROW LEVEL SECURITY in psql
commit : 0b46e82c06b0c4b0dc6a94a890d23945ebf720fd
author : Michael Paquier <[email protected]>
date : Sat, 24 Oct 2020 10:29:55 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 24 Oct 2020 10:29:55 +0900
This completes both the FORCE and NO FORCE options, NO INHERIT needing a
small adjustment.
Author: Li Japin
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/tab-complete.c
Fix more portability issues in new amcheck code.
commit : 321633e17b07968e68ca5341429e2c8bbf15c331
author : Tom Lane <[email protected]>
date : Fri, 23 Oct 2020 19:08:01 -0400
committer: Tom Lane <[email protected]>
date : Fri, 23 Oct 2020 19:08:01 -0400
verify_heapam() wasn't being careful to sanity-check tuple line
pointers before using them, resulting in SIGBUS on alignment-picky
architectures. Fix that, add some more test coverage.
Mark Dilger, some tweaking by me
Discussion: https://postgr.es/m/[email protected]
M contrib/amcheck/t/001_verify_heapam.pl
M contrib/amcheck/verify_heapam.c
Allow psql to re-use connection parameters after a connection loss.
commit : 1b62d0fb3e50ede570d0d4e4a2be69d5645b48a7
author : Tom Lane <[email protected]>
date : Fri, 23 Oct 2020 17:07:15 -0400
committer: Tom Lane <[email protected]>
date : Fri, 23 Oct 2020 17:07:15 -0400
Instead of immediately PQfinish'ing a dead connection, save it aside
so that we can still extract its parameters for \connect attempts.
(This works because PQconninfo doesn't care whether the PGconn is in
CONNECTION_BAD state.) This allows developers to reconnect with
just \c after a database crash and restart.
It's tempting to use the same approach instead of closing the old
connection after a failed non-interactive \connect command. However,
that would not be very safe: consider a script containing
\c db1 user1 live_server
\c db2 user2 dead_server
\c db3
The script would be expecting to connect to db3 at dead_server, but
if we re-use parameters from the first connection then it might
successfully connect to db3 at live_server. This'd defeat the goal
of not letting a script accidentally execute commands against the
wrong database.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/common.c
M src/bin/psql/describe.c
M src/bin/psql/settings.h
M src/bin/psql/startup.c
Fix portability issues in new amcheck test.
commit : 860593ec3bd15e8969effdfcb5cbd98c561dd722
author : Tom Lane <[email protected]>
date : Fri, 23 Oct 2020 14:02:00 -0400
committer: Tom Lane <[email protected]>
date : Fri, 23 Oct 2020 14:02:00 -0400
The tests added by commit 866e24d47 failed on big-endian machines
due to lack of attention to endianness considerations. Fix that.
While here, improve a few small cosmetic things, such as running
it through perltidy.
Mark Dilger
Discussion: https://postgr.es/m/[email protected]
M contrib/amcheck/t/001_verify_heapam.pl
Fix broken XML formatting in EXPLAIN output for incremental sorts.
commit : 87a174c0e77eed0bec7d53ef6d470d60335f6444
author : Tom Lane <[email protected]>
date : Fri, 23 Oct 2020 11:32:33 -0400
committer: Tom Lane <[email protected]>
date : Fri, 23 Oct 2020 11:32:33 -0400
The ExplainCloseGroup arguments for incremental sort usage data
didn't match the corresponding ExplainOpenGroup. This only matters
for XML-format output, which is probably why we'd not noticed.
Daniel Gustafsson, per bug #16683 from Frits Jalvingh
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/explain.c
doc: Fix order of protocol messages in listing
commit : 83d727e5b23c43f3fb7221963ddec24277c1126f
author : Peter Eisentraut <[email protected]>
date : Fri, 23 Oct 2020 13:01:39 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 23 Oct 2020 13:01:39 +0200
The order of AuthenticationGSSContinue and AuthenticationSSPI was
swapped, based on the other Authentication* protocol messages being
listed in subcode order.
M doc/src/sgml/protocol.sgml
doc: Remove reference to pre-8.2 pg_dump behaviour
commit : 902b57c9bf27b3ea2963ef832cddb38b1203130e
author : Heikki Linnakangas <[email protected]>
date : Fri, 23 Oct 2020 11:49:59 +0300
committer: Heikki Linnakangas <[email protected]>
date : Fri, 23 Oct 2020 11:49:59 +0300
The behavioural change in the -t/--table option happened around 15 years
ago and there seems little point in keeping it around.
Author: Ian Barwick
Discussion: https://www.postgresql.org/message-id/CAB8KJ%3Dh-XALik4M7gv-pX48%3D%2BSPWexfaYwa%2ByTnPwD3DxceXrg%40mail.gmail.com
M doc/src/sgml/ref/pg_dump.sgml
Fix initialization of es_result_relations in EvalPlanQualStart().
commit : 22b73d3cb0b5bb4c141421f98dd67f091dda3c20
author : Heikki Linnakangas <[email protected]>
date : Fri, 23 Oct 2020 09:30:08 +0300
committer: Heikki Linnakangas <[email protected]>
date : Fri, 23 Oct 2020 09:30:08 +0300
Thinko in commit 1375422c782. EvalPlanQualStart() was mistakenly
resetting the parent EState's es_result_relations, when it should
initialize the field in the child EPQ EState it just created.
That was clearly wrong, but it didn't cause any ill effects, because
es_result_relations is currently not used after the ExecInit* phase.
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqFEuq8AAAmxXsTDVZ1r38cHbfYuiPQx_%3DYyKe2DC-6q4A%40mail.gmail.com
M src/backend/executor/execMain.c
Improve performance of Unicode {de,re}composition in the backend
commit : 783f0cc64dcc05e3d112a06b1cd181e5a1ca9099
author : Michael Paquier <[email protected]>
date : Fri, 23 Oct 2020 11:05:46 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 23 Oct 2020 11:05:46 +0900
This replaces the existing binary search with two perfect hash functions
for the composition and the decomposition in the backend code, at the
cost of slightly-larger binaries there (35kB in libpgcommon_srv.a). Per
the measurements done, this improves the speed of the recomposition and
decomposition by up to 30~40 times for the NFC and NFKC conversions,
while all other operations get at least 40% faster. This is not as
"good" as what libicu has, but it closes the gap a lot as per the
feedback from Daniel Verite.
The decomposition table remains the same, getting used for the binary
search in the frontend code, where we care more about the size of the
libraries like libpq over performance as this gets involved only in code
paths related to the SCRAM authentication. In consequence, note that
the perfect hash function for the recomposition needs to use a new
inverse lookup array back to to the existing decomposition table.
The size of all frontend deliverables remains unchanged, even with
--enable-debug, including libpq.
Author: John Naylor
Reviewed-by: Michael Paquier, Tom Lane
Discussion: https://postgr.es/m/CAFBsxsHUuMFCt6-pU+oG-F1==CmEp8wR+O+bRouXWu6i8kXuqA@mail.gmail.com
M src/common/unicode/Makefile
M src/common/unicode/generate-unicode_norm_table.pl
M src/common/unicode_norm.c
A src/include/common/unicode_norm_hashfunc.h
M src/tools/pgindent/exclude_file_patterns
Update time zone data files to tzdata release 2020d.
commit : 7d6d6bce43c60bb7b77237e2cc6ab845646b911f
author : Tom Lane <[email protected]>
date : Thu, 22 Oct 2020 21:23:47 -0400
committer: Tom Lane <[email protected]>
date : Thu, 22 Oct 2020 21:23:47 -0400
DST law changes in Palestine, with a whopping 120 hours' notice.
Also some historical corrections for Palestine.
M src/timezone/data/tzdata.zi
Sync our copy of the timezone library with IANA release tzcode2020d.
commit : c5054da0d7ee59a93ca6d7f7fc669ef0210dbad3
author : Tom Lane <[email protected]>
date : Thu, 22 Oct 2020 21:15:22 -0400
committer: Tom Lane <[email protected]>
date : Thu, 22 Oct 2020 21:15:22 -0400
There's no functional change at all here, but I'm curious to see
whether this change successfully shuts up Coverity's warning about
a useless strcmp(), which appeared with the previous update.
Discussion: http://mm.icann.org/pipermail/tz/2020-October/029370.html
M src/timezone/README
M src/timezone/zic.c
Add documentation and tests for quote marks in ECPG literal queries.
commit : c16a1bbcf498f0aa053a3e55008f57d7f67357dd
author : Tom Lane <[email protected]>
date : Thu, 22 Oct 2020 18:29:40 -0400
committer: Tom Lane <[email protected]>
date : Thu, 22 Oct 2020 18:29:40 -0400
ECPG's PREPARE ... FROM and EXECUTE IMMEDIATE can optionally take
the target query as a simple literal, rather than the more usual
string-variable reference. This was previously documented as
being a C string literal, but that's a lie in one critical respect:
you can't write a data double quote as \" in such literals. That's
because the lexer is in SQL mode at this point, so it'll parse
double-quoted strings as SQL identifiers, within which backslash
is not special, so \" ends the literal.
I looked into making this work as documented, but getting the lexer
to switch behaviors at just the right point is somewhere between
very difficult and impossible. It's not really worth the trouble,
because these cases are next to useless: if you have a fixed SQL
statement to execute or prepare, you might as well write it as
a direct EXEC SQL, saving the messiness of converting it into
a string literal and gaining the opportunity for compile-time
SQL syntax checking.
Instead, let's just document (and test) the workaround of writing
a double quote as an octal escape (\042) in such cases.
There's no code behavioral change here, so in principle this could
be back-patched, but it's such a niche case I doubt it's worth
the trouble.
Per report from 1250kv.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/preproc/pgc.l
M src/interfaces/ecpg/test/expected/sql-execute.c
M src/interfaces/ecpg/test/expected/sql-execute.stderr
M src/interfaces/ecpg/test/sql/execute.pgc
Avoid premature de-doubling of quote marks in ECPG strings.
commit : 3dfb1942d9b8748b93094a430289e2f7f3b3ae0d
author : Tom Lane <[email protected]>
date : Thu, 22 Oct 2020 17:34:32 -0400
committer: Tom Lane <[email protected]>
date : Thu, 22 Oct 2020 17:34:32 -0400
If you write the literal 'abc''def' in an EXEC SQL command, that will
come out the other end as 'abc'def', triggering a syntax error in the
backend. Likewise, "abc""def" is reduced to "abc"def" which is wrong
syntax for a quoted identifier.
The cause is that the lexer thinks it should emit just one quote
mark, whereas what it really should do is keep the string as-is.
Add some docs and test cases, too.
Although this seems clearly a bug, I fear users wouldn't appreciate
changing it in minor releases. Some may well be working around it
by applying an extra doubling of affected quotes, as for example
sql/dyntest.pgc has been doing.
Per investigation of a report from 1250kv, although this isn't
exactly what he/she was on about.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/preproc/pgc.l
M src/interfaces/ecpg/test/expected/preproc-strings.c
M src/interfaces/ecpg/test/expected/preproc-strings.stderr
M src/interfaces/ecpg/test/expected/preproc-strings.stdout
M src/interfaces/ecpg/test/preproc/strings.pgc
M src/interfaces/ecpg/test/sql/dyntest.pgc
Try to avoid a compiler warning about using fxid uninitialized.
commit : 8bb0c9770e80fa79f15b27af8f3c3f2833225aa3
author : Robert Haas <[email protected]>
date : Thu, 22 Oct 2020 16:13:58 -0400
committer: Robert Haas <[email protected]>
date : Thu, 22 Oct 2020 16:13:58 -0400
Mark Dilger, with a couple of stray semicolons removed by me.
Discussion: http://postgr.es/m/[email protected]
M contrib/amcheck/verify_heapam.c
Clean up some unpleasant behaviors in psql's \connect command.
commit : 94929f1cf6cb3ea070d0919d1303379b525a72a3
author : Tom Lane <[email protected]>
date : Thu, 22 Oct 2020 14:04:21 -0400
committer: Tom Lane <[email protected]>
date : Thu, 22 Oct 2020 14:04:21 -0400
The check for whether to complain about not having an old connection
to get parameters from was seriously out of date: it had not been
rethought when we invented connstrings, nor when we invented the
-reuse-previous option. Replace it with a check that throws an
error if reuse-previous is active and we lack an old connection to
reuse. While that doesn't move the goalposts very far in terms of
easing reconnection after a server crash, at least it's consistent.
If the user specifies a connstring plus additional parameters
(which is invalid per the documentation), the extra parameters were
silently ignored. That seems like it could be really confusing,
so let's throw a syntax error instead.
Teach the connstring code path to re-use the old connection's password
in the same cases as the old-style-syntax code path would, ie if we
are reusing parameters and the values of username, host/hostaddr, and
port are not being changed. Document this behavior, too, since it was
unmentioned before. Also simplify the implementation a bit, giving
rise to two new and useful properties: if there's a "password=xxx" in
the connstring, we'll use it not ignore it, and by default (i.e.,
except with --no-password) we will prompt for a password if the
re-used password or connstring password doesn't work. The previous
code just failed if the re-used password didn't work.
Given the paucity of field complaints about these issues, I don't
think that they rise to the level of back-patchable bug fixes,
and in any case they might represent undesirable behavior changes
in minor releases. So no back-patch.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
Extend amcheck to check heap pages.
commit : 866e24d47db1743dfcff5bd595b57e3a143f2cb1
author : Robert Haas <[email protected]>
date : Thu, 22 Oct 2020 08:44:18 -0400
committer: Robert Haas <[email protected]>
date : Thu, 22 Oct 2020 08:44:18 -0400
Mark Dilger, reviewed by Peter Geoghegan, Andres Freund, Álvaro Herrera,
Michael Paquier, Amul Sul, and by me. Some last-minute cosmetic
revisions by me.
Discussion: http://postgr.es/m/[email protected]
M contrib/amcheck/Makefile
A contrib/amcheck/amcheck–1.2–1.3.sql
M contrib/amcheck/amcheck.control
A contrib/amcheck/expected/check_heap.out
A contrib/amcheck/sql/check_heap.sql
A contrib/amcheck/t/001_verify_heapam.pl
A contrib/amcheck/verify_heapam.c
M doc/src/sgml/amcheck.sgml
M src/backend/access/heap/hio.c
M src/backend/access/transam/multixact.c
M src/include/access/multixact.h
M src/tools/pgindent/typedefs.list
Use croak instead of die in Perl code when appropriate
commit : f8721bd752790859df747905bc44fb5ad8dbf07d
author : Peter Eisentraut <[email protected]>
date : Thu, 22 Oct 2020 13:29:39 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 22 Oct 2020 13:29:39 +0200
M src/test/perl/TestLib.pm
Optimize a few list_delete_ptr calls
commit : e7c2b95d37a2b9c01367f7ccc55703555b39c81c
author : David Rowley <[email protected]>
date : Thu, 22 Oct 2020 14:36:32 +1300
committer: David Rowley <[email protected]>
date : Thu, 22 Oct 2020 14:36:32 +1300
There is a handful of places where we called list_delete_ptr() to remove
some element from a List. In many of these places we know, or with very
little additional effort know the index of the ListCell that we need to
remove.
Here we change all of those places to instead either use one of;
list_delete_nth_cell(), foreach_delete_current() or list_delete_last().
Each of these saves from having to iterate over the list to search for the
element to remove by its pointer value.
There are some small performance gains to be had by doing this, but in the
general case, none of these lists are likely to be very large, so the
lookup was probably never that expensive anyway. However, some of the
calls are in fairly hot code paths, e.g process_equivalence(). So any
small gains there are useful.
Author: Zhijie Hou and David Rowley
Discussion: https://postgr.es/m/b3517353ec7c4f87aa560678fbb1034b@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/path/joinpath.c
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_utilcmd.c
M src/backend/rewrite/rewriteHandler.c
Fix connection string handling in psql's \connect command.
commit : 85c54287af56fe351b53913ea2b81e9d6145f964
author : Tom Lane <[email protected]>
date : Wed, 21 Oct 2020 16:18:40 -0400
committer: Tom Lane <[email protected]>
date : Wed, 21 Oct 2020 16:18:40 -0400
psql's \connect claims to be able to re-use previous connection
parameters, but in fact it only re-uses the database name, user name,
host name (and possibly hostaddr, depending on version), and port.
This is problematic for assorted use cases. Notably, pg_dump[all]
emits "\connect databasename" commands which we would like to have
re-use all other parameters. If such a script is loaded in a psql run
that initially had "-d connstring" with some non-default parameters,
those other parameters would be lost, potentially causing connection
failure. (Thus, this is the same kind of bug addressed in commits
a45bc8a4f and 8e5793ab6, although the details are much different.)
To fix, redesign do_connect() so that it pulls out all properties
of the old PGconn using PQconninfo(), and then replaces individual
properties in that array. In the case where we don't wish to re-use
anything, get libpq's default settings using PQconndefaults() and
replace entries in that, so that we don't need different code paths
for the two cases.
This does result in an additional behavioral change for cases where
the original connection parameters allowed multiple hosts, say
"psql -h host1,host2", and the \connect request allows re-use of the
host setting. Because the previous coding relied on PQhost(), it
would only permit reconnection to the same host originally selected.
Although one can think of scenarios where that's a good thing, there
are others where it is not. Moreover, that behavior doesn't seem to
meet the principle of least surprise, nor was it documented; nor is
it even clear it was intended, since that coding long pre-dates the
addition of multi-host support to libpq. Hence, this patch is content
to drop it and re-use the host list as given.
Per Peter Eisentraut's comments on bug #16604. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
Use fast checkpoint in PostgresNode::backup()
commit : 831611b11c34ed0066633864f42ff67a10286aee
author : Alvaro Herrera <[email protected]>
date : Wed, 21 Oct 2020 14:37:26 -0300
committer: Alvaro Herrera <[email protected]>
date : Wed, 21 Oct 2020 14:37:26 -0300
Should cause tests to be a bit faster
M src/test/perl/PostgresNode.pm
Remove the option to build thread_test.c outside configure.
commit : 8a2121185b26d7eb439860c7a96a08c0aa3c8508
author : Tom Lane <[email protected]>
date : Wed, 21 Oct 2020 12:08:48 -0400
committer: Tom Lane <[email protected]>
date : Wed, 21 Oct 2020 12:08:48 -0400
Theoretically one could go into src/test/thread and build/run this
program there. In practice, that hasn't worked since 96bf88d52,
and probably much longer on some platforms (likely including just
the sort of hoary leftovers where this test might be of interest).
While it wouldn't be too hard to repair the breakage, the fact that
nobody has noticed for two years shows that there is zero usefulness
in maintaining this build pathway. Let's get rid of it and decree
that thread_test.c is *only* meant to be built/used in configure.
Given that decision, it makes sense to put thread_test.c under config/
and get rid of src/test/thread altogether, so that's what I did.
In passing, update src/test/README, which had been ignored by some
not-so-recent additions of subdirectories.
Discussion: https://postgr.es/m/[email protected]
R093 src/test/thread/thread_test.c config/thread_test.c
M configure
M configure.ac
M src/Makefile
M src/port/thread.c
M src/test/Makefile
M src/test/README
D src/test/thread/.gitignore
D src/test/thread/Makefile
D src/test/thread/README
Remove obsolete ifdefs
commit : 555eb1a4f0114e1c4017093fe840045dde1ed9ea
author : Peter Eisentraut <[email protected]>
date : Wed, 21 Oct 2020 14:20:50 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 21 Oct 2020 14:20:50 +0200
Commit 8dace66e0735ca39b779922d02c24ea2686e6521 added #ifdefs for a
number of errno symbols because they were not present on Windows.
Later, commit 125ad539a275db5ab8f4647828b80a16d02eabd2 added
replacement #defines for some of those symbols. So some of the
changes from the first commit are made dead code by the second commit
and can now be removed.
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/port/strerror.c
Fix -Wcast-function-type warnings on Windows/MinGW
commit : 8a58347a3c10126ce9eb38872a46c55e91faa174
author : Peter Eisentraut <[email protected]>
date : Wed, 21 Oct 2020 08:17:51 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 21 Oct 2020 08:17:51 +0200
After de8feb1f3a23465b5737e8a8c160e8ca62f61339, some warnings remained
that were only visible when using GCC on Windows. Fix those as well.
Note that the ecpg test source files don't use the full pg_config.h,
so we can't use pg_funcptr_t there but have to do it the long way.
M src/backend/libpq/auth.c
M src/backend/port/win32/crashdump.c
M src/bin/pg_ctl/pg_ctl.c
M src/common/restricted_token.c
M src/interfaces/ecpg/test/expected/thread-thread.c
M src/interfaces/ecpg/test/expected/thread-thread_implicit.c
M src/interfaces/ecpg/test/thread/thread.pgc
M src/interfaces/ecpg/test/thread/thread_implicit.pgc
M src/port/getaddrinfo.c
M src/port/win32env.c
M src/port/win32stat.c
Review format of code generated by PerfectHash.pm
commit : 19ae53c92d5f5bdfb971d560a562e84c5f65c8b0
author : Michael Paquier <[email protected]>
date : Wed, 21 Oct 2020 09:22:27 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 21 Oct 2020 09:22:27 +0900
80f8eb7 has added to the normalization quick check headers some code
generated by PerfectHash.pm that is incompatible with the settings of
gitattributes for this repository, as whitespaces followed a set of tabs
for the first element of a line in the table. Instead of adding a new
exception to gitattributes, rework the format generated so as a right
padding with spaces is used instead of a left padding. This keeps the
table generated in a readable shape with its set of columns, making
unnecessary an update of gitattributes.
Reported-by: Peter Eisentraut
Author: John Naylor
Discussion: https://postgr.es/m/[email protected]
M src/include/common/unicode_normprops_table.h
M src/tools/PerfectHash.pm
Fix ALTER TABLE .. ENABLE/DISABLE TRIGGER recursion
commit : bbb927b4db9b3b449ccd0f76c1296de382a2f0c1
author : Alvaro Herrera <[email protected]>
date : Tue, 20 Oct 2020 19:22:09 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 20 Oct 2020 19:22:09 -0300
More precisely, correctly handle the ONLY flag indicating not to
recurse. This was implemented in 86f575948c77 by recursing in
trigger.c, but that's the wrong place; use ATSimpleRecursion instead,
which behaves properly. However, because legacy inheritance has never
recursed in that situation, make sure to do that only for new-style
partitioning.
I noticed this problem while testing a fix for another bug in the
vicinity.
This has been wrong all along, so backpatch to 11.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Change the attribute name in pg_stat_replication_slots view.
commit : 03d51b776d88badbeb4452c1ab452feb78dce36a
author : Amit Kapila <[email protected]>
date : Tue, 20 Oct 2020 10:24:36 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 20 Oct 2020 10:24:36 +0530
Change the attribute 'name' to 'slot_name' in pg_stat_replication_slots
view to make it clear and that way we will be consistent with the other
places like pg_stat_wal_receiver view where we display the same attribute.
In the passing, fix the typo in one of the macros in the related code.
Bump the catversion as we have modified the name in the catalog as well.
Reported-by: Noriyoshi Shinoda
Author: Noriyoshi Shinoda
Reviewed-by: Sawada Masahiko and Amit Kapila
Discussion: https://postgr.es/m/CA+fd4k5_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/rules.out
Fix connection string handling in src/bin/scripts/ programs.
commit : 8e5793ab60bba65ffaa0f2237b39c9580d8972c7
author : Tom Lane <[email protected]>
date : Mon, 19 Oct 2020 19:03:46 -0400
committer: Tom Lane <[email protected]>
date : Mon, 19 Oct 2020 19:03:46 -0400
When told to process all databases, clusterdb, reindexdb, and vacuumdb
would reconnect by replacing their --maintenance-db parameter with the
name of the target database. If that parameter is a connstring (which
has been allowed for a long time, though we failed to document that
before this patch), we'd lose any other options it might specify, for
example SSL or GSS parameters, possibly resulting in failure to connect.
Thus, this is the same bug as commit a45bc8a4f fixed in pg_dump and
pg_restore. We can fix it in the same way, by using libpq's rules for
handling multiple "dbname" parameters to add the target database name
separately. I chose to apply the same refactoring approach as in that
patch, with a struct to handle the command line parameters that need to
be passed through to connectDatabase. (Maybe someday we can unify the
very similar functions here and in pg_dump/pg_restore.)
Per Peter Eisentraut's comments on bug #16604. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/clusterdb.sgml
M doc/src/sgml/ref/createdb.sgml
M doc/src/sgml/ref/dropdb.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
M src/bin/scripts/clusterdb.c
M src/bin/scripts/common.c
M src/bin/scripts/common.h
M src/bin/scripts/createdb.c
M src/bin/scripts/createuser.c
M src/bin/scripts/dropdb.c
M src/bin/scripts/dropuser.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/scripts_parallel.c
M src/bin/scripts/scripts_parallel.h
M src/bin/scripts/vacuumdb.c
Fix list-munging bug that broke SQL function result coercions.
commit : c8ab9701791e22f7a8e1badf362654db179c9703
author : Tom Lane <[email protected]>
date : Mon, 19 Oct 2020 14:33:01 -0400
committer: Tom Lane <[email protected]>
date : Mon, 19 Oct 2020 14:33:01 -0400
Since commit 913bbd88d, check_sql_fn_retval() can either insert type
coercion steps in-line in the Query that produces the SQL function's
results, or generate a new top-level Query to perform the coercions,
if modifying the Query's output in-place wouldn't be safe. However,
it appears that the latter case has never actually worked, because
the code tried to inject the new Query back into the query list it was
passed ... which is not the list that will be used for later processing
when we execute the SQL function "normally" (without inlining it).
So we ended up with no coercion happening at run-time, leading to
wrong results or crashes depending on the datatypes involved.
While the regression tests look like they cover this area well enough,
through a huge bit of bad luck all the test cases that exercise the
separate-Query path were checking either inline-able cases (which
accidentally didn't have the bug) or cases that are no-ops at runtime
(e.g., varchar to text), so that the failure to perform the coercion
wasn't obvious. The fact that the cases that don't work weren't
allowed at all before v13 probably contributed to not noticing the
problem sooner, too.
To fix, get rid of the separate "flat" list of Query nodes and instead
pass the real two-level list that is going to be used later. I chose
to make the same change in check_sql_fn_statements(), although that has
no actual bug, just so that we don't need that data structure at all.
This is an API change, as evidenced by the adjustments needed to
callers outside functions.c. That's a bit scary to be doing in a
released branch, but so far as I can tell from a quick search,
there are no outside callers of these functions (and they are
sufficiently specific to our semantics for SQL-language functions that
it's not apparent why any extension would need to call them). In any
case, v13 already changed the API of check_sql_fn_retval() compared to
prior branches.
Per report from pinker. Back-patch to v13 where this code came in.
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/pg_proc.c
M src/backend/executor/functions.c
M src/backend/optimizer/util/clauses.c
M src/include/executor/functions.h
M src/test/regress/expected/rangefuncs.out
M src/test/regress/sql/rangefuncs.sql
Misc documentation fixes.
commit : c5f42daa6077a4c309c5280a47d0e114c12dc572
author : Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 19:28:54 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 19:28:54 +0300
- Misc grammar and punctuation fixes.
- Stylistic cleanup: use spaces between function arguments and JSON fields
in examples. For example "foo(a,b)" -> "foo(a, b)". Add semicolon after
last END in a few PL/pgSQL examples that were missing them.
- Make sentence that talked about "..." and ".." operators more clear,
by avoiding to end the sentence with "..". That makes it look the same
as "..."
- Fix syntax description for HAVING: HAVING conditions cannot be repeated
Patch by Justin Pryzby, per Yaroslav Schekin's report. Backpatch to all
supported versions, to the extent that the patch applies easily.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/dblink.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/hstore.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/isn.sgml
M doc/src/sgml/ltree.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/parallel.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/select.sgml
M doc/src/sgml/ref/select_into.sgml
M doc/src/sgml/rules.sgml
M doc/src/sgml/seg.sgml
M doc/src/sgml/textsearch.sgml
Fix TRUNCATE doc: ALTER SEQUENCE RESTART is now transactional.
commit : 2a972e0165c9b34da4cff3001a2dad8e5a9a5a2b
author : Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 19:02:25 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 19:02:25 +0300
ALTER SEQUENCE RESTART was made transactional in commit 3d79013b97.
Backpatch to v10, where that was introduced.
Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/ref/truncate.sgml
Fix output of tsquery example in docs.
commit : c0bc4c682ee2a84dee2b562db3d17d173a6abc06
author : Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 18:50:33 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 18:50:33 +0300
The output for this query changed in commit 4e2477b7b8. Backport to 9.6
like that commit.
Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/textsearch.sgml
Fix doc for full text search distance operator.
commit : 1a64c7636f24096155f7aa15d7164eba3cdef075
author : Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 17:58:38 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 17:58:38 +0300
Commit 028350f619 changed its behavior from "at most" to "exactly", but
forgot to update the documentation. Backpatch to 9.6.
Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/textsearch.sgml
Update link for pllua
commit : b4d5b458e6fb4c861f92888753deea6a5005a68d
author : Magnus Hagander <[email protected]>
date : Mon, 19 Oct 2020 13:47:09 +0200
committer: Magnus Hagander <[email protected]>
date : Mon, 19 Oct 2020 13:47:09 +0200
Author: Daniel Gustafsson <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/external-projects.sgml
Remove PartitionRoutingInfo struct.
commit : fb5883da86154c3126264bfd97b0cd6f293bcebd
author : Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 14:11:57 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 14:11:57 +0300
The extra indirection neeeded to access its members via its enclosing
ResultRelInfo seems pointless. Move all the fields from
PartitionRoutingInfo to ResultRelInfo.
Author: Amit Langote
Reviewed-by: Alvaro Herrera
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqFViT47Zbr_ASBejiK7iDG8%3DQ1swQ-tjM6caRPQ67pT%3Dw%40mail.gmail.com
M src/backend/commands/copy.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/backend/replication/logical/worker.c
M src/include/executor/execPartition.h
M src/include/nodes/execnodes.h
Revise child-to-root tuple conversion map management.
commit : 6973533650c04653d9018f61c1ac99ecb11094bd
author : Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 14:11:54 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 14:11:54 +0300
Store the tuple conversion map to convert a tuple from a child table's
format to the root format in a new ri_ChildToRootMap field in
ResultRelInfo. It is initialized if transition tuple capture for FOR
STATEMENT triggers or INSERT tuple routing on a partitioned table is
needed. Previously, ModifyTable kept the maps in the per-subplan
ModifyTableState->mt_per_subplan_tupconv_maps array, or when tuple
routing was used, in
ResultRelInfo->ri_Partitioninfo->pi_PartitionToRootMap. The new field
replaces both of those.
Now that the child-to-root tuple conversion map is always available in
ResultRelInfo (when needed), remove the TransitionCaptureState.tcs_map
field. The callers of Exec*Trigger() functions no longer need to set or
save it, which is much less confusing and bug-prone. Also, as a future
optimization, this will allow us to delay creating the map for a given
result relation until the relation is actually processed during
execution.
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqHtCWLdK-LO%3DNEsvOdHx%2B7yv4mE_zYK0i3BH7dXb-wxog%40mail.gmail.com
M src/backend/commands/copy.c
M src/backend/commands/trigger.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/include/commands/trigger.h
M src/include/executor/execPartition.h
M src/include/nodes/execnodes.h
Clean up code to resolve the "root target relation" in nodeModifyTable.c
commit : f49b85d783f6781138f33bbe5f6e98da86907d84
author : Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 14:11:44 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 19 Oct 2020 14:11:44 +0300
When executing DDL on a partitioned table or on a table with inheritance
children, statement-level triggers must be fired against the table given
in the original statement. The code to look that up was a bit messy and
duplicative. Commit 501ed02cf6 added a helper function,
getASTriggerResultRelInfo() (later renamed to getTargetResultRelInfo())
for it, but for some reason it was only used when firing AFTER STATEMENT
triggers and the code to fire BEFORE STATEMENT triggers duplicated the
logic.
Determine the target relation in ExecInitModifyTable(), and set it always
in ModifyTableState. Code that used to call getTargetResultRelInfo() can
now use ModifyTableState->rootResultRelInfo directly.
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqFViT47Zbr_ASBejiK7iDG8%3DQ1swQ-tjM6caRPQ67pT%3Dw%40mail.gmail.com
M src/backend/executor/nodeModifyTable.c
M src/include/nodes/execnodes.h
Avoid invalid alloc size error in shm_mq
commit : 26ec6b5948a73d0e07ed9435ee4554594acdf34f
author : Peter Eisentraut <[email protected]>
date : Mon, 19 Oct 2020 08:52:25 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 19 Oct 2020 08:52:25 +0200
In shm_mq_receive(), a huge payload could trigger an unjustified
"invalid memory alloc request size" error due to the way the buffer
size is increased.
Add error checks (documenting the upper limit) and avoid the error by
limiting the allocation size to MaxAllocSize.
Author: Markus Wanner <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/3bb363e7-ac04-0ac4-9fe8-db1148755bfa%402ndquadrant.com
M src/backend/storage/ipc/shm_mq.c
Improve whitespace
commit : afa0d53d4de6d2f43fbc0a76c88dceb22ba65e86
author : Peter Eisentraut <[email protected]>
date : Mon, 19 Oct 2020 08:32:53 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 19 Oct 2020 08:32:53 +0200
The SQL file was using a mix of tabs and spaces inconsistently.
Convert all to spaces and adjust indentation accordingly.
M src/test/regress/expected/hash_func.out
M src/test/regress/sql/hash_func.sql
Change the docs for PARALLEL option of Vacuum.
commit : 560d260d7852dc54a8c587c1b388843e8c433bc8
author : Amit Kapila <[email protected]>
date : Mon, 19 Oct 2020 09:13:17 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 19 Oct 2020 09:13:17 +0530
The rules to choose the number of parallel workers to perform parallel
vacuum operation were not clearly specified.
Reported-by: Peter Eisentraut
Author: Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/vacuum.sgml
Fix potential memory leak in pgcrypto
commit : ca2a12c935f75fb56c3b14527d6f2ff6f549ea85
author : Michael Paquier <[email protected]>
date : Mon, 19 Oct 2020 09:36:56 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 19 Oct 2020 09:36:56 +0900
When allocating a EVP context, it would have been possible to leak some
memory allocated directly by OpenSSL, that PostgreSQL lost track of if
the initialization of the context allocated failed. The cleanup can be
done with EVP_MD_CTX_destroy().
Note that EVP APIs exist since OpenSSL 0.9.7 and we have in the tree
equivalent implementations for older versions since ce9b75d (code
removed with 9b7cd59a as of 10~). However, in 9.5 and 9.6, the existing
code makes use of EVP_MD_CTX_destroy() and EVP_MD_CTX_create() without
an equivalent implementation when building the tree with OpenSSL 0.9.6
or older, meaning that this code is in reality broken with such versions
since it got introduced in e2838c5. As we have heard no complains about
that, it does not seem worth bothering with in 9.5 and 9.6, so I have
left that out for simplicity.
Author: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M contrib/pgcrypto/openssl.c
Prevent overly large and NaN row estimates in relations
commit : a90c950fc7fd8796daa8c7948e7046bceb272894
author : David Rowley <[email protected]>
date : Mon, 19 Oct 2020 10:53:52 +1300
committer: David Rowley <[email protected]>
date : Mon, 19 Oct 2020 10:53:52 +1300
Given a query with enough joins, it was possible that the query planner,
after multiplying the row estimates with the join selectivity that the
estimated number of rows would exceed the limits of the double data type
and become infinite.
To give an indication on how extreme a case is required to hit this, the
particular example case reported required 379 joins to a table without any
statistics, which resulted in the 1.0/DEFAULT_NUM_DISTINCT being used for
the join selectivity. This eventually caused the row estimates to go
infinite and resulted in an assert failure in initial_cost_mergejoin()
where the infinite row estimated was multiplied by an outerstartsel of 0.0
resulting in NaN. The failing assert verified that NaN <= Inf, which is
false.
To get around this we use clamp_row_est() to cap row estimates at a
maximum of 1e100. This value is thought to be low enough that costs
derived from it would remain within the bounds of what the double type can
represent.
Aside from fixing the failing Assert, this also has the added benefit of
making it so add_path() will still receive proper numerical values as
costs which will allow it to make more sane choices when determining the
cheaper path in extreme cases such as the one described above.
Additionally, we also get rid of the isnan() checks in the join costing
functions. The actual case which originally triggered those checks to be
added in the first place never made it to the mailing lists. It seems
likely that the new code being added to clamp_row_est() will result in
those becoming checks redundant, so just remove them.
The fairly harmless assert failure problem does also exist in the
backbranches, however, a more minimalistic fix will be applied there.
Reported-by: Onder Kalaci
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/DM6PR21MB1211FF360183BCA901B27F04D80B0@DM6PR21MB1211.namprd21.prod.outlook.com
M src/backend/optimizer/path/costsize.c
Update the Winsock API version requested by libpq.
commit : d5a9a661fcd2f5db037274157f931863a52004fd
author : Tom Lane <[email protected]>
date : Sun, 18 Oct 2020 12:56:43 -0400
committer: Tom Lane <[email protected]>
date : Sun, 18 Oct 2020 12:56:43 -0400
According to Microsoft's documentation, 2.2 has been the current
version since Windows 98 or so. Moreover, that's what the Postgres
backend has been requesting since 2004 (cf commit 4cdf51e64).
So there seems no reason for libpq to keep asking for 1.1.
Bring thread_test along, too, so that we're uniformly asking for 2.2
in all our WSAStartup calls.
It's not clear whether there's any point in back-patching this,
so for now I didn't.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-connect.c
M src/test/thread/thread_test.c
In pg_restore's dump_lo_buf(), work a little harder on error handling.
commit : 929c69aa1970b3ae30bbb5a159b9dc530ec34d5c
author : Tom Lane <[email protected]>
date : Sun, 18 Oct 2020 12:26:02 -0400
committer: Tom Lane <[email protected]>
date : Sun, 18 Oct 2020 12:26:02 -0400
Failure to write data to a large object during restore led to an ugly
and uninformative error message. To add insult to injury, it then
fatal'd out, where other SQL-level errors usually result in pressing on.
Report the underlying error condition, rather than just giving not-very-
useful byte counts, and use warn_or_exit_horribly() so as to adhere to
pg_restore's general policy about whether to continue or not.
Also recognize that lo_write() returns int not size_t.
Per report from Justin Pryzby, though I didn't use his patch.
Given the lack of comparable complaints, I'm not sure this is
worth back-patching.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_backup_archiver.c
In libpq for Windows, call WSAStartup once and WSACleanup not at all.
commit : 7d00a6b2de8ab1e95e052663064defb0bc3022f6
author : Tom Lane <[email protected]>
date : Sat, 17 Oct 2020 16:53:48 -0400
committer: Tom Lane <[email protected]>
date : Sat, 17 Oct 2020 16:53:48 -0400
The Windows documentation insists that every WSAStartup call should
have a matching WSACleanup call. However, if that ever had actual
relevance, it wasn't in this century. Every remotely-modern Windows
kernel is capable of cleaning up when a process exits without doing
that, and must be so to avoid resource leaks in case of a process
crash. Moreover, Postgres backends have done WSAStartup without
WSACleanup since commit 4cdf51e64 in 2004, and we've never seen any
indication of a problem with that.
libpq's habit of doing WSAStartup during connection start and
WSACleanup during shutdown is also rather inefficient, since a
series of non-overlapping connection requests leads to repeated,
quite expensive DLL unload/reload cycles. We document a workaround
for that (having the application call WSAStartup for itself), but
that's just a kluge. It's also worth noting that it's far from
uncommon for applications to exit without doing PQfinish, and
we've not heard reports of trouble from that either.
However, the real reason for acting on this is that recent
experiments by Alexander Lakhin suggest that calling WSACleanup
during PQfinish might be triggering the symptom we occasionally see
that a process using libpq fails to emit expected stdio output.
Therefore, let's change libpq so that it calls WSAStartup only
once per process, during the first connection attempt, and never
calls WSACleanup at all.
While at it, get rid of the only other WSACleanup call in our code
tree, in pg_dump/parallel.c; that presumably is equally useless.
If this proves to suppress the fairly-common ecpg test failures
we see on Windows, I'll back-patch, but for now let's just do it
in HEAD and see what happens.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
M src/bin/pg_dump/parallel.c
M src/interfaces/libpq/fe-connect.c
Doc: caution against misuse of 'now' and related datetime literals.
commit : 540849814cdc22ea025777d374ff6705b4d64a0f
author : Tom Lane <[email protected]>
date : Sat, 17 Oct 2020 16:02:47 -0400
committer: Tom Lane <[email protected]>
date : Sat, 17 Oct 2020 16:02:47 -0400
Section 8.5.1.4, which defines these literals, made only a vague
reference to the fact that they might be evaluated too soon to be
safe in non-interactive contexts. Provide a more explicit caution
against misuse. Also, generalize the wording in the related tip in
section 9.9.4: while it clearly described this problem, it implied
(or really, stated outright) that the problem only applies to table
DEFAULT clauses.
Per gripe from Tijs van Dam. Back-patch to all supported branches.
Discussion: https://postgr.es/m/c2LuRv9BiRT3bqIo5mMQiVraEXey_25B4vUn0kDqVqilwOEu_iVF1tbtvLnyQK7yDG3PFaz_GxLLPil2SDkj1MCObNRVaac-7j1dVdFERk8=@thalex.com
M doc/src/sgml/datatype.sgml
M doc/src/sgml/func.sgml
Update time zone data files to tzdata release 2020c.
commit : c4a803ac7e7539d5f4538e0e677bb543f2a5eaa0
author : Tom Lane <[email protected]>
date : Fri, 16 Oct 2020 21:53:33 -0400
committer: Tom Lane <[email protected]>
date : Fri, 16 Oct 2020 21:53:33 -0400
DST law changes in Morocco, Canadian Yukon, Fiji, Macquarie Island,
Casey Station (Antarctica). Historical corrections for France,
Hungary, Monaco.
M src/timezone/data/tzdata.zi
Sync our copy of the timezone library with IANA release tzcode2020c.
commit : ce0e97f808683dc1732880ec3196443875efa262
author : Tom Lane <[email protected]>
date : Fri, 16 Oct 2020 21:40:16 -0400
committer: Tom Lane <[email protected]>
date : Fri, 16 Oct 2020 21:40:16 -0400
This changes zic's default output format from "-b fat" to "-b slim".
We were already using "slim" in v13/HEAD, so those branches drop
the explicit -b switch in the Makefiles. Instead, add an explicit
"-b fat" in v12 and before, so that we don't change the output file
format in those branches. (This is perhaps excessively conservative,
but we decided not to do so in a12079109, and I'll stick with that.)
Other non-cosmetic changes are to drop support for zic's long-obsolete
"-y" switch, and to ensure that strftime() does not change errno
unless it fails.
As usual with tzcode changes, back-patch to all supported branches.
M src/timezone/Makefile
M src/timezone/README
M src/timezone/strftime.c
M src/timezone/zic.c
M src/tools/msvc/Install.pm
Add missing error check in pgcrypto/crypt-md5.c.
commit : 02a75f8369b1b14b709349ee60b29f79b70320f6
author : Tom Lane <[email protected]>
date : Fri, 16 Oct 2020 11:59:13 -0400
committer: Tom Lane <[email protected]>
date : Fri, 16 Oct 2020 11:59:13 -0400
In theory, the second px_find_digest call in px_crypt_md5 could fail
even though the first one succeeded, since resource allocation is
required. Don't skip testing for a failure. (If one did happen,
the likely result would be a crash rather than clean recovery from
an OOM failure.)
The code's been like this all along, so back-patch to all supported
branches.
Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M contrib/pgcrypto/crypt-md5.c
Doc: tweak column widths in synchronous-commit-matrix table.
commit : bc49f8780b361591141e35c03ffb17d4035847cb
author : Tom Lane <[email protected]>
date : Fri, 16 Oct 2020 11:36:34 -0400
committer: Tom Lane <[email protected]>
date : Fri, 16 Oct 2020 11:36:34 -0400
Commit a97e85f2b caused "exceed the available area" warnings in PDF
builds. Fine-tune colwidth values to avoid that.
Back-patch to 9.6, like the prior patch. (This is of dubious value
before v13, since we were far from free of such warnings in older
branches. But we might as well keep the SGML looking the same in all
branches.)
Per buildfarm.
M doc/src/sgml/config.sgml
postgres_fdw: Restructure connection retry logic.
commit : 7fc1a81e4982b6722405b032e21be38dbbef5b80
author : Fujii Masao <[email protected]>
date : Fri, 16 Oct 2020 13:58:45 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 16 Oct 2020 13:58:45 +0900
Commit 32a9c0bdf introduced connection retry logic into postgres_fdw.
Previously it used goto statement for retry. This commit gets rid of that
goto from the logic to make the code simpler and easier-to-read.
When getting out of PG_CATCH() for the retry, the error state should be
cleaned up and the memory context should be reset. But commit 32a9c0bdf
forgot to do that. This commit also fixes this bug.
Previously only PQstatus()==CONNECTION_BAD was verified to detect
connection failure. But this could cause false detection in the case where
any error other than connection failure (e.g., out-of-memory) was thrown
after a broken connection was detected in libpq and CONNECTION_BAD is set.
To fix this issue, this commit changes the logic so that it also checks
the error's sqlstate is ERRCODE_CONNECTION_FAILURE.
Author: Fujii Masao
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/connection.c
llvmjit: Work around bug in LLVM 3.9 causing crashes after 72559438f92.
commit : fe2a16d8b3e6fde07a8a990057e983854ad50616
author : Andres Freund <[email protected]>
date : Thu, 15 Oct 2020 17:38:00 -0700
committer: Andres Freund <[email protected]>
date : Thu, 15 Oct 2020 17:38:00 -0700
Unfortunately in LLVM 3.9 LLVMGetAttributeCountAtIndex(func, index)
crashes when called with an index that has 0 attributes. Since there's
no way to work around this in the C API, add a small C++ wrapper doing
so.
The only reason this didn't fail before 72559438f92 is that there
always are function attributes...
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch: 11-, like 72559438f92
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_wrap.cpp
M src/include/jit/llvmjit.h
pg_upgrade: remove C99 compiler req. from commit 3c0471b5fd
commit : 536de14e2bdf467aefc5e25f85056fe1adf4846d
author : Bruce Momjian <[email protected]>
date : Thu, 15 Oct 2020 20:37:20 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 15 Oct 2020 20:37:20 -0400
This commit required support for inline variable definition, which is
not a requirement.
RELEASE NOTE AUTHOR: the author of commit 3c0471b5fd
(pg_upgrade/tablespaces) was Justin Pryzby, not me.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M src/bin/pg_upgrade/check.c
pg_upgrade: generate check error for left-over new tablespace
commit : 3c0471b5fdd847febb30ec805a3e44f3eac5b66f
author : Bruce Momjian <[email protected]>
date : Thu, 15 Oct 2020 19:33:36 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 15 Oct 2020 19:33:36 -0400
Previously, if pg_upgrade failed, and the user recreated the cluster but
did not remove the new cluster tablespace directory, a later pg_upgrade
would fail since the new tablespace directory would already exists.
This adds error reporting for this during check.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M src/bin/pg_upgrade/check.c
llvmjit: Also copy parameter / return value attributes from template functions.
commit : 72559438f92f9a7d9205192bef601be953c7bd2a
author : Andres Freund <[email protected]>
date : Thu, 15 Oct 2020 13:39:41 -0700
committer: Andres Freund <[email protected]>
date : Thu, 15 Oct 2020 13:39:41 -0700
Previously we only copied the function attributes. That caused problems at
least on s390x: Because we didn't copy the 'zeroext' attribute for
ExecAggTransReparent()'s *IsNull parameters, expressions invoking it didn't
ensure that the upper bytes of the registers were zeroed. In the - relatively
rare - cases where not, ExecAggTransReparent() wrongly ended up in the
newValueIsNull branch due to the register not being zero. Subsequently causing
a crash.
It's quite possible that this would cause problems on other platforms, and in
other places than just ExecAggTransReparent() on s390x.
Thanks to Christoph (and the Debian project) for providing me with access to a
s390x machine, allowing me to debug this.
Reported-By: Christoph Berg
Author: Andres Freund
Discussion: https://postgr.es/m/[email protected]
Backpatch: 11-, where JIT was added
M src/backend/jit/llvm/llvmjit.c
doc: improve description of synchronous_commit modes
commit : a97e85f2be400a08d863c1d65f172eb48af3e9b5
author : Bruce Momjian <[email protected]>
date : Thu, 15 Oct 2020 15:15:29 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 15 Oct 2020 15:15:29 -0400
Previously it wasn't clear exactly what each of the synchronous_commit
modes accomplished. This clarifies that, and adds a table describing it.
Only backpatched through 9.6 since 9.5 doesn't have all the options.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M doc/src/sgml/config.sgml
M src/include/access/xact.h
Revert "Remove pointless HeapTupleHeaderIndicatesMovedPartitions calls"
commit : 93f84d59f80d11a3d8ade9ae71560162d6f3ecb2
author : Alvaro Herrera <[email protected]>
date : Thu, 15 Oct 2020 15:16:11 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 15 Oct 2020 15:16:11 -0300
This reverts commit 85adb5e91ec2. It was not intended for commit just
yet.
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_visibility.c
Remove pointless HeapTupleHeaderIndicatesMovedPartitions calls
commit : 85adb5e91ec2f45a388bef7a92a3d988c7e45501
author : Alvaro Herrera <[email protected]>
date : Thu, 15 Oct 2020 14:32:34 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 15 Oct 2020 14:32:34 -0300
Pavan Deolasee recently noted that a few of the
HeapTupleHeaderIndicatesMovedPartitions calls added by commit
5db6df0c0117 are useless, since they are done after comparing t_self
with t_ctid. But because t_self can never be set to the magical values
that indicate that the tuple moved partition, this can never succeed: if
the first test fails (so we know t_self equals t_ctid), necessarily the
second test will also fail.
So these checks can be removed and no harm is done.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_visibility.c
Install pg_isolation_regress and isolationtester
commit : 2203ede9ae85b6423f850466122606275ea09b17
author : Alvaro Herrera <[email protected]>
date : Thu, 15 Oct 2020 13:09:29 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 15 Oct 2020 13:09:29 -0300
We already install assorted tools for testing extensions, but these two
were missing. Having them installed, and after ISOLATION support was
added to PGXS's makefiles by d3c09b9b1307, helps third-party modules
usefully include isolation tests. Compare c3a0818460a8.
Author: Craig Ringer <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/CAMsr+YFsCMH3B4uOPFE+2qWM6k=o=hf9LGiPNCfwqKdUPz_BsQ@mail.gmail.com
M src/test/isolation/Makefile
Review logical replication tablesync code
commit : b05fe7b442fd185b9705c537adb602b3b9d49518
author : Alvaro Herrera <[email protected]>
date : Thu, 15 Oct 2020 11:33:48 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 15 Oct 2020 11:33:48 -0300
Most importantly, remove optimization in LogicalRepSyncTableStart that
skips the normal walrcv_startstreaming/endstreaming dance. The
optimization is not critically important for production uses anyway,
since it only fires in cases with no activity, and saves an
uninteresting amount of work even then. Critically, it obscures bugs by
hiding the interesting code path from test cases.
Also: in GetSubscriptionRelState, remove pointless relation open; access
pg_subscription_rel->srsubstate with GETSTRUCT as is typical rather than
SysCacheGetAttr; remove unused 'missing_ok' argument.
In wait_for_relation_state_change, use explicit catalog snapshot
invalidation rather than obscurely (and expensively) through
GetLatestSnapshot.
In various places: sprinkle comments more liberally and rewrite a number
of them. Other cosmetic code improvements.
No backpatch, since no bug is being fixed here.
Author: Álvaro Herrera <[email protected]>
Reviewed-by: Petr Jelínek <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/pg_subscription.c
M src/backend/replication/logical/relation.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/include/catalog/pg_subscription_rel.h
Refactor code for cross-partition updates to a separate function.
commit : c5b097f8fa69861ab83470c0bfe732ab90df9e62
author : Heikki Linnakangas <[email protected]>
date : Thu, 15 Oct 2020 17:08:10 +0300
committer: Heikki Linnakangas <[email protected]>
date : Thu, 15 Oct 2020 17:08:10 +0300
ExecUpdate() is very long, so extract the part of it that deals with
cross-partition updates to a separate function to make it more readable.
Per Andres Freund's suggestion.
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqEUgb5RdUgxR7Sqco4S09jzJstHiaT2vnCRPGR4JCAPqA%40mail.gmail.com
M src/backend/executor/nodeModifyTable.c
Fix query in new test to check tables are synced
commit : 7f4708818317f59a0452f549dedf4ce35182c13f
author : Alvaro Herrera <[email protected]>
date : Thu, 15 Oct 2020 09:48:36 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 15 Oct 2020 09:48:36 -0300
Rather than looking for tablesync workers, it is more reliable to see
the sync state of the tables.
Per note from Amit Kapila.
Discussion: https://postgr.es/m/CAA4eK1JSSD7FVwq+_rOme86jUZTQFzjsNU06hQ4-LiRt1xFmSg@mail.gmail.com
M src/test/subscription/t/100_bugs.pl
Replace calls of htonl()/ntohl() with pg_bswap.h for GSSAPI encryption
commit : 86dba33217bb0b48fcf59da76912c3382b4418cd
author : Michael Paquier <[email protected]>
date : Thu, 15 Oct 2020 17:03:56 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 15 Oct 2020 17:03:56 +0900
The in-core equivalents can make use of built-in functions if the
compiler supports this option, making optimizations possible. 0ba99c8
replaced all existing calls in the code base at this time, but b0b39f7
(GSSAPI encryption) has forgotten to do the switch.
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/be-secure-gssapi.c
M src/interfaces/libpq/fe-secure-gssapi.c
Improve tab-completion for FETCH/MOVE.
commit : 8176afd8b764e54fc6e34240928effc661fc40cd
author : Fujii Masao <[email protected]>
date : Thu, 15 Oct 2020 16:50:57 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 15 Oct 2020 16:50:57 +0900
Author: Naoki Nakamichi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/tab-complete.c
Fixup some appendStringInfo and appendPQExpBuffer calls
commit : 110d81728a0a006abcf654543fc15346f8043dc0
author : David Rowley <[email protected]>
date : Thu, 15 Oct 2020 20:35:17 +1300
committer: David Rowley <[email protected]>
date : Thu, 15 Oct 2020 20:35:17 +1300
A number of places were using appendStringInfo() when they could have been
using appendStringInfoString() instead. While there's no functionality
change there, it's just more efficient to use appendStringInfoString()
when no formatting is required. Likewise for some
appendStringInfoString() calls which were just appending a single char.
We can just use appendStringInfoChar() for that.
Additionally, many places were using appendPQExpBuffer() when they could
have used appendPQExpBufferStr(). Change those too.
Patch by Zhijie Hou, but further searching by me found significantly more
places that deserved the same treatment.
Author: Zhijie Hou, David Rowley
Discussion: https://postgr.es/m/cb172cf4361e4c7ba7167429070979d4@G08CNEXMBPEKD05.g08.fujitsu.local
M contrib/postgres_fdw/postgres_fdw.c
M contrib/test_decoding/test_decoding.c
M src/backend/access/rmgrdesc/dbasedesc.c
M src/backend/commands/explain.c
M src/backend/replication/backup_manifest.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/tablesync.c
M src/backend/utils/adt/jsonpath.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/adt/ruleutils.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_upgrade/version.c
M src/bin/psql/describe.c
M src/bin/scripts/reindexdb.c
M src/pl/plpython/plpy_elog.c
Add documentation link to attributes supported by Clang
commit : 73c381cee71e7dd8a78a394731958b0bbb4d8991
author : Peter Eisentraut <[email protected]>
date : Thu, 15 Oct 2020 08:54:16 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 15 Oct 2020 08:54:16 +0200
M src/include/c.h
Handle EACCES errors from kevent() better.
commit : 70516a178ad0fc54d0f0d2a88175af3e1e92f83f
author : Thomas Munro <[email protected]>
date : Thu, 15 Oct 2020 18:23:30 +1300
committer: Thomas Munro <[email protected]>
date : Thu, 15 Oct 2020 18:23:30 +1300
While registering for postmaster exit events, we have to handle a couple
of edge cases where the postmaster is already gone. Commit 815c2f09
missed one: EACCES must surely imply that PostmasterPid no longer
belongs to our postmaster process (or alternatively an unexpected
permissions model has been imposed on us). Like ESRCH, this should be
treated as a WL_POSTMASTER_DEATH event, rather than being raised with
ereport().
No known problems reported in the wild. Per code review from Tom Lane.
Back-patch to 13.
Reported-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/3624029.1602701929%40sss.pgh.pa.us
M src/backend/storage/ipc/latch.c
Execute invalidation messages for each XLOG_XACT_INVALIDATIONS message during logical decoding.
commit : d7eb52d7181d83cf2363570f7a205b8eb1008dbc
author : Amit Kapila <[email protected]>
date : Thu, 15 Oct 2020 08:17:51 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 15 Oct 2020 08:17:51 +0530
Prior to commit c55040ccd0 we have no way of knowing the invalidations
before commit. So, while decoding we use to execute all the invalidations
at each command end as we had no way of knowing which invalidations
happened before that command. Due to this, transactions involving large
amounts of DDLs use to take more time and also lead to high CPU usage. But
now we know specific invalidations at each command end so we execute only
required invalidations.
It has been observed that decoding of a transaction containing truncation
of a table with 1000 partitions would be finished in 1s whereas before
this patch it used to take 4-5 minutes.
Author: Dilip Kumar
Reviewed-by: Amit Kapila and Keisuke Kuroda
Discussion: https://postgr.es/m/CANDwggKYveEtXjXjqHA6RL3AKSHMsQyfRY6bK+NqhAWJyw8psQ@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h
doc: Mention that toast_tuple_target affects also column marked as Main.
commit : 564a410c81b4a9d289c948690a3135d0cd25411b
author : Fujii Masao <[email protected]>
date : Thu, 15 Oct 2020 11:04:07 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 15 Oct 2020 11:04:07 +0900
Previously it was documented that toast_tuple_target affected column
marked as only External or Extended. But this description is not correct
and toast_tuple_target affects also column marked as Main.
Back-patch to v11 where toast_tuple_target reloption was introduced.
Author: Shinya Okano
Reviewed-by: Tatsuhito Kasahara, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_table.sgml
Restore replication protocol's duplicate command tags
commit : 4e9821b6fac5042e872d5397f711a67984b165f8
author : Alvaro Herrera <[email protected]>
date : Wed, 14 Oct 2020 20:12:26 -0300
committer: Alvaro Herrera <[email protected]>
date : Wed, 14 Oct 2020 20:12:26 -0300
I removed the duplicate command tags for START_REPLICATION inadvertently
in commit 07082b08cc5d, but the replication protocol requires them. The
fact that the replication protocol was broken was not noticed because
all our test cases use an optimized code path that exits early, failing
to verify that the behavior is correct for non-optimized cases. Put
them back.
Also document this protocol quirk.
Add a test case that shows the failure. It might still succeed even
without the patch when run on a fast enough server, but it suffices to
show the bug in enough cases that it would be noticed in buildfarm.
Author: Álvaro Herrera <[email protected]>
Reported-by: Henry Hinze <[email protected]>
Reviewed-by: Petr Jelínek <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/protocol.sgml
M src/backend/replication/logical/worker.c
M src/backend/replication/walsender.c
M src/test/subscription/t/100_bugs.pl
Make WL_POSTMASTER_DEATH level-triggered on kqueue builds.
commit : b94109ce375b137f235149bfba3559c69f4573e7
author : Thomas Munro <[email protected]>
date : Thu, 15 Oct 2020 11:31:20 +1300
committer: Thomas Munro <[email protected]>
date : Thu, 15 Oct 2020 11:31:20 +1300
If WaitEventSetWait() reports that the postmaster has gone away, later
calls to WaitEventSetWait() should continue to report that. Otherwise
further waits that occur in the proc_exit() path after we already
noticed the postmaster's demise could block forever.
Back-patch to 13, where the kqueue support landed.
Reported-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/3624029.1602701929%40sss.pgh.pa.us
M src/backend/storage/ipc/latch.c
Remove es_result_relation_info from EState.
commit : a04daa97a4339c38e304cd6164d37da540d665a8
author : Heikki Linnakangas <[email protected]>
date : Wed, 14 Oct 2020 11:41:40 +0300
committer: Heikki Linnakangas <[email protected]>
date : Wed, 14 Oct 2020 11:41:40 +0300
Maintaining 'es_result_relation_info' correctly at all times has become
cumbersome, especially with partitioning where each partition gets its
own result relation info. Having to set and reset it across arbitrary
operations has caused bugs in the past.
This changes all the places that used 'es_result_relation_info', to
receive the currently active ResultRelInfo via function parameters
instead.
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqGEmiib8FLiHMhKB%2BCH5dRgHSLc5N5wnvc4kym%2BZYpQEQ%40mail.gmail.com
M src/backend/commands/copy.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execIndexing.c
M src/backend/executor/execMain.c
M src/backend/executor/execReplication.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeModifyTable.c
M src/backend/replication/logical/worker.c
M src/include/executor/executor.h
M src/include/executor/nodeModifyTable.h
M src/include/nodes/execnodes.h
M src/test/regress/expected/insert.out
M src/test/regress/sql/insert.sql
Include result relation info in direct modify ForeignScan nodes.
commit : 178f2d560dde05b356ab2f586b8bc62514f454aa
author : Heikki Linnakangas <[email protected]>
date : Wed, 14 Oct 2020 10:58:38 +0300
committer: Heikki Linnakangas <[email protected]>
date : Wed, 14 Oct 2020 10:58:38 +0300
FDWs that can perform an UPDATE/DELETE remotely using the "direct modify"
set of APIs need to access the ResultRelInfo of the target table. That's
currently available in EState.es_result_relation_info, but the next
commit will remove that field.
This commit adds a new resultRelation field in ForeignScan, to store the
target relation's RT index, and the corresponding ResultRelInfo in
ForeignScanState. The FDW's PlanDirectModify callback is expected to set
'resultRelation' along with 'operation'. The core code doesn't need them
for anything, they are for the convenience of FDW's Begin- and
IterateDirectModify callbacks.
Authors: Amit Langote, Etsuro Fujita
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqGEmiib8FLiHMhKB%2BCH5dRgHSLc5N5wnvc4kym%2BZYpQEQ%40mail.gmail.com
M contrib/postgres_fdw/postgres_fdw.c
M doc/src/sgml/fdwhandler.sgml
M src/backend/executor/nodeForeignscan.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/setrefs.c
M src/include/nodes/execnodes.h
M src/include/nodes/plannodes.h
Use https for gnu.org links
commit : 39b4a951003a6545268e141272e123929d0d710f
author : Peter Eisentraut <[email protected]>
date : Wed, 14 Oct 2020 08:24:54 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 14 Oct 2020 08:24:54 +0200
Mostly already done, but there were some stragglers.
M doc/src/sgml/installation.sgml
M src/include/c.h
M src/include/port/atomics/generic-gcc.h
M src/interfaces/ecpg/Makefile
M src/tools/RELEASE_CHANGES
Correct error message
commit : 4e118fc33e3ca5244c11a81a71bd25cf9ed3d484
author : Peter Eisentraut <[email protected]>
date : Wed, 14 Oct 2020 07:54:14 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 14 Oct 2020 07:54:14 +0200
Apparently copy-and-pasted from nearby.
M src/backend/parser/parse_target.c
Paper over regression failures in infinite_recurse() on PPC64 Linux.
commit : ae0f7b11f143d9748e2201c3647330893c4c1c5a
author : Tom Lane <[email protected]>
date : Tue, 13 Oct 2020 17:44:56 -0400
committer: Tom Lane <[email protected]>
date : Tue, 13 Oct 2020 17:44:56 -0400
Our infinite_recurse() test to verify sane stack-overrun behavior
is affected by a bug of the Linux kernel on PPC64: it will get SIGSEGV
if it receives a signal when the stack depth is (a) over 1MB and
(b) within a few kB of filling the current physical stack allocation.
See https://bugzilla.kernel.org/show_bug.cgi?id=205183.
Since this test is a bit time-consuming and we run it in parallel with
test scripts that do a lot of DDL, it can be expected to get an sinval
catchup interrupt at some point, leading to failure if the timing is
wrong. This has caused more than 100 buildfarm failures over the
past year or so.
While a fix exists for the kernel bug, it might be years before that
propagates into all production kernels, particularly in some of the
older distros we have in the buildfarm. For now, let's just back off
and not run this test on Linux PPC64; that loses nothing in test
coverage so far as our own code is concerned.
To do that, split this test into a new script infinite_recurse.sql
and skip the test when the platform name is powerpc64...-linux-gnu.
Back-patch to v12. Branches before that have not been seen to get
this failure. No doubt that's because the "errors" test was not
run in parallel with other tests before commit 798070ec0, greatly
reducing the odds of an sinval catchup being necessary.
I also back-patched 3c8553547 into v12, just so the new regression
script would look the same in all branches having it.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/20190723162703.GM22387%40telsasoft.com
M src/test/regress/expected/errors.out
A src/test/regress/expected/infinite_recurse.out
A src/test/regress/expected/infinite_recurse_1.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
M src/test/regress/sql/errors.sql
A src/test/regress/sql/infinite_recurse.sql
Create ResultRelInfos later in InitPlan, index them by RT index.
commit : 1375422c7826a2bf387be29895e961614f69de4b
author : Heikki Linnakangas <[email protected]>
date : Tue, 13 Oct 2020 12:57:02 +0300
committer: Heikki Linnakangas <[email protected]>
date : Tue, 13 Oct 2020 12:57:02 +0300
Instead of allocating all the ResultRelInfos upfront in one big array,
allocate them in ExecInitModifyTable(). es_result_relations is now an
array of ResultRelInfo pointers, rather than an array of structs, and it
is indexed by the RT index.
This simplifies things: we get rid of the separate concept of a "result
rel index", and don't need to set it in setrefs.c anymore. This also
allows follow-up optimizations (not included in this commit yet) to skip
initializing ResultRelInfos for target relations that were not needed at
runtime, and removal of the es_result_relation_info pointer.
The EState arrays of regular result rels and root result rels are merged
into one array. Similarly, the resultRelations and rootResultRelations
lists in PlannedStmt are merged into one. It's not actually clear to me
why they were kept separate in the first place, but now that the
es_result_relations array is indexed by RT index, it certainly seems
pointless.
The PlannedStmt->resultRelations list is now only needed for
ExecRelationIsTargetRelation(). One visible effect of this change is that
ExecRelationIsTargetRelation() will now return 'true' also for the
partition root, if a partitioned table is updated. That seems like a good
thing, although the function isn't used in core code, and I don't see any
reason for an FDW to call it on a partition root.
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqGEmiib8FLiHMhKB%2BCH5dRgHSLc5N5wnvc4kym%2BZYpQEQ%40mail.gmail.com
M src/backend/commands/copy.c
M src/backend/commands/explain.c
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/backend/executor/execMain.c
M src/backend/executor/execParallel.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeModifyTable.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/nodes/readfuncs.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/replication/logical/worker.c
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/plannodes.h
Fix the unstable output of tests added by commit 8fccf75834.
commit : 2050832d0d637358a376a99514071941aa93ed31
author : Amit Kapila <[email protected]>
date : Tue, 13 Oct 2020 12:46:38 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 13 Oct 2020 12:46:38 +0530
The test cases added by that commit were trying to test the exact number of
times a particular transaction has spilled. However, that number can vary if
any background transaction (say by autovacuum) happens in parallel to the main
transaction. So let's not try to verify the exact count.
Author: Amit Kapila
Reviewed-by: Sawada Masahiko
Discussion: https://postgr.es/m/CA+fd4k5_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com
M contrib/test_decoding/expected/stats.out
M contrib/test_decoding/sql/stats.sql
doc: Expand recursive query documentation
commit : 323ae003e464cac35f86790f3708b383c18df9e2
author : Peter Eisentraut <[email protected]>
date : Tue, 13 Oct 2020 06:29:06 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 13 Oct 2020 06:29:06 +0200
Break the section up with subsection headings. Add examples for depth-
and breadth-first search ordering. For consistency with the SQL
search clause, start the depth counting at 0 instead of 1 in the
examples.
Discussion: https://www.postgresql.org/message-id/[email protected]
M doc/src/sgml/queries.sgml
Add tests for logical replication spilled stats.
commit : 8fccf75834c11798e3b13a4cebedfb8ae683857a
author : Amit Kapila <[email protected]>
date : Tue, 13 Oct 2020 08:30:35 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 13 Oct 2020 08:30:35 +0530
Commit 9868167500 added a mechanism to track statistics corresponding to
the spilling of changes from ReorderBuffer but didn't add any tests.
Author: Amit Kapila and Sawada Masahiko
Discussion: https://postgr.es/m/CA+fd4k5_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com
M contrib/test_decoding/Makefile
A contrib/test_decoding/expected/stats.out
A contrib/test_decoding/sql/stats.sql
Fix GiST buffering build to work when there are included columns.
commit : 371668a8389d792279bd7aff8ca796e284f595df
author : Tom Lane <[email protected]>
date : Mon, 12 Oct 2020 18:01:34 -0400
committer: Tom Lane <[email protected]>
date : Mon, 12 Oct 2020 18:01:34 -0400
gistRelocateBuildBuffersOnSplit did not get the memo about which
attribute count to use. This could lead to a crash if there were
included columns and buffering build was chosen. (Because there
are random page-split decisions elsewhere in GiST index build,
the crashes are not entirely deterministic.)
Back-patch to v12 where GiST gained support for included columns.
Pavel Borisov
Discussion: https://postgr.es/m/CALT9ZEECCV5m7wvxg46PC-7x-EybUmnpupBGhSFMoAAay+r6HQ@mail.gmail.com
M src/backend/access/gist/gistbuildbuffers.c
Re-allow testing of GiST buffered builds.
commit : 78c0b6ed273a1262f96efe94004bc92d99865005
author : Tom Lane <[email protected]>
date : Mon, 12 Oct 2020 17:09:50 -0400
committer: Tom Lane <[email protected]>
date : Mon, 12 Oct 2020 17:09:50 -0400
Commit 16fa9b2b3 broke the ability to reliably test GiST buffered builds,
because it caused sorted builds to be done instead if sortsupport is
available, regardless of any attempt to override that. While a would-be
test case could try to work around that by choosing an opclass that has
no sortsupport function, coverage would be silently lost the moment
someone decides it'd be a good idea to add a sortsupport function.
Hence, rearrange the logic in gistbuild() so that if "buffering = on"
is specified in CREATE INDEX, we will use that method, sortsupport or no.
Also document the interaction between sorting and the buffering
parameter, as 16fa9b2b3 failed to do.
(Note that in fact we still lack any test coverage of buffered builds,
but this is a prerequisite to adding a non-fragile test.)
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/gist.sgml
M doc/src/sgml/ref/create_index.sgml
M src/backend/access/gist/gistbuild.c
Fix memory leak when guc.c decides a setting can't be applied now.
commit : 397ea901e85b83e6381a0edeba7a45d794063569
author : Tom Lane <[email protected]>
date : Mon, 12 Oct 2020 13:31:24 -0400
committer: Tom Lane <[email protected]>
date : Mon, 12 Oct 2020 13:31:24 -0400
The prohibitValueChange code paths in set_config_option(), which
are executed whenever we re-read a PGC_POSTMASTER variable from
postgresql.conf, neglected to free anything before exiting. Thus
we'd leak the proposed new value of a PGC_STRING variable, as noted
by BoChen in bug #16666. For all variable types, if the check hook
creates an "extra" chunk, we'd also leak that.
These are malloc not palloc chunks, so there is no mechanism for
recovering the leaks before process exit. Fortunately, the values
are typically not very large, meaning you'd have to go through an
awful lot of SIGHUP configuration-reload cycles to make the leakage
amount to anything. Still, for a long-lived postmaster process it
could potentially be a problem.
Oversight in commit 2594cf0e8. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/misc/guc.c
Minor cleanup for win32stat.c.
commit : fcd11329db5bca9993207f099a642a7d1cd59ff8
author : Tom Lane <[email protected]>
date : Mon, 12 Oct 2020 11:13:02 -0400
committer: Tom Lane <[email protected]>
date : Mon, 12 Oct 2020 11:13:02 -0400
Use GetLastError(), rather than assuming that CreateFile() failure
must map to ENOENT. Noted by Michael Paquier.
Discussion: https://postgr.es/m/CAC+AXB0g44SbvSpC86o_1HWh8TAU2pZrMRW6tJT-dkijotx5Qg@mail.gmail.com
M src/port/win32stat.c
Fix compilation warning in unicode_norm.c
commit : e578c17d81662b4f4f63a2797bc1be64af3c8f93
author : Michael Paquier <[email protected]>
date : Mon, 12 Oct 2020 20:34:55 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 12 Oct 2020 20:34:55 +0900
80f8eb7 has introduced in unicode_norm.c some new code that uses
htonl(). On at least some FreeBSD environments, it is possible to find
that this function is undeclared, causing a compilation warning. It is
worth noting that no buildfarm members have reported this issue.
Instead of adding a new inclusion to arpa/inet.h, switch to use
the equivalent defined in pg_bswap.h, to benefit from any built-in
function if the compiler has one.
Reported-by: Masahiko Sawada
Discussion: https://postgr.es/m/CA+fd4k7D4b12ShywWj=AbcHZzV1-OqMjNe7RZAu+tgz5rd_11A@mail.gmail.com
M src/common/unicode_norm.c
Fix estimates for ModifyTable paths without RETURNING.
commit : f0f13a3a08b2757997410f3a1c38bdc22973c525
author : Thomas Munro <[email protected]>
date : Mon, 12 Oct 2020 20:41:16 +1300
committer: Thomas Munro <[email protected]>
date : Mon, 12 Oct 2020 20:41:16 +1300
In the past, we always estimated that a ModifyTable node would emit the
same number of rows as its subpaths. Without a RETURNING clause, the
correct estimate is zero. Fix, in preparation for a proposed parallel
write patch that is sensitive to that number.
A remaining problem is that for RETURNING queries, the estimated width
is based on subpath output rather than the RETURNING tlist.
Reviewed-by: Greg Nancarrow <[email protected]>
Discussion: https://postgr.es/m/CAJcOf-cXnB5cnMKqWEp2E2z7Mvcd04iLVmV%3DqpFJrR3AcrTS3g%40mail.gmail.com
M src/backend/optimizer/util/pathnode.c
Adjust cycle detection examples and tests
commit : 3fb676504da9c019540c7384423c7e3d7d394110
author : Peter Eisentraut <[email protected]>
date : Mon, 12 Oct 2020 07:46:20 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 12 Oct 2020 07:46:20 +0200
Adjust the existing cycle detection example and test queries to put
the cycle column before the path column. This is mainly because the
SQL-standard CYCLE clause puts them in that order, and so if we added
that feature that would make the sequence of examples more consistent
and easier to follow.
Discussion: https://www.postgresql.org/message-id/[email protected]
M doc/src/sgml/queries.sgml
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Choose ppc compare_exchange constant path for more operand values.
commit : 88ea7a1188d1afb25695124045e0ff81870f55e3
author : Noah Misch <[email protected]>
date : Sun, 11 Oct 2020 21:31:37 -0700
committer: Noah Misch <[email protected]>
date : Sun, 11 Oct 2020 21:31:37 -0700
The implementation uses smaller code when the "expected" operand is a
small constant, but the implementation needlessly defined the set of
acceptable constants more narrowly than the ABI does. Core PostgreSQL
and PGXN don't use the constant path at all, so this is future-proofing.
Back-patch to v13, where commit 30ee5d17c20dbb282a9952b3048d6ad52d56c371
introduced this code.
Reviewed by Tom Lane. Reported by Christoph Berg.
Discussion: https://postgr.es/m/[email protected]
M src/include/port/atomics/arch-ppc.h
M src/test/regress/regress.c
For ppc gcc, implement 64-bit compare_exchange and fetch_add with asm.
commit : f5c1167b173d6e7a5d4c938fe716f0d29ae7228d
author : Noah Misch <[email protected]>
date : Sun, 11 Oct 2020 21:31:37 -0700
committer: Noah Misch <[email protected]>
date : Sun, 11 Oct 2020 21:31:37 -0700
While xlc defines __64BIT__, gcc does not. Due to this oversight in
commit 30ee5d17c20dbb282a9952b3048d6ad52d56c371, gcc builds continued
implementing 64-bit atomics by way of intrinsics. Back-patch to v13,
where that commit first appeared.
Reviewed by Tom Lane.
Discussion: https://postgr.es/m/[email protected]
M src/include/port/atomics/arch-ppc.h
Use perfect hash for NFC and NFKC Unicode Normalization quick check
commit : 80f8eb79e24d9b7963eaf17ce846667e2c6b6e6f
author : Michael Paquier <[email protected]>
date : Sun, 11 Oct 2020 19:09:01 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 11 Oct 2020 19:09:01 +0900
This makes the normalization quick check about 30% faster for NFC and
50% faster for NFKC than the binary search used previously. The hash
lookup reuses the existing array of bit fields used for the binary
search to get the quick check property and is generated as part of "make
update-unicode" in src/common/unicode/.
Author: John Naylor
Reviewed-by: Mark Dilger, Michael Paquier
Discussion: https://postgr.es/m/CACPNZCt4fbJ0_bGrN5QPt34N4whv=mszM0LMVQdoa2rC9UMRXA@mail.gmail.com
M src/common/unicode/generate-unicode_normprops_table.pl
M src/common/unicode_norm.c
M src/include/common/unicode_normprops_table.h
M src/tools/pgindent/exclude_file_patterns
M src/tools/pgindent/typedefs.list
Band-aid new postgres_fdw test case to remove error text dependency.
commit : 85d08b8b721fb3b9359bca9325bc425cc95c30b1
author : Tom Lane <[email protected]>
date : Sat, 10 Oct 2020 19:57:25 -0400
committer: Tom Lane <[email protected]>
date : Sat, 10 Oct 2020 19:57:25 -0400
Buildfarm member lorikeet is still failing the test from commit
32a9c0bdf, but now it's down to the should-have-foreseen-it problem
that the error message isn't what the expected-output file expects.
Let's see if we can get stable results by printing just the SQLSTATE.
I believe we'll reliably see ERRCODE_CONNECTION_FAILURE, since
pgfdw_report_error() will report that for any libpq-originated error.
There may be a better way to do this, but I'd like to get the
buildfarm back to green before we discuss further improvements.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Remove pointless error-code checking in pg_dump/parallel.c.
commit : eeb01eb1f560d90625ac4c8ee210f29421ba4f0d
author : Tom Lane <[email protected]>
date : Sat, 10 Oct 2020 15:33:54 -0400
committer: Tom Lane <[email protected]>
date : Sat, 10 Oct 2020 15:33:54 -0400
Commit fe27009cb tried to make parallel.c's Windows implementation of
piperead() translate Windows socket errors to Unix, but that didn't
actually work because TranslateSocketError() is backend-internal code
(and not even public there). But on closer inspection, the sole
caller of this function doesn't actually care whether the result is
zero or negative, much less inspect the errno. So the whole exercise
is totally useless, and has been since this code was introduced.
Rip it out and just call recv() directly.
Per buildfarm.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/parallel.c
Minor cleanup for win32stat.c.
commit : 961e07b8ccb56cb3979185b066b503b3d4f7c036
author : Tom Lane <[email protected]>
date : Sat, 10 Oct 2020 14:53:23 -0400
committer: Tom Lane <[email protected]>
date : Sat, 10 Oct 2020 14:53:23 -0400
Fix silly typo in previous commit.
Discussion: https://postgr.es/m/CAC+AXB0g44SbvSpC86o_1HWh8TAU2pZrMRW6tJT-dkijotx5Qg@mail.gmail.com
M src/port/win32stat.c
Minor cleanup for win32stat.c.
commit : c94cfb38c32a68ef9b021a6851fff25f9ecdf297
author : Tom Lane <[email protected]>
date : Sat, 10 Oct 2020 13:39:21 -0400
committer: Tom Lane <[email protected]>
date : Sat, 10 Oct 2020 13:39:21 -0400
Ensure that CloseHandle() can't clobber the errno we set for
failure exits, and make a couple of tweaks for pgindent.
Juan José Santamaría Flecha
Discussion: https://postgr.es/m/CAC+AXB0g44SbvSpC86o_1HWh8TAU2pZrMRW6tJT-dkijotx5Qg@mail.gmail.com
M src/port/win32stat.c
Recognize network-failure errnos as indicating hard connection loss.
commit : fe27009cbb5fff53f87969786ca8dac82ed7deba
author : Tom Lane <[email protected]>
date : Sat, 10 Oct 2020 13:28:12 -0400
committer: Tom Lane <[email protected]>
date : Sat, 10 Oct 2020 13:28:12 -0400
Up to now, only ECONNRESET (and EPIPE, in most but not quite all places)
received special treatment in our error handling logic. This patch
changes things so that related error codes such as ECONNABORTED are
also recognized as indicating that the connection's dead and unlikely
to come back.
We continue to think, however, that only ECONNRESET and EPIPE should be
reported as probable server crashes; the other cases indicate network
connectivity problems but prove little about the server's state. Thus,
there's no change in the error message texts that are output for such
cases. The key practical effect is that errcode_for_socket_access()
will report ERRCODE_CONNECTION_FAILURE rather than
ERRCODE_INTERNAL_ERROR for a network failure. It's expected that this
will fix buildfarm member lorikeet's failures since commit 32a9c0bdf,
as that seems to be due to not treating ECONNABORTED equivalently to
ECONNRESET.
The set of errnos treated this way now includes ECONNABORTED, EHOSTDOWN,
EHOSTUNREACH, ENETDOWN, ENETRESET, and ENETUNREACH. Several of these
were second-class citizens in terms of their handling in places like
get_errno_symbol(), so upgrade the infrastructure where necessary.
As committed, this patch assumes that all these symbols are defined
everywhere. POSIX specifies all of them except EHOSTDOWN, but that
seems to exist on all platforms of interest; we'll see what the
buildfarm says about that.
Probably this should be back-patched, but let's see what the buildfarm
thinks of it first.
Fujii Masao and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/port/win32/socket.c
M src/backend/utils/error/elog.c
M src/bin/pg_dump/parallel.c
M src/include/port.h
M src/include/port/win32_port.h
M src/interfaces/libpq/fe-misc.c
M src/interfaces/libpq/fe-secure.c
M src/interfaces/libpq/win32.h
M src/port/strerror.c
plperl.h should #undef fstat along with stat and lstat.
commit : ed30b1a60dadf2b7cc58bce5009ad8676b8fe479
author : Tom Lane <[email protected]>
date : Fri, 9 Oct 2020 17:54:34 -0400
committer: Tom Lane <[email protected]>
date : Fri, 9 Oct 2020 17:54:34 -0400
Needed now that commit bed90759f caused win32_port.h to provide
a #define for that too. Per buildfarm.
M src/pl/plperl/plperl.h
Fix our Windows stat() emulation to handle file sizes > 4GB.
commit : bed90759fcbcd72d4d06969eebab81e47326f9a2
author : Tom Lane <[email protected]>
date : Fri, 9 Oct 2020 16:20:12 -0400
committer: Tom Lane <[email protected]>
date : Fri, 9 Oct 2020 16:20:12 -0400
Hack things so that our idea of "struct stat" is equivalent to Windows'
struct __stat64, allowing it to have a wide enough st_size field.
Instead of relying on native stat(), use GetFileInformationByHandle().
This avoids a number of issues with Microsoft's multiple and rather
slipshod emulations of stat(). We still need to jump through hoops
to deal with ERROR_DELETE_PENDING, though :-(
Pull the relevant support code out of dirmod.c and put it into
its own file, win32stat.c.
Still TODO: do we need to do something different with lstat(),
rather than treating it identically to stat()?
Juan José Santamaría Flecha, reviewed by Emil Iggland;
based on prior work by Michael Paquier, Sergey Zubkovsky, and others
Discussion: https://postgr.es/m/1803D792815FC24D871C00D17AE95905CF5099@g01jpexmbkw24
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.ac
M src/include/port/win32_port.h
M src/port/dirmod.c
A src/port/win32stat.c
M src/tools/msvc/Mkvcbuild.pm
Fix typos in logical.c and reorderbuffer.c.
commit : f13f2e484172a1c865cd067796cee3568467dd51
author : Amit Kapila <[email protected]>
date : Fri, 9 Oct 2020 08:15:53 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 9 Oct 2020 08:15:53 +0530
Reviewed-by: Sawada Masahiko
Discussion: https://postgr.es/m/CAA4eK1K6zTpuqf_d7wXCBjo_EF0_B6Fz3Ecp71Vq18t=wG-nzg@mail.gmail.com
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/reorderbuffer.c
Avoid gratuitous inaccuracy in numeric width_bucket().
commit : 7538708394e7a70105a4e601e253adf80f47cca8
author : Tom Lane <[email protected]>
date : Thu, 8 Oct 2020 13:06:27 -0400
committer: Tom Lane <[email protected]>
date : Thu, 8 Oct 2020 13:06:27 -0400
Multiply before dividing, not the reverse, so that cases that should
produce exact results do produce exact results. (width_bucket_float8
got this right already.) Even when the result is inexact, this avoids
making it more inexact, since only the division step introduces any
imprecision.
While at it, fix compute_bucket() to not uselessly repeat the sign
check already done by its caller, and avoid duplicating the
multiply/divide steps by adjusting variable usage.
Per complaint from Martin Visser. Although this seems like a bug fix,
I'm hesitant to risk changing width_bucket()'s results in stable
branches, so no back-patch.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Fix numeric width_bucket() to allow its first argument to be infinite.
commit : 8ce423b1912b8303dbec5dc3ec78a7a725acf6c2
author : Tom Lane <[email protected]>
date : Thu, 8 Oct 2020 12:37:59 -0400
committer: Tom Lane <[email protected]>
date : Thu, 8 Oct 2020 12:37:59 -0400
While the calculation is not well-defined if the bounds arguments are
infinite, there is a perfectly sane outcome if the test operand is
infinite: it's just like any other value that's before the first bucket
or after the last one. width_bucket_float8() got this right, but
I was too hasty about the case when adding infinities to numerics
(commit a57d312a7), so that width_bucket_numeric() just rejected it.
Fix that, and sync the relevant error message strings.
No back-patch needed, since infinities-in-numeric haven't shipped yet.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Fix typo in multixact.c
commit : b90b79e1409b7cbffcadf89ae2e85c7ba1332818
author : Michael Paquier <[email protected]>
date : Thu, 8 Oct 2020 14:06:12 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 8 Oct 2020 14:06:12 +0900
AtEOXact_MultiXact() was referenced in two places with an incorrect
routine name.
Author: Hou Zhijie
Discussion: https://postgr.es/m/1b41e9311e8f474cb5a360292f0b3cb1@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/access/transam/multixact.c
Improve set of candidate multipliers for perfect hash function generation
commit : 2a7316458164369436e252e5e60a5957b17103c3
author : Michael Paquier <[email protected]>
date : Thu, 8 Oct 2020 13:16:43 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 8 Oct 2020 13:16:43 +0900
The previous set of multipliers was not adapted for large sets of short
keys, and this new set of multipliers allows to generate perfect hash
functions for larger sets without having an impact for existing callers
of those functions, as experimentation has showed. A future commit will
make use of that to improve the performance of unicode normalization.
All multipliers compile to shift-and-add instructions on most platforms.
This has been tested as far back as gcc 4.1 and clang 3.8.
Author: John Naylor
Reviewed-by: Mark Dilger, Michael Paquier
Discussion: https://postgr.es/m/CACPNZCt4fbJ0_bGrN5QPt34N4whv=mszM0LMVQdoa2rC9UMRXA@mail.gmail.com
M src/tools/PerfectHash.pm
Track statistics for spilling of changes from ReorderBuffer.
commit : 98681675002d852d926a49d7bc4d4b4856b2fc4a
author : Amit Kapila <[email protected]>
date : Thu, 8 Oct 2020 09:09:08 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 8 Oct 2020 09:09:08 +0530
This adds the statistics about transactions spilled to disk from
ReorderBuffer. Users can query the pg_stat_replication_slots view to check
these stats and call pg_stat_reset_replication_slot to reset the stats of
a particular slot. Users can pass NULL in pg_stat_reset_replication_slot
to reset stats of all the slots.
This commit extends the statistics collector to track this information
about slots.
Author: Sawada Masahiko and Amit Kapila
Reviewed-by: Amit Kapila and Dilip Kumar
Discussion: https://postgr.es/m/CA+fd4k5_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/slot.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/include/replication/logical.h
M src/include/replication/reorderbuffer.h
M src/include/replication/slot.h
M src/test/regress/expected/rules.out
M src/tools/pgindent/typedefs.list
Fix optimization hazard in gram.y's makeOrderedSetArgs(), redux.
commit : 8d2a01ae12cd657b33ffd50eace86a341636c586
author : Tom Lane <[email protected]>
date : Wed, 7 Oct 2020 18:41:39 -0400
committer: Tom Lane <[email protected]>
date : Wed, 7 Oct 2020 18:41:39 -0400
It appears that commit cf63c641c, which intended to prevent
misoptimization of the result-building step in makeOrderedSetArgs,
didn't go far enough: buildfarm member hornet's version of xlc
is now optimizing back to the old, broken behavior in which
list_length(directargs) is fetched only after list_concat() has
changed that value. I'm not entirely convinced whether that's
an undeniable compiler bug or whether it can be justified by a
sufficiently aggressive interpretation of C sequence points.
So let's just change the code to make it harder to misinterpret.
Back-patch to all supported versions, just in case.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/gram.y
Prevent internal overflows in date-vs-timestamp and related comparisons.
commit : 3db322eaab9688d57643b4d2a5f52b7f350ef46f
author : Tom Lane <[email protected]>
date : Wed, 7 Oct 2020 17:10:26 -0400
committer: Tom Lane <[email protected]>
date : Wed, 7 Oct 2020 17:10:26 -0400
The date-vs-timestamp, date-vs-timestamptz, and timestamp-vs-timestamptz
comparators all worked by promoting the first type to the second and
then doing a simple same-type comparison. This works fine, except
when the conversion result is out of range, in which case we throw an
entirely avoidable error. The sources of such failures are
(a) type date can represent dates much farther in the future than
the timestamp types can;
(b) timezone rotation might cause a just-in-range timestamp value to
become a just-out-of-range timestamptz value.
Up to now we just ignored these corner-case issues, but now we have
an actual user complaint (bug #16657 from Huss EL-Sheikh), so let's
do something about it.
It turns out that commit 52ad1e659 already built all the necessary
infrastructure to support error-free comparisons, but neglected to
actually use it in the main-line code paths. Fix that, do a little
bit of code style review, and remove the now-duplicate logic in
jsonpath_exec.c.
Back-patch to v13 where 52ad1e659 came in. We could take this back
further by back-patching said infrastructure, but given the small
number of complaints so far, I don't feel a great need to.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/date.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/adt/timestamp.c
M src/include/utils/date.h
M src/include/utils/timestamp.h
M src/test/regress/expected/horology.out
M src/test/regress/sql/horology.sql
Clean up after newly-added tests for pg_test_fsync and pg_test_timing.
commit : 6c05e5b77471dfadebe50ad4a8bdedef02ad0078
author : Tom Lane <[email protected]>
date : Wed, 7 Oct 2020 13:27:33 -0400
committer: Tom Lane <[email protected]>
date : Wed, 7 Oct 2020 13:27:33 -0400
Oversight in 4d29e6dbd.
M src/bin/pg_test_fsync/Makefile
M src/bin/pg_test_timing/Makefile
Rethink recent fix for pg_dump's handling of extension config tables.
commit : 9e5f1f21ad6a10662bc043046d1a2adea05007f6
author : Tom Lane <[email protected]>
date : Wed, 7 Oct 2020 12:50:54 -0400
committer: Tom Lane <[email protected]>
date : Wed, 7 Oct 2020 12:50:54 -0400
Commit 3eb3d3e78 was a few bricks shy of a load: while it correctly
set the table's "interesting" flag when deciding to dump the data of
an extension config table, it was not correct to clear that flag
if we concluded we shouldn't dump the data. This led to the crash
reported in bug #16655, because in fact we'll traverse dumpTableSchema
anyway for all extension tables (to see if they have user-added
seclabels or RLS policies).
The right thing to do is to force "interesting" true in makeTableDataInfo,
and otherwise leave the flag alone. (Doing it there is more future-proof
in case additional calls are added, and it also avoids setting the flag
unnecessarily if that function decides the table is non-dumpable.)
This investigation also showed that while only the --inserts code path
had an obvious failure in the case considered by 3eb3d3e78, the COPY
code path also has a problem with not having loaded table subsidiary
data. That causes fmtCopyColumnList to silently return an empty string
instead of the correct column list. That accidentally mostly works,
which perhaps is why we didn't notice this before. It would only fail
if the restore column order is different from the dump column order,
which only happens in weird inheritance cases, so it's not surprising
nobody had hit the case with an extension config table. Nonetheless,
it's a bug, and it goes a long way back, not just to v12 where the
--inserts code path started to have a problem with this.
In hopes of catching such cases a bit sooner in future, add some
Asserts that "interesting" has been set in both dumpTableData and
dumpTableSchema. Adjust the test case added by 3eb3d3e78 so that it
checks the COPY rather than INSERT form of that bug, allowing it to
detect the longer-standing symptom.
Per bug #16655 from Cameron Daniel. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_dump.c
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/modules/test_pg_dump/test_pg_dump–1.0.sql
Display the names of missing columns in error during logical replication.
commit : f07707099c17e7ff66eac7d38cbd1148672f7ee4
author : Amit Kapila <[email protected]>
date : Wed, 7 Oct 2020 08:14:19 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 7 Oct 2020 08:14:19 +0530
In logical replication when a subscriber is missing some columns, it
currently emits an error message that says "some" columns are missing, but
it doesn't specify the missing column names. Change that to display
missing column names which makes an error to be more informative to the
user.
We have decided not to backpatch this commit as this is a minor usability
improvement and no user has reported this.
Reported-by: Bharath Rupireddy
Author: Bharath Rupireddy
Reviewed-by: Kyotaro Horiguchi and Amit Kapila
Discussion: https://postgr.es/m/CALj2ACVkW-EXH_4pmBK8tNeHRz5ksUC4WddGactuCjPiBch-cg@mail.gmail.com
M src/backend/replication/logical/relation.c
pg_upgrade: remove pre-8.4 code and >= 8.4 check
commit : 069179767f79c03381056b23ded0a54156674456
author : Bruce Momjian <[email protected]>
date : Tue, 6 Oct 2020 14:31:22 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 6 Oct 2020 14:31:22 -0400
We only support upgrading from >= 8.4 so no need for this code or tests.
Reported-by: Magnus Hagander
Discussion: https://postgr.es/m/CABUevEx-D0PNVe00tkeQRGennZQwDtBJn=493MJt-x6sppbUxA@mail.gmail.com
Backpatch-through: 9.5
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/relfilenode.c
pg_upgrade; change major version comparisons to use <=, not <
commit : bc1fbc960bf5efbb692f4d1bf91bf9bc6390425a
author : Bruce Momjian <[email protected]>
date : Tue, 6 Oct 2020 12:12:09 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 6 Oct 2020 12:12:09 -0400
This makes checking for older major versions more consistent.
Backpatch-through: 9.5
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/exec.c
M src/bin/pg_upgrade/function.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/server.c
Build EC members for child join rels in the right memory context.
commit : d7885b1f87fa066606acc54700edee28812be8c1
author : Tom Lane <[email protected]>
date : Tue, 6 Oct 2020 11:43:53 -0400
committer: Tom Lane <[email protected]>
date : Tue, 6 Oct 2020 11:43:53 -0400
This patch prevents crashes or wrong plans when partition-wise joins
are considered during GEQO planning, as a consequence of the
EquivalenceClass data structures becoming corrupt after a GEQO
context reset.
A remaining problem is that successive GEQO cycles will make multiple
copies of the required EC members, since add_child_join_rel_equivalences
has no idea that such members might exist already. For now we'll just
live with that. The lack of field complaints of crashes suggests that
this is a mighty little-used situation.
Back-patch to v12 where this code was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/path/equivclass.c
Further improvements on documentation for pg_dump -t
commit : 1b22224945e1e7162fe92aecb1cd2b373fba24c2
author : Magnus Hagander <[email protected]>
date : Tue, 6 Oct 2020 15:50:03 +0200
committer: Magnus Hagander <[email protected]>
date : Tue, 6 Oct 2020 15:50:03 +0200
Ian submitted an updated patch just as I was pushing the previous one,
so use this newer wording instead.
Author: Ian Barwick
M doc/src/sgml/ref/pg_dump.sgml
Clarify documentation around pg_dump -t option
commit : b8c4d385120cb7fe70006812234b2cbc950f4a61
author : Magnus Hagander <[email protected]>
date : Tue, 6 Oct 2020 15:46:36 +0200
committer: Magnus Hagander <[email protected]>
date : Tue, 6 Oct 2020 15:46:36 +0200
The behavior is different for different types of objects, so make that
more clear.
Author: Ian Barwick
M doc/src/sgml/ref/pg_dump.sgml
Expand installation documentation to cover binary installations
commit : 5b36221c46f9ca9c5df6a6e108790b5c6c5d2cc3
author : Magnus Hagander <[email protected]>
date : Tue, 6 Oct 2020 14:15:32 +0200
committer: Magnus Hagander <[email protected]>
date : Tue, 6 Oct 2020 14:15:32 +0200
Reviewed-By: David G. Johnston, Daniel Gustafsson
M doc/src/sgml/filelist.sgml
A doc/src/sgml/install-binaries.sgml
M doc/src/sgml/install-windows.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/postgres.sgml
Fix compilation warning in xlog.c
commit : 0a3c864c32751fd29d021929cf70af421fd27370
author : Michael Paquier <[email protected]>
date : Tue, 6 Oct 2020 15:29:34 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 6 Oct 2020 15:29:34 +0900
Oversight in 9d0bd95.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
Try to unbreak 021_row_visibility.pl on mingw.
commit : 1df2b50dbebb72a75708c3cfbc21bcaa71993147
author : Andres Freund <[email protected]>
date : Mon, 5 Oct 2020 19:20:17 -0700
committer: Andres Freund <[email protected]>
date : Mon, 5 Oct 2020 19:20:17 -0700
Thanks to Andrew for proposing and testing this fix.
It's possible that we should address this on a more fundamental basis,
e.g. by configuring PerlIO to to CR/LF conversion for us, but this
approach already exists in other places. And it's nice to unbreak the
BF.
Proposed-By: Andrew Dunstan <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/021_row_visibility.pl
postgres_fdw: reestablish new connection if cached one is detected as broken.
commit : 32a9c0bdf493cf5fc029ab44a22384d547290eff
author : Fujii Masao <[email protected]>
date : Tue, 6 Oct 2020 10:31:09 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 6 Oct 2020 10:31:09 +0900
In postgres_fdw, once remote connections are established, they are cached
and re-used for subsequent queries and transactions. There can be some
cases where those cached connections are unavaiable, for example,
by the restart of remote server. In these cases, previously an error was
reported and the query accessing to remote server failed if new remote
transaction failed to start because the cached connection was broken.
This commit improves postgres_fdw so that new connection is remade
if broken connection is detected when starting new remote transaction.
This is useful to avoid unnecessary failure of queries when connection is
broken but can be reestablished.
Author: Bharath Rupireddy, tweaked a bit by Fujii Masao
Reviewed-by: Ashutosh Bapat, Tatsuhito Kasahara, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACUAi23vf1WiHNar_LksM9EDOWXcbHCo-fD4Mbr1d=78YQ@mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
doc: show functions returning record types and use of ROWS FROM
commit : dd0a64ed435d4a266ed16adb8204e7222af6c164
author : Bruce Momjian <[email protected]>
date : Mon, 5 Oct 2020 16:27:33 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 5 Oct 2020 16:27:33 -0400
Previously it was unclear exactly how ROWS FROM behaved and how to cast
the data types of columns returned by FROM functions. Also document
that only non-OUT record functions can have their columns cast to data
types.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/queries.sgml
Overhaul pg_hba.conf clientcert's API
commit : 253f1025da8c8d6e52f96f764658b76eb59290ad
author : Bruce Momjian <[email protected]>
date : Mon, 5 Oct 2020 15:48:40 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 5 Oct 2020 15:48:40 -0400
Since PG 12, clientcert no longer supported only on/off, so remove 1/0
as possible values, and instead support only the text strings
'verify-ca' and 'verify-full'.
Remove support for 'no-verify' since that is possible by just not
specifying clientcert.
Also, throw an error if 'verify-ca' is used and 'cert' authentication is
used, since cert authentication requires verify-full.
Also improve the docs.
THIS IS A BACKWARD INCOMPATIBLE API CHANGE.
Reported-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
Author: Kyotaro Horiguchi
Backpatch-through: master
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/runtime.sgml
M src/backend/libpq/hba.c
Include the process PID in assertion-failure messages.
commit : 18c170a08ee23d03a06d235ea628fecb057d974f
author : Tom Lane <[email protected]>
date : Mon, 5 Oct 2020 13:40:28 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Oct 2020 13:40:28 -0400
This should help to identify what happened when studying the postmaster
log after-the-fact.
While here, clean up some old comments in the same function.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/error/assert.c
Fix two latent(?) bugs in equivclass.c.
commit : 53c6daff4364219256119fcd79b2d71b57c13e37
author : Tom Lane <[email protected]>
date : Mon, 5 Oct 2020 13:15:39 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Oct 2020 13:15:39 -0400
get_eclass_for_sort_expr() computes expr_relids and nullable_relids
early on, even though they won't be needed unless we make a new
EquivalenceClass, which we often don't. Aside from the probably-minor
inefficiency, there's a memory management problem: these bitmapsets will
be built in the caller's context, leading to dangling pointers if that
is shorter-lived than root->planner_cxt. This would be a live bug if
get_eclass_for_sort_expr() could be called with create_it = true during
GEQO join planning. So far as I can find, the core code never does
that, but it's hard to be sure that no extensions do, especially since
the comments make it clear that that's supposed to be a supported case.
Fix by not computing these values until we've switched into planner_cxt
to build the new EquivalenceClass.
generate_join_implied_equalities() uses inner_rel->relids to look up
relevant eclasses, but it ought to be using nominal_inner_relids.
This is presently harmless because a child RelOptInfo will always have
exactly the same eclass_indexes as its topmost parent; but that might
not be true forever, and anyway it makes the code confusing.
The first of these is old (introduced by me in f3b3b8d5b), so back-patch
to all supported branches. The second only dates to v13, but we might
as well back-patch it to keep the code looking similar across branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/path/equivclass.c
Doc: fix parameter names in the docs of a couple of functions.
commit : 9cc3d614a9eb6ce17c0f6f6bf77c5d431617267e
author : Tom Lane <[email protected]>
date : Mon, 5 Oct 2020 11:42:33 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Oct 2020 11:42:33 -0400
The descriptions of make_interval() and pg_options_to_table()
were randomly different from the reality embedded in pg_proc.
(These are not all the discrepancies I found in a quick search,
but the others perhaps require more discussion, since there's
at least a case to be made for changing pg_proc not the docs.)
make_interval issue noted by Thomas Kellerer.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
Support for OUT parameters in procedures
commit : 2453ea142233ae57af452019c3b9a443dad1cdd0
author : Peter Eisentraut <[email protected]>
date : Mon, 5 Oct 2020 09:09:09 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 5 Oct 2020 09:09:09 +0200
Unlike for functions, OUT parameters for procedures are part of the
signature. Therefore, they have to be listed in pg_proc.proargtypes
as well as mentioned in ALTER PROCEDURE and DROP PROCEDURE.
Reviewed-by: Andrew Dunstan <[email protected]>
Reviewed-by: Pavel Stehule <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/ref/alter_extension.sgml
M doc/src/sgml/ref/alter_procedure.sgml
M doc/src/sgml/ref/comment.sgml
M doc/src/sgml/ref/create_procedure.sgml
M doc/src/sgml/ref/drop_procedure.sgml
M doc/src/sgml/ref/security_label.sgml
M doc/src/sgml/xfunc.sgml
M src/backend/catalog/pg_proc.c
M src/backend/commands/functioncmds.c
M src/backend/executor/functions.c
M src/backend/parser/gram.y
M src/backend/utils/fmgr/funcapi.c
M src/include/catalog/pg_proc.h
M src/include/funcapi.h
M src/pl/plperl/expected/plperl_call.out
M src/pl/plperl/sql/plperl_call.sql
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/sql/plpgsql_call.sql
M src/pl/plpython/expected/plpython_call.out
M src/pl/plpython/plpy_procedure.c
M src/pl/plpython/sql/plpython_call.sql
M src/pl/tcl/expected/pltcl_call.out
M src/pl/tcl/sql/pltcl_call.sql
M src/test/regress/expected/create_procedure.out
M src/test/regress/sql/create_procedure.sql
Improve stability of identity.sql regression test.
commit : e899742081fa24bf52d4a32103ef854a3a85865d
author : Tom Lane <[email protected]>
date : Sun, 4 Oct 2020 20:45:06 -0400
committer: Tom Lane <[email protected]>
date : Sun, 4 Oct 2020 20:45:06 -0400
I noticed while trying to run the regression tests under a low
geqo_threshold that one query on information_schema.columns had
unstable (as in, variable from one run to the next) output order.
This is pretty unsurprising given the complexity of the underlying
plan. Interestingly, of this test's three nigh-identical queries on
information_schema.columns, the other two already had ORDER BY clauses
guaranteeing stable output. Let's make this one look the same.
Back-patch to v10 where this test was added. We've not heard field
reports of the test failing, but this experience shows that it can
happen when testing under even slightly unusual conditions.
M src/test/regress/expected/identity.out
M src/test/regress/sql/identity.sql
Fix handling of redundant options with COPY for "freeze" and "header"
commit : 10c5291cc2c6497d5abe04dde7df649d898becc4
author : Michael Paquier <[email protected]>
date : Mon, 5 Oct 2020 09:43:17 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 5 Oct 2020 09:43:17 +0900
The handling of those options was inconsistent, as the processing used
directly the value assigned to the option to check if it was redundant,
leading to patterns like this one to succeed (note that false is
specified first):
COPY hoge to '/path/to/file/' (header off, header on);
And the opposite would fail correctly (note that true is first here):
COPY hoge to '/path/to/file/' (header on, header off);
While on it, add some tests to check for all redundant patterns with the
options of COPY. I have gone through the code and did not notice
similar mistakes for other commands.
"header" got it wrong since b63990c, and "freeze" was wrong from the
start as of 8de72b6. No backpatch is done per the lack of complaints.
Reported-by: Rémi Lapeyre
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/copy.c
M src/test/regress/expected/copy2.out
M src/test/regress/sql/copy2.sql
Make postgres.bki use the same literal-string syntax as postgresql.conf.
commit : 97b61448262eae5e1b4a631aeac63b11d902a474
author : Tom Lane <[email protected]>
date : Sun, 4 Oct 2020 16:09:55 -0400
committer: Tom Lane <[email protected]>
date : Sun, 4 Oct 2020 16:09:55 -0400
The BKI file's string quoting conventions were previously quite weird,
perhaps as a result of repurposing a function built to scan
single-quoted strings to scan double-quoted ones. Change to use the
same rules as we use in GUC files, allowing some simplifications in
genbki.pl and initdb.c.
While at it, completely remove the backend's scanstr() function, which
was essentially a duplicate of the string dequoting code in guc-file.l.
Instead export that one (under a less generic name than it had) and let
bootscanner.l use it. Now we can clarify that scansup.c exists only to
support the main lexer. We could alternatively have removed GUC_scanstr,
but this way seems better since the previous arrangement could mislead
a reader into thinking that scanstr() had something to do with the main
lexer's handling of string literals. Maybe it did once, but if so it
was a long time ago.
This patch does not bump catversion, since the initially-installed
catalog contents don't change. Note however that successful initdb
after applying this patch will require up-to-date postgres.bki as well
as postgres and initdb executables.
In passing, remove a bunch of very-long-obsolete #include's in
bootparse.y and bootscanner.l.
John Naylor
Discussion: https://postgr.es/m/CACPNZCtDpd18T0KATTmCggO2GdVC4ow86ypiq5ENff1VnauL8g@mail.gmail.com
M doc/src/sgml/bki.sgml
M src/backend/bootstrap/bootparse.y
M src/backend/bootstrap/bootscanner.l
M src/backend/catalog/genbki.pl
M src/backend/parser/scansup.c
M src/backend/utils/misc/guc-file.l
M src/bin/initdb/initdb.c
M src/include/parser/scansup.h
M src/include/utils/guc.h
Improve <xref> vs. <command> formatting in the documentation
commit : 9081bddbd75e4e8994ca243c820ca63387bd33f7
author : Peter Eisentraut <[email protected]>
date : Sat, 3 Oct 2020 16:16:51 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 3 Oct 2020 16:16:51 +0200
SQL commands are generally marked up as <command>, except when a link
to a reference page is used using <xref>. But the latter doesn't
create monospace markup, so this looks strange especially when a
paragraph contains a mix of links and non-links.
We considered putting <command> in the <refentrytitle> on the target
side, but that creates some formatting side effects elsewhere.
Generally, it seems safer to solve this on the link source side.
We can't put the <xref> inside the <command>; the DTD doesn't allow
this. DocBook 5 would allow the <command> to have the linkend
attribute itself, but we are not there yet.
So to solve this for now, convert the <xref>s to <link> plus
<command>. This gives the correct look and also gives some more
flexibility what we can put into the link text (e.g., subcommands or
other clauses). In the future, these could then be converted to
DocBook 5 style.
I haven't converted absolutely all xrefs to SQL command reference
pages, only those where we care about the appearance of the link text
or where it was otherwise appropriate to make the appearance match a
bit better. Also in some cases, the links where repetitive, so in
those cases the links where just removed and replaced by a plain
<command>. In cases where we just want the link and don't
specifically care about the generated link text (typically phrased
"for further information see <xref ...>") the xref is kept.
Reported-by: Dagfinn Ilmari Mannsåker <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/backup.sgml
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/dml.sgml
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/queries.sgml
M doc/src/sgml/ref/abort.sgml
M doc/src/sgml/ref/alter_aggregate.sgml
M doc/src/sgml/ref/alter_domain.sgml
M doc/src/sgml/ref/alter_foreign_table.sgml
M doc/src/sgml/ref/alter_group.sgml
M doc/src/sgml/ref/alter_index.sgml
M doc/src/sgml/ref/alter_materialized_view.sgml
M doc/src/sgml/ref/alter_role.sgml
M doc/src/sgml/ref/alter_statistics.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/alter_trigger.sgml
M doc/src/sgml/ref/alter_type.sgml
M doc/src/sgml/ref/alter_user.sgml
M doc/src/sgml/ref/analyze.sgml
M doc/src/sgml/ref/begin.sgml
M doc/src/sgml/ref/close.sgml
M doc/src/sgml/ref/cluster.sgml
M doc/src/sgml/ref/copy.sgml
M doc/src/sgml/ref/create_aggregate.sgml
M doc/src/sgml/ref/create_cast.sgml
M doc/src/sgml/ref/create_database.sgml
M doc/src/sgml/ref/create_foreign_table.sgml
M doc/src/sgml/ref/create_function.sgml
M doc/src/sgml/ref/create_index.sgml
M doc/src/sgml/ref/create_language.sgml
M doc/src/sgml/ref/create_materialized_view.sgml
M doc/src/sgml/ref/create_operator.sgml
M doc/src/sgml/ref/create_procedure.sgml
M doc/src/sgml/ref/create_role.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/create_table_as.sgml
M doc/src/sgml/ref/create_transform.sgml
M doc/src/sgml/ref/create_trigger.sgml
M doc/src/sgml/ref/create_type.sgml
M doc/src/sgml/ref/create_user.sgml
M doc/src/sgml/ref/create_view.sgml
M doc/src/sgml/ref/createdb.sgml
M doc/src/sgml/ref/createuser.sgml
M doc/src/sgml/ref/declare.sgml
M doc/src/sgml/ref/delete.sgml
M doc/src/sgml/ref/drop_group.sgml
M doc/src/sgml/ref/drop_language.sgml
M doc/src/sgml/ref/drop_owned.sgml
M doc/src/sgml/ref/drop_role.sgml
M doc/src/sgml/ref/drop_table.sgml
M doc/src/sgml/ref/drop_user.sgml
M doc/src/sgml/ref/dropdb.sgml
M doc/src/sgml/ref/dropuser.sgml
M doc/src/sgml/ref/end.sgml
M doc/src/sgml/ref/explain.sgml
M doc/src/sgml/ref/fetch.sgml
M doc/src/sgml/ref/grant.sgml
M doc/src/sgml/ref/lock.sgml
M doc/src/sgml/ref/postgres-ref.sgml
M doc/src/sgml/ref/prepare.sgml
M doc/src/sgml/ref/prepare_transaction.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/ref/reassign_owned.sgml
M doc/src/sgml/ref/refresh_materialized_view.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/revoke.sgml
M doc/src/sgml/ref/rollback.sgml
M doc/src/sgml/ref/rollback_to.sgml
M doc/src/sgml/ref/savepoint.sgml
M doc/src/sgml/ref/select.sgml
M doc/src/sgml/ref/select_into.sgml
M doc/src/sgml/ref/set_role.sgml
M doc/src/sgml/ref/set_session_auth.sgml
M doc/src/sgml/ref/start_transaction.sgml
M doc/src/sgml/ref/vacuumdb.sgml
M doc/src/sgml/sepgsql.sgml
M doc/src/sgml/tsm-system-rows.sgml
M doc/src/sgml/tsm-system-time.sgml
M doc/src/sgml/user-manag.sgml
M doc/src/sgml/xaggr.sgml
M doc/src/sgml/xfunc.sgml
M doc/src/sgml/xplang.sgml
doc: libpq connection options can override command-line flags
commit : 1a9388bd0fefccc81430192d0b2c87938bb62d38
author : Bruce Momjian <[email protected]>
date : Fri, 2 Oct 2020 22:19:31 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 2 Oct 2020 22:19:31 -0400
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/ref/clusterdb.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_isready.sgml
M doc/src/sgml/ref/pg_receivewal.sgml
M doc/src/sgml/ref/pg_recvlogical.sgml
M doc/src/sgml/ref/pg_restore.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
doc: clarify the use of ssh port forwarding
commit : 472e518a44eacd9caac7d618f1b6451672ca4481
author : Bruce Momjian <[email protected]>
date : Fri, 2 Oct 2020 21:39:33 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 2 Oct 2020 21:39:33 -0400
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/runtime.sgml
Tidy up error reporting when converting PL/Python arrays.
commit : 26b8361518c393c2f152e9e3837daf605b34bef8
author : Heikki Linnakangas <[email protected]>
date : Fri, 2 Oct 2020 18:23:39 +0300
committer: Heikki Linnakangas <[email protected]>
date : Fri, 2 Oct 2020 18:23:39 +0300
Use PLy_elog() only when a call to a Python C API function failed, and
ereport() for other errors. Add an error code to the "wrong length of
inner sequence" ereport().
Reviewed-by: Daniel Gustafsson
Discussion: https://www.postgresql.org/message-id/B8B72889-D6D7-48FF-B782-D670A6CA4D37%40yesql.se
M src/pl/plpython/plpy_typeio.c
doc: Improve some documentation about HA and replication
commit : 8550cbd0bae7c3004034351cb3447b51a552e56c
author : Michael Paquier <[email protected]>
date : Fri, 2 Oct 2020 10:36:35 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 2 Oct 2020 10:36:35 +0900
This clarifies some wording in the description of the options available
as replication solutions. While on it, this replaces some instances of
"master" with "primary", for consistency with recent changes like
9e101cf.
Author: Robert Treat
Reviewed-by: Magnus Hagander, Michael Paquier
Discussion: https://postgr.es/m/CAJSLCQ2TPaK_K8raofCamrqELCxY-H6mJrpDNRzc-LKpPY7c+g@mail.gmail.com
M doc/src/sgml/high-availability.sgml
Add pg_stat_wal statistics view.
commit : 8d9a935965f01b7759a8c23ff6291000b670a2bf
author : Fujii Masao <[email protected]>
date : Fri, 2 Oct 2020 10:17:11 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 2 Oct 2020 10:17:11 +0900
This view shows the statistics about WAL activity. Currently it has only
two columns: wal_buffers_full and stats_reset. wal_buffers_full column
indicates the number of times WAL data was written to the disk because
WAL buffers got full. This information is useful when tuning wal_buffers.
stats_reset column indicates the time at which these statistics were
last reset.
pg_stat_wal view is also the basic infrastructure to expose other
various statistics about WAL activity later.
Bump PGSTAT_FILE_FORMAT_ID due to the change in pgstat format.
Bump catalog version.
Author: Masahiro Ikeda
Reviewed-by: Takayuki Tsunakawa, Kyotaro Horiguchi, Amit Kapila, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
M src/backend/access/transam/xlog.c
M src/backend/catalog/system_views.sql
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/pgstat.c
M src/backend/utils/adt/pgstatfuncs.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/pgstat.h
M src/test/regress/expected/rules.out
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/sysviews.sql
M src/tools/pgindent/typedefs.list
Add block information in error context of WAL REDO apply loop
commit : 9d0bd95fa90a7243047a74e29f265296a9fc556d
author : Michael Paquier <[email protected]>
date : Fri, 2 Oct 2020 09:31:50 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 2 Oct 2020 09:31:50 +0900
Providing this information can be useful for example when diagnosing
problems related to recovery conflicts or for recovery issues without
having to go through the output generated by pg_waldump to get some
information about the blocks a WAL record works on.
The block information is printed in the same format as pg_waldump. This
already existed in xlog.c for debugging purposes with -DWAL_DEBUG, so
adding the block information in the callback has required just a small
refactoring.
Author: Bertrand Drouvot
Reviewed-by: Michael Paquier, Masahiko Sawada
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
Put back explicit setting of replication values within TAP tests.
commit : 49642530486852f8fef248964e563aaab25ecae7
author : Tom Lane <[email protected]>
date : Thu, 1 Oct 2020 10:59:20 -0400
committer: Tom Lane <[email protected]>
date : Thu, 1 Oct 2020 10:59:20 -0400
Commit 151c0c5f7 neglected the possibility that a TEMP_CONFIG file
would explicitly set max_wal_senders=0; as indeed buildfarm member
thorntail does, so that it can test wal_level=minimal in other test
suites. Hence, rather than assuming that max_wal_senders=10 will
prevail if we say nothing, set it explicitly.
Set max_replication_slots=10 explicitly too, just to be safe.
Back-patch to v10, like the previous patch.
Discussion: https://postgr.es/m/[email protected]
M src/test/perl/PostgresNode.pm
Fix incorrect assertion on number of array dimensions.
commit : e1761871c096514ef08a211bf4a9b557753383dc
author : Heikki Linnakangas <[email protected]>
date : Thu, 1 Oct 2020 11:48:48 +0300
committer: Heikki Linnakangas <[email protected]>
date : Thu, 1 Oct 2020 11:48:48 +0300
This has been wrong ever since the support for multi-dimensional
arrays as PL/python function arguments and return values was
introduced in commit 94aceed317.
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/61647b8e-961c-0362-d5d3-c8a18f4a7ec6%40iki.fi
M src/pl/plpython/plpy_typeio.c
Set right-links during sorted GiST index build.
commit : 265ea567852a692e644fe6b207def4a2fc812adb
author : Heikki Linnakangas <[email protected]>
date : Thu, 1 Oct 2020 11:10:43 +0300
committer: Heikki Linnakangas <[email protected]>
date : Thu, 1 Oct 2020 11:10:43 +0300
This is not strictly necessary, as the right-links are only needed by
scans that are concurrent with page splits, and neither scans or page
splits can happen during sorted index build. But it seems like a good
idea to set them anyway, if we e.g. want to add a check to amcheck in
the future to verify that the chain of right-links is complete.
Author: Andrey Borodin
Discussion: https://www.postgresql.org/message-id/4D68C21F-9FB9-41DA-B663-FDFC8D143788%40yandex-team.ru
M src/backend/access/gist/gistbuild.c
Remove logging.c from the shared library of src/common/
commit : 6b1c5cacec5e8305a3880b441526c47bd47c64cd
author : Michael Paquier <[email protected]>
date : Thu, 1 Oct 2020 10:37:34 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 1 Oct 2020 10:37:34 +0900
As fe0a1dc has proved, it is not a good concept to add to libpq
dependencies that would enforce the error output to a central logging
facility because it breaks the promise of reporting the error back to
an application in a consistent way, with the application to potentially
exit() suddenly if using pieces from for example jsonapi.c. prairiedog
has allowed to report an actual design problem with fe0a1dc, but it will
not be around forever, so removing logging.c from libpgcommon_shlib is a
simple and much better long-term way to prevent any attempt to load the
central logging in libraries with general purposes.
Author: Michael Paquier
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/common/Makefile
Fix and test snapshot behavior on standby.
commit : 7b28913bcab8d1bf3dbf59c9d8fb4b51cef57664
author : Andres Freund <[email protected]>
date : Wed, 30 Sep 2020 17:28:51 -0700
committer: Andres Freund <[email protected]>
date : Wed, 30 Sep 2020 17:28:51 -0700
I (Andres) broke this in 623a9CA79bx, because I didn't think about the
way snapshots are built on standbys sufficiently. Unfortunately our
existing tests did not catch this, as they are all just querying with
psql (therefore ending up with fresh snapshots).
The fix is trivial, we just need to increment the transaction
completion counter in ExpireTreeKnownAssignedTransactionIds(), which
is the equivalent of ProcArrayEndTransaction() during recovery.
This commit also adds a new test doing some basic testing of the
correctness of snapshots built on standbys. To avoid the
aforementioned issue of one-shot psql's not exercising the snapshot
caching, the test uses a long lived psqls, similar to
013_crash_restart.pl. It'd be good to extend the test further.
Reported-By: Ian Barwick <[email protected]>
Author: Andres Freund <[email protected]>
Author: Ian Barwick <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/procarray.c
A src/test/recovery/t/021_row_visibility.pl
Reword partitioning error message
commit : 9fc21227128980f88b6eb6d5306625af0428ef45
author : Alvaro Herrera <[email protected]>
date : Wed, 30 Sep 2020 18:25:23 -0300
committer: Alvaro Herrera <[email protected]>
date : Wed, 30 Sep 2020 18:25:23 -0300
The error message about columns in the primary key not including all of
the partition key was unclear; reword it.
Backpatch all the way to pg11, where it appeared.
Reported-by: Nagaraj Raj <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/indexcmds.c
M src/test/regress/expected/indexing.out
Fix handling of BC years in to_date/to_timestamp.
commit : 489c9c3407cbfd473c2f8d7863ffaaf6d2e8fcf8
author : Tom Lane <[email protected]>
date : Wed, 30 Sep 2020 15:40:23 -0400
committer: Tom Lane <[email protected]>
date : Wed, 30 Sep 2020 15:40:23 -0400
Previously, a conversion such as
to_date('-44-02-01','YYYY-MM-DD')
would result in '0045-02-01 BC', as the code attempted to interpret
the negative year as BC, but failed to apply the correction needed
for our internal handling of BC years. Fix the off-by-one problem.
Also, arrange for the combination of a negative year and an
explicit "BC" marker to cancel out and produce AD. This is how
the negative-century case works, so it seems sane to do likewise.
Continue to read "year 0000" as 1 BC. Oracle would throw an error,
but we've accepted that case for a long time so I'm hesitant to
change it in a back-patch.
Per bug #16419 from Saeed Hubaishan. Back-patch to all supported
branches.
Dar Alathar-Yemen and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/utils/adt/formatting.c
M src/test/regress/expected/horology.out
M src/test/regress/sql/horology.sql
pgbench: Use PQExpBuffer to simplify code that constructs SQL.
commit : 9796f455c38e9e79847443529ff2166b1a6f714f
author : Heikki Linnakangas <[email protected]>
date : Wed, 30 Sep 2020 10:58:09 +0300
committer: Heikki Linnakangas <[email protected]>
date : Wed, 30 Sep 2020 10:58:09 +0300
Author: Fabien Coelho
Reviewed-by: Jeevan Ladhe
Discussion: https://www.postgresql.org/message-id/alpine.DEB.2.21.1910220826570.15559%40lancre
M src/bin/pgbench/pgbench.c
Fix XML id to match GUC name
commit : 300b6984a58ad48d0ff2eb2f35c745613f07ad9c
author : Peter Eisentraut <[email protected]>
date : Wed, 30 Sep 2020 07:39:38 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 30 Sep 2020 07:39:38 +0200
For some reason, the id of the description of
max_parallel_maintenance_workers has been
guc-max-parallel-workers-maintenance since the beginning. Flip that
around to make it consistent.
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/create_index.sgml
M doc/src/sgml/ref/vacuum.sgml
Remove obsolete replication settings within TAP tests.
commit : 151c0c5f7277fd572ec02cb4ee992fdd23365a2f
author : Tom Lane <[email protected]>
date : Tue, 29 Sep 2020 20:02:58 -0400
committer: Tom Lane <[email protected]>
date : Tue, 29 Sep 2020 20:02:58 -0400
PostgresNode.pm set "max_wal_senders = 5" for replication testing,
but this seems to be slightly too low for our current test suite.
Slower buildfarm members frequently report "number of requested standby
connections exceeds max_wal_senders" failures, due to old walsenders
not exiting instantaneously. Usually, the test does not fail overall
because of automatic walreceiver restart, but sometimes the failure
becomes visible; and in any case such retries slow down the test.
That value came in with commit 89ac7004d, but was soon obsoleted by
f6d6d2920, which raised the built-in default from zero to 10; so that
PostgresNode.pm is actually setting it to less than the conservative
built-in default. That seems pretty pointless, so let's remove the
special setting and let the default prevail, in hopes of making
the TAP tests more robust.
Likewise, the setting "max_replication_slots = 5" is obsolete and
can be removed.
While here, reverse-engineer a comment about why we're choosing
less-than-default values for some other settings.
(Note: before v12, max_wal_senders counted against max_connections
so that the latter setting also needs some fiddling with.)
Back-patch to v10 where the subscription tests were added.
It's likely that the older branches aren't pushing the boundaries
of max_wal_senders, but I'm disinclined to spend time trying to
figure out exactly when it started to be a problem.
Discussion: https://postgr.es/m/[email protected]
M src/test/perl/PostgresNode.pm
Doc: Improve clarity on partitioned table limitations
commit : 2b888647d864c5341fd0fb628c3773304282dc1d
author : David Rowley <[email protected]>
date : Wed, 30 Sep 2020 13:02:08 +1300
committer: David Rowley <[email protected]>
date : Wed, 30 Sep 2020 13:02:08 +1300
Explicitly mention that primary key constraints are also included in the
limitation that the constraint columns must be a superset of the partition key
columns.
Wording suggestion from Tom Lane.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11, where unique constraints on partitioned tables were added
M doc/src/sgml/ddl.sgml
Fix make_timestamp[tz] to accept negative years as meaning BC.
commit : a094c8ff53523e88ff9dd28ad467618039e27b58
author : Tom Lane <[email protected]>
date : Tue, 29 Sep 2020 13:48:06 -0400
committer: Tom Lane <[email protected]>
date : Tue, 29 Sep 2020 13:48:06 -0400
Previously we threw an error. But make_date already allowed the case,
so it is inconsistent as well as unhelpful for make_timestamp not to.
Both functions continue to reject year zero.
Code and test fixes by Peter Eisentraut, doc changes by me
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/date.out
M src/test/regress/expected/timestamp.out
M src/test/regress/sql/date.sql
M src/test/regress/sql/timestamp.sql
Fix memory leak in plpgsql's CALL processing.
commit : a6b1f5365d58356b5d42829e9cd89a6c725d7a0a
author : Tom Lane <[email protected]>
date : Tue, 29 Sep 2020 11:18:30 -0400
committer: Tom Lane <[email protected]>
date : Tue, 29 Sep 2020 11:18:30 -0400
When executing a CALL or DO in a non-atomic context (i.e., not inside
a function or query), plpgsql creates a new plan each time through,
as a rather hacky solution to some resource management issues. But
it failed to free this plan until exit of the current procedure or DO
block, resulting in serious memory bloat in procedures that called
other procedures many times. Fix by remembering to free the plan,
and by being more honest about restoring the previous state (otherwise,
recursive procedure calls have a problem).
There was also a smaller leak associated with recalculation of the
"target" list of output variables. Fix that by using the statement-
lifespan context to hold non-permanent values.
Back-patch to v11 where procedures were introduced.
Pavel Stehule and Tom Lane
Discussion: https://postgr.es/m/CAFj8pRDiiU1dqym+_P4_GuTWm76knJu7z9opWayBJTC0nQGUUA@mail.gmail.com
M src/pl/plpgsql/src/pl_exec.c
Support for ISO 8601 in the jsonpath .datetime() method
commit : 927d9abb6538e441aa97bb3eea7a15d4cda70715
author : Alexander Korotkov <[email protected]>
date : Tue, 29 Sep 2020 11:41:46 +0300
committer: Alexander Korotkov <[email protected]>
date : Tue, 29 Sep 2020 11:41:46 +0300
The SQL standard doesn't require jsonpath .datetime() method to support the
ISO 8601 format. But our to_json[b]() functions convert timestamps to text in
the ISO 8601 format in the sake of compatibility with javascript. So, we add
support of the ISO 8601 to the jsonpath .datetime() in the sake compatibility
with to_json[b]().
The standard mode of datetime parsing currently supports just template patterns
and separators in the format string. In order to implement ISO 8601, we have to
add support of the format string double quotes to the standard parsing mode.
Discussion: https://postgr.es/m/94321be0-cc96-1a81-b6df-796f437f7c66%40postgrespro.ru
Author: Nikita Glukhov, revised by me
Backpatch-through: 13
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/jsonb_jsonpath.out
M src/test/regress/sql/jsonb_jsonpath.sql
Remove excess space from jsonpath .datetime() default format string
commit : c2aa562ea5c2bf28c347503731434f08097cf1e5
author : Alexander Korotkov <[email protected]>
date : Tue, 29 Sep 2020 11:00:22 +0300
committer: Alexander Korotkov <[email protected]>
date : Tue, 29 Sep 2020 11:00:22 +0300
bffe1bd684 has introduced jsonpath .datetime() method, but default formats
for time and timestamp contain excess space between time and timezone. This
commit removes this excess space making behavior of .datetime() method
standard-compliant.
Discussion: https://postgr.es/m/94321be0-cc96-1a81-b6df-796f437f7c66%40postgrespro.ru
Author: Nikita Glukhov
Backpatch-through: 13
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/jsonb_jsonpath.out
M src/test/regress/sql/jsonb_jsonpath.sql
Archive timeline history files in standby if archive_mode is set to "always".
commit : fd26f78231830b20d8b5f8391f97d2ccebd492b7
author : Fujii Masao <[email protected]>
date : Tue, 29 Sep 2020 16:21:46 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 29 Sep 2020 16:21:46 +0900
Previously the standby server didn't archive timeline history files
streamed from the primary even when archive_mode is set to "always",
while it archives the streamed WAL files. This could cause the PITR to
fail because there was no required timeline history file in the archive.
The cause of this issue was that walreceiver didn't mark those files as
ready for archiving.
This commit makes walreceiver mark those streamed timeline history
files as ready for archiving if archive_mode=always. Then the archiver
process archives the marked timeline history files.
Back-patch to all supported versions.
Reported-by: Grigory Smolkin
Author: Grigory Smolkin, Fujii Masao
Reviewed-by: David Zhang, Anastasia Lubennikova
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/high-availability.sgml
M src/backend/replication/walreceiver.c
Fix progress reporting of REINDEX CONCURRENTLY
commit : e66bcfb4c66ebc97020b1f7484b1529bd7993f23
author : Michael Paquier <[email protected]>
date : Tue, 29 Sep 2020 14:15:57 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 29 Sep 2020 14:15:57 +0900
This addresses a couple of issues with the so-said subject:
- Report the correct parent relation with the index actually being
rebuilt or validated. Previously, the command status remained set to
the last index created for the progress of the index build and
validation, which would be incorrect when working on a table that has
more than one index.
- Use the correct phase when waiting before the drop of the old
indexes. Previously, this was reported with the same status as when
waiting before the old indexes are marked as dead.
Author: Matthias van de Meent, Michael Paquier
Discussion: https://postgr.es/m/CAEze2WhqFgcwe1_tv=sFYhLWV2AdpfukumotJ6JNcAOQs3jufg@mail.gmail.com
Backpatch-through: 12
M src/backend/commands/indexcmds.c
Add for_each_from, to simplify loops starting from non-first list cells.
commit : 56fe008996bc1a547ce60c8dddd2ca821cac163e
author : Tom Lane <[email protected]>
date : Mon, 28 Sep 2020 20:32:53 -0400
committer: Tom Lane <[email protected]>
date : Mon, 28 Sep 2020 20:32:53 -0400
We have a dozen or so places that need to iterate over all but the
first cell of a List. Prior to v13 this was typically written as
for_each_cell(lc, lnext(list_head(list)))
Commit 1cff1b95a changed these to
for_each_cell(lc, list, list_second_cell(list))
This patch introduces a new macro for_each_from() which expresses
the start point as a list index, allowing these to be written as
for_each_from(lc, list, 1)
This is marginally more efficient, since ForEachState.i can be
initialized directly instead of backing into it from a ListCell
address. It also seems clearer and less typo-prone.
Some of the remaining uses of for_each_cell() look like they could
profitably be changed to for_each_from(), but here I confined myself
to changing uses of list_second_cell().
Also, fix for_each_cell_setup() and for_both_cell_setup() to
const-ify their arguments; that's a simple oversight in 1cff1b95a.
Back-patch into v13, on the grounds that (1) the const-ification
is a minor bug fix, and (2) it's better for back-patching purposes
if we only have two ways to write these loops rather than three.
In HEAD, also remove list_third_cell() and list_fourth_cell(),
which were also introduced in 1cff1b95a, and are unused as of
cc99baa43. It seems unlikely that any third-party code would
have started to use them already; anyone who has can be directed
to list_nth_cell instead.
Discussion: https://postgr.es/m/CAApHDvpo1zj9KhEpU2cCRZfSM3Q6XGdhzuAS2v79PH7WJBkYVA@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/backend/nodes/nodeFuncs.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/planner.c
M src/backend/parser/parse_agg.c
M src/backend/utils/adt/jsonpath_gram.y
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/adt/selfuncs.c
M src/include/nodes/pg_list.h
Revert "Change SHA2 implementation based on OpenSSL to use EVP digest routines"
commit : fe0a1dc52c7332a65b44db8e8408a5fd1d8fc8fb
author : Michael Paquier <[email protected]>
date : Tue, 29 Sep 2020 09:25:51 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 29 Sep 2020 09:25:51 +0900
This reverts commit e21cbb4, as the switch to EVP routines requires a
more careful design where we would need to have at least our wrapper
routines return a status instead of issuing an error by themselves to
let the caller do the error handling. The memory handling was also
incorrect and could cause leaks in the backend if a failure happened,
requiring most likely a callback to do the necessary cleanup as the only
clean way to be able to allocate an EVP context requires the use of an
allocation within OpenSSL. The potential rework of the wrappers also
impacts the fallback implementation when not building with OpenSSL.
Originally, prairiedog has reported a compilation failure, but after
discussion with Tom Lane this needs a better design.
Discussion: https://postgr.es/m/[email protected]
M src/common/sha2_openssl.c
M src/include/common/sha2.h
Stabilize create_table regression test.
commit : 042d8017ec7bca244db4a53bc3d72d218495136d
author : Tom Lane <[email protected]>
date : Mon, 28 Sep 2020 14:48:01 -0400
committer: Tom Lane <[email protected]>
date : Mon, 28 Sep 2020 14:48:01 -0400
Adding \d+ to the test in commit 2dfa3fea8 was ill-advised,
because the partitions' names are such that their sort order
is locale dependent. We could rename them to avoid that,
but it doesn't seem worth the trouble; just take \d+ out again.
Per buildfarm.
M src/test/regress/expected/create_table.out
M src/test/regress/sql/create_table.sql
Assign collations in partition bound expressions.
commit : 72647ac3bf0f11732483eac2cd9b5cf4972f2e1f
author : Tom Lane <[email protected]>
date : Mon, 28 Sep 2020 14:12:38 -0400
committer: Tom Lane <[email protected]>
date : Mon, 28 Sep 2020 14:12:38 -0400
Failure to do this can result in errors during evaluation of
the bound expression, as illustrated by the new regression test.
Back-patch to v12 where the ability for partition bounds to be
expressions was added.
Discussion: https://postgr.es/m/CAJV4CdrZ5mKuaEsRSbLf2URQ3h6iMtKD=hik8MaF5WwdmC9uZw@mail.gmail.com
M src/backend/parser/parse_utilcmd.c
M src/test/regress/expected/create_table.out
M src/test/regress/sql/create_table.sql
Remove complaints about COLLATE clauses in partition bound values.
commit : 2dfa3fea88bc951d0812a18649d801f07964c9b9
author : Tom Lane <[email protected]>
date : Mon, 28 Sep 2020 13:44:01 -0400
committer: Tom Lane <[email protected]>
date : Mon, 28 Sep 2020 13:44:01 -0400
transformPartitionBoundValue went out of its way to do the wrong
thing: there is no reason to complain about a non-matching COLLATE
clause in a partition boundary expression. We're coercing the
bound expression to the target column type as though by an
implicit assignment, and the rules for implicit assignment say
that collations can be implicitly converted.
What we *do* need to do, and the code is not doing, is apply
assign_expr_collations() to the bound expression. While this is
merely a definition disagreement, that is a bug that needs to be
back-patched, so I'll commit it separately.
Discussion: https://postgr.es/m/CAJV4CdrZ5mKuaEsRSbLf2URQ3h6iMtKD=hik8MaF5WwdmC9uZw@mail.gmail.com
M src/backend/parser/parse_utilcmd.c
M src/test/regress/expected/create_table.out
M src/test/regress/sql/create_table.sql
Cache the result of converting now() to a struct pg_tm.
commit : 0a87ddff5c83589e90de236bd55e6a19b017fe9a
author : Tom Lane <[email protected]>
date : Mon, 28 Sep 2020 12:05:03 -0400
committer: Tom Lane <[email protected]>
date : Mon, 28 Sep 2020 12:05:03 -0400
SQL operations such as CURRENT_DATE, CURRENT_TIME, LOCALTIME, and
conversion of "now" in a datetime input string have to obtain the
transaction start timestamp ("now()") as a broken-down struct pg_tm.
This is a remarkably expensive conversion, and since now() does not
change intra-transaction, it doesn't really need to be done more than
once per transaction. Introducing a simple cache provides visible
speedups in queries that compute these values many times, for example
insertion of many rows that use a default value of CURRENT_DATE.
Peter Smith, with a bit of kibitzing by me
Discussion: https://postgr.es/m/CAHut+Pu89TWjq530V2gY5O6SWi=OEJMQ_VHMt8bdZB_9JFna5A@mail.gmail.com
M src/backend/utils/adt/date.c
M src/backend/utils/adt/datetime.c
Change SHA2 implementation based on OpenSSL to use EVP digest routines
commit : e21cbb4b893b85b5f1cf203b9a77ca0d9ee671d1
author : Michael Paquier <[email protected]>
date : Mon, 28 Sep 2020 12:47:13 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 28 Sep 2020 12:47:13 +0900
The use of low-level hash routines is not recommended by upstream
OpenSSL since 2000, and pgcrypto already switched to EVP as of 5ff4a67.
Note that this also fixes a failure with SCRAM authentication when using
FIPS in OpenSSL, but as there have been few complaints about this
problem and as this causes an ABI breakage, no backpatch is done.
Author: Michael Paquier, Alessandro Gherardi
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/common/sha2_openssl.c
M src/include/common/sha2.h
Minor mop-up for List improvements.
commit : 9d299a492454f9ffdf0fd8fe1fd6303c8ddf805a
author : Tom Lane <[email protected]>
date : Sun, 27 Sep 2020 22:30:44 -0400
committer: Tom Lane <[email protected]>
date : Sun, 27 Sep 2020 22:30:44 -0400
Fix a few places that were using written-out versions of the
pg_list.h macros that commit cc99baa43 just improved, making them
also use those macros so as to gain whatever performance improvement
is to be had.
Discussion: https://postgr.es/m/CAApHDvpo1zj9KhEpU2cCRZfSM3Q6XGdhzuAS2v79PH7WJBkYVA@mail.gmail.com
M src/backend/catalog/objectaddress.c
M src/backend/commands/sequence.c
M src/backend/nodes/list.c
Improve tab-completion for DEALLOCATE.
commit : 0baf82fa0cbe98fe317483f1e1c6612fc41e1dad
author : Fujii Masao <[email protected]>
date : Mon, 28 Sep 2020 11:23:15 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 28 Sep 2020 11:23:15 +0900
Author: Naoki Nakamichi
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/tab-complete.c
Improve pg_list.h's linitial(), lsecond() and co macros
commit : cc99baa43e0ed83ac18fcbde31f2ab7274eb26cf
author : David Rowley <[email protected]>
date : Mon, 28 Sep 2020 14:47:19 +1300
committer: David Rowley <[email protected]>
date : Mon, 28 Sep 2020 14:47:19 +1300
Prior to this commit, the linitial(), lsecond(), lthird(), lfourth()
macros and their int and Oid list cousins would call their corresponding
inlined function to fetch the cell of interest. Those inline functions
were kind enough to return NULL if the particular cell did not exist.
Unfortunately, the care that these functions took was of no relevance to
the calling macros as they proceeded to directly dereference the returned
value without any regard to whether that value was NULL or not. If it had
been, we'd have segfaulted.
Of course, the fact that we would have segfaulted on misuse of these
macros just goes to prove that nobody is relying on the empty or list too
small checks. So here we just get rid of those checks completely.
The existing inline functions have been left alone as someone may be using
those directly. We just replace the call within each macro to use
list_nth_cell().
For the llast*() case we require a new list_last_cell() inline function to
get away from the multiple evaluation hazard that we'd get if we fetched
->length on the macro's parameter.
Author: David Rowley
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/CAApHDvpo1zj9KhEpU2cCRZfSM3Q6XGdhzuAS2v79PH7WJBkYVA@mail.gmail.com
M src/include/nodes/pg_list.h
Improve range checks of options for pg_test_fsync and pg_test_timing
commit : 4d29e6dbd0bb6d8c3a48d0f3c7d65dc1def1b07e
author : Michael Paquier <[email protected]>
date : Mon, 28 Sep 2020 10:13:59 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 28 Sep 2020 10:13:59 +0900
Both tools never had safeguard checks for the options provided, and it
was possible to make pg_test_fsync run an infinite amount of time or
pass down buggy values to pg_test_timing.
These behaviors have existed for a long time, with no actual complaints,
so no backpatch is done. Basic TAP tests are introduced for both tools.
Author: Michael Paquier
Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_test_fsync/.gitignore
M src/bin/pg_test_fsync/Makefile
M src/bin/pg_test_fsync/pg_test_fsync.c
A src/bin/pg_test_fsync/t/001_basic.pl
M src/bin/pg_test_timing/.gitignore
M src/bin/pg_test_timing/Makefile
M src/bin/pg_test_timing/pg_test_timing.c
A src/bin/pg_test_timing/t/001_basic.pl
Move resolution of AlternativeSubPlan choices to the planner.
commit : 41efb8340877e8ffd0023bb6b2ef22ffd1ca014d
author : Tom Lane <[email protected]>
date : Sun, 27 Sep 2020 12:51:28 -0400
committer: Tom Lane <[email protected]>
date : Sun, 27 Sep 2020 12:51:28 -0400
When commit bd3daddaf introduced AlternativeSubPlans, I had some
ambitions towards allowing the choice of subplan to change during
execution. That has not happened, or even been thought about, in the
ensuing twelve years; so it seems like a failed experiment. So let's
rip that out and resolve the choice of subplan at the end of planning
(in setrefs.c) rather than during executor startup. This has a number
of positive benefits:
* Removal of a few hundred lines of executor code, since
AlternativeSubPlans need no longer be supported there.
* Removal of executor-startup overhead (particularly, initialization
of subplans that won't be used).
* Removal of incidental costs of having a larger plan tree, such as
tree-scanning and copying costs in the plancache; not to mention
setrefs.c's own costs of processing the discarded subplans.
* EXPLAIN no longer has to print a weird (and undocumented)
representation of an AlternativeSubPlan choice; it sees only the
subplan actually used. This should mean less confusion for users.
* Since setrefs.c knows which subexpression of a plan node it's
working on at any instant, it's possible to adjust the estimated
number of executions of the subplan based on that. For example,
we should usually estimate more executions of a qual expression
than a targetlist expression. The implementation used here is
pretty simplistic, because we don't want to expend a lot of cycles
on the issue; but it's better than ignoring the point entirely,
as the executor had to.
That last point might possibly result in shifting the choice
between hashed and non-hashed EXISTS subplans in a few cases,
but in general this patch isn't meant to change planner choices.
Since we're doing the resolution so late, it's really impossible
to change any plan choices outside the AlternativeSubPlan itself.
Patch by me; thanks to David Rowley for review.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/nodeSubplan.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/jit/llvm/llvmjit_types.c
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/backend/utils/adt/ruleutils.c
M src/include/executor/execExpr.h
M src/include/executor/nodeSubplan.h
M src/include/nodes/execnodes.h
M src/include/nodes/nodes.h
M src/include/nodes/pathnodes.h
M src/include/nodes/primnodes.h
M src/test/regress/expected/insert_conflict.out
M src/test/regress/expected/subselect.out
M src/test/regress/expected/updatable_views.out
M src/test/regress/sql/subselect.sql
Further stabilize output from rolenames regression test.
commit : 3c8819955023694feeaa456ee60853d0d6d0e60a
author : Tom Lane <[email protected]>
date : Sat, 26 Sep 2020 17:42:20 -0400
committer: Tom Lane <[email protected]>
date : Sat, 26 Sep 2020 17:42:20 -0400
Commit e5209bf37 didn't quite get the job done, as I failed to
notice that chksetconfig() also needed to have its ORDER BY
extended. Per buildfarm member dory.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dory&dt=2020-09-26%2020%3A10%3A13
M src/test/modules/unsafe_tests/expected/rolenames.out
M src/test/modules/unsafe_tests/sql/rolenames.sql
Revise RelationBuildRowSecurity() to avoid memory leaks.
commit : e55f718fc429fb7971eb7351debfa5931f9811bf
author : Tom Lane <[email protected]>
date : Sat, 26 Sep 2020 16:04:06 -0400
committer: Tom Lane <[email protected]>
date : Sat, 26 Sep 2020 16:04:06 -0400
This function leaked some memory while loading qual clauses for
an RLS policy. While ordinarily negligible, that could build up
in some repeated-reload cases, as reported by Konstantin Knizhnik.
We can improve matters by borrowing the coding long used in
RelationBuildRuleLock: build stringToNode's result directly in
the target context, and remember to explicitly pfree the
input string.
This patch by no means completely guarantees zero leaks within
this function, since we have no real guarantee that the catalog-
reading subroutines it calls don't leak anything. However,
practical tests suggest that this is enough to resolve the issue.
In any case, any remaining leaks are similar to those risked by
RelationBuildRuleLock and other relcache-loading subroutines.
If we need to fix them, we should adopt a more global approach
such as that used by the RECOVER_RELATION_BUILD_MEMORY hack.
While here, let's remove the need for an expensive PG_TRY block by
using MemoryContextSetParent to reparent an initially-short-lived
context for the RLS data.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/policy.c
Fix the logical replication from HEAD to lower versions.
commit : 079d0cacf4fefc326bf776ecc2df5dba995f50de
author : Amit Kapila <[email protected]>
date : Sat, 26 Sep 2020 10:08:00 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 26 Sep 2020 10:08:00 +0530
Commit 464824323e changed the logical replication protocol to allow the
streaming of in-progress transactions and used the new version of protocol
irrespective of the server version. Use the appropriate version of the
protocol based on the server version.
Reported-by: Ashutosh Sharma
Author: Dilip Kumar
Reviewed-by: Ashutosh Sharma and Amit Kapila
Discussion: https://postgr.es/m/CAE9k0P=9OpXcNrcU5Gsvd5MZ8GFpiN833vNHzX6Uc=8+h1ft1Q@mail.gmail.com
M src/backend/replication/logical/worker.c
M src/backend/replication/pgoutput/pgoutput.c
M src/include/replication/logicalproto.h
Defer flushing of SLRU files.
commit : dee663f7843902535a15ae366cede8b4089f1144
author : Thomas Munro <[email protected]>
date : Fri, 25 Sep 2020 18:49:43 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 25 Sep 2020 18:49:43 +1200
Previously, we called fsync() after writing out individual pg_xact,
pg_multixact and pg_commit_ts pages due to cache pressure, leading to
regular I/O stalls in user backends and recovery. Collapse requests for
the same file into a single system call as part of the next checkpoint,
as we already did for relation files, using the infrastructure developed
by commit 3eb77eba. This can cause a significant improvement to
recovery performance, especially when it's otherwise CPU-bound.
Hoist ProcessSyncRequests() up into CheckPointGuts() to make it clearer
that it applies to all the SLRU mini-buffer-pools as well as the main
buffer pool. Rearrange things so that data collected in CheckpointStats
includes SLRU activity.
Also remove the Shutdown{CLOG,CommitTS,SUBTRANS,MultiXact}() functions,
because they were redundant after the shutdown checkpoint that
immediately precedes them. (I'm not sure if they were ever needed, but
they aren't now.)
Reviewed-by: Tom Lane <[email protected]> (parts)
Tested-by: Jakub Wartak <[email protected]>
Discussion: https://postgr.es/m/CA+hUKGLJ=84YT+NvhkEEDAuUtVHMfQ9i-N7k_o50JmQ6Rpj_OQ@mail.gmail.com
M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/access/transam/xlog.c
M src/backend/commands/async.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/lmgr/predicate.c
M src/backend/storage/sync/sync.c
M src/include/access/clog.h
M src/include/access/commit_ts.h
M src/include/access/multixact.h
M src/include/access/slru.h
M src/include/storage/sync.h
M src/tools/pgindent/typedefs.list
Remove custom memory allocation layer in pgcrypto
commit : ca7f8e2b86e5f15a40b67e6199d714f45a467ff1
author : Michael Paquier <[email protected]>
date : Fri, 25 Sep 2020 10:25:55 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 25 Sep 2020 10:25:55 +0900
PX_OWN_ALLOC was intended as a way to disable the use of palloc(), and
over the time new palloc() or equivalent calls have been added like in
32984d8, making this extra layer losing its original purpose. This
simplifies on the way some code paths to use palloc0() rather than
palloc() followed by memset(0).
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M contrib/pgcrypto/imath.c
M contrib/pgcrypto/internal-sha2.c
M contrib/pgcrypto/internal.c
M contrib/pgcrypto/mbuf.c
M contrib/pgcrypto/openssl.c
M contrib/pgcrypto/pgp-cfb.c
M contrib/pgcrypto/pgp-compress.c
M contrib/pgcrypto/pgp-decrypt.c
M contrib/pgcrypto/pgp-encrypt.c
M contrib/pgcrypto/pgp-mpi-internal.c
M contrib/pgcrypto/pgp-mpi.c
M contrib/pgcrypto/pgp-pubenc.c
M contrib/pgcrypto/pgp-pubkey.c
M contrib/pgcrypto/pgp.c
M contrib/pgcrypto/px-hmac.c
M contrib/pgcrypto/px.c
M contrib/pgcrypto/px.h
Fix handling of -d "connection string" in pg_dump/pg_restore.
commit : a45bc8a4f6495072bc48ad40a5aa0304979114f7
author : Tom Lane <[email protected]>
date : Thu, 24 Sep 2020 18:19:38 -0400
committer: Tom Lane <[email protected]>
date : Thu, 24 Sep 2020 18:19:38 -0400
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_restore.c
Fix two bugs in MaintainOldSnapshotTimeMapping.
commit : 55b7e2f4d78d8aa7b4a5eae9a0a810601d03c563
author : Robert Haas <[email protected]>
date : Thu, 24 Sep 2020 15:27:19 -0400
committer: Robert Haas <[email protected]>
date : Thu, 24 Sep 2020 15:27:19 -0400
The previous coding was confused about whether head_timestamp was
intended to represent the timestamp for the newest bucket in the
mapping or the oldest timestamp for the oldest bucket in the mapping.
Decide that it's intended to be the oldest one, and repair
accordingly.
To do that, we need to do two things. First, when advancing to a
new bucket, don't categorically set head_timestamp to the new
timestamp. Do this only if we're blowing out the map completely
because a lot of time has passed since we last maintained it. If
we're replacing entries one by one, advance head_timestamp by
1 minute for each; if we're filling in unused entries, don't
advance head_timestamp at all.
Second, fix the computation of how many buckets we need to advance.
The previous formula would be correct if head_timestamp were the
timestamp for the new bucket, but we're now making all the code
agree that it's the timestamp for the oldest bucket, so adjust the
formula accordingly.
This is certainly a bug fix, but I don't feel good about
back-patching it without the introspection tools added by commit
aecf5ee2bb36c597d3c6142e367e38d67816c777, and perhaps also some
actual tests. Since back-patching the introspection tools might
not attract sufficient support and since there are no automated
tests of these fixes yet, I'm just committing this to master for
now.
Patch by me, reviewed by Thomas Munro, Dilip Kumar, Hamid Akhtar.
Discussion: http://postgr.es/m/CA+TgmoY=aqf0zjTD+3dUWYkgMiNDegDLFjo+6ze=Wtpik+3XqA@mail.gmail.com
M src/backend/utils/time/snapmgr.c
Standardize the printf format for st_size
commit : c005eb00e7d878cb869854f592103f774e15d01e
author : Peter Eisentraut <[email protected]>
date : Thu, 24 Sep 2020 20:45:57 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 24 Sep 2020 20:45:57 +0200
Existing code used various inconsistent ways to printf struct stat's
st_size member. The type of that is off_t, which is in most cases a
signed 64-bit integer, so use the long long int format for it.
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xlogarchive.c
M src/bin/pg_basebackup/pg_receivewal.c
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/fe_utils/archive.c
Add new 'old_snapshot' contrib module.
commit : aecf5ee2bb36c597d3c6142e367e38d67816c777
author : Robert Haas <[email protected]>
date : Thu, 24 Sep 2020 13:55:47 -0400
committer: Robert Haas <[email protected]>
date : Thu, 24 Sep 2020 13:55:47 -0400
You can use this to view the contents of the time to XID mapping
which the server maintains when old_snapshot_threshold != -1.
Being able to view that information may be interesting for users,
and it's definitely useful for figuring out whether the mapping
is being maintained correctly. It isn't, so that will need to be
fixed in a subsequent commit.
Patch by me, reviewed by Thomas Munro, Dilip Kumar, Hamid Akhtar.
Discussion: http://postgr.es/m/CA+TgmoY=aqf0zjTD+3dUWYkgMiNDegDLFjo+6ze=Wtpik+3XqA@mail.gmail.com
M contrib/Makefile
A contrib/old_snapshot/Makefile
A contrib/old_snapshot/old_snapshot–1.0.sql
A contrib/old_snapshot/old_snapshot.control
A contrib/old_snapshot/time_mapping.c
M doc/src/sgml/contrib.sgml
M doc/src/sgml/filelist.sgml
A doc/src/sgml/oldsnapshot.sgml
Expose oldSnapshotControl definition via new header.
commit : f5ea92e8d620a1260d3427b92fa350bbf36594a2
author : Robert Haas <[email protected]>
date : Thu, 24 Sep 2020 13:32:39 -0400
committer: Robert Haas <[email protected]>
date : Thu, 24 Sep 2020 13:32:39 -0400
This makes it possible for code outside snapmgr.c to examine the
contents of this data structure. This commit does not add any code
which actually does so; a subsequent commit will make that change.
Patch by me, reviewed by Thomas Munro, Dilip Kumar, Hamid Akhtar.
Discussion: http://postgr.es/m/CA+TgmoY=aqf0zjTD+3dUWYkgMiNDegDLFjo+6ze=Wtpik+3XqA@mail.gmail.com
M src/backend/utils/time/snapmgr.c
A src/include/utils/old_snapshot.h
Doc: sync lobj.sgml's copy of testlo.c with the latter file.
commit : fc5f107a8c08f55cc8b810d6f212dd723a07fe9a
author : Tom Lane <[email protected]>
date : Thu, 24 Sep 2020 10:39:11 -0400
committer: Tom Lane <[email protected]>
date : Thu, 24 Sep 2020 10:39:11 -0400
Zhijie Hou
Discussion: https://postgr.es/m/ce2cd951fe9b448a9cda99dc1a871fb9@G08CNEXMBPEKD05.g08.fujitsu.local
M doc/src/sgml/lobj.sgml
Improve behavior of tsearch_readline(), and remove t_readline().
commit : 83b61319a16ceabc7ea7d1143dcc045826c112d2
author : Tom Lane <[email protected]>
date : Wed, 23 Sep 2020 20:26:58 -0400
committer: Tom Lane <[email protected]>
date : Wed, 23 Sep 2020 20:26:58 -0400
Commit fbeb9da22, which added the tsearch_readline APIs, left
t_readline() in place as a compatibility measure. But that function
has been unused and deprecated for twelve years now, so that seems
like enough time to remove it. Doing so, and merging t_readline's
code into tsearch_readline, aids in making several useful
improvements:
* The hard-wired 4K limit on line length in tsearch data files is
removed, by using a StringInfo buffer instead of a fixed-size buffer.
* We can buy back the per-line palloc/pfree added by 3ea7e9550
in the common case where encoding conversion is not required.
* We no longer need a separate pg_verify_mbstr call, as that
functionality was folded into encoding conversion some time ago.
(We could have done some of this stuff while keeping t_readline as a
separate API, but there seems little point, since there's no reason
for anyone to still be using t_readline directly.)
Discussion: https://postgr.es/m/[email protected]
M src/backend/tsearch/dict_thesaurus.c
M src/backend/tsearch/ts_locale.c
M src/include/tsearch/ts_locale.h
Fix missing fsync of SLRU directories.
commit : aca74843e40f43d0cceffd314aec6aa6f80e83d5
author : Thomas Munro <[email protected]>
date : Thu, 24 Sep 2020 09:26:09 +1200
committer: Thomas Munro <[email protected]>
date : Thu, 24 Sep 2020 09:26:09 +1200
Harmonize behavior by moving reponsibility for fsyncing directories down
into slru.c. In 10 and later, only the multixact directories were
missed (see commit 1b02be21), and in older branches all SLRUs were
missed.
Back-patch to all supported releases.
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGLtsTUOScnNoSMZ-2ZLv%2BwGh01J6kAo_DM8mTRq1sKdSQ%40mail.gmail.com
M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/slru.c
Improve error cursor positions for problems with partition bounds.
commit : 6b2c4e59d016ea40c42fdc66367d6463e792c125
author : Tom Lane <[email protected]>
date : Wed, 23 Sep 2020 18:04:53 -0400
committer: Tom Lane <[email protected]>
date : Wed, 23 Sep 2020 18:04:53 -0400
We failed to pass down the query string to check_new_partition_bound,
so that its attempts to provide error cursor positions were for naught;
one must have the query string to get parser_errposition to do anything.
Adjust its API to require a ParseState to be passed down.
Also, improve the logic inside check_new_partition_bound so that the
cursor points at the partition bound for the specific column causing
the issue, when one can be identified.
That part is also for naught if we can't determine the query position of
the column with the problem. Improve transformPartitionBoundValue so
that it makes sure that const-simplified partition expressions will be
properly labeled with positions. In passing, skip calling evaluate_expr
if the value is already a Const, which is surely the most common case.
Alexandra Wang, Ashwin Agrawal, Amit Langote; reviewed by Ashutosh Bapat
Discussion: https://postgr.es/m/CACiyaSopZoqssfMzgHk6fAkp01cL6vnqBdmTw2C5_KJaFR_aMg@mail.gmail.com
Discussion: https://postgr.es/m/CAJV4CdrZ5mKuaEsRSbLf2URQ3h6iMtKD=hik8MaF5WwdmC9uZw@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/backend/parser/parse_utilcmd.c
M src/backend/partitioning/partbounds.c
M src/include/partitioning/partbounds.h
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/create_table.out
Avoid possible dangling-pointer access in tsearch_readline_callback.
commit : 3ea7e9550e58c4016625c7e3e4a1340edfc9adf4
author : Tom Lane <[email protected]>
date : Wed, 23 Sep 2020 11:36:13 -0400
committer: Tom Lane <[email protected]>
date : Wed, 23 Sep 2020 11:36:13 -0400
tsearch_readline() saves the string pointer it returns to the caller
for possible use in the associated error context callback. However,
the caller will usually pfree that string sometime before it next
calls tsearch_readline(), so that there is a window where an ereport
will try to print an already-freed string.
The built-in users of tsearch_readline() happen to all do that pfree
at the bottoms of their loops, so that the window is effectively
empty for them. However, this is not documented as a requirement,
and contrib/dict_xsyn doesn't do it like that, so it seems likely
that third-party dictionaries might have live bugs here.
The practical consequences of this seem pretty limited in any case,
since production builds wouldn't clobber the freed string immediately,
besides which you'd not expect syntax errors in dictionary files
being used in production. Still, it's clearly a bug waiting to bite
somebody.
Fix by pstrdup'ing the string to be saved for the error callback,
and then pfree'ing it next time through. It's been like this for
a long time, so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/tsearch/ts_locale.c
Allow WaitLatch() to be used without a latch.
commit : 733fa9aa51c526582f100aa0d375e0eb9a6bce8b
author : Thomas Munro <[email protected]>
date : Wed, 23 Sep 2020 15:17:30 +1200
committer: Thomas Munro <[email protected]>
date : Wed, 23 Sep 2020 15:17:30 +1200
Due to flaws in commit 3347c982bab, using WaitLatch() without
WL_LATCH_SET could cause an assertion failure or crash. Repair.
While here, also add a check that the latch we're switching to belongs
to this backend, when changing from one latch to another.
Discussion: https://postgr.es/m/CA%2BhUKGK1607VmtrDUHQXrsooU%3Dap4g4R2yaoByWOOA3m8xevUQ%40mail.gmail.com
M src/backend/storage/ipc/latch.c
Simplify SortTocFromFile() by removing fixed buffer-size limit.
commit : 2e3c19462da930d1d018caa3daabca00159b4b18
author : Tom Lane <[email protected]>
date : Tue, 22 Sep 2020 16:03:32 -0400
committer: Tom Lane <[email protected]>
date : Tue, 22 Sep 2020 16:03:32 -0400
pg_restore previously coped with overlength TOC-file lines using some
complicated logic to ignore additional bufferloads. While this isn't
wrong, since we don't expect that the interesting part of a line would
run to more than a dozen or so bytes, it's more complex than it needs
to be. Use a StringInfo instead of a fixed-size buffer so that we can
process long lines as single entities and thus not need the extra
logic.
Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_backup_archiver.c
Remove arbitrary line length limit for libpq service files.
commit : c0cb87fbb6642222a99593785f77d318af06ef02
author : Tom Lane <[email protected]>
date : Tue, 22 Sep 2020 15:59:23 -0400
committer: Tom Lane <[email protected]>
date : Tue, 22 Sep 2020 15:59:23 -0400
Use a StringInfo instead of a fixed-size buffer in parseServiceInfo().
While we've not heard complaints about the existing 255-byte limit,
it certainly seems possible that complex cases could run afoul of it.
Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-connect.c
Rethink API for pg_get_line.c, one more time.
commit : 931487018c409a3102452f965ccaa48367244a41
author : Tom Lane <[email protected]>
date : Tue, 22 Sep 2020 15:55:13 -0400
committer: Tom Lane <[email protected]>
date : Tue, 22 Sep 2020 15:55:13 -0400
Further experience says that the appending behavior offered by
pg_get_line_append is useful to only a very small minority of callers.
For most, the requirement to reset the buffer after each line is just
an error-prone nuisance. Hence, invent another alternative call
pg_get_line_buf, which takes care of that detail.
Noted while reviewing a patch from Daniel Gustafsson.
Discussion: https://postgr.es/m/[email protected]
M src/bin/initdb/initdb.c
M src/common/pg_get_line.c
M src/include/common/string.h
M src/interfaces/ecpg/test/pg_regress_ecpg.c
M src/test/regress/pg_regress.c
Exclude fmgrprotos.h from pgindent processing.
commit : c4133ec169dfe47803656325dbfb8397f85a70ea
author : Tom Lane <[email protected]>
date : Tue, 22 Sep 2020 11:32:10 -0400
committer: Tom Lane <[email protected]>
date : Tue, 22 Sep 2020 11:32:10 -0400
pgindent messes up entries in this file if their names match
typedef names. While there's reason to avoid choosing conflicting
names, we have some historical exceptions, and there's no guarantee
that more duplicates won't appear in future. Since this is a derived
file anyway, there's little harm in just excluding it.
I said yesterday that all our derived files are pgindent-clean, or else
explicitly excluded from indentation, but I'd forgotten about this one.
Now that project is really done, as confirmed by a test run.
Discussion: https://postgr.es/m/[email protected]
M src/tools/pgindent/README
M src/tools/pgindent/exclude_file_patterns
Improve the error message for an inappropriate column definition list.
commit : ce90f075f0d831ca4085ba73891b7da2a2f7047e
author : Tom Lane <[email protected]>
date : Tue, 22 Sep 2020 10:49:11 -0400
committer: Tom Lane <[email protected]>
date : Tue, 22 Sep 2020 10:49:11 -0400
The existing message about "a column definition list is only allowed for
functions returning "record"" could be given in some cases where it was
fairly confusing; in particular, a function with multiple OUT parameters
*does* return record according to pg_proc. Break it down into a couple
more cases to deliver a more on-point complaint. Per complaint from
Bruce Momjian.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/parse_relation.c
M src/test/regress/expected/rangefuncs.out
M src/test/regress/sql/rangefuncs.sql
Fix a few more generator scripts to produce pgindent-clean output.
commit : f859c2ffa01d83a079652ce7d953cd61eb3d7171
author : Tom Lane <[email protected]>
date : Mon, 21 Sep 2020 13:58:26 -0400
committer: Tom Lane <[email protected]>
date : Mon, 21 Sep 2020 13:58:26 -0400
This completes the project of making all our derived files be
pgindent-clean (or else explicitly excluded from indentation),
so that no surprises result when running pgindent in a built-out
development tree.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/sort/gen_qsort_tuple.pl
M src/pl/plpgsql/src/generate-plerrcodes.pl
M src/pl/plpython/generate-spiexceptions.pl
M src/pl/tcl/generate-pltclerrcodes.pl
Copy editing: fix a bunch of misspellings and poor wording.
commit : 9436041ed848debb3d64fb5fbff6cdb35bc46d04
author : Tom Lane <[email protected]>
date : Mon, 21 Sep 2020 12:43:42 -0400
committer: Tom Lane <[email protected]>
date : Mon, 21 Sep 2020 12:43:42 -0400
99% of this is docs, but also a couple of comments. No code changes.
Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/btree.sgml
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/alter_statistics.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_rewind.sgml
M doc/src/sgml/ref/pgbench.sgml
M doc/src/sgml/ref/reindex.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
M doc/src/sgml/runtime.sgml
M doc/src/sgml/sources.sgml
M src/backend/access/gin/README
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/partition_join.out
M src/test/regress/sql/partition_join.sql
Standardize order of use strict and use warnings in Perl code
commit : 80fc96eceb817d89cdd11c02c8ab913c8d060a3e
author : Peter Eisentraut <[email protected]>
date : Mon, 21 Sep 2020 15:22:41 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 21 Sep 2020 15:22:41 +0200
The standard order in PostgreSQL and other code is use strict first,
but some code was uselessly inconsistent about this.
M doc/src/sgml/generate-errcodes-table.pl
M src/backend/parser/check_keywords.pl
M src/backend/storage/lmgr/generate-lwlocknames.pl
M src/backend/utils/generate-errcodes.pl
M src/pl/plpgsql/src/generate-plerrcodes.pl
M src/pl/plpython/generate-spiexceptions.pl
M src/pl/tcl/generate-pltclerrcodes.pl
Fix checksum calculation in the new sorting GiST build.
commit : c47a240fe6db8293e2de1565233faee40afa64b6
author : Heikki Linnakangas <[email protected]>
date : Mon, 21 Sep 2020 14:50:07 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 21 Sep 2020 14:50:07 +0300
Since we're bypassing the buffer manager, we need to call
PageSetChecksumInplace() directly. As reported by Justin Pryzby.
In the passing, add RelationOpenSmgr() calls before all smgrwrite() and
smgrextend() calls. Tom added one before the first smgrextend() call in
commit c2bb287025, which seems to be enough, but let's play it safe and
do it before each one. That's how it's done in the similar code in
nbtsort.c, too.
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/backend/access/gist/gistbuild.c
Fix new GIST build code to work under CLOBBER_CACHE_ALWAYS.
commit : c2bb287025189d93c8df16b28f2a3d9d07d25655
author : Tom Lane <[email protected]>
date : Sun, 20 Sep 2020 17:08:49 -0400
committer: Tom Lane <[email protected]>
date : Sun, 20 Sep 2020 17:08:49 -0400
Can't say if this fixes *all* cases, but at least we get through
the "point" regression test now, which hyrax's last run did not.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hyrax&dt=2020-09-19%2021%3A27%3A23
M src/backend/access/gist/gistbuild.c
Fix whitespace
commit : 3d13867a2c0b0b9cd1215107d4d6aa94e975b33b
author : Peter Eisentraut <[email protected]>
date : Sun, 20 Sep 2020 14:42:54 +0200
committer: Peter Eisentraut <[email protected]>
date : Sun, 20 Sep 2020 14:42:54 +0200
M src/backend/storage/ipc/procarray.c
Remove precedence hacks no longer needed without postfix operators.
commit : 28a61fc6c58fbfb83e416411f55e7d6ec47279ff
author : Tom Lane <[email protected]>
date : Sat, 19 Sep 2020 15:11:26 -0400
committer: Tom Lane <[email protected]>
date : Sat, 19 Sep 2020 15:11:26 -0400
It's no longer necessary to assign explicit precedences to GENERATED,
NULL_P, PRESERVE, or STRIP_P.
Actually, we don't need to assign precedence to IDENT either; that was
really just there to govern the behavior of target_el's "a_expr IDENT"
production, which no longer ends with that terminal. However, it seems
like a good idea to continue to do so, because it provides a reference
point for a precedence level that we can assign to other unreserved
keywords that lack a natural precedence level.
Research by Peter Eisentraut and John Naylor; comment rewrite by me.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/gram.y
Remove unused parameters
commit : 8354e7b27ebec8bf78088e22b5b9c1caa46db8e1
author : Peter Eisentraut <[email protected]>
date : Tue, 25 Aug 2020 07:24:15 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 25 Aug 2020 07:24:15 +0200
Remove various unused parameters in pg_dump code. These have all
become unused over time or were never used.
Discussion: https://www.postgresql.org/message-id/flat/511bb100-f829-ba21-2f10-9f952ec06ead%402ndquadrant.com
M src/bin/pg_dump/dumputils.c
M src/bin/pg_dump/dumputils.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_tar.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
Code review for dynahash change.
commit : ff28809feb442eabd303955277f05cd16d9c6d8a
author : Thomas Munro <[email protected]>
date : Sat, 19 Sep 2020 15:39:48 +1200
committer: Thomas Munro <[email protected]>
date : Sat, 19 Sep 2020 15:39:48 +1200
Commit be0a6666 left behind a comment about the order of some tests that
didn't make sense without the expensive division, and in fact we might
as well change the order to one that fails more cheaply most of the time
as a micro-optimization. Also, remove the "+ 1" applied to max_bucket,
to drop an instruction and match the original behavior. Per review
from Tom Lane.
Discussion: https://postgr.es/m/VI1PR0701MB696044FC35013A96FECC7AC8F62D0%40VI1PR0701MB6960.eurprd07.prod.outlook.com
M src/backend/utils/hash/dynahash.c
Remove large fill factor support from dynahash.c.
commit : be0a6666656ec3f68eb7d8e7abab5139fcd47012
author : Thomas Munro <[email protected]>
date : Sat, 19 Sep 2020 11:28:34 +1200
committer: Thomas Munro <[email protected]>
date : Sat, 19 Sep 2020 11:28:34 +1200
Since ancient times we have had support for a fill factor (maximum load
factor) to be set for a dynahash hash table, but:
1. It was an integer, whereas for in-memory hash tables interesting
load factor targets are probably somewhere near the 0.75-1.0 range.
2. It was implemented in a way that performed an expensive division
operation that regularly showed up in profiles.
3. We are not aware of anyone ever having used a non-default value.
Therefore, remove support, effectively fixing it at 1.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Tomas Vondra <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Reviewed-by: David Rowley <[email protected]>
Discussion: https://postgr.es/m/VI1PR0701MB696044FC35013A96FECC7AC8F62D0%40VI1PR0701MB6960.eurprd07.prod.outlook.com
M src/backend/utils/hash/dynahash.c
M src/include/utils/hsearch.h
Allow most keywords to be used as column labels without requiring AS.
commit : 06a7c3154f5bfad65549810cc84f0e3a77b408bf
author : Tom Lane <[email protected]>
date : Fri, 18 Sep 2020 16:46:26 -0400
committer: Tom Lane <[email protected]>
date : Fri, 18 Sep 2020 16:46:26 -0400
Up to now, if you tried to omit "AS" before a column label in a SELECT
list, it would only work if the column label was an IDENT, that is not
any known keyword. This is rather unfriendly considering that we have
so many keywords and are constantly growing more. In the wake of commit
1ed6b8956 it's possible to improve matters quite a bit.
We'd originally tried to make this work by having some of the existing
keyword categories be allowed without AS, but that didn't work too well,
because each category contains a few special cases that don't work
without AS. Instead, invent an entirely orthogonal keyword property
"can be bare column label", and mark all keywords that way for which
we don't get shift/reduce errors by doing so.
It turns out that of our 450 current keywords, all but 39 can be made
bare column labels, improving the situation by over 90%. This number
might move around a little depending on future grammar work, but it's
a pretty nice improvement.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M doc/src/sgml/generate-keywords-table.pl
M doc/src/sgml/keywords.sgml
M doc/src/sgml/queries.sgml
M src/backend/parser/check_keywords.pl
M src/backend/parser/gram.y
M src/backend/parser/scan.l
M src/backend/utils/adt/misc.c
M src/common/keywords.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/common/keywords.h
M src/include/parser/kwlist.h
M src/interfaces/ecpg/preproc/keywords.c
pg_surgery: Try to stabilize regression tests.
commit : 0811f766fd740018a72e222521553f8b22e7b3d6
author : Robert Haas <[email protected]>
date : Fri, 18 Sep 2020 13:26:48 -0400
committer: Robert Haas <[email protected]>
date : Fri, 18 Sep 2020 13:26:48 -0400
According to buildfarm member sungazer, the behavior of VACUUM can be
unstable in these tests even if we prevent autovacuum from running on
the tables in question, apparently because even a manual vacuum can
behave differently depending on whether anything else is running that
holds back the global xmin. So use a temporary table instead, which
as of commit a7212be8b9e0885ee769e8c55f99ef742cda487b enables
vacuuming using a more aggressive cutoff.
This approach can't be used for the regression test that involves a
materialized view, but that test doesn't run vacuum, so it shouldn't
be prone to this particular failure mode.
Analysis by Tom Lane. Patch by Ashutosh Sharma and me.
Discussion: http://postgr.es/m/[email protected]
M contrib/pg_surgery/expected/heap_surgery.out
M contrib/pg_surgery/sql/heap_surgery.sql
Update file header comments for logical/relation.c.
commit : 24fb35e11148b2b2cae2bddc5f2d6c566675c2b2
author : Amit Kapila <[email protected]>
date : Fri, 18 Sep 2020 10:14:30 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 18 Sep 2020 10:14:30 +0530
Author: Amit Langote
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CA+HiwqE20oZoix13JyCeALpTf_SmjarZWtBFe5sND6zz+iupAw@mail.gmail.com
M src/backend/replication/logical/relation.c
Fix comments in heapam.c.
commit : 0d32511eca5aec205cb6b609638ea67129ef6665
author : Amit Kapila <[email protected]>
date : Fri, 18 Sep 2020 09:40:04 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 18 Sep 2020 09:40:04 +0530
After commits 85f6b49c2c and 3ba59ccc89, we can allow parallel inserts
which was earlier not possible as parallel group members won't conflict
for relation extension and page lock. In those commits, we forgot to
update comments at few places.
Author: Amit Kapila
Reviewed-by: Robert Haas and Dilip Kumar
Backpatch-through: 13
Discussion: https://postgr.es/m/CAFiTN-tMrQh5FFMPx5aWJ+1gi1H6JxktEhq5mDwCHgnEO5oBkA@mail.gmail.com
M src/backend/access/heap/heapam.c
Try to stabilize output from rolenames regression test.
commit : e5209bf37a2752f0f7a34309ba53c210d21dbbb6
author : Tom Lane <[email protected]>
date : Thu, 17 Sep 2020 21:02:55 -0400
committer: Tom Lane <[email protected]>
date : Thu, 17 Sep 2020 21:02:55 -0400
It's not quite clear why commit 45b980570 has resulted in
some instability here, though interference from concurrent
autovacuum runs seems like a reasonable guess. What is
clear is that the output ordering of the test queries is
underdetermined for no very good reason. Extend the
ORDER BY keys in hopes of fixing the buildfarm.
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/unsafe_tests/expected/rolenames.out
M src/test/modules/unsafe_tests/sql/rolenames.sql
Remove support for postfix (right-unary) operators.
commit : 1ed6b895634ce0dc5fd4bd040e87252b32182cba
author : Tom Lane <[email protected]>
date : Thu, 17 Sep 2020 19:38:05 -0400
committer: Tom Lane <[email protected]>
date : Thu, 17 Sep 2020 19:38:05 -0400
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/deparse.c
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/alter_extension.sgml
M doc/src/sgml/ref/alter_operator.sgml
M doc/src/sgml/ref/alter_opfamily.sgml
M doc/src/sgml/ref/comment.sgml
M doc/src/sgml/ref/create_opclass.sgml
M doc/src/sgml/ref/create_operator.sgml
M doc/src/sgml/ref/drop_operator.sgml
M doc/src/sgml/syntax.sgml
M doc/src/sgml/typeconv.sgml
M doc/src/sgml/xoper.sgml
M src/backend/catalog/namespace.c
M src/backend/catalog/pg_operator.c
M src/backend/commands/operatorcmds.c
M src/backend/nodes/print.c
M src/backend/parser/gram.y
M src/backend/parser/parse_expr.c
M src/backend/parser/parse_oper.c
M src/backend/utils/adt/ruleutils.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_upgrade/check.c
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_operator.h
M src/include/parser/parse_oper.h
M src/test/regress/expected/create_operator.out
M src/test/regress/expected/opr_sanity.out
M src/test/regress/sql/create_operator.sql
M src/test/regress/sql/opr_sanity.sql
M src/tutorial/complex.source
M src/tutorial/syscat.source
Remove factorial operators, leaving only the factorial() function.
commit : 76f412ab310554acb970a0b73c8d1f37f35548c6
author : Tom Lane <[email protected]>
date : Thu, 17 Sep 2020 16:17:27 -0400
committer: Tom Lane <[email protected]>
date : Thu, 17 Sep 2020 16:17:27 -0400
The "!" operator is our only built-in postfix operator. Remove it,
on the way to removal of grammar support for postfix operators.
There is also a "!!" prefix operator, but since it's been marked
deprecated for most of its existence, we might as well remove it too.
Also zap the SQL alias function numeric_fac(), which seems to have
equally little reason to live.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/func.sgml
M src/include/catalog/catversion.h
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/create_operator.out
M src/test/regress/expected/numeric.out
M src/test/regress/sql/create_operator.sql
M src/test/regress/sql/numeric.sql
Further improve pgindent's list of file exclusions.
commit : 74d4608f506ba423e4c84609c1c0e03e1e92cb91
author : Tom Lane <[email protected]>
date : Thu, 17 Sep 2020 14:16:18 -0400
committer: Tom Lane <[email protected]>
date : Thu, 17 Sep 2020 14:16:18 -0400
I despair of people keeping the README file's notes in sync with
the actual exclusion list, so move the notes into the exclusion file.
Adjust the pgindent script to explicitly ignore comments in the file,
just in case (though it's hard to believe any would match filenames).
Extend the list so that it doesn't process any files we'd not wish it to
even in a fully-built-out development directory. (There are still a
couple of derived files that it wants to reformat, but fixing the
generator scripts for those seems like fit material for a separate patch.)
Discussion: https://postgr.es/m/[email protected]
M src/tools/pgindent/README
M src/tools/pgindent/exclude_file_patterns
M src/tools/pgindent/pgindent
Improve common/logging.c's support for multiple verbosity levels.
commit : 99175141c9254318e5894ac30b9fdb622612acda
author : Tom Lane <[email protected]>
date : Thu, 17 Sep 2020 12:52:18 -0400
committer: Tom Lane <[email protected]>
date : Thu, 17 Sep 2020 12:52:18 -0400
Instead of hard-wiring specific verbosity levels into the option
processing of client applications, invent pg_logging_increase_verbosity()
and encourage clients to implement --verbose by calling that. Then,
the common convention that more -v's gets you more verbosity just works.
In particular, this allows resurrection of the debug-grade messages that
have long existed in pg_dump and its siblings. They were unreachable
before this commit due to lack of a way to select PG_LOG_DEBUG logging
level. (It appears that they may have been unreachable for some time
before common/logging.c was introduced, too, so I'm not specifically
blaming cc8d41511 for the oversight. One reason for thinking that is
that it's now apparent that _allocAH()'s message needs a null-pointer
guard. Testing might have failed to reveal that before 96bf88d52.)
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_restore.sgml
M src/bin/pg_archivecleanup/pg_archivecleanup.c
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_dump/pg_restore.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pgbench/pgbench.c
M src/common/logging.c
M src/include/common/logging.h
Update parallel BTree scan state when the scan keys can't be satisfied.
commit : b7f2dd959a5082540adbeee1dea0c0b1c154374f
author : Amit Kapila <[email protected]>
date : Thu, 17 Sep 2020 15:16:46 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 17 Sep 2020 15:16:46 +0530
For parallel btree scan to work for array of scan keys, it should reach
BTPARALLEL_DONE state once for every distinct combination of array keys.
This is required to ensure that the parallel workers don't try to seize
blocks at the same time for different scan keys. We missed to update this
state when we discovered that the scan keys can't be satisfied.
Author: James Hunter
Reviewed-by: Amit Kapila
Tested-by: Justin Pryzby
Backpatch-through: 10, where it was introduced
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/nbtree/nbtsearch.c
Allow CURRENT_ROLE where CURRENT_USER is accepted
commit : 45b9805706fdc726906fc9187c9a7b38c559755c
author : Peter Eisentraut <[email protected]>
date : Thu, 17 Sep 2020 11:39:28 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 17 Sep 2020 11:39:28 +0200
In the particular case of GRANTED BY, this is specified in the SQL
standard. Since in PostgreSQL, CURRENT_ROLE is equivalent to
CURRENT_USER, and CURRENT_USER is already supported here, adding
CURRENT_ROLE is trivial. The other cases are PostgreSQL extensions,
but for the same reason it also makes sense there.
Reviewed-by: Vik Fearing <[email protected]>
Reviewed-by: Asif Rehman <[email protected]>
Reviewed-by: Alvaro Herrera <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/f2feac44-b4c5-f38f-3699-2851d6a76dc9%402ndquadrant.com
M doc/src/sgml/ref/alter_aggregate.sgml
M doc/src/sgml/ref/alter_collation.sgml
M doc/src/sgml/ref/alter_conversion.sgml
M doc/src/sgml/ref/alter_database.sgml
M doc/src/sgml/ref/alter_domain.sgml
M doc/src/sgml/ref/alter_event_trigger.sgml
M doc/src/sgml/ref/alter_foreign_data_wrapper.sgml
M doc/src/sgml/ref/alter_foreign_table.sgml
M doc/src/sgml/ref/alter_function.sgml
M doc/src/sgml/ref/alter_group.sgml
M doc/src/sgml/ref/alter_language.sgml
M doc/src/sgml/ref/alter_large_object.sgml
M doc/src/sgml/ref/alter_materialized_view.sgml
M doc/src/sgml/ref/alter_opclass.sgml
M doc/src/sgml/ref/alter_operator.sgml
M doc/src/sgml/ref/alter_opfamily.sgml
M doc/src/sgml/ref/alter_policy.sgml
M doc/src/sgml/ref/alter_procedure.sgml
M doc/src/sgml/ref/alter_publication.sgml
M doc/src/sgml/ref/alter_role.sgml
M doc/src/sgml/ref/alter_routine.sgml
M doc/src/sgml/ref/alter_schema.sgml
M doc/src/sgml/ref/alter_sequence.sgml
M doc/src/sgml/ref/alter_server.sgml
M doc/src/sgml/ref/alter_statistics.sgml
M doc/src/sgml/ref/alter_subscription.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/alter_tablespace.sgml
M doc/src/sgml/ref/alter_tsconfig.sgml
M doc/src/sgml/ref/alter_tsdictionary.sgml
M doc/src/sgml/ref/alter_type.sgml
M doc/src/sgml/ref/alter_user.sgml
M doc/src/sgml/ref/alter_user_mapping.sgml
M doc/src/sgml/ref/alter_view.sgml
M doc/src/sgml/ref/create_policy.sgml
M doc/src/sgml/ref/create_schema.sgml
M doc/src/sgml/ref/create_tablespace.sgml
M doc/src/sgml/ref/create_user_mapping.sgml
M doc/src/sgml/ref/drop_owned.sgml
M doc/src/sgml/ref/drop_user_mapping.sgml
M doc/src/sgml/ref/grant.sgml
M doc/src/sgml/ref/reassign_owned.sgml
M doc/src/sgml/ref/revoke.sgml
M src/backend/parser/gram.y
M src/backend/utils/adt/acl.c
M src/include/nodes/parsenodes.h
M src/test/modules/unsafe_tests/expected/rolenames.out
M src/test/modules/unsafe_tests/sql/rolenames.sql
Add support for building GiST index by sorting.
commit : 16fa9b2b30a357b4aea982bd878ec2e5e002dbcc
author : Heikki Linnakangas <[email protected]>
date : Thu, 17 Sep 2020 11:33:40 +0300
committer: Heikki Linnakangas <[email protected]>
date : Thu, 17 Sep 2020 11:33:40 +0300
This adds a new optional support function to the GiST access method:
sortsupport. If it is defined, the GiST index is built by sorting all data
to the order defined by the sortsupport's comparator function, and packing
the tuples in that order to GiST pages. This is similar to how B-tree
index build works, and is much faster than inserting the tuples one by
one. The resulting index is smaller too, because the pages are packed more
tightly, upto 'fillfactor'. The normal build method works by splitting
pages, which tends to lead to more wasted space.
The quality of the resulting index depends on how good the opclass-defined
sort order is. A good order preserves locality of the input data.
As the first user of this facility, add 'sortsupport' function to the
point_ops opclass. It sorts the points in Z-order (aka Morton Code), by
interleaving the bits of the X and Y coordinates.
Author: Andrey Borodin
Reviewed-by: Pavel Borisov, Thomas Munro
Discussion: https://www.postgresql.org/message-id/1A36620E-CAD8-4267-9067-FB31385E7C0D%40yandex-team.ru
M doc/src/sgml/gist.sgml
M src/backend/access/gist/gistbuild.c
M src/backend/access/gist/gistproc.c
M src/backend/access/gist/gistutil.c
M src/backend/access/gist/gistvalidate.c
M src/backend/access/transam/xloginsert.c
M src/backend/utils/sort/sortsupport.c
M src/backend/utils/sort/tuplesort.c
M src/include/access/gist.h
M src/include/access/gist_private.h
M src/include/access/xloginsert.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_amproc.dat
M src/include/catalog/pg_proc.dat
M src/include/utils/sortsupport.h
M src/include/utils/tuplesort.h
M src/test/regress/expected/create_index.out
doc: Apply more consistently <productname> markup for OpenSSL
commit : 089da3c4778fdc1931f721a265caa0c6fca38584
author : Michael Paquier <[email protected]>
date : Thu, 17 Sep 2020 16:33:22 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 17 Sep 2020 16:33:22 +0900
OpenSSL was quoted in inconsistent ways in many places of the docs,
sometimes with <application>, <productname> or just nothing.
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/sslinfo.sgml
Improve tab completion of IMPORT FOREIGN SCHEMA in psql
commit : 7307df16a05984396649db33df0faf6d840cd223
author : Michael Paquier <[email protected]>
date : Thu, 17 Sep 2020 11:49:29 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 17 Sep 2020 11:49:29 +0900
It is not possible to get a list of foreign schemas as the server is not
known, so this provides instead a list of local schemas, which is more
useful than nothing if using a loopback server or having schema names
matching in the local and remote servers.
Author: Jeff Janes
Reviewed-by: Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/CAMkU=1wr7Roj41q-XiJs=Uyc2xCmHhcGGy7J-peJQK-e+w=ghw@mail.gmail.com
M src/bin/psql/tab-complete.c
Teach walsender to update its process title for replication commands.
commit : babef40c9a999949abe0ae8e82240cac3f154237
author : Tom Lane <[email protected]>
date : Wed, 16 Sep 2020 21:06:50 -0400
committer: Tom Lane <[email protected]>
date : Wed, 16 Sep 2020 21:06:50 -0400
Because the code path taken for SQL commands executed in a walsender
will update the process title, we pretty much have to update the
title for replication commands as well. Otherwise, the title shows
"idle" for the rest of a logical walsender's lifetime once it's
executed any SQL command.
Playing with this, I confirm that a walsender now typically spends
most of its life reporting
walsender postgres [local] START_REPLICATION
Considering this in isolation, it might be better to have it say
walsender postgres [local] sending replication data
However, consistency with the other cases seems to be a stronger
argument.
In passing, remove duplicative pgstat_report_activity call.
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/walsender.c
Improve formatting of create_help.pl and plperl_opmask.pl output.
commit : add105840b673ab3949abc2568da0d4f2fd496c5
author : Tom Lane <[email protected]>
date : Wed, 16 Sep 2020 20:31:19 -0400
committer: Tom Lane <[email protected]>
date : Wed, 16 Sep 2020 20:31:19 -0400
Adjust the whitespace in the emitted files so that it matches
what pgindent would do. This makes the generated files look
like they match project style, and avoids confusion if someone
does run pgindent on the generated files.
Also, add probes.h to pgindent's exclusion list, because it can
confuse pgindent, plus there's not much point in processing it.
Daniel Gustafsson, additional fixes by me
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/create_help.pl
M src/pl/plperl/plperl_opmask.pl
M src/tools/pgindent/README
M src/tools/pgindent/exclude_file_patterns
Fix bogus completion tag usage in walsender
commit : 07082b08cc5d3c378d22c105c65841ec0952e3ed
author : Alvaro Herrera <[email protected]>
date : Wed, 16 Sep 2020 13:04:38 -0300
committer: Alvaro Herrera <[email protected]>
date : Wed, 16 Sep 2020 13:04:38 -0300
Since commit fd5942c18f97 (2012, 9.3-era), walsender has been sending
completion tags for certain replication commands twice -- and they're
not even consistent. Apparently neither libpq nor JDBC have a problem
with it, but it's not kosher. Fix by remove the EndCommand() call in
the common code path for them all, and inserting specific calls to
EndReplicationCommand() specifically in those places where it's needed.
EndReplicationCommand() is a new simple function to send the completion
tag for replication commands. Do this instead of sending a generic
SELECT completion tag for them all, which was also pretty bogus (if
innocuous). While at it, change StartReplication() to use
EndReplicationCommand() instead of pg_puttextmessage().
In commit 2f9661311b83, I failed to realize that replication commands
are not close-enough kin of regular SQL commands, so the
DROP_REPLICATION_SLOT tag I added is undeserved and a type pun. Take it
out.
Backpatch to 13, where the latter commit appeared. The duplicate tag
has been sent since 9.3, but since nothing is broken, it doesn't seem
worth fixing.
Per complaints from Tom Lane.
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/walsender.c
M src/backend/tcop/dest.c
M src/include/tcop/cmdtaglist.h
M src/include/tcop/dest.h
Centralize setup of SIGQUIT handling for postmaster child processes.
commit : 44fc6e259b799f9924de206eeddc1b1fcbcd172f
author : Tom Lane <[email protected]>
date : Wed, 16 Sep 2020 16:04:36 -0400
committer: Tom Lane <[email protected]>
date : Wed, 16 Sep 2020 16:04:36 -0400
We decided that the policy established in commit 7634bd4f6 for
the bgwriter, checkpointer, walwriter, and walreceiver processes,
namely that they should accept SIGQUIT at all times, really ought
to apply uniformly to all postmaster children. Therefore, get
rid of the duplicative and inconsistent per-process code for
establishing that signal handler and removing SIGQUIT from BlockSig.
Instead, make InitPostmasterChild do it.
The handler set up by InitPostmasterChild is SignalHandlerForCrashExit,
which just summarily does _exit(2). In interactive backends, we
almost immediately replace that with quickdie, since we would prefer
to try to tell the client that we're dying. However, this patch is
changing the behavior of autovacuum (both launcher and workers), as
well as walsenders. Those processes formerly also used quickdie,
but AFAICS that was just mindless copy-and-paste: they don't have
any interactive client that's likely to benefit from being told this.
The stats collector continues to be an outlier, in that it thinks
SIGQUIT means normal exit. That should probably be changed for
consistency, but there's another patch set where that's being
dealt with, so I didn't do so here.
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/bgwriter.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/pgarch.c
M src/backend/postmaster/postmaster.c
M src/backend/postmaster/startup.c
M src/backend/postmaster/walwriter.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walsender.c
M src/backend/tcop/postgres.c
M src/backend/utils/init/miscinit.c
Don't fetch partition check expression during InitResultRelInfo.
commit : 2000b6c10aa6777929f1a8b613f30426bb90f849
author : Tom Lane <[email protected]>
date : Wed, 16 Sep 2020 14:28:11 -0400
committer: Tom Lane <[email protected]>
date : Wed, 16 Sep 2020 14:28:11 -0400
Since there is only one place that actually needs the partition check
expression, namely ExecPartitionCheck, it's better to fetch it from
the relcache there. In this way we will never fetch it at all if
the query never has use for it, and we still fetch it just once when
we do need it.
The reason for taking an interest in this is that if the relcache
doesn't already have the check expression cached, fetching it
requires obtaining AccessShareLock on the partition root. That
means that operations that look like they should only touch the
partition itself will also take a lock on the root. In particular
we observed that TRUNCATE on a partition may take a lock on the
partition's root, contributing to a deadlock situation in parallel
pg_restore.
As written, this patch does have a small cost, which is that we
are microscopically reducing efficiency for the case where a partition
has an empty check expression. ExecPartitionCheck will be called,
and will go through the motions of setting up and checking an empty
qual, where before it would not have been called at all. We could
avoid that by adding a separate boolean flag to track whether there
is a partition expression to test. However, this case only arises
for a default partition with no siblings, which surely is not an
interesting case in practice. Hence adding complexity for it
does not seem like a good trade-off.
Amit Langote, per a suggestion by me
Discussion: https://postgr.es/m/VI1PR03MB31670CA1BD9625C3A8C5DD05EB230@VI1PR03MB3167.eurprd03.prod.outlook.com
M src/backend/commands/copy.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/execReplication.c
M src/backend/executor/nodeModifyTable.c
M src/backend/replication/logical/worker.c
M src/include/nodes/execnodes.h
Fix amcheck child check pg_upgrade bug.
commit : aac80bfcddf7df3fbd1eb73e8a386115d922c714
author : Peter Geoghegan <[email protected]>
date : Wed, 16 Sep 2020 10:42:30 -0700
committer: Peter Geoghegan <[email protected]>
date : Wed, 16 Sep 2020 10:42:30 -0700
Commit d114cc53 overlooked the fact that pg_upgrade'd B-Tree indexes
have leaf page high keys whose offset numbers do not match the one from
the copy of the tuple one level up (the copy stored with a downlink for
leaf page's right sibling page). This led to false positive reports of
corruption from bt_index_parent_check() when it was called to verify a
pg_upgrade'd index.
To fix, skip comparing the offset number on pg_upgrade'd B-Tree indexes.
Author: Anastasia Lubennikova <[email protected]>
Author: Peter Geoghegan <[email protected]>
Reported-By: Andrew Bille <[email protected]>
Diagnosed-By: Anastasia Lubennikova <[email protected]>
Bug: #16619
Discussion: https://postgr.es/m/[email protected]
Backpatch: 13-, where child check was enhanced.
M contrib/amcheck/verify_nbtree.c
Avoid unnecessary recursion to child tables in ALTER TABLE SET NOT NULL.
commit : e5fac1cb1941e4adbcb88206f914e2035e5cccf2
author : Tom Lane <[email protected]>
date : Wed, 16 Sep 2020 13:38:26 -0400
committer: Tom Lane <[email protected]>
date : Wed, 16 Sep 2020 13:38:26 -0400
If a partitioned table's column is already marked NOT NULL, there is
no need to examine its partitions, because we can rely on previous
DDL to have enforced that the child columns are NOT NULL as well.
(Unfortunately, the same cannot be said for traditional inheritance,
so for now we have to restrict the optimization to partitioned tables.)
Hence, we may skip recursing to child tables in this situation.
The reason this case is worth worrying about is that when pg_dump dumps
a partitioned table having a primary key, it will include the requisite
NOT NULL markings in the CREATE TABLE commands, and then add the
primary key as a separate step. The primary key addition generates a
SET NOT NULL as a subcommand, just to be sure. So the situation where
a SET NOT NULL is redundant does arise in the real world.
Skipping the recursion does more than just save a few cycles: it means
that a command such as "ALTER TABLE ONLY partition_parent ADD PRIMARY
KEY" will take locks only on the partition parent table, not on the
partitions. It turns out that parallel pg_restore is effectively
assuming that that's true, and has little choice but to do so because
the dependencies listed for such a TOC entry don't include the
partitions. pg_restore could thus issue this ALTER while data restores
on the partitions are still in progress. Taking unnecessary locks on
the partitions not only hurts concurrency, but can lead to actual
deadlock failures, as reported by Domagoj Smoljanovic.
(A contributing factor in the deadlock is that TRUNCATE on a child
partition wants a non-exclusive lock on the parent. This seems
likewise unnecessary, but the fix for it is more invasive so we
won't consider back-patching it. Fortunately, getting rid of one
of these two poor behaviors is enough to remove the deadlock.)
Although support for partitioned primary keys came in with v11,
this patch is dependent on the SET NOT NULL refactoring done by
commit f4a3fdfbd, so we can only patch back to v12.
Patch by me; thanks to Alvaro Herrera and Amit Langote for review.
Discussion: https://postgr.es/m/VI1PR03MB31670CA1BD9625C3A8C5DD05EB230@VI1PR03MB3167.eurprd03.prod.outlook.com
M src/backend/commands/tablecmds.c
Fix bogus cache-invalidation logic in logical replication worker.
commit : 3d65b0593c5578014f62e09d4008006f1783f64d
author : Tom Lane <[email protected]>
date : Wed, 16 Sep 2020 12:07:31 -0400
committer: Tom Lane <[email protected]>
date : Wed, 16 Sep 2020 12:07:31 -0400
The code recorded cache invalidation events by zeroing the "localreloid"
field of affected cache entries. However, it's possible for an inval
event to occur even while we have the entry open and locked. So an
ill-timed inval could result in "cache lookup failed for relation 0"
errors, if the worker's code tried to use the cleared field. We can
fix that by creating a separate bool field to record whether the entry
needs to be revalidated. (In the back branches, cram the bool into
what had been padding space, to avoid an ABI break in the somewhat
unlikely event that any extension is looking at this struct.)
Also, rearrange the logic in logicalrep_rel_open so that it
does the right thing in cases where table_open would fail.
We should retry the lookup by name in that case, but we didn't.
The real-world impact of this is probably small. In the first place,
the error conditions are very low probability, and in the second place,
the worker would just exit and get restarted. We only noticed because
in a CLOBBER_CACHE_ALWAYS build, the failure can occur repeatedly,
preventing the worker from making progress. Nonetheless, it's clearly
a bug, and it impedes a useful type of testing; so back-patch to v10
where this code was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/relation.c
M src/include/replication/logicalrelation.h
Add leader_pid field into the example of file_fdw for csvlog.
commit : e568ed0eb07239b7e53d948565ebaeb6f379630f
author : Fujii Masao <[email protected]>
date : Wed, 16 Sep 2020 18:47:39 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 16 Sep 2020 18:47:39 +0900
Commit b8fdee7d0c added leader_pid field into csvlog,
but forgot to update the example of file_fdw for csvlog.
Author: Yuta Katsuragi
M doc/src/sgml/file-fdw.sgml
Avoid retrieval of CHECK constraints and DEFAULT exprs in data-only dump
commit : 5423853feebd30772b7ff9b306885dcb02b79e76
author : Michael Paquier <[email protected]>
date : Wed, 16 Sep 2020 16:26:50 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 16 Sep 2020 16:26:50 +0900
Those extra queries are not necessary when doing a data-only dump. With
this change, this means that the dependencies between CHECK/DEFAULT and
the parent table are not tracked anymore for a data-only dump. However,
these dependencies are only used for the schema generation and we have
never guaranteed that a dump can be reloaded if a CHECK constraint uses
a custom function whose behavior changes when loading the data, like
when using cross-table references in the CHECK function.
Author: Julien Rouhaud
Reviewed-by: Daniel Gustafsson, Michael Paquier
Discussion: https://postgr.es/m/20200712054850.GA92357@nol
M src/bin/pg_dump/pg_dump.c
Change LogicalTapeSetBlocks() to use nBlocksWritten.
commit : c8aeaf3ab31edeedf1791e37c74bcedf61a916ed
author : Jeff Davis <[email protected]>
date : Tue, 15 Sep 2020 21:34:05 -0700
committer: Jeff Davis <[email protected]>
date : Tue, 15 Sep 2020 21:34:05 -0700
Previously, it was based on nBlocksAllocated to account for tapes with
open write buffers that may not have made it to the BufFile yet.
That was unnecessary, because callers do not need to get the number of
blocks while a tape has an open write buffer; and it also conflicted
with the preallocation logic added for HashAgg.
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
M src/backend/utils/sort/logtape.c
HashAgg: release write buffers sooner by rewinding tape.
commit : 3bd35d4f516adfc492360b20e72911949c961e47
author : Jeff Davis <[email protected]>
date : Tue, 15 Sep 2020 21:16:31 -0700
committer: Jeff Davis <[email protected]>
date : Tue, 15 Sep 2020 21:16:31 -0700
This was an oversight. The purpose of 7fdd919ae7 was to avoid keeping
tape buffers around unnecessisarily, but HashAgg didn't rewind early
enough.
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
Fix initialization of RelationSyncEntry for streaming transactions.
commit : 69bd60672af63eaa8b19cdcea175df5ff172e80e
author : Amit Kapila <[email protected]>
date : Wed, 16 Sep 2020 07:45:44 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 16 Sep 2020 07:45:44 +0530
In commit 464824323e, for each RelationSyncEntry we maintained the list
of xids (streamed_txns) for which we have already sent the schema. This
helps us to track when to send the schema to the downstream node for
replication of streaming transactions. Before this list got initialized,
we were processing invalidation messages which access this list and led
to an assertion failure.
In passing, clean up the nearby code:
* Initialize the list of xids with NIL instead of NULL which is our usual
coding practice.
* Remove the MemoryContext switch for creating a RelationSyncEntry in dynahash.
Diagnosed-by: Amit Kapila and Tom Lane
Author: Amit Kapila
Reviewed-by: Tom Lane and Dilip Kumar
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/pgoutput/pgoutput.c
Optimize compactify_tuples function
commit : 19c60ad69a91f346edf66996b2cf726f594d3d2b
author : David Rowley <[email protected]>
date : Wed, 16 Sep 2020 13:22:20 +1200
committer: David Rowley <[email protected]>
date : Wed, 16 Sep 2020 13:22:20 +1200
This function could often be seen in profiles of vacuum and could often
be a significant bottleneck during recovery. The problem was that a qsort
was performed in order to sort an array of item pointers in reverse offset
order so that we could use that to safely move tuples up to the end of the
page without overwriting the memory of yet-to-be-moved tuples. i.e. we
used to compact the page starting at the back of the page and move towards
the front. The qsort that this required could be expensive for pages with
a large number of tuples.
In this commit, we take another approach to tuple compactification.
Now, instead of sorting the remaining item pointers array we first check
if the array is presorted and only memmove() the tuples that need to be
moved. This presorted check can be done very cheaply in the calling
functions when the array is being populated. This presorted case is very
fast.
When the item pointer array is not presorted we must copy tuples that need
to be moved into a temp buffer before copying them back into the page
again. This differs from what we used to do here as we're now copying the
tuples back into the page in reverse line pointer order. Previously we
left the existing order alone. Reordering the tuples results in an
increased likelihood of hitting the pre-sorted case the next time around.
Any newly added tuple which consumes a new line pointer will also maintain
the correct sort order of tuples in the page which will also result in the
presorted case being hit the next time. Only consuming an unused line
pointer can cause the order of tuples to go out again, but that will be
corrected next time the function is called for the page.
Benchmarks have shown that the non-presorted case is at least equally as
fast as the original qsort method even when the page just has a few
tuples. As the number of tuples becomes larger the new method maintains
its performance whereas the original qsort method became much slower when
the number of tuples on the page became large.
Author: David Rowley
Reviewed-by: Thomas Munro
Tested-by: Jakub Wartak
Discussion: https://postgr.es/m/CA+hUKGKMQFVpjr106gRhwk6R-nXv0qOcTreZuQzxgpHESAL6dw@mail.gmail.com
M src/backend/storage/page/bufpage.c
Fix use-after-free bug with event triggers in an extension script
commit : ced138e8cbac7f5a840de8679e9882665478c680
author : Alvaro Herrera <[email protected]>
date : Tue, 15 Sep 2020 21:03:14 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 15 Sep 2020 21:03:14 -0300
ALTER TABLE commands in an extension script are added to an event
trigger command list; but starting with commit b5810de3f4 they do so in
a memory context that's too short-lived, so when execution ends and time
comes to use the entries, they've already been freed.
(This would also be a problem with ALTER TABLE commands in a
multi-command query string, but these serendipitously end in
PortalContext -- which probably explains why it took so long for this to
be reported.)
Fix by using the memory context specifically set for that, instead.
Backpatch to 13, where the aforementioned commit appeared.
Reported-by: Philippe Beaudoin
Author: Jehan-Guillaume de Rorthais <[email protected]>
Discussion: https://postgr.es/m/20200902193715.6e0269d4@firost
M src/backend/commands/event_trigger.c
M src/test/modules/test_extensions/Makefile
M src/test/modules/test_extensions/expected/test_extensions.out
M src/test/modules/test_extensions/sql/test_extensions.sql
A src/test/modules/test_extensions/test_ext_evttrig–1.0–2.0.sql
A src/test/modules/test_extensions/test_ext_evttrig–1.0.sql
A src/test/modules/test_extensions/test_ext_evttrig.control
Report resource usage at the end of recovery
commit : 10a5b35a0061d9747ac7594c5e6faa0513276593
author : David Rowley <[email protected]>
date : Wed, 16 Sep 2020 11:25:46 +1200
committer: David Rowley <[email protected]>
date : Wed, 16 Sep 2020 11:25:46 +1200
Reporting this has been rather useful in some recent recovery speedup
work. It also seems like something that will be useful to the average DBA
too.
Author: David Rowley
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/CAApHDvqYVORiZxq2xPvP6_ndmmsTkvr6jSYv4UTNaFa5i1kd%3DQ%40mail.gmail.com
M src/backend/access/transam/xlog.c
Allow incremental sorts for windowing functions
commit : 62e221e1c01e3985d2b8e4b68c364f8486c327ab
author : David Rowley <[email protected]>
date : Tue, 15 Sep 2020 23:44:45 +1200
committer: David Rowley <[email protected]>
date : Tue, 15 Sep 2020 23:44:45 +1200
This expands on the work done in d2d8a229b and allows incremental sort
to be considered during create_window_paths().
Author: David Rowley
Reviewed-by: Daniel Gustafsson, Tomas Vondra
Discussion: https://postgr.es/m/CAApHDvoOHobiA2x13NtWnWLcTXYj9ddpCkv9PnAJQBMegYf_xw%40mail.gmail.com
M src/backend/optimizer/plan/planner.c
M src/test/regress/expected/window.out
M src/test/regress/sql/window.sql
Fix compiler warning
commit : fe4f36bcde182d57dee5dba898076aba5d826515
author : David Rowley <[email protected]>
date : Tue, 15 Sep 2020 15:07:57 +1200
committer: David Rowley <[email protected]>
date : Tue, 15 Sep 2020 15:07:57 +1200
Introduced in 0aa8f7640.
MSVC warned about performing 32-bit bit shifting when it appeared like we
might like a 64-bit result. We did, but it just so happened that none of
the calls to this function could have caused the 32-bit shift to overflow.
Here we just cast the constant to int64 to make the compiler happy.
Discussion: https://postgr.es/m/CAApHDvofA_vsrpC13mq_hZyuye5B-ssKEaer04OouXYCO5-uXQ@mail.gmail.com
M src/backend/utils/adt/dbsize.c
Make walsenders show their replication commands in pg_stat_activity.
commit : f560209c6e99e000f3f6c972f34f1d9dc3857f25
author : Tom Lane <[email protected]>
date : Mon, 14 Sep 2020 12:35:00 -0400
committer: Tom Lane <[email protected]>
date : Mon, 14 Sep 2020 12:35:00 -0400
A walsender process that has executed a SQL command left the text of
that command in pg_stat_activity.query indefinitely, which is quite
confusing if it's in RUNNING state but not doing that query. An easy
and useful fix is to treat replication commands as if they were SQL
queries, and show them in pg_stat_activity according to the same rules
as for regular queries. While we're at it, it seems also sensible to
set debug_query_string, allowing error logging and debugging to see
the replication command.
While here, clean up assorted silliness in exec_replication_command:
* The SQLCmd path failed to restore CurrentMemoryContext to the caller's
value, and failed to delete the temp context created in this routine.
It's only through great good fortune that these oversights did not
result in long-term memory leaks or other problems. It seems cleaner
to code SQLCmd as a separate early-exit path, so do it like that.
* Remove useless duplicate call of SnapBuildClearExportedSnapshot().
* replication_scanner_finish() was never called.
None of those things are significant enough to merit a backpatch,
so this is for HEAD only.
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/walsender.c
Fix interpolation in test name.
commit : 47a3a1c3d4981c85f1819bb495ec4ff70f1b9456
author : Noah Misch <[email protected]>
date : Sun, 13 Sep 2020 23:29:51 -0700
committer: Noah Misch <[email protected]>
date : Sun, 13 Sep 2020 23:29:51 -0700
A pre-commit review had reported the problem, but the fix reached only
v10 and earlier. Back-patch to v11.
Discussion: https://postgr.es/m/[email protected]
M src/test/recovery/t/020_archive_status.pl
Fix typos.
commit : 95233011a08fcfa872f4505b295bc501b2844a08
author : Fujii Masao <[email protected]>
date : Mon, 14 Sep 2020 14:16:07 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 14 Sep 2020 14:16:07 +0900
Author: Naoki Nakamichi
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/procarray.c
M src/backend/utils/time/snapmgr.c
Make index_set_state_flags() transactional
commit : 83158f74d3ab8dd48aed92e84b6207562992d13d
author : Michael Paquier <[email protected]>
date : Mon, 14 Sep 2020 13:56:41 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 14 Sep 2020 13:56:41 +0900
3c84046 is the original commit that introduced index_set_state_flags(),
where the presence of SnapshotNow made necessary the use of an in-place
update. SnapshotNow has been removed in 813fb03, so there is no actual
reasons to not make this operation transactional.
Note that while making the operation more robust, using a transactional
operation in this routine was not strictly necessary as there was no use
case for it yet. However, some future features are going to need a
transactional behavior, like support for CREATE/DROP INDEX CONCURRENTLY
with partitioned tables, where indexes in a partition tree need to have
all their pg_index.indis* flags updated in the same transaction to make
the operation stable to the end-user by keeping partition trees
consistent, even with a failure mid-flight.
REINDEX CONCURRENTLY uses already transactional updates when swapping
the old and new indexes, making this change more consistent with the
index-swapping logic.
Author: Michael Paquier
Reviewed-by: Anastasia Lubennikova
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/index.c
Message fixes and style improvements
commit : 3e0242b24c3c059870890644b69d6c4491a45651
author : Peter Eisentraut <[email protected]>
date : Mon, 14 Sep 2020 06:42:07 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 14 Sep 2020 06:42:07 +0200
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/transam/xlog.c
M src/backend/commands/opclasscmds.c
M src/backend/commands/tablecmds.c
M src/backend/libpq/hba.c
M src/backend/nodes/params.c
M src/backend/parser/gram.y
M src/backend/parser/parse_clause.c
M src/backend/replication/basebackup.c
M src/backend/storage/ipc/procarray.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/fmgr/fmgr.c
M src/backend/utils/misc/guc.c
M src/backend/utils/sort/sharedtuplestore.c
M src/bin/pg_verifybackup/parse_manifest.c
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/bin/pg_verifybackup/t/005_bad_manifest.pl
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/scripts/vacuumdb.c
M src/fe_utils/archive.c
M src/test/modules/test_misc/t/001_constraint_validation.pl
M src/test/regress/expected/alter_generic.out
M src/test/regress/expected/jsonb_jsonpath.out
M src/test/regress/expected/limit.out
Avoid useless allocations for information of dumpable objects in pg_dump/
commit : ac673a1aaff197f3e01f7bac69da0dd700854e13
author : Michael Paquier <[email protected]>
date : Mon, 14 Sep 2020 10:44:23 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 14 Sep 2020 10:44:23 +0900
If there are no objects of a certain type, there is no need to do an
allocation for a set of DumpableObject items. The previous coding did
an allocation of 1 byte instead as per the fallback of pg_malloc() in
the event of an allocation size of zero. This assigns NULL instead for
a set of dumpable objects.
A similar rule already applied to findObjectByOid(), so this makes the
code more defensive as we would just fail with a pointer dereference
instead of attempting to use some incorrect data if a non-existing,
positive, OID is given by a caller of this function.
Author: Daniel Gustafsson
Reviewed-by: Julien Rouhaud, Ranier Vilela
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/common.c
Use the properly transformed RangeVar for expandTableLikeClause().
commit : 19f5a37b9fc48a12c77edafb732543875da2f4a3
author : Tom Lane <[email protected]>
date : Sun, 13 Sep 2020 12:51:21 -0400
committer: Tom Lane <[email protected]>
date : Sun, 13 Sep 2020 12:51:21 -0400
transformCreateStmt() adjusts the transformed statement's RangeVar
to specify the target schema explicitly, for the express reason
of making sure that auxiliary statements derived by parse
transformation operate on the right table. But the refactoring
I did in commit 502898192 got this wrong and passed the untransformed
RangeVar to expandTableLikeClause(). This could lead to assertion
failures or weird misbehavior if the wrong table was accessed.
Per report from Alexander Lakhin. Like the previous patch, back-patch
to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/tcop/utility.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Fix inconsistency in determining the timestamp of the db statfile.
commit : 03c7f1f37a1f3bb69e348f915296af2289257e89
author : Amit Kapila <[email protected]>
date : Sat, 12 Sep 2020 08:02:54 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 12 Sep 2020 08:02:54 +0530
We use the timestamp of the global statfile if we are not able to
determine it for a particular database in case the entry for that database
doesn't exist. However, we were using it even when the statfile is
corrupt.
As there is no user reported issue and it is not clear if there is any
impact of this on actual application so decided not to backpatch.
Reported-by: Amit Kapila
Author: Amit Kapila
Reviewed-by: Sawada Masahiko, Magnus Hagander and Alvaro Herrera
Discussion: https://postgr.es/m/CAA4eK1J3oTJKyVq6v7K4d3jD+vtnruG9fHRib6UuWWsrwAR6Aw@mail.gmail.com
M src/backend/postmaster/pgstat.c
Remove unused function declaration in logicalproto.h.
commit : ddd5f6d2609b25f252d81dc43746347e0113bfbe
author : Amit Kapila <[email protected]>
date : Sat, 12 Sep 2020 07:47:53 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 12 Sep 2020 07:47:53 +0530
In the passing, fix a typo in pgoutput.c.
Reported-by: Tomas Vondra
Author: Tomas Vondra
Reviewed-by: Dilip Kumar
Discussion: https://postgr.es/m/20200909084353.pncuclpbwlr7vylh@development
M src/backend/replication/pgoutput/pgoutput.c
M src/include/replication/logicalproto.h
logtape.c: do not preallocate for tapes when sorting
commit : 07589649639410032df281e98469db88a0b86271
author : Jeff Davis <[email protected]>
date : Fri, 11 Sep 2020 17:10:02 -0700
committer: Jeff Davis <[email protected]>
date : Fri, 11 Sep 2020 17:10:02 -0700
The preallocation logic is only useful for HashAgg, so disable it when
sorting.
Also, adjust an out-of-date comment.
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-Wzn_o7tE2+hRVvwSFghRb75AJ5g-nqGzDUqLYMexjOAe=g@mail.gmail.com
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
M src/backend/utils/sort/logtape.c
M src/backend/utils/sort/tuplesort.c
M src/include/utils/logtape.h
Accept SIGQUIT during error recovery in auxiliary processes.
commit : 7634bd4f6d38bdef1fe442df5c2e0da73f1f90f4
author : Tom Lane <[email protected]>
date : Fri, 11 Sep 2020 16:01:28 -0400
committer: Tom Lane <[email protected]>
date : Fri, 11 Sep 2020 16:01:28 -0400
The bgwriter, checkpointer, walwriter, and walreceiver processes
claimed to allow SIGQUIT "at all times". In reality SIGQUIT
would get re-blocked during error recovery, because we didn't
update the actual signal mask immediately, so sigsetjmp() would
save and reinstate a mask that includes SIGQUIT.
This appears to be simply a coding oversight. There's never a
good reason to hold off SIGQUIT in these processes, because it's
going to just call _exit(2) which should be safe enough, especially
since the postmaster is going to tear down shared memory afterwards.
Hence, stick in PG_SETMASK() calls to install the modified BlockSig
mask immediately.
Also try to improve the comments around sigsetjmp blocks. Most of
them were just referencing postgres.c, which is misleading because
actually postgres.c manages the signals differently.
No back-patch, since there's no evidence that this is causing any
problems in the field.
Discussion: https://postgr.es/m/CALDaNm1d1hHPZUg3xU4XjtWBOLCrA+-2cJcLpw-cePZ=GgDVfA@mail.gmail.com
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/bgwriter.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/walwriter.c
M src/backend/replication/walreceiver.c
psql: Display stats target of extended statistics
commit : 3c99230b4f0d10c9eac5f4efdd2394eccb2af3a0
author : Alvaro Herrera <[email protected]>
date : Fri, 11 Sep 2020 16:15:47 -0300
committer: Alvaro Herrera <[email protected]>
date : Fri, 11 Sep 2020 16:15:47 -0300
The stats target can be set since commit d06215d03, but wasn't shown by
psql.
Author: Justin Pryzby <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Reviewed-by: Georgios Kokolatos <[email protected]>
Reviewed-by: Tatsuro Yamada <[email protected]>
M src/bin/psql/describe.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Log a message when resorting to SIGKILL during shutdown/crash recovery.
commit : 10095ca634fb39d78cfae8000489a19f4f4e27ef
author : Tom Lane <[email protected]>
date : Fri, 11 Sep 2020 12:24:46 -0400
committer: Tom Lane <[email protected]>
date : Fri, 11 Sep 2020 12:24:46 -0400
Currently, no useful trace is left in the logs when the postmaster
is forced to use SIGKILL to shut down children that failed to respond
to SIGQUIT. Some questions were raised about how often that scenario
happens in the buildfarm, so let's add a LOG-level message showing
that it happened.
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/postmaster.c
Don't run atexit callbacks during signal exits from ProcessStartupPacket.
commit : 6693a96b329ec46f1df916f2a28d640cc9a9977d
author : Tom Lane <[email protected]>
date : Fri, 11 Sep 2020 12:20:16 -0400
committer: Tom Lane <[email protected]>
date : Fri, 11 Sep 2020 12:20:16 -0400
Although 58c6feccf fixed the case for SIGQUIT, we were still calling
proc_exit() from signal handlers for SIGTERM and timeout failures in
ProcessStartupPacket. Fortunately, at the point where that code runs,
we haven't yet connected to shared memory in any meaningful way, so
there is nothing we need to undo in shared memory. This means it
should be safe to use _exit(1) here, ie, not run any atexit handlers
but also inform the postmaster that it's not a crash exit.
To make sure nobody breaks the "nothing to undo" expectation, add
a cross-check that no on-shmem-exit or before-shmem-exit handlers
have been registered yet when we finish using these signal handlers.
This change is simple enough that maybe it could be back-patched,
but I won't risk that right now.
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/postmaster.c
M src/backend/storage/ipc/ipc.c
M src/include/storage/ipc.h
Update copyright year
commit : 6a68a233cef86d81b35fb4c1c21a094768875af2
author : Alvaro Herrera <[email protected]>
date : Fri, 11 Sep 2020 12:53:25 -0300
committer: Alvaro Herrera <[email protected]>
date : Fri, 11 Sep 2020 12:53:25 -0300
Thinko in 40b3e2c201af.
Reported-by: "Wang, Shenhao" <[email protected]>
Discussion: https://postgr.es/m/ed98706b82694b57a8c0d339a10732aa@G08CNEXMBPEKD06.g08.fujitsu.local
M src/backend/catalog/pg_cast.c
Skip empty transaction stream in test_decoding.
commit : 0ba5181c00eb0216bdfd9afbf3d680fee67d34b3
author : Amit Kapila <[email protected]>
date : Fri, 11 Sep 2020 10:00:01 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 11 Sep 2020 10:00:01 +0530
We were decoding empty transactions via streaming APIs added in commit
45fdc9738b even when the user used the option 'skip-empty-xacts'. The APIs
makes no effort to skip empty xacts under the assumption that we will
never try to stream such transactions. However, that is not true because
we can pick to stream a transaction that has change messages for
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT and we don't send such messages to
downstream rather they are just to update the internal state. So, we need
to skip such xacts when plugin uses the option 'skip-empty-xacts'.
Diagnosed-By: Amit Kapila
Author: Dilip Kumar
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAA4eK1+OqgFNZkf7=ETe_y5ntjgDk3T0wcdkd4Sot_u1hySGfw@mail.gmail.com
M contrib/test_decoding/Makefile
A contrib/test_decoding/expected/concurrent_stream.out
M contrib/test_decoding/expected/stream.out
A contrib/test_decoding/specs/concurrent_stream.spec
M contrib/test_decoding/test_decoding.c
Print WAL logical message contents in pg_waldump
commit : 9f1cf97bb5387a6243c8a6c9725616ef7447962e
author : Alvaro Herrera <[email protected]>
date : Thu, 10 Sep 2020 19:37:02 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 10 Sep 2020 19:37:02 -0300
This helps debuggability when looking at WAL streams containing logical
messages.
Author: Ashutosh Bapat <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/CAExHW5sWx49rKmXbg5H1Xc1t+nRv9PaYKQmgw82HPt6vWDVmDg@mail.gmail.com
M src/backend/access/rmgrdesc/logicalmsgdesc.c
M src/backend/replication/logical/message.c
M src/include/replication/message.h
Use _exit(2) for SIGQUIT during ProcessStartupPacket, too.
commit : 58c6feccfae1321ea4ff16c535c6f3fb90a07d69
author : Tom Lane <[email protected]>
date : Thu, 10 Sep 2020 12:06:26 -0400
committer: Tom Lane <[email protected]>
date : Thu, 10 Sep 2020 12:06:26 -0400
Bring the signal handling for startup-packet collection into line
with the policy established in commits bedadc732 and 8e19a8264,
namely don't risk running atexit callbacks when handling SIGQUIT.
Ideally, we'd not do so for SIGTERM or timeout interrupts either,
but that change seems a bit too risky for the back branches.
For now, just improve the comments in this area to describe the risk.
Also relocate where BackendInitialize re-disables these interrupts,
to minimize the code span where they're active. This doesn't buy
a whole lot of safety, but it can't hurt.
In passing, rename startup_die() to remove confusion about whether
it is for the startup process.
Like the previous commits, back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/postmaster.c
New contrib module, pg_surgery, with heap surgery functions.
commit : 34a947ca13e52389eb3e81f1ceb38e592ad53110
author : Robert Haas <[email protected]>
date : Thu, 10 Sep 2020 11:10:55 -0400
committer: Robert Haas <[email protected]>
date : Thu, 10 Sep 2020 11:10:55 -0400
Sometimes it happens that the visibility information for a tuple
becomes corrupted, either due to bugs in the database software or
external factors. Provide a function heap_force_kill() that can
be used to truncate such dead tuples to dead line pointers, and
a function heap_force_freeze() that can be used to overwrite the
visibility information in such a way that the tuple becomes
all-visible.
These functions are unsafe, in that you can easily use them to
corrupt a database that was not previously corrupted, and you can
use them to further corrupt an already-corrupted database or to
destroy data. The documentation accordingly cautions against
casual use. However, in some cases they permit recovery of data
that would otherwise be very difficult to recover, or to allow a
system to continue to function when it would otherwise be difficult
to do so.
Because we may want to add other functions for performing other
kinds of surgery in the future, the new contrib module is called
pg_surgery rather than something specific to these functions. I
proposed back-patching this so that it could be more easily used
by people running existing releases who are facing these kinds of
problems, but that proposal did not attract enough support, so
no back-patch for now.
Ashutosh Sharma, reviewed and tested by Andrey M. Borodin,
M. Beena Emerson, Masahiko Sawada, Rajkumar Raghuwanshi,
Asim Praveen, and Mark Dilger, and somewhat revised by me.
Discussion: http://postgr.es/m/CA+TgmoZW1fsU-QUNCRUQMGUygBDPVeOTLCqRdQZch=EYZnctSA@mail.gmail.com
M contrib/Makefile
A contrib/pg_surgery/.gitignore
A contrib/pg_surgery/Makefile
A contrib/pg_surgery/expected/heap_surgery.out
A contrib/pg_surgery/heap_surgery.c
A contrib/pg_surgery/pg_surgery–1.0.sql
A contrib/pg_surgery/pg_surgery.control
A contrib/pg_surgery/sql/heap_surgery.sql
M doc/src/sgml/contrib.sgml
M doc/src/sgml/filelist.sgml
A doc/src/sgml/pgsurgery.sgml
M src/tools/pgindent/typedefs.list
Remove unused parameter
commit : c02767d2415d17329e2998a9a182fc240a90b554
author : Peter Eisentraut <[email protected]>
date : Thu, 10 Sep 2020 16:13:19 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 10 Sep 2020 16:13:19 +0200
Apparently, this was never used when
introduced (3dad73e71f08abd86564d5090a58ca71740e07e0).
Discussion: https://www.postgresql.org/message-id/flat/511bb100-f829-ba21-2f10-9f952ec06ead%402ndquadrant.com
M src/bin/pg_basebackup/receivelog.c
Add libpq's openssl dependencies to pkg-config file
commit : beff361bc1edc24ee5f8b2073a1e5e4c92ea66eb
author : Peter Eisentraut <[email protected]>
date : Thu, 10 Sep 2020 15:55:31 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 10 Sep 2020 15:55:31 +0200
Add libssl and libcrypto to libpq.pc's Requires.private. This allows
static linking to work if those libssl or libcrypto themselves have
dependencies in their *.private fields, such as -lz in some cases.
Reported-by: Sandro Mani <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/interfaces/libpq/Makefile
doc: Remove buggy ICU collation from documentation
commit : 4fff515e9efff0053d50fc0b811612a4844b1c65
author : Peter Eisentraut <[email protected]>
date : Thu, 10 Sep 2020 15:31:09 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 10 Sep 2020 15:31:09 +0200
We have had multiple reports that point to the
'@colReorder=latn-digit' collation customization being buggy. We have
reported this to ICU and are waiting for a fix. In the meantime,
remove references to this from the documentation and replace it by
another reordering example. Apparently, many users have been picking
up this example specifically from the documentation.
Author: Jehan-Guillaume de Rorthais <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/153201618542.1404.3611626898935613264%40wrigleys.postgresql.org
M doc/src/sgml/charset.sgml
Add more tests for EXTRACT of date type
commit : 540612fa469eaae3345ede7a160b146dd903e7ee
author : Peter Eisentraut <[email protected]>
date : Thu, 10 Sep 2020 14:52:36 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 10 Sep 2020 14:52:36 +0200
EXTRACT of date type is implemented as a wrapper around EXTRACT of
timestamp, so the code is already tested there. But the externally
visible behavior of EXTRACT on date is not recorded anywhere. Since
there is some discussion about reimplementing or refactoring some of
this, add some more explicit tests of EXTRACT on date, similar in
structure to existing EXTRACT tests on other data types.
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/test/regress/expected/date.out
M src/test/regress/expected/expressions.out
M src/test/regress/sql/date.sql
Fix title in reference section
commit : 994a58407c89724917b05a1564dcf1f2f7f3ea9a
author : Magnus Hagander <[email protected]>
date : Thu, 10 Sep 2020 14:15:26 +0200
committer: Magnus Hagander <[email protected]>
date : Thu, 10 Sep 2020 14:15:26 +0200
Reported-by: Robert Kahlert
Author: Daniel Gustafsson
M doc/src/sgml/biblio.sgml
Clean up some code and comments in partbounds.c.
commit : 3857f98f143166f96c6f6359810ab4241ab4590f
author : Etsuro Fujita <[email protected]>
date : Thu, 10 Sep 2020 18:00:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Thu, 10 Sep 2020 18:00:00 +0900
Do some minor cleanup for commit c8434d64c: 1) remove a useless
assignment (in normal builds) and 2) improve comments a little.
Back-patch to v13 where the aforementioned commit went in.
Author: Etsuro Fujita
Reviewed-by: Alvaro Herrera
Discussion: https://postgr.es/m/CAPmGK16yCd2R4=bQ4g8N2dT9TtA5ZU+qNmJ3LPc_nypbNy4_2A@mail.gmail.com
M src/backend/partitioning/partbounds.c
doc: Fix some grammar and inconsistencies
commit : aad546bd0a51059aafe44e8c4a783afa1557d1d9
author : Michael Paquier <[email protected]>
date : Thu, 10 Sep 2020 15:50:19 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 10 Sep 2020 15:50:19 +0900
Some comments are fixed while on it.
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_subscription.sgml
M doc/src/sgml/ref/pg_verifybackup.sgml
M doc/src/sgml/sources.sgml
M src/backend/replication/logical/relation.c
M src/backend/storage/lmgr/proc.c
Fix rd_firstRelfilenodeSubid for nailed relations, in parallel workers.
commit : fe4d022c8e171ba3a9165bd55fa6b2ca3a40fa4e
author : Noah Misch <[email protected]>
date : Wed, 9 Sep 2020 18:50:24 -0700
committer: Noah Misch <[email protected]>
date : Wed, 9 Sep 2020 18:50:24 -0700
Move applicable code out of RelationBuildDesc(), which nailed relations
bypass. Non-assert builds experienced no known problems. Back-patch to
v13, where commit c6b92041d38512a4176ed76ad06f713d2e6c01a8 introduced
rd_firstRelfilenodeSubid.
Kyotaro Horiguchi. Reported by Justin Pryzby.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/cache/relcache.c
M src/test/regress/expected/reindex_catalog.out
M src/test/regress/sql/reindex_catalog.sql
Make archiver's SIGQUIT handler exit via _exit().
commit : bedadc73220f7b09f29a4741dccd143a21a08dda
author : Tom Lane <[email protected]>
date : Wed, 9 Sep 2020 15:32:34 -0400
committer: Tom Lane <[email protected]>
date : Wed, 9 Sep 2020 15:32:34 -0400
Commit 8e19a8264 changed the SIGQUIT handlers of almost all server
processes not to run atexit callbacks. The archiver process was
skipped, perhaps because it's not connected to shared memory; but
it's just as true here that running atexit callbacks in a signal
handler is unsafe. So let's make it work like the rest.
In HEAD and v13, we can use the common SignalHandlerForCrashExit
handler. Before that, just tweak pgarch_exit to use _exit(2)
explicitly.
Like the previous commit, back-patch to all supported branches.
Kyotaro Horiguchi, back-patching by me
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/pgarch.c
Expose internal function for converting int64 to numeric
commit : 0aa8f764088ea0f36620ae2955fa6c54ec736c46
author : Peter Eisentraut <[email protected]>
date : Wed, 9 Sep 2020 20:16:28 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 9 Sep 2020 20:16:28 +0200
Existing callers had to take complicated detours via
DirectFunctionCall1(). This simplifies a lot of code.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M contrib/btree_gist/btree_numeric.c
M contrib/jsonb_plperl/jsonb_plperl.c
M src/backend/utils/adt/cash.c
M src/backend/utils/adt/dbsize.c
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/adt/numeric.c
M src/include/utils/numeric.h
Doc: adjust documentation related to index support functions.
commit : a273dcc6fd34c8b6aa290fafa45e516ccd8d907d
author : Tom Lane <[email protected]>
date : Wed, 9 Sep 2020 12:00:49 -0400
committer: Tom Lane <[email protected]>
date : Wed, 9 Sep 2020 12:00:49 -0400
Commit 15cb2bd27 neglected to make the running text match the
tables, leaving the reader with the strong impression that
we cannot count. Also, don't drop an unrelated para between
a table and the para describing it.
M doc/src/sgml/xindex.sgml
Minor fixes in docs and error messages.
commit : f3e1e6619655bf219131d93ec85348cad75c7f12
author : Tom Lane <[email protected]>
date : Wed, 9 Sep 2020 11:53:39 -0400
committer: Tom Lane <[email protected]>
date : Wed, 9 Sep 2020 11:53:39 -0400
Alexander Lakhin
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/bgworker.sgml
M doc/src/sgml/btree.sgml
M doc/src/sgml/intarray.sgml
M src/backend/replication/backup_manifest.c
M src/backend/utils/misc/guc.c
Add missing quote in docs
commit : 60df530c571a8a5fbae82dc82e4931bb6325ddc0
author : Magnus Hagander <[email protected]>
date : Wed, 9 Sep 2020 12:20:53 +0200
committer: Magnus Hagander <[email protected]>
date : Wed, 9 Sep 2020 12:20:53 +0200
Mistake in commit 68b603e1a9.
Reported-by: Ian Barwick
M doc/src/sgml/file-fdw.sgml
Add some more numeric test coverage
commit : fe2bf8f81096ea1b7b2fc58501309fc5e41ccd47
author : Peter Eisentraut <[email protected]>
date : Wed, 9 Sep 2020 09:58:12 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 9 Sep 2020 09:58:12 +0200
max(numeric) wasn't tested at all, min(numeric) was only used by some
unrelated tests. Add explicit tests with the other numeric aggregate
functions.
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Check default partitions constraints while descending
commit : f481d2823297e692e9d44c9edfd18e626e768b0c
author : Alvaro Herrera <[email protected]>
date : Tue, 8 Sep 2020 19:35:15 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 8 Sep 2020 19:35:15 -0300
Partitioning tuple route code assumes that the partition chosen while
descending the partition hierarchy is always the correct one. This is
true except when the partition is the default partition and another
partition has been added concurrently: the partition constraint changes
and we don't recheck it. This can lead to tuples mistakenly being added
to the default partition that should have been rejected.
Fix by rechecking the default partition constraint while descending the
hierarchy.
An isolation test based on the reproduction steps described by Hao Wu
(with tweaks for extra coverage) is included.
Backpatch to 12, where this bug came in with 898e5e3290a7.
Reported by: Hao Wu <[email protected]>
Author: Amit Langote <[email protected]>
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/CA+HiwqFqBmcSSap4sFnCBUEL_VfOMmEKaQ3gwUhyfa4c7J_-nA@mail.gmail.com
Discussion: https://postgr.es/m/DM5PR0501MB3910E97A9EDFB4C775CF3D75A42F0@DM5PR0501MB3910.namprd05.prod.outlook.com
M src/backend/executor/execPartition.c
A src/test/isolation/expected/partition-concurrent-attach.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/partition-concurrent-attach.spec
Install an error check into cancel_before_shmem_exit().
commit : c9ae5cbb8834dd440cdfa7058c03443f98436284
author : Tom Lane <[email protected]>
date : Tue, 8 Sep 2020 15:54:25 -0400
committer: Tom Lane <[email protected]>
date : Tue, 8 Sep 2020 15:54:25 -0400
Historically, cancel_before_shmem_exit() just silently did nothing
if the specified callback wasn't the top-of-stack. The folly of
ignoring this case was exposed by the bugs fixed in 303640199 and
bab150045, so let's make it throw elog(ERROR) instead.
There is a decent argument to be made that PG_ENSURE_ERROR_CLEANUP
should use some separate infrastructure, so it wouldn't break if
something inside the guarded code decides to register a new
before_shmem_exit callback. However, a survey of the surviving
uses of before_shmem_exit() and PG_ENSURE_ERROR_CLEANUP doesn't
show any plausible conflicts of that sort today, so for now we'll
forgo the extra complexity. (It will almost certainly become
necessary if anyone ever wants to wrap PG_ENSURE_ERROR_CLEANUP
around arbitrary user-defined actions, though.)
No backpatch, since this is developer support not a production issue.
Bharath Rupireddy, per advice from Andres Freund, Robert Haas, and myself
Discussion: https://postgr.es/m/CALj2ACWk7j4F2v2fxxYfrroOF=AdFNPr1WsV+AGtHAFQOqm_pw@mail.gmail.com
M src/backend/storage/ipc/ipc.c
Fix autovacuum cancellation.
commit : 5871f09c98588acd486a31d7bb7bd084a6553a39
author : Andres Freund <[email protected]>
date : Tue, 8 Sep 2020 11:25:34 -0700
committer: Andres Freund <[email protected]>
date : Tue, 8 Sep 2020 11:25:34 -0700
The problem is caused by me (Andres) having ProcSleep() look at the
wrong PGPROC entry in 5788e258bb2.
Unfortunately it seems hard to write a reliable test for autovacuum
cancellations. Perhaps somebody will come up with a good approach, but
it seems worth fixing the issue even without a test.
Reported-By: Jeff Janes <[email protected]>
Author: Jeff Janes <[email protected]>
Discussion: https://postgr.es/m/CAMkU=1wH2aUy+wDRDz+5RZALdcUnEofV1t9PzXS_gBJO9vZZ0Q@mail.gmail.com
M src/backend/storage/lmgr/proc.c
Use plain memset() in numeric.c, not MemSet and friends.
commit : 3438c988fd757c74ea4ddd80cdbb7405f4a1bc39
author : Tom Lane <[email protected]>
date : Tue, 8 Sep 2020 11:47:37 -0400
committer: Tom Lane <[email protected]>
date : Tue, 8 Sep 2020 11:47:37 -0400
This essentially reverts a micro-optimization I made years ago,
as part of the much larger commit d72f6c750. It's doubtful
that there was any hard evidence for it being helpful even then,
and the case is even more dubious now that modern compilers
are so much smarter about inlining memset().
The proximate reason for undoing it is to get rid of the type punning
inherent in MemSet, for fear that that may cause problems now that
we're applying additional optimization switches to numeric.c.
At the very least this'll silence some warnings from a few old
buildfarm animals.
(It's probably past time for another look at whether MemSet is still
worth anything at all, but I do not propose to tackle that question
right now.)
Discussion: https://postgr.es/m/CAJ3gD9evtA_vBo+WMYMyT-u=keHX7-r8p2w7OSRfXf42LTwCZQ@mail.gmail.com
M src/backend/utils/adt/numeric.c
Use <unnamed> for name of unnamed portal's memory context
commit : 728d4bc16b79dea0dd6d540aa5c55a84ac722a09
author : Peter Eisentraut <[email protected]>
date : Tue, 8 Sep 2020 17:11:16 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 8 Sep 2020 17:11:16 +0200
Otherwise just printing an empty string makes the memory context debug
output slightly confusing.
Discussion: https://www.postgresql.org/message-id/flat/ccb353ef-89ff-09b3-8046-1d2514624b9c%402ndquadrant.com
M src/backend/utils/mmgr/portalmem.c
Remove unused parameter
commit : 1dec091d5b0bfad1a9b061cac820236e98cbd72c
author : Peter Eisentraut <[email protected]>
date : Tue, 8 Sep 2020 10:08:46 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 8 Sep 2020 10:08:46 +0200
unused since f0d6f20278b7c5c412ce40a9b86c6b31dc2fbfdd
Discussion: https://www.postgresql.org/message-id/flat/511bb100-f829-ba21-2f10-9f952ec06ead%402ndquadrant.com
M contrib/pg_stat_statements/pg_stat_statements.c
Remove isolation test reindex-partitions
commit : 6785974287afa27221a49ae7202ea3d0695c0a84
author : Michael Paquier <[email protected]>
date : Tue, 8 Sep 2020 11:15:21 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 8 Sep 2020 11:15:21 +0900
The isolation test added by a6642b3 is proving to be unstable, as once
the first transaction holding a lock on the top-most partitioned table
or on a partition commits, the commit order of the follow-up DROP TABLE
and REINDEX could become reversed depending on the timing.
The only part of the test that could be entirely reliable is the one
using a SHARE lock, allowing REINDEX to commit first, but it is the
least interesting of the set.
Per buildfarm members rorqual and mylodon.
Discussion: https://postgr.es/m/[email protected]
D src/test/isolation/expected/reindex-partitions.out
M src/test/isolation/isolation_schedule
D src/test/isolation/specs/reindex-partitions.spec
Add support for partitioned tables and indexes in REINDEX
commit : a6642b3ae060976b42830b7dc8f29ec190ab05e4
author : Michael Paquier <[email protected]>
date : Tue, 8 Sep 2020 10:09:22 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 8 Sep 2020 10:09:22 +0900
Until now, REINDEX was not able to work with partitioned tables and
indexes, forcing users to reindex partitions one by one. This extends
REINDEX INDEX and REINDEX TABLE so as they can accept a partitioned
index and table in input, respectively, to reindex all the partitions
assigned to them with physical storage (foreign tables, partitioned
tables and indexes are then discarded).
This shares some logic with schema and database REINDEX as each
partition gets processed in its own transaction after building a list of
relations to work on. This choice has the advantage to minimize the
number of invalid indexes to one partition with REINDEX CONCURRENTLY in
the event a cancellation or failure in-flight, as the only indexes
handled at once in a single REINDEX CONCURRENTLY loop are the ones from
the partition being working on.
Isolation tests are added to emulate some cases I bumped into while
developing this feature, particularly with the concurrent drop of a
leaf partition reindexed. However, this is rather limited as LOCK would
cause REINDEX to block in the first transaction building the list of
partitions.
Per its multi-transaction nature, this new flavor cannot run in a
transaction block, similarly to REINDEX SCHEMA, SYSTEM and DATABASE.
Author: Justin Pryzby, Michael Paquier
Reviewed-by: Anastasia Lubennikova
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/ref/reindex.sgml
M src/backend/catalog/index.c
M src/backend/commands/indexcmds.c
M src/backend/tcop/utility.c
M src/include/commands/defrem.h
A src/test/isolation/expected/reindex-partitions.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/reindex-partitions.spec
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
M src/tools/pgindent/typedefs.list
Adjust cost model for HashAgg that spills to disk.
commit : a547e6867527ca16628a3fb1cf3ef6f785210a31
author : Jeff Davis <[email protected]>
date : Mon, 7 Sep 2020 13:31:59 -0700
committer: Jeff Davis <[email protected]>
date : Mon, 7 Sep 2020 13:31:59 -0700
Tomas Vondra observed that the IO behavior for HashAgg tends to be
worse than for Sort. Penalize HashAgg IO costs accordingly.
Also, account for the CPU effort of spilling the tuples and reading
them back.
Discussion: https://postgr.es/m/20200906212112.nzoy5ytrzjjodpfh@development
Reviewed-by: Tomas Vondra
Backpatch-through: 13
M src/backend/optimizer/path/costsize.c
Clarify comments in enforce_generic_type_consistency().
commit : 53367e6c62e3111eccdd578a3fa607c48c3e1fb6
author : Tom Lane <[email protected]>
date : Mon, 7 Sep 2020 14:52:33 -0400
committer: Tom Lane <[email protected]>
date : Mon, 7 Sep 2020 14:52:33 -0400
Some of the pre-existing comments were vague about whether they
referred to all polymorphic types or only the old-style ones.
Also be more consistent about using the "family 1" vs "family 2"
terminology.
Himanshu Upadhyaya and Tom Lane
Discussion: https://postgr.es/m/CAPF61jBUg9XoMPNuLpoZ+h6UZ2VxKdNt3rQL1xw1GOBwjWzAXQ@mail.gmail.com
M src/backend/parser/parse_coerce.c
Frob numeric.c loop so that clang will auto-vectorize it too.
commit : 9c79e646c6f0f8df06d966c536d0c6aa33bf1b06
author : Tom Lane <[email protected]>
date : Mon, 7 Sep 2020 12:03:04 -0400
committer: Tom Lane <[email protected]>
date : Mon, 7 Sep 2020 12:03:04 -0400
Experimentation shows that clang will auto-vectorize the critical
multiplication loop if the termination condition is written "i2 < limit"
rather than "i2 <= limit". This seems unbelievably stupid, but I've
reproduced it on both clang 9.0.1 (RHEL8) and 11.0.3 (macOS Catalina).
gcc doesn't care, so tweak the code to do it that way.
Discussion: https://postgr.es/m/CAJ3gD9evtA_vBo+WMYMyT-u=keHX7-r8p2w7OSRfXf42LTwCZQ@mail.gmail.com
M src/backend/utils/adt/numeric.c
Add d_type to our Windows dirent emulation.
commit : 87e6ed7c8c6abb6dd62119259d2fd89169a04ac0
author : Thomas Munro <[email protected]>
date : Mon, 7 Sep 2020 23:18:22 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 7 Sep 2020 23:18:22 +1200
This allows us to skip some stat calls, by extending commit 861c6e7c to
cover Windows systems.
Author: Juan José Santamaría Flecha <[email protected]>
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Magnus Hagander <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKG%2BFzxupGGN4GpUdbzZN%2Btn6FQPHo8w0Q%2BAPH5Wz8RG%2Bww%40mail.gmail.com
M src/include/port/win32_msvc/dirent.h
M src/port/dirent.c
doc: Fix table cell overflow
commit : 74ff28197c14de0aa709497ce707e31bb9af7253
author : Peter Eisentraut <[email protected]>
date : Mon, 7 Sep 2020 09:59:50 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 7 Sep 2020 09:59:50 +0200
Fix one instance of a table cell overflow by adding a zero-width
space. The visual impact of this is minimal, but since this is
currently the only such case reported by FOP ("contents of ... exceed
the available area"), it seems worth getting rid of.
M doc/src/sgml/high-availability.sgml
Skip unnecessary stat() calls in walkdir().
commit : 861c6e7c8e4dfdd842442dde47cc653764baff4f
author : Thomas Munro <[email protected]>
date : Mon, 7 Sep 2020 18:11:46 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 7 Sep 2020 18:11:46 +1200
Some kernels can tell us the type of a "dirent", so we can avoid a call
to stat() or lstat() in many cases. Define a new function
get_dirent_type() to contain that logic, for use by the backend and
frontend versions of walkdir(), and perhaps other callers in future.
Reviewed-by: Tom Lane <[email protected]>
Reviewed-by: Juan José Santamaría Flecha <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKG%2BFzxupGGN4GpUdbzZN%2Btn6FQPHo8w0Q%2BAPH5Wz8RG%2Bww%40mail.gmail.com
M src/backend/storage/file/fd.c
M src/common/Makefile
M src/common/file_utils.c
M src/include/common/file_utils.h
M src/tools/msvc/Mkvcbuild.pm
M src/tools/pgindent/typedefs.list
doc: Tweak sentence for pg_checksums when enabling checksums
commit : f0942b1327e8fa32b38a02eaff627c16b517c3f9
author : Michael Paquier <[email protected]>
date : Mon, 7 Sep 2020 14:34:59 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 7 Sep 2020 14:34:59 +0900
The previous version of the docs mentioned that files are rewritten,
implying that a second copy of each file gets created, but each file is
updated in-place.
Author: Michael Banck
Reviewed-by: Daniel Gustafsson, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M doc/src/sgml/ref/pg_checksums.sgml
Add additional tests to test streaming of in-progress transactions.
commit : 58b5ae9d62bdd44879c6f9a570849903ccd6cd66
author : Amit Kapila <[email protected]>
date : Mon, 7 Sep 2020 08:08:58 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 7 Sep 2020 08:08:58 +0530
This covers the functionality tests for streaming in-progress
subtransactions, streaming transactions containing rollback to savepoints,
and streaming transactions having DDLs.
Author: Tomas Vondra, Amit Kapila and Dilip Kumar
Reviewed-by: Dilip Kumar
Discussion: https://postgr.es/m/[email protected]
A src/test/subscription/t/016_stream_subxact.pl
A src/test/subscription/t/017_stream_ddl.pl
A src/test/subscription/t/018_stream_subxact_abort.pl
A src/test/subscription/t/019_stream_subxact_ddl_abort.pl
Apply auto-vectorization to the inner loop of numeric multiplication.
commit : 88709176236caf3cb9655acda6bad2df0323ac8f
author : Tom Lane <[email protected]>
date : Sun, 6 Sep 2020 21:40:39 -0400
committer: Tom Lane <[email protected]>
date : Sun, 6 Sep 2020 21:40:39 -0400
Compile numeric.c with -ftree-vectorize where available, and adjust
the innermost loop of mul_var() so that it is amenable to being
auto-vectorized. (Mainly, that involves making it process the arrays
left-to-right not right-to-left.)
Applying -ftree-vectorize actually makes numeric.o smaller, at least
with my compiler (gcc 8.3.1 on x86_64), and it's a little faster too.
Independently of that, fixing the inner loop to be vectorizable also
makes things a bit faster. But doing both is a huge win for
multiplications with lots of digits. For me, the numeric regression
test is the same speed to within measurement noise, but numeric_big
is a full 45% faster.
We also looked into applying -funroll-loops, but that makes numeric.o
bloat quite a bit, and the additional speed improvement is very
marginal.
Amit Khandekar, reviewed and edited a little by me
Discussion: https://postgr.es/m/CAJ3gD9evtA_vBo+WMYMyT-u=keHX7-r8p2w7OSRfXf42LTwCZQ@mail.gmail.com
M src/backend/utils/adt/Makefile
M src/backend/utils/adt/numeric.c
Split Makefile symbol CFLAGS_VECTOR into two symbols.
commit : 695de5d1eda6382b20fadb0a2b2d286b57a6a61c
author : Tom Lane <[email protected]>
date : Sun, 6 Sep 2020 21:28:16 -0400
committer: Tom Lane <[email protected]>
date : Sun, 6 Sep 2020 21:28:16 -0400
Replace CFLAGS_VECTOR with CFLAGS_UNROLL_LOOPS and CFLAGS_VECTORIZE,
allowing us to distinguish whether we want to apply -funroll-loops,
-ftree-vectorize, or both to a particular source file. Up to now
the only consumer of the symbol has been checksum.c which wants
both, so that there was no need to distinguish; but that's about
to change.
Amit Khandekar, reviewed and edited a little by me
Discussion: https://postgr.es/m/CAJ3gD9evtA_vBo+WMYMyT-u=keHX7-r8p2w7OSRfXf42LTwCZQ@mail.gmail.com
M configure
M configure.ac
M src/Makefile.global.in
M src/backend/storage/page/Makefile
Remove arbitrary line length limits in pg_regress (plain and ECPG).
commit : 784b1ba1a2b9306697544bedb2ef9425185dd206
author : Tom Lane <[email protected]>
date : Sun, 6 Sep 2020 14:13:02 -0400
committer: Tom Lane <[email protected]>
date : Sun, 6 Sep 2020 14:13:02 -0400
Refactor replace_string() to use a StringInfo for the modifiable
string argument. This allows the string to be of indefinite size
initially and/or grow substantially during replacement. The previous
logic in convert_sourcefiles_in() had a hard-wired limit of 1024
bytes on any line in input/*.sql or output/*.out files. While we've
not had reports of trouble yet, it'd surely have bit us someday.
This also fixes replace_string() so it won't get into an infinite
loop if the string-to-be-replaced is a substring of the replacement.
That's unlikely to happen in current usage, but the function surely
shouldn't depend on it.
Also fix ecpg_filter() to use a StringInfo and thereby remove its
hard limit of 300 bytes on the length of an ecpg source line.
Asim Rama Praveen and Georgios Kokolatos,
reviewed by Alvaro Herrera and myself
Discussion: https://postgr.es/m/y9Dlk2QhiZ39DhaB1QE9mgZ95HcOQKZCNtGwN7XCRKMdBRBnX_0woaRUtTjloEp4PKA6ERmcUcfq3lPGfKPOJ5xX2TV-5WoRYyySeNHRzdw=@protonmail.com
M src/interfaces/ecpg/test/pg_regress_ecpg.c
M src/test/regress/pg_regress.c
M src/test/regress/pg_regress.h
Refactor pg_get_line() to expose an alternative StringInfo-based API.
commit : 8e3c58e6e459b285d37edb6129e412ed25cd90c1
author : Tom Lane <[email protected]>
date : Sun, 6 Sep 2020 13:57:10 -0400
committer: Tom Lane <[email protected]>
date : Sun, 6 Sep 2020 13:57:10 -0400
Letting the caller provide a StringInfo to read into is helpful when
the caller needs to merge lines or otherwise modify the data after
it's been read. Notably, now the code added by commit 8f8154a50
can use pg_get_line_append() instead of having its own copy of that
logic. A follow-on commit will also make use of this.
Also, since StringInfo buffers are a minimum of 1KB long, blindly
using pg_get_line() in a loop can eat a lot more memory than one would
expect. I discovered for instance that commit e0f05cd5b caused initdb
to consume circa 10MB to read postgres.bki, even though that's under
1MB worth of data. A less memory-hungry alternative is to re-use the
same StringInfo for all lines and pg_strdup the results.
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/hba.c
M src/bin/initdb/initdb.c
M src/common/pg_get_line.c
M src/include/common/string.h
Change path in example of file_fdw for logs
commit : 68b603e1a934dcd82e259b7776565ec1776e7a29
author : Magnus Hagander <[email protected]>
date : Sun, 6 Sep 2020 19:28:32 +0200
committer: Magnus Hagander <[email protected]>
date : Sun, 6 Sep 2020 19:28:32 +0200
It's better to use a relative path into the data directory, than to a
hardcoded home directory of user 'josh'.
Discussion: https://postgr.es/m/CABUevEyuf67Yu_r9gpDMs5MKifK7+-+pe=ZjKzya4JEn9kUk1w@mail.gmail.com
M doc/src/sgml/file-fdw.sgml
Fix typo in comment
commit : 2a093355aa629468eab82242ced83a1c7fff76a8
author : Magnus Hagander <[email protected]>
date : Sun, 6 Sep 2020 19:26:55 +0200
committer: Magnus Hagander <[email protected]>
date : Sun, 6 Sep 2020 19:26:55 +0200
Author: Hou, Zhijie
M src/backend/storage/ipc/procsignal.c
Fix misleading error message about inconsistent moving-aggregate types.
commit : 19ad7e1d7b8b54ea3aa405e60057ab4baefcedbf
author : Tom Lane <[email protected]>
date : Sun, 6 Sep 2020 12:55:13 -0400
committer: Tom Lane <[email protected]>
date : Sun, 6 Sep 2020 12:55:13 -0400
We reported the wrong types when complaining that an aggregate's
moving-aggregate implementation is inconsistent with its regular
implementation.
This was wrong since the feature was introduced, so back-patch
to all supported branches.
Jeff Janes
Discussion: https://postgr.es/m/CAMkU=1x808LH=LPhZp9mNSP0Xd1xDqEd+XeGcvEe48dfE6xV=A@mail.gmail.com
M src/backend/catalog/pg_aggregate.c
Remove useless lstat() call in pg_rewind.
commit : 666e9a90f09a1ab89959774273597764e6e0f9ad
author : Tom Lane <[email protected]>
date : Sun, 6 Sep 2020 11:50:40 -0400
committer: Tom Lane <[email protected]>
date : Sun, 6 Sep 2020 11:50:40 -0400
This is duplicative of an lstat that was just done by the calling
function (traverse_datadir), besides which we weren't really doing
anything with the results. There's not much point in checking to
see if someone removed the file since the previous lstat, since the
FILE_ACTION_REMOVE code would have to deal with missing-file cases
anyway. Moreover, the "exists = false" assignment was a dead store;
nothing was done with that value later.
A syscall saved is a syscall earned, so back-patch to 9.5
where this code was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_rewind/filemap.c
doc: Don't hide the "Up" link when it is the same as "Home"
commit : cd153b54eb6542d0e0b4addf1f10a1b34a328e17
author : Peter Eisentraut <[email protected]>
date : Sun, 6 Sep 2020 16:46:13 +0200
committer: Peter Eisentraut <[email protected]>
date : Sun, 6 Sep 2020 16:46:13 +0200
The original stylesheets seemed to think this was a good idea, but our
users find it confusing and unhelpful, so undo that logic.
Reported-by: Fabien COELHO <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/alpine.DEB.2.22.394.2006210914370.859381%40pseudo
M doc/src/sgml/stylesheet.xsl
Remove unused parameter
commit : 76af9744db168cfe96f45769b8ec68eb222d8fcf
author : Peter Eisentraut <[email protected]>
date : Sun, 6 Sep 2020 09:32:16 +0200
committer: Peter Eisentraut <[email protected]>
date : Sun, 6 Sep 2020 09:32:16 +0200
unused since 84d723b6cefcf25b8c800f8aa6cf3c9538a546b4
Discussion: https://www.postgresql.org/message-id/flat/511bb100-f829-ba21-2f10-9f952ec06ead%402ndquadrant.com
M contrib/tablefunc/tablefunc.c
Improve some ancient, crufty code in bootstrap + initdb.
commit : e0f05cd5ba76a75e2ce3b85ba050e48e857dca00
author : Tom Lane <[email protected]>
date : Sat, 5 Sep 2020 16:20:04 -0400
committer: Tom Lane <[email protected]>
date : Sat, 5 Sep 2020 16:20:04 -0400
At some point back in the last century, somebody felt that reading
all of pg_type twice was cheaper, or at least easier, than using
repalloc() to resize the Typ[] array dynamically. That seems like an
entirely wacko proposition, so rewrite the code to do it the other
way. (To add insult to injury, there were two not-quite-identical
copies of said code.)
initdb.c's readfile() function had the same disease of preferring
to do double the I/O to avoid resizing its output array. Here,
we can make things easier by using the just-invented pg_get_line()
function to handle reading individual lines without a predetermined
notion of how long they are.
On my machine, it's difficult to detect any net change in the
overall runtime of initdb from these changes; but they should
help on slower buildfarm machines (especially since a buildfarm
cycle involves a lot of initdb's these days).
My attention was drawn to these places by scan-build complaints,
but on inspection they needed a lot more work than just suppressing
dead stores :-(
M src/backend/bootstrap/bootstrap.c
M src/bin/initdb/initdb.c
Yet more elimination of dead stores and useless initializations.
commit : a5cc4dab6d1d694f113912a2aca7012a95262f0b
author : Tom Lane <[email protected]>
date : Sat, 5 Sep 2020 13:17:32 -0400
committer: Tom Lane <[email protected]>
date : Sat, 5 Sep 2020 13:17:32 -0400
I'm not sure what tool Ranier was using, but the ones I contributed
were found by using a newer version of scan-build than I tried before.
Ranier Vilela and Tom Lane
Discussion: https://postgr.es/m/CAEudQAo1+AcGppxDSg8k+zF4+Kv+eJyqzEDdbpDg58-=MQcerQ@mail.gmail.com
M src/backend/access/common/heaptuple.c
M src/backend/access/gist/gistutil.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/catalog/storage.c
M src/backend/commands/async.c
M src/backend/storage/ipc/procarray.c
M src/backend/tsearch/spell.c
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/tsrank.c
M src/bin/pg_dump/pg_backup_custom.c
Switch to multi-inserts when registering dependencies for many code paths
commit : 8febfd1855450f50f17419def41c2ea9bcf994d5
author : Michael Paquier <[email protected]>
date : Sat, 5 Sep 2020 21:33:53 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 5 Sep 2020 21:33:53 +0900
This commit improves the dependency registrations by taking advantage of
the preliminary work done in 63110c62, to group together the insertion
of dependencies of the same type to pg_depend. With the current layer
of routines available, and as only dependencies of the same type can be
grouped, there are code paths still doing more than one multi-insert
when it is necessary to register dependencies of multiple types
(constraint and index creation are two cases doing that).
While on it, this refactors some of the code to use ObjectAddressSet()
when manipulating object addresses.
Author: Daniel Gustafsson, Michael Paquier
Reviewed-by: Andres Freund, Álvaro Herrera
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/pg_aggregate.c
M src/backend/catalog/pg_cast.c
M src/backend/catalog/pg_constraint.c
M src/backend/catalog/pg_operator.c
M src/backend/catalog/pg_proc.c
M src/backend/catalog/pg_range.c
M src/backend/catalog/pg_type.c
M src/backend/commands/functioncmds.c
M src/backend/commands/proclang.c
M src/backend/commands/tsearchcmds.c
M src/test/regress/expected/create_index.out
Extend SQL function tests lightly
commit : 11b80d900fe4297e8e4bc231f6a41b53d604ed9e
author : Peter Eisentraut <[email protected]>
date : Sat, 5 Sep 2020 13:28:05 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 5 Sep 2020 13:28:05 +0200
The basic tests that defined SQL functions didn't actually run the
functions to see if they worked. Add that, and also fix a minor
mistake in a function that was revealed by this. (This is not a
question of test coverage, since there are other places where SQL
functions are run, but it is a bit of a silly test design.)
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/test/regress/expected/create_function_3.out
M src/test/regress/sql/create_function_3.sql
Fix typo in comment
commit : 556cbdfce4ffe01410b89dbf01b27315aa201bbf
author : Peter Eisentraut <[email protected]>
date : Sat, 5 Sep 2020 11:32:20 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 5 Sep 2020 11:32:20 +0200
M src/backend/libpq/auth-scram.c
Use multi-inserts for pg_depend
commit : 63110c6264a5363863209c8fbdd0498c03535507
author : Michael Paquier <[email protected]>
date : Sat, 5 Sep 2020 13:52:47 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 5 Sep 2020 13:52:47 +0900
This is a follow-up of the work done in e3931d01. This case is a bit
different than pg_attribute and pg_shdepend: the maximum number of items
to insert is known in advance, but there is no need to handle pinned
dependencies. Hence, the base allocation for slots is done based on the
number of items and the maximum allowed with a cap at 64kB. Slots are
initialized once used to minimize the overhead of the operation.
The insertions can be done for dependencies of the same type. More
could be done by grouping the insertion of multiple dependency types in
a single batch. This is left as future work.
Some of the multi-insert logic is also simplified for pg_shdepend, as
per the feedback discussed for this specific patch. This also moves to
indexing.h the variable capping the maximum amount of data that can be
used at once for a multi-insert, instead of having separate definitions
for pg_attribute, pg_depend and pg_shdepend.
Author: Daniel Gustafsson, Michael Paquier
Reviewed-by: Andres Freund, Álvaro Herrera
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/heap.c
M src/backend/catalog/pg_depend.c
M src/backend/catalog/pg_shdepend.c
M src/include/catalog/indexing.h
Make new authentication test case more robust.
commit : 4d41823c5267184fb021ac456caf95ef734cc8b0
author : Tom Lane <[email protected]>
date : Fri, 4 Sep 2020 21:01:58 -0400
committer: Tom Lane <[email protected]>
date : Fri, 4 Sep 2020 21:01:58 -0400
I happened to notice that the new test case I added in b55b4dad9
falls over if one runs "make check" repeatedly; though not in branches
after v10. That's because it was assuming that tmp_check/pgpass
wouldn't exist already. However, it's only been since v11 that the
Makefiles forcibly remove all of tmp_check/ before starting a TAP run.
This fix to unlink the file is therefore strictly necessary only in
v10 ... but it seems wisest to do it across the board, rather than
let the test rely on external logic to get the conditions right.
M src/test/authentication/t/001_password.pl
Fix over-eager ping'ing in logical replication receiver.
commit : c8746f999ea2feba22832b485fba37b5301b54a3
author : Tom Lane <[email protected]>
date : Fri, 4 Sep 2020 20:20:05 -0400
committer: Tom Lane <[email protected]>
date : Fri, 4 Sep 2020 20:20:05 -0400
Commit 3f60f690f only partially fixed the broken-status-tracking
issue in LogicalRepApplyLoop: we need ping_sent to have the same
lifetime as last_recv_timestamp. The effects are much less serious
than what that commit fixed, though. AFAICS this would just lead to
extra ping requests being sent, once per second until the sender
responds. Still, it's a bug, so backpatch to v10 as before.
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/worker.c
Remove still more useless assignments.
commit : 9a851039aac6fc7b78a80e60699e7bd8924b6f12
author : Tom Lane <[email protected]>
date : Fri, 4 Sep 2020 18:17:47 -0400
committer: Tom Lane <[email protected]>
date : Fri, 4 Sep 2020 18:17:47 -0400
Fix some more things scan-build pointed to as dead stores. In some of
these cases, rearranging the code a little leads to more readable
code IMO. It's all cosmetic, though.
Discussion: https://postgr.es/m/CAEudQAo1+AcGppxDSg8k+zF4+Kv+eJyqzEDdbpDg58-=MQcerQ@mail.gmail.com
M src/backend/replication/slotfuncs.c
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/jsonfuncs.c
M src/bin/scripts/vacuumdb.c
Fix bogus MaxAllocSize check in logtape.c.
commit : 0852006a946aa9795b4913bccebb88d623942ca6
author : Jeff Davis <[email protected]>
date : Fri, 4 Sep 2020 12:01:58 -0700
committer: Jeff Davis <[email protected]>
date : Fri, 4 Sep 2020 12:01:58 -0700
Reported-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-Wz=NZPZc3-fkdmvu=w2itx0PiB-G6QpxHXZOjuvFAzPdZw@mail.gmail.com
Backpatch-through: 13
M src/backend/utils/sort/logtape.c
Report expected contrecord length on mismatch
commit : f43e295f68c3e04ef891627f62016a5b3d8ed4a8
author : Alvaro Herrera <[email protected]>
date : Fri, 4 Sep 2020 14:58:32 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 4 Sep 2020 14:58:32 -0400
When reading a WAL record fails to find continuation record(s) of the
proper length, report what it expects, for clarity.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlogreader.c
Remove some more useless assignments.
commit : 38a2d703298c9a891dc9c24c0c087f417f555c70
author : Tom Lane <[email protected]>
date : Fri, 4 Sep 2020 14:32:10 -0400
committer: Tom Lane <[email protected]>
date : Fri, 4 Sep 2020 14:32:10 -0400
Found with clang's scan-build tool. It also whines about a lot of
other dead stores that we should *not* change IMO, either as a matter
of style or future-proofing. But these places seem like clear
oversights.
Discussion: https://postgr.es/m/CAEudQAo1+AcGppxDSg8k+zF4+Kv+eJyqzEDdbpDg58-=MQcerQ@mail.gmail.com
M src/backend/access/brin/brin_tuple.c
M src/backend/access/gin/ginbtree.c
M src/backend/catalog/pg_depend.c
M src/backend/optimizer/plan/planner.c
M src/backend/parser/parse_utilcmd.c
M src/backend/partitioning/partbounds.c
M src/backend/statistics/extended_stats.c
M src/backend/tsearch/spell.c
Collect attribute data on extension owned tables being dumped
commit : 3eb3d3e7822d5eecfcaba871a90263c6025c5216
author : Andrew Dunstan <[email protected]>
date : Fri, 4 Sep 2020 13:53:09 -0400
committer: Andrew Dunstan <[email protected]>
date : Fri, 4 Sep 2020 13:53:09 -0400
If this data is not collected, pg_dump segfaults if asked for column
inserts.
Fix by Fabrízio de Royes Mello
Backpatch to release 12 where the bug was introduced.
M src/bin/pg_dump/pg_dump.c
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/modules/test_pg_dump/test_pg_dump–1.0.sql
C comment: correct use of 64-"byte" cache line size
commit : 3b5af0e95ad5a3d9b478826336a11ad1d201c378
author : Bruce Momjian <[email protected]>
date : Fri, 4 Sep 2020 13:27:52 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 4 Sep 2020 13:27:52 -0400
Reported-by: Kelly Min
Discussion: https://postgr.es/m/CAPSbxatOiQO90LYpSC3+svAU9-sHgDfEP4oFhcEUt_X=DqFA9g@mail.gmail.com
Backpatch-through: 9.5
M src/include/storage/buf_internals.h
Fix rare deadlock failure in create_am regression test.
commit : d54f99e41541de848a6ca53b3ec060f461e9ab71
author : Tom Lane <[email protected]>
date : Fri, 4 Sep 2020 12:40:28 -0400
committer: Tom Lane <[email protected]>
date : Fri, 4 Sep 2020 12:40:28 -0400
The "DROP ACCESS METHOD gist2" test will require locking the index
to be dropped and then its table; while most ordinary operations
lock a table first then its index. While no concurrent test scripts
should be touching fast_emp4000, autovacuum might chance to be
processing that table when the DROP runs, resulting in a deadlock
failure. This is pretty rare but we see it in the buildfarm from
time to time.
To fix, acquire a lock on fast_emp4000 before issuing the DROP.
Since the point of the exercise is mostly to prevent buildfarm
failures, back-patch to 9.6 where this test was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/create_am.out
M src/test/regress/sql/create_am.sql
doc: Change table alias names to lower case in tutorial chapter
commit : 49d7165117893405ae9b5b8d8e7877acff33c0e7
author : Peter Eisentraut <[email protected]>
date : Fri, 4 Sep 2020 08:45:57 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 4 Sep 2020 08:45:57 +0200
This is needlessly different from our usual style otherwise.
Author: Jürgen Purtz <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/query.sgml
M src/tutorial/basics.source
doc: Fix whitespace issue in PDF
commit : 79fd620b20b7721c2b591ef495be79b6347286c1
author : Peter Eisentraut <[email protected]>
date : Fri, 4 Sep 2020 08:39:01 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 4 Sep 2020 08:39:01 +0200
Move <indexterm> outside of <para> to avoid whitespace issue in PDF
output.
Author: Jürgen Purtz <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/query.sgml
doc: Use tags consistently in the tutorial chapter
commit : 6eee73e4e5b6d499f1a5529f209ce4af86ef5152
author : Peter Eisentraut <[email protected]>
date : Fri, 4 Sep 2020 08:19:20 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 4 Sep 2020 08:19:20 +0200
Make more consistent use of <screen> and <programlisting>.
Author: Jürgen Purtz <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/query.sgml
Remove unused parameter
commit : 96cfcadd26e26e138ae60f4f3f94beddafba60bb
author : Peter Eisentraut <[email protected]>
date : Fri, 4 Sep 2020 08:02:58 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 4 Sep 2020 08:02:58 +0200
unused since 93ee38eade1b2b4964354b95b01b09e17d6f098d
Discussion: https://www.postgresql.org/message-id/flat/511bb100-f829-ba21-2f10-9f952ec06ead%402ndquadrant.com
M contrib/pageinspect/btreefuncs.c
Fix inline marking introduced in commit 464824323e.
commit : ac15b499f7f92c26661835c327bfb0228a9b5e73
author : Amit Kapila <[email protected]>
date : Fri, 4 Sep 2020 11:25:16 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 4 Sep 2020 11:25:16 +0530
Forgot to add inline marking in changes_filename() declaration. In the passing, add
inline marking for a similar function subxact_filename().
Reported-By: Nathan Bossart
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/worker.c
remove redundant initializations
commit : e36e936e0ee664e07588732d1a8d105c5ec8e57d
author : Bruce Momjian <[email protected]>
date : Thu, 3 Sep 2020 22:57:35 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 3 Sep 2020 22:57:35 -0400
Reported-by: Ranier Vilela
Discussion: https://postgr.es/m/CAEudQAo1+AcGppxDSg8k+zF4+Kv+eJyqzEDdbpDg58-=MQcerQ@mail.gmail.com
Author: Ranier Vilela
Backpatch-through: master
M src/backend/access/gist/gistxlog.c
M src/backend/catalog/heap.c
M src/backend/storage/smgr/md.c
M src/backend/utils/adt/json.c
M src/backend/utils/mmgr/mcxt.c
Remove variable "concurrent" from ReindexStmt
commit : 844c05abc3f1c1703bf17cf44ab66351ed9711d2
author : Michael Paquier <[email protected]>
date : Fri, 4 Sep 2020 10:36:35 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 4 Sep 2020 10:36:35 +0900
This node already handles multiple options using a bitmask, so having a
separate boolean flag is not necessary. This simplifies the code a bit
with less arguments to give to the reindex routines, by replacing the
boolean with an equivalent bitmask value.
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/indexcmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/parser/gram.y
M src/backend/tcop/utility.c
M src/include/commands/defrem.h
M src/include/nodes/parsenodes.h
Remove arbitrary restrictions on password length.
commit : 67a472d71c98c3d2fa322a1b4013080b20720b98
author : Tom Lane <[email protected]>
date : Thu, 3 Sep 2020 20:09:18 -0400
committer: Tom Lane <[email protected]>
date : Thu, 3 Sep 2020 20:09:18 -0400
This patch started out with the goal of harmonizing various arbitrary
limits on password length, but after awhile a better idea emerged:
let's just get rid of those fixed limits.
recv_password_packet() has an arbitrary limit on the packet size,
which we don't really need, so just drop it. (Note that this doesn't
really affect anything for MD5 or SCRAM password verification, since
those will hash the user's password to something shorter anyway.
It does matter for auth methods that require a cleartext password.)
Likewise remove the arbitrary error condition in pg_saslprep().
The remaining limits are mostly in client-side code that prompts
for passwords. To improve those, refactor simple_prompt() so that
it allocates its own result buffer that can be made as big as
necessary. Actually, it proves best to make a separate routine
pg_get_line() that has essentially the semantics of fgets(), except
that it allocates a suitable result buffer and hence will never
return a truncated line. (pg_get_line has a lot of potential
applications to replace randomly-sized fgets buffers elsewhere,
but I'll leave that for another patch.)
I built pg_get_line() atop stringinfo.c, which requires moving
that code to src/common/; but that seems fine since it was a poor
fit for src/port/ anyway.
This patch is mostly mine, but it owes a good deal to Nathan Bossart
who pressed for a solution to the password length problem and
created a predecessor patch. Also thanks to Peter Eisentraut and
Stephen Frost for ideas and discussion.
Discussion: https://postgr.es/m/[email protected]
M contrib/oid2name/oid2name.c
M contrib/vacuumlo/vacuumlo.c
M src/backend/libpq/auth.c
M src/bin/initdb/initdb.c
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pgbench/pgbench.c
M src/bin/psql/command.c
M src/bin/psql/startup.c
M src/bin/scripts/common.c
M src/bin/scripts/createuser.c
M src/bin/scripts/dropuser.c
M src/common/Makefile
A src/common/pg_get_line.c
M src/common/saslprep.c
R081 src/port/sprompt.c src/common/sprompt.c
M src/include/common/string.h
M src/include/port.h
M src/port/Makefile
M src/tools/msvc/Mkvcbuild.pm
Avoid lockup of a parallel worker when reporting a long error message.
commit : be4b0c0077e6a1f7be0965f8d93696e0e0eadb52
author : Tom Lane <[email protected]>
date : Thu, 3 Sep 2020 16:52:09 -0400
committer: Tom Lane <[email protected]>
date : Thu, 3 Sep 2020 16:52:09 -0400
Because sigsetjmp() will restore the initial state with signals blocked,
the code path in bgworker.c for reporting an error and exiting would
execute that way. Usually this is fairly harmless; but if a parallel
worker had an error message exceeding the shared-memory communication
buffer size (16K) it would lock up, because it would wait for a
resume-sending signal from its parallel leader which it would never
detect.
To fix, just unblock signals at the appropriate point.
This can be shown to fail back to 9.6. The lack of parallel query
infrastructure makes it difficult to provide a simple test case for
9.5; but I'm pretty sure the issue exists in some form there as well,
so apply the code change there too.
Vignesh C, reviewed by Bharath Rupireddy, Robert Haas, and myself
Discussion: https://postgr.es/m/CALDaNm1d1hHPZUg3xU4XjtWBOLCrA+-2cJcLpw-cePZ=GgDVfA@mail.gmail.com
M src/backend/postmaster/bgworker.c
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql
Allow records to span multiple lines in pg_hba.conf and pg_ident.conf.
commit : 8f8154a503c71a18ad72c64f4aefb9d847c45b86
author : Tom Lane <[email protected]>
date : Thu, 3 Sep 2020 12:16:48 -0400
committer: Tom Lane <[email protected]>
date : Thu, 3 Sep 2020 12:16:48 -0400
A backslash at the end of a line now causes the next line to be appended
to the current one (effectively, the backslash and newline are discarded).
This allows long HBA entries to be created without legibility problems.
While we're here, get rid of the former hard-wired length limit on
pg_hba.conf lines, by using an expansible StringInfo buffer instead
of a fixed-size local variable.
Since the same code is used to read the ident map file, these changes
apply there as well.
Fabien Coelho, reviewed by Justin Pryzby and David Zhang
Discussion: https://postgr.es/m/alpine.DEB.2.21.2003251906140.15243@pseudo
M doc/src/sgml/client-auth.sgml
M src/backend/libpq/hba.c
M src/test/authentication/t/001_password.pl
Doc: mention packager-supplied tools for server start/stop, initdb, etc.
commit : d2511d71328cb30b0d6b3f43af4809df83c4f0cc
author : Tom Lane <[email protected]>
date : Thu, 3 Sep 2020 11:45:26 -0400
committer: Tom Lane <[email protected]>
date : Thu, 3 Sep 2020 11:45:26 -0400
The majority of our audience is probably using a pre-packaged Postgres
build rather than raw sources. For them, much of runtime.sgml is not
too relevant, and they should be reading the packager's docs instead.
Add some notes pointing that way in appropriate places.
Text by me; thanks to Daniel Gustafsson for review and discussion,
and to Laurenz Albe for an earlier version.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/runtime.sgml
doc: Make SQL command names in the catalog documentation links
commit : 5ccf32211832477a147b1428b190817baea08f2f
author : Peter Eisentraut <[email protected]>
date : Thu, 3 Sep 2020 13:15:53 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 3 Sep 2020 13:15:53 +0200
In passing, fix the initdb references to be <application> rather than
<command>, which is what we normally use.
Author: Dagfinn Ilmari Mannsåker <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/catalogs.sgml
doc: Add missing cross-links in system catalog documentation
commit : cb6eb4a09e8adf3cc25e8eff9fdd43d8d9cd9c86
author : Peter Eisentraut <[email protected]>
date : Thu, 3 Sep 2020 13:15:53 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 3 Sep 2020 13:15:53 +0200
This makes the first mention of a system catalog or view in each
paragraph in the system system catalog and view documentation pages
hyperlinks, for easier navigation.
Also linkify the first mention of pg_hba.conf in pg_hba_file_rules, as
that's more specific and easier to spot than the link to the client
authentication chapter.
Author: Dagfinn Ilmari Mannsåker <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M doc/src/sgml/catalogs.sgml
Fix XML id to match containing page
commit : 4220e5721a27286aab726e87d3db5d1ee475fe23
author : Peter Eisentraut <[email protected]>
date : Thu, 3 Sep 2020 12:38:32 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 3 Sep 2020 12:38:32 +0200
This was apparently a typo when this part of the documentation was
first added.
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/psql-ref.sgml
Add support for streaming to built-in logical replication.
commit : 464824323e57dc4b397e8b05854d779908b55304
author : Amit Kapila <[email protected]>
date : Thu, 3 Sep 2020 07:54:07 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 3 Sep 2020 07:54:07 +0530
To add support for streaming of in-progress transactions into the
built-in logical replication, we need to do three things:
* Extend the logical replication protocol, so identify in-progress
transactions, and allow adding additional bits of information (e.g.
XID of subtransactions).
* Modify the output plugin (pgoutput) to implement the new stream
API callbacks, by leveraging the extended replication protocol.
* Modify the replication apply worker, to properly handle streamed
in-progress transaction by spilling the data to disk and then
replaying them on commit.
We however must explicitly disable streaming replication during
replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover we don't have a replication connection open so we
don't have where to send the data anyway.
Author: Tomas Vondra, Dilip Kumar and Amit Kapila
Reviewed-by: Amit Kapila, Kuntal Ghosh and Ajin Cherian
Tested-by: Neha Sharma, Mahendra Singh Thalor and Ajin Cherian
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/ref/alter_subscription.sgml
M doc/src/sgml/ref/create_subscription.sgml
M src/backend/catalog/pg_subscription.c
M src/backend/catalog/system_views.sql
M src/backend/commands/subscriptioncmds.c
M src/backend/postmaster/pgstat.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/proto.c
M src/backend/replication/logical/worker.c
M src/backend/replication/pgoutput/pgoutput.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_subscription.h
M src/include/pgstat.h
M src/include/replication/logicalproto.h
M src/include/replication/walreceiver.h
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
A src/test/subscription/t/015_stream.pl
M src/tools/pgindent/typedefs.list
Add string_to_table() function.
commit : 66f163068030b5c5fe792a0daee27822dac43791
author : Tom Lane <[email protected]>
date : Wed, 2 Sep 2020 18:23:56 -0400
committer: Tom Lane <[email protected]>
date : Wed, 2 Sep 2020 18:23:56 -0400
This splits a string at occurrences of a delimiter. It is exactly like
string_to_array() except for producing a set of values instead of an
array of values. Thus, the relationship of these two functions is
the same as between regexp_split_to_table() and regexp_split_to_array().
Although the same results could be had from unnest(string_to_array()),
this is somewhat faster than that, and anyway it seems reasonable to
have it for symmetry with the regexp functions.
Pavel Stehule, reviewed by Peter Smith
Discussion: https://postgr.es/m/CAFj8pRD8HOpjq2TqeTBhSo_QkzjLOhXzGCpKJ4nCs7Y9SQkuPw@mail.gmail.com
M doc/src/sgml/func.sgml
M src/backend/utils/adt/varlena.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/arrays.out
M src/test/regress/sql/arrays.sql
Remove unused parameter
commit : fd5e3b291415e6cf55408af1282585c945464c8f
author : Peter Eisentraut <[email protected]>
date : Wed, 2 Sep 2020 15:17:33 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 2 Sep 2020 15:17:33 +0200
unused since 39bd3fd1db6f3aa3764d4a1bebcd71c4e9c00281
Discussion: https://www.postgresql.org/message-id/flat/511bb100-f829-ba21-2f10-9f952ec06ead%402ndquadrant.com
M src/pl/plpgsql/src/pl_gram.y
Add access method names to \d[i|m|t]+ in psql
commit : 07f386ede026ae8c3f2adeba0c22139df19bf2ff
author : Michael Paquier <[email protected]>
date : Wed, 2 Sep 2020 16:59:22 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 2 Sep 2020 16:59:22 +0900
Listing a full set of relations with those psql meta-commands, without a
matching pattern, has never showed the access method associated with
each relation. This commit adds the access method of tables, indexes
and matviews, masking it for relation kinds where it does not apply.
Note that when HIDE_TABLEAM is enabled, the information does not show
up. This is available when connecting to a backend version of at least
12, where table AMs have been introduced.
Author: Georgios Kokolatos
Reviewed-by: Vignesh C, Michael Paquier, Justin Pryzby
Discussion: https://postgr.es/m/svaS1VTOEscES9CLKVTeKItjJP1EEJuBhTsA0ESOdlnbXeQSgycYwVlliL5zt8Jwcfo4ATYDXtEqsExxjkSkkhCSTCL8fnRgaCAJdr0unUg=@protonmail.com
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/describe.c
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
Fix thinko with definition of REINDEXOPT_MISSING_OK
commit : 01767533e37fa255722cc0088ed67a0309875963
author : Michael Paquier <[email protected]>
date : Wed, 2 Sep 2020 14:56:59 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 2 Sep 2020 14:56:59 +0900
This had no direct consequences, but let's be consistent and it would be
confusing when adding new flags. Oversight in 1d65416.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M src/include/nodes/parsenodes.h
Avoid unnecessary acquisition of SyncRepLock in transaction commit time.
commit : be9788e9989a0744ba60ab100153340fd123b786
author : Fujii Masao <[email protected]>
date : Wed, 2 Sep 2020 10:55:55 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 2 Sep 2020 10:55:55 +0900
In SyncRepWaitForLSN() routine called in transaction commit time,
SyncRepLock is necessary to atomically both check the shared
sync_standbys_defined flag and operate the sync replication wait-queue.
On the other hand, when the flag is false, the lock is not necessary
because the wait-queue is not touched. But due to the changes by
commit 48c9f49265, previously the lock was taken whatever the flag was.
This could cause unnecessary performance overhead in every transaction
commit time. Therefore this commit avoids that unnecessary aquisition
of SyncRepLock.
Author: Fujii Masao
Reviewed-by: Asim Praveen, Masahiko Sawada,
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/syncrep.c
Fix typo in comment
commit : 05c16b827f3ae66ec718e9af49e4d96906fa2ffb
author : Alvaro Herrera <[email protected]>
date : Tue, 1 Sep 2020 20:43:23 -0400
committer: Alvaro Herrera <[email protected]>
date : Tue, 1 Sep 2020 20:43:23 -0400
Introduced by 8b08f7d4820f; backpatch to 11.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_dump.h
Improve handling of dropped relations for REINDEX DATABASE/SCHEMA/SYSTEM
commit : 1d65416661bbb0b165865a521ce038ffb61b12ad
author : Michael Paquier <[email protected]>
date : Wed, 2 Sep 2020 09:08:12 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 2 Sep 2020 09:08:12 +0900
When multiple relations are reindexed, a scan of pg_class is done first
to build the list of relations to work on. However the REINDEX logic
has never checked if a relation listed still exists when beginning the
work on it, causing for example sudden cache lookup failures.
This commit adds safeguards against dropped relations for REINDEX,
similarly to VACUUM or CLUSTER where we try to open the relation,
ignoring it if it is missing. A new option is added to the REINDEX
routines to control if a missed relation is OK to ignore or not.
An isolation test, based on REINDEX SCHEMA, is added for the concurrent
and non-concurrent cases.
Author: Michael Paquier
Reviewed-by: Anastasia Lubennikova
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/table/table.c
M src/backend/catalog/index.c
M src/backend/commands/indexcmds.c
M src/include/access/table.h
M src/include/nodes/parsenodes.h
A src/test/isolation/expected/reindex-schema.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/reindex-schema.spec
Improve test coverage of ginvacuum.c.
commit : 4c51a2d1e4b750bc11b8de9a85b079a14f798741
author : Tom Lane <[email protected]>
date : Tue, 1 Sep 2020 18:40:37 -0400
committer: Tom Lane <[email protected]>
date : Tue, 1 Sep 2020 18:40:37 -0400
Add a test case that exercises vacuum's deletion of empty GIN
posting pages. Since this is a temp table, it should now work
reliably to delete a bunch of rows and immediately VACUUM.
Before the preceding commit, this would not have had the desired
effect, at least not in parallel regression tests.
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/gin.out
M src/test/regress/sql/gin.sql
Set cutoff xmin more aggressively when vacuuming a temporary table.
commit : a7212be8b9e0885ee769e8c55f99ef742cda487b
author : Tom Lane <[email protected]>
date : Tue, 1 Sep 2020 18:37:12 -0400
committer: Tom Lane <[email protected]>
date : Tue, 1 Sep 2020 18:37:12 -0400
Since other sessions aren't allowed to look into a temporary table
of our own session, we do not need to worry about the global xmin
horizon when setting the vacuum XID cutoff. Indeed, if we're not
inside a transaction block, we may set oldestXmin to be the next
XID, because there cannot be any in-doubt tuples in a temp table,
nor any tuples that are dead but still visible to some snapshot of
our transaction. (VACUUM, of course, is never inside a transaction
block; but we need to test that because CLUSTER shares the same code.)
This approach allows us to always clean out a temp table completely
during VACUUM, independently of concurrent activity. Aside from
being useful in its own right, that simplifies building reproducible
test cases.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/cluster.c
M src/backend/commands/vacuum.c
M src/include/commands/cluster.h
M src/include/commands/vacuum.h
doc: clarify that max_wal_size is "during" checkpoints
commit : db864c3c36035e4620afd114c783af7d777d78b0
author : Bruce Momjian <[email protected]>
date : Tue, 1 Sep 2020 17:00:10 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 1 Sep 2020 17:00:10 -0400
Previous wording was "between".
Reported-by: Pavel Luzanov
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
Raise error on concurrent drop of partitioned index
commit : afc7e0ad556a4f720c466cb4815fc77d310fc50a
author : Alvaro Herrera <[email protected]>
date : Tue, 1 Sep 2020 13:40:43 -0400
committer: Alvaro Herrera <[email protected]>
date : Tue, 1 Sep 2020 13:40:43 -0400
We were already raising an error for DROP INDEX CONCURRENTLY on a
partitioned table, albeit a different and confusing one:
ERROR: DROP INDEX CONCURRENTLY must be first action in transaction
Change that to throw a more comprehensible error:
ERROR: cannot drop partitioned index \"%s\" concurrently
Michael Paquier authored the test case for indexes on temporary
partitioned tables.
Backpatch to 11, where indexes on partitioned tables were added.
Reported-by: Jan Mussler <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/drop_index.sgml
M src/backend/commands/tablecmds.c
M src/test/regress/expected/indexing.out
M src/test/regress/sql/indexing.sql
Teach libpq to handle arbitrary-length lines in .pgpass files.
commit : b55b4dad99e99d5306744a4e8ef8021fa3a922e4
author : Tom Lane <[email protected]>
date : Tue, 1 Sep 2020 13:14:44 -0400
committer: Tom Lane <[email protected]>
date : Tue, 1 Sep 2020 13:14:44 -0400
Historically there's been a hard-wired assumption here that no line of
a .pgpass file could be as long as NAMEDATALEN*5 bytes. That's a bit
shaky to start off with, because (a) there's no reason to suppose that
host names fit in NAMEDATALEN, and (b) this figure fails to allow for
backslash escape characters. However, it fails completely if someone
wants to use a very long password, and we're now hearing reports of
people wanting to use "security tokens" that can run up to several
hundred bytes. Another angle is that the file is specified to allow
comment lines, but there's no reason to assume that long comment lines
aren't possible.
Rather than guessing at what might be a more suitable limit, let's
replace the fixed-size buffer with an expansible PQExpBuffer. That
adds one malloc/free cycle to the typical use-case, but that's surely
pretty cheap relative to the I/O this code has to do.
Also, add TAP test cases to exercise this code, because there was no
test coverage before.
This reverts most of commit 2eb3bc588, as there's no longer a need for
a warning message about overlength .pgpass lines. (I kept the explicit
check for comment lines, though.)
In HEAD and v13, this also fixes an oversight in 74a308cf5: there's not
much point in explicit_bzero'ing the line buffer if we only do so in two
of the three exit paths.
Back-patch to all supported branches, except that the test case only
goes back to v10 where src/test/authentication/ was added.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-connect.c
M src/test/authentication/t/001_password.pl
Fix the SharedFileSetUnregister API.
commit : 4ab77697f67aa5b90b032b9175b46901859da6d7
author : Amit Kapila <[email protected]>
date : Tue, 1 Sep 2020 08:11:39 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 1 Sep 2020 08:11:39 +0530
Commit 808e13b282 introduced a few APIs to extend the existing Buffile
interface. In SharedFileSetDeleteOnProcExit, it tries to delete the list
element while traversing the list with 'foreach' construct which makes the
behavior of list traversal unpredictable.
Author: Amit Kapila
Reviewed-by: Dilip Kumar
Tested-by: Dilip Kumar and Neha Sharma
Discussion: https://postgr.es/m/CAA4eK1JhLatVcQ2OvwA_3s0ih6Hx9+kZbq107cXVsSWWukH7vA@mail.gmail.com
M src/backend/storage/file/sharedfileset.c
doc: document how the backup manifest is transferred
commit : ab3c6d41552411ea2fe4904ec8294951c52c113d
author : Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 18:48:38 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 18:48:38 -0400
Reported-by: Bernd Helmle
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M doc/src/sgml/protocol.sgml
doc: add commas after 'i.e.' and 'e.g.'
commit : 953c64e0f67d44b1db97a3d33b329ccd4691f4c6
author : Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 18:33:37 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 18:33:37 -0400
This follows the American format,
https://jakubmarian.com/comma-after-i-e-and-e-g/. There is no intention
of requiring this format for future text, but making existing text
consistent every few years makes sense.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/bki.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/glossary.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/install-windows.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/parallel.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/queries.sgml
M doc/src/sgml/ref/create_database.sgml
M doc/src/sgml/ref/create_event_trigger.sgml
M doc/src/sgml/ref/create_function.sgml
M doc/src/sgml/ref/create_procedure.sgml
M doc/src/sgml/ref/create_statistics.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_restore.sgml
M doc/src/sgml/ref/pg_rewind.sgml
M doc/src/sgml/ref/pgbench.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/postgres-ref.sgml
M doc/src/sgml/ref/prepare.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/replication-origins.sgml
M doc/src/sgml/runtime.sgml
M doc/src/sgml/sepgsql.sgml
M doc/src/sgml/sources.sgml
M doc/src/sgml/sslinfo.sgml
M doc/src/sgml/tableam.sgml
M doc/src/sgml/textsearch.sgml
M doc/src/sgml/wal.sgml
M doc/src/sgml/xfunc.sgml
M doc/src/sgml/xml2.sgml
pg_upgrade doc: mention saving postgresql.conf.auto files
commit : 50ed605b3e311f26497cea979887a8deed3eabf6
author : Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 17:36:23 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 17:36:23 -0400
Also mention files included by postgresql.conf.
Reported-by: Álvaro Herrera
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/ref/pgupgrade.sgml
doc: Update partitioning limitation on BEFORE triggers
commit : 0ebe82a9413fcd5621266ec57d9f4bc50f0457d8
author : Alvaro Herrera <[email protected]>
date : Mon, 31 Aug 2020 17:04:40 -0400
committer: Alvaro Herrera <[email protected]>
date : Mon, 31 Aug 2020 17:04:40 -0400
Reported-by: Erwin Brandstetter <[email protected]>
Discussion: https://postgr.es/m/CAGHENJ6Le7S3qJJx2TvWvTwRNS3N=BtoNeb7AF2rZvfNBMeQcg@mail.gmail.com
M doc/src/sgml/ddl.sgml
docs: in mapping SQL to C data types, timestamp isn't a pointer
commit : 9524fa1aa557b19c9f049b3d79dc93e53cb9d57b
author : Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 17:05:53 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 17:05:53 -0400
It is an int64.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/xfunc.sgml
doc: cross-link file-fdw and CSV config log sections
commit : de2d1920ddf54ec8dbcc87821ade5346f0cf8926
author : Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 16:59:59 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 16:59:59 -0400
There is an file-fdw example that reads the server config file, so cross
link them.
Reported-by: Oleg Samoilov
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
M doc/src/sgml/file-fdw.sgml
docs: clarify intermediate certificate creation instructions
commit : b1ae70b3b4fd54220a0901eaf3dd4c5ca0827108
author : Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 16:21:03 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 16:21:03 -0400
Specifically, explain the v3_ca openssl specification.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/runtime.sgml
docs: replace "stable storage" with "durable" in descriptions
commit : 70e791f47e976a810254d52d38fbc33acdc8705f
author : Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 15:23:19 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 15:23:19 -0400
For PG, "durable storage" has a clear meaning, while "stable storage"
does not, so use the former.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
C comment: remove mention of use of t_hoff WAL structure member
commit : 1e0512ff23e600e9bc19e7f1a1c5ce0597c7bd47
author : Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 13:58:00 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 13:58:00 -0400
Reported-by: Antonin Houska
Discussion: https://postgr.es/m/21643.1595353537@antos
Backpatch-through: 9.5
M src/include/access/heapam_xlog.h
doc: improve description of subscripting of arrays
commit : 06eba0fd10b1c19d578b90f6ab792834fe9a7418
author : Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 13:49:17 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 13:49:17 -0400
It wasn't clear the non-integers are cast to integers for subscripting,
rather than throwing an error.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/syntax.sgml
docs: improve 'capitals' inheritance example
commit : 47c427d006267d752fee4655543ec99dabe9e61d
author : Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 13:43:05 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 13:43:05 -0400
Adds constraints and improves wording.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/advanced.sgml
doc: clarify the useful features of procedures
commit : 243a3b92a67519d4da1712b4fbad194bded3afb5
author : Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 13:20:04 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 31 Aug 2020 13:20:04 -0400
This was not clearly documented when procedures were added in PG 11.
Reported-by: Robin Abbi
Discussion: https://postgr.es/m/CAGmg_NX327KKVuJmbWZD=pGutYFxzZjX1rU+3ji8UuX=8ONn9Q@mail.gmail.com
Backpatch-through: 11
M doc/src/sgml/xfunc.sgml
Fix docs bug stating file_fdw requires absolute paths
commit : 3a788db60108fed2e51f62a79a5f425401300338
author : Magnus Hagander <[email protected]>
date : Mon, 31 Aug 2020 13:03:54 +0200
committer: Magnus Hagander <[email protected]>
date : Mon, 31 Aug 2020 13:03:54 +0200
It has always (since the first commit) worked with relative paths, so
use the same wording as other parts of the documentation.
Author: Bruce Momjian
Discussion: https://postgr.es/m/CABUevExx-hm=cit+A9LeKBH39srvk8Y2tEZeEAj5mP8YfzNKUg@mail.gmail.com
M doc/src/sgml/file-fdw.sgml
Mark factorial operator, and postfix operators in general, as deprecated.
commit : 6ca547cf75ef6e922476c51a3fb5e253eef5f1b6
author : Tom Lane <[email protected]>
date : Sun, 30 Aug 2020 14:37:24 -0400
committer: Tom Lane <[email protected]>
date : Sun, 30 Aug 2020 14:37:24 -0400
Per discussion, we're planning to remove parser support for postfix
operators in order to simplify the grammar. So it behooves us to
put out a deprecation notice at least one release before that.
There is only one built-in postfix operator, ! for factorial.
Label it deprecated in the docs and in pg_description, and adjust
some examples that formerly relied on it. (The sister prefix
operator !! is also deprecated. We don't really have to remove
that one, but since we're suggesting that people use factorial()
instead, it seems better to remove both operators.)
Also state in the CREATE OPERATOR ref page that postfix operators
in general are going away.
Although this changes the initial contents of pg_description,
I did not force a catversion bump; it doesn't seem essential.
In v13, also back-patch 4c5cf5431, so that there's someplace for
the <link>s to point to.
Mark Dilger and John Naylor, with some adjustments by me
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/create_operator.sgml
M doc/src/sgml/syntax.sgml
M doc/src/sgml/typeconv.sgml
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_proc.dat
Redefine pg_class.reltuples to be -1 before the first VACUUM or ANALYZE.
commit : 3d351d916b20534f973eda760cde17d96545d4c4
author : Tom Lane <[email protected]>
date : Sun, 30 Aug 2020 12:21:51 -0400
committer: Tom Lane <[email protected]>
date : Sun, 30 Aug 2020 12:21:51 -0400
Historically, we've considered the state with relpages and reltuples
both zero as indicating that we do not know the table's tuple density.
This is problematic because it's impossible to distinguish "never yet
vacuumed" from "vacuumed and seen to be empty". In particular, a user
cannot use VACUUM or ANALYZE to override the planner's normal heuristic
that an empty table should not be believed to be empty because it is
probably about to get populated. That heuristic is a good safety
measure, so I don't care to abandon it, but there should be a way to
override it if the table is indeed intended to stay empty.
Hence, represent the initial state of ignorance by setting reltuples
to -1 (relpages is still set to zero), and apply the minimum-ten-pages
heuristic only when reltuples is still -1. If the table is empty,
VACUUM or ANALYZE (but not CREATE INDEX) will override that to
reltuples = relpages = 0, and then we'll plan on that basis.
This requires a bunch of fiddly little changes, but we can get rid of
some ugly kluges that were formerly needed to maintain the old definition.
One notable point is that FDWs' GetForeignRelSize methods will see
baserel->tuples = -1 when no ANALYZE has been done on the foreign table.
That seems like a net improvement, since those methods were formerly
also in the dark about what baserel->tuples = 0 really meant. Still,
it is an API change.
I bumped catversion because code predating this change would get confused
by seeing reltuples = -1.
Discussion: https://postgr.es/m/[email protected]
M contrib/file_fdw/file_fdw.c
M contrib/pgstattuple/pgstatapprox.c
M contrib/postgres_fdw/postgres_fdw.c
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/fdwhandler.sgml
M src/backend/access/gin/ginvacuum.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/table/tableam.c
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/commands/vacuum.c
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/util/plancat.c
M src/backend/postmaster/autovacuum.c
M src/backend/rewrite/rewriteDefine.c
M src/backend/utils/cache/relcache.c
M src/include/access/genam.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_class.h
Reset indisreplident for an invalid index in DROP INDEX CONCURRENTLY
commit : 9511fb37ac78c77736e5483118265f7e83cd9f3c
author : Michael Paquier <[email protected]>
date : Sun, 30 Aug 2020 14:14:34 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 30 Aug 2020 14:14:34 +0900
A failure when dropping concurrently an index used in a replica identity
could leave in pg_index an index marked as !indisvalid and
indisreplident. Reindexing this index would switch back indisvalid to
true, and if the replica identity of the parent relation was switched to
use a different index, it would be possible to finish with more than one
index marked as indisreplident. If that were to happen, this could mess
up with the relation cache as an incorrect index could be used for the
replica identity.
Indexes marked as invalid are discarded as candidates for the replica
identity, as of RelationGetIndexList(), so similarly to what is done
with indisclustered, resetting indisreplident when the index is marked
as invalid keeps things consistent. REINDEX CONCURRENTLY's swapping
already resets the flag for the old index, while the new index inherits
the value of the old index to-be-dropped, so only DROP INDEX was an
issue.
Even if this is a bug, the sequence able to reproduce a problem requires
a failure while running DROP INDEX CONCURRENTLY, something unlikely
going to happen in the field, so no backpatch is done.
Author: Michael Paquier
Reviewed-by: Dmitry Dolgov
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/index.c
doc: Rework tables for built-in operator classes of index AMs
commit : 7a1cd5260aa20bc13aec8960a57904b5623d1830
author : Michael Paquier <[email protected]>
date : Fri, 28 Aug 2020 16:54:59 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 28 Aug 2020 16:54:59 +0900
The tables listing all the operator classes available for BRIN, GIN,
GiST and SP-GiST had a confusing format where the same operator could be
listed multiple times, for different data types. This improves the
shape of these tables by adding the types associated to each operator,
for their associated operator class.
Each table included previously the data type that could be used for an
operator class in an extra column. This is removed to reduce the width
of the tables as this is now described within each operator. This also
makes the tables fit better in the PDF documentation.
Reported-by: osdba
Author: Michael Paquier
Reviewed-by: Álvaro Herrera, Tom Lane, Bruce Momjian
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/brin.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/spgist.sgml
doc: Update cracklib URL
commit : 42aaed60c83ff51aa736f50ad96e43653fc539da
author : Peter Eisentraut <[email protected]>
date : Fri, 28 Aug 2020 08:19:12 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 28 Aug 2020 08:19:12 +0200
Author: Daniel Gustafsson <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/f7266133-618a-0adc-52ef-f43c78806b0e%402ndquadrant.com
M doc/src/sgml/passwordcheck.sgml
passwordcheck: Log cracklib diagnostics
commit : 924123a87f40c12063a2bb2500805447cddc02a3
author : Peter Eisentraut <[email protected]>
date : Fri, 28 Aug 2020 08:16:32 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 28 Aug 2020 08:16:32 +0200
When calling cracklib to check the password, the diagnostic from
cracklib was thrown away. This would hide essential information such
as no dictionary being installed. Change this to show the cracklib
error message using errdetail_log().
Reviewed-by: Daniel Gustafsson <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/f7266133-618a-0adc-52ef-f43c78806b0e%402ndquadrant.com
M contrib/passwordcheck/passwordcheck.c
Fix code for re-finding scan position in a multicolumn GIN index.
commit : 10564ee02ca380f8d614eabc4e80c5d39ea4edad
author : Tom Lane <[email protected]>
date : Thu, 27 Aug 2020 17:36:13 -0400
committer: Tom Lane <[email protected]>
date : Thu, 27 Aug 2020 17:36:13 -0400
collectMatchBitmap() needs to re-find the index tuple it was previously
looking at, after transiently dropping lock on the index page it's on.
The tuple should still exist and be at its prior position or somewhere
to the right of that, since ginvacuum never removes tuples but
concurrent insertions could add one. However, there was a thinko in
that logic, to the effect of expecting any inserted tuples to have the
same index "attnum" as what we'd been scanning. Since there's no
physical separation of tuples with different attnums, it's not terribly
hard to devise scenarios where this fails, leading to transient "lost
saved point in index" errors. (While I've duplicated this with manual
testing, it seems impossible to make a reproducible test case with our
available testing technology.)
Fix by just continuing the scan when the attnum doesn't match.
While here, improve the error message used if we do fail, so that it
matches the wording used in btree for a similar case.
collectMatchBitmap()'s posting-tree code path was previously not
exercised at all by our regression tests. While I can't make
a regression test that exhibits the bug, I can at least improve
the code coverage here, so do that. The test case I made for this
is an extension of one added by 4b754d6c1, so it only works in
HEAD and v13; didn't seem worth trying hard to back-patch it.
Per bug #16595 from Jesse Kinkead. This has been broken since
multicolumn capability was added to GIN (commit 27cb66fdf),
so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/gin/ginget.c
M src/test/regress/expected/gin.out
M src/test/regress/sql/gin.sql
Fix comment in procarray.c
commit : 77c7267c37f7fa8e5e48abda4798afdbecb2b95a
author : Michael Paquier <[email protected]>
date : Thu, 27 Aug 2020 16:40:34 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 27 Aug 2020 16:40:34 +0900
The description of GlobalVisDataRels was missing, GlobalVisCatalogRels
being mentioned instead.
Author: Jim Nasby
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/procarray.c
Suppress compiler warning in non-cassert builds.
commit : e942af7b8261cd8070d0eeaf518dbc1a664859fd
author : Tom Lane <[email protected]>
date : Wed, 26 Aug 2020 17:08:11 -0400
committer: Tom Lane <[email protected]>
date : Wed, 26 Aug 2020 17:08:11 -0400
Oversight in 808e13b28, reported by Bruce Momjian.
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/file/sharedfileset.c
Add regression tests for REPLICA IDENTITY with dropped indexes
commit : fe7fd4e9613f58262d30782a34b01cc0c4cbbeb5
author : Michael Paquier <[email protected]>
date : Wed, 26 Aug 2020 20:42:27 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 26 Aug 2020 20:42:27 +0900
REPLICA IDENTITY USING INDEX behaves the same way as NOTHING if the
associated index is dropped, even if there is a primary key that could
be used as a fallback for the changes generated. There have never been
any tests to cover such scenarios, so this commit closes the gap.
Author: Michael Paquier
Reviewed-by: Masahiko Sawada, Rahila Syed, Euler Taveira
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/expected/ddl.out
M contrib/test_decoding/sql/ddl.sql
Add additional information in the vacuum error context.
commit : 7e453634bb62f06a048f5562ba59d52aa1f28d12
author : Amit Kapila <[email protected]>
date : Wed, 26 Aug 2020 09:40:52 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 26 Aug 2020 09:40:52 +0530
The additional information added will be an offset number for heap
operations. This information will help us in finding the exact tuple due
to which the error has occurred.
Author: Mahendra Singh Thalor and Amit Kapila
Reviewed-by: Sawada Masahiko, Justin Pryzby and Amit Kapila
Discussion: https://postgr.es/m/CAKYtNApK488TDF4bMbw+1QH8HJf9cxdNDXquhU50TK5iv_FtCQ@mail.gmail.com
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/vacuumlazy.c
M src/include/access/heapam.h
Extend the BufFile interface.
commit : 808e13b282efa7e7ac7b78e886aca5684f4bccd3
author : Amit Kapila <[email protected]>
date : Wed, 26 Aug 2020 07:36:43 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 26 Aug 2020 07:36:43 +0530
Allow BufFile to support temporary files that can be used by the single
backend when the corresponding files need to be survived across the
transaction and need to be opened and closed multiple times. Such files
need to be created as a member of a SharedFileSet.
Additionally, this commit implements the interface for BufFileTruncate to
allow files to be truncated up to a particular offset and extends the
BufFileSeek API to support the SEEK_END case. This also adds an option to
provide a mode while opening the shared BufFiles instead of always opening
in read-only mode.
These enhancements in BufFile interface are required for the upcoming
patch to allow the replication apply worker, to handle streamed
in-progress transactions.
Author: Dilip Kumar, Amit Kapila
Reviewed-by: Amit Kapila
Tested-by: Neha Sharma
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
M src/backend/postmaster/pgstat.c
M src/backend/storage/file/buffile.c
M src/backend/storage/file/fd.c
M src/backend/storage/file/sharedfileset.c
M src/backend/utils/sort/logtape.c
M src/backend/utils/sort/sharedtuplestore.c
M src/include/pgstat.h
M src/include/storage/buffile.h
M src/include/storage/fd.h
M src/include/storage/sharedfileset.h
Add regression test for pg_backend_memory_contexts.
commit : adc8fc6167aa3f68b951ddd60ea32a62b13f18d6
author : Fujii Masao <[email protected]>
date : Wed, 26 Aug 2020 10:52:02 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 26 Aug 2020 10:52:02 +0900
Author: Atsushi Torikoshi
Reviewed-by: Michael Paquier, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/sysviews.sql
Move codes for pg_backend_memory_contexts from mmgr/mcxt.c to adt/mcxtfuncs.c.
commit : 50db5964ee333bc148e0c8844ffafaf585c719c6
author : Fujii Masao <[email protected]>
date : Wed, 26 Aug 2020 10:51:31 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 26 Aug 2020 10:51:31 +0900
Previously the codes for pg_backend_memory_contexts were in
src/backend/utils/mmgr/mcxt.c. This commit moves them to
src/backend/utils/adt/mcxtfuncs.c so that mcxt.c basically includes
only the low-level interface for memory contexts.
Author: Atsushi Torikoshi
Reviewed-by: Michael Paquier, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/Makefile
A src/backend/utils/adt/mcxtfuncs.c
M src/backend/utils/mmgr/mcxt.c
Prevent non-superusers from reading pg_backend_memory_contexts, by default.
commit : 29dd6d8bc631eebc3e50493c115f7a215f03bd0a
author : Fujii Masao <[email protected]>
date : Wed, 26 Aug 2020 10:50:02 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 26 Aug 2020 10:50:02 +0900
pg_backend_memory_contexts view contains some internal information of
memory contexts. Since exposing them to any users by default may cause
security issue, this commit allows only superusers to read this view,
by default, like we do for pg_shmem_allocations view.
Bump catalog version.
Author: Atsushi Torikoshi
Reviewed-by: Michael Paquier, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/system_views.sql
M src/include/catalog/catversion.h
Fixup some misusages of bms_num_members()
commit : c34605daed563fcade07a9f45bcf440459599c00
author : David Rowley <[email protected]>
date : Wed, 26 Aug 2020 10:51:36 +1200
committer: David Rowley <[email protected]>
date : Wed, 26 Aug 2020 10:51:36 +1200
It's a bit inefficient to test if a Bitmapset is empty by counting all the
members and seeing if that number is zero. It's much better just to use
bms_is_empty(). Likewise for checking if there are at least two members,
just use bms_membership(), which does not need to do anything more after
finding two members.
Discussion: https://postgr.es/m/CAApHDvpvwm_QjbDOb5xga%2BKmX9XkN9xQavNGm3SvDbVnCYOerQ%40mail.gmail.com
Reviewed-by: Tomas Vondra
M src/backend/optimizer/path/clausesel.c
M src/backend/statistics/dependencies.c
docs: client certificates are always sent to the server
commit : ff60394a8c9a7af8b32de420ccb54a20a0f019c1
author : Bruce Momjian <[email protected]>
date : Tue, 25 Aug 2020 09:53:12 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 25 Aug 2020 09:53:12 -0400
They are not "requested" by the server.
Reported-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
doc: Fix up title case
commit : 7f055fba3fa99d807837a229967fd6c5dd720530
author : Peter Eisentraut <[email protected]>
date : Tue, 25 Aug 2020 07:29:05 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 25 Aug 2020 07:29:05 +0200
This fixes some instances that were missed in earlier processings and
that now look a bit strange because they are inconsistent with nearby
titles.
M doc/src/sgml/dml.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/plpgsql.sgml
doc: Fix some markups for support functions of index AMs
commit : 77c1537f512e6ac2513f8695c795dc94cbf207ee
author : Michael Paquier <[email protected]>
date : Mon, 24 Aug 2020 16:46:52 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 24 Aug 2020 16:46:52 +0900
All the documentation of index AMs has been using <replaceable> for
local_relopts. This is a structure, so <structname> is a much better
choice.
Alexander has found the inconsistency for btree, while I have spotted
the rest when applying the concept of consistency to the docs.
Author: Alexander Lakhin, Michael Paquier
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/brin.sgml
M doc/src/sgml/btree.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/spgist.sgml
Improve the vacuum error context phase information.
commit : a3c66de6c5e1ee9dd41ce1454496568622fb7712
author : Amit Kapila <[email protected]>
date : Mon, 24 Aug 2020 08:16:19 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 24 Aug 2020 08:16:19 +0530
We were displaying the wrong phase information for 'info' message in the
index clean up phase because we were switching to the previous phase a bit
early. We were also not displaying context information for heap phase
unless the block number is valid which is fine for error cases but for
messages at 'info' or lower error level it appears to be inconsistent with
index phase information.
Reported-by: Sawada Masahiko
Author: Sawada Masahiko
Reviewed-by: Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CA+fd4k4HcbhPnCs7paRTw1K-AHin8y4xKomB9Ru0ATw0UeTy2w@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Avoid pushing quals down into sub-queries that have grouping sets.
commit : 4d346def1555ea55b3adf76fc4afa3d3495ecfdd
author : Tom Lane <[email protected]>
date : Sat, 22 Aug 2020 14:46:40 -0400
committer: Tom Lane <[email protected]>
date : Sat, 22 Aug 2020 14:46:40 -0400
The trouble with doing this is that an apparently-constant subquery
output column isn't really constant if it is a grouping column that
appears in only some of the grouping sets. A qual using such a
column would be subject to incorrect const-folding after push-down,
as seen in bug #16585 from Paul Sivash.
To fix, just disable qual pushdown altogether if the sub-query has
nonempty groupingSets. While we could imagine far less restrictive
solutions, there is not much point in working harder right now,
because subquery_planner() won't move HAVING clauses to WHERE within
such a subquery. If the qual stays in HAVING it's not going to be
a lot more useful than if we'd kept it at the outer level.
Having said that, this restriction could be removed if we used a
parsetree representation that distinguished such outputs from actual
constants, which is something I hope to do in future. Hence, make
the patch a minimal addition rather than integrating it more tightly
(e.g. by renumbering the existing items in subquery_is_pushdown_safe's
comment).
Back-patch to 9.5 where grouping sets were introduced.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/path/allpaths.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Fix ALTER TABLE's scheduling rules for AT_AddConstraint subcommands.
commit : 5b02d68e758307e0ae8fae4d7bbcd687f1dd6ce1
author : Tom Lane <[email protected]>
date : Sat, 22 Aug 2020 12:34:17 -0400
committer: Tom Lane <[email protected]>
date : Sat, 22 Aug 2020 12:34:17 -0400
Commit 1281a5c90 rearranged the logic in this area rather drastically,
and it broke the case of adding a foreign key constraint in the same
ALTER that adds the pkey or unique constraint it depends on. While
self-referential fkeys are surely a pretty niche case, this used to
work so we shouldn't break it.
To fix, reorganize the scheduling rules in ATParseTransformCmd so
that a transformed AT_AddConstraint subcommand will be delayed into
a later pass in all cases, not only when it's been spit out as a
side-effect of parsing some other command type.
Also tweak the logic so that we won't run ATParseTransformCmd twice
while doing this. It seems to work even without that, but it's
surely wasting cycles to do so.
Per bug #16589 from Jeremy Evans. Back-patch to v13 where the new
code was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
doc: Fix format, incorrect structure names and markup inconsistencies
commit : c3a288649e152612791121fa6d17a1322b8f2814
author : Michael Paquier <[email protected]>
date : Sat, 22 Aug 2020 22:26:10 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 22 Aug 2020 22:26:10 +0900
Author: Alexander Lakhin
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M doc/src/sgml/datetime.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/protocol.sgml
docs: improve description of how to handle multiple databases
commit : 2a9f37243b0b0b3621f1851a6a8644d4ca2749d6
author : Bruce Momjian <[email protected]>
date : Fri, 21 Aug 2020 20:23:09 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 21 Aug 2020 20:23:09 -0400
This is a redesign of the intro to the managing databases chapter.
Discussion: https://postgr.es/m/[email protected]
Author: David G. Johnston
Backpatch-through: 9.5
M doc/src/sgml/manage-ag.sgml
docs: add COMMENT examples for new features, rename rtree
commit : bfd78c0b41c5d59e6850dee412f32748da0a3c11
author : Bruce Momjian <[email protected]>
date : Fri, 21 Aug 2020 18:29:37 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 21 Aug 2020 18:29:37 -0400
Reported-by: Jürgen Purtz
Discussion: https://postgr.es/m/[email protected]
Author: Jürgen Purtz
Backpatch-through: 11
M doc/src/sgml/ref/comment.sgml
Fix handling of CREATE TABLE LIKE with inheritance.
commit : 50289819230d8ddad510879ee4793b04a05cf13b
author : Tom Lane <[email protected]>
date : Fri, 21 Aug 2020 15:00:42 -0400
committer: Tom Lane <[email protected]>
date : Fri, 21 Aug 2020 15:00:42 -0400
If a CREATE TABLE command uses both LIKE and traditional inheritance,
Vars in CHECK constraints and expression indexes that are absorbed
from a LIKE parent table tended to get mis-numbered, resulting in
wrong answers and/or bizarre error messages (though probably not any
actual crashes, thanks to validation occurring in the executor).
In v12 and up, the same could happen to Vars in GENERATED expressions,
even in cases with no LIKE clause but multiple traditional-inheritance
parents.
The cause of the problem for LIKE is that parse_utilcmd.c supposed
it could renumber such Vars correctly during transformCreateStmt(),
which it cannot since we have not yet accounted for columns added via
inheritance. Fix that by postponing processing of LIKE INCLUDING
CONSTRAINTS, DEFAULTS, GENERATED, INDEXES till after we've performed
DefineRelation().
The error with GENERATED and multiple inheritance is a simple oversight
in MergeAttributes(); it knows it has to renumber Vars in inherited
CHECK constraints, but forgot to apply the same processing to inherited
GENERATED expressions (a/k/a defaults).
Per bug #16272 from Tom Gottfried. The non-GENERATED variants of the
issue are ancient, presumably dating right back to the addition of
CREATE TABLE LIKE; hence back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/backend/parser/parse_utilcmd.c
M src/backend/tcop/utility.c
M src/include/nodes/parsenodes.h
M src/include/parser/parse_utilcmd.h
M src/test/modules/test_ddl_deparse/expected/create_table.out
M src/test/modules/test_ddl_deparse/test_ddl_deparse.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Fix explain regression test failure.
commit : eabba4a3eb71b3886d0ec581155df6202b96b15a
author : Fujii Masao <[email protected]>
date : Sat, 22 Aug 2020 01:22:55 +0900
committer: Fujii Masao <[email protected]>
date : Sat, 22 Aug 2020 01:22:55 +0900
Commit 9d701e624f caused the regression test for EXPLAIN to fail on
the buildfarm member prion. This happened because of instability of
test output, i.e., in text format, whether "Planning:" line is output
varies depending on the system state.
This commit updated the regression test so that it ignores that
"Planning:" line to produce more stable test output and get rid of
the test failure.
Back-patch to v13.
Author: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/explain.out
M src/test/regress/sql/explain.sql
Rework EXPLAIN for planner's buffer usage.
commit : 9d701e624f4b9386cbd99519dab7936afe3d5aed
author : Fujii Masao <[email protected]>
date : Fri, 21 Aug 2020 20:48:59 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 21 Aug 2020 20:48:59 +0900
Commit ce77abe63c allowed EXPLAIN (BUFFERS) to report the information
on buffer usage during planning phase. However three issues were
reported regarding this feature.
(1) Previously, EXPLAIN option BUFFERS required ANALYZE. So the query
had to be actually executed by specifying ANALYZE even when we
want to see only the planner's buffer usage. This was inconvenient
especially when the query was write one like DELETE.
(2) EXPLAIN included the planner's buffer usage in summary
information. So SUMMARY option had to be enabled to report that.
Also this format was confusing.
(3) The output structure for planning information was not consistent
between TEXT format and the others. For example, "Planning" tag
was output in JSON format, but not in TEXT format.
For (1), this commit allows us to perform EXPLAIN (BUFFERS) without
ANALYZE to report the planner's buffer usage.
For (2), this commit changed EXPLAIN output so that the planner's
buffer usage is reported before summary information.
For (3), this commit made the output structure for planning
information more consistent between the formats.
Back-patch to v13 where the planner's buffer usage was allowed to
be reported in EXPLAIN.
Reported-by: Pierre Giraud, David Rowley
Author: Fujii Masao
Reviewed-by: David Rowley, Julien Rouhaud, Pierre Giraud
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/explain.sgml
M src/backend/commands/explain.c
M src/test/regress/expected/explain.out
M src/test/regress/sql/explain.sql
Fix typos in comments.
commit : d259afa7365165760004c2fdbe2520a94ddf2600
author : Fujii Masao <[email protected]>
date : Fri, 21 Aug 2020 12:33:30 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 21 Aug 2020 12:33:30 +0900
Author: Masahiko Sawada
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CA+fd4k4m9hFSrRLB3etPWO5_v5=MujVZWRtz63q+55hM0Dz25Q@mail.gmail.com
M src/backend/storage/ipc/procarray.c
Fix a few typos in JIT comments and README
commit : 8431d33079a2c552aaa223ebcfd470572d90146b
author : David Rowley <[email protected]>
date : Fri, 21 Aug 2020 09:33:56 +1200
committer: David Rowley <[email protected]>
date : Fri, 21 Aug 2020 09:33:56 +1200
Reviewed-by: Abhijit Menon-Sen
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/CAApHDvobgmCs6CohqhKTUf7D8vffoZXQTCBTERo9gbOeZmvLTw%40mail.gmail.com
Backpatch-through: 11, where JIT was added
M src/backend/jit/README
M src/include/jit/llvmjit_emit.h
Revert "Make vacuum a bit more verbose to debug BF failure."
commit : c62a0a49f33a0d45a97aa1d3a5bc6ddc83f10d82
author : Andres Freund <[email protected]>
date : Thu, 20 Aug 2020 12:59:00 -0700
committer: Andres Freund <[email protected]>
date : Thu, 20 Aug 2020 12:59:00 -0700
This reverts commit 49967da65aec970fcda123acc681f1df5d70bfc6.
Enough time has passed that we can be confident that 07f32fcd23a
resolved the issue. Therefore we can remove the temporary debugging
aids.
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam.c
M src/backend/access/heap/vacuumlazy.c
Revise REINDEX CONCURRENTLY recovery instructions
commit : 0784c333728dd454b80c0bd0faec916782370810
author : Alvaro Herrera <[email protected]>
date : Thu, 20 Aug 2020 13:49:04 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 20 Aug 2020 13:49:04 -0400
When the leftover invalid index is "ccold", there's no need to re-run
the command. Reword the instructions to make that explicit.
Backpatch to 12, where REINDEX CONCURRENTLY appeared.
Author: Álvaro Herrera <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Reviewed-by: Julien Rouhaud <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/reindex.sgml
Acquire ProcArrayLock exclusively in ProcArrayClearTransaction.
commit : 1fe1f42e3e85279e1cb8b004b3b076a04bde4cee
author : Andres Freund <[email protected]>
date : Wed, 19 Aug 2020 18:19:52 -0700
committer: Andres Freund <[email protected]>
date : Wed, 19 Aug 2020 18:19:52 -0700
This corrects an oversight by me in 20729324078, which made
ProcArrayClearTransaction() increment xactCompletionCount. That requires an
exclusive lock, obviously.
There's other approaches that avoid the exclusive acquisition, but given that a
2PC commit is fairly heavyweight, it doesn't seem worth doing so. I've not been
able to measure a performance difference, unsurprisingly. I did add a
comment documenting that we could do so, should it ever become a bottleneck.
Reported-By: Tom Lane <[email protected]>
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/procarray.c
Suppress unnecessary RelabelType nodes in yet more cases.
commit : 20729324078055a4d9654fc5af9570fe625786a5
author : Tom Lane <[email protected]>
date : Wed, 19 Aug 2020 14:07:49 -0400
committer: Tom Lane <[email protected]>
date : Wed, 19 Aug 2020 14:07:49 -0400
Commit a477bfc1d fixed eval_const_expressions() to ensure that it
didn't generate unnecessary RelabelType nodes, but I failed to notice
that some other places in the planner had the same issue. Really
noplace in the planner should be using plain makeRelabelType(), for
fear of generating expressions that should be equal() to semantically
equivalent trees, but aren't.
An example is that because canonicalize_ec_expression() failed
to be careful about this, we could end up with an equivalence class
containing both a plain Const, and a Const-with-RelabelType
representing exactly the same value. So far as I can tell this led to
no visible misbehavior, but we did waste a bunch of cycles generating
and evaluating "Const = Const-with-RelabelType" to prove such entries
are redundant.
Hence, move the support function added by a477bfc1d to where it can
be more generally useful, and use it in the places where planner code
previously used makeRelabelType.
Back-patch to v12, like the previous patch. While I have no concrete
evidence of any real misbehavior here, it's certainly possible that
I overlooked a case where equivalent expressions that aren't equal()
could cause a user-visible problem. In any case carrying extra
RelabelType nodes through planning to execution isn't very desirable.
Discussion: https://postgr.es/m/[email protected]
M src/backend/nodes/nodeFuncs.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/clauses.c
M src/include/nodes/nodeFuncs.h
Add pg_backend_memory_contexts system view.
commit : 3e98c0bafb28de87ae095b341687dc082371af54
author : Fujii Masao <[email protected]>
date : Wed, 19 Aug 2020 15:34:43 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 19 Aug 2020 15:34:43 +0900
This view displays the usages of all the memory contexts of the server
process attached to the current session. This information is useful to
investigate the cause of backend-local memory bloat.
This information can be also collected by calling
MemoryContextStats(TopMemoryContext) via a debugger. But this technique
cannot be uesd in some environments because no debugger is available there.
And it outputs lots of text messages and it's not easy to analyze them.
So, pg_backend_memory_contexts view allows us to access to backend-local
memory contexts information more easily.
Bump catalog version.
Author: Atsushi Torikoshi, Fujii Masao
Reviewed-by: Tatsuhito Kasahara, Andres Freund, Daniel Gustafsson, Robert Haas, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/system_views.sql
M src/backend/utils/mmgr/mcxt.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/rules.out
Fix race condition in snapshot caching when 2PC is used.
commit : 07f32fcd23ac81898ed47f88beb569c631a2f223
author : Andres Freund <[email protected]>
date : Tue, 18 Aug 2020 16:31:12 -0700
committer: Andres Freund <[email protected]>
date : Tue, 18 Aug 2020 16:31:12 -0700
When preparing a transaction xactCompletionCount needs to be
incremented, even though the transaction has not committed
yet. Otherwise the snapshot used within the transaction otherwise can
get reused outside of the prepared transaction. As GetSnapshotData()
does not include the current xid when building a snapshot, reuse would
not be correct.
Somewhat surprisingly the regression tests only rarely show incorrect
results without the fix. The reason for that is that often the
snapshot's xmax will be >= the backend xid, yielding a snapshot that
is correct, despite the bug.
I'm working on a reliable test for the bug, but it seems worth seeing
whether this fixes all the BF failures while I do.
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/procarray.c
Avoid non-constant format string argument to fprintf().
commit : 734478200ababcbb328ec3f02a74047bc470cae2
author : Heikki Linnakangas <[email protected]>
date : Tue, 18 Aug 2020 13:13:09 +0300
committer: Heikki Linnakangas <[email protected]>
date : Tue, 18 Aug 2020 13:13:09 +0300
As Tom Lane pointed out, it could defeat the compiler's printf() format
string verification.
Backpatch to v12, like that patch that introduced it.
Discussion: https://www.postgresql.org/message-id/1069283.1597672779%40sss.pgh.pa.us
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_checksums/pg_checksums.c
M src/bin/pg_rewind/pg_rewind.c
snapshot scalability: cache snapshots using a xact completion counter.
commit : 623a9ba79bbdd11c5eccb30b8bd5c446130e521c
author : Andres Freund <[email protected]>
date : Mon, 17 Aug 2020 21:07:10 -0700
committer: Andres Freund <[email protected]>
date : Mon, 17 Aug 2020 21:07:10 -0700
Previous commits made it faster/more scalable to compute snapshots. But not
building a snapshot is still faster. Now that GetSnapshotData() does not
maintain RecentGlobal* anymore, that is actually not too hard:
This commit introduces xactCompletionCount, which tracks the number of
top-level transactions with xids (i.e. which may have modified the database)
that completed in some form since the start of the server.
We can avoid rebuilding the snapshot's contents whenever the current
xactCompletionCount is the same as it was when the snapshot was
originally built. Currently this check happens while holding
ProcArrayLock. While it's likely possible to perform the check without
acquiring ProcArrayLock, it seems better to do that separately /
later, some careful analysis is required. Even with the lock this is a
significant win on its own.
On a smaller two socket machine this gains another ~1.03x, on a larger
machine the effect is roughly double (earlier patch version tested
though). If we were able to safely avoid the lock there'd be another
significant gain on top of that.
Author: Andres Freund <[email protected]>
Reviewed-By: Robert Haas <[email protected]>
Reviewed-By: Thomas Munro <[email protected]>
Reviewed-By: David Rowley <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/snapbuild.c
M src/backend/storage/ipc/procarray.c
M src/backend/utils/time/snapmgr.c
M src/include/access/transam.h
M src/include/utils/snapshot.h
Fix use-after-release issue in PL/Sample
commit : 51300b45db95b6fd29f88534ab0739fdc9df1699
author : Michael Paquier <[email protected]>
date : Tue, 18 Aug 2020 12:24:22 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 18 Aug 2020 12:24:22 +0900
Introduced in adbe62d0. Per buildfarm member prion, when using
RELCACHE_FORCE_RELEASE.
M src/test/modules/plsample/plsample.c
Add PL/Sample to src/test/modules/
commit : adbe62d04b360bbd408d97e447932d8078485972
author : Michael Paquier <[email protected]>
date : Tue, 18 Aug 2020 11:10:50 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 18 Aug 2020 11:10:50 +0900
PL/Sample is an example template of procedural-language handler. This
can be used as a base to implement a custom PL, or as a facility to test
APIs dedicated to PLs. Much more could be done in this module, like
adding a simple validator, but this is left as future work.
The documentation included originally some C code to understand the
basics of PL handler implementation, but it was outdated, and not really
helpful either if trying to implement a new procedural language,
particularly when it came to the integration of a PL installation with
CREATE EXTENSION.
Author: Mark Wong
Reviewed-by: Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/plhandler.sgml
M src/test/modules/Makefile
A src/test/modules/plsample/.gitignore
A src/test/modules/plsample/Makefile
A src/test/modules/plsample/README
A src/test/modules/plsample/expected/plsample.out
A src/test/modules/plsample/plsample–1.0.sql
A src/test/modules/plsample/plsample.c
A src/test/modules/plsample/plsample.control
A src/test/modules/plsample/sql/plsample.sql
Disable autovacuum for BRIN test table
commit : 6e70443edacfc86674995c0c10ade0aec7a4fddf
author : Alvaro Herrera <[email protected]>
date : Mon, 17 Aug 2020 16:20:06 -0400
committer: Alvaro Herrera <[email protected]>
date : Mon, 17 Aug 2020 16:20:06 -0400
This should improve stability in the tests.
Per buildfarm member hyrax (CLOBBER_CACHE_ALWAYS) via Tom Lane.
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/brin.out
M src/test/regress/sql/brin.sql
Doc: fix description of UNION/CASE/etc type unification.
commit : 22e75a341ecc841bdc1db417d11a643b0a42df4f
author : Tom Lane <[email protected]>
date : Mon, 17 Aug 2020 15:40:07 -0400
committer: Tom Lane <[email protected]>
date : Mon, 17 Aug 2020 15:40:07 -0400
The description of what select_common_type() does was not terribly
accurate. Improve it.
David Johnston and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/typeconv.sgml
Mark commit and abort WAL records with XLR_SPECIAL_REL_UPDATE.
commit : a28d731a1187e8d9d8c2b6319375fcbf0a8debd5
author : Heikki Linnakangas <[email protected]>
date : Mon, 17 Aug 2020 10:52:58 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 17 Aug 2020 10:52:58 +0300
If a commit or abort record includes "dropped relfilenodes", then replaying
the record will remove data files. That is surely a "special rel update",
but the records were not marked as such. Fix that, teach pg_rewind to
expect and ignore them, and add a test case to cover it.
It's always been like this, but no backporting for fear of breaking
existing applications. If an application parsed the WAL but was not
handling commit/abort records, it would stop working. That might be a good
thing if it really needed to handle the dropped rels, but it will be caught
when the application is updated to work with PostgreSQL v14 anyway.
Discussion: https://www.postgresql.org/message-id/07b33e2c-46a6-86a1-5f9e-a7da73fddb95%40iki.fi
Reviewed-by: Amit Kapila, Michael Paquier
M src/backend/access/transam/xact.c
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/t/001_basic.pl
Make xact.h usable in frontend.
commit : 3941eb6341d8274dd63a26972042da6632533f2b
author : Heikki Linnakangas <[email protected]>
date : Mon, 17 Aug 2020 10:50:13 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 17 Aug 2020 10:50:13 +0300
xact.h included utils/datetime.h, which cannot be used in the frontend
(it includes fmgr.h, which needs Datum). But xact.h only needs the
definition of TimestampTz from it, which is available directly in
datatypes/timestamp.h. Change xact.h to include that instead of
utils/datetime.h, so that it can be used in client programs.
M contrib/pg_prewarm/autoprewarm.c
M contrib/postgres_fdw/connection.c
M src/backend/nodes/params.c
M src/backend/utils/time/snapmgr.c
M src/include/access/xact.h
Fix printing last progress report line in client programs.
commit : d7ec8337f9093b097f08f94e5ecec36303ad73fd
author : Heikki Linnakangas <[email protected]>
date : Mon, 17 Aug 2020 09:27:29 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 17 Aug 2020 09:27:29 +0300
A number of client programs have a "--progress" option that when printing
to a TTY, updates the current line by printing a '\r' and overwriting it.
After the last line, '\n' needs to be printed to move the cursor to the
next line. pg_basebackup and pgbench got this right, but pg_rewind and
pg_checksums were slightly wrong. pg_rewind printed the newline to stdout
instead of stderr, and pg_checksums printed the newline even when not
printing to a TTY. Fix them, and also add a 'finished' argument to
pg_basebackup's progress_report() function, to keep it consistent with
the other programs.
Backpatch to v12. pg_rewind's newline was broken with the logging changes
in commit cc8d415117 in v12, and pg_checksums was introduced in v12.
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_checksums/pg_checksums.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/pg_rewind.h
doc: Fix description about bgwriter and checkpoint in HA section
commit : b4f16397af460d9d6ead31b86cb3e7f562806866
author : Michael Paquier <[email protected]>
date : Mon, 17 Aug 2020 10:23:17 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 17 Aug 2020 10:23:17 +0900
Since 806a2ae, the work of the bgwriter is split the checkpointer, but a
portion of the documentation did not get the message.
Author: Masahiko Sawada
Discussion: https://postgr.es/m/CA+fd4k6jXxjAtjMVC=wG3=QGpauZBtcgN3Jhw+oV7zXGKVLKzQ@mail.gmail.com
Backpatch-through: 9.5
M doc/src/sgml/high-availability.sgml
Fix use of wrong index in ComputeXidHorizons().
commit : f6661d3df228dbbf50efb04f2b760774a6f2bfff
author : Andres Freund <[email protected]>
date : Sun, 16 Aug 2020 14:21:37 -0700
committer: Andres Freund <[email protected]>
date : Sun, 16 Aug 2020 14:21:37 -0700
This bug, recently introduced in 941697c3c1a, at least lead to vacuum
failing because it found tuples inserted by a running transaction, but
below the freeze limit. The freeze limit in turn is directly affected
by the aforementioned bug.
Thanks to Tom Lane figuring how to make the bug reproducible.
We should add a few more assertions to make sure this type of bug
isn't as hard to notice, but it's not yet clear how to best do so.
Co-Diagnosed-By: Tom Lane <[email protected]>
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/procarray.c
Make vacuum a bit more verbose to debug BF failure.
commit : 49967da65aec970fcda123acc681f1df5d70bfc6
author : Andres Freund <[email protected]>
date : Sun, 16 Aug 2020 12:57:01 -0700
committer: Andres Freund <[email protected]>
date : Sun, 16 Aug 2020 12:57:01 -0700
This is temporary. While possibly some more error checking / debugging
in this path would be a good thing, it'll not look exactly like this.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam.c
M src/backend/access/heap/vacuumlazy.c
Correct several behavior descriptions in comments.
commit : 676a9c3cc4b5f1d262c29de318868948513f0fa0
author : Noah Misch <[email protected]>
date : Sat, 15 Aug 2020 20:21:52 -0700
committer: Noah Misch <[email protected]>
date : Sat, 15 Aug 2020 20:21:52 -0700
Reuse cautionary language from src/test/ssl/README in
src/test/kerberos/README. SLRUs have had access to six-character
segments names since commit 73c986adde5d73a5e2555da9b5c8facedb146dcd,
and recovery stopped calling HeapTupleHeaderAdvanceLatestRemovedXid() in
commit 558a9165e081d1936573e5a7d576f5febd7fb55a. The other corrections
are more self-evident.
M src/backend/access/heap/heapam.c
M src/backend/access/transam/README
M src/backend/access/transam/varsup.c
M src/backend/commands/async.c
M src/backend/commands/vacuum.c
M src/backend/storage/buffer/bufmgr.c
M src/bin/pg_waldump/pg_waldump.c
M src/include/access/xlog_internal.h
M src/test/kerberos/README
M src/test/perl/PostgresNode.pm
M src/test/recovery/t/010_logical_decoding_timelines.pl
M src/test/ssl/t/SSLServer.pm
Doc: various improvements for pg_basebackup reference page.
commit : db659a3416b967d716806e558efbb9d1ec610cd1
author : Tom Lane <[email protected]>
date : Sat, 15 Aug 2020 15:43:34 -0400
committer: Tom Lane <[email protected]>
date : Sat, 15 Aug 2020 15:43:34 -0400
Put the -r option in the right section (it certainly isn't an
option controlling "the location and format of the output").
Clarify the behavior of the tablespace and waldir options
(that part per gripe from [email protected]).
Make a large number of small copy-editing fixes in text that
visibly wasn't written by native speakers, and try to avoid
grammatical inconsistencies between the descriptions of
the different options.
Back-patch to v13, since HEAD hasn't meaningfully diverged yet.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/pg_basebackup.sgml
Prevent concurrent SimpleLruTruncate() for any given SLRU.
commit : 566372b3d6435639e4cc4476d79b8505a0297c87
author : Noah Misch <[email protected]>
date : Sat, 15 Aug 2020 10:15:53 -0700
committer: Noah Misch <[email protected]>
date : Sat, 15 Aug 2020 10:15:53 -0700
The SimpleLruTruncate() header comment states the new coding rule. To
achieve this, add locktype "frozenid" and two LWLocks. This closes a
rare opportunity for data loss, which manifested as "apparent
wraparound" or "could not access status of transaction" errors. Data
loss is more likely in pg_multixact, due to released branches' thin
margin between multiStopLimit and multiWrapLimit. If a user's physical
replication primary logged ": apparent wraparound" messages, the user
should rebuild standbys of that primary regardless of symptoms. At less
risk is a cluster having emitted "not accepting commands" errors or
"must be vacuumed" warnings at some point. One can test a cluster for
this data loss by running VACUUM FREEZE in every database. Back-patch
to 9.5 (all supported versions).
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/monitoring.sgml
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/commands/async.c
M src/backend/commands/vacuum.c
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lwlocknames.txt
M src/backend/utils/adt/lockfuncs.c
M src/include/storage/lmgr.h
M src/include/storage/lock.h
Remove no-longer-usable hstore--1.0--1.1.sql update script.
commit : d4d443b3bbbb3eb9cdc511564ef3c57fde7dd3ac
author : Tom Lane <[email protected]>
date : Sat, 15 Aug 2020 12:04:19 -0400
committer: Tom Lane <[email protected]>
date : Sat, 15 Aug 2020 12:04:19 -0400
Since commit 865f14a2d made "=>" unusable as an operator name,
it's been impossible either to install hstore 1.0 or to execute
this update script. There's not much point in continuing
to ship it.
Discussion: https://postgr.es/m/[email protected]
M contrib/hstore/Makefile
D contrib/hstore/hstore–1.0–1.1.sql
Remove obsolete cygwin.h hack
commit : 53095b5fe650270118bc2ab77416d08e19472cd3
author : Peter Eisentraut <[email protected]>
date : Sat, 15 Aug 2020 11:23:18 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 15 Aug 2020 11:23:18 +0200
The version being checked for is 20 years old.
Reviewed-by: Marco Atzeri <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/aa266ede-baaa-f4e6-06cf-5b1737610e9a%402ndquadrant.com
M src/include/port/cygwin.h
Remove obsolete HAVE_BUGGY_SOLARIS_STRTOD
commit : bacda6a327efb820d0e9f3262b81e803b2d5702b
author : Peter Eisentraut <[email protected]>
date : Sat, 15 Aug 2020 11:23:18 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 15 Aug 2020 11:23:18 +0200
Fixed more than 10 years ago.
Reviewed-by: Noah Misch <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/aa266ede-baaa-f4e6-06cf-5b1737610e9a%402ndquadrant.com
M src/backend/utils/adt/float.c
M src/include/port/solaris.h
Mark a few logical decoding related variables with PGDLLIMPORT.
commit : b48cac3b10a02fea2bed684469dd4d36a6616405
author : Amit Kapila <[email protected]>
date : Sat, 15 Aug 2020 08:34:48 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 15 Aug 2020 08:34:48 +0530
Commit 7259736a6e added two variables CheckXidAlive and bsysscan to detect
concurrent aborts and used these in inline functions that are part of the
API that can be used by extensions. So it is better to mark them with
PGDLLIMPORT.
Reported-by: Thomas Munro
Discussion: https://postgr.es/m/CA+hUKGJ7+HYupd=Pz9+QrXa-C_YnbC4rAYu8V+=OKg=UgdzMeg@mail.gmail.com
M src/include/access/xact.h
Be more careful about the shape of hashable subplan clauses.
commit : 1e7629d2c95ffd290ab0e18d7618ca9d9da94265
author : Tom Lane <[email protected]>
date : Fri, 14 Aug 2020 22:14:03 -0400
committer: Tom Lane <[email protected]>
date : Fri, 14 Aug 2020 22:14:03 -0400
nodeSubplan.c expects that the testexpr for a hashable ANY SubPlan
has the form of one or more OpExprs whose LHS is an expression of the
outer query's, while the RHS is an expression over Params representing
output columns of the subquery. However, the planner only went as far
as verifying that the clauses were all binary OpExprs. This works
99.99% of the time, because the clauses have the right shape when
emitted by the parser --- but it's possible for function inlining to
break that, as reported by PegoraroF10. To fix, teach the planner
to check that the LHS and RHS contain the right things, or more
accurately don't contain the wrong things. Given that this has been
broken for years without anyone noticing, it seems sufficient to just
give up hashing when it happens, rather than go to the trouble of
commuting the clauses back again (which wouldn't necessarily work
anyway).
While poking at that, I also noticed that nodeSubplan.c had a baked-in
assumption that the number of hash clauses is identical to the number
of subquery output columns. Again, that's fine as far as parser output
goes, but it's not hard to break it via function inlining. There seems
little reason for that assumption though --- AFAICS, the only thing
it's buying us is not having to store the number of hash clauses
explicitly. Adding code to the planner to reject such cases would take
more code than getting nodeSubplan.c to cope, so I fixed it that way.
This has been broken for as long as we've had hashable SubPlans,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/nodeSubplan.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/clauses.c
M src/include/nodes/execnodes.h
M src/include/optimizer/clauses.h
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
snapshot scalability: Move subxact info to ProcGlobal, remove PGXACT.
commit : 73487a60fc1063ba4b5178b69aee4ee210c182c4
author : Andres Freund <[email protected]>
date : Fri, 14 Aug 2020 14:30:38 -0700
committer: Andres Freund <[email protected]>
date : Fri, 14 Aug 2020 14:30:38 -0700
Similar to the previous changes this increases the chance that data
frequently needed by GetSnapshotData() stays in l2 cache. In many
workloads subtransactions are very rare, and this makes the check for
that considerably cheaper.
As this removes the last member of PGXACT, there is no need to keep it
around anymore.
On a larger 2 socket machine this and the two preceding commits result
in a ~1.07x performance increase in read-only pgbench. For read-heavy
mixed r/w workloads without row level contention, I see about 1.1x.
Author: Andres Freund <[email protected]>
Reviewed-By: Robert Haas <[email protected]>
Reviewed-By: Thomas Munro <[email protected]>
Reviewed-By: David Rowley <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/clog.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/varsup.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/lmgr/proc.c
M src/include/storage/proc.h
M src/tools/pgindent/typedefs.list
snapshot scalability: Move PGXACT->vacuumFlags to ProcGlobal->vacuumFlags.
commit : 5788e258bb26495fab65ff3aa486268d1c50b123
author : Andres Freund <[email protected]>
date : Wed, 15 Jul 2020 15:35:07 -0700
committer: Andres Freund <[email protected]>
date : Wed, 15 Jul 2020 15:35:07 -0700
Similar to the previous commit this increases the chance that data
frequently needed by GetSnapshotData() stays in l2 cache. As we now
take care to not unnecessarily write to ProcGlobal->vacuumFlags, there
should be very few modifications to the ProcGlobal->vacuumFlags array.
Author: Andres Freund <[email protected]>
Reviewed-By: Robert Haas <[email protected]>
Reviewed-By: Thomas Munro <[email protected]>
Reviewed-By: David Rowley <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/twophase.c
M src/backend/commands/vacuum.c
M src/backend/postmaster/autovacuum.c
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/lmgr/deadlock.c
M src/backend/storage/lmgr/proc.c
M src/include/storage/proc.h
snapshot scalability: Introduce dense array of in-progress xids.
commit : 941697c3c1ae5d6ee153065adb96e1e63ee11224
author : Andres Freund <[email protected]>
date : Fri, 14 Aug 2020 12:15:38 -0700
committer: Andres Freund <[email protected]>
date : Fri, 14 Aug 2020 12:15:38 -0700
The new array contains the xids for all connected backends / in-use
PGPROC entries in a dense manner (in contrast to the PGPROC/PGXACT
arrays which can have unused entries interspersed).
This improves performance because GetSnapshotData() always needs to
scan the xids of all live procarray entries and now there's no need to
go through the procArray->pgprocnos indirection anymore.
As the set of running top-level xids changes rarely, compared to the
number of snapshots taken, this substantially increases the likelihood
of most data required for a snapshot being in l2 cache. In
read-mostly workloads scanning the xids[] array will sufficient to
build a snapshot, as most backends will not have an xid assigned.
To keep the xid array dense ProcArrayRemove() needs to move entries
behind the to-be-removed proc's one further up in the array. Obviously
moving array entries cannot happen while a backend sets it
xid. I.e. locking needs to prevent that array entries are moved while
a backend modifies its xid.
To avoid locking ProcArrayLock in GetNewTransactionId() - a fairly hot
spot already - ProcArrayAdd() / ProcArrayRemove() now needs to hold
XidGenLock in addition to ProcArrayLock. Adding / Removing a procarray
entry is not a very frequent operation, even taking 2PC into account.
Due to the above, the dense array entries can only be read or modified
while holding ProcArrayLock and/or XidGenLock. This prevents a
concurrent ProcArrayRemove() from shifting the dense array while it is
accessed concurrently.
While the new dense array is very good when needing to look at all
xids it is less suitable when accessing a single backend's xid. In
particular it would be problematic to have to acquire a lock to access
a backend's own xid. Therefore a backend's xid is not just stored in
the dense array, but also in PGPROC. This also allows a backend to
only access the shared xid value when the backend had acquired an
xid.
The infrastructure added in this commit will be used for the remaining
PGXACT fields in subsequent commits. They are kept separate to make
review easier.
Author: Andres Freund <[email protected]>
Reviewed-By: Robert Haas <[email protected]>
Reviewed-By: Thomas Munro <[email protected]>
Reviewed-By: David Rowley <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam_visibility.c
M src/backend/access/transam/README
M src/backend/access/transam/clog.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/varsup.c
M src/backend/commands/vacuum.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/sinvaladt.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
M src/include/storage/proc.h
pg_dump: fix dependencies on FKs to partitioned tables
commit : 2ba5b2db7943742e100834d99548c5d2661a105b
author : Alvaro Herrera <[email protected]>
date : Fri, 14 Aug 2020 17:33:31 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 14 Aug 2020 17:33:31 -0400
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Backpatch to 12, where FKs to partitioned tables were introduced.
Reported-by: Tom Lane <[email protected]>
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch: 12-master
M src/bin/pg_dump/pg_dump.c
Fix obsolete comment in xlogutils.c.
commit : 914140e85a79c63853c86334afa2d7e6e930c11a
author : Peter Geoghegan <[email protected]>
date : Fri, 14 Aug 2020 11:09:08 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 14 Aug 2020 11:09:08 -0700
Oversight in commit 2c03216d831.
M src/backend/access/transam/xlogutils.c
Fix postmaster's behavior during smart shutdown.
commit : 0038f943878286ce84b2dfac10d64e00eab02edd
author : Tom Lane <[email protected]>
date : Fri, 14 Aug 2020 13:26:57 -0400
committer: Tom Lane <[email protected]>
date : Fri, 14 Aug 2020 13:26:57 -0400
Up to now, upon receipt of a SIGTERM ("smart shutdown" command), the
postmaster has immediately killed all "optional" background processes,
and subsequently refused to launch new ones while it's waiting for
foreground client processes to exit. No doubt this seemed like an OK
policy at some point; but it's a pretty bad one now, because it makes
for a seriously degraded environment for the remaining clients:
* Parallel queries are killed, and new ones fail to launch. (And our
parallel-query infrastructure utterly fails to deal with the case
in a reasonable way --- it just hangs waiting for workers that are
not going to arrive. There is more work needed in that area IMO.)
* Autovacuum ceases to function. We can tolerate that for awhile,
but if bulk-update queries continue to run in the surviving client
sessions, there's eventually going to be a mess. In the worst case
the system could reach a forced shutdown to prevent XID wraparound.
* The bgwriter and walwriter are also stopped immediately, likely
resulting in performance degradation.
Hence, let's rearrange things so that the only immediate change in
behavior is refusing to let in new normal connections. Once the last
normal connection is gone, shut everything down as though we'd received
a "fast" shutdown. To implement this, remove the PM_WAIT_BACKUP and
PM_WAIT_READONLY states, instead staying in PM_RUN or PM_HOT_STANDBY
while normal connections remain. A subsidiary state variable tracks
whether or not we're letting in new connections in those states.
This also allows having just one copy of the logic for killing child
processes in smart and fast shutdown modes. I moved that logic into
PostmasterStateMachine() by inventing a new state PM_STOP_BACKENDS.
Back-patch to 9.6 where parallel query was added. In principle
this'd be a good idea in 9.5 as well, but the risk/reward ratio
is not as good there, since lack of autovacuum is not a problem
during typical uses of smart shutdown.
Per report from Bharath Rupireddy.
Patch by me, reviewed by Thomas Munro
Discussion: https://postgr.es/m/CALj2ACXAZ5vKxT9P7P89D87i3MDO9bfS+_bjMHgnWJs8uwUOOw@mail.gmail.com
M doc/src/sgml/ref/pg_ctl-ref.sgml
M src/backend/postmaster/postmaster.c
M src/backend/utils/init/postinit.c
M src/include/libpq/libpq-be.h
Fix typo in test comment.
commit : 5bdf694568ef0b9eebef32002a9ebc1918dd0b4b
author : Heikki Linnakangas <[email protected]>
date : Fri, 14 Aug 2020 10:40:50 +0300
committer: Heikki Linnakangas <[email protected]>
date : Fri, 14 Aug 2020 10:40:50 +0300
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Fix compilation warnings with libselinux 3.1 in contrib/sepgsql/
commit : 1f32136a9960df2e135e7b36363ea1f087b514a0
author : Michael Paquier <[email protected]>
date : Fri, 14 Aug 2020 09:30:34 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 14 Aug 2020 09:30:34 +0900
Upstream SELinux has recently marked security_context_t as officially
deprecated, causing warnings with -Wdeprecated-declarations. This is
considered as legacy code for some time now by upstream as
security_context_t got removed from most of the code tree during the
development of 2.3 back in 2014.
This removes all the references to security_context_t in sepgsql/ to be
consistent with SELinux, fixing the warnings. Note that this does not
impact the minimum version of libselinux supported.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
M contrib/sepgsql/label.c
M contrib/sepgsql/selinux.c
M contrib/sepgsql/uavc.c
Doc: improve examples for json_populate_record() and related functions.
commit : a9306f10b95992ec7229cae3de507a9fa2f6aa3c
author : Tom Lane <[email protected]>
date : Thu, 13 Aug 2020 20:00:38 -0400
committer: Tom Lane <[email protected]>
date : Thu, 13 Aug 2020 20:00:38 -0400
Make these examples self-contained by providing declarations of the
user-defined row types they rely on. There wasn't room to do this
in the old doc format, but now there is, and I think it makes the
examples a good bit less confusing.
M doc/src/sgml/func.sgml
snapshot scalability: Move PGXACT->xmin back to PGPROC.
commit : 1f51c17c68d05c28d5b9294d8013cb9e7e653160
author : Andres Freund <[email protected]>
date : Thu, 13 Aug 2020 16:25:21 -0700
committer: Andres Freund <[email protected]>
date : Thu, 13 Aug 2020 16:25:21 -0700
Now that xmin isn't needed for GetSnapshotData() anymore, it leads to
unnecessary cacheline ping-pong to have it in PGXACT, as it is updated
considerably more frequently than the other PGXACT members.
After the changes in dc7420c2c92, this is a very straight-forward change.
For highly concurrent, snapshot acquisition heavy, workloads this change alone
can significantly increase scalability. E.g. plain pgbench on a smaller 2
socket machine gains 1.07x for read-only pgbench, 1.22x for read-only pgbench
when submitting queries in batches of 100, and 2.85x for batches of 100
'SELECT';. The latter numbers are obviously not to be expected in the
real-world, but micro-benchmark the snapshot computation
scalability (previously spending ~80% of the time in GetSnapshotData()).
Author: Andres Freund <[email protected]>
Reviewed-By: Robert Haas <[email protected]>
Reviewed-By: Thomas Munro <[email protected]>
Reviewed-By: David Rowley <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/gist/gistxlog.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/transam/README
M src/backend/access/transam/twophase.c
M src/backend/commands/indexcmds.c
M src/backend/replication/logical/snapbuild.c
M src/backend/replication/walsender.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/sinvaladt.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/time/snapmgr.c
M src/include/storage/proc.h
Handle new HOT chains in index-build table scans
commit : a811ea5bde2fbf450095994b5726dcbf64d68668
author : Alvaro Herrera <[email protected]>
date : Thu, 13 Aug 2020 17:33:49 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 13 Aug 2020 17:33:49 -0400
When a table is scanned by heapam_index_build_range_scan (née
IndexBuildHeapScan) and the table lock being held allows concurrent data
changes, it is possible for new HOT chains to sprout in a page that were
unknown when the scan of a page happened. This leads to an error such
as
ERROR: failed to find parent tuple for heap-only tuple at (X,Y) in table "tbl"
because the root tuple was not present when we first obtained the list
of the page's root tuples. This can be fixed by re-obtaining the list
of root tuples, if we see that a heap-only tuple appears to point to a
non-existing root.
This was reported by Anastasia as occurring for BRIN summarization
(which exists since 9.5), but I think it could theoretically also happen
with CREATE INDEX CONCURRENTLY (much older) or REINDEX CONCURRENTLY
(very recent). It seems a happy coincidence that BRIN forces us to
backpatch this all the way to 9.5.
Reported-by: Anastasia Lubennikova <[email protected]>
Diagnosed-by: Anastasia Lubennikova <[email protected]>
Co-authored-by: Anastasia Lubennikova <[email protected]>
Co-authored-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch: 9.5 - master
M src/backend/access/heap/heapam_handler.c
M src/backend/access/heap/pruneheap.c
Fix out-of-date version reference, grammar.
commit : b8443eae72b5c36e6b443a2f09b9c605c61a589d
author : Andres Freund <[email protected]>
date : Wed, 12 Aug 2020 17:04:51 -0700
committer: Andres Freund <[email protected]>
date : Wed, 12 Aug 2020 17:04:51 -0700
Time appears to be passing fast.
Reported-By: Peter Geoghegan <[email protected]>
M src/backend/access/nbtree/README
M src/backend/access/transam/README
snapshot scalability: Don't compute global horizons while building snapshots.
commit : dc7420c2c9274a283779ec19718d2d16323640c0
author : Andres Freund <[email protected]>
date : Wed, 12 Aug 2020 16:03:49 -0700
committer: Andres Freund <[email protected]>
date : Wed, 12 Aug 2020 16:03:49 -0700
To make GetSnapshotData() more scalable, it cannot not look at at each proc's
xmin: While snapshot contents do not need to change whenever a read-only
transaction commits or a snapshot is released, a proc's xmin is modified in
those cases. The frequency of xmin modifications leads to, particularly on
higher core count systems, many cache misses inside GetSnapshotData(), despite
the data underlying a snapshot not changing. That is the most
significant source of GetSnapshotData() scaling poorly on larger systems.
Without accessing xmins, GetSnapshotData() cannot calculate accurate horizons /
thresholds as it has so far. But we don't really have to: The horizons don't
actually change that much between GetSnapshotData() calls. Nor are the horizons
actually used every time a snapshot is built.
The trick this commit introduces is to delay computation of accurate horizons
until there use and using horizon boundaries to determine whether accurate
horizons need to be computed.
The use of RecentGlobal[Data]Xmin to decide whether a row version could be
removed has been replaces with new GlobalVisTest* functions. These use two
thresholds to determine whether a row can be pruned:
1) definitely_needed, indicating that rows deleted by XIDs >= definitely_needed
are definitely still visible.
2) maybe_needed, indicating that rows deleted by XIDs < maybe_needed can
definitely be removed
GetSnapshotData() updates definitely_needed to be the xmin of the computed
snapshot.
When testing whether a row can be removed (with GlobalVisTestIsRemovableXid())
and the tested XID falls in between the two (i.e. XID >= maybe_needed && XID <
definitely_needed) the boundaries can be recomputed to be more accurate. As it
is not cheap to compute accurate boundaries, we limit the number of times that
happens in short succession. As the boundaries used by
GlobalVisTestIsRemovableXid() are never reset (with maybe_needed updated by
GetSnapshotData()), it is likely that further test can benefit from an earlier
computation of accurate horizons.
To avoid regressing performance when old_snapshot_threshold is set (as that
requires an accurate horizon to be computed), heap_page_prune_opt() doesn't
unconditionally call TransactionIdLimitedForOldSnapshots() anymore. Both the
computation of the limited horizon, and the triggering of errors (with
SetOldSnapshotThresholdTimestamp()) is now only done when necessary to remove
tuples.
This commit just removes the accesses to PGXACT->xmin from
GetSnapshotData(), but other members of PGXACT residing in the same
cache line are accessed. Therefore this in itself does not result in a
significant improvement. Subsequent commits will take advantage of the
fact that GetSnapshotData() now does not need to access xmins anymore.
Note: This contains a workaround in heap_page_prune_opt() to keep the
snapshot_too_old tests working. While that workaround is ugly, the tests
currently are not meaningful, and it seems best to address them separately.
Author: Andres Freund <[email protected]>
Reviewed-By: Robert Haas <[email protected]>
Reviewed-By: Thomas Munro <[email protected]>
Reviewed-By: David Rowley <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M contrib/amcheck/verify_nbtree.c
M contrib/pg_visibility/pg_visibility.c
M contrib/pgstattuple/pgstatapprox.c
M src/backend/access/gin/ginvacuum.c
M src/backend/access/gist/gistutil.c
M src/backend/access/gist/gistxlog.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/heap/heapam_visibility.c
M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/index/indexam.c
M src/backend/access/nbtree/README
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/spgist/spgvacuum.c
M src/backend/access/transam/README
M src/backend/access/transam/xlog.c
M src/backend/commands/analyze.c
M src/backend/commands/vacuum.c
M src/backend/postmaster/autovacuum.c
M src/backend/replication/logical/launcher.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walsender.c
M src/backend/storage/ipc/procarray.c
M src/backend/utils/adt/selfuncs.c
M src/backend/utils/init/postinit.c
M src/backend/utils/time/snapmgr.c
M src/include/access/ginblock.h
M src/include/access/heapam.h
M src/include/access/transam.h
M src/include/storage/bufpage.h
M src/include/storage/proc.h
M src/include/storage/procarray.h
M src/include/utils/snapmgr.h
M src/include/utils/snapshot.h
M src/tools/pgindent/typedefs.list
BRIN: Handle concurrent desummarization properly
commit : 1f42d35a1d6144a23602b2c0bc7f97f3046cf890
author : Alvaro Herrera <[email protected]>
date : Wed, 12 Aug 2020 15:33:36 -0400
committer: Alvaro Herrera <[email protected]>
date : Wed, 12 Aug 2020 15:33:36 -0400
If a page range is desummarized at just the right time concurrently with
an index walk, BRIN would raise an error indicating index corruption.
This is scary and unhelpful; silently returning that the page range is
not summarized is sufficient reaction.
This bug was introduced by commit 975ad4e602ff as additional protection
against a bug whose actual fix was elsewhere. Backpatch equally.
Reported-By: Anastasia Lubennikova <[email protected]>
Diagnosed-By: Alexander Lakhin <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch: 9.5 - master
M src/backend/access/brin/brin_revmap.c
Improve comments for postmaster.c's BackendList.
commit : 3546cf8a7a9dc57e6aa98f5fc1ac5476ad6b99ff
author : Tom Lane <[email protected]>
date : Wed, 12 Aug 2020 11:54:16 -0400
committer: Tom Lane <[email protected]>
date : Wed, 12 Aug 2020 11:54:16 -0400
This had gotten a little disjointed over time, and some of the grammar
was sloppy. Rewrite for more clarity.
In passing, re-pgindent some recently added comments.
No code changes.
M src/backend/postmaster/postmaster.c
Track latest completed xid as a FullTransactionId.
commit : 3bd7f9969a240827bc2effa399170b7565238fd2
author : Andres Freund <[email protected]>
date : Tue, 11 Aug 2020 17:41:18 -0700
committer: Andres Freund <[email protected]>
date : Tue, 11 Aug 2020 17:41:18 -0700
The reason for doing so is that a subsequent commit will need that to
avoid wraparound issues. As the subsequent change is large this was
split out for easier review.
The reason this is not a perfect straight-forward change is that we do
not want track 64bit xids in the procarray or the WAL. Therefore we
need to advance lastestCompletedXid in relation to 32 bit xids. The
code for that is now centralized in MaintainLatestCompletedXid*.
Author: Andres Freund
Reviewed-By: Thomas Munro, Robert Haas, David Rowley
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/varsup.c
M src/backend/access/transam/xlog.c
M src/backend/storage/ipc/procarray.c
M src/include/access/transam.h
Rename VariableCacheData.nextFullXid to nextXid.
commit : fea10a64340e529805609126740a540c8f9daab4
author : Andres Freund <[email protected]>
date : Tue, 11 Aug 2020 11:25:23 -0700
committer: Andres Freund <[email protected]>
date : Tue, 11 Aug 2020 11:25:23 -0700
Including Full in variable names duplicates the type information and
leads to overly long names. As FullTransactionId cannot accidentally
be casted to TransactionId that does not seem necessary.
Author: Andres Freund
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/gist/gistxlog.c
M src/backend/access/rmgrdesc/xlogdesc.c
M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/subtrans.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/varsup.c
M src/backend/access/transam/xact.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/predicate.c
M src/backend/utils/misc/pg_controldata.c
M src/bin/pg_controldata/pg_controldata.c
M src/bin/pg_resetwal/pg_resetwal.c
M src/include/access/transam.h
M src/include/catalog/pg_control.h
M src/include/storage/standby.h
M src/include/storage/standbydefs.h
Improve tab completion of REINDEX in psql
commit : 1f75b454134cce6a67a9bcdb01b5c018221dd359
author : Michael Paquier <[email protected]>
date : Tue, 11 Aug 2020 14:37:38 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 11 Aug 2020 14:37:38 +0900
This allows the tab completion of REINDEX to handle an optional
parenthesized list of options. This case is more complicated than
VACUUM or ANALYZE because of CONCURRENTLY and the different object types
to consider with the reindex.
Author: Justin Pryzby
Reviewed-by: Alexey Kondratov, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/tab-complete.c
Replace remaining StrNCpy() by strlcpy()
commit : 1784f278a63866cc144fcd0a2127cadba6a2b7f8
author : Peter Eisentraut <[email protected]>
date : Mon, 10 Aug 2020 18:51:31 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 10 Aug 2020 18:51:31 +0200
They are equivalent, except that StrNCpy() zero-fills the entire
destination buffer instead of providing just one trailing zero. For
all but a tiny number of callers, that's just overhead rather than
being desirable.
Remove StrNCpy() as it is now unused.
In some cases, namestrcpy() is the more appropriate function to use.
While we're here, simplify the API of namestrcpy(): Remove the return
value, don't check for NULL input. Nothing was using that anyway.
Also, remove a few unused name-related functions.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/44f5e198-36f6-6cdb-7fa9-60e34784daae%402ndquadrant.com
M contrib/pgcrypto/crypt-des.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/xlogarchive.c
M src/backend/catalog/pg_constraint.c
M src/backend/commands/indexcmds.c
M src/backend/commands/statscmds.c
M src/backend/commands/tablecmds.c
M src/backend/postmaster/pgstat.c
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/name.c
M src/backend/utils/adt/pg_locale.c
M src/backend/utils/adt/ruleutils.c
M src/common/exec.c
M src/include/c.h
M src/include/utils/builtins.h
M src/interfaces/ecpg/pgtypeslib/dt_common.c
M src/interfaces/ecpg/test/pg_regress_ecpg.c
M src/test/modules/ssl_passphrase_callback/ssl_passphrase_func.c
Document clashes between logical replication and untrusted users.
commit : cec57b1a0fbcd3833086ba686897c5883e0a2afc
author : Noah Misch <[email protected]>
date : Mon, 10 Aug 2020 09:22:54 -0700
committer: Noah Misch <[email protected]>
date : Mon, 10 Aug 2020 09:22:54 -0700
Back-patch to v10, which introduced logical replication.
Security: CVE-2020-14349
M doc/src/sgml/logical-replication.sgml
Empty search_path in logical replication apply worker and walsender.
commit : 11da97024abbe76b8c81e3f2375b2a62e9717c67
author : Noah Misch <[email protected]>
date : Mon, 10 Aug 2020 09:22:54 -0700
committer: Noah Misch <[email protected]>
date : Mon, 10 Aug 2020 09:22:54 -0700
This is like CVE-2018-1058 commit
582edc369cdbd348d68441fc50fa26a84afd0c1a. Today, a malicious user of a
publisher or subscriber database can invoke arbitrary SQL functions
under an identity running replication, often a superuser. This fix may
cause "does not exist" or "no schema has been selected to create in"
errors in a replication process. After upgrading, consider watching
server logs for these errors. Objects accruing schema qualification in
the wake of the earlier commit are unlikely to need further correction.
Back-patch to v10, which introduced logical replication.
Security: CVE-2020-14349
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/worker.c
M src/test/subscription/t/001_rep_changes.pl
Move connect.h from fe_utils to src/include/common.
commit : e078fb5d4eeb23d0d09932e0b183a8e7bdfb17b4
author : Noah Misch <[email protected]>
date : Mon, 10 Aug 2020 09:22:54 -0700
committer: Noah Misch <[email protected]>
date : Mon, 10 Aug 2020 09:22:54 -0700
Any libpq client can use the header. Clients include backend components
postgres_fdw, dblink, and logical replication apply worker. Back-patch
to v10, because another fix needs this. In released branches, just copy
the header and keep the original.
M contrib/oid2name/oid2name.c
M contrib/vacuumlo/vacuumlo.c
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_rewind/libpq_fetch.c
M src/bin/pg_upgrade/server.c
M src/bin/scripts/common.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/vacuumdb.c
M src/fe_utils/cancel.c
R096 src/include/fe_utils/connect.h src/include/common/connect.h
M src/tools/findoidjoins/findoidjoins.c
Make contrib modules' installation scripts more secure.
commit : 7eeb1d9861b0a3f453f8b31c7648396cdd7f1e59
author : Tom Lane <[email protected]>
date : Mon, 10 Aug 2020 10:44:42 -0400
committer: Tom Lane <[email protected]>
date : Mon, 10 Aug 2020 10:44:42 -0400
Hostile objects located within the installation-time search_path could
capture references in an extension's installation or upgrade script.
If the extension is being installed with superuser privileges, this
opens the door to privilege escalation. While such hazards have existed
all along, their urgency increases with the v13 "trusted extensions"
feature, because that lets a non-superuser control the installation path
for a superuser-privileged script. Therefore, make a number of changes
to make such situations more secure:
* Tweak the construction of the installation-time search_path to ensure
that references to objects in pg_catalog can't be subverted; and
explicitly add pg_temp to the end of the path to prevent attacks using
temporary objects.
* Disable check_function_bodies within installation/upgrade scripts,
so that any security gaps in SQL-language or PL-language function bodies
cannot create a risk of unwanted installation-time code execution.
* Adjust lookup of type input/receive functions and join estimator
functions to complain if there are multiple candidate functions. This
prevents capture of references to functions whose signature is not the
first one checked; and it's arguably more user-friendly anyway.
* Modify various contrib upgrade scripts to ensure that catalog
modification queries are executed with secure search paths. (These
are in-place modifications with no extension version changes, since
it is the update process itself that is at issue, not the end result.)
Extensions that depend on other extensions cannot be made fully secure
by these methods alone; therefore, revert the "trusted" marking that
commit eb67623c9 applied to earthdistance and hstore_plperl, pending
some better solution to that set of issues.
Also add documentation around these issues, to help extension authors
write secure installation scripts.
Patch by me, following an observation by Andres Freund; thanks
to Noah Misch for review.
Security: CVE-2020-14350
M contrib/btree_gist/btree_gist–1.1–1.2.sql
M contrib/citext/citext–1.1–1.2.sql
M contrib/citext/citext–1.2–1.3.sql
M contrib/cube/cube–1.1–1.2.sql
M contrib/cube/cube–1.3–1.4.sql
M contrib/earthdistance/earthdistance–1.1.sql
M contrib/earthdistance/earthdistance.control
M contrib/hstore/hstore–1.1–1.2.sql
M contrib/hstore/hstore–1.3–1.4.sql
M contrib/hstore_plperl/hstore_plperl.control
M contrib/intagg/intagg–1.0–1.1.sql
M contrib/intarray/intarray–1.1–1.2.sql
M contrib/ltree/ltree–1.0–1.1.sql
M contrib/pg_trgm/pg_trgm–1.2–1.3.sql
M contrib/seg/seg–1.0–1.1.sql
M contrib/seg/seg–1.2–1.3.sql
M doc/src/sgml/earthdistance.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/hstore.sgml
M doc/src/sgml/ltree.sgml
M doc/src/sgml/ref/create_extension.sgml
M src/backend/commands/extension.c
M src/backend/commands/operatorcmds.c
M src/backend/commands/typecmds.c
Correct nbtree page split lock coupling comment.
commit : d129c07499dbf0d5960115173515e3ce384c662a
author : Peter Geoghegan <[email protected]>
date : Sun, 9 Aug 2020 12:01:15 -0700
committer: Peter Geoghegan <[email protected]>
date : Sun, 9 Aug 2020 12:01:15 -0700
There is no reason to distinguish between readers and writers here.
M src/backend/access/nbtree/nbtinsert.c
Check for fseeko() failure in pg_dump's _tarAddFile().
commit : 1b9cde51246c7773eac119b84cc18095118735de
author : Tom Lane <[email protected]>
date : Sun, 9 Aug 2020 12:39:07 -0400
committer: Tom Lane <[email protected]>
date : Sun, 9 Aug 2020 12:39:07 -0400
Coverity pointed out, not unreasonably, that we checked fseeko's
result at every other call site but these. Failure to seek in the
temp file (note this is NOT pg_dump's output file) seems quite
unlikely, and even if it did happen the file length cross-check
further down would probably detect the problem. Still, that's a
poor excuse for not checking the result of a system call.
M src/bin/pg_dump/pg_backup_tar.c
Remove useless Assert.
commit : 1c164ef3d28dfab445a885a03e80cfd0d552f64a
author : Tom Lane <[email protected]>
date : Sun, 9 Aug 2020 11:32:31 -0400
committer: Tom Lane <[email protected]>
date : Sun, 9 Aug 2020 11:32:31 -0400
Testing that an unsigned variable is >= 0 is pretty pointless,
as noted by Coverity and numerous buildfarm members.
In passing, add comment about new uses of "volatile" --- Coverity
doesn't much like that either, but it seems probably necessary.
M src/backend/replication/logical/reorderbuffer.c
Remove <@ from contrib/intarray's GiST operator classes.
commit : 20e7e1fe316467720d8d062e1a1429f798fc31bf
author : Tom Lane <[email protected]>
date : Sat, 8 Aug 2020 17:26:29 -0400
committer: Tom Lane <[email protected]>
date : Sat, 8 Aug 2020 17:26:29 -0400
Since commit efc77cf5f, an indexed query using <@ has required a
full-index scan, so that it actually performs worse than a plain seqscan
would do. As I noted at the time, we'd be better off to not treat <@ as
being indexable by such indexes at all; and that's what this patch does.
It would have been difficult to remove these opclass members without
dropping the whole opclass before commit 9f9682783 fixed GiST opclass
member dependency rules, but now it's quite simple, so let's do it.
I left the existing support code in place for the time being, with
comments noting it's now unreachable. At some point, perhaps we should
remove that code in favor of throwing an error telling people to upgrade
the extension version.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M contrib/intarray/Makefile
M contrib/intarray/_int_gist.c
M contrib/intarray/_intbig_gist.c
A contrib/intarray/intarray–1.3–1.4.sql
M contrib/intarray/intarray.control
M doc/src/sgml/intarray.sgml
Teach amcheck to verify sibling links in all cases.
commit : 39132b784aeaaacf5ddfb5c35b6e29a6926f4345
author : Peter Geoghegan <[email protected]>
date : Sat, 8 Aug 2020 11:12:01 -0700
committer: Peter Geoghegan <[email protected]>
date : Sat, 8 Aug 2020 11:12:01 -0700
Teach contrib/amcheck's bt_index_check() function to check agreement
between siblings links. The left sibling's right link should point to a
right sibling page whose left link points back to the same original left
sibling. This extends a check that bt_index_parent_check() always
performed to bt_index_check().
This is the first time amcheck has been taught to perform buffer lock
coupling, which we have explicitly avoided up until now. The sibling
link check tends to catch a lot of real world index corruption with
little overhead, so it seems worth accepting the complexity. Note that
the new lock coupling logic would not work correctly on replica servers
without the changes made by commits 0a7d771f and 9a9db08a (there could
be false positives without those changes).
Author: Andrey Borodin, Peter Geoghegan
Discussion: https://postgr.es/m/[email protected]
M contrib/amcheck/verify_nbtree.c
walsnd: Don't set waiting_for_ping_response spuriously
commit : 470687b4a5bb3b9f2b5bf7c9235680b3c91bd050
author : Alvaro Herrera <[email protected]>
date : Sat, 8 Aug 2020 12:31:55 -0400
committer: Alvaro Herrera <[email protected]>
date : Sat, 8 Aug 2020 12:31:55 -0400
Ashutosh Bapat noticed that when logical walsender needs to wait for
WAL, and it realizes that it must send a keepalive message to
walreceiver to update the sent-LSN, which *does not* request a reply
from walreceiver, it wrongly sets the flag that it's going to wait for
that reply. That means that any future would-be sender of feedback
messages ends up not sending a feedback message, because they all
believe that a reply is expected.
With built-in logical replication there's not much harm in this, because
WalReceiverMain will send a ping-back every wal_receiver_timeout/2
anyway; but with other logical replication systems (e.g. pglogical) it
can cause significant pain.
This problem was introduced in commit 41d5f8ad734, where the
request-reply flag was changed from true to false to WalSndKeepalive,
without at the same time removing the line that sets
waiting_for_ping_response.
Just removing that line would be a sufficient fix, but it seems better
to shift the responsibility of setting the flag to WalSndKeepalive
itself instead of requiring caller to do it; this is clearly less
error-prone.
Author: Álvaro Herrera <[email protected]>
Reported-by: Ashutosh Bapat <[email protected]>
Backpatch: 9.5 and up
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/walsender.c
Fix the logical streaming test.
commit : 82a0ba7707e010a29f5fe1a0020d963c82b8f1cb
author : Amit Kapila <[email protected]>
date : Sat, 8 Aug 2020 12:13:18 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 8 Aug 2020 12:13:18 +0530
Commit 7259736a6e added the capability to stream changes in ReorderBuffer
which has some tests to test the streaming mode. It is quite possible that
while this test is running a parallel transaction could be logged by
autovacuum. Such a transaction won't perform any insert/update/delete to
non-catalog tables so will be shown as an empty transaction. Fix it by
skipping the empty transactions during this test.
Per report by buildfarm.
M contrib/test_decoding/expected/stream.out
M contrib/test_decoding/sql/stream.sql
Add some const decorations
commit : a13421c96c0e8ffa34310f92d03d0e6f3bfa27f8
author : Peter Eisentraut <[email protected]>
date : Sat, 8 Aug 2020 07:31:52 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 8 Aug 2020 07:31:52 +0200
M src/backend/replication/logical/logical.c
M src/include/replication/logical.h
Implement streaming mode in ReorderBuffer.
commit : 7259736a6e5b7c7588fff9578370736a6648acbb
author : Amit Kapila <[email protected]>
date : Sat, 8 Aug 2020 07:34:39 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 8 Aug 2020 07:34:39 +0530
Instead of serializing the transaction to disk after reaching the
logical_decoding_work_mem limit in memory, we consume the changes we have
in memory and invoke stream API methods added by commit 45fdc9738b.
However, sometimes if we have incomplete toast or speculative insert we
spill to the disk because we can't generate the complete tuple and stream.
And, as soon as we get the complete tuple we stream the transaction
including the serialized changes.
We can do this incremental processing thanks to having assignments
(associating subxact with toplevel xacts) in WAL right away, and
thanks to logging the invalidation messages at each command end. These
features are added by commits 0bead9af48 and c55040ccd0 respectively.
Now that we can stream in-progress transactions, the concurrent aborts
may cause failures when the output plugin consults catalogs (both system
and user-defined).
We handle such failures by returning ERRCODE_TRANSACTION_ROLLBACK
sqlerrcode from system table scan APIs to the backend or WALSender
decoding a specific uncommitted transaction. The decoding logic on the
receipt of such a sqlerrcode aborts the decoding of the current
transaction and continue with the decoding of other transactions.
We have ReorderBufferTXN pointer in each ReorderBufferChange by which we
know which xact it belongs to. The output plugin can use this to decide
which changes to discard in case of stream_abort_cb (e.g. when a subxact
gets discarded).
We also provide a new option via SQL APIs to fetch the changes being
streamed.
Author: Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke
Reviewed-by: Amit Kapila, Kuntal Ghosh, Ajin Cherian
Tested-by: Neha Sharma, Mahendra Singh Thalor and Ajin Cherian
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/Makefile
A contrib/test_decoding/expected/stream.out
M contrib/test_decoding/expected/truncate.out
A contrib/test_decoding/sql/stream.sql
M contrib/test_decoding/sql/truncate.sql
M contrib/test_decoding/test_decoding.c
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/test-decoding.sgml
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_visibility.c
M src/backend/access/index/genam.c
M src/backend/access/table/tableam.c
M src/backend/access/transam/xact.c
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/reorderbuffer.c
M src/include/access/heapam_xlog.h
M src/include/access/tableam.h
M src/include/access/xact.h
M src/include/replication/logical.h
M src/include/replication/reorderbuffer.h
Make nbtree split REDO locking match original execution.
commit : 0a7d771f0f63eb120e7f0a60aecd543ab25ba197
author : Peter Geoghegan <[email protected]>
date : Fri, 7 Aug 2020 15:27:56 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 7 Aug 2020 15:27:56 -0700
Make the nbtree page split REDO routine consistent with original
execution in its approach to acquiring and releasing buffer locks (at
least for pages on the tree level of the page being split). This brings
btree_xlog_split() in line with btree_xlog_unlink_page(), which was
taught to couple buffer locks by commit 9a9db08a.
Note that the precise order in which we both acquire and release sibling
buffer locks in btree_xlog_split() now matches original execution
exactly (the precise order in which the locks are released probably
doesn't matter much, but we might as well be consistent about it).
The rule for nbtree REDO routines from here on is that same-level locks
should be acquired in an order that's consistent with original
execution. It's not practical to have a similar rule for cross-level
page locks, since for the most part original execution holds those locks
for a period that spans multiple atomic actions/WAL records. It's also
not necessary, because clearly the cross-level lock coupling is only
truly needed during original execution because of the presence of
concurrent inserters.
This is not a bug fix (unlike the similar aforementioned commit, commit
9a9db08a). The immediate reason to tighten things up in this area is to
enable an upcoming enhancement to contrib/amcheck that allows it to
verify that sibling links are in agreement with only an AccessShareLock
(this check produced false positives when run on a replica server on
account of the inconsistency fixed by this commit). But that's not the
only reason to be stricter here.
It is generally useful to make locking on replicas be as close to what
happens during original execution as practically possible. It makes it
less likely that hard to catch bugs will slip in in the future. The
previous state of affairs seems to be a holdover from before the
introduction of Hot Standby, when buffer lock acquisitions during
recovery were totally unnecessary. See also: commit 3bbf668d, which
tightened things up in this area a few years after the introduction of
Hot Standby.
Discussion: https://postgr.es/m/CAH2-Wz=465cJj11YXD9RKH8z=nhQa2dofOZ_23h67EXUGOJ00Q@mail.gmail.com
M src/backend/access/nbtree/README
M src/backend/access/nbtree/nbtxlog.c
Remove PROC_IN_ANALYZE and derived flags
commit : cea3d55898655582e3a3835a7bed2c3a1b002fef
author : Alvaro Herrera <[email protected]>
date : Fri, 7 Aug 2020 17:24:40 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 7 Aug 2020 17:24:40 -0400
These flags are unused and always have been.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/analyze.c
M src/include/storage/proc.h
M src/include/storage/procarray.h
Support testing of cases where table schemas change after planning.
commit : 6f0b632f083ba08fabb6c496caf733802cee9d2e
author : Tom Lane <[email protected]>
date : Fri, 7 Aug 2020 14:30:41 -0400
committer: Tom Lane <[email protected]>
date : Fri, 7 Aug 2020 14:30:41 -0400
We have various cases where we allow DDL on tables to be performed with
less than full AccessExclusiveLock. This requires concurrent queries
to be able to cope with the DDL change mid-flight, but up to now we had
no repeatable way to test such cases. To improve that, invent a test
module that allows halting a backend after planning and then resuming
execution once we've done desired actions in another session. (The same
approach could be used to inject delays in other places, if there's a
suitable hook available.)
This commit includes a single test case, which is meant to exercise the
previously-untestable ExecCreatePartitionPruneState code repaired by
commit 7a980dfc6. We'd probably not bother with this if that were the
only foreseen benefit, but I expect additional test cases will use this
infrastructure in the future.
Test module by Andy Fan, partition-addition test case by me.
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/Makefile
A src/test/modules/delay_execution/.gitignore
A src/test/modules/delay_execution/Makefile
A src/test/modules/delay_execution/delay_execution.c
A src/test/modules/delay_execution/expected/partition-addition.out
A src/test/modules/delay_execution/specs/partition-addition.spec
Rename nbtree split REDO routine variables.
commit : 3df92bbd1dba98f72e3f005406463b0718193a0f
author : Peter Geoghegan <[email protected]>
date : Fri, 7 Aug 2020 09:53:27 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 7 Aug 2020 09:53:27 -0700
Make the nbtree page split REDO routine variable names consistent with
_bt_split() (which handles the original execution of page splits).
These names make the code easier to follow by making the distinction
between the original page and the left half of the split clear. (The
left half of the split page is a temp page that REDO creates to replace
the origpage contents.)
Also reduce the elevel used when adding a new high key to the temp page
from PANIC to ERROR to be consistent. We already only raise an ERROR
when data item PageAddItem() temp page calls fail.
M src/backend/access/nbtree/nbtxlog.c
Fix yet another issue with step generation in partition pruning.
commit : 199cec9779504c08aaa8159c6308283156547409
author : Etsuro Fujita <[email protected]>
date : Fri, 7 Aug 2020 14:45:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Fri, 7 Aug 2020 14:45:00 +0900
Commit 13838740f fixed some issues with step generation in partition
pruning, but there was yet another one: get_steps_using_prefix() assumes
that clauses in the passed-in prefix list are sorted in ascending order
of their partition key numbers, but the caller failed to ensure this for
range partitioning, which led to an assertion failure in debug builds.
Adjust the caller function to arrange the clauses in the prefix list in
the required order for range partitioning.
Back-patch to v11, like the previous commit.
Patch by me, reviewed by Amit Langote.
Discussion: https://postgr.es/m/CAPmGK16jkXiFG0YqMbU66wte-oJTfW6D1HaNvQf%3D%2B5o9%3Dm55wQ%40mail.gmail.com
M src/backend/partitioning/partprune.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Remove obsolete amcheck comment.
commit : 3a3be80641c01e675d0ed484f15df8ec536d0a06
author : Peter Geoghegan <[email protected]>
date : Thu, 6 Aug 2020 16:23:52 -0700
committer: Peter Geoghegan <[email protected]>
date : Thu, 6 Aug 2020 16:23:52 -0700
Oversight in commit d114cc53871.
M contrib/amcheck/verify_nbtree.c
amcheck: Sanitize metapage's allequalimage field.
commit : c254d8d7b20bf629420b407a5451c3b32d1a7b0b
author : Peter Geoghegan <[email protected]>
date : Thu, 6 Aug 2020 15:25:49 -0700
committer: Peter Geoghegan <[email protected]>
date : Thu, 6 Aug 2020 15:25:49 -0700
This will be helpful if it ever proves necessary to revoke an opclass's
support for deduplication.
Backpatch: 13-, where nbtree deduplication was introduced.
M contrib/amcheck/verify_nbtree.c
Fix bogus EXPLAIN output for Hash Aggregate
commit : d5e96520ffca8eeeefc11f8fc82af610f68e63a8
author : David Rowley <[email protected]>
date : Fri, 7 Aug 2020 10:22:18 +1200
committer: David Rowley <[email protected]>
date : Fri, 7 Aug 2020 10:22:18 +1200
9bdb300de modified the EXPLAIN output for Hash Aggregate to show details
from parallel workers. However, it neglected to consider that a given
parallel worker may not have assisted with the given Hash Aggregate. This
can occur when workers fail to start or during Parallel Append with
enable_partitionwise_join enabled when only a single worker is working on
a non-parallel aware sub-plan. It could also happen if a worker simply
wasn't fast enough to get any work done before other processes went and
finished all the work.
The bogus output came from the fact that ExplainOpenWorker() skipped
showing any details for non-initialized workers but show_hashagg_info()
did show details from the worker. This meant that the worker properties
that were shown were not properly attributed to the worker that they
belong to.
In passing, we also now don't show Hash Aggregate properties for the
leader process when it did not contribute any work to the Hash Aggregate.
This can occur either during Parallel Append when only a parallel worker
worked on a given sub plan or with parallel_leader_participation set to
off. This aims to make the behavior of Hash Aggregate's EXPLAIN output
more similar to Sort's.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20200805012105.GZ28072%40telsasoft.com
Backpatch-through: 13, where the original breakage was introduced
M src/backend/commands/explain.c
Register llvm_shutdown using on_proc_exit, not before_shmem_exit.
commit : bab150045bd9766869f471ede88734ea0989261c
author : Robert Haas <[email protected]>
date : Thu, 6 Aug 2020 14:13:03 -0400
committer: Robert Haas <[email protected]>
date : Thu, 6 Aug 2020 14:13:03 -0400
This seems more correct, because other before_shmem_exit calls may
expect the infrastructure that is needed to run queries and access the
database to be working, and also because this cleanup has nothing to
do with shared memory.
There are no known user-visible consequences to this, though, apart
from what was previous fixed by commit
303640199d0436c5e7acdf50b837a027b5726594 and back-patched as commit
bcbc27251d35336a6442761f59638138a772b839 and commit
f7013683d9bb663a6a917421b1374306a32f165b, so for now, no back-patch.
Bharath Rupireddy
Discussion: http://postgr.es/m/CALj2ACWk7j4F2v2fxxYfrroOF=AdFNPr1WsV+AGtHAFQOqm_pw@mail.gmail.com
M src/backend/jit/llvm/llvmjit.c
doc: clarify "state" table reference in tutorial
commit : a6775352476ac92d6b3eb3ae2dfd2775e3622afe
author : Bruce Momjian <[email protected]>
date : Wed, 5 Aug 2020 17:12:10 -0400
committer: Bruce Momjian <[email protected]>
date : Wed, 5 Aug 2020 17:12:10 -0400
Reported-by: Vyacheslav Shablistyy
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/advanced.sgml
Fix matching of sub-partitions when a partitioned plan is stale.
commit : 7a980dfc6c15add6ec3309932cf3061bb6745f65
author : Tom Lane <[email protected]>
date : Wed, 5 Aug 2020 15:38:55 -0400
committer: Tom Lane <[email protected]>
date : Wed, 5 Aug 2020 15:38:55 -0400
Since we no longer require AccessExclusiveLock to add a partition,
the executor may see that a partitioned table has more partitions
than the planner saw. ExecCreatePartitionPruneState's code for
matching up the partition lists in such cases was faulty, and would
misbehave if the planner had successfully pruned any partitions from
the query. (Thus, trouble would occur only if a partition addition
happens concurrently with a query that uses both static and dynamic
partition pruning.) This led to an Assert failure in debug builds,
and probably to crashes or query misbehavior in production builds.
To repair the bug, just explicitly skip zeroes in the plan's
relid_map[] list. I also made some cosmetic changes to make the code
more readable (IMO anyway). Also, convert the cross-checking Assert
to a regular test-and-elog, since it's now apparent that this logic
is more fragile than one would like.
Currently, there's no way to repeatably exercise this code, except
with manual use of a debugger to stop the backend between planning
and execution. Hence, no test case in this patch. We oughta do
something about that testability gap, but that's for another day.
Amit Langote and Tom Lane, per report from Justin Pryzby. Oversight
in commit 898e5e329; backpatch to v12 where that appeared.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execPartition.c
Remove btree page items after page unlink
commit : f47b5e139579a77c1f7c63400f01ea39d515e8c8
author : Alexander Korotkov <[email protected]>
date : Wed, 5 Aug 2020 02:15:34 +0300
committer: Alexander Korotkov <[email protected]>
date : Wed, 5 Aug 2020 02:15:34 +0300
Currently, page unlink leaves remaining items "as is", but replay of
corresponding WAL-record re-initializes page leaving it with no items.
For the sake of consistency, this commit makes primary delete all the items
during page unlink as well.
Thanks to this change, we now don't mask contents of deleted btree page for
WAL consistency checking.
Discussion: https://postgr.es/m/CAPpHfdt_OTyQpXaPJcWzV2N-LNeNJseNB-K_A66qG%3DL518VTFw%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Peter Geoghegan
M contrib/amcheck/verify_nbtree.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtxlog.c
Increase hard-wired timeout values in ecpg regression tests.
commit : 0f76294260b92849c4958fb706ecd5b5cd73e40e
author : Tom Lane <[email protected]>
date : Tue, 4 Aug 2020 15:20:31 -0400
committer: Tom Lane <[email protected]>
date : Tue, 4 Aug 2020 15:20:31 -0400
A couple of test cases had connect_timeout=14, a value that seems
to have been plucked from a hat. While it's more than sufficient
for normal cases, slow/overloaded buildfarm machines can get a timeout
failure here, as per recent report from "sungazer". Increase to 180
seconds, which is in line with our typical timeouts elsewhere in
the regression tests.
Back-patch to 9.6; the code looks different in 9.5, and this doesn't
seem to be quite worth the effort to adapt to that.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=sungazer&dt=2020-08-04%2007%3A12%3A22
M src/interfaces/ecpg/test/connect/test1.pgc
M src/interfaces/ecpg/test/connect/test5.pgc
M src/interfaces/ecpg/test/expected/connect-test1-minGW32.stderr
M src/interfaces/ecpg/test/expected/connect-test1.c
M src/interfaces/ecpg/test/expected/connect-test1.stderr
M src/interfaces/ecpg/test/expected/connect-test5.c
M src/interfaces/ecpg/test/expected/connect-test5.stderr
Make new SSL TAP test for channel_binding more robust
commit : dd877998d498c511352bd3640fd57f041c90ea62
author : Michael Paquier <[email protected]>
date : Tue, 4 Aug 2020 14:36:01 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 4 Aug 2020 14:36:01 +0900
The test would fail in an environment including a certificate file in
~/.postgresql/. bdd6e9b fixed a similar failure, and d6e612f introduced
the same problem again with a new test.
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/test/ssl/t/002_scram.pl
Fix replica backward scan race condition.
commit : 9a9db08ae46209edcc5ecb120328a2bf92fd6069
author : Peter Geoghegan <[email protected]>
date : Mon, 3 Aug 2020 15:54:38 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 3 Aug 2020 15:54:38 -0700
It was possible for the logic used by backward scans (which must reason
about concurrent page splits/deletions in its own peculiar way) to
become confused when running on a replica. Concurrent replay of a WAL
record that describes the second phase of page deletion could cause
_bt_walk_left() to get confused. btree_xlog_unlink_page() simply failed
to adhere to the same locking protocol that we use on the primary, which
is obviously wrong once you consider these two disparate functions
together. This bug is present in all stable branches.
More concretely, the problem was that nothing stopped _bt_walk_left()
from observing inconsistencies between the deletion's target page and
its original sibling pages when running on a replica. This is true even
though the second phase of page deletion is supposed to work as a single
atomic action. Queries running on replicas raised "could not find left
sibling of block %u in index %s" can't-happen errors when they went back
to their scan's "original" page and observed that the page has not been
marked deleted (even though it really was concurrently deleted).
There is no evidence that this actually happened in the real world. The
issue came to light during unrelated feature development work. Note
that _bt_walk_left() is the only code that cares about the difference
between a half-dead page and a fully deleted page that isn't also
exclusively used by nbtree VACUUM (unless you include contrib/amcheck
code). It seems very likely that backward scans are the only thing that
could become confused by the inconsistency. Even amcheck's complex
bt_right_page_check_scankey() dance was unaffected.
To fix, teach btree_xlog_unlink_page() to lock the left sibling, target,
and right sibling pages in that order before releasing any locks (just
like _bt_unlink_halfdead_page()). This is the simplest possible
approach. There doesn't seem to be any opportunity to be more clever
about lock acquisition in the REDO routine, and it hardly seems worth
the trouble in any case.
This fix might enable contrib/amcheck verification of leaf page sibling
links with only an AccessShareLock on the relation. An amcheck patch
from Andrey Borodin was rejected back in January because it clashed with
btree_xlog_unlink_page()'s lax approach to locking pages. It now seems
likely that the real problem was with btree_xlog_unlink_page(), not the
patch.
This is a low severity, low likelihood bug, so no backpatch.
Author: Michail Nikolaev
Diagnosed-By: Michail Nikolaev
Discussion: https://postgr.es/m/CANtu0ohkR-evAWbpzJu54V8eCOtqjJyYp3PQ_SGoBTRGXWhWRw@mail.gmail.com
M src/backend/access/nbtree/README
M src/backend/access/nbtree/nbtxlog.c
Add nbtree page deletion assertion.
commit : a451b7d44249b8655db8d40476ace9f84d76ab88
author : Peter Geoghegan <[email protected]>
date : Mon, 3 Aug 2020 13:04:42 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 3 Aug 2020 13:04:42 -0700
Add a documenting assertion that's similar to the nearby assertion added
by commit cd8c73a3. This conveys that the entire call to _bt_pagedel()
does no work if it isn't possible to get a descent stack for the initial
scanblkno page.
M src/backend/access/nbtree/nbtpage.c
Remove unnecessary "DISTINCT" in psql's queries for \dAc and \dAf.
commit : 9e496768b8a7303ea07888ea1baae8e2a57dda7b
author : Tom Lane <[email protected]>
date : Mon, 3 Aug 2020 14:02:35 -0400
committer: Tom Lane <[email protected]>
date : Mon, 3 Aug 2020 14:02:35 -0400
A moment's examination of these queries is sufficient to see that
they do not produce duplicate rows, unless perhaps there's
catalog corruption. Using DISTINCT anyway is inefficient and
confusing; moreover it sets a poor example for anyone who
refers to psql -E output to see how to query the catalogs.
M src/bin/psql/describe.c
Doc: fix obsolete info about allowed range of TZ offsets in timetz.
commit : eeb01e3122bb0acb6f8575d352e8e63101662ae7
author : Tom Lane <[email protected]>
date : Mon, 3 Aug 2020 13:11:16 -0400
committer: Tom Lane <[email protected]>
date : Mon, 3 Aug 2020 13:11:16 -0400
We've allowed UTC offsets up to +/- 15:59 since commit cd0ff9c0f, but
that commit forgot to fix the documentation about timetz.
Per bug #16571 from osdba.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/datatype.sgml
Fix behavior of ecpg's "EXEC SQL elif name".
commit : 5f28b21eb3c5c2fb72c24608bc686acd7c9b113c
author : Tom Lane <[email protected]>
date : Mon, 3 Aug 2020 09:46:12 -0400
committer: Tom Lane <[email protected]>
date : Mon, 3 Aug 2020 09:46:12 -0400
This ought to work much like C's "#elif defined(name)"; but the code
implemented it in a way equivalent to endif followed by ifdef, so that
it didn't matter whether any previous branch of the IF construct had
succeeded. Fix that; add some test cases covering elif and nested IFs;
and improve the documentation, which also seemed a bit confused.
AFAICS the code has been like this since the feature was added in 1999
(commit b57b0e044). So while it's surely wrong, there might be code
out there relying on the current behavior. Hence, don't back-patch
into stable branches. It seems all right to fix it in v13 though.
Per report from Ashutosh Sharma. Reviewed by Ashutosh Sharma and
Michael Meskes.
Discussion: https://postgr.es/m/CAE9k0P=dQk9X0cU2tN49S7a9tv733-e1pVdpB1P-pWJ5PdTktg@mail.gmail.com
M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/preproc/pgc.l
M src/interfaces/ecpg/test/expected/preproc-define.c
M src/interfaces/ecpg/test/expected/preproc-define.stderr
M src/interfaces/ecpg/test/preproc/define.pgc
Add %P to log_line_prefix for parallel group leader
commit : b8fdee7d0ca8bd2165d46fb1468f75571b706a01
author : Michael Paquier <[email protected]>
date : Mon, 3 Aug 2020 13:38:48 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 3 Aug 2020 13:38:48 +0900
This is useful for monitoring purposes with log parsing. Similarly to
pg_stat_activity, the leader's PID is shown only for active parallel
workers, minimizing the log footprint for the leaders as the equivalent
shared memory field is set as long as a backend is alive.
Author: Justin Pryzby
Reviewed-by: Álvaro Herrera, Michael Paquier, Julien Rouhaud, Tom Lane
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/config.sgml
M src/backend/utils/error/elog.c
M src/backend/utils/misc/postgresql.conf.sample
Fix rare failure in LDAP tests.
commit : f44b9b625bedd8e0bca67b3b42ba10ce482fa31b
author : Thomas Munro <[email protected]>
date : Mon, 3 Aug 2020 12:39:15 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 3 Aug 2020 12:39:15 +1200
Instead of writing a query to psql's stdin, use -c. This avoids a
failure where psql exits before we write, seen a few times on the build
farm. Thanks to Tom Lane for the suggestion.
Back-patch to 11, where the LDAP tests arrived.
Reviewed-by: Noah Misch <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGLFmW%2BHQYPeKiwSp5sdFFHtFViCpw4Mh6yAgEx74r5-Cw%40mail.gmail.com
M src/test/ldap/t/001_auth.pl
Correct comment in simplehash.h.
commit : 63e9aa6879cc5b87c77bab9afea3740748a4f00b
author : Thomas Munro <[email protected]>
date : Mon, 3 Aug 2020 12:17:41 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 3 Aug 2020 12:17:41 +1200
Post-commit review for commit 84c0e4b9.
Author: David Rowley <[email protected]>
Discussion: https://postgr.es/m/CAApHDvptBx_%2BUPAzY0uXzopbvPVGKPeZ6Hoy8rnPcWz20Cr0Bw%40mail.gmail.com
M src/include/lib/simplehash.h
Fix minor issues in psql's new \dAc and related commands.
commit : 533020d05045046a3481fdd92777de7bb2e30ab3
author : Tom Lane <[email protected]>
date : Sun, 2 Aug 2020 17:00:26 -0400
committer: Tom Lane <[email protected]>
date : Sun, 2 Aug 2020 17:00:26 -0400
The type-name pattern in \dAc and \dAf was matched only to the actual
pg_type.typname string, which is fairly user-unfriendly in cases where
that is not what's shown to the user by format_type (compare "_int4"
and "integer[]"). Make this code match what \dT does, i.e. match the
pattern against either typname or format_type() output. Also fix its
broken handling of schema-name restrictions. (IOW, make these
processSQLNamePattern calls match \dT's.) While here, adjust
whitespace to make the query a little prettier in -E output, too.
Also improve some inaccuracies and shaky grammar in the related
documentation.
Noted while working on a patch for intarray's opclasses; I wondered
why I couldn't get a match to "integer*" for the input type name.
M doc/src/sgml/indices.sgml
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/describe.c
Use int64 instead of long in incremental sort code
commit : 6ee3b5fb990ea11992b0db960d79b1fbe7b5e8e5
author : David Rowley <[email protected]>
date : Sun, 2 Aug 2020 14:24:46 +1200
committer: David Rowley <[email protected]>
date : Sun, 2 Aug 2020 14:24:46 +1200
Windows 64bit has 4-byte long values which is not suitable for tracking
disk space usage in the incremental sort code. Let's just make all these
fields int64s.
Author: James Coleman
Discussion: https://postgr.es/m/CAApHDvpky%2BUhof8mryPf5i%3D6e6fib2dxHqBrhp0Qhu0NeBhLJw%40mail.gmail.com
Backpatch-through: 13, where the incremental sort code was added
M src/backend/commands/explain.c
M src/include/nodes/execnodes.h
M src/include/utils/tuplesort.h
Change XID and mxact limits to warn at 40M and stop at 3M.
commit : cd5e82256de5895595cdd99ecb03aea15b346f71
author : Noah Misch <[email protected]>
date : Sat, 1 Aug 2020 15:31:01 -0700
committer: Noah Misch <[email protected]>
date : Sat, 1 Aug 2020 15:31:01 -0700
We have edge-case bugs when assigning values in the last few dozen pages
before the wrap limit. We may introduce similar bugs in the future. At
default BLCKSZ, this makes such bugs unreachable outside of single-user
mode. Also, when VACUUM began to consume mxacts, multiStopLimit did not
change to compensate.
pg_upgrade may fail on a cluster that was already printing "must be
vacuumed" warnings. Follow the warning's instructions to clear the
warning, then run pg_upgrade again. One can still, peacefully consume
98% of XIDs or mxacts, so DBAs need not change routine VACUUM settings.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/maintenance.sgml
M src/backend/access/transam/multixact.c
M src/backend/access/transam/varsup.c
Invent "amadjustmembers" AM method for validating opclass members.
commit : 9f9682783bea74bf8d93cac4f7dd65fa677f5dc7
author : Tom Lane <[email protected]>
date : Sat, 1 Aug 2020 17:12:47 -0400
committer: Tom Lane <[email protected]>
date : Sat, 1 Aug 2020 17:12:47 -0400
This allows AM-specific knowledge to be applied during creation of
pg_amop and pg_amproc entries. Specifically, the AM knows better than
core code which entries to consider as required or optional. Giving
the latter entries the appropriate sort of dependency allows them to
be dropped without taking out the whole opclass or opfamily; which
is something we'd like to have to correct obsolescent entries in
extensions.
This callback also opens the door to performing AM-specific validity
checks during opclass creation, rather than hoping than an opclass
developer will remember to test with "amvalidate". For the most part
I've not actually added any such checks yet; that can happen in a
follow-on patch. (Note that we shouldn't remove any tests from
"amvalidate", as those are still needed to cross-check manually
constructed entries in the initdb data. So adding tests to
"amadjustmembers" will be somewhat duplicative, but it seems like
a good idea anyway.)
Patch by me, reviewed by Alexander Korotkov, Hamid Akhtar, and
Anastasia Lubennikova.
Discussion: https://postgr.es/m/[email protected]
M contrib/bloom/blutils.c
M doc/src/sgml/indexam.sgml
M src/backend/access/brin/brin.c
M src/backend/access/gin/ginutil.c
M src/backend/access/gin/ginvalidate.c
M src/backend/access/gist/gist.c
M src/backend/access/gist/gistvalidate.c
M src/backend/access/hash/hash.c
M src/backend/access/hash/hashvalidate.c
M src/backend/access/index/amvalidate.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtvalidate.c
M src/backend/access/spgist/spgutils.c
M src/backend/access/spgist/spgvalidate.c
M src/backend/commands/opclasscmds.c
M src/bin/pg_dump/t/002_pg_dump.pl
M src/include/access/amapi.h
M src/include/access/amvalidate.h
M src/include/access/gin_private.h
M src/include/access/gist_private.h
M src/include/access/hash.h
M src/include/access/nbtree.h
M src/include/access/spgist.h
D src/include/catalog/opfam_internal.h
Use pg_pread() and pg_pwrite() in slru.c.
commit : e2b37d9e7cabc90633c4bd822e1bcfdd1bda44c4
author : Thomas Munro <[email protected]>
date : Sat, 1 Aug 2020 23:39:36 +1200
committer: Thomas Munro <[email protected]>
date : Sat, 1 Aug 2020 23:39:36 +1200
This avoids lseek() system calls at every SLRU I/O, as was
done for relation files in commit c24dcd0c.
Reviewed-by: Ashwin Agrawal <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKG%2Biqke4uTRFj8D8uEUUgj%2BRokPSp%2BCWM6YYzaaamG9Wvg%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2BhUKGJ%2BoHhnvqjn3%3DHro7xu-YDR8FPr0FL6LF35kHRX%3D_bUzg%40mail.gmail.com
M src/backend/access/transam/slru.c
Minimize slot creation for multi-inserts of pg_shdepend
commit : 022350b8495a8a7ff0ff8dd6791572e91e7cd6fe
author : Michael Paquier <[email protected]>
date : Sat, 1 Aug 2020 11:49:13 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 1 Aug 2020 11:49:13 +0900
When doing multiple insertions in pg_shdepend for the copy of
dependencies from a template database in CREATE DATABASE, the same
number of slots would have been created and used all the time. As the
number of items to insert is not known in advance, this makes most of
the slots created for nothing. This improves the slot handling so as
slot creation only happens when needed, minimizing the overhead of the
operation.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/pg_shdepend.c
Improve programmer docs for simplehash and dynahash.
commit : 84c0e4b9bce794da914fe9c062753bf21369745f
author : Thomas Munro <[email protected]>
date : Sat, 1 Aug 2020 12:16:15 +1200
committer: Thomas Munro <[email protected]>
date : Sat, 1 Aug 2020 12:16:15 +1200
When reading the code it's not obvious when one should prefer dynahash
over simplehash and vice-versa, so, for programmer-friendliness, add
comments to inform that decision.
Show sample simplehash method signatures.
Author: James Coleman <[email protected]>
Discussion: https://postgr.es/m/CAAaqYe_dOF39gAJ8rL-a3YO3Qo96MHMRQ2whFjK5ZcU6YvMQSA%40mail.gmail.com
M src/backend/utils/hash/dynahash.c
M src/include/lib/simplehash.h
Restore lost amcheck TOAST test coverage.
commit : c79aed4f793086300abfc188def94b5c0bd0b45d
author : Peter Geoghegan <[email protected]>
date : Fri, 31 Jul 2020 15:34:28 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 31 Jul 2020 15:34:28 -0700
Commit eba77534 fixed an amcheck false positive bug involving
inconsistencies in TOAST input state between table and index. A test
case was added that verified that such an inconsistency didn't result in
a spurious corruption related error.
Test coverage from the test was accidentally lost by commit 501e41dd,
which propagated ALTER TABLE ... SET STORAGE attstorage state to
indexes. This broke the test because the test specifically relied on
attstorage not being propagated. This artificially forced there to be
index tuples whose datums were equivalent to the datums in the heap
without the datums actually being bitwise equal.
Fix this by updating pg_attribute directly instead. Commit 501e41dd
made similar changes to a test_decoding TOAST-related test case which
made the same assumption, but overlooked the amcheck test case.
Backpatch: 11-, just like commit eba77534 (and commit 501e41dd).
M contrib/amcheck/expected/check_btree.out
M contrib/amcheck/sql/check_btree.sql
Fix oversight in ALTER TYPE: typmodin/typmodout must propagate to arrays.
commit : 3d2376d55c6f2d364a6a1a95cc350c531f6d9423
author : Tom Lane <[email protected]>
date : Fri, 31 Jul 2020 17:11:28 -0400
committer: Tom Lane <[email protected]>
date : Fri, 31 Jul 2020 17:11:28 -0400
If a base type supports typmods, its array type does too, with the
same interpretation. Hence changes in pg_type.typmodin/typmodout
must be propagated to the array type.
While here, improve AlterTypeRecurse to not recurse to domains if
there is nothing we'd need to change.
Oversight in fe30e7ebf. Back-patch to v13 where that came in.
M src/backend/commands/typecmds.c
M src/test/regress/expected/create_type.out
M src/test/regress/sql/create_type.sql
Fix recently-introduced performance problem in ts_headline().
commit : 78e73e87548a1e0b71b6f2425f76ea6e9c85b2eb
author : Tom Lane <[email protected]>
date : Fri, 31 Jul 2020 11:43:12 -0400
committer: Tom Lane <[email protected]>
date : Fri, 31 Jul 2020 11:43:12 -0400
The new hlCover() algorithm that I introduced in commit c9b0c678d
turns out to potentially take O(N^2) or worse time on long documents,
if there are many occurrences of individual query words but few or no
substrings that actually satisfy the query. (One way to hit this
behavior is with a "common_word & rare_word" type of query.) This
seems unavoidable given the original goal of checking every substring
of the document, so we have to back off that idea. Fortunately, it
seems unlikely that anyone would really want headlines spanning all of
a long document, so we can avoid the worse-than-linear behavior by
imposing a maximum length of substring that we'll consider.
For now, just hard-wire that maximum length as a multiple of max_words
times max_fragments. Perhaps at some point somebody will argue for
exposing it as a ts_headline parameter, but I'm hesitant to make such
a feature addition in a back-patched bug fix.
I also noted that the hlFirstIndex() function I'd added in that
commit was unnecessarily stupid: it really only needs to check whether
a HeadlineWordEntry's item pointer is null or not. This wouldn't make
all that much difference in typical cases with queries having just
a few terms, but a cycle shaved is a cycle earned.
In addition, add a CHECK_FOR_INTERRUPTS call in TS_execute_recurse.
This ensures that hlCover's loop is cancellable if it manages to take
a long time, and it may protect some other TS_execute callers as well.
Back-patch to 9.6 as the previous commit was. I also chose to add the
CHECK_FOR_INTERRUPTS call to 9.5. The old hlCover() algorithm seems
to avoid the O(N^2) behavior, at least on the test case I tried, but
nonetheless it's not very quick on a long document.
Per report from Stephen Frost.
Discussion: https://postgr.es/m/[email protected]
M src/backend/tsearch/wparser_def.c
M src/backend/utils/adt/tsvector_op.c
Fix compiler warning from Clang.
commit : 7be04496a9f763fc4d4c1d06ce9ccc250e52df31
author : Thomas Munro <[email protected]>
date : Fri, 31 Jul 2020 19:08:09 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 31 Jul 2020 19:08:09 +1200
Per build farm.
Discussion: https://postgr.es/m/20200731062626.GD3317%40paquier.xyz
M src/backend/utils/misc/guc.c
Preallocate some DSM space at startup.
commit : 84b1c63ad41872792d47e523363fce1f0e230022
author : Thomas Munro <[email protected]>
date : Fri, 31 Jul 2020 17:27:09 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 31 Jul 2020 17:27:09 +1200
Create an optional region in the main shared memory segment that can be
used to acquire and release "fast" DSM segments, and can benefit from
huge pages allocated at cluster startup time, if configured. Fall back
to the existing mechanisms when that space is full. The size is
controlled by a new GUC min_dynamic_shared_memory, defaulting to 0.
Main region DSM segments initially contain whatever garbage the memory
held last time they were used, rather than zeroes. That change revealed
that DSA areas failed to initialize themselves correctly in memory that
wasn't zeroed first, so fix that problem.
Discussion: https://postgr.es/m/CA%2BhUKGLAE2QBv-WgGp%2BD9P_J-%3Dyne3zof9nfMaqq1h3EGHFXYQ%40mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/storage/ipc/dsm.c
M src/backend/storage/ipc/dsm_impl.c
M src/backend/storage/ipc/ipci.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/backend/utils/mmgr/dsa.c
M src/include/storage/dsm.h
M src/include/storage/dsm_impl.h
Fix comment in instrument.h
commit : 7b1110d2fd3da3d7536530d14952d4f4d9c25438
author : Michael Paquier <[email protected]>
date : Fri, 31 Jul 2020 14:17:29 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 31 Jul 2020 14:17:29 +0900
local_blks_dirtied tracks the number of local blocks dirtied, not shared
ones.
Author: Kirk Jamison
Discussion: https://postgr.es/m/OSBPR01MB2341760686DC056DE89D2AB9EF710@OSBPR01MB2341.jpnprd01.prod.outlook.com
M src/include/executor/instrument.h
Cache smgrnblocks() results in recovery.
commit : c5315f4f44843c20ada876fdb0d0828795dfbdf5
author : Thomas Munro <[email protected]>
date : Fri, 31 Jul 2020 14:15:18 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 31 Jul 2020 14:15:18 +1200
Avoid repeatedly calling lseek(SEEK_END) during recovery by caching
the size of each fork. For now, we can't use the same technique in
other processes, because we lack a shared invalidation mechanism.
Do this by generalizing the pre-existing caching used by FSM and VM
to support all forks.
Discussion: https://postgr.es/m/CAEepm%3D3SSw-Ty1DFcK%3D1rU-K6GSzYzfdD4d%2BZwapdN7dTa6%3DnQ%40mail.gmail.com
M contrib/pg_visibility/pg_visibility.c
M src/backend/access/heap/visibilitymap.c
M src/backend/catalog/storage.c
M src/backend/storage/freespace/freespace.c
M src/backend/storage/smgr/smgr.c
M src/include/storage/smgr.h
Use multi-inserts for pg_attribute and pg_shdepend
commit : e3931d01f3afef14703827eda1dad0a3fb3b5d07
author : Michael Paquier <[email protected]>
date : Fri, 31 Jul 2020 10:54:26 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 31 Jul 2020 10:54:26 +0900
For pg_attribute, this allows to insert at once a full set of attributes
for a relation (roughly 15% of WAL reduction in extreme cases). For
pg_shdepend, this reduces the work done when creating new shared
dependencies from a database template. The number of slots used for the
insertion is capped at 64kB of data inserted for both, depending on the
number of items to insert and the length of the rows involved.
More can be done for other catalogs, like pg_depend. This part requires
a different approach as the number of slots to use depends also on the
number of entries discarded as pinned dependencies. This is also
related to the rework or dependency handling for ALTER TABLE and CREATE
TABLE, mainly.
Author: Daniel Gustafsson
Reviewed-by: Andres Freund, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam.c
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/indexing.c
M src/backend/catalog/pg_shdepend.c
M src/backend/commands/tablecmds.c
M src/include/catalog/heap.h
M src/include/catalog/indexing.h
Doc: fix high availability solutions comparison.
commit : cab2556f3ab289b81a9c6a75e669b6ec78356ffc
author : Tatsuo Ishii <[email protected]>
date : Fri, 31 Jul 2020 07:18:41 +0900
committer: Tatsuo Ishii <[email protected]>
date : Fri, 31 Jul 2020 07:18:41 +0900
In "High Availability, Load Balancing, and Replication" chapter,
certain descriptions of Pgpool-II were not correct at this point. It
does not need conflict resolution. Also "Multiple-Server Parallel
Query Execution" is not supported anymore.
Discussion: https://postgr.es/m/20200726.230128.53842489850344110.t-ishii%40sraoss.co.jp
Author: Tatsuo Ishii
Reviewed-by: Bruce Momjian
Backpatch-through: 9.5
M doc/src/sgml/high-availability.sgml
Use pg_bitutils for HyperLogLog.
commit : fd734f387d8780d9989d750942d026167de8cf3c
author : Jeff Davis <[email protected]>
date : Thu, 30 Jul 2020 08:44:58 -0700
committer: Jeff Davis <[email protected]>
date : Thu, 30 Jul 2020 08:44:58 -0700
Using pg_leftmost_one_post32() yields substantial performance benefits.
Backpatching to version 13 because HLL is used for HashAgg
improvements in 9878b643, which was also backpatched to 13.
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-WzkGvDKVDo+0YvfvZ+1CE=iCi88DCOGFF3i1hTGGaxcKPw@mail.gmail.com
Backpatch-through: 13
M src/backend/lib/hyperloglog.c
Include partitioned tables for tab completion of VACUUM in psql
commit : f1af75c5f2516ec5b20cfe4b3a474071a318ae1e
author : Michael Paquier <[email protected]>
date : Thu, 30 Jul 2020 16:57:37 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 30 Jul 2020 16:57:37 +0900
The relkinds that support indexing are the same as the ones supporting
VACUUM, so the code gets refactored a bit with the completion query used
for CLUSTER, but there is no change for CLUSTER in this commit.
Author: Justin Pryzby
Reviewed-by: Fujii Masao, Michael Paquier, Masahiko Sawada
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/tab-complete.c
doc: Mention index references in pg_inherits
commit : 903134fcc0ccd188803fdbc2b7c06b898749153a
author : Michael Paquier <[email protected]>
date : Thu, 30 Jul 2020 15:48:44 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 30 Jul 2020 15:48:44 +0900
Partitioned indexes are also registered in pg_inherits, but the
description of this catalog did not reflect that.
Author: Dagfinn Ilmari Mannsåker
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M doc/src/sgml/catalogs.sgml
Introduce a WaitEventSet for the stats collector.
commit : e7591fd3cae6c64236ef29d3c87e69b96608a19b
author : Thomas Munro <[email protected]>
date : Thu, 30 Jul 2020 17:25:48 +1200
committer: Thomas Munro <[email protected]>
date : Thu, 30 Jul 2020 17:25:48 +1200
This avoids avoids some epoll/kqueue system calls for every wait.
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw%40mail.gmail.com
M src/backend/postmaster/pgstat.c
Use WaitLatch() for condition variables.
commit : e2d394df5df28ab5ee4bfac6b13837e99e24045d
author : Thomas Munro <[email protected]>
date : Thu, 30 Jul 2020 17:23:32 +1200
committer: Thomas Munro <[email protected]>
date : Thu, 30 Jul 2020 17:23:32 +1200
Previously, condition_variable.c created a long lived WaitEventSet to
avoid extra system calls. WaitLatch() now uses something similar
internally, so there is no point in wasting an extra kernel descriptor.
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw%40mail.gmail.com
M src/backend/storage/lmgr/condition_variable.c
Use a long lived WaitEventSet for WaitLatch().
commit : 3347c982bab0dd56d5b6cb784521233ba2bbac27
author : Thomas Munro <[email protected]>
date : Thu, 30 Jul 2020 17:08:11 +1200
committer: Thomas Munro <[email protected]>
date : Thu, 30 Jul 2020 17:08:11 +1200
Create LatchWaitSet at backend startup time, and use it to implement
WaitLatch(). This avoids repeated epoll/kqueue setup and teardown
system calls.
Reorder SubPostmasterMain() slightly so that we restore the postmaster
pipe and Windows signal emulation before we reach InitPostmasterChild(),
to make this work in EXEC_BACKEND builds.
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw%40mail.gmail.com
M src/backend/postmaster/postmaster.c
M src/backend/storage/ipc/latch.c
M src/backend/utils/init/miscinit.c
M src/include/storage/latch.h
Add hash_mem_multiplier GUC.
commit : d6c08e29e7bc8bc3bf49764192c4a9c71fc0b097
author : Peter Geoghegan <[email protected]>
date : Wed, 29 Jul 2020 14:14:58 -0700
committer: Peter Geoghegan <[email protected]>
date : Wed, 29 Jul 2020 14:14:58 -0700
Add a GUC that acts as a multiplier on work_mem. It gets applied when
sizing executor node hash tables that were previously size constrained
using work_mem alone.
The new GUC can be used to preferentially give hash-based nodes more
memory than the generic work_mem limit. It is intended to enable admin
tuning of the executor's memory usage. Overall system throughput and
system responsiveness can be improved by giving hash-based executor
nodes more memory (especially over sort-based alternatives, which are
often much less sensitive to being memory constrained).
The default value for hash_mem_multiplier is 1.0, which is also the
minimum valid value. This means that hash-based nodes continue to apply
work_mem in the traditional way by default.
hash_mem_multiplier is generally useful. However, it is being added now
due to concerns about hash aggregate performance stability for users
that upgrade to Postgres 13 (which added disk-based hash aggregation in
commit 1f39bce0). While the old hash aggregate behavior risked
out-of-memory errors, it is nevertheless likely that many users actually
benefited. Hash agg's previous indifference to work_mem during query
execution was not just faster; it also accidentally made aggregation
resilient to grouping estimate problems (at least in cases where this
didn't create destabilizing memory pressure).
hash_mem_multiplier can provide a certain kind of continuity with the
behavior of Postgres 12 hash aggregates in cases where the planner
incorrectly estimates that all groups (plus related allocations) will
fit in work_mem/hash_mem. This seems necessary because hash-based
aggregation is usually much slower when only a small fraction of all
groups can fit. Even when it isn't possible to totally avoid hash
aggregates that spill, giving hash aggregation more memory will reliably
improve performance (the same cannot be said for external sort
operations, which appear to be almost unaffected by memory availability
provided it's at least possible to get a single merge pass).
The PostgreSQL 13 release notes should advise users that increasing
hash_mem_multiplier can help with performance regressions associated
with hash aggregation. That can be taken care of by a later commit.
Author: Peter Geoghegan
Reviewed-By: Álvaro Herrera, Jeff Davis
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CAH2-WzmD%2Bi1pG6rc1%2BCjc4V6EaFJ_qSuKCCHVnH%3DoruqD-zqow%40mail.gmail.com
Backpatch: 13-, where disk-based hash aggregation was introduced.
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/postgres-ref.sgml
M doc/src/sgml/runtime.sgml
M src/backend/executor/execGrouping.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeHashjoin.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/pathnode.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/init/globals.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/executor/hashjoin.h
M src/include/executor/nodeHash.h
M src/include/miscadmin.h
pg_stat_statements: track number of rows processed by some utility commands.
commit : 6023b7ea717ca04cf1bd53709d9c862db07eaefb
author : Fujii Masao <[email protected]>
date : Wed, 29 Jul 2020 23:21:55 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 29 Jul 2020 23:21:55 +0900
This commit makes pg_stat_statements track the total number
of rows retrieved or affected by CREATE TABLE AS, SELECT INTO,
CREATE MATERIALIZED VIEW and FETCH commands.
Suggested-by: Pascal Legrand
Author: Fujii Masao
Reviewed-by: Asif Rehman
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
Remove non-fast promotion.
commit : b5310e4ff6b7b0b14a5ee2443839fbf3553623ea
author : Fujii Masao <[email protected]>
date : Wed, 29 Jul 2020 21:24:26 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 29 Jul 2020 21:24:26 +0900
When fast promotion was supported in 9.3, non-fast promotion became
undocumented feature and it's basically not available for ordinary users.
However we decided not to remove non-fast promotion at that moment,
to leave it for a release or two for debugging purpose or as an emergency
method because fast promotion might have some issues, and then to
remove it later. Now, several versions were released since that decision
and there is no longer reason to keep supporting non-fast promotion.
Therefore this commit removes non-fast promotion.
Author: Fujii Masao
Reviewed-by: Hamid Akhtar, Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
M src/backend/postmaster/postmaster.c
M src/bin/pg_ctl/pg_ctl.c
M src/include/access/xlog.h
HashAgg: use better cardinality estimate for recursive spilling.
commit : 9878b643f37b1e4167f64a9941244bfabed60623
author : Jeff Davis <[email protected]>
date : Tue, 28 Jul 2020 23:15:47 -0700
committer: Jeff Davis <[email protected]>
date : Tue, 28 Jul 2020 23:15:47 -0700
Use HyperLogLog to estimate the group cardinality in a spilled
partition. This estimate is used to choose the number of partitions if
we recurse.
The previous behavior was to use the number of tuples in a spilled
partition as the estimate for the number of groups, which lead to
overpartitioning. That could cause the number of batches to be much
higher than expected (with each batch being very small), which made it
harder to interpret EXPLAIN ANALYZE results.
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
M src/include/executor/nodeAgg.h
Fix incorrect print format in json.c
commit : f2130e77da51f35d37fd15a343bc1c4a4527e0fd
author : Michael Paquier <[email protected]>
date : Wed, 29 Jul 2020 14:44:32 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 29 Jul 2020 14:44:32 +0900
Oid is unsigned, so %u needs to be used and not %d. The code path
involved here is not normally reachable, so no backpatch is done.
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/json.c
Move syncscan.c to src/backend/access/common.
commit : cb04ad498551dcdb91a834c2e8730cdf0b77e70a
author : Thomas Munro <[email protected]>
date : Wed, 29 Jul 2020 16:46:58 +1200
committer: Thomas Munro <[email protected]>
date : Wed, 29 Jul 2020 16:46:58 +1200
Since the tableam.c code needs to make use of the syncscan.c routines
itself, and since other block-oriented AMs might also want to use it one
day, it didn't make sense for it to live under src/backend/access/heap.
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGLCnG%3DNEAByg6bk%2BCT9JZD97Y%3DAxKhh27Su9FeGWOKvDg%40mail.gmail.com
M src/backend/access/common/Makefile
R098 src/backend/access/heap/syncscan.c src/backend/access/common/syncscan.c
M src/backend/access/heap/Makefile
M src/backend/access/heap/heapam.c
M src/backend/access/heap/heapam_handler.c
M src/backend/access/table/tableam.c
M src/backend/storage/ipc/ipci.c
M src/include/access/heapam.h
A src/include/access/syncscan.h
Rename another "hash_mem" local variable.
commit : c49c74d19241b1fc8da6c215ebb40fd6b71c1bff
author : Peter Geoghegan <[email protected]>
date : Tue, 28 Jul 2020 17:59:16 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 28 Jul 2020 17:59:16 -0700
Missed by my commit 564ce621.
Backpatch: 13-, where disk-based hash aggregation was introduced.
M src/backend/executor/nodeAgg.c
Correct obsolete UNION hash aggs comment.
commit : b1d79127ed875f04720d2c4677a75f43528bfe08
author : Peter Geoghegan <[email protected]>
date : Tue, 28 Jul 2020 17:14:07 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 28 Jul 2020 17:14:07 -0700
Oversight in commit 1f39bce0, which added disk-based hash aggregation.
Backpatch: 13-, where disk-based hash aggregation was introduced.
M src/backend/optimizer/prep/prepunion.c
Doc: Remove obsolete CREATE AGGREGATE note.
commit : f36e82072c8866ba2eca08d88d1a5c3e0c3d1eb4
author : Peter Geoghegan <[email protected]>
date : Tue, 28 Jul 2020 16:59:01 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 28 Jul 2020 16:59:01 -0700
The planner is in fact willing to use hash aggregation when work_mem is
not set high enough for everything to fit in memory. This has been the
case since commit 1f39bce0, which added disk-based hash aggregation.
There are a few remaining cases in which hash aggregation is avoided as
a matter of policy when the planner surmises that spilling will be
necessary. For example, callers of choose_hashed_setop() still
conservatively avoid hash aggregation when spilling is anticipated.
That doesn't seem like a good enough reason to mention hash aggregation
in this context.
Backpatch: 13-, where disk-based hash aggregation was introduced.
M doc/src/sgml/ref/create_aggregate.sgml
Make EXPLAIN ANALYZE of HashAgg more similar to Hash Join
commit : 0e3e1c4e1cea68073132fe817fb3a98cb5c1b805
author : David Rowley <[email protected]>
date : Wed, 29 Jul 2020 11:42:21 +1200
committer: David Rowley <[email protected]>
date : Wed, 29 Jul 2020 11:42:21 +1200
There were various unnecessary differences between Hash Agg's EXPLAIN
ANALYZE output and Hash Join's. Here we modify the Hash Agg output so
that it's better aligned to Hash Join's.
The following changes have been made:
1. Start batches counter at 1 instead of 0.
2. Always display the "Batches" property, even when we didn't spill to
disk.
3. Use the text "Batches" instead of "HashAgg Batches" for text format.
4. Use the text "Memory Usage" instead of "Peak Memory Usage" for text
format.
5. Include "Batches" before "Memory Usage" in both text and non-text
formats.
In passing also modify the "Planned Partitions" property so that we show
it regardless of if the value is 0 or not for non-text EXPLAIN formats.
This was pointed out by Justin Pryzby and probably should have been part
of 40efbf870.
Reviewed-by: Justin Pryzby, Jeff Davis
Discussion: https://postgr.es/m/CAApHDvrshRnA6C0VFnu7Fb9TVvgGo80PUMm5+2DiaS1gEkPvtw@mail.gmail.com
Backpatch-through: 13, where HashAgg batching was introduced
M src/backend/commands/explain.c
M src/backend/executor/nodeAgg.c
Doc: Improve documentation for pg_jit_available()
commit : d7c8576ebe3949a644c700a9f54d88e7e373a647
author : David Rowley <[email protected]>
date : Tue, 28 Jul 2020 22:52:03 +1200
committer: David Rowley <[email protected]>
date : Tue, 28 Jul 2020 22:52:03 +1200
Per complaint from Scott Ribe. Based on wording suggestion from Tom Lane.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11, where pg_jit_available() was added
M doc/src/sgml/func.sgml
Extend the logical decoding output plugin API with stream methods.
commit : 45fdc9738b36d1068d3ad8fdb06436d6fd14436b
author : Amit Kapila <[email protected]>
date : Tue, 28 Jul 2020 08:06:44 +0530
committer: Amit Kapila <[email protected]>
date : Tue, 28 Jul 2020 08:06:44 +0530
This adds seven methods to the output plugin API, adding support for
streaming changes of large in-progress transactions.
* stream_start
* stream_stop
* stream_abort
* stream_commit
* stream_change
* stream_message
* stream_truncate
Most of this is a simple extension of the existing methods, with
the semantic difference that the transaction (or subtransaction)
is incomplete and may be aborted later (which is something the
regular API does not really need to deal with).
This also extends the 'test_decoding' plugin, implementing these
new stream methods.
The stream_start/start_stop are used to demarcate a chunk of changes
streamed for a particular toplevel transaction.
This commit simply adds these new APIs and the upcoming patch to "allow
the streaming mode in ReorderBuffer" will use these APIs.
Author: Tomas Vondra, Dilip Kumar, Amit Kapila
Reviewed-by: Amit Kapila
Tested-by: Neha Sharma and Mahendra Singh Thalor
Discussion: https://postgr.es/m/[email protected]
M contrib/test_decoding/test_decoding.c
M doc/src/sgml/logicaldecoding.sgml
M src/backend/replication/logical/logical.c
M src/include/replication/logical.h
M src/include/replication/output_plugin.h
M src/include/replication/reorderbuffer.h
Fix some issues with step generation in partition pruning.
commit : 13838740f61fc455aa4196d257efc0b761daba1f
author : Etsuro Fujita <[email protected]>
date : Tue, 28 Jul 2020 11:00:00 +0900
committer: Etsuro Fujita <[email protected]>
date : Tue, 28 Jul 2020 11:00:00 +0900
In the case of range partitioning, get_steps_using_prefix() assumes that
the passed-in prefix list contains at least one clause for each of the
partition keys earlier than one specified in the passed-in
step_lastkeyno, but the caller (ie, gen_prune_steps_from_opexps())
didn't take it into account, which led to a server crash or incorrect
results when the list contained no clauses for such partition keys, as
reported in bug #16500 and #16501 from Kobayashi Hisanori. Update the
caller to call that function only when the list created there contains
at least one clause for each of the earlier partition keys in the case
of range partitioning.
While at it, fix some other issues:
* The list to pass to get_steps_using_prefix() is allowed to contain
multiple clauses for the same partition key, as described in the
comment for that function, but that function actually assumed that the
list contained just a single clause for each of middle partition keys,
which led to an assertion failure when the list contained multiple
clauses for such partition keys. Update that function to match the
comment.
* In the case of hash partitioning, partition keys are allowed to be
NULL, in which case the list to pass to get_steps_using_prefix()
contains no clauses for NULL partition keys, but that function treats
that case as like the case of range partitioning, which led to the
assertion failure. Update the assertion test to take into account
NULL partition keys in the case of hash partitioning.
* Fix a typo in a comment in get_steps_using_prefix_recurse().
* gen_partprune_steps() failed to detect self-contradiction from
strict-qual clauses and an IS NULL clause for the same partition key
in some cases, producing incorrect partition-pruning steps, which led
to incorrect results of partition pruning, but didn't cause any
user-visible problems fortunately, as the self-contradiction is
detected later in the query planning. Update that function to detect
the self-contradiction.
Per bug #16500 and #16501 from Kobayashi Hisanori. Patch by me, initial
diagnosis for the reported issue and review by Dmitry Dolgov.
Back-patch to v11, where partition pruning was introduced.
Discussion: https://postgr.es/m/16500-d1613f2a78e1e090%40postgresql.org
Discussion: https://postgr.es/m/16501-5234a9a0394f6754%40postgresql.org
M src/backend/partitioning/partprune.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Remove hashagg_avoid_disk_plan GUC.
commit : bcbf9446a2983b6452c19cc50050456be262f7c5
author : Peter Geoghegan <[email protected]>
date : Mon, 27 Jul 2020 17:53:19 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 27 Jul 2020 17:53:19 -0700
Note: This GUC was originally named enable_hashagg_disk when it appeared
in commit 1f39bce0, which added disk-based hash aggregation. It was
subsequently renamed in commit 92c58fd9.
Author: Peter Geoghegan
Reviewed-By: Jeff Davis, Álvaro Herrera
Discussion: https://postgr.es/m/9d9d1e1252a52ea1bad84ea40dbebfd54e672a0f.camel%40j-davis.com
Backpatch: 13-, where disk-based hash aggregation was introduced.
M doc/src/sgml/config.sgml
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/utils/misc/guc.c
M src/include/optimizer/cost.h
Fix corner case with 16kB-long decompression in pgcrypto, take 2
commit : a3ab7a707d9eda4b2162273348cba52252c0f0c9
author : Michael Paquier <[email protected]>
date : Mon, 27 Jul 2020 15:58:32 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 27 Jul 2020 15:58:32 +0900
A compressed stream may end with an empty packet. In this case
decompression finishes before reading the empty packet and the
remaining stream packet causes a failure in reading the following
data. This commit makes sure to consume such extra data, avoiding a
failure when decompression the data. This corner case was reproducible
easily with a data length of 16kB, and existed since e94dd6a. A cheap
regression test is added to cover this case based on a random,
incompressible string.
The first attempt of this patch has allowed to find an older failure
within the compression logic of pgcrypto, fixed by b9b6105. This
involved SLES 15 with z390 where a custom flavor of libz gets used.
Bonus thanks to Mark Wong for providing access to the specific
environment.
Reported-by: Frank Gagnepain
Author: Kyotaro Horiguchi, Michael Paquier
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M contrib/pgcrypto/expected/pgp-compression.out
M contrib/pgcrypto/pgp-compress.c
M contrib/pgcrypto/sql/pgp-compression.sql
Fix handling of structure for bytea data type in ECPG
commit : e971357961f2bf5bddebb3f68ba8b55954709486
author : Michael Paquier <[email protected]>
date : Mon, 27 Jul 2020 10:28:06 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 27 Jul 2020 10:28:06 +0900
Some code paths dedicated to bytea used the structure for varchar. This
did not lead to any actual bugs, as bytea and varchar have the same
definition, but it could become a trap if one of these definitions
changes for a new feature or a bug fix.
Issue introduced by 050710b.
Author: Shenhao Wang
Reviewed-by: Vignesh C, Michael Paquier
Discussion: https://postgr.es/m/07ac7dee1efc44f99d7f53a074420177@G08CNEXMBPEKD06.g08.fujitsu.local
Backpatch-through: 12
M src/interfaces/ecpg/ecpglib/data.c
M src/interfaces/ecpg/ecpglib/descriptor.c
M src/interfaces/ecpg/ecpglib/execute.c
Fix LookupTupleHashEntryHash() pipeline-stall issue.
commit : 200f6100a9f9fc71273aeb6aceac4430f3437195
author : Jeff Davis <[email protected]>
date : Sun, 26 Jul 2020 14:55:52 -0700
committer: Jeff Davis <[email protected]>
date : Sun, 26 Jul 2020 14:55:52 -0700
Refactor hash lookups in nodeAgg.c to improve performance.
Author: Andres Freund and Jeff Davis
Discussion: https://postgr.es/m/20200612213715.op4ye4q7gktqvpuo%40alap3.anarazel.de
Backpatch-through: 13
M src/backend/executor/execGrouping.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeRecursiveunion.c
M src/backend/executor/nodeSetOp.c
M src/backend/executor/nodeSubplan.c
M src/include/executor/executor.h
Allocate consecutive blocks during parallel seqscans
commit : 56788d2156fc32bd5737e7ac716d70e6a269b7bc
author : David Rowley <[email protected]>
date : Sun, 26 Jul 2020 21:02:45 +1200
committer: David Rowley <[email protected]>
date : Sun, 26 Jul 2020 21:02:45 +1200
Previously we would allocate blocks to parallel workers during a parallel
sequential scan 1 block at a time. Since other workers were likely to
request a block before a worker returns for another block number to work
on, this could lead to non-sequential I/O patterns in each worker which
could cause the operating system's readahead to perform poorly or not at
all.
Here we change things so that we allocate consecutive "chunks" of blocks
to workers and have them work on those until they're done, at which time
we allocate another chunk for the worker. The size of these chunks is
based on the size of the relation.
Initial patch here was by Thomas Munro which showed some good improvements
just having a fixed chunk size of 64 blocks with a simple ramp-down near
the end of the scan. The revisions of the patch to make the chunk size
based on the relation size and the adjusted ramp-down in powers of two was
done by me, along with quite extensive benchmarking to determine the
optimal chunk sizes.
For the most part, benchmarks have shown significant performance
improvements for large parallel sequential scans on Linux, FreeBSD and
Windows using SSDs. It's less clear how this affects the performance of
cloud providers. Tests done so far are unable to obtain stable enough
performance to provide meaningful benchmark results. It is possible that
this could cause some performance regressions on more obscure filesystems,
so we may need to later provide users with some ability to get something
closer to the old behavior. For now, let's leave that until we see that
it's really required.
Author: Thomas Munro, David Rowley
Reviewed-by: Ranier Vilela, Soumyadeep Chakraborty, Robert Haas
Reviewed-by: Amit Kapila, Kirk Jamison
Discussion: https://postgr.es/m/CA+hUKGJ_EErDv41YycXcbMbCBkztA34+z1ts9VQH+ACRuvpxig@mail.gmail.com
M src/backend/access/heap/heapam.c
M src/backend/access/table/tableam.c
M src/include/access/relscan.h
M src/include/access/tableam.h
Tweak behavior of pg_stat_activity.leader_pid
commit : 11a68e4b53ffccf336a2faf5fa380acda28e880b
author : Michael Paquier <[email protected]>
date : Sun, 26 Jul 2020 16:32:11 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 26 Jul 2020 16:32:11 +0900
The initial implementation of leader_pid in pg_stat_activity added by
b025f32 took the approach to strictly print what a PGPROC entry
includes. In short, if a backend has been involved in parallel query at
least once, leader_pid would remain set as long as the backend is alive.
For a parallel group leader, this means that the field would always be
set after it participated at least once in parallel query, and after
more discussions this could be confusing if using for example a
connection pooler.
This commit changes the data printed so as leader_pid becomes always
NULL for a parallel group leader, showing up a non-NULL value only for
the parallel workers, and actually as long as a parallel query is
running as workers are shut down once the query has completed.
This does not change the definition of any catalog, so no catalog bump
is needed. Per discussion with Justin Pryzby, Álvaro Herrera, Julien
Rouhaud and me.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M doc/src/sgml/monitoring.sgml
M src/backend/utils/adt/pgstatfuncs.c
Remove optimization for RAND_poll() failing.
commit : 15e441972276e95639f8c3d9f5f66c2318fe9348
author : Noah Misch <[email protected]>
date : Sat, 25 Jul 2020 14:50:59 -0700
committer: Noah Misch <[email protected]>
date : Sat, 25 Jul 2020 14:50:59 -0700
The loop to generate seed data will exit on RAND_status(), so we don't
need to handle the case of RAND_poll() failing separately. Failures
here are rare, so this a code cleanup, essentially.
Daniel Gustafsson, reviewed by David Steele and Michael Paquier.
Discussion: https://postgr.es/m/[email protected]
M src/port/pg_strong_random.c
Use RAND_poll() for seeding randomness after fork().
commit : ce4939ff70890fa658a4095b9fe457f8432b2575
author : Noah Misch <[email protected]>
date : Sat, 25 Jul 2020 14:50:59 -0700
committer: Noah Misch <[email protected]>
date : Sat, 25 Jul 2020 14:50:59 -0700
OpenSSL deprecated RAND_cleanup(), and OpenSSL 1.1.0 made it into a
no-op. Replace it with RAND_poll(), per an OpenSSL community
recommendation. While this has no user-visible consequences under
OpenSSL defaults, it might help under non-default settings.
Daniel Gustafsson, reviewed by David Steele and Michael Paquier.
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/fork_process.c
Improve performance of binary COPY FROM through better buffering.
commit : 0a0727ccfc5f4e2926623abe877bdc0b5bfd682e
author : Tom Lane <[email protected]>
date : Sat, 25 Jul 2020 16:34:35 -0400
committer: Tom Lane <[email protected]>
date : Sat, 25 Jul 2020 16:34:35 -0400
At least on Linux and macOS, fread() turns out to have far higher
per-call overhead than one could wish. Reading 64KB of data at a time
and then parceling it out with our own memcpy logic makes binary COPY
from a file significantly faster --- around 30% in simple testing for
cases with narrow text columns (on Linux ... even more on macOS).
In binary COPY from frontend, there's no per-call fread(), and this
patch introduces an extra layer of memcpy'ing, but it still manages
to eke out a small win. Apparently, the control-logic overhead in
CopyGetData() is enough to be worth avoiding for small fetches.
Bharath Rupireddy and Amit Langote, reviewed by Vignesh C,
cosmetic tweaks by me
Discussion: https://postgr.es/m/CALj2ACU5Bz06HWLwqSzNMN=Gupoj6Rcn_QVC+k070V4em9wu=A@mail.gmail.com
M src/backend/commands/copy.c
Mark built-in coercion functions as leakproof where possible.
commit : 8a37951eebffd9bf528cb06d46127fb721d0e452
author : Tom Lane <[email protected]>
date : Sat, 25 Jul 2020 12:54:58 -0400
committer: Tom Lane <[email protected]>
date : Sat, 25 Jul 2020 12:54:58 -0400
Making these leakproof seems helpful since (for example) if you have a
function f(int8) that is leakproof, you don't want it to effectively
become non-leakproof when you apply it to an int4 or int2 column.
But that's what happens today, since the implicit up-coercion will
not be leakproof.
Most of the coercion functions that visibly can't throw errors are
functions that convert numeric datatypes to other, wider ones.
Notable is that float4_numeric and float8_numeric can be marked
leakproof; before commit a57d312a7 they could not have been.
I also marked the functions that coerce strings to "name" as leakproof;
that's okay today because they truncate silently, but if we ever
reconsidered that behavior then they could no longer be leakproof.
I desisted from marking rtrim1() as leakproof; it appears so right now,
but the code seems a little too complex and perhaps subject to change,
since it's shared with other SQL functions.
Discussion: https://postgr.es/m/[email protected]
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/opr_sanity.out
Fix buffer usage stats for nodes above Gather Merge.
commit : 2a2494229a709b880a6db82d8b267017fccf671f
author : Amit Kapila <[email protected]>
date : Sat, 25 Jul 2020 10:20:39 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 25 Jul 2020 10:20:39 +0530
Commit 85c9d347 addressed a similar problem for Gather and Gather
Merge nodes but forgot to account for nodes above parallel nodes. This
still works for nodes above Gather node because we shut down the workers
for Gather node as soon as there are no more tuples. We can do a similar
thing for Gather Merge as well but it seems better to account for stats
during nodes shutdown after completing the execution.
Reported-by: Stéphane Lorek, Jehan-Guillaume de Rorthais
Author: Jehan-Guillaume de Rorthais <[email protected]>
Reviewed-by: Amit Kapila
Backpatch-through: 10, where it was introduced
Discussion: https://postgr.es/m/20200718160206.584532a2@firost
M src/backend/executor/execProcnode.c
Replace TS_execute's TS_EXEC_CALC_NOT flag with TS_EXEC_SKIP_NOT.
commit : 79d6d1a277ee1cdda90f9a66d7970ac3885822de
author : Tom Lane <[email protected]>
date : Fri, 24 Jul 2020 15:43:56 -0400
committer: Tom Lane <[email protected]>
date : Fri, 24 Jul 2020 15:43:56 -0400
It's fairly silly that ignoring NOT subexpressions is TS_execute's
default behavior. It's wrong on its face and it encourages errors
of omission. Moreover, the only two remaining callers that aren't
specifying CALC_NOT are in ts_headline calculations, and it's very
arguable that those are bugs: if you've specified "!foo" in your
query, why would you want to get a headline that includes "foo"?
Hence, rip that out and change the default behavior to be to calculate
NOT accurately. As a concession to the slim chance that there is still
somebody somewhere who needs the incorrect behavior, provide a new
SKIP_NOT flag to explicitly request that.
Back-patch into v13, mainly because it seems better to change this
at the same time as the previous commit's rejiggering of TS_execute
related APIs. Any outside callers affected by this change are
probably also affected by that one.
Discussion: https://postgr.es/m/CALT9ZEE-aLotzBg-pOp2GFTesGWVYzXA3=mZKzRDa_OKnLF7Mg@mail.gmail.com
M src/backend/utils/adt/tsginidx.c
M src/backend/utils/adt/tsgistidx.c
M src/backend/utils/adt/tsrank.c
M src/backend/utils/adt/tsvector_op.c
M src/include/tsearch/ts_utils.h
Fix assorted bugs by changing TS_execute's callback API to ternary logic.
commit : 2f2007fbb255be178aca586780967f43885203a7
author : Tom Lane <[email protected]>
date : Fri, 24 Jul 2020 15:26:51 -0400
committer: Tom Lane <[email protected]>
date : Fri, 24 Jul 2020 15:26:51 -0400
Text search sometimes failed to find valid matches, for instance
'!crew:A'::tsquery might fail to locate 'crew:1B'::tsvector during
an index search. The root of the issue is that TS_execute's callback
functions were not changed to use ternary (yes/no/maybe) reporting
when we made the search logic itself do so. It's somewhat annoying
to break that API, but on the other hand we now see that any code
using plain boolean logic is almost certainly broken since the
addition of phrase search. There seem to be very few outside callers
of this code anyway, so we'll just break them intentionally to get
them to adapt.
This allows removal of tsginidx.c's private re-implementation of
TS_execute, since that's now entirely duplicative. It's also no
longer necessary to avoid use of CALC_NOT in tsgistidx.c, since
the underlying callbacks can now do something reasonable.
Back-patch into v13. We can't change this in stable branches,
but it seems not quite too late to fix it in v13.
Tom Lane and Pavel Borisov
Discussion: https://postgr.es/m/CALT9ZEE-aLotzBg-pOp2GFTesGWVYzXA3=mZKzRDa_OKnLF7Mg@mail.gmail.com
M src/backend/tsearch/wparser_def.c
M src/backend/utils/adt/tsginidx.c
M src/backend/utils/adt/tsgistidx.c
M src/backend/utils/adt/tsrank.c
M src/backend/utils/adt/tsvector_op.c
M src/include/tsearch/ts_utils.h
M src/test/regress/expected/tsearch.out
M src/test/regress/expected/tstypes.out
M src/test/regress/sql/tsearch.sql
M src/test/regress/sql/tstypes.sql
Rename configure.in to configure.ac
commit : 25244b8972a34b838c4033fe9efc1d31cba9d0e3
author : Peter Eisentraut <[email protected]>
date : Fri, 24 Jul 2020 10:34:16 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 24 Jul 2020 10:34:16 +0200
The new name has been preferred by Autoconf for a long time. Future
versions of Autoconf will warn about the old name.
Discussion: https://www.postgresql.org/message-id/flat/e796c185-5ece-8569-248f-dd3799701be1%402ndquadrant.com
M config/general.m4
R099 configure.in configure.ac
M src/backend/catalog/Makefile
M src/include/pg_config.h.in
M src/tools/msvc/Solution.pm
M src/tools/version_stamp.pl
Fix ancient violation of zlib's API spec.
commit : b9b610577d7f70d959968c3697557011699b3a54
author : Tom Lane <[email protected]>
date : Thu, 23 Jul 2020 17:19:37 -0400
committer: Tom Lane <[email protected]>
date : Thu, 23 Jul 2020 17:19:37 -0400
contrib/pgcrypto mishandled the case where deflate() does not consume
all of the offered input on the first try. It reset the next_in pointer
to the start of the input instead of leaving it alone, causing the wrong
data to be fed to the next deflate() call.
This has been broken since pgcrypto was committed. The reason for the
lack of complaints seems to be that it's fairly hard to get stock zlib
to not consume all the input, so long as the output buffer is big enough
(which it normally would be in pgcrypto's usage; AFAICT the input is
always going to be packetized into packets no larger than ZIP_OUT_BUF).
However, IBM's zlibNX implementation for AIX evidently will do it
in some cases.
I did not add a test case for this, because I couldn't find one that
would fail with stock zlib. When we put back the test case for
bug #16476, that will cover the zlibNX situation well enough.
While here, write deflate()'s second argument as Z_NO_FLUSH per its
API spec, instead of hard-wiring the value zero.
Per buildfarm results and subsequent investigation.
Discussion: https://postgr.es/m/[email protected]
M contrib/pgcrypto/pgp-compress.c
doc: Document that ssl_ciphers does not affect TLS 1.3
commit : 5733fa0fe4a73efa46801aa4189f7da17dd2b4bf
author : Peter Eisentraut <[email protected]>
date : Thu, 23 Jul 2020 17:13:00 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 23 Jul 2020 17:13:00 +0200
TLS 1.3 uses a different way of specifying ciphers and a different
OpenSSL API. PostgreSQL currently does not support setting those
ciphers. For now, just document this. In the future, support for
this might be added somehow.
Reviewed-by: Jonathan S. Katz <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
M doc/src/sgml/config.sgml
Fix error message.
commit : 42dee8b8e362bae05de2234a4fc7a3aa9dacdf6f
author : Thomas Munro <[email protected]>
date : Thu, 23 Jul 2020 21:10:49 +1200
committer: Thomas Munro <[email protected]>
date : Thu, 23 Jul 2020 21:10:49 +1200
Remove extra space. Back-patch to all releases, like commit 7897e3bb.
Author: Lu, Chenyang <[email protected]>
Discussion: https://postgr.es/m/795d03c6129844d3803e7eea48f5af0d%40G08CNEXMBPEKD04.g08.fujitsu.local
M src/backend/storage/file/buffile.c
WAL Log invalidations at command end with wal_level=logical.
commit : c55040ccd017962b7b8d3fbcdc184aa90c722a21
author : Amit Kapila <[email protected]>
date : Thu, 23 Jul 2020 08:19:07 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 23 Jul 2020 08:19:07 +0530
When wal_level=logical, write invalidations at command end into WAL so
that decoding can use this information.
This patch is required to allow the streaming of in-progress transactions
in logical decoding. The actual work to allow streaming will be committed
as a separate patch.
We still add the invalidations to the cache and write them to WAL at
commit time in RecordTransactionCommit(). This uses the existing
XLOG_INVALIDATIONS xlog record type, from the RM_STANDBY_ID resource
manager (see LogStandbyInvalidations for details).
So existing code relying on those invalidations (e.g. redo) does not need
to be changed.
The invalidations written at command end uses a new xlog record type
XLOG_XACT_INVALIDATIONS, from RM_XACT_ID resource manager. See
LogLogicalInvalidations for details.
These new xlog records are ignored by existing redo procedures, which
still rely on the invalidations written to commit records.
The invalidations are decoded and accumulated in top-transaction, and then
executed during replay. This obviates the need to decode the
invalidations as part of a commit record.
Bump XLOG_PAGE_MAGIC, since this introduces XLOG_XACT_INVALIDATIONS.
Author: Dilip Kumar, Tomas Vondra, Amit Kapila
Reviewed-by: Amit Kapila
Tested-by: Neha Sharma and Mahendra Singh Thalor
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/rmgrdesc/xactdesc.c
M src/backend/access/transam/xact.c
M src/backend/replication/logical/decode.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/utils/cache/inval.c
M src/include/access/xact.h
M src/include/access/xlog_internal.h
M src/include/replication/reorderbuffer.h
M src/include/utils/inval.h
Revert "Fix corner case with PGP decompression in pgcrypto"
commit : 38f60f174e3279069b5547b5f4015eb4a8492037
author : Michael Paquier <[email protected]>
date : Thu, 23 Jul 2020 08:29:08 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 23 Jul 2020 08:29:08 +0900
This reverts commit 9e10898, after finding out that buildfarm members
running SLES 15 on z390 complain on the compression and decompression
logic of the new test: pipistrelles, barbthroat and steamerduck.
Those hosts are visibly using hardware-specific changes to improve zlib
performance, requiring more investigation.
Thanks to Tom Lane for the discussion.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M contrib/pgcrypto/expected/pgp-compression.out
M contrib/pgcrypto/pgp-compress.c
M contrib/pgcrypto/sql/pgp-compression.sql
Support infinity and -infinity in the numeric data type.
commit : a57d312a7706321d850faa048a562a0c0c01b835
author : Tom Lane <[email protected]>
date : Wed, 22 Jul 2020 19:19:44 -0400
committer: Tom Lane <[email protected]>
date : Wed, 22 Jul 2020 19:19:44 -0400
Add infinities that behave the same as they do in the floating-point
data types. Aside from any intrinsic usefulness these may have,
this closes an important gap in our ability to convert floating
values to numeric and/or replace float-based APIs with numeric.
The new values are represented by bit patterns that were formerly
not used (although old code probably would take them for NaNs).
So there shouldn't be any pg_upgrade hazard.
Patch by me, reviewed by Dean Rasheed and Andrew Gierth
Discussion: https://postgr.es/m/[email protected]
M contrib/jsonb_plperl/jsonb_plperl.c
M contrib/jsonb_plpython/jsonb_plpython.c
M doc/src/sgml/datatype.sgml
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/numeric.c
M src/include/utils/numeric.h
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/numeric.out
M src/test/regress/expected/window.out
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/numeric.sql
M src/test/regress/sql/window.sql
Fix corner case with PGP decompression in pgcrypto
commit : 9e108984fb35d8f9e2c2bffa10c0034f9161e802
author : Michael Paquier <[email protected]>
date : Wed, 22 Jul 2020 14:52:23 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 22 Jul 2020 14:52:23 +0900
A compressed stream may end with an empty packet, and PGP decompression
finished before reading this empty packet in the remaining stream. This
caused a failure in pgcrypto, handling this case as corrupted data.
This commit makes sure to consume such extra data, avoiding a failure
when decompression the entire stream. This corner case was reproducible
with a data length of 16kB, and existed since its introduction in
e94dd6a. A cheap regression test is added to cover this case.
Thanks to Jeff Janes for the extra investigation.
Reported-by: Frank Gagnepain
Author: Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M contrib/pgcrypto/expected/pgp-compression.out
M contrib/pgcrypto/pgp-compress.c
M contrib/pgcrypto/sql/pgp-compression.sql
Fix conversion table generator scripts.
commit : a5073871ea655e37759f22f30c4c70359ad9759b
author : Thomas Munro <[email protected]>
date : Wed, 22 Jul 2020 16:38:20 +1200
committer: Thomas Munro <[email protected]>
date : Wed, 22 Jul 2020 16:38:20 +1200
convutils.pm used implicit conversion of undefined value to integer
zero. Some of conversion scripts are susceptible to regexp greediness.
Fix, avoiding whitespace changes in the output. Also update ICU URLs
that moved.
No need to back-patch, because the output of these scripts is also in
the source tree so we shouldn't need to rerun them on back-branches.
Author: Kyotaro Horiguchi <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJ7SEGLbj%3D%3DTQCcyKRA9aqj8%2B6L%3DexSq1y25TA%3DWxLziQ%40mail.gmail.com
M src/backend/utils/mb/Unicode/Makefile
M src/backend/utils/mb/Unicode/UCS_to_EUC_JIS_2004.pl
M src/backend/utils/mb/Unicode/UCS_to_EUC_JP.pl
M src/backend/utils/mb/Unicode/UCS_to_SHIFT_JIS_2004.pl
M src/backend/utils/mb/Unicode/convutils.pm
Fix comment in sha2.h
commit : e47c2602aa4d35a4e3eb6ada40454c6c0f1279bf
author : Michael Paquier <[email protected]>
date : Wed, 22 Jul 2020 10:16:21 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 22 Jul 2020 10:16:21 +0900
An incorrect reference to SHA-1 was present.
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/include/common/sha2.h
neqjoinsel must now pass through collation to eqjoinsel.
commit : bd0d893aa7aa303d7f344e267a9d3e53b0219491
author : Tom Lane <[email protected]>
date : Tue, 21 Jul 2020 19:40:44 -0400
committer: Tom Lane <[email protected]>
date : Tue, 21 Jul 2020 19:40:44 -0400
Since commit 044c99bc5, eqjoinsel passes the passed-in collation
to any operators it invokes. However, neqjoinsel failed to pass
on whatever collation it got, so that if we invoked a
collation-dependent operator via that code path, we'd get "could not
determine which collation to use for string comparison" or the like.
Per report from Justin Pryzby. Back-patch to v12, like the previous
commit.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/selfuncs.c
Add nbtree Valgrind buffer lock checks.
commit : 4a70f829d86cb8dbd68f561720e6329f5e818c94
author : Peter Geoghegan <[email protected]>
date : Tue, 21 Jul 2020 15:50:58 -0700
committer: Peter Geoghegan <[email protected]>
date : Tue, 21 Jul 2020 15:50:58 -0700
Holding just a buffer pin (with no buffer lock) on an nbtree buffer/page
provides very weak guarantees, especially compared to heapam, where it's
often safe to read a page while only holding a buffer pin. This commit
has Valgrind enforce the following rule: it is never okay to access an
nbtree buffer without holding both a pin and a lock on the buffer.
A draft version of this patch detected questionable code that was
cleaned up by commits fa7ff642 and 7154aa16. The code in question used
to access an nbtree buffer page's special/opaque area with no buffer
lock (only a buffer pin). This practice (which isn't obviously unsafe)
is hereby formally disallowed in nbtree. There doesn't seem to be any
reason to allow it, and banning it keeps things simple for Valgrind.
The new checks are implemented by adding custom nbtree client requests
(located in LockBuffer() wrapper functions); these requests are
"superimposed" on top of the generic bufmgr.c Valgrind client requests
added by commit 1e0dfd16. No custom resource management cleanup code is
needed to undo the effects of marking buffers as non-accessible under
this scheme.
Author: Peter Geoghegan
Reviewed-By: Anastasia Lubennikova, Georgios Kokolatos
Discussion: https://postgr.es/m/CAH2-WzkLgyN3zBvRZ1pkNJThC=xi_0gpWRUb_45eexLH1+k2_Q@mail.gmail.com
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/nbtree/nbtpage.c
M src/backend/access/nbtree/nbtree.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/storage/buffer/bufmgr.c
M src/include/access/nbtree.h
M src/include/pg_config_manual.h
Weaken type-OID-matching checks in array_recv and record_recv.
commit : 670c0a1d474bf296dbcc1d6de912d4841f2ed643
author : Tom Lane <[email protected]>
date : Tue, 21 Jul 2020 15:19:46 -0400
committer: Tom Lane <[email protected]>
date : Tue, 21 Jul 2020 15:19:46 -0400
Rather than always insisting on an exact match of the type OID in the
data to the element type or column type we expect, complain only when
both OIDs fall within the manually-assigned range. This acknowledges
the reality that user-defined types don't have stable OIDs, while
still preserving some of the mistake-detection value of the old test.
(It's not entirely clear whether to error if one OID is manually
assigned and the other isn't. But perhaps that case could arise in
cross-version cases where a former extension type has been imported
into core, so I let it pass.)
This change allows us to remove the prohibition on binary transfer
of user-defined arrays and composites in the recently-landed support
for binary logical replication (commit 9de77b545). We can just
unconditionally drop that check, since if the client has asked for
binary transfer it must be >= v14 and must have this change.
Discussion: https://postgr.es/m/CADK3HH+R3xMn=8t3Ct+uD+qJ1KD=Hbif5NFMJ+d5DkoCzp6Vgw@mail.gmail.com
M src/backend/replication/logical/proto.c
M src/backend/utils/adt/arrayfuncs.c
M src/backend/utils/adt/rowtypes.c
Glossary: Add term "base backup"
commit : 606c3845988ddd9497cbbbf6fc559b91c76ed65d
author : Alvaro Herrera <[email protected]>
date : Tue, 21 Jul 2020 13:11:23 -0400
committer: Alvaro Herrera <[email protected]>
date : Tue, 21 Jul 2020 13:11:23 -0400
Author: Jürgen Purtz <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/glossary.sgml
Minor glossary tweaks
commit : a0b2d583db9f040e2c156570b741e46bc33c6aec
author : Alvaro Herrera <[email protected]>
date : Tue, 21 Jul 2020 13:09:42 -0400
committer: Alvaro Herrera <[email protected]>
date : Tue, 21 Jul 2020 13:09:42 -0400
Add "(process)" qualifier to two terms, remove self-reference in one
term.
Author: Jürgen Purtz <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/glossary.sgml
Be more careful about marking catalog columns NOT NULL by default.
commit : fc032bed2fb809ee69eaf2fece349f65274c876f
author : Tom Lane <[email protected]>
date : Tue, 21 Jul 2020 13:03:48 -0400
committer: Tom Lane <[email protected]>
date : Tue, 21 Jul 2020 13:03:48 -0400
The bug fixed in commit 72eab84a5 would not have occurred if initdb
had a less surprising rule about which columns should be marked
NOT NULL by default. Let's make that rule be strictly that the
column must be fixed-width and its predecessors must be fixed-width
and NOT NULL, removing the hacky and unsafe exceptions for oidvector
and int2vector.
Since we do still want all existing oidvector and int2vector columns
to be marked NOT NULL, we have to put BKI_FORCE_NOT_NULL labels on
them. But making this less magic and more documented seems like a
good idea, even if it's a shade more verbose.
I didn't bump catversion since the initial catalog contents are
not actually changed by this patch. Note however that the
contents of postgres.bki do change, and feeding an old copy of
that to a new backend will produce wrong results.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/bki.sgml
M src/backend/bootstrap/bootstrap.c
M src/backend/catalog/genbki.pl
M src/include/catalog/genbki.h
M src/include/catalog/pg_index.h
M src/include/catalog/pg_partitioned_table.h
M src/include/catalog/pg_proc.h
M src/include/catalog/pg_statistic_ext.h
M src/include/catalog/pg_trigger.h
Assert that we don't insert nulls into attnotnull catalog columns.
commit : 3e66019f15549d3e1e13da665b2549684061d630
author : Tom Lane <[email protected]>
date : Tue, 21 Jul 2020 12:38:08 -0400
committer: Tom Lane <[email protected]>
date : Tue, 21 Jul 2020 12:38:08 -0400
The executor checks for this error, and so does the bootstrap catalog
loader, but we never checked for it in retail catalog manipulations.
The folly of that has now been exposed, so let's add assertions
checking it. Checking in CatalogTupleInsert[WithInfo] and
CatalogTupleUpdate[WithInfo] should be enough to cover this.
Back-patch to v10; the aforesaid functions didn't exist before that,
and it didn't seem worth adapting the patch to the oldest branches.
But given the risk of JIT crashes, I think we certainly need this
as far back as v11.
Pre-v13, we have to explicitly exclude pg_subscription.subslotname
and pg_subscription_rel.srsublsn from the checks, since they are
mismarked. (Even if we change our mind about applying BKI_FORCE_NULL
in the branch tips, it doesn't seem wise to have assertions that
would fire in existing databases.)
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/bki.sgml
M src/backend/catalog/indexing.c
Rework tab completion of COPY and \copy in psql
commit : c273d9d8ce412a475082b1729462845d2abb5ae8
author : Michael Paquier <[email protected]>
date : Tue, 21 Jul 2020 12:05:07 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 21 Jul 2020 12:05:07 +0900
This corrects and simplifies $subject in a number of ways:
- Remove from the completion the pre-9.0 grammar still supported for
compatibility purposes. This simplifies the code, and allows to extend
it more easily with new patterns.
- Add completion for the options of FORMAT within a WITH clause.
- Complete WHERE and WITH clauses correctly depending on if TO or FROM
are used, WHERE being only available with COPY FROM.
Author: Vignesh C, Michael Paquier
Reviewed-by: Ahsan Hadi
Discussion: https://postgr.es/m/CALDaNm3zWr=OmxeNqOqfT=uZTSdam_j-gkX94CL8eTNfgUtf6A@mail.gmail.com
M src/bin/psql/tab-complete.c
Fix some corner cases for window ranges with infinite offsets.
commit : a4faef8f8fe1493397679c014cf11a6e27c0f1e5
author : Tom Lane <[email protected]>
date : Mon, 20 Jul 2020 22:03:18 -0400
committer: Tom Lane <[email protected]>
date : Mon, 20 Jul 2020 22:03:18 -0400
Many situations where the offset is infinity were not handled sanely.
We should generally allow the val versus base +/- offset comparison to
proceed according to the normal rules of IEEE arithmetic; however, we
must do something special for the corner cases where base +/- offset
would produce NaN due to subtracting two like-signed infinities.
That corresponds to asking which values infinitely precede +inf or
infinitely follow -inf, which should certainly be true of any finite
value or of the opposite-signed infinity. After some discussion it
seems that the best decision is to make it true of the same-signed
infinity as well, ie, just return constant TRUE if the calculation
would produce a NaN.
(We could write this with a bit less code by subtracting anyway,
and then checking for a NaN result. However, I prefer this
formulation because it'll be easier to transpose into numeric.c.)
Although this seems like clearly a bug fix with respect to finite
values, it is less obviously correct for infinite values. Between
that and the fact that the whole issue only arises for very strange
window specifications (e.g. RANGE BETWEEN 'inf' PRECEDING AND 'inf'
PRECEDING), I'll desist from back-patching.
Noted by Dean Rasheed.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/float.c
M src/test/regress/expected/window.out
M src/test/regress/sql/window.sql
Make floating-point "NaN / 0" return NaN instead of raising an error.
commit : 4fb6aeb4f6e807c8ce3e140d2d2281f50eb6fb1a
author : Tom Lane <[email protected]>
date : Mon, 20 Jul 2020 19:44:41 -0400
committer: Tom Lane <[email protected]>
date : Mon, 20 Jul 2020 19:44:41 -0400
This is more consistent with the IEEE 754 spec and our treatment of
NaNs elsewhere; in particular, the case has always acted that way in
"numeric" arithmetic.
Noted by Dean Rasheed.
Discussion: https://postgr.es/m/[email protected]
M src/include/utils/float.h
M src/test/regress/expected/float4-misrounded-input.out
M src/test/regress/expected/float4.out
M src/test/regress/expected/float8.out
M src/test/regress/sql/float4.sql
M src/test/regress/sql/float8.sql
Assert that buffer is pinned in LockBuffer().
commit : 6ca7cd89a2d1998b16e8168dda62d43a9e0fdaff
author : Peter Geoghegan <[email protected]>
date : Mon, 20 Jul 2020 16:03:38 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 20 Jul 2020 16:03:38 -0700
Strengthen the LockBuffer() assertion that verifies BufferIsValid() by
making it verify BufferIsPinned() instead. Do the same in nearby
related functions.
There is probably not much chance that anybody will try to lock a buffer
that is not already pinned, but we might as well make sure of that.
M src/backend/storage/buffer/bufmgr.c
Correctly mark pg_subscription_rel.srsublsn as nullable.
commit : 0fa0b487b5d75d2b8576dec86a317212542642b8
author : Tom Lane <[email protected]>
date : Mon, 20 Jul 2020 14:55:56 -0400
committer: Tom Lane <[email protected]>
date : Mon, 20 Jul 2020 14:55:56 -0400
The code has always set this column to NULL when it's not valid,
but the catalog header's description failed to reflect that,
as did the SGML docs, as did some of the code. To prevent future
coding errors of the same ilk, let's hide the field from C code
as though it were variable-length (which, in a sense, it is).
As with commit 72eab84a5, we can only fix this cleanly in HEAD
and v13; the problem extends further back but we'll need some
klugery in the released branches.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/pg_subscription.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_subscription_rel.h
Fix construction of updated-columns bitmap in logical replication.
commit : d5daae47db5e8a61ce6ed7afaa3e3a99af108c06
author : Tom Lane <[email protected]>
date : Mon, 20 Jul 2020 13:40:16 -0400
committer: Tom Lane <[email protected]>
date : Mon, 20 Jul 2020 13:40:16 -0400
Commit b9c130a1f failed to apply the publisher-to-subscriber column
mapping while checking which columns were updated. Perhaps less
significantly, it didn't exclude dropped columns either. This could
result in an incorrect updated-columns bitmap and thus wrong decisions
about whether to fire column-specific triggers on the subscriber while
applying updates. In HEAD (since commit 9de77b545), it could also
result in accesses off the end of the colstatus array, as detected by
buildfarm member skink. Fix the logic, and adjust 003_constraints.pl
so that the problem is exposed in unpatched code.
In HEAD, also add some assertions to check that we don't access off
the ends of these newly variable-sized arrays.
Back-patch to v10, as b9c130a1f was.
Discussion: https://postgr.es/m/CAH2-Wz=79hKQ4++c5A060RYbjTHgiYTHz=fw6mptCtgghH2gJA@mail.gmail.com
M src/backend/replication/logical/proto.c
M src/backend/replication/logical/worker.c
M src/include/replication/logicalproto.h
M src/test/subscription/t/003_constraints.pl
Update btree_gist extension for parallel query
commit : d98c08cdc638a290d80fb69b20b8288c3924d875
author : Alexander Korotkov <[email protected]>
date : Mon, 20 Jul 2020 13:59:50 +0300
committer: Alexander Korotkov <[email protected]>
date : Mon, 20 Jul 2020 13:59:50 +0300
All functions provided by this extension are PARALLEL SAFE.
Discussion: https://postgr.es/m/AM5PR0901MB1587E47B1ACF23C6089DFCA3FD9B0%40AM5PR0901MB1587.eurprd09.prod.outlook.com
Author: Steven Winfield
M contrib/btree_gist/Makefile
A contrib/btree_gist/btree_gist–1.5–1.6.sql
M contrib/btree_gist/btree_gist.control
Rename wal_keep_segments to wal_keep_size.
commit : c3fe108c025e4a080315562d4c15ecbe3f00405e
author : Fujii Masao <[email protected]>
date : Mon, 20 Jul 2020 13:30:18 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 20 Jul 2020 13:30:18 +0900
max_slot_wal_keep_size that was added in v13 and wal_keep_segments are
the GUC parameters to specify how much WAL files to retain for
the standby servers. While max_slot_wal_keep_size accepts the number of
bytes of WAL files, wal_keep_segments accepts the number of WAL files.
This difference of setting units between those similar parameters could
be confusing to users.
To alleviate this situation, this commit renames wal_keep_segments to
wal_keep_size, and make users specify the WAL size in it instead of
the number of WAL files.
There was also the idea to rename max_slot_wal_keep_size to
max_slot_wal_keep_segments, in the discussion. But we have been moving
away from measuring in segments, for example, checkpoint_segments was
replaced by max_wal_size. So we concluded to rename wal_keep_segments
to wal_keep_size.
Back-patch to v13 where max_slot_wal_keep_size was added.
Author: Fujii Masao
Reviewed-by: Álvaro Herrera, Kyotaro Horiguchi, David Steele
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/xlog.c
M src/backend/replication/slotfuncs.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/pg_rewind/t/RewindTest.pm
M src/include/access/xlog.h
M src/test/recovery/t/019_replslot_limit.pl
Immediately WAL-log subtransaction and top-level XID association.
commit : 0bead9af484c1d0a67e690fda47011addaa5bc9d
author : Amit Kapila <[email protected]>
date : Mon, 20 Jul 2020 08:48:26 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 20 Jul 2020 08:48:26 +0530
The logical decoding infrastructure needs to know which top-level
transaction the subxact belongs to, in order to decode all the
changes. Until now that might be delayed until commit, due to the
caching (GPROC_MAX_CACHED_SUBXIDS), preventing features requiring
incremental decoding.
So we also write the assignment info into WAL immediately, as part
of the next WAL record (to minimize overhead) only when wal_level=logical.
We can not remove the existing XLOG_XACT_ASSIGNMENT WAL as that is
required for avoiding overflow in the hot standby snapshot.
Bump XLOG_PAGE_MAGIC, since this introduces XLR_BLOCK_ID_TOPLEVEL_XID.
Author: Tomas Vondra, Dilip Kumar, Amit Kapila
Reviewed-by: Amit Kapila
Tested-by: Neha Sharma and Mahendra Singh Thalor
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xact.c
M src/backend/access/transam/xloginsert.c
M src/backend/access/transam/xlogreader.c
M src/backend/replication/logical/decode.c
M src/include/access/xact.h
M src/include/access/xlog.h
M src/include/access/xlog_internal.h
M src/include/access/xlogreader.h
M src/include/access/xlogrecord.h
Add generic_plans and custom_plans fields into pg_prepared_statements.
commit : d05b172a760e0ccb3008a2144f96053720000b12
author : Fujii Masao <[email protected]>
date : Mon, 20 Jul 2020 11:55:50 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 20 Jul 2020 11:55:50 +0900
There was no easy way to find how many times generic and custom plans
have been executed for a prepared statement. This commit exposes those
numbers of times in pg_prepared_statements view.
Author: Atsushi Torikoshi, Kyotaro Horiguchi
Reviewed-by: Tatsuro Yamada, Masahiro Ikeda, Fujii Masao
Discussion: https://postgr.es/m/CACZ0uYHZ4M=NZpofH6JuPHeX=__5xcDELF8hT8_2T+R55w4RQw@mail.gmail.com
M doc/src/sgml/catalogs.sgml
M src/backend/commands/prepare.c
M src/backend/utils/cache/plancache.c
M src/include/catalog/pg_proc.dat
M src/include/utils/plancache.h
M src/test/regress/expected/plancache.out
M src/test/regress/expected/rules.out
M src/test/regress/sql/plancache.sql
Fix minor typo in nodeIncrementalSort.c.
commit : 044dc7b964147ec6303d59320fb743693b22af30
author : Amit Kapila <[email protected]>
date : Mon, 20 Jul 2020 07:45:26 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 20 Jul 2020 07:45:26 +0530
Author: Vignesh C
Reviewed-by: James Coleman
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CALDaNm0WjZqRvdeL59ZfYH0o4mLbKQ23jm-bnjXcFzgpANx55g@mail.gmail.com
M src/backend/executor/nodeIncrementalSort.c
Avoid harmless Valgrind no-buffer-pin errors.
commit : a766d6ca22ac7c233e69c896ae0c5f19de916db4
author : Peter Geoghegan <[email protected]>
date : Sun, 19 Jul 2020 16:12:51 -0700
committer: Peter Geoghegan <[email protected]>
date : Sun, 19 Jul 2020 16:12:51 -0700
Valgrind builds with assertions enabled sometimes perform a
theoretically unsafe page access inside an assertion in
heapam_tuple_lock(). This happened when the eval-plan-qual isolation
test ran one of the permutations added by commit a2418f9e238.
Avoid complaints from Valgrind by moving the assertion ever so slightly.
This is minor cleanup for commit 1e0dfd16, which added Valgrind buffer
access instrumentation.
No backpatch, since this only happens within an assertion, and seems
very unlikely to cause any real problems even with assert-enabled
builds.
M src/backend/access/heap/heapam_handler.c
Mark buffers as defined to Valgrind consistently.
commit : 46ef520b9566d9eccb095ceafa53e5c305cf80b9
author : Peter Geoghegan <[email protected]>
date : Sun, 19 Jul 2020 09:46:44 -0700
committer: Peter Geoghegan <[email protected]>
date : Sun, 19 Jul 2020 09:46:44 -0700
Make PinBuffer() mark buffers as defined to Valgrind unconditionally,
including when the buffer header spinlock must be acquired. Failure to
handle that case could lead to false positive reports from Valgrind.
This theoretically creates a risk that we'll mark buffers defined even
when external callers don't end up with a buffer pin. That seems
perfectly acceptable, though, since in general we make no guarantees
about buffers that are unsafe to access being reliably marked as unsafe.
Oversight in commit 1e0dfd16, which added valgrind buffer access
instrumentation.
M src/backend/storage/buffer/bufmgr.c
Correctly mark pg_subscription.subslotname as nullable.
commit : 72eab84a565cbc0677bf8907cd4bfaddf064bd64
author : Tom Lane <[email protected]>
date : Sun, 19 Jul 2020 12:37:23 -0400
committer: Tom Lane <[email protected]>
date : Sun, 19 Jul 2020 12:37:23 -0400
Due to the layout of this catalog, subslotname has to be explicitly
marked BKI_FORCE_NULL, else initdb will default to the assumption
that it's non-nullable. Since, in fact, CREATE/ALTER SUBSCRIPTION
will store null values there, the existing marking is just wrong,
and has been since this catalog was invented.
We haven't noticed because not much in the system actually depends
on attnotnull being truthful. However, JIT'ed tuple deconstruction
does depend on that in some cases, allowing crashes or wrong answers
in queries that inspect pg_subscription. Commit 9de77b545 quite
accidentally exposed this on the buildfarm members that force JIT
activation.
Back-patch to v13. The problem goes further back, but we cannot
force initdb in released branches, so some klugier solution will
be needed there. Before working on that, push this simple fix
to try to get the buildfarm back to green.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M src/include/catalog/catversion.h
M src/include/catalog/pg_subscription.h
Define OPENSSL_API_COMPAT
commit : 4d3db13621be64fbac2faf7c01c4879d20885c1b
author : Peter Eisentraut <[email protected]>
date : Sun, 19 Jul 2020 12:14:42 +0200
committer: Peter Eisentraut <[email protected]>
date : Sun, 19 Jul 2020 12:14:42 +0200
This avoids deprecation warnings from newer OpenSSL versions (3.0.0 in
particular).
Discussion: https://www.postgresql.org/message-id/flat/FEF81714-D479-4512-839B-C769D2605F8A%40yesql.se
M configure
M configure.in
M src/include/pg_config.h.in
M src/tools/msvc/Solution.pm
Fix replication/worker_internal.h to compile without other headers.
commit : 9b14280b207053f8b09e20b5719785ec4413b9d2
author : Tom Lane <[email protected]>
date : Sat, 18 Jul 2020 14:58:18 -0400
committer: Tom Lane <[email protected]>
date : Sat, 18 Jul 2020 14:58:18 -0400
This header hasn't changed recently, so the fact that it now fails
headerscheck/cpluspluscheck testing must be due to changes in what
it includes. Probably f21916791 is to blame, but I didn't try to
verify that.
Discussion: https://postgr.es/m/[email protected]
M src/include/replication/worker_internal.h
Allow logical replication to transfer data in binary format.
commit : 9de77b5453130242654ff0b30a551c9c862ed661
author : Tom Lane <[email protected]>
date : Sat, 18 Jul 2020 12:44:51 -0400
committer: Tom Lane <[email protected]>
date : Sat, 18 Jul 2020 12:44:51 -0400
This patch adds a "binary" option to CREATE/ALTER SUBSCRIPTION.
When that's set, the publisher will send data using the data type's
typsend function if any, rather than typoutput. This is generally
faster, if slightly less robust.
As committed, we won't try to transfer user-defined array or composite
types in binary, for fear that type OIDs won't match at the subscriber.
This might be changed later, but it seems like fit material for a
follow-on patch.
Dave Cramer, reviewed by Daniel Gustafsson, Petr Jelinek, and others;
adjusted some by me
Discussion: https://postgr.es/m/CADK3HH+R3xMn=8t3Ct+uD+qJ1KD=Hbif5NFMJ+d5DkoCzp6Vgw@mail.gmail.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/ref/alter_subscription.sgml
M doc/src/sgml/ref/create_subscription.sgml
M src/backend/catalog/pg_subscription.c
M src/backend/catalog/system_views.sql
M src/backend/commands/subscriptioncmds.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/proto.c
M src/backend/replication/logical/worker.c
M src/backend/replication/pgoutput/pgoutput.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/psql/describe.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_subscription.h
M src/include/replication/logicalproto.h
M src/include/replication/pgoutput.h
M src/include/replication/walreceiver.h
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
A src/test/subscription/t/014_binary.pl
doc: Refresh more URLs in the docs
commit : 9add405014f8e47e038af7124528b7601249a2ac
author : Michael Paquier <[email protected]>
date : Sat, 18 Jul 2020 22:43:35 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 18 Jul 2020 22:43:35 +0900
This updates some URLs that are redirections, mostly to an equivalent
using https. One URL referring to generalized partial indexes was
outdated.
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/acronyms.sgml
M doc/src/sgml/biblio.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/cube.sgml
M doc/src/sgml/dfunc.sgml
M doc/src/sgml/earthdistance.sgml
M doc/src/sgml/external-projects.sgml
M doc/src/sgml/geqo.sgml
M doc/src/sgml/install-windows.sgml
M doc/src/sgml/intro.sgml
M doc/src/sgml/isn.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/nls.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/plperl.sgml
M doc/src/sgml/pltcl.sgml
M doc/src/sgml/seg.sgml
M doc/src/sgml/textsearch.sgml
Adjust minor comment in reorderbuffer.c.
commit : f41fbee7e704947fd1d2241f35bc8d0e37705919
author : Amit Kapila <[email protected]>
date : Sat, 18 Jul 2020 09:57:23 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 18 Jul 2020 09:57:23 +0530
Author: Dave Cramer
Reviewed-by: David G. Johnston
Discussion: https://postgr.es/m/CADK3HHL8do4Fp1bsymgNasx375njV3AR7zY3UgYwzbL_Dx-n2Q@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
Fix comments in reorderbuffer.c.
commit : df7c5cb16e8fcf960e3302355fa6547fba428f5e
author : Amit Kapila <[email protected]>
date : Sat, 18 Jul 2020 09:47:38 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 18 Jul 2020 09:47:38 +0530
Author: Dave Cramer
Reviewed-by: David G. Johnston
Discussion: https://postgr.es/m/CADK3HHL8do4Fp1bsymgNasx375njV3AR7zY3UgYwzbL_Dx-n2Q@mail.gmail.com
M src/backend/replication/logical/reorderbuffer.c
doc: Fix description of \copy for psql
commit : b74d449a02b3c972051b1847f3915128da8775dc
author : Michael Paquier <[email protected]>
date : Sat, 18 Jul 2020 10:42:41 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 18 Jul 2020 10:42:41 +0900
The WHERE clause introduced by 31f3817 was not described. While on it,
split the grammar of \copy FROM and TO into two distinct parts for
clarity as they support different set of options.
Author: Vignesh C
Discussion: https://postgr.es/m/CALDaNm3zWr=OmxeNqOqfT=uZTSdam_j-gkX94CL8eTNfgUtf6A@mail.gmail.com
Backpatch-through: 12
M doc/src/sgml/ref/psql-ref.sgml
Rename "hash_mem" local variable.
commit : 564ce62164cd57c0311752ae07ec10439c78d599
author : Peter Geoghegan <[email protected]>
date : Fri, 17 Jul 2020 18:24:23 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 17 Jul 2020 18:24:23 -0700
The term "hash_mem" will take on new significance when pending work to
add a new hash_mem_multiplier GUC is committed. Rename a local variable
that happens to have been called hash_mem now to avoid confusion.
M src/backend/executor/nodeAgg.c
Add Valgrind buffer access instrumentation.
commit : 1e0dfd166b3fa7fc79e4fad73b6fae056bab598a
author : Peter Geoghegan <[email protected]>
date : Fri, 17 Jul 2020 17:49:45 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 17 Jul 2020 17:49:45 -0700
Teach Valgrind memcheck to maintain the "defined-ness" of each shared
buffer based on whether the backend holds at least one pin at the point
it is accessed by access method code. Bugs like the one fixed by commit
b0229f26 can be detected using this new instrumentation.
Note that backends running with Valgrind naturally have their own
independent ideas about whether any given byte in shared memory is safe
or unsafe to access. There is no risk that concurrent access by
multiple backends to the same shared memory will confuse Valgrind's
instrumentation, because everything already works at the process level
(or at the memory mapping level, if you prefer).
Author: Álvaro Herrera, Peter Geoghegan
Reviewed-By: Anastasia Lubennikova
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CAH2-WzkLgyN3zBvRZ1pkNJThC=xi_0gpWRUb_45eexLH1+k2_Q@mail.gmail.com
M src/backend/storage/buffer/bufmgr.c
M src/include/pg_config_manual.h
Cope with data-offset-less archive files during out-of-order restores.
commit : f009591d6eddbeece955aab70c35e7002fd01aec
author : Tom Lane <[email protected]>
date : Fri, 17 Jul 2020 13:03:50 -0400
committer: Tom Lane <[email protected]>
date : Fri, 17 Jul 2020 13:03:50 -0400
pg_dump produces custom-format archive files that lack data offsets
when it is unable to seek its output. Up to now that's been a hazard
for pg_restore. But if pg_restore is able to seek in the archive
file, there is no reason to throw up our hands when asked to restore
data blocks out of order. Instead, whenever we are searching for a
data block, record the locations of the blocks we passed over (that
is, fill in the missing data-offset fields in our in-memory copy of
the TOC data). Then, when we hit a case that requires going
backwards, we can just seek back.
Also track the furthest point that we've searched to, and seek back
to there when beginning a search for a new data block. This avoids
possible O(N^2) time consumption, by ensuring that each data block
is examined at most twice. (On Unix systems, that's at most twice
per parallel-restore job; but since Windows uses threads here, the
threads can share block location knowledge, reducing the amount of
duplicated work.)
We can also improve the code a bit by using fseeko() to skip over
data blocks during the search.
This is all of some use even in simple restores, but it's really
significant for parallel pg_restore. In that case, we require
seekability of the input already, and we will very probably need
to do out-of-order restores.
Back-patch to v12, as this fixes a regression introduced by commit
548e50976. Before that, parallel restore avoided requesting
out-of-order restores, so it would work on a data-offset-less
archive. Now it will again.
Ideally this patch would include some test coverage, but there are
other open bugs that need to be fixed before we can extend our
coverage of parallel restore very much. Plan to revisit that later.
David Gilman and Tom Lane; reviewed by Justin Pryzby
Discussion: https://postgr.es/m/CALBH9DDuJ+scZc4MEvw5uO-=vRyR2=QF9+Yh=3hPEnKHWfS81A@mail.gmail.com
M doc/src/sgml/ref/pg_restore.sgml
M src/bin/pg_dump/pg_backup_custom.c
Remove manual tracking of file position in pg_dump/pg_backup_custom.c.
commit : a8d0732ac2b5527ce47cce5b325f8df93f4d19cc
author : Tom Lane <[email protected]>
date : Fri, 17 Jul 2020 12:14:28 -0400
committer: Tom Lane <[email protected]>
date : Fri, 17 Jul 2020 12:14:28 -0400
We do not really need to track the file position by hand. We were
already relying on ftello() whenever the archive file is seekable,
while if it's not seekable we don't need the file position info
anyway because we're not going to be able to re-write the TOC.
Moreover, that tracking was buggy since it failed to account for
the effects of fseeko(). Somewhat remarkably, that seems not to
have made for any live bugs up to now. We could fix the oversights,
but it seems better to just get rid of the whole error-prone mess.
In itself this is merely code cleanup. However, it's necessary
infrastructure for an upcoming bug-fix patch (because that code
*does* need valid file position after fseeko). The bug fix
needs to go back as far as v12; hence, back-patch that far.
Discussion: https://postgr.es/m/CALBH9DDuJ+scZc4MEvw5uO-=vRyR2=QF9+Yh=3hPEnKHWfS81A@mail.gmail.com
M src/bin/pg_dump/pg_backup_custom.c
Avoid CREATE INDEX unique index deduplication.
commit : 5da8bf8bbb5c119d4bd767dbdfaf10efd348c0fd
author : Peter Geoghegan <[email protected]>
date : Fri, 17 Jul 2020 09:50:48 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 17 Jul 2020 09:50:48 -0700
There is no advantage to attempting deduplication for a unique index
during CREATE INDEX, since there cannot possibly be any duplicates.
Doing so wastes cycles due to unnecessary copying. Make sure that we
avoid it consistently.
We already avoided unique index deduplication in the case where there
were some spool2 tuples to merge. That didn't account for the fact that
spool2 is removed early/unset in the common case where it has no tuples
that need to be merged (i.e. it failed to account for the "spool2 turns
out to be unnecessary" optimization in _bt_spools_heapscan()).
Oversight in commit 0d861bbb, which added nbtree deduplication
Backpatch: 13-, where nbtree deduplication was introduced.
M src/backend/access/nbtree/nbtsort.c
Ensure that distributed timezone abbreviation files are plain ASCII.
commit : 7fe3083f4cc9cb213f99deecf1bf775a9270b3b2
author : Tom Lane <[email protected]>
date : Fri, 17 Jul 2020 11:03:55 -0400
committer: Tom Lane <[email protected]>
date : Fri, 17 Jul 2020 11:03:55 -0400
We had two occurrences of "Mitteleuropäische Zeit" in Europe.txt,
though the corresponding entries in Default were spelled
"Mitteleuropaeische Zeit". Standardize on the latter spelling to
avoid questions of which encoding to use.
While here, correct a couple of other trivial inconsistencies between
the Default file and the supposedly-matching entries in the *.txt
files, as exposed by some checking with comm(1). Also, add BDST to
the Europe.txt file; it previously was only listed in Default.
None of this has any direct functional effect.
Per complaint from Christoph Berg. As usual for timezone data patches,
apply to all branches.
Discussion: https://postgr.es/m/[email protected]
M src/timezone/tznames/Antarctica.txt
M src/timezone/tznames/Australia.txt
M src/timezone/tznames/Default
M src/timezone/tznames/Europe.txt
Fix whitespace
commit : 20ef35516328350a244d22b590e37f24ea958526
author : Peter Eisentraut <[email protected]>
date : Fri, 17 Jul 2020 15:16:13 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 17 Jul 2020 15:16:13 +0200
M src/backend/utils/adt/genfile.c
Resolve gratuitous tabs in SQL file
commit : 44f34365b81fd1be575d212593c2bd9c47060aa6
author : Peter Eisentraut <[email protected]>
date : Fri, 17 Jul 2020 15:07:54 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 17 Jul 2020 15:07:54 +0200
M src/backend/catalog/system_views.sql
Fix signal handler setup for SIGHUP in the apply launcher process.
commit : 01160a3de3d89346c9597414ff85416b77f5391f
author : Amit Kapila <[email protected]>
date : Fri, 17 Jul 2020 09:02:44 +0530
committer: Amit Kapila <[email protected]>
date : Fri, 17 Jul 2020 09:02:44 +0530
Commit 1e53fe0e70 has unified the usage of the config-file reload flag by
using the same signal handler function for the SIGHUP signal at many places
in the code. By mistake, it used the wrong SIGNAL in apply launcher
process for the SIGHUP signal handler function.
Author: Bharath Rupireddy
Reviewed-by: Dilip Kumar
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CALj2ACVzHCRnS20bOiEHaLtP5PVBENZQn4khdsSJQgOv_GM-LA@mail.gmail.com
M src/backend/replication/logical/launcher.c
Use MinimalTuple for tuple queues.
commit : cdc7169509113018cc389da740e950c587b5751f
author : Thomas Munro <[email protected]>
date : Fri, 17 Jul 2020 14:57:50 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 17 Jul 2020 14:57:50 +1200
This representation saves 8 bytes per tuple compared to HeapTuple, and
avoids the need to allocate, copy and free on the receiving side.
Gather can emit the returned MinimalTuple directly, but GatherMerge now
needs to make an explicit copy because it buffers multiple tuples at a
time. That should be no worse than before.
Reviewed-by: Soumyadeep Chakraborty <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKG%2B8T_ggoUTAE-U%3DA%2BOcPc4%3DB0nPPHcSfffuQhvXXjML6w%40mail.gmail.com
M src/backend/executor/nodeGather.c
M src/backend/executor/nodeGatherMerge.c
M src/backend/executor/tqueue.c
M src/backend/optimizer/plan/createplan.c
M src/include/executor/tqueue.h
Add huge_page_size setting for use on Linux.
commit : d2bddc2500fb74d56e5bc53a1cfa269e2e846510
author : Thomas Munro <[email protected]>
date : Fri, 17 Jul 2020 14:33:00 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 17 Jul 2020 14:33:00 +1200
This allows the huge page size to be set explicitly. The default is 0,
meaning it will use the system default, as before.
Author: Odin Ugedal <[email protected]>
Discussion: https://postgr.es/m/20200608154639.20254-1-odin%40ugedal.com
M doc/src/sgml/config.sgml
M doc/src/sgml/runtime.sgml
M src/backend/port/sysv_shmem.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/storage/pg_shmem.h
Enable almost all TAP tests involving symlinks on Windows
commit : d66b23b032d75614e1be47ca182020960d89206d
author : Andrew Dunstan <[email protected]>
date : Thu, 16 Jul 2020 14:48:37 -0400
committer: Andrew Dunstan <[email protected]>
date : Thu, 16 Jul 2020 14:48:37 -0400
Windows has junction points which function as symbolic links for
directories. This patch introduces a new function TestLib::dir_symlink()
which creates a junction point on Windows and a standard Unix type
symbolic link elsewhere.
The function TestLib::perl2host is also modified, first to use cygpath
where it's available (e.g. msys2) and second to allow it to succeed if
the gandparent directory exists but the parent does not.
Given these changes the only symlink tests that need to be skipped on
Windows are those related to permissions or to use of readlink. The
relevant tests for pg_basebackup and pg_rewind are therefore adjusted
accordingly.
Andrew Dunstan, reviewed by Peter Eisentraut and Michael Paquier.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_dump/t/010_dump_connstr.pl
M src/bin/pg_rewind/t/004_pg_xlog_symlink.pl
M src/test/perl/TestLib.pm
Switch pg_test_fsync to use binary mode on Windows
commit : 932f9fb504a57f296cf698d15bd93462ddfe2776
author : Michael Paquier <[email protected]>
date : Thu, 16 Jul 2020 15:52:37 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 16 Jul 2020 15:52:37 +0900
pg_test_fsync has always opened files using the text mode on Windows, as
this is the default mode used if not enforced by _setmode().
This fixes a failure when running pg_test_fsync down to 12 because
O_DSYNC and the text mode are not able to work together nicely. We
fixed the handling of O_DSYNC in 12~ for the tool by switching to the
concurrent-safe version of fopen() in src/port/ with 0ba06e0. And
40cfe86, by enforcing the text mode for compatibility reasons if O_TEXT
or O_BINARY are not specified by the caller, broke pg_test_fsync. For
all versions, this avoids any translation overhead, and pg_test_fsync
should test binary writes, so it is a gain in all cases.
Note that O_DSYNC is still not handled correctly in ~11, leading to
pg_test_fsync to show insanely high numbers for open_datasync() (using
this property it is easy to notice that the binary mode is much
faster). This would require a backpatch of 0ba06e0 and 40cfe86, which
could potentially break existing applications, so this is left out.
There are no TAP tests for this tool yet, so I have checked all builds
manually using MSVC. We could invent a new option to run a single
transaction instead of using a duration of 1s to make the tests a
maximum short, but this is left as future work.
Thanks to Bruce Momjian for the discussion.
Reported-by: Jeff Janes
Author: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M src/bin/pg_test_fsync/pg_test_fsync.c
pg_dump: Reorganize dumpFunc() and dumpAgg()
commit : ed2c7f65bd9f15f8f7cd21ad61602f983b1e72e9
author : Peter Eisentraut <[email protected]>
date : Wed, 15 Jul 2020 14:23:22 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 15 Jul 2020 14:23:22 +0200
Similar to daa9fe8a5264a3f192efa5ddee8fb011ad9da365, instead of
repeating the almost same large query in each version branch, use one
query and add a few columns to the SELECT list depending on the
version. This saves a lot of duplication.
Reviewed-by: Fabien COELHO <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/bin/pg_dump/pg_dump.c
Fix handling of missing files when using pg_rewind with online source
commit : 1d09fb1f0a9c7edc3298104fc4350e49169f6f2a
author : Michael Paquier <[email protected]>
date : Wed, 15 Jul 2020 15:17:23 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 15 Jul 2020 15:17:23 +0900
When working with an online source cluster, pg_rewind gets a list of all
the files in the source data directory using a WITH RECURSIVE query,
returning a NULL result for a file's metadata if it gets removed between
the moment it is listed in a directory and the moment its metadata is
obtained with pg_stat_file() (say a recycled WAL segment). The query
result was processed in such a way that for each tuple we checked only
that the first file's metadata was NULL. This could have two
consequences, both resulting in a failure of the rewind:
- If the first tuple referred to a removed file, all files from the
source would be ignored.
- Any file actually missing would not be considered as such.
While on it, rework slightly the code so as no values are saved if we
know that a file is going to be skipped.
Issue introduced by b36805f, so backpatch down to 9.5.
Author: Justin Pryzby, Michael Paquier
Reviewed-by: Daniel Gustafsson, Masahiko Sawada
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M src/bin/pg_rewind/libpq_fetch.c
Fix compilation failure with sepgsql
commit : e949137397383de58f0c566f5a92b017343c5269
author : Michael Paquier <[email protected]>
date : Wed, 15 Jul 2020 09:42:21 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 15 Jul 2020 09:42:21 +0900
One change for getObjectIdentity() has been missed in 2a10fdc, causing
the module to not compile properly. This was actually the only problem,
and it happens that it is easy enough to check the compilation of the
module on Debian after installing libselinux1-dev.
Per buildfarm member rhinoceros.
M contrib/sepgsql/relation.c
Eliminate cache lookup errors in SQL functions for object addresses
commit : 2a10fdc4307a667883f7a3369cb93a721ade9680
author : Michael Paquier <[email protected]>
date : Wed, 15 Jul 2020 09:03:10 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 15 Jul 2020 09:03:10 +0900
When using the following functions, users could see various types of
errors of the type "cache lookup failed for OID XXX" with elog(), that
can only be used for internal errors:
* pg_describe_object()
* pg_identify_object()
* pg_identify_object_as_address()
The set of APIs managing object addresses for all object types are made
smarter by gaining a new argument "missing_ok" that allows any caller to
control if an error is raised or not on an undefined object. The SQL
functions listed above are changed to handle the case where an object is
missing.
Regression tests are added for all object types for the cases where
these are undefined. Before this commit, these cases failed with cache
lookup errors, and now they basically return NULL (minus the name of the
object type requested).
Author: Michael Paquier
Reviewed-by: Aleksander Alekseev, Dmitry Dolgov, Daniel Gustafsson,
Álvaro Herrera, Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAB7nPqSZxrSmdHK-rny7z8mi=EAFXJ5J-0RbzDw6aus=wB5azQ@mail.gmail.com
M contrib/sepgsql/database.c
M contrib/sepgsql/dml.c
M contrib/sepgsql/label.c
M contrib/sepgsql/proc.c
M contrib/sepgsql/relation.c
M contrib/sepgsql/schema.c
M doc/src/sgml/func.sgml
M src/backend/catalog/dependency.c
M src/backend/catalog/objectaddress.c
M src/backend/catalog/pg_depend.c
M src/backend/catalog/pg_shdepend.c
M src/backend/commands/event_trigger.c
M src/backend/commands/extension.c
M src/backend/commands/tablecmds.c
M src/backend/utils/adt/regproc.c
M src/include/catalog/objectaddress.h
M src/include/utils/regproc.h
M src/test/regress/expected/object_address.out
M src/test/regress/sql/object_address.sql
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
commit : 689696c7110f148ede8004aae50d7543d05b5587
author : Tom Lane <[email protected]>
date : Tue, 14 Jul 2020 18:56:49 -0400
committer: Tom Lane <[email protected]>
date : Tue, 14 Jul 2020 18:56:49 -0400
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/util/pathnode.c
M src/test/regress/expected/partition_join.out
M src/test/regress/sql/partition_join.sql
Fix -Wcast-function-type warnings
commit : de8feb1f3a23465b5737e8a8c160e8ca62f61339
author : Peter Eisentraut <[email protected]>
date : Tue, 14 Jul 2020 19:36:30 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 14 Jul 2020 19:36:30 +0200
Three groups of issues needed to be addressed:
load_external_function() and related functions returned PGFunction,
even though not necessarily all callers are looking for a function of
type PGFunction. Since these functions are really just wrappers
around dlsym(), change to return void * just like dlsym().
In dynahash.c, we are using strlcpy() where a function with a
signature like memcpy() is expected. This should be safe, as the new
comment there explains, but the cast needs to be augmented to avoid
the warning.
In PL/Python, methods all need to be cast to PyCFunction, per Python
API, but this now runs afoul of these warnings. (This issue also
exists in core CPython.)
To fix the second and third case, we add a new type pg_funcptr_t that
is defined specifically so that gcc accepts it as a special function
pointer that can be cast to any other function pointer without the
warning.
Also add -Wcast-function-type to the standard warning flags, subject
to configure check.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/1e97628e-6447-b4fd-e230-d109cec2d584%402ndquadrant.com
M configure
M configure.in
M src/backend/utils/fmgr/dfmgr.c
M src/backend/utils/hash/dynahash.c
M src/include/c.h
M src/include/fmgr.h
M src/pl/plpython/plpy_plpymodule.c
Add comment to explain an unused function parameter
commit : 101f903e51f52bf595cd8177d2e0bc6fe9000762
author : David Rowley <[email protected]>
date : Tue, 14 Jul 2020 17:29:52 +1200
committer: David Rowley <[email protected]>
date : Tue, 14 Jul 2020 17:29:52 +1200
Removing the unused 'miinfo' parameter has been raised a couple of times
now. It was decided in the 2nd discussion below that we're going to leave
it alone. It seems like it might be useful to add a comment to mention
this fact so that nobody wastes any time in the future proposing its
removal again.
Discussion: https://postgr.es/m/CAApHDvpCf-qR5HC1rXskUM4ToV+3YDb4-n1meY=vpAHsRS_1PA@mail.gmail.com
Discussion: https://postgr.es/m/CAE9k0P%3DFvcDswnSVtRpSyZMpcAWC%3DGp%3DifZ0HdfPaRQ%3D__LBtw%40mail.gmail.com
M src/backend/commands/copy.c
Fix timing issue with ALTER TABLE's validate constraint
commit : f1fcf2d3b2e00b3d7ad3e0d2b1d26b77f5a48413
author : David Rowley <[email protected]>
date : Tue, 14 Jul 2020 16:54:47 +1200
committer: David Rowley <[email protected]>
date : Tue, 14 Jul 2020 16:54:47 +1200
An ALTER TABLE to validate a foreign key in which another subcommand
already caused a pending table rewrite could fail due to ALTER TABLE
attempting to validate the foreign key before the actual table rewrite
takes place. This situation could result in an error such as:
ERROR: could not read block 0 in file "base/nnnnn/nnnnn": read only 0 of 8192 bytes
The failure here was due to the SPI call which validates the foreign key
trying to access an index which is yet to be rebuilt.
Similarly, we also incorrectly tried to validate CHECK constraints before
the heap had been rewritten.
The fix for both is to delay constraint validation until phase 3, after
the table has been rewritten. For CHECK constraints this means a slight
behavioral change. Previously ALTER TABLE VALIDATE CONSTRAINT on
inheritance tables would be validated from the bottom up. This was
different from the order of evaluation when a new CHECK constraint was
added. The changes made here aligns the VALIDATE CONSTRAINT evaluation
order for inheritance tables to be the same as ADD CONSTRAINT, which is
generally top-down.
Reported-by: Nazli Ugur Koyluoglu, using SQLancer
Discussion: https://postgr.es/m/CAApHDvp%3DZXv8wiRyk_0rWr00skhGkt8vXDrHJYXRMft3TjkxCA%40mail.gmail.com
Backpatch-through: 9.5 (all supported versions)
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
Fix some header identifications
commit : b8401c32bacd0869bd00bc71fd64f1d6071bcd6a
author : Michael Paquier <[email protected]>
date : Tue, 14 Jul 2020 13:39:45 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 14 Jul 2020 13:39:45 +0900
The following header files missed the shot:
- jsonfuncs.h, as of ce0425b.
- jsonapi.h, as of beb4699.
- llvmjit_emit.h as of 7ec0d80.
- partdesc.h, as of 1bb5e78.
Author: Jesse Zhang
Discussion: https://postgr.es/m/CAGf+fX4-8xULEOz09DE2dZGjT+q8VJ--rqfTpvcFwc+A4fc-3Q@mail.gmail.com
M src/include/common/jsonapi.h
M src/include/jit/llvmjit_emit.h
M src/include/partitioning/partdesc.h
M src/include/utils/jsonfuncs.h
Fix comments related to table AMs
commit : 9168793d7275b4b318c153d607fba55d14098c19
author : Michael Paquier <[email protected]>
date : Tue, 14 Jul 2020 13:17:11 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 14 Jul 2020 13:17:11 +0900
Incorrect function names were referenced. As this fixes some portions
of tableam.h, that is mentioned in the docs as something to look at when
implementing a table AM, backpatch down to 12 where this has been
introduced.
Author: Hironobu Suzuki
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M src/backend/access/heap/heapam.c
M src/include/access/tableam.h
Cope with lateral references in the quals of a subquery RTE.
commit : a742ecf9c63d454ccb107a357288c8ec1444ca12
author : Tom Lane <[email protected]>
date : Mon, 13 Jul 2020 20:38:20 -0400
committer: Tom Lane <[email protected]>
date : Mon, 13 Jul 2020 20:38:20 -0400
The qual pushdown logic assumed that all Vars in a restriction clause
must be Vars referencing subquery outputs; but since we introduced
LATERAL, it's possible for such a Var to be a lateral reference instead.
This led to an assertion failure in debug builds. In a non-debug
build, there might be no ill effects (if qual_is_pushdown_safe decided
the qual was unsafe anyway), or we could get failures later due to
construction of an invalid plan. I've not gone to much length to
characterize the possible failures, but at least segfaults in the
executor have been observed.
Given that this has been busted since 9.3 and it took this long for
anybody to notice, I judge that the case isn't worth going to great
lengths to optimize. Hence, fix by just teaching qual_is_pushdown_safe
that such quals are unsafe to push down, matching the previous behavior
when it accidentally didn't fail.
Per report from Tom Ellis. Back-patch to all supported branches.
Discussion: https://postgr.es/m/20200713175124.GQ8220@cloudinit-builder
M src/backend/optimizer/path/allpaths.c
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
Fix uninitialized value in segno calculation
commit : b5b4c0fef9fb905d98f93c1f455397449c6e63a8
author : Alvaro Herrera <[email protected]>
date : Mon, 13 Jul 2020 13:49:51 -0400
committer: Alvaro Herrera <[email protected]>
date : Mon, 13 Jul 2020 13:49:51 -0400
Remove previous hack in KeepLogSeg that added a case to deal with a
(badly represented) invalid segment number. This was added for the sake
of GetWALAvailability. But it's not needed if in that function we
initialize the segment number to be retreated to the currently being
written segment, so do that instead.
Per valgrind-running buildfarm member skink, and some sparc64 animals.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
Fix bugs in libpq's management of GSS encryption state.
commit : 25fe5ac45a736e50d2b85280a3cb0ce169f583f3
author : Tom Lane <[email protected]>
date : Mon, 13 Jul 2020 11:57:55 -0400
committer: Tom Lane <[email protected]>
date : Mon, 13 Jul 2020 11:57:55 -0400
GSS-related resources should be cleaned up in pqDropConnection,
not freePGconn, else the wrong things happen when resetting
a connection or trying to switch to a different server.
It's also critical to reset conn->gssenc there.
During connection setup, initialize conn->try_gss at the correct
place, else switching to a different server won't work right.
Remove now-redundant cleanup of GSS resources around one (and, for
some reason, only one) pqDropConnection call in connectDBStart.
Per report from Kyotaro Horiguchi that psql would freeze up,
rather than successfully resetting a GSS-encrypted connection
after a server restart.
This is YA oversight in commit b0b39f72b, so back-patch to v12.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-connect.c
Improvements to psql \dAo and \dAp commands
commit : 8d2ed66e4107ef27d05aef682c68af5952af7690
author : Alexander Korotkov <[email protected]>
date : Sat, 11 Jul 2020 14:14:49 +0300
committer: Alexander Korotkov <[email protected]>
date : Sat, 11 Jul 2020 14:14:49 +0300
* Strategy number and purpose are essential information for opfamily operator.
So, show those columns in non-verbose output.
* "Left/right arg type" \dAp column names are confusing, because those type
don't necessary match to function arguments. Rename them to "Registered
left/right type".
* Replace manual assembling of operator/procedure names with casts to
regoperator/regprocedure.
* Add schema-qualification for pg_catalog functions and tables.
Reported-by: Peter Eisentraut, Tom Lane
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/2edc7b27-031f-b2b6-0db2-864241c91cb9%402ndquadrant.com
Backpatch-through: 13
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
HashAgg: before spilling tuples, set unneeded columns to NULL.
commit : 2302302236a0c7aeba3377d78dd9d80ba53247df
author : Jeff Davis <[email protected]>
date : Sun, 12 Jul 2020 17:48:49 -0700
committer: Jeff Davis <[email protected]>
date : Sun, 12 Jul 2020 17:48:49 -0700
This is a replacement for 4cad2534. Instead of projecting all tuples
going into a HashAgg, only remove unnecessary attributes when actually
spilling. This avoids the regression for the in-memory case.
Discussion: https://postgr.es/m/a2fb7dfeb4f50aa0a123e42151ee3013933cb802.camel%40j-davis.com
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
M src/include/nodes/execnodes.h
Revert "Use CP_SMALL_TLIST for hash aggregate"
commit : 0babd109801e5ecd90df29589c23c6daf3ae69f7
author : Jeff Davis <[email protected]>
date : Sun, 12 Jul 2020 16:46:19 -0700
committer: Jeff Davis <[email protected]>
date : Sun, 12 Jul 2020 16:46:19 -0700
This reverts commit 4cad2534da6d17067d98cf04be2dfc1bda8f2cd0 due to a
performance regression. It will be replaced by a new approach in an
upcoming commit.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M contrib/postgres_fdw/expected/postgres_fdw.out
M src/backend/optimizer/plan/createplan.c
Revert "Track statistics for spilling of changes from ReorderBuffer".
commit : d973747281caece520236e93d255c654cc613ec9
author : Amit Kapila <[email protected]>
date : Mon, 13 Jul 2020 08:06:05 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 13 Jul 2020 08:06:05 +0530
The stats with this commit was available only for WALSenders, however,
users might want to see for backends doing logical decoding via SQL API.
Then, users might want to reset and access these stats across server
restart which was not possible with the current patch.
List of commits reverted:
caa3c4242c Don't call elog() while holding spinlock.
e641b2a995 Doc: Update the documentation for spilled transaction
statistics.
5883f5fe27 Fix unportable printf format introduced in commit 9290ad198.
9290ad198b Track statistics for spilling of changes from ReorderBuffer.
Additionaly, remove the release notes entry for this feature.
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CA+fd4k5_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/catalog/system_views.sql
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/walsender.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/replication/reorderbuffer.h
M src/include/replication/walsender_private.h
M src/test/regress/expected/rules.out
Fix timestamp range handling in regression tests of modules/commit_ts/
commit : 5bfe6a3c485d3259f59fa2d2e1d34dea1a3baeba
author : Michael Paquier <[email protected]>
date : Mon, 13 Jul 2020 10:54:26 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 13 Jul 2020 10:54:26 +0900
Switching the regression tests to use tstzrange() has proved to not be a
good idea for environments where the timestamp precision is low, as
internal range checks exclude the upper bound. So, if the commit
timestamp of a transaction matched with now() from the next query,
the test would fail. This changes to use two bound checks instead of
the range function, where the upper bound is inclusive.
Per buildfarm member jacana.
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/commit_ts/expected/commit_timestamp.out
M src/test/modules/commit_ts/expected/commit_timestamp_1.out
M src/test/modules/commit_ts/sql/commit_timestamp.sql
Fix test failure with -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
commit : ea3e15d1691ec4cadc67f160cc91c7f237a705ae
author : Michael Paquier <[email protected]>
date : Sun, 12 Jul 2020 21:25:36 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 12 Jul 2020 21:25:36 +0900
Replication origins created by regression tests should have names
starting with "regress_", and the test introduced in b1e48bb for commit
timestamps did not do that.
Per buildfarm member longfin.
Discussion: https://postgr.es/m/[email protected]
M src/test/modules/commit_ts/expected/commit_timestamp.out
M src/test/modules/commit_ts/expected/commit_timestamp_1.out
M src/test/modules/commit_ts/sql/commit_timestamp.sql
Include replication origins in SQL functions for commit timestamp
commit : b1e48bbe64a411666bb1928b9741e112e267836d
author : Michael Paquier <[email protected]>
date : Sun, 12 Jul 2020 20:47:15 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 12 Jul 2020 20:47:15 +0900
This includes two changes:
- Addition of a new function pg_xact_commit_timestamp_origin() able, for
a given transaction ID, to return the commit timestamp and replication
origin of this transaction. An equivalent function existed in
pglogical.
- Addition of the replication origin to pg_last_committed_xact().
The commit timestamp manager includes already APIs able to return the
replication origin of a transaction on top of its commit timestamp, but
the code paths for replication origins were never stressed as those
functions have never looked for a replication origin, and the SQL
functions available have never included this information since their
introduction in 73c986a.
While on it, refactor a test of modules/commit_ts/ to use tstzrange() to
check that a transaction timestamp is within the wanted range, making
the test a bit easier to read.
Bump catalog version.
Author: Movead Li
Reviewed-by: Madan Kumar, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/access/transam/commit_ts.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/modules/commit_ts/expected/commit_timestamp.out
M src/test/modules/commit_ts/expected/commit_timestamp_1.out
M src/test/modules/commit_ts/sql/commit_timestamp.sql
Avoid useless buffer allocations during binary COPY FROM.
commit : cd22d3cdb9bd9963c694c01a8c0232bbae3ddcfb
author : Tom Lane <[email protected]>
date : Sat, 11 Jul 2020 14:21:28 -0400
committer: Tom Lane <[email protected]>
date : Sat, 11 Jul 2020 14:21:28 -0400
The raw_buf and line_buf buffers aren't used when reading binary format,
so skip allocating them. raw_buf is 64K so that seems like a worthwhile
savings. An unused line_buf only wastes 1K, but as long as we're checking
it's free to avoid allocating that too.
Bharath Rupireddy, tweaked a bit by me
Discussion: https://postgr.es/m/CALj2ACXcCKaGPY0whowqrJ4OPJvDnTssgpGCzvuFQu5z0CXb-g@mail.gmail.com
M src/backend/commands/copy.c
Avoid trying to restore table ACLs and per-column ACLs in parallel.
commit : ea9125304dc6e90eabad165bd120eb1e667525d4
author : Tom Lane <[email protected]>
date : Sat, 11 Jul 2020 13:36:50 -0400
committer: Tom Lane <[email protected]>
date : Sat, 11 Jul 2020 13:36:50 -0400
Parallel pg_restore has always supposed that ACL items for different
objects are independent and can be restored in parallel without
conflicts. However, there is one case where this fails: because
REVOKE on a table is defined to also revoke the privilege(s) at
column level, we can't restore per-column ACLs till after we restore
any table-level privileges on their table. Failure to honor this
restriction can lead to "tuple concurrently updated" errors during
parallel restore, or even to the per-column ACLs silently disappearing
because the table-level REVOKE is executed afterwards.
To fix, add a dependency from each column-level ACL item to its table's
ACL item, if there is one. Note that this doesn't fix the hazard
for pre-existing archive files, only for ones made with a corrected
pg_dump. Given that the bug's been there quite awhile without
field reports, I think this is acceptable.
This requires changing the API of pg_dump's dumpACL() function.
To keep its argument list from getting even longer, I removed the
"CatalogId objCatId" argument, which has been unused for ages.
Per report from Justin Pryzby. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_dump.c
doc: Add link from pg_dump --encoding to supported encodings
commit : 64fe120b57c6a928a527880476e9882b9bf7ae8a
author : Peter Eisentraut <[email protected]>
date : Sat, 11 Jul 2020 13:47:29 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 11 Jul 2020 13:47:29 +0200
Reported-by: Lee Dong Wook <[email protected]>
M doc/src/sgml/ref/pg_dump.sgml
Rename field "relkind" to "objtype" for CTAS and ALTER TABLE nodes
commit : cc35d8933a211d9965eb1c1d2749a903d5735db2
author : Michael Paquier <[email protected]>
date : Sat, 11 Jul 2020 13:32:28 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 11 Jul 2020 13:32:28 +0900
"relkind" normally refers to the char field from pg_class. However, in
the parse nodes AlterTableStmt and CreateTableAsStmt, "relkind" was used
for a field of type enum ObjectType, that could refer to other object
types than those possible for a relkind. Such fields being usually
named "objtype", switch the name in both structures to make things more
consistent. Note that this led to some confusion in functions that
also operate on a RangeTableEntry object, which also has a field named
"relkind".
This naming goes back to commit 09d4e96, where only OBJECT_TABLE and
OBJECT_INDEX were used. This got extended later to use as well
OBJECT_TYPE with e440e12, not really a relation kind.
Author: Mark Dilger
Reviewed-by: Daniel Gustafsson, Álvaro Herrera, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/parser/analyze.c
M src/backend/parser/gram.y
M src/backend/parser/parse_utilcmd.c
M src/backend/tcop/utility.c
M src/include/nodes/parsenodes.h
Forbid numeric NaN in jsonpath
commit : df646509f371069c65f84309eb5749642e8650b3
author : Alexander Korotkov <[email protected]>
date : Sat, 11 Jul 2020 03:21:00 +0300
committer: Alexander Korotkov <[email protected]>
date : Sat, 11 Jul 2020 03:21:00 +0300
SQL standard doesn't define numeric Inf or NaN values. It appears even more
ridiculous to support then in jsonpath assuming JSON doesn't support these
values as well. This commit forbids returning NaN from .double(), which was
previously allowed. NaN can't be result of inner-jsonpath computation over
non-NaNs. So, we can not expect NaN in the jsonpath output.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/203949.1591879542%40sss.pgh.pa.us
Author: Alexander Korotkov
Reviewed-by: Tom Lane
Backpatch-through: 12
M src/backend/utils/adt/jsonb_util.c
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/jsonb_jsonpath.out
Improve error reporting for jsonpath .double() method
commit : 0657181167467ec5cfb5a28aacf47ca02f5f4588
author : Alexander Korotkov <[email protected]>
date : Sat, 11 Jul 2020 03:20:46 +0300
committer: Alexander Korotkov <[email protected]>
date : Sat, 11 Jul 2020 03:20:46 +0300
When jsonpath .double() method detects that numeric or string can't be
converted to double precision, it throws an error. This commit makes these
errors explicitly express the reason of failure.
Discussion: https://postgr.es/m/CAPpHfdtqJtiSXkP7tOXez18NxhLUH_-75bL8%3DOce4Ki%2Bbv7V6Q%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Tom Lane
Backpatch-through: 12
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/jsonb_jsonpath.out
M src/test/regress/sql/jsonb_jsonpath.sql
Doc: update or remove dead external links.
commit : e91cd951b1114a009a0256f211a41d0ec873b2fc
author : Tom Lane <[email protected]>
date : Fri, 10 Jul 2020 13:16:00 -0400
committer: Tom Lane <[email protected]>
date : Fri, 10 Jul 2020 13:16:00 -0400
Re-point comp.ai.genetic FAQ link to a more stable address.
Remove stale links to AIX documentation; we don't really need to
tell AIX users how to use their systems.
Remove stale links to HP documentation about SSL. We've had to
update those twice before, making it increasingly obvious that
HP does not intend them to be stable landing points. They're
not particularly authoritative, either. (This change effectively
reverts bbd3bdba3.)
Daniel Gustafsson and Álvaro Herrera, per a gripe from
Kyotaro Horiguchi. Back-patch, since these links are
just as dead in the back branches.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/geqo.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/runtime.sgml
Add missing <application> tags in application doc <refentrytitle>s
commit : 72a16cb3ee3017ccd8e4d58ea3c0c7bceec8b702
author : Peter Eisentraut <[email protected]>
date : Fri, 10 Jul 2020 16:51:29 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 10 Jul 2020 16:51:29 +0200
Most of them already have this, but some were missing.
Author: Author: Dagfinn Ilmari Mannsåker <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/87o8pco34z.fsf%40wibble.ilmari.org
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_config-ref.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_receivewal.sgml
M doc/src/sgml/ref/pg_restore.sgml
M doc/src/sgml/ref/pg_verifybackup.sgml
Revert "Remove reset of testtablespace from pg_regress on Windows"
commit : 61be85afabba0cd85bb1bcaacbea8efa8641f564
author : Michael Paquier <[email protected]>
date : Fri, 10 Jul 2020 17:08:13 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 10 Jul 2020 17:08:13 +0900
This reverts commit 2b2a070, that moved the reset of path
"testtablespace" used by the regression tests as a path for tablespaces
(via --outputdir) from pg_regress to the MSVC script vcregress.pl, as
this broke the behavior added by ce5d342 to be able to safely run the
regression test suite with an administrative Windows account using a
restricted token.
Note that before 2b2a070, the code doing the reset in pg_regress.c
included a comment telling that we had better move that out to a
different place, leading to the mistake done in 2b2a070. Fix this
comment, and document instead that we had better never remove this code,
for the sake of not breaking again the behavior we expect on Windows.
Thanks to Thomas Munro and Andrew Dunstan for the discussion.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CA+hUKGLiieEzfrdWxWFE+_wnXho_F5Smx972X1wEubhS7v1q9g@mail.gmail.com
M src/test/regress/pg_regress.c
M src/tools/msvc/vcregress.pl
Log the location field before any backtrace
commit : ff61359ad7cd10e48a9ce662b5f2a5e7f3ca23e1
author : Peter Eisentraut <[email protected]>
date : Fri, 10 Jul 2020 08:27:00 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 10 Jul 2020 08:27:00 +0200
This order makes more sense because the location is effectively at the
lowest level of the backtrace.
Discussion: https://www.postgresql.org/message-id/flat/90f5fa04-c410-a54e-9449-aa3749fb7972%402ndquadrant.com
M src/backend/utils/error/elog.c
Remove WARNING message from brin_desummarize_range
commit : 986529ce40c6edcd7f83689c00fcfaab8d9436d8
author : Alvaro Herrera <[email protected]>
date : Thu, 9 Jul 2020 20:13:25 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 9 Jul 2020 20:13:25 -0400
This message was being emitted on the grounds that only crashed
summarization could cause it, but in reality even an aborted vacuum
could do it ... which makes it way too noisy, particularly since it
shows up in regression tests and makes them die.
Reported by Tom Lane.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/brin/brin_revmap.c
Fix pg_current_logfile() to not emit a carriage return on Windows.
commit : 183926da3162b1807904710e46882c004beff5fb
author : Tom Lane <[email protected]>
date : Thu, 9 Jul 2020 16:02:23 -0400
committer: Tom Lane <[email protected]>
date : Thu, 9 Jul 2020 16:02:23 -0400
Due to not having our signals straight about CRLF vs. LF line
termination, the output of pg_current_logfile() included a trailing
\r on Windows. To fix, force the file descriptor it uses into text
mode.
While here, move a couple of local variable declarations to make
the function's logic clearer.
In v12 and v13, also back-patch the test added by 1c4e88e2f so that
this function has some test coverage. However, the 004_logrotate.pl
test script doesn't exist before v12, and it didn't seem worth adding
to older branches just for this.
Per report from Thomas Kellerer. Back-patch to v10 where this
function was added.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/misc.c
Further tighten Windows CRLF conversion in our TAP test scripts.
commit : ffb4cee43bdccb54f0cf072a51dedc74e343b6f1
author : Tom Lane <[email protected]>
date : Thu, 9 Jul 2020 11:37:21 -0400
committer: Tom Lane <[email protected]>
date : Thu, 9 Jul 2020 11:37:21 -0400
Buildfarm results now imply that Perl's IPC::Run does CRLF conversion
for us if we're using native Perl, but not when using MSys Perl.
Restrict the conversions done by PostgresNode.pm to act only in the
latter case. (Similar conversions done in TestLib.pm and RewindTest.pm
were already handled this way.)
Discussion: https://postgr.es/m/[email protected]
M src/test/perl/PostgresNode.pm
pg_dump: Further reorganize getTableAttrs()
commit : 991c444e7ad3a1ff7bcffff4121e6ff67c4783fc
author : Peter Eisentraut <[email protected]>
date : Thu, 9 Jul 2020 09:47:43 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 9 Jul 2020 09:47:43 +0200
After further discussion after
daa9fe8a5264a3f192efa5ddee8fb011ad9da365, reorder the version-specific
sections from oldest to newest. Also, remove the variable assignments
from PQfnumber() to reduce vertical space.
Reviewed-by: Fabien COELHO <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M src/bin/pg_dump/pg_dump.c
doc: Correct the description about the length of pg_stat_activity.query.
commit : a5cd7047e7c6dacdef79a763a26802469ac4e02b
author : Fujii Masao <[email protected]>
date : Thu, 9 Jul 2020 13:31:33 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 9 Jul 2020 13:31:33 +0900
pg_stat_activity.query text is truncated at 1024 bytes. But previously
the document described that it's truncated at 1024 characters.
This was not accurate when considering multibyte characters.
Back-patch to v10 where this inaccurate description was added.
Author: Atsushi Torikoshi
Reviewed-by: Daniel Gustafsson, Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/monitoring.sgml
Tighten up Windows CRLF conversion in our TAP test scripts.
commit : 91bdf499b37b0bbef34e2bab5cc40dde52bda52c
author : Tom Lane <[email protected]>
date : Wed, 8 Jul 2020 20:25:52 -0400
committer: Tom Lane <[email protected]>
date : Wed, 8 Jul 2020 20:25:52 -0400
The previous approach was to search-and-destroy all \r occurrences
no matter what. That seems more likely to hide bugs than anything
else; indeed it seems to be hiding one now. Fix things so that
we only transform \r\n to \n.
Side effects: must do this before, not after, chomp'ing if we're
going to chomp, else we'd fail to clean up a trailing \r\n. Also,
remove safe_psql's redundant repetition of what psql already did;
else it might reduce \r\r\n to \n, which is exactly the scenario
I'm hoping to expose.
Perhaps this should be back-patched, but for now I'm content to
see what happens in HEAD.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_rewind/t/RewindTest.pm
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
Fix whitespace in HashAgg EXPLAIN ANALYZE
commit : 2b7dbc0db6ecf38fc305284d006a1ef8f5354fbb
author : David Rowley <[email protected]>
date : Thu, 9 Jul 2020 10:06:24 +1200
committer: David Rowley <[email protected]>
date : Thu, 9 Jul 2020 10:06:24 +1200
The Sort node does not put a space between the number of kilobytes and
the "kB" of memory or disk space used, but HashAgg does. Here we align
HashAgg to do the same as Sort. Sort has been displaying this
information for longer than HashAgg, so it makes sense to align HashAgg
to Sort rather than the other way around.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13, where the hashagg started showing these details
M src/backend/commands/explain.c
code: replace most remaining uses of 'master'.
commit : a9a4a7ad565b136cbee735d4bb505d98d06da522
author : Andres Freund <[email protected]>
date : Mon, 15 Jun 2020 10:14:40 -0700
committer: Andres Freund <[email protected]>
date : Mon, 15 Jun 2020 10:14:40 -0700
Author: Andres Freund
Reviewed-By: David Steele
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
M src/backend/catalog/index.c
M src/backend/catalog/toasting.c
M src/backend/commands/vacuum.c
M src/backend/libpq/hba.c
M src/backend/libpq/pqcomm.c
M src/backend/optimizer/plan/planner.c
M src/backend/parser/gram.y
M src/backend/snowball/README
M src/backend/utils/time/snapmgr.c
M src/include/nodes/execnodes.h
M src/include/utils/snapmgr.h
M src/pl/tcl/pltcl.c
docs: replace 'master process' with 'supervisor process' where appropriate.
commit : 7c89f8a5b810d10dae300ec58ea7d70024e9123e
author : Andres Freund <[email protected]>
date : Mon, 15 Jun 2020 10:19:32 -0700
committer: Andres Freund <[email protected]>
date : Mon, 15 Jun 2020 10:19:32 -0700
Author: Andres Freund
Reviewed-By: David Steele
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/arch-dev.sgml
M doc/src/sgml/runtime.sgml
M doc/src/sgml/start.sgml
docs: replace 'master' with 'root' where appropriate.
commit : 09dfd430118f1fadf52a782db5ee161b1eb16337
author : Andres Freund <[email protected]>
date : Mon, 15 Jun 2020 10:18:41 -0700
committer: Andres Freund <[email protected]>
date : Mon, 15 Jun 2020 10:18:41 -0700
These uses of 'master' refer to partitioning / inheritance. 'root'
seems more descriptive than 'master'.
Author: Andres Freund
Reviewed-By: David Steele
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ddl.sgml
docs: replace 'master' with 'primary' where appropriate.
commit : 9e101cf60612f4be4f855d7393531900c2986a55
author : Andres Freund <[email protected]>
date : Mon, 15 Jun 2020 10:12:58 -0700
committer: Andres Freund <[email protected]>
date : Mon, 15 Jun 2020 10:12:58 -0700
Also changed "in the primary" to "on the primary", and added a few
"the" before "primary".
Author: Andres Freund
Reviewed-By: David Steele
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/amcheck.sgml
M doc/src/sgml/backup.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/external-projects.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/pgstandby.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_rewind.sgml
M doc/src/sgml/runtime.sgml
M doc/src/sgml/wal.sgml
code: replace 'master' with 'leader' where appropriate.
commit : e07633646a22734e85d7fc58a66855f747128e6b
author : Andres Freund <[email protected]>
date : Sun, 14 Jun 2020 14:22:47 -0700
committer: Andres Freund <[email protected]>
date : Sun, 14 Jun 2020 14:22:47 -0700
Leader already is the more widely used terminology, but a few places
didn't get the message.
Author: Andres Freund
Reviewed-By: David Steele
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_prewarm/autoprewarm.c
M doc/src/sgml/ref/pg_dump.sgml
M src/backend/access/transam/parallel.c
M src/backend/access/transam/xact.c
M src/backend/executor/execGrouping.c
M src/backend/libpq/pqmq.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/util/clauses.c
M src/backend/utils/init/globals.c
M src/backend/utils/misc/guc.c
M src/bin/pg_dump/parallel.c
M src/bin/pg_dump/parallel.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_directory.c
M src/bin/pg_dump/pg_dump.c
M src/include/catalog/pg_proc.h
M src/include/libpq/pqmq.h
M src/include/storage/backendid.h
code: replace 'master' with 'primary' where appropriate.
commit : 5e7bbb528638c0f6d585bab107ec7a19e3a39deb
author : Andres Freund <[email protected]>
date : Sun, 14 Jun 2020 14:05:18 -0700
committer: Andres Freund <[email protected]>
date : Sun, 14 Jun 2020 14:05:18 -0700
Also changed "in the primary" to "on the primary", and added a few
"the" before "primary".
Author: Andres Freund
Reviewed-By: David Steele
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/common/bufmask.c
M src/backend/access/gist/gistxlog.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/pruneheap.c
M src/backend/access/nbtree/README
M src/backend/access/nbtree/nbtxlog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogutils.c
M src/backend/catalog/namespace.c
M src/backend/commands/tablecmds.c
M src/backend/postmaster/postmaster.c
M src/backend/replication/README
M src/backend/replication/basebackup.c
M src/backend/replication/logical/worker.c
M src/backend/replication/walreceiver.c
M src/backend/replication/walsender.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/README
M src/backend/storage/page/README
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/pg_basebackup/pg_recvlogical.c
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_rewind/copy_fetch.c
M src/bin/pg_rewind/filemap.c
M src/bin/pg_rewind/parsexlog.c
M src/include/access/xlog.h
M src/include/tcop/utility.h
M src/include/utils/guc_tables.h
tap tests: replace 'master' with 'primary'.
commit : 229f8c219f8fffacc253eca6023eab10a16eb009
author : Andres Freund <[email protected]>
date : Sun, 14 Jun 2020 11:47:37 -0700
committer: Andres Freund <[email protected]>
date : Sun, 14 Jun 2020 11:47:37 -0700
We've largely replaced master with primary in docs etc, but tap test
still widely used master.
Author: Andres Freund
Reviewed-By: David Steele
Discussion: https://postgr.es/m/[email protected]
M contrib/bloom/t/001_wal.pl
M src/bin/pg_rewind/t/001_basic.pl
M src/bin/pg_rewind/t/002_databases.pl
M src/bin/pg_rewind/t/003_extrafiles.pl
M src/bin/pg_rewind/t/004_pg_xlog_symlink.pl
M src/bin/pg_rewind/t/005_same_timeline.pl
M src/bin/pg_rewind/t/RewindTest.pm
M src/bin/pg_verifybackup/t/002_algorithm.pl
M src/bin/pg_verifybackup/t/003_corruption.pl
M src/bin/pg_verifybackup/t/004_options.pl
M src/bin/pg_verifybackup/t/006_encoding.pl
M src/bin/pg_verifybackup/t/007_wal.pl
M src/test/authentication/t/001_password.pl
M src/test/authentication/t/002_saslprep.pl
M src/test/modules/commit_ts/t/002_standby.pl
M src/test/modules/commit_ts/t/003_standby_2.pl
M src/test/modules/commit_ts/t/004_restart.pl
M src/test/modules/test_misc/t/001_constraint_validation.pl
M src/test/perl/PostgresNode.pm
M src/test/perl/README
M src/test/recovery/t/001_stream_rep.pl
M src/test/recovery/t/002_archiving.pl
M src/test/recovery/t/003_recovery_targets.pl
M src/test/recovery/t/004_timeline_switch.pl
M src/test/recovery/t/005_replay_delay.pl
M src/test/recovery/t/006_logical_decoding.pl
M src/test/recovery/t/007_sync_rep.pl
M src/test/recovery/t/008_fsm_truncation.pl
M src/test/recovery/t/009_twophase.pl
M src/test/recovery/t/010_logical_decoding_timelines.pl
M src/test/recovery/t/011_crash_recovery.pl
M src/test/recovery/t/012_subtransactions.pl
M src/test/recovery/t/013_crash_restart.pl
M src/test/recovery/t/019_replslot_limit.pl
M src/test/recovery/t/020_archive_status.pl
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
Don't treat DumpOptions->dump_inserts like a boolean
commit : 2661a793ff59149af1d844d6be504e421c04733e
author : Alvaro Herrera <[email protected]>
date : Wed, 8 Jul 2020 14:52:39 -0400
committer: Alvaro Herrera <[email protected]>
date : Wed, 8 Jul 2020 14:52:39 -0400
This has been an integer count since 7e413a0f82c8 so treat it explicitly
like an integer.
No backpatch since this is just cosmetic.
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_dump.c
Add test coverage for pg_current_logfile() function.
commit : 1c4e88e2fe41dbd1cb3431fbfed5ee6d92bfe089
author : Tom Lane <[email protected]>
date : Wed, 8 Jul 2020 13:41:52 -0400
committer: Tom Lane <[email protected]>
date : Wed, 8 Jul 2020 13:41:52 -0400
There has been no coverage at all up to now. Given Thomas Kellerer's
recent report, I suspect this may fail on (some?) Windows machines,
but let's find out.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_ctl/t/004_logrotate.pl
Fix incorrect variable datatype.
commit : 654242fd81cfbdd229ec2a752a9f857a84cc039b
author : Fujii Masao <[email protected]>
date : Wed, 8 Jul 2020 21:24:34 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 8 Jul 2020 21:24:34 +0900
Since slot_keep_segs indicates the number of WAL segments not LSN,
its datatype should not be XLogRecPtr.
Back-patch to v13 where this issue was added.
Reported-by: Atsushi Torikoshi
Author: Atsushi Torikoshi, tweaked by Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
Remove junk in test file
commit : eb2c8a8f816b6c411047eeab7a326e8356a2c0a8
author : Peter Eisentraut <[email protected]>
date : Wed, 8 Jul 2020 11:17:52 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 8 Jul 2020 11:17:52 +0200
Remove a redundant and failing command, probably a typo.
M src/test/modules/unsafe_tests/expected/rolenames.out
M src/test/modules/unsafe_tests/sql/rolenames.sql
Fix typo
commit : 98f0eba5b7840197ee43f52833ef08f9090fbc44
author : Magnus Hagander <[email protected]>
date : Wed, 8 Jul 2020 10:11:43 +0200
committer: Magnus Hagander <[email protected]>
date : Wed, 8 Jul 2020 10:11:43 +0200
Author: Daniel Gustafsson
M src/backend/postmaster/pgstat.c
Fix function name in comment.
commit : 5e574d170e2eb112e1da8a40f011741b82340f68
author : Fujii Masao <[email protected]>
date : Wed, 8 Jul 2020 11:00:23 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 8 Jul 2020 11:00:23 +0900
Author: Masahiro Ikeda
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/cache/relfilenodemap.c
doc: Fix inconsistencies in GIN, BRIN and SP-GiST for optional opclass methods
commit : d92be269103acaf58e87ba4ba732eac0e3c4d78a
author : Michael Paquier <[email protected]>
date : Wed, 8 Jul 2020 10:41:53 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 8 Jul 2020 10:41:53 +0900
The GIN and SP-GiST parts were out-of-sync since the changes of 14903f2,
and the BRIN part was wrong since its introduction in 15cb2bd.
Author: Guillaume Lelarge
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/CAECtzeXKvEPEr967h0PRYRi39uTmdEms=oUtc_PWGjZRNN1prw@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/brin.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/spgist.sgml
Un-break pg_upgrade from pre-v12 servers.
commit : 3f96af4619c8b129ec8d5f4fb961df4310999383
author : Tom Lane <[email protected]>
date : Tue, 7 Jul 2020 18:10:42 -0400
committer: Tom Lane <[email protected]>
date : Tue, 7 Jul 2020 18:10:42 -0400
I neglected to test this scenario while preparing commit f3faf35f3,
so of course it was broken, thanks to some very obscure and undocumented
code in pg_dump. Pre-v12 databases might have toast tables attached to
partitioned tables, which we need to ignore since newer servers never
create such useless toast tables. There was a filter for this case in
binary_upgrade_set_type_oids_by_rel_oid(), which appeared to just
prevent the pg_type OID from being copied. But actually it managed to
prevent the toast table from being created at all --- or it did before
I took out that logic. But that was a fundamentally bizarre place to be
making the test in the first place. The place where the filter should
have been, one would think, is binary_upgrade_set_pg_class_oids(), so
add it there.
While at it, reorganize binary_upgrade_set_pg_class_oids() so that it
doesn't make a completely useless query when it knows it's being
invoked for an index. And correct a comment that mis-described the
scenario where we need to force creation of a TOAST table.
Per buildfarm.
M src/bin/pg_dump/pg_dump.c
Don't create pg_type entries for sequences or toast tables.
commit : f3faf35f370f558670c8213a08f2683f3811ffc7
author : Tom Lane <[email protected]>
date : Tue, 7 Jul 2020 15:43:22 -0400
committer: Tom Lane <[email protected]>
date : Tue, 7 Jul 2020 15:43:22 -0400
Commit f7f70d5e2 left one inconsistency behind: we're still creating
pg_type entries for the composite types of sequences and toast tables,
but not arrays over those composites. But there seems precious little
reason to have named composite types for toast tables, and not much more
to have them for sequences (especially given the thought that sequences
may someday not be standalone relations at all).
So, let's close that inconsistency by removing these composite types,
rather than adding arrays for them. This buys back a little bit of
the initial pg_type bloat added by the previous patch, and could be
a significant savings in a large database with many toast tables.
Aside from a small logic rearrangement in heap_create_with_catalog,
this patch mostly needs to clean up some places that were assuming that
pg_class.reltype always has a valid value. Those are really pre-existing
bugs, given that it's documented otherwise; notably, the plpgsql changes
fix code that gives "cache lookup failed for type 0" on indexes today.
But none of these seem interesting enough to back-patch.
Also, remove the pg_dump/pg_upgrade infrastructure for propagating
a toast table's pg_type OID into the new database, since we no longer
need that.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/heap.c
M src/backend/catalog/toasting.c
M src/backend/commands/tablecmds.c
M src/backend/nodes/makefuncs.c
M src/backend/utils/adt/pg_upgrade_support.c
M src/backend/utils/cache/relcache.c
M src/bin/pg_dump/pg_dump.c
M src/include/catalog/binary_upgrade.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/pl/plpgsql/src/pl_comp.c
Morph pg_replication_slots.min_safe_lsn to safe_wal_size
commit : a8aaa0c786b3a27706172cc7979684a122ebb706
author : Alvaro Herrera <[email protected]>
date : Tue, 7 Jul 2020 13:08:00 -0400
committer: Alvaro Herrera <[email protected]>
date : Tue, 7 Jul 2020 13:08:00 -0400
The previous definition of the column was almost universally disliked,
so provide this updated definition which is more useful for monitoring
purposes: a large positive value is good, while zero or a negative value
means danger. This should be operationally more convenient.
Backpatch to 13, where the new column to pg_replication_slots (and the
feature it represents) were added.
Author: Kyotaro Horiguchi <[email protected]>
Author: Álvaro Herrera <[email protected]>
Reported-by: Fujii Masao <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M src/backend/access/transam/xlog.c
M src/backend/catalog/system_views.sql
M src/backend/replication/slotfuncs.c
M src/include/access/xlog_internal.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/recovery/t/019_replslot_limit.pl
M src/test/regress/expected/rules.out
Check ssl_in_use flag when reporting statistics
commit : 6a5c750f3f72899f4f982f921d5bf5665f55651e
author : Magnus Hagander <[email protected]>
date : Tue, 7 Jul 2020 16:57:27 +0200
committer: Magnus Hagander <[email protected]>
date : Tue, 7 Jul 2020 16:57:27 +0200
Previously we checked that the ssl pointer was not null, but this puts a
requirement on there being such a pointer which may not be true in
future multi-ssl-library supporting times. This seems to have been an
oversight in 9029f4b3740, but hasn't really had any effect since we only
have one library.
Author: Daniel Gustafsson
M src/backend/postmaster/pgstat.c
Resolve gratuitous tabs in test SQL file
commit : 71ec58e7fc1e180165fbed7a983d8179a03ddfc6
author : Peter Eisentraut <[email protected]>
date : Tue, 7 Jul 2020 10:21:54 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 7 Jul 2020 10:21:54 +0200
M src/test/modules/unsafe_tests/expected/rolenames.out
M src/test/modules/unsafe_tests/sql/rolenames.sql
Remove unnecessary PageIsEmpty() nbtree build check.
commit : 28c16f4947df0a98c124ec1724faff629d6c77f7
author : Peter Geoghegan <[email protected]>
date : Mon, 6 Jul 2020 13:47:29 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 6 Jul 2020 13:47:29 -0700
nbtree index builds cannot write out an empty page. That would mean
that there was no way to create a pivot tuple pointing to the page one
level up, since _bt_truncate() generates one based on page's firstright
tuple.
Replace the unnecessary PageIsEmpty() check with an assertion that
checks that the page has space for at least two line pointers (the
would-be high key line pointer, plus at least one valid "data item"
tuple line pointer).
The PageIsEmpty() check was added by commit 5d9f146c over 20 years ago.
It looks like it has always been unnecessary.
M src/backend/access/nbtree/nbtsort.c
Create composite array types for initdb-created relations.
commit : f7f70d5e22aa2330b8cc31dfc8732cd26ef0bbdd
author : Tom Lane <[email protected]>
date : Mon, 6 Jul 2020 14:21:16 -0400
committer: Tom Lane <[email protected]>
date : Mon, 6 Jul 2020 14:21:16 -0400
When we invented arrays of composite types (commit bc8036fc6),
we excluded system catalogs, basically just on the grounds of not
wanting to bloat pg_type. However, it's definitely inconsistent that
catalogs' composite types can't be put into arrays when others can.
Another problem is that the exclusion is done by checking
IsUnderPostmaster in heap_create_with_catalog, which means that
(1) If a user tries to create a table in single-user mode, it doesn't
get an array type. That's bad in itself, plus it breaks pg_upgrade.
(2) If someone drops and recreates a system view or information_schema
view (as we occasionally recommend doing), it will now have an array
type where it did not before, making for still more inconsistency.
So this is all pretty messy. Let's just get rid of the inconsistency
and decree that system-created relations should have array types if
similar user-created ones would, i.e. it only depends on the relkind.
As of HEAD, that means that the initial contents of pg_type grow from
411 rows to 605, which is a lot of growth percentage-wise, but it's
still quite a small catalog compared to others.
Wenjing Zeng, reviewed by Shawn Wang, further hacking by me
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/heap.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_type.dat
Fix typo in test
commit : bae9e8a58bf642aa383f5dc01b2c5947bae533dd
author : Peter Eisentraut <[email protected]>
date : Mon, 6 Jul 2020 09:53:37 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 6 Jul 2020 09:53:37 +0200
The test was supposed to error but didn't. Apparently, a copy and
paste and string replace mistake from a nearby similar test.
M src/test/modules/unsafe_tests/expected/rolenames.out
M src/test/modules/unsafe_tests/sql/rolenames.sql
doc: Add note about possible performance overhead by enabling track_planning.
commit : 321fa6a4a26c9b649a0fbec9fc8b019f19e62289
author : Fujii Masao <[email protected]>
date : Mon, 6 Jul 2020 14:27:09 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 6 Jul 2020 14:27:09 +0900
Enabling pg_stat_statements.track_plaanning may incur a noticeable
performance penalty, especially when a fewer kinds of queries are executed
on many concurrent connections. This commit documents this note.
Back-patch to v13 where pg_stat_statements.track_plaanning was added.
Suggested-by: Pavel Stehule
Author: Fujii Masao
Reviewed-by: Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRC9Jxa8r5i0TNBWLb8mzuaYzEoLq3QOvip0jVpHPOLbVA@mail.gmail.com
M doc/src/sgml/pgstatstatements.sgml
Refactor routines for name lookups of procedures and operators
commit : aa38434824c4fa52e55a9c7465b24c246caddc81
author : Michael Paquier <[email protected]>
date : Mon, 6 Jul 2020 13:06:08 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 6 Jul 2020 13:06:08 +0900
This introduces a new set of extended routines for procedure and
operator name lookups, with a flag bitmask argument that can modify the
result. The following options are available:
- Force schema qualification, ignoring search_path. This is similar to
the existing option for format_{operator|procedure}_qualified().
- Force NULL as result instead of a numeric OID for an undefined
object. This option is new.
This is a refactoring similar to 1185c78, that will be used for a future
patch to improve the SQL functions providing information using object
addresses for undefined objects.
Author: Michael Paquier
Reviewed-by: Aleksander Alekseev, Dmitry Dolgov, Daniel Gustafsson,
Álvaro Herrera
Discussion: https://postgr.es/m/CAB7nPqSZxrSmdHK-rny7z8mi=EAFXJ5J-0RbzDw6aus=wB5azQ@mail.gmail.com
M src/backend/utils/adt/regproc.c
M src/include/utils/regproc.h
Remove extra whitespace in comments atop ReorderBufferCheckMemoryLimit.
commit : 04c7f4144fec3bcddc82b6abdbfebd88989f7953
author : Amit Kapila <[email protected]>
date : Mon, 6 Jul 2020 08:44:33 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 6 Jul 2020 08:44:33 +0530
Backpatch-through: 13, where it was introduced
M src/backend/replication/logical/reorderbuffer.c
Add new flag to format_type_extended() to get NULL for undefined type
commit : 1185c782943c6d36835db481b199dce8e6f30246
author : Michael Paquier <[email protected]>
date : Mon, 6 Jul 2020 12:12:11 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 6 Jul 2020 12:12:11 +0900
If a type scanned is undefined, type format routines have two behaviors
depending on if FORMAT_TYPE_ALLOW_INVALID is used by the caller or not:
- Issue a cache lookup error
- Return an undefined type name "???", "???[]" or "-"
The current interface is not really helpful for callers willing to
format properly a type name, but still make sure that the type is
defined as there could be types matching the strings generated when
looking for an undefined type, even if that should not be a problem in
practice. In order to counter that, add a new flag called
FORMAT_TYPE_INVALID_AS_NULL that returns a NULL result instead of "???
or "-" which does not generate an error. This flag will be used in a
follow-up patch improving the set of SQL functions showing information
for object addresses when it comes to undefined objects.
Author: Michael Paquier
Reviewed-by: Aleksander Alekseev, Dmitry Dolgov, Daniel Gustafsson,
Álvaro Herrera
Discussion: https://postgr.es/m/CAB7nPqSZxrSmdHK-rny7z8mi=EAFXJ5J-0RbzDw6aus=wB5azQ@mail.gmail.com
M src/backend/utils/adt/format_type.c
M src/include/utils/builtins.h
Remove unused function parameter in end_parallel_vacuum.
commit : 231ef5b90d3a8a4ddf9de137e8e0997409c7a8b1
author : Amit Kapila <[email protected]>
date : Mon, 6 Jul 2020 08:21:52 +0530
committer: Amit Kapila <[email protected]>
date : Mon, 6 Jul 2020 08:21:52 +0530
Author: Vignesh C
Reviewed-by: Sawada Masahiko
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CALDaNm3Ppt71NafGY5mk3V2i3Q+mm93pVibDq-0NpW7WU67Jcg@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Improve perl script in MSVC to build binaries
commit : 404b912c5cab05d4a2ddac85696a300e00783c73
author : Michael Paquier <[email protected]>
date : Mon, 6 Jul 2020 09:16:17 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 6 Jul 2020 09:16:17 +0900
This commit includes two improvements for build.pl in src/tools/msvc/:
- Fix two warnings related to $ARGV[0] being uninitialized, something
that happens when calling build.pl (or build.bat) without arguments to
compile all the components with a release quality.
- If calling the script with more than two arguments, exit immediately
and show a new help output. build.pl was not failing in this case
before this commit, and the extra arguments were just ignored, but the
new behavior helps in understanding how to run the script without
looking at the documentation for the Windows builds.
Reported-by: Ranier Vilela
Author: Michael Paquier
Reviewed-by: Juan José Santamaría Flecha, David Zhang
Discussion: https://postgr.es/m/CAEudQAo38dfR_0Ugt2OHy4mq-6Hz93XPSBAGEUV67RhKdgp8Zg@mail.gmail.com
M src/tools/msvc/build.pl
Inline the fast path of plpgsql's exec_cast_value().
commit : fe2e206cdb00a2d5dcebc3c8e8119017392d9781
author : Tom Lane <[email protected]>
date : Sun, 5 Jul 2020 13:12:31 -0400
committer: Tom Lane <[email protected]>
date : Sun, 5 Jul 2020 13:12:31 -0400
In the common case where this function isn't actually asked to perform
any type conversion, there's nothing it has to do beyond comparing the
arguments. Arrange for that part to be inlined into callers, with the
slower path remaining out-of-line. This seems to be good for several
percent speedup on simple cases, with only minimal code bloat.
Amit Khandekar
Discussion: https://postgr.es/m/CAJ3gD9eBNrmUD7WBBLG8ohaZ485H9y+4eihQTgr+K8Lhka3vcQ@mail.gmail.com
M src/pl/plpgsql/src/pl_exec.c
doc: Spell checking
commit : 90b2d8c1adc86d1fc46f83c73444a1fb5b28dd3c
author : Peter Eisentraut <[email protected]>
date : Sun, 5 Jul 2020 15:37:57 +0200
committer: Peter Eisentraut <[email protected]>
date : Sun, 5 Jul 2020 15:37:57 +0200
M contrib/pg_stat_statements/pg_stat_statements.c
M doc/src/sgml/backup-manifest.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/glossary.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_verifybackup.sgml
M doc/src/sgml/ref/pgbench.sgml
doc: Fix incorrect reference to textout in plpgsql examples
commit : 8cea55e99b766fd654431f24987a774622594c9b
author : Michael Paquier <[email protected]>
date : Sun, 5 Jul 2020 19:35:56 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 5 Jul 2020 19:35:56 +0900
This error has survived for 22 years, and has been introduced by
da63386.
Reported-by: Erwin Brandstetter
Discussion: https://postgr.es/m/CAGHENJ57wogGOvGXo5LgWYcqswxafLck8ELqHDR+zrkTPgs_OQ@mail.gmail.com
Backpatch-through: 9.5
M doc/src/sgml/plpgsql.sgml
Rename enable_incrementalsort for clarity
commit : e61225ffab3f8726ecd053b1c22ff249c417f9ba
author : Peter Eisentraut <[email protected]>
date : Sun, 5 Jul 2020 11:41:52 +0200
committer: Peter Eisentraut <[email protected]>
date : Sun, 5 Jul 2020 11:41:52 +0200
Author: James Coleman <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/df652910-e985-9547-152c-9d4357dc3979%402ndquadrant.com
M doc/src/sgml/config.sgml
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/optimizer/cost.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/expected/partition_aggregate.out
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/incremental_sort.sql
M src/test/regress/sql/partition_aggregate.sql
Fix "ignoring return value" complaints from commit 96d1f423f9
commit : 1d05627fcf54b26e0cbd7527f9858f165d117813
author : Joe Conway <[email protected]>
date : Sat, 4 Jul 2020 13:46:31 -0400
committer: Joe Conway <[email protected]>
date : Sat, 4 Jul 2020 13:46:31 -0400
The cfbot and some BF animals are complaining about the previous
read_binary_file commit because of ignoring return value of ‘fread’.
So let's make everyone happy by testing the return value even though
not strictly needed.
Reported by Justin Pryzby, and suggested patch by Tom Lane. Backpatched
to v11 same as the previous commit.
Reported-By: Justin Pryzby
Reviewed-By: Tom Lane
Discussion: https://postgr.es/m/flat/969b8d82-5bb2-5fa8-4eb1-f0e685c5d736%40joeconway.com
Backpatch-through: 11
M src/backend/utils/adt/genfile.c
Read until EOF vice stat-reported size in read_binary_file
commit : 96d1f423f95d15c74989336e0776e1d55561a3f1
author : Joe Conway <[email protected]>
date : Sat, 4 Jul 2020 06:26:53 -0400
committer: Joe Conway <[email protected]>
date : Sat, 4 Jul 2020 06:26:53 -0400
read_binary_file(), used by SQL functions pg_read_file() and friends,
uses stat to determine file length to read, when not passed an explicit
length as an argument. This is problematic, for example, if the file
being read is a virtual file with a stat-reported length of zero.
Arrange to read until EOF, or StringInfo data string lenth limit, is
reached instead.
Original complaint and patch by me, with significant review, corrections,
advice, and code optimizations by Tom Lane. Backpatched to v11. Prior to
that only paths relative to the data and log dirs were allowed for files,
so no "zero length" files were reachable anyway.
Reviewed-By: Tom Lane
Discussion: https://postgr.es/m/flat/969b8d82-5bb2-5fa8-4eb1-f0e685c5d736%40joeconway.com
Backpatch-through: 11
M contrib/adminpack/expected/adminpack.out
M src/backend/utils/adt/genfile.c
Clamp total-tuples estimates for foreign tables to ensure planner sanity.
commit : ca5e93f769e1bf5e90d080cfcc8c0368ef649a7c
author : Tom Lane <[email protected]>
date : Fri, 3 Jul 2020 19:01:21 -0400
committer: Tom Lane <[email protected]>
date : Fri, 3 Jul 2020 19:01:21 -0400
After running GetForeignRelSize for a foreign table, adjust rel->tuples
to be at least as large as rel->rows. This prevents bizarre behavior
in estimate_num_groups() and perhaps other places, especially in the
scenario where rel->tuples is zero because pg_class.reltuples is
(suggesting that ANALYZE has never been run for the table). As things
stood, we'd end up estimating one group out of any GROUP BY on such a
table, whereas the default group-count estimate is more likely to result
in a sane plan.
Also, clarify in the documentation that GetForeignRelSize has the option
to override the rel->tuples value if it has a better idea of what to use
than what is in pg_class.reltuples.
Per report from Jeff Janes. Back-patch to all supported branches.
Patch by me; thanks to Etsuro Fujita for review
Discussion: https://postgr.es/m/CAMkU=1xNo9cnan+Npxgz0eK7394xmjmKg-QEm8wYG9P5-CcaqQ@mail.gmail.com
M doc/src/sgml/fdwhandler.sgml
M src/backend/optimizer/path/allpaths.c
Fix temporary tablespaces for shared filesets some more.
commit : f7b5988cc0d879d6cc9d481ba898e55747a53bbe
author : Tom Lane <[email protected]>
date : Fri, 3 Jul 2020 17:01:34 -0400
committer: Tom Lane <[email protected]>
date : Fri, 3 Jul 2020 17:01:34 -0400
Commit ecd9e9f0b fixed the problem in the wrong place, causing unwanted
side-effects on the behavior of GetNextTempTableSpace(). Instead,
let's make SharedFileSetInit() responsible for subbing in the value
of MyDatabaseTableSpace when the default tablespace is called for.
The convention about what is in the tempTableSpaces[] array is
evidently insufficiently documented, so try to improve that.
It also looks like SharedFileSetInit() is doing the wrong thing in the
case where temp_tablespaces is empty. It was hard-wiring use of the
pg_default tablespace, but it seems like using MyDatabaseTableSpace
is more consistent with what happens for other temp files.
Back-patch the reversion of PrepareTempTablespaces()'s behavior to
9.5, as ecd9e9f0b was. The changes in SharedFileSetInit() go back
to v11 where that was introduced. (Note there is net zero code change
before v11 from these two patch sets, so nothing to release-note.)
Magnus Hagander and Tom Lane
Discussion: https://postgr.es/m/CABUevExg5YEsOvqMxrjoNvb3ApVyH+9jggWGKwTDFyFCVWczGQ@mail.gmail.com
M src/backend/commands/tablespace.c
M src/backend/storage/file/fd.c
M src/backend/storage/file/sharedfileset.c
Inline plpgsql's exec_stmt() into exec_stmts().
commit : 1f902d499eda862a98e881a2227b8d2cf3565374
author : Tom Lane <[email protected]>
date : Fri, 3 Jul 2020 15:42:10 -0400
committer: Tom Lane <[email protected]>
date : Fri, 3 Jul 2020 15:42:10 -0400
This saves one level of C function call per plpgsql statement executed,
and permits a tiny additional optimization of not saving and restoring
estate->err_stmt for each statement in a block. The net effect seems
nearly un-measurable on x86_64, but there's a clear win on aarch64,
amounting to two or three percent in a loop over a few simple plpgsql
statements.
To do this, we have to get rid of the other existing call sites for
exec_stmt(). Replace them with exec_toplevel_block(), which is just
defined to do what exec_stmts() does, but for a single
PLpgSQL_stmt_block statement. Hard-wiring the expectation of which
statement type applies here allows us to skip the dispatch switch,
making this not much uglier than the previous factorization.
Amit Khandekar, tweaked a bit by me
Discussion: https://postgr.es/m/CAJ3gD9eBNrmUD7WBBLG8ohaZ485H9y+4eihQTgr+K8Lhka3vcQ@mail.gmail.com
M src/pl/plpgsql/src/pl_exec.c
Fix temporary tablespaces for shared filesets
commit : ecd9e9f0bc141550726b2205dd6f5745a58e9ecd
author : Magnus Hagander <[email protected]>
date : Fri, 3 Jul 2020 15:09:06 +0200
committer: Magnus Hagander <[email protected]>
date : Fri, 3 Jul 2020 15:09:06 +0200
A likely copy/paste error in 98e8b480532 from back in 2004 would
cause temp tablespace to be reset to InvalidOid if temp_tablespaces
was set to the same value as the primary tablespace in the database.
This would cause shared filesets (such as for parallel hash joins)
to ignore them, putting the temporary files in the default tablespace
instead of the configured one. The bug is in the old code, but it
appears to have been exposed only once we had shared filesets.
Reviewed-By: Daniel Gustafsson
Discussion: https://postgr.es/m/CABUevExg5YEsOvqMxrjoNvb3ApVyH+9jggWGKwTDFyFCVWczGQ@mail.gmail.com
Backpatch-through: 9.5
M src/backend/commands/tablespace.c
doc: Correct description of restart_lsn in pg_replication_slots
commit : 8f9b6d40570bd8991f18a089a8445cc5275c1f49
author : Fujii Masao <[email protected]>
date : Fri, 3 Jul 2020 12:08:35 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 3 Jul 2020 12:08:35 +0900
Previously the document explained that restart_lsn indicates the LSN of
oldest WAL won't be automatically removed during checkpoints. But
since v13 this was no longer true thanks to max_slot_wal_keep_size.
Back-patch to v13 where max_slot_wal_keep_size was added.
Author: Fujii Masao
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
Change default of pg_stat_statements.track_planning to off.
commit : d1763ea8c9c32837d373a196ed0c2e1256a55824
author : Fujii Masao <[email protected]>
date : Fri, 3 Jul 2020 11:35:22 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 3 Jul 2020 11:35:22 +0900
Since v13 pg_stat_statements is allowed to track the planning time of
statements when track_planning option is enabled. Its default was on.
But this feature could cause more terrible spinlock contentions in
pg_stat_statements. As a result of this, Robins Tharakan reported that
v13 beta1 showed ~45% performance drop at high DB connection counts
(when compared with v12.3) during fully-cached SELECT-only test using
pgbench.
To avoid this performance regression by the default setting,
this commit changes default of pg_stat_statements.track_planning to off.
Back-patch to v13 where pg_stat_statements.track_planning was introduced.
Reported-by: Robins Tharakan
Author: Fujii Masao
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
M doc/src/sgml/pgstatstatements.sgml
Initialize work_mem using current guc.c default.
commit : 947456a823d6b0973b68c6b38c8623a0504054e7
author : Peter Geoghegan <[email protected]>
date : Thu, 2 Jul 2020 16:34:54 -0700
committer: Peter Geoghegan <[email protected]>
date : Thu, 2 Jul 2020 16:34:54 -0700
Do the same for the maintenance_work_mem global variable.
Oversight in commit 848ae330a49, which increased the previous defaults
for work_mem and maintenance_work_mem by 4X.
M src/backend/utils/init/globals.c
nbtree: Rename _bt_search() variables.
commit : e25d462a38f853621a2ad529638d463836230766
author : Peter Geoghegan <[email protected]>
date : Thu, 2 Jul 2020 14:54:55 -0700
committer: Peter Geoghegan <[email protected]>
date : Thu, 2 Jul 2020 14:54:55 -0700
Make some of the variable names in _bt_search() consistent with
corresponding variables within _bt_getstackbuf(). This naming scheme is
clearer because the variable names always express a relationship between
the currently locked buffer/page and some other page.
M src/backend/access/nbtree/nbtsearch.c
Move description of libpqwalreceiver hooks out of the replication's README
commit : 641dd167a3424f97eed8416b5ef1c21dbbf8cf12
author : Michael Paquier <[email protected]>
date : Thu, 2 Jul 2020 13:57:03 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 2 Jul 2020 13:57:03 +0900
src/backend/replication/README includes since 32bc08b a basic
description of the WAL receiver hooks available in walreceiver.h for a
module like libpqwalreceiver, but the README has never been updated to
reflect changes done to the hooks, so the contents of the README have
rotten with the time. This commit moves the description from the README
to walreceiver.h, where it will be hard to miss that a description
update or addition is needed depending on the modifications done to the
hooks.
Each hook now includes a description of what it does in walreceiver.h,
and the replication's README mentions walreceiver.h.
Thanks also to Amit Kapila for the discussion.
Author: Michael Paquier
Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/README
M src/include/replication/walreceiver.h
Refactor ObjectAddress field assignments in more places
commit : 4315e8c23b9a897e12fcf91de7bfd734621096bf
author : Michael Paquier <[email protected]>
date : Wed, 1 Jul 2020 17:03:50 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 1 Jul 2020 17:03:50 +0900
This is a follow-up commit similar to 68de144, with more places in the
backend code simplified with the macros able to assign values to the
fields of ObjectAddress. The code paths changed here could be
transitioned later into using more grouping when inserting dependency
records, simplifying this future work.
Author: Daniel Gustafsson, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/catalog/pg_aggregate.c
M src/backend/catalog/pg_constraint.c
M src/backend/catalog/pg_operator.c
M src/backend/catalog/pg_proc.c
Improve vacuum error context handling.
commit : a69e041d0c91759fb60ab52e7e21e3ec3752c69b
author : Amit Kapila <[email protected]>
date : Wed, 1 Jul 2020 07:58:36 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 1 Jul 2020 07:58:36 +0530
Use separate functions to save and restore error context information as
that made code easier to understand. Also, make it clear that the index
information required for error context is sane.
Author: Andres Freund, Justin Pryzby, Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CAA4eK1LWo+v1OWu=Sky27GTGSCuOmr7iaURNbc5xz6jO+SaPeA@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
M src/tools/pgindent/typedefs.list
Refactor creation of normal dependency records when creating extension
commit : 684b4f29b729e5b135d256802d6e712222b9551f
author : Michael Paquier <[email protected]>
date : Wed, 1 Jul 2020 11:12:33 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 1 Jul 2020 11:12:33 +0900
When creating an extension, the same type of dependency is used when
registering a dependency to a schema and required extensions. This
improves the code so as those dependencies are not recorded one-by-one,
but grouped together. Note that this has as side effect to remove
duplicate dependency entries, even if it should not happen in practice
as extensions listed as required in a control file should be listed only
once.
Extracted from a larger patch by the same author.
Author: Daniel Dustafsson
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/extension.c
Fix removal of files generated by TAP tests for SSL
commit : c4342c932abb6792f6c7d8528b6224ca4d5e1a84
author : Michael Paquier <[email protected]>
date : Wed, 1 Jul 2020 10:47:24 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 1 Jul 2020 10:47:24 +0900
001_ssltests.pl and 002_scram.pl both generated an extra file for a
client key used in the tests that were not removed. In Debian, this
causes repeated builds to fail.
The code refactoring done in 4dc6355 broke the cleanup done in
001_ssltests.pl, and the new tests added in 002_scram.pl via d6e612f
forgot the removal of one file. While on it, fix a second issue
introduced in 002_scram.pl where we use the same file name in 001 and
002 for the temporary client key whose permissions are changed in the
test, as using the same file name in both tests could cause failures
with parallel jobs of src/test/ssl/ if one test removes a file still
needed by the second test.
Reported-by: Felix Lechner
Author: Daniel Gustafsson, Felix Lechner
Reviewed-by: Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/CAFHYt543sjX=Cm_aEeoejStyP47C+Y3+Wh6WbirLXsgUMaw7iw@mail.gmail.com
Backpatch-through: 13
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
Further adjustments to Hashagg EXPLAIN ANALYZE output
commit : 40efbf8706cdd96e06bc4d1754272e46d9857875
author : David Rowley <[email protected]>
date : Wed, 1 Jul 2020 12:15:59 +1200
committer: David Rowley <[email protected]>
date : Wed, 1 Jul 2020 12:15:59 +1200
The "Disk Usage" and "HashAgg Batches" properties in the EXPLAIN ANALYZE
output for HashAgg were previously only shown if the number of batches
was greater than 0. Here we change this so that these properties are
always shown for EXPLAIN ANALYZE formats other than "text". The idea here
is that since the HashAgg could have spilled to disk if there had been
more data or groups to aggregate, then it's relevant that we're clear in
the EXPLAIN ANALYZE output when no spilling occurred in this particular
execution of the given plan.
For the "text" EXPLAIN format, we still hide these properties when no
spilling occurs. This EXPLAIN format is designed to be easy for humans
to read. To maintain the readability we have a higher threshold for which
properties we display for this format.
Discussion: https://postgr.es/m/CAApHDvo_dmNozQQTmN-2jGp1vT%3Ddxx7Q0vd%2BMvD1cGpv2HU%3DSg%40mail.gmail.com
Backpatch-through: 13, where the hashagg spilling code was added.
M src/backend/commands/explain.c
Fix ecpg crash with bytea and cursor variables.
commit : e576f71fbe713d9357d1b480b4993043f7fcc73a
author : Michael Meskes <[email protected]>
date : Tue, 30 Jun 2020 17:31:08 +0200
committer: Michael Meskes <[email protected]>
date : Tue, 30 Jun 2020 17:31:08 +0200
Author: Jehan-Guillaume de Rorthais <[email protected]>
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/test/expected/sql-bytea.c
M src/interfaces/ecpg/test/expected/sql-bytea.stderr
M src/interfaces/ecpg/test/expected/sql-bytea.stdout
M src/interfaces/ecpg/test/sql/bytea.pgc
doc: clarify that storage parameter values are optional
commit : aa90d9957bd2d9ce39f9078d15914a37d99af072
author : Bruce Momjian <[email protected]>
date : Tue, 30 Jun 2020 12:26:51 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 30 Jun 2020 12:26:51 -0400
In a few cases, the documented syntax specified storage parameter values
as required.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/ref/alter_index.sgml
M doc/src/sgml/ref/alter_materialized_view.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_index.sgml
doc: change pg_upgrade wal_level to be not minimal
commit : 2a06cb86dbacdf19a19d5bfd00d5156584f2188a
author : Bruce Momjian <[email protected]>
date : Tue, 30 Jun 2020 11:55:53 -0400
committer: Bruce Momjian <[email protected]>
date : Tue, 30 Jun 2020 11:55:53 -0400
Previously it was specified to be only replica.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/ref/pgupgrade.sgml
Add +(pg_lsn,numeric) and -(pg_lsn,numeric) operators.
commit : 9bae7e4cde7c9786ee61dac4a3e032b346350a88
author : Fujii Masao <[email protected]>
date : Tue, 30 Jun 2020 23:55:07 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 30 Jun 2020 23:55:07 +0900
By using these operators, the number of bytes can be added into and
subtracted from LSN.
Bump catalog version.
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Michael Paquier, Asif Rehman
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/datatype.sgml
M src/backend/utils/adt/numeric.c
M src/backend/utils/adt/pg_lsn.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_proc.dat
M src/test/regress/expected/numeric.out
M src/test/regress/expected/pg_lsn.out
M src/test/regress/sql/numeric.sql
M src/test/regress/sql/pg_lsn.sql
Prevent compilation of frontend-only files in src/common/ with backend
commit : 324435eb14e4f41cd430f96c9b13ad9b160e45e4
author : Michael Paquier <[email protected]>
date : Tue, 30 Jun 2020 13:26:11 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 30 Jun 2020 13:26:11 +0900
Any frontend-only file of src/common/ should include a protection to
prevent such code to be included in the backend compilation.
fe_memutils.c and restricted_token.c have been doing that, while
file_utils.c (since bf5bb2e) and logging.c (since fc9a62a) forgot it.
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/common/file_utils.c
M src/common/logging.c
pgstattuple: Have pgstattuple_approx accept TOAST tables
commit : ee0202d552791f14bc407ce299628ce8d50eebe3
author : Peter Eisentraut <[email protected]>
date : Tue, 30 Jun 2020 00:29:35 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 30 Jun 2020 00:29:35 +0200
TOAST tables have a visibility map and a free space map, so they can
be supported by pgstattuple_approx just fine.
Add test cases to show how various pgstattuple functions accept TOAST
tables. Also add similar tests to pg_visibility, which already
accepted TOAST tables correctly but had no test coverage for them.
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/[email protected]
M contrib/pg_visibility/expected/pg_visibility.out
M contrib/pg_visibility/sql/pg_visibility.sql
M contrib/pgstattuple/expected/pgstattuple.out
M contrib/pgstattuple/pgstatapprox.c
M contrib/pgstattuple/sql/pgstattuple.sql
Remove support for timezone "posixrules" file.
commit : ea57e531b9487e042131ca1151a3ef5d655f40ec
author : Tom Lane <[email protected]>
date : Mon, 29 Jun 2020 18:55:01 -0400
committer: Tom Lane <[email protected]>
date : Mon, 29 Jun 2020 18:55:01 -0400
The IANA tzcode library has a feature to read a time zone file named
"posixrules" and apply the daylight-savings transition dates and times
therein, when it is given a POSIX-style time zone specification that
lacks an explicit transition rule. However, there's a problem with
that code: it doesn't work for dates past the Y2038 time_t rollover.
(Effectively, all times beyond that point are treated as standard
time.) The IANA crew regard this feature as legacy, so their plan is
to remove it not fix it. The time frame in which that will happen
is unclear, but presumably it'll happen well before 2038.
Moreover, effective with the next IANA data update (probably this
fall), the recommended default will be to not install a "posixrules"
file in the first place. The time frame in which tzdata packagers
might adopt that suggestion is likewise unclear, but at least some
platforms will probably do it in the next year or so. While we could
ignore that recommendation so far as PG-supplied tzdata trees are
concerned, builds using --with-system-tzdata will be subject to
whatever the platform's tzdata packager decides to do.
Thus, whether or not we do anything, some increasing fraction of
Postgres users will be exposed to the behavior observed when there
is no "posixrules" file; and if we do nothing, we'll have essentially
no control over the timing of that change.
The best thing to do to ameliorate the uncertainty seems to be to
proactively remove the posixrules-reading feature. If we do that in
a scheduled release then at least we can release-note the behavioral
change, rather than having users be surprised by it after a routine
tzdata update.
The change in question is fairly minor anyway: to be affected,
you have to be using a POSIX-style timezone spec, it has to not
have an explicit rule, and it has to not be one of the four traditional
continental-USA zone names (EST5EDT, CST6CDT, MST7MDT, or PST8PDT),
as those are special-cased. Since the default "posixrules" file
provides USA DST rules, the number of people who are likely to find
such a zone spec useful is probably quite small. Moreover, the
fallback behavior with no explicit rule and no "posixrules" file is to
apply current USA rules, so the only thing that really breaks is the
DST transitions in years before 2007 (and you get the countervailing
fix that transitions after 2038 will be applied).
Now, some installations might have replaced the "posixrules" file,
allowing e.g. EU rules to be applied to a POSIX-style timezone spec.
That won't work anymore. But it's not exactly clear why this solution
would be preferable to using a regular named zone. In any case, given
the Y2038 issue, we need to be pushing users to stop depending on this.
Back-patch into v13; it hasn't been released yet, so it seems OK to
change its behavior. (Personally I think we ought to back-patch
further, but I've been outvoted.)
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/datetime.sgml
M src/timezone/Makefile
M src/timezone/README
M src/timezone/localtime.c
M src/tools/msvc/Install.pm
Mop up some no-longer-necessary hacks around printf %.*s format.
commit : c410af098c46949e36607eb13689e697fa2def97
author : Tom Lane <[email protected]>
date : Mon, 29 Jun 2020 17:12:38 -0400
committer: Tom Lane <[email protected]>
date : Mon, 29 Jun 2020 17:12:38 -0400
Commit 54cd4f045 added some kluges to work around an old glibc bug,
namely that %.*s could misbehave if glibc thought any characters in
the supplied string were incorrectly encoded. Now that we use our
own snprintf.c implementation, we need not worry about that bug (even
if it still exists in the wild). Revert a couple of particularly
ugly hacks, and remove or improve assorted comments.
Note that there can still be encoding-related hazards here: blindly
clipping at a fixed length risks producing wrongly-encoded output
if the clip splits a multibyte character. However, code that's
doing correct multibyte-aware clipping doesn't really need a comment
about that, while code that isn't needs an explanation why not,
rather than a red-herring comment about an obsolete bug.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/copy.c
M src/backend/parser/scansup.c
M src/backend/tsearch/wparser_def.c
M src/backend/utils/adt/datetime.c
M src/backend/utils/adt/ruleutils.c
M src/fe_utils/print.c
M src/interfaces/ecpg/ecpglib/error.c
M src/interfaces/ecpg/pgtypeslib/dt_common.c
M src/interfaces/libpq/fe-misc.c
nbtree: Correct inaccurate split location comment.
commit : f7a476f0d656bbc673474b9165cd05fa548c18c9
author : Peter Geoghegan <[email protected]>
date : Mon, 29 Jun 2020 12:30:39 -0700
committer: Peter Geoghegan <[email protected]>
date : Mon, 29 Jun 2020 12:30:39 -0700
Minor oversight in commit fab25024338.
M src/backend/access/nbtree/nbtsplitloc.c
Avoid using %c printf format for potentially non-ASCII characters.
commit : 16e3ad5d143795b05a21dc887c2ab384cce4bcb8
author : Tom Lane <[email protected]>
date : Mon, 29 Jun 2020 11:41:19 -0400
committer: Tom Lane <[email protected]>
date : Mon, 29 Jun 2020 11:41:19 -0400
Since %c only passes a C "char" to printf, it's incapable of dealing
with multibyte characters. Passing just the first byte of such a
character leads to an output string that is visibly not correctly
encoded, resulting in undesirable behavior such as encoding conversion
failures while sending error messages to clients.
We've lived with this issue for a long time because it was inconvenient
to avoid in a portable fashion. However, now that we always use our own
snprintf code, it's reasonable to use the %.*s format to print just one
possibly-multibyte character in a string. (We previously avoided that
obvious-looking answer in order to work around glibc's bug #6530, cf
commits 54cd4f045 and ed437e2b2.)
Hence, run around and fix a bunch of places that used %c to report
a character found in a user-supplied string. For simplicity, I did
not touch places that were emitting non-user-facing debug messages,
or reporting catalog data that should always be ASCII. (It's also
unclear how useful this approach could be in frontend code, where
it's less certain that we know what encoding we're dealing with.)
In passing, improve a couple of poorly-written error messages in
pageinspect/heapfuncs.c.
This is a longstanding issue, but I'm hesitant to back-patch because
of the impact on translatable message strings. In any case this fix
would not work reliably before v12.
Tom Lane and Quan Zongliang
Discussion: https://postgr.es/m/[email protected]
M contrib/hstore/hstore_io.c
M contrib/pageinspect/heapfuncs.c
M src/backend/utils/adt/encode.c
M src/backend/utils/adt/jsonpath_gram.y
M src/backend/utils/adt/regexp.c
M src/backend/utils/adt/varbit.c
M src/backend/utils/adt/varlena.c
Add current substring regular expression syntax
commit : 78c887679d7632c1211f85eb95723f3226bf1b46
author : Peter Eisentraut <[email protected]>
date : Mon, 29 Jun 2020 11:04:42 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 29 Jun 2020 11:04:42 +0200
SQL:1999 had syntax
SUBSTRING(text FROM pattern FOR escapechar)
but this was replaced in SQL:2003 by the more clear
SUBSTRING(text SIMILAR pattern ESCAPE escapechar)
but this was never implemented in PostgreSQL. This patch adds that
new syntax as an alternative in the parser, and updates documentation
and tests to indicate that this is the preferred alternative now.
Reviewed-by: Pavel Stehule <[email protected]>
Reviewed-by: Vik Fearing <[email protected]>
Reviewed-by: Fabien COELHO <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/a15db31c-d0f8-8ce0-9039-578a31758adb%402ndquadrant.com
M contrib/citext/expected/citext.out
M contrib/citext/expected/citext_1.out
M contrib/citext/sql/citext.sql
M doc/src/sgml/func.sgml
M src/backend/catalog/information_schema.sql
M src/backend/parser/gram.y
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
Clean up grammar a bit
commit : aafefb4dcbf79e8cb1439e888a9cdb3dfefa7657
author : Peter Eisentraut <[email protected]>
date : Mon, 29 Jun 2020 10:36:52 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 29 Jun 2020 10:36:52 +0200
Simplify the grammar specification of substring() and overlay() a bit,
simplify and update some comments.
Reviewed-by: Pavel Stehule <[email protected]>
Reviewed-by: Vik Fearing <[email protected]>
Reviewed-by: Fabien COELHO <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/a15db31c-d0f8-8ce0-9039-578a31758adb%402ndquadrant.com
M src/backend/parser/gram.y
Refactor ObjectAddress field assignments for type dependencies
commit : 68de1440c79d75e529ff8c7395d698252370f992
author : Michael Paquier <[email protected]>
date : Mon, 29 Jun 2020 09:56:52 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 29 Jun 2020 09:56:52 +0900
The logic used to build the set of dependencies needed for a type is
rather repetitive with direct assignments for each ObjectAddress field.
This refactors the code to use the macro ObjectAddressSet() instead, to
do the same work. There are more areas of the backend code that could
use this macro, but these are left for a follow-up patch that will
partially rework the way dependencies are recorded as well. Type
dependencies are left out of the follow-up patch, so they are refactored
separately here.
Extracted from a larger patch by the same author.
Author: Daniel Gustafsson
Discussion: https://potgr.es/m/[email protected]
M src/backend/catalog/pg_type.c
Fix documentation of "must be vacuumed within" warning.
commit : 96879a0efb65b9cde0a688201516633aa79fd5b0
author : Noah Misch <[email protected]>
date : Sat, 27 Jun 2020 22:05:04 -0700
committer: Noah Misch <[email protected]>
date : Sat, 27 Jun 2020 22:05:04 -0700
Warnings start 10M transactions before xidStopLimit, which is 11M
transactions before wraparound. The sample WARNING output showed a
value greater than 11M, and its HINT message predated commit
25ec228ef760eb91c094cc3b6dea7257cc22ffb5. Hence, the sample was
impossible. Back-patch to 9.5 (all supported versions).
M doc/src/sgml/maintenance.sgml
Fix list of SSL error codes for older OpenSSL versions.
commit : e1cc25f59a8a90d821aaf894e1691575ed94454e
author : Tom Lane <[email protected]>
date : Sat, 27 Jun 2020 13:26:17 -0400
committer: Tom Lane <[email protected]>
date : Sat, 27 Jun 2020 13:26:17 -0400
Apparently 1.0.1 lacks SSL_R_VERSION_TOO_HIGH and
SSL_R_VERSION_TOO_LOW. Per buildfarm.
M src/backend/libpq/be-secure-openssl.c
M src/interfaces/libpq/fe-secure-openssl.c
Add hints about protocol-version-related SSL connection failures.
commit : b63dd3d88f479947ef7fb7cbf5db27de66ae0654
author : Tom Lane <[email protected]>
date : Sat, 27 Jun 2020 12:47:58 -0400
committer: Tom Lane <[email protected]>
date : Sat, 27 Jun 2020 12:47:58 -0400
OpenSSL's native reports about problems related to protocol version
restrictions are pretty opaque and inconsistent. When we get an
SSL error that is plausibly due to this, emit a hint message that
includes the range of SSL protocol versions we (think we) are
allowing. This should at least get the user thinking in the right
direction to resolve the problem, even if the hint isn't totally
accurate, which it might not be for assorted reasons.
Back-patch to v13 where we increased the default minimum protocol
version, thereby increasing the risk of this class of failure.
Patch by me, reviewed by Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/be-secure-openssl.c
M src/include/common/openssl.h
M src/interfaces/libpq/fe-secure-openssl.c
Change libpq's default ssl_min_protocol_version to TLSv1.2.
commit : 6e682f61a5bdb08164a805419144318db6b7229f
author : Tom Lane <[email protected]>
date : Sat, 27 Jun 2020 12:20:33 -0400
committer: Tom Lane <[email protected]>
date : Sat, 27 Jun 2020 12:20:33 -0400
When we initially created this parameter, in commit ff8ca5fad, we left
the default as "allow any protocol version" on grounds of backwards
compatibility. However, that's inconsistent with the backend's default
since b1abfec82; protocol versions prior to 1.2 are not considered very
secure; and OpenSSL has had TLSv1.2 support since 2012, so the number
of PG servers that need a lesser minimum is probably quite small.
On top of those things, it emerges that some popular distros (including
Debian and RHEL) set MinProtocol=TLSv1.2 in openssl.cnf. Thus, far
from having "allow any protocol version" behavior in practice, what
we actually have as things stand is a platform-dependent lower limit.
So, change our minds and set the min version to TLSv1.2. Anybody
wanting to connect with a new libpq to a pre-2012 server can either
set ssl_min_protocol_version=TLSv1 or accept the fallback to non-SSL.
Back-patch to v13 where the aforementioned patches appeared.
Patch by me, reviewed by Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-connect.c
Remove duplicate check added by commit b2a5545bd6.
commit : e7b476c657ebe4c0a47fa14b8f1c7ec767067585
author : Amit Kapila <[email protected]>
date : Sat, 27 Jun 2020 09:54:51 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 27 Jun 2020 09:54:51 +0530
As this doesn't cause any harm so we decided to this clean up in HEAD only.
Author: Ádám Balogh
Discussion: https://postgr.es/m/VI1PR0702MB36631BD67559461AFDE1FEEE81920@VI1PR0702MB3663.eurprd07.prod.outlook.com
M src/backend/access/transam/xlog.c
Persist slot invalidation correctly
commit : 4ae08cd5fd19d566538005c15e7bf992ebae4c72
author : Alvaro Herrera <[email protected]>
date : Fri, 26 Jun 2020 20:41:29 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 26 Jun 2020 20:41:29 -0400
We failed to save slot to disk after invalidating it, so the state was
lost in case of server restart or crash. Fix by marking it dirty and
flushing.
Also, if the slot is known invalidated we don't need to reason about the
LSN at all -- it's known invalidated. Only test the LSN if the slot is
known not invalidated.
Author: Fujii Masao <[email protected]>
Author: Kyotaro Horiguchi <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
Doc: explain that "timestamp - timestamp" applies justify_hours().
commit : eca08f58d05f45c4cae02bca5e1556ba58732fc4
author : Tom Lane <[email protected]>
date : Fri, 26 Jun 2020 13:54:01 -0400
committer: Tom Lane <[email protected]>
date : Fri, 26 Jun 2020 13:54:01 -0400
Back-patch to v13; before that, there's not really space for this
kind of detail.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
doc: mention trigger helper functions in CREATE TRIGGER docs
commit : 81d46ea12cef2391a4cae63c1f3951401e3dd883
author : Bruce Momjian <[email protected]>
date : Thu, 25 Jun 2020 18:33:28 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 25 Jun 2020 18:33:28 -0400
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/ref/create_trigger.sgml
docs: clarify that CREATE DATABASE does not copy db permissions
commit : d352de8d8eb7102e51e6adf0a965a9eae09e3f39
author : Bruce Momjian <[email protected]>
date : Thu, 25 Jun 2020 18:22:44 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 25 Jun 2020 18:22:44 -0400
That is, those database permissions set by GRANT.
Diagnosed-by: Joseph Nahmias
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/ref/create_database.sgml
Fix misuse of table_index_fetch_tuple_check().
commit : 10f1ab2cb8bea3c6741a78f6dc19a5c91c0a34e1
author : Peter Geoghegan <[email protected]>
date : Thu, 25 Jun 2020 10:55:28 -0700
committer: Peter Geoghegan <[email protected]>
date : Thu, 25 Jun 2020 10:55:28 -0700
Commit 0d861bbb, which added deduplication to nbtree, had
_bt_check_unique() pass a TID to table_index_fetch_tuple_check() that
isn't safe to mutate. table_index_fetch_tuple_check()'s tid argument is
modified when the TID in question is not the latest visible tuple in a
hot chain, though this wasn't documented.
To fix, go back to using a local copy of the TID in _bt_check_unique(),
and update comments above table_index_fetch_tuple_check().
Backpatch: 13-, where B-Tree deduplication was introduced.
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/table/tableam.c
M src/include/access/tableam.h
Doc: correct nitpicky mistakes in array_position/array_positions examples.
commit : 463b808e757928f053490dd397af77a80b4e7baa
author : Tom Lane <[email protected]>
date : Thu, 25 Jun 2020 13:28:30 -0400
committer: Tom Lane <[email protected]>
date : Thu, 25 Jun 2020 13:28:30 -0400
Daniel Gustafsson and Erik Rijkers, per report from nick@cleaton
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/array.sgml
Remove erroneous assertion from pg_copy_logical_replication_slot().
commit : a82ba066ea217e7fe4da3c20ced01e7ca976a351
author : Fujii Masao <[email protected]>
date : Thu, 25 Jun 2020 11:13:13 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 25 Jun 2020 11:13:13 +0900
If restart_lsn of logical replication slot gets behind more than
max_slot_wal_keep_size from the current LSN, the logical replication slot
would be invalidated and its restart_lsn is reset to an invalid LSN.
If this logical replication slot with an invalid restart_lsn was specified as
the source slot in pg_copy_logical_replication_slot(), the function caused
the assertion failure unexpectedly.
This assertion was added because restart_lsn should not be invalid before.
But in v13, it can be invalid thanks to max_slot_wal_keep_size. So since this
assertion is no longer useful, this commit removes it.
This commit also changes the errcode in the error message that
pg_copy_logical_replication_slot() emits when the slot with an invalid
restart_lsn is specified, to more appropriate one.
Back-patch to v13 where max_slot_wal_keep_size was added and
the assertion was no longer valid.
Author: Fujii Masao
Reviewed-by: Alvaro Herrera, Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/slotfuncs.c
Fix compiler warning induced by commit d8b15eeb8.
commit : 235c0f6eed2d9f5650f9b6ee0c51601792eff8e4
author : Tom Lane <[email protected]>
date : Wed, 24 Jun 2020 15:47:30 -0400
committer: Tom Lane <[email protected]>
date : Wed, 24 Jun 2020 15:47:30 -0400
I forgot that INT64_FORMAT can't be used with sscanf on Windows.
Use the same trick of sscanf'ing into a temp variable as we do in
some other places in zic.c.
The upstream IANA code avoids the portability problem by relying on
<inttypes.h>'s SCNdFAST64 macro. Once we're requiring C99 in all
branches, we should do likewise and drop this set of diffs from
upstream. For now, though, a hack seems fine, since we do not
actually care about leapseconds anyway.
Discussion: https://postgr.es/m/[email protected]
M src/timezone/zic.c
Adjust max_slot_wal_keep_size behavior per review
commit : b8fd4e02c6d01183bf6def5897ad6cf7766bfff4
author : Alvaro Herrera <[email protected]>
date : Wed, 24 Jun 2020 14:23:39 -0400
committer: Alvaro Herrera <[email protected]>
date : Wed, 24 Jun 2020 14:23:39 -0400
In pg_replication_slot, change output from normal/reserved/lost to
reserved/extended/unreserved/ lost, which better expresses the possible
states particularly near the time where segments are no longer safe but
checkpoint has not run yet.
Under the new definition, reserved means the slot is consuming WAL
that's still under the normal WAL size constraints; extended means it's
consuming WAL that's being protected by wal_keep_segments or the slot
itself, whose size is below max_slot_wal_keep_size; unreserved means the
WAL is no longer safe, but checkpoint has not yet removed those files.
Such as slot is in imminent danger, but can still continue for a little
while and may catch up to the reserved WAL space.
Also, there were some bugs in the calculations used to report the
status; fixed those.
Backpatch to 13.
Reported-by: Fujii Masao <[email protected]>
Author: Kyotaro Horiguchi <[email protected]>
Reviewed-by: Fujii Masao <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/catalogs.sgml
M src/backend/access/transam/xlog.c
M src/backend/replication/slotfuncs.c
M src/include/access/xlog.h
M src/test/recovery/t/019_replslot_limit.pl
Save slot's restart_lsn when invalidated due to size
commit : 0188bb82531f1b0ae3648fb81a4bd4a4f6242127
author : Alvaro Herrera <[email protected]>
date : Wed, 24 Jun 2020 14:15:17 -0400
committer: Alvaro Herrera <[email protected]>
date : Wed, 24 Jun 2020 14:15:17 -0400
We put it aside as invalidated_at, which let us show "lost" in
pg_replication slot. Prior to this change, the state value was reported
as NULL.
Backpatch to 13.
Author: Kyotaro Horiguchi <[email protected]>
Reviewed-by: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/include/access/xlog.h
M src/include/replication/slot.h
M src/test/recovery/t/019_replslot_limit.pl
Add parens to ConvertToXSegs macro
commit : 368d7f3297e7e1304da03904d2e1310d79fa82a9
author : Alvaro Herrera <[email protected]>
date : Wed, 24 Jun 2020 14:00:37 -0400
committer: Alvaro Herrera <[email protected]>
date : Wed, 24 Jun 2020 14:00:37 -0400
The current definition is dangerous. No bugs exist in our code at
present, but backpatch to 11 nonetheless where it was introduced.
Author: Álvaro Herrera <[email protected]>
M src/backend/access/transam/xlog.c
Fix comment in heap.c
commit : a3554b2d718520cbd16c13ff5c9f2e8257846170
author : Michael Paquier <[email protected]>
date : Wed, 24 Jun 2020 15:14:04 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 24 Jun 2020 15:14:04 +0900
The description of InsertPgAttributeTuple() does not match its handling
of pg_attribute contents with NULL values for a long time, with 911e702
making things more inconsistent. This adjusts the description to match
the reality.
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/heap.c
Doc fixup for hashagg_avoid_disk_plan GUC.
commit : 7ce461560159948ba0c802c767e42c5f5ae08b4a
author : Jeff Davis <[email protected]>
date : Mon, 22 Jun 2020 12:14:55 -0700
committer: Jeff Davis <[email protected]>
date : Mon, 22 Jun 2020 12:14:55 -0700
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backport-through: 13
M doc/src/sgml/config.sgml
Undo double-quoting of index names in non-text EXPLAIN output formats.
commit : 63d2ac23b018c2b173f42d274ae46b7b0c3263df
author : Tom Lane <[email protected]>
date : Mon, 22 Jun 2020 11:46:41 -0400
committer: Tom Lane <[email protected]>
date : Mon, 22 Jun 2020 11:46:41 -0400
explain_get_index_name() applied quote_identifier() to the index name.
This is fine for text output, but the non-text output formats all have
their own quoting conventions and would much rather start from the
actual index name. For example in JSON you'd get something like
"Index Name": "\"My Index\"",
which is surely not desirable, especially when the same does not
happen for table names. Hence, move the responsibility for applying
quoting out to the callers, where it can go into already-existing
special code paths for text format.
This changes the API spec for users of explain_get_index_name_hook:
before, they were supposed to apply quote_identifier() if necessary,
now they should not. Research suggests that the only publicly
available user of the hook is hypopg, and it actually forgot to
apply quoting anyway, so it's fine. (In any case, there's no
behavioral change for the output of a hook as seen in non-text
EXPLAIN formats, so this won't break any case that programs should
be relying on.)
Digging in the commit logs, it appears that quoting was included in
explain_get_index_name's duties when commit 604ffd280 invented it;
and that was fine at the time because we only had text output format.
This should have been rethought when non-text formats were invented,
but it wasn't.
This is a fairly clear bug for users of non-text EXPLAIN formats,
so back-patch to all supported branches.
Per bug #16502 from Maciek Sakrejda. Patch by me (based on
investigation by Euler Taveira); thanks to Julien Rouhaud for review.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/explain.c
Fix inconsistent markups in catalogs.sgml
commit : fe186b4c200b76a5c0f03379fe8645ed1c70a844
author : Michael Paquier <[email protected]>
date : Mon, 22 Jun 2020 13:40:04 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 22 Jun 2020 13:40:04 +0900
Some fields related to pg_opclass and pg_opfamily were using incorrect
markups, listing them as structname instead of structfield.
Author: Fabien Coelho
Discussion: https://postgr.es/m/alpine.DEB.2.22.394.2006210903560.859381@pseudo
M doc/src/sgml/catalogs.sgml
Add --no-index-cleanup and --no-truncate to vacuumdb.
commit : 9550ea3027aa4f290c998afd8836a927df40b09d
author : Michael Paquier <[email protected]>
date : Mon, 22 Jun 2020 13:23:38 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 22 Jun 2020 13:23:38 +0900
Both INDEX_CLEANUP and TRUNCATE have been available since v12, and are
enabled by default except if respectively vacuum_index_cleanup and
vacuum_truncate are disabled for a given relation. This change adds
support for disabling these options from vacuumdb.
Author: Nathan Bossart
Reviewed-by: Michael Paquier, Masahiko Sawada
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/vacuumdb.sgml
M src/bin/scripts/t/100_vacuumdb.pl
M src/bin/scripts/vacuumdb.c
Language fixes for docs related to opclass options
commit : 14903f238ec999802df5c539010c6be32d72f8cd
author : Alexander Korotkov <[email protected]>
date : Sun, 21 Jun 2020 04:48:03 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 21 Jun 2020 04:48:03 +0300
Discussion: https://postgr.es/m/20200620232145.GB17995%40telsasoft.com
Author: Justin Pryzby
Backpatch-through: 13
M doc/src/sgml/brin.sgml
M doc/src/sgml/btree.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/spgist.sgml
M doc/src/sgml/xindex.sgml
Doc: Tweak description of B-Tree duplicate tuples.
commit : 48c6959864491964ae3f893af8242d5770b3d3d2
author : Peter Geoghegan <[email protected]>
date : Sat, 20 Jun 2020 17:34:07 -0700
committer: Peter Geoghegan <[email protected]>
date : Sat, 20 Jun 2020 17:34:07 -0700
Defining duplicates as "close by" to each other was unclear. Simplify
the definition.
Backpatch: 13-, where deduplication was introduced (by commit 0d861bbb)
M doc/src/sgml/btree.sgml
Minor corrections to docs related to opclass options
commit : 3be015c9fc02316f1f1579301391fb5c67731057
author : Alexander Korotkov <[email protected]>
date : Sun, 21 Jun 2020 00:35:42 +0300
committer: Alexander Korotkov <[email protected]>
date : Sun, 21 Jun 2020 00:35:42 +0300
Reported-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-WzmwhYbxuoL0WjTLaiCxW3gj6qadeNpBhWAo_KZsE5-FGw%40mail.gmail.com
M doc/src/sgml/btree.sgml
M doc/src/sgml/spgist.sgml
Fix masking of SP-GiST pages during xlog consistency check
commit : a44dd932ff3816de7fe0414063cfcc5656117c3a
author : Alexander Korotkov <[email protected]>
date : Sat, 20 Jun 2020 17:34:51 +0300
committer: Alexander Korotkov <[email protected]>
date : Sat, 20 Jun 2020 17:34:51 +0300
spg_mask() didn't take into account that pd_lower equal to SizeOfPageHeaderData
is still valid value. This commit fixes that. Backpatch to 11, where
spg_mask() pg_lower check was introduced.
Reported-by: Michael Paquier
Discussion: https://postgr.es/m/20200615131405.GM52676%40paquier.xyz
Backpatch-through: 11
M src/backend/access/spgist/spgxlog.c
Add documentation for opclass options
commit : 15cb2bd27009f73a84a35c2ba60fdd105b4bf263
author : Alexander Korotkov <[email protected]>
date : Sat, 20 Jun 2020 13:34:54 +0300
committer: Alexander Korotkov <[email protected]>
date : Sat, 20 Jun 2020 13:34:54 +0300
911e7020770 added opclass options and adjusted documentation for each
particular affected opclass. However, documentation for extendability was
not adjusted. This commit adjusts documentation for interfaces of index AMs
and opclasses.
Discussion: https://postgr.es/m/CAH2-WzmQnW6%2Bz5F9AW%2BSz%2BzEcEvXofTwh_A9J3%3D_WA-FBP0wYg%40mail.gmail.com
Author: Alexander Korotkov
Reported-by: Peter Geoghegan
Reviewed-by: Peter Geoghegan
M doc/src/sgml/brin.sgml
M doc/src/sgml/btree.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/spgist.sgml
M doc/src/sgml/xindex.sgml
Remove dead forceSync parameter of XactLogCommitRecord().
commit : d28ab91e7155353d4377abad5a7d5b0f07450867
author : Noah Misch <[email protected]>
date : Sat, 20 Jun 2020 01:25:40 -0700
committer: Noah Misch <[email protected]>
date : Sat, 20 Jun 2020 01:25:40 -0700
The function has been reading global variable forceSyncCommit, mirroring
the intent of the caller that passed forceSync=forceSyncCommit. The
other caller, RecordTransactionCommitPrepared(), passed false. Since
COMMIT PREPARED can't share a transaction with any command, it certainly
doesn't share a transaction with a command that sets forceSyncCommit.
Reviewed by Michael Paquier.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xact.c
M src/include/access/xact.h
Removal unused function parameter in CopyReadBinaryAttribute.
commit : 74b4d78e037c1d4d55fd61f1b58c161e6249dfde
author : Amit Kapila <[email protected]>
date : Sat, 20 Jun 2020 09:18:57 +0530
committer: Amit Kapila <[email protected]>
date : Sat, 20 Jun 2020 09:18:57 +0530
The function parameter column_no is not used in CopyReadBinaryAttribute,
this can be removed.
Commit 0e319c7ad7 removed the usage of column_no parameter in function
CopyReadBinaryAttribute but forgot to remove the parameter.
Reported-by: Vignesh C
Author: Vignesh C
Discussion: https://postgr.es/m/CALDaNm1TYSNTfqx_jfz9_mwEZ2Er=dZnu++duXpC1uQo1cG=WA@mail.gmail.com
M src/backend/commands/copy.c
Ensure write failure reports no-disk-space
commit : ae3259c55067c926d25c745d70265fca15c2d26b
author : Alvaro Herrera <[email protected]>
date : Fri, 19 Jun 2020 16:46:07 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 19 Jun 2020 16:46:07 -0400
A few places calling fwrite and gzwrite were not setting errno to ENOSPC
when reporting errors, as is customary; this led to some failures being
reported as
"could not write file: Success"
which makes us look silly. Make a few of these places in pg_dump and
pg_basebackup use our customary pattern.
Backpatch-to: 9.5
Author: Justin Pryzby <[email protected]>
Author: Tom Lane <[email protected]>
Author: Álvaro Herrera <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_dump/pg_backup_directory.c
Future-proof regression tests against possibly-missing posixrules file.
commit : 2c8ef9363db199aed9e8f17edba866b1215803c6
author : Tom Lane <[email protected]>
date : Fri, 19 Jun 2020 13:55:21 -0400
committer: Tom Lane <[email protected]>
date : Fri, 19 Jun 2020 13:55:21 -0400
The IANA time zone folk have deprecated use of a "posixrules" file in
the tz database. While for now it's our choice whether to keep
supplying one in our own builds, installations built with
--with-system-tzdata will soon be needing to cope with that file not
being present, at least on some platforms.
This causes a problem for the horology test, which expected the
nonstandard POSIX zone spec "CST7CDT" to apply pre-2007 US daylight
savings rules. That does happen if the posixrules file supplies such
information, but otherwise the test produces undesired results.
To fix, add an explicit transition date rule that matches 2005 practice.
(We could alternatively have switched the test to use some real time
zone, but it seems useful to have coverage of this type of zone spec.)
While at it, update a documentation example that also relied on
"CST7CDT"; use a real-world zone name instead. Also, document why
the zone names EST5EDT, CST6CDT, MST7MDT, PST8PDT aren't subject to
similar failures when "posixrules" is missing.
Back-patch to all supported branches, since the hazard is the same
for all.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/datetime.sgml
M doc/src/sgml/func.sgml
M src/test/regress/expected/horology.out
M src/test/regress/sql/horology.sql
Adjust some glossary terms
commit : 816cbb59e3008112c5b217af7b9213b7a09881bf
author : Alvaro Herrera <[email protected]>
date : Fri, 19 Jun 2020 12:55:43 -0400
committer: Alvaro Herrera <[email protected]>
date : Fri, 19 Jun 2020 12:55:43 -0400
Mostly in response to Jürgen Purtz critique of previous definitions,
though I added many other changes.
Author: Álvaro Herrera <[email protected]>
Reviewed-by: Jürgen Purtz <[email protected]>
Reviewed-by: Justin Pryzby <[email protected]>
Reviewed-by: Erik Rijkers <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/glossary.sgml
Fix deduplication "single value" strategy bug.
commit : be14f884d57bc9c8ec8415edafea35ba5d31af59
author : Peter Geoghegan <[email protected]>
date : Fri, 19 Jun 2020 08:57:24 -0700
committer: Peter Geoghegan <[email protected]>
date : Fri, 19 Jun 2020 08:57:24 -0700
It was possible for deduplication's single value strategy to mistakenly
believe that a very small duplicate tuple counts as one of the six large
tuples that it aims to leave behind after the page finally splits. This
could cause slightly suboptimal space utilization with very low
cardinality indexes, though only under fairly narrow conditions.
To fix, be particular about what kind of tuple counts as a
maxpostingsize-capped tuple. This avoids confusion in the event of a
small tuple that gets "wedged" between two large tuples, where all
tuples on the page are duplicates of the same value.
Discussion: https://postgr.es/m/CAH2-Wz=Y+sgSFc-O3LpiZX-POx2bC+okec2KafERHuzdVa7-rQ@mail.gmail.com
Backpatch: 13-, where deduplication was introduced (by commit 0d861bbb)
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/access/nbtree/nbtxlog.c
M src/include/access/nbtree.h
Fix issues in invalidation of obsolete replication slots.
commit : f9e9704f09daf882f5a1cf1fbe3f5a3150ae2bb9
author : Fujii Masao <[email protected]>
date : Fri, 19 Jun 2020 17:15:52 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 19 Jun 2020 17:15:52 +0900
This commit fixes the following issues.
1. There is the case where the slot is dropped while trying to invalidate it.
InvalidateObsoleteReplicationSlots() did not handle this case, and
which could cause checkpoint to fail.
2. InvalidateObsoleteReplicationSlots() could emit the same log message
multiple times unnecessary. It should be logged only once.
3. When marking the slot as used, we always searched the target slot from
all the replication slots even if we already found it. This could cause
useless waste of cycles.
Back-patch to v13 where these issues were added as a part of
max_slot_wal_keep_size code.
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Alvaro Herrera
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/slot.c
Fix EXPLAIN ANALYZE for parallel HashAgg plans
commit : 9bdb300dedf086cc54edf740088208e6b24307ef
author : David Rowley <[email protected]>
date : Fri, 19 Jun 2020 17:24:27 +1200
committer: David Rowley <[email protected]>
date : Fri, 19 Jun 2020 17:24:27 +1200
Since 1f39bce02, HashAgg nodes have had the ability to spill to disk when
memory consumption exceeds work_mem. That commit added new properties to
EXPLAIN ANALYZE to show the maximum memory usage and disk usage, however,
it didn't quite go as far as showing that information for parallel
workers. Since workers may have experienced something very different from
the main process, we should show this information per worker, as is done
in Sort.
Reviewed-by: Justin Pryzby
Reviewed-by: Jeff Davis
Discussion: https://postgr.es/m/CAApHDvpEKbfZa18mM1TD7qV6PG+w97pwCWq5tVD0dX7e11gRJw@mail.gmail.com
Backpatch-through: 13, where the hashagg spilling code was added.
M src/backend/commands/explain.c
M src/backend/executor/execParallel.c
M src/backend/executor/nodeAgg.c
M src/include/executor/nodeAgg.h
M src/include/nodes/execnodes.h
Clean up includes of s_lock.h.
commit : f219167910ad33dfd8f1b0bba15323d71a91c4e9
author : Andres Freund <[email protected]>
date : Thu, 18 Jun 2020 19:40:09 -0700
committer: Andres Freund <[email protected]>
date : Thu, 18 Jun 2020 19:40:09 -0700
Users of spinlocks should use spin.h, not s_lock.h. And lwlock.h
hasn't utilized spinlocks for quite a while.
Discussion: https://postgr.es/m/[email protected]
M src/backend/main/main.c
M src/include/storage/condition_variable.h
M src/include/storage/lwlock.h
Fix deadlock danger when atomic ops are done under spinlock.
commit : cf1234a10e50ff9be0dc85184689ee4ebc57cd77
author : Andres Freund <[email protected]>
date : Mon, 8 Jun 2020 16:50:37 -0700
committer: Andres Freund <[email protected]>
date : Mon, 8 Jun 2020 16:50:37 -0700
This was a danger only for --disable-spinlocks in combination with
atomic operations unsupported by the current platform.
While atomics.c was careful to signal that a separate semaphore ought
to be used when spinlock emulation is active, spin.c didn't actually
implement that mechanism. That's my (Andres') fault, it seems to have
gotten lost during the development of the atomic operations support.
Fix that issue and add test for nesting atomic operations inside a
spinlock.
Author: Andres Freund
Discussion: https://postgr.es/m/[email protected]
Backpatch: 9.5-
M src/backend/storage/lmgr/spin.c
M src/test/regress/regress.c
Add basic spinlock tests to regression tests.
commit : 3b37a6de027c27f1e4ac865aaa34dd92cf5dc7a1
author : Andres Freund <[email protected]>
date : Mon, 8 Jun 2020 16:36:51 -0700
committer: Andres Freund <[email protected]>
date : Mon, 8 Jun 2020 16:36:51 -0700
As s_lock_test, the already existing test for spinlocks, isn't run in
an automated fashion (and doesn't test a normal backend environment),
adding tests that are run as part of a normal regression run is a good
idea. Particularly in light of several recent and upcoming spinlock
related fixes.
Currently the new tests are run as part of the pre-existing
test_atomic_ops() test. That perhaps can be quibbled about, but for
now seems ok.
The only operations that s_lock_test tests but the new tests don't are
the detection of a stuck spinlock and S_LOCK_FREE (which is otherwise
unused, not implemented on all platforms, and will be removed).
This currently contains a test for more than INT_MAX spinlocks (only
run with --disable-spinlocks), to ensure the recent commit fixing a
bug with more than INT_MAX spinlock initializations is correct. That
test is somewhat slow, so we might want to disable it after a few
days.
It might be worth retiring s_lock_test after this. The added coverage
of a stuck spinlock probably isn't worth the added complexity?
Author: Andres Freund
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/regress.c
Doc: document POSIX-style time zone specifications in full.
commit : a3235a53ae9f6f21f823081c610b0901db6aa665
author : Tom Lane <[email protected]>
date : Thu, 18 Jun 2020 16:27:18 -0400
committer: Tom Lane <[email protected]>
date : Thu, 18 Jun 2020 16:27:18 -0400
We'd glossed over most of this complexity for years, but it's hard
to avoid writing it all down now, so that we can explain what happens
when there's no "posixrules" file in the IANA time zone database.
That was at best a tiny minority situation till now, but it's likely
to become quite common in the future, so we'd better explain it.
Nonetheless, we don't really encourage people to use POSIX zone specs;
picking a named zone is almost always what you really want, unless
perhaps you're stuck with an out-of-date zone database. Therefore,
let's shove all this detail into an appendix.
Patch by me; thanks to Robert Haas for help with some awkward wording.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/datatype.sgml
M doc/src/sgml/datetime.sgml
Fix oldest xmin and LSN computation across repslots after advancing
commit : b48df818dcbd1a5e34ab7a2d9f98828b7b62140c
author : Michael Paquier <[email protected]>
date : Thu, 18 Jun 2020 16:34:59 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 18 Jun 2020 16:34:59 +0900
Advancing a replication slot did not recompute the oldest xmin and LSN
values across replication slots, preventing resource removal like
segments not recycled at checkpoint time. The original commit that
introduced the slot advancing in 9c7d06d never did the update of those
oldest values, and b0afdca removed this code.
This commit adds a TAP test to check segment recycling with advancing
for physical slots, enforcing an extra segment switch before advancing
to check if the segment gets correctly recycled after a checkpoint.
Reported-by: Andres Freund
Reviewed-by: Alexey Kondratov, Kyptaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M src/backend/replication/slotfuncs.c
M src/test/recovery/t/001_stream_rep.pl
Disallow factorial of negative numbers
commit : 0a40563eadc67472d6fd50dabf7002afa25c3330
author : Peter Eisentraut <[email protected]>
date : Thu, 18 Jun 2020 08:41:31 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 18 Jun 2020 08:41:31 +0200
The previous implementation returned 1 for all negative numbers, which
is not sensible under any definition.
Discussion: https://www.postgresql.org/message-id/flat/6ce1df0e-86a3-e544-743a-f357ff663f68%402ndquadrant.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
Expand tests for factorial
commit : 9d402c73ade412bdeb9064c81fc4ed071c4e93f8
author : Peter Eisentraut <[email protected]>
date : Thu, 18 Jun 2020 08:41:31 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 18 Jun 2020 08:41:31 +0200
Move from int4 to numeric test. (They were originally int4 functions,
but were reimplemented for numeric in
04a4821adef38155b7920ba9eb83c4c3c29156f8.) Add some tests for edge
cases.
Discussion: https://www.postgresql.org/message-id/flat/6ce1df0e-86a3-e544-743a-f357ff663f68%402ndquadrant.com
M src/test/regress/expected/int4.out
M src/test/regress/expected/numeric.out
M src/test/regress/sql/int4.sql
M src/test/regress/sql/numeric.sql
Remove reset of testtablespace from pg_regress on Windows
commit : 2b2a070d98b2f2c7ecc031e582cfefa400316ce3
author : Michael Paquier <[email protected]>
date : Thu, 18 Jun 2020 10:40:10 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 18 Jun 2020 10:40:10 +0900
testtablespace is an extra path used as tablespace location in the main
regression test suite, computed from --outputdir as defined by the
caller of pg_regress (current directory if undefined).
This special handling was introduced as of f10589e to be specific to
MSVC, as we let pg_regress' Makefile handle this cleanup in other
environments. This moves the cleanup to the MSVC script running
regression tests instead where needed: check, installcheck and
upgradecheck. I have also checked this patch on MSVC with repeated runs
of each target.
Author: Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/pg_regress.c
M src/tools/msvc/vcregress.pl
Sync our copy of the timezone library with IANA release tzcode2020a.
commit : d8b15eeb8a1acbe01b502ddd3390d7f1824c7a25
author : Tom Lane <[email protected]>
date : Wed, 17 Jun 2020 18:29:29 -0400
committer: Tom Lane <[email protected]>
date : Wed, 17 Jun 2020 18:29:29 -0400
This absorbs a leap-second-related bug fix in localtime.c, and
teaches zic to handle an expiration marker in the leapseconds file.
Neither are of any interest to us (for the foreseeable future
anyway), but we need to stay more or less in sync with upstream.
Also adjust some over-eager changes in the README from commit 957338418.
I have no intention of making changes that require C99 in this code,
until such time as all the live back branches require C99. Otherwise
back-patching will get too exciting.
For the same reason, absorb assorted whitespace and other cosmetic
changes from HEAD into the back branches; mostly this reflects use of
improved versions of pgindent.
All in all then, quite a boring update. But I figured I'd get it
done while I was looking at this code.
M src/timezone/README
M src/timezone/localtime.c
M src/timezone/zic.c
Fix nbtree.h dedup state comment.
commit : 6924c37f772cd7701d3e1267a1fb3221ca159ba4
author : Peter Geoghegan <[email protected]>
date : Wed, 17 Jun 2020 15:23:55 -0700
committer: Peter Geoghegan <[email protected]>
date : Wed, 17 Jun 2020 15:23:55 -0700
Oversight in commit 0d861bbb.
M src/include/access/nbtree.h
spinlock emulation: Fix bug when more than INT_MAX spinlocks are initialized.
commit : 4d4ca24efe8ebda9547337f47dcb61d3163be765
author : Andres Freund <[email protected]>
date : Mon, 8 Jun 2020 15:25:49 -0700
committer: Andres Freund <[email protected]>
date : Mon, 8 Jun 2020 15:25:49 -0700
Once the counter goes negative we ended up with spinlocks that errored
out on first use (due to check in tas_sema).
Author: Andres Freund
Reviewed-By: Robert Haas
Discussion: https://postgr.es/m/[email protected]
Backpatch: 9.5-
M src/backend/storage/lmgr/spin.c
Avoid potential spinlock in a signal handler as part of global barriers.
commit : fd49d53807575e009f7b66771d48c9356344d7d1
author : Andres Freund <[email protected]>
date : Mon, 15 Jun 2020 18:23:10 -0700
committer: Andres Freund <[email protected]>
date : Mon, 15 Jun 2020 18:23:10 -0700
On platforms without support for 64bit atomic operations where we also
cannot rely on 64bit reads to have single copy atomicity, such atomics
are implemented using a spinlock based fallback. That means it's not
safe to even read such atomics from within a signal handler (since the
signal handler might run when the spinlock already is held).
To avoid this issue defer global barrier processing out of the signal
handler. Instead of checking local / shared barrier generation to
determine whether to set ProcSignalBarrierPending, introduce
PROCSIGNAL_BARRIER and always set ProcSignalBarrierPending when
receiving such a signal. Additionally avoid redundant work in
ProcessProcSignalBarrier if ProcSignalBarrierPending is unnecessarily.
Also do a small amount of other polishing.
Author: Andres Freund
Reviewed-By: Robert Haas
Discussion: https://postgr.es/m/[email protected]
Backpatch: 13-, where the code was introduced.
M src/backend/storage/ipc/procsignal.c
M src/include/storage/procsignal.h
Improve server code to read files as part of a base backup.
commit : 2fd2effc50824a8775a088435a13f47b7a6f3b94
author : Robert Haas <[email protected]>
date : Wed, 17 Jun 2020 11:39:17 -0400
committer: Robert Haas <[email protected]>
date : Wed, 17 Jun 2020 11:39:17 -0400
Don't use fread(), since that doesn't necessarily set errno. We could
use read() instead, but it's even better to use pg_pread(), which
allows us to avoid some extra calls to seek to the desired location in
the file.
Also, advertise a wait event while reading from a file, as we do for
most other places where we're reading data from files.
Patch by me, reviewed by Hamid Akhtar.
Discussion: http://postgr.es/m/CA+TgmobBw-3573vMosGj06r72ajHsYeKtksT_oTxH8XvTL7DxA@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M src/backend/postmaster/pgstat.c
M src/backend/replication/basebackup.c
M src/include/pgstat.h
Minor code cleanup for perform_base_backup().
commit : 453e0e3f0ef3202386b553719f628cef93ff95a7
author : Robert Haas <[email protected]>
date : Wed, 17 Jun 2020 11:05:42 -0400
committer: Robert Haas <[email protected]>
date : Wed, 17 Jun 2020 11:05:42 -0400
Merge two calls to sendDir() that are exactly the same except for
the fifth argument. Adjust comments to match.
Also, don't bother checking whether tblspc_map_file is NULL. We
initialize it in all cases, so it can't be.
Patch by me, reviewed by Amit Kapila and Kyotaro Horiguchi.
Discussion: http://postgr.es/m/CA+TgmoYq+59SJ2zBbP891ngWPA9fymOqntqYcweSDYXS2a620A@mail.gmail.com
M src/backend/replication/basebackup.c
Don't export basebackup.c's sendTablespace().
commit : 1fa092913d260056b1aaf627ebc9cd9655c3a27c
author : Robert Haas <[email protected]>
date : Wed, 17 Jun 2020 10:57:34 -0400
committer: Robert Haas <[email protected]>
date : Wed, 17 Jun 2020 10:57:34 -0400
Commit 72d422a5227ef6f76f412486a395aba9f53bf3f0 made xlog.c call
sendTablespace() with the 'sizeonly' argument set to true, which
required basebackup.c to export sendTablespace(). However, that's
kind of ugly, so instead defer the call to sendTablespace() until
basebackup.c regains control. That way, it can still be a static
function.
Patch by me, reviewed by Amit Kapila and Kyotaro Horiguchi.
Discussion: http://postgr.es/m/CA+TgmoYq+59SJ2zBbP891ngWPA9fymOqntqYcweSDYXS2a620A@mail.gmail.com
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogfuncs.c
M src/backend/replication/basebackup.c
M src/include/access/xlog.h
M src/include/replication/basebackup.h
Remove STATUS_WAITING
commit : a513f1dfbf2c29a51b0f7cbd5913ce2d2ee452c5
author : Peter Eisentraut <[email protected]>
date : Wed, 17 Jun 2020 09:14:37 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 17 Jun 2020 09:14:37 +0200
Add a separate enum for use in the locking APIs, which were the only
user.
Discussion: https://www.postgresql.org/message-id/flat/a6f91ead-0ce4-2a34-062b-7ab9813ea308%402ndquadrant.com
M src/backend/access/transam/twophase.c
M src/backend/storage/lmgr/lock.c
M src/backend/storage/lmgr/proc.c
M src/include/c.h
M src/include/storage/proc.h
Doc: fix copy-and-pasteo in ecpg docs.
commit : 42aa1f0ab321fd43cbfdd875dd9e13940b485900
author : Tom Lane <[email protected]>
date : Tue, 16 Jun 2020 16:41:11 -0400
committer: Tom Lane <[email protected]>
date : Tue, 16 Jun 2020 16:41:11 -0400
The synopsis for PGTYPESinterval_free() used the wrong name.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ecpg.sgml
Lobotomize test for float -Inf ^ -2, at least for now.
commit : 3a990a1263560ef28707bc623e875e75ae42041c
author : Tom Lane <[email protected]>
date : Tue, 16 Jun 2020 15:28:37 -0400
committer: Tom Lane <[email protected]>
date : Tue, 16 Jun 2020 15:28:37 -0400
Per POSIX this case should produce +0, but buildfarm member fossa
(with icc (ICC) 19.0.5.281 20190815) is reporting -0. icc has a
boatload of unsafe floating-point optimizations, with a corresponding
boatload of not-too-well-documented compiler switches, and it seems our
default use of "-mp1" isn't whacking it hard enough to keep it from
misoptimizing the stanza in dpow() that checks whether y is odd.
There's nothing wrong with that code (seeing that no other buildfarm
member has trouble with it), so I'm content to blame this on the
compiler. But without access to the compiler I'm not going to guess at
what switches might be needed to fix it. For now, tweak the test case
so it will accept either -0 or +0 as a correct answer.
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/float8.out
M src/test/regress/sql/float8.sql
Fix file reference in nls.mk
commit : eab6e460e00d61e5aae95298a0fe8c79a1653184
author : Peter Eisentraut <[email protected]>
date : Tue, 16 Jun 2020 17:25:20 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 16 Jun 2020 17:25:20 +0200
Broken by move of fe_archive.c to fe_utils.
M src/bin/pg_rewind/nls.mk
In dpow(), remove redundant check for whether y is an integer.
commit : 400f16937328d50a7e8d7f0d158f7c92be1da45a
author : Tom Lane <[email protected]>
date : Tue, 16 Jun 2020 11:09:42 -0400
committer: Tom Lane <[email protected]>
date : Tue, 16 Jun 2020 11:09:42 -0400
I failed to notice that we don't really need to check for y being an
integer in the code path where x = -inf; we already did.
Also make some further cosmetic rearrangements in that spot in hopes
of dodging the seeming compiler bug that buildfarm member fossa is
hitting. And be consistent about declaring variables as "float8"
not "double", since the pre-existing variables in this function are
like that.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/float.c
Remove useless variable.
commit : 4dd804a99cd66d8d336ce7060c91e23db3263c31
author : Thomas Munro <[email protected]>
date : Tue, 16 Jun 2020 17:40:06 +1200
committer: Thomas Munro <[email protected]>
date : Tue, 16 Jun 2020 17:40:06 +1200
M src/backend/storage/file/buffile.c
Make BufFileWrite() void.
commit : f5d18862bbb327b67cd0f33f3d11351096080665
author : Thomas Munro <[email protected]>
date : Tue, 16 Jun 2020 17:23:36 +1200
committer: Thomas Munro <[email protected]>
date : Tue, 16 Jun 2020 17:23:36 +1200
It now either returns after it wrote all the data you gave it, or raises
an error. Not done in back-branches, because it might cause problems
for external code.
Discussion: https://postgr.es/m/CA%2BhUKGJE04G%3D8TLK0DLypT_27D9dR8F1RQgNp0jK6qR0tZGWOw%40mail.gmail.com
M src/backend/storage/file/buffile.c
M src/include/storage/buffile.h
Fix buffile.c error handling.
commit : 7897e3bb902c557412645b82120f4d95f7474906
author : Thomas Munro <[email protected]>
date : Tue, 16 Jun 2020 13:50:56 +1200
committer: Thomas Munro <[email protected]>
date : Tue, 16 Jun 2020 13:50:56 +1200
Convert buffile.c error handling to use ereport. This fixes cases where
I/O errors were indistinguishable from EOF or not reported. Also remove
"%m" from error messages where errno would be bogus. While we're
modifying those strings, add block numbers and short read byte counts
where appropriate.
Back-patch to all supported releases.
Reported-by: Amit Khandekar <[email protected]>
Reviewed-by: Melanie Plageman <[email protected]>
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Robert Haas <[email protected]>
Reviewed-by: Ibrar Ahmed <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJE04G%3D8TLK0DLypT_27D9dR8F1RQgNp0jK6qR0tZGWOw%40mail.gmail.com
M src/backend/access/gist/gistbuildbuffers.c
M src/backend/executor/nodeHashjoin.c
M src/backend/replication/backup_manifest.c
M src/backend/storage/file/buffile.c
M src/backend/utils/sort/logtape.c
M src/backend/utils/sort/sharedtuplestore.c
M src/backend/utils/sort/tuplestore.c
doc: Document factorial function
commit : 4c5cf5431410f8e28fb855c892cb2880649df850
author : Peter Eisentraut <[email protected]>
date : Mon, 15 Jun 2020 08:51:46 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 15 Jun 2020 08:51:46 +0200
This has existed for a very long time, equivalent to the ! and !!
operators, but it was never documented.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/6ce1df0e-86a3-e544-743a-f357ff663f68%402ndquadrant.com
M doc/src/sgml/func.sgml
pg_upgrade: set vacuum_defer_cleanup_age to zero
commit : 3f5863e15664757393c7a1416181fb58deac37a6
author : Bruce Momjian <[email protected]>
date : Mon, 15 Jun 2020 20:59:40 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 15 Jun 2020 20:59:40 -0400
Non-zero vacuum_defer_cleanup_age values cause pg_upgrade freezing of
the system catalogs to be incomplete, or do nothing. This will cause
the upgrade to fail in confusing ways.
Reported-by: Laurenz Albe
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M src/bin/pg_upgrade/server.c
Fix power() for large inputs yet more.
commit : 5674eb98762ed41c7f9672a3bc56edf925e9d97d
author : Tom Lane <[email protected]>
date : Mon, 15 Jun 2020 19:10:30 -0400
committer: Tom Lane <[email protected]>
date : Mon, 15 Jun 2020 19:10:30 -0400
Buildfarm results for commit e532b1d57 reveal the error in my thinking
about the unexpected-EDOM case. I'd supposed this was no longer really
a live issue, but it seems the fix for glibc's bug #3866 is not all that
old, and we still have at least one buildfarm animal (lapwing) with the
bug. Hence, resurrect essentially the previous logic (but, I hope, less
opaquely presented), and explain what it is we're really doing here.
Also, blindly try to fix fossa's failure by tweaking the logic that
figures out whether y is an odd integer when x is -inf. This smells
a whole lot like a compiler bug, but I lack access to icc to try to
pin it down. Maybe doing division instead of multiplication will
dodge the issue.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/float.c
Assorted cleanup of tar-related code.
commit : 2961c9711c17e5fe05fb1fbd72c3e47f3b601ee4
author : Robert Haas <[email protected]>
date : Fri, 24 Apr 2020 10:38:10 -0400
committer: Robert Haas <[email protected]>
date : Fri, 24 Apr 2020 10:38:10 -0400
Introduce TAR_BLOCK_SIZE and replace many instances of 512 with
the new constant. Introduce function tarPaddingBytesRequired
and use it to replace numerous repetitions of (x + 511) & ~511.
Add preprocessor guards against multiple inclusion to pgtar.h.
Reformat the prototype for tarCreateHeader so it doesn't extend
beyond 80 characters.
Discussion: http://postgr.es/m/CA+TgmobWbfReO9-XFk8urR1K4wTNwqoHx_v56t7=T8KaiEoKNw@mail.gmail.com
M src/backend/replication/basebackup.c
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_dump/pg_backup_tar.c
M src/include/pgtar.h
Fix power() for infinity inputs some more.
commit : e532b1d57df9b55557263303dba883e06521b0d2
author : Tom Lane <[email protected]>
date : Mon, 15 Jun 2020 12:15:56 -0400
committer: Tom Lane <[email protected]>
date : Mon, 15 Jun 2020 12:15:56 -0400
Buildfarm results for commit decbe2bfb show that AIX and illumos
have non-POSIX-compliant pow() functions, as do ancient NetBSD
and HPUX releases. While it's dubious how much we should care
about the latter two platforms, the former two are probably enough
reason to put in manual handling of infinite-input cases. Hence,
do so, and clean up the post-pow() error handling to reflect its
now-more-limited scope. (Notably, while we no longer expect to
ever see EDOM from pow(), report it as a domain error if we do.
The former coding had the net effect of expensively converting the
error to ERANGE, which seems highly questionable: if pow() wanted
to report ERANGE, it would have done so.)
Patch by me; thanks to Michael Paquier for review.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/float.c
M src/test/regress/expected/float8.out
M src/test/regress/sql/float8.sql
Fix some comments referring to past features
commit : 7a3543c2ea23d032865f207c2605942b7a32e9ba
author : Michael Paquier <[email protected]>
date : Mon, 15 Jun 2020 21:18:14 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 15 Jun 2020 21:18:14 +0900
Timestamp can only be an int64 since b9d092c, and support for WITH OIDS
has been removed as of 578b229.
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/common/tupdesc.c
M src/backend/commands/tablecmds.c
M src/backend/utils/adt/selfuncs.c
pg_dump: Unbreak dumping of aggregates from very old server versions
commit : 3baa7e38d51579b69e1228f3e1a43f56001b6d64
author : Peter Eisentraut <[email protected]>
date : Mon, 15 Jun 2020 10:47:59 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 15 Jun 2020 10:47:59 +0200
Recently broken by d9fa17aa7c34dea66ce64da6fb4c643e75ba452c.
M src/bin/pg_dump/pg_dump.c
Error message refactoring
commit : 47d4d0cfad6921fe1f6aabacc8a7a328ce9acb76
author : Peter Eisentraut <[email protected]>
date : Mon, 15 Jun 2020 08:22:52 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 15 Jun 2020 08:22:52 +0200
Take some untranslatable things out of the message and replace by
format placeholders, to reduce translatable strings and reduce
translation mistakes.
M src/bin/pg_basebackup/pg_basebackup.c
M src/interfaces/libpq/fe-connect.c
Bump catversion for ACL changes on replication origin functions
commit : 46241b28d84551bf6aeec3aa7e2391cda2757a21
author : Michael Paquier <[email protected]>
date : Mon, 15 Jun 2020 15:26:51 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 15 Jun 2020 15:26:51 +0900
Oversight in cc07264.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/include/catalog/catversion.h
Doc: Add references for SI and SSI.
commit : 1575fbf1ac3c8464b2dade245deff537a3aa2498
author : Thomas Munro <[email protected]>
date : Mon, 15 Jun 2020 11:33:13 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 15 Jun 2020 11:33:13 +1200
Our documentation failed to point out that REPEATABLE READ is really
snapshot isolation, which might be important to some users. Point to
the standard reference paper for this complicated topic.
Likewise, add a reference to the VLDB paper about PostgreSQL SSI, for
technical information about our SSI implementation and how it compares
to S2PL.
While here, add a note about catalog access using a lower isolation
level, per recent user complaint.
Back-patch to all releases.
Reported-by: Kyle Kingsbury <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Peter Geoghegan <[email protected]>
Reviewed-by: Tatsuo Ishii <[email protected]>
Discussion: https://postgr.es/m/db7b729d-0226-d162-a126-8a8ab2dc4443%40jepsen.io
Discussion: https://postgr.es/m/16454-9408996bb1750faf%40postgresql.org
M doc/src/sgml/biblio.sgml
M doc/src/sgml/mvcc.sgml
Fix behavior of exp() and power() for infinity inputs.
commit : decbe2bfb1051c5ab6c382b19e1d909e34227a22
author : Tom Lane <[email protected]>
date : Sun, 14 Jun 2020 11:00:07 -0400
committer: Tom Lane <[email protected]>
date : Sun, 14 Jun 2020 11:00:07 -0400
Previously, these functions tended to throw underflow errors for
negative-infinity exponents. The correct thing per POSIX is to
return 0, so let's do that instead. (Note that the SQL standard
is silent on such issues, as it lacks the concepts of either Inf
or NaN; so our practice is to follow POSIX whenever a corresponding
C-library function exists.)
Also, add a bunch of test cases verifying that exp() and power()
actually do follow POSIX for Inf and NaN inputs. While this patch
should guarantee that exp() passes the tests, power() will not unless
the platform's pow(3) is fully POSIX-compliant. I already know that
gaur fails some of the tests, and I am suspicious that the Windows
animals will too; the extent of compliance of other old platforms
remains to be seen. We might choose to drop failing test cases, or
to work harder at overriding pow(3) for these cases, but first let's
see just how good or bad the situation is.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/float.c
M src/test/regress/expected/float8.out
M src/test/regress/sql/float8.sql
Add test coverage for EXTRACT()
commit : 378badc8ebebc8fece7a18001f6b876cc00b12c0
author : Peter Eisentraut <[email protected]>
date : Sun, 14 Jun 2020 07:48:15 +0200
committer: Peter Eisentraut <[email protected]>
date : Sun, 14 Jun 2020 07:48:15 +0200
The variants for time and timetz had zero test coverage, the variant
for interval only very little. This adds practically full coverage
for those functions.
Reviewed-by: Vik Fearing <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/c3306ac7-fcae-a1b8-1e30-6a379d605bcb%402ndquadrant.com
M src/test/regress/expected/date.out
M src/test/regress/expected/interval.out
M src/test/regress/expected/time.out
M src/test/regress/expected/timetz.out
M src/test/regress/sql/date.sql
M src/test/regress/sql/interval.sql
M src/test/regress/sql/time.sql
M src/test/regress/sql/timetz.sql
Replace superuser check by ACLs for replication origin functions
commit : cc072641d41c55c6aa24a331fc1f8029e0a8d799
author : Michael Paquier <[email protected]>
date : Sun, 14 Jun 2020 12:40:37 +0900
committer: Michael Paquier <[email protected]>
date : Sun, 14 Jun 2020 12:40:37 +0900
This patch removes the hardcoded check for superuser privileges when
executing replication origin functions. Instead, execution is revoked
from public, meaning that those functions can be executed by a superuser
and that access to them can be granted.
Author: Martín Marqués
Reviewed-by: Kyotaro Horiguchi, Michael Paquier, Masahiko Sawada
Discussion: https:/postgr.es/m/CAPdiE1xJMZOKQL3dgHMUrPqysZkgwzSMXETfKkHYnBAB7-0VRQ@mail.gmail.com
M contrib/test_decoding/expected/replorigin.out
M contrib/test_decoding/sql/replorigin.sql
M doc/src/sgml/func.sgml
M src/backend/catalog/system_views.sql
M src/backend/replication/logical/origin.c
Sync behavior of var_samp and stddev_samp for single NaN inputs.
commit : 23cbeda50b94c817bed4f7d2127ee09c4e8c8b86
author : Tom Lane <[email protected]>
date : Sat, 13 Jun 2020 14:01:46 -0400
committer: Tom Lane <[email protected]>
date : Sat, 13 Jun 2020 14:01:46 -0400
var_samp(numeric) and stddev_samp(numeric) disagreed with their float
cousins about what to do for a single non-null input value that is NaN.
The float versions return NULL on the grounds that the calculation is
only defined for more than one non-null input, which seems like the
right answer. But the numeric versions returned NaN, as a result of
dealing with edge cases in the wrong order. Fix that. The patch
also gets rid of an insignificant memory leak in such cases.
This inconsistency is of long standing, but on the whole it seems best
not to back-patch the change into stable branches; nobody's complained
and it's such an obscure point that nobody's likely to complain.
(Note that v13 and v12 now contain test cases that will notice if we
accidentally back-patch this behavior change in future.)
Report and patch by me; thanks to Dean Rasheed for review.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/aggregates.out
Fix behavior of float aggregates for single Inf or NaN inputs.
commit : 03109a53020e4663df3a8d822cdc665a7d712a93
author : Tom Lane <[email protected]>
date : Sat, 13 Jun 2020 13:43:24 -0400
committer: Tom Lane <[email protected]>
date : Sat, 13 Jun 2020 13:43:24 -0400
When there is just one non-null input value, and it is infinity or NaN,
aggregates such as stddev_pop and covar_pop should produce a NaN
result, because the calculation is not well-defined. They used to do
so, but since we adopted Youngs-Cramer aggregation in commit e954a727f,
they produced zero instead. That's an oversight, so fix it. Add tests
exercising these edge cases.
Affected aggregates are
var_pop(double precision)
stddev_pop(double precision)
var_pop(real)
stddev_pop(real)
regr_sxx(double precision,double precision)
regr_syy(double precision,double precision)
regr_sxy(double precision,double precision)
regr_r2(double precision,double precision)
regr_slope(double precision,double precision)
regr_intercept(double precision,double precision)
covar_pop(double precision,double precision)
corr(double precision,double precision)
Back-patch to v12 where the behavior change was accidentally introduced.
Report and patch by me; thanks to Dean Rasheed for review.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/float.c
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql
Silence _bt_check_unique compiler warning.
commit : d64f1cdf2f4bef1454c74af1028c9ea0c3280322
author : Peter Geoghegan <[email protected]>
date : Sat, 13 Jun 2020 09:33:33 -0700
committer: Peter Geoghegan <[email protected]>
date : Sat, 13 Jun 2020 09:33:33 -0700
Reported-By: Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/nbtree/nbtinsert.c
Refactor AlterExtensionContentsStmt grammar
commit : 8f5b5967441f05e56446fa4cdeffd0774c01e553
author : Peter Eisentraut <[email protected]>
date : Sat, 13 Jun 2020 09:03:28 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 13 Jun 2020 09:03:28 +0200
Make use of the general object support already used by COMMENT, DROP,
and SECURITY LABEL.
Discussion: https://www.postgresql.org/message-id/flat/163c00a5-f634-ca52-fc7c-0e53deda8735%402ndquadrant.com
M src/backend/commands/extension.c
M src/backend/parser/gram.y
M src/test/modules/test_pg_dump/expected/test_pg_dump.out
Grammar object type refactoring
commit : a332b366d4fa19ee3578a864993a8dc7abb47177
author : Peter Eisentraut <[email protected]>
date : Sat, 13 Jun 2020 09:03:28 +0200
committer: Peter Eisentraut <[email protected]>
date : Sat, 13 Jun 2020 09:03:28 +0200
Unify the grammar of COMMENT, DROP, and SECURITY LABEL further. They
all effectively just take an object address for later processing, so
we can make the grammar more generalized. Some extra checking about
which object types are supported can be done later in the statement
execution.
Discussion: https://www.postgresql.org/message-id/flat/163c00a5-f634-ca52-fc7c-0e53deda8735%402ndquadrant.com
M src/backend/commands/seclabel.c
M src/backend/parser/gram.y
Create by default sql/ and expected/ for output directory in pg_regress
commit : e78900afd217fa3eaa77c51e23a94c1466af421c
author : Michael Paquier <[email protected]>
date : Sat, 13 Jun 2020 14:04:56 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 13 Jun 2020 14:04:56 +0900
Using --outputdir with a custom output repository has never created by
default the sql/ and expected/ paths generated with contents from
respectively input/ and output/ if they don't exist, while the base
output directory gets created if it does not exist. If sql/ and
expected/ are not present, pg_regress would fail with the path missing,
requiring test scripts to create those extra paths by themselves. This
commit changes pg_regress so as both get created by default if they do
not exist, removing the need for external test scripts to do so.
This cleans up two code paths in the tree for pg_upgrade tests in MSVC
and environments able to use test.sh. sql/ and expected/ were created
as part of each test script, but this is not needed anymore as
pg_regress handles the work now.
Author: Roman Zharkov, Daniel Gustafsson
Reviewed-by: Michael Paquier, Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_upgrade/test.sh
M src/test/regress/pg_regress.c
M src/tools/msvc/vcregress.pl
Add more TAP tests for pg_dump options with range checks
commit : 64725728e790b76c97984b1029d9ffe90bcb2ec4
author : Michael Paquier <[email protected]>
date : Sat, 13 Jun 2020 09:34:38 +0900
committer: Michael Paquier <[email protected]>
date : Sat, 13 Jun 2020 09:34:38 +0900
This adds two tests for --extra-float-digits and --rows-per-insert,
similar to what exists for --compress.
Author: Dong Wook Lee
Discussion: https://postgr.es/m/CAAcByaJsgrB-qc-ALb0mALprRGLAdmcBap7SZxO4kCAU-JEHcQ@mail.gmail.com
M src/bin/pg_dump/t/001_basic.pl
Have pg_itoa, pg_ltoa and pg_lltoa return the length of the string
commit : dad75eb4a8d5835ecc795d7a7978e7702e4d5912
author : David Rowley <[email protected]>
date : Sat, 13 Jun 2020 12:32:00 +1200
committer: David Rowley <[email protected]>
date : Sat, 13 Jun 2020 12:32:00 +1200
Core by no means makes excessive use of these functions, but quite a large
number of those usages do require the caller to call strlen() on the
returned string. This is quite wasteful since these functions do already
have a good idea of the length of the string, so we might as well just
have them return that.
Reviewed-by: Andrew Gierth
Discussion: https://postgr.es/m/CAApHDvrm2A5x2uHYxsqriO2cUaGcFvND%2BksC9e7Tjep0t2RK_A%40mail.gmail.com
M src/backend/access/common/printsimple.c
M src/backend/utils/adt/int.c
M src/backend/utils/adt/int8.c
M src/backend/utils/adt/numutils.c
M src/include/utils/builtins.h
Add missing extern keyword for a couple of numutils functions
commit : 9a7fccd9eac85726ced3f3794a743eeab447f334
author : David Rowley <[email protected]>
date : Sat, 13 Jun 2020 11:27:25 +1200
committer: David Rowley <[email protected]>
date : Sat, 13 Jun 2020 11:27:25 +1200
In passing, also remove a few surplus empty lines from pg_ltoa and
pg_ulltoa_n in numutils.c
Reported-by: Andrew Gierth
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13, where these changes were introduced
M src/backend/utils/adt/numutils.c
M src/include/utils/builtins.h
Avoid using a cursor in plpgsql's RETURN QUERY statement.
commit : 2f48ede080f42b97b594fb14102c82ca1001b80c
author : Tom Lane <[email protected]>
date : Fri, 12 Jun 2020 12:14:32 -0400
committer: Tom Lane <[email protected]>
date : Fri, 12 Jun 2020 12:14:32 -0400
plpgsql has always executed the query given in a RETURN QUERY command
by opening it as a cursor and then fetching a few rows at a time,
which it turns around and dumps into the function's result tuplestore.
The point of this was to keep from blowing out memory with an oversized
SPITupleTable result (note that while a tuplestore can spill tuples
to disk, SPITupleTable cannot). However, it's rather inefficient, both
because of extra data copying and because of executor entry/exit
overhead. In recent versions, a new performance problem has emerged:
use of a cursor prevents use of a parallel plan for the executed query.
We can improve matters by skipping use of a cursor and having the
executor push result tuples directly into the function's result
tuplestore. However, a moderate amount of new infrastructure is needed
to make that idea work:
* We can use the existing tstoreReceiver.c DestReceiver code to funnel
executor output to the tuplestore, but it has to be extended to support
plpgsql's requirement for possibly applying a tuple conversion map.
* SPI needs to be extended to allow use of a caller-supplied
DestReceiver instead of its usual receiver that puts tuples into
a SPITupleTable. Two new API calls are needed to handle both the
RETURN QUERY and RETURN QUERY EXECUTE cases.
I also felt that I didn't want these new API calls to use the legacy
method of specifying query parameter values with "char" null flags
(the old ' '/'n' convention); rather they should accept ParamListInfo
objects containing the parameter type and value info. This required
a bit of additional new infrastructure since we didn't yet have any
parse analysis callback that would interpret $N parameter symbols
according to type data supplied in a ParamListInfo. There seems to be
no harm in letting makeParamList install that callback by default,
rather than leaving a new ParamListInfo's parserSetup hook as NULL.
(Indeed, as of HEAD, I couldn't find anyplace that was using the
parserSetup field at all; plpgsql was using parserSetupArg for its
own purposes, but parserSetup seemed to be write-only.)
We can actually get plpgsql out of the business of using legacy null
flags altogether, and using ParamListInfo instead of its ad-hoc
PreparedParamsData structure; but this requires inventing one more
SPI API call that can replace SPI_cursor_open_with_args. That seems
worth doing, though.
SPI_execute_with_args and SPI_cursor_open_with_args are now unused
anywhere in the core PG distribution. Perhaps someday we could
deprecate/remove them. But cleaning up the crufty bits of the SPI
API is a task for a different patch.
Per bug #16040 from Jeremy Smith. This is unfortunately too invasive to
consider back-patching. Patch by me; thanks to Hamid Akhtar for review.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/spi.sgml
M src/backend/commands/portalcmds.c
M src/backend/executor/spi.c
M src/backend/executor/tstoreReceiver.c
M src/backend/nodes/params.c
M src/backend/tcop/pquery.c
M src/include/executor/spi.h
M src/include/executor/tstoreReceiver.h
M src/pl/plpgsql/src/pl_exec.c
Fix typos and some format mistakes in comments
commit : aaf8c990502f7bb28c10f6bab1d23fe2f9f0b537
author : Michael Paquier <[email protected]>
date : Fri, 12 Jun 2020 21:05:10 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 12 Jun 2020 21:05:10 +0900
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/origin.c
M src/backend/replication/logical/relation.c
M src/backend/utils/mmgr/README
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_custom.c
Make more use of RELKIND_HAS_STORAGE()
commit : ffd2582297b86f640b60ba46097b70743f920d35
author : Peter Eisentraut <[email protected]>
date : Fri, 12 Jun 2020 08:51:16 +0200
committer: Peter Eisentraut <[email protected]>
date : Fri, 12 Jun 2020 08:51:16 +0200
Make use of RELKIND_HAS_STORAGE() where appropriate, instead of
listing out the relkinds individually. No behavior change intended.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/7a22bf51-2480-d999-1794-191ba67ff47c%402ndquadrant.com
M src/backend/catalog/heap.c
M src/backend/postmaster/pgstat.c
M src/backend/utils/adt/dbsize.c
Improve comments for [Heap]CheckForSerializableConflictOut().
commit : 7aa4fb592530b74bf05f62c52b736ee3910691b9
author : Thomas Munro <[email protected]>
date : Fri, 12 Jun 2020 10:44:32 +1200
committer: Thomas Munro <[email protected]>
date : Fri, 12 Jun 2020 10:44:32 +1200
Rewrite the documentation of these functions, in light of recent bug fix
commit 5940ffb2.
Back-patch to 13 where the check-for-conflict-out code was split up into
AM-specific and generic parts, and new documentation was added that now
looked wrong.
Reviewed-by: Peter Geoghegan <[email protected]>
Discussion: https://postgr.es/m/db7b729d-0226-d162-a126-8a8ab2dc4443%40jepsen.io
M src/backend/access/heap/heapam.c
M src/backend/storage/lmgr/predicate.c
doc: document problems with using xreflabel in XML docs
commit : 59fa7eb60364b8e71fe0f515d6ad87c274af839b
author : Bruce Momjian <[email protected]>
date : Thu, 11 Jun 2020 18:44:49 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 11 Jun 2020 18:44:49 -0400
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: master
M doc/src/sgml/README.links
doc: remove xreflabels from commits 75fcdd2ae2 and 85af628da5
commit : 0dd1eb3aea636c0b28a18e1290ef108629e5eddd
author : Bruce Momjian <[email protected]>
date : Thu, 11 Jun 2020 18:19:25 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 11 Jun 2020 18:19:25 -0400
xreflabels prevent references to the chapter numbers of sections id's.
It should only be used in specific cases.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/geqo.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/plperl.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/plpython.sgml
M doc/src/sgml/spgist.sgml
M doc/src/sgml/spi.sgml
M doc/src/sgml/storage.sgml
M doc/src/sgml/vacuumlo.sgml
Fix mishandling of NaN counts in numeric_[avg_]combine.
commit : 77a3be32f7c16538bc4e05edad85560d9f88369b
author : Tom Lane <[email protected]>
date : Thu, 11 Jun 2020 17:38:42 -0400
committer: Tom Lane <[email protected]>
date : Thu, 11 Jun 2020 17:38:42 -0400
When merging two NumericAggStates, the code missed adding the new
state's NaNcount unless its N was also nonzero; since those counts
are independent, this is wrong.
This would only have visible effect if some partial aggregate scans
found only NaNs while earlier ones found only non-NaNs; then we could
end up falsely deciding that there were no NaNs and fail to return a
NaN final result as expected. That's pretty improbable, so it's no
surprise this hasn't been reported from the field. Still, it's a bug.
I didn't try to produce a regression test that would show the bug,
but I did notice that these functions weren't being reached at all
in our regression tests, so I improved the tests to at least
exercise them. With these additions, I see pretty complete code
coverage on the aggregation-related functions in numeric.c.
Back-patch to 9.6 where this code was introduced. (I only added
the improved test case as far back as v10, though, since the
relevant part of aggregates.sql isn't there at all in 9.6.)
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql
Rework HashAgg GUCs.
commit : 92c58fd94801dd5c81ee20e26c5bb71ad64552a8
author : Jeff Davis <[email protected]>
date : Thu, 11 Jun 2020 11:58:16 -0700
committer: Jeff Davis <[email protected]>
date : Thu, 11 Jun 2020 11:58:16 -0700
Eliminate enable_groupingsets_hash_disk, which was primarily useful
for testing grouping sets that use HashAgg and spill. Instead, hack
the table stats to convince the planner to choose hashed aggregation
for grouping sets that will spill to disk. Suggested by Melanie
Plageman.
Rename enable_hashagg_disk to hashagg_avoid_disk_plan, and invert the
meaning of on/off. The new name indicates more strongly that it only
affects the planner. Also, the word "avoid" is less definite, which
should avoid surprises when HashAgg still needs to use the
disk. Change suggested by Justin Pryzby, though I chose a different
GUC name.
Discussion: https://postgr.es/m/CAAKRu_aisiENMsPM2gC4oUY1hHG3yrCwY-fXUg22C6_MJUwQdA%40mail.gmail.com
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M doc/src/sgml/config.sgml
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/utils/misc/guc.c
M src/include/optimizer/cost.h
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/groupingsets.out
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/groupingsets.sql
Avoid update conflict out serialization anomalies.
commit : 5940ffb221316ab73e6fdc780dfe9a07d4221ebb
author : Peter Geoghegan <[email protected]>
date : Thu, 11 Jun 2020 10:09:47 -0700
committer: Peter Geoghegan <[email protected]>
date : Thu, 11 Jun 2020 10:09:47 -0700
SSI's HeapCheckForSerializableConflictOut() test failed to correctly
handle conditions involving a concurrently inserted tuple which is later
concurrently updated by a separate transaction . A SELECT statement
that called HeapCheckForSerializableConflictOut() could end up using the
same XID (updater's XID) for both the original tuple, and the successor
tuple, missing the XID of the xact that created the original tuple
entirely. This only happened when neither tuple from the chain was
visible to the transaction's MVCC snapshot.
The observable symptoms of this bug were subtle. A pair of transactions
could commit, with the later transaction failing to observe the effects
of the earlier transaction (because of the confusion created by the
update to the non-visible row). This bug dates all the way back to
commit dafaa3ef, which added SSI.
To fix, make sure that we check the xmin of concurrently inserted tuples
that happen to also have been updated concurrently.
Author: Peter Geoghegan
Reported-By: Kyle Kingsbury
Reviewed-By: Thomas Munro
Discussion: https://postgr.es/m/[email protected]
Backpatch: All supported versions
M src/backend/access/heap/heapam.c
A src/test/isolation/expected/update-conflict-out.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/update-conflict-out.spec
pg_dump: Remove dead code
commit : d9fa17aa7c34dea66ce64da6fb4c643e75ba452c
author : Peter Eisentraut <[email protected]>
date : Thu, 11 Jun 2020 14:14:12 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 11 Jun 2020 14:14:12 +0200
Remove some code relevant only for dumping from pre-7.1 servers,
support for which had already been removed by
64f3524e2c8deebc02808aa5ebdfa17859473add.
M src/bin/pg_dump/pg_dump.c
Fix typos.
commit : ad9291f5e6f81ebca978c4438a1c7a448ca5b9ad
author : Amit Kapila <[email protected]>
date : Thu, 11 Jun 2020 14:10:43 +0530
committer: Amit Kapila <[email protected]>
date : Thu, 11 Jun 2020 14:10:43 +0530
Reported-by: John Naylor
Author: John Naylor
Backpatch-through: 9.5
Discussion: https://postgr.es/m/CACPNZCtRuvs6G+EYqejhVJgBq2AKeZdXRVJsbX4syhO9gn5SNQ@mail.gmail.com
M src/test/regress/input/largeobject.source
M src/test/regress/output/largeobject.source
M src/test/regress/output/largeobject_1.source
Refactor DROP LANGUAGE grammar
commit : 3fbd4bb6f494dd70cc5075536a754261853de167
author : Peter Eisentraut <[email protected]>
date : Thu, 11 Jun 2020 11:18:15 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 11 Jun 2020 11:18:15 +0200
Fold it into the generic DropStmt.
Discussion: https://www.postgresql.org/message-id/flat/163c00a5-f634-ca52-fc7c-0e53deda8735%402ndquadrant.com
M src/backend/parser/gram.y
Remove deprecated syntax from CREATE/DROP LANGUAGE
commit : 5333e014ab943b201fe71cfaf409419c204f36be
author : Peter Eisentraut <[email protected]>
date : Thu, 11 Jun 2020 10:26:12 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 11 Jun 2020 10:26:12 +0200
Remove the option to specify the language name as a single-quoted
string. This has been obsolete since ee8ed85da3b. Removing it allows
better grammar refactoring.
The syntax of the CREATE FUNCTION LANGUAGE clause is not changed.
Discussion: https://www.postgresql.org/message-id/flat/163c00a5-f634-ca52-fc7c-0e53deda8735%402ndquadrant.com
M doc/src/sgml/ref/create_language.sgml
M doc/src/sgml/ref/drop_language.sgml
M src/backend/parser/gram.y
Move frontend-side archive APIs from src/common/ to src/fe_utils/
commit : a3b2bf1fe7ce7cf88af6af2c100c6ed61c976780
author : Michael Paquier <[email protected]>
date : Thu, 11 Jun 2020 15:48:46 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 11 Jun 2020 15:48:46 +0900
fe_archive.c was compiled only for the frontend in src/common/, but as
it will never share anything with the backend, it makes most sense to
move this file to src/fe_utils/.
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13
M src/bin/pg_rewind/parsexlog.c
M src/common/Makefile
M src/fe_utils/Makefile
R094 src/common/fe_archive.c src/fe_utils/archive.c
R091 src/include/common/fe_archive.h src/include/fe_utils/archive.h
M src/tools/msvc/Mkvcbuild.pm
Fold AlterForeignTableStmt into AlterTableStmt
commit : c4325cefba512772efc108baf8ef7182c3833716
author : Peter Eisentraut <[email protected]>
date : Thu, 11 Jun 2020 08:21:24 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 11 Jun 2020 08:21:24 +0200
All other relation types are handled by AlterTableStmt, so it's
unnecessary to make a different statement for foreign tables.
Discussion: https://www.postgresql.org/message-id/flat/163c00a5-f634-ca52-fc7c-0e53deda8735%402ndquadrant.com
M src/backend/parser/gram.y
Remove redundant grammar symbols
commit : c2bd1fec32ab5407a3675272af1a06f425cb5161
author : Peter Eisentraut <[email protected]>
date : Wed, 10 Jun 2020 22:58:46 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 10 Jun 2020 22:58:46 +0200
access_method, database_name, and index_name are all just name, and
they are not used consistently for their alleged purpose, so remove
them. They have been around since ancient times but have no current
reason for existing. Removing them can simplify future grammar
refactoring.
Discussion: https://www.postgresql.org/message-id/flat/163c00a5-f634-ca52-fc7c-0e53deda8735%402ndquadrant.com
M src/backend/parser/gram.y
M src/interfaces/ecpg/preproc/ecpg.trailer
Change default of password_encryption to scram-sha-256
commit : c7eab0e97e6cf1d0c136c22269c10ae11ba874c4
author : Peter Eisentraut <[email protected]>
date : Wed, 10 Jun 2020 16:16:37 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 10 Jun 2020 16:16:37 +0200
Also, the legacy values on/true/yes/1 for password_encryption that
mapped to md5 are removed. The only valid values are now
scram-sha-256 and md5.
Reviewed-by: Jonathan S. Katz <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/d5b0ad33-7d94-bdd1-caac-43a1c782cab2%402ndquadrant.com
M doc/src/sgml/config.sgml
M src/backend/commands/user.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/initdb/initdb.c
M src/test/regress/expected/password.out
M src/test/regress/sql/password.sql
Update description of parameter password_encryption
commit : 5a4ada71a8f944600c348a6e4f5feb388ba8bd37
author : Peter Eisentraut <[email protected]>
date : Wed, 10 Jun 2020 11:57:41 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 10 Jun 2020 11:57:41 +0200
The previous description string still described the pre-PostgreSQL
10 (pre eb61136dc75a76caef8460fa939244d8593100f2) behavior of
selecting between encrypted and unencrypted, but it is now choosing
between encryption algorithms.
M src/backend/utils/misc/guc.c
Fix ReorderBuffer memory overflow check.
commit : c5c000b1038e3037289806f7f29c203f05a2b1e3
author : Amit Kapila <[email protected]>
date : Wed, 10 Jun 2020 10:20:10 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 10 Jun 2020 10:20:10 +0530
Commit cec2edfa78 introduced logical_decoding_work_mem to limit
ReorderBuffer memory usage. We spill the changes once the memory occupied
by changes exceeds logical_decoding_work_mem. There was an assumption
in the code that by evicting the largest (sub)transaction we will come
under the memory limit as the selected transaction will be at least as
large as the most recent change (which caused us to go over the memory
limit). However, that is not true because a user can reduce the
logical_decoding_work_mem to a smaller value before the most recent
change.
We fix it by allowing to evict the transactions until we reach under the
memory limit.
Reported-by: Fujii Masao
Author: Amit Kapila
Reviewed-by: Fujii Masao
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/reorderbuffer.c
Spelling adjustments
commit : 350f47786c3444d822d6d27829dd6a5426487150
author : Peter Eisentraut <[email protected]>
date : Tue, 9 Jun 2020 10:41:41 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 9 Jun 2020 10:41:41 +0200
similar to 0fd2a79a637f9f96b9830524823df0454e962f96
M contrib/pg_trgm/trgm_regexp.c
M src/backend/commands/explain.c
M src/backend/optimizer/plan/planner.c
M src/pl/plpgsql/src/expected/plpgsql_control.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
Unify drop-by-OID functions
commit : b1d32d3e3230f00b5baba08f75b4f665c7d6dac6
author : Peter Eisentraut <[email protected]>
date : Tue, 9 Jun 2020 09:10:14 +0200
committer: Peter Eisentraut <[email protected]>
date : Tue, 9 Jun 2020 09:10:14 +0200
There are a number of Remove${Something}ById() functions that are
essentially identical in structure and only different in which catalog
they are working on. Refactor this to be one generic function. The
information about which oid column, index, etc. to use was already
available in ObjectProperty for most catalogs, in a few cases it was
easily added.
Reviewed-by: Pavel Stehule <[email protected]>
Reviewed-by: Robert Haas <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/331d9661-1743-857f-1cbb-d5728bcd62cb%402ndquadrant.com
M src/backend/catalog/aclchk.c
M src/backend/catalog/dependency.c
M src/backend/catalog/objectaddress.c
M src/backend/catalog/pg_collation.c
M src/backend/catalog/pg_conversion.c
M src/backend/commands/amcmds.c
M src/backend/commands/event_trigger.c
M src/backend/commands/foreigncmds.c
M src/backend/commands/functioncmds.c
M src/backend/commands/opclasscmds.c
M src/backend/commands/proclang.c
M src/backend/commands/publicationcmds.c
M src/backend/commands/schemacmds.c
M src/backend/commands/tsearchcmds.c
M src/include/catalog/objectaddress.h
M src/include/catalog/pg_collation.h
M src/include/catalog/pg_conversion.h
M src/include/commands/defrem.h
M src/include/commands/event_trigger.h
M src/include/commands/proclang.h
M src/include/commands/publicationcmds.h
M src/include/commands/schemacmds.h
M src/include/utils/acl.h
Fix invalid function references in a few comments
commit : b27c90bbe4a3d607f8fc6703c7183e56e4039acd
author : David Rowley <[email protected]>
date : Tue, 9 Jun 2020 18:43:15 +1200
committer: David Rowley <[email protected]>
date : Tue, 9 Jun 2020 18:43:15 +1200
These appear to have been forgotten when the functions were renamed in
1fd687a03.
Backpatch-through: 13, where the functions were renamed
M src/backend/utils/adt/numutils.c
Repair unstable regression test.
commit : 05dea2427c2f6ba83f0b11f4e9472db2032dc1c7
author : Tom Lane <[email protected]>
date : Tue, 9 Jun 2020 01:17:59 -0400
committer: Tom Lane <[email protected]>
date : Tue, 9 Jun 2020 01:17:59 -0400
Commit 0c882e52a tried to force table atest12 to have more-accurate-
than-default statistics; but transiently setting default_statistics_target
isn't enough for that, because autovacuum could come along and overwrite
the stats later. This evidently explains some intermittent buildfarm
failures we've seen since then. Repair by disabling autovac on this table.
Thanks to David Rowley for correctly diagnosing the cause.
Discussion: https://postgr.es/m/CA+hUKG+OUkQSOUTg=qo=S=fWa_tbm99i7rB7mfbHz1SYm4v-jQ@mail.gmail.com
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Fix HashAgg regression from choosing too many initial buckets.
commit : 1b2c29469a58cd9086bd86e20c708eb437564a80
author : Jeff Davis <[email protected]>
date : Mon, 8 Jun 2020 20:59:45 -0700
committer: Jeff Davis <[email protected]>
date : Mon, 8 Jun 2020 20:59:45 -0700
Diagnosis by Andres.
Reported-by: Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRDLVakD5Aagt3yZeEQeTeEWaS3YE5h8XC3Q3qJ6TYkc2Q%40mail.gmail.com
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
Avoid need for valgrind suppressions for pg_atomic_init_u64 on some platforms.
commit : 47c718792b885c2a06e32cf86f4caca69ce5cda4
author : Andres Freund <[email protected]>
date : Mon, 8 Jun 2020 19:52:19 -0700
committer: Andres Freund <[email protected]>
date : Mon, 8 Jun 2020 19:52:19 -0700
Previously we used pg_atomic_write_64_impl inside
pg_atomic_init_u64. That works correctly, but on platforms without
64bit single copy atomicity it could trigger spurious valgrind errors
about uninitialized memory, because we use compare_and_swap for atomic
writes on such platforms.
I previously suppressed one instance of this problem (6c878edc1df),
but as Tom reports that wasn't enough. As the atomic variable cannot
yet be concurrently accessible during initialization, it seems better
to have pg_atomic_init_64_impl set the value directly.
Change pg_atomic_init_u32_impl for symmetry.
Reported-By: Tom Lane
Author: Andres Freund
Discussion: https://postgr.es/m/[email protected]
Backpatch: 9.5-
M src/include/port/atomics/generic.h
M src/tools/valgrind.supp
Update documentation for snowball update
commit : c2e71cb355a43d3ea0eaa6433d07f3681f934d54
author : Peter Eisentraut <[email protected]>
date : Mon, 8 Jun 2020 22:44:15 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 8 Jun 2020 22:44:15 +0200
Discussion: https://www.postgresql.org/message-id/flat/a8eeabd6-2be1-43fe-401e-a97594c38478%402ndquadrant.com
M doc/src/sgml/textsearch.sgml
Update snowball
commit : cbcc8726bb1c3075e58b9907547104271ff5899b
author : Peter Eisentraut <[email protected]>
date : Mon, 8 Jun 2020 07:58:51 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 8 Jun 2020 07:58:51 +0200
Update to snowball tag v2.0.0. Major changes are new stemmers for
Basque, Catalan, and Hindi.
Discussion: https://www.postgresql.org/message-id/flat/a8eeabd6-2be1-43fe-401e-a97594c38478%402ndquadrant.com
M src/backend/snowball/Makefile
M src/backend/snowball/README
M src/backend/snowball/dict_snowball.c
M src/backend/snowball/libstemmer/api.c
A src/backend/snowball/libstemmer/stem_ISO_8859_1_basque.c
A src/backend/snowball/libstemmer/stem_ISO_8859_1_catalan.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_danish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_dutch.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_english.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_finnish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_french.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_german.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_indonesian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_irish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_italian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_norwegian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_porter.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_portuguese.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_spanish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_1_swedish.c
M src/backend/snowball/libstemmer/stem_ISO_8859_2_hungarian.c
M src/backend/snowball/libstemmer/stem_ISO_8859_2_romanian.c
M src/backend/snowball/libstemmer/stem_KOI8_R_russian.c
M src/backend/snowball/libstemmer/stem_UTF_8_arabic.c
A src/backend/snowball/libstemmer/stem_UTF_8_basque.c
A src/backend/snowball/libstemmer/stem_UTF_8_catalan.c
M src/backend/snowball/libstemmer/stem_UTF_8_danish.c
M src/backend/snowball/libstemmer/stem_UTF_8_dutch.c
M src/backend/snowball/libstemmer/stem_UTF_8_english.c
M src/backend/snowball/libstemmer/stem_UTF_8_finnish.c
M src/backend/snowball/libstemmer/stem_UTF_8_french.c
M src/backend/snowball/libstemmer/stem_UTF_8_german.c
M src/backend/snowball/libstemmer/stem_UTF_8_greek.c
A src/backend/snowball/libstemmer/stem_UTF_8_hindi.c
M src/backend/snowball/libstemmer/stem_UTF_8_hungarian.c
M src/backend/snowball/libstemmer/stem_UTF_8_indonesian.c
M src/backend/snowball/libstemmer/stem_UTF_8_irish.c
M src/backend/snowball/libstemmer/stem_UTF_8_italian.c
M src/backend/snowball/libstemmer/stem_UTF_8_lithuanian.c
M src/backend/snowball/libstemmer/stem_UTF_8_nepali.c
M src/backend/snowball/libstemmer/stem_UTF_8_norwegian.c
M src/backend/snowball/libstemmer/stem_UTF_8_porter.c
M src/backend/snowball/libstemmer/stem_UTF_8_portuguese.c
M src/backend/snowball/libstemmer/stem_UTF_8_romanian.c
M src/backend/snowball/libstemmer/stem_UTF_8_russian.c
M src/backend/snowball/libstemmer/stem_UTF_8_spanish.c
M src/backend/snowball/libstemmer/stem_UTF_8_swedish.c
M src/backend/snowball/libstemmer/stem_UTF_8_tamil.c
M src/backend/snowball/libstemmer/stem_UTF_8_turkish.c
M src/backend/snowball/libstemmer/utilities.c
M src/bin/initdb/initdb.c
M src/include/snowball/libstemmer/header.h
A src/include/snowball/libstemmer/stem_ISO_8859_1_basque.h
A src/include/snowball/libstemmer/stem_ISO_8859_1_catalan.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_danish.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_dutch.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_english.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_finnish.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_french.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_german.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_indonesian.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_irish.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_italian.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_norwegian.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_porter.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_portuguese.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_spanish.h
M src/include/snowball/libstemmer/stem_ISO_8859_1_swedish.h
M src/include/snowball/libstemmer/stem_ISO_8859_2_hungarian.h
M src/include/snowball/libstemmer/stem_ISO_8859_2_romanian.h
M src/include/snowball/libstemmer/stem_KOI8_R_russian.h
M src/include/snowball/libstemmer/stem_UTF_8_arabic.h
A src/include/snowball/libstemmer/stem_UTF_8_basque.h
A src/include/snowball/libstemmer/stem_UTF_8_catalan.h
M src/include/snowball/libstemmer/stem_UTF_8_danish.h
M src/include/snowball/libstemmer/stem_UTF_8_dutch.h
M src/include/snowball/libstemmer/stem_UTF_8_english.h
M src/include/snowball/libstemmer/stem_UTF_8_finnish.h
M src/include/snowball/libstemmer/stem_UTF_8_french.h
M src/include/snowball/libstemmer/stem_UTF_8_german.h
M src/include/snowball/libstemmer/stem_UTF_8_greek.h
A src/include/snowball/libstemmer/stem_UTF_8_hindi.h
M src/include/snowball/libstemmer/stem_UTF_8_hungarian.h
M src/include/snowball/libstemmer/stem_UTF_8_indonesian.h
M src/include/snowball/libstemmer/stem_UTF_8_irish.h
M src/include/snowball/libstemmer/stem_UTF_8_italian.h
M src/include/snowball/libstemmer/stem_UTF_8_lithuanian.h
M src/include/snowball/libstemmer/stem_UTF_8_nepali.h
M src/include/snowball/libstemmer/stem_UTF_8_norwegian.h
M src/include/snowball/libstemmer/stem_UTF_8_porter.h
M src/include/snowball/libstemmer/stem_UTF_8_portuguese.h
M src/include/snowball/libstemmer/stem_UTF_8_romanian.h
M src/include/snowball/libstemmer/stem_UTF_8_russian.h
M src/include/snowball/libstemmer/stem_UTF_8_spanish.h
M src/include/snowball/libstemmer/stem_UTF_8_swedish.h
M src/include/snowball/libstemmer/stem_UTF_8_tamil.h
M src/include/snowball/libstemmer/stem_UTF_8_turkish.h
Fix locking bugs that could corrupt pg_control.
commit : 57cb8063089a7d6c54e27adfd5d028cc01f21536
author : Thomas Munro <[email protected]>
date : Mon, 8 Jun 2020 13:57:24 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 8 Jun 2020 13:57:24 +1200
The redo routines for XLOG_CHECKPOINT_{ONLINE,SHUTDOWN} must acquire
ControlFileLock before modifying ControlFile->checkPointCopy, or the
checkpointer could write out a control file with a bad checksum.
Likewise, XLogReportParameters() must acquire ControlFileLock before
modifying ControlFile and calling UpdateControlFile().
Back-patch to all supported releases.
Author: Nathan Bossart <[email protected]>
Author: Fujii Masao <[email protected]>
Reviewed-by: Fujii Masao <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Discussion: https://postgr.es/m/70BF24D6-DC51-443F-B55A-95735803842A%40amazon.com
M src/backend/access/transam/xlog.c
Doc: Update example symptom of systemd misconfiguration.
commit : d094bf93014b467cc3c129cc0d7d3f0f69968c96
author : Thomas Munro <[email protected]>
date : Mon, 8 Jun 2020 13:20:46 +1200
committer: Thomas Munro <[email protected]>
date : Mon, 8 Jun 2020 13:20:46 +1200
In PostgreSQL 10, we stopped using System V semaphores on Linux
systems. Update the example we give of an error message from a
misconfigured system to show what people are most likely to see these
days.
Back-patch to 10, where PREFERRED_SEMAPHORES=UNNAMED_POSIX arrived.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGLmJUSwybaPQv39rB8ABpqJq84im2UjZvyUY4feYhpWMw%40mail.gmail.com
M doc/src/sgml/runtime.sgml
Fix crash in WAL sender when starting physical replication
commit : 879ad9f90e83b94db14b8be11f1cc1fb38187cc0
author : Michael Paquier <[email protected]>
date : Mon, 8 Jun 2020 10:12:24 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 8 Jun 2020 10:12:24 +0900
Since database connections can be used with WAL senders in 9.4, it is
possible to use physical replication. This commit fixes a crash when
starting physical replication with a WAL sender using a database
connection, caused by the refactoring done in 850196b.
There have been discussions about forbidding the use of physical
replication in a database connection, but this is left for later,
taking care only of the crash new to 13.
While on it, add a test to check for a failure when attempting logical
replication if the WAL sender does not have a database connection. This
part is extracted from a larger patch by Kyotaro Horiguchi.
Reported-by: Vladimir Sitnikov
Author: Michael Paquier, Kyotaro Horiguchi
Reviewed-by: Kyotaro Horiguchi, Álvaro Herrera
Discussion: https://postgr.es/m/CAB=Je-GOWMj1PTPkeUhjqQp-4W3=nW-pXe2Hjax6rJFffB5_Aw@mail.gmail.com
Backpatch-through: 13
M src/backend/access/transam/xlogreader.c
M src/backend/replication/walsender.c
M src/include/access/xlogreader.h
M src/test/recovery/t/006_logical_decoding.pl
MSVC: Avoid warning when testing a TAP suite without PROVE_FLAGS.
commit : 5a2398b0b2d30315469ee00ca289ea88b03ccfd8
author : Noah Misch <[email protected]>
date : Sun, 7 Jun 2020 16:27:13 -0700
committer: Noah Misch <[email protected]>
date : Sun, 7 Jun 2020 16:27:13 -0700
Commit 7be5d8df1f74b78620167d3abf32ee607e728919 surfaced the logic
error, which had no functional implications, by adding "use warnings".
The buildfarm always customizes PROVE_FLAGS, so the warning did not
appear there. Back-patch to 9.5 (all supported versions).
M src/tools/msvc/vcregress.pl
Stamp HEAD as 14devel.
commit : d10b19e224c2ead10b59382f695f0c2bd65039fa
author : Tom Lane <[email protected]>
date : Sun, 7 Jun 2020 17:16:30 -0400
committer: Tom Lane <[email protected]>
date : Sun, 7 Jun 2020 17:16:30 -0400
Let the hacking begin ...
M configure
M configure.in
M doc/src/sgml/filelist.sgml
D doc/src/sgml/release-13.sgml
A doc/src/sgml/release-14.sgml
M doc/src/sgml/release.sgml
M src/tools/git_changelog
M src/tools/version_stamp.pl