Fix CLUSTER progress reporting of number of blocks scanned.
commit : fce17e486f3579fd2617c12f332608a5e78fefac author : Fujii Masao <firstname.lastname@example.org> date : Fri, 27 Nov 2020 20:16:44 +0900 committer: Fujii Masao <email@example.com> date : Fri, 27 Nov 2020 20:16:44 +0900
Previously pg_stat_progress_cluster view reported the current block number in heap scan as the number of heap blocks scanned (i.e., heap_blks_scanned). This reported number could be incorrect when synchronize_seqscans is enabled, because it allowed the heap scan to start at block in middle. This could result in wraparounds in the heap_blks_scanned column when the heap scan wrapped around. This commit fixes the bug by calculating the number of blocks from the block that the heap scan starts at to the current block in scan, and reporting that number in the heap_blks_scanned column. Also, in pg_stat_progress_cluster view, previously heap_blks_scanned could not reach heap_blks_total at the end of heap scan phase if the last pages scanned were empty. This commit fixes the bug by manually updating heap_blks_scanned to the same value as heap_blks_total when the heap scan phase finishes. Back-patch to v12 where pg_stat_progress_cluster view was introduced. Reported-by: Matthias van de Meent Author: Matthias van de Meent Reviewed-by: Fujii Masao Discussion: https://postgr.es/m/CAEze2WjCBWSGkVfYag001Rc4+-nNLDpWM7QbyD6yPvuhKs-gYQ@mail.gmail.com
In psql's \d commands, don't truncate attribute default values.
commit : ea7a167daa3e7c9c7b03fb09432aafcea1cf4d88 author : Tom Lane <firstname.lastname@example.org> date : Wed, 25 Nov 2020 16:19:25 -0500 committer: Tom Lane <email@example.com> date : Wed, 25 Nov 2020 16:19:25 -0500
Historically, psql has truncated the text of a column's default expression at 128 characters. This is unlike any other behavior in describe.c, and it's become particularly confusing now that the limit is only applied to the expression proper and not to the "generated always as (...) stored" text that may get wrapped around it. Excavation in our git history suggests that the original motivation for this limit was not really to limit the display width (as I'd long supposed), but to make it safe to use a fixed-width output buffer to store the result. That implementation restriction is long gone of course, but the limit remained. Let's just get rid of it. While here, rearrange the logic about when to free the output string so that it's not so dependent on unstated assumptions about the possible values of attidentity and attgenerated. Per bug #16743 from David Turon. Back-patch to v12 where GENERATED came in. (Arguably we could take it back further, but I'm hesitant to change the behavior of long-stable branches for this.) Discussion: https://firstname.lastname@example.org
doc: Fix typos
commit : b608645c17d7979357ebdbc9f461bfca54a6a415 author : Peter Eisentraut <email@example.com> date : Wed, 25 Nov 2020 09:49:00 +0100 committer: Peter Eisentraut <firstname.lastname@example.org> date : Wed, 25 Nov 2020 09:49:00 +0100
Author: Justin Pryzby <email@example.com> Discussion: https://www.postgresql.org/message-id/20201121194105.GO24784@telsasoft.com
Properly check index mark/restore in ExecSupportsMarkRestore.
commit : ae5aa26dc39d7729b044013e8b213e79e24e613b author : Andrew Gierth <firstname.lastname@example.org> date : Tue, 24 Nov 2020 20:58:32 +0000 committer: Andrew Gierth <email@example.com> date : Tue, 24 Nov 2020 20:58:32 +0000
Previously this code assumed that all IndexScan nodes supported mark/restore, which is not true since it depends on optional index AM support functions. This could lead to errors about missing support functions in rare edge cases of mergejoins with no sort keys, where an unordered non-btree index scan was placed on the inner path without a protecting Materialize node. (Normally, the fact that merge join requires ordered input would avoid this error.) Backpatch all the way since this bug is ancient. Per report from Eugen Konkov on irc. Discussion: https://firstname.lastname@example.org
Skip allocating hash table in EXPLAIN-only mode.
commit : 888fa2baeb39fa4069b1b55552c0ced85b533313 author : Heikki Linnakangas <email@example.com> date : Fri, 20 Nov 2020 14:41:14 +0200 committer: Heikki Linnakangas <firstname.lastname@example.org> date : Fri, 20 Nov 2020 14:41:14 +0200
This is a backpatch of commit 2cccb627f1, backpatched due to popular demand. Backpatch to all supported versions. Author: Alexey Bashtanov Discussion: https://www.postgresql.org/message-id/36823f65-050d-ae24-aa4d-a37726998240%40imap.cc
On macOS, use -isysroot in link steps as well as compile steps.
commit : 5b83604270a6ffc06e8e9406d19b9f2e845b74f9 author : Tom Lane <email@example.com> date : Fri, 20 Nov 2020 00:58:26 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Fri, 20 Nov 2020 00:58:26 -0500
We previously put the -isysroot switch only into CPPFLAGS, theorizing that it was only needed to find the right copies of include files. However, it seems that we also need to use it while linking programs, to find the right stub ".tbd" files for libraries. We got away without that up to now, but apparently that was mostly luck. It may also be that failures are only observed when the Xcode version is noticeably out of sync with the host macOS version; the case that's prompting action right now is that builds fail when using latest Xcode (12.2) on macOS Catalina, even though it's fine on Big Sur. Hence, add -isysroot to LDFLAGS as well. (It seems that the more common practice is to put it in CFLAGS, whence it'd be included at both compile and link steps. However, we can't mess with CFLAGS in the platform template file without confusing configure's logic for choosing default CFLAGS.) Back-patch of 49407dc32 into all supported branches. Report and patch by James Hilliard (some cosmetic mods by me) Discussion: https://email@example.com
Adjust DSM and DSA slot usage constants (back-patch).
commit : 2ded1f1fbb6e8e2b7927cce93238775604069bc3 author : Thomas Munro <firstname.lastname@example.org> date : Fri, 20 Nov 2020 10:44:09 +1300 committer: Thomas Munro <email@example.com> date : Fri, 20 Nov 2020 10:44:09 +1300
1. Previously, a DSA area would create up to four segments at each size before doubling the size. After this commit, it will create only two at each size, so it ramps up faster and therefore needs fewer slots. 2. Previously, the total limit on DSM slots allowed for 2 per connection. Switch to 5 per connection. This back-patches commit d061ea21 from release 13 into 10-12 based on a field complaint. Discussion: https://postgr.es/m/CAO03teA%2BjE1qt5iWDWzHqaufqBsF6EoOgZphnazps_tr_jDPZA%40mail.gmail.com Discussion: https://postgr.es/m/CA%2BhUKGL6H2BpGbiF7Lj6QiTjTGyTLW_vLR%3DSn2tEBeTcYXiMKw%40mail.gmail.com
Further fixes for CREATE TABLE LIKE: cope with self-referential FKs.
commit : 87ab464219259abdc06b0492f0d42bc6734ae490 author : Tom Lane <firstname.lastname@example.org> date : Thu, 19 Nov 2020 15:03:17 -0500 committer: Tom Lane <email@example.com> date : Thu, 19 Nov 2020 15:03:17 -0500
Commit 502898192 was too careless about the order of execution of the additional ALTER TABLE operations generated by expandTableLikeClause. It just stuck them all at the end, which seems okay for most purposes. But it falls down in the case where LIKE is importing a primary key or unique index and the outer CREATE TABLE includes a FOREIGN KEY constraint that needs to depend on that index. Weird as that is, it used to work, so we ought to keep it working. To fix, make parse_utilcmd.c insert LIKE clauses between index-creation and FK-creation commands in the transformed list of commands, and change utility.c so that the commands generated by expandTableLikeClause are executed immediately not at the end. One could imagine scenarios where this wouldn't work either; but currently expandTableLikeClause only makes column default expressions, CHECK constraints, and indexes, and this ordering seems fine for those. Per bug #16730 from Sofoklis Papasofokli. Like the previous patch, back-patch to all supported branches. Discussion: https://firstname.lastname@example.org
Don't Insert() a VFD entry until it's fully built.
commit : 4097a79069be10c9883d1fef566f2921621e1852 author : Tom Lane <email@example.com> date : Mon, 16 Nov 2020 20:32:35 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Mon, 16 Nov 2020 20:32:35 -0500
Otherwise, if FDDEBUG is enabled, the debugging output fails because it tries to read the fileName, which isn't set up yet (and should in fact always be NULL). AFAICT, this has been wrong since Berkeley. Before 96bf88d52, it would accidentally fail to crash on platforms where snprintf() is forgiving about being passed a NULL pointer for %s; but the file name intended to be included in the debug output wouldn't ever have shown up. Report and fix by Greg Nancarrow. Although this is only visibly broken in custom-made builds, it still seems worth back-patching to all supported branches, as the FDDEBUG code is pretty useless as it stands. Discussion: https://postgr.es/m/CAJcOf-cUDgm9qYtC_B6XrC6MktMPNRby2p61EtSGZKnfotMArw@mail.gmail.com
Do not return NULL for error cases in satisfies_hash_partition().
commit : 54a94064407375ac3c108e1b791303a42baf030a author : Tom Lane <email@example.com> date : Mon, 16 Nov 2020 16:39:59 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Mon, 16 Nov 2020 16:39:59 -0500
Since this function is used as a CHECK constraint condition, returning NULL is tantamount to returning TRUE, which would have the effect of letting in a row that doesn't satisfy the hash condition. Admittedly, the cases for which this is done should be unreachable in practice, but that doesn't make it any less a bad idea. It also seems like a dartboard was used to decide which error cases should throw errors as opposed to returning NULL. For the checks for NULL input values, I just switched it to returning false. There's some argument that an error would be better; but the case really should be can't-happen in a generated hash constraint, so it's likely not worth more code for. For the parent-relation-open-failure case, it seems like we might as well let relation_open throw an error, instead of having an impossible-to-diagnose constraint failure. Back-patch to v11 where this code came in. Discussion: https://email@example.com
Use "true" not "TRUE" in one ICU function call.
commit : 029fa664ecb8687eb60633b4ccf230b457a1fb77 author : Tom Lane <firstname.lastname@example.org> date : Mon, 16 Nov 2020 15:16:39 -0500 committer: Tom Lane <email@example.com> date : Mon, 16 Nov 2020 15:16:39 -0500
This was evidently missed in commit 6337865f3, which generally did s/TRUE/true/ everywhere. It escaped notice up to now because ICU versions before ICU 68 provided definitions of "TRUE" and "FALSE" regardless. With ICU 68, it fails to compile. Per report from Condor. Back-patch to v11 where 6337865f3 came in. (I've not tested v10, where this call originated, but I imagine it's fine since we defined TRUE in c.h back then.) Discussion: https://firstname.lastname@example.org
doc: update bgwriter description
commit : 139c439730b1444be88203cf867e60dd7533b294 author : Bruce Momjian <email@example.com> date : Mon, 16 Nov 2020 13:13:43 -0500 committer: Bruce Momjian <firstname.lastname@example.org> date : Mon, 16 Nov 2020 13:13:43 -0500
This clarifies exactly what the bgwriter does, which should help with tuning. Reported-by: Chris Wilson Discussion: https://email@example.com Backpatch-through: 9.5
doc: clarify how to find pg_type_d.h in the install tree
commit : 3da1110b8dfdd3d4c70715ef5cf2870ae94a30c3 author : Bruce Momjian <firstname.lastname@example.org> date : Mon, 16 Nov 2020 12:36:17 -0500 committer: Bruce Momjian <email@example.com> date : Mon, 16 Nov 2020 12:36:17 -0500
Followup to patch 152ed04799. Reported-by: Alvaro Herrera Discussion: https://postgr.es/m/20201112202900.GA28098@alvherre.pgsql Backpatch-through: 9.5
doc: improve wording of the need for analyze of exp. indexes
commit : 260493aed2e8863fe76e7c5f0c02f535f40000fe author : Bruce Momjian <firstname.lastname@example.org> date : Mon, 16 Nov 2020 10:26:17 -0500 committer: Bruce Momjian <email@example.com> date : Mon, 16 Nov 2020 10:26:17 -0500
This is a followup commit on 3370207986. Reported-by: Justin Pryzby Discussion: https://postgr.es/m/20201112211143.GL30691@telsasoft.com Backpatch-through: 9.5
commit : b3e7c202b79e4d6bce6a65c957fe7f813d5138f8 author : Alvaro Herrera <firstname.lastname@example.org> date : Mon, 16 Nov 2020 10:54:11 -0300 committer: Alvaro Herrera <email@example.com> date : Mon, 16 Nov 2020 10:54:11 -0300
Introduced in 90fdc259866e; backpatch to 12. Author: Erik Rijkers <firstname.lastname@example.org> Discussion: https://email@example.com
Fix fuzzy thinking about amcanmulticol versus amcaninclude.
commit : 4ac8ee9d4878dadccc2331d8d1b1769c97f4ebe9 author : Tom Lane <firstname.lastname@example.org> date : Sun, 15 Nov 2020 16:10:48 -0500 committer: Tom Lane <email@example.com> date : Sun, 15 Nov 2020 16:10:48 -0500
These flags should be independent: in particular an index AM should be able to say that it supports include columns without necessarily supporting multiple key columns. The included-columns patch got this wrong, possibly aided by the fact that it didn't bother to update the documentation. While here, clarify some text about amcanreturn, which was a little vague about what should happen when amcanreturn reports that only some of the index columns are returnable. Noted while reviewing the SP-GiST included-columns patch, which quite incorrectly (and unsafely) changed SP-GiST to claim amcanmulticol = true as a workaround for this bug. Backpatch to v11 where included columns were introduced.
Doc: improve partitioning discussion in ddl.sgml.
commit : 2b77595085423970036afd6263046c5747534dd0 author : Tom Lane <firstname.lastname@example.org> date : Sat, 14 Nov 2020 13:09:53 -0500 committer: Tom Lane <email@example.com> date : Sat, 14 Nov 2020 13:09:53 -0500
This started with the intent to explain that range upper bounds are exclusive, which previously you could only find out by reading the CREATE TABLE man page. But I soon found that section 5.11 really could stand a fair amount of editorial attention. It's apparently been revised several times without much concern for overall flow, nor careful copy-editing. Back-patch to v11, which is as far as the patch goes easily. Per gripe from Edson Richter. Thanks to David Johnston for review. Discussion: https://postgr.es/m/DM6PR13MB3988736CF8F5DC5720440231CFE60@DM6PR13MB3988.namprd13.prod.outlook.com
doc: clarify where to find pg_type_d.h (PG 11+) and pg_type.h
commit : 36051198dfa75c9b7e19ae59e458c12f48d67abf author : Bruce Momjian <firstname.lastname@example.org> date : Thu, 12 Nov 2020 15:13:01 -0500 committer: Bruce Momjian <email@example.com> date : Thu, 12 Nov 2020 15:13:01 -0500
These files are in compiled directories and install directories. Reported-by: firstname.lastname@example.org Discussion: https://email@example.com Backpatch-through: 9.5
docs: mention that expression indexes need analyze
commit : 90fdc259866ed800283620c8fe6329015ad58e5c author : Bruce Momjian <firstname.lastname@example.org> date : Thu, 12 Nov 2020 15:00:44 -0500 committer: Bruce Momjian <email@example.com> date : Thu, 12 Nov 2020 15:00:44 -0500
Expression indexes can't benefit from pre-computed statistics on columns. Reported-by: Nikolay Samokhvalov Discussion: https://postgr.es/m/CANNMO++5rw9RDA=p40iMVbMNPaW6O=S0AFzTU=KpYHRpCd1voA@mail.gmail.com Author: Nikolay Samokhvalov, modified Backpatch-through: 9.5
doc: wire protocol data type for history file content is bytea
commit : d7fa90fae104a81cc5a1a76a9607afcd3d0ad070 author : Bruce Momjian <firstname.lastname@example.org> date : Thu, 12 Nov 2020 14:33:28 -0500 committer: Bruce Momjian <email@example.com> date : Thu, 12 Nov 2020 14:33:28 -0500
Document that though the history file content is marked as bytea, it is the same a text, and neither is btyea-escaped or encoding converted. Reported-by: Brar Piening Discussion: https://firstname.lastname@example.org Backpatch-through: 13 - 9.5 (not master)
pg_trgm: fix crash in 2-item picksplit
commit : 7f69ed4aebe342365dc7db96f33b473dec9f054b author : Andrew Gierth <email@example.com> date : Thu, 12 Nov 2020 14:34:37 +0000 committer: Andrew Gierth <firstname.lastname@example.org> date : Thu, 12 Nov 2020 14:34:37 +0000
Whether from size overflow in gistSplit or from secondary splits, picksplit is (rarely) called with exactly two items to split. Formerly, due to special-case handling of the last item, this would lead to access to an uninitialized cache entry; prior to PG 13 this might have been harmless or at worst led to an incorrect union datum, but in 13 onwards it can cause a backend crash from using an uninitialized pointer. Repair by removing the special case, which was deemed not to have been appropriate anyway. Backpatch all the way, because this bug has existed since pg_trgm was added. Per report on IRC from user "ftzdomino". Analysis and testing by me, patch from Alexander Korotkov. Discussion: https://email@example.com
Remove duplicate code in brin_memtuple_initialize
commit : 0d0626e27d56a52f78cd71a796ee2e8fe216f217 author : Tomas Vondra <firstname.lastname@example.org> date : Wed, 11 Nov 2020 18:37:36 +0100 committer: Tomas Vondra <email@example.com> date : Wed, 11 Nov 2020 18:37:36 +0100
Commit 8bf74967dab moved some of the code from brin_new_memtuple to brin_memtuple_initialize, but this resulted in some of the code being duplicate. Fix by removing the duplicate lines and backpatch to 10. Author: Tomas Vondra Backpatch-through: 10 Discussion: https://postgr.es/m/5eb50c97-9a8e-b691-8c40-1b2a55611c4c%40enterprisedb.com
Fix and simplify some usages of TimestampDifference().
commit : 171c457cd06051b5910cf357ea6a77643b69088a author : Tom Lane <firstname.lastname@example.org> date : Tue, 10 Nov 2020 22:51:19 -0500 committer: Tom Lane <email@example.com> date : Tue, 10 Nov 2020 22:51:19 -0500
Introduce TimestampDifferenceMilliseconds() to simplify callers that would rather have the difference in milliseconds, instead of the select()-oriented seconds-and-microseconds format. This gets rid of at least one integer division per call, and it eliminates some apparently-easy-to-mess-up arithmetic. Two of these call sites were in fact wrong: * pg_prewarm's autoprewarm_main() forgot to multiply the seconds by 1000, thus ending up with a delay 1000X shorter than intended. That doesn't quite make it a busy-wait, but close. * postgres_fdw's pgfdw_get_cleanup_result() thought it needed to compute microseconds not milliseconds, thus ending up with a delay 1000X longer than intended. Somebody along the way had noticed this problem but misdiagnosed the cause, and imposed an ad-hoc 60-second limit rather than fixing the units. This was relatively harmless in context, because we don't care that much about exactly how long this delay is; still, it's wrong. There are a few more callers of TimestampDifference() that don't have a direct need for seconds-and-microseconds, but can't use TimestampDifferenceMilliseconds() either because they do need microsecond precision or because they might possibly deal with intervals long enough to overflow 32-bit milliseconds. It might be worth inventing another API to improve that, but that seems outside the scope of this patch; so those callers are untouched here. Given the fact that we are fixing some bugs, and the likelihood that future patches might want to back-patch code that uses this new API, back-patch to all supported branches. Alexey Kondratov and Tom Lane Discussion: https://firstname.lastname@example.org
doc: fix spelling "connction" to "connection"
commit : 1393acb3836d32e512801bb00f0cc6b4d2a58dc8 author : Bruce Momjian <email@example.com> date : Tue, 10 Nov 2020 19:18:35 -0500 committer: Bruce Momjian <firstname.lastname@example.org> date : Tue, 10 Nov 2020 19:18:35 -0500
Was wrong in commit 1a9388bd0f. Reported-by: Tom Lane, Justin Pryzby Discussion: https://postgr.es/m/20201102063333.GE22691@telsasoft.com Backpatch-through: 9.5
Work around cross-version-upgrade issues created by commit 9e38c2bb5.
commit : 5ce51f8280d3e2a59dbcd93c9d8244e3bf96944d author : Tom Lane <email@example.com> date : Tue, 10 Nov 2020 18:32:36 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Tue, 10 Nov 2020 18:32:36 -0500
Summarily changing the STYPE of regression-test aggregates that depend on array_append or array_cat is an issue for the buildfarm's cross-version-upgrade tests, because those aggregates (as defined in the back branches) now won't load into HEAD. Although this seems like only a minimal risk for genuine user-defined aggregates, we need to do something for the buildfarm. Hence, adjust the aggregate definitions, in both HEAD and the back branches. Discussion: https://email@example.com Discussion: https://postgr.es/m/E1kaQ2c-0005lx-Eg@gemulon.postgresql.org