commit : 9848bd725e3b021bbe53e606ac9c261079ee82ba author : Tom Lane <email@example.com> date : Mon, 11 Feb 2019 16:21:19 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Mon, 11 Feb 2019 16:21:19 -0500
commit : 248fd6cb3a9d01348be0fef12f370c453dba8a77 author : Peter Eisentraut <email@example.com> date : Mon, 11 Feb 2019 14:21:55 +0100 committer: Peter Eisentraut <firstname.lastname@example.org> date : Mon, 11 Feb 2019 14:21:55 +0100
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: f3ca37534288ad85c0436fa00755b8aba69e57d3
Release notes for 11.2, 10.7, 9.6.12, 9.5.16, 9.4.21.
commit : 73f95272b9412a26c352099a2b1bbcd6cf0f36f8 author : Tom Lane <email@example.com> date : Sun, 10 Feb 2019 15:44:05 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Sun, 10 Feb 2019 15:44:05 -0500
Solve cross-version-upgrade testing problem induced by 1fb57af92.
commit : 4b235947c7c63a084a90c2e049d79a09ff03b9b3 author : Tom Lane <email@example.com> date : Sat, 9 Feb 2019 21:02:06 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Sat, 9 Feb 2019 21:02:06 -0500
Renaming varchar_transform to varchar_support had a side effect I hadn't foreseen: the core regression tests leave around a transform object that relies on that function, so the name change breaks cross-version upgrade tests, because the name used in the older branches doesn't match. Since the dependency on varchar_transform was chosen with the aid of a dartboard anyway (it would surely not work as a language transform support function), fix by just choosing a different random builtin function with the right signature. Also add some comments explaining why this isn't horribly unsafe. I chose to make the same substitution in a couple of other copied-and-pasted test cases, for consistency, though those aren't directly contributing to the testing problem. Per buildfarm. Back-patch, else it doesn't fix the problem.
Repair unsafe/unportable snprintf usage in pg_restore.
commit : 2b6009e2a2794df286939e5ce1196a23cca07d68 author : Tom Lane <email@example.com> date : Sat, 9 Feb 2019 19:45:38 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Sat, 9 Feb 2019 19:45:38 -0500
warn_or_exit_horribly() was blithely passing a potentially-NULL string pointer to a %s format specifier. That works (at least to the extent of not crashing) on some platforms, but not all, and since we switched to our own snprintf.c it doesn't work for us anywhere. Of the three string fields being handled this way here, I think that only "owner" is supposed to be nullable ... but considering that this is error-reporting code, it has very little business assuming anything, so put in defenses for all three. Per a crash observed on buildfarm member crake and then reproduced here. Because of the portability aspect, back-patch to all supported versions.
Call set_rel_pathlist_hook before generate_gather_paths, not after.
commit : 6257f525aff4e711d7eb5405626e7eeac37e2ff5 author : Tom Lane <email@example.com> date : Sat, 9 Feb 2019 11:41:09 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Sat, 9 Feb 2019 11:41:09 -0500
The previous ordering of these steps satisfied the nominal requirement that set_rel_pathlist_hook could editorialize on the whole set of Paths constructed for a base relation. In practice, though, trying to change the set of partial paths was impossible. Adding one didn't work because (a) it was too late to be included in Gather paths made by the core code, and (b) calling add_partial_path after generate_gather_paths is unsafe, because it might try to delete a path it thinks is dominated, but that is already embedded in some Gather path(s). Nor could the hook safely remove partial paths, for the same reason that they might already be embedded in Gathers. Better to call extensions first, let them add partial paths as desired, and then gather. In v11 and up, we already doubled down on that ordering by postponing gathering even further for single-relation queries; so even if the hook wished to editorialize on Gather path construction, it could not. Report and patch by KaiGai Kohei. Back-patch to 9.6 where Gather paths were added. Discussion: https://postgr.es/m/CAOP8fzahwpKJRTVVTqo2AE=mDTz_efVzV6Get_0=U3SO+-ha1A@mail.gmail.com
Defend against null error message reported by libxml2.
commit : ef922faea703aff4996eab17520cb5201a793701 author : Tom Lane <email@example.com> date : Fri, 8 Feb 2019 13:30:42 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Fri, 8 Feb 2019 13:30:42 -0500
While this isn't really supposed to happen, it can occur in OOM situations and perhaps others. Instead of crashing, substitute "(no message provided)". I didn't worry about localizing this text, since we aren't localizing anything else here; besides, if we're on the edge of OOM, it's unlikely gettext() would work. Report and fix by Sergio Conde Gómez in bug #15624. Discussion: https://email@example.com
Doc: fix thinko in description of how to escape a backslash in bytea.
commit : 2bcbf687e4265cad739d7fd6cb5f405bae560ed5 author : Tom Lane <firstname.lastname@example.org> date : Fri, 8 Feb 2019 12:49:36 -0500 committer: Tom Lane <email@example.com> date : Fri, 8 Feb 2019 12:49:36 -0500
Also clean up some discussion that had been left in a very confused state thanks to half-hearted adjustments for the change to standard_conforming_strings being the default. Discussion: https://firstname.lastname@example.org
Ensure that foreign scans with lateral refs are planned correctly.
commit : d468da0d65905b762dbaa647aaf8ba1ea5bca04d author : Tom Lane <email@example.com> date : Thu, 7 Feb 2019 13:10:46 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Thu, 7 Feb 2019 13:10:46 -0500
As reported in bug #15613 from Srinivasan S A, file_fdw and postgres_fdw neglected to mark plain baserel foreign paths as parameterized when the relation has lateral_relids. Other FDWs have surely copied this mistake, so rather than just patching those two modules, install a band-aid fix in create_foreignscan_path to rectify the mistake centrally. Although the band-aid is enough to fix the visible symptom, correct the calls in file_fdw and postgres_fdw anyway, so that they are valid examples for external FDWs. Also, since the band-aid isn't enough to make this work for parameterized foreign joins, throw an elog(ERROR) if such a case is passed to create_foreignscan_path. This shouldn't pose much of a problem for existing external FDWs, since it's likely they aren't trying to make such paths anyway (though some of them may need a defense against joins with lateral_relids, similar to the one this patch installs into postgres_fdw). Add some assertions in relnode.c to catch future occurrences of the same error --- in particular, as backstop against core-code mistakes like the one fixed by commit bdd9a99aa. Discussion: https://email@example.com
Fix searchpath and module location for pg_rewind and ssl TAP tests
commit : a29e93e8f9da8f9176174901526bca4681b586e1 author : Andrew Dunstan <firstname.lastname@example.org> date : Thu, 7 Feb 2019 10:22:49 -0500 committer: Andrew Dunstan <email@example.com> date : Thu, 7 Feb 2019 10:22:49 -0500
The modules RewindTest.pm and ServerSetup.pm are really only useful for TAP tests, so they really belong in the TAP test directories. In addition, ServerSetup.pm is renamed to SSLServer.pm. The test scripts have their own directories added to the search path so that the relocated modules will be found, regardless of where the tests are run from, even on modern perl where "." is no longer in the searchpath. Discussion: https://postgr.es/m/e4b0f366-269c-73c3-9c90-d9cb0f4db1f9@2ndQuadrant.com Backpatch as appropriate to 9.5
R100 src/bin/pg_rewind/RewindTest.pm src/bin/pg_rewind/t/RewindTest.pm
R099 src/test/ssl/ServerSetup.pm src/test/ssl/t/SSLServer.pm
Propagate lateral-reference information to indirect descendant relations.
commit : 41740b9efb110e3187241e2c8c99ab63976f3ce3 author : Tom Lane <firstname.lastname@example.org> date : Wed, 6 Feb 2019 12:44:58 -0500 committer: Tom Lane <email@example.com> date : Wed, 6 Feb 2019 12:44:58 -0500
create_lateral_join_info() computes a bunch of information about lateral references between base relations, and then attempts to propagate those markings to appendrel children of the original base relations. But the original coding neglected the possibility of indirect descendants (grandchildren etc). During v11 development we noticed that this was wrong for partitioned-table cases, but failed to realize that it was just as wrong for any appendrel. While the case can't arise for appendrels derived from traditional table inheritance (because we make a flat appendrel for that), nested appendrels can arise from nested UNION ALL subqueries. Failure to mark the lower-level relations as having lateral references leads to confusion in add_paths_to_append_rel about whether unparameterized paths can be built. It's not very clear whether that leads to any user-visible misbehavior; the lack of field reports suggests that it may cause nothing worse than minor cost misestimation. Still, it's a bug, and it leads to failures of Asserts that I intend to add later. To fix, we need to propagate information from all appendrel parents, not just those that are RELOPT_BASERELs. We can still do it in one pass, if we rely on the append_rel_list to be ordered with ancestor relationships before descendant ones; add assertions checking that. While fixing this, we can make a small performance improvement by traversing the append_rel_list just once instead of separately for each appendrel parent relation. Noted while investigating bug #15613, though this patch does not fix that (which is why I'm not committing the related Asserts yet). Discussion: https://firstname.lastname@example.org
Unify searchpath and do file logic in MSVC build scripts.
commit : 324c926468826c1ddc780806c35691a31523d83a author : Andrew Dunstan <email@example.com> date : Wed, 6 Feb 2019 07:32:35 -0500 committer: Andrew Dunstan <firstname.lastname@example.org> date : Wed, 6 Feb 2019 07:32:35 -0500
Commit f83419b739 failed to notice that mkvcbuild.pl and build.pl use different searchpath and do-file logic, breaking the latter, so it is adjusted to use the same logic as mkvcbuild.pl.
Fix included file path for modern perl
commit : 42345284c7992a9c0022fb1035c13303fe160f90 author : Andrew Dunstan <email@example.com> date : Tue, 5 Feb 2019 18:57:12 -0500 committer: Andrew Dunstan <firstname.lastname@example.org> date : Tue, 5 Feb 2019 18:57:12 -0500
Contrary to the comment on 772d4b76, only paths starting with "./" or "../" are considered relative to the current working directory by perl's "do" function. So this patch converts all the relevant cases to use "./" paths. This only affects MSVC. Backpatch to all live branches.
More fixed for modern perl on back branches
commit : 772d4b769d30090dbf4434d7d192569b11371a53 author : Andrew Dunstan <email@example.com> date : Tue, 5 Feb 2019 18:31:10 -0500 committer: Andrew Dunstan <firstname.lastname@example.org> date : Tue, 5 Feb 2019 18:31:10 -0500
Use "do" instead of "require" for included files, as it doesn't look for them in the search path but relative to the current working directory. These changes have already been made to REL_10_STABLE and later, to satisfy the demands of perlcritic, but need backporting now to earlier branches.
Keep perl style checker happy
commit : 749cb71de2d6217ef2f4ca76aa8461e4a7275f4c author : Andrew Dunstan <email@example.com> date : Tue, 5 Feb 2019 15:16:55 -0500 committer: Andrew Dunstan <firstname.lastname@example.org> date : Tue, 5 Feb 2019 15:16:55 -0500
It doesn't like code before "use strict;".
Update time zone data files to tzdata release 2018i.
commit : f6db9f8abfd6a46f5b58423d8cef2568bf945a0b author : Tom Lane <email@example.com> date : Tue, 5 Feb 2019 10:58:53 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Tue, 5 Feb 2019 10:58:53 -0500
DST law changes in Kazakhstan, Metlakatla, and São Tomé and Príncipe. Kazakhstan's Qyzylorda zone is split in two, creating a new zone Asia/Qostanay, as some areas did not change UTC offset. Historical corrections for Hong Kong and numerous Pacific islands.
Fix searchpath for modern Perl for genbki.pl
commit : be037c11bf2dbaf9048a988c6ac24b18b2016dd0 author : Andrew Dunstan <email@example.com> date : Tue, 5 Feb 2019 09:59:46 -0500 committer: Andrew Dunstan <firstname.lastname@example.org> date : Tue, 5 Feb 2019 09:59:46 -0500
This was fixed for MSVC tools by commit 1df92eeafefac4, but per buildfarm member bowerbird genbki.pl needs the same treatment. Backpatch to all live branches.
Doc: in each release branch, keep only that branch's own release notes.
commit : 60b1d6c2a736a1279b297f698cf507d5c81fa31a author : Tom Lane <email@example.com> date : Mon, 4 Feb 2019 19:18:50 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Mon, 4 Feb 2019 19:18:50 -0500
Historically we've had each release branch include all prior branches' notes, including minor-release changes, back to the beginning of the project. That's basically an O(N^2) proposition, and it was starting to catch up with us: as of HEAD the back-branch release notes alone accounted for nearly 30% of the documentation. While there's certainly some value in easy access to back-branch notes, this is getting out of hand. Hence, switch over to the rule that each branch contains only its own release notes. So as to not make older notes too hard to find, each branch will provide URLs for the immediately preceding branches' release notes on the project website. There might be value in providing aggregated notes across all branches somewhere on the website, but that's a task for another day. Discussion: https://email@example.com
Fix dumping of matviews with indirect dependencies on primary keys.
commit : 16e0464a11d8c2840cf8127394a4b4315111440a author : Tom Lane <firstname.lastname@example.org> date : Mon, 4 Feb 2019 17:20:02 -0500 committer: Tom Lane <email@example.com> date : Mon, 4 Feb 2019 17:20:02 -0500
Commit 62215de29 turns out to have been not quite on-the-mark. When we are forced to postpone dumping of a materialized view into the dump's post-data section (because it depends on a unique index that isn't created till that section), we may also have to postpone dumping other matviews that depend on said matview. The previous fix didn't reliably work for such cases: it'd break the dependency loops properly, producing a workable object ordering, but it didn't necessarily mark all the matviews as "postponed_def". This led to harmless bleating about "archive items not in correct section order", as reported by Tom Cassidy in bug #15602. Less harmlessly, selective-restore options such as --section might misbehave due to the matview dump objects not being properly labeled. The right way to fix it is to consider that each pre-data dependency we break amounts to moving the no-longer-dependent object into post-data, and hence we should mark that object if it's a matview. Back-patch to all supported versions, since the issue's been there since matviews were introduced. Discussion: https://firstname.lastname@example.org
Move port-specific parts of with_temp_install to port makefile.
commit : e7f1d14137afa51bb782a02b7f1bd39d9b9e048b author : Andrew Gierth <email@example.com> date : Mon, 4 Feb 2019 18:47:33 +0000 committer: Andrew Gierth <firstname.lastname@example.org> date : Mon, 4 Feb 2019 18:47:33 +0000
Rather than define ld_library_path_ver with a big nested $(if), just put the overriding values in the makefiles for the relevant ports. Also add a variable for port makefiles to append their own stuff to with_temp_install, and use it to set LD_LIBRARY_PATH_RPATH=1 on FreeBSD which is needed to make LD_LIBRARY_PATH override DT_RPATH if DT_RUNPATH is not set (which seems to depend in unpredictable ways on the choice of compiler, at least on my system). Backpatch for the benefit of anyone doing regression tests on FreeBSD. (For other platforms there should be no functional change.)
Add PG_CFLAGS, PG_CXXFLAGS, and PG_LDFLAGS variables to PGXS
commit : 5e63df82708ba6e350a1beb568c059e109f440a0 author : Michael Paquier <email@example.com> date : Sun, 3 Feb 2019 17:48:53 +0900 committer: Michael Paquier <firstname.lastname@example.org> date : Sun, 3 Feb 2019 17:48:53 +0900
Add PG_CFLAGS, PG_CXXFLAGS, and PG_LDFLAGS variables to pgxs.mk which will be appended or prepended to the corresponding make variables. Notably, there was previously no way to pass custom CXXFLAGS to third party extension module builds, COPT and PROFILE supporting only CFLAGS and LDFLAGS. Backpatch all the way down to ease integration with existing extensions. Author: Christoph Berg Reviewed-by: Andres Freund, Tom Lane, Michael Paquier Discussion: https://postgr.es/m/20181113104005.GA32154@msg.credativ.de Backpatch-through: 9.4
Avoid possible deadlock while locking multiple heap pages.
commit : 073afae81989c0eec572c9f5850b9265c9e07933 author : Amit Kapila <email@example.com> date : Sat, 2 Feb 2019 08:54:33 +0530 committer: Amit Kapila <firstname.lastname@example.org> date : Sat, 2 Feb 2019 08:54:33 +0530
To avoid deadlock, backend acquires a lock on heap pages in block number order. In certain cases, lock on heap pages is dropped and reacquired. In this case, the locks are dropped for reading in corresponding VM page/s. The issue is we re-acquire locks in bufferId order whereas the intention was to acquire in blockid order. This commit ensures that we will always acquire locks on heap pages in blockid order. Reported-by: Nishant Fnu Author: Nishant Fnu Reviewed-by: Amit Kapila and Robert Haas Backpatch-through: 9.4 Discussion: https://postgr.es/m/5883C831-2ED1-47C8-BFAC-2D5BAE5A8CAE@amazon.com
Fix use of dangling pointer in heap_delete() when logging replica identity
commit : 014763e97668d1597d3cc2abe9ec4bdbf1554e5f author : Michael Paquier <email@example.com> date : Fri, 1 Feb 2019 10:35:52 +0900 committer: Michael Paquier <firstname.lastname@example.org> date : Fri, 1 Feb 2019 10:35:52 +0900
When logging the replica identity of a deleted tuple, XLOG_HEAP_DELETE records include references of the old tuple. Its data is stored in an intermediate variable used to register this information for the WAL record, but this variable gets away from the stack when the record gets actually inserted. Spotted by clang's AddressSanitizer. Author: Stas Kelvish Discussion: https://postgr.es/m/085C8825-AD86-4E93-AF80-E26CDF03D1EA@postgrespro.ru Backpatch-through: 9.4
Fix a crash in logical replication
commit : 650296f8f607365278f64c9b9a7438c321f2d4ec author : Peter Eisentraut <email@example.com> date : Mon, 28 Jan 2019 22:09:33 +0100 committer: Peter Eisentraut <firstname.lastname@example.org> date : Mon, 28 Jan 2019 22:09:33 +0100
The bug was that determining which columns are part of the replica identity index using RelationGetIndexAttrBitmap() would run eval_const_expressions() on index expressions and predicates across all indexes of the table, which in turn might require a snapshot, but there wasn't one set, so it crashes. There were actually two separate bugs, one on the publisher and one on the subscriber. To trigger the bug, a table that is part of a publication or subscription needs to have an index with a predicate or expression that lends itself to constant expressions simplification. The fix is to avoid the constant expressions simplification in RelationGetIndexAttrBitmap(), so that it becomes safe to call in these contexts. The constant expressions simplification comes from the calls to RelationGetIndexExpressions()/RelationGetIndexPredicate() via BuildIndexInfo(). But RelationGetIndexAttrBitmap() calling BuildIndexInfo() is overkill. The latter just takes pg_index catalog information, packs it into the IndexInfo structure, which former then just unpacks again and throws away. We can just do this directly with less overhead and skip the troublesome calls to eval_const_expressions(). This also removes the awkward cross-dependency between relcache.c and index.c. Bug: #15114 Reported-by: Петър Славов <email@example.com> Reviewed-by: Noah Misch <firstname.lastname@example.org> Reviewed-by: Michael Paquier <email@example.com> Discussion: https://firstname.lastname@example.org/
Fix psql's "\g target" meta-command to work with COPY TO STDOUT.
commit : ae4c7d5ab88668275ab69913169564ce9bcbf27d author : Tom Lane <email@example.com> date : Sat, 26 Jan 2019 14:15:42 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Sat, 26 Jan 2019 14:15:42 -0500
Previously, \g would successfully execute the COPY command, but the target specification if any was ignored, so that the data was always dumped to the regular query output target. This seems like a clear bug, so let's not just fix it but back-patch it. While at it, adjust the documentation for \copy to recommend "COPY ... TO STDOUT \g foo" as a plausible alternative. Back-patch to 9.5. The problem exists much further back, but the code associated with \g was refactored enough in 9.5 that we'd need a significantly different patch for 9.4, and it doesn't seem worth the trouble. Daniel Vérité, reviewed by Fabien Coelho Discussion: https://email@example.com
Allow UNLISTEN in hot-standby mode.
commit : 0d5b2735122557417e24a73f92a518638d2a5a32 author : Tom Lane <firstname.lastname@example.org> date : Fri, 25 Jan 2019 21:14:31 -0500 committer: Tom Lane <email@example.com> date : Fri, 25 Jan 2019 21:14:31 -0500
Since LISTEN is (still) disallowed, UNLISTEN must be a no-op in a hot-standby session, and so there's no harm in allowing it. This change allows client code to not worry about whether it's connected to a primary or standby server when performing session-state-reset type activities. (Note that DISCARD ALL, which includes UNLISTEN, was already allowed, making it inconsistent to reject UNLISTEN.) Per discussion, back-patch to all supported versions. Shay Rojansky, reviewed by Mi Tar Discussion: https://postgr.es/m/CADT4RqCf2gA_TJtPAjnGzkC3ZiexfBZiLmA-mV66e4UyuVv8bA@mail.gmail.com
Remove infinite-loop hazards in ecpg test suite.
commit : adc2553b7069abb46ea72a663eb0208f98bca29e author : Tom Lane <firstname.lastname@example.org> date : Thu, 24 Jan 2019 16:46:55 -0500 committer: Tom Lane <email@example.com> date : Thu, 24 Jan 2019 16:46:55 -0500
A report from Andrew Dunstan showed that an ecpglib breakage that causes repeated query failures could lead to infinite loops in some ecpg test scripts, because they contain "while(1)" loops with no exit condition other than successful test completion. That might be all right for manual testing, but it seems entirely unacceptable for automated test environments such as our buildfarm. We don't want buildfarm owners to have to intervene manually when a test goes wrong. To fix, just change all those while(1) loops to exit after at most 100 iterations (which is more than any of them expect to iterate). This seems sufficient since we'd see discrepancies in the test output if any loop executed the wrong number of times. I tested this by dint of intentionally breaking ecpg_do_prologue to always fail, and verifying that the tests still got to completion. Back-patch to all supported branches, since the whole point of this exercise is to protect the buildfarm against future mistakes. Discussion: https://firstname.lastname@example.org
Blind attempt to fix _configthreadlocale() failures on MinGW.
commit : 1d334ab6e0b5845cb8d4d68edc706a990774a3b9 author : Tom Lane <email@example.com> date : Wed, 23 Jan 2019 22:46:45 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Wed, 23 Jan 2019 22:46:45 -0500
Apparently, some builds of MinGW contain a version of _configthreadlocale() that always returns -1, indicating failure. Rather than treating that as a curl-up-and-die condition, soldier on as though the function didn't exist. This leaves us without thread safety on such MinGW versions, but we didn't have it anyway. Discussion: https://postgr.es/m/d06a16bc-52d6-9f0d-2379-21242d7dbe81@2ndQuadrant.com
Fix misc typos in comments.
commit : 6bf9f4e6e38b0b8d6f4ba26da473d6d5bb7fa49d author : Heikki Linnakangas <email@example.com> date : Wed, 23 Jan 2019 13:39:00 +0200 committer: Heikki Linnakangas <firstname.lastname@example.org> date : Wed, 23 Jan 2019 13:39:00 +0200
Spotted mostly by Fabien Coelho. Discussion: https://www.postgresql.org/message-id/alpine.DEB.2.21.1901230947050.16643@lancre
Avoid thread-safety problem in ecpglib.
commit : eb3e90cad1e48ce5ad55335514d639744a295b78 author : Tom Lane <email@example.com> date : Mon, 21 Jan 2019 23:18:58 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Mon, 21 Jan 2019 23:18:58 -0500
ecpglib attempts to force the LC_NUMERIC locale to "C" while reading server output, to avoid problems with strtod() and related functions. Historically it's just issued setlocale() calls to do that, but that has major problems if we're in a threaded application. setlocale() itself is not required by POSIX to be thread-safe (and indeed is not, on recent OpenBSD). Moreover, its effects are process-wide, so that we could cause unexpected results in other threads, or another thread could change our setting. On platforms having uselocale(), which is required by POSIX:2008, we can avoid these problems by using uselocale() instead. Windows goes its own way as usual, but we can make it safe by using _configthreadlocale(). Platforms having neither continue to use the old code, but that should be pretty much nobody among current systems. (Subsequent buildfarm results show that recent NetBSD versions still lack uselocale(), but it's not a big problem because they also do not support non-"C" settings for LC_NUMERIC.) Back-patch of commits 8eb4a9312 and ee27584c4. Michael Meskes and Tom Lane; thanks also to Takayuki Tsunakawa. Discussion: https://email@example.com
Flush relcache entries when their FKs are meddled with
commit : 4aead13a751f86a9e143d3ee931171a11fd7bf7e author : Alvaro Herrera <firstname.lastname@example.org> date : Mon, 21 Jan 2019 19:34:11 -0300 committer: Alvaro Herrera <email@example.com> date : Mon, 21 Jan 2019 19:34:11 -0300
Back in commit 100340e2dcd0, we made relcache entries keep lists of the foreign keys applying to the relation -- but we forgot to update CacheInvalidateHeapTuple to flush those entries when new FKs got created or existing ones updated/deleted. No bugs appear to have been reported that would be explained by this ommission, but I noticed the problem while working on an unrelated bugfix which clearly showed it. Fix by adding relcache flush on relevant foreign key changes. Backpatch to 9.6, like the aforementioned commit. Discussion: https://firstname.lastname@example.org Reviewed-by: Tom Lane
Revert "Add valgrind suppressions for wcsrtombs optimizations"
commit : 058e2757b9bf44968af4cc93cd8334e3dba6f362 author : Tomas Vondra <email@example.com> date : Sat, 19 Jan 2019 20:42:44 +0100 committer: Tomas Vondra <firstname.lastname@example.org> date : Sat, 19 Jan 2019 20:42:44 +0100
This reverts commit 71b2951ccc433a6dd799efedff9f5927c32440f6. Per discussion, it's not desirable to add valgrind suppressions for outside our own code base (e.g. glibc in this case), especially when the suppressions may be platform-specific. There are better ways to deal with that, e.g. by providing local suppressions. Discussion: https://www.postgresql.org/message-id/flat/90ac0452-e907-e7a4-b3c8-15bd33780e62%402ndquadrant.com
Use our own getopt() on OpenBSD.
commit : 167ba6b1569d5a96d0452fbe708e866222d5b8a8 author : Tom Lane <email@example.com> date : Fri, 18 Jan 2019 15:06:26 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Fri, 18 Jan 2019 15:06:26 -0500
Recent OpenBSD (at least 5.9 and up) has a version of getopt(3) that will not cope with the "-:" spec we use to accept double-dash options in postgres.c and postmaster.c. Admittedly, that's a hack because POSIX only requires getopt() to allow alphanumeric option characters. I have no desire to find another way, however, so let's just do what we were already doing on Solaris: force use of our own src/port/getopt.c implementation. In passing, improve some of the comments around said implementation. Per buildfarm and local testing. Back-patch to all supported branches. Discussion: https://email@example.com
Replace references to mailinglists with @lists.postgresql.org
commit : 9450b91cdf140e7becb5e9bb477da0c1f6cdbdb1 author : Magnus Hagander <firstname.lastname@example.org> date : Thu, 17 Jan 2019 13:52:51 +0100 committer: Magnus Hagander <email@example.com> date : Thu, 17 Jan 2019 13:52:51 +0100
The namespace for all lists have changed a while ago, so all references should use the correct address.
Remove references to Majordomo
commit : 14a0481194ee51623197b911b3719b0a0b949039 author : Magnus Hagander <firstname.lastname@example.org> date : Thu, 17 Jan 2019 13:47:24 +0100 committer: Magnus Hagander <email@example.com> date : Thu, 17 Jan 2019 13:47:24 +0100
Lists are not handled by Majordomo anymore and haven't been for a while, so remove the reference and instead direct people to the list server.
Postpone aggregate checks until after collation is assigned.
commit : 624046abe23f3da003b8db07d7028c70f1c672bd author : Andrew Gierth <firstname.lastname@example.org> date : Thu, 17 Jan 2019 05:33:01 +0000 committer: Andrew Gierth <email@example.com> date : Thu, 17 Jan 2019 05:33:01 +0000
Previously, parseCheckAggregates was run before assign_query_collations, but this causes problems if any expression has already had a collation assigned by some transform function (e.g. transformCaseExpr) before parseCheckAggregates runs. The differing collations would cause expressions not to be recognized as equal to the ones in the GROUP BY clause, leading to spurious errors about unaggregated column references. The result was that CASE expr WHEN val ... would fail when "expr" contained a GROUPING() expression or matched one of the group by expressions, and where collatable types were involved; whereas the supposedly identical CASE WHEN expr = val ... would succeed. Backpatch all the way; this appears to have been wrong ever since collations were introduced. Per report from Guillaume Lelarge, analysis and patch by me. Discussion: https://postgr.es/m/CAECtzeVSO_US8C2Khgfv54ZMUOBR4sWq+6_bLrETnWExHT=rFg@mail.gmail.com Discussion: https://firstname.lastname@example.org
Fix typo in documentation
commit : 381409686a81c2680c3d6ac416b5a9d823f6cf83 author : Michael Paquier <email@example.com> date : Tue, 15 Jan 2019 08:47:20 +0900 committer: Michael Paquier <firstname.lastname@example.org> date : Tue, 15 Jan 2019 08:47:20 +0900
These have been found while cross-checking for the use of unique words in the documentation. Author: Alexander Lakhin Discussion: https://email@example.com
commit : e0f9fade3591796c1c47523bdc010b1867ef74d7 author : Andrew Dunstan <firstname.lastname@example.org> date : Sun, 13 Jan 2019 16:43:14 -0500 committer: Andrew Dunstan <email@example.com> date : Sun, 13 Jan 2019 16:43:14 -0500
Make DLSUFFIX easily discoverable by build scripts
commit : 149f91fa0b473656cf1733f6e455bdaaa2eded23 author : Andrew Dunstan <firstname.lastname@example.org> date : Sun, 13 Jan 2019 15:59:35 -0500 committer: Andrew Dunstan <email@example.com> date : Sun, 13 Jan 2019 15:59:35 -0500
This will enable things like the buildfarm client to discover more reliably if certain libraries have been installed. Discussion: https://postgr.es/m/859e7c91-7ef4-d4b4-2ca2-8046e0cbee09@2ndQuadrant.com Backpatch to all live branches.
Avoid sharing PARAM_EXEC slots between different levels of NestLoop.
commit : 4f80974990dce96771ef0e44c45999d8cc029c22 author : Tom Lane <firstname.lastname@example.org> date : Fri, 11 Jan 2019 15:53:34 -0500 committer: Tom Lane <email@example.com> date : Fri, 11 Jan 2019 15:53:34 -0500
Up to now, createplan.c attempted to share PARAM_EXEC slots for NestLoopParams across different plan levels, if the same underlying Var was being fed down to different righthand-side subplan trees by different NestLoops. This was, I think, more of an artifact of using subselect.c's PlannerParamItem infrastructure than an explicit design goal, but anyway that was the end result. This works well enough as long as the plan tree is executing synchronously, but the feature whereby Gather can execute the parallelized subplan locally breaks it. An upper NestLoop node might execute for a row retrieved from a parallel worker, and assign a value for a PARAM_EXEC slot from that row, while the leader's copy of the parallelized subplan is suspended with a different active value of the row the Var comes from. When control eventually returns to the leader's subplan, it gets the wrong answers if the same PARAM_EXEC slot is being used within the subplan, as reported in bug #15577 from Bartosz Polnik. This is pretty reminiscent of the problem fixed in commit 46c508fbc, and the proper fix seems to be the same: don't try to share PARAM_EXEC slots across different levels of controlling NestLoop nodes. This requires decoupling NestLoopParam handling from PlannerParamItem handling, although the logic remains somewhat similar. To avoid bizarre division of labor between subselect.c and createplan.c, I decided to move all the param-slot-assignment logic for both cases out of those files and put it into a new file paramassign.c. Hopefully it's a bit better documented now, too. A regression test case for this might be nice, but we don't know a test case that triggers the problem with a suitably small amount of data. Back-patch to 9.6 where we added Gather nodes. It's conceivable that related problems exist in older branches; but without some evidence for that, I'll leave the older branches alone. Discussion: https://firstname.lastname@example.org
Doc: update our docs about kernel IPC parameters on *BSD.
commit : 2763cc4a0367ef21601ecdc0c3ab1c5db24e458d author : Tom Lane <email@example.com> date : Tue, 8 Jan 2019 12:03:54 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Tue, 8 Jan 2019 12:03:54 -0500
runtime.sgml said that you couldn't change SysV IPC parameters on OpenBSD except by rebuilding the kernel. That's definitely wrong in OpenBSD 6.x, and excavation in their man pages says it changed in OpenBSD 3.3. Update NetBSD and OpenBSD sections to recommend adjustment of the SEMMNI and SEMMNS settings, which are painfully small by default on those platforms. (The discussion thread contemplated recommending that people select POSIX semaphores instead, but the performance consequences of that aren't really clear, so I'll refrain.) Remove pointless discussion of SEMMNU and SEMMAP from the FreeBSD section. Minor other wordsmithing. Discussion: https://email@example.com
doc: document that INFO messages always go to client.
commit : bbce27c6dd6ddaed3adc5c149b4f7053a62fc119 author : Andrew Gierth <firstname.lastname@example.org> date : Mon, 7 Jan 2019 18:19:46 +0000 committer: Andrew Gierth <email@example.com> date : Mon, 7 Jan 2019 18:19:46 +0000
In passing add a couple of links to the message severity table. Backpatch because it's always been this way. Author: Karl O. Pinc <firstname.lastname@example.org>
doc: Update RFC URLs
commit : feb4b80388696a3493044c4ea8554e41f8548a85 author : Michael Paquier <email@example.com> date : Sat, 5 Jan 2019 12:48:55 +0900 committer: Michael Paquier <firstname.lastname@example.org> date : Sat, 5 Jan 2019 12:48:55 +0900
Consistently use the IETF HTML links instead of a random mix of different sites and formats. This also fixes one broken link for JSON documentation.
Improve ANALYZE's handling of concurrent-update scenarios.
commit : 60de80cd2dacdc843374cd3722deb10fc72b1345 author : Tom Lane <email@example.com> date : Thu, 3 Jan 2019 17:00:08 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Thu, 3 Jan 2019 17:00:08 -0500
This patch changes the rule for whether or not a tuple seen by ANALYZE should be included in its sample. When we last touched this logic, in commit 51e1445f1, we weren't thinking very hard about tuples being UPDATEd by a long-running concurrent transaction. In such a case, we might see the pre-image as either LIVE or DELETE_IN_PROGRESS depending on timing; and we might see the post-image not at all, or as INSERT_IN_PROGRESS. Since the existing code will not sample either DELETE_IN_PROGRESS or INSERT_IN_PROGRESS tuples, this leads to concurrently-updated rows being omitted from the sample entirely. That's not very helpful, and it's especially the wrong thing if the concurrent transaction ends up rolling back. The right thing seems to be to sample DELETE_IN_PROGRESS rows just as if they were live. This makes the "sample it" and "count it" decisions the same, which seems good for consistency. It's clearly the right thing if the concurrent transaction ends up rolling back; in effect, we are sampling as though IN_PROGRESS transactions haven't happened yet. Also, this combination of choices ensures maximum robustness against the different combinations of whether and in which state we might see the pre- and post-images of an update. It's slightly annoying that we end up recording immediately-out-of-date stats in the case where the transaction does commit, but on the other hand the stats are fine for columns that didn't change in the update. And the alternative of sampling INSERT_IN_PROGRESS rows instead seems like a bad idea, because then the sampling would be inconsistent with the way rows are counted for the stats report. Per report from Mark Chambers; thanks to Jeff Janes for diagnosing what was happening. Back-patch to all supported versions. Discussion: https://postgr.es/m/CAFh58O_Myr6G3tcH3gcGrF-=OExB08PJdWZcSBcEcovaiPsrHA@mail.gmail.com
Update ssl test certificates and keys
commit : ba26b3172f037b3afa863aaa4db67aa2bcba81db author : Peter Eisentraut <email@example.com> date : Thu, 3 Jan 2019 15:06:53 +0100 committer: Peter Eisentraut <firstname.lastname@example.org> date : Thu, 3 Jan 2019 15:06:53 +0100
Debian testing and newer now require that RSA and DHE keys are at least 2048 bit long and no longer allow SHA-1 for signatures in certificates. This is currently causing the ssl tests to fail there because the test certificates and keys have been created in violation of those conditions. Update the parameters to create the test files and create a new set of test files. Author: Kyotaro HORIGUCHI <email@example.com> Reported-by: Michael Paquier <firstname.lastname@example.org> Discussion: https://www.postgresql.org/message-id/flat/20180917131340.GE31460%40paquier.xyz
Don't believe MinMaxExpr is leakproof without checking.
commit : c27c3993ef6dbdff1f196670877a47f374d884ad author : Tom Lane <email@example.com> date : Wed, 2 Jan 2019 16:33:48 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Wed, 2 Jan 2019 16:33:48 -0500
MinMaxExpr invokes the btree comparison function for its input datatype, so it's only leakproof if that function is. Many such functions are indeed leakproof, but others are not, and we should not just assume that they are. Hence, adjust contain_leaked_vars to verify the leakproofness of the referenced function explicitly. I didn't add a regression test because it would need to depend on some particular comparison function being leaky, and that's a moving target, per discussion. This has been wrong all along, so back-patch to supported branches. Discussion: https://email@example.com
Update copyright for 2019
commit : b66a07430e9d17ad892c5e3241c19b1b33d73e46 author : Bruce Momjian <firstname.lastname@example.org> date : Wed, 2 Jan 2019 12:44:25 -0500 committer: Bruce Momjian <email@example.com> date : Wed, 2 Jan 2019 12:44:25 -0500
Backpatch-through: certain files through 9.4
pg_regress: Promptly detect failed postmaster startup.
commit : 13602a9b6b72d0152200a146fef16e12d7a4a10a author : Noah Misch <firstname.lastname@example.org> date : Mon, 31 Dec 2018 13:50:32 -0800 committer: Noah Misch <email@example.com> date : Mon, 31 Dec 2018 13:50:32 -0800
Detect it the way pg_ctl's wait_for_postmaster() does. When pg_regress spawned a postmaster that failed startup, we were detecting that only with "pg_regress: postmaster did not respond within 60 seconds". Back-patch to 9.4 (all supported versions). Reviewed by Tom Lane. Discussion: https://postgr.es/m/20181231172922.GA199150@gust.leadboat.com
pg_rewind: Add missing newline to error message
commit : 55fde981c299550d96742799f1bfa36220cc1b7c author : Peter Eisentraut <firstname.lastname@example.org> date : Sat, 29 Dec 2018 13:02:51 +0100 committer: Peter Eisentraut <email@example.com> date : Sat, 29 Dec 2018 13:02:51 +0100
Have DISCARD ALL/TEMP remove leftover temp tables
commit : aa2e84d2cc8dd08f3531a1f36eaa0490117e5baa author : Alvaro Herrera <firstname.lastname@example.org> date : Thu, 27 Dec 2018 16:17:40 -0300 committer: Alvaro Herrera <email@example.com> date : Thu, 27 Dec 2018 16:17:40 -0300
Previously, it would only remove temp tables created in the same session; but if the session uses the BackendId of a previously crashed backend that left temp tables around, those would not get removed. Since autovacuum would not drop them either (because it sees that the BackendId is in use by the current session) these can cause annoying xid-wraparound warnings. Apply to branches 9.4 to 10. This is not a problem since version 11, because commit 943576bddcb5 added state tracking that makes autovacuum realize that those temp tables are not ours, so it removes them. This is useful to handle in DISCARD, because even though it does not handle all situations, it does handle the common one where a connection pooler keeps the same session open for an indefinitely long time. Discussion: https://firstname.lastname@example.org Reviewed-by: Takayuki Tsunakawa, Michaël Paquier
Make autovacuum more selective about temp tables to keep
commit : 7fc2363a615842e42f927d915331a4b3bd67af19 author : Alvaro Herrera <email@example.com> date : Thu, 27 Dec 2018 16:00:39 -0300 committer: Alvaro Herrera <firstname.lastname@example.org> date : Thu, 27 Dec 2018 16:00:39 -0300
When temp tables are in danger of XID wraparound, autovacuum drops them; however, it preserves those that are owned by a working session. This is desirable, except when the session is connected to a different database (because the temp tables cannot be from that session), so make it only keep the temp tables only if the backend is in the same database as the temp tables. This is not bulletproof: it fails to detect temp tables left by a session whose backend ID is reused in the same database but the new session does not use temp tables. Commit 943576bddcb5 fixes that case too, for branches 11 and up (which is why we don't apply this fix to those branches), but back-patching that one is not universally agreed on. Discussion: https://email@example.com Reviewed-by: Takayuki Tsunakawa, Michaël Paquier
Ignore inherited temp relations from other sessions when truncating
commit : 285abc8df4d7741fc4e303fb6a906a364345ebbe author : Michael Paquier <firstname.lastname@example.org> date : Thu, 27 Dec 2018 10:17:21 +0900 committer: Michael Paquier <email@example.com> date : Thu, 27 Dec 2018 10:17:21 +0900
Inheritance trees can include temporary tables if the parent is permanent, which makes possible the presence of multiple temporary children from different sessions. Trying to issue a TRUNCATE on the parent in this scenario causes a failure, so similarly to any other queries just ignore such cases, which makes TRUNCATE work transparently. This makes truncation behave similarly to any other DML query working on the parent table with queries which need to be issues on children. A set of isolation tests is added to cover basic cases. Reported-by: Zhou Digoal Author: Amit Langote, Michael Paquier Discussion: https://firstname.lastname@example.org Backpatch-through: 9.4
Fix portability failure introduced in commits d2b0b60e7 et al.
commit : 811821748b5b833766444325c105b4bc25aac847 author : Tom Lane <email@example.com> date : Wed, 26 Dec 2018 15:30:10 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Wed, 26 Dec 2018 15:30:10 -0500
I made a frontend fprintf() format use %m, forgetting that that's only safe in HEAD not the back branches; prior to 96bf88d52 and d6c55de1f, it would work on glibc platforms but not elsewhere. Revert to using %s ... strerror(errno) as the code did before. We could have left HEAD as-is, but for code consistency across branches, I chose to apply this patch there too. Per Coverity and a few buildfarm members.
Prioritize history files when archiving
commit : 9378701243b5a9075a5c0e11909cb3eec7428801 author : Michael Paquier <email@example.com> date : Mon, 24 Dec 2018 20:26:11 +0900 committer: Michael Paquier <firstname.lastname@example.org> date : Mon, 24 Dec 2018 20:26:11 +0900
At the end of recovery for the post-promotion process, a new history file is created followed by the last partial segment of the previous timeline. Based on the timing, the archiver would first try to archive the last partial segment and then the history file. This can delay the detection of a new timeline taken, particularly depending on the time it takes to transfer the last partial segment as it delays the moment the history file of the new timeline gets archived. This can cause promoted standbys to use the same timeline as one already taken depending on the circumstances if multiple instances look at archives at the same location. This commit changes the order of archiving so as history files are archived in priority over other file types, which reduces the likelihood of the same timeline being taken (still not reducing the window to zero), and it makes the archiver behave more consistently with the startup process doing its post-promotion business. Author: David Steele Reviewed-by: Michael Paquier, Kyotaro Horiguchi Discussion: https://email@example.com Backpatch-through: 9.5
Fix ancient compiler warnings and typos in !HAVE_SYMLINK code
commit : ffcd98c7fcfb2ed4ef9069591c06d3748669941c author : Peter Eisentraut <firstname.lastname@example.org> date : Sat, 22 Dec 2018 07:21:40 +0100 committer: Peter Eisentraut <email@example.com> date : Sat, 22 Dec 2018 07:21:40 +0100
This has never been correct since this code was introduced.
Check for conflicting queries during replay of gistvacuumpage()
commit : 1b02807457318fa61cb564b101669f82e1b7be23 author : Alexander Korotkov <firstname.lastname@example.org> date : Fri, 21 Dec 2018 02:33:37 +0300 committer: Alexander Korotkov <email@example.com> date : Fri, 21 Dec 2018 02:33:37 +0300
013ebc0a7b implements so-called GiST microvacuum. That is gistgettuple() marks index tuples as dead when kill_prior_tuple is set. Later, when new tuple insertion claims page space, those dead index tuples are physically deleted from page. When this deletion is replayed on standby, it might conflict with read-only queries. But 013ebc0a7b doesn't handle this. That may lead to disappearance of some tuples from read-only snapshots on standby. This commit implements resolving of conflicts between replay of GiST microvacuum and standby queries. On the master we implement new WAL record type XLOG_GIST_DELETE, which comprises necessary information. On stable releases we've to be tricky to keep WAL compatibility. Information required for conflict processing is just appended to data of XLOG_GIST_PAGE_UPDATE record. So, PostgreSQL version, which doesn't know about conflict processing, will just ignore that. Reported-by: Andres Freund Diagnosed-by: Andres Freund Discussion: https://postgr.es/m/20181212224524.scafnlyjindmrbe6%40alap3.anarazel.de Author: Alexander Korotkov Backpatch-through: 9.6
Doc: fix ancient mistake in search_path documentation.
commit : e13d8a783d25691a63370c4b03acb7254934a041 author : Tom Lane <firstname.lastname@example.org> date : Thu, 20 Dec 2018 13:55:11 -0500 committer: Tom Lane <email@example.com> date : Thu, 20 Dec 2018 13:55:11 -0500
"$user" in a search_path string is replaced by CURRENT_USER not SESSION_USER. (It actually was SESSION_USER in the initial implementation, but we changed it shortly later, and evidently forgot to fix the docs to match.) Noted by firstname.lastname@example.org Discussion: https://email@example.com
Fix ADD IF NOT EXISTS used in conjunction with ALTER TABLE ONLY
commit : 5668afeb6d25ef842ff14239c5ba3aa20cf56937 author : Greg Stark <firstname.lastname@example.org> date : Wed, 19 Dec 2018 18:28:35 -0500 committer: Greg Stark <email@example.com> date : Wed, 19 Dec 2018 18:28:35 -0500
The flag for IF NOT EXISTS was only being passed down in the normal recursing case. It's been this way since originally added in 9.6 in commit 2cd40adb85 so backpatch back to 9.6.
Fix ancient thinko in mergejoin cost estimation.
commit : 7c2bc40b8b78442d414db76366e6d15d2fa6ebd0 author : Tom Lane <firstname.lastname@example.org> date : Tue, 18 Dec 2018 11:19:39 -0500 committer: Tom Lane <email@example.com> date : Tue, 18 Dec 2018 11:19:39 -0500
"rescanratio" was computed as 1 + rescanned-tuples / total-inner-tuples, which is sensible if it's to be multiplied by total-inner-tuples or a cost value corresponding to scanning all the inner tuples. But in reality it was (mostly) multiplied by inner_rows or a related cost, numbers that take into account the possibility of stopping short of scanning the whole inner relation thanks to a limited key range in the outer relation. This'd still make sense if we could expect that stopping short would result in a proportional decrease in the number of tuples that have to be rescanned. It does not, however. The argument that establishes the validity of our estimate for that number is independent of whether we scan all of the inner relation or stop short, and experimentation also shows that stopping short doesn't reduce the number of rescanned tuples. So the correct calculation is 1 + rescanned-tuples / inner_rows, and we should be sure to multiply that by inner_rows or a corresponding cost value. Most of the time this doesn't make much difference, but if we have both a high rescan rate (due to lots of duplicate values) and an outer key range much smaller than the inner key range, then the error can be significant, leading to a large underestimate of the cost associated with rescanning. Per report from Vijaykumar Jain. This thinko appears to go all the way back to the introduction of the rescan estimation logic in commit 70fba7043, so back-patch to all supported branches. Discussion: https://postgr.es/m/CAE7uO5hMb_TZYJcZmLAgO6iD68AkEK6qCe7i=vZUkCpoKns+EQ@mail.gmail.com
Update project link of pgBadger in documentation
commit : 91c39daa3a4760f082a58133bb57b42b6965a5c3 author : Michael Paquier <firstname.lastname@example.org> date : Tue, 18 Dec 2018 10:03:08 +0900 committer: Michael Paquier <email@example.com> date : Tue, 18 Dec 2018 10:03:08 +0900
The project has moved to a new place. Reported-by: Peter Neave Discussion: https://firstname.lastname@example.org
Fix use-after-free bug when renaming constraints
commit : 419bd6371d557f5aa687e56ebe04e4946db0b8a4 author : Michael Paquier <email@example.com> date : Mon, 17 Dec 2018 12:43:57 +0900 committer: Michael Paquier <firstname.lastname@example.org> date : Mon, 17 Dec 2018 12:43:57 +0900
This is an oversight from recent commit b13fd344. While on it, tweak the previous test with a better name for the renamed primary key. Detected by buildfarm member prion which forces relation cache release with -DRELCACHE_FORCE_RELEASE. Back-patch down to 9.4 as the previous commit.
Make constraint rename issue relcache invalidation on target relation
commit : d79cd555da115a15f6e63f5e687521d1e274ff9a author : Michael Paquier <email@example.com> date : Mon, 17 Dec 2018 10:36:29 +0900 committer: Michael Paquier <firstname.lastname@example.org> date : Mon, 17 Dec 2018 10:36:29 +0900
When a constraint gets renamed, it may have associated with it a target relation (for example domain constraints don't have one). Not invalidating the target relation cache when issuing the renaming can result in issues with subsequent commands that refer to the old constraint name using the relation cache, causing various failures. One pattern spotted was using CREATE TABLE LIKE after a constraint renaming. Reported-by: Stuart <email@example.com> Author: Amit Langote Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/2047094.V130LYfLq4@station53.ousa.org
Make error handling in parallel pg_upgrade less bogus.
commit : f7642cf31601b206cb7cfb99339e6b4edfec0b58 author : Tom Lane <firstname.lastname@example.org> date : Sun, 16 Dec 2018 14:51:48 -0500 committer: Tom Lane <email@example.com> date : Sun, 16 Dec 2018 14:51:48 -0500
reap_child() basically ignored the possibility of either an error in waitpid() itself or a child process failure on signal. We don't really need to do more than report and crash hard, but proceeding as though nothing is wrong is definitely Not Acceptable. The error report for nonzero child exit status was pretty off-point, as well. Noted while fooling around with child-process failure detection logic elsewhere. It's been like this a long time, so back-patch to all supported branches.
Prevent GIN deleted pages from being reclaimed too early
commit : cd24b4eaece3ee816b8a7decacc8dd00e5f16906 author : Alexander Korotkov <firstname.lastname@example.org> date : Thu, 13 Dec 2018 06:12:31 +0300 committer: Alexander Korotkov <email@example.com> date : Thu, 13 Dec 2018 06:12:31 +0300
When GIN vacuum deletes a posting tree page, it assumes that no concurrent searchers can access it, thanks to ginStepRight() locking two pages at once. However, since 9.4 searches can skip parts of posting trees descending from the root. That leads to the risk that page is deleted and reclaimed before concurrent search can access it. This commit prevents the risk of above by waiting for every transaction, which might wait to reference this page, to finish. Due to binary compatibility we can't change GinPageOpaqueData to store corresponding transaction id. Instead we reuse page header pd_prune_xid field, which is unused in index pages. Discussion: https://postgr.es/m/31a702a.14dd.166c1366ac1.Coremail.chjischj%40163.com Author: Andrey Borodin, Alexander Korotkov Reviewed-by: Alexander Korotkov Backpatch-through: 9.4
Prevent deadlock in ginRedoDeletePage()
commit : 80d4d8d71d57cc336e00e448f14c1b70ad8bc85f author : Alexander Korotkov <firstname.lastname@example.org> date : Thu, 13 Dec 2018 06:12:25 +0300 committer: Alexander Korotkov <email@example.com> date : Thu, 13 Dec 2018 06:12:25 +0300
On standby ginRedoDeletePage() can work concurrently with read-only queries. Those queries can traverse posting tree in two ways. 1) Using rightlinks by ginStepRight(), which locks the next page before unlocking its left sibling. 2) Using downlinks by ginFindLeafPage(), which locks at most one page at time. Original lock order was: page, parent, left sibling. That lock order can deadlock with ginStepRight(). In order to prevent deadlock this commit changes lock order to: left sibling, page, parent. Note, that position of parent in locking order seems insignificant, because we only lock one page at time while traversing downlinks. Reported-by: Chen Huajun Diagnosed-by: Chen Huajun, Peter Geoghegan, Andrey Borodin Discussion: https://postgr.es/m/31a702a.14dd.166c1366ac1.Coremail.chjischj%40163.com Author: Alexander Korotkov Backpatch-through: 9.4
Repair bogus EPQ plans generated for postgres_fdw foreign joins.
commit : 12cb7ea599fb651bddadcff5d55188cdc4633190 author : Tom Lane <firstname.lastname@example.org> date : Wed, 12 Dec 2018 16:08:30 -0500 committer: Tom Lane <email@example.com> date : Wed, 12 Dec 2018 16:08:30 -0500
postgres_fdw's postgresGetForeignPlan() assumes without checking that the outer_plan it's given for a join relation must have a NestLoop, MergeJoin, or HashJoin node at the top. That's been wrong at least since commit 4bbf6edfb (which could cause insertion of a Sort node on top) and it seems like a pretty unsafe thing to Just Assume even without that. Through blind good fortune, this doesn't seem to have any worse consequences today than strange EXPLAIN output, but it's clearly trouble waiting to happen. To fix, test the node type explicitly before touching Join-specific fields, and avoid jamming the new tlist into a node type that can't do projection. Export a new support function from createplan.c to avoid building low-level knowledge about the latter into FDWs. Back-patch to 9.6 where the faulty coding was added. Note that the associated regression test cases don't show any changes before v11, apparently because the tests back-patched with 4bbf6edfb don't actually exercise the problem case before then (there's no top-level Sort in those plans). Discussion: https://firstname.lastname@example.org
Repair bogus handling of multi-assignment Params in upper plan levels.
commit : 10bad8c0f02e43dd2448316c4658f50b75f9cba8 author : Tom Lane <email@example.com> date : Wed, 12 Dec 2018 13:49:42 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Wed, 12 Dec 2018 13:49:42 -0500
Our support for multiple-set-clauses in UPDATE assumes that the Params referencing a MULTIEXPR_SUBLINK SubPlan will appear before that SubPlan in the targetlist of the plan node that calculates the updated row. (Yeah, it's a hack...) In some PG branches it's possible that a Result node gets inserted between the primary calculation of the update tlist and the ModifyTable node. setrefs.c did the wrong thing in this case and left the upper-level Params as Params, causing a crash at runtime. What it should do is replace them with "outer" Vars referencing the child plan node's output. That's a result of careless ordering of operations in fix_upper_expr_mutator, so we can fix it just by reordering the code. Fix fix_join_expr_mutator similarly for consistency, even though join nodes could never appear in such a context. (In general, it seems likely to be a bit cheaper to use Vars than Params in such situations anyway, so this patch might offer a tiny performance improvement.) The hazard extends back to 9.5 where the MULTIEXPR_SUBLINK stuff was introduced, so back-patch that far. However, this may be a live bug only in 9.6.x and 10.x, as the other branches don't seem to want to calculate the final tlist below the Result node. (That plan shape change between branches might be a mini-bug in itself, but I'm not really interested in digging into the reasons for that right now. Still, add a regression test memorializing what we expect there, so we'll notice if it changes again.) Per bug report from Eduards Bezverhijs. Discussion: https://email@example.com
Fix test_rls_hooks to assign expression collations properly.
commit : d2267b61adef7811ab190dc2fa8c0399646434fe author : Tom Lane <firstname.lastname@example.org> date : Tue, 11 Dec 2018 11:48:00 -0500 committer: Tom Lane <email@example.com> date : Tue, 11 Dec 2018 11:48:00 -0500
This module overlooked this necessary fixup step on the results of transformWhereClause(). It accidentally worked anyway, because the constructed expression involved type "name" which is not collatable, but it fell over while I was experimenting with changing "name" to be collatable. Back-patch, not because there's any live bug here in back branches, but because somebody might use this code as a model for some real application and then not understand why it doesn't work.
Doc: improve documentation about ALTER LARGE OBJECT requirements.
commit : b48cfafd5fe74b98bc26da2e8873591971c8a800 author : Tom Lane <firstname.lastname@example.org> date : Tue, 11 Dec 2018 11:21:36 -0500 committer: Tom Lane <email@example.com> date : Tue, 11 Dec 2018 11:21:36 -0500
Unlike other ALTER ref pages, this one neglected to mention that ALTER OWNER requires being a member of the new owning role. Per bug #15546 from Stefan Kadow. Discussion: https://firstname.lastname@example.org
Raise some timeouts to 180s, in test code.
commit : 752278e7748e068088b3ac5ace7da82186a7dd6c author : Noah Misch <email@example.com> date : Mon, 10 Dec 2018 20:15:42 -0800 committer: Noah Misch <firstname.lastname@example.org> date : Mon, 10 Dec 2018 20:15:42 -0800
Slow runs of buildfarm members chipmunk, hornet and mandrill saw the shorter timeouts expire. The 180s timeout in poll_query_until has been trouble-free since 2a0f89cd717ce6d49cdc47850577823682167e87 introduced it two years ago, so use 180s more widely. Back-patch to 9.6, where the first of these timeouts was introduced. Reviewed by Michael Paquier. Discussion: https://postgr.es/m/20181209001601.GC2973271@rfd.leadboat.com
Add stack depth checks to key recursive functions in backend/nodes/*.c.
commit : 0f9fd7419a19c3bfa000117d8b475e134824259a author : Tom Lane <email@example.com> date : Mon, 10 Dec 2018 11:12:43 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Mon, 10 Dec 2018 11:12:43 -0500
Although copyfuncs.c has a check_stack_depth call in its recursion, equalfuncs.c, outfuncs.c, and readfuncs.c lacked one. This seems unwise. Likewise fix planstate_tree_walker(), in branches where that exists. Discussion: https://email@example.com
Fix misapplication of pgstat_count_truncate to wrong relation.
commit : 078303f79b513ef8f50fdf2a95181c61519e4559 author : Tom Lane <firstname.lastname@example.org> date : Fri, 7 Dec 2018 12:12:00 -0500 committer: Tom Lane <email@example.com> date : Fri, 7 Dec 2018 12:12:00 -0500
The stanza of ExecuteTruncate[Guts] that truncates a target table's toast relation re-used the loop local variable "rel" to reference the toast rel. This was safe enough when written, but commit d42358efb added code below that that supposed "rel" still pointed to the parent table. Therefore, the stats counter update was applied to the wrong relcache entry (the toast rel not the user rel); and if we were unlucky and that relcache entry had been flushed during reindex_relation, very bad things could ensue. (I'm surprised that CLOBBER_CACHE_ALWAYS testing hasn't found this. I'm even more surprised that the problem wasn't detected during the development of d42358efb; it must not have been tested in any case with a toast table, as the incorrect stats counts are very obvious.) To fix, replace use of "rel" in that code branch with a more local variable. Adjust test cases added by d42358efb so that some of them use tables with toast tables. Per bug #15540 from Pan Bian. Back-patch to 9.5 where d42358efb came in. Discussion: https://firstname.lastname@example.org
Improve our response to invalid format strings, and detect more cases.
commit : d398119c19ef4d19e1d87d19e2cbbd67c4e63556 author : Tom Lane <email@example.com> date : Thu, 6 Dec 2018 15:08:44 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Thu, 6 Dec 2018 15:08:44 -0500
Places that are testing for *printf failure ought to include the format string in their error reports, since bad-format-string is one of the more likely causes of such failure. This both makes it easier to find and repair the mistake, and provides at least some useful info to the user who stumbles across such a problem. Also, tighten snprintf.c to report EINVAL for an invalid flag or final character in a format %-spec (including the case where the %-spec is missing a final character altogether). This seems like better project policy, and it also allows removing an instruction or two from the hot code path. Back-patch the error reporting change in pvsnprintf, since it should be harmless and may be helpful; but not the snprintf.c change. Per discussion of bug #15511 from Ertuğrul Kahveci, which reported an invalid translated format string. These changes don't fix that error, but they should improve matters next time we make such a mistake. Discussion: https://email@example.com
Document handling of invalid/ambiguous timestamp input near DST boundaries.
commit : 4173a6180a161a6d1905f31c3f058dfec8f9056b author : Tom Lane <firstname.lastname@example.org> date : Thu, 29 Nov 2018 18:28:11 -0500 committer: Tom Lane <email@example.com> date : Thu, 29 Nov 2018 18:28:11 -0500
The source code comments documented this, but the user-facing docs, not so much. Add a section to Appendix B that discusses it. In passing, improve a couple other things in Appendix B --- notably, a long-obsolete claim that time zone abbreviations are looked up in a fixed table. Per bug #15527 from Michael Davidson. Discussion: https://firstname.lastname@example.org
Ensure static libraries have correct mod time even if ranlib messes it up.
commit : 8993d3a10d13475e0471fcd3cf66d3639f6094a8 author : Tom Lane <email@example.com> date : Thu, 29 Nov 2018 15:53:44 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Thu, 29 Nov 2018 15:53:44 -0500
In at least Apple's version of ranlib, the output file is updated to have a mod time equal to the max of the timestamps of its components, and that data only has seconds precision. On a filesystem with sub-second file timestamp precision --- say, APFS --- this can result in the finished static library appearing older than its input files, which causes useless rebuilds and possible outright failures in parallel makes. We've only seen this reported in the field from people using Apple's ranlib with a non-Apple make, because Apple's make doesn't know about sub-second timestamps either so it doesn't decide rebuilds are needed. But Apple's ranlib presumably shares code with at least some BSDen, so it's not that unlikely that the same problem could arise elsewhere. To fix, just "touch" the output file after ranlib finishes. We seem to need this in only one place. There are other calls of ranlib in our makefiles, but they are working on intermediate files whose timestamps are not actually important, or else on an installed static library for which sub-second timestamp precision is unlikely to matter either. (Also, so far as I can tell, Apple's ranlib doesn't mess up the file timestamp in the latter usage anyhow.) In passing, change "ranlib" to "$(RANLIB)" in one place that was bypassing the make macro for no good reason. Per bug #15525 from Jack Kelly (via Alyssa Ross). Back-patch to all supported branches. Discussion: https://email@example.com
Fix handling of synchronous replication for stopping WAL senders
commit : 8340eba6d8473b072a58f3fd50dbeb8fdb706cb2 author : Michael Paquier <firstname.lastname@example.org> date : Thu, 29 Nov 2018 09:12:53 +0900 committer: Michael Paquier <email@example.com> date : Thu, 29 Nov 2018 09:12:53 +0900
This fixes an oversight from c6c3334 which forgot that if a subset of WAL senders are stopping and in a sync state, other WAL senders could still be waiting for a WAL position to be synced while committing a transaction. However the subset of stopping senders would not release waiters, potentially breaking synchronous replication guarantees. This commit makes sure that even WAL senders stopping are able to release waiters and are tracked properly. On 9.4, this can also trigger an assertion failure when setting for example max_wal_senders to 1 where a WAL sender is not able to find itself as in synchronous state when the instance stops. Reported-by: Paul Guo Author: Paul Guo, Michael Paquier Discussion: https://postgr.es/m/CAEET0ZEv8VFqT3C-cQm6byOB4r4VYWcef1J21dOX-gcVhCSpmA@mail.gmail.com Backpatch-through: 9.4
Don't set PAM_RHOST for Unix sockets.
commit : 63d8350665c820203fd48e5690af01c33801fe46 author : Thomas Munro <firstname.lastname@example.org> date : Wed, 28 Nov 2018 14:00:57 +1300 committer: Thomas Munro <email@example.com> date : Wed, 28 Nov 2018 14:00:57 +1300
Since commit 2f1d2b7a we have set PAM_RHOST to "[local]" for Unix sockets. This caused Linux PAM's libaudit integration to make DNS requests for that name. It's not exactly clear what value PAM_RHOST should have in that case, but it seems clear that we shouldn't set it to an unresolvable name, so don't do that. Back-patch to 9.6. Bug #15520. Author: Thomas Munro Reviewed-by: Peter Eisentraut Reported-by: Albert Schabhuetl Discussion: https://postgr.es/m/15520-4c266f986998e1c5%40postgresql.org
Do not decode TOAST data for table rewrites
commit : b86d148ae339be7937ba8aa707de6905314a78c4 author : Tomas Vondra <firstname.lastname@example.org> date : Wed, 28 Nov 2018 01:11:15 +0100 committer: Tomas Vondra <email@example.com> date : Wed, 28 Nov 2018 01:11:15 +0100
During table rewrites (VACUUM FULL and CLUSTER), the main heap is logged using XLOG / FPI records, and thus (correctly) ignored in decoding. But the associated TOAST table is WAL-logged as plain INSERT records, and so was logically decoded and passed to reorder buffer. That has severe consequences with TOAST tables of non-trivial size. Firstly, reorder buffer has to keep all those changes, possibly spilling them to a file, incurring I/O costs and disk space. Secondly, ReoderBufferCommit() was stashing all those TOAST chunks into a hash table, which got discarded only after processing the row from the main heap. But as the main heap is not decoded for rewrites, this never happened, so all the TOAST data accumulated in memory, resulting either in excessive memory consumption or OOM. The fix is simple, as commit e9edc1ba already introduced infrastructure (namely HEAP_INSERT_NO_LOGICAL flag) to skip logical decoding of TOAST tables, but it only applied it to system tables. So simply use it for all TOAST data in raw_heap_insert(). That would however solve only the memory consumption issue - the TOAST changes would still be decoded and added to the reorder buffer, and spilled to disk (although without TOAST tuple data, so much smaller). But we can solve that by tweaking DecodeInsert() to just ignore such INSERT records altogether, using XLH_INSERT_CONTAINS_NEW_TUPLE flag, instead of skipping them later in ReorderBufferCommit(). Review: Masahiko Sawada Discussion: https://www.postgresql.org/message-id/flat/1a17c643-e9af-3dba-486b-fbe31bc1823a%402ndquadrant.com Backpatch: 9.4-, where logical decoding was introduced
Fix ac218aa4f6 to work on versions before 9.5.
commit : a52c31b8190518ac5776c228c314c946dc51eb62 author : Andres Freund <firstname.lastname@example.org> date : Mon, 26 Nov 2018 23:26:05 -0800 committer: Andres Freund <email@example.com> date : Mon, 26 Nov 2018 23:26:05 -0800
Unfortunately ac218aa4f6 missed the fact that a reference to 'pg_catalog.regnamespace'::regclass wouldn't work before that type is known. Fix that, by replacing the regtype usage with a join to pg_type. Reported-By: Tom Lane Author: Andres Freund Discussion: https://firstname.lastname@example.org Backpatch: 9.5-, like ac218aa4f6
Update pg_upgrade test for reg* to include regrole and regnamespace.
commit : 17024c08e8e07bebdb02917304c4aa2a9004d917 author : Andres Freund <email@example.com> date : Mon, 26 Nov 2018 17:00:43 -0800 committer: Andres Freund <firstname.lastname@example.org> date : Mon, 26 Nov 2018 17:00:43 -0800
When the regrole (0c90f6769) and regnamespace (cb9fa802b) types were added in 9.5, pg_upgrade's check for reg* types wasn't updated. While regrole currently is safe, regnamespace is not. It seems unlikely that anybody uses regnamespace inside catalog tables across a pg_upgrade, but the tests should be correct nevertheless. While at it, reorder the types checked in the query to be alphabetical. Otherwise it's annoying to compare existing and tested for types. Author: Andres Freund Discussion: https://postgr.es/m/037e152a-cb25-3bcb-4f35-bdc9988f8204@2ndQuadrant.com Backpatch: 9.5-, as regrole/regnamespace
doc: fix wording for plpgsql, add "and"
commit : e37d20e2ae90ba14385020d64fa377c2bd6fcdf6 author : Bruce Momjian <email@example.com> date : Mon, 26 Nov 2018 19:41:18 -0500 committer: Bruce Momjian <firstname.lastname@example.org> date : Mon, 26 Nov 2018 19:41:18 -0500
Reported-by: Anthony Greene Discussion: https://postgr.es/m/CAPRNmnsSZ4QL75FUjcS8ND_oV+WjgyPbZ4ch2RUwmW6PWzF38w@mail.gmail.com Backpatch-through: 9.4
Fix translation of special characters in psql's LaTeX output modes.
commit : ac305ff8c3dfe44d3441297f665ac3c602a87199 author : Tom Lane <email@example.com> date : Mon, 26 Nov 2018 17:32:51 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Mon, 26 Nov 2018 17:32:51 -0500
latex_escaped_print() mistranslated \ and failed to provide any translation for # ^ and ~, all of which would typically lead to LaTeX document syntax errors. In addition it didn't translate < > and |, which would typically render as unexpected characters. To some extent this represents shortcomings in ancient versions of LaTeX, which if memory serves had no easy way to render these control characters as ASCII text. But that's been fixed for, um, decades. In any case there is no value in emitting guaranteed-to-fail output for these characters. Noted while fooling with test cases added by commit 9a98984f4. Back-patch the code change to all supported versions.
Revert "Fix typo in documentation of toast storage"
commit : bd567b6a4147531267a9d4f4edcc5c985f06e785 author : Michael Paquier <email@example.com> date : Mon, 26 Nov 2018 16:43:19 +0900 committer: Michael Paquier <firstname.lastname@example.org> date : Mon, 26 Nov 2018 16:43:19 +0900
This reverts commit 058ef3a, per complains from Magnus Hagander and Vik Fearing.
Fix typo in documentation of toast storage
commit : 0d5e2dd001e5958d6971376e4d8af58dadf1feda author : Michael Paquier <email@example.com> date : Mon, 26 Nov 2018 15:49:23 +0900 committer: Michael Paquier <firstname.lastname@example.org> date : Mon, 26 Nov 2018 15:49:23 +0900
Author: Nawaz Ahmed Discussion: https://email@example.com
Fix hstore hash function for empty hstores upgraded from 8.4.
commit : 239abfff12a41a0611d24e2c7b67895a7359e6bf author : Andrew Gierth <firstname.lastname@example.org> date : Sat, 24 Nov 2018 09:59:49 +0000 committer: Andrew Gierth <email@example.com> date : Sat, 24 Nov 2018 09:59:49 +0000
Hstore data generated on pg 8.4 and pg_upgraded to current versions remains in its original on-disk format unless modified. The same goes for values generated by the addon hstore-new module on pre-9.0 versions. (The hstoreUpgrade function converts old values on the fly when read in, but the on-disk value is not modified by this.) Since old-format empty hstores (and hstore-new hstores) have representations compatible with the new format, hstoreUpgrade thought it could get away without modifying such values; but this breaks hstore_hash (and the new hstore_hash_extended) which assumes bit-perfect matching between semantically identical hstore values. Only one bit actually differs (the "new version" flag in the count field) but that of course is enough to break the hash. Fix by making hstoreUpgrade unconditionally convert all old values to new format. Backpatch all the way, even though this changes a hash value in some cases, because in those cases the hash value is already failing - for example, a hash join between old- and new-format empty hstores will be failing to match, or a hash index on an hstore column containing an old-format empty value will be failing to find the value since it will be searching for a hash derived from a new-format datum. (There are no known field reports of this happening, probably because hashing of hstores has only been useful in limited circumstances and there probably isn't much upgraded data being used this way.) Per concerns arising from discussion of commit eb6f29141be. Original bug is my fault. Discussion: https://postgr.es/m/60b1fd3b-7332-40f0-7e7f-f2f04f777747%402ndquadrant.com
Update additional float4/8 expected-output files.
commit : 1f99d0867020377defa251cc61943f3d4239e9eb author : Tom Lane <firstname.lastname@example.org> date : Sat, 24 Nov 2018 13:53:12 -0500 committer: Tom Lane <email@example.com> date : Sat, 24 Nov 2018 13:53:12 -0500
I forgot that the back branches have more variant files than HEAD :-(. Per buildfarm. Discussion: https://firstname.lastname@example.org
Fix float-to-integer coercions to handle edge cases correctly.
commit : 93eec1238663beab8521605f08a5937dbd8ec114 author : Tom Lane <email@example.com> date : Sat, 24 Nov 2018 12:45:50 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Sat, 24 Nov 2018 12:45:50 -0500
ftoi4 and its sibling coercion functions did their overflow checks in a way that looked superficially plausible, but actually depended on an assumption that the MIN and MAX comparison constants can be represented exactly in the float4 or float8 domain. That fails in ftoi4, ftoi8, and dtoi8, resulting in a possibility that values near the MAX limit will be wrongly converted (to negative values) when they need to be rejected. Also, because we compared before rounding off the fractional part, the other three functions threw errors for values that really ought to get rounded to the min or max integer value. Fix by doing rint() first (requiring an assumption that it handles NaN and Inf correctly; but dtoi8 and ftoi8 were assuming that already), and by comparing to values that should coerce to float exactly, namely INTxx_MIN and -INTxx_MIN. Also remove some random cosmetic discrepancies between these six functions. This back-patches commits cbdb8b4c0 and 452b637d4. In the 9.4 branch, also back-patch the portion of 62e2a8dc2 that added PG_INTnn_MIN and related constants to c.h, so that these functions can rely on them. Per bug #15519 from Victor Petrovykh. Patch by me; thanks to Andrew Gierth for analysis and discussion. Discussion: https://email@example.com
Avoid crashes in contrib/intarray gist__int_ops (bug #15518)
commit : 5f11a500fac69da4c3de61bf878d66f73f4fc8d3 author : Andrew Gierth <firstname.lastname@example.org> date : Fri, 23 Nov 2018 23:56:39 +0000 committer: Andrew Gierth <email@example.com> date : Fri, 23 Nov 2018 23:56:39 +0000
1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://firstname.lastname@example.org
doc: adjust time zone names text, v2
commit : 49c57f81828a86833dc6375d8ba30dfb66c73dfb author : Bruce Momjian <email@example.com> date : Wed, 21 Nov 2018 17:20:15 -0500 committer: Bruce Momjian <firstname.lastname@example.org> date : Wed, 21 Nov 2018 17:20:15 -0500
Removed one too many words. Fix for 7906de847f229f391b9e6b5892b4b4a89f29edb4. Reported-by: Thomas Munro Backpatch-through: 9.4
doc: adjust time zone names text
commit : 16e33ae5d67011974e4f7046aeecfd04fde29b46 author : Bruce Momjian <email@example.com> date : Wed, 21 Nov 2018 16:55:39 -0500 committer: Bruce Momjian <firstname.lastname@example.org> date : Wed, 21 Nov 2018 16:55:39 -0500
Reported-by: Kevin <email@example.com> Discussion: https://firstname.lastname@example.org Backpatch-through: 9.4
doc: Clarify CREATE TYPE ENUM documentation
commit : 390301ce630c9bbececef169130694acc06fafe2 author : Peter Eisentraut <email@example.com> date : Tue, 13 Nov 2018 10:42:43 +0100 committer: Peter Eisentraut <firstname.lastname@example.org> date : Tue, 13 Nov 2018 10:42:43 +0100
The documentation claimed that an enum type requires "one or more" labels, but since 1fd9883ff49, zero labels are also allowed. Reported-by: Lukas Eder <email@example.com> Bug: #15356
Fix configure's AC_CHECK_DECLS tests to work correctly with clang.
commit : 8af1511e9cb5fbcfe47cb081ff9a1800fee73348 author : Tom Lane <firstname.lastname@example.org> date : Mon, 19 Nov 2018 12:01:47 -0500 committer: Tom Lane <email@example.com> date : Mon, 19 Nov 2018 12:01:47 -0500
The test case that Autoconf uses to discover whether a function has been declared doesn't work reliably with clang, because clang reports a warning not an error if the name is a known built-in function. On some platforms, this results in a lot of compile-time warnings about strlcpy and related functions not having been declared. There is a fix for this (by Noah Misch) in the upstream Autoconf sources, but since they've not made a release in years and show no indication of doing so anytime soon, let's just absorb their fix directly. We can revert this when and if we update to a newer Autoconf release. Back-patch to all supported branches. Discussion: https://firstname.lastname@example.org
PANIC on fsync() failure.
commit : b9cce9ddfa1748c76d1d35c62aa35fe70ddc2dbe author : Thomas Munro <email@example.com> date : Mon, 19 Nov 2018 13:40:57 +1300 committer: Thomas Munro <firstname.lastname@example.org> date : Mon, 19 Nov 2018 13:40:57 +1300
On some operating systems, it doesn't make sense to retry fsync(), because dirty data cached by the kernel may have been dropped on write-back failure. In that case the only remaining copy of the data is in the WAL. A subsequent fsync() could appear to succeed, but not have flushed the data. That means that a future checkpoint could apparently complete successfully but have lost data. Therefore, violently prevent any future checkpoint attempts by panicking on the first fsync() failure. Note that we already did the same for WAL data; this change extends that behavior to non-temporary data files. Provide a GUC data_sync_retry to control this new behavior, for users of operating systems that don't eject dirty data, and possibly forensic/testing uses. If it is set to on and the write-back error was transient, a later checkpoint might genuinely succeed (on a system that does not throw away buffers on failure); if the error is permanent, later checkpoints will continue to fail. The GUC defaults to off, meaning that we panic. Back-patch to all supported releases. There is still a narrow window for error-loss on some operating systems: if the file is closed and later reopened and a write-back error occurs in the intervening time, but the inode has the bad luck to be evicted due to memory pressure before we reopen, we could miss the error. A later patch will address that with a scheme for keeping files with dirty data open at all times, but we judge that to be too complicated to back-patch. Author: Craig Ringer, with some adjustments by Thomas Munro Reported-by: Craig Ringer Reviewed-by: Robert Haas, Thomas Munro, Andres Freund Discussion: https://postgr.es/m/20180427222842.in2e4mibx45zdth5%40alap3.anarazel.de
Don't forget about failed fsync() requests.
commit : a4d6d6a25c12ca9a19a5e7ec6f76115bced69521 author : Thomas Munro <email@example.com> date : Mon, 19 Nov 2018 13:40:50 +1300 committer: Thomas Munro <firstname.lastname@example.org> date : Mon, 19 Nov 2018 13:40:50 +1300
If fsync() fails, md.c must keep the request in its bitmap, so that future attempts will try again. Back-patch to all supported releases. Author: Thomas Munro Reviewed-by: Amit Kapila Reported-by: Andrew Gierth Discussion: https://postgr.es/m/87y3i1ia4w.fsf%40news-spur.riddles.org.uk
Add valgrind suppressions for wcsrtombs optimizations
commit : 71b2951ccc433a6dd799efedff9f5927c32440f6 author : Tomas Vondra <email@example.com> date : Sat, 17 Nov 2018 23:50:21 +0100 committer: Tomas Vondra <firstname.lastname@example.org> date : Sat, 17 Nov 2018 23:50:21 +0100
wcsrtombs (called through wchar2char from common functions like lower, upper, etc.) uses various optimizations that may look like access to uninitialized data, triggering valgrind reports. For example AVX2 instructions load data in 256-bit chunks, and gconv does something similar with 32-bit chunks. This is faster than accessing the bytes one by one, and the uninitialized part of the buffer is not actually used. So suppress the bogus reports. The exact stack depends on possible optimizations - it might be AVX, SSE (as in the report by Aleksander Alekseev) or something else. Hence the last frame is wildcarded, to deal with this. Backpatch all the way back to 9.4. Author: Tomas Vondra Discussion: https://www.postgresql.org/message-id/flat/90ac0452-e907-e7a4-b3c8-15bd33780e62%402ndquadrant.com Discussion: https://www.postgresql.org/message-id/20180220150838.GD18315@e733.localdomain
Doc: remove claim that all \pset format options are unique in 1 letter.
commit : 033d45a1008f6846ae2380568abaee22da6ca19a author : Tom Lane <email@example.com> date : Wed, 14 Nov 2018 16:29:57 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Wed, 14 Nov 2018 16:29:57 -0500
This hasn't been correct since 9.3 added "latex-longtable". I left the phraseology "Unique abbreviations are allowed" alone. It's correct as far as it goes, and we are studiously refraining from specifying exactly what happens if you give a non-unique abbreviation. (The answer in the back branches is "you get a backwards-compatible choice", and the answer in HEAD will shortly be "you get an error", but there seems no need to mention such details here.) Daniel Vérité Discussion: https://email@example.com
Second try at fixing numeric data passed through an ECPG SQLDA.
commit : f9e25ba140776c01ee4bfdffc611b24a430e0872 author : Tom Lane <firstname.lastname@example.org> date : Wed, 14 Nov 2018 11:27:31 -0500 committer: Tom Lane <email@example.com> date : Wed, 14 Nov 2018 11:27:31 -0500
In commit ecfd55795, I removed sqlda.c's checks for ndigits != 0 on the grounds that we should duplicate the state of the numeric value's digit buffer even when all the digits are zeroes. However, that still isn't quite right, because another possible state of the digit buffer is buf == digits == NULL (this occurs for a NaN). As the code now stands, it'll invoke memcpy with a NULL source address and zero bytecount, which we know a few platforms crash on. Hence, reinstate the no-copy short-circuit, but make it test specifically for buf != NULL rather than some other condition. In hindsight, the ndigits test (added by commit f2ae9f9c3) was almost certainly meant to fix the NaN case not the all-zeroes case as the associated thread alleged. As before, back-patch to all supported versions. Discussion: https://postgr.es/m/1803D792815FC24D871C00D17AE95905C71161@g01jpexmbkw24
Initialize TransactionState and user ID consistently at transaction start
commit : 0d472b1e12efc3b860705cdc629a70d8972f0400 author : Michael Paquier <firstname.lastname@example.org> date : Wed, 14 Nov 2018 16:48:17 +0900 committer: Michael Paquier <email@example.com> date : Wed, 14 Nov 2018 16:48:17 +0900
If a failure happens when a transaction is starting between the moment the transaction status is changed from TRANS_DEFAULT to TRANS_START and the moment the current user ID and security context flags are fetched via GetUserIdAndSecContext(), or before initializing its basic fields, then those may get reset to incorrect values when the transaction aborts, leaving the session in an inconsistent state. One problem reported is that failing a starting transaction at the first query of a session could cause several kinds of system crashes on the follow-up queries. In order to solve that, move the initialization of the transaction state fields and the call of GetUserIdAndSecContext() in charge of fetching the current user ID close to the point where the transaction status is switched to TRANS_START, where there cannot be any error triggered in-between, per an idea of Tom Lane. This properly ensures that the current user ID, the security context flags and that the basic fields of TransactionState remain consistent even if the transaction fails while starting. Reported-by: Richard Guo Diagnosed-By: Richard Guo Author: Michael Paquier Reviewed-by: Tom Lane Discussion: https://postgr.es/m/CAN_9JTxECSb=pEPcb0a8d+6J+bDcOZ4=DgRo_B7Y5gRHJUM=Rw@mail.gmail.com Backpatch-through: 9.4
Fix incorrect results for numeric data passed through an ECPG SQLDA.
commit : e1f25901268cd07095329ce2e705df1de244f913 author : Tom Lane <firstname.lastname@example.org> date : Tue, 13 Nov 2018 15:46:08 -0500 committer: Tom Lane <email@example.com> date : Tue, 13 Nov 2018 15:46:08 -0500
Numeric values with leading zeroes were incorrectly copied into a SQLDA (SQL Descriptor Area), leading to wrong results in ECPG programs. Report and patch by Daisuke Higuchi. Back-patch to all supported versions. Discussion: https://postgr.es/m/1803D792815FC24D871C00D17AE95905C71161@g01jpexmbkw24
Fix the initialization of atomic variable introduced by the group clearing mechanism.
commit : 1a8bde490c982cbdd9d174e72d42f6042e9bf4ee author : Amit Kapila <firstname.lastname@example.org> date : Tue, 13 Nov 2018 10:43:35 +0530 committer: Amit Kapila <email@example.com> date : Tue, 13 Nov 2018 10:43:35 +0530
Commit 0e141c0fbb introduced initialization of atomic variable in InitProcess which means that it's not safe to look at it for backends that aren't currently in use. Fix that by initializing the same during postmaster startup. Reported-by: Andres Freund Author: Amit Kapila Backpatch-through: 9.6 Discussion: https://firstname.lastname@example.org
Limit the number of index clauses considered in choose_bitmap_and().
commit : fe66fc6f94221c5d884b63902b8d48ad87aba5e6 author : Tom Lane <email@example.com> date : Mon, 12 Nov 2018 11:19:04 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Mon, 12 Nov 2018 11:19:04 -0500
classify_index_clause_usage() is O(N^2) in the number of distinct index qual clauses it considers, because of its use of a simple search list to store them. For nearly all queries, that's fine because only a few clauses will be considered. But Alexander Kuzmenkov reported a machine-generated query with 80000 (!) index qual clauses, which caused this code to take forever. Somewhat remarkably, this is the only O(N^2) behavior we now have for such a query, so let's fix it. We can get rid of the O(N^2) runtime for cases like this without much damage to the functionality of choose_bitmap_and() by separating out paths with "too many" qual or pred clauses, and deeming them to always be nonredundant with other paths. Then their clauses needn't go into the search list, so it doesn't get too long, but we don't lose the ability to consider bitmap AND plans altogether. I set the threshold for "too many" to be 100 clauses per path, which should be plenty to ensure no change in planning behavior for normal queries. There are other things we could do to make this go faster, but it's not clear that it's worth any additional effort. 80000 qual clauses require a whole lot of work in many other places, too. The code's been like this for a long time, so back-patch to all supported branches. The troublesome query only works back to 9.5 (in 9.4 it fails with stack overflow in the parser); so I'm not sure that fixing this in 9.4 has any real-world benefit, but perhaps it does. Discussion: https://email@example.com
Fix incorrect author name in release notes
commit : 6b41ccb6900b299a437906e23e8e9636834f98f7 author : Michael Paquier <firstname.lastname@example.org> date : Mon, 12 Nov 2018 23:01:04 +0900 committer: Michael Paquier <email@example.com> date : Mon, 12 Nov 2018 23:01:04 +0900
Author: Alexander Lakhin Discussion: https://firstname.lastname@example.org
Fix missing role dependencies for some schema and type ACLs.
commit : d431dff1af8c220490b84dd978aa3a508f71d415 author : Tom Lane <email@example.com> date : Fri, 9 Nov 2018 20:42:03 -0500 committer: Tom Lane <firstname.lastname@example.org> date : Fri, 9 Nov 2018 20:42:03 -0500
This patch fixes several related cases in which pg_shdepend entries were never made, or were lost, for references to roles appearing in the ACLs of schemas and/or types. While that did no immediate harm, if a referenced role were later dropped, the drop would be allowed and would leave a dangling reference in the object's ACL. That still wasn't a big problem for normal database usage, but it would cause obscure failures in subsequent dump/reload or pg_upgrade attempts, taking the form of attempts to grant privileges to all-numeric role names. (I think I've seen field reports matching that symptom, but can't find any right now.) Several cases are fixed here: 1. ALTER DOMAIN SET/DROP DEFAULT would lose the dependencies for any existing ACL entries for the domain. This case is ancient, dating back as far as we've had pg_shdepend tracking at all. 2. If a default type privilege applies, CREATE TYPE recorded the ACL properly but forgot to install dependency entries for it. This dates to the addition of default privileges for types in 9.2. 3. If a default schema privilege applies, CREATE SCHEMA recorded the ACL properly but forgot to install dependency entries for it. This dates to the addition of default privileges for schemas in v10 (commit ab89e465c). Another somewhat-related problem is that when creating a relation rowtype or implicit array type, TypeCreate would apply any available default type privileges to that type, which we don't really want since such an object isn't supposed to have privileges of its own. (You can't, for example, drop such privileges once they've been added to an array type.) ab89e465c is also to blame for a race condition in the regression tests: privileges.sql transiently installed globally-applicable default privileges on schemas, which sometimes got absorbed into the ACLs of schemas created by concurrent test scripts. This should have resulted in failures when privileges.sql tried to drop the role holding such privileges; but thanks to the bug fixed here, it instead led to dangling ACLs in the final state of the regression database. We'd managed not to notice that, but it became obvious in the wake of commit da906766c, which allowed the race condition to occur in pg_upgrade tests. To fix, add a function recordDependencyOnNewAcl to encapsulate what callers of get_user_default_acl need to do; while the original call sites got that right via ad-hoc code, none of the later-added ones have. Also change GenerateTypeDependencies to generate these dependencies, which requires adding the typacl to its parameter list. (That might be annoying if there are any extensions calling that function directly; but if there are, they're most likely buggy in the same way as the core callers were, so they need work anyway.) While I was at it, I changed GenerateTypeDependencies to accept most of its parameters in the form of a Form_pg_type pointer, making its parameter list a bit less unwieldy and mistake-prone. The test race condition is fixed just by wrapping the addition and removal of default privileges into a single transaction, so that that state is never visible externally. We might eventually prefer to separate out tests of default privileges into a script that runs by itself, but that would be a bigger change and would make the tests run slower overall. Back-patch relevant parts to all supported branches. Discussion: https://email@example.com
Disallow setting client_min_messages higher than ERROR.
commit : 041ad9a66d28c3106a8bba79d94af447efc60347 author : Tom Lane <firstname.lastname@example.org> date : Thu, 8 Nov 2018 17:33:26 -0500 committer: Tom Lane <email@example.com> date : Thu, 8 Nov 2018 17:33:26 -0500
Previously it was possible to set client_min_messages to FATAL or PANIC, which had the effect of suppressing transmission of regular ERROR messages to the client. Perhaps that seemed like a useful option in the past, but the trouble with it is that it breaks guarantees that are explicitly made in our FE/BE protocol spec about how a query cycle can end. While libpq and psql manage to cope with the omission, that's mostly because they are not very bright; client libraries that have more semantic knowledge are likely to get confused. Notably, pgODBC doesn't behave very sanely. Let's fix this by getting rid of the ability to set client_min_messages above ERROR. In HEAD, just remove the FATAL and PANIC options from the set of allowed enum values for client_min_messages. (This change also affects trace_recovery_messages, but that's OK since these aren't useful values for that variable either.) In the back branches, there was concern that rejecting these values might break applications that are explicitly setting things that way. I'm pretty skeptical of that argument, but accommodate it by accepting these values and then internally setting the variable to ERROR anyway. In all branches, this allows a couple of tiny simplifications in the logic in elog.c, so do that. Also respond to the point that was made that client_min_messages has exactly nothing to do with the server's logging behavior, and therefore does not belong in the "When To Log" subsection of the documentation. The "Statement Behavior" subsection is a better match, so move it there. Jonah Harris and Tom Lane Discussion: https://firstname.lastname@example.org Discussion: https://email@example.com
GUC: adjust effective_cache_size SQL descriptions
commit : 6fc2738edd9f40fce099ea1be3f592caef4ac6c5 author : Bruce Momjian <firstname.lastname@example.org> date : Tue, 6 Nov 2018 13:40:02 -0500 committer: Bruce Momjian <email@example.com> date : Tue, 6 Nov 2018 13:40:02 -0500
Follow on patch for commit 3e0f1a4741f564c1a2fa6e944729d6967355d8c7. Reported-by: Peter Eisentraut Discussion: https://firstname.lastname@example.org Backpatch-through: 9.4