Stamp 9.6.9.
commit : 2f895f784fb31bf008162e9c52cc0b2c717823e8
author : Tom Lane <[email protected]>
date : Mon, 7 May 2018 16:53:38 -0400
committer: Tom Lane <[email protected]>
date : Mon, 7 May 2018 16:53:38 -0400
M configure
M configure.in
M doc/bug.template
M src/include/pg_config.h.win32
M src/interfaces/libpq/libpq.rc.in
M src/port/win32ver.rc
Last-minute updates for release notes.
commit : e71d9f6951fa61f386bebe3c97735a80def8a1e2
author : Tom Lane <[email protected]>
date : Mon, 7 May 2018 13:13:27 -0400
committer: Tom Lane <[email protected]>
date : Mon, 7 May 2018 13:13:27 -0400
The set of functions that need parallel-safety adjustments isn't the
same in 9.6 as 10, so I shouldn't have blindly back-patched that list.
Adjust as needed. Also, provide examples of the commands to issue.
M doc/src/sgml/release-9.3.sgml
M doc/src/sgml/release-9.4.sgml
M doc/src/sgml/release-9.5.sgml
M doc/src/sgml/release-9.6.sgml
Translation updates
commit : 2a771988f4daf6495ab63e9cb69c9d3a84eca1f0
author : Peter Eisentraut <[email protected]>
date : Mon, 7 May 2018 11:52:49 -0400
committer: Peter Eisentraut <[email protected]>
date : Mon, 7 May 2018 11:52:49 -0400
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: ffe3ce875edb7ff07db5351ed1640897fe9eb92d
M src/backend/po/de.po
M src/backend/po/fr.po
M src/backend/po/ru.po
M src/bin/pg_basebackup/po/de.po
M src/bin/pg_basebackup/po/ru.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/ru.po
M src/bin/scripts/po/de.po
M src/bin/scripts/po/ru.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/libpq/po/ru.po
M src/pl/plpython/po/ru.po
Last-minute updates for release notes.
commit : 1950d5b7a2aa6570dfc170f2fef1e921271d0024
author : Tom Lane <[email protected]>
date : Mon, 7 May 2018 11:50:05 -0400
committer: Tom Lane <[email protected]>
date : Mon, 7 May 2018 11:50:05 -0400
Security: CVE-2018-1115
M doc/src/sgml/release-9.6.sgml
adminpack: Revoke EXECUTE on pg_logfile_rotate()
commit : 53b79ab4fe722b1030b7e9a1580b283fca956e30
author : Stephen Frost <[email protected]>
date : Mon, 7 May 2018 10:10:45 -0400
committer: Stephen Frost <[email protected]>
date : Mon, 7 May 2018 10:10:45 -0400
In 9.6, we moved a number of functions over to using the GRANT system to
control access instead of having hard-coded superuser checks.
As it turns out, adminpack was creating another function in the catalog
for one of those backend functions where the superuser check was
removed, specifically pg_rotate_logfile(), but it didn't get the memo
about having to REVOKE EXECUTE on the alternative-name function
(pg_logfile_rotate()), meaning that in any installations with adminpack
on 9.6 and higher, any user is able to run the pg_logfile_rotate()
function, which then calls pg_rotate_logfile() and rotates the logfile.
Fix by adding a new version of adminpack (1.1) which handles the REVOKE.
As this function should have only been available to the superuser, this
is a security issue, albeit a minor one.
Security: CVE-2018-1115
M contrib/adminpack/Makefile
A contrib/adminpack/adminpack–1.0–1.1.sql
A contrib/adminpack/adminpack–1.1.sql
M contrib/adminpack/adminpack.control
Release notes for 10.4, 9.6.9, 9.5.13, 9.4.18, 9.3.23.
commit : 5c4049472db9088e9325e94fcac6d994ea3f0089
author : Tom Lane <[email protected]>
date : Sun, 6 May 2018 15:30:44 -0400
committer: Tom Lane <[email protected]>
date : Sun, 6 May 2018 15:30:44 -0400
M doc/src/sgml/release-9.3.sgml
M doc/src/sgml/release-9.4.sgml
M doc/src/sgml/release-9.5.sgml
M doc/src/sgml/release-9.6.sgml
Clear severity 5 perlcritic warnings from vcregress.pl
commit : 289bafdbc469fd0ed2a4485eca22005dbe668db5
author : Andrew Dunstan <[email protected]>
date : Sun, 6 May 2018 07:37:05 -0400
committer: Andrew Dunstan <[email protected]>
date : Sun, 6 May 2018 07:37:05 -0400
My recent update for python3 support used some idioms that are
unapproved. This fixes them. Backpatch to all live branches like the
original.
M src/tools/msvc/vcregress.pl
Tweak tests to support Python 3.7
commit : ab7825eadf75f6369efe98951bb3b93c06aec783
author : Peter Eisentraut <[email protected]>
date : Tue, 13 Feb 2018 16:13:20 -0500
committer: Peter Eisentraut <[email protected]>
date : Tue, 13 Feb 2018 16:13:20 -0500
Python 3.7 removes the trailing comma in the repr() of
BaseException (see <https://bugs.python.org/issue30399>), leading to
test output differences. Work around that by composing the equivalent
test output in a more manual way.
M src/pl/plpython/expected/plpython_subtransaction.out
M src/pl/plpython/expected/plpython_subtransaction_0.out
M src/pl/plpython/expected/plpython_subtransaction_5.out
M src/pl/plpython/sql/plpython_subtransaction.sql
Remove extra newlines after PQerrorMessage()
commit : 7ff50cf60b20fa8c333e427f5e61b59f913844fe
author : Peter Eisentraut <[email protected]>
date : Sat, 5 May 2018 10:51:38 -0400
committer: Peter Eisentraut <[email protected]>
date : Sat, 5 May 2018 10:51:38 -0400
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_dump/pg_dumpall.c
Fix scenario where streaming standby gets stuck at a continuation record.
commit : 7b7521d6577295413db2088beac6136d39d61e4e
author : Heikki Linnakangas <[email protected]>
date : Sat, 5 May 2018 01:34:53 +0300
committer: Heikki Linnakangas <[email protected]>
date : Sat, 5 May 2018 01:34:53 +0300
If a continuation record is split so that its first half has already been
removed from the master, and is only present in pg_wal, and there is a
recycled WAL segment in the standby server that looks like it would
contain the second half, recovery would get stuck. The code in
XLogPageRead() incorrectly started streaming at the beginning of the
WAL record, even if we had already read the first page.
Backpatch to 9.4. In principle, older versions have the same problem, but
without replication slots, there was no straightforward mechanism to
prevent the master from recycling old WAL that was still needed by standby.
Without such a mechanism, I think it's reasonable to assume that there's
enough slack in how many old segments are kept around to not run into this,
or you have a WAL archive.
Reported by Jonathon Nelson. Analysis and patch by Kyotaro HORIGUCHI, with
some extra comments by me.
Discussion: https://www.postgresql.org/message-id/CACJqAM3xVz0JY1XFDKPP%2BJoJAjoGx%3DGNuOAshEDWCext7BFvCQ%40mail.gmail.com
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogreader.c
M src/include/access/xlogreader.h
Don't mark pages all-visible spuriously
commit : 3a11485a524ca23f404147c5736fbc2ef6b2eff2
author : Alvaro Herrera <[email protected]>
date : Fri, 4 May 2018 15:24:44 -0300
committer: Alvaro Herrera <[email protected]>
date : Fri, 4 May 2018 15:24:44 -0300
Dan Wood diagnosed a long-standing problem that pages containing tuples
that are locked by multixacts containing live lockers may spuriously end
up as candidates for getting their all-visible flag set. This has the
long-term effect that multixacts remain unfrozen; this may previously
pass undetected, but since commit XYZ it would be reported as
"ERROR: found multixact 134100944 from before relminmxid 192042633"
because when a later vacuum tries to freeze the page it detects that a
multixact that should have gotten frozen, wasn't.
Dan proposed a (correct) patch that simply sets a variable to its
correct value, after a bogus initialization. But, per discussion, it
seems better coding to avoid the bogus initializations altogether, since
they could give rise to more bugs later. Therefore this fix rewrites
the logic a little bit to avoid depending on the bogus initializations.
This bug was part of a family introduced in 9.6 by commit a892234f830e;
later, commit 38e9f90a227d fixed most of them, but this one was
unnoticed.
Authors: Dan Wood, Pavan Deolasee, Álvaro Herrera
Reviewed-by: Masahiko Sawada, Pavan Deolasee, Álvaro Herrera
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam.c
Provide for testing on python3 modules when under MSVC
commit : a9fbf550b1f1a09baa505153820d95ccd76b56b8
author : Andrew Dunstan <[email protected]>
date : Fri, 4 May 2018 15:22:48 -0400
committer: Andrew Dunstan <[email protected]>
date : Fri, 4 May 2018 15:22:48 -0400
This should have been done some years ago as promised in commit
c4dcdd0c2. However, better late than never.
Along the way do a little housekeeping, including using a simpler test
for the python version being tested, and removing a redundant subroutine
parameter. These changes only apply back to release 9.5.
Backpatch to all live releases.
M src/tools/msvc/Install.pm
M src/tools/msvc/vcregress.pl
Allow MSYS as well as MINGW in Msys uname
commit : 679b07469a17eb08271dfeec1f015f9239ce40bd
author : Andrew Dunstan <[email protected]>
date : Fri, 4 May 2018 14:54:04 -0400
committer: Andrew Dunstan <[email protected]>
date : Fri, 4 May 2018 14:54:04 -0400
Msys2's uname -s outputs a string beginning MSYS rather than MINGW as is
output by Msys. Allow either in pg_upgrade's test.sh.
Backpatch to all live branches.
M src/bin/pg_upgrade/test.sh
Sync our copy of the timezone library with IANA release tzcode2018e.
commit : 7a83323f2daaa0aa73e60b82ba48f651c6934bc5
author : Tom Lane <[email protected]>
date : Fri, 4 May 2018 12:26:25 -0400
committer: Tom Lane <[email protected]>
date : Fri, 4 May 2018 12:26:25 -0400
The non-cosmetic changes involve teaching the "zic" tzdata compiler about
negative DST. While I'm not currently intending that we start using
negative-DST data right away, it seems possible that somebody would try
to use our copy of zic with bleeding-edge IANA data. So we'd better be
out in front of this change code-wise, even though it doesn't matter for
the data file we're shipping.
Discussion: https://postgr.es/m/[email protected]
M src/timezone/README
M src/timezone/localtime.c
M src/timezone/strftime.c
M src/timezone/zic.c
Add HOLD_INTERRUPTS section into FinishPreparedTransaction.
commit : d9b3bc55201a185e6a9fb37f703592efde69c6e2
author : Teodor Sigaev <[email protected]>
date : Thu, 3 May 2018 20:09:28 +0300
committer: Teodor Sigaev <[email protected]>
date : Thu, 3 May 2018 20:09:28 +0300
If an interrupt arrives in the middle of FinishPreparedTransaction
and any callback decide to call CHECK_FOR_INTERRUPTS (e.g.
RemoveTwoPhaseFile can write a warning with ereport, which checks for
interrupts) then it's possible to leave current GXact undeleted.
Backpatch to all supported branches
Stas Kelvich
Discussion: ihttps://www.postgresql.org/message-id/[email protected]
M src/backend/access/transam/twophase.c
Revert back-branch changes in power()'s behavior for NaN inputs.
commit : eab8d6312f15b8efbab269fea66da8f5a1e7c65f
author : Tom Lane <[email protected]>
date : Wed, 2 May 2018 17:32:40 -0400
committer: Tom Lane <[email protected]>
date : Wed, 2 May 2018 17:32:40 -0400
Per discussion, the value of fixing these bugs in the back branches
doesn't outweigh the downsides of changing corner-case behavior in
a minor release. Hence, revert commits 217d8f3a1 and 4d864de48 in
the v10 branch and the corresponding commits in 9.3-9.6.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/float.c
M src/test/regress/expected/float8-exp-three-digits-win32.out
M src/test/regress/expected/float8-small-is-zero.out
M src/test/regress/expected/float8-small-is-zero_1.out
M src/test/regress/expected/float8.out
M src/test/regress/sql/float8.sql
Fix bogus list-iteration code in pg_regress.c, affecting ecpg tests only.
commit : 938c6f42d8524ccc8828aa5a1a46578e4226562d
author : Tom Lane <[email protected]>
date : Sun, 29 Apr 2018 21:56:28 -0400
committer: Tom Lane <[email protected]>
date : Sun, 29 Apr 2018 21:56:28 -0400
While looking at a recent buildfarm failure in the ecpg tests, I wondered
why the pg_regress output claimed the stderr part of the test failed, when
the regression diffs were clearly for the stdout part. Looking into it,
the reason is that pg_regress.c's logic for iterating over three parallel
lists is wrong, and has been wrong since it was written: it advances the
"tag" pointer at a different place in the loop than the other two pointers.
Fix that.
M src/test/regress/pg_regress.c
Avoid wrong results for power() with NaN input on more platforms.
commit : d6ec3d2375ac3b94d2480c01d7ee7739b236642c
author : Tom Lane <[email protected]>
date : Sun, 29 Apr 2018 18:15:16 -0400
committer: Tom Lane <[email protected]>
date : Sun, 29 Apr 2018 18:15:16 -0400
Buildfarm results show that the modern POSIX rule that 1 ^ NaN = 1 is not
honored on *BSD until relatively recently, and really old platforms don't
believe that NaN ^ 0 = 1 either. (This is unsurprising, perhaps, since
SUSv2 doesn't require either behavior.) In hopes of getting to platform
independent behavior, let's deal with all the NaN-input cases explicitly
in dpow().
Note that numeric_power() doesn't know either of these special cases.
But since that behavior is platform-independent, I think it should be
addressed separately, and probably not back-patched.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/float.c
M src/test/regress/expected/float8-exp-three-digits-win32.out
M src/test/regress/expected/float8-small-is-zero.out
M src/test/regress/expected/float8-small-is-zero_1.out
M src/test/regress/expected/float8.out
M src/test/regress/sql/float8.sql
Update time zone data files to tzdata release 2018d.
commit : 2acbeea48c38b1574b5f59f4e2e05fbad2e45ec5
author : Tom Lane <[email protected]>
date : Sun, 29 Apr 2018 15:50:08 -0400
committer: Tom Lane <[email protected]>
date : Sun, 29 Apr 2018 15:50:08 -0400
DST law changes in Palestine and Antarctica (Casey Station). Historical
corrections for Portugal and its colonies, as well as Enderbury, Jamaica,
Turks & Caicos Islands, and Uruguay.
M src/timezone/data/tzdata.zi
Avoid wrong results for power() with NaN input on some platforms.
commit : 48e0f8cbe0d9bbc08de6e97ccb6241a670731a97
author : Tom Lane <[email protected]>
date : Sun, 29 Apr 2018 15:21:45 -0400
committer: Tom Lane <[email protected]>
date : Sun, 29 Apr 2018 15:21:45 -0400
Per spec, the result of power() should be NaN if either input is NaN.
It appears that on some versions of Windows, the libc function does
return NaN, but it also sets errno = EDOM, confusing our code that
attempts to work around shortcomings of other platforms. Hence, add
guard tests to avoid substituting a wrong result for the right one.
It's been like this for a long time (and the odd behavior only appears
in older MSVC releases, too) so back-patch to all supported branches.
Dang Minh Huong, reviewed by David Rowley
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/float.c
M src/test/regress/expected/float8-exp-three-digits-win32.out
M src/test/regress/expected/float8-small-is-zero.out
M src/test/regress/expected/float8-small-is-zero_1.out
M src/test/regress/expected/float8.out
M src/test/regress/sql/float8.sql
docs: remove "III" version text from pgAdmin link
commit : 1d91af81cac75ae6328b0a248c2d478499e4c1d7
author : Bruce Momjian <[email protected]>
date : Thu, 26 Apr 2018 11:10:43 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 26 Apr 2018 11:10:43 -0400
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.3
M doc/src/sgml/external-projects.sgml
Correct pg_recvlogical server version test.
commit : 32c247629367c827b09e65d135d0a26fc91eb8fa
author : Noah Misch <[email protected]>
date : Wed, 25 Apr 2018 18:50:29 -0700
committer: Noah Misch <[email protected]>
date : Wed, 25 Apr 2018 18:50:29 -0700
The predecessor test boiled down to "PQserverVersion(NULL) >= 100000",
which is always false. No release includes that, so it could not have
reintroduced CVE-2018-1058. Back-patch to 9.4, like the addition of the
predecessor in commit 8d2814f274def85f39fbe997d454b01628cb5667.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_basebackup/streamutil.c
Fix race conditions when an event trigger is added concurrently with DDL.
commit : c76d0eed270ee6d160d6e37feb814bf8b53189a2
author : Tom Lane <[email protected]>
date : Fri, 20 Apr 2018 17:15:31 -0400
committer: Tom Lane <[email protected]>
date : Fri, 20 Apr 2018 17:15:31 -0400
EventTriggerTableRewrite crashed if there were table_rewrite triggers
present, but there had not been when the calling command started.
EventTriggerDDLCommandEnd called ddl_command_end triggers if present,
even if there had been no such triggers when the calling command started,
which would lead to a failure in pg_event_trigger_ddl_commands.
In both cases, fix by doing nothing; it's better to wait till the next
command when things will be properly initialized.
In passing, remove an elog(DEBUG1) call that might have seemed interesting
four years ago but surely isn't today.
We found this because of intermittent failures in the buildfarm. Thanks
to Alvaro Herrera and Andrew Gierth for analysis.
Back-patch to 9.5; some of this code exists before that, but the specific
hazards we need to guard against don't.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/event_trigger.c
Change more places to be less trusting of RestrictInfo.is_pushed_down.
commit : 64ad85860ce6b3a7ce01a392f90a322bf61d068f
author : Tom Lane <[email protected]>
date : Fri, 20 Apr 2018 15:19:17 -0400
committer: Tom Lane <[email protected]>
date : Fri, 20 Apr 2018 15:19:17 -0400
On further reflection, commit e5d83995e didn't go far enough: pretty much
everywhere in the planner that examines a clause's is_pushed_down flag
ought to be changed to use the more complicated behavior where we also
check the clause's required_relids. Otherwise we could make incorrect
decisions about whether, say, a clause is safe to use as a hash clause.
Some (many?) of these places are safe as-is, either because they are
never reached while considering a parameterized path, or because there
are additional checks that would reject a pushed-down clause anyway.
However, it seems smarter to just code them all the same way rather
than rely on easily-broken reasoning of that sort.
In support of that, invent a new macro RINFO_IS_PUSHED_DOWN that should
be used in place of direct tests on the is_pushed_down flag.
Like the previous patch, back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/joinpath.c
M src/backend/optimizer/path/joinrels.c
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/util/restrictinfo.c
M src/include/nodes/relation.h
Fix broken extract_actual_join_clauses call in 9.6 postgres_fdw.
commit : 306d6e59f7511fef3e05457b15ecad0fc00b5e1e
author : Tom Lane <[email protected]>
date : Thu, 19 Apr 2018 18:29:39 -0400
committer: Tom Lane <[email protected]>
date : Thu, 19 Apr 2018 18:29:39 -0400
In commits e5d83995e et al, I changed the signature of
extract_actual_join_clauses, thinking that it was not called from
anywhere but createplan.c. I missed that postgres_fdw uses it
in the 9.6 branch only.
This opens up the question of whether any third-party modules might
be calling it, and whether we need to take steps to avoid an API break
for them. But for the moment, just get the buildfarm green again.
M contrib/postgres_fdw/postgres_fdw.c
Fix incorrect handling of join clauses pushed into parameterized paths.
commit : 0c141fcaa7dd806752986401b25de8f665ceb3fe
author : Tom Lane <[email protected]>
date : Thu, 19 Apr 2018 15:49:12 -0400
committer: Tom Lane <[email protected]>
date : Thu, 19 Apr 2018 15:49:12 -0400
In some cases a clause attached to an outer join can be pushed down into
the outer join's RHS even though the clause is not degenerate --- this
can happen if we choose to make a parameterized path for the RHS. If
the clause ends up attached to a lower outer join, we'd misclassify it
as being a "join filter" not a plain "filter" condition at that node,
leading to wrong query results.
To fix, teach extract_actual_join_clauses to examine each join clause's
required_relids, not just its is_pushed_down flag. (The latter now
seems vestigial, or at least in need of rethinking, but we won't do
anything so invasive as redefining it in a bug-fix patch.)
This has been wrong since we introduced parameterized paths in 9.2,
though it's evidently hard to hit given the lack of previous reports.
The test case used here involves a lateral function call, and I think
that a lateral reference may be required to get the planner to select
a broken plan; though I wouldn't swear to that. In any case, even if
LATERAL is needed to trigger the bug, it still affects all supported
branches, so back-patch to all.
Per report from Andreas Karlsson. Thanks to Andrew Gierth for
preliminary investigation.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/restrictinfo.c
M src/include/optimizer/restrictinfo.h
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Enlarge find_other_exec's meager fgets buffer
commit : b3f4742aab82463166b327dbf26b3d4a5039e1c6
author : Alvaro Herrera <[email protected]>
date : Thu, 19 Apr 2018 10:45:15 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 19 Apr 2018 10:45:15 -0300
The buffer was 100 bytes long, which is barely sufficient when the
version string gets longer (such as by configure --with-extra-version).
Set it to MAXPGPATH.
Author: Nikhil Sontakke
Discussion: https://postgr.es/m/CAMGcDxfLfpYU_Jru++L6ARPCOyxr0W+2O3Q54TDi5XdYeU36ow@mail.gmail.com
M src/common/exec.c
Better fix for deadlock hazard in CREATE INDEX CONCURRENTLY.
commit : 69e3a548e99f81babf28140526670dded7f94dcf
author : Tom Lane <[email protected]>
date : Wed, 18 Apr 2018 12:07:37 -0400
committer: Tom Lane <[email protected]>
date : Wed, 18 Apr 2018 12:07:37 -0400
Commit 54eff5311 did not account for the possibility that we'd have
a transaction snapshot due to default_transaction_isolation being
set high enough to require one. The transaction snapshot is enough
to hold back our advertised xmin and thus risk deadlock anyway.
The only way to get rid of that snap is to start a new transaction,
so let's do that instead. Also throw in an assert checking that we
really have gotten to a state where no xmin is being advertised.
Back-patch to 9.4, like the previous commit.
Discussion: https://postgr.es/m/CAMkU=1ztk3TpQdcUNbxq93pc80FrXUjpDWLGMeVBDx71GHNwZQ@mail.gmail.com
M src/backend/commands/indexcmds.c
Fix broken collation-aware searches in SP-GiST text opclass.
commit : d90b2904c7118ca088d14f488a00294a27da5fc4
author : Tom Lane <[email protected]>
date : Mon, 16 Apr 2018 16:06:47 -0400
committer: Tom Lane <[email protected]>
date : Mon, 16 Apr 2018 16:06:47 -0400
spg_text_leaf_consistent() supposed that it should compare only
Min(querylen, entrylen) bytes of the two strings, and then deal with
any excess bytes in one string or the other by assuming the longer
string is greater if the prefixes are equal. Quite aside from the
fact that that's just wrong in some locales (e.g., 'ch' is not less
than 'd' in cs_CZ), it also risked passing incomplete multibyte
characters to strcoll(), with ensuing bad results.
Instead, just pass the full strings to varstr_cmp, and let it decide
what to do about unequal-length strings.
Fortunately, this error doesn't imply any index corruption, it's just
that searches might return the wrong set of entries.
Per report from Emre Hasegeli, though this is not his patch.
Thanks to Peter Geoghegan for review and discussion.
This code was born broken, so back-patch to all supported branches.
In HEAD, I failed to resist the temptation to do a bit of cosmetic
cleanup/pgindent'ing on 710d90da1, too.
Discussion: https://postgr.es/m/CAE2gYzzb6K51VnTq5i5p52z+j9p2duEa-K1T3RrC_GQEynAKEg@mail.gmail.com
M src/backend/access/spgist/spgtextproc.c
Fix potentially-unportable code in contrib/adminpack.
commit : 65f2e868b262ff4480cb10d063ccd9b689a0a77f
author : Tom Lane <[email protected]>
date : Sun, 15 Apr 2018 13:02:11 -0400
committer: Tom Lane <[email protected]>
date : Sun, 15 Apr 2018 13:02:11 -0400
Spelling access(2)'s second argument as "2" is just horrid.
POSIX makes no promises as to the numeric values of W_OK and related
macros. Even if it accidentally works as intended on every supported
platform, it's still unreadable and inconsistent with adjacent code.
In passing, don't spell "NULL" as "0" either. Yes, that's legal C;
no, it's not project style.
Back-patch, just in case the unportability is real and not theoretical.
(Most likely, even if a platform had different bit assignments for
access()'s modes, there'd not be an observable behavior difference
here; but I'm being paranoid today.)
M contrib/adminpack/adminpack.c
In libpq, free any partial query result before collecting a server error.
commit : 131f6a9583600cc108891ff638e94633336d9f17
author : Tom Lane <[email protected]>
date : Fri, 13 Apr 2018 12:53:46 -0400
committer: Tom Lane <[email protected]>
date : Fri, 13 Apr 2018 12:53:46 -0400
We'd throw away the partial result anyway after parsing the error message.
Throwing it away beforehand costs nothing and reduces the risk of
out-of-memory failure. Also, at least in systems that behave like
glibc/Linux, if the partial result was very large then the error PGresult
would get allocated at high heap addresses, preventing the heap storage
used by the partial result from being released to the OS until the error
PGresult is freed.
In psql >= 9.6, we hold onto the error PGresult until another error is
received (for \errverbose), so that this behavior causes a seeming
memory leak to persist for awhile, as in a recent complaint from
Darafei Praliaskouski. This is a potential performance regression from
older versions, justifying back-patching at least that far. But similar
behavior may occur in other client applications, so it seems worth just
back-patching to all supported branches.
Discussion: https://postgr.es/m/CAC8Q8tJ=7cOkPePyAbJE_Pf691t8nDFhJp0KZxHvnq_uicfyVg@mail.gmail.com
M src/interfaces/libpq/fe-protocol2.c
M src/interfaces/libpq/fe-protocol3.c
Fix bogus affix-merging code.
commit : 0f439c8dd28935bf9600722c1a25534add9390f0
author : Tom Lane <[email protected]>
date : Thu, 12 Apr 2018 18:39:51 -0400
committer: Tom Lane <[email protected]>
date : Thu, 12 Apr 2018 18:39:51 -0400
NISortAffixes() compared successive compound affixes incorrectly,
thus possibly failing to merge identical affixes, or (less likely)
merging ones that shouldn't be merged. The user-visible effects
of this are unclear, to me anyway.
Per bug #15150 from Alexander Lakhin. It's been broken for a long time,
so back-patch to all supported branches.
Arthur Zakirov
Discussion: https://postgr.es/m/[email protected]
M src/backend/tsearch/spell.c
Ignore nextOid when replaying an ONLINE checkpoint.
commit : 060bb38d0750a04870b6e15fbb2a995a9dcd2b0a
author : Tom Lane <[email protected]>
date : Wed, 11 Apr 2018 18:11:30 -0400
committer: Tom Lane <[email protected]>
date : Wed, 11 Apr 2018 18:11:30 -0400
The nextOid value is from the start of the checkpoint and may well be stale
compared to values from more recent XLOG_NEXTOID records. Previously, we
adopted it anyway, allowing the OID counter to go backwards during a crash.
While this should be harmless, it contributed to the severity of the bug
fixed in commit 0408e1ed5, by allowing duplicate TOAST OIDs to be assigned
immediately following a crash. Without this error, that issue would only
have arisen when TOAST objects just younger than a multiple of 2^32 OIDs
were deleted and then not vacuumed in time to avoid a conflict.
Pavan Deolasee
Discussion: https://postgr.es/m/CABOikdOgWT2hHkYG3Wwo2cyZJq2zfs1FH0FgX-=h4OLosXHf9w@mail.gmail.com
M src/backend/access/transam/xlog.c
Do not select new object OIDs that match recently-dead entries.
commit : 8bba10f7e8349546924a4e8247346c0a48180e8f
author : Tom Lane <[email protected]>
date : Wed, 11 Apr 2018 17:41:10 -0400
committer: Tom Lane <[email protected]>
date : Wed, 11 Apr 2018 17:41:10 -0400
When selecting a new OID, we take care to avoid picking one that's already
in use in the target table, so as not to create duplicates after the OID
counter has wrapped around. However, up to now we used SnapshotDirty when
scanning for pre-existing entries. That ignores committed-dead rows, so
that we could select an OID matching a deleted-but-not-yet-vacuumed row.
While that mostly worked, it has two problems:
* If recently deleted, the dead row might still be visible to MVCC
snapshots, creating a risk for duplicate OIDs when examining the catalogs
within our own transaction. Such duplication couldn't be visible outside
the object-creating transaction, though, and we've heard few if any field
reports corresponding to such a symptom.
* When selecting a TOAST OID, deleted toast rows definitely *are* visible
to SnapshotToast, and will remain so until vacuumed away. This leads to
a conflict that will manifest in errors like "unexpected chunk number 0
(expected 1) for toast value nnnnn". We've been seeing reports of such
errors from the field for years, but the cause was unclear before.
The fix is simple: just use SnapshotAny to search for conflicting rows.
This results in a slightly longer window before object OIDs can be
recycled, but that seems unlikely to create any large problems.
Pavan Deolasee
Discussion: https://postgr.es/m/CABOikdOgWT2hHkYG3Wwo2cyZJq2zfs1FH0FgX-=h4OLosXHf9w@mail.gmail.com
M src/backend/access/heap/tuptoaster.c
M src/backend/catalog/catalog.c
Make local copy of client hostnames in backend status array.
commit : 74dc05e01e5a195e432c6bd31f6504eb24b8316b
author : Heikki Linnakangas <[email protected]>
date : Wed, 11 Apr 2018 23:39:48 +0300
committer: Heikki Linnakangas <[email protected]>
date : Wed, 11 Apr 2018 23:39:48 +0300
The other strings, application_name and query string, were snapshotted to
local memory in pgstat_read_current_status(), but we forgot to do that for
client hostnames. As a result, the client hostname would appear to change in
the local copy, if the client disconnected.
Backpatch to all supported versions.
Author: Edmund Horner
Reviewed-by: Michael Paquier
Discussion: https://www.postgresql.org/message-id/CAMyN-kA7aOJzBmrYFdXcc7Z0NmW%2B5jBaf_m%3D_-77uRNyKC9r%3DA%40mail.gmail.com
M src/backend/postmaster/pgstat.c
Fix incorrect close() call in dsm_impl_mmap().
commit : 494f3cb5bbdae3d75b0100559a8a05020e8338ff
author : Tom Lane <[email protected]>
date : Tue, 10 Apr 2018 18:34:40 -0400
committer: Tom Lane <[email protected]>
date : Tue, 10 Apr 2018 18:34:40 -0400
One improbable error-exit path in this function used close() where
it should have used CloseTransientFile(). This is unlikely to be
hit in the field, and I think the consequences wouldn't be awful
(just an elog(LOG) bleat later). But a bug is a bug, so back-patch
to 9.4 where this code came in.
Pan Bian
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/dsm_impl.c
Remove wrongly backpatched piece of code in cube.c
commit : 5b0fa06f65e959f73c145c8104cb3fa8505fca73
author : Teodor Sigaev <[email protected]>
date : Tue, 10 Apr 2018 14:59:27 +0300
committer: Teodor Sigaev <[email protected]>
date : Tue, 10 Apr 2018 14:59:27 +0300
Due to sloppy division of changes between f50c80dbb (which was not
back-patched) and 563a053bd, this piece of code was wrongly backpatched to
REL_10_STABLE and REL9_6_STABLE. This code never causes real error because
its condition is never satisfied, but it's a dead code, which needs to be
removed.
Alexander Korotkov per gripe from Tom Lane
M contrib/cube/cube.c
Doc: clarify explanation of pg_dump usage.
commit : 3d465826f2aa86fc42b0930a6e4942585ed4ed43
author : Tom Lane <[email protected]>
date : Sun, 8 Apr 2018 16:35:43 -0400
committer: Tom Lane <[email protected]>
date : Sun, 8 Apr 2018 16:35:43 -0400
This section confusingly used both "infile" and "outfile" to refer
to the same file, i.e. the textual output of pg_dump. Use "dumpfile"
for both cases, per suggestion from Jonathan Katz.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/backup.sgml
Remove overzeleous assertions in pg_atomic_flag code.
commit : 5faead30d83189536ab1cf3ea0f8940239cc9838
author : Andres Freund <[email protected]>
date : Sat, 7 Apr 2018 18:27:14 -0700
committer: Andres Freund <[email protected]>
date : Sat, 7 Apr 2018 18:27:14 -0700
The atomics code asserts proper alignment in various places. That's
mainly because the alignment of 64bit integers is not sufficient for
atomic operations on all platforms. Some ABIs only have four byte
alignment, but don't have atomic behavior when crossing page
boundaries.
The flags code isn't affected by that however, as the type alignment
always is sufficient for atomic operations. Nevertheless the code
asserted alignment requirements. Before 8c3debbb it was only broken on
hppa, after it probably affect further platforms.
Thus remove the assertions for pg_atomic_flag operators.
Per buildfarm animal pademelon.
Discussion: https://postgr.es/m/[email protected]
Backpatch: 9.5-
M src/include/port/atomics.h
Fix and improve pg_atomic_flag fallback implementation.
commit : 1f3bbe00b7b5fd4f2acd43db1c63d5a0328509c3
author : Andres Freund <[email protected]>
date : Fri, 6 Apr 2018 20:02:02 -0700
committer: Andres Freund <[email protected]>
date : Fri, 6 Apr 2018 20:02:02 -0700
The atomics fallback implementation for pg_atomic_flag was broken,
returning the inverted value from pg_atomic_test_set_flag(). This was
unnoticed because a) atomic flags were unused until recently b) the
test code wasn't run when the fallback implementation was in
use (because it didn't allow to test for some edge cases).
Fix the bug, and improve the fallback so it has the same behaviour as
the non-fallback implementation in the problematic edge cases. That
breaks ABI compatibility in the back branches when fallbacks are in
use, but given they were broken until now...
Author: Andres Freund
Reported-by: Daniel Gustafsson
Discussion:
https://postgr.es/m/[email protected]
https://postgr.es/m/[email protected]
Backpatch: 9.5-, where the atomics abstraction was introduced.
M src/backend/port/atomics.c
M src/include/port/atomics/fallback.h
M src/test/regress/regress.c
doc: remove mention of the DMOZ catalog in ltree docs
commit : e17745070867d073243be7e50f2e42465a533500
author : Bruce Momjian <[email protected]>
date : Thu, 5 Apr 2018 15:55:41 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 5 Apr 2018 15:55:41 -0400
Discussion: https://postgr.es/m/CAF4Au4xYem_W3KOuxcKct7=G4j8Z3uO9j3DUKTFJqUsfp_9pQg@mail.gmail.com
Author: Oleg Bartunov
Backpatch-through: 9.3
M doc/src/sgml/ltree.sgml
docs: update ltree URL for the DMOZ catalog
commit : 374204ce816d63a8d0a22696322042c015123f36
author : Bruce Momjian <[email protected]>
date : Wed, 4 Apr 2018 15:06:21 -0400
committer: Bruce Momjian <[email protected]>
date : Wed, 4 Apr 2018 15:06:21 -0400
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Author: Oleg Bartunov
Backpatch-through: 9.3
M doc/src/sgml/ltree.sgml
Also fix the descriptions in pg_config.h.win32.
commit : 0d2012c9f04afe375200d16d539a9ec5c0093c07
author : Heikki Linnakangas <[email protected]>
date : Wed, 4 Apr 2018 11:33:39 +0300
committer: Heikki Linnakangas <[email protected]>
date : Wed, 4 Apr 2018 11:33:39 +0300
I missed pg_config.h.win32 in the previous commit that fixed these in
pg_config.h.in.
M src/include/pg_config.h.win32
Fix incorrect description of USE_SLICING_BY_8_CRC32C.
commit : 7be511a767cf41f808ee77911bec0f341d04c5fc
author : Heikki Linnakangas <[email protected]>
date : Wed, 4 Apr 2018 11:20:53 +0300
committer: Heikki Linnakangas <[email protected]>
date : Wed, 4 Apr 2018 11:20:53 +0300
And a typo in the description of USE_SSE42_CRC32C_WITH_RUNTIME_CHECK,
spotted by Daniel Gustafsson.
M configure.in
M src/include/pg_config.h.in
doc: document "IS NOT DOCUMENT"
commit : 77b9c507d4cf94580ff1b58379a5769f11df08eb
author : Bruce Momjian <[email protected]>
date : Mon, 2 Apr 2018 16:41:46 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 2 Apr 2018 16:41:46 -0400
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Author: Euler Taveira
Backpatch-through: 9.3
M doc/src/sgml/func.sgml
Fix assorted issues in parallel vacuumdb.
commit : 6cd110c477b483cee7fc4b3ce0fdfb961dbe39b9
author : Tom Lane <[email protected]>
date : Sat, 31 Mar 2018 16:28:52 -0400
committer: Tom Lane <[email protected]>
date : Sat, 31 Mar 2018 16:28:52 -0400
Avoid storing the result of PQsocket() in a pgsocket variable; it's
declared as int, and the no-socket test is properly written as "x < 0"
not "x == PGINVALID_SOCKET". This accidentally had no bad effect
because we never got to init_slot() with a bad connection, but it's
still wrong.
Actually, it seems like we should avoid storing the result for a long
period at all. The function's not so expensive that it's worth avoiding,
and the existing coding technique here would fail if anyone tried to
PQreset the connection during the life of the program. Hence, just
re-call PQsocket every time we construct a select(2) mask.
Speaking of select(), GetIdleSlot imagined that it could compute the
select mask once and continue to use it over multiple calls to
select_loop(), which is pretty bogus since that would stomp on the
mask on return. This could only matter if the function's outer loop
iterated more than once, which is unlikely (it'd take some connection
receiving data, but not enough to complete its command). But if it
did happen, we'd acquire "tunnel vision" and stop watching the other
connections for query termination, with the effect of losing parallelism.
Another way in which GetIdleSlot could lose parallelism is that once
PQisBusy returns false, it would lock in on that connection and do
PQgetResult until that returns NULL; in some cases that could result
in blocking. (Perhaps this can never happen in vacuumdb due to the
limited set of commands that it can issue, but I'm not quite sure
of that, and even if true today it's not a future-proof assumption.)
Refactor the code to do that properly, so that it risks blocking in
PQgetResult only in cases where we need to wait anyway.
Another loss-of-parallelism problem, which *is* easily demonstrable,
is that any setup queries issued during prepare_vacuum_command() were
always issued on the last-to-be-created connection, whether or not
that was idle. Long-running operations on that connection thus
prevented issuance of additional operations on the other ones, except
in the limited cases where no preparatory query was needed. Instead,
wait till we've identified a free connection and use that one.
Also, avoid core dump due to undersized malloc request in the case
that no tables are identified to be vacuumed.
The bogus no-socket test was noted by CharSyam, the other problems
identified in my own code review. Back-patch to 9.5 where parallel
vacuumdb was introduced.
Discussion: https://postgr.es/m/CAMrLSE6etb33-192DTEUGkV-TsvEcxtBDxGWG1tgNOMnQHwgDA@mail.gmail.com
M src/bin/scripts/vacuumdb.c
Fix bogus provolatile/proparallel markings on a few built-in functions.
commit : 91d82317d252e41885c06597c394d25845faeafc
author : Tom Lane <[email protected]>
date : Fri, 30 Mar 2018 18:14:51 -0400
committer: Tom Lane <[email protected]>
date : Fri, 30 Mar 2018 18:14:51 -0400
Richard Yen reported that pg_upgrade failed if the target cluster had
force_parallel_mode = on, because binary_upgrade_create_empty_extension()
is marked parallel restricted, allowing it to be executed in parallel
mode, which complains because it tries to acquire an XID.
In general, no function that might try to modify database data should
be considered parallel safe or restricted, since execution of it might
force XID acquisition. We found several other examples of this mistake.
Furthermore, functions that execute user-supplied SQL queries or query
fragments, or pull data from user-supplied cursors, had better be marked
both volatile and parallel unsafe, because we don't know what the supplied
query or cursor might try to do. There were several tsquery and XML
functions that had the wrong proparallel marking for this, and some of
them were even mislabeled as to volatility.
All these bugs are old, dating back to 9.6 for the proparallel mistakes
and much further for the provolatile mistakes. We can't force a
catversion bump in the back branches, but we can at least ensure that
installations initdb'd in future have the right values.
Thomas Munro and Tom Lane
Discussion: https://postgr.es/m/CAEepm=2sNDScSLTfyMYu32Q=ob98ZGW-vM_2oLxinzSABGQ6VA@mail.gmail.com
M src/include/catalog/pg_proc.h
docs: add parameter with brackets around varbit()
commit : 64f76ac689fe0697a906ef2390e34baba43655ff
author : Bruce Momjian <[email protected]>
date : Fri, 30 Mar 2018 13:34:12 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 30 Mar 2018 13:34:12 -0400
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Author: Euler Taveira
Backpatch-through: 9.3
M doc/src/sgml/datatype.sgml
Fix handling of files that source server removes during pg_rewind is running.
commit : 52c32d8d8d3cc29d9c2ab4a0c360fb09627231e1
author : Fujii Masao <[email protected]>
date : Thu, 29 Mar 2018 04:00:21 +0900
committer: Fujii Masao <[email protected]>
date : Thu, 29 Mar 2018 04:00:21 +0900
After processing the filemap to build the list of chunks that will be
fetched from the source to rewing the target server, it is possible that
a file which was previously processed is removed from the source. A
simple example of such an occurence is a WAL segment which gets recycled
on the target in-between. When the filemap is processed, files not
categorized as relation files are first truncated to prepare for its
full copy of which is going to be taken from the source, divided into a
set of junks. However, for a recycled WAL segment, this would result in
a segment which has a zero-byte size. With such an empty file,
post-rewind recovery thinks that records are saved but they are actually
not because of the truncation which happened when processing the
filemap, resulting in data loss.
In order to fix the problem, make sure that files which are found as
removed on the source when receiving chunks of them are as well deleted
on the target server for consistency.
Back-patch to 9.5 where pg_rewind was added.
Author: Tsunakawa Takayuki
Reviewed-by: Michael Paquier
Reported-by: Tsunakawa Takayuki
Discussion: https://postgr.es/m/0A3221C70F24FB45833433255569204D1F8DAAA2%40G01JPEXMBYT05
M src/bin/pg_rewind/file_ops.c
M src/bin/pg_rewind/file_ops.h
M src/bin/pg_rewind/libpq_fetch.c
Fix actual and potential double-frees around tuplesort usage.
commit : 90decdba3786d13752bacfe6bbb7f8da452d11ff
author : Tom Lane <[email protected]>
date : Wed, 28 Mar 2018 13:26:43 -0400
committer: Tom Lane <[email protected]>
date : Wed, 28 Mar 2018 13:26:43 -0400
tuplesort_gettupleslot() passed back tuples allocated in the tuplesort's
own memory context, even when the caller was responsible to free them.
This created a double-free hazard, because some callers might destroy
the tuplesort object (via tuplesort_end) before trying to clean up the
last returned tuple. To avoid this, change the API to specify that the
tuple is allocated in the caller's memory context. v10 and HEAD already
did things that way, but in 9.5 and 9.6 this is a live bug that can
demonstrably cause crashes with some grouping-set usages.
In 9.5 and 9.6, this requires doing an extra tuple copy in some cases,
which is unfortunate. But the amount of refactoring needed to avoid it
seems excessive for a back-patched change, especially since the cases
where an extra copy happens are less performance-critical.
Likewise change tuplesort_getdatum() to return pass-by-reference Datums
in the caller's context not the tuplesort's context. There seem to be
no live bugs among its callers, but clearly the same sort of situation
could happen in future.
For other tuplesort fetch routines, continue to allocate the memory in
the tuplesort's context. This is a little inconsistent with what we now
do for tuplesort_gettupleslot() and tuplesort_getdatum(), but that's
preferable to adding new copy overhead in the back branches where it's
clearly unnecessary. These other fetch routines provide the weakest
possible guarantees about tuple memory lifespan from v10 on, anyway,
so this actually seems more consistent overall.
Adjust relevant comments to reflect these API redefinitions.
Arguably, we should change the pre-9.5 branches as well, but since
there are no known failure cases there, it seems not worth the risk.
Peter Geoghegan, per report from Bernd Helmle. Reviewed by Kyotaro
Horiguchi; thanks also to Andreas Seltenreich for extracting a
self-contained test case.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/orderedsetaggs.c
M src/backend/utils/sort/tuplesort.c
Fix thinko in comment
commit : 43cd7abdcc2891abd1aaec5f4c062fa81a9c8f98
author : Alvaro Herrera <[email protected]>
date : Mon, 26 Mar 2018 12:00:25 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 26 Mar 2018 12:00:25 -0300
The listed numbers disagreed with the ones being used in the symbols;
but instead of just fixing the numbers in the comment, use the symbolic
name instead, which seems clearer.
This has been wrong all along, so apply back to 9.5 where BRIN was
introduced.
Reported-by: Tomas Vondra
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/brin/brin_inclusion.c
Doc: add example of type resolution in nested UNIONs.
commit : 356f85f95cb0b0e99ce51932c2a9dbd292f0b292
author : Tom Lane <[email protected]>
date : Sun, 25 Mar 2018 16:15:16 -0400
committer: Tom Lane <[email protected]>
date : Sun, 25 Mar 2018 16:15:16 -0400
Section 10.5 didn't say explicitly that multiple UNIONs are resolved
pairwise. Since the resolution algorithm is described as taking any
number of inputs, readers might well think that a query like
"select x union select y union select z" would be resolved by
considering x, y, and z in one resolution step. But that's not what
happens (and I think that behavior is per SQL spec). Add an example
clarifying this point.
Per bug #15129 from Philippe Beaudoin.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/typeconv.sgml
Doc: remove extra comma in syntax summary for array_fill().
commit : 7b55a3b167514ee78a9284a834abfd3ccb0547c0
author : Tom Lane <[email protected]>
date : Sun, 25 Mar 2018 12:38:21 -0400
committer: Tom Lane <[email protected]>
date : Sun, 25 Mar 2018 12:38:21 -0400
Noted by Scott Ure. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
Don't qualify type pg_catalog.text in extend-extensions-example.
commit : 2c8974e6a02d24be3543741f9bf6ceab384b30d6
author : Noah Misch <[email protected]>
date : Fri, 23 Mar 2018 20:31:03 -0700
committer: Noah Misch <[email protected]>
date : Fri, 23 Mar 2018 20:31:03 -0700
Extension scripts begin execution with pg_catalog at the front of the
search path, so type names reliably refer to pg_catalog. Remove these
superfluous qualifications. Earlier <programlisting> of this <sect1>
already omitted them. Back-patch to 9.3 (all supported versions).
M doc/src/sgml/extend.sgml
Fix make rules that generate multiple output files.
commit : 36c07fc2996b252a144819f4e43d3e715ca6b10d
author : Tom Lane <[email protected]>
date : Fri, 23 Mar 2018 13:45:38 -0400
committer: Tom Lane <[email protected]>
date : Fri, 23 Mar 2018 13:45:38 -0400
For years, our makefiles have correctly observed that "there is no correct
way to write a rule that generates two files". However, what we did is to
provide empty rules that "generate" the secondary output files from the
primary one, and that's not right either. Depending on the details of
the creating process, the primary file might end up timestamped later than
one or more secondary files, causing subsequent make runs to consider the
secondary file(s) out of date. That's harmless in a plain build, since
make will just re-execute the empty rule and nothing happens. But it's
fatal in a VPATH build, since make will expect the secondary file to be
rebuilt in the build directory. This would manifest as "file not found"
failures during VPATH builds from tarballs, if we were ever unlucky enough
to ship a tarball with apparently out-of-date secondary files. (It's not
clear whether that has ever actually happened, but it definitely could.)
To ensure that secondary output files have timestamps >= their primary's,
change our makefile convention to be that we provide a "touch $@" action
not an empty rule. Also, make sure that this rule actually gets invoked
during a distprep run, else the hazard remains.
It's been like this a long time, so back-patch to all supported branches.
In HEAD, I skipped the changes in src/backend/catalog/Makefile, because
those rules are due to get replaced soon in the bootstrap data format
patch, and there seems no need to create a merge issue for that patch.
If for some reason we fail to land that patch in v11, we'll need to
back-fill the changes in that one makefile from v10.
Discussion: https://postgr.es/m/[email protected]
M src/Makefile.shlib
M src/backend/Makefile
M src/backend/catalog/Makefile
M src/backend/parser/Makefile
M src/backend/storage/lmgr/Makefile
M src/backend/utils/Makefile
M src/bin/psql/Makefile
M src/interfaces/ecpg/preproc/Makefile
M src/pl/plpgsql/src/Makefile
M src/test/isolation/Makefile
Fix tuple counting in SP-GiST index build.
commit : db35bf507ffd98dca5ef5e5b73aa282d733ed47c
author : Tom Lane <[email protected]>
date : Thu, 22 Mar 2018 13:23:48 -0400
committer: Tom Lane <[email protected]>
date : Thu, 22 Mar 2018 13:23:48 -0400
Count the number of tuples in the index honestly, instead of assuming
that it's the same as the number of tuples in the heap. (It might be
different if the index is partial.)
Back-patch to all supported versions.
Tomas Vondra
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/spgist/spginsert.c
Fix errors in contrib/bloom index build.
commit : df90401556ce77f861d9a2205c90256968939152
author : Tom Lane <[email protected]>
date : Thu, 22 Mar 2018 13:13:58 -0400
committer: Tom Lane <[email protected]>
date : Thu, 22 Mar 2018 13:13:58 -0400
Count the number of tuples in the index honestly, instead of assuming
that it's the same as the number of tuples in the heap. (It might be
different if the index is partial.)
Fix counting of tuples in current index page, too. This error would
have led to failing to write out the final page of the index if it
contained exactly one tuple, so that the last tuple of the relation
would not get indexed.
Back-patch to 9.6 where contrib/bloom was added.
Tomas Vondra and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M contrib/bloom/blinsert.c
Fix mishandling of quoted-list GUC values in pg_dump and ruleutils.c.
commit : 8132f0f3856d0d36b140ffadb08bb9a38e4b7837
author : Tom Lane <[email protected]>
date : Wed, 21 Mar 2018 20:03:28 -0400
committer: Tom Lane <[email protected]>
date : Wed, 21 Mar 2018 20:03:28 -0400
Code that prints out the contents of setconfig or proconfig arrays in
SQL format needs to handle GUC_LIST_QUOTE variables differently from
other ones, because for those variables, flatten_set_variable_args()
already applied a layer of quoting. The value can therefore safely
be printed as-is, and indeed must be, or flatten_set_variable_args()
will muck it up completely on reload. For all other GUC variables,
it's necessary and sufficient to quote the value as a SQL literal.
We'd recognized the need for this long ago, but mis-analyzed the
need slightly, thinking that all GUC_LIST_INPUT variables needed
the special treatment. That's actually wrong, since a valid value
of a LIST variable might include characters that need quoting,
although no existing variables accept such values.
More to the point, we hadn't made any particular effort to keep the
various places that deal with this up-to-date with the set of variables
that actually need special treatment, meaning that we'd do the wrong
thing with, for example, temp_tablespaces values. This affects dumping
of SET clauses attached to functions, as well as ALTER DATABASE/ROLE SET
commands.
In ruleutils.c we can fix it reasonably honestly by exporting a guc.c
function that allows discovering the flags for a given GUC variable.
But pg_dump doesn't have easy access to that, so continue the old method
of having a hard-wired list of affected variable names. At least we can
fix it to have just one list not two, and update the list to match
current reality.
A remaining problem with this is that it only works for built-in
GUC variables. pg_dump's list obvious knows nothing of third-party
extensions, and even the "ask guc.c" method isn't bulletproof since
the relevant extension might not be loaded. There's no obvious
solution to that, so for now, we'll just have to discourage extension
authors from inventing custom GUCs that need GUC_LIST_QUOTE.
This has been busted for a long time, so back-patch to all supported
branches.
Michael Paquier and Tom Lane, reviewed by Kyotaro Horiguchi and
Pavel Stehule
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/misc/guc.c
M src/bin/pg_dump/dumputils.c
M src/bin/pg_dump/dumputils.h
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
M src/include/utils/guc.h
M src/test/regress/expected/rules.out
M src/test/regress/sql/rules.sql
Fix typo.
commit : 36e1d45718c3415d42a12faddf4d6b2f0c488f2b
author : Tatsuo Ishii <[email protected]>
date : Wed, 21 Mar 2018 23:08:43 +0900
committer: Tatsuo Ishii <[email protected]>
date : Wed, 21 Mar 2018 23:08:43 +0900
Patch by me.
M doc/src/sgml/protocol.sgml
Rework word_similarity documentation, make it close to actual algorithm.
commit : 4c7feb16116540091e9d3766350c70dbb3c525d7
author : Teodor Sigaev <[email protected]>
date : Wed, 21 Mar 2018 14:37:51 +0300
committer: Teodor Sigaev <[email protected]>
date : Wed, 21 Mar 2018 14:37:51 +0300
word_similarity before claimed as returning similarity of closest word in
string, but, actually it returns similarity of substring. Also fix mistyped
comments.
Author: Alexander Korotkov
Review by: David Steele, Liudmila Mantrova
Discussionis:
https://www.postgresql.org/message-id/flat/CY4PR17MB13207ED8310F847CF117EED0D85A0@CY4PR17MB1320.namprd17.prod.outlook.com
https://www.postgresql.org/message-id/flat/f43b242d-000c-f4c8-cb8b-d37e9752cd93%40postgrespro.ru
M contrib/pg_trgm/trgm_op.c
M doc/src/sgml/pgtrgm.sgml
Doc: typo fix, "PG_" should be "TG_" here.
commit : eb63b72388b662039a886850257294d7d3f84f8f
author : Tom Lane <[email protected]>
date : Tue, 20 Mar 2018 11:34:12 -0400
committer: Tom Lane <[email protected]>
date : Tue, 20 Mar 2018 11:34:12 -0400
Too much PG on the brain in commit 769159fd3, evidently.
Noted by [email protected].
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/plpgsql.sgml
Prevent query-lifespan memory leakage of SP-GiST traversal values.
commit : 57ef2da434e235f6295bf49a065fce3da398fd8b
author : Tom Lane <[email protected]>
date : Mon, 19 Mar 2018 23:59:17 -0400
committer: Tom Lane <[email protected]>
date : Mon, 19 Mar 2018 23:59:17 -0400
The original coding of the SP-GiST scan traversalValue feature (commit
ccd6eb49a) arranged for traversal values to be stored in the query's main
executor context. That's fine if there's only one index scan per query,
but if there are many, we have a memory leak as successive scans create
new traversal values. Fix it by creating a separate memory context for
traversal values, which we can reset during spgrescan(). Back-patch
to 9.6 where this code was introduced.
In principle, adding the traversalCxt field to SpGistScanOpaqueData
creates an ABI break in the back branches. But I (tgl) have little
sympathy for extensions including spgist_private.h, so I'm not very
worried about that. Alternatively we could stick the new field at the
end of the struct in back branches, but that has its own downsides.
Anton Dignös, reviewed by Alexander Kuzmenkov
Discussion: https://postgr.es/m/CALNdv1jb6y2Te-m8xHLxLX12RsBmZJ1f4hESX7J0HjgyOhA9eA@mail.gmail.com
M src/backend/access/spgist/spgscan.c
M src/include/access/spgist_private.h
Fix some corner-case issues in REFRESH MATERIALIZED VIEW CONCURRENTLY.
commit : 7d7b59aa65918f59e3c122318b31e0f77f12f2cd
author : Tom Lane <[email protected]>
date : Mon, 19 Mar 2018 18:49:53 -0400
committer: Tom Lane <[email protected]>
date : Mon, 19 Mar 2018 18:49:53 -0400
refresh_by_match_merge() has some issues in the way it builds a SQL
query to construct the "diff" table:
1. It doesn't require the selected unique index(es) to be indimmediate.
2. It doesn't pay attention to the particular equality semantics enforced
by a given index, but just assumes that they must be those of the column
datatype's default btree opclass.
3. It doesn't check that the indexes are btrees.
4. It's insufficiently careful to ensure that the parser will pick the
intended operator when parsing the query. (This would have been a
security bug before CVE-2018-1058.)
5. It's not careful about indexes on system columns.
The way to fix #4 is to make use of the existing code in ri_triggers.c
for generating an arbitrary binary operator clause. I chose to move
that to ruleutils.c, since that seems a more reasonable place to be
exporting such functionality from than ri_triggers.c.
While #1, #3, and #5 are just latent given existing feature restrictions,
and #2 doesn't arise in the core system for lack of alternate opclasses
with different equality behaviors, #4 seems like an issue worth
back-patching. That's the bulk of the change anyway, so just back-patch
the whole thing to 9.4 where this code was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/matview.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/adt/ruleutils.c
M src/include/utils/builtins.h
Fix performance hazard in REFRESH MATERIALIZED VIEW CONCURRENTLY.
commit : ebcf34d4628e175a7f35714ccec561b596fe6d0f
author : Tom Lane <[email protected]>
date : Mon, 19 Mar 2018 17:23:07 -0400
committer: Tom Lane <[email protected]>
date : Mon, 19 Mar 2018 17:23:07 -0400
Jeff Janes discovered that commit 7ca25b7de made one of the queries run by
REFRESH MATERIALIZED VIEW CONCURRENTLY perform badly. The root cause is
bad cardinality estimation for correlated quals, but a principled solution
to that problem is some way off, especially since the planner lacks any
statistics about whole-row variables. Moreover, in non-error cases this
query produces no rows, meaning it must be run to completion; but use of
LIMIT 1 encourages the planner to pick a fast-start, slow-completion plan,
exactly not what we want. Remove the LIMIT clause, and instead rely on
the count parameter we pass to SPI_execute() to prevent excess work if the
query does return some rows.
While we've heard no field reports of planner misbehavior with this query,
it could be that people are having performance issues that haven't reached
the level of pain needed to cause a bug report. In any case, that LIMIT
clause can't possibly do anything helpful with any existing version of the
planner, and it demonstrably can cause bad choices in some cases, so
back-patch to 9.4 where the code was introduced.
Thomas Munro
Discussion: https://postgr.es/m/CAMkU=1z-JoGymHneGHar1cru4F1XDfHqJDzxP_CtK5cL3DOfmg@mail.gmail.com
M src/backend/commands/matview.c
Doc: note that statement-level view triggers require an INSTEAD OF trigger.
commit : 7b544c4e053b93851280ee14da8e3b1049b7347c
author : Tom Lane <[email protected]>
date : Sun, 18 Mar 2018 15:10:28 -0400
committer: Tom Lane <[email protected]>
date : Sun, 18 Mar 2018 15:10:28 -0400
If a view lacks an INSTEAD OF trigger, DML on it can only work by rewriting
the command into a command on the underlying base table(s). Then we will
fire triggers attached to those table(s), not those for the view. This
seems appropriate from a consistency standpoint, but nowhere was the
behavior explicitly documented, so let's do that.
There was some discussion of throwing an error or warning if a statement
trigger is created on a view without creating a row INSTEAD OF trigger.
But a simple implementation of that would result in dump/restore ordering
hazards. Given that it's been like this all along, and we hadn't heard
a complaint till now, a documentation improvement seems sufficient.
Per bug #15106 from Pu Qun. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_trigger.sgml
M doc/src/sgml/trigger.sgml
Fix pg_recvlogical for pre-10 versions
commit : 59743deca9a19126ad711874b05c88ab6a59fc47
author : Magnus Hagander <[email protected]>
date : Sun, 18 Mar 2018 13:08:25 +0100
committer: Magnus Hagander <[email protected]>
date : Sun, 18 Mar 2018 13:08:25 +0100
In e170b8c8, protection against modified search_path was added. However,
PostgreSQL versions prior to 10 does not accept SQL commands over a
replication connection, so the protection would generate a syntax error.
Since we cannot run SQL commands on it, we are also not vulnerable to
the issue that e170b8c8 fixes, so we can just skip this command for
older versions.
Author: Michael Paquier <[email protected]>
M src/bin/pg_basebackup/streamutil.c
Fix overflow handling in plpgsql's integer FOR loops.
commit : 5917297bfd25ea78416506b2d606316a8052b028
author : Tom Lane <[email protected]>
date : Sat, 17 Mar 2018 15:38:15 -0400
committer: Tom Lane <[email protected]>
date : Sat, 17 Mar 2018 15:38:15 -0400
The test to exit the loop if the integer control value would overflow
an int32 turns out not to work on some ICC versions, as it's dependent
on the assumption that the compiler will execute the code as written
rather than "optimize" it. ICC lacks any equivalent of gcc's -fwrapv
switch, so it was optimizing on the assumption of no integer overflow,
and that breaks this. Rewrite into a form that in fact does not
do any overflowing computations.
Per Tomas Vondra and buildfarm member fulmar. It's been like this
for a long time, although it was not till we added a regression test
case covering the behavior (in commit dd2243f2a) that the problem
became apparent. Back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]
M src/pl/plpgsql/src/pl_exec.c
Fix WHERE CURRENT OF when the referenced cursor uses an index-only scan.
commit : 12d18b4870a2561e39d36999c5ba94899ac07f44
author : Tom Lane <[email protected]>
date : Sat, 17 Mar 2018 14:59:31 -0400
committer: Tom Lane <[email protected]>
date : Sat, 17 Mar 2018 14:59:31 -0400
"UPDATE/DELETE WHERE CURRENT OF cursor_name" failed, with an error message
like "cannot extract system attribute from virtual tuple", if the cursor
was using a index-only scan for the target table. Fix it by digging the
current TID out of the indexscan state.
It seems likely that the same failure could occur for CustomScan plans
and perhaps some FDW plan types, so that leaving this to be treated as an
internal error with an obscure message isn't as good an idea as it first
seemed. Hence, add a bit of heaptuple.c infrastructure to let us deliver
a more on-topic message. I chose to make the message match what you get
for the case where execCurrentOf can't identify the target scan node at
all, "cursor "foo" is not a simply updatable scan of table "bar"".
Perhaps it should be different, but we can always adjust that later.
In the future, it might be nice to provide hooks that would let custom
scan providers and/or FDWs deal with this in other ways; but that's
not a suitable topic for a back-patchable bug fix.
It's been like this all along, so back-patch to all supported branches.
Yugo Nagata and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/common/heaptuple.c
M src/backend/executor/execCurrent.c
M src/include/executor/tuptable.h
M src/test/regress/expected/portals.out
M src/test/regress/sql/portals.sql
Fix query-lifespan memory leakage in repeatedly executed hash joins.
commit : 5149dc93467465e38645a5c318d7d90d806b5e53
author : Tom Lane <[email protected]>
date : Fri, 16 Mar 2018 16:03:45 -0400
committer: Tom Lane <[email protected]>
date : Fri, 16 Mar 2018 16:03:45 -0400
ExecHashTableCreate allocated some memory that wasn't freed by
ExecHashTableDestroy, specifically the per-hash-key function information.
That's not a huge amount of data, but if one runs a query that repeats
a hash join enough times, it builds up. Fix by arranging for the data
in question to be kept in the hashtable's hashCxt instead of leaving it
"loose" in the query-lifespan executor context. (This ensures that we'll
also clean up anything that the hash functions allocate in fn_mcxt.)
Per report from Amit Khandekar. It's been like this forever, so back-patch
to all supported branches.
Discussion: https://postgr.es/m/CAJ3gD9cFofAWGvcxLOxDHC=B0hjtW8yGmUsF2hdGh97CM38=7g@mail.gmail.com
M src/backend/executor/nodeHash.c
Doc: explicitly point out that enum values can't be dropped.
commit : 02c4ad357d2189a48d744449a8b9d165ea42ad24
author : Tom Lane <[email protected]>
date : Fri, 16 Mar 2018 13:44:34 -0400
committer: Tom Lane <[email protected]>
date : Fri, 16 Mar 2018 13:44:34 -0400
This was not stated in so many words anywhere. Document it to make
clear that it's a design limitation and not just an oversight or
documentation omission.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/datatype.sgml
test_ddl_deparse: rename matview
commit : eb1c481ac3975574c1709042efe2c6c32a937a23
author : Alvaro Herrera <[email protected]>
date : Thu, 15 Mar 2018 15:14:15 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 15 Mar 2018 15:14:15 -0300
Should have done this in e69f5e0efca ...
Per note from Tom Lane.
M src/test/modules/test_ddl_deparse/expected/matviews.out
M src/test/modules/test_ddl_deparse/sql/matviews.sql
test_ddl_deparse: Don't use pg_class as source for a matview
commit : 91797fa43817b1a29bf783df7479915e89770fe6
author : Alvaro Herrera <[email protected]>
date : Thu, 15 Mar 2018 09:51:12 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 15 Mar 2018 09:51:12 -0300
Doing so causes a pg_upgrade of a database containing these objects to
fail whenever pg_class changes. And it's pointless anyway: we have more
interesting tables anyhow.
Discussion: https://postgr.es/m/CAD5tBc+s8pW9WvH2+_z=B4x95FD4QuzZKcaMpff_9H4rS0VU1A@mail.gmail.com
M src/test/modules/test_ddl_deparse/expected/matviews.out
M src/test/modules/test_ddl_deparse/sql/matviews.sql
Fix double frees in ecpg.
commit : 8e3f3ab5b85a230e9a008c743402c3e5a43085a1
author : Michael Meskes <[email protected]>
date : Wed, 14 Mar 2018 00:47:49 +0100
committer: Michael Meskes <[email protected]>
date : Wed, 14 Mar 2018 00:47:49 +0100
Patch by Patrick Krecker <[email protected]>
M src/interfaces/ecpg/preproc/ecpg.c
When updating reltuples after ANALYZE, just extrapolate from our sample.
commit : c2c4bc628bbe7f1c77a6d3044b1380753c2acbbe
author : Tom Lane <[email protected]>
date : Tue, 13 Mar 2018 13:24:27 -0400
committer: Tom Lane <[email protected]>
date : Tue, 13 Mar 2018 13:24:27 -0400
The existing logic for updating pg_class.reltuples trusted the sampling
results only for the pages ANALYZE actually visited, preferring to
believe the previous tuple density estimate for all the unvisited pages.
While there's some rationale for doing that for VACUUM (first that
VACUUM is likely to visit a very nonrandom subset of pages, and second
that we know for sure that the unvisited pages did not change), there's
no such rationale for ANALYZE: by assumption, it's looked at an unbiased
random sample of the table's pages. Furthermore, in a very large table
ANALYZE will have examined only a tiny fraction of the table's pages,
meaning it cannot slew the overall density estimate very far at all.
In a table that is physically growing, this causes reltuples to increase
nearly proportionally to the change in relpages, regardless of what is
actually happening in the table. This has been observed to cause reltuples
to become so much larger than reality that it effectively shuts off
autovacuum, whose threshold for doing anything is a fraction of reltuples.
(Getting to the point where that would happen seems to require some
additional, not well understood, conditions. But it's undeniable that if
reltuples is seriously off in a large table, ANALYZE alone will not fix it
in any reasonable number of iterations, especially not if the table is
continuing to grow.)
Hence, restrict the use of vac_estimate_reltuples() to VACUUM alone,
and in ANALYZE, just extrapolate from the sample pages on the assumption
that they provide an accurate model of the whole table. If, by very bad
luck, they don't, at least another ANALYZE will fix it; in the old logic
a single bad estimate could cause problems indefinitely.
In HEAD, let's remove vac_estimate_reltuples' is_analyze argument
altogether; it was never used for anything and now it's totally pointless.
But keep it in the back branches, in case any third-party code is calling
this function.
Per bug #15005. Back-patch to all supported branches.
David Gould, reviewed by Alexander Kuzmenkov, cosmetic changes by me
Discussion: https://postgr.es/m/20180117164916.3fdcf2e9@engels
M src/backend/commands/analyze.c
M src/backend/commands/vacuum.c
Avoid holding AutovacuumScheduleLock while rechecking table statistics.
commit : 4b0e717053e36f931f3cfc9b24060b281db7900b
author : Tom Lane <[email protected]>
date : Tue, 13 Mar 2018 12:28:15 -0400
committer: Tom Lane <[email protected]>
date : Tue, 13 Mar 2018 12:28:15 -0400
In databases with many tables, re-fetching the statistics takes some time,
so that this behavior seriously decreases the available concurrency for
multiple autovac workers. There's discussion afoot about more complete
fixes, but a simple and back-patchable amelioration is to claim the table
and release the lock before rechecking stats. If we find out there's no
longer a reason to process the table, re-taking the lock to un-claim the
table is cheap enough.
(This patch is quite old, but got lost amongst a discussion of more
aggressive fixes. It's not clear when or if such a fix will be
accepted, but in any case it'd be unlikely to get back-patched.
Let's do this now so we have some improvement for the back branches.)
In passing, make the normal un-claim step take AutovacuumScheduleLock
not AutovacuumLock, since that is what is documented to protect the
wi_tableoid field. This wasn't an actual bug in view of the fact that
readers of that field hold both locks, but it creates some concurrency
penalty against operations that need only AutovacuumLock.
Back-patch to all supported versions.
Jeff Janes
Discussion: https://postgr.es/m/[email protected]
M src/backend/postmaster/autovacuum.c
Set connection back to NULL after freeing it.
commit : 44a36a8d9a6e7ba209253d694dd2ebf3e13f0b5d
author : Michael Meskes <[email protected]>
date : Mon, 12 Mar 2018 23:52:08 +0100
committer: Michael Meskes <[email protected]>
date : Mon, 12 Mar 2018 23:52:08 +0100
Patch by Jeevan Ladhe <[email protected]>
M src/interfaces/ecpg/preproc/output.c
Fix improper uses of canonicalize_qual().
commit : 976e5844ef53ee2e777a4725ae20db1392887098
author : Tom Lane <[email protected]>
date : Sun, 11 Mar 2018 18:10:42 -0400
committer: Tom Lane <[email protected]>
date : Sun, 11 Mar 2018 18:10:42 -0400
One of the things canonicalize_qual() does is to remove constant-NULL
subexpressions of top-level AND/OR clauses. It does that on the assumption
that what it's given is a top-level WHERE clause, so that NULL can be
treated like FALSE. Although this is documented down inside a subroutine
of canonicalize_qual(), it wasn't mentioned in the documentation of that
function itself, and some callers hadn't gotten that memo.
Notably, commit d007a9505 caused get_relation_constraints() to apply
canonicalize_qual() to CHECK constraints. That allowed constraint
exclusion to misoptimize situations in which a CHECK constraint had a
provably-NULL subclause, as seen in the regression test case added here,
in which a child table that should be scanned is not. (Although this
thinko is ancient, the test case doesn't fail before 9.2, for reasons
I've not bothered to track down in detail. There may be related cases
that do fail before that.)
More recently, commit f0e44751d added an independent bug by applying
canonicalize_qual() to index expressions, which is even sillier since
those might not even be boolean. If they are, though, I think this
could lead to making incorrect index entries for affected index
expressions in v10. I haven't attempted to prove that though.
To fix, add an "is_check" parameter to canonicalize_qual() to specify
whether it should assume WHERE or CHECK semantics, and make it perform
NULL-elimination accordingly. Adjust the callers to apply the right
semantics, or remove the call entirely in cases where it's not known
that the expression has one or the other semantics. I also removed
the call in some cases involving partition expressions, where it should
be a no-op because such expressions should be canonical already ...
and was a no-op, independently of whether it could in principle have
done something, because it was being handed the qual in implicit-AND
format which isn't what it expects. In HEAD, add an Assert to catch
that type of mistake in future.
This represents an API break for external callers of canonicalize_qual().
While that's intentional in HEAD to make such callers think about which
case applies to them, it seems like something we probably wouldn't be
thanked for in released branches. Hence, in released branches, the
extra parameter is added to a new function canonicalize_qual_ext(),
and canonicalize_qual() is a wrapper that retains its old behavior.
Patch by me with suggestions from Dean Rasheed. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepqual.c
M src/backend/optimizer/util/plancat.c
M src/backend/utils/cache/relcache.c
M src/include/optimizer/prep.h
M src/test/regress/expected/inherit.out
M src/test/regress/sql/inherit.sql
Fix warnings in man page build
commit : 2f09dc11936b8656328ad66f7dc7d2dd5e45ed9e
author : Peter Eisentraut <[email protected]>
date : Wed, 28 Feb 2018 08:22:51 -0500
committer: Peter Eisentraut <[email protected]>
date : Wed, 28 Feb 2018 08:22:51 -0500
The changes in the CREATE POLICY man page from commit
87c2a17fee784c7e1004ba3d3c5d8147da676783 triggered a stylesheet bug that
created some warning messages and incorrect output. This installs a
workaround.
Also improve the whitespace a bit so it looks better.
M doc/src/sgml/ref/create_policy.sgml
M doc/src/sgml/stylesheet-man.xsl
Refrain from duplicating data in reorderbuffers
commit : 8e5c2afa98d9c28c9883c4c42890f3a6dd99ba4e
author : Alvaro Herrera <[email protected]>
date : Tue, 6 Mar 2018 15:57:20 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 6 Mar 2018 15:57:20 -0300
If a walsender exits leaving data in reorderbuffers, the next walsender
that tries to decode the same transaction would append its decoded data
in the same spill files without truncating it first, which effectively
duplicate the data. Avoid that by removing any leftover reorderbuffer
spill files when a walsender starts.
Backpatch to 9.4; this bug has been there from the very beginning of
logical decoding.
Author: Craig Ringer, revised by me
Reviewed by: Álvaro Herrera, Petr Jelínek, Masahiko Sawada
M src/backend/replication/logical/reorderbuffer.c
Fix pg_rewind to handle relation data files in tablespaces properly.
commit : 7aba4f23f5b7717f70f6ea0cb4361c481bb4863b
author : Fujii Masao <[email protected]>
date : Tue, 6 Mar 2018 02:08:18 +0900
committer: Fujii Masao <[email protected]>
date : Tue, 6 Mar 2018 02:08:18 +0900
pg_rewind checks whether each file is a relation data file, from its path.
Previously this check logic had the bug which made pg_rewind fail to
recognize any relation data files in tablespaces. Which also caused
an assertion failure in pg_rewind.
Back-patch to 9.5 where pg_rewind was added.
Author: Takayuki Tsunakawa
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/0A3221C70F24FB45833433255569204D1F8D6C7A@G01JPEXMBYT05
M src/bin/pg_rewind/filemap.c
Fix assorted issues in convert_to_scalar().
commit : e2108f58131f6618477ca7c1adaeaeec52c16ca1
author : Tom Lane <[email protected]>
date : Sat, 3 Mar 2018 20:31:35 -0500
committer: Tom Lane <[email protected]>
date : Sat, 3 Mar 2018 20:31:35 -0500
If convert_to_scalar is passed a pair of datatypes it can't cope with,
its former behavior was just to elog(ERROR). While this is OK so far as
the core code is concerned, there's extension code that would like to use
scalarltsel/scalargtsel/etc as selectivity estimators for operators that
work on non-core datatypes, and this behavior is a show-stopper for that
use-case. If we simply allow convert_to_scalar to return FALSE instead of
outright failing, then the main logic of scalarltsel/scalargtsel will work
fine for any operator that behaves like a scalar inequality comparison.
The lack of conversion capability will mean that we can't estimate to
better than histogram-bin-width precision, since the code will effectively
assume that the comparison constant falls at the middle of its bin. But
that's still a lot better than nothing. (Someday we should provide a way
for extension code to supply a custom version of convert_to_scalar, but
today is not that day.)
While poking at this issue, we noted that the existing code for handling
type bytea in convert_to_scalar is several bricks shy of a load.
It assumes without checking that if the comparison value is type bytea,
the bounds values are too; in the worst case this could lead to a crash.
It also fails to detoast the input values, so that the comparison result is
complete garbage if any input is toasted out-of-line, compressed, or even
just short-header. I'm not sure how often such cases actually occur ---
the bounds values, at least, are probably safe since they are elements of
an array and hence can't be toasted. But that doesn't make this code OK.
Back-patch to all supported branches, partly because author requested that,
but mostly because of the bytea bugs. The change in API for the exposed
routine convert_network_to_scalar() is theoretically a back-patch hazard,
but it seems pretty unlikely that any third-party code is calling that
function directly.
Tomas Vondra, with some adjustments by me
Discussion: https://postgr.es/m/[email protected]
M contrib/btree_gist/btree_inet.c
M src/backend/utils/adt/network.c
M src/backend/utils/adt/selfuncs.c
M src/include/utils/builtins.h
doc: Fix links to pg_stat_replication
commit : 79300d09fc4c61945c008237e0457042e23e8ea8
author : Peter Eisentraut <[email protected]>
date : Sat, 3 Mar 2018 14:11:39 -0500
committer: Peter Eisentraut <[email protected]>
date : Sat, 3 Mar 2018 14:11:39 -0500
In PostgreSQL 9.5, the documentation for pg_stat_replication was moved,
so some of the links pointed to an appropriate location.
Author: Maksim Milyutin <[email protected]>
M doc/src/sgml/config.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/release-9.1.sgml
M doc/src/sgml/release-9.5.sgml
Fix VM buffer pin management in heap_lock_updated_tuple_rec().
commit : 96d2df840b0669aad6792b459805367ba70641af
author : Tom Lane <[email protected]>
date : Fri, 2 Mar 2018 17:40:48 -0500
committer: Tom Lane <[email protected]>
date : Fri, 2 Mar 2018 17:40:48 -0500
Sloppy coding in this function could lead to leaking a VM buffer pin,
or to attempting to free the same pin twice. Repair. While at it,
reduce the code's tendency to free and reacquire the same page pin.
Back-patch to 9.6; before that, this routine did not concern itself
with VM pages.
Amit Kapila and Tom Lane
Discussion: https://postgr.es/m/CAA4eK1KJKwhc=isgTQHjM76CAdVswzNeAuZkh_cx-6QgGkSEgA@mail.gmail.com
M src/backend/access/heap/heapam.c
Make gistvacuumcleanup() count the actual number of index tuples.
commit : 529137cac9a8f0db6cca23a5c305b533cd2dd880
author : Tom Lane <[email protected]>
date : Fri, 2 Mar 2018 11:22:42 -0500
committer: Tom Lane <[email protected]>
date : Fri, 2 Mar 2018 11:22:42 -0500
Previously, it just returned the heap tuple count, which might be only an
estimate, and would be completely the wrong thing if the index is partial.
Since this function scans every index page anyway to find free pages,
it's practically free to count the surviving index tuples. Let's do that
and return an accurate count.
This is easily visible as a wrong reltuples value for a partial GiST
index following VACUUM, so back-patch to all supported branches.
Andrey Borodin, reviewed by Michail Nikolaev
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/gist/gistvacuum.c
Use ereport not elog for some corrupt-HOT-chain reports.
commit : 8e25be4859783f0971c5a1b9f9d12bcea18c59ec
author : Tom Lane <[email protected]>
date : Thu, 1 Mar 2018 16:23:30 -0500
committer: Tom Lane <[email protected]>
date : Thu, 1 Mar 2018 16:23:30 -0500
These errors have been seen in the field in corrupted-data situations.
It seems worthwhile to report them with ERRCODE_DATA_CORRUPTED, rather
than the generic ERRCODE_INTERNAL_ERROR, for the benefit of log monitoring
and tools like amcheck. However, use errmsg_internal so that the text
strings still aren't translated; it seems unlikely to be worth
translators' time to do so.
Back-patch to 9.3, like the predecessor commit d70cf811f that introduced
these elog calls originally (replacing Asserts).
Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-Wzmn4-Pg-UGFwyuyK-wiTih9j32pwg_7T9iwqXpAUZr=Mg@mail.gmail.com
M src/backend/catalog/index.c
Relax overly strict sanity check for upgraded ancient databases
commit : 0ddaaa4cf4ff6e200767daf831e4cd955bafaa61
author : Alvaro Herrera <[email protected]>
date : Thu, 1 Mar 2018 18:07:46 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 1 Mar 2018 18:07:46 -0300
Commit 4800f16a7ad0 added some sanity checks to ensure we don't
accidentally corrupt data, but in one of them we failed to consider the
effects of a database upgraded from 9.2 or earlier, where a tuple
exclusively locked prior to the upgrade has a slightly different bit
pattern. Fix that by using the macro that we fixed in commit
74ebba84aeb6 for similar situations.
Reported-by: Alexandre Garcia
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/CAPYLKR6yxV4=pfW0Gwij7aPNiiPx+3ib4USVYnbuQdUtmkMaEA@mail.gmail.com
Andres suspects that this bug may have wider ranging consequences, but I
couldn't find anything.
M src/backend/access/heap/heapam.c
Fix IOS planning when only some index columns can return an attribute.
commit : 3f26be83ec1e8976566d020da738c032c4aee3f3
author : Tom Lane <[email protected]>
date : Thu, 1 Mar 2018 15:35:03 -0500
committer: Tom Lane <[email protected]>
date : Thu, 1 Mar 2018 15:35:03 -0500
Since 9.5, it's possible that some but not all columns of an index
support returning the indexed value for index-only scans. If the
same indexed column appears in index columns that behave both ways,
check_index_only() supposed that it'd be OK to do an index-only scan
testing that column; but that fails if we have to recheck the indexed
condition on one of the columns that doesn't support this.
In principle we could make this work by remapping the recheck expressions
to pull the value from a column that does support returning the indexed
value. But such cases are so weird and rare that, at least for now,
it doesn't seem worth the trouble. Instead, just teach check_index_only
that a value is returnable only if all the index columns containing it
are returnable, rather than any of them.
Per report from David Pereiro Lagares. Back-patch to 9.5 where the
possibility of this situation appeared.
Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M contrib/btree_gist/expected/inet.out
M contrib/btree_gist/sql/inet.sql
M src/backend/optimizer/path/indxpath.c
Rename base64 routines to avoid conflict with Solaris built-in functions.
commit : 11e7700e584e818b40013ef8e6bb17d04d8210f8
author : Tom Lane <[email protected]>
date : Wed, 28 Feb 2018 18:33:45 -0500
committer: Tom Lane <[email protected]>
date : Wed, 28 Feb 2018 18:33:45 -0500
Solaris 11.4 has built-in functions named b64_encode and b64_decode.
Rename ours to something else to avoid the conflict (fortunately,
ours are static so the impact is limited).
One could wish for less duplication of code in this area, but that
would be a larger patch and not very suitable for back-patching.
Since this is a portability fix, we want to put it into all supported
branches.
Report and initial patch by Rainer Orth, reviewed and adjusted a bit
by Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M contrib/pgcrypto/pgp-armor.c
M src/backend/utils/adt/encode.c
Remove restriction on SQL block length in isolationtester scanner.
commit : a030b997ae33628bd77b069d23b9adc71cac9f8c
author : Tom Lane <[email protected]>
date : Wed, 28 Feb 2018 16:57:37 -0500
committer: Tom Lane <[email protected]>
date : Wed, 28 Feb 2018 16:57:37 -0500
specscanner.l had a fixed limit of 1024 bytes on the length of
individual SQL stanzas in an isolation test script. People are
starting to run into that, so fix it by making the buffer resizable.
Once we allow this in HEAD, it seems inevitable that somebody will
try to back-patch a test that exceeds the old limit, so back-patch
this change as a preventive measure.
Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
M src/test/isolation/specscanner.l
Fix up ecpg's configuration so it handles "long long int" in MSVC builds.
commit : 7ee8005ceddf900d1061ae692f236dc086f4cae7
author : Tom Lane <[email protected]>
date : Tue, 27 Feb 2018 16:46:52 -0500
committer: Tom Lane <[email protected]>
date : Tue, 27 Feb 2018 16:46:52 -0500
Although configure-based builds correctly define HAVE_LONG_LONG_INT when
appropriate (in both pg_config.h and ecpg_config.h), builds using the MSVC
scripts failed to do so. This currently has no impact on the backend,
since it uses that symbol nowhere; but it does prevent ecpg from
supporting "long long int". Fix that.
Also, adjust Solution.pm so that in the constructed ecpg_config.h file,
the "#if (_MSC_VER > 1200)" covers only the LONG_LONG_INT-related
#defines, not the whole file. AFAICS this was a thinko on somebody's
part: ENABLE_THREAD_SAFETY should always be defined in Windows builds,
and in branches using USE_INTEGER_DATETIMES, the setting of that shouldn't
depend on the compiler version either. If I'm wrong, I imagine the
buildfarm will say so.
Per bug #15080 from Jonathan Allen; issue diagnosed by Michael Meskes
and Andrew Gierth. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/include/pg_config.h.win32
M src/tools/msvc/Solution.pm
Remove regression tests' CREATE FUNCTION commands for unused C functions.
commit : df3962c00b36efd406aac2c1a7ba5177d5a7d1e1
author : Tom Lane <[email protected]>
date : Tue, 27 Feb 2018 15:04:21 -0500
committer: Tom Lane <[email protected]>
date : Tue, 27 Feb 2018 15:04:21 -0500
I removed these functions altogether in HEAD, in commit db3af9feb, and
it emerges that that causes trouble for cross-branch upgrade testing.
We could put back stub functions but that seems pretty silly. Instead,
back-patch a minimal subset of db3af9feb, namely just removing the
CREATE FUNCTION commands.
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/input/create_function_1.source
M src/test/regress/input/create_function_2.source
M src/test/regress/output/create_function_1.source
M src/test/regress/output/create_function_2.source
Prevent dangling-pointer access when update trigger returns old tuple.
commit : 06f47297e2b05971e89dafca01ed00f12313ca16
author : Tom Lane <[email protected]>
date : Tue, 27 Feb 2018 13:27:38 -0500
committer: Tom Lane <[email protected]>
date : Tue, 27 Feb 2018 13:27:38 -0500
A before-update row trigger may choose to return the "new" or "old" tuple
unmodified. ExecBRUpdateTriggers failed to consider the second
possibility, and would proceed to free the "old" tuple even if it was the
one returned, leading to subsequent access to already-deallocated memory.
In debug builds this reliably leads to an "invalid memory alloc request
size" failure; in production builds it might accidentally work, but data
corruption is also possible.
This is a very old bug. There are probably a couple of reasons it hasn't
been noticed up to now. It would be more usual to return NULL if one
wanted to suppress the update action; returning "old" is significantly less
efficient since the update will occur anyway. Also, none of the standard
PLs would ever cause this because they all returned freshly-manufactured
tuples even if they were just copying "old". But commit 4b93f5799 changed
that for plpgsql, making it possible to see the bug with a plpgsql trigger.
Still, this is certainly legal behavior for a trigger function, so it's
ExecBRUpdateTriggers's fault not plpgsql's.
It seems worth creating a test case that exercises returning "old" directly
with a C-language trigger; testing this through plpgsql seems unreliable
because its behavior might change again.
Report and fix by Rushabh Lathia; regression test case by me.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/CAGPqQf1P4pjiNPrMof=P_16E-DFjt457j+nH2ex3=nBTew7tXw@mail.gmail.com
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/input/create_function_1.source
M src/test/regress/output/create_function_1.source
M src/test/regress/regress.c
M src/test/regress/sql/triggers.sql