Stamp 9.6.11.
commit : 518d5492911d445b126e9d5c669a83b5cae43e50
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Nov 2018 16:47:37 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Nov 2018 16:47:37 -0500
M configure
M configure.in
M doc/bug.template
M src/include/pg_config.h.win32
M src/interfaces/libpq/libpq.rc.in
M src/port/win32ver.rc
Fix copy-paste error in errhint() introduced in 691d79a07933.
commit : a16e8b440132d0418d7d0c8986ac944b2370a1d4
author : Andres Freund <andres@anarazel.de>
date : Mon, 5 Nov 2018 12:02:25 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 5 Nov 2018 12:02:25 -0800
Reported-By: Petr Jelinek
Discussion: https://postgr.es/m/c95a620b-34f0-7930-aeb5-f7ab804f26cb@2ndquadrant.com
Backpatch: 9.4-, like the previous commit
M src/backend/replication/slot.c
Translation updates
commit : 1dc05482fee545279951af06c3f0a17500b1b41f
author : Peter Eisentraut <peter_e@gmx.net>
date : Mon, 5 Nov 2018 14:52:07 +0100
committer: Peter Eisentraut <peter_e@gmx.net>
date : Mon, 5 Nov 2018 14:52:07 +0100
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 0c3afa7194c2708cf0b1f6f3de858ed69b60abba
M src/backend/po/de.po
M src/backend/po/fr.po
M src/backend/po/it.po
M src/backend/po/ru.po
M src/backend/po/sv.po
M src/bin/initdb/po/fr.po
M src/bin/initdb/po/ru.po
M src/bin/pg_basebackup/po/fr.po
M src/bin/pg_basebackup/po/it.po
M src/bin/pg_basebackup/po/ru.po
M src/bin/pg_controldata/po/fr.po
M src/bin/pg_controldata/po/it.po
M src/bin/pg_controldata/po/ru.po
M src/bin/pg_controldata/po/sv.po
M src/bin/pg_ctl/po/it.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_dump/po/it.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_resetxlog/po/ru.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/it.po
M src/bin/pg_rewind/po/ru.po
M src/bin/psql/po/it.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/sv.po
M src/bin/scripts/po/fr.po
M src/bin/scripts/po/it.po
M src/interfaces/ecpg/preproc/po/it.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/libpq/po/ru.po
Release notes for 11.1, 10.6, 9.6.11, 9.5.15, 9.4.20, 9.3.25.
commit : 94c53b73a7f6a5cae1534c8aa6dbb4ea05c64e00
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 4 Nov 2018 16:57:14 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 4 Nov 2018 16:57:14 -0500
M doc/src/sgml/release-9.3.sgml
M doc/src/sgml/release-9.4.sgml
M doc/src/sgml/release-9.5.sgml
M doc/src/sgml/release-9.6.sgml
Make ts_locale.c's character-type functions cope with UTF-16.
commit : 73dbaed93b460f9ac1cf9b13d591f2cdc4d06aa1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 3 Nov 2018 13:56:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 3 Nov 2018 13:56:10 -0400
On Windows, in UTF8 database encoding, what char2wchar() produces is
UTF16 not UTF32, ie, characters above U+FFFF will be represented by
surrogate pairs. t_isdigit() and siblings did not account for this
and failed to provide a large enough result buffer. That in turn
led to bogus "invalid multibyte character for locale" errors, because
contrary to what you might think from char2wchar()'s documentation,
its Windows code path doesn't cope sanely with buffer overflow.
The solution for t_isdigit() and siblings is pretty clear: provide
a 3-wchar_t result buffer not 2.
char2wchar() also needs some work to provide more consistent, and more
accurately documented, buffer overrun behavior. But that's a bigger job
and it doesn't actually have any immediate payoff, so leave it for later.
Per bug #15476 from Kenji Uno, who deserves credit for identifying the
cause of the problem. Back-patch to all active branches.
Discussion: https://postgr.es/m/15476-4314f480acf0f114@postgresql.org
M src/backend/tsearch/ts_locale.c
Yet further rethinking of build changes for macOS Mojave.
commit : 401202b7988ca75549c6cb6609ac958675f0aeb7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 2 Nov 2018 18:54:00 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 2 Nov 2018 18:54:00 -0400
The solution arrived at in commit e74dd00f5 presumes that the compiler
has a suitable default -isysroot setting ... but further experience
shows that in many combinations of macOS version, XCode version, Xcode
command line tools version, and phase of the moon, Apple's compiler
will *not* supply a default -isysroot value.
We could potentially go back to the approach used in commit 68fc227dd,
but I don't have a lot of faith in the reliability or life expectancy of
that either. Let's just revert to the approach already shipped in 11.0,
namely specifying an -isysroot switch globally. As a partial response to
the concerns raised by Jakob Egger, adjust the contents of Makefile.global
to look like
CPPFLAGS = -isysroot $(PG_SYSROOT) ...
PG_SYSROOT = /path/to/sysroot
This allows overriding the sysroot path at build time in a relatively
painless way.
Add documentation to installation.sgml about how to use the PG_SYSROOT
option. I also took the opportunity to document how to work around
macOS's "System Integrity Protection" feature.
As before, back-patch to all supported versions.
Discussion: https://postgr.es/m/20840.1537850987@sss.pgh.pa.us
M configure
M configure.in
M doc/src/sgml/installation.sgml
M src/Makefile.global.in
M src/template/darwin
docs: adjust simpler language for NULL return from ANY/ALL
commit : f35187b3689f53dab03a560b436640139b602094
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Nov 2018 13:05:30 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Nov 2018 13:05:30 -0400
Adjustment to commit 8610c973ddf1cbf0befc1369d2cf0d56c0efcd0a.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/17406.1541168421@sss.pgh.pa.us
Backpatch-through: 9.3
M doc/src/sgml/func.sgml
GUC: adjust effective_cache_size docs and SQL description
commit : 2a279b6255c2a7a1938b85dc5c9d787b59eefee0
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Nov 2018 09:11:00 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Nov 2018 09:11:00 -0400
Clarify that effective_cache_size is both kernel buffers and shared
buffers.
Reported-by: nat@makarevitch.org
Discussion: https://postgr.es/m/153685164808.22334.15432535018443165207@wrigleys.postgresql.org
Backpatch-through: 9.3
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc.c
Fix some spelling errors in the documentation
commit : 4f67ff17e2b302a97b1a10127d91746d54f4655c
author : Magnus Hagander <magnus@hagander.net>
date : Fri, 2 Nov 2018 13:55:57 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Fri, 2 Nov 2018 13:55:57 +0100
Author: Daniel Gustafsson <daniel@yesql.se>
M doc/src/sgml/libpq.sgml
M doc/src/sgml/lobj.sgml
doc: use simpler language for NULL return from ANY/ALL
commit : f5382cfc7036325867b4d99f156ef7800aa15501
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Nov 2018 08:54:34 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Nov 2018 08:54:34 -0400
Previously the combination of "does not return" and "any row" caused
ambiguity.
Reported-by: KES <kes-kes@yandex.ru>
Discussion: https://postgr.es/m/153701242703.22334.1476830122267077397@wrigleys.postgresql.org
Reviewed-by: David G. Johnston
Backpatch-through: 9.3
M doc/src/sgml/func.sgml
Fix error message typo introduced 691d79a07933.
commit : 87f76f1324b21f3871e24028c609045945fcde91
author : Andres Freund <andres@anarazel.de>
date : Thu, 1 Nov 2018 10:44:29 -0700
committer: Andres Freund <andres@anarazel.de>
date : Thu, 1 Nov 2018 10:44:29 -0700
Reported-By: Michael Paquier
Discussion: https://postgr.es/m/20181101003405.GB1727@paquier.xyz
Backpatch: 9.4-, like the previous commit
M src/backend/replication/slot.c
Disallow starting server with insufficient wal_level for existing slot.
commit : d35fd17cb58db72122912b57c75c8825ebf24285
author : Andres Freund <andres@anarazel.de>
date : Wed, 31 Oct 2018 14:47:41 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 31 Oct 2018 14:47:41 -0700
Previously it was possible to create a slot, change wal_level, and
restart, even if the new wal_level was insufficient for the
slot. That's a problem for both logical and physical slots, because
the necessary WAL records are not generated.
This removes a few tests in newer versions that, somewhat
inexplicably, whether restarting with a too low wal_level worked (a
buggy behaviour!).
Reported-By: Joshua D. Drake
Author: Andres Freund
Discussion: https://postgr.es/m/20181029191304.lbsmhshkyymhw22w@alap3.anarazel.de
Backpatch: 9.4-, where replication slots where introduced
M src/backend/replication/logical/logical.c
M src/backend/replication/slot.c
Fix memory leak in repeated SPGIST index scans.
commit : 558571afc7c42c47243836c411ad86fc6c0577c5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Oct 2018 17:04:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Oct 2018 17:04:43 -0400
spgendscan neglected to pfree all the memory allocated by spgbeginscan.
It's possible to get away with that in most normal queries, since the
memory is allocated in the executor's per-query context which is about
to get deleted anyway; but it causes severe memory leakage during
creation or filling of large exclusion-constraint indexes.
Also, document that amendscan is supposed to free what ambeginscan
allocates. The docs' lack of clarity on that point probably caused this
bug to begin with. (There is discussion of changing that API spec going
forward, but I don't think it'd be appropriate for the back branches.)
Per report from Bruno Wolff. It's been like this since the beginning,
so back-patch to all active branches.
In HEAD, also fix an independent leak caused by commit 2a6368343
(allocating memory during spgrescan instead of spgbeginscan, which
might be all right if it got cleaned up, but it didn't). And do a bit
of code beautification on that commit, too.
Discussion: https://postgr.es/m/20181024012314.GA27428@wolff.to
M doc/src/sgml/indexam.sgml
M src/backend/access/spgist/spgscan.c
Sync our copy of the timezone library with IANA release tzcode2018g.
commit : 407decd3af095e2451e33fbe789b9744da0613d1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Oct 2018 09:47:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Oct 2018 09:47:53 -0400
This patch absorbs an upstream fix to "zic" for a recently-introduced
bug that made it output data that some 32-bit clients couldn't read.
Given the current source data, the bug only manifests in zones with
leap seconds, which we don't generate, so that there's no actual
change in our installed timezone data files from this. Still, in
case somebody uses our copy of "zic" to do something else, it seems
best to apply the fix promptly.
Also, update the README's notes about converting upstream code to
our conventions.
M src/timezone/README
M src/timezone/zic.c
Update time zone data files to tzdata release 2018g.
commit : bb761c6a056dd037e01c6e6f26b357328ae8ed99
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Oct 2018 08:35:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 31 Oct 2018 08:35:50 -0400
DST law changes in Morocco (with, effectively, zero notice).
Historical corrections for Hawaii.
M src/timezone/data/tzdata.zi
Fix missing whitespace in pg_dump ref page
commit : 679ad2f969772297e4725715ea24e5d38526590d
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 29 Oct 2018 12:34:49 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 29 Oct 2018 12:34:49 +0100
Author: Daniel Gustafsson <daniel@yesql.se>
M doc/src/sgml/ref/pg_dump.sgml
Fix perl searchpath for modern perl for MSVC tools
commit : 9fd6d4eae3009658d0d941262f7dec5ca9b877c1
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 28 Oct 2018 12:22:32 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 28 Oct 2018 12:22:32 -0400
Modern versions of perl no longer include the current directory in the
perl searchpath, as it's insecure. Instead of adding the current
directory, we get around the problem by adding the directory where the
script lives.
Problem noted by Victor Wagner.
Solution adapted from buildfarm client code.
Backpatch to all live versions.
M src/tools/msvc/install.pl
M src/tools/msvc/mkvcbuild.pl
M src/tools/msvc/vcregress.pl
Fix some grammar errors in bloom.sgml
commit : 6889428769255253cbbaaf1113474382b59a2f1c
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 22 Oct 2018 00:23:26 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 22 Oct 2018 00:23:26 +0300
Discussion: https://postgr.es/m/CAEepm%3D3sijpGr8tXdyz-7EJJZfhQHABPKEQ29gpnb7-XSy%2B%3D5A%40mail.gmail.com
Reported-by: Thomas Munro
Backpatch-through: 9.6
M doc/src/sgml/bloom.sgml
Lower privilege level of programs calling regression_main
commit : 42a93da25a5ce8e3937eabbf12f2be6837ab7e82
author : Andrew Dunstan <andrew@dunslane.net>
date : Sat, 20 Oct 2018 09:02:36 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sat, 20 Oct 2018 09:02:36 -0400
On Windows this mean that the regression tests can now safely and
successfully run as Administrator, which is useful in situations like
Appveyor. Elsewhere it's a no-op.
Backpatch to 9.5 - this is harder in earlier branches and not worth the
trouble.
Discussion: https://postgr.es/m/650b0c29-9578-8571-b1d2-550d7f89f307@2ndQuadrant.com
M src/test/regress/pg_regress.c
Client-side fixes for delayed NOTIFY receipt.
commit : 34aad21cbf03fd0d49e7322efdd18c9bf692f53d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 19 Oct 2018 22:22:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 19 Oct 2018 22:22:57 -0400
PQnotifies() is defined to just process already-read data, not try to read
any more from the socket. (This is a debatable decision, perhaps, but I'm
hesitant to change longstanding library behavior.) The documentation has
long recommended calling PQconsumeInput() before PQnotifies() to ensure
that any already-arrived message would get absorbed and processed.
However, psql did not get that memo, which explains why it's not very
reliable about reporting notifications promptly.
Also, most (not quite all) callers called PQconsumeInput() just once before
a PQnotifies() loop. Taking this recommendation seriously implies that we
should do PQconsumeInput() before each call. This is more important now
that we have "payload" strings in notification messages than it was before;
that increases the probability of having more than one packet's worth
of notify messages. Hence, adjust code as well as documentation examples
to do it like that.
Back-patch to 9.5 to match related server fixes. In principle we could
probably go back further with these changes, but given lack of field
complaints I doubt it's worthwhile.
Discussion: https://postgr.es/m/CAOYf6ec-TmRYjKBXLLaGaB-jrd=mjG1Hzn1a1wufUAR39PQYhw@mail.gmail.com
M doc/src/sgml/libpq.sgml
M src/bin/psql/common.c
M src/interfaces/ecpg/ecpglib/execute.c
M src/interfaces/libpq/fe-exec.c
M src/test/examples/testlibpq2.c
Server-side fix for delayed NOTIFY and SIGTERM processing.
commit : cbab94077e126bf32d5fb47695d7a43fe7cb60ac
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 19 Oct 2018 21:39:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 19 Oct 2018 21:39:22 -0400
Commit 4f85fde8e introduced some code that was meant to ensure that we'd
process cancel, die, sinval catchup, and notify interrupts while waiting
for client input. But there was a flaw: it supposed that the process
latch would be set upon arrival at secure_read() if any such interrupt
was pending. In reality, we might well have cleared the process latch
at some earlier point while those flags remained set -- particularly
notifyInterruptPending, which can't be handled as long as we're within
a transaction.
To fix the NOTIFY case, also attempt to process signals (except
ProcDiePending) before trying to read.
Also, if we see that ProcDiePending is set before we read, forcibly set the
process latch to ensure that we will handle that signal promptly if no data
is available. I also made it set the process latch on the way out, in case
there is similar logic elsewhere. (It remains true that we won't service
ProcDiePending here unless we need to wait for input.)
The code for handling ProcDiePending during a write needs those changes,
too.
Also be a little more careful about when to reset whereToSendOutput,
and improve related comments.
Back-patch to 9.5 where this code was added. I'm not entirely convinced
that older branches don't have similar issues, but the complaint at hand
is just about the >= 9.5 code.
Jeff Janes and Tom Lane
Discussion: https://postgr.es/m/CAOYf6ec-TmRYjKBXLLaGaB-jrd=mjG1Hzn1a1wufUAR39PQYhw@mail.gmail.com
M src/backend/libpq/be-secure.c
M src/backend/tcop/postgres.c
Sync our copy of the timezone library with IANA release tzcode2018f.
commit : 5dfb53faaeb6d7c4052c1d6eaea5a88364735770
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 19 Oct 2018 19:36:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 19 Oct 2018 19:36:34 -0400
About half of this is purely cosmetic changes to reduce the diff between
our code and theirs, like inserting "const" markers where they have them.
The other half is tracking actual code changes in zic.c and localtime.c.
I don't think any of these represent near-term compatibility hazards, but
it seems best to stay up to date.
I also fixed longstanding bugs in our code for producing the
known_abbrevs.txt list, which by chance hadn't been exposed before,
but which resulted in some garbage output after applying the upstream
changes in zic.c. Notably, because upstream removed their old phony
transitions at the Big Bang, it's now necessary to cope with TZif files
containing no DST transition times at all.
M src/timezone/README
M src/timezone/localtime.c
M src/timezone/pgtz.h
M src/timezone/private.h
M src/timezone/strftime.c
M src/timezone/tzfile.h
M src/timezone/zic.c
Update time zone data files to tzdata release 2018f.
commit : 185f135c91e907e7ac27a0e1d1ef27e2c1d189e7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 19 Oct 2018 17:01:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 19 Oct 2018 17:01:34 -0400
DST law changes in Chile, Fiji, and Russia (Volgograd).
Historical corrections for China, Japan, Macau, and North Korea.
Note: like the previous tzdata update, this involves a depressingly
large amount of semantically-meaningless churn in tzdata.zi. That
is a consequence of upstream's data compression method assigning
unstable abbreviations to DST rulesets. I complained about that
to them last time, and this version now uses an assignment method
that pays some heed to not changing abbreviations unnecessarily.
So hopefully, that'll be better going forward.
M src/timezone/data/tzdata.zi
M src/timezone/known_abbrevs.txt
M src/timezone/tznames/America.txt
M src/timezone/tznames/Asia.txt
M src/timezone/tznames/Default
M src/timezone/tznames/Pacific.txt
Still further rethinking of build changes for macOS Mojave.
commit : 1b92ca9e2c22fac54e4f77c80b6bbe8469ed472d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Oct 2018 14:55:23 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Oct 2018 14:55:23 -0400
To avoid the sorts of problems complained of by Jakob Egger, it'd be
best if configure didn't emit any references to the sysroot path at all.
In the case of PL/Tcl, we can do that just by keeping our hands off the
TCL_INCLUDE_SPEC string altogether. In the case of PL/Perl, we need to
substitute -iwithsysroot for -I in the compile commands, which is easily
handled if we change to using a configure output variable that includes
the switch not only the directory name. Since PL/Tcl and PL/Python
already do it like that, this seems like good consistency cleanup anyway.
Hence, this replaces the advice given to Perl-related extensions in commit
5e2217131; instead of writing "-I$(perl_archlibexp)/CORE", they should
just write "$(perl_includespec)". (The old way continues to work, but not
on recent macOS.)
It's still the case that configure needs to be aware of the sysroot
path internally, but that's cleaner than what we had before.
As before, back-patch to all supported versions.
Discussion: https://postgr.es/m/20840.1537850987@sss.pgh.pa.us
M configure
M configure.in
M contrib/hstore_plperl/Makefile
M src/Makefile.global.in
M src/pl/plperl/GNUmakefile
M src/template/darwin
Fix minor bug in isolationtester.
commit : 4c68883891a560f0e4a234bd207a8e8bf43fd08f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Oct 2018 15:06:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Oct 2018 15:06:38 -0400
If the lock wait query failed, isolationtester would report the
PQerrorMessage from some other connection, meaning there would be
no message or an unrelated one. This seems like a pretty unlikely
occurrence, but if it did happen, this bug could make it really
difficult/confusing to figure out what happened. That seems to
justify patching all the way back.
In passing, clean up another place where the "wrong" conn was used
for an error report. That one's not actually buggy because it's
a different alias for the same connection, but it's still confusing
to the reader.
M src/test/isolation/isolationtester.c
Improve tzparse's handling of TZDEFRULES ("posixrules") zone data.
commit : 1164389e12b0009e510501d1e7fd1b1025d15734
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Oct 2018 12:26:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Oct 2018 12:26:48 -0400
In the IANA timezone code, tzparse() always tries to load the zone
file named by TZDEFRULES ("posixrules"). Previously, we'd hacked
that logic to skip the load in the "lastditch" code path, which we use
only to initialize the default "GMT" zone during GUC initialization.
That's critical for a couple of reasons: since we do not support leap
seconds, we *must not* allow "GMT" to have leap seconds, and since this
case runs before the GUC subsystem is fully alive, we'd really rather
not take the risk of pg_open_tzfile throwing any errors.
However, that still left the code reading TZDEFRULES on every other
call, something we'd noticed to the extent of having added code to cache
the result so it was only done once per process not a lot of times.
Andres Freund complained about the static data space used up for the
cache; but as long as the logic was like this, there was no point in
trying to get rid of that space.
We can improve matters by looking a bit more closely at what the IANA
code actually needs the TZDEFRULES data for. One thing it does is
that if "posixrules" is a leap-second-aware zone, the leap-second
behavior will be absorbed into every POSIX-style zone specification.
However, that's a behavior we'd really prefer to do without, since
for our purposes the end effect is to render every POSIX-style zone
name unsupported. Otherwise, the TZDEFRULES data is used only if
the POSIX zone name specifies DST but doesn't include a transition
date rule (e.g., "EST5EDT" rather than "EST5EDT,M3.2.0,M11.1.0").
That is a minority case for our purposes --- in particular, it
never happens when tzload() invokes tzparse() to interpret a
transition date rule string found in a tzdata zone file.
Hence, if we legislate that we're going to ignore leap-second data
from "posixrules", we can postpone the TZDEFRULES load into the path
where we actually need to substitute for a missing date rule string.
That means it will never happen at all in common scenarios, making it
reasonable to dynamically allocate the cache space when it does happen.
Even when the data is already loaded, this saves some cycles in the
common code path since we avoid a memcpy of 23KB or so. And, IMO at
least, this is a less ugly hack on the IANA logic than what we had
before, since it's not messing with the lastditch-vs-regular code paths.
Back-patch to all supported branches, not so much because this is a
critical change as that I want to keep all our copies of the IANA
timezone code in sync.
Discussion: https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
M src/timezone/README
M src/timezone/localtime.c
Back off using -isysroot on Darwin.
commit : 5777a9ff8f217401986c66e20415e02e558e5b44
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Oct 2018 16:27:15 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Oct 2018 16:27:15 -0400
Rethink the solution applied in commit 5e2217131 to get PL/Tcl to
build on macOS Mojave. I feared that adding -isysroot globally might
have undesirable consequences, and sure enough Jakob Egger reported
one: it complicates building extensions with a different Xcode version
than was used for the core server. (I find that a risky proposition
in general, but apparently it works most of the time, so we shouldn't
break it if we don't have to.)
We'd already adopted the solution for PL/Perl of inserting the sysroot
path directly into the -I switches used to find Perl's headers, and we
can do the same thing for PL/Tcl by changing the -iwithsysroot switch
that Apple's tclConfig.sh reports. This restricts the risks to PL/Perl
and PL/Tcl themselves and directly-dependent extensions, which is a lot
more pleasing in general than a global -isysroot switch.
Along the way, tighten the test to see if we need to inject the sysroot
path into $perl_includedir, as I'd speculated about upthread but not
gotten round to doing.
As before, back-patch to all supported versions.
Discussion: https://postgr.es/m/20840.1537850987@sss.pgh.pa.us
M configure
M configure.in
M src/template/darwin
Avoid rare race condition in privileges.sql regression test.
commit : 75b3b137009991b996d56e91648cfacd1ed9f66a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Oct 2018 13:56:58 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Oct 2018 13:56:58 -0400
We created a temp table, then switched to a new session, leaving
the old session to clean up its temp objects in background. If that
took long enough, the eventual attempt to drop the user that owns
the temp table could fail, as exhibited today by sidewinder.
Fix by dropping the temp table explicitly when we're done with it.
It's been like this for quite some time, so back-patch to all
supported branches.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=sidewinder&dt=2018-10-16%2014%3A45%3A00
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Fix mis-backpatch of c015ccb306ec81bca3023818c9cf0113cae25be1.
commit : a485bacef177b4cd476b9437f8fb6cb8dce58944
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Oct 2018 13:24:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Oct 2018 13:24:42 -0400
Per buildfarm.
M src/test/perl/PostgresNode.pm
Make PostgresNode.pm's poll_query_until() more chatty about failures.
commit : 0ed4a47fbfbd0f22806dc14f70dcc42de07ce9f1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Oct 2018 12:27:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Oct 2018 12:27:14 -0400
Reporting only the stderr is unhelpful when the problem is that the
server output we're getting doesn't match what was expected. So we
should report the query output too; and just for good measure, let's
print the query we used and the output we expected.
Back-patch to 9.5 where poll_query_until was introduced.
Discussion: https://postgr.es/m/17913.1539634756@sss.pgh.pa.us
M src/test/perl/PostgresNode.pm
Improve stability of recently-added regression test case.
commit : 3e406417b8da9fe7139374f35cab5f9d55881760
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Oct 2018 12:01:19 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Oct 2018 12:01:19 -0400
Commit b5febc1d1 added a contrib/btree_gist test case that has been
observed to fail in the buildfarm as a result of background auto-analyze
updating stats and changing the selected plan. Forestall that by
forcibly analyzing in foreground, instead. The new plan choice is
just as good for our purposes, since we really only care that an
index-only plan does not get selected.
Back-patch to 9.5, like the previous patch.
Discussion: https://postgr.es/m/14643.1539629304@sss.pgh.pa.us
M contrib/btree_gist/expected/inet.out
M contrib/btree_gist/sql/inet.sql
Avoid statically allocating gmtsub()'s timezone workspace.
commit : 5ef71d4bd3226bfd4e733a2fc0b6598df0cc26cf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Oct 2018 11:50:19 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Oct 2018 11:50:19 -0400
localtime.c's "struct state" is a rather large object, ~23KB. We were
statically allocating one for gmtsub() to use to represent the GMT
timezone, even though that function is not at all heavily used and is
never reached in most backends. Let's malloc it on-demand, instead.
This does pose the question of how to handle a malloc failure, but
there's already a well-defined error report convention here, ie
set errno and return NULL.
We have but one caller of pg_gmtime in HEAD, and two in back branches,
neither of which were troubling to check for error. Make them do so.
The possible errors are sufficiently unlikely (out-of-range timestamp,
and now malloc failure) that I think elog() is adequate.
Back-patch to all supported branches to keep our copies of the IANA
timezone code in sync. This particular change is in a stanza that
already differs from upstream, so it's a wash for maintenance purposes
--- but only as long as we keep the branches the same.
Discussion: https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
M src/backend/utils/adt/nabstime.c
M src/backend/utils/adt/timestamp.c
M src/timezone/localtime.c
Check for stack overrun in standard_ProcessUtility().
commit : ca361554c24e51eed38c8975780fa7dd98d760ed
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 15 Oct 2018 14:01:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 15 Oct 2018 14:01:38 -0400
ProcessUtility can recurse, and indeed can be driven to infinite
recursion, so it ought to have a check_stack_depth() call. This
covers the reported bug (portal trying to execute itself) and a bunch
of other cases that could perhaps arise somewhere.
Per bug #15428 from Malthe Borch. Back-patch to all supported branches.
Discussion: https://postgr.es/m/15428-b3c2915ec470b033@postgresql.org
M src/backend/tcop/utility.c
contrib/bloom documentation improvement
commit : a05f7053e842d1463c35713ae79e6795b8751dc2
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 15 Oct 2018 00:40:17 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Mon, 15 Oct 2018 00:40:17 +0300
This commit documents rounding of "length" parameter and absence of support
for unique indexes and NULLs searching. Backpatch to 9.6 where contrib/bloom
was introduced.
Discussion: https://postgr.es/m/CAF4Au4wPQQ7EHVSnzcLjsbY3oLSzVk6UemZLD1Sbmwysy3R61g%40mail.gmail.com
Author: Oleg Bartunov with minor editorialization by me
Backpatch-through: 9.6
M doc/src/sgml/bloom.sgml
Avoid duplicate XIDs at recovery when building initial snapshot
commit : 010041ddcc39323ed786fc0e0b94789abdf430e9
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 14 Oct 2018 22:23:43 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 14 Oct 2018 22:23:43 +0900
On a primary, sets of XLOG_RUNNING_XACTS records are generated on a
periodic basis to allow recovery to build the initial state of
transactions for a hot standby. The set of transaction IDs is created
by scanning all the entries in ProcArray. However it happens that its
logic never counted on the fact that two-phase transactions finishing to
prepare can put ProcArray in a state where there are two entries with
the same transaction ID, one for the initial transaction which gets
cleared when prepare finishes, and a second, dummy, entry to track that
the transaction is still running after prepare finishes. This way
ensures a continuous presence of the transaction so as callers of for
example TransactionIdIsInProgress() are always able to see it as alive.
So, if a XLOG_RUNNING_XACTS takes a standby snapshot while a two-phase
transaction finishes to prepare, the record can finish with duplicated
XIDs, which is a state expected by design. If this record gets applied
on a standby to initial its recovery state, then it would simply fail,
so the odds of facing this failure are very low in practice. It would
be tempting to change the generation of XLOG_RUNNING_XACTS so as
duplicates are removed on the source, but this requires to hold on
ProcArrayLock for longer and this would impact all workloads,
particularly those using heavily two-phase transactions.
XLOG_RUNNING_XACTS is also actually used only to initialize the standby
state at recovery, so instead the solution is taken to discard
duplicates when applying the initial snapshot.
Diagnosed-by: Konstantin Knizhnik
Author: Michael Paquier
Discussion: https://postgr.es/m/0c96b653-4696-d4b4-6b5d-78143175d113@postgrespro.ru
Backpatch-through: 9.3
M src/backend/storage/ipc/procarray.c
Remove abstime, reltime, tinterval tables from old regression databases.
commit : 2ad422ce10a4c84cea2fede1cdfc6831940707f0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Oct 2018 19:33:56 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Oct 2018 19:33:56 -0400
In the back branches, drop these tables after the regression tests are
done with them. This fixes failures of cross-branch pg_upgrade testing
caused by these types having been removed in v12. We do lose the ability
to test dump/restore behavior with these types in the back branches, but
the actual loss of code coverage seems to be nil given that there's nothing
very special about these types.
Discussion: https://postgr.es/m/20181009192237.34wjp3nmw7oynmmr@alap3.anarazel.de
M src/test/regress/expected/horology.out
M src/test/regress/expected/sanity_check.out
M src/test/regress/sql/horology.sql
Fix logical decoding error when system table w/ toast is repeatedly rewritten.
commit : a88482dd24aafeca555cf80aa58cf9ae39f25f9d
author : Andres Freund <andres@anarazel.de>
date : Wed, 10 Oct 2018 13:53:02 -0700
committer: Andres Freund <andres@anarazel.de>
date : Wed, 10 Oct 2018 13:53:02 -0700
Repeatedly rewriting a mapped catalog table with VACUUM FULL or
CLUSTER could cause logical decoding to fail with:
ERROR, "could not map filenode \"%s\" to relation OID"
To trigger the problem the rewritten catalog had to have live tuples
with toasted columns.
The problem was triggered as during catalog table rewrites the
heap_insert() check that prevents logical decoding information to be
emitted for system catalogs, failed to treat the new heap's toast table
as a system catalog (because the new heap is not recognized as a
catalog table via RelationIsLogicallyLogged()). The relmapper, in
contrast to the normal catalog contents, does not contain historical
information. After a single rewrite of a mapped table the new relation
is known to the relmapper, but if the table is rewritten twice before
logical decoding occurs, the relfilenode cannot be mapped to a
relation anymore. Which then leads us to error out. This only
happens for toast tables, because the main table contents aren't
re-inserted with heap_insert().
The fix is simple, add a new heap_insert() flag that prevents logical
decoding information from being emitted, and accept during decoding
that there might not be tuple data for toast tables.
Unfortunately that does not fix pre-existing logical decoding
errors. Doing so would require not throwing an error when a filenode
cannot be mapped to a relation during decoding, and that seems too
likely to hide bugs. If it's crucial to fix decoding for an existing
slot, temporarily changing the ERROR in ReorderBufferCommit() to a
WARNING appears to be the best fix.
Author: Andres Freund
Discussion: https://postgr.es/m/20180914021046.oi7dm4ra3ot2g2kt@alap3.anarazel.de
Backpatch: 9.4-, where logical decoding was introduced
M contrib/test_decoding/expected/rewrite.out
M contrib/test_decoding/sql/rewrite.sql
M src/backend/access/heap/heapam.c
M src/backend/access/heap/rewriteheap.c
M src/backend/replication/logical/reorderbuffer.c
M src/include/access/heapam.h
Silence compiler warning in Assert()
commit : a653569c14c3a0b7f48a874a2770d58ce39e07d0
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 8 Oct 2018 10:37:21 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 8 Oct 2018 10:37:21 -0300
gcc 6.3 does not whine about this mistake I made in 39808e8868c8 but
evidently lots of other compilers do, according to Michael Paquier,
Peter Eisentraut, Arthur Zakirov, Tomas Vondra.
Discussion: too many to list
M src/backend/commands/event_trigger.c
Fix event triggers for partitioned tables
commit : b2f266f58d4c24b372b3413e75acce9ff32e126a
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 6 Oct 2018 19:17:46 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 6 Oct 2018 19:17:46 -0300
Index DDL cascading on partitioned tables introduced a way for ALTER
TABLE to be called reentrantly. This caused an an important deficiency
in event trigger support to be exposed: on exiting the reentrant call,
the alter table state object was clobbered, causing a crash when the
outer alter table tries to finalize its processing. Fix the crash by
creating a stack of event trigger state objects. There are still ways
to cause things to misbehave (and probably other crashers) with more
elaborate tricks, but at least it now doesn't crash in the obvious
scenario.
Backpatch to 9.5, where DDL deparsing of event triggers was introduced.
Reported-by: Marco Slot
Authors: Michaël Paquier, Álvaro Herrera
Discussion: https://postgr.es/m/CANNhMLCpi+HQ7M36uPfGbJZEQLyTy7XvX=5EFkpR-b1bo0uJew@mail.gmail.com
M src/backend/catalog/index.c
M src/backend/commands/event_trigger.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/commands/view.c
M src/include/catalog/index.h
M src/include/tcop/deparse_utility.h
Propagate xactStartTimestamp and stmtStartTimestamp to parallel workers.
commit : bdc2e7a19af7066ba5688fa0e0d3e6c03558c0c0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 6 Oct 2018 12:00:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 6 Oct 2018 12:00:10 -0400
Previously, a worker process would establish values for these based on
its own start time. In v10 and up, this can trivially be shown to cause
misbehavior of transaction_timestamp(), timestamp_in(), and related
functions which are (perhaps unwisely?) marked parallel-safe. It seems
likely that other behaviors might diverge from what happens in the parent
as well.
It's not as trivial to demonstrate problems in 9.6 or 9.5, but I'm sure
it's still possible, so back-patch to all branches containing parallel
worker infrastructure.
In HEAD only, mark now() and statement_timestamp() as parallel-safe
(other affected functions already were). While in theory we could
still squeeze that change into v11, it doesn't seem important enough
to force a last-minute catversion bump.
Konstantin Knizhnik, whacked around a bit by me
Discussion: https://postgr.es/m/6406dbd2-5d37-4cb6-6eb2-9c44172c7e7c@postgrespro.ru
M src/backend/access/transam/parallel.c
M src/backend/access/transam/xact.c
M src/include/access/xact.h
Allow btree comparison functions to return INT_MIN.
commit : 60cc2414beac4749e0846ce7ce3f8bead20c0745
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Oct 2018 16:01:30 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Oct 2018 16:01:30 -0400
Historically we forbade datatype-specific comparison functions from
returning INT_MIN, so that it would be safe to invert the sort order
just by negating the comparison result. However, this was never
really safe for comparison functions that directly return the result
of memcmp(), strcmp(), etc, as POSIX doesn't place any such restriction
on those library functions. Buildfarm results show that at least on
recent Linux on s390x, memcmp() actually does return INT_MIN sometimes,
causing sort failures.
The agreed-on answer is to remove this restriction and fix relevant
call sites to not make such an assumption; code such as "res = -res"
should be replaced by "INVERT_COMPARE_RESULT(res)". The same is needed
in a few places that just directly negated the result of memcmp or
strcmp.
To help find places having this problem, I've also added a compile option
to nbtcompare.c that causes some of the commonly used comparators to
return INT_MIN/INT_MAX instead of their usual -1/+1. It'd likely be
a good idea to have at least one buildfarm member running with
"-DSTRESS_SORT_INT_MIN". That's far from a complete test of course,
but it should help to prevent fresh introductions of such bugs.
This is a longstanding portability hazard, so back-patch to all supported
branches.
Discussion: https://postgr.es/m/20180928185215.ffoq2xrq5d3pafna@alap3.anarazel.de
M contrib/ltree/ltree_op.c
M contrib/pgcrypto/imath.c
M src/backend/access/nbtree/nbtcompare.c
M src/backend/access/nbtree/nbtsearch.c
M src/backend/access/nbtree/nbtutils.c
M src/backend/executor/nodeIndexscan.c
M src/backend/executor/nodeMergeAppend.c
M src/bin/pg_rewind/filemap.c
M src/include/access/nbtree.h
M src/include/c.h
M src/include/utils/sortsupport.h
MAXALIGN the target address where we store flattened value.
commit : dca44d07c585637d8245a46a29be732241fa40bf
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 3 Oct 2018 09:54:01 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 3 Oct 2018 09:54:01 +0530
The API (EOH_flatten_into) that flattens the expanded value representation
expects the target address to be maxaligned. All it's usage adhere to that
principle except when serializing datums for parallel query. Fix that
usage.
Diagnosed-by: Tom Lane
Author: Tom Lane and Amit Kapila
Backpatch-through: 9.6
Discussion: https://postgr.es/m/11629.1536550032@sss.pgh.pa.us
M src/backend/utils/adt/datum.c
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql
Set snprintf.c's maximum number of NL arguments to be 31.
commit : d5895717e000f3b6a89a9e290a47e9edb64eea28
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Oct 2018 12:41:28 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Oct 2018 12:41:28 -0400
Previously, we used the platform's NL_ARGMAX if any, otherwise 16.
The trouble with this is that the platform value is hugely variable,
ranging from the POSIX-minimum 9 to as much as 64K on recent FreeBSD.
Values of more than a dozen or two have no practical use and slow down
the initialization of the argtypes array. Worse, they cause snprintf.c
to consume far more stack space than was the design intention, possibly
resulting in stack-overflow crashes.
Standardize on 31, which is comfortably more than we need (it looks like
no existing translatable message has more than about 10 parameters).
I chose that, not 32, to make the array sizes powers of 2, for some
possible small gain in speed of the memset.
The lack of reported crashes suggests that the set of platforms we
use snprintf.c on (in released branches) may have no overlap with
the set where NL_ARGMAX has unreasonably large values. But that's
not entirely clear, so back-patch to all supported branches.
Per report from Mateusz Guzik (via Thomas Munro).
Discussion: https://postgr.es/m/CAEepm=3VF=PUp2f8gU8fgZB22yPE_KBS0+e1AHAtQ=09schTHg@mail.gmail.com
M src/port/snprintf.c
Fix corner-case failures in has_foo_privilege() family of functions.
commit : 6d73983be61adde122b876308dbc6b6d52113419
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Oct 2018 11:54:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 2 Oct 2018 11:54:12 -0400
The variants of these functions that take numeric inputs (OIDs or
column numbers) are supposed to return NULL rather than failing
on bad input; this rule reduces problems with snapshot skew when
queries apply the functions to all rows of a catalog.
has_column_privilege() had careless handling of the case where the
table OID didn't exist. You might get something like this:
select has_column_privilege(9999,'nosuchcol','select');
ERROR: column "nosuchcol" of relation "(null)" does not exist
or you might get a crash, depending on the platform's printf's response
to a null string pointer.
In addition, while applying the column-number variant to a dropped
column returned NULL as desired, applying the column-name variant
did not:
select has_column_privilege('mytable','........pg.dropped.2........','select');
ERROR: column "........pg.dropped.2........" of relation "mytable" does not exist
It seems better to make this case return NULL as well.
Also, the OID-accepting variants of has_foreign_data_wrapper_privilege,
has_server_privilege, and has_tablespace_privilege didn't follow the
principle of returning NULL for nonexistent OIDs. Superusers got TRUE,
everybody else got an error.
Per investigation of Jaime Casanova's report of a new crash in HEAD.
These behaviors have been like this for a long time, so back-patch to
all supported branches.
Patch by me; thanks to Stephen Frost for discussion and review
Discussion: https://postgr.es/m/CAJGNTeP=-6Gyqq5TN9OvYEydi7Fv1oGyYj650LGTnW44oAzYCg@mail.gmail.com
M src/backend/utils/adt/acl.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Fix documentation of pgrowlocks using "lock_type" instead of "modes"
commit : 0353f48f0e7c85bc1a053710ed42ab23123b0b7c
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 2 Oct 2018 16:36:11 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 2 Oct 2018 16:36:11 +0900
The example used in the documentation is outdated as well. This is an
oversight from 0ac5ad5, which bumped up pgrowlocks but forgot some bits
of the documentation.
Reported-by: Chris Wilson
Discussion: https://postgr.es/m/153838692816.2950.12001142346234155699@wrigleys.postgresql.org
Backpatch-through: 9.3
M doc/src/sgml/pgrowlocks.sgml
Fix tuple_data_split() to not open a relation without any lock.
commit : ec5f71aeadf977399707a54e4798aedc963c85e7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Oct 2018 11:51:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Oct 2018 11:51:07 -0400
contrib/pageinspect's tuple_data_split() function thought it could get
away with opening the referenced relation with NoLock. In practice
there's no guarantee that the current session holds any lock on that
rel (even if we just read a page from it), so that this is unsafe.
Switch to using AccessShareLock. Also, postpone closing the relation,
so that we needn't copy its tupdesc. Also, fix unsafe use of
att_isnull() for attributes past the end of the tuple.
Per testing with a patch that complains if we open a relation without
holding any lock on it. I don't plan to back-patch that patch, but we
should close the holes it identifies in all supported branches.
Discussion: https://postgr.es/m/2038.1538335244@sss.pgh.pa.us
M contrib/pageinspect/heapfuncs.c
Fix ALTER COLUMN TYPE to not open a relation without any lock.
commit : 0360c539f2a2a979a98b9bb4ae4b3d8b77ee199b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Oct 2018 11:39:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 1 Oct 2018 11:39:14 -0400
If the column being modified is referenced by a foreign key constraint
of another table, ALTER TABLE would open the other table (to re-parse
the constraint's definition) without having first obtained a lock on it.
This was evidently intentional, but that doesn't mean it's really safe.
It's especially not safe in 9.3, which pre-dates use of MVCC scans for
catalog reads, but even in current releases it doesn't seem like a good
idea.
We know we'll need AccessExclusiveLock shortly to drop the obsoleted
constraint, so just get that a little sooner to close the hole.
Per testing with a patch that complains if we open a relation without
holding any lock on it. I don't plan to back-patch that patch, but we
should close the holes it identifies in all supported branches.
Discussion: https://postgr.es/m/2038.1538335244@sss.pgh.pa.us
M src/backend/commands/tablecmds.c
Fix detection of the result type of strerror_r().
commit : 2855421ec728ef5f871c765390ab432ffa6ec8a6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 30 Sep 2018 16:24:56 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 30 Sep 2018 16:24:56 -0400
The method we've traditionally used, of redeclaring strerror_r() to
see if the compiler complains of inconsistent declarations, turns out
not to work reliably because some compilers only report a warning,
not an error. Amazingly, this has gone undetected for years, even
though it certainly breaks our detection of whether strerror_r
succeeded.
Let's instead test whether the compiler will take the result of
strerror_r() as a switch() argument. It's possible this won't
work universally either, but it's the best idea I could come up with
on the spur of the moment.
Back-patch of commit 751f532b9. Buildfarm results indicate that only
icc-on-Linux actually has an issue here; perhaps the lack of field
reports indicates that people don't build PG for production that way.
Discussion: https://postgr.es/m/10877.1537993279@sss.pgh.pa.us
M config/c-library.m4
M configure
M src/include/pg_config.h.in
M src/include/pg_config.h.win32
Fix assertion failure when updating full_page_writes for checkpointer.
commit : e315bd7db96a8cd41207cec1f036afffb282640e
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 28 Sep 2018 12:45:00 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 28 Sep 2018 12:45:00 +0530
When the checkpointer receives a SIGHUP signal to update its configuration,
it may need to update the shared memory for full_page_writes and need to
write a WAL record for it. Now, it is quite possible that the XLOG
machinery has not been initialized by that time and it will lead to
assertion failure while doing that. Fix is to allow the initialization of
the XLOG machinery outside critical section.
This bug has been introduced by the commit 2c03216d83 which added the XLOG
machinery initialization in RecoveryInProgress code path.
Reported-by: Dilip Kumar
Author: Dilip Kumar
Reviewed-by: Michael Paquier and Amit Kapila
Backpatch-through: 9.5
Discussion: https://postgr.es/m/CAFiTN-u4BA8KXcQUWDPNgaKAjDXC=C2whnzBM8TAcv=stckYUw@mail.gmail.com
M src/backend/access/transam/xlog.c
Fix WAL recycling on standbys depending on archive_mode
commit : f4fa92f267faf655d67f840bce4b8b4c9a8ae7ad
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 28 Sep 2018 11:56:04 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 28 Sep 2018 11:56:04 +0900
A restart point or a checkpoint recycling WAL segments treats segments
marked with neither ".done" (archiving is done) or ".ready" (segment is
ready to be archived) in archive_status the same way for archive_mode
being "on" or "always". While for a primary this is fine, a standby
running a restart point with archive_mode = on would try to mark such a
segment as ready for archiving, which is something that will never
happen except after the standby is promoted.
Note that this problem applies only to WAL segments coming from the
local pg_wal the first time archive recovery is run. Segments part of a
self-contained base backup are the most common case where this could
happen, however even in this case normally the .done markers would be
most likely part of the backup. Segments recovered from an archive are
marked as .ready or .done by the startup process, and segments finished
streaming are marked as such by the WAL receiver, so they are handled
already.
Reported-by: Haruka Takatsuka
Author: Michael Paquier
Discussion: https://postgr.es/m/15402-a453c90ed4cf88b2@postgresql.org
Backpatch-through: 9.5, where archive_mode = always has been added.
M src/backend/access/transam/xlogarchive.c
Recurse to sequences on ownership change for all relkinds
commit : bdf11d6889610061b2723496bf759f76a136f6f3
author : Peter Eisentraut <peter_e@gmx.net>
date : Thu, 14 Jun 2018 23:22:14 -0400
committer: Peter Eisentraut <peter_e@gmx.net>
date : Thu, 14 Jun 2018 23:22:14 -0400
When a table ownership is changed, we must apply that also to any owned
sequences. (Otherwise, it would result in a situation that cannot be
restored, because linked sequences must have the same owner as the
table.) But this was previously only applied to regular tables and
materialized views. But it should also apply to at least foreign
tables. This patch removes the relkind check altogether, because it
doesn't save very much and just introduces the possibility of similar
omissions.
Bug: #15238
Reported-by: Christoph Berg <christoph.berg@credativ.de>
M src/backend/commands/tablecmds.c
Rework activation of commit timestamps during recovery
commit : e513a3d854a13c141e7d73e209ad44d789352479
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Sep 2018 10:29:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 26 Sep 2018 10:29:49 +0900
The activation and deactivation of commit timestamp tracking has not
been handled consistently for a primary or standbys at recovery. The
facility can be activated at three different moments of recovery:
- The beginning, where a primary would use the GUC value for the
decision-making, and where a standby relies on the contents of the
control file.
- When replaying a XLOG_PARAMETER_CHANGE record at redo.
- The end, where both primary and standby rely on the GUC value.
Using the GUC value for a primary at the beginning of recovery causes
problems with commit timestamp access when doing crash recovery.
Particularly, when replaying transaction commits, it could be possible
that an attempt to read commit timestamps is done for a transaction
which committed at a moment when track_commit_timestamp was disabled.
A test case is added to reproduce the failure. The test works down to
v11 as it takes advantage of transaction commits within procedures.
Reported-by: Hailong Li
Author: Masahiko Sawasa, Michael Paquier
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/11224478-a782-203b-1f17-e4797b39bdf0@qunar.com
Backpatch-through: 9.5, where commit timestamps have been introduced.
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/xlog.c
Make some fixes to allow building Postgres on macOS 10.14 ("Mojave").
commit : 0a4456a70dfa004b2904c6a315dd32f044336bcc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 25 Sep 2018 13:23:29 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 25 Sep 2018 13:23:29 -0400
Apple's latest rearrangements of the system-supplied headers have broken
building of PL/Perl and PL/Tcl. The only practical way to fix PL/Tcl is to
start using the "-isysroot" compiler flag to point to SDK-supplied headers,
as Apple expects. We must also start distinguishing where to find Perl's
headers from where to find its shared library; but that seems like good
cleanup anyway.
Extensions that formerly did something like -I$(perl_archlibexp)/CORE
should now do -I$(perl_includedir)/CORE instead. perl_archlibexp
is still the place to look for libperl.so, though.
If for some reason you don't like the default -isysroot setting, you can
override that by setting PG_SYSROOT in configure's arguments. I don't
currently think people would need to do so, unless maybe for cross-version
build purposes.
In addition, teach configure where to find tclConfig.sh. Our traditional
method of searching $auto_path hasn't worked for the last couple of macOS
releases, and it now seems clear that Apple's not going to change that.
The workaround of manually specifying --with-tclconfig was annoying
already, but Mojave's made it a lot more so because the sysroot path now
has to be included as well. Let's just wire the knowledge into configure
instead. To avoid breaking builds against non-default Tcl installations
(e.g. MacPorts) wherein the $auto_path method probably still works,
arrange to try the additional case only after all else has failed.
Back-patch to all supported versions, since at least the buildfarm
cares about that. The changes are set up to not do anything on macOS
releases that are old enough to not have functional sysroot trees.
M config/tcl.m4
M configure
M configure.in
M contrib/hstore_plperl/Makefile
M src/Makefile.global.in
M src/pl/plperl/GNUmakefile
M src/template/darwin
Fix over-allocation of space for array_out()'s result string.
commit : ac863108f56b721a63df400086ab26c63b61dc21
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 24 Sep 2018 11:30:51 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 24 Sep 2018 11:30:51 -0400
array_out overestimated the space needed for its output, possibly by
a very substantial amount if the array is multi-dimensional, because
of wrong order of operations in the loop that counts the number of
curly-brace pairs needed. While the output string is normally
short-lived, this could still cause problems in extreme cases.
An additional minor error was that it counted one more delimiter than
is actually needed.
Repair those errors, add an Assert that the space is now correctly
calculated, and make some minor improvements in the comments.
I also failed to resist the temptation to get rid of an integer
modulus operation per array element; a simple comparison is sufficient.
This bug dates clear back to Berkeley days, so back-patch to all
supported versions.
Keiichi Hirobe, minor additional work by me
Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
M src/backend/utils/adt/arrayfuncs.c
Initialize random() in bootstrap/stand-alone postgres and in initdb.
commit : 329cacb902705c44c66098535c61f8e84f6f6ee0
author : Noah Misch <noah@leadboat.com>
date : Sun, 23 Sep 2018 22:56:39 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 23 Sep 2018 22:56:39 -0700
This removes a difference between the standard IsUnderPostmaster
execution environment and that of --boot and --single. In a stand-alone
backend, "SELECT random()" always started at the same seed.
On a system capable of using posix shared memory, initdb could still
conclude "selecting dynamic shared memory implementation ... sysv".
Crashed --boot or --single postgres processes orphaned shared memory
objects having names that collided with the not-actually-random names
that initdb probed. The sysv fallback appeared after ten crashes of
--boot or --single postgres. Since --boot and --single are rare in
production use, systems used for PostgreSQL development are the
principal candidate to notice this symptom.
Back-patch to 9.3 (all supported versions). PostgreSQL 9.4 introduced
dynamic shared memory, but 9.3 does share the "SELECT random()" problem.
Reviewed by Tom Lane and Kyotaro HORIGUCHI.
Discussion: https://postgr.es/m/20180915221546.GA3159382@rfd.leadboat.com
M src/backend/utils/init/miscinit.c
M src/bin/initdb/initdb.c
Fix failure in WHERE CURRENT OF after rewinding the referenced cursor.
commit : 77d2a48660a2bad63654c01fdbdebc1b18379520
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 Sep 2018 16:05:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 23 Sep 2018 16:05:45 -0400
In a case where we have multiple relation-scan nodes in a cursor plan,
such as a scan of an inheritance tree, it's possible to fetch from a
given scan node, then rewind the cursor and fetch some row from an
earlier scan node. In such a case, execCurrent.c mistakenly thought
that the later scan node was still active, because ExecReScan hadn't
done anything to make it look not-active. We'd get some sort of
failure in the case of a SeqScan node, because the node's scan tuple
slot would be pointing at a HeapTuple whose t_self gets reset to
invalid by heapam.c. But it seems possible that for other relation
scan node types we'd actually return a valid tuple TID to the caller,
resulting in updating or deleting a tuple that shouldn't have been
considered current. To fix, forcibly clear the ScanTupleSlot in
ExecScanReScan.
Another issue here, which seems only latent at the moment but could
easily become a live bug in future, is that rewinding a cursor does
not necessarily lead to *immediately* applying ExecReScan to every
scan-level node in the plan tree. Upper-level nodes will think that
they can postpone that call if their child node is already marked
with chgParam flags. I don't see a way for that to happen today in
a plan tree that's simple enough for execCurrent.c's search_plan_tree
to understand, but that's one heck of a fragile assumption. So, add
some logic in search_plan_tree to detect chgParam flags being set on
nodes that it descended to/through, and assume that that means we
should consider lower scan nodes to be logically reset even if their
ReScan call hasn't actually happened yet.
Per bug #15395 from Matvey Arye. This has been broken for a long time,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/153764171023.14986.280404050547008575@wrigleys.postgresql.org
M src/backend/executor/execCurrent.c
M src/backend/executor/execScan.c
M src/test/regress/expected/portals.out
M src/test/regress/sql/portals.sql
docs: remove use of escape strings and use bytea hex output
commit : 2f0c19ce4eaf902f9ae5eaaa2e9a8544decf41e5
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Sep 2018 19:55:07 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Sep 2018 19:55:07 -0400
standard_conforming_strings defaulted to 'on' in PG 9.1.
bytea_output defaulted to 'hex' in PG 9.0.
Reported-by: André Hänsel
Discussion: https://postgr.es/m/12e601d447ac$345994a0$9d0cbde0$@webkr.de
Backpatch-through: 9.3
M doc/src/sgml/array.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/lobj.sgml
M doc/src/sgml/rowtypes.sgml
Error out for clang on x86-32 without SSE2 support, no -fexcess-precision.
commit : e553997e3f59024664d19251df8f50066c7bbb7f
author : Andres Freund <andres@anarazel.de>
date : Thu, 20 Sep 2018 18:10:52 -0700
committer: Andres Freund <andres@anarazel.de>
date : Thu, 20 Sep 2018 18:10:52 -0700
As clang currently doesn't support -fexcess-precision=standard,
compiling x86-32 code with SSE2 disabled, can lead to problems with
floating point overflow checks and the like.
This issue was noticed because clang, on at least some BSDs, defaults
to i386 compatibility, whereas it defaults to pentium4 on Linux. Our
forced usage of __builtin_isinf() lead to some overflow checks not
triggering when compiling for i386, e.g. when the result of the
calculation didn't overflow in 80bit registers, but did so in 64bit.
While we could just fall back to a non-builtin isinf, it seems likely
that the use of 80bit registers leads to other problems (which is why
we force the flag for GCC already). Therefore error out when
detecting clang in that situation.
Reported-By: Victor Wagner
Analyzed-By: Andrew Gierth and Andres Freund
Author: Andres Freund
Discussion: https://postgr.es/m/20180905005130.ewk4xcs5dgyzcy45@alap3.anarazel.de
Backpatch: 9.3-, all supported versions are affected
M configure
M configure.in
Defer restoration of libraries in parallel workers.
commit : de4fe83c7e6d14d996babec814c4fc12490889ef
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 20 Sep 2018 14:02:40 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 20 Sep 2018 14:02:40 +1200
Several users of extensions complained of crashes in parallel workers
that turned out to be due to syscache access from their _PG_init()
functions. Reorder the initialization of parallel workers so that
libraries are restored after the caches are initialized, and inside a
transaction.
This was reported in bug #15350 and elsewhere. We don't consider it
to be a bug: extensions shouldn't do that, because then they can't be
used in shared_preload_libraries. However, it's a fairly obscure
hazard and these extensions worked in practice before parallel query
came along. So let's make it work. Later commits might add a warning
message and eventually an error.
Back-patch to 9.6, where parallel query landed.
Author: Thomas Munro
Reviewed-by: Amit Kapila
Reported-by: Kieran McCusker, Jimmy
Discussion: https://postgr.es/m/153512195228.1489.8545997741965926448%40wrigleys.postgresql.org
M src/backend/access/transam/parallel.c
Don't ignore locktable-full failures in StandbyAcquireAccessExclusiveLock.
commit : 265ac024572b639eb315eab1b68faf45bcc8f4f2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 19 Sep 2018 12:43:51 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 19 Sep 2018 12:43:51 -0400
Commit 37c54863c removed the code in StandbyAcquireAccessExclusiveLock
that checked the return value of LockAcquireExtended. That created a
bug, because it's still passing reportMemoryError = false to
LockAcquireExtended, meaning that LOCKACQUIRE_NOT_AVAIL will be returned
if we're out of shared memory for the lock table.
In such a situation, the startup process would believe it had acquired an
exclusive lock even though it hadn't, with potentially dire consequences.
To fix, just drop the use of reportMemoryError = false, which allows us
to simplify the call into a plain LockAcquire(). It's unclear that the
locktable-full situation arises often enough that it's worth having a
better recovery method than crash-and-restart. (I strongly suspect that
the only reason the code path existed at all was that it was relatively
simple to do in the pre-37c54863c implementation. But now it's not.)
LockAcquireExtended's reportMemoryError parameter is now dead code and
could be removed. I refrained from doing so, however, because there
was some interest in resurrecting the behavior if we do get reports of
locktable-full failures in the field. Also, it seems unwise to remove
the parameter concurrently with shipping commit f868a8143, which added a
parameter; if there are any third-party callers of LockAcquireExtended,
we want them to get a wrong-number-of-parameters compile error rather
than a possibly-silent misinterpretation of its last parameter.
Back-patch to 9.6 where the bug was introduced.
Discussion: https://postgr.es/m/6202.1536359835@sss.pgh.pa.us
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/lock.c
Allow DSM allocation to be interrupted.
commit : f547035a0e1bfb338cd2113976d7f384854a64e3
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 18 Sep 2018 22:56:36 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 18 Sep 2018 22:56:36 +1200
Chris Travers reported that the startup process can repeatedly try to
cancel a backend that is in a posix_fallocate()/EINTR loop and cause it
to loop forever. Teach the retry loop to give up if an interrupt is
pending. Don't actually check for interrupts in that loop though,
because a non-local exit would skip some clean-up code in the caller.
Back-patch to 9.4 where DSM was added (and posix_fallocate() was later
back-patched).
Author: Chris Travers
Reviewed-by: Ildar Musin, Murat Kabilov, Oleksii Kliukin
Tested-by: Oleksii Kliukin
Discussion: https://postgr.es/m/CAN-RpxB-oeZve_J3SM_6%3DHXPmvEG%3DHX%2B9V9pi8g2YR7YW0rBBg%40mail.gmail.com
M src/backend/storage/ipc/dsm_impl.c
Fix failure with initplans used conditionally during EvalPlanQual rechecks.
commit : 2a97a0ad34daefde117cc9001c97ad3c0ef82dfb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 15 Sep 2018 13:42:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 15 Sep 2018 13:42:34 -0400
The EvalPlanQual machinery assumes that any initplans (that is,
uncorrelated sub-selects) used during an EPQ recheck would have already
been evaluated during the main query; this is implicit in the fact that
execPlan pointers are not copied into the EPQ estate's es_param_exec_vals.
But it's possible for that assumption to fail, if the initplan is only
reached conditionally. For example, a sub-select inside a CASE expression
could be reached during a recheck when it had not been previously, if the
CASE test depends on a column that was just updated.
This bug is old, appearing to date back to my rewrite of EvalPlanQual in
commit 9f2ee8f28, but was not detected until Kyle Samson reported a case.
To fix, force all not-yet-evaluated initplans used within the EPQ plan
subtree to be evaluated at the start of the recheck, before entering the
EPQ environment. This could be inefficient, if such an initplan is
expensive and goes unused again during the recheck --- but that's piling
one layer of improbability atop another. It doesn't seem worth adding
more complexity to prevent that, at least not in the back branches.
It was convenient to use the new-in-v11 ExecEvalParamExecParams function
to implement this, but I didn't like either its name or the specifics of
its API, so revise that.
Back-patch all the way. Rather than rewrite the patch to avoid depending
on bms_next_member() in the oldest branches, I chose to back-patch that
function into 9.4 and 9.3. (This isn't the first time back-patches have
needed that, and it exhausted my patience.) I also chose to back-patch
some test cases added by commits 71404af2a and 342a1ffa2 into 9.4 and 9.3,
so that the 9.x versions of eval-plan-qual.spec are all the same.
Andrew Gierth diagnosed the problem and contributed the added test cases,
though the actual code changes are by me.
Discussion: https://postgr.es/m/A033A40A-B234-4324-BE37-272279F7B627@tripadvisor.com
M src/backend/executor/execMain.c
M src/backend/executor/nodeSubplan.c
M src/include/executor/nodeSubplan.h
M src/test/isolation/expected/eval-plan-qual.out
M src/test/isolation/specs/eval-plan-qual.spec
Don't allow LIMIT/OFFSET clause within sub-selects to be pushed to workers.
commit : 568b4e1fdeb37361b1f5c5b1e1d5a1b998f7a9e9
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 14 Sep 2018 10:17:31 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 14 Sep 2018 10:17:31 +0530
Allowing sub-select containing LIMIT/OFFSET in workers can lead to
inconsistent results at the top-level as there is no guarantee that the
row order will be fully deterministic. The fix is to prohibit pushing
LIMIT/OFFSET within sub-selects to workers.
Reported-by: Andrew Fletcher
Bug: 15324
Author: Amit Kapila
Reviewed-by: Dilip Kumar
Backpatch-through: 9.6
Discussion: https://postgr.es/m/153417684333.10284.11356259990921828616@wrigleys.postgresql.org
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/plan/planner.c
M src/include/optimizer/planner.h
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql
Back-patch "Fix parallel hash join path search."
commit : 271b678436ce83bdd6bb355f9fcc011dafdd0122
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 14 Sep 2018 08:36:21 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 14 Sep 2018 08:36:21 +0530
Back-patch commit 655393a022bd653e2b48dbf20b69236981e35195 to 9.6. This
synchronizes the relavant code and helps in generating parallel paths in
some cases in 9.6. This also helps in back-patch of future patches where
we can get the same plan in all branches.
Discussion: https://postgr.es/m/CAA4eK1+XK_875cJA1HPVpx9C7C8Fp7i4QzLJ17T3igfU2iadxQ@mail.gmail.com
M src/backend/optimizer/path/joinpath.c
Attach FPI to the first record after full_page_writes is turned on.
commit : fd4f2af774db6a978130da88e94e7425c07651d5
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 13 Sep 2018 16:08:55 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 13 Sep 2018 16:08:55 +0530
XLogInsert fails to attach a required FPI to the first record after
full_page_writes is turned on by the last checkpoint. This bug got
introduced in 9.5 due to code rearrangement in commits 2c03216d83 and
2076db2aea. Fix it by ensuring that XLogInsertRecord performs a
recomputation when the given record is generated with FPW as off but
found that the flag has been turned on while actually inserting the
record.
Reported-by: Kyotaro Horiguchi
Author: Kyotaro Horiguchi
Reviewed-by: Amit Kapila
Backpatch-through: 9.5 where this problem was introduced
Discussion: https://postgr.es/m/20180420.151043.74298611.horiguchi.kyotaro@lab.ntt.co.jp
M src/backend/access/transam/xlog.c
doc: Update broken links
commit : 4659632dbaadc33baa9fa42c49582d5f8e77f711
author : Peter Eisentraut <peter_e@gmx.net>
date : Tue, 14 Aug 2018 22:54:52 +0200
committer: Peter Eisentraut <peter_e@gmx.net>
date : Tue, 14 Aug 2018 22:54:52 +0200
Discussion: https://www.postgresql.org/message-id/flat/153044458767.13254.16049977382403131287%40wrigleys.postgresql.org
M doc/src/sgml/libpq.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/runtime.sgml
Repair bug in regexp split performance improvements.
commit : 03e0bc1171c864b9add64f0ef5351126faa4e3f5
author : Andrew Gierth <rhodiumtoad@postgresql.org>
date : Wed, 12 Sep 2018 19:31:06 +0100
committer: Andrew Gierth <rhodiumtoad@postgresql.org>
date : Wed, 12 Sep 2018 19:31:06 +0100
Commit c8ea87e4b introduced a temporary conversion buffer for
substrings extracted during regexp splits. Unfortunately the code that
sized it was failing to ignore the effects of ignored degenerate
regexp matches, so for regexp_split_* calls it could under-size the
buffer in such cases.
Fix, and add some regression test cases (though those will only catch
the bug if run in a multibyte encoding).
Backpatch to 9.3 as the faulty code was.
Thanks to the PostGIS project, Regina Obe and Paul Ramsey for the
report (via IRC) and assistance in analysis. Patch by me.
M src/backend/utils/adt/regexp.c
M src/test/regress/expected/strings.out
M src/test/regress/sql/strings.sql
Repair double-free in SP-GIST rescan (bug #15378)
commit : 84a3a1e55c86a104a8e259a43ed31814e010954f
author : Andrew Gierth <rhodiumtoad@postgresql.org>
date : Tue, 11 Sep 2018 18:14:19 +0100
committer: Andrew Gierth <rhodiumtoad@postgresql.org>
date : Tue, 11 Sep 2018 18:14:19 +0100
spgrescan would first reset traversalCxt, and then traverse a
potentially non-empty stack containing pointers to traversalValues
which had been allocated in those contexts, freeing them a second
time. This bug originates in commit ccd6eb49a where traversalValue was
introduced.
Repair by traversing the stack before the context reset; this isn't
ideal, since it means doing retail pfree in a context that's about to
be reset, but the freeing of a stack entry is also done in other
places in the code during the scan so it's not worth trying to
refactor it further. Regression test added.
Backpatch to 9.6 where the problem was introduced.
Per bug #15378; analysis and patch by me, originally from a report on
IRC by user velix; see also PostGIS ticket #4174; review by Alexander
Korotkov.
Discussion: https://postgr.es/m/153663176628.23136.11901365223750051490@wrigleys.postgresql.org
M src/backend/access/spgist/spgscan.c
M src/test/regress/expected/spgist.out
M src/test/regress/sql/spgist.sql
Fix past pd_upper write in ginRedoRecompress()
commit : f9e66f2fbbb49a493045c8d8086a9b15d95b8f18
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 9 Sep 2018 21:19:29 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 9 Sep 2018 21:19:29 +0300
ginRedoRecompress() replays actions over compressed segments of posting list
in-place. However, it might lead to write past pg_upper, because intermediate
state during playing the changes can take more space than both original state
and final state. This commit fixes that by refuse from in-place modification.
Instead page tail is copied once modification is started, and then it's used
as the source of original segments. Backpatch to 9.4 where posting list
compression was introduced.
Reported-by: Sivasubramanian Ramasubramanian
Discussion: https://postgr.es/m/1536091151804.6588%40amazon.com
Author: Alexander Korotkov based on patch from and ideas by Sivasubramanian Ramasubramanian
Review: Sivasubramanian Ramasubramanian
Backpatch-through: 9.4
M src/backend/access/gin/ginxlog.c
Save/restore SPI's global variables in SPI_connect() and SPI_finish().
commit : 82ebf39fcb25236b60f05e4483d86cb4144fce8a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Sep 2018 20:09:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Sep 2018 20:09:57 -0400
This patch removes two sources of interference between nominally
independent functions when one SPI-using function calls another,
perhaps without knowing that it does so.
Chapman Flack pointed out that xml.c's query_to_xml_internal() expects
SPI_tuptable and SPI_processed to stay valid across datatype output
function calls; but it's possible that such a call could involve
re-entrant use of SPI. It seems likely that there are similar hazards
elsewhere, if not in the core code then in third-party SPI users.
Previously SPI_finish() reset SPI's API globals to zeroes/nulls, which
would typically make for a crash in such a situation. Restoring them
to the values they had at SPI_connect() seems like a considerably more
useful behavior, and it still meets the design goal of not leaving any
dangling pointers to tuple tables of the function being exited.
Also, cause SPI_connect() to reset these variables to zeroes/nulls after
saving them. This prevents interference in the opposite direction: it's
possible that a SPI-using function that's only ever been tested standalone
contains assumptions that these variables start out as zeroes. That was
the case as long as you were the outermost SPI user, but not so much for
an inner user. Now it's consistent.
Report and fix suggestion by Chapman Flack, actual patch by me.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/9fa25bef-2e4f-1c32-22a4-3ad0723c4a17@anastigmatix.net
M src/backend/executor/spi.c
M src/include/executor/spi_priv.h
Limit depth of forced recursion for CLOBBER_CACHE_RECURSIVELY.
commit : 2ef5c12ad5b613fd4f4755122aa837689f7c5fd5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Sep 2018 18:13:29 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Sep 2018 18:13:29 -0400
It's somewhat surprising that we got away with this before. (Actually,
since nobody tests this routinely AFAIK, it might've been broken for
awhile. But it's definitely broken in the wake of commit f868a8143.)
It seems sufficient to limit the forced recursion to a small number
of levels.
Back-patch to all supported branches, like the preceding patch.
Discussion: https://postgr.es/m/12259.1532117714@sss.pgh.pa.us
M src/backend/utils/cache/inval.c
Fix longstanding recursion hazard in sinval message processing.
commit : 395f310b04c59250e58d131cc00c7cbf80e94198
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Sep 2018 18:04:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Sep 2018 18:04:38 -0400
LockRelationOid and sibling routines supposed that, if our session already
holds the lock they were asked to acquire, they could skip calling
AcceptInvalidationMessages on the grounds that we must have already read
any remote sinval messages issued against the relation being locked.
This is normally true, but there's a critical special case where it's not:
processing inside AcceptInvalidationMessages might attempt to access system
relations, resulting in a recursive call to acquire a relation lock.
Hence, if the outer call had acquired that same system catalog lock, we'd
fall through, despite the possibility that there's an as-yet-unread sinval
message for that system catalog. This could, for example, result in
failure to access a system catalog or index that had just been processed
by VACUUM FULL. This is the explanation for buildfarm failures we've been
seeing intermittently for the past three months. The bug is far older
than that, but commits a54e1f158 et al added a new recursion case within
AcceptInvalidationMessages that is apparently easier to hit than any
previous case.
To fix this, we must not skip calling AcceptInvalidationMessages until
we have *finished* a call to it since acquiring a relation lock, not
merely acquired the lock. (There's already adequate logic inside
AcceptInvalidationMessages to deal with being called recursively.)
Fortunately, we can implement that at trivial cost, by adding a flag
to LOCALLOCK hashtable entries that tracks whether we know we have
completed such a call.
There is an API hazard added by this patch for external callers of
LockAcquire: if anything is testing for LOCKACQUIRE_ALREADY_HELD,
it might be fooled by the new return code LOCKACQUIRE_ALREADY_CLEAR
into thinking the lock wasn't already held. This should be a fail-soft
condition, though, unless something very bizarre is being done in
response to the test.
Also, I added an additional output argument to LockAcquireExtended,
assuming that that probably isn't called by any outside code given
the very limited usefulness of its additional functionality.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/12259.1532117714@sss.pgh.pa.us
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lock.c
M src/include/storage/lock.h
doc: wording fix
commit : fb4045caadd9c08d30e7a10bbab3ba9e57e037bc
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 6 Sep 2018 20:42:24 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 6 Sep 2018 20:42:24 -0400
Author: Liudmila Mantrova
Backpatch-through: 9.6 and 10 only
M doc/src/sgml/pgtrgm.sgml
Make contrib/unaccent's unaccent() function work when not in search path.
commit : 594ee1ada5bc9affa37751d7ee840acb636a1f32
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 6 Sep 2018 10:49:45 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 6 Sep 2018 10:49:45 -0400
Since the fixes for CVE-2018-1058, we've advised people to schema-qualify
function references in order to fix failures in code that executes under
a minimal search_path setting. However, that's insufficient to make the
single-argument form of unaccent() work, because it looks up the "unaccent"
text search dictionary using the search path.
The most expedient answer seems to be to remove the search_path dependency
by making it look in the same schema that the unaccent() function itself
is declared in. This will definitely work for the normal usage of this
function with the unaccent dictionary provided by the extension.
It's barely possible that there are people who were relying on the
search-path-dependent behavior to select other dictionaries with the same
name; but if there are any such people at all, they can still get that
behavior by writing unaccent('unaccent', ...), or possibly
unaccent('unaccent'::text::regdictionary, ...) if the lookup has to be
postponed to runtime.
Per complaint from Gunnlaugur Thor Briem. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/CAPs+M8LCex6d=DeneofdsoJVijaG59m9V0ggbb3pOH7hZO4+cQ@mail.gmail.com
M contrib/unaccent/unaccent.c
M doc/src/sgml/unaccent.sgml
Make argument names of pg_get_object_address consistent, and fix docs.
commit : 5b114b045a479e584be9252dedac4c357c456311
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 5 Sep 2018 13:47:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 5 Sep 2018 13:47:16 -0400
pg_get_object_address and pg_identify_object_as_address are supposed
to be inverses, but they disagreed as to the names of the arguments
representing the textual form of an object address. Moreover, the
documented argument names didn't agree with reality at all, either
for these functions or pg_identify_object.
In HEAD and v11, I think we can get away with renaming the input
arguments of pg_get_object_address to match the outputs of
pg_identify_object_as_address. In theory that might break queries
using named-argument notation to call pg_get_object_address, but
it seems really unlikely that anybody is doing that, or that they'd
have much trouble adjusting if they were. In older branches, we'll
just live with the lack of consistency.
Aside from fixing the documentation of these functions to match reality,
I couldn't resist the temptation to do some copy-editing.
Per complaint from Jean-Pierre Pelletier. Back-patch to 9.5 where these
functions were introduced. (Before v11, this is a documentation change
only.)
Discussion: https://postgr.es/m/CANGqjDnWH8wsTY_GzDUxbt4i=y-85SJreZin4Hm8uOqv1vzRQA@mail.gmail.com
M doc/src/sgml/func.sgml
docs: improve AT TIME ZONE description
commit : a704f8327f56b9a6b38e8d88e465ae021a01bc52
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 4 Sep 2018 22:34:07 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 4 Sep 2018 22:34:07 -0400
The previous description was unclear. Also add a third example, change
use of time zone acronyms to more verbose descriptions, and add a
mention that using 'time' with AT TIME ZONE uses the current time zone
rules.
Backpatch-through: 9.3
M doc/src/sgml/func.sgml
Prohibit pushing subqueries containing window function calculation to workers.
commit : f658235a448aa9462249f9d3448168be1e63a55f
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 4 Sep 2018 11:01:25 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 4 Sep 2018 11:01:25 +0530
Allowing window function calculation in workers leads to inconsistent
results because if the input row ordering is not fully deterministic, the
output of window functions might vary across workers. The fix is to treat
them as parallel-restricted.
In the passing, improve the coding pattern in max_parallel_hazard_walker
so that it has a chain of mutually-exclusive if ... else if ... else if
... else if ... IsA tests.
Reported-by: Marko Tiikkaja
Bug: 15324
Author: Amit Kapila
Reviewed-by: Tom Lane
Backpatch-through: 9.6
Discussion: https://postgr.es/m/CAL9smLAnfPJCDUUG4ckX2iznj53V7VSMsYefzZieN93YxTNOcw@mail.gmail.com
M src/backend/optimizer/util/clauses.c
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql
Fix initial sync of slot parent directory when restoring status
commit : d8030c6842076156c8792cb1613eea1b1422059e
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 2 Sep 2018 12:40:52 -0700
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 2 Sep 2018 12:40:52 -0700
At the beginning of recovery, information from replication slots is
recovered from disk to memory. In order to ensure the durability of the
information, the status file as well as its parent directory are
synced. It happens that the sync on the parent directory was done
directly using the status file path, which is logically incorrect, and
the current code has been doing a sync on the same object twice in a
row.
Reported-by: Konstantin Knizhnik
Diagnosed-by: Konstantin Knizhnik
Author: Michael Paquier
Discussion: https://postgr.es/m/9eb1a6d5-b66f-2640-598d-c5ea46b8f68a@postgrespro.ru
Backpatch-through: 9.4-
M src/backend/replication/slot.c
Doc: fix oversights in "Client/Server Character Set Conversions" table.
commit : 25fb6ba11575515d754477492a5ff83ef0c126c4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 1 Sep 2018 16:02:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 1 Sep 2018 16:02:47 -0400
This table claimed that JOHAB could be used as a server encoding, which
was true originally but hasn't been true since 8.3. It also lacked
entries for EUC_JIS_2004 and SHIFT_JIS_2004.
JOHAB problem noted by Lars Kanis, the others by me.
Discussion: https://postgr.es/m/c0f514a1-b7a9-b9ea-1c02-c34aead56c06@greiz-reinsdorf.de
M doc/src/sgml/charset.sgml
Avoid using potentially-under-aligned page buffers.
commit : 826980424538cb149670f9c89eedf34387ff674f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 1 Sep 2018 15:27:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 1 Sep 2018 15:27:13 -0400
There's a project policy against using plain "char buf[BLCKSZ]" local
or static variables as page buffers; preferred style is to palloc or
malloc each buffer to ensure it is MAXALIGN'd. However, that policy's
been ignored in an increasing number of places. We've apparently got
away with it so far, probably because (a) relatively few people use
platforms on which misalignment causes core dumps and/or (b) the
variables chance to be sufficiently aligned anyway. But this is not
something to rely on. Moreover, even if we don't get a core dump,
we might be paying a lot of cycles for misaligned accesses.
To fix, invent new union types PGAlignedBlock and PGAlignedXLogBlock
that the compiler must allocate with sufficient alignment, and use
those in place of plain char arrays.
I used these types even for variables where there's no risk of a
misaligned access, since ensuring proper alignment should make
kernel data transfers faster. I also changed some places where
we had been palloc'ing short-lived buffers, for coding style
uniformity and to save palloc/pfree overhead.
Since this seems to be a live portability hazard (despite the lack
of field reports), back-patch to all supported versions.
Patch by me; thanks to Michael Paquier for review.
Discussion: https://postgr.es/m/1535618100.1286.3.camel@credativ.de
M contrib/bloom/blinsert.c
M contrib/pg_prewarm/pg_prewarm.c
M src/backend/access/gin/ginentrypage.c
M src/backend/access/gin/ginfast.c
M src/backend/access/hash/hashpage.c
M src/backend/access/heap/heapam.c
M src/backend/access/heap/visibilitymap.c
M src/backend/access/transam/generic_xlog.c
M src/backend/access/transam/xlog.c
M src/backend/access/transam/xloginsert.c
M src/backend/access/transam/xlogreader.c
M src/backend/commands/tablecmds.c
M src/backend/replication/walsender.c
M src/backend/storage/file/buffile.c
M src/backend/storage/freespace/freespace.c
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_resetxlog/pg_resetxlog.c
M src/bin/pg_rewind/copy_fetch.c
M src/bin/pg_upgrade/file.c
M src/include/c.h
Ignore server-side delays when enforcing wal_sender_timeout.
commit : 081e4104a4317709c1adf0fab42a1546ebf8d6b2
author : Noah Misch <noah@leadboat.com>
date : Fri, 31 Aug 2018 22:59:58 -0700
committer: Noah Misch <noah@leadboat.com>
date : Fri, 31 Aug 2018 22:59:58 -0700
Healthy clients of servers having poor I/O performance, such as
buildfarm members hamster and tern, saw unexpected timeouts. That
disagreed with documentation. This fix adds one gettimeofday() call
whenever ProcessRepliesIfAny() finds no client reply messages.
Back-patch to 9.4; the bug's symptom is rare and mild, and the code all
moved between 9.3 and 9.4.
Discussion: https://postgr.es/m/20180826034600.GA1105084@rfd.leadboat.com
M src/backend/replication/walsender.c
Ensure correct minimum consistent point on standbys
commit : 4a9a5bb3fd2a6aafda1e3bd8ef25c1fb004f5c87
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 31 Aug 2018 11:04:33 -0700
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 31 Aug 2018 11:04:33 -0700
Startup process has improved its calculation of incorrect minimum
consistent point in 8d68ee6, which ensures that all WAL available gets
replayed when doing crash recovery, and has introduced an incorrect
calculation of the minimum recovery point for non-startup processes,
which can cause incorrect page references on a standby when for example
the background writer flushed a couple of pages on-disk but was not
updating the control file to let a subsequent crash recovery replay to
where it should have.
The only case where this has been reported to be a problem is when a
standby needs to calculate the latest removed xid when replaying a btree
deletion record, so one would need connections on a standby that happen
just after recovery has thought it reached a consistent point. Using a
background worker which is started after the consistent point is reached
would be the easiest way to get into problems if it connects to a
database. Having clients which attempt to connect periodically could
also be a problem, but the odds of seeing this problem are much lower.
The fix used is pretty simple, as the idea is to give access to the
minimum recovery point written in the control file to non-startup
processes so as they use a reference, while the startup process still
initializes its own references of the minimum consistent point so as the
original problem with incorrect page references happening post-promotion
with a crash do not show up.
Reported-by: Alexander Kukushkin
Diagnosed-by: Alexander Kukushkin
Author: Michael Paquier
Reviewed-by: Kyotaro Horiguchi, Alexander Kukushkin
Discussion: https://postgr.es/m/153492341830.1368.3936905691758473953@wrigleys.postgresql.org
Backpatch-through: 9.3
M src/backend/access/transam/xlog.c
Enforce cube dimension limit in all cube construction functions
commit : 5fed7b24aed50a50e2e922de2bcc51458a067210
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 30 Aug 2018 14:18:53 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 30 Aug 2018 14:18:53 +0300
contrib/cube has a limit to 100 dimensions for cube datatype. However, it's
not enforced everywhere, and one can actually construct cube with more than
100 dimensions having then trouble with dump/restore. This commit add checks
for dimensions limit in all functions responsible for cube construction.
Backpatch to all supported versions.
Reported-by: Andrew Gierth
Discussion: https://postgr.es/m/87va7uybt4.fsf%40news-spur.riddles.org.uk
Author: Andrey Borodin with small additions by me
Review: Tom Lane
Backpatch-through: 9.3
M contrib/cube/cube.c
M contrib/cube/expected/cube.out
M contrib/cube/sql/cube.sql
Split contrib/cube platform-depended checks into separate test
commit : df85f8ecea535eee425ea66b3ecf852739e54cea
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 30 Aug 2018 14:09:25 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 30 Aug 2018 14:09:25 +0300
We're currently maintaining two outputs for cube regression test. But that
appears to be unsuitable, because these outputs are different in out few checks
involving scientific notation. So, split checks involving scientific notation
into separate test, making contrib/cube easier to maintain. Backpatch to all
supported versions in order to make further backpatching easier.
Discussion: https://postgr.es/m/CAPpHfdvJgWjxHsJTtT%2Bo1tz3OR8EFHcLQjhp-d3%2BUcmJLh-fQA%40mail.gmail.com
Author: Alexander Korotkov
Backpatch-through: 9.3
M contrib/cube/Makefile
M contrib/cube/expected/cube.out
D contrib/cube/expected/cube_1.out
D contrib/cube/expected/cube_2.out
D contrib/cube/expected/cube_3.out
A contrib/cube/expected/cube_sci.out
A contrib/cube/expected/cube_sci_1.out
A contrib/cube/expected/cube_sci_2.out
A contrib/cube/expected/cube_sci_3.out
M contrib/cube/sql/cube.sql
A contrib/cube/sql/cube_sci.sql
Make checksum_impl.h safe to compile with -fstrict-aliasing.
commit : d6ef17ed7bba02c83408296f5fff09766e4f14dd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 31 Aug 2018 12:26:20 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 31 Aug 2018 12:26:20 -0400
In general, Postgres requires -fno-strict-aliasing with compilers that
implement C99 strict aliasing rules. There's little hope of getting
rid of that overall. But it seems like it would be a good idea if
storage/checksum_impl.h in particular didn't depend on it, because
that header is explicitly intended to be included by external programs.
We don't have a lot of control over the compiler switches that an
external program might use, as shown by Michael Banck's report of
failure in a privately-modified version of pg_verify_checksums.
Hence, switch to using a union in place of willy-nilly pointer casting
inside this file. I think this makes the code a bit more readable
anyway.
checksum_impl.h hasn't changed since it was introduced in 9.3,
so back-patch all the way.
Discussion: https://postgr.es/m/1535618100.1286.3.camel@credativ.de
M src/include/storage/checksum_impl.h
Stop bgworkers during fast shutdown with postmaster in startup phase
commit : f6feb8e3851f8393305ac6d45fe49fe6b9295976
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 29 Aug 2018 17:11:27 -0700
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 29 Aug 2018 17:11:27 -0700
When a postmaster gets into its phase PM_STARTUP, it would start
background workers using BgWorkerStart_PostmasterStart mode immediately,
which would cause problems for a fast shutdown as the postmaster forgets
to send SIGTERM to already-started background workers. With smart and
immediate shutdowns, this correctly happened, and fast shutdown is the
only mode missing the shot.
Author: Alexander Kukushkin
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/CAFh8B=mvnD8+DZUfzpi50DoaDfZRDfd7S=gwj5vU9GYn8UvHkA@mail.gmail.com
Backpatch-through: 9.5
M src/backend/postmaster/postmaster.c
Make pg_restore's identify_locking_dependencies() more bulletproof.
commit : 97aa524f18c0c5d4f800a6b37bb340890dba3224
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 28 Aug 2018 19:46:59 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 28 Aug 2018 19:46:59 -0400
This function had a blacklist of dump object types that it believed
needed exclusive lock ... but we hadn't maintained that, so that it
was missing ROW SECURITY, POLICY, and INDEX ATTACH items, all of
which need (or should be treated as needing) exclusive lock.
Since the same oversight seems likely in future, let's reverse the
sense of the test so that the code has a whitelist of safe object
types; better to wrongly assume a command can't be run in parallel
than the opposite. Currently the only POST_DATA object type that's
safe is CREATE INDEX ... and that list hasn't changed in a long time.
Back-patch to 9.5 where RLS came in.
Discussion: https://postgr.es/m/11450.1535483506@sss.pgh.pa.us
M src/bin/pg_dump/pg_backup_archiver.c
postgres_fdw: don't push ORDER BY with no vars (bug #15352)
commit : 639bdbb96fed7f8975111bf8081172fd8c7f623d
author : Andrew Gierth <rhodiumtoad@postgresql.org>
date : Tue, 28 Aug 2018 14:43:51 +0100
committer: Andrew Gierth <rhodiumtoad@postgresql.org>
date : Tue, 28 Aug 2018 14:43:51 +0100
Commit aa09cd242 changed a condition in find_em_expr_for_rel from
being a bms_equal comparison of relids to bms_is_subset, in order to
support order by clauses on foreign joins. But this also allows
through the degenerate case of expressions with no Vars at all (and
hence empty relids), including integer constants which will be parsed
unexpectedly on the remote (viz. "ERROR: ORDER BY position 0 is not in
select list" as in the bug report).
Repair by adding an additional !bms_is_empty test.
Backpatch through to 9.6 where the aforementioned change was made.
Per bug #15352 from Maksym Boguk; analysis and patch by me.
Discussion: https://postgr.es/m/153518420278.1478.14875560810251994661@wrigleys.postgresql.org
M contrib/postgres_fdw/postgres_fdw.c
Avoid quadratic slowdown in regexp match/split functions.
commit : 450b247415125e08821dfbb68b0a15a4f0f7eb22
author : Andrew Gierth <rhodiumtoad@postgresql.org>
date : Tue, 28 Aug 2018 09:52:25 +0100
committer: Andrew Gierth <rhodiumtoad@postgresql.org>
date : Tue, 28 Aug 2018 09:52:25 +0100
regexp_matches, regexp_split_to_table and regexp_split_to_array all
work by compiling a list of match positions as character offsets (NOT
byte positions) in the source string.
Formerly, they then used text_substr to extract the matched text; but
in a multi-byte encoding, that counts the characters in the string,
and the characters needed to reach the starting byte position, on
every call. Accordingly, the performance degraded as the product of
the input string length and the number of match positions, such that
splitting a string of a few hundred kbytes could take many minutes.
Repair by keeping the wide-character copy of the input string
available (only in the case where encoding_max_length is not 1) after
performing the match operation, and extracting substrings from that
instead. This reduces the complexity to being linear in the number of
result bytes, discounting the actual regexp match itself (which is not
affected by this patch).
In passing, remove cleanup using retail pfree() which was obsoleted by
commit ff428cded (Feb 2008) which made cleanup of SRF multi-call
contexts automatic. Also increase (to ~134 million) the maximum number
of matches and provide an error message when it is reached.
Backpatch all the way because this has been wrong forever.
Analysis and patch by me; review by Kaiting Chen.
Discussion: https://postgr.es/m/87pnyn55qh.fsf@news-spur.riddles.org.uk
see also https://postgr.es/m/87lg996g4r.fsf@news-spur.riddles.org.uk
M src/backend/utils/adt/regexp.c
Fix missing dependency for pg_dump's ENABLE ROW LEVEL SECURITY items.
commit : 173df4cd36dfc8cbb36de908cea6bd03b13c0cb8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 27 Aug 2018 15:11:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 27 Aug 2018 15:11:12 -0400
The archive should show a dependency on the item's table, but it failed
to include one. This could cause failures in parallel restore due to
emitting ALTER TABLE ... ENABLE ROW LEVEL SECURITY before restoring
the table's data. In practice the odds of a problem seem low, since
you would typically need to have set FORCE ROW LEVEL SECURITY as well,
and you'd also need a very high --jobs count to have any chance of this
happening. That probably explains the lack of field reports.
Still, it's a bug, so back-patch to 9.5 where RLS was introduced.
Discussion: https://postgr.es/m/19784.1535390902@sss.pgh.pa.us
M src/bin/pg_dump/pg_dump.c
Make syslogger more robust against failures in opening CSV log files.
commit : 93ca07fd8abf600bf56a56ed8b7f882ab58321f3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 26 Aug 2018 14:21:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 26 Aug 2018 14:21:55 -0400
The previous coding figured it'd be good enough to postpone opening
the first CSV log file until we got a message we needed to write there.
This is unsafe, though, because if the open fails we end up in infinite
recursion trying to report the failure. Instead make the CSV log file
management code look as nearly as possible like the longstanding logic
for the stderr log file. In particular, open it immediately at postmaster
startup (if enabled), or when we get a SIGHUP in which we find that
log_destination has been changed to enable CSV logging.
It seems OK to fail if a postmaster-start-time open attempt fails, as
we've long done for the stderr log file. But we can't die if we fail
to open a CSV log file during SIGHUP, so we're still left with a problem.
In that case, write any output meant for the CSV log file to the stderr
log file. (This will also cover race-condition cases in which backends
send CSV log data before or after we have the CSV log file open.)
This patch also fixes an ancient oversight that, if CSV logging was
turned off during a SIGHUP, we never actually closed the last CSV
log file.
In passing, remember to reset whereToSendOutput = DestNone during syslogger
start, since (unlike all other postmaster children) it's forked before the
postmaster has done that. This made for a platform-dependent difference
in error reporting behavior between the syslogger and other children:
except on Windows, it'd report problems to the original postmaster stderr
as well as the normal error log file(s). It's barely possible that that
was intentional at some point; but it doesn't seem likely to be desirable
in production, and the platform dependency definitely isn't desirable.
Per report from Alexander Kukushkin. It's been like this for a long time,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/CAFh8B==iLUD_gqC-dAENS0V+kVrCeGiKujtKqSQ7++S-caaChw@mail.gmail.com
M src/backend/postmaster/syslogger.c
doc: correct syntax of pgtrgm examples in older releases
commit : c2ea62f58575045bd85156581f55b57067f33fd8
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 25 Aug 2018 15:03:32 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 25 Aug 2018 15:03:32 -0400
Reported-by: Liudmila Mantrova
Discussion: https://postgr.es/m/ded40ecb-557e-8c50-7d58-69f4b5226664@postgrespro.ru
Backpatch-through: 9.6 and 10 only
M doc/src/sgml/pgtrgm.sgml
doc: "Latest checkpoint location" will not match in pg_upgrade
commit : 50ff8844bb3b32fbc2a4d1b2d4fe22b4e5efd779
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 25 Aug 2018 13:35:14 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 25 Aug 2018 13:35:14 -0400
Mention that "Latest checkpoint location" will not match in pg_upgrade
if the standby server is still running during the upgrade, which is
possible. "Match" text first appeared in PG 9.5.
Reported-by: Paul Bonaud
Discussion: https://postgr.es/m/c7268794-edb4-1772-3bfd-04c54585c24e@trainline.com
Backpatch-through: 9.5
M doc/src/sgml/ref/pgupgrade.sgml
doc: add doc link for 'applicable_roles'
commit : a65f22fc6223117650fec31a3821f181f15d9366
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 25 Aug 2018 13:01:24 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 25 Aug 2018 13:01:24 -0400
Reported-by: Ashutosh Sharma
Discussion: https://postgr.es/m/CAE9k0PnhnL6MNDLuvkk8USzOa_DpzDzFQPAM_uaGuXbh9HMKYw@mail.gmail.com
Author: Ashutosh Sharma
Backpatch-through: 9.3
M doc/src/sgml/information_schema.sgml
docs: clarify plpython SD and GD dictionary behavior
commit : 06d47f6dc6e338e44700a51ad6dff1e855a6aa33
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 25 Aug 2018 11:52:29 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 25 Aug 2018 11:52:29 -0400
Reported-by: Adam Bielański
Discussion: https://postgr.es/m/153484305538.1370.7605856225879294548@wrigleys.postgresql.org
Backpatch-through: 9.3
M doc/src/sgml/plpython.sgml
Fix lexing of standard multi-character operators in edge cases.
commit : 5ec70a9286211b4734795419a292128165ca433e
author : Andrew Gierth <rhodiumtoad@postgresql.org>
date : Thu, 23 Aug 2018 18:29:18 +0100
committer: Andrew Gierth <rhodiumtoad@postgresql.org>
date : Thu, 23 Aug 2018 18:29:18 +0100
Commits c6b3c939b (which fixed the precedence of >=, <=, <> operators)
and 865f14a2d (which added support for the standard => notation for
named arguments) created a class of lexer tokens which look like
multi-character operators but which have their own token IDs distinct
from Op. However, longest-match rules meant that following any of
these tokens with another operator character, as in (1<>-1), would
cause them to be incorrectly returned as Op.
The error here isn't immediately obvious, because the parser would
usually still find the correct operator via the Op token, but there
were more subtle problems:
1. If immediately followed by a comment or +-, >= <= <> would be given
the old precedence of Op rather than the correct new precedence;
2. If followed by a comment, != would be returned as Op rather than as
NOT_EQUAL, causing it not to be found at all;
3. If followed by a comment or +-, the => token for named arguments
would be lexed as Op, causing the argument to be mis-parsed as a
simple expression, usually causing an error.
Fix by explicitly checking for the operators in the {operator} code
block in addition to all the existing special cases there.
Backpatch to 9.5 where the problem was introduced.
Analysis and patch by me; review by Tom Lane.
Discussion: https://postgr.es/m/87va851ppl.fsf@news-spur.riddles.org.uk
M src/backend/parser/scan.l
M src/fe_utils/psqlscan.l
M src/interfaces/ecpg/preproc/pgc.l
M src/test/regress/expected/create_operator.out
M src/test/regress/expected/polymorphism.out
M src/test/regress/sql/create_operator.sql
M src/test/regress/sql/polymorphism.sql
Reduce an unnecessary O(N^3) loop in lexer.
commit : 4854ead60a293bb1ca235bd110c6a56c8aaaafd3
author : Andrew Gierth <rhodiumtoad@postgresql.org>
date : Thu, 23 Aug 2018 20:00:33 +0100
committer: Andrew Gierth <rhodiumtoad@postgresql.org>
date : Thu, 23 Aug 2018 20:00:33 +0100
The lexer's handling of operators contained an O(N^3) hazard when
dealing with long strings of + or - characters; it seems hard to
prevent this case from being O(N^2), but the additional N multiplier
was not needed.
Backpatch all the way since this has been there since 7.x, and it
presents at least a mild hazard in that trying to do Bind, PREPARE or
EXPLAIN on a hostile query could take excessive time (without
honouring cancels or timeouts) even if the query was never executed.
M src/backend/parser/scan.l
M src/fe_utils/psqlscan.l
M src/interfaces/ecpg/preproc/pgc.l
Fix set of NLS translation issues
commit : 90b0f30aea0ee456614ec97f518ffaf8ca6c3b05
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 21 Aug 2018 15:18:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 21 Aug 2018 15:18:00 +0900
While monitoring the code, a couple of issues related to string
translation has showed up:
- Some routines for auto-updatable views return an error string, which
sometimes missed the shot. A comment regarding string translation is
added for each routine to help with future features.
- GSSAPI authentication missed two translations.
- vacuumdb handles non-translated strings.
Reported-by: Kyotaro Horiguchi
Author: Kyotaro Horiguchi
Reviewed-by: Michael Paquier, Tom Lane
Discussion: https://postgr.es/m/20180810.152131.31921918.horiguchi.kyotaro@lab.ntt.co.jp
Backpatch-through: 9.3
M src/backend/commands/tablecmds.c
M src/backend/commands/view.c
M src/backend/libpq/auth.c
M src/backend/rewrite/rewriteHandler.c
M src/bin/scripts/vacuumdb.c
Ensure schema qualification in pg_restore DISABLE/ENABLE TRIGGER commands.
commit : 72329ba03b601bcd0cfe1349a9acdd251972cbd0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Aug 2018 17:12:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Aug 2018 17:12:21 -0400
Previously, this code blindly followed the common coding pattern of
passing PQserverVersion(AH->connection) as the server-version parameter
of fmtQualifiedId. That works as long as we have a connection; but in
pg_restore with text output, we don't. Instead we got a zero from
PQserverVersion, which fmtQualifiedId interpreted as "server is too old to
have schemas", and so the name went unqualified. That still accidentally
managed to work in many cases, which is probably why this ancient bug went
undetected for so long. It only became obvious in the wake of the changes
to force dump/restore to execute with restricted search_path.
In HEAD/v11, let's deal with this by ripping out fmtQualifiedId's server-
version behavioral dependency, and just making it schema-qualify all the
time. We no longer support pg_dump from servers old enough to need the
ability to omit schema name, let alone restoring to them. (Also, the few
callers outside pg_dump already didn't work with pre-schema servers.)
In older branches, that's not an acceptable solution, so instead just
tweak the DISABLE/ENABLE TRIGGER logic to ensure it will schema-qualify
its output regardless of server version.
Per bug #15338 from Oleg somebody. Back-patch to all supported branches.
Discussion: https://postgr.es/m/153452458706.1316.5328079417086507743@wrigleys.postgresql.org
M src/bin/pg_dump/pg_backup_archiver.c
Set scan direction appropriately for SubPlans (bug #15336)
commit : 6302fe6b28e24e87d78e457e85362dc2fa9ca119
author : Andrew Gierth <rhodiumtoad@postgresql.org>
date : Fri, 17 Aug 2018 15:04:26 +0100
committer: Andrew Gierth <rhodiumtoad@postgresql.org>
date : Fri, 17 Aug 2018 15:04:26 +0100
When executing a SubPlan in an expression, the EState's direction
field was left alone, resulting in an attempt to execute the subplan
backwards if it was encountered during a backwards scan of a cursor.
Also, though much less likely, it was possible to reach the execution
of an InitPlan while in backwards-scan state.
Repair by saving/restoring estate->es_direction and forcing forward
scan mode in the relevant places.
Backpatch all the way, since this has been broken since 8.3 (prior to
commit c7ff7663e, SubPlans had their own EStates rather than sharing
the parent plan's, so there was no confusion over scan direction).
Per bug #15336 reported by Vladimir Baranoff; analysis and patch by
me, review by Tom Lane.
Discussion: https://postgr.es/m/153449812167.1304.1741624125628126322@wrigleys.postgresql.org
M src/backend/executor/nodeSubplan.c
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
pg_upgrade: issue helpful error message for use on standbys
commit : 87be73e3ff1cd2e4474f8305a214a9d61535d2fb
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 17 Aug 2018 10:25:48 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 17 Aug 2018 10:25:48 -0400
Commit 777e6ddf1723306bd2bf8fe6f804863f459b0323 checked for a shut down
message from a standby and allowed it to continue. This patch reports a
helpful error message in these cases, suggesting to use rsync as
documented.
Diagnosed-by: Martín Marqués
Discussion: https://postgr.es/m/CAPdiE1xYCow-reLjrhJ9DqrMu-ppNq0ChUUEvVdxhdjGRD5_eA@mail.gmail.com
Backpatch-through: 9.3
M src/bin/pg_upgrade/controldata.c
Mention ownership requirements for REFRESH MATERIALIZED VIEW in docs
commit : 86e873c01650a0d3cd605e897946d36073ca240e
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 17 Aug 2018 11:29:15 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 17 Aug 2018 11:29:15 +0900
Author: Dian Fay
Discussion: https://postgr.es/m/745abbd2-a1a0-ead8-2cb2-768c16747d97@gmail.com
Backpatch-through: 9.3
M doc/src/sgml/ref/refresh_materialized_view.sgml
Proof-reading for documentation.
commit : 3fe8c13a31bcdd0220c0c0891c2a161087600a5d
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 17 Aug 2018 11:38:44 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 17 Aug 2018 11:38:44 +1200
Somebody accidentally a word. Back-patch to 9.6.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20180816195431.GA23707%40telsasoft.com
M doc/src/sgml/parallel.sgml
Close the file descriptor in ApplyLogicalMappingFile
commit : 5257b9bfbafdbb17fa7b3a1d387dfaa7d40626cb
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 16 Aug 2018 16:49:10 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Thu, 16 Aug 2018 16:49:10 +0200
The function was forgetting to close the file descriptor, resulting
in failures like this:
ERROR: 53000: exceeded maxAllocatedDescs (492) while trying to open
file "pg_logical/mappings/map-4000-4eb-1_60DE1E08-5376b5-537c6b"
LOCATION: OpenTransientFile, fd.c:2161
Simply close the file at the end, and backpatch to 9.4 (where logical
decoding was introduced). While at it, fix a nearby typo.
Discussion: https://www.postgresql.org/message-id/flat/738a590a-2ce5-9394-2bef-7b1caad89b37%402ndquadrant.com
M src/backend/replication/logical/reorderbuffer.c
Make snprintf.c follow the C99 standard for snprintf's result value.
commit : c2a2e331da17e86ba874ff9449b1d2c9e7e829fd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 15 Aug 2018 17:25:23 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 15 Aug 2018 17:25:23 -0400
C99 says that the result should be the number of bytes that would have
been emitted given a large enough buffer, not the number we actually
were able to put in the buffer. It's time to make our substitute
implementation comply with that. Not doing so results in inefficiency
in buffer-enlargement cases, and also poses a portability hazard for
third-party code that might expect C99-compliant snprintf behavior
within Postgres.
In passing, remove useless tests for str == NULL; neither C99 nor
predecessor standards ever allowed that except when count == 0,
so I see no reason to expend cycles on making that a non-crash case
for this implementation. Also, don't waste a byte in pg_vfprintf's
local I/O buffer; this might have performance benefits by allowing
aligned writes during flushbuffer calls.
Back-patch of commit 805889d7d. There was some concern about this
possibly breaking code that assumes pre-C99 behavior, but there is
much more risk (and reality, in our own code) of code that assumes
C99 behavior and hence fails to detect buffer overrun without this.
Discussion: https://postgr.es/m/17245.1534289329@sss.pgh.pa.us
M src/port/snprintf.c
Update FSM on WAL replay of page all-visible/frozen
commit : 3cbd190e1579546bd89b650d9d388ae3e80b4eae
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 15 Aug 2018 18:09:29 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 15 Aug 2018 18:09:29 -0300
We aren't very strict about keeping FSM up to date on WAL replay,
because per-page freespace values aren't critical in replicas (can't
write to heap in a replica; and if the replica is promoted, the values
would be updated by VACUUM anyway). However, VACUUM since 9.6 can skip
processing pages marked all-visible or all-frozen, and if such pages are
recorded in FSM with wrong values, those values are blindly propagated
to FSM's upper layers by VACUUM's FreeSpaceMapVacuum. (This rationale
assumes that crashes are not very frequent, because those would cause
outdated FSM to occur in the primary.)
Even when the FSM is outdated in standby, things are not too bad
normally, because, most per-page FSM values will be zero (other than
those propagated with the base-backup that created the standby); only
once the remaining free space is less than 0.2*BLCKSZ the per-page value
is maintained by WAL replay of heap ins/upd/del. However, if
wal_log_hints=on causes complete FSM pages to be propagated to a standby
via full-page images, many too-optimistic per-page values can end up
being registered in the standby.
Incorrect per-page values aren't critical in most cases, since an
inserter that is given a page that doesn't actually contain the claimed
free space will update FSM with the correct value, and retry until it
finds a usable page. However, if there are many such updates to do, an
inserter can spend a long time doing them before a usable page is found;
in a heavily trafficked insert-only table with many concurrent inserters
this has been observed to cause several second stalls, causing visible
application malfunction.
To fix this problem, it seems sufficient to have heap_xlog_visible
(replay of setting all-visible and all-frozen VM bits for a heap page)
update the FSM value for the page being processed. This fixes the
per-page counters together with making the page skippable to vacuum, so
when vacuum does FreeSpaceMapVacuum, the values propagated to FSM upper
layers are the correct ones, avoiding the problem.
While at it, apply the same fix to heap_xlog_clean (replay of tuple
removal by HOT pruning and vacuum). This makes any space freed by the
cleaning available earlier than the next vacuum in the promoted replica.
Backpatch to 9.6, where this problem was diagnosed on an insert-only
table with all-frozen pages, which were introduced as a concept in that
release. Theoretically it could apply with all-visible pages to older
branches, but there's been no report of that and it doesn't backpatch
cleanly anyway.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20180802172857.5skoexsilnjvgruk@alvherre.pgsql
M src/backend/access/heap/heapam.c
Clean up assorted misuses of snprintf()'s result value.
commit : c182c1e0b895c8e6baf7c5a9d3cd98307f420168
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 15 Aug 2018 16:29:32 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 15 Aug 2018 16:29:32 -0400
Fix a small number of places that were testing the result of snprintf()
but doing so incorrectly. The right test for buffer overrun, per C99,
is "result >= bufsize" not "result > bufsize". Some places were also
checking for failure with "result == -1", but the standard only says
that a negative value is delivered on failure.
(Note that this only makes these places correct if snprintf() delivers
C99-compliant results. But at least now these places are consistent
with all the other places where we assume that.)
Also, make psql_start_test() and isolation_start_test() check for
buffer overrun while constructing their shell commands. There seems
like a higher risk of overrun, with more severe consequences, here
than there is for the individual file paths that are made elsewhere
in the same functions, so this seemed like a worthwhile change.
Also fix guc.c's do_serialize() to initialize errno = 0 before
calling vsnprintf. In principle, this should be unnecessary because
vsnprintf should have set errno if it returns a failure indication ...
but the other two places this coding pattern is cribbed from don't
assume that, so let's be consistent.
These errors are all very old, so back-patch as appropriate. I think
that only the shell command overrun cases are even theoretically
reachable in practice, but there's not much point in erroneous error
checks.
Discussion: https://postgr.es/m/17245.1534289329@sss.pgh.pa.us
M src/backend/libpq/ip.c
M src/backend/postmaster/pgstat.c
M src/backend/utils/misc/guc.c
M src/interfaces/ecpg/pgtypeslib/common.c
M src/port/getaddrinfo.c
M src/test/isolation/isolation_main.c
M src/test/regress/pg_regress.c
M src/test/regress/pg_regress_main.c
pg_upgrade: fix shutdown check for standby servers
commit : 54db0e5e17c1d2da3347d40e6d9143938b3c31a4
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 14 Aug 2018 17:19:02 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 14 Aug 2018 17:19:02 -0400
Commit 244142d32afd02e7408a2ef1f249b00393983822 only tested for the
pg_controldata output for primary servers, but standby servers have
different "Database cluster state" output, so check for that too.
Diagnosed-by: Michael Paquier
Discussion: https://postgr.es/m/20180810164240.GM13638@paquier.xyz
Backpatch-through: 9.3
M src/bin/pg_upgrade/controldata.c
Prohibit shutting down resources if there is a possibility of back up.
commit : 69de17186ad9d668bc0500595b47d9a24def238f
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 13 Aug 2018 08:56:37 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 13 Aug 2018 08:56:37 +0530
Currently, we release the asynchronous resources as soon as it is evident
that no more rows will be needed e.g. when a Limit is filled. This can be
problematic especially for custom and foreign scans where we can scan
backward. Fix that by disallowing the shutting down of resources in such
cases.
Reported-by: Robert Haas
Analysed-by: Robert Haas and Amit Kapila
Author: Amit Kapila
Reviewed-by: Robert Haas
Backpatch-through: 9.6 where this code was introduced
Discussion: https://postgr.es/m/86137f17-1dfb-42f9-7421-82fd786b04a1@anayrat.info
M src/backend/executor/execMain.c
M src/backend/executor/nodeLimit.c
docs: Only first instance of a PREPARE parameter sets data type
commit : 0d428b6553bcedf8faf51ed9947c3a59238f8da8
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 9 Aug 2018 10:13:15 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 9 Aug 2018 10:13:15 -0400
If the first reference to $1 is "($1 = col) or ($1 is null)", the data
type can be determined, but not for "($1 is null) or ($1 = col)". This
change documents this.
Reported-by: Morgan Owens
Discussion: https://postgr.es/m/153233728858.1404.15268121695358514937@wrigleys.postgresql.org
Backpatch-through: 9.3
M doc/src/sgml/ref/prepare.sgml
Don't run atexit callbacks in quickdie signal handlers.
commit : 8e4e783ee4f1cb64a46798989e5fd04c7c0a2f53
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 8 Aug 2018 19:08:10 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 8 Aug 2018 19:08:10 +0300
exit() is not async-signal safe. Even if the libc implementation is, 3rd
party libraries might have installed unsafe atexit() callbacks. After
receiving SIGQUIT, we really just want to exit as quickly as possible, so
we don't really want to run the atexit() callbacks anyway.
The original report by Jimmy Yih was a self-deadlock in startup_die().
However, this patch doesn't address that scenario; the signal handling
while waiting for the startup packet is more complicated. But at least this
alleviates similar problems in the SIGQUIT handlers, like that reported
by Asim R P later in the same thread.
Backpatch to 9.3 (all supported versions).
Discussion: https://www.postgresql.org/message-id/CAOMx_OAuRUHiAuCg2YgicZLzPVv5d9_H4KrL_OFsFP%3DVPekigA%40mail.gmail.com
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/bgwriter.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/startup.c
M src/backend/postmaster/walwriter.c
M src/backend/replication/walreceiver.c
M src/backend/tcop/postgres.c
Don't record FDW user mappings as members of extensions.
commit : f3ed5364e666f89d4e2856e20628eae9e5eb2f7c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 7 Aug 2018 16:32:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 7 Aug 2018 16:32:50 -0400
CreateUserMapping has a recordDependencyOnCurrentExtension call that's
been there since extensions were introduced (very possibly my fault).
However, there's no support anywhere else for user mappings as members
of extensions, nor are they listed as a possible member object type in
the documentation. Nor does it really seem like a good idea for user
mappings to belong to extensions when roles don't. Hence, remove the
bogus call.
(As we saw in bug #15310, the lack of any pg_dump support for this case
ensures that any such membership record would silently disappear during
pg_upgrade. So there's probably no need for us to do anything else
about cleaning up after this mistake.)
Discussion: https://postgr.es/m/27952.1533667213@sss.pgh.pa.us
M src/backend/commands/foreigncmds.c
Fix incorrect initialization of BackendActivityBuffer.
commit : f73a31370624d7a67fc9c409df5be3da3778b462
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 7 Aug 2018 16:00:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 7 Aug 2018 16:00:44 -0400
Since commit c8e8b5a6e, this has been zeroed out using the wrong length.
In practice the length would always be too small, leading to not zeroing
the whole buffer rather than clobbering additional memory; and that's
pretty harmless, both because shmem would likely start out as zeroes
and because we'd reinitialize any given entry before use. Still,
it's bogus, so fix it.
Reported by Petru-Florin Mihancea (bug #15312)
Discussion: https://postgr.es/m/153363913073.1303.6518849192351268091@wrigleys.postgresql.org
M src/backend/postmaster/pgstat.c
Fix pg_upgrade to handle event triggers in extensions correctly.
commit : 92d5dd36ecf8a49e400d14bfc8b4e939b585ed2b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 7 Aug 2018 15:43:49 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 7 Aug 2018 15:43:49 -0400
pg_dump with --binary-upgrade must emit ALTER EXTENSION ADD commands
for all objects that are members of extensions. It forgot to do so for
event triggers, as per bug #15310 from Nick Barnes. Back-patch to 9.3
where event triggers were introduced.
Haribabu Kommi
Discussion: https://postgr.es/m/153360083872.1395.4593932457718151600@wrigleys.postgresql.org
M src/bin/pg_dump/pg_dump.c
Ensure pg_dump_sort.c sorts null vs non-null namespace consistently.
commit : 6b6327d938ed61b45eb674912e63372cd349c4a3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 7 Aug 2018 13:13:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 7 Aug 2018 13:13:42 -0400
The original coding here (which is, I believe, my fault) supposed that
it didn't need to concern itself with the possibility that one object
of a given type-priority has a namespace while another doesn't. But
that's not reliably true anymore, if it ever was; and if it does happen
then it's possible that DOTypeNameCompare returns self-inconsistent
comparison results. That leads to unspecified behavior in qsort()
and a resultant weird output order from pg_dump.
This should end up being only a cosmetic problem, because any ordering
constraints that actually matter should be enforced by the later
dependency-based sort. Still, it's a bug, so back-patch.
Report and fix by Jacob Champion, though I editorialized on his
patch to the extent of making NULL sort after non-NULL, for consistency
with our usual sorting definitions.
Discussion: https://postgr.es/m/CABAq_6Hw+V-Kj7PNfD5tgOaWT_-qaYkc+SRmJkPLeUjYXLdxwQ@mail.gmail.com
M src/bin/pg_dump/pg_dump_sort.c