Fix edge case leading to agg transitions skipping ExecAggTransReparent() calls.
commit : f651976d94bfd6fd19c3b27e9649f166030923bd
author : Andres Freund <andres@anarazel.de>
date : Mon, 20 Jan 2020 23:26:51 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 20 Jan 2020 23:26:51 -0800 The code checking whether an aggregate transition value needs to be
reparented into the current context has always only compared the
transition return value with the previous transition value by datum,
i.e. without regard for NULLness. This normally works, because when
the transition function returns NULL (via fcinfo->isnull), it'll
return a value that won't be the same as its input value.
But there's no hard requirement that that's the case. And it turns
out, it's possible to hit this case (see discussion or reproducers),
leading to a non-null transition value not being reparented, followed
by a crash caused by that.
Instead of adding another comparison of NULLness, instead have
ExecAggTransReparent() ensure that pergroup->transValue ends up as 0
when the new transition value is NULL. That avoids having to add an
additional branch to the much more common cases of the transition
function returning the old transition value (which is a pointer in
this case), and when the new value is different, but not NULL.
In branches since 69c3936a149, also deduplicate the reparenting code
between the expression evaluation based transitions, and the path for
ordered aggregates.
Reported-By: Teodor Sigaev, Nikita Glukhov
Author: Andres Freund
Discussion: https://postgr.es/m/bd34e930-cfec-ea9b-3827-a8bc50891393@sigaev.ru
Backpatch: 9.4-, this issue has existed since at least 7.4 M src/backend/executor/nodeAgg.c
Add GUC variables for stat tracking and timeout as PGDLLIMPORT
commit : 4a49149b9d74d2f9479cb73227e9646c901c84e6
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 21 Jan 2020 13:47:13 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 21 Jan 2020 13:47:13 +0900 This helps integration of extensions with Windows. The following
parameters are changed:
- idle_in_transaction_session_timeout (9.6 and newer versions)
- lock_timeout
- statement_timeout
- track_activities
- track_counts
- track_functions
Author: Pascal Legrand
Reviewed-by: Amit Kamila, Julien Rouhaud, Michael Paquier
Discussion: https://postgr.es/m/1579298868581-0.post@n3.nabble.com
Backpatch-through: 9.4 M src/include/pgstat.h
M src/include/storage/proc.h
Fix pg_dump's sigTermHandler() to use _exit() not exit().
commit : b1392a9502016fcd3c93ba18ab418e11424b711e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jan 2020 12:57:17 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jan 2020 12:57:17 -0500 sigTermHandler() tried to be careful to invoke only operations that
are safe to do in a signal handler. But for some reason we forgot
that exit(3) is not among those, because it calls atexit handlers
that might do various random things. (pg_dump itself installs no
atexit handlers, but e.g. OpenSSL does.) That led to crashes or
lockups when attempting to terminate a parallel dump or restore
via a signal.
Fix by calling _exit() instead.
Per bug #16199 from Raúl Marín. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16199-cb2f121146a96f9b@postgresql.org M src/bin/pg_dump/parallel.c
Fix crash in BRIN inclusion op functions, due to missing datum copy.
commit : 98f0d283774b68895bc41413d7dd9c19e5608231
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 20 Jan 2020 10:36:35 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 20 Jan 2020 10:36:35 +0200 The BRIN add_value() and union() functions need to make a longer-lived
copy of the argument, if they want to store it in the BrinValues struct
also passed as argument. The functions for the "inclusion operator
classes" used with box, range and inet types didn't take into account
that the union helper function might return its argument as is, without
making a copy. Check for that case, and make a copy if necessary. That
case arises at least with the range_union() function, when one of the
arguments is an 'empty' range:
CREATE TABLE brintest (n numrange);
CREATE INDEX brinidx ON brintest USING brin (n);
INSERT INTO brintest VALUES ('empty');
INSERT INTO brintest VALUES (numrange(0, 2^1000::numeric));
INSERT INTO brintest VALUES ('(-1, 0)');
SELECT brin_desummarize_range('brinidx', 0);
SELECT brin_summarize_range('brinidx', 0);
Backpatch down to 9.5, where BRIN was introduced.
Discussion: https://www.postgresql.org/message-id/e6e1d6eb-0a67-36aa-e779-bcca59167c14%40iki.fi
Reviewed-by: Emre Hasegeli, Tom Lane, Alvaro Herrera M src/backend/access/brin/brin_inclusion.c
Repair more failures with SubPlans in multi-row VALUES lists.
commit : 3964722780d811430521b6051bc350ead03fb708
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Jan 2020 16:17:18 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Jan 2020 16:17:18 -0500 Commit 9b63c13f0 turns out to have been fundamentally misguided:
the parent node's subPlan list is by no means the only way in which
a child SubPlan node can be hooked into the outer execution state.
As shown in bug #16213 from Matt Jibson, we can also get short-lived
tuple table slots added to the outer es_tupleTable list. At this point
I have little faith that there aren't other possible connections as
well; the long time it took to notice this problem shows that this
isn't a heavily-exercised situation.
Therefore, revert that fix, returning to the coding that passed a
NULL parent plan pointer down to the transiently-built subexpressions.
That gives us a pretty good guarantee that they won't hook into the
outer executor state in any way. But then we need some other solution
to make SubPlans work. Adopt the solution speculated about in the
previous commit's log message: do expression initialization at plan
startup for just those VALUES rows containing SubPlans, abandoning the
goal of reclaiming memory intra-query for those rows. In practice it
seems unlikely that queries containing a vast number of VALUES rows
would be using SubPlans in them, so this should not give up much.
(BTW, this test case also refutes my claim in connection with the prior
commit that the issue only arises with use of LATERAL. That was just
wrong: some variants of SubLink always produce SubPlans.)
As with previous patch, back-patch to all supported branches.
Discussion: https://postgr.es/m/16213-871ac3bc208ecf23@postgresql.org M src/backend/executor/nodeValuesscan.c
M src/include/nodes/execnodes.h
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
Set ReorderBufferTXN->final_lsn more eagerly
commit : 58997ace5b372cc137770292f462d5b8854c832d
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 17 Jan 2020 18:00:39 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 17 Jan 2020 18:00:39 -0300 ... specifically, set it incrementally as each individual change is
spilled down to disk. This way, it is set correctly when the
transaction disappears without trace, ie. without leaving an XACT_ABORT
wal record. (This happens when the server crashes midway through a
transaction.)
Failing to have final_lsn prevents ReorderBufferRestoreCleanup() from
working, since it needs the final_lsn in order to know the endpoint of
its iteration through spilled files.
Commit df9f682c7bf8 already tried to fix the problem, but it didn't set
the final_lsn in all cases. Revert that, since it's no longer needed.
Author: Vignesh C
Reviewed-by: Amit Kapila, Dilip Kumar
Discussion: https://postgr.es/m/CALDaNm2CLk+K9JDwjYST0sPbGg5AQdvhUt0jbKyX_HdAE0jk3A@mail.gmail.com M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h
Make rewriter prevent auto-updates on views with conditional INSTEAD rules.
commit : bb09a9414f1e149aac6064d217b5d5cab8f13723
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 14 Jan 2020 09:48:44 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 14 Jan 2020 09:48:44 +0000 A view with conditional INSTEAD rules and no unconditional INSTEAD
rules or INSTEAD OF triggers is not auto-updatable. Previously we
relied on a check in the executor to catch this, but that's
problematic since the planner may fail to properly handle such a query
and thus return a particularly unhelpful error to the user, before
reaching the executor check.
Instead, trap this in the rewriter and report the correct error there.
Doing so also allows us to include more useful error detail than the
executor check can provide. This doesn't change the existing behaviour
of updatable views; it merely ensures that useful error messages are
reported when a view isn't updatable.
Per report from Pengzhou Tang, though not adopting that suggested fix.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/CAG4reAQn+4xB6xHJqWdtE0ve_WqJkdyCV4P=trYr4Kn8_3_PEA@mail.gmail.com M src/backend/executor/execMain.c
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/updatable_views.out
M src/test/regress/sql/updatable_views.sql
Fix edge-case crashes and misestimation in range containment selectivity.
commit : 784c58da1957cecdea037428ce08953b852be85b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 12 Jan 2020 14:37:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 12 Jan 2020 14:37:00 -0500 When estimating the selectivity of "range_var <@ range_constant" or
"range_var @> range_constant", if the upper (or respectively lower)
bound of the range_constant was above the last bin of the range_var's
histogram, the code would access uninitialized memory and potentially
crash (though it seems the probability of a crash is quite low).
Handle the endpoint cases explicitly to fix that.
While at it, be more paranoid about the possibility of getting NaN
or other silly results from the range type's subdiff function.
And improve some comments.
Ordinarily we'd probably add a regression test case demonstrating
the bug in unpatched code. But it's too hard to get it to crash
reliably because of the uninitialized-memory dependence, so skip that.
Per bug #16122 from Adam Scott. It's been broken from the beginning,
apparently, so backpatch to all supported branches.
Diagnosis by Michael Paquier, patch by Andrey Borodin and Tom Lane.
Discussion: https://postgr.es/m/16122-eb35bc248c806c15@postgresql.org M src/backend/utils/adt/rangetypes_selfuncs.c
doc: Fix naming of SELinux
commit : 8d55879f089563cf1d62266d921c0a00d2b9fff8
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 10 Jan 2020 09:37:38 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 10 Jan 2020 09:37:38 +0900 Reported-by: Tham Nguyen
Discussion: https://postgr.es/m/157851402876.29175.12977878383183540468@wrigleys.postgresql.org
Backpatch-through: 9.4 M doc/src/sgml/ref/security_label.sgml
M src/test/modules/dummy_seclabel/README
Revert "Forbid DROP SCHEMA on temporary namespaces"
commit : 86949c2f120792a6cfcafee1f30cf4231085b6d8
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 8 Jan 2020 10:36:46 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 8 Jan 2020 10:36:46 +0900 This reverts commit a052f6c, following complains from Robert Haas and
Tom Lane. Backpatch down to 9.4, like the previous commit.
Discussion: https://postgr.es/m/CA+TgmobL4npEX5=E5h=5Jm_9mZun3MT39Kq2suJFVeamc9skSQ@mail.gmail.com
Backpatch-through: 9.4 M src/backend/commands/dropcmds.c
Fix running out of file descriptors for spill files.
commit : a6f4f407ada026007d37ab9d494bb6380ec12527
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 2 Jan 2020 12:28:02 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 2 Jan 2020 12:28:02 +0530 Currently while decoding changes, if the number of changes exceeds a
certain threshold, we spill those to disk. And this happens for each
(sub)transaction. Now, while reading all these files, we don't close them
until we read all the files. While reading these files, if the number of
such files exceeds the maximum number of file descriptors, the operation
errors out.
Use PathNameOpenFile interface to open these files as that internally has
the mechanism to release kernel FDs as needed to get us under the
max_safe_fds limit.
Reported-by: Amit Khandekar
Author: Amit Khandekar
Reviewed-by: Amit Kapila
Backpatch-through: 9.4
Discussion: https://postgr.es/m/CAJ3gD9c-sECEn79zXw4yBnBdOttacoE-6gAyP0oy60nfs_sabQ@mail.gmail.com M src/backend/replication/logical/reorderbuffer.c
Update copyrights for 2020
commit : 78b381c5ddadb25e915ee4f380c28a676851998c
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 1 Jan 2020 12:21:44 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 1 Jan 2020 12:21:44 -0500 Backpatch-through: update all files in master, backpatch legal files through 9.4 M COPYRIGHT
M doc/src/sgml/legal.sgml
doc: add examples of creative use of unique expression indexes
commit : 937cf1b09146ef91195780f747900cf69e8d0fd8
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 27 Dec 2019 14:49:08 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 27 Dec 2019 14:49:08 -0500 Unique expression indexes can constrain data in creative ways, so show
two examples.
Reported-by: Tuomas Leikola
Discussion: https://postgr.es/m/156760275564.1127.12321702656456074572@wrigleys.postgresql.org
Backpatch-through: 9.4 M doc/src/sgml/indices.sgml
docs: clarify infinite range values from data-type infinities
commit : ebd3c132f5304f351e43e34a4565233e2587154f
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 27 Dec 2019 14:33:30 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 27 Dec 2019 14:33:30 -0500 The previous docs referenced these distinct ideas confusingly.
Reported-by: Eugen Konkov
Discussion: https://postgr.es/m/376945611.20191026161529@yandex.ru
Backpatch-through: 9.4 M doc/src/sgml/rangetypes.sgml
Forbid DROP SCHEMA on temporary namespaces
commit : 12cb5478a2a451fd291a535b8b1b387b3a81914a
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 27 Dec 2019 17:59:32 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 27 Dec 2019 17:59:32 +0900 This operation was possible for the owner of the schema or a superuser.
Down to 9.4, doing this operation would cause inconsistencies in a
session whose temporary schema was dropped, particularly if trying to
create new temporary objects after the drop. A more annoying
consequence is a crash of autovacuum on an assertion failure when
logging information about an orphaned temp table dropped. Note that
because of 246a6c8 (present in v11~), which has made the removal of
orphaned temporary tables more aggressive, the failure could be
triggered more easily, but it is possible to reproduce down to 9.4.
Reported-by: Mahendra Singh, Prabhat Sahu
Author: Michael Paquier
Reviewed-by: Kyotaro Horiguchi, Mahendra Singh
Discussion: https://postgr.es/m/CAKYtNAr9Zq=1-ww4etHo-VCC-k120YxZy5OS01VkaLPaDbv2tg@mail.gmail.com
Backpatch-through: 9.4 M src/backend/commands/dropcmds.c
Rotate instead of shifting hash join batch number.
commit : 893eaf0be8be32f1d6ee364d5d9e2dae0d87ebfd
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 24 Dec 2019 11:31:24 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 24 Dec 2019 11:31:24 +1300 Our algorithm for choosing batch numbers turned out not to work
effectively for multi-billion key inner relations. We would use
more hash bits than we have, and effectively concentrate all tuples
into a smaller number of batches than we intended. While ideally
we should switch to wider hashes, for now, change the algorithm to
one that effectively gives up bits from the bucket number when we
don't have enough bits. That means we'll finish up with longer
bucket chains than would be ideal, but that's better than having
batches that don't fit in work_mem and can't be divided.
Batch-patch to all supported releases.
Author: Thomas Munro
Reviewed-by: Tom Lane, thanks also to Tomas Vondra, Alvaro Herrera, Andres Freund for testing and discussion
Reported-by: James Coleman
Discussion: https://postgr.es/m/16104-dc11ed911f1ab9df%40postgresql.org M src/backend/executor/nodeHash.c
Disallow null category in crosstab_hash
commit : 70fc6c4ef3c3790284796db3935d6f2d040b60bd
author : Joe Conway <mail@joeconway.com>
date : Mon, 23 Dec 2019 13:34:05 -0500
committer: Joe Conway <mail@joeconway.com>
date : Mon, 23 Dec 2019 13:34:05 -0500 While building a hash map of categories in load_categories_hash,
resulting category names have not thus far been checked to ensure
they are not null. Prior to pg12 null category names worked to the
extent that they did not crash on some platforms. This is because
those system libraries have an snprintf which can deal with being
passed a null pointer argument for a string. But even in those cases
null categories did nothing useful. And on some platforms it crashed.
As of pg12, our own version of snprintf gets called, and it does
not deal with null pointer arguments at all, and crashes consistently.
Fix that by disallowing null categories. They never worked usefully,
and no one has ever asked for them to work previously. Back-patch to
all supported branches.
Reported-By: Ireneusz Pluta
Discussion: https://postgr.es/m/16176-7489719b05e4303c@postgresql.org M contrib/tablefunc/tablefunc.c
Prevent a rowtype from being included in itself via a range.
commit : 6609c3ad984bde125ff6ec4e3e056a5b23f68210
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Dec 2019 12:08:24 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Dec 2019 12:08:24 -0500 We probably should have thought of this case when ranges were added,
but we didn't. (It's not the fault of commit eb51af71f, because
ranges didn't exist then.)
It's an old bug, so back-patch to all supported branches.
Discussion: https://postgr.es/m/7782.1577051475@sss.pgh.pa.us M src/backend/catalog/heap.c
M src/test/regress/expected/rangetypes.out
M src/test/regress/sql/rangetypes.sql
Avoid low-probability regression test failures in timestamp[tz] tests.
commit : 365052abbc114a4269405bd7ac97b4b86dcfdca7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 22 Dec 2019 18:00:18 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 22 Dec 2019 18:00:18 -0500 If the first transaction block in these tests were entered exactly
at midnight (California time), they'd report a bogus failure due
to 'now' and 'midnight' having the same values. Commit 8c2ac75c5
had dismissed this as being of negligible probability, but we've
now seen it happen in the buildfarm, so let's prevent it. We can
get pretty much the same test coverage without an it's-not-midnight
assumption by moving the does-'now'-work cases into their own test step.
While here, apply commit 47169c255's s/DELETE/TRUNCATE/ change to
timestamptz as well as timestamp (not sure why that didn't
occur to me at the time; the risk of failure is the same).
Back-patch to all supported branches, since the main point is
to get rid of potential buildfarm failures.
Discussion: https://postgr.es/m/14821.1577031117@sss.pgh.pa.us M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
In pgwin32_open, loop after ERROR_ACCESS_DENIED only if we can't stat.
commit : 35b28d98335e704a8f3fffb241e837daec95533a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 21 Dec 2019 17:39:37 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 21 Dec 2019 17:39:37 -0500 This fixes a performance problem introduced by commit 6d7547c21.
ERROR_ACCESS_DENIED is returned in some other cases besides the
delete-pending case considered by that commit; notably, if the
given path names a directory instead of a plain file. In that
case we'll uselessly loop for 1 second before returning the
failure condition. That slows down some usage scenarios enough
to cause test timeout failures on our Windows buildfarm critters.
To fix, try to stat() the file, and sleep/loop only if that fails.
It will fail in the delete-pending case, and also in the case where
the deletion completed before we could stat(), so we have the cases
where we want to loop covered. In the directory case, the stat()
should succeed, letting us exit without a wait.
One case where we'll still wait uselessly is if the access-denied
problem pertains to a directory in the given pathname. But we don't
expect that to happen in any performance-critical code path.
There might be room to refine this further, but I'll push it now
in hopes of making the buildfarm green again.
Back-patch, like the preceding commit.
Alexander Lakhin and Tom Lane
Discussion: https://postgr.es/m/23073.1576626626@sss.pgh.pa.us M src/port/open.c
docs: clarify handling of column lists in COPY TO/FROM
commit : dd4c4eaab724835f4553656efb6451b79ffbb9e1
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 21 Dec 2019 12:44:38 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 21 Dec 2019 12:44:38 -0500 Previously it was unclear how COPY FROM handled cases where not all
columns were specified, or if the order didn't match.
Reported-by: pavlo.golub@gmail.com
Discussion: https://postgr.es/m/157487729344.7213.14245726713444755296@wrigleys.postgresql.org
Backpatch-through: 9.4 M doc/src/sgml/ref/copy.sgml
libpq should expose GSS-related parameters even when not implemented.
commit : 5e22a111185ccf5fe1d84b980d947b5427a50e64
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 20 Dec 2019 15:34:08 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 20 Dec 2019 15:34:08 -0500 We realized years ago that it's better for libpq to accept all
connection parameters syntactically, even if some are ignored or
restricted due to lack of the feature in a particular build.
However, that lesson from the SSL support was for some reason never
applied to the GSSAPI support. This is causing various buildfarm
members to have problems with a test case added by commit 6136e94dc,
and it's just a bad idea from a user-experience standpoint anyway,
so fix it.
While at it, fix some places where parameter-related infrastructure
was added with the aid of a dartboard, or perhaps with the aid of
the anti-pattern "add new stuff at the end". It should be safe
to rearrange the contents of struct pg_conn even in released
branches, since that's private to libpq (and we'd have to move
some fields in some builds to fix this, anyway).
Back-patch to all supported branches.
Discussion: https://postgr.es/m/11297.1576868677@sss.pgh.pa.us M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/libpq-int.h
Fix error reporting for index expressions of prohibited types.
commit : da5dd421833b2cd074cb752dd8ecfbe6a4735a4f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Dec 2019 17:44:28 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Dec 2019 17:44:28 -0500 If CheckAttributeType() threw an error about the datatype of an
index expression column, it would report an empty column name,
which is pretty unhelpful and certainly not the intended behavior.
I (tgl) evidently broke this in commit cfc5008a5, by not noticing
that the column's attname was used above where I'd placed the
assignment of it.
In HEAD and v12, this is trivially fixable by moving up the
assignment of attname. Before v12 the code is a bit more messy;
to avoid doing substantial refactoring, I took the lazy way out
and just put in two copies of the assignment code.
Report and patch by Amit Langote. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/CA+HiwqFA+BGyBFimjiYXXMa2Hc3fcL0+OJOyzUNjhU4NCa_XXw@mail.gmail.com M src/backend/catalog/index.c
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
On Windows, wait a little to see if ERROR_ACCESS_DENIED goes away.
commit : cd03803512bf9484d2a045bf877a8ddc2cb47059
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Dec 2019 15:10:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Dec 2019 15:10:55 -0500 Attempting to open a file fails with ERROR_ACCESS_DENIED if the file
is flagged for deletion but not yet actually gone (another in a long
list of reasons why Windows is broken, if you ask me). This seems
likely to explain a lot of irreproducible failures we see in the
buildfarm. This state generally persists for only a millisecond or so,
so just wait a bit and retry. If it's a real permissions problem,
we'll eventually give up and report it as such. If it's the pending
deletion case, we'll see file-not-found and report that after the
deletion completes, and the caller will treat that in an appropriate
way.
In passing, rejigger the existing retry logic for some other error
cases so that we don't uselessly wait an extra time when we're
not going to retry anymore.
Alexander Lakhin (with cosmetic tweaks by me). Back-patch to all
supported branches, since this seems like a pretty safe change and
the problem is definitely real.
Discussion: https://postgr.es/m/16161-7a985d2f1bbe8f71@postgresql.org M src/port/open.c
Fix EXTRACT(ISOYEAR FROM timestamp) for years BC.
commit : 323c47925af9e53c4c0476553e3e13a86f3d8173
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 12 Dec 2019 12:30:44 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 12 Dec 2019 12:30:44 -0500 The test cases added by commit 26ae3aa80 exposed an old oversight in
timestamp[tz]_part: they didn't correct the result of date2isoyear()
for BC years, so that we produced an off-by-one answer for such years.
Fix that, and back-patch to all supported branches.
Discussion: https://postgr.es/m/SG2PR06MB37762CAE45DB0F6CA7001EA9B6550@SG2PR06MB3776.apcprd06.prod.outlook.com M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
Remove redundant function calls in timestamp[tz]_part().
commit : e284e0e2f9da268754ee0deb814b5a7310db0d0e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 12 Dec 2019 12:12:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 12 Dec 2019 12:12:36 -0500 The DTK_DOW/DTK_ISODOW and DTK_DOY switch cases in timestamp_part() and
timestamptz_part() contained calls of timestamp2tm() that were fully
redundant with the ones done just above the switch. This evidently crept
in during commit 258ee1b63, which relocated that code from another place
where the calls were indeed needed. Just delete the redundant calls.
I (tgl) noted that our test coverage of these functions left quite a
bit to be desired, so extend timestamp.sql and timestamptz.sql to
cover all the branches.
Back-patch to all supported branches, as the previous commit was.
There's no real issue here other than some wasted cycles in some
not-too-heavily-used code paths, but the test coverage seems valuable.
Report and patch by Li Japin; test case adjustments by me.
Discussion: https://postgr.es/m/SG2PR06MB37762CAE45DB0F6CA7001EA9B6550@SG2PR06MB3776.apcprd06.prod.outlook.com M src/backend/utils/adt/timestamp.c
M src/test/regress/expected/timestamp.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamp.sql
M src/test/regress/sql/timestamptz.sql
Doc: back-patch documentation about limitations of CHECK constraints.
commit : 6bf23e8c6e066fb7554b9076355afdfc103ad134
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 11 Dec 2019 15:53:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 11 Dec 2019 15:53:36 -0500 Back-patch commits 36d442a25 and 1f66c657f into all supported
branches. I'd considered doing this when putting in the latter
commit, but failed to pull the trigger. Now that we've had an
actual field complaint about the lack of such docs, let's do it.
Per bug #16158 from Piotr Jander. Original patches by Lætitia Avrot,
Patrick Francelle, and me.
Discussion: https://postgr.es/m/16158-7ccf2f74b3d655db@postgresql.org M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/alter_domain.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_domain.sgml
M doc/src/sgml/ref/create_table.sgml
Fix race condition in our Windows signal emulation.
commit : 1a0c65120137786bb2667cb935954b2982d5d96f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Dec 2019 15:03:52 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Dec 2019 15:03:52 -0500 pg_signal_dispatch_thread() responded to the client (signal sender)
and disconnected the pipe before actually setting the shared variables
that make the signal visible to the backend process's main thread.
In the worst case, it seems, effective delivery of the signal could be
postponed for as long as the machine has any other work to do.
To fix, just move the pg_queue_signal() call so that we do it before
responding to the client. This essentially makes pgkill() synchronous,
which is a stronger guarantee than we have on Unix. That may be
overkill, but on the other hand we have not seen comparable timing bugs
on any Unix platform.
While at it, add some comments to this sadly underdocumented code.
Problem diagnosis and fix by Amit Kapila; I just added the comments.
Back-patch to all supported versions, as it appears that this can cause
visible NOTIFY timing oddities on all of them, and there might be
other misbehavior due to slow delivery of other signals.
Discussion: https://postgr.es/m/32745.1575303812@sss.pgh.pa.us M src/backend/port/win32/signal.c
Document search_path security with untrusted dbowner or CREATEROLE.
commit : 3056258149c1aea7341a4d81bd502e1a1c8198a6
author : Noah Misch <noah@leadboat.com>
date : Sun, 8 Dec 2019 11:06:26 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sun, 8 Dec 2019 11:06:26 -0800 Commit 5770172cb0c9df9e6ce27c507b449557e5b45124 wrote, incorrectly, that
certain schema usage patterns are secure against CREATEROLE users and
database owners. When an untrusted user is the database owner or holds
CREATEROLE privilege, a query is secure only if its session started with
SELECT pg_catalog.set_config('search_path', '', false) or equivalent.
Back-patch to 9.4 (all supported versions).
Discussion: https://postgr.es/m/20191013013512.GC4131753@rfd.leadboat.com M doc/src/sgml/ddl.sgml
Ensure maxlen is at leat 1 in dict_int
commit : a2fdeb7863a684b661b0fcbaf90f00595be11bd0
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 3 Dec 2019 16:55:51 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 3 Dec 2019 16:55:51 +0100 The dict_int text search dictionary template accepts maxlen parameter,
which is then used to cap the length of input strings. The value was
not properly checked, and the code simply does
txt[d->maxlen] = '\0';
to insert a terminator, leading to segfaults with negative values.
This commit simply rejects values less than 1. The issue was there since
dct_int was introduced in 9.3, so backpatch all the way back to 9.4
which is the oldest supported version.
Reported-by: cili
Discussion: https://postgr.es/m/16144-a36a5bef7657047d@postgresql.org
Backpatch-through: 9.4 M contrib/dict_int/dict_int.c
M contrib/dict_int/expected/dict_int.out
M contrib/dict_int/sql/dict_int.sql
Fix misbehavior with expression indexes on ON COMMIT DELETE ROWS tables.
commit : cfffa8a6b22d8813c30fd524affc07c0c580ea58
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Dec 2019 13:09:27 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Dec 2019 13:09:27 -0500 We implement ON COMMIT DELETE ROWS by truncating tables marked that
way, which requires also truncating/rebuilding their indexes. But
RelationTruncateIndexes asks the relcache for up-to-date copies of any
index expressions, which may cause execution of eval_const_expressions
on them, which can result in actual execution of subexpressions.
This is a bad thing to have happening during ON COMMIT. Manuel Rigger
reported that use of a SQL function resulted in crashes due to
expectations that ActiveSnapshot would be set, which it isn't.
The most obvious fix perhaps would be to push a snapshot during
PreCommit_on_commit_actions, but I think that would just open the door
to more problems: CommitTransaction explicitly expects that no
user-defined code can be running at this point.
Fortunately, since we know that no tuples exist to be indexed, there
seems no need to use the real index expressions or predicates during
RelationTruncateIndexes. We can set up dummy index expressions
instead (we do need something that will expose the right data type,
as there are places that build index tupdescs based on this), and
just ignore predicates and exclusion constraints.
In a green field it'd likely be better to reimplement ON COMMIT DELETE
ROWS using the same "init fork" infrastructure used for unlogged
relations. That seems impractical without catalog changes though,
and even without that it'd be too big a change to back-patch.
So for now do it like this.
Per private report from Manuel Rigger. This has been broken forever,
so back-patch to all supported branches. M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/backend/utils/cache/relcache.c
M src/include/catalog/index.h
M src/include/utils/relcache.h
M src/test/regress/expected/temp.out
M src/test/regress/sql/temp.sql
Fix off-by-one error in PGTYPEStimestamp_fmt_asc
commit : a17602de18f7ff7821054b3b7f51eadf9fb59d62
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 30 Nov 2019 14:51:27 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 30 Nov 2019 14:51:27 +0100 When using %b or %B patterns to format a date, the code was simply using
tm_mon as an index into array of month names. But that is wrong, because
tm_mon is 1-based, while array indexes are 0-based. The result is we
either use name of the next month, or a segfault (for December).
Fix by subtracting 1 from tm_mon for both patterns, and add a regression
test triggering the issue. Backpatch to all supported versions (the bug
is there far longer, since at least 2003).
Reported-by: Paul Spencer
Backpatch-through: 9.4
Discussion: https://postgr.es/m/16143-0d861eb8688d3fef%40postgresql.org M src/interfaces/ecpg/pgtypeslib/timestamp.c
M src/interfaces/ecpg/test/expected/pgtypeslib-dt_test.c
M src/interfaces/ecpg/test/expected/pgtypeslib-dt_test.stderr
M src/interfaces/ecpg/test/expected/pgtypeslib-dt_test.stdout
M src/interfaces/ecpg/test/pgtypeslib/dt_test.pgc
Fix typo in comment.
commit : 5f55e4c061d376304f979fef0f9aebae94e286bf
author : Etsuro Fujita <efujita@postgresql.org>
date : Wed, 27 Nov 2019 16:00:53 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Wed, 27 Nov 2019 16:00:53 +0900 M src/backend/optimizer/util/relnode.c
Avoid assertion failure with LISTEN in a serializable transaction.
commit : 864e8080e19007ed3377d679447d2fc06f148bc8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 24 Nov 2019 15:57:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 24 Nov 2019 15:57:32 -0500 If LISTEN is the only action in a serializable-mode transaction,
and the session was not previously listening, and the notify queue
is not empty, predicate.c reported an assertion failure. That
happened because we'd acquire the transaction's initial snapshot
during PreCommit_Notify, which was called *after* predicate.c
expects any such snapshot to have been established.
To fix, just swap the order of the PreCommit_Notify and
PreCommit_CheckForSerializationFailure calls during CommitTransaction.
This will imply holding the notify-insertion lock slightly longer,
but the difference could only be meaningful in serializable mode,
which is an expensive option anyway.
It appears that this is just an assertion failure, with no
consequences in non-assert builds. A snapshot used only to scan
the notify queue could not have been involved in any serialization
conflicts, so there would be nothing for
PreCommit_CheckForSerializationFailure to do except assign it a
prepareSeqNo and set the SXACT_FLAG_PREPARED flag. And given no
conflicts, neither of those omissions affect the behavior of
ReleasePredicateLocks. This admittedly once-over-lightly analysis
is backed up by the lack of field reports of trouble.
Per report from Mark Dilger. The bug is old, so back-patch to all
supported branches; but the new test case only goes back to 9.6,
for lack of adequate isolationtester infrastructure before that.
Discussion: https://postgr.es/m/3ac7f397-4d5f-be8e-f354-440020675694@gmail.com
Discussion: https://postgr.es/m/13881.1574557302@sss.pgh.pa.us M src/backend/access/transam/xact.c
Defend against self-referential views in relation_is_updatable().
commit : bcd541897fcf89b60494700f1f15ff70fae5ced7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 21 Nov 2019 16:21:44 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 21 Nov 2019 16:21:44 -0500 While a self-referential view doesn't actually work, it's possible
to create one, and it turns out that this breaks some of the
information_schema views. Those views call relation_is_updatable(),
which neglected to consider the hazards of being recursive. In
older PG versions you get a "stack depth limit exceeded" error,
but since v10 it'd recurse to the point of stack overrun and crash,
because commit a4c35ea1c took out the expression_returns_set() call
that was incidentally checking the stack depth.
Since this function is only used by information_schema views, it
seems like it'd be better to return "not updatable" than suffer
an error. Hence, add tracking of what views we're examining,
in just the same way that the nearby fireRIRrules() code detects
self-referential views. I added a check_stack_depth() call too,
just to be defensive.
Per private report from Manuel Rigger. Back-patch to all
supported versions. M src/backend/rewrite/rewriteHandler.c
M src/backend/utils/adt/misc.c
M src/include/rewrite/rewriteHandler.h
Remove incorrect markup
commit : 31d3da740cf888b7d59ab7ea82e329fe536a38d6
author : Magnus Hagander <magnus@hagander.net>
date : Wed, 20 Nov 2019 17:03:07 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Wed, 20 Nov 2019 17:03:07 +0100 Author: Daniel Gustafsson <daniel@yesql.se> M doc/src/sgml/libpq.sgml
Revise GIN README
commit : 8165384babd96258ed521c4ea2567c70a1849948
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 19 Nov 2019 23:11:24 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 19 Nov 2019 23:11:24 +0300 We find GIN concurrency bugs from time to time. One of the problems here is
that concurrency of GIN isn't well-documented in README. So, it might be even
hard to distinguish design bugs from implementation bugs.
This commit revised concurrency section in GIN README providing more details.
Some examples are illustrated in ASCII art.
Also, this commit add the explanation of how is tuple layout in internal GIN
B-tree page different in comparison with nbtree.
Discussion: https://postgr.es/m/CAPpHfduXR_ywyaVN4%2BOYEGaw%3DcPLzWX6RxYLBncKw8de9vOkqw%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Peter Geoghegan
Backpatch-through: 9.4 M src/backend/access/gin/README
Fix traversing to the deleted GIN page via downlink
commit : 4fc4856849deb51c8d8822d79b6f10d571dab4f6
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 19 Nov 2019 23:08:14 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 19 Nov 2019 23:08:14 +0300 Current GIN code appears to don't handle traversing to the deleted page via
downlink. This commit fixes that by stepping right from the delete page like
we do in nbtree.
This commit also fixes setting 'deleted' flag to the GIN pages. Now other page
flags are not erased once page is deleted. That helps to keep our assertions
true if we arrive deleted page via downlink.
Discussion: https://postgr.es/m/CAPpHfdvMvsw-NcE5bRS7R1BbvA4BxoDnVVjkXC5W0Czvy9LVrg%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Peter Geoghegan
Backpatch-through: 9.4 M src/backend/access/gin/ginbtree.c
M src/backend/access/gin/gindatapage.c
M src/backend/access/gin/ginvacuum.c
M src/backend/access/gin/ginxlog.c
Doc: clarify use of RECURSIVE in WITH.
commit : 4871270e38e5ab0490b78c37ea7d4ccf3240edce
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Nov 2019 14:43:37 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Nov 2019 14:43:37 -0500 Apparently some people misinterpreted the syntax as being that
RECURSIVE is a prefix of individual WITH queries. It's a modifier
for the WITH clause as a whole, so state that more clearly.
Discussion: https://postgr.es/m/ca53c6ce-a0c6-b14a-a8e3-162f0b2cc119@a-kretschmer.de M doc/src/sgml/ref/select.sgml
Doc: clarify behavior of ALTER DEFAULT PRIVILEGES ... IN SCHEMA.
commit : de8c2d38fddc4be2a4cd0c5108b428daa5cc781e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Nov 2019 14:21:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Nov 2019 14:21:42 -0500 The existing text stated that "Default privileges that are specified
per-schema are added to whatever the global default privileges are for
the particular object type". However, that bare-bones observation is
not quite clear enough, as demonstrated by the complaint in bug #16124.
Flesh it out by stating explicitly that you can't revoke built-in
default privileges this way, and by providing an example to drive
the point home.
Back-patch to all supported branches, since it's been like this
from the beginning.
Discussion: https://postgr.es/m/16124-423d8ee4358421bc@postgresql.org M doc/src/sgml/ref/alter_default_privileges.sgml
Further fix dumping of views that contain just VALUES(...).
commit : ecb533af623738f49a8bd748e688c7f02c1c0f9f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 16 Nov 2019 20:00:20 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 16 Nov 2019 20:00:20 -0500 It turns out that commit e9f1c01b7 missed a case: we must print a
VALUES clause in long format if get_query_def is given a resultDesc
that would require the query's output column name(s) to be different
from what the bare VALUES clause would produce.
This applies in case an ALTER ... RENAME COLUMN has been done to
a view that formerly could be printed in simple format, as shown
in the added regression test case. It also explains bug #16119
from Dmitry Telpt, because it turns out that (unlike CREATE VIEW)
CREATE MATERIALIZED VIEW fails to apply any column aliases it's
given to the stored ON SELECT rule. So to get them to be printed,
we have to account for the resultDesc renaming. It might be worth
changing the matview code so that it creates the ON SELECT rule
with the correct aliases; but we'd still need these messy checks in
get_simple_values_rte to handle the case of a subsequent column
rename, so any such change would be just neatnik-ism not a bug fix.
Like the previous patch, back-patch to all supported branches.
Discussion: https://postgr.es/m/16119-e64823f30a45a754@postgresql.org M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/rules.out
M src/test/regress/sql/rules.sql
Handle arrays and ranges in pg_upgrade's test for non-upgradable types.
commit : fb26754af4da8bdb25ca3bc8841714c4101c4107
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Nov 2019 11:35:37 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Nov 2019 11:35:37 -0500 pg_upgrade needs to check whether certain non-upgradable data types
appear anywhere on-disk in the source cluster. It knew that it has
to check for these types being contained inside domains and composite
types; but it somehow overlooked that they could be contained in
arrays and ranges, too. Extend the existing recursive-containment
query to handle those cases.
We probably should have noticed this oversight while working on
commit 0ccfc2822 and follow-ups, but we failed to :-(. The whole
thing's possibly a bit overdesigned, since we don't really expect
that any of these types will appear on disk; but if we're going to
the effort of doing a recursive search then it's silly not to cover
all the possibilities.
While at it, refactor so that we have only one copy of the search
logic, not three-and-counting. Also, to keep the branches looking
more alike, back-patch the output wording change of commit 1634d3615.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/31473.1573412838@sss.pgh.pa.us M src/bin/pg_upgrade/version.c