Stamp 12.6.
commit : 1b9eb7cde7e9cdacd62d206b75ab66bfe6a9a0a4
author : Tom Lane <[email protected]>
date : Mon, 8 Feb 2021 16:56:03 -0500
committer: Tom Lane <[email protected]>
date : Mon, 8 Feb 2021 16:56:03 -0500
M configure
M configure.in
M src/include/pg_config.h.win32
M src/interfaces/libpq/libpq.rc.in
M src/port/win32ver.rc
Translation updates
commit : 73844356cfab995b9314f2a66e1de38bc8365f3f
author : Peter Eisentraut <[email protected]>
date : Mon, 8 Feb 2021 17:50:22 +0100
committer: Peter Eisentraut <[email protected]>
date : Mon, 8 Feb 2021 17:50:22 +0100
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 08f1c10dca3d7b8efc365107c737b87c1c3a82ee
M src/backend/po/de.po
M src/backend/po/fr.po
M src/backend/po/ja.po
M src/backend/po/ru.po
M src/bin/pg_checksums/po/de.po
M src/bin/pg_checksums/po/fr.po
M src/bin/pg_checksums/po/ru.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_rewind/po/ru.po
M src/bin/psql/po/de.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ru.po
M src/interfaces/libpq/po/ru.po
M src/pl/plpgsql/src/po/ru.po
Last-minute updates for release notes.
commit : 392c530d10352978bb3701b7eb425626dc9a4d3e
author : Tom Lane <[email protected]>
date : Mon, 8 Feb 2021 11:10:40 -0500
committer: Tom Lane <[email protected]>
date : Mon, 8 Feb 2021 11:10:40 -0500
Security: CVE-2021-3393, CVE-2021-20229
M doc/src/sgml/release-12.sgml
Fix permission checks on constraint violation errors on partitions.
commit : f50e888990d3fa197e588991b637f9e43e56e53f
author : Heikki Linnakangas <[email protected]>
date : Mon, 8 Feb 2021 11:01:51 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 8 Feb 2021 11:01:51 +0200
If a cross-partition UPDATE violates a constraint on the target partition,
and the columns in the new partition are in different physical order than
in the parent, the error message can reveal columns that the user does not
have SELECT permission on. A similar bug was fixed earlier in commit
804b6b6db4.
The cause of the bug is that the callers of the
ExecBuildSlotValueDescription() function got confused when constructing
the list of modified columns. If the tuple was routed from a parent, we
converted the tuple to the parent's format, but the list of modified
columns was grabbed directly from the child's RTE entry.
ExecUpdateLockMode() had a similar issue. That lead to confusion on which
columns are key columns, leading to wrong tuple lock being taken on tables
referenced by foreign keys, when a row is updated with INSERT ON CONFLICT
UPDATE. A new isolation test is added for that corner case.
With this patch, the ri_RangeTableIndex field is no longer set for
partitions that don't have an entry in the range table. Previously, it was
set to the RTE entry of the parent relation, but that was confusing.
NOTE: This modifies the ResultRelInfo struct, replacing the
ri_PartitionRoot field with ri_RootResultRelInfo. That's a bit risky to
backpatch, because it breaks any extensions accessing the field. The
change that ri_RangeTableIndex is not set for partitions could potentially
break extensions, too. The ResultRelInfos are visible to FDWs at least,
and this patch required small changes to postgres_fdw. Nevertheless, this
seem like the least bad option. I don't think these fields widely used in
extensions; I don't think there are FDWs out there that uses the FDW
"direct update" API, other than postgres_fdw. If there is, you will get a
compilation error, so hopefully it is caught quickly.
Backpatch to 11, where support for both cross-partition UPDATEs, and unique
indexes on partitioned tables, were added.
Reviewed-by: Amit Langote
Security: CVE-2021-3393
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/access/common/tupconvert.c
M src/backend/commands/copy.c
M src/backend/commands/explain.c
M src/backend/commands/trigger.c
M src/backend/executor/execMain.c
M src/backend/executor/execPartition.c
M src/backend/executor/execUtils.c
M src/backend/executor/nodeModifyTable.c
M src/include/access/tupconvert.h
M src/include/executor/executor.h
M src/include/nodes/execnodes.h
A src/test/isolation/expected/tuplelock-partition.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/tuplelock-partition.spec
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Release notes for 13.2, 12.6, 11.11, 10.16, 9.6.21, 9.5.25.
commit : e474abdadd84490544517e846f7eb45f74c8f316
author : Tom Lane <[email protected]>
date : Sun, 7 Feb 2021 15:46:38 -0500
committer: Tom Lane <[email protected]>
date : Sun, 7 Feb 2021 15:46:38 -0500
M doc/src/sgml/release-12.sgml
Revert "Propagate CTE property flags when copying a CTE list into a rule."
commit : 46f2971135984a2666a279faab8b61aa1f1904bd
author : Tom Lane <[email protected]>
date : Sun, 7 Feb 2021 12:54:08 -0500
committer: Tom Lane <[email protected]>
date : Sun, 7 Feb 2021 12:54:08 -0500
This reverts commit ed290896335414c6c069b9ccae1f3dcdd2fac6ba and
equivalent back-branch commits. The issue is subtler than I thought,
and it's far from new, so just before a release deadline is no time
to be fooling with it. We'll consider what to do at a bit more
leisure.
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Propagate CTE property flags when copying a CTE list into a rule.
commit : 2a7664a79c1668f83a658209c10bad6ebdc34a6b
author : Tom Lane <[email protected]>
date : Sat, 6 Feb 2021 19:28:39 -0500
committer: Tom Lane <[email protected]>
date : Sat, 6 Feb 2021 19:28:39 -0500
rewriteRuleAction() neglected this step, although it was careful to
propagate other similar flags such as hasSubLinks or hasRowSecurity.
Omitting to transfer hasRecursive is just cosmetic at the moment,
but omitting hasModifyingCTE is a live bug, since the executor
certainly looks at that.
The proposed test case only fails back to v10, but since the executor
examines hasModifyingCTE in 9.x as well, I suspect that a test case
could be devised that fails in older branches. Given the nearness
of the release deadline, though, I'm not going to spend time looking
for a better test.
Report and patch by Greg Nancarrow, cosmetic changes by me
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Disallow converting an inheritance child table to a view.
commit : f7332195708d3eb846938609c52ec2f6be3075e1
author : Tom Lane <[email protected]>
date : Sat, 6 Feb 2021 15:17:01 -0500
committer: Tom Lane <[email protected]>
date : Sat, 6 Feb 2021 15:17:01 -0500
Generally, members of inheritance trees must be plain tables (or,
in more recent versions, foreign tables). ALTER TABLE INHERIT
rejects creating an inheritance relationship that has a view at
either end. When DefineQueryRewrite attempts to convert a relation
to a view, it already had checks prohibiting doing so for partitioning
parents or children as well as traditional-inheritance parents ...
but it neglected to check that a traditional-inheritance child wasn't
being converted. Since the planner assumes that any inheritance
child is a table, this led to making plans that tried to do a physical
scan on a view, causing failures (or even crashes, in recent versions).
One could imagine trying to support such a case by expanding the view
normally, but since the rewriter runs before the planner does
inheritance expansion, it would take some very fundamental refactoring
to make that possible. There are probably a lot of other parts of the
system that don't cope well with such a situation, too. For now,
just forbid it.
Per bug #16856 from Yang Lin. Back-patch to all supported branches.
(In versions before v10, this includes back-patching the portion of
commit 501ed02cf that added has_superclass(). Perhaps the lack of
that infrastructure partially explains the missing check.)
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/pg_inherits.c
M src/backend/rewrite/rewriteDefine.c
M src/test/regress/expected/rules.out
M src/test/regress/sql/rules.sql
Fix backslash-escaping multibyte chars in COPY FROM.
commit : 9b0ce895794c88b6556d6990cdf13e8da35d11de
author : Heikki Linnakangas <[email protected]>
date : Fri, 5 Feb 2021 11:14:56 +0200
committer: Heikki Linnakangas <[email protected]>
date : Fri, 5 Feb 2021 11:14:56 +0200
If a multi-byte character is escaped with a backslash in TEXT mode input,
and the encoding is one of the client-only encodings where the bytes after
the first one can have an ASCII byte "embedded" in the char, we didn't
skip the character correctly. After a backslash, we only skipped the first
byte of the next character, so if it was a multi-byte character, we would
try to process its second byte as if it was a separate character. If it
was one of the characters with special meaning, like '\n', '\r', or
another '\\', that would cause trouble.
One such exmple is the byte sequence '\x5ca45c2e666f6f' in Big5 encoding.
That's supposed to be [backslash][two-byte character][.][f][o][o], but
because the second byte of the two-byte character is 0x5c, we incorrectly
treat it as another backslash. And because the next character is a dot, we
parse it as end-of-copy marker, and throw an "end-of-copy marker corrupt"
error.
Backpatch to all supported versions.
Reviewed-by: John Naylor, Kyotaro Horiguchi
Discussion: https://www.postgresql.org/message-id/a897f84f-8dca-8798-3139-07da5bb38728%40iki.fi
M src/backend/commands/copy.c
Avoid crash when rolling back within a prepared statement.
commit : 77e760d5cc71b8605762b770f539ea77b8db64f1
author : Tom Lane <[email protected]>
date : Wed, 3 Feb 2021 19:38:29 -0500
committer: Tom Lane <[email protected]>
date : Wed, 3 Feb 2021 19:38:29 -0500
If a portal is used to run a prepared CALL or DO statement that
contains a ROLLBACK, PortalRunMulti fails because the portal's
statement list gets cleared by the rollback. (Since the grammar
doesn't allow CALL/DO in PREPARE, the only easy way to get to this is
via extended query protocol, which treats all inputs as prepared
statements.) It's difficult to avoid resetting the portal early
because of resource-management issues, so work around this by teaching
PortalRunMulti to be wary of portal->stmts having suddenly become NIL.
The crash has only been seen to occur in v13 and HEAD (as a
consequence of commit 1cff1b95a having added an extra touch of
portal->stmts). But even before that, the code involved touching a
List that the portal no longer has any claim on. In the test case at
hand, the List will still exist because of another refcount on the
cached plan; but I'm far from convinced that it's impossible for the
cached plan to have been dropped by the time control gets back to
PortalRunMulti. Hence, backpatch to v11 where nested transactions
were added.
Thomas Munro and Tom Lane, per bug #16811 from James Inform
Discussion: https://postgr.es/m/[email protected]
M src/backend/tcop/pquery.c
pg_dump: Fix dumping of inherited generated columns
commit : 1dd6baf7880240102aff76a1a28098228915de14
author : Peter Eisentraut <[email protected]>
date : Wed, 3 Feb 2021 11:27:13 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 3 Feb 2021 11:27:13 +0100
Generation expressions of generated columns are always inherited, so
there is no need to set them separately in child tables, and there is
no syntax to do so either. The code previously used the code paths
for the handling of default values, for which different rules apply;
in particular it might want to set a default value explicitly for an
inherited column. This resulted in unrestorable dumps. For generated
columns, just skip them in inherited tables.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/15830.1575468847%40sss.pgh.pa.us
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/t/002_pg_dump.pl
Remove extra increment of plpgsql's statement counter for FOR loops.
commit : 78fab843386bbaaaa4f74b26859d669e89d7db63
author : Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 14:35:12 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 14:35:12 -0500
This left gaps in the internal statement numbering, which is not
terribly harmful (else we'd have noticed sooner), but it's not
great either.
Oversight in bbd5c207b; backpatch to v12 where that came in.
Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRDXyQaJmpotNTQVc-t-WxdWZC35V2PnmwOaV1-taidFWA@mail.gmail.com
M src/pl/plpgsql/src/pl_gram.y
Fix ancient memory leak in contrib/auto_explain.
commit : 7428469586e837747f86566c790f6ccceac1b1f3
author : Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 13:49:08 -0500
committer: Tom Lane <[email protected]>
date : Tue, 2 Feb 2021 13:49:08 -0500
The ExecutorEnd hook is invoked in a context that could be quite
long-lived, not the executor's own per-query context as I think
we were sort of assuming. Thus, any cruft generated while producing
the EXPLAIN output could accumulate over multiple queries. This can
result in spectacular leakage if log_nested_statements is on, and
even without that I'm surprised nobody complained before.
To fix, just switch into the executor's context so that anything we
allocate will be released when standard_ExecutorEnd frees the executor
state. We might as well nuke the code's retail pfree of the explain
output string, too; that's laughably inadequate to the need.
Japin Li, per report from Jeff Janes. This bug is old, so
back-patch to all supported branches.
Discussion: https://postgr.es/m/CAMkU=1wCVtbeRn0s9gt12KwQ7PLXovbpM8eg25SYocKW3BT4hg@mail.gmail.com
M contrib/auto_explain/auto_explain.c
Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions.
commit : be843ce29737ee4752fedabe07a75cf9db1e969f
author : Noah Misch <[email protected]>
date : Sat, 30 Jan 2021 00:00:27 -0800
committer: Noah Misch <[email protected]>
date : Sat, 30 Jan 2021 00:00:27 -0800
In a cluster having used CREATE INDEX CONCURRENTLY while having enabled
prepared transactions, queries that use the resulting index can silently
fail to find rows. Fix this for future CREATE INDEX CONCURRENTLY by
making it wait for prepared transactions like it waits for ordinary
transactions. This expands the VirtualTransactionId structure domain to
admit prepared transactions. It may be necessary to reindex to recover
from past occurrences. Back-patch to 9.5 (all supported versions).
Andrey Borodin, reviewed (in earlier versions) by Tom Lane and Michael
Paquier.
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lock.c
M src/include/storage/lock.h
M src/test/isolation/Makefile
M src/test/isolation/README
A src/test/isolation/expected/prepared-transactions-cic.out
A src/test/isolation/specs/prepared-transactions-cic.spec
Document behavior of the .** jsonpath accessor in the lax mode
commit : bd6366129b1b97769c5293da1ec7f112cc68a444
author : Alexander Korotkov <[email protected]>
date : Fri, 29 Jan 2021 15:27:55 +0300
committer: Alexander Korotkov <[email protected]>
date : Fri, 29 Jan 2021 15:27:55 +0300
When the .** jsonpath accessor handles the array, it selects both array and
each of its elements. When using lax mode, subsequent accessors automatically
unwrap arrays. So, the content of each array element may be selected twice.
Even though this behavior is counterintuitive, it's correct because everything
works as designed. This commit documents it.
Backpatch to 12 where the jsonpath language was introduced.
Reported-by: Thomas Kellerer
Bug: #16828
Discussion: https://postgr.es/m/16828-2b0229babfad2d8c%40postgresql.org
Discussion: https://postgr.es/m/CAPpHfdtS-nNidT%3DEqZbAYOPcnNOWh_sd6skVdu2CAQUGdvpT8Q%40mail.gmail.com
Author: Alexandex Korotkov, revised by Tom Lane
Reviewed-by: Alvaro Herrera, Thomas Kellerer, Tom Lane
Backpatch-through: 12
M doc/src/sgml/func.sgml
Silence another gcc 11 warning.
commit : 22f71a6e1d76c08f3f2a11459ae7a391ea223a97
author : Tom Lane <[email protected]>
date : Thu, 28 Jan 2021 17:18:23 -0500
committer: Tom Lane <[email protected]>
date : Thu, 28 Jan 2021 17:18:23 -0500
Per buildfarm and local experimentation, bleeding-edge gcc isn't
convinced that the MemSet in reorder_function_arguments() is safe.
Shut it up by adding an explicit check that pronargs isn't negative,
and by changing MemSet to memset. (It appears that either change is
enough to quiet the warning at -O2, but let's do both to be sure.)
M src/backend/optimizer/util/clauses.c
Fix hash partition pruning with asymmetric partition sets.
commit : 7c53a80675dffd63250cb3fbb9495d4552c68313
author : Tom Lane <[email protected]>
date : Thu, 28 Jan 2021 13:41:55 -0500
committer: Tom Lane <[email protected]>
date : Thu, 28 Jan 2021 13:41:55 -0500
perform_pruning_combine_step() was not taught about the number of
partition indexes used in hash partitioning; more embarrassingly,
get_matching_hash_bounds() also had it wrong. These errors are masked
in the common case where all the partitions have the same modulus
and no partition is missing. However, with missing or unequal-size
partitions, we could erroneously prune some partitions that need
to be scanned, leading to silently wrong query answers.
While a minimal-footprint fix for this could be to export
get_partition_bound_num_indexes and make the incorrect functions use it,
I'm of the opinion that that function should never have existed in the
first place. It's not reasonable data structure design that
PartitionBoundInfoData lacks any explicit record of the length of
its indexes[] array. Perhaps that was all right when it could always
be assumed equal to ndatums, but something should have been done about
it as soon as that stopped being true. Putting in an explicit
"nindexes" field makes both partition_bounds_equal() and
partition_bounds_copy() simpler, safer, and faster than before,
and removes explicit knowledge of the number-of-partition-indexes
rules from some other places too.
This change also makes get_hash_partition_greatest_modulus obsolete.
I left that in place in case any external code uses it, but no core
code does anymore.
Per bug #16840 from MichaĆ Albrycht. Back-patch to v11 where the
hash partitioning code came in. (In the back branches, add the new
field at the end of PartitionBoundInfoData to minimize ABI risks.)
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execPartition.c
M src/backend/partitioning/partbounds.c
M src/backend/partitioning/partprune.c
M src/include/partitioning/partbounds.h
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Make ecpg's rjulmdy() and rmdyjul() agree with their declarations.
commit : 25f9a7266282cdac858c1bbc77fac2497349ef8b
author : Tom Lane <[email protected]>
date : Thu, 28 Jan 2021 11:17:13 -0500
committer: Tom Lane <[email protected]>
date : Thu, 28 Jan 2021 11:17:13 -0500
We had "short *mdy" in the extern declarations, but "short mdy[3]"
in the actual function definitions. Per C99 these are equivalent,
but recent versions of gcc have started to issue warnings about
the inconsistency. Clean it up before the warnings get any more
widespread.
Back-patch, in case anyone wants to build older PG versions with
bleeding-edge compilers.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/ecpg/compatlib/informix.c
pgbench: Remove dead code
commit : 30854d94c627adf60cdb774c18c5b6b1290f6345
author : Alvaro Herrera <[email protected]>
date : Thu, 28 Jan 2021 12:50:40 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 28 Jan 2021 12:50:40 -0300
doConnect() never returns connections in state CONNECTION_BAD, so
checking for that is pointless. Remove the code that does.
This code has been dead since ba708ea3dc84, 20 years ago.
Discussion: https://postgr.es/m/[email protected]
Reviewed-by: Tom Lane <[email protected]>
M src/bin/pgbench/pgbench.c
Don't add bailout adjustment for non-strict deserialize calls.
commit : 30f912a41378bd0ce363276b2e6b9f590520af1d
author : Andrew Gierth <[email protected]>
date : Thu, 28 Jan 2021 10:53:10 +0000
committer: Andrew Gierth <[email protected]>
date : Thu, 28 Jan 2021 10:53:10 +0000
When building aggregate expression steps, strict checks need a bailout
jump for when a null value is encountered, so there is a list of steps
that require later adjustment. Adding entries to that list for steps
that aren't actually strict would be harmless, except that there is an
Assert which catches them. This leads to spurious errors on asserts
builds, for data sets that trigger parallel aggregation of an
aggregate with a non-strict deserialization function (no such
aggregates exist in the core system).
Repair by not adding the adjustment entry when it's not needed.
Backpatch back to 11 where the code was introduced.
Per a report from Darafei (Komzpa) of the PostGIS project; analysis
and patch by me.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execExpr.c
doc: Remove reference to views for TRUNCATE privilege
commit : d43e23505285b2f39dbda615d8cab51d0905fadc
author : Michael Paquier <[email protected]>
date : Wed, 27 Jan 2021 13:41:06 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 27 Jan 2021 13:41:06 +0900
The page about privilege rights mentioned that TRUNCATE could be applied
to views or even other relation types. This is confusing as this
command can be used only on tables and on partitioned tables.
Oversight in afc4a78.
Reported-by: Harisai Hari
Reviewed-by: Laurenz Albe
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M doc/src/sgml/ddl.sgml
Report the true database name on connection errors
commit : fdf9d005401dc2aa0b6321794310cd3cdb634b2d
author : Alvaro Herrera <[email protected]>
date : Tue, 26 Jan 2021 16:42:13 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 26 Jan 2021 16:42:13 -0300
When reporting connection errors, we might show a database name in the
message that's not the one we actually tried to connect to, if the
database was taken from libpq defaults instead of from user parameters.
Fix such error messages to use PQdb(), which reports the correct name.
(But, per commit 2930c05634bc, make sure not to try to print NULL.)
Apply to branches 9.5 through 13. Branch master has already been
changed differently by commit 58cd8dca3de0.
Reported-by: Robert Haas <[email protected]>
Discussion: https://postgr.es/m/CA+TgmobssJ6rS22dspWnu-oDxXevGmhMD8VcRBjmj-b9UDqRjw@mail.gmail.com
M contrib/oid2name/oid2name.c
M contrib/vacuumlo/vacuumlo.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pgbench/pgbench.c
Code review for psql's helpSQL() function.
commit : 82f97d33f4d947fa5d4492654f68b2cbaf19c9ff
author : Tom Lane <[email protected]>
date : Tue, 26 Jan 2021 13:04:52 -0500
committer: Tom Lane <[email protected]>
date : Tue, 26 Jan 2021 13:04:52 -0500
The loops to identify word boundaries could access past the end of
the input string. Likely that would never result in an actual
crash, but it makes valgrind unhappy.
The logic to try different numbers of words didn't work when the
input has two words but we only have a match to the first, eg
"\h with select". (We must "continue" the pass loop, not "break".)
The logic to compute nl_count was bizarrely managed, and in at
least two code paths could end up calling PageOutput with
nl_count = 0, resulting in failing to paginate output that should
have been fed to the pager. Also, in v12 and up, the nl_count
calculation hadn't been updated to account for the addition of a URL.
The PQExpBuffer holding the command syntax details wasn't freed,
resulting in a session-lifespan memory leak.
While here, improve some comments, choose a more descriptive name
for a variable, fix inconsistent datatype choice for another variable.
Per bug #16837 from Alexander Lakhin. This code is very old,
so back-patch to all supported branches.
Kyotaro Horiguchi and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/help.c
Don't clobber the calling user's credentials cache in Kerberos test.
commit : 820aa9ef4c357d540d21b17f2bf518d7ff2d9b12
author : Tom Lane <[email protected]>
date : Mon, 25 Jan 2021 14:53:13 -0500
committer: Tom Lane <[email protected]>
date : Mon, 25 Jan 2021 14:53:13 -0500
Embarrassing oversight in this test script, which fortunately is not
run by default.
Report and patch by Jacob Champion.
Discussion: https://postgr.es/m/[email protected]
M src/test/kerberos/t/001_auth.pl
Fix broken ruleutils support for function TRANSFORM clauses.
commit : 4641b2a30f0a2353e875cba48193e33d8ca0b173
author : Tom Lane <[email protected]>
date : Mon, 25 Jan 2021 13:03:11 -0500
committer: Tom Lane <[email protected]>
date : Mon, 25 Jan 2021 13:03:11 -0500
I chanced to notice that this dumped core due to a faulty Assert.
To add insult to injury, the output has been misformatted since v11.
Obviously we need some regression testing here.
Discussion: https://postgr.es/m/[email protected]
M contrib/hstore_plpython/expected/hstore_plpython.out
M contrib/hstore_plpython/sql/hstore_plpython.sql
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/fmgr/funcapi.c
Fix hypothetical bug in heap backward scans
commit : 06cdfe21d3233e3010d5332d60f27886c28b1a25
author : David Rowley <[email protected]>
date : Mon, 25 Jan 2021 19:53:27 +1300
committer: David Rowley <[email protected]>
date : Mon, 25 Jan 2021 19:53:27 +1300
Both heapgettup() and heapgettup_pagemode() incorrectly set the first page
to scan in a backward scan in which the number of pages to scan was
specified by heap_setscanlimits(). The code incorrectly started the scan
at the end of the relation when startBlk was 0, or otherwise at
startBlk - 1, neither of which is correct when only scanning a subset of
pages.
The fix here checks if heap_setscanlimits() has changed the number of
pages to scan and if so we set the first page to scan as the final page in
the specified range during backward scans.
Proper adjustment of this code was forgotten when heap_setscanlimits() was
added in 7516f5259 back in 9.5. However, practice, nowhere in core code
performs backward scans after having used heap_setscanlimits(), yet, it is
possible an extension uses the heap functions in this way, hence
backpatch.
An upcoming patch does use heap_setscanlimits() with backward scans, so
this must be fixed before that can go in.
Author: David Rowley
Discussion: https://postgr.es/m/CAApHDvpGc9h0_oVD2CtgBcxCS1N-qDYZSeBRnUh+0CWJA9cMaA@mail.gmail.com
Backpatch-through: 9.5, all supported versions
M src/backend/access/heap/heapam.c
Update time zone data files to tzdata release 2021a.
commit : 5db6ba3034bfe03df860d1a26e40e3ed63ae47b9
author : Tom Lane <[email protected]>
date : Sun, 24 Jan 2021 16:29:47 -0500
committer: Tom Lane <[email protected]>
date : Sun, 24 Jan 2021 16:29:47 -0500
DST law changes in Russia (Volgograd zone) and South Sudan.
Historical corrections for Australia, Bahamas, Belize, Bermuda,
Ghana, Israel, Kenya, Nigeria, Palestine, Seychelles, and Vanuatu.
Notably, the Australia/Currie zone has been corrected to the point
where it is identical to Australia/Hobart.
M src/timezone/data/tzdata.zi
Doc: improve directions for building on macOS.
commit : 29ba00598fe17ad2cc3aa26eccc2f88e408959e9
author : Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 18:58:25 -0500
committer: Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 18:58:25 -0500
In light of recent discussions, we should instruct people to
install Apple's command line tools; installing Xcode is secondary.
Also, fix sample command for finding out the default sysroot,
as we now know that the command originally recommended can give
a result that doesn't match your OS version.
Also document the workaround to use if you really don't want
configure to select a sysroot at all.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/installation.sgml
Doc: remove misleading claim in documentation of PQreset().
commit : 878cf600e36796e25a93c75d77c491a048047d27
author : Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 11:29:43 -0500
committer: Tom Lane <[email protected]>
date : Fri, 22 Jan 2021 11:29:43 -0500
This text claimed that the reconnection would occur "to the same
server", but there is no such guarantee in the code, nor would
insisting on that be an improvement.
Back-patch to v10 where multi-host connection strings were added.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
commit : 1cce024fd26be40412c41e9ec59f389cf5979da2
author : Tom Lane <[email protected]>
date : Thu, 21 Jan 2021 15:37:23 -0500
committer: Tom Lane <[email protected]>
date : Thu, 21 Jan 2021 15:37:23 -0500
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/[email protected]
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/optimizer/path/clausesel.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/path/pathkeys.c
M src/backend/optimizer/path/tidpath.c
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/optimizer/util/clauses.c
M src/backend/optimizer/util/inherit.c
M src/backend/optimizer/util/orclauses.c
M src/backend/optimizer/util/placeholder.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/optimizer/util/var.c
M src/backend/utils/adt/selfuncs.c
M src/include/optimizer/clauses.h
M src/include/optimizer/optimizer.h
M src/include/optimizer/paths.h
M src/include/optimizer/planmain.h
M src/include/optimizer/restrictinfo.h
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Further tweaking of PG_SYSROOT heuristics for macOS.
commit : 561dd8d8a2eab5097fedddabcc6dee4981c73550
author : Tom Lane <[email protected]>
date : Wed, 20 Jan 2021 12:07:23 -0500
committer: Tom Lane <[email protected]>
date : Wed, 20 Jan 2021 12:07:23 -0500
It emerges that in some phases of the moon (perhaps to do with
directory entry order?), xcrun will report that the SDK path is
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
which is normally a symlink to a version-numbered sibling directory.
Our heuristic to skip non-version-numbered pathnames was rejecting
that, which is the wrong thing to do. We'd still like to end up
with a version-numbered PG_SYSROOT value, but we can have that by
dereferencing the symlink.
Like the previous fix, back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]
M src/template/darwin
Disable vacuum page skipping in selected test cases.
commit : 53a24faaa4033303ff9d8d103456cae8c2791619
author : Tom Lane <[email protected]>
date : Wed, 20 Jan 2021 11:49:29 -0500
committer: Tom Lane <[email protected]>
date : Wed, 20 Jan 2021 11:49:29 -0500
By default VACUUM will skip pages that it can't immediately get
exclusive access to, which means that even activities as harmless
and unpredictable as checkpoint buffer writes might prevent a page
from being processed. Ordinarily this is no big deal, but we have
a small number of test cases that examine the results of VACUUM's
processing and therefore will fail if the page of interest is skipped.
This seems to be the explanation for some rare buildfarm failures.
To fix, add the DISABLE_PAGE_SKIPPING option to the VACUUM commands
in tests where this could be an issue.
In passing, remove a duplicated query in pageinspect/sql/page.sql.
Back-patch as necessary (some of these cases are as old as v10).
Discussion: https://postgr.es/m/[email protected]
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/sql/page.sql
M contrib/pg_visibility/expected/pg_visibility.out
M contrib/pg_visibility/sql/pg_visibility.sql
Fix bug in detecting concurrent page splits in GiST insert
commit : 0326635dd484fe1d5272dd0c551b78e3f779acb4
author : Heikki Linnakangas <[email protected]>
date : Wed, 20 Jan 2021 11:58:03 +0200
committer: Heikki Linnakangas <[email protected]>
date : Wed, 20 Jan 2021 11:58:03 +0200
In commit 9eb5607e699, I got the condition on checking for split or
deleted page wrong: I used && instead of ||. The comment correctly said
"concurrent split _or_ deletion".
As a result, GiST insertion could miss a concurrent split, and insert to
wrong page. Duncan Sands demonstrated this with a test script that did a
lot of concurrent inserts.
Backpatch to v12, where this was introduced. REINDEX is required to fix
indexes that were affected by this bug.
Backpatch-through: 12
Reported-by: Duncan Sands
Discussion: https://www.postgresql.org/message-id/a9690483-6c6c-3c82-c8ba-dc1a40848f11%40deepbluecap.com
M src/backend/access/gist/gist.c
Fix ALTER DEFAULT PRIVILEGES with duplicated objects
commit : 5ad672fc2f6541ec5a0c533fd44d3819bff1447a
author : Michael Paquier <[email protected]>
date : Wed, 20 Jan 2021 11:39:17 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 20 Jan 2021 11:39:17 +0900
Specifying duplicated objects in this command would lead to unique
constraint violations in pg_default_acl or "tuple already updated by
self" errors. Similarly to GRANT/REVOKE, increment the command ID after
each subcommand processing to allow this case to work transparently.
A regression test is added by tweaking one of the existing queries of
privileges.sql to stress this case.
Reported-by: Andrus
Author: Michael Paquier
Reviewed-by: Ălvaro Herrera
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M src/backend/catalog/aclchk.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Remove faulty support for MergeAppend plan with WHERE CURRENT OF.
commit : 6253159965d563dd0e416e064b760e381a60b8e8
author : Tom Lane <[email protected]>
date : Tue, 19 Jan 2021 13:25:33 -0500
committer: Tom Lane <[email protected]>
date : Tue, 19 Jan 2021 13:25:33 -0500
Somebody extended search_plan_tree() to treat MergeAppend exactly
like Append, which is 100% wrong, because unlike Append we can't
assume that only one input node is actively returning tuples.
Hence a cursor using a MergeAppend across a UNION ALL or inheritance
tree could falsely match a WHERE CURRENT OF query at a row that
isn't actually the cursor's current output row, but coincidentally
has the same TID (in a different table) as the current output row.
Delete the faulty code; this means that such a case will now return
an error like 'cursor "foo" is not a simply updatable scan of table
"bar"', instead of silently misbehaving. Users should not find that
surprising though, as the same cursor query could have failed that way
already depending on the chosen plan. (It would fail like that if the
sort were done with an explicit Sort node instead of MergeAppend.)
Expand the clearly-inadequate commentary to be more explicit about
what this code is doing, in hopes of forestalling future mistakes.
It's been like this for awhile, so back-patch to all supported
branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execCurrent.c
doc: adjust alignment of doc file list for "pg_waldump.sgml"
commit : 6b7436202604fc6be923c464123f3398b9c07485
author : Bruce Momjian <[email protected]>
date : Mon, 18 Jan 2021 18:48:25 -0500
committer: Bruce Momjian <[email protected]>
date : Mon, 18 Jan 2021 18:48:25 -0500
Backpatch-through: 10
M doc/src/sgml/ref/allfiles.sgml
Avoid crash with WHERE CURRENT OF and a custom scan plan.
commit : ba80489c66cf721da04dc5b230b3592f97f6be4c
author : Tom Lane <[email protected]>
date : Mon, 18 Jan 2021 18:32:30 -0500
committer: Tom Lane <[email protected]>
date : Mon, 18 Jan 2021 18:32:30 -0500
execCurrent.c's search_plan_tree() assumed that ForeignScanStates
and CustomScanStates necessarily have a valid ss_currentRelation.
This is demonstrably untrue for postgres_fdw's remote join and
remote aggregation plans, and non-leaf custom scans might not have
an identifiable scan relation either. Avoid crashing by ignoring
such nodes when the field is null.
This solution will lead to errors like 'cursor "foo" is not a
simply updatable scan of table "bar"' in cases where maybe we
could have allowed WHERE CURRENT OF to work. That's not an issue
for postgres_fdw's usages, since joins or aggregations would render
WHERE CURRENT OF invalid anyway. But an otherwise-transparent
upper level custom scan node might find this annoying. When and if
someone cares to expend work on such a scenario, we could invent a
custom-scan-provider callback to determine what's safe.
Report and patch by David Geier, commentary by me. It's been like
this for awhile, so back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execCurrent.c
Fix pg_dump for GRANT OPTION among initial privileges.
commit : a0d31b1c94143c12c98da85652aeb5ff240489c7
author : Noah Misch <[email protected]>
date : Sat, 16 Jan 2021 12:21:35 -0800
committer: Noah Misch <[email protected]>
date : Sat, 16 Jan 2021 12:21:35 -0800
The context is an object that no longer bears some aclitem that it bore
initially. (A user issued REVOKE or GRANT statements upon the object.)
pg_dump is forming SQL to reproduce the object ACL. Since initdb
creates no ACL bearing GRANT OPTION, reaching this bug requires an
extension where the creation script establishes such an ACL. No PGXN
extension does that. If an installation did reach the bug, pg_dump
would have omitted a semicolon, causing a REVOKE and the next SQL
statement to fail. Separately, since the affected code exists to
eliminate an entire aclitem, it wants plain REVOKE, not REVOKE GRANT
OPTION FOR. Back-patch to 9.6, where commit
23f34fa4ba358671adab16773e79c17c92cbc870 first appeared.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/dumputils.c
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/modules/test_pg_dump/test_pg_dump–1.0.sql
Prevent excess SimpleLruTruncate() deletion.
commit : 66087f7e922c7279ee2c1967760de0c1f942522b
author : Noah Misch <[email protected]>
date : Sat, 16 Jan 2021 12:21:35 -0800
committer: Noah Misch <[email protected]>
date : Sat, 16 Jan 2021 12:21:35 -0800
Every core SLRU wraps around. With the exception of pg_notify, the wrap
point can fall in the middle of a page. Account for this in the
PagePrecedes callback specification and in SimpleLruTruncate()'s use of
said callback. Update each callback implementation to fit the new
specification. This changes SerialPagePrecedesLogically() from the
style of asyncQueuePagePrecedes() to the style of CLOGPagePrecedes().
(Whereas pg_clog and pg_serial share a key space, pg_serial is nothing
like pg_notify.) The bug fixed here has the same symptoms and user
followup steps as 592a589a04bd456410b853d86bd05faa9432cbbb. Back-patch
to 9.5 (all supported versions).
Reviewed by Andrey Borodin and (in earlier versions) by Tom Lane.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/commands/async.c
M src/backend/storage/lmgr/predicate.c
M src/include/access/slru.h
Disallow CREATE STATISTICS on system catalogs
commit : 943a113bcb6fb913814ae32cbd4dd1ed29498750
author : Tomas Vondra <[email protected]>
date : Fri, 15 Jan 2021 23:24:19 +0100
committer: Tomas Vondra <[email protected]>
date : Fri, 15 Jan 2021 23:24:19 +0100
Add a check that CREATE STATISTICS does not add extended statistics on
system catalogs, similarly to indexes etc. It can be overriden using
the allow_system_table_mods GUC.
This bug exists since 7b504eb282c, adding the extended statistics, so
backpatch all the way back to PostgreSQL 10.
Author: Tomas Vondra
Reported-by: Dean Rasheed
Backpatch-through: 10
Discussion: https://postgr.es/m/CAEZATCXAPrrOKwEsyZKQ4uzzJQWBCt6QAvOcgqRGdWwT1zb%2BrQ%40mail.gmail.com
M src/backend/commands/statscmds.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Improve our heuristic for selecting PG_SYSROOT on macOS.
commit : f5d044eaeff01a26869444de9753be977b5a24b1
author : Tom Lane <[email protected]>
date : Fri, 15 Jan 2021 11:28:51 -0500
committer: Tom Lane <[email protected]>
date : Fri, 15 Jan 2021 11:28:51 -0500
In cases where Xcode is newer than the underlying macOS version,
asking xcodebuild for the SDK path will produce a pointer to the
SDK shipped with Xcode, which may end up building code that does
not work on the underlying macOS version. It appears that in
such cases, xcodebuild's answer also fails to match the default
behavior of Apple's compiler: assuming one has installed Xcode's
"command line tools", there will be an SDK for the OS's own version
in /Library/Developer/CommandLineTools, and the compiler will
default to using that. This is all pretty poorly documented,
but experimentation suggests that "xcrun --show-sdk-path" gives
the sysroot path that the compiler is actually using, at least
in some cases. Hence, try that first, but revert to xcodebuild
if xcrun fails (in very old Xcode, it is missing or lacks the
--show-sdk-path switch).
Also, "xcrun --show-sdk-path" may give a path that is valid but lacks
any OS version identifier. We don't really want that, since most
of the motivation for wiring -isysroot into the build flags at all
is to ensure that all parts of a PG installation are built against
the same SDK, even when considering extensions built later and/or on
a different machine. Insist on finding "N.N" in the directory name
before accepting the result. (Adding "--sdk macosx" to the xcrun
call seems to produce the same answer as xcodebuild, but usually
more quickly because it's cached, so we also try that as a fallback.)
The core reason why we don't want to use Xcode's default SDK in cases
like this is that Apple's technology for introducing new syscalls
does not play nice with Autoconf: for example, configure will think
that preadv/pwritev exist when using a Big Sur SDK, even when building
on an older macOS version where they don't exist. It'd be nice to
have a better solution to that problem, but this patch doesn't attempt
to fix that.
Per report from Sergey Shinderuk. Back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]
M src/template/darwin
Fix calculation of how much shared memory is required to store a TOC.
commit : 19a1d768531a75cf729dc294322c6a4460f67ed9
author : Fujii Masao <[email protected]>
date : Fri, 15 Jan 2021 12:44:17 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 15 Jan 2021 12:44:17 +0900
Commit ac883ac453 refactored shm_toc_estimate() but changed its calculation
of shared memory size for TOC incorrectly. Previously this could cause too
large memory to be allocated.
Back-patch to v11 where the bug was introduced.
Author: Takayuki Tsunakawa
Discussion: https://postgr.es/m/TYAPR01MB2990BFB73170E2C4921E2C4DFEA80@TYAPR01MB2990.jpnprd01.prod.outlook.com
M src/backend/storage/ipc/shm_toc.c
pg_dump: label PUBLICATION TABLE ArchiveEntries with an owner.
commit : 0d221ececeb9697b4e0c3a1c22861e8f54bd050d
author : Tom Lane <[email protected]>
date : Thu, 14 Jan 2021 16:19:38 -0500
committer: Tom Lane <[email protected]>
date : Thu, 14 Jan 2021 16:19:38 -0500
This is the same fix as commit 9eabfe300 applied to INDEX ATTACH
entries, but for table-to-publication attachments. As in that
case, even though the backend doesn't record "ownership" of the
attachment, we still ought to label it in the dump archive with
the role name that should run the ALTER PUBLICATION command.
The existing behavior causes the ALTER to be done by the original
role that started the restore; that will usually work fine, but
there may be corner cases where it fails.
The bulk of the patch is concerned with changing struct
PublicationRelInfo to include a pointer to the associated
PublicationInfo object, so that we can get the owner's name
out of that when the time comes. While at it, I rewrote
getPublicationTables() to do just one query of pg_publication_rel,
not one per table.
Back-patch to v10 where this code was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
Prevent drop of tablespaces used by partitioned relations
commit : c9b702552accae338f7fb2efe1a23cfdc2fff535
author : Alvaro Herrera <[email protected]>
date : Thu, 14 Jan 2021 15:32:14 -0300
committer: Alvaro Herrera <[email protected]>
date : Thu, 14 Jan 2021 15:32:14 -0300
When a tablespace is used in a partitioned relation (per commits
ca4103025dfe in pg12 for tables and 33e6c34c3267 in pg11 for indexes),
it is possible to drop the tablespace, potentially causing various
problems. One such was reported in bug #16577, where a rewriting ALTER
TABLE causes a server crash.
Protect against this by using pg_shdepend to keep track of tablespaces
when used for relations that don't keep physical files; we now abort a
tablespace if we see that the tablespace is referenced from any
partitioned relations.
Backpatch this to 11, where this problem has been latent all along. We
don't try to create pg_shdepend entries for existing partitioned
indexes/tables, but any ones that are modified going forward will be
protected.
Note slight behavior change: when trying to drop a tablespace that
contains both regular tables as well as partitioned ones, you'd
previously get ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE and now you'll
get ERRCODE_DEPENDENT_OBJECTS_STILL_EXIST. Arguably, the latter is more
correct.
It is possible to add protecting pg_shdepend entries for existing
tables/indexes, by doing
ALTER TABLE ONLY some_partitioned_table SET TABLESPACE pg_default;
ALTER TABLE ONLY some_partitioned_table SET TABLESPACE original_tablespace;
for each partitioned table/index that is not in the database default
tablespace. Because these partitioned objects do not have storage, no
file needs to be actually moved, so it shouldn't take more time than
what's required to acquire locks.
This query can be used to search for such relations:
SELECT ... FROM pg_class WHERE relkind IN ('p', 'I') AND reltablespace <> 0
Reported-by: Alexander Lakhin <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Author: Ălvaro Herrera <[email protected]>
Reviewed-by: Michael Paquier <[email protected]>
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/heap.c
M src/backend/catalog/pg_shdepend.c
M src/backend/commands/tablecmds.c
M src/backend/commands/tablespace.c
M src/include/catalog/dependency.h
M src/test/regress/input/tablespace.source
M src/test/regress/output/tablespace.source
Call out vacuum considerations in create index docs
commit : 259b21233032d21cf193b1a6041310e96afd9f9a
author : Alvaro Herrera <[email protected]>
date : Wed, 13 Jan 2021 17:55:41 -0300
committer: Alvaro Herrera <[email protected]>
date : Wed, 13 Jan 2021 17:55:41 -0300
Backpatch to pg12, which is as far as it goes without conflicts.
Author: James Coleman <[email protected]>
Reviewed-by: "David G. Johnston" <[email protected]>
Discussion: https://postgr.es/m/CAAaqYe9oEfbz7AxXq7OX+FFVi5w5p1e_Of8ON8ZnKO9QqBfmjg@mail.gmail.com
M doc/src/sgml/ref/create_index.sgml
M doc/src/sgml/ref/reindex.sgml
Disallow a digit as the first character of a variable name in pgbench.
commit : 3e214fafcea73d92f97da91bd76339e8ba7607b2
author : Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 14:52:49 -0500
committer: Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 14:52:49 -0500
The point of this restriction is to avoid trying to substitute variables
into timestamp literal values, which may contain strings like '12:34'.
There is a good deal more that should be done to reduce pgbench's
tendency to substitute where it shouldn't. But this is sufficient to
solve the case complained of by Jaime Soler, and it's simple enough
to back-patch.
Back-patch to v11; before commit 9d36a3866, pgbench had a slightly
different definition of what a variable name is, and anyway it seems
unwise to change long-stable branches for this.
Fabien Coelho
Discussion: https://postgr.es/m/alpine.DEB.2.22.394.2006291740420.805678@pseudo
M doc/src/sgml/ref/pgbench.sgml
M src/bin/pgbench/pgbench.c
Doc: clarify behavior of back-half options in pg_dump.
commit : 0363b7e908cdbcac060b82e97232b35ab3caa484
author : Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 13:30:04 -0500
committer: Tom Lane <[email protected]>
date : Wed, 13 Jan 2021 13:30:04 -0500
Options that change how the archive data is converted to SQL text
are ignored when dumping to archive formats. The documentation
previously said "not meaningful", which is not helpful.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/pg_dump.sgml
Fix memory leak in SnapBuildSerialize.
commit : 0b2ae3c928b2926fafa1c423b07b949a4c3bc402
author : Amit Kapila <[email protected]>
date : Wed, 13 Jan 2021 08:31:45 +0530
committer: Amit Kapila <[email protected]>
date : Wed, 13 Jan 2021 08:31:45 +0530
The memory for the snapshot was leaked while serializing it to disk during
logical decoding. This memory will be freed only once walsender stops
streaming the changes. This can lead to a huge memory increase when master
logs Standby Snapshot too frequently say when the user is trying to create
many replication slots.
Reported-by: [email protected]
Diagnosed-by: [email protected]
Author: Amit Kapila
Backpatch-through: 9.5
Discussion: https://postgr.es/m/[email protected]
M src/backend/replication/logical/snapbuild.c
pg_dump: label INDEX ATTACH ArchiveEntries with an owner.
commit : 436d9c5cabfcc3ffe83453b85299044f3aa0d86e
author : Tom Lane <[email protected]>
date : Tue, 12 Jan 2021 13:37:38 -0500
committer: Tom Lane <[email protected]>
date : Tue, 12 Jan 2021 13:37:38 -0500
Although a partitioned index's attachment to its parent doesn't
have separate ownership, the ArchiveEntry for it needs to be
marked with an owner anyway, to ensure that the ALTER command
is run by the appropriate role when restoring with
--use-set-session-authorization. Without this, the ALTER will
be run by the role that started the restore session, which will
usually work but it's formally the wrong thing.
Back-patch to v11 where this type of ArchiveEntry was added.
In HEAD, add equivalent commentary to the just-added TABLE ATTACH
case, which I'd made do the right thing already.
Discussion: https://postgr.es/m/[email protected]
M src/bin/pg_dump/pg_dump.c
Doc: fix description of privileges needed for ALTER PUBLICATION.
commit : a2768e44e879e62d9bb6f0bda4d695cd73d03843
author : Tom Lane <[email protected]>
date : Tue, 12 Jan 2021 12:52:14 -0500
committer: Tom Lane <[email protected]>
date : Tue, 12 Jan 2021 12:52:14 -0500
Adding a table to a publication requires ownership of the table
(in addition to ownership of the publication). This was mentioned
nowhere.
M doc/src/sgml/ref/alter_publication.sgml
Fix thinko in comment
commit : ddf0be8ca47c70564b077f24888ed178f5162e22
author : Alvaro Herrera <[email protected]>
date : Tue, 12 Jan 2021 11:48:45 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 12 Jan 2021 11:48:45 -0300
This comment has been wrong since its introduction in commit
2c03216d8311.
Author: Masahiko Sawada <[email protected]>
Discussion: https://postgr.es/m/CAD21AoAzz6qipFJBbGEaHmyWxvvNDp8httbwLR9tUQWaTjUs2Q@mail.gmail.com
M src/backend/access/transam/xlogreader.c
doc: expand description of how non-SELECT queries are processed
commit : 931c0bda2dd77781e0efdae47dda5ec478fde083
author : Bruce Momjian <[email protected]>
date : Sat, 9 Jan 2021 12:11:16 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 9 Jan 2021 12:11:16 -0500
The previous description of how the executor processes non-SELECT
queries was very dense, causing lack of clarity. This expanded text
spells it out more simply.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/arch-dev.sgml
Fix ancient bug in parsing of BRE-mode regular expressions.
commit : 8354371d0a539c0da9fcca4e4341872db60860e1
author : Tom Lane <[email protected]>
date : Fri, 8 Jan 2021 12:16:00 -0500
committer: Tom Lane <[email protected]>
date : Fri, 8 Jan 2021 12:16:00 -0500
brenext(), when parsing a '*' quantifier, forgot to return any "value"
for the token; per the equivalent case in next(), it should return
value 1 to indicate that greedy rather than non-greedy behavior is
wanted. The result is that the compiled regexp could behave like 'x*?'
rather than the intended 'x*', if we were unlucky enough to have
a zero in v->nextvalue at this point. That seems to happen with some
reliability if we have '.*' at the beginning of a BRE-mode regexp,
although that depends on the initial contents of a stack-allocated
struct, so it's not guaranteed to fail.
Found by Alexander Lakhin using valgrind testing. This bug seems
to be aboriginal in Spencer's code, so back-patch all the way.
Discussion: https://postgr.es/m/[email protected]
M src/backend/regex/regc_lex.c
Adjust createdb TAP tests to work on recent OpenBSD.
commit : 29556bf1be91510ba282e02f3ec890c3b6bf4f94
author : Tom Lane <[email protected]>
date : Thu, 7 Jan 2021 20:36:09 -0500
committer: Tom Lane <[email protected]>
date : Thu, 7 Jan 2021 20:36:09 -0500
We found last February that the error-case tests added by commit
008cf0409 failed on OpenBSD, because that platform doesn't really
check locale names. At the time it seemed that that was only an issue
for LC_CTYPE, but testing on a more recent version of OpenBSD shows
that it's now equally lax about LC_COLLATE.
Rather than dropping the LC_COLLATE test too, put back LC_CTYPE
(reverting c4b0edb07), and adjust these tests to accept the different
error message that we get if setlocale() doesn't reject a bogus locale
name. The point of these tests is not really what the backend does
with the locale name, but to show that createdb quotes funny locale
names safely; so we're not losing test reliability this way.
Back-patch as appropriate.
Discussion: https://postgr.es/m/[email protected]
M src/bin/scripts/t/020_createdb.pl
Further second thoughts about idle_session_timeout patch.
commit : f2a69e90472f9a78b1d57bcfda4f2cb15ede2ab2
author : Tom Lane <[email protected]>
date : Thu, 7 Jan 2021 11:45:09 -0500
committer: Tom Lane <[email protected]>
date : Thu, 7 Jan 2021 11:45:09 -0500
On reflection, the order of operations in PostgresMain() is wrong.
These timeouts ought to be shut down before, not after, we do the
post-command-read CHECK_FOR_INTERRUPTS, to guarantee that any
timeout error will be detected there rather than at some ill-defined
later point (possibly after having wasted a lot of work).
This is really an error in the original idle_in_transaction_timeout
patch, so back-patch to 9.6 where that was introduced.
M src/backend/tcop/postgres.c
doc: Remove reference to recovery params for divergence lookup in pg_rewind
commit : 820f882b3b4aa3c5f521cc1c83737a4958eb3e3f
author : Michael Paquier <[email protected]>
date : Thu, 7 Jan 2021 20:50:35 +0900
committer: Michael Paquier <[email protected]>
date : Thu, 7 Jan 2021 20:50:35 +0900
The documentation of pg_rewind mentioned the use of restore_command and
primary_conninfo as options available at startup to fetch missing WAL
segments that could be used to find the point of divergence for the
rewind. This is confusing because when finding the point of divergence
the target cluster is offline, so this option is not available.
Issue introduced by 878bd9a, so backpatch down to 9.6. The
documentation of 13 and HEAD was already right as this sentence has been
changed by a7e8ece when introducing -c/--restore-target-wal.
Reported-by: Amine Tengilimoglu
Discussion: https://postgr.es/m/CADTdw-w_0MP=iQrfizeU4QU5fcZb+w8P_oPeLL+WznWf0kbn3w@mail.gmail.com
Backpatch-through: 9.6
M doc/src/sgml/ref/pg_rewind.sgml
Detect the deadlocks between backends and the startup process.
commit : 9f540f840665936132dd30bd8e58e9a67e648f22
author : Fujii Masao <[email protected]>
date : Wed, 6 Jan 2021 12:29:43 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 6 Jan 2021 12:29:43 +0900
The deadlocks that the recovery conflict on lock is involved in can
happen between hot-standby backends and the startup process.
If a backend takes an access exclusive lock on the table and which
finally triggers the deadlock, that deadlock can be detected
as expected. On the other hand, previously, if the startup process
took an access exclusive lock and which finally triggered the deadlock,
that deadlock could not be detected and could remain even after
deadlock_timeout passed. This is a bug.
The cause of this bug was that the code for handling the recovery
conflict on lock didn't take care of deadlock case at all. It assumed
that deadlocks involving the startup process and backends were able
to be detected by the deadlock detector invoked within backends.
But this assumption was incorrect. The startup process also should
have invoked the deadlock detector if necessary.
To fix this bug, this commit makes the startup process invoke
the deadlock detector if deadlock_timeout is reached while handling
the recovery conflict on lock. Specifically, in that case, the startup
process requests all the backends holding the conflicting locks to
check themselves for deadlocks.
Back-patch to v9.6. v9.5 has also this bug, but per discussion we decided
not to back-patch the fix to v9.5. Because v9.5 doesn't have some
infrastructure codes (e.g., 37c54863cf) that this bug fix patch depends on.
We can apply those codes for the back-patch, but since the next minor
version release is the final one for v9.5, it's risky to do that. If we
unexpectedly introduce new bug to v9.5 by the back-patch, there is no
chance to fix that. We determined that the back-patch to v9.5 would give
more risk than gain.
Author: Fujii Masao
Reviewed-by: Bertrand Drouvot, Masahiko Sawada, Kyotaro Horiguchi
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/proc.c
M src/backend/tcop/postgres.c
M src/include/storage/procarray.h
doc: Fix description about default behavior of recovery_target_timeline.
commit : c919ed191dc0a6b4d0cbd33f2d42b8c584f35759
author : Fujii Masao <[email protected]>
date : Wed, 6 Jan 2021 11:58:23 +0900
committer: Fujii Masao <[email protected]>
date : Wed, 6 Jan 2021 11:58:23 +0900
The default value of recovery_target_timeline was changed in v12,
but the description about the default behavior of that was not updated.
Back-patch to v12 where the default behavior of recovery_target_timeline
was changed.
Author: Benoit Lobréau
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAPE8EZ7c3aruEmM24GYkj8y8WmHKD1m9TtPtgCF0nQ3zw4LCkQ@mail.gmail.com
M doc/src/sgml/backup.sgml
doc: improve NLS instruction wording
commit : 3e9cea229413a5f62fa069a625d4c4c5515c7b99
author : Bruce Momjian <[email protected]>
date : Tue, 5 Jan 2021 14:26:37 -0500
committer: Bruce Momjian <[email protected]>
date : Tue, 5 Jan 2021 14:26:37 -0500
Reported-by: "Tang, Haiying"
Discussion: https://postgr.es/m/bbbccf7a3c2d436e85d45869d612fd6b@G08CNEXMBPEKD05.g08.fujitsu.local
Author: "Tang, Haiying"
Backpatch-through: 9.5
M doc/src/sgml/nls.sgml
Add an explicit cast to double when using fabs().
commit : 740780a6157f0688829082dec793fce6153a36a3
author : Dean Rasheed <[email protected]>
date : Tue, 5 Jan 2021 11:50:11 +0000
committer: Dean Rasheed <[email protected]>
date : Tue, 5 Jan 2021 11:50:11 +0000
Commit bc43b7c2c0 used fabs() directly on an int variable, which
apparently requires an explicit cast on some platforms.
Per buildfarm.
M src/backend/utils/adt/numeric.c
Fix numeric_power() when the exponent is INT_MIN.
commit : 258b7700551c4fc01ecd55d7ead4085184d4dce3
author : Dean Rasheed <[email protected]>
date : Tue, 5 Jan 2021 11:07:12 +0000
committer: Dean Rasheed <[email protected]>
date : Tue, 5 Jan 2021 11:07:12 +0000
In power_var_int(), the computation of the number of significant
digits to use in the computation used log(Abs(exp)), which isn't safe
because Abs(exp) returns INT_MIN when exp is INT_MIN. Use fabs()
instead of Abs(), so that the exponent is cast to a double before the
absolute value is taken.
Back-patch to 9.6, where this was introduced (by 7d9a4737c2).
Discussion: https://postgr.es/m/CAEZATCVd6pMkz=BrZEgBKyqqJrt2xghr=fNc8+Z=5xC6cgWrWA@mail.gmail.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Fix integer-overflow corner cases in substring() functions.
commit : 7c98759e0485d26303f2572ce7724b6026fddd9c
author : Tom Lane <[email protected]>
date : Mon, 4 Jan 2021 18:32:40 -0500
committer: Tom Lane <[email protected]>
date : Mon, 4 Jan 2021 18:32:40 -0500
If the substring start index and length overflow when added together,
substring() misbehaved, either throwing a bogus "negative substring
length" error on a case that should succeed, or failing to complain that
a negative length is negative (and instead returning the whole string,
in most cases). Unsurprisingly, the text, bytea, and bit variants of
the function all had this issue. Rearrange the logic to ensure that
negative lengths are always rejected, and add an overflow check to
handle the other case.
Also install similar guards into detoast_attr_slice() (nee
heap_tuple_untoast_attr_slice()), since it's far from clear that
no other code paths leading to that function could pass it values
that would overflow.
Patch by myself and Pavel Stehule, per bug #16804 from Rafi Shamim.
Back-patch to v11. While these bugs are old, the common/int.h
infrastructure for overflow-detecting arithmetic didn't exist before
commit 4d6ad3125, and it doesn't seem like these misbehaviors are bad
enough to justify developing a standalone fix for the older branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/tuptoaster.c
M src/backend/utils/adt/varbit.c
M src/backend/utils/adt/varlena.c
M src/test/regress/expected/bit.out
M src/test/regress/expected/strings.out
M src/test/regress/sql/bit.sql
M src/test/regress/sql/strings.sql
Update copyright for 2021
commit : 9a968e7ceb0e8f97abc02923a11f617e4fb17ce7
author : Bruce Momjian <[email protected]>
date : Sat, 2 Jan 2021 13:06:24 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 2 Jan 2021 13:06:24 -0500
Backpatch-through: 9.5
M COPYRIGHT
M doc/src/sgml/legal.sgml
Doc: improve explanation of EXTRACT(EPOCH) for timestamp without tz.
commit : 5706c4871cbb5313092977b05e57ff1cb1b0e13e
author : Tom Lane <[email protected]>
date : Fri, 1 Jan 2021 15:51:09 -0500
committer: Tom Lane <[email protected]>
date : Fri, 1 Jan 2021 15:51:09 -0500
Try to be clearer about what computation is actually happening here.
Per bug #16797 from Dana Burd.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
Get heap page max offset with buffer lock held.
commit : 5f95664e191974d275381577db87858cc121e0f4
author : Peter Geoghegan <[email protected]>
date : Wed, 30 Dec 2020 17:21:39 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 30 Dec 2020 17:21:39 -0800
On further reflection it seems better to call PageGetMaxOffsetNumber()
after acquiring a buffer lock on the page. This shouldn't really
matter, but doing it this way is cleaner.
Follow-up to commit 42288174.
Backpatch: 12-, just like commit 42288174
M src/backend/access/heap/heapam.c
Fix index deletion latestRemovedXid bug.
commit : 4f70e0910ec875e34b645116e7b565e5bce28c3f
author : Peter Geoghegan <[email protected]>
date : Wed, 30 Dec 2020 16:29:01 -0800
committer: Peter Geoghegan <[email protected]>
date : Wed, 30 Dec 2020 16:29:01 -0800
The logic for determining the latest removed XID for the purposes of
generating recovery conflicts in REDO routines was subtly broken. It
failed to follow links from HOT chains, and so failed to consider all
relevant heap tuple headers in some cases.
To fix, expand the loop that deals with LP_REDIRECT line pointers to
also deal with HOT chains. The new version of the loop is loosely based
on a similar loop from heap_prune_chain().
The impact of this bug is probably quite limited, since the horizon code
necessarily deals with heap tuples that are pointed to by LP_DEAD-set
index tuples. The process of setting LP_DEAD index tuples (e.g. within
the kill_prior_tuple mechanism) is highly correlated with opportunistic
pruning of pointed-to heap tuples. Plus the question of generating a
recovery conflict usually comes up some time after index tuple LP_DEAD
bits were initially set, unlike heap pruning, where a latestRemovedXid
is generated at the point of the pruning operation (heap pruning has no
deferred "would-be page split" style processing that produces conflicts
lazily).
Only backpatch to Postgres 12, the first version where this logic runs
during original execution (following commit 558a9165e08). The index
latestRemovedXid mechanism has had the same bug since it first appeared
over 10 years ago (in commit a760893d), but backpatching to all
supported versions now seems like a bad idea on balance. Running the
new improved code during recovery seems risky, especially given the lack
of complaints from the field.
Author: Peter Geoghegan <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wz=Eib393+HHcERK_9MtgNS7Ew1HY=RDC_g6GL46zM5C6Q@mail.gmail.com
Backpatch: 12-
M src/backend/access/heap/heapam.c
M src/backend/storage/ipc/standby.c
Doc: spell out comparison behaviors for the date/time types.
commit : e3bfdf216166f5464235e71fc812e48454718f49
author : Tom Lane <[email protected]>
date : Wed, 30 Dec 2020 17:48:43 -0500
committer: Tom Lane <[email protected]>
date : Wed, 30 Dec 2020 17:48:43 -0500
The behavior of cross-type comparisons among date/time data types was
not really explained anywhere. You could probably infer it if you
recognized the applicability of comments elsewhere about datatype
conversions, but it seems worthy of explicit documentation.
Per bug #16797 from Dana Burd.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
Fix up usage of krb_server_keyfile GUC parameter.
commit : 3ca19490b4f8875c802999a62a207f9dabe813b2
author : Tom Lane <[email protected]>
date : Wed, 30 Dec 2020 11:38:42 -0500
committer: Tom Lane <[email protected]>
date : Wed, 30 Dec 2020 11:38:42 -0500
secure_open_gssapi() installed the krb_server_keyfile setting as
KRB5_KTNAME unconditionally, so long as it's not empty. However,
pg_GSS_recvauth() only installed it if KRB5_KTNAME wasn't set already,
leading to a troubling inconsistency: in theory, clients could see
different sets of server principal names depending on whether they
use GSSAPI encryption. Always using krb_server_keyfile seems like
the right thing, so make both places do that. Also fix up
secure_open_gssapi()'s lack of a check for setenv() failure ---
it's unlikely, surely, but security-critical actions are no place
to be sloppy.
Also improve the associated documentation.
This patch does nothing about secure_open_gssapi()'s use of setenv(),
and indeed causes pg_GSS_recvauth() to use it too. That's nominally
against project portability rules, but since this code is only built
with --with-gssapi, I do not feel a need to do something about this
in the back branches. A fix will be forthcoming for HEAD though.
Back-patch to v12 where GSSAPI encryption was introduced. The
dubious behavior in pg_GSS_recvauth() goes back further, but it
didn't have anything to be inconsistent with, so let it be.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/config.sgml
M src/backend/libpq/auth.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/utils/misc/postgresql.conf.sample
In pg_upgrade cross-version test, handle lack of oldstyle_length().
commit : 635140ace0b84aa5f177bddc368acb64f56a8e59
author : Noah Misch <[email protected]>
date : Wed, 30 Dec 2020 01:43:43 -0800
committer: Noah Misch <[email protected]>
date : Wed, 30 Dec 2020 01:43:43 -0800
This suffices for testing v12 -> v13; some other version pairs need more
changes. Back-patch to v10, which removed the function.
M src/bin/pg_upgrade/test.sh
doc: Improve some grammar and sentences
commit : 9ad2689889ad47274550afadef430bc4929a9cd7
author : Michael Paquier <[email protected]>
date : Tue, 29 Dec 2020 18:19:11 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 29 Dec 2020 18:19:11 +0900
90fbf7c has taken care of that for HEAD. This includes the portion of
the fixes that applies to the documentation, where needed depending on
the branch.
Author: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/sources.sgml
Improve log messages related to pg_hba.conf not matching a connection.
commit : 7ed616f818f0a5842a927493d049cc393b09b9a1
author : Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 17:58:58 -0500
committer: Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 17:58:58 -0500
Include details on whether GSS encryption has been activated;
since we added "hostgssenc" type HBA entries, that's relevant info.
Kyotaro Horiguchi and Tom Lane. Back-patch to v12 where
GSS encryption was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/backend/libpq/auth.c
Fix assorted issues in backend's GSSAPI encryption support.
commit : 4cfdd8a47a9e865a45d38f944c6b0e6c7bc50628
author : Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 17:44:17 -0500
committer: Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 17:44:17 -0500
Unrecoverable errors detected by GSSAPI encryption can't just be
reported with elog(ERROR) or elog(FATAL), because attempting to
send the error report to the client is likely to lead to infinite
recursion or loss of protocol sync. Instead make this code do what
the SSL encryption code has long done, which is to just report any
such failure to the server log (with elevel COMMERROR), then pretend
we've lost the connection by returning errno = ECONNRESET.
Along the way, fix confusion about whether message translation is done
by pg_GSS_error() or its callers (the latter should do it), and make
the backend version of that function work more like the frontend
version.
Avoid allocating the port->gss struct until it's needed; we surely
don't need to allocate it in the postmaster.
Improve logging of "connection authorized" messages with GSS enabled.
(As part of this, I back-patched the code changes from dc11f31a1.)
Make BackendStatusShmemSize() account for the GSS-related space that
will be allocated by CreateSharedBackendStatus(). This omission
could possibly cause out-of-shared-memory problems with very high
max_connections settings.
Remove arbitrary, pointless restriction that only GSS authentication
can be used on a GSS-encrypted connection.
Improve documentation; notably, document the fact that libpq now
prefers GSS encryption over SSL encryption if both are possible.
Per report from Mikael Gustavsson. Back-patch to v12 where
this code was introduced.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/runtime.sgml
M src/backend/libpq/auth.c
M src/backend/libpq/be-gssapi-common.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/libpq/be-secure.c
M src/backend/libpq/hba.c
M src/backend/libpq/pqcomm.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/utils/init/postinit.c
M src/include/libpq/be-gssapi-common.h
M src/include/libpq/libpq-be.h
Fix bugs in libpq's GSSAPI encryption support.
commit : b3a5bf719cf7cbc2eb3f15c0e05a8032aa502cf3
author : Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 15:43:44 -0500
committer: Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 15:43:44 -0500
The critical issue fixed here is that if a GSSAPI-encrypted connection
is successfully made, pqsecure_open_gss() cleared conn->allow_ssl_try,
as an admittedly-hacky way of preventing us from then trying to tunnel
SSL encryption over the already-encrypted connection. The problem
with that is that if we abandon the GSSAPI connection because of a
failure during authentication, we would not attempt SSL encryption
in the next try with the same server. This can lead to unexpected
connection failure, or silently getting a non-encrypted connection
where an encrypted one is expected.
Fortunately, we'd only manage to make a GSSAPI-encrypted connection
if both client and server hold valid tickets in the same Kerberos
infrastructure, which is a relatively uncommon environment.
Nonetheless this is a very nasty bug with potential security
consequences. To fix, don't reset the flag, instead adding a
check for conn->gssenc being already true when deciding whether
to try to initiate SSL.
While here, fix some lesser issues in libpq's GSSAPI code:
* Use the need_new_connection stanza when dropping an attempted
GSSAPI connection, instead of partially duplicating that code.
The consequences of this are pretty minor: AFAICS it could only
lead to auth_req_received or password_needed remaining set when
they shouldn't, which is not too harmful.
* Fix pg_GSS_error() to not repeat the "mprefix" it's given multiple
times, and to notice any failure return from gss_display_status().
* Avoid gratuitous dependency on NI_MAXHOST in
pg_GSS_load_servicename().
Per report from Mikael Gustavsson. Back-patch to v12 where
this code was introduced.
Discussion: https://postgr.es/m/[email protected]
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-gssapi-common.c
M src/interfaces/libpq/fe-secure-gssapi.c
Further fix thinko in plpgsql memory leak fix.
commit : d37965965ad30aacad4d46cfa5ca42b416331ce6
author : Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 11:55:23 -0500
committer: Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 11:55:23 -0500
There's a second call of get_eval_mcontext() that should also be
get_stmt_mcontext(). This is actually dead code, since no
interesting allocations happen before switching back to the
original context, but we should keep it in sync with the other
call to forestall possible future bugs.
Discussion: https://postgr.es/m/[email protected]
M src/pl/plpgsql/src/pl_exec.c
Fix thinko in plpgsql memory leak fix.
commit : fd1347f9d8def3a892a7335ec026bff0227db1af
author : Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 11:41:25 -0500
committer: Tom Lane <[email protected]>
date : Mon, 28 Dec 2020 11:41:25 -0500
Commit a6b1f5365 intended to place the transient "target" list of
a CALL statement in the function's statement-lifespan context,
but I fat-fingered that and used get_eval_mcontext() instead of
get_stmt_mcontext(). The eval_mcontext belongs to the "simple
expression" infrastructure, which is destroyed at transaction end.
The net effect is that a CALL in a procedure to another procedure
that has OUT or INOUT parameters would fail if the called procedure
did a COMMIT.
Per report from Peter Eisentraut. Back-patch to v11, like the
prior patch.
Discussion: https://postgr.es/m/[email protected]
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_call.sql
Fix inconsistent code with shared invalidations of snapshots
commit : 190f0af0d1277429467ef28332818952c1fd395f
author : Michael Paquier <[email protected]>
date : Mon, 28 Dec 2020 22:17:02 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 28 Dec 2020 22:17:02 +0900
The code in charge of processing a single invalidation message has been
using since 568d413 the structure for relation mapping messages. This
had fortunately no consequence as both locate the database ID at the
same location, but it could become a problem in the future if this area
of the code changes.
Author: Konstantin Knizhnik
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M src/backend/utils/cache/inval.c
postgres_fdw: Fix connection leak.
commit : e792ca4aca2688a7084f739248d34e4f7a812b87
author : Fujii Masao <[email protected]>
date : Mon, 28 Dec 2020 19:59:00 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 28 Dec 2020 19:59:00 +0900
In postgres_fdw, the cached connections to foreign servers will not be
closed until the local session exits if the user mappings or foreign servers
that those connections depend on are dropped. Those connections can be
leaked.
To fix that connection leak issue, after a change to a pg_foreign_server
or pg_user_mapping catalog entry, this commit makes postgres_fdw close
the connections depending on that entry immediately if current
transaction has not used those connections yet. Otherwise, mark those
connections as invalid and then close them at the end of current transaction,
since they cannot be closed in the midst of the transaction using them.
Closed connections will be remade at the next opportunity if necessary.
Back-patch to all supported branches.
Author: Bharath Rupireddy
Reviewed-by: Zhihong Yu, Zhijie Hou, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVNcGH_6qLY-4_tXz8JLvA+4yeBThRfxMz7Oxbk1aHcpQ@mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Invalidate acl.c caches when pg_authid changes.
commit : c9669880604221489bd87b8dbe686462d19b4ce4
author : Noah Misch <[email protected]>
date : Fri, 25 Dec 2020 10:41:59 -0800
committer: Noah Misch <[email protected]>
date : Fri, 25 Dec 2020 10:41:59 -0800
This makes existing sessions reflect "ALTER ROLE ... [NO]INHERIT" as
quickly as they have been reflecting "GRANT role_name". Back-patch to
9.5 (all supported versions).
Reviewed by Nathan Bossart.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/adt/acl.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Avoid time-of-day-dependent failure in log rotation test.
commit : f853ffa22055a27f8d2239982edff6e4b8b958d4
author : Tom Lane <[email protected]>
date : Thu, 24 Dec 2020 21:37:46 -0500
committer: Tom Lane <[email protected]>
date : Thu, 24 Dec 2020 21:37:46 -0500
Buildfarm members pogona and petalura have shown a failure when
pg_ctl/t/004_logrotate.pl starts just before local midnight.
The default rotate-at-midnight behavior occurs just before the
Perl script examines current_logfiles, so it figures that the
rotation it's already requested has occurred ... but in reality,
that rotation happens just after it looks, so the expected new
log data goes into a different file than the one it's examining.
In HEAD, src/test/kerberos/t/001_auth.pl has acquired similar code
that evidently has a related failure mode. Besides being quite new,
few buildfarm critters run that test, so it's unsurprising that
we've not yet seen a failure there.
Fix both cases by setting log_rotation_age = 0 so that no time-based
rotation can occur. Also absorb 004_logrotate.pl's decision to
set lc_messages = 'C' into the kerberos test, in hopes that it will
work in non-English prevailing locales.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=pogona&dt=2020-12-24%2022%3A10%3A04
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=petalura&dt=2020-02-01%2022%3A20%3A04
M src/bin/pg_ctl/t/004_logrotate.pl
Fix race condition between shutdown and unstarted background workers.
commit : 3d8068edce13464dc30837f7c9e6483920fc53fd
author : Tom Lane <[email protected]>
date : Thu, 24 Dec 2020 17:00:43 -0500
committer: Tom Lane <[email protected]>
date : Thu, 24 Dec 2020 17:00:43 -0500
If a database shutdown (smart or fast) is commanded between the time
some process decides to request a new background worker and the time
that the postmaster can launch that worker, then nothing happens
because the postmaster won't launch any bgworkers once it's exited
PM_RUN state. This is fine ... unless the requesting process is
waiting for that worker to finish (or even for it to start); in that
case the requestor is stuck, and only manual intervention will get us
to the point of being able to shut down.
To fix, cancel pending requests for workers when the postmaster sends
shutdown (SIGTERM) signals, and similarly cancel any new requests that
arrive after that point. (We can optimize things slightly by only
doing the cancellation for workers that have waiters.) To fit within
the existing bgworker APIs, the "cancel" is made to look like the
worker was started and immediately stopped, causing deregistration of
the bgworker entry. Waiting processes would have to deal with
premature worker exit anyway, so this should introduce no bugs that
weren't there before. We do have a side effect that registration
records for restartable bgworkers might disappear when theoretically
they should have remained in place; but since we're shutting down,
that shouldn't matter.
Back-patch to v10. There might be value in putting this into 9.6
as well, but the management of bgworkers is a bit different there
(notably see 8ff518699) and I'm not convinced it's worth the effort
to validate the patch for that branch.
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_prewarm/autoprewarm.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/postmaster.c
M src/include/postmaster/bgworker_internals.h
docs: document which server-side languages can create procs
commit : 8985e021df36d9d65183c8349351464addab4536
author : Bruce Momjian <[email protected]>
date : Wed, 23 Dec 2020 09:37:38 -0500
committer: Bruce Momjian <[email protected]>
date : Wed, 23 Dec 2020 09:37:38 -0500
This was missed when the feature was added.
Reported-by: Daniel Westermann
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 11
M doc/src/sgml/plperl.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/plpython.sgml
M doc/src/sgml/pltcl.sgml
M doc/src/sgml/spi.sgml
Fix portability issues with parsing of recovery_target_xid
commit : 67a9411d2fcaffee2e7e0f3e384d13c406d5115e
author : Michael Paquier <[email protected]>
date : Wed, 23 Dec 2020 12:51:39 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 23 Dec 2020 12:51:39 +0900
The parsing of this parameter has been using strtoul(), which is not
portable across platforms. On most Unix platforms, unsigned long has a
size of 64 bits, while on Windows it is 32 bits. It is common in
recovery scenarios to rely on the output of txid_current() or even the
newer pg_current_xact_id() to get a transaction ID for setting up
recovery_target_xid. The value returned by those functions includes the
epoch in the computed result, which would cause strtoul() to fail where
unsigned long has a size of 32 bits once the epoch is incremented.
WAL records and 2PC data include only information about 32-bit XIDs and
it is not possible to have XIDs across more than one epoch, so
discarding the high bits from the transaction ID set has no impact on
recovery. On the contrary, the use of strtoul() prevents a consistent
behavior across platforms depending on the size of unsigned long.
This commit changes the parsing of recovery_target_xid to use
pg_strtouint64() instead, available down to 9.6. There is one TAP test
stressing recovery with recovery_target_xid, where a tweak based on
pg_reset{xlog,wal} is added to bump the XID epoch so as this change gets
tested, as per an idea from Alexander Lakhin.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.6
M src/backend/utils/misc/guc.c
M src/test/recovery/t/003_recovery_targets.pl
Improve autoprewarm's handling of early-shutdown scenarios.
commit : f581e5383648913385d3e7789ad9323a3ebfb1d0
author : Tom Lane <[email protected]>
date : Tue, 22 Dec 2020 13:23:49 -0500
committer: Tom Lane <[email protected]>
date : Tue, 22 Dec 2020 13:23:49 -0500
Bad things happen if the DBA issues "pg_ctl stop -m fast" before
autoprewarm finishes loading its list of blocks to prewarm.
The current worker process successfully terminates early, but
(if this wasn't the last database with blocks to prewarm) the
leader process will just try to launch another worker for the
next database. Since the postmaster is now in PM_WAIT_BACKENDS
state, it ignores the launch request, and the leader just sits
until it's killed manually.
This is mostly the fault of our half-baked design for launching
background workers, but a proper fix for that is likely to be
too invasive to be back-patchable. To ameliorate the situation,
fix apw_load_buffers() to check whether SIGTERM has arrived
just before trying to launch another worker. That leaves us with
only a very narrow window in each worker launch where SIGTERM
could occur between the launch request and successful worker start.
Another issue is that if the leader process does manage to exit,
it unconditionally rewrites autoprewarm.blocks with only the
blocks currently in shared buffers, thus forgetting any blocks
that we hadn't reached yet while prewarming. This seems quite
unhelpful, since the next database start will then not have the
expected prewarming benefit. Fix it to not modify the file if
we shut down before the initial load attempt is complete.
Per bug #16785 from John Thompson. Back-patch to v11 where
the autoprewarm code was introduced.
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_prewarm/autoprewarm.c
Remove "invalid concatenation of jsonb objects" error case.
commit : 38bef9e433eff68ffbfcf0a52ba7c2dc83c7466d
author : Tom Lane <[email protected]>
date : Mon, 21 Dec 2020 13:11:29 -0500
committer: Tom Lane <[email protected]>
date : Mon, 21 Dec 2020 13:11:29 -0500
The jsonb || jsonb operator arbitrarily rejected certain combinations
of scalar and non-scalar inputs, while being willing to concatenate
other combinations. This was of course quite undocumented. Rather
than trying to document it, let's just remove the restriction,
creating a uniform rule that unless we are handling an object-to-object
concatenation, non-array inputs are converted to one-element arrays,
resulting in an array-to-array concatenation. (This does not change
the behavior for any case that didn't throw an error before.)
Per complaint from Joel Jacobson. Back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/func.sgml
M src/backend/utils/adt/jsonfuncs.c
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql
Doc: fix description of how to use src/tutorial files.
commit : def510c9fac03965840eca805c5877cea819eca2
author : Tom Lane <[email protected]>
date : Sun, 20 Dec 2020 15:28:22 -0500
committer: Tom Lane <[email protected]>
date : Sun, 20 Dec 2020 15:28:22 -0500
The separate "cd" command before invoking psql made sense (or at least
I thought so) when it was added in commit ed1939332. But 4e3a61635
removed the supporting text that explained when to use it, making it
just confusing. So drop it.
Also switch from four-dot to three-dot filler for the unsupplied
part of the path, since at least one person has read the four-dot
filler as a typo for "../..". And fix these/those inconsistency.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/query.sgml
Doc: improve description of pgbench script weights.
commit : 3989fdab48d937b448198e9886bb07c6027b00ca
author : Tom Lane <[email protected]>
date : Sun, 20 Dec 2020 13:37:25 -0500
committer: Tom Lane <[email protected]>
date : Sun, 20 Dec 2020 13:37:25 -0500
Point out the workaround to be used if you want to write a script
file name that includes "@". Clean up the text a little.
Fabien Coelho, additional wordsmithing by me
Discussion: https://postgr.es/m/1c4e81550d214741827a03292222db8d@G08CNEXMBPEKD06.g08.fujitsu.local
M doc/src/sgml/ref/pgbench.sgml
Avoid memcpy() with same source and destination during relmapper init.
commit : ba04cedf1d262f3d7dfe6950b13a7ed1d80a9502
author : Tom Lane <[email protected]>
date : Fri, 18 Dec 2020 15:46:44 -0500
committer: Tom Lane <[email protected]>
date : Fri, 18 Dec 2020 15:46:44 -0500
A narrow reading of the C standard says that memcpy(x,x,n) is undefined,
although it's hard to envision an implementation that would really
misbehave. However, analysis tools such as valgrind might whine about
this; accordingly, let's band-aid relmapper.c to not do it.
See also 5b630501e, d3f4e8a8a, ad7b48ea0, and other similar fixes.
Apparently, none of those folk tried valgrinding initdb? This has been
like this for long enough that I'm surprised it hasn't been reported
before.
Back-patch, just in case anybody wants to use a back branch on a platform
that complains about this; we back-patched those earlier fixes too.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/cache/relmapper.c
doc: clarify COPY TO for partitioning/inheritance
commit : f32c6873556f5101b2ce7436f2508b9f44ef30d6
author : Bruce Momjian <[email protected]>
date : Tue, 15 Dec 2020 19:20:15 -0500
committer: Bruce Momjian <[email protected]>
date : Tue, 15 Dec 2020 19:20:15 -0500
It was not clear how COPY TO behaved with partitioning/inheritance
because the paragraphs were so far apart. Also reword to simplify.
Discussion: https://postgr.es/m/[email protected]
Author: Justin Pryzby
Backpatch-through: 10
M doc/src/sgml/ref/copy.sgml
Use native methods to open input in TestLib::slurp_file on Windows.
commit : bf88c6d6021c1abbc73197803c98dad0f033b462
author : Andrew Dunstan <[email protected]>
date : Tue, 15 Dec 2020 10:00:18 -0500
committer: Andrew Dunstan <[email protected]>
date : Tue, 15 Dec 2020 10:00:18 -0500
This is a backport of commits 114541d58e and 6f59826f0 to the remaining
live branches.
M src/test/perl/TestLib.pm
Revert "Cannot use WL_SOCKET_WRITEABLE without WL_SOCKET_READABLE."
commit : 706d84fe701ef9a8a5b04d2df013e29ea3f7ca74
author : Jeff Davis <[email protected]>
date : Mon, 14 Dec 2020 23:48:04 -0800
committer: Jeff Davis <[email protected]>
date : Mon, 14 Dec 2020 23:48:04 -0800
This reverts commit 3a9e64aa0d96c8ffb6c682b082d0f72b1d373327.
Commit 4bad60e3 fixed the root of the problem that 3a9e64aa worked
around.
This enables proper pipelining of commands after terminating
replication, eliminating an undocumented limitation.
Discussion: https://postgr.es/m/3d57bc29-4459-578b-79cb-7641baf53c57%40iki.fi
Backpatch-through: 9.5
M src/backend/replication/walsender.c
initdb: complete getopt_long alphabetization
commit : 8e456b8e597ed944eca7c24d6613ebff94aacb0b
author : Bruce Momjian <[email protected]>
date : Sat, 12 Dec 2020 12:59:09 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 12 Dec 2020 12:59:09 -0500
Backpatch-through: 9.5
M src/bin/initdb/initdb.c
initdb: properly alphabetize getopt_long options in C string
commit : 035c7dfddff1197a9229559a5c6e23e2a4cbf46c
author : Bruce Momjian <[email protected]>
date : Sat, 12 Dec 2020 12:51:16 -0500
committer: Bruce Momjian <[email protected]>
date : Sat, 12 Dec 2020 12:51:16 -0500
Backpatch-through: 9.5
M src/bin/initdb/initdb.c
Teach contain_leaked_vars that assignment SubscriptingRefs are leaky.
commit : 2f1997b1551a417b52d75621817a813f5171c48f
author : Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 17:50:54 -0500
committer: Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 17:50:54 -0500
array_get_element and array_get_slice qualify as leakproof, since
they will silently return NULL for bogus subscripts. But
array_set_element and array_set_slice throw errors for such cases,
making them clearly not leakproof. contain_leaked_vars was evidently
written with only the former case in mind, as it gave the wrong answer
for assignment SubscriptingRefs (nee ArrayRefs).
This would be a live security bug, were it not that assignment
SubscriptingRefs can only occur in INSERT and UPDATE target lists,
while we only care about leakproofness for qual expressions; so the
wrong answer can't occur in practice. Still, that's a rather shaky
answer for a security-related question; and maybe in future somebody
will want to ask about leakproofness of a tlist. So it seems wise to
fix and even back-patch this correction.
(We would need some change here anyway for the upcoming
generic-subscripting patch, since extensions might make different
tradeoffs about whether to throw errors. Commit 558d77f20 attempted
to lay groundwork for that by asking check_functions_in_node whether a
SubscriptingRef contains leaky functions; but that idea fails now that
the implementation methods of a SubscriptingRef are not SQL-visible
functions that could be marked leakproof or not.)
Back-patch to 9.6. While 9.5 has the same issue, the code's a bit
different. It seems quite unlikely that we'd introduce any actual bug
in the short time 9.5 has left to live, so the work/risk/reward balance
isn't attractive for changing 9.5.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/util/clauses.c
Doc: clarify that CREATE TABLE discards redundant unique constraints.
commit : e824ddc37f3c57a52eec0d56131c4949cd66220c
author : Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 13:09:48 -0500
committer: Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 13:09:48 -0500
The SQL standard says that redundant unique constraints are disallowed,
but we long ago decided that throwing an error would be too
user-unfriendly, so we just drop redundant ones. The docs weren't very
clear about that though, as this behavior was only explained for PRIMARY
KEY vs UNIQUE, not UNIQUE vs UNIQUE.
While here, I couldn't resist doing some copy-editing and markup-fixing
on the adjacent text about INCLUDE options.
Per bug #16767 from Matthias vd Meent.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_table.sgml
Doc: explain that the string types can't store \0 (ASCII NUL).
commit : 101c8ae02cc007101bec711563655727f14efb0b
author : Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 12:06:19 -0500
committer: Tom Lane <[email protected]>
date : Tue, 8 Dec 2020 12:06:19 -0500
This restriction was mentioned in connection with string literals,
but it wasn't made clear that it's a general restriction not just
a syntactic limitation in query strings.
Per unsigned documentation comment.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/datatype.sgml
pgcrypto: Detect errors with EVP calls from OpenSSL
commit : d7ecba937b909c26cc266d06261ddaf5a540630e
author : Michael Paquier <[email protected]>
date : Tue, 8 Dec 2020 15:22:43 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 8 Dec 2020 15:22:43 +0900
The following routines are called within pgcrypto when handling digests
but there were no checks for failures:
- EVP_MD_CTX_size (can fail with -1 as of 3.0.0)
- EVP_MD_CTX_block_size (can fail with -1 as of 3.0.0)
- EVP_DigestInit_ex
- EVP_DigestUpdate
- EVP_DigestFinal_ex
A set of elog(ERROR) is added by this commit to detect such failures,
that should never happen except in the event of a processing failure
internal to OpenSSL.
Note that it would be possible to use ERR_reason_error_string() to get
more context about such errors, but these refer mainly to the internals
of OpenSSL, so it is not really obvious how useful that would be. This
is left out for simplicity.
Per report from Coverity. Thanks to Tom Lane for the discussion.
Backpatch-through: 9.5
M contrib/pgcrypto/openssl.c
jit: Correct parameter type for generated expression evaluation functions.
commit : 27b57f806dc2da9d43657c0ef9569be9aac11e1c
author : Andres Freund <[email protected]>
date : Mon, 7 Dec 2020 18:21:06 -0800
committer: Andres Freund <[email protected]>
date : Mon, 7 Dec 2020 18:21:06 -0800
clang only uses the 'i1' type for scalar booleans, not for pointers to
booleans (as the pointer might be pointing into a larger memory
allocation). Therefore a pointer-to-bool needs to the "storage" boolean.
There's no known case of wrong code generation due to this, but it seems quite
possible that it could cause problems (see e.g. 72559438f92).
Author: Andres Freund
Discussion: https://postgr.es/m/[email protected]
Backpatch: 11-, where jit support was added
M src/backend/jit/llvm/llvmjit_expr.c
jit: configure: Explicitly reference 'native' component.
commit : b07490b912467e6760b3a410347959c3eb9fc306
author : Andres Freund <[email protected]>
date : Mon, 7 Dec 2020 18:12:23 -0800
committer: Andres Freund <[email protected]>
date : Mon, 7 Dec 2020 18:12:23 -0800
Until recently 'native' was implicitly included via 'orcjit', but a change
included in LLVM 11 (not yet released) removed a number of such indirect
component references.
Reported-By: Fabien COELHO <[email protected]>
Reported-By: Andres Freund <[email protected]>
Reported-By: Thomas Munro <[email protected]>
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch: 11-, where jit support was added
M config/llvm.m4
M configure
backpatch "jit: Add support for LLVM 12."
commit : fa7a52fb9943aa1c362a19fa886db284c82b62cd
author : Andres Freund <[email protected]>
date : Mon, 9 Nov 2020 20:01:33 -0800
committer: Andres Freund <[email protected]>
date : Mon, 9 Nov 2020 20:01:33 -0800
As there haven't been problem on the buildfarm due to this change,
backpatch 6c57f2ed16e now.
Author: Andres Freund
Discussion: https://postgr.es/m/[email protected]
Backpatch: 11-, where jit support was added
M src/backend/jit/llvm/llvmjit.c
M src/tools/pgindent/typedefs.list
Fix more race conditions in the newly-added pg_rewind test.
commit : 98f8cdd48cc5702c0cd3ef896efb957888e848b0
author : Heikki Linnakangas <[email protected]>
date : Mon, 7 Dec 2020 14:44:34 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 7 Dec 2020 14:44:34 +0200
pg_rewind looks at the control file to check what timeline a server is on.
But promotion doesn't immediately write a checkpoint, it merely writes
an end-of-recovery WAL record. If pg_rewind runs immediately after
promotion, before the checkpoint has completed, it will think think that
the server is still on the earlier timeline. We ran into this issue a long
time ago already, see commit 484a848a73f.
It's a bit bogus that pg_rewind doesn't determine the timeline correctly
until the end-of-recovery checkpoint has completed. We probably should
fix that. But for now work around it by waiting for the checkpoint
to complete before running pg_rewind, like we did in commit 484a848a73f.
In the passing, tidy up the new test a little bit. Rerder the INSERTs so
that the comments make more sense, remove a spurious CHECKPOINT call after
pg_rewind has already run, and add --debug option, so that if this fails
again, we'll have more data.
Per buildfarm failure at https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=rorqual&dt=2020-12-06%2018%3A32%3A19&stg=pg_rewind-check.
Backpatch to all supported versions.
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/bin/pg_rewind/t/008_min_recovery_point.pl
Fix missed step in removal of useless RESULT RTEs in the planner.
commit : 77a94c30da6152b9cbe5383d29e2aa9661260fbe
author : Tom Lane <[email protected]>
date : Sat, 5 Dec 2020 16:16:13 -0500
committer: Tom Lane <[email protected]>
date : Sat, 5 Dec 2020 16:16:13 -0500
Commit 4be058fe9 forgot that the append_rel_list would already be
populated at the time we remove useless result RTEs, and it might contain
PlaceHolderVars that need to be adjusted like the ones in the main parse
tree. This could lead to "no relation entry for relid N" failures later
on, when the planner tries to do something with an unadjusted PHV.
Per report from Tom Ellis. Back-patch to v12 where the bug came in.
Discussion: https://postgr.es/m/20201205173056.GF30712@cloudinit-builder
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Fix race conditions in newly-added test.
commit : ad3fb04b9cc2d490e64d4a16e516b5f9eeadc7f3
author : Heikki Linnakangas <[email protected]>
date : Fri, 4 Dec 2020 18:20:18 +0200
committer: Heikki Linnakangas <[email protected]>
date : Fri, 4 Dec 2020 18:20:18 +0200
Buildfarm has been failing sporadically on the new test. I was able to
reproduce this by adding a random 0-10 s delay in the walreceiver, just
before it connects to the primary. There's a race condition where node_3
is promoted before it has fully caught up with node_1, leading to diverged
timelines. When node_1 is later reconfigured as standby following node_3,
it fails to catch up:
LOG: primary server contains no more WAL on requested timeline 1
LOG: new timeline 2 forked off current database system timeline 1 before current recovery point 0/30000A0
That's the situation where you'd need to use pg_rewind, but in this case
it happens already when we are just setting up the actual pg_rewind
scenario we want to test, so change the test so that it waits until
node_3 is connected and fully caught up before promoting it, so that you
get a clean, controlled failover.
Also rewrite some of the comments, for clarity. The existing comments
detailed what each step in the test did, but didn't give a good overview
of the situation the steps were trying to create.
For reasons I don't understand, the test setup had to be written slightly
differently in 9.6 and 9.5 than in later versions. The 9.5/9.6 version
needed node 1 to be reinitialized from backup, whereas in later versions
it could be shut down and reconfigured to be a standby. But even 9.5 should
support "clean switchover", where primary makes sure that pending WAL is
replicated to standby on shutdown. It would be nice to figure out what's
going on there, but that's independent of pg_rewind and the scenario that
this test tests.
Discussion: https://www.postgresql.org/message-id/b0a3b95b-82d2-6089-6892-40570f8c5e60%40iki.fi
M src/bin/pg_rewind/t/008_min_recovery_point.pl
doc: remove unnecessary blank before command option text
commit : f3f033a28426796653cadbc576fe83ec3ab60673
author : Bruce Momjian <[email protected]>
date : Thu, 3 Dec 2020 11:33:24 -0500
committer: Bruce Momjian <[email protected]>
date : Thu, 3 Dec 2020 11:33:24 -0500
Backpatch-through: 11
M doc/src/sgml/ref/pg_checksums.sgml
M doc/src/sgml/ref/pg_controldata.sgml
M doc/src/sgml/ref/pg_resetwal.sgml
docs: list single-letter options first in command-line summary
commit : d415eec304c3398caa12b910877f5b2bd8545d5f
author : Bruce Momjian <[email protected]>
date : Thu, 3 Dec 2020 10:28:58 -0500
committer: Bruce Momjian <[email protected]>
date : Thu, 3 Dec 2020 10:28:58 -0500
In a few places, the long-version options were listed before the
single-letter ones in the command summary of a few commands. This
didn't match other commands, and didn't match the option ordering later
in the same reference page.
Backpatch-through: 9.5
M doc/src/sgml/ref/pg_controldata.sgml
M doc/src/sgml/ref/pg_resetwal.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
Fix pg_rewind bugs when rewinding a standby server.
commit : 020f153dbb48519ebb9780ddbfe87b0532324d49
author : Heikki Linnakangas <[email protected]>
date : Thu, 3 Dec 2020 15:57:48 +0200
committer: Heikki Linnakangas <[email protected]>
date : Thu, 3 Dec 2020 15:57:48 +0200
If the target is a standby server, its WAL doesn't end at the last
checkpoint record, but at minRecoveryPoint. We must scan all the
WAL from the last common checkpoint all the way up to minRecoveryPoint
for modified pages, and also consider that portion when determining
whether the server needs rewinding.
Backpatch to all supported versions.
Author: Ian Barwick and me
Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
A src/bin/pg_rewind/t/008_min_recovery_point.pl
pg_checksums: data_checksum_version is unsigned so use %u not %d
commit : c50b6f7a9ca7786219646fb6b266ab6b18082aa3
author : Bruce Momjian <[email protected]>
date : Tue, 1 Dec 2020 20:27:05 -0500
committer: Bruce Momjian <[email protected]>
date : Tue, 1 Dec 2020 20:27:05 -0500
While the previous behavior didn't generate a warning, we might as well
use an accurate *printf specification.
Backpatch-through: 12
M src/bin/pg_checksums/pg_checksums.c
Ensure that expandTableLikeClause() re-examines the same table.
commit : 5f9b05ada1bb4b5576944227a2d330cde29683cb
author : Tom Lane <[email protected]>
date : Tue, 1 Dec 2020 14:02:28 -0500
committer: Tom Lane <[email protected]>
date : Tue, 1 Dec 2020 14:02:28 -0500
As it stood, expandTableLikeClause() re-did the same relation_openrv
call that transformTableLikeClause() had done. However there are
scenarios where this would not find the same table as expected.
We hold lock on the LIKE source table, so it can't be renamed or
dropped, but another table could appear before it in the search path.
This explains the odd behavior reported in bug #16758 when cloning a
table as a temp table of the same name. This case worked as expected
before commit 502898192 introduced the need to open the source table
twice, so we should fix it.
To make really sure we get the same table, let's re-open it by OID not
name. That requires adding an OID field to struct TableLikeClause,
which is a little nervous-making from an ABI standpoint, but as long
as it's at the end I don't think there's any serious risk.
Per bug #16758 from Marc Boeren. Like the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_utilcmd.c
M src/include/nodes/parsenodes.h
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Avoid memcpy() with a NULL source pointer and count == 0
commit : f0ff52f25cb9773de67303a77e6afe3b8a2c4d48
author : Alvaro Herrera <[email protected]>
date : Tue, 1 Dec 2020 11:46:56 -0300
committer: Alvaro Herrera <[email protected]>
date : Tue, 1 Dec 2020 11:46:56 -0300
When memcpy() is called on a pointer, the compiler is entitled to assume
that the pointer is not null, which can lead to optimizing nearby code
in potentially undesirable ways. We still want such optimizations
(gcc's -fdelete-null-pointer-checks) in cases where they're valid.
Related: commit 13bba02271dc.
Backpatch to pg11, where this particular instance appeared.
Reported-by: Ranier Vilela <[email protected]>
Reported-by: Zhihong Yu <[email protected]>
Discussion: https://postgr.es/m/CAEudQApUndmQkr5fLrCKXQ7+ib44i7S+Kk93pyVThS85PnG3bQ@mail.gmail.com
Discussion: https://postgr.es/m/CALNJ-vSdhwSM5f4tnNn1cdLHvXMVe_S+V3nR5GwNrmCPNB2VtQ@mail.gmail.com
M src/backend/commands/indexcmds.c
Free disk space for dropped relations on commit.
commit : 50f338574242be955c79ed0b3e193ce3ff4cd5eb
author : Thomas Munro <[email protected]>
date : Tue, 1 Dec 2020 13:21:03 +1300
committer: Thomas Munro <[email protected]>
date : Tue, 1 Dec 2020 13:21:03 +1300
When committing a transaction that dropped a relation, we previously
truncated only the first segment file to free up disk space (the one
that won't be unlinked until the next checkpoint).
Truncate higher numbered segments too, even though we unlink them on
commit. This frees the disk space immediately, even if other backends
have open file descriptors and might take a long time to get around to
handling shared invalidation events and closing them. Also extend the
same behavior to the first segment, in recovery.
Back-patch to all supported releases.
Bug: #16663
Reported-by: Denis Patron <[email protected]>
Reviewed-by: Pavel Borisov <[email protected]>
Reviewed-by: Neil Chen <[email protected]>
Reviewed-by: David Zhang <[email protected]>
Discussion: https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org
M src/backend/storage/smgr/md.c
Document concurrent indexes waiting on each other
commit : b2603f16ad7531a5e09b1f38a0b4382a946a139f
author : Alvaro Herrera <[email protected]>
date : Mon, 30 Nov 2020 18:24:55 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 30 Nov 2020 18:24:55 -0300
Because regular CREATE INDEX commands are independent, and there's no
logical data dependency, it's not immediately obvious that transactions
held by concurrent index builds on one table will block the second phase
of concurrent index creation on an unrelated table, so document this
caveat.
Backpatch this all the way back. In branch master, mention that only
some indexes are involved.
Author: James Coleman <[email protected]>
Reviewed-by: David Johnston <[email protected]>
Discussion: https://postgr.es/m/CAAaqYe994=PUrn8CJZ4UEo_S-FfRr_3ogERyhtdgHAb2WG_Ufg@mail.gmail.com
M doc/src/sgml/ref/create_index.sgml
Remove configure-time probe for DocBook DTD.
commit : ef1a785fa430bdb75f21fd01287dccd74e579a01
author : Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 15:24:13 -0500
committer: Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 15:24:13 -0500
Checking for DocBook being installed was valuable when we were on the
OpenSP docs toolchain, because that was rather hard to get installed
fully. Nowadays, as long as you have xmllint and xsltproc installed,
you're good, because those programs will fetch the DocBook files off
the net at need. Moreover, testing this at configure time means that
a network access may well occur whether or not you have any interest
in building the docs later. That can be slow (typically 2 or 3
seconds, though much higher delays have been reported), and it seems
not very nice to be doing an off-machine access without warning, too.
Hence, drop the PGAC_CHECK_DOCBOOK probe, and adjust related
documentation. Without that macro, there's not much left of
config/docbook.m4 at all, so I just removed it.
Back-patch to v11, where we started to use xmllint in the
PGAC_CHECK_DOCBOOK probe.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M aclocal.m4
D config/docbook.m4
M configure
M configure.in
M doc/src/sgml/docguide.sgml
Prevent parallel index build in a standalone backend.
commit : 16a1e1ea4fa323d5fc34883cbb2f61eccae8d45f
author : Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 14:38:00 -0500
committer: Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 14:38:00 -0500
This can't work if there's no postmaster, and indeed the code got an
assertion failure trying. There should be a check on IsUnderPostmaster
gating the use of parallelism, as the planner has for ordinary
parallel queries.
Commit 40d964ec9 got this right, so follow its model of checking
IsUnderPostmaster at the same place where we check for
max_parallel_maintenance_workers == 0. In general, new code
implementing parallel utility operations should do the same.
Report and patch by Yulin Pei, cosmetically adjusted by me.
Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/HK0PR01MB22747D839F77142D7E76A45DF4F50@HK0PR01MB2274.apcprd01.prod.exchangelabs.com
M src/backend/optimizer/plan/planner.c
Fix miscomputation of direct_lateral_relids for join relations.
commit : 4f728a1f5f06dab85f6b94fd062752a25239c6b6
author : Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 12:22:43 -0500
committer: Tom Lane <[email protected]>
date : Mon, 30 Nov 2020 12:22:43 -0500
If a PlaceHolderVar is to be evaluated at a join relation, but
its value is only needed there and not at higher levels, we neglected
to update the joinrel's direct_lateral_relids to include the PHV's
source rel. This causes problems because join_is_legal() then won't
allow joining the joinrel to the PHV's source rel at all, leading
to "failed to build any N-way joins" planner failures.
Per report from Andreas Seltenreich. Back-patch to 9.5
where the problem originated.
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/util/placeholder.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Remove leftover comments, left behind by removal of WITH OIDS.
commit : ee42f78910b7e4097044bea188a02564bff70d4e
author : Heikki Linnakangas <[email protected]>
date : Mon, 30 Nov 2020 10:26:43 +0200
committer: Heikki Linnakangas <[email protected]>
date : Mon, 30 Nov 2020 10:26:43 +0200
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqGaRoF3XrhPW-Y7P%2BG7bKo84Z_h%3DkQHvMh-80%3Dav3wmOw%40mail.gmail.com
M contrib/file_fdw/file_fdw.c
M src/backend/commands/copy.c
Fix recently-introduced breakage in psql's \connect command.
commit : 3958abdad33755cbdcae66dada1eb1d5b13db3d4
author : Tom Lane <[email protected]>
date : Sun, 29 Nov 2020 15:22:04 -0500
committer: Tom Lane <[email protected]>
date : Sun, 29 Nov 2020 15:22:04 -0500
Through my misreading of what the existing code actually did,
commits 85c54287a et al. broke psql's behavior for the case where
"\c connstring" provides a password in the connstring. We should
use that password in such a case, but as of 85c54287a we ignored it
(and instead, prompted for a password).
Commit 94929f1cf fixed that in HEAD, but since I thought it was
cleaning up a longstanding misbehavior and not one I'd just created,
I didn't back-patch it.
Hence, back-patch the portions of 94929f1cf having to do with
password management. In addition to fixing the introduced bug,
this means that "\c -reuse-previous=on connstring" will allow
re-use of an existing connection's password if the connstring
doesn't change user/host/port. That didn't happen before, but
it seems like a bug fix, and anyway I'm loath to have significant
differences in this code across versions.
Also fix an error with the same root cause about whether or not to
override a connstring's setting of client_encoding. As of 85c54287a
we always did so; restore the previous behavior of overriding only
when stdin/stdout are a terminal and there's no environment setting
of PGCLIENTENCODING. (I find that definition a bit surprising, but
right now doesn't seem like the time to revisit it.)
Per bug #16746 from Krzysztof Gradek. As with the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
Doc: clarify behavior of PQconnectdbParams().
commit : db83c0450929f89b293ae30a8d68d520a91b7ead
author : Tom Lane <[email protected]>
date : Sun, 29 Nov 2020 13:58:30 -0500
committer: Tom Lane <[email protected]>
date : Sun, 29 Nov 2020 13:58:30 -0500
The documentation omitted the critical tidbit that a keyword-array entry
is simply ignored if its corresponding value-array entry is NULL or an
empty string; it will *not* override any previously-obtained value for
the parameter. (See conninfo_array_parse().) I'd supposed that would
force the setting back to default, which is what led me into bug #16746;
but it doesn't.
While here, I couldn't resist the temptation to do some copy-editing,
both in the description of PQconnectdbParams() and in the section
about connection URI syntax.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/libpq.sgml
Retry initial slurp_file("current_logfiles"), in test 004_logrotate.pl.
commit : 7b5b90be95663f3feb1f7c141b07485d593268cf
author : Noah Misch <[email protected]>
date : Sat, 28 Nov 2020 21:52:27 -0800
committer: Noah Misch <[email protected]>
date : Sat, 28 Nov 2020 21:52:27 -0800
Buildfarm member topminnow failed when the test script attempted this
before the syslogger would have created the file. Back-patch to v12,
which introduced the test.
M src/bin/pg_ctl/t/004_logrotate.pl
Fix a recently-introduced race condition in LISTEN/NOTIFY handling.
commit : cbc7a7a10c2d91cfa0401e9f68b17f19b7d5e7d2
author : Tom Lane <[email protected]>
date : Sat, 28 Nov 2020 14:03:40 -0500
committer: Tom Lane <[email protected]>
date : Sat, 28 Nov 2020 14:03:40 -0500
Commit 566372b3d fixed some race conditions involving concurrent
SimpleLruTruncate calls, but it introduced new ones in async.c.
A newly-listening backend could attempt to read Notify SLRU pages that
were in process of being truncated, possibly causing an error. Also,
the QUEUE_TAIL pointer could become set to a value that's not equal to
the queue position of any backend. While that's fairly harmless in
v13 and up (thanks to commit 51004c717), in older branches it resulted
in near-permanent disabling of the queue truncation logic, so that
continued use of NOTIFY led to queue-fill warnings and eventual
inability to send any more notifies. (A server restart is enough to
make that go away, but it's still pretty unpleasant.)
The core of the problem is confusion about whether QUEUE_TAIL
represents the "logical" tail of the queue (i.e., the oldest
still-interesting data) or the "physical" tail (the oldest data we've
not yet truncated away). To fix, split that into two variables.
QUEUE_TAIL regains its definition as the logical tail, and we
introduce a new variable to track the oldest un-truncated page.
Per report from Mikael Gustavsson. Like the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/async.c
Fix CLUSTER progress reporting of number of blocks scanned.
commit : fce17e486f3579fd2617c12f332608a5e78fefac
author : Fujii Masao <[email protected]>
date : Fri, 27 Nov 2020 20:16:44 +0900
committer: Fujii Masao <[email protected]>
date : Fri, 27 Nov 2020 20:16:44 +0900
Previously pg_stat_progress_cluster view reported the current block
number in heap scan as the number of heap blocks scanned (i.e.,
heap_blks_scanned). This reported number could be incorrect when
synchronize_seqscans is enabled, because it allowed the heap scan to
start at block in middle. This could result in wraparounds in the
heap_blks_scanned column when the heap scan wrapped around.
This commit fixes the bug by calculating the number of blocks from
the block that the heap scan starts at to the current block in scan,
and reporting that number in the heap_blks_scanned column.
Also, in pg_stat_progress_cluster view, previously heap_blks_scanned
could not reach heap_blks_total at the end of heap scan phase
if the last pages scanned were empty. This commit fixes the bug by
manually updating heap_blks_scanned to the same value as
heap_blks_total when the heap scan phase finishes.
Back-patch to v12 where pg_stat_progress_cluster view was introduced.
Reported-by: Matthias van de Meent
Author: Matthias van de Meent
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAEze2WjCBWSGkVfYag001Rc4+-nNLDpWM7QbyD6yPvuhKs-gYQ@mail.gmail.com
M src/backend/access/heap/heapam_handler.c
In psql's \d commands, don't truncate attribute default values.
commit : ea7a167daa3e7c9c7b03fb09432aafcea1cf4d88
author : Tom Lane <[email protected]>
date : Wed, 25 Nov 2020 16:19:25 -0500
committer: Tom Lane <[email protected]>
date : Wed, 25 Nov 2020 16:19:25 -0500
Historically, psql has truncated the text of a column's default
expression at 128 characters. This is unlike any other behavior
in describe.c, and it's become particularly confusing now that
the limit is only applied to the expression proper and not to
the "generated always as (...) stored" text that may get wrapped
around it.
Excavation in our git history suggests that the original motivation
for this limit was not really to limit the display width (as I'd long
supposed), but to make it safe to use a fixed-width output buffer to
store the result. That implementation restriction is long gone of
course, but the limit remained. Let's just get rid of it.
While here, rearrange the logic about when to free the output string
so that it's not so dependent on unstated assumptions about the
possible values of attidentity and attgenerated.
Per bug #16743 from David Turon. Back-patch to v12 where GENERATED
came in. (Arguably we could take it back further, but I'm hesitant
to change the behavior of long-stable branches for this.)
Discussion: https://postgr.es/m/[email protected]
M src/bin/psql/describe.c
doc: Fix typos
commit : b608645c17d7979357ebdbc9f461bfca54a6a415
author : Peter Eisentraut <[email protected]>
date : Wed, 25 Nov 2020 09:49:00 +0100
committer: Peter Eisentraut <[email protected]>
date : Wed, 25 Nov 2020 09:49:00 +0100
Author: Justin Pryzby <[email protected]>
Discussion: https://www.postgresql.org/message-id/[email protected]
M doc/src/sgml/contrib.sgml
Properly check index mark/restore in ExecSupportsMarkRestore.
commit : ae5aa26dc39d7729b044013e8b213e79e24e613b
author : Andrew Gierth <[email protected]>
date : Tue, 24 Nov 2020 20:58:32 +0000
committer: Andrew Gierth <[email protected]>
date : Tue, 24 Nov 2020 20:58:32 +0000
Previously this code assumed that all IndexScan nodes supported
mark/restore, which is not true since it depends on optional index AM
support functions. This could lead to errors about missing support
functions in rare edge cases of mergejoins with no sort keys, where an
unordered non-btree index scan was placed on the inner path without a
protecting Materialize node. (Normally, the fact that merge join
requires ordered input would avoid this error.)
Backpatch all the way since this bug is ancient.
Per report from Eugen Konkov on irc.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/execAmi.c
M src/backend/optimizer/util/plancat.c
M src/include/nodes/pathnodes.h
Skip allocating hash table in EXPLAIN-only mode.
commit : 888fa2baeb39fa4069b1b55552c0ced85b533313
author : Heikki Linnakangas <[email protected]>
date : Fri, 20 Nov 2020 14:41:14 +0200
committer: Heikki Linnakangas <[email protected]>
date : Fri, 20 Nov 2020 14:41:14 +0200
This is a backpatch of commit 2cccb627f1, backpatched due to popular
demand. Backpatch to all supported versions.
Author: Alexey Bashtanov
Discussion: https://www.postgresql.org/message-id/36823f65-050d-ae24-aa4d-a37726998240%40imap.cc
M src/backend/executor/nodeAgg.c
On macOS, use -isysroot in link steps as well as compile steps.
commit : 5b83604270a6ffc06e8e9406d19b9f2e845b74f9
author : Tom Lane <[email protected]>
date : Fri, 20 Nov 2020 00:58:26 -0500
committer: Tom Lane <[email protected]>
date : Fri, 20 Nov 2020 00:58:26 -0500
We previously put the -isysroot switch only into CPPFLAGS, theorizing
that it was only needed to find the right copies of include files.
However, it seems that we also need to use it while linking programs,
to find the right stub ".tbd" files for libraries. We got away
without that up to now, but apparently that was mostly luck. It may
also be that failures are only observed when the Xcode version is
noticeably out of sync with the host macOS version; the case that's
prompting action right now is that builds fail when using latest Xcode
(12.2) on macOS Catalina, even though it's fine on Big Sur.
Hence, add -isysroot to LDFLAGS as well. (It seems that the more
common practice is to put it in CFLAGS, whence it'd be included at
both compile and link steps. However, we can't mess with CFLAGS in
the platform template file without confusing configure's logic for
choosing default CFLAGS.)
Back-patch of 49407dc32 into all supported branches.
Report and patch by James Hilliard (some cosmetic mods by me)
Discussion: https://postgr.es/m/[email protected]
M configure
M configure.in
M src/template/darwin
Adjust DSM and DSA slot usage constants (back-patch).
commit : 2ded1f1fbb6e8e2b7927cce93238775604069bc3
author : Thomas Munro <[email protected]>
date : Fri, 20 Nov 2020 10:44:09 +1300
committer: Thomas Munro <[email protected]>
date : Fri, 20 Nov 2020 10:44:09 +1300
1. Previously, a DSA area would create up to four segments at each size
before doubling the size. After this commit, it will create only two at
each size, so it ramps up faster and therefore needs fewer slots.
2. Previously, the total limit on DSM slots allowed for 2 per connection.
Switch to 5 per connection.
This back-patches commit d061ea21 from release 13 into 10-12 based on a
field complaint.
Discussion: https://postgr.es/m/CAO03teA%2BjE1qt5iWDWzHqaufqBsF6EoOgZphnazps_tr_jDPZA%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2BhUKGL6H2BpGbiF7Lj6QiTjTGyTLW_vLR%3DSn2tEBeTcYXiMKw%40mail.gmail.com
M src/backend/storage/ipc/dsm.c
M src/backend/utils/mmgr/dsa.c
Further fixes for CREATE TABLE LIKE: cope with self-referential FKs.
commit : 87ab464219259abdc06b0492f0d42bc6734ae490
author : Tom Lane <[email protected]>
date : Thu, 19 Nov 2020 15:03:17 -0500
committer: Tom Lane <[email protected]>
date : Thu, 19 Nov 2020 15:03:17 -0500
Commit 502898192 was too careless about the order of execution of the
additional ALTER TABLE operations generated by expandTableLikeClause.
It just stuck them all at the end, which seems okay for most purposes.
But it falls down in the case where LIKE is importing a primary key
or unique index and the outer CREATE TABLE includes a FOREIGN KEY
constraint that needs to depend on that index. Weird as that is,
it used to work, so we ought to keep it working.
To fix, make parse_utilcmd.c insert LIKE clauses between index-creation
and FK-creation commands in the transformed list of commands, and change
utility.c so that the commands generated by expandTableLikeClause are
executed immediately not at the end. One could imagine scenarios where
this wouldn't work either; but currently expandTableLikeClause only
makes column default expressions, CHECK constraints, and indexes, and
this ordering seems fine for those.
Per bug #16730 from Sofoklis Papasofokli. Like the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/parse_utilcmd.c
M src/backend/tcop/utility.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Don't Insert() a VFD entry until it's fully built.
commit : 4097a79069be10c9883d1fef566f2921621e1852
author : Tom Lane <[email protected]>
date : Mon, 16 Nov 2020 20:32:35 -0500
committer: Tom Lane <[email protected]>
date : Mon, 16 Nov 2020 20:32:35 -0500
Otherwise, if FDDEBUG is enabled, the debugging output fails because
it tries to read the fileName, which isn't set up yet (and should in
fact always be NULL).
AFAICT, this has been wrong since Berkeley. Before 96bf88d52,
it would accidentally fail to crash on platforms where snprintf()
is forgiving about being passed a NULL pointer for %s; but the
file name intended to be included in the debug output wouldn't
ever have shown up.
Report and fix by Greg Nancarrow. Although this is only visibly
broken in custom-made builds, it still seems worth back-patching
to all supported branches, as the FDDEBUG code is pretty useless
as it stands.
Discussion: https://postgr.es/m/CAJcOf-cUDgm9qYtC_B6XrC6MktMPNRby2p61EtSGZKnfotMArw@mail.gmail.com
M src/backend/storage/file/fd.c
Do not return NULL for error cases in satisfies_hash_partition().
commit : 54a94064407375ac3c108e1b791303a42baf030a
author : Tom Lane <[email protected]>
date : Mon, 16 Nov 2020 16:39:59 -0500
committer: Tom Lane <[email protected]>
date : Mon, 16 Nov 2020 16:39:59 -0500
Since this function is used as a CHECK constraint condition,
returning NULL is tantamount to returning TRUE, which would have the
effect of letting in a row that doesn't satisfy the hash condition.
Admittedly, the cases for which this is done should be unreachable
in practice, but that doesn't make it any less a bad idea. It also
seems like a dartboard was used to decide which error cases should
throw errors as opposed to returning NULL.
For the checks for NULL input values, I just switched it to returning
false. There's some argument that an error would be better; but the
case really should be can't-happen in a generated hash constraint,
so it's likely not worth more code for.
For the parent-relation-open-failure case, it seems like we might
as well let relation_open throw an error, instead of having an
impossible-to-diagnose constraint failure.
Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/[email protected]
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/hash_part.out
Use "true" not "TRUE" in one ICU function call.
commit : 029fa664ecb8687eb60633b4ccf230b457a1fb77
author : Tom Lane <[email protected]>
date : Mon, 16 Nov 2020 15:16:39 -0500
committer: Tom Lane <[email protected]>
date : Mon, 16 Nov 2020 15:16:39 -0500
This was evidently missed in commit 6337865f3, which generally did
s/TRUE/true/ everywhere. It escaped notice up to now because ICU
versions before ICU 68 provided definitions of "TRUE" and "FALSE"
regardless. With ICU 68, it fails to compile.
Per report from Condor. Back-patch to v11 where 6337865f3 came in.
(I've not tested v10, where this call originated, but I imagine
it's fine since we defined TRUE in c.h back then.)
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/collationcmds.c
doc: update bgwriter description
commit : 139c439730b1444be88203cf867e60dd7533b294
author : Bruce Momjian <[email protected]>
date : Mon, 16 Nov 2020 13:13:43 -0500
committer: Bruce Momjian <[email protected]>
date : Mon, 16 Nov 2020 13:13:43 -0500
This clarifies exactly what the bgwriter does, which should help with
tuning.
Reported-by: Chris Wilson
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
doc: clarify how to find pg_type_d.h in the install tree
commit : 3da1110b8dfdd3d4c70715ef5cf2870ae94a30c3
author : Bruce Momjian <[email protected]>
date : Mon, 16 Nov 2020 12:36:17 -0500
committer: Bruce Momjian <[email protected]>
date : Mon, 16 Nov 2020 12:36:17 -0500
Followup to patch 152ed04799.
Reported-by: Alvaro Herrera
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
doc: improve wording of the need for analyze of exp. indexes
commit : 260493aed2e8863fe76e7c5f0c02f535f40000fe
author : Bruce Momjian <[email protected]>
date : Mon, 16 Nov 2020 10:26:17 -0500
committer: Bruce Momjian <[email protected]>
date : Mon, 16 Nov 2020 10:26:17 -0500
This is a followup commit on 3370207986.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/ref/create_index.sgml
Fix typo
commit : b3e7c202b79e4d6bce6a65c957fe7f813d5138f8
author : Alvaro Herrera <[email protected]>
date : Mon, 16 Nov 2020 10:54:11 -0300
committer: Alvaro Herrera <[email protected]>
date : Mon, 16 Nov 2020 10:54:11 -0300
Introduced in 90fdc259866e; backpatch to 12.
Author: Erik Rijkers <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/create_index.sgml
Fix fuzzy thinking about amcanmulticol versus amcaninclude.
commit : 4ac8ee9d4878dadccc2331d8d1b1769c97f4ebe9
author : Tom Lane <[email protected]>
date : Sun, 15 Nov 2020 16:10:48 -0500
committer: Tom Lane <[email protected]>
date : Sun, 15 Nov 2020 16:10:48 -0500
These flags should be independent: in particular an index AM should
be able to say that it supports include columns without necessarily
supporting multiple key columns. The included-columns patch got
this wrong, possibly aided by the fact that it didn't bother to
update the documentation.
While here, clarify some text about amcanreturn, which was a little
vague about what should happen when amcanreturn reports that only
some of the index columns are returnable.
Noted while reviewing the SP-GiST included-columns patch, which
quite incorrectly (and unsafely) changed SP-GiST to claim
amcanmulticol = true as a workaround for this bug.
Backpatch to v11 where included columns were introduced.
M doc/src/sgml/indexam.sgml
M src/backend/commands/indexcmds.c
Doc: improve partitioning discussion in ddl.sgml.
commit : 2b77595085423970036afd6263046c5747534dd0
author : Tom Lane <[email protected]>
date : Sat, 14 Nov 2020 13:09:53 -0500
committer: Tom Lane <[email protected]>
date : Sat, 14 Nov 2020 13:09:53 -0500
This started with the intent to explain that range upper bounds
are exclusive, which previously you could only find out by reading
the CREATE TABLE man page. But I soon found that section 5.11
really could stand a fair amount of editorial attention. It's
apparently been revised several times without much concern for
overall flow, nor careful copy-editing.
Back-patch to v11, which is as far as the patch goes easily.
Per gripe from Edson Richter. Thanks to David Johnston for review.
Discussion: https://postgr.es/m/DM6PR13MB3988736CF8F5DC5720440231CFE60@DM6PR13MB3988.namprd13.prod.outlook.com
M doc/src/sgml/ddl.sgml
doc: clarify where to find pg_type_d.h (PG 11+) and pg_type.h
commit : 36051198dfa75c9b7e19ae59e458c12f48d67abf
author : Bruce Momjian <[email protected]>
date : Thu, 12 Nov 2020 15:13:01 -0500
committer: Bruce Momjian <[email protected]>
date : Thu, 12 Nov 2020 15:13:01 -0500
These files are in compiled directories and install directories.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
docs: mention that expression indexes need analyze
commit : 90fdc259866ed800283620c8fe6329015ad58e5c
author : Bruce Momjian <[email protected]>
date : Thu, 12 Nov 2020 15:00:44 -0500
committer: Bruce Momjian <[email protected]>
date : Thu, 12 Nov 2020 15:00:44 -0500
Expression indexes can't benefit from pre-computed statistics on
columns.
Reported-by: Nikolay Samokhvalov
Discussion: https://postgr.es/m/CANNMO++5rw9RDA=p40iMVbMNPaW6O=S0AFzTU=KpYHRpCd1voA@mail.gmail.com
Author: Nikolay Samokhvalov, modified
Backpatch-through: 9.5
M doc/src/sgml/ref/create_index.sgml
doc: wire protocol data type for history file content is bytea
commit : d7fa90fae104a81cc5a1a76a9607afcd3d0ad070
author : Bruce Momjian <[email protected]>
date : Thu, 12 Nov 2020 14:33:28 -0500
committer: Bruce Momjian <[email protected]>
date : Thu, 12 Nov 2020 14:33:28 -0500
Document that though the history file content is marked as bytea, it is
the same a text, and neither is btyea-escaped or encoding converted.
Reported-by: Brar Piening
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 13 - 9.5 (not master)
M doc/src/sgml/protocol.sgml
M src/backend/replication/walsender.c
pg_trgm: fix crash in 2-item picksplit
commit : 7f69ed4aebe342365dc7db96f33b473dec9f054b
author : Andrew Gierth <[email protected]>
date : Thu, 12 Nov 2020 14:34:37 +0000
committer: Andrew Gierth <[email protected]>
date : Thu, 12 Nov 2020 14:34:37 +0000
Whether from size overflow in gistSplit or from secondary splits,
picksplit is (rarely) called with exactly two items to split.
Formerly, due to special-case handling of the last item, this would
lead to access to an uninitialized cache entry; prior to PG 13 this
might have been harmless or at worst led to an incorrect union datum,
but in 13 onwards it can cause a backend crash from using an
uninitialized pointer.
Repair by removing the special case, which was deemed not to have been
appropriate anyway. Backpatch all the way, because this bug has
existed since pg_trgm was added.
Per report on IRC from user "ftzdomino". Analysis and testing by me,
patch from Alexander Korotkov.
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_trgm/trgm_gist.c
Remove duplicate code in brin_memtuple_initialize
commit : 0d0626e27d56a52f78cd71a796ee2e8fe216f217
author : Tomas Vondra <[email protected]>
date : Wed, 11 Nov 2020 18:37:36 +0100
committer: Tomas Vondra <[email protected]>
date : Wed, 11 Nov 2020 18:37:36 +0100
Commit 8bf74967dab moved some of the code from brin_new_memtuple to
brin_memtuple_initialize, but this resulted in some of the code being
duplicate. Fix by removing the duplicate lines and backpatch to 10.
Author: Tomas Vondra
Backpatch-through: 10
Discussion: https://postgr.es/m/5eb50c97-9a8e-b691-8c40-1b2a55611c4c%40enterprisedb.com
M src/backend/access/brin/brin_tuple.c
Fix and simplify some usages of TimestampDifference().
commit : 171c457cd06051b5910cf357ea6a77643b69088a
author : Tom Lane <[email protected]>
date : Tue, 10 Nov 2020 22:51:19 -0500
committer: Tom Lane <[email protected]>
date : Tue, 10 Nov 2020 22:51:19 -0500
Introduce TimestampDifferenceMilliseconds() to simplify callers
that would rather have the difference in milliseconds, instead of
the select()-oriented seconds-and-microseconds format. This gets
rid of at least one integer division per call, and it eliminates
some apparently-easy-to-mess-up arithmetic.
Two of these call sites were in fact wrong:
* pg_prewarm's autoprewarm_main() forgot to multiply the seconds
by 1000, thus ending up with a delay 1000X shorter than intended.
That doesn't quite make it a busy-wait, but close.
* postgres_fdw's pgfdw_get_cleanup_result() thought it needed to compute
microseconds not milliseconds, thus ending up with a delay 1000X longer
than intended. Somebody along the way had noticed this problem but
misdiagnosed the cause, and imposed an ad-hoc 60-second limit rather
than fixing the units. This was relatively harmless in context, because
we don't care that much about exactly how long this delay is; still,
it's wrong.
There are a few more callers of TimestampDifference() that don't
have a direct need for seconds-and-microseconds, but can't use
TimestampDifferenceMilliseconds() either because they do need
microsecond precision or because they might possibly deal with
intervals long enough to overflow 32-bit milliseconds. It might be
worth inventing another API to improve that, but that seems outside
the scope of this patch; so those callers are untouched here.
Given the fact that we are fixing some bugs, and the likelihood
that future patches might want to back-patch code that uses this
new API, back-patch to all supported branches.
Alexey Kondratov and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M contrib/pg_prewarm/autoprewarm.c
M contrib/postgres_fdw/connection.c
M src/backend/access/transam/xlog.c
M src/backend/replication/walreceiverfuncs.c
M src/backend/replication/walsender.c
M src/backend/utils/adt/timestamp.c
M src/include/utils/timestamp.h
doc: fix spelling "connction" to "connection"
commit : 1393acb3836d32e512801bb00f0cc6b4d2a58dc8
author : Bruce Momjian <[email protected]>
date : Tue, 10 Nov 2020 19:18:35 -0500
committer: Bruce Momjian <[email protected]>
date : Tue, 10 Nov 2020 19:18:35 -0500
Was wrong in commit 1a9388bd0f.
Reported-by: Tom Lane, Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 9.5
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_receivewal.sgml
Work around cross-version-upgrade issues created by commit 9e38c2bb5.
commit : 5ce51f8280d3e2a59dbcd93c9d8244e3bf96944d
author : Tom Lane <[email protected]>
date : Tue, 10 Nov 2020 18:32:36 -0500
committer: Tom Lane <[email protected]>
date : Tue, 10 Nov 2020 18:32:36 -0500
Summarily changing the STYPE of regression-test aggregates that
depend on array_append or array_cat is an issue for the buildfarm's
cross-version-upgrade tests, because those aggregates (as defined
in the back branches) now won't load into HEAD. Although this seems
like only a minimal risk for genuine user-defined aggregates, we
need to do something for the buildfarm. Hence, adjust the aggregate
definitions, in both HEAD and the back branches.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
M src/test/regress/expected/polymorphism.out
M src/test/regress/sql/polymorphism.sql