PostgreSQL 11.14 (upcoming) commit log

Fix variable shadowing in procarray.c.

commit   : 8d28ec21f0c3b2defb94afd168efa0a1c60b1a2d    
  
author   : Fujii Masao <fujii@postgresql.org>    
date     : Thu, 16 Sep 2021 13:07:29 +0900    
  
committer: Fujii Masao <fujii@postgresql.org>    
date     : Thu, 16 Sep 2021 13:07:29 +0900    

Click here for diff

ProcArrayGroupClearXid function has a parameter named "proc",  
but the same name was used for its local variables. This commit fixes  
this variable shadowing, to improve code readability.  
  
Back-patch to all supported versions, to make future back-patching  
easy though this patch is classified as refactoring only.  
  
Reported-by: Ranier Vilela  
Author: Ranier Vilela, Aleksander Alekseev  
https://postgr.es/m/CAEudQAqyoTZC670xWi6w-Oe2_Bk1bfu2JzXz6xRfiOUzm7xbyQ@mail.gmail.com  

M src/backend/storage/ipc/procarray.c

jit: Do not try to shut down LLVM state in case of LLVM triggered errors.

commit   : dccffd9a274dfbf343008c4da8029ed8b1f35ea2    
  
author   : Andres Freund <andres@anarazel.de>    
date     : Mon, 13 Sep 2021 18:07:19 -0700    
  
committer: Andres Freund <andres@anarazel.de>    
date     : Mon, 13 Sep 2021 18:07:19 -0700    

Click here for diff

If an allocation failed within LLVM it is not safe to call back into LLVM as  
LLVM is not generally safe against exceptions / stack-unwinding. Thus errors  
while in LLVM code are promoted to FATAL. However llvm_shutdown() did call  
back into LLVM even in such cases, while llvm_release_context() was careful  
not to do so.  
  
We cannot generally skip shutting down LLVM, as that can break profiling. But  
it's OK to do so if there was an error from within LLVM.  
  
Reported-By: Jelte Fennema <Jelte.Fennema@microsoft.com>  
Author: Andres Freund <andres@anarazel.de>  
Author: Justin Pryzby <pryzby@telsasoft.com>  
Discussion: https://postgr.es/m/AM5PR83MB0178C52CCA0A8DEA0207DC14F7FF9@AM5PR83MB0178.EURPRD83.prod.outlook.com  
Backpatch: 11-, where jit was introduced  

M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_error.cpp
M src/include/jit/llvmjit.h

Fix EXIT out of outermost block in plpgsql.

commit   : bdd6ce48d0356fbfaa0fe4a1fee20068f99abb70    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 13 Sep 2021 12:42:03 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 13 Sep 2021 12:42:03 -0400    

Click here for diff

Ordinarily, using EXIT this way would draw "control reached end of  
function without RETURN".  However, if the function is one where we  
don't require an explicit RETURN (such as a DO block), that should  
not happen.  It did anyway, because add_dummy_return() neglected to  
account for the case.  
  
Per report from Herwig Goemans.  Back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/868ae948-e3ca-c7ec-95a6-83cfc08ef750@gmail.com  

M src/pl/plpgsql/src/expected/plpgsql_control.out
M src/pl/plpgsql/src/pl_comp.c
M src/pl/plpgsql/src/sql/plpgsql_control.sql

Fix error handling with threads on OOM in ECPG connection logic

commit   : b6a10ff4abe9538f16d9f52bda6bc04f90b77d0d    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Mon, 13 Sep 2021 13:24:35 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Mon, 13 Sep 2021 13:24:35 +0900    

Click here for diff

An out-of-memory failure happening when allocating the structures to  
store the connection parameter keywords and values would mess up with  
the set of connections saved, as on failure the pthread mutex would  
still be hold with the new connection object listed but free()'d.  
  
Rather than just unlocking the mutex, which would leave the static list  
of connections into an inconsistent state, move the allocation for the  
structures of the connection parameters before beginning the test  
manipulation.  This ensures that the list of connections and the  
connection mutex remain consistent all the time in this code path.  
  
This error is unlikely going to happen, but this could mess up badly  
with ECPG clients in surprising ways, so backpatch all the way down.  
  
Reported-by: ryancaicse  
Discussion: https://postgr.es/m/17186-b4cfd8f0eb4d1dee@postgresql.org  
Backpatch-through: 9.6  

M src/interfaces/ecpg/ecpglib/connect.c

Make pg_regexec() robust against out-of-range search_start.

commit   : 3be381a900df75de6a085cc3a465d85ec8bf39cd    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 11 Sep 2021 15:19:31 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 11 Sep 2021 15:19:31 -0400    

Click here for diff

If search_start is greater than the length of the string, we should just  
return REG_NOMATCH immediately.  (Note that the equality case should  
*not* be rejected, since the pattern might be able to match zero  
characters.)  This guards various internal assumptions that the min of a  
range of string positions is not more than the max.  Violation of those  
assumptions could allow an attempt to fetch string[search_start-1],  
possibly causing a crash.  
  
Jaime Casanova pointed out that this situation is reachable with the  
new regexp_xxx functions that accept a user-specified start position.  
I don't believe it's reachable via any in-core call site in v14 and  
below.  However, extensions could possibly call pg_regexec with an  
out-of-range search_start, so let's back-patch the fix anyway.  
  
Discussion: https://postgr.es/m/20210911180357.GA6870@ahch-to  

M src/backend/regex/regexec.c

Fix some anomalies with NO SCROLL cursors.

commit   : 9ea8d3d24a9fb2bbe99cb17f6ee08747f662aabd    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 10 Sep 2021 13:18:32 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 10 Sep 2021 13:18:32 -0400    

Click here for diff

We have long forbidden fetching backwards from a NO SCROLL cursor,  
but the prohibition didn't extend to cases in which we rewind the  
query altogether and then re-fetch forwards.  I think the reason is  
that this logic was mainly meant to protect plan nodes that can't  
be run in the reverse direction.  However, re-reading the query output  
is problematic if the query is volatile (which includes SELECT FOR  
UPDATE, not just queries with volatile functions): the re-read can  
produce different results, which confuses the cursor navigation logic  
completely.  Another reason for disliking this approach is that some  
code paths will either fetch backwards or rewind-and-fetch-forwards  
depending on the distance to the target row; so that seemingly  
identical use-cases may or may not draw the "cursor can only scan  
forward" error.  Hence, let's clean things up by disallowing rewind  
as well as fetch-backwards in a NO SCROLL cursor.  
  
Ordinarily we'd only make such a definitional change in HEAD, but  
there is a third reason to consider this change now.  Commit ba2c6d6ce  
created some new user-visible anomalies for non-scrollable cursors  
WITH HOLD, in that navigation in the cursor result got confused if the  
cursor had been partially read before committing.  The only good way  
to resolve those anomalies is to forbid rewinding such a cursor, which  
allows removal of the incorrect cursor state manipulations that  
ba2c6d6ce added to PersistHoldablePortal.  
  
To minimize the behavioral change in the back branches (including  
v14), refuse to rewind a NO SCROLL cursor only when it has a holdStore,  
ie has been held over from a previous transaction due to WITH HOLD.  
This should avoid breaking most applications that have been sloppy  
about whether to declare cursors as scrollable.  We'll enforce the  
prohibition across-the-board beginning in v15.  
  
Back-patch to v11, as ba2c6d6ce was.  
  
Discussion: https://postgr.es/m/3712911.1631207435@sss.pgh.pa.us  

M src/backend/commands/portalcmds.c
M src/backend/tcop/pquery.c
M src/test/regress/expected/portals.out
M src/test/regress/sql/portals.sql

Avoid fetching from an already-terminated plan.

commit   : 7813451c20c3b341c8af542d80bb941c9c893825    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 9 Sep 2021 13:36:31 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 9 Sep 2021 13:36:31 -0400    

Click here for diff

Some plan node types don't react well to being called again after  
they've already returned NULL.  PortalRunSelect() has long dealt  
with this by calling the executor with NoMovementScanDirection  
if it sees that we've already run the portal to the end.  However,  
commit ba2c6d6ce overlooked this point, so that persisting an  
already-fully-fetched cursor would fail if it had such a plan.  
  
Per report from Tomas Barton.  Back-patch to v11, as the faulty  
commit was.  (I've omitted a test case because the type of plan  
that causes a problem isn't all that stable.)  
  
Discussion: https://postgr.es/m/CAPV2KRjd=ErgVGbvO2Ty20tKTEZZr6cYsYLxgN_W3eAo9pf5sw@mail.gmail.com  

M src/backend/commands/portalcmds.c

Check for relation length overrun soon enough.

commit   : 1a23b669db576460a42b257379335a447229ffa6    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 9 Sep 2021 11:45:48 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 9 Sep 2021 11:45:48 -0400    

Click here for diff

We don't allow relations to exceed 2^32-1 blocks, because block  
numbers are 32 bits and the last possible block number is reserved  
to mean InvalidBlockNumber.  There is a check for this in mdextend,  
but that's really way too late, because the smgr API requires us to  
create a buffer for the block-to-be-added, and we do not want to  
have any buffer with blocknum InvalidBlockNumber.  (Such a case  
can trigger assertions in bufmgr.c, plus I think it might confuse  
ReadBuffer's logic for data-past-EOF later on.)  So put the check  
into ReadBuffer.  
  
Per report from Christoph Berg.  It's been like this forever,  
so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/YTn1iTkUYBZfcODk@msg.credativ.de  

M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/smgr/md.c

Fix issue with WAL archiving in standby.

commit   : aacb3cfaf5e8b69da19446ad6a8e7ce426edb898    
  
author   : Fujii Masao <fujii@postgresql.org>    
date     : Thu, 9 Sep 2021 23:58:54 +0900    
  
committer: Fujii Masao <fujii@postgresql.org>    
date     : Thu, 9 Sep 2021 23:58:54 +0900    

Click here for diff

Previously, walreceiver always closed the currently-opened WAL segment  
and created its archive notification file, after it finished writing  
the current segment up and received any WAL data that should be  
written into the next segment. If walreceiver exited just before  
any WAL data in the next segment arrived at standby, it did not  
create the archive notification file of the current segment  
even though that's known completed. This behavior could cause  
WAL archiving of the segment to be delayed until subsequent  
restartpoints or checkpoints created its notification file.  
  
To fix the issue, this commit changes walreceiver so that it creates  
an archive notification file of a current WAL segment immediately  
if that's known completed before receiving next WAL data.  
  
Back-patch to all supported branches.  
  
Reported-by: Kyotaro Horiguchi  
Author: Fujii Masao  
Reviewed-by: Kyotaro Horiguchi  
Discussion: https://postgr.es/m/20200630.165503.1465894182551545886.horikyota.ntt@gmail.com  

M src/backend/replication/walreceiver.c

Avoid useless malloc/free traffic around getFormattedTypeName().

commit   : 4559c161cb5db350690c2b942fa6fecb3a3e79d3    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 8 Sep 2021 15:09:43 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 8 Sep 2021 15:09:43 -0400    

Click here for diff

Coverity complained that one caller of getFormattedTypeName() failed  
to free the returned string.  Which is true, but rather than fixing  
that one, let's get rid of this tedious and error-prone requirement.  
Now that getFormattedTypeName() caches its result, strdup'ing that  
result and expecting the caller to free it accomplishes little except  
to waste cycles.  We do create a leak in the case where getTypes didn't  
make a TypeInfo for the type, but that basically shouldn't ever happen.  
  
Back-patch, as commit 6c450a861 was.  This isn't a particularly  
interesting bug fix, but the API change seems like a hazard for  
future back-patching activity if we don't back-patch it.  

M src/bin/pg_dump/pg_dump.c

Fix rewriter to set hasModifyingCTE correctly on rewritten queries.

commit   : 882b7e7288353ad07e9419ffe62743c552b8008d    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 8 Sep 2021 12:05:43 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 8 Sep 2021 12:05:43 -0400    

Click here for diff

If we copy data-modifying CTEs from the original query to a replacement  
query (from a DO INSTEAD rule), we must set hasModifyingCTE properly  
in the replacement query.  Failure to do this can cause various  
unpleasantness, such as unsafe usage of parallel plans.  The code also  
neglected to propagate hasRecursive, though that's only cosmetic at  
the moment.  
  
A difficulty arises if the rule action is an INSERT...SELECT.  We  
attach the original query's RTEs and CTEs to the sub-SELECT Query, but  
data-modifying CTEs are only allowed to appear in the topmost Query.  
For the moment, throw an error in such cases.  It would probably be  
possible to avoid this error by attaching the CTEs to the top INSERT  
Query instead; but that would require a bunch of new code to adjust  
ctelevelsup references.  Given the narrowness of the use-case, and  
the need to back-patch this fix, it does not seem worth the trouble  
for now.  We can revisit this if we get field complaints.  
  
Per report from Greg Nancarrow.  Back-patch to all supported branches.  
(The test case added here does not fail before v10, but there are  
plenty of places checking top-level hasModifyingCTE in 9.6, so I have  
no doubt that this code change is necessary there too.)  
  
Greg Nancarrow and Tom Lane  
  
Discussion: https://postgr.es/m/CAJcOf-f68DT=26YAMz_i0+Au3TcLO5oiHY5=fL6Sfuits6r+_w@mail.gmail.com  
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com  

M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql

Invalidate relcache for publications defined for all tables.

commit   : 96e38fa5e549e9891b0f0dc3baf41e273ad3ede5    
  
author   : Amit Kapila <akapila@postgresql.org>    
date     : Wed, 8 Sep 2021 10:59:21 +0530    
  
committer: Amit Kapila <akapila@postgresql.org>    
date     : Wed, 8 Sep 2021 10:59:21 +0530    

Click here for diff

Updates/Deletes on a relation were allowed even without replica identity  
after we define the publication for all tables. This would later lead to  
an error on subscribers. The reason was that for such publications we were  
not invalidating the relcache and the publication information for  
relations was not getting rebuilt. Similarly, we were not invalidating the  
relcache after dropping of such publications which will prohibit  
Updates/Deletes without replica identity even without any publication.  
  
Author: Vignesh C and Hou Zhijie  
Reviewed-by: Hou Zhijie, Kyotaro Horiguchi, Amit Kapila  
Backpatch-through: 10, where it was introduced  
Discussion: https://postgr.es/m/CALDaNm0pF6zeWqCA8TCe2sDuwFAy8fCqba=nHampCKag-qLixg@mail.gmail.com  

M src/backend/commands/publicationcmds.c
M src/test/regress/expected/publication.out
M src/test/regress/sql/publication.sql

Fix bogus timetz_zone() results for DYNTZ abbreviations.

commit   : 90b4647f63dae68181557d244717a7615831243d    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 6 Sep 2021 11:29:52 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 6 Sep 2021 11:29:52 -0400    

Click here for diff

timetz_zone() delivered completely wrong answers if the zone was  
specified by a dynamic TZ abbreviation, because it failed to account  
for the difference between the POSIX conventions for field values in  
struct pg_tm and the conventions used in PG-specific datetime code.  
  
As a stopgap fix, just adjust the tm_year and tm_mon fields to match  
PG conventions.  This is fixed in a different way in HEAD (388e71af8)  
but I don't want to back-patch the change of reference point.  
  
Discussion: https://postgr.es/m/CAJ7c6TOMG8zSNEZtCn5SPe+cCk3Lfxb71ZaQwT2F4T7PJ_t=KA@mail.gmail.com  

M src/backend/utils/adt/date.c

Further portability tweaks for float4/float8 hash functions.

commit   : 8782a84529035baca1639c3680458e2062ef9b44    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 4 Sep 2021 16:29:08 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 4 Sep 2021 16:29:08 -0400    

Click here for diff

Attempting to make hashfloat4() look as much as possible like  
hashfloat8(), I'd figured I could replace NaNs with get_float4_nan()  
before widening to float8.  However, results from protosciurus  
and topminnow show that on some platforms that produces a different  
bit-pattern from get_float8_nan(), breaking the intent of ce773f230.  
Rearrange so that we use the result of get_float8_nan() for all NaN  
cases.  As before, back-patch.  

M src/backend/access/hash/hashfunc.c

Revert "Avoid creating archive status ".ready" files too early"

commit   : c801c3a1ff73c34fcc7ab22b2dd6a85041e6cef3    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Sat, 4 Sep 2021 12:14:30 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Sat, 4 Sep 2021 12:14:30 -0400    

Click here for diff

This reverts commit 515e3d84a0b5 and equivalent commits in back  
branches.  This solution to the problem has a number of problems, so  
we'll try again with a different approach.  
  
Per note from Andres Freund  
  
Discussion: https://postgr.es/m/20210831042949.52eqp5xwbxgrfank@alap3.anarazel.de  

M src/backend/access/transam/xlog.c
M src/backend/postmaster/walwriter.c
M src/include/access/xlog.h
M src/include/access/xlogdefs.h

Remove arbitrary MAXPGPATH limit on command lengths in pg_ctl.

commit   : beb404d3b165ab1cc0dae56b71a6929b4742a10b    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 3 Sep 2021 21:04:44 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 3 Sep 2021 21:04:44 -0400    

Click here for diff

Replace fixed-length command buffers with psprintf() calls.  We didn't  
have anything as convenient as psprintf() when this code was written,  
but now that we do, there's little reason for the limitation to  
stand.  Removing it eliminates some corner cases where (for example)  
starting the postmaster with a whole lot of options fails.  
  
Most individual file names that pg_ctl deals with are still restricted  
to MAXPGPATH, but we've seldom had complaints about that limitation  
so long as it only applies to one filename.  
  
Back-patch to all supported branches.  
  
Phil Krylov  
  
Discussion: https://postgr.es/m/567e199c6b97ee19deee600311515b86@krylov.eu  

M src/bin/pg_ctl/pg_ctl.c

Disallow creating an ICU collation if the DB encoding won't support it.

commit   : 9ebe2852e5435d88d5e3d655b05887b66059cc4d    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 3 Sep 2021 16:38:55 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 3 Sep 2021 16:38:55 -0400    

Click here for diff

Previously this was allowed, but the collation effectively vanished  
into the ether because of the way lookup_collation() works: you could  
not use the collation, nor even drop it.  Seems better to give an  
error up front than to leave the user wondering why it doesn't work.  
  
(Because this test is in DefineCollation not CreateCollation, it does  
not prevent pg_import_system_collations from creating ICU collations,  
regardless of the initially-chosen encoding.)  
  
Per bug #17170 from Andrew Bille.  Back-patch to v10 where ICU support  
was added.  
  
Discussion: https://postgr.es/m/17170-95845cf3f0a9c36d@postgresql.org  

M src/backend/commands/collationcmds.c

Fix portability issue in tests from commit ce773f230.

commit   : 2836d57e44c67cc30453c83a33fbcf745f5d87ef    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 3 Sep 2021 10:01:02 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 3 Sep 2021 10:01:02 -0400    

Click here for diff

Modern POSIX seems to require strtod() to accept "-NaN", but there's  
nothing about NaN in SUSv2, and some of our oldest buildfarm members  
don't like it.  Let's try writing it as -'NaN' instead; that seems  
to produce the same result, at least on Intel hardware.  
  
Per buildfarm.  

M src/test/regress/expected/hash_func.out
M src/test/regress/sql/hash_func.sql

Fix float4/float8 hash functions to produce uniform results for NaNs.

commit   : ad66373ead6db1b54cfb0af5cda6c24e4cfcaa6b    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 2 Sep 2021 17:24:42 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 2 Sep 2021 17:24:42 -0400    

Click here for diff

The IEEE 754 standard allows a wide variety of bit patterns for NaNs,  
of which at least two ("NaN" and "-NaN") are pretty easy to produce  
from SQL on most machines.  This is problematic because our btree  
comparison functions deem all NaNs to be equal, but our float hash  
functions know nothing about NaNs and will happily produce varying  
hash codes for them.  That causes unexpected results from queries  
that hash a column containing different NaN values.  It could also  
produce unexpected lookup failures when using a hash index on a  
float column, i.e. "WHERE x = 'NaN'" will not find all the rows  
it should.  
  
To fix, special-case NaN in the float hash functions, not too much  
unlike the existing special case that forces zero and minus zero  
to hash the same.  I arranged for the most vanilla sort of NaN  
(that coming from the C99 NAN constant) to still have the same  
hash code as before, to reduce the risk to existing hash indexes.  
  
I dithered about whether to back-patch this into stable branches,  
but ultimately decided to do so.  It's a clear improvement for  
queries that hash internally.  If there is anybody who has -NaN  
in a hash index, they'd be well advised to re-index after applying  
this patch ... but the misbehavior if they don't will not be much  
worse than the misbehavior they had before.  
  
Per bug #17172 from Ma Liangzhu.  
  
Discussion: https://postgr.es/m/17172-7505bea9e04e230f@postgresql.org  

M src/backend/access/hash/hashfunc.c
M src/test/regress/expected/hash_func.out
M src/test/regress/sql/hash_func.sql

doc: Replace some uses of "which" by "that" in parallel.sgml

commit   : 6eadfd485f56cb28606adf4f112c83cd98f7a2b3    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Thu, 2 Sep 2021 11:36:11 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Thu, 2 Sep 2021 11:36:11 +0900    

Click here for diff

This makes the documentation more accurate grammatically.  
  
Author: Elena Indrupskaya  
Discussion: https://postgr.es/m/1c994b3d-951e-59bb-1ac2-7b9221c0e4cf@postgrespro.ru  
Backpatch-through: 9.6  

M doc/src/sgml/parallel.sgml

Fix the random test failure in 001_rep_changes.

commit   : 56683e1684efc07faefb6dc932305d8077588791    
  
author   : Amit Kapila <akapila@postgresql.org>    
date     : Wed, 1 Sep 2021 09:36:50 +0530    
  
committer: Amit Kapila <akapila@postgresql.org>    
date     : Wed, 1 Sep 2021 09:36:50 +0530    

Click here for diff

The check to test whether the subscription workers were restarting after a  
change in the subscription was failing. The reason was that the test was  
assuming the walsender started before it reaches the 'streaming' state and  
the walsender was exiting due to an error before that. Now, the walsender  
was erroring out before reaching the 'streaming' state because it tries to  
acquire the slot before the previous walsender has exited.  
  
In passing, improve the die messages so that it is easier to investigate  
the failures in the future if any.  
  
Reported-by: Michael Paquier, as per buildfarm  
Author: Ajin Cherian  
Reviewed-by: Masahiko Sawada, Amit Kapila  
Backpatch-through: 10, where this test was introduced  
Discussion: https://postgr.es/m/YRnhFxa9bo73wfpV@paquier.xyz  

M src/test/subscription/t/001_rep_changes.pl

In pg_dump, avoid doing per-table queries for RLS policies.

commit   : a60860ff33fbd41685943e1c59a199fe95ee96bd    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 31 Aug 2021 15:04:05 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 31 Aug 2021 15:04:05 -0400    

Click here for diff

For no particularly good reason, getPolicies() queried pg_policy  
separately for each table.  We can collect all the policies in  
a single query instead, and attach them to the correct TableInfo  
objects using findTableByOid() lookups.  On the regression  
database, this reduces the number of queries substantially, and  
provides a visible savings even when running against a local  
server.  
  
Per complaint from Hubert Depesz Lubaczewski.  Since this is such  
a simple fix and can have a visible performance benefit, back-patch  
to all supported branches.  
  
Discussion: https://postgr.es/m/20210826084430.GA26282@depesz.com  

M src/bin/pg_dump/pg_dump.c

Cache the results of format_type() queries in pg_dump.

commit   : c4b298ee15aa67a953e65c8efa1a33eeea721446    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 31 Aug 2021 13:53:33 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 31 Aug 2021 13:53:33 -0400    

Click here for diff

There's long been a "TODO: there might be some value in caching  
the results" annotation on pg_dump's getFormattedTypeName function;  
but we hadn't gotten around to checking what it was costing us to  
repetitively look up type names.  It turns out that when dumping the  
current regression database, about 10% of the total number of queries  
issued are duplicative format_type() queries.  However, Hubert Depesz  
Lubaczewski reported a not-unusual case where these account for over  
half of the queries issued by pg_dump.  Individually these queries  
aren't expensive, but when network lag is a factor, they add up to a  
problem.  We can very easily add some caching to getFormattedTypeName  
to solve it.  
  
Since this is such a simple fix and can have a visible performance  
benefit, back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/20210826084430.GA26282@depesz.com  

M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h

Rename the role in stats_ext to have regress_ prefix

commit   : bce24d1edbcf9b3034a6976b4fa2c46c5c85e260    
  
author   : Tomas Vondra <tomas.vondra@postgresql.org>    
date     : Tue, 31 Aug 2021 19:21:29 +0200    
  
committer: Tomas Vondra <tomas.vondra@postgresql.org>    
date     : Tue, 31 Aug 2021 19:21:29 +0200    

Click here for diff

Commit 5be8ce82e8 added a new role to the stats_ext regression suite,  
but the role name did not start with regress_ causing failures when  
running with ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS. Fixed by  
renaming the role to start with the expected regress_ prefix.  
  
Backpatch-through: 10, same as the new regression test  
Discussion: https://postgr.es/m/1F238937-7CC2-4703-A1B1-6DC225B8978A%40enterprisedb.com  

M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql

Fix lookup error in extended stats ownership check

commit   : b46ff4b50f449e01fd1646c4a65c182e7519a48b    
  
author   : Tomas Vondra <tomas.vondra@postgresql.org>    
date     : Tue, 31 Aug 2021 18:03:05 +0200    
  
committer: Tomas Vondra <tomas.vondra@postgresql.org>    
date     : Tue, 31 Aug 2021 18:03:05 +0200    

Click here for diff

When an ownership check on extended statistics object failed, the code  
was calling aclcheck_error_type to report the failure, which is clearly  
wrong, resulting in cache lookup errors. Fix by calling aclcheck_error.  
  
This issue exists since the introduction of extended statistics, so  
backpatch all the way back to PostgreSQL 10. It went unnoticed because  
there were no tests triggering the error, so add one.  
  
Reported-by: Mark Dilger  
Backpatch-through: 10, where extended stats were introduced  
Discussion: https://postgr.es/m/1F238937-7CC2-4703-A1B1-6DC225B8978A%40enterprisedb.com  

M src/backend/catalog/objectaddress.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql

Report tuple address in data-corruption error message

commit   : 0105b7aaa3f4c020934790311b2587fce54a9afd    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Mon, 30 Aug 2021 16:29:12 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Mon, 30 Aug 2021 16:29:12 -0400    

Click here for diff

Most data-corruption reports mention the location of the problem, but  
this one failed to.  Add it.  
  
Backpatch all the way back.  In 12 and older, also assign the  
ERRCODE_DATA_CORRUPTED error code as was done in commit fd6ec93bf890 for  
13 and later.  
  
Discussion: https://postgr.es/m/202108191637.oqyzrdtnheir@alvherre.pgsql  

M src/backend/executor/execMain.c

Fix incorrect error code in StartupReplicationOrigin().

commit   : a8e7eb159c38d33a90574d0409c556c554c5967c    
  
author   : Amit Kapila <akapila@postgresql.org>    
date     : Mon, 30 Aug 2021 09:48:56 +0530    
  
committer: Amit Kapila <akapila@postgresql.org>    
date     : Mon, 30 Aug 2021 09:48:56 +0530    

Click here for diff

ERRCODE_CONFIGURATION_LIMIT_EXCEEDED was used for checksum failure, use  
ERRCODE_DATA_CORRUPTED instead.  
  
Reported-by: Tatsuhito Kasahara  
Author: Tatsuhito Kasahara  
Backpatch-through: 9.6, where it was introduced  
Discussion: https://postgr.es/m/CAP0=ZVLHtYffs8SOWcFJWrBGoRzT9QQbk+_aP+E5AHLNXiOorA@mail.gmail.com  

M src/backend/replication/logical/origin.c

Fix data loss in wal_level=minimal crash recovery of CREATE TABLESPACE.

commit   : 6ebd2426bda2260fc9bc33555da81dfc8c623f32    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Fri, 27 Aug 2021 23:33:23 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Fri, 27 Aug 2021 23:33:23 -0700    

Click here for diff

If the system crashed between CREATE TABLESPACE and the next checkpoint,  
the result could be some files in the tablespace unexpectedly containing  
no rows.  Affected files would be those for which the system did not  
write WAL; see the wal_skip_threshold documentation.  Before v13, a  
different set of conditions governed the writing of WAL; see v12's  
<sect2 id="populate-pitr">.  (The v12 conditions were broader in some  
ways and narrower in others.)  Users may want to audit non-default  
tablespaces for unexpected short files.  The bug could have truncated an  
index without affecting the associated table, and reindexing the index  
would fix that particular problem.  
  
This fixes the bug by making create_tablespace_directories() more like  
TablespaceCreateDbspace().  create_tablespace_directories() was  
recursively removing tablespace contents, reasoning that WAL redo would  
recreate everything removed that way.  That assumption holds for other  
wal_level values.  Under wal_level=minimal, the old approach could  
delete files for which no other copy existed.  Back-patch to 9.6 (all  
supported versions).  
  
Reviewed by Robert Haas and Prabhat Sahu.  Reported by Robert Haas.  
  
Discussion: https://postgr.es/m/CA+TgmoaLO9ncuwvr2nN-J4VEP5XyAcy=zKiHxQzBbFRxxGxm0w@mail.gmail.com  

M src/backend/commands/tablespace.c

Count SP-GiST index scans in pg_stat statistics.

commit   : 7c1b0f4c35354edf5460cf6e87c63a4a65be1e00    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 27 Aug 2021 19:42:42 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 27 Aug 2021 19:42:42 -0400    

Click here for diff

Somehow, spgist overlooked the need to call pgstat_count_index_scan().  
Hence, pg_stat_all_indexes.idx_scan and equivalent columns never  
became nonzero for an SP-GiST index, although the related per-tuple  
counters worked fine.  
  
This fix works a bit differently from other index AMs, in that the  
counter increment occurs in spgrescan not spggettuple/spggetbitmap.  
It looks like this won't make the user-visible semantics noticeably  
different, so I won't go to the trouble of introducing an is-this-  
the-first-call flag just to make the counter bumps happen in the  
same places.  
  
Per bug #17163 from Christian Quest.  Back-patch to all supported  
versions.  
  
Discussion: https://postgr.es/m/17163-b8c5cc88322a5e92@postgresql.org  

M src/backend/access/spgist/spgscan.c

docs: clarify bgw_restart_time documentation

commit   : 05fec0b74facdc7a0f96fd1bc18e5d402f497f7c    
  
author   : Daniel Gustafsson <dgustafsson@postgresql.org>    
date     : Fri, 27 Aug 2021 22:50:19 +0200    
  
committer: Daniel Gustafsson <dgustafsson@postgresql.org>    
date     : Fri, 27 Aug 2021 22:50:19 +0200    

Click here for diff

Author: Dave Cramer <davecramer@gmail.com>  
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>  
Discussion: https://postgr.es/m/CADK3HHLZmqAQZ2ByPDQQ9yhGqax36kksq6sDkV0yYzsxw6ipvQ@mail.gmail.com  

M doc/src/sgml/bgworker.sgml

Fix broken snapshot handling in parallel workers.

commit   : 198cf81e2c64bdf2d4fa3c24d54c1eacf47a3d38    
  
author   : Robert Haas <rhaas@postgresql.org>    
date     : Wed, 25 Aug 2021 08:32:04 -0400    
  
committer: Robert Haas <rhaas@postgresql.org>    
date     : Wed, 25 Aug 2021 08:32:04 -0400    

Click here for diff

Pengchengliu reported an assertion failure in a parallel woker while  
performing a parallel scan using an overflowed snapshot. The proximate  
cause is that TransactionXmin was set to an incorrect value.  The  
underlying cause is incorrect snapshot handling in parallel.c.  
  
In particular, InitializeParallelDSM() was unconditionally calling  
GetTransactionSnapshot(), because I (rhaas) mistakenly thought that  
was always retrieving an existing snapshot whereas, at isolation  
levels less than REPEATABLE READ, it's actually taking a new one. So  
instead do this only at higher isolation levels where there actually  
is a single snapshot for the whole transaction.  
  
By itself, this is not a sufficient fix, because we still need to  
guarantee that TransactionXmin gets set properly in the workers. The  
easiest way to do that seems to be to install the leader's active  
snapshot as the transaction snapshot if the leader did not serialize a  
transaction snapshot. This doesn't affect the results of future  
GetTrasnactionSnapshot() calls since those have to take a new snapshot  
anyway; what we care about is the side effect of setting TransactionXmin.  
  
Report by Pengchengliu. Patch by Greg Nancarrow, except for some comment  
text which I supplied.  
  
Discussion: https://postgr.es/m/002f01d748ac$eaa781a0$bff684e0$@tju.edu.cn  

M src/backend/access/transam/parallel.c

Fix toast rewrites in logical decoding.

commit   : bfdbda24b9aa5fc335f370414e904daccc35a370    
  
author   : Amit Kapila <akapila@postgresql.org>    
date     : Wed, 25 Aug 2021 09:43:33 +0530    
  
committer: Amit Kapila <akapila@postgresql.org>    
date     : Wed, 25 Aug 2021 09:43:33 +0530    

Click here for diff

Commit 325f2ec555 introduced pg_class.relwrite to skip operations on  
tables created as part of a heap rewrite during DDL. It links such  
transient heaps to the original relation OID via this new field in  
pg_class but forgot to do anything about toast tables. So, logical  
decoding was not able to skip operations on internally created toast  
tables. This leads to an error when we tried to decode the WAL for the  
next operation for which it appeared that there is a toast data where  
actually it didn't have any toast data.  
  
To fix this, we set pg_class.relwrite for internally created toast tables  
as well which allowed skipping operations on them during logical decoding.  
  
Author: Bertrand Drouvot  
Reviewed-by: David Zhang, Amit Kapila  
Backpatch-through: 11, where it was introduced  
Discussion: https://postgr.es/m/b5146fb1-ad9e-7d6e-f980-98ed68744a7c@amazon.com  

M contrib/test_decoding/expected/toast.out
M contrib/test_decoding/sql/toast.sql
M src/backend/catalog/toasting.c
M src/backend/commands/cluster.c
M src/backend/commands/tablecmds.c
M src/include/catalog/toasting.h
M src/include/commands/tablecmds.h

Avoid using ambiguous word "positive" in error message.

commit   : f23b2a488b062563fe09fe3552979f18055e8f53    
  
author   : Fujii Masao <fujii@postgresql.org>    
date     : Wed, 25 Aug 2021 11:46:25 +0900    
  
committer: Fujii Masao <fujii@postgresql.org>    
date     : Wed, 25 Aug 2021 11:46:25 +0900    

Click here for diff

There are two identical error messages about valid value of modulus for  
hash partition, in PostgreSQL source code. Commit 0e1275fb07 improved  
only one of them so that ambiguous word "positive" was avoided there,  
and forgot to improve the other. This commit improves the other.  
Which would reduce translator burden.  
  
Back-pach to v11 where the error message exists.  
  
Author: Kyotaro Horiguchi  
Reviewed-by: Fujii Masao  
Discussion: https://postgr.es/m/20210819.170315.1413060634876301811.horikyota.ntt@gmail.com  

M src/backend/parser/parse_utilcmd.c
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/create_table.out

Improve error message about valid value for distance in phrase operator.

commit   : 3095dc1c47ca4ab198ce0817da39637f616df679    
  
author   : Fujii Masao <fujii@postgresql.org>    
date     : Wed, 25 Aug 2021 11:43:56 +0900    
  
committer: Fujii Masao <fujii@postgresql.org>    
date     : Wed, 25 Aug 2021 11:43:56 +0900    

Click here for diff

The distance in phrase operator must be an integer value between zero  
and MAXENTRYPOS inclusive. But previously the error message about  
its valid value included the information about its upper limit  
but not lower limit (i.e., zero). This commit improves the error message  
so that it also includes the information about its lower limit.  
  
Back-patch to v9.6 where full-text phrase search was supported.  
  
Author: Kyotaro Horiguchi  
Reviewed-by: Fujii Masao  
Discussion: https://postgr.es/m/20210819.170315.1413060634876301811.horikyota.ntt@gmail.com  

M src/backend/utils/adt/tsquery.c

Fix regexp misbehavior with capturing parens inside "{0}".

commit   : 3ebd32e70c09d7c35072a1ebff0783e5596eec91    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 24 Aug 2021 16:37:27 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 24 Aug 2021 16:37:27 -0400    

Click here for diff

Regexps like "(.){0}...\1" drew an "invalid backreference number".  
That's not unreasonable on its face, since the capture group will  
never be matched if it's iterated zero times.  However, other engines  
such as Perl's don't complain about this, nor do we throw an error for  
related cases such as "(.)|\1", even though that backref can never  
succeed either.  Also, if the zero-iterations case happens at runtime  
rather than compile time --- say, "(x)*...\1" when there's no "x" to  
be found --- that's not an error, we just deem the backref to not  
match.  Making this even less defensible, no error was thrown for  
nested cases such as "((.)){0}...\2"; and to add insult to injury,  
those cases could result in assertion failures instead.  (It seems  
that nothing especially bad happened in non-assert builds, though.)  
  
Let's just fix it so that no error is thrown and instead the backref  
is deemed to never match, so that compile-time detection of no  
iterations behaves the same as run-time detection.  
  
Per report from Mark Dilger.  This appears to be an aboriginal error  
in Spencer's library, so back-patch to all supported versions.  
  
Pre-v14, it turns out to also be necessary to back-patch one aspect of  
commits cb76fbd7e/00116dee5, namely to create capture-node subREs with  
the begin/end states of their subexpressions, not the current lp/rp  
of the outer parseqatom invocation.  Otherwise delsub complains that  
we're trying to disconnect a state from itself.  This is a bit scary  
but code examination shows that it's safe: in the pre-v14 code, if we  
want to wrap iteration around the subexpression, the first thing we do  
is overwrite the atom's begin/end fields with new states.  So the  
bogus values didn't survive long enough to be used for anything, except  
if no iteration is required, in which case it doesn't matter.  
  
Discussion: https://postgr.es/m/A099E4A8-4377-4C64-A98C-3DEDDC075502@enterprisedb.com  

M src/backend/regex/regcomp.c
M src/test/regress/expected/regex.out
M src/test/regress/sql/regex.sql

Prevent regexp back-refs from sometimes matching when they shouldn't.

commit   : 08e080756d11c4466b878832f27d741f0507be5a    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 23 Aug 2021 17:41:07 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 23 Aug 2021 17:41:07 -0400    

Click here for diff

The recursion in cdissect() was careless about clearing match data  
for capturing parentheses after rejecting a partial match.  This  
could allow a later back-reference to succeed when by rights it  
should fail for lack of a defined referent.  
  
To fix, think a little more rigorously about what the contract  
between different levels of cdissect's recursion needs to be.  
With the right spec, we can fix this using fewer rather than more  
resets of the match data; the key decision being that a failed  
sub-match is now explicitly responsible for clearing any matches  
it may have set.  
  
There are enough other cross-checks and optimizations in the code  
that it's not especially easy to exhibit this problem; usually, the  
match will fail as-expected.  Plus, regexps that are even potentially  
vulnerable are most likely user errors, since there's just not much  
point in writing a back-ref that doesn't always have a referent.  
These facts perhaps explain why the issue hasn't been detected,  
even though it's almost certainly a couple of decades old.  
  
Discussion: https://postgr.es/m/151435.1629733387@sss.pgh.pa.us  

M src/backend/regex/regexec.c
M src/test/regress/expected/regex.out
M src/test/regress/sql/regex.sql

Avoid creating archive status ".ready" files too early

commit   : cf69f7109cd8bd5908c2c23190e39ec3d69be309    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Mon, 23 Aug 2021 15:50:35 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Mon, 23 Aug 2021 15:50:35 -0400    

Click here for diff

WAL records may span multiple segments, but XLogWrite() does not  
wait for the entire record to be written out to disk before  
creating archive status files.  Instead, as soon as the last WAL page of  
the segment is written, the archive status file is created, and the  
archiver may process it.  If PostgreSQL crashes before it is able to  
write and flush the rest of the record (in the next WAL segment), the  
wrong version of the first segment file lingers in the archive, which  
causes operations such as point-in-time restores to fail.  
  
To fix this, keep track of records that span across segments and ensure  
that segments are only marked ready-for-archival once such records have  
been completely written to disk.  
  
This has always been wrong, so backpatch all the way back.  
  
Author: Nathan Bossart <bossartn@amazon.com>  
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>  
Reviewed-by: Ryo Matsumura <matsumura.ryo@fujitsu.com>  
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>  
Discussion: https://postgr.es/m/CBDDFA01-6E40-46BB-9F98-9340F4379505@amazon.com  

M src/backend/access/transam/xlog.c
M src/backend/postmaster/walwriter.c
M src/include/access/xlog.h
M src/include/access/xlogdefs.h

Fix performance bug in regexp's citerdissect/creviterdissect.

commit   : 9610852ab3a4b4309d8b0d0d3616b83033871a41    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 20 Aug 2021 14:19:04 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 20 Aug 2021 14:19:04 -0400    

Click here for diff

After detecting a sub-match "dissect" failure (i.e., a backref match  
failure) in the i'th sub-match of an iteration node, we should proceed  
by adjusting the attempted length of the i'th submatch.  As coded,  
though, these functions changed the attempted length of the *last*  
sub-match, and only after exhausting all possibilities for that would  
they back up to adjust the next-to-last sub-match, and then the  
second-from-last, etc; all of which is wasted effort, since only  
changing the start or length of the i'th sub-match can possibly make  
it succeed.  This oversight creates the possibility for exponentially  
bad performance.  Fortunately the problem is masked in most cases by  
optimizations or constraints applied elsewhere; which explains why  
we'd not noticed it before.  But it is possible to reach the problem  
with fairly simple, if contrived, regexps.  
  
Oversight in my commit 173e29aa5.  That's pretty ancient now,  
so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/1808998.1629412269@sss.pgh.pa.us  

M src/backend/regex/regexec.c

Avoid trying to lock OLD/NEW in a rule with FOR UPDATE.

commit   : fbc1eed8a8dd3178de70de53a0e95786c80f9dbc    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 19 Aug 2021 12:12:35 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 19 Aug 2021 12:12:35 -0400    

Click here for diff

transformLockingClause neglected to exclude the pseudo-RTEs for  
OLD/NEW when processing a rule's query.  This led to odd errors  
or even crashes later on.  This bug is very ancient, but it's  
not terribly surprising that nobody noticed, since the use-case  
for SELECT FOR UPDATE in a non-view rule is somewhere between  
thin and non-existent.  Still, crashing is not OK.  
  
Per bug #17151 from Zhiyong Wu.  Thanks to Masahiko Sawada  
for analysis of the problem.  
  
Discussion: https://postgr.es/m/17151-c03a3e6e4ec9aadb@postgresql.org  

M src/backend/parser/analyze.c
M src/include/nodes/parsenodes.h
M src/test/regress/expected/rules.out
M src/test/regress/sql/rules.sql

Fix check_agg_arguments' examination of aggregate FILTER clauses.

commit   : a65319f093a61cecda7e27f94521704855861046    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 18 Aug 2021 18:12:51 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 18 Aug 2021 18:12:51 -0400    

Click here for diff

Recursion into the FILTER clause was mis-implemented, such that a  
relevant Var or Aggref at the very top of the FILTER clause would  
be ignored.  (Of course, that'd have to be a plain boolean Var or  
boolean-returning aggregate.)  The consequence would be  
mis-identification of the correct semantic level of the aggregate,  
which could lead to not-per-spec query behavior.  If the FILTER  
expression is an aggregate, this could also lead to failure to issue  
an expected "aggregate function calls cannot be nested" error, which  
would likely result in a core dump later on, since the planner and  
executor aren't expecting such cases to appear.  
  
The root cause is that commit b560ec1b0 blindly copied some code  
that assumed it's recursing into a List, and thus didn't examine the  
top-level node.  To forestall questions about why this call doesn't  
look like the others, as well as possible future copy-and-paste  
mistakes, let's change all three check_agg_arguments_walker calls in  
check_agg_arguments, even though only the one for the filter clause  
is really broken.  
  
Per bug #17152 from Zhiyong Wu.  This has been wrong since we  
implemented FILTER, so back-patch to all supported versions.  
(Testing suggests that pre-v11 branches manage to avoid crashing  
in the bad-Aggref case, thanks to "redundant" checks in ExecInitAgg.  
But I'm not sure how thorough that protection is, and anyway the  
wrong-behavior issue remains, so fix 9.6 and 10 too.)  
  
Discussion: https://postgr.es/m/17152-c7f906cc1a88e61b@postgresql.org  

M src/backend/parser/parse_agg.c
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql

Set type identifier on BIO

commit   : f1d5a94fc738a7de89664e0984766622a89d540d    
  
author   : Daniel Gustafsson <dgustafsson@postgresql.org>    
date     : Tue, 17 Aug 2021 14:27:37 +0200    
  
committer: Daniel Gustafsson <dgustafsson@postgresql.org>    
date     : Tue, 17 Aug 2021 14:27:37 +0200    

Click here for diff

In OpenSSL there are two types of BIO's (I/O abstractions):  
source/sink and filters. A source/sink BIO is a source and/or  
sink of data, ie one acting on a socket or a file. A filter  
BIO takes a stream of input from another BIO and transforms it.  
In order for BIO_find_type() to be able to traverse the chain  
of BIO's and correctly find all BIO's of a certain type they  
shall have the type bit set accordingly, source/sink BIO's  
(what PostgreSQL implements) use BIO_TYPE_SOURCE_SINK and  
filter BIO's use BIO_TYPE_FILTER. In addition to these, file  
descriptor based BIO's should have the descriptor bit set,  
BIO_TYPE_DESCRIPTOR.  
  
The PostgreSQL implementation didn't set the type bits, which  
went unnoticed for a long time as it's only really relevant  
for code auditing the OpenSSL installation, or doing similar  
tasks. It is required by the API though, so this fixes it.  
  
Backpatch through 9.6 as this has been wrong for a long time.  
  
Author: Itamar Gafni  
Discussion: https://postgr.es/m/SN6PR06MB39665EC10C34BB20956AE4578AF39@SN6PR06MB3966.namprd06.prod.outlook.com  
Backpatch-through: 9.6  

M src/backend/libpq/be-secure-openssl.c
M src/interfaces/libpq/fe-secure-openssl.c

doc: \123 and \x12 escapes in COPY are in database encoding.

commit   : b889d6c7196d120c9e9a357dea5cd77d68018600    
  
author   : Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Tue, 17 Aug 2021 10:00:06 +0300    
  
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Tue, 17 Aug 2021 10:00:06 +0300    

Click here for diff

The backslash sequences, including \123 and \x12 escapes, are interpreted  
after encoding conversion. The docs failed to mention that.  
  
Backpatch to all supported versions.  
  
Reported-by: Andreas Grob  
Discussion: https://www.postgresql.org/message-id/17142-9181542ca1df75ab%40postgresql.org  

M doc/src/sgml/ref/copy.sgml

Refresh apply delay on reload of recovery_min_apply_delay at recovery

commit   : d392e86fa9c4ef5170eebf700d81f77ede4363ad    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Mon, 16 Aug 2021 12:11:59 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Mon, 16 Aug 2021 12:11:59 +0900    

Click here for diff

This commit ensures that the wait interval in the replay delay loop  
waiting for an amount of time defined by recovery_min_apply_delay is  
correctly handled on reload, recalculating the delay if this GUC value  
is updated, based on the timestamp of the commit record being replayed.  
  
The previous behavior would be problematic for example with replay  
still waiting even if the delay got reduced or just cancelled.  If the  
apply delay was increased to a larger value, the wait would have just  
respected the old value set, finishing earlier.  
  
Author: Soumyadeep Chakraborty, Ashwin Agrawal  
Reviewed-by: Kyotaro Horiguchi, Michael Paquier  
Discussion: https://postgr.es/m/CAE-ML+93zfr-HLN8OuxF0BjpWJ17O5dv1eMvSE5jsj9jpnAXZA@mail.gmail.com  
Backpatch-through: 9.6  

M src/backend/access/transam/xlog.c

Add RISC-V spinlock support in s_lock.h.

commit   : 8024ff478ba7ef65dfe25594f36a02265faed133    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 13 Aug 2021 13:58:47 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 13 Aug 2021 13:58:47 -0400    

Click here for diff

Like the ARM case, just use gcc's __sync_lock_test_and_set();  
that will compile into AMOSWAP.W.AQ which does what we need.  
  
At some point it might be worth doing some work on atomic ops  
for RISC-V, but this should be enough for a creditable port.  
  
Back-patch to all supported branches, just in case somebody  
wants to try them on RISC-V.  
  
Marek Szuba  
  
Discussion: https://postgr.es/m/dea97b6d-f55f-1f6d-9109-504aa7dfa421@gentoo.org  

M src/include/storage/s_lock.h

Fix incorrect hash table resizing code in simplehash.h

commit   : 5a6b0f21e67921118dfd290b1d005789ac006fb7    
  
author   : David Rowley <drowley@postgresql.org>    
date     : Fri, 13 Aug 2021 16:43:46 +1200    
  
committer: David Rowley <drowley@postgresql.org>    
date     : Fri, 13 Aug 2021 16:43:46 +1200    

Click here for diff

This fixes a bug in simplehash.h which caused an incorrect size mask to be  
used when the hash table grew to SH_MAX_SIZE (2^32).  The code was  
incorrectly setting the size mask to 0 when the hash tables reached the  
maximum possible number of buckets.  This would result always trying to  
use the 0th bucket causing an  infinite loop of trying to grow the hash  
table due to there being too many collisions.  
  
Seemingly it's not that common for simplehash tables to ever grow this big  
as this bug dates back to v10 and nobody seems to have noticed it before.  
However, probably the most likely place that people would notice it would  
be doing a large in-memory Hash Aggregate with something close to at least  
2^31 groups.  
  
After this fix, the code now works correctly with up to within 98% of 2^32  
groups and will fail with the following error when trying to insert any  
more items into the hash table:  
  
ERROR:  hash table size exceeded  
  
However, the work_mem (or hash_mem_multiplier in newer versions) settings  
will generally cause Hash Aggregates to spill to disk long before reaching  
that many groups.  The minimal test case I did took a work_mem setting of  
over 192GB to hit the bug.  
  
simplehash hash tables are used in a few other places such as Bitmap Index  
Scans, however, again the size that the hash table can become there is  
also limited to work_mem and it would take a relation of around 16TB  
(2^31) pages and a very large work_mem setting to hit this.  With smaller  
work_mem values the table would become lossy and never grow large enough  
to hit the problem.  
  
Author: Yura Sokolov  
Reviewed-by: David Rowley, Ranier Vilela  
Discussion: https://postgr.es/m/b1f7f32737c3438136f64b26f4852b96@postgrespro.ru  
Backpatch-through: 10, where simplehash.h was added  

M src/include/lib/simplehash.h

Make EXEC_BACKEND more convenient on macOS.

commit   : 0bfefb7dafe0019715075b6e8bf58ed5de58bffa    
  
author   : Thomas Munro <tmunro@postgresql.org>    
date     : Fri, 13 Aug 2021 10:38:22 +1200    
  
committer: Thomas Munro <tmunro@postgresql.org>    
date     : Fri, 13 Aug 2021 10:38:22 +1200    

Click here for diff

It's hard to disable ASLR on current macOS releases, for testing with  
-DEXEC_BACKEND.  You could already set the environment variable  
PG_SHMEM_ADDR to something not likely to collide with mappings created  
earlier in process startup.  Let's also provide a default value that  
works on current releases and architectures, for developer convenience.  
  
As noted in the pre-existing comment, this is a horrible hack, but  
-DEXEC_BACKEND is only used by Unix-based PostgreSQL developers for  
testing some otherwise Windows-only code paths, so it seems excusable.  
  
Back-patch to all supported branches.  
  
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>  
Discussion: https://postgr.es/m/20210806032944.m4tz7j2w47mant26%40alap3.anarazel.de  

M src/backend/port/sysv_shmem.c

Fix failure of btree_gin indexscans with "char" type and </<= operators.

commit   : eefa4c2b5ed647fed9795f79ed4287898fae9ce6    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 10 Aug 2021 18:10:30 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 10 Aug 2021 18:10:30 -0400    

Click here for diff

As a result of confusion about whether the "char" type is signed or  
unsigned, scans for index searches like "col < 'x'" or "col <= 'x'"  
would start at the middle of the index not the left end, thus missing  
many or all of the entries they should find.  Fortunately, this  
is not a symptom of index corruption.  It's only the search logic  
that is broken, and we can fix it without unpleasant side-effects.  
  
Per report from Jason Kim.  This has been wrong since btree_gin's  
beginning, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/20210810001649.htnltbh7c63re42p@jasonk.me  

M contrib/btree_gin/btree_gin.c
M contrib/btree_gin/expected/char.out