Fix extreme skew detection in Parallel Hash Join.
commit : 98c7c7152d2d4670a349c0fe48f05cf95c0b1185
author : Thomas Munro <[email protected]>
date : Thu, 17 Oct 2024 15:52:24 +1300
committer: Thomas Munro <[email protected]>
date : Thu, 17 Oct 2024 15:52:24 +1300
Click here for diff
After repartitioning the inner side of a hash join that would have
exceeded the allowed size, we check if all the tuples from a parent
partition moved to one child partition. That is evidence that it
contains duplicate keys and later attempts to repartition will also
fail, so we should give up trying to limit memory (for lack of a better
fallback strategy).
A thinko prevented the check from working correctly in partition 0 (the
one that is partially loaded into memory already). After
repartitioning, we should check for extreme skew if the *parent*
partition's space_exhausted flag was set, not the child partition's.
The consequence was repeated futile repartitioning until per-partition
data exceeded various limits including "ERROR: invalid DSA memory alloc
request size 1811939328", OS allocation failure, or temporary disk space
errors. (We could also do something about some of those symptoms, but
that's material for separate patches.)
This problem only became likely when PostgreSQL 16 introduced support
for Parallel Hash Right/Full Join, allowing NULL keys into the hash
table. Repartitioning always leaves NULL in partition 0, no matter how
many times you do it, because the hash value is all zero bits. That's
unlikely for other hashed values, but they might still have caused
wasted extra effort before giving up.
Back-patch to all supported releases.
Reported-by: Craig Milhiser <[email protected]>
Reviewed-by: Andrei Lepikhov <[email protected]>
Discussion: https://postgr.es/m/CA%2BwnhO1OfgXbmXgC4fv_uu%3DOxcDQuHvfoQ4k0DFeB0Qqd-X-rQ%40mail.gmail.com
M src/backend/executor/nodeHash.c
Stamp 17.0.
commit : d7ec59a63d745ba74fba0e280bbf85dc6d1caa3e
author : Tom Lane <[email protected]>
date : Mon, 23 Sep 2024 16:02:53 -0400
committer: Tom Lane <[email protected]>
date : Mon, 23 Sep 2024 16:02:53 -0400
Click here for diff
M configure
M configure.ac
M meson.build
Translation updates
commit : 29d483fb33229f4322ca6cd040422ac508c678c1
author : Peter Eisentraut <[email protected]>
date : Mon, 23 Sep 2024 12:06:47 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 23 Sep 2024 12:06:47 +0200
Click here for diff
v2.53.1: PostgreSQL 17beta3 Support
commit : b84a3dd08c7eeea81ce4c77d0bb0c7ad16f61f4c
author : David Steele <[email protected]>
date : Mon, 19 Aug 2024 11:16:14 +0700
committer: David Steele <[email protected]>
date : Mon, 19 Aug 2024 11:16:14 +0700
Click here for diff
Bug Fixes:
* Fix permissions when restore run as root user. (Reviewed by Stefan Fercot. Reported by Will M.)
* Fix segfault on delayed connection errors. (Reviewed by David Christensen. Reported by Anton Glushakov.)
* Skip local repository duplicate check for SFTP. (Fixed by Reid Thompson. Reviewed by David Steele. Reported by Anton Kurochkin.)
Improvements:
* PostgreSQL 17beta3 support.
M CONTRIBUTING.md
M README.md
M doc/resource/exe.cache
M doc/resource/git-history.cache
M doc/xml/auto/metric-coverage-report.auto.xml
M doc/xml/release/2024/2.53.1.xml
M meson.build
M src/build/configure.ac
M src/configure
M src/version.h
Stamp 12.20.
commit : 76265a851b13bbb001a218481c0cb6315c0fdfe6
author : Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 16:11:34 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 16:11:34 -0400
Click here for diff
M configure
M configure.in
M src/include/pg_config.h.win32
M src/interfaces/libpq/libpq.rc.in
M src/port/win32ver.rc
Last-minute updates for release notes.
commit : 1b85e65846aa16ede7aeddd67a9bd16ec73a18ae
author : Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 14:03:20 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 14:03:20 -0400
Click here for diff
Stamp 13.16.
commit : 0dc08bbfcc93ac4f04d2a3f4b9d1231a80e2cc0c
author : Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 16:09:59 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 16:09:59 -0400
Click here for diff
M configure
M configure.in
Last-minute updates for release notes.
commit : 429560193282f2c29a1bb9f28a055c87b7605253
author : Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 14:03:20 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 14:03:20 -0400
Click here for diff
Stamp 14.13.
commit : 33965476b6ef6a37af2bc9bb1b5b2af17c35dc5e
author : Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 16:08:36 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 16:08:36 -0400
Click here for diff
M configure
M configure.ac
Last-minute updates for release notes.
commit : b942cc42b7c659da70c205151bde5e1f9af2ece0
author : Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 14:03:20 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 14:03:20 -0400
Click here for diff
Stamp 15.8.
commit : 3cc5e51ab4507b33acc82684b7d79ac43b8a6b5d
author : Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 16:07:10 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 16:07:10 -0400
Click here for diff
M configure
M configure.ac
Last-minute updates for release notes.
commit : 6eca8a5673307ce52f892f083df0b958d35f09e1
author : Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 14:03:20 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 14:03:20 -0400
Click here for diff
Stamp 16.4.
commit : 2caa85f4aae689e6f6721d7363b4c66a2a6417d6
author : Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 16:05:35 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 16:05:35 -0400
Click here for diff
M configure
M configure.ac
M meson.build
Last-minute updates for release notes.
commit : c04778592d6db6197819f95028347af135709e4b
author : Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 14:03:20 -0400
committer: Tom Lane <[email protected]>
date : Mon, 5 Aug 2024 14:03:20 -0400
Click here for diff
v2.53: Concurrent Backups
commit : 6e8a45f650e303fe74a1e34ee8e76a193dbc6c4a
author : David Steele <[email protected]>
date : Mon, 22 Jul 2024 09:33:31 +0700
committer: David Steele <[email protected]>
date : Mon, 22 Jul 2024 09:33:31 +0700
Click here for diff
IMPORTANT NOTE: The log-level-stderr option default has been changed from warn to off. This makes it easier to capture errors when only redirecting stdout. To preserve the prior behavior set log-level-stderr=warn.
NOTE TO PACKAGERS: The lz4 library is now required by the meson build.
NOTE TO PACKAGERS: Compiler support for __builtin_clzl() and __builtin_bswap64() is now required by the meson build.
Bug Fixes:
* Fix SFTP renaming failure when file already exists. (Fixed by Reid Thompson. Reviewed by David Steele. Reported by ahmed112212.)
Features:
* Allow backups to run concurrently on different repositories. (Reviewed by Reid Thompson, Stefan Fercot.)
* Support IP-based SANs for TLS certificate validation. (Contributed by David Christensen. Reviewed by David Steele.)
Improvements:
* Default log-level-stderr option to off. (Reviewed by Greg Sabino Mullane, Stefan Fercot.)
* Allow alternative WAL segment sizes for PostgreSQL ≤ 10. (Contributed by Viktor Kurilko. Reviewed by David Steele.)
* Add hint to check SFTP authorization log. (Contributed by Vitalii Zurian. Reviewed by Reid Thompson, David Steele.)
Documentation Improvements:
* Clarify archive-push multi-repo behavior. (Reviewed by Stefan Fercot.)
M CONTRIBUTING.md
M README.md
M doc/lib/pgBackRestDoc/Custom/DocCustomRelease.pm
M doc/resource/exe.cache
M doc/resource/git-history.cache
M doc/xml/auto/metric-coverage-report.auto.xml
M doc/xml/release.xml
A doc/xml/release/2024/2.52.1.xml
M doc/xml/release/2024/2.53.xml
M meson.build
M src/build/configure.ac
M src/configure
M src/version.h
M test/code-count/file-type.yaml
v2.52.1: Bug Fix
commit : cc9035c2456e00eef463724dfc6900f68dd9c3fc
author : David Steele <[email protected]>
date : Tue, 25 Jun 2024 11:46:44 +0800
committer: David Steele <[email protected]>
date : Tue, 25 Jun 2024 11:46:44 +0800
Click here for diff
Bug Fixes:
* Fix issue with files larger on the replica than on the primary. (Reviewed by Stefan Fercot. Reported by Nicolas Lassimonne.)
M CONTRIBUTING.md
M README.md
M doc/resource/exe.cache
M doc/resource/git-history.cache
M doc/xml/release/2024/2.52.1.xml
M meson.build
M src/build/configure.ac
M src/configure
M src/version.h
v2.52: PostgreSQL 17 support
commit : dfc14b193454b3003c10eb2d021699cbd6294004
author : David Steele <[email protected]>
date : Mon, 27 May 2024 11:13:16 +1000
committer: David Steele <[email protected]>
date : Mon, 27 May 2024 11:13:16 +1000
Click here for diff
NOTE TO PACKAGERS: The build system for pgBackRest is now meson. The autoconf/make build will not receive any new features and will be removed after a few releases.
Features:
* Add GCS batch delete support. (Reviewed by Reid Thompson.)
* S3 SSE-C encryption support. (Reviewed by Tim Jones. Suggested by Tim Jones.)
* PostgreSQL 17 support. (Reviewed by Stefan Fercot.)
Improvements:
* Allow explicit disabling of optional dependencies in meson builds. (Contributed by Michael Schout. Reviewed by David Steele.)
* Dynamically find python in meson build. (Contributed by Michael Schout. Reviewed by David Steele.)
* Tag pgbackrest build target in meson as installable. (Contributed by Bradford Boyle. Reviewed by David Steele.)
Documentation Improvements:
* Update start/stop documentation to reflect actual functionality. (Reviewed by Stefan Fercot.)
M CONTRIBUTING.md
M README.md
M doc/resource/exe.cache
M doc/resource/git-history.cache
M doc/xml/auto/metric-coverage-report.auto.xml
M doc/xml/release/2024/2.52.xml
M meson.build
M src/build/configure.ac
M src/configure
M src/version.h
M test/code-count/file-type.yaml
Stamp 12.19.
commit : 01aeb431c12a3388594a445ca97e71cbed410ed2
author : Tom Lane <[email protected]>
date : Mon, 6 May 2024 16:27:39 -0400
committer: Tom Lane <[email protected]>
date : Mon, 6 May 2024 16:27:39 -0400
Click here for diff
M configure
M configure.in
M src/include/pg_config.h.win32
M src/interfaces/libpq/libpq.rc.in
M src/port/win32ver.rc
Translation updates
commit : 51da12633a3d7b4b9f715c41deac4d7d4c94248b
author : Peter Eisentraut <[email protected]>
date : Mon, 6 May 2024 12:15:05 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 6 May 2024 12:15:05 +0200
Click here for diff
Stamp 13.15.
commit : 8e5faba4b918ba6142339c8f55eaa4eb99776a03
author : Tom Lane <[email protected]>
date : Mon, 6 May 2024 16:26:10 -0400
committer: Tom Lane <[email protected]>
date : Mon, 6 May 2024 16:26:10 -0400
Click here for diff
M configure
M configure.in
Translation updates
commit : 38993fe3a2bbc16c812b1a9633063129130860c5
author : Peter Eisentraut <[email protected]>
date : Mon, 6 May 2024 12:13:39 +0200
committer: Peter Eisentraut <[email protected]>
date : Mon, 6 May 2024 12:13:39 +0200
Click here for diff
Stamp 14.12.
commit : cf07a47dc0c3b16778291e62d730537b45da51ef
author : Tom Lane <[email protected]>
date : Mon, 6 May 2024 16:24:45 -0400
committer: Tom Lane <[email protected]>
date : Mon, 6 May 2024 16:24:45 -0400
Click here for diff
M configure
M configure.ac
Last-minute updates for release notes.
commit : b7712704f0bb3dc82f286516a7057b0fa5b1d260
author : Tom Lane <[email protected]>
date : Mon, 6 May 2024 12:27:27 -0400
committer: Tom Lane <[email protected]>
date : Mon, 6 May 2024 12:27:27 -0400
Click here for diff
Stamp 15.7.
commit : 4a53584cf2d676e685d899d01cde18c075fbeca7
author : Tom Lane <[email protected]>
date : Mon, 6 May 2024 16:23:18 -0400
committer: Tom Lane <[email protected]>
date : Mon, 6 May 2024 16:23:18 -0400
Click here for diff
M configure
M configure.ac
Last-minute updates for release notes.
commit : 7b2ac0f6039f0f64d80c7488f0f718094cfb89a2
author : Tom Lane <[email protected]>
date : Mon, 6 May 2024 12:27:26 -0400
committer: Tom Lane <[email protected]>
date : Mon, 6 May 2024 12:27:26 -0400
Click here for diff
Stamp 16.3.
commit : 05ffe9398b758bbb8d30cc76e9bbc638dab2d477
author : Tom Lane <[email protected]>
date : Mon, 6 May 2024 16:21:25 -0400
committer: Tom Lane <[email protected]>
date : Mon, 6 May 2024 16:21:25 -0400
Click here for diff
M configure
M configure.ac
M meson.build
Last-minute updates for release notes.
commit : a62be834aeb899dfc2e10e3cde6d0f48ba128373
author : Tom Lane <[email protected]>
date : Mon, 6 May 2024 12:27:26 -0400
committer: Tom Lane <[email protected]>
date : Mon, 6 May 2024 12:27:26 -0400
Click here for diff
v2.51: Meson Build System
commit : dc07fb1e5bd38026d4fefb67a0525a89ac98b520
author : David Steele <[email protected]>
date : Mon, 25 Mar 2024 09:53:22 +1300
committer: David Steele <[email protected]>
date : Mon, 25 Mar 2024 09:53:22 +1300
Click here for diff
NOTE TO PACKAGERS: The build system for pgBackRest is now meson. The autoconf/make build will not receive any new features and will be removed after a few releases.
Bug Fixes:
* Skip zero-length files for block incremental delta restore. (Reviewed by Sebastian Krause, René Højbjerg Larsen. Reported by Sebastian Krause.)
* Fix performance regression in storage list. (Reviewed by Stephen Frost. Reported by Maksym Boguk.)
* Fix progress logging when file size changes during backup. (Reviewed by Stephen Frost. Reported by samkingno.)
Improvements:
* Improved support for dual stack connections. (Reviewed by Stephen Frost. Suggested by Timothée Peignier.)
* Make meson the primary build system. (Reviewed by Stephen Frost.)
* Detect files that have not changed during non-delta incremental backup. (Reviewed by Stephen Frost.)
* Prevent invalid recovery when backup_label removed. (Reviewed by Stephen Frost.)
* Improve archive-push WAL segment queue handling. (Reviewed by Stephen Frost.)
* Limit resume functionality to full backups. (Reviewed by Stephen Frost, Stefan Fercot.)
* Update resume functionality for block incremental. (Reviewed by Stephen Frost.)
* Allow --version and --help for version and help. (Reviewed by Greg Sabino Mullane. Suggested by Greg Sabino Mullane.)
* Add detailed backtrace to autoconf/make build. (Reviewed by Stephen Frost.)
Documentation Improvements:
* Update references to recovery.conf. (Reviewed by Stefan Fercot. Suggested by Stephen Frost.)
M CONTRIBUTING.md
M README.md
M doc/resource/exe.cache
M doc/resource/git-history.cache
M doc/xml/auto/metric-coverage-report.auto.xml
M doc/xml/release/2024/2.51.xml
M meson.build
M src/build/configure.ac
M src/configure
M src/version.h
M test/code-count/file-type.yaml
Stamp 12.18.
commit : 3ba17930941ea629b2ffb5cd252f3055d4d4a9a6
author : Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 16:48:53 -0500
committer: Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 16:48:53 -0500
Click here for diff
M configure
M configure.in
M src/include/pg_config.h.win32
M src/interfaces/libpq/libpq.rc.in
M src/port/win32ver.rc
Last-minute updates for release notes.
commit : 84e6a6e40ffc8e7e42279c6932bd1b9b7f6e24d9
author : Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 11:51:11 -0500
committer: Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 11:51:11 -0500
Click here for diff
Stamp 13.14.
commit : b3b2bf3b9cc1f31170d3b4e21cb6ac0cf24bf90a
author : Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 16:47:02 -0500
committer: Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 16:47:02 -0500
Click here for diff
M configure
M configure.in
Last-minute updates for release notes.
commit : 3344db3cfd9f569462937a41af8f536cdd3ac251
author : Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 11:51:11 -0500
committer: Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 11:51:11 -0500
Click here for diff
Stamp 14.11.
commit : d1a2a93766b0a27eb4ffecdb63ed3879088dbfbe
author : Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 16:45:28 -0500
committer: Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 16:45:28 -0500
Click here for diff
M configure
M configure.ac
Last-minute updates for release notes.
commit : 25ee45b7caa992896477841baf8791d8e7fb4e79
author : Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 11:51:11 -0500
committer: Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 11:51:11 -0500
Click here for diff
Stamp 15.6.
commit : 496a1dc44bf1261053da9b3f7e430769754298b4
author : Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 16:43:21 -0500
committer: Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 16:43:21 -0500
Click here for diff
M configure
M configure.ac
Last-minute updates for release notes.
commit : cb620968b6f7339113b7873b9abf5885cb4aa1ee
author : Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 11:51:11 -0500
committer: Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 11:51:11 -0500
Click here for diff
Stamp 16.2.
commit : b78fa8547d02fc72ace679fb4d5289dccdbfc781
author : Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 16:41:37 -0500
committer: Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 16:41:37 -0500
Click here for diff
M configure
M configure.ac
M meson.build
Last-minute updates for release notes.
commit : da21a3a7d5d37e34f2adef07b1f2f6231d61c3cc
author : Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 11:51:11 -0500
committer: Tom Lane <[email protected]>
date : Mon, 5 Feb 2024 11:51:11 -0500
Click here for diff
commit : 1add35624dbf4061bcb19521022b318418b0bba5
author : David Steele <[email protected]>
date : Mon, 22 Jan 2024 09:54:59 -0300
committer: David Steele <[email protected]>
date : Mon, 22 Jan 2024 09:54:59 -0300
Click here for diff
Bug Fixes:
* Fix short read in block incremental restore. (Reviewed by Stephen Frost, Brent Graveland. Reported by Adol Rodriguez, Brent Graveland.)
* Fix overflow suppressing backup progress in info output. (Fixed by Robert Donovan. Reviewed by Joe Wildish.)
Improvements:
* Preserve partial files during block incremental delta restore. (Reviewed by Stephen Frost.)
* Add support for alternate compile-time page sizes. (Contributed by Viktor Kurilko. Reviewed by David Steele.)
* Skip files truncated during backup when bundling. (Contributed by Georgy Shelkovy. Reviewed by David Steele.)
* Improve SFTP storage error messages. (Contributed by Reid Thompson. Reviewed by David Steele.)
M CONTRIBUTING.md
M README.md
M doc/resource/exe.cache
M doc/resource/git-history.cache
M doc/xml/auto/metric-coverage-report.auto.xml
M doc/xml/release/2024/2.50.xml
M meson.build
M src/build/configure.ac
M src/configure
M src/version.h
M test/code-count/file-type.yaml
v2.49: Remove PostgreSQL 9.3 Support
commit : 3cb891e3ca49ed1166fb0b513b126412448f5b09
author : David Steele <[email protected]>
date : Mon, 27 Nov 2023 08:55:56 -0300
committer: David Steele <[email protected]>
date : Mon, 27 Nov 2023 08:55:56 -0300
Click here for diff
Bug Fixes:
* Fix regression in retries. (Reviewed by Stephen Frost. Reported by Norman Adkins, Tanel Suurhans, Jordan English, Timothée Peignier.)
* Fix recursive path remove in SFTP storage driver. (Fixed by Reid Thompson. Reviewed by Stephen Frost. Reported by Luc.)
Improvements:
* Remove support for PostgreSQL 9.3. (Reviewed by Stephen Frost.)
Documentation Features:
* Document maintainer options. (Reviewed by Stefan Fercot.)
* Update point-in-time recovery documentation for PostgreSQL >= 13.
Test Suite Improvements:
* Allow config/load unit test to run without libssh2 installed. (Contributed by Reid Thompson. Reviewed by David Steele. Suggested by Wu Ning.)
M CONTRIBUTING.md
M README.md
M doc/resource/exe.cache
M doc/resource/git-history.cache
M doc/xml/auto/metric-coverage-report.auto.xml
M doc/xml/release/2023/2.49.xml
M meson.build
M src/build/configure.ac
M src/configure
M src/version.h
M test/code-count/file-type.yaml
Stamp 11.22.
commit : fd851f9e4a13d81cccc4ac5d6059d732c7518111
author : Tom Lane <[email protected]>
date : Mon, 6 Nov 2023 17:13:58 -0500
committer: Tom Lane <[email protected]>
date : Mon, 6 Nov 2023 17:13:58 -0500
Click here for diff
M configure
M configure.in
M doc/bug.template
M src/include/pg_config.h.win32
M src/interfaces/libpq/libpq.rc.in
M src/port/win32ver.rc
Last-minute updates for release notes.
commit : 3dbb3dd2d641af0da92e0a25b490b5bc38490e58
author : Tom Lane <[email protected]>
date : Mon, 6 Nov 2023 13:26:33 -0500
committer: Tom Lane <[email protected]>
date : Mon, 6 Nov 2023 13:26:33 -0500
Click here for diff