pgBackRest 2.62 commit log

commit   : eca9290a2a6583fd0dd9156ee6ef6cf26899336e    
  
author   : David Steele <david@pgbackrest.org>    
date     : Mon, 21 Jul 2025 10:12:22 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Mon, 21 Jul 2025 10:12:22 -0400    

Bug Fixes:  
  
* Fix issue with adhoc expiration when no backups in a repository. (Reviewed by Stefan Fercot. Reported by Anup Gupta.)  
  
Features:  
  
* Add restore progress to info command output. (Contributed by Denis Garsh, Maxim Michkov. Reviewed by David Steele.)  
* Add progress-only detail level for info command output. (Contributed by Denis Garsh. Reviewed by David Steele, Stefan Fercot.)  
  
Improvements:  
  
* Retry failed reads on object stores. (Reviewed by David Christensen.)  
* Fix defaults in command-line help. (Reviewed by David Christensen, Chris Bandy.)  
  
Documentation Improvements:  
  
* Describe discrete option values in a list where appropriate. (Contributed by Anton Kurochkin. Reviewed by David Steele.)  
* Fix "less than" in help output for archive-mode option. (Contributed by Anton Kurochkin. Reviewed by David Steele.)  

commit   : a2a246bba9f71b4b80b3df77688d260862534949    
  
author   : David Steele <david@pgbackrest.org>    
date     : Fri, 18 Jul 2025 11:42:12 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Fri, 18 Jul 2025 11:42:12 -0400    

commit   : 8cdd9ce1c4ab6cca508932a41a3013374d7547ef    
  
author   : Denis Garsh <d.garsh@arenadata.io>    
date     : Wed, 16 Jul 2025 19:49:34 +0300    
  
committer: GitHub <noreply@github.com>    
date     : Wed, 16 Jul 2025 19:49:34 +0300    

The prior implementation of the restore command did not provide any progress information. Implement it similarly to the backup command and also update the info command to display restore progress alongside backup progress.  
  
Restore is a read operation and should not block other commands. The only exception is that multiple restores with the same lock and repository path are not allowed, as each restore must write progress to a separate file. Therefore, no lock is needed for restores with a remote role and the restore command should not be terminated when the stop command is executed.

commit   : d55836e7e80a0524c46db2a763dee5194354cd33    
  
author   : David Steele <david@pgbackrest.org>    
date     : Tue, 15 Jul 2025 09:45:36 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Tue, 15 Jul 2025 09:45:36 -0400    

20.04 is EOL and Docker is now noisy during installation so update.  

commit   : ff944585caeea912c2897302e56ff4c8e6b0df2e    
  
author   : David Steele <david@pgbackrest.org>    
date     : Mon, 7 Jul 2025 16:57:08 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Mon, 7 Jul 2025 16:57:08 -0400    

commit   : 2706168f609af544b0453d384c7fb620626fc205    
  
author   : Denis Garsh <d.garsh@arenadata.io>    
date     : Fri, 27 Jun 2025 00:40:05 +0300    
  
committer: GitHub <noreply@github.com>    
date     : Fri, 27 Jun 2025 00:40:05 +0300    

The info command fetches a lot of information from the repository about backups and archives, so this operation can be slow. Because progress data is stored in local lock files, accessing the repository is unnecessary when only progress information is required.  
  
This patch introduces the `--detail-level=[progress|full]` option, with `full` as the default. The `progress` level limits the info command output to progress details without querying the repository. The only remaining operations are scanning the folder structure to list available stanzas and reading lock files.  
  
Note: When `progress` is selected, the info command performs no checks beyond verifying stanza availability.

commit   : 277b4db46f20a52d9c363fe3478adb6acc31c039    
  
author   : David Steele <david@pgbackrest.org>    
date     : Thu, 26 Jun 2025 08:04:33 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Thu, 26 Jun 2025 08:04:33 -0400    

commit   : 638815b4cde1ba1322516e256742c51676af66f0    
  
author   : David Steele <david@pgbackrest.org>    
date     : Tue, 24 Jun 2025 12:35:18 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Tue, 24 Jun 2025 12:35:18 -0400    

If there are no backups in one or more repositories then the following error occurs during adhoc expiration:  
  
ASSERT: [025]: cannot get index 0 from list with 0 value(s)  
  
Fix this by skipping the adhoc logic when there are no backups in a repository.  

commit   : 378005786670fad5c6f8b027f6447565e10489d6    
  
author   : David Steele <david@pgbackrest.org>    
date     : Sun, 22 Jun 2025 17:46:54 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Sun, 22 Jun 2025 17:46:54 -0400    

commit   : d0072945fa33d4af6b64ffd7921df376885c1717    
  
author   : David Steele <david@pgbackrest.org>    
date     : Sun, 22 Jun 2025 15:38:32 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Sun, 22 Jun 2025 15:38:32 -0400    

The nonstring variable attribute specifies that an object or member declaration with type array of char, signed char, or unsigned char, or pointer to such a type is intended to store character arrays that do not necessarily contain a terminating NUL.  
  
Newer versions of gcc will warn if this attribute is missing.  

commit   : 44e3ee33d2360127ac715ebb16866cf995863949    
  
author   : David Steele <david@pgbackrest.org>    
date     : Mon, 16 Jun 2025 17:21:33 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Mon, 16 Jun 2025 17:21:33 -0400    

Some defaults and allow lists were determined in cfgLoadUpdateOption() because they depended on the values of other options, .e.g. compress-level.  
  
Instead build this functionality into the config parser. Not only does this standardize the defaults and allow lists but it makes it possible to automate the documentation, which is also done in this commit.  

commit   : 435f8a3ad7104fe28bcb79bf9bb00d1f0b7ee32f    
  
author   : David Steele <david@pgbackrest.org>    
date     : Mon, 16 Jun 2025 17:05:35 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Mon, 16 Jun 2025 17:05:35 -0400    

It is currently not possible to determine the default of all options knowing just the command. Some defaults are set in cfgLoadUpdateOption() and in an upcoming commit defaults may be based on the value of other options.  
  
It would be possible to update parser to provide this information but that will complicate the parser and since the logic is only used to simplify options passed to remotes it does not seem worth the effort.  
  
For the same reason cfgParseOptionRequired() can also be removed.  

commit   : 5e14692588b9fd8a566c9a1795c866ba2e83eebd    
  
author   : David Steele <david@pgbackrest.org>    
date     : Mon, 16 Jun 2025 16:25:50 -0400    
  
committer: GitHub <noreply@github.com>    
date     : Mon, 16 Jun 2025 16:25:50 -0400    

There is currently a retry if the initial get request fails (depending on the error code) but if the read fails later on while fetching blocks of data it is fatal. In most cases there is a higher level retry (e.g. restore) but restarting the restore job might be expensive depending on how many files are being restored.  
  
Add a retry that will catch read errors and retry from where the last data was successfully read.  
  
A bit of history -- this patch was first started three years ago but the memory context model at that time would not allow the interface (StorageRead) to own the driver (e.g. StorageReadS3). Subsequent improvements in memory contexts have allowed this ownership model and in fact it is now the default so no ownership changes are required in this patch except in StorageReadRemote which was not updated in f6e3073.

commit   : 1e23e38730af88152f16e12aa95dd0e6761b317a    
  
author   : David Steele <david@pgbackrest.org>    
date     : Mon, 16 Jun 2025 16:04:35 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Mon, 16 Jun 2025 16:04:35 -0400    

commit   : 2e76e9fc699d22286f6bf05fddc271d8fa22523e    
  
author   : Anton Kurochkin <45575813+woblerr@users.noreply.github.com>    
date     : Mon, 9 Jun 2025 17:34:17 +0300    
  
committer: GitHub <noreply@github.com>    
date     : Mon, 9 Jun 2025 17:34:17 +0300    

In the documentation and help output the possible values for most options are described as a list but there were several commands for which formatting was done in a different style.  
  
Formatting has been applied to commands and their options to display option values as a list:  
  
* `info` command, `--type` option  
* `verify` command, `--output` option  
* `version` command, `--output` option  
* `manifest` command, `--output` option

commit   : 8aa2d88bac07225df8786ceb98c4eeac4fb58e5e    
  
author   : David Steele <david@pgbackrest.org>    
date     : Thu, 5 Jun 2025 17:33:18 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Thu, 5 Jun 2025 17:33:18 -0400    

A few bugs have been reported over the years on Alpine with musl libc and generally it has been possible to figure them out without testing on that platform but a few newer ones cannot be reproduced elsewhere.  
  
Also, testing with additional libc implementations helps portability so it makes sense to add support.  
  
For the most part musl libc works as expected with a few caveats:  
  
1) The FD_*() macros won't accept an int file descriptor without warning when -Wsign-conversion is enabled. I opened https://www.openwall.com/lists/musl/2025/05/23/4 to discuss this and I was referred to https://www.openwall.com/lists/musl/2024/07/16/1 which explains why this happens. It was not a very satisfying answer but clearly it is not going to be addressed so a meson probe was added to detect the issue and disable the warning.  
  
2) Floating point numbers are rounded differently than in GNU/Mac libc when formatted with printf() and friends. This is fine for the core code but causes issues in the unit tests that expect log entries to match exactly. This was solved in ad7ba46b by adding our own rough and ready formatting routines.  
  
3) Some error messages are different from GNU/Mac libc. This was solved with a new error macro that accepts multiple messages in b5fbb16c.  
  
4) For some reason ninja on Alpine outputs "nothing to do" messages to stderr whereas they go to stdout on other distributions. Redirecting stderr to stdout is our standard fix for this issue so do that. A non-zero result code will let us know that something has gone wrong.  
  
5) It appears that profiling is not supported on Alpine, which is pretty surprising. For now fix this by only unit testing the profiling code when coverage is enabled. This is not a great solution since we would rather test profiling on any system that supports it but for now this will do.  

commit   : b5fbb16cce1dc47d153e3f96171e6544407f714f    
  
author   : David Steele <david@pgbackrest.org>    
date     : Thu, 5 Jun 2025 15:35:28 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Thu, 5 Jun 2025 15:35:28 -0400    

This macro allows the error to be tested against multiple strings and it passes if any of them match.  
  
This is useful for supporting multiple libc versions/implementations that have different error messages and is needed for an upcoming commit to add unit test support for musl libc.  

commit   : ad7ba46bc6d9aa3074a33eca7eefc53ebb721cde    
  
author   : David Steele <david@pgbackrest.org>    
date     : Thu, 5 Jun 2025 13:27:45 -0400    
  
committer: GitHub <noreply@github.com>    
date     : Thu, 5 Jun 2025 13:27:45 -0400    

Some C libraries (e.g.musl) render floating point numbers differently when using printf(). This does not cause any problems for the core code but the unit tests require more predictability to function smoothly without a lot of exceptions.  
  
To accomplish this, wrap the various floating point operations in functions that mostly continue doing math using double but format the output string using integers. This leads to more predictable output at the cost of some complexity.  
  
Rounding could also be accomplished using nearbyint() but this would require linking the math library, which does not seem worth it for a fairly simple operation.

commit   : a30442f1688968b9fc133667b6860d18474f2ac6    
  
author   : David Steele <david@pgbackrest.org>    
date     : Sat, 31 May 2025 10:08:13 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Sat, 31 May 2025 10:08:13 -0400    

The prior check would result in a segfault from musl libc, which is understandable but not OK for a test.  
  
Instead set all the flags to get the expected invalid result.  
  
The second invalid test was not required for coverage so remove it. This was likely added before the earlier test was required for additional coverage.  

commit   : cb673225e90604a8865ce30c188d15661f945e5e    
  
author   : Anton Kurochkin <45575813+woblerr@users.noreply.github.com>    
date     : Wed, 28 May 2025 22:33:38 +0300    
  
committer: GitHub <noreply@github.com>    
date     : Wed, 28 May 2025 22:33:38 +0300    

commit   : f494dd21ade34bc09692515e13d7ca60b53373b0    
  
author   : David Steele <david@pgbackrest.org>    
date     : Tue, 27 May 2025 17:14:18 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Tue, 27 May 2025 17:14:18 -0400    

When the architecture was not specified it was being set to x86_64 rather than the host architecture.  

commit   : f4e1adf0591ab06c1ced79cf94a77331de16db52    
  
author   : David Steele <david@pgbackrest.org>    
date     : Tue, 27 May 2025 16:33:32 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Tue, 27 May 2025 16:33:32 -0400    

The list pointer was not nulled out so calling lstClear() caused a double free.  
  
This is not a production issue but was noticed in some upcoming test code.  

commit   : b5a26d99a04120ce0946cfe20829e44a7351cc9a    
  
author   : David Steele <david@pgbackrest.org>    
date     : Mon, 26 May 2025 12:49:47 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Mon, 26 May 2025 12:49:47 -0400    

The assignment was obviously not kosher but seemed fine for a test in short-lived fork. However, musl libc defines stderr as read-only so this trick won't work.  
  
Instead use dup2(), which is pretty clearly the correct way to do this.  

commit   : 8737cad566974c37529595715ecdc06571420752    
  
author   : David Steele <david@pgbackrest.org>    
date     : Mon, 12 May 2025 13:33:42 -0400    
  
committer: GitHub <noreply@github.com>    
date     : Mon, 12 May 2025 13:33:42 -0400    

This is another attempt to fix broken defaults in the command-line help. PR #2395 tried to do this by removing current and default values (at least from the list) but this removed much of the value of command-line help being context sensitive. Improvements were made to current values in b3ca2e34 but defaults were left as-is, i.e. broken in some cases.  
  
This new approach handles defaults much the same way as current values. If there are multiple defaults for an option then it is displayed in the option list as <multi> and enumerated in the specific option help. For example, if repo1-type=s3 and repo2-type=azure then the defaults for repo1-storage-upload-chunk-size and repo2-storage-upload-chunk-size will differ.  
  
However, getting the above right means we can no longer display defaults for options that are not valid in the current context. For instance, if repo1-type=posix then none of the repo1-s3-* options are valid and therefore their defaults are not displayed in help. In this case they don't have defaults and if you tried to set one of these options you would get an error.

commit   : 7b1b1f5322fe2b96c2325bb283178fb4ca1b602a    
  
author   : David Steele <david@pgbackrest.org>    
date     : Fri, 9 May 2025 17:45:05 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Fri, 9 May 2025 17:45:05 -0400    

Since the addition of libcurl4-openssl-dev requires a rebuild of the Debian containers go ahead and rebuild all containers to include new PostgreSQL minor release versions.  

commit   : 34b916f3231d32cd350cc7532ff3b30cf09065b9    
  
author   : David Steele <david@pgbackrest.org>    
date     : Fri, 9 May 2025 12:56:57 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Fri, 9 May 2025 12:56:57 -0400    

This aligns with what is reported by 'uname -m' and required by Docker naming conventions.  
  
Also convert a 'aarch64' string to the VM_ARCH_AARCH64 constant.  

commit   : 7786b5d8472221a94dda3dc3ac7871aab2c0e5be    
  
author   : David Steele <david@pgbackrest.org>    
date     : Fri, 9 May 2025 09:25:38 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Fri, 9 May 2025 09:25:38 -0400    

These data types are equivalent so no code changes required -- just renaming.  
  
The advantage is that uint8_t makes it clear that this is byte data. It also has the advantage of being shorter.  

commit   : 1e98684148c0095d2c8e4cabc033e2aa3b0c2554    
  
author   : David Steele <david@pgbackrest.org>    
date     : Thu, 8 May 2025 09:16:30 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Thu, 8 May 2025 09:16:30 -0400    

These defaults were being set in cfgLoadUpdate() but they can be better documented if they are part of the option rules. This is part of a general trend to move away from custom-coded defaults.  

commit   : 6869543b7ac312fa230ae52cf22a4727c74fb684    
  
author   : David Steele <david@pgbackrest.org>    
date     : Thu, 8 May 2025 09:12:38 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Thu, 8 May 2025 09:12:38 -0400    

This allows default types to be added without adding more boolean flags.  

commit   : 32b69e9accb02c0c83959a6d97798707f7b6dc3b    
  
author   : David Steele <david@pgbackrest.org>    
date     : Wed, 7 May 2025 09:35:54 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Wed, 7 May 2025 09:35:54 -0400    

This aligns better with naming in an upcoming commit.  

commit   : d45da0363e3dcf0df277255edf097a1bfb7194a7    
  
author   : David Steele <david@pgbackrest.org>    
date     : Tue, 6 May 2025 14:12:38 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Tue, 6 May 2025 14:12:38 -0400    

These options had defaults so this doesn't really change anything but it does prevent some difficult to explain churn in a future commit.  

commit   : e38e3e009035a4feab45279aca7f14112a0572b4    
  
author   : David Steele <david@pgbackrest.org>    
date     : Mon, 5 May 2025 13:40:14 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Mon, 5 May 2025 13:40:14 -0400    

commit   : 975516ae6d55676fb24562272d771fb3da295b23    
  
author   : David Steele <david@pgbackrest.org>    
date     : Sun, 4 May 2025 13:56:13 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Sun, 4 May 2025 13:56:13 -0400    

Once upon a time storing the default required creating a new string that needed to be assigned to the config mem context but that was a very long time ago and now only constants are used, so no need for a context switch.  

commit   : beeb93225f16c0f63c6ba0c8e5dd4eb2583863fc    
  
author   : David Steele <david@pgbackrest.org>    
date     : Sat, 3 May 2025 17:29:38 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Sat, 3 May 2025 17:29:38 -0400    

commit   : 96dfdce0859811a4e01d94cb7f401f872094bde8    
  
author   : David Steele <david@pgbackrest.org>    
date     : Tue, 29 Apr 2025 15:12:11 -0400    
  
committer: GitHub <noreply@github.com>    
date     : Tue, 29 Apr 2025 15:12:11 -0400    

20bfd14 removed content-md5 where allowed by the specification but failed to notice that either content-md5 or x-amz-content-* is required for PUT when object lock is enabled.  
  
On top of that it appears Scality S3 (at least?) won't accept alternate content checksums when object lock is enabled. Technically this is a violation of the specification but nonetheless the change breaks working installations.  
  
For now it seems safer to revert this change and pursue a better solution for a future feature release.

commit   : 9958b178344898476979bb27ef1c37f082d2ee5d    
  
author   : David Steele <david@pgbackrest.org>    
date     : Mon, 28 Apr 2025 14:33:56 -0400    
  
committer: GitHub <noreply@github.com>    
date     : Mon, 28 Apr 2025 14:33:56 -0400    

Specifying an indexed option with a key of 0, e.g. pg0-path, would lead to a segfault.  
  
Add lower bounds checking to fix the issue.

commit   : 1d89e3d75b9c19ee09b1d5906f4296914be2b896    
  
author   : David Steele <david@pgbackrest.org>    
date     : Wed, 23 Apr 2025 13:17:19 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Wed, 23 Apr 2025 13:17:19 -0400    

commit   : 01c8c0f8649f5cdc9c0cf83cfa78ee5c2bd72a18    
  
author   : David Steele <david@pgbackrest.org>    
date     : Mon, 21 Apr 2025 18:51:50 -0400    
  
committer: David Steele <david@pgbackrest.org>    
date     : Mon, 21 Apr 2025 18:51:50 -0400