pgBackRest 2.49 commit log

commit   : a7ab686d0e3d48b3200986d91ea3489488800382    
  
author   : David Steele <[email protected]>    
date     : Mon, 25 Sep 2023 09:32:15 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 25 Sep 2023 09:32:15 -0400    

Bug Fixes:  
  
* Fix issue restoring block incremental without a block list. (Reviewed by Stephen Frost, Burak Yurdakul. Reported by Burak Yurdakul.)  
  
Features:  
  
* Add --repo-storage-tag option to create object tags. (Reviewed by Stephen Frost, Stefan Fercot, Timothée Peignier.)  
* Add known hosts checking for SFTP storage driver. (Contributed by Reid Thompson. Reviewed by Stephen Frost, David Steele.)  
* Support for dual stack connections. (Reviewed by Stephen Frost.)  
* Add backup size completed/total to info command JSON output. (Contributed by Stefan Fercot. Reviewed by David Steele.)  
  
Improvements:  
  
* Multi-stanza check command. (Reviewed by Stephen Frost.)  
* Retry reads of pg_control until checksum is valid. (Reviewed by Stefan Fercot, Stephen Frost.)  
* Optimize WAL segment check after successful backup. (Reviewed by Stephen Frost.)  
* Improve GCS multi-part performance. (Reviewed by Reid Thompson.)  
* Allow archive-get command to run when stanza is stopped. (Reviewed by Tom Swartz, David Christensen, Reid Thompson.)  
* Accept leading tilde in paths for SFTP public/private keys. (Contributed by Reid Thompson. Reviewed by David Steele.)  
* Reload GCS credentials before renewing authentication token. (Reviewed by Stephen Frost. Suggested by Daniel Farina.)  
  
Documentation Bug Fixes:  
  
* Fix configuration reference example for the tls-server-address option. (Fixed by Hartmut Goebel. Reviewed by David Steele.)  
* Fix command reference example for the filter option.  
  
Test Suite Improvements:  
  
* Allow storage/sftp unit test to run without libssh2 installed. (Contributed by Reid Thompson. Reviewed by David Steele. Suggested by Wu Ning.)  

commit   : cb3ff6ed4336af9e4c0643d8224d5bc3d790a655    
  
author   : David Steele <[email protected]>    
date     : Sat, 23 Sep 2023 13:41:03 -0400    
  
committer: David Steele <[email protected]>    
date     : Sat, 23 Sep 2023 13:41:03 -0400    

This example was broken by 24f7252. Revert to (almost) the prior code to fix this example until something better can be committed. The something better is in progress but it adds new build requirements so it is too late to include it for the release.  
  
Technically this breaks some other examples, but they are all internal and not visible in the user-facing documentation.  

commit   : 6f0f2b371e6604552001b56e8931c6d15ceab41f    
  
author   : David Steele <[email protected]>    
date     : Tue, 19 Sep 2023 19:01:16 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 19 Sep 2023 19:01:16 -0400    

Fix capitalization and remove pluralization that is implied.  

commit   : c3c0834e170196a1241a368779162fcca65f9ad5    
  
author   : Reid Thompson <[email protected]>    
date     : Tue, 19 Sep 2023 16:26:13 -0400    
  
committer: GitHub <[email protected]>    
date     : Tue, 19 Sep 2023 16:26:13 -0400    

Add missing #ifdefs and update tests to prevent test compilation failure and test run issues when libssh2 is not present.

commit   : bb752cd111b1257088e57137bdbe6715e96eb292    
  
author   : David Steele <[email protected]>    
date     : Tue, 19 Sep 2023 12:08:09 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 19 Sep 2023 12:08:09 -0400    

These tests are already run as part of storage/posix and do not need to be duplicated in storage/sftp.  

commit   : 31de127cf42438b1fac04c161c47264aa1fb1060    
  
author   : David Steele <[email protected]>    
date     : Tue, 19 Sep 2023 11:30:29 -0400    
  
committer: GitHub <[email protected]>    
date     : Tue, 19 Sep 2023 11:30:29 -0400    

It is currently possible for a block map to be written without any accompanying blocks. This happens when a file timestamp is updated but the file has not changed. On restore, this caused problems when encryption was enabled, the block map was bundled after a file that had been stored without block incremental, and both files were included in a single bundle read. In this case the block map would not be decrypted and the encrypted data was passed to blockMapNewRead() with unpredictable results. In many cases built-in retries would rectify this problem as long as delta was enabled since block maps would move to the beginning of the bundle read and be decrypted properly. If enough files were affected, however, it could overwhelm the retries and throw an error. Subsequent delta restores would eventually be able to produce a valid result.
  

  
Fix this by moving block map decryption so it works correctly no matter where the block map is located in the read. This has the additional benefit of limiting how far the block map can read so it will error earlier if corrupt. Though in this case there was no repository corruption involved, it appeared that way to blockMapNewRead() since it was reading encrypted data.
  

  
Arguably block maps without blocks should not be written at all, but it would be better to consider that as a separate change. This pattern clearly exists in the wild and needs to be handled, plus the new implementation has other benefits.

commit   : 88edea4571d954e54a561247de587f747eaaadca    
  
author   : David Steele <[email protected]>    
date     : Mon, 18 Sep 2023 11:30:42 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 18 Sep 2023 11:30:42 -0400    

Log that block incremental was used and the delta size if less than the entire file was updated.  

commit   : d27533b40f039ff8da3a284d77fccff101083492    
  
author   : David Steele <[email protected]>    
date     : Sun, 17 Sep 2023 17:06:42 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 17 Sep 2023 17:06:42 -0400    

pg_data/ was appended at the beginning of the filter, which meant that files in tablespaces could never be queried directly.  
  
Update the filter to require the full path, including pg_data/ or pg_tblspc/.  

commit   : ed88f0483e8c87fde6dc098102b1c49e281d96e1    
  
author   : David Steele <[email protected]>    
date     : Sat, 16 Sep 2023 09:35:27 -0400    
  
committer: David Steele <[email protected]>    
date     : Sat, 16 Sep 2023 09:35:27 -0400    

commit   : ce9ba0fadee652d409488bee701ea6a51cafec75    
  
author   : Reid Thompson <[email protected]>    
date     : Fri, 15 Sep 2023 20:22:38 -0400    
  
committer: GitHub <[email protected]>    
date     : Fri, 15 Sep 2023 20:22:38 -0400    

By default require a known hosts match as part of the SFTP storage driver's authentication process, i.e. repo-sftp-host-key-check-type=strict. The match is expected to be found in the default list or in a list of known hosts files provided by the user. An exception is made if a fingerprint has been manually configured with repo-sftp-host-fingerprint or repo-sftp-host-key-check-type=accept-new can be used to automatically add new hosts.
  

  
Also allow host key verification to be skipped, as before, but require the user to explicitly set this (repo-sftp-host-key-check-type=none) rather than it being the default.

commit   : f5c730fd03ff27088bd91c148ed285c63b5e7502    
  
author   : David Steele <[email protected]>    
date     : Fri, 15 Sep 2023 18:15:02 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 15 Sep 2023 18:15:02 -0400    

Older versions of meson fail when a build target in a subproject has the same name as another subproject.  
  
This has been fixed in newer versions, but we still need to support older versions and in any case this seems cleaner and the help build target is already prefixed in this fashion.  

commit   : 9039d20b5bd73d3732ed6dac0a2bba4558176d8f    
  
author   : David Steele <[email protected]>    
date     : Fri, 15 Sep 2023 09:30:40 -0400    
  
committer: GitHub <[email protected]>    
date     : Fri, 15 Sep 2023 09:30:40 -0400    

This option is intended to eventually create a comprehensive report about the state of the pgBackRest configuration based on the results of the check command.
  

  
Implement a detailed report of the configuration options in the environment and configuration files. This should be useful information when debugging configuration errors, since invalid options and configurations are automatically noted. While custom config locations will not be found automatically, it will at least be clear that the config is not in a standard location.
  

  
For now keep this option internal since there is a lot of work to be done, but commit it so that it can be used when needed and tested in various environments.
  

  
Note that for now when --report is specified, the check command is not being run at all. Only the config report is generated. This behavior will be improved in the future.

commit   : 657c1a3e069910106bc3642153c27b0542c61dc3    
  
author   : David Steele <[email protected]>    
date     : Thu, 14 Sep 2023 18:41:36 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 14 Sep 2023 18:41:36 -0400    

commit   : 1b4e0cce5f3ff5ea3ee0b889e6d4ce4de1430b04    
  
author   : David Steele <[email protected]>    
date     : Thu, 14 Sep 2023 08:22:21 -0400    
  
committer: GitHub <[email protected]>    
date     : Thu, 14 Sep 2023 08:22:21 -0400    

This new option allows tags to be added to objects in S3, GCS, and Azure repositories.
  

  
This was fairly straightforward for S3 and Azure, but GCS does not allow tags for a simple upload using the JSON interface. If tags are required then the resumable interface must be used even if the file falls below the limit that usually triggers a resumable upload (i.e. size < repo-storage-upload-chunk-size).
  

  
This option is structured so that tags must be specified per-repo rather than globally for all repos. This seems logical since the tag keys and values may vary by service, e.g. S3 vs GCS.
  

  
These storage tags are independent of backup annotations since they are likely to be used for different purposes, e.g. billing, while the backup annotations are primarily intended for monitoring.

commit   : 3b9c31f6e3a0c23674f97b02c42133192fb51724    
  
author   : Reid Thompson <[email protected]>    
date     : Wed, 13 Sep 2023 16:59:52 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 13 Sep 2023 16:59:52 -0400    

commit   : 39bb8a0d3a96eef63219dff262a9306a4f59dae9    
  
author   : David Steele <[email protected]>    
date     : Tue, 12 Sep 2023 18:09:58 -0400    
  
committer: GitHub <[email protected]>    
date     : Tue, 12 Sep 2023 18:09:58 -0400    

The prior code would only connect to the first address provided by getaddrinfo().
  

  
Instead try each address in the list. If all connections fail then wait and try them all again until timeout.
  

  
Currently a round robin approach is used where each connection attempt must fail before the next connection is attempted. This works fine, for example, when an ipv6 address has no route to the host, but will work less well when a host answers but doesn't respond in a timely fashion.
  

  
We may consider a Happy Eyeballs approach in the future, but since pgBackRest is primarily a background process, it is not clear that slightly improved response time (in the case of failed connections) is worth the extra complexity.

commit   : 9d3a605900644059ff0e030d71a818b3965eb031    
  
author   : David Steele <[email protected]>    
date     : Sun, 10 Sep 2023 12:45:58 -0400    
  
committer: GitHub <[email protected]>    
date     : Sun, 10 Sep 2023 12:45:58 -0400    

The prior code did one list command against the storage for each WAL segment. This led to a lot of lists and was especially inefficient when the WAL (or the majority of it) was already present.
  

  
Optimize to keep the contents of a WAL directory and use them on a subsequent search. Leave the optimizations for a single WAL segment since other places still use that mode.

commit   : edbd520c81f5d437963aaf42d1c79b2c51d01665    
  
author   : David Steele <[email protected]>    
date     : Sun, 10 Sep 2023 10:36:58 -0400    
  
committer: GitHub <[email protected]>    
date     : Sun, 10 Sep 2023 10:36:58 -0400    

sckHostLookup() only returned the first address record returned from getaddrinfo(). The new AddressInfo object provides a full list of values returned from getaddrinfo(). Freeing the list is also handled by the object so there is no longer a need for FINALLY blocks to ensure the list is freed.
  

  
Add the selected address to the client/server names for debugging purposes.
  

  
This code does not attempt to connect to multiple addresses. It just lays the groundwork for a future commit to do so.

commit   : f42d927d2dcd58cf5a80d68eaf6888bcf2c67a4b    
  
author   : David Steele <[email protected]>    
date     : Sun, 10 Sep 2023 09:47:49 -0400    
  
committer: GitHub <[email protected]>    
date     : Sun, 10 Sep 2023 09:47:49 -0400    

On certain file systems (e.g. ext4) pg_control may appear torn if there is a concurrent write while reading the file. To prevent an invalid read, retry until the checksum matches the control data.
  

  
Special handling is required for the pg-version-force feature since the offset of the checksum is not known. In this case, scan from the default position to the end of the data looking for a checksum match. This is a bit imprecise, but better than nothing, and the chance of a random collision in the control data seems very remote considering the ratio of data size (< 512 bytes) to checksum size (4 bytes).
  

  
This was discovered and a possible solution proposed for PostgreSQL in [1]. The proposed solution may work for backup, but pgBackRest needs to be able to read pg_control reliably outside of backup. So no matter what fix is adopted for PostgreSQL, pgBackRest need retries. Further adjustment may be required as the PostgreSQL fix evolves.
  

  
[1] https://www.postgresql.org/message-id/20221123014224.xisi44byq3cf5psi%40awork3.anarazel.de

commit   : c1805134b370f0c5c54c40a2df8978599328784f    
  
author   : David Steele <[email protected]>    
date     : Sat, 9 Sep 2023 12:54:55 -0400    
  
committer: David Steele <[email protected]>    
date     : Sat, 9 Sep 2023 12:54:55 -0400    

If there are a lot of retries then the output might be very large and even be truncated by the error module. Either way, it is not good information for the user.  
  
When a message is repeated, aggregate so that total retries and time range are output for the message. This provides helpful information about what happened without overwhelming the user with data.  

commit   : 5314dbffc761bc7507db02693f2421c1578efe10    
  
author   : David Steele <[email protected]>    
date     : Sat, 9 Sep 2023 11:22:33 -0400    
  
committer: David Steele <[email protected]>    
date     : Sat, 9 Sep 2023 11:22:33 -0400    

The prior code gave a "free" extra iteration at the end of the wait, functionality that was copied directly from the equivalent code in Perl. This works and is mostly negligible except when wait loops are nested, in which case outer loops will always run twice even if an inner loop times out, which has a multiplying effect. For example, three nested wait loops with a timeout of three seconds will result in the inner loop being run four times (for a total of twelve seconds) even if it times out each time.  
  
Instead make waitMore() stop exactly when time is up. This makes more sense because a complete failure and timeout of an inner loop means retrying an outer loop is probably a waste of time since that inner loop will likely continue to fail.  
  
Also make waitRemaining() recalculate the remaining time rather than depending on the prior result.  
  
Some tests needed to be adjusted to take into account there being one less loop. In general this led to a simplification of the tests.  
  
Reinit a begin value in the wait unit tests. This is not related to the current change but it does make the time measurements more accurate and less likely to fail on an edge case, which has been observed from time to time.  
  
This change appears to have a benefit for test runtime, which seems plausible especially for nested waits, but a larger sample of CI runs are needed to be sure.  

commit   : d57900839addeac56b89a445352a8efc96a197f0    
  
author   : David Steele <[email protected]>    
date     : Mon, 4 Sep 2023 18:51:44 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 4 Sep 2023 18:51:44 -0400    

The current tests only generate small quantities of WAL per backup but sometimes it is useful to generate large quantities for testing.  
  
Fix the issues with generating large quantities of WAL and also improve memory management.  

commit   : 43524a47a4c6a847f76514812f94b87cb7af4c05    
  
author   : Hartmut Goebel <[email protected]>    
date     : Mon, 4 Sep 2023 19:39:11 +0200    
  
committer: GitHub <[email protected]>    
date     : Mon, 4 Sep 2023 19:39:11 +0200    

The prior example (::*) was not valid and would result in the following error:
  

  
ERROR: [049]: unable to get address for '::*': [-2] Name or service not known
  

  
Correct values are either * for IPv4 or :: for IPv6. The IPv4 value is used as the example since only one example is allowed.

commit   : c0935b79924686ebf4efef99c7dc69aea9efb28d    
  
author   : David Steele <[email protected]>    
date     : Mon, 4 Sep 2023 12:00:06 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 4 Sep 2023 12:00:06 -0400    

The release.xml file was getting pretty unwieldy so break release notes into separate files. Also break contributors into a separate file.  
  
In theory most of release.xml could now be generated automatically but adding a new release does not represent a serious maintenance burden, so for the time being it does not seem worth it.  

commit   : 6cb9c40fb8ea18e6572bf3f6e44bbb1d8f922682    
  
author   : David Steele <[email protected]>    
date     : Mon, 4 Sep 2023 10:39:19 -0400    
  
committer: GitHub <[email protected]>    
date     : Mon, 4 Sep 2023 10:39:19 -0400    

The prior code avoided uploading a chunk if it was not clear whether the write was complete or not. This was primarily due to the GCS documentation being very vague on what to do in the case of a zero-size chunk.
  

  
Now chunks are uploaded as they are available. This should improve performance and also reduces the diff against a future commit that absolutely requires zero-size chunks.

commit   : fd9c6b0e9ddd38de57475d1325bc53b79833a983    
  
author   : David Steele <[email protected]>    
date     : Fri, 1 Sep 2023 12:03:39 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 1 Sep 2023 12:03:39 -0400    

commit   : 4f52015b14a1f4e5df9373707179a5447cb5a4a6    
  
author   : David Steele <[email protected]>    
date     : Thu, 31 Aug 2023 20:22:01 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 31 Aug 2023 20:22:01 -0400    

commit   : e3fcd8cd9e0be42a32fc4cbd2ae3db2c072eabcd    
  
author   : Stefan Fercot <[email protected]>    
date     : Thu, 31 Aug 2023 22:36:52 +0200    
  
committer: GitHub <[email protected]>    
date     : Thu, 31 Aug 2023 22:36:52 +0200    

This allows for backup progress to be monitored by external processes, whereas before backup progress was only available via text output.

commit   : fbd992adc7b18db5e70774d675c160c8c8e2f031    
  
author   : David Steele <[email protected]>    
date     : Tue, 29 Aug 2023 13:49:49 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 29 Aug 2023 13:49:49 -0400    

The restore command can run while the stanza is stopped so it makes sense for the archive-get command to follow the same rule.  
  
The important thing is to ensure that all commands that write to the repository are stopped when the stanza is stopped.  

commit   : 8424737697edfee7cbea195b3bccb4077e1c2ee5    
  
author   : David Steele <[email protected]>    
date     : Tue, 29 Aug 2023 12:35:29 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 29 Aug 2023 12:35:29 -0400    

The key also needs to be encoded (not just the value).  
  
This is not currently an issue because none of the keys in use require encoding.  

commit   : d24180e4da49a8e2ef2a10a02e76cd204fe1ae8f    
  
author   : David Steele <[email protected]>    
date     : Tue, 29 Aug 2023 12:28:02 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 29 Aug 2023 12:28:02 -0400    

The tests expect the group name/id to match between the host system and the container. If there is a conflict rename the group with the required id to the expected name.  
  
This could have unintended consequences but it seems reasonably safe since we control everything that runs in the container and there should never be any system processes running.  

commit   : 1e6efd73e1fcf7bbe3d3f04d925fefde44847109    
  
author   : David Steele <[email protected]>    
date     : Wed, 23 Aug 2023 12:56:49 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 23 Aug 2023 12:56:49 -0400    

Store the value of storageReadIo() rather than calling it each time. This is slightly more efficient, but more importantly it will be needed for an upcoming commit.  

commit   : 82c12fed123df057fce109757aa5c72dce605440    
  
author   : David Steele <[email protected]>    
date     : Mon, 21 Aug 2023 13:10:48 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 21 Aug 2023 13:10:48 -0400    

Coverity complained that: Overrunning array "optionGroupIndexKeep[optionGroupIdx]" of 512 bytes at byte offset 4294967550 using index "keyIdx" (which evaluates to 4294967294).  
  
This does not seem possible but adjust the code to make Coverity happy, as usual.  

commit   : eb32d6de5e1575a37efb166abbf7df400d93352e    
  
author   : Reid Thompson <[email protected]>    
date     : Tue, 8 Aug 2023 03:41:34 -0400    
  
committer: GitHub <[email protected]>    
date     : Tue, 8 Aug 2023 03:41:34 -0400    

The documentation indicates that leading tilde file paths for public/private keys are valid but the functionality was omitted from the original implementation.

commit   : 1141dc20708cf0c68296f83e74c872a7db9c2bd9    
  
author   : David Steele <[email protected]>    
date     : Mon, 7 Aug 2023 17:03:09 +0100    
  
committer: GitHub <[email protected]>    
date     : Mon, 7 Aug 2023 17:03:09 +0100    

Check command now checks multiple stanzas when the stanza option is omitted.
  

  
The stanza list is extracted from the current configuration rather than scanning the repository like the info command. Scanning the repository is a problem because configuration for each stanza may not be present in the current configuration. Since this functionality is new for check there is no regression.
  

  
Add a new section to the user guide to cover multi-stanza configuration and provide additional coverage for this feature.
  

  
Also fix a small issue in the parser when an indexed option has a dependency on a non-indexed option. There were no examples of this case in the previous configuration.

commit   : 995a8e96691d23eaf3914a425669f89deb3ce002    
  
author   : David Steele <[email protected]>    
date     : Mon, 7 Aug 2023 13:30:50 +0100    
  
committer: GitHub <[email protected]>    
date     : Mon, 7 Aug 2023 13:30:50 +0100    

This allows the service key to be updated while a command is running. The new key should be written atomically and ideally the old key should remain valid for some period of time to avoid a race condition if the old token happens to expire at the same time that the new key is being generated.

commit   : cbafcfabf2eb593de9ee95483da93f278c9ff1ee    
  
author   : David Steele <[email protected]>    
date     : Mon, 7 Aug 2023 12:38:33 +0100    
  
committer: David Steele <[email protected]>    
date     : Mon, 7 Aug 2023 12:38:33 +0100    

The ZLIB_CONST macro may be used since 1.2.5.2 (17 Dec 2011) to specify that the input buffers are const. This is sufficiently old to cover all non-EOL distributions, i.e. everything we test on.  
  
Compiling on older distributions may generate warnings but will continue to work.  

commit   : 6c75c99d0b3024311ea7b254953f1c6a901bcb70    
  
author   : David Steele <[email protected]>    
date     : Mon, 31 Jul 2023 11:09:17 +0200    
  
committer: David Steele <[email protected]>    
date     : Mon, 31 Jul 2023 11:09:17 +0200    

This release fixes compilation errors on some platforms and contains numerous performance improvements.  

commit   : f55c8f1b813fadbc7b19a1cc4eb8f90e36b2b18c    
  
author   : David Steele <[email protected]>    
date     : Wed, 26 Jul 2023 10:45:12 +0200    
  
committer: David Steele <[email protected]>    
date     : Wed, 26 Jul 2023 10:45:12 +0200    

Rename to HRN_PG_CONTROL_OVERRIDE_VERSION_PUT() since other types of overrides are possible (and coming).  

commit   : 65fb3b3b2d89e2fc80698e52a8378395b18e6d2a    
  
author   : David Steele <[email protected]>    
date     : Mon, 24 Jul 2023 09:17:46 +0200    
  
committer: David Steele <[email protected]>    
date     : Mon, 24 Jul 2023 09:17:46 +0200    

commit   : c585ddaf48b806b5d038f0c90e7848875f7a533e    
  
author   : David Steele <[email protected]>    
date     : Mon, 24 Jul 2023 09:15:38 +0200    
  
committer: David Steele <[email protected]>    
date     : Mon, 24 Jul 2023 09:15:38 +0200