pgBackRest 1.3 commit log

v1.03: Check Command and Bug Fixes

  
commit   : 1b62354dcb6cd815fc8a5b235b0aaf52eb85a692    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 2 Jul 2016 10:22:52 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 2 Jul 2016 10:22:52 -0400    

Click here for diff

  
Bug Fixes:  
  
* Fixed an issue where keep-alives could be starved out by lots of small files during multi-threaded backup. They were also completely absent from single/multi-threaded backup resume and restore checksumming. (Reported by Janice Parkinson, Chris Barber.)  
* Fixed an issue where the expire command would refuse to run when explicitly called from the command line if the db-host option was set. This was not an issue when expire was run automatically after a backup (Reported by Chris Barber.)  
* Fixed an issue where validation was being running on archive_command even when the archive-check option was disabled.  
  
Features:  
  
* Added check command to validate that pgBackRest is configured correctly for archiving and backups. (Contributed by Cynthia Shang.)  
* Added the protocol-timeout option. Previously protocol-timeout was set as db-timeout + 30 seconds.  
* Failure to shutdown remotes at the end of the backup no longer throws an exception. Instead a warning is generated that recommends a higher protocol-timeout.  
* Experimental support for non-exclusive backups in PostgreSQL 9.6 beta2. Changes to the control/catalog/WAL versions in subsequent betas may break compatibility but pgBackRest will be updated with each release to keep pace.  
  
Refactoring:  
  
* The pg_xlogfile_name() function is no longer used to construct WAL filenames from LSNs. While this function is convenient it is not available on a standby. Instead, the archive is searched for the LSN in order to find the timeline. If due to some misadventure the LSN appears on multiple timelines then an error will be thrown, whereas before this condition would have passed unnoticed.  
* Option handling is now far more strict. Previously it was possible for a command to use an option that was not explicitly assigned to it. This was especially true for the backup-host and db-host options which are used to determine locality.  
* Improved handling of users/groups captured during backup that do not exist on the restore host. Also explicitly handle the case where user/group is not mapped to a name.  
* Changed version variable to a constant. It had originally been designed to play nice with a specific packaging tool but that tool was never used.  
  

Update WAL/catalog versions for 9.5 beta 2.

  
commit   : 186ca18659f42e1cc287f0737db0f2138fa3dc54    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 2 Jul 2016 09:19:10 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 2 Jul 2016 09:19:10 -0400    

Click here for diff

  
  

Order database versions during test.pl –vm-build.

  
commit   : 5df2eed72916bc68dbf2bb0d772b4a5e3fcfbe55    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 2 Jul 2016 09:07:48 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 2 Jul 2016 09:07:48 -0400    

Click here for diff

  
  

Closed #213: Archive command should not be validate when archive-check=n

  
commit   : 418039027b32c680680f83183dabb1f7e1adbcb4    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 2 Jul 2016 09:02:55 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 2 Jul 2016 09:02:55 -0400    

Click here for diff

  
  

Fix for container build in Vagrantfile.

  
commit   : 0f2eb424bb6d53f2c6619f70535706e649f7c9a9    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 2 Jul 2016 08:59:11 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 2 Jul 2016 08:59:11 -0400    

Click here for diff

  
  

Added documentation for asynchronous archiving.

  
commit   : 9f9396943f20e00157b53494bc074d355456b689    
  
author   : Cynthia Shang <cynthia.shang@crunchydata.com>    
date     : Thu, 30 Jun 2016 19:38:28 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Thu, 30 Jun 2016 19:38:28 -0400    

Click here for diff

  
Also removed some obsolete sections.  
  
Contributed by Cynthia Shang.  
  

Release note edits.

  
commit   : 304d890f6c0ead8d49cf8f31fcef9f918904896a    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 27 Jun 2016 23:07:47 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 27 Jun 2016 23:07:47 -0400    

Click here for diff

  
  

Closed #209: Dont’ use “localtime” for HTML-docs.

  
commit   : 9516db4ad660a4be29fd43b34ab9e0ef848daaf8    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 27 Jun 2016 20:37:25 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 27 Jun 2016 20:37:25 -0400    

Click here for diff

  
The release-date-static var has been added to allow the html footer to use the last release date instead of localtime().  
  

User/group permissions improvements.

  
commit   : cc2a8777d5efdaebfc17b053f747e8683a8ce3b1    
  
author   : David Steele <david@pgmasters.net>    
date     : Sun, 26 Jun 2016 21:01:20 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sun, 26 Jun 2016 21:01:20 -0400    

Click here for diff

  
Improved handling of users/groups captured during backup that do not exist on the restore host. Also explicitly handle the case where user/group is not mapped to a name.  
  

Minor refactor of version variable to a constant.

  
commit   : b571218930c4ef5578e2c42f73c3e24bcfaba216    
  
author   : David Steele <david@pgmasters.net>    
date     : Sun, 26 Jun 2016 20:53:45 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sun, 26 Jun 2016 20:53:45 -0400    

Click here for diff

  
It had originally been designed to play nice with a specific packaging tool but that tool was never used.  
  

Added more keep-alives.

  
commit   : 6ee485cc76ec05421ac4f45e50cd90911739163d    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 24 Jun 2016 13:16:07 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 24 Jun 2016 13:16:07 -0400    

Click here for diff

  
Keep-alives were also missing from multi-threaded backup/restore and single-threaded restore.  
  
Found by Chris Barber.  
  

Rename test paths for clarity.

  
commit   : bd25223fd6167ef2e1cc4e9788de46371a647572    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 24 Jun 2016 10:54:31 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 24 Jun 2016 10:54:31 -0400    

Click here for diff

  
This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved.  Keeping them separate made it easier to audit the changes in the refactor.  
  

Major refactor of the test suite.

  
commit   : c8f806a293813323affea96c73b1aa177a0ac15a    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 24 Jun 2016 08:12:58 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 24 Jun 2016 08:12:58 -0400    

Click here for diff

  
* Make the code more modular and object-oriented.  
* Multiple Docker containers can now be created for a single test to simulate more realistic environments.  
  

Stop using pg_xlogfile_name().

  
commit   : 23a39118301384d939f431a344a724c8fc39f5e0    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 24 Jun 2016 08:06:20 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 24 Jun 2016 08:06:20 -0400    

Click here for diff

  
The pg_xlogfile_name() function is no longer used to construct WAL filenames from LSNs. While this function is convenient it is not available on a standby. Instead, the archive is searched for the LSN in order to find the timeline. If due to some misadventure the LSN appears on multiple timelines then an error will be thrown, whereas before this condition would have passed unnoticed.  
  

Protocol timeout option and keep-alive fixes.

  
commit   : 41b27c0456793694316daa882ffbd2209fd249c3    
  
author   : David Steele <david@pgmasters.net>    
date     : Wed, 22 Jun 2016 18:01:18 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 22 Jun 2016 18:01:18 -0400    

Click here for diff

  
* Fixed an issue where keep-alives could be starved out by lots of small files during multi-threaded operation and were completely absent during single-threaded operation when resuming from a previous incomplete backup.  
  
Reported by Janice Parkinson.  
  
* Added the protocol-timeout option. Previously protocol-timeout was set as db-timeout + 30 seconds.  
* Failure to shutdown remotes at the end of the backup no longer throws an exception. A warning is still generated that recommends a higher protocol-timeout.  
  

Fixed pgBackRest config sections being blank in the output when not loaded from cache.

  
commit   : 4cff5bb0e69f03f047f8e8fe96a9ea336d7b4f31    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 18 Jun 2016 10:59:45 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 18 Jun 2016 10:59:45 -0400    

Click here for diff

  
  

Closed #207: Expire fails with unhandled exception.

  
commit   : 012405a33b01a3f215a0ee610a2648c5f5afb0f6    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 18 Jun 2016 09:55:00 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 18 Jun 2016 09:55:00 -0400    

Click here for diff

  
* Fixed an issue where the expire command would refuse to run when explicitly called from the command line if the db-host option was set.  This was not an issue when expire was run after a backup, which is the usual case.  
* Option handling is now far more strict. Previously it was possible for a command to use an option that was not explicitly assigned to it. This was especially true for the backup-host and db-host options which are used to determine locality.  
  
Reported by Chris Barber.  
  

Fixed issue in doc info output with –require.

  
commit   : e988b96eceafd1619d2f8c51c94e38523a051f54    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 18 Jun 2016 09:22:26 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 18 Jun 2016 09:22:26 -0400    

Click here for diff

  
Section names were being repeated in the info output when multiple --require options depended on the same sections.  
  

VM build improvements.

  
commit   : cb32a8ab1f1aa2f71070cfd61c70d7fb4933abb4    
  
author   : David Steele <david@crunchydata.com>    
date     : Mon, 13 Jun 2016 11:18:06 -0400    
  
committer: David Steele <david@crunchydata.com>    
date     : Mon, 13 Jun 2016 11:18:06 -0400    

Click here for diff

  
* Properly set user env before calling test.pl --vm-build.  
* Remove squid proxy made obsolete by more efficient builds.  
  

Added check command.

  
commit   : 7e45ed8366f8933a0e8f52b5777f6618d57766a2    
  
author   : Cynthia Shang <cynthia.shang@crunchydata.com>    
date     : Sun, 12 Jun 2016 09:13:46 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sun, 12 Jun 2016 09:13:46 -0400    

Click here for diff

  
The check command validates that pgBackRest is configured correctly for archiving and backups.  
  
Contributed by Cynthia Shang.  
  

Add cmd-description tag.

  
commit   : 7c9eaf7210181ca806c1e8b477bc8a0df8d4af3a    
  
author   : David Steele <david@pgmasters.net>    
date     : Sun, 12 Jun 2016 09:01:56 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sun, 12 Jun 2016 09:01:56 -0400    

Click here for diff

  
Allow command summaries to be inserted anywhere in the documentation to avoid duplication.  
  

Docker container improvements.

  
commit   : f1e92804cef8e2fa7f85d231aaf5d5710130bee9    
  
author   : David Steele <david@pgmasters.net>    
date     : Sun, 12 Jun 2016 09:00:16 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sun, 12 Jun 2016 09:00:16 -0400    

Click here for diff

  
* Containers now use a squid proxy for apt/yum to speed builds.  
* Obsolete containers are removed by the <br-option>--vm-force</br-option> option.  
* Greatly reduced the quantity of Docker containers built by default. Containers are only built for PostgreSQL versions specified in db-minimal and those required to build documentation. Additional containers can be built with --db-version=all or by specifying a version, e.g. --db-version=9.4.  
  

Update TeX Live to 2016 version.

  
commit   : 3d761b726aeb243ffda53e6c8d122aa9d323ac32    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 10 Jun 2016 21:22:36 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 10 Jun 2016 21:22:36 -0400    

Click here for diff

  
  

Fixed DTDs not being loaded from test.pl.

  
commit   : 5660f69859f02be53b328f7b7773b651ba4e13c1    
  
author   : David Steele <david@crunchydata.com>    
date     : Mon, 6 Jun 2016 13:27:52 -0400    
  
committer: David Steele <david@crunchydata.com>    
date     : Mon, 6 Jun 2016 13:27:52 -0400    

Click here for diff

  
  

PostgreSQL version is now a variable to allow multi-version documentation.

  
commit   : 3f7ca876c255f7e3af772cf374ba9bc44a2be701    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 4 Jun 2016 12:24:31 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 4 Jun 2016 12:24:31 -0400    

Click here for diff

  
  

  
commit   : bfed78ef42599ea5c7e883b16efef973ee84b23a    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 4 Jun 2016 12:04:09 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 4 Jun 2016 12:04:09 -0400    

Click here for diff

  
  

Fixed pgBackRest-specific xml that was loaded for non-pgBackRest projects.

  
commit   : 1ddfb1e453aef26590b649c77330d5206978917d    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 4 Jun 2016 11:40:33 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 4 Jun 2016 11:40:33 -0400    

Click here for diff

  
  

Fixed DTD search path.

  
commit   : 8ddfab0cbc646bfd11fcee1b4a47d3a3b617ae7a    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 4 Jun 2016 11:38:23 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 4 Jun 2016 11:38:23 -0400    

Click here for diff

  
The search path that did not work properly when --doc-path was used.  
  

Add cmd-extra field.

  
commit   : e796f563c4af7a265a1f8ecd94e7f3f795a78839    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 3 Jun 2016 20:07:28 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 3 Jun 2016 20:07:28 -0400    

Click here for diff

  
Allow hidden options to be added to a command. This allows certain commands (like apt-get) to be forced during the build without making that a part of the documentation.  
  

Update installation documentation.

  
commit   : 36cf0b18f5c7db870f8b0338ee43e84c32ad4661    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 3 Jun 2016 18:59:12 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 3 Jun 2016 18:59:12 -0400    

Click here for diff

  
* Recommended install location for pgBackRest modules is now /usr/share/perl5 since /usr/lib/perl5 has been removed from the search path in newer versions of Perl.  
  
* Added instructions for removing prior versions of pgBackRest.