pgBackRest 1.18 commit log · Life with Open Source

12 Apr 2017, 19:17 by Yogesh Sharma

`v1.18: Stanza Upgrade, Refactoring, and Locking Improvements`

commit   : 76161bd8451e581a8f9aa11c1f37d14863585d29    
  
author   : David Steele <david@pgmasters.net>    
date     : Wed, 12 Apr 2017 19:17:39 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 12 Apr 2017 19:17:39 -0400

Click here for diff

Bug Fixes:  
  
* Fixed an issue where read-only operations that used local worker processes (i.e. restore) were creating write locks that could interfere with parallel archive-push. (Reported by Jens Wilke.)  
  
Features:  
  
* Added the stanza-upgrade command to provide a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. (Contributed by Cynthia Shang.)  
* Added validation of pgbackrest.conf to display warnings if options are not valid or are not in the correct section. (Contributed by Cynthia Shang.)  
  
Refactoring:  
  
* Simplify locking scheme. Now, only the master process will hold write locks (for archive-push and backup commands) and not all local and remote worker processes as before.  
* Refactor Ini.pm to facilitate testing.  
* Do not set timestamps of files in the backup directories to match timestamps in the cluster directory. This was originally done to enable backup resume, but that process is now implemented with checksums.  
* Improved error message when the restore command detects the presence of postmaster.pid. (Suggested by Yogesh Sharma.)  
* Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. (Suggested by Yogesh Sharma.)  
* The backup and restore commands no longer copy via temp files. In both cases the files are checksummed on resume so there's no danger of partial copies.  
* Allow functions to accept optional parameters as a hash.  
* Refactor File->list() and fileList() to accept optional parameters.  
* Refactor backupLabel() and add unit tests.  
* Silence some perl critic warnings. (Contributed by Cynthia Shang.)

M README.md
M doc/release.pl
M doc/resource/exe.cache
M doc/xml/index.xml
M doc/xml/release.xml
M lib/pgBackRest/Version.pm
M libc/lib/pgBackRest/LibC.pm

`Go back to using static user for documentation.`

commit   : 5c635e0f0a1f5dc8ab8ef8c65204e81d3495afcb    
  
author   : David Steele <david@pgmasters.net>    
date     : Wed, 12 Apr 2017 18:36:33 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 12 Apr 2017 18:36:33 -0400

Click here for diff

Making this dynamic in commit 5d2e792 broke doc builds from cache.  The long-term solution is to create a special user for doc builds but that’s beyond the scope of this release.

M doc/lib/BackRestDoc/Common/DocExecute.pm
M doc/xml/user-guide.xml

`Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions.`

commit   : f207dc71238097a0fded50cc87bb7ef2afb006de    
  
author   : David Steele <david@pgmasters.net>    
date     : Wed, 12 Apr 2017 12:41:34 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 12 Apr 2017 12:41:34 -0400

Click here for diff

Suggested by Yogesh Sharma.

M doc/xml/release.xml
M doc/xml/user-guide.xml
M lib/pgBackRest/Common/Exception.pm
M lib/pgBackRest/Common/Log.pm
M lib/pgBackRest/Config/Config.pm
M lib/pgBackRest/Protocol/CommandMaster.pm
M test/expect/archive-get-001.log
M test/expect/archive-get-002.log
M test/expect/archive-get-003.log
M test/expect/archive-get-004.log
M test/expect/archive-get-005.log
M test/expect/archive-get-006.log
M test/expect/archive-get-007.log
M test/expect/archive-get-008.log
M test/expect/archive-push-001.log
M test/expect/archive-push-002.log
M test/expect/archive-push-003.log
M test/expect/archive-push-004.log
M test/expect/archive-push-005.log
M test/expect/archive-push-006.log
M test/expect/archive-push-007.log
M test/expect/archive-push-008.log
M test/expect/archive-stop-001.log
M test/expect/archive-stop-002.log
M test/expect/archive-stop-003.log
M test/expect/archive-stop-004.log
M test/expect/archive-stop-005.log
M test/expect/archive-stop-006.log
M test/expect/full-real-001.log
M test/expect/full-real-007.log
M test/expect/full-synthetic-001.log
M test/expect/full-synthetic-002.log
M test/expect/full-synthetic-003.log
M test/expect/full-synthetic-004.log
M test/expect/full-synthetic-005.log
M test/expect/full-synthetic-006.log
M test/expect/full-synthetic-007.log
M test/expect/full-synthetic-008.log
M test/expect/stanza-create-001.log
M test/expect/stanza-create-002.log
M test/expect/stanza-upgrade-001.log
M test/expect/stanza-upgrade-002.log

`Improved error message when the restore command detects the presence of postmaster.pid.`

commit   : f5f8d51366faae6ac31f355d021eaff67bef8f7f    
  
author   : David Steele <david@pgmasters.net>    
date     : Wed, 12 Apr 2017 09:53:07 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 12 Apr 2017 09:53:07 -0400

Click here for diff

Suggested by Yogesh Sharma.

M doc/xml/release.xml
M lib/pgBackRest/Restore.pm
M test/expect/full-synthetic-001.log
M test/lib/pgBackRestTest/Full/FullSyntheticTest.pm

`Do not set timestamps of files in the backup directories to match timestamps in the cluster directory.`

commit   : b0119914d4633431b208a5a327a3758a06258087    
  
author   : David Steele <david@pgmasters.net>    
date     : Wed, 12 Apr 2017 09:22:51 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 12 Apr 2017 09:22:51 -0400

Click here for diff

This was originally done to enable backup resume, but that process is now implemented with checksums.

M doc/xml/release.xml
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/BackupFile.pm

`Backup and restore commands no longer copy via temp files.`

commit   : fe02b40293fec08c819c5f41110203b8919fbae4    
  
author   : David Steele <david@pgmasters.net>    
date     : Wed, 12 Apr 2017 09:05:57 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 12 Apr 2017 09:05:57 -0400

Click here for diff

In both cases the files are checksummed on resume so there's no danger of partial copies.

M doc/xml/release.xml
M lib/pgBackRest/BackupFile.pm
M lib/pgBackRest/File.pm
M lib/pgBackRest/FileCommon.pm
M lib/pgBackRest/RestoreFile.pm
M test/expect/archive-get-002.log
M test/expect/archive-get-004.log
M test/expect/archive-get-006.log
M test/expect/archive-get-008.log
M test/expect/archive-push-001.log
M test/expect/archive-push-003.log
M test/expect/archive-push-005.log
M test/expect/archive-push-007.log
M test/expect/full-synthetic-001.log
M test/expect/full-synthetic-002.log
M test/expect/full-synthetic-003.log
M test/expect/full-synthetic-004.log
M test/expect/full-synthetic-005.log
M test/expect/full-synthetic-006.log
M test/expect/full-synthetic-007.log
M test/expect/full-synthetic-008.log
M test/expect/stanza-create-001.log
M test/expect/stanza-create-002.log

`Silence some perl critic warnings.`

commit   : 21a000d002bae2fa87cd09952740cbe14770aada    
  
author   : Cynthia Shang <cynthia.shang@crunchydata.com>    
date     : Mon, 10 Apr 2017 17:23:39 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 10 Apr 2017 17:23:39 -0400

Click here for diff

* Variables::RequireNegativeIndice  
* Variables::ProhibitUnusedVariables  
* Variables::ProhibitAugmentedAssignmentInDeclaration  
* Modules::ProhibitConditionalUseStatements

M doc/lib/BackRestDoc/Common/DocExecute.pm
M doc/lib/BackRestDoc/Common/DocRender.pm
M doc/lib/BackRestDoc/Html/DocHtmlPage.pm
M doc/xml/release.xml
M lib/pgBackRest/Common/Log.pm
M lib/pgBackRest/File.pm
M lib/pgBackRest/Info.pm
M test/lib/pgBackRestTest/Common/LogTest.pm
M test/lib/pgBackRestTest/Expire/ExpireEnvTest.pm
M test/lint/perlcritic.policy

`Ignore clock skew in container libc/package builds using make.`

commit   : 8546571461215eb65ffcff649c6baa17bc57d605    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 10 Apr 2017 17:20:21 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 10 Apr 2017 17:20:21 -0400

Click here for diff

It is common for containers to have clock skew so the build process takes care of this issue independently.

M doc/xml/release.xml
M test/test.pl

`Added ability to test warning messages.`

commit   : b63ede5614fed0dce32449f1450b7a5627eeeb3d    
  
author   : Cynthia Shang <cynthia.shang@crunchydata.com>    
date     : Mon, 10 Apr 2017 13:53:19 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 10 Apr 2017 13:53:19 -0400

Click here for diff

Contributed by Cynthia Shang.

M doc/xml/release.xml
M lib/pgBackRest/Common/Log.pm
M lib/pgBackRest/Config/Config.pm
M test/lib/pgBackRestTest/Common/RunTest.pm
M test/lib/pgBackRestTest/Config/ConfigUnitTest.pm

`Improvements to Ini.pm.`

commit   : 3d84f2ce5e02c5bd7f9bdcbb3b4579b146bf7ccf    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 10 Apr 2017 13:24:45 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 10 Apr 2017 13:24:45 -0400

Click here for diff

* Refactor Ini.pm to facilitate testing.  
* Complete statement/branch coverage for Ini.pm.  
* Improved functions used to test/munge manifest and info files.

M doc/lib/BackRestDoc/Common/DocExecute.pm
M doc/xml/release.xml
M lib/pgBackRest/Archive/ArchiveInfo.pm
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/BackupInfo.pm
M lib/pgBackRest/Common/Ini.pm
M lib/pgBackRest/Config/Config.pm
M lib/pgBackRest/Manifest.pm
M lib/pgBackRest/Restore.pm
M lib/pgBackRest/Stanza.pm
M test/expect/full-synthetic-001.log
M test/expect/full-synthetic-002.log
M test/expect/full-synthetic-003.log
M test/expect/full-synthetic-004.log
M test/expect/full-synthetic-005.log
M test/expect/full-synthetic-006.log
M test/expect/full-synthetic-007.log
M test/expect/full-synthetic-008.log
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/lib/pgBackRestTest/Common/Host/HostBackupTest.pm
M test/lib/pgBackRestTest/Common/Host/HostDbCommonTest.pm
M test/lib/pgBackRestTest/Config/ConfigConfigTest.pm
M test/lib/pgBackRestTest/Full/FullSyntheticTest.pm
A test/lib/pgBackRestTest/Info/InfoIniUnitTest.pm

`Coverage testing always enabled on Debian-based containers.`

commit   : 367e06f4be9133f399d4b7380ace8c5ca087c4b6    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 10 Apr 2017 12:31:30 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 10 Apr 2017 12:31:30 -0400

Click here for diff

* Full coverage is verified when specified.  
* Modules marked with partial coverage will error if they are actually fully covered.  
* Simplified test representation is DefineTest.  
* Added new representation for queries in DefineTest and added API functions.  
* Update modules using DefineTest to use new API.

M .travis.yml
M doc/xml/release.xml
M lib/pgBackRest/Common/String.pm
M test/Vagrantfile
M test/lib/pgBackRestTest/Common/CiTest.pm
M test/lib/pgBackRestTest/Common/ContainerTest.pm
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/lib/pgBackRestTest/Common/JobTest.pm
M test/lib/pgBackRestTest/Common/ListTest.pm
M test/lib/pgBackRestTest/Common/RunTest.pm
M test/lib/pgBackRestTest/Common/VmTest.pm
M test/test.pl

`Fixed locking bug and refactoring.`

commit   : eb4918d01881c20448e37da4d931d6e8a19f659a    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 7 Apr 2017 10:31:49 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 7 Apr 2017 10:31:49 -0400

Click here for diff

* Fixed an issue where read-only operations that used local worker processes (i.e. restore) were creating write locks that could interfere with parallel archive-push. (Reported by Jens Wilke.)  
* Simplify locking scheme. Now, only the master process will hold write locks (archive-push, backup) and not all the local and remote worker processes as before.

M bin/pgbackrest
M doc/xml/release.xml
M lib/pgBackRest/Archive/ArchivePushAsync.pm
M lib/pgBackRest/Common/Lock.pm
M lib/pgBackRest/Protocol/LocalMinion.pm
M lib/pgBackRest/Protocol/Protocol.pm
M test/expect/archive-get-001.log
M test/expect/archive-get-002.log
M test/expect/archive-get-003.log
M test/expect/archive-get-004.log
M test/expect/archive-get-005.log
M test/expect/archive-get-006.log
M test/expect/archive-get-007.log
M test/expect/archive-get-008.log
M test/expect/archive-push-001.log
M test/expect/archive-push-003.log
M test/expect/archive-push-005.log
M test/expect/archive-push-007.log
M test/expect/archive-stop-004.log
M test/expect/archive-stop-006.log
M test/expect/full-synthetic-001.log
M test/expect/full-synthetic-002.log
M test/expect/full-synthetic-003.log
M test/expect/full-synthetic-004.log
M test/expect/full-synthetic-005.log
M test/expect/full-synthetic-006.log
M test/expect/full-synthetic-007.log
M test/expect/full-synthetic-008.log
M test/expect/stanza-create-001.log
M test/expect/stanza-create-002.log
M test/lib/pgBackRestTest/Archive/ArchivePushUnitTest.pm

`Update wording for release note sections.`

commit   : edfb0962b678a5ebb3021d37d487fafb843dbf2a    
  
author   : David Steele <david@pgmasters.net>    
date     : Thu, 6 Apr 2017 21:47:43 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Thu, 6 Apr 2017 21:47:43 -0400

Click here for diff

M doc/lib/BackRestDoc/Custom/DocCustomRelease.pm
M doc/xml/release.xml

`Improvements and additional tests for stanza-upgrade.`

commit   : b620e6b282d3fc0bd1354b9490fbd84ee8d22d6e    
  
author   : Cynthia Shang <cynthia.shang@crunchydata.com>    
date     : Thu, 6 Apr 2017 19:36:37 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Thu, 6 Apr 2017 19:36:37 -0400

Click here for diff

Contributed by Cynthia Shang.

M doc/xml/user-guide.xml
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/BackupInfo.pm
M lib/pgBackRest/Expire.pm
M test/expect/expire-expire-001.log
M test/expect/expire-expire-002.log
M test/expect/full-synthetic-001.log
M test/expect/full-synthetic-002.log
M test/expect/full-synthetic-003.log
M test/expect/full-synthetic-004.log
M test/expect/full-synthetic-005.log
M test/expect/full-synthetic-006.log
M test/expect/full-synthetic-007.log
M test/expect/full-synthetic-008.log
M test/expect/stanza-upgrade-001.log
M test/expect/stanza-upgrade-002.log
A test/lib/pgBackRestTest/Backup/BackupInfoUnitTest.pm
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/lib/pgBackRestTest/Expire/ExpireExpireTest.pm
M test/lib/pgBackRestTest/Stanza/StanzaUnitTest.pm
M test/lib/pgBackRestTest/Stanza/StanzaUpgradeTest.pm

`Require description in every call to testResult() and make iWaitSeconds optional.`

commit   : 0da216c034d484927fa8ad0845e08f25c5227b5a    
  
author   : David Steele <david@crunchydata.com>    
date     : Wed, 5 Apr 2017 10:50:32 -0400    
  
committer: David Steele <david@crunchydata.com>    
date     : Wed, 5 Apr 2017 10:50:32 -0400

Click here for diff

M doc/xml/release.xml
M test/lib/pgBackRestTest/Archive/ArchivePushTest.pm
M test/lib/pgBackRestTest/Archive/ArchiveStopTest.pm
M test/lib/pgBackRestTest/Common/RunTest.pm
M test/lib/pgBackRestTest/File/FileOwnerTest.pm
M test/lib/pgBackRestTest/File/FileUnitTest.pm

`Refactor backupLabel() and add unit tests.`

commit   : c7b61a200258b2f10d1c62f2aedc7441e27bb6b1    
  
author   : David Steele <david@crunchydata.com>    
date     : Wed, 5 Apr 2017 10:16:16 -0400    
  
committer: David Steele <david@crunchydata.com>    
date     : Wed, 5 Apr 2017 10:16:16 -0400

Click here for diff

M bin/pgbackrest
M doc/xml/release.xml
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/BackupCommon.pm
M test/lib/pgBackRestTest/Backup/BackupUnitTest.pm
M test/lib/pgBackRestTest/Common/DefineTest.pm

`Allow functions to accept optional parameters as a hash.`

commit   : b2f43b56014f3d43404a17531887a09a78eb0db1    
  
author   : David Steele <david@pgmasters.net>    
date     : Tue, 4 Apr 2017 21:17:19 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Tue, 4 Apr 2017 21:17:19 -0400

Click here for diff

Refactor File->list() and fileList() to accept optional parameters.

M doc/xml/release.xml
M lib/pgBackRest/Archive/ArchiveCommon.pm
M lib/pgBackRest/Archive/ArchiveInfo.pm
M lib/pgBackRest/Archive/ArchivePush.pm
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/BackupInfo.pm
M lib/pgBackRest/Common/Log.pm
M lib/pgBackRest/Expire.pm
M lib/pgBackRest/File.pm
M lib/pgBackRest/FileCommon.pm
M lib/pgBackRest/Info.pm
M lib/pgBackRest/Stanza.pm
M test/lib/pgBackRestTest/Archive/ArchiveStopTest.pm
M test/lib/pgBackRestTest/Common/Host/HostBackupTest.pm
M test/lib/pgBackRestTest/File/FileListTest.pm

`Fixed flapping archive stop tests.`

commit   : 9eae36a7028a7c08869ec2e7c2f31f0aa49ecb4c    
  
author   : David Steele <david@pgmasters.net>    
date     : Tue, 4 Apr 2017 21:15:22 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Tue, 4 Apr 2017 21:15:22 -0400

Click here for diff

M doc/xml/release.xml
M test/lib/pgBackRestTest/Archive/ArchiveStopTest.pm

`Added stanza-upgrade command.`

commit   : beead043ac1f988e3757dc8689638ee5d443d9f9    
  
author   : Cynthia Shang <cynthia.shang@crunchydata.com>    
date     : Mon, 3 Apr 2017 10:42:55 -0400    
  
committer: David Steele <david@crunchydata.com>    
date     : Mon, 3 Apr 2017 10:42:55 -0400

Click here for diff

The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL.  
  
Contributed by Cynthia Shang.

M bin/pgbackrest
M doc/xml/reference.xml
M doc/xml/release.xml
M doc/xml/user-guide.xml
M lib/pgBackRest/Archive/ArchiveInfo.pm
M lib/pgBackRest/BackupInfo.pm
M lib/pgBackRest/Check/Check.pm
M lib/pgBackRest/Config/Config.pm
M lib/pgBackRest/Config/ConfigHelpData.pm
M lib/pgBackRest/Expire.pm
M lib/pgBackRest/Info.pm
A lib/pgBackRest/InfoCommon.pm
M lib/pgBackRest/Stanza.pm
A test/data/backup.pg_control_92.bin
M test/data/backup.pg_control_93.bin
A test/data/backup.pg_control_95.bin
A test/data/backup.wal1_92.bin
A test/data/backup.wal1_93.bin
A test/data/backup.wal1_95.bin
M test/expect/expire-expire-001.log
A test/expect/expire-expire-002.log
M test/expect/full-real-001.log
M test/expect/full-real-007.log
M test/expect/full-synthetic-001.log
M test/expect/full-synthetic-002.log
M test/expect/full-synthetic-003.log
M test/expect/full-synthetic-004.log
M test/expect/full-synthetic-005.log
M test/expect/full-synthetic-006.log
M test/expect/full-synthetic-007.log
M test/expect/full-synthetic-008.log
M test/expect/help-help-001.log
M test/expect/stanza-create-001.log
M test/expect/stanza-create-002.log
A test/expect/stanza-upgrade-001.log
A test/expect/stanza-upgrade-002.log
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/lib/pgBackRestTest/Common/Env/EnvHostTest.pm
M test/lib/pgBackRestTest/Common/Host/HostBackupTest.pm
M test/lib/pgBackRestTest/Expire/ExpireEnvTest.pm
M test/lib/pgBackRestTest/Expire/ExpireExpireTest.pm
M test/lib/pgBackRestTest/Full/FullRealTest.pm
M test/lib/pgBackRestTest/Full/FullSyntheticTest.pm
M test/lib/pgBackRestTest/Info/InfoUnitTest.pm
M test/lib/pgBackRestTest/Stanza/StanzaCreateTest.pm
A test/lib/pgBackRestTest/Stanza/StanzaUnitTest.pm
A test/lib/pgBackRestTest/Stanza/StanzaUpgradeTest.pm

`Added validation of pgbackrest.conf to display warnings if options are not valid or are not in the correct section.`

commit   : af7e4f4b4b6070e566865872e5240b824a14983f    
  
author   : Cynthia Shang <cynthia.shang@crunchydata.com>    
date     : Wed, 22 Mar 2017 19:03:28 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 22 Mar 2017 19:03:28 -0400

Click here for diff

Contributed by Cynthia Shang.

M doc/xml/release.xml
M lib/pgBackRest/Config/Config.pm
M lib/pgBackRest/Version.pm
M libc/lib/pgBackRest/LibC.pm
M test/lib/pgBackRestTest/Common/DefineTest.pm
A test/lib/pgBackRestTest/Config/ConfigUnitTest.pm

`Fixed user change in docs missed in vagrant upgrade.`

commit   : 5d2e79242da79854ee922c1a84dec0e67d52f090    
  
author   : David Steele <david@pgmasters.net>    
date     : Thu, 16 Mar 2017 19:50:59 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Thu, 16 Mar 2017 19:50:59 -0400

Click here for diff

M doc/lib/BackRestDoc/Common/DocExecute.pm
M doc/xml/user-guide.xml

`Updated vagrant to new version and image.`

commit   : cbd879a9830d2585b3df3209f59c4a9581a3c166    
  
author   : David Steele <david@pgmasters.net>    
date     : Tue, 14 Mar 2017 22:05:17 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Tue, 14 Mar 2017 22:05:17 -0400

Click here for diff

M .gitignore
A doc/.gitignore
M doc/xml/release.xml
A test/.gitignore
M test/Vagrantfile