pgBackRest 1.6 commit log

by Yogesh Sharma
pgBackRest
commit log

v1.06: Backup from Standby and Bug Fixes

commit   : be77136eaeca4d0943bd00280afcbe3f5a53a096    
  
author   : David Steele <david@pgmasters.net>    
date     : Thu, 25 Aug 2016 11:49:09 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Thu, 25 Aug 2016 11:49:09 -0400

Click here for diff

Bug Fixes:  
  
* Fixed an issue where a tablespace link that referenced another link would not produce an error, but instead skip the tablespace entirely. (Reported by Michael Vitale.)  
* Fixed an issue where options that should not allow multiple values could be specified multiple times in pgbackrest.conf without an error being raised. (Reported by Michael Vitale.)  
* Fixed an issue where the protocol-timeout option was not automatically increased when the db-timeout option was increased. (Reported by Todd Vernick.)  
  
Features:  
  
* Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.  
* More flexible configuration for databases. Master and standby can both be configured on the backup server and pgBackRest will automatically determine which is the master. This means no configuration changes for backup are required after failing over from a master to standby when a separate backup server is used.  
* Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include pgsql_tmp and pg_stat_tmp. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.  
* Experimental support for non-exclusive backups in PostgreSQL 9.6 beta4. Changes to the control/catalog/WAL versions in subsequent betas may break compatibility but pgBackRest will be updated with each release to keep pace.  
  
Refactoring:  
  
* Simplify protocol creation and identifying which host is local/remote.  
* Removed all OP_* function constants that were used only for debugging, not in the protocol, and replaced with __PACKAGE__.  
* Improvements in Db module: separated out connect() function, allow executeSql() calls that do not return data, and improve error handling.  
* Improve error message for links that reference links in manifest build.  
* Added hints to error message when relative paths are detected in archive-push or archive-get.  
* Improve backup log messages to indicate which host the files are being copied from.

M README.md
M doc/resource/exe.cache
M doc/xml/release.xml
M lib/pgBackRest/Version.pm

Fixed timeout issue.

commit   : c9bdf6a9ae2f2657ba06eb97a7fbaba38251ef1c    
  
author   : David Steele <david@pgmasters.net>    
date     : Thu, 25 Aug 2016 11:27:00 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Thu, 25 Aug 2016 11:27:00 -0400

Click here for diff

The protocol-timeout option was not automatically increased when the db-timeout option was increased.  
  
Reported by Todd Vernick.

M doc/xml/release.xml
M lib/pgBackRest/Config/Config.pm
M test/lib/pgBackRestTest/Config/ConfigTest.pm

Backup from a standby cluster.

commit   : 5ada189a92813a57b9050dec54ba3ddf69f68bf0    
  
author   : David Steele <david@pgmasters.net>    
date     : Thu, 25 Aug 2016 11:25:46 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Thu, 25 Aug 2016 11:25:46 -0400

Click here for diff

A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.

M doc/xml/reference.xml
M doc/xml/release.xml
M doc/xml/user-guide.xml
M lib/pgBackRest/ArchiveCommon.pm
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/BackupInfo.pm
M lib/pgBackRest/Config/Config.pm
M lib/pgBackRest/Config/ConfigHelpData.pm
M lib/pgBackRest/Db.pm
M lib/pgBackRest/Manifest.pm
M test/expect/backup-expire-001.log
M test/expect/backup-full-005.log
M test/expect/backup-full-006.log
M test/expect/backup-full-011.log
M test/expect/backup-synthetic-001.log
M test/expect/backup-synthetic-002.log
M test/expect/backup-synthetic-003.log
M test/expect/backup-synthetic-004.log
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log
M test/lib/pgBackRestTest/Backup/BackupTest.pm
M test/lib/pgBackRestTest/Backup/Common/ExpireCommonTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbCommonTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbTest.pm

Improve host tag rendering.

commit   : ecf827579edc9dc620a27985c0bd75a1d8cb3d23    
  
author   : David Steele <david@pgmasters.net>    
date     : Thu, 25 Aug 2016 10:04:46 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Thu, 25 Aug 2016 10:04:46 -0400

Click here for diff

M doc/lib/BackRestDoc/Common/DocRender.pm
M doc/resource/html/default.css
M doc/xml/dtd/doc.dtd
M doc/xml/release.xml

More flexible configuration for databases

commit   : d0b6f78b20cb7c74b43717a97242fce7160d2604    
  
author   : David Steele <david@crunchydata.com>    
date     : Wed, 24 Aug 2016 12:39:27 -0400    
  
committer: David Steele <david@crunchydata.com>    
date     : Wed, 24 Aug 2016 12:39:27 -0400

Click here for diff

Master and standby can both be configured on the backup server and pgBackRest will automatically determine which is the master. This means no configuration changes for backup are required after failing over from a master to standby when a separate backup server is used.

M bin/pgbackrest
M doc/lib/BackRestDoc/Common/DocConfig.pm
M doc/lib/BackRestDoc/Common/DocExecute.pm
M doc/xml/reference.xml
M doc/xml/release.xml
M doc/xml/user-guide.xml
M lib/pgBackRest/Archive.pm
A lib/pgBackRest/ArchiveCommon.pm
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/BackupFile.pm
M lib/pgBackRest/Common/Exception.pm
M lib/pgBackRest/Common/Exit.pm
M lib/pgBackRest/Common/Ini.pm
M lib/pgBackRest/Config/Config.pm
M lib/pgBackRest/Config/ConfigHelpData.pm
M lib/pgBackRest/Db.pm
M lib/pgBackRest/DbVersion.pm
M lib/pgBackRest/Expire.pm
M lib/pgBackRest/File.pm
M lib/pgBackRest/Info.pm
M lib/pgBackRest/Protocol/Common.pm
M lib/pgBackRest/Protocol/CommonMaster.pm
A lib/pgBackRest/Protocol/Protocol.pm
M lib/pgBackRest/Protocol/RemoteMaster.pm
M lib/pgBackRest/Protocol/RemoteMinion.pm
M lib/pgBackRest/Protocol/ThreadGroup.pm
M lib/pgBackRest/Restore.pm
M lib/pgBackRest/Version.pm
M test/expect/backup-archive-get-001.log
M test/expect/backup-archive-get-002.log
M test/expect/backup-archive-get-003.log
M test/expect/backup-archive-get-004.log
M test/expect/backup-archive-get-005.log
M test/expect/backup-archive-get-006.log
M test/expect/backup-archive-get-007.log
M test/expect/backup-archive-get-008.log
M test/expect/backup-archive-push-001.log
M test/expect/backup-archive-push-002.log
M test/expect/backup-archive-push-003.log
M test/expect/backup-archive-push-004.log
M test/expect/backup-archive-push-005.log
M test/expect/backup-archive-push-006.log
M test/expect/backup-archive-push-007.log
M test/expect/backup-archive-push-008.log
M test/expect/backup-archive-stop-001.log
M test/expect/backup-archive-stop-002.log
M test/expect/backup-archive-stop-003.log
M test/expect/backup-archive-stop-004.log
M test/expect/backup-archive-stop-005.log
M test/expect/backup-archive-stop-006.log
A test/expect/backup-full-001.log
A test/expect/backup-full-002.log
A test/expect/backup-full-003.log
A test/expect/backup-full-004.log
A test/expect/backup-full-005.log
A test/expect/backup-full-006.log
A test/expect/backup-full-007.log
A test/expect/backup-full-008.log
A test/expect/backup-full-009.log
A test/expect/backup-full-010.log
A test/expect/backup-full-011.log
M test/expect/backup-synthetic-001.log
M test/expect/backup-synthetic-002.log
M test/expect/backup-synthetic-003.log
M test/expect/backup-synthetic-004.log
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log
M test/lib/pgBackRestTest/Backup/BackupCommonTest.pm
M test/lib/pgBackRestTest/Backup/BackupTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostBackupTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostBaseTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbCommonTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbSyntheticTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbTest.pm
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/lib/pgBackRestTest/Common/ExecuteTest.pm
M test/lib/pgBackRestTest/Common/HostTest.pm
M test/lib/pgBackRestTest/Common/LogTest.pm
M test/lib/pgBackRestTest/Config/ConfigTest.pm
M test/lib/pgBackRestTest/File/FileTest.pm
M test/test.pl

Revert some backup exclusions until they have been tested more thoroughly.

commit   : cd6278e5af22119cb0c4e3714dc2a66d3cd219c0    
  
author   : David Steele <david@crunchydata.com>    
date     : Wed, 24 Aug 2016 12:27:48 -0400    
  
committer: David Steele <david@crunchydata.com>    
date     : Wed, 24 Aug 2016 12:27:48 -0400

Click here for diff

M doc/xml/release.xml
M lib/pgBackRest/Manifest.pm
M test/expect/backup-synthetic-001.log
M test/lib/pgBackRestTest/Backup/BackupTest.pm

mprove backup log messages to indicated which host the files are being copied from.

commit   : 9c13ca1d091e38dddcd38656fc8af748e8c19675    
  
author   : David Steele <david@crunchydata.com>    
date     : Wed, 24 Aug 2016 12:01:08 -0400    
  
committer: David Steele <david@crunchydata.com>    
date     : Wed, 24 Aug 2016 12:01:08 -0400

Click here for diff

M doc/xml/release.xml
M lib/pgBackRest/BackupFile.pm
M lib/pgBackRest/Protocol/RemoteMaster.pm
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log

Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup.

commit   : f1412baccf9ab4b5015d839ae0b598d92dd32586    
  
author   : David Steele <david@crunchydata.com>    
date     : Tue, 16 Aug 2016 09:35:16 -0400    
  
committer: David Steele <david@crunchydata.com>    
date     : Tue, 16 Aug 2016 09:35:16 -0400

Click here for diff

These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.

M doc/xml/release.xml
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/Manifest.pm
M test/expect/backup-synthetic-001.log
M test/expect/backup-synthetic-002.log
M test/expect/backup-synthetic-003.log
M test/expect/backup-synthetic-004.log
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log
M test/lib/pgBackRestTest/Backup/BackupTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbCommonTest.pm

Added hints to error message when relative paths are detected in archive-push or archive-get.

commit   : 332055971e29beb031276d7ce32ad4827dabf6d2    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 15 Aug 2016 20:54:17 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 15 Aug 2016 20:54:17 -0400

Click here for diff

M doc/xml/release.xml
M lib/pgBackRest/Archive.pm

Fixed #229: Check fails on target db host.

commit   : b25f10949a016cc4dc6d2e3ca1c8a63f764059c9    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 15 Aug 2016 20:15:17 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 15 Aug 2016 20:15:17 -0400

Click here for diff

Options that should not allow multiple values could be specified multiple times in pgbackrest.conf without an error being raised.

M doc/xml/release.xml
M lib/pgBackRest/Common/Exception.pm
M lib/pgBackRest/Config/Config.pm
M test/lib/pgBackRestTest/Config/ConfigTest.pm

Improvements in Db module.

commit   : 88d9bd3d37475c907a4fffdbf907deafc0b44bc3    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 15 Aug 2016 17:44:51 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 15 Aug 2016 17:44:51 -0400

Click here for diff

* Separated out connect() function.  
* Allow executeSql() calls that do not return data.  
* Improve error handling.

M doc/xml/release.xml
M lib/pgBackRest/Db.pm
M lib/pgBackRest/Protocol/RemoteMinion.pm

commit   : 1e0f15f4253319ff141455a46d597d2f387c8ff1    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 15 Aug 2016 17:23:37 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 15 Aug 2016 17:23:37 -0400

Click here for diff

M doc/xml/release.xml
M lib/pgBackRest/Manifest.pm
M test/expect/backup-synthetic-001.log
M test/expect/backup-synthetic-002.log
M test/expect/backup-synthetic-003.log
M test/expect/backup-synthetic-004.log
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log

Fixed #236: Recursive user tablespace symlink.

commit   : f9fa1270b24e2511717d4b4a3ddbee861bb32760    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 15 Aug 2016 17:11:45 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 15 Aug 2016 17:11:45 -0400

Click here for diff

A tablespace link that referenced another link would not produce an error, but instead skip the tablespace entirely.

M doc/xml/release.xml
M lib/pgBackRest/Manifest.pm
M test/expect/backup-synthetic-001.log
M test/lib/pgBackRestTest/Backup/BackupTest.pm

Update control version for PostgreSQL beta4.

commit   : 35931305d71186935534b26d0b08c25946decd58    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 15 Aug 2016 16:48:24 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 15 Aug 2016 16:48:24 -0400

Click here for diff

M doc/xml/release.xml
M lib/pgBackRest/Db.pm

Database version refactoring.

commit   : 17b79d62799ac82876b71d79a57d5e5259a2431d    
  
author   : David Steele <david@pgmasters.net>    
date     : Thu, 11 Aug 2016 22:35:24 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Thu, 11 Aug 2016 22:35:24 -0400

Click here for diff

* Refactor db version constants into a separate module.  
* Update synthetic backup tests to PostgreSQL 9.4.

M doc/xml/release.xml
M lib/pgBackRest/Archive.pm
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/Db.pm
A lib/pgBackRest/DbVersion.pm
M lib/pgBackRest/Manifest.pm
M lib/pgBackRest/Restore.pm
A test/data/pg_control_93
A test/data/pg_control_94
M test/expect/backup-synthetic-001.log
M test/expect/backup-synthetic-002.log
M test/expect/backup-synthetic-003.log
M test/expect/backup-synthetic-004.log
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log
M test/lib/pgBackRestTest/Backup/BackupTest.pm
M test/lib/pgBackRestTest/Backup/Common/ExpireCommonTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbCommonTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbSyntheticTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbTest.pm
M test/lib/pgBackRestTest/Common/FileTest.pm
M test/lib/pgBackRestTest/Common/VmTest.pm
M test/test.pl

Removed all OP_* function constants that were used only for debugging, not in the protocol, and replaced with __PACKAGE__.

commit   : bff262ac4791fae834f13ee4c422da8a4e7cdb1d    
  
author   : David Steele <david@pgmasters.net>    
date     : Thu, 11 Aug 2016 17:32:28 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Thu, 11 Aug 2016 17:32:28 -0400

Click here for diff

M doc/xml/release.xml
M lib/pgBackRest/Archive.pm
M lib/pgBackRest/ArchiveInfo.pm
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/BackupCommon.pm
M lib/pgBackRest/BackupFile.pm
M lib/pgBackRest/BackupInfo.pm
M lib/pgBackRest/Common/Exit.pm
M lib/pgBackRest/Common/Ini.pm
M lib/pgBackRest/Common/Lock.pm
M lib/pgBackRest/Common/Log.pm
M lib/pgBackRest/Db.pm
M lib/pgBackRest/Expire.pm
M lib/pgBackRest/File.pm
M lib/pgBackRest/FileCommon.pm
M lib/pgBackRest/Info.pm
M lib/pgBackRest/Manifest.pm
M lib/pgBackRest/Protocol/Common.pm
M lib/pgBackRest/Protocol/CommonMaster.pm
M lib/pgBackRest/Protocol/CommonMinion.pm
M lib/pgBackRest/Protocol/IO.pm
M lib/pgBackRest/Protocol/RemoteMaster.pm
M lib/pgBackRest/Protocol/RemoteMinion.pm
M lib/pgBackRest/Restore.pm
M lib/pgBackRest/RestoreFile.pm
M lib/pgBackRest/Version.pm
M test/expect/backup-archive-get-001.log
M test/expect/backup-archive-get-002.log
M test/expect/backup-archive-get-003.log
M test/expect/backup-archive-get-004.log
M test/expect/backup-archive-get-005.log
M test/expect/backup-archive-get-006.log
M test/expect/backup-archive-get-007.log
M test/expect/backup-archive-get-008.log
M test/expect/backup-archive-push-001.log
M test/expect/backup-archive-push-002.log
M test/expect/backup-archive-push-003.log
M test/expect/backup-archive-push-004.log
M test/expect/backup-archive-push-005.log
M test/expect/backup-archive-push-006.log
M test/expect/backup-archive-push-007.log
M test/expect/backup-archive-push-008.log
M test/expect/backup-archive-stop-001.log
M test/expect/backup-archive-stop-002.log
M test/expect/backup-archive-stop-003.log
M test/expect/backup-archive-stop-004.log
M test/expect/backup-archive-stop-005.log
M test/expect/backup-archive-stop-006.log
M test/expect/backup-synthetic-001.log
M test/expect/backup-synthetic-002.log
M test/expect/backup-synthetic-003.log
M test/expect/backup-synthetic-004.log
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log
M test/lib/pgBackRestTest/Backup/Common/ExpireCommonTest.pm
M test/lib/pgBackRestTest/Common/ExecuteTest.pm
M test/lib/pgBackRestTest/Common/LogTest.pm