v1.06: Backup from Standby and Bug Fixes
commit : be77136eaeca4d0943bd00280afcbe3f5a53a096
author : David Steele <[email protected]>
date : Thu, 25 Aug 2016 11:49:09 -0400
committer: David Steele <[email protected]>
date : Thu, 25 Aug 2016 11:49:09 -0400
Bug Fixes:
* Fixed an issue where a tablespace link that referenced another link would not produce an error, but instead skip the tablespace entirely. (Reported by Michael Vitale.)
* Fixed an issue where options that should not allow multiple values could be specified multiple times in pgbackrest.conf without an error being raised. (Reported by Michael Vitale.)
* Fixed an issue where the protocol-timeout option was not automatically increased when the db-timeout option was increased. (Reported by Todd Vernick.)
Features:
* Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
* More flexible configuration for databases. Master and standby can both be configured on the backup server and pgBackRest will automatically determine which is the master. This means no configuration changes for backup are required after failing over from a master to standby when a separate backup server is used.
* Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include pgsql_tmp and pg_stat_tmp. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
* Experimental support for non-exclusive backups in PostgreSQL 9.6 beta4. Changes to the control/catalog/WAL versions in subsequent betas may break compatibility but pgBackRest will be updated with each release to keep pace.
Refactoring:
* Simplify protocol creation and identifying which host is local/remote.
* Removed all OP_* function constants that were used only for debugging, not in the protocol, and replaced with __PACKAGE__.
* Improvements in Db module: separated out connect() function, allow executeSql() calls that do not return data, and improve error handling.
* Improve error message for links that reference links in manifest build.
* Added hints to error message when relative paths are detected in archive-push or archive-get.
* Improve backup log messages to indicate which host the files are being copied from.
M README.md
M doc/resource/exe.cache
M doc/xml/release.xml
M lib/pgBackRest/Version.pm
Fixed timeout issue.
commit : c9bdf6a9ae2f2657ba06eb97a7fbaba38251ef1c
author : David Steele <[email protected]>
date : Thu, 25 Aug 2016 11:27:00 -0400
committer: David Steele <[email protected]>
date : Thu, 25 Aug 2016 11:27:00 -0400
The protocol-timeout option was not automatically increased when the db-timeout option was increased.
Reported by Todd Vernick.
M doc/xml/release.xml
M lib/pgBackRest/Config/Config.pm
M test/lib/pgBackRestTest/Config/ConfigTest.pm
Backup from a standby cluster.
commit : 5ada189a92813a57b9050dec54ba3ddf69f68bf0
author : David Steele <[email protected]>
date : Thu, 25 Aug 2016 11:25:46 -0400
committer: David Steele <[email protected]>
date : Thu, 25 Aug 2016 11:25:46 -0400
A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
M doc/xml/reference.xml
M doc/xml/release.xml
M doc/xml/user-guide.xml
M lib/pgBackRest/ArchiveCommon.pm
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/BackupInfo.pm
M lib/pgBackRest/Config/Config.pm
M lib/pgBackRest/Config/ConfigHelpData.pm
M lib/pgBackRest/Db.pm
M lib/pgBackRest/Manifest.pm
M test/expect/backup-expire-001.log
M test/expect/backup-full-005.log
M test/expect/backup-full-006.log
M test/expect/backup-full-011.log
M test/expect/backup-synthetic-001.log
M test/expect/backup-synthetic-002.log
M test/expect/backup-synthetic-003.log
M test/expect/backup-synthetic-004.log
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log
M test/lib/pgBackRestTest/Backup/BackupTest.pm
M test/lib/pgBackRestTest/Backup/Common/ExpireCommonTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbCommonTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbTest.pm
Improve host tag rendering.
commit : ecf827579edc9dc620a27985c0bd75a1d8cb3d23
author : David Steele <[email protected]>
date : Thu, 25 Aug 2016 10:04:46 -0400
committer: David Steele <[email protected]>
date : Thu, 25 Aug 2016 10:04:46 -0400
M doc/lib/BackRestDoc/Common/DocRender.pm
M doc/resource/html/default.css
M doc/xml/dtd/doc.dtd
M doc/xml/release.xml
More flexible configuration for databases
commit : d0b6f78b20cb7c74b43717a97242fce7160d2604
author : David Steele <[email protected]>
date : Wed, 24 Aug 2016 12:39:27 -0400
committer: David Steele <[email protected]>
date : Wed, 24 Aug 2016 12:39:27 -0400
Master and standby can both be configured on the backup server and pgBackRest will automatically determine which is the master. This means no configuration changes for backup are required after failing over from a master to standby when a separate backup server is used.
M bin/pgbackrest
M doc/lib/BackRestDoc/Common/DocConfig.pm
M doc/lib/BackRestDoc/Common/DocExecute.pm
M doc/xml/reference.xml
M doc/xml/release.xml
M doc/xml/user-guide.xml
M lib/pgBackRest/Archive.pm
A lib/pgBackRest/ArchiveCommon.pm
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/BackupFile.pm
M lib/pgBackRest/Common/Exception.pm
M lib/pgBackRest/Common/Exit.pm
M lib/pgBackRest/Common/Ini.pm
M lib/pgBackRest/Config/Config.pm
M lib/pgBackRest/Config/ConfigHelpData.pm
M lib/pgBackRest/Db.pm
M lib/pgBackRest/DbVersion.pm
M lib/pgBackRest/Expire.pm
M lib/pgBackRest/File.pm
M lib/pgBackRest/Info.pm
M lib/pgBackRest/Protocol/Common.pm
M lib/pgBackRest/Protocol/CommonMaster.pm
A lib/pgBackRest/Protocol/Protocol.pm
M lib/pgBackRest/Protocol/RemoteMaster.pm
M lib/pgBackRest/Protocol/RemoteMinion.pm
M lib/pgBackRest/Protocol/ThreadGroup.pm
M lib/pgBackRest/Restore.pm
M lib/pgBackRest/Version.pm
M test/expect/backup-archive-get-001.log
M test/expect/backup-archive-get-002.log
M test/expect/backup-archive-get-003.log
M test/expect/backup-archive-get-004.log
M test/expect/backup-archive-get-005.log
M test/expect/backup-archive-get-006.log
M test/expect/backup-archive-get-007.log
M test/expect/backup-archive-get-008.log
M test/expect/backup-archive-push-001.log
M test/expect/backup-archive-push-002.log
M test/expect/backup-archive-push-003.log
M test/expect/backup-archive-push-004.log
M test/expect/backup-archive-push-005.log
M test/expect/backup-archive-push-006.log
M test/expect/backup-archive-push-007.log
M test/expect/backup-archive-push-008.log
M test/expect/backup-archive-stop-001.log
M test/expect/backup-archive-stop-002.log
M test/expect/backup-archive-stop-003.log
M test/expect/backup-archive-stop-004.log
M test/expect/backup-archive-stop-005.log
M test/expect/backup-archive-stop-006.log
A test/expect/backup-full-001.log
A test/expect/backup-full-002.log
A test/expect/backup-full-003.log
A test/expect/backup-full-004.log
A test/expect/backup-full-005.log
A test/expect/backup-full-006.log
A test/expect/backup-full-007.log
A test/expect/backup-full-008.log
A test/expect/backup-full-009.log
A test/expect/backup-full-010.log
A test/expect/backup-full-011.log
M test/expect/backup-synthetic-001.log
M test/expect/backup-synthetic-002.log
M test/expect/backup-synthetic-003.log
M test/expect/backup-synthetic-004.log
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log
M test/lib/pgBackRestTest/Backup/BackupCommonTest.pm
M test/lib/pgBackRestTest/Backup/BackupTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostBackupTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostBaseTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbCommonTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbSyntheticTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbTest.pm
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/lib/pgBackRestTest/Common/ExecuteTest.pm
M test/lib/pgBackRestTest/Common/HostTest.pm
M test/lib/pgBackRestTest/Common/LogTest.pm
M test/lib/pgBackRestTest/Config/ConfigTest.pm
M test/lib/pgBackRestTest/File/FileTest.pm
M test/test.pl
Revert some backup exclusions until they have been tested more thoroughly.
commit : cd6278e5af22119cb0c4e3714dc2a66d3cd219c0
author : David Steele <[email protected]>
date : Wed, 24 Aug 2016 12:27:48 -0400
committer: David Steele <[email protected]>
date : Wed, 24 Aug 2016 12:27:48 -0400
M doc/xml/release.xml
M lib/pgBackRest/Manifest.pm
M test/expect/backup-synthetic-001.log
M test/lib/pgBackRestTest/Backup/BackupTest.pm
mprove backup log messages to indicated which host the files are being copied from.
commit : 9c13ca1d091e38dddcd38656fc8af748e8c19675
author : David Steele <[email protected]>
date : Wed, 24 Aug 2016 12:01:08 -0400
committer: David Steele <[email protected]>
date : Wed, 24 Aug 2016 12:01:08 -0400
M doc/xml/release.xml
M lib/pgBackRest/BackupFile.pm
M lib/pgBackRest/Protocol/RemoteMaster.pm
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup.
commit : f1412baccf9ab4b5015d839ae0b598d92dd32586
author : David Steele <[email protected]>
date : Tue, 16 Aug 2016 09:35:16 -0400
committer: David Steele <[email protected]>
date : Tue, 16 Aug 2016 09:35:16 -0400
These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
M doc/xml/release.xml
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/Manifest.pm
M test/expect/backup-synthetic-001.log
M test/expect/backup-synthetic-002.log
M test/expect/backup-synthetic-003.log
M test/expect/backup-synthetic-004.log
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log
M test/lib/pgBackRestTest/Backup/BackupTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbCommonTest.pm
Added hints to error message when relative paths are detected in archive-push or archive-get.
commit : 332055971e29beb031276d7ce32ad4827dabf6d2
author : David Steele <[email protected]>
date : Mon, 15 Aug 2016 20:54:17 -0400
committer: David Steele <[email protected]>
date : Mon, 15 Aug 2016 20:54:17 -0400
M doc/xml/release.xml
M lib/pgBackRest/Archive.pm
Fixed #229: Check fails on target db host.
commit : b25f10949a016cc4dc6d2e3ca1c8a63f764059c9
author : David Steele <[email protected]>
date : Mon, 15 Aug 2016 20:15:17 -0400
committer: David Steele <[email protected]>
date : Mon, 15 Aug 2016 20:15:17 -0400
Options that should not allow multiple values could be specified multiple times in pgbackrest.conf without an error being raised.
M doc/xml/release.xml
M lib/pgBackRest/Common/Exception.pm
M lib/pgBackRest/Config/Config.pm
M test/lib/pgBackRestTest/Config/ConfigTest.pm
Improvements in Db module.
commit : 88d9bd3d37475c907a4fffdbf907deafc0b44bc3
author : David Steele <[email protected]>
date : Mon, 15 Aug 2016 17:44:51 -0400
committer: David Steele <[email protected]>
date : Mon, 15 Aug 2016 17:44:51 -0400
* Separated out connect() function.
* Allow executeSql() calls that do not return data.
* Improve error handling.
M doc/xml/release.xml
M lib/pgBackRest/Db.pm
M lib/pgBackRest/Protocol/RemoteMinion.pm
Improve error message for links that reference links in manifest build.
commit : 1e0f15f4253319ff141455a46d597d2f387c8ff1
author : David Steele <[email protected]>
date : Mon, 15 Aug 2016 17:23:37 -0400
committer: David Steele <[email protected]>
date : Mon, 15 Aug 2016 17:23:37 -0400
M doc/xml/release.xml
M lib/pgBackRest/Manifest.pm
M test/expect/backup-synthetic-001.log
M test/expect/backup-synthetic-002.log
M test/expect/backup-synthetic-003.log
M test/expect/backup-synthetic-004.log
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log
Fixed #236: Recursive user tablespace symlink.
commit : f9fa1270b24e2511717d4b4a3ddbee861bb32760
author : David Steele <[email protected]>
date : Mon, 15 Aug 2016 17:11:45 -0400
committer: David Steele <[email protected]>
date : Mon, 15 Aug 2016 17:11:45 -0400
A tablespace link that referenced another link would not produce an error, but instead skip the tablespace entirely.
M doc/xml/release.xml
M lib/pgBackRest/Manifest.pm
M test/expect/backup-synthetic-001.log
M test/lib/pgBackRestTest/Backup/BackupTest.pm
Update control version for PostgreSQL beta4.
commit : 35931305d71186935534b26d0b08c25946decd58
author : David Steele <[email protected]>
date : Mon, 15 Aug 2016 16:48:24 -0400
committer: David Steele <[email protected]>
date : Mon, 15 Aug 2016 16:48:24 -0400
M doc/xml/release.xml
M lib/pgBackRest/Db.pm
Database version refactoring.
commit : 17b79d62799ac82876b71d79a57d5e5259a2431d
author : David Steele <[email protected]>
date : Thu, 11 Aug 2016 22:35:24 -0400
committer: David Steele <[email protected]>
date : Thu, 11 Aug 2016 22:35:24 -0400
* Refactor db version constants into a separate module.
* Update synthetic backup tests to PostgreSQL 9.4.
M doc/xml/release.xml
M lib/pgBackRest/Archive.pm
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/Db.pm
A lib/pgBackRest/DbVersion.pm
M lib/pgBackRest/Manifest.pm
M lib/pgBackRest/Restore.pm
A test/data/pg_control_93
A test/data/pg_control_94
M test/expect/backup-synthetic-001.log
M test/expect/backup-synthetic-002.log
M test/expect/backup-synthetic-003.log
M test/expect/backup-synthetic-004.log
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log
M test/lib/pgBackRestTest/Backup/BackupTest.pm
M test/lib/pgBackRestTest/Backup/Common/ExpireCommonTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbCommonTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbSyntheticTest.pm
M test/lib/pgBackRestTest/Backup/Common/HostDbTest.pm
M test/lib/pgBackRestTest/Common/FileTest.pm
M test/lib/pgBackRestTest/Common/VmTest.pm
M test/test.pl
Removed all OP_* function constants that were used only for debugging, not in the protocol, and replaced with __PACKAGE__.
commit : bff262ac4791fae834f13ee4c422da8a4e7cdb1d
author : David Steele <[email protected]>
date : Thu, 11 Aug 2016 17:32:28 -0400
committer: David Steele <[email protected]>
date : Thu, 11 Aug 2016 17:32:28 -0400
M doc/xml/release.xml
M lib/pgBackRest/Archive.pm
M lib/pgBackRest/ArchiveInfo.pm
M lib/pgBackRest/Backup.pm
M lib/pgBackRest/BackupCommon.pm
M lib/pgBackRest/BackupFile.pm
M lib/pgBackRest/BackupInfo.pm
M lib/pgBackRest/Common/Exit.pm
M lib/pgBackRest/Common/Ini.pm
M lib/pgBackRest/Common/Lock.pm
M lib/pgBackRest/Common/Log.pm
M lib/pgBackRest/Db.pm
M lib/pgBackRest/Expire.pm
M lib/pgBackRest/File.pm
M lib/pgBackRest/FileCommon.pm
M lib/pgBackRest/Info.pm
M lib/pgBackRest/Manifest.pm
M lib/pgBackRest/Protocol/Common.pm
M lib/pgBackRest/Protocol/CommonMaster.pm
M lib/pgBackRest/Protocol/CommonMinion.pm
M lib/pgBackRest/Protocol/IO.pm
M lib/pgBackRest/Protocol/RemoteMaster.pm
M lib/pgBackRest/Protocol/RemoteMinion.pm
M lib/pgBackRest/Restore.pm
M lib/pgBackRest/RestoreFile.pm
M lib/pgBackRest/Version.pm
M test/expect/backup-archive-get-001.log
M test/expect/backup-archive-get-002.log
M test/expect/backup-archive-get-003.log
M test/expect/backup-archive-get-004.log
M test/expect/backup-archive-get-005.log
M test/expect/backup-archive-get-006.log
M test/expect/backup-archive-get-007.log
M test/expect/backup-archive-get-008.log
M test/expect/backup-archive-push-001.log
M test/expect/backup-archive-push-002.log
M test/expect/backup-archive-push-003.log
M test/expect/backup-archive-push-004.log
M test/expect/backup-archive-push-005.log
M test/expect/backup-archive-push-006.log
M test/expect/backup-archive-push-007.log
M test/expect/backup-archive-push-008.log
M test/expect/backup-archive-stop-001.log
M test/expect/backup-archive-stop-002.log
M test/expect/backup-archive-stop-003.log
M test/expect/backup-archive-stop-004.log
M test/expect/backup-archive-stop-005.log
M test/expect/backup-archive-stop-006.log
M test/expect/backup-synthetic-001.log
M test/expect/backup-synthetic-002.log
M test/expect/backup-synthetic-003.log
M test/expect/backup-synthetic-004.log
M test/expect/backup-synthetic-005.log
M test/expect/backup-synthetic-006.log
M test/expect/backup-synthetic-007.log
M test/expect/backup-synthetic-008.log
M test/lib/pgBackRestTest/Backup/Common/ExpireCommonTest.pm
M test/lib/pgBackRestTest/Common/ExecuteTest.pm
M test/lib/pgBackRestTest/Common/LogTest.pm