pgBackRest 2.2 commit log

v2.02: Parallel Asynchronous Archive Get and Configuration Includes

commit   : ee8aafb3cadd77682c97bcc74adb457e0723ebee    
  
author   : David Steele <[email protected]>    
date     : Sun, 6 May 2018 19:53:42 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 6 May 2018 19:53:42 -0400    

Click here for diff

Bug Fixes:  
  
* Fix directory syncs running recursively when only the specified directory should be synced. (Reported by Craig A. James.)  
* Fix archive-copy throwing "path not found" error for incr/diff backups. (Reported by yummyliu, Vitaliy Kukharik.)  
* Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. (Reported by Vitaliy Kukharik.)  
* Fix delta restore failing when a linked file is missing.  
* Fix rendering of key/value and list options in help. (Reported by Clinton Adams.)  
  
Features:  
  
* Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.  
* Add support for additional pgBackRest configuration files in the directory specified by the --config-include-path option. Add --config-path option for overriding the default base path of the --config and --config-include-path option. (Contributed by Cynthia Shang.)  
* Add repo-s3-token option to allow temporary credentials tokens to be configured. pgBackRest currently has no way to request new credentials so the entire command (e.g. backup, restore) must complete before the credentials expire. (Contributed by Yogesh Sharma.)  
  
Improvements:  
  
* Update the archive-push-queue-max, manifest-save-threshold, and buffer-size options to accept values in KB, MB, GB, TB, or PB where the multiplier is a power of 1024. (Contributed by Cynthia Shang.)  
* Make backup/restore path sync more efficient. Scanning the entire directory can be very expensive if there are a lot of small tables. The backup manifest contains the path list so use it to perform syncs instead of scanning the backup/restore path.  
* Show command parameters as well as command options in initial info log message.  
* Rename archive-queue-max option to archive-push-queue-max to avoid confusion with the new archive-get-queue-max option. The old option name will continue to be accepted.  

M README.md
M doc/resource/exe.cache
M doc/xml/index.xml
M doc/xml/release.xml
M lib/pgBackRest/Version.pm
M libc/lib/pgBackRest/LibCAuto.pm
M src/version.h
M test/code-count/file-type.yaml

Add HARNESS_FORK for tests that require fork().

commit   : 4d6a51ac47e168811531cb4d388f0dde2b89cbcd    
  
author   : David Steele <[email protected]>    
date     : Sun, 6 May 2018 08:56:42 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 6 May 2018 08:56:42 -0400    

Click here for diff

A standard pattern for tests makes fork() easier to use and should help prevent some common mistakes.  

M doc/xml/release.xml
A test/src/common/harnessFork.h
M test/src/module/archive/getTest.c
M test/src/module/common/errorTest.c
M test/src/module/common/exitTest.c
M test/src/module/common/forkTest.c
M test/src/module/common/lockTest.c
M test/src/module/storage/storageTest.c

Enforce full branch coverage in C code.

commit   : 790f7c73128a4b3f0f98600aa7475d88f774fd1b    
  
author   : David Steele <[email protected]>    
date     : Sat, 5 May 2018 11:50:11 -0400    
  
committer: David Steele <[email protected]>    
date     : Sat, 5 May 2018 11:50:11 -0400    

Click here for diff

M doc/xml/release.xml
M test/lib/pgBackRestTest/Common/JobTest.pm
M test/src/module/archive/getTest.c
M test/test.pl

Full branch coverage for command/help/help, common/error, common/ini, and common/log modules.

commit   : 0a860e0b6049bd6922a1583ede6748b46cfb9d23    
  
author   : David Steele <[email protected]>    
date     : Sat, 5 May 2018 09:38:09 -0400    
  
committer: David Steele <[email protected]>    
date     : Sat, 5 May 2018 09:38:09 -0400    

Click here for diff

M doc/xml/release.xml
M src/command/help/help.c
M src/common/error.c
M src/common/ini.c
M src/common/log.c
M test/define.yaml
M test/src/module/archive/pushTest.c
M test/src/module/common/errorTest.c
M test/src/module/common/iniTest.c
M test/src/module/common/logTest.c
M test/src/module/help/helpTest.c

Full branch coverage for config module.

commit   : 90aadc6534be12e8a49d4da5ff231a1f439e378c    
  
author   : David Steele <[email protected]>    
date     : Fri, 4 May 2018 12:49:25 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 4 May 2018 12:49:25 -0400    

Click here for diff

M src/command/help/help.c
M src/common/log.c
M src/common/log.h
M src/config/define.c
M src/config/define.h
M src/config/load.c
M src/config/load.h
M src/config/parse.c
M test/define.yaml
M test/src/module/config/defineTest.c
M test/src/module/config/loadTest.c

Improve module branch coverage.

commit   : 6c3b40152fa88e101564867ed55338bb63eff626    
  
author   : David Steele <[email protected]>    
date     : Fri, 4 May 2018 09:21:32 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 4 May 2018 09:21:32 -0400    

Click here for diff

Add full branch coverage for cipher/block, command/command, common/wait, and postgres/pageChecksum modules.  

M src/cipher/block.c
M src/postgres/pageChecksum.c
M src/postgres/type.h
M test/src/module/command/commandTest.c
M test/src/module/common/waitTest.c
M test/src/module/postgres/pageChecksumTest.c

Add _FMT variants for all THROW macros so format types are checked by the compiler.

commit   : c3a8fbe706ed31e2414fa454d6090d0346f34f4a    
  
author   : David Steele <[email protected]>    
date     : Thu, 3 May 2018 11:24:29 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 3 May 2018 11:24:29 -0400    

Click here for diff

M doc/xml/release.xml
M src/cipher/block.c
M src/command/archive/common.c
M src/command/archive/push/push.c
M src/command/help/help.c
M src/common/assert.h
M src/common/encode.c
M src/common/encode/base64.c
M src/common/error.c
M src/common/error.h
M src/common/ini.c
M src/common/io/handle.c
M src/common/lock.c
M src/common/log.c
M src/common/memContext.c
M src/common/type/list.c
M src/common/type/string.c
M src/common/type/variant.c
M src/config/config.c
M src/config/define.c
M src/config/load.c
M src/config/parse.c
M src/postgres/info.c
M src/postgres/pageChecksum.c
M src/storage/driver/posix/driver.c
M src/storage/driver/posix/driverFile.c
M src/storage/driver/posix/driverRead.c
M src/storage/driver/posix/driverWrite.c
M src/storage/helper.c
M src/storage/storage.c
M test/lib/pgBackRestTest/Module/Common/CommonEncodePerlTest.pm
M test/src/common/harnessTest.h
M test/src/common/logTest.c
M test/src/module/archive/getTest.c
M test/src/module/common/encodeTest.c
M test/src/module/common/errorTest.c
M test/src/module/common/forkTest.c
M test/src/module/config/configTest.c
M test/src/module/config/defineTest.c
M test/src/module/storage/storageTest.c

Add repo-s3-token option to allow temporary credentials tokens to be configured.

commit   : 6a40c916d4307c2de79de250ed444237d47e6b7a    
  
author   : Yogesh Sharma <[email protected]>    
date     : Wed, 2 May 2018 14:06:40 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 2 May 2018 14:06:40 -0400    

Click here for diff

pgBackRest currently has no way to request new credentials so the entire command (e.g. backup, restore) must complete before the credentials expire.  
  
Contributed by Yogesh Sharma.  

M build/lib/pgBackRestBuild/Config/Data.pm
M doc/xml/reference.xml
M doc/xml/release.xml
M lib/pgBackRest/Protocol/Storage/Helper.pm
M lib/pgBackRest/Storage/S3/Auth.pm
M lib/pgBackRest/Storage/S3/Request.pm
M libc/lib/pgBackRest/LibCAuto.pm
M libc/xs/config/config.auto.xsh
M src/config/config.auto.c
M src/config/config.auto.h
M src/config/define.auto.c
M src/config/define.auto.h
M src/config/parse.auto.c
M test/expect/mock-stanza-003.log
M test/lib/pgBackRestTest/Module/Storage/StorageS3AuthPerlTest.pm
M test/lib/pgBackRestTest/Module/Storage/StorageS3CertPerlTest.pm
M test/src/module/help/helpTest.c

Fix failure in manifest build when two or more files in PGDATA are linked to the same directory.

commit   : 5e090ba30501b76717dcb034420aba06847955f2    
  
author   : David Steele <[email protected]>    
date     : Wed, 2 May 2018 12:19:54 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 2 May 2018 12:19:54 -0400    

Click here for diff

Reported by Vitaliy Kukharik.  

M doc/xml/release.xml
M lib/pgBackRest/Manifest.pm
M test/expect/mock-all-001.log
M test/expect/mock-all-002.log
M test/expect/mock-all-003.log
M test/lib/pgBackRestTest/Module/Mock/MockAllTest.pm

Fix delta restore failing when a linked file was missing.

commit   : 6a01db2b3297fc103577b8bd25da011f09fc8a17    
  
author   : David Steele <[email protected]>    
date     : Wed, 2 May 2018 12:06:12 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 2 May 2018 12:06:12 -0400    

Click here for diff

M doc/xml/release.xml
M lib/pgBackRest/RestoreFile.pm

Add example of command section usage to archiving configuration.

commit   : 9d9fad88a25b0f4bafcae1f2e9b1cf81d00b8d9e    
  
author   : David Steele <[email protected]>    
date     : Wed, 2 May 2018 07:31:44 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 2 May 2018 07:31:44 -0400    

Click here for diff

Suggested by Christophe Courtois.  

M doc/xml/release.xml
M doc/xml/user-guide.xml

Fix rendering of key/value and list options in help.

commit   : fd1984239e5d60293b86419d77bcb1c990dbbc7e    
  
author   : David Steele <[email protected]>    
date     : Tue, 1 May 2018 14:07:08 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 1 May 2018 14:07:08 -0400    

Click here for diff

Reported by Clinton Adams.  

M doc/xml/release.xml
M src/command/help/help.c
M test/src/module/help/helpTest.c

Update help test to use restore command to provide better coverage.

commit   : 1a1ed8d6b9973e069325b1dee7b54c9bbc2740d1    
  
author   : David Steele <[email protected]>    
date     : Tue, 1 May 2018 13:20:48 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 1 May 2018 13:20:48 -0400    

Click here for diff

M test/src/module/help/helpTest.c

Fix archive-copy throwing "path not found" error for incr/diff backups.

commit   : 8dbbf2de929813d5ed2068c13b8cc5b3c3cfafc2    
  
author   : David Steele <[email protected]>    
date     : Tue, 1 May 2018 12:37:06 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 1 May 2018 12:37:06 -0400    

Click here for diff

Reported by yummyliu, Vitaliy Kukharik.  

M doc/xml/release.xml
M lib/pgBackRest/Backup/Backup.pm
M test/expect/real-all-001.log
M test/expect/real-all-002.log
M test/expect/real-all-003.log
M test/expect/real-all-004.log
M test/expect/real-all-005.log
M test/expect/real-all-006.log
M test/lib/pgBackRestTest/Module/Real/RealAllTest.pm

Use path list in the backup manifest to do restore path syncs.

commit   : 71ba08f579be228a8975c0a59bd485e5c11bebe5    
  
author   : David Steele <[email protected]>    
date     : Tue, 1 May 2018 11:05:37 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 1 May 2018 11:05:37 -0400    

Click here for diff

Remove recursive path sync functionality since it is no longer used.  

M doc/xml/release.xml
M lib/pgBackRest/Backup/Backup.pm
M lib/pgBackRest/Restore.pm
M lib/pgBackRest/Storage/Local.pm
M lib/pgBackRest/Storage/Posix/Driver.pm
M test/expect/mock-all-001.log
M test/expect/mock-all-002.log
M test/lib/pgBackRestTest/Module/Mock/MockAllTest.pm

Add asynchronous, parallel archive-get.

commit   : 54dd6f3ed4fcfeabe0254fdb9d88f560efd5f8ca    
  
author   : David Steele <[email protected]>    
date     : Mon, 30 Apr 2018 17:27:39 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 30 Apr 2018 17:27:39 -0400    

Click here for diff

This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.  

M build/lib/pgBackRestBuild/Config/Data.pm
M doc/xml/reference.xml
M doc/xml/release.xml
M doc/xml/user-guide.xml
A lib/pgBackRest/Archive/Get/Async.pm
M lib/pgBackRest/Archive/Get/File.pm
M lib/pgBackRest/Archive/Get/Get.pm
M lib/pgBackRest/Archive/Push/Async.pm
M lib/pgBackRest/Archive/Push/Push.pm
M lib/pgBackRest/Main.pm
M lib/pgBackRest/Protocol/Helper.pm
M lib/pgBackRest/Protocol/Local/Minion.pm
M lib/pgBackRest/Storage/Helper.pm
M libc/lib/pgBackRest/LibCAuto.pm
M libc/xs/config/config.auto.xsh
M src/Makefile
M src/command/archive/common.c
M src/command/archive/common.h
A src/command/archive/get/get.c
A src/command/archive/get/get.h
M src/command/archive/push/push.c
M src/config/config.auto.c
M src/config/config.auto.h
M src/config/config.c
M src/config/define.auto.c
M src/config/define.auto.h
M src/config/parse.auto.c
M src/main.c
M src/storage/helper.c
M src/storage/helper.h
M test/define.yaml
M test/expect/mock-archive-001.log
M test/expect/mock-archive-002.log
M test/expect/mock-archive-003.log
M test/expect/mock-stanza-001.log
M test/expect/mock-stanza-002.log
M test/expect/mock-stanza-003.log
M test/lib/pgBackRestTest/Env/HostEnvTest.pm
M test/lib/pgBackRestTest/Module/Archive/ArchiveGetPerlTest.pm
M test/lib/pgBackRestTest/Module/Mock/MockArchiveTest.pm
M test/src/common/harnessConfig.c
M test/src/module/archive/commonTest.c
A test/src/module/archive/getTest.c
M test/src/module/help/helpTest.c
M test/src/module/storage/helperTest.c

Update docs with 32-bit support and caveats.

commit   : c48b0a2a1e40764a33b42485a7edf21a544b0db1    
  
author   : David Steele <[email protected]>    
date     : Mon, 30 Apr 2018 08:13:56 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 30 Apr 2018 08:13:56 -0400    

Click here for diff

32-bit support was added in v1.26.  
  
Reported by Viorel Tabara.  

M doc/xml/release.xml
M doc/xml/user-guide.xml

Add walSegmentNext() and walSegmentRange().

commit   : 321a28f6b0c6c522b8b8120e9fccfec09de4c6f4    
  
author   : David Steele <[email protected]>    
date     : Sun, 29 Apr 2018 11:47:50 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 29 Apr 2018 11:47:50 -0400    

Click here for diff

M doc/xml/release.xml
M src/command/archive/common.c
M src/command/archive/common.h
M test/define.yaml
M test/src/module/archive/commonTest.c

Add pgControlInfo() to read pg_control and determine the PostgreSQL version.

commit   : be02c6750376e9a0312b7c707c5c96025eb1cfa6    
  
author   : David Steele <[email protected]>    
date     : Sun, 29 Apr 2018 11:20:51 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 29 Apr 2018 11:20:51 -0400    

Click here for diff

M doc/xml/release.xml
A src/postgres/info.c
A src/postgres/info.h
A src/postgres/type.h
A src/postgres/version.h
M test/define.yaml
A test/src/module/postgres/infoTest.c

Add storageInfo() and track size in read objects.

commit   : 8c6e2bdbc7ac2b0e887d9d30931a532141974821    
  
author   : David Steele <[email protected]>    
date     : Sun, 29 Apr 2018 11:02:21 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 29 Apr 2018 11:02:21 -0400    

Click here for diff

M build/error.yaml
M lib/pgBackRest/Common/ExceptionAuto.pm
M src/common/error.auto.c
M src/common/error.auto.h
M src/storage/driver/posix/driver.c
M src/storage/driver/posix/driver.h
M src/storage/driver/posix/driverRead.c
M src/storage/driver/posix/driverRead.h
M src/storage/fileRead.c
M src/storage/fileRead.h
A src/storage/info.h
M src/storage/storage.c
M src/storage/storage.h
M test/define.yaml
M test/src/module/storage/fileTest.c
M test/src/module/storage/storageTest.c

Add bufNewC() and only reallocate buffer when the size has changed.

commit   : d74c167420b1c2eba74b29cf13b87f1143951b19    
  
author   : David Steele <[email protected]>    
date     : Sun, 29 Apr 2018 10:41:05 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 29 Apr 2018 10:41:05 -0400    

Click here for diff

M doc/xml/release.xml
M src/common/type/buffer.c
M src/common/type/buffer.h
M test/src/module/common/typeBufferTest.c

Add strLstExists(), strLstExistsZ(), strSub(), and strSubN() to String and StringList objects.

commit   : d44848baa0cc6583208d1d5ddb2db8cf69b866f3    
  
author   : David Steele <[email protected]>    
date     : Sun, 29 Apr 2018 10:32:46 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 29 Apr 2018 10:32:46 -0400    

Click here for diff

M doc/xml/release.xml
M src/common/type/string.c
M src/common/type/string.h
M src/common/type/stringList.c
M src/common/type/stringList.h
M test/define.yaml
M test/src/module/common/typeStringListTest.c
M test/src/module/common/typeStringTest.c

Refactor archive common functions in preparation for parallel async archive-get.

commit   : 89d3476e325a94cb75457354e26b1ab4f5bcc8b0    
  
author   : David Steele <[email protected]>    
date     : Sun, 29 Apr 2018 10:16:59 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 29 Apr 2018 10:16:59 -0400    

Click here for diff

M lib/pgBackRest/Archive/Base.pm
M lib/pgBackRest/Archive/Common.pm
A lib/pgBackRest/Archive/Get/File.pm
M lib/pgBackRest/Archive/Get/Get.pm
M lib/pgBackRest/Archive/Push/Async.pm
M lib/pgBackRest/Archive/Push/File.pm
M lib/pgBackRest/Archive/Push/Push.pm
M lib/pgBackRest/Check/Check.pm
M lib/pgBackRest/Protocol/Helper.pm
M lib/pgBackRest/Protocol/Remote/Minion.pm
M src/Makefile
A src/command/archive/common.c
A src/command/archive/common.h
M src/command/archive/push/push.c
M test/define.yaml
M test/expect/mock-archive-001.log
M test/expect/mock-archive-002.log
M test/expect/mock-stanza-001.log
M test/expect/mock-stanza-002.log
M test/expect/mock-stanza-003.log
M test/lib/pgBackRestTest/Module/Archive/ArchiveCommonPerlTest.pm
M test/lib/pgBackRestTest/Module/Archive/ArchiveGetPerlTest.pm
M test/lib/pgBackRestTest/Module/Archive/ArchivePushPerlTest.pm
A test/src/module/archive/commonTest.c
M test/src/module/archive/pushTest.c

Update the archive-push-queue-max, manifest-save-threshold, and buffer-size options to accept values in KB, MB, GB, TB, or PB where the multiplier is a power of 1024.

commit   : 721432f6ae55c2a06d645629703c5103f8d89190    
  
author   : Cynthia Shang <[email protected]>    
date     : Wed, 25 Apr 2018 15:46:05 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 25 Apr 2018 15:46:05 -0400    

Click here for diff

Contributed by Cynthia Shang.  

M build/lib/pgBackRestBuild/Config/Data.pm
M doc/xml/reference.xml
M doc/xml/release.xml
M libc/lib/pgBackRest/LibCAuto.pm
M libc/xs/config/define.auto.xsh
M src/config/config.c
M src/config/define.auto.c
M src/config/define.auto.h
M src/config/parse.c
M src/perl/config.c
M test/define.yaml
M test/src/module/config/parseTest.c
M test/src/module/help/helpTest.c

Add strUpper, strLower, strChr, and strTrunc to String object.

commit   : 8eabc52f3b5872c898cc14b2691a01a2559ad6bd    
  
author   : Cynthia Shang <[email protected]>    
date     : Wed, 25 Apr 2018 15:16:42 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 25 Apr 2018 15:16:42 -0400    

Click here for diff

Contributed by Cynthia Shang.  

M doc/xml/release.xml
M src/common/type/string.c
M src/common/type/string.h
M test/define.yaml
M test/src/module/common/typeStringTest.c

Divide tests into three types (unit, integration, performance).

commit   : 88a633da17d53261e0131e2d77209f22403112d6    
  
author   : David Steele <[email protected]>    
date     : Tue, 24 Apr 2018 09:12:25 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 24 Apr 2018 09:12:25 -0400    

Click here for diff

Many options that were set per test can instead be inferred from the types, i.e. container, c, expect, and individual.  
  
Also finish renaming Perl unit tests with the -perl suffix.  

M doc/xml/release.xml
M test/define.yaml
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/lib/pgBackRestTest/Common/ListTest.pm
R099 test/lib/pgBackRestTest/Module/Archive/ArchiveInfoUnitTest.pm test/lib/pgBackRestTest/Module/Archive/ArchiveInfoUnitPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Backup/BackupInfoUnitTest.pm test/lib/pgBackRestTest/Module/Backup/BackupInfoUnitPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Backup/BackupUnitTest.pm test/lib/pgBackRestTest/Module/Backup/BackupUnitPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Common/CommonHttpClientTest.pm test/lib/pgBackRestTest/Module/Common/CommonHttpClientPerlTest.pm
R098 test/lib/pgBackRestTest/Module/Common/CommonIoBufferedTest.pm test/lib/pgBackRestTest/Module/Common/CommonIoBufferedPerlTest.pm
R096 test/lib/pgBackRestTest/Module/Common/CommonIoProcessTest.pm test/lib/pgBackRestTest/Module/Common/CommonIoProcessPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Info/InfoUnitTest.pm test/lib/pgBackRestTest/Module/Info/InfoUnitPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Manifest/ManifestAllTest.pm test/lib/pgBackRestTest/Module/Manifest/ManifestAllPerlTest.pm
R097 test/lib/pgBackRestTest/Module/Protocol/ProtocolCommonMinionTest.pm test/lib/pgBackRestTest/Module/Protocol/ProtocolCommonMinionPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Protocol/ProtocolHelperTest.pm test/lib/pgBackRestTest/Module/Protocol/ProtocolHelperPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Stanza/StanzaAllTest.pm test/lib/pgBackRestTest/Module/Stanza/StanzaAllPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Storage/StorageFilterCipherBlockTest.pm test/lib/pgBackRestTest/Module/Storage/StorageFilterCipherBlockPerlTest.pm
R098 test/lib/pgBackRestTest/Module/Storage/StorageFilterGzipTest.pm test/lib/pgBackRestTest/Module/Storage/StorageFilterGzipPerlTest.pm
R097 test/lib/pgBackRestTest/Module/Storage/StorageFilterShaTest.pm test/lib/pgBackRestTest/Module/Storage/StorageFilterShaPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Storage/StorageLocalTest.pm test/lib/pgBackRestTest/Module/Storage/StorageLocalPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Storage/StoragePosixTest.pm test/lib/pgBackRestTest/Module/Storage/StoragePosixPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Storage/StorageS3AuthTest.pm test/lib/pgBackRestTest/Module/Storage/StorageS3AuthPerlTest.pm
R098 test/lib/pgBackRestTest/Module/Storage/StorageS3CertTest.pm test/lib/pgBackRestTest/Module/Storage/StorageS3CertPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Storage/StorageS3Test.pm test/lib/pgBackRestTest/Module/Storage/StorageS3PerlTest.pm
R099 test/lib/pgBackRestTest/Module/Storage/StorageS3RequestTest.pm test/lib/pgBackRestTest/Module/Storage/StorageS3RequestPerlTest.pm

Storage object improvements.

commit   : bb8c315cadc6329a03cd975a6c865b1ce29d751c    
  
author   : David Steele <[email protected]>    
date     : Mon, 23 Apr 2018 17:26:27 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 23 Apr 2018 17:26:27 -0400    

Click here for diff

* Add storageCopy(), storageMove(), and storagePathSync().  
* Separate StorageFile object into separate read and write objects.  
* Abstract out Posix file read/write objects.  

M build/error.yaml
M doc/xml/release.xml
M lib/pgBackRest/Common/ExceptionAuto.pm
M libc/LibC.xs
M libc/Makefile.PL
M src/Makefile
M src/command/archive/push/push.c
M src/common/error.auto.c
M src/common/error.auto.h
M src/common/ini.c
M src/config/parse.c
R063 src/storage/driver/posix.c src/storage/driver/posix/driver.c
R069 src/storage/driver/posix.h src/storage/driver/posix/driver.h
A src/storage/driver/posix/driverFile.c
A src/storage/driver/posix/driverFile.h
A src/storage/driver/posix/driverRead.c
A src/storage/driver/posix/driverRead.h
A src/storage/driver/posix/driverWrite.c
A src/storage/driver/posix/driverWrite.h
D src/storage/file.h
A src/storage/fileRead.c
A src/storage/fileRead.h
A src/storage/fileWrite.c
R054 src/storage/file.c src/storage/fileWrite.h
M src/storage/storage.c
M src/storage/storage.h
M test/define.yaml
M test/src/common/logTest.c
M test/src/module/archive/pushTest.c
M test/src/module/common/iniTest.c
M test/src/module/common/logTest.c
M test/src/module/config/parseTest.c
M test/src/module/help/helpTest.c
M test/src/module/storage/fileTest.c
M test/src/module/storage/helperTest.c
M test/src/module/storage/storageTest.c

Add THROWP_* macro variants for error handling.

commit   : 02cc8ccbd45221ef03f30cd6b068fc7675011592    
  
author   : David Steele <[email protected]>    
date     : Mon, 23 Apr 2018 10:24:04 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 23 Apr 2018 10:24:04 -0400    

Click here for diff

These macros allow an ErrorType pointer to be passed and are required for functions that may return different errors based on a parameter.  

M doc/xml/release.xml
M src/common/error.h

Add bufEq and bufCat to Buffer object.

commit   : 18882cb8822ab8af04ff57853c2bc35145cb7b65    
  
author   : David Steele <[email protected]>    
date     : Mon, 23 Apr 2018 10:19:45 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 23 Apr 2018 10:19:45 -0400    

Click here for diff

M doc/xml/release.xml
M src/common/type/buffer.c
M src/common/type/buffer.h
M test/define.yaml
M test/src/module/common/typeBufferTest.c

Add TEST_ERROR_FMT macro to simplify testing of formatted error messages.

commit   : f146502eae44d7366d79f3d73f695209d3189040    
  
author   : David Steele <[email protected]>    
date     : Mon, 23 Apr 2018 10:10:55 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 23 Apr 2018 10:10:55 -0400    

Click here for diff

M doc/xml/release.xml
M test/src/common/harnessTest.h

Add support for additional pgBackRest configuration files.

commit   : 27aa5f756e32e8e1713588695b86451fde2e5b02    
  
author   : Cynthia Shang <[email protected]>    
date     : Thu, 19 Apr 2018 11:09:39 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 19 Apr 2018 11:09:39 -0400    

Click here for diff

Configuration files are loaded from the directory specified by the --config-include-path option.  
  
Add --config-path option for overriding the default base path of the --config and --config-include-path option.  
  
Contributed by Cynthia Shang.  

M build/lib/pgBackRestBuild/Config/Data.pm
M doc/xml/reference.xml
M doc/xml/release.xml
M doc/xml/user-guide.xml
M lib/pgBackRest/Protocol/Helper.pm
M libc/lib/pgBackRest/LibCAuto.pm
M libc/xs/config/config.auto.xsh
M src/config/config.auto.c
M src/config/config.auto.h
M src/config/config.c
M src/config/define.auto.c
M src/config/define.auto.h
M src/config/parse.auto.c
M src/config/parse.c
M test/define.yaml
M test/expect/mock-all-001.log
M test/lib/pgBackRestTest/Module/Protocol/ProtocolHelperTest.pm
M test/src/module/config/parseTest.c
M test/src/module/help/helpTest.c

Rename archive-queue-max option to archive-push-queue-max.

commit   : 79b7552940971ab73cc63676655c6614fd535e21    
  
author   : David Steele <[email protected]>    
date     : Wed, 18 Apr 2018 19:37:35 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 18 Apr 2018 19:37:35 -0400    

Click here for diff

This avoids confusion with the archive-get-queue-max option. The old option name will continue to be accepted.  

M build/lib/pgBackRestBuild/Config/Data.pm
M doc/xml/reference.xml
M doc/xml/release.xml
M doc/xml/user-guide.xml
M lib/pgBackRest/Archive/Push/Async.pm
M lib/pgBackRest/Archive/Push/Push.pm
M libc/lib/pgBackRest/LibCAuto.pm
M libc/xs/config/config.auto.xsh
M src/config/config.auto.c
M src/config/config.auto.h
M src/config/define.auto.c
M src/config/define.auto.h
M src/config/parse.auto.c
M test/expect/mock-archive-stop-001.log
M test/expect/mock-archive-stop-002.log
M test/expect/mock-archive-stop-003.log
M test/expect/mock-archive-stop-004.log
M test/expect/mock-archive-stop-005.log
M test/expect/mock-archive-stop-006.log
M test/expect/mock-archive-stop-007.log
M test/expect/mock-stanza-001.log
M test/expect/mock-stanza-002.log
M test/expect/mock-stanza-003.log
M test/lib/pgBackRestTest/Env/Host/HostDbCommonTest.pm
M test/lib/pgBackRestTest/Module/Archive/ArchivePushPerlTest.pm
M test/lib/pgBackRestTest/Module/Mock/MockArchiveStopTest.pm
M test/src/module/config/configTest.c
M test/src/module/config/defineTest.c
M test/src/module/config/parseTest.c
M test/src/module/help/helpTest.c
M test/src/module/perl/configTest.c

Don't copy CFGDEF_NAME_ALT or CFGDEF_INHERIT when processing config option inheritance.

commit   : ab26c765d454cd8f757ccc7ada593746dbddae0d    
  
author   : David Steele <[email protected]>    
date     : Wed, 18 Apr 2018 18:57:45 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 18 Apr 2018 18:57:45 -0400    

Click here for diff

M build/lib/pgBackRestBuild/Config/Data.pm
M doc/xml/release.xml

Try tweaking time sync settings to prevent clock drift rather than restarting VBoxService on every test run.

commit   : ea4046a607cddef588b5eb061f1df1e602a497bf    
  
author   : David Steele <[email protected]>    
date     : Wed, 18 Apr 2018 18:47:43 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 18 Apr 2018 18:47:43 -0400    

Click here for diff

M doc/xml/release.xml
M test/Vagrantfile
M test/test.pl

Fix document update missed in e9790fd3.

commit   : b3e668421b03b2869c0dac1be6f22a9faf753341    
  
author   : David Steele <[email protected]>    
date     : Tue, 17 Apr 2018 20:25:42 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 17 Apr 2018 20:25:42 -0400    

Click here for diff

M src/config/define.auto.c

Show command parameters as well as command options in initial info message.

commit   : 0381945caa198f833adde7806af2368c2627678a    
  
author   : David Steele <[email protected]>    
date     : Tue, 17 Apr 2018 18:47:14 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 17 Apr 2018 18:47:14 -0400    

Click here for diff

M doc/xml/release.xml
M src/command/command.c
M test/expect/mock-archive-001.log
M test/expect/mock-archive-002.log
M test/expect/mock-stanza-001.log
M test/expect/mock-stanza-002.log
M test/expect/mock-stanza-003.log
M test/src/module/command/commandTest.c

Update out-of-date description for the spool-path option.

commit   : e9790fd3b13f87a7f4726471019ead36d5b7b08d    
  
author   : David Steele <[email protected]>    
date     : Tue, 17 Apr 2018 17:38:25 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 17 Apr 2018 17:38:25 -0400    

Click here for diff

M doc/xml/reference.xml
M doc/xml/release.xml
M src/config/define.auto.c
M test/src/module/help/helpTest.c

Rename Perl tests so they don't conflict with their C counterparts.

commit   : cf0ec44ededb357c4fe2783814a2575119a4c498    
  
author   : David Steele <[email protected]>    
date     : Tue, 17 Apr 2018 12:03:03 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 17 Apr 2018 12:03:03 -0400    

Click here for diff

M doc/xml/release.xml
M test/define.yaml
R099 test/lib/pgBackRestTest/Module/Archive/ArchiveCommonTest.pm test/lib/pgBackRestTest/Module/Archive/ArchiveCommonPerlTest.pm
R099 test/lib/pgBackRestTest/Module/Archive/ArchiveGetTest.pm test/lib/pgBackRestTest/Module/Archive/ArchiveGetPerlTest.pm

Generate code counts for all source files.

commit   : dce11ce3cf0473b127e56c21436dd4c4159d8646    
  
author   : David Steele <[email protected]>    
date     : Mon, 16 Apr 2018 15:52:17 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 16 Apr 2018 15:52:17 -0400    

Click here for diff

The source files are also classified by type and purpose.  

M doc/xml/release.xml
M test/Vagrantfile
A test/code-count/code.def
A test/code-count/file-type.yaml
A test/lib/pgBackRestTest/Common/CodeCountTest.pm
M test/test.pl

Use pip 9.03 in test VMs.

commit   : 3e229774840f82e4488d0077b07ae80971f7c385    
  
author   : David Steele <[email protected]>    
date     : Mon, 16 Apr 2018 15:25:50 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 16 Apr 2018 15:25:50 -0400    

Click here for diff

pip 10 drops support for Python 2.6 which is still used by the older test VMs.  

M doc/xml/release.xml
M test/lib/pgBackRestTest/Common/ContainerTest.pm

Split cfgLoad() into multiple functions to make testing easier.

commit   : 2a4ff2f904b411f623239f02f54cdb2622017298    
  
author   : David Steele <[email protected]>    
date     : Fri, 13 Apr 2018 16:05:52 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 13 Apr 2018 16:05:52 -0400    

Click here for diff

Mainly this helps with unit tests that need to do log expect testing. Add harnessCfgLoad() test function, which allows a new config to be loaded for unit testing without resetting log functions, opening a log file, or taking locks.  

M doc/xml/release.xml
M libc/LibC.xs
M libc/xs/config/configTest.xs
M src/config/load.c
M src/config/load.h
M test/lib/pgBackRestTest/Env/ConfigEnvTest.pm
A test/src/common/harnessConfig.c
A test/src/common/harnessConfig.h
M test/src/common/logTest.c
M test/src/module/archive/pushTest.c
M test/src/module/config/loadTest.c

Add monitoring examples using PostgreSQL and jq.

commit   : 49fc737cd0fc053339f26b4e3b01d2b598124bb4    
  
author   : David Steele <[email protected]>    
date     : Fri, 13 Apr 2018 14:31:33 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 13 Apr 2018 14:31:33 -0400    

Click here for diff

Suggested by Stephen Frost, Brian Faherty.  

A doc/example/pgsql-pgbackrest-info.sql
A doc/example/pgsql-pgbackrest-query.sql
M doc/xml/release.xml
M doc/xml/user-guide.xml

Add logic to find the real oid of the test1 database during restore testing.

commit   : d924f3d50a1be296a65096b0b1014ade14c2dd59    
  
author   : David Steele <[email protected]>    
date     : Fri, 13 Apr 2018 12:00:22 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 13 Apr 2018 12:00:22 -0400    

Click here for diff

M doc/xml/release.xml
M doc/xml/user-guide.xml

Move async forking and more error handling to C.

commit   : f0250dab4b7fba2eda57fc0a483352f7cd70d0ab    
  
author   : David Steele <[email protected]>    
date     : Thu, 12 Apr 2018 20:42:26 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 12 Apr 2018 20:42:26 -0400    

Click here for diff

The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.  

M doc/xml/release.xml
M doc/xml/user-guide.xml
M lib/pgBackRest/Archive/Push/Async.pm
M lib/pgBackRest/Archive/Push/Push.pm
M lib/pgBackRest/Backup/Backup.pm
M lib/pgBackRest/Check/Check.pm
D lib/pgBackRest/Common/Exit.pm
M lib/pgBackRest/Config/Config.pm
M lib/pgBackRest/Info.pm
M lib/pgBackRest/Main.pm
M lib/pgBackRest/Protocol/Base/Minion.pm
M lib/pgBackRest/Stanza.pm
M src/Makefile
M src/command/archive/push/push.c
M src/command/command.c
M src/command/command.h
M src/common/exit.c
M src/common/exit.h
A src/common/fork.c
A src/common/fork.h
M src/common/lock.c
M src/common/lock.h
M src/common/log.h
M src/config/load.c
M src/config/load.h
M src/main.c
M src/perl/config.h
M src/perl/exec.c
M src/perl/exec.h
M test/define.yaml
M test/expect/mock-all-001.log
M test/expect/mock-all-002.log
M test/expect/mock-all-003.log
M test/expect/mock-archive-001.log
M test/expect/mock-archive-002.log
M test/expect/mock-stanza-001.log
M test/expect/mock-stanza-002.log
M test/expect/mock-stanza-003.log
M test/lib/pgBackRestTest/Common/JobTest.pm
M test/lib/pgBackRestTest/Module/Archive/ArchivePushPerlTest.pm
M test/lib/pgBackRestTest/Module/Info/InfoUnitTest.pm
M test/lib/pgBackRestTest/Module/Stanza/StanzaAllTest.pm
M test/src/module/archive/pushTest.c
M test/src/module/command/commandTest.c
M test/src/module/common/exitTest.c
A test/src/module/common/forkTest.c
M test/src/module/common/lockTest.c
M test/src/module/perl/execTest.c
M test/test.pl

Improved lock implementation written in C.

commit   : 6fd0c3dcaa68e168b05364c699f6c1a6f87f1de0    
  
author   : David Steele <[email protected]>    
date     : Wed, 11 Apr 2018 09:36:12 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 11 Apr 2018 09:36:12 -0400    

Click here for diff

Now only two types of locks can be taken: archive and backup. Most commands use one or the other but the stanza-* commands acquire both locks. This provides better protection than the old command-based locking scheme.  

M build/lib/pgBackRestBuild/Config/Build.pm
M build/lib/pgBackRestBuild/Config/Data.pm
M doc/xml/release.xml
M lib/pgBackRest/Archive/Push/Async.pm
M lib/pgBackRest/Common/Exit.pm
M lib/pgBackRest/Common/Lock.pm
M lib/pgBackRest/Main.pm
M lib/pgBackRest/Protocol/Base/Minion.pm
M libc/LibC.xs
M libc/Makefile.PL
M libc/build/lib/pgBackRestLibC/Build.pm
M libc/lib/pgBackRest/LibCAuto.pm
A libc/xs/common/lock.xs
M src/Makefile
A src/common/lock.c
A src/common/lock.h
M src/config/config.auto.c
M src/config/config.c
M src/config/config.h
M src/config/load.c
M test/define.yaml
M test/expect/mock-all-001.log
M test/expect/mock-all-002.log
M test/expect/mock-archive-001.log
M test/expect/mock-archive-002.log
M test/expect/mock-stanza-001.log
M test/expect/mock-stanza-002.log
M test/expect/mock-stanza-003.log
M test/lib/pgBackRestTest/Env/ConfigEnvTest.pm
A test/src/module/common/lockTest.c
M test/src/module/config/configTest.c
M test/src/module/config/loadTest.c

Add storagePathRemove() and use it in the Perl Posix driver.

commit   : 4744eb93878da6b7aa2173be16928b081c865b1d    
  
author   : David Steele <[email protected]>    
date     : Wed, 11 Apr 2018 08:21:09 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 11 Apr 2018 08:21:09 -0400    

Click here for diff

This implementation should be faster because it does not stat each file.  It simply assumes that most directory entries are files so attempts an unlink() first.  If the entry is reported by error codes to be a directory then it attempts an rmdir().  

M doc/xml/release.xml
M lib/pgBackRest/Storage/Posix/Driver.pm
M libc/LibC.h
M libc/LibC.xs
M libc/build/lib/pgBackRestLibC/Build.pm
M libc/lib/pgBackRest/LibCAuto.pm
A libc/xs/storage/storage.xs
M src/storage/driver/posix.c
M src/storage/driver/posix.h
M src/storage/storage.c
M src/storage/storage.h
M test/define.yaml
M test/src/module/storage/storageTest.c

Skip version checks when testing in --dev mode.

commit   : c9ce20d41a0c6edc3087511eaff5e0a80e95c8da    
  
author   : David Steele <[email protected]>    
date     : Mon, 9 Apr 2018 16:46:36 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 9 Apr 2018 16:46:36 -0400    

Click here for diff

M doc/xml/release.xml
M test/test.pl

Move archive-stop and expire tests to the mock module.

commit   : 3bcf637aeeebfeb135e693fb0191dbb2c70f9420    
  
author   : David Steele <[email protected]>    
date     : Sun, 8 Apr 2018 12:57:02 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 8 Apr 2018 12:57:02 -0400    

Click here for diff

These are mock integration tests so they should be grouped with the other mock integration tests.  

M doc/xml/release.xml
M test/define.yaml
R100 test/expect/archive-stop-001.log test/expect/mock-archive-stop-001.log
R100 test/expect/archive-stop-002.log test/expect/mock-archive-stop-002.log
R100 test/expect/archive-stop-003.log test/expect/mock-archive-stop-003.log
R100 test/expect/archive-stop-004.log test/expect/mock-archive-stop-004.log
R100 test/expect/archive-stop-005.log test/expect/mock-archive-stop-005.log
R100 test/expect/archive-stop-006.log test/expect/mock-archive-stop-006.log
R100 test/expect/archive-stop-007.log test/expect/mock-archive-stop-007.log
R100 test/expect/expire-expire-001.log test/expect/mock-expire-001.log
R100 test/expect/expire-expire-002.log test/expect/mock-expire-002.log
R097 test/lib/pgBackRestTest/Module/Archive/ArchiveStopTest.pm test/lib/pgBackRestTest/Module/Mock/MockArchiveStopTest.pm
R099 test/lib/pgBackRestTest/Module/Expire/ExpireExpireTest.pm test/lib/pgBackRestTest/Module/Mock/MockExpireTest.pm

Move help/version integration tests to mock/all.

commit   : 3b661f4ad550cfce993afc5d767ed7f2ac4cea81    
  
author   : David Steele <[email protected]>    
date     : Sun, 8 Apr 2018 12:16:32 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 8 Apr 2018 12:16:32 -0400    

Click here for diff

Help and version are covered by unit tests, so we really just to need to make sure there is output when called from the command line.  

M doc/xml/release.xml
M test/define.yaml
M test/expect/mock-all-001.log
D test/expect/real-help-001.log
M test/lib/pgBackRestTest/Module/Mock/MockAllTest.pm
D test/lib/pgBackRestTest/Module/Real/RealHelpTest.pm

Move test definitions to test/define.yaml.

commit   : 741acfd7799b7b1b2afe850ccf078cf84a421353    
  
author   : David Steele <[email protected]>    
date     : Sun, 8 Apr 2018 10:19:24 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 8 Apr 2018 10:19:24 -0400    

Click here for diff

The location is better because it is no longer buried in the Perl test libs. Also, the data can be easily accessed from C.  

M .travis.yml
M build/lib/pgBackRestBuild/Error/Data.pm
M doc/xml/release.xml
M test/Vagrantfile
A test/define.yaml
M test/lib/pgBackRestTest/Common/CiTest.pm
M test/lib/pgBackRestTest/Common/ContainerTest.pm
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/test.pl

Allow -DDEBUG_UNIT to be suppressed to test how debug macros behave.

commit   : b77b4250964ca47ac622c83ae72bfc9d453ea5d7    
  
author   : David Steele <[email protected]>    
date     : Sat, 7 Apr 2018 19:44:00 -0400    
  
committer: David Steele <[email protected]>    
date     : Sat, 7 Apr 2018 19:44:00 -0400    

Click here for diff

M doc/xml/release.xml
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/lib/pgBackRestTest/Common/JobTest.pm
M test/lib/pgBackRestTest/Common/ListTest.pm
M test/src/common/harnessTest.h
M test/src/module/common/debugOffTest.c
M test/src/module/common/debugOnTest.c

Split debug and assert code into separate headers.

commit   : 82751b3b511e9a55724d3836b2454df8c97d8a25    
  
author   : David Steele <[email protected]>    
date     : Sat, 7 Apr 2018 13:12:35 -0400    
  
committer: David Steele <[email protected]>    
date     : Sat, 7 Apr 2018 13:12:35 -0400    

Click here for diff

Assert can be used earlier because it only depends on the error-handler and not logging.  

M doc/xml/release.xml
M src/command/command.c
A src/common/assert.h
M src/common/debug.h
M src/common/log.c
M src/common/memContext.c
M src/common/type/variant.c
M src/config/config.c
M src/config/parse.c
M src/storage/file.c
M src/storage/storage.c
M test/lib/pgBackRestTest/Common/DefineTest.pm
A test/src/module/common/assertOffTest.c
A test/src/module/common/assertOnTest.c
M test/src/module/common/debugOffTest.c
M test/src/module/common/debugOnTest.c
M test/src/module/postgres/pageChecksumTest.c

More C type cleanup.

commit   : e00f2dd4ad574c7680bce2126df4d320c9c8d6ac    
  
author   : David Steele <[email protected]>    
date     : Fri, 6 Apr 2018 21:46:45 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 6 Apr 2018 21:46:45 -0400    

Click here for diff

* Remove typec.h entirely.  
* Order all typdefs above local includes.  

M doc/xml/release.xml
M src/cipher/block.h
M src/cipher/cipher.h
M src/cipher/random.h
M src/command/archive/push/push.c
M src/command/help/help.c
M src/common/encode.c
M src/common/encode.h
M src/common/encode/base64.h
M src/common/error.c
M src/common/error.h
M src/common/log.c
M src/common/log.h
M src/common/memContext.h
M src/common/regExp.c
M src/common/regExp.h
M src/common/time.c
M src/common/time.h
M src/common/type/keyValue.c
M src/common/type/keyValue.h
M src/common/type/list.h
M src/common/type/string.c
M src/common/type/string.h
M src/common/type/stringList.h
M src/common/type/variant.c
M src/common/type/variant.h
M src/common/type/variantList.h
D src/common/typec.h
M src/common/wait.h
M src/config/define.c
M src/config/define.h
M src/perl/exec.c
M src/postgres/pageChecksum.h
M src/storage/file.h
M src/storage/storage.h
M test/lib/pgBackRestTest/Common/DefineTest.pm

Allow MemContext objects to be copied to a new parent.

commit   : f3b56cf8f328dc68dc4c978f0d82d3f5efbbe939    
  
author   : David Steele <[email protected]>    
date     : Fri, 6 Apr 2018 17:04:52 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 6 Apr 2018 17:04:52 -0400    

Click here for diff

This makes it easier to create objects and then copy them to another context when they are complete without having to worry about freeing them on error. Update List, StringList, and Buffer to allow moves. Update Ini and Storage to take advantage of moves.  

M doc/xml/release.xml
M src/common/ini.c
M src/common/memContext.c
M src/common/memContext.h
M src/common/type/buffer.c
M src/common/type/buffer.h
M src/common/type/list.c
M src/common/type/list.h
M src/common/type/stringList.c
M src/common/type/stringList.h
M src/storage/driver/posix.c
M src/storage/storage.c
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/src/module/common/memContextTest.c
M test/src/module/common/typeBufferTest.c
M test/src/module/common/typeListTest.c
M test/src/module/common/typeStringListTest.c

Cleanup C types.

commit   : b9f636cab470165ff0d69baa54efa5f8b2a79d43    
  
author   : David Steele <[email protected]>    
date     : Thu, 5 Apr 2018 22:38:11 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 5 Apr 2018 22:38:11 -0400    

Click here for diff

* Remove pgBackRest defined types included with the C types.  
* Remove redefinition of standard C types.  

M doc/xml/release.xml
M src/cipher/cipher.h
M src/command/archive/push/push.c
M src/command/help/help.c
M src/common/encode.h
M src/common/encode/base64.h
M src/common/error.h
M src/common/ini.h
M src/common/io/handle.h
M src/common/log.h
M src/common/memContext.h
M src/common/regExp.c
M src/common/regExp.h
M src/common/time.c
M src/common/time.h
M src/common/type/buffer.h
M src/common/type/keyValue.c
M src/common/type/keyValue.h
M src/common/type/variant.c
M src/common/type/variant.h
M src/common/type/variantList.h
R071 src/common/type.h src/common/typec.h
M src/config/config.c
M src/config/config.h
M src/config/define.c
M src/config/define.h
M src/perl/config.h
M src/perl/exec.c
M src/postgres/pageChecksum.c
M src/postgres/pageChecksum.h
M src/storage/storage.h
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/src/common/harnessTest.h
M test/src/common/logTest.c
R098 test/src/module/common/typeTest.c test/src/module/common/typecTest.c

Abstract Posix storage driver code into a separate module.

commit   : bd25248df02d4a21315e73c1a4dc28fe8458fe97    
  
author   : David Steele <[email protected]>    
date     : Thu, 5 Apr 2018 14:48:16 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 5 Apr 2018 14:48:16 -0400    

Click here for diff

M doc/xml/release.xml
M libc/Makefile.PL
M src/Makefile
A src/storage/driver/posix.c
A src/storage/driver/posix.h
M src/storage/storage.c
M src/storage/storage.h
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/src/module/storage/storageTest.c

Make backup directory sync more efficient.

commit   : 348278bb68ea4cebc3140404729f12fc63d6071d    
  
author   : David Steele <[email protected]>    
date     : Tue, 3 Apr 2018 21:30:15 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 3 Apr 2018 21:30:15 -0400    

Click here for diff

Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.  

M doc/xml/release.xml
M lib/pgBackRest/Backup/Backup.pm
M lib/pgBackRest/Storage/Base.pm
M test/expect/mock-all-001.log
M test/expect/mock-all-002.log

Make backup.history sync more efficient.

commit   : 0dce8ba2fa096faafa3871517458bfe6d742350f    
  
author   : David Steele <[email protected]>    
date     : Tue, 3 Apr 2018 20:54:18 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 3 Apr 2018 20:54:18 -0400    

Click here for diff

Only the backup.history/[year] directory was being synced, so check if the backup.history is newly created and sync it as well.  

M doc/xml/release.xml
M lib/pgBackRest/Backup/Backup.pm
M test/expect/mock-all-001.log
M test/expect/mock-all-002.log

Fix directory syncs running recursively when only the specified directory should be synced.

commit   : 58902722477518d7daa202241772bf499ea136a3    
  
author   : David Steele <[email protected]>    
date     : Tue, 3 Apr 2018 18:12:03 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 3 Apr 2018 18:12:03 -0400    

Click here for diff

Reported by Craig A. James.  

M doc/xml/release.xml
M lib/pgBackRest/Storage/Local.pm

Fix header exclusion defines that do not match the general pattern.

commit   : 46b6be6d7269ae35ff633736c4c5ddcde313d0d2    
  
author   : David Steele <[email protected]>    
date     : Tue, 3 Apr 2018 13:35:56 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 3 Apr 2018 13:35:56 -0400    

Click here for diff

M doc/xml/release.xml
M src/cipher/cipher.h
M src/cipher/random.h
M src/command/archive/push/push.h
M src/command/command.h
M src/command/help/help.h
M src/common/encode.h
M src/common/encode/base64.h
M src/common/error.h
M src/common/memContext.h
M src/common/type/stringList.h
M src/common/type/variantList.h
M src/config/config.h
M src/postgres/pageChecksum.h

Storage object improvements.

commit   : 93fdb98d1500ca72a1d3355ed0c7ff23e27e074c    
  
author   : David Steele <[email protected]>    
date     : Tue, 3 Apr 2018 12:25:21 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 3 Apr 2018 12:25:21 -0400    

Click here for diff

* Convert all functions to variadic functions.  
* Enforce read-only storage.  
* Add storageLocalWrite() helper function. Add storageExists(), storagePathCreate(), storageRemove(), and storageStat().  
* Add StorageFile object and storageOpenRead()/storageOpenWrite().  

M CODING.md
M doc/xml/coding.xml
M doc/xml/release.xml
M libc/Makefile.PL
M src/Makefile
M src/command/archive/push/push.c
M src/common/ini.c
M src/common/type/string.c
M src/common/type/string.h
M src/common/wait.c
M src/config/parse.c
A src/storage/file.c
A src/storage/file.h
M src/storage/helper.c
M src/storage/helper.h
M src/storage/storage.c
M src/storage/storage.h
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/lib/pgBackRestTest/Common/JobTest.pm
M test/src/common/logTest.c
M test/src/module/archive/pushTest.c
M test/src/module/common/iniTest.c
M test/src/module/common/logTest.c
M test/src/module/common/timeTest.c
M test/src/module/common/typeStringTest.c
M test/src/module/config/parseTest.c
M test/src/module/help/helpTest.c
A test/src/module/storage/fileTest.c
M test/src/module/storage/helperTest.c
M test/src/module/storage/storageTest.c

Remove documentation describing info --output=json as experimental.

commit   : 90f980fe91b8dff96523a8089613b31b4485b923    
  
author   : David Steele <[email protected]>    
date     : Fri, 30 Mar 2018 21:33:30 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 30 Mar 2018 21:33:30 -0400    

Click here for diff

M doc/xml/reference.xml
M doc/xml/release.xml
M src/config/define.auto.c
M test/expect/real-help-001.log

Add ASSERT() that is preserved in production builds.

commit   : 635caff573315d5fe0dd267f968a4c40c98d524f    
  
author   : David Steele <[email protected]>    
date     : Fri, 30 Mar 2018 19:10:34 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 30 Mar 2018 19:10:34 -0400    

Click here for diff

M doc/xml/release.xml
M src/common/debug.h
M test/lib/pgBackRestTest/Common/DefineTest.pm
M test/src/module/common/debugOffTest.c
M test/src/module/common/debugOnTest.c

Improve logic for smart builds to include version changes.

commit   : a8721ffe118e0c6ae93f6bc45c6bb79aaf78b440    
  
author   : David Steele <[email protected]>    
date     : Tue, 27 Mar 2018 22:48:22 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 27 Mar 2018 22:48:22 -0400    

Click here for diff

M doc/xml/release.xml
M test/test.pl

Full branch coverage for storage/storage module.

commit   : 5518a2ac51be91f71bfed0e27eba5e8bb6e465d2    
  
author   : David Steele <[email protected]>    
date     : Tue, 27 Mar 2018 20:40:33 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 27 Mar 2018 20:40:33 -0400    

Click here for diff

M src/common/type/buffer.c
M src/storage/storage.c
M test/src/module/common/typeBufferTest.c
M test/src/module/storage/storageTest.c

Start work on C handle io object and use it to output help.

commit   : c1ab7b3c98c0a21b2b1b78ef3e64dfd891f68a11    
  
author   : David Steele <[email protected]>    
date     : Sun, 25 Mar 2018 11:39:24 -0400    
  
committer: David Steele <[email protected]>    
date     : Sun, 25 Mar 2018 11:39:24 -0400    

Click here for diff

M doc/xml/release.xml
M src/Makefile
M src/command/help/help.c
A src/common/io/handle.c
A src/common/io/handle.h
M test/lib/pgBackRestTest/Common/DefineTest.pm
R098 test/lib/pgBackRestTest/Module/Common/CommonIoHandleTest.pm test/lib/pgBackRestTest/Module/Common/CommonIoHandlePerlTest.pm
A test/src/module/common/ioHandleTest.c

Replace THROW_ON_SYS_ERROR() with THROW_SYS_ERROR().

commit   : 9001b9b9576cb2d852984214e8bdd3b583dc552f    
  
author   : David Steele <[email protected]>    
date     : Sat, 24 Mar 2018 14:11:29 -0400    
  
committer: David Steele <[email protected]>    
date     : Sat, 24 Mar 2018 14:11:29 -0400    

Click here for diff

The former macro was hiding missing branch coverage for critical error handling.  

M doc/xml/release.xml
M src/command/archive/push/push.c
M src/command/help/help.c
M src/common/error.c
M src/common/error.h
M src/common/log.c
M src/storage/storage.c
M test/src/module/common/errorTest.c
M test/src/module/help/helpTest.c
M test/src/module/perl/execTest.c

Document build improvements.

commit   : c7a6e9d2f2b58ca7832c58c3e25c65c1d03079e3    
  
author   : David Steele <[email protected]>    
date     : Fri, 23 Mar 2018 16:08:32 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 23 Mar 2018 16:08:32 -0400    

Click here for diff

* Perform apt-get update to ensure packages are up to date before installing.  
* Add -p to the repository mkdir so it won't fail if the directory already exists, handy for testing packages.  

M doc/xml/release.xml
M doc/xml/user-guide.xml

Include VM type in gcov path to avoid conflicts between VMs with different architectures.

commit   : dabf727d91704dac6bfd59019d112857cbf963f3    
  
author   : David Steele <[email protected]>    
date     : Wed, 21 Mar 2018 16:08:08 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 21 Mar 2018 16:08:08 -0400    

Click here for diff

M doc/xml/release.xml
M test/lib/pgBackRestTest/Common/JobTest.pm

Full branch coverage for config/parse module.

commit   : c6457b070f5bdf26bb9146004e5ba2116b161df7    
  
author   : David Steele <[email protected]>    
date     : Wed, 21 Mar 2018 13:46:08 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 21 Mar 2018 13:46:08 -0400    

Click here for diff

M doc/xml/release.xml
M src/config/parse.c
M test/src/module/config/parseTest.c

Refactor usec to msec in common/time.c.

commit   : b234f43c9d8e6abde99947c64e0776e0054091c8    
  
author   : David Steele <[email protected]>    
date     : Wed, 21 Mar 2018 09:18:48 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 21 Mar 2018 09:18:48 -0400    

Click here for diff

The implementation provides usec resolution but this is not needed in practice and it makes the interface more complicated due to the extra zeros.  

M doc/xml/release.xml
M lib/pgBackRest/Version.pm
M libc/lib/pgBackRest/LibCAuto.pm
M src/common/log.c
M src/common/time.c
M src/common/time.h
M src/common/wait.c
M src/version.h
M test/src/module/common/timeTest.c
M test/src/module/common/waitTest.c

Full branch coverage for all common/type modules.

commit   : 31830bdc55bd603e90a13b54b81b919526cf03a5    
  
author   : David Steele <[email protected]>    
date     : Tue, 20 Mar 2018 16:08:52 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 20 Mar 2018 16:08:52 -0400    

Click here for diff

M src/common/type/variant.c
M test/src/module/common/typeBufferTest.c
M test/src/module/common/typeKeyValueTest.c
M test/src/module/common/typeListTest.c
M test/src/module/common/typeStringListTest.c
M test/src/module/common/typeStringTest.c
M test/src/module/common/typeVariantTest.c

Specify prefix for lcov output.

commit   : a2030da20048845c879c45279e839c9109a92418    
  
author   : David Steele <[email protected]>    
date     : Tue, 20 Mar 2018 16:01:38 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 20 Mar 2018 16:01:38 -0400    

Click here for diff

This prevents pages from moving around depending on the subtest.  

M test/test.pl

Add switch() to lcov branch exclusions.

commit   : f4ed36d128ba7bda602f0c6451325ce377f422b4    
  
author   : David Steele <[email protected]>    
date     : Tue, 20 Mar 2018 10:17:53 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 20 Mar 2018 10:17:53 -0400    

Click here for diff

lcov requires default to show complete coverage but --Wswitch-enum enforces all enum values be present so no default is needed.  
  
Add documentation for each branch exclusion.  

M test/src/lcov.conf

Update lcov settings.

commit   : d34142dee82e41f983517a181f721d6009d70a81    
  
author   : David Steele <[email protected]>    
date     : Tue, 20 Mar 2018 09:47:51 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 20 Mar 2018 09:47:51 -0400    

Click here for diff

Mark any coverage less that 100% as yellow, less that 90% as red.  
  
Add comments to config file and make genhtml use it as well as lcov.  

M test/src/lcov.conf
M test/test.pl

Use lcov for C unit test coverage reporting.

commit   : 07f38f584a6014e10888fe0f8016febab79ce67b    
  
author   : David Steele <[email protected]>    
date     : Mon, 19 Mar 2018 23:33:28 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 19 Mar 2018 23:33:28 -0400    

Click here for diff

Switch from Devel::Cover because it would not report on branch coverage for reports converted from gcov.  
  
Branch coverage is not complete, so for the time being errors will only be generated when statement coverage is not complete. Coverage of unit tests is not displayed in the report unless they are incomplete for either statement or branch coverage.  

M .travis.yml
M doc/xml/release.xml
M src/common/regExp.c
M test/Vagrantfile
M test/lib/pgBackRestTest/Common/CiTest.pm
M test/lib/pgBackRestTest/Common/ContainerTest.pm
M test/lib/pgBackRestTest/Common/JobTest.pm
A test/src/lcov.conf
M test/src/module/common/memContextTest.c
M test/src/module/common/typeBufferTest.c
M test/test.pl