pgBackRest 2.13 commit log

v2.13: Bug Fixes

commit   : 41f3874822faf3f859f46867c0fe965f7156b123    
  
author   : David Steele <[email protected]>    
date     : Thu, 18 Apr 2019 21:26:02 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 18 Apr 2019 21:26:02 -0400    

Click here for diff

Bug Fixes:  
  
* Fix zero-length reads causing problems for IO filters that did not expect them. (Reported by brunre01, jwpit, Tomasz Kontusz, guruguruguru.)  
* Fix reliability of error reporting from local/remote processes.  
* Fix Posix/CIFS error messages reporting the wrong filename on write/sync/close.  

M README.md
M doc/resource/exe.cache
M doc/xml/release.xml
M lib/pgBackRest/Version.pm
M src/perl/embed.auto.c
M src/version.h
M test/code-count/file-type.yaml
M test/lib/pgBackRestTest/Common/LogTest.pm

Harden IO filters against zero input and optimize zero output case.

commit   : 7390952d8e99649077a6d4b577a483e6c38a3ecd    
  
author   : David Steele <[email protected]>    
date     : Thu, 18 Apr 2019 21:24:10 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 18 Apr 2019 21:24:10 -0400    

Click here for diff

Add production checks to ensure no filter gets a zero-size input buffer.  
  
Also, optimize the case where a filter returns no output.  There's no sense in running downstream filters if they have no new input.  

M doc/xml/release.xml
M src/common/io/filter/filter.c
M src/common/io/filter/group.c
M test/src/module/common/cryptoTest.c
M test/src/module/common/ioTest.c

Fix zero-length reads causing problems for IO filters that did not expect them.

commit   : 2d73de1d360e0b7bc66b4db9e6b3a0e441927230    
  
author   : David Steele <[email protected]>    
date     : Thu, 18 Apr 2019 21:21:35 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 18 Apr 2019 21:21:35 -0400    

Click here for diff

The IoRead object was passing zero-length buffers into the filter processing code but not all the filters were happy about getting them.  
  
In particular, the gzip compression filter failed if it was given no input directly after it had flushed all of its buffers.  This made the problem rather intermittent even though a zero-length buffer was being passed to the filter at the end of every file.  It also explains why tweaking compress-level or buffer-size allowed the file to go through.  
  
Since this error was happening after all processing had completed, there does not appear to be any risk that successfully processed files were corrupted.  
  
Reported by brunre01, jwpit, Tomasz Kontusz, guruguruguru.  

M doc/xml/release.xml
M src/common/io/read.c
M test/src/module/common/ioTest.c

Add CHECK() macro for production assertions.

commit   : 670fa88a98c7e12c4e2e948d92476918d3048f7e    
  
author   : David Steele <[email protected]>    
date     : Thu, 18 Apr 2019 13:21:24 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 18 Apr 2019 13:21:24 -0400    

Click here for diff

CHECK() works just like ASSERT() but is kept in production builds.  

M doc/xml/release.xml
M src/common/assert.h

Move lockRelease() to the end of exitSafe().

commit   : b258aec0adcf6fdaf7a2494c66f1540aa720e345    
  
author   : David Steele <[email protected]>    
date     : Thu, 18 Apr 2019 13:19:05 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 18 Apr 2019 13:19:05 -0400    

Click here for diff

Releasing the lock too early was allowing other async processes to sneak in and start running before the current process was completely shut down.  
  
The only symptom seems to have been mixed up log messages so not a very serious issue.  

M doc/xml/release.xml
M src/common/exit.c
M test/expect/mock-all-001.log

Fix reliability of error reporting from local/remote processes.

commit   : b960919cf7322f03ee4f9681f0cb06da6b4c52de    
  
author   : David Steele <[email protected]>    
date     : Thu, 18 Apr 2019 10:36:21 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 18 Apr 2019 10:36:21 -0400    

Click here for diff

Asserts were only only reported on stderr rather than being returned through the protocol layer.  This did not appear to be very reliable.  
  
Instead, report the assert through the protocol layer like any other error.  Add a stack trace if an assert error or debug logging is enabled.  

M doc/xml/release.xml
M src/command/remote/remote.c
M src/protocol/client.c
M src/protocol/client.h
M src/protocol/server.c
M src/protocol/server.h
M test/src/module/protocol/protocolTest.c

Option to build documentation from current apt.postgres.org packages.

commit   : 281d2848b92bf7dba36018127cb9238662d7875b    
  
author   : David Steele <[email protected]>    
date     : Wed, 17 Apr 2019 18:39:35 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 17 Apr 2019 18:39:35 -0400    

Click here for diff

This is useful for testing any issue currently being seen in the field.  
  
We'll also need to add a "yum" option at some point.  

M doc/xml/release.xml
M doc/xml/user-guide.xml

Fix Posix/CIFS error messages reporting the wrong filename on write/sync/close.

commit   : 0d4ba3a39f10667bb0234a94850548ab2dd7ab2d    
  
author   : David Steele <[email protected]>    
date     : Wed, 17 Apr 2019 18:18:55 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 17 Apr 2019 18:18:55 -0400    

Click here for diff

The reported file name was the destination file, not the temp file that is written to during atomic write operations.  

M doc/xml/release.xml
M src/storage/driver/posix/fileWrite.c
M test/src/module/storage/posixTest.c

Add macros to create constant Variant types.

commit   : 4c13955c05662c2df3fb13cfc1d5538c2a0010a0    
  
author   : David Steele <[email protected]>    
date     : Wed, 17 Apr 2019 08:04:22 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 17 Apr 2019 08:04:22 -0400    

Click here for diff

These work almost exactly like the String constant macros. However, a struct per variant type was required which meant custom constructors and destructors for each type.  
  
Propagate the variant constants out into the codebase wherever they are useful.  

M doc/xml/release.xml
M src/Makefile
M src/command/archive/common.c
M src/command/archive/get/get.c
M src/command/archive/get/protocol.c
M src/command/archive/push/file.c
M src/command/archive/push/protocol.c
M src/command/archive/push/push.c
M src/command/help/help.c
M src/command/info/info.c
M src/common/ini.c
M src/common/io/filter/group.c
M src/common/io/http/header.c
M src/common/io/http/query.c
M src/common/type/json.c
M src/common/type/variant.c
M src/common/type/variant.h
M src/config/exec.c
M src/config/load.c
M src/config/parse.c
M src/info/info.c
M src/info/infoBackup.c
M src/info/infoManifest.c
M src/info/infoManifest.h
M src/info/infoPg.c
M src/info/infoPg.h
M src/perl/config.c
M src/protocol/client.c
M src/protocol/command.c
M src/protocol/helper.c
M src/protocol/server.c
M src/storage/driver/remote/fileRead.c
M src/storage/driver/remote/fileWrite.c
M src/storage/driver/remote/protocol.c
M src/storage/driver/remote/storage.c
M test/src/module/command/infoTest.c
M test/src/module/common/typeVariantTest.c

Add STR() macro to create constant String objects from runtime strings.

commit   : 2dac4b598620693d1d564c0ecb2c124693357d6f    
  
author   : David Steele <[email protected]>    
date     : Tue, 16 Apr 2019 13:39:58 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 16 Apr 2019 13:39:58 -0400    

Click here for diff

The STRING_CONST() macro worked fine for constants but was not able to constify strings created at runtime.  
  
Add the STR() macro to do this by using strlen() to get the size.  
  
Also rename STRING_CONST() to STRDEF() for brevity and to match the other macro name.  

M doc/xml/release.xml
M src/command/archive/common.c
M src/command/archive/get/file.c
M src/command/archive/get/get.c
M src/command/archive/push/push.c
M src/command/help/help.c
M src/command/remote/remote.c
M src/common/lock.c
M src/common/type/string.c
M src/common/type/string.h
M src/common/type/variant.c
M src/config/exec.c
M src/config/parse.c
M src/info/infoBackup.c
M src/perl/config.c
M src/postgres/interface.c
M src/protocol/client.c
M src/protocol/helper.c
M src/protocol/parallel.c
M src/protocol/server.c
M src/storage/driver/posix/storage.c
M src/storage/driver/remote/fileWrite.c
M src/storage/driver/remote/protocol.c
M src/storage/driver/s3/storage.c
M src/storage/helper.c
M src/storage/storage.c
M test/src/module/common/typeStringTest.c

Use a macro instead of a nested struct to create common String variables.

commit   : a029eba8d56dc2167814669bc8c5ba12f2eaab80    
  
author   : David Steele <[email protected]>    
date     : Tue, 16 Apr 2019 13:18:30 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 16 Apr 2019 13:18:30 -0400    

Click here for diff

This makes the code a bit less obfuscated while still ensuring that these variables will be common to both String structures.  

M doc/xml/release.xml
M src/common/type/string.c
M src/common/type/string.h
M test/src/module/common/typeStringTest.c

Migrate backupRegExp() to C.

commit   : a7281878ac8e058b528e49f80a525c09f62a39ed    
  
author   : Cynthia Shang <[email protected]>    
date     : Mon, 15 Apr 2019 08:29:25 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 15 Apr 2019 08:29:25 -0400    

Click here for diff

Removed the "anchor" parameter because it was never used in any calls in the Perl code so it was just a dead parameter that always defaulted to true.  
  
Contributed by Cynthia Shang.  

M doc/xml/release.xml
M src/Makefile
A src/command/backup/common.c
A src/command/backup/common.h
M test/define.yaml
A test/src/module/command/backupCommonTest.c

Various comment and assert fixes in the info modules.

commit   : 39ed4f763b923905b651938b7fe65c48e099b880    
  
author   : David Steele <[email protected]>    
date     : Fri, 12 Apr 2019 18:13:01 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 12 Apr 2019 18:13:01 -0400    

Click here for diff

M src/info/info.c
M src/info/infoArchive.c
M src/info/infoBackup.c
M src/info/infoManifest.h
M src/info/infoPg.c

Convert commented cipherPass params to test params.

commit   : 65a26ac4ba111c1e7e4bfd6c0a0bf1d84afd66e8    
  
author   : David Steele <[email protected]>    
date     : Fri, 12 Apr 2019 18:10:39 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 12 Apr 2019 18:10:39 -0400    

Click here for diff

This param will show up in stack traces for test builds, but not production.  

M src/command/archive/get/file.c
M src/info/info.c
M src/info/infoArchive.c
M src/info/infoBackup.c
M src/info/infoPg.c

Automatically generate constants for command and option names.

commit   : c5157c03345eaf4b8e9e7e06190d75021e9d8e39    
  
author   : David Steele <[email protected]>    
date     : Fri, 12 Apr 2019 09:03:34 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 12 Apr 2019 09:03:34 -0400    

Click here for diff

These constants are easier than using cfgOptionName() and cfgCommandName() and lead to cleaner code and simpler to construct messages.  
  
String versions are provided. Eventually all the strings will be used in the config structures, but for now they are useful to avoid wrapping with strNew().  

M build/lib/pgBackRestBuild/Config/Build.pm
M build/lib/pgBackRestBuild/Config/BuildParse.pm
M doc/xml/release.xml
M src/command/archive/common.c
M src/command/archive/get/get.c
M src/command/archive/push/push.c
M src/config/config.auto.c
M src/config/config.auto.h
M src/config/load.c
M src/config/parse.auto.c
M src/main.c
M src/protocol/helper.c

Begin v2.13 development.

commit   : 867690c08d5710d0cab27a9ee5ba879f7216abda    
  
author   : David Steele <[email protected]>    
date     : Fri, 12 Apr 2019 08:33:10 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 12 Apr 2019 08:33:10 -0400    

Click here for diff

Also update CentOS packages so documentation builds.  

M doc/xml/release.xml
M doc/xml/user-guide.xml
M lib/pgBackRest/Version.pm
M src/perl/embed.auto.c
M src/version.h