pgBackRest 2.24 commit log · Life with Open Source

26 Jan 2020, 22:38 by Yogesh Sharma

`v2.23: Bug Fix`

commit   : 2358d3448507f5dc4fd50edf65172285bca69c4b    
  
author   : David Steele <[email protected]>    
date     : Sun, 26 Jan 2020 22:38:21 -0700    
  
committer: David Steele <[email protected]>    
date     : Sun, 26 Jan 2020 22:38:21 -0700

Click here for diff

Bug Fixes:  
  
* Fix missing files corrupting the manifest. If a file was removed by PostgreSQL during the backup (or was missing from the standby) then the next file might not be copied and updated in the manifest. If this happened then the backup would error when restored. (Reviewed by Cynthia Shang. Reported by Vitaliy Kukharik.)  
  
Improvements:  
  
* Use pkg-config instead of xml2-config for libxml2 build options. (Contributed by David Steele, Adrian Vondendriesch.)  
* Validate checksums are set in the manifest on backup/restore. (Reviewed by Cynthia Shang.)

M README.md
M doc/resource/exe.cache
M doc/resource/git-history.cache
M doc/xml/auto/metric-coverage-report.auto.xml
M doc/xml/release.xml
M lib/pgBackRest/Version.pm
M src/configure
M src/configure.ac
M src/version.h

`Validate checksums are set in the manifest on backup/restore.`

commit   : 7ab07dc580989452d807332bc91d26a42e5a0d3a    
  
author   : David Steele <[email protected]>    
date     : Sun, 26 Jan 2020 21:58:59 -0700    
  
committer: David Steele <[email protected]>    
date     : Sun, 26 Jan 2020 21:58:59 -0700

Click here for diff

This is a modest start but it addresses the specific issue that was caused by the bug fixed in 45ec694a.  This validation will produce an immediate error rather than erroring out partway through the restore.  
  
More validations are planned but this is the most important one and seems safest for this release.

M doc/xml/release.xml
M src/command/backup/backup.c
M src/command/restore/restore.c
M src/info/manifest.c
M src/info/manifest.h
M test/src/module/info/manifestTest.c

`Fix missing files corrupting the manifest.`

commit   : 45ec694af22dda84b124f0df242bbcfdd28726d1    
  
author   : David Steele <[email protected]>    
date     : Sun, 26 Jan 2020 13:19:13 -0700    
  
committer: David Steele <[email protected]>    
date     : Sun, 26 Jan 2020 13:19:13 -0700

Click here for diff

If a file was removed by PostgreSQL during the backup (or was missing from the standby) then the next file might not be copied and updated in the manifest. If this happened then the backup would error when restored.  
  
The issue was that removing files from the manifest invalidated the pointers stored in the processing queues.  When a file was removed, all the pointers shifted to the next file in the list, causing a file to be unprocessed.  Since the unprocessed file was still in the manifest it would be saved with no checksum, causing a failure on restore.  
  
When process-max was > 1 then the bug would often not express since the file had already been pulled from the queue and updates to the manifest are done by name rather than by pointer.

M doc/xml/release.xml
M src/command/backup/backup.c
M test/src/module/command/backupTest.c

`Sort last processing queue on backup from standby.`

commit   : 9b47ff2746b74bf41d20fa43d6ed2697b0a99087    
  
author   : David Steele <[email protected]>    
date     : Sun, 26 Jan 2020 12:29:53 -0700    
  
committer: David Steele <[email protected]>    
date     : Sun, 26 Jan 2020 12:29:53 -0700

Click here for diff

The last queue was not being sorted when a primary queue was added first.  
  
This did not affect the backup or integrity but could lead to slightly lower performance since large files were not always copied first.

M src/command/backup/backup.c

`Remove obsolete include to ../libc.`

commit   : 0444d3741423004e0f94116b528d94090c1e4303    
  
author   : David Steele <[email protected]>    
date     : Fri, 24 Jan 2020 10:43:47 -0700    
  
committer: David Steele <[email protected]>    
date     : Fri, 24 Jan 2020 10:43:47 -0700

Click here for diff

M src/configure
M src/configure.ac

`Add lib path for libpq in case it is in a non-standard location.`

commit   : b1c5885017c035320804c5d16ed57b149a33ab10    
  
author   : Marc Cousin <[email protected]>    
date     : Fri, 24 Jan 2020 10:40:42 -0700    
  
committer: David Steele <[email protected]>    
date     : Fri, 24 Jan 2020 10:40:42 -0700

Click here for diff

M src/configure
M src/configure.ac

`Use pkg-config instead of xml2-config for libxml2 build options.`

commit   : 90abc3cf170a3d5da8bbc5307cf5032d6af5ed6c    
  
author   : David Steele <[email protected]>    
date     : Fri, 24 Jan 2020 10:08:05 -0700    
  
committer: David Steele <[email protected]>    
date     : Fri, 24 Jan 2020 10:08:05 -0700

Click here for diff

pkg-config is a generic way to get build options rather than relying on a package-specific utility.  
  
XML2_CONFIG can be used to override this utility for systems that do not ship pkg-config.

M doc/xml/release.xml
M doc/xml/user-guide.xml
M libc/Makefile.PL
M src/configure
M src/configure.ac
M test/container.yaml
M test/lib/pgBackRestTest/Common/ContainerTest.pm
M test/lib/pgBackRestTest/Common/JobTest.pm
M test/travis.pl

`Use designated initializers to initialize structs.`

commit   : b134175fc7a98836f49f20d552f7c31138b66b1b    
  
author   : David Steele <[email protected]>    
date     : Thu, 23 Jan 2020 14:15:58 -0700    
  
committer: David Steele <[email protected]>    
date     : Thu, 23 Jan 2020 14:15:58 -0700

Click here for diff

Previously memNew() used memset() to initialize all struct members to 0, NULL, false, etc.  While this appears to work in practice, it is a violation of the C specification.  For instance, NULL == 0 must be true but neither NULL nor 0 must be represented with all zero bits.  
  
Instead use designated initializers to initialize structs.  These guarantee that struct members will be properly initialized even if they are not specified in the initializer.  Note that due to a quirk in the C99 specification at least one member must be explicitly initialized even if it needs to be the default value.  
  
Since pre-zeroed memory is no longer required, adjust memAllocInternal()/memReallocInternal() to return raw memory and update dependent functions accordingly.  All instances of memset() have been removed except in debug/test code where needed.  
  
Add memMewPtrArray() to allocate an array of pointers and automatically set all pointers to NULL.  
  
Rename memGrowRaw() to the more logical memResize().

M src/command/backup/pageChecksum.c
M src/command/restore/restore.c
M src/common/compress/gzip/compress.c
M src/common/compress/gzip/decompress.c
M src/common/crypto/cipherBlock.c
M src/common/crypto/hash.c
M src/common/exec.c
M src/common/ini.c
M src/common/io/bufferRead.c
M src/common/io/bufferWrite.c
M src/common/io/filter/buffer.c
M src/common/io/filter/filter.c
M src/common/io/filter/group.c
M src/common/io/filter/sink.c
M src/common/io/filter/size.c
M src/common/io/handleRead.c
M src/common/io/handleWrite.c
M src/common/io/http/cache.c
M src/common/io/http/client.c
M src/common/io/http/header.c
M src/common/io/http/query.c
M src/common/io/read.c
M src/common/io/tls/client.c
M src/common/io/write.c
M src/common/memContext.c
M src/common/memContext.h
M src/common/regExp.c
M src/common/stackTrace.c
M src/common/type/buffer.c
M src/common/type/keyValue.c
M src/common/type/list.c
M src/common/type/mcv.c
M src/common/type/string.c
M src/common/type/variant.c
M src/common/type/xml.c
M src/common/wait.c
M src/config/config.c
M src/config/config.h
M src/config/parse.c
M src/db/db.c
M src/info/info.c
M src/info/infoArchive.c
M src/info/infoBackup.c
M src/info/infoPg.c
M src/info/manifest.c
M src/postgres/client.c
M src/postgres/interface/version.intern.h
M src/protocol/client.c
M src/protocol/command.c
M src/protocol/helper.c
M src/protocol/parallel.c
M src/protocol/parallelJob.c
M src/protocol/server.c
M src/storage/helper.c
M src/storage/posix/read.c
M src/storage/posix/storage.c
M src/storage/posix/write.c
M src/storage/read.c
M src/storage/remote/read.c
M src/storage/remote/storage.c
M src/storage/remote/write.c
M src/storage/s3/read.c
M src/storage/s3/storage.c
M src/storage/s3/write.c
M src/storage/storage.c
M src/storage/write.c
M test/src/module/command/backupCommonTest.c
M test/src/module/command/backupTest.c
M test/src/module/common/cryptoTest.c
M test/src/module/common/exitTest.c
M test/src/module/common/ioTest.c
M test/src/module/common/memContextTest.c
M test/src/module/common/objectTest.c
M test/src/module/config/configTest.c
M test/src/module/config/parseTest.c
M test/src/module/postgres/interfaceTest.c
M test/src/module/postgres/pageChecksumTest.c

`Add XML2_CONFIG environment variable to configure.`

commit   : cf2024beafba82b3899d9c46c7ea64716835a292    
  
author   : David Steele <[email protected]>    
date     : Tue, 21 Jan 2020 18:47:14 -0700    
  
committer: David Steele <[email protected]>    
date     : Tue, 21 Jan 2020 18:47:14 -0700

Click here for diff

This allows the default 'xml2-config' to be replaced with, e.g. 'pkg-config libxml-2.0', for libxml2 configuration.

M src/configure
M src/configure.ac

`Set client_encoding to UTF8 on PostgreSQL connect.`

commit   : 600a51815feacb149add2bb777d6a3b4d284c8aa    
  
author   : David Steele <[email protected]>    
date     : Tue, 21 Jan 2020 18:42:22 -0700    
  
committer: David Steele <[email protected]>    
date     : Tue, 21 Jan 2020 18:42:22 -0700

Click here for diff

This is the only non-ASCII character encoding we have tested so make sure that's all we get from PostgreSQL.

M src/db/db.c
M test/src/common/harnessPq.h
M test/src/module/db/dbTest.c

`Set encoding to UTF8 for Debian documentation containers.`

commit   : 1706c599bd1c4d1eeac5bf44d1811e0259809027    
  
author   : David Steele <[email protected]>    
date     : Tue, 21 Jan 2020 18:37:43 -0700    
  
committer: David Steele <[email protected]>    
date     : Tue, 21 Jan 2020 18:37:43 -0700

Click here for diff

This allows testing multi-byte encodings in PostgreSQL.

M doc/xml/user-guide.xml

`Begin v2.23 development.`

commit   : 382ddfd79dfec12f4cdf821f36a47dd404a51356    
  
author   : David Steele <[email protected]>    
date     : Tue, 21 Jan 2020 16:43:44 -0700    
  
committer: David Steele <[email protected]>    
date     : Tue, 21 Jan 2020 16:43:44 -0700

Click here for diff

M doc/resource/git-history.cache
M doc/xml/release.xml
M lib/pgBackRest/Version.pm
M src/configure
M src/configure.ac
M src/version.h