pgBackRest 2.31 commit log

v2.30: PostgreSQL 13 Support

commit   : 9229d6c866ee6296180bfefdbce42660b9e59b98    
  
author   : David Steele <[email protected]>    
date     : Mon, 5 Oct 2020 11:51:45 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 5 Oct 2020 11:51:45 -0400    

Click here for diff

Bug Fixes:  
  
* Error with hints when backup user cannot read pg_settings. (Reviewed by Stefan Fercot, Cynthia Shang. Reported by Mohamed Insaf K.)  
  
Features:  
  
* PostgreSQL 13 support. (Reviewed by Cynthia Shang.)  
  
Improvements:  
  
* Improve PostgreSQL version identification. (Reviewed by Cynthia Shang, Stephen Frost.)  
* Improve working directory error message. (Reviewed by Stefan Fercot.)  
* Add hint about starting the stanza when WAL segment not found. (Contributed by David Christensen. Reviewed by David Steele.)  
* Add hint for protocol version mismatch. (Reviewed by Cynthia Shang. Suggested by loop-evgeny.)  
  
Documentation Improvements:  
  
* Add note that pgBackRest versions must match when running remotely. (Reviewed by Cynthia Shang. Suggested by loop-evgeny.)  
* Move info command text to the reference and link to user guide. (Reviewed by Cynthia Shang. Suggested by Christophe Courtois.)  
* Update yum repository path for CentOS/RHEL user guide. (Contributed by Heath Lord. Reviewed by David Steele.)  

M CONTRIBUTING.md
M README.md
M doc/RELEASE.md
M doc/resource/exe.cache
M doc/resource/git-history.cache
M doc/xml/auto/metric-coverage-report.auto.xml
M doc/xml/release.xml
M src/build/configure.ac
M src/configure
M src/version.h
M test/code-count/file-type.yaml

Add apt-get update to contributing guide.

commit   : 74151d57ba03a4e9386f8e988d7f832ccfbe1e25    
  
author   : David Steele <[email protected]>    
date     : Mon, 5 Oct 2020 08:42:18 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 5 Oct 2020 08:42:18 -0400    

Click here for diff

This allows apt-get to work correctly even if the base container is a bit out of date.  

M doc/xml/contributing.xml

commit   : 293add28fdd306b863b077d27b0f3438b6594110    
  
author   : David Steele <[email protected]>    
date     : Mon, 5 Oct 2020 08:37:48 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 5 Oct 2020 08:37:48 -0400    

Click here for diff

Since links are not possible in the command line help just display the name of the linked section.  
  
Also, during reference text rendering there is no out key so make sure it is defined before trying to use it.  

M doc/lib/pgBackRestDoc/Common/DocRender.pm
M src/config/define.auto.c

commit   : 597739fafecf349af470134200aafc65a53eabf2    
  
author   : David Steele <[email protected]>    
date     : Fri, 25 Sep 2020 11:26:27 -0400    
  
committer: GitHub <[email protected]>    
date     : Fri, 25 Sep 2020 11:26:27 -0400    

Click here for diff

This means the same text will appear in both places, which should make it easier to find.
  

  
Also update the link code to allow both page and section to be specified rather than only one or the other.

M doc/lib/pgBackRestDoc/Common/DocRender.pm
M doc/xml/reference.xml
M doc/xml/release.xml
M doc/xml/user-guide.xml

Add hint and documentation for protocol version mismatches.

commit   : 31962377d9d53aed4b8b01accdba7801fa5ce435    
  
author   : David Steele <[email protected]>    
date     : Fri, 25 Sep 2020 10:30:29 -0400    
  
committer: GitHub <[email protected]>    
date     : Fri, 25 Sep 2020 10:30:29 -0400    

Click here for diff

Update the documentation to explicitly state that versions must match across hosts when running remotely.
  

  
Add a hint to the protocol version mismatch error to help the user identify the problem.

M doc/xml/release.xml
M doc/xml/user-guide.xml
M src/protocol/client.c
M test/src/module/protocol/protocolTest.c

Update test containers for PostgreSQL 13.

commit   : b096a25b49875b1303c066b6f659dd5afffebc56    
  
author   : David Steele <[email protected]>    
date     : Thu, 24 Sep 2020 11:19:51 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 24 Sep 2020 11:19:51 -0400    

Click here for diff

Add older PostgreSQL versions to the u18 container that were not available before.  
  
This also updates all minor versions for prior versions of PostgreSQL.  

M doc/xml/release.xml
M doc/xml/user-guide.xml
M test/container.yaml
M test/lib/pgBackRestTest/Common/ContainerTest.pm
M test/lib/pgBackRestTest/Common/VmTest.pm

Add internal verify command.

commit   : ad79932ba5bf3df7e59677c29b32477b7022ea16    
  
author   : Cynthia Shang <[email protected]>    
date     : Tue, 22 Sep 2020 11:57:38 -0400    
  
committer: GitHub <[email protected]>    
date     : Tue, 22 Sep 2020 11:57:38 -0400    

Click here for diff

Scan the WAL archive for missing or invalid files and build up ranges of WAL that will be used to verify backup integrity. A number of errors and warnings are currently emitted but they should not be considered authoritative (yet).
  

  
The command is incomplete so is marked internal.

M build/lib/pgBackRestBuild/Config/Data.pm
M doc/xml/reference.xml
M doc/xml/release.xml
M src/Makefile.in
M src/command/local/local.c
A src/command/verify/file.c
A src/command/verify/file.h
A src/command/verify/protocol.c
A src/command/verify/protocol.h
A src/command/verify/verify.c
A src/command/verify/verify.h
M src/config/config.auto.c
M src/config/config.auto.h
M src/config/define.auto.c
M src/config/define.auto.h
M src/main.c
M test/define.yaml
A test/src/module/command/verifyTest.c

Update yum repository path for CentOS/RHEL user guide.

commit   : 3e8a6febe66ef30dee104a895fd7e3c5f8ad0660    
  
author   : Heath Lord <[email protected]>    
date     : Mon, 21 Sep 2020 14:37:07 -0400    
  
committer: GitHub <[email protected]>    
date     : Mon, 21 Sep 2020 14:37:07 -0400    

Click here for diff

Update to the current yum.p.o repository so the CentOS/RHEL user guide will build on PostgreSQL 13.

M doc/xml/release.xml
M doc/xml/user-guide.xml

Improve PostgreSQL version identification.

commit   : 927d9adbee780a51639955edb74ce7a4850db100    
  
author   : David Steele <[email protected]>    
date     : Fri, 18 Sep 2020 16:55:26 -0400    
  
committer: GitHub <[email protected]>    
date     : Fri, 18 Sep 2020 16:55:26 -0400    

Click here for diff

Previously, catalog versions were fixed for all versions which made maintaining the catalog versions during PostgreSQL beta and release candidate cycles very painful. A version of pgBackRest which was functionally compatible was rendered useless by a catalog version bump in PostgreSQL.
  

  
Instead use only the control version to identify a PostgreSQL version when possible. Some older versions require a catalog version to positively identify a PostgreSQL version, so include them when required.
  

  
Since the catalog number is required to work with tablespaces it will need to be stored. There's already a copy of it in backup.info so use that (even though we have been ignoring it in the C versions).

M doc/xml/release.xml
M src/command/backup/backup.c
M src/command/restore/restore.c
M src/command/stanza/create.c
M src/command/stanza/upgrade.c
M src/info/infoArchive.c
M src/info/infoBackup.c
M src/info/infoBackup.h
M src/info/infoPg.c
M src/info/infoPg.h
M src/info/manifest.c
M src/info/manifest.h
M src/postgres/interface.c
M src/postgres/interface.h
M src/postgres/interface/version.h
M src/postgres/interface/version.intern.h
M src/postgres/interface/version.vendor.h
M test/src/module/command/backupTest.c
M test/src/module/command/checkTest.c
M test/src/module/command/restoreTest.c
M test/src/module/command/stanzaTest.c
M test/src/module/info/infoBackupTest.c
M test/src/module/info/infoPgTest.c
M test/src/module/info/manifestTest.c
M test/src/module/performance/typeTest.c
M test/src/module/postgres/interfaceTest.c

Add missing semicolon in backtrace debug code.

commit   : 94475bfbe6a694b150091d5f7208b0d01ab07a40    
  
author   : David Steele <[email protected]>    
date     : Thu, 17 Sep 2020 15:17:13 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 17 Sep 2020 15:17:13 -0400    

Click here for diff

Apparently backtrace has not been used for debugging since it was broken in 7fba1f0b.  
  
Even though this is test code it might be good to find a way to test it to prevent regressions.  

M src/common/stackTrace.c

Update PostgreSQL 13 test catalog versions missed in 6bb111c1.

commit   : 4cd61152f510140a30525e995b0901590d55012c    
  
author   : David Steele <[email protected]>    
date     : Thu, 17 Sep 2020 12:39:30 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 17 Sep 2020 12:39:30 -0400    

Click here for diff

These values are not used by the Perl integration tests so maybe it would be better to remove them, but for now just update since they should not be changing again for PG13.  

M test/lib/pgBackRestTest/Env/ExpireEnvTest.pm
M test/lib/pgBackRestTest/Env/HostEnvTest.pm

Error with hints when backup user cannot read pg_settings.

commit   : 9db314397302bc01d81a24cb0edaceefa2b2e9d0    
  
author   : David Steele <[email protected]>    
date     : Thu, 17 Sep 2020 10:35:27 -0400    
  
committer: GitHub <[email protected]>    
date     : Thu, 17 Sep 2020 10:35:27 -0400    

Click here for diff

This condition used to give a not-very-clear error which we have been intending to improve. But in the meantime the changes in fbff299 resulted in a segfault for this condition instead because the data_directory was assumed to be non-NULL.
  

  
Fix this by explicitly throwing an error with hints when any row in pg_settings cannot be selected.

M doc/xml/release.xml
M src/db/db.c
M test/src/module/db/dbTest.c

Fix incorrect control/catalog versions in command/stanza test.

commit   : c71609879bb3782dd7da40c14313e75e78262bfc    
  
author   : David Steele <[email protected]>    
date     : Wed, 16 Sep 2020 09:33:37 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 16 Sep 2020 09:33:37 -0400    

Click here for diff

These were not caught because backup info load automatically corrects them.  

M test/src/module/command/stanzaTest.c

Ignore backup_manifest in PG13.

commit   : 8dce7bbb608c65a16a4e41ae2cd95c588154e9e0    
  
author   : David Steele <[email protected]>    
date     : Mon, 14 Sep 2020 10:15:40 -0400    
  
committer: GitHub <[email protected]>    
date     : Mon, 14 Sep 2020 10:15:40 -0400    

Click here for diff

This file is created by pg_basebackup so might be in the data directory if the cluster was restored from a pg_basebackup backup. Also exclude backup_manifest.tmp since it is possible to find that in the backup directory.

M doc/xml/release.xml
M src/info/manifest.c
M src/postgres/interface.h
M test/src/module/info/manifestTest.c

Improve working directory error message.

commit   : fc77c511824feb77c0a5f4cd5d95795184b65d2e    
  
author   : David Steele <[email protected]>    
date     : Fri, 11 Sep 2020 10:10:25 -0400    
  
committer: GitHub <[email protected]>    
date     : Fri, 11 Sep 2020 10:10:25 -0400    

Click here for diff

Improve the wording of the error message and add a hint to make it clearer what is wrong and how the user can fix it.
  

  
Also change the assert to a regular error since this is not an internal error.

M doc/xml/release.xml
M src/command/archive/common.c
M test/src/module/command/archiveCommonTest.c

Improve alignment in TEST_ERROR output.

commit   : dac182c06e84ba94a3e03cc0769367bdc89ba3b7    
  
author   : David Steele <[email protected]>    
date     : Fri, 11 Sep 2020 08:23:26 -0400    
  
committer: David Steele <[email protected]>    
date     : Fri, 11 Sep 2020 08:23:26 -0400    

Click here for diff

Align the error types and messages so they are easier to compare.  

M test/src/common/harnessTest.h

Add infoArchiveMove() and infoBackupMove().

commit   : ecb112f802be958fdccf003be2e9b62aada98bb3    
  
author   : Cynthia Shang <[email protected]>    
date     : Tue, 8 Sep 2020 13:02:13 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 8 Sep 2020 13:02:13 -0400    

Click here for diff

M src/info/infoArchive.c
M src/info/infoArchive.h
M src/info/infoBackup.c
M src/info/infoBackup.h
M test/src/module/info/infoArchiveTest.c
M test/src/module/info/infoBackupTest.c

Extern infoBackupNewLoad() and infoArchiveNewLoad().

commit   : 5941f5631afc73e504d67827587a90ca91ead399    
  
author   : Cynthia Shang <[email protected]>    
date     : Tue, 8 Sep 2020 12:55:56 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 8 Sep 2020 12:55:56 -0400    

Click here for diff

M src/info/infoArchive.c
M src/info/infoArchive.h
M src/info/infoBackup.c
M src/info/infoBackup.h

Add manifestFree().

commit   : 4239f8a81aa5da52667149a4933956347492b748    
  
author   : Cynthia Shang <[email protected]>    
date     : Tue, 8 Sep 2020 12:47:35 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 8 Sep 2020 12:47:35 -0400    

Click here for diff

M src/info/manifest.c
M src/info/manifest.h
M test/src/module/info/manifestTest.c

Move WAL constants to postgres/interface header.

commit   : fead0046918ad50eafa30991c2e8d6aabbf254a0    
  
author   : Cynthia Shang <[email protected]>    
date     : Tue, 8 Sep 2020 12:44:08 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 8 Sep 2020 12:44:08 -0400    

Click here for diff

M src/postgres/interface.c
M src/postgres/interface.h

Move archiveIdComparator() to archive/common module.

commit   : b8efb13bcb11027fcbb3bd386446280cd8b625ce    
  
author   : Cynthia Shang <[email protected]>    
date     : Tue, 8 Sep 2020 12:28:56 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 8 Sep 2020 12:28:56 -0400    

Click here for diff

M src/command/archive/common.c
M src/command/archive/common.h
M src/command/expire/expire.c
M test/define.yaml
M test/src/module/command/archiveCommonTest.c
M test/src/module/command/expireTest.c

Add hint about starting the stanza when WAL segment not found.

commit   : 9fd31913a8cbff1f1371ccabfebb003cb3c47c05    
  
author   : David Christensen <[email protected]>    
date     : Thu, 3 Sep 2020 07:49:49 -0400    
  
committer: David Steele <[email protected]>    
date     : Thu, 3 Sep 2020 07:49:49 -0400    

Click here for diff

If a stop command has been issued the check command fails due to archiving timing out.  
  
Provide a hint to document this situation and point the user in the proper direction.  

M doc/xml/release.xml
M src/command/archive/common.c
M test/src/module/command/archiveCommonTest.c
M test/src/module/command/backupTest.c
M test/src/module/command/checkTest.c

Fix contributor name.

commit   : 76900d676416758e82b12ea3966daadf6e3b3464    
  
author   : David Steele <[email protected]>    
date     : Wed, 2 Sep 2020 14:45:09 -0400    
  
committer: David Steele <[email protected]>    
date     : Wed, 2 Sep 2020 14:45:09 -0400    

Click here for diff

M doc/xml/release.xml

Allow ProtocolParallel to complete with no jobs.

commit   : 41118e1b6813ea41a13c0441c726f3e7c28051d9    
  
author   : David Steele <[email protected]>    
date     : Wed, 2 Sep 2020 11:03:49 -0400    
  
committer: GitHub <[email protected]>    
date     : Wed, 2 Sep 2020 11:03:49 -0400    

Click here for diff

If the callback never returned any jobs then protocolParallelDone() would never be true. The reason is that the done state was being set in protocolParallelResult(), which never gets called if there are no results.
  

  
Calling protocolParallelResult() doesn't make much sense in this case so instead move the done logic to protocolParallelDone().
  

  
For current usage of ProtocolParallel we ensure there are jobs before processing so this is not a live issue, but the new behavior is required for future development.

M doc/xml/release.xml
M src/protocol/parallel.c
M src/protocol/parallel.h
M test/src/module/protocol/protocolTest.c

commit   : ce6c492620d9408a432f5b7c04a437ec812788ac    
  
author   : David Steele <[email protected]>    
date     : Tue, 1 Sep 2020 11:02:44 -0400    
  
committer: David Steele <[email protected]>    
date     : Tue, 1 Sep 2020 11:02:44 -0400    

Click here for diff

M doc/FORMAT.md

Update config.guess and config.sub to latest versions.

commit   : bb87d989b840e77d1ceafc8f699972101fb94d96    
  
author   : David Steele <[email protected]>    
date     : Mon, 31 Aug 2020 08:20:40 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 31 Aug 2020 08:20:40 -0400    

Click here for diff

M src/build/config.guess
M src/build/config.sub

Begin v2.30 development.

commit   : 05ef03feb5550f7151ee34afe6b0358637f65fc8    
  
author   : David Steele <[email protected]>    
date     : Mon, 31 Aug 2020 07:41:48 -0400    
  
committer: David Steele <[email protected]>    
date     : Mon, 31 Aug 2020 07:41:48 -0400    

Click here for diff

M doc/resource/git-history.cache
M doc/xml/release.xml
M src/build/configure.ac
M src/configure
M src/version.h