pgBackRest 2.30 (upcoming) commit log

Rename handle to fd.

commit   : cde2c756ea8ae32663e0a817098fc9f5d4f2a1f2    
  
author   : David Steele <david@pgmasters.net>    
date     : Wed, 5 Aug 2020 18:25:07 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 5 Aug 2020 18:25:07 -0400    

Click here for diff

Pretty much everywhere handle is used what is really meant is file descriptor (fd). This terminology got migrated over from Perl and is just not quite correct, or at least not as correct as fd.  
  
There were also plenty of places fd was used so now all uses are consistent.  
  
The Perl code was not updated but might be in a future commit.  

M src/Makefile.in
M src/command/control/stop.c
M src/command/help/help.c
M src/command/info/info.c
M src/command/local/local.c
M src/command/local/local.h
M src/command/remote/remote.c
M src/command/remote/remote.h
M src/command/repo/get.c
M src/command/repo/ls.c
M src/command/repo/put.c
M src/command/restore/file.c
M src/common/exec.c
M src/common/fork.c
R073 src/common/io/handleRead.c src/common/io/fdRead.c
R075 src/common/io/handleRead.h src/common/io/fdRead.h
R064 src/common/io/handleWrite.c src/common/io/fdWrite.c
R074 src/common/io/handleWrite.h src/common/io/fdWrite.h
M src/common/io/read.c
M src/common/io/read.h
M src/common/io/read.intern.h
M src/common/io/write.c
M src/common/io/write.h
M src/common/io/write.intern.h
M src/common/lock.c
M src/common/log.c
M src/protocol/parallel.c
M src/storage/posix/read.c
M src/storage/posix/storage.c
M src/storage/posix/write.c
M test/code-count/file-type.yaml
M test/define.yaml
M test/src/common/harnessFork.h
M test/src/common/harnessLog.c
M test/src/module/command/archivePushTest.c
M test/src/module/command/controlTest.c
M test/src/module/command/localTest.c
M test/src/module/command/remoteTest.c
M test/src/module/common/execTest.c
M test/src/module/common/ioHttpTest.c
M test/src/module/common/ioTest.c
M test/src/module/common/ioTlsTest.c
M test/src/module/common/lockTest.c
M test/src/module/common/logTest.c
M test/src/module/config/protocolTest.c
M test/src/module/db/dbTest.c
M test/src/module/performance/storageTest.c
M test/src/module/protocol/protocolTest.c
M test/src/module/storage/azureTest.c
M test/src/module/storage/posixTest.c
M test/src/module/storage/s3Test.c

Fix macro spacing.

commit   : a1d7d50223510eda06d68234a03e3c19f972dc47    
  
author   : David Steele <david@pgmasters.net>    
date     : Wed, 5 Aug 2020 17:30:38 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 5 Aug 2020 17:30:38 -0400    

Click here for diff

M test/src/common/harnessFork.h

Remove unused port struct member.

commit   : d5edc3fe371e16d5612e841861ae9c10ae69cca7    
  
author   : David Steele <david@pgmasters.net>    
date     : Wed, 5 Aug 2020 15:24:13 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 5 Aug 2020 15:24:13 -0400    

Click here for diff

This does not appear to have been used in quite some time and the tests are equally useless because they don't prove the correct port was passed to httpClientNew().  

M src/storage/s3/storage.c
M test/src/module/storage/s3Test.c

Add memContextFreeing().

commit   : d74c8f4b5813f3b0ce5df246c36469257a549e4d    
  
author   : David Steele <david@pgmasters.net>    
date     : Wed, 5 Aug 2020 14:07:04 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 5 Aug 2020 14:07:04 -0400    

Click here for diff

This can be used to determine if functions are running in a callback from memContextFree(). In this case, data in child mem contexts is not valid.  

M src/common/memContext.c
M src/common/memContext.h
M test/src/module/common/memContextTest.c

Disable s390x builds on Travis-CI.

commit   : 6fc25c7a69a09f105f9fdd41ddb33c700d983a2e    
  
author   : David Steele <david@pgmasters.net>    
date     : Wed, 5 Aug 2020 13:38:44 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Wed, 5 Aug 2020 13:38:44 -0400    

Click here for diff

s390x has not been building for about a day which is clogging the integration branch.  

M .travis.yml

Improve TLS error reporting.

commit   : 847e61ce21710cc0cb7baa9af9b3740d9250b4fa    
  
author   : David Steele <dwsteele@users.noreply.github.com>    
date     : Tue, 4 Aug 2020 15:15:24 -0400    
  
committer: GitHub <noreply@github.com>    
date     : Tue, 4 Aug 2020 15:15:24 -0400    

Click here for diff

Before 9f2d647 TLS errors included additional details in at least some cases. After 9f2d647 a connection to an HTTP server threw `TLS error [1]` instead of `unable to negotiate TLS connection: [336031996] unknown protocol`.
  

  
Bring back the detailed messages to make debugging TLS errors easier. Since the error routine is now generic the `unable to negotiate TLS connection context` is not available so the error looks like `TLS error [1:336031996] unknown protocol`.

M doc/xml/release.xml
M src/common/io/tls/session.c
M test/src/module/common/ioTlsTest.c

Proactively close file descriptors after forking async process.

commit   : 94d3a01f7368a547dfecaafce67995e24738be78    
  
author   : David Steele <dwsteele@users.noreply.github.com>    
date     : Tue, 4 Aug 2020 13:20:01 -0400    
  
committer: GitHub <noreply@github.com>    
date     : Tue, 4 Aug 2020 13:20:01 -0400    

Click here for diff

PostgreSQL may be using most of the available file descriptors when it executes the the archive-get/archive-push commands (especially archive-get). This can lead to problems depending on how many file descriptors are needed for parallelism in the async process.
  

  
Proactively free file descriptors between 3 and 1023 to help ensure there are enough available for reasonable values of process-max, i.e. <= 300.

M doc/xml/release.xml
M src/command/archive/common.c

Improve memory usage of unlogged relation detection in manifest build.

commit   : e81533bbabb2e4f809596c0a88671b2b6616da01    
  
author   : David Steele <dwsteele@users.noreply.github.com>    
date     : Tue, 4 Aug 2020 10:16:51 -0400    
  
committer: GitHub <noreply@github.com>    
date     : Tue, 4 Aug 2020 10:16:51 -0400    

Click here for diff

This loop was using a lot of memory without freeing it at intervals.
  

  
Rewrite to use char arrays when possible to reduce memory that needs to be allocated and freed.

M doc/xml/release.xml
M src/info/manifest.c
M test/lib/pgBackRestTest/Common/JobTest.pm

Use forkSafe() in common/exec module.

commit   : e044eaf55faf102d84389b04ff79a7a00fef0965    
  
author   : David Steele <david@pgmasters.net>    
date     : Tue, 4 Aug 2020 09:23:59 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Tue, 4 Aug 2020 09:23:59 -0400    

Click here for diff

forkSafe() will throw an exception if the fork fails. This is a pretty unlikely scenario but the function is already there so may as well use it.  

M src/common/exec.c

Add functions for Zigzag encoding/decoding.

commit   : 5a9856c2f9aeb6efccada86c229713665100eba0    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 1 Aug 2020 09:42:03 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 1 Aug 2020 09:42:03 -0400    

Click here for diff

Zigzag encoding places the sign bit in the least significant bit so that -1 is encoded as 1, 1 as 2, etc. This moves as many bits as possible into the low order bits which is good for other types of encoding, e.g. base-128.  
  
See https://en.wikipedia.org/wiki/Variable-length_quantity#Zigzag_encoding.  

M src/common/type/convert.h
M test/define.yaml
M test/src/module/common/typeConvertTest.c

Improve comment locations.

commit   : f6d22ccb6fab7d7308d26c11c770362e312e600f    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 31 Jul 2020 17:33:15 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 31 Jul 2020 17:33:15 -0400    

Click here for diff

M src/storage/remote/protocol.c

Accept NULL filenames in hrnStorageInfoListCallback().

commit   : caef2ba482a887113d08fd926a582ff68786a39f    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 31 Jul 2020 16:35:58 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 31 Jul 2020 16:35:58 -0400    

Click here for diff

This allows it to be used with StorageInfo objects returned from storageInfo(), which do not contain a filename.  

M test/src/common/harnessStorage.c

Add lstGetLast() and lstRemoveLast().

commit   : a8e47c38c6d70c202070fa585d4096592a9961cf    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 31 Jul 2020 16:27:57 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 31 Jul 2020 16:27:57 -0400    

Click here for diff

When a list is being treated as a stack it is useful to get/remove the last (top) item.  

M src/common/type/list.c
M src/common/type/list.h
M test/src/module/common/typeListTest.c

Add file name to make performance/storage test more realistic.

commit   : bfb489a82d7451830abec45e3d8a79f4d8580827    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 31 Jul 2020 16:18:56 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 31 Jul 2020 16:18:56 -0400    

Click here for diff

Also add timing information.  

M test/src/module/performance/storageTest.c

Remove special protocol encoding for StorageType.

commit   : 9f1e7d88da845083a1e45d99c123b1718af9808d    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 31 Jul 2020 16:13:05 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 31 Jul 2020 16:13:05 -0400    

Click here for diff

It seems like overkill to encode this when other enums (e.g. StorageInfoLevel) are passed as integers.  
  
Instead note that StorageType values should not be changed and remove the special encoding.  

M src/storage/info.h
M src/storage/remote/protocol.c
M src/storage/remote/storage.c
M test/src/module/storage/remoteTest.c

Fix missing space.

commit   : afe1568bb1713c3520618bac87ba15e4374274bb    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 31 Jul 2020 15:55:17 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 31 Jul 2020 15:55:17 -0400    

Click here for diff

M src/common/type/buffer.h

Preserve Variant parsed from JSON in iniLoad().

commit   : ac72e1f193d8d4a9de7d8877528e78e317c135e2    
  
author   : David Steele <dwsteele@users.noreply.github.com>    
date     : Thu, 30 Jul 2020 10:59:50 -0400    
  
committer: GitHub <noreply@github.com>    
date     : Thu, 30 Jul 2020 10:59:50 -0400    

Click here for diff

The fix for = characters in info files (039d314) added JSON validation but discarded the resulting Variant which means the JSON is being parsed twice. This nearly doubles the time to load a manifest since a lot of complex JSON is involved.
  

  
Time to load a million file manifest:
  
Before 039d314: 7.8s
  
039d314: 15.5s
  
This patch: 7.5s
  

  
To fix this regression return the Variant in the callback so the caller does not have to parse it again. The new code appears slightly more efficient overall, probably because there are fewer operations against Strings.

M doc/xml/release.xml
M src/common/ini.c
M src/common/ini.h
M src/info/info.c
M src/info/info.h
M src/info/infoBackup.c
M src/info/infoPg.c
M src/info/manifest.c
M test/src/common/harnessInfo.c
M test/src/common/harnessInfo.h
M test/src/module/common/iniTest.c
M test/src/module/performance/typeTest.c

Rename strPtr()/strPtrNull() to strZ()/strZNull().

commit   : 3e9dce0d7615f0f56f3eff754095c57c6d56fd6b    
  
author   : David Steele <david@pgmasters.net>    
date     : Thu, 30 Jul 2020 07:49:06 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Thu, 30 Jul 2020 07:49:06 -0400    

Click here for diff

We use the Z suffix in many functions to indicate that we are expecting a zero-terminated string so make this function conform to the pattern.  
  
As a bonus the new name is a bit shorter, which is a good quality in a commonly-used function.  

M CODING.md
M CONTRIBUTING.md
M doc/xml/coding.xml
M doc/xml/contributing.xml
M src/command/archive/common.c
M src/command/archive/get/file.c
M src/command/archive/get/get.c
M src/command/archive/get/protocol.c
M src/command/archive/push/file.c
M src/command/archive/push/push.c
M src/command/backup/backup.c
M src/command/backup/common.c
M src/command/backup/file.c
M src/command/check/check.c
M src/command/check/common.c
M src/command/command.c
M src/command/control/common.c
M src/command/control/start.c
M src/command/control/stop.c
M src/command/expire/expire.c
M src/command/help/help.c
M src/command/info/info.c
M src/command/repo/get.c
M src/command/repo/ls.c
M src/command/restore/file.c
M src/command/restore/protocol.c
M src/command/restore/restore.c
M src/command/stanza/create.c
M src/command/stanza/delete.c
M src/command/stanza/upgrade.c
M src/common/compress/helper.c
M src/common/crypto/cipherBlock.c
M src/common/crypto/common.c
M src/common/crypto/hash.c
M src/common/exec.c
M src/common/ini.c
M src/common/io/filter/filter.c
M src/common/io/handleRead.c
M src/common/io/handleWrite.c
M src/common/io/http/common.c
M src/common/io/http/header.c
M src/common/io/http/query.c
M src/common/io/http/request.c
M src/common/io/http/response.c
M src/common/io/socket/client.c
M src/common/io/socket/common.c
M src/common/io/socket/session.c
M src/common/io/tls/client.c
M src/common/lock.c
M src/common/regExp.c
M src/common/type/buffer.c
M src/common/type/buffer.h
M src/common/type/json.c
M src/common/type/string.c
M src/common/type/string.h
M src/common/type/stringList.c
M src/common/type/variant.c
M src/common/type/variant.h
M src/common/type/xml.c
M src/common/user.c
M src/config/config.c
M src/config/exec.c
M src/config/load.c
M src/config/parse.c
M src/config/protocol.c
M src/db/db.c
M src/info/info.c
M src/info/infoArchive.c
M src/info/infoBackup.c
M src/info/infoPg.c
M src/info/manifest.c
M src/postgres/client.c
M src/postgres/interface.c
M src/protocol/client.c
M src/protocol/command.c
M src/protocol/helper.c
M src/protocol/parallelJob.c
M src/protocol/server.c
M src/storage/azure/read.c
M src/storage/azure/storage.c
M src/storage/helper.c
M src/storage/posix/read.c
M src/storage/posix/storage.c
M src/storage/posix/write.c
M src/storage/read.c
M src/storage/remote/protocol.c
M src/storage/remote/storage.c
M src/storage/s3/read.c
M src/storage/s3/storage.c
M src/storage/storage.c
M src/storage/write.c
M test/src/common/harnessInfo.c
M test/src/common/harnessLog.c
M test/src/common/harnessPq.c
M test/src/common/harnessPq.h
M test/src/common/harnessStorage.c
M test/src/common/harnessTest.h
M test/src/common/harnessTls.c
M test/src/module/command/archiveCommonTest.c
M test/src/module/command/archiveGetTest.c
M test/src/module/command/archivePushTest.c
M test/src/module/command/backupTest.c
M test/src/module/command/checkTest.c
M test/src/module/command/controlTest.c
M test/src/module/command/expireTest.c
M test/src/module/command/helpTest.c
M test/src/module/command/infoTest.c
M test/src/module/command/repoTest.c
M test/src/module/command/restoreTest.c
M test/src/module/command/stanzaTest.c
M test/src/module/common/compressTest.c
M test/src/module/common/execTest.c
M test/src/module/common/iniTest.c
M test/src/module/common/ioTest.c
M test/src/module/common/ioTlsTest.c
M test/src/module/common/lockTest.c
M test/src/module/common/logTest.c
M test/src/module/common/regExpTest.c
M test/src/module/common/typeStringTest.c
M test/src/module/config/configTest.c
M test/src/module/config/loadTest.c
M test/src/module/config/parseTest.c
M test/src/module/info/manifestTest.c
M test/src/module/performance/typeTest.c
M test/src/module/postgres/clientTest.c
M test/src/module/storage/azureTest.c
M test/src/module/storage/posixTest.c
M test/src/module/storage/remoteTest.c
M test/src/module/storage/s3Test.c

Fix issue with = character in file or database names.

commit   : 039d314438f7881e7c7062d645f7800ee2dd8872    
  
author   : David Steele <dwsteele@users.noreply.github.com>    
date     : Tue, 28 Jul 2020 14:00:23 -0400    
  
committer: GitHub <noreply@github.com>    
date     : Tue, 28 Jul 2020 14:00:23 -0400    

Click here for diff

The manifest uses the = character as the key/value separator so = characters in the key cause parsing errors and lead to an error or segfault.
  

  
Since the value must be valid JSON we can keep checking the value on the right side of the = and stop building the key when the value is valid. It's a bit hackish but it does seem to do the job without breaking the manifest format.
  

  
Unsurprisingly this makes parsing about 50% slower but it's still more than fast enough. Parsing 10 million key/values takes about 6.5s for the old code and 10s for the new code. Since the value is used as JSON downstream we can reclaim most of this time by just passing the JSON value rather than making the callback reparse it. We'll save that for another commit, though.

M doc/xml/release.xml
M src/common/ini.c
M test/src/module/common/iniTest.c
M test/src/module/info/manifestTest.c

Suppress errors when closing local/remote processes.

commit   : 63a93db6fd97c6e8e22478ee69b05487fa3db69d    
  
author   : David Steele <dwsteele@users.noreply.github.com>    
date     : Tue, 28 Jul 2020 12:15:33 -0400    
  
committer: GitHub <noreply@github.com>    
date     : Tue, 28 Jul 2020 12:15:33 -0400    

Click here for diff

Since the command has completed it is counterproductive to throw an error but still warn to indicate that something unusual happened.
  

  
Also fix the related issue that the local processes were not being shut down when they completed, which meant that they might timeout before being closed when pgbackrest terminated.

M doc/xml/release.xml
M src/common/exit.c
M src/protocol/helper.c
M src/protocol/helper.h
M src/protocol/parallel.c
M test/define.yaml
M test/src/module/protocol/protocolTest.c

Add quarterly press release instructions to doc/RELEASE.md.

commit   : d9309f13a4c7b962b401832881a593b92f29c438    
  
author   : David Steele <david@pgmasters.net>    
date     : Tue, 28 Jul 2020 11:20:07 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Tue, 28 Jul 2020 11:20:07 -0400    

Click here for diff

M doc/RELEASE.md

Add zstd to CentOS/RHEL 6 test container.

commit   : a260d4a53b68e40f8ebbdc9129143ac61d0ac88b    
  
author   : David Steele <david@pgmasters.net>    
date     : Tue, 28 Jul 2020 08:09:10 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Tue, 28 Jul 2020 08:09:10 -0400    

Click here for diff

Zstd is now required by the upstream yum package.  

M test/container.yaml
M test/lib/pgBackRestTest/Common/VmTest.pm

Add strBaseZ().

commit   : aa37a21084f75c7af8d8ca047be0376057a86bb6    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 25 Jul 2020 13:37:39 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 25 Jul 2020 13:37:39 -0400    

Click here for diff

Since the base name is at the end of the filename it is possible to return the char pointer without making a copy, which is preferred in some cases.  

M src/common/type/string.c
M src/common/type/string.h
M src/config/parse.c

Build new manifest inside a temporary context.

commit   : 8e140ad7e06f7e4abdd2c6fa32018ff463d63a6f    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 25 Jul 2020 11:39:24 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 25 Jul 2020 11:39:24 -0400    

Click here for diff

There is a fair amount of temporary data that is created during the build and it should not end up as a permanent part of the manifest.  

M src/info/manifest.c

Add manifestNewBuild() to performance/type test.

commit   : 8e776cc537117292504bc17a1691481fce9bae3d    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 25 Jul 2020 11:26:14 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 25 Jul 2020 11:26:14 -0400    

Click here for diff

Use a test storage driver to allow manifestNewBuild() to be run against a test cluster at any scale without having to write files to disk.  
  
Simplify the test by using the output of manifestNewBuild() to feed manifestSave() and manifestNewLoad().  
  
Also add manifest size to the output.  

M test/src/module/performance/typeTest.c

Add memContextSize().

commit   : f9d923ca3b2c7b2d3eacb3a3cd4fba347cef701e    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 25 Jul 2020 11:06:25 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 25 Jul 2020 11:06:25 -0400    

Click here for diff

Calculates the memory used by the context and all child contexts.  
  
This is primarily useful for debugging but it is not conditional on DEBUG because it is useful for profile/performance tests.  

M src/common/memContext.c
M src/common/memContext.h
M test/src/module/common/memContextTest.c

Move dummy storage driver to test harness.

commit   : 216a61d93626683acdcd0fbf2e145c8751aa7064    
  
author   : David Steele <david@pgmasters.net>    
date     : Sat, 25 Jul 2020 08:44:41 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Sat, 25 Jul 2020 08:44:41 -0400    

Click here for diff

The dummy driver is the basis for creating test storage drivers so it makes sense to locate it in the harness where all tests can access it.  

M test/src/common/harnessStorage.c
M test/src/common/harnessStorage.h
M test/src/module/performance/storageTest.c

Fix non-JSON ini values in tests.

commit   : cbf5dd19c2e273ce8ec65eab256bde9591fb0760    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 24 Jul 2020 11:05:09 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 24 Jul 2020 11:05:09 -0400    

Click here for diff

A number of tests used invalid JSON values where an error was expected or the value would be ignored.  
  
Update these tests to use valid JSON values so all values in the file can be validated even if they are not used.  

M test/src/module/common/iniTest.c
M test/src/module/info/infoBackupTest.c
M test/src/module/info/infoPgTest.c
M test/src/module/info/manifestTest.c
M test/src/module/performance/typeTest.c

Error in jsonToVar() when input not entirely consumed.

commit   : 9d8f6d9fc2a91f20dfc1d3c531459f37fa642a49    
  
author   : David Steele <david@pgmasters.net>    
date     : Fri, 24 Jul 2020 10:47:51 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Fri, 24 Jul 2020 10:47:51 -0400    

Click here for diff

Something like 3="string" would return an Int64 variant and ignore the invalid portion after the integer. Other JSON interface functions have this check but it was forgotten here.  
  
There are no current issues because of this but we want to be able to validate arbitrary JSON strings and this function was not working correctly for that usage.  

M src/common/type/json.c
M test/src/module/common/typeJsonTest.c

Add storage parameter to pgWalFromFile().

commit   : 78ef442a18d62368f4bd05d946660bdcfb7ea40b    
  
author   : Cynthia Shang <cynthia.shang@crunchydata.com>    
date     : Tue, 21 Jul 2020 16:28:05 -0400    
  
committer: GitHub <noreply@github.com>    
date     : Tue, 21 Jul 2020 16:28:05 -0400    

Click here for diff

M src/command/archive/push/file.c
M src/postgres/interface.c
M src/postgres/interface.h
M test/src/module/postgres/interfaceTest.c

Clarify that expire must be run regularly when expire-auto is disabled.

commit   : ed8829386103b27053c4a1d9e29f1d767d0238ce    
  
author   : David Steele <david@pgmasters.net>    
date     : Tue, 21 Jul 2020 10:57:47 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Tue, 21 Jul 2020 10:57:47 -0400    

Click here for diff

M doc/xml/reference.xml
M doc/xml/release.xml
M src/config/define.auto.c

Rearrange Travis CI jobs based on current runtimes.

commit   : d1ac2f7637866c2044f0d40068dcae2de373aa3b    
  
author   : David Steele <david@pgmasters.net>    
date     : Tue, 21 Jul 2020 07:34:02 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Tue, 21 Jul 2020 07:34:02 -0400    

Click here for diff

In general it is better to have slower jobs run earlier.  

M .travis.yml

Remove unused constructor strLstNewParam().

commit   : 90ffde8881f50ba8b91d9d9e881bf59c6488052e    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 18:09:32 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 18:09:32 -0400    

Click here for diff

This function is not used in the core code so remove it and update the test where it was used.  
  
There may eventually be a need for a strLstNewP() function but it doesn't seem worth the code churn until there is an actual requirement.  

M src/common/type/stringList.c
M src/common/type/stringList.h
M test/src/module/command/expireTest.c

Remove lstNew() constructor in favor of lstNewP().

commit   : 1783e0490abe266233e65ba1904670a7adde48bb    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 15:22:33 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 15:22:33 -0400    

Click here for diff

The old constructor was left around to reduce code churn during the migration but it just makes the code harder to read and search.  
  
Remove the old constructor and rename all remaining instances to lstNewP(), which by default has the same semantics.  

M src/command/backup/backup.c
M src/command/expire/expire.c
M src/command/restore/restore.c
M src/common/io/filter/group.c
M src/common/io/http/client.c
M src/common/type/keyValue.c
M src/common/type/list.c
M src/common/type/list.h
M src/common/type/mcv.c
M src/common/type/variantList.c
M src/common/type/xml.c
M src/db/protocol.c
M src/info/infoPg.c
M src/info/manifest.c
M src/protocol/parallel.c
M src/protocol/server.c
M test/src/common/harnessLog.c
M test/src/module/common/typeListTest.c
M test/src/module/protocol/protocolTest.c

Small wording change in website index / Github readme.

commit   : a92b1f9776acea52627d59fc734707ec90d78b4e    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 13:44:11 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 13:44:11 -0400    

Click here for diff

M doc/xml/index.xml

Update config.guess and config.sub to latest versions.

commit   : 738900a3a9e9fb7d63fbc5e4e4642977b8b65e2e    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 11:08:18 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 11:08:18 -0400    

Click here for diff

Also update the policy in doc/RELEASE.md to get the latest versions at the beginning of the release cycle. The older policy was created when we were getting new versions right before the release.  

M doc/RELEASE.md
M src/build/config.guess
M src/build/config.sub

Add support for testing on 64-bit big-endian architectures.

commit   : 3f0b41eb9c1137b0b19138d22d2c26009d477602    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 09:59:16 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 09:59:16 -0400    

Click here for diff

In particular add support for s390x but we hope this will work for other 64-bit big-endian architectures.  
  
Run basic unit tests on Travis CI for 390x.  

M .travis.yml
M test/src/common/harnessTest.h
M test/src/module/command/archivePushTest.c
M test/src/module/command/backupTest.c
M test/src/module/postgres/interfaceTest.c

Reduce reliance on static checksums in unit tests.

commit   : 55277357b8457e3a215bc00e8e97de523949eba6    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 09:47:43 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 09:47:43 -0400    

Click here for diff

Testing against static checksums is valuable but it can be become burdensome when supporting multiple architectures.  
  
Reduce the number of tests we are doing against static checksums when the architecture can cause the checksum to vary.  

M test/src/common/harnessTest.h
M test/src/module/command/archivePushTest.c
M test/src/module/command/backupTest.c

Fix ill-advised cast in CipherBlock.

commit   : f8c1957127c148e36843d77ef0bd26a8c8c78fd1    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 09:22:53 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 09:22:53 -0400    

Click here for diff

Little-endian architectures store the low-order bytes in the lowest memory location so this worked even in the case that size_t and int had different byte representations. Since buffer sizes are constrained there was no chance of the integer becoming negative and causing a problem that way.  
  
On big-endian architectures this cast caused the low-order bytes to get loaded into the high-order bytes resulting in a huge buffer size that immediately triggered an assertion (and without the assertion would have certainly segfaulted).  
  
Instead use a temporary int variable and cast that to size_t after the function call. This is the correct way to do it regardless of architecture.  
  
This issue was detected while testing on the s390x architecture.  

M src/common/crypto/cipherBlock.c

Begin v2.29 development.

commit   : 615c41e52564a8629e57b9892fd3361c92ae22f6    
  
author   : David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 09:08:59 -0400    
  
committer: David Steele <david@pgmasters.net>    
date     : Mon, 20 Jul 2020 09:08:59 -0400    

Click here for diff

M doc/resource/git-history.cache
M doc/xml/release.xml
M src/build/configure.ac
M src/configure
M src/version.h