Skip to main content

Browse the archive

https://github.com/postgres/postgres
07 April 2024, 13:56:23 UTC
sort by:
Revision Author Date Message Commit Date
4191e37 Tom Lane 06 August 2018, 20:05:31 UTC Stamp 10.5. 06 August 2018, 20:05:31 UTC
4ae1121 Peter Eisentraut 06 August 2018, 18:03:55 UTC Translation updates Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 3463878fc340cb1436153b18a300f8cfdcb12adb 06 August 2018, 18:03:55 UTC
9d3072d Tom Lane 06 August 2018, 17:13:41 UTC Last-minute updates for release notes. Security: CVE-2018-10915, CVE-2018-10925 06 August 2018, 17:13:41 UTC
ab54004 Tom Lane 06 August 2018, 14:53:35 UTC Fix failure to reset libpq's state fully between connection attempts. The logic in PQconnectPoll() did not take care to ensure that all of a PGconn's internal state variables were reset before trying a new connection attempt. If we got far enough in the connection sequence to have changed any of these variables, and then decided to try a new server address or server name, the new connection might be completed with some state that really only applied to the failed connection. While this has assorted bad consequences, the only one that is clearly a security issue is that password_needed didn't get reset, so that if the first server asked for a password and the second didn't, PQconnectionUsedPassword() would return an incorrect result. This could be leveraged by unprivileged users of dblink or postgres_fdw to allow them to use server-side login credentials that they should not be able to use. Other notable problems include the possibility of forcing a v2-protocol connection to a server capable of supporting v3, or overriding "sslmode=prefer" to cause a non-encrypted connection to a server that would have accepted an encrypted one. Those are certainly bugs but it's harder to paint them as security problems in themselves. However, forcing a v2-protocol connection could result in libpq having a wrong idea of the server's standard_conforming_strings setting, which opens the door to SQL-injection attacks. The extent to which that's actually a problem, given the prerequisite that the attacker needs control of the client's connection parameters, is unclear. These problems have existed for a long time, but became more easily exploitable in v10, both because it introduced easy ways to force libpq to abandon a connection attempt at a late stage and then try another one (rather than just giving up), and because it provided an easy way to specify multiple target hosts. Fix by rearranging PQconnectPoll's state machine to provide centralized places to reset state properly when moving to a new target host or when dropping and retrying a connection to the same host. Tom Lane, reviewed by Noah Misch. Our thanks to Andrew Krasichkov for finding and reporting the problem. Security: CVE-2018-10915 06 August 2018, 14:53:35 UTC
e155ddd Peter Eisentraut 11 June 2018, 21:19:11 UTC Adjust error message Makes it look more similar to other ones, and avoids the need for pluralization. 06 August 2018, 08:53:51 UTC
9468474 Tom Lane 05 August 2018, 20:38:43 UTC Release notes for 10.5, 9.6.10, 9.5.14, 9.4.19, 9.3.24. 05 August 2018, 20:38:43 UTC
aa72d54 Tom Lane 05 August 2018, 17:03:42 UTC Doc: fix incorrectly stated argument list for pgcrypto's hmac() function. The bytea variant takes (bytea, bytea, text). Per unsigned report. Discussion: https://postgr.es/m/153344327294.1404.654155870612982042@wrigleys.postgresql.org 05 August 2018, 17:04:05 UTC
f6a124d Tom Lane 04 August 2018, 23:38:58 UTC Fix INSERT ON CONFLICT UPDATE through a view that isn't just SELECT *. When expanding an updatable view that is an INSERT's target, the rewriter failed to rewrite Vars in the ON CONFLICT UPDATE clause. This accidentally worked if the view was just "SELECT * FROM ...", as the transformation would be a no-op in that case. With more complicated view targetlists, this omission would often lead to "attribute ... has the wrong type" errors or even crashes, as reported by Mario De Frutos Dieguez. Fix by adding code to rewriteTargetView to fix up the data structure correctly. The easiest way to update the exclRelTlist list is to rebuild it from scratch looking at the new target relation, so factor the code for that out of transformOnConflictClause to make it sharable. In passing, avoid duplicate permissions checks against the EXCLUDED pseudo-relation, and prevent useless view expansion of that relation's dummy RTE. The latter is only known to happen (after this patch) in cases where the query would fail later due to not having any INSTEAD OF triggers for the view. But by exactly that token, it would create an unintended and very poorly tested state of the query data structure, so it seems like a good idea to prevent it from happening at all. This has been broken since ON CONFLICT was introduced, so back-patch to 9.5. Dean Rasheed, based on an earlier patch by Amit Langote; comment-kibitzing and back-patching by me Discussion: https://postgr.es/m/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg@mail.gmail.com 04 August 2018, 23:38:58 UTC
7124e64 Michael Paquier 04 August 2018, 20:32:12 UTC Reset properly errno before calling write() 6cb3372 enforces errno to ENOSPC when less bytes than what is expected have been written when it is unset, though it forgot to properly reset errno before doing a system call to write(), causing errno to potentially come from a previous system call. Reported-by: Tom Lane Author: Michael Paquier Reviewed-by: Tom Lane Discussion: https://postgr.es/m/31797.1533326676@sss.pgh.pa.us 04 August 2018, 20:32:12 UTC
c83408a Peter Geoghegan 03 August 2018, 21:44:56 UTC Add table relcache invalidation to index builds. It's necessary to make sure that owning tables have a relcache invalidation prior to advancing the command counter to make newly-entered catalog tuples for the index visible. inval.c must be able to maintain the consistency of the local caches in the event of transaction abort. There is usually only a problem when CREATE INDEX transactions abort, since there is a generic invalidation once we reach index_update_stats(). This bug is of long standing. Problems were made much more likely by the addition of parallel CREATE INDEX (commit 9da0cc35284), but it is strongly suspected that similar problems can be triggered without involving plan_create_index_workers(). (plan_create_index_workers() triggers a relcache build or rebuild, which previously only happened in rare edge cases.) Author: Peter Geoghegan Reported-By: Luca Ferrari Diagnosed-By: Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAKoxK+5fVodiCtMsXKV_1YAKXbzwSfp7DgDqUmcUAzeAhf=HEQ@mail.gmail.com Backpatch: 9.3- 03 August 2018, 21:44:56 UTC
2e83837 Alvaro Herrera 03 August 2018, 20:45:08 UTC Add 'n' to list of possible values to pg_default_acl.defaclobjtype This was missed in commit ab89e465cb20; backpatch to v10. Author: Fabien Coelho <coelho@cri.ensmp.fr> Discussion: https://postgr.es/m/alpine.DEB.2.21.1807302243001.13230@lancre 03 August 2018, 20:45:08 UTC
d9c99c6 Alvaro Herrera 03 August 2018, 20:34:59 UTC Fix pg_replication_slot example output The example output of pg_replication_slot is wrong. Correct it and make the output stable by explicitly listing columns to output. Author: Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/20180731.190909.42582169.horiguchi.kyotaro@lab.ntt.co.jp 03 August 2018, 20:34:59 UTC
ed5d819 Tom Lane 03 August 2018, 16:20:47 UTC Remove no-longer-appropriate special case in psql's \conninfo code. \conninfo prints the results of PQhost() and some other libpq functions. It used to override the PQhost() result with the hostaddr parameter if that'd been given, but that's unhelpful when multiple hosts were listed in the connection string. Furthermore, it seems unnecessary in the wake of commit 1944cdc98, since PQhost does any useful substitution itself. So let's just remove the extra code and print PQhost()'s result without any editorialization. Back-patch to v10, as 1944cdc98 (just) was. Discussion: https://postgr.es/m/23287.1533227021@sss.pgh.pa.us 03 August 2018, 16:20:47 UTC
8d00858 Tom Lane 03 August 2018, 16:12:10 UTC Change libpq's internal uses of PQhost() to inspect host field directly. Commit 1944cdc98 changed PQhost() to return the hostaddr value when that is specified and host isn't. This is a good idea in general, but fe-auth.c and related files contain PQhost() calls for which it isn't. Specifically, when we compare SSL certificates or other server identity information to the host field, we do not want to use hostaddr instead; that's not what's documented, that's not what happened pre-v10, and it doesn't seem like a good idea. Instead, we can just look at connhost[].host directly. This does what we want in v10 and up; in particular, if neither host nor hostaddr were given, the host field will be replaced with the default host name. That seems useful, and it's likely the reason that these places were coded to call PQhost() originally (since pre-v10, the stored field was not replaced with the default). Back-patch to v10, as 1944cdc98 (just) was. Discussion: https://postgr.es/m/23287.1533227021@sss.pgh.pa.us 03 August 2018, 16:12:10 UTC
6203881 Tom Lane 03 August 2018, 15:30:34 UTC libpq: PQhost to return active connected host or hostaddr Previously, PQhost didn't return the connected host details when the connection type was CHT_HOST_ADDRESS (i.e., via hostaddr). Instead, it returned the complete host connection parameter (which could contain multiple hosts) or the default host details, which was confusing and arguably incorrect. Change this to return the actually connected host or hostaddr irrespective of the connection type. When hostaddr but no host was specified, hostaddr is now returned. Never return the original host connection parameter, and document that PQhost cannot be relied on before the connection is established. PQport is similarly changed to always return the active connection port and never the original connection parameter. Back-patch of commit 1944cdc98273dbb8439ad9b387ca2858531afcf0 into the v10 branch. Author: Hari Babu <kommi.haribabu@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> Reviewed-by: David G. Johnston <david.g.johnston@gmail.com> 03 August 2018, 15:30:34 UTC
b805b63 Amit Kapila 03 August 2018, 05:57:11 UTC Fix buffer usage stats for parallel nodes. The buffer usage stats is accounted only for the execution phase of the node. For Gather and Gather Merge nodes, such stats are accumulated at the time of shutdown of workers which is done after execution of node due to which we missed to account them for such nodes. Fix it by treating nodes as running while we shut down them. We can also miss accounting for a Limit node when Gather or Gather Merge is beneath it, because it can finish the execution before shutting down such nodes. So we allow a Limit node to shut down the resources before it completes the execution. In the passing fix the gather node code to allow workers to shut down as soon as we find that all the tuples from the workers have been retrieved. The original code use to do that, but is accidently removed by commit 01edb5c7fc. Reported-by: Adrien Nayrat Author: Amit Kapila and Robert Haas Reviewed-by: Robert Haas and Andres Freund Backpatch-through: 9.6 where this code was introduced Discussion: https://postgr.es/m/86137f17-1dfb-42f9-7421-82fd786b04a1@anayrat.info 03 August 2018, 05:57:11 UTC
3f02b51 Amit Kapila 03 August 2018, 04:20:24 UTC Match the buffer usage tracking for leader and worker backends. In the leader backend, we don't track the buffer usage for ExecutorStart phase whereas in worker backend we track it for ExecutorStart phase as well. This leads to different value for buffer usage stats for the parallel and non-parallel query. Change the code so that worker backend also starts tracking buffer usage after ExecutorStart. Author: Amit Kapila and Robert Haas Reviewed-by: Robert Haas and Andres Freund Backpatch-through: 9.6 where this code was introduced Discussion:https://postgr.es/m/86137f17-1dfb-42f9-7421-82fd786b04a1@anayrat.info 03 August 2018, 04:20:24 UTC
71e3b28 Tom Lane 01 August 2018, 16:30:36 UTC Fix libpq's code for searching .pgpass; rationalize empty-list-item cases. Before v10, we always searched ~/.pgpass using the host parameter, and nothing else, to match to the "hostname" field of ~/.pgpass. (However, null host or host matching DEFAULT_PGSOCKET_DIR was replaced by "localhost".) In v10, this got broken by commit 274bb2b38, repaired by commit bdac9836d, and broken again by commit 7b02ba62e; in the code actually shipped, we'd search with hostaddr if both that and host were specified --- though oddly, *not* if only hostaddr were specified. Since this is directly contrary to the documentation, and not backwards-compatible, it's clearly a bug. However, the change wasn't totally without justification, even though it wasn't done quite right, because the pre-v10 behavior has arguably been buggy since we added hostaddr. If hostaddr is specified and host isn't, the pre-v10 code will search ~/.pgpass for "localhost", and ship that password off to a server that most likely isn't local at all. That's unhelpful at best, and could be a security breach at worst. Therefore, rather than just revert to that old behavior, let's define the behavior as "search with host if provided, else with hostaddr if provided, else search for localhost". (As before, a host name matching DEFAULT_PGSOCKET_DIR is replaced by localhost.) This matches the behavior of the actual connection code, so that we don't pick up an inappropriate password; and it allows useful searches to happen when only hostaddr is given. While we're messing around here, ensure that empty elements within a host or hostaddr list select the same behavior as a totally-empty field would; for instance "host=a,,b" is equivalent to "host=a,/tmp,b" if DEFAULT_PGSOCKET_DIR is /tmp. Things worked that way in some cases already, but not consistently so, which contributed to the confusion about what key ~/.pgpass would get searched with. Update documentation accordingly, and also clarify some nearby text. Back-patch to v10 where the host/hostaddr list functionality was introduced. Discussion: https://postgr.es/m/30805.1532749137@sss.pgh.pa.us 01 August 2018, 16:30:36 UTC
d8dd8d2 Bruce Momjian 31 July 2018, 22:10:06 UTC pg_upgrade: fix --check for live source server checks Fix for commit 244142d32afd02e7408a2ef1f249b00393983822. Backpatch-through: 9.3 31 July 2018, 22:10:06 UTC
31b29b1 Tom Lane 31 July 2018, 17:00:08 UTC Further fixes for quoted-list GUC values in pg_dump and ruleutils.c. Commits 742869946 et al turn out to be a couple bricks shy of a load. We were dumping the stored values of GUC_LIST_QUOTE variables as they appear in proconfig or setconfig catalog columns. However, although that quoting rule looks a lot like SQL-identifier double quotes, there are two critical differences: empty strings ("") are legal, and depending on which variable you're considering, values longer than NAMEDATALEN might be valid too. So the current technique fails altogether on empty-string list entries (as reported by Steven Winfield in bug #15248) and it also risks truncating file pathnames during dump/reload of GUC values that are lists of pathnames. To fix, split the stored value without any downcasing or truncation, and then emit each element as a SQL string literal. This is a tad annoying, because we now have three copies of the comma-separated-string splitting logic in varlena.c as well as a fourth one in dumputils.c. (Not to mention the randomly-different-from-those splitting logic in libpq...) I looked at unifying these, but it would be rather a mess unless we're willing to tweak the API definitions of SplitIdentifierString, SplitDirectoriesString, or both. That might be worth doing in future; but it seems pretty unsafe for a back-patched bug fix, so for now accept the duplication. Back-patch to all supported branches, as the previous fix was. Discussion: https://postgr.es/m/7585.1529435872@sss.pgh.pa.us 31 July 2018, 17:00:08 UTC
2c4d0f3 Alvaro Herrera 30 July 2018, 20:30:07 UTC Set ActiveSnapshot when logically replaying inserts Input functions for the inserted tuples may require a snapshot, when they are replayed by native logical replication. An example is a domain with a constraint using a SQL-language function, which prior to this commit failed to apply on the subscriber side. Reported-by: Mai Peng <maily.peng@webedia-group.com> Co-authored-by: Minh-Quan TRAN <qtran@itscaro.me> Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m/4EB4BD78-BFC3-4D04-B8DA-D53DF7160354@webedia-group.com Discussion: https://postgr.es/m/153211336163.1404.11721804383024050689@wrigleys.postgresql.org 30 July 2018, 20:30:07 UTC
96b1d98 Tom Lane 30 July 2018, 16:35:49 UTC Fix pg_dump's failure to dump REPLICA IDENTITY for constraint indexes. pg_dump knew about printing ALTER TABLE ... REPLICA IDENTITY USING INDEX for indexes declared as indexes, but it failed to print that for indexes declared as unique or primary-key constraints. Per report from Achilleas Mantzios. This has been broken since the feature was introduced, AFAICS. Back-patch to 9.4. Discussion: https://postgr.es/m/1e6cc5ad-b84a-7c07-8c08-a4d0c3cdc938@matrix.gatewaynet.com 30 July 2018, 16:35:49 UTC
406df42 Tom Lane 30 July 2018, 15:54:41 UTC Doc: fix oversimplified example for CREATE POLICY. As written, this policy constrained only the post-image not the pre-image of rows, meaning that users could delete other users' rows or take ownership of such rows, contrary to what the docs claimed would happen. We need two separate policies to achieve the documented effect. While at it, try to explain what's happening a bit more fully. Per report from Олег Самойлов. Back-patch to 9.5 where this was added. Thanks to Stephen Frost for off-list discussion. Discussion: https://postgr.es/m/3298321532002010@sas1-2b3c3045b736.qloud-c.yandex.net 30 July 2018, 15:54:41 UTC
e768869 Noah Misch 29 July 2018, 19:02:07 UTC Fix earthdistance test suite function name typo. Affected test queries have been testing the wrong thing since their introduction in commit 4c1383efd132e4f532213c8a8cc63a455f55e344. Back-patch to 9.3 (all supported versions). 29 July 2018, 19:02:10 UTC
ca4a6de Noah Misch 29 July 2018, 03:08:01 UTC Document security implications of qualified names. Commit 5770172cb0c9df9e6ce27c507b449557e5b45124 documented secure schema usage, and that advice suffices for using unqualified names securely. Document, in typeconv-func primarily, the additional issues that arise with qualified names. Back-patch to 9.3 (all supported versions). Reviewed by Jonathan S. Katz. Discussion: https://postgr.es/m/20180721012446.GA1840594@rfd.leadboat.com 29 July 2018, 03:08:21 UTC
d524a11 Bruce Momjian 28 July 2018, 19:34:06 UTC pgtest: run clean, build, and check stages separately This allows for cleaner error reporting. Backpatch-through: 9.5 28 July 2018, 19:34:06 UTC
9a13e7f Bruce Momjian 28 July 2018, 19:01:55 UTC pg_upgrade: check for clean server shutdowns Previously pg_upgrade checked for the pid file and started/stopped the server to force a clean shutdown. However, "pg_ctl -m immediate" removes the pid file but doesn't do a clean shutdown, so check pg_controldata for a clean shutdown too. Diagnosed-by: Vimalraj A Discussion: https://postgr.es/m/CAFKBAK5e4Q-oTUuPPJ56EU_d2Rzodq6GWKS3ncAk3xo7hAsOZg@mail.gmail.com Backpatch-through: 9.3 28 July 2018, 19:01:55 UTC
fe25526 Bruce Momjian 28 July 2018, 15:35:52 UTC pgtest: grab possible warnings from install.log Since PG 9.5, 'make check' records the build output in install.log, so look in there for warnings too. Backpatch-through: 9.5 28 July 2018, 15:35:52 UTC
ff8ce0b Amit Kapila 27 July 2018, 05:35:06 UTC Fix the buffer release order for parallel index scans. During parallel index scans, if the current page to be read is deleted, we skip it and try to get the next page for a scan without releasing the buffer lock on the current page. To get the next page, sometimes it needs to wait for another process to complete its scan and advance it to the next page. Now, it is quite possible that the master backend has errored out before advancing the scan and issued a termination signal for all workers. The workers failed to notice the termination request during wait because the interrupts are held due to buffer lock on the previous page. This lead to all workers being stuck. The fix is to release the buffer lock on current page before trying to get the next page. We are already doing same in backward scans, but missed it for forward scans. Reported-by: Victor Yegorov Bug: 15290 Diagnosed-by: Thomas Munro and Amit Kapila Author: Amit Kapila Reviewed-by: Thomas Munro Tested-By: Thomas Munro and Victor Yegorov Backpatch-through: 10 where parallel index scans were introduced Discussion: https://postgr.es/m/153228422922.1395.1746424054206154747@wrigleys.postgresql.org 27 July 2018, 05:35:06 UTC
46201d6 Thomas Munro 24 July 2018, 22:58:54 UTC Pad semaphores to avoid false sharing. In a USE_UNNAMED_SEMAPHORES build, the default on Linux and FreeBSD since commit ecb0d20a, we have an array of sem_t objects. This turned out to reduce performance compared to the previous default USE_SYSV_SEMAPHORES on an 8 socket system. Testing showed that the lost performance could be regained by padding the array elements so that they have their own cache lines. This matches what we do for similar hot arrays (see LWLockPadded, WALInsertLockPadded). Back-patch to 10, where unnamed semaphores were adopted as the default semaphore interface on those operating systems. Author: Thomas Munro Reviewed-by: Andres Freund Reported-by: Mithun Cy Tested-by: Mithun Cy, Tom Lane, Thomas Munro Discussion: https://postgr.es/m/CAD__OugYDM3O%2BdyZnnZSbJprSfsGFJcQ1R%3De59T3hcLmDug4_w%40mail.gmail.com 24 July 2018, 23:00:53 UTC
360cbf1 Andres Freund 24 July 2018, 17:51:09 UTC doc: Fix reference to "decoder" to instead be the correct "output plugin". Author: Jonathan Katz Discussion: https://postgr.es/m/DD02DD86-5989-4BFD-8712-468541F68383@postgresql.org Backpatch: 9.4-, where logical decoding was added 24 July 2018, 17:51:09 UTC
dfbad3f Tom Lane 21 July 2018, 19:40:51 UTC Further portability hacking in pg_upgrade's test script. I blew the dust off a Bourne shell (file date 1996, yea verily) and tried to run test.sh with it. It mostly worked, but I found that the temp-directory creation code introduced by commit be76a6d39 was not compatible, for a couple of reasons: this shell thinks "set -e" should force an exit if a command within backticks fails, and it also thinks code within braces should be executed by a sub-shell, meaning that variable settings don't propagate back up to the parent shell. In view of Victor Wagner's report that Solaris is still using pre-POSIX shells, seems like we oughta make this case work. It's not like the code is any less idiomatic this way; the prior coding technique appeared nowhere else. (There is a remaining bash-ism here, which is that $RANDOM doesn't do what the code hopes in non-bash shells. But the use of $$ elsewhere in that path should be enough to ensure uniqueness and some amount of randomness, so I think it's okay as-is.) Back-patch to all supported branches, as the previous commit was. Discussion: https://postgr.es/m/20180720153820.69e9ae6c@fafnir.local.vm 21 July 2018, 19:40:51 UTC
8212004 Dean Rasheed 20 July 2018, 07:58:37 UTC Guard against rare RAND_bytes() failures in pg_strong_random(). When built using OpenSSL, pg_strong_random() uses RAND_bytes() to generate the random number. On very rare occasions that can fail, if its PRNG has not been seeded with enough data. Additionally, once it does fail, all subsequent calls will also fail until more seed data is added. Since this is required during backend startup, this can result in all new backends failing to start until a postmaster restart. Guard against that by checking the state of OpenSSL's PRNG using RAND_status(), and if necessary (very rarely), seeding it using RAND_poll(). Back-patch to v10, where pg_strong_random() was introduced. Dean Rasheed and Michael Paquier. Discussion: https://postgr.es/m/CAEZATCXMtxbzSAvyKKk5uCRf9pNt4UV%2BF_5v%3DgLfJUuPxU4Ytg%40mail.gmail.com 20 July 2018, 07:58:37 UTC
2131d45 Tom Lane 19 July 2018, 19:41:46 UTC Remove undocumented restriction against duplicate partition key columns. transformPartitionSpec rejected duplicate simple partition columns (e.g., "PARTITION BY RANGE (x,x)") but paid no attention to expression columns, resulting in inconsistent behavior. Worse, cases like "PARTITION BY RANGE (x,(x))") were accepted but would then result in dump/reload failures, since the expression (x) would get simplified to a plain column later. There seems no better reason for this restriction than there was for the one against duplicate included index columns (cf commit 701fd0bbc), so let's just remove it. Back-patch to v10 where this code was added. Report and patch by Yugo Nagata. Discussion: https://postgr.es/m/20180712165939.36b12aff.nagata@sraoss.co.jp 19 July 2018, 19:41:46 UTC
0d26812 Alexander Korotkov 19 July 2018, 18:04:17 UTC Fix handling of empty uncompressed posting list pages in GIN PostgreSQL 9.4 introduces posting list compression in GIN. This feature supports online upgrade, so that after pg_upgrade uncompressed posting lists are compressed on-the-fly. Underlying code appears to always expect at least one item on uncompressed posting list page. But there could be completely empty pages, because VACUUM never deletes leftmost and rightmost pages from posting trees. This commit fixes that. Reported-by: Sivasubramanian Ramasubramanian Discussion: https://postgr.es/m/1531867212836.63354%40amazon.com Author: Sivasubramanian Ramasubramanian, Alexander Korotkov Backpatch-through: 9.4 19 July 2018, 18:12:43 UTC
ff4fb4c Heikki Linnakangas 19 July 2018, 17:24:29 UTC Fix error message when a hostaddr cannot be parsed. We were incorrectly passing hostname, not hostaddr, in the error message, and because of that, you got: $ psql 'hostaddr=foo' psql: could not parse network address "(null)": Name or service not known Backpatch to v10, where this was broken (by commit 7b02ba62e9). Report and fix by Robert Haas. Discussion: https://www.postgresql.org/message-id/CA+TgmoapFQA30NomGKEaZCu3iN7mF7fux8fbbk9SouVOT2JP7w@mail.gmail.com 19 July 2018, 17:25:05 UTC
a1dc4ea Heikki Linnakangas 19 July 2018, 13:08:09 UTC Rephrase a few comments for clarity. I was confused by what "intended to be parallel serially" meant, until Robert Haas and David G. Johnston explained it. Rephrase the comment to make it more clear, using David's suggested wording. Discussion: https://www.postgresql.org/message-id/1fec9022-41e8-e484-70ce-2179b08c2092%40iki.fi 19 July 2018, 13:08:49 UTC
49d506d Michael Paquier 19 July 2018, 00:55:15 UTC Fix print of Path nodes when using OPTIMIZER_DEBUG GatherMergePath (introduced in 10) and CustomPath (introduced in 9.5) have gone missing. The order of the Path nodes was inconsistent with what is listed in nodes.h, so make the order consistent at the same time to ease future checks and additions. Author: Sawada Masahiko Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/CAD21AoBQMLoc=ohH-oocuAPsELrmk8_EsRJjOyR8FQLZkbE0wA@mail.gmail.com 19 July 2018, 00:55:15 UTC
5c513db Tom Lane 18 July 2018, 21:39:27 UTC Remove race-prone hot_standby_feedback test cases in 001_stream_rep.pl. This script supposed that if it turned hot_standby_feedback on and then shut down the standby server, at least one feedback message would be guaranteed to be sent before the standby stops. But there is no such guarantee, if the standby's walreceiver process is slow enough --- and we've seen multiple failures in the buildfarm showing that that does happen in practice. While we could rearrange the walreceiver logic to make it less likely, it seems probably impossible to create a really bulletproof guarantee of that sort; and if we tried, we might create situations where the walreceiver wouldn't react in a timely manner to shutdown commands. It seems better instead to remove the script's assumption that feedback will occur before shutdown. But once we do that, these last few tests seem quite redundant with the earlier tests in the script. So let's just drop them altogether and save some buildfarm cycles. Backpatch to v10 where these tests were added. Discussion: https://postgr.es/m/1922.1531592205@sss.pgh.pa.us 18 July 2018, 21:39:27 UTC
ed529fa Heikki Linnakangas 09 July 2018, 13:10:44 UTC Fix misc typos, mostly in comments. A collection of typos I happened to spot while reading code, as well as grepping for common mistakes. Backpatch to all supported versions, as applicable, to avoid conflicts when backporting other commits in the future. 18 July 2018, 13:18:27 UTC
4beb25c Robert Haas 16 July 2018, 21:33:22 UTC Add subtransaction handling for table synchronization workers. Since the old logic was completely unaware of subtransactions, a change made in a subsequently-aborted subtransaction would still cause workers to be stopped at toplevel transaction commit. Fix that by managing a stack of worker lists rather than just one. Amit Khandekar and Robert Haas Discussion: http://postgr.es/m/CAJ3gD9eaG_mWqiOTA2LfAug-VRNn1hrhf50Xi1YroxL37QkZNg@mail.gmail.com 16 July 2018, 21:55:13 UTC
0bb28ca Tom Lane 14 July 2018, 15:59:12 UTC Fix hashjoin costing mistake introduced with inner_unique optimization. In final_cost_hashjoin(), commit 9c7f5229a allowed inner_unique cases to follow a code path previously used only for SEMI/ANTI joins; but it neglected to fix an if-test within that path that assumed SEMI and ANTI were the only possible cases. This resulted in a wrong value for hashjointuples, and an ensuing bad cost estimate, for inner_unique normal joins. Fortunately, for inner_unique normal joins we can assume the number of joined tuples is the same as for a SEMI join; so there's no need for more code, we just have to invert the test to check for ANTI not SEMI. It turns out that in two contrib tests in which commit 9c7f5229a changed the plan expected for a query, the change was actually wrong and induced by this estimation error, not by any real improvement. Hence this patch also reverts those changes. Per report from RK Korlapati. Backpatch to v10 where the error was introduced. David Rowley Discussion: https://postgr.es/m/CA+SNy03bhq0fodsfOkeWDCreNjJVjsdHwUsb7AG=jpe0PtZc_g@mail.gmail.com 14 July 2018, 15:59:12 UTC
1f47eb0 Tom Lane 13 July 2018, 22:45:30 UTC Fix crash in contrib/ltree's lca() function for empty input array. lca_inner() wasn't prepared for the possibility of getting no inputs. Fix that, and make some cosmetic improvements to the code while at it. Also, I thought the documentation of this function as returning the "longest common prefix" of the paths was entirely misleading; it really returns a path one shorter than the longest common prefix, for the typical definition of "prefix". Don't use that term in the docs, and adjust the examples to clarify what really happens. This has been broken since its beginning, so back-patch to all supported branches. Per report from Hailong Li. Thanks to Pierre Ducroquet for diagnosing and for the initial patch, though I whacked it around some and added test cases. Discussion: https://postgr.es/m/5b0d8e4f-f2a3-1305-d612-e00e35a7be66@qunar.com 13 July 2018, 22:45:30 UTC
f1963a1 Tom Lane 13 July 2018, 15:52:50 UTC Fix inadequate buffer locking in FSM and VM page re-initialization. When reading an existing FSM or VM page that was found to be corrupt by the buffer manager, the code applied PageInit() to reinitialize the page, but did so without any locking. There is thus a hazard that two backends might concurrently do PageInit, which in itself would still be OK, but the slower one might then zero over subsequent data changes applied by the faster one. Even that is unlikely to be fatal; but it's not desirable, so add locking to prevent it. This does not add any locking overhead in the normal code path where the page is OK. It's not immediately obvious that that's safe, but I believe it is, for reasons explained in the added comments. Problem noted by R P Asim. It's been like this for a long time, so back-patch to all supported branches. Discussion: https://postgr.es/m/CANXE4Te4G0TGq6cr0-TvwP0H4BNiK_-hB5gHe8mF+nz0mcYfMQ@mail.gmail.com 13 July 2018, 15:53:12 UTC
c81e49b Bruce Momjian 13 July 2018, 15:16:55 UTC docs: Remove "New" description of the libpqxx interface Backpatch-through: 9.3 13 July 2018, 15:16:55 UTC
93532db Tom Lane 12 July 2018, 16:28:43 UTC Doc: minor improvement in pl/pgsql FETCH/MOVE documentation. Explain that you can use any integer expression for the "count" in pl/pgsql's versions of FETCH/MOVE, unlike the SQL versions which only allow a constant. Remove the duplicate version of this para under MOVE. I don't see a good reason to maintain two identical paras when we just said that MOVE works exactly like FETCH. Per Pavel Stehule, though I didn't use his text. Discussion: https://postgr.es/m/CAFj8pRAcvSXcNdUGx43bOK1e3NNPbQny7neoTLN42af+8MYWEA@mail.gmail.com 12 July 2018, 16:28:43 UTC
11abea3 Michael Paquier 12 July 2018, 01:20:08 UTC Make logical WAL sender report streaming state appropriately WAL senders sending logically-decoded data fail to properly report in "streaming" state when starting up, hence as long as one extra record is not replayed, such WAL senders would remain in a "catchup" state, which is inconsistent with the physical cousin. This can be easily reproduced by for example using pg_recvlogical and restarting the upstream server. The TAP tests have been slightly modified to detect the failure and strengthened so as future tests also make sure that a node is in streaming state when waiting for its catchup. Backpatch down to 9.4 where this code has been introduced. Reported-by: Sawada Masahiko Author: Simon Riggs, Sawada Masahiko Reviewed-by: Petr Jelinek, Michael Paquier, Vaishnavi Prabakaran Discussion: https://postgr.es/m/CAD21AoB2ZbCCqOx=bgKMcLrAvs1V0ZMqzs7wBTuDySezTGtMZA@mail.gmail.com 12 July 2018, 01:20:08 UTC
c350320 Tom Lane 11 July 2018, 19:25:29 UTC Fix create_scan_plan's handling of sortgrouprefs for physical tlists. We should only run apply_pathtarget_labeling_to_tlist if CP_LABEL_TLIST was specified, because only in that case has use_physical_tlist checked that the labeling will succeed; otherwise we may get an "ORDER/GROUP BY expression not found in targetlist" error. (This subsumes the previous test about gating_clauses, because we reset "flags" to zero earlier if there are gating clauses to apply.) The only known case in which a failure can occur is with a ProjectSet path directly atop a table scan path, although it seems likely that there are other cases or will be such in future. This means that the failure is currently only visible in the v10 branch: 9.6 didn't have ProjectSet, while in v11 and HEAD, apply_scanjoin_target_to_paths for some weird reason is using create_projection_path not apply_projection_to_path, masking the problem because there's a ProjectionPath in between. Nonetheless this code is clearly wrong on its own terms, so back-patch to 9.6 where this logic was introduced. Per report from Regina Obe. Discussion: https://postgr.es/m/001501d40f88$75186950$5f493bf0$@pcorp.us 11 July 2018, 19:25:29 UTC
7c644b7 Alvaro Herrera 10 July 2018, 19:07:28 UTC Better handle pseudotypes as partition keys We fail to handle polymorphic types properly when they are used as partition keys: we were unnecessarily adding a RelabelType node on top, which confuses code examining the nodes. In particular, this makes predtest.c-based partition pruning not to work, and ruleutils.c to emit expressions that are uglier than needed. Fix it by not adding RelabelType when not needed. In master/11 the new pruning code is separate so it doesn't suffer from this problem, since we already fixed it (in essentially the same way) in e5dcbb88a15d, which also added a few tests; back-patch those tests to pg10 also. But since UPDATE/DELETE still uses predtest.c in pg11, this change improves partitioning for those cases too. Add tests for this. The ruleutils.c behavior change is relevant in pg11/master too. Co-authored-by: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Robert Haas <robertmhaas@gmail.com> Discussion: https://postgr.es/m/54745d13-7ed4-54ac-97d8-ea1eec95ae25@lab.ntt.co.jp 10 July 2018, 19:07:28 UTC
7f0911f Peter Eisentraut 10 July 2018, 09:14:53 UTC Fix typos 10 July 2018, 09:15:59 UTC
59b2dcb Tom Lane 09 July 2018, 23:26:19 UTC Avoid emitting a bogus WAL record when recycling an all-zero btree page. Commit fafa374f2 caused _bt_getbuf() to possibly emit a WAL record for a page that it was about to recycle. However, it failed to distinguish all-zero pages from dead pages, which is important because only the latter have valid btpo.xact values, or indeed any special space at all. Recycling an all-zero page with XLogStandbyInfoActive() enabled therefore led to an Assert failure, or to emission of a WAL record containing a bogus cutoff XID, which might lead to unnecessary query cancellations on hot standby servers. Per reports from Antonin Houska and 自己. Amit Kapila was first to propose this fix, and Robert Haas, myself, and Kyotaro Horiguchi reviewed it at various times. This is an old bug, so back-patch to all supported branches. Discussion: https://postgr.es/m/2628.1474272158@localhost Discussion: https://postgr.es/m/48875502.f4a0.1635f0c27b0.Coremail.zoulx1982@163.com 09 July 2018, 23:26:19 UTC
c74f48a Tom Lane 09 July 2018, 21:23:31 UTC Prevent accidental linking of system-supplied copies of libpq.so etc. Back-patch commit dddfc4cb2, which broke LDFLAGS and related Makefile variables into two parts, one for within-build-tree library references and one for external libraries, to ensure that the order of -L flags has all of the former before all of the latter. This turns out to fix a problem recently noted on buildfarm member peripatus, that we attempted to incorporate code from libpgport.a into a shared library. That will fail on platforms that are sticky about putting non-PIC code into shared libraries. (It's quite surprising we hadn't seen such failures before, since the code in question has been like that for a long time.) I think that peripatus' problem could have been fixed with just a subset of this patch; but since the previous issue of accidentally linking to the wrong copy of a Postgres shlib seems likely to bite people in the field, let's just back-patch the whole change. Now that commit dddfc4cb2 has survived some beta testing, I'm less afraid to back-patch it than I was at the time. This also fixes undesired inclusion of "-DFRONTEND" in pg_config's CPPFLAGS output (in 9.6 and up) and undesired inclusion of "-L../../src/common" in its LDFLAGS output (in all supported branches). Back-patch to v10 and older branches; this is already in v11. Discussion: https://postgr.es/m/20180704234304.bq2dxispefl65odz@ler-imac.local 09 July 2018, 21:23:31 UTC
b0da7ec Bruce Momjian 09 July 2018, 15:19:18 UTC rel notes: mention enabling of parallelism in PG 10 Reported-by: Justin Pryzby Discussion: https://postgr.es/m/20180525010025.GT30060@telsasoft.com Backpatch-through: 10 09 July 2018, 15:19:18 UTC
c030db3 Michael Paquier 09 July 2018, 01:26:18 UTC Rework order of end-of-recovery actions to delay timeline history write A critical failure in some of the end-of-recovery actions before the end-of-recovery record is written can cause PostgreSQL to react inconsistently with the rest of the cluster in the event of a crash before the final record is written. Two such failures are for example an error while processing a two-phase state files or when operating on recovery.conf. With this commit, the failures are still considered FATAL, but the write of the timeline history file is delayed as much as possible so as the window between the moment the file is written and the end-of-recovery record is generated gets minimized. This way, in the event of a crash or a failure, the new timeline decided at promotion will not seem taken by other nodes in the cluster. It is not really possible to reduce to zero this window, hence one could still see failures if a crash happens between the history file write and the end-of-recovery record, so any future code should be careful when adding new end-of-recovery actions. The original report from Magnus Hagander mentioned a renamed recovery.conf as original end-of-recovery failure which caused a timeline to be seen as taken but the subsequent processing on the now-missing recovery.conf cause the startup process to issue stop on FATAL, which at follow-up startup made the system inconsistent because of on-disk changes which already happened. Processing of two-phase state files still needs some work as corrupted entries are simply ignored now. This is left as a future item and this commit fixes the original complain. Reported-by: Magnus Hagander Author: Heikki Linnakangas Reviewed-by: Alexander Korotkov, Michael Paquier, David Steele Discussion: https://postgr.es/m/CABUevEz09XY2EevA2dLjPCY-C5UO4Hq=XxmXLmF6ipNFecbShQ@mail.gmail.com 09 July 2018, 01:26:18 UTC
4230397 Michael Paquier 06 July 2018, 23:10:39 UTC Add note in pg_rewind documentation about read-only files When performing pg_rewind, the presence of a read-only file which is not accessible for writes will cause a failure while processing. This can cause the control file of the target data folder to be truncated, causing it to not be reusable with a successive run. Also, when pg_rewind fails mid-flight, there is likely no way to be able to recover the target data folder anyway, in which case a new base backup is the best option. A note is added in the documentation as well about. Reported-by: Christian H. Author: Michael Paquier Reviewed-by: Andrew Dunstan Discussion: https://postgr.es/m/20180104200633.17004.16377%40wrigleys.postgresql.org 06 July 2018, 23:10:39 UTC
a1f680d Alvaro Herrera 06 July 2018, 20:38:29 UTC Allow replication slots to be dropped in single-user mode Starting with commit 9915de6c1cb2, replication slot drop uses a condition variable sleep to wait until the current user of the slot goes away. This is more user friendly than the previous behavior of erroring out if the slot is in use, but it fails with a not-for-user-consumption error message in single-user mode; plus, if you're using single-user mode because you don't want to start the server in the regular mode (say, disk is full and WAL won't recycle because of the slot), it's inconvenient. Fix by skipping the cond variable sleep in single-user mode, since there can't be anybody to wait for anyway. Reported-by: tushar <tushar.ahuja@enterprisedb.com> Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m/3b2f809f-326c-38dd-7a9e-897f957a4eb1@enterprisedb.com 06 July 2018, 20:38:29 UTC
bba8c61 Alvaro Herrera 05 July 2018, 21:42:37 UTC logical decoding: beware of an unset specinsert change Coverity complains that there is no protection in the code (at least in non-assertion-enabled builds) against speculative insertion failing to follow the expected protocol. Add an elog(ERROR) for the case. 05 July 2018, 21:42:37 UTC
7acbb48 Alvaro Herrera 05 July 2018, 20:37:32 UTC Reduce cost of test_decoding's new oldest_xmin test Change a whole-database VACUUM into doing just pg_attribute, which is the portion that verifies what we want it to do. The original formulation wastes a lot of CPU time, which leads the test to fail when runtime exceeds isolationtester timeout when it's super-slow, such as under CLOBBER_CACHE_ALWAYS. Per buildfarm member friarbird. It turns out that the previous shape of the test doesn't always detect the condition it is supposed to detect (on unpatched reorderbuffer code): the reason is that there is a good chance of encountering a xl_running_xacts record (logged every 15 seconds) before the checkpoint -- and because we advance the xmin when we receive that WAL record, and we *don't* advance the xmin twice consecutively without receiving a client message in between, that means the xmin is not advanced enough for the tuple to be pruned from pg_attribute by VACUUM. So the test would spuriously pass. The reason this test deficiency wasn't detected earlier is that HOT pruning removes the tuple anyway, even if vacuum leaves it in place, so the test correctly fails (detecting the coding mistake), but for the wrong reason. To fix this mess, run the s0_get_changes step twice before vacuum instead of once: this seems to cause the xmin to be advanced reliably, wreaking havoc with more certainty. Author: Arseny Sher Discussion: https://postgr.es/m/87h8lkuxoa.fsf@ars-thinkpad 05 July 2018, 20:58:30 UTC
b1def47 Peter Eisentraut 05 July 2018, 20:52:19 UTC doc: Fix typos Author: Justin Pryzby <pryzby@telsasoft.com> 05 July 2018, 20:52:19 UTC
fbbe29b Peter Eisentraut 03 July 2018, 18:18:56 UTC Fix typo 05 July 2018, 06:20:15 UTC
a197434 Peter Eisentraut 27 June 2018, 05:51:20 UTC doc: Improve wording and fix whitespace 05 July 2018, 06:20:15 UTC
6716f2f Michael Paquier 05 July 2018, 01:47:01 UTC Prevent references to invalid relation pages after fresh promotion If a standby crashes after promotion before having completed its first post-recovery checkpoint, then the minimal recovery point which marks the LSN position where the cluster is able to reach consistency may be set to a position older than the first end-of-recovery checkpoint while all the WAL available should be replayed. This leads to the instance thinking that it contains inconsistent pages, causing a PANIC and a hard instance crash even if all the WAL available has not been replayed for certain sets of records replayed. When in crash recovery, minRecoveryPoint is expected to always be set to InvalidXLogRecPtr, which forces the recovery to replay all the WAL available, so this commit makes sure that the local copy of minRecoveryPoint from the control file is initialized properly and stays as it is while crash recovery is performed. Once switching to archive recovery or if crash recovery finishes, then the local copy minRecoveryPoint can be safely updated. Pavan Deolasee has reported and diagnosed the failure in the first place, and the base fix idea to rely on the local copy of minRecoveryPoint comes from Kyotaro Horiguchi, which has been expanded into a full-fledged patch by me. The test included in this commit has been written by Álvaro Herrera and Pavan Deolasee, which I have modified to make it faster and more reliable with sleep phases. Backpatch down to all supported versions where the bug appears, aka 9.3 which is where the end-of-recovery checkpoint is not run by the startup process anymore. The test gets easily supported down to 10, still it has been tested on all branches. Reported-by: Pavan Deolasee Diagnosed-by: Pavan Deolasee Reviewed-by: Pavan Deolasee, Kyotaro Horiguchi Author: Michael Paquier, Kyotaro Horiguchi, Pavan Deolasee, Álvaro Herrera Discussion: https://postgr.es/m/CABOikdPOewjNL=05K5CbNMxnNtXnQjhTx2F--4p4ruorCjukbA@mail.gmail.com 05 July 2018, 01:47:01 UTC
0095809 Andres Freund 04 July 2018, 21:53:30 UTC Check for interrupts inside the nbtree page deletion code. When deleting pages the nbtree code has to walk through siblings of a tree node. When those sibling links are corrupted that can lead to endless loops - which are currently not interruptible. This is especially problematic if autovacuum is repeatedly blocked on such indexes, as it can be hard to get out of that situation without resorting to single user mode. Thus add interrupt checks to appropriate places in such loops. Unfortunately in one of the cases it's it's not easy to do so. Between 9.3 and 9.4 the page deletion (and page split) code changed significantly. Before it was significantly less robust against interruptions. Therefore don't backpatch to 9.3. Author: Andres Freund Discussion: https://postgr.es/m/20180627191629.wkunw2qbibnvlz53@alap3.anarazel.de Backpatch: 9.4- 04 July 2018, 21:58:39 UTC
8463be0 Fujii Masao 04 July 2018, 17:21:15 UTC Improve the performance of relation deletes during recovery. When multiple relations are deleted at the same transaction, the files of those relations are deleted by one call to smgrdounlinkall(), which leads to scan whole shared_buffers only one time. OTOH, previously, during recovery, smgrdounlink() (not smgrdounlinkall()) was called for each file to delete, which led to scan shared_buffers multiple times. Obviously this could cause to increase the WAL replay time very much especially when shared_buffers was huge. To alleviate this situation, this commit changes the recovery so that it also calls smgrdounlinkall() only one time to delete multiple relation files. This is just fix for oversight of commit 279628a0a7, not new feature. So, per discussion on pgsql-hackers, we concluded to backpatch this to all supported versions. Author: Fujii Masao Reviewed-by: Michael Paquier, Andres Freund, Thomas Munro, Kyotaro Horiguchi, Takayuki Tsunakawa Discussion: https://postgr.es/m/CAHGQGwHVQkdfDqtvGVkty+19cQakAydXn1etGND3X0PHbZ3+6w@mail.gmail.com 04 July 2018, 17:26:22 UTC
c2c69d4 Peter Eisentraut 01 July 2018, 12:06:40 UTC Fix libpq example programs When these programs call pg_catalog.set_config, they need to check for PGRES_TUPLES_OK instead of PGRES_COMMAND_OK. Fix for 5770172cb0c9df9e6ce27c507b449557e5b45124. Reported-by: Ideriha, Takeshi <ideriha.takeshi@jp.fujitsu.com> 01 July 2018, 12:09:03 UTC
b241c11 Michael Paquier 29 June 2018, 13:17:37 UTC Replace search.cpan.org with metacpan.org search.cpan.org has been EOL'd, with metacpan.org being the official replacement to which URLs now redirect. Update links to match the new URL. Also update links to CPAN to use https as it will redirect from http. Author: Daniel Gustafsson Discussion: https://postgr.es/m/B74C0219-6BA9-46E1-A524-5B9E8CD3BDB3@yesql.se 29 June 2018, 13:17:37 UTC
5b0539a Bruce Momjian 28 June 2018, 13:12:07 UTC doc: backpatch mention use of cross platform logical replication Backpatch 21c1f0c607f0344ae8f71ecaae1fe6f58cf7ff9a to PG 10 docs. Reported-by: Haribabu Kommi Discussion: https://postgr.es/m/CAJrrPGfdknoqZcMipPy8XnH3hO3uRic6JTD=jv35oj1DWqL07g@mail.gmail.com Backpatch-through: only 10 28 June 2018, 13:12:07 UTC
5521e3b Amit Kapila 27 June 2018, 12:59:42 UTC Fix thinko in comments. A slot can not be stored in a tuple but it's vice versa. Reported-by: Ashutosh Bapat Author: Ashutosh Bapat Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/CAFjFpRcHhNhXdegyJv3KKDWrwO1_NB_KYZM_ZSDeMOZaL1A5jQ@mail.gmail.com 27 June 2018, 12:59:42 UTC
b767b3f Alvaro Herrera 26 June 2018, 20:38:34 UTC Fix "base" snapshot handling in logical decoding Two closely related bugs are fixed. First, xmin of logical slots was advanced too early. During xl_running_xacts processing, xmin of the slot was set to the oldest running xid in the record, but that's wrong: actually, snapshots which will be used for not-yet-replayed transactions might consider older txns as running too, so we need to keep xmin back for them. The problem wasn't noticed earlier because DDL which allows to delete tuple (set xmax) while some another not-yet-committed transaction looks at it is pretty rare, if not unique: e.g. all forms of ALTER TABLE which change schema acquire ACCESS EXCLUSIVE lock conflicting with any inserts. The included test case (test_decoding's oldest_xmin) uses ALTER of a composite type, which doesn't have such interlocking. To deal with this, we must be able to quickly retrieve oldest xmin (oldest running xid among all assigned snapshots) from ReorderBuffer. To fix, add another list of ReorderBufferTXNs to the reorderbuffer, where transactions are sorted by base-snapshot-LSN. This is slightly different from the existing (sorted by first-LSN) list, because a transaction can have an earlier LSN but a later Xmin, if its first record does not obtain an xmin (eg. xl_xact_assignment). Note this new list doesn't fully replace the existing txn list: we still need that one to prevent WAL recycling. The second issue concerns SnapBuilder snapshots and subtransactions. SnapBuildDistributeNewCatalogSnapshot never assigned a snapshot to a transaction that is known to be a subtxn, which is good in the common case that the top-level transaction already has one (no point in doing so), but a bug otherwise. To fix, arrange to transfer the snapshot from the subtxn to its top-level txn as soon as the kinship gets known. test_decoding's snapshot_transfer verifies this. Also, fix a minor memory leak: refcount of toplevel's old base snapshot was not decremented when the snapshot is transferred from child. Liberally sprinkle code comments, and rewrite a few existing ones. This part is my (Álvaro's) contribution to this commit, as I had to write all those comments in order to understand the existing code and Arseny's patch. Reported-by: Arseny Sher <a.sher@postgrespro.ru> Diagnosed-by: Arseny Sher <a.sher@postgrespro.ru> Co-authored-by: Arseny Sher <a.sher@postgrespro.ru> Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Antonin Houska <ah@cybertec.at> Discussion: https://postgr.es/m/87lgdyz1wj.fsf@ars-thinkpad 26 June 2018, 20:38:34 UTC
09879f7 Fujii Masao 26 June 2018, 15:45:21 UTC Fix documentation bug related to backup history file. The backup history file has been no longer necessary for recovery since the version 9.0. It's now basically just for informational purpose. But previously the documentations still described that a recovery requests the backup history file to proceed. The commit fixes this documentation bug. Back-patch to all supported versions. Author: Yugo Nagata Reviewed-by: Kyotaro Horiguchi Discussion: https://postgr.es/m/20180626174752.0ce505e3.nagata@sraoss.co.jp 26 June 2018, 15:49:47 UTC
3566873 Thomas Munro 18 June 2018, 06:33:53 UTC Add PGTYPESchar_free() to avoid cross-module problems on Windows. On Windows, it is sometimes important for corresponding malloc() and free() calls to be made from the same DLL, since some build options can result in multiple allocators being active at the same time. For that reason we already provided PQfreemem(). This commit adds a similar function for freeing string results allocated by the pgtypes library. Author: Takayuki Tsunakawa Reviewed-by: Kyotaro Horiguchi Discussion: https://postgr.es/m/0A3221C70F24FB45833433255569204D1F8AD5D6%40G01JPEXMBYT05 26 June 2018, 07:49:52 UTC
88554c0 Thomas Munro 26 June 2018, 05:17:27 UTC Move RecoveryLockList into a hash table. Standbys frequently need to release all locks held by a given xid. Instead of searching one big list linearly, let's create one list per xid and put them in a hash table, so we can find what we need in O(1) time. Earlier analysis and a prototype were done by David Rowley, though this isn't his patch. Back-patch all the way. Author: Thomas Munro Diagnosed-by: David Rowley, Andres Freund Reviewed-by: Andres Freund, Tom Lane, Robert Haas Discussion: https://postgr.es/m/CAEepm%3D1mL0KiQ2KJ4yuPpLGX94a4Ns_W6TL4EGRouxWibu56pA%40mail.gmail.com Discussion: https://postgr.es/m/CAKJS1f9vJ841HY%3DwonnLVbfkTWGYWdPN72VMxnArcGCjF3SywA%40mail.gmail.com 26 June 2018, 05:17:27 UTC
324076a Michael Paquier 26 June 2018, 00:56:55 UTC Correct handling of fsync failures with tar mode of walmethods.c This file has been missing the fact that it needs to report back to callers a proper failure on fsync calls. I have spotted the one in tar_finish() while Kuntal has spotted the one in tar_close(). Backpatch down to 10 where this code has been introduced. Reported by: Michael Paquier, Kuntal Ghosh Author: Michael Paquier Reviewed-by: Kuntal Ghosh, Magnus Hagander Discussion: https://postgr.es/m/20180625024356.GD1146@paquier.xyz 26 June 2018, 00:56:55 UTC
99fb443 Alvaro Herrera 25 June 2018, 19:36:33 UTC Update obsolete comments Commit 9fab40ad32ef removed some pre-allocating logic in reorderbuffer.c, but left outdated comments in place. Repair. Author: Álvaro Herrera 25 June 2018, 19:43:05 UTC
6eec672 Michael Paquier 25 June 2018, 02:20:19 UTC Address set of issues with errno handling System calls mixed up in error code paths are causing two issues which several code paths have not correctly handled: 1) For write() calls, sometimes the system may return less bytes than what has been written without errno being set. Some paths were careful enough to consider that case, and assumed that errno should be set to ENOSPC, other calls missed that. 2) errno generated by a system call is overwritten by other system calls which may succeed once an error code path is taken, causing what is reported to the user to be incorrect. This patch uses the brute-force approach of correcting all those code paths. Some refactoring could happen in the future, but this is let as future work, which is not targeted for back-branches anyway. Author: Michael Paquier Reviewed-by: Ashutosh Sharma Discussion: https://postgr.es/m/20180622061535.GD5215@paquier.xyz 25 June 2018, 02:20:19 UTC
6350dcc Bruce Momjian 24 June 2018, 22:07:00 UTC doc: adjust order of NUMERIC arguments to match syntax Specifically, mention precision before scale Reported-by: claytonjsalem@gmail.com Discussion: https://postgr.es/m/152967566691.1268.1062965601465200209@wrigleys.postgresql.org Backpatch-through: 9.3 24 June 2018, 22:07:00 UTC
c1975c1 Bruce Momjian 24 June 2018, 03:32:41 UTC doc: show how interval's 3 unit buckets behave using EXTRACT() This clarifies when justify_days() and justify_hours() are useful. Paragraph moved too. Reported-by: vodevsh@gmail.com Discussion: https://postgr.es/m/152698651482.26744.15456677499485530703@wrigleys.postgresql.org Backpatch-through: 9.3 24 June 2018, 03:32:41 UTC
b742010 Alvaro Herrera 21 June 2018, 21:01:10 UTC Disclaim support for default namespace in XMLTABLE Pavel Stehule's original patch had support for default namespace, but I ripped it out before commit -- hence the docs were correct when written, and I broke them by omission :-(. Remove the offending phrase. Author: Daniel Gustafsson Discussion: https://postgr.es/m/1550C5E5-FC70-4493-A226-AA137D831E8D@yesql.se 21 June 2018, 21:01:10 UTC
b8a1d03 Tom Lane 21 June 2018, 20:18:34 UTC Fix partial aggregation for variance(int4) and related aggregates. A typo in numeric_poly_combine caused bogus results for queries using it, but of course would only manifest if parallel aggregation is performed. Reported by Rajkumar Raghuwanshi. David Rowley did the diagnosis and the fix; I editorialized rather heavily on his regression test additions. Back-patch to v10 where the breakage was introduced (by 9cca11c91). Discussion: https://postgr.es/m/CAKcux6nU4E2x8nkSBpLOT2DPvQ5LviJ3SGyAN6Sz7qDH4G4+Pw@mail.gmail.com 21 June 2018, 20:18:34 UTC
a4c95b0 Tom Lane 21 June 2018, 14:58:42 UTC Fix mishandling of sortgroupref labels while splitting SRF targetlists. split_pathtarget_at_srfs() neglected to worry about sortgroupref labels in the intermediate PathTargets it constructs. I think we'd supposed that their labeling didn't matter, but it does at least for the case that GroupAggregate/GatherMerge nodes appear immediately under the ProjectSet step(s). This results in "ERROR: ORDER/GROUP BY expression not found in targetlist" during create_plan(), as reported by Rajkumar Raghuwanshi. To fix, make this logic track the sortgroupref labeling of expressions, not just their contents. This also restores the pre-v10 behavior that separate GROUP BY expressions will be kept distinct even if they are textually equal(). Discussion: https://postgr.es/m/CAKcux6=1_Ye9kx8YLBPmJs_xE72PPc6vNi5q2AOHowMaCWjJ2w@mail.gmail.com 21 June 2018, 14:58:42 UTC
04ab840 Alvaro Herrera 20 June 2018, 17:02:46 UTC Update expected XML output with disabled XML, too 20 June 2018, 17:02:46 UTC
e10bc16 Alvaro Herrera 20 June 2018, 16:58:12 UTC Accept TEXT and CDATA nodes in XMLTABLE's column_expression. Column expressions that match TEXT or CDATA nodes must return the contents of the nodes themselves, not the content of non-existing children (i.e. the empty string). Author: Markus Winand Reported-by: Markus Winand Reviewed-by: Álvaro Herrera Discussion: https://postgr.es/m/0684A598-002C-42A2-AE12-F024A324EAE4@winand.at 20 June 2018, 16:58:12 UTC
99ba8d2 Magnus Hagander 20 June 2018, 14:07:07 UTC Fix typo Reported using the website comment form 20 June 2018, 14:07:07 UTC
5862174 Michael Paquier 20 June 2018, 01:48:28 UTC Clarify use of temporary tables within partition trees Since their introduction, partition trees have been a bit lossy regarding temporary relations. Inheritance trees respect the following patterns: 1) a child relation can be temporary if the parent is permanent. 2) a child relation can be temporary if the parent is temporary. 3) a child relation cannot be permanent if the parent is temporary. 4) The use of temporary relations also imply that when both parent and child need to be from the same sessions. Partitions share many similar patterns with inheritance, however the handling of the partition bounds make the situation a bit tricky for case 1) as the partition code bases a lot of its lookup code upon PartitionDesc which does not really look after relpersistence. This causes for example a temporary partition created by session A to be visible by another session B, preventing this session B to create an extra partition which overlaps with the temporary one created by A with a non-intuitive error message. There could be use-cases where mixing permanent partitioned tables with temporary partitions make sense, but that would be a new feature. Partitions respect 2), 3) and 4) already. It is a bit depressing to see those error checks happening in MergeAttributes() whose purpose is different, but that's left as future refactoring work. Back-patch down to 10, which is where partitioning has been introduced, except that default partitions do not apply there. Documentation also includes limitations related to the use of temporary tables with partition trees. Reported-by: David Rowley Author: Amit Langote, Michael Paquier Reviewed-by: Ashutosh Bapat, Amit Langote, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f94Ojk0og9GMkRHGt8wHTW=ijq5KzJKuoBoqWLwSVwGmw@mail.gmail.com 20 June 2018, 01:48:28 UTC
a6fb937 Bruce Momjian 19 June 2018, 17:43:40 UTC doc: explain use of json_populate_record{set}() The set-returning nature of these functions make their use unclear. The modified paragraph was added in PG 9.4. Reported-by: yshaladi@denodo.com Discussion: https://postgr.es/m/152571684246.9460.18059951267371255159@wrigleys.postgresql.org Backpatch-through: 9.4 19 June 2018, 17:43:40 UTC
7594b7a Tom Lane 18 June 2018, 19:55:06 UTC Fix contrib/hstore_plperl to look through scalar refs. Bring this transform function into sync with the policy established by commit 3a382983d. Also, fix it to make sure that what it drills down to is indeed a hash, and not some other kind of Perl SV. Previously, the test cases added here provoked crashes. Because of the crash hazard, back-patch to 9.5 where this module was introduced. Discussion: https://postgr.es/m/28336.1528393969@sss.pgh.pa.us 18 June 2018, 19:55:06 UTC
fb28104 Michael Paquier 18 June 2018, 01:43:27 UTC Prevent hard failures of standbys caused by recycled WAL segments When a standby's WAL receiver stops reading WAL from a WAL stream, it writes data to the current WAL segment without having priorily zero'ed the page currently written to, which can cause the WAL reader to read junk data from a past recycled segment and then it would try to get a record from it. While sanity checks in place provide most of the protection needed, in some rare circumstances, with chances increasing when a record header crosses a page boundary, then the startup process could fail violently on an allocation failure, as follows: FATAL: invalid memory alloc request size XXX This is confusing for the user and also unhelpful as this requires in the worst case a manual restart of the instance, impacting potentially the availability of the cluster, and this also makes WAL data look like it is in a corrupted state. The chances of seeing failures are higher if the connection between the standby and its root node is unstable, causing WAL pages to be written in the middle. A couple of approaches have been discussed, like zero-ing new WAL pages within the WAL receiver itself but this has the disadvantage of impacting performance of any existing instances as this breaks the sequential writes done by the WAL receiver. This commit deals with the problem with a more simple approach, which has no performance impact without reducing the detection of the problem: if a record is found with a length higher than 1GB for backends, then do not try any allocation and report a soft failure which will force the standby to retry reading WAL. It could be possible that the allocation call passes and that an unnecessary amount of memory is allocated, however follow-up checks on records would just fail, making this allocation short-lived anyway. This patch owes a great deal to Tsunakawa Takayuki for reporting the failure first, and then discussing a couple of potential approaches to the problem. Backpatch down to 9.5, which is where palloc_extended has been introduced. Reported-by: Tsunakawa Takayuki Reviewed-by: Tsunakawa Takayuki Author: Michael Paquier Discussion: https://postgr.es/m/0A3221C70F24FB45833433255569204D1F8B57AD@G01JPEXMBYT05 18 June 2018, 01:43:42 UTC
416e3e3 Tom Lane 16 June 2018, 19:34:07 UTC Use -Wno-format-truncation and -Wno-stringop-truncation, if available. gcc 8 has started emitting some warnings that are largely useless for our purposes, particularly since they complain about code following the project-standard coding convention that path names are assumed to be shorter than MAXPGPATH. Even if we make the effort to remove that assumption in some future release, the changes wouldn't get back-patched. Hence, just suppress these warnings, on compilers that have these switches. Backpatch to all supported branches. Discussion: https://postgr.es/m/1524563856.26306.9.camel@gunduz.org 16 June 2018, 19:34:07 UTC
8c92485 Tom Lane 16 June 2018, 18:58:11 UTC Avoid unnecessary use of strncpy in a couple of places in ecpg. Use of strncpy with a length limit based on the source, rather than the destination, is non-idiomatic and draws warnings from gcc 8. Replace with memcpy, which does exactly the same thing in these cases, but with less chance for confusion. Backpatch to all supported branches. Discussion: https://postgr.es/m/21789.1529170195@sss.pgh.pa.us 16 June 2018, 18:58:21 UTC
1893326 Tom Lane 16 June 2018, 18:45:47 UTC Use snprintf not sprintf in pg_waldump's timestamptz_to_str. This could only cause an issue if strftime returned a ridiculously long timezone name, which seems unlikely; and it wouldn't qualify as a security problem even then, since pg_waldump (nee pg_xlogdump) is a debug tool not part of the server. But gcc 8 has started issuing warnings about it, so let's use snprintf and be safe. Backpatch to 9.3 where this code was added. Discussion: https://postgr.es/m/21789.1529170195@sss.pgh.pa.us 16 June 2018, 18:45:47 UTC
3b5f4f1 Alvaro Herrera 14 June 2018, 16:51:32 UTC Fail BRIN control functions during recovery explicitly They already fail anyway, but prior to this patch they raise an ugly error message about a lock that cannot be acquired. This just improves the message. Author: Masahiko Sawada Reported-by: Masahiko Sawada Discussion: https://postgr.es/m/CAD21AoBZau4g4_NUf3BKNd=CdYK+xaPdtJCzvOC1TxGdTiJx_Q@mail.gmail.com Reviewed-by: Kuntal Ghosh, Alexander Korotkov, Simon Riggs, Michaël Paquier, Álvaro Herrera 14 June 2018, 16:51:32 UTC
476ba1e Alexander Korotkov 13 June 2018, 15:31:18 UTC Documentation improvement for pg_trgm Documentation of word_similarity() and strict_word_similarity() functions contains some vague wordings which could confuse users. This patch makes those wordings more clear. word_similarity() was introduced in PostgreSQL 9.6, and corresponding part of documentation needs to be backpatched. Author: Bruce Momjian, Alexander Korotkov Discussion: https://postgr.es/m/20180526165648.GB12510%40momjian.us Backpatch: 9.6, where word_similarity() was introduced 13 June 2018, 15:32:03 UTC
2ce64ca Andres Freund 12 June 2018, 18:13:21 UTC Fix bugs in vacuum of shared rels, by keeping their relcache entries current. When vacuum processes a relation it uses the corresponding relcache entry's relfrozenxid / relminmxid as a cutoff for when to remove tuples etc. Unfortunately for nailed relations (i.e. critical system catalogs) bugs could frequently lead to the corresponding relcache entry being stale. This set of bugs could cause actual data corruption as vacuum would potentially not remove the correct row versions, potentially reviving them at a later point. After 699bf7d05c some corruptions in this vein were prevented, but the additional error checks could also trigger spuriously. Examples of such errors are: ERROR: found xmin ... from before relfrozenxid ... and ERROR: found multixact ... from before relminmxid ... To be caused by this bug the errors have to occur on system catalog tables. The two bugs are: 1) Invalidations for nailed relations were ignored, based on the theory that the relcache entry for such tables doesn't change. Which is largely true, except for fields like relfrozenxid etc. This means that changes to relations vacuumed in other sessions weren't picked up by already existing sessions. Luckily autovacuum doesn't have particularly longrunning sessions. 2) For shared *and* nailed relations, the shared relcache init file was never invalidated while running. That means that for such tables (e.g. pg_authid, pg_database) it's not just already existing sessions that are affected, but even new connections are as well. That explains why the reports usually were about pg_authid et. al. To fix 1), revalidate the rd_rel portion of a relcache entry when invalid. This implies a bit of extra complexity to deal with bootstrapping, but it's not too bad. The fix for 2) is simpler, simply always remove both the shared and local init files. Author: Andres Freund Reviewed-By: Alvaro Herrera Discussion: https://postgr.es/m/20180525203736.crkbg36muzxrjj5e@alap3.anarazel.de https://postgr.es/m/CAMa1XUhKSJd98JW4o9StWPrfS=11bPgG+_GDMxe25TvUY4Sugg@mail.gmail.com https://postgr.es/m/CAKMFJucqbuoDRfxPDX39WhA3vJyxweRg_zDVXzncr6+5wOguWA@mail.gmail.com https://postgr.es/m/CAGewt-ujGpMLQ09gXcUFMZaZsGJC98VXHEFbF-tpPB0fB13K+A@mail.gmail.com Backpatch: 9.3- 12 June 2018, 18:13:21 UTC
b10edaf Tom Lane 11 June 2018, 23:17:50 UTC Fix access to just-closed relcache entry. It might be impossible for this to cause a problem in non-debug builds, since there'd be no opportunity for the relcache entry to get recycled before the fetch. It blows up nicely with -DRELCACHE_FORCE_RELEASE plus valgrind, though. Evidently introduced by careless refactoring in commit f0e44751d. Back-patch accordingly. Discussion: https://postgr.es/m/27543.1528758304@sss.pgh.pa.us 11 June 2018, 23:17:50 UTC
8585718 Michael Paquier 11 June 2018, 00:54:25 UTC Fix grammar in documentation related to checkpoint_flush_after Reported-by: Christopher Jones 11 June 2018, 00:57:09 UTC
fd73508 Michael Paquier 10 June 2018, 13:44:17 UTC Fix grammar in REVOKE documentation Reported-by: Erwin Brandstetter 10 June 2018, 13:48:29 UTC
a25c207 Alvaro Herrera 08 June 2018, 20:18:40 UTC Teach SHOW ALL to honor pg_read_all_settings membership Also, fix the pg_settings view to display source filename and line number when invoked by a pg_read_all_settings member. This addition by me (Álvaro). Also, fix wording of the comment in GetConfigOption regarding the restriction it implements, renaming the parameter for extra clarity. Noted by Michaël. These were all oversight in commit 25fff40798fc; backpatch to pg10, where that commit first appeared. Author: Laurenz Albe Reviewed-by: Michaël Paquier, Álvaro Herrera Discussion: https://postgr.es/m/1519917758.6586.8.camel@cybertec.at 08 June 2018, 20:27:56 UTC
6695e95 Peter Eisentraut 08 June 2018, 15:55:12 UTC Fix typo 08 June 2018, 15:55:25 UTC
0692f29 Heikki Linnakangas 07 June 2018, 06:56:22 UTC Fix obsolete comment. The 'orig_slot' argument was removed in commit c0a8ae7be392, but that commit forgot to update the comment. Author: Amit Langote Discussion: https://www.postgresql.org/message-id/194ac4bf-7b4a-c887-bf26-bc1a85ea995a@lab.ntt.co.jp 07 June 2018, 07:02:09 UTC
1d5b249 Alvaro Herrera 06 June 2018, 18:46:53 UTC Fix function code in error report This bug causes a lseek() failure to be reported as a "could not open" failure in the error message, muddling bug reports. I introduced this copy-and-pasteo in commit 78e122010422. Noticed while reviewing code for bug report #15221, from lily liang. In version 10 the affected function is only used by multixact.c and commit_ts, and only in corner-case circumstances, neither of which are involved in the reported bug (a pg_subtrans failure.) Author: Álvaro Herrera 06 June 2018, 18:48:03 UTC
back to top