Skip to main content

Browse the archive

https://github.com/postgres/postgres
02 May 2024, 11:03:54 UTC
sort by:
Revision Author Date Message Commit Date
364998d Tom Lane 07 May 2018, 20:57:35 UTC Stamp 9.4.18. 07 May 2018, 20:57:35 UTC
d289dcf Tom Lane 07 May 2018, 17:13:27 UTC Last-minute updates for release notes. The set of functions that need parallel-safety adjustments isn't the same in 9.6 as 10, so I shouldn't have blindly back-patched that list. Adjust as needed. Also, provide examples of the commands to issue. 07 May 2018, 17:13:47 UTC
dc441d5 Peter Eisentraut 07 May 2018, 15:47:28 UTC Translation updates Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: d73a8c239ac494c183e266261c657580526d4cba 07 May 2018, 15:47:28 UTC
d1e913f Tom Lane 06 May 2018, 19:30:44 UTC Release notes for 10.4, 9.6.9, 9.5.13, 9.4.18, 9.3.23. 06 May 2018, 19:30:44 UTC
1eb2472 Andrew Dunstan 06 May 2018, 11:37:05 UTC Clear severity 5 perlcritic warnings from vcregress.pl My recent update for python3 support used some idioms that are unapproved. This fixes them. Backpatch to all live branches like the original. 06 May 2018, 11:40:04 UTC
af9e0d5 Peter Eisentraut 13 February 2018, 21:13:20 UTC Tweak tests to support Python 3.7 Python 3.7 removes the trailing comma in the repr() of BaseException (see <https://bugs.python.org/issue30399>), leading to test output differences. Work around that by composing the equivalent test output in a more manual way. 06 May 2018, 03:53:05 UTC
280cf0f Peter Eisentraut 05 May 2018, 14:51:38 UTC Remove extra newlines after PQerrorMessage() 05 May 2018, 14:54:00 UTC
c06380e Heikki Linnakangas 04 May 2018, 22:34:53 UTC Fix scenario where streaming standby gets stuck at a continuation record. If a continuation record is split so that its first half has already been removed from the master, and is only present in pg_wal, and there is a recycled WAL segment in the standby server that looks like it would contain the second half, recovery would get stuck. The code in XLogPageRead() incorrectly started streaming at the beginning of the WAL record, even if we had already read the first page. Backpatch to 9.4. In principle, older versions have the same problem, but without replication slots, there was no straightforward mechanism to prevent the master from recycling old WAL that was still needed by standby. Without such a mechanism, I think it's reasonable to assume that there's enough slack in how many old segments are kept around to not run into this, or you have a WAL archive. Reported by Jonathon Nelson. Analysis and patch by Kyotaro HORIGUCHI, with some extra comments by me. Discussion: https://www.postgresql.org/message-id/CACJqAM3xVz0JY1XFDKPP%2BJoJAjoGx%3DGNuOAshEDWCext7BFvCQ%40mail.gmail.com 04 May 2018, 22:35:18 UTC
134db37 Andrew Dunstan 04 May 2018, 19:22:48 UTC Provide for testing on python3 modules when under MSVC This should have been done some years ago as promised in commit c4dcdd0c2. However, better late than never. Along the way do a little housekeeping, including using a simpler test for the python version being tested, and removing a redundant subroutine parameter. These changes only apply back to release 9.5. Backpatch to all live releases. 04 May 2018, 19:51:31 UTC
ade3b27 Andrew Dunstan 04 May 2018, 18:54:04 UTC Allow MSYS as well as MINGW in Msys uname Msys2's uname -s outputs a string beginning MSYS rather than MINGW as is output by Msys. Allow either in pg_upgrade's test.sh. Backpatch to all live branches. 04 May 2018, 19:04:10 UTC
2d123b3 Tom Lane 04 May 2018, 16:26:25 UTC Sync our copy of the timezone library with IANA release tzcode2018e. The non-cosmetic changes involve teaching the "zic" tzdata compiler about negative DST. While I'm not currently intending that we start using negative-DST data right away, it seems possible that somebody would try to use our copy of zic with bleeding-edge IANA data. So we'd better be out in front of this change code-wise, even though it doesn't matter for the data file we're shipping. Discussion: https://postgr.es/m/30996.1525445902@sss.pgh.pa.us 04 May 2018, 16:26:48 UTC
6bd659f Teodor Sigaev 03 May 2018, 17:10:11 UTC Add HOLD_INTERRUPTS section into FinishPreparedTransaction. If an interrupt arrives in the middle of FinishPreparedTransaction and any callback decide to call CHECK_FOR_INTERRUPTS (e.g. RemoveTwoPhaseFile can write a warning with ereport, which checks for interrupts) then it's possible to leave current GXact undeleted. Backpatch to all supported branches Stas Kelvich Discussion: ihttps://www.postgresql.org/message-id/3AD85097-A3F3-4EBA-99BD-C38EDF8D2949@postgrespro.ru 03 May 2018, 17:10:11 UTC
7021145 Tom Lane 02 May 2018, 21:32:40 UTC Revert back-branch changes in power()'s behavior for NaN inputs. Per discussion, the value of fixing these bugs in the back branches doesn't outweigh the downsides of changing corner-case behavior in a minor release. Hence, revert commits 217d8f3a1 and 4d864de48 in the v10 branch and the corresponding commits in 9.3-9.6. Discussion: https://postgr.es/m/75DB81BEEA95B445AE6D576A0A5C9E936A73E741@BPXM05GP.gisp.nec.co.jp 02 May 2018, 21:32:40 UTC
8109a3c Tom Lane 30 April 2018, 01:56:28 UTC Fix bogus list-iteration code in pg_regress.c, affecting ecpg tests only. While looking at a recent buildfarm failure in the ecpg tests, I wondered why the pg_regress output claimed the stderr part of the test failed, when the regression diffs were clearly for the stdout part. Looking into it, the reason is that pg_regress.c's logic for iterating over three parallel lists is wrong, and has been wrong since it was written: it advances the "tag" pointer at a different place in the loop than the other two pointers. Fix that. 30 April 2018, 01:56:28 UTC
59c2df3 Tom Lane 29 April 2018, 22:15:16 UTC Avoid wrong results for power() with NaN input on more platforms. Buildfarm results show that the modern POSIX rule that 1 ^ NaN = 1 is not honored on *BSD until relatively recently, and really old platforms don't believe that NaN ^ 0 = 1 either. (This is unsurprising, perhaps, since SUSv2 doesn't require either behavior.) In hopes of getting to platform independent behavior, let's deal with all the NaN-input cases explicitly in dpow(). Note that numeric_power() doesn't know either of these special cases. But since that behavior is platform-independent, I think it should be addressed separately, and probably not back-patched. Discussion: https://postgr.es/m/75DB81BEEA95B445AE6D576A0A5C9E936A73E741@BPXM05GP.gisp.nec.co.jp 29 April 2018, 22:15:16 UTC
37c02b2 Tom Lane 29 April 2018, 19:50:08 UTC Update time zone data files to tzdata release 2018d. DST law changes in Palestine and Antarctica (Casey Station). Historical corrections for Portugal and its colonies, as well as Enderbury, Jamaica, Turks & Caicos Islands, and Uruguay. 29 April 2018, 19:50:43 UTC
44ccd11 Tom Lane 29 April 2018, 19:21:45 UTC Avoid wrong results for power() with NaN input on some platforms. Per spec, the result of power() should be NaN if either input is NaN. It appears that on some versions of Windows, the libc function does return NaN, but it also sets errno = EDOM, confusing our code that attempts to work around shortcomings of other platforms. Hence, add guard tests to avoid substituting a wrong result for the right one. It's been like this for a long time (and the odd behavior only appears in older MSVC releases, too) so back-patch to all supported branches. Dang Minh Huong, reviewed by David Rowley Discussion: https://postgr.es/m/75DB81BEEA95B445AE6D576A0A5C9E936A73E741@BPXM05GP.gisp.nec.co.jp 29 April 2018, 19:21:45 UTC
367e57f Bruce Momjian 26 April 2018, 15:10:43 UTC docs: remove "III" version text from pgAdmin link Reported-by: vodevsh@gmail.com Discussion: https://postgr.es/m/152404286919.19366.7988650271505173666@wrigleys.postgresql.org Backpatch-through: 9.3 26 April 2018, 15:10:43 UTC
bb53285 Noah Misch 26 April 2018, 01:50:29 UTC Correct pg_recvlogical server version test. The predecessor test boiled down to "PQserverVersion(NULL) >= 100000", which is always false. No release includes that, so it could not have reintroduced CVE-2018-1058. Back-patch to 9.4, like the addition of the predecessor in commit 8d2814f274def85f39fbe997d454b01628cb5667. Discussion: https://postgr.es/m/20180422215551.GB2676194@rfd.leadboat.com 26 April 2018, 01:50:34 UTC
58fec95 Tom Lane 20 April 2018, 19:19:17 UTC Change more places to be less trusting of RestrictInfo.is_pushed_down. On further reflection, commit e5d83995e didn't go far enough: pretty much everywhere in the planner that examines a clause's is_pushed_down flag ought to be changed to use the more complicated behavior where we also check the clause's required_relids. Otherwise we could make incorrect decisions about whether, say, a clause is safe to use as a hash clause. Some (many?) of these places are safe as-is, either because they are never reached while considering a parameterized path, or because there are additional checks that would reject a pushed-down clause anyway. However, it seems smarter to just code them all the same way rather than rely on easily-broken reasoning of that sort. In support of that, invent a new macro RINFO_IS_PUSHED_DOWN that should be used in place of direct tests on the is_pushed_down flag. Like the previous patch, back-patch to all supported branches. Discussion: https://postgr.es/m/f8128b11-c5bf-3539-48cd-234178b2314d@proxel.se 20 April 2018, 19:19:17 UTC
a347d52 Tom Lane 19 April 2018, 19:49:12 UTC Fix incorrect handling of join clauses pushed into parameterized paths. In some cases a clause attached to an outer join can be pushed down into the outer join's RHS even though the clause is not degenerate --- this can happen if we choose to make a parameterized path for the RHS. If the clause ends up attached to a lower outer join, we'd misclassify it as being a "join filter" not a plain "filter" condition at that node, leading to wrong query results. To fix, teach extract_actual_join_clauses to examine each join clause's required_relids, not just its is_pushed_down flag. (The latter now seems vestigial, or at least in need of rethinking, but we won't do anything so invasive as redefining it in a bug-fix patch.) This has been wrong since we introduced parameterized paths in 9.2, though it's evidently hard to hit given the lack of previous reports. The test case used here involves a lateral function call, and I think that a lateral reference may be required to get the planner to select a broken plan; though I wouldn't swear to that. In any case, even if LATERAL is needed to trigger the bug, it still affects all supported branches, so back-patch to all. Per report from Andreas Karlsson. Thanks to Andrew Gierth for preliminary investigation. Discussion: https://postgr.es/m/f8128b11-c5bf-3539-48cd-234178b2314d@proxel.se 19 April 2018, 19:49:12 UTC
e668507 Alvaro Herrera 19 April 2018, 13:45:15 UTC Enlarge find_other_exec's meager fgets buffer The buffer was 100 bytes long, which is barely sufficient when the version string gets longer (such as by configure --with-extra-version). Set it to MAXPGPATH. Author: Nikhil Sontakke Discussion: https://postgr.es/m/CAMGcDxfLfpYU_Jru++L6ARPCOyxr0W+2O3Q54TDi5XdYeU36ow@mail.gmail.com 19 April 2018, 13:45:15 UTC
7490ce7 Tom Lane 18 April 2018, 16:07:38 UTC Better fix for deadlock hazard in CREATE INDEX CONCURRENTLY. Commit 54eff5311 did not account for the possibility that we'd have a transaction snapshot due to default_transaction_isolation being set high enough to require one. The transaction snapshot is enough to hold back our advertised xmin and thus risk deadlock anyway. The only way to get rid of that snap is to start a new transaction, so let's do that instead. Also throw in an assert checking that we really have gotten to a state where no xmin is being advertised. Back-patch to 9.4, like the previous commit. Discussion: https://postgr.es/m/CAMkU=1ztk3TpQdcUNbxq93pc80FrXUjpDWLGMeVBDx71GHNwZQ@mail.gmail.com 18 April 2018, 16:07:38 UTC
92b503c Tom Lane 18 April 2018, 15:57:37 UTC Revert "Add temporary debug logging, in 9.4 branch only." This reverts commit e55380f3b60108d402f64131fe655b0e5ccc1f31. It's served its purpose. 18 April 2018, 15:57:37 UTC
248c268 Tom Lane 18 April 2018, 15:56:56 UTC Revert "Add more temporary debug logging, in 9.4 branch only." This reverts commit eef1a609adfd0c41361aac2e04020bd199fb61fb. It's served its purpose. 18 April 2018, 15:56:56 UTC
eef1a60 Tom Lane 17 April 2018, 15:26:37 UTC Add more temporary debug logging, in 9.4 branch only. Last night's results were inconclusive, but after more staring at the code I've thought of some more data to gather. Discussion: https://postgr.es/m/6744.1523833660@sss.pgh.pa.us 17 April 2018, 15:26:37 UTC
608d1f9 Tom Lane 16 April 2018, 20:06:47 UTC Fix broken collation-aware searches in SP-GiST text opclass. spg_text_leaf_consistent() supposed that it should compare only Min(querylen, entrylen) bytes of the two strings, and then deal with any excess bytes in one string or the other by assuming the longer string is greater if the prefixes are equal. Quite aside from the fact that that's just wrong in some locales (e.g., 'ch' is not less than 'd' in cs_CZ), it also risked passing incomplete multibyte characters to strcoll(), with ensuing bad results. Instead, just pass the full strings to varstr_cmp, and let it decide what to do about unequal-length strings. Fortunately, this error doesn't imply any index corruption, it's just that searches might return the wrong set of entries. Per report from Emre Hasegeli, though this is not his patch. Thanks to Peter Geoghegan for review and discussion. This code was born broken, so back-patch to all supported branches. In HEAD, I failed to resist the temptation to do a bit of cosmetic cleanup/pgindent'ing on 710d90da1, too. Discussion: https://postgr.es/m/CAE2gYzzb6K51VnTq5i5p52z+j9p2duEa-K1T3RrC_GQEynAKEg@mail.gmail.com 16 April 2018, 20:06:47 UTC
e55380f Tom Lane 16 April 2018, 17:44:39 UTC Add temporary debug logging, in 9.4 branch only. Commit 5ee940e1c served its purpose by demonstrating that buildfarm member okapi is seeing some sort of locally-visible state mismanagement, not a cross-process data visibility problem as I'd first theorized. Put in some elog(LOG) messages in hopes of gathering more info about exactly what's happening there. Again, this is temporary code to be reverted once we have buildfarm results. Discussion: https://postgr.es/m/6744.1523833660@sss.pgh.pa.us 16 April 2018, 17:44:39 UTC
fea5bfd Tom Lane 16 April 2018, 17:23:35 UTC Revert "Add temporary debugging assertion, in 9.4 branch only." This reverts commit 5ee940e1cdb6af3af52bb01e44aac63f3a73a28d. Further debugging is needed, but it'll look different than this, so for simplicity revert this first. 16 April 2018, 17:23:35 UTC
5ee940e Tom Lane 16 April 2018, 00:23:59 UTC Add temporary debugging assertion, in 9.4 branch only. Buildfarm member okapi has been failing the multiple-cic isolation test for months now, but only in 9.4. To narrow down the possible causes, add an Assert testing that CREATE INDEX CONCURRENTLY is advertising zero xmin before waiting for other transactions to end. I'm not sure that this would hold in general, so this assertion isn't meant to get released, but it passes all 9.4 regression tests for me. Will revert once we see how okapi responds. 16 April 2018, 00:23:59 UTC
e6b7172 Tom Lane 15 April 2018, 17:02:12 UTC Fix potentially-unportable code in contrib/adminpack. Spelling access(2)'s second argument as "2" is just horrid. POSIX makes no promises as to the numeric values of W_OK and related macros. Even if it accidentally works as intended on every supported platform, it's still unreadable and inconsistent with adjacent code. In passing, don't spell "NULL" as "0" either. Yes, that's legal C; no, it's not project style. Back-patch, just in case the unportability is real and not theoretical. (Most likely, even if a platform had different bit assignments for access()'s modes, there'd not be an observable behavior difference here; but I'm being paranoid today.) 15 April 2018, 17:02:12 UTC
3dd36aa Tom Lane 13 April 2018, 16:53:46 UTC In libpq, free any partial query result before collecting a server error. We'd throw away the partial result anyway after parsing the error message. Throwing it away beforehand costs nothing and reduces the risk of out-of-memory failure. Also, at least in systems that behave like glibc/Linux, if the partial result was very large then the error PGresult would get allocated at high heap addresses, preventing the heap storage used by the partial result from being released to the OS until the error PGresult is freed. In psql >= 9.6, we hold onto the error PGresult until another error is received (for \errverbose), so that this behavior causes a seeming memory leak to persist for awhile, as in a recent complaint from Darafei Praliaskouski. This is a potential performance regression from older versions, justifying back-patching at least that far. But similar behavior may occur in other client applications, so it seems worth just back-patching to all supported branches. Discussion: https://postgr.es/m/CAC8Q8tJ=7cOkPePyAbJE_Pf691t8nDFhJp0KZxHvnq_uicfyVg@mail.gmail.com 13 April 2018, 16:53:46 UTC
f71d803 Tom Lane 12 April 2018, 22:39:51 UTC Fix bogus affix-merging code. NISortAffixes() compared successive compound affixes incorrectly, thus possibly failing to merge identical affixes, or (less likely) merging ones that shouldn't be merged. The user-visible effects of this are unclear, to me anyway. Per bug #15150 from Alexander Lakhin. It's been broken for a long time, so back-patch to all supported branches. Arthur Zakirov Discussion: https://postgr.es/m/152353327780.31225.13445405496721177988@wrigleys.postgresql.org 12 April 2018, 22:39:51 UTC
6943fb9 Tom Lane 11 April 2018, 22:11:30 UTC Ignore nextOid when replaying an ONLINE checkpoint. The nextOid value is from the start of the checkpoint and may well be stale compared to values from more recent XLOG_NEXTOID records. Previously, we adopted it anyway, allowing the OID counter to go backwards during a crash. While this should be harmless, it contributed to the severity of the bug fixed in commit 0408e1ed5, by allowing duplicate TOAST OIDs to be assigned immediately following a crash. Without this error, that issue would only have arisen when TOAST objects just younger than a multiple of 2^32 OIDs were deleted and then not vacuumed in time to avoid a conflict. Pavan Deolasee Discussion: https://postgr.es/m/CABOikdOgWT2hHkYG3Wwo2cyZJq2zfs1FH0FgX-=h4OLosXHf9w@mail.gmail.com 11 April 2018, 22:11:30 UTC
5b3ed6b Tom Lane 11 April 2018, 21:41:10 UTC Do not select new object OIDs that match recently-dead entries. When selecting a new OID, we take care to avoid picking one that's already in use in the target table, so as not to create duplicates after the OID counter has wrapped around. However, up to now we used SnapshotDirty when scanning for pre-existing entries. That ignores committed-dead rows, so that we could select an OID matching a deleted-but-not-yet-vacuumed row. While that mostly worked, it has two problems: * If recently deleted, the dead row might still be visible to MVCC snapshots, creating a risk for duplicate OIDs when examining the catalogs within our own transaction. Such duplication couldn't be visible outside the object-creating transaction, though, and we've heard few if any field reports corresponding to such a symptom. * When selecting a TOAST OID, deleted toast rows definitely *are* visible to SnapshotToast, and will remain so until vacuumed away. This leads to a conflict that will manifest in errors like "unexpected chunk number 0 (expected 1) for toast value nnnnn". We've been seeing reports of such errors from the field for years, but the cause was unclear before. The fix is simple: just use SnapshotAny to search for conflicting rows. This results in a slightly longer window before object OIDs can be recycled, but that seems unlikely to create any large problems. Pavan Deolasee Discussion: https://postgr.es/m/CABOikdOgWT2hHkYG3Wwo2cyZJq2zfs1FH0FgX-=h4OLosXHf9w@mail.gmail.com 11 April 2018, 21:41:27 UTC
310d137 Heikki Linnakangas 11 April 2018, 20:39:48 UTC Make local copy of client hostnames in backend status array. The other strings, application_name and query string, were snapshotted to local memory in pgstat_read_current_status(), but we forgot to do that for client hostnames. As a result, the client hostname would appear to change in the local copy, if the client disconnected. Backpatch to all supported versions. Author: Edmund Horner Reviewed-by: Michael Paquier Discussion: https://www.postgresql.org/message-id/CAMyN-kA7aOJzBmrYFdXcc7Z0NmW%2B5jBaf_m%3D_-77uRNyKC9r%3DA%40mail.gmail.com 11 April 2018, 20:40:27 UTC
f530af8 Tom Lane 10 April 2018, 22:34:40 UTC Fix incorrect close() call in dsm_impl_mmap(). One improbable error-exit path in this function used close() where it should have used CloseTransientFile(). This is unlikely to be hit in the field, and I think the consequences wouldn't be awful (just an elog(LOG) bleat later). But a bug is a bug, so back-patch to 9.4 where this code came in. Pan Bian Discussion: https://postgr.es/m/152056616579.4966.583293218357089052@wrigleys.postgresql.org 10 April 2018, 22:34:40 UTC
4716c9e Tom Lane 08 April 2018, 20:35:43 UTC Doc: clarify explanation of pg_dump usage. This section confusingly used both "infile" and "outfile" to refer to the same file, i.e. the textual output of pg_dump. Use "dumpfile" for both cases, per suggestion from Jonathan Katz. Discussion: https://postgr.es/m/152311295239.31235.6487236091906987117@wrigleys.postgresql.org 08 April 2018, 20:35:43 UTC
ec45e60 Bruce Momjian 05 April 2018, 19:55:41 UTC doc: remove mention of the DMOZ catalog in ltree docs Discussion: https://postgr.es/m/CAF4Au4xYem_W3KOuxcKct7=G4j8Z3uO9j3DUKTFJqUsfp_9pQg@mail.gmail.com Author: Oleg Bartunov Backpatch-through: 9.3 05 April 2018, 19:55:41 UTC
9305257 Bruce Momjian 04 April 2018, 19:06:21 UTC docs: update ltree URL for the DMOZ catalog Reported-by: bbrincat@gmail.com Discussion: https://postgr.es/m/152283596377.1441.11672249301622760943@wrigleys.postgresql.org Author: Oleg Bartunov Backpatch-through: 9.3 04 April 2018, 19:06:21 UTC
b71741b Bruce Momjian 02 April 2018, 20:41:46 UTC doc: document "IS NOT DOCUMENT" Reported-by: scott.ure@caseware.com Discussion: https://postgr.es/m/152056505045.4963.16783351661813640274@wrigleys.postgresql.org Author: Euler Taveira Backpatch-through: 9.3 02 April 2018, 20:41:46 UTC
b7537ff Tom Lane 30 March 2018, 22:14:51 UTC Fix bogus provolatile/proparallel markings on a few built-in functions. Richard Yen reported that pg_upgrade failed if the target cluster had force_parallel_mode = on, because binary_upgrade_create_empty_extension() is marked parallel restricted, allowing it to be executed in parallel mode, which complains because it tries to acquire an XID. In general, no function that might try to modify database data should be considered parallel safe or restricted, since execution of it might force XID acquisition. We found several other examples of this mistake. Furthermore, functions that execute user-supplied SQL queries or query fragments, or pull data from user-supplied cursors, had better be marked both volatile and parallel unsafe, because we don't know what the supplied query or cursor might try to do. There were several tsquery and XML functions that had the wrong proparallel marking for this, and some of them were even mislabeled as to volatility. All these bugs are old, dating back to 9.6 for the proparallel mistakes and much further for the provolatile mistakes. We can't force a catversion bump in the back branches, but we can at least ensure that installations initdb'd in future have the right values. Thomas Munro and Tom Lane Discussion: https://postgr.es/m/CAEepm=2sNDScSLTfyMYu32Q=ob98ZGW-vM_2oLxinzSABGQ6VA@mail.gmail.com 30 March 2018, 22:14:51 UTC
839e26e Bruce Momjian 30 March 2018, 17:34:12 UTC docs: add parameter with brackets around varbit() Reported-by: scott.ure@caseware.com Discussion: https://postgr.es/m/152074343671.1853.18284519607571497106@wrigleys.postgresql.org Author: Euler Taveira Backpatch-through: 9.3 30 March 2018, 17:34:12 UTC
428d246 Tom Lane 25 March 2018, 20:15:16 UTC Doc: add example of type resolution in nested UNIONs. Section 10.5 didn't say explicitly that multiple UNIONs are resolved pairwise. Since the resolution algorithm is described as taking any number of inputs, readers might well think that a query like "select x union select y union select z" would be resolved by considering x, y, and z in one resolution step. But that's not what happens (and I think that behavior is per SQL spec). Add an example clarifying this point. Per bug #15129 from Philippe Beaudoin. Discussion: https://postgr.es/m/152196085023.32649.9916472370480121694@wrigleys.postgresql.org 25 March 2018, 20:15:16 UTC
8f991f4 Tom Lane 25 March 2018, 16:38:21 UTC Doc: remove extra comma in syntax summary for array_fill(). Noted by Scott Ure. Back-patch to all supported branches. Discussion: https://postgr.es/m/152199346794.4544.1888397173908716912@wrigleys.postgresql.org 25 March 2018, 16:38:45 UTC
60c6236 Noah Misch 24 March 2018, 03:31:03 UTC Don't qualify type pg_catalog.text in extend-extensions-example. Extension scripts begin execution with pg_catalog at the front of the search path, so type names reliably refer to pg_catalog. Remove these superfluous qualifications. Earlier <programlisting> of this <sect1> already omitted them. Back-patch to 9.3 (all supported versions). 24 March 2018, 03:31:06 UTC
4c26965 Tom Lane 23 March 2018, 17:45:38 UTC Fix make rules that generate multiple output files. For years, our makefiles have correctly observed that "there is no correct way to write a rule that generates two files". However, what we did is to provide empty rules that "generate" the secondary output files from the primary one, and that's not right either. Depending on the details of the creating process, the primary file might end up timestamped later than one or more secondary files, causing subsequent make runs to consider the secondary file(s) out of date. That's harmless in a plain build, since make will just re-execute the empty rule and nothing happens. But it's fatal in a VPATH build, since make will expect the secondary file to be rebuilt in the build directory. This would manifest as "file not found" failures during VPATH builds from tarballs, if we were ever unlucky enough to ship a tarball with apparently out-of-date secondary files. (It's not clear whether that has ever actually happened, but it definitely could.) To ensure that secondary output files have timestamps >= their primary's, change our makefile convention to be that we provide a "touch $@" action not an empty rule. Also, make sure that this rule actually gets invoked during a distprep run, else the hazard remains. It's been like this a long time, so back-patch to all supported branches. In HEAD, I skipped the changes in src/backend/catalog/Makefile, because those rules are due to get replaced soon in the bootstrap data format patch, and there seems no need to create a merge issue for that patch. If for some reason we fail to land that patch in v11, we'll need to back-fill the changes in that one makefile from v10. Discussion: https://postgr.es/m/18556.1521668179@sss.pgh.pa.us 23 March 2018, 17:45:38 UTC
7f6f8cc Tom Lane 22 March 2018, 17:23:48 UTC Fix tuple counting in SP-GiST index build. Count the number of tuples in the index honestly, instead of assuming that it's the same as the number of tuples in the heap. (It might be different if the index is partial.) Back-patch to all supported versions. Tomas Vondra Discussion: https://postgr.es/m/3b3d8eac-c709-0d25-088e-b98339a1b28a@2ndquadrant.com 22 March 2018, 17:23:48 UTC
67e02cd Tom Lane 22 March 2018, 00:03:28 UTC Fix mishandling of quoted-list GUC values in pg_dump and ruleutils.c. Code that prints out the contents of setconfig or proconfig arrays in SQL format needs to handle GUC_LIST_QUOTE variables differently from other ones, because for those variables, flatten_set_variable_args() already applied a layer of quoting. The value can therefore safely be printed as-is, and indeed must be, or flatten_set_variable_args() will muck it up completely on reload. For all other GUC variables, it's necessary and sufficient to quote the value as a SQL literal. We'd recognized the need for this long ago, but mis-analyzed the need slightly, thinking that all GUC_LIST_INPUT variables needed the special treatment. That's actually wrong, since a valid value of a LIST variable might include characters that need quoting, although no existing variables accept such values. More to the point, we hadn't made any particular effort to keep the various places that deal with this up-to-date with the set of variables that actually need special treatment, meaning that we'd do the wrong thing with, for example, temp_tablespaces values. This affects dumping of SET clauses attached to functions, as well as ALTER DATABASE/ROLE SET commands. In ruleutils.c we can fix it reasonably honestly by exporting a guc.c function that allows discovering the flags for a given GUC variable. But pg_dump doesn't have easy access to that, so continue the old method of having a hard-wired list of affected variable names. At least we can fix it to have just one list not two, and update the list to match current reality. A remaining problem with this is that it only works for built-in GUC variables. pg_dump's list obvious knows nothing of third-party extensions, and even the "ask guc.c" method isn't bulletproof since the relevant extension might not be loaded. There's no obvious solution to that, so for now, we'll just have to discourage extension authors from inventing custom GUCs that need GUC_LIST_QUOTE. This has been busted for a long time, so back-patch to all supported branches. Michael Paquier and Tom Lane, reviewed by Kyotaro Horiguchi and Pavel Stehule Discussion: https://postgr.es/m/20180111064900.GA51030@paquier.xyz 22 March 2018, 00:03:28 UTC
9312fcf Tatsuo Ishii 21 March 2018, 14:08:43 UTC Fix typo. Patch by me. 21 March 2018, 14:12:18 UTC
e1f186d Tom Lane 19 March 2018, 22:49:53 UTC Fix some corner-case issues in REFRESH MATERIALIZED VIEW CONCURRENTLY. refresh_by_match_merge() has some issues in the way it builds a SQL query to construct the "diff" table: 1. It doesn't require the selected unique index(es) to be indimmediate. 2. It doesn't pay attention to the particular equality semantics enforced by a given index, but just assumes that they must be those of the column datatype's default btree opclass. 3. It doesn't check that the indexes are btrees. 4. It's insufficiently careful to ensure that the parser will pick the intended operator when parsing the query. (This would have been a security bug before CVE-2018-1058.) 5. It's not careful about indexes on system columns. The way to fix #4 is to make use of the existing code in ri_triggers.c for generating an arbitrary binary operator clause. I chose to move that to ruleutils.c, since that seems a more reasonable place to be exporting such functionality from than ri_triggers.c. While #1, #3, and #5 are just latent given existing feature restrictions, and #2 doesn't arise in the core system for lack of alternate opclasses with different equality behaviors, #4 seems like an issue worth back-patching. That's the bulk of the change anyway, so just back-patch the whole thing to 9.4 where this code was introduced. Discussion: https://postgr.es/m/13836.1521413227@sss.pgh.pa.us 19 March 2018, 22:49:53 UTC
b6ba94e Tom Lane 19 March 2018, 21:23:07 UTC Fix performance hazard in REFRESH MATERIALIZED VIEW CONCURRENTLY. Jeff Janes discovered that commit 7ca25b7de made one of the queries run by REFRESH MATERIALIZED VIEW CONCURRENTLY perform badly. The root cause is bad cardinality estimation for correlated quals, but a principled solution to that problem is some way off, especially since the planner lacks any statistics about whole-row variables. Moreover, in non-error cases this query produces no rows, meaning it must be run to completion; but use of LIMIT 1 encourages the planner to pick a fast-start, slow-completion plan, exactly not what we want. Remove the LIMIT clause, and instead rely on the count parameter we pass to SPI_execute() to prevent excess work if the query does return some rows. While we've heard no field reports of planner misbehavior with this query, it could be that people are having performance issues that haven't reached the level of pain needed to cause a bug report. In any case, that LIMIT clause can't possibly do anything helpful with any existing version of the planner, and it demonstrably can cause bad choices in some cases, so back-patch to 9.4 where the code was introduced. Thomas Munro Discussion: https://postgr.es/m/CAMkU=1z-JoGymHneGHar1cru4F1XDfHqJDzxP_CtK5cL3DOfmg@mail.gmail.com 19 March 2018, 21:23:07 UTC
b1e48cc Tom Lane 18 March 2018, 19:10:28 UTC Doc: note that statement-level view triggers require an INSTEAD OF trigger. If a view lacks an INSTEAD OF trigger, DML on it can only work by rewriting the command into a command on the underlying base table(s). Then we will fire triggers attached to those table(s), not those for the view. This seems appropriate from a consistency standpoint, but nowhere was the behavior explicitly documented, so let's do that. There was some discussion of throwing an error or warning if a statement trigger is created on a view without creating a row INSTEAD OF trigger. But a simple implementation of that would result in dump/restore ordering hazards. Given that it's been like this all along, and we hadn't heard a complaint till now, a documentation improvement seems sufficient. Per bug #15106 from Pu Qun. Back-patch to all supported branches. Discussion: https://postgr.es/m/152083391168.1215.16892140713507052796@wrigleys.postgresql.org 18 March 2018, 19:10:28 UTC
af5fbb1 Magnus Hagander 18 March 2018, 12:08:25 UTC Fix pg_recvlogical for pre-10 versions In e170b8c8, protection against modified search_path was added. However, PostgreSQL versions prior to 10 does not accept SQL commands over a replication connection, so the protection would generate a syntax error. Since we cannot run SQL commands on it, we are also not vulnerable to the issue that e170b8c8 fixes, so we can just skip this command for older versions. Author: Michael Paquier <michael@paquier.xyz> 18 March 2018, 12:11:58 UTC
092401b Tom Lane 17 March 2018, 19:38:15 UTC Fix overflow handling in plpgsql's integer FOR loops. The test to exit the loop if the integer control value would overflow an int32 turns out not to work on some ICC versions, as it's dependent on the assumption that the compiler will execute the code as written rather than "optimize" it. ICC lacks any equivalent of gcc's -fwrapv switch, so it was optimizing on the assumption of no integer overflow, and that breaks this. Rewrite into a form that in fact does not do any overflowing computations. Per Tomas Vondra and buildfarm member fulmar. It's been like this for a long time, although it was not till we added a regression test case covering the behavior (in commit dd2243f2a) that the problem became apparent. Back-patch to all supported versions. Discussion: https://postgr.es/m/50562fdc-0876-9843-c883-15b8566c7511@2ndquadrant.com 17 March 2018, 19:38:15 UTC
0a0721f Tom Lane 17 March 2018, 18:59:31 UTC Fix WHERE CURRENT OF when the referenced cursor uses an index-only scan. "UPDATE/DELETE WHERE CURRENT OF cursor_name" failed, with an error message like "cannot extract system attribute from virtual tuple", if the cursor was using a index-only scan for the target table. Fix it by digging the current TID out of the indexscan state. It seems likely that the same failure could occur for CustomScan plans and perhaps some FDW plan types, so that leaving this to be treated as an internal error with an obscure message isn't as good an idea as it first seemed. Hence, add a bit of heaptuple.c infrastructure to let us deliver a more on-topic message. I chose to make the message match what you get for the case where execCurrentOf can't identify the target scan node at all, "cursor "foo" is not a simply updatable scan of table "bar"". Perhaps it should be different, but we can always adjust that later. In the future, it might be nice to provide hooks that would let custom scan providers and/or FDWs deal with this in other ways; but that's not a suitable topic for a back-patchable bug fix. It's been like this all along, so back-patch to all supported branches. Yugo Nagata and Tom Lane Discussion: https://postgr.es/m/20180201013349.937dfc5f.nagata@sraoss.co.jp 17 March 2018, 18:59:31 UTC
2709549 Tom Lane 16 March 2018, 20:03:45 UTC Fix query-lifespan memory leakage in repeatedly executed hash joins. ExecHashTableCreate allocated some memory that wasn't freed by ExecHashTableDestroy, specifically the per-hash-key function information. That's not a huge amount of data, but if one runs a query that repeats a hash join enough times, it builds up. Fix by arranging for the data in question to be kept in the hashtable's hashCxt instead of leaving it "loose" in the query-lifespan executor context. (This ensures that we'll also clean up anything that the hash functions allocate in fn_mcxt.) Per report from Amit Khandekar. It's been like this forever, so back-patch to all supported branches. Discussion: https://postgr.es/m/CAJ3gD9cFofAWGvcxLOxDHC=B0hjtW8yGmUsF2hdGh97CM38=7g@mail.gmail.com 16 March 2018, 20:03:45 UTC
21c90df Tom Lane 16 March 2018, 17:44:34 UTC Doc: explicitly point out that enum values can't be dropped. This was not stated in so many words anywhere. Document it to make clear that it's a design limitation and not just an oversight or documentation omission. Discussion: https://postgr.es/m/152089733343.1222.6927268289645380498@wrigleys.postgresql.org 16 March 2018, 17:44:34 UTC
fcc15bf Michael Meskes 13 March 2018, 23:47:49 UTC Fix double frees in ecpg. Patch by Patrick Krecker <patrick@judicata.com> 13 March 2018, 23:52:21 UTC
25a2ba3 Tom Lane 13 March 2018, 17:24:27 UTC When updating reltuples after ANALYZE, just extrapolate from our sample. The existing logic for updating pg_class.reltuples trusted the sampling results only for the pages ANALYZE actually visited, preferring to believe the previous tuple density estimate for all the unvisited pages. While there's some rationale for doing that for VACUUM (first that VACUUM is likely to visit a very nonrandom subset of pages, and second that we know for sure that the unvisited pages did not change), there's no such rationale for ANALYZE: by assumption, it's looked at an unbiased random sample of the table's pages. Furthermore, in a very large table ANALYZE will have examined only a tiny fraction of the table's pages, meaning it cannot slew the overall density estimate very far at all. In a table that is physically growing, this causes reltuples to increase nearly proportionally to the change in relpages, regardless of what is actually happening in the table. This has been observed to cause reltuples to become so much larger than reality that it effectively shuts off autovacuum, whose threshold for doing anything is a fraction of reltuples. (Getting to the point where that would happen seems to require some additional, not well understood, conditions. But it's undeniable that if reltuples is seriously off in a large table, ANALYZE alone will not fix it in any reasonable number of iterations, especially not if the table is continuing to grow.) Hence, restrict the use of vac_estimate_reltuples() to VACUUM alone, and in ANALYZE, just extrapolate from the sample pages on the assumption that they provide an accurate model of the whole table. If, by very bad luck, they don't, at least another ANALYZE will fix it; in the old logic a single bad estimate could cause problems indefinitely. In HEAD, let's remove vac_estimate_reltuples' is_analyze argument altogether; it was never used for anything and now it's totally pointless. But keep it in the back branches, in case any third-party code is calling this function. Per bug #15005. Back-patch to all supported branches. David Gould, reviewed by Alexander Kuzmenkov, cosmetic changes by me Discussion: https://postgr.es/m/20180117164916.3fdcf2e9@engels 13 March 2018, 17:24:27 UTC
95f08d3 Tom Lane 13 March 2018, 16:28:16 UTC Avoid holding AutovacuumScheduleLock while rechecking table statistics. In databases with many tables, re-fetching the statistics takes some time, so that this behavior seriously decreases the available concurrency for multiple autovac workers. There's discussion afoot about more complete fixes, but a simple and back-patchable amelioration is to claim the table and release the lock before rechecking stats. If we find out there's no longer a reason to process the table, re-taking the lock to un-claim the table is cheap enough. (This patch is quite old, but got lost amongst a discussion of more aggressive fixes. It's not clear when or if such a fix will be accepted, but in any case it'd be unlikely to get back-patched. Let's do this now so we have some improvement for the back branches.) In passing, make the normal un-claim step take AutovacuumScheduleLock not AutovacuumLock, since that is what is documented to protect the wi_tableoid field. This wasn't an actual bug in view of the fact that readers of that field hold both locks, but it creates some concurrency penalty against operations that need only AutovacuumLock. Back-patch to all supported versions. Jeff Janes Discussion: https://postgr.es/m/26118.1520865816@sss.pgh.pa.us 13 March 2018, 16:28:39 UTC
bd7eb6f Michael Meskes 12 March 2018, 22:52:08 UTC Set connection back to NULL after freeing it. Patch by Jeevan Ladhe <jeevan.ladhe@enterprisedb.com> 12 March 2018, 22:54:22 UTC
e556fb1 Tom Lane 11 March 2018, 22:10:42 UTC Fix improper uses of canonicalize_qual(). One of the things canonicalize_qual() does is to remove constant-NULL subexpressions of top-level AND/OR clauses. It does that on the assumption that what it's given is a top-level WHERE clause, so that NULL can be treated like FALSE. Although this is documented down inside a subroutine of canonicalize_qual(), it wasn't mentioned in the documentation of that function itself, and some callers hadn't gotten that memo. Notably, commit d007a9505 caused get_relation_constraints() to apply canonicalize_qual() to CHECK constraints. That allowed constraint exclusion to misoptimize situations in which a CHECK constraint had a provably-NULL subclause, as seen in the regression test case added here, in which a child table that should be scanned is not. (Although this thinko is ancient, the test case doesn't fail before 9.2, for reasons I've not bothered to track down in detail. There may be related cases that do fail before that.) More recently, commit f0e44751d added an independent bug by applying canonicalize_qual() to index expressions, which is even sillier since those might not even be boolean. If they are, though, I think this could lead to making incorrect index entries for affected index expressions in v10. I haven't attempted to prove that though. To fix, add an "is_check" parameter to canonicalize_qual() to specify whether it should assume WHERE or CHECK semantics, and make it perform NULL-elimination accordingly. Adjust the callers to apply the right semantics, or remove the call entirely in cases where it's not known that the expression has one or the other semantics. I also removed the call in some cases involving partition expressions, where it should be a no-op because such expressions should be canonical already ... and was a no-op, independently of whether it could in principle have done something, because it was being handed the qual in implicit-AND format which isn't what it expects. In HEAD, add an Assert to catch that type of mistake in future. This represents an API break for external callers of canonicalize_qual(). While that's intentional in HEAD to make such callers think about which case applies to them, it seems like something we probably wouldn't be thanked for in released branches. Hence, in released branches, the extra parameter is added to a new function canonicalize_qual_ext(), and canonicalize_qual() is a wrapper that retains its old behavior. Patch by me with suggestions from Dean Rasheed. Back-patch to all supported branches. Discussion: https://postgr.es/m/24475.1520635069@sss.pgh.pa.us 11 March 2018, 22:10:42 UTC
6d30e3a Alvaro Herrera 06 March 2018, 18:57:20 UTC Refrain from duplicating data in reorderbuffers If a walsender exits leaving data in reorderbuffers, the next walsender that tries to decode the same transaction would append its decoded data in the same spill files without truncating it first, which effectively duplicate the data. Avoid that by removing any leftover reorderbuffer spill files when a walsender starts. Backpatch to 9.4; this bug has been there from the very beginning of logical decoding. Author: Craig Ringer, revised by me Reviewed by: Álvaro Herrera, Petr Jelínek, Masahiko Sawada 06 March 2018, 19:10:23 UTC
165fa27 Tom Lane 04 March 2018, 01:31:35 UTC Fix assorted issues in convert_to_scalar(). If convert_to_scalar is passed a pair of datatypes it can't cope with, its former behavior was just to elog(ERROR). While this is OK so far as the core code is concerned, there's extension code that would like to use scalarltsel/scalargtsel/etc as selectivity estimators for operators that work on non-core datatypes, and this behavior is a show-stopper for that use-case. If we simply allow convert_to_scalar to return FALSE instead of outright failing, then the main logic of scalarltsel/scalargtsel will work fine for any operator that behaves like a scalar inequality comparison. The lack of conversion capability will mean that we can't estimate to better than histogram-bin-width precision, since the code will effectively assume that the comparison constant falls at the middle of its bin. But that's still a lot better than nothing. (Someday we should provide a way for extension code to supply a custom version of convert_to_scalar, but today is not that day.) While poking at this issue, we noted that the existing code for handling type bytea in convert_to_scalar is several bricks shy of a load. It assumes without checking that if the comparison value is type bytea, the bounds values are too; in the worst case this could lead to a crash. It also fails to detoast the input values, so that the comparison result is complete garbage if any input is toasted out-of-line, compressed, or even just short-header. I'm not sure how often such cases actually occur --- the bounds values, at least, are probably safe since they are elements of an array and hence can't be toasted. But that doesn't make this code OK. Back-patch to all supported branches, partly because author requested that, but mostly because of the bytea bugs. The change in API for the exposed routine convert_network_to_scalar() is theoretically a back-patch hazard, but it seems pretty unlikely that any third-party code is calling that function directly. Tomas Vondra, with some adjustments by me Discussion: https://postgr.es/m/b68441b6-d18f-13ab-b43b-9a72188a4e02@2ndquadrant.com 04 March 2018, 01:31:35 UTC
947f06c Tom Lane 02 March 2018, 16:22:42 UTC Make gistvacuumcleanup() count the actual number of index tuples. Previously, it just returned the heap tuple count, which might be only an estimate, and would be completely the wrong thing if the index is partial. Since this function scans every index page anyway to find free pages, it's practically free to count the surviving index tuples. Let's do that and return an accurate count. This is easily visible as a wrong reltuples value for a partial GiST index following VACUUM, so back-patch to all supported branches. Andrey Borodin, reviewed by Michail Nikolaev Discussion: https://postgr.es/m/151956654251.6915.675951950408204404.pgcf@coridan.postgresql.org 02 March 2018, 16:22:42 UTC
a4fed31 Tom Lane 01 March 2018, 21:23:30 UTC Use ereport not elog for some corrupt-HOT-chain reports. These errors have been seen in the field in corrupted-data situations. It seems worthwhile to report them with ERRCODE_DATA_CORRUPTED, rather than the generic ERRCODE_INTERNAL_ERROR, for the benefit of log monitoring and tools like amcheck. However, use errmsg_internal so that the text strings still aren't translated; it seems unlikely to be worth translators' time to do so. Back-patch to 9.3, like the predecessor commit d70cf811f that introduced these elog calls originally (replacing Asserts). Peter Geoghegan Discussion: https://postgr.es/m/CAH2-Wzmn4-Pg-UGFwyuyK-wiTih9j32pwg_7T9iwqXpAUZr=Mg@mail.gmail.com 01 March 2018, 21:23:50 UTC
3ee2383 Alvaro Herrera 01 March 2018, 21:07:46 UTC Relax overly strict sanity check for upgraded ancient databases Commit 4800f16a7ad0 added some sanity checks to ensure we don't accidentally corrupt data, but in one of them we failed to consider the effects of a database upgraded from 9.2 or earlier, where a tuple exclusively locked prior to the upgrade has a slightly different bit pattern. Fix that by using the macro that we fixed in commit 74ebba84aeb6 for similar situations. Reported-by: Alexandre Garcia Reviewed-by: Andres Freund Discussion: https://postgr.es/m/CAPYLKR6yxV4=pfW0Gwij7aPNiiPx+3ib4USVYnbuQdUtmkMaEA@mail.gmail.com Andres suspects that this bug may have wider ranging consequences, but I couldn't find anything. 01 March 2018, 21:07:46 UTC
d07f79a Tom Lane 28 February 2018, 23:33:45 UTC Rename base64 routines to avoid conflict with Solaris built-in functions. Solaris 11.4 has built-in functions named b64_encode and b64_decode. Rename ours to something else to avoid the conflict (fortunately, ours are static so the impact is limited). One could wish for less duplication of code in this area, but that would be a larger patch and not very suitable for back-patching. Since this is a portability fix, we want to put it into all supported branches. Report and initial patch by Rainer Orth, reviewed and adjusted a bit by Michael Paquier Discussion: https://postgr.es/m/ydd372wk28h.fsf@CeBiTec.Uni-Bielefeld.DE 28 February 2018, 23:33:45 UTC
cadb14c Tom Lane 28 February 2018, 21:57:38 UTC Remove restriction on SQL block length in isolationtester scanner. specscanner.l had a fixed limit of 1024 bytes on the length of individual SQL stanzas in an isolation test script. People are starting to run into that, so fix it by making the buffer resizable. Once we allow this in HEAD, it seems inevitable that somebody will try to back-patch a test that exceeds the old limit, so back-patch this change as a preventive measure. Daniel Gustafsson Discussion: https://postgr.es/m/8D628BE4-6606-4FF6-A3FF-8B2B0E9B43D0@yesql.se 28 February 2018, 21:57:38 UTC
49f9014 Tom Lane 27 February 2018, 21:46:52 UTC Fix up ecpg's configuration so it handles "long long int" in MSVC builds. Although configure-based builds correctly define HAVE_LONG_LONG_INT when appropriate (in both pg_config.h and ecpg_config.h), builds using the MSVC scripts failed to do so. This currently has no impact on the backend, since it uses that symbol nowhere; but it does prevent ecpg from supporting "long long int". Fix that. Also, adjust Solution.pm so that in the constructed ecpg_config.h file, the "#if (_MSC_VER > 1200)" covers only the LONG_LONG_INT-related #defines, not the whole file. AFAICS this was a thinko on somebody's part: ENABLE_THREAD_SAFETY should always be defined in Windows builds, and in branches using USE_INTEGER_DATETIMES, the setting of that shouldn't depend on the compiler version either. If I'm wrong, I imagine the buildfarm will say so. Per bug #15080 from Jonathan Allen; issue diagnosed by Michael Meskes and Andrew Gierth. Back-patch to all supported branches. Discussion: https://postgr.es/m/151935568942.1461.14623890240535309745@wrigleys.postgresql.org 27 February 2018, 21:46:52 UTC
5cedaec Tom Lane 27 February 2018, 20:04:21 UTC Remove regression tests' CREATE FUNCTION commands for unused C functions. I removed these functions altogether in HEAD, in commit db3af9feb, and it emerges that that causes trouble for cross-branch upgrade testing. We could put back stub functions but that seems pretty silly. Instead, back-patch a minimal subset of db3af9feb, namely just removing the CREATE FUNCTION commands. Discussion: https://postgr.es/m/11927.1519756619@sss.pgh.pa.us 27 February 2018, 20:04:53 UTC
5ccb775 Tom Lane 27 February 2018, 18:27:38 UTC Prevent dangling-pointer access when update trigger returns old tuple. A before-update row trigger may choose to return the "new" or "old" tuple unmodified. ExecBRUpdateTriggers failed to consider the second possibility, and would proceed to free the "old" tuple even if it was the one returned, leading to subsequent access to already-deallocated memory. In debug builds this reliably leads to an "invalid memory alloc request size" failure; in production builds it might accidentally work, but data corruption is also possible. This is a very old bug. There are probably a couple of reasons it hasn't been noticed up to now. It would be more usual to return NULL if one wanted to suppress the update action; returning "old" is significantly less efficient since the update will occur anyway. Also, none of the standard PLs would ever cause this because they all returned freshly-manufactured tuples even if they were just copying "old". But commit 4b93f5799 changed that for plpgsql, making it possible to see the bug with a plpgsql trigger. Still, this is certainly legal behavior for a trigger function, so it's ExecBRUpdateTriggers's fault not plpgsql's. It seems worth creating a test case that exercises returning "old" directly with a C-language trigger; testing this through plpgsql seems unreliable because its behavior might change again. Report and fix by Rushabh Lathia; regression test case by me. Back-patch to all supported branches. Discussion: https://postgr.es/m/CAGPqQf1P4pjiNPrMof=P_16E-DFjt457j+nH2ex3=nBTew7tXw@mail.gmail.com 27 February 2018, 18:27:38 UTC
5181beb Magnus Hagander 27 February 2018, 13:08:53 UTC Revert restructuring of bin/scripts/Makefile The Makefile portion of 91f3ffc5249eff99c311fb27e7b29a44d9c62be1 broke the MSVC build. This patch reverts the changes to the Makefile and adjusts it to work with the new code, while keeping the actual code changes from the original patch. Author: Victor Wagner <vitus@wagner.pp.ru> 27 February 2018, 13:11:09 UTC
f46bc90 Tom Lane 26 February 2018, 22:17:45 UTC Stamp 9.4.17. 26 February 2018, 22:17:45 UTC
dfebb71 Tom Lane 26 February 2018, 17:14:05 UTC Last-minute updates for release notes. Security: CVE-2018-1058 26 February 2018, 17:14:05 UTC
f28955e Noah Misch 26 February 2018, 15:39:44 UTC Document security implications of search_path and the public schema. The ability to create like-named objects in different schemas opens up the potential for users to change the behavior of other users' queries, maliciously or accidentally. When you connect to a PostgreSQL server, you should remove from your search_path any schema for which a user other than yourself or superusers holds the CREATE privilege. If you do not, other users holding CREATE privilege can redefine the behavior of your commands, causing them to perform arbitrary SQL statements under your identity. "SET search_path = ..." and "SELECT pg_catalog.set_config(...)" are not vulnerable to such hijacking, so one can use either as the first command of a session. As special exceptions, the following client applications behave as documented regardless of search_path settings and schema privileges: clusterdb createdb createlang createuser dropdb droplang dropuser ecpg (not programs it generates) initdb oid2name pg_archivecleanup pg_basebackup pg_config pg_controldata pg_ctl pg_dump pg_dumpall pg_isready pg_receivewal pg_recvlogical pg_resetwal pg_restore pg_rewind pg_standby pg_test_fsync pg_test_timing pg_upgrade pg_waldump reindexdb vacuumdb vacuumlo. Not included are core client programs that run user-specified SQL commands, namely psql and pgbench. PostgreSQL encourages non-core client applications to do likewise. Document this in the context of libpq connections, psql connections, dblink connections, ECPG connections, extension packaging, and schema usage patterns. The principal defense for applications is "SELECT pg_catalog.set_config('search_path', '', false)", and the principal defense for databases is "REVOKE CREATE ON SCHEMA public FROM PUBLIC". Either one is sufficient to prevent attack. After a REVOKE, consider auditing the public schema for objects named like pg_catalog objects. Authors of SECURITY DEFINER functions use some of the same defenses, and the CREATE FUNCTION reference page already covered them thoroughly. This is a good opportunity to audit SECURITY DEFINER functions for robust security practice. Back-patch to 9.3 (all supported versions). Reviewed by Michael Paquier and Jonathan S. Katz. Reported by Arseniy Sharoglazov. Security: CVE-2018-1058 26 February 2018, 15:39:48 UTC
928bca1 Noah Misch 26 February 2018, 15:39:44 UTC Empty search_path in Autovacuum and non-psql/pgbench clients. This makes the client programs behave as documented regardless of the connect-time search_path and regardless of user-created objects. Today, a malicious user with CREATE permission on a search_path schema can take control of certain of these clients' queries and invoke arbitrary SQL functions under the client identity, often a superuser. This is exploitable in the default configuration, where all users have CREATE privilege on schema "public". This changes behavior of user-defined code stored in the database, like pg_index.indexprs and pg_extension_config_dump(). If they reach code bearing unqualified names, "does not exist" or "no schema has been selected to create in" errors might appear. Users may fix such errors by schema-qualifying affected names. After upgrading, consider watching server logs for these errors. The --table arguments of src/bin/scripts clients have been lax; for example, "vacuumdb -Zt pg_am\;CHECKPOINT" performed a checkpoint. That now fails, but for now, "vacuumdb -Zt 'pg_am(amname);CHECKPOINT'" still performs a checkpoint. Back-patch to 9.3 (all supported versions). Reviewed by Tom Lane, though this fix strategy was not his first choice. Reported by Arseniy Sharoglazov. Security: CVE-2018-1058 26 February 2018, 15:39:48 UTC
461c32b Noah Misch 26 February 2018, 15:39:48 UTC Back-patch non-static ExecuteSqlQueryForSingleRow(). Back-patch a subset of commit 47e59697679a0877e0525c565b1be437487604a7 to 9.4 and 9.3. The next commit adds calls to this function. Security: CVE-2018-1058 26 February 2018, 15:39:48 UTC
9f6e529 Tom Lane 26 February 2018, 15:18:22 UTC Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058 26 February 2018, 15:18:22 UTC
fd090d0 Peter Eisentraut 26 February 2018, 13:37:35 UTC Translation updates Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: e859c0a3a8ac0fc7ae28150aff33153ec532ef04 26 February 2018, 13:37:35 UTC
548e849 Tom Lane 25 February 2018, 19:52:51 UTC Release notes for 10.3, 9.6.8, 9.5.12, 9.4.17, 9.3.22. 25 February 2018, 19:52:51 UTC
3ba503b Tom Lane 23 February 2018, 19:38:19 UTC Allow auto_explain.log_min_duration to go up to INT_MAX. The previous limit of INT_MAX / 1000 seems to have been cargo-culted in from somewhere else. Or possibly the value was converted to microseconds at some point; but in all supported releases, it's just compared to other values, so there's no need for the restriction. This change raises the effective limit from ~35 minutes to ~24 days, which conceivably is useful to somebody, and anyway it's more consistent with the range of the core log_min_duration_statement GUC. Per complaint from Kevin Bloch. Back-patch to all supported releases. Discussion: https://postgr.es/m/8ea82d7e-cb78-8e05-0629-73aa14d2a0ca@codingthat.com 23 February 2018, 19:39:22 UTC
da27155 Noah Misch 23 February 2018, 19:24:04 UTC Synchronize doc/ copies of src/test/examples/. This is mostly cosmetic, but it might fix build failures, on some platform, when copying from the documentation. Back-patch to 9.3 (all supported versions). 23 February 2018, 19:24:08 UTC
f6dd084 Tom Lane 23 February 2018, 18:47:33 UTC Fix planner failures with overlapping mergejoin clauses in an outer join. Given overlapping or partially redundant join clauses, for example t1 JOIN t2 ON t1.a = t2.x AND t1.b = t2.x the planner's EquivalenceClass machinery will ordinarily refactor the clauses as "t1.a = t1.b AND t1.a = t2.x", so that join processing doesn't see multiple references to the same EquivalenceClass in a list of join equality clauses. However, if the join is outer, it's incorrect to derive a restriction clause on the outer side from the join conditions, so the clause refactoring does not happen and we end up with overlapping join conditions. The code that attempted to deal with such cases had several subtle bugs, which could result in "left and right pathkeys do not match in mergejoin" or "outer pathkeys do not match mergeclauses" planner errors, if the selected join plan type was a mergejoin. (It does not appear that any actually incorrect plan could have been emitted.) The core of the problem really was failure to recognize that the outer and inner relations' pathkeys have different relationships to the mergeclause list. A join's mergeclause list is constructed by reference to the outer pathkeys, so it will always be ordered the same as the outer pathkeys, but this cannot be presumed true for the inner pathkeys. If the inner sides of the mergeclauses contain multiple references to the same EquivalenceClass ({t2.x} in the above example) then a simplistic rendering of the required inner sort order is like "ORDER BY t2.x, t2.x", but the pathkey machinery recognizes that the second sort column is redundant and throws it away. The mergejoin planning code failed to account for that behavior properly. One error was to try to generate cut-down versions of the mergeclause list from cut-down versions of the inner pathkeys in the same way as the initial construction of the mergeclause list from the outer pathkeys was done; this could lead to choosing a mergeclause list that fails to match the outer pathkeys. The other problem was that the pathkey cross-checking code in create_mergejoin_plan treated the inner and outer pathkey lists identically, whereas actually the expectations for them must be different. That led to false "pathkeys do not match" failures in some cases, and in principle could have led to failure to detect bogus plans in other cases, though there is no indication that such bogus plans could be generated. Reported by Alexander Kuzmenkov, who also reviewed this patch. This has been broken for years (back to around 8.3 according to my testing), so back-patch to all supported branches. Discussion: https://postgr.es/m/5dad9160-4632-0e47-e120-8e2082000c01@postgrespro.ru 23 February 2018, 18:47:33 UTC
2d12c55 Tom Lane 21 February 2018, 23:40:24 UTC Repair pg_upgrade's failure to preserve relfrozenxid for matviews. This oversight led to data corruption in matviews, manifesting as "could not access status of transaction" before our most recent releases, and "found xmin from before relfrozenxid" errors since then. The proximate cause of the problem seems to have been confusion between the task of preserving dropped-column status and the task of preserving frozenxid status. Those are required for distinct sets of relkinds, and the reasoning was entirely undocumented in the source code. In hopes of forestalling future errors of the same kind, try to improve the commentary in this area. In passing, also improve the remarkably unhelpful comments around pg_upgrade's set_frozenxids(). That's not actually buggy AFAICS, but good luck figuring out what it does from the old comments. Per report from Claudio Freire. It appears that bug #14852 from Alexey Ermakov is an earlier report of the same issue, and there may be other cases that we failed to identify at the time. Patch by me based on analysis by Andres Freund. The bug dates back to the introduction of matviews, so back-patch to all supported branches. Discussion: https://postgr.es/m/CAGTBQpbrY9CdRGGhyBZ9yqY4jWaGC85rUF4X+R7d-aim=mBNsw@mail.gmail.com Discussion: https://postgr.es/m/20171013115320.28049.86457@wrigleys.postgresql.org 21 February 2018, 23:40:24 UTC
e11b648 Tom Lane 19 February 2018, 21:00:18 UTC Fix misbehavior of CTE-used-in-a-subplan during EPQ rechecks. An updating query that reads a CTE within an InitPlan or SubPlan could get incorrect results if it updates rows that are concurrently being modified. This is caused by CteScanNext supposing that nothing inside its recursive ExecProcNode call could change which read pointer is selected in the CTE's shared tuplestore. While that's normally true because of scoping considerations, it can break down if an EPQ plan tree gets built during the call, because EvalPlanQualStart builds execution trees for all subplans whether they're going to be used during the recheck or not. And it seems like a pretty shaky assumption anyway, so let's just reselect our own read pointer here. Per bug #14870 from Andrei Gorita. This has been broken since CTEs were implemented, so back-patch to all supported branches. Discussion: https://postgr.es/m/20171024155358.1471.82377@wrigleys.postgresql.org 19 February 2018, 21:00:18 UTC
1bb87c0 Tom Lane 15 February 2018, 18:56:38 UTC Doc: fix minor bug in CREATE TABLE example. One example in create_table.sgml claimed to be showing table constraint syntax, but it was really column constraint syntax due to the omission of a comma. This is both wrong and confusing, so fix it in all supported branches. Per report from neil@postgrescompare.com. Discussion: https://postgr.es/m/151871659877.1393.2431103178451978795@wrigleys.postgresql.org 15 February 2018, 18:57:14 UTC
bd87186 Tom Lane 14 February 2018, 19:47:18 UTC Fix broken logic for reporting PL/Python function names in errcontext. plpython_error_callback() reported the name of the function associated with the topmost PL/Python execution context. This was not merely wrong if there were nested PL/Python contexts, but it risked a core dump if the topmost one is an inline code block rather than a named function. That will have proname = NULL, and so we were passing a NULL pointer to snprintf("%s"). It seems that none of the PL/Python-testing machines in the buildfarm will dump core for that, but some platforms do, as reported by Marina Polyakova. Investigation finds that there actually is an existing regression test that used to prove that the behavior was wrong, though apparently no one had noticed that it was printing the wrong function name. It stopped showing the problem in 9.6 when we adjusted psql to not print CONTEXT by default for NOTICE messages. The problem is masked (if your platform avoids the core dump) in error cases, because PL/Python will throw away the originally generated error info in favor of a new traceback produced at the outer level. Repair by using ErrorContextCallback.arg to pass the correct context to the error callback. Add a regression test illustrating correct behavior. Back-patch to all supported branches, since they're all broken this way. Discussion: https://postgr.es/m/156b989dbc6fe7c4d3223cf51da61195@postgrespro.ru 14 February 2018, 19:47:18 UTC
5811893 Magnus Hagander 07 February 2018, 10:03:55 UTC Change default git repo URL to https Since we now support the server side handler for git over https (so we're no longer using the "dumb protocol"), make https the primary choice for cloning the repository, and the git protocol the secondary choice. In passing, also change the links to git-scm.com from http to https. Reviewed by Stefan Kaltenbrunner and David G. Johnston 07 February 2018, 10:05:13 UTC
b7e1ca7 Tom Lane 05 February 2018, 21:07:03 UTC Stamp 9.4.16. 05 February 2018, 21:07:03 UTC
9a12ac3 Tom Lane 05 February 2018, 19:43:40 UTC Last-minute updates for release notes. Security: CVE-2018-1052, CVE-2018-1053 05 February 2018, 19:44:08 UTC
0a5dcba Peter Eisentraut 05 February 2018, 17:45:45 UTC Translation updates Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 1bde68354ce9389733ebee17e76817f3bda1edf1 05 February 2018, 17:45:45 UTC
c345620 Tom Lane 05 February 2018, 15:58:27 UTC Ensure that all temp files made during pg_upgrade are non-world-readable. pg_upgrade has always attempted to ensure that the transient dump files it creates are inaccessible except to the owner. However, refactoring in commit 76a7650c4 broke that for the file containing "pg_dumpall -g" output; since then, that file was protected according to the process's default umask. Since that file may contain role passwords (hopefully encrypted, but passwords nonetheless), this is a particularly unfortunate oversight. Prudent users of pg_upgrade on multiuser systems would probably run it under a umask tight enough that the issue is moot, but perhaps some users are depending only on pg_upgrade's umask changes to protect their data. To fix this in a future-proof way, let's just tighten the umask at process start. There are no files pg_upgrade needs to write at a weaker security level; and if there were, transiently relaxing the umask around where they're created would be a safer approach. Report and patch by Tom Lane; the idea for the fix is due to Noah Misch. Back-patch to all supported branches. Security: CVE-2018-1053 05 February 2018, 15:58:27 UTC
f20a997 Tom Lane 04 February 2018, 20:13:45 UTC Release notes for 10.2, 9.6.7, 9.5.11, 9.4.16, 9.3.21. 04 February 2018, 20:13:45 UTC
27a5eaa Bruce Momjian 31 January 2018, 21:54:33 UTC doc: in contrib-spi, mention and link to the meaning of SPI Also remove outdated comment about SPI subtransactions. Reported-by: gregory@arenius.com Discussion: https://postgr.es/m/151726276676.1240.10501743959198501067@wrigleys.postgresql.org Backpatch-through: 9.3 31 January 2018, 21:54:33 UTC
7c348e1 Robert Haas 31 January 2018, 21:28:11 UTC pgcrypto's encrypt() supports AES-128, AES-192, and AES-256 Previously, only 128 was mentioned, but the others are also supported. Thomas Munro, reviewed by Michael Paquier and extended a bit by me. Discussion: http://postgr.es/m/CAEepm=1XbBHXYJKofGjnM2Qfz-ZBVqhGU4AqvtgR+Hegy4fdKg@mail.gmail.com 31 January 2018, 21:33:39 UTC
9f050c0 Peter Eisentraut 31 January 2018, 00:21:19 UTC Exclude common/int128.h from cpluspluscheck It uses static assertions, which are not supported under C++ in this branch. This change only goes into the 9.4 branch, because 9.5 and beyond will primarily use the USE_NATIVE_INT128 branch, so cpluspluscheck isn't bothered. In PG11 we will have C++ support for static assertions, so the issue will go away altogether. 31 January 2018, 00:21:19 UTC
b37422d Peter Eisentraut 29 January 2018, 19:04:32 UTC psql documentation fixes Update the documentation for \pset to mention columns|linestyle. Author: Дилян Палаузов <dpa-postgres@aegee.org> 29 January 2018, 19:18:11 UTC
06efc5c Tom Lane 28 January 2018, 18:39:07 UTC Add stack-overflow guards in set-operation planning. create_plan_recurse lacked any stack depth check. This is not per our normal coding rules, but I'd supposed it was safe because earlier planner processing is more complex and presumably should eat more stack. But bug #15033 from Andrew Grossman shows this isn't true, at least not for queries having the form of a many-thousand-way INTERSECT stack. Further testing showed that recurse_set_operations is also capable of being crashed in this way, since it likewise will recurse to the bottom of a parsetree before calling any support functions that might themselves contain any stack checks. However, its stack consumption is only perhaps a third of create_plan_recurse's. It's possible that this particular problem with create_plan_recurse can only manifest in 9.6 and later, since before that we didn't build a Path tree for set operations. But having seen this example, I now have no faith in the proposition that create_plan_recurse doesn't need a stack check, so back-patch to all supported branches. Discussion: https://postgr.es/m/20180127050845.28812.58244@wrigleys.postgresql.org 28 January 2018, 18:39:07 UTC
back to top