Skip to main content

Browse the archive

https://github.com/postgres/postgres
07 April 2024, 13:56:23 UTC
sort by:
Revision Author Date Message Commit Date
f3eff7b Tom Lane 06 November 2017, 22:15:48 UTC Stamp 9.3.20. 06 November 2017, 22:15:48 UTC
fb3930a Tom Lane 06 November 2017, 17:02:30 UTC Last-minute updates for release notes. Security: CVE-2017-12172, CVE-2017-15098, CVE-2017-15099 06 November 2017, 17:02:30 UTC
c0c8807 Tom Lane 06 November 2017, 15:29:17 UTC Make json{b}_populate_recordset() use the right tuple descriptor. json{b}_populate_recordset() used the tuple descriptor created from the query-level AS clause without worrying about whether it matched the actual input record type. If it didn't, that would usually result in a crash, though disclosure of server memory contents seems possible as well, for a skilled attacker capable of issuing crafted SQL commands. Instead, use the query-supplied descriptor only when there is no input tuple to look at, and otherwise get a tuple descriptor based on the input tuple's own type marking. The core code will detect any type mismatch in the latter case. Michael Paquier and Tom Lane, per a report from David Rowley. Back-patch to 9.3 where this functionality was introduced. Security: CVE-2017-15098 06 November 2017, 15:29:42 UTC
b500297 Noah Misch 06 November 2017, 15:11:10 UTC start-scripts: switch to $PGUSER before opening $PGLOG. By default, $PGUSER has permission to unlink $PGLOG. If $PGUSER replaces $PGLOG with a symbolic link, the server will corrupt the link-targeted file by appending log messages. Since these scripts open $PGLOG as root, the attack works regardless of target file ownership. "make install" does not install these scripts anywhere. Users having manually installed them in the past should repeat that process to acquire this fix. Most script users have $PGLOG writable to root only, located in $PGDATA. Just before updating one of these scripts, such users should rename $PGLOG to $PGLOG.old. The script will then recreate $PGLOG with proper ownership. Reviewed by Peter Eisentraut. Reported by Antoine Scemama. Security: CVE-2017-12172 06 November 2017, 15:11:13 UTC
1ea3f6a Peter Eisentraut 05 November 2017, 22:05:18 UTC Translation updates Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 8cd11c648f4c98378ff0a2b5e1e92ab54f69a4a5 05 November 2017, 22:05:18 UTC
0f89484 Tom Lane 05 November 2017, 18:47:57 UTC Release notes for 10.1, 9.6.6, 9.5.10, 9.4.15, 9.3.20, 9.2.24. In the v10 branch, also back-patch the effects of 1ff01b390 and c29c57890 on these files, to reduce future maintenance issues. (I'd do it further back, except that the 9.X branches differ anyway due to xlog-to-wal link tag renaming.) 05 November 2017, 18:47:57 UTC
deb429b Michael Meskes 03 November 2017, 10:14:30 UTC Improve error message for incorrect number inputs in libecpg. 03 November 2017, 11:40:42 UTC
7a35507 Michael Meskes 02 November 2017, 19:46:34 UTC Fix float parsing in ecpg INFORMIX mode. 02 November 2017, 19:51:44 UTC
f05ae2f Alvaro Herrera 02 November 2017, 14:51:05 UTC Revert bogus fixes of HOT-freezing bug It turns out we misdiagnosed what the real problem was. Revert the previous changes, because they may have worse consequences going forward. A better fix is forthcoming. The simplistic test case is kept, though disabled. Discussion: https://postgr.es/m/20171102112019.33wb7g5wp4zpjelu@alap3.anarazel.de 02 November 2017, 14:51:05 UTC
e89867c Tom Lane 02 November 2017, 02:07:14 UTC Doc: update URL for check_postgres. Reported by Dan Vianello. Discussion: https://postgr.es/m/e6e12f18f70e46848c058084d42fb651@KSTLMEXGP001.CORP.CHARTERCOM.com 02 November 2017, 02:07:14 UTC
d64a4d3 Michael Meskes 01 November 2017, 12:32:18 UTC Make sure ecpglib does accepts digits behind decimal point even for integers in Informix mode. Spotted and fixed by 高增琦 <pgf00a@gmail.com> 01 November 2017, 12:41:21 UTC
e06b9e9 Tom Lane 27 October 2017, 22:16:25 UTC Dept of second thoughts: keep aliasp_item in sync with tlistitem. Commit d5b760ecb wasn't quite right, on second thought: if the caller didn't ask for column names then it would happily emit more Vars than if the caller did ask for column names. This is surely not a good idea. Advance the aliasp_item whether or not we're preparing a colnames list. 27 October 2017, 22:16:25 UTC
9d15b8b Tom Lane 27 October 2017, 21:10:21 UTC Fix crash when columns have been added to the end of a view. expandRTE() supposed that an RTE_SUBQUERY subquery must have exactly as many non-junk tlist items as the RTE has column aliases for it. This was true at the time the code was written, and is still true so far as parse analysis is concerned --- but when the function is used during planning, the subquery might have appeared through insertion of a view that now has more columns than it did when the outer query was parsed. This results in a core dump if, for instance, we have to expand a whole-row Var that references the subquery. To avoid crashing, we can either stop expanding the RTE when we run out of aliases, or invent new aliases for the added columns. While the latter might be more useful, the former is consistent with what expandRTE() does for composite-returning functions in the RTE_FUNCTION case, so it seems like we'd better do it that way. Per bug #14876 from Samuel Horwitz. This has been busted since commit ff1ea2173 allowed views to acquire more columns, so back-patch to all supported branches. Discussion: https://postgr.es/m/20171026184035.1471.82810@wrigleys.postgresql.org 27 October 2017, 21:10:21 UTC
be203c3 Tom Lane 27 October 2017, 16:18:57 UTC Rethink the dependencies recorded for FieldSelect/FieldStore nodes. On closer investigation, commits f3ea3e3e8 et al were a few bricks shy of a load. What we need is not so much to lock down the result type of a FieldSelect, as to lock down the existence of the column it's trying to extract. Otherwise, we can break it by dropping that column. The dependency on the result type is then held indirectly through the column, and doesn't need to be recorded explicitly. Out of paranoia, I left in the code to record a dependency on the result type, but it's used only if we can't identify the pg_class OID for the column. That shouldn't ever happen right now, AFAICS, but it seems possible that in future the input node could be marked as being of type RECORD rather than some specific composite type. Likewise for FieldStore. Like the previous patch, back-patch to all supported branches. Discussion: https://postgr.es/m/22571.1509064146@sss.pgh.pa.us 27 October 2017, 16:18:57 UTC
7102efd Tom Lane 27 October 2017, 14:46:07 UTC Doc: mention that you can't PREPARE TRANSACTION after NOTIFY. The NOTIFY page said this already, but the PREPARE TRANSACTION page missed it. Discussion: https://postgr.es/m/20171024010602.1488.80066@wrigleys.postgresql.org 27 October 2017, 14:46:07 UTC
0cf7212 Andrew Dunstan 26 October 2017, 14:10:37 UTC Improve gendef.pl diagnostic on failure to open sym file There have been numerous buildfarm failures but the diagnostic is currently silent about the reason for failure to open the file. Let's see if we can get to the bottom of it. Backpatch to all live branches. 26 October 2017, 14:16:35 UTC
6dd7a12 Tom Lane 25 October 2017, 23:32:24 UTC Fix libpq to not require user's home directory to exist. Some people like to run libpq-using applications in environments where there's no home directory. We've broken that scenario before (cf commits 5b4067798 and bd58d9d88), and commit ba005f193 broke it again, by making it a hard error if we fail to get the home directory name while looking for ~/.pgpass. The previous precedent is that if we can't get the home directory name, we should just silently act as though the file we hoped to find there doesn't exist. Rearrange the new code to honor that. Looking around, the service-file code added by commit 41a4e4595 had the same disease. Apparently, that escaped notice because it only runs when a service name has been specified, which I guess the people who use this scenario don't do. Nonetheless, it's wrong too, so fix that case as well. Add a comment about this policy to pqGetHomeDirectory, in the probably vain hope of forestalling the same error in future. And upgrade the rather miserable commenting in parseServiceInfo, too. In passing, also back off parseServiceInfo's assumption that only ENOENT is an ignorable error from stat() when checking a service file. We would need to ignore at least ENOTDIR as well (cf 5b4067798), and seeing that the far-better-tested code for ~/.pgpass treats all stat() failures alike, I think this code ought to as well. Per bug #14872 from Dan Watson. Back-patch the .pgpass change to v10 where ba005f193 came in. The service-file bugs are far older, so back-patch the other changes to all supported branches. Discussion: https://postgr.es/m/20171025200457.1471.34504@wrigleys.postgresql.org 25 October 2017, 23:32:24 UTC
da82bb1 Tom Lane 23 October 2017, 22:15:36 UTC Update time zone data files to tzdata release 2017c. DST law changes in Fiji, Namibia, Northern Cyprus, Sudan, Tonga, and Turks & Caicos Islands. Historical corrections for Alaska, Apia, Burma, Calcutta, Detroit, Ireland, Namibia, and Pago Pago. 23 October 2017, 22:16:00 UTC
9c74dd2 Tom Lane 23 October 2017, 21:54:09 UTC Sync our copy of the timezone library with IANA release tzcode2017c. This is a trivial update containing only cosmetic changes. The point is just to get back to being synced with an official release of tzcode, rather than some ad-hoc point in their commit history, which is where commit 47f849a3c left it. 23 October 2017, 21:54:09 UTC
dde99de Tom Lane 23 October 2017, 17:57:46 UTC Fix some oversights in expression dependency recording. find_expr_references() neglected to record a dependency on the result type of a FieldSelect node, allowing a DROP TYPE to break a view or rule that contains such an expression. I think we'd omitted this case intentionally, reasoning that there would always be a related dependency ensuring that the DROP would cascade to the view. But at least with nested field selection expressions, that's not true, as shown in bug #14867 from Mansur Galiev. Add the dependency, and for good measure a dependency on the node's exposed collation. Likewise add a dependency on the result type of a FieldStore. I think here the reasoning was that it'd only appear within an assignment to a field, and the dependency on the field's column would be enough ... but having seen this example, I think that's wrong for nested-composites cases. Looking at nearby code, I notice we're not recording a dependency on the exposed collation of CoerceViaIO, which seems inconsistent with our choices for related node types. Maybe that's OK but I'm feeling suspicious of this code today, so let's add that; it certainly can't hurt. This patch does not do anything to protect already-existing views, only views created after it's installed. But seeing that the issue has been there a very long time and nobody noticed till now, that's probably good enough. Back-patch to all supported branches. Discussion: https://postgr.es/m/20171023150118.1477.19174@wrigleys.postgresql.org 23 October 2017, 17:57:46 UTC
7c70a12 Tom Lane 20 October 2017, 21:12:27 UTC Fix typcache's failure to treat ranges as container types. Like the similar logic for arrays and records, it's necessary to examine the range's subtype to decide whether the range type can support hashing. We can omit checking the subtype for btree-defined operations, though, since range subtypes are required to have those operations. (Possibly that simplification for btree cases led us to overlook that it does not apply for hash cases.) This is only an issue if the subtype lacks hash support, which is not true of any built-in range type, but it's easy to demonstrate a problem with a range type over, eg, money: you can get a "could not identify a hash function" failure when the planner is misled into thinking that hash join or aggregation would work. This was born broken, so back-patch to all supported branches. 20 October 2017, 21:12:27 UTC
06b2a73 Tom Lane 17 October 2017, 16:15:08 UTC Fix misparsing of non-newline-terminated pg_hba.conf files. This back-patches the v10-cycle commit 1e5a5d03d into 9.3 - 9.6. I had noticed at the time that that was fixing a bug, namely that next_token() might advance *lineptr past the line-terminating '\0', but given the lack of field complaints I too easily convinced myself that the problem was only latent. It's not, because tokenize_file() decides whether there's more on the line using "strlen(lineptr)". The bug is indeed latent on a newline-terminated line, because then the newline-stripping bit in tokenize_file() means we'll have two or more consecutive '\0's in the buffer, masking the fact that we accidentally advanced over the first one. But the last line in the file might not be null-terminated, allowing the loop to see and process garbage, as reported by Mark Jones in bug #14859. The bug doesn't exist in <= 9.2; there next_token() is reading directly from a file, and termination of the outer loop relies on an feof() test not a buffer pointer check. Probably commit 7f49a67f9 can be blamed for this bug, but I didn't track it down exactly. Commit 1e5a5d03d does a bit more than the minimum needed to fix the bug, but I felt the rest of it was good cleanup, so applying it all. Discussion: https://postgr.es/m/20171017141814.8203.27280@wrigleys.postgresql.org 17 October 2017, 16:15:08 UTC
f7126b4 Tom Lane 11 October 2017, 20:56:23 UTC Doc: fix missing explanation of default object privileges. The GRANT reference page, which lists the default privileges for new objects, failed to mention that USAGE is granted by default for data types and domains. As a lesser sin, it also did not specify anything about the initial privileges for sequences, FDWs, foreign servers, or large objects. Fix that, and add a comment to acldefault() in the probably vain hope of getting people to maintain this list in future. Noted by Laurenz Albe, though I editorialized on the wording a bit. Back-patch to all supported branches, since they all have this behavior. Discussion: https://postgr.es/m/1507620895.4152.1.camel@cybertec.at 11 October 2017, 20:56:59 UTC
7573d12 Tom Lane 11 October 2017, 18:28:33 UTC Fix low-probability loss of NOTIFY messages due to XID wraparound. Up to now async.c has used TransactionIdIsInProgress() to detect whether a notify message's source transaction is still running. However, that function has a quick-exit path that reports that XIDs before RecentXmin are no longer running. If a listening backend is doing nothing but listening, and not running any queries, there is nothing that will advance its value of RecentXmin. Once 2 billion transactions elapse, the RecentXmin check causes active transactions to be reported as not running. If they aren't committed yet according to CLOG, async.c decides they aborted and discards their messages. The timing for that is a bit tight but it can happen when multiple backends are sending notifies concurrently. The net symptom therefore is that a sufficiently-long-surviving listen-only backend starts to miss some fraction of NOTIFY traffic, but only under heavy load. The only function that updates RecentXmin is GetSnapshotData(). A brute-force fix would therefore be to take a snapshot before processing incoming notify messages. But that would add cycles, as well as contention for the ProcArrayLock. We can be smarter: having taken the snapshot, let's use that to check for running XIDs, and not call TransactionIdIsInProgress() at all. In this way we reduce the number of ProcArrayLock acquisitions from one per message to one per notify interrupt; that's the same under light load but should be a benefit under heavy load. Light testing says that this change is a wash performance-wise for normal loads. I looked around for other callers of TransactionIdIsInProgress() that might be at similar risk, and didn't find any; all of them are inside transactions that presumably have already taken a snapshot. Problem report and diagnosis by Marko Tiikkaja, patch by me. Back-patch to all supported branches, since it's been like this since 9.0. Discussion: https://postgr.es/m/20170926182935.14128.65278@wrigleys.postgresql.org 11 October 2017, 18:28:33 UTC
d45fb6e Tom Lane 06 October 2017, 16:20:13 UTC Fix access-off-end-of-array in clog.c. Sloppy loop coding in set_status_by_pages() resulted in fetching one array element more than it should from the subxids[] array. The odds of this resulting in SIGSEGV are pretty small, but we've certainly seen that happen with similar mistakes elsewhere. While at it, we can get rid of an extra TransactionIdToPage() calculation per loop. Per report from David Binderman. Back-patch to all supported branches, since this code is quite old. Discussion: https://postgr.es/m/HE1PR0802MB2331CBA919CBFFF0C465EB429C710@HE1PR0802MB2331.eurprd08.prod.outlook.com 06 October 2017, 16:20:13 UTC
b052d52 Alvaro Herrera 06 October 2017, 15:14:42 UTC Fix traversal of half-frozen update chains When some tuple versions in an update chain are frozen due to them being older than freeze_min_age, the xmax/xmin trail can become broken. This breaks HOT (and probably other things). A subsequent VACUUM can break things in more serious ways, such as leaving orphan heap-only tuples whose root HOT redirect items were removed. This can be seen because index creation (or REINDEX) complain like ERROR: XX000: failed to find parent tuple for heap-only tuple at (0,7) in table "t" Because of relfrozenxid contraints, we cannot avoid the freezing of the early tuples, so we must cope with the results: whenever we see an Xmin of FrozenTransactionId, consider it a match for whatever the previous Xmax value was. This problem seems to have appeared in 9.3 with multixact changes, though strictly speaking it seems unrelated. Since 9.4 we have commit 37484ad2a "Change the way we mark tuples as frozen", so the fix is simple: just compare the raw Xmin (still stored in the tuple header, since freezing merely set an infomask bit) to the Xmax. But in 9.3 we rewrite the Xmin value to FrozenTransactionId, so the original value is lost and we have nothing to compare the Xmax with. To cope with that case we need to compare the Xmin with FrozenXid, assume it's a match, and hope for the best. Sadly, since you can pg_upgrade a 9.3 instance containing half-frozen pages to newer releases, we need to keep the old check in newer versions too, which seems a bit brittle; I hope we can somehow get rid of that. I didn't optimize the new function for performance. The new coding is probably a bit slower than before, since there is a function call rather than a straight comparison, but I'd rather have it work correctly than be fast but wrong. This is a followup after 20b655224249 fixed a few related problems. Apparently, in 9.6 and up there are more ways to get into trouble, but in 9.3 - 9.5 I cannot reproduce a problem anymore with this patch, so there must be a separate bug. Reported-by: Peter Geoghegan Diagnosed-by: Peter Geoghegan, Michael Paquier, Daniel Wood, Yi Wen Wong, Álvaro Discussion: https://postgr.es/m/CAH2-Wznm4rCrhFAiwKPWTpEw2bXDtgROZK7jWWGucXeH3D1fmA@mail.gmail.com 06 October 2017, 15:14:42 UTC
b24f15f Alvaro Herrera 03 October 2017, 12:58:25 UTC Fix coding rules violations in walreceiver.c 1. Since commit b1a9bad9e744 we had pstrdup() inside a spinlock-protected critical section; reported by Andreas Seltenreich. Turn those into strlcpy() to stack-allocated variables instead. Backpatch to 9.6. 2. Since commit 9ed551e0a4fd we had a pfree() uselessly inside a spinlock-protected critical section. Tom Lane noticed in code review. Move down. Backpatch to 9.6. 3. Since commit 64233902d22b we had GetCurrentTimestamp() (a kernel call) inside a spinlock-protected critical section. Tom Lane noticed in code review. Move it up. Backpatch to 9.2. 4. Since commit 1bb2558046cc we did elog(PANIC) while holding spinlock. Tom Lane noticed in code review. Release spinlock before dying. Backpatch to 9.2. Discussion: https://postgr.es/m/87h8vhtgj2.fsf@ansel.ydns.eu 03 October 2017, 12:58:25 UTC
d149aa7 Alvaro Herrera 28 September 2017, 14:44:01 UTC Fix freezing of a dead HOT-updated tuple Vacuum calls page-level HOT prune to remove dead HOT tuples before doing liveness checks (HeapTupleSatisfiesVacuum) on the remaining tuples. But concurrent transaction commit/abort may turn DEAD some of the HOT tuples that survived the prune, before HeapTupleSatisfiesVacuum tests them. This happens to activate the code that decides to freeze the tuple ... which resuscitates it, duplicating data. (This is especially bad if there's any unique constraints, because those are now internally violated due to the duplicate entries, though you won't know until you try to REINDEX or dump/restore the table.) One possible fix would be to simply skip doing anything to the tuple, and hope that the next HOT prune would remove it. But there is a problem: if the tuple is older than freeze horizon, this would leave an unfrozen XID behind, and if no HOT prune happens to clean it up before the containing pg_clog segment is truncated away, it'd later cause an error when the XID is looked up. Fix the problem by having the tuple freezing routines cope with the situation: don't freeze the tuple (and keep it dead). In the cases that the XID is older than the freeze age, set the HEAP_XMAX_COMMITTED flag so that there is no need to look up the XID in pg_clog later on. An isolation test is included, authored by Michael Paquier, loosely based on Daniel Wood's original reproducer. It only tests one particular scenario, though, not all the possible ways for this problem to surface; it be good to have a more reliable way to test this more fully, but it'd require more work. In message https://postgr.es/m/20170911140103.5akxptyrwgpc25bw@alvherre.pgsql I outlined another test case (more closely matching Dan Wood's) that exposed a few more ways for the problem to occur. Backpatch all the way back to 9.3, where this problem was introduced by multixact juggling. In branches 9.3 and 9.4, this includes a backpatch of commit e5ff9fefcd50 (of 9.5 era), since the original is not correctable without matching the coding pattern in 9.5 up. Reported-by: Daniel Wood Diagnosed-by: Daniel Wood Reviewed-by: Yi Wen Wong, Michaël Paquier Discussion: https://postgr.es/m/E5711E62-8FDF-4DCA-A888-C200BF6B5742@amazon.com 28 September 2017, 14:44:01 UTC
2e82fba Tom Lane 27 September 2017, 21:05:54 UTC Fix behavior when converting a float infinity to numeric. float8_numeric() and float4_numeric() failed to consider the possibility that the input is an IEEE infinity. The results depended on the platform-specific behavior of sprintf(): on most platforms you'd get something like ERROR: invalid input syntax for type numeric: "inf" but at least on Windows it's possible for the conversion to succeed and deliver a finite value (typically 1), due to a nonstandard output format from sprintf and lack of syntax error checking in these functions. Since our numeric type lacks the concept of infinity, a suitable conversion is impossible; the best thing to do is throw an explicit error before letting sprintf do its thing. While at it, let's use snprintf not sprintf. Overrunning the buffer should be impossible if sprintf does what it's supposed to, but this is cheap insurance against a stack smash if it doesn't. Problem reported by Taiki Kondo. Patch by me based on fix suggestion from KaiGai Kohei. Back-patch to all supported branches. Discussion: https://postgr.es/m/12A9442FBAE80D4E8953883E0B84E088C8C7A2@BPXM01GP.gisp.nec.co.jp 27 September 2017, 21:05:54 UTC
4366192 Noah Misch 27 September 2017, 05:39:44 UTC Don't recommend "DROP SCHEMA information_schema CASCADE". It drops objects outside information_schema that depend on objects inside information_schema. For example, it will drop a user-defined view if the view query refers to information_schema. Discussion: https://postgr.es/m/20170831025345.GE3963697@rfd.leadboat.com 27 September 2017, 05:39:47 UTC
6c77b47 Tom Lane 26 September 2017, 19:25:57 UTC Improve wording of error message added in commit 714805010. Per suggestions from Peter Eisentraut and David Johnston. Back-patch, like the previous commit. Discussion: https://postgr.es/m/E1dv9jI-0006oT-Fn@gemulon.postgresql.org 26 September 2017, 19:25:57 UTC
e0f5710 Peter Eisentraut 22 September 2017, 20:50:59 UTC Fix saving and restoring umask In two cases, we set a different umask for some piece of code and restore it afterwards. But if the contained code errors out, the umask is not restored. So add TRY/CATCH blocks to fix that. 23 September 2017, 14:05:40 UTC
2020f90 Tom Lane 22 September 2017, 04:04:21 UTC Sync our copy of the timezone library with IANA tzcode master. This patch absorbs a few unreleased fixes in the IANA code. It corresponds to commit 2d8b944c1cec0808ac4f7a9ee1a463c28f9cd00a in https://github.com/eggert/tz. Non-cosmetic changes include: TZDEFRULESTRING is updated to match current US DST practice, rather than what it was over ten years ago. This only matters for interpretation of POSIX-style zone names (e.g., "EST5EDT"), and only if the timezone database doesn't include either an exact match for the zone name or a "posixrules" entry. The latter should not be true in any current Postgres installation, but this could possibly matter when using --with-system-tzdata. Get rid of a nonportable use of "++var" on a bool var. This is part of a larger fix that eliminates some vestigial support for consecutive leap seconds, and adds checks to the "zic" compiler that the data files do not specify that. Remove a couple of ancient compatibility hacks. The IANA crew think these are obsolete, and I tend to agree. But perhaps our buildfarm will think different. Back-patch to all supported branches, in line with our policy that all branches should be using current IANA code. Before v10, this includes application of current pgindent rules, to avoid whitespace problems in future back-patches. Discussion: https://postgr.es/m/E1dsWhf-0000pT-F9@gemulon.postgresql.org 22 September 2017, 04:04:21 UTC
a09d8be Tom Lane 21 September 2017, 22:13:11 UTC Give a better error for duplicate entries in VACUUM/ANALYZE column list. Previously, the code didn't think about this case and would just try to analyze such a column twice. That would fail at the point of inserting the second version of the pg_statistic row, with obscure error messsages like "duplicate key value violates unique constraint" or "tuple already updated by self", depending on context and PG version. We could allow the case by ignoring duplicate column specifications, but it seems better to reject it explicitly. The bogus error messages seem like arguably a bug, so back-patch to all supported versions. Nathan Bossart, per a report from Michael Paquier, and whacked around a bit by me. Discussion: https://postgr.es/m/E061A8E3-5E3D-494D-94F0-E8A9B312BBFC@amazon.com 21 September 2017, 22:13:11 UTC
149cfdb Michael Meskes 11 September 2017, 19:10:36 UTC Fixed ECPG to correctly handle out-of-scope cursor declarations with pointers or array variables. 18 September 2017, 21:08:24 UTC
b1be335 Tom Lane 17 September 2017, 18:50:01 UTC Fix possible dangling pointer dereference in trigger.c. AfterTriggerEndQuery correctly notes that the query_stack could get repalloc'd during a trigger firing, but it nonetheless passes the address of a query_stack entry to afterTriggerInvokeEvents, so that if such a repalloc occurs, afterTriggerInvokeEvents is already working with an obsolete dangling pointer while it scans the rest of the events. Oops. The only code at risk is its "delete_ok" cleanup code, so we can prevent unsafe behavior by passing delete_ok = false instead of true. However, that could have a significant performance penalty, because the point of passing delete_ok = true is to not have to re-scan possibly a large number of dead trigger events on the next time through the loop. There's more than one way to skin that cat, though. What we can do is delete all the "chunks" in the event list except the last one, since we know all events in them must be dead. Deleting the chunks is work we'd have had to do later in AfterTriggerEndQuery anyway, and it ends up saving rescanning of just about the same events we'd have gotten rid of with delete_ok = true. In v10 and HEAD, we also have to be careful to mop up any per-table after_trig_events pointers that would become dangling. This is slightly annoying, but I don't think that normal use-cases will traverse this code path often enough for it to be a performance problem. It's pretty hard to hit this in practice because of the unlikelihood of the query_stack getting resized at just the wrong time. Nonetheless, it's definitely a live bug of ancient standing, so back-patch to all supported branches. Discussion: https://postgr.es/m/2891.1505419542@sss.pgh.pa.us 17 September 2017, 18:50:01 UTC
a42f8d9 Tom Lane 03 September 2017, 15:01:08 UTC Fix macro-redefinition warning on MSVC. In commit 9d6b160d7, I tweaked pg_config.h.win32 to use "#define HAVE_LONG_LONG_INT_64 1" rather than defining it as empty, for consistency with what happens in an autoconf'd build. But Solution.pm injects another definition of that macro into ecpg_config.h, leading to justifiable (though harmless) compiler whining. Make that one consistent too. Back-patch, like the previous patch. Discussion: https://postgr.es/m/CAEepm=1dWsXROuSbRg8PbKLh0S=8Ou-V8sr05DxmJOF5chBxqQ@mail.gmail.com 03 September 2017, 15:01:08 UTC
727add8 Peter Eisentraut 02 September 2017, 02:59:27 UTC doc: Fix typos and other minor issues Author: Alexander Lakhin <exclusion@gmail.com> 02 September 2017, 03:23:15 UTC
dd344de Tom Lane 01 September 2017, 19:14:18 UTC Make [U]INT64CONST safe for use in #if conditions. Instead of using a cast to force the constant to be the right width, assume we can plaster on an L, UL, LL, or ULL suffix as appropriate. The old approach to this is very hoary, dating from before we were willing to require compilers to have working int64 types. This fix makes the PG_INT64_MIN, PG_INT64_MAX, and PG_UINT64_MAX constants safe to use in preprocessor conditions, where a cast doesn't work. Other symbolic constants that might be defined using [U]INT64CONST are likewise safer than before. Also fix the SIZE_MAX macro to be similarly safe, if we are forced to provide a definition for that. The test added in commit 2e70d6b5e happens to do what we want even with the hack "(size_t) -1" definition, but we could easily get burnt on other tests in future. Back-patch to all supported branches, like the previous commits. Discussion: https://postgr.es/m/15883.1504278595@sss.pgh.pa.us 01 September 2017, 19:14:18 UTC
074985b Tom Lane 01 September 2017, 17:52:54 UTC Ensure SIZE_MAX can be used throughout our code. Pre-C99 platforms may lack <stdint.h> and thereby SIZE_MAX. We have a couple of places using the hack "(size_t) -1" as a fallback, but it wasn't universally available; which means the code added in commit 2e70d6b5e fails to compile everywhere. Move that hack to c.h so that we can rely on having SIZE_MAX everywhere. Per discussion, it'd be a good idea to make the macro's value safe for use in #if-tests, but that will take a bit more work. This is just a quick expedient to get the buildfarm green again. Back-patch to all supported branches, like the previous commit. Discussion: https://postgr.es/m/15883.1504278595@sss.pgh.pa.us 01 September 2017, 17:52:54 UTC
1e6c362 Tom Lane 29 August 2017, 19:38:05 UTC Doc: document libpq's restriction to INT_MAX rows in a PGresult. As long as PQntuples, PQgetvalue, etc, use "int" for row numbers, we're pretty much stuck with this limitation. The documentation formerly stated that the result of PQntuples "might overflow on 32-bit operating systems", which is just nonsense: that's not where the overflow would happen, and if you did reach an overflow it would not be on a 32-bit machine, because you'd have OOM'd long since. Discussion: https://postgr.es/m/CA+FnnTxyLWyjY1goewmJNxC==HQCCF4fKkoCTa9qR36oRAHDPw@mail.gmail.com 29 August 2017, 19:38:40 UTC
d391fb6 Tom Lane 29 August 2017, 19:18:01 UTC Teach libpq to detect integer overflow in the row count of a PGresult. Adding more than 1 billion rows to a PGresult would overflow its ntups and tupArrSize fields, leading to client crashes. It'd be desirable to use wider fields on 64-bit machines, but because all of libpq's external APIs use plain "int" for row counters, that's going to be hard to accomplish without an ABI break. Given the lack of complaints so far, and the general pain that would be involved in using such huge PGresults, let's settle for just preventing the overflow and reporting a useful error message if it does happen. Also, for a couple more lines of code we can increase the threshold of trouble from INT_MAX/2 to INT_MAX rows. To do that, refactor pqAddTuple() to allow returning an error message that replaces the default assumption that it failed because of out-of-memory. Along the way, fix PQsetvalue() so that it reports all failures via pqInternalNotice(). It already did so in the case of bad field number, but neglected to report anything for other error causes. Because of the potential for crashes, this seems like a back-patchable bug fix, despite the lack of field reports. Michael Paquier, per a complaint from Igor Korot. Discussion: https://postgr.es/m/CA+FnnTxyLWyjY1goewmJNxC==HQCCF4fKkoCTa9qR36oRAHDPw@mail.gmail.com 29 August 2017, 19:18:01 UTC
669bef9 Tom Lane 29 August 2017, 13:34:21 UTC Improve docs about numeric formatting patterns (to_char/to_number). The explanation about "0" versus "9" format characters was confusing and arguably wrong; the discussion of sign handling wasn't very good either. Notably, while it's accurate to say that "FM" strips leading zeroes in date/time values, what it really does with numeric values is to strip *trailing* zeroes, and then only if you wrote "9" rather than "0". Per gripes from Erwin Brandstetter. Discussion: https://postgr.es/m/CAGHENJ7jgRbTn6nf48xNZ=FHgL2WQ4X8mYsUAU57f-vq8PubEw@mail.gmail.com Discussion: https://postgr.es/m/CAGHENJ45ymd=GOCu1vwV9u7GmCR80_5tW0fP9C_gJKbruGMHvQ@mail.gmail.com 29 August 2017, 13:34:21 UTC
93067c5 Tom Lane 28 August 2017, 21:28:16 UTC Stamp 9.3.19. 28 August 2017, 21:28:16 UTC
a711d6a Tom Lane 28 August 2017, 15:40:48 UTC Doc: adjust release-note credit for parallel pg_restore fix. Discussion: https://postgr.es/m/CAFcNs+pJ6_Ud-zg3vY_Y0mzfESdM34Humt8avKrAKq_H+v18Cg@mail.gmail.com 28 August 2017, 15:40:48 UTC
1e5446e Peter Eisentraut 28 August 2017, 14:09:04 UTC Translation updates Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: ef3340570fcdce8fd9695fa85904448acdb92e06 28 August 2017, 14:09:04 UTC
bacbfae Tom Lane 27 August 2017, 21:35:04 UTC Release notes for 9.6.5, 9.5.9, 9.4.14, 9.3.19, 9.2.23. 27 August 2017, 21:35:04 UTC
5b286ca Andres Freund 22 August 2017, 14:46:05 UTC Backpatch introduction of TupleDescAttr(tupdesc, i). 2cd70845240 / c6293249d change the way individual attributes in a TupleDesc are stored / accessed. To reduce the effort of making extensions compatible with postgresql 11, and to ease future backpatching, backpatch introduction of TupleDescAttr() to all releases. Do not backpatch change in storage, as that'd be a breaking change for existing and working extensions. Author: Andres Freund Discussion: https://postgr.es/m/20170820181723.tdswdinzptbcwhrr@alap3.anarazel.de Backpatch: 9.2- 22 August 2017, 14:47:50 UTC
ece4bd9 Tom Lane 19 August 2017, 17:39:38 UTC Fix possible core dump in parallel restore when using a TOC list. Commit 3eb9a5e7c unintentionally introduced an ordering dependency into restore_toc_entries_prefork(). The existing coding of reduce_dependencies() contains a check to skip moving a TOC entry to the ready_list if it wasn't initially in the pending_list. This used to suffice to prevent reduce_dependencies() from trying to move anything into the ready_list during restore_toc_entries_prefork(), because the pending_list stayed empty throughout that phase; but it no longer does. The problem doesn't manifest unless the TOC has been reordered by SortTocFromFile, which is how I missed it in testing. To fix, just add a test for ready_list == NULL, converting the call with NULL from a poor man's sanity check into an explicit command not to touch TOC items' list membership. Clarify some of the comments around this; in particular, note the primary purpose of the check for pending_list membership, which is to ensure that we can't try to restore the same item twice, in case a TOC list forces it to be restored before its dependency count goes to zero. Per report from Fabrízio de Royes Mello. Back-patch to 9.3, like the previous commit. Discussion: https://postgr.es/m/CAFcNs+pjuv0JL_x4+=71TPUPjdLHOXA4YfT32myj_OrrZb4ohA@mail.gmail.com 19 August 2017, 17:39:38 UTC
bc44044 Tom Lane 17 August 2017, 17:15:36 UTC Further tweaks to compiler flags for PL/Perl on Windows. It now emerges that we can only rely on Perl to tell us we must use -D_USE_32BIT_TIME_T if it's Perl 5.13.4 or later. For older versions, revert to our previous practice of assuming we need that symbol in all 32-bit Windows builds. This is not ideal, but inquiring into which compiler version Perl was built with seems far too fragile. In any case, we had not previously had complaints about these old Perl versions, so let's assume this is Good Enough. (It's still better than the situation ante commit 5a5c2feca, in that at least the effects are confined to PL/Perl rather than the whole PG build.) Back-patch to all supported versions, like 5a5c2feca and predecessors. Discussion: https://postgr.es/m/CANFyU97OVQ3+Mzfmt3MhuUm5NwPU=-FtbNH5Eb7nZL9ua8=rcA@mail.gmail.com 17 August 2017, 17:15:36 UTC
f8bc6b2 Michael Meskes 14 August 2017, 09:29:34 UTC Changed ecpg parser to allow RETURNING clauses without attached C variables. 16 August 2017, 11:30:09 UTC
9f0f4ef Peter Eisentraut 15 August 2017, 23:27:22 UTC Include foreign tables in information_schema.table_privileges This appears to have been an omission in the original commit 0d692a0dc9f. All related information_schema views already include foreign tables. Reported-by: Nicolas Thauvin <nicolas.thauvin@dalibo.com> 15 August 2017, 23:32:52 UTC
cd18427 Tom Lane 14 August 2017, 19:43:20 UTC Handle elog(FATAL) during ROLLBACK more robustly. Stress testing by Andreas Seltenreich disclosed longstanding problems that occur if a FATAL exit (e.g. due to receipt of SIGTERM) occurs while we are trying to execute a ROLLBACK of an already-failed transaction. In such a case, xact.c is in TBLOCK_ABORT state, so that AbortOutOfAnyTransaction would skip AbortTransaction and go straight to CleanupTransaction. This led to an assert failure in an assert-enabled build (due to the ROLLBACK's portal still having a cleanup hook) or without assertions, to a FATAL exit complaining about "cannot drop active portal". The latter's not disastrous, perhaps, but it's messy enough to want to improve it. We don't really want to run all of AbortTransaction in this code path. The minimum required to clean up the open portal safely is to do AtAbort_Memory and AtAbort_Portals. It seems like a good idea to do AtAbort_Memory unconditionally, to be entirely sure that we are starting with a safe CurrentMemoryContext. That means that if the main loop in AbortOutOfAnyTransaction does nothing, we need an extra step at the bottom to restore CurrentMemoryContext = TopMemoryContext, which I chose to do by invoking AtCleanup_Memory. This'll result in calling AtCleanup_Memory twice in many of the paths through this function, but that seems harmless and reasonably inexpensive. The original motivation for the assertion in AtCleanup_Portals was that we wanted to be sure that any user-defined code executed as a consequence of the cleanup hook runs during AbortTransaction not CleanupTransaction. That still seems like a valid concern, and now that we've seen one case of the assertion firing --- which means that exactly that would have happened in a production build --- let's replace the Assert with a runtime check. If we see the cleanup hook still set, we'll emit a WARNING and just drop the hook unexecuted. This has been like this a long time, so back-patch to all supported branches. Discussion: https://postgr.es/m/877ey7bmun.fsf@ansel.ydns.eu 14 August 2017, 19:43:20 UTC
25169b9 Tom Lane 14 August 2017, 15:48:59 UTC Absorb -D_USE_32BIT_TIME_T switch from Perl, if relevant. Commit 3c163a7fc's original choice to ignore all #define symbols whose names begin with underscore turns out to be too simplistic. On Windows, some Perl installations are built with -D_USE_32BIT_TIME_T, and we must absorb that or we get the wrong result for sizeof(PerlInterpreter). This effectively re-reverts commit ef58b87df, which injected that symbol in a hacky way, making it apply to all of Postgres not just PL/Perl. More significantly, it did so on *all* 32-bit Windows builds, even when the Perl build to be used did not select this option; so that it fails to work properly with some newer Perl builds. By making this change, we would be introducing an ABI break in 32-bit Windows builds; but fortunately we have not used type time_t in any exported Postgres APIs in a long time. So it should be OK, both for PL/Perl itself and for third-party extensions, if an extension library is built with a different _USE_32BIT_TIME_T setting than the core code. Patch by me, based on research by Ashutosh Sharma and Robert Haas. Back-patch to all supported branches, as commit 3c163a7fc was. Discussion: https://postgr.es/m/CANFyU97OVQ3+Mzfmt3MhuUm5NwPU=-FtbNH5Eb7nZL9ua8=rcA@mail.gmail.com 14 August 2017, 15:48:59 UTC
bb11ff2 Tom Lane 13 August 2017, 20:15:14 UTC Remove AtEOXact_CatCache(). The sole useful effect of this function, to check that no catcache entries have positive refcounts at transaction end, has really been obsolete since we introduced ResourceOwners in PG 8.1. We reduced the checks to assertions years ago, so that the function was a complete no-op in production builds. There have been previous discussions about removing it entirely, but consensus up to now was that it had some small value as a cross-check for bugs in the ResourceOwner logic. However, it now emerges that it's possible to trigger these assertions if you hit an assert-enabled backend with SIGTERM during a call to SearchCatCacheList, because that function temporarily increases the refcounts of entries it's intending to add to a catcache list construct. In a normal ERROR scenario, the extra refcounts are cleaned up by SearchCatCacheList's PG_CATCH block; but in a FATAL exit we do a transaction abort and exit without ever executing PG_CATCH handlers. There's a case to be made that this is a generic hazard and we should consider restructuring elog(FATAL) handling so that pending PG_CATCH handlers do get run. That's pretty scary though: it could easily create more problems than it solves. Preliminary stress testing by Andreas Seltenreich suggests that there are not many live problems of this ilk, so we rejected that idea. There are more-localized ways to fix the problem; the most principled one would be to use PG_ENSURE_ERROR_CLEANUP instead of plain PG_TRY. But adding cycles to SearchCatCacheList isn't very appealing. We could also weaken the assertions in AtEOXact_CatCache in some more or less ad-hoc way, but that just makes its raison d'etre even less compelling. In the end, the most reasonable solution seems to be to just remove AtEOXact_CatCache altogether, on the grounds that it's not worth trying to fix it. It hasn't found any bugs for us in many years. Per report from Jeevan Chalke. Back-patch to all supported branches. Discussion: https://postgr.es/m/CAM2+6=VEE30YtRQCZX7_sCFsEpoUkFBV1gZazL70fqLn8rcvBA@mail.gmail.com 13 August 2017, 20:15:14 UTC
06931a9 Tom Lane 09 August 2017, 21:03:10 UTC Fix handling of container types in find_composite_type_dependencies. find_composite_type_dependencies correctly found columns that are of the specified type, and columns that are of arrays of that type, but not columns that are domains or ranges over the given type, its array type, etc. The most general way to handle this seems to be to assume that any type that is directly dependent on the specified type can be treated as a container type, and processed recursively (allowing us to handle nested cases such as ranges over domains over arrays ...). Since a type's array type already has such a dependency, we can drop the existing special case for the array type. The very similar logic in get_rels_with_domain was likewise a few bricks shy of a load, as it supposed that a directly dependent type could *only* be a sub-domain. This is already wrong for ranges over domains, and it'll someday be wrong for arrays over domains. Add test cases illustrating the problems, and back-patch to all supported branches. Discussion: https://postgr.es/m/15268.1502309024@sss.pgh.pa.us 09 August 2017, 21:03:10 UTC
962b26b Alvaro Herrera 08 August 2017, 22:46:16 UTC Reword some unclear comments 08 August 2017, 22:49:23 UTC
a5915db Tom Lane 07 August 2017, 21:17:46 UTC Stamp 9.3.18. 07 August 2017, 21:17:46 UTC
61f5285 Peter Eisentraut 07 August 2017, 17:51:07 UTC Translation updates Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 0858deb5ec238f269d1311ff1299f003fc168cab 07 August 2017, 17:51:07 UTC
d64440f Tom Lane 07 August 2017, 15:46:20 UTC Last-minute updates for release notes. Security: CVE-2017-7546, CVE-2017-7547, CVE-2017-7548 07 August 2017, 15:46:20 UTC
5e8e009 Noah Misch 07 August 2017, 14:09:28 UTC Again match pg_user_mappings to information_schema.user_mapping_options. Commit 3eefc51053f250837c3115c12f8119d16881a2d7 claimed to make pg_user_mappings enforce the qualifications user_mapping_options had been enforcing, but its removal of a longstanding restriction left them distinct when the current user is the subject of a mapping yet has no server privileges. user_mapping_options emits no rows for such a mapping, but pg_user_mappings includes full umoptions. Change pg_user_mappings to show null for umoptions. Back-patch to 9.2, like the above commit. Reviewed by Tom Lane. Reported by Jeff Janes. Security: CVE-2017-7547 07 August 2017, 14:09:31 UTC
b2f833e Heikki Linnakangas 07 August 2017, 14:03:42 UTC Don't allow logging in with empty password. Some authentication methods allowed it, others did not. In the client-side, libpq does not even try to authenticate with an empty password, which makes using empty passwords hazardous: an administrator might think that an account with an empty password cannot be used to log in, because psql doesn't allow it, and not realize that a different client would in fact allow it. To clear that confusion and to be be consistent, disallow empty passwords in all authentication methods. All the authentication methods that used plaintext authentication over the wire, except for BSD authentication, already checked that the password received from the user was not empty. To avoid forgetting it in the future again, move the check to the recv_password_packet function. That only forbids using an empty password with plaintext authentication, however. MD5 and SCRAM need a different fix: * In stable branches, check that the MD5 hash stored for the user does not not correspond to an empty string. This adds some overhead to MD5 authentication, because the server needs to compute an extra MD5 hash, but it is not noticeable in practice. * In HEAD, modify CREATE and ALTER ROLE to clear the password if an empty string, or a password hash that corresponds to an empty string, is specified. The user-visible behavior is the same as in the stable branches, the user cannot log in, but it seems better to stop the empty password from entering the system in the first place. Secondly, it is fairly expensive to check that a SCRAM hash doesn't correspond to an empty string, because computing a SCRAM hash is much more expensive than an MD5 hash by design, so better avoid doing that on every authentication. We could clear the password on CREATE/ALTER ROLE also in stable branches, but we would still need to check at authentication time, because even if we prevent empty passwords from being stored in pg_authid, there might be existing ones there already. Reported by Jeroen van der Ham, Ben de Graaff and Jelte Fennema. Security: CVE-2017-7546 07 August 2017, 14:04:12 UTC
73c2b36 Tom Lane 06 August 2017, 21:56:49 UTC Release notes for 9.6.4, 9.5.8, 9.4.13, 9.3.18, 9.2.22. 06 August 2017, 21:56:49 UTC
dda04b9 Tom Lane 04 August 2017, 15:07:10 UTC Disallow SSL session tickets. We don't actually support session tickets, since we do not create an SSL session identifier. But it seems that OpenSSL will issue a session ticket on-demand anyway, which will then fail when used. This results in reconnection failures when using ticket-aware client-side SSL libraries (such as the Npgsql .NET driver), as reported by Shay Rojansky. To fix, just tell OpenSSL not to issue tickets. At some point in the far future, we might consider enabling tickets instead. But the security implications of that aren't entirely clear; and besides it would have little benefit except for very short-lived database connections, which is Something We're Bad At anyhow. It would take a lot of other work to get to a point where that would really be an exciting thing to do. While at it, also tell OpenSSL not to use a session cache. This doesn't really do anything, since a backend would never populate the cache anyway, but it might gain some micro-efficiencies and/or reduce security exposures. Patch by me, per discussion with Heikki Linnakangas and Shay Rojansky. Back-patch to all supported versions. Discussion: https://postgr.es/m/CADT4RqBU8N-csyZuzaook-c795dt22Zcwg1aHWB6tfVdAkodZA@mail.gmail.com 04 August 2017, 15:07:10 UTC
b7d1bc8 Peter Eisentraut 04 August 2017, 00:49:07 UTC Add missing ALTER USER variants ALTER USER ... SET did not support all the syntax variants of ALTER ROLE ... SET. Reported-by: Pavel Golub <pavel@microolap.com> 04 August 2017, 01:29:36 UTC
035bb82 Tom Lane 03 August 2017, 21:36:23 UTC Fix pg_dump/pg_restore to emit REFRESH MATERIALIZED VIEW commands last. Because we push all ACL (i.e. GRANT/REVOKE) restore steps to the end, materialized view refreshes were occurring while the permissions on referenced objects were still at defaults. This led to failures if, say, an MV owned by user A reads from a table owned by user B, even if B had granted the necessary privileges to A. We've had multiple complaints about that type of restore failure, most recently from Jordan Gigov. The ideal fix for this would be to start treating ACLs as dependency- sortable objects, rather than hard-wiring anything about their dump order (the existing approach is a messy kluge dating to commit dc0e76ca3). But that's going to be a rather major change, and it certainly wouldn't lead to a back-patchable fix. As a short-term solution, convert the existing two-pass hack (ie, normal objects then ACLs) to a three-pass hack, ie, normal objects then ACLs then matview refreshes. Because this happens in RestoreArchive(), it will also fix the problem when restoring from an existing archive-format dump. (Note this means that if a matview refresh would have failed under the permissions prevailing at dump time, it'll fail during restore as well. We'll define that as user error rather than something we should try to work around.) To avoid performance loss in parallel restore, we need the matview refreshes to still be parallelizable. Hence, clean things up enough so that both ACLs and matviews are handled by the parallel restore infrastructure, instead of reverting back to serial restore for ACLs. There is still a final serial step, but it shouldn't normally have to do anything; it's only there to try to recover if we get stuck due to some problem like unresolved circular dependencies. Patch by me, but it owes something to an earlier attempt by Kevin Grittner. Back-patch to 9.3 where materialized views were introduced. Discussion: https://postgr.es/m/28572.1500912583@sss.pgh.pa.us 03 August 2017, 21:36:23 UTC
fb31e3a Tom Lane 02 August 2017, 20:55:03 UTC Add pgtcl back to the list of externally-maintained client interfaces. FlightAware is still maintaining this, and indeed is seemingly being more active with it than the pgtclng fork is. List both, for the time being anyway. In the back branches, also back-port commit e20f679f6 and other recent updates to the client-interfaces list. I think these are probably of current interest to users of back branches. I did not touch the list of externally maintained PLs in the back branches, though. Those are much more likely to be server version sensitive, and I don't know which of these PLs work all the way back. Discussion: https://postgr.es/m/20170730162612.1449.58796@wrigleys.postgresql.org 02 August 2017, 20:55:03 UTC
3d9ae20 Tom Lane 02 August 2017, 19:07:20 UTC Silence warning from modern perl about unescaped braces Back-patch commit 76a1c97bf21c301f61bb41345e0cdd0d365b2086 into the older branches (9.5 and before). This is needed because perl 5.26 and later treats the case as an error not just a warning. Original patch by Andrew Dunstan, need for back-patch noted by Ashutosh Sharma Discussion: https://postgr.es/m/CAE9k0PkfNcmj9pA7Yx4qQ=K=3aY4TuiRhp7KYpayDWm9MYsnjg@mail.gmail.com 02 August 2017, 19:07:20 UTC
bf19a6f Tatsuo Ishii 31 July 2017, 23:00:11 UTC Fix comment. XLByteToSeg and XLByteToPrevSeg calculate only a segment number. The definition of these macros were modified by commit dfda6ebaec6763090fb78b458a979b558c50b39b but the comment remain unchanged. Patch by Yugo Nagata. Back patched to 9.3 and beyond. 31 July 2017, 23:09:03 UTC
3bf894f Tom Lane 31 July 2017, 17:42:48 UTC Doc: specify that the minimum supported version of Perl is 5.8.3. Previously the docs just said "5.8 or later". Experimentation shows that while you can build on Unix from a git checkout with 5.8.0, compiling recent PL/Perl requires at least 5.8.1, and you won't be able to run the TAP tests with less than 5.8.3 because that's when they added "prove". (I do not have any information on just what the MSVC build scripts require.) Since all these versions are quite ancient, let's not split hairs in the docs, but just say that 5.8.3 is the minimum requirement. Discussion: https://postgr.es/m/16894.1501392088@sss.pgh.pa.us 31 July 2017, 17:42:48 UTC
0d8f015 Tom Lane 31 July 2017, 16:38:35 UTC PL/Perl portability fix: absorb relevant -D switches from Perl. Back-patch of commit 3c163a7fc76debbbdad1bdd3c43721cffe72f4db, which see for more info. Also throw in commit b4cc35fbb709bd6fcae8998f041fd731c9acbf42, so Coverity doesn't whine about the back branches. Ashutosh Sharma, some adjustments by me Discussion: https://postgr.es/m/CANFyU97OVQ3+Mzfmt3MhuUm5NwPU=-FtbNH5Eb7nZL9ua8=rcA@mail.gmail.com 31 July 2017, 16:38:35 UTC
b92f172 Tom Lane 31 July 2017, 16:10:36 UTC PL/Perl portability fix: avoid including XSUB.h in plperl.c. Back-patch of commit bebe174bb4462ef079a1d7eeafb82ff969f160a4, which see for more info. Patch by me, with some help from Ashutosh Sharma Discussion: https://postgr.es/m/CANFyU97OVQ3+Mzfmt3MhuUm5NwPU=-FtbNH5Eb7nZL9ua8=rcA@mail.gmail.com 31 July 2017, 16:10:36 UTC
c89d4a2 Tatsuo Ishii 31 July 2017, 02:24:51 UTC Add missing comment in postgresql.conf. current_source requires to restart server to reflect the new value. Per Yugo Nagata and Masahiko Sawada. Back patched to 9.2 and beyond. 31 July 2017, 02:28:34 UTC
689646b Tom Lane 27 July 2017, 18:13:15 UTC Fix psql tab completion for CREATE USER MAPPING. After typing CREATE USER M..., it would not fill in MAPPING FOR, even though that was clearly intended behavior. Jeff Janes Discussion: https://postgr.es/m/CAMkU=1wo2iQ6jWnN=egqOb5NxEPn0PpANEtKHr3uPooQ+nYPtw@mail.gmail.com 27 July 2017, 18:13:15 UTC
00cf718 Tom Lane 26 July 2017, 23:35:35 UTC Clean up SQL emitted by psql/describe.c. Fix assorted places that had not bothered with the convention of prefixing catalog and function names with "pg_catalog.". That could possibly result in query failure when running with a nondefault search_path. Also fix two places that weren't quoting OID literals. I think the latter hasn't mattered much since about 7.3, but it's still a bad idea to be doing it in 99 places and not in 2 others. Also remove a useless EXISTS sub-select that someone had stuck into describeOneTableDetails' queries for child tables. We just got the OID out of pg_class, so I hardly see how checking that it exists in pg_class was doing anything helpful. In passing, try to improve the emitted formatting of a couple of these queries, though I didn't work really hard on that. And merge unnecessarily duplicative coding in some other places. Much of this was new in HEAD, but some was quite old; back-patch as appropriate. 26 July 2017, 23:36:22 UTC
2efbfb9 Alvaro Herrera 26 July 2017, 21:24:16 UTC Fix concurrent locking of tuple update chain If several sessions are concurrently locking a tuple update chain with nonconflicting lock modes using an old snapshot, and they all succeed, it may happen that some of them fail because of restarting the loop (due to a concurrent Xmax change) and getting an error in the subsequent pass while trying to obtain a tuple lock that they already have in some tuple version. This can only happen with very high concurrency (where a row is being both updated and FK-checked by multiple transactions concurrently), but it's been observed in the field and can have unpleasant consequences such as an FK check failing to see a tuple that definitely exists: ERROR: insert or update on table "child_table" violates foreign key constraint "fk_constraint_name" DETAIL: Key (keyid)=(123456) is not present in table "parent_table". (where the key is observably present in the table). Discussion: https://postgr.es/m/20170714210011.r25mrff4nxjhmf3g@alvherre.pgsql 26 July 2017, 21:24:16 UTC
4ecee11 Tom Lane 24 July 2017, 20:45:47 UTC Fix race condition in predicate-lock init code in EXEC_BACKEND builds. Trading a little too heavily on letting the code path be the same whether we were creating shared data structures or only attaching to them, InitPredicateLocks() inserted the "scratch" PredicateLockTargetHash entry unconditionally. This is just wrong if we're in a postmaster child, which would only reach this code in EXEC_BACKEND builds. Most of the time, the hash_search(HASH_ENTER) call would simply report that the entry already existed, causing no visible effect since the code did not bother to check for that possibility. However, if this happened while some other backend had transiently removed the "scratch" entry, then that other backend's eventual RestoreScratchTarget would suffer an assert failure; this appears to be the explanation for a recent failure on buildfarm member culicidae. In non-assert builds, there would be no visible consequences there either. But nonetheless this is a pretty bad bug for EXEC_BACKEND builds, for two reasons: 1. Each new backend would perform the hash_search(HASH_ENTER) call without holding any lock that would prevent concurrent access to the PredicateLockTargetHash hash table. This creates a low but certainly nonzero risk of corruption of that hash table. 2. In the event that the race condition occurred, by reinserting the scratch entry too soon, we were defeating the entire purpose of the scratch entry, namely to guarantee that transaction commit could move hash table entries around with no risk of out-of-memory failure. The odds of an actual OOM failure are quite low, but not zero, and if it did happen it would again result in corruption of the hash table. The user-visible symptoms of such corruption are a little hard to predict, but would presumably amount to misbehavior of SERIALIZABLE transactions that'd require a crash or postmaster restart to fix. To fix, just skip the hash insertion if IsUnderPostmaster. I also inserted a bunch of assertions that the expected things happen depending on whether IsUnderPostmaster is true. That might be overkill, since most comparable code in other functions isn't quite that paranoid, but once burnt twice shy. In passing, also move a couple of lines to places where they seemed to make more sense. Diagnosis of problem by Thomas Munro, patch by me. Back-patch to all supported branches. Discussion: https://postgr.es/m/10593.1500670709@sss.pgh.pa.us 24 July 2017, 20:45:47 UTC
b6d6400 Tom Lane 24 July 2017, 19:16:31 UTC Ensure that pg_get_ruledef()'s output matches pg_get_viewdef()'s. Various cases involving renaming of view columns are handled by having make_viewdef pass down the view's current relation tupledesc to get_query_def, which then takes care to use the column names from the tupledesc for the output column names of the SELECT. For some reason though, we'd missed teaching make_ruledef to do similarly when it is printing an ON SELECT rule, even though this is exactly the same case. The results from pg_get_ruledef would then be different and arguably wrong. In particular, this breaks pre-v10 versions of pg_dump, which in some situations would define views by means of emitting a CREATE RULE ... ON SELECT command. Third-party tools might not be happy either. In passing, clean up some crufty code in make_viewdef; we'd apparently modernized the equivalent code in make_ruledef somewhere along the way, and missed this copy. Per report from Gilles Darold. Back-patch to all supported versions. Discussion: https://postgr.es/m/ec05659a-40ff-4510-fc45-ca9d965d0838@dalibo.com 24 July 2017, 19:16:31 UTC
9842c1b Noah Misch 24 July 2017, 06:53:27 UTC MSVC: Accept tcl86.lib in addition to tcl86t.lib. ActiveTcl8.6.4.1.299124-win32-x86_64-threaded.exe ships just tcl86.lib. Back-patch to 9.2, like the commit recognizing tcl86t.lib. 24 July 2017, 06:53:51 UTC
68a22bc Tom Lane 23 July 2017, 00:20:10 UTC Fix pg_dump's handling of event triggers. pg_dump with the --clean option failed to emit DROP EVENT TRIGGER commands for event triggers. In a closely related oversight, it also did not emit ALTER OWNER commands for event triggers. Since only superusers can create event triggers, the latter oversight is of little practical consequence ... but if we're going to record an owner for event triggers, then surely pg_dump should preserve it. Per complaint from Greg Atkins. Back-patch to 9.3 where event triggers were introduced. Discussion: https://postgr.es/m/20170722191142.yi4e7tzcg3iacclg@gmail.com 23 July 2017, 00:20:10 UTC
6d9de66 Tom Lane 21 July 2017, 18:20:43 UTC Stabilize postgres_fdw regression tests. The new test cases added in commit 8bf58c0d9 turn out to have output that can vary depending on the lc_messages setting prevailing on the test server. Hide the remote end's error messages to ensure stable output. This isn't a terribly desirable solution; we'd rather know that the connection failed for the expected reason and not some other one. But there seems little choice for the moment. Per buildfarm. Discussion: https://postgr.es/m/18419.1500658570@sss.pgh.pa.us 21 July 2017, 18:20:43 UTC
6971141 Tom Lane 21 July 2017, 16:51:38 UTC Re-establish postgres_fdw connections after server or user mapping changes. Previously, postgres_fdw would keep on using an existing connection even if the user did ALTER SERVER or ALTER USER MAPPING commands that should affect connection parameters. Teach it to watch for catcache invals on these catalogs and re-establish connections when the relevant catalog entries change. Per bug #14738 from Michal Lis. In passing, clean up some rather crufty decisions in commit ae9bfc5d6 about where fields of ConnCacheEntry should be reset. We now reset all the fields whenever we open a new connection. Kyotaro Horiguchi, reviewed by Ashutosh Bapat and myself. Back-patch to 9.3 where postgres_fdw appeared. Discussion: https://postgr.es/m/20170710113917.7727.10247@wrigleys.postgresql.org 21 July 2017, 16:51:38 UTC
a6088e1 Tom Lane 20 July 2017, 16:41:26 UTC Doc: clarify description of degenerate NATURAL joins. Claiming that NATURAL JOIN is equivalent to CROSS JOIN when there are no common column names is only strictly correct if it's an inner join; you can't say e.g. CROSS LEFT JOIN. Better to explain it as meaning JOIN ON TRUE, instead. Per a suggestion from David Johnston. Discussion: https://postgr.es/m/CAKFQuwb+mYszQhDS9f_dqRrk1=Pe-S6D=XMkAXcDf4ykKPmgKQ@mail.gmail.com 20 July 2017, 16:41:45 UTC
e947838 Tom Lane 20 July 2017, 15:29:36 UTC Fix dumping of outer joins with empty qual lists. Normally, a JoinExpr would have empty "quals" only if it came from CROSS JOIN syntax. However, it's possible to get to this state by specifying NATURAL JOIN between two tables with no common column names, and there might be other ways too. The code previously printed no ON clause if "quals" was empty; that's right for CROSS JOIN but syntactically invalid if it's some type of outer join. Fix by printing ON TRUE in that case. This got broken by commit 2ffa740be, which stopped using NATURAL JOIN syntax in ruleutils output due to its brittleness in the face of column renamings. Back-patch to 9.3 where that commit appeared. Per report from Tushar Ahuja. Discussion: https://postgr.es/m/98b283cd-6dda-5d3f-f8ac-87db8c76a3da@enterprisedb.com 20 July 2017, 15:29:36 UTC
cd969b2 Tom Lane 19 July 2017, 16:58:36 UTC Doc: add missing note about permissions needed to change log_lock_waits. log_lock_waits is PGC_SUSET, but config.sgml lacked the standard boilerplate sentence noting that. Report: https://postgr.es/m/20170719100838.19352.16320@wrigleys.postgresql.org 19 July 2017, 16:59:10 UTC
a64726e Tom Lane 17 July 2017, 20:43:03 UTC Doc: explain dollar quoting in the intro part of the pl/pgsql chapter. We're throwing people into the guts of the syntax with not much context; let's back up one step and point out that this goes inside a literal in a CREATE FUNCTION command. Per suggestion from Kurt Kartaltepe. Discussion: https://postgr.es/m/CACawnnyWAmH+au8nfZhLiFfWKjXy4d0kY+eZWfcxPRnjVfaa_Q@mail.gmail.com 17 July 2017, 20:43:40 UTC
1a3ec19 Tom Lane 17 July 2017, 19:28:17 UTC Merge large_object.sql test into largeobject.source. It seems pretty confusing to have tests named both largeobject and large_object. The latter is of very recent vintage (commit ff992c074), so get rid of it in favor of merging into the former. Also, enable the LO comment test that was added by commit 70ad7ed4e, since the later commit added the then-missing pg_upgrade functionality. The large_object.sql test case is almost completely redundant with that, but not quite: it seems like creating a user-defined LO with an OID in the system range might be an interesting case for pg_upgrade, so let's keep it. Like the earlier patch, back-patch to all supported branches. Discussion: https://postgr.es/m/18665.1500306372@sss.pgh.pa.us 17 July 2017, 19:28:17 UTC
f363368 Heikki Linnakangas 14 July 2017, 13:02:53 UTC Fix pg_basebackup output to stdout on Windows. When writing a backup to stdout with pg_basebackup on Windows, put stdout to binary mode. Any CR bytes in the output will otherwise be output incorrectly as CR+LF. In the passing, standardize on using "_setmode" instead of "setmode", for the sake of consistency. They both do the same thing, but according to MSDN documentation, setmode is deprecated. Fixes bug #14634, reported by Henry Boehlert. Patch by Haribabu Kommi. Backpatch to all supported versions. Discussion: https://www.postgresql.org/message-id/20170428082818.24366.13134@wrigleys.postgresql.org 14 July 2017, 13:03:23 UTC
0ecc407 Tom Lane 13 July 2017, 23:24:44 UTC Fix dumping of FUNCTION RTEs that contain non-function-call expressions. The grammar will only accept something syntactically similar to a function call in a function-in-FROM expression. However, there are various ways to input something that ruleutils.c won't deparse that way, potentially leading to a view or rule that fails dump/reload. Fix by inserting a dummy CAST around anything that isn't going to deparse as a function (which is one of the ways to get something like that in there in the first place). In HEAD, also make use of the infrastructure added by this to avoid emitting unnecessary parentheses in CREATE INDEX deparsing. I did not change that in back branches, thinking that people might find it to be unexpected/unnecessary behavioral change. In HEAD, also fix incorrect logic for when to add extra parens to partition key expressions. Somebody apparently thought they could get away with simpler logic than pg_get_indexdef_worker has, but they were wrong --- a counterexample is PARTITION BY LIST ((a[1])). Ignoring the prettyprint flag for partition expressions isn't exactly a nice solution anyway. This has been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/10477.1499970459@sss.pgh.pa.us 13 July 2017, 23:24:44 UTC
5c5af32 Heikki Linnakangas 13 July 2017, 12:47:02 UTC Fix race between GetNewTransactionId and GetOldestActiveTransactionId. The race condition goes like this: 1. GetNewTransactionId advances nextXid e.g. from 100 to 101 2. GetOldestActiveTransactionId reads the new nextXid, 101 3. GetOldestActiveTransactionId loops through the proc array. There are no active XIDs there, so it returns 101 as the oldest active XID. 4. GetNewTransactionid stores XID 100 to MyPgXact->xid So, GetOldestActiveTransactionId returned XID 101, even though 100 only just started and is surely still running. This would be hard to hit in practice, and even harder to spot any ill effect if it happens. GetOldestActiveTransactionId is only used when creating a checkpoint in a master server, and the race condition can only happen on an online checkpoint, as there are no backends running during a shutdown checkpoint. The oldestActiveXid value of an online checkpoint is only used when starting up a hot standby server, to determine the starting point where pg_subtrans is initialized from. For the race condition to happen, there must be no other XIDs in the proc array that would hold back the oldest-active XID value, which means that the missed XID must be a top transaction's XID. However, pg_subtrans is not used for top XIDs, so I believe an off-by-one error is in fact inconsequential. Nevertheless, let's fix it, as it's clearly wrong and the fix is simple. This has been wrong ever since hot standby was introduced, so backport to all supported versions. Discussion: https://www.postgresql.org/message-id/e7258662-82b6-7a45-56d4-99b337a32bf7@iki.fi 13 July 2017, 12:48:30 UTC
aea1a3f Tom Lane 12 July 2017, 22:00:04 UTC Fix ruleutils.c for domain-over-array cases, too. Further investigation shows that ruleutils isn't quite up to speed either for cases where we have a domain-over-array: it needs to be prepared to look past a CoerceToDomain at the top level of field and element assignments, else it decompiles them incorrectly. Potentially this would result in failure to dump/reload a rule, if it looked like the one in the new test case. (I also added a test for EXPLAIN; that output isn't broken, but clearly we need more test coverage here.) Like commit b1cb32fb6, this bug is reachable in cases we already support, so back-patch all the way. 12 July 2017, 22:00:04 UTC
11854de Heikki Linnakangas 12 July 2017, 19:03:38 UTC Reduce memory usage of tsvector type analyze function. compute_tsvector_stats() detoasted and kept in memory every tsvector value in the sample, but that can be a lot of memory. The original bug report described a case using over 10 gigabytes, with statistics target of 10000 (the maximum). To fix, allocate a separate copy of just the lexemes that we keep around, and free the detoasted tsvector values as we go. This adds some palloc/pfree overhead, when you have a lot of distinct lexemes in the sample, but it's better than running out of memory. Fixes bug #14654 reported by James C. Reviewed by Tom Lane. Backport to all supported versions. Discussion: https://www.postgresql.org/message-id/20170514200602.1451.46797@wrigleys.postgresql.org 12 July 2017, 19:04:08 UTC
cf3e276 Heikki Linnakangas 12 July 2017, 14:07:35 UTC Fix variable and type name in comment. Kyotaro Horiguchi Discussion: https://www.postgresql.org/message-id/20170711.163441.241981736.horiguchi.kyotaro@lab.ntt.co.jp 12 July 2017, 14:09:16 UTC
66a3ea0 Heikki Linnakangas 12 July 2017, 09:30:50 UTC Remove unnecessary braces, to match the surrounding style. Mostly in the new subscription-related commands. Backport the few that were also present in older versions. Thomas Munro Discussion: https://www.postgresql.org/message-id/CAEepm=3CyW1QmXcXJXmqiJXtXzFDc8SvSfnxkEGD3Bkv2SrkeQ@mail.gmail.com 12 July 2017, 09:31:20 UTC
521fede Tom Lane 11 July 2017, 20:48:59 UTC Fix multiple assignments to a column of a domain type. We allow INSERT and UPDATE commands to assign to the same column more than once, as long as the assignments are to subfields or elements rather than the whole column. However, this failed when the target column was a domain over array rather than plain array. Fix by teaching process_matched_tle() to look through CoerceToDomain nodes, and add relevant test cases. Also add a group of test cases exercising domains over array of composite. It's doubtless accidental that CREATE DOMAIN allows this case while not allowing straight domain over composite; but it does, so we'd better make sure we don't break it. (I could not find any documentation mentioning either side of that, so no doc changes.) It's been like this for a long time, so back-patch to all supported branches. Discussion: https://postgr.es/m/4206.1499798337@sss.pgh.pa.us 11 July 2017, 20:48:59 UTC
59892b1 Tom Lane 10 July 2017, 15:00:09 UTC On Windows, retry process creation if we fail to reserve shared memory. We've heard occasional reports of backend launch failing because pgwin32_ReserveSharedMemoryRegion() fails, indicating that something has already used that address space in the child process. It's not very clear what, given that we disable ASLR in Windows builds, but suspicion falls on antivirus products. It'd be better if we didn't have to disable ASLR, anyway. So let's try to ameliorate the problem by retrying the process launch after such a failure, up to 100 times. Patch by me, based on previous work by Amit Kapila and others. This is a longstanding issue, so back-patch to all supported branches. Discussion: https://postgr.es/m/CAA4eK1+R6hSx6t_yvwtx+NRzneVp+MRqXAdGJZChcau8Uij-8g@mail.gmail.com 10 July 2017, 15:00:09 UTC
1b2e875 Tom Lane 10 July 2017, 04:08:19 UTC Doc: clarify wording about tool requirements in sourcerepo.sgml. Original wording had confusingly vague antecedent for "they", so replace that with a more repetitive but clearer formulation. In passing, make the link to the installation requirements section more specific. Per gripe from Martin Mai, though this is not the fix he initially proposed. Discussion: https://postgr.es/m/CAN_NWRu-cWuNaiXUjV3m4H-riWURuPW=j21bSaLADs6rjjzXgQ@mail.gmail.com 10 July 2017, 04:08:53 UTC
f371cc9 Teodor Sigaev 06 July 2017, 14:20:56 UTC Fix potential data corruption during freeze Fix oversight in 3b97e6823b94 bug fix. Bitwise AND is used instead of OR and it cleans all bits in t_infomask heap tuple field. Backpatch to 9.3 06 July 2017, 14:20:56 UTC
543e00b Heikki Linnakangas 03 July 2017, 11:51:51 UTC Treat clean shutdown of an SSL connection same as the non-SSL case. If the client closes an SSL connection, treat it the same as EOF on a non-SSL connection. In particular, don't write a message in the log about that. Michael Paquier. Discussion: https://www.postgresql.org/message-id/CAB7nPqSfyVV42Q2acFo%3DvrvF2gxoZAMJLAPq3S3KkjhZAYi7aw@mail.gmail.com 03 July 2017, 11:53:10 UTC
cc154d9 Tom Lane 28 June 2017, 16:30:16 UTC Second try at fixing tcp_keepalives_idle option on Solaris. Buildfarm evidence shows that TCP_KEEPALIVE_THRESHOLD doesn't exist after all on Solaris < 11. This means we need to take positive action to prevent the TCP_KEEPALIVE code path from being taken on that platform. I've chosen to limit it with "&& defined(__darwin__)", since it's unclear that anyone else would follow Apple's precedent of spelling the symbol that way. Also, follow a suggestion from Michael Paquier of eliminating code duplication by defining a couple of intermediate symbols for the socket option. In passing, make some effort to reduce the number of translatable messages by replacing "setsockopt(foo) failed" with "setsockopt(%s) failed", etc, throughout the affected files. And update relevant documentation so that it doesn't claim to provide an exhaustive list of the possible socket option names. Like the previous commit (f0256c774), back-patch to all supported branches. Discussion: https://postgr.es/m/20170627163757.25161.528@wrigleys.postgresql.org 28 June 2017, 16:30:16 UTC
back to top