https://github.com/postgres/postgres

sort by:
Revision Author Date Message Commit Date
b96df2c Stamp 9.5beta1. 05 October 2015, 19:09:44 UTC
7d88b3d docs: update guidelines on when to use GIN and GiST indexes Report by Tomas Vondra Backpatch through 9.5 05 October 2015, 17:38:36 UTC
d623591 Docs: explain contrib/pg_stat_statements' handling of GC failure. Failure to perform garbage collection now has a user-visible effect, so explain that and explain that reducing pgss_max is the way to prevent it. Per gripe from Andrew Dunstan. 05 October 2015, 16:44:20 UTC
c0f058e Fix insufficiently-portable regression test case. Some of the buildfarm members are evidently miserly enough of stack space to pass the originally-committed form of this test. Increase the requirement 10X to hopefully ensure that it fails as-expected everywhere. Security: CVE-2015-5289 05 October 2015, 16:19:14 UTC
149a8cd Translation updates Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 23a52bc86edcd39c3c6b80ee1f7374759c8711f8 05 October 2015, 15:01:07 UTC
808f1bd Last-minute updates for release notes. Add entries for security and not-quite-security issues. Security: CVE-2015-5288, CVE-2015-5289 05 October 2015, 14:57:44 UTC
56805a4 Remove outdated comment about relation level autovacuum freeze limits. The documentation for the autovacuum_multixact_freeze_max_age and autovacuum_freeze_max_age relation level parameters contained: "Note that while you can set autovacuum_multixact_freeze_max_age very small, or even zero, this is usually unwise since it will force frequent vacuuming." which hasn't been true since these options were made relation options, instead of residing in the pg_autovacuum table (834a6da4f7). Remove the outdated sentence. Even the lowered limits from 2596d70 are high enough that this doesn't warrant calling out the risk in the CREATE TABLE docs. Per discussion with Tom Lane and Alvaro Herrera Discussion: 26377.1443105453@sss.pgh.pa.us Backpatch: 9.0- (in parts) 05 October 2015, 14:51:03 UTC
f9bb9c0 Add regression tests for INSERT/UPDATE+RETURNING This adds regressions tests which are specific to INSERT+RETURNING and UPDATE+RETURNING to ensure that the SELECT policies are added as WithCheckOptions (and should therefore throw an error when the policy is violated). Per suggestion from Andres. Back-patch to 9.5 as the prior commit was. 05 October 2015, 14:14:51 UTC
7bed97d Prevent stack overflow in query-type functions. The tsquery, ltxtquery and query_int data types have a common ancestor. Having acquired check_stack_depth() calls independently, each was missing at least one call. Back-patch to 9.0 (all supported versions). 05 October 2015, 14:06:33 UTC
acf0da1 Prevent stack overflow in container-type functions. A range type can name another range type as its subtype, and a record type can bear a column of another record type. Consequently, functions like range_cmp() and record_recv() are recursive. Functions at risk include operator family members and referents of pg_type regproc columns. Treat as recursive any such function that looks up and calls the same-purpose function for a record column type or the range subtype. Back-patch to 9.0 (all supported versions). An array type's element type is never itself an array type, so array functions are unaffected. Recursion depth proportional to array dimensionality, found in array_dim_to_jsonb(), is fine thanks to MAXDIM. 05 October 2015, 14:06:33 UTC
98f30d2 Prevent stack overflow in json-related functions. Sufficiently-deep recursion heretofore elicited a SIGSEGV. If an application constructs PostgreSQL json or jsonb values from arbitrary user input, application users could have exploited this to terminate all active database connections. That applies to 9.3, where the json parser adopted recursive descent, and later versions. Only row_to_json() and array_to_json() were at risk in 9.2, both in a non-security capacity. Back-patch to 9.2, where the json type was introduced. Oskari Saarenmaa, reviewed by Michael Paquier. Security: CVE-2015-5289 05 October 2015, 14:06:33 UTC
4d67522 pgcrypto: Detect and report too-short crypt() salts. Certain short salts crashed the backend or disclosed a few bytes of backend memory. For existing salt-induced error conditions, emit a message saying as much. Back-patch to 9.0 (all supported versions). Josh Kupershmidt Security: CVE-2015-5288 05 October 2015, 14:06:33 UTC
bd90147 Apply SELECT policies in INSERT/UPDATE+RETURNING Similar to 7d8db3e, given that INSERT+RETURNING requires SELECT rights on the table, apply the SELECT policies as WCOs to the tuples being inserted. Apply the same logic to UPDATE+RETURNING. Back-patch to 9.5 where RLS was added. 05 October 2015, 11:55:11 UTC
31fb4df Do not write out WCOs in Query The WithCheckOptions list in Query are only populated during rewrite and do not need to be written out or read in as part of a Query structure. Further, move WithCheckOptions to the bottom and add comments to clarify that it is only populated during rewrite. Back-patch to 9.5 with a catversion bump, as we are still in alpha. 05 October 2015, 11:38:56 UTC
0577821 Re-Align *_freeze_max_age reloption limits with corresponding GUC limits. In 020235a5754 I lowered the autovacuum_*freeze_max_age minimums to allow for easier testing of wraparounds. I did not touch the corresponding per-table limits. While those don't matter for the purpose of wraparound, it seems more consistent to lower them as well. It's noteworthy that the previous reloption lower limit for autovacuum_multixact_freeze_max_age was too high by one magnitude, even before 020235a5754. Discussion: 26377.1443105453@sss.pgh.pa.us Backpatch: back to 9.0 (in parts), like the prior patch 05 October 2015, 09:56:58 UTC
90f334d ALTER TABLE .. FORCE ROW LEVEL SECURITY To allow users to force RLS to always be applied, even for table owners, add ALTER TABLE .. FORCE ROW LEVEL SECURITY. row_security=off overrides FORCE ROW LEVEL SECURITY, to ensure pg_dump output is complete (by default). Also add SECURITY_NOFORCE_RLS context to avoid data corruption when ALTER TABLE .. FORCE ROW SECURITY is being used. The SECURITY_NOFORCE_RLS security context is used only during referential integrity checks and is only considered in check_enable_rls() after we have already checked that the current user is the owner of the relation (which should always be the case during referential integrity checks). Back-patch to 9.5 where RLS was added. 05 October 2015, 01:05:18 UTC
e78dc6b Release notes for 9.5beta1, 9.4.5, 9.3.10, 9.2.14, 9.1.19, 9.0.23. 04 October 2015, 23:38:00 UTC
39a716d Improve contrib/pg_stat_statements' handling of garbage collection failure. If we can't read the query texts file (whether because out-of-memory, or for some other reason), give up and reset the file to empty, discarding all stored query texts, though not the statistics per se. We used to leave things alone and hope for better luck next time, but the problem is that the file is only going to get bigger and even harder to slurp into memory. Better to do something that will get us out of trouble. Likewise reset the file to empty for any other failure within gc_qtexts(). The previous behavior after a write error was to discard query texts but not do anything to truncate the file, which is just weird. Also, increase the maximum supported file size from MaxAllocSize to MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage collection successfully. Also, fix recalculation of mean_query_len within entry_dealloc() to match the calculation in gc_qtexts(). The previous coding overlooked the possibility of dropped texts (query_len == -1) and would underestimate the mean of the remaining entries in such cases, thus possibly causing excess garbage collection cycles. In passing, add some errdetail to the log entry that complains about insufficient memory to read the query texts file, which after all was Jim Nasby's original complaint. Back-patch to 9.4 where the current handling of query texts was introduced. Peter Geoghegan, rather editorialized upon by me 04 October 2015, 21:58:30 UTC
e5c94c7 Further twiddling of nodeHash.c hashtable sizing calculation. On reflection, the submitted patch didn't really work to prevent the request size from exceeding MaxAllocSize, because of the fact that we'd happily round nbuckets up to the next power of 2 after we'd limited it to max_pointers. The simplest way to enforce the limit correctly is to round max_pointers down to a power of 2 when it isn't one already. (Note that the constraint to INT_MAX / 2, if it were doing anything useful at all, is properly applied after that.) 04 October 2015, 19:55:07 UTC
ca5b42d Fix some issues in new hashtable size calculations in nodeHash.c. Limit the size of the hashtable pointer array to not more than MaxAllocSize, per reports from Kouhei Kaigai and others of "invalid memory alloc request size" failures. There was discussion of allowing the array to get larger than that by using the "huge" palloc API, but so far no proof that that is actually a good idea, and at this point in the 9.5 cycle major changes from old behavior don't seem like the way to go. Fix a rather serious secondary bug in the new code, which was that it didn't ensure nbuckets remained a power of 2 when recomputing it for the multiple-batch case. Clean up sloppy division of labor between ExecHashIncreaseNumBuckets and its sole call site. 04 October 2015, 18:06:51 UTC
544ccf6 Disallow invalid path elements in jsonb_set Null path elements and, where the object is an array, invalid integer elements now cause an error. Incorrect behaviour noted by Thom Brown, patch from Dmitry Dolgov. Backpatch to 9.5 where jsonb_set was introduced 04 October 2015, 17:48:24 UTC
e45f8f8 Group cluster_name and update_process_title settings together 04 October 2015, 16:29:50 UTC
4365d9c Document that row_security is a boolean GUC. Oversight in commit 537bd178c73b1d25938347b17e9e3e62898fc231. Back-patch to 9.5, like that commit. 04 October 2015, 00:20:52 UTC
01ba789 Make BYPASSRLS behave like superuser RLS bypass. Specifically, make its effect independent from the row_security GUC, and make it affect permission checks pertinent to views the BYPASSRLS role owns. The row_security GUC thereby ceases to change successful-query behavior; it can only make a query fail with an error. Back-patch to 9.5, where BYPASSRLS was introduced. 04 October 2015, 00:20:50 UTC
cfddb5d Improve errhint() about replication slot naming restrictions. The existing hint talked about "may only contain letters", but the actual requirement is more strict: only lower case letters are allowed. Reported-By: Rushabh Lathia Author: Rushabh Lathia Discussion: AGPqQf2x50qcwbYOBKzb4x75sO_V3g81ZsA8+Ji9iN5t_khFhQ@mail.gmail.com Backpatch: 9.4-, where replication slots were added 03 October 2015, 13:29:35 UTC
7285d66 Fix several bugs related to ON CONFLICT's EXCLUDED pseudo relation. Four related issues: 1) attnos/varnos/resnos for EXCLUDED were out of sync when a column after one dropped in the underlying relation was referenced. 2) References to whole-row variables (i.e. EXCLUDED.*) lead to errors. 3) It was possible to reference system columns in the EXCLUDED pseudo relations, even though they would not have valid contents. 4) References to EXCLUDED were rewritten by the RLS machinery, as EXCLUDED was treated as if it were the underlying relation. To fix the first two issues, generate the excluded targetlist with dropped columns in mind and add an entry for whole row variables. Instead of unconditionally adding a wholerow entry we could pull up the expression if needed, but doing it unconditionally seems simpler. The wholerow entry is only really needed for ruleutils/EXPLAIN support anyway. The remaining two issues are addressed by changing the EXCLUDED RTE to have relkind = composite. That fits with EXCLUDED not actually being a real relation, and allows to treat it differently in the relevant places. scanRTEForColumn now skips looking up system columns when the RTE has a composite relkind; fireRIRrules() already had a corresponding check, thereby preventing RLS expansion on EXCLUDED. Also add tests for these issues, and improve a few comments around excluded handling in setrefs.c. Reported-By: Peter Geoghegan, Geoff Winkless Author: Andres Freund, Amit Langote, Peter Geoghegan Discussion: CAEzk6fdzJ3xYQZGbcuYM2rBd2BuDkUksmK=mY9UYYDugg_GgZg@mail.gmail.com, CAM3SWZS+CauzbiCEcg-GdE6K6ycHE_Bz6Ksszy8AoixcMHOmsA@mail.gmail.com Backpatch: 9.5, where ON CONFLICT was introduced 03 October 2015, 13:12:50 UTC
0777a88 doc: Update URLs of external projects 03 October 2015, 01:51:56 UTC
5f90492 doc: Make some index terms and terminology more consistent 03 October 2015, 01:23:44 UTC
19b06cc Update time zone data files to tzdata release 2015g. DST law changes in Cayman Islands, Fiji, Moldova, Morocco, Norfolk Island, North Korea, Turkey, Uruguay. New zone America/Fort_Nelson for Canadian Northern Rockies. 02 October 2015, 23:15:49 UTC
63e86ec Clarify FDW documentation about ON CONFLICT. Etsuro Fujita, reviewed by Peter Geoghegan 02 October 2015, 20:57:28 UTC
bdc5d95 Add recursion depth protection to LIKE matching. Since MatchText() recurses, it could in principle be driven to stack overflow, although quite a long pattern would be needed. 02 October 2015, 19:00:52 UTC
20c6277 Add recursion depth protections to regular expression matching. Some of the functions in regex compilation and execution recurse, and therefore could in principle be driven to stack overflow. The Tcl crew has seen this happen in practice in duptraverse(), though their fix was to put in a hard-wired limit on the number of recursive levels, which is not too appetizing --- fortunately, we have enough infrastructure to check the actually available stack. Greg Stark has also seen it in other places while fuzz testing on a machine with limited stack space. Let's put guards in to prevent crashes in all these places. Since the regex code would leak memory if we simply threw elog(ERROR), we have to introduce an API that checks for stack depth without throwing such an error. Fortunately that's not difficult. 02 October 2015, 18:51:58 UTC
51f2359 Fix potential infinite loop in regular expression execution. In cfindloop(), if the initial call to shortest() reports that a zero-length match is possible at the current search start point, but then it is unable to construct any actual match to that, it'll just loop around with the same start point, and thus make no progress. We need to force the start point to be advanced. This is safe because the loop over "begin" points has already tried and failed to match starting at "close", so there is surely no need to try that again. This bug was introduced in commit e2bd904955e2221eddf01110b1f25002de2aaa83, wherein we allowed continued searching after we'd run out of match possibilities, but evidently failed to think hard enough about exactly where we needed to search next. Because of the way this code works, such a match failure is only possible in the presence of backrefs --- otherwise, shortest()'s judgment that a match is possible should always be correct. That probably explains how come the bug has escaped detection for several years. The actual fix is a one-liner, but I took the trouble to add/improve some comments related to the loop logic. After fixing that, the submitted test case "()*\1" didn't loop anymore. But it reported failure, though it seems like it ought to match a zero-length string; both Tcl and Perl think it does. That seems to be from overenthusiastic optimization on my part when I rewrote the iteration match logic in commit 173e29aa5deefd9e71c183583ba37805c8102a72: we can't just "declare victory" for a zero-length match without bothering to set match data for capturing parens inside the iterator node. Per fuzz testing by Greg Stark. The first part of this is a bug in all supported branches, and the second part is a bug since 9.2 where the iteration rewrite happened. 02 October 2015, 18:26:36 UTC
bb704a7 Add some more query-cancel checks to regular expression matching. Commit 9662143f0c35d64d7042fbeaf879df8f0b54be32 added infrastructure to allow regular-expression operations to be terminated early in the event of SIGINT etc. However, fuzz testing by Greg Stark disclosed that there are still cases where regex compilation could run for a long time without noticing a cancel request. Specifically, the fixempties() phase never adds new states, only new arcs, so it doesn't hit the cancel check I'd put in newstate(). Add one to newarc() as well to cover that. Some experimentation of my own found that regex execution could also run for a long time despite a pending cancel. We'd put a high-level cancel check into cdissect(), but there was none inside the core text-matching routines longest() and shortest(). Ordinarily those inner loops are very very fast ... but in the presence of lookahead constraints, not so much. As a compromise, stick a cancel check into the stateset cache-miss function, which is enough to guarantee a cancel check at least once per lookahead constraint test. Making this work required more attention to error handling throughout the regex executor. Henry Spencer had apparently originally intended longest() and shortest() to be incapable of incurring errors while running, so neither they nor their subroutines had well-defined error reporting behaviors. However, that was already broken by the lookahead constraint feature, since lacon() can surely suffer an out-of-memory failure --- which, in the code as it stood, might never be reported to the user at all, but just silently be treated as a non-match of the lookahead constraint. Normalize all that by inserting explicit error tests as needed. I took the opportunity to add some more comments to the code, too. Back-patch to all supported branches, like the previous patch. 02 October 2015, 17:45:39 UTC
c56b2aa Docs: add disclaimer about hazards of using regexps from untrusted sources. It's not terribly hard to devise regular expressions that take large amounts of time and/or memory to process. Recent testing by Greg Stark has also shown that machines with small stack limits can be driven to stack overflow by suitably crafted regexps. While we intend to fix these things as much as possible, it's probably impossible to eliminate slow-execution cases altogether. In any case we don't want to treat such things as security issues. The history of that code should already discourage prudent DBAs from allowing execution of regexp patterns coming from possibly-hostile sources, but it seems like a good idea to warn about the hazard explicitly. Currently, similar_escape() allows access to enough of the underlying regexp behavior that the warning has to apply to SIMILAR TO as well. We might be able to make it safer if we tightened things up to allow only SQL-mandated capabilities in SIMILAR TO; but that would be a subtly non-backwards-compatible change, so it requires discussion and probably could not be back-patched. Per discussion among pgsql-security list. 02 October 2015, 17:30:42 UTC
1dc6f55 Docs: add another example of creating a range type. The "floatrange" example is a bit too simple because float8mi can be used without any additional type conversion. Add an example that does have to account for that, and do some minor other wordsmithing. 02 October 2015, 16:20:08 UTC
65dc1fc Don't disable commit_ts in standby if enabled locally Bug noticed by Fujii Masao 02 October 2015, 15:49:01 UTC
0f51a84 pg_rewind: Improve some messages The output of a typical pg_rewind run contained a mix of capitalized and not-capitalized and punctuated and not-punctuated phrases for no apparent reason. Make that consistent. Also fix some problems in other messages. 02 October 2015, 01:52:46 UTC
867bc68 Fix message punctuation according to style guide 02 October 2015, 01:52:46 UTC
5ea47e8 Fix pg_dump to handle inherited NOT VALID check constraints correctly. This case seems to have been overlooked when unvalidated check constraints were introduced, in 9.2. The code would attempt to dump such constraints over again for each child table, even though adding them to the parent table is sufficient. In 9.2 and 9.3, also fix contrib/pg_upgrade/Makefile so that the "make clean" target fully cleans up after a failed test. This evidently got dealt with at some point in 9.4, but it wasn't back-patched. I ran into it while testing this fix ... Per bug #13656 from Ingmar Brouns. 01 October 2015, 20:20:15 UTC
a742ef8 Fix commit_ts for standby Module initialization was still not completely correct after commit 6b61955135e9, per crash report from Takashi Ohnishi. To fix, instead of trying to monkey around with the value of the GUC setting directly, add a separate boolean flag that enables the feature on a standby, but only for the startup (recovery) process, when it sees that its master server has the feature enabled. Discussion: http://www.postgresql.org/message-id/ca44c6c7f9314868bdc521aea4f77cbf@MP-MSGSS-MBX004.msg.nttdata.co.jp Also change the deactivation routine to delete all segment files rather than leaving the last one around. (This doesn't need separate WAL-logging, because on recovery we execute the same deactivation routine anyway.) In passing, clean up the code structure somewhat, particularly so that xlog.c doesn't know so much about when to activate/deactivate the feature. Thanks to Fujii Masao for testing and Petr Jelínek for off-list discussion. Back-patch to 9.5, where commit_ts was introduced. 01 October 2015, 18:06:55 UTC
d312cc3 Fix documentation error in commit 8703059c6b55c427100e00a09f66534b6ccbfaa1. Etsuro Fujita spotted a thinko in the README commentary. 01 October 2015, 14:32:12 UTC
c9a8d05 Fix mention of htup.h in storage.sgml Previously it was documented that the details on HeapTupleHeaderData struct could be found in htup.h. This is not correct because it's now defined in htup_details.h. Back-patch to 9.3 where the definition of HeapTupleHeaderData struct was moved from htup.h to htup_details.h. Michael Paquier 01 October 2015, 14:13:07 UTC
8c8a834 Improve LISTEN startup time when there are many unread notifications. If some existing listener is far behind, incoming new listener sessions would start from that session's read pointer and then need to advance over many already-committed notification messages, which they have no interest in. This was expensive in itself and also thrashed the pg_notify SLRU buffers a lot more than necessary. We can improve matters considerably in typical scenarios, without much added cost, by starting from the furthest-ahead read pointer, not the furthest-behind one. We do have to consider only sessions in our own database when doing this, which requires an extra field in the data structure, but that's a pretty small cost. Back-patch to 9.0 where the current LISTEN/NOTIFY logic was introduced. Matt Newell, slightly adjusted by me 01 October 2015, 03:32:23 UTC
91d97f0 Don't dump core when destroying an unused ParallelContext. If a transaction or subtransaction creates a ParallelContext but ends without calling InitializeParallelDSM, the previous code would seg fault. Fix that. 30 September 2015, 22:42:10 UTC
75096c4 Include policies based on ACLs needed When considering which policies should be included, rather than look at individual bits of the query (eg: if a RETURNING clause exists, or if a WHERE clause exists which is referencing the table, or if it's a FOR SHARE/UPDATE query), consider any case where we've determined the user needs SELECT rights on the relation while doing an UPDATE or DELETE to be a case where we apply SELECT policies, and any case where we've deteremind that the user needs UPDATE rights on the relation while doing a SELECT to be a case where we apply UPDATE policies. This simplifies the logic and addresses concerns that a user could use UPDATE or DELETE with a WHERE clauses to determine if rows exist, or they could use SELECT .. FOR UPDATE to lock rows which they are not actually allowed to modify through UPDATE policies. Use list_append_unique() to avoid adding the same quals multiple times, as, on balance, the cost of checking when adding the quals will almost always be cheaper than keeping them and doing busywork for each tuple during execution. Back-patch to 9.5 where RLS was added. 30 September 2015, 11:39:31 UTC
3c4c5ac Fix incorrect tps number calculation in "excluding connections establishing". The tolerance (larger than actual tps number) increases as the number of threads decreases. The bug has been there since the thread support was introduced in 9.0. Because back patching introduces incompatible behavior changes regarding the tps number, the fix is committed to master and 9.5 stable branches only. Problem spotted by me and fix proposed by Fabien COELHO. Note that his original patch included more than fixes (a code re-factoring) which is not related to the problem and I omitted the part. 30 September 2015, 01:36:23 UTC
d8c7bb2 Code review for transaction commit timestamps There are three main changes here: 1. No longer cause a start failure in a standby if the feature is disabled in postgresql.conf but enabled in the master. This reverts one part of commit 4f3924d9cd43; what we keep is the ability of the standby to activate/deactivate the module (which includes creating and removing segments as appropriate) during replay of such actions in the master. 2. Replay WAL records affecting commitTS even if the feature is disabled. This means the standby will always have the same state as the master after replay. 3. Have COMMIT PREPARE record the transaction commit time as well. We were previously only applying it in the normal transaction commit path. Author: Petr Jelínek Discussion: http://www.postgresql.org/message-id/CAHGQGwHereDzzzmfxEBYcVQu3oZv6vZcgu1TPeERWbDc+gQ06g@mail.gmail.com Discussion: http://www.postgresql.org/message-id/CAHGQGwFuzfO4JscM9LCAmCDCxp_MfLvN4QdB+xWsS-FijbjTYQ@mail.gmail.com Additionally, I cleaned up nearby code related to replication origins, which I found a bit hard to follow, and fixed a couple of typos. Backpatch to 9.5, where this code was introduced. Per bug reports from Fujii Masao and subsequent discussion. 29 September 2015, 17:40:56 UTC
a16b9b1 Fix plperl to handle non-ASCII error message texts correctly. We were passing error message texts to croak() verbatim, which turns out not to work if the text contains non-ASCII characters; Perl mangles their encoding, as reported in bug #13638 from Michal Leinweber. To fix, convert the text into a UTF8-encoded SV first. It's hard to test this without risking failures in different database encodings; but we can follow the lead of plpython, which is already assuming that no-break space (U+00A0) has an equivalent in all encodings we care about running the regression tests in (cf commit 2dfa15de5). Back-patch to 9.1. The code is quite different in 9.0, and anyway it seems too risky to put something like this into 9.0's final minor release. Alex Hunsaker, with suggestions from Tim Bunce and Tom Lane 29 September 2015, 14:52:22 UTC
9a6fbc2 Comment update for join pushdown. Etsuro Fujita 29 September 2015, 11:50:36 UTC
60fcee9 Fix compiler warning for non-TIOCGWINSZ case Backpatch to 9.5 where the error was introduced. 28 September 2015, 22:43:33 UTC
c4e6d50 Fix compiler warning about unused function in non-readline case. Backpatch to all live branches to keep the code in sync. 28 September 2015, 22:35:06 UTC
aea76d1 Fix "sesssion" typo It was introduced alongside replication origins, by commit 5aa2350426c, so backpatch to 9.5. Pointed out by Fujii Masao 28 September 2015, 22:13:42 UTC
b67c9c1 Fix poor errno handling in libpq's version of our custom OpenSSL BIO. Thom Brown reported that SSL connections didn't seem to work on Windows in 9.5. Asif Naeem figured out that the cause was my_sock_read() looking at "errno" when it needs to look at "SOCK_ERRNO". This mistake was introduced in commit 680513ab79c7e12e402a2aad7921b95a25a4bcc8, which cloned the backend's custom SSL BIO code into libpq, and didn't translate the errno handling properly. Moreover, it introduced unnecessary errno save/restore logic, which was particularly confusing because it was incomplete; and it failed to check for all three of EINTR, EAGAIN, and EWOULDBLOCK in my_sock_write. (That might not be necessary; but since we're copying well-tested backend code that does do that, it seems prudent to copy it faithfully.) 28 September 2015, 22:02:44 UTC
ce58502 Ensure a few policies remain for pg_upgrade To make sure that pg_dump/pg_restore function properly with RLS policies, arrange to have a few of them left around at the end of the regression tests. Back-patch to 9.5 where RLS was added. 28 September 2015, 19:48:45 UTC
90586ef Fix ON CONFLICT DO UPDATE for tables with oids. When taking the UPDATE path in an INSERT .. ON CONFLICT .. UPDATE tables with oids were not supported. The tuple generated by the update target list was projected without space for an oid - a simple oversight. Reported-By: Peter Geoghegan Author: Andres Freund Backpatch: 9.5, where ON CONFLICT was introduced 28 September 2015, 17:30:03 UTC
80e2694 Don't try to create a temp install without abs_top_builddir. Otherwise, we effectively act as if abs_top_builddir were the root directory, which is quite dangerous if the user happens to have permissions to do things there. This can crop up in PGXS builds, for example. Report by Sandro Santilli, patch by me, review by Noah Misch. 28 September 2015, 14:52:19 UTC
27af56b pg_dump: Fix some messages Make quoting style match existing style. Improve plural support. 28 September 2015, 00:34:40 UTC
90d0377 reindexdb: Fix mistake in help output 27 September 2015, 15:38:34 UTC
63ab1a3 pg_ctl: Improve help formatting and order 27 September 2015, 01:10:51 UTC
0160c1d doc: Tweak "cube" index entry With the arrival of the CUBE key word/feature, the index entries for the cube extension and the CUBE feature were collapsed into one. Tweak the entry for the cube extension so they are separate entries. 27 September 2015, 01:02:24 UTC
6e8af37 Remove legacy multixact truncation support. In 9.5 and master there is no need to support legacy truncation. This is just committed separately to make it easier to backpatch the WAL logged multixact truncation to 9.3 and 9.4 if we later decide to do so. I bumped master's magic from 0xD086 to 0xD088 and 9.5's from 0xD085 to 0xD087 to avoid 9.5 reusing a value that has been in use on master while keeping the numbers increasing between major versions. Discussion: 20150621192409.GA4797@alap3.anarazel.de Backpatch: 9.5 26 September 2015, 17:04:25 UTC
bd7c348 Rework the way multixact truncations work. The fact that multixact truncations are not WAL logged has caused a fair share of problems. Amongst others it requires to do computations during recovery while the database is not in a consistent state, delaying truncations till checkpoints, and handling members being truncated, but offset not. We tried to put bandaids on lots of these issues over the last years, but it seems time to change course. Thus this patch introduces WAL logging for multixact truncations. This allows: 1) to perform the truncation directly during VACUUM, instead of delaying it to the checkpoint. 2) to avoid looking at the offsets SLRU for truncation during recovery, we can just use the master's values. 3) simplify a fair amount of logic to keep in memory limits straight, this has gotten much easier During the course of fixing this a bunch of additional bugs had to be fixed: 1) Data was not purged from memory the member's SLRU before deleting segments. This happened to be hard or impossible to hit due to the interlock between checkpoints and truncation. 2) find_multixact_start() relied on SimpleLruDoesPhysicalPageExist - but that doesn't work for offsets that haven't yet been flushed to disk. Add code to flush the SLRUs to fix. Not pretty, but it feels slightly safer to only make decisions based on actual on-disk state. 3) find_multixact_start() could be called concurrently with a truncation and thus fail. Via SetOffsetVacuumLimit() that could lead to a round of emergency vacuuming. The problem remains in pg_get_multixact_members(), but that's quite harmless. For now this is going to only get applied to 9.5+, leaving the issues in the older branches in place. It is quite possible that we need to backpatch at a later point though. For the case this gets backpatched we need to handle that an updated standby may be replaying WAL from a not-yet upgraded primary. We have to recognize that situation and use "old style" truncation (i.e. looking at the SLRUs) during WAL replay. In contrast to before, this now happens in the startup process, when replaying a checkpoint record, instead of the checkpointer. Doing truncation in the restartpoint is incorrect, they can happen much later than the original checkpoint, thereby leading to wraparound. To avoid "multixact_redo: unknown op code 48" errors standbys would have to be upgraded before primaries. A later patch will bump the WAL page magic, and remove the legacy truncation codepaths. Legacy truncation support is just included to make a possible future backpatch easier. Discussion: 20150621192409.GA4797@alap3.anarazel.de Reviewed-By: Robert Haas, Alvaro Herrera, Thomas Munro Backpatch: 9.5 for now 26 September 2015, 17:04:25 UTC
c9645f7 Second try at fixing O(N^2) problem in foreign key references. This replaces ill-fated commit 5ddc72887a012f6a8b85707ef27d85c274faf53d, which was reverted because it broke active uses of FK cache entries. In this patch, we still do nothing more to invalidatable cache entries than mark them as needing revalidation, so we won't break active uses. To keep down the overhead of InvalidateConstraintCacheCallBack(), keep a list of just the currently-valid cache entries. (The entries are large enough that some added space for list links doesn't seem like a big problem.) This would still be O(N^2) when there are many valid entries, though, so when the list gets too long, just force the "sinval reset" behavior to remove everything from the list. I set the threshold at 1000 entries, somewhat arbitrarily. Possibly that could be fine-tuned later. Another item for future study is whether it's worth adding reference counting so that we could safely remove invalidated entries. As-is, problem cases are likely to end up with large and mostly invalid FK caches. Like the previous attempt, backpatch to 9.3. Jan Wieck and Tom Lane 25 September 2015, 17:16:30 UTC
5eb7024 Further fix for psql's code for locale-aware formatting of numeric output. (Third time's the charm, I hope.) Additional testing disclosed that this code could mangle already-localized output from the "money" datatype. We can't very easily skip applying it to "money" values, because the logic is tied to column right-justification and people expect "money" output to be right-justified. Short of decoupling that, we can fix it in what should be a safe enough way by testing to make sure the string doesn't contain any characters that would not be expected in plain numeric output. 25 September 2015, 16:20:45 UTC
da4af91 Further fix for psql's code for locale-aware formatting of numeric output. On closer inspection, those seemingly redundant atoi() calls were not so much inefficient as just plain wrong: the author of this code either had not read, or had not understood, the POSIX specification for localeconv(). The grouping field is *not* a textual digit string but separate integers encoded as chars. We'll follow the existing code as well as the backend's cash.c in only honoring the first group width, but let's at least honor it correctly. This doesn't actually result in any behavioral change in any of the locales I have installed on my Linux box, which may explain why nobody's complained; grouping width 3 is close enough to universal that it's barely worth considering other cases. Still, wrong is wrong, so back-patch. 25 September 2015, 04:00:43 UTC
f1ee153 Fix psql's code for locale-aware formatting of numeric output. This code did the wrong thing entirely for numbers with an exponent but no decimal point (e.g., '1e6'), as reported by Jeff Janes in bug #13636. More generally, it made lots of unverified assumptions about what the input string could possibly look like. Rearrange so that it only fools with leading digits that it's directly verified are there, and an immediately adjacent decimal point. While at it, get rid of some useless inefficiencies, like converting the grouping count string to integer over and over (and over). This has been broken for a long time, so back-patch to all supported branches. 25 September 2015, 03:01:04 UTC
45d256c Allow planner to use expression-index stats for function calls in WHERE. Previously, a function call appearing at the top level of WHERE had a hard-wired selectivity estimate of 0.3333333, a kludge conveniently dated in the source code itself to July 1992. The expectation at the time was that somebody would soon implement estimator support functions analogous to those for operators; but no such code has appeared, nor does it seem likely to in the near future. We do have an alternative solution though, at least for immutable functions on single relations: creating an expression index on the function call will allow ANALYZE to gather stats about the function's selectivity. But the code in clause_selectivity() failed to make use of such data even if it exists. Refactor so that that will happen. I chose to make it try this technique for any clause type for which clause_selectivity() doesn't have a special case, not just functions. To avoid adding unnecessary overhead in the common case where we don't learn anything new, make selfuncs.c provide an API that hooks directly to examine_variable() and then var_eq_const(), rather than the previous coding which laboriously constructed an OpExpr only so that it could be expensively deconstructed again. I preserved the behavior that the default estimate for a function call is 0.3333333. (For any other expression node type, it's 0.5, as before.) I had originally thought to make the default be 0.5 across the board, but changing a default estimate that's survived for twenty-three years seems like something not to do without a lot more testing than I care to put into it right now. Per a complaint from Jehan-Guillaume de Rorthais. Back-patch into 9.5, but not further, at least for the moment. 24 September 2015, 22:35:46 UTC
59d765b Improve handling of collations in contrib/postgres_fdw. If we have a local Var of say varchar type with default collation, and we apply a RelabelType to convert that to text with default collation, we don't want to consider that as creating an FDW_COLLATE_UNSAFE situation. It should be okay to compare that to a remote Var, so long as the remote Var determines the comparison collation. (When we actually ship such an expression to the remote side, the local Var would become a Param with default collation, meaning the remote Var would in fact control the comparison collation, because non-default implicit collation overrides default implicit collation in parse_collate.c.) To fix, be more precise about what FDW_COLLATE_NONE means: it applies either to a noncollatable data type or to a collatable type with default collation, if that collation can't be traced to a remote Var. (When it can, FDW_COLLATE_SAFE is appropriate.) We were essentially using that interpretation already at the Var/Const/Param level, but we weren't bubbling it up properly. An alternative fix would be to introduce a separate FDW_COLLATE_DEFAULT value to describe the second situation, but that would add more code without changing the actual behavior, so it didn't seem worthwhile. Also, since we're clarifying the rule to be that we care about whether operator/function input collations match, there seems no need to fail immediately upon seeing a Const/Param/non-foreign-Var with nondefault collation. We only have to reject if it appears in a collation-sensitive context (for example, "var IS NOT NULL" is perfectly safe from a collation standpoint, whatever collation the var has). So just set the state to UNSAFE rather than failing immediately. Per report from Jeevan Chalke. This essentially corrects some sloppy thinking in commit ed3ddf918b59545583a4b374566bc1148e75f593, so back-patch to 9.3 where that logic appeared. 24 September 2015, 16:47:30 UTC
eac3b33 Make pg_controldata report newest XID with valid commit timestamp Previously pg_controldata didn't report newestCommitTs and this was an oversight in commit 73c986a. Also this patch changes pg_resetxlog so that it uses the same sentences as pg_controldata does, regarding oldestCommitTs and newestCommitTs, for the sake of consistency. Back-patch to 9.5 where track_commit_timestamp was added. Euler Taveira 24 September 2015, 14:31:44 UTC
ef4fccd Lower *_freeze_max_age minimum values. The old minimum values are rather large, making it time consuming to test related behaviour. Additionally the current limits, especially for multixacts, can be problematic in space-constrained systems. 10000000 multixacts can contain a lot of members. Since there's no good reason for the current limits, lower them a good bit. Setting them to 0 would be a bad idea, triggering endless vacuums, so still retain a limit. While at it fix autovacuum_multixact_freeze_max_age to refer to multixact.c instead of varsup.c. Reviewed-By: Robert Haas Discussion: CA+TgmoYmQPHcrc3GSs7vwvrbTkbcGD9Gik=OztbDGGrovkkEzQ@mail.gmail.com Backpatch: back to 9.0 (in parts) 24 September 2015, 12:53:33 UTC
cfb2024 Make ANALYZE compute basic statistics even for types with no "=" operator. Previously, ANALYZE simply ignored columns of datatypes that have neither a btree nor hash opclass (which means they have no recognized equality operator). Without a notion of equality, we can't identify most-common values nor estimate the number of distinct values. But we can still count nulls and compute the average physical column width, and those stats might be of value. Moreover there are some tools out there that don't work so well if rows are missing from pg_statistic. So let's add suitable logic for this case. While this is arguably a bug fix, it also has the potential to change query plans, and the gain seems not worth taking a risk of that in stable branches. So back-patch into 9.5 but not further. Oleksandr Shulgin, rewritten a bit by me. 23 September 2015, 22:26:58 UTC
fe6d2ab Docs: fix typo in to_char() example. Per bug #13631 from KOIZUMI Satoru. 22 September 2015, 14:40:25 UTC
55728ea test_decoding: Protect against rare spurious test failures. A bunch of tests missed specifying that empty transactions shouldn't be displayed. That causes problems when e.g. autovacuum runs in an unfortunate moment. The tests in question only run for a very short time, making this quite unlikely. Reported-By: Buildfarm member axolotl Backpatch: 9.4, where logical decoding was introduced 22 September 2015, 13:46:57 UTC
62e503b Correct value of LW_SHARED_MASK. The previous wrong value lead to wrong LOCK_DEBUG output, never showing any shared lock holders. Reported-By: Alexander Korotkov Discussion: CAPpHfdsPmWqz9FB0AnxJrwp1=KLF0n=-iB+QvR0Q8GSmpFVdUQ@mail.gmail.com Backpatch: 9.5, where the bug was introduced. 22 September 2015, 09:12:28 UTC
265728e doc: Tweak synopsis indentation for consistency 22 September 2015, 03:32:19 UTC
a8bb248 Fix whitespace 21 September 2015, 17:41:04 UTC
3dfffac Fix possible internal overflow in numeric multiplication. mul_var() postpones propagating carries until it risks overflow in its internal digit array. However, the logic failed to account for the possibility of overflow in the carry propagation step, allowing wrong results to be generated in corner cases. We must slightly reduce the when-to-propagate-carries threshold to avoid that. Discovered and fixed by Dean Rasheed, with small adjustments by me. This has been wrong since commit d72f6c75038d8d37e64a29a04b911f728044d83b, so back-patch to all supported branches. 21 September 2015, 16:11:45 UTC
bbdb9df Remove the SECURITY_ROW_LEVEL_DISABLED security context bit. This commit's parent made superfluous the bit's sole usage. Referential integrity checks have long run as the subject table's owner, and that now implies RLS bypass. Safe use of the bit was tricky, requiring strict control over the SQL expressions evaluating therein. Back-patch to 9.5, where the bit was introduced. Based on a patch by Stephen Frost. 21 September 2015, 00:47:36 UTC
6dae6ed Remove the row_security=force GUC value. Every query of a single ENABLE ROW SECURITY table has two meanings, with the row_security GUC selecting between them. With row_security=force available, every function author would have been advised to either set the GUC locally or test both meanings. Non-compliance would have threatened reliability and, for SECURITY DEFINER functions, security. Authors already face an obligation to account for search_path, and we should not mimic that example. With this change, only BYPASSRLS roles need exercise the aforementioned care. Back-patch to 9.5, where the row_security GUC was introduced. Since this narrows the domain of pg_db_role_setting.setconfig and pg_proc.proconfig, one might bump catversion. A row_security=force setting in one of those columns will elicit a clear message, so don't. 21 September 2015, 00:45:54 UTC
1be9d65 Restrict file mode creation mask during tmpfile(). Per Coverity. Back-patch to 9.0 (all supported versions). Michael Paquier, reviewed (in earlier versions) by Heikki Linnakangas. 21 September 2015, 00:42:44 UTC
3d3bc29 Be more wary about partially-valid LOCALLOCK data in RemoveLocalLock(). RemoveLocalLock() must consider the possibility that LockAcquireExtended() failed to palloc the initial space for a locallock's lockOwners array. I had evidently meant to cope with this hazard when the code was originally written (commit 1785acebf2ed14fd66955e2d9a55d77a025f418d), but missed that the pfree needed to be protected with an if-test. Just to make sure things are left in a clean state, reset numLockOwners as well. Per low-memory testing by Andreas Seltenreich. Back-patch to all supported branches. 20 September 2015, 20:48:44 UTC
b46f5eb Let compiler handle size calculation of bool types. Back in the day this did not work, but modern compilers should handle it themselves. 19 September 2015, 09:13:00 UTC
866a034 Simplify GETTEXT_FILES list 19 September 2015, 02:40:41 UTC
bd313ba Add missing serial comma 19 September 2015, 02:40:10 UTC
1e1ae6e Cache argument type information in json(b) aggregate functions. These functions have been looking up type info for every row they process. Instead of doing that we only look them up the first time through and stash the information in the aggregate state object. Affects json_agg, json_object_agg, jsonb_agg and jsonb_object_agg. There is plenty more work to do in making these more efficient, especially the jsonb functions, but this is a virtually cost free improvement that can be done right away. Backpatch to 9.5 where the jsonb variants were introduced. 18 September 2015, 18:45:28 UTC
a39331f Fix low-probability memory leak in regex execution. After an internal failure in shortest() or longest() while pinning down the exact location of a match, find() forgot to free the DFA structure before returning. This is pretty unlikely to occur, since we just successfully ran the "search" variant of the DFA; but it could happen, and it would result in a session-lifespan memory leak since this code uses malloc() directly. Problem seems to have been aboriginal in Spencer's library, so back-patch all the way. In passing, correct a thinko in a comment I added awhile back about the meaning of the "ntree" field. I happened across these issues while comparing our code to Tcl's version of the library. 18 September 2015, 17:55:17 UTC
e8e2999 Order some new options on man pages more sensibly, minor improvements 18 September 2015, 00:56:58 UTC
104184d Honour TEMP_CONFIG when testing pg_upgrade This setting contains extra configuration for the temp instance, as used in pg_regress' --temp-config flag. Backpatch to 9.2 where test.sh was introduced. 17 September 2015, 16:00:41 UTC
d97bdb0 Fix documentation of regular expression character-entry escapes. The docs claimed that \uhhhh would be interpreted as a Unicode value regardless of the database encoding, but it's never been implemented that way: \uhhhh and \xhhhh actually mean exactly the same thing, namely the character that pg_mb2wchar translates to 0xhhhh. Moreover we were falsely dismissive of the usefulness of Unicode code points above FFFF. Fix that. It's been like this for ages, so back-patch to all supported branches. 16 September 2015, 18:50:23 UTC
3047a9b Don't use "#" as an abbreviation for "number" in PL/Tcl error messages. Also, rewrite one error message to make it follow our message style guidelines better. Euler Taveira and Tom Lane 16 September 2015, 16:09:05 UTC
5b7aef8 Remove no-longer-used T_PrivGrantee node tag. Oversight in commit 31eae6028eca4365e7165f5f33fee1ed0486aee0, which replaced PrivGrantee nodes with RoleSpec nodes. Spotted by Yugo Nagata. 16 September 2015, 14:48:21 UTC
ea00ff5 Review program help output for wording and formatting 16 September 2015, 04:37:39 UTC
68b5201 Enforce ALL/SELECT policies in RETURNING for RLS For the UPDATE/DELETE RETURNING case, filter the records which are not visible to the user through ALL or SELECT policies from those considered for the UPDATE or DELETE. This is similar to how the GRANT system works, which prevents RETURNING unless the caller has SELECT rights on the relation. Per discussion with Robert, Dean, Tom, and Kevin. Back-patch to 9.5 where RLS was introduced. 15 September 2015, 19:49:40 UTC
23a4b89 RLS refactoring This refactors rewrite/rowsecurity.c to simplify the handling of the default deny case (reducing the number of places where we check for and add the default deny policy from three to one) by splitting up the retrival of the policies from the application of them. This also allowed us to do away with the policy_id field. A policy_name field was added for WithCheckOption policies and is used in error reporting, when available. Patch by Dean Rasheed, with various mostly cosmetic changes by me. Back-patch to 9.5 where RLS was introduced to avoid unnecessary differences, since we're still in alpha, per discussion with Robert. 15 September 2015, 19:49:40 UTC
ed301d6 Revert "Fix an O(N^2) problem in foreign key references". Commit 5ddc72887a012f6a8b85707ef27d85c274faf53d does not actually work because it will happily blow away ri_constraint_cache entries that are in active use in outer call levels. In any case, it's a very ugly, brute-force solution to the problem of limiting the cache size. Revert until it can be redesigned. 15 September 2015, 15:09:16 UTC
225f539 Add POLICY to COMMENT documentation COMMENT supports POLICY but the documentation hadn't caught up with that fact. Patch by Charles Clavadetscher Back-patch to 9.5 where POLICY was added. 15 September 2015, 14:56:40 UTC
fb98859 Improve log messages related to tablespace_map file This patch changes the log message which is logged when the server successfully renames backup_label file to *.old but fails to rename tablespace_map file during the shutdown. Previously the WARNING message "online backup mode was not canceled" was logged in that case. However this message is confusing because the backup mode is treated as canceled whenever backup_label is successfully renamed. So this commit makes the server log the message "online backup mode canceled" in that case. Also this commit changes errdetail messages so that they follow the error message style guide. Back-patch to 9.5 where tablespace_map file is introduced. Original patch by Amit Kapila, heavily modified by me. 15 September 2015, 14:23:35 UTC
f243072 Fix the fastpath rule for jsonb_concat with an empty operand. To prevent perverse results, we now only return the other operand if it's not scalar, and if both operands are of the same kind (array or object). Original bug complaint and patch from Oskari Saarenmaa, extended by me to cover the cases of different kinds of jsonb. Backpatch to 9.5 where jsonb_concat was introduced. 13 September 2015, 21:14:46 UTC
63c0f5b doc: Remove dead links The web pages of Andy Dong at Berkeley don't exist anymore, and he is no longer there. 13 September 2015, 03:50:54 UTC
back to top