Skip to main content

Browse the archive

https://github.com/postgres/postgres
07 April 2024, 13:56:23 UTC
sort by:
Revision Author Date Message Commit Date
45b8826 Tom Lane 10 February 2020, 22:14:51 UTC Stamp 12.2. 10 February 2020, 22:14:51 UTC
ce5a2d2 Tom Lane 10 February 2020, 17:51:07 UTC Last-minute updates for release notes. Security: CVE-2020-1720 10 February 2020, 17:51:07 UTC
87d014d Alvaro Herrera 10 February 2020, 15:14:58 UTC createuser: fix parsing of --connection-limit argument The original coding failed to quote the argument properly. Reported-by: Daniel Gustafsson Discussion: 1B8AE66C-85AB-4728-9BB4-612E8E61C219@yesql.se 10 February 2020, 15:14:58 UTC
2ad1253 Alvaro Herrera 10 February 2020, 14:47:09 UTC Fix priv checks for ALTER <object> DEPENDS ON EXTENSION Marking an object as dependant on an extension did not have any privilege check whatsoever; this allowed any user to mark objects as droppable by anyone able to DROP EXTENSION, which could be used to cause system-wide havoc. Disallow by checking that the calling user owns the mentioned object. (No constraints are placed on the extension.) Security: CVE-2020-1720 Reported-by: Tom Lane Discussion: 31605.1566429043@sss.pgh.pa.us 10 February 2020, 14:47:09 UTC
3264316 Peter Eisentraut 10 February 2020, 12:15:42 UTC Translation updates Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: bcdfb83b81a7aa3c3948c0a5221f9c68d7010ac5 10 February 2020, 12:15:42 UTC
198f0ba Michael Paquier 10 February 2020, 06:48:35 UTC Revert "pg_upgrade: Fix quoting of some arguments in pg_ctl command" This reverts commit d1c0b61. The patch has some downsides that require more attention, as discussed with Noah Misch. Backpatch-through: 9.5 10 February 2020, 06:48:35 UTC
a6a8616 Amit Kapila 10 February 2020, 03:04:43 UTC doc: Spell checking Reported-by: Justin Pryzby Author: Justin Pryzby Backpatch-through: 9.6 Discussion: https://postgr.es/m/20200206021432.GA24549@telsasoft.com 10 February 2020, 03:13:52 UTC
770eca3 Michael Paquier 10 February 2020, 01:49:38 UTC pg_upgrade: Fix quoting of some arguments in pg_ctl command The previous coding forgot to apply shell quoting to the socket directory and the data folder, leading to failures when running pg_upgrade. This refactors the code generating the pg_ctl command starting clusters to use a more correct shell quoting. Failures are easier to trigger in 12 and newer versions by using a value of --socketdir that includes quotes, but it is also possible to cause failures with quotes included in the default socket directory used by pg_upgrade or the data folders of the clusters involved in the upgrade. As 9.4 is going to be EOL'd with the next minor release, nobody is likely going to upgrade to it now so this branch is not included in the set of branches fixed. Author: Michael Paquier Reviewed-by: Álvaro Herrera, Noah Misch Backpatch-through: 9.5 10 February 2020, 01:49:38 UTC
2db27ff Tom Lane 09 February 2020, 19:20:26 UTC Revert "docs: change "default role" wording to "predefined role"" This reverts commit 29af9c542f6cdc9a6456b175fdcdfe7a28b77d6c. Per discussion, we can't change the section title without some web-site work, so revert this change temporarily. Discussion: https://postgr.es/m/157742545062.1149.11052653770497832538@wrigleys.postgresql.org 09 February 2020, 19:20:26 UTC
0522d6f Tom Lane 09 February 2020, 19:14:18 UTC Release notes for 12.2, 11.7, 10.12, 9.6.17, 9.5.21, 9.4.26. 09 February 2020, 19:14:18 UTC
baf4871 Tom Lane 09 February 2020, 17:02:57 UTC Store the deletion horizon XID for a deleted GIN page on the right page. Commit b10714080 moved the GinPageSetDeleteXid() call to a spot where the "page" variable was pointing to the wrong page, causing the XID to be inserted on a page that's not being deleted, thus allowing later GinPageIsRecyclable tests to recycle the deleted page too soon. It might be a good idea to stop using the single "page" variable for multiple purposes in this function. But for the moment I just moved the GinPageSetDeleteXid() call down beside the GinPageSetDeleted() call, which seems like a more logical place for it anyway. Back-patch to v11, as the faulty patch was. (Fortunately, the bug hasn't made it into any release yet.) Discussion: https://postgr.es/m/21620.1581098806@sss.pgh.pa.us 09 February 2020, 17:02:57 UTC
4faea7f Fujii Masao 08 February 2020, 03:29:38 UTC Revert recent change of ScanOption values and renumber SO_TYPE_TIDSCAN. Commit 598b466e80 in v12 renumbered ScanOption enum values to add the option for Tid scan. But the change of the codes assigned to the existing values caused an ABI break and may break some extensions depending on them. This should be avoided in minor version. This commit reverts the renumbering of ScanOption enum values made by commit 598b466e80 in v12 and put the ScanOption for Tid scan with new value. This is applied only to v12. Per complaint from Tom Lane. Discussion: https://postgr.es/m/5261.1581103527@sss.pgh.pa.us 08 February 2020, 03:29:38 UTC
388d435 Tom Lane 07 February 2020, 21:52:55 UTC First-draft release notes for 12.2. As usual, the release notes for other branches will be made by cutting these down, but put them up for community review first. 07 February 2020, 21:53:23 UTC
2c80a65 Alvaro Herrera 07 February 2020, 21:27:18 UTC Fix failure to create FKs correctly in partitions On a multi-level partioned table, when adding a partition not directly connected to the root table, foreign key constraints referencing the root were not cloned to the new partition, leading to the FK being possibly inadvertently violated later on. This was caused by fuzzy thinking in CloneFkReferenced (commit f56f8f8da6af): it was skipping constraints marked as having parents on the theory that cloning those would create duplicates; but that's only correct for the top level of the partitioning hierarchy. For levels below that one, such constraints must still be considered and only skipped if later on we see that we'd create duplicates. Apparently, I (Álvaro) wrote the comments right but the code implemented something slightly different. Author: Jehan-Guillaume de Rorthais Discussion: https://postgr.es/m/20200206004948.238352db@firost 07 February 2020, 21:27:18 UTC
ce054a8 Alvaro Herrera 07 February 2020, 20:09:36 UTC Fix TRUNCATE .. CASCADE on partitions When running TRUNCATE CASCADE on a child of a partitioned table referenced by another partitioned table, the truncate was not applied to partitions of the referencing table; this could leave rows violating the constraint in the referencing partitioned table. Repair by walking the pg_constraint chain all the way up to the topmost referencing table. Note: any partitioned tables containing FKs that reference other partitioned tables should be checked for possible violating rows, if TRUNCATE has occurred in partitions of the referenced table. Reported-by: Christophe Courtois Author: Jehan-Guillaume de Rorthais Discussion: https://postgr.es/m/20200204183906.115f693e@firost 07 February 2020, 20:09:36 UTC
598b466 Fujii Masao 07 February 2020, 13:00:21 UTC Fix bug in Tid scan. Commit 147e3722f7 changed Tid scan so that it calls table_beginscan() and uses the scan option for seq scan. This change caused two issues. (1) The change caused Tid scan to take a predicate lock on the entire relation in serializable transaction even when relation-level lock is not necessary. This could lead to an unexpected serialization error. (2) The change caused Tid scan to increment the number of seq_scan in pg_stat_*_tables views even though it's not seq scan. This could confuse the users. This commit adds the scan option for Tid scan and makes Tid scan use it, to avoid those issues. Back-patch to v12, where the bug was introduced. Author: Tatsuhito Kasahara Reviewed-by: Kyotaro Horiguchi, Masahiko Sawada, Fujii Masao Discussion: https://postgr.es/m/CAP0=ZVKy+gTbFmB6X_UW0pP3WaeJ-fkUWHoD-pExS=at3CY76g@mail.gmail.com 07 February 2020, 13:07:44 UTC
2f47339 Michael Paquier 06 February 2020, 23:11:01 UTC Revert "Add GUC checks for ssl_min_protocol_version and ssl_max_protocol_version" This reverts commit 41aadee, as the GUC checks could run on older values with the new values used, and result in incorrect errors if both parameters are changed at the same time. Per complaint from Tom Lane. Discussion: https://postgr.es/m/27574.1581015893@sss.pgh.pa.us Backpatch-through: 12 06 February 2020, 23:11:01 UTC
4988d7e Fujii Masao 06 February 2020, 15:33:11 UTC Add note about access permission checks by inherited TRUNCATE and LOCK TABLE. Inherited queries perform access permission checks on the parent table only. But there are two exceptions to this rule in v12 or before; TRUNCATE and LOCK TABLE commands through a parent table check the permissions on not only the parent table but also the children tables. Previously these exceptions were not documented. This commit adds the note about these exceptions, into the document. Back-patch to v9.4. But we don't apply this commit to the master because commit e6f1e560e4 already got rid of the exception about inherited TRUNCATE and upcoming commit will do for the exception about inherited LOCK TABLE. Author: Amit Langote Reviewed-by: Fujii Masao Discussion: https://postgr.es/m/CA+HiwqHfTnMU6SUkyHxCmpHUKk7ERLHCR3vZVq19ZOQBjPBLmQ@mail.gmail.com 06 February 2020, 15:34:20 UTC
c5060c1 Amit Kapila 06 February 2020, 10:30:54 UTC Fix typo. Reported-by: Amit Langote Author: Amit Langote Backpatch-through: 9.6, where it was introduced Discussion: https://postgr.es/m/CA+HiwqFNADeukaaGRmTqANbed9Fd81gLi08AWe_F86_942Gspw@mail.gmail.com 06 February 2020, 10:30:54 UTC
3d214a8 Fujii Masao 06 February 2020, 05:43:21 UTC Fix bug in LWLock statistics mechanism. Previously PostgreSQL built with -DLWLOCK_STATS could report more than one LWLock statistics entries for the same backend process and the same LWLock. This is strange and only one statistics should be output in that case, instead. The cause of this issue is that the key variable used for LWLock stats hash table was not fully initialized. The key consists of two fields and they were initialized. But the following 4 bytes allocated in the key variable for the alignment was not initialized. So even if the same key was specified, hash_search(HASH_ENTER) could not find the existing entry for that key and created new one. This commit fixes this issue by initializing the key variable with zero. As the side effect of this commit, the volume of LWLock statistics output would be reduced very much. Back-patch to v10, where commit 3761fe3c20 introduced the issue. Author: Fujii Masao Reviewed-by: Julien Rouhaud, Kyotaro Horiguchi Discussion: https://postgr.es/m/26359edb-798a-568f-d93a-6aafac49752d@oss.nttdata.com 06 February 2020, 05:44:22 UTC
0e37489 Andrew Gierth 05 February 2020, 19:49:47 UTC Force tuple conversion when the source has missing attributes. Tuple conversion incorrectly concluded that no conversion was needed as long as all the attributes lined up. But if the source tuple has a missing attribute (from addition of a column with default), then the destination tupdesc might not reflect the same default. The typical symptom was that the affected columns would be unexpectedly NULL. Repair by always forcing conversion if the source has missing attributes, which will be filled in by the deform operation. (In theory we could optimize for when the destination has the same default, but that seemed overkill.) Backpatch to 11 where missing attributes were added. Per bug #16242. Vik Fearing (discovery, code, testing) and me (analysis, testcase). Discussion: https://postgr.es/m/16242-d1c9fca28445966b@postgresql.org 05 February 2020, 20:08:11 UTC
308724b Alvaro Herrera 05 February 2020, 18:06:11 UTC ALTER SUBSCRIPTION / REFRESH docs: explain copy_data The docs are ambiguous as to which tables would be copied over when the copy_data parameter is true in ALTER SUBSCRIPTION ... REFRESH PUBLICATION. Make it clear that it only applies to tables which are new in the publication. Author: David Christensen (reword by Álvaro Herrera) Discussion: https://postgr.es/m/95339420-7F09-4F8C-ACC0-8F1CFAAD9CD7@endpoint.com 05 February 2020, 18:06:11 UTC
78a26c3 Noah Misch 05 February 2020, 16:26:41 UTC When a TAP file has non-zero exit status, retain temporary directories. PostgresNode already retained base directories in such cases. Stop using $SIG{__DIE__}, which is redundant with the exit status check, in lieu of proliferating it to TestLib. Back-patch to 9.6, where commit 88802e068017bee8cea7a5502a712794e761c7b5 introduced retention on failure. Reviewed by Daniel Gustafsson. Discussion: https://postgr.es/m/20200202170155.GA3264196@rfd.leadboat.com 05 February 2020, 16:26:44 UTC
1fb8976 Fujii Masao 26 December 2019, 06:07:43 UTC Add note about how each partition's default value is treated, into the doc. Column defaults may be specified separately for each partition. But INSERT via a partitioned table ignores those partition's default values. The former is documented, but the latter restriction not. This commit adds the note about that restriction into the document. Back-patch to v10 where partitioning was introduced. Author: Fujii Masao Reviewed-by: Amit Langote Discussion: https://postgr.es/m/CAHGQGwEs-59omrfGF7hOHz9iMME3RbKy5ny+iftDx3LHTEn9sA@mail.gmail.com 05 February 2020, 04:55:01 UTC
2e2351b Thomas Munro 04 February 2020, 23:21:03 UTC Handle lack of DSM slots in parallel btree build, take 2. Commit 74618e77 added a new check intended to fix a bug, but put it in the wrong place so that parallel btree build was always disabled. Do the check after we've actually tried to create a DSM segment. Back-patch to 11, like the earlier commit. Reviewed-by: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmDABkJzrNnvf%2BOULK-_A_j9gkYg_Dz-H62jzNv4eKQTw%40mail.gmail.com 04 February 2020, 23:28:33 UTC
9a85860 Tom Lane 04 February 2020, 18:07:13 UTC Fix handling of "Subplans Removed" field in EXPLAIN output. Commit 499be013d added this field in a rather poorly-thought-through manner, with the result being that rather than being a field of the Append or MergeAppend plan node as intended (and as it seems to be, in text format), it was actually an element of the "Plans" subgroup. At least in JSON format, that's flat out invalid syntax, because "Plans" is an array not an object. While it's not hard to move the generation of the field so that it appears where it's supposed to, this does result in a visible change in field order in text format, in cases where a Append or MergeAppend plan node has any InitPlans attached. That's slightly annoying to do in stable branches; but the alternative of continuing to emit broken non-text formats seems worse. Also, since the set of fields emitted is not supposed to be data-dependent in non-text formats, make sure that "Subplans Removed" appears in Append and MergeAppend nodes even when it's zero, in those formats. (The previous coding made it look like it could appear in some other node types such as BitmapAnd, but we don't actually support runtime pruning there, so don't emit it in those cases.) Per bug #16171 from Mahadevan Ramachandran. Fix by Daniel Gustafsson and Tom Lane, reviewed by Hamid Akhtar. Back-patch to v11 where this code came in. Discussion: https://postgr.es/m/16171-b72259ab75505fa2@postgresql.org 04 February 2020, 18:07:13 UTC
42e3187 Alvaro Herrera 03 February 2020, 21:59:12 UTC Add missing break out seqscan loop in logical replication When replica identity is FULL (an admittedly unusual case), the loop that searches for tuples in execReplication.c didn't stop scanning the table when once a matching tuple was found. Add the missing 'break'. Note slight behavior change: we now return the first matching tuple rather than the last one. They are supposed to be indistinguishable anyway, so this shouldn't matter. Author: Konstantin Knizhnik Discussion: https://postgr.es/m/379743f6-ae91-b866-f7a2-5624e6d2b0a4@postgrespro.ru 03 February 2020, 21:59:12 UTC
0d9f307 Fujii Masao 03 February 2020, 03:37:59 UTC Revert commit de0177788b. This commit reverts the fix "Make inherited TRUNCATE perform access permission checks on parent table only" only in the back branches. It's not hard to imagine that there are some applications expecting the old behavior and the fix breaks their security. To avoid this compatibility problem, we decided to apply the fix only in HEAD and revert it in all supported back branches. Discussion: https://postgr.es/m/21015.1580400165@sss.pgh.pa.us 03 February 2020, 03:37:59 UTC
24dd34a Thomas Munro 01 February 2020, 01:29:13 UTC Fix memory leak on DSM slot exhaustion. If we attempt to create a DSM segment when no slots are available, we should return the memory to the operating system. Previously we did that if the DSM_CREATE_NULL_IF_MAXSEGMENTS flag was passed in, but we didn't do it if an error was raised. Repair. Back-patch to 9.4, where DSM segments arrived. Author: Thomas Munro Reviewed-by: Robert Haas Reported-by: Julian Backes Discussion: https://postgr.es/m/CA%2BhUKGKAAoEw-R4om0d2YM4eqT1eGEi6%3DQot-3ceDR-SLiWVDw%40mail.gmail.com 01 February 2020, 01:49:57 UTC
65aa155 Tom Lane 31 January 2020, 22:03:55 UTC Fix CheckAttributeType's handling of collations for ranges. Commit fc7695891 changed CheckAttributeType to recurse into ranges, but made it pass down the wrong collation (always InvalidOid, since ranges as such have no collation). This would result in guaranteed failure when considering a range type whose subtype is collatable. Embarrassingly, we lack any regression tests that would expose such a problem (but fortunately, somebody noticed before we shipped this bug in any release). Fix it to pass down the range's subtype collation property instead, and add some regression test cases to exercise collatable-subtype ranges a bit more. Back-patch to all supported branches, as the previous patch was. Report and patch by Julien Rouhaud, test cases tweaked by me Discussion: https://postgr.es/m/CAOBaU_aBWqNweiGUFX0guzBKkcfJ8mnnyyGC_KBQmO12Mj5f_A@mail.gmail.com 31 January 2020, 22:03:55 UTC
0c84199 Tom Lane 31 January 2020, 19:41:49 UTC Fix parallel pg_dump/pg_restore for failure to create worker processes. If we failed to fork a worker process, or create a communication pipe for one, WaitForTerminatingWorkers would suffer an assertion failure if assert-enabled, otherwise crash or go into an infinite loop. This was a consequence of not accounting for the startup condition where we've not yet forked all the workers. The original bug was that ParallelBackupStart would set workerStatus to WRKR_IDLE before it had successfully forked a worker. I made things worse in commit b7b8cc0cf by not understanding the undocumented fact that the WRKR_TERMINATED state was also meant to represent the case where a worker hadn't been started yet: I changed enum T_WorkerStatus so that *all* the worker slots were initially in WRKR_IDLE state. But this wasn't any more broken in practice, since even one slot in the wrong state would keep WaitForTerminatingWorkers from terminating. In v10 and later, introduce an explicit T_WorkerStatus value for worker-not-started, in hopes of preventing future oversights of the same ilk. Before that, just document that WRKR_TERMINATED is supposed to cover that case (partly because it wasn't actively broken, and partly because the enum is exposed outside parallel.c in those branches, so there's microscopically more risk involved in changing it). In all branches, introduce a WORKER_IS_RUNNING status test macro to hide which T_WorkerStatus values mean that, and be more careful not to access ParallelSlot fields till we're sure they're valid. Per report from Vignesh C, though this is my patch not his. Back-patch to all supported branches. Discussion: https://postgr.es/m/CALDaNm1Luv-E3sarR+-unz-BjchquHHyfP+YC+2FS2pt_J+wxg@mail.gmail.com 31 January 2020, 19:41:49 UTC
706ad6a Michael Paquier 31 January 2020, 04:58:05 UTC Fix typo in recently-added TAP test for replication slots Oversight in commit b0afdca. 31 January 2020, 04:58:05 UTC
a4484a6 Tom Lane 30 January 2020, 23:25:55 UTC In jsonb_plpython.c, suppress warning message from gcc 10. Very recent gcc complains that PLyObject_ToJsonbValue could return a pointer to a local variable. I think it's wrong; but the coding is fragile enough, and the savings of one palloc() minimal enough, that it seems better to just do a palloc() all the time. (My other idea of tweaking the if-condition doesn't suppress the warning.) Back-patch to v11 where this code was introduced. Discussion: https://postgr.es/m/21547.1580170366@sss.pgh.pa.us 30 January 2020, 23:26:13 UTC
1fcf62e Thomas Munro 30 January 2020, 21:25:34 UTC Handle lack of DSM slots in parallel btree build. If no DSM slots are available, a ParallelContext can still be created, but its seg pointer is NULL. Teach parallel btree build to cope with that by falling back to a regular non-parallel build, to avoid crashing with a segmentation fault. Back-patch to 11, where parallel CREATE INDEX landed. Reported-by: Nicola Contu Reviewed-by: Peter Geoghegan Discussion: https://postgr.es/m/CA%2BhUKGJgJEBnkuODBVomyK3MWFvDBbMVj%3Dgdt6DnRPU-5sQ6UQ%40mail.gmail.com 30 January 2020, 21:27:53 UTC
de01777 Fujii Masao 30 January 2020, 15:42:06 UTC Make inherited TRUNCATE perform access permission checks on parent table only. Previously, TRUNCATE command through a parent table checked the permissions on not only the parent table but also the children tables inherited from it. This was a bug and inherited queries should perform access permission checks on the parent table only. This commit fixes that bug. Back-patch to all supported branches. Author: Amit Langote Reviewed-by: Fujii Masao Discussion: https://postgr.es/m/CAHGQGwFHdSvifhJE+-GSNqUHSfbiKxaeQQ7HGcYz6SC2n_oDcg@mail.gmail.com 30 January 2020, 15:43:28 UTC
3228512 Michael Paquier 30 January 2020, 02:15:28 UTC Fix slot data persistency when advancing physical replication slots Advancing a physical replication slot with pg_replication_slot_advance() did not mark the slot as dirty if any advancing was done, preventing the follow-up checkpoint to flush the slot data to disk. This caused the advancing to be lost even on clean restarts. This does not happen for logical slots as any advancing marked the slot as dirty. Per discussion, the original feature has been implemented so as in the event of a crash the slot may move backwards to a past LSN. This property is kept and more documentation is added about that. This commit adds some new TAP tests to check the persistency of physical and logical slots after advancing across clean restarts. Author: Alexey Kondratov, Michael Paquier Reviewed-by: Andres Freund, Kyotaro Horiguchi, Craig Ringer Discussion: https://postgr.es/m/059cc53a-8b14-653a-a24d-5f867503b0ee@postgrespro.ru Backpatch-through: 11 30 January 2020, 02:15:28 UTC
b558f6d Michael Paquier 29 January 2020, 06:08:26 UTC Fix dispsize for libpq connection parameters channel_binding and gssencmode channel_binding's longest allowed value is not "7", it is actually "8". gssencmode also got that wrong. A similar mistake has been fixed as of f4051e3. Backpatch down to v12, where gssencmode has been introduced. Reviewed-by: Daniel Gustafsson Discussion: https://postgr.es/m/20200128053633.GD1552@paquier.xyz Backpatch-through: 12 29 January 2020, 06:08:26 UTC
87fed2a Tom Lane 28 January 2020, 22:26:37 UTC Fix dangling pointer in EvalPlanQual machinery. EvalPlanQualStart() supposed that it could re-use the relsubs_rowmark and relsubs_done arrays from a prior instantiation. But since they are allocated in the es_query_cxt of the recheckestate, that's just wrong; EvalPlanQualEnd() will blow away that storage. Therefore we were using storage that could have been reallocated to something else, causing all sorts of havoc. I think this was modeled on the old code's handling of es_epqTupleSlot, but since the code was anyway clearing the arrays at re-use, there's clearly no expectation of importing any outside state. So it's just a dubious savings of a couple of pallocs, which is negligible compared to setting up a new planstate tree. Therefore, just allocate the arrays always. (I moved the allocations slightly for readability.) In principle this bug could cause a problem whenever EPQ rechecks are needed in more than one target table of a ModifyTable plan node. In practice it seems not quite so easy to trigger as that; I couldn't readily duplicate a crash with a partitioned target table, for instance. That's probably down to incidental choices about when to free or reallocate stuff. The added isolation test case does seem to reliably show an assertion failure, though. Per report from Oleksii Kliukin. Back-patch to v12 where the bug was introduced (evidently by commit 3fb307bc4). Discussion: https://postgr.es/m/EEF05F66-2871-4786-992B-5F45C92FEE2E@hintbits.com 28 January 2020, 22:26:37 UTC
f9d0be2 Thomas Munro 26 January 2020, 23:52:08 UTC Avoid unnecessary shm writes in Parallel Hash Join. Currently, Parallel Hash Join cannot be used for full/right joins, so there is no point in setting the match flag. It turns out that the cache coherence traffic generated by those writes slows down large systems running many-core joins, so let's stop doing that. In future, if we need to use match bits in parallel joins, we might want to consider setting them only if not already set. Back-patch to 11, where Parallel Hash Join arrived. Reported-by: Deng, Gang Discussion: https://postgr.es/m/0F44E799048C4849BAE4B91012DB910462E9897A%40SHSMSX103.ccr.corp.intel.com 27 January 2020, 01:11:24 UTC
bad4943 Tom Lane 26 January 2020, 21:31:48 UTC Fix EXPLAIN (SETTINGS) to follow policy about when to print empty fields. In non-TEXT output formats, the "Settings" field should appear when requested, even if it would be empty. Also, get rid of the premature optimization of counting all the GUC_EXPLAIN variables at startup. Since there was no provision for adjusting that count later, all it'd take would be some extension marking a parameter as GUC_EXPLAIN to risk an assertion failure or memory stomp. We could make get_explain_guc_options() count those variables on-the-fly, or dynamically resize its array ... but TBH I do not think that making a transient array of pointers a bit smaller is worth any extra complication, especially when you consider all the other transient space EXPLAIN eats. So just allocate that array at the max possible size. In HEAD, also add some regression test coverage for this feature. Because of the memory-stomp hazard, back-patch to v12 where this feature was added. Discussion: https://postgr.es/m/19416.1580069629@sss.pgh.pa.us 26 January 2020, 21:31:48 UTC
7294f99 Tom Lane 26 January 2020, 19:31:08 UTC In postgres_fdw, don't try to ship MULTIEXPR updates to remote server. In a statement like "UPDATE remote_tab SET (x,y) = (SELECT ...)", we'd conclude that the statement could be directly executed remotely, because the sub-SELECT is in a resjunk tlist item that's not examined for shippability. Currently that ends up crashing if the sub-SELECT contains any remote Vars. Prevent the crash by deeming MULTIEXEC Params to be unshippable. This is a bit of a brute-force solution, since if the sub-SELECT *doesn't* contain any remote Vars, the current execution technology would work; but that's not a terribly common use-case for this syntax, I think. In any case, we generally don't try to ship sub-SELECTs, so it won't surprise anybody that this doesn't end up as a remote direct update. I'd be inclined to see if that general limitation can be fixed before worrying about this case further. Per report from Lukáš Sobotka. Back-patch to 9.6. 9.5 had MULTIEXPR, but we didn't try to perform remote direct updates then, so the case didn't arise anyway. Discussion: https://postgr.es/m/CAJif3k+iA_ekBB5Zw2hDBaE1wtiQa4LH4_JUXrrMGwTrH0J01Q@mail.gmail.com 26 January 2020, 19:31:08 UTC
c4c76d1 Michael Paquier 24 January 2020, 00:55:51 UTC Doc: Fix list of storage parameters available for ALTER TABLE Only the parameter parallel_workers can be used directly with ALTER TABLE. Issue introduced in 6f3a13f, so backpatch down to 10. Author: Justin Pryzby Discussion: https://postgr.es/m/20200106025623.GA12066@telsasoft.com Backpatch-through: 10 24 January 2020, 00:55:51 UTC
f309c81 Tom Lane 23 January 2020, 21:15:32 UTC Fix an oversight in commit 4c70098ff. I had supposed that the from_char_seq_search() call sites were all passing the constant arrays you'd expect them to pass ... but on looking closer, the one for DY format was passing the days[] array not days_short[]. This accidentally worked because the day abbreviations in English are all the same as the first three letters of the full day names. However, once we took out the "maximum comparison length" logic, it stopped working. As penance for that oversight, add regression test cases covering this, as well as every other switch case in DCH_from_char() that was not reached according to the code coverage report. Also, fold the DCH_RM and DCH_rm cases into one --- now that seq_search is case independent, there's no need to pass different comparison arrays for those cases. Back-patch, as the previous commit was. 23 January 2020, 21:15:32 UTC
be13f22 Tom Lane 23 January 2020, 18:42:10 UTC Clean up formatting.c's logic for matching constant strings. seq_search(), which is used to match input substrings to constants such as month and day names, had a lot of bizarre and unnecessary behaviors. It was mostly possible to avert our eyes from that before, but we don't want to duplicate those behaviors in the upcoming patch to allow recognition of non-English month and day names. So it's time to clean this up. In particular: * seq_search scribbled on the input string, which is a pretty dangerous thing to do, especially in the badly underdocumented way it was done here. Fortunately the input string is a temporary copy, but that was being made three subroutine levels away, making it something easy to break accidentally. The behavior is externally visible nonetheless, in the form of odd case-folding in error reports about unrecognized month/day names. The scribbling is evidently being done to save a few calls to pg_tolower, but that's such a cheap function (at least for ASCII data) that it's pretty pointless to worry about. In HEAD I switched it to be pg_ascii_tolower to ensure it is cheap in all cases; but there are corner cases in Turkish where this'd change behavior, so leave it as pg_tolower in the back branches. * seq_search insisted on knowing the case form (all-upper, all-lower, or initcap) of the constant strings, so that it didn't have to case-fold them to perform case-insensitive comparisons. This likewise seems like excessive micro-optimization, given that pg_tolower is certainly very cheap for ASCII data. It seems unsafe to assume that we know the case form that will come out of pg_locale.c for localized month/day names, so it's better just to define the comparison rule as "downcase all strings before comparing". (The choice between downcasing and upcasing is arbitrary so far as English is concerned, but it might not be in other locales, so follow citext's lead here.) * seq_search also had a parameter that'd cause it to report a match after a maximum number of characters, even if the constant string were longer than that. This was not actually used because no caller passed a value small enough to cut off a comparison. Replicating that behavior for localized month/day names seems expensive as well as useless, so let's get rid of that too. * from_char_seq_search used the maximum-length parameter to truncate the input string in error reports about not finding a matching name. This leads to rather confusing reports in many cases. Worse, it is outright dangerous if the input string isn't all-ASCII, because we risk truncating the string in the middle of a multibyte character. That'd lead either to delivering an illegible error message to the client, or to encoding-conversion failures that obscure the actual data problem. Get rid of that in favor of truncating at whitespace if any (a suggestion due to Alvaro Herrera). In addition to fixing these things, I const-ified the input string pointers of DCH_from_char and its subroutines, to make sure there aren't any other scribbling-on-input problems. The risk of generating a badly-encoded error message seems like enough of a bug to justify back-patching, so patch all supported branches. Discussion: https://postgr.es/m/29432.1579731087@sss.pgh.pa.us 23 January 2020, 18:42:10 UTC
817a1b8 Michael Paquier 22 January 2020, 00:49:24 UTC Fix concurrent indexing operations with temporary tables Attempting to use CREATE INDEX, DROP INDEX or REINDEX with CONCURRENTLY on a temporary relation with ON COMMIT actions triggered unexpected errors because those operations use multiple transactions internally to complete their work. Here is for example one confusing error when using ON COMMIT DELETE ROWS: ERROR: index "foo" already contains data Issues related to temporary relations and concurrent indexing are fixed in this commit by enforcing the non-concurrent path to be taken for temporary relations even if using CONCURRENTLY, transparently to the user. Using a non-concurrent path does not matter in practice as locks cannot be taken on a temporary relation by a session different than the one owning the relation, and the non-concurrent operation is more effective. The problem exists with REINDEX since v12 with the introduction of CONCURRENTLY, and with CREATE/DROP INDEX since CONCURRENTLY exists for those commands. In all supported versions, this caused only confusing error messages to be generated. Note that with REINDEX, it was also possible to issue a REINDEX CONCURRENTLY for a temporary relation owned by a different session, leading to a server crash. The idea to enforce transparently the non-concurrent code path for temporary relations comes originally from Andres Freund. Reported-by: Manuel Rigger Author: Michael Paquier, Heikki Linnakangas Reviewed-by: Andres Freund, Álvaro Herrera, Heikki Linnakangas Discussion: https://postgr.es/m/CA+u7OA6gP7YAeCguyseusYcc=uR8+ypjCcgDDCTzjQ+k6S9ksQ@mail.gmail.com Backpatch-through: 9.4 22 January 2020, 00:49:24 UTC
21fdfd0 Andres Freund 21 January 2020, 07:26:51 UTC Fix edge case leading to agg transitions skipping ExecAggTransReparent() calls. The code checking whether an aggregate transition value needs to be reparented into the current context has always only compared the transition return value with the previous transition value by datum, i.e. without regard for NULLness. This normally works, because when the transition function returns NULL (via fcinfo->isnull), it'll return a value that won't be the same as its input value. But there's no hard requirement that that's the case. And it turns out, it's possible to hit this case (see discussion or reproducers), leading to a non-null transition value not being reparented, followed by a crash caused by that. Instead of adding another comparison of NULLness, instead have ExecAggTransReparent() ensure that pergroup->transValue ends up as 0 when the new transition value is NULL. That avoids having to add an additional branch to the much more common cases of the transition function returning the old transition value (which is a pointer in this case), and when the new value is different, but not NULL. In branches since 69c3936a149, also deduplicate the reparenting code between the expression evaluation based transitions, and the path for ordered aggregates. Reported-By: Teodor Sigaev, Nikita Glukhov Author: Andres Freund Discussion: https://postgr.es/m/bd34e930-cfec-ea9b-3827-a8bc50891393@sigaev.ru Backpatch: 9.4-, this issue has existed since at least 7.4 21 January 2020, 07:29:35 UTC
ef8e6b2 Michael Paquier 21 January 2020, 04:46:55 UTC Add GUC variables for stat tracking and timeout as PGDLLIMPORT This helps integration of extensions with Windows. The following parameters are changed: - idle_in_transaction_session_timeout (9.6 and newer versions) - lock_timeout - statement_timeout - track_activities - track_counts - track_functions Author: Pascal Legrand Reviewed-by: Amit Kamila, Julien Rouhaud, Michael Paquier Discussion: https://postgr.es/m/1579298868581-0.post@n3.nabble.com Backpatch-through: 9.4 21 January 2020, 04:46:55 UTC
71b121f Tom Lane 20 January 2020, 17:57:17 UTC Fix pg_dump's sigTermHandler() to use _exit() not exit(). sigTermHandler() tried to be careful to invoke only operations that are safe to do in a signal handler. But for some reason we forgot that exit(3) is not among those, because it calls atexit handlers that might do various random things. (pg_dump itself installs no atexit handlers, but e.g. OpenSSL does.) That led to crashes or lockups when attempting to terminate a parallel dump or restore via a signal. Fix by calling _exit() instead. Per bug #16199 from Raúl Marín. Back-patch to all supported branches. Discussion: https://postgr.es/m/16199-cb2f121146a96f9b@postgresql.org 20 January 2020, 17:57:17 UTC
fd436bb Heikki Linnakangas 20 January 2020, 08:36:35 UTC Fix crash in BRIN inclusion op functions, due to missing datum copy. The BRIN add_value() and union() functions need to make a longer-lived copy of the argument, if they want to store it in the BrinValues struct also passed as argument. The functions for the "inclusion operator classes" used with box, range and inet types didn't take into account that the union helper function might return its argument as is, without making a copy. Check for that case, and make a copy if necessary. That case arises at least with the range_union() function, when one of the arguments is an 'empty' range: CREATE TABLE brintest (n numrange); CREATE INDEX brinidx ON brintest USING brin (n); INSERT INTO brintest VALUES ('empty'); INSERT INTO brintest VALUES (numrange(0, 2^1000::numeric)); INSERT INTO brintest VALUES ('(-1, 0)'); SELECT brin_desummarize_range('brinidx', 0); SELECT brin_summarize_range('brinidx', 0); Backpatch down to 9.5, where BRIN was introduced. Discussion: https://www.postgresql.org/message-id/e6e1d6eb-0a67-36aa-e779-bcca59167c14%40iki.fi Reviewed-by: Emre Hasegeli, Tom Lane, Alvaro Herrera 20 January 2020, 08:36:45 UTC
c7c2cc6 Tom Lane 20 January 2020, 00:15:15 UTC Fix out-of-memory handling in ecpglib. ecpg_build_params() would crash on a null pointer dereference if realloc() failed, due to updating the persistent "stmt" struct too aggressively. (Even without the crash, this would've leaked the old storage that we were trying to realloc.) Per Coverity. This seems to have been broken in commit 0cc050794, so back-patch into v12. 20 January 2020, 00:15:15 UTC
ac2dcca Michael Paquier 18 January 2020, 03:32:55 UTC Add GUC checks for ssl_min_protocol_version and ssl_max_protocol_version Mixing incorrect bounds set in the SSL context leads to confusing error messages generated by OpenSSL which are hard to act on. New checks are added within the GUC machinery to improve the user experience as they apply to any SSL implementation, not only OpenSSL, and doing the checks beforehand avoids the creation of a SSL during a reload (or startup) which we know will never be used anyway. Backpatch down to 12, as those parameters have been introduced by e73e67c. Author: Michael Paquier Reviewed-by: Daniel Gustafsson Discussion: https://postgr.es/m/20200114035420.GE1515@paquier.xyz Backpatch-through: 12 18 January 2020, 03:32:55 UTC
2e26460 Tom Lane 17 January 2020, 21:17:17 UTC Repair more failures with SubPlans in multi-row VALUES lists. Commit 9b63c13f0 turns out to have been fundamentally misguided: the parent node's subPlan list is by no means the only way in which a child SubPlan node can be hooked into the outer execution state. As shown in bug #16213 from Matt Jibson, we can also get short-lived tuple table slots added to the outer es_tupleTable list. At this point I have little faith that there aren't other possible connections as well; the long time it took to notice this problem shows that this isn't a heavily-exercised situation. Therefore, revert that fix, returning to the coding that passed a NULL parent plan pointer down to the transiently-built subexpressions. That gives us a pretty good guarantee that they won't hook into the outer executor state in any way. But then we need some other solution to make SubPlans work. Adopt the solution speculated about in the previous commit's log message: do expression initialization at plan startup for just those VALUES rows containing SubPlans, abandoning the goal of reclaiming memory intra-query for those rows. In practice it seems unlikely that queries containing a vast number of VALUES rows would be using SubPlans in them, so this should not give up much. (BTW, this test case also refutes my claim in connection with the prior commit that the issue only arises with use of LATERAL. That was just wrong: some variants of SubLink always produce SubPlans.) As with previous patch, back-patch to all supported branches. Discussion: https://postgr.es/m/16213-871ac3bc208ecf23@postgresql.org 17 January 2020, 21:17:32 UTC
bc21406 Alvaro Herrera 17 January 2020, 21:00:39 UTC Set ReorderBufferTXN->final_lsn more eagerly ... specifically, set it incrementally as each individual change is spilled down to disk. This way, it is set correctly when the transaction disappears without trace, ie. without leaving an XACT_ABORT wal record. (This happens when the server crashes midway through a transaction.) Failing to have final_lsn prevents ReorderBufferRestoreCleanup() from working, since it needs the final_lsn in order to know the endpoint of its iteration through spilled files. Commit df9f682c7bf8 already tried to fix the problem, but it didn't set the final_lsn in all cases. Revert that, since it's no longer needed. Author: Vignesh C Reviewed-by: Amit Kapila, Dilip Kumar Discussion: https://postgr.es/m/CALDaNm2CLk+K9JDwjYST0sPbGg5AQdvhUt0jbKyX_HdAE0jk3A@mail.gmail.com 17 January 2020, 21:00:39 UTC
162c951 Tomas Vondra 17 January 2020, 13:06:28 UTC Allocate freechunks bitmap as part of SlabContext The bitmap used by SlabCheck to cross-check free chunks in a block used to be allocated for each SlabCheck call, and was never freed. The memory leak could be fixed by simply adding a pfree call, but it's actually a bad idea to do any allocations in SlabCheck at all as it assumes the state of the memory management as a whole is sane. So instead we allocate the bitmap as part of SlabContext, which means we don't need to do any allocations in SlabCheck and the bitmap goes away together with the SlabContext. Backpatch to 10, where the Slab context was introduced. Author: Tomas Vondra Reported-by: Andres Freund Reviewed-by: Tom Lane Backpatch-through: 10 Discussion: https://www.postgresql.org/message-id/20200116044119.g45f7pmgz4jmodxj%40alap3.anarazel.de 17 January 2020, 14:30:51 UTC
0fca3d0 Michael Paquier 15 January 2020, 04:58:41 UTC Fix buggy logic in isTempNamespaceInUse() The logic introduced in this routine as of 246a6c8 would report an incorrect result when a session calls it to check if the temporary namespace owned by the session is in use or not. It is possible to optimize more the routine in this case to avoid a PGPROC lookup, but let's keep the logic simple. As this routine is used only by autovacuum for now, there were no live bugs, still let's be correct for any future code involving it. Author: Michael Paquier Reviewed-by: Julien Rouhaud Discussion: https://postgr.es/m/20200113093703.GA41902@paquier.xyz Backpatch-through: 11 15 January 2020, 04:58:41 UTC
29af9c5 Bruce Momjian 14 January 2020, 18:13:04 UTC docs: change "default role" wording to "predefined role" The new wording was determined to be more accurate. Also, update release note links that reference these sections. Reported-by: rirans@comcast.net Discussion: https://postgr.es/m/157742545062.1149.11052653770497832538@wrigleys.postgresql.org Backpatch-through: 9.6 14 January 2020, 18:13:04 UTC
fd5476b Dean Rasheed 14 January 2020, 09:51:28 UTC Make rewriter prevent auto-updates on views with conditional INSTEAD rules. A view with conditional INSTEAD rules and no unconditional INSTEAD rules or INSTEAD OF triggers is not auto-updatable. Previously we relied on a check in the executor to catch this, but that's problematic since the planner may fail to properly handle such a query and thus return a particularly unhelpful error to the user, before reaching the executor check. Instead, trap this in the rewriter and report the correct error there. Doing so also allows us to include more useful error detail than the executor check can provide. This doesn't change the existing behaviour of updatable views; it merely ensures that useful error messages are reported when a view isn't updatable. Per report from Pengzhou Tang, though not adopting that suggested fix. Back-patch to all supported branches. Discussion: https://postgr.es/m/CAG4reAQn+4xB6xHJqWdtE0ve_WqJkdyCV4P=trYr4Kn8_3_PEA@mail.gmail.com 14 January 2020, 09:51:28 UTC
fa1eaeb Amit Kapila 11 January 2020, 05:14:39 UTC Revert test added by commit d207038053. This test was trying to test the mechanism to release kernel FDs as needed to get us under the max_safe_fds limit in case of spill files. To do that, it needs to set max_files_per_process to a very low value which doesn't even permit starting of the server in the case when there are a few already opened files. This test also won't work on platforms where we use one FD per semaphore. Backpatch-through: 10, till where this test was added Discussion: https://postgr.es/m/CAA4eK1LHhERi06Q+MmP9qBXBBboi+7WV3910J0aUgz71LcnKAw@mail.gmail.com https://postgr.es/m/6485.1578583522@sss.pgh.pa.us 14 January 2020, 02:35:05 UTC
bf65f3c Peter Eisentraut 13 January 2020, 12:27:39 UTC Fix base backup with database OIDs larger than INT32_MAX The use of pg_atoi() for parsing a string into an Oid fails for values larger than INT32_MAX, since OIDs are unsigned. Instead, use atooid(). While this has less error checking, the contents of the data directory are expected to be trustworthy, so we don't need to go out of our way to do full error checking. Discussion: https://www.postgresql.org/message-id/flat/dea47fc8-6c89-a2b1-07e3-754ff1ab094b%402ndquadrant.com 13 January 2020, 12:41:09 UTC
52bc730 Amit Kapila 13 January 2020, 09:14:55 UTC Fix typo. Reported-by: Antonin Houska Author: Antonin Houska Backpatch-through: 11, where it was introduced Discussion: https://postgr.es/m/2246.1578900133@antos 13 January 2020, 09:22:59 UTC
70c17a8 Tom Lane 12 January 2020, 19:37:00 UTC Fix edge-case crashes and misestimation in range containment selectivity. When estimating the selectivity of "range_var <@ range_constant" or "range_var @> range_constant", if the upper (or respectively lower) bound of the range_constant was above the last bin of the range_var's histogram, the code would access uninitialized memory and potentially crash (though it seems the probability of a crash is quite low). Handle the endpoint cases explicitly to fix that. While at it, be more paranoid about the possibility of getting NaN or other silly results from the range type's subdiff function. And improve some comments. Ordinarily we'd probably add a regression test case demonstrating the bug in unpatched code. But it's too hard to get it to crash reliably because of the uninitialized-memory dependence, so skip that. Per bug #16122 from Adam Scott. It's been broken from the beginning, apparently, so backpatch to all supported branches. Diagnosis by Michael Paquier, patch by Andrey Borodin and Tom Lane. Discussion: https://postgr.es/m/16122-eb35bc248c806c15@postgresql.org 12 January 2020, 19:37:00 UTC
7c21a96 Michael Paquier 12 January 2020, 13:44:59 UTC Remove incorrect assertion for INSERT in logical replication's publisher On the publisher, it was assumed that an INSERT change cannot happen for a relation with no replica identity. However this is true only for a change that needs references to old rows, aka UPDATE or DELETE, so trying to use logical replication with a relation that has no replica identity led to an assertion failure in the publisher when issuing an INSERT. This commit removes the incorrect assertion, and adds more regression tests to provide coverage for relations without replica identity. Reported-by: Neha Sharma Author: Dilip Kumar, Michael Paquier Reviewed-by: Andres Freund Discussion: https://postgr.es/m/CANiYTQsL1Hb8_Km08qd32svrqNumXLJeoGo014O7VZymgOhZEA@mail.gmail.com Backpatch-through: 10 12 January 2020, 13:44:59 UTC
fde1554 Tom Lane 11 January 2020, 22:14:08 UTC Extensive code review for GSSAPI encryption mechanism. Fix assorted bugs in handling of non-blocking I/O when using GSSAPI encryption. The encryption layer could return the wrong status information to its caller, resulting in effectively dropping some data (or possibly in aborting a not-broken connection), or in a "livelock" situation where data remains to be sent but the upper layers think transmission is done and just go to sleep. There were multiple small thinkos contributing to that, as well as one big one (failure to think through what to do when a send fails after having already transmitted data). Note that these errors could cause failures whether the client application asked for non-blocking I/O or not, since both libpq and the backend always run things in non-block mode at this level. Also get rid of use of static variables for GSSAPI inside libpq; that's entirely not okay given that multiple connections could be open at once inside a single client process. Also adjust a bunch of random small discrepancies between the frontend and backend versions of the send/receive functions -- except for error handling, they should be identical, and now they are. Also extend the Kerberos TAP tests to exercise cases where nontrivial amounts of data need to be pushed through encryption. Before, those tests didn't provide any useful coverage at all for the cases of interest here. (They still might not, depending on timing, but at least there's a chance.) Per complaint from pmc@citylink and subsequent investigation. Back-patch to v12 where this code was introduced. Discussion: https://postgr.es/m/20200109181822.GA74698@gate.oper.dinoex.org 11 January 2020, 22:14:08 UTC
93078e6 Noah Misch 11 January 2020, 02:31:22 UTC Maintain valid md.c state when FileClose() fails. FileClose() failure ordinarily causes a PANIC. Suppose the user disables that PANIC via data_sync_retry=on. After mdclose() issued a FileClose() that failed, calls into md.c raised SIGSEGV. This fix adds repalloc() calls during mdclose(); update a comment about ignoring repalloc() cost. The rate of relation segment count change is a minor factor; more relevant to overall performance is the rate of mdclose() and subsequent re-opening of segments. Back-patch to v10, where commit 45e191e3aa62d47a8bc1a33f784286b2051f45cb introduced the bug. Reviewed by Kyotaro Horiguchi. Discussion: https://postgr.es/m/20191222091930.GA1280238@rfd.leadboat.com 11 January 2020, 02:31:25 UTC
2e89a12 Michael Paquier 10 January 2020, 00:37:16 UTC doc: Fix naming of SELinux Reported-by: Tham Nguyen Discussion: https://postgr.es/m/157851402876.29175.12977878383183540468@wrigleys.postgresql.org Backpatch-through: 9.4 10 January 2020, 00:37:16 UTC
e05a3b4 Peter Eisentraut 09 January 2020, 15:02:23 UTC doc: Add link to upgrading chapter to release notes Author: Vik Fearing <vik.fearing@2ndquadrant.com> Discussion: https://www.postgresql.org/message-id/flat/54c208b9-7e2c-6211-0ba0-ffb0429cf20b@2ndquadrant.com 09 January 2020, 15:02:23 UTC
20c4df8 Alvaro Herrera 08 January 2020, 17:33:49 UTC Reimplement nullification of walsender timestamp Make the value null only at pg_stat_activity-output time, as suggested by Tom Lane, instead of messing with the internal state. This should appease buildfarm members with force_parallel_mode=regress, which are running parallel queries on logical replication walsenders. The fact that walsenders can run parallel queries should perhaps be studied more carefully, but for the moment let's get rid of the red blots in buildfarm. Backpatch to pg10, like the previous commit. Discussion: https://postgr.es/m/30804.1578438763@sss.pgh.pa.us 08 January 2020, 17:33:49 UTC
c24f3b7 Tom Lane 08 January 2020, 14:42:53 UTC Fix handling of generated columns in ALTER TABLE. ALTER TABLE failed if a column referenced in a GENERATED expression had been added or changed in type earlier in the ALTER command. That's because the GENERATED expression needs to be evaluated against the table's updated tuples, but it was being evaluated against the original tuples. (Fortunately the executor has adequate cross-checks to notice the mismatch, so we just got an obscure error message and not anything more dangerous.) Per report from Andreas Joseph Krogh. Back-patch to v12 where GENERATED was added. Discussion: https://postgr.es/m/VisenaEmail.200.231b0a41523275d0.16ea7f800c7@tc7-visena 08 January 2020, 14:42:53 UTC
a1c003e Michael Paquier 08 January 2020, 01:36:22 UTC Revert "Forbid DROP SCHEMA on temporary namespaces" This reverts commit a052f6c, following complains from Robert Haas and Tom Lane. Backpatch down to 9.4, like the previous commit. Discussion: https://postgr.es/m/CA+TgmobL4npEX5=E5h=5Jm_9mZun3MT39Kq2suJFVeamc9skSQ@mail.gmail.com Backpatch-through: 9.4 08 January 2020, 01:36:22 UTC
fce9ba8 Alvaro Herrera 07 January 2020, 20:38:48 UTC pg_stat_activity: show NULL stmt start time for walsenders Returning a non-NULL time is pointless, sinc a walsender is not a process that would be running normal transactions anyway, but the code was unintentionally exposing the process start time intermittently, which was not only bogus but it also confused monitoring systems looking for idle transactions. Fix by avoiding all updates in walsenders. Backpatch to 11, where walsenders started appearing in pg_stat_activity. Reported-by: Tomas Vondra Discussion: https://postgr.es/m/20191209234409.exe7osmyalwkt5j4@development 07 January 2020, 20:38:48 UTC
b898452 Tom Lane 06 January 2020, 21:42:20 UTC Reduce the number of GetFlushRecPtr() calls done by walsenders. Since the WAL flush position only moves forward, it's safe to cache its previous value within each walsender process, and update from shared memory only once we've caught up to the previously-seen value. When there are many active walsenders, this makes for a very significant reduction in the amount of contention on the XLogCtl->info_lck spinlock. This patch also adjusts the logic so that we update our idea of the flush position after processing a WAL record, rather than beforehand. This may cause us to realize we're not caught up when the preceding coding would've thought that we were, but that seems all to the good; it may avoid a useless sleep-and-wakeup cycle. Back-patch to v12. The contention problem exists in prior branches, but it's much less severe (due to inefficiencies elsewhere) so there seems no need to take any risk of back-patching further. Pierre Ducroquet, reviewed by Julien Rouhaud Discussion: https://postgr.es/m/2931018.Vxl9zapr77@pierred-pdoc 06 January 2020, 21:42:20 UTC
8c2bfd9 Peter Eisentraut 06 January 2020, 07:21:14 UTC Have logical replication subscriber fire column triggers The logical replication apply worker did not fire per-column update triggers because the updatedCols bitmap in the RTE was not populated. This fixes that. Reviewed-by: Euler Taveira <euler@timbira.com.br> Discussion: https://www.postgresql.org/message-id/flat/21673e2d-597c-6afe-637e-e8b10425b240%402ndquadrant.com 06 January 2020, 09:43:55 UTC
6c1860b Tatsuo Ishii 05 January 2020, 10:50:27 UTC Docs: use more standard terminology "round-to-nearest-even" instead of "round-to-even". Per suggestion from Tom Lane. Discussion: https://postgr.es/m/flat/20191230.093451.1762483750956466101.t-ishii%40sraoss.co.jp 05 January 2020, 10:50:27 UTC
0c8836a Amit Kapila 03 January 2020, 05:22:46 UTC Fix typos in parallel query docs. Reported-by: Jon Jensen Author: Jon Jensen Reviewed-by: Amit Kapila and Robert Haas Backpatch-through: 10 Discussion: https://postgr.es/m/nycvar.YSQ.7.76.1912301807510.9899@ybpnyubfg 03 January 2020, 05:27:26 UTC
d732148 Alvaro Herrera 02 January 2020, 20:04:24 UTC Fix cloning of row triggers to sub-partitions When row triggers exist in partitioned partitions that are not either part of FKs or deferred unique constraints, they are not correctly cloned to their partitions. That's because they are marked "internal", and those are purposefully skipped when doing the clone triggers dance. Fix by relaxing the condition on which internal triggers are skipped. Amit Langote initially diagnosed the problem and proposed a fix, but I used a different approach. Reported-by: Petr Fedorov Discussion: https://postgr.es/m/6b3f0646-ba8c-b3a9-c62d-1c6651a1920f@phystech.edu 02 January 2020, 20:04:24 UTC
98b75c3 Peter Eisentraut 02 January 2020, 13:40:18 UTC Fix comment in test The comment was apparently copy-and-pasted and did not reflect the actual test outcome. 02 January 2020, 13:41:04 UTC
f8a6d8e Amit Kapila 02 January 2020, 05:10:32 UTC Fix running out of file descriptors for spill files. Currently while decoding changes, if the number of changes exceeds a certain threshold, we spill those to disk.  And this happens for each (sub)transaction.  Now, while reading all these files, we don't close them until we read all the files.  While reading these files, if the number of such files exceeds the maximum number of file descriptors, the operation errors out. Use PathNameOpenFile interface to open these files as that internally has the mechanism to release kernel FDs as needed to get us under the max_safe_fds limit. Reported-by: Amit Khandekar Author: Amit Khandekar Reviewed-by: Amit Kapila Backpatch-through: 9.4 Discussion: https://postgr.es/m/CAJ3gD9c-sECEn79zXw4yBnBdOttacoE-6gAyP0oy60nfs_sabQ@mail.gmail.com 02 January 2020, 06:23:53 UTC
8f3e44a Bruce Momjian 01 January 2020, 17:21:45 UTC Update copyrights for 2020 Backpatch-through: update all files in master, backpatch legal files through 9.4 01 January 2020, 17:21:45 UTC
b4507a2 Alvaro Herrera 27 December 2019, 21:34:30 UTC Add pg_dump test for triggers on partitioned tables This currently works, but add this test to ensure it continues to work. Lack of this test became evident after a recent bugfix submission that would have inadvertently broken it, in https://postgr.es/m/CA+HiwqFM2=i+uHB9o4OkLbE2S3sjPHoVe2wXuAD1GLJ4+Pk9eg@mail.gmail.com 27 December 2019, 21:34:30 UTC
2034bcc Bruce Momjian 27 December 2019, 19:49:08 UTC doc: add examples of creative use of unique expression indexes Unique expression indexes can constrain data in creative ways, so show two examples. Reported-by: Tuomas Leikola Discussion: https://postgr.es/m/156760275564.1127.12321702656456074572@wrigleys.postgresql.org Backpatch-through: 9.4 27 December 2019, 19:49:08 UTC
5d72f85 Bruce Momjian 27 December 2019, 19:33:30 UTC docs: clarify infinite range values from data-type infinities The previous docs referenced these distinct ideas confusingly. Reported-by: Eugen Konkov Discussion: https://postgr.es/m/376945611.20191026161529@yandex.ru Backpatch-through: 9.4 27 December 2019, 19:33:30 UTC
1dd8820 Michael Paquier 27 December 2019, 08:59:12 UTC Forbid DROP SCHEMA on temporary namespaces This operation was possible for the owner of the schema or a superuser. Down to 9.4, doing this operation would cause inconsistencies in a session whose temporary schema was dropped, particularly if trying to create new temporary objects after the drop. A more annoying consequence is a crash of autovacuum on an assertion failure when logging information about an orphaned temp table dropped. Note that because of 246a6c8 (present in v11~), which has made the removal of orphaned temporary tables more aggressive, the failure could be triggered more easily, but it is possible to reproduce down to 9.4. Reported-by: Mahendra Singh, Prabhat Sahu Author: Michael Paquier Reviewed-by: Kyotaro Horiguchi, Mahendra Singh Discussion: https://postgr.es/m/CAKYtNAr9Zq=1-ww4etHo-VCC-k120YxZy5OS01VkaLPaDbv2tg@mail.gmail.com Backpatch-through: 9.4 27 December 2019, 08:59:12 UTC
883c27a Tom Lane 26 December 2019, 20:19:39 UTC Fix possible loss of sync between rectypeid and underlying PLpgSQL_type. When revalidate_rectypeid() acts to update a stale record type OID in plpgsql's data structures, it fixes the active PLpgSQL_rec struct as well as the PLpgSQL_type struct it references. However, the latter is shared across function executions while the former is not. In a later function execution, the PLpgSQL_rec struct would be reinitialized by copy_plpgsql_datums and would then contain a stale type OID, typically leading to "could not open relation with OID NNNN" errors. revalidate_rectypeid() can easily fix this, fortunately, just by treating typ->typoid as authoritative. Per report and diagnosis from Ashutosh Sharma, though this is not his suggested fix. Back-patch to v11 where this code came in. Discussion: https://postgr.es/m/CAE9k0Pkd4dZwt9J5pS9xhJFWpUtqs05C9xk_GEwPzYdV=GxwWg@mail.gmail.com 26 December 2019, 20:19:39 UTC
50fa688 Michael Paquier 26 December 2019, 13:26:26 UTC Fix some comments related to logical repslot advancing confirmed_flush is part of a replication slot's information, but not confirmed_lsn. Author: Kyotaro Horiguchi Discussion: https://postgr.es/m/20191226.175919.17237335658671970.horikyota.ntt@gmail.com Backpatch-through: 11 26 December 2019, 13:26:26 UTC
8052aaf Thomas Munro 23 December 2019, 22:31:24 UTC Rotate instead of shifting hash join batch number. Our algorithm for choosing batch numbers turned out not to work effectively for multi-billion key inner relations. We would use more hash bits than we have, and effectively concentrate all tuples into a smaller number of batches than we intended. While ideally we should switch to wider hashes, for now, change the algorithm to one that effectively gives up bits from the bucket number when we don't have enough bits. That means we'll finish up with longer bucket chains than would be ideal, but that's better than having batches that don't fit in work_mem and can't be divided. Batch-patch to all supported releases. Author: Thomas Munro Reviewed-by: Tom Lane, thanks also to Tomas Vondra, Alvaro Herrera, Andres Freund for testing and discussion Reported-by: James Coleman Discussion: https://postgr.es/m/16104-dc11ed911f1ab9df%40postgresql.org 24 December 2019, 00:07:54 UTC
b5e7569 Joe Conway 23 December 2019, 18:33:34 UTC Disallow null category in crosstab_hash While building a hash map of categories in load_categories_hash, resulting category names have not thus far been checked to ensure they are not null. Prior to pg12 null category names worked to the extent that they did not crash on some platforms. This is because those system libraries have an snprintf which can deal with being passed a null pointer argument for a string. But even in those cases null categories did nothing useful. And on some platforms it crashed. As of pg12, our own version of snprintf gets called, and it does not deal with null pointer arguments at all, and crashes consistently. Fix that by disallowing null categories. They never worked usefully, and no one has ever asked for them to work previously. Back-patch to all supported branches. Reported-By: Ireneusz Pluta Discussion: https://postgr.es/m/16176-7489719b05e4303c@postgresql.org 23 December 2019, 18:33:34 UTC
7fbb39a Tom Lane 23 December 2019, 17:53:13 UTC Disallow partition key expressions that return pseudo-types. This wasn't checked originally, but it should have been, because in general pseudo-types can't be stored to and retrieved from disk. Notably, partition bound values of type "record" would not be interpretable by another session. In v12 and HEAD, add another flag to CheckAttributeType's repertoire so that it can produce a specific error message for this case. That's infeasible in older branches without an ABI break, so fall back to a slightly-less-nicely-worded error message in v10 and v11. Problem noted by Amit Langote, though this patch is not his initial solution. Back-patch to v10 where partitioning was introduced. Discussion: https://postgr.es/m/CA+HiwqFUzjfj9HEsJtYWcr1SgQ_=iCAvQ=O2Sx6aQxoDu4OiHw@mail.gmail.com 23 December 2019, 17:53:13 UTC
976cb11 Tom Lane 23 December 2019, 17:08:23 UTC Prevent a rowtype from being included in itself via a range. We probably should have thought of this case when ranges were added, but we didn't. (It's not the fault of commit eb51af71f, because ranges didn't exist then.) It's an old bug, so back-patch to all supported branches. Discussion: https://postgr.es/m/7782.1577051475@sss.pgh.pa.us 23 December 2019, 17:08:23 UTC
e1c056c Tom Lane 22 December 2019, 23:00:17 UTC Avoid low-probability regression test failures in timestamp[tz] tests. If the first transaction block in these tests were entered exactly at midnight (California time), they'd report a bogus failure due to 'now' and 'midnight' having the same values. Commit 8c2ac75c5 had dismissed this as being of negligible probability, but we've now seen it happen in the buildfarm, so let's prevent it. We can get pretty much the same test coverage without an it's-not-midnight assumption by moving the does-'now'-work cases into their own test step. While here, apply commit 47169c255's s/DELETE/TRUNCATE/ change to timestamptz as well as timestamp (not sure why that didn't occur to me at the time; the risk of failure is the same). Back-patch to all supported branches, since the main point is to get rid of potential buildfarm failures. Discussion: https://postgr.es/m/14821.1577031117@sss.pgh.pa.us 22 December 2019, 23:00:17 UTC
90281a3 Tom Lane 21 December 2019, 22:39:36 UTC In pgwin32_open, loop after ERROR_ACCESS_DENIED only if we can't stat. This fixes a performance problem introduced by commit 6d7547c21. ERROR_ACCESS_DENIED is returned in some other cases besides the delete-pending case considered by that commit; notably, if the given path names a directory instead of a plain file. In that case we'll uselessly loop for 1 second before returning the failure condition. That slows down some usage scenarios enough to cause test timeout failures on our Windows buildfarm critters. To fix, try to stat() the file, and sleep/loop only if that fails. It will fail in the delete-pending case, and also in the case where the deletion completed before we could stat(), so we have the cases where we want to loop covered. In the directory case, the stat() should succeed, letting us exit without a wait. One case where we'll still wait uselessly is if the access-denied problem pertains to a directory in the given pathname. But we don't expect that to happen in any performance-critical code path. There might be room to refine this further, but I'll push it now in hopes of making the buildfarm green again. Back-patch, like the preceding commit. Alexander Lakhin and Tom Lane Discussion: https://postgr.es/m/23073.1576626626@sss.pgh.pa.us 21 December 2019, 22:39:36 UTC
96aa9b6 Bruce Momjian 21 December 2019, 17:44:38 UTC docs: clarify handling of column lists in COPY TO/FROM Previously it was unclear how COPY FROM handled cases where not all columns were specified, or if the order didn't match. Reported-by: pavlo.golub@gmail.com Discussion: https://postgr.es/m/157487729344.7213.14245726713444755296@wrigleys.postgresql.org Backpatch-through: 9.4 21 December 2019, 17:44:38 UTC
e8f60e6 Tom Lane 20 December 2019, 20:34:07 UTC libpq should expose GSS-related parameters even when not implemented. We realized years ago that it's better for libpq to accept all connection parameters syntactically, even if some are ignored or restricted due to lack of the feature in a particular build. However, that lesson from the SSL support was for some reason never applied to the GSSAPI support. This is causing various buildfarm members to have problems with a test case added by commit 6136e94dc, and it's just a bad idea from a user-experience standpoint anyway, so fix it. While at it, fix some places where parameter-related infrastructure was added with the aid of a dartboard, or perhaps with the aid of the anti-pattern "add new stuff at the end". It should be safe to rearrange the contents of struct pg_conn even in released branches, since that's private to libpq (and we'd have to move some fields in some builds to fix this, anyway). Back-patch to all supported branches. Discussion: https://postgr.es/m/11297.1576868677@sss.pgh.pa.us 20 December 2019, 20:34:07 UTC
e5a37d9 Tom Lane 19 December 2019, 14:42:46 UTC Doc: add a short summary of available authentication methods. The "auth-methods" <sect1> used to include descriptions of all our authentication methods. Commit 56811e573 promoted its child <sect2>'s to <sect1>'s, which has advantages but also created some issues: * The auth-methods page itself is essentially empty/useless. * Links that pointed to "auth-methods" as a placeholder for all auth methods were rendered a bit nonsensical. * DocBook no longer provides a subsection table-of-contents here, which formerly was a useful if terse summary of available auth methods. To improve matters, add a handwritten list of all the auth methods. Per gripe from Dave Cramer. Back-patch to v11 where the previous commit came in. Discussion: https://postgr.es/m/CADK3HH+xQLhcPgg=kWqfogtXGGZr-JdSo=x=WQC0PkAVyxUWyQ@mail.gmail.com 19 December 2019, 14:42:58 UTC
a9f304f Robert Haas 19 December 2019, 14:24:44 UTC Update neglected comment. Commit d986d4e87f61c68f52c68ebc274960dc664b7b4e renamed a variable but neglected to update the corresponding comment. Amit Langote 19 December 2019, 14:30:12 UTC
c74111d Amit Kapila 16 December 2019, 09:53:46 UTC Fix subscriber invalid memory access on DDL. This patch allows building the local relmap cache for a subscribed relation after processing pending invalidation messages and potential relcache updates. Without this, the attributes in the local cache don't tally with the updated relcache entry leading to invalid memory access. Reported-by Jehan-Guillaume de Rorthais Author: Jehan-Guillaume de Rorthais and Vignesh C Reviewed-by: Amit Kapila Backpatch-through: 10 Discussion: https://postgr.es/m/20191025175929.7e90dbf5@firost 18 December 2019, 02:27:01 UTC
35c0152 Michael Paquier 18 December 2019, 01:11:31 UTC Remove shadow variables linked to RedoRecPtr in xlog.c This changes the routines in charge of recycling WAL segments past the last redo LSN to not use anymore "RedoRecPtr" as a local variable, which is also available in the context of the session as a static declaration, replacing it with "lastredoptr". This confusion has been introduced by d9fadbf, so backpatch down to v11 like the other commit. Thanks to Tom Lane, Robert Haas, Alvaro Herrera, Mark Dilger and Kyotaro Horiguchi for the input provided. Author: Ranier Vilela Discussion: https://postgr.es/m/MN2PR18MB2927F7B5F690065E1194B258E35D0@MN2PR18MB2927.namprd18.prod.outlook.com Backpatch-through: 11 18 December 2019, 01:11:31 UTC
97ba30f Tom Lane 17 December 2019, 22:44:28 UTC Fix error reporting for index expressions of prohibited types. If CheckAttributeType() threw an error about the datatype of an index expression column, it would report an empty column name, which is pretty unhelpful and certainly not the intended behavior. I (tgl) evidently broke this in commit cfc5008a5, by not noticing that the column's attname was used above where I'd placed the assignment of it. In HEAD and v12, this is trivially fixable by moving up the assignment of attname. Before v12 the code is a bit more messy; to avoid doing substantial refactoring, I took the lazy way out and just put in two copies of the assignment code. Report and patch by Amit Langote. Back-patch to all supported branches. Discussion: https://postgr.es/m/CA+HiwqFA+BGyBFimjiYXXMa2Hc3fcL0+OJOyzUNjhU4NCa_XXw@mail.gmail.com 17 December 2019, 22:44:28 UTC
0eac283 Amit Kapila 12 December 2019, 06:21:30 UTC Change overly strict Assert in TransactionGroupUpdateXidStatus. This Assert thought that an overflowed transaction can never get registered for the group update.  But that is not true, because even when the number of children for a transaction got reduced, the overflow flag is not changed. And, for group update, we only care about the current number of children for a transaction that is being committed. Based on comments by Andres Freund, remove a redundant Assert in TransactionIdSetPageStatus as we already had a static Assert for the same condition a few lines earlier. Reported-by: Vignesh C Author: Dilip Kumar Reviewed-by: Amit Kapila Backpatch-through: 11 Discussion: https://postgr.es/m/CAFiTN-s5=uJw-Z6JC9gcqtBSjXsrHnU63PXBrA=pnBjqnkm5UA@mail.gmail.com 17 December 2019, 04:09:21 UTC
95f43fe Tom Lane 16 December 2019, 20:10:55 UTC On Windows, wait a little to see if ERROR_ACCESS_DENIED goes away. Attempting to open a file fails with ERROR_ACCESS_DENIED if the file is flagged for deletion but not yet actually gone (another in a long list of reasons why Windows is broken, if you ask me). This seems likely to explain a lot of irreproducible failures we see in the buildfarm. This state generally persists for only a millisecond or so, so just wait a bit and retry. If it's a real permissions problem, we'll eventually give up and report it as such. If it's the pending deletion case, we'll see file-not-found and report that after the deletion completes, and the caller will treat that in an appropriate way. In passing, rejigger the existing retry logic for some other error cases so that we don't uselessly wait an extra time when we're not going to retry anymore. Alexander Lakhin (with cosmetic tweaks by me). Back-patch to all supported branches, since this seems like a pretty safe change and the problem is definitely real. Discussion: https://postgr.es/m/16161-7a985d2f1bbe8f71@postgresql.org 16 December 2019, 20:10:55 UTC
42d1acd Heikki Linnakangas 16 December 2019, 11:57:41 UTC Fix yet another crash in page split during GiST index creation. Commit a7ee7c8513 fixed a bug in GiST page split during index creation, where we failed to re-find the position of a downlink after the page containing it was split. However, that fix was incomplete; the other call to gistinserttuples() in the same function needs to also clear 'downlinkoffnum'. Fixes bug #16134 reported by Alexander Lakhin, for real this time. The previous fix was enough to fix the crash with the reproducer script for bug #16162, but the original script for #16134 was still crashing. Backpatch to v12, like the previous incomplete fix. Discussion: https://www.postgresql.org/message-id/d869f537-abe4-d2ea-0510-38cd053f5152%40gmail.com 16 December 2019, 11:58:07 UTC
back to top