Skip to main content

Browse the archive

https://github.com/postgres/postgres
07 April 2024, 13:56:23 UTC
sort by:
Revision Author Date Message Commit Date
6ac9161 Tom Lane 09 May 2022, 21:20:11 UTC Stamp 11.16. 09 May 2022, 21:20:11 UTC
b5576e1 Tom Lane 09 May 2022, 18:29:53 UTC Last-minute updates for release notes. Security: CVE-2022-1552 09 May 2022, 18:29:53 UTC
539f8c5 Tom Lane 09 May 2022, 18:15:37 UTC Fix core dump in transformValuesClause when there are no columns. The parser code that transformed VALUES from row-oriented to column-oriented lists failed if there were zero columns. You can't write that straightforwardly (though probably you should be able to), but the case can be reached by expanding a "tab.*" reference to a zero-column table. Per bug #17477 from Wang Ke. Back-patch to all supported branches. Discussion: https://postgr.es/m/17477-0af3c6ac6b0a6ae0@postgresql.org 09 May 2022, 18:15:37 UTC
c0406fa Tom Lane 09 May 2022, 15:02:37 UTC Revert "Disallow infinite endpoints in generate_series() for timestamps." This reverts commit eafdf9de06e9b60168f5e47cedcfceecdc6d4b5f and its back-branch counterparts. Corey Huinker pointed out that we'd discussed this exact change back in 2016 and rejected it, on the grounds that there's at least one usage pattern with LIMIT where an infinite endpoint can usefully be used. Perhaps that argument needs to be re-litigated, but there's no time left before our back-branch releases. To keep our options open, restore the status quo ante; if we do end up deciding to change things, waiting one more quarter won't hurt anything. Rather than just doing a straight revert, I added a new test case demonstrating the usage with LIMIT. That'll at least remind us of the issue if we forget again. Discussion: https://postgr.es/m/3603504.1652068977@sss.pgh.pa.us Discussion: https://postgr.es/m/CADkLM=dzw0Pvdqp5yWKxMd+VmNkAMhG=4ku7GnCZxebWnzmz3Q@mail.gmail.com 09 May 2022, 15:40:42 UTC
34ff156 Noah Misch 09 May 2022, 15:35:08 UTC In REFRESH MATERIALIZED VIEW, set user ID before running user code. It intended to, but did not, achieve this. Adopt the new standard of setting user ID just after locking the relation. Back-patch to v10 (all supported versions). Reviewed by Simon Riggs. Reported by Alvaro Herrera. Security: CVE-2022-1552 09 May 2022, 15:35:13 UTC
48ca290 Noah Misch 09 May 2022, 15:35:08 UTC Make relation-enumerating operations be security-restricted operations. When a feature enumerates relations and runs functions associated with all found relations, the feature's user shall not need to trust every user having permission to create objects. BRIN-specific functionality in autovacuum neglected to account for this, as did pg_amcheck and CLUSTER. An attacker having permission to create non-temp objects in at least one schema could execute arbitrary SQL functions under the identity of the bootstrap superuser. CREATE INDEX (not a relation-enumerating operation) and REINDEX protected themselves too late. This change extends to the non-enumerating amcheck interface. Back-patch to v10 (all supported versions). Sergey Shinderuk, reviewed (in earlier versions) by Alexander Lakhin. Reported by Alexander Lakhin. Security: CVE-2022-1552 09 May 2022, 15:35:13 UTC
d8ab73f Peter Eisentraut 09 May 2022, 10:27:34 UTC Translation updates Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: a148941910171cea2b5391c98cdd90c5cd980d93 09 May 2022, 10:27:34 UTC
4caa294 Andres Freund 09 May 2022, 00:59:30 UTC Disable 031_recovery_conflict.pl until after minor releases. f40d362a667 disabled part of 031_recovery_conflict.pl due to instability that's not trivial to fix in the back branches. That fixed most of the issues. But there was one more failure (on lapwing / REL_10_STABLE). That failure looks like it might be caused by a genuine problem. Disable the test until after the set of releases, to avoid packagers etc potentially having to fight with a test failure they can't do anything about. Discussion: https://postgr.es/m/3447060.1652032749@sss.pgh.pa.us Backpatch: 10-14 09 May 2022, 01:06:40 UTC
0f17150 Tom Lane 08 May 2022, 16:36:38 UTC Release notes for 14.3, 13.7, 12.11, 11.16, 10.21. 08 May 2022, 16:36:38 UTC
0371d6c Noah Misch 07 May 2022, 16:12:56 UTC Fix back-patch of "Under has_wal_read_bug, skip .../001_wal.pl." Per buildfarm members tadarida, snapper, and kittiwake. Back-patch to v10 (all supported versions). 07 May 2022, 16:13:37 UTC
8fa3386 Noah Misch 07 May 2022, 07:33:15 UTC Under has_wal_read_bug, skip contrib/bloom/t/001_wal.pl. Per buildfarm members snapper and kittiwake. Back-patch to v10 (all supported versions). Discussion: https://postgr.es/m/20220116210241.GC756210@rfd.leadboat.com 07 May 2022, 07:40:19 UTC
fb965ab Andres Freund 06 May 2022, 16:01:08 UTC Temporarily skip recovery deadlock test in back branches. The recovery deadlock test has a timing issue that was fixed in 5136967f1eb in HEAD. Unfortunately the same fix doesn't quite work in the back branches: 1) adjust_conf() doesn't exist, which is easy enough to work around 2) a restart cleares the recovery conflict stats < 15. These issues can be worked around, but given the upcoming set of minor releases, skip the problematic test for now. The buildfarm doesn't show failures in other parts of 031_recovery_conflict.pl. Discussion: https://postgr.es/m/20220506155827.dfnaheq6ufylwrqf@alap3.anarazel.de Backpatch: 10-14 06 May 2022, 16:07:42 UTC
47f47a8 Andres Freund 06 May 2022, 15:39:14 UTC Backpatch addition of pump_until() more completely. In a2ab9c06ea1 I just backpatched the introduction of pump_until(), without changing the existing local definitions (as 6da65a3f9a9). The necessary changes seemed more verbose than desirable. However, that leads to warnings, as I failed to realize... Backpatch to all versions containing pump_until() calls before f74496dd611 (there's none in 10). Discussion: https://postgr.es/m/2808491.1651802860@sss.pgh.pa.us Discussion: https://postgr.es/m/18b37361-b482-b9d8-f30d-6115cd5ce25c@enterprisedb.com Backpatch: 11-14 06 May 2022, 15:39:14 UTC
da72ff0 Tom Lane 05 May 2022, 18:54:53 UTC Update time zone data files to tzdata release 2022a. DST law changes in Palestine. Historical corrections for Chile and Ukraine. 05 May 2022, 18:55:22 UTC
8b9991c Andres Freund 04 May 2022, 21:15:36 UTC Revert "Fix timing issue in deadlock recovery conflict test." This reverts commit 7a1267f9fe74c8ab7bab08e58dfdb965fafc6e11. 04 May 2022, 21:15:36 UTC
7a1267f Andres Freund 04 May 2022, 19:50:38 UTC Fix timing issue in deadlock recovery conflict test. Per buildfarm members longfin and skink. Discussion: https://postgr.es/m/20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de Backpatch: 10- 04 May 2022, 19:57:28 UTC
25d5494 Andres Freund 03 May 2022, 01:29:35 UTC Backpatch 031_recovery_conflict.pl. The prior commit showed that the introduction of recovery conflict tests was a good idea. Without these tests it's hard to know that the fix didn't break something... 031_recovery_conflict.pl was introduced in 9f8a050f68d and extended in 21e184403bf. Discussion: https://postgr.es/m/20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de Backpatch: 10-14 03 May 2022, 01:30:15 UTC
9cda785 Andres Freund 03 May 2022, 01:25:00 UTC Fix possibility of self-deadlock in ResolveRecoveryConflictWithBufferPin(). The tests added in 9f8a050f68d failed nearly reliably on FreeBSD in CI, and occasionally on the buildfarm. That turns out to be caused not by a bug in the test, but by a longstanding bug in recovery conflict handling. The standby timeout handler, used by ResolveRecoveryConflictWithBufferPin(), executed SendRecoveryConflictWithBufferPin() inside a signal handler. A bad idea, because the deadlock timeout handler (or a spurious latch set) could have interrupted ProcWaitForSignal(). If unlucky that could cause a self-deadlock on ProcArrayLock, if the deadlock check is in SendRecoveryConflictWithBufferPin()->CancelDBBackends(). To fix, set a flag in StandbyTimeoutHandler(), and check the flag in ResolveRecoveryConflictWithBufferPin(). Subsequently the recovery conflict tests will be backpatched. Discussion: https://postgr.es/m/20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de Backpatch: 10- 03 May 2022, 01:30:15 UTC
2adb8de Andres Freund 03 May 2022, 01:09:44 UTC Backpatch addition of wait_for_log(), pump_until(). These were originally introduced in a2ab9c06ea1 and a2ab9c06ea1, as they are needed by a about-to-be-backpatched test. Discussion: https://postgr.es/m/20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de Backpatch: 10-14 03 May 2022, 01:09:44 UTC
1dd49df Etsuro Fujita 02 May 2022, 07:45:07 UTC Fix typo in comment. 02 May 2022, 07:45:07 UTC
b90ce0d Andrew Dunstan 25 April 2022, 19:02:13 UTC Inhibit mingw CRT's auto-globbing of command line arguments For some reason by default the mingw C Runtime takes it upon itself to expand program arguments that look like shell globbing characters. That has caused much scratching of heads and mis-attribution of the causes of some TAP test failures, so stop doing that. This removes an inconsistency with Windows binaries built with MSVC, which have no such behaviour. Per suggestion from Noah Misch. Backpatch to all live branches. Discussion: https://postgr.es/m/20220423025927.GA1274057@rfd.leadboat.com 25 April 2022, 19:51:27 UTC
8b910d9 Andrew Dunstan 21 April 2022, 13:28:47 UTC Support new perl module namespace in stable branches Commit b3b4d8e68a moved our perl test modules to a better namespace structure, but this has made life hard for people wishing to backpatch improvements in the TAP tests. Here we alleviate much of that difficulty by implementing the new module names on top of the old modules, mostly by using a little perl typeglob aliasing magic, so that we don't have a dual maintenance burden. This should work both for the case where a new test is backpatched and the case where a fix to an existing test that uses the new namespace is backpatched. Reviewed by Michael Paquier Per complaint from Andres Freund Discussion: https://postgr.es/m/20220418141530.nfxtkohefvwnzncl@alap3.anarazel.de Applied to branches 10 through 14 21 April 2022, 13:28:47 UTC
adb2d84 Peter Geoghegan 21 April 2022, 00:17:35 UTC Fix CLUSTER tuplesorts on abbreviated expressions. CLUSTER sort won't use the datum1 SortTuple field when clustering against an index whose leading key is an expression. This makes it unsafe to use the abbreviated keys optimization, which was missed by the logic that sets up SortSupport state. Affected tuplesorts output tuples in a completely bogus order as a result (the wrong SortSupport based comparator was used for the leading attribute). This issue is similar to the bug fixed on the master branch by recent commit cc58eecc5d. But it's a far older issue, that dates back to the introduction of the abbreviated keys optimization by commit 4ea51cdfe8. Backpatch to all supported versions. Author: Peter Geoghegan <pg@bowt.ie> Author: Thomas Munro <thomas.munro@gmail.com> Discussion: https://postgr.es/m/CA+hUKG+bA+bmwD36_oDxAoLrCwZjVtST2fqe=b4=qZcmU7u89A@mail.gmail.com Backpatch: 10- 21 April 2022, 00:17:35 UTC
e7adbd2 Tom Lane 20 April 2022, 22:08:15 UTC Disallow infinite endpoints in generate_series() for timestamps. Such cases will lead to infinite loops, so they're of no practical value. The numeric variant of generate_series() already threw error for this, so borrow its message wording. Per report from Richard Wesley. Back-patch to all supported branches. Discussion: https://postgr.es/m/91B44E7B-68D5-448F-95C8-B4B3B0F5DEAF@duckdblabs.com 20 April 2022, 22:08:15 UTC
9130f8c Tom Lane 20 April 2022, 03:03:59 UTC Fix breakage in AlterFunction(). An ALTER FUNCTION command that tried to update both the function's proparallel property and its proconfig list failed to do the former, because it stored the new proparallel value into a tuple that was no longer the interesting one. Carelessness in 7aea8e4f2. (I did not bother with a regression test, because the only likely future breakage would be for someone to ignore the comment I added and add some other field update after the heap_modify_tuple step. A test using existing function properties could not catch that.) Per report from Bryn Llewellyn. Back-patch to all supported branches. Discussion: https://postgr.es/m/8AC9A37F-99BD-446F-A2F7-B89AD0022774@yugabyte.com 20 April 2022, 03:03:59 UTC
a90de82 Amit Kapila 19 April 2022, 03:59:34 UTC Fix the check to limit sync workers. We don't allow to invoke more sync workers once we have reached the sync worker limit per subscription. But the check to enforce this also doesn't allow to launch an apply worker if it gets restarted. This code was introduced by commit de43897122 but we caught the problem only with the test added by recent commit c91f71b9dc which started failing occasionally in the buildfarm. As per buildfarm. Diagnosed-by: Amit Kapila, Masahiko Sawada, Tomas Vondra Author: Amit Kapila Backpatch-through: 10 Discussion: https://postgr.es/m/CAH2L28vddB_NFdRVpuyRBJEBWjz4BSyTB=_ektNRH8NJ1jf95g@mail.gmail.com https://postgr.es/m/f90d2b03-4462-ce95-a524-d91464e797c8@enterprisedb.com 19 April 2022, 03:59:34 UTC
20af446 Tom Lane 18 April 2022, 16:16:45 UTC Avoid invalid array reference in transformAlterTableStmt(). Don't try to look at the attidentity field of system attributes, because they're not there in the TupleDescAttr array. Sometimes this is harmless because we accidentally pick up a zero, but otherwise we'll report "no owned sequence found" from an attempt to alter a system attribute. (It seems possible that a SIGSEGV could occur, too, though I've not seen it in testing.) It's not in this function's charter to complain that you can't alter a system column, so instead just hard-wire an assumption that system attributes aren't identities. I didn't bother with a regression test because the appearance of the bug is very erratic. Per bug #17465 from Roman Zharkov. Back-patch to all supported branches. (There's not actually a live bug before v12, because before that get_attidentity() did the right thing anyway. But for consistency I changed the test in the older branches too.) Discussion: https://postgr.es/m/17465-f2a554a6cb5740d3@postgresql.org 18 April 2022, 16:16:45 UTC
4751a13 Michael Paquier 18 April 2022, 02:40:25 UTC Fix race in TAP test 002_archiving.pl when restoring history file This test, introduced in df86e52, uses a second standby to check that it is able to remove correctly RECOVERYHISTORY and RECOVERYXLOG at the end of recovery. This standby uses the archives of the primary to restore its contents, with some of the archive's contents coming from the first standby previously promoted. In slow environments, it was possible that the test did not check what it should, as the history file generated by the promotion of the first standby may not be stored yet on the archives the second standby feeds on. So, it could be possible that the second standby selects an incorrect timeline, without restoring a history file at all. This commits adds a wait phase to make sure that the history file required by the second standby is archived before this cluster is created. This relies on poll_query_until() with pg_stat_file() and an absolute path, something not supported in REL_10_STABLE. While on it, this adds a new test to check that the history file has been restored by looking at the logs of the second standby. This ensures that a RECOVERYHISTORY, whose removal needs to be checked, is created in the first place. This should make the test more robust. This test has been introduced by df86e52, but it came in light as an effect of the bug fixed by acf1dd42, where the extra restore_command calls made the test much slower. Reported-by: Andres Freund Discussion: https://postgr.es/m/YlT23IvsXkGuLzFi@paquier.xyz Backpatch-through: 11 18 April 2022, 02:40:25 UTC
d9f4348 Noah Misch 17 April 2022, 00:43:54 UTC Add a temp-install prerequisite to src/interfaces/ecpg "checktcp". The target failed, tested $PATH binaries, or tested a stale temporary installation. Commit c66b438db62748000700c9b90b585e756dd54141 missed this. Back-patch to v10 (all supported versions). 17 April 2022, 00:43:58 UTC
6270ee4 Robert Haas 14 April 2022, 15:10:16 UTC Rethink the delay-checkpoint-end mechanism in the back-branches. The back-patch of commit bbace5697df12398e87ffd9879171c39d27f5b33 had the unfortunate effect of changing the layout of PGPROC in the back-branches, which could break extensions. This happened because it changed the delayChkpt from type bool to type int. So, change it back, and add a new bool delayChkptEnd field instead. The new field should fall within what used to be padding space within the struct, and so hopefully won't cause any extensions to break. Per report from Markus Wanner and discussion with Tom Lane and others. Patch originally by me, somewhat revised by Markus Wanner per a suggestion from Michael Paquier. A very similar patch was developed by Kyotaro Horiguchi, but I failed to see the email in which that was posted before writing one of my own. Discussion: http://postgr.es/m/CA+Tgmoao-kUD9c5nG5sub3F7tbo39+cdr8jKaOVEs_1aBWcJ3Q@mail.gmail.com Discussion: http://postgr.es/m/20220406.164521.17171257901083417.horikyota.ntt@gmail.com 14 April 2022, 15:10:16 UTC
79fed07 Michael Paquier 14 April 2022, 06:09:42 UTC pageinspect: Fix handling of all-zero pages Getting from get_raw_page() an all-zero page is considered as a valid case by the buffer manager and it can happen for example when finding a corrupted page with zero_damaged_pages enabled (using zero_damaged_pages to look at corrupted pages happens), or after a crash when a relation file is extended before any WAL for its new data is generated (before a vacuum or autovacuum job comes in to do some cleanup). However, all the functions of pageinspect, as of the index AMs (except hash that has its own idea of new pages), heap, the FSM or the page header have never worked with all-zero pages, causing various crashes when going through the page internals. This commit changes all the pageinspect functions to be compliant with all-zero pages, where the choice is made to return NULL or no rows for SRFs when finding a new page. get_raw_page() still works the same way, returning a batch of zeros in the bytea of the page retrieved. A hard error could be used but NULL, while more invasive, is useful when scanning relation files in full to get a batch of results for a single relation in one query. Tests are added for all the code paths impacted. Reported-by: Daria Lepikhova Author: Michael Paquier Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru Backpatch-through: 10 14 April 2022, 06:09:42 UTC
d53f689 Tom Lane 12 April 2022, 22:21:04 UTC Doc: tweak textsearch.sgml for SEO purposes. Google seems to like to return textsearch.html for queries about GIN and GiST indexes, even though it's not a primary reference for either. It seems likely that that's because those keywords appear in the page title. Since "GIN and GiST Index Types" is not a very apposite title for this material anyway, rename the section in hopes of stopping that. Also provide explicit links to the GIN and GiST chapters, to help anyone who finds their way to this page regardless. Per gripe from Jan Piotrowski. Back-patch to supported branches. (Unfortunately Google is likely to continue returning the 9.1 version of this page, but improving that situation is a matter for the www team.) Discussion: https://postgr.es/m/164978902252.1276550.9330175733459697101@wrigleys.postgresql.org 12 April 2022, 22:21:30 UTC
b9eb041 Tom Lane 31 March 2022, 18:29:24 UTC Fix postgres_fdw to check shippability of sort clauses properly. postgres_fdw would push ORDER BY clauses to the remote side without verifying that the sort operator is safe to ship. Moreover, it failed to print a suitable USING clause if the sort operator isn't default for the sort expression's type. The net result of this is that the remote sort might not have anywhere near the semantics we expect, which'd be disastrous for locally-performed merge joins in particular. We addressed similar issues in the context of ORDER BY within an aggregate function call in commit 7012b132d, but failed to notice that query-level ORDER BY was broken. Thus, much of the necessary logic already existed, but it requires refactoring to be usable in both cases. Back-patch to all supported branches. In HEAD only, remove the core code's copy of find_em_expr_for_rel, which is no longer used and really should never have been pushed into equivclass.c in the first place. Ronan Dunklau, per report from David Rowley; reviews by David Rowley, Ranier Vilela, and myself Discussion: https://postgr.es/m/CAApHDvr4OeC2DBVY--zVP83-K=bYrTD7F8SZDhN4g+pj2f2S-A@mail.gmail.com 31 March 2022, 18:29:24 UTC
1430431 Tom Lane 31 March 2022, 15:24:26 UTC Add missing newline in one libpq error message. Oversight in commit a59c79564. Back-patch, as that was. Noted by Peter Eisentraut. Discussion: https://postgr.es/m/7f85ef6d-250b-f5ec-9867-89f0b16d019f@enterprisedb.com 31 March 2022, 15:24:26 UTC
a5d4a38 Daniel Gustafsson 31 March 2022, 10:03:33 UTC doc: Fix typo in ANALYZE documentation Commit 61fa6ca79b3 accidentally wrote constrast instead of contrast. Backpatch-through: 10 Discussion: https://postgr.es/m/88903179-5ce2-3d4d-af43-7830372bdcb6@enterprisedb.com 31 March 2022, 10:03:33 UTC
166f143 Etsuro Fujita 30 March 2022, 10:00:07 UTC Fix typo in comment. 30 March 2022, 10:00:07 UTC
0fc2455 Tomas Vondra 28 March 2022, 12:27:36 UTC Document autoanalyze limitations for partitioned tables When dealing with partitioned tables, counters for partitioned tables are not updated when modifying child tables. This means autoanalyze may not update optimizer statistics for the parent relations, which can result in poor plans for some queries. It's worth documenting this limitation, so that people are aware of it and can take steps to mitigate it (e.g. by setting up a script executing ANALYZE regularly). Backpatch to v10. Older branches are affected too, of couse, but we no longer maintain those. Author: Justin Pryzby Reviewed-by: Zhihong Yu, Tomas Vondra Backpatch-through: 10 Discussion: https://postgr.es/m/20210913035409.GA10647%40telsasoft.com 28 March 2022, 12:31:45 UTC
7d935bd Andres Freund 23 March 2022, 23:38:43 UTC waldump: fix use-after-free in search_directory(). After closedir() dirent->d_name is not valid anymore. As there alerady are a few places relying on the limited lifetime of pg_waldump, do so here as well, and just pg_strdup() the string. The bug was introduced in fc49e24fa69a. Found by UBSan, run locally. Backpatch: 11-, like fc49e24fa69 itself. 28 March 2022, 01:15:17 UTC
1a2fdf8 Michael Paquier 27 March 2022, 08:54:03 UTC pageinspect: Add more sanity checks to prevent out-of-bound reads A couple of code paths use the special area on the page passed by the function caller, expecting to find some data in it. However, feeding an incorrect page can lead to out-of-bound reads when trying to access the page special area (like a heap page that has no special area, leading PageGetSpecialPointer() to grab a pointer outside the allocated page). The functions used for hash and btree indexes have some protection already against that, while some other functions using a relation OID as argument would make sure that the access method involved is correct, but functions taking in input a raw page without knowing the relation the page is attached to would run into problems. This commit improves the set of checks used in the code paths of BRIN, btree (including one check if a leaf page is found with a non-zero level), GIN and GiST to verify that the page given in input has a special area size that fits with each access method, which is done though PageGetSpecialSize(), becore calling PageGetSpecialPointer(). The scope of the checks done is limited to work with pages that one would pass after getting a block with get_raw_page(), as it is possible to craft byteas that could bypass existing code paths. Having too many checks would also impact the usability of pageinspect, as the existing code is very useful to look at the content details in a corrupted page, so the focus is really to avoid out-of-bound reads as this is never a good thing even with functions whose execution is limited to superusers. The safest approach could be to rework the functions so as these fetch a block using a relation OID and a block number, but there are also cases where using a raw page is useful. Tests are added to cover all the code paths that needed such checks, and an error message for hash indexes is reworded to fit better with what this commit adds. Reported-By: Alexander Lakhin Author: Julien Rouhaud, Michael Paquier Discussion: https://postgr.es/m/16527-ef7606186f0610a1@postgresql.org Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru Backpatch-through: 10 27 March 2022, 08:54:03 UTC
a2a1aa8 Tom Lane 26 March 2022, 18:29:29 UTC Suppress compiler warning in relptr_store(). clang 13 with -Wextra warns that "performing pointer subtraction with a null pointer has undefined behavior" in the places where freepage.c tries to set a relptr variable to constant NULL. This appears to be a compiler bug, but it's unlikely to get fixed instantly. Fortunately, we can work around it by introducing an inline support function, which seems like a good change anyway because it removes the macro's existing double-evaluation hazard. Backpatch to v10 where this code was introduced. Patch by me, based on an idea of Andres Freund's. Discussion: https://postgr.es/m/48826.1648310694@sss.pgh.pa.us 26 March 2022, 18:29:56 UTC
3d263b0 Tom Lane 25 March 2022, 18:23:26 UTC Harden TAP tests that intentionally corrupt page checksums. The previous method for doing that was to write zeroes into a predetermined set of page locations. However, there's a roughly 1-in-64K chance that the existing checksum will match by chance, and yesterday several buildfarm animals started to reproducibly see that, resulting in test failures because no checksum mismatch was reported. Since the checksum includes the page LSN, test success depends on the length of the installation's WAL history, which is affected by (at least) the initial catalog contents, the set of locales installed on the system, and the length of the pathname of the test directory. Sooner or later we were going to hit a chance match, and today is that day. Harden these tests by specifically inverting the checksum field and leaving all else alone, thereby guaranteeing that the checksum is incorrect. In passing, fix places that were using seek() to set up for syswrite(), a combination that the Perl docs very explicitly warn against. We've probably escaped problems because no regular buffered I/O is done on these filehandles; but if it ever breaks, we wouldn't deserve or get much sympathy. Although we've only seen problems in HEAD, now that we recognize the environmental dependencies it seems like it might be just a matter of time until someone manages to hit this in back-branch testing. Hence, back-patch to v11 where we started doing this kind of test. Discussion: https://postgr.es/m/3192026.1648185780@sss.pgh.pa.us 25 March 2022, 18:23:26 UTC
118f1a3 Robert Haas 24 March 2022, 18:49:08 UTC Fix possible recovery trouble if TRUNCATE overlaps a checkpoint. If TRUNCATE causes some buffers to be invalidated and thus the checkpoint does not flush them, TRUNCATE must also ensure that the corresponding files are truncated on disk. Otherwise, a replay from the checkpoint might find that the buffers exist but have the wrong contents, which may cause replay to fail. Report by Teja Mupparti. Patch by Kyotaro Horiguchi, per a design suggestion from Heikki Linnakangas, with some changes to the comments by me. Review of this and a prior patch that approached the issue differently by Heikki Linnakangas, Andres Freund, Álvaro Herrera, Masahiko Sawada, and Tom Lane. Discussion: http://postgr.es/m/BYAPR06MB6373BF50B469CA393C614257ABF00@BYAPR06MB6373.namprd06.prod.outlook.com 24 March 2022, 18:49:08 UTC
2121d58 Andres Freund 23 March 2022, 20:05:25 UTC Don't call fwrite() with len == 0 when writing out relcache init file. Noticed via -fsanitize=undefined. Backpatch to all branches, for the same reasons as 46ab07ffda9. Discussion: https://postgr.es/m/20220323173537.ll7klrglnp4gn2um@alap3.anarazel.de Backpatch: 10- 23 March 2022, 20:13:40 UTC
a77dddb Andres Freund 23 March 2022, 19:43:14 UTC configure: check for dlsym instead of dlopen. When building with sanitizers the sanitizer library provides dlopen, but not dlsym(), making configure think that -ldl isn't needed. Just checking for dlsym() ought to suffice, hard to see dlsym() being provided without dlopen() also being provided. Backpatch to all branches, for the same reasons as 46ab07ffda9. Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/20220323173537.ll7klrglnp4gn2um@alap3.anarazel.de Backpatch: 10- 23 March 2022, 19:43:36 UTC
efe44fb Alvaro Herrera 23 March 2022, 18:23:51 UTC pg_upgrade: Upgrade an Assert to a real 'if' test It seems possible for the condition being tested to be true in production, and nobody would never know (except when some data eventually becomes corrupt?). Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m//202109040001.zky3wgv2qeqg@alvherre.pgsql 23 March 2022, 18:23:51 UTC
199cd7b Alvaro Herrera 23 March 2022, 17:22:10 UTC Fix "missing continuation record" after standby promotion Invalidate abortedRecPtr and missingContrecPtr after a missing continuation record is successfully skipped on a standby. This fixes a PANIC caused when a recently promoted standby attempts to write an OVERWRITE_RECORD with an LSN of the previously read aborted record. Backpatch to 10 (all stable versions). Author: Sami Imseih <simseih@amazon.com> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m/44D259DE-7542-49C4-8A52-2AB01534DCA9@amazon.com 23 March 2022, 17:22:10 UTC
cd1951b Andres Freund 22 March 2022, 15:22:02 UTC Add missing dependency of pg_dumpall to WIN32RES. When cross-building to windows, or building with mingw on windows, the build could fail with x86_64-w64-mingw32-gcc: error: win32ver.o: No such file or director because pg_dumpall didn't depend on WIN32RES, but it's recipe references it. The build nevertheless succeeded most of the time, due to pg_dump/pg_restore having the required dependency, causing win32ver.o to be built. Reported-By: Thomas Munro <thomas.munro@gmail.com> Discussion: https://postgr.es/m/CA+hUKGJeekpUPWW6yCVdf9=oBAcCp86RrBivo4Y4cwazAzGPng@mail.gmail.com Backpatch: 10-, omission present on all live branches 22 March 2022, 15:28:54 UTC
635abe6 Michael Paquier 22 March 2022, 04:21:54 UTC Fix failures in SSL tests caused by out-of-tree keys and certificates This issue is environment-sensitive, where the SSL tests could fail in various way by feeding on defaults provided by sslcert, sslkey, sslrootkey, sslrootcert, sslcrl and sslcrldir coming from a local setup, as of ~/.postgresql/ by default. Horiguchi-san has reported two failures, but more advanced testing from me (aka inclusion of garbage SSL configuration in ~/.postgresql/ for all the configuration parameters) has showed dozens of failures that can be triggered in the whole test suite. History has showed that we are not good when it comes to address such issues, fixing them locally like in dd87799, and such problems keep appearing. This commit strengthens the entire test suite to put an end to this set of problems by embedding invalid default values in all the connection strings used in the tests. The invalid values are prefixed in each connection string, relying on the follow-up values passed in the connection string to enforce any invalid value previously set. Note that two tests related to CRLs are required to fail with certain pre-set configurations, but we can rely on enforcing an empty value instead after the invalid set of values. Reported-by: Kyotaro Horiguchi Reviewed-by: Andrew Dunstan, Daniel Gustafsson, Kyotaro Horiguchi Discussion: https://postgr.es/m/20220316.163658.1122740600489097632.horikyota.ntt@gmail.com backpatch-through: 10 22 March 2022, 04:21:54 UTC
5de2441 Tom Lane 21 March 2022, 21:44:29 UTC Fix assorted missing logic for GroupingFunc nodes. The planner needs to treat GroupingFunc like Aggref for many purposes, in particular with respect to processing of the argument expressions, which are not to be evaluated at runtime. A few places hadn't gotten that memo, notably including subselect.c's processing of outer-level aggregates. This resulted in assertion failures or wrong plans for cases in which a GROUPING() construct references an outer aggregation level. Also fix missing special cases for GroupingFunc in cost_qual_eval (resulting in wrong cost estimates for GROUPING(), although it's not clear that that would affect plan shapes in practice) and in ruleutils.c (resulting in excess parentheses in pretty-print mode). Per bug #17088 from Yaoguang Chen. Back-patch to all supported branches. Richard Guo, Tom Lane Discussion: https://postgr.es/m/17088-e33882b387de7f5c@postgresql.org 21 March 2022, 21:44:29 UTC
b8ae17f Tom Lane 21 March 2022, 16:22:13 UTC Fix risk of deadlock failure while dropping a partitioned index. DROP INDEX needs to lock the index's table before the index itself, else it will deadlock against ordinary queries that acquire the relation locks in that order. This is correctly mechanized for plain indexes by RangeVarCallbackForDropRelation; but in the case of a partitioned index, we neglected to lock the child tables in advance of locking the child indexes. We can fix that by traversing the inheritance tree and acquiring the needed locks in RemoveRelations, after we have acquired our locks on the parent partitioned table and index. While at it, do some refactoring to eliminate confusion between the actual and expected relkind in RangeVarCallbackForDropRelation. We can save a couple of syscache lookups too, by having that function pass back info that RemoveRelations will need. Back-patch to v11 where partitioned indexes were added. Jimmy Yih, Gaurab Dey, Tom Lane Discussion: https://postgr.es/m/BYAPR05MB645402330042E17D91A70C12BD5F9@BYAPR05MB6454.namprd05.prod.outlook.com 21 March 2022, 16:22:13 UTC
f693e99 Tom Lane 20 March 2022, 16:39:40 UTC Doc: fix our example systemd script. The example used "TimeoutSec=0", but systemd's documented way to get the desired effect is "TimeoutSec=infinity". Discussion: https://postgr.es/m/164770078557.670.5467111518383664377@wrigleys.postgresql.org 20 March 2022, 16:40:01 UTC
b68929d Michael Paquier 19 March 2022, 07:37:57 UTC doc: Mention SET TABLESPACE clause for ALTER MATERIALIZED VIEW This command flavor is supported, but there was nothing in the documentation about it. Author: Yugo Nagata Discussion: https://postgr.es/m/20220316133337.5dc9740abfa24c25ec9f67f5@sraoss.co.jp Backpatch-through: 10 19 March 2022, 07:37:57 UTC
84f3ecd Tom Lane 18 March 2022, 20:01:42 UTC Fix incorrect xmlschema output for types timetz and timestamptz. The output of table_to_xmlschema() and allied functions includes a regex describing valid values for these types ... but the regex was itself invalid, as it failed to escape a literal "+" sign. Report and fix by Renan Soares Lopes. Back-patch to all supported branches. Discussion: https://postgr.es/m/7f6fabaa-3f8f-49ab-89ca-59fbfe633105@me.com 18 March 2022, 20:01:42 UTC
13b54d1 Tom Lane 17 March 2022, 22:18:05 UTC Revert applying column aliases to the output of whole-row Vars. In commit bf7ca1587, I had the bright idea that we could make the result of a whole-row Var (that is, foo.*) track any column aliases that had been applied to the FROM entry the Var refers to. However, that's not terribly logically consistent, because now the output of the Var is no longer of the named composite type that the Var claims to emit. bf7ca1587 tried to handle that by changing the output tuple values to be labeled with a blessed RECORD type, but that's really pretty disastrous: we can wind up storing such tuples onto disk, whereupon they're not readable by other sessions. The only practical fix I can see is to give up on what bf7ca1587 tried to do, and say that the column names of tuples produced by a whole-row Var are always those of the underlying named composite type, query aliases or no. While this introduces some inconsistencies, it removes others, so it's not that awful in the abstract. What *is* kind of awful is to make such a behavioral change in a back-patched bug fix. But corrupt data is worse, so back-patched it will be. (A workaround available to anyone who's unhappy about this is to introduce an extra level of sub-SELECT, so that the whole-row Var is referring to the sub-SELECT's output and not to a named table type. Then the Var is of type RECORD to begin with and there's no issue.) Per report from Miles Delahunty. The faulty commit dates to 9.5, so back-patch to all supported branches. Discussion: https://postgr.es/m/2950001.1638729947@sss.pgh.pa.us 17 March 2022, 22:18:05 UTC
ca522c6 Thomas Munro 16 March 2022, 04:20:24 UTC Fix race between DROP TABLESPACE and checkpointing. Commands like ALTER TABLE SET TABLESPACE may leave files for the next checkpoint to clean up. If such files are not removed by the time DROP TABLESPACE is called, we request a checkpoint so that they are deleted. However, there is presently a window before checkpoint start where new unlink requests won't be scheduled until the following checkpoint. This means that the checkpoint forced by DROP TABLESPACE might not remove the files we expect it to remove, and the following ERROR will be emitted: ERROR: tablespace "mytblspc" is not empty To fix, add a call to AbsorbSyncRequests() just before advancing the unlink cycle counter. This ensures that any unlink requests forwarded prior to checkpoint start (i.e., when ckpt_started is incremented) will be processed by the current checkpoint. Since AbsorbSyncRequests() performs memory allocations, it cannot be called within a critical section, so we also need to move SyncPreCheckpoint() to before CreateCheckPoint()'s critical section. This is an old bug, so back-patch to all supported versions. Author: Nathan Bossart <nathandbossart@gmail.com> Reported-by: Nathan Bossart <nathandbossart@gmail.com> Reviewed-by: Thomas Munro <thomas.munro@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/20220215235845.GA2665318%40nathanxps13 16 March 2022, 04:38:55 UTC
09c9774 Michael Paquier 16 March 2022, 03:30:02 UTC pageinspect: Fix memory context allocation of page in brin_revmap_data() This caused the function to fail, as the aligned copy of the raw page given by the function caller was not saved in the correct memory context, which needs to be multi_call_memory_ctx in this case. Issue introduced by 076f4d9. Per buildfarm members sifika, mylodon and longfin. I have reproduced that locally with macos. Discussion: https://postgr.es/m/YjFPOtfCW6yLXUeM@paquier.xyz Backpatch-through: 10 16 March 2022, 03:30:02 UTC
2389ee8 Michael Paquier 16 March 2022, 02:20:57 UTC pageinspect: Fix handling of page sizes and AM types This commit fixes a set of issues related to the use of the SQL functions in this module when the caller is able to pass down raw page data as input argument: - The page size check was fuzzy in a couple of places, sometimes looking after only a sub-range, but what we are looking for is an exact match on BLCKSZ. After considering a few options here, I have settled down to do a generalization of get_page_from_raw(). Most of the SQL functions already used that, and this is not strictly required if not accessing an 8-byte-wide value from a raw page, but this feels safer in the long run for alignment-picky environment, particularly if a code path begins to access such values. This also reduces the number of strings that need to be translated. - The BRIN function brin_page_items() uses a Relation but it did not check the access method of the opened index, potentially leading to crashes. All the other functions in need of a Relation already did that. - Some code paths could fail on elog(), but we should to use ereport() for failures that can be triggered by the user. Tests are added to stress all the cases that are fixed as of this commit, with some junk raw pages (\set VERBOSITY ensures that this works across all page sizes) and unexpected index types when functions open relations. Author: Michael Paquier, Justin Prysby Discussion: https://postgr.es/m/20220218030020.GA1137@telsasoft.com Backpatch-through: 10 16 March 2022, 02:20:57 UTC
986d240 Thomas Munro 15 March 2022, 22:35:00 UTC Back-patch LLVM 14 API changes. Since LLVM 14 has stopped changing and is about to be released, back-patch the following changes from the master branch: e6a7600202105919bffd62b3dfd941f4a94e082b 807fee1a39de6bb8184082012e643951abb9ad1d a56e7b66010f330782243de9e25ac2a6596be0e1 Back-patch to 11, where LLVM JIT support came in. 15 March 2022, 22:35:00 UTC
789c83b Michael Paquier 09 March 2022, 05:59:30 UTC doc: Add ALTER/DROP ROUTINE to the event trigger matrix ALTER ROUTINE triggers the events ddl_command_start and ddl_command_end, and DROP ROUTINE triggers sql_drop, ddl_command_start and ddl_command_end, but this was not mention on the matrix table. Reported-by: Leslie Lemaire Discussion: https://postgr.es/m/164647533363.646.5802968483136493025@wrigleys.postgresql.org Backpatch-through: 11 09 March 2022, 05:59:30 UTC
49e8a5d Noah Misch 05 March 2022, 02:53:13 UTC Introduce PG_TEST_TIMEOUT_DEFAULT for TAP suite non-elapsing timeouts. Slow hosts may avoid load-induced, spurious failures by setting environment variable PG_TEST_TIMEOUT_DEFAULT to some number of seconds greater than 180. Developers may see faster failures by setting that environment variable to some lesser number of seconds. In tests, write $PostgreSQL::Test::Utils::timeout_default wherever the convention has been to write 180. This change raises the default for some briefer timeouts. Back-patch to v10 (all supported versions). Discussion: https://postgr.es/m/20220218052842.GA3627003@rfd.leadboat.com 05 March 2022, 02:53:17 UTC
566e1c0 Tom Lane 04 March 2022, 00:03:17 UTC Fix bogus casting in BlockIdGetBlockNumber(). This macro cast the result to BlockNumber after shifting, not before, which is the wrong thing. Per the C spec, the uint16 fields would promote to int not unsigned int, so that (for 32-bit int) the shift potentially shifts a nonzero bit into the sign position. I doubt there are any production systems where this would actually end with the wrong answer, but it is undefined behavior per the C spec, and clang's -fsanitize=undefined option reputedly warns about it on some platforms. (I can't reproduce that right now, but the code is undeniably wrong per spec.) It's easy to fix by casting to BlockNumber (uint32) in the proper places. It's been wrong for ages, so back-patch to all supported branches. Report and patch by Zhihong Yu (cosmetic tweaking by me) Discussion: https://postgr.es/m/CALNJ-vT9r0DSsAOw9OXVJFxLENoVS_68kJ5x0p44atoYH+H4dg@mail.gmail.com 04 March 2022, 00:03:50 UTC
f2087e2 Tom Lane 03 March 2022, 23:13:24 UTC Clean up assorted failures under clang's -fsanitize=undefined checks. Most of these are cases where we could call memcpy() or other libc functions with a NULL pointer and a zero count, which is forbidden by POSIX even though every production version of libc allows it. We've fixed such things before in a piecemeal way, but apparently never made an effort to try to get them all. I don't claim that this patch does so either, but it gets every failure I observe in check-world, using clang 12.0.1 on current RHEL8. numeric.c has a different issue that the sanitizer doesn't like: "ln(-1.0)" will compute log10(0) and then try to assign the resulting -Inf to an integer variable. We don't actually use the result in such a case, so there's no live bug. Back-patch to all supported branches, with the idea that we might start running a buildfarm member that tests this case. This includes back-patching c1132aae3 (Check the size in COPY_POINTER_FIELD), which previously silenced some of these issues in copyfuncs.c. Discussion: https://postgr.es/m/CALNJ-vT9r0DSsAOw9OXVJFxLENoVS_68kJ5x0p44atoYH+H4dg@mail.gmail.com 03 March 2022, 23:13:24 UTC
5bb3d91 Tom Lane 02 March 2022, 16:57:02 UTC Allow root-owned SSL private keys in libpq, not only the backend. This change makes libpq apply the same private-key-file ownership and permissions checks that we have used in the backend since commit 9a83564c5. Namely, that the private key can be owned by either the current user or root (with different file permissions allowed in the two cases). This allows system-wide management of key files, which is just as sensible on the client side as the server, particularly when the client is itself some application daemon. Sync the comments about this between libpq and the backend, too. Back-patch of a59c79564 and 50f03473e into all supported branches. David Steele Discussion: https://postgr.es/m/f4b7bc55-97ac-9e69-7398-335e212f7743@pgmasters.net 02 March 2022, 16:57:02 UTC
31befa6 Tom Lane 25 February 2022, 22:40:21 UTC Disallow execution of SPI functions during plperl function compilation. Perl can be convinced to execute user-defined code during compilation of a plperl function (or at least a plperlu function). That's not such a big problem as long as the activity is confined within the Perl interpreter, and it's not clear we could do anything about that anyway. However, if such code tries to use plperl's SPI functions, we have a bigger problem. In the first place, those functions are likely to crash because current_call_data->prodesc isn't set up yet. In the second place, because it isn't set up, we lack critical info such as whether the function is supposed to be read-only. And in the third place, this path allows code execution during function validation, which is strongly discouraged because of the potential for security exploits. Hence, reject execution of the SPI functions until compilation is finished. While here, add check_spi_usage_allowed() calls to various functions that hadn't gotten the memo about checking that. I think that perhaps plperl_sv_to_literal may have been intentionally omitted on the grounds that it was safe at the time; but if so, the addition of transforms functionality changed that. The others are more recently added and seem to be flat-out oversights. Per report from Mark Murawski. Back-patch to all supported branches. Discussion: https://postgr.es/m/9acdf918-7fff-4f40-f750-2ffa84f083d2@intellasoft.net 25 February 2022, 22:40:45 UTC
51c3416 Andres Freund 25 February 2022, 18:40:32 UTC pg_waldump: Fix error message for WAL files smaller than XLOG_BLCKSZ. When opening a WAL file smaller than XLOG_BLCKSZ (e.g. 0 bytes long) while determining the wal_segment_size, pg_waldump checked errno, despite errno not being set by the short read. Resulting in a bogus error message. Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/20220214.181847.775024684568733277.horikyota.ntt@gmail.com Backpatch: 11-, the bug was introducedin fc49e24fa 25 February 2022, 18:40:32 UTC
3faa21b Andres Freund 19 February 2022, 20:27:20 UTC Fix temporary object cleanup failing due to toast access without snapshot. When cleaning up temporary objects during process exit the cleanup could fail with: FATAL: cannot fetch toast data without an active snapshot The bug is caused by RemoveTempRelationsCallback() not setting up a snapshot. If an object with toasted catalog data needs to be cleaned up, init_toast_snapshot() could fail with the above error. Most of the time however the the problem is masked due to cached catalog snapshots being returned by GetOldestSnapshot(). But dropping an object can cause catalog invalidations to be emitted. If no further catalog accesses are necessary between the invalidation processing and the next toast datum deletion, the bug becomes visible. It's easy to miss this bug because it typically happens after clients disconnect and the FATAL error just ends up in the log. Luckily temporary table cleanup at the next use of the same temporary schema or during DISCARD ALL does not have the same problem. Fix the bug by pushing a snapshot in RemoveTempRelationsCallback(). Also add isolation tests for temporary object cleanup, including objects with toasted catalog data. A future HEAD only commit will add more assertions. Reported-By: Miles Delahunty Author: Andres Freund Discussion: https://postgr.es/m/CAOFAq3BU5Mf2TTvu8D9n_ZOoFAeQswuzk7yziAb7xuw_qyw5gw@mail.gmail.com Backpatch: 10- 21 February 2022, 16:59:34 UTC
f1f8230 Andrew Dunstan 20 February 2022, 16:50:54 UTC Remove most msys special processing in TAP tests Following migration of Windows buildfarm members running TAP tests to use of ucrt64 perl for those tests, special processing for msys perl is no longer necessary and so is removed. Backpatch to release 10 Discussion: https://postgr.es/m/c65a8781-77ac-ea95-d185-6db291e1baeb@dunslane.net 20 February 2022, 16:50:54 UTC
b4a6cee Andrew Dunstan 20 February 2022, 13:55:06 UTC Remove PostgreSQL::Test::Utils::perl2host completely Commit f1ac4a74de disabled this processing, and as nothing has broken (as expected) here we proceed to remove the routine and adjust all the call sites. Backpatch to release 10 Discussion: https://postgr.es/m/0ba775a2-8aa0-0d56-d780-69427cf6f33d@dunslane.net Discussion: https://postgr.es/m/20220125023609.5ohu3nslxgoygihl@alap3.anarazel.de 20 February 2022, 15:50:48 UTC
48bf71a Etsuro Fujita 16 February 2022, 06:15:07 UTC Doc: Update documentation for modifying postgres_fdw foreign tables. Document that they can be modified using COPY as well. Back-patch to v11 where commit 3d956d956 added support for COPY in postgres_fdw. 16 February 2022, 06:15:07 UTC
efae440 Tom Lane 15 February 2022, 17:21:28 UTC Improve subscriber's error message for wrong publication relkind. Pre-v13 versions only support logical replication from plain tables, while v13 and later also allow partitioned tables to be published. If you tried to subscribe an older server to such a publication, you got "table XXX not found on publisher", which is pretty unhelpful/confusing. Arrange to deliver a more on-point error message. As commit c314c147c did in v13, remove the relkind check from the query WHERE clause altogether, so that "not there" is distinguishable from "wrong relkind". Per report from Radoslav Nedyalkov. Patch v10-v12. Discussion: https://postgr.es/m/2952568.1644876730@sss.pgh.pa.us 15 February 2022, 17:21:28 UTC
1cd5802 Amit Kapila 14 February 2022, 03:07:23 UTC WAL log unchanged toasted replica identity key attributes. Currently, during UPDATE, the unchanged replica identity key attributes are not logged separately because they are getting logged as part of the new tuple. But if they are stored externally then the untoasted values are not getting logged as part of the new tuple and logical replication won't be able to replicate such UPDATEs. So we need to log such attributes as part of the old_key_tuple during UPDATE. Reported-by: Haiying Tang Author: Dilip Kumar and Amit Kapila Reviewed-by: Alvaro Herrera, Haiying Tang, Andres Freund Backpatch-through: 10 Discussion: https://postgr.es/m/OS0PR01MB611342D0A92D4F4BF26C0F47FB229@OS0PR01MB6113.jpnprd01.prod.outlook.com 14 February 2022, 03:07:23 UTC
0d55477 Alexander Korotkov 14 February 2022, 00:26:55 UTC Fix memory leak in IndexScan node with reordering Fix ExecReScanIndexScan() to free the referenced tuples while emptying the priority queue. Backpatch to all supported versions. Discussion: https://postgr.es/m/CAHqSB9gECMENBQmpbv5rvmT3HTaORmMK3Ukg73DsX5H7EJV7jw%40mail.gmail.com Author: Aliaksandr Kalenik Reviewed-by: Tom Lane, Alexander Korotkov Backpatch-through: 10 14 February 2022, 01:04:19 UTC
14ee565 Tom Lane 11 February 2022, 20:23:52 UTC Don't use_physical_tlist for an IOS with non-returnable columns. createplan.c tries to save a runtime projection step by specifying a scan plan node's output as being exactly the table's columns, or index's columns in the case of an index-only scan, if there is not a reason to do otherwise. This logic did not previously pay attention to whether an index's columns are returnable. That worked, sort of accidentally, until commit 9a3ddeb51 taught setrefs.c to reject plans that try to read a non-returnable column. I have no desire to loosen setrefs.c's new check, so instead adjust use_physical_tlist() to not try to optimize this way when there are non-returnable column(s). Per report from Ryan Kelly. Like the previous patch, back-patch to all supported branches. Discussion: https://postgr.es/m/CAHUie24ddN+pDNw7fkhNrjrwAX=fXXfGZZEHhRuofV_N_ftaSg@mail.gmail.com 11 February 2022, 20:23:52 UTC
69cc15c Tom Lane 10 February 2022, 21:49:39 UTC Make pg_ctl stop/restart/promote recheck postmaster aliveness. "pg_ctl stop/restart" checked that the postmaster PID is valid just once, as a side-effect of sending the stop signal, and then would wait-till-timeout for the postmaster.pid file to go away. This neglects the case wherein the postmaster dies uncleanly after we signal it. Similarly, once "pg_ctl promote" has sent the signal, it'd wait for the corresponding on-disk state change to occur even if the postmaster dies. I'm not sure how we've managed not to notice this problem, but it seems to explain slow execution of the 017_shm.pl test script on AIX since commit 4fdbf9af5, which added a speculative "pg_ctl stop" with the idea of making real sure that the postmaster isn't there. In the test steps that kill-9 and then restart the postmaster, it's possible to get past the initial signal attempt before kill() stops working for the doomed postmaster. If that happens, pg_ctl waited till PGCTLTIMEOUT before giving up ... and the buildfarm's AIX members have that set very high. To fix, include a "kill(pid, 0)" test (similar to what postmaster_is_alive uses) in these wait loops, so that we'll give up immediately if the postmaster PID disappears. While here, I chose to refactor those loops out of where they were. do_stop() and do_restart() can perfectly well share one copy of the wait-for-stop loop, and it seems desirable to put a similar function beside that for wait-for-promote. Back-patch to all supported versions, since pg_ctl's wait logic is substantially identical in all, and we're seeing the slow test behavior in all branches. Discussion: https://postgr.es/m/20220210023537.GA3222837@rfd.leadboat.com 10 February 2022, 21:49:39 UTC
e2d104e Andrew Dunstan 10 February 2022, 18:44:05 UTC Use gendef instead of pexports for building windows .def files Modern msys systems lack pexports but have gendef instead, so use that. Discussion: https://postgr.es/m/3ccde7a9-e4f9-e194-30e0-0936e6ad68ba@dunslane.net Backpatch to release 9.4 to enable building with perl on older branches. Before that pexports is not used for plperl. 10 February 2022, 18:51:59 UTC
2373429 Noah Misch 10 February 2022, 02:16:59 UTC Use Test::Builder::todo_start(), replacing $::TODO. Some pre-2017 Test::More versions need perfect $Test::Builder::Level maintenance to find the variable. Buildfarm member snapper reported an overall failure that the file intended to hide via the TODO construct. That trouble was reachable in v11 and v10. For later branches, this serves as defense in depth. Back-patch to v10 (all supported versions). Discussion: https://postgr.es/m/20220202055556.GB2745933@rfd.leadboat.com 10 February 2022, 02:17:03 UTC
3cc89d9 Noah Misch 10 February 2022, 02:16:56 UTC Fix back-patch of "Avoid race in RelationBuildDesc() ..." The back-patch of commit fdd965d074d46765c295223b119ca437dbcac973 broke CLOBBER_CACHE_ALWAYS for v9.6 through v13. It updated the InvalidateSystemCaches() call for CLOBBER_CACHE_RECURSIVELY, neglecting the one for CLOBBER_CACHE_ALWAYS. Back-patch to v13, v12, v11, and v10. Reviewed by Tomas Vondra. Reported by Tomas Vondra. Discussion: https://postgr.es/m/df7b4c0b-7d92-f03f-75c4-9e08b269a716@enterprisedb.com 10 February 2022, 02:16:59 UTC
3a6e3a8 Tom Lane 09 February 2022, 00:25:56 UTC Remove ppport.h's broken re-implementation of eval_pv(). Recent versions of Devel::PPPort try to redefine eval_pv() to dodge a bug in pre-5.31 Perl versions. Unfortunately the redefinition fails on compilers that don't support statements nested within expressions. However, we aren't actually interested in this bug fix, since we always call eval_pv() with croak_on_error = FALSE. So, until there's an upstream fix for this breakage, just comment out the macro to revert to the older behavior. Per report from Wei Sun, as well as previous buildfarm failure on pademelon (which I'd unfortunately not looked at carefully enough to understand the cause). Back-patch to all supported versions, since we're using the same ppport.h in all. Discussion: https://postgr.es/m/tencent_2EFCC8BA0107B6EC0F97179E019A8A43C806@qq.com Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=pademelon&dt=2022-02-02%2001%3A22%3A58 09 February 2022, 00:26:26 UTC
ff2a6d2 Tom Lane 07 February 2022, 21:20:23 UTC Stamp 11.15. 07 February 2022, 21:20:23 UTC
d793dcf Peter Eisentraut 07 February 2022, 12:51:36 UTC Translation updates Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 250dd3558ab52b3b5f7c9b6d0cfe9ff81d726de2 07 February 2022, 12:51:36 UTC
714343b Tom Lane 06 February 2022, 19:24:55 UTC Release notes for 14.2, 13.6, 12.10, 11.15, 10.20. 06 February 2022, 19:24:55 UTC
9419987 Tom Lane 05 February 2022, 17:55:44 UTC Doc: be clearer that foreign-table partitions need user-added constraints. A very well-informed user might deduce this from what we said already, but I'd bet against it. Lay it out explicitly. While here, rewrite the comment about tuple routing to be more intelligible to an average SQL user. Per bug #17395 from Alexander Lakhin. Back-patch to v11. (The text in this area is different in v10 and I'm not sufficiently excited about this point to adapt the patch.) Discussion: https://postgr.es/m/17395-8c326292078d1a57@postgresql.org 05 February 2022, 17:55:44 UTC
46bf1f2 Tom Lane 05 February 2022, 16:59:30 UTC Test, don't just Assert, that mergejoin's inputs are in order. There are two Asserts in nodeMergejoin.c that are reachable if the input data is not in the expected order. This seems way too fragile. Alexander Lakhin reported a case where the assertions could be triggered with misconfigured foreign-table partitions, and bitter experience with unstable operating system collation definitions suggests another easy route to hitting them. Neither Assert is in a place where we can't afford one more test-and-branch, so replace 'em with plain test-and-elog logic. Per bug #17395. While the reported symptom is relatively recent, collation changes could happen anytime, so back-patch to all supported branches. Discussion: https://postgr.es/m/17395-8c326292078d1a57@postgresql.org 05 February 2022, 16:59:30 UTC
6d511a4 Bruce Momjian 03 February 2022, 02:53:51 UTC doc: clarify syntax notation, particularly parentheses Also move TCL syntax to the PL/tcl section. Reported-by: davs2rt@gmail.com Discussion: https://postgr.es/m/164308146320.12460.3590769444508751574@wrigleys.postgresql.org Backpatch-through: 10 03 February 2022, 02:53:51 UTC
0b25f5e Peter Eisentraut 02 February 2022, 08:14:26 UTC doc: Fix mistake in PL/Python documentation Small thinko introduced by 94aceed317730953476bec490ce0148b2af3c383 Reported-by: nassehk@gmail.com 02 February 2022, 08:16:31 UTC
e41176b Tom Lane 02 February 2022, 00:03:41 UTC Replace use of deprecated Python module distutils.sysconfig, take 2. With Python 3.10, configure spits out warnings about the module distutils.sysconfig being deprecated and scheduled for removal in Python 3.12. Change the uses in configure to use the module sysconfig instead. The logic stays largely the same, although we have to rely on INCLUDEPY instead of the deprecated get_python_inc function. Note that sysconfig exists since Python 2.7, so this moves the minimum required version up from Python 2.6 (or 2.4, before v13). Also, sysconfig didn't exist in Python 3.1, so the minimum 3.x version is now 3.2. Back-patch of commit bd233bdd8 into all supported branches. In v10, this also includes back-patching v11's beff4bb9c, primarily because this opinion is clearly out-of-date: While at it, get rid of the code's assumption that both the major and minor numbers contain exactly one digit. That will foreseeably be broken by Python 3.10 in perhaps four or five years. That's far enough out that we probably don't need to back-patch this. Peter Eisentraut, Tom Lane, Andres Freund Discussion: https://postgr.es/m/c74add3c-09c4-a9dd-1a03-a846e5b2fc52@enterprisedb.com 02 February 2022, 00:03:41 UTC
e368910 Tom Lane 31 January 2022, 20:07:39 UTC Revert "plperl: Fix breakage of c89f409749c in back branches." This reverts commits d81cac47a et al. We shouldn't need that hack after the preceding commits. Discussion: https://postgr.es/m/20220131015130.shn6wr2fzuymerf6@alap3.anarazel.de 31 January 2022, 20:07:39 UTC
923f9f4 Tom Lane 31 January 2022, 20:01:05 UTC plperl: update ppport.h to Perl 5.34.0. Also apply the changes suggested by running perl ppport.h --compat-version=5.8.0 And remove some no-longer-required NEED_foo declarations. Dagfinn Ilmari Mannsåker Back-patch of commit 05798c9f7 into all supported branches. At the time we thought this update was mostly cosmetic, but the lack of it has caused trouble, while the patch itself hasn't. Discussion: https://postgr.es/m/87y278s6iq.fsf@wibble.ilmari.org Discussion: https://postgr.es/m/20220131015130.shn6wr2fzuymerf6@alap3.anarazel.de 31 January 2022, 20:01:05 UTC
ac37448 Andres Freund 31 January 2022, 01:53:53 UTC plperl: Fix breakage of c89f409749c in back branches. ppport.h was only updated in 05798c9f7f0 (master). Unfortunately my commit c89f409749c uses PERL_VERSION_LT which came in with that update. Breaking most buildfarm animals. I should have noticed that... We might want to backpatch the ppport update instead, but for now lets get the buildfarm green again. Discussion: https://postgr.es/m/20220131015130.shn6wr2fzuymerf6@alap3.anarazel.de Backpatch: 10-14, master doesn't need it 31 January 2022, 02:01:15 UTC
ad95a63 Andres Freund 30 January 2022, 22:29:04 UTC plperl: windows: Use Perl_setlocale on 5.28+, fixing compile failure. For older versions we need our own copy of perl's setlocale(), because it was not exposed (why we need the setlocale in the first place is explained in plperl_init_interp) . The copy stopped working in 5.28, as some of the used macros are not public anymore. But Perl_setlocale is available in 5.28, so use that. Author: Victor Wagner <vitus@wagner.pp.ru> Reviewed-By: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> Discussion: https://postgr.es/m/20200501134711.08750c5f@antares.wagner.home Backpatch: all versions 31 January 2022, 00:42:49 UTC
5cb8864 Tomas Vondra 27 January 2022, 16:53:53 UTC Fix ordering of XIDs in ProcArrayApplyRecoveryInfo Commit 8431e296ea reworked ProcArrayApplyRecoveryInfo to sort XIDs before adding them to KnownAssignedXids. But the XIDs are sorted using xidComparator, which compares the XIDs simply as uint32 values, not logically. KnownAssignedXidsAdd() however expects XIDs in logical order, and calls TransactionIdFollowsOrEquals() to enforce that. If there are XIDs for which the two orderings disagree, an error is raised and the recovery fails/restarts. Hitting this issue is fairly easy - you just need two transactions, one started before the 4B limit (e.g. XID 4294967290), the other sometime after it (e.g. XID 1000). Logically (4294967290 <= 1000) but when compared using xidComparator we try to add them in the opposite order. Which makes KnownAssignedXidsAdd() fail with an error like this: ERROR: out-of-order XID insertion in KnownAssignedXids This only happens during replica startup, while processing RUNNING_XACTS records to build the snapshot. Once we reach STANDBY_SNAPSHOT_READY, we skip these records. So this does not affect already running replicas, but if you restart (or create) a replica while there are transactions with XIDs for which the two orderings disagree, you may hit this. Long-running transactions and frequent replica restarts increase the likelihood of hitting this issue. Once the replica gets into this state, it can't be started (even if the old transactions are terminated). Fixed by sorting the XIDs logically - this is fine because we're dealing with normal XIDs (because it's XIDs assigned to backends) and from the same wraparound epoch (otherwise the backends could not be running at the same time on the primary node). So there are no problems with the triangle inequality, which is why xidComparator compares raw values. Investigation and root cause analysis by Abhijit Menon-Sen. Patch by me. This issue is present in all releases since 9.4, however releases up to 9.6 are EOL already so backpatch to 10 only. Reviewed-by: Abhijit Menon-Sen Reviewed-by: Alvaro Herrera Backpatch-through: 10 Discussion: https://postgr.es/m/36b8a501-5d73-277c-4972-f58a4dce088a%40enterprisedb.com 27 January 2022, 19:18:22 UTC
e092f00 Noah Misch 27 January 2022, 02:06:19 UTC On sparc64+ext4, suppress test failures from known WAL read failure. Buildfarm members kittiwake, tadarida and snapper began to fail frequently when commits 3cd9c3b921977272e6650a5efbeade4203c4bca2 and f47ed79cc8a0cfa154dc7f01faaf59822552363f added tests of concurrency, but the problem was reachable before those commits. Back-patch to v10 (all supported versions). Discussion: https://postgr.es/m/20220116210241.GC756210@rfd.leadboat.com 27 January 2022, 02:06:23 UTC
bbb1caf Tom Lane 25 January 2022, 17:17:40 UTC Revert "graceful shutdown" changes for Windows, in back branches only. This reverts commits 6051857fc and ed52c3707, but only in the back branches. Further testing has shown that while those changes do fix some things, they also break others; in particular, it looks like walreceivers fail to detect walsender-initiated connection close reliably if the walsender shuts down this way. We'll keep trying to improve matters in HEAD, but it now seems unwise to push these changes into stable releases. Discussion: https://postgr.es/m/CA+hUKG+OeoETZQ=Qw5Ub5h3tmwQhBmDA=nuNO3KG=zWfUypFAw@mail.gmail.com 25 January 2022, 17:17:40 UTC
acd4f44 Michael Paquier 25 January 2022, 01:49:48 UTC doc: Fix some grammar This is an extraction of the user-visible changes done in 410aa24, including all the relevant documentation parts. Author: Justin Pryzby Discussion: https://postgr.es/m/20220124030001.GQ23027@telsasoft.com Backpatch-through: 10 25 January 2022, 01:49:48 UTC
4ec5449 Tom Lane 24 January 2022, 20:33:34 UTC Fix limitations on what SQL commands can be issued to a walsender. In logical replication mode, a WalSender is supposed to be able to execute any regular SQL command, as well as the special replication commands. Poor design of the replication-command parser caused it to fail in various cases, notably: * semicolons embedded in a command, or multiple SQL commands sent in a single message; * dollar-quoted literals containing odd numbers of single or double quote marks; * commands starting with a comment. The basic problem here is that we're trying to run repl_scanner.l across the entire input string even when it's not a replication command. Since repl_scanner.l does not understand all of the token types known to the core lexer, this is doomed to have failure modes. We certainly don't want to make repl_scanner.l as big as scan.l, so instead rejigger stuff so that we only lex the first token of a non-replication command. That will usually look like an IDENT to repl_scanner.l, though a comment would end up getting reported as a '-' or '/' single-character token. If the token is a replication command keyword, we push it back and proceed normally with repl_gram.y parsing. Otherwise, we can drop out of exec_replication_command() without examining the rest of the string. (It's still theoretically possible for repl_scanner.l to fail on the first token; but that could only happen if it's an unterminated single- or double-quoted string, in which case you'd have gotten largely the same error from the core lexer too.) In this way, repl_gram.y isn't involved at all in handling general SQL commands, so we can get rid of the SQLCmd node type. (In the back branches, we can't remove it because renumbering enum NodeTag would be an ABI break; so just leave it sit there unused.) I failed to resist the temptation to clean up some other sloppy coding in repl_scanner.l while at it. The only externally-visible behavior change from that is it now accepts \r and \f as whitespace, same as the core lexer. Per bug #17379 from Greg Rychlewski. Back-patch to all supported branches. Discussion: https://postgr.es/m/17379-6a5c6cfb3f1f5e77@postgresql.org 24 January 2022, 20:33:34 UTC
449a696 Tom Lane 24 January 2022, 17:09:46 UTC Remember to reset yy_start state when firing up repl_scanner.l. Without this, we get odd behavior when the previous cycle of lexing exited in a non-default exclusive state. Every other copy of this code is aware that it has to do BEGIN(INITIAL), but repl_scanner.l did not get that memo. The real-world impact of this is probably limited, since most replication clients would abandon their connection after getting a syntax error. Still, it's a bug. This mistake is old, so back-patch to all supported branches. Discussion: https://postgr.es/m/1874781.1643035952@sss.pgh.pa.us 24 January 2022, 17:09:46 UTC
bd110e2 Tom Lane 23 January 2022, 16:09:00 UTC Suppress variable-set-but-not-used warning from clang 13. In the normal configuration where GEQO_DEBUG isn't defined, recent clang versions have started to complain that geqo_main.c accumulates the edge_failures count but never does anything with it. As a minimal back-patchable fix, insert a void cast to silence this warning. (I'd speculated about ripping out the GEQO_DEBUG logic altogether, but I don't think we'd wish to back-patch that.) Per recently-established project policy, this is a candidate for back-patching into out-of-support branches: it suppresses an annoying compiler warning but changes no behavior. Hence, back-patch all the way to 9.2. Discussion: https://postgr.es/m/CA+hUKGLTSZQwES8VNPmWO9AO0wSeLt36OCPDAZTccT1h7Q7kTQ@mail.gmail.com 23 January 2022, 16:09:38 UTC
5e9fe25 Tomas Vondra 23 January 2022, 02:36:55 UTC Correct type of front_pathkey to PathKey In sort_inner_and_outer we iterate a list of PathKey elements, but the variable is declared as (List *). This mistake is benign, because we only pass the pointer to lcons() and never dereference it. This exists since ~2004, but it's confusing. So fix and backpatch to all supported branches. Backpatch-through: 10 Discussion: https://postgr.es/m/bf3a6ea1-a7d8-7211-0669-189d5c169374%40enterprisedb.com 23 January 2022, 02:56:44 UTC
26c841e Tom Lane 22 January 2022, 18:32:40 UTC Flush table's relcache during ALTER TABLE ADD PRIMARY KEY USING INDEX. Previously, unless we had to add a NOT NULL constraint to the column, this command resulted in updating only the index's relcache entry. That's problematic when replication behavior is being driven off the existence of a primary key: other sessions (and ours too for that matter) failed to recalculate their opinion of whether the table can be replicated. Add a relcache invalidation to fix it. This has been broken since pg_class.relhaspkey was removed in v11. Before that, updating the table's relhaspkey value sufficed to cause a cache flush. Hence, backpatch to v11. Report and patch by Hou Zhijie Discussion: https://postgr.es/m/OS0PR01MB5716EBE01F112C62F8F9B786947B9@OS0PR01MB5716.jpnprd01.prod.outlook.com 22 January 2022, 18:32:40 UTC
37f5dc8 Tom Lane 21 January 2022, 20:36:12 UTC Fix race condition in gettext() initialization in libpq and ecpglib. In libpq and ecpglib, multiple threads can concurrently enter the initialization logic for message localization. Since we set the its-done flag before actually doing the work, it'd be possible for some threads to reach gettext() before anyone has called bindtextdomain(). Barring bugs in libintl itself, this would not result in anything worse than failure to localize some early messages. Nonetheless, it's a bug, and an easy one to fix. Noted while investigating bug #17299 from Clemens Zeidler (much thanks to Liam Bowen for followup investigation on that). It currently appears that that actually *is* a bug in libintl itself, but that doesn't let us off the hook for this bit. Back-patch to all supported versions. Discussion: https://postgr.es/m/17299-7270741958c0b1ab@postgresql.org Discussion: https://postgr.es/m/CAE7q7Eit4Eq2=bxce=Fm8HAStECjaXUE=WBQc-sDDcgJQ7s7eg@mail.gmail.com 21 January 2022, 20:36:29 UTC
back to top