Skip to main content

Browse the archive

https://github.com/postgres/postgres
07 April 2024, 13:56:23 UTC
sort by:
Revision Author Date Message Commit Date
118558e Tom Lane 04 April 2024, 19:31:53 UTC Fix ecpg's mechanism for detecting unsupported cases in the grammar. ecpg wants to emit a warning if it parses a SQL construct that the backend can parse but will immediately throw a FEATURE_NOT_SUPPORTED error for. The way it was testing for this was to see if the string ERRCODE_FEATURE_NOT_SUPPORTED appeared anywhere in the gram.y code. This is, of course, not nearly good enough, as there are plenty of rules in gram.y that throw that error only conditionally. There was a hack dating to 2008 to suppress the warning in one rule that doesn't even exist anymore, but nothing for other cases we've created since then. End result was that you could get "unsupported feature will be passed to server" warnings while compiling perfectly good SQL code in ecpg. Somehow we'd not heard complaints about this, but it was exposed by the recent addition of an ecpg test for a SQL/JSON construct. To fix, suppress the warning if the rule contains any "if" statement. Manual comparison of gram.y with the generated preproc.y file shows that the warning is now emitted only in rules where it's sensible. This problem has existed for a long time, so back-patch to all supported branches. Discussion: https://postgr.es/m/603615.1712245382@sss.pgh.pa.us 04 April 2024, 19:31:53 UTC
f6f61a4 Etsuro Fujita 04 April 2024, 08:25:00 UTC Fix bogus coding in ExecAppendAsyncEventWait(). No configured-by-FDW events would result in "return" directly out of a PG_TRY block, making the exception stack dangling. Repair. Oversight in commit 501cfd07d; back-patch to v14, like that commit, but as we do not have this issue in HEAD (cf. commit 50c67c201), no need to apply this patch to it. In passing, improve a comment about the handling of in-process requests in a postgres_fdw.c function called from this function. Alexander Pyhalov, with comment adjustment/improvement by me. Discussion: https://postgr.es/m/425fa29a429b21b0332737c42a4fdc70%40postgrespro.ru 04 April 2024, 08:25:00 UTC
bafad43 Alexander Korotkov 03 April 2024, 18:29:18 UTC Fix the parameters order for TableAmRoutine.relation_copy_for_cluster() Specify OldTable first, NewTable second as used by table_relation_copy_for_cluster() and as implemented in heapam_relation_copy_for_cluster(). Backpatch to PostgreSQL 12, where TableAmRoutine was introduced. Discussion: https://postgr.es/m/ME3P282MB3166860D4911AE82F92DF7C5B63F2%40ME3P282MB3166.AUSP282.PROD.OUTLOOK.COM Author: Japin Li Reviewed-by: Pavel Borisov Backpatch-through: 12 03 April 2024, 18:42:38 UTC
cbfbb14 Tom Lane 02 April 2024, 18:59:04 UTC Avoid deadlock during orphan temp table removal. If temp tables have dependencies (such as sequences) then it's possible for autovacuum's cleanup of orphan temp tables to deadlock against an incoming backend that's trying to clean out the temp namespace for its own use. That can happen because RemoveTempRelations' performDeletion call can visit objects within the namespace in an order different from the order in which a per-table deletion will visit them. To fix, observe that performDeletion will begin by taking an exclusive lock on the temp namespace (even though it won't actually delete it). So, if we can get a shared lock on the namespace, we can be sure we're not running concurrently with RemoveTempRelations, while also not conflicting with ordinary use of the namespace. This requires introducing a conditional version of LockDatabaseObject, but that's no big deal. (It's surprising we've got along without that this long.) Report and patch by Mikhail Zhilin. Back-patch to all supported branches. Discussion: https://postgr.es/m/c43ce028-2bc2-4865-9b89-3f706246eed5@postgrespro.ru 02 April 2024, 18:59:04 UTC
3235a11 Tom Lane 01 April 2024, 23:01:18 UTC Avoid "unused variable" warning on non-USE_SSL_ENGINE platforms. If we are building with openssl but USE_SSL_ENGINE didn't get set, initialize_SSL's variable "pkey" is declared but used nowhere. Apparently this combination hasn't been exercised in the buildfarm before now, because I've not seen this warning before, even though the code has been like this a long time. Move the declaration to silence the warning (and remove its useless initialization). Per buildfarm member sawshark. Back-patch to all supported branches. 01 April 2024, 23:01:18 UTC
14e4e8f Tom Lane 01 April 2024, 19:15:03 UTC Avoid possible longjmp-induced logic error in PLy_trigger_build_args. The "pltargs" variable wasn't marked volatile, which makes it unsafe to change its value within the PG_TRY block. It looks like the worst outcome would be to fail to release a refcount on Py_None during an (improbable) error exit, which would likely go unnoticed in the field. Still, it's a bug. A one-liner fix could be to mark pltargs volatile, but on the whole it seems cleaner to arrange things so that we don't change its value within PG_TRY. Per report from Xing Guo. This has been there for quite awhile, so back-patch to all supported branches. Discussion: https://postgr.es/m/CACpMh+DLrk=fDv07MNpBT4J413fDAm+gmMXgi8cjPONE+jvzuw@mail.gmail.com 01 April 2024, 19:15:03 UTC
ca4c60e Bruce Momjian 27 March 2024, 21:58:10 UTC doc: fix CREATE ROLE typo This wording typo was added in PG 16. Reported-by: s.bailey@chorusintel.com Discussion: https://postgr.es/m/171150077554.7105.801523271545956671@wrigleys.postgresql.org Backpatch-through: 16 27 March 2024, 21:58:10 UTC
a94f51a Tom Lane 27 March 2024, 17:39:03 UTC Fix unnecessary use of moving-aggregate mode with non-moving frame. When a plain aggregate is used as a window function, and the window frame start is specified as UNBOUNDED PRECEDING, the frame's head cannot move so we do not need to use moving-aggregate mode. The check for that was put into initialize_peragg(), failing to notice that ExecInitWindowAgg() calls that function before it's filled in winstate->frameOptions. Since makeNode() would have zeroed the field, this didn't provoke uninitialized-value complaints, nor would the erroneous decision have resulted in more than a little inefficiency. Still, it's wrong, so move the initialization of winstate->frameOptions earlier to make it work properly. While here, also fix a thinko in a comment. Both errors crept in in commit a9d9acbf2 which introduced the moving-aggregate mode. Spotted by Vallimaharajan G. Back-patch to all supported branches. Discussion: https://postgr.es/m/18e7f2a5167.fe36253866818.977923893562469143@zohocorp.com 27 March 2024, 17:39:03 UTC
f5d0e86 David Rowley 27 March 2024, 11:14:17 UTC Fix unstable aggregate regression test Buildfarm member avocet has shown a plan change by switching the finalize aggregate stage to use a GroupAggregate rather than a HashAggregate. This is consistent with autovacuum having triggered on the table, per analysis by Alexander Lakhin. Fix this by disabling autovacuum on the table. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/d4493a28-589a-5328-fed5-250f2d7d3e2a@gmail.com Backpatch-through: 16, where this test was added. 27 March 2024, 11:14:17 UTC
7445f09 Tom Lane 26 March 2024, 19:28:16 UTC Fix failure of ALTER FOREIGN TABLE SET SCHEMA to move sequences. Ordinary ALTER TABLE SET SCHEMA will also move any owned sequences into the new schema. We failed to do likewise for foreign tables, because AlterTableNamespaceInternal believed that only certain relkinds could have indexes, owned sequences, or constraints. We could simply add foreign tables to that relkind list, but it seems likely that the same oversight could be made again in future. Instead let's remove the relkind filter altogether. These functions shouldn't cost much when there are no objects that they need to process, and surely this isn't an especially performance-critical case anyway. Per bug #18407 from Vidushi Gupta. Back-patch to all supported branches. Discussion: https://postgr.es/m/18407-4fd07373d252c6a0@postgresql.org 26 March 2024, 19:28:16 UTC
7651fd3 Tom Lane 26 March 2024, 15:44:49 UTC Allow "make check"-style testing to work with musl C library. The musl dynamic linker saves a pointer to the process' environment value of LD_LIBRARY_PATH very early in startup. When we move/clobber the environment to make more room for ps status strings, we clobber that value and thereby prevent libraries from being found via LD_LIBRARY_PATH, which breaks the use of a temporary installation for testing purposes. To fix, stop collecting usable space for ps status if we notice that the variable we are about to clobber is LD_LIBRARY_PATH. This will result in some reduction in how long the ps status can be, but it's only likely to occur in temporary test contexts, so it doesn't seem like a big problem. In any case, we don't have to do it if we see we are on glibc, which surely is where the majority of our Linux testing is done. Thomas Munro, Bruce Momjian, and Tom Lane, per report from Wolfgang Walther. Back-patch to all supported branches, with the hope that we'll set up a buildfarm animal to test on this platform. Discussion: https://postgr.es/m/fddd1cd6-dc16-40a2-9eb5-d7fef2101488@technowledgy.de 26 March 2024, 15:44:49 UTC
408209d Andres Freund 26 March 2024, 02:58:10 UTC Avoid edge case in pg_visibility test with small shared_buffers Since 82a4edabd27 we can bulk extend relations. The bulk relation extension logic has a heuristic component. Normally the heurstic does not trigger in the occasionally-failing test case, as the relation is only extended once. But with very small shared_buffers the limits for the number of buffers pinned at once prevent the extension from happening at once. With the second "bulk" extension, the heuristic kicks in, and the relation ends up one block bigger. That's ok from a correctness perspective, but changes the results of the test query due to one additional block. We discussed a few more expansive fixes, but for now have decided to avoid this by making the table a bit smaller. Author: Heikki Linnakangas <hlinnaka@iki.fi> Reported-by: Discussion: https://postgr.es/m/29c74104-210b-ef39-2522-27a6aa7a704f@iki.fi Discussion: https://postgr.es/m/20230916000011.2ugpkkkp7bpp4cfh@awork3.anarazel.de Backpatch: 16-, where the new relation extension logic was added 26 March 2024, 03:20:34 UTC
90677b5 Andres Freund 25 March 2024, 20:06:58 UTC ci: macos: Choose python version The CI base image used to have a python3 with headers etc installed in PATH, but doesn't anymore. Instead of relying on a specific version in the base image, explicitly install one ourselves. On 16 and HEAD this lead to a build without python support, but on 15 CI failed, due to explicitly enabled python3 support. 25 March 2024, 20:14:28 UTC
022068e Jeff Davis 25 March 2024, 18:51:44 UTC Clarify comment for LogicalTapeSetBlocks(). Discussion: https://postgr.es/m/1229327.1711160246@sss.pgh.pa.us Backpatch-through: 13 25 March 2024, 19:00:30 UTC
d82dc79 Nathan Bossart 24 March 2024, 20:24:09 UTC doc: Clarify requirements for SET ROLE. Since commit 3d14e171e9, SET ROLE has required the current session user to have membership with the SET option in the target role, but the SET ROLE documentation only mentions the membership requirement. This commit adds this important detail to the SET ROLE page. Reviewed-by: Robert Haas Discussion: https://postgr.es/m/CA%2BRLCQysHtME0znk2KUMJN343ksboSRQSU-hCnOjesX6VK300Q%40mail.gmail.com Backpatch-through: 16 24 March 2024, 20:24:09 UTC
3676b84 Alexander Korotkov 23 March 2024, 21:00:06 UTC amcheck: Normalize index tuples containing uncompressed varlena It might happen that the varlena value wasn't compressed by index_form_tuple() due to current storage parameters. If compression is currently enabled, we need to compress such values to match index tuple coming from the heap. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Andrey Borodin Reviewed-by: Alexander Lakhin, Michael Zhilin, Jian He, Alexander Korotkov Backpatch-through: 12 23 March 2024, 21:02:30 UTC
a6ddb8a Alexander Korotkov 23 March 2024, 20:59:56 UTC amcheck: Support for different header sizes of short varlena datum In the heap, tuples may contain short varlena datum with both 1B header and 4B headers. But the corresponding index tuple should always have such varlena's with 1B headers. So, for fingerprinting, we need to convert. Backpatch to all supported versions. Discussion: https://postgr.es/m/flat/7bdbe559-d61a-4ae4-a6e1-48abdf3024cc%40postgrespro.ru Author: Michael Zhilin Reviewed-by: Alexander Lakhin, Andrey Borodin, Jian He, Alexander Korotkov Backpatch-through: 12 23 March 2024, 21:02:30 UTC
14e991d Tom Lane 22 March 2024, 21:13:53 UTC Use a hash table for catcache.c's CatCList objects. Up to now, all of the "catcache list" objects within a catalog cache were just chained together on a single dlist, requiring O(N) time to search. Remarkably, we've not had serious performance problems with that so far; but we got a complaint of a bad performance regression from v15 in a case with a large number of roles in the system, which traced down to O(N^2) total time when we probed N catcache lists. Replace that data structure with a hashtable having an enlargeable number of dlists, in an exactly parallel way to the data structure we've used for years for the plain CatCTup cache members. The extra cost of maintaining a hash table seems negligible, since we were already computing a hash value for list searches. Normally this'd be HEAD-only material, but in view of the performance regression it seems advisable to back-patch into v16. In the v16 version of the patch, leave the dead cc_lists field where it is and add the new fields at the end of struct catcache, to avoid possible ABI breakage in case any external code is looking at these structs. (We assume no external code is actually allocating new catcache structs.) Per report from alex work. Discussion: https://postgr.es/m/CAGvXd3OSMbJQwOSc-Tq-Ro1CAz=vggErdSG7pv2s6vmmTOLJSg@mail.gmail.com 22 March 2024, 21:13:53 UTC
5863bac Daniel Gustafsson 21 March 2024, 22:31:57 UTC Fix dumping role comments when using --no-role-passwords Commit 9a83d56b38c added support for allowing pg_dumpall to dump roles without including passwords, which accidentally made dumps omit COMMENTs on roles. This fixes it by using pg_authid to get the comment. Backpatch to all supported versions. Patch simultaneously written independently by Álvaro and myself. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Author: Daniel Gustafsson <daniel@yesql.se> Reported-by: Bartosz Chroł <bartosz.chrol@handen.pl> Discussion: https://postgr.es/m/AS8P194MB1271CDA0ADCA7B75FCD8E767F7332@AS8P194MB1271.EURP194.PROD.OUTLOOK.COM Discussion: https://postgr.es/m/CAEP4nAz9V4H41_4ESJd1Gf0v%3DdevkqO1%3Dpo91jUw-GJSx8Hxqg%40mail.gmail.com Backpatch-through: v12 21 March 2024, 22:31:57 UTC
e6c4e01 Alvaro Herrera 20 March 2024, 14:28:14 UTC Review wording on tablespaces w.r.t. partitioned tables Remove a redundant comment, and document pg_class.reltablespace properly in catalogs.sgml. After commits a36c84c3e4a9, 87259588d0ab and others. Backpatch to 12. Discussion: https://postgr.es/m/202403191013.w2kr7wqlamqz@alvherre.pgsql 20 March 2024, 14:28:14 UTC
1f4eb73 Heikki Linnakangas 18 March 2024, 12:04:13 UTC Fix EXPLAIN Bitmap heap scan to count pages with no visible tuples Previously, bitmap heap scans only counted lossy and exact pages for explain when there was at least one visible tuple on the page. heapam_scan_bitmap_next_block() returned true only if there was a "valid" page with tuples to be processed. However, the lossy and exact page counters in EXPLAIN should count the number of pages represented in a lossy or non-lossy way in the constructed bitmap, regardless of whether or not the pages ultimately contained visible tuples. Backpatch to all supported versions. Author: Melanie Plageman Discussion: https://www.postgresql.org/message-id/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA@mail.gmail.com 18 March 2024, 12:04:13 UTC
34c854b Dean Rasheed 17 March 2024, 10:19:31 UTC Fix EXPLAIN output for subplans in MERGE. Given a subplan in a MERGE query, EXPLAIN would sometimes fail to properly display expressions involving Params referencing variables in other parts of the plan tree. This would affect subplans outside the topmost join plan node, for which expansion of Params would go via the top-level ModifyTable plan node. The problem was that "inner_tlist" for the ModifyTable node's deparse_namespace was set to the join node's targetlist, but "inner_plan" was set to the ModifyTable node itself, rather than the join node, leading to incorrect results when descending to the referenced variable. Fix and backpatch to v15, where MERGE was introduced. Discussion: https://postgr.es/m/CAEZATCWAv-sZuH%2BwG5xJ-%2BGt7qGNGX8wUQd3XYydMFDKgRB9nw%40mail.gmail.com 17 March 2024, 10:19:31 UTC
5643262 Daniel Gustafsson 15 March 2024, 16:02:07 UTC Fix handling of expecteddir in pg_regress Commit c855872074b introduced a new parameter to pg_regress to set the directory where to look for expected files, but accidentally only implemented it for when compiling pg_regress for ECPG tests. Fix by adding support for the parameter to the main regression test compilation of pg_regress as well. Backpatch to v16 where --expecteddir was introduced. Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Discussion: https://postgr.es/m/CAO6_Xqq5yKJHcJsq__LPcKwSY0XHRqVERNWGxx5ttNXXj7+W=A@mail.gmail.com Backpatch-through: 16 15 March 2024, 16:02:07 UTC
4e1ff2a David Rowley 14 March 2024, 22:55:50 UTC Trim ORDER BY/DISTINCT aggregate pathkeys in gather_grouping_paths Similar to d8a295389, trim off any PathKeys which are for ORDER BY / DISTINCT aggregate functions from the PathKey List for the Gather Merge paths created by gather_grouping_paths(). These additional PathKeys are not valid to use after grouping has taken place as these PathKeys belong to columns which are inputs to an aggregate function and, therefore are unavailable after aggregation. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/cf63174c-8c89-3953-cb49-48f41f74941a@gmail.com Backpatch-through: 16, where 1349d2790 was added 14 March 2024, 22:55:50 UTC
52898c6 Tom Lane 14 March 2024, 18:57:16 UTC Make INSERT-from-multiple-VALUES-rows handle domain target columns. Commit a3c7a993d fixed some cases involving target columns that are arrays or composites by applying transformAssignedExpr to the VALUES entries, and then stripping off any assignment ArrayRefs or FieldStores that the transformation added. But I forgot about domains over arrays or composites :-(. Such cases would either fail with surprising complaints about mismatched datatypes, or insert unexpected coercions that could lead to odd results. To fix, extend the stripping logic to get rid of CoerceToDomain if it's atop an ArrayRef or FieldStore. While poking at this, I realized that there's a poorly documented and not-at-all-tested behavior nearby: we coerce each VALUES column to the domain type separately, and rely on the rewriter to merge those operations so that the domain constraints are checked only once. If that merging did not happen, it's entirely possible that we'd get unexpected domain constraint failures due to checking a partially-updated container value. There's no bug there, but while we're here let's improve the commentary about it and add some test cases that explicitly exercise that behavior. Per bug #18393 from Pablo Kharo. Back-patch to all supported branches. Discussion: https://postgr.es/m/18393-65fedb1a0de9260d@postgresql.org 14 March 2024, 18:57:16 UTC
d4c573d Dean Rasheed 13 March 2024, 13:15:07 UTC doc: Improve a couple of places in the MERGE docs. In the synopsis, make the syntax for merge_update consistent with the syntax for a plain UPDATE command. It was missing the optional "ROW" keyword that can be used in a multi-column assignment, and the option to assign from a multi-column subquery, both of which have been supported by MERGE since it was introduced. In the parameters section for the with_query parameter, mention that WITH RECURSIVE isn't supported, since this is different from plain INSERT, UPDATE, and DELETE commands. While at it, move that entry to the top of the list, for consistency with the other pages. Back-patch to v15, where MERGE was introduced. Discussion: https://postgr.es/m/CAEZATCWoQyWkMFfu7JXXQr8dA6%3DgxjhYzgpuBP2oz0QoJTxGWw%40mail.gmail.com 13 March 2024, 13:15:07 UTC
9158e4b Andres Freund 13 March 2024, 08:40:48 UTC meson: macos: Avoid warnings on Sonoma Starting with the Sonoma toolchain macos' linker emits warnings when the same library is linked to twice. That's ill considered, as the same library can be used by multiple subsidiary libraries. Luckily there's a flag to suppress that warning. On Ventura meson's default of -Wl,-undefined,dynamic_lookup caused warnings, which we suppressed with -Wl,-undefined,error. Unfortunately that causes a warning on Sonoma, which is absurd, as it's documented linker default. To avoid that warning, only add -Wl,-undefined,error if it does not trigger warnings. Luckily dynamic_lookup doesn't trigger a warning on Sonoma anymore. Discussion: https://postgr.es/m/20231201040515.p5bshhhtfru7d3da@awork3.anarazel.de Backpatch: 16-, where the meson build was added 13 March 2024, 08:40:48 UTC
40d1bde Tom Lane 12 March 2024, 22:16:10 UTC Fix confusion about the return rowtype of SQL-language procedures. There is a very ancient hack in check_sql_fn_retval that allows a single SELECT targetlist entry of composite type to be taken as supplying all the output columns of a function returning composite. (This is grotty and fundamentally ambiguous, but it's really hard to do nested composite-returning functions without it.) As far as I know, that doesn't cause any problems in ordinary functions. It's disastrous for procedures however. All procedures that have any output parameters are labeled with prorettype RECORD, and the CALL code expects it will get back a record with one column per output parameter, regardless of whether any of those parameters is composite. Doing something else leads to an assertion failure or core dump. This is simple enough to fix: we just need to not apply that rule when considering procedures. However, that requires adding another argument to check_sql_fn_retval, which at least in principle might be getting called by external callers. Therefore, in the back branches convert check_sql_fn_retval into an ABI-preserving wrapper around a new function check_sql_fn_retval_ext. Per report from Yahor Yuzefovich. This has been broken since we implemented procedures, so back-patch to all supported branches. Discussion: https://postgr.es/m/CABz5gWHSjj2df6uG0NRiDhZ_Uz=Y8t0FJP-_SVSsRsnrQT76Gg@mail.gmail.com 12 March 2024, 22:16:10 UTC
539e328 Heikki Linnakangas 12 March 2024, 08:18:32 UTC Disconnect if socket cannot be put into non-blocking mode Commit 387da18874 moved the code to put socket into non-blocking mode from socket_set_nonblocking() into the one-time initialization function, pq_init(). In socket_set_nonblocking(), there indeed was a risk of recursion on failure like the comment said, but in pq_init(), ERROR or FATAL is fine. There's even another elog(FATAL) just after this, if setting FD_CLOEXEC fails. Note that COMMERROR merely logged the error, it did not close the connection, so if putting the socket to non-blocking mode failed we would use the connection anyway. You might not immediately notice, because most socket operations in a regular backend wait for the socket to become readable/writable anyway. But e.g. replication will be quite broken. Backpatch to all supported versions. Discussion: https://www.postgresql.org/message-id/d40a5cd0-2722-40c5-8755-12e9e811fa3c@iki.fi 12 March 2024, 08:18:44 UTC
0d795a1 Bruce Momjian 11 March 2024, 17:31:13 UTC doc: add missing word "the" Reported-by: doughale@gmail.com Discussion: https://postgr.es/m/170993253510.640.5664117187431542912@wrigleys.postgresql.org Backpatch-through: 12 11 March 2024, 17:31:13 UTC
407cb6c Heikki Linnakangas 11 March 2024, 07:28:09 UTC Set all_visible_according_to_vm correctly with DISABLE_PAGE_SKIPPING It's important for 'all_visible_according_to_vm' to correctly reflect whether the VM bit is set or not, even when we are not trusting the VM to skip pages, because contrary to what the comment said, lazy_scan_prune() relies on it. If it's incorrectly set to 'false', when the VM bit is in fact set, lazy_scan_prune() will try to set the VM bit again and dirty the page unnecessarily. As a result, if you used DISABLE_PAGE_SKIPPING, all heap pages were dirtied, even if there were no changes. We would also fail to clear any VM bits that were set incorrectly. This was broken in commit 980ae17310, so backpatch to v16. Backpatch-through: 16 Reviewed-by: Melanie Plageman, Peter Geoghegan Discussion: https://www.postgresql.org/message-id/3df2b582-dc1c-46b6-99b6-38eddd1b2784@iki.fi 11 March 2024, 07:28:21 UTC
348233c David Rowley 11 March 2024, 05:20:39 UTC Fix incorrect accessing of pfree'd memory in Memoize For pass-by-reference types, the code added in 0b053e78b, which aimed to resolve a memory leak, was overly aggressive in resetting the per-tuple memory context which could result in pfree'd memory being accessed resulting in failing to find previously cached results in the hash table. What was happening was prepare_probe_slot() was switching to the per-tuple memory context and calling ExecEvalExpr(). ExecEvalExpr() may have required a memory allocation. Both MemoizeHash_hash() and MemoizeHash_equal() were aggressively resetting the per-tuple context and after determining the hash value, the context would have gotten reset before MemoizeHash_equal() was called. This could have resulted in MemoizeHash_equal() looking at pfree'd memory. This is less likely to have caused issues on a production build as some other allocation would have had to have reused the pfree'd memory to overwrite it. Otherwise, the original contents would have been intact. However, this clearly caused issues on MEMORY_CONTEXT_CHECKING builds. Author: Tender Wang, Andrei Lepikhov Reported-by: Tender Wang (using SQLancer) Reviewed-by: Andrei Lepikhov, Richard Guo, David Rowley Discussion: https://postgr.es/m/CAHewXNnT6N6UJkya0z-jLFzVxcwGfeRQSfhiwA+NyLg-x8iGew@mail.gmail.com Backpatch-through: 14, where Memoize was added 11 March 2024, 05:20:39 UTC
7607671 Alexander Korotkov 16 February 2024, 14:02:00 UTC Backpatch missing check_stack_depth() to some recursive functions Backpatch changes from d57b7cc333, 75bcba6cbd to all supported branches per proposal of Egor Chindyaskin. Discussion: https://postgr.es/m/DE5FD776-A8CD-4378-BCFA-3BF30F1F6D60%40mail.ru 11 March 2024, 01:06:55 UTC
6a9e2cb David Rowley 10 March 2024, 23:27:46 UTC Fix deparsing of Consts in postgres_fdw ORDER BY For UNION ALL queries where a union child query contained a foreign table, if the targetlist of that query contained a constant, and the top-level query performed an ORDER BY which contained the column for the constant value, then postgres_fdw would find the EquivalenceMember with the Const and then try to produce an ORDER BY containing that Const. This caused problems with INT typed Consts as these could appear to be requests to order by an ordinal column position rather than the constant value. This could lead to either an error such as: ERROR: ORDER BY position <int const> is not in select list or worse, if the constant value is a valid column, then we could just sort by the wrong column altogether. Here we fix this issue by just not including these Consts in the ORDER BY clause. In passing, add a new section for testing ORDER BY in the postgres_fdw tests and move two existing tests which were misplaced in the WHERE clause testing section into it. Reported-by: Michał Kłeczek Reviewed-by: Ashutosh Bapat, Richard Guo Bug: #18381 Discussion: https://postgr.es/m/0714C8B8-8D82-4ABB-9F8D-A0C3657E7B6E%40kleczek.org Discussion: https://postgr.es/m/18381-137456acd168bf93%40postgresql.org Backpatch-through: 12, oldest supported version 10 March 2024, 23:27:46 UTC
6a2c80e Tom Lane 08 March 2024, 00:37:51 UTC Cope with a deficiency in OpenSSL 3.x's error reporting. In OpenSSL 3.0.0 and later, ERR_reason_error_string randomly refuses to provide a string for error codes representing system errno values (e.g., "No such file or directory"). There is a poorly-documented way to extract the errno from the SSL error code in this case, so do that and apply strerror, rather than falling back to reporting the error code's numeric value as we were previously doing. Problem reported by David Zhang, although this is not his proposed patch; it's instead based on a suggestion from Heikki Linnakangas. Back-patch to all supported branches, since any of them are likely to be used with recent OpenSSL. Discussion: https://postgr.es/m/b6fb018b-f05c-4afd-abd3-318c649faf18@highgo.ca 08 March 2024, 00:37:51 UTC
dd73d10 Dean Rasheed 07 March 2024, 09:55:39 UTC Fix handling of self-modified tuples in MERGE. When an UPDATE or DELETE action in MERGE returns TM_SelfModified, there are 2 possible causes: 1). The target tuple was already updated or deleted by the current command. This can happen if the target row joins to more than one source row, and the SQL standard explicitly says that this must be an error. 2). The target tuple was already updated or deleted by a later command in the current transaction. This can happen if the tuple is modified by a BEFORE trigger or a volatile function used in the query, and should be an error for the same reason that it is in a plain UPDATE or DELETE command. In MERGE's primary error handling block, it failed to check for (2), causing it to return a misleading error message in such cases. In the secondary error handling block, following a concurrent update from another session, it failed to check for (1), causing it to silently ignore target rows joined to more than one source row, instead of reporting an error. Fix this, and add tests for both of these cases. Per report from Wenjiang Zhang. Back-patch to v15, where MERGE was introduced. Discussion: https://postgr.es/m/tencent_41DE0FF443FE14B94A5898D373792109E408%40qq.com 07 March 2024, 09:55:39 UTC
c46817e Michael Paquier 06 March 2024, 23:31:00 UTC Revert "Fix parallel-safety check of expressions and predicate for index builds" This reverts commit eae7be600be7, following a discussion with Tom Lane, due to concerns that this impacts the decisions made by the planner for the number of workers spawned based on the inlining and const-folding of index expressions and predicate for cases that would have worked until this commit. Discussion: https://postgr.es/m/162802.1709746091@sss.pgh.pa.us Backpatch-through: 12 06 March 2024, 23:31:00 UTC
1b3029b Tom Lane 06 March 2024, 19:41:13 UTC Fix type-checking of RECORD-returning functions in FROM. In the corner case where a function returning RECORD has been simplified to a RECORD constant or an inlined ROW() expression, ExecInitFunctionScan failed to cross-check the function's result rowtype against the coldeflist provided by the calling query. That happened because get_expr_result_type is able to extract a tupdesc from such expressions, which led ExecInitFunctionScan to ignore the coldeflist. (Instead, it used the extracted tupdesc to check the function's output, which of course always succeeds.) I have not been able to demonstrate any really serious consequences from this, because if some column of the result is of the wrong type and is directly referenced by a Var of the calling query, CheckVarSlotCompatibility will catch it. However, we definitely do fail to report the case where the function returns more columns than the coldeflist expects, and in the converse case where it returns fewer columns, we get an assert failure (but, seemingly, no worse results in non-assert builds). To fix, always build the expected tupdesc from the coldeflist if there is one, and consult get_expr_result_type only when there isn't one. Also remove the failing Assert, even though it is no longer reached after this fix. It doesn't seem to be adding anything useful, since later checking will deal with cases with the wrong number of columns. The only other place I could find that is doing something similar is inline_set_returning_function. There's no live bug there because we cannot be looking at a Const or RowExpr, but for consistency change that code to agree with ExecInitFunctionScan. Per report from PetSerAl. After some debate I've concluded that this should be back-patched. There is a small risk that somebody has been relying on such a case not throwing an error, but I judge this outweighed by the risk that I've missed some way in which the failure to cross-check has worse consequences than sketched above. Discussion: https://postgr.es/m/CAKygsHSerA1eXsJHR9wft3Gn3wfHQ5RfP8XHBzF70_qcrrRvEg@mail.gmail.com 06 March 2024, 19:41:13 UTC
4ec8f77 Michael Paquier 06 March 2024, 08:24:05 UTC Fix parallel-safety check of expressions and predicate for index builds As coded, the planner logic that calculates the number of parallel workers to use for a parallel index build uses expressions and predicates from the relcache, which are flattened for the planner by eval_const_expressions(). As reported in the bug, an immutable parallel-unsafe function flattened in the relcache would become a Const, which would be considered as parallel-safe, even if the predicate or the expressions including the function are not safe in parallel workers. Depending on the expressions or predicate used, this could cause the parallel build to fail. Tests are included that check parallel index builds with parallel-unsafe predicate and expressions. Two routines are added to lsyscache.h to be able to retrieve expressions and predicate of an index from its pg_index data. Reported-by: Alexander Lakhin Author: Tender Wang Reviewed-by: Jian He, Michael Paquier Discussion: https://postgr.es/m/CAHewXN=UaAaNn9ruHDH3Os8kxLVmtWqbssnf=dZN_s9=evHUFA@mail.gmail.com Backpatch-through: 12 06 March 2024, 08:24:05 UTC
8870c54 Jeff Davis 05 March 2024, 17:25:04 UTC docs: Update HOT update docs for 19d8e2308b Commit 19d8e2308b changed when the HOT update optimization is possible but neglected to update the Heap-Only Tuples (HOT) documentation. This patch updates that documentation accordingly. Author: Elizabeth Christensen Backpatch-through: 16 Reviewed-By: Stephen Frost, Alvaro Herrera Discussion: https://postgr.es/m/CABoUFXRjisr58Ct_3VsFEdQx+fJeQTWTdJnM7XAp=8MUbtoa9A@mail.gmail.com 05 March 2024, 17:25:04 UTC
ac7e6a0 David Rowley 05 March 2024, 03:17:53 UTC Fix incorrectly reported stats kind in "can't happen" ERROR The error message(s) were reporting the stats kind of 'f', which is not correct as that's for the "dependencies" statistics kind. Reported-by: Horst Reiterer Reviewed-by: Richard Guo Discussion: https://postgr.es/m/18375-ba99383eb9062d6a@postgresql.org Backpatch-through: 12, where MCV extended stats were added. 05 March 2024, 03:17:53 UTC
b78f4d2 Tom Lane 04 March 2024, 17:00:39 UTC Fix initdb's -c option to treat the GUC name case-insensitively. The backend treats GUC names case-insensitively, so this code should too. This avoids ending up with a confusing set of redundant entries in the generated postgresql.conf file. Per report from Kyotaro Horiguchi. Back-patch to v16 where this feature was added (in commit 3e51b278d). Discussion: https://postgr.es/m/20230928.164904.2153358973162534034.horikyota.ntt@gmail.com 04 March 2024, 17:00:39 UTC
896bd6d Daniel Gustafsson 04 March 2024, 09:52:19 UTC doc: Fix datatype for postgres_fdw option The datatype for analyze_sampling had accidentally been set to text and not string. Backpatch to v16 where analyze_sampling first was introduced. Author: Shinya Kato <Shinya11.Kato@oss.nttdata.com> Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at> Discussion: https://postgr.es/m/7fd9166b9fda267411793f39986d7f24@oss.nttdata.com Backpatch-through: v16 04 March 2024, 09:52:19 UTC
a30a305 Daniel Gustafsson 29 February 2024, 11:19:52 UTC Fix integer underflow in shared memory debugging dsa_dump would print a large negative number instead of zero for segment bin 0. Fix by explicitly checking for underflow and add special case for bin 0. Backpatch to all supported versions. Author: Ian Ilyasov <ianilyasov@outlook.com> Reviewed-by: Robert Haas <robertmhaas@gmail.com> Discussion: https://postgr.es/m/GV1P251MB1004E0D09D117D3CECF9256ECD502@GV1P251MB1004.EURP251.PROD.OUTLOOK.COM Backpatch-through: v12 29 February 2024, 11:19:52 UTC
17db543 Tom Lane 28 February 2024, 19:00:30 UTC Fix mis-rounding and overflow hazards in date_bin(). In the case where the target timestamp is before the origin timestamp and their difference is already an exact multiple of the stride, the code incorrectly subtracted the stride anyway. Also detect several integer-overflow cases that previously produced bogus results. (The submitted patch tried to avoid overflow, but I'm not convinced it's right, and problematic cases are so far out of the plausibly-useful range that they don't seem worth sweating over. Let's just use overflow-detecting arithmetic and throw errors.) timestamp_bin() and timestamptz_bin() are basically identical and so had identical bugs. Fix both. Report and patch by Moaaz Assali, adjusted some by me. Back-patch to v14 where date_bin() was introduced. Discussion: https://postgr.es/m/CALkF+nvtuas-2kydG-WfofbRSJpyODAJWun==W-yO5j2R4meqA@mail.gmail.com 28 February 2024, 19:00:30 UTC
c4bd6ff Alvaro Herrera 26 February 2024, 17:19:03 UTC Revise MERGE documentation Add a note about the additional privileges required after the fix in 4989ce72644b (wording per Tom Lane); also change marked-up mentions of "target_table_name" to be simply "the target table" or the like. Also, note that "join_condition" is scouted for requisite privileges. Backpatch to 15. Discussion: https://postgr.es/m/202402211653.zuh6objy3z72@alvherre.pgsql 26 February 2024, 17:19:03 UTC
22cecad Amit Kapila 26 February 2024, 04:03:57 UTC Back-patch test modifications that were done as part of b6df0798a5. This commit fixes the intermittent buildfarm failures in 031_column_list. I missed to back-patch while committing b6df0798a5 in the HEAD. Diagnosed-by: Amit Kapila Author: Vignesh C Backpatch-through: 15 Discussion: https://postgr.es/m/3307255.1706911634@sss.pgh.pa.us 26 February 2024, 04:03:57 UTC
8c785d3 Tom Lane 25 February 2024, 21:15:07 UTC Promote assertion about !ReindexIsProcessingIndex to runtime error. When this assertion was installed (in commit d2f60a3ab), I thought it was only for catching server logic errors that caused accesses to catalogs that were undergoing index rebuilds. However, it will also fire in case of a user-defined index expression that attempts to access its own table. We occasionally see reports of people trying to do that, and typically getting unintelligible low-level errors as a result. We can provide a more on-point message by making this a regular runtime check. While at it, adjust the similar error check in systable_beginscan_ordered to use the same message text. That one is (probably) not reachable without a coding bug, but we might as well use a translatable message if we have one. Per bug #18363 from Alexander Lakhin. Back-patch to all supported branches. Discussion: https://postgr.es/m/18363-e3598a5a572d0699@postgresql.org 25 February 2024, 21:15:07 UTC
b3b3eff Tom Lane 25 February 2024, 20:29:09 UTC Doc: fix minor typos in two ECPG function descriptions. Noted by Aidar Imamov. Discussion: https://postgr.es/m/170869935022.643.3709087848818148291@wrigleys.postgresql.org 25 February 2024, 20:29:23 UTC
ef0333e Tom Lane 23 February 2024, 20:21:53 UTC Avoid dangling-pointer problem with partitionwise joins under GEQO. build_child_join_sjinfo creates a derived SpecialJoinInfo in the short-lived GEQO context, but afterwards the semi_rhs_exprs from that may be used in a UniquePath for a child base relation. This breaks the expectation that all base-relation-level structures are in the planning-lifespan context, leading to use of a dangling pointer with probable ensuing crash later on in create_unique_plan. To fix, copy the expression trees when making a UniquePath. Per bug #18360 from Alexander Lakhin. This has been broken since partitionwise joins were added, so back-patch to all supported branches. Discussion: https://postgr.es/m/18360-a23caf3157f34e62@postgresql.org 23 February 2024, 20:21:53 UTC
a6155c4 Peter Eisentraut 23 February 2024, 13:40:25 UTC Fix mistake in SQL features list Fix for c9f57541d97: Feature F302-02 was renamed to F305, but that commit failed to delete the old line. Reported-by: Satoru Koizumi (小泉 悟) <koizumistr@minos.ocn.ne.jp> Discussion: https://www.postgresql.org/message-id/flat/170866661469.645.14101429540172934386%40wrigleys.postgresql.org 23 February 2024, 13:40:25 UTC
a3f5d20 Alvaro Herrera 21 February 2024, 16:18:52 UTC MERGE ... DO NOTHING: require SELECT privileges Verify that a user running MERGE with a DO NOTHING clause has privileges to read the table, even if no columns are referenced. Such privileges were already required if the ON clause or any of the WHEN conditions referenced any column at all, so there's no functional change in practice. This change fixes an assertion failure in the case where no column is referenced by the command and the WHEN clauses are all DO NOTHING. Backpatch to 15, where MERGE was introduced. Reported-by: Alena Rybakina <a.rybakina@postgrespro.ru> Reported-by: Alexander Lakhin <exclusion@gmail.com> Discussion: https://postgr.es/m/4d65a385-7efa-4436-a825-0869f89d9d92@postgrespro.ru 21 February 2024, 16:18:52 UTC
21d521d Daniel Gustafsson 21 February 2024, 08:22:18 UTC doc: Fix link to pg_ident_file_mappings view Commit a2c84990bea7 accidentally used the link for pg_hba_file_rules when linking to pg_ident_file_mappings. Backpatch to v16 where this was introduced. Author: Erik Wienhold <ewie@ewie.name> Discussion: https://postgr.es/m/qt5hvgvfi4qzlgml2dfssaut2t2x5nwf7b5l63fklr7fpxwm6g@hle3mtglpm4y Backpatch-through: v16 21 February 2024, 08:22:18 UTC
3ad319b Tom Lane 20 February 2024, 19:35:12 UTC Doc: improve explanation of type interval, especially extract(). The explanation of interval's behavior in datatype.sgml wasn't wrong exactly, but it was unclear, partly because it buried the lede about there being three internal fields. Rearrange and wordsmith for more clarity. The discussion of extract() claimed that input of type date was handled by casting, but actually there's been a separate SQL function taking date for a very long time. Also, it was mostly silent about how interval inputs are handled, but there are several field types for which it seems useful to be specific. Improve discussion of justify_days()/justify_hours() too. In passing, remove vertical space in some groups of examples, as there was little consistency about whether to have such space or not. (I only did this within the datetime functions section; there are some related inconsistencies elsewhere.) Per discussion of bug #18348 from Michael Bondarenko. There may be some code changes coming out of that discussion too, but we likely won't back-patch them. This docs-only patch seems useful to back-patch, though I only carried it back to v13 because it didn't apply easily in v12. Discussion: https://postgr.es/m/18348-b097a3587dfde8a4@postgresql.org 20 February 2024, 19:35:12 UTC
5ad6c72 Tom Lane 20 February 2024, 16:58:28 UTC Doc: correct minor error in back-branch release notes. Commits 1b2c6b756 et al affected the core BRIN "bloom" opclasses, not contrib/bloom. This only corrected a bad assertion so it's not too significant to end users, but since we documented it we should do so accurately. Spotted by Takatsuka Haruka. Discussion: https://postgr.es/m/18353-926aa99cfe58aa78@postgresql.org 20 February 2024, 16:58:28 UTC
59cea09 Michael Paquier 20 February 2024, 04:43:56 UTC Fix race leading to incorrect conflict cause in InvalidatePossiblyObsoleteSlot() The invalidation of an active slot is done in two steps: - Termination of the backend holding it, if any. - Report that the slot is obsolete, with a conflict cause depending on the slot's data. This can be racy because between these two steps the slot mutex would be released while doing system calls, which means that the effective_xmin and effective_catalog_xmin could advance during that time, detecting a conflict cause different than the one originally wanted before the process owning a slot is terminated. Holding the mutex longer is not an option, so this commit changes the code to record the LSNs stored in the slot during the termination of the process owning the slot. Bonus thanks to Alexander Lakhin for the various tests and the analysis. Author: Bertrand Drouvot Reviewed-by: Michael Paquier, Bharath Rupireddy Discussion: https://postgr.es/m/ZaTjW2Xh+TQUCOH0@ip-10-97-1-34.eu-west-3.compute.internal Backpatch-through: 16 20 February 2024, 04:43:56 UTC
ec4fcf4 Michael Paquier 20 February 2024, 02:59:10 UTC doc: Use system-username instead of system-user This inconsistency has been introduced in efb6f4a4f9b6. Reported-by: Julien Rouhaud Author: Bertrand Drouvot Discussion: https://postgr.es/m/ZdMWux1HpIebkEmd@ip-10-97-1-34.eu-west-3.compute.internal Backpatch-through: 16 20 February 2024, 02:59:10 UTC
fb95cc7 David Rowley 19 February 2024, 23:50:09 UTC Fix incorrect pruning of NULL partition for boolean IS NOT clauses Partition pruning wrongly assumed that, for a table partitioned on a boolean column, a clause in the form "boolcol IS NOT false" and "boolcol IS NOT true" could be inverted to correspondingly become "boolcol IS true" and "boolcol IS false". These are not equivalent as the NOT version matches the opposite boolean value *and* NULLs. This incorrect assumption meant that partition pruning pruned away partitions that could contain NULL values. Here we fix this by correctly not pruning partitions which could store NULLs. To be affected by this, the table must be partitioned by a NULLable boolean column and queries would have to contain "boolcol IS NOT false" or "boolcol IS NOT true". This could result in queries filtering out NULL values with a LIST partitioned table and "ERROR: invalid strategy number 0" for RANGE and HASH partitioned tables. Reported-by: Alexander Lakhin Bug: #18344 Discussion: https://postgr.es/m/18344-8d3f00bada6d09c6@postgresql.org Backpatch-through: 12 19 February 2024, 23:50:09 UTC
c59a973 Noah Misch 19 February 2024, 20:52:28 UTC Fix test race between primary XLOG_RUNNING_XACTS and standby logical slot. Before the previous commit, the test could hang until LOG_SNAPSHOT_INTERVAL_MS (15s), until checkpoint_timeout (300s), or indefinitely. An indefinite hang was awfully improbable. It entailed the test reaching checkpoint_timeout before the DecodingContextFindStartpoint() of a CREATE SUBSCRIPTION, yet after the preceding WAL record. Back-patch to v16, which introduced the test. Bertrand Drouvot, reported by Noah Misch. Discussion: https://postgr.es/m/20240211010227.a2.nmisch@google.com 19 February 2024, 20:52:49 UTC
f024746 Noah Misch 19 February 2024, 20:52:07 UTC Bound waits in 035_standby_logical_decoding.pl. One IPC::Run::start() used an IPC::Run::timer() without checking for expiration. The other used no timeout or timer. Back-patch to v16, which introduced the test. Reviewed by Bertrand Drouvot. Discussion: https://postgr.es/m/20240211010227.a2.nmisch@google.com 19 February 2024, 20:52:48 UTC
6257ebf Tom Lane 19 February 2024, 19:17:11 UTC Doc: fix typo in SECURITY LABEL synopsis. One case missed its trailing "|". Reported by Tim Needham. Discussion: https://postgr.es/m/170833547220.3279712.700702770281879175@wrigleys.postgresql.org 19 February 2024, 19:17:22 UTC
88e03d0 Michael Paquier 19 February 2024, 02:38:44 UTC ecpg: Fix zero-termination of string generated by intoasc() intoasc(), a wrapper for PGTYPESinterval_to_asc that converts an interval to its textual representation, used a plain memcpy() when copying its result. This could miss a zero-termination in the result string, leading to an incorrect result. The routines in informix.c do not provide the length of their result buffer, which would allow a replacement of strcpy() to safer strlcpy() calls, but this requires an ABI breakage and that cannot happen in back-branches. Author: Oleg Tselebrovskiy Reviewed-by: Ashutosh Bapat Discussion: https://postgr.es/m/bf47888585149f83b276861a1662f7e4@postgrespro.ru Backpatch-through: 12 19 February 2024, 02:38:44 UTC
2c7f2eb Peter Eisentraut 16 February 2024, 10:39:09 UTC Remove non-existing file from .gitattributes The file was removed by ac25173cdbc. Author: Jelte Fennema-Nio <postgres@jeltef.nl> Discussion: https://www.postgresql.org/message-id/flat/CAGECzQQGzbroAXi%2BYicp3HvcCo4%3Dg84kaOgjuvQ5MW9F0ubOGg%40mail.gmail.com 16 February 2024, 10:40:53 UTC
1a6dcfe Tom Lane 15 February 2024, 21:45:03 UTC Doc: improve a couple of comments in postgresql.conf.sample. Clarify comments associated with max_parallel_workers and related settings. Per bug #18343 from Christopher Kline. Discussion: https://postgr.es/m/18343-3a5e903d1d3692ab@postgresql.org 15 February 2024, 21:45:03 UTC
aafa451 Daniel Gustafsson 14 February 2024, 10:05:10 UTC doc: Remove links to further reading from pgcrypto docs The pgcrypto docs contained a set of links for useful reading and technical references. These sets of links were however not actively curated and had stale content and dead links. Rather than investing time into maintining these, this removes them altogether since there are lots of resources online which are actively maintained. Backpatch to all supported versions since these links have been in the docs for a long time. Reported-by: Hanefi Onaldi <hanefi.onaldi@microsoft.com> Reviewed-by: Magnus Hagander <magnus@hagander.net> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/170774255387.3279713.2822272755998870925@wrigleys.postgresql.org Backpatch-through: v12 14 February 2024, 10:05:10 UTC
f2f09b8 Heikki Linnakangas 13 February 2024, 19:23:41 UTC Fix 'mmap' DSM implementation with allocations larger than 4 GB Fixes bug #18341. Backpatch to all supported versions. Discussion: https://www.postgresql.org/message-id/18341-ce16599e7fd6228c@postgresql.org 13 February 2024, 19:24:45 UTC
f15d01c Tom Lane 13 February 2024, 17:18:25 UTC Use a safer outfuncs/readfuncs representation for BitStrings. For a long time, our outfuncs.c code has supposed that the string contents of a BitString node could just be printed literally with no concern for quoting/escaping. Now, that's okay if the string literal contains only valid binary or hex digits ... but our lexer doesn't check that, preferring to let bitin() be the sole authority on what's valid. So we could have raw parse trees that contain incorrect BitString literals, and that can result in failures when WRITE_READ_PARSE_PLAN_TREES debugging is enabled. Fix by using outToken() to print the string field, and debackslash() to read it. This results in a change in the emitted representation only in cases that would have failed before, and don't represent valid SQL in the first place. Between that and the fact that we don't store raw parse trees in the catalogs, I judge this safe to apply without a catversion bump. Per bug #18340 from Alexander Lakhin. Back-patch to v16; before that, we lacked readfuncs support for BitString nodes, so that the problem was only cosmetic. Discussion: https://postgr.es/m/18340-4aa1ae6ed4121912@postgresql.org 13 February 2024, 17:18:25 UTC
1032358 Daniel Gustafsson 13 February 2024, 12:47:12 UTC Skip .DS_Store files in server side utils The macOS Finder application creates .DS_Store files in directories when opened, which creates problems for serverside utilities which expect all files to be PostgreSQL specific files. Skip these files when encountered in pg_checksums, pg_rewind and pg_basebackup. This was extracted from a larger patchset for skipping hidden files and system files, where the concencus was to just skip these. Since this is equally likely to happen in every version, backpatch to all supported versions. Reported-by: Mark Guertin <markguertin@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Tobias Bussmann <t.bussmann@gmx.net> Discussion: https://postgr.es/m/E258CE50-AB0E-455D-8AAD-BB4FE8F882FB@gmail.com Backpatch-through: v12 13 February 2024, 12:47:12 UTC
0460e4e Thomas Munro 11 February 2024, 21:47:57 UTC Fix gai_strerror() thread-safety on Windows. Commit 5579388d removed code that supplied a fallback implementation of getaddrinfo(), which was dead code on modern systems. One tiny piece of the removed code was still doing something useful on Windows, though: that OS's own gai_strerror()/gai_strerrorA() function returns a pointer to a static buffer that it overwrites each time, so it's not thread-safe. In rare circumstances, a multi-threaded client program could get an incorrect or corrupted error message. Restore the replacement gai_strerror() function, though now that it's only for Windows we can put it into a win32-specific file and cut it down to the errors that Windows documents. The error messages here are taken from FreeBSD, because Windows' own messages seemed too verbose. Back-patch to 16. Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA%2BhUKGKz%2BF9d2PTiXwfYV7qJw%2BWg2jzACgSDgPizUw7UG%3Di58A%40mail.gmail.com 11 February 2024, 22:14:42 UTC
4eb2611 Tom Lane 09 February 2024, 17:29:41 UTC Remove race condition in pg_get_expr(). Since its introduction, pg_get_expr() has intended to silently return NULL if called with an invalid relation OID, as can happen when scanning the catalogs concurrently with relation drops. However, there is a race condition: we check validity of the OID at the start, but it could get dropped just afterward, leading to failures. This is the cause of some intermittent instability we're seeing in a proposed new test case, and presumably it's a hazard in the field as well. We can fix this by AccessShareLock-ing the target relation for the duration of pg_get_expr(). Since we don't require any permissions on the target relation, this is semantically a bit undesirable. But it turns out that the set_relation_column_names() subroutine already takes a transient AccessShareLock on that relation, and has done since commit 2ffa740be in 2012. Given the lack of complaints about that, it seems like there should be no harm in holding the lock a bit longer. Back-patch to all supported branches. Discussion: https://postgr.es/m/31ddcc01-a71b-4e8c-9948-01d1c47293ca@eisentraut.org 09 February 2024, 17:29:41 UTC
52afe56 Tom Lane 09 February 2024, 16:21:08 UTC Avoid concurrent calls to bindtextdomain(). We previously supposed that it was okay for different threads to call bindtextdomain() concurrently (cf. commit 1f655fdc3). It now emerges that there's at least one gettext implementation in which that triggers an abort() crash, so let's stop doing that. Add mutexes guarding libpq's and ecpglib's calls, which are the only ones that need worry about multithreaded callers. Note: in libpq, we could perhaps have piggybacked on default_threadlock() to avoid defining a new mutex variable. I judge that not terribly safe though, since libpq_gettext could be called from code that is holding the default mutex. If that were the first such call in the process, it'd fail. An extra mutex is cheap insurance against unforeseen interactions. Per bug #18312 from Christian Maurer. Back-patch to all supported versions. Discussion: https://postgr.es/m/18312-bbbabc8113592b78@postgresql.org Discussion: https://postgr.es/m/264860.1707163416@sss.pgh.pa.us 09 February 2024, 16:21:08 UTC
9440d23 Tom Lane 09 February 2024, 16:11:39 UTC Clean up Windows-specific mutex code in libpq and ecpglib. Fix pthread-win32.h and pthread-win32.c to provide a more complete emulation of POSIX pthread mutexes: define PTHREAD_MUTEX_INITIALIZER and make sure that pthread_mutex_lock() can operate on a mutex object that's been initialized that way. Then we don't need the duplicative platform-specific logic in default_threadlock() and pgtls_init(), which we'd otherwise need yet a third copy of for an upcoming bug fix. Also, since default_threadlock() supposes that pthread_mutex_lock() cannot fail, try to ensure that that's actually true, by getting rid of the malloc call that was formerly involved in initializing an emulated mutex. We can define an extra state for the spinlock field instead. Also, replace the similar code in ecpglib/misc.c with this version. While ecpglib's version at least had a POSIX-compliant API, it also had the potential of failing during mutex init (but here, because of CreateMutex failure rather than malloc failure). Since all of misc.c's callers ignore failures, it seems like a wise idea to avoid failures here too. A further improvement in this area could be to unify libpq's and ecpglib's implementations into a src/port/pthread-win32.c file. But that doesn't seem like a bug fix, so I'll desist for now. In preparation for the aforementioned bug fix, back-patch to all supported branches. Discussion: https://postgr.es/m/264860.1707163416@sss.pgh.pa.us 09 February 2024, 16:11:39 UTC
e3e05ad Alexander Korotkov 08 February 2024, 10:45:26 UTC Fix wrong logic in TransactionIdInRecentPast() The TransactionIdInRecentPast() should return false for all the transactions older than TransamVariables->oldestClogXid. However, the function contains a bug in comparison FullTransactionId to TransactionID allowing full transactions between nextXid - 2^32 and oldestClogXid - 2^31. This commit fixes TransactionIdInRecentPast() by turning the oldestClogXid into FullTransactionId first, then performing the comparison. Backpatch to all supported versions. Reported-by: Egor Chindyaskin Bug: 18212 Discussion: https://postgr.es/m/18212-547307f8adf57262%40postgresql.org Author: Karina Litskevich Reviewed-by: Kyotaro Horiguchi Backpatch-through: 12 09 February 2024, 10:37:21 UTC
86d2b43 Peter Eisentraut 09 February 2024, 06:57:31 UTC Fix propagation of persistence to sequences in ALTER TABLE / ADD COLUMN Fix for 344d62fb9a9: That commit introduced unlogged sequences and made it so that identity/serial sequences automatically get the persistence level of their owning table. But this works only for CREATE TABLE and not for ALTER TABLE / ADD COLUMN. The latter would always create the sequence as logged (default), independent of the persistence setting of the table. This is fixed here. Note: It is allowed to change the persistence of identity sequences directly using ALTER SEQUENCE. So mistakes in existing databases can be fixed manually. Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/c4b6e2ed-bcdf-4ea7-965f-e49761094827%40eisentraut.org 09 February 2024, 07:09:59 UTC
e13205a Daniel Gustafsson 08 February 2024, 11:19:34 UTC doc: Remove superfluous bracket in synopsis Commit 9c08aea6a30 accidentally added one too many end brackets in the synopsis for CREATE DATABASE .. strategy = strat. Fix by removing. Backpatch to v15 where it was introduced. Reported-by: tim.needham2@gmail.com Discussion: https://postgr.es/m/170734160862.3279712.810853722572951776@wrigleys.postgresql.org Backpatch-through: v15 08 February 2024, 11:19:34 UTC
5c699c8 Peter Eisentraut 08 February 2024, 09:18:50 UTC Update comment The documentation output format htmlhelp is no longer supported, but a comment still mentioned it. 08 February 2024, 09:19:55 UTC
17f3196 Alvaro Herrera 07 February 2024, 18:25:07 UTC Update PQparameterStatus and ParameterStatus docs Cover scram_iterations, which was added in commit b577743000cd. While at it, turn the list into a <simplelist> with 2 columns, which is much nicer to read. In master, remove mentions of antediluvian versions before which some parameters were not reported. Noticed while investigating a question by Maiquel Grassi. Backpatch to 16. Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl> Discussion: https://postgr.es/m/202401301236.mc5ebrohhtsd@alvherre.pgsql 07 February 2024, 18:25:07 UTC
b78fa85 Tom Lane 05 February 2024, 21:41:37 UTC Stamp 16.2. 05 February 2024, 21:41:37 UTC
da21a3a Tom Lane 05 February 2024, 16:51:11 UTC Last-minute updates for release notes. Security: CVE-2024-0985 (not CVE-2023-5869 as claimed in prior commit msg) 05 February 2024, 16:51:11 UTC
246d16e Peter Eisentraut 05 February 2024, 13:45:29 UTC Translation updates Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 7465ae7935588bbbafa9aac1c2b8c5863de50cbb 05 February 2024, 13:45:29 UTC
fb38368 Heikki Linnakangas 05 February 2024, 09:01:30 UTC Fix assertion if index is dropped during REFRESH CONCURRENTLY When assertions are disabled, the built SQL statement is invalid and you get a "syntax error". So this isn't a serious problem, but let's avoid the assertion failure. Backpatch to all supported versions. Reviewed-by: Noah Misch 05 February 2024, 09:03:03 UTC
d6a61cb Heikki Linnakangas 05 February 2024, 09:01:23 UTC Run REFRESH MATERIALIZED VIEW CONCURRENTLY in right security context The internal commands in REFRESH MATERIALIZED VIEW CONCURRENTLY are correctly executed in SECURITY_RESTRICTED_OPERATION mode, except for creating the temporary "diff" table, because you cannot create temporary tables in SRO mode. But creating the temporary "diff" table is a pretty complex CTAS command that selects from another temporary table created earlier in the command. If you can cajole that CTAS command to execute code defined by the table owner, the table owner can run code with the privileges of the user running the REFRESH command. The proof-of-concept reported to the security team relied on CREATE RULE to convert the internally-built temp table to a view. That's not possible since commit b23cd185fd, and I was not able to find a different way to turn the SELECT on the temp table into code execution, so as far as I know this is only exploitable in v15 and below. That's a fiddly assumption though, so apply this patch to master and all stable versions. Thanks to Pedro Gallegos for the report. Security: CVE-2023-5869 Reviewed-by: Noah Misch 05 February 2024, 09:02:56 UTC
272aecf Tom Lane 04 February 2024, 19:17:14 UTC Release notes for 16.2, 15.6, 14.11, 13.14, 12.18. 04 February 2024, 19:17:14 UTC
cd307c3 Heikki Linnakangas 02 February 2024, 22:18:21 UTC Fix typo in comments Backpatch-through: v16 02 February 2024, 22:19:21 UTC
a153781 Tom Lane 02 February 2024, 20:34:29 UTC Translate ENOMEM to ERRCODE_OUT_OF_MEMORY in errcode_for_file_access(). Previously you got ERRCODE_INTERNAL_ERROR, which seems inappropriate, especially given that we're trying to avoid emitting that in reachable cases. Alexander Kuzmenkov Discussion: https://postgr.es/m/CALzhyqzgQph0BY8-hFRRGdHhF8CoqmmDHW9S=hMZ-HMzLxRqDQ@mail.gmail.com 02 February 2024, 20:34:29 UTC
87dcc5e Tom Lane 02 February 2024, 17:52:29 UTC First-draft release notes for 16.2. As usual, the release notes for other branches will be made by cutting these down, but put them up for community review first. 02 February 2024, 17:52:29 UTC
48a6bf5 Noah Misch 01 February 2024, 21:44:19 UTC Sync PG_VERSION file in CREATE DATABASE. An OS crash could leave PG_VERSION empty or missing. The same symptom appeared in a backup by block device snapshot, taken after the next checkpoint and before the OS flushes the PG_VERSION blocks. Device snapshots are not a documented backup method, however. Back-patch to v15, where commit 9c08aea6a3090a396be334cc58c511edab05776a introduced STRATEGY=WAL_LOG and made it the default. Discussion: https://postgr.es/m/20240130195003.0a.nmisch@google.com 01 February 2024, 21:44:22 UTC
6d423e9 Noah Misch 01 February 2024, 21:44:19 UTC Handle interleavings between CREATE DATABASE steps and base backup. Restoring a base backup taken in the middle of CreateDirAndVersionFile() or write_relmap_file() would lose the function's effects. The symptom was absence of the database directory, PG_VERSION file, or pg_filenode.map. If missing the directory, recovery would fail. Either missing file would not fail recovery but would render the new database unusable. Fix CreateDirAndVersionFile() with the transam/README "action first and then write a WAL entry" strategy. That has a side benefit of moving filesystem mutations out of a critical section, reducing the ways to PANIC. Fix the write_relmap_file() call with a lock acquisition, so it interacts with checkpoints like non-CREATE DATABASE calls do. Back-patch to v15, where commit 9c08aea6a3090a396be334cc58c511edab05776a introduced STRATEGY=WAL_LOG and made it the default. Discussion: https://postgr.es/m/20240130195003.0a.nmisch@google.com 01 February 2024, 21:44:22 UTC
b4fb76f Tom Lane 01 February 2024, 20:57:53 UTC Update time zone data files to tzdata release 2024a. DST law changes in Ittoqqortoormiit, Greenland (America/Scoresbysund), Kazakhstan (Asia/Almaty and Asia/Qostanay) and Palestine; as well as updates for the Antarctic stations Casey and Vostok. Historical corrections for Vietnam, Toronto, and Miquelon. 01 February 2024, 20:57:53 UTC
90bad72 Andrew Dunstan 01 February 2024, 20:17:41 UTC Avoid package qualification of $windows_os Further fallout from commit 6ee26c6a4b. To keep code in sync and avoid issues on older releases with different package names, simply use the unqualified name like many other places in our code. 01 February 2024, 20:33:58 UTC
62f1202 Tom Lane 01 February 2024, 17:34:21 UTC Apply band-aid fix for an oversight in reparameterize_path_by_child. The path we wish to reparameterize is not a standalone object: in particular, it implicitly references baserestrictinfo clauses in the associated RelOptInfo, and if it's a SampleScan path then there is also the TableSampleClause in the RTE to worry about. Both of those could contain lateral references to the join partner relation, which would need to be modified to refer to its child. Since we aren't doing that, affected queries can give wrong answers, or odd failures such as "variable not found in subplan target list", or executor crashes. But we can't just summarily modify those expressions, because they are shared with other paths for the rel. We'd break things if we modify them and then end up using some non-partitioned-join path. In HEAD, we plan to fix this by postponing reparameterization until create_plan(), when we know that those other paths are no longer of interest, and then adjusting those expressions along with the ones in the path itself. That seems like too big a change for stable branches however. In the back branches, let's just detect whether any troublesome lateral references actually exist in those expressions, and fail reparameterization if so. This will result in not performing a partitioned join in such cases. Given the lack of field complaints, nobody's likely to miss the optimization. Report and patch by Richard Guo. Apply to 12-16 only, since the intended fix for HEAD looks quite different. We're not quite ready to push the HEAD fix, but with back-branch releases coming up soon, it seems wise to get this stopgap fix in place there. Discussion: https://postgr.es/m/CAMbWs496+N=UAjOc=rcD3P7B6oJe4rZw08e_TZRUsWbPxZW3Tw@mail.gmail.com 01 February 2024, 17:34:21 UTC
88be3f0 Bruce Momjian 01 February 2024, 11:11:53 UTC doc: improve role option documentation Role option management was changed in Postgres 16. This patch improves the docs around these changes, including CREATE ROLE's INHERIT option, inheritance handling, and grant's ability to change role options. Discussion: https://postgr.es/m/Zab9GiV63EENDcWG@momjian.us Co-authored-by: David G. Johnston Backpatch-through: 16 01 February 2024, 11:11:53 UTC
7743046 Daniel Gustafsson 01 February 2024, 09:45:37 UTC doc: remove incorrect grammar for ALTER EVENT TRIGGER The Parameters subsection had an extra TRIGGER in the grammar for DISABLE/ENABLE which is incorrect. Backpatch down to all supported versions since it's been like this all along. Discussion: https://postgr.es/m/0AFB171E-7E78-4A90-A140-46AB270212CA@yesql.se Backpatch-through: v12 01 February 2024, 09:45:37 UTC
0df4f75 Daniel Gustafsson 01 February 2024, 08:36:34 UTC doc: Fix incorrect openssl option The openssl command for displaying the DN of a client certificate was using --subject and not the single-dash option -subject. While recent versions of openssl handles double dash options, earlier does not so fix by using just -subject (which is per the openssl documentation). Backpatch to v14 where this was introduced. Reported-by: konkove@gmail.com Discussion: https://postgr.es/m/170672168899.666.10442618407194498217@wrigleys.postgresql.org Backpatch-through: v14 01 February 2024, 08:36:34 UTC
781bc12 Michael Paquier 01 February 2024, 08:13:09 UTC Fix stats_fetch_consistency with stats for fixed-numbered objects This impacts the statistics retrieved in transactions for the following views when updating the value of stats_fetch_consistency, leading to behaviors contrary to what is documented since 605994651b6a as an update of this parameter should discard all statistics snapshot data: - pg_stat_archiver - pg_stat_bgwriter - pg_stat_checkpointer - pg_stat_io - pg_stat_slru - pg_stat_wal For example, updating stats_fetch_consistency from "snapshot" to "cache" in a transaction did not re-fetch any fresh data, using data cached from the time when "snapshot" was in use. Author: Shinya Kato Discussion: https://postgr.es/m/d77fc5190d4dbe1738d77231488e768b@oss.nttdata.com Backpatch-through: 15 01 February 2024, 08:13:09 UTC
f33e832 Michael Paquier 31 January 2024, 04:16:43 UTC Fix various issues with ALTER TEXT SEARCH CONFIGURATION This commit addresses a set of issues when changing token type mappings in a text search configuration when using duplicated token names: - ADD MAPPING would fail on insertion because of a constraint failure after inserting the same mapping. - ALTER MAPPING with an "overridden" configuration failed with "tuple already updated by self" when the token mappings are removed. - DROP MAPPING failed with "tuple already updated by self", like previously, but in a different code path. The code is refactored so the token names (with their numbers) are handled as a List with unique members rather than an array with numbers, ensuring that no duplicates mess up with the catalog inserts, updates and deletes. The list is generated by getTokenTypes(), with the same error handling as previously while duplicated tokens are discarded from the list used to work on the catalogs. Regression tests are expanded to cover much more ground for the cases fixed by this commit, as there was no coverage for the code touched in this commit. A bit more is done regarding the fact that a token name not supported by a configuration's parser should result in an error even if IF EXISTS is used in a DROP MAPPING clause. This is implied in the code but there was no coverage for that, and it was very easy to miss. These issues exist since at least their introduction in core with 140d4ebcb46e, so backpatch all the way down. Reported-by: Alexander Lakhin Author: Tender Wang, Michael Paquier Discussion: https://postgr.es/m/18310-1eb233c5908189c8@postgresql.org Backpatch-through: 12 31 January 2024, 04:16:43 UTC
52a9af3 Andrew Dunstan 30 January 2024, 22:09:44 UTC Fix 003_extrafiles.pl test for the Windows File::Find converts backslashes to slashes in the newer Perl versions. See: https://github.com/Perl/perl5/commit/414f14df98cb1c9a20f92c5c54948b67c09f072d So, do the same conversion for Windows before comparing paths. To support all Perl versions, always convert them on Windows regardless of the Perl's version. Author: Nazir Bilal Yavuz <byavuz81@gmail.com> Backpatch to all live branches 30 January 2024, 22:11:47 UTC
5d726c7 Daniel Gustafsson 30 January 2024, 10:15:46 UTC pgcrypto: Fix check for buffer size The code copying the PGP block into the temp buffer failed to account for the extra 2 bytes in the buffer which are needed for the prefix. If the block was oversized, subsequent checks of the prefix would have exceeded the buffer size. Since the block sizes are hardcoded in the list of supported ciphers it can be verified that there is no live bug here. Backpatch all the way for consistency though, as this bug is old. Author: Mikhail Gribkov <youzhick@gmail.com> Discussion: https://postgr.es/m/CAMEv5_uWvcMCMdRFDsJLz2Q8g16HEa9xWyfrkr+FYMMFJhawOw@mail.gmail.com Backpatch-through: v12 30 January 2024, 10:15:46 UTC
1c8336e David Rowley 29 January 2024, 21:15:51 UTC Doc: mention foreign keys can reference unique indexes We seem to have only documented a foreign key can reference the columns of a primary key or unique constraint. Here we adjust the documentation to mention columns in a non-partial unique index can be mentioned too. The header comment for transformFkeyCheckAttrs() also didn't mention unique indexes, so fix that too. In passing make that header comment reflect reality in the various other aspects where it deviated from it. Bug: 18295 Reported-by: Gilles PARC Author: Laurenz Albe, David Rowley Discussion: https://www.postgresql.org/message-id/18295-0ed0fac5c9f7b17b%40postgresql.org Backpatch-through: 12 29 January 2024, 21:15:51 UTC
1b924a8 Nathan Bossart 29 January 2024, 18:09:03 UTC Move is_valid_ascii() to ascii.h. This function requires simd.h, which is a rather large dependency for a widely-used header file like pg_wchar.h. Furthermore, there is a report of a third-party tool that is struggling to use pg_wchar.h due to its dependence on simd.h (presumably because simd.h uses several intrinsics). Moving the function to the much less popular ascii.h resolves these issues for now. This commit is back-patched for the benefit of the aforementioned third-party tool. The simd.h dependency was only added in v16, but we've opted to back-patch to v15 so that is_valid_ascii() lives in the same file for all versions where it exists. This could break existing third-party code that uses the function, but we couldn't find any examples of such code. It should be possible to fix any code that this commit breaks by including ascii.h in the file that uses is_valid_ascii(). Author: Jubilee Young Reviewed-by: Tom Lane, John Naylor, Andres Freund, Eric Ridge Discussion: https://postgr.es/m/CAPNHn3oKJJxMsYq%2BqLYzVJOFrUcOr4OF1EC-KtFT-qh8nOOOtQ%40mail.gmail.com Backpatch-through: 15 29 January 2024, 18:09:03 UTC
back to top