Skip to main content

Browse the archive

https://forge.softwareheritage.org/source/swh-scheduler.git
13 March 2024, 03:48:39 UTC
Revision e051b320e4050bdc75502bf23de8b5d53d368809 authored by Nicolas Dandrimont on 09 December 2021, 13:54:09 UTC, committed by Nicolas Dandrimont on 09 December 2021, 14:16:06 UTC 
Use a temporary table to update scheduler metrics
 
When using ``insert into <...> select <...>``, PostgreSQL disables
parallel querying. Under some circumstances (in our large production
database), this makes updating the scheduler metrics take a (very) long
time.

Parallel querying is allowed for ``create table <...> as select <...>``,
and doing so restores the small(er) runtimes for this query (15 minutes
instead of multiple hours). To use that, we have to turn the function
into plpgsql instead of plain sql.
1 parent a8edbdb
History
Tip revision: e051b320e4050bdc75502bf23de8b5d53d368809 authored by Nicolas Dandrimont on 09 December 2021, 13:54:09 UTC
Use a temporary table to update scheduler metrics
Tip revision: e051b32
File Mode Size
data
docs
sql
swh
.gitignore -rw-r--r-- 129 bytes
.pre-commit-config.yaml -rw-r--r-- 1.0 KB
AUTHORS -rw-r--r-- 112 bytes
CODE_OF_CONDUCT.md -rw-r--r-- 3.3 KB
CONTRIBUTORS -rw-r--r-- 14 bytes
LICENSE -rw-r--r-- 34.3 KB
LICENSE.Celery -rw-r--r-- 2.6 KB
MANIFEST.in -rw-r--r-- 154 bytes
Makefile -rw-r--r-- 163 bytes
README.md -rw-r--r-- 292 bytes
conftest.py -rw-r--r-- 664 bytes
mypy.ini -rw-r--r-- 750 bytes
pyproject.toml -rw-r--r-- 237 bytes
pytest.ini -rw-r--r-- 33 bytes
requirements-journal.txt -rw-r--r-- 12 bytes
requirements-simulator.txt -rw-r--r-- 21 bytes
requirements-swh.txt -rw-r--r-- 50 bytes
requirements-test.txt -rw-r--r-- 155 bytes
requirements.txt -rw-r--r-- 364 bytes
setup.cfg -rw-r--r-- 247 bytes
setup.py -rwxr-xr-x 2.5 KB
tox.ini -rw-r--r-- 1.5 KB

README.md

back to top