Skip to main content

Browse the archive

https://forge.softwareheritage.org/source/swh-scheduler.git
13 March 2024, 03:48:39 UTC
Revision e051b320e4050bdc75502bf23de8b5d53d368809 authored by Nicolas Dandrimont on 09 December 2021, 13:54:09 UTC, committed by Nicolas Dandrimont on 09 December 2021, 14:16:06 UTC 
Use a temporary table to update scheduler metrics
 
When using ``insert into <...> select <...>``, PostgreSQL disables
parallel querying. Under some circumstances (in our large production
database), this makes updating the scheduler metrics take a (very) long
time.

Parallel querying is allowed for ``create table <...> as select <...>``,
and doing so restores the small(er) runtimes for this query (15 minutes
instead of multiple hours). To use that, we have to turn the function
into plpgsql instead of plain sql.
1 parent a8edbdb
Raw File
Tip revision: e051b320e4050bdc75502bf23de8b5d53d368809 authored by Nicolas Dandrimont on 09 December 2021, 13:54:09 UTC
Use a temporary table to update scheduler metrics
Tip revision: e051b32
.pre-commit-config.yaml 
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
  rev: v2.4.0
  hooks:
  - id: trailing-whitespace
  - id: check-json
  - id: check-yaml

- repo: https://gitlab.com/pycqa/flake8
  rev: 3.8.3
  hooks:
  - id: flake8

- repo: https://github.com/codespell-project/codespell
  rev: v1.16.0
  hooks:
  - id: codespell
    args: [-L simpy, -L hist]

- repo: local
  hooks:
  - id: mypy
    name: mypy
    entry: mypy
    args: [swh]
    pass_filenames: false
    language: system
    types: [python]

- repo: https://github.com/PyCQA/isort
  rev: 5.5.2
  hooks:
  - id: isort

- repo: https://github.com/python/black
  rev: 19.10b0
  hooks:
  - id: black

# unfortunately, we are far from being able to enable this...
# - repo: https://github.com/PyCQA/pydocstyle.git
#   rev: 4.0.0
#   hooks:
#   - id: pydocstyle
#     name: pydocstyle
#     description: pydocstyle is a static analysis tool for checking compliance with Python docstring conventions.
#     entry: pydocstyle --convention=google
#     language: python
#     types: [python]
back to top