Revision - a76bb02 - Make scheduling policy used in schedule_recurrent configurable - origin: https://forge.softwareheritage.org/source/swh-scheduler.git

visit type:

https://forge.softwareheritage.org/source/swh-scheduler.git

13 March 2024, 03:48:39 UTC

Revision a76bb02f0e94bf1c61124c9133d48a03f3d1a05f authored by David Douard on 15 April 2022, 16:08:49 UTC, committed by David Douard on 20 April 2022, 14:34:23 UTC

Make scheduling policy used in schedule_recurrent configurable

Add support for a configuration option "scheduling_policy" in the config
file loaded by the 'swh scheduler schedule-recurrent' command. This
config entry allows to specify the scheduling policies used by the
schedule-recurrent tool, instead of having them hardcoded in the source
code.

A visit type policy config entry should have at least a 'weight' value
for each policy.

Default values are unchanged.

Eg.:

  scheduling_policy:
    git:
      - policy: already_visited_order_by_lag
        weight: 55
        tablesample: 0.5
      - policy: never_visited_oldest_update_first
        weight: 45
        tablesample: 0.5

Note: there may not be configuration entries for all visit types, but if
      a visit type policy is configured, the config entry should be complete
      (in other words, the merging of the configuration with the default
      values is only done at first config level).

1 parent 5302efd

Files
Changes

Permalinks

Tip revision: a76bb02f0e94bf1c61124c9133d48a03f3d1a05f authored by David Douard on 15 April 2022, 16:08:49 UTC
Make scheduling policy used in schedule_recurrent configurable

Tip revision: a76bb02

File	Mode	Size
data
docs
sql
swh
.git-blame-ignore-revs	-rw-r--r--	140 bytes
.gitignore	-rw-r--r--	129 bytes
.pre-commit-config.yaml	-rw-r--r--	976 bytes
AUTHORS	-rw-r--r--	112 bytes
CODE_OF_CONDUCT.md	-rw-r--r--	3.3 KB
CONTRIBUTORS	-rw-r--r--	14 bytes
LICENSE	-rw-r--r--	34.3 KB
LICENSE.Celery	-rw-r--r--	2.6 KB
MANIFEST.in	-rw-r--r--	154 bytes
Makefile	-rw-r--r--	163 bytes
README.md	-rw-r--r--	292 bytes
conftest.py	-rw-r--r--	664 bytes
mypy.ini	-rw-r--r--	750 bytes
pyproject.toml	-rw-r--r--	237 bytes
pytest.ini	-rw-r--r--	39 bytes
requirements-journal.txt	-rw-r--r--	12 bytes
requirements-simulator.txt	-rw-r--r--	21 bytes
requirements-swh.txt	-rw-r--r--	45 bytes
requirements-test.txt	-rw-r--r--	155 bytes
requirements.txt	-rw-r--r--	358 bytes
setup.cfg	-rw-r--r--	342 bytes
setup.py	-rwxr-xr-x	2.5 KB
tox.ini	-rw-r--r--	1.6 KB

Showing with 0 additions and 0 deletions (0 / 0 diffs computed)

Computing file changes ...

https://forge.softwareheritage.org/source/swh-scheduler.git

Make scheduling policy used in schedule_recurrent configurable

README.md