Skip to main content

Browse the archive

https://forge.softwareheritage.org/source/swh-scheduler.git
13 March 2024, 03:48:39 UTC
Revision d107a5553414ec7f2745a739dbc82e56eb62514e authored by Nicolas Dandrimont on 16 June 2020, 08:25:08 UTC, committed by Nicolas Dandrimont on 16 June 2020, 08:25:08 UTC 
Implement storage of listed origins
 
This new API endpoint allows listers to record the origins they have seen during
their current run.

Origins are identified by the lister instance, the url of the origin, and the
type of loader that should be used to load this origin.

The implementation allows listers just send the list of origins they've
seen (with some lightweight extra information), leaving the backend to handle
whether to do an insertion or an update to an existing origin.

The current implementation doesn't disable origins that have disappeared when
doing a full listing run. This step will be done by a separate "origin garbage
collection" endpoint, which will peruse the `last_seen` field.
1 parent e0fa5c5
History
Tip revision: d107a5553414ec7f2745a739dbc82e56eb62514e authored by Nicolas Dandrimont on 16 June 2020, 08:25:08 UTC
Implement storage of listed origins
Tip revision: d107a55
File Mode Size
data
docs
sql
swh
.gitignore -rw-r--r-- 129 bytes
.pre-commit-config.yaml -rw-r--r-- 882 bytes
AUTHORS -rw-r--r-- 112 bytes
CODE_OF_CONDUCT.md -rw-r--r-- 3.3 KB
CONTRIBUTORS -rw-r--r-- 14 bytes
LICENSE -rw-r--r-- 34.3 KB
LICENSE.Celery -rw-r--r-- 2.6 KB
MANIFEST.in -rw-r--r-- 154 bytes
Makefile -rw-r--r-- 163 bytes
README.md -rw-r--r-- 292 bytes
conftest.py -rw-r--r-- 292 bytes
mypy.ini -rw-r--r-- 550 bytes
pyproject.toml -rw-r--r-- 39 bytes
pytest.ini -rw-r--r-- 54 bytes
requirements-swh.txt -rw-r--r-- 51 bytes
requirements-test.txt -rw-r--r-- 92 bytes
requirements.txt -rw-r--r-- 452 bytes
setup.cfg -rw-r--r-- 247 bytes
setup.py -rwxr-xr-x 2.3 KB
tox.ini -rw-r--r-- 664 bytes

README.md

back to top