https://forge.softwareheritage.org/source/swh-scheduler.git
Revision 889839446eb8645a5520237513a54c892d3a3104 authored by Nicolas Dandrimont on 21 January 2021, 16:45:23 UTC, committed by Valentin Lorentz on 26 January 2021, 12:31:57 UTC
This was a significant bottleneck of the simulator. To work around this,
we:

 - Generate snapshot ids consistently in the OriginModel
 - Cache the origin data locally in the simulator, to compute the
   eventfulness of visits
 - Cache the last visit time for all origins to compute the estimated
   run time of visit tasks.
1 parent c92ead5
History
Tip revision: 889839446eb8645a5520237513a54c892d3a3104 authored by Nicolas Dandrimont on 21 January 2021, 16:45:23 UTC
simulator: stop using the database as a cache for origin data
Tip revision: 8898394
File Mode Size
data
docs
sql
swh
.gitignore -rw-r--r-- 129 bytes
.pre-commit-config.yaml -rw-r--r-- 1.0 KB
AUTHORS -rw-r--r-- 112 bytes
CODE_OF_CONDUCT.md -rw-r--r-- 3.3 KB
CONTRIBUTORS -rw-r--r-- 14 bytes
LICENSE -rw-r--r-- 34.3 KB
LICENSE.Celery -rw-r--r-- 2.6 KB
MANIFEST.in -rw-r--r-- 154 bytes
Makefile -rw-r--r-- 163 bytes
README.md -rw-r--r-- 292 bytes
conftest.py -rw-r--r-- 620 bytes
mypy.ini -rw-r--r-- 750 bytes
pyproject.toml -rw-r--r-- 237 bytes
pytest.ini -rw-r--r-- 33 bytes
requirements-journal.txt -rw-r--r-- 12 bytes
requirements-simulator.txt -rw-r--r-- 21 bytes
requirements-swh.txt -rw-r--r-- 47 bytes
requirements-test.txt -rw-r--r-- 65 bytes
requirements.txt -rw-r--r-- 403 bytes
setup.cfg -rw-r--r-- 247 bytes
setup.py -rwxr-xr-x 2.5 KB
tox.ini -rw-r--r-- 673 bytes

README.md

back to top