Revision cf45ec629a53850d7446b4c4005c643a348a6a1a authored by Antoine R. Dumont (@ardumont) on 30 January 2020, 11:22:16 UTC, committed by Antoine R. Dumont (@ardumont) on 30 January 2020, 11:22:16 UTC
Currently, wrong "hash collisions" are happening a lot on ingestion [1] [2] [3] The last loading step (flush) is failing on most loaders (git, npm, etc...). This commits adds the retry behavior to the current pipeline storage deployed. Which should decrease the frequency of that error. The remaining hash collision which won't subside should be then real hash collisions. [1] https://sentry.softwareheritage.org/share/issue/102aace238fe4ba6b49bcc5531f7c2bf/ [2] https://sentry.softwareheritage.org/share/issue/8e8b48a1d94c465b8109e76311ecdbe7/ [3] https://sentry.softwareheritage.org/share/issue/d4f1208b7eec4b43b11e38494ff039cc/
1 parent 1608fcd
| File | Mode | Size |
|---|---|---|
| bin | ||
| docs | ||
| sql | ||
| swh | ||
| .gitignore | -rw-r--r-- | 136 bytes |
| .pre-commit-config.yaml | -rw-r--r-- | 1.1 KB |
| AUTHORS | -rw-r--r-- | 112 bytes |
| CODE_OF_CONDUCT.md | -rw-r--r-- | 3.3 KB |
| CONTRIBUTORS | -rw-r--r-- | 46 bytes |
| LICENSE | -rw-r--r-- | 34.3 KB |
| MANIFEST.in | -rw-r--r-- | 242 bytes |
| Makefile | -rw-r--r-- | 163 bytes |
| Makefile.local | -rw-r--r-- | 69 bytes |
| README.md | -rw-r--r-- | 4.2 KB |
| mypy.ini | -rw-r--r-- | 839 bytes |
| pytest.ini | -rw-r--r-- | 319 bytes |
| requirements-swh-journal.txt | -rw-r--r-- | 22 bytes |
| requirements-swh.txt | -rw-r--r-- | 73 bytes |
| requirements-test.txt | -rw-r--r-- | 305 bytes |
| requirements.txt | -rw-r--r-- | 66 bytes |
| setup.py | -rwxr-xr-x | 2.4 KB |
| tox.ini | -rw-r--r-- | 507 bytes |
Computing file changes ...
