| 9cf1d67 | Jenkins for Software Heritage | 14 April 2020, 12:22:04 UTC | New upstream version 0.0.185 | 14 April 2020, 12:22:04 UTC |
| 2cc263d | David Douard | 07 April 2020, 14:17:39 UTC | test: update storage tests to (future) swh.journal 0.0.30 which will handle swh.model objects everywhere instead of dicts. Also add a BW compat version os the InMemoryJournalWriter so tests will pass with current version of swh.journal (0.0.29). | 14 April 2020, 09:37:31 UTC |
| e5e5943 | Antoine R. Dumont (@ardumont) | 14 April 2020, 08:54:14 UTC | storage.filter: Remove internal state | 14 April 2020, 08:54:14 UTC |
| 0e5731c | Jenkins for Software Heritage | 10 April 2020, 14:14:17 UTC | New upstream version 0.0.184 | 10 April 2020, 14:14:17 UTC |
| ddac3d2 | Antoine R. Dumont (@ardumont) | 10 April 2020, 09:40:39 UTC | test_retry: Add missing skipped_content_add tests | 10 April 2020, 11:53:34 UTC |
| 54b2907 | Antoine R. Dumont (@ardumont) | 08 April 2020, 13:27:08 UTC | storage*: Add flush endpoints to storage implems (backend, proxy) All storage defines one endpoint even if it's mostly noop. This avoids introspection surprises. Related to D2966 (to be consistent) | 10 April 2020, 09:08:05 UTC |
| d6ecf54 | Jenkins for Software Heritage | 09 April 2020, 10:46:26 UTC | New upstream version 0.0.183 | 09 April 2020, 10:46:26 UTC |
| b0b0313 | Antoine R. Dumont (@ardumont) | 08 April 2020, 09:27:30 UTC | test_filter: Extract the filter storage into a fixture | 09 April 2020, 07:37:20 UTC |
| 566c325 | Antoine R. Dumont (@ardumont) | 07 April 2020, 12:56:46 UTC | storage*: Add `clear_buffers` operation for proxy storages This also adds the endpoint as noop for the main backend implementations. Related to T2352 | 09 April 2020, 07:37:20 UTC |
| ed4097c | David Douard | 08 April 2020, 19:42:32 UTC | Add a pyproject.toml file to target py37 for black | 08 April 2020, 20:11:21 UTC |
| cd52a03 | David Douard | 08 April 2020, 12:33:42 UTC | Enable black - blackify all the python files, - enable black in pre-commit, - add a black tox environment. | 08 April 2020, 13:16:34 UTC |
| 0fe4665 | David Douard | 08 April 2020, 13:14:48 UTC | Fix Storage.origin_visit_update(); ensure it raises a StorageArgumentException | 08 April 2020, 13:16:34 UTC |
| be954f2 | Valentin Lorentz | 08 April 2020, 10:02:40 UTC | buffer: filter out duplicate objects. | 08 April 2020, 11:01:00 UTC |
| c51139e | David Douard | 07 April 2020, 14:09:48 UTC | Make Storage.origin_visit_update() add an OriginVisit model entity in the journal instead of a dict, to comply with next version of swh.journal (which will require swh.model objects). | 08 April 2020, 10:03:38 UTC |
| bf48cfe | David Douard | 07 April 2020, 14:09:05 UTC | Make swh/storage/storage.py flake8 compliant | 08 April 2020, 10:03:38 UTC |
| fbb51aa | David Douard | 07 April 2020, 14:06:25 UTC | Add a setup.cfg file to configure flake8 for black compatibility | 08 April 2020, 10:03:38 UTC |
| 8e8577e | Valentin Lorentz | 08 April 2020, 08:30:28 UTC | Prevent erroneous HashCollisions by using the same ctime for all rows. 'swh_content_add' tries to avoid this issue with a DISTINCT clause on the entire row; but it is useless because 'ctime' cells differ by a few microseconds. This commit ensures all ctime values are exactly the same, so they are filtered out. An alternative would be to change 'swh_content_add' to do: ``` select distinct on (sha1, sha1_git, sha256, blake2s256, length, status) sha1, sha1_git, sha256, blake2s256, length, status, ctime from tmp_content ``` instead of: ``` select distinct sha1, sha1_git, sha256, blake2s256, length, status, ctime from tmp_content ``` but this is more verbose and there's no good reason to call 'now()' for every row. | 08 April 2020, 08:30:41 UTC |
| 82b41ba | Valentin Lorentz | 02 April 2020, 14:40:00 UTC | Remove magic CassObject class, use dicts instead. | 02 April 2020, 14:40:00 UTC |
| df3207a | David Douard | 26 March 2020, 16:28:48 UTC | Adapt cassandra backend to validating model types This is required to be able to activate type validation in the model (in swh.model.model). It requires to replace the "distorded" usage that was done of model entities to build objects compatible with CqlRunner's object addition logic. Since we cannot create invalid model entities any more in this context, we add a new CassObject type (just a dict with __getattr__=__getitem__) and use it as object passed to the CqlRunner for entity types that need special care (namely Revision and Release). This should still work with swh.model v0.0.62 (without type validation) as well as the (next) v0.0.63 which will come with type valdation. | 01 April 2020, 12:19:41 UTC |
| 20baa1b | David Douard | 30 March 2020, 13:34:55 UTC | test: convert test_converts.py to pytest style | 01 April 2020, 12:19:41 UTC |
| fa4a043 | David Douard | 25 March 2020, 15:15:46 UTC | test: get rid of normalized_xxx in tests This is not needed any more with properly typed test data. | 01 April 2020, 12:19:41 UTC |
| fcca905 | David Douard | 25 March 2020, 15:08:06 UTC | test: ensure timestamp in test data are properly typed according to model declaration, a timestamp must be a dict with 2 keys, 'seconds' and 'microseconds'. Also add a few more tests for the date_to_db helper function so that the test coverage of this later remains. | 01 April 2020, 12:19:41 UTC |
| 377e6a8 | Antoine R. Dumont (@ardumont) | 01 April 2020, 09:54:33 UTC | tests: Prepare tests for origin_visit_update objects Related to T2310 | 01 April 2020, 09:56:05 UTC |
| 9c22156 | Antoine R. Dumont (@ardumont) | 27 March 2020, 16:28:56 UTC | storage*: Add missing type annotations on origin_visit_get* endpoints | 01 April 2020, 09:02:03 UTC |
| ff0a538 | Antoine R. Dumont (@ardumont) | 01 April 2020, 08:48:29 UTC | tests: Skip internal origin_visit_update model object generation Beside making the tests fail, they are not helpful right now. This commit will avoid the current master build from breaking. Also, the model bump is a mandatory preparatory work for making the origin visit immutable. Related to T2310 | 01 April 2020, 08:52:54 UTC |
| 2856004 | Antoine R. Dumont (@ardumont) | 31 March 2020, 14:28:25 UTC | Ensure visit id is set in origin_visit_upsert before journal writes | 31 March 2020, 14:28:25 UTC |
| acf057e | Antoine R. Dumont (@ardumont) | 31 March 2020, 13:54:30 UTC | storage*: Unify validation exception capture across storages | 31 March 2020, 13:56:35 UTC |
| 81e7575 | Antoine R. Dumont (@ardumont) | 31 March 2020, 13:43:45 UTC | storage*: Stop duplication and use storage.utils.now function | 31 March 2020, 13:43:45 UTC |
| eb82792 | Antoine R. Dumont (@ardumont) | 30 March 2020, 18:30:44 UTC | cassandra/cql: Simplify type using Iterator | 31 March 2020, 13:02:30 UTC |
| 46fa27e | Antoine R. Dumont (@ardumont) | 27 March 2020, 13:40:36 UTC | storage*: Add types to origin_visit_get | 31 March 2020, 12:58:55 UTC |
| 4bdde50 | Antoine R. Dumont (@ardumont) | 26 March 2020, 07:57:18 UTC | storage*: Align origin_visit_update interface and implementations This also adds an unused (yet) optional parameter date. It will soon be used in the context of origin_visit_update use. Related to T2310 | 31 March 2020, 12:42:08 UTC |
| 69862b0 | Antoine R. Dumont (@ardumont) | 27 March 2020, 14:08:35 UTC | storage: Define a now() function | 31 March 2020, 12:35:00 UTC |
| 8e8e3a9 | Antoine R. Dumont (@ardumont) | 31 March 2020, 12:34:22 UTC | test_retry: Remove unused import | 31 March 2020, 12:34:22 UTC |
| c53433d | Antoine R. Dumont (@ardumont) | 31 March 2020, 10:51:26 UTC | test_retry: Use datetime instead of string Reuse a date_visit from the sample storage | 31 March 2020, 12:24:36 UTC |
| 623a1b7 | David Douard | 23 March 2020, 09:52:40 UTC | test: add a small test to check for type validation (using release_add) | 27 March 2020, 14:56:01 UTC |
| 90c4112 | David Douard | 20 March 2020, 10:11:07 UTC | validate: fix type annotation for origin_visit_add date argument is expected to be a datetime. | 27 March 2020, 08:50:12 UTC |
| 1916fd7 | David Douard | 20 March 2020, 10:05:19 UTC | validate: ensure StorageArgumentException is always encodable by embeding a string representation of the original Exception as StorageArgumentException args instead of the original exc.args since this can contain any python (possibli non-serializable) object. This is needed e.g. when swh.model has runtime type validation. | 27 March 2020, 08:50:12 UTC |
| c67fe21 | David Douard | 20 March 2020, 10:03:17 UTC | writer: fix skipped_content_add type declaration to use SkippedContent instead of plain Content. | 27 March 2020, 08:50:12 UTC |
| 19be96f | David Douard | 20 March 2020, 09:48:37 UTC | tests: fix types of several test data sets these are currently accepted by swh.model, but won't be any more as soon as we activate type validation in swh.model. | 27 March 2020, 08:50:12 UTC |
| 982023a | Jenkins for Software Heritage | 27 March 2020, 06:13:14 UTC | New upstream version 0.0.182 | 27 March 2020, 06:13:14 UTC |
| 570dce2 | Nicolas Dandrimont | 26 March 2020, 19:42:39 UTC | Shut down cassandra connection before closing the fixture down | 26 March 2020, 19:43:30 UTC |
| ce5d2bf | Antoine R. Dumont (@ardumont) | 26 March 2020, 13:36:45 UTC | storage*: Update origin_visit_update to make status parameter mandatory This actually aligns with the origin_visit model whose status is already mandatory. | 26 March 2020, 15:13:45 UTC |
| 40a7569 | Antoine R. Dumont (@ardumont) | 26 March 2020, 07:53:50 UTC | test: Adapt origin validation test according to latest model changes Origin model no longer allows to have a type. Related to f533f62bbf114cfcc29f7c72307c4dfbe99cf048 | 26 March 2020, 14:44:24 UTC |
| 0a22e72 | Valentin Lorentz | 26 March 2020, 12:04:01 UTC | Respec discovery_date as a Python datetime instead of an ISO string. For consistency with the rest of the API. | 26 March 2020, 12:04:13 UTC |
| 74fd15e | Valentin Lorentz | 26 March 2020, 09:45:56 UTC | origin_visit_add: Add missing db/cur argument to call to origin_get. | 26 March 2020, 10:07:58 UTC |
| c99ec11 | Jenkins for Software Heritage | 25 March 2020, 09:03:41 UTC | New upstream version 0.0.181 | 25 March 2020, 09:03:41 UTC |
| fd29fcb | Antoine R. Dumont (@ardumont) | 24 March 2020, 12:32:33 UTC | storage*: Hex encode content hashes in HashCollision exception Related to T2332#42793 | 24 March 2020, 17:40:48 UTC |
| b7477e5 | Valentin Lorentz | 24 March 2020, 11:14:38 UTC | Add format of discovery_date in the metadata specification. It was not specified what the format should be. | 24 March 2020, 11:14:38 UTC |
| 92a87ea | Valentin Lorentz | 23 March 2020, 14:50:01 UTC | Store the value of token(partition_key) in skipped_content_by_* table, instead of three hashes. As was done for content_by_*. | 23 March 2020, 14:51:13 UTC |
| a24ab3f | Valentin Lorentz | 10 March 2020, 12:51:15 UTC | Store the value of token(partition_key) in content_by_* table, instead of three hashes. That's a big win in terms of disk space, and shouldn't affect performance negatively. | 23 March 2020, 14:16:46 UTC |
| a72370d | Jenkins for Software Heritage | 18 March 2020, 17:45:34 UTC | New upstream version 0.0.180 | 18 March 2020, 17:45:34 UTC |
| 456e15a | Nicolas Dandrimont | 18 March 2020, 17:10:36 UTC | Don't double-count added origins in origin_add origin_add_one already counts origins; this other send_metric would have us count added origins twice. | 18 March 2020, 17:10:36 UTC |
| d99f08b | Nicolas Dandrimont | 18 March 2020, 17:07:53 UTC | Don't count origins len(url) times when calling origin_add_one I guess the `origins` variable name was carried over from a refactoring, but it doesn't match what db.origin_add actually returns. Overall this variable name made us overcount origins a little. | 18 March 2020, 17:08:32 UTC |
| b259b70 | Jenkins for Software Heritage | 18 March 2020, 15:50:48 UTC | New upstream version 0.0.179 | 18 March 2020, 15:50:48 UTC |
| 209de5d | Nicolas Dandrimont | 18 March 2020, 14:36:01 UTC | Serialize objstorage and database writes in content_add Considering that the objstorage is idempotent, and that there's no rollback feature, intermixing both "transactions" has no concrete benefit. This avoids doing database transactions that are longer than needed. | 18 March 2020, 14:36:01 UTC |
| aaa0e54 | Nicolas Dandrimont | 18 March 2020, 11:04:46 UTC | Don't nest transactions in content_add/skipped_content_add Seems like the cur/db arguments have been missed in one of the various refactorings, creating separate transactions for these function calls. | 18 March 2020, 11:04:46 UTC |
| 1dbb732 | Nicolas Dandrimont | 18 March 2020, 11:04:05 UTC | Don't create a transaction for content_get_partition It just calls out to another function which, itself, creates a transaction. | 18 March 2020, 11:04:05 UTC |
| 9b3735b | Antoine Lambert | 17 March 2020, 14:34:12 UTC | requirements-swh.txt: Use >= instead of == for swh-core version check This fixes installation of swh modules in virtualenv when executing "pip install $(./bin/pip-swh-packages)" in swh-environment. | 17 March 2020, 14:34:25 UTC |
| 0e68cbe | Jenkins for Software Heritage | 16 March 2020, 11:59:16 UTC | New upstream version 0.0.178 | 16 March 2020, 11:59:16 UTC |
| da98f5f | Antoine R. Dumont (@ardumont) | 12 March 2020, 18:54:21 UTC | origin_visit_add: Adapt endpoint signature to return OriginVisit Prior to this commit, there was: - no signature in the method - discrepancy between checks on the different backend origin_visit_add endpoint is now typed ``` def origin_visit_add( self, origin_url: str, date: Union[str, datetime.datetime], type: str) -> OriginVisit: ``` This also: - renames appropriately the origin_url parameter (removing 1 FIXME) - align backend implementations' check which were different | 13 March 2020, 13:09:42 UTC |
| 0456cce | Antoine R. Dumont (@ardumont) | 11 March 2020, 16:34:53 UTC | origin_visit_upsert: Use OriginVisit object as input This aligns with other `_add` endpoints. Only the journal depends on this. Related to D2812#67298 | 12 March 2020, 18:05:26 UTC |
| aa39be1 | David Douard | 10 March 2020, 15:44:07 UTC | storage/writer: refactor JournalWriter.content_add to send model objects to the journal writer, as it already does with other objet types (instead of dicts). | 10 March 2020, 15:44:07 UTC |
| a97781d | David Douard | 10 March 2020, 15:42:49 UTC | storage/validate: small code formatting | 10 March 2020, 15:42:49 UTC |
| 5ab89b0 | Jenkins for Software Heritage | 10 March 2020, 10:48:10 UTC | New upstream version 0.0.177 | 10 March 2020, 10:48:10 UTC |
| 05a4fca | Antoine R. Dumont (@ardumont) | 06 March 2020, 12:58:51 UTC | storage: Identify and provide the collision hashes in exception This matches, what's done in other storage backends. There is no consistency for now though, storage backends provides as HashCollision exception parameters the content information as: - cassandra: (algo: str, hash_id: bytes, Content as cassandra Row) - in_memory: (algo: str, hash_id: bytes, Content as Tuple[str, bytes] - pgstorage: algo: Optional[str] Opening this diff to discuss how to properly land this. | 10 March 2020, 08:27:55 UTC |
| 88fe942 | Nicolas Dandrimont | 05 March 2020, 14:16:48 UTC | Guarantee the order of results for revision_get and release_get It's a bit silly, but we depend on it for some tests. | 09 March 2020, 15:53:34 UTC |
| 7ee3972 | Valentin Lorentz | 06 March 2020, 13:59:21 UTC | Mock calls to time.sleep in retry tests. It makes the tests faster. | 06 March 2020, 13:59:21 UTC |
| 6fe9de4 | Valentin Lorentz | 06 March 2020, 13:45:25 UTC | Fix retry tests. mock_memory.has_calls does not exist; so calling it returns a MagicMock. | 06 March 2020, 13:58:33 UTC |
| 3b8b718 | David Douard | 06 March 2020, 08:29:09 UTC | sql: do not attempt to create the plpgsql lang if already exists This is needed in case the pg user is not super user (but language already exists.) | 06 March 2020, 08:29:09 UTC |
| 8e41bcc | Nicolas Dandrimont | 02 March 2020, 14:18:02 UTC | Update requirement on swh.core for RPCClient method overrides | 02 March 2020, 14:18:02 UTC |
| 2b64be8 | Jenkins for Software Heritage | 28 February 2020, 15:21:25 UTC | New upstream version 0.0.176 | 28 February 2020, 15:21:25 UTC |
| 5222352 | Valentin Lorentz | 27 February 2020, 15:59:52 UTC | Use Content.hashes() instead of Content.to_dict() where it makes sense. .hashes() returns a subset of .to_dict(), so it was accidentally used instead. | 27 February 2020, 15:59:52 UTC |
| d096542 | Valentin Lorentz | 27 February 2020, 15:58:00 UTC | Make the RPC client and objstorage helper fetch Content.data. This is needed when a lazy subclass of Content is used, eg. from swh.model.from_disk. | 27 February 2020, 15:58:00 UTC |
| 3996e5d | Valentin Lorentz | 27 February 2020, 15:56:06 UTC | Move ctime out of the validation proxy. It's not the right place to set the ctime (it should be on the server side). | 27 February 2020, 15:56:06 UTC |
| caf51a0 | Valentin Lorentz | 27 February 2020, 15:04:08 UTC | Accept cassandra-driver >= 3.22. The bug that affected us in 3.21 is resolved ( https://datastax-oss.atlassian.net/browse/PYTHON-1205 ), so we can now use v3.22 to get wheels. | 27 February 2020, 15:04:08 UTC |
| 79e1f7c | Jenkins for Software Heritage | 20 February 2020, 13:18:32 UTC | New upstream version 0.0.175 | 20 February 2020, 13:18:32 UTC |
| b093a5a | Antoine Lambert | 20 February 2020, 12:24:02 UTC | retry: Add support for tenacity < 5.0 This fixes swh-storage debian package build on buster as python3-tenacity version is 4.12. | 20 February 2020, 12:38:48 UTC |
| a20779f | Jenkins for Software Heritage | 19 February 2020, 15:00:30 UTC | New upstream version 0.0.174 | 19 February 2020, 15:00:30 UTC |
| 7cf0864 | Valentin Lorentz | 14 February 2020, 16:26:09 UTC | Add support for (de)serializing swh-model in RPC calls. This allows running the validating proxy on the client side instead of the server side. | 18 February 2020, 13:22:33 UTC |
| 80befa5 | Valentin Lorentz | 12 February 2020, 15:49:01 UTC | Make storage proxies use swh-model objects instead of dicts. This means that instead of having the validation proxy right before the backend class, it must now be at the beginning of pipelines. | 18 February 2020, 12:45:46 UTC |
| 29b0948 | Valentin Lorentz | 12 February 2020, 15:18:29 UTC | Fix FilteringProxy to not drop skipped-contents with a missing sha1_git. Passes them all to the backend instead of silently dropping them all if any of them is not missing. | 18 February 2020, 12:45:22 UTC |
| 51b2016 | Valentin Lorentz | 12 February 2020, 15:08:10 UTC | Fix inconsistent behavior of skipped_content_missing across backends. Two fixes: * in-mem ignored None keys * cassandra yielded input dicts as-is instead of a dict with just the hashes | 14 February 2020, 16:49:05 UTC |
| 03c2a02 | Valentin Lorentz | 06 February 2020, 13:22:29 UTC | Re-raise StorageArgumentException through API calls. So clients will get a nice exception looking like the original one, instead of generic RemoteApiError. | 12 February 2020, 15:51:26 UTC |
| b668651 | Jenkins for Software Heritage | 12 February 2020, 13:13:45 UTC | New upstream version 0.0.172 | 12 February 2020, 13:13:45 UTC |
| 652ecf0 | Antoine R. Dumont (@ardumont) | 07 February 2020, 01:44:43 UTC | storages: Refactor journal operations with a dedicated writer collab Prior to this commit, the code was triplicated across the storage backends. Now all storages use the same collaborator whose concern is writing to the storage. Could be a stepping stone to make that a proxy storage. | 11 February 2020, 14:40:36 UTC |
| adcbf95 | Valentin Lorentz | 11 February 2020, 13:12:12 UTC | Fix RecursionError when storage proxies are deepcopied or unpickled. They both get attributes (eg. __setattr__) before setting any attribute, and don't call the constructor; so self.storage is not set when __getattr__ is called for the first times. | 11 February 2020, 13:12:12 UTC |
| de7797d | Antoine R. Dumont (@ardumont) | 10 February 2020, 16:17:59 UTC | tests: Remove print statement | 10 February 2020, 16:18:33 UTC |
| 5b3c940 | Antoine R. Dumont (@ardumont) | 10 February 2020, 12:18:21 UTC | storages: Refactor objstorage operations with a dedicated collaborator Prior to this commit, the code was triplicated across the storage backends. Now all storages use the same collaborator whose concern is writing to the objstorage. | 10 February 2020, 12:38:29 UTC |
| 68ff23c | Valentin Lorentz | 07 February 2020, 15:35:20 UTC | Add a validation proxy for _add() methods. It converts input dictionaries into swh-model objects, which validates them and raises an appropriate error. This removes duplicated validation code that is currently present in all three storage backends. It also uses well-defined object types instead of loose dicts, which gives these _add() methods a more strict type. | 10 February 2020, 11:03:49 UTC |
| 6383637 | Valentin Lorentz | 06 February 2020, 12:26:47 UTC | Unify exception raised by invalid input to API endpoints. This is a first step toward not pickling exceptions. | 07 February 2020, 10:40:23 UTC |
| be225df | Jenkins for Software Heritage | 06 February 2020, 14:07:35 UTC | New upstream version 0.0.171 | 06 February 2020, 14:07:35 UTC |
| 2b029b7 | Valentin Lorentz | 05 February 2020, 11:17:59 UTC | Split 'content_add' method into 'content_add' and 'skipped_content_add'. Respectively to add present content and skipped content. This simplifies the logic of both methods, and is a necessary step to typing / using swh-model objects everywhere, as contents have quite different attributes depending on whether they are present or missing. | 06 February 2020, 13:29:31 UTC |
| 93ea487 | Valentin Lorentz | 04 February 2020, 12:38:32 UTC | Increase Cassandra requests timeout to 1 second. 100ms worked fine so far, but we're starting to get some timeouts on the Azure test cluster. Multiplying the timeout by 10 should give us ample room to work with. | 04 February 2020, 12:38:32 UTC |
| a66e16c | Jenkins for Software Heritage | 03 February 2020, 13:23:46 UTC | New upstream version 0.0.170 | 03 February 2020, 13:23:46 UTC |
| b315f9d | Valentin Lorentz | 31 January 2020, 16:09:31 UTC | Tune Cassandra test config for lower test latency. | 03 February 2020, 12:31:28 UTC |
| 25941d5 | Valentin Lorentz | 03 February 2020, 11:25:49 UTC | Make tests reuse the same keyspace/schema instead of recreating it for each test. This makes tests run 16 times faster than https://forge.softwareheritage.org/D2612 (which is itself 3 times faster than this commit's parent) | 03 February 2020, 11:26:24 UTC |
| eb155ad | Valentin Lorentz | 21 January 2020, 13:59:26 UTC | Add Cassandra backend. | 31 January 2020, 15:05:53 UTC |
| 523f2eb | Jenkins for Software Heritage | 30 January 2020, 13:26:21 UTC | New upstream version 0.0.169 | 30 January 2020, 13:26:21 UTC |
| cf45ec6 | Antoine R. Dumont (@ardumont) | 30 January 2020, 11:22:16 UTC | retry: Add retry behavior on pipeline storage with flushing failure Currently, wrong "hash collisions" are happening a lot on ingestion [1] [2] [3] The last loading step (flush) is failing on most loaders (git, npm, etc...). This commits adds the retry behavior to the current pipeline storage deployed. Which should decrease the frequency of that error. The remaining hash collision which won't subside should be then real hash collisions. [1] https://sentry.softwareheritage.org/share/issue/102aace238fe4ba6b49bcc5531f7c2bf/ [2] https://sentry.softwareheritage.org/share/issue/8e8b48a1d94c465b8109e76311ecdbe7/ [3] https://sentry.softwareheritage.org/share/issue/d4f1208b7eec4b43b11e38494ff039cc/ | 30 January 2020, 11:22:16 UTC |
| 3e6d2bf | Jenkins for Software Heritage | 30 January 2020, 10:25:29 UTC | New upstream version 0.0.168 | 30 January 2020, 10:25:29 UTC |
