swh:1:snp:eb70f1f85391e4b077c211bec36af0061c4bf937

sort by:
Revision Author Date Message Commit Date
e734695 Updated backport on buster-swh from debian/0.9.3-1_swh1 (unstable-swh) 06 July 2020, 08:11:52 UTC
9af4e73 Merge tag 'debian/0.9.3-1_swh1' into debian/buster-swh 06 July 2020, 08:11:51 UTC
8828f5d Updated debian changelog for version 0.9.3 06 July 2020, 08:06:13 UTC
cd6b2fc Update upstream source from tag 'debian/upstream/0.9.3' Update to upstream version '0.9.3' with Debian dir 75517c4f83b27d028160c6a892308555ff2febaf 06 July 2020, 08:06:12 UTC
8154981 New upstream version 0.9.3 06 July 2020, 08:06:11 UTC
8010848 storage: Send metrics from the origin_add endpoint Prior to this commit, since the loaders got migrated to use the main endpoint, no metrics were sent for the origin any longer. This commit fixes it. It also drops the send_metrics call from the deprecated endpoint origin_add_one (which, as an implementation details calls the other one). 06 July 2020, 07:45:40 UTC
ddec157 Updated backport on buster-swh from debian/0.9.2-1_swh1 (unstable-swh) 03 July 2020, 16:59:53 UTC
a98cc42 Merge tag 'debian/0.9.2-1_swh1' into debian/buster-swh 03 July 2020, 16:59:52 UTC
ab6706c Updated debian changelog for version 0.9.2 03 July 2020, 16:54:13 UTC
12740a4 Update upstream source from tag 'debian/upstream/0.9.2' Update to upstream version '0.9.2' with Debian dir 56ec491d91dc54522e4bc3ac723e3e3b0e6b5c47 03 July 2020, 16:54:12 UTC
6b7027b New upstream version 0.9.2 03 July 2020, 16:54:10 UTC
95fd660 pg-storage: Add missing cur parameter passing Although, this also pulled a refactoring on the insertion query as the default naive approach ended up with issues on cur already being closed [1] [1] Related to P715 Related to D3416 03 July 2020, 15:54:04 UTC
ffa9206 Updated backport on buster-swh from debian/0.9.1-1_swh1 (unstable-swh) 03 July 2020, 15:06:24 UTC
9ac73da Merge tag 'debian/0.9.1-1_swh1' into debian/buster-swh 03 July 2020, 15:06:23 UTC
b7ff639 Updated debian changelog for version 0.9.1 03 July 2020, 15:00:33 UTC
b3e9669 Update upstream source from tag 'debian/upstream/0.9.1' Update to upstream version '0.9.1' with Debian dir 02ff8c6b2b9df66b00a95c56c91b76a84d436e78 03 July 2020, 15:00:32 UTC
4a03544 New upstream version 0.9.1 03 July 2020, 15:00:30 UTC
348bc7b storage.db: Drop db.origin_visit_upsert behavior The initial desired behavior was to allow creation of origin-visit if they already had their id set. This is the what's needed for the replayer to actually work. But somehow, this left the possibility to update the origin-visit... This commit fixes it by dropping conflictual origin-visits if any. In effect, we can no longer overwrite origin-visits (pg-storage wise). Related to T2310 03 July 2020, 14:42:20 UTC
248c277 Move tests of content_metadata_* next to origin_metadata_* For consistency with the main code. 02 July 2020, 09:04:43 UTC
f2619b6 Rework 157 migration to ease replication setup Past experience showed that altering tables is more stressful than plain creation. As in here. Related to T2306 Related P707 01 July 2020, 13:41:16 UTC
90c9772 Updated backport on buster-swh from debian/0.9.0-1_swh1 (unstable-swh) 01 July 2020, 08:07:50 UTC
28344a0 Merge tag 'debian/0.9.0-1_swh1' into debian/buster-swh 01 July 2020, 08:07:50 UTC
74fba45 Updated debian changelog for version 0.9.0 01 July 2020, 08:02:09 UTC
a1d68e7 Update upstream source from tag 'debian/upstream/0.9.0' Update to upstream version '0.9.0' with Debian dir e7f3e620018c387391bd5555d28fbfd2415e640c 01 July 2020, 08:02:08 UTC
80cf105 New upstream version 0.9.0 01 July 2020, 08:02:06 UTC
312127a storage*: Drop intermediary conversion step into OriginVisit This is no longer possible as OriginVisit no longer hold the same information as OriginVisitStatus. This will allow to drop entirely those fields in the model. Related to T2310 30 June 2020, 13:54:01 UTC
953bd29 pg: use 'on conflict do nothing' strategy for duplicate metadata rows. "updates are a problem for postgresql logical replication" 30 June 2020, 13:25:53 UTC
00f97f0 Document the behavior of adding a duplicate non-intrinsic object is unspecified. 30 June 2020, 13:06:03 UTC
4c2bdad Make the code location of metadata endpoints consistent across backends. 30 June 2020, 12:56:20 UTC
ffe6b92 Add content_metadata_{add,get}. 30 June 2020, 10:31:59 UTC
869679a Add context columns to object_metadata table and object_metadata_{add,get}. Not used/tested yet; will be used when I introduce content_metadata_{get,add}. 30 June 2020, 10:31:59 UTC
27e9426 Generalize origin_metadata to allow support for other object types in the future. 30 June 2020, 10:31:21 UTC
7d410f8 Updated backport on buster-swh from debian/0.8.1-1_swh1 (unstable-swh) 30 June 2020, 08:42:24 UTC
6549fe9 Merge tag 'debian/0.8.1-1_swh1' into debian/buster-swh 30 June 2020, 08:42:23 UTC
2298df9 Updated debian changelog for version 0.8.1 30 June 2020, 08:36:45 UTC
346d50d Update upstream source from tag 'debian/upstream/0.8.1' Update to upstream version '0.8.1' with Debian dir 7f0a4728948d5075fb349093cb33b181cf6d5723 30 June 2020, 08:36:44 UTC
cced77a New upstream version 0.8.1 30 June 2020, 08:36:42 UTC
1f0e256 Work around the segmentation faults caused by pytest-coverage + multiprocessing. 30 June 2020, 08:23:25 UTC
dc1878b Make release_add support adding the same object twice in the same call This is an edge case, but the mirror infrastructure is apparently hitting it. We modify the SQL query to be properly idempotent. Also ensure in_memory and cassandra backends behave the same. Note: this revision was mostly written by Nicolas Dandrimont <nicolas@dandrimont.eu>. 29 June 2020, 15:27:21 UTC
006c612 Updated backport on buster-swh from debian/0.8.0-1_swh1 (unstable-swh) 29 June 2020, 07:49:46 UTC
cff86d2 Merge tag 'debian/0.8.0-1_swh1' into debian/buster-swh 29 June 2020, 07:49:46 UTC
8e9f87c Updated debian changelog for version 0.8.0 29 June 2020, 07:44:00 UTC
545d51b Update upstream source from tag 'debian/upstream/0.8.0' Update to upstream version '0.8.0' with Debian dir 7c90b60fe9d71a40cd885b736d00a3156a4aafad 29 June 2020, 07:43:59 UTC
57aaf30 New upstream version 0.8.0 29 June 2020, 07:43:58 UTC
10443b8 Iterate over paginated visits in batches to retrieve latest visit/snapshot This should stops the current timeouts on origin with a high number of visits. Related to T2310 26 June 2020, 15:38:22 UTC
182ee49 storage*: Open order parameter to origin-visit-get endpoint This allows clients to search from most recent to oldest visit when calling the endpoint with the "order" parameter set to "desc" (visit id desc). This keeps and explicits the existing sorting order as visit id "asc". Related to T2310 26 June 2020, 11:22:40 UTC
f75cd41 tests*: Drop obsolete origin visit fields Related to T2310 26 June 2020, 10:28:06 UTC
8620519 replayer: Drop obsolete fields from origin-visit Otherwise, we won't be able to replay them. Related T2310 26 June 2020, 07:50:38 UTC
b991e69 test_storage: Add missing tests on origin_visit_get method 25 June 2020, 12:47:11 UTC
89e9dae storage: Given origin-visit index a name to avoid future dev/prod divergence Related to D3342#inline-23217 25 June 2020, 12:37:39 UTC
12d729b Relax checks on journal writes regarding origin-visit* 25 June 2020, 12:35:38 UTC
c6e6f33 replayer: Fix isoformat datetime string for origin-visit We no longer write datetime as strings in the journal. Still, the current journal must have those old values within. Related to D3336 Related to D3345 25 June 2020, 09:19:55 UTC
e5e80ef storage*: Drop obsolete fields from origin_visit Related to T2310 25 June 2020, 08:35:18 UTC
621fc8d Deprecate the origin_add_one() endpoint This endpoint is not really useful since the origin_add() can be used instead. Using a single API endpoint would also make the API a bit more consistant (most other endpoints only provide a xxx_add endpoint) ; having a single endpoint per object_type make is enough and make the whole API simpler. 23 June 2020, 14:07:09 UTC
fb603e1 Make Storage.add_origin() return a sumary dict make it consistent with other add_xxx methods by making it return a summary dict `{"origin:add": int}`. 23 June 2020, 13:58:54 UTC
07ed7b4 Updated backport on buster-swh from debian/0.7.0-1_swh1 (unstable-swh) 22 June 2020, 14:15:07 UTC
e8e36cf Merge tag 'debian/0.7.0-1_swh1' into debian/buster-swh 22 June 2020, 14:15:07 UTC
6069d91 Updated debian changelog for version 0.7.0 22 June 2020, 14:09:33 UTC
2d3ad44 Update upstream source from tag 'debian/upstream/0.7.0' Update to upstream version '0.7.0' with Debian dir eff55695245fa9a450104fc50500133e3736ff24 22 June 2020, 14:09:32 UTC
c1f0e27 New upstream version 0.7.0 22 June 2020, 14:09:31 UTC
2d497ff test_origin: Rename appropriately tests So one can trigger tests separately by name tagging. 22 June 2020, 12:39:32 UTC
e9f4554 algos: Improve origin visit get latest visit status algorithm Prior to this commit, this looked up only the latest visit information. This now looks up across multiple visits up (from most recent visit to the oldest) until one visit which match the criteria is elected. 22 June 2020, 12:39:32 UTC
041543d test_snapshot: Do not use origin_visit_add returned result This api will be realigned with other add endpoints. 22 June 2020, 09:33:15 UTC
32fded1 algos.snapshot: Fix edge case when snapshot is not resolved Fixes [1] [1] https://sentry.softwareheritage.org/share/issue/9848d9ea23d94d6ba8855bc7a7d7d297/ 22 June 2020, 09:19:38 UTC
53c4392 Ensure ids are correct in tests' storage_data Also add an "objects" dict to easily retrieve available objects from their object_type. 22 June 2020, 08:57:47 UTC
46ac997 Fix tests' storage_data revisions one of them was actually invalid (extra_header metadata being used in hash computation) 22 June 2020, 08:57:39 UTC
19354bc SQL: replace the hash(url) index by a unique btree(url) on the origin table This ensures unicity of url in the origin table. 22 June 2020, 08:09:23 UTC
9514a1d Make sure the pagination in swh_snapshot_get_by_id uses the proper indexes 19 June 2020, 15:14:59 UTC
54f1ceb Updated backport on buster-swh from debian/0.6.0-1_swh1 (unstable-swh) 19 June 2020, 12:52:54 UTC
da48ef5 Merge tag 'debian/0.6.0-1_swh1' into debian/buster-swh 19 June 2020, 12:52:54 UTC
63e03ce Updated debian changelog for version 0.6.0 19 June 2020, 12:45:33 UTC
6ea558e Update upstream source from tag 'debian/upstream/0.6.0' Update to upstream version '0.6.0' with Debian dir 3d65a941245977b04338c5f582ef910e79429ab7 19 June 2020, 12:45:32 UTC
f6936c8 New upstream version 0.6.0 19 June 2020, 12:45:30 UTC
1600907 Move deprecated endpoint snapshot_get_latest from api endpoint to algos This allows to avoid repeating the same pattern of retrieving the last snapshot for a given origin. Note that this also makes the new function return a Snapshot model object as well. Related to T2310 19 June 2020, 09:19:58 UTC
5480b7b algos.origin: Open origin-get-latest-visit-status function This will allow to avoid repeating the same pattern of retrieving the last visit status for a given origin. Related to T2310 18 June 2020, 11:40:16 UTC
c498901 storage*: Allow origin-visit-get-latest to filter on type 18 June 2020, 10:25:12 UTC
822d96b test_origin: Align storage initialization within tests This aligns consistently the storage initialization with other tests. 18 June 2020, 06:55:34 UTC
5eec974 Updated backport on buster-swh from debian/0.5.0-1_swh1 (unstable-swh) 17 June 2020, 14:26:27 UTC
cfd9a67 Merge tag 'debian/0.5.0-1_swh1' into debian/buster-swh 17 June 2020, 14:26:27 UTC
6c7e3c1 Updated debian changelog for version 0.5.0 17 June 2020, 14:19:37 UTC
312d787 Update upstream source from tag 'debian/upstream/0.5.0' Update to upstream version '0.5.0' with Debian dir 7424c826a8fad8859e62627cf4d5d3edb0d08037 17 June 2020, 14:19:36 UTC
7406df2 New upstream version 0.5.0 17 June 2020, 14:19:35 UTC
c3d177b test_storage: Fix flakiness in round to milliseconds test util method Prior to this commit, the tests would fail [1] for no good reason [2]. This fixes it. [1] https://jenkins.softwareheritage.org/job/DSTO/job/tests/1264/console [2] microseconds would exceed a limit of 999999 from time to time 17 June 2020, 13:13:41 UTC
7319495 storage*: Add origin-visit-status-get-latest endpoint So we can read the latest origin-visit-status out of a storage Related to T2310 17 June 2020, 10:20:48 UTC
692bfa3 Fix/update the backfiller The backfiller has not been updated to match recent changes in several places. This has not been detected because there was no proper test of the backfiller function as a whole. This is now done. 17 June 2020, 09:35:35 UTC
057c6fd validate: accept model objects as well as dicts on all add endpoints This generalizes work by Antoine Dumont to all object addition endpoints, as a further step towards completely dropping the validate proxy in tests. 17 June 2020, 09:22:52 UTC
d153a80 cql: Fix blackified strings 16 June 2020, 18:11:31 UTC
5e053f8 storage: Add missing cur parameter 16 June 2020, 16:11:35 UTC
c2b673b Fix db_to_author() converter to return None is all fields are None Fix T2455. 16 June 2020, 10:32:20 UTC
fc3804c Updated backport on buster-swh from debian/0.4.0-1_swh1 (unstable-swh) 16 June 2020, 08:43:08 UTC
3491e02 Merge tag 'debian/0.4.0-1_swh1' into debian/buster-swh 16 June 2020, 08:43:07 UTC
d208787 Updated debian changelog for version 0.4.0 16 June 2020, 07:58:23 UTC
e364df3 Update upstream source from tag 'debian/upstream/0.4.0' Update to upstream version '0.4.0' with Debian dir f7b34f8df2db72a4ab983c75032bdf6f28003f95 16 June 2020, 07:58:22 UTC
ae131a9 New upstream version 0.4.0 16 June 2020, 07:58:21 UTC
8f1ac4c storage*: Drop leftover code This is no longer used, it should have been dropped with previous commits. Related to T2310 15 June 2020, 13:28:38 UTC
d6144d2 storage*: Drop origin_visit_upsert endpoint Related to T2310 15 June 2020, 12:26:42 UTC
c7f3060 storage*: Remove origin-visit-update endpoint Related to T2310 15 June 2020, 12:11:52 UTC
2bcbc82 replay: Replay origin-visit and origin-visit-status This now uses the respective origin-visit-add and origin-visit-status-add endpoints. Related to T2310 15 June 2020, 12:06:10 UTC
0183fec in_memory: Make origin-visit-status-add respect "on conflict ignore" policy Prior to this commit, that behavior was not properly tested and inconsistent between backends. All backends except in-memory were respecting it. This commit aligns the in-memory backend implementation and test it. Related to T2310 15 June 2020, 11:44:06 UTC
46a7839 test_storage: Add journal behavior coverage for origin-visit-*add This was missing some coverage on origin-visit-add and origin-visit-status-add for the journal part. Related to T2310 15 June 2020, 09:50:33 UTC
back to top