| dc7c1e0 | Valentin Lorentz | 05 June 2019, 11:18:14 UTC | Add test for snapshot/release counters. | 17 June 2019, 09:00:55 UTC |
| aec9571 | Valentin Lorentz | 05 June 2019, 09:41:50 UTC | Add origin_visit counter to the in-mem storage. | 17 June 2019, 09:00:55 UTC |
| c541224 | Valentin Lorentz | 13 June 2019, 14:27:50 UTC | Actually run the remote-pg-storage tests on a remote-pg. They were run on a local pg instead of via the HTTP API. | 17 June 2019, 08:40:27 UTC |
| 34c2c34 | Antoine R. Dumont (@ardumont) | 15 June 2019, 06:33:43 UTC | 136: Drop the extra 'text' mention which fails the alter query | 15 June 2019, 06:34:26 UTC |
| c891a5b | Antoine R. Dumont (@ardumont) | 15 June 2019, 06:28:40 UTC | 135: Dropping production index after new ones replacing it are built Also, the index creations must be done sequentially. Those new indexes need to be built prior to remove the old index. Otherwise, we break the api (db answering too slow leads to the api breaking). | 15 June 2019, 06:34:26 UTC |
| 2d610ab | Valentin Lorentz | 13 June 2019, 14:31:07 UTC | Add a new origin_visit_get_latest endpoint. snapshot_get_latest did two things: find the latest origin, and get its snapshot. Now, it only calls origin_visit_get_latest, then snapshot_get. snapshot_get_latest should also be deprecated, because it only fetches some of the branches (for large snapshots). Finally, with the new data model (types are associated to visits, not origins), origin_visit_get_latest is required to know the type of the loader that got the snapshot (needed in swh-indexer). Also, note that origin_visit_get_latest only supports URLs as argument, not origin ids. Since we are migrating away from origin ids, I don't think it makes sense to add support for origin ids in a new endpoint. | 14 June 2019, 11:28:49 UTC |
| df461d1 | Valentin Lorentz | 14 June 2019, 11:12:48 UTC | Ignore Hypothesis timeout error for test_add_arbitrary. | 14 June 2019, 11:13:10 UTC |
| 5e16e60 | Valentin Lorentz | 05 June 2019, 09:35:32 UTC | Add test for date parsing of origin_visit_add. | 13 June 2019, 12:10:03 UTC |
| 16b6dc5 | Valentin Lorentz | 05 June 2019, 11:23:19 UTC | Add tests for content_missing. | 13 June 2019, 12:08:36 UTC |
| 9afe736 | Valentin Lorentz | 05 June 2019, 11:40:18 UTC | Test date parsing of origin_metadata_add. | 13 June 2019, 12:04:12 UTC |
| 34b8c54 | Valentin Lorentz | 05 June 2019, 11:33:13 UTC | Add a test checking that directory_ls does not recurse into subdirs by default. | 12 June 2019, 13:44:28 UTC |
| 21e26e3 | Valentin Lorentz | 05 June 2019, 11:39:23 UTC | Test that content_find filters out contents that don't match. | 12 June 2019, 13:21:23 UTC |
| eed16c2 | Valentin Lorentz | 05 June 2019, 11:16:05 UTC | Add test for ctime in content_add. | 12 June 2019, 13:07:44 UTC |
| fb88f61 | Valentin Lorentz | 05 June 2019, 11:26:54 UTC | Add test for content_add when one of the content is already known. | 12 June 2019, 12:29:58 UTC |
| cb48f63 | Valentin Lorentz | 05 June 2019, 11:21:30 UTC | Add test for snapshot_get_branches with both filter and pagination. Reviewers: #reviewers, douardda Reviewed By: #reviewers, douardda Differential Revision: https://forge.softwareheritage.org/D1543 | 12 June 2019, 12:26:37 UTC |
| aef6dbf | Antoine R. Dumont (@ardumont) | 11 June 2019, 13:32:54 UTC | upgrades/136: Upgrade update instructions | 11 June 2019, 13:33:15 UTC |
| e4e5b5c | Valentin Lorentz | 11 June 2019, 13:24:49 UTC | Mark network tests, so they can be disabled. | 11 June 2019, 13:24:49 UTC |
| 5a9f544 | Valentin Lorentz | 06 June 2019, 15:05:03 UTC | Add support for using URL instead of ID in snapshot_get_latest. | 11 June 2019, 10:01:50 UTC |
| 0386716 | Valentin Lorentz | 06 June 2019, 15:03:22 UTC | Add support for using URL instead of ID in origin_visit_{add,update}. | 11 June 2019, 10:01:08 UTC |
| 21b5142 | Valentin Lorentz | 06 June 2019, 13:30:13 UTC | Add a 'type' column to origin_visit. This is a step toward dropping the 'type' column of origins. | 11 June 2019, 10:00:00 UTC |
| d8f0a7b | David Douard | 11 June 2019, 08:09:55 UTC | storage: small typo in Storage.content_add_metadata's doc string | 11 June 2019, 08:11:58 UTC |
| 710e0bf | Valentin Lorentz | 06 June 2019, 12:08:27 UTC | Add support for omitting origin type when calling origin_get. | 06 June 2019, 14:21:07 UTC |
| 3349788 | Valentin Lorentz | 05 June 2019, 09:34:18 UTC | Make origin_visit_upsert parse date strings. | 05 June 2019, 12:31:11 UTC |
| 75ac20f | Valentin Lorentz | 05 June 2019, 11:25:05 UTC | Add test for content_get on missing content. | 05 June 2019, 11:45:23 UTC |
| 09666c0 | Valentin Lorentz | 05 June 2019, 11:21:30 UTC | Add test for snapshot_get_branches with both filter and pagination. | 05 June 2019, 11:21:30 UTC |
| 909c69b | Valentin Lorentz | 04 June 2019, 13:14:40 UTC | Fix recursive entry names in in_memory.py's directory_entry_get_by_path | 04 June 2019, 13:14:40 UTC |
| efda98e | Valentin Lorentz | 29 May 2019, 12:52:46 UTC | Fix origin_search in the in-mem storage to search for sub-strings. | 04 June 2019, 11:29:24 UTC |
| 3556b74 | Valentin Lorentz | 29 May 2019, 12:54:47 UTC | Unify argument names of the pg and in-mem storage. Argument names matter for the remote API. | 29 May 2019, 16:13:03 UTC |
| 17d4112 | David Douard | 17 May 2019, 12:43:09 UTC | Update the README file especially for latest CLI modifications. Add a slightly more detailed step-by-step 'quick start' section. | 17 May 2019, 13:08:59 UTC |
| 289b233 | David Douard | 17 May 2019, 12:42:26 UTC | cli: small improvements of 'rpc-serve --help' message | 17 May 2019, 13:07:24 UTC |
| 39a1c45 | Valentin Lorentz | 17 May 2019, 12:53:46 UTC | Make content_find return an empty list instead of None. | 17 May 2019, 12:53:46 UTC |
| ea3f2d3 | Valentin Lorentz | 17 May 2019, 12:51:17 UTC | Update import of swh.core.db.tests.db_testing. | 17 May 2019, 12:51:17 UTC |
| 01ed375 | Valentin Lorentz | 17 May 2019, 11:32:55 UTC | Update import of swh.core.api.tests.server_testing. | 17 May 2019, 11:32:59 UTC |
| fa02fa4 | David Douard | 15 May 2019, 15:08:04 UTC | cli: add a 'storage' cli group with a 'rpc-serve' subcommand we do add this 'storage' group even if it contains only one subcommand (serve) for UI consistency: swh storage rpc-serve We do keep the main swh-storage console_script entry point as is for bw compat, however. | 16 May 2019, 15:28:29 UTC |
| 02134a7 | mihir(faux__) | 24 March 2019, 16:17:31 UTC | Changes the output of content_find method to a list in case of hash collisions and makes the sql query on python side and added test duplicate input, colliding sha256 and colliding blake2s256 | 16 May 2019, 09:06:06 UTC |
| 85b3da9 | David Douard | 07 May 2019, 15:12:11 UTC | bump dependency to swh.core[db,http] >= 0.0.60 so tests are not broken when next swh.core version is published. | 07 May 2019, 15:12:11 UTC |
| 2e70516 | Antoine Lambert | 23 April 2019, 14:56:29 UTC | algos.revisions_walker: Add methods to query produced history state Add methods is_history_truncated and missing_revisions to check if the produced history got truncated or not. Closes T1677 | 06 May 2019, 10:02:21 UTC |
| 6510b5e | Antoine Lambert | 18 April 2019, 14:17:16 UTC | algos.revisions_walker: Handle truncated/shallow histories It might exist cases where a revision's parent is referenced but doesn't exist in the archive resulting in a truncated/shallow history when walking on the revisions graph. That commit ensures the revision walkers will not end up in errors when encountering such cases. Closes T1675 | 18 April 2019, 15:30:17 UTC |
| 685b601 | Valentin Lorentz | 15 April 2019, 10:43:31 UTC | Make revision_add and directory_add call `_missing` using the same transaction. | 16 April 2019, 08:27:07 UTC |
| 7baf562 | Nicolas Dandrimont | 12 April 2019, 10:03:09 UTC | Update coverage gitignore | 12 April 2019, 10:03:09 UTC |
| 58b49f9 | Nicolas Dandrimont | 11 April 2019, 09:42:19 UTC | Use the proper order to record content byte addition The current way the value is returned creates a metric whose name is backwards. | 11 April 2019, 12:55:43 UTC |
| 63f50a5 | Nicolas Dandrimont | 11 April 2019, 10:15:49 UTC | Fix backwards-compatibility for snapshot_add in the server | 11 April 2019, 12:55:43 UTC |
| cf99702 | Valentin Lorentz | 09 April 2019, 14:40:49 UTC | Use the db_transaction decorator on all _add() methods. So they gracefully release the connection on error instead of relying on reference-counting to call the Db's `__del__` (which does not happen in Hypothesis tests) because a ref to it is kept via the traceback object. | 09 April 2019, 16:06:24 UTC |
| dab5bf8 | Valentin Lorentz | 09 April 2019, 14:36:31 UTC | Rewrite skipped_content_missing to not use a temporary table. | 09 April 2019, 16:05:46 UTC |
| 5aacb79 | Valentin Lorentz | 09 April 2019, 14:44:03 UTC | Add a test that adds arbitrary objects to the storage. | 09 April 2019, 15:58:28 UTC |
| cd0d309 | Valentin Lorentz | 09 April 2019, 14:24:36 UTC | Fix person name clashes in revision_add and release_add. When two revisions/releases have authors with the same fullname but different names (which should never happen in practice, but since name/email parsing is on a best-effort basis, you never know...). Bug found by Hypothesis. | 09 April 2019, 15:49:38 UTC |
| 70c43ba | Valentin Lorentz | 09 April 2019, 14:42:19 UTC | Fix order of results of person_get. | 09 April 2019, 15:31:07 UTC |
| 8672a27 | Valentin Lorentz | 09 April 2019, 14:28:38 UTC | Add test for skipped_content_missing. | 09 April 2019, 15:27:12 UTC |
| 95c0b27 | Valentin Lorentz | 09 April 2019, 14:12:00 UTC | Fix support of releases with no author or date. Bug found by Hypothesis. | 09 April 2019, 15:24:12 UTC |
| ba3a9af | Antoine R. Dumont (@ardumont) | 09 April 2019, 09:48:49 UTC | listener: Remove | 09 April 2019, 15:19:54 UTC |
| 681070f | Valentin Lorentz | 09 April 2019, 14:09:06 UTC | Make test_origin_get_range run faster (less objects). | 09 April 2019, 14:09:06 UTC |
| 2aa256b | Valentin Lorentz | 08 April 2019, 20:24:10 UTC | Fix hypothesis tests. Why does test_origin_get_range even work? | 08 April 2019, 20:24:10 UTC |
| 00e5f33 | Valentin Lorentz | 08 April 2019, 13:40:24 UTC | Make test_origin_get_range run faster. | 08 April 2019, 13:40:24 UTC |
| 92a575f | Valentin Lorentz | 05 April 2019, 18:00:42 UTC | Move Hypothesis strategies to swh-model. | 08 April 2019, 13:23:51 UTC |
| a574fe5 | Valentin Lorentz | 05 April 2019, 14:19:38 UTC | Add a new endpoint origin_visit_upsert to add an origin_visit with a specific id. Also allows skipping an extra origin_visit_update query to set snapshot/metadata/status. | 05 April 2019, 15:40:28 UTC |
| d110b58 | Valentin Lorentz | 04 April 2019, 18:42:32 UTC | Make content_add_metadata require a ctime argument. This makes Python set the ctime instead of pgsql. | 05 April 2019, 09:15:18 UTC |
| a62b1ad | Valentin Lorentz | 04 April 2019, 18:44:56 UTC | Prevent the 'snapshots' hypothesis strategy from generating loops. | 04 April 2019, 18:44:56 UTC |
| ee62e05 | Valentin Lorentz | 04 April 2019, 13:54:03 UTC | Add a new endpoint content_add_metadata. | 04 April 2019, 15:15:51 UTC |
| 5913718 | Valentin Lorentz | 04 April 2019, 13:52:07 UTC | Refactor content_add's code into smaller functions. | 04 April 2019, 15:10:50 UTC |
| 4c8a0ac | Valentin Lorentz | 04 April 2019, 13:14:57 UTC | Make origin_add copy objects before changing them. hypothesis seems to reuse objects across tests now. | 04 April 2019, 13:14:57 UTC |
| 81a9954 | Valentin Lorentz | 03 April 2019, 11:38:58 UTC | Don't leak origin ids to the journal. | 04 April 2019, 09:47:18 UTC |
| e9a3198 | Antoine R. Dumont (@ardumont) | 03 April 2019, 14:35:55 UTC | api.server: Add tests around send_metric function | 03 April 2019, 15:56:49 UTC |
| 1a401c0 | Antoine R. Dumont (@ardumont) | 03 April 2019, 14:09:16 UTC | api.server: Make send_metric be in charge of the metric parsing | 03 April 2019, 14:20:12 UTC |
| 7d6d894 | Antoine R. Dumont (@ardumont) | 03 April 2019, 13:47:06 UTC | api.server: Use constants for all metrics | 03 April 2019, 13:50:26 UTC |
| 1da7b50 | Antoine R. Dumont (@ardumont) | 03 April 2019, 13:42:03 UTC | api.server: Rename send_metric as we really send only 1 metric | 03 April 2019, 13:50:26 UTC |
| ff05590 | Antoine R. Dumont (@ardumont) | 03 April 2019, 13:37:43 UTC | api.server: Rename constant | 03 April 2019, 13:38:18 UTC |
| 2f1e763 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:59:55 UTC | api.server: Refactor repetition into a send_metrics function | 03 April 2019, 13:06:02 UTC |
| 6dee5f5 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:50:37 UTC | api.server: Fix wrong api call | 03 April 2019, 13:06:02 UTC |
| 04792f8 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:43:23 UTC | server: Add metrics on tool_add endpoint | 03 April 2019, 13:06:02 UTC |
| ee934b6 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:42:43 UTC | storage: Make tool_add endpoint return a list instead of a generator This aligns with other endpoints and allow to extract metrics easily | 03 April 2019, 13:06:02 UTC |
| 9b35b2d | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:39:25 UTC | server: Add metrics on origin_metadata_add & metadata_provider_add | 03 April 2019, 13:06:02 UTC |
| 6064a31 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:28:34 UTC | api.{client,server}: Remove dead code about entity | 03 April 2019, 13:06:02 UTC |
| 907c4ad | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:27:30 UTC | api.server: Add metrics on origin_add and origin_add_one | 03 April 2019, 13:06:02 UTC |
| b7d9896 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:22:06 UTC | api.server: Add metrics to origin_visit_add | 03 April 2019, 13:06:02 UTC |
| e357a38 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:13:40 UTC | api.server: Include the encoding step when timing | 03 April 2019, 13:06:02 UTC |
| bb19ce3 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:00:10 UTC | server.api: Fix metric names respecting prometheus conventions [1] [1] https://prometheus.io/docs/practices/naming/ | 03 April 2019, 13:06:02 UTC |
| 4518363 | Antoine R. Dumont (@ardumont) | 03 April 2019, 08:49:41 UTC | api.server.process_metrics: Skip processing when value is 0 | 03 April 2019, 13:06:02 UTC |
| cb7b9ee | Antoine R. Dumont (@ardumont) | 03 April 2019, 08:48:59 UTC | api.server: Rename to process_metrics function name | 03 April 2019, 13:06:02 UTC |
| 529c4be | Antoine R. Dumont (@ardumont) | 03 April 2019, 08:48:35 UTC | storage: Fix typo in variable name | 03 April 2019, 13:06:02 UTC |
| ce999b5 | Antoine R. Dumont (@ardumont) | 02 April 2019, 16:45:52 UTC | api.server: Exploit new endpoint to compute metrics | 03 April 2019, 13:06:01 UTC |
| f18af30 | Antoine R. Dumont (@ardumont) | 02 April 2019, 16:44:24 UTC | swh.storage: Adapt and unify _add endpoints | 03 April 2019, 13:06:01 UTC |
| ddd1835 | Antoine R. Dumont (@ardumont) | 01 April 2019, 14:27:23 UTC | api.server: Send computed metrics | 03 April 2019, 12:57:30 UTC |
| 007f3de | Antoine R. Dumont (@ardumont) | 01 April 2019, 14:04:25 UTC | storage*.snapshot_add: Return summary | 03 April 2019, 12:57:30 UTC |
| ab7f008 | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:55:07 UTC | storage*.release_add: Return summary | 03 April 2019, 12:53:50 UTC |
| 01b8a9a | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:54:42 UTC | storage.in_memory: Fix missing filtering on release_add | 03 April 2019, 12:53:50 UTC |
| f366446 | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:48:36 UTC | storage*.revision_add: Return summary | 03 April 2019, 12:53:50 UTC |
| fb4be03 | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:40:44 UTC | storage: Align implementations to compute correct metrics It was simpler to return the correct number instead of making the in-memory returns wrong results. | 03 April 2019, 12:53:50 UTC |
| 25da407 | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:11:16 UTC | directory_add: Normalize metrics keys | 03 April 2019, 12:53:50 UTC |
| cd78364 | Antoine R. Dumont (@ardumont) | 01 April 2019, 11:59:43 UTC | content_add: Normalize the summary keys | 03 April 2019, 12:53:50 UTC |
| b341446 | Antoine R. Dumont (@ardumont) | 01 April 2019, 10:27:57 UTC | api.server.directory_add: Add counter metrics | 03 April 2019, 12:53:50 UTC |
| ecbb611 | Antoine R. Dumont (@ardumont) | 01 April 2019, 10:27:42 UTC | storage*.directory_add: Return summary | 03 April 2019, 12:53:50 UTC |
| 71bd5bf | Antoine R. Dumont (@ardumont) | 01 April 2019, 10:05:07 UTC | storage.in_memory.content_add: Adapt implementation to send summary | 03 April 2019, 12:53:50 UTC |
| a284ec0 | Antoine R. Dumont (@ardumont) | 01 April 2019, 09:58:06 UTC | storage: Update tests to check the summary result act as expected | 03 April 2019, 12:53:50 UTC |
| ee62f46 | Antoine R. Dumont (@ardumont) | 01 April 2019, 09:53:50 UTC | api.server: Remove encode step from metrics | 03 April 2019, 12:53:49 UTC |
| 7f798bb | Antoine R. Dumont (@ardumont) | 01 April 2019, 09:53:30 UTC | api.server: Update docstrings on content_add to explicit the output | 03 April 2019, 12:53:49 UTC |
| 3e45179 | Antoine R. Dumont (@ardumont) | 29 March 2019, 16:13:58 UTC | Install counter metrics | 03 April 2019, 12:53:49 UTC |
| 2539858 | Antoine R. Dumont (@ardumont) | 29 March 2019, 09:39:12 UTC | api.server: Remove timed metric on the get_storage function | 03 April 2019, 12:53:49 UTC |
| 86308f6 | Valentin Lorentz | 01 April 2019, 14:11:44 UTC | Make snapshot_add take an iterable of snapshot. For uniformity with other _add() endpoints (except origin_visit_add). Noone uses the new signature of snapshot_add except the tests yet, so it's the perfect time to make this breaking change. | 03 April 2019, 12:47:02 UTC |
| 1ddb145 | Valentin Lorentz | 01 April 2019, 10:00:51 UTC | Use sha1 instead of bigint as FK from origin_visit to snapshot (part 3: cleanup) | 01 April 2019, 13:21:24 UTC |
| 7b56701 | Valentin Lorentz | 01 April 2019, 09:56:43 UTC | Use sha1 instead of bigint as FK from origin_visit to snapshot (part 2: backfill + start using the new colum for reading) | 01 April 2019, 13:21:24 UTC |
