swh:1:snp:eb70f1f85391e4b077c211bec36af0061c4bf937

sort by:
Revision Author Date Message Commit Date
7543dc8 Updated debian changelog for version 0.11.4 27 July 2020, 14:16:18 UTC
548f018 Update upstream source from tag 'debian/upstream/0.11.4' Update to upstream version '0.11.4' with Debian dir a95acf15bcf503742e7f5ce212010932a5cc49a5 27 July 2020, 14:16:17 UTC
b64c58d New upstream version 0.11.4 27 July 2020, 14:16:13 UTC
119d01e storage*: origin_visit_get_by -> Optional[OriginVisit] Related to T645 27 July 2020, 12:44:16 UTC
2d51be9 Rename object_metadata to raw_extrinsic_metadata. For consistency with the name in swh-model. 27 July 2020, 12:18:49 UTC
57e305e storage*: origin_visit_find_by_date -> Optional[OriginVisit] Related to T645 27 July 2020, 10:46:35 UTC
b31c304 algos.origin: Simplify origin_get_latest_visit_status function 27 July 2020, 09:56:33 UTC
5344a6f storage*: type origin_visit_get_latest endpoint result The endpoint returns an optional OriginVisit object instead of a dict: ``` def origin_visit_get_latest(...) -> Optional[OriginVisit] ``` It also fixes the in-memory storage implementation which filtered data too early. It only filtered on the latest origin visit status associated to the origin visit. So depending on filters, this could have been wrong. It was not much of a problem as there is no longer any direct clients of this api (they are using [1] now). [1] swh.storage.algos.origin.origin_get_latest_origin_visit_status function Related to T645 27 July 2020, 09:56:33 UTC
6f0946d Updated debian changelog for version 0.11.3 27 July 2020, 06:08:56 UTC
2324094 Update upstream source from tag 'debian/upstream/0.11.3' Update to upstream version '0.11.3' with Debian dir b2477219c2018c32c320df9ddcf9f011f7185c12 27 July 2020, 06:08:54 UTC
65ea9e8 New upstream version 0.11.3 27 July 2020, 06:08:50 UTC
789972f metadata_{authority,fetcher}_add: Fix crash when the iterable argument is empty. 24 July 2020, 07:28:33 UTC
7e94767 storage*: origin_get(Iterable[str]) -> Iterable[Optional[Origin]] This: - drops the legacy behavior (no more input as list of dicts or even one dict). - aligns with other _get endpoints (only 1 iterable of identifiers as input, here the origin urls). - migrates towards returning an iterable of optional origin model objects (again the optional part is alignment with existing get endpoint) Related to T645 23 July 2020, 16:16:38 UTC
d8583eb storage*.origin_visit_get_random: Read model objects Related to T645 23 July 2020, 12:10:07 UTC
ccbcee9 Updated debian changelog for version 0.11.2 23 July 2020, 10:18:15 UTC
1bd6fa1 Update upstream source from tag 'debian/upstream/0.11.2' Update to upstream version '0.11.2' with Debian dir d959a8c4b061404f92da4b96189a773245484f35 23 July 2020, 10:18:14 UTC
8052056 New upstream version 0.11.2 23 July 2020, 10:18:11 UTC
b2055f4 pgstorage: Drop unnecessary indirection from reading origin_visit It's a missing left-over from the migration to making the origin-visit immutable. 23 July 2020, 08:28:09 UTC
ccbd2e9 pytest_plugin: Make sample_data an object Note that this: - drops the no longer needed copy done by the StorageData instance (used by sample_data) since now it returned immutable BaseModel objects. - centralizes some left-over tests to use sample_data as well 22 July 2020, 10:41:08 UTC
67a909e pytest_plugin: Rename sample_data_model to sample_data Related to T2494 22 July 2020, 09:37:40 UTC
e005900 pytest_plugin: Drop sample_data in favor of sample_data_model Related to T2494 22 July 2020, 09:30:33 UTC
bbe840e storage_data: Expose snapshots as model objects Related to T2494 21 July 2020, 15:43:35 UTC
d0cf317 storage_data: Expose release as model objects Related to T2494 21 July 2020, 15:32:03 UTC
3be5327 storage_data: Expose origin_visits as model objects Related to T2494 21 July 2020, 15:21:20 UTC
bcc0aee storage_data: Expose origins as model objects Related to T2494 21 July 2020, 15:03:26 UTC
d4cd33c storage_data: Expose revisions as model objects Related to T2494 21 July 2020, 14:24:16 UTC
955b6e2 storage_data: Expose directories as directory model objects Related to T2494 21 July 2020, 13:53:49 UTC
95dbdf7 storage_data: Remove unused fixture data Less to maintain Related to T2494 21 July 2020, 13:53:49 UTC
98a87fe storage_data: Expose contents as content model object Related to T2494 21 July 2020, 13:53:49 UTC
a23b748 pytest_plugin: Drop unnecessary back and forth conversion This is preparatory work to incrementally migrate the sample_data fixture to use model objects directly. Related to T2494 21 July 2020, 13:53:49 UTC
6338ad2 Drop validate proxy The validate proxy was initially an helper to ease the transition from the use of dicts towards model objects in "*_add" production endpoints. It was not removed immediately and grew some behavior it should not have (notably revision conversion so the comparison within those related tests work). After finally migrated away from dicts within the tests, we can now drop it [1]. Note that this moves the extra revision conversion behavior from the validate proxy to those related tests. This extra step will also disappear when we finally move the "*_get" endpoints to return model objects as well. Note: - This drops fixture redefinitions in the process (introduced so we could have that validate proxy at the time). - Remove the "validate" keyword from the get_storage function (so no longer possible to instantiate one [2]) [1] T2994 [2] which, practically, is the case today, nothing runs on production with it. Related to T2499 21 July 2020, 11:25:24 UTC
e0152b0 157: Fix migration script a posteriori Data has been fixed in production 21 July 2020, 10:35:40 UTC
96b2636 tests: Convert left-over dicts to model objects Related to T2494 21 July 2020, 08:41:22 UTC
42ae56d test_storage: Migrate last storage to use model objects Related to T2494 20 July 2020, 20:23:09 UTC
d4f896e test_storage: test_origin: Use data model object Related to T2494 20 July 2020, 20:21:56 UTC
6bdfd85 test_storage: origin_metadata: Centralize objects within sample_data_model Related to T2494 20 July 2020, 19:44:09 UTC
a6f70c3 test_storage: content_metadata: Centralize objects within sample_data_model Related to T2494 20 July 2020, 19:43:25 UTC
c9e921e test_storage: test_object_find_by_sha1: Use data model object Related to T2494 20 July 2020, 15:59:00 UTC
caa7f79 test_storage: content_find: Use data model object Related to T2494 20 July 2020, 15:55:35 UTC
6453504 test_storage: stat_counters: Use data model object Related to T2494 20 July 2020, 15:55:35 UTC
cf80d3c test_storage: snapshot: Use data model object Related to T2494 20 July 2020, 14:22:35 UTC
4f10171 test_storage: origin_visit/origin_visit_status: Use data model object Related to T2494 20 July 2020, 13:36:05 UTC
1c1bef9 test_storage: revision/release: Drop no longer needed conditionals Related to T2494 20 July 2020, 12:52:37 UTC
cdf6f58 test_storage: origin: Use data model object Related to T2494 20 July 2020, 12:51:22 UTC
bbdd7ed tests: Drop deprecated storage.origin_add_one use This is no longer used anywhere. Related to T2494 20 July 2020, 11:44:08 UTC
4971c25 test_storage: release: Use data model object Related to T2494 20 July 2020, 11:20:59 UTC
927b95d Updated debian changelog for version 0.11.1 20 July 2020, 11:14:39 UTC
4d5d3c8 Update upstream source from tag 'debian/upstream/0.11.1' Update to upstream version '0.11.1' with Debian dir 28028312f22d69d4ca9d0b7c2a432af895a0fbc9 20 July 2020, 11:14:38 UTC
f600de9 New upstream version 0.11.1 20 July 2020, 11:14:37 UTC
a17c412 test_storage: revision: Use data model object Related to T2494 20 July 2020, 11:03:18 UTC
03c6e15 Rename 'deposit' authority type to 'deposit_client'. It makes more sense semantically; as the client is the authority not the deposit server. 20 July 2020, 11:00:22 UTC
87b1070 test_storage: directory: Use data model object Related to T2494 20 July 2020, 09:58:34 UTC
4b86a12 Updated debian changelog for version 0.11.0 20 July 2020, 09:12:25 UTC
b616ec3 Update upstream source from tag 'debian/upstream/0.11.0' Update to upstream version '0.11.0' with Debian dir a87b48762949ad167037acd21d8ba78e4b06fca9 20 July 2020, 09:12:25 UTC
0a24452 New upstream version 0.11.0 20 July 2020, 09:12:23 UTC
1a2b85f test_storage: Make swh_contents fixture generate content model objects Note that this will be improved upon after the storage migration. We can take a look at the swh.model.tests.generate_testdata.gen_contents and make it generate BaseContent objects directly if it's possible. This was not done here so the impacts is limited to storage. Related to T2494 20 July 2020, 09:09:08 UTC
99a28ad tests.generate_data_test: Remove dead code The storage fixtures now uses the swh.model.tests.generate_data_test instead Related to T2494 20 July 2020, 09:01:26 UTC
03e17c3 test_storage: skipped_content/content_missing: Use data model object Related to T2494 20 July 2020, 09:01:26 UTC
7131dcb Make metadata-related endpoints consistent with other endpoints by using Iterables of swh-model objects instead of a dict. 20 July 2020, 08:48:35 UTC
997ec1d test.storage: content_add_metadata: Use data model object Related to T2494 17 July 2020, 15:51:08 UTC
8c2d635 test.storage: content_add: Use data model object Related to T2494 17 July 2020, 15:16:20 UTC
2b239f0 test_cassandra: Use data model object Related to T2494 17 July 2020, 14:37:31 UTC
eb2bf8c test_db: Drop redundant test This is already tested through the test_storage scenario 17 July 2020, 12:50:34 UTC
04d25df test_cli: Use snapshot model object within test That commit is not so interesting. But at least we validate the snapshot is correct prior to sending it. Also that removes a bit duplicated storage configuration. Related to T2494 16 July 2020, 15:52:50 UTC
2d4f727 algos.test_origin: Use data model object and drop validate proxy use Related to T2494 16 July 2020, 15:16:36 UTC
d28e4de Updated debian changelog for version 0.10.6 16 July 2020, 13:36:34 UTC
f15a814 Update upstream source from tag 'debian/upstream/0.10.6' Update to upstream version '0.10.6' with Debian dir cd1a0ab09fced23e3f2fdfc2180319f53e381296 16 July 2020, 13:36:34 UTC
b0ab58f New upstream version 0.10.6 16 July 2020, 13:36:32 UTC
97a0721 algos.test_snapshot: Use model objects from sample_data_model This opens up origin_visits and add new snapshots to the fixture. So we can reuse those. Related to T2494 16 July 2020, 13:34:23 UTC
b6971b5 pytest_plugin: Ensure fixture instantiates correctly Related to T2484 Should fix [1] [1] https://jenkins.softwareheritage.org/view/Debian%20packages/job/debian/job/packages/job/DLDBASE/job/gbp-buildpackage/154/console 16 July 2020, 13:13:05 UTC
e784318 Updated debian changelog for version 0.10.5 16 July 2020, 12:34:44 UTC
dfde77a Update upstream source from tag 'debian/upstream/0.10.5' Update to upstream version '0.10.5' with Debian dir bb81c343fd86d41d4dfc1a42d9261ada325e4aa4 16 July 2020, 12:34:44 UTC
53a6699 New upstream version 0.10.5 16 July 2020, 12:34:42 UTC
3abf6b3 pytest_plugin: Do not expose the validate proxy storage Only the storage module needs it. This should fix the debian jenkins build [1] [1] https://jenkins.softwareheritage.org/view/Debian%20packages/job/debian/job/packages/job/DLDBASE/job/gbp-buildpackage/153/console 16 July 2020, 12:00:46 UTC
a688e82 test_revision_bw_compat: Use revision model object Related to T2494 16 July 2020, 10:08:33 UTC
b5ecd61 Updated debian changelog for version 0.10.4 16 July 2020, 09:37:24 UTC
9a134a0 Update upstream source from tag 'debian/upstream/0.10.4' Update to upstream version '0.10.4' with Debian dir a72efe1f10ce0a794b9f88ea6fb9eff0c9c65437 16 July 2020, 09:37:24 UTC
7d51310 New upstream version 0.10.4 16 July 2020, 09:37:22 UTC
21efe2a test_filter: Use model objects in tests and drop validate proxy 16 July 2020, 09:28:21 UTC
2ff4c6f test_buffer: Use model objects in tests and drop validate proxy 16 July 2020, 09:28:21 UTC
df45641 test_retry: Drop validate proxy when we can When we use the sample_data_model (almost all object types except the metadata ones), we can use a storage with no validate proxy. Depends on D3510 16 July 2020, 09:28:21 UTC
14b1648 test_retry: Use sample_data_model fixture to manipulate model objects 16 July 2020, 09:28:20 UTC
df3f46d pytest-plugin: Expose a sample_data_model fixture This is almost the same fixture as sample_data except: - it's BaseModel object instance within - not complete as we cannot convert yet the metadata objects (there is a diff pending which will allow it but right now we cannot). The next commits will use this fixture to allow the switch from dict to model objects. 16 July 2020, 09:28:20 UTC
8bc7944 pytest_plugin: Avoid fixture client to declare optional dependency Prior to this commit, this would make swh_storage_backend_config fixture clients need to declare an optional dependency on swh.journal. Otherwise, it would not work [1]. This commit fixes it by dropping this configuration in the main pytest plugin. It keeps the storage tests testing with that journal_writer collaborator though by declaring an override which still provides it. This fixes the debian build [1] [1] https://jenkins.softwareheritage.org/view/Debian%20packages/job/debian/job/packages/job/DLDBASE/job/gbp-buildpackage/152/console 16 July 2020, 07:33:00 UTC
f5811da Allow cassandra binary path to be configured through env variable The current hard-coded value won't work for other distributions not relying on standard conventions (e.g. nixos...). This keeps the original behavior and only allow to diverge based on the environment variable SWH_CASSANDRA_BIN. This also: - fixes an issue on log path inexistence which raises. - renames the other env variable LOG_CASSANDRA to SWH_CASSANDRA_LOG (for consistency) 15 July 2020, 10:09:55 UTC
1a8924b 158: Make schema and migration converge so the migration works In the end, the order of the revision entry matters whether we select * or not. So the select must match the order defined in the revision_entry type type. Otherwise, a mismatch type error occurs [1] [1] psql:sql/upgrades/158.sql:74: ERROR: return type mismatch in function declared to return revision_entry 11 July 2020, 06:52:26 UTC
bca3e4d Updated debian changelog for version 0.10.3 10 July 2020, 14:40:29 UTC
39820c5 Update upstream source from tag 'debian/upstream/0.10.3' Update to upstream version '0.10.3' with Debian dir 838097bcec631592b5a930eda6b7bcaf82653459 10 July 2020, 14:40:28 UTC
312a1d4 New upstream version 0.10.3 10 July 2020, 14:40:26 UTC
9219a23 in_memory: Fix snapshot_get_branches regression with target_types When providing target_types parameter, snapshot branches must be sorted when iterating otherwise wrong branches can be returned. 10 July 2020, 14:15:54 UTC
be19086 d/changelog: Bump new release 10 July 2020, 07:21:43 UTC
e61adec d/rules: Avoid double pytest plugin loading clash 10 July 2020, 07:17:27 UTC
319abde Updated debian changelog for version 0.10.2 10 July 2020, 06:52:42 UTC
83f3869 Update upstream source from tag 'debian/upstream/0.10.2' Update to upstream version '0.10.2' with Debian dir 6a7cb32f7c2b24f5a1a828148ccddc05ecf59f29 10 July 2020, 06:52:41 UTC
da3fd3f New upstream version 0.10.2 10 July 2020, 06:52:39 UTC
23318c2 setup: Do no expose the pytest-plugin any longer Defining the pytest-plugin though the pytest-plugin [1] makes it loaded by default. This creates loading issues on modules depending on storage but not on the pytest plugin storage exposes. It was explained in the doc and I did not realize [2] Instead we'll explicitely define to modules depending on the pytest plugins in their root conftest [3]: ``` pytest_plugins = [ "swh.storage.pytest_plugin" ] ``` [1] https://docs.pytest.org/en/stable/writing_plugins.html#setuptools-entry-points [2] https://docs.pytest.org/en/stable/writing_plugins.html#plugin-discovery-order-at-tool-startup [3] https://docs.pytest.org/en/stable/writing_plugins.html#requiring-loading-plugins-in-a-test-module-or-conftest-file Related to T2484 10 July 2020, 06:19:39 UTC
124e76d Rework dia -> pdf pipeline for inkscape 1.0 - Use dia directly to convert from .dia to .svg (inkscape would use dia via a plugin anyway) - Add proper runes to detect inkscape >= 1 and use the export options for that. 09 July 2020, 17:38:29 UTC
de38cd1 Remove overhead of to_dict/from_dict in test_snapshot_large. This should make it fast enough not to exceed the deadline. 09 July 2020, 16:02:10 UTC
e415488 in_memory: Fix quadratic run time in snapshot_get_branches. snapshot.branches is now an ImmutableDict, which is backed by a tuple of tuples; so random accesses now take a linear time instead of a constant time. This commit replaces random accesses with a single scan of all the items, and does existence checks in a set instead. 09 July 2020, 15:59:48 UTC
c3803ef Fix a typo I introduced in previous revision dict(x if x is not None else None) != dict(x) if x is not None else None... 09 July 2020, 09:56:35 UTC
back to top