https://github.com/SoftwareHeritage/swh-storage

sort by:
Revision Author Date Message Commit Date
3b40ce8 Updated backport on buster-swh from debian/0.11.1-1_swh1 (unstable-swh) 20 July 2020, 11:20:20 UTC
0c2e50d Merge tag 'debian/0.11.1-1_swh1' into debian/buster-swh 20 July 2020, 11:20:19 UTC
927b95d Updated debian changelog for version 0.11.1 20 July 2020, 11:14:39 UTC
4d5d3c8 Update upstream source from tag 'debian/upstream/0.11.1' Update to upstream version '0.11.1' with Debian dir 28028312f22d69d4ca9d0b7c2a432af895a0fbc9 20 July 2020, 11:14:38 UTC
f600de9 New upstream version 0.11.1 20 July 2020, 11:14:37 UTC
03c6e15 Rename 'deposit' authority type to 'deposit_client'. It makes more sense semantically; as the client is the authority not the deposit server. 20 July 2020, 11:00:22 UTC
87b1070 test_storage: directory: Use data model object Related to T2494 20 July 2020, 09:58:34 UTC
e88c85a Updated backport on buster-swh from debian/0.11.0-1_swh1 (unstable-swh) 20 July 2020, 09:25:05 UTC
ab88a89 Merge tag 'debian/0.11.0-1_swh1' into debian/buster-swh 20 July 2020, 09:25:05 UTC
4b86a12 Updated debian changelog for version 0.11.0 20 July 2020, 09:12:25 UTC
b616ec3 Update upstream source from tag 'debian/upstream/0.11.0' Update to upstream version '0.11.0' with Debian dir a87b48762949ad167037acd21d8ba78e4b06fca9 20 July 2020, 09:12:25 UTC
0a24452 New upstream version 0.11.0 20 July 2020, 09:12:23 UTC
1a2b85f test_storage: Make swh_contents fixture generate content model objects Note that this will be improved upon after the storage migration. We can take a look at the swh.model.tests.generate_testdata.gen_contents and make it generate BaseContent objects directly if it's possible. This was not done here so the impacts is limited to storage. Related to T2494 20 July 2020, 09:09:08 UTC
99a28ad tests.generate_data_test: Remove dead code The storage fixtures now uses the swh.model.tests.generate_data_test instead Related to T2494 20 July 2020, 09:01:26 UTC
03e17c3 test_storage: skipped_content/content_missing: Use data model object Related to T2494 20 July 2020, 09:01:26 UTC
7131dcb Make metadata-related endpoints consistent with other endpoints by using Iterables of swh-model objects instead of a dict. 20 July 2020, 08:48:35 UTC
997ec1d test.storage: content_add_metadata: Use data model object Related to T2494 17 July 2020, 15:51:08 UTC
8c2d635 test.storage: content_add: Use data model object Related to T2494 17 July 2020, 15:16:20 UTC
2b239f0 test_cassandra: Use data model object Related to T2494 17 July 2020, 14:37:31 UTC
eb2bf8c test_db: Drop redundant test This is already tested through the test_storage scenario 17 July 2020, 12:50:34 UTC
04d25df test_cli: Use snapshot model object within test That commit is not so interesting. But at least we validate the snapshot is correct prior to sending it. Also that removes a bit duplicated storage configuration. Related to T2494 16 July 2020, 15:52:50 UTC
2d4f727 algos.test_origin: Use data model object and drop validate proxy use Related to T2494 16 July 2020, 15:16:36 UTC
edd74d8 Updated backport on buster-swh from debian/0.10.6-1_swh1 (unstable-swh) 16 July 2020, 13:41:58 UTC
01058e2 Merge tag 'debian/0.10.6-1_swh1' into debian/buster-swh 16 July 2020, 13:41:58 UTC
d28e4de Updated debian changelog for version 0.10.6 16 July 2020, 13:36:34 UTC
f15a814 Update upstream source from tag 'debian/upstream/0.10.6' Update to upstream version '0.10.6' with Debian dir cd1a0ab09fced23e3f2fdfc2180319f53e381296 16 July 2020, 13:36:34 UTC
b0ab58f New upstream version 0.10.6 16 July 2020, 13:36:32 UTC
97a0721 algos.test_snapshot: Use model objects from sample_data_model This opens up origin_visits and add new snapshots to the fixture. So we can reuse those. Related to T2494 16 July 2020, 13:34:23 UTC
b6971b5 pytest_plugin: Ensure fixture instantiates correctly Related to T2484 Should fix [1] [1] https://jenkins.softwareheritage.org/view/Debian%20packages/job/debian/job/packages/job/DLDBASE/job/gbp-buildpackage/154/console 16 July 2020, 13:13:05 UTC
cc97ee5 Updated backport on buster-swh from debian/0.10.5-1_swh1 (unstable-swh) 16 July 2020, 12:40:17 UTC
eb2859d Merge tag 'debian/0.10.5-1_swh1' into debian/buster-swh 16 July 2020, 12:40:17 UTC
e784318 Updated debian changelog for version 0.10.5 16 July 2020, 12:34:44 UTC
dfde77a Update upstream source from tag 'debian/upstream/0.10.5' Update to upstream version '0.10.5' with Debian dir bb81c343fd86d41d4dfc1a42d9261ada325e4aa4 16 July 2020, 12:34:44 UTC
53a6699 New upstream version 0.10.5 16 July 2020, 12:34:42 UTC
3abf6b3 pytest_plugin: Do not expose the validate proxy storage Only the storage module needs it. This should fix the debian jenkins build [1] [1] https://jenkins.softwareheritage.org/view/Debian%20packages/job/debian/job/packages/job/DLDBASE/job/gbp-buildpackage/153/console 16 July 2020, 12:00:46 UTC
a688e82 test_revision_bw_compat: Use revision model object Related to T2494 16 July 2020, 10:08:33 UTC
93768be Updated backport on buster-swh from debian/0.10.4-1_swh1 (unstable-swh) 16 July 2020, 09:42:50 UTC
c07ee1c Merge tag 'debian/0.10.4-1_swh1' into debian/buster-swh 16 July 2020, 09:42:49 UTC
b5ecd61 Updated debian changelog for version 0.10.4 16 July 2020, 09:37:24 UTC
9a134a0 Update upstream source from tag 'debian/upstream/0.10.4' Update to upstream version '0.10.4' with Debian dir a72efe1f10ce0a794b9f88ea6fb9eff0c9c65437 16 July 2020, 09:37:24 UTC
7d51310 New upstream version 0.10.4 16 July 2020, 09:37:22 UTC
21efe2a test_filter: Use model objects in tests and drop validate proxy 16 July 2020, 09:28:21 UTC
2ff4c6f test_buffer: Use model objects in tests and drop validate proxy 16 July 2020, 09:28:21 UTC
df45641 test_retry: Drop validate proxy when we can When we use the sample_data_model (almost all object types except the metadata ones), we can use a storage with no validate proxy. Depends on D3510 16 July 2020, 09:28:21 UTC
14b1648 test_retry: Use sample_data_model fixture to manipulate model objects 16 July 2020, 09:28:20 UTC
df3f46d pytest-plugin: Expose a sample_data_model fixture This is almost the same fixture as sample_data except: - it's BaseModel object instance within - not complete as we cannot convert yet the metadata objects (there is a diff pending which will allow it but right now we cannot). The next commits will use this fixture to allow the switch from dict to model objects. 16 July 2020, 09:28:20 UTC
8bc7944 pytest_plugin: Avoid fixture client to declare optional dependency Prior to this commit, this would make swh_storage_backend_config fixture clients need to declare an optional dependency on swh.journal. Otherwise, it would not work [1]. This commit fixes it by dropping this configuration in the main pytest plugin. It keeps the storage tests testing with that journal_writer collaborator though by declaring an override which still provides it. This fixes the debian build [1] [1] https://jenkins.softwareheritage.org/view/Debian%20packages/job/debian/job/packages/job/DLDBASE/job/gbp-buildpackage/152/console 16 July 2020, 07:33:00 UTC
f5811da Allow cassandra binary path to be configured through env variable The current hard-coded value won't work for other distributions not relying on standard conventions (e.g. nixos...). This keeps the original behavior and only allow to diverge based on the environment variable SWH_CASSANDRA_BIN. This also: - fixes an issue on log path inexistence which raises. - renames the other env variable LOG_CASSANDRA to SWH_CASSANDRA_LOG (for consistency) 15 July 2020, 10:09:55 UTC
1a8924b 158: Make schema and migration converge so the migration works In the end, the order of the revision entry matters whether we select * or not. So the select must match the order defined in the revision_entry type type. Otherwise, a mismatch type error occurs [1] [1] psql:sql/upgrades/158.sql:74: ERROR: return type mismatch in function declared to return revision_entry 11 July 2020, 06:52:26 UTC
d191afc Updated backport on buster-swh from debian/0.10.3-1_swh1 (unstable-swh) 10 July 2020, 14:46:24 UTC
fc3270c Merge tag 'debian/0.10.3-1_swh1' into debian/buster-swh 10 July 2020, 14:46:24 UTC
bca3e4d Updated debian changelog for version 0.10.3 10 July 2020, 14:40:29 UTC
39820c5 Update upstream source from tag 'debian/upstream/0.10.3' Update to upstream version '0.10.3' with Debian dir 838097bcec631592b5a930eda6b7bcaf82653459 10 July 2020, 14:40:28 UTC
312a1d4 New upstream version 0.10.3 10 July 2020, 14:40:26 UTC
9219a23 in_memory: Fix snapshot_get_branches regression with target_types When providing target_types parameter, snapshot branches must be sorted when iterating otherwise wrong branches can be returned. 10 July 2020, 14:15:54 UTC
983cc9c Updated backport on buster-swh from debian/0.10.2-1_swh2 (unstable-swh) 10 July 2020, 07:30:38 UTC
29e7aa6 Merge tag 'debian/0.10.2-1_swh2' into debian/buster-swh 10 July 2020, 07:30:38 UTC
be19086 d/changelog: Bump new release 10 July 2020, 07:21:43 UTC
e61adec d/rules: Avoid double pytest plugin loading clash 10 July 2020, 07:17:27 UTC
319abde Updated debian changelog for version 0.10.2 10 July 2020, 06:52:42 UTC
83f3869 Update upstream source from tag 'debian/upstream/0.10.2' Update to upstream version '0.10.2' with Debian dir 6a7cb32f7c2b24f5a1a828148ccddc05ecf59f29 10 July 2020, 06:52:41 UTC
da3fd3f New upstream version 0.10.2 10 July 2020, 06:52:39 UTC
23318c2 setup: Do no expose the pytest-plugin any longer Defining the pytest-plugin though the pytest-plugin [1] makes it loaded by default. This creates loading issues on modules depending on storage but not on the pytest plugin storage exposes. It was explained in the doc and I did not realize [2] Instead we'll explicitely define to modules depending on the pytest plugins in their root conftest [3]: ``` pytest_plugins = [ "swh.storage.pytest_plugin" ] ``` [1] https://docs.pytest.org/en/stable/writing_plugins.html#setuptools-entry-points [2] https://docs.pytest.org/en/stable/writing_plugins.html#plugin-discovery-order-at-tool-startup [3] https://docs.pytest.org/en/stable/writing_plugins.html#requiring-loading-plugins-in-a-test-module-or-conftest-file Related to T2484 10 July 2020, 06:19:39 UTC
124e76d Rework dia -> pdf pipeline for inkscape 1.0 - Use dia directly to convert from .dia to .svg (inkscape would use dia via a plugin anyway) - Add proper runes to detect inkscape >= 1 and use the export options for that. 09 July 2020, 17:38:29 UTC
de38cd1 Remove overhead of to_dict/from_dict in test_snapshot_large. This should make it fast enough not to exceed the deadline. 09 July 2020, 16:02:10 UTC
e415488 in_memory: Fix quadratic run time in snapshot_get_branches. snapshot.branches is now an ImmutableDict, which is backed by a tuple of tuples; so random accesses now take a linear time instead of a constant time. This commit replaces random accesses with a single scan of all the items, and does existence checks in a set instead. 09 July 2020, 15:59:48 UTC
c3803ef Fix a typo I introduced in previous revision dict(x if x is not None else None) != dict(x) if x is not None else None... 09 July 2020, 09:56:35 UTC
8bf3794 Convert ImmutableDict to dict before passing it to json.dumps. To work with the new swh-model version, which uses ImmutableDict in model objects. 09 July 2020, 09:31:50 UTC
35143e0 Updated backport on buster-swh from debian/0.10.1-1_swh2 (unstable-swh) 08 July 2020, 13:06:52 UTC
d9be965 Merge tag 'debian/0.10.1-1_swh2' into debian/buster-swh 08 July 2020, 13:06:52 UTC
7fef9a3 d/changelog: Bump new release 08 July 2020, 12:56:55 UTC
dc13687 d/control: Add missing runtime dependency 08 July 2020, 12:53:59 UTC
378be70 Updated debian changelog for version 0.10.1 08 July 2020, 12:39:15 UTC
b21bdf3 Update upstream source from tag 'debian/upstream/0.10.1' Update to upstream version '0.10.1' with Debian dir ef92c0f03a58e915e8c533907b05f3cbbf47c0b2 08 July 2020, 12:39:14 UTC
4b735d9 New upstream version 0.10.1 08 July 2020, 12:39:13 UTC
f64a1cf Updated debian changelog for version 0.10.0 08 July 2020, 10:11:10 UTC
dd9e095 Update upstream source from tag 'debian/upstream/0.10.0' Update to upstream version '0.10.0' with Debian dir 9dcf01bf2be5bceee11f69c2c4aa312d2e03db82 08 July 2020, 10:11:09 UTC
5f8614a New upstream version 0.10.0 08 July 2020, 10:11:07 UTC
c21d0e3 Move sharable fixtures out of conftest into a dedicated pytest plugin This will allow loaders to reuse those dedicated fixtures within their code base without having to import the swh.storage.tests.conftest module. Related to T2484 08 July 2020, 09:50:21 UTC
e45ca76 Migrate from vcversioner to setuptools-scm Related to T2105 07 July 2020, 15:42:30 UTC
5ab7023 Extract revision's extra_header as a top level attribute Follows swh.model's evolution for the Revision model class. In Postgresql, store the extra_headers as a bytea[][]. Ensure data present in postgres with extra_headers in the metadata field are properly supported by the pg-backed storage. Get rid of the (now useless) git_headers_to_db() converter function. In Cassandra, store them as frozen<list<list<blob>>>. 07 July 2020, 14:49:14 UTC
e734695 Updated backport on buster-swh from debian/0.9.3-1_swh1 (unstable-swh) 06 July 2020, 08:11:52 UTC
9af4e73 Merge tag 'debian/0.9.3-1_swh1' into debian/buster-swh 06 July 2020, 08:11:51 UTC
8828f5d Updated debian changelog for version 0.9.3 06 July 2020, 08:06:13 UTC
cd6b2fc Update upstream source from tag 'debian/upstream/0.9.3' Update to upstream version '0.9.3' with Debian dir 75517c4f83b27d028160c6a892308555ff2febaf 06 July 2020, 08:06:12 UTC
8154981 New upstream version 0.9.3 06 July 2020, 08:06:11 UTC
8010848 storage: Send metrics from the origin_add endpoint Prior to this commit, since the loaders got migrated to use the main endpoint, no metrics were sent for the origin any longer. This commit fixes it. It also drops the send_metrics call from the deprecated endpoint origin_add_one (which, as an implementation details calls the other one). 06 July 2020, 07:45:40 UTC
ddec157 Updated backport on buster-swh from debian/0.9.2-1_swh1 (unstable-swh) 03 July 2020, 16:59:53 UTC
a98cc42 Merge tag 'debian/0.9.2-1_swh1' into debian/buster-swh 03 July 2020, 16:59:52 UTC
ab6706c Updated debian changelog for version 0.9.2 03 July 2020, 16:54:13 UTC
12740a4 Update upstream source from tag 'debian/upstream/0.9.2' Update to upstream version '0.9.2' with Debian dir 56ec491d91dc54522e4bc3ac723e3e3b0e6b5c47 03 July 2020, 16:54:12 UTC
6b7027b New upstream version 0.9.2 03 July 2020, 16:54:10 UTC
95fd660 pg-storage: Add missing cur parameter passing Although, this also pulled a refactoring on the insertion query as the default naive approach ended up with issues on cur already being closed [1] [1] Related to P715 Related to D3416 03 July 2020, 15:54:04 UTC
ffa9206 Updated backport on buster-swh from debian/0.9.1-1_swh1 (unstable-swh) 03 July 2020, 15:06:24 UTC
9ac73da Merge tag 'debian/0.9.1-1_swh1' into debian/buster-swh 03 July 2020, 15:06:23 UTC
b7ff639 Updated debian changelog for version 0.9.1 03 July 2020, 15:00:33 UTC
b3e9669 Update upstream source from tag 'debian/upstream/0.9.1' Update to upstream version '0.9.1' with Debian dir 02ff8c6b2b9df66b00a95c56c91b76a84d436e78 03 July 2020, 15:00:32 UTC
4a03544 New upstream version 0.9.1 03 July 2020, 15:00:30 UTC
348bc7b storage.db: Drop db.origin_visit_upsert behavior The initial desired behavior was to allow creation of origin-visit if they already had their id set. This is the what's needed for the replayer to actually work. But somehow, this left the possibility to update the origin-visit... This commit fixes it by dropping conflictual origin-visits if any. In effect, we can no longer overwrite origin-visits (pg-storage wise). Related to T2310 03 July 2020, 14:42:20 UTC
248c277 Move tests of content_metadata_* next to origin_metadata_* For consistency with the main code. 02 July 2020, 09:04:43 UTC
back to top