https://github.com/SoftwareHeritage/swh-storage

sort by:
Revision Author Date Message Commit Date
68331b5 Updated backport on buster-swh from debian/0.21.1-1_swh1 (unstable-swh) 28 January 2021, 13:24:27 UTC
3f1dc5c Merge tag 'debian/0.21.1-1_swh1' into debian/buster-swh 28 January 2021, 13:24:26 UTC
3d71ec6 Updated debian changelog for version 0.21.1 28 January 2021, 13:19:24 UTC
9580e45 Update upstream source from tag 'debian/upstream/0.21.1' Update to upstream version '0.21.1' with Debian dir 086f067ed910d482b6d861ee9c917a169e5d82de 28 January 2021, 13:19:23 UTC
57d3066 New upstream version 0.21.1 28 January 2021, 13:19:21 UTC
76de53c Correctly return origin_visit_status.type value everywhere If the type is not present on an origin_visit_status, it should be computed from the origin_visit. There were some methods which only return the origin_visit_status value. It breaks the webapp mangling the type to empty value on the search result page. Related to T3001 28 January 2021, 11:15:11 UTC
55d7898 Updated backport on buster-swh from debian/0.21.0-1_swh1 (unstable-swh) 20 January 2021, 14:56:52 UTC
7f014af Merge tag 'debian/0.21.0-1_swh1' into debian/buster-swh 20 January 2021, 14:56:51 UTC
5bd8ed5 Updated debian changelog for version 0.21.0 20 January 2021, 14:52:21 UTC
ef7b545 Update upstream source from tag 'debian/upstream/0.21.0' Update to upstream version '0.21.0' with Debian dir 12ba6aba188abf4a213a5074e30832a0a4676790 20 January 2021, 14:52:20 UTC
47e0a4c New upstream version 0.21.0 20 January 2021, 14:52:18 UTC
e433255 db: Allow new status values not_found, failed to OriginVisitStatus Related to T2961 20 January 2021, 14:36:12 UTC
47f78d7 Updated backport on buster-swh from debian/0.20.0-1_swh1 (unstable-swh) 20 January 2021, 09:34:29 UTC
fc66b25 Merge tag 'debian/0.20.0-1_swh1' into debian/buster-swh 20 January 2021, 09:34:28 UTC
7c47e51 Updated debian changelog for version 0.20.0 20 January 2021, 09:29:54 UTC
37a1ac6 Update upstream source from tag 'debian/upstream/0.20.0' Update to upstream version '0.20.0' with Debian dir 8a95cef4e85584d63f5645c29998daf165907e77 20 January 2021, 09:29:54 UTC
45803cf New upstream version 0.20.0 20 January 2021, 09:29:52 UTC
d04165f Add type to the origin_visit_status topic useful when the type is not yet populated in the database Related to T2966 18 January 2021, 10:49:34 UTC
c24d35f Add persistence of the field OriginVisitStatus.type (!) A new database upgrade is needed (165.sql) for postgresql backend Related to T2964 15 January 2021, 11:38:38 UTC
da55308 Make test_content_add_race fail for the right reason. Since 209de5dbaa127dacd114fbbd084f22632982eb77, it was failing because of: TypeError("content_add() got an unexpected keyword argument 'db'") 15 January 2021, 10:30:36 UTC
530d11f Updated backport on buster-swh from debian/0.19.0-1_swh1 (unstable-swh) 14 January 2021, 10:23:13 UTC
b6ee9fe Merge tag 'debian/0.19.0-1_swh1' into debian/buster-swh 14 January 2021, 10:23:13 UTC
753936e Updated debian changelog for version 0.19.0 14 January 2021, 10:18:32 UTC
cee711b Update upstream source from tag 'debian/upstream/0.19.0' Update to upstream version '0.19.0' with Debian dir f25b1bc13ecf51db73cfd3eb91a6550d40da724b 14 January 2021, 10:18:31 UTC
2204346 New upstream version 0.19.0 14 January 2021, 10:18:30 UTC
0b44b37 Adapt cassandra storage to ignore the new OriginVisitStatus.type field Depends on D4848 Related to T2443 13 January 2021, 10:06:12 UTC
728c3ee Allow to use the JAVA_HOME environment for cassandra tests This allows to enforce a specific version of java to be used. For example, since cassandra seems not to support java 14 yet, this allows to run tests on bullseye: JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64/ pytest swh 13 January 2021, 09:56:07 UTC
30945a5 Enforce hypothesis <6 to prevent test breakage hypothesis 6 upgraded a warning into an error: now raises a FailedHealthCheck when using a pytest fixture with a @given generative test set. See https://hypothesis.readthedocs.io/en/latest/healthchecks.html 13 January 2021, 09:42:42 UTC
74e6f58 Make the CREATE_TABLES_QUERIES in cassandra/schema.py an explicit list prevent being fooled by a missing '\n'. 08 January 2021, 13:20:08 UTC
2b35198 Add a cli section in the doc 18 December 2020, 12:41:23 UTC
04ae89f storage.backfill: Allow cli run for origin_visit_status as well 24 November 2020, 17:21:21 UTC
64ee845 conftest: Reference swh.core.db.pytest_plugin As it's exposed through the swh.storage.pytest_plugin itself used by other swh modules, this needs to be declared to avoid other swh module build failures. Related to T2746 24 November 2020, 13:08:12 UTC
ec2f392 Updated backport on buster-swh from debian/0.18.0-1_swh1 (unstable-swh) 23 November 2020, 14:00:20 UTC
1fc26df Merge tag 'debian/0.18.0-1_swh1' into debian/buster-swh 23 November 2020, 14:00:19 UTC
cbbaeea Updated debian changelog for version 0.18.0 23 November 2020, 13:52:33 UTC
e5bce82 Update upstream source from tag 'debian/upstream/0.18.0' Update to upstream version '0.18.0' with Debian dir 6ef3ab996694ae4582886e07c25624fa073dc956 23 November 2020, 13:52:32 UTC
4c46835 New upstream version 0.18.0 23 November 2020, 13:52:31 UTC
e289593 requirements-test.txt: Drop no longer needed pytest-postgresql requirement requirements-swh.txt already declares the swh.core[db] dependency which transitively pulls it. Related to T2746 23 November 2020, 12:07:45 UTC
b1375bc d/control: Rework test dependency Related to T2746 23 November 2020, 10:54:40 UTC
0065d4d backfill: Reverse flawed logic in SnapshotBranch generation The previous code would nullify all non-null branches, and try to create a SnapshotBranch out of null branches. 13 November 2020, 15:51:29 UTC
f501136 migrate_extrinsic_metadata: don't crash when deb revisions aren't referenced by any snapshot As this happens for about 50 revisions in the archive. 13 November 2020, 14:26:15 UTC
dc65326 Updated backport on buster-swh from debian/0.17.2-1_swh1 (unstable-swh) 13 November 2020, 11:13:32 UTC
7a4ba12 Merge tag 'debian/0.17.2-1_swh1' into debian/buster-swh 13 November 2020, 11:13:32 UTC
4bec746 Updated debian changelog for version 0.17.2 13 November 2020, 11:05:35 UTC
bf7d9c7 Update upstream source from tag 'debian/upstream/0.17.2' Update to upstream version '0.17.2' with Debian dir cb682d738260a73e1a5c714e32fe464181268a21 13 November 2020, 11:05:34 UTC
6089094 New upstream version 0.17.2 13 November 2020, 11:05:33 UTC
20d3f8e backfill: only flush the journal writer on every batch This module's use of write_addition predated the introduction of reliable writing in swh.journal; Since this introduction, the backfiller has been flushing the kafka writer after writing each single object, leading to a 3x measured slowdown on backfilling contents. 13 November 2020, 10:17:31 UTC
248a04b Don't use string expansions in debug logging 12 November 2020, 17:07:14 UTC
3eba73d migrate_extrinsic_metadata: Remove log output when a CRAN origin is missing as this happens quite often and isn't an error. 09 November 2020, 15:32:20 UTC
f3652a9 migrate_extrinsic_metadata: add support for guessing the origin of more PyPI packages from filenames. 09 November 2020, 15:32:20 UTC
c0a3d96 migrate_extrinsic_metadata: use the retry proxy Because it makes a lot of get requests and doesn't handle failures, it crashed often. 09 November 2020, 15:32:20 UTC
aded45b Make the retry proxy work on all functions. The metadata migration script kept crashing otherwise. 09 November 2020, 15:32:20 UTC
2e7d489 Set the value_sanitizer argument of get_journal_writer. The next version of swh-journal will remove the default value. 09 November 2020, 15:32:20 UTC
24cdc85 cassandra: Fix content_missing_per_sha1_git implementation 09 November 2020, 13:13:10 UTC
2f5b41f Updated backport on buster-swh from debian/0.17.1-1_swh1 (unstable-swh) 06 November 2020, 14:50:58 UTC
9cc48a1 Merge tag 'debian/0.17.1-1_swh1' into debian/buster-swh 06 November 2020, 14:50:57 UTC
886999c Updated debian changelog for version 0.17.1 05 November 2020, 12:56:53 UTC
a0b1792 Update upstream source from tag 'debian/upstream/0.17.1' Update to upstream version '0.17.1' with Debian dir 1a06182f324d3f658eec3faac9d724b9c9776f3a 05 November 2020, 12:56:53 UTC
d8a6720 New upstream version 0.17.1 05 November 2020, 12:56:51 UTC
84984a6 algos.snapshot.snapshot_resolve_alias: Don't return the branch list. It complicates the signature and the code, and we don't have any use for it currently. 05 November 2020, 11:08:13 UTC
fa86834 Add test for snapshot_resolve_alias with a missing branch. 05 November 2020, 10:57:08 UTC
1826b2b Simplify algos.snapshot.snapshot_resolve_alias. 1. rename branch_info to last_branch 2. exclude the last_branch from 'branches', so that: a) it never needs to contain a None value, so we don't need a cast b) no need for slicing 05 November 2020, 10:57:08 UTC
f5092b9 Updated backport on buster-swh from debian/0.17.0-1_swh1 (unstable-swh) 03 November 2020, 18:15:38 UTC
ab33a6b Merge tag 'debian/0.17.0-1_swh1' into debian/buster-swh 03 November 2020, 18:15:38 UTC
328907a Updated debian changelog for version 0.17.0 03 November 2020, 17:20:46 UTC
9128c5b Update upstream source from tag 'debian/upstream/0.17.0' Update to upstream version '0.17.0' with Debian dir 170e6e6bb959e47c04b6d85458d4b4076ed71012 03 November 2020, 17:20:45 UTC
57956ce New upstream version 0.17.0 03 November 2020, 17:20:44 UTC
943e440 Rename the `id` argument of raw_extrinsic_metadata_get to `target` Consistently with the new name for this attribute in swh.model 0.7.2. 03 November 2020, 14:59:52 UTC
48b6dbe cassandra/in_memory: rename raw_extrinsic_metadata.id to target For consistency with swh.model v0.7.2, to prepare for the addition of an (intrinsic) id field to RawExtrinsicMetadata objects. 03 November 2020, 13:56:22 UTC
4fbf481 PostgreSQL: rename raw_extrinsic_metadata.id to target For consistency with swh.model v0.7.2, to prepare for the addition of an (intrinsic) id field to RawExtrinsicMetadata objects. 03 November 2020, 13:56:16 UTC
8b18155 algos/snapshot: Add function to resolve branch alias to real target Related to T2734 03 November 2020, 11:49:07 UTC
6e3e350 migrate_extrinsic_metadata: Write metadata on directories instead of revisions. To match the new behavior of package loaders. 29 October 2020, 10:14:48 UTC
97d0b05 pre-commit: Fix codespell regexp related error 27 October 2020, 15:43:36 UTC
9645aef Replace RawExtrinsicMetadata `id` attribute with `target`. The old attribute was deprecated in swh.model 0.7.2 27 October 2020, 14:37:13 UTC
5819683 Update swh.storage.validate for swh.model 0.7.2 swh.model.model.ModelObject.compute_hash was changed to a method instead of a staticmethod. 27 October 2020, 13:11:34 UTC
4f35f7f Add black change on swh.storage.backfill 27 October 2020, 12:47:33 UTC
474ee72 --amend 22 October 2020, 20:35:32 UTC
eb3952f migrate_extrinsic_metadata: Make pypi_origin_from_filename fix project names when possible using PyPI's API. 22 October 2020, 14:23:30 UTC
b1a3b80 migrate_extrinsic_metadata: move pypi_origin_from_filename to its own function. Instead of bloating handle_row, which is already way too long. 22 October 2020, 14:22:46 UTC
aeb72c7 migrate_extrinsic_metadata: add support for guix revisions 22 October 2020, 10:32:48 UTC
73dc5e3 migrate_extrinsic_metadata: allow deposits with 'id' missing from their metadata. 22 October 2020, 10:28:18 UTC
c483066 migrate_extrinsic_metadata: add support for guessing the origin of more PyPI packages from filenames. It now supports all pypi revisions with an id starting with an hex digit from 0 to 5. 22 October 2020, 10:27:16 UTC
2bfd9fe storage.pytest_plugin: Reuse swh.core.db.db_utils postgresql_fact 22 October 2020, 09:58:03 UTC
d93429f api.server: Add missing coverage on make_app_from_configfile factory This is actually what starts the server, so it sounds more reasonable to test that part. 19 October 2020, 13:08:01 UTC
ca8e6aa api.server: Drop the % in the error message 19 October 2020, 13:08:01 UTC
8de381b Updated backport on buster-swh from debian/0.16.0-1_swh1 (unstable-swh) 17 October 2020, 10:00:13 UTC
95b7c89 Merge tag 'debian/0.16.0-1_swh1' into debian/buster-swh 17 October 2020, 10:00:13 UTC
49d787c storage.api.server: Add type to load_and_check_config then refactor tests This also drops the type parameter from load_and_check_config which is never used. 16 October 2020, 13:57:11 UTC
1a9687f backfill: use get_journal_writer instead if instantiating JournalWriter directly. A future version of swh-journal will add a mandatory argument to JournalWriter, whic get_journal_writer sets by default. 12 October 2020, 17:21:09 UTC
b425b5c migrate_extrinsic_metadata: add support for the new deposit metadata formats introduced in late september. * https://forge.softwareheritage.org/D4065 * https://forge.softwareheritage.org/D4105 12 October 2020, 13:07:27 UTC
cd07c36 Updated debian changelog for version 0.16.0 09 October 2020, 16:33:11 UTC
d97d189 Update upstream source from tag 'debian/upstream/0.16.0' Update to upstream version '0.16.0' with Debian dir fe2380c191d3dfe05572b51a4bc92614491ed390 09 October 2020, 16:33:10 UTC
aade84f New upstream version 0.16.0 09 October 2020, 16:33:09 UTC
a11d58a Remove a bunch of deprecated instances of `args` in configurations Notably, `get_objstorage`'s `args` has been deprecated as of swh.objstorage 0.2.2. 09 October 2020, 15:29:10 UTC
a085b7e backfill: use the common `storage` top-level config key This makes the backfiller configuration compatible with all other modules. 08 October 2020, 18:35:49 UTC
dceeb74 backfill: support arbitrary journal writer configuration This allows more settings than the previous hardcoded three, e.g. the `privileged` flag to backfill a journal containing anonymous topics. 08 October 2020, 18:35:49 UTC
a6af589 retry: don't retry on keyboardinterrupt. Otherwise, Ctrl-C is ignored if pressed while sending a request. 02 October 2020, 09:07:09 UTC
889bd87 migrate_extrinsic_metadata: add support for guessing the origin of more PyPI packages from filenames. It now supports all pypi revisions with an id starting with 0, 1, or 2. 02 October 2020, 09:07:09 UTC
9ddbb69 migrate_extrinsic_metadata: allow dash in deposit client and collection names. 02 October 2020, 09:07:09 UTC
59e7e68 migrate_extrinsic_metadata: update name of column deposit.swhid_context. It was renamed in 4d72d1be529a568784842f5c0864e862a4b4705c. 02 October 2020, 09:07:09 UTC
back to top