swh:1:snp:eb70f1f85391e4b077c211bec36af0061c4bf937

sort by:
Revision Author Date Message Commit Date
043c181 Updated backport on buster-swh from debian/0.15.3-1_swh1 (unstable-swh) 24 September 2020, 18:28:40 UTC
25a1c97 Merge tag 'debian/0.15.3-1_swh1' into debian/buster-swh 24 September 2020, 18:28:39 UTC
19167e1 Updated debian changelog for version 0.15.3 24 September 2020, 18:24:14 UTC
55d1e74 Update upstream source from tag 'debian/upstream/0.15.3' Update to upstream version '0.15.3' with Debian dir 44405d462345831220c37a2a71fff14720013476 24 September 2020, 18:24:13 UTC
c0440b0 New upstream version 0.15.3 24 September 2020, 18:24:12 UTC
ac3c537 Update sql paths for the moved SQL files This should fix the currently failing documentation build. 24 September 2020, 18:08:56 UTC
2819694 Updated backport on buster-swh from debian/0.15.2-1_swh1 (unstable-swh) 24 September 2020, 17:50:25 UTC
3bc7d87 Merge tag 'debian/0.15.2-1_swh1' into debian/buster-swh 24 September 2020, 17:50:24 UTC
68b6733 Updated debian changelog for version 0.15.2 24 September 2020, 17:28:22 UTC
370ad96 Update upstream source from tag 'debian/upstream/0.15.2' Update to upstream version '0.15.2' with Debian dir 7efa6a39fe821feba8691b02fed4b6355ed4d6ea 24 September 2020, 17:28:22 UTC
20808a4 New upstream version 0.15.2 24 September 2020, 17:28:20 UTC
f677b84 Updated debian changelog for version 0.15.1 24 September 2020, 16:44:22 UTC
cdf972d Update upstream source from tag 'debian/upstream/0.15.1' Update to upstream version '0.15.1' with Debian dir 7a4e6e501403f48373a18b995786d9b017dcd3c6 24 September 2020, 16:44:21 UTC
ee6041c New upstream version 0.15.1 24 September 2020, 16:44:20 UTC
96be9bd Fix default value handling in constructor Use a more simple default value and do not identity check against it. 24 September 2020, 16:33:09 UTC
8368e34 Updated backport on buster-swh from debian/0.15.0-1_swh1 (unstable-swh) 24 September 2020, 15:08:30 UTC
1df6824 Merge tag 'debian/0.15.0-1_swh1' into debian/buster-swh 24 September 2020, 15:08:30 UTC
02d9d0d Updated debian changelog for version 0.15.0 24 September 2020, 15:03:58 UTC
7aea535 Update upstream source from tag 'debian/upstream/0.15.0' Update to upstream version '0.15.0' with Debian dir d2dad0b2426ce586558c43b33f25576ab6825d82 24 September 2020, 15:03:57 UTC
e6808fa New upstream version 0.15.0 24 September 2020, 15:03:56 UTC
829118a Add the SQL commands used to set up the logical replication publication 24 September 2020, 11:57:18 UTC
5d3de06 Support different database flavors in the SQL scripts This uses a new database table and some psql conditionals to introduce three different flavors for the swh.storage Postgres database: - the 'default' flavor has all the deduplication features, foreign keys and read indexes - the 'mirror' flavor has all the deduplication features and read indexes; it drops some foreign keys to allow for out of order addition of some object types - the 'read_replica' flavor has the minimal set of indexes to support read queries, and replication using the PostgreSQL logical replication feature Related to T2604. 24 September 2020, 11:57:14 UTC
63426e6 pytest_plugin: Use psql to load SQL files instead of connecting with psycopg2 This avoids running into issues when the SQL files contain psql-specific features like backslash-escapes. 24 September 2020, 11:54:38 UTC
38b1dbf Output a warning when the version of the database is different than expected 24 September 2020, 11:54:38 UTC
e37f639 Improve code quality and doc in BufferedProxyStorage - better names related to the object buffers - extracted parameter dicts from the constructor - used more generic typing in function parameters and more specific in other contexts in order to apply the principle of robustness 23 September 2020, 22:21:54 UTC
c97b23b Adapt cli declaration entrypoint to swh.core 0.3 23 September 2020, 14:13:01 UTC
924621f pytest_plugin: Order the fixture definitions in dependency order 23 September 2020, 10:30:55 UTC
6286e18 pytest_plugin: Change dbname to storage to avoid clash in tests Other similar fixtures in other modules which use the same "tests" db already. Clash can then happen when table names exists in different modules (e.g. dbversion exist both in scheduler and storage dbs). 23 September 2020, 10:28:52 UTC
8c44a29 pytest_plugin: Reuse swh_storage_postgresql connection string The `swh_storage_postgresql.dsn` string already contains the connection information necessary for the tests to run. 23 September 2020, 10:27:02 UTC
30cdb78 Drop the -swh- part of sql files it does not bring any meaningful info and makes it somewhat inconsistent with the new -superuser- "tag". 22 September 2020, 08:18:07 UTC
915575d Rename 10-swh-init.sql as 10-superuser-init.sql so the db initialization from swh.core (>= 0.3) executes this during the database creation step (i.e. while having a superuser level connection to the database). 22 September 2020, 08:14:15 UTC
67ee86b Warn about skipped_content sneaking the 'content' topics 21 September 2020, 08:43:00 UTC
8de6564 Small fix in the graph replayer to prevent a wrong warning 18 September 2020, 14:30:37 UTC
c43ae73 Updated backport on buster-swh from debian/0.14.3-1_swh1 (unstable-swh) 17 September 2020, 17:04:09 UTC
da299c1 Merge tag 'debian/0.14.3-1_swh1' into debian/buster-swh 17 September 2020, 17:04:09 UTC
61e880d Updated debian changelog for version 0.14.3 17 September 2020, 16:53:57 UTC
7045a62 Update upstream source from tag 'debian/upstream/0.14.3' Update to upstream version '0.14.3' with Debian dir 66526310514244ae6c614cfdc030860320fde0eb 17 September 2020, 16:53:55 UTC
8387ef5 New upstream version 0.14.3 17 September 2020, 16:53:52 UTC
b0027ab python: Reorder imports with isort Related to T2610 17 September 2020, 16:06:07 UTC
d27a046 pre-commit: Add isort hook and configuration Related to T2610 17 September 2020, 16:06:06 UTC
469c38c Make origin_add() handle multiple occurences of an origin properly this is needed to prevent some traceback in case an origin is present several times in the same origin_add batch, which situation has been seen in some mirror tests. 17 September 2020, 14:47:27 UTC
37ce2c4 pre-commit: Update flake8 hook configuration flake8 hook has been removed from https://github.com/pre-commit/pre-commit-hooks so now use the one from https://gitlab.com/pycqa/flake8 17 September 2020, 11:57:15 UTC
f008a59 migrate_extrinsic_metadata: improve pypi_project_from_filename to support suffixes after the version number. 16 September 2020, 15:00:42 UTC
8e32ad0 migrate_extrinsic_metadata: guess PyPI origins. This works by guessing the package name from the original_artifact data, then building an origin that would match the package name, then filtering checking if the revision can be reached from it. 16 September 2020, 15:00:37 UTC
0bbcd91 migrate_extrinsic_metadata.test_pypi: use the in-memory storage instead of mocks in a future commit, migrating pypi revisions will become more interactive with the storage, so it's easier to have a real one instead of a mock. 16 September 2020, 15:00:33 UTC
1676478 migrate_extrinsic_metadata.test_debian: use the in-memory storage instead of mocks in tests that need to read in the storage. Using mocks just makes it more complicated, and we decided not to do that a while ago. 16 September 2020, 15:00:29 UTC
723e728 migrate_extrinsic_metadata: fix crash on dangling branch. 16 September 2020, 15:00:23 UTC
89d23cb migrate_extrinsic_metadata: fix crash when a Debian revision is missing. https://forge.softwareheritage.org/T997 16 September 2020, 15:00:19 UTC
2ad5600 migrate_extrinsic_metadata: guess Debian origins. This works by guessing the package name from the original_artifact data, then building origins that would match the package name, then filtering out origins by checking if the revision can be reached from them. 16 September 2020, 15:00:16 UTC
744dfca Updated backport on buster-swh from debian/0.14.2-1_swh1 (unstable-swh) 15 September 2020, 13:22:08 UTC
0dc98ea Merge tag 'debian/0.14.2-1_swh1' into debian/buster-swh 15 September 2020, 13:22:07 UTC
3b781a8 sql: Make the extra_headers not null a constraint Due to the data volume, the basic not null instruction constraint (which is blocking) impose us downtime for loaders otherwise. Related to T2547 14 September 2020, 11:07:31 UTC
3577381 Updated debian changelog for version 0.14.2 11 September 2020, 13:37:11 UTC
e6886ef Update upstream source from tag 'debian/upstream/0.14.2' Update to upstream version '0.14.2' with Debian dir 7b9883751d27ae0e4410fe5f1da1f92fc639aa05 11 September 2020, 13:37:09 UTC
eb961e9 New upstream version 0.14.2 11 September 2020, 13:37:06 UTC
ed55e9c Load the "fast" hypothesis profile by default otherwise, running pytest directly (without specifying a --hypothesis-profile option) will use default hypothesis values that are unsuitable for us. 11 September 2020, 10:27:17 UTC
fd6d72f cli: speedup the `swh` cli command startup time by moving import statements in functions, as well as preventing the loading of swh.storage.interface in swh.storage unless needed for type checking. Related to T2575. 11 September 2020, 09:15:58 UTC
d24a1e7 migrate_extrinsic_metadata: retry in case of database errors. They are likely to happen since this script takes a long time to run. 10 September 2020, 07:39:55 UTC
5ec70a6 Add a Python script to migrate extrinsic metadata from revision metadata. 10 September 2020, 07:39:55 UTC
93458a4 Import get_objstorage from swh.objstorage.factory instead of swh.objstorage (deprecated). 09 September 2020, 17:26:00 UTC
90eda98 directory_ls: Don't return None for status/length/sha1/... if the content is known but skipped. 08 September 2020, 11:52:33 UTC
3198e11 Add a test for directory_ls when the contents are missing. 08 September 2020, 11:52:33 UTC
4e4efec Updated backport on buster-swh from debian/0.14.1-1_swh1 (unstable-swh) 04 September 2020, 14:04:07 UTC
8c3cfa7 Merge tag 'debian/0.14.1-1_swh1' into debian/buster-swh 04 September 2020, 14:04:06 UTC
f7d7830 Updated debian changelog for version 0.14.1 04 September 2020, 13:52:17 UTC
e6b8fe2 Update upstream source from tag 'debian/upstream/0.14.1' Update to upstream version '0.14.1' with Debian dir dc20fd4421375511834b932efce5c132307e25de 04 September 2020, 13:52:16 UTC
85189d3 New upstream version 0.14.1 04 September 2020, 13:52:12 UTC
374e01c algos.diff: Add missed revision_get conversion This fixes by doing the minimum required so the diff module still works and fix [1]. Will release release 0.14.1 asap. [1] https://forge.softwareheritage.org/harbormaster/unit/view/918517/ Related to T645 04 September 2020, 13:37:24 UTC
506843c Updated backport on buster-swh from debian/0.14.0-1_swh1 (unstable-swh) 04 September 2020, 11:11:33 UTC
fde873f Merge tag 'debian/0.14.0-1_swh1' into debian/buster-swh 04 September 2020, 11:11:32 UTC
c64324a Updated debian changelog for version 0.14.0 04 September 2020, 10:59:54 UTC
c14acc0 Update upstream source from tag 'debian/upstream/0.14.0' Update to upstream version '0.14.0' with Debian dir 8dec6a262f13d2d677c12bde6169148cea62f7b5 04 September 2020, 10:59:53 UTC
08312f5 New upstream version 0.14.0 04 September 2020, 10:59:49 UTC
356eacd Refactor revision_get storage API to return Revision objects The signature becomes revision_get(...) -> List[Optional[Revision]] Related to T645 03 September 2020, 15:05:11 UTC
36d284c cassandra: Discard Content ctime field in content_get_partition That field should only be returned by content_find method. Add tests to check expected behaviors according to methods. 02 September 2020, 13:10:57 UTC
b05e48d Updated backport on buster-swh from debian/0.13.3-1_swh1 (unstable-swh) 01 September 2020, 13:17:17 UTC
cf6b013 Merge tag 'debian/0.13.3-1_swh1' into debian/buster-swh 01 September 2020, 13:17:16 UTC
e80e15a Updated debian changelog for version 0.13.3 01 September 2020, 12:40:29 UTC
95ecb31 Update upstream source from tag 'debian/upstream/0.13.3' Update to upstream version '0.13.3' with Debian dir f2f40a96246f11fbc294dae34a02357008161d70 01 September 2020, 12:40:27 UTC
3962bb7 New upstream version 0.13.3 01 September 2020, 12:40:24 UTC
e6fcfb9 storage*: release_get(...) -> List[Optional[Release]] Related to T645 01 September 2020, 12:19:19 UTC
e6c17f6 Make StorageInterface a Protocol. We're already using 'StorageInterface' as type annotation where we expect an object that implements the same methods, but it isn't technically correct since these objects aren't an instance of a subclass. Making StorageInterface a protocol reflects how we already use it, without lying to mypy. 01 September 2020, 08:38:36 UTC
5afd985 Add a validating storage proxy, to check ids before insertion. It can be used to prevent buggy or malicious clients of the storage from adding invalid objects. 28 August 2020, 11:18:53 UTC
4532a4d Add a --check-config option for cli commands this allows to specify on command line wether to check the configuration and read or write access to the storage at startup time (especially for `rpc-server` and `replay` commands.) Warning: this option defaults to "read" for the backfill command and "write" for rpc-serve and replay commands, so now the creation of the Storage instance used in cli commands *will be checked*. Closes T2525. 25 August 2020, 08:50:50 UTC
2a35c0b Remove the deprecated config-path option from `swh storage rpc-serve` command include the validation of the presence of a "storage" config section in the main `storage` click.group, where the config-file is actually parsed. 25 August 2020, 08:50:50 UTC
e8b1b21 Tell pytest not to recurse in dotdirs. pytest wastes a lot of time in .hypothesis and .git; this commit excludes them. Before: $ time pytest --collect-only > /dev/null pytest --collect-only > /dev/null 6.93s user 0.51s system 100% cpu 7.425 total After: $ time pytest --collect-only > /dev/null pytest --collect-only > /dev/null 2.39s user 0.10s system 100% cpu 2.475 total 25 August 2020, 08:01:20 UTC
f3870dc 161: Fix sql upgrade script Related to T2524 25 August 2020, 07:35:31 UTC
cc33dd3 Add support for a new "check_config" config option in get_storage() if "check_config" is present in get_storage()'s kwargs, call the storage.check_config() method and raise an exception is the result fails. The storage.check_config() method is called with the content of the "check_config" config value (so expected value for this currently is `check_config={"check_write": <bool>}`). 24 August 2020, 14:58:17 UTC
4dd9597 Check for db version mismatch in PgStorage.check_config() 24 August 2020, 14:58:17 UTC
c16ff50 Add a check_dbversion() method to the Db class This method compares the version stored in the database in the `dbversion` table with the currently declared version. This current version is declared as a simple `current_version` attribute on the Db class. **It must be updated jointly in 30-swh-schema.sql** when modifying the db schema. Note that if this is forgotten, the added test `test_dbversion` should fail. Related to T2525. 24 August 2020, 14:58:17 UTC
629d2d1 Fix pytest_plugin's database janitor: do not truncate the dbversion table 24 August 2020, 14:58:17 UTC
f570f93 algos.snapshot: Add visits_and_snapshots_get_from_revision Its code is moved from snapshot_id_get_from_revision so it's a rather small change; and the revision metadata migration script (bin/migrate-extrinsic-metadata.py) will need it. 24 August 2020, 13:55:00 UTC
2bd172b Updated backport on buster-swh from debian/0.13.2-1_swh2 (unstable-swh) 21 August 2020, 10:34:28 UTC
97cfb65 Merge tag 'debian/0.13.2-1_swh2' into debian/buster-swh 21 August 2020, 10:34:28 UTC
119baf9 Add mypy-extensions to build-dependencies 21 August 2020, 10:18:01 UTC
d1f19e9 storage/interface: Remove deprecated diff endpoints They are not of interest anymore as swh-web is the only client and now directly uses functions defined in swh.storage.algos.diff. 20 August 2020, 15:15:15 UTC
5390a4c storage_tests: Remove duplicated postgresql-specific tests. They got copied to test_postgresql.py instead of moved. 20 August 2020, 14:04:30 UTC
3ac332e tests: Fix failures after test_storage renaming to storage_tests Also move the round_to_milliseconds function to the storage.utils module. 20 August 2020, 11:43:01 UTC
8db4459 Updated debian changelog for version 0.13.2 20 August 2020, 07:18:50 UTC
ead0aaf Update upstream source from tag 'debian/upstream/0.13.2' Update to upstream version '0.13.2' with Debian dir a6820e85659fbf232784d1b9e6a516e705844bcb 20 August 2020, 07:18:49 UTC
back to top