https://github.com/opencv/cvat

sort by:
Revision Author Date Message Commit Date
bef3325 Update server schema 20 February 2024, 10:33:57 UTC
1fd6358 Disable authentication for public endpoints 19 February 2024, 19:34:23 UTC
7286d65 Drop support of the TFRecord format (#7416) Usage statistics on app.cvat.ai show that this format is rarely used, with single-digit numbers of projects/tasks/jobs being exported or imported using this format. Moreover, TensorFlow's popularity appears to be shrinking, so I'm not expecting this format to make a comeback. Meanwhile, supporting this format has a cost that has to be borne by everyone deploying CVAT, because it requires TensorFlow to be installed and loaded. This has various ill effects: 1. Loading time is increased. In my testing, even a command as trivial as `manage.py --help` is slowed down by 3.6 seconds. This may not seem like much, but the effect is compounded, because we have multiple processes (server + workers) all loading the same codebase. Plus, the container entrypoint may execute several Django commands. 2. Memory usage is increased. TensorFlow adds ~100MB of RAM usage per process with data alone; and the libraries add more (although it's hard to estimate the impact of library code, since it can be shared between processes in RAM). 3. Docker image size is increased by ~1.5GB (when unpacked). This is more than half of the current total size! Building time is increased as well. Overall, it seems that the drawbacks of keeping support for this format outweigh the benefits, so it's time to drop it. 19 February 2024, 14:41:40 UTC
f0af22a Removed extra cypress configs (#7488) 19 February 2024, 13:45:25 UTC
42be015 Add events for importing/exporting annotations/datasets (#7476) This can provide useful statistics on the use of this functionality. Note that I chose to use the same scope for operations on projects, tasks and jobs, because the import/export operations work more-or-less the same on each of them and have the same parameters. It's still possible to distinguish which type of object the operation was performed on by examining the various `*_id` fields in the event. 16 February 2024, 13:03:07 UTC
9c048f2 Don't save all fields of a model when bumping the update date (#7450) Currently, there are a lot of places where changing one object requires updating the `updated_date` field on other objects. Usually, the `.save` method is used for this purpose, which results in a generic SQL `UPDATE` query that replaces the values in all columns. Introduce and use a specialized method that only changes the date. This provides the following advantages: * Prevents the date updates from overwriting updates to other fields that may happen at the same time. * Possibly makes the updates more efficient due to fewer columns being updated (I have no data on this, though). * Makes it easier to recognize the purpose of the query when analyzing logs/live statistics. * Makes the purpose of the code more obvious. Additionally, update `__save_job_handler`, where we don't want to save _just_ the date, but we still only need to save two fields. 16 February 2024, 13:01:30 UTC
972a218 Make event-related identifiers in cvat-core consistent (#7482) 16 February 2024, 12:31:21 UTC
6260232 Record DB-related events only after changes are committed (#7477) This is a small addendum to #7460, which I couldn't include in that PR, because it was blocked by #7430. 15 February 2024, 16:49:16 UTC
6673bb1 Modify backend entrypoint to wait redis (inmem and ondisk) (#7479) Currently CVAT is just waiting for the database to start the server process and the worker processes, but if redis instances (for queue and cache processing) are not running we will get a non-functional cvat. This PR adds availability checking for redis_ondisk and redis_inmem 15 February 2024, 14:08:45 UTC
11cec4c Migrate quality UI changes (#7441) 15 February 2024, 13:31:53 UTC
e8384f8 Run actions documentation added (#7448) 13 February 2024, 14:00:57 UTC
f477e0b Defer most signal handling code until after the end of the current transaction (#7460) Non-database actions like deleting directories, sending webhooks, scheduling reports should only be done after the current transaction is committed. If we do it immediately, and the transaction is later aborted, then we will have (for example) sent a webhook about an event that didn't actually happen. There's also a secondary benefit to moving this action outside of the transaction; the less time we spend inside a transaction, the better, since a transaction may lock out other clients from working on the affected DB rows. In addition, prevent the affected actions from crashing the view handler with an exception (using the `robust=True` option). I don't think it's reasonable to (for example) return a 500 response to a `PATCH` request just because we failed to send the corresponding webhook. There is one more type of action that should be modified in this way (sending events), but it would be easier to do that after a refactoring that I did in another patch, so I'll do it later. 13 February 2024, 13:43:08 UTC
b2a1065 Add failure_ttl when enqueueing task creation job (#7475) By default failed rq jobs are kept for 1 year. It is too long and leads to keeping a lot of unnecessary results in Redis. 13 February 2024, 13:42:08 UTC
58173c4 Reduce repetitive code in cvat/apps/events/handlers.py (#7430) <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> This is a preparatory PR before I add more events. ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - ~~[ ] I have created a changelog fragment~~ <!-- see top comment in CHANGELOG.md --> - ~~[ ] I have updated the documentation accordingly~~ - ~~[ ] I have added tests to cover my changes~~ - ~~[ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~ - ~~[ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~ ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. 13 February 2024, 13:02:28 UTC
ad60b82 Add the ability to disable sign-in/sign-up in the UI (#7348) Co-authored-by: Boris Sekachev <boris.sekachev@yandex.ru> 12 February 2024, 14:34:23 UTC
4e2b2df Remove the "Cancelling Duplicates" workflow (#7467) 09 February 2024, 18:00:28 UTC
24479c0 Click instead of hover to open dropdown menus (#7431) 09 February 2024, 13:41:40 UTC
8bf8fe5 Merge pull request #7468 from opencv/dev-release-2.10.3 Update develop after v2.10.3 09 February 2024, 11:19:04 UTC
eb2dfa8 Update develop after v2.10.3 09 February 2024, 11:12:10 UTC
73bf676 Prepare release v2.10.3 09 February 2024, 10:12:43 UTC
b4eed78 Reblacken Python code (#7461) Black 24 is upon us, and with it a new stable code style. Reformat all code to prevent CI failures in future PRs. 09 February 2024, 07:22:08 UTC
8e6e46a Exception event: 'not authentificated' was removed from analytics (#7457) 08 February 2024, 18:07:23 UTC
9f517d1 Reduced number of requests from client to server (#7446) 08 February 2024, 16:12:50 UTC
22d4e04 Add an `application_name` PostgreSQL connection parameter (#7275) <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> This will let us more easily identify PostgreSQL sessions that come from CVAT. See <https://www.postgresql.org/docs/15/runtime-config-logging.html#GUC-APPLICATION-NAME>. ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ```sql SELECT * FROM pg_stat_activity; ``` ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - ~~[ ] I have created a changelog fragment~~ <!-- see top comment in CHANGELOG.md --> - ~~[ ] I have updated the documentation accordingly~~ - ~~[ ] I have added tests to cover my changes~~ - ~~[ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~ - ~~[ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~ ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. 07 February 2024, 07:43:13 UTC
dc40be2 Bump actions/checkout and actions/setup-node to v4 (#7442) 06 February 2024, 16:34:37 UTC
edfafff Fix unhandled Redis exception in the /api/server/health/ endpoint (#7417) <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> `CustomCacheBackend` does not handle `RedisError`, so if Redis is not responding, the backend will propagate the exception, and the endpoint will crash with a 500 response code. This problem has been fixed upstream in django-health-check 3.18, so just update to the latest version and remove our custom cache backend. The upstream version, unlike ours, doesn't handle `sqlite3.DatabaseError`, but we're not using SQLite for caching anymore, so I don't think it matters. ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> I checked the endpoint with both working and non-working Redis. ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [x] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - ~~[ ] I have updated the documentation accordingly~~ - ~~[ ] I have added tests to cover my changes~~ - ~~[ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~ - ~~[ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~ ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. 05 February 2024, 15:09:39 UTC
e1ad07c Removed extra data from docker context (#7429) 05 February 2024, 10:20:54 UTC
1e91dd4 Added documentation to open guides automatically (#7440) 05 February 2024, 10:19:20 UTC
b7cc562 Open annotations guide automatically (#7410) 01 February 2024, 22:03:11 UTC
bdd002a Stabilized cypress test: issue_2900_creating_more_one_tasks_from_project_per_time.js (#7423) 01 February 2024, 19:52:21 UTC
27f8aa7 Fixed exception "Cannot read properties of null (reading 'plot')" (#7422) 01 February 2024, 11:53:09 UTC
b539802 Fixed "Cannot read properties of undefined (reading 'toLowerCase')" (#7421) 01 February 2024, 11:33:18 UTC
e4aa63f [Snyk] Security upgrade cryptography from 41.0.6 to 42.0.2 (#7419) 01 February 2024, 09:29:52 UTC
c794836 Strip useless linebreak in the payload of send:exception events (#7407) `traceback.format_exception` adds a `\n` to each element of the resulting list, so the message will always have a pointless `\n` at the end. Removing it makes CSV reports that include the message easier to parse visually, because there's no longer a linebreak in the middle of each line. 31 January 2024, 12:53:44 UTC
7a9c1f4 Update annotation-with-brush-tool.md (#7413) 31 January 2024, 07:22:38 UTC
ed3ddf3 Prevent zombie processes from accumulating in the Kvrocks container (#7412) <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> I saw several hundreds of zombie processes accumulate in the Kvrocks container in production. This should fix that. See the comment in `docker-compose.yml` for more explanation. Note that as far as I'm aware, Kubernetes does not use healthcheck commands configured in Docker images, so this issue should be irrelevant for Kubernetes deployments. ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [x] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - ~~[ ] I have updated the documentation accordingly~~ - ~~[ ] I have added tests to cover my changes~~ - ~~[ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~ - ~~[ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~ ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. 31 January 2024, 06:59:03 UTC
3db7be6 Merge pull request #7402 from opencv/dev-release-2.10.2 Update develop after v2.10.2 29 January 2024, 07:33:06 UTC
a6b37ba Subscription management documentation updated (#7393) 27 January 2024, 16:45:15 UTC
3992769 Update develop after v2.10.2 26 January 2024, 18:14:51 UTC
3733a07 Prepare release v2.10.2 26 January 2024, 13:35:45 UTC
ed9439b Fixed TrackerMIL one frame late (#7399) 26 January 2024, 12:13:55 UTC
34bc61e Added test for empty masks issues (#7381) 26 January 2024, 09:16:38 UTC
b5e1d19 Fixed 3D format for contextual images in documentation (#7390) 25 January 2024, 16:29:14 UTC
45c4356 Fix excessive memory usage when exporting projects with multiple video tasks (#7374) Currently, the following situation causes the export worker to consume more memory than necessary: * A user exports a project with images included, and * The project contains multiple tasks with video chunks. The main reason for this is that the `image_maker_per_task` dictionary in `CVATProjectDataExtractor.__init__` indirectly references a distinct `FrameProvider` for each task, which in turn, indirectly references an `av.containers.Container` object corresponding to the most recent chunk opened in that task. Initially, none of the chunks are opened, so the memory usage is low. But as Datumaro iterates over each frame, it eventually requests at least one image from each task of the project. This causes the corresponding `FrameProvider` to open a chunk file and keep a handle to it. The only way for a `FrameProvider` to close this chunk file is to open a new chunk when a frame from that chunk is requested. Therefore, each `FrameProvider` keeps at least one chunk open from the moment Datumaro requests the first frame from its task and _until the end of the export_. This manifests in the export worker's memory usage growing and growing as Datumaro goes from task to task. An open chunk consumes whatever Python objects represent it, but more importantly, any C-level buffers allocated by FFmpeg, which can be quite significant. In my testing, the size of the per-chunk memory was on the order of tens of megabytes. An open chunk also takes up a file descriptor. The fix for this is conceptually simple: ensure that only one `FrameProvider` object exists at a time. AFAICS, when a project is exported, all frames from a given task are grouped together, so we shouldn't need multiple tasks' chunks to be open at the same time anyway. I had to restructure the code to make this work, so I took the opportunity to fix a few other things, as well: * The code currently relies on garbage collection of PyAV's `Container` objects to free the resources used by them. Even though `VideoReader.__iter__` uses a `with` block to close the container, the `with` block can only do so if the container is iterated all the way to the end. This doesn't happen when `FrameProvider` uses it, since `FrameProvider` seeks to the needed frame and then stops iterating. I don't have evidence that this causes any issues at the moment, but Python does not guarantee that objects are GC'd promptly, so this could become another source of excessive memory usage. I added cleanup methods (`close`/`unload`/`__exit__`) at several layers of the code to ensure that each chunk is closed as soon as it is no longer needed. * I factored out and merged the code used to generate `dm.Image` objects when exporting projects and jobs/tasks. It's likely possible to merge even more code, but I don't want to expand the scope of the patch too much. * I fixed a seemingly useless optimization in the handling for 3D frames. Specifically, `CVATProjectDataExtractor` performs queries of the form: task.data.images.prefetch_related().get(id=i) The prefetch here looks useless, as only a single object is queried - it wouldn't be any less efficient to just let Django fetch the `Image`'s `related_files` when needed. I rewrote this code to prefetch `related_files` for all images in the task at once instead. 24 January 2024, 13:38:16 UTC
4cf45e5 Fix a typo. "cvat" service does not exist. It should be "cvat_server". (#7368) Fix a typo. refer to docker-compose.yml 23 January 2024, 11:25:14 UTC
8a81868 [Snyk] Security upgrade jinja2 from 3.1.2 to 3.1.3 (#7350) 23 January 2024, 11:19:52 UTC
a5a0e77 Bump pillow from 10.1.0 to 10.2.0 in /tests/python (#7384) 23 January 2024, 11:18:26 UTC
79aaa84 Bump pillow from 10.1.0 to 10.2.0 in /utils/dicom_converter (#7383) 23 January 2024, 11:17:18 UTC
a5669aa Enhanced errors notifications (#7371) 19 January 2024, 11:30:29 UTC
209826c Remove empty masks (#7295) 18 January 2024, 16:08:02 UTC
f688908 Merge pull request #7373 from opencv/dev-release-2.10.1 Update develop after v2.10.1 18 January 2024, 12:22:11 UTC
1d87216 Update develop after v2.10.1 18 January 2024, 11:10:54 UTC
c21062f Prepare release v2.10.1 18 January 2024, 10:25:37 UTC
7a1a4b1 Replace keydb with kvrocks (#7339) Resolved #7345 18 January 2024, 10:08:45 UTC
d25b863 Fixed context menu for skeleton elements (#7362) 17 January 2024, 14:58:19 UTC
0579b00 Documentation: Introduction to CVAT (#7229) 17 January 2024, 14:43:45 UTC
0433043 Added tests for annotations actions (#7358) 17 January 2024, 10:32:53 UTC
422dd68 Added test for GT issues (#7357) 17 January 2024, 07:25:43 UTC
300146c [WIP] invite users to org updated (#7305) <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [ ] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning)) ### License - [ ] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. --------- Co-authored-by: Kirill Lakhov <kirill.9992@gmail.com> 16 January 2024, 13:22:12 UTC
906260c Fix export of resources to cloud storage (#7317) <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> In the previous implementation logic to upload created files to cloud storage was in the code that processes requests. This resulted in responses with 504 status when uploading a file took longer than the request timeout. This PR fixes the described problem. ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [x] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [x] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning)) ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. 15 January 2024, 13:04:41 UTC
c02580c Bump follow-redirects from 1.15.3 to 1.15.4 (#7342) 15 January 2024, 08:24:45 UTC
8c2db1d Merge pull request #7341 from opencv/dev-release-2.10.0 Update develop after v2.10.0 10 January 2024, 13:47:50 UTC
a684c0c Update develop after v2.10.0 10 January 2024, 13:06:45 UTC
c36ec24 Prepare release v2.10.0 10 January 2024, 11:40:22 UTC
cfbad35 Fixed CI badge URL (#7337) 09 January 2024, 14:14:28 UTC
40dc2a9 Fix updating Azure connection string && increase credentials max_length (#7336) 09 January 2024, 13:37:13 UTC
9b99f6a Remove dependency on distutils module (#7318) The distutils module is [deprecated](https://peps.python.org/pep-0632) and will be removed from the standard library in Python 3.12. 09 January 2024, 11:17:09 UTC
64d6d7a Correct updating server ID after patching annotations (#7315) 04 January 2024, 11:15:29 UTC
39190c5 Bump tj-actions/changed-files from 35.9.2 to 41.0.0 in /.github/workflows (#7306) 03 January 2024, 13:25:19 UTC
397d006 Add absent skeleton elements automatically in cvat-core (#7302) 03 January 2024, 13:23:55 UTC
56f37ac Adjusted text color for label matcher (#7309) 03 January 2024, 11:19:45 UTC
c585209 Fixed two issues with GT annotations (#7299) 29 December 2023, 11:52:31 UTC
e269f13 Update LICENSE (#7301) 29 December 2023, 07:29:33 UTC
4bfd128 Updated pillow, unlocked test (#7100) 28 December 2023, 15:34:48 UTC
bf65513 Fix rq scheduler launch (#7293) 27 December 2023, 10:58:23 UTC
fb610a4 504 on save annotation, workaround (#7286) 27 December 2023, 08:36:08 UTC
06c6a76 Fixed docs building (#7294) 26 December 2023, 12:50:57 UTC
c51184b Fix bug: chinese can not input correctly (#6916) 23 December 2023, 06:26:14 UTC
ec5fe82 Small refactoring (#7149) 23 December 2023, 06:25:33 UTC
e11b6e0 Ground truth annotations is not injected to annotation job (#7281) 21 December 2023, 13:26:27 UTC
d9e7b2c Added cypress tests for slice/join (#7249) 19 December 2023, 14:25:41 UTC
48ab12b Use separate services for storing job queues and cache (#7245) <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> These types of data have different characteristics and we have different expectations on them: * job queues are small and we'd rather not lose them (although losing them is not fatal); * cached chunks are large and we don't care if we lose them. We currently store both in KeyDB, which has shown itself to not be especially reliable. A few times we've had to clear the KeyDB store due to data corruption, which destroyed the queues as well. While we'll probably end up replacing KeyDB with something else, it would still be useful to have the ability to just clear the cache volume without taking out the job queues in the process. As a solution to this, add a Redis service to be used only for the queues (and potentially for other small data items). Using the original Redis instead of KeyDB should also help with reliability (at least as far as the job queues are concerned). ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> I checked the CVAT can still start using the development environment instructions, the Compose file and the Helm chart. ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [x] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - ~~[ ] I have updated the documentation accordingly~~ - ~~[ ] I have added tests to cover my changes~~ - ~~[ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~ - ~~[ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~ ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. 19 December 2023, 13:33:56 UTC
80e1212 Remove deprecated VS Code settings (#7244) 18 December 2023, 10:38:53 UTC
fb0fe9e Use OPA from the Docker image for testing (#7273) Currently, the testing pipeline downloads a copy of OPA from the official site. But we already have a perfectly usable copy in the Docker image that we download for the Django unit tests. So just use that copy instead. This reduces the amount of downloading the pipeline has to do, and eliminates a hardcoded version number. 15 December 2023, 20:53:04 UTC
2b7d01a Improve invite users to organization feature (#7138) <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> This PR improves workflow with invitations to organizations. It allows to view, accept, decline invitations to organizations (previously all the invitations were auto-accepted). It fixes problems with inviting unregistered users to organization allowing them to register without invitation. ![image](https://github.com/opencv/cvat/assets/50956430/71bededa-be48-4e7a-80b6-febba86c2a3f) ![image](https://github.com/opencv/cvat/assets/50956430/773119a2-4d88-46c1-bad6-ed24734ce0c2) ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> The best way to test this PR is to add those lines to `cvat/settings/base.py`: Enable email verification: ``` ACCOUNT_AUTHENTICATION_METHOD = 'username_email' ACCOUNT_CONFIRM_EMAIL_ON_GET = True ACCOUNT_EMAIL_REQUIRED = True ACCOUNT_EMAIL_VERIFICATION = 'mandatory' ``` Setup console email backend ``` # Email backend settings for Django EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend' ``` And edit `ORG_INVITATION_CONFIRM` to `Yes` ``` ORG_INVITATION_CONFIRM = 'Yes' ``` ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [ ] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning)) ### License - [ ] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. --------- Co-authored-by: Boris Sekachev <boris@cvat.ai> Co-authored-by: Boris Sekachev <boris.sekachev@yandex.ru> 15 December 2023, 17:42:55 UTC
9fb582d Simplify the dev environment setup instructions by reusing Compose files (#7254) The advantages of this are as follows: * It's much easier for a developer to use one `docker compose up` command to bring everything up than to run a custom command for each service. * We eliminate possible divergence of configuration (e.g. versions, command-line parameters) between what we actually use and what's listed in the documentation. * It makes it easier to update the developer guide if new dependencies are introduced. * And speaking of new dependencies, we have KeyDB now, which hasn't been added to the dev guide. The disadvantage is that we have to run an extra copy of the CVAT server, because otherwise OPA can't fetch its rules. I don't think it's a significant issue, since it doesn't prevent you from debugging anything. <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> Working on #7245, I realized that I don't want to add another custom command for running Redis in the development environment to the dev guide. So I wanted to remove the custom commands entirely. ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> By manually following the updated instructions. ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - ~~[ ] I have created a changelog fragment~~ <!-- see top comment in CHANGELOG.md --> - [x] I have updated the documentation accordingly - ~~[ ] I have added tests to cover my changes~~ - [x] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - ~~[ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~ ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. 15 December 2023, 14:17:16 UTC
1bb674a Update Clickhouse to the latest version (#7268) <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> The main reason for this is to update the Helm chart, because the current one conflicts with the Redis chart I want to add (in #7245). But also the version we have is quite old and out of support, so it's due for an update anyway. The change in `objects.py` is due to the fact that with the new version of Clickhouse, the value of `date` somehow became a timezone-aware datetime object, so the output of `isoparse` now includes a UTC offset, and appending a "Z" now creates a malformed date. I chose to replace this with a custom format string, which is consistent with other date formatting code in the `analytics_report` app. ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> I manually tried both the Helm chart and the Compose file, making sure that the analytics are still recorded and displayed. I also checked that a database created with the previous Clickhouse version can be loaded with the new version (it can). ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - ~~[ ] I have updated the documentation accordingly~~ - ~~[ ] I have added tests to cover my changes~~ - [x] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - ~~[ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~ ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. 15 December 2023, 12:38:06 UTC
3520645 Removed `PermissionDenied` error thrown before OPA call (#7259) At first, this part of the code was an optimization. If we know that a user is not a member but tries to access org resources we throw an error. But during the development of other features(like invitations to organization) we need to give user an access to such resources. That leads to rewriting this part of the code with lots of `if` statements. Generally, all permission decisions should be made by OPA. 15 December 2023, 10:32:35 UTC
d7bbc10 Fix compatibility with older versions of Docker Compose (#7170) <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> In 6ae1cffd, I accidentally left behind the empty `command:` key. This works, but only with Docker Compose v2.17.1+. Since we don't actually need this key, we might as well remove it and restore compatibility with older versions. Also, rearrange the commented out code so that you only need to uncomment one block to enable the Traefik dashboard. Fixes #7153. ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> Manual testing. ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [x] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - ~~[ ] I have updated the documentation accordingly~~ - ~~[ ] I have added tests to cover my changes~~ - [x] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - ~~[ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~ ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. 14 December 2023, 07:54:41 UTC
cbd5f43 Fix GCS bucket name validation (#7227) It is not possible to attach a GCS cloud storage bucket whose name contains a dot. As stated [here](https://cloud.google.com/storage/docs/buckets#naming), GCS bucket names allow dots. This is a major problem for us, and this PR fixes this issue. --------- Co-authored-by: Jonas Otten <jootten@uos.de> Co-authored-by: Boris Sekachev <sekachev.bs@gmail.com> Co-authored-by: Maria Khrustaleva <maria@cvat.ai> 13 December 2023, 10:29:49 UTC
2aae952 Merge pull request #7255 from opencv/dev-release-2.9.2 Update develop after v2.9.2 11 December 2023, 18:40:13 UTC
0eec271 Update develop after v2.9.2 11 December 2023, 18:25:36 UTC
6263ba1 Prepare release v2.9.2 11 December 2023, 16:49:01 UTC
e472fad Recreate a chunk in case of UnpicklingError (#7251) Resolved #7251 11 December 2023, 16:40:37 UTC
72379e2 Revert "Update keydb to 6.3.4 (#7118)" (#7236) <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> That KeyDB version turned out rather unstable in practice, with multiple crashes and freezes observed in production. Resolved #7238 This reverts commit 118cc720ca6da62ffdf8048d2d150b8a44ba48bf. ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - ~~[ ] I have updated the documentation accordingly~~ - ~~[ ] I have added tests to cover my changes~~ - ~~[ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~ - ~~[ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~ ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. 11 December 2023, 13:09:10 UTC
588df39 Added workaround for corrupted zip chunks (#7243) 08 December 2023, 20:27:09 UTC
405a019 Use different tags for GPU versions of serverless functions (#7215) Having both CPU and GPU images use the same image name and tag is confusing, since you can't tell which one you have on your system without deep inspection. 08 December 2023, 15:37:57 UTC
de6e50c cvat-core: retry HTTP requests if a 429 status is returned (#7216) This should help reduce breakage if a user hits an API rate limit. 08 December 2023, 15:19:34 UTC
a4f0cea [Snyk] Security upgrade cryptography from 41.0.5 to 41.0.6 (#7203) Co-authored-by: snyk-bot <snyk-bot@snyk.io> 08 December 2023, 12:02:09 UTC
6cfab41 Added validation of `duration` field for `change:frame` (#7211) Fix exceptions with 500 code, like this in our logging system: ``` Error: KeyError: 'duration' Error count: 131 Stack: Traceback (most recent call last): File "/opt/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch response = handler(request, *args, **kwargs) File "/home/django/cvat/apps/events/views.py", line 32, in create serializer.is_valid(raise_exception=True) File "/opt/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 227, in is_valid self._validated_data = self.run_validation(self.initial_data) File "/opt/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 426, in run_validation value = self.to_internal_value(data) File "/home/django/cvat/apps/events/serializers.py", line 59, in to_internal_value event_duration += datetime.timedelta(milliseconds=event["duration"]) KeyError: 'duration' ``` 08 December 2023, 11:54:45 UTC
back to top